Modeling sub-collections in MongoDB Realm Sync - ios

I'm new to MongoDB as well as to MongoDB Realm Sync. I was following the Realm Sync tutorial and Realm data model docs, but I wanted to learn more so I tweaked the Atlas collection structure as follows.
Projects > Tasks // i.e. tasks is a sub-collection in each project.
What I don't know is how to come up with Realm Sync Schema which can support Atlas sub-collections.
The best I came up with is a Schema where Tasks are modelled as an array within the Project. But, I'm worried that this can hit the 16MB (although a lot!) document limit for projects with a lot of the tasks.
{
"bsonType": "object",
"properties": {
"_id": {
"bsonType": "objectId"
},
"_partition": {
"bsonType": "string"
},
"name": {
"bsonType": "string"
},
"tasks": {
"bsonType": "array",
"items": {
"bsonType": "object",
"title": "Task",
"properties": {
"name": {
"bsonType": "string"
},
"status": {
"bsonType": "string"
}
}
}
}
},
"required": [
"_id",
"_partition",
"name",
],
"title": "Project"
}
Looking forward on how to model sub-collection the right way.
Edit
Here's my client side Realm models.
import Foundation
import RealmSwift
class Project: Object {
#objc dynamic var _id: String = ObjectId.generate().stringValue
#objc dynamic var _partition: String = "" // user.id
#objc dynamic var name: String = ""
var tasks = RealmSwift.List<Task>()
override static func primaryKey() -> String? {
return "_id"
}
}
class Task: EmbeddedObject {
#objc dynamic var name: String = ""
#objc dynamic var status: String = "Pending"
}
As far the CRUD operations are concerned, I only create a new project and read existing projects as follows.
// Read projects
realm.objects(Project.self).forEach { (project) in
// Access fields
}
// Create a new project
try! realm.write {
realm.add(project)
}

Your code looks great and your heading the right direction, so this answer is more explanation and suggestions on modeling than hard code.
First, Realm objects are lazily loaded which means they are only loaded when used. Tens of thousands of objects will have very little impact on a devices memory. So suppose you have 10,000 users and you 'load them all in'
let myTenThousandUsers = realm.objects(UserClass.self)
meh, no big deal. However, doing this
let someFilteredUsers = myTenThousandUsers.filter { $0.blah == "blah" }
will (could) create a problem - if that returns 10,000 users they are all loaded into memory possibly overwhelming the device. That's a Swift function and 'converting' Realms lazy data using Swift should generally be avoided (use case dependent)
The observation of this code using Swift .forEach
realm.objects(Project.self).forEach { (project) in
// Access fields
}
could cause issues depending on what's being done with those project objects - using them as a tableView dataSource could be trouble if there are a lot of them.
Second thing is the question about the 16Mb limit per document. For clarity an Atlas document is this
{
field1: value1,
field2: value2,
field3: value3,
...
fieldN: valueN
}
where value can be any of the BSON data types such as other documents, arrays, and arrays of documents.
In your structure, the var tasks = RealmSwift.List<Task>() where Task is an embedded object. While conceptually embedded objects are objects, I believe they count toward a single document limit because they are embedded (correct me if I am wrong); as the number of them grows, the size of the enclosing document grows - keeping in mind that 16Mb of text is an ENORMOUS of text so that would/could equate to millions of tasks per project.
The simple solution is to not embed them and have them stand on their own.
class Task: Object {
#objc dynamic var _id: String = ObjectId.generate().stringValue
#objc dynamic var _partition: String = ""
#objc dynamic var name: String = ""
#objc dynamic var status: String = "Pending"
override static func primaryKey() -> String? {
return "_id"
}
}
Then each one can be 16Mb, and an 'unlimited number' can be associated with a single project. One advantage of embedded objects is a type of cascade delete where when the parent object is deleted, the child objects are as well, but with a 1-many relationship from Project to Tasks - deleting a bunch of tasks belonging to a parent is easy.
Oh - another case for not using embedded objects - especially for this use case - is they cannot have indexed properties. Indexing can greatly speed up some queries.

Related

How to replace/update dictionary in Array of Dictionary based on Type

I'm getting below JSON response from server, and displaying phone number on screen.
Now user can change/update any of phone number, so we have to update particular mobile number in same object and send it to server.
"phone_numbers": [
{
"type": "MOBILE",
"number": "8091212121"
},
{
"type": "HOME",
"number": "4161212943"
},
{
"type": "BUSINESS",
"number": "8091212344"
}
]
My model class is looks like this:
public struct Contact: Decodable {
public let phone_numbers: [Phone]?
}
public struct Phone: Decodable {
public let type: PhoneType?
public let number: String?
}
I'm struggling to update this JSON object for particular phone number.
For example, if I want to update BUSINESS number only in above array, What's best way to do it.
I'm using XCode 11 and Swift 5.
Because all your properties are defined as constants (let), nothing can be updated. You have to initialize and return a new Contact object with the updated phone numbers.
If you change the properties to var, then you can update:
public enum PhoneType: String, Decodable {
case mobile = "MOBILE"
case home = "HOME"
case business = "BUSINESS"
}
public struct Contact: Decodable {
public var phone_numbers: [Phone]?
mutating func update(phoneNumber: String, for type: PhoneType) {
guard let phone_numbers = self.phone_numbers else { return }
for (i, number) in phone_numbers.enumerated() {
if number.type == type {
self.phone_numbers![i].number = phoneNumber
}
}
}
}
public struct Phone: Decodable {
public var type: PhoneType?
public var number: String?
}
var contact = try! JSONDecoder().decode(Contact.self, from: jsonData)
contact.update(phoneNumber: "123456", for: .business)
I'm struggling to update this JSON object for particular phone number.
It shouldn't be a JSON object when you update it. Think of JSON as just a format for transferring data. Once transferred, you should parse it into something that you can work with, like an array of dictionaries or whatever. If you've done that, then more specific questions you might ask are:
How can I find a specific entry in an array?
How can I modify the fields of a struct?
How can I replace one entry in an array with another?
After looking at the definitions of your structures, I think the problem you're having probably has to do with how you've declared them:
public struct Phone: Decodable {
public let type: PhoneType?
public let number: String?
}
Because you used let to declare type and number, those fields cannot be changed after initialization. If you want the fields of a Phone struct to be modifiable, you need to declare them with var instead of let.
The same thing is true for your Contact struct:
public struct Contact: Decodable {
public let phone_numbers: [Phone]?
}
You've declared phone_numbers as an immutable array because you used let instead of var. If you want to be able to add, remove, or modify the array in phone_numbers, you need to use var instead.
The struct declarations you have right now work fine for reading the data from JSON because all the components of the JSON data are constructed using the values from the JSON. But again, you'll need to make those structs modifiable by switching to var declarations if you want to be able to make changes.
There are a couple of ways to approach this (I'm assuming PhoneType is an enum you have somewhere)
You can iterate over the array and guard for only business numbers, like so
for phone in phone_numbers{
guard phone.type == .MOBILE else { continue }
// Code that modifies phone
}
You can filter and iterate over the array, like so
phone_numbers.filter {$0.type == .BUSINESS }.forEach { phone in
// Modify phone here
}
You can then modify the right value in the array with it's index, like this
for (phoneIndex, phone) in phone_numbers.enumerated() {
guard phone.type == .BUSINESS else { continue }
phone_numbers[phoneIndex].type = ANOTHER_TYPE
}
Some can argue that the second is preferred over the first, because it is an higher order function, but in my day to day activities, I tend to use both and believe that this is a matter of taste

Multiple Realms architecture

I have a question regarding my architecture of multiple Realms in my project. Consider the following model:
#objcMembers class Book: Object {
dynamic var bookID = UUID().uuidString
dynamic var title: String = ""
dynamic var year: Int = 0
override static func primaryKey() -> String {
return "bookID"
}
}
What I would like to do is to be able to either like a book or to put it into a shopping list.
The first solution I came up with was to have three default Realms for:
Books - RealmBooks
Books liked - RealmLiked
Books added to a shopping list - RealmShopping
With this fairly simple approach I was forced to have several Realms. In the future I would like to sync the data from the above Realms to the Realm Cloud. So to avoid this is it possible to have an architecture that combines the data of let's say RealmLiked and RealmShopping? This could be of an object type UserSettings:
#objcMembers class UserSettings: Object {
dynamic var userID = UUID().uuidString
let likedBooks = List<Book>()
let shoppingListBooks = List<Book>()
override static func primaryKey() -> String {
return "userID"
}
}
Then I would simply sync UserSettings and this would act like the master Realm. Obviously the remaining RealmBooks would be synced in a separate Realm. The reasons for all of this is due to trying to keep my project simple as well as the limitation of having Up to 3 Realm Instances synced in the standard version of Realm Platform Cloud.
I would like to know the best practice in this scenario. Any help is much appreciated.

How to correctly wrap a Flux inside a Mono object

I have a web-service which returns student and enrolled class details.
{
"name": "student-name",
"classes": [
{
"className": "reactor-101",
"day": "Tuesday"
},
{
"className": "reactor-102",
"day": "Friday"
}
]
}
The DTO for this class is as below:
public class Student {
private String name;
private Flux<StudentClass> classes;
#Data
#AllArgsConstructor
#JsonInclude(JsonInclude.Include.NON_DEFAULT)
public static class StudentClass {
private String className;
private String day;
}
}
The main REST controller logic to fetch the student is as follows:
Flux<StudentClass> studentClassFlux = studentClassRepository.getStudentClass(studentName);
return Mono.just(new Student(studentName, studentClassFlux));
The problem with this is, I get the following output after making the REST call:
{
"name": "student-name",
"classes": {
"prefetch": 32,
"scanAvailable": true
}
}
I can achieve the desired output by blocking on the flux request to get completed and then convert the output to list.
List<StudentClass> studentClassList = studentClassRepository.getStudentClass(studentName)..toStream().collect(Collectors.toList());
return Mono.just(new Student(studentName, studentClassList)); // Change the Student#classes from flux to list
I am new to reactive-programming.
What is the correct way of using the Flux & Mono here to get the desired output?
Reactive types aren't meant to be serialized when wrapped in each other.
In that particular case, you probably want your Student object to contain a List<StudentClass>. You can achieve that like this:
public Mono<Student> findStudent(String studentName) {
return studentClassRepository
.getStudentClass(studentName)
.collectList()
.map(studentClasses -> new Student(studentName, studentClasses));
}
I think, in the case that you really need a Flux in your result, you would want to break down the API so that you have separate methods to retrieve the entities.
One for student properties, and another for their classes. The student GET method could be a Mono, while the classes would return a Flux.

RestKit dynamic mapping with nested items

I am having trouble configuring a dynamic mapping for an array of results. The result objects I'd like to map are embedded within a "container" that describes the type of the enclosed object.
I have JSON similar to the following:
"list": {
"name": ""
"item_infos": [
{
"type": "TaskItem",
"item": {
"id": 0
"title": "A Task Item",
"task": "..."
}
"url": "..."
},
{
"type": "ReminderItem",
"item": {
"id": 0
"title": "A Reminder Item",
"reminder": "..."
}
"url": "..."
}]
}
I'd like to map this as a List object with an array of Items, where each item might be of a different type. The different types are finite and known.
class List: NSManagedObject {
#NSManaged var name: String
#NSManaged var items: NSOrderedSet
}
class Item: NSManagedObject {
#NSManaged var identifier: Int32
#NSManaged var title: String
// Each mappable `Item` is responsible for providing its own
// mapping object. It's overridden in the subclasses below.
class func responseMappingForManagedObjectStore(dos: RKManagedObjectStore) -> RKEntityMapping { ... }
}
class TaskItem: Item {
#NSManaged var task: String
}
class ReminderItem: Item {
#NSManaged var reminder: String
}
How can I map the embedded item directly under the list using RKDynamicMapping while still making use of the type field? I'm trying something along these lines for the response mapping:
let listMapping = RKEntityMapping(forEntityForName: "List", inManagedObjectStore: store)
responseMapping.addAttributeMappingsFromDictionary(["name": "title"])
let dynamicItemMapping = RKDynamicMapping()
dynamicItemMapping.setObjectMappingForRepresentationBlock { representation in
let itemMapping: RKEntityMapping
switch representation.valueForKeyPath("type") as? String {
case .Some("TaskItem"): itemMapping = TaskItem.responseMappingForManagedObjectStore(objectStore)
case .Some("ReminderItem"): itemMapping = ReminderItem.responseMappingForManagedObjectStore(objectStore)
default: return nil
// This is the bit I'm failing to solve. How can I return a mapping
// that essentially "skips" a level of the JSON, and just maps the
// embedded `item`, not the `item_info`.
let itemInfoMapping = RKObjectMapping(forClass: NSMutableDictionary.self)
itemInfoMapping.addRelationshipMappingWithSourceKeyPath("item", mapping: itemMapping)
return itemInfoMapping
}
listMapping.addPropertyMapping(RKRelationshipMapping(
fromKeyPath: "item_infos",
toKeyPath: "items",
withMapping: dynamicItemMapping))
With this mapping, an exception is raised:
-[__NSDictionaryM _isKindOfEntity:]: unrecognized selector sent to instance 0x7fd2603cb400
Which doesn't surprise me, as the way the dynamic mapping is set up just doesn't feel right anyway – I'm looking for a way to "skip" a level of the JSON and only map the embedded item.
An alternative attempt was to fully specify the fromKeyPath as "item_infos.item" for the relationship to the listMapping, but then I cannot use the type field in the dynamic mapping block to determine the type of Item mapping to use:
// ...
dynamicItemMapping.setObjectMappingForRepresentationBlock { representation in
// `type` inaccessible from the nested item representation
switch representation.valueForKeyPath("type") as? String {
case .Some("TaskItem"): return TaskItem.responseMappingForManagedObjectStore(objectStore)
case .Some("ReminderItem"): return ReminderItem.responseMappingForManagedObjectStore(objectStore)
default: return nil
}
listMapping.addPropertyMapping(RKRelationshipMapping(
fromKeyPath: "item_infos.item",
toKeyPath: "items",
withMapping: dynamicItemMapping))
You can't do exactly what you're trying to do, because you're connecting a core data relationship to what will be an array of dictionaries which contains managed objects.
The simplest solution is to take your skip logic out from where it is and create all of the mappings for the managed objects (task and reminder) by adding item. at the start of the source key (path). So
"item.title" : "title"

Building APIs to access Neo4j data

I have a huge Neo4j database that I created using the batch import tool. Now I want to expose certain parts of the data via APIs (that will run a query in the backend) to my users. My requirements are pretty general:
1. Latency should be minimum
2. Support qps of about ~10-20.
Can someone give me recommendations on what I should use for this and any documentation on how to go about this? I see several examples of ruby/rails and REST APIs -- however they are specific to exposing the data as is without any complex queries in the backend. I am not sure how to translate that into the specific APIs that I want. Any help would be appreciated.
Thanks.
I wrote a simple Flask API example that interfaces with Neo4j for a simple demo (backend for a messaging iOS app).
You might find it a helpful reference: https://github.com/johnymontana/messages-api
There are also a few resources online for using Flask with Neo4j:
http://nicolewhite.github.io/neo4j-flask/
http://neo4j.com/blog/building-python-web-application-using-flask-neo4j/
https://github.com/nicolewhite/neo4j-flask
Check out the GraphAware Framework. You can build the APIs directly on top of Neo4j (same JVM) but you have to use Cypher, Java, or Scala.
I'd start with Cypher, because you can write it very quickly, then optimise for performance, and finally, if all else fails and your latency is still to high, convert to Java.
You can expose subgraphs (or even partially hydrated nodes and relationship, i.e. only certain properties) very easily. Checkout out the stuff in the api package. Example code:
You'd write a controller to return a person's graph, but only include nodes' names (not ages or anything else):
#RestController
public class ApiExample {
private final GraphDatabaseService database;
#Autowired
public ApiExample(GraphDatabaseService database) {
this.database = database;
}
#RequestMapping(path = "person/{name}")
public JsonGraph getPersonGraph(#PathVariable(value = "name") String name) {
JsonGraph<?> result = new JsonGraph() {
#Override
protected JsonGraph self() {
return this;
}
};
try (Transaction tx = database.beginTx()) {
Node person = database.findNode(label("Person"), "name", name);
if (person == null) {
throw new NotFoundException(); //eventually translate to 404
}
result.addNode(person, IncludeOnlyNameNodeTransformer.INSTANCE);
for (Relationship worksFor : person.getRelationships(withName("WORKS_FOR"), Direction.OUTGOING)) {
result.addRelationship(worksFor);
result.addNode(worksFor.getEndNode(), IncludeOnlyNameNodeTransformer.INSTANCE);
}
tx.success();
}
return result;
}
private static final class IncludeOnlyNameNodeTransformer implements NodeTransformer<LongIdJsonNode> {
private static final IncludeOnlyNameNodeTransformer INSTANCE = new IncludeOnlyNameNodeTransformer();
private IncludeOnlyNameNodeTransformer() {
}
#Override
public LongIdJsonNode transform(Node node) {
return new LongIdJsonNode(node, new String[]{"name"});
}
}
}
Running this test
public class ApiExampleTest extends GraphAwareApiTest {
#Override
protected void populateDatabase(GraphDatabaseService database) {
database.execute("CREATE INDEX ON :Person(name)");
database.execute("CREATE (:Person {name:'Michal', age:32})-[:WORKS_FOR {since:2013}]->(:Company {name:'GraphAware', est:2013})");
}
#Test
public void testExample() {
System.out.println(httpClient.get(baseUrl() + "/person/Michal/", 200));
}
}
would return the following JSON
{
"nodes": [
{
"properties": {
"name": "GraphAware"
},
"labels": [
"Company"
],
"id": 1
},
{
"properties": {
"name": "Michal"
},
"labels": [
"Person"
],
"id": 0
}
],
"relationships": [
{
"properties": {
"since": 2013
},
"type": "WORKS_FOR",
"id": 0,
"startNodeId": 0,
"endNodeId": 1
}
]
}
Obviously you can roll your own using frameworks like Rails / Sinatra. If you want a standard for the way that your API is formatted I quite like the JSON API standard:
http://jsonapi.org/
Here is an episode of The Changelog podcast talking about it:
https://changelog.com/189/
There's also a gem for creating resource objects which determine what is exposed and what is not:
https://github.com/cerebris/jsonapi-resources
I tried it out a bit with the neo4j gem and it works at a basic level, though once you start getting into includes there seems to be some dependencies on ActiveRecord. I'd love to see issues like that worked out, though.
You might also check out the GraphQL standard which was created by Facebook:
https://github.com/facebook/graphql
There's a Ruby gem for it:
https://github.com/rmosolgo/graphql-ruby
And, of course, another episode of The Changelog ;)
http://5by5.tv/changelog/149
Various other API resources for Ruby:
https://github.com/webmachine/webmachine-ruby
https://github.com/ruby-grape/grape
Use grest.
You can simply define your primary model(s) and its relation(s) (as secondary) and build an API with minimal coding and as quickly as possible!

Resources