I'm making an app where it stores how many minutes a user has studied with my app. My Firestore database starts with a "users" collection, and each user has their own document that is named by their userID generated in Auth.
My question is if I read their userID document, which has many documents in its sub collections, does that count as one read or does it also count the number of documents in the sub collections as well?
Thank You in advance.
The answer here from Torewin is mostly correct, but it missing one important detail. It says:
if you retrieve a document; anywhere, it counts as a read
This is not entirely true. Cached document reads are not billed as reads. This is one important feature of the Firestore client SDKs that helps lower billing costs. If you get a single document using the source option cache (options are "cache" or "server" or "default"), then the cache will be consulted first, and you get the document without billing. The cache is also used for query results when the app is offline.
The same is true for query results. If a document comes from cache for some reason, there is no billing for that read.
I am uncertain what Torewin means by this in comments: "They recommend you make multiple reads instead of 1 big one because you will save money that way". All reads are the same "size" in terms of billing, considering only the cost of the read itself. The size of the document matters only for the cost of internet egress usage, for which there is documentation on pricing.
It's worth noting that documents can't "contain" other documents. Documents are contained in collections or subcollections. These collection just have a "path" that describes where they live. A subcollection can exist without a "parent" document. When a document doesn't exist, but a collection is organized under it, the document ID is shown in italics in the console. When you delete a document using the client API, none of its subcollections are deleted. Deletes are said to be "shallow" in this respect.
If you are referring to is it 1 read to access a Document (in this case your generatedUserID) from FireStore?
I would imagine the answer would be yes.
Any query or read from Firestore only pulls the reference that you are mapping to. For example, if you grab the 3rd document in your User -> userID -> 3rd document, only the 3rd document will be returned. None of the other documents in that collection or any of the collections besides the userID.
Does that answer your question or are you asking something completely different?
For reference: https://firebase.google.com/docs/firestore/pricing#operations
Edit: Each individual Document that is pulled from the query will be charged. For example, if you pull the parent collection (with 6 documents in it), you will be charged for all 6 documents. The idea is to only grab the documents you need or use a cursor which let's you resume a long-running query. For example, if you only want the document pertaining to use data on a specific date (if your data is set up like that), you'd only retrieve that specific document and not retrieve all of the documents in the collection for the other days.
A simple way of thinking about it is: if you retrieve a document; anywhere, it counts as a read.
Related
This is how my Firestore DB is structured
For example, now I want userA to share the Recipe2 with userB.
I want to list the Recipe2 in UserB UI as well when I fetch his/her recipes.
The way I am fetching the recipes in my Swift code is that a user can only view his/her docs or collections, i.e. userB can never query/read the userA's recipes.
Keeping in view that I might allow userB to modify shared recipes in future, how can I achieve sharing feature? I don't want to create multiple copies, my Firestore rules are set to not allow any user to read other user's data, with all this what is best possible solution?
There are a bunch of ways to do it handle this, and providing the best possible solution would be impossible without understanding the entire use case.
Let me suggest this:
When you have a situation where there's my data, your data and OUR data, it's often best to put OUR data in a separate collection, using uid's and rules to determine who can access what data. For example
users
UserA
...
UserB
...
recipes
recipe0
owner: UserA
name: "pasta fazul"
ingredients
...
visible_to
UserB
Then a rule that would allow a user to read recipes where their uid is listed in visible_to, and only the owner matching the owner uid can make changes.
Like I said in the comments, you must either grant users access to each other's documents or denormalize the documents themselves. Denormalizing data in NoSQL is not only standard operating procedure but a highly-effective optimization tool and one that should be used when needed.
Firstly, Firestore documents cannot be larger than 1 MB which means that with a relatively small userbase you can opt for a different architecture (that would save you money on reads and writes). However, I would advise against architecting data that doesn't scale with the userbase because userbase size can be the most difficult metric to predict. This solution can scale with a userbase of practically any size. Secondly, this solution is designed so that when the owner of a recipe modifies that recipe, their shared instances can easily be found and also be updated (if needed). Thirdly, this solution is designed so that when a user who is displaying a shared recipe modifies that recipe, the original author can easily be found and that recipe can be updated or notated (if needed). Finally, this solution makes displaying a user's recipes (that the user both owns and borrows) possible with just two simple queries that don't require accessing another user's data.
[collection]
<document>
[users]
<userA>
[my-recipes]
<recipe1>
<recipe2>
<recipe3>
[sharing-with]
<userB> (acts as a bookmark with the user's info)
<userB>
[my-recipes]
<recipe4>
<recipe5>
<recipe6>
[shared-recipes]
<recipe3> (contains an owner field)
I have some data that needs to be loaded from the server (backend). For example, let's just say I have an entities of user and event. The relationship between them is many-to-many (user can attend many events and event can have many attendees). All the data is stored remotely on backend and locally in Core Data. When I download data from backend I convert it into NSManagedObjects and store it in NSManagedObjectContext. Everything's very simple, but...
When I download a list of events I want to know, how many attendees this event has. But I cannot download a list of users in the same request, because it's totally overkill. What I need is to download, let's say, a list of users' unique ids so that I can have two things: total number of attendees and means to download detailed data of concrete users (via unique id). Or there's another example: I need to know total number of attendees and download a limited set of them, so I can create some entities in CoreData, but not all of them.
So the main question is how am I supposed to store such information in my CoreData? Meaning I need to know that for some entity there are some related entities in relationship that are not actually currently present in CoreData, but I know how many of them there should be. The first thing that came in my mind is to have a attribute called something like usersCount in my event entity, but that seems to be kind of dirty. What is the best practice for such situation?
Please comment if the question is not clear enough so I can maybe add some more specifics.
When you download an event with a list of corresponding user ids, then you can create
the Event object and also the related User objects, but you fill only the "userId"
attribute in the user object.
Later, when you download the complete user info, you update the existing (incomplete) objects
or create new user objects. Implementing Find-or-Create Efficiently in the "Core Data Programming Guide"
describes a pattern that might be useful.
So the idea is to create Core Data objects with incomplete information first and update the
objects with detailed information later. The advantage is that you can set up all relationships immediatly, and e.g. counting related users works even if the user information
is yet incomplete.
There is nothing dirty about having an attribute to store the count, especially if those entities are retrieved and paged via separate requests.
I've been pouring over the new documentation for the Google Docs API for more efficient ways to synchronize my application's resources and I came upon the changes feed. So far, all the documentation leads me to believe that the changes feed applies to the entire documents list only without the ability to inquire about changes to a specific collection. My application doesn't care about any documents besides the ones belonging to the collections the user has specified. Does anyone know if querying a specific collection for a changes feed is possible? Thank you.
That is not possible, but the nature of the changes feed is such that it should be cheaper to grab just the latest few changes, by storing the largest timestamp. Once you have the changes feed, you can filter them by looking at the entry's #parent link, that looks like:
<link rel='http://schemas.google.com/docs/2007#parent'
type='application/atom+xml'
href='https://docs.google.com/feeds/default/private/full/folder%3A0B343'
title='Some API Documentation' />
I'm starting a MongoDB project just for kicks and as a chance to learn MongoDB/NoSQL schemas. It'll be a live chat app and the stack includes: Rails 3, Ruby 1.9.2, Devise, Mongoid/MongoDB, CarrierWave, Redis, JQuery.
I'll be handling the live chat polling/message queueing separately. Not sure how yet, either Node.js, APE or custom EventMachine app. But in regards to Mongo, I'm thinking to use it for everything else in the app, specifically chat logs and historical transcripts.
My question is how best to design the schema as all my previous experience has been with MySQL and relational DB schema's. And as a sub-question, when is it best to us embedded documents vs related documents.
The app will have:
Multiple accounts which have multiple rooms
Multiple rooms
Multiple users per room
List of rooms a user is allowed to be in
Multiple user chats per room
Searchable chat logs on a per room and per user basis
Optional file attachment for a given chat
Given Mongo (at least last time I checked) has a document limit of 4MB, I don't think having a collection for rooms and storing all room chats as embedded documents would work out so well.
From what I've thought about so far, I'm thinking of doing something like:
A collection for accounts
A collection for rooms
Each room relates back to an account
Related documents in chats collections for all chat messages in the room
Embedded Document listing all users currently in the room
A collection for users
Embedded Document listing all the rooms the user is currently in
Embedded Document listing all the rooms the user is allowed to be in
A collection for chats
Each chat relates back to a room in the rooms collection
Each chat relates back to a user in the users collection
Embedded document with info about optional uploaded file attachment.
My main concern is how far do I go until this ends up looking like a relational schema and I defeat the purpose? There is definitely more relating than embedding going on.
Another concern is that referencing related documents is much slower than accessing embedded documents I've heard.
I want to make generic queries such as:
Give me all rooms for an account
Give me all chats in a room (or filtered via date range)
Give me all chats from a specific user
Give me all uploaded files in a given room or for a given org
etc
Any suggestions on how to structure the schema efficiently in a way that scales? Thanks everyone.
I think you're pretty much on the right track. I'd use a capped collection for chat lines, with each line containing the user ID, room ID, timestamp, and what was said. This data would expire once the capped collection's "end" is reached, so if you needed a historical log you'd want to copy data out of the capped collection into a "log" collection periodically, but capped collections are specifically designed for logging-style applications where you aren't going to be deleting documents, and insertion order matters. In the case of chat, it's a perfect match.
The only other change I'd suggest would be to maintain uploads in a separate collection, as well.
I am a big fan of mongodb as a document database aswell. But are you sure you are using mongodb for the right reason? What is mongodb powerful at?
Its a subjective question but for me in-place (atomic) updates over documents is what makes mongodb powerful. And I can't really see you using it that much. And on top of that you are hitting the document size limit problem aswell.(With experience I can tell you that embedding files to mongodb is not a good idea). You want to have a live chat application on top of database too.
Your document schema's seems logical. But I wouldn't go with mongodb for this kind of project where your application heavily depends on inserts. I would go for CouchDB.
With CouchDB you wouldn't have to worry about attachments problem, you can embed them easily. "_changes" would make your life much much easier to eighter build a live chat application / long pooling / feeding search engine (if you want to implement one).
And I saw an open source showcase project in couchone. It has some similarities with your goals: Anologue. You should check it out.
PS : Sorry it was a little off topic but I couldn't hold myself.
I am developing a web app which displays sales from local stores around the United States. The sales and stores listed vary by location. Is there a RESTful URL scheme for describing this information while avoiding duplicate content?
Specifically, the app needs to list local stores, and list items sold at a particular store. Zip (postal) codes seem a convenient way to refer to location, so consider this scheme:
/stores/zip - list stores near zip, with links to particular stores
/store/name/lat+long - list items at a particular store
There is a problem. The page at /store/name/lat+long needs to link back to the list of stores, but which zip code should it choose? Say it chooses the zip code closest to the lat+long coordinate. A user might arrive at a particular store page from a link on /stores/zipA yet the store page could refer them back to a slightly different list, /stores/zipB.
We could solve that problem by carrying the zip code information forward. So the list at /stores/zip, could link to /store/name/lat+long/zip. This is not logical, however, because all information needed to identify a store is provided by the lat+long coordinate; the zip code is redundant. In fact the same page content would have duplicate URLs.
Another solution would be to save the last-viewed zip code as a cookie, but that's not RESTful. Is there a solution?
Add that information as an optional query parameter.
/stores/name/lat+long?search=zip
The /stores/name/lat+long represents the resource uniquely, while the optional query parameter provides the extra information you need for your breadcrumb back to their original search.
If you have links that come from somewhere other than a search for that zip code, then you could just leave the query parameter off. When the query parameter is missing, default to linking back to the closest zip code, or leaving the breadcrumb link off entirely.
Another option would be to just let the browser history do this for you, by using JavaScript to navigate the user to the previous page in their history:
Back to search