Am working on an app for iPad, to organise sports, using Firebase.
The requirement is that:
Each Sport may contain many events.
Many teams would register for an event.
The queries to be handled would be:
Displaying team names given Sports name and Event name
Displaying details of all registered teams given Sports name and Event name
Am new to designing the database model for firebase. I have designed the firebase structure, as shown in the image.
Will this structure support my queries optimally?
Kindly suggest changes that I would have to make.
Thanks in advance:)
Firebase structure
1st, firebase push keys have the very handy feature of sorting chronological, meaning just apply alphanumeric sort and they will be in chronological order even for offline writes.
Ideally, you design your document in such a way that you don't need to query them. If you're coming from a MSSQL world, your documents will be analogous to MSSQL views, but with no tables to back them up.
Or in other words, store the documents as you expect the results of your queries.
Applying to an app
Let's suppose:
You have a view in your app showing the start datetime, # of teams joined, and prize, for all events in a given sport.
Clicking any of the events, you have a view showing start datetime, end datetime, venue, sponsors, and the team name of all teams joined.
This is essentially two nodes dedicated to such views:
So, what i get from your question is something like
Entities: Sports, Teams, Events
and it will be a many to many relationship between each entity but as you are dealing with non-sql based solution so you can get results for required queries with a structure similar to this one
Sports:{
sport_name,
id,
Teams:
[{team_name, id}]
}
Events: {
event_id
event_name
teams :
[
{team_name, team_id}
]
}
To get detail of every team you can have another node of teams in which you can keep all details.
So every time you need to get teams registerd for any event you can go directly to registered teams under teams node and same for the other query.
Related
We recently had a new business rule that will require our users to pay for individual modules in our web application.
So, all the features we build in the application will not apply to all users. Some users can choose to add features that they want.
I've tried researching into an architecture/mindset to how to approach this development.
If I could get an idea on how to get started with this.. I would very much appreciate it.
I work with .NET web applications, and Microsoft SQL Server.
Thanks.
First list what "objects" or things you need to keep track of.
Users
userid
fullname
can manage his features? You said not all users can
...
Features:
featureid
description
cost
...
UserHasFeature
a link between a user and a feature
each line is userid, featureid
Using this you can query which user has what feature. Or list the users that have access to a particular feature.
In your web app, you will need administrator functions:
users management: add, remove, modify, list
feature: add, remove, modify, list
link management: add, remove, list
Reports: whatever reports you want to have
And user functions:
user: signin, modify, reset password, view all features, view features the user already has, add a new feature, remove a feature
reports: total cost of features the user is using, others
Now this is a very quick first draft. There are a lot of missing requirements:
approval workflow: can a user modify his features without the approbation of X?
payment methods
project number for internal billing
cost structure: monthly, one time, ...?
managers can view the features of the employees he manages?
...
This to remember:
Start with objects in your projet. These become tables.
Characteristics of the objects become fields in your tables.
If the same characteristic appears in many object tables, with the same values, consider creating a new table for these. Ex. in an address, you would not leave the country value as a simple VARCHAR field. You would link to another table with the country values.
List the relations. These become foreign keys, or link tables.
Split your objects. So apply 1NF, 2NF and 3NF at least. It is enough for most applications. (NF == Normal Form).
Each table and links require administrator pages (CRUD)
Users have a limited view related to their features only.
This is a huge subject, I could go on and on, but this could get you started.
Have fun!
I am trying to learn RavenDB by replacing my RDBMS in a project that I've already worked on so that I'm using it in a real situation. I've come to a standstill while trying to create the database, and I'd love to know the best way to model this in a document database. Every possibility I come up with either ends up looking like a relational database or ends up repeating vasts amount of information. Repeating the information in the database isn't a big deal, but keeping it all up to date when changes occur would be.
I'm hoping that I'm stuck in SQL mode and I'm just completely unable to see an obvious answer.
Here are the basic objects I need to record data for:
-Event
-Person
-Organization
-Cabin
Simple Requirements:
-A person can be a part of multiple organizations.
-An organization can have many members (people).
-A person can attend multiple events.
-An event has many people that attend.
-Some details about a cabin may change depending on the event (e.g. Accommodations).
Complex Requirements:
-I need to be able to reserve cabins for an event so that a single cabin is not used by two events at once. (with RDBMS I would just create an "EventCabins" table).
-I need to be able to record which people are attending an event. People attending an event will have information associated with them that is not part of Person or Event.
-I need to be able to record which organizations are attending an event. Organizations attending will have information associated with them that is not part of Organization or Event.
-I need to be able to record which People are assigned to which cabins in a particular event.
-I need to be able to record which People are attending a particular event as a part of an organization (it's not required to attend as a part of an organization). Even though a person can belong to more than one organization, he/she can only attend as a part of one of those organizations for a particular event. He/she might attend as a part of a different organization for another event.
-In the program, the user will be looking at only one event at a time. In that event, the user can look at attenders grouped by cabin or grouped by organization.
It seems obvious that I will need separate collections for Events, People, Organizations, and Cabins. Fulfilling the complex requirements above is where I hit the wall.
Do I put Attenders inside the Event collection? If so, then what do I do with Cabins and Organizations?
Do I create a separate collection for Attenders? If so, then there will be 4 different related collections that I will need to store Ids for and query at various times (Organizations, Cabins, Events, People). This seems opposite of the document database approach.
Thanks!
It seems to me that you should just use a relational database for this project.
If you want to use RavenDB I would suggest to use completely separated collections for all of these objects, but keeping references to other documents. Then you could query database using .Include functionality. And the best way - to create map/reduce indecies for all of the possible cases, like an index returning object for Event filled with all of invited people.
I'm building a Rails 3.1 application that allows people to submit events. One of the fields for the event is a venue. On the create/edit form, the venue_name field has autocomplete functionality so it displays venues with a similar name, but the user is able to enter any name.
When the form is submitted, I'm using find_or_create_by_name when attaching the venue to the event model.
I'm doing this because it's not possible for us to maintain a complete list of venues and I don't want to prevent people from submitting an event because the venue isn't in the list.
The problem is that it's quite likely we'll get duplicates over time like "Venue Name" and "The Venue Name" or any number of other possibilities.
I was thinking that I probably just need to create an administrative tool that allows the admin to review recent venues and if he/she thinks they're duplicates to search/select a master record and have the duplicate record's association copied over to the master record and once successful to delete the duplicate record.
Is this a good approach? In terms of the data manipulation would it be best to handle this in a transaction? Would it be best to add this functionality in a sort of utility class - or directly in the Venue model?
Thanks for your time.
If I were going to put together a system like that, I'd probably try to find a unique identifier I could associate with each venue - perhaps an address or a phone number?
So, if I had "The Clubhouse" with a phone number 503-555-1212, and someone tried to input a new venue called "Clubhouse" with the phone number 503-555-1212, I might take them to an interstitial page where I ask them "Did you mean this location?"
Barring that, I might ask for a phone number or address first, then present a list of possible matches with the option to create a new venue.
Otherwise, you're introducing a lot of potential for error at the admin level, plus you run into a scalability problem. If your admin has to review 10 entries a month, maybe not so bad - but if your app takes off and that number goes to 1000, that becomes unmanageable fast!
I'm starting a MongoDB project just for kicks and as a chance to learn MongoDB/NoSQL schemas. It'll be a live chat app and the stack includes: Rails 3, Ruby 1.9.2, Devise, Mongoid/MongoDB, CarrierWave, Redis, JQuery.
I'll be handling the live chat polling/message queueing separately. Not sure how yet, either Node.js, APE or custom EventMachine app. But in regards to Mongo, I'm thinking to use it for everything else in the app, specifically chat logs and historical transcripts.
My question is how best to design the schema as all my previous experience has been with MySQL and relational DB schema's. And as a sub-question, when is it best to us embedded documents vs related documents.
The app will have:
Multiple accounts which have multiple rooms
Multiple rooms
Multiple users per room
List of rooms a user is allowed to be in
Multiple user chats per room
Searchable chat logs on a per room and per user basis
Optional file attachment for a given chat
Given Mongo (at least last time I checked) has a document limit of 4MB, I don't think having a collection for rooms and storing all room chats as embedded documents would work out so well.
From what I've thought about so far, I'm thinking of doing something like:
A collection for accounts
A collection for rooms
Each room relates back to an account
Related documents in chats collections for all chat messages in the room
Embedded Document listing all users currently in the room
A collection for users
Embedded Document listing all the rooms the user is currently in
Embedded Document listing all the rooms the user is allowed to be in
A collection for chats
Each chat relates back to a room in the rooms collection
Each chat relates back to a user in the users collection
Embedded document with info about optional uploaded file attachment.
My main concern is how far do I go until this ends up looking like a relational schema and I defeat the purpose? There is definitely more relating than embedding going on.
Another concern is that referencing related documents is much slower than accessing embedded documents I've heard.
I want to make generic queries such as:
Give me all rooms for an account
Give me all chats in a room (or filtered via date range)
Give me all chats from a specific user
Give me all uploaded files in a given room or for a given org
etc
Any suggestions on how to structure the schema efficiently in a way that scales? Thanks everyone.
I think you're pretty much on the right track. I'd use a capped collection for chat lines, with each line containing the user ID, room ID, timestamp, and what was said. This data would expire once the capped collection's "end" is reached, so if you needed a historical log you'd want to copy data out of the capped collection into a "log" collection periodically, but capped collections are specifically designed for logging-style applications where you aren't going to be deleting documents, and insertion order matters. In the case of chat, it's a perfect match.
The only other change I'd suggest would be to maintain uploads in a separate collection, as well.
I am a big fan of mongodb as a document database aswell. But are you sure you are using mongodb for the right reason? What is mongodb powerful at?
Its a subjective question but for me in-place (atomic) updates over documents is what makes mongodb powerful. And I can't really see you using it that much. And on top of that you are hitting the document size limit problem aswell.(With experience I can tell you that embedding files to mongodb is not a good idea). You want to have a live chat application on top of database too.
Your document schema's seems logical. But I wouldn't go with mongodb for this kind of project where your application heavily depends on inserts. I would go for CouchDB.
With CouchDB you wouldn't have to worry about attachments problem, you can embed them easily. "_changes" would make your life much much easier to eighter build a live chat application / long pooling / feeding search engine (if you want to implement one).
And I saw an open source showcase project in couchone. It has some similarities with your goals: Anologue. You should check it out.
PS : Sorry it was a little off topic but I couldn't hold myself.
Social networking website probably maintain tables for users, friends and events...
How do they use these tables to compute friends events in an efficient and scalable manner?
Many of the social networking sites like Twitter don't use an RDBMS at all but a Message Queue application. A lot of them start out with a already present application like RabbitMQ. Some of them get big enough they have to heavily customize or build their own. Twitter is in the process of doing this for the second time.
A message queue application works by holding messages from one service for one or more other services. For instance say service Frank is publishing messages to a queue foo. Joe and Jill are subscribed to Franks foo queue. the application will keep track of whether or not Joe or Jill have recieved the messages and once every subscriber to the queue has recieved the message it discards it. Frank fires messages and forgets about it. Joe and Jill ask for messages from foo and get whatever messages they haven't gotten yet. Joe and Jill do whatever they need to do with the message. Perhaps keeping it around perhaps not.
The message queue application guarantees that everyone who is supposed to get the message can and will get the message when they request them. The publisher can send the messages confident that subscriber can get them eventually. This has the benefit of being completely asynchronous and not requiring costly joins.
EDIT: I should mention also that usually the storage for these kind of things at high scale are heavily denormalized. So Joe and Jill may be storing a copy of the exact same message. This is considered ok because it helps the application scale to billions of users.
Other reading:
http://www.rabbitmq.com/
http://qpid.apache.org/
The mainstay data structure of social networking sites is the graph. On facebook the graph is undirected (When you're someone's friend, they're you're friend). On twitter the graph is directed (You follow someone, but they don't necessarily follow you).
The two popular ways to represent graphs are adjacency lists and adjacency matrices.
An adjacency list is simply a list of edges on the graph. Consider a user with an integer userid.
User1, User2
1 2
1 3
2 3
The undirected interpretation of these records is that user 1 is friends with users 2 and 3 and user 2 is also friends with user 3.
Representing this in a database table is trivial. It is the many to many relationship join table that we are familiar with. SQL queries to find friends of a particular user are quite easy to write.
Now that you know a particular user's friends, you just need to join those results to the updates table. This table contains all the user's updates indexed by user id.
As long as all these tables are properly indexed, you'd have a pretty easy time designing efficient queries to answer the questions you're interested in.
Travis wrote a great post on this ,
Activity Logs and Friend Feeds on Rails & pfeed
For the small scale doing a join on users.friends and users.events and query caching is probably fine but does slow down pretty quickly as friends and events grow. You could also try an event based model in which every time a user creates an event an entry is created in a join table (perhaps called "friends_events"). Thus whenever a user wants to see what events their friends have created they can simply do a join between their own id and the friends_events table and find out. In this way you avoid grabbing all a users with friends and then joining their friends with the events table.