I am creating a management system and I want to know how "Ruby on Rails" can support me in the mission of ensuring that each customer has their information, records and tables independent from other customers.
Is it better to put everything in a database and put a customer identifier to pull information through this parameter in queries or create a database for each customer automatically?
I admit that the second option attracts me more ... And I know that putting everything in one database will be detrimental to performance, because I assume that customers and their data will increase exponentially!
I want to know which option is more viable in the long run. And if the best option is to create separate databases, how can I do this with Ruby on Rails ??
There are pro and cons for both solutions which really depend on your use case.
Separating each customer in its own database has definitely advantages for scaling, running in different data centres or even onsite. However, this comes with higher complexity. For instance you can't query across customers anymore, you would need to run queries for each customers and aggregate the results. This approach is called multi tenancy (or shardening). There is a good gem called Apartment available (https://github.com/influitive/apartment).
Keeping everything in one database might be simpler to start of as it's less complex but it really depends on your use case.
Edit
Adding some more information based on the questions.
There are several reasons to use a one db per client architecture.
You have clearly separated tenants. In case it might make sense to go with the one db approach.
Scale. Having separated databases for each tenant makes scaling of course easier.
If 2) is the main reason you want to go for a one db per client approach I would strongly advise you against it. You add so much more complexity to your app which you might not need for years to come (if ever).
If scaling is your main concern I recommend reading Designing Data Intensive Applications by Martin Kleppmann. But basically, don't worry about scale for the first few years and focus on your product.
so I am trying to set up a data warehouse for a service where each customer has their own database with a unique schema. How do I go about setting up a warehouse so each customer has their own semantic layer / relational model set up automatically (since we (centrally) do not know what is in each database) So that each customer can easily report on their data? Is there any automatic process we can follow? Am I missing something?
It depends on whether you want a consolidated view of the data, or if each customer's data is to remain segregated.
If consolidation is the objective (and there are huge benefits for a multi-tenant SAAS vendor to have a consolidated overview of customer data) then Nithin B's suggestion is good.
If separate warehouses are required, then you'll need to think about how to optimise your costs. The two biggest components will be ETL/ELT, and database hosting.
The fastest way to ETL/ELT is data warehouse automation. You'll find a good list of vendors on our web site (http://ajilius.com/competitors). Look for a solution that will give you the flexibility to meet your deployment options (cloud and/or on-premise), as well as the geographic reach you'll need for accessing customer data.
Will you be hosting your own databases or in the cloud? How much data will each tenant require? A good starting point would be PostgreSQL or SQL Server (SMP), and Ajilius gives you the flexibility to instantly migrate to MPP platforms if your needs outgrow those platforms.
There are many ways to address this.
Land all the tables in a Landing area in different schemas.
Stage the data into appropriate staging tables for dim and fact loads.
Create a dim table to identify the Customer Area. For eg: Dim_Source
Load the data into the fact tables. Any specific customers can filter the data from the facts by using the Dim_Source values.
This design would help overall Enterprise reporting as well.
Hope that helps.
I would start with a Kimball BUS Matrix.
Cheers
Nithin
I am about to embark on the creation of one of my first meaningful rails apps, and am a bit unsure of how to structure things.
Here's my situation. I am using a SAAS inventory software that keeps track of approximately 4000 products. I need an app that can perform routine maintenance functions on the products. For example, to give you an idea:
Every week, calculate and set the "low stock alert quantity" for each product based on historic sales
Assign products to product categories based on rules (i.e. if the word "t-shirt" is in the title, automatically add it to the t-shirts category)
etc etc
My questions are as follows:
I'm not sure exactly how to structure the data in this app. Should I query the API each time I need to retrieve the products? Or should I build a local copy of the inventory in a local database, for faster querying? If I am to build a local copy of the database, what would be an efficient way of keeping an up to date version of each product without consuming too much server resources? Obviously I can't pull 4000 products via the API at once...and I also don't want a cron job to be running every minute of the day.
Where should I place the code for my remote API functions? Do I create a class, module, or something else?
Thanks for any advice.
First of all, do you really need a web interface? I don't see so in your app description. If what you want to build is a maintenance set of tools that would be executed periodically, why would you use Rails?
The way of structuring data depends on your needs and resources. Does your SAAS provide instant notifications on products update, such as webhooks? Does your SAAS API let you fetch only products that were updated after a certain date? What's more important to you, speed or working with up to date data? Answering those questions to yourself should help you to decide what the best approach is.
Regarding the app design, it looks that you could write a gem where you could have different modules to manage data, execute operations with that data and connect to the third party service, and use that gem in scripts that would execute periodically. I would recommend you reading a book on Object Oriented design that I truly believe it'd help you, Practical Object-Oriented Design in Ruby.
Just one last word, I'm taking too many assumptions here, no one but you know better your requirements, and the design of your software depends on that.
If your API follows REST then it can very easily be integrated into rails using ActiveResource. If the API is more complicated, you could use ActiveModel and roll your own implementation. Just create a base class of your own that handles connection and authentication and extend your models from that. Given the limited description of your problem, there's not much more I can say.
I have a rails application which manages different types of items and users who own them. Items of different types might have different features. There is a number of sinatra services which have to access items (read-only, every service one specific item type).
Is it a good idea to create separate tables / databases for every service and to keep them in sync with the rails DB? In this case the main DB will hold all items. It's postgres, so hstore could be used for different features. On all updates a sync message will be sent using Redis pub/sub or RabbitMQ messaging. Services will subscribe and update service specific tables.
The system should be really reliable, scalable, and prepared for high-load and new not yet known item categories. What do you think? Does it make sense or are there better approaches for these requirements? Thank you in advance, I really appreciate your help!
There is no one-size-fits-all answer here. The answer depends on your requirements and these will decide which of two approaches you might take.
The first approach is conceptually the simplest, which is to have every service hit the same database. The advantage here is that you can scale up relatively easily, the system is simple and flexible, and you can do a lot with the database to keep things working well. The disadvantage is that db downtime will take down all services at once.
The second approach is to keep every service (or group of closely related services) as separate self-contained service, kept in sync with some sort of message passing. This has the advantage of being more robust in terms of delivering basic services, but far less robust in terms of everything staying in sync (because the CAP theorem's consistency requirement is sacrificed for availability, and your data is effectively partitioned).
I don't know which one you will want to use. To the extent possible I would usually choose the single db approach but I am a Postgres guy, not a Rails guy. The second approach also works quite well in some cases but it does have a complexity cost.
At my company we're about to build a new site using ASP.NET MVC. My boss (marketing guy) would like to know some more about the technology so I've tried to find a really good, simple and pedagogical presentation of the MVC concept without any luck. Most of them require quite a lot of basic knowledge in programming.
Any suggestions for a good video, slides or other?
Craig Strong has a pretty nice article about MVC in general and how to explain its benefits to business. Check it out here: Updated link.
Define MVC in layman’s terms
Remember you’re technically minded and close to the code. MVC to you
is as clear as day, but saying to the business ‘Model, View,
Contoller’ could give them the impression that you are suffering from
some form tourette syndrome. MVC won’t mean much to the business even
after you define them in relation to the code. To get the business to
understand why this is the answer and least of all what it is, can be
more of a task than expected in my experience. Even some fellow
developers have difficulty understanding this on occasion.
To get the listener to understand what MVC is and why it works what I
have tried in the pass is to apply MVC to a different industries where
the listeners have had more involvement. An example that has worked
for me in the past in a comparison to the property or even the
vehicles. Most people have had dealing’s with builders, carpenters,
plumbers, electricians or have watched the flood of property shows on
the TV. This experience is a good platform to use and to explain why
separation such as MVC works. I know you’re probably thinking that
won’t work as it’s not the same as in software, but remember you’re
not trying to train the business to become developers or have an in
depth understanding of MVC, simply explaining to them that separation
in production is required and that’s what an MVC structure offers.
To give an example of how you could describe this I have very briefly
explained how separation works in property. Keep in mind this is
focused on using the system not developing which could be a completely
different angle of explanation.
View
The view in MVC is the presentation layer. This is what the end user
of a product will see and interact with. A system can have multiple
views of all different types ranging from command line output to
rendered HTML. The view doesn’t consist of business logic in most
clear designs. The interface is fit for purpose and is the area of
interaction. Therefore you could simply output HTML for consumers to
interact with or output SOAP/XML for businesses to interact with. Both
use the same business logic behind the system otherwise known as the
models and controllers.
In the world of property you could think of the view as the interior
of a property or the outer layer of a property that the inhabitants
interact with. The interior can be customised for purpose and the same
property can have many different types of tenants. For example a
property of a particular design could contain residential dwellings.
The same internal space could easily be used as office space, where
although in the same property has a different purpose. However the
property structure is the same. Therefore the environment in which the
users interact does not interfere with the structure of the building.
Controllers
The controller is where the magic happens and defines the business
application logic. This could be where the user has sent a response
from the view, then this response is used to process the internal
workings of the request and processes the response back to the user.
Taking a typical response where a user has requested to buy a book.
The controller has the user id, payment details, shipping address and
item choice. These elements are then processed through the business
logic to complete a purchase. The data is passed through the system
into the model layer and eventually after the entire request satisfies
the business definitions, the order is constructed and the user
receives their item.
If we compare this to a property, we could compare the ordering of a
book online to turning on a light switch. A tenant will flick the
switch to on just like ordering a book. The switch itself is an
element in the view layer which sends the request to the controller
just like clicking a checkout button on a web site. The business logic
in this case is what the electrician installed and are embedded within
the property designs. The switch is flicked, which completes the
circuit. Electricity runs through all the wires including the fuse box
straight through to the light bulb. Just like the user receiving a
book, in this case the tenant receives light. The whole process behind
the scenes involving the electricity cabling is not visible to the the
tenant. They simply interact with the switch within the space and from
there the controller handles the request.
Models
The models in MVC are the bottom most layer and handle the core logic
of the system. In most cases this could be seen as the layer that
interacts with the data source. In systems using MVC, the controller
will pass information to the model in order to store and retrieve
data. Following on from the example above controller definition, this
is where the order details are stored. Additional data such as stock
levels, physical location of product of the book amongst many things
are all stored here. If that was the last book in stock ordered, the
next request for this item may check if it’s available and disallow
the order as the item is no longer available.
Sticking with our example of turning on a light switch, this level in
our structure could be the electricity supply. When the tenant flicks
the switch, the internal circuit must request electricity to power the
request which is similar when the user requested data from the
database, as in data is needed to process a request. If the dwelling
isn’t connected to an electric supply, it cannot complete the process.
Business benefits from using MVC
After you get the message across explaining what MVC is, you will then
have to see what benefits can be obtained from it. I’m not going to go
into a huge amount of detail here are I’m sure you can apply benefits
more accurately which are directly related to you actual situation. To
list just some of the common benefits of an MVC based system here are
a few examples:
Different skill levels can work on different system levels. For example designers can work on the interface (View) with very little
development knowledge and developers can work on the business logic
(Controller) with very little concern for the design level. Then they
simply integrate together on completion.
As a result of the above separation projects can be managed easier and quicker. The designer can start the interfaces before the
developer and vice versa. This development process can be parallel as
opposed to being sequential therefore reducing development time.
Easy to have multiple view types using the same business logic.
Clear route through the system. You clearly know where there different levels of the system are. With a clear route of the system,
logic can be shared and improved. This has added security benefits as
you clearly know the permitted route from the data to the user and can
have clear security checks along the route.
Each layer is responsible for itself. (Relates to point 1) This means that you can have clean file structure which can be maintained
and managed much easier and quicker than a tightly couple system where
you may have lots of duplicate logic.
Having a clear structure means development will be more transparent which should result in reduced development time,
maintenance problems and release cycles if applied properly.
M-V-C Think of it as:
"Order Details (including Customer & Employee info)", "HTML/ASP Form (to display the OrderDetails)" and "Order details service class (having methods to SaveOrderDetails, GetOrderDetails etc.).
The Model (Data Class e.g. OrderDetails)
The data you want to Display
The Controller (Service class)
Knows about the Model (Order Details)
Has methods to manage the Model
And as such can be unit tested Its Single Responsibility is to manage the OrderDetails CRUD operations.
It knows NOTHING about the View
The View (ASP Page)
Displays the Model (OrderDetail's ViewData).
It has to know about the Model's structure so it can correctly display the data to the users on screen.
The View's structure (style, layout, HTML etc., locale) can be changed at anytime without it changing anything in the application's functionality.
And as such, many Views can display the same Model in many different ways.
In multi-tenant web applications, Customer specific Views can be stored in a database table and displayed based on Customer information
You have to explain the benefits of ASP.NET MVC, not the features
You have control over your URLs -- that means SEO for the site will be better -- that means your site will be higher in google
The code is cleaner, which means that it's easier to change, which means that you can add features faster
etc.
How do you save money, make money, reduce risk? That's what your boss wants to know.
Imagine a control room in a factory, the model is the machine itself, the monitoring equipment is the view and the instrument panel is the controller. You could have several different control rooms for the same machine and changes in the controls in one control room would reflect on the monitors in all control rooms.
The point is that you should only model once and then view or control however is most convenient.
The model is the data access layer, which can just be a wrapper for a few simple queries to an ORM that manages the data entity relationships itself. It handles communication to the data source, retrieves data and usually organizes it into objects defined in your application.
The views are just html files with bits of html and css with some templating engine (smarty, mako, etc) code to display the data passed to it the way you want.
The controller puts it all together. Requests made to your page will be routed to a controller (class) and an action (method) within the controller. Just like any other application, the action will do what's requested of it, but it's still part of the controller.
So, the controller uses the model to query data (users, content, etc), then passes the data to a view to be rendered and displayed the way you want.
I wouldn't try to explain the technology to him, I'd try to explain what the MVC architectural principle is all about.
MVC was designed to separate concerns. Plain and simple. Explain to him that when you build anything that what you're building can be classified in two different categories: what the business need is (the domain), and everything else.
MVC separates the Domain from the everything else by introducing layers to separate out the concerns. M is for Model, which is your domain. V is for View, which is the visible part to him, what he sees. C is for Controller, the part that controls what is going on in between the Domain and the View.
The marketing guy would just be interested in the "V" part, the View. Depending on how you design things, the View would just be basic HTML/CSS "templates" that the marketing person could modify. Technically without breaking anything.
Ideally the Model (database) and Controller (logic) shouldn't care if the View (presentation) is XML, HTML, text, etc. The marketing person shouldn't care what the Model and Controller do, except for requesting additional functionality.
Going further with the "ideal", you should technically be able to replace ASP with PHP, Java, Ruby, etc as the Controller without touching the Model or View.
You can very easily do this, that is if you understand marking speak. I dont but I imagine it would go something like this...
This should be use. MVC (if done right) will allow you to decouple the UI from the data (model) and control of the UI (controler). This will allow the UI to be more flexible which will in turn allow to better market it self faster.
To a marketing guy, perhaps the best way to explain the reason for ASP.Net MVC is the ability to broaden your product's reach.
By using MVC, the code is already separated in a fashion that will let you more easily build an interface that feels natural on a desktop, and then the different interface that caters to a general mobile device user, and a still-slightly-different interface that caters to an iPhone user, without risking the backend code getting out of sync and introducing subtle and company-harming bugs. And, if there's a smart client desktop app that could be a product... it, too, can rest on the same codebase.
The Model is "how things work inside the box". The Controller is "what you can touch on the outside of the box" and the view is "what comes out of the box"...
The most important thing for your marketing guy is money, budget, TCO ...
When you don't use MVC you usually mix design, application logic etc. alltogether.
Programmer then must know html design, programming etc... That could mean you need powerful professional to do it all.
if you use MVC, everything is divided into "separate parts". Html coder can prepare html layer, programmer only works with application logic etc...
MVC brings better granularity and everybody can focus on what he or she can do the best!
Listen, for example xhtml validity and css cleanliness is so hard that there is a lot of people who focuses only on this while lot of browsers and platforms compatibility on mind.
Usually one person is NOT the best asp.net programmer, xhtml coder in one ;-)
This is a pretty simple one
http://en.wikipedia.org/wiki/Model-view-controller#Pattern_description
The best way I can thing of is that the model is the data representation, the view is the presentation to the user and the controller is what collects user interaction that changes the model.
The important word in the title of the manager in this case is "marketing." He is a Marketing manager. The concerns one has as a marketing manager have to do with strategy and tactics. These two are not the same thing. Strategy is the big picture word that embraces among other things how a company conceptually addresses customer needs and how the company differentiates itself from its competition. Strategy is typically not what software can portray to a user. Tactics, on the other hand, are the direct methods or approaches that a company takes in winning the business of the customer. Tactics tend to change far more frequently than strategies, and it is likely that the marketing manager, when he asks what advantage MVC may give him, is really asking, "How rapidly can you change whatever it is that you create into something that conforms to new realities in the way we have to deal with customers." In other words, how quickly can you change an offer of "buy 1 and get 1 free" into "buy 2 on Friday and get 1 on the following Tuesday if it is raining in Albany."
Marketing management is about results measured in dollars and cents, not finery and nuanced explanations that are littered with conceptual words lacking any real specificity. Everything a programmer may say might make sense to himself, but a marketing manager needs to know the real likelihood of rapid response to changing customer perceptions or rapid implementation to a different approach to selling to the same customers. He needs to know if it will cost more than an existing method because if he sells $1 million more in product while spending $1.25 million in software development, he will probably lose his job.
So, in short, he is looking for flexibility and cost-effectiveness. He needs software that be adapted to changing conditions quickly, just as he changes his pitch first one way and then another to a difficult-to-persuade prospective customer, and he needs to know that he won't have to be liable for a huge price tag for that flexibility.
Frankly, I don't think that you would be able to deliver on such promises if they were made because in spite of all the advantages of MVC from a development point of view, we are still talking about software here, and as we all know, software is a rigid, demanding taskmaster that takes it own sweet time to mature to the point of trustworthiness and to be rid of its bugs. We as programmers are always in search of the holy grail of software reusability, and while we flail about trying one thing and then another (MVC, MVP, MVVM, and whatever else someone may conceive), the rest of the world is simply asking for something that works. So the best of luck to you. I hope you are able to win your case.