Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
I have a website that customers can log into and buy products from a specific range (based on their customer number). The navbar has a dropdown for product categories, but doesn't show any where there would be no products.
At the minute, the database is queried on each page to populate the product categories menu, but this feels inefficient. I'm now wondering if the list of 'allowed' products should be stored in either cookies or session variables at the beginning of a session.
From what I have gathered from other questions, a session variable would normally be used, but they have warned against storing a lot of data in session variables. In this case, I would be storing a potentially large list of products for each user, so would cookies be better? I wouldn't class a list of products as being particularly sensitive, and I would do a server-side check before placing the order. Or should I stick with the current solution of querying the database each time?
To be clear, I will still store the information in the database as well, this question is only asking about a temporary storage for quick access throughout the session.
I have already looked at the following questions, but I still don't feel like I've quite got the answer for my particular scenario, and the questions that are close haven't mentioned cookies.
Cache VS Session VS cookies?
Cookie VS Session
Session Cookie vs Persistent Cookie
Trade off between user data in session vs database?
session variables vs database
The approach I would take would probably be something like this:
Store the product data in a cache that is shared between all customers and use the primary key of each product as (part of) the cache key for each product. The cache strategy used could vary depending on scalability requirements (System.Runtime.Caching vs redis vs file caching). In general, I would probably use a sliding expiration or LRU cache (see this .NET example) for the product data so the most popular products stay cached longer than less popular products. When a request comes for a particular primary key, the cache is checked first for the key and if it returns null, the product is looked up from the database and cache populated before returning the product (see this example).
For each customer, store only the primary key and other relevant data (what class of discount they may get, etc.). If the primary key list is small enough, you can probably get by with a cookie (encrypted) for this. If you need this to scale to give each customer more than will fit in a cookie, then query the database for the primary keys.
When the customer requests the data, get the list of primary keys first, then use those keys to access the pre-cached product data to build the view. You can store as many different primary key lists as you need to meet your requirements without having to get the product data from the database for every customer or every use case for an individual customer.
Think twice about using session state - the advice there is not to use session state for user profile data.
Of course, the above doesn't take into consideration any other marketing requirements that may be needed. So you may need to adjust this strategy for your needs. For example, if you need to update the product information faster than the cache would normally expire, you might consider an approach that updates the product data in the database first and, if successful, gets a write lock on the cache and updates the cache, too. That would allow for near-real-time cache updates without having to invalidate and reload the cache from the database.
Related
I have already read Rails - How do I temporarily store a rails model instance? and similar questions but I cannot find a successful answer.
Imagine I have the model Customer, which may contain a huge amount of information attached (simple attributes, data in other tables through has_many relation, etc...). I want the application's user to access all data in a single page with a single Save button on it. As the user makes changes in the data (i.e. he changes simple attributes, adds or deletes has_many items,...) I want the application to update the model, but without committing changes to the database. Only when the user clicks on Save, the model must be committed.
For achieving this I need the model to be kept by Rails between HTTP requests. Furthermore, two different users may be changing the model's data at the same time, so these temporary instances should be bound to the Rails session.
Is there any way to achieve this? Is it actually a good idea? And, if not, how can one design a web application in which changes in a model cannot be retained in the browser but in the server until the user wants to commit them?
EDIT
Based on user smallbutton.com's proposal, I wonder if serializing the model instance to a temporary file (whose path would be stored in the session hash), and then reloading it each time a new request arrives, would do the trick. Would it work in all cases? Is there any piece of information that would be lost during serialization/deserialization?
As HTTP requests are stateless you need some kind of storeage between requests. The session is the easiest way to store data between requests. As for you the session will not be enough because you need it to be accessed by multiple users.
I see two ways to achive your goal:
1) Get some fast external data storage like a key-value server (redis, or anything you prefer http://nosql-database.org/) where you put your objects via serializing/deserializing (eg. JSON).
This may be fast depending on your design choices and data model but this is the harder approach.
2) Just store your Objects in the DB as you would regularly do and get them versioned: (https://github.com/airblade/paper_trail). Then you can just store a timestamp when people hit the save-button and you can always go back to this state. This would be the easier approach i guess but may be a bit slower depending on the size of your data model changes ( but I think it'll do )
EDIT: If you need real-time collaboration between users you should probably have a look at something like Firebase
EDIT2: Anwer to your second question, whether you can put the data into a file:
Sure you can do that. But you would need some kind of locking to prevent data loss if more than one person is editing. You will need that aswell if you go for 1) but tools like redis already include locks to achive your goal (eg. redis-semaphore). Depending on your data you may need to build some logic for merging different changes of different users.
3) Another aproach that came to my mind would be doing all editing with Javascript and save it in one db-transaction. This would go well with synchronization tools like firebase (or your own synchronization via Rails streaming API)
I am looking for solution of logging data changes for public API.
There is a need to tell client app which tables form db has changed and need to be synchronised since the app synchronised last time and also need to be for specific brand and country.
Current Solution:
Version table with class_names of models which is touched from every model on create, delete, touch and save action.
When we are touching version for specific model we also look at the reflected associations and touch them too.
Version model is scoped to brand and country
REST API is responding to a request that includes last_sync_at:timestamp, brand and country
Rails look at Version with given attributes and return class_names of models which were changed since lans_sync_at timestamp.
This solution works but the problem is performance and is also hard to maintenance.
UPDATE 1:
Maybe the simple question is.
What is the best practice how to find out and tell frontend apps when and what needs to be synchronized. In terms of whole concept.
Conditions:
Front end apps needs to download only their own content changes not whole dataset.
Does not invoked synchronization when application from different country or brand needs to be synchronized.
Thank you.
I think that the best solution would be to use redis (or some other key-value store) and save your information there. Writing to redis is much faster than any sql db. You can write some service class that would save the data like:
RegisterTableUpdate.set(table_name, country_id, brand_id, timestamp)
Such call would save given timestamp under key that could look like i.e. table-update-1-1-users, where first number is country id, second number is brand id, followed by table name (or you could use country and brand names if needed). If you would like to find out which tables have changed you would just need to find redis keys with query "table-update-1-1-*", iterate through them and check which are newer than timestamp sent through api.
It is worth to rmember that redis is not as reliable as sql databases. Its reliability depends on configuration so you might want to read redis guidelines and decide if you would like to go for it.
You can take advantage of the fact that ActiveModel automatically logs every time it updates a table row (the 'Updated at' column)
When checking what needs to be updated, select the objects you are interested in and compare their 'Updated at' with the timestamp from the client app
The advantage of this approach is that you don't need to keep an additional table that lists all the updates on models, which should speed things up for the API users and be easier to maintain.
The disadvantage is that you cannot see the changes in data over time, you only know that a change occurred and you can access the latest version. If you need to track changes in data over time efficiently, than I'm afraid you'll have to rework things from the top.
(read last part - this is what you are interested in)
I would recommend that you use the decorator design pattern for changing the client queries. So the client sends a query of what he wants and the server decides what to give him based on the client's last update.
so:
the client sends a query that includes the time it last synched
the server sees the query and takes into account the client's nature (device-country)
the server decorates (changes accordingly) the query to request from the DB only the relevant data, and if that is not possible:
after the data are returned from the database manager they are trimmed to be relevant to where they are going
returns to the client all the new stuff that the client cares about.
I assume that you have a time entered field on your DB entries.
In that case the "decoration" of the query (abstractly) would be just to add something like a "WHERE" clause in your query and state you want data entered after the last update.
Finally, if you want that to be done for many devices/locales/whatever implement a decorator for the query and the result of the query and serve them to your clients as they should be served. (Keep in mind that in contrast with a subclassing approach you will only have to implement one decorator for each device/locale/whatever - not for all combinations!
Hope this helped!
I'm using acts_as_taggable_on for tagging items across my system so that they're easily searchable.
Now I have a UX problem: I'm noticing lots of places where users choose certain minor states (for example, closing a one-time help box or moving to the next javascript-run step in a given page). We have here situations that are both too minor/numerous/dynamic/fast-changing to be put into a database table (imagine having to migrate with every UX change!), and that there is a need to persist some of these choices beyond the session.
In this case, is there anything wrong with using tags to store these simple decisions? For example, user.set_tags_on(:ui, "closed_index_help") or user.set_tags_on(:ui, "tutorial_1_done"), then showing/hiding these elements in the future by looking at the user's ui_list.
Are there drawbacks to this I'm not considering or is this a prudent way to go?
Another way might be to store the information in the SESSION. You will of course have to migrate the session information to be stored in the DB rather than the cookie, but at least that way - you only have to retrieve the session once.
I'm just starting to learn ASP.NET MVC and I'd like to know how I can retain model objects between subsequent requests to controller action methods?
For example say I'm creating a contact list web app. Users can create, update, rename, and delete contacts in their list. However, I also want users to be able to upload a contact list exported from other programs. Yet I don't want to just automatically add all the contacts in the uploaded file I want to give the user a secondary form where they can pick which uploaded contacts should be actualy added to their list.
So first I have a ContactController.Upload() method which shows an upload form. This submits to ContactController.Upload(HttpPostedFileBase file) which reads the file that was posted into a set of Contact model objects. Then I want to display a list of all the names of the contacts in the list and allow the user to select those that should be added to their contact list. This might be a long list that needs to be split up into multiple pages, and I might also want to allow the user to edit the details of the contacts before they are actually added to their contact list.
Where should I save the model objects between when a user uploads a file and when they finally submit the specific contacts they want? I'd rather not immediately load all the uploaded contacts into the back end database, as the user may end up only selecting a handful to actually add. Then the rest would need to be deleted. Also I would have to account for the case when a user uploads a file, but never actually completes the upload.
From what I understand an instance of a controller only lasts for one request. So should I create a static property on my Contact controller that contains all the latest uploaded contact model object collections? And then have some process that periodically checks the age of these collections and clears out any that are older then some specified expiration time?
A static property on the controller is trouble. First off, it won't work in a web farm and second it you'd have to deal with multiple requests from different users. If you really don't want to use your database you could use the ASP.NET Session.
No, you don't want a static property, as that would be static to all instances of the controller, even for other users.
Instead, you should create a table used to upload the data to. This table would be used as an intermediary between when the user uploads the data, and completes the process. Upon completion, you copy the contacts you want to keep into your permanent table, then delete the temporary data. You can then run a process every so often that purges incomplete data that is older than a specified time limit.
You could also use the HttpContext.Cache, which supports expiration (and sliding expiration) out-of-the box.
Alternatively, and perhaps even better (but more work) you could use cookies and have the user modify the data using javascript in her browser before finally posting it to you.
However, I'd strongly recommend to store the uploaded information in the database instead.
As you pointed out, it might be a lot of data and the user might want to edit it before clicking 'confirm'. What happens if the user's machine (or browser) crashes or she has to leave urgently?
Depending on the way you store the data the data in this scenario will probably be lost. Even if you used the user id as a cache key, a server restart, cache expiration or cache overflow would cause data loss.
The best solution is probably a combination of database and cookie storage where the DB keeps the information in a temporary collection. Every n minutes, or upon pagination, the modified data is sent to the server and updated in the DB.
The problem with storing the data in session or memory is what happens if the user uploads 50k contacts or more. You then have a very large data set in memory to deal with which depending on your platform may effect application performance.
If this is never going to be an issue and the size of the imported contacts list is manageable you can use either the session or cache to store the dataset for further modifications. Just remember to clear it when the user has committed the changes, you don't want a few heavy datasets hanging around in session.
If you store the dataset in session using your application controller then it will be available to all controllers while it is needed.
I'm trying to run my site off the lazy idea of not having user registrations.
Anyway, I want a user to be able to "favorite" items on the site when they click "favorite" off of an "item"
I'm assuming I need to use cookies for this but I don't really know the next step. Could anyone point me in the right direction?
Thanks!
Not rails specific, but I dislike the idea of storing too much stuff in a cookie. Instead I store it on a database on my web site, and just put a primary key value in the cookie. That way you can store as much or as little as you like without transmitting too much over the net. The disadvantage of the "no registrations" approach is that they lose all their stored data if they change to a new computer, or even a new browser on the same computer.
What I do in my app is to store a cookie with a "session id". Then I have several tables, one stores one-off data like the last date that session id was seen, and others store multiple items per session id. I have a "session_states(session_id, state)" table, for instance, that stores one record for each state a person chooses from a list of US states.
One reason for storing the last date is that I purge any session ids from the database that haven't been seen in two years (because I give an expiration of +2 years when I create the cookie).
Maybe you could learn more about cookies on Rails. This link has a lot of information about this topic. I think that you will appreciate it!
But take some care to not store too much information using cookies.