I have an spring-boot application which I would like to deploy it into multiple docker instances and there is a load balance before the instances.
However, the application uses ehcache to cache some data from a database. It makes the application stateful.
So without session sticky, a same customer might hit different docker instances and see different results.
My question is if I can't apply session sticky in load balance, what is the best practice to deploy an app with cache feature via docker style and still comply the rule of should-be-stateless?
I explain here in this devoxx video how clustered caching can help each of you docker instance share the same cache
First of all, if you really have a pure caching use case, there should be no correctness impact only a performance one. Which of course can in itself be a bad thing for your application.
But effectively, if you want to use caching to provide performance and at the same time have a multi-node ability without sticky sessions, you have to move into the realm of distributed caching. This will give you the ability to share the cache content amongst the different nodes and thus make it (more) transparent for a given request in a conversation to hit any node of your application.
In the Ehcache world, this means backing the cache with a Terracotta server, see the documentation for details.
It is common to combine ehcache with terracotta to allow distributed caching among nodes.
Regards
Related
The documentation for the ActiveSupport::Cache::RedisCacheStore states:
Take care to use a dedicated Redis cache rather than pointing this at your existing Redis server. It won't cope well with mixed usage patterns and it won't expire cache entries by default.
Is this advice still true in general, especially when talking about custom data caches, not page (fragment) caches?
Or, more specifically, If I'm building a custom cache for specific "costly" backend calls to a slow 3rd-party API and I set an explicit expires_in value on my cache (or all my cached values), does this advice apply to me at all?
TLDR; yes, as long as you set an eviction policy. Which one? Read on.
On the same page, the docs for #new state that:
No expiry is set on cache entries by default. Redis is expected to be configured with an eviction policy that automatically deletes least-recently or -frequently used keys when it reaches max memory. See redis.io/topics/lru-cache for cache server setup.
This is more about memory management and access patterns than what's being cached. The Redis eviction policy documentation has a detailed section for policy choice and mixed usage (whether to use a single instance or not):
Picking the right eviction policy is important depending on the access pattern of your application, however you can reconfigure the policy at runtime while the application is running, and monitor the number of cache misses and hits using the Redis INFO output to tune your setup.
In general as a rule of thumb:
Use the allkeys-lru policy when you expect a power-law distribution in the popularity of your requests. That is, you expect a subset of elements will be accessed far more often than the rest. This is a good pick if you are unsure.
Use the allkeys-random if you have a cyclic access where all the keys are scanned continuously, or when you expect the distribution to be uniform.
Use the volatile-ttl if you want to be able to provide hints to Redis about what are good candidate for expiration by using different TTL values when you create your cache objects.
The volatile-lru and volatile-random policies are mainly useful when you want to use a single instance for both caching and to have a set of persistent keys. However it is usually a better idea to run two Redis instances to solve such a problem.
It is also worth noting that setting an expire value to a key costs memory, so using a policy like allkeys-lru is more memory efficient since there is no need for an expire configuration for the key to be evicted under memory pressure.
You do not have mixed usage. For example, you do not persist Sidekiq jobs in Redis, which have no TTL/expiry by default. So, you can treat your Redis instance as cache-only.
I have a rails application running on a single VPS that uses passenger, apache and MySQL. I am moving this to Amazon AWS with the following simple setup:
ELB > Web Server > MySQL
Lets say I am expecting a huge spike in daily users and want to start to scale this out on Amazon AWS using multiple instances. Where does a newbie start on this journey? Do I simply create an AMI from my production configured web server and get the ASG to launch these when required?
I understand that AWS increases the number of instances using auto scale groups as the load demands it, but do I need to architect anything differently in my Rails application for it to run at scale across multiple interfaces?
The problem with scaling horizontally is that it really depends on the application. There's no "just-add-water" ways to do it.
But there are some generic recipes you can follow in the beginning:
Extract MySQL server into a separate instance, which is capable of holding a higher load. Then create as many worker (i.e. app) instances that connect to the MySQL database as you need. You can keep doing so before your MySQL server gets saturated with requests, and can no longer keep up with the load.
When you're done with step 1, you can add MySQL replicas and setup a master-slave replication. This will leave you with a MySQL cluster, where one server can accept writes and all the others are read-only. After your set it up, change your application to send SELECT's to read-only replicas and INSERT/DELETE/UPDATE's to the writeable master server. This approach is based on the fact that most of the applications do reads way more often than writes. It can be not the case for you, but if it is, it'll keep your afloat pretty long. Right before you saturate MySQL master server write performance.
Once you've squeezed everything from step 2, you can go ahead and shard the data. This is now becoming more and more dependent on your application. But I will provide a blind example in order to convey the idea. Say, you have a user-centric application (e.g. a private photo-album, with no sharing capabilities), and each user has a name. In this case you can make two completely independent clusters, where the first one will serve users with names starting A-M, and the second one will serve ones with N-Z. It essentially makes the load twice as less, but complicates the whole architecture.
Though generic, these recipes can help you build a pretty solid application capable of serving millions of users daily before you're forced to bring up more exotic ways of scaling.
Hope this helps!
Some web services in my company are built with different web apps.(Rails, Django, PHP)
What's the better practice to share the session status
So that user won't have to login again and again among different servers.
I build my Rails apps in AWS auto scaling group.
That is, even I browse the same website, but next time I may browser on another server, so that I have to login again. because the server doesn't have my session status.
I need some better idea or keywords for me to search about that kind of issue.
Thanks in advance
I can think of two ways in which you can achieve this objective
Implement a custom session handling mechanism that makes use of database session management, i.e. all sessions will be stored in a special table in the database and will be accessible to all the servers.
Have a Central Authentication Service (CAS) which will act as a proxy to all the other servers. This will then mean that this step has to happen before the requests reach the load balancer.
If you look around, option 1 might be recommended by many, but it may also be an overkill since you'll need custom session management in each of the servers. However, your choice would probably depend on the specific objectives you want to achieve, the overall flexibility of the system architecture and the amount of time you have on your hands. The CAS might be a more straightforward way of solving the problem.
Storing user sessions in your applications database wouldn't be recommended option for AWS.
The biggest problem with using a database, is that you need to write some clean up script that runs every so often to clear the table of all the expired user sessions. This is messy, creates more overhead, and puts more pressure on your DB.
However, if you do want to use an actual database for this, you should use a NoSQL database like Dynamo. This will give you much better performance than a relational database. It's probably more cost effective too in terms of data transfer. However, the biggest problem with this is that you still need that annoying clean up script. Note There is built in support in the SDK for using PHP with Dynamo for storing the user's session:
http://docs.aws.amazon.com/aws-sdk-php/v2/guide/feature-dynamodb-session-handler.html
The best but most costly solution is to use an ElastiCache cluster. You can set a TTL of your choice which means you won't have to worry about clean up scripts. Also ElastiCache will get you much better performance than Dynamo or any relational DB as the data is stored in the RAM of the ElastiCache nodes. The main drawback of ElastiCache is that currently, it can't scale dynamically. So if too many users logged in at once, if you didn't have a big enough cluster already provisioned, things could get ugly.
http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/WhatIs.html
But you can bet that all the biggest, baddest and best applications being hosted on AWS are either using Dynamo, ElastiCache or a custom NoSQL or Cache cluster solution for storing user sessions.
Note that both of these services are supported in all of the AWS SDKs:
https://aws.amazon.com/tools/
I know redis is powerful and I use it for caching in my rails application. Could anyone give me an comparison between rails default caching and redis? What's the trade off as cache.
The main point is distribution.
With Redis the cache can be shared across all back-ends (eventually running on multiple hosts). This is the most scalable solution (because you can multiply the number of back-end hosts). The downside is you will pay for an extra network roundtrip for each cache access. Also, you require an extra component to deploy and manage (Redis).
With ActiveSupport::FileStore, the cache can be shared across back-ends instances provided they run on the same host. Easy to use.
With ActiveSupport::MemStore, the cache cannot be shared across back-ends (even if they run on the same host). However, this is the fastest solution. Easy to use.
I don't think I'm at the point where I need to go through and get memcached setup for my Rails app, but I would like to do some simple caching on a few things.
Is using file_store for as the config.cache_store setting sufficient enough? Or will having to access files for data over and over kill the benefit of caching in the first place from a server load stand point?
Or maybe I'm not really understanding the difference between file_store and mem_cache_store...
I don't think I'm at the point where I need to go through and get memcached setup for my Rails app, but I would like to do some simple caching on a few things
Then use your existing database to store your cached items. (You are using a database, right?)
memcached is only a fast-but-dumb database. If you don't need the ‘fast’ part(*) then don't introduce the extra complexity, inconsistency and overhead of having a separate cache layer.
memcache with file_store is a dumb-but-not-even-fast database, and thus of little use to anyone except for compatibility/testing.
(*: and really, most sites don't. Memcache should be a last resort when you can't optimise your schema, denormalise it for common queries or pre-calculate complex operations any further. It's not something the average web application should be considering as essential for scalability.)
The file_store will cache stuff in files in a filesystem.
If that filesystem is LOCAL to your web server, then clearly, it will only be relevant to that particular web server, therefore you'll lose cache hit rate when a cached entity exists on one server but not another.
How many web servers do you have? 2? 10? 100?
The file_store for caching does not scale properly and will reduce your hit rate over a shared store (for example the memcached one).
The purpose of using memcached is so that a pool of web servers can access a single cache (even though it can be split on multiple physical servers). Using the file_store will not (unless it's a network filesystem, and that will almost certainly be fraught with its own problems).
Always measure the cache hit rate on any cache; if you're not getting a high hit % then it's usually not worth it. Be sure to trend it over time in your monitoring.