I am working on Informatica 9.6 version.
I am trying to load records which are greater than session endtime.
How can I load parameter file with session_end_time?
When I pass timestamp>=$$session_end_time how to load parameter file with session_end_time.
You probably need to refer to the end time of your previous session run. There is no such variable available, so you will need to store a value somewhere (a paramfile or database) at the end of your session run. And read it and use to set a variable in a separate session before the main one.
Related
I want to initialize a variable in rails and use it in a controller's action, but the variable should have a fixed value when the server starts. I have tried defining it inside a controller's action and it get's the same initialized value for every request. For example, i want to initialize a date.now and have the same date after 15 days also.
Update
I am implementing a coming soon page in which a timer is shown 15 days from now. If i implement it in a action inside a controller, it shows new date every time the action is invoked.
Please Help
If you want to create a CONSTANT in rails then you can simply put it into the initializer file. For eg, create a file name constants.rb inside initializer:
#config/initializers/constants.rb
TEST_VALUE = 10
And to access this CONSTANT from your controller, just call for TEST_VALUE. for eg,
#controllers_path/some_controller.rb
.....
def some_def
#value = TEST_VALUE # this will be enough to fetch the constants.
end
But, make sure you restart your server after changing the intializer.
You're looking to create a constant, which is basically a variable which doesn't change its value
You'd typically set these with initializers:
#config/initializers/your_initializer.rb
YOUR_CONSTANT = your_date
To maintain a persistent date, you'll have to give some more context on what you're using it for. It will be difficult to create this each time Rails loads (how to know which Time.now to use), so giving us more info will be a good help
You can also use an opposite approach. Assuming you should know the date when the new feature comes (for example 2014-04-04 18:00) you can just find a number of seconds left till the target date:
#seconds_left = Time.parse('2014-04-04 18:00').to_i - Time.now.to_i
then pass it to client side and implement a timer. So you'll just need to store a string representation of a target date.
Obvious disadvantage of this approach is that you should adjust that target time each time you want to introduce a new feature.
I am looking to find information on how the caching mechanism in Rails 4 prevents against multiple users trying to regenerate cache keys at once, aka a cache stampede: http://en.wikipedia.org/wiki/Cache_stampede
I've not been able to find out much information via Googling. If I look at other systems (such as Drupal) cache stampede prevention is implemented via a semaphores table in the database.
Rails does not have a built-in mechanism to prevent cache stampedes.
According to the README for atomic_mem_cache_store (a replacement for ActiveSupport::Cache::MemCacheStore that mitigates cache stampedes):
Rails (and any framework relying on active support cache store) does
not offer any built-in solution to this problem
Unfortunately, I'm guessing that this gem won't solve your problem either. It supports fragment caching, but it only works with time-based expiration.
Read more about it here:
https://github.com/nel/atomic_mem_cache_store
Update and possible solution:
I thought about this a bit more and came up with what seems to me to be a plausible solution. I haven't verified that this works, and there are probably better ways to do it, but I was trying to think of the smallest change that would mitigate the majority of the problem.
I assume you're doing something like cache model do in your templates as described by DHH (http://37signals.com/svn/posts/3113-how-key-based-cache-expiration-works). The problem is that when the model's updated_at column changes, the cache_key likewise changes, and all your servers try to re-create the template at the same time. In order to prevent the servers from stampeding, you would need to retain the old cache_key for a brief time.
You might be able to do this by (dum da dum) caching the cache_key of the object with a short expiration (say, 1 second) and a race_condition_ttl.
You could create a module like this and include it in your models:
module StampedeAvoider
def cache_key
orig_cache_key = super
Rails.cache.fetch("/cache-keys/#{self.class.table_name}/#{self.id}", expires_in: 1, race_condition_ttl: 2) { orig_cache_key }
end
end
Let's review what would happen. There are a bunch of servers calling cache model. If your model includes StampedeAvoider, then its cache_key will now be fetching /cache-keys/models/1, and returning something like /models/1-111 (where 111 is the timestamp), which cache will use to fetch the compiled template fragment.
When you update the model, model.cache_key will begin returning /models/1-222 (assuming 222 is the new timestamp), but for the first second after that, cache will keep seeing /models/1-111, since that is what is returned by cache_key. Once 1 second passes, all of the servers will get a cache-miss on /cache-keys/models/1 and will try to regenerate it. If they all recreated it immediately, it would defeat the point of overriding cache_key. But because we set race_condition_ttl to 2, all of the servers except for the first will be delayed for 2 seconds, during which time they will continue to fetch the old cached template based on the old cache key. Once the 2 seconds have passed, fetch will begin returning the new cache key (which will have been updated by the first thread which tried to read/update /cache-keys/models/1) and they will get a cache hit, returning the template compiled by that first thread.
Ta-da! Stampede averted.
Note that if you did this, you would be doing twice as many cache reads, but depending on how common stampedes are, it could be worth it.
I haven't tested this. If you try it, please let me know how it goes :)
The :race_condition_ttl setting in ActiveSupport::Cache::Store#fetch should help avoid this problem. As the documentation says:
Setting :race_condition_ttl is very useful in situations where a cache entry is used very frequently and is under heavy load. If a cache expires and due to heavy load seven different processes will try to read data natively and then they all will try to write to cache. To avoid that case the first process to find an expired cache entry will bump the cache expiration time by the value set in :race_condition_ttl. Yes, this process is extending the time for a stale value by another few seconds. Because of extended life of the previous cache, other processes will continue to use slightly stale data for a just a bit longer. In the meantime that first process will go ahead and will write into cache the new value. After that all the processes will start getting new value. The key is to keep :race_condition_ttl small.
Great question. A partial answer that applies to single multi-threaded Rails servers but not multiprocess(or) environments (thanks to Nick Urban for drawing this distinction) is that the ActionView template compilation code blocks on a mutex that is per template. See line 230 in template.rb here. Notice there is a check for completed compilation both before grabbing the lock and after.
The effect is to serialize attempts to compile the same template, where only the first will actually do the compilation and the rest will get the already completed result.
Very interesting question. I searched on google (you get more results if you search for "dog pile" instead of "stampede") but like you, did I not get any answers, except this one blog post: protecting from dogpile using memcache.
Basically does it store you fragment in two keys: key:timestamp (where timestamp would be updated_at for active record objects) and key:last.
def custom_write_dogpile(key, timestamp, fragment, options)
Rails.cache.write(key + ':' + timestamp.to_s, fragment)
Rails.cache.write(key + ':last', fragment)
Rails.cache.delete(key + ':refresh-thread')
fragment
end
Now when reading from the cache, and trying to fetch a non existing cache, will it instead try to fecth the key:last fragment instead:
def custom_read_dogpile(key, timestamp, options)
result = Rails.cache.read(timestamp_key(name, timestamp))
if result.blank?
Rails.cache.write(name + ':refresh-thread', 0, raw: true, unless_exist: true, expires_in: 5.seconds)
if Rails.cache.increment(name + ':refresh-thread') == 1
# The cache didn't exists
result = nil
else
# Fetch the last cache, as the new one has not been created yet
result = Rails.cache.read(name + ':last')
end
end
result
end
This is a simplified summary of the by Moshe Bergman that i linked to before, or you can find here.
There is no protection against memcache stampedes. This is a real problem when multiple machines are involved and multiple processes on those multiple machines. -Ouch-.
The problem is compounded when one of the key processes has "died" leaving any "locking" ... locked.
In order to prevent stampedes you have to re-compute the data before it expires. So, if your data is valid for 10 minutes, you need to regenerate again at the 5th minute and re-set the data with a new expiration for 10 more minutes. Thus you don't wait until the data expires to set it again.
Should also not allow your data to expire at the 10 minute mark, but re-compute it every 5 minutes, and it should never expire. :)
You can use wget & cron to periodically call the code.
I recommend using redis, which will allow you to save the data and reload it in the advent of a crash.
-daniel
A reasonable strategy would be to:
use a :race_condition_ttl with at least the expected time it takes to refresh the resource. Setting it to less time than expected to perform a refresh is not advisable as the angry mob will end up trying to refresh it, resulting in a stampede.
use an :expires_in time calculated as the maximum acceptable expiry time minus the :race_condition_ttl to allow for refreshing the resource by a single worker and avoiding a stampede.
Using the above strategy will ensure that you don't exceed your expiry/staleness deadline and also avoid a stampede. It works because only one worker gets through to refresh, whilst the angry mob are held off using the cache value with the race_condition_ttl extension time right up to the originally intended expiry time.
I'm working on a project where i want to do a mysql query from time to time. The query is too long, and actually it's done when the user does a request.
I'm afraid if many users does the request, the application will be too slow to respond. So, I want to do the query and load it with the query response from time to time, and then, on a request, the action from the controller will use this variable, instead of doing the query again and again.
How can I do that using Whenever?
on the schedule.rb
every 5.minutes do
runner "variable = Model.method"
end
and on the controller
def some_action
"the variable should be loaded here"
end
I agree with Damien Roche, you need to cache the results of the query. But, I don't think the example he gives is the best answer for you because you don't want for a user to wait for the query when it isn't cached, at the times when the cache is expired, even if this is a rare occurrence.
So you need to combine the periodic query with whenever, like you suggested, with a caching mechanism to store your query result, and retrieve it from the cache in your controller. since the runner is a different process, you will have to use a cache that is available to both the runner and your app. I recommend you look into Redis. it should be very simple to get it to work so that the runner runs the query and when it finishes writes a result set to the Redis cahce. The controller will then read the result set from the cache.
I am working with some session data in a Rails site that was created before my time. One of the session variables is not working and I'd like to see what variables exist in my session. Is there a way to get key/value pairs for my session variables and then print them out in the log?
Assuming the default log level of info:
Rails.logger.info(session.inspect)
If that doesn't work try:
Rails.logger.error(session.inspect)
We recently lost a database and I want to recover the data from de Production.log.
Every request is logged like this:
Processing ChamadosController#create (for XXX.XXX.XXX.40 at 2008-07-30 11:07:30) [POST]
Session ID: 74c865cefa0fdd96b4e4422497b828f9
Parameters: {"commit"=>"Gravar", "action"=>"create", "funcionario"=>"6" ... (all other parameters go here).
But some stuff to post on de database were in the session. In the request I have the Session ID, and I also have all the session files from the server.
Is there anyway I can, from this Session ID, open de session file and get it's contents?
It's probably best to load the session file into a hash -- using the session-id as the key -- and then go through all the log files in chronological order, and parse out the relevant info for each session, and modify your database with it.
I guess you're starting out with an old database backup? Make sure to do this in a separate Rails environment -- e.g. don't do this in production; create and use a separate "recovery" environment / DB.
think about some sanity checks you can run on the database afterwards, to make sure that the state of the records makes sense
Going forward:
make sure that you do regular backups going forward (e.g. with mysqldump if you use MySQL).
make sure to set up your database for master/slave replication
hope this helps -- good luck!
Have you tried using Marshal#load? I'm not sure how you're generating those session files, but it's quite possible Rails just uses Marshal.
A client exactly had the same problem a few weeks ago. I came up with the following solution:
play back the latest backup you have (in our case it was one year
old)
write a small parser that moves all the requests from production in a temporary database (i chose mongodb for that): i used a rake task and "eval" to create the hash.
play back the data in the following order
play in the first create of an object, if it does not already exist.
find the last update (by date) and play it back.
here is the regex for scanning the production.log:
file = File.open("location_of_your_production.log", "rb")
contents = file.read
contents.scan(/(Started POST \"(.*?)\" for (.*?) at (.*?)\n.*?Parameters: \{(.*?)\}\n.*?Completed (.*?) in (.*?)ms)/m).each do |x|
# now you can collect all the important data.
# do the same for GET requests as well, if you need it.
end
In my case, the temporary database speeded up the process of the logfile parsing, so the above noted steps could be taken. Of course, everything that was not sent over production.log will be lost. Also, updates of the objects would send the whole information, it might be different in your case. I could also recreate the image uploads, since the images were sent base64 encoded in the production.log.
good luck!