I need to do periodical background task fetching data from a weather site.
How in rails can I fetch json from a remote url and store it on the server? Also, there doesn't seem to be any point in storing this in a db so how in rails can I store a variable available to all users?
Why don't you store the data in the cache (local OR memcached).
Write/update the cache when you retrieve the feed data:
def load_feed
Rails.cache.write("feed_data", get_data_from_feed)
end
Read from cache, when you need to access the data:
def read_feed
Rails.cache.fetch("feed_data") { get_data_from_feed }
end
I will only partialy answer your question.
If you want to store it as some variable availble to all users, then probably you can create a my_new_variable.rb in config/initializers. And you can generate some code that defines and initializes your variable (maybe constant is better). Then is a bad part of this approach - you have to restart your server. If you are using passanger, then just touch tmp/restart.txt and it's done.
You can also store it in yml file and load it on server start.
Even if you would store it in a different way, probably the easiest way is to restart server to load this new variable. Otherwise on every request you should check if there is new variable availble (on example check last update time on a file), or reload this file.
So, for me it looks like the easiest way is to store it in db.
Related
I'm writing a rails app which saves items imported from the external API into the DB.
I'm looking for a way permanently to store the number of the imported items and the value can be updated. And the value is single, so I thought it would be right not to use the DB.
The Stored value should only be used locally and should not be shared with other servers using the app. So, the other server will store its own new value at the first.
I did googling and I found some ways for my goal, such as 'ENV' and PStore.
But ENV is not available to update the value and stores temporary value.
PStore should be used in the transaction block. I don't need such that excessive structure.
I just want to store the value permanently and locally, and use anywhere in the ruby code.
Can anyone give me a hint?
Thanks!
You can use redis. Store key/value on each server. Try this: https://www.rubyguides.com/2019/04/ruby-redis/
In my project, a user is uploading a text file, that needs to be read.
File, can be of any size, the file I am using is 1 MB and has ~1500 lines. The file can be bigger as well. Hence instead of putting all in db, i thought of processing the file and retaining the data in instance variable.
But instance variables are not available across HTTP request. Hence what are the options available to me to retain the instance variable values across HTTP request.The other reason for not choosing DB was, I dont need the data to be persisted. As long as user is logged in, data needs to be present for that time duration only. Once user is logged out, I can discard the data.
Please let me know if you need further information.
As #xyious advises, I would say avoid storing that much data in session, it is just not a good practice. You could, however, do the following:
Setup a system-wide configuration setting that holds a path where you store temporary files, in this case, the files uploaded by the user
Generate a random (maybe with SecureRandom.hex) filename when the user uploads the file and store this file in the path mentioned on point #1
Store this random filename in the user session, that way, even if you change between requests you can still access the filename
On each request, whenever you need to process the data, pull the filename from the user's session and join the path of the setting of #1, read the file from the filesystem and do the processing as necessary
Add a callback on your login/sessions controller so that when a user logs out you go and find the filename and delete it before logging out, that way you don't keep unused files around
I would advise against it, but you could store the data in a session variable, or in a cookie.
Why would you need that much data to be stored while the user is logged in ? Is it possible to just save important bits ?
Using instance variables to store content is not a right approach since you don't have a limit on the size of file uploaded and you end up passing the data everytime.
Firstly, decide something on the size limit since you expect text file from users and then upload the file temporarily with a reference path in DB. This file can be cleaned up when required and will make accessing the content simple. To further improve this, enable caching mechanism and setup a caching server for the uploaded files.
If you are not fine with this then other option i can think of is using session variables which is already suggested. So this data will stay per session which fits your requirement. you can just session[:file_Data] = "put parsed content here"
I need to store a global/class variable that is updated by managers from a web dashboard. The variable will be an array, lets call it car_types. About once a week managers need to go in and change the value. So maybe they'll update from ['suv', 'convertible', 'sedan'] to ['suv', 'convertible'].
What I'm not sure on is where to store this variable.
I could certainly create a database table with one record in it that gets updated, but that seems like overkill.
We use memecached, so I could send the variable there, though I'm not sure if thats persistent enough.
I was thinking of having the dashboard update a class variable, but we have dozens of servers running the same app, and I'm unclear if the change would be replicated to all boxes or just stay on one box.
thanks
Global variables are prefixed by $, example: $cars
But What if your application goes down? The global var is reinitialized to its default value.
I would recommend a database, eventually with caching if you want to save on performances.
You could cache your database values in you $cars variable
That's my personal approach: database + cache for records that being updated not often. cache is cleared when a change is made in the table, and cache is created (with a db request) during first fetch of the record.
As a result its all good cause you have the flexibility to change the records sometimes, no problem arise when the server goes down, or with multi-threading, and the cache permit not to kill performances
I'm current writing a personal time management system, and it turns out that I need to store the configuration data such as the time of availability for other people to sign up for the web app. However, since this system is dedicated for one person's schedule, I do not think I need to use sql database as it will wind up with only one entry. So where can I store this configuration? Thank you.
You can do this sort of thing, put it in config/environment.rb
AppName::Application.configure do
config.key = value
config.key2 = value2
end
And in your controller, you can access this by
AppName::Application.config.key
I have a system I've built in MVC 3 that currently provides a yearly submission cycle where the system proceeds through a serious of seven steps tied to dates stored in the web.config as AppSettings. However, each year, I always have to roll the system back and forth between previous steps in order to accommodate the end users. I would like to give the administrator the ability to control the system status without having to contact a developer. What is the best way to do this?
I plan to build a page with proper validation that lets the administrator set the dates. I've considered a couple options of how I should store those date, but none of them seem correct. Our entire permission system uses these dates, and various bits of text on the pages turns on and off based on what period we're currently in.
So far I've come up with two options:
Option 1: Create a database table – This was my first thought. I’ve set up properties on the MvcApplication class in the global.asax and pulled them from the database. Using a lazy loader, I can set the properties the first time they're needed. However, when they change in the database, I don't have a way to force the system to “reset” and read the date changes. If I do this action on Begin_Request(), I'm constantly opening the connection and resetting the properties for each file that the web browser opens on the server, regardless if it's static content or not.
I could directly fetch the dates from the database every time I need one of the dates, but then I'm having to redo a lot of functionality to reduce repeated database calls. I'd like to cache the dates for each request, and only pull them when I need them,
Option 2: Allow editing a config file through the application – I've looked up how to split the web.config file so I can have a separate file that just contains the appSettings. Then I could just update the new config file from a controller action. I think this would work nicely, and not require me to rewrite any of the existing functionality, but it feels like I would be introducing a bad design pattern into the code.
I'd vote for the database. For the sake of performance you can cache those parameter values in a static class inside your app and provide a method to reread them from DB in the same class. So:
When a user makes request, check if those properties are already cached. If they are - use cached values, if no - read them from DB
When administrator makes changes to those parameters - store them to database and enforce your static caching class to reread them from DB.
I would suggest an approach that doesn't care whether the settings are stored in database or key/value pairs in config file.
Since you want the settings to be accessed globally by all users you can cache the settings and the cache implementation should be generic and distributed. There are plenty of online resources available how to create such an interface.
Since you want the cache to be sync with the underlying data you have to set cache dependencies (AppFabric won't supports sql cache dependency see this thread, while NCache supports both sql and file).
I would store the values in a database and use a distributed cache to persist the data across the web farm. MS AppFabric Caching has worked well for me. You will need to implement a standard caching pattern (check the cache, if null load from db and insert into cache).I would probably just create a static Load() method that abstracts this logic away. When the admins update the db you could update the cache or just delete the cachekey.
Therr are other considerations to be added to performance. Namely if you modify the config file thr application pool is re iniyializrd, while the database solution doesnt cause application reinitialization
...so do you need to re initialize the app after the changes or not?...If there i no way to avoid the inizialization whitout drastic changmes to the application ptobably the config filr solution is better