1) I am downloading datafeeds (xml) files from a URL to unzip/import them into the database. Where should I store them in the rails file structure?
2) How does the rails file structure work, can rails access the entire hosting environment? I basically mean, if I store my XML feed in /lib/files would I use that path in my models, or the longer full linux path?
Appreciate any advice!
You should probably use the tmp/ folder to store those temporary files
Its a good practice to always use the full path. You can get the rails root dir via Rails.root
Rails can access any thing that the user account under which the rails process is running, can access. ie: if you run the rails server process under root (which is not a good idea BTW), the app could access any path that root can access. This might of course be limited by whatever access control mechanisms in place by the OS(ex: SELinux).
Related
I've been working on a Ruby parser, that fetches data from different API sources, and compile this data into a clear read-to-use JSON file.
For my use case, i need to store the data i'm initially fetching from the different sources as i don't want to fetch them each time I use the code.
For now i'm writing the JSON i'm receiving from the API sources locally into different JSON files stored in a data folder where my ruby script is. Then i read those files again, parse them and generate my new formatted JSON file that i'm gonna use later in a Rails app.
For that matter i want to create a Gem from this ruby script, which i'm currently working on. Nevertheless i'm not sure to fully understand how and where i should store that data (the one i'm fetching and the one i'm generating).
For now i have tried to simply keep the code as is and simply try to write the file like so:
URI.open("path/to/where/i/wanna/store/file.json", "wb") do |file|
file << URI.open(fetched_data_url).read
end
But wherever i try to write the data i get a :
No such file or directory # rb_sysopen path/to/where/i/wanna/store/file.json
Which in a way does not surprise me that much as i expected it to work in different ways in the context of a Gem. But i'm still missing something here about how to handle this. I'm not sure to fully understand how that all works, especially when you use paths in a gem that will ultimately be used in a rails project.
So several questions here:
Whenever you use a path to write a file inside a Gem, is that path relative to the gem or to the project that will ultimately use that Gem? (and consequently will the file be written inside the project that uses the Gem?)
In that precise use case here, what should i do about it? Where and how do i store my data so that i can use it later? knowing that i need to store it as a JSON file and that for now any attempt of writing a file ends up with an error.
Any input on what i'm misunderstanding here would be much appreciated ! Thanks !
Whenever you use a path to write a file inside a Gem, is that path relative to the gem or to the project that will ultimately use that Gem?
There is nothing special about using file paths whether the code is part of a Gem or not.
path/to/where/i/wanna/store/file.json is a relative path, which means it is looked up relative to the current working directory of the user who started the script. That's nothing special about Gems, that's not even anything to do with Ruby. That is just how file paths work. Relative paths are relative to the current working directory, absolute paths are not.
Where and how do i store my data so that i can use it later?
This depends largely on the Operating System Environment. Different OS Environments have different conventions where to store what kind of files. E.g. your files look like they fit the definition of a cache and Windows has a dedicated folder for caches, as does macOS, as do Linux distributions that follow the Linux Standard Base, as do Desktop Environments that follow the Free Desktop Standards, as does Android, as does iOS, …
For example, the Free Desktop Group has the XDG Base Directory Specification, which defines directories for application state, application data, application cache, and many other things for XDG-compliant environments. Microsoft has similar specifications for Windows. The LSB has something to say as well.
Am using EngineYard to host my Rails 3.2 app. This application allows users to post images/assets. I save them in the public directory (using Paperclip Gem). Now, my problem is that - with a new deployment, I am having to manually copy over the assets to the CURRENT version.
Though, I could use AmazonS3, I still want to figure out if there is a way in EngineYard which lets me save/serve the assets from a different directory than the code, say /data/assets.
Please, let me know if you see any other alternate implementations too.
Typically your structure would look like
/data
myapp/
shared/
images
releases/
20120613000000
20120601000000
...
current (symlink to one of the releases)
When you deploy, you symlink public/images to shared/images and so your images always get stored in a non release dependant location.
I would encourage you to use something like s3: you'll make things a lot easier for when you want to host the app on multiple instances.
For every project it's like having two parts: the Rails application and then all documents and pictures related to it.
I wonder how you organize them both.
Do you put everything under the same project root folder and apply Git on that folder or don't you protect your documents with Git at all?
Then, what if the docs are too sensitive or only for owners of that project. Then I probably should't have it under the same folder right?
How have you structured both your Rails code and belonging business documents?
Share your solutions!
If you're deploying with capistrano, as a lot of Rails apps are, the standard solution seems to be to keep these sorts of assets within the shared folder, and then get cap to symlink them into the application at the point of deploy.
My rails application runs on a Ubuntu server machine.
I need to create temporary files in order to "feed" them to a second, independent app (I'll be using rake tasks for this, in case this information is needed)
My question is: what is the best way of creating temporary fields on a rails application?
Since I'm in ubuntu, I could create them on /tmp/whatever, but what would work only in linux.
I'd like my application to be as portable as possible - so it can be installed on Windows machines & mac, if needed.
Any ideas?
Thanks a lot.
tmp/ is definitively the right place to put the files.
The best way I've found of creating files on that folder is using ruby's tempfile library.
The code looks like this:
require 'tempfile'
def foo()
# creates a temporary file in tmp/
Tempfile.open('prefix', Rails.root.join('tmp') ) do |f|
f.print('a temp message')
f.flush
#... do more stuff with f
end
end
I like this solution because:
It generates random file names automatically (you can provide a prefix)
It automatically deletes the files when they are no longer used. For example, if invoked on a rake task, the files are removed when the rake task ends.
Rails apps also have their own tmp/ directory. I guess that one is always available and thus a good candidate to use and keep your application portable.
I am generating some large files in my Rails application. Fortunately, they only need to be generated once. I would like to save these files to disk so that I don't have to generate them again.
Which directory in my Rails application is the most appropriate place to put application generated files?
Thanks!
If security of the files is not an issue you can put them in a subdirectory of public (for example, public/assets) which in your deploy script is symlinked to a directory in shared/public so that when you redeploy the files are retained.
If security is an issue, the solution is similar, though you would not want the directory to be web accessible. Instead you would use a controller to control access and serve up the files with send_file.