Storing data inside a ruby gem, where / how to write files? - ruby-on-rails

I've been working on a Ruby parser, that fetches data from different API sources, and compile this data into a clear read-to-use JSON file.
For my use case, i need to store the data i'm initially fetching from the different sources as i don't want to fetch them each time I use the code.
For now i'm writing the JSON i'm receiving from the API sources locally into different JSON files stored in a data folder where my ruby script is. Then i read those files again, parse them and generate my new formatted JSON file that i'm gonna use later in a Rails app.
For that matter i want to create a Gem from this ruby script, which i'm currently working on. Nevertheless i'm not sure to fully understand how and where i should store that data (the one i'm fetching and the one i'm generating).
For now i have tried to simply keep the code as is and simply try to write the file like so:
URI.open("path/to/where/i/wanna/store/file.json", "wb") do |file|
file << URI.open(fetched_data_url).read
end
But wherever i try to write the data i get a :
No such file or directory # rb_sysopen path/to/where/i/wanna/store/file.json
Which in a way does not surprise me that much as i expected it to work in different ways in the context of a Gem. But i'm still missing something here about how to handle this. I'm not sure to fully understand how that all works, especially when you use paths in a gem that will ultimately be used in a rails project.
So several questions here:
Whenever you use a path to write a file inside a Gem, is that path relative to the gem or to the project that will ultimately use that Gem? (and consequently will the file be written inside the project that uses the Gem?)
In that precise use case here, what should i do about it? Where and how do i store my data so that i can use it later? knowing that i need to store it as a JSON file and that for now any attempt of writing a file ends up with an error.
Any input on what i'm misunderstanding here would be much appreciated ! Thanks !

Whenever you use a path to write a file inside a Gem, is that path relative to the gem or to the project that will ultimately use that Gem?
There is nothing special about using file paths whether the code is part of a Gem or not.
path/to/where/i/wanna/store/file.json is a relative path, which means it is looked up relative to the current working directory of the user who started the script. That's nothing special about Gems, that's not even anything to do with Ruby. That is just how file paths work. Relative paths are relative to the current working directory, absolute paths are not.
Where and how do i store my data so that i can use it later?
This depends largely on the Operating System Environment. Different OS Environments have different conventions where to store what kind of files. E.g. your files look like they fit the definition of a cache and Windows has a dedicated folder for caches, as does macOS, as do Linux distributions that follow the Linux Standard Base, as do Desktop Environments that follow the Free Desktop Standards, as does Android, as does iOS, …
For example, the Free Desktop Group has the XDG Base Directory Specification, which defines directories for application state, application data, application cache, and many other things for XDG-compliant environments. Microsoft has similar specifications for Windows. The LSB has something to say as well.

Related

True Paperclip Replacement (Speficially Structure of the File System)

With Rails 6, I need to replace Paperclip, but I can't find any substitutions that actually easily replicate it.
Specifically, the file structure paperclip used:
:model/:attachmant_field/000/000/000/:identifier/:style/:original_file_name
Over the last decade we have built several tools that rely on that structure (or something similar) and in addition our users expect that after uploading an image, they can reference the styles with the same file name and a permanent url (not a randomly generated name like ActiveStorage and Shrine does) and change the "style" component in the url to a different one in their html.
I've spent several days both on Shrine and ActiveStorage working to get the file structure and naming to work on and keep failing, as despite being "natural replacements" they don't actually handle things in the same way.
Our end system is on Amazon S3, though integrating with that hasn't been the issue, just the file system.
Thanks for your help, it's been really frustrating having to remove something that works great when there seems to be nothing that actually replaces it, if you want/need things done in the same way. I'd rather not have to start rewriting all of tools that we developed and resetting our customers expectations to work with a new structure.
Thanks so much.
Have you tried Carrierwave? You can specify any storage path and build it dynamically using model name (model.class.to_s.underscore), attachment field (mounted_as), model id (model.id). The original file name is also available as original_filename.

General advice and recommended folder structure - Sinatra

How would you structure a simple Sinatra app?
I'm making it right now and I want the app to have these features:
The "app" is more of an admin dashboard for all the information inside it. Then another app will access the information via REST. I have not created the dashboard yet, just the getting of things from the database
Sessions and authentication (have not implemented this yet)
You can upload pictures, and other the other app can display those pictures
I have created a testing file using RSpec
Reports generation via Prawn
Currently the setup is just this:
app.rb
test_app.rb
because I have literally just the app and testing file. So far I have used Datamapper for the ORM, SQLite for the database. This is my first Ruby/Sinatra project, so any and all advice is welcome - what other libraries should I be using, should I put stuff like config.ru, etc.
Sinatra is not opinionated when it comes to your file structure, you can place files however you like. When I first started I just dropped everything in the top level, but over time reading how people structure their code, reading over the source code of gems I've broken up my code into smaller .rb files that fulfill a specific function and places all of them under /lib, it's a convention carried over from rails perhaps but does not have any of the magic associated with it in rails. If you use scss or coffee script they depend on certain folders to exist, you will discover for yourself over time (and even then you can reconfigure them however you wish) and from this you will figure out what works best for you.
if you write a restful api, check out grape - https://github.com/intridea/grape
you will also find sinatra-contrib to be very useful - https://github.com/sinatra/sinatra-contrib
As for what to do with your config.ru - https://github.com/rack/rack/wiki/%28tutorial%29-rackup-howto

Rails 3.2 zip multiple text files

I need to write about 10 text files from query results then zip and send them.
Is there a way to do this all in memory or do I need to write the files to /tmp or database first? What is best practise for a Rails 3.2.11 application?
I don't need any functionality beyond creating the files, zipping and sending them in a single action. The files are not large.
You will need to create some temporary files. Where you chose to put them is up to, you, however.
Here's a blog post (not mine, and not tested, but I see no reason the process described shouldn't work) that describes using Rails to zip some files and send the resulting archive to the user. It shouldn't be too hard to adapt it for your needs.

Why is there Rails.rb files all over the place?

Was digging around my Rails applications and noticed that there are rails.rb files all over the place. In my ruby gems directories like:
...gems\devise-2.0.4\lib\devise\rails.rb
...gems\cucumber-rails-1.3.0\lib\cucumber\rails.rb
...gems\railties-3.2.3\lib\rails.rb
I am assuming that there are executed whenever you issue some command like "rails xxx". So all these extra rails.rb files combine with the original rails.rb file to essentially make one big rails.rb file. Essentially, when we type in "rails xxx" it goes thru all them all?
Just looking for some confirmation PLUS a little more knowledge about this. Thanks.
The best way to understand what these rails.rb files are doing, is to read the source code.
ralties
devise
cucumber-rails
As you can see, in any library the file assumes a different scope. The common behaviour is that the file rails.rb normally contains the code required to initialize the library when loaded from a Rails project.
BTW, this has nothing to do with the script/rails command and there is no "big rails.rb" file.
The files are not generated but are simply source files of these libraries you are using.
In this case they are probably rails-related classes that either extend Rails in some way or modify it or make the library interact with Rails.
Rails is a very common framework in Ruby land so most if not all libraries will have some sort of integration with Rails.
By no means are all of those loaded when you run rails XXX but rather when your application loads these libraries their rails.rb files may be executed to provide some sort of integration with Rails.

Rails - Creating temp files in a portable way

My rails application runs on a Ubuntu server machine.
I need to create temporary files in order to "feed" them to a second, independent app (I'll be using rake tasks for this, in case this information is needed)
My question is: what is the best way of creating temporary fields on a rails application?
Since I'm in ubuntu, I could create them on /tmp/whatever, but what would work only in linux.
I'd like my application to be as portable as possible - so it can be installed on Windows machines & mac, if needed.
Any ideas?
Thanks a lot.
tmp/ is definitively the right place to put the files.
The best way I've found of creating files on that folder is using ruby's tempfile library.
The code looks like this:
require 'tempfile'
def foo()
# creates a temporary file in tmp/
Tempfile.open('prefix', Rails.root.join('tmp') ) do |f|
f.print('a temp message')
f.flush
#... do more stuff with f
end
end
I like this solution because:
It generates random file names automatically (you can provide a prefix)
It automatically deletes the files when they are no longer used. For example, if invoked on a rake task, the files are removed when the rake task ends.
Rails apps also have their own tmp/ directory. I guess that one is always available and thus a good candidate to use and keep your application portable.

Resources