I have been asked in a recent interview why we make the database as a yml file instead of rb. Initially, I was baffled by this question and I try to give an answer that we can serialize and deserialize the yml file but the answer was not satisfactory. So, can someone share his views over it
As I noticed,
YAML is a superset of JSON. YAML is visually easier to look and easy to read.
We can use "anchors" to reference other data in YAML so it can handle relational data.
YAML is more robust about embedding other serialization formats such as JSON.
It will reduce unnecessary creation of objects like done in .rb file.
So storing configuration where only key-value pairs to be stored, are backed up by yaml files.
In short, data and code should be separated for sanity. From a perspective of functionality, data plays a different role with code. That's why we store data in a database, or we serialized them to JSON and YAML.
Config from YAML is deserialization. It's human readable, free from bothering with irrelevant language concerns, and if you want to migrate from an old codebase, it's much easier when you have configurations in text format. YAML is over JSON here because of readability.
Rails is built on the concept of separating layers based on their logic functionality. Like, MVC is designed for the same reason. And you will have a separated auth layer out of the three if necessary.
Related
I created a web-survey application using rails. The administrator specify each survey sections, questions, validations and possible answers/types of answers in yaml config files.
For now, I load the yaml config files for sections and questions at each request (on each page). I know I can use cache to optimize database queries, but would it be faster to use cache also for reading yaml config files? And what kind of cache would you suggest? memcache, binary file, other?
Thanks for any help on this issue!
I think Knuth said it best with...
"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil"
Unless its a problem, move on with your life. However to avoid being ranked down for snarkyness...
In this case, you could read your config files into Class constants to avoid File I/O when you need to retrieve the information if you really think it's going to be a problem.
However based upon your problem what you're probably trying to do could be done better with Backbone Forms but then you'd be adding complexity with JavaScript.
I would recommend not using yaml files actually, unless there is a very good reason for it. You should use your DB instead. It would be much more user friendly and easier to maintain if you used nested models with the correct associations. This railscast actually is almost identical to your situation.
I want to store and load diverse program data in a Delphi project. This data ranges from simple strings to more complex recurring configuration object data.
As we all know ini files provide a fast and easy way to store program data but are limited to key-value representations.
XML is often the weapon of choice when it comes to requirements like this but I want to know if there is an alternative to XML.
Recently I found superobject for Delphi which seems to be a lot easier to handle than XML. Is there anything to be said against using JSON for such "non web task"?
Are you aware of other options that support data storage and load in plain text (like ini, xml, json) in Delphi?
In fact it doesn't matter which storing format you choose (ini, xml, json, whatever). Build an abstract Configuration class that fits all your needs and after that think about the concrete class and the concrete storing format, and decide by how easy to implement and maybe human readability
In some cases you also want to have different configuration aspects (global, machine, user).
With your configuration class you can easily mix them together (use global if not user defined) and can also mix up storing formats (global-config from DB, machine-config from Registry, user-config from file).
Good old INI Files work great for me, in combination with the built in TIniFile and TMemIniFile classes in the IniFiles unit
Benefits of INI files;
Not binary.
Easier to move from machine to machine than Registry settings.
Easy to inspect and view.
Unlike XML, it's simple and human readable
INI files are easy to modify either by hand or by tool and are almost bulletproof, whereas it's easy to make a malformed JSON or XML that is completely unreadable, it's hard to do more than "damage one section" of an INI file. Simplicity wins.
Drawbacks:
Unlike XML and Registry it's more or less "two levels", sections and items.
TMemIniFile doesn't order the results in any controllable way. I often wish I could control the order of items in my ini files if they are generated by a human being, I would like the order to be preserved, and TMemIniFile does not preserve order, thus I find I do not love TMemIniFile as much as love plain old TIniFile.
We've got a rails app that processes large amounts of xml data imports. Right now we're storing these ~5MB xml docs in Postgres. This is not ideal given that we use each xml doc once or twice for parsing. We'd like to have an intelligent way of storing and archiving these docs, but not overly complicate the retrieval process for the sake of space. We've considered moving the docs to Mongo (which we're also using), but then aren't we just artificially boosting the memory requirements of our Mongo db servers?
What's the best way for us to deal with this?
I would just store a link to the file in the DB if you use it only for parsing once or twice and then load the file from the given link. Another aproach is to use a XML DB, e.g. eXist.
You could try eXist, an XML database. If you are just archiving them, though, why not just store them in a directory tree?
You may want to look into DB2's PureXML capabilities. To play with it, you can download the free DB2 Express-C version here. For the record, IBM is also the only database provider officially supporting their Ruby driver and Rails adapter, so you wouldn't be on your own.
What harm are they doing where they are? They will take up 'space' wherever you put them.
If are confident you will never need them again then there is a case for archival to less expensive storage (eg tape?) - otherwise whatever you do will 'overly complicate the retrieval process'
You could consider compressing them in-place if you are not already doing so
I have some experience with Pragmatic-Programmer-type code generation: specifying a data structure in a platform-neutral format and writing templates for a code generator that consume these data structure files and produce code that pulls raw bytes into language-specific data structures, does scaling on the numeric data, prints out the data, etc. The nice pragmatic(TM) ideas are that (a) I can change data structures by modifying my specification file and regenerating the source (which is DRY and all that) and (b) I can add additional functions that can be generated for all of my structures just by modifying my templates.
What I had used was a Perl script called Jeeves which worked, but it's general purpose, and any functions I wanted to write to manipulate my data I was writing from the ground up.
Are there any frameworks that are well-suited for creating parsers for structured binary data? What I've read of Antlr suggests that that's overkill. My current target langauges of interest are C#, C++, and Java, if it matters.
Thanks as always.
Edit: I'll put a bounty on this question. If there are any areas that I should be looking it (keywords to search on) or other ways of attacking this problem that you've developed yourself, I'd love to hear about them.
Also you may look to a relatively new project Kaitai Struct, which provides a language for that purpose and also has a good IDE:
Kaitai.io
You might find ASN.1 interesting, as it provide an absract way to describe the data you might be processing. If you use ASN.1 to describe the data abstractly, you need a way to map that abstract data to concrete binary streams, for which ECN (Encoding Control Notation) is likely the right choice.
The New Jersey Machine Toolkit is actually focused on binary data streams corresponding to instruction sets, but I think that's a superset of just binary streams. It has very nice facilities for defining fields in terms of bit strings, and automatically generating accessors and generators of such. This might be particularly useful
if your binary data structures contain pointers to other parts of the data stream.
I would like to use a delimited text file (xml/csv etc) as a replacement for a database within Ruby on Rails. Solutions?
(This is a class project requirement, I would much rather use a database if I had the choice.)
I'm fine serializing the data and sending it to the text file myself.
The best way is probably to take the ActiveModel API and build your methods that parse your files in the appropriate ways.
Here's a good presentation about ActiveModel and ActiveRelation where he builds a custom model, which should have a lot of similar concepts (but different backend.) And also a good blog post by Yehuda about the ActiveModel API
Have you thought of using SQLite? It is much better solution.
It uses a single file.
It is way faster than doing the serialization yourself.
It is zero configuration. Very simple to use.
You get ACID compliance, transactions sub selects etc etc.
MySQL has a way to store tables in CSV. It has some pretty serious limitations, but it sounds like your requirements demand something with some pretty serious limitations anyway.
I've never set up a Rails project that way, and I don't know what it would take, but it seems like it might be possible.
HSQLDB seems to work by storing data on disk as a SQL script that creates your database. It records changes in memory and a log file, and when you shut down it recreates a single SQL script again. I've not used this one myself.
HSQLDB doesn't appear to be one of the supported databases in Rails. I don't know what it would take to add support for a new database.