How to specify markers in YAML file - ruby-on-rails

Question

I have a YAML file that is going to be parsed by two different machines, so I want to specify some sort of markers in the file to indicate which machine has the right to read a specific block.
As an example I want the block 1 to be parsed by machine 1 and block 2 to be parsed by machine2:
# BLOCK 1 - Machine 1
-
:id: 1234
:worker: Foo1
:opts:
:ftpaccount: user1
:limit: 10
# BLOCK 2 - Machine 2
-
:id: 5678
:worker: Foo2
:opts:
:ftpaccount: user2
:limit: 10
How can I achieve something like this? How you implement something similar to this? Thanks.

Answer 1

Treat the blocks as hash entries with the key being the hostname:
require 'yaml'
yaml = <<EOT
host1:
# BLOCK 1 - Machine 1
-
:id: 1234
:worker: Foo1
:opts:
:ftpaccount: user1
:limit: 10
host2:
# BLOCK 2 - Machine 2
-
:id: 5678
:worker: Foo2
:opts:
:ftpaccount: user2
:limit: 10
EOT
config = YAML.load(yaml)
# => {"host1"=>
# [{:id=>1234,
# :worker=>"Foo1",
# :opts=>{:ftpaccount=>"user1", :limit=>10}}],
# "host2"=>
# [{:id=>5678,
# :worker=>"Foo2",
# :opts=>{:ftpaccount=>"user2", :limit=>10}}]}
At this point you can grab the chunk you need:
config['host1']
# => [{:id=>1234, :worker=>"Foo1", :opts=>{:ftpaccount=>"user1", :limit=>10}}]
config['host2']
# => [{:id=>5678, :worker=>"Foo2", :opts=>{:ftpaccount=>"user2", :limit=>10}}]
You don't even have to hard-code the hostname; You can ask the machine what its name is:
`hostname`.chomp # => "MyHost"
Actually, I'd change the YAML a little, so it's a hash of hashes. As is, your YAML returns a hash of arrays of hashes, which, because of the array, makes it more awkward to use:
host1:
# BLOCK 1 - Machine 1
:id: 1234
:worker: Foo1
:opts:
:ftpaccount: user1
:limit: 10
host2:
# BLOCK 2 - Machine 2
:id: 5678
:worker: Foo2
:opts:
:ftpaccount: user2
:limit: 10
Results in:
config = YAML.load(yaml)
# => {"host1"=>
# {:id=>1234, :worker=>"Foo1", :opts=>{:ftpaccount=>"user1", :limit=>10}},
# "host2"=>
# {:id=>5678, :worker=>"Foo2", :opts=>{:ftpaccount=>"user2", :limit=>10}}}
config['host1']
# => {:id=>1234, :worker=>"Foo1", :opts=>{:ftpaccount=>"user1", :limit=>10}}
config['host2']
# => {:id=>5678, :worker=>"Foo2", :opts=>{:ftpaccount=>"user2", :limit=>10}}
Finally, if your YAML file is complex, or long, or has repeated sections, seriously consider writing code that emits that file for you. Ruby makes it really easy to generate the YAML in a very smart way that automatically uses aliases. For instance:
require 'yaml'
SOME_COMMON_DATA = {
'shared_db_dsn' => 'mysql://user:password#host/db'
}
HOST1 = 'foo.com'
HOST1_DATA = {
HOST1 => {
'id' => 1234,
'worker' => 'Foo1',
'opts' => {
'ftpaccount' => 'user1',
'limit' => 10
},
'dsn' => SOME_COMMON_DATA
}
}
HOST2 = 'bar.com'
HOST2_DATA = {
HOST2 => {
'id' => 5678,
'worker' => 'Foo2',
'opts' => {
'ftpaccount' => 'user2',
'limit' => 10
},
'dsn' => SOME_COMMON_DATA
}
}
data = {
HOST1 => HOST1_DATA,
HOST2 => HOST2_DATA,
}
puts data.to_yaml
# >> ---
# >> foo.com:
# >> foo.com:
# >> id: 1234
# >> worker: Foo1
# >> opts:
# >> ftpaccount: user1
# >> limit: 10
# >> dsn: &1
# >> shared_db_dsn: mysql://user:password#host/db
# >> bar.com:
# >> bar.com:
# >> id: 5678
# >> worker: Foo2
# >> opts:
# >> ftpaccount: user2
# >> limit: 10
# >> dsn: *1
Notice how YAML converted "dsn" into an alias and referenced it in the second host's definition using an anchor. This can add up to serious space savings, depending on how you define your variables and build the data structure. See "Aliases and Anchors" for more information.
Also, I'd highly recommend avoiding the use of symbols for your hash keys. By doing so your YAML can be easily loaded by other languages, not just Ruby. At that point, your YAML becomes even more useful when building big systems.

Answer 2

Here's a simple state machine that assembles a string based on the most recent matching comment in the yaml file. The YAML string is then loaded into the parser. If your files are really large, you could easily modify this to use Tempfile or some other IO class.
require 'yaml'
class YAMLSplitter
attr_reader :flag, :mode, :raw
def initialize(flag)
#flag = flag
#mode = :match
#raw = ""
end
def parse(file)
File.read(file).each_line do |line|
process_line(line)
end
YAML.load(raw)
end
private
def process_line(line)
set_match_status(line)
write_line(line) if match?
end
def set_match_status(line)
if line.start_with?("#")
if line.match(flag)
match!
else
nomatch!
end
end
end
def write_line(line)
puts "WRITE_LINE #{mode.inspect} #{line.inspect}"
raw << line
end
def match?
mode == :match
end
def match!
#mode = :match
end
def nomatch!
#mode = :nomatch
end
end
YAML:
---
# machine 1
- 1
- 2
- 3
- 4
# machine 2
- 5
- 6
- 7
- 8
- 9
- 10
- 11
# machine 1
- 12
Execution:
splitter = YAMLSplitter.new('machine 1')
yaml = splitter.parse('test.yml')