I need to validate a whole bunch of YAML files.
I tried the yaml online parser (http://yaml-online-parser.appspot.com/) which works perfect, but it's too much manual work to copy each YAML file content into the box and parse them.
Is there a way to parse/validate YAML files in bulk?
This is fairly straightforward in any scripting language that has a YAML library. For example, here's how you might do it in Ruby:
#!/usr/bin/env ruby
require "yaml"
def check_file(filename)
YAML.parse_file(filename)
puts "OK"
0
rescue Psych::SyntaxError => ex
puts "Error#{ex.message[/: .+/]}"
1
end
exit_code = 0
max_filename_length = ARGV.max_by(&:size).size
ARGV.each do |filename|
printf "%-*s ", max_filename_length, filename
exit_code |= check_file(filename)
end
exit exit_code
Usage:
$ ruby check_yaml.rb *.yml
config-1.yml OK
config-2.yml OK
invalid.yml Error: did not find expected key while parsing a block mapping at line 2 column 3
xyzzy.yml OK
$ echo $EXIT_CODE
1
Just want to share another YAML parser that I found useful too.
https://www.npmjs.com/package/yaml-to-json
Thank you all for helping with this!
Related
I have installed Erlang/OTP and Elixir, and compiled the HelloWorld program into a BEAM using the command:
elixirc test.ex
Which produced a file named Elixir.Hello.beam
How do I run this file?
Short answer: no way to know for sure without also knowing the contents of your source file :)
There are a few ways to run Elixir code. This answer will be an overview of various workflows that can be used with Elixir.
When you are just getting started and want to try things out, launching iex and evaluating expressions one at a time is the way to go.
iex(5)> Enum.reverse [1,2,3,4]
[4, 3, 2, 1]
You can also get help on Elixir modules and functions in iex. Most of the functions have examples in their docs.
iex(6)> h Enum.reverse
def reverse(collection)
Reverses the collection.
[...]
When you want to put some code into a file to reuse it later, the recommended (and de facto standard) way is to create a mix project and start adding modules to it. But perhaps, you would like to know what's going on under the covers before relying on mix to perform common tasks like compiling code, starting applications, and so on. Let me explain that.
The simplest way to put some expressions into a file and run it would be to use the elixir command.
x = :math.sqrt(1234)
IO.puts "Your square root is #{x}"
Put the above fragment of code into a file named simple.exs and run it with elixir simple.exs. The .exs extension is just a convention to indicate that the file is meant to be evaluated (and that is what we did).
This works up until the point you want to start building a project. Then you will need to organize your code into modules. Each module is a collection of functions. It is also the minimal compilation unit: each module is compiled into a .beam file. Usually people have one module per source file, but it is also fine to define more than one. Regardless of the number of modules in a single source file, each module will end up in its own .beam file when compiled.
defmodule M do
def hi(name) do
IO.puts "Hello, #{name}"
end
end
We have defined a module with a single function. Save it to a file named mymod.ex. We can use it in multiple ways:
launch iex and evaluate the code in the spawned shell session:
$ iex mymod.ex
iex> M.hi "Alex"
Hello, Alex
:ok
evaluate it before running some other code. For example, to evaluate a single expression on the command line, use elixir -e <expr>. You can "require" (basically, evaluate and load) one or more files before it:
$ elixir -r mymod.ex -e 'M.hi "Alex"'
Hello, Alex
compile it and let the code loading facility of the VM find it
$ elixirc mymod.ex
$ iex
iex> M.hi "Alex"
Hello, Alex
:ok
In that last example we compiled the module which produced a file named Elixir.M.beam in the current directory. When you then run iex in the same directory, the module will be loaded the first time a function from it is called. You could also use other ways to evaluate code, like elixir -e 'M.hi "..."'. As long as the .beam file can be found by the code loader, the module will be loaded and the appropriate function in it will be executed.
However, this was all about trying to play with some code examples. When you are ready to build a project in Elixir, you will need to use mix. The workflow with mix is more or less as follows:
$ mix new myproj
* creating README.md
* creating .gitignore
* creating mix.exs
[...]
$ cd myproj
# 'mix new' has generated a dummy test for you
# see test/myproj_test.exs
$ mix test
Add new modules in the lib/ directory. It is customary to prefix all module names with your project name. So if you take the M module we defined above and put it into the file lib/m.ex, it'll look like this:
defmodule Myproj.M do
def hi(name) do
IO.puts "Hello, #{name}"
end
end
Now you can start a shell with the Mix project loaded in it.
$ iex -S mix
Running the above will compile all your source file and will put them under the _build directory. Mix will also set up the code path for you so that the code loader can locate .beam files in that directory.
Evaluating expressions in the context of a mix project looks like this:
$ mix run -e 'Myproj.M.hi "..."'
Again, no need to compile anything. Most mix tasks will recompile any changed files, so you can safely assume that any modules you have defined are available when you call functions from them.
Run mix help to see all available tasks and mix help <task> to get a detailed description of a particular task.
To specifically address the question:
$ elixirc test.ex
will produce a file named Elixir.Hello.beam, if the file defines a Hello module.
If you run elixir or iex from the directory containing this file, the module will be available. So:
$ elixir -e Hello.some_function
or
$ iex
iex(1)> Hello.some_function
Assume that I write an Elixir program like this:
defmodule PascalTriangle do
defp next_row(m), do: for(x <- (-1..Map.size(m)-1), do: { (x+1), Map.get(m, x, 0) + Map.get(m, x+1, 0) } ) |> Map.new
def draw(1), do: (IO.puts(1); %{ 0 => 1})
def draw(n) do
(new_map = draw(n - 1) |> next_row ) |> Map.values |> Enum.join(" ") |> IO.puts
new_map
end
end
The module PascalTriangle can be used like this: PascalTriangle.draw(8)
When you use elixirc to compile the ex file, it will create a file called Elixir.PascalTriangle.beam.
From command line, you can execute the beam file like this:
elixir -e "PascalTriangle.draw(8)"
You can see the output similar to the photo:
I'm new to ruby and I'm trying to run 5 different files in a folder. I'd like to execute them all in series mean one after another.
the list of my files are:
Scripts/aaa. rb
Scripts/bbb. rb
Scripts/ccc. rb
Scripts/ddd. rb
Scripts/eee. rb
My script should run the aaa.rb first and then rest
Do as below using Dir::chdir, Dir::glob:
Dir.chdir("path/to/the/.rb files") do |path|
Dir.glob("*.rb").sort.each do |name|
system("ruby #{name}")
end
end
create another ruby script to run them in order
["aaa", "bbb", "ccc", "ddd", "eee"].each do |name|
system("ruby #{name}.rb") #give the full path of the file here
end
Depending on your OS, you can use a batch file, or a shell script. In either case, that's exactly their purpose. Assuming you have the executable bit set on the files:
On *nix or Mac OS:
#!/bin/sh
Scripts/aaa.rb
Scripts/bbb.rb
Scripts/ccc.rb
Scripts/ddd.rb
Scripts/eee.rb
as a simplistic script.
For something a bit more "DRY":
#!/bin/sh
pushdir Scripts
for i in aaa bbb ccc ddd eee
do
$i.rb
done
popdir
If the executable-bit isn't set, precede the name of the file with the path to your Ruby executable, something like:
/usr/local/bin/ruby Scripts/aaa.rb
There are a lot of other ways to point to the Ruby executable, so your mileage might vary.
In Rails 4 rails console I can type history and it will behave just like the history command from the bash shell. E.g.:
[25] my_rails_project ยป history
1: Nomination
2: {:ad => "asdfsdasadf"}
3: Nomination.count
4: Nomination.count.to_sql
5: Nomination.all.class
6: Nomination.all.to_sql
...
Is there a way to search that history, e.g. history | grep Nomination? How about tail?
Note: When I initially wrote this question I thought the history command came from Rails itself, but it comes from the pry gem which I have in my system (my Gemfile specifies the jazz_hands gem which pulls in pry). Pry does in fact have a grep feature, e.g. history --grep Nomination will give me lines 1 and 3-6 above. It also has a tail feature. These are documented here: https://github.com/pry/pry/wiki/History
You can use ~/.irb-history for this purpose. So, the following can be used (you must be knowing how to do this, but this is only for reference):
tail -fn0 ~/.irb-history # for tailing
cat ~/.irb-history | grep something # for searching
Note that, you might have both the files: ~/.irb-history and ~/.irb_history, and any one of them can be more up to date than the other. I have not been able to resolve this mystery yet. So, use the one you find more suitable for yourself.
UPDATE: You can access history in a variable using the following logic (it took me a while to read the code Pry uses, try edit history inside pry):
def pry_history
arr = []
history = Pry::History.new
history.send(:read_from_file) do |line|
arr.push line.chomp
end
arr
end
Now, you can simply call pry_history to get an array of pry's history. You can further save it inside your .pryrc configuration file and use it whenever you want in pry.
Go to rails console
press <ctrl> + r and type few letters of the command.
use <ctrl> + r to choose other commands that match the search
use <ctrl> + <shift> + r to choose the command in the reverse direction
Exactly like the <ctrl> + r in bash shell
Probably this is what you are looking for:
def search_history(filter)
puts `cat ~/.irb-history | grep "#{filter}"`
end
It's worth noting that if you are using pry then the history file will be located at ~/.pry_history. You may also come across this history file .byebug_history.
I've added the following commands to my ~/.pryrc config file.
Pry.config.commands.alias_command 'find.history', 'hist -G'
Pry::Commands.command 'search.history', 'filter commands in the history using fzf and output the selected command' do
puts `cat ~/.pry_history | sort | uniq | fzf --tac --tiebreak=index --height=20`
end
Pry::Commands.command 'search.history.run', 'filter commands in the history using fzf and execute the selected command' do
_pry_.input = StringIO.new(`cat ~/.pry_history | sort | uniq | fzf --tac --tiebreak=index --height=20 | perl -pe 'chomp if eof'`)
end
I have a script written in Ironruby that uses a C# .dll to retrieve a hash. I then use that hash throughout the rest of my Ruby code. I would rather not run my entire script off of the Ironruby interpreter. Is there anyway to run a section of code on the IR interpreter, get the hash, and execute the rest of the code via the regular Ruby interpreter?
Thanks
One possible solution is to split up the script into two parts,
the first part executed by iron ruby has to save his state in a yaml file before handing control to the second part which will run by ruby
here a small demo:
C:\devkit\home\demo>demo
"running program:demo_ir.rb"
"the first part of the script running by the iron_ruby interpreter"
"my_hash attributes:"
"attr1: first value"
"attr2: second value"
"attr3: 2012"
"hash_store_filename:temp.yaml"
"running program:demo_ruby.rb"
"hash_store_filename:temp.yaml"
"the second part of the script running by ruby 1.8.x interpreter"
"my_hash attributes:"
"attr1: first value"
"attr2: second value"
"attr3: 2012"
here the source of the first part for ironruby (demo_ir.rb):
require "yaml"
p "running program:#{$0}"
hash_store_filename = ARGV[0]
my_hash = { attr1: 'first value', attr2: 'second value', attr3: 2012}
p "the first part of the script running by the iron_ruby interpreter"
p "my_hash attributes:"
p "attr1: #{my_hash[:attr1]}"
p "attr2: #{my_hash[:attr2]}"
p "attr3: #{my_hash[:attr3]}"
# save the state of the script in an array where my_hash is the first element
p "hash_store_filename:#{hash_store_filename}"
File.open( hash_store_filename, 'w' ) do |out|
YAML.dump( [my_hash], out )
end
here the code of the second part for ruby 1.8 (demo_ruby.rb)
require "yaml"
p "running program:#{$0}"
hash_store_filename = ARGV[0]
p "hash_store_filename:#{hash_store_filename}"
ar = YAML.load_file(hash_store_filename)
my_hash=ar[0]
p "the second part of the script running by ruby 1.8.x interpreter"
p "my_hash attributes:"
p "attr1: #{my_hash[:attr1]}"
p "attr2: #{my_hash[:attr2]}"
p "attr3: #{my_hash[:attr3]}"
and the launcher:
#ECHO OFF
REM file: demo.bat
SET TEMP_YAML=temp.yaml
ir demo_ir.rb %TEMP_YAML%
ruby demo_ruby.rb %TEMP_YAML%
del %TEMP_YAML%
if you run the script in a concurrent environment you can generate a unique temporary name of the yaml file in the ironruby script avoiding that two process ( or thread ) try to write the same file.
If you prefer you could use some C# line of code, instead of a .bat, to integrate the two parts of the script, but this is a is a bit more difficult (IMHO)
I successfully test this solution using:
C:\devkit\home\demo>ir -v
IronRuby 1.1.3.0 on .NET 4.0.30319.239
C:\devkit\home\demo>ruby -v
ruby 1.8.7 (2011-12-28 patchlevel 357) [i386-mingw32]
ask if you need some clarification
On a bash console, if I do this:
cd mydir
ls -l > mydir.txt
The > operator captures the standard input and redirects it to a file; so I get the listing of files in mydir.txt instead of in the standard output.
Is there any way to do something similar on the rails console?
I've got a ruby statement that generates lots of prints (~8k lines) and I'd like to be able to see it completely, but the console only "remembers" the last 1024 lines or so. So I thought about redirecting to a file - If anyone knows a better option, I'm all ears.
A quick one-off solution:
irb:001> f = File.new('statements.xml', 'w')
irb:002> f << Account.find(1).statements.to_xml
irb:003> f.close
Create a JSON fixture:
irb:004> f = File.new(Rails.root + 'spec/fixtures/qbo/amy_cust.json', 'w')
irb:005> f << JSON.pretty_generate((q.get :customer, 1).as_json)
irb:006> f.close
You can use override $stdout to redirect the console output:
$stdout = File.new('console.out', 'w')
You may also need to call this once:
$stdout.sync = true
To restore:
$stdout = STDOUT
Apart from Veger's answer, there is one of more way to do it which also provides many other additional options.
Just open your rails project directory and enter the command:
rails c | tee output.txt
tee command also has many other options which you can check out by:
man tee
If you write the following code in your environment file, it should work.
if "irb" == $0
config.logger = Logger.new(Rails.root.join('path_to_log_file.txt'))
end
You can also rotate the log file using
config.logger = Logger.new(Rails.root.join('path_to_log_file.txt'), number_of_files, file_roation_size_threshold)
For logging only active record related operations, you can do
ActiveRecord::Base.logger = Logger.new(Rails.root.join('path_to_log_file.txt'))
This also lets you have different logger config/file for different environments.
Using Hirb, you can choose to log only the Hirb output to a text file.
That makes you able to still see the commands you type in into the console window, and just the model output will go to the file.
From the Hirb readme:
Although views by default are printed to STDOUT, they can be easily modified to write anywhere:
# Setup views to write to file 'console.log'.
>> Hirb::View.render_method = lambda {|output| File.open("console.log", 'w') {|f| f.write(output) } }
# Doesn't write to file because Symbol doesn't have a view and thus defaults to irb's echo mode.
>> :blah
=> :blah
# Go back to printing Hirb views to STDOUT.
>> Hirb::View.reset_render_method
Use hirb. It automatically pages any output in irb that is longer than a screenful. Put this in a console session to see this work:
>> require 'rubygems'
>> require 'hirb'
>> Hirb.enable
For more on how this works, read this post.
Try using script utility if you are on Unix-based OS.
script -c "rails runner -e development lib/scripts/my_script.rb" report.txt
That helped me capture a Rails runner script's very-very long output easily to a file.
I tried using redirecting to a file but it got written only at the end of script.
That didn't helped me because I had few interactive commands in my script.
Then I used just script and then ran the rails runner in script session but it didn't wrote everything. Then I found this script -c "runner command here" output_file and it saved all the output as was desired. This was on Ubuntu 14.04 LTS
References:
https://askubuntu.com/questions/290322/how-to-get-and-copy-a-too-long-output-completely-in-terminal#comment1668695_715798
Writing Ruby Console Output to Text File