Is there a way to achieve grep like functionality on output inside the Rails Console - ruby-on-rails

In shell, I can do
$ cat name_of_file_with_a_lot_of_text | grep "What I am looking for"
Inside the Rails Console, can I achieve something similar, say when I run a command and the output is huge, especially say a DB query.
I am aware of outputting it as YAML but that Is not what I am looking for.
Thanks.

Yes, you can. The method is called gr... wait for it ...ep. Ruby's grep works on String, Array and many other built-in objects. For example to get all to_xxx methods of a number, just do:
1.methods.grep(/to_/)

I had the same question and wasn't very satisfied with the somewhat snarky response from ream88, so I decided to take a crack at it.
# Allows you to filter output to the console using grep
# Ex:
# def foo
# puts "Some debugging output here"
# puts "The value of x is y"
# puts "The value of foo is bar"
# end
#
# grep_stdout(/value/) { foo }
# # => The value of x is y
# # => The value of foo is bar
# # => nil
def grep_stdout(expression)
# First we need to create a ruby "pipe" which is two sets of IO subclasses
# the first is read only (which represents a fake $stdin) and the second is
# write only (which represents a fake $stdout).
placeholder_in, placeholder_out = IO.pipe
# This child process handles the grep'ing. Its done in a child process so that
# it can operate in parallel with the main process.
pid = fork {
# sync $stdout so we can report any matches asap
$stdout.sync
# replace $stdout with placeholder_out
$stdin.reopen(placeholder_in)
# we have to close both placeholder_out and placeholder_in because all instances
# of an IO stream must be closed in order for it to ever reach EOF. There's two
# in this method; one in the child process and one in the main process.
placeholder_in.close
placeholder_out.close
# loop continuously until we reach EOF (which happens when all
# instances of placeholder_out have closed)
read_buffer = ''
loop do
begin
read_buffer << $stdin.readpartial(4096)
if line_match = read_buffer.match(/(.*\n)(.*)/)
print line_match[1].grep(expression) # grep complete lines
read_buffer = line_match[2] # save remaining partial line for the next iteration
end
rescue EOFError
print read_buffer.grep(expression) # grep any remaining partial line at EOF
break
end
end
}
# Save the original stdout out to a variable so we can use it again after this
# method is done
original_stdout = $stdout
# Redirect stdout to our pipe
$stdout = placeholder_out
# sync $stdout so that we can start operating on it as soon as possible
$stdout.sync
# allow the block to execute and save its return value
return_value = yield
# Set stdout back to the original so output will flow again
$stdout = original_stdout
# close the main instances of placeholder_in and placeholder_out
placeholder_in.close
placeholder_out.close
# Wait for the child processes to finish
Process.wait pid
# Because the connection to the database has a tendency to go away when calling this, reconnect here
# if we're using ActiveRecord
if defined?(ActiveRecord)
suppress_stdout { ActiveRecord::Base.verify_active_connections! }
end
# return the value of the block
return_value
end
The obvious drawback of my solution is that the output is lost. I'm not sure how to get around that without calling yield twice.
EDIT I've changed my answer to only call fork once, which allows me to keep the output of the block and return it at the end. Win.
EDIT 2 You can get all of this functionality (and more!) in this gem now https://github.com/FutureAdvisor/console_util

Related

Calling a command inside a ruby script

Is there a way to call a command like last inside a ruby script? I can use %x to call commands like ls and ls -l within a script, but would that be acceptable for the complex and constantly expanding log information provided by the last command?
Here's an example, to retrieve user name, ip, startup and duration :
%x(last -i).each_line do |line|
line.chomp! # Removes newline
break if line.empty? # last is done
# 3 possibilities to extract information :
user, *other_columns = line.split(' ') # 1. Use split
start = Time.parse(line[39,16]) # 2. Use known position of a column
ip = line[22,17].strip
if line =~/\((\d+):(\d+)\)/ # 3. Use a regex
duration = $1.to_i*60+$2.to_i
else
duration = nil
end
info={user: user, ip: ip, start: start, duration: duration}
#TODO: Check that user isn't "reboot"
puts info
end
# {:user=>"ricou", :ip=>"0.0.0.0", :start=>2016-11-01 21:29:00 +0100, :duration=>141}
# {:user=>"ricou", :ip=>"0.0.0.0", :start=>2016-11-01 15:21:00 +0100, :duration=>57}
Which information do you need exactly?

Running code asynchronously inside pollers

In my ruby script,I am using celluloid-zmq gem. where I am trying to run evaluate_response asynchronously inside pollers using,
async.evaluate_response(socket.read_multipart)
But if I remove sleep from loop, somehow thats not working out, It is not reaching to "evaluate_response" method. But if I put sleep inside loop it works perfectly.
require 'celluloid/zmq'
Celluloid::ZMQ.init
module Celluloid
module ZMQ
class Socket
def socket
#socket
end
end
end
end
class Indefinite
include Celluloid::ZMQ
## Readers
attr_reader :dealersock,:pullsock,:pollers
def initialize
prepare_dealersock and prepare_pullsock and prepare_pollers
end
## prepare DEALER SOCK
def prepare_dealersock
#dealersock = DealerSocket.new
#dealersock.identity = "IDENTITY"
#dealersock.connect("tcp://localhost:20482")
end
## prepare PULL SOCK
def prepare_pullsock
#pullsock = PullSocket.new
#pullsock.connect("tcp://localhost:20483")
end
## prepare the Pollers
def prepare_pollers
#pollers = ZMQ::Poller.new
#pollers.register_readable(dealersock.socket)
#pollers.register_readable(pullsock.socket)
end
def run!
loop do
pollers.poll ## this is blocking operation never mind though we need it
pollers.readables.each do |socket|
## we know socket.read_multipart is blocking call this would give celluloid the chance to run other process in mean time.
async.evaluate_response(socket.read_multipart)
end
## If you remove the sleep the async evaluate response would never be executed.
## sleep 0.2
end
end
def evaluate_response(message)
## Hmmm, the code just not reaches over here
puts "got message: #{message}"
...
...
...
...
end
end
## Code is invoked like this
Indefinite.new.run!
Any idea why this is happening?
The question was 100% changed, so my previous answer does not help.
Now, the issues are...
ZMQ::Poller is not part of Celluloid::ZMQ
You are directly using the ffi-rzmq bindings, and not using the Celluloid::ZMQ wrapping, which provides evented & threaded handling of the socket(s).
It would be best to make multiple actors -- one per socket -- or to just use Celluloid::ZMQ directly in one actor, rather than undermining it.
Your actor never gets time to work with the response
This part makes it a duplicate of:
Celluloid async inside ruby blocks does not work
The best answer is to use after or every and not loop ... which is dominating your actor.
You need to either:
Move evaluate_response to another actor.
Move each socket to their own actor.
This code needs to be broken up into several actors to work properly, with a main sleep at the end of the program. But before all that, try using after or every instead of loop.

Can I display the log of system call in Ruby?

I need to call a command(in a sinatra or rails app) like this:
`command sub`
Some log will be outputed when the command is executing.
I want to see the log displaying continuously in the process.
But I just can get the log string after it's done with:
result = `command sub`
So, is there a way to implement this?
On windows i have the best experience with IO.popen
Here is a sample
require 'logger'
$log = Logger.new( "#{__FILE__}.log", 'monthly' )
#here comes the full command line, here it is a java program
command = %Q{java -jar getscreen.jar #{$userid} #{$password}}
$log.debug command
STDOUT.sync = true
begin
# Note the somewhat strange 2> syntax. This denotes the file descriptor to pipe to a file. By convention, 0 is stdin, 1 is stdout, 2 is stderr.
IO.popen(command+" 2>&1") do |pipe|
pipe.sync = true
while str = pipe.gets #for every line the external program returns
#do somerthing with the capturted line
end
end
rescue => e
$log.error "#{__LINE__}:#{e}"
$log.error e.backtrace
end
There's six ways to do it, but the way you're using isn't the correct one because it waits for the process the return.
Pick one from here:
http://tech.natemurray.com/2007/03/ruby-shell-commands.html
I would use IO#popen3 if I was you.

read and execute ruby script

script.rb:
puts 'hello'
puts 'foo'
main.rb:
puts `jruby script.rb` # receive the expected result
The question:
How can the same be achieved with reading the "script" before execution?
main.rb:
code=File.open('script.rb', 'r').read.gsub('"', '\"')
# puts `jruby -e '#{code}'` # Does not work for relatively big files;
Windows and unicode are the reasons of this question;
Please note that `jruby script.rb' creates a new process which is essential.
Store the modified script in a Tempfile and run that instead of passing the whole contents as an eval argument:
require 'tempfile'
code = IO.read('script.rb').gsub('"', '\"')
begin
tempfile = Tempfile.new 'mytempfile'
f.write code
f.close
puts `jruby '#{f.path}'`
ensure
f.close
f.unlink
end
The reason you’re likely getting an error is either a lack of proper escaping or a limit on the maximum argument length in the shell.
Also, beware that in your original implementation you never close the original file. I’ve fixed that by instead using IO.read.
In the command line, using
$ getconf ARG_MAX
will give the upper limit on how many bytes can be used for the command line argument and environment variables.
#Andrew Marshall's answer is better, but suppose you don't want to use a temp file, and assuming we can use fork in JRuby,
require 'ffi'
module Exec
extend FFI::Library
ffi_lib FFI::Platform::LIBC
attach_function :fork, [], :int
end
code = IO.read('script.rb')
pid = Exec.fork
if 0 == pid
eval code
exit 0
else
Process.waitpid pid
end
use require
main.rb:
require "script.rb"

ipython redirect stdout display corruption

I'm developing a system in python, and one functionality I need is the ability to have console output go to both the console and a user-specified file. This is replicating the Diary function in MATLAB. I have the following that works perfectly well on both IDLE on windows and python cmdline in ubuntu (this all exists inside a module that gets loaded):
class diaryout(object):
def __init__(self):
self.terminal = sys.stdout
self.save = None
def __del__(self):
try:
self.save.flush()
self.save.close()
except:
# do nothing, just catch the error; maybe it self was instantiated, but never opened
1/1
self.save = None
def dclose(self):
self.__del__()
def write(self, message):
self.terminal.write(message)
self.save.write(message)
def dopen(self,outfile):
self.outfile = outfile
try:
self.save = open(self.outfile, "a")
except Exception, e:
# just pass out the error here so the Diary function can handle it
raise e
def Diary(outfile = None):# NEW TO TEST
global this_diary
if outfile == None:
# None passed, so close the diary file if one is open
if isinstance(this_diary, diaryout):
sys.stdout = this_diary.terminal # set the stdout back to stdout
this_diary.dclose() # flush and close the file
this_diary = None # "delete" it
else:
# file passed, so let's open it and set it for the output
this_diary = diaryout() # instantiate
try:
this_diary.dopen(outfile) # open & test that it opened
except IOError:
raise IOError("Can't open %s for append!"%outfile)
this_dairy=none # must uninstantiate it, since already did that
except TypeError:
raise TypeError("Invalid input detected - must be string filename or None: %s"%Diary.__doc__)
this_dairy=none # must uninbstantiate it, since already did that
sys.stdout = this_diary # set stdout to it
Far superior to both IDLE and the plain python cmline, I'm using ipython; herein my problem lies. I can turn on the "diary" perfectly fine with no error but the display on the console gets messed. The attached screenshot shows this . The output file also becomes similarly garbled. Everything goes back to normal when I undo the redirection with Diary(None). I have tried editing the code so that it never even writes to the file, with no effect. It seems almost like something is forcing an unsupported character set or something I don't understand.
Anyone have an idea about this?

Resources