I'm using libreoffice/openoffice as a headless process to handle some document conversion tasks on my server that I "submit" via unoconv. Once in a while, the process that actually does the work, soffice.bin, seems to get wedged. I tried playing around with strace and saw that when launching new unoconv instances, they still connect and talk to the soffice process, just that nothing else happens after the 'bad' document goes in. If it were so simply as to just detect that soffice does not talk to incoming sockets any more, it'd be easy to write a watchdog. But it's not that simple, apparently. Any ideas how to tell when things have gone south?
Here's what I set up as a cron job:
def monitor_unoconv
retval = false
target_dir = "/tmp/monitor_unoconv"
begin
Timeout::timeout(30) do
FileUtils.mkdir_p(target_dir)
FileUtils.cp(File.dirname(__FILE__) + "/../hello.odt", target_dir)
Dir.chdir target_dir do
retval = system("unoconv -f html hello.odt")
end
end
rescue => e
STDERR.puts "Caught error #{e.inspect}"
retval = false
end
if !retval
STDERR.puts "soffice process appears hung. Killing it"
STDERR.puts `killall soffice.bin`
sleep 5
STDERR.puts `killall -9 soffice.bin`
end
end
Seems to work ok.
Issue might be with soffice mutiple threads,
so resolving trick might be like:
Using unoconv as a service.
creating a init.d script and starting as a daemon.
So instead of unoconv calling soffice to start , unoconv as a service will keep running a single and maintain it.
create starting process file as follows:
!/bin/sh
case "$1" in
start)
/usr/bin/unoconv --listener &
;;
stop)
killall soffice.bin
;;
restart)
killall soffice.bin
sleep 1
/usr/bin/unoconv --listener &
;;
esac
Related
My Ruby on Rails application remotely starts some scripts on a distant SuSe server (SUSE Linux Enterprise Server 15 SP2). It relies on the net-ssh gem which is declared in the Gemfile: gem 'net-ssh'.
The script is triggerd remotely through the following block:
Net::SSH.start(remote_host, remote_user, password: remote_password) do |ssh|
feed_back = ssh.exec!("#{event.statement}")
end
This works as expected as long as long as the Rails server runs on Windows Server 2016, which is my DEV environment. But when I deploy to the Validation environment, which is SUSE Linux Enterprise Server 15 SP2, I get this error message:
Errno::ENOTTY in myController#myMethod
Inappropriate ioctl for device
On another hand, issuing the SSH request through the command line - from SUSE to SUSE - works as expected too. Reading around I did not find a relevant parameter for the Net::SSH module to solve this.
Your suggestions are welcome!
I finally found out that the message refers to the operating mode of SSH: it requires a sort of terminal emulation - so called pty - wrapped into a SSH chanel.
So I implemented it this way:
Net::SSH.start(remote_host, remote_user, password: remote_password) do |session|
session.open_channel do |channel|
channel.request_pty do |ch, success|
raise "Error requesting pty" unless success
puts "------------ pty successfully obtained"
end
channel.exec "#{#task.statement}" do |ch, success|
abort "could not execute command" unless success
channel.on_data do |ch, data|
puts "------------ got stdout: #{data}"
#task.update_attribute(:return_value, data)
end
channel.on_extended_data do |ch, type, data|
puts "------------ got stderr: #{data}"
end
channel.on_close do |ch|
puts "------------ channel is closing!"
end
end
end
### Wait until the session closes
session.loop
end
This solved my issue.
Note:
The answer proposed above was only a part of the solution. The same error occured again with this source code when deploying to the production server.
The issue appears to be the password to the SSH target: I retyped it by hand instead of doing the usual copy/paste from MS Excel, and the SSH connection is now successful!
As the error raised is not a simple "connection refused", I suspect that the password string had a specific character encoding, or an unexpected ending character.
As the first proposed solution provides a working example, I leave it there.
I want to start a ngrok process when server starts. To achieve this, I coded a ngrok.rb lib and I call it within an initializer
app/lib/ngrok.rb
require "singleton"
class Ngrok
include Singleton
attr_accessor :api_url, :front_url
def start
if is_running?
return fetch_urls
end
authenticate
started = system("ngrok start --all -log=stdout > #{ENV['APP_ROOT']}/log/ngrok.log &")
system("sleep 1")
if !started
return { api: nil, front: nil }
end
urls = fetch_urls
sync_urls(urls["api_url"], urls["front_url"])
return urls
end
def sync_urls(api_url, front_url)
NgrokSyncJob.perform_later(api_url, front_url)
end
def is_running?
return system("ps aux | grep ngrok")
end
def restart
stop
return start
end
def stop
return system("pkill ngrok")
end
def authenticate
has_file = system("ls ~/.ngrok2/ngrok.yml")
if has_file
return true
else
file_created = system("ngrok authtoken #{ENV['NGROK_TOKEN']}")
if file_created
return system("cat " + ENV['APP_ROOT'] + '/essentials/ngrok/example.yml >> ~/.ngrok2/ngrok.yml')
else
return false
end
end
end
def fetch_urls
logfile = ENV['APP_ROOT'] + '/log/ngrok.log'
file = File.open logfile
text = file.read
api_url = nil
front_url = nil
text.split("\n").each do |line|
next if !line.include?("url=") || !line.include?("https")
if line.split("name=")[1].split(" addr=")[0] == "ncommerce-api"
api_url = line.split("url=")[1]
elsif line.split("name=")[1].split(" addr=")[0] == "ncommerce"
front_url = line.split("url=")[1]
end
end
file.close
self.api_url = api_url
self.front_url = front_url
res = {}
res["api_url"] = api_url
res["front_url"] = front_url
return res
end
end
config/initializers/app-init.rb
module AppModule
class Application < Rails::Application
config.after_initialize do
puts "XXXXXXXXXXXXXXXXXXXXXXX"
Ngrok.instance.start
puts "XXXXXXXXXXXXXXXXXXXXXXX"
end
end
end
When I type rails serve, here is a sample of the output
So we know for sure my initializer is being called, but when I look at rails console if it's running, it's not!
But when I type Ngrok.instance.start in rails console, here's the output:
And it starts!
So, my question is: WHY ON EARTH is system("ngrok start --all -log=stdout > #{ENV['APP_ROOT']}/log/ngrok.log &") NOT working on rails serve, but it is on rails console?
UPDATE
If I use 'byebug' within ngrok.rb and use rails serve, when I exit byebug with "continue", the ngrok process is created and works
You're creating an orphaned process in the way that you use system() to start the ngrok process in the background:
system("ngrok start --all -log=stdout > #{ENV['APP_ROOT']}/log/ngrok.log &")
Note the & at the end of the commandline.
I'd need more details about your runtime environment to tell precisely which system policy kills the orphaned ngrok process right after starting it (which OS? if Linux, is it based on systemd? how do you start rails server, from a terminal or as a system service?).
But what's happening is this:
system() starts an instance of /bin/sh to interpret the commandline
/bin/sh starts the ngrok process in the background and terminates
ngrok is now "orphaned", meaning that its parent process /bin/sh is terminated, so that the ngrok process can't be wait(2)ed for
depending on the environment, the terminating /bin/sh may kill ngrok with a SIGHUP signal
or the OS re-parents ngrok, normally to the init-process (but this depends)
When you use the rails console or byebug, in both cases you're entering an interactive environment, which prepares "process groups", "session ids" and "controlling terminals" in a way suitable for interactive execution. These properties are inherited by child processes, like ngrok. This influences system policies regarding the handling of the orphaned background process.
When ngrok is started from rails server, these properties will be different (depending on the way rails server is started).
Here's a nice article about some of the OS mechanisms that might be involved: https://www.jstorimer.com/blogs/workingwithcode/7766093-daemon-processes-in-ruby
You would probably have better success by using Ruby's Process.spawn to start the background process, in combination with Process.detach in your case. This would avoid orphaning the ngrok process.
I am working on importing data from web based CSV into database, so I created a rake task that imports data into database. However I tried to make running my rails app a little seamless and I integrated the import rake task and running rails server into foreman.
However, when I run foreman start, the processes start but terminates after the rake task finishes. I also will like that rake task to start first before running rails s
Here is what I have done below:
lib/tasks/web_import.rake
require 'open-uri'
require 'csv'
namespace :web_import do
desc 'Import users from csv'
task users: :environment do
url = 'http://blablabla.com/content/people.csv'
# I forced encoding so avoid UndefinedConversionError "\xC3" from ASCII-8BIT to UTF-8
csv_string = open(url).read.force_encoding('UTF-8')
counter = 0
duplicate_counter = 0
user = []
CSV.parse(csv_string, headers: true, header_converters: :symbol) do |row|
next unless row[:name].present? && row[:email_address].present?
user = CsvImporter::User.create row.to_h
if user.persisted?
counter += 1
else
duplicate_counter += 1
end
end
p "Email duplicate record: #{user.email_address} - #{user.errors.full_messages.join(',')}" if user.errors.any?
p "Imported #{counter} users, #{duplicate_counter} duplicate rows ain't added in total"
end
end
Procfile
rake: rake web_import:users
server: rails s
when I run forman start, the image below shows the process
I will like the rake task in the foreman to run first before running rails s command. I also don't want it to terminate by itself. I don't know what am doing wrong.
Any help is appreciated.
I solved this by refactoring the Procfile. Instead of having two tasks, I merged it to just one command using && so I can determine which command takes the prefix and which one takes the suffix.
So I changed the profile to:
tasks: rake web_import:users && rails s -p 3000
With this I have my import run first and server command being the last.
If you noticed, I added port with -p flap so as not make sure server is listening on port 3000. Note adding port is optional.
I hope this helps someone as well.
I am writing a web interface for a hardware device and am currently doing the network configuration wizard. I want to have the user choose the SSID and if it is secured, then enter a password. I want to check then and there that the password is correct without going any further in the wizard.
Is there any easier way to do this than throwing the details at wpa_supplicant and parsing log output?
Well, could find much, maybe my Google fu sucks. I ended up making this script to do it for me. It uses wpa_supplicant to try to connect and then parses the logs to look for certain strings that indicate auth failure or success. It is only tested with v2.3, if the log output is different in other versions it may not work properly.
It will always cause a current connection on the wireless interface you are testing from to drop momentarily but will come back as soon as the wpa_supplicant started by this script is killed by the script exiting.
#!/usr/bin env ruby
# This script will run WPA to check authentication to a Wifi AP. It will return status 0 for success
# or status 1 for failure, as well as logging some information about what the script is doing.
#
# It has a built in timeout (default 15sec) in case something holds the script up, on a BeagleboneBlack
# this typically took ~10s for a failed auth, and ~2s for a successful auth. In most cases the WPA
# supplicant was killed before DHCP could configure the interface.
#
# In the case where the interface is already connected to an AP and configured via WPA/DHCP, this script
# will cause that connection to drop, regardless of AP auth success or failure. With the latter the
# connection is promptly restored after the script is finished, however with the former the connection
# may be momentarily reconfigured via DHCP to the new APs details, before control is given back to the
# original WPA process. It is unclear if this disconnection can be avoided.
#
# This has only been tested with wpa_supplicant v2.3
require 'logger'
require 'fileutils'
LOG_FILE = "/tmp/wpalog"
PIDFILE = "/tmp/wpapid"
LOG = Logger.new(STDOUT)
ssid = ARGV[0]
pass = ARGV[1]
timeout = 15
dev = "wlan0"
abort "Usage: #{$0} <ssid> <pass>" if ssid.nil? or pass.nil?
File.write(LOG_FILE, "")
# make sure we don't leave wpa running
at_exit do
kill_wpa!
end
# kill the auth process if it's pidfile exists
def kill_wpa!
if File.exist?(PIDFILE)
pid = File.read(PIDFILE).strip
LOG.info "Killing WPA on PID #{pid}"
Process.kill 9, pid.to_i
FileUtils.rm PIDFILE
end
end
# parse the log for indications of auth success/failure
def parse_log
log = File.read(LOG_FILE)
if log.include? "WPA: Key negotiation completed"
return true
end
if log.include?("pre-shared key may be incorrect") || log.include?("auth_failures=1")
return false
end
nil
end
# timeout so we don't keep going forever if theres some issue
Thread.new do
sleep timeout
LOG.fatal "Operation timed out"
exit
end
# run the process to try to auth to the AP
s = Time.now
LOG.info "Starting WPA Supplicant"
system "bash -c 'wpa_supplicant -Dwext -c <(wpa_passphrase \"#{ssid}\" \"#{pass}\") -B -P #{PIDFILE} -f #{LOG_FILE} -i #{dev} 2>/dev/null'"
result = nil
# loop until the parse_log gives us a non nil result indicating auth success or failure
LOG.info "Searching WPA log for authentication state"
loop do
result = parse_log
sleep 0.2 and next if result.nil?
break
end
f = Time.now
duration = (f - s).to_f.round(2)
LOG.info "Found authentication state in #{duration} seconds"
# kill WPA ASAP before DHCP takes over and changes the interface config
kill_wpa!
if result
LOG.info "Authentication successful"
else
LOG.error "Authentication failed"
end
# empty the log in case there are creds in it
File.write(LOG_FILE, "")
# use return values to signal auth failure or success
exit result ? 0 : 1
I would like to connect to a remote computer via another using ruby.
This scheme is the following :
Local -> proxy -> remote
I have this code which is doing the work for a direct access :
require 'net/ssh'
Net::SSH.start(remote_host, remote_user) do |ssh|
puts ssh.exec!'hostname'
end
However, when I try with the proxy, the command 'hostname' is executed and correct, but then the code freezes, same if I call ssh.close.
Here is the code :
require 'net/ssh'
require 'net/ssh/proxy/command'
proxy_cmd = Net::SSH::Proxy::Command.new('ssh proxy_user#proxy_host nc %h %p')
Net::SSH.start(remote_host, remote_user, :proxy => proxy) do |ssh|
puts ssh.exec!'hostname'
end
The loggin is done without password thanks to a rsa key. And the proxycommand is working (I was using it in bash before)
Would someone knows what I am doing wrong ?
Thank you very much for your interest,
EDIT : here is the last line in the logs, it blocks there :
I, [2013-10-16T23:01:19.304778 #3785] INFO -- net.ssh.connection.session[4555128]: closing remaining channels (0 open)
I've just bumped in the same issue - command line ssh was working and net/ssh was hanging on me when using proxycommand.
Debuging net/ssh brought me as far as: https://github.com/net-ssh/net-ssh/blob/master/lib/net/ssh/transport/session.rb#L113 and the whole thing was hanging on the .close call of the socket.
I'm not sure what caused this, but adding timeout to nc command seems to have solved it:
ProxyCommand ssh proxy_server#proxy_server nc -q 1 %h %p