TCL exec echo does not work with enviornment varaibles - environment-variables

Let us say that I have environment variable PO, with value 1.If I use the LINUX echo command I get:
>echo $PO
1
However, if I use TCL and exec, I do not get interpolation:
>exec echo "\$PO"
$PO
Now, if I do something more elaborate, by using regsub to replace every ${varname} with [ lindex array get env varname ] 0 ], and use substr, it works:
>subst [ regsub -all {\$\{(\S+?)\}} "\${PO}/1" "\[ lindex \[ array get env \\1 \] 1 \]" ]
1/1
I have some corner cases, sure. But why is the exec not giving back what the shell would do?

why is the exec not giving back what the shell would do?
Because exec is not a shell.
When you do echo $PO from a shell, echo is not responsible for resolving the value. It is the shell that converts $PO to the value 1 before calling echo. echo never sees $PO when calling it from the shell.
If you are trying to emulate what the shell does, then you need to do the same work as the shell (or, invoke an actual shell to do the work for you).

Tcl is a lot more careful about where it does interpolation than Unix shells normally are. It keeps environment variables out of the way so that you don't trip over them by accident, and does far less processing when it invokes a subprocess. This is totally by design!
As much as possible (with a few exceptions) Tcl passes the arguments to exec through to the subprocesses it creates. It also has standard mechanisms for quoting strings so that you can control exactly what substitutions happen before the arguments are actually passed to exec. This means that when you do:
exec echo "\$PO"
Tcl is going to do its normal substitution rules and get these exact arguments to the command dispatch: exec, echo, and $PO. This then calls into the exec command, which launches the echo program with one argument, $PO, which does exactly that. (Shells usually substitute the value first.) If you'd instead done:
exec echo {$PO}
you would have got the same effect. Or even if you'd done:
exec {*}{echo $PO}
You still end up feeding the exact same characters into exec as its arguments. If you want to run the shell on it, you should explicitly ask for it:
exec /bin/sh -c {echo $PO}
The bit in the braces there is a full (small) shell script, and will be evaluated as such. And you could do this even:
exec /bin/sh -c {exec echo '$PO'}
It's a bit of a useless thing to do but it works.
You can also do whatever substitutions you want from your own code. My current favourite from Tcl 8.7 (in development) is this:
exec echo [regsub -all -command {\$(\w+)} "\$PO" {apply {- name} {
global env
return $env($name)
}}]
OK, total overkill for this but since you can use any old complex RE and script to do the substitutions, it's a major power tool. (You can do similar things with string map, regsub and subst in older Tcl, but that's quite a bit harder to do.) The sky and your imagination are the only limits.

Related

Docker run env variable does not work with single quotes?

I was running the following command:
docker run -e 'SOME_ENV_VARIABLE=somevalue' somecommand
And this did not pass the env variable to somecommand (Note: SOME_ENV_VARIABLE and somevalue did not have un-escaped single quotes, they were capital snake case and lowercase respectively without special character.
Later, I tried:
docker run -e "SOME_ENV_VARIABLE=somevalue" somecommand
And this somehow worked (note the double quotes).
Why is this? I have seen documentation with single quotes.
I tried this on Windows.
Quotation is handled by your shell - since you are on windows, that is cmd.exe. And, from what I understand, since cmd.exe does not handle single quotes, it will pass that argument with the quotes to docker.
It would work on Linux though (which usually has the bash shell), being the the environment many people would use docker on, and as such the reason it is present in a lot of documentation.

Can't use commands from ipython or julia repls with zsh

When I try to run a shell command in ipython or the julia repl it just says
shell> ls
zsh:1: command not found: ls
Not sure if it matters, but I have my path set in zshenv instead of zshrc so that emacs shell works.
Any ideas?
Edit:
I'm on macOS 10.14.6
For Julia, The shell> REPL prompt does in fact use a shell to execute its commands (on non-Windows systems). It effectively does something like run(`$shell -c ls`), and for most shells (including zsh) this means "non-interactive" mode and limits the number of init files that get loaded. You want to make sure your shell is working in this mode; I'd guess that if you type zsh -c ls at your terminal it'll be similarly broken.
Alternatively, you can customize which shell Julia uses through an environment variable. Setting JULIA_SHELL=/bin/sh is probably a safe bet — Julia uses that environment variable if it is set, otherwise it uses SHELL, and finally it falls back to /bin/sh if neither is set.
I'm not as familiar with ipython, but I'd wager it's doing something similar.

User job not enabled

i have a codestains.conf file in ~/.init folder
description "Codestains"
author "Varun Mundra"
start on virtual-filesystems
stop on runlevel [06]
env PATH=/opt/www/codestains.com/current/bin:/usr/local/rbenv/shims:/usr/local/rbenv/bin:/usr/local/bin:/us$
env RAILS_ENV=production
env RACK_ENV=production
setuid ubuntu
setgid sudo
chdir /opt/www/codestains.com
pre-start script
exec >/home/ubuntu/codestains.log 2>&1
exec /opt/www/codestains.com/current/bin/unicorn -D -c /opt/www/codestains.com/current/config/unicorn.rb $
end script
post-stop script
exec kill 'cat /tmp/unicorn.codestains.pid'
end script
I have added https://gist.github.com/bradleyayers/1660182 in /etc/dbus-1/system.d/Upstart.conf` to enable Upstart user jobs
But everytime I run
start codestains
sudo start codestains
I get "start: Unknown job: codestains".
I have tried a lot of things available online. Nothing seems to help.
Also,
init-checkconf codestains.conf
gives "File codestains.conf: syntax ok"
I spot one error that is certainly a problem; I do not know if it is the only problem. I haven't made any attempt to test it. However, this bit:
exec kill 'cat /tmp/unicorn.codestains.pid'
is definitely wrong, it would pass the string cat /tmp/unicorn.codestains.pid to the kill command, which will not do what you want.
You may have seen an example, and missed that they are backtick characters, which causes the shell to execute cat /tmp/unicorn.codestains.pid, capture its STDOUT, and then interpolate the result where you put the backticks; IOW it passes the contents of that pid file to the kill command.
Like this:
exec kill `cat /tmp/unicorn.codestains.pid`
Note the subtly different backtick character
Which shells (bash, at least) will treat specially as I described: http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_03_04.html
(see the section on "Command substitution")
HTH

Ruby background process STDOUT is empty

I'm having a weird issue with a start-up script which runs a Sinatra script using the shell's "daemon" function. The problem is that when I run the command at the command line, I get output to STDOUT. If I run the command at the command line exactly as it is in the script -- less the daemon part -- the output is correctly redirected to the output file. However, when the startup script runs it (see below), I get stuff to the STDERR log but not to the STDOUT log.
The relevant lines of the script:
#!/bin/sh
# (which is and has been a symlink to /bin/bash
# Source function library.
. /etc/init.d/functions
# Set Some Variables
RUNAS="joeuser"
PID=/var/run/myapp.pid
LOG="/var/log/myapp/app-out.log"
ERR_LOG="/var/log/myapp/app-err.log"
APPLICATION_COMMAND="RAILS_ENV=production ruby /opt/myapp/lib/daemons/my-sinatra-app.rb -p 8002 2>>${ERR_LOG} >>${LOG} &"
# Snip a bunch. This is the applicable line from the "start" case:
daemon --user $RUNAS --pidfile $PID $APPLICATION_COMMAND &> /dev/null
Now, the funky parts:
The error log is written to correctly via the redirect of STDERR.
If I reverse the order of the >> and the 2>> (I'm grasping at straws, here!), the behavior does not change: I still get STDERR logged correctly and STDOUT is empty.
If the output log doesn't exist, the STDOUT redirect creates the file. But, the file remains 0-length.
This used to work. The log directory is maintained by log-rotate. All of the more-recent 'out' logs are 0-length. The older ones are not. It seems like it stopped working some time in April. The ruby code didn't change at any time near then; neither did the startup script.
We're running three different services in this way. Two of them are ruby daemons (one uses sinatra, one does not) and the other is a background java process. This is occurring for BOTH of the ruby processes but is not happening on the java process. Maybe something changed in Ruby?
FTR, we've got ruby 1.8.5 and RHEL 5.4.
I've done some more probing. The daemon function does a bunch of stuff, but the meat of the matter is that it runs the program using runuser. The command essentially looks like this:
runuser -s /bin/bash - joeuser -c "ulimit -S -c 0 >/dev/null 2>&1 ; RAILS_ENV=production ruby /opt/myapp/lib/daemons/my-sinatra-app.rb -p 8002 '</dev/null' '>>/var/log/myapp/app-out.log' '2>>/var/log/myapp/app-err.log' '&'"
When I run exactly that at the command line (both with and without the single-ticks that got added somewhere along the line), I get the exact same screwy behavior w.r.t. the output log. So, it seems to me that this is an issue of how ruby (?) interacts with runuser?
Too long to put in a comment :-)
change the shebang to add #!/bin/sh -x and verify that everything is expanded according to your expectations. Also, when executing from terminal, your .bashrc file is sourced, when executing from script, it is not; might be something in you're environment that differ. One way to find out is to do env from terminal and from script and diff the output
env > env_terminal
env > env_script
diff env_terminal env_script
Happy hunting...

Analysing a shell script

This would be part of a reverse-engineering project.
To determine and document what a shell script (ksh, bash, sh) does, it is comfortable, if you have information about what other programs/scripts it calls.
How could one automate this task? Do you know any program or framework that can parse a shell script? This way for instance, I could recognize external command calls -- a step to the right direction.
For bash/sh/ksh, I think you can easily modify their source to log what has been executed. That would be a solution.
How about:
Get a list of distinct words in that script
Search $PATH to find a hit for each
?
bash -v script.sh ?
Bash's xtrace is your friend.
You can invoke it with:
set -x at the top of your script,
by calling your script with bash -x (or even bash --debugger -x),
or recursively by doing (set -x; export SHELLOPTS; your-script; )
If you can't actually run the script, try loading it into a text editor that supports syntax highlighting for Bash. It will color-code all of the text and should help indicate what is a reserved word, variable, external command, etc.

Resources