I have a supervisord file where like this
[program:decrypt]
command=export KEYTOKEN=$(aws kms decrypt --ciphertext-blob fileb://<(echo %(ENV_TOKENENC)s | base64 -d) --output text --query Plaintext --region %(ENV_REGION)s | base64 -d )
I am passing the environment ENV_TOKENENC,ENV_REGION to the container and I can echo those variables and confirm that the docker container is getting them, also the command to decrypt kms value also works.But when I put the kms decrypt command in supervised file it throws error saying ('ENV_REGION')&('ENV_CONSULTOKENENC') which cannot be expanded.
Am I putting the right value in supervisord file?
Setting an environment variable is easy, if you're setting it to a constant value:
[program:decrypt]
command=/usr/bin/env foo=bar baz=qux /path/to/something ...
or, with less overhead:
environment=foo="bar",baz="qux"
command=/path/to/something ...
However, dynamically generating that variable's value requires a shell:
[program:decrypt]
command=/bin/sh -c 'foo=$(generate-bar) /path/to/something'
Note that export is not actually needed here, as var=value something as part of a single command exports var having value value during the execution of something.
Related
I am trying to use the "az pipelines variable-group variable create ..." command to create a variable which references a different variable. e.g.
az pipelines variable-group variable create --project MyTestProject --
group-id 15 --name ‘ApplicationName’ --value 'TestApp-$(env)'
where the variable “env” is defined in a variable-group within the same project library.
When I run the command above it gives the error:
“Failed to load python executable” “exit /b 1”.
Despite an error being generated the variable is created; in the example above the variable 'ApplicationName' has the value 'TestApp-$(env', the trailing bracket character, ")", is missing and seems to be causing the problem.
The dollar sign, "$", and/or the opening bracket character "(" don't generate any error when used without the closing bracket ")".
I have tried escaping the closing bracket character with backslashes "\" and caret "`" characters but couldn't find any combination that would create the desired variable value, "TestApp-$(env)".
Could someone tell me how to escape the closing bracket so the variable is correctly created within the variable group.
I am running the following versions of az:
azure-cli 2.0.73
command-modules-nspkg 2.0.3
core 2.0.73
nspkg 3.0.4
telemetry 1.0.3
Extensions:
azure-devops 0.12.0
Python (Windows) 3.6.6
Many Thanks,
Gary
I have finally managed to figure out how to escape a variable whose name itself contains a different variable name. By calling the az cli command and wrapping the variable value in double quotes and a single quote, the variable is correctly created in DevOps:
pipelines variable-group variable create --project MyTestProject -- group-id 15 --name ApplicationName' --value '"TestApp-$(env)"'
DevOps-LibraryVariable-screenshot
As a further expansion on this topic I had a need to pass the value as a variable. My source was in a key value pair. In this case I used the following.
$key = $var.Key
$value = '"{0}"' -f $var.Value
az pipelines variable-group variable update --group-id $groupId --org $org --project $project `
--name $key --value $value
It depends on your OS and tools.
For example, in Windows OS, you can get environment variable with %variable_name%. So, the following would be right:
az pipelines variable-group variable create --project keyvault --group-id 1 --name "ApplicationName" --value "TestApp-%java_home%"
However, in PowerShell, you can get environment with "$env:variable_name". So, the following would be right:
az pipelines variable-group variable create --project keyvault --group-id 1 --name "ApplicationName2" --value "TestApp-$($env:java_home)"
Update:
So, in Azure Pipeline, you can use group variables as following:
pool:
name: Hosted VS2017
demands:
- msbuild
- visualstudio
- vstest
variables:
- group: vargroup
steps:
- task: AzureCLI#1
inputs:
azureSubscription: 'CSP Azure (e5b0fcfa-e859-43f3-8d84-5e5fe29f4c68)'
scriptLocation: 'inlineScript'
inlineScript: |
echo the variable var1:%var1%
I have a variable group:
And you can see that: echo the variable var1:%var1% will be echo the variable var1:value1
I'm starting a python script with supervisord on a linux debian platform. The user selected for executing the script shall depend on the value of an environmental variable. How can i make the field "user=" in a supervisord configuration file conditional?
First, I have added to the supervisor.service an environmental variable SPECIALUSER=myuser (file /lib/systemd/system/supervisor.service)
[Service]
ExecStart=/usr/bin/supervisord -n -c /etc/supervisor/supervisord.conf
ExecStop=/usr/bin/supervisorctl $OPTIONS shutdown
ExecReload=/usr/bin/supervisorctl -c /etc/supervisor/supervisord.conf $OPTIONS reload
KillMode=process
Restart=on-failure
Environment=SPECIALUSER=myuser
Then I try to use the variable inside my supervisord.conf file:
[program:myprogram]
command=python myscript.py
user="if [ %(ENV_SPECIALUSER)s = myuser]; then root; else standarduser; fi"
But I get the following error when i try to reread the supervisord.conf
ERROR: CANT_REREAD: Invalid user name "if [ myuser = myuser ]; then root; else standarduser; fi" in section 'program:myprogram' (file: '/etc/supervisor/conf.d/supervisord.conf')
The environmental variable is interpreted correctly but the bash script, not.
I thought about entering the name of the user directly in the variable Environment=SPECIALUSER=root, but the environmental varialble is not always available.
If the environment variable is set to SPECIALUSER=myuser, I expect supervisor.d to interpret my program as
[program:myprogram]
command=python myscript.py
user=root
In all other cases as
[program:myprogram]
command=python myscript.py
user=standarduser
According to the documentation the user parameter value is never "interpreted" or sent to a shell. This means that it tries to use the entire value as the username.
http://supervisord.org/configuration.html#program-x-section-settings
All parameters aren't interpreted or sent to a shell. This means that you can't insert conditionals generally in parameters in your supervisord.conf.
If your goal is to just use different users on say different platforms or one for development and another on a deploymentserver I suggest creating a dedicated user for the service.
If your goal is to only sometimes run as superuser I suggest always using user=root in your supervisord.conf and wrapping your program in a small shell script that interprets this environment variable and drops privileges accordingly.
This other SO question might help you:
https://unix.stackexchange.com/questions/132663/how-do-i-drop-root-privileges-in-shell-scripts
I'm trying to use a Jenkins Active Choice Reactive Parameter and using Groovy to run an aws kms cli command, but the value doesn't seem to be returning. I've looked at the couple other posts around using aws cli in Groovy but none of them resolve my issue. I'm not a Java developer by any means so any help is welcomed. If I use the code below with a "ls" command then the value is being populated in the parameter.
Yo
def command = "aws kms decrypt --query Plaintext --output text --ciphertext-blob fileb://<(echo 'my-cipher' | base64 -d') | base64 -d".execute()
command.waitFor()
return [command.text]
Thanks in advance for any help
I figured this one out using the below from Run a compound shell command from Java/Groovy. Thanks to #cfrick for the comment above.
def out = ['bash', '-c', "aws kms decrypt --query Plaintext --output text --ciphertext-blob fileb://<(echo 'my-cipher' | base64 -d) | base64 -d"].execute([], new File('/tmp')).text
return [out]
New to using SQLCMD.
If I run the following sqlcmd I get an error
Incorrect syntax near '.209
Command:
sqlcmd -E -S MyServer\SQLEXPRESS -d MyDatebase-Q "EXEC spRunThisPS #IP=$(IP)" /v IP="192.168.209.4"
If I just have "192.168" as the parameter the script will run (obviously the PS fails because invalid ip). Not sure if the amount of "." causes it to fail or not.
Any thoughts or suggestions?
Nick
You probably need to wrap the parameter value in quotes, either by specifying the IP value as IP="'192.168.209.4'", or by adding them around the parameter value in the EXEC command (EXEC spRunThisPS #IP='$(IP)').
Otherwise it's trying to interpret it as a numeric value rather than a string, which is why 192.168 is ok and 192.168.209.4 isn't.
I am trying to check whether variable GROUP exist in SAS data set file or not from the UNIX command but unfortunately it's showing that GROUP variable does not exist in the data set,However GROUP variable is present in SAS data set.
In my command for case sensitive and whole word match I am using i and w options of grep command respectively. But still UNIX command is not giving the expected result.I s there any way to fix this issue?
Below is the command which I am using:
sasfile="sasdata"
rwords="GROUP"
cat $sasfile | grep -iqw "$rwords"
Thank you
As mentioned in earlier comment
SAS data sets are stored in disk files using a proprietary format.
There may be encodings and storage methodologies that do not yield the
information you seek in a plain text examination of said disk file.
Running SAS code in a SAS session is the definitive way to glean information about a data set.
What will that code look like ?
Proc CONTENTS
Data step or macro code that uses VARNAME function
... many other ways ...
In UNIX SAS can use stdio.
From "SAS(R) 9.2 Companion for UNIX Environments", STDIO System Option: UNIX
Details
This option tells SAS to take its input from standard input (stdin),
to write its log to standard error (stderr), and to write its output
to standard output (stdout).
This option is designed for running SAS
in batch mode or from a shell script. If you specify this option
interactively, SAS starts a line mode session.
The STDIO option
overrides the DMS, DMSEXP, and EXPLORER system options. The STDIO
option does not affect the assignment of the Stdio, Stdin, and Stderr
filerefs. See Filerefs Assigned by SAS in UNIX Environments for more
information.
For example, in the following SAS command, the file
myinput is used as the source program, and files myoutput and mylog
are used for the procedure output and log respectively.
sas -stdio < myinput > myoutput 2> mylog
If you are using the C shell, you should
use parentheses:
(sas -stdio < myinput > myoutput ) >& output_log
With -stdio you want a short SAS program that can indicate if a variable is present in a data set, or perhaps emit a list of variables in a data set for further shell processing. A Proc CONTENTS step is short and sweet.
So looking for your proverbial needle in a haystack
sasfile=<path to data set file>/<dataset>.sas7bdat
needle=GROUP
echo "Proc CONTENTS data=""$sasfile""" | sas -stdio | grep $needle
The default CONTENTS output might contain yield some false matches. So you could also try
echo "Proc CONTENTS noprint data=""$sasfile"" out=list;data _null_;set list;file print;put name;"
| sas -stdio
| grep -i "GROUP"
You could try:
sasfile="sasdata"
rwords="GROUP"
grep -iw "$rwords" "$sasfile"
The only difference between these and your original commands is that I omitted cat and grep's quiet flag -q.
Sample input in sasdata:
fasd group
fdsfds fdsfdsa
fdsfd as GROUP afdsfdsa
Output:
fasd group
fdsfd as GROUP afdsfdsa
The -q flag of grep will suppress standard output but echo $? can retrieve the return value of grep. Using the same input file as before:
grep -iqw "$rwords" "$sasfile" # No stout
echo $? # Prints 0, means grep succeeded
grep -iqw "word" "$sasfile" # No stout
echo $? # Prints 1, means grep failed