First what I think I've understood:
nix-shell -p git
is the same as doing
nix-shell
if in the same directory there is a file default.nix with this content:
with import <nixpkgs> {};
stdenv.mkDerivation rec {
name = "name";
env = buildEnv { name = name; paths = buildInputs; };
buildInputs = [git ]; }
In the case I am right and I 've created an environment with the first method,
How can I get a default.nix file that I can reuse to recreate the same environment?
Related
I'm trying to use bash variables in a shellScript in jetbrains space automation to no success.
My .space.kts is as follows;
job("mvn compile"){
container(displayName="mvn", image="maven:3.8.5-eclipse-temurin-17"){
shellScript {
content = """
FOO="bar"
echo $FOO
"""
}
}
}
in the above i'd expect "bar" to be echoed, but instead im getting the following error when this tries to run;
Dsl file '/tmp/16487320722162400386/.space.kts' downloaded in 1736 ms
Compiling DSL script /tmp/16487320722162400386/.space.kts...
downloading /home/pipelines-config-dsl-compile-container/space-automation-runtime.jar ...
[SUCCESSFUL ] com.jetbrains#space-automation-runtime;1.1.100932!space-automation-runtime.jar (71ms)
Compilation failed in 8.652797664s.
ERROR Unresolved reference: FOO (.space.kts:9:23)
Cleaned up the output folder: /tmp/16487320722162400386
DSL processing failed: Compilation exited with non zero exit code: 2. Exit code: 102
I had planned on parsing the branch name from JB_SPACE_GIT_BRANCH and storing it in a variable to use in a call to mvn to build and tag a container using Jib
Is there anyway that i can use variables within the content of a shellScript? or should/ can this be done in a different way?
You need to replace $ by ${"$"}:
job("mvn compile") {
container(displayName="mvn", image="maven:3.8.5-eclipse-temurin-17") {
shellScript {
content = """
FOO="bar"
echo ${"$"}FOO
"""
}
}
}
Or use a sh file file.sh like this:
FOO="bar"
echo $FOO
.
job("mvn compile") {
container(displayName="mvn", image="maven:3.8.5-eclipse-temurin-17") {
shellScript {
content = """
./file.sh
"""
}
}
}
In gradle 7 I have created this method:
def shellCmd(String cmd) {
exec {
executable "sh"
args "-c", cmd
}
}
And this "pure" groovy version:
def shellCmd2(String cmd) {
def process = cmd.execute()
def output = new StringWriter(), error = new StringWriter()
process.waitForProcessOutput(output, error)
}
Which I call from another method e.g.:
def myMethod() {
shellCmd('ls -la')
}
I am now experimenting with getting it to work with multi-line (jenkins like) shell commands:
def myMethod() {
def cmd = """
for i in \$(ls -la); do
if [[ \$i == 'settings.gradle' ]]; then
echo "Found $i"
fi
done
"""
shellCmd(cmd)
}
but it fails with:
script '/home/user/samples/build.gradle': 47: Unexpected input: 'i' # line 47, column 5.
for i in $(ls -la); do
^
1 error
Probably I am breaking all the rules here but any input?
Also tried most of the suggestions here:
What's wrong with Groovy multi-line String?
but no luck so far.
Also based on the suggestion below I have tried to use shellCmd2 method (I am leaning towards using a plain groovy method for this to make it easier to debug outside of gradle) with:
def myMethod() {
def cmd = """
for i in \$(ls -la); do
if [ \$i = 'settings.gradle' ]; then
echo "Found \$i"
fi
done
"""
shellCmd2(cmd)
}
But that gives:
Caught: java.io.IOException: Cannot run program "for": error=2, No such file or directory
java.io.IOException: Cannot run program "for": error=2, No such file or directory
So seems the for keyword is now causing an issue.
There is nothing wrong with your multiline string. I had to change three things to make your code work:
Use single brackets [ and ]
Use single equal sign (=)
Escape the missing $i variable which was being interpreted as a Groovy variable.
You are using sh so you should only use POSIX compatible features.
Code:
def shellCmd(String cmd) {
exec {
executable "sh" // or use another shell like bash or zsh (less portable)
args "-c", cmd
}
}
def myMethod() {
def cmd = """
for i in \$(ls -la); do
if [ \$i = 'settings.gradle' ]; then
echo "Found \$i"
fi
done
"""
shellCmd(cmd)
}
You can compare with double brackets with shells like zsh and bash.
I have a variable that I need to add as Jenkins Environment Variable so that the $jenkins_home/init.d groovy scripts can use them. I have this variable set in the host machine(where iam installing jenkins) - Amazon linux EC2 via a .sh file in /etc/profile.d and sourceing the /etc/profile . This did not help as I am not able to access the variable from jenkins. Can anybody please help on how I can achieve this.
In Jenkins, you can setup the global variables using the below given groovy script. You will have to use this in a shell script that you use to provision a jenkins server. This should be executed right after install so that, jenkins can use then while running jobs.
The below given groovy script creates the environment variables one by one,
createGlobalEnvironmentVariables('Var1','DummyValue')
This adds a global variable with name as Var1 and value as DummyValue
If you have multiple values, you can use the ones given below.
envVars.put("Key1", "Value1)
envVars.put("Key2", "Value2)
before calling instance.save(). We have a shell script file that has this and many other functions to install and configure jenkins in a single shot.
import hudson.EnvVars;
import hudson.slaves.EnvironmentVariablesNodeProperty;
import hudson.slaves.NodeProperty;
import hudson.slaves.NodePropertyDescriptor;
import hudson.util.DescribableList;
import jenkins.model.Jenkins;
public createGlobalEnvironmentVariables(String key, String value){
Jenkins instance = Jenkins.getInstance();
DescribableList<NodeProperty<?>, NodePropertyDescriptor> globalNodeProperties = instance.getGlobalNodeProperties();
List<EnvironmentVariablesNodeProperty> envVarsNodePropertyList = globalNodeProperties.getAll(EnvironmentVariablesNodeProperty.class);
EnvironmentVariablesNodeProperty newEnvVarsNodeProperty = null;
EnvVars envVars = null;
if ( envVarsNodePropertyList == null || envVarsNodePropertyList.size() == 0 ) {
newEnvVarsNodeProperty = new hudson.slaves.EnvironmentVariablesNodeProperty();
globalNodeProperties.add(newEnvVarsNodeProperty);
envVars = newEnvVarsNodeProperty.getEnvVars();
} else {
envVars = envVarsNodePropertyList.get(0).getEnvVars();
}
envVars.put(key, value)
instance.save()
}
createGlobalEnvironmentVariables('Var1','DummyValue')
I'm trying to configure a container within nixos containers for example:
containers.abc123 = {
config = { config, pkgs, ... }:
{
systemd.services = {
finder-email-requests = {
description = "";
enable = true;
environment = {
TESTING = "abcxyz";
};
serviceConfig = {
Type = "simple";
ExecStart = "bash -c \"echo '$TESTING hello' >> /tmp/abcxyz\"";
Restart = "always";
RestartSec = 30;
};
wantedBy = [ "default.target" ];
};
};
};
};
However needing to test/compile this means running nixos-rebuild test which can take 10+ seconds on my machine (or 7s on a newly installed VM I just tried).
Is there some way I can more quickly test this container config independent from my entire host's Nixos config? For example building just the container config itself rather than the entire instance of this nixos config?
I've found that the nixos-rebuild command is a small shell script for example at https://github.com/NixOS/nixpkgs/blob/216f0e6ee4a65219f37caed95afda1a2c66188dc/nixos/modules/installer/tools/nixos-rebuild.sh
However after reading through it, I don't quite understand whats going on in terms of the relationship between this 'containers' unit and the general 'nixos config'.
NixOS Containers don't have a testing facility of their own (as of Nov 2020). They behave like normal NixOS systems for most intents and purposes. If you want to test your container, a normal NixOS test should be adequate.
Writing NixOS tests is documented in the NixOS manual. You can use the pkgs.nixosTest function to write your own tests outside of the Nixpkgs repo.
You could test either an approximation of the host system with your container, or just the container configuration as if it was the "top-level" NixOS system. I'd go with the latter, unless you need to test multiple containers in conjunction, or if you need to test interactions between host and container.
However to test the container definition building correctly, we can use pkg.nixos. For example, a nix expression for your container (that can be be built with the usual nix-build method):
{ a = let pkgs = import <nixpkgs> {};
in (pkgs.nixos
{
fileSystems."/".device = "x";
boot.loader.grub.enable = false;
systemd.services = {
finder-email-requests-2 = {
description = "";
enable = true;
environment = {
TESTING = "abcxyz";
};
serviceConfig = {
Type = "simple";
ExecStart = "bash -c \"echo '$TESTING hello' >> /tmp/abcxyz\"";
Restart = "always";
RestartSec = 30;
};
wantedBy = [ "default.target" ];
};
};
}).toplevel;
}
I'm trying to reference a secret file to run with newman under a sub directory of the workspace like so:
String integrationFile = "src/test/postman/integration.json"
String environmentFile = "src/test/postman/environment-dev.json"
String reportFile = "integrationReport.xml"
String reportArgs = "--reporters cli,junit --reporter-junit-export ${reportFile}"
node {
withCredentials([file(credentialsId: "${env.FILE_KEY}", variable: "FILE_PATH")]) {
String dataFile = "${env.FILE_PATH}"
dir('sub-dir') {
git branch: 'master',
credentialsId: "${config.GitHubKeyId}",
url: 'https://github.com/xxx/repo.git'
withEnv(["PATH=${tool 'nodejs-12.8.0'}/bin:${env.PATH}"]) {
try {
sh ("newman run \"${integrationFile}\" -e \"${environmentFile}\" --global-var \"baseUrl=${route}\" -d ${dataFile} ${reportArgs}")
} catch (error) {
throw error
} finally {
junit "${reportFile}"
}
}
}
}
}
}
But when I run the code above, Jenkins throws an error:
error: iteration data could not be loaded
ENOENT: no such file or directory, open '/var/jenkins_home/workspace/Platform'
The path looks to be truncated because when I run a pwd command before the node closure runs, the workspace should be:
/var/jenkins_home/workspace/Platform Management/JJob#2
My question is, why is Jenkins doing this? Do I need to format the variable of the secret another way? Or should I reference it differently?
I know the file exists because in another Jenkins pipeline that does not have a sub directory (dir("")), it works fine.
I see from the Jenkins docs about withCredentials shows that how the file is reference gets tricky when you move between directories, see here: https://www.jenkins.io/doc/pipeline/steps/credentials-binding/
Here are the things that I've tried:
${env.FILE_PATH}
${FILE_PATH}
$FILE_PATH
(all of the above with double and single quotes around the sh command)
Any help is appreciated,
Thanks!
Ok - after playing around alot more with it, I eventually added a double quote around the variable to keep the spaces. This SO helped me out: Jenkins coping with spaces in batch file arguments
In the newman command line script, I just had add double quotes within the param args, like so:
sh ("newman run \"${integrationFile}\" -e \"${environmentFile}\" --global-var \"baseUrl=${route}\" -d \"${dataFile}\" ${reportArgs}")