I'm trying to create a function in a Jenkins shared library that spits out the last commit hash of the parent branch of the current git branch.
ParentBranch = foo
ChildBranch(Forked from parent) = bar
What I want to get is the commit hash for the last commit in foo branch
I've written this script and tested it on my local machine
#! /bin/bash
PARENT_BRANCH=$(git show-branch -a | grep '\*' | grep -v `git rev-parse --abbrev-ref HEAD` | head -n1 | sed 's/.*\[\(.*\)\].*/\1/' | sed 's/[\^~].*//')
PARENT_COMMIT=$(git rev-parse $PARENT_BRANCH | cut -c-20)
printf $PARENT_COMMIT
And this works perfectly.
But when I put it in a function in Jenkins, it doesn't seem to do anything. I've tried to escape the special characters too.
def call() {
sh '''
cat <<EOF > gitBranch.sh
#! /bin/bash
PARENT_BRANCH="$(git show-branch | grep '*' | grep -v "$(git rev-parse --abbrev-ref HEAD)" | head -n1 | sed 's/.*\\[\\(.*\\)\\].*/\\1/' | sed 's/[\\^~].*//')"
PARENT_COMMIT="$(git rev-parse ${PARENT_BRANCH} | cut -c-20)"
printf "${PARENT_COMMIT}"
EOF
'''
}
In my Jenkins console output, when I cat the file gitBranch.sh, this is what I get
#! /bin/bash
PARENT_BRANCH=""
PARENT_COMMIT=""
printf ""
Nothing gets expanded.
What am I doing wrong please?
Your problem here is Jenkins (so I removed the non-Jenkins tags): Jenkins uses Groovy for commands, and Groovy does its own $ interpolation in strings. You just need to protect these particular dollar signs. See also Using "$" in Groovy and illegal string body character after dollar sign : either escape a literal dollar sign "\$5" or bracket the value expression.
(I am not a Jenkins or Groovy expert and I have no love at all for Jenkins, but this is something anyone using it should know, because it's pretty evil that way.)
Related
After a lengthy pipe which ends with a grep, I correctly end up with a set of matching absolute paths/files and match string separated by a comma delimiter for each. I want to tag each file with its match string. Complicated also in that the path has spaces but there is none between the delimiter and the preceding and succeeding characters.
I need to be able to deal with an absolute path rather than just the filename within the directory. The match strings are space_free but the filename might not be:
So by way of example, the output of the pipe might look like:
pipe1 | pipe2 |
outputs
/Users/bloggs/Directory One/matched_file.doc,attributes_0001ABC
/Users/bloggs/Directory One/matched_file1.doc,attributeY_2
/Users/bloggs/Directory One/match_file_00x.doc,Attribute_00201
/Users/bloggs/Directory One/matching file 2.doc,attribute_0004
I want to tag each using something which will probably include:
tag --add "$attribute" "$file"
Where attribute refers to the match string eg "Attribute_00201"
Normally I'd just say eg:
tag --add Attribute_00201 /Users/bloggs/Directory\ One/match_file_00x.doc
At this point I am stuck how to parse each line ideally via another pipe and to deal with spaces correctly and execute the tag command. Grateful for any help
So I'm looking for a new pipe, pipe3 to execute or give me the correctly formatted tag command:
pipe1 | pipe2 | pipe3
delivers eg
tag --add Attribute_00201 /Users/bloggs/Directory\ One/match_file_00x.doc
etc
etc
This seems to work
| tee >(cut -f2 -d","| sed 's/^/tag --add /' > temp_out.txt) >(cut -d"," -f1 | sed -e 's/[[:space:]]/\\ /g' > temp_out1.txt) > /dev/null && paste -d' ' temp_out.txt temp_out1.txt > command.sh && chmod +x ./command.sh
I'm trying to perform a shell script from a groovy function loaded by a jenkins-pipeline to retrieve a zip file from an external location. I am building the address out in the function and passing it into the shell script via $. But I am getting a syntax error and I'm not sure why.
I've tried escaping the $ but dont think thats the correct approach here and my code has been coverted from triple single quotes (''') to triple double (""") so I can pass the variable in.
def DownloadBaseLineFromNexus(groupID, artifactID){
//add code for this method
def nexusLink = "${GetNexusLink()}/${GetNexusProdRepo()}/${groupID}/${artifactID}/"
sh """
# retrieving all available version from release repo to versionFile.xml
curl ${nexusLink} | grep "<a href=.*</a>" | grep "http" | cut -d'>' -f3 |cut -d'/' -f1 > versionFile.xml
# creating array from versionFile.xml
fileItemString=$(cat versionFile.xml |tr "\n" " ")
fileItemArray=($fileItemString)
# Finding maximum of array element
maxValue=`printf "%d\n" "${fileItemArray[#]}" | sort -rn | head -1`
# Download latest version artifact from nexus
curl -o ${(artifactID)}.zip ${(nexusLink)}/${(artifactID)}-$maxValue.zip
# Unzip the tool
unzip ${(artifactID)}.zip
"""
}
the results I get are:
Script1.groovy: 28: illegal string body character after dollar sign;
solution: either escape a literal dollar sign "\$5" or bracket the value expression "${5}" # line 28, column 22.
curl "${nexusLink}" | grep "" | grep "http" | cut -d'>' -f3 |cut -d'/' -f1 > versionFile.xml
You have to add escape characters like below:-
curl ${nexusLink} | grep \"<a href=.*</a>\" | grep \"http\" | cut -d'>' -f3 |cut -d'/' -f1 > versionFile.xml
I am running into an issue where I am trying to run the following command:
aws ecs list-task-definitions | grep Foo-Task-Testing | awk -F '/' '{print $2}'
This returns exactly what I am looking for which is just the task definition name.
When running the command in the CLI with just grep i get this:
"arn:aws:ecs:us-east-1:xxxxxxxxxx:task-definition/Foo-Task-Testing-TaskDefinition-OYBZ78KBUI57:1",
When including Awk, I get:
Foo-Task-Testing-TaskDefinition-OYBZ78KBUI57:1"
However, when I try to add this to my Jenkins pipeline:
ecsTaskDefinitionName = Foo-Task-Testing
ecsTaskDefinition = sh(returnStdout: true, script: "aws ecs list-task-definitions | grep $ecsTaskDefinitionName | awk -F '/' '{print \$2}'").trim()
I always get this error message:
/home/jenkins/workspace/foo_test_PR-828#tmp/durable-a5ce4670/script.sh: 1: /home/jenkins/workspace/foo_test_PR-828#tmp/durable-a5ce4670/script.sh: Syntax error: Unterminated quoted string
I have a feeling this has to do with how I am using Awk in Groovy but I can't seem to find enough examples online to confirm this. Can anyone either provide a way of doing this in Groovy w/o using Awk or any experienced Groovy programmers can tell me the correct way of passing Awk?
You can avoid the need for awk with grep -o:
... | grep -o Foo-Task-Testing.*
returns
Foo-Task-Testing-TaskDefinition-OYBZ78KBUI57:1
(-o only returns the match, .* greedily matches everything after)
I am trying to get the current branch name from where I am pushing the code to the branch in the remote server .
Example: My local branch name is s095_Development and I am pushing the code from my local to master .
I want to get the local branch name in my pre-receive hook.
current_branch=$(git symbolic-ref HEAD | sed -e 's,.*/\(.*\),\1,')
current_branch=$(git rev-parse --abbrev-ref HEAD | cut -d'_' -f1 | cut -d'/' -f3 )
current_branch=$(git name-rev --name-only HEAD | cut -d'_' -f1 | cut -d'/' -f3 )
I have used all the above but nothing seems to work for me - I am not able to get the current branch name.
Use the following command in the directory where your project is.
git branch
When using git push, it pushes to that branch but in the remote server. Thus, if you are currently in master, you'll be pushing to remotes/origin/master.
This is working for me using PHP executing a pre-receive hook on server
#!/usr/bin/php
<?php
$stdin = trim(fgets(STDIN));
$branchName = end(explode('/', end(explode(' ', $stdin))));
I have created spec/lint/rubocop_spec.rb which runs Rubocop style checker on the files changed between current branch and master. This works when I test locally but not when the test run on the build server Circle.ci.
I suspect it is because only the branch in question is downloaded, so it does not find any differences between master. Is there a better way than git co master && git pull origin master?
Can I query the Github API perhaps to get the files changed listed?
require 'spec_helper'
describe 'Check that the files we have changed have correct syntax' do
before do
current_sha = `git rev-parse --verify HEAD`.strip!
files = `git diff master #{current_sha} --name-only | grep .rb`
files.tr!("\n", ' ')
#report = 'nada'
if files.present?
puts "Changed files: #{files}"
#report = `rubocop #{files}`
puts "Report: #{#report}"
end
end
it { #report.match('Offenses').should_not be true }
end
You don't have to use github api, or even ruby (unless you want to wrap the responses) you can just run:
git fetch && git diff-tree -r --no-commit-id --name-only master#\{u\} head | xargs ls -1 2>/dev/null | xargs rubocop --force-exclusion
see http://www.red56.uk/2017/03/26/running-rubocop-on-changed-files/ for longer write-up of this
I fixed it by querying api.github.com.
This will run rubocop on all files that has been changed between current_sha and the master branch.
require 'spec_helper'
describe 'Check that the files we have changed have correct syntax' do
before do
current_sha = `git rev-parse --verify HEAD`.strip!
token = 'YOUR GITHUB TOKEN'
url = 'https://api.github.com/repos/orwapp/orwapp/compare/' \
"master...#{current_sha}?access_token=#{token}"
files = `curl -i #{url} | grep filename | cut -f2 -d: | grep \.rb | tr '"', '\ '`
files.tr!("\n", ' ')
#report = 'nada'
if files.present?
puts "Changed files: #{files}"
#report = `rubocop #{files}`
puts "Report: #{#report}"
end
end
it { expect(#report.match('Offenses')).to be_falsey }
end
I found https://github.com/m4i/rubocop-git which works very well. However it works on your git diff (optionally with --cached) so it does not allow you to compare branches.
I don't have high enough reputation to comment on an answer, so I am posting an answer to add a refinement I found useful:
git fetch && git diff-tree -r --no-commit-id --name-only master#\{u\} HEAD | xargs ls -1 2>/dev/null | grep '\.rb$' | xargs bundle exec rubocop --force-exclusion
The addition of --force-exclusion makes RuboCop respect the Exclude declarations in its config file (here using the default ./.rubocop.yml). You put those declarations in for a reason, right?! ;)
Here's another alternative that compares the current branch to origin/master (should work with any repo hosting - just tried it on circleci with a bitbucket repo). It also passes a .rubocop.yml config file option (you can remove that part if you don't need it).
require 'spec_helper'
RSpec.describe 'Check that the files we have changed have correct syntax' do
before do
current_sha = 'origin/master..HEAD'
#files = `git diff-tree --no-commit-id --name-only -r #{current_sha} | grep .rb`
#files.tr!("\n", ' ')
end
it 'runs rubocop on changed ruby files' do
if #files.empty?
puts "Linting not performed. No ruby files changed."
else
puts "Running rubocop for changed files: #{#files}"
result = system "bundle exec rubocop --config .rubocop.yml --fail-level warn #{#files}"
expect(result).to be(true)
end
end
end
Original gist here: https://gist.github.com/djburdick/5104d15f612c15dde65f#gistcomment-2029606
One simpler solution:
git diff origin/master --name-only | xargs rubocop --force-exclusion
Explanation: I rarely have master up to date locally, but doing git fetch updates origin/master so I want to diff against that. I can't get the other proposed solutions with diff-tree and origin/master to work.
Maybe you could leverage CircleCI’s dynamic configuration feature.
There is a specific guide on how to execute specific workflows or steps based on which files are modified (https://circleci.com/docs/using-dynamic-configuration/#execute-specific-workflows-or-steps-based-on-which-files-are-modified).