Must not run with sudo - devops

Hi I am new in github actions and I am trying to create a CICD pipline using Github action. I am using a digital ocean droplet as my server and I am trying to create a runner as said in github->settings->actions
When I wrote the following command
./config.sh --url https://github.com/basobaasnepal/BasobaasWeb --token DFGFSDF234sf3fg45hd
I got this:
Must not run with sudo
I tried to change the from root user to non root user but didn't work. I also tried export {AGENT_ALLOW_RUNASROOT="1"} bur

Are you setting the right variable? I think you can set the RUNNER_ALLOW_RUNASROOT variable to get by this problem using export RUNNER_ALLOW_RUNASROOT=1, or you can provide it directly to the command:
RUNNER_ALLOW_RUNASROOT="1" ./config.sh --url https://github.com/basobaasnepal/BasobaasWeb --token DFGFSDF234sf3fg45hd

Related

How can I add a secret in GitHub Actions using my terminal?

I know I can add secrets using the GUI. I also know it's possible to use the new GitHub Actions Secrets API
What I need is a simple way to add, update, delete and list secrets in my GitHub repository, using my terminal only. By using a CLI, it will ease my day-to-day work, and also allow me to rotate secrets in my GitHub Actions CI/CD processes.
Is there a CLI or anything like that?
I've created a simple CLI that can help you achieve that - https://github.com/unfor19/githubsecrets
This CLI is based on the official GitHub Actions Secrets API
Install with pip
$ pip install githubsecrets
...
$ ghs --help
...
Or use with Docker; you must supply all arguments, prompts are not available in Docker mode
macOS and Linux
$ docker run -v $HOME:/root unfor19/githubsecrets --help
Usage: ghs [OPTIONS] COMMAND [ARGS]...
Windows
$ docker run --rm -v c:/Temp:/root unfor19/githubsecrets --help
Usage: ghs [OPTIONS] COMMAND [ARGS]...
Another one, in ruby:
$ gem install secret_hub
also using the official Github API, and supports bulk operations.
https://github.com/dannyben/secret_hub
Full disclosure: I wrote it, following this discussion about the missing feature in GitHub to have organization-level secrets.

Google Endpoints YAML file update: Is there a simpler method

When using Google Endpoints with Cloud Run to provide the container service, one creates a YAML file (stagger 2.0 format) to specify the paths with all configurations. For EVERY CHANGE the following is what I do (based on the documentation (https://cloud.google.com/endpoints/docs/openapi/get-started-cloud-functions)
Step 1: Deploying the Endpoints configuration
gcloud endpoints services deploy openapi-functions.yaml \
--project ESP_PROJECT_ID
This gives me the following output:
Service Configuration [CONFIG_ID] uploaded for service [CLOUD_RUN_HOSTNAME]
Then,
Step 2: Download the script to local machine
chmod +x gcloud_build_image
./gcloud_build_image -s CLOUD_RUN_HOSTNAME \
-c CONFIG_ID -p ESP_PROJECT_ID
Then,
Step 3: Re deploy the Cloud Run service
gcloud run deploy CLOUD_RUN_SERVICE_NAME \
--image="gcr.io/ESP_PROJECT_ID/endpoints-runtime-serverless:CLOUD_RUN_HOSTNAME-CONFIG_ID" \
--allow-unauthenticated \
--platform managed \
--project=ESP_PROJECT_ID
Is this the process for every API path change? Or is there a simpler direct method of updating the YAML file and uploading it somewhere?
Thanks.
Based on the documentation, yes, this would be the process for every API path change. However, this may change in the future as this feature is currently on beta as stated on the documentation you shared.
You may want to look over here in order to create a feature request to GCP so they can improve this feature in the future.
In the meantime, I could advise to create a script for this process as it is always the same steps and doing something in bash that runs these commands would help you automatize the task.
Hope you find this useful.
When you use the default Cloud Endpoint image as described in the documentation the parameter --rollout_strategy=managed is automatically set.
You have to wait up to 1 minutes to use the new configuration. Personally it's what I observe in my deployments. Have a try on it!

Scanning Rest API's through OWASP zap inside a docker environment

I set an Azure devops CI/CD build that will start a vm where Owasp Zap is running as a proxy and where the Owasp zap Azure devops task will run on a target url and copy my report in an Azure Storage.
Followed this guy's beautiful tutorial: https://kasunkodagoda.com/2017/09/03/introducing-owasp-zed-attack-proxy-task-for-visual-studio-team-services/
(also the guy who created the Azure devops task)
All well and good but recently I wanted to use an REST Api as a target url. The Owasp zap task in azure devops doesn't have the ability. Even asked the creator (https://github.com/kasunkv/owasp-zap-vsts-task/issues/30#issuecomment-452258621) and he also didn't think this is available through the Azure devops task and only through docker.
On my next quest I am now trying to get it running inside a docker image. (Firstly inside Azure devops but that wasn't smooth https://github.com/zaproxy/zaproxy/issues/5176 )
And finally getting on this tutorial (https://zaproxy.blogspot.com/2017/06/scanning-apis-with-zap.html)
Where I am trying to run a docker image with the following steps:
--- docker pull owasp/zap2docker-weekly
--running the container
-------command : docker run -v ${pwd}:/zap/wrk/:rw -t owasp/zap2docker-weekly zap-api-scan.py -t https://apiurl/api.json -f openapi -z "-configfile /zap/wrk/options.prop"
------- options.prop file
-config replacer.full_list\(0\).description=auth1 \
-config replacer.full_list\(0\).enabled=true \
-config replacer.full_list\(0\).matchtype=REQ_HEADER \
-config replacer.full_list\(0\).matchstr=Authorization \
-config replacer.full_list\(0\).regex=false \
-config replacer.full_list\(0\).replacement=Bearer xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
But This scans only the root url not every URL. As I am typing this question i tried to download the json file from the root and running the docker run command with passing the json file with the -t I am getting number of imported url's : what seems to be everything. But this seems to freeze inside powershell.
Which step do i miss to get a full recursive scan on my rest api ?
Any one some ideas or some help pls ?
Firstly, your property file format is wrong. You only need the '-config' and '\'s if you set the options directly on the command line. In the property file you should have:
replacer.full_list(0).description=auth1
replacer.full_list(0).enabled=true
replacer.full_list(0).matchtype=REQ_HEADER
replacer.full_list(0).matchstr=Authorization
replacer.full_list(0).regex=false
replacer.full_list(0).replacement=Bearer xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Secondly, what does https://apiurl/api.json return and have you checked you can access it from within your docker container?
Try running
curl https://apiurl/api.json
and see what you get.

Cannot start fabric-ca server natively

I have been following this for setting up a fabric-ca server in my network of 2 organizations, 4 peers (2 in each).
I have two questions:
In the documentation, it says that we can start server locally. When I try to do the same, I'm not able to do it and getting the following error :
fabric-ca-server: command not found . So I tried using a docker image and the server now works as a docker image.
Now when I try to run the fabric-ca-client command, it cannot find the client configuration in the fabric-ca-client home. The FABRIC_CA_HOME environment variable is set as `/etc/hyperledger/fabric-ca-server' in the container. I'm confused as to what I might be missing here.
If you followed the instructions, then the fabric-ca-server executable will be under $GOPATH/bin, you will need to add this to your PATH, via export PATH=$PATH:$GOPATH/bin. Remember to also set FABRIC_CA_HOME.
Assuming you're also using the client natively, it should also be under $GOPATH/bin. In a separate terminal, set FABRIC_CA_HOME to a different path. Then you can enroll the admin user, for example: fabric-ca-client enroll -u http://admin:password#localhost:7054.
The issue is coming because you have not set the GOPATH path.
After cloning the CA repo set the GOPATH to the given directory.
For setting up GOPATH:
(Ubuntu)
If you don’t set a GOPATH, the default will be used.
You have to add $GOPATH/bin to your PATH to execute any binary installed in $GOPATH/bin, or you need to type $GOPATH/bin/the-command.
Add this to your ~/.bash_profile
export PATH=$GOPATH/bin:$PATH
Current GOPATH command:
go env GOPATH
Changing the GOPATH command:
export GOPATH=$HOME/your-desired-path
So. change the 'your-desired-path' to your Fabric-CA repo directory.
You will be facing issue with certain versions of golang. Set the flags explicitly using CGO_LDFLAGS_ALLOW='-Wl,--no-as-needed'
Reference -
Error while running make command using Fabric 1.0.6 after all the 15 steps
https://github.com/golang/go/issues/23739
➜ fabric-ca git:(release-1.0) go get -u github.com/hyperledger/fabric-ca/cmd/...
go build github.com/hyperledger/fabric-ca/vendor/github.com/miekg/pkcs11: invalid flag in #cgo LDFLAGS: -I/usr/local/share/libtool
➜ fabric-ca git:(release-1.0) export CGO_LDFLAGS_ALLOW='-Wl,--no-as-needed'
➜ fabric-ca git:(release-1.0) make

How to "Unlock Jenkins"?

I am installing Jenkins 2 on windows,after installing,a page is opened,URL is:
http://localhost:8080/login?from=%2F
content of the page is like this:
Question:
How to "Unlock Jenkins"?
PS:I have looked for the answer in documentation and google.
Starting from version 2.0 of Jenkins you may use
-Djenkins.install.runSetupWizard=false
to prevent this screen.
Accroding to documentation
jenkins.install.runSetupWizard - Set to false to skip install wizard. Note that this leaves Jenkins unsecured by default.
Development-mode only: Set to true to not skip showing the setup wizard during Jenkins development.
More details about Jenkins properties can be found on this Jenkins Wiki page.
Check https://wiki.jenkins-ci.org/display/JENKINS/Logging to see where Jenkins is logging its files.
e.g. for Linux, use the command: less /var/log/jenkins/jenkins.log
And scroll down to the part: "Jenkins initial setup is required. An admin user has been created ... to proceed to installation:
[randompasswordhere] <--- Copy and paste
Linux
By default logs should be made available in /var/log/jenkins/jenkins.log, unless customized in /etc/default/jenkins (for *.deb) or via /etc/sysconfig/jenkins (for */rpm)
Windows
By default logs should be at %JENKINS_HOME%/jenkins.out and %JENKINS_HOME%/jenkins.err, unless customized in %JENKINS_HOME%/jenkins.xml
Mac OS X
Log files should be at /var/log/jenkins/jenkins.log, unless customized in org.jenkins-ci.plist
open file: e:\Program Files (x86)\Jenkins\secrets\initialAdminPassword
copy content file: 47c5d4f760014e54a6bffc27bd95c077
paste in input: http://localhost:8080/login?from=%2F
DONE
Some of the above instructions seem to have gone out of date. As of the released version 2.0, creating the following file will cause Jenkins to skip the unlock screen:
${JENKINS_HOME}/jenkins.install.InstallUtil.lastExecVersion
This file must contain the string 2.0 without any line terminators. I'm not sure if this is required but Jenkins also sets the owner/group to be the same as the Jenkins server, so that's probably a good thing to mimic as well.
I did not need to create the upgraded or .last_exec_version files.
I assume you were running jenkins.war manually with java -jar jenkins.war, then all logging information by default is output to standard out, just type the token to unlock jenkins2.0.
If you were not running jenkins with java -jar jenkins.war, then you can always follow this Official Document to find the correct log location.
Open your terminal and type code below to find all the containers.
docker container list -a
You will find jenkinsci/blueocean and/or docker:dind if not than
docker container run --name jenkins-docker --rm --detach ^
--privileged --network jenkins --network-alias docker ^
--env DOCKER_TLS_CERTDIR=/certs ^
--volume jenkins-docker-certs:/certs/client ^
--volume jenkins-data:/var/jenkins_home ^
--volume "%HOMEDRIVE%%HOMEPATH%":/home ^
docker:dind
and
docker container run --name jenkins-blueocean --rm --detach ^
--network jenkins --env DOCKER_HOST=tcp://docker:2376 ^
--env DOCKER_CERT_PATH=/certs/client --env DOCKER_TLS_VERIFY=1 ^
--volume jenkins-data:/var/jenkins_home ^
--volume jenkins-docker-certs:/certs/client:ro ^
--volume "%HOMEDRIVE%%HOMEPATH%":/home ^
--publish 8080:8080 --publish 50000:50000 jenkinsci/blueocean
run command
docker run jenkinsci/blueocean
or
docker run docker:dind
Copy and Paste the secret key.
One method to prevent the installation wizard is to do the following in $JENKINS_HOME:
Create an empty file named .last_exec_version
Create a file named upgraded
If left empty, a banner will prompt you to "upgrade" to 2.0 (which just means install a bunch of new plugins like Pipeline)
If the contents of that file is 2.0, you'll receive no banner and it will act like an regular old Jenkins install
Remember, this wizard is in place to prevent unauthorized access to Jenkins during setup. However, bypassing this wizard can be useful if, for example, you want to deploy automated installations of Jenkins with something like Ansible/Puppet/etc.
This was tested against Jenkins 2.0-beta-1 – so these instructions may not work in future beta or stable releases.
In the mac use:
sudo more /Users/Shared/Jenkins/Home/secrets/initialAdminPassword
I have seen a lot of response to the question, most of them were actually solution to it but they solve the problem when jenkins is actually run as a standalone CI without Application container using the command:
java -jar jenkins.war
But when running on Tomcat as it is the case in this scenario, Jenkins logs are displayed on the catalina logs since the software is running on a container.
So you need to go to the logs folder:
C:\Program Files\tomcat_folder\Tomcat 8.5\logs\catalina.log
in my own case. Just scroll almost to the middle to look for a generated password which is essentially a token and copy and paste it to unlock jenkins.
I hope this fix your problem.
Step 1: Open the terminal on your mac
Step 2: Concatenate or Paste
sudo cat **/Users/Shared/Jenkins/Home/secrets/initialAdminPassword**
Step 3: It will ask for password, type your mac password and enter
Step 4: A key would be generated.
Step 5: Copy and paste the security token in Jenkins
Step 6: Click continue
I found the token in the following file in the installation dir:
<jenkins install dir>\users\admin\config.xml
and the element
<jenkins.install.SetupWizard_-AuthenticationKey>
<key> THE KEY </key>
</jenkins.install.SetupWizard_-AuthenticationKey>
You might see it in the catalina.out. I installed Jenkins war in tomcat and I can see this in the catalina.out
The below method does not work anymore on 2.42.2
Create an empty file named .last_exec_version
Create a file named upgraded
If left empty, a banner will prompt you to "upgrade" to 2.0 (which just means install a bunch of new plugins like Pipeline)
If the contents of that file is 2.0, you'll receive no banner and it will act like an regular old Jenkins install
mostly jenkins will show you the path for initialAdminPassword if you dont find it there, then you have to check jenkins logs
in log you will see
05-May-2017 01:01:41.854 INFO [Jenkins initialization thread] jenkins.install.SetupWizard.init
Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:
7c249e4ed93c4596972f57e55f7ff32e
This may also be found at: /opt/tomcat/.jenkins/secrets/initialAdminPassword
Use a lil shortcut to get to the folder:
cmd + shift + g
then insert /Users/Shared/Jenkins
there u can see the secrets folder - probably shows that it's locked.
to unlock it: right click on the folder and click info + click on the lock at the bottom. now u can change the rights shown at the bottom of the window
hope that helped :)
Greetings, Stefanie ^__^
If unable to find Jenkins password in the location C:\Windows\System32\config\systemprofile\.jenkins\secrets\initialAdminPassword
by installing Jenkins generic war on Tomcat server, try below
Solution:
Set environmental variable JENKINS_HOME to your jenkins path say
JENKINS_HOME ="C:/users/username/apachetomcat/webapps/jenkins"
Place Jenkins.war in the webapp folder of Tomcat and start Tomcat,
initial admin password gets generated in the path
C:\Program Files (x86)\Apache Software Foundation\Tomcat 9.0\webapps\jenkins\secrets\initialAdminPassword
Yet another way to bypass the unlock screen is to copy the UpgradeWizard state to the InstallUtil last execution version, add an install.runSetupWizard file with the contents 'false', and update the config.xml installStateName from NEW to RUNNING.
docker exec -it jenkins bash
sed -i s/NEW/RUNNING/ /var/jenkins_home/config.xml
echo 'false' > /var/jenkins_home/jenkins.install.runSetupWizard
cp /var/jenkins_home/jenkins.install.UpgradeWizard.state /var/jenkins_home/jenkins.install.InstallUtil.lastExecVersion
exit
docker restart jenkins
For reference, this is the command I use to run jenkins:
docker run --rm --name jenkins --network host -u root -d -v jenkins:/var/jenkins_home -v /var/run/docker.sock:/var/run/docker.sock jenkinsci/blueocean:1.16.0
You will also want to update the config with the Root URL:
echo "<?xml version='1.1' encoding='UTF-8'?><jenkins.model.JenkinsLocationConfiguration><jenkinsUrl>http://<IP>:8080/</jenkinsUrl></jenkins.model.JenkinsLocationConfiguration>" > jenkins.model.JenkinsLocationConfiguration.xml
exit
docker restart jenkins
In case, if you installed/upgraded new versions of jenkins and unable to find admin credentials on server then, ...
if you are using old version of jenkins and on the top of it you are trying to reinstall/upgrade new version of jenkins then,
the files under "JENKINS_HOME", namely -
${JENKINS_HOME}/jenkins.install.InstallUtil.lastExecVersion
${JENKINS_HOME}/jenkins.install.UpgradeWizard.state
will cause jenkins to skip the unlock (or admin credentials screen) and webpage directly ask you for username and password. even on server you will not able to find "${JENKINS_HOME}/secrets/initialAdminPassword".
In such case, don't get panic. just try to use old admin user creds in newly installed/upgraded jenkins server.
In simple language, if you have admin creds as admin/admin in old version of jenkins server then, after upgrading jenkins server, the new server won't ask you set password for admin user again. in fact it will use old creds only.
I have found the password in C:\Program Files\Jenkins\jenkins.err. Open jenkins.err text file and scroll down, and you will find the password.
Go to C:\Program Files (x86)\Jenkins\secrets
then with notepad ++ open file initail Admin Password and paste its content.
thats it
-->if you are using linux machine, then login as root user: sudo su
-->then go to the below path: cd /var/lib/jenkins/secrets
-->just view the IntialAdminPassword file ,you can see the secret key.
-->paste the secret key into jenkins window,it will be unlocked.
https://issues.jenkins-ci.org/browse/JENKINS-35981
Try this %2Fjenkins%2F instead of %2Fjenkins in the browser
Open the terminal on your mac and open new window (command+T)
Paste sudo cat /Users/Shared/Jenkins/Home/secrets/initialAdminPassword
It will ask for password, type your password(i gave my mac password, i haven't check if any other password would work) and enter
A key would be generated.
Copy the key and paste it where it asks you to enter admin password
click continue
The problem can be fixed in latest version: mine is 2.4. The error comes because of %2fjenkins%2f in URL. The previous version was coming with %2fjenkins and the same error used to come. They have resolved the issue, but the URL has been changed from %2fjenkins to %2fjenkins%. So as a summary in the URL currently %2fjenkins% is coming. before passing the admin password change it to %2fjenkins. Along with that add a .last_exec_version empty file.
If someone chooses running Jenkins as a Docker container, may face the same problem with me.
Because accessing-the-jenkins-blue-ocean-docker-container is quite different,
Common problem is /var/lib/jenkins/secrets: No such file or directory
You need to access through Docker, the link Jenkins provide is quite helpful.
Except <docker-container-name> maybe not specified, then you may need to use the container ID.
After
docker exec -it jenkins-blueocean bash
or
docker exec -it YOUR_JENKINS_CONTAINER_ID bash
The /var/lib/jenkins/secrets/initialAdminPassword would be accessible.
The password would be there.
I have setup Jenkins using Brew, But when I restarted Mac Jenkins was asking for initialAdminPassword(The screenshot attached in question)
And the problem was it was not generated under sercret directory.
So I'd found the Jenkins process which was running on port: 8080 using: $ sudo lsof -i -n -P | grep TCP and killed it using $ sudo kill 66(66 was process id).
Then I downloaded the latest jenkins .war file from: https://jenkins.io/download/
And executed command: $ java -jar jenkins.war (Make sure you are in jenkins.war directory).
And that's it everything is working fine.
This works well when you are stuck with Docker on Windows and are using Git-Bash
Presuming something like:
# docker run --detach --publish 8080:8080 --volume jenkins_home:/var/jenkins_home --name jenkins jenkins/jenkins:lts
Execute to get the Container ID, for example "d56686cb700d"
# docker ps -l
Now tell Docker to return the password written in the logs for that Container ID:
# docker logs d56686cb700d 2>&1 | grep -A5 -B5 Admin
2>&1 redirects stderr to stdout
-A5 includes 5 lines AFTER the line with "Admin" in it
-B5 includes 5 lines BEFORE the line with "Admin" in it
Output example:
Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:
47647383733f4387a0d53c873334b707
This may also be found at: /var/jenkins_home/secrets/initialAdminPassword
*************************************************************
*************************************************************
*************************************************************
I found it under below directory. Full issue detail https://github.com/jenkinsci/ibm-security-appscansource-scanner-plugin/issues/2
C:\Windows\SysWOW64\config\systemprofile\AppData\Local\Jenkins\.jenkins
Open jenkins.err file in C:\Program Files\Jenkins\.
In that file check for a hash key after this line
Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:
And paste it there in the jenkins prompt. Worked for me.
To solve this problem for docker container in Ubuntu 18.04.5 LTS (Bionic Beaver) - Ubuntu Releases
1- connect to your docker server or ubuntu server witch ssh or other method
2- run sudo docker ps
3- copy the container name parameter ("NAMES")
4- run sudo docker logs "your_parameters_NAMES_VALUES"
5- Find the folowing sentence "Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:" and copy the password

Resources