How can I upload files to a HTTPS site using CURL?
The site is also password protected.
What's the command line used to upload for that particular site?
Yes, you can,
$ curl --user login:password --upload-file your.file.txt https://the.url/where/that/should/go
You may need to add --insecure if you don't want cURL to check the certificate chain.
Related
Is there any way to store or update a CSV file to One-drive after Jenkins build run? I want it to happen without any third-party plugin.
You can do this using OneDrive API and curl, e.g.:
curl https://graph.microsoft.com/v1.0/me/drive/root:/data.csv:/content -X PUT -d #data.csv -H "Authorization: bearer access_token_here"
One limitation is that your csv file couldn't be more than 4MB.
I am trying to trigger a Jenkins build by post request from my Windows computer, using the following commands.
First, I obtain the crumb using...
curl http://JENKINS-URL/crumbIssuer/api/xml?xpath=//crumb
The response is this...
<crumb>string-of-digits<\crumb>
I then use the command
curl -u USERNAME:PASSWORD -X POST -H "Jenkins-Crumb:string-of-digits" http://JENKINS-URL/job/my-job/buildWithParameters?token=my-token
The username is correct, the password is correct, the crumb is exactly the string of digits that I got from the first command (everything between the crumb> at the beginning and <\crumb> at the end), the token matches the token I have specified in Jenkins. But still, I am getting the response
Error 403 No valid crumb was included in the request
In Configure Global Security -> CSRF Protection, I also have "Enable proxy compatibility" checked and am using "Default Crumb Issuer".
Does it look like I am missing anything here? Forgetting any steps? Improperly formatted commands? Anything else you might be able to think of?
I have followed the steps and formatted my command like the solution here as well, and still no luck..
https://linuxacademy.com/community/posts/show/topic/28964-no-valid-crumb-was-included-in-the-request
In addition to this, I have also tried saving the cookies from the first request to get the crumb, and then I pass the file I saved the cookies to into the second curl command, see below for the command. Still I am getting the 403 No valid crumb...
Here are the commands using cookies...
wget --keep-session-cookies --save-cookies cookies.txt --auth-no-challenge --user admin --password my_password -q --output-document - http://JENKINS-URL/crumbIssuer/api/xml?xpath=//crumb
curl --cookie cookies.txt -u admin:my_password -H "JenkinsCrumb: string-of-digits-from-stdout" -X POST http://JENKINS-URL/job/my-job/buildWithParameters?token=my-token
I have finally discovered the answer. I thought that I could specify my actual account password, but it turns out that I needed to use an API token to authenticate. Using the API token instead of my password on the above commands with cookies allowed me to trigger my build remotely.
enter image description here
I finally resolved this issue by selecting the jenkins own user database in security realm
As part of an automated tests suite I have to use OpenShift's REST APIs to send commands and get OpenShift's status. To authenticate these API calls I need to embed an authorization token in every call.
Currently, I get this token by executing the following commands with ssh on the machine where OpenShift is installed:
oc login --username=<uname> --password=<password>
oc whoami --show-token
I would like to stop using the oc tool completely and get this token using HTTP calls to the APIs but am not really able to find a document that explains how to use it. If I use the option --loglevel=10 when calling oc commands I can see the HTTP calls made by oc when logging in but it is quite difficult for me to reverse-engineer the process from these logs.
Theoretically this is not something specific to OpenShift but rather to the OAuth protocol, I have found some documentation like the one posted here but I still find it difficult to implement without specific examples.
If that helps, I am developing this tool using ruby (not rails).
P.S. I know that normally for this type of job one should use Service Account Tokens but since this is a testing environment the OpenShift installation gets removed and reinstalled fairly often. This would force me to re-create the service account every time with the oc command line tool and again prevent me from automatizing the process.
I have found the answer in this GitHub issue.
Surprisingly, one curl command is enough to get the token:
curl -u joe:password -kv -H "X-CSRF-Token: xxx" 'https://master.cluster.local:8443/oauth/authorize?client_id=openshift-challenging-client&response_type=token'
The response is going to be an HTTP 302 trying to redirect to another URL. The redirection URL will contain the token, for example:
Location: https://master.cluster.local:8443/oauth/token/display#access_token=VO4dAgNGLnX5MGYu_wXau8au2Rw0QAqnwq8AtrLkMfU&expires_in=86400&token_type=bearer
You can use token or combination user/password.
To use username:password in header, you can use Authorizartion: Basic. The oc client commands are doing simple authentication with your user and password in header. Like this
curl -H "Authorization: Basic <SOMEHASH>"
where the hash is exactly base64 encoded username:password. (try it with echo -n "username:password" | base64).
To use token, you can obtain the token here with curl:
curl -H Authorization: Basic $(echo -n username:password | base64)" https://openshift.example.com:8443/oauth/authorize\?response_type\=token\&client_id\=openshift-challenging-client
But the token is replied in the ugly format format. You can try to grep it
... | grep -oP "access_token=\K[ˆ&]*"
You need to use the correct url for your oauth server. In my case, I use openshift 4.7 and this is the url:
https://oauth-openshift.apps.<clustername><domain>/oauth/authorize\?response_type\=token\&client_id\=openshift-challenging-client
oc get route oauth-openshift -n openshift-authentication -o json | jq .spec.host
In case you are using OpenShift CRC:
Then the URL is: https://oauth-openshift.apps-crc.testing/oauth/authorize
Command to get the Token:
curl -v --insecure --user developer:developer --header "X-CSRF-Token: xxx" --url "https://oauth-openshift.apps-crc.testing/oauth/authorize?response_type=token&client_id=openshift-challenging-client" 2>&1 | grep -oP "access_token=\K[^&]*"
Note:
2>&1 is required, because curl writes to standard error
--insecure: because I have not set up TLS certificate
Adjust the user and password developer as needed (crc developer/developer is standard user in crc, therefore good for testing.)
Token is per default 24h vaild
Export the Token to an environment Variable
export TOKEN=$(curl -v --insecure --user developer:developer --header "X-CSRF-Token: xxx" --url "https://oauth-openshift.apps-crc.testing/oauth/authorize?response_type=token&client_id=openshift-challenging-client" 2>&1 | grep -oP "access_token=\K[^&]*")
And Use the token then in, e.g., oc login:
oc login --token=$TOKEN --server=https://api.crc.testing:6443
I want the user to upload to our server and have our server then upload the file to a specific folder in Box using the API. How can I do this? What credentials should I use?
curl https://app.box.com/api/oauth2/token \
-d 'grant_type=client_credentials&client_id=CLIENT_ID&client_secret=SECRET'
returns
{"access_token":"TOKEN","expires_in":3793,"restricted_to":[],"token_type":"bearer"}
curl https://upload.box.com/api/2.0/files/content \
-H "Authorization: Bearer TOKEN" -X POST -F \
file=#package.json -F folder_id=3306197480
returns
{"type":"error","status":404,"code":"not_found","context_info":{"errors":[{"reason":"invalid_parameter","name":"parent","message":"Invalid value 'd_3306197480'. 'parent' with value 'd_3306197480' not found"}]},"help_url":"http://developers.box.com/docs/#errors","message":"Not Found","request_id":"1157604954550c7c754b9c9"}
It sounds like the API is returning a 404 because the user associated with the access token doesn't have permissions to see the folder you're uploading to. Your server will need to authenticate as a user that is collaborated on that folder instead of authenticating as the user uploading the file.
When adding a URL into a web browser, I get the usual prompt to open the XML file and view it. However, when I use the same URL within a Curl batch file it only appears to download the login aspx page.
//stuff/stuff/Report.aspx?Report=All_Nodes_IP_Report&DataFormat=XML&AccountID=<UID>&Password=<password>
My batch file looks like this:
curl -L "//stuff/stuff/Report.aspx?Report=All_Nodes_IP_Report&DataFormat=XML&AccountID=<UID>&Password=<Password>" -o "local.xml" -v
pause
What am I doing wrong? There's no proxy server between me and the report URL..? The web site is https but I can't include that as the validation checker keeps moaning at me :)
why use CURL when you can use one application called MGET that i create.
Download Link:
http://bit.ly/1i1FpGE
Syntax of the command:
MGET //stuff/stuff/Report.aspx?Report=All_Nodes_IP_Report&DataFormat=XML&AccountID=<UID>&Password=<Password> local.xml
And if you want to use HTTPS do it, for best experience use HTTP