We are trying to run a command wget -O xyz.xls --user=COldPolar --password=GlacierICe --ignore-length=on "http://Colder.near.com:8080/sr/jira.issueviews:searchrequest-excel-current-fields/temp/SearchRequest.xls?&runQuery=true(jqlQuery=project%3DCCD)&tempMax=1000"
This is returning a 3kb output
If we open IE and use the following "http://Colder.near.com:8080/sr/jira.issueviews:searchrequest-excel-current-fields/temp/SearchRequest.xls?&runQuery=true(jqlQuery=project%3DCCD)&tempMax=1000" This allows us to save a 1.7MB file. Please advise how to get the wget to work
If you can use cURL you can do:
curl -o xyz.xls -u COldPolar:GlacierICe 'http://Colder.near.com:8080/sr/jira.issueviews:searchrequest-excel-current-fields/temp/SearchRequest.xls?&runQuery=true(jqlQuery=project%3DCCD)&tempMax=1000'
What I managed to get wget to work was to do this first:
wget --save-cookies cookies.txt --post-data 'os_username=COldPolar&os_password=GlacierICe&os_cookie=true' http://Colder.near.com:8080/login.jsp
And then:
wget -O xyz.xls --load-cookies cookies.txt "http://Colder.near.com:8080/sr/jira.issueviews:searchrequest-excel-current-fields/temp/SearchRequest.xls?&runQuery=true(jqlQuery=project%3DCCD)&tempMax=1000"
One of the things to watch out for is a self-signed certificate. You can rule this out by running with --no-check-certificate.
Related
I've already tried pip install wget in my cmd, which reads
>pip install wget
Requirement already satisfied: wget in c:\users\user\...\python\python38-32\lib\site-packages (3.2)
however when I try the command in git bash, it keeps showing
$ wget
bash: wget: command not found
I've made sure both the python file and the git file are in PATH.
What am I doing wrong here?
If you would like to use curl on Git Bash, here is an example:
$ curl -kLSs https://github.com/opscode/chef-repo/tarball/master -o master.tar.gz
$ ls master.tar.gz
master.tar.gz
-L follow redirects
-o (lower case O) to write output to file instead of stdout.
Ss silent mode, but show errors, if any
k allows curl to proceed and operate even for server connections otherwise considered insecure.
Reference: curl manpage.
With the command:
pip install wget
you installed this Python library https://pypi.org/project/wget/, so you can use that from inside Python:
import wget
I imagine what you actually want is to be able to use wget from inside Git bash. To do what, install Wget for Windows and add the executable to the path. Or, alternatively, use curl.
if you are just looking for having wget in the git bash without pip or any other dependency, you can follow the nice and quick tutorial from this page:
How to add more to Git Bash on Windows
the essence of it is:
Download wget binaries for Windows here (preferrably as ZIP) eternallybored
extract the wget.exe from the zip
copy the EXE file to your git bash binaries folder e.g. "c:\Program Files\Git\mingw64\bin"
done :)
Quick and dirty replacement for the single argument, fetch a file usecase:
alias wget='curl -O'
-O, --remote-name Write output to a file named as the remote file
Maybe give the alias a different name so you don't try to use wget flags in curl.
I am trying to find a way to make these work together. Whereas I can run this successfully using Wget for Windows:
wget --html-extension -r http://www.sitename.com
this downloads every single file on my server that is directory linked from the root domain. I'd rather download only the pages in my sitemap. For this, I found the following trick which uses CygWin:
wget --quiet https://www.sitename.com/sitemap.xml --output-document - | egrep -o
"http://www\.sitename\.com[^<]+" | wget --spider -i - --wait 1
However this is only checking that the pages exist, not downloading them as static HTML files as the prior wget command is doing.
Is there a way to merge these and download the sitemap pages as local html files?
If you look at the man page for wget, you will see that the --spider entry is as follows:
--spider
When invoked with this option, Wget will behave as a Web spider, which means that it will not download the pages, just check that they are there.
All you need to do to actually download the file is remove the --spider from your command.
wget --quiet https://www.sitename.com/sitemap.xml --output-document - | egrep -o \
"https?://www\.sitename\.com[^<]+" | wget -i - --wait 1
Every now and then (<5%) Travis builds stall because wget calling a PHP page times out. Example: https://travis-ci.org/marcelstoer/nodemcu-custom-build/builds/69239694 I was using curl first for several weeks but then tried wget because of this issue.
cURL
curl -m 60 url
wget
wget -qO- url &> /dev/null
The server which serves the PHP page doesn't report any errors and in the Apache access log the request to the PHP page is reported as returning HTTP 200.
What could I do to further analyze?
Following these instructions:
Ubuntu installation
on Ubuntu Server 12.04. I've set my https_proxy in /etc/environment. Next I do:
sudo wget https://get.docker.com/
and the response is "cannot verify get.docker.com's certificate... to connect insecurely use '--no-check-certificate'.
So I do:
sudo wget --no-check-certificate https://get.docker.com/
I'm still getting a message complaining "cannot verify get.docker.com's certificate" and wget downloads the index.html file from get.docker.com rather than an installation package.
I am very new to Linux - please can anyone tell me what I'm doing wrong?
You are doing this:
sudo wget https://get.docker.com/
The instructions to which you linked tell you to do this:
wget -qO- https://get.docker.com/ | sh
That retrieves the shell script and pipes it to the shell for execution. For the record I am morally opposed to this sort of installation, but that's what you need to do to follow those instructions.
I am trying to download a site (with permission) and grepping a particular text from that. The problem is I want to grep on the go without saving any files on local drive. Following command does not help.
wget --mirror site.com -O - | grep TEXT
wget command manual (man page) tells, the usage of the command should be:
wget [option]... [URL]...
in your case, it should be:
wget --mirror -O - site.com|grep TXT
You can use curl:
curl -s http://www.site.com | grep TEXT
how about this
wget -qO- site.com |grep TEXT
and
curl -vs site.com 2>&1 |grep TEXT