I want to parse the HTML from this site, I already try to use :
file_get_contents
Simple HTML DOM
but the result is awful like this :
then I try PHP CURL by using php html parser, I found the error is SSL certificate problem: unable to get local issuer certificate
After searching the answer, then I read using curl to access https, and I try to get the certificate by using the step in that article but the result is same
the question is, why the parsing result is like that? now, I don't know what to do :(
The page is compressed with gzip.
curl -k --compressed https://inaproc.lkpp.go.id/v3/daftar_lpse
For PHP
$ch = curl_init("https://inaproc.lkpp.go.id/v3/daftar_lpse");
curl_setopt($ch, CURLOPT_ENCODING , "");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
$output = curl_exec($ch);
curl_close($ch);
echo $output;
Related
I am trying to upload text files to a ruby on rails server. I was successful at uploading the file through CLI:
curl -X POST --data-urlencode data#/etc/filepath/ www.example.com/path
Now, trying with libCurl through C:
FILE *fd;
fd = fopen(file_path, "rb");
if (!fd)
return -1;
curl_easy_setopt(curl, CURLOPT_URL, url);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, data);
curl_easy_setopt(curl, CURLOPT_POST, 1L);
curl_easy_setopt(curl, CURLOPT_UPLOAD, 1L);
curl_easy_setopt(curl, CURLOPT_READDATA, fd);
}
Any ideas why running through CLI would work, but libCurl does not? Am I missing some curl options or does my server require alternate configuration?
Thanks
You want CURLOPT_POSTFIELDS or CURLOPT_UPLOAD basically. Do you want to provide the data as a string or read it from the file?
You also don't want both CURLOPT_POST and CURLOPT_UPLOAD, as the later implies PUT.
i'm trying to send a parse.com push notification from ruby 1.8.7.
i got a test working with curl. but with ruby's net::http i'm getting Timeout::Error: Resource temporarily unavailable
how can i debug this? i don't know how to see why the parse server is responding differently or otherwise see what's happening. i tried sending the request to my own server and the headers looked ok to me.
i simplified what i'm doing to this:
http = Net::HTTP.new('api.parse.com', 443)
response = http.post("/1/push", "{\"where\":{},\"data\":{\"alert\":\"Elliot net http json test 1\"}}", {"X-Parse-Application-Id"=>"xxxxx", "Content-Type"=>"application/json", "X-Parse-REST-API-Key"=>"xxxxx"})
the json there is hard to read, it's from:
api_req = {:where => {}, :data => {:alert => "Elliot net http json test 1"}}.to_json
puts api_req
# {"where":{},"data":{"alert":"Elliot net http json test 1"}}
i also tried several other ways of sending a request with net::http. same result.
the curl request that worked was:
curl -X POST \
-H "X-Parse-Application-Id: xxxxxx" \
-H "X-Parse-REST-API-Key: xxxxx" \
-H "Content-Type: application/json" \
-d '{
"where": {},
"data": {
"alert": "Elliot curl test #4"
}
}' \
https://api.parse.com/1/push
i'm not using parse-ruby-client because i ran into problems with dependencies assuming a newer version of ruby. all i need to do is send some simple push notifications, and it seems like this should work without too much trouble.
can anyone help me get this working or tell me how to get some useful info about what's happening to debug?
As per the REST API Developers guide,
All API access is over HTTPS, and accessed via the https://api.parse.com domain.
So all you need to do is to add http.use_ssl = true.
I followed the instructions on https://developers.google.com/youtube/v3/guides/authentication#OAuth2_Calling_a_Google_API
to check my access to the YouTube v3 API.
I got an access_token from this:
curl --data "code=4/XXX.YYY" \
--data "client_id=foo123.apps.googleusercontent.com" \
--data "client_secret=YYY" \
--data "redirect_uri=urn:ietf:wg:oauth:2.0:oob" \
--data "grant_type=authorization_code" \
https://accounts.google.com/o/oauth2/token
which returned:
{
"access_token" : "ya29.XYXYXY",
"token_type" : "Bearer",
"expires_in" : 3600
}
If I now check access with curl or in the browser
https://www.googleapis.com/youtube/v3/video?access_token=ya29.XYXYXY
I just got back "Not Found".
If I use another (phantasy) access_token or leave out the parameter the result is always the same, just "Not Found". I tried with "http" and "https".
Am I doing something wrong? Is the service just temporarily not available? Should I use v2?
Thanks for your time,
Axel.
Found it myself. The documentation is misleading for me. The sample it uses
https://www.googleapis.com/youtube/v3/video?access_token=ya29.XYXYXY
doesn't say that you need further parameters. The response "Not found" from the server is not helpful. Furthermore "video" must be "videos".
A correct test would be
https://www.googleapis.com/youtube/v3/videos?access_token=ya29.XYXYXY&id=12345678&part=snippet
I am using cURL command line utility to send HTTP POST to a web service. I want to include a file's contents as the body entity of the POST. I have tried using -d </path/to/filename> as well as other variants with type info like --data </path/to/filename> --data-urlencode </path/to/filename> etc... the file is always attached. I need it as the body entity.
I believe you're looking for the #filename syntax, e.g.:
strip new lines
curl --data "#/path/to/filename" http://...
keep new lines
curl --data-binary "#/path/to/filename" http://...
curl will strip all newlines from the file. If you want to send the file with newlines intact, use --data-binary in place of --data
I know the question has been answered, but in my case I was trying to send the content of a text file to the Slack Webhook api and for some reason the above answer did not work. Anywho, this is what finally did the trick for me:
curl -X POST -H --silent --data-urlencode "payload={\"text\": \"$(cat file.txt | sed "s/\"/'/g")\"}" https://hooks.slack.com/services/XXX
In my case, # caused some sort of encoding problem, I still prefer my old way:
curl -d "$(cat /path/to/file)" https://example.com
curl https://upload.box.com/api/2.0/files/3300/content -H "Authorization: Bearer $access_token" -F file=#"C:\Crystal Reports\Crystal Reports\mysales.pdf"
I need to get the Google validation token to use with Google APIs, but my code does not work.
$client_id = '495225261106.apps.googleusercontent.com';
$client_secret = urlencode('MY_SECRET_CDE');
$redirect_uri = urlencode('http://MYPAGE.net/test.php');
//$grant_type = urlencode('authorization_code'); //it does not work either.
$grant_type = 'authorization_code';
$post_string = "code=4/P7q7W91a-oMsCeLvIaQm6bTrgtp6&client_id={$client_id}&client_secret={$client_secret}&redirect_uri={$redirect_uri}&grant_type={$grant_type}";
//echo_key_value('post_string',$post_string);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://accounts.google.com/o/oauth2/token');
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_string);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$result = curl_exec($ch); // Execute the HTTP command
$errmsg = curl_error($ch);
if($errmsg) echo $errmsg;
The output is:
{"error":"invalid_grant"}
You may find it easier to use Google APIs, especially OAuth stuff, via one of the official client libraries.
Here's a link to the PHP one: http://code.google.com/p/google-api-php-client/
And a link to the docs on OAuth 2.0 with the library (with some great example code): http://code.google.com/p/google-api-php-client/wiki/OAuth2
Don't you have to put " curl_setopt($ch, CURLOPT_POST, true); " before using postfields? Mine is working and except that and I didn't used urlencode on my secret, it's the same
Setup Instructions
Go to the Google Developers Console
https://console.developers.google.com/project Select your project or
create a new one (and then select it)
Enable the API for your
project In the sidebar on the left, expand APIs & auth > APIs Search
for "drive" Click on "Drive API" click the blue "Enable API" button
Create a service account for your project In the sidebar on the left,
expand APIs & auth > Credentials Click blue "Add credentials" button
Select the "Service account" option
Select "Furnish a new private
key" checkbox Select the "JSON" key type option
Click blue "Create"
button your JSON key file is generated and downloaded to your machine
(it is the only copy!)
open the json file and save your private key to a file called rsa
note your service account's email address
(also available in the JSON key file) Share the doc (or docs) with
your service account using the email noted above
based on information from ( a fantastic doc )
https://developers.google.com/identity/protocols/OAuth2ServiceAccount#delegatingauthority
for a list of possible API scopes set
https://developers.google.com/identity/protocols/googlescopes#sheetsv4
for a purely bash based solution
#!/bin/bash
client_email='your client email'
scope='https://www.googleapis.com/auth/spreadsheets.readonly'
jwt1=`echo -n '{"alg":"RS256","typ":"JWT"}' | openssl base64 -e`
exp=$(($(date +%s)+3600))
iat=$(date +%s)
jwt2=`echo -n '{\
"iss":"'"$client_email"'",\
"scope":"'"$scope"'",\
"aud":"https://accounts.google.com/o/oauth2/token",\
"exp":'$exp',\
"iat":'$iat'}' | openssl base64 -e`
jwt3=`echo -n "$jwt1.$jwt2" | tr -d '\n' | tr -d '=' | tr '/+' '_-'`
jwt4=`echo -n "$jwt3" | openssl sha -sha256 -sign rsa | openssl base64 -e`
jwt5=`echo -n "$jwt4" | tr -d '\n' | tr -d '=' | tr '/+' '_-'`
echo $jwt3
echo $jwt5
curl -H -vvv "Content-type: application/x-www-form-urlencoded" -X POST "https://accounts.google.com/o/oauth2/token" -d \
"grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer&assertion=$jwt3.$jwt5"
for a javascript nodejs based solution see
https://gist.github.com/cloverbox/5ce51a1d8889da9045c5b128a3a2502f