How does one change user agent strings in http requests made in R? And how do I figure out what my current user agent string looks like?
Thanks in advance.
options("HTTPUserAgent") or getOption("HTTPUserAgent") prints your current settings, and options(HTTPUserAgent="My settings") is the way to change it.
To temporary change this option use: withr::with_options:
withr::with_options(list(HTTPUserAgent="My settings"), download.file(..something..))
or Droplet answer if you use download.file.
The solution using options() in the accepted answer will change the setting globally for the whole session (unless you change it back).
To change the User-Agent temporarily in a request made by download.file(), you need to use the headers argument:
download.file("https://httpbin.org/user-agent",
destfile = "test.txt",
headers = c("User-Agent" = "My Custom User Agent"))
Since R 4.0.0, you can also use this argument in available.packages() and install.packages() and it will be forwarded to download.file().
Related
I'm trying to use Apigee's GetOAuthv2Info policy but think I'm missing something really simple (still new to Apigee/APIs)...
The policy is configured based on the sample:
<GetOAuthV2Info name="GetTokenAttributes">
<AccessToken>request.queryParam.access_token</AccessToken>
</GetOAuthV2Info>`
If I try to pass a valid token via a query parameter:
http://{host}/path/to/endpoint?access_token=tUbvXzh97UtRRUuBpGUNpXESJtD1, I get a 404 Not Found error code with:
{"fault":"{\"detail\":
{\"errorcode\":\"keymanagement.service.invalid_access_token\"},
\"faultstring\":\"Invalid Access Token\"}"}
Is this the correct way to pass the access token to the policy?
Thanks in advance!
Try:
<GetOAuthV2Info name="GetTokenAttributes">
<AccessToken ref="request.queryparam.access_token"></AccessToken>
</GetOAuthV2Info>
or
<GetOAuthV2Info name="GetTokenAttributes">
<AccessToken>{request.queryparam.access_token}</AccessToken>
</GetOAuthV2Info>
Your original policy is probably using the literal string value of request.queryParam.access_token as the token reference. Also noticed that queryparam is complete lower case.
I can't seem to make any progress with this one. My CI session settings are these:
$config['sess_cookie_name'] = 'ci_session';
$config['sess_expiration'] = 0;
$config['sess_expire_on_close'] = FALSE;
$config['sess_encrypt_cookie'] = FALSE;
$config['sess_use_database'] = TRUE;
$config['sess_table_name'] = 'ci_sessions';
$config['sess_match_ip'] = FALSE;
$config['sess_match_useragent'] = FALSE;
$config['sess_time_to_update'] = 7200;
$config['cookie_prefix'] = "";
$config['cookie_domain'] = "";
$config['cookie_path'] = "/";
$config['cookie_secure'] = FALSE;
The session library is loaded on autoload. I've commented the sess_update function to prevent an AJAX bug that I've found about reading the CI forum.
The ci_sessions table in the database has collation utf8_general_ci (there was a bug that lost the session after every redirect() call and it was linked to the fact that the collation was latin1_swedish_ci by default).
It always breaks after a user of my admin section tries to add a long article and clicks the save button. The save action looks like this:
function save($id = 0){
if($this->my_model->save_article($id)){
$this->session->set_flashdata('message', 'success!');
redirect('admin/article_listing');
}else{
$this->session->set_flashdata('message', 'errors encountered');
redirect('admin/article_add');
}
}
If you spend more than 20minutes and click save, the article will be added but on redirect the user will be logged out.
I've also enabled logging and sometimes when the error occurs i get the message The session cookie data did not match what was expected. This could be a possible hacking attempt. but only half of the time. The other half I get nothing: a message that I've placed at the end of the Session constructor is displayed and nothing else. In all the cases if I look at the cookie stored in my browser, after the error the cookie's first part doesn't match the hash.
Also, although I know Codeigniter doesn't use native sessions, I've set session.gc_maxlifetime to 86400.
Another thing to mention is that I'm unable to reproduce the error on my computer but on all the other computers I've tested this bug appears by the same pattern as mentioned above.
If you have any ideas on what to do next, I'd greatly appreciate them. Changing to a new version or using a native session class (the old one was for CI 1.7, will it still work?) are also options I'm willing to consider.
Edit : I've run a diff between the Session class in CI 2.0.3 and the latest CI Session class and they're the same.
Here's how I solved it: the standards say that a browser shouldn't allow redirects after a POST request. CI's redirect() method is sending a 302 redirect by default. The logical way would be to send a 307 redirect, which solved my problem but has the caveat of showing a confirm dialog about the redirect. Other options are a 301 (meaning moved permanently) redirect or, the solution I've chosen, a javascript redirect.
I'd like to have a dynamic redirect URL for my Facebook OAuth2 integration. For example, if my redirect URL is this in my Facebook app:
http://www.mysite.com/oauth_callback?foo=bar
I'd like the redirect URL for a specific request be something like this, so that on the server, I have some context about how to process the auth code:
http://www.mysite.com/oauth_callback?foo=bar&user=6234
My redirect gets invoked after the authorization dialog is submitted, and I get back an auth code, but when I try to get my access token, I'm getting an OAuthException error back from Facebook. My request looks like this (line breaks added for clarity):
https://graph.facebook.com/oauth/access_token
?client_id=MY_CLIENT_ID
&redirect_uri=http%3A%2F%2Fwww.mysite.com%2Foauth_callback%3Ffoo%3Dbar%26user%3D6234
&client_secret=MY_SECRET
&code=RECEIVED_CODE
All of my parameters are URL-encoded, and the code looks valid, so my only guess is that the problem parameter is my redirect_uri. I've tried setting redirect_uri to all of the following, to no avail:
The actual URL of the request to my site
The URL of the request to my site, minus the code parameter
The URL specified in my Facebook application's configuration
Are custom redirect URI parameters supported? If so, am I specifying them correctly? If not, will I be forced to set a cookie, or is there some better pattern for supplying context to my web site?
I figured out the answer; rather than adding additional parameters to the redirect URL, you can add a state parameter to the request to https://www.facebook.com/dialog/oauth:
https://www.facebook.com/dialog/oauth
?client_id=MY_CLIENT_ID
&scope=MY_SCOPE
&redirect_uri=http%3A%2F%2Fwww.mysite.com%2Foauth_callback%3Ffoo%3Dbar
&state=6234
That state parameter is then passed to the callback URL.
If, for any reason, you can't use the option that Jacob suggested as it's my case, you can urlencode your redirect_uri parameter before passing it and it will work, even with a complete querystring like foo=bar&morefoo=morebar in it.
I was trying to implement a Facebook login workflow against API v2.9 following this tutorial. I tried the solutions described above. Manuel's answer is sort of correct, but what I observed is url encoding is not needed. Plus, you can only pass one parameter. Only the first query parameter will be considered, the rest will be ignored. Here is an example,
Request a code via https://www.facebook.com/v2.9/dialog/oauth?client_id={app-id}&redirect_uri=http://{url}/login-redirect?myExtraParameter={some-value}
You'd get a callback for your url. It will look like http://{url}/login-redirect?code={code-from-facebook}&myExtraParameter={value-passed-in-step-1}. Note that facebook would make a callback with myExtraParameter. You can extract the value for myExtraParameter from callback url.
Then you can request access token with https://graph.facebook.com/v2.9/oauth/access_token?client_id={app-id}&client_secret={app-secret}&code={code-from-facebook}&redirect_uri=http://{url}/login-redirect?myExtraParameter={value-extracted-in-step-2}
Additional parameter passed in step 1 after the first query parameter will be ignored. Also make sure to not include any invalid characters in your query parameter (see this for more information).
You're best off specifying a unique callback for each oAuth provider, /oauth/facebook, /oauth/twitter etc.
If you really want the same file to respond to all oAuth requests, either include it in the individual files or setup a path that will call the same file on your server using .htaccess redirects or something similar: /oauth/* > oauth_callback.ext
You should set your custom state parameter using the login helper as such:
use Facebook\Facebook;
use Illuminate\Support\Str;
$fb = new Facebook([
'app_id' => env('FB_APP_ID'),
'app_secret' => env('FB_APP_SECRET'),
'default_graph_version' => env('FB_APP_VER'),
]);
$helper = $fb->getRedirectLoginHelper();
$permissions = [
'public_profile',
'user_link',
'email',
'read_insights',
'pages_show_list',
'instagram_basic',
'instagram_manage_insights',
'manage_pages'
];
$random = Str::random(20);
$OAuth2Client = $fb->getOAuth2Client();
$redirectLoginHelper = $fb->getRedirectLoginHelper();
$persistentDataHandler = $redirectLoginHelper->getPersistentDataHandler();
$persistentDataHandler->set('state', $random);
$loginUrl = $OAuth2Client->getAuthorizationUrl(
url('/') . '/auth/facebook',
$random,
$permissions
);
Hey if you are using official facebook php skd then you can set custom state param like this
$helper = $fb->getRedirectLoginHelper();
$helper->getPersistentDataHandler()->set('state',"any_data");
$url = $helper->getLoginUrl($callback_url, $fb_permissions_array);
I am having a problem with Twitter's oauth authentication and using a callback url.
I am coding in php and using the sample code referenced by the twitter wiki, http://github.com/abraham/twitteroauth
I got that code, and tried a simple test and it worked nicely. However I want to programatically specify the callback url, and the example did not support that.
So I quickly modified the getRequestToken() method to take in a parameter and now it looks like this:
function getRequestToken($params = array()) {
$r = $this->oAuthRequest($this->requestTokenURL(), $params);
$token = $this->oAuthParseResponse($r);
$this->token = new OAuthConsumer($token['oauth_token'], $token['oauth_token_secret']);
return $token;
}
and my call looks like this
$tok = $to->getRequestToken(array('oauth_callback' => 'http://127.0.0.1/twitter_prompt/index.php'));
This is the only change I made, and the redirect works like a charm, however I am getting an error when I then try and use my newly granted access to try and make a call. I get a "Could not authenticate you" error. Also the application never actually gets added to the users authorized connections.
Now I read the specs and I thought all I had to do was specify the parameter when getting the request token. Could someone a little more seasoned in oauth and twitter possibly give me a hand? Thank You
I think this is fixed by twitter by now or you might have missed to provide a default callback url in your application settings, which is required for dynamic callback url to work as mentioned by others above.
Any case, I got this working by passing the oath_callback parameter while retrieving the request token. I am using twitter-async PHP library and had to make a small tweak to make the library pass the callback url.
If you are using twitter-async, the change is below:
modified getRequestToken and getAuthenticateURL functions to take callback url as parameter
public function getRequestToken($callback_url = null)
{
$params = empty($callback_url) ? null : array('oauth_callback'=>$callback_url);
$resp = $this->httpRequest('GET', $this->requestTokenUrl, $params);
return new EpiOAuthResponse($resp);
}
public function getAuthenticateUrl($callback_url = null)
{
$token = $this->getRequestToken($callback_url);
return $this->authenticateUrl . '?oauth_token=' . $token->oauth_token;
}
And pass the callback url from your PHP code.
$twitterObj->getAuthenticateUrl('http://localhost/twitter/confirm.php');
#Ian, twitter now allows 127.0.0.1 and has made some other recent changes.
#jtymann, check my answer here and see if it helps
Twitter oauth_callback parameter being ignored!
GL
jingles
even me to was getting 401 error.. but its resolved..
during registering your application to twitter you need to give callback url...
like http://localhost:8080.
i have done this using java...
so my code is: String CallbackURL="http://localhost:8080/tweetproj/index.jsp";
provider.retrieveRequestToken(consumer,CallbackURL);
where tweetproj is my project name
and index.jsp is just one jsp page...
Hope this may helps u...
After the user authorizes the application on twitter.com and they return to your callback URL you have to exchange the request token for an access token.
Twitter does not honor the oauth_callback parameter and will only use the one specified in the registered application settings.
It also doesn't allow for 127.0.0.1 or localhost names in that callback, so I've setup http://dev.twipler.com which is setup for 127.0.0.1 in DNS so you can safely use;
http://dev.twipler.com/twitter_prompt/index.php
Is anyone else having a difficult time getting Twitters oAuth's callback URL to hit their localhost development environment.
Apparently it has been disabled recently. http://code.google.com/p/twitter-api/issues/detail?id=534#c1
Does anyone have a workaround. I don't really want to stop my development
Alternative 1.
Set up your .hosts (Windows) or etc/hosts file to point a live domain to your localhost IP. such as:
127.0.0.1 xyz.example
where xyz.example is your real domain.
Alternative 2.
Also, the article gives the tip to alternatively use a URL shortener service. Shorten your local URL and provide the result as callback.
Alternative 3.
Furthermore, it seems that it works to provide for example http://127.0.0.1:8080 as callback to Twitter, instead of http://localhost:8080.
I just had to do this last week. Apparently localhost doesn't work but 127.0.0.1 does Go figure.
This of course assumes that you are registering two apps with Twitter, one for your live www.mysite.example and another for 127.0.0.1.
Just put http://127.0.0.1:xxxx/ as the callback URL, where xxxx is the port for your framework
Yes, it was disabled because of the recent security issue that was found in OAuth. The only solution for now is to create two OAuth applications - one for production and one for development. In the development application you set your localhost callback URL instead of the live one.
Callback URL edited
http://localhost:8585/logintwitter.aspx
Convert to
http://127.0.0.1:8585/logintwitter.aspx
This is how i did it:
Registered Callback URL:
http://127.0.0.1/Callback.aspx
OAuthTokenResponse authorizationTokens =
OAuthUtility.GetRequestToken(ConfigSettings.getConsumerKey(),
ConfigSettings.getConsumerSecret(),
"http://127.0.0.1:1066/Twitter/Callback.aspx");
ConfigSettings:
public static class ConfigSettings
{
public static String getConsumerKey()
{
return System.Configuration.ConfigurationManager.AppSettings["ConsumerKey"].ToString();
}
public static String getConsumerSecret()
{
return System.Configuration.ConfigurationManager.AppSettings["ConsumerSecret"].ToString();
}
}
Web.config:
<appSettings>
<add key="ConsumerKey" value="xxxxxxxxxxxxxxxxxxxx"/>
<add key="ConsumerSecret" value="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"/>
</appSettings>
Make sure you set the property 'use dynamic ports' of you project to 'false' and enter a static port number instead. (I used 1066).
I hope this helps!
Use http://smackaho.st
What it does is a simple DNS association to 127.0.0.1 which allows you to bypass the filters on localhost or 127.0.0.1 :
smackaho.st. 28800 IN A 127.0.0.1
So if you click on the link, it will display you what you have on your local webserver (and if you don't have one, you'll get a 404). You can of course set it to any page/port you want :
http://smackaho.st:54878/twitter/callback
I was working with Twitter callback url on my localhost. If you are not sure how to create a virtual host ( this is important ) use Ampps. He is really cool and easy. In a few steps you have your own virtual host and then every url will work on it. For example:
download and install ampps
Add new domain. ( here you can set for example twitter.local) that means your virtual host will be http://twitter.local and it will work after step 3.
I am working on Win so go under to your host file -> C:\Windows\System32\Drivers\etc\hosts and add line: 127.0.0.1 twitter.local
Restart your Ampps and you can use your callback. You can specify any url, even if you are using some framework MVC or you have htaccess url rewrite.
Hope This Help!
Cheers.
Seems nowadays http://127.0.0.1 also stopped working.
A simple solution is to use http://localtest.me instead of http://localhost it is always pointing to 127.0.0.1 And you can even add any arbitrary subdomain to it, and it will still point to 127.0.0.1
See Website
When I develop locally, I always set up a locally hosted dev name that reflects the project I'm working on. I set this up in xampp through xampp\apache\conf\extra\httpd-vhosts.conf and then also in \Windows\System32\drivers\etc\hosts.
So if I am setting up a local dev site for example.com, I would set it up as example.dev in those two files.
Short Answer: Once this is set up properly, you can simply treat this url (http://example.dev) as if it were live (rather than local) as you set up your Twitter Application.
A similar answer was given here: https://dev.twitter.com/discussions/5749
Direct Quote (emphasis added):
You can provide any valid URL with a domain name we recognize on the
application details page. OAuth 1.0a requires you to send a
oauth_callback value on the request token step of the flow and we'll
accept a dynamic locahost-based callback on that step.
This worked like a charm for me. Hope this helps.
It can be done very conveniently with Fiddler:
Open menu Tools > HOSTS...
Insert a line like 127.0.0.1 your-production-domain.com, make sure that "Enable remapping of requests..." is checked. Don't forget to press Save.
If access to your real production server is needed, simply exit Fiddler or disable remapping.
Starting Fiddler again will turn on remapping (if it is checked).
A pleasant bonus is that you can specify a custom port, like this:
127.0.0.1:3000 your-production-domain.com (it would be impossible to achieve this via the hosts file). Also, instead of IP you can use any domain name (e.g., localhost).
This way, it is possible (but not necessary) to register your Twitter app only once (provided that you don't mind using the same keys for local development and production).
edit this function on TwitterAPIExchange.php at line #180
public function performRequest($return = true)
{
if (!is_bool($return))
{
throw new Exception('performRequest parameter must be true or false');
}
$header = array($this->buildAuthorizationHeader($this->oauth), 'Expect:');
$getfield = $this->getGetfield();
$postfields = $this->getPostfields();
$options = array(
CURLOPT_HTTPHEADER => $header,
CURLOPT_HEADER => false,
CURLOPT_URL => $this->url,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_SSL_VERIFYHOST => false
);
if (!is_null($postfields))
{
$options[CURLOPT_POSTFIELDS] = $postfields;
}
else
{
if ($getfield !== '')
{
$options[CURLOPT_URL] .= $getfield;
}
}
$feed = curl_init();
curl_setopt_array($feed, $options);
$json = curl_exec($feed);
curl_close($feed);
if ($return) { return $json; }
}
I had the same challenge and I was not able to give localhost as a valid callback URL. So I created a simple domain to help us developers out:
https://tolocalhost.com
It will redirect any path to your localhost domain and port you need. Hope it can be of use to other developers.
set callbackurl in twitter app : 127.0.0.1:3000
and set WEBrick to bind on 127.0.0.1 instead of 0.0.0.0
command : rails s -b 127.0.0.1
Looks like Twitter now allows localhost alongside whatever you have in the Callback URL settings, so long as there is a value there.
I struggled with this and followed a dozen solutions, in the end all I had to do to work with any ssl apis on local host was:
Go download: cacert.pem file
In php.ini * un-comment and change:
curl.cainfo = "c:/wamp/bin/php/php5.5.12/cacert.pem"
You can find where your php.ini file is on your machine by running php --ini in your CLI
I placed my cacert.pem in the same directory as php.ini for ease.
These are the steps that worked for me to get Facebook working with a local application on my laptop:
goto apps.twitter.com
enter the name, app description and your site URL
Note: for localhost:8000, use 127.0.0.1:8000 since the former will not work
enter the callback URL matching your callback URL defined in TWITTER_REDIRECT_URI your application
Note: eg: http://127.0.0.1/login/twitter/callback (localhost will not work).
Important enter both the "privacy policy" and "terms of use" URLs if you wish to request the user's email address
check the agree to terms checkbox
click [Create Your Twitter Application]
switch to the [Keys and Access Tokens] tab at the top
copy the "Consumer Key (API Key)" and "Consumer Secret (API Secret)" to TWITTER_KEY and TWITTER_SECRET in your application
click the "Permissions" tab and set appropriately to "read only", "read and write" or "read, write and direct message" (use the least intrusive option needed for your application, for just and OAuth login "read only" is sufficient
Under "Additional Permissions" check the "request email addresses from users" checkbox if you wish for the user's email address to be returned to the OAuth login data (in most cases check yes)