I'm trying to extract data from twitter. When I use the code in my home network, it works very well, but in my university we use a proxy server. So I need to configure the request. I'm not specialised on computer science, so I don't know how to modify the request.
I read the documentation in https://dev.twitter.com/ but I couldn't find a solution!
This is an exemple of code :
import tweepy
from tweepy import OAuthHandler
consumer_key = 'XXXXXXXXXX'
consumer_secret = 'XXXXXXXXXXXX'
access_token = 'XXXXXXXXXXXXXXXXXXXX'
access_secret = 'XXXXXXXXXXXXXXXXXXXXXXXXXXX'
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_secret)
api = tweepy.API(auth)
for status in tweepy.Cursor(api.home_timeline).items(100):
# Process a single status
print(status.text)
This is the error I got:
Traceback (most recent call last):
File "D:\formation\python\github tools\fbp.py", line 14, in <module>
for status in tweepy.Cursor(api.home_timeline).items(100):
File "C:\Anaconda\lib\site-packages\tweepy\cursor.py", line 197, in next
self.current_page = self.page_iterator.next()
File "C:\Anaconda\lib\site-packages\tweepy\cursor.py", line 108, in next
data = self.method(max_id=self.max_id, parser=RawParser(), *self.args, **self.kargs)
File "C:\Anaconda\lib\site-packages\tweepy\binder.py", line 245, in _call
return method.execute()
File "C:\Anaconda\lib\site-packages\tweepy\binder.py", line 189, in execute
raise TweepError('Failed to send request: %s' % e)
TweepError: Failed to send request: HTTPSConnectionPool(host='api.twitter.com', port=443): Max retries exceeded with url: /1.1/statuses/home_timeline.json (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 407 Authentification proxy requise',)))
You can add a proxy using the follow code:
api =tweepy.API(auth,proxy="127.0.0.1:1080")
It works on my computer. You can try to modify you code like this.
Related
apologies upfront for being a complete SSL noob.
I´m starting to work with Micropython and ran into a problem that with HHTPS requests. With the below code on an ESP8266MOD 12F I try to get some information from a test API, but all I get is an error.
import urequests as requests
import network
WIFI_SSID = "***********"
WIFI_PW = "************"
def connect_wifi(wlan, ssid, password):
if not wlan.isconnected():
print("\nConnecting...")
wlan.connect(ssid, password)
while not wlan.isconnected():
pass
print("Wifi connected")
wlan = network.WLAN(network.STA_IF)
wlan.active(True)
connect_wifi(wlan, WIFI_SSID, WIFI_PW)
if not wlan.isconnected():
connect_wifi(wlan, WIFI_SSID, WIFI_PW)
else:
url = "http://api.coindesk.com/v1/bpi/currentprice.json"
r = requests.get(url)
print(r.text)
The error I get is:
Traceback (most recent call last):
File "<stdin>", line 25, in <module>
File "urequests.py", line 116, in get
File "urequests.py", line 62, in request
OSError: [Errno 107] ENOTCONN
My original attempt was to get to the Telelgram Bot API so I can send Telegram messages from my IoT devices. If I use an URL for the Telelgram API in the form of
url = "https://api.telegram.org/bot*****:***********/sendMessage?text=Klingel&chat_id=*****"
I get a different error which is
Traceback (most recent call last):
File "<stdin>", line 25, in <module>
File "urequests.py", line 116, in get
File "urequests.py", line 60, in request
OSError: [Errno 103] ECONNABORTED
When changing the Coindesk URL to HTTP, everything works fine. Unfortunately the Telegram API does not accept HTTP requests.
I tried several different urequests libraries as the on e isntalles via pip seemed to be a bit dated. The one I´m currently trying I got from github here. My ESP is running the current firmware esp8266-20220618-v1.19.1 from micropython.org.
I searched the internet for 4 days now and did not find anything that helped me. Nevertheless there seem to be quite some people to managed to use SSL in this configuration. As I am a true beginner in this whole topic, any help would be appreciated.
Thanks
Marcus
After some deeper digging I found out that the problem was caused by my Adguard DNS server which blocked the Telegram API. I did not see it on my laptop because it used a different DNS server. Now that I fixed the DNS issue, I´m getting OSError -40 which seems to be a common problem with the axTLS library used in the ESP8266. At least that makes sense to me now.
Thanks Andy for the quick reply.,
I am trying to authenticate using bearer token.
When I am trying to call the api with valid oauth2.0 access token
https://**************/api/method/frappe.auth.get_logged_user
I am getting this error response.
{
"exc": "[\"Traceback (most recent call last):\\n File \\\"/home/frappe/frappe-bench/apps/frappe/frappe/app.py\\\", line 66, in application\\n response = frappe.api.handle()\\n File \\\"/home/frappe/frappe-bench/apps/frappe/frappe/api.py\\\", line 56, in handle\\n return frappe.handler.handle()\\n File \\\"/home/frappe/frappe-bench/apps/frappe/frappe/handler.py\\\", line 21, in handle\\n data = execute_cmd(cmd)\\n File \\\"/home/frappe/frappe-bench/apps/frappe/frappe/handler.py\\\", line 54, in execute_cmd\\n is_whitelisted(method)\\n File \\\"/home/frappe/frappe-bench/apps/frappe/frappe/handler.py\\\", line 64, in is_whitelisted\\n raise frappe.PermissionError('Not Allowed, {0}'.format(method))\\nPermissionError: Not Allowed, <function get_logged_user at 0x7f9c027a9c08>\\n\"]",
"_server_messages": "[\"{\\\"message\\\": \\\"Not permitted\\\"}\"]"
}
Hard to tell without the code base. But it looks like the
execute_cmd(..)
is raising a Permissions error. I think the problem is with the script itself running on that machine. i.e. Not necessarily the access token. But without code this is a guess. You could verify by manually running that script when logged into the machine with the same user/permissions as when running as a web server.
We are trying to use the JIRA application link for creating the token. But we are getting the error as 400 response
Traceback (most recent call last):
File "generate_access_token.py", line 30, in
request_token = oauth.fetch_request_token(REQUEST_TOKEN_URL)
File "/usr/lib/python2.7/site-packages/requests_oauthlib/oauth1_session.py", line 284, in fetch_request_token
token = self._fetch_token(url, **request_kwargs)
File "/usr/lib/python2.7/site-packages/requests_oauthlib/oauth1_session.py", line 368, in _fetch_token
raise TokenRequestDenied(error % (r.status_code, r.text), r)
requests_oauthlib.oauth1_session.TokenRequestDenied: Token request failed with code 400, response was 'oauth_parameters_absent=oauth_consumer_key&oauth_problem=parameter_absent'.
Can someone please help?
The error seems to explain that a value for oauth_consumer_key is not being passed.
The parameter_absent error means that you are not sending the required OAuth parameters to the actual endpoint.
Can you check if this is being passed correctly or if it is not being passed totally? Adding your code snippet should help others help you identify the issue quickly!
Hope this answers your question!
trying to test out the ECS driver of libcloud for a project.
i wrote a simple test:
with open("conf.json") as f:
data = json.loads(f.read())
driver = get_driver(getattr(Provider,'ECS'))
with open('.cloud_credentials/'+data['provider']+'/access_id', 'r') as myfile:
ACCESS_KEY_ID = myfile.read().replace('\n', '')
print ACCESS_KEY_ID
with open('.cloud_credentials/'+data['provider']+'/secret_key', 'r') as myfile:
SECRET_KEY = myfile.read().replace('\n', '')
print SECRET_KEY
timer.start()
conn = driver(access_id = ACCESS_KEY_ID,
secret = SECRET_KEY,
region = 'us-east-1')
timer.stop()
cluster = conn.create_cluster('pippo','us-east-1')
However running it produce this error:
Traceback (most recent call last):
File "launch.py", line 32, in <module>
cluster = conn.create_cluster('pippo',data['region'])
File "/home/sergio/anaconda2/lib/python2.7/site-packages/libcloud/container/drivers/ecs.py", line 151, in create_cluster
headers=self._get_headers('CreateCluster')
File "/home/sergio/anaconda2/lib/python2.7/site-packages/libcloud/common/base.py", line 604, in request
headers=headers, stream=stream)
File "/home/sergio/anaconda2/lib/python2.7/site-packages/libcloud/http.py", line 212, in request
verify=self.verification
File "/home/sergio/anaconda2/lib/python2.7/site-packages/requests/sessions.py", line 468, in request
resp = self.send(prep, **send_kwargs)
File "/home/sergio/anaconda2/lib/python2.7/site-packages/requests/sessions.py", line 576, in send
r = adapter.send(request, **kwargs)
File "/home/sergio/anaconda2/lib/python2.7/site-packages/requests/adapters.py", line 437, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='ecs.%s.amazonaws.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f5f16dca950>: Failed to establish a new connection: [Errno -2] Name or service not known',))
I Think that the problem lies with the host string, containing that %s. Digging in the library looks like that string is a dummy one initialized in the file:
/libcloud/container/drivers/ecs.py
I suppose that there should be somewhere in those files a function supposed to overwrite the base-string, that probably doesn't get called for some reasons but, being a novice with the libcloud library, makes it kind of difficult to find the culprit.
Curiously, inspecting the headers that are being sent, looks like they contain the correct(?) address there:
headers: {'Host': u'ecs.us-east-1.amazonaws.com', 'User-Agent': 'libcloud/2.2.1 (Amazon Elastic Container Service) ', 'Content-Type': 'application/x-amz-json-1.1', 'Accept-Encoding': 'gzip,deflate', 'x-amz-target': 'AmazonEC2ContainerServiceV20141113.CreateCluster'}
Anyone with more experience could point me in the right direction? It would be much appreciated.
This looks like a bug, it's not something you've done.
I am getting an error "unsupported grant type" when I try to request an OAuth token for a service account using curl. I'm following the example for OAuth 2.0 for service accounts (https://developers.google.com/identity/protocols/OAuth2ServiceAccount) and I think I have everything setup correctly. I have a service account setup in Google Cloud and I'm using that email address in the OAuth request.
The documentation says to use the URL encoded grant type "urn:ietf:params:oauth:grant-type:jwt-bearer" but it isn't clear if this is the only option for the grant type or what other options might be.
I am sending the the base64 encoded header
{"alg":"RS256","typ":"JWT"}
and "."
and base64 encoded claims
{
"iss":"chargepubadmin#xxxxxxxx.iam.gserviceaccount.com",
"scope":"https://www.googleapis.com/auth/pubsub",
"aud":"https://www.googleapis.com/oauth2/v4/token",
"exp":1497159875,
"iat":1497156275
}
and "."
and base64 encoded signature
{base64 header}.{base64 claims}
.
curl -X POST -d 'grant_type=http%3A%2F%2Foauth.net%2Fgrant_type%2Fdevice%2F1.0%26assertion=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.ew0KICAiaXNzIjoiY2.......' "https://www.googleapis.com/oauth2/v4/token"
I'm using an online base64 encoding tool which matches the example base64 encoding.
Can anyone enlighten me as to the what the grant type is or should be?
The grant type should be set as urn:ietf:params:oauth:grant-type:jwt-bearer documented here under the REST API Making the access token request section.
Working example using google-auth library
It will be very easy and simple, if you used the google-auth library which automatically takes care of parsing the private key json file, fetching access tokens, refreshing them and actually including them as part of the requests.
You only need to provide the request URL and body, the library takes care of the rest. Here is a simplified example:
#!/usr/bin/env python
from google.auth.transport.requests import AuthorizedSession
from google.oauth2.service_account import Credentials
# BEGIN CONFIGURATION - change as needed.
# Path to the JSON file containing the service account private key and email.
PRIVATE_KEY_JSON = '/path/to/json/file'
# The API scope this token will be valid for.
API_SCOPES = ['https://www.googleapis.com/auth/pubsub']
# END CONFIGURATION
if __name__ == '__main__':
credentials = Credentials.from_service_account_file(
PRIVATE_KEY_JSON, scopes=API_SCOPES)
authed_session = AuthorizedSession(credentials)
url = 'https://pubsub.googleapis.com/v1/<SOMETHING>'
response = authed_session.get(url)
print str(response.content)
Working example without additional libraries
If you do not want to use any additional libraries but can use the standard python libraries, here is a working sample (tested personally with a service account of my own) in Python (supports both 2.x and 3.x versions) which takes care of all the steps:
#!/usr/bin/env python
import Crypto.PublicKey.RSA as RSA
import Crypto.Hash.SHA256 as SHA
import Crypto.Signature.PKCS1_v1_5 as PKCS1_v1_5
import base64
import json
import time
try:
from urllib.request import urlopen
except ImportError:
from urllib2 import urlopen
try:
from urllib.parse import urlencode
except ImportError:
from urllib import urlencode
# BEGIN CONFIGURATION - change as needed.
# Path to the JSON file containing the service account private key and email.
PRIVATE_KEY_JSON = '/path/to/json/file'
# The API scope this token will be valid for.
API_SCOPE = 'https://www.googleapis.com/auth/pubsub'
# The validity of the token in seconds. Max allowed is 3600s.
ACCESS_TOKEN_VALIDITY_SECS = 3600
# END CONFIGURATION
class OauthAccessTokenGetter:
"""Fetches a new Google OAuth 2.0 access token.
The code is based on the steps described here: https://developers.go
ogle.com/identity/protocols/OAuth2ServiceAccount#authorizingrequests
"""
ACCESS_TOKEN_AUD = 'https://www.googleapis.com/oauth2/v4/token'
REQUEST_URL = 'https://www.googleapis.com/oauth2/v4/token'
GRANT_TYPE = 'urn:ietf:params:oauth:grant-type:jwt-bearer'
def __init__(self, private_key_json_file, scope, token_valid_secs=3600):
self.private_key_json = self.LoadPrivateKeyJsonFromFile(
private_key_json_file)
self.scope = scope
self.token_valid_secs = token_valid_secs
#classmethod
def Base64UrlEncode(cls, data):
"""Returns the base64url encoded string for the specified data."""
return base64.urlsafe_b64encode(data)
#classmethod
def LoadPrivateKeyJsonFromFile(cls, private_key_json_file):
"""Returns JSON object by parsing the specified private key JSON
file."""
with open(private_key_json_file) as private_key_json_file:
return json.load(private_key_json_file)
def GetPrivateKey(self):
"""Returns the imported RSA private key from the JSON data."""
return RSA.importKey(self.private_key_json['private_key'])
def GetSigner(self):
"""Returns a PKCS1-V1_5 object for signing."""
return PKCS1_v1_5.new(self.GetPrivateKey())
#classmethod
def GetEncodedJwtHeader(cls):
"""Returns the base64url encoded JWT header."""
return cls.Base64UrlEncode(json.dumps({'alg': 'RS256', 'typ': 'JWT'}).encode('utf-8'))
def GetEncodedJwtClaimSet(self):
"""Returns the base64url encoded JWT claim set."""
current_time_secs = int(time.time())
jwt_claims = {
'iss': self.private_key_json['client_email'],
'scope': self.scope,
'aud': self.ACCESS_TOKEN_AUD,
'exp': current_time_secs + self.token_valid_secs,
'iat': current_time_secs
}
return self.Base64UrlEncode(json.dumps(jwt_claims).encode('utf-8'))
def GetJwtSignature(self, message):
"""Returns signature of JWT as per JSON Web Signature (JWS) spec."""
signed_message = self.GetSigner().sign(SHA.new(message))
return self.Base64UrlEncode(signed_message)
def GetSignedJwt(self):
"""Returns signed JWT."""
header = self.GetEncodedJwtHeader()
jwt_claim_set = self.GetEncodedJwtClaimSet()
signature = self.GetJwtSignature(header + b'.' + jwt_claim_set)
return header + b'.' + jwt_claim_set + b'.' + signature
def SendRequest(self, body):
"""Returns the response by sending the specified request."""
return urlopen(self.REQUEST_URL, urlencode(body).encode('utf-8')).read()
def GetAccessToken(self):
"""Returns the access token."""
body = {
'grant_type': self.GRANT_TYPE,
'assertion': self.GetSignedJwt()
}
response = json.loads(self.SendRequest(body))
return response['access_token']
if __name__ == '__main__':
print (OauthAccessTokenGetter(PRIVATE_KEY_JSON, API_SCOPE,
ACCESS_TOKEN_VALIDITY_SECS).GetAccessToken())
After you get the access token, you need to include it as the Bearer header in the requests you send as described here.
GET /drive/v2/files HTTP/1.1
Authorization: Bearer <access_token>
Host: www.googleapis.com/
Equivalently in curl as:
curl -H "Authorization: Bearer <access_token>" https://www.googleapis.com/drive/v2/files
Although it is described here that you can specify the token using access_token= parameter, I could not get it working at least for Google Compute Engine APIs, may be it works with PubSub, but the Bearer header approach has worked always in my experience.
UPDATE: As per the discovery doc for PubSub API, there seems to be a query parameter for access_token=, so it might very well work too.
"access_token": {
"description": "OAuth access token.",
"type": "string",
"location": "query"
},
And the discovery doc for Compute Engine APIs indicate the use of oauth_token query parameter instead and I did verify that it worked.
"oauth_token": {
"type": "string",
"description": "OAuth 2.0 token for the current user.",
"location": "query"
},