v1 api you could do
print(e.response)
print(e.api_errors)
print(e.api_codes)
print(e.api_messages)
with v2 (client) the Error message doesn't seem to have any of this. Any ideas how to get at the api_codes and api_messages. I've been googling for hours and am not finding any clear reference on how to move this to V2. Very frustrating.
With code sample as requested. I hope I'm missing something simple.
try:
status = api.update_status(status="test", in_reply_to_status_id = 1514213177210425346 , auto_populate_reply_metadata=True)
except tweepy.errors.Forbidden as e:
print(e.response)
print(e.api_errors)
print(e.api_codes)
print(e.api_messages)
# <Response [403]>
# [{'code': 433, 'message': 'The original Tweet author restricted who can reply to this Tweet.'}]
# [433]
# ['The original Tweet author restricted who can reply to this Tweet.']
# Deleted or not visible.
try:
response = client.create_tweet(text="test", in_reply_to_tweet_id=1514213177210425346)
except tweepy.errors.Forbidden as e:
print(e.response)
print(e.api_errors)
print(e.api_codes)
print(e.api_messages)
# <Response [403]>
# []
# []
# []
Related
I'm working on getting my Rails app interacting with the AWS Comprehend service for text entity extraction. I'm using the aws-sdk-comprehend gem. I have successfully gotten my app working with the AWS Rekognition service for image analysis using the aws-sdk-rekognition gem.
I can't seem to get the AWS Comprehend authentication correct as all of my calls result in an Aws::Comprehend::Errors::InvalidRequestException.
I have all of the following ENV variables set:
AWSAccessKeyId
AWSSecretKey
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
My code looks something like this:
class MyApp::Aws::ComprehendService < MyApp::ServiceBase
def call
#credentials = Aws::Credentials.new(ENV['AWSAccessKeyId'], ENV['AWSSecretKey'])
#client = Aws::Comprehend::Client.new(region: "us-west-1", credentials: credentials)
#client.detect_entities({text: "this is a simply little blob of text", language_code: "en"})
end
end
This resulted in Aws::Comprehend::Errors::InvalidRequestException. So I also tried:
class MyApp::Aws::ComprehendService < MyApp::ServiceBase
def call
# use ENV credential format I've seen in examples...
#credentials = Aws::Credentials.new(ENV['AWS_ACCESS_KEY_ID'], ENV['AWS_SECRET_ACCESS_KEY'])
#client = Aws::Comprehend::Client.new(region: "us-west-1", credentials: credentials)
#client.detect_entities({text: "this is a simply little blob of text", language_code: "en"})
end
end
I found an example that didn't use the #credential approach. The example claimed "The initialize method will load the credentials environment variables by itself". So I tried this:
class MyApp::Aws::ComprehendService < MyApp::ServiceBase
def call
# ignore setting the credentials
#client = Aws::Comprehend::Client.new(region: "us-west-1")
#client.detect_entities({text: "this is a simply little blob of text", language_code: "en"}).
end
end
This also resulted in Aws::Comprehend::Errors::InvalidRequestException.
Can you see anything I'm doing wrong? Has anyone had success in using this gem to interact with the Comprehend API?
Per the Documentation for Aws::Comprehend::Client#detect_entities
If the system detects a document-level error in your input document, the API returns an InvalidRequestException error response. For details about this exception, see Errors in semi-structured documents in the Comprehend Developer Guide.
So it appears your usage is not necessarily the issue but rather the input documents themselves.
The response however should include what the actual issue is according to the Developer Guide:
Document-level errors
If the ClassifyDocument or DetectEntities API operation detects a document-level error in your input document, the API returns an InvalidRequestException error response.
In the error response, the Reason field contains the value INVALID_DOCUMENT.
The Detail field contains one of the following values:
DOCUMENT_SIZE_EXCEEDED – Document size is too large. Check the size of your file and resubmit the request.
UNSUPPORTED_DOC_TYPE – Document type is not supported. Check the file type and resubmit the request.
PAGE_LIMIT_EXCEEDED – Too many pages in the document. Check the number of pages in your file and resubmit the request.
TEXTRACT_ACCESS_DENIED_EXCEPTION – Access denied to Amazon Textract. Verify that your account has permission to use the Amazon Textract DetectDocumentText and AnalyzeDocument API operations and resubmit the request.
The Aws::Comprehend::Errors::InvalidRequestException object is documented so it appears you could potentially figure out what is wrong via
def call
# use ENV credential format I've seen in examples...
#credentials = Aws::Credentials.new(ENV['AWS_ACCESS_KEY_ID'], ENV['AWS_SECRET_ACCESS_KEY'])
#client = Aws::Comprehend::Client.new(region: "us-west-1", credentials: credentials)
begin
#client.detect_entities({text: "this is a simply little blob of text", language_code: "en"})
rescue Aws::Comprehend::Errors::InvalidRequestException => e
# interrogate the error object here e.g.
puts {reason: e.reason, detail: e.detail}
end
end
I wanted to write a program to fetch tweets from Twitter and then do sentiment analysis. I wrote the following code and got the error even after importing all the necessary libraries. I'm relatively new to data science, so please help me.
I could not understand the reason for this error:
class TwitterClient(object):
def __init__(self):
# keys and tokens from the Twitter Dev Console
consumer_key = 'XXXXXXXXX'
consumer_secret = 'XXXXXXXXX'
access_token = 'XXXXXXXXX'
access_token_secret = 'XXXXXXXXX'
api = Api(consumer_key, consumer_secret, access_token, access_token_secret)
def preprocess(tweet, ascii=True, ignore_rt_char=True, ignore_url=True, ignore_mention=True, ignore_hashtag=True,letter_only=True, remove_stopwords=True, min_tweet_len=3):
sword = stopwords.words('english')
if ascii: # maybe remove lines with ANY non-ascii character
for c in tweet:
if not (0 < ord(c) < 127):
return ''
tokens = tweet.lower().split() # to lower, split
res = []
for token in tokens:
if remove_stopwords and token in sword: # ignore stopword
continue
if ignore_rt_char and token == 'rt': # ignore 'retweet' symbol
continue
if ignore_url and token.startswith('https:'): # ignore url
continue
if ignore_mention and token.startswith('#'): # ignore mentions
continue
if ignore_hashtag and token.startswith('#'): # ignore hashtags
continue
if letter_only: # ignore digits
if not token.isalpha():
continue
elif token.isdigit(): # otherwise unify digits
token = '<num>'
res += token, # append token
if min_tweet_len and len(res) < min_tweet_len: # ignore tweets few than n tokens
return ''
else:
return ' '.join(res)
for line in api.GetStreamSample():
if 'text' in line and line['lang'] == u'en': # step 1
text = line['text'].encode('utf-8').replace('\n', ' ') # step 2
p_t = preprocess(text)
# attempt authentication
try:
# create OAuthHandler object
self.auth = OAuthHandler(consumer_key, consumer_secret)
# set access token and secret
self.auth.set_access_token(access_token, access_token_secret)
# create tweepy API object to fetch tweets
self.api = tweepy.API(self.auth)
except:
print("Error: Authentication Failed")
Assume all the necessary libraries are imported. The error is on line 69.
for line in api.GetStreamSample():
if 'text' in line and line['lang'] == u'en': # step 1
text = line['text'].encode('utf-8').replace('\n', ' ') # step 2
p_t = preprocess(text)
I tried checking on the internet the reason for the error but could not get any solution.
Error was:
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(0 bytes read, 512 more expected)', IncompleteRead(0 bytes read, 512 more expected))
I'm using Python 2.7 and requests version 2.14, the latest one.
If you set stream to True when making a request, Requests cannot release the connection back to the pool unless you consume all the data or call Response.close. This can lead to inefficiency with connections. If you find yourself partially reading request bodies (or not reading them at all) while using stream=True, you should make the request within a with statement to ensure it’s always closed:
with requests.get('http://httpbin.org/get', stream=True) as r:
# Do things with the response here.
I had the same problem but without stream, and as stone mini said, just apply "with" clause before to make sure your request is closed before a new request.
with requests.request("POST", url_base, json=task, headers=headers) as report:
print('report: ', report)
actually the problem with your django2.7 or earlier version based application. that django versions by default allowed 2.5mb data upload memory size of request body.
I was facing the same issue with django2.7 based application, I just updated the setting.py file of my django application where my urls(endpoints) were working.
DATA_UPLOAD_MAX_MEMORY_SIZE = None
I just add the above variable in my application's settings.py file.
you can also readout about that from here
I'm pretty sure this will work for you.
I'm building an app using Dailymotion API
https://developer.dailymotion.com/
with official python sdk
https://github.com/dailymotion/dailymotion-sdk-python
and writing video CRUD(Create, Read, Update, Delete) simply.
Create, Read, Delete were succeccfully completed, but encountering a weird response of the API about "Update".
Here is a simplified snippet of my code in my Django project,
def update(request, video_id):
user = request.user
video = get_object_or_404(Video, pk=video_id)
file_path = # define file_path from uploaded file object
input_title = # define input_title from post request
input_description = # define input_description from post request
d = get_dailymotion_d(user)
if d == 'revoked':
# do actions of logout and delete the user
try:
# get url for upload with the file_path on my server
url = d.upload(file_path)
# update
response = d.post('/video/' + video.dailymotion_video_id, {'url': url, 'title': input_title, 'description': input_description, 'published': 'true', 'channel': 'creation'})
# delete the video from my sever
video.file_field.delete()
return redirect('/videos')
except Exception as e:
print(e.args)
print('update failed..!')
# delete the video from my server
video.file_field.delete()
return redirect('/videos')
def get_dailymotion_d(user):
d = dailymotion.Dailymotion()
d.set_grant_type('token', api_key=settings.DAILYMOTION_API_KEY, api_secret=settings.DAILYMOTION_API_SECRET, scope=['email', 'userinfo', 'manage_videos'], info={'redirect_uri': settings.DAILYMOTION_REDIRECT_URI})
# get credentiaols from database
access_token = user.dailymotionuser.access_token
expires = user.dailymotionuser.expires
refresh_token = user.dailymotionuser.refresh_token
session_params = {'access_token': access_token, 'expires': expires, 'refresh_token': refresh_token}
# set the credentials
d._session_store.set(session_params)
# check if the user revoked or not
try:
force_refreshed_access_token = d.get_access_token(force_refresh=True)
except dailymotion.DailymotionAuthError as e:
print(e.args[0])
return 'revoked'
# get valid access token
valid_access_token = d.get_access_token()
# update database with the valid access token
DailymotionUser.objects.filter(user=user).update(access_token=valid_access_token, expires=expires, refresh_token=refresh_token)
# prepare dic of the valid access token
valid_access_token_dic = {'access_token': valid_access_token}
# set the valid access token
d._session_store.set(valid_access_token_dic)
return d
but the update fails except title field with the following message,
'access_forbidden: You are not allowed to change existing video source.'
from doc,
access_forbidden: Thrown when the user doesn't have the permission to access the data (e.g. missing a required scope to access certain fields).
but I'm sure that the permission has manage_videos scope which is an enough scope to update the existing video source because the doc says,
manage_videos: Allows to modify or delete the user's uploaded videos and to publish new ones.
and described above,
Only the title field of the video is updated with the input_title properly.
Thanks for reading, I researched their docs carefully, but still don't understand this response.
Only partner users can update the video source urls.
Best,
I'm trying to use OAuth2 for Server to Server Applications in conjunction with Google's Content API for Shopping using the google-api-client gem and Ruby on Rails 3.2.5. Also, I have already set up my merchant account as prescribed in the Content API documentation.
This was the best way I found to be able to:
create/update products in the background
have created products fall under my company's Google Products 'umbrella'
not require every user to authenticate/authorize when their token expires
Using lines 1 - 23 from this sample as a starting point, I've begun to write the following module for use in background jobs:
require 'httparty'
require 'google/api_client'
module GoogleProducts
GOOGLE_CONFIG = YAML.load_file(File.join(Rails.root, "config", "google.yml"))[Rails.env]
CLIENT_ID = "XXXXXXXXXXXX#developer.gserviceaccount.com"
MERCHANT_ID = "XXXXXXX"
SCOPE = "https://www.googleapis.com/auth/structuredcontent"
KEY_FILE_PATH = File.join(Rails.root, "config", "my-privatekey.p12")
KEY_FILE_PASS = "XXXXXXXXXX"
def self.add_item(item_id)
self.fetch_token
xml = self.gen_item_xml(item_id)
headers = {"Content-type" => "application/atom+xml", "Content-Length" => xml.length.to_s}
url = "https://content.googleapis.com/content/v1/#{MERCHANT_ID}/items/products/generic?access_token=#{$gp_token}"
response = HTTParty.post(url, :body => xml, :headers => headers).parsed_response
end
def self.gen_item_xml(item_id)
#building product xml
end
private
def self.fetch_token
api_client = Google::APIClient.new(:authorization => :oauth2)
key = Google::APIClient::PKCS12.load_key(KEY_FILE_PATH, KEY_FILE_PASS)
asserter = Google::APIClient::JWTAsserter.new(CLIENT_ID, SCOPE, key)
begin
api_client.authorization = asserter.authorize
#todo - store in something other than a global
$gp_token = api_client.authorization.access_token
rescue Signet::AuthorizationError => e
puts e.message
ensure
return $gp_token
end
end
end
Everything seemingly works fine - the authentication, the handling of the auth token - until I attempt to actually add an item, which I get the following when I do:
<errors xmlns='http://schemas.google.com/g/2005'>
<error>
<domain>GData</domain>
<code>ServiceForbiddenException</code>
<internalReason>Could not find authenticated customer</internalReason>
</error>
</errors>
Any ideas?
After much anguish and mental toil, I've finally solved my issue!
Since I am using OAuth 2 Server to Server authentication the suggestion hjblok gave didn't apply (thanks for giving it a shot, though!).
I simply added the email address that was associated with my Service Account key from the Google API Console (e.g. XXXXXXXXXXXX#developer.gserviceaccount.com) to my Google Merchant account (Settings > Users on the merchant admin page), and it worked.
If there's any clarification needed, please feel free to comment!
The Google Content API documentation says you need to set it up in the Settings page of the Google Merchant Center:
https://developers.google.com/shopping-content/getting-started/usingapi-products
EDIT rewrote the answer after diving into the Google's API documentation
Did you already try to use Google's OAuth 2.0 playground? I was able to successfully access https://content.googleapis.com/content/v1/#{MERCHANT_ID}/items/products/generic.
In "Step 1" I've chosen the "Content API for Shopping" and then authorized the API with my account.
Then in "Step 2" I've "exchanged authorization code for tokens", which results in a "refresh token" and an "access token".
Then in "Step 3" I've invoked a GET request to https://content.googleapis.com/content/v1/1234567/items/products/generic. Because 1234567 is not a valid MERCHANT_ID it returns an Error. But the Error Messages contains a MERCHANT_ID which actually belongs to your account.
I repeated "Step 3" but now with the correct MERCHANT_ID. Which returns a HTTP/1.1 200 OK with the requested items in the body.
Furthermore I'm not sure, but doesn't Google API expect an Authorization header to be present with the access_token ($gp_token)? Within the OAuth 2.0 playground this Authorization header is used to sent the access_token.
I also found the Structured Content API demo page (https://google-content-api-tools.appspot.com/demo/demo.html), which is more specific to the Content API for Shopping.
is any possible & simple way to make a twitter bot that will reply to some tweets (depend on search terms) in certain time interval. can anyone help me.
for example twitter.com/shastribot
Thanks
If you like Ruby, then I suggest using the Twitter gem: https://github.com/jnunemaker/twitter
It makes things very easy.
You could then write a script that checks whether there are any replies to the bot and if there are any new ones sends out a message. Then set it up as a cron job running as often as you think is necessary.
There's also the Twitter Bot interface to Twitter, I haven't used it myself but might be worth a look: http://integrum.rubyforge.org/twitter_bot/
You should try tweebot. It's python micro framework for twitter bots. This lib provides built-in blocks (like Filters, Selectors and Actions) that you can combine to achieve your requirements. For example, next code demonstrates how-to create canonical implementation of "retweet" bot (more examples).
# Next code demonstrates how to create simple twitter bot that select all
# friends' tweets with your mentiones and retweet they.
import tweebot as twb
def main():
# Step 1. setup context configuration
repeater = twb.Context({
'app_name' : 'repeater',
'username' : '<YOUR ACCOUNT NAME>',
'consumer_key' : '<YOUR CONSUMER KEY>',
'consumer_secret' : '<YOUR CONSUMER SECRET>',
'access_key' : '<YOUR ACCESS KEY>',
'access_secret' : '<YOUR ACCESS SECRET>',
'timeout' : 10 * 60, # 10 min, ensure twitter api limits
'history_file' : 'history.json', # don't repeat answered tweets
})
# Step 2. enable pretty logging (stdout by default)
twb.enable_logging(repeater)
# Step 3. setup chain Selector->Filters->Action
chain = (
# Select recently tweets with current user mentions.
twb.SearchMentions(),
# Apply several filters to selected tweets:
twb.MultiPart.And(
# exclude answered, blocked and own tweets
twb.BaseFilter,
# then leave only friends tweets (friends list will be cached)
twb.UsersFilter.Friends(),
# and finally, exclude tweets with invalid content
twb.BadTweetFilter),
# And now, retweet remain tweets
twb.ReplyRetweet)
# Step 4. start processing
repeater.start_forever(*chain)
if __name__ == '__main__':
main()
Ruby's twitter gem is a very good one. You can make use of twitter API to see the available methods.
You can start with a Twitter::REST::Client like following:
twitter_client = Twitter::REST::Client.new do |config|
config.consumer_key = "YOUR_CONSUMER_KEY"
config.consumer_secret = "YOUR_CONSUMER_SECRET"
config.access_token = "YOUR_ACCESS_TOKEN"
config.access_token_secret = "YOUR_ACCESS_SECRET"
end
Then you can you use your twitter_client for various purpose. For example you can post a tweet to your profile using this:
twitter_client.update("I am posting this tweet from my Ruby program")
You can get a list of all tweets by providing the twitter username like this:
twitter_client.user_timeline("YOUR_TWITTER_USER_NAME").each do |tweet|
puts tweet.text
end
For searching for tweets, take a look at this.