Searching through a Twitter feed for an Alexa Skill - twitter

I'm working on a fairly basic Alexa skill that, in essence, searches through a specific Twitter feed looking for a hashtag, parses that tweet, and reads it back. What's the simplest way to pull data from a Twitter feed? I've been having issues working with the Twitter API (see below) and scraping data from Twitter appears to be against the TOS.
... crawling the Services is permissible if done in accordance with the provisions of the robots.txt file, however, scraping the Services without the prior consent of Twitter is expressly prohibited.
To write an Alexa skill
Follow a tutorial.
Simple enough. Use a pretty interface to add an invocation, some intents, and a slot type with the data I want.
Write an AWS Lambda function to handle everything.
Python because I know it better than JavaScript. Pick one of the Python wrappers for the Twitter API, then realize that because Twitter, all of their API requires authentication - even basic searches.
Register an app with Twitter so I have keys and tokens.
Realize having authentication keys and tokens in plaintext in an application is a bad idea, and decide to figure out account linking for Alexa.
Try following the one tutorial around, twice. Have trouble and go to StackExchange.
Why the need for an external webapp?
...we need an OAuth implementation of our own to make the integration work correctly
What's wrong with the one provided by Twitter? Why can't any issues be fixed within the Lambda method, since the account integration isn't being touched otherwise AFAIK? Isn't having the tokens passed around via the URL a bad idea too? Their example code seems to require that the Consumer Secret be hard coded too.
Enter: “https://alexa-twitter-airport-info.herokuapp.com/oauth/request_token?vendor_id=XXXXXX&consumer_key=YYYYYY&consumer_secret=ZZZZZZ”.
At the very least, their webapp seems to be down for the time being, and it'd be nice to have an option that doesn't require paying money to host another copy.
I've seen this post discussing a Node.js OAuth implementation, but the necessity for such an implementation still escapes me.

I found twitter Node package useful doing this. I used search/tweets GET API for my purpose which is shown below. However you have various APIs which are available to work with the tweets. The code below only shows the parts needed to access Twitter API from within Node JS.
var Twitter = require('twitter');
var client = new Twitter({
// I added the keys as AWS Environment variables
consumer_key: process.env.consumer_key,
consumer_secret: process.env.consumer_secret,
access_token_key: process.env.access_token_key,
access_token_secret: process.env.access_token_secret
});
client.get('search/tweets', {q: <<your search query>>,count:<<if you want to receive specific number of tweets>>}, function(error, tweets, response) {
var noOfTweets = tweets.statuses.length;
if(error) {
speechOutput = "I could not find tweets for <<your reason>>"
self.emit(':tell', speechOutput);
} else if(noOfTweets === 0) {
speechOutput = "another speech output";
self.emit(':tell', speechOutput);
} else {
tweets.statuses.forEach(function(tweet) {
//process the tweet the way you want to.
});
}
});

Related

Use Tweepy to extract Twitter follower information (API incompatibility issues)?

I was following this tutorial, https://towardsdatascience.com/how-to-download-twitter-friends-or-followers-for-free-b9d5ac23812, which was written in 2021. It should've worked fine, however, they have to 'fix' the things that just work.
Specifically, running this line
for fid in Cursor(api.followers_ids, screen_name=screen_name, count=5000).items():
ids.append(fid)
gives the error:
"tweepy.error.TweepError: [{'message': 'You currently have Essential access which includes access to Twitter API v2 endpoints only. If you need access to this endpoint, you’ll need to apply for Elevated access via the Developer Portal. You can learn more here: https://developer.twitter.com/en/docs/twitter-api/getting-started/about-twitter-api#v2-access-leve', 'code': 453}]"
I could have pulled the data in five minutes. Now debugging this already cost one hour+ because they just break the things that work. Is there anyway to make this old code snippet work? The application to use API 1.1 takes weeks, and I don't have time to watch their bad documents of how to migrate from API 1.1 to 2.0 and then the documents of migrating from Tweepy 3.9.0 to 4.0.0. Five minutes' task would just become half a day. Thanks in advance for any help.
First of all, have you at least tried to apply for the Elevated access?
It can take some time, it's true, but it can also be instantaneous.
The other solution would be to use the Twitter API V2.
You don't need any tutorial, just read the documentation:
Here for the authentication ;
Here for the retrieval of the followers ;
Here for the pagination.
And you should get something like that:
import tweepy
client = tweepy.Client("Bearer Token here")
paginator = tweepy.Paginator(
client.get_users_followers,
id=..., # ID only, no screename
max_results=1000
).flatten()
for follower in paginator:
print(follower.id)
Finally, even if I understand your frustration (and developing Twitter applications can be very frustrating), I think that you should try to keep it out your SO questions. Good luck!

How do I use the oauth2 access token with the youtube data api

I am able to generate an oauth2 access token (from a refresh token), which I believe should give me the ability to access the youtube data api functionally to delete/upload content.
Using python and the youtube api I need to delete and upload a (new) video to youtube periodically, say hourly.
All google python samples I've found seem to call the "DENY/ALLOW" screen which requires a copy/paste back in the calling app.
I can do this occasionally but otherwise want the process to be automated. I've read about service accounts which, which according to the linked post, are not supported by the youtube api. Offline access et.al. is also mentioned but in somewhat abstract terms i.e. no concrete python examples (that I have yet found). Another source mentioned an http get like below:
"GET access_token=ya29.GlxBBS89....ast987&part=snippet&mine=true"
but the following in python doesn't seem to work returning "response [400]" (bad request)
url = 'https://www.googleapis.com/youtube/v3/channels'
args = 'access_token: ' + token var + ', part: snippet, mine: true'
get_token = requests.get(url, data = args)
I have used Can we use google youtube data api without OAuth (and others) to get to this stage but need clarification for the next step.
**********************************Update*********************************
I have found that I can only generate access tokens for clients credentials configured as web apps. I am writing a desktop app so I may be barking up the wrong tree.
Or learning Django...
I found examples at https://developers.google.com/youtube/v3/guides/auth/installed-apps that helped.
curl -H "Authorization: Bearer <access_token>" https://www.googleapis.com/youtube/v3/channels?part=snippet&mine=true
curl https://www.googleapis.com/youtube/v3/channels?access_token=<access_token>&part=snippet&mine=true
The curl samples especially provided confirmation that I'm reaching the endpoint and returned helpful debugging info. They have exposed other issues which I'll ask in another question.

How to use Twitter User Streams

I am new in Twitter API. I have a requirement like I have to pull the information of an authenticated twitter user to know the number of followers. I can do this using REST API but I have a problem here. So I have taken the way to use Twitter User Streams.
I am not able to integrate this functionality in the site. I am using twitter console( https://dev.twitter.com/console ). But there I am not able to get any kind of result, it actually hangs/shows gateway timeout. And that's for I am not able to configure that how to write the code for this or how to use this in site. I am not passing any kind of parameters.
So, any body can please help in this would be very grateful( a clean sample code for this or any link where to get the good documentation ). I have already checked with twitter dev documentations.
Thanks in advance :)

How to get results with twitter search api?

I wish to get tweets with a keyword. But There is no result with any keyword.
http://search.twitter.com/search.json?q=summer
How to get results with Twitter Search API?
Version 1 of the twitter API has been deprecated and is being removed. Not sure how you can miss the giant warnings on the twitter dev site ;) This means simple code like the above will not work any more.
So, you now need to make authenticated requests (OAuth) using the 1.1 API, and it's nowhere near as simple as just doing a (in PHP) file_get_contents(http://search.twitter.com/ ...).
I couldn't see any server-side languages you use from your profile, but I wrote a lengthy post explaining the issue (with pictures) and how to use a php library to perform authenticated requests.

Twitter tweets about a certain url via salmon and/or pubsubhubbub?

For a web-application I am writing (Rails) I am looking for ways to post back publick mentions about that post. Mentions on Twitter, Facebook, Reddit and so on.
Starting with Twitter, I am looking for a way to get these tweets back into the application. Obviously I could simply monitor the twitter firehose (through their search, most probably) for a certain url (the url of the post), if found, extract that and place that tweet as new comment to the post. But this requires me to invent something similar for each and every social network.
Whereas there is a generic protocol for this: Salmon, allowing services to push such comments (tweets mentioning your url) through PubSubHubbub (PSH).
I did not find any mention of this on the Twitter API itself; they are not using PSH or Salmon (yet) it seems.
But maybe there are thirdpartis that do? Know any? Are there other projects that wrap around Twitters firehose and allow you to approach Twitter as if it is a Salmon-speaking pubsubhubbub? And What about Reddit, or Facebook?
The Twitter API TOS explicitly forbids the re-distribution of tweets via APIs*, except for Twitter's partners. These are DataSift and Gnip - but they don't do what you are looking for.
Your best option would be to simply monitor the filterhose.
* Actually, it's allowed to share IDs, but those are pretty useless since you have to look them up manually.

Resources