Use Tweepy to extract Twitter follower information (API incompatibility issues)? - twitter

I was following this tutorial, https://towardsdatascience.com/how-to-download-twitter-friends-or-followers-for-free-b9d5ac23812, which was written in 2021. It should've worked fine, however, they have to 'fix' the things that just work.
Specifically, running this line
for fid in Cursor(api.followers_ids, screen_name=screen_name, count=5000).items():
ids.append(fid)
gives the error:
"tweepy.error.TweepError: [{'message': 'You currently have Essential access which includes access to Twitter API v2 endpoints only. If you need access to this endpoint, you’ll need to apply for Elevated access via the Developer Portal. You can learn more here: https://developer.twitter.com/en/docs/twitter-api/getting-started/about-twitter-api#v2-access-leve', 'code': 453}]"
I could have pulled the data in five minutes. Now debugging this already cost one hour+ because they just break the things that work. Is there anyway to make this old code snippet work? The application to use API 1.1 takes weeks, and I don't have time to watch their bad documents of how to migrate from API 1.1 to 2.0 and then the documents of migrating from Tweepy 3.9.0 to 4.0.0. Five minutes' task would just become half a day. Thanks in advance for any help.

First of all, have you at least tried to apply for the Elevated access?
It can take some time, it's true, but it can also be instantaneous.
The other solution would be to use the Twitter API V2.
You don't need any tutorial, just read the documentation:
Here for the authentication ;
Here for the retrieval of the followers ;
Here for the pagination.
And you should get something like that:
import tweepy
client = tweepy.Client("Bearer Token here")
paginator = tweepy.Paginator(
client.get_users_followers,
id=..., # ID only, no screename
max_results=1000
).flatten()
for follower in paginator:
print(follower.id)
Finally, even if I understand your frustration (and developing Twitter applications can be very frustrating), I think that you should try to keep it out your SO questions. Good luck!

Related

What was the latest version of google-oauth2 updated?

We are using the following url for Google OAuth2 v3 now.
https://accounts.google.com/o/oauth2/auth
https://www.googleapis.com/oauth2/v3/token
But, I confirmed version-up in the latest document(https://developers.google.com/identity/protocols/OAuth2WebServer).
https://accounts.google.com/o/oauth2/v2/auth
https://www.googleapis.com/oauth2/v4/token
What was changed?
If possible, we want to update it.
This is the third time i think i have seen them change them in the last five years. Exactly why they change them they never inform us. I would suspect that the new endpoints are OpenIDConnect compliant.
The old ones should continue to work for a while i have not heard anything about them shutting them down.
You can also check the Discovery document

Google YOLO stop working : The client origin is not permitted to use this API

I assume it has something to do with this:
For me Google one Tap stopped working on all my sites that previously worked. I added API HTTP refer to restriction in console.developer.com, but I still get a warning message "The client origin is not permitted to use this API." any thoughts? If you go to the page https://www.wego.com/ you can see that Google one tap still works...
https://news.ycombinator.com/item?id=17044518#17045809
but Google YOLO stop working for everyone. I use it like many people for login and it just stop work.
My domain are obviously added on console.developers.google.com
Any ETA for fix this? Some information would be great for people who rely on it.
Google YOLO is not disabled. It is open to a small list of Google Partners.
The reason you were able to access it earlier was because it was open for a short period of time but the whitelist is now readded/enabled.
Reference:
https://twitter.com/sirdarckcat/status/994867137704587264
Google YOLO was put on whitelist after a client-side exploit became clear to google.
People could cover the login button of the prompt with something like a cookie consent (which we all know people automatically accept).
Therefor people could easily steal their gmail or other details due to this google decided to put it on whitelist and review the sites that are using this technology in order to ensure that they are using it as they should.
Google retroactively labeled One-Tap as a "closed beta".
https://developers.google.com/identity/one-tap/web
The beta test program for this API is currently closed. We are improving the API's cross-browser functionality and will provide updates here in the coming months.
The link for the entire project is currently 404, but the beta statement is visible on the wayback machine.

How to get results with twitter search api?

I wish to get tweets with a keyword. But There is no result with any keyword.
http://search.twitter.com/search.json?q=summer
How to get results with Twitter Search API?
Version 1 of the twitter API has been deprecated and is being removed. Not sure how you can miss the giant warnings on the twitter dev site ;) This means simple code like the above will not work any more.
So, you now need to make authenticated requests (OAuth) using the 1.1 API, and it's nowhere near as simple as just doing a (in PHP) file_get_contents(http://search.twitter.com/ ...).
I couldn't see any server-side languages you use from your profile, but I wrote a lengthy post explaining the issue (with pictures) and how to use a php library to perform authenticated requests.

Migrating command line script to FT API 1.0

I have a simple server-side command line python script, based on the now deprecated FT SQL API using Client Login for authorization.
Every few hours I refresh my FT by inserting the updated data. It's a small table, 3000 rows.
Read the FT Migration Guide, have a Google User id, enabled the FT API and have an API key, etc.
(Switching my Google Maps application to the new API was straight forward)
But I'm confused about how to proceed with migrating a command line python script. I see that OAuth2 is the recommended approach for authentication but it appears to me that this is a much more complicated approach than I truly need. It's not even clear to me that it will work for command line scripts.
Truly no need to prompt the user for login info (it's only me). I realize that this must be an "Installed app" and I found the Hello,python example at
https://developers.google.com/fusiontables/docs/sample_code#ftAPIv1
Haven't tried this yet but wanted to see if anyone has experience with migrating command line scripts. (PHP would work as well)
In particular I was intrigued by a quote I found here:
https://developers.google.com/fusiontables/docs/v1/using
"The Fusion Tables API also supports older authorization options, such as OAuth 1.0, AuthSub, or ClientLogin; however, in most cases we don't recommend using those other options. If your application already uses those options, we recommend migrating to OAuth 2.0 if possible."
Because, frankly I'd rather not switch to OAuth 2 for such a simple task.
UPDATE
Forgot to mention, I run this from cron, every 3 hours. So user input is not really possible. Though as David suggested below a one time user input is doable.
You should be able to use your Client Login token as your auth token for your existing server-side command line app with the new API. You'll also need to include your developer key in the request as the "key" parameter.
There is a very simple python implementation of the OAuth2 Installed Application flow here https://developers.google.com/fusiontables/docs/samples/python as you may have found, and yes, you should switch to this. ClientLogin deprecated just like the SQL API.
The Code requires you to open a browser and prompts you every time you use it and I'm going to speak to the author about optimizing it and having it store the Refresh Token and so it would only prompt the first time you use it.

Google Docs upload works on some accounts but not others

I have written a client in C to upload files to Google Docs, I get an authentication token, and use it for the upload. However, I have 4 Google accounts, the upload worked on 2 accounts, but not the other 2. And now, it only works on one account.
On authentication, all accounts behave the same, and ClientLogin issues an "Auth=xxx" token. I use the token 100% successfully on one Google account, it used to work on another, and has never worked on the other 2.
The only error is "HTTP/1.1 401 Token invalid".
One account used to work, and now does not, which does not make much sense. The ClientLogin calls never seem to ask for a CAPTCHA.
Currently I can't invest the time to move to the Drive API, and that may present the same issues anyway.
Any thoughts?
Thank you
Garry
Try them individually first, if this works you can start combining untill all 4 works,
doing it step by step is the way to succes, I can't help you with programming code, according to the fact that I'm just a noobie :) tho I thought myself PHP by just doing it step by step, if you got 1 done, then it's just copy/paste and change abit :)
Goodluck tho :)
You should move your code away from ClientLogin and use OAuth 2.0 instead. You didn't mention what language your script uses but Google has some samples for performing OAuth 2.0 at:
https://developers.google.com/google-apps/documents-list/#authorizing_requests_with_oauth_20
and
https://developers.google.com/api-client-library/
Jay

Resources