I do some analysis on twitter users features like number of following , number of retweet, number of friend , etc
I have all my information from Twitter Rest API
But there is an Rate Limit Exceeded error occurred when I tried to retrieve all data
can I have all these data from Twitter Streaming API , if I can , how I can you it?
If not what Is the Solution?
Thanks for Help
On each response you can call getRateLimitStatus() to get a RateLimitStatus and if remaining calls is 0, sleep the thread till time limit is over.
do {
TwitterResponse response = twitter.getFollowersIDs(userId, cursor);
RateLimitStatus status = response.getRateLimitStatus();
if(status.getRemaining() == 0) {
try {
Thread.sleep(status.getSecondsUntilReset() * 1000);
}
catch(InterruptedException e) {
// ...
}
}
} while(cursor > 0);
Twitter API update limits error 403
Related
I tried to get the redirect url in google sheet with the cache function to avoid the error message: Service invoked for too many times in one day. However, the following code return an error message, may i know how to fix it
function getRedirects(url) {
var params = {
'followRedirects': false,
'muteHttpExceptions': true
};
var followedUrls = [url];
var cache = CacheService.getScriptCache();
var properties = PropertiesService.getScriptProperties();
try {
let res = cache.get(url);
while (true) {
var res = UrlFetchApp.fetch(url, params);
if (res.getResponseCode() < 300 || res.getResponseCode() > 399) {
return followedUrls;
}
var url = res.getHeaders()['Location'];
followedUrls.push(url);
}
}
}
You are reaching Apps Script quota limit since there's a while-true
As per the documentation Quotas are set at different levels for users of consumer (such as gmail.com). Which in your case is URL Fetch calls - 20,000 / day
I'd change the while (true) line and foresee how many requests will be made beforehand. When you use var res = UrlFetchApp.fetch(url, params); a request / day is consumed so keep in mind that unless you own a Workspace Account you won't be able to change your quota limit.
Reference
Quotas for Google Services
I'm trying to use the egg_mode crate to retrieve a stream of tweets as per the example here but with just a slight difference in the way I input my token.
#[tokio::main]
async fn main() {
let con_token = egg_mode::KeyPair::new(CONSUMER_KEY, CONSUMER_SECRET);
println!("Live streaming tweets...");
println!("Ctrl-C to quit\n");
let stream = egg_mode::stream::filter()
.track(&["rustlang"])
.start(&Token::Bearer(BEARERTOKEN.to_string()))
.try_for_each(|m| {
if let StreamMessage::Tweet(tweet) = m {
println!("{}\n{}",tweet.created_at,tweet.text);
} else {
println!("{:?}",m);
}
futures::future::ok(())
});
if let Err(e) = stream.await {
println!("Stream error: {}", e);
println!("Disconnected")
}
}
However I keep getting a 401 unauthorized error.
Using the same bearer token, I am able to retrieve tweets with no problem using egg_mode::tweet::user_timeline as per the example here.
What could be the problem?
It has succeeded after passing the Access variant of the Token enum instead of the Bearer variant.
is there any API to check monetize status of Youtube channel? also for youtube video. i try youtube data api but not getting.or any other api to know that monetization is on or off.
I know that this was asked a long time ago, but I still found the need of this nowadays. So after searching a lot inside YouTube API documentation I figure out that Google doesn't provide this, so I create a little workaround.
Basically what I do is generate an analytics report using the metric estimatedRevenue, If I gof 'Forbidden error' or 403 I tried again changing the metrics to not have estimatedRevenue. If this time a get the report data without errors that means that this channel doesn't have any revenue and therefore is not monetized.
I'm using nodejs to illustrate this because it's what I'm using in my project but you can adapt to any other language. You may want to look at the Google official client libraries:
https://developers.google.com/youtube/v3/libraries
The code snippet comes to this:
let isMonetized = true;
this.metrics = 'views,comments,likes,dislikes,estimatedMinutesWatched,grossRevenue,estimatedRevenue'
while (true) {
try {
const youtubeAnalytics = googleapis.google.youtubeAnalytics({ version: 'v2', auth });
const response = await youtubeAnalytics.reports
.query({
endDate: '2030-12-30',
ids: 'channel==MINE',
metrics: this.reportMetrics,
startDate: '2000-01-01',
});
const responseData = response.data;
const analyticsInfo = {
channelId,
channelName: youtubeTokens[channelId].channelName,
views: responseData.rows[0][0],
comments: responseData.rows[0][1],
likes: responseData.rows[0][2],
dislikes: responseData.rows[0][3],
estimatedMinutesWatched: responseData.rows[0][4],
grossRevenue: responseData.rows[0][5] !== undefined
? responseData.rows[0][5]
: 'Not monetized',
estimatedRevenue: responseData.rows[0][6] !== undefined
? responseData.rows[0][6]
: 'Not monetized',
};
return analyticsInfo;
} catch (error) {
if (error.code === 403 && isMonetized) {
console.log('Could not get reports. Trying again without revenue metrics');
this.reportMetrics = 'views,comments,likes,dislikes,estimatedMinutesWatched';
isMonetized = false;
} else {
console.log(error);
return false;
}
}
}
I want to get tweets by particular user by entering its userId.
I can search for a text by:
Query query = new Query("Hi");
QueryResult result;
do {
result = twitter.search(query);
List<Status> tweets = result.getTweets();
for (Status tweet : tweets) {
System.out.println("#" + tweet.getUser().getScreenName() +
" - " + tweet.getText());
}
} while ((query = result.nextQuery()) != null);
but how can I search for the tweets by entering particular userId, is there any direct method or I have to apply logic ?
I am trying:
Status status = twitter.showStatus(id);
if (status != null){
System.out.println("#" + status.getUser().getScreenName()
+ " - " + status.getText());}
where id is userId, but by doing this, I am getting the error:
Failed to search tweets: 404:The URI requested is invalid or the
resource requested, such as a user, does not exists. Also returned
when the requested format is not supported by the requested method.
message - No status found with that ID. code - 144
Can anyone please help me with this?
With the Twitter API you can get up to ~3200 tweets from an user, to do this you can get the time line from an specific user, see those questions
Get tweets of a public twitter profile
Twitter4J: Get all statuses from Twitter account
By the way, you are getting that error because you are using twitter.showStatus(id); with an userid, you need to call twitter.showUser(id) and you won't get that error
I have a list of tweets with information about the user who tweeted them that I am using for an undergrad research project. To build a social network graph of these tweets I need to grab their friend and follower lists. I have tried using the GET Follower IDs call through the twitter4j platform. My authentication is Oauth with Read, write, and direct messages. I get a 400 response code with no further error code. I also get the following exception code
exceptionCode=[92c30ec6-19bed99c 70a5018f-1e1c55ac 70a5018f-1e1c55aa], statusCode=-1, message=null, code=-1, retryAfter=-1, rateLimitStatus=null, version=3.0.3}
This tells me that I'm not authenticated to make this request which from what I have read is because the people are not followers of mine. Is there a way I can request this information without having this relationship with the user?
here is my code
public static void main (String[] args){
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setDebugEnabled(true)
.setOAuthConsumerKey("something")
.setOAuthConsumerSecret("something else")
.setOAuthAccessToken("another thing")
.setOAuthAccessTokenSecret("a secret thing")
.setUseSSL(true)
.setUserStreamRepliesAllEnabled(true);
Twitter twitter = new TwitterFactory(cb.build()).getInstance();
long cursor = -1;
IDs ids = null;
String[] users = new String[16717];
BufferedReader br = null;
try {//getting user screen names
String sCurrentLine;
br = new BufferedReader(new FileReader("users.txt"));
int i = 0;
while ((sCurrentLine = br.readLine()) != null) {
users[i]=sCurrentLine;
i++;
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (br != null)br.close();
} catch (IOException ex) {
ex.printStackTrace();
}
}
for(int i=0;i<users.length;i++){
System.out.println("==================="+users[i]+"===================");
do {
try {
ids = twitter.getFollowersIDs(users[i], cursor);
for (long id : ids.getIDs()) {
System.out.println(id);
User user = twitter.showUser(id);
System.out.println(user.getName());
}
} catch (TwitterException e) {
e.printStackTrace();
}
} while ((cursor = ids.getNextCursor()) != 0);
}
}
One can obtain the list of followers IDs for any public twitter user using this API from twitter. I don't use twitter4j but it should work fine.
Main thing to be conscious of, outside of authentication, is that twitter allows fetching maximum 5000 IDs in one call and rate limits aggressively (15 calls per app/user token) so your application has to be designed and built to honor those considerations/limitations with appropriate tokens/sleeps etc.
For e.g. if you use the application token and a given user has 100K followers, twitter will start returning rate_limit_exceeded errors after fetching 75K (5K * 15) followers IDs.
In an extremely embarrassing turn of events it turns out that the reason the request could not be made is that the usernames I was searching with had an extra space. So I trimmed each name and it works now.