I want to fetch various social media accounts data through Radian6 API (particularly Twitter).
I have my defined url as:
headers = {"access_token": key}
url = 'https://api.socialstudio.radian6.com/v3/posts?topics=1234567&startDate='
+ startDate + '&endDate=' + endDate
resp = session.get(url, headers=headers)
Json = json.loads(resp.content)
where duration between start and end date covers all my records in the current day.
In the JSON that I am getting, for the data fetched from sources other than twitter, I am getting the content text but for tweets, I am not getting any data.
Do I need to add special privileges to my account or is getting tweets data not possible through the API, or is it I am doing something wrong. Need help in this.
Related
I have the Academic Research access to Twitter's API and have been using Tweepy to access the API. My problem is I cannot retrieve the tweets from older tweets
This is the code attempting to retrieve the tweets using the conversation_id, from 2014
# https://twitter.com/NintendoAmerica/status/535462600294035456
start_time = '2014-11-01T00:00:00Z'
end_time = '2014-12-12T00:00:00Z'
tweets = client.search_all_tweets(query = 'conversation_id:535462600294035456', max_results = 500, start_time=start_time, end_time=end_time)
and the output is:
Response(data=[<Tweet id=535465221679489024 text='#NintendoAmerica #Pokemon [this was a link I had to remove]'>], includes={}, errors=[], meta={'newest_id': '535465221679489024', 'oldest_id': '535465221679489024', 'result_count': 1})
which is only one seemingly random tweet amongst many.
However, when I tried running the same code on a more recent tweet, it retrieved all the tweets. I do not have to specify a start/end time because it's a tweet from the past 30 days.
# https://twitter.com/380kmh/status/1545477360916373504
tweets = client.search_all_tweets(query = 'conversation_id:1545477360916373504', max_results = 500)
the output was complete (shortened, I removed the tweets):
Response(data=[...], meta={'newest_id': '1546465585093087235', 'oldest_id': '1545477768229670912', 'result_count': 18})
I followed Tweepy's documentation here, using Client.search_all_tweets:
https://docs.tweepy.org/en/stable/client.html#tweepy.Client.search_all_tweets
I also tried using Postman to retrieve the tweets but it came out empty, even though I followed the documentation here:
https://developer.twitter.com/en/docs/twitter-api/tweets/search/api-reference/get-tweets-search-all#tab0
Query: https://api.twitter.com/2/tweets/search/all?query=conversation_id%3A537923834557771776&start_time=2014-11-01T00:00:00.000Z&end_time=2014-12-18T00:00:00.000Z&tweet.fields=in_reply_to_user_id,text
Output:
{
"meta": {
"result_count": 0
}
}
What am I doing wrong?
We got to build our own reporting database for our Youtube channel to measure the channel and video performance.
To support this, we implemented an ETL job to extract using Youtube Analytics API and used below python code to get the data.
def GetAnalyticsData(extractDate,accessToken, channelId):
channelId = 'channel%3D%3D{0}'.format(channelId)
headers = {'Authorization': 'Bearer {}'.format(accessToken),
'accept': 'application/json'}
url = 'https://youtubeanalytics.googleapis.com/v2/reports?dimensions={dimensions}&endDate={enddate}&ids={ids}&maxResults={maxresults}&metrics={metrics}&startDate={startdate}&alt={alt}&sort={sort}'.format(
dimensions='video',
ids=channelId,
enddate= extractDate,
startdate=extractDate,
metrics = 'views%2Ccomments%2Clikes%2Cdislikes%2Cshares%2CestimatedMinutesWatched%2CsubscribersGained%2CsubscribersLost%2CannotationClicks%2CannotationClickThroughRate%2CaverageViewDuration%2CaverageViewPercentage%2CannotationCloseRate%2CannotationImpressions%2CannotationClickableImpressions%2CannotationClosableImpressions%2CannotationCloses',
maxresults = 200,
alt ='json',
sort='-views'
)
return requests.get(url,headers=headers)
We hit this API everyday and get all the video metric and sorted by views in descending order.
This solved our need partially and it returns only 200 videos, if we specify maxResults more than 200, its return 400 error code.
The challenge is, how to get all videos for the given date and given channel?
Thanks in advance.
Regards,
Guna
I am not keen on YouTube analytics API, but it seems that you are looking for startIndex.
startIndex
integer
The 1-based index of the first entity to retrieve. (The default value is 1.) Use this parameter as a pagination mechanism along with the max-results parameter.
I am trying to make an application for an organization which will required to fetch all past as well present tweets with some particular hashtag like #airtel, #airtel etc, how should I get past tweet, I am able to fetch the present tweet with the following url : "https://api.twitter.com/1.1/search/tweets.json?q=%23airtel"
Thanks
You can get up to a maximum of 100 tweets with the twitter rest api see the following twitter documentation. The best you can do is use the count parameter https://api.twitter.com/1.1/search/tweets.json?q=%23airtel&count=100
After various search on Google I found some useful library for fetching tweets e.g:
TwitterSearch [https://github.com/ckoepp/TwitterSearch], you would I find the twitter profile # https://twitter.com/twittersearch
Tweepy [https://github.com/tweepy/tweepy], you could find more info at http://www.tweepy.org/
I have implemented using both. Tweepy implementation is as follows:
import tweepy
import json
consumer_key = "get"
consumer_secret = "From"
access_token = "Twitter"
access_token_secret = "site"
# Authenticate twitter Api
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)
#made a cursor
c = tweepy.Cursor(api.search, q='%23Coursera')
c.pages(15) # you can change it make get tweets
#Lets save the selected part of the tweets inot json
tweetJson = []
for tweet in c.items():
if tweet.lang == 'en':
createdAt = str(tweet.created_at)
authorCreatedAt = str(tweet.author.created_at)
tweetJson.append(
{'tweetText':tweet.text,
'tweetCreatedAt':createdAt,
'authorName': tweet.author.name,
})
#dump the data into json format
print json.dumps(tweetJson)
If any one have problem, let me know, will provide git repo for this.
Thanks
Krishna
I am new to Twitter APIs. I am implementing a module in Java which shows popularity of a #hashtag. So I have to collect tweets for regular time interval.
Is there a way to get all the tweets that contains particular #hashtag, so I can the count total tweets each time and get the popularity? Below URL give me tweets but it's not consistent. Doesn't give me all tweets.
http://search.twitter.com/search.json?q=%23example
Why are you using urls use Api Twitter 4j
http://twitter4j.org/en/
You can perform this function eaisly with that
With Twitter4J, you can easily integrate your Java application with the Twitter service.
But Twitter4J is an unofficial library.
Use this function of that api
Search for Tweets
// The factory instance is re-useable and thread safe.
Twitter twitter = TwitterFactory.getSingleton();
Query query = new Query("source:twitter4j yusukey");
QueryResult result = twitter.search(query);
for (Status status : result.getStatuses()) {
System.out.println("#" + status.getUser().getScreenName() + ":" + status.getText());
}
Check this url for snippets
http://twitter4j.org/en/code-examples.html
Is it possible to get tweets which contain photos? I am currently using twitter search api and getting all the tweets having entity details by setting include_entities=true. I see picture details under media object but is there anyway to filter and get tweets objects which just have these media items. Or is there anyway in Twitter4j to do this query?
There is no specific way to specify that I need only photos or videos but you can filter the results based on filter:links or filter:images or filter:videos in your query along with include_entities=true.
For Example: To get the tweets that contain links since 2012-01-31, you query should have include_entities parameter as well as filter:links as shown as follows:
https://search.twitter.com/search.json?q=from%3Agoogle%20since%3A2012-01-31%20filter%3Alinks&include_entities=true"
As your need is to filter your tweets based on images/photos, I think you should use filter:images.
An example of your case would look like:
https://search.twitter.com/search.json?q=from%3Agoogle%20since%3A2012-01-31%20filter%3Aimages&include_entities=true"
Hope this helps.
With Latest twitter API I couldn't get filters work and I couldn't find either any explanation in their docs. Althought you can get all the tweets and then parse only the media ones. You can fire this if inside your parsing script:
if(this.entities.media != null){
//Parse the tweet
}
This is not the best solution but the worst part comes to twitter who's giving you more information and using more of its own resources.
In the lastest twitter API you can do it in the ConfigurationBuilder instance, before creating the Twitter instance:
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setDebugEnabled(false);
cb.setOAuthConsumerKey(API_KEY);
cb.setOAuthConsumerSecret(API_SECRET);
cb.setOAuthAccessToken(ACCESS_TOKEN);
cb.setOAuthAccessTokenSecret(SECRET_KEY);
// enabling include_entities parameters
cb.setIncludeEntitiesEnabled(true);
Twitter twitterInstance = new TwitterFactory(cb.build()).getInstance();
Also, after enabling the entities, in the search string you have to had the condition "filter:images".
List<String> keywords = new ArrayList<String>();
keywords.add("#pet");
keywords.add("cat");
// String.join for Java 8
String twitterSearchString = "((" + String.join(" OR ", keywords) + ")";
// adding the filter condition
twitterSearchString += " AND filter:images)";
Query q = new Query(twitterSearchString);
And you will get just results with images (tested with twitter4j-core 4.0.4).
For filter in twitter API you can check official document for latest version as on date Apr 02 2019
https://developer.twitter.com/en/docs/tweets/search/guides/standard-operators.html