I'm trying to create images streams from social media hashtag.
When i try to using Twitter, there's a problem when getting tweets images. I already succeed displaying the images but there's a lot of duplicate tweets! The reason is there's a few users who tweet same tweet, and i think they all are bot because when i stalk their profile they all got same tweets.
Here is the example :
http://t.co/NMX3B68eBT http://t.co/xDeBSS7GRr http://t.co/VmmNezY8Qg
Here is the query :
$response = $twitter->get('search/tweets.json?q=%23holifestival -filter:retweets filter:media&count=20');
How do i fix the duplicate tweets like this? is it possible?
if it's isn't, is there any workaround? please help me.
thanks
You really only have two options.
Compare the text (ignore the URL) and see if it matches a previous tweet. If it does; discard it.
Compare the images. After downloading the image, compare it to other images you've downloaded (either filesize or MD5 hash). If it matches, discard it.
One of Twitter's problems is that multiple spam accounts can be used to repeatedly broadcast identical content. Other than examining the content, there's no way in the API to filter it.
Related
I've got a bunch of free online HTML, CSS, and JS tutorials under my belt and I want to try using them to make a browser extension. But I want to make sure that the data I want to use is actually accessible before getting started.
My goal is to make a browser extension for twitter.com that shows the number of impressions of any tweet next to the likes, retweets, and replies. My basic idea is to get the status URL of any given tweet, poll the Twitter API for the number of impressions of that tweet, store that in a variable, and then use CSS to display a little eye icon and the number stored in the impressions variable.
I know that I can find the number of impressions of all of my tweets, both through Twitter Analytics, and also just going to my profile page and clicking the little bar chart icon next to views, retweets, etc. But I'm not clear on whether I can do that for other people's tweets via Twitter's API or anything else. Can you?
For the record, I'm not too concerned about the varying definition of "impression," since it will be consistently applied across all tweets and I'm mostly interested in giving users a comparison between tweets. This is part of a research project to see how this might change how people engage with social media if they know how many views a given post has. If there's a simpler way to go about that using existing platforms, I'm open to suggestions.
Thanks for the advice!
No, impressions data is private. If you are authenticated to the Twitter API then you can use the new Twitter Developer Labs Tweets API to get private metrics like impressions, but you cannot get that for other people's Tweets. Also, the Twitter API does not support CORS, so I don't think you'll be successful trying to use it from a browser extension.
I'm looking to get back photos from a specific album on Facebook that contain a given string in their title. Can this be done? The only alternative I know of would be to download ALL photos from the album (at the moment there's about 500 and this number will grow over time), parse them all and then eventually filter. That could become (and already kind of is) an extremely costly operation I'm looking to avoid.
I have looked all over and have yet to find anything somewhat related, so if there's an answer to this question already please link me to it, thanks!
You do have to go through the album, there is no Search API for that.
/album-id/photos
check for the string in the title
use paging to get the next batch of photos
repeat from number 2 until there is no next batch
You can also cache the ids/titles for later, so you don´t have to hit the API for a all the photos in a new search, but that´s completely up to you. There is no other way.
I'm trying to develop some code in order to get all the tweets that were generated with certain hashtags, then parse them and finally analyse them. I believe I've already thought and solve the last two parts of this but I'm having some trouble with the first one. I've already read the Twitter Search API documentation but I haven't realised yet how to do this. Can anyone help me?
If you want to retrieve the tweets sent recently, you should use the search/tweets endpoint of twitter' REST API, and mention the hashtag inside q parameter
In case you want to listen to tweets containing the hashtag and receive them in real time, then twitter's streaming API is what you should use (statuses/filter endPoint).
Have a look at the documentation on twitter's website, there's also plenty of information on how to do this all around the web.
I've explored most of this topic using the previous question How do I get a list of uploaded videos for a certain channel with the new YouTube Data API (V3)?, but I wanted some specifics that build upon the answer given.
Is it possible to use the channel's username without calling a search query to get the channel ID? I have an https request that can get a search list shown below, but I'd rather not waste API quotas when I know the exact channel username.
https://www.googleapis.com/youtube/v3/search?q={channel_username}&key={my_api_key}&part=snippet
My hack for this is just take a videoid from the channel and extract a uploader ID from there, but its not elegant and won't work if the upload is removed.
Also, once I've worked my way down to getting the uploads key, how can I get a json that shows all of the uploads and not just a list of 25? If I do have to recursively call pages of uploads, how can I do that?
Thanks.
I think that to get a channel, using the channel's username, the only option is to use the search, and the best strategy to minimize the use of quota is to use a cache, or save the channel ID.
For a full list of videos uploaded, as JSON, you can use contentDetails.relatedPlaylists.uploads (the ID of the playlist that contains the channel's uploaded videos) and retrieve the PlaylistItems, using pages with 50 items per page, and using the nextPageToken value to retrieve each page in the result set.
I've found the answer. The correct way to go about it is to query the site using the search, but directly state the type of list to respond with, in effect searching only channels. This is done by
https://www.googleapis.com/youtube/v3/search?q={search_term}&key={API_key}&type=channel&part=snippet
As you can see the type is channel. By searching with a search term of the exact channel username, you'll get a return list with the first item object as the channel, and you can store the channel ID and upload ID from there.
To update this answer, you need to recognise the "quota limits".
As such, to use the "search?q=[search]" will incur a 100 quota usage.
Better to use "channels?part=id&forUsername={username]" as is only a quota of 1.
Then you can use "channels?part=id,snippet ....... &id={channel_ID]" to grab all your data
There have been quite a few number of start-up pertaining to analyzing Twitter data. There is CrowdBooster, then there is Klout, which use Twitter data to tell the user their True reach.
I have got the following two questions:
1) Is there a way to find out who has viewed one's tweet, or the number of people that have viewed a tweet. Crowdbooster claims to tell you how many impression one received per tweet. How do they do it?
2) Thousands and thousands of links are shared each day on Twitter. Can we find out which user has clicked the link in a tweet?
I have looked through Twitter API and some of the companies that have licensed Twitter's Firehose, but have not found anything that meet my needs.
Also, to give you a short answer to your 2nd question. Now that we've established that view analysis is impossible. Can you find out which user has clicked on that link, absolutely. And depending on what your talking about, user as far as the user who has clicked on the link or the user that has the link on their Twitter stream. Both are possible,
in the case of A, you would get the referring users IP address. Methods vary depending on language.
But what I think your asking for is scenario B, finding out which user has the link in their Twitter stream. This can be done by querying the link, the API response you will get can include tweet entities which will list all this information out for you and more. Open up a firehose with your link and watch what comes in.
https://dev.twitter.com/docs/streaming-api/methods
1) Is there a way to find out who has viewed one's tweet, or the
number of people that have viewed a tweet. Crowdbooster claims to tell
you how many impression one received per tweet. How do they do it?
No, in the case of a view - this would be impossible. The tweet impression can happen in multiple silos. On the website, in a widget, in a mobile app. You can imagine that it's simply not possible to get the impression of a tweet on a view because of this reason and because unlike a click, there is no I viewed this tweet identifier sent when a view has been enacted. I spent a great deal of time researching for a way to get the tweet impression even based on a similar clicked link and this is not even possible. (edit: it's possible see the last paragraph) This brings us to question 2.
2) Thousands and thousands of links are shared each day on Twitter.
Can we find out which user has clicked the link in a tweet?
Yes, what these websites are mainly doing is analyzing links that you process through their website. If you can have a unique hash marker on a link then analysis becomes possible. Without a unique hash marker, Twitter will re-interrupt two of the same links in a exactly the same way, even in the case that it shortens your link to it's custom t.co wrapper.
This means the only reliable way to do tweet analysis is by including a unique link marker code on your tweet and analyze the the fact that somebody that has hit your server has clicked on that link.
There is a somewhat hidden Twitter API feature that helps you understand how popular a particular link is. That being the link count API .. http://urls.api.twitter.com/1/urls/count.json?url=
Something really outside of the box you can do if your set on analyzing multiple versions of exactly the same link without using markers and if your also using the Streaming (firehose) would be to analyze the tweet views (using the link count API) on similar links that hit your server. The link that got the +1 boost in view is the one that hit your server. But that's about the extent of creative analysis you can get with your tweets and more specifically the links, as mentioned links are the only thing your really able to analyze when it comes to Twitter.
1) Is there a way to find out who has viewed one's tweet, or the number of people that have viewed a tweet. Crowdbooster claims to tell you how many impression one received per tweet. How do they do it?
Yes, sign up for Twitter Analytics https://analytics.twitter.com (free service provided by Twitter) and you can see how many people view (impressions) for each tweet and totals for specific dates or a date range.
2) Thousands and thousands of links are shared each day on Twitter. Can we find out which user has clicked the link in a tweet?
Yes, you can do this. Using a URL shortening service like Bitly.com you can track how many clicks you had from Twitter (only give out that Bitly link on Twitter to do this). But if you want more indept information you may need to create a tracking software, as I don't know of any available. To do that you would need the tracking software to track the link and find out the refer header and see if it's from Twitter (or better yet, just give out a unique URL for your tweets), then you would need to use the Twitter API to find out the handle (username) of that visitor who clicked your link. Lastly store this information in a database so you can review who clicked what link.