When someone tweets sometime to a page, and twitter crawls the page for data such as checking Twitter Cards, does anyone know what User Agent Twitter uses?
Twitter uses the User-Agent of Twitterbot (with version, such as Twitterbot/1.0) when crawling pages to find Twitter Cards markup. Twitter's crawler respects robots.txt when scanning URLs if you ever need to create exceptions.
Read more on Twitter Cards URL Crawling & Caching.
Related
If someone on my website sent a POST request with some data, how would I redirect that to another API in Ruby on Rails? I am trying to make a rails website where a user on my website can input some data like a title, body text, some media and send that to the APIs of social media platforms like Facebook, Instagram, Pinterest, YouTube, TikTok, LinkedIn, and Twitter. So the user can connect their accounts and create a post or a video on that social media platform from my website. Is that possible, and if so, can someone help me with how I do that entire thing from my website? Thanks.
I didn't try a lot of things since I don't exactly know what to do.
I want to crawl the tweets from twitter for sentimental analysis and want to know if by using my own crawler and without OathAuthentification i.e by registering my app on twitter can I crawl twitter
Background:
I'm using the react-boilerplate to build a personal website, and I would like to display my latest tweet in the footer. I do not want the embedded card, only the tweet itself (similar to the Momentum extension).
This appears to be what I am looking for:
https://dev.twitter.com/rest/reference/get/statuses/user_timeline
Question:
How is my website supposed to authenticate with the Twitter API, without including the authentication data in the website's codebase?
Should I have an app in the middle that processes the requests, or is it possible for my website to directly retrieve my latest tweet each time a user accesses my page?
Thanks
In Google Analytics' Pages report, I'm looking at a particular page on my site, then, under secondary dimensions, pulling up "Source/Medium." I'm seeing rows for both of these:
t.co / referral
twitter.com / social
What's the difference between these two? I understand that links shared on Twitter get wrapped in t.co URLs, but then what are the visits coming from twitter.com?
t.co is a link shortner similar to bitly, but with some assurance from twitter that the link isn't harmful (they say they scan the page before giving out the shortened URL). People refered to your sites are getting there from a page that shared on twitter, and the people coming from twitter.com came from a twitter post.
So, basically:
t.co/referral implies Twitter -> other page -> your site
twitter/social implies Twitter -> your site
I'm not sure that #j-a-streich is right about that. From what I've seen, dealing with google analytics, you can't 100% predict why google ends up with the info they get - there are a lot of edge cases and gotchas. Here is what I have uncovered:
Twitter's official stance on how they handle referrer information: https://dev.twitter.com/docs/tco-redirection-behavior
t.co currently handles redirections by context and known user agents. We've taken care to preserve original referrers in all contexts where they are reliably provided.
But that doesn't actually help much, because they don't differentiate in that doc between t.co and twitter.com.
I did some testing, and it appears that whenever someone comes to your site direct from clicking a t.co wrapped link on twitter.com the referrer is the t.co link. That includes tweets and profile bios. That is also true for urls that were directly pasted into twitter and those that were pre-shortened by another shortener.
I also checked links from a client (old tweetdeck) and that showed up as t.co.
I wasn't actually able to find a valid source of twitter.com referrers.
My main problem with what #j-a-streich says is that he is appearing to say that the referrer will be maintained if someone goes from twitter to another site, then to you - and that won't happen. The referrer will be the other site.
I deal with this issue in my analytics as well, but I've just come to accept that referral data is not perfect and I treat it as so. I currently see direct, t.Co and twitter.Com referrals in GA and generally associate them all with Twitter. It's obviously hard to make a distinction with the direct visits, but there isn't much to do about that.
I think mobile device referrals play a part in it too because of the applications browser. I've noticed clicks in the android app show direct visits instead of t.co or twitter.
I'm currently using Twitter's RSS API (example) to retrieve people's tweets for an app. However, once version 1.1 of their API comes out, I won't be able to do this anymore. What's a simple replacement I can use for this?
Unfortunately the answer is: "There is none."
Twitter continues to lock down their platform, as they have since the switch to OAuth. If you want to add content to Twitter, you can do that easily. If you want to get content out of Twitter, well, you'll have to conform to Twitter's requirements.
RSS was an easy way of getting content out of Twitter, and so they killed it. They'll tell you that it wasn't really used, but that's a load of bull and they know it, but it sounds good and most people will simply swallow it.
There is a workaround that you can use until March 2013:
https://api.twitter.com/1/statuses/user_timeline/UserName.rss
Replace 'UserName' with the actual twitter username you want.
But after sometime in March/April 2013, that will cease to work and you'll have to use OAuth and json to parse Twitter feeds.
Use Twitter RSS - Google Apps Script from github to convert Twitter API 1.1 to RSS. Using this option you can get search results, users' timelines, users’ favorites or even Twitter Lists.
I've put together two mit licensed php scripts and instructions so you can fetch and cache twitter user timelines as RSS..
The most important is to cache the tweets since you are only allowed to call the API 15 times per 15 minutes..
Have a look at twitterrss.net
I've been looking for an answer to this for a while, basically as I wanted to provide a replacement for the Twitter triggers in IFTTT. When IFTTT removed them we could use Twitter's RSS feeds. However now, with v1 of their API retired, you have to authenticate any requests and they no longer off RSS feeds. It's a shame because it's closing the door for a lot of newbie developers out there.
I'd definitely recommend checking out twitter-rss.com in the answer above, but you might want to check out a solution I've come up with. It is called Twools and it's almost ready.
Twools started off as a way to get back RSS feeds from Twitter. Twitter
retired version 1 of their API in June 2013 as well as retiring RSS feeds. As
well as not having access to RSS feeds, you need to be authenticated to use
the API which
raises the bar for Twitter users and developers. It has been my intention to
make this as easy as possible for you install, however you do need to install
this on your own website (PHP is required). It is just a case of editing a few
lines in a file (with your Twitter app details) and uploading to your website.
Twools offers a number of RSS feeds from the Twitter API:
Your home timeline
User timeline (yours or another user's)
Your mentions
Your favorites
List timeline (from a list that you own)
Direct messages received (disabled by default)
Direct messages Sent (disabled by default)
Twitter Search
New followers
New friends
You can filter the results from the Twitter API by only outputting tweets
containing certain keywords, hashtags or screen names or even by the app used
to send the Tweet. As well as that you can filter based on the user (currently
their location and language). Finally, you can modify the data outputted in
the RSS feed by removing URLs, screen names, hashtags etc or share your recipe
with other folders. As well as an RSS feed generator, Twools has a basic
direct message spam filter (experimental- use at your own risk!) and a simple
social media management tool (basically an HTML version of the RSS filters).
You can also keep an eye on your API usage on the Status page.
With Twools and your app you could
Your tweets when you mention a specific hashtag
Your Twitter Faves:
Get new followers from the a certain country or location
Get tweets when someone mentions Starbucks negatively in a Tweet in the UK
Get tweets when someone tweets to an article on your website
For more information, see http://twools.it/