By following the guide Create Your Own Google Pagespeed & Mobile Usability Tracking Google Sheet in 5 Steps I managed to set up mobile pagespeed score for a list of (up to 50) URLs.
However since late 2017 or something there is real data available from the Chrome User Experience Report that displays an average load time in seconds for a page based on chrome user data.
(This data is being used for example when using Pagespeed Insights by google.)
Instead of pulling a page score I as described above I would like to pull the average load time into my google sheet.
Is it possible to adapt the script used from the article above to pull load time in seconds instead of pagescore? Or is there any other way to do this?
Thanks in advance your help is much appreciated.
This is the script I run in script editor to get pagescore into google sheet according to the linked article with function =checkAll(C3):
/**
* Returns Mobile Pagespeed, Mobile Usability, and Desktop Pagespeed values in three adjacent columns
* by Cagri Sarigoz
*/
function checkAll(Url) {
//CHANGE YOUR API KEY WITH YOUR_API_KEY BELOW
var key = "AIzaSyB2SeOumbCd6YNfFWRg5Jo_WpISZi4gCFs";
var serviceUrlMobile = "https://www.googleapis.com/pagespeedonline/v2/runPagespeed?url="+Url+"&strategy=mobile&key="+key;
var serviceUrlDesktop = "https://www.googleapis.com/pagespeedonline/v2/runPagespeed?url="+Url+"&strategy=desktop&key="+key;
var array = [];
if (key == "YOUR_API_KEY")
return "Please enter your API key to the script";
var responseMobile = UrlFetchApp.fetch(serviceUrlMobile);
if(responseMobile.getResponseCode() == 200) {
var contentMobile = JSON.parse(responseMobile.getContentText());
if ( (contentMobile != null) && (contentMobile["ruleGroups"] != null) )
{
if (contentMobile["responseCode"] == 200)
{
var speedScoreMobile = contentMobile["ruleGroups"]["SPEED"]["score"];
var usabilityScoreMobile = contentMobile["ruleGroups"]["USABILITY"]["score"];
}
else
{
array.push(["Not Found!", "Not Found!", "Not Found!"]);
return array;
}
}
}
var responseDesktop = UrlFetchApp.fetch(serviceUrlDesktop);
if(responseDesktop.getResponseCode() == 200) {
var contentDesktop = JSON.parse(responseDesktop.getContentText());
if ( (contentDesktop != null) && (contentDesktop["ruleGroups"] != null) )
var speedScoreDesktop = contentDesktop["ruleGroups"]["SPEED"]["score"];
}
array.push([speedScoreMobile, usabilityScoreMobile, speedScoreDesktop]);
return array;
}
I am the writer of the blog post that you shared. As you said, the Google Apps Script there was using Google Pagespeed API v2. The current API version is v4, and v2 will be depreciated on June 30th.
So I updated the code with v4 on my own copy of the spreadsheet. You can make your own copy from here.
I also wanted to add the mobile-friendly test results but it turned out that Google Search Console's API quota restrictions were too tight, returning error almost all the time. So I commented out that part of the code for the time being.
I didn't have the time to update my blog post yet. You can see the new version of the script here.
How can I get UserId and ScreenName of the users that a given user is FOLLOWING (not followers) in LinqToTwitter?
??
The Twitter API uses the terms follower to mean people who follow a user and friends to mean people that a user follows and LINQ to Twitter continues that approach. So, you would use a Friendship/FriendshipType.FriendsList query, like this:
static async Task FriendsListAsync(TwitterContext twitterCtx)
{
Friendship friendship;
long cursor = -1;
do
{
friendship =
await
(from friend in twitterCtx.Friendship
where friend.Type == FriendshipType.FriendsList &&
friend.ScreenName == "JoeMayo" &&
friend.Cursor == cursor &&
friend.Count == 200
select friend)
.SingleOrDefaultAsync();
if (friendship != null &&
friendship.Users != null &&
friendship.CursorMovement != null)
{
cursor = friendship.CursorMovement.Next;
friendship.Users.ForEach(friend =>
Console.WriteLine(
"ID: {0} Name: {1}",
friend.UserIDResponse, friend.ScreenNameResponse));
}
} while (cursor != 0);
}
This example pages through the results in a do/while loop. Notice that the cursor is set to -1, which starts off the query without a Twitter API cursor. Each query assigns the cursor, which gets the next page of users. In the if block, the first statement reads the friendship.CursorMovement.Next to the get cursor for the next page of users. When the next cursor is 0, you've read all of the followers.
After the query executes, the Users property has a List<User> where you can get user information. This demo prints each member of the list.
One of the things you might run into with large friend lists is that Twitter will return an error for exceeding the rate limit. You'll be able to catch this, in a try/catch block, by catching TwitterQueryException and examining properties for the Rate Limit Exceeded. To minimize rate limit propblems, set count to 200, the max. Otherwise count defaults to 20.
You can download samples and view documentation for this on the LINQ to Twitter Web site.
I am using Linq2Twitter in my ASP.net Web Forms application to return recent user tweets
var tweets = await
(from tweet in ctx.Status
where (
(tweet.Type == StatusType.User)
&& (tweet.ScreenName == screenName)
&& (tweet.ExcludeReplies == true)
&& (tweet.IncludeMyRetweet == false)
&& (tweet.Count == 10)
&& (tweet.RetweetCount < 1)
)
select tweet)
.Take(count)
.ToListAsync();
This seems to work well and I get the expected Json return, but...
When I try and construct a link to the original tweet...
“https://twitter.com/” + ScreenName + “/status/” + data.StatusId
I get a "Sorry, page does not exist error".
Upon investigation it appears that the returned StatusId is incorrect. For example, the returned StatusId is:
500244784682774500
When the actual tweet refers to:
500244784682774528
In other words, in this case, the StatusId seem to be 28 adrift.
Can anyone throw any light on what is happening/what I am doing wrong?
IThanks.
After some debugging I found that the ID returned to the LinqtoTwitter application was correct, the problem occurred either in the JSON converter or in JavaScript itself being unable to handle the unsigned-integer id value.
The solution was to create a simple view model from the returned results (using an extension method against the LinqToTwitter.Status object) and passing that to the client instead of the whole data graph.
On a MVC 5 web site I would like visitors to be able to read the full version of a post only after they shared it on Facebook or Twitter.
I have seen this example in a few web sites ... What would be the best way to do this?
There is no real security issues here ... It is just a way to spread the word ...
My first idea would be to save a cookie with a post KEY (Guid) ... This key is not visible to the user so he will not know the value.
The problem is how do I know that he shared the url ... How do I get the confirmation?
Thank You,
Miguel
You get confirmation as follows, per the Facebook Developers Docs:
FB.ui(
{
method: 'feed',
name: 'Facebook Dialogs',
link: 'https://developers.facebook.com/docs/dialogs/',
picture: 'http://fbrell.com/f8.jpg',
caption: 'Reference Documentation',
description: 'Dialogs provide a simple, consistent interface for applications to interface with users.'
},
function(response) {
if (response && response.post_id) {
alert('Post was shared.'); //give access to article
} else {
alert('Post was not shared.'); //they chose not share... don't give access
}
});
I implemented this code almost verbatim in a .NET project (just replaced the alerts with my own functionality) where I gave users two entries into a contest if they shared the contest page (instead of one entry if they didn't share or did nothing).
As for Twitter, I've personally not implemented something similar, but your best bet is probably JavaScript Interfaces for Twitter for Websites.
I don't know about Facebook, but with Twitter a retweet is the same as a share. The statuses/retweeters/ids should work. If you have the id of the tweet, then you can hold a list of who retweeted it, updating as needed to get new ids.
https://dev.twitter.com/docs/api/1.1/get/statuses/retweeters/ids
If you don't want to write all of the code to authenticate and configure the endpoint, you could use a 3rd party library. Here's an example from my library, LINQ to Twitter v3.0 Beta:
ulong tweetID = 210591841312190464;
var status =
await
(from tweet in twitterCtx.Status
where tweet.Type == StatusType.Retweeters &&
tweet.ID == tweetID
select tweet)
.SingleOrDefaultAsync();
if (status != null && status.User != null)
status.Users.ForEach(
userID => Console.WriteLine("User ID: " + userID));
BTW, there's also a Facebook SDK for .NET.
Is there a way in the Twitter API to get the replies to a particular tweet? Thanks
Here is the procedure to get the replies for a tweets
when you fetch the tweet store the tweetId ie., id_str
using twitter search api do the following query
[q="to:$tweeterusername", sinceId = $tweetId]
Loop all the results , the results matching the in_reply_to_status_id_str to $tweetid is the replies for the post.
From what I understand, there's not a way to do that directly (at least not now). Seems like something that should be added. They recently added some 'retweet' capabilities, seem logical to add this as well.
Here's one possible way to do this, first sample tweet data (from status/show):
<status>
<created_at>Tue Apr 07 22:52:51 +0000 2009</created_at>
<id>1472669360</id>
<text>At least I can get your humor through tweets. RT #abdur: I don't mean this in a bad way, but genetically speaking your a cul-de-sac.</text>
<source>TweetDeck</source>
<truncated>false</truncated>
<in_reply_to_status_id></in_reply_to_status_id>
<in_reply_to_user_id></in_reply_to_user_id>
<favorited>false</favorited>
<in_reply_to_screen_name></in_reply_to_screen_name>
<user>
<id>1401881</id>
...
From status/show you can find the user's id. Then statuses/mentions_timeline will return a list of status for a user. Just parse that return looking for a in_reply_to_status_id matching the original tweet's id.
The Twitter API v2 supports this now using a conversation_id field. You can read more in the docs.
First, request the conversation_id field of the tweet.
https://api.twitter.com/2/tweets?ids=1225917697675886593&tweet.fields=conversation_id
Second, then search tweets using the conversation_id as the query.
https://api.twitter.com/2/tweets/search/recent?query=conversation_id:1225912275971657728
This is a minimal example, so you should add other fields as you need to the URL.
Twitter has an undocumented api called related_results. It will give you replies for the specified tweet id. Not sure how reliable it is as its experimental, however this is the same api call that is called on twitter web.
Use at your own risk. :)
https://api.twitter.com/1/related_results/show/172019363942117377.json?include_entities=1
For more info, check out this discussion on dev.twitter:
https://dev.twitter.com/discussions/293
Here is my solution. It utilizes Abraham's Twitter Oauth PHP library: https://github.com/abraham/twitteroauth
It requires you to know the Twitter user's screen_name attribute as well as the id_str attribute of the tweet in question. This way, you can get an arbitrary conversation feed from any arbitrary user's tweet:
*UPDATE: Refreshed code to reflect object access vs array access:
function get_conversation($id_str, $screen_name, $return_type = 'json', $count = 100, $result_type = 'mixed', $include_entities = true) {
$params = array(
'q' => 'to:' . $screen_name, // no need to urlencode this!
'count' => $count,
'result_type' => $result_type,
'include_entities' => $include_entities,
'since_id' => $id_str
);
$feed = $connection->get('search/tweets', $params);
$comments = array();
for ($index = 0; $index < count($feed->statuses); $index++) {
if ($feed->statuses[$index]->in_reply_to_status_id_str == $id_str) {
array_push($comments, $feed->statuses[$index]);
}
}
switch ($return_type) {
case 'array':
return $comments;
break;
case 'json':
default:
return json_encode($comments);
break;
}
}
Here I am sharing simple R code to fetch reply of specific tweet
userName = "SrBachchan"
##fetch tweets from #userName timeline
tweets = userTimeline(userName,n = 1)
## converting tweets list to DataFrame
tweets <- twListToDF(tweets)
## building queryString to fetch retweets
queryString = paste0("to:",userName)
## retrieving tweet ID for which reply is to be fetched
Id = tweets[1,"id"]
## fetching all the reply to userName
rply = searchTwitter(queryString, sinceID = Id)
rply = twListToDF(rply)
## eliminate all the reply other then reply to required tweet Id
rply = rply[!rply$replyToSID > Id,]
rply = rply[!rply$replyToSID < Id,]
rply = rply[complete.cases(rply[,"replyToSID"]),]
## now rply DataFrame contains all the required replies.
You can use twarc package in python to collect all the replies to a tweet.
twarc replies 824077910927691778 > replies.jsonl
Also, it is possible to collect all the reply chains (replies to the replies) to a tweet using command below:
twarc replies 824077910927691778 --recursive
Not in an easy pragmatic way. There is an feature request in for it:
http://code.google.com/p/twitter-api/issues/detail?id=142
There are a couple of third-party websites that provide APIs but they often miss statuses.
I've implemented this in the following way:
1) statuses/update returns id of the last status (if include_entities is true)
2) Then you can request statuses/mentions and filter the result by in_reply_to_status_id. The latter should be equal to the particular id from step 1
As states satheesh it works great. Here is REST API code what I used
ini_set('display_errors', 1);
require_once('TwitterAPIExchange.php');
/** Set access tokens here - see: https://dev.twitter.com/apps/ **/
$settings = array(
'oauth_access_token' => "xxxx",
'oauth_access_token_secret' => "xxxx",
'consumer_key' => "xxxx",
'consumer_secret' => "xxxx"
);
// Your specific requirements
$url = 'https://api.twitter.com/1.1/search/tweets.json';
$requestMethod = 'GET';
$getfield = '?q=to:screen_name&sinceId=twitter_id';
// Perform the request
$twitter = new TwitterAPIExchange($settings);
$b = $twitter->setGetfield($getfield)
->buildOauth($url, $requestMethod)
->performRequest();
$arr = json_decode($b,TRUE);
echo "Replies <pre>";
print_r($arr);
die;
I came across the same issue a few months ago at work, as I was previously using their related_tweets endpoint in REST V1.
So I had to create a workaround, which I have documented here:
http://adriancrepaz.com/twitter_conversations_api Mirror - Github fork
This class should do exactly what you want.
It scrapes the HTML of the mobile site, and parses a conversation. I've used it for a while and it seems very reliable.
To fetch a conversation...
Request
<?php
require_once 'acTwitterConversation.php';
$twitter = new acTwitterConversation;
$conversation = $twitter->fetchConversion(324215761998594048);
print_r($conversation);
?>
Response
Array
(
[error] => false
[tweets] => Array
(
[0] => Array
(
[id] => 324214451756728320
[state] => before
[username] => facebook
[name] => Facebook
[content] => Facebook for iOS v6.0 ? Now with chat heads and stickers in private messages, and a more beautiful News Feed on iPad itunes.apple.com/us/app/faceboo?
[date] => 16 Apr
[images] => Array
(
[thumbnail] => https://pbs.twimg.com/profile_images/3513354941/24aaffa670e634a7da9a087bfa83abe6_normal.png
[large] => https://pbs.twimg.com/profile_images/3513354941/24aaffa670e634a7da9a087bfa83abe6.png
)
)
[1] => Array
(
[id] => 324214861728989184
[state] => before
[username] => michaelschultz
[name] => Michael Schultz
[content] => #facebook good April Fools joke Facebook?.chat hasn?t changed. No new features.
[date] => 16 Apr
[images] => Array
(
[thumbnail] => https://pbs.twimg.com/profile_images/414193649073668096/dbIUerA8_normal.jpeg
[large] => https://pbs.twimg.com/profile_images/414193649073668096/dbIUerA8.jpeg
)
)
....
)
)
since statuses/mentions_timeline will return the 20 most recent mention this won't be that efficient to call, and it has limitations like 75 requests per window (15min) , insted of this we can use user_timeline
the best way: 1. get the screen_name or user_id parameters From status/show.
2. now use user_timeline
GET https://api.twitter.com/1.1/statuses/user_timeline.json?screen_name=screen_name&count=count
(screen_name== name which we got From status/show)
(count== 1 to max 200)
count: Specifies the number of Tweets to try and retrieve, up to a maximum of 200 per distinct request.
from the result Just parse that return looking for an in_reply_to_status_id matching the original tweet's id.
Obviously, it's not ideal, but it will work.
If you need all replies related to one user for ANY DATE RANGE, and you only need to do it once (like for downloading your stuff), it is doable.
Make a Twitter development application
Apply for elevated credentials. You will instantly get them after filling out the forms. At least I did on two separate accounts today.
Your development account now has access to the v1.1 API search in the "Sandbox" tier. You get 50 requests against the tweets/search/fullarchive endpoint maxing out at 5000 returned tweets.
Make an environment for your development application.
Make a script to query https://api.twitter.com/1.1/tweets/search/fullarchive/<env name>.json where <env name> is the name of your environment. Make your query to:your_twitter_username and fromDate when you created your account, toDate today.
Iterate over the results
This will not get your replies recursively