I'm using YQL to retrieve site specific results (I don't need Google results for just my own site but for a variety of sites.
However, I'm getting inconsistent results. Using the YQL console for my query:
Executing a Test for the same query multiple times, sometimes returns result but quite often gives no result without producing an error.
When there are results, the number of results is often significantly less from manually executing a Google query on google.com
An example query that shows this behaviour is
select * from google.search where q = 'site:www.freelas.net'
Any suggestions on this?
Kind regards,
Marc
The older Google Search API that the google.search YQL table uses has been deprecated by Google. It's still running, but I think there is a 100 query/day limit. When you see no results in YQL console, it likely means that it's being rate-limited.
As an alternative you might try using the newer Google Custom Search API. Note that it's only free up to 100 queries/day, but they offer a paid plan above that.
Related
I hope that you guys are fine. I want to build a simple spreadsheet and I thought I could be able to make one but blank sheet looks horrible to me. I am sure that you guys are kind enough to help me out.
I want to perform multiple Google search queries in Google spreadsheet and want to parse results of each search (top 10 results of each search)
Something like this: https://www.youtube.com/watch?v=tBwEbuMRFlI
But when I tried his given formula in description to play test, Google returned #Error to me, I don't know why.
Can you guys please help me out in making a simple spreadsheet compatible for multiple queries at once? Like one column for keywords (where I could paste my list of keywords) and then 10 columns of search results. All results for one keyword should come in one row
Something like this:
My 1st Example Query = 1st search result, 2nd search result, 3rd result and so on.
My 2nd Example Query = 1st search result, 2nd search result, 3rd result and so on.
It must be easy to code but yeah, it might be time-consuming and I would be very grateful if anyone of you could help me about it.
Looking forward to your help guys.
The problem is that you want to scrape out of Spreadsheets, that's a bad approach and is almost certainly not going to work. Even if you manage to write a scraper inside that limited environment it will easily be spotted by Google.
As you said time is not a problem, I would suggest another route.
Use a backend tool/script that scrapes the data
Use a backend tool/script that creates/modifies the Google spreadsheet
You can run such a script(s) manually on your PC or from a server full automated using a scheduler/cron job.
To create/modify spreadsheets look here: How do I access the Google Spreadsheets API in PHP?
To scrape Google look here: Is it ok to scrape data from Google results?
So this is PHP as language of choice but you can do the exactly same in Java or Python or C#
There is a third party solution like SerpApi you could use for this. It's a paid API with a free trial.
Google Sheets Add-on: SerpApi - Search Engine Results and Ranks
Example code to extract title from the first result:
=SERPAPI_RESULT("engine=google&q=coffee&location=Austin, Texas, United States&google_domain=google.com&gl=us&hl=en", "organic_results.0.title")
I'm trying to do large amount of search queries automatically, and YQL has very attractive limitation settings. However, when I was following this post, hoping I could use YQL Bing Search or BOSS v2, through both YQL Console and url, both told me no defination of the tables...
But I found yql-tables here, which includes Bing search and BOSS search and many other tables that cannot be found through YQl Console Show Tables
The tutorials about YQL I have found are all majorly happened in 2011, that's many years ago...
So, I'm wondering, whether tables here are still valid? If so, do you know how can I use microsoft.bing.web or boss.search to get the query results?
... that cannot be found through YQl Console Show Tables
Be sure to enable "Show Community Tables" (on the left of the page, above the tables list) as shown below.
I'm implementing a sorting algorithm and I want to query the Insights for Twitter API in order to find if a topic is trendy in comparison to another topic.
As far as I've investigated, there is a Count API Operation which will return the number of Tweets based on a given query. Therefore I could query how many Tweets have the particular "keyword" in their body and then compare it to other different keywords to establish an order.
Is there is a more direct query to know if a keyword is trendy?
Your approach using the /api/v1/messages/count endpoint looks like the right one when working with the Insights for Twitter API.
I guess you could also use the posted query parameter to build trending charts over time for your different keywords.
I am trying to fetch the result of Search engine optimization queries in Google Analytics Query Explorer 2.
But I am not able to get the desired results. I have tried the below query
Any suggestions how to achieve this? or Is there a way I can query it using google analytics api.
Kindly Checkout Internal Link Checker of Elite Site Optimizer helps in Identifying and fixing the Link Errors
I am developing an application to guess locations of tornadoes by analyzing twitter data. For this, I would first need to train a neural network on some manually annotated tweets. I am trying to get tweets from last year which have the word 'tornado' in them. This is my code below :-
Query query = new Query("tornado");
query.setRpp(100);
query.setSince("2010-11-01");
query.setUntil("2011-01-13");
QueryResult queryResult = instance.search(query);
tweetList = queryResult.getTweets();
I am able to retrieve tweets from periods closer to now such as last week and such, but am unable to get any results for periods such as the one listed above. Any clues, suggestions would be help. Thanks in advance.
I just found out the reason through a different medium, thought i'd share the answer in case there are other people with the same issue.
It turns out that the twitter search api does not return tweets older than around a week and also, depending on the server load, at times this could be as low as 24 hours ! Hence, any 3rd party libraries (such as twitter4j) which have a wrapper for the twitter search api will behave similarly.
The best way to go about this would be to use third party search and indexing sites such as snapbird, topsy, etc..