Can somebody please help me in finding official limits of Google Spreadsheet api "RESPONSE".
Is it size based or num of cell based. I have to pull Grid data for number of columns from a quite big google sheet(30*57000).
But it fails to respond if I pull range of more then 18 columns in single request.
The error I receives
com.google.api.client.googleapis.json.GoogleJsonResponseException: 502 Bad Gateway
<!DOCTYPE html>
<html lang=en>
<meta charset=utf-8>
<meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
<title>Error 502 (Server Error)!!1</title>
<style>
*{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}#media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}#media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}#media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
</style>
<a href=//www.google.com/><span id=logo aria-label=Google></span></a>
<p><b>502.</b> <ins>That’s an error.</ins>
<p>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds. <ins>That’s all we know.</ins>`
Get request looks like:
{fields=sheets.data.rowData.values.effectiveFormat, ranges=[Sheet1!A2:A, Sheet1!M2:M, Sheet1!N2:N, Sheet1!O2:O, Sheet1!R2:R, Sheet1!S2:S, Sheet1!T2:T, Sheet1!U2:U, Sheet1!V2:V, Sheet1!X2:X, Sheet1!Y2:Y, Sheet1!Z2:Z, Sheet1!AC2:AC, Sheet1!AD2:AD, Sheet1!AE2:AE, Sheet1!AF2:AF, Sheet1!AG2:AG, Sheet1!AH2:AH, Sheet1!AI2:AI, Sheet1!AJ2:AJ], spreadsheetId=1_qVz-XXXXXXXXXXXXXXXXrXXXXXXXXXXXXXXXX}
Based on your example I was able to replicate the behaviour. That 502 appears when getting huge ranges while including the grid data in the response. In order to prevent this issue the docs lists «For large spreadsheets, it is recommended to retrieve only the specific fields of the spreadsheet that you want». Based on that approach, I strongly suggest you to get only the relevant ranges for the operation at hand. If you really need to get 18 columns (each with 57,457 rows), then you should spread your requests in several batches. Then you could join the responses so you could have a big array to interact with. Please ask me to clarify any point if needed.
Related
I am using the Microsoft Graph API to programmatically add content to a OneNote Page on my Office365 OneNote Notebook. For an example page like this:
OneNote snip consisting of Title & 2 separate divs/blocks, this is the generated HTML that I get by making a GET to the pages/{myPageID}/content?includeids=true endpoint
<html lang="en-US">
<head>
<title>Thursday, December 6, 2018</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="created" content="2018-12-06T19:55:00.0000000" />
</head>
<body data-absolute-enabled="true" style="font-family:Calibri;font-size:11pt">
<div id="div:{95790d05-168a-4f61-81a7-5c13e3124069}{4}" style="position:absolute;left:48px;top:115px;width:624px">
<p id="p:{95790d05-168a-4f61-81a7-5c13e3124069}{10}" style="margin-top:0pt;margin-bottom:0pt">It is a jolly good day</p>
</div>
<div id="div:{3a6b78e9-2af8-4a5b-abf8-09871fb6eec5}{20}" style="position:absolute;left:44px;top:180px;width:624px">
<p id="p:{3a6b78e9-2af8-4a5b-abf8-09871fb6eec5}{24}" style="margin-top:0pt;margin-bottom:0pt">Lets do this shall we</p>
</div>
</body>
</html>
I now try adding a new div/block by making the following PATCH using the requests library in Python. The myPageID and access_token are collected previously by making GET requests to the graph API:
patchPageURL='https://graph.microsoft.com/v1.0/users/'+userID+'/onenote/pages/'+myPageID+'/content?includeIDs=true'
patchPageHeaders={
'Content-Type':'application/json',
'Authorization':'Bearer '+access_token
}
patchPageBody=[{
'target':'div:{3a6b78e9-2af8-4a5b-abf8-09871fb6eec5}{20}',
'action':'insert',
'position':'after',
'content':'<div><p> And God said let there be new blocks </p></div>'
}]
patchResp = requests.patch(patchPageURL, headers=patchPageHeaders, json=patchPageBody)
However, I end up with a 400 error as a response to my PATCH request with the following code and message:
code: 20135
message: The entity type is not supported for this operation.
If I change the opening div tag for the content to something like <div data-id="new-div">, I still receive the same 400 error, code 20135.
I am following the instructions from this documentation here. What subtlety is being missed here? There are no authentication or other similar errors. The PATCH works fine if I change the target to body and the action to append but that doesn't make a new div/block and only adds the content as a child of the first div in the page.
Is there a straightforward way to add new blocks/divs as a sibling of existing blocks or even just something as simple as adding a new block at the end of the OneNote page?
I read these 2 doc pages: summary and
summary-card-with-large-image, but I don't really see the difference.
Example:
<meta name="twitter:card" content="summary" /> <!-- or summary_large_image -->
<meta name="twitter:title" content="Small Island Developing States Photo Submission" />
<meta name="twitter:image" content="https://farm6.staticflickr.com/5510/14338202952_93595258ff_z.jpg" />
What is the actual difference betwen both at the end? The rendering looks identical:
You're quite right and I'll make a note to update these documentation pages. A summary card usually has a small image on the left of the text in the timeline - this variance in the docs may be the result of using embedded Tweets. Both examples in the documentation show what I'd expect a summary card with large image to render as.
As suggested by #AndyPiper, this is probably a documentation problem. Here is the result of a content="summary", screenshot from Twitter in Chrome browser (desktop):
The Card validator is displaying my summary card correctly, however, when I actually try to share it buy clicking the "twitter-share-button" it does not display a twitter card, it only shows a regular tweet.
This what the validator says:
Your site is whitelisted for summary_large_image card
INFO: Page fetched successfully INFO: 18 metatags were found INFO:
twitter:card = summary_large_image tag found INFO: Card loaded
successfully
These are the tags I'm using:
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:site" content="#username">
<meta name="twitter:creator" content="#username">
<meta name="twitter:title" content="Some Title">
<meta name="twitter:description" content="Some description.">
<meta name="twitter:image" content="http://example.com/images/dog.jpg">
This is the html:
<a class="twitter-share-button"
href="https://twitter.com/intent/tweet?text=Testing Testing Testing">
Tweet</a>
For anyone strugling with tyhe same issue, I'm responding to my own question.
Here's what I found out:
1. First off, keep in mind that the card only shows in the final posted Tweet - not in the Tweet composer window. However, in my case, it wasn't showing in the final posted tweet either.
2. You have to include the link to the page you want to share after the text parameter. Additionally you need to encode its URL value. So, instead of text=http://example.com use text=http%3A%2F%2Fexample.com. If you go to http://www.w3schools.com/tags/ref_urlencode.asp you can make this encoding convertion automaticaly.
3. I've been told that sometimes there is a delay as twitter's crawler caches your site, which would explain why the card doesn't show in the final posted tweet in your fist attempts to share it. In my case it took 48 hours, after which, clicking the share button will post the card immediately.
Hope this my be helpful to someone else...
This is the current twitter card data I have inside <head>
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:site" content="#tradukka">
<meta name="twitter:title" content="Currency exchange: United States Dollar (USD) - Argentine Peso (ARS)">
<meta name="twitter:description" content="Currencies exchange updated in real time: United States Dollar (USD) - Argentine Peso (ARS) / 1 USD...">
<meta name="twitter:image" content="http://tradukka.com/meta/359ac65be829e896c12a3126fa9c71d0.jpg">
<meta name="twitter:image:width" content="750">
<meta name="twitter:image:height" content="370">
<meta name="twitter:url" content="http://tradukka.com/currency/usd/ars">
If I send a tweet with that link, all the information is being shown in the tweet, except the image:
I also tried using the card validator:
...it fetches everything except the image.
Also, I already tried using twitter:image:src as well. The domain is whitelisted. Any ideas on why the image is not being displayed inside the tweet?
The issue was caused by robots.txt. Make sure that the twitter bot is allowed to crawl the directory containing the twitter images (in this case, /meta).
I am also facing the same problem.
The solution is just switching from summary card with large image to summary.
To apply this you have to edit the script
"meta content='summary_large_image' name='twitter:card'"
as
"meta content='summary' name='twitter:card'"
This solves the problem, but I don't know why the large image type cards are not working.
To avoid reinventing the wheel, which approaches are known to extract product information / service information from previously unknown webpages via machine-learning?
Which keywords in a search engine might give me better results about this topic?
One method is the following:
take a raw html page
strip all html/script tag
count all words and placing them in a list
strip all common words like 'is', 'that', 'we',..
Then you do what you have to do with your keywords.. If you want to crawl you take all links and repeat the procedure
Most webpages have meta-tags in de head-section, those might be useful too:
<head>
<meta name="description" content="webpage with product content">
<meta name="keywords" content="Cameras">
<meta name="author" content="">
<meta charset="UTF-8">
</head>