I use the youtube api v3 and i would like to understand how does the Etag. I would like to use it for what it takes to cache purpose but I do not know what to do in PHP.
Could you tell me the steps to follow once the etag recovered ? please. Thanks for help.
According to the youtube docs (https://developers.google.com/youtube/v3/getting-started#etags), an eTag is basically used to determine if a resource has changed. Use them for:
Optimization - Caching youtube resources in your app can reduce bandwidth and latency. When caching, store the eTag so that you can include it when getting a resource. If the resource has not changed, you will get a 304 response code (NOT MODIFIED), which means you can use your cached version. Otherwise, you will get the updated version of the resource.
Quota Usage - You can reduce the amount you tap into your quota by caching youtube data. The first time you get the resource, you will tap into your quota. Before displaying the resource, first check to see if your cached resource has changed, which will only cost you 1 quota unit. If the resource has not changed, youtube will return a 304 response. If it has changed, you can get the resource again, costing various quota units depending on what you are getting. For more on your quota: (https://developers.google.com/youtube/v3/getting-started#quota).
Overwrite protection - If you are overwriting a resource, including the eTag will ensure that you are not overwriting a newer version of the resource.
eTags are part of the HTTP 1.1 spec (http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.19) and are used in the headers of the request/response. Here's a good article that talks about them at a low level: http://www.ibuildings.com/blog/2013/07/etags-uninitiated
As far as using eTags in PHP, I can only suggest a couple things since I've never done it. YouTube returns eTags for feeds AND individual items within a feed, and I'm not sure how to use them for individual items within a feed. But to get the original feed itself, essentially you would use curl and add the eTag to the header of your request (PHP cURL custom headers). You might also want to check out http_cache_etag (http://www.php.net/manual/en/function.http-cache-etag.php)
I was looking for similar information, but I couldn't find a clear example on the youtube website. On the other hand, it seems facebook is using a similar approach (Etags to check whether a resource has changed) and these two links I found on facebook developers area might be of help:
https://developers.facebook.com/docs/reference/ads-api/etags-reference/ and https://developers.facebook.com/blog/post/627/
The first one explains in a simpler and more detailed way how etags are used and provide some request/response examples.
The second link provides a PHP example on how to retrieve a resource and extract the etag and use it in a subsequent request.
Of course these links contain information related to facebook website, but for the great part they can be applied to youtube as well.
I am not sure if anyone would still be interested but I have posted an answer on how to use etag in using the youtube api here. The idea works not only for the youtube api. The post is quite long but hope it can help.
Related
I'm building an alternative client for YouTube subscriptions browsing (folder based subscriptions with according feed generated), and I'm making a lot of requests to YouTube to aggregate that data.
I'm caching a lot of requests as it is not needed to refresh them once it has been fetched on any other day before the current one.
The fact is, current-day refreshes are consuming a lot, and I reach my quota pretty fast even though those requests are read-only.
I submitted that YouTube quota increase request form, but still, I'm quite afraid.
Am I missing something with the userIp & quotaUser parameters ?
Shouldn't those requests - as they are pretty much the same that a normal user would do on the regular YouTube client - be considered as "Queries per 100 seconds per user" ?
My main quota, the "Queries per day" currently seems to handle ALL the requests coming from my app, even though I added the quotaUser parameter on all my requests made by a user on the frontend.
I think I am missing something as my app should not be considered as "data consuming" as it is sending almost nothing to YouTube in terms of data, and it is just reading data that is also available on the YouTube main client, but not in the same format..
Thanks for your help.
Does the YouTube Data API Client Library for Java use Etags and/or gzip, as described at Getting started page?
Documentation is short (only find java docs) and don't say anything about it, so i guess is just a wrapper.
Based from this link, Etags are supported by youtube but it depends on what kind of data you are asking.
To use the etag, create a header request and put "If-None-Match" equal to your etag value. Note this should be a request header and not appended to the endpoint call. You can also use "If-Match".
Depending on what kind of API you are using, the way of inserting a new value to the request header may differ slightly. The ETag response-header field provides the current value of the entity tag for the requested variant.
You may also check on this related thread.
I'm trying to use etags in order to reduce both my bandwidth and my quota usage but /search returns an new etag even when nothing changed. It also still sends the content if I specify the previous etag in the header.
Is it supported for that api call or am I probably doing something wrong?
Etags are supported by youtube but it depends on what kind of data you are asking
ETags, a standard part of the HTTP protocol, allow applications to refer to a specific version of a particular API resource. The resource could be an entire feed or an item in that feed. This functionality supports the following use cases:
Caching and conditional retrieval – Your application can cache API
resources and their ETags. Then, when your application requests a
stored resource again, it specifies the ETag associated with that
resource. If the resource has changed, the API returns the modified
resource and the ETag associated with that version of the resource. If
the resource has not changed, the API returns an HTTP 304 response
(Not Modified), which indicates that the resource has not changed.
Your application can reduce latency and bandwidth usage by serving
cached resources in this manner.
The client libraries for Google APIs differ in their support of ETags.
For example, the JavaScript client library supports ETags via a
whitelist for allowed request headers that includes If-Match and
If-None-Match. The whitelist allows normal browser caching to occur so
that if a resource's ETag has not changed, the resource can be served
from the browser cache. The Obj-C client, on the other hand, does not
support ETags. Protecting against inadvertent overwrites of changes –
ETags help to ensure that multiple API clients don't inadvertently
overwrite each other's changes. When updating or deleting a resource,
your application can specify the resource's ETag. If the ETag doesn't
match the most recent version of that resource, then the API request
fails.
Using ETags in your application provides several benefits:
The API responds more quickly to requests for cached but unchanged
resources, yielding lower latency and lower bandwidth usage. Your
application will not inadvertently overwrite changes to a resource
that were made from another API client.
https://developers.google.com/youtube/v3/getting-started#etags
I usually scrape Youtube for Videos searches and I just store the Etag that it return. To use the etag, create a header request and put "If-None-Match" equal to your etag value. Note this should be a request header and not appended to the endpoint call. You can also use "If-Match".
Depending on what kind of API you are using, the way of inserting a new value to the request header may differ slightly
I want to cache results to reduce the calls I make to the API based on whether something actually changed in the channel/playlist.
Is there a way to know when a channel or playlist was last updated using version 3 of the API?
In the previous version for example ProfileEntry had an Updated property.
The YouTube API uses etags for every resource and so you may be able to use them ... it would allow you to send the etag with your request and if nothing has changed, you get back a very quick 304 (so it may not reduce the number of calls you make, but it still reduces latency because your app won't have to wait for large payloads to be returned when your cached version is up-to-date).
https://developers.google.com/youtube/v3/getting-started#etags
So I built a website that uses Twitch.tv API, which is a gaming live stream website. The requests are long and slow, and I would like to cache it somehow. The problem is that there are a lot of dynamic attributes, if they are still online, or how many viewers there are. Since the traffic to my website is low at the moment, expiring Cache early isn't going to help much. Also, I have a page where it lists all the live streams, and it requests to see if the stream is online. So even if no one is online it still takes a while to load. Is there anyway to retrieve api faster without caching?
here is twitch.tv api doc
Since you don't own the Twitch.tv API, unfortunately I would say there is really nothing you can do to make their calls faster.
The good news is that you can cache the calls you make to them, which will make things appear faster to your users.
The way to cache the calls is to create a key and then cache the return JSON from the API. To create the key I would just use the URL you are calling for the API. Then just give the cached value an expiration time of a few minutes and when it expires you make another API call to re-populate the cache.
Also I'd look at Varnish (https://www.varnish-cache.org/) which does a lot of HTTP caching really well. Could work really well for you and it has the concept of a grace period that tries to hide the expensive calls made when the cache expires.