Retrieving Data in a While Loop from the Youtube API - youtube-api

Question

I do have a hughe database where some data sets link to certain youtube videos. As we all know some youtube videos disappear after a while from youtube and this leads to my solution and my problem as well --> I'd like to check if the youtube video still exists by simply checking via JSON if there is data to retrieve from a video. If not than I'd simply delete that certain data set.
So the first part of my solution would be to go through each row of my data table and checking for each id if there is data to retrieve from youtube as seen in the following code:
$result = $db->query("SELECT id, link FROM songs");
while($row = $result->fetch_assoc())
{
$number = 1+$rown++;
$id = $row['id'];
$link = $row['link'];
$video_ID = $link;
$JSON = file_get_contents("https://gdata.youtube.com/feeds/api/videos/{$video_ID}?v=2&alt=json");
$JSON_Data = json_decode($JSON);
$views = $JSON_Data->{'entry'}->{'yt$statistics'}->{'viewCount'};
echo $number .' row<br />';
echo $link .' link<br />';
echo $views .' views<br /><br />';
}
This attempt works fine and outputs me the data I need. The only problem is, that it just gets me data from the first 150-190 rows and that's it. Now I am checking for a solution that checks each row for empty youtube data and this lead to two concrete questions I have:
1st) Might youtube be responsible for that due to a restriction in retrieving data from one single query?
2nd) Might this be a server issue of mine that stops queries after x-seconds (but I already expand the time limit by putting a line set_time_limit (10000000); into my php code but without success)?
Hope you can help, thanks in advance.

Answer 1

YouTube, naturally, enforces limits on how many requests you can make per period of time. Unfortunately, there are no clear guidelines on what those limits are ... for v2, the guidelines merely state:
The YouTube API enforces quotas to prevent problems associated with
irregular API usage. Specifically, a too_many_recent_calls error
indicates that the API servers have received too many calls from the
same caller in a short amount of time. If you receive this type of
error, then we recommend that you wait a few minutes and then try your
request again.
If time isn't an issue for you, you could slow down each query so that you only make 1 request per every 10-15 seconds or so. Alternatively, you'd probably have better luck batch processing. With this, you can make up to 50 requests at once (this counts as 50 requests against your overall request per day quota, but only as one against your per time quota). Batch processing with v2 of the API is a little involved, as you make a POST request to a batch endpoint first, and then based on those results you can send in the multiple requests. Here's the documentation:
https://developers.google.com/youtube/2.0/developers_guide_protocol?hl=en#Batch_processing
Batch processing is much easier with v3, as you just have the videoId parameter be a comma delimited list of the videos you want info on -- so in your case, you'd execute file_get_contents on a URL like this:
https://www.googleapis.com/youtube/v3/videos?part=id&id={comma-separated-list-of-IDs}&maxResults=50&key={YOUR_API_KEY}
Any video ID in your list that doesn't come back in the JSON response doesn't exist anymore. IF you do 50 at a time, wait for 15 seconds, do another 50, etc., that should give you better performance.