I have a problem on video search. With a French IP, everything works well, but with Ireland IP no.
I try :
https://api.dailymotion.com/user/xm44zy/videos?search=RC%20Lens on French IP -> 500+ results
https://api.dailymotion.com/user/xm44zy/videos?search=RC%20Lens on Ireland IP -> 1 result
Maybe Video access error (DM007 Video geo-restricted by its owner) https://developer.dailymotion.com/api#access-error ?
No because https://api.dailymotion.com/video/x6f8qjq works well. Not found on search query (with Ireland IP) but I have access to the detail of the video
an idea ?
Thanks.
The API will always returns information about a specific video if you request data with its own resource URI (like: https://api.dailymotion.com/video/ID), even if the video is restricted in your country.
However, when using listings (i.e. asking for a video list, or when searching for videos based on search terms), the list will automatically exclude videos which can't be played in your country.
You can know if a video is geoblocked in your country using the "geoblocking" field:
https://developer.dailymotion.com/api/internal#video-geoblocking-field.
In your case, your example is geo-restricted everywhere else France, that's why it won't show in search results (or any listings): https://api.dailymotion.com/video/x6f8qjq?fields=id,title,geoblocking
returns: "geoblocking": [
"allow",
"fr"
]
This means you can only watch it ( and appears in search results ) in France
Related
I'm working on a website to load multiple youtube channels live streams. At first i was trying to figure out a way to do this without utilizing youtube's api but have decided to give in.
To find whether a channel is live streaming and to get the live stream links I've been using:
https://www.googleapis.com/youtube/v3/search?part=snippet&channelId={CHANNEL_ID}&eventType=live&maxResults=10&type=video&key={API_KEY}
However with the minimum quota being 10000 and each search being worth 100, Im only able to do about 100 searches before I exceed my quota limit which doesn't help at all. I ended up exceeding the quota limit in about 10 minutes. :(
Does anyone know of a better way to figure out if a channel is currently live streaming and what the live stream links are, using as minimal quota points as possible?
I want to reload youtube data for each user every 3 minutes, save it into a database, and display the information using my own api to save server resources as well as quota points.
Hopefully someone has a good solution to this problem!
If nothing can be done about links just determining if the user is live without using 100 quota points each time would be a big help.
Since the question only specified that Search API quotas should not be used in finding out if the channel is streaming, I thought I would share a sort of work-around method. It might require a bit more work than a simple API call, but it reduces API quota use to practically nothing:
I used a simple Perl GET request to retrieve a Youtube channel's main page. Several unique elements are found in the HTML of a channel page that is streaming live:
The number of live viewers tag, e.g. <li>753 watching</li>. The LIVE NOW
badge tag: <span class="yt-badge yt-badge-live" >Live now</span>.
To ascertain whether a channel is currently streaming live requires a simple match to see if the unique HTML tag is contained in the GET request results. Something like: if ($get_results =~ /$unique_html/) (Perl). Then, an API call can be made only to a channel ID that is actually streaming, in order to obtain the video ID of the stream.
The advantage of this is that you already know the channel is streaming, instead of using thousands of quota points to find out. My test script successfully identifies whether a channel is streaming, by looking in the HTML code for: <span class="yt-badge yt-badge-live" > (note the weird extra spaces in the code from Youtube).
I don't know what language OP is using, or I would help with a basic GET request in that language. I used Perl, and included browser headers, User Agent and cookies, to look like a normal computer visit.
Youtube's robots.txt doesn't seem to forbid crawling a channel's main page, only the community page of a channel.
Let me know what you think about the pros and cons of this method, and please comment with what might be improved rather than disliking if you find a flaw. Thanks, happy coding!
2020 UPDATE
The yt-badge-live seems to have been deprecated, it no longer reliably shows whether the channel is streaming. Instead, I now check the HTML for this string:
{"text":" watching"}
If I get a match, it means the page is streaming. (Non-streaming channels don't contain this string.) Again, note the weird extra whitespace. I also escape all the quotation marks since I'm using Perl.
Here are my two suggestions:
Check my answer where I explain how you can check how retrieve videos from channels who are livesrteaming.
Another option could be use the following URL and somehow make request(s) each time for check if there's a livestreaming.
https://www.youtube.com/channel/<CHANNEL_ID>/live
Where CHANNEL_ID is the channel id you want check if that channel is livestreaming1.
1 Just notice that maybe the URL wont work in all channels (and that depends of the channel itself).
For example, if you check the channel_id UC7_YxT-KID8kRbqZo7MyscQ - link to this channel livestreaming - https://www.youtube.com/channel/UC4nprx9Vd84-ly7N-1Ce6Og/live, this channel will show if he is livestreaming, but, with his channel id UC4nprx9Vd84-ly7N-1Ce6Og - link to this channel livestreaming -, it will show his main page instead.
Adding to the answer by Bman70, I tried eliminating the need of making a costly search request after knowing that the channel is streaming live. I did this using two indicators in the HTML response from channels page who are streaming live.
function findLiveStreamVideoId(channelId, cb){
$.ajax({
url: 'https://www.youtube.com/channel/'+channelId,
type: "GET",
headers: {
'Access-Control-Allow-Origin': '*',
'Accept-Language': 'en-US, en;q=0.5'
}}).done(function(resp) {
//one method to find live video
let n = resp.search(/\{"videoId[\sA-Za-z0-9:"\{\}\]\[,\-_]+BADGE_STYLE_TYPE_LIVE_NOW/i);
//If found
if(n>=0){
let videoId = resp.slice(n+1, resp.indexOf("}",n)-1).split("\":\"")[1]
return cb(videoId);
}
//If not found, then try another method to find live video
n = resp.search(/https:\/\/i.ytimg.com\/vi\/[A-Za-z0-9\-_]+\/hqdefault_live.jpg/i);
if (n >= 0){
let videoId = resp.slice(n,resp.indexOf(".jpg",n)-1).split("/")[4]
return cb(videoId);
}
//No streams found
return cb(null, "No live streams found");
}).fail(function() {
return cb(null, "CORS Request blocked");
});
}
However, there's a tradeoff. This method confuses a recently ended stream with currently live streams. A workaround for this issue is to get status of the videoId returned from Youtube API (costs a single unit from your quota).
I found youtube API to be very restrictive given the cost of search operation. Apparently the accepted answer did not work for me as I found the string on non live streams as well. Web scraping with aiohttp and beautifulsoup was not an option since the better indicators required javascript support. Hence I turned to selenium. I looked for the css selector
#info-text
and then search for the string Started streaming or with watching now in it.
To reduce load on my tiny server that would have otherwise required lot more resources, I moved this test of functionality to a heroku dyno with a small flask app.
# import flask dependencies
import os
from flask import Flask, request, make_response, jsonify
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
base = "https://www.youtube.com/watch?v={0}"
delay = 3
# initialize the flask app
app = Flask(__name__)
# default route
#app.route("/")
def index():
return "Hello World!"
# create a route for webhook
#app.route("/islive", methods=["GET", "POST"])
def is_live():
chrome_options = Options()
chrome_options.binary_location = os.environ.get('GOOGLE_CHROME_BIN')
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
chrome_options.add_argument('--headless')
chrome_options.add_argument('--remote-debugging-port=9222')
driver = webdriver.Chrome(executable_path=os.environ.get('CHROMEDRIVER_PATH'), chrome_options=chrome_options)
url = request.args.get("url")
if "youtube.com" in url:
video_id = url.split("?v=")[-1]
else:
video_id = url
url = base.format(url)
print(url)
response = { "url": url, "is_live": False, "ok": False, "video_id": video_id }
driver.get(url)
try:
element = WebDriverWait(driver, delay).until(EC.presence_of_element_located((By.CSS_SELECTOR, "#info-text")))
result = element.text.lower().find("Started streaming".lower())
if result != -1:
response["is_live"] = True
else:
result = element.text.lower().find("watching now".lower())
if result != -1:
response["is_live"] = True
response["ok"] = True
return jsonify(response)
except Exception as e:
print(e)
return jsonify(response)
finally:
driver.close()
# run the app
if __name__ == "__main__":
app.run()
You'll however need to add the following buildpacks in settings
https://github.com/heroku/heroku-buildpack-google-chrome
https://github.com/heroku/heroku-buildpack-chromedriver
https://github.com/heroku/heroku-buildpack-python
Set the following Config Vars in settings
CHROMEDRIVER_PATH=/app/.chromedriver/bin/chromedriver
GOOGLE_CHROME_BIN=/app/.apt/usr/bin/google-chrome
You can find supported python runtime here but anything below python 3.9 should be good since selenium had problems with improper use of is operator
I hope youtube will provide better alternatives than workarounds.
I know this is a old thread, but i thought i share my way of checking to for example grab the status code to use in an app.
This is for a single Channel, but you could easly do a foreach with it.
<?php
#####
$ytchannelID = "UCd0BTXriKLvOs1ANx3puZ3Q";
#####
$ytliveurl = "https://www.youtube.com/channel/".$ytchannelID."/live";
$ytchannelLIVE = '{"text":" watching now"}';
$contents = file_get_contents($ytliveurl);
if ( strpos($contents, $ytchannelLIVE) !== false ){http_response_code(200);} else {http_response_code(201);}
unset($ytliveurl);
?>
Adding onto the other answers here, I use a GET request to https://www.youtube.com/c/<CHANNEL_NAME>/live and then search for "isLive":true (rather than {"text":" watching"})
I do not want my complete site to be blocked in a country, instead I want to block some countries or IPs from accessing some pages on site.
If someone can just give me a push in the right direction that would be great.
You did not specify the language you are using, but you can either use a local IP to country database or query an API and reject the visitor if its country code matches your block list :
$country = file_get_contents("http://api.db-ip.com/v2/free/{$_SERVER['REMOTE_ADDR']}/countryCode");
if (in_array($country, [ "US", "CA" ])) {
header("Status: 403 Forbidden");
die();
}
I am using v3 api and videoEmbeddable="true" in my search request.
However, when I try to play the video(s) on my webpage, it says the content is blocked to be displayed on this site. Watch it on Youtube.
Is there a parameter that I can use which will not return non-embeddable videos?
*EDIT *
Video Response:
{u'items': [{u'snippet': {u'title': u'Jerez - Yamaha Preview'}, u'contentDetails': {u'definition': u'hd', u'contentRating': {u'ytRating': u'ytAllAudiences'}, u'caption': u'false', u'duration': u'PT1M21S', u'licensedContent': True, u'dimension': u'2d'}, u'status': {u'publicStatsViewable': True, u'privacyStatus': u'public', u'uploadStatus': u'processed', u'license': u'youtube', u'embeddable': True}, u'id': u'aaR72Xf_4wc'}]}
Query:
return yt_service_v3.videos().list(
id='aaR72Xf_4wc',
part="id,snippet,contentDetails,status",
fields="items(id,snippet(title),contentDetails,status)",
).execute()
This video is not embeddable on my webpage and in the response there is nothing to suggest that this video is not embeddable.
I request both contentDetails and status. I am not requesting via mobile device
Please provide pointers.
A small subset of videos have domain-level whitelists or blacklists applied to them by their content owners. Those restrictions are not reflected in API responses.
This is what you're seeing with aaR72Xf_4wc and the relevant portion of the blog post about playback restrictions is
There are even more subtle restrictions that occasionally come into
play. Not all of these are currently queryable via the API. For
instance, some videos are only playable on a certain set of domains.
I assume it wasn't there 2.5 years ago, but the current API does have indication for regional block, when requesting contentDetails. here's a response for such request for 2 videos, the first one is blocked and the second one is not. Note the regionRestriction key:
{u'etag': u'"iDqJ1j7zKs4x3o3ZsFlBOwgWAHU/ZqEbAZJ-JFnYX3v37ZRuhK8FFN8"',
u'items': [{u'contentDetails': {u'caption': u'false',
u'definition': u'sd',
u'dimension': u'2d',
u'duration': u'PT2M5S',
u'licensedContent': False,
u'regionRestriction': {u'blocked': [u'NU',
u'BY',
u'NP',
u'NR',
u'WS',
u'TD',
u'SY',
u'NZ',
u'NE',
.
.
.
u'TG']}},
u'etag': u'"iDqJ1j7zKs4x3o3ZsFlBOwgWAHU/hM4srkgNUiZj2GtAhLDFoFrHnlk"',
u'id': u'WGf4mSBoCzQ',
u'kind': u'youtube#video'},
{u'contentDetails': {u'caption': u'false',
u'definition': u'sd',
u'dimension': u'2d',
u'duration': u'PT17M2S',
u'licensedContent': False},
u'etag': u'"iDqJ1j7zKs4x3o3ZsFlBOwgWAHU/JNjXh3lMwcaC9gtbzM-KmOQLh44"',
u'id': u'Tw75Dt8fBBs',
u'kind': u'youtube#video'}],
u'kind': u'youtube#videoListResponse',
u'pageInfo': {u'resultsPerPage': 2, u'totalResults': 2}}
You can also filter by syndication if you are accessing from a mobile device. Last but not the least there are region specific blockages.
Maybe you need to also include the filter for videoSyndicated. I don't know for sure though; it's a little confusing to me.
Using the YouTube API, how can I fetch structured data about a video? For instance, given a video that is a movie trailer, how can I find the movie title or topic in a structured format, and other interesting metadata such as the celebrities featured or the director?
Yes. The v3 API allows you do do this via videos/list. Here's an example API call using a trailer for The Dark Knight Rises:
GET https://www.googleapis.com/youtube/v3/videos?part=topicDetails&id=g8evyE9TuYk&fields=items%2FtopicDetails%2FtopicIds&key={YOUR_API_KEY}
(To get a developer key, follow the instructions in our video tutorial).
That sets:
topic: topicDetails
id: g8evyE9TuYk
fields: items/topicDetails/topicIds
In the response body, you get:
{
"items": [
{
"topicDetails": {
"topicIds": [
"/m/0bpm4yw",
"/m/01d5g",
"/m/0btpm6",
"/m/0184dt",
"/m/02vxn"
]
}
}
]
}
These correspond to Freebase mids, which you can lookup using the Freebase API. For instance, the first response corresponds to the Freebase entity The Dark Knight Rises (movie). Once you look up the entity, you can look up related entities such as the director, actors, or genre.
Play around with the API explorer a bit. When you're ready to translate this into code, watch out video about turning the API reference into code.
I found that I got the metadata details I wanted via this request url:
https://www.googleapis.com/youtube/v3/videos?part=snippet,contentDetails&id={YouTubeVideoID}&key={YourAPIkey}
Here is documentation with description of all the returned data fields.
Here's info on how to get an API key.
I have tried this YouTube API v3 call from curl today:
http://www.googleapis.com/youtube/v3/videos?id=ZCJgvabihQ8&key=apiKey&part=snippet,recordingDetails
Everything else is fine, but I didn't find the recordingDetails object in the response json:
{
"kind":"youtube#videoListResponse",
"etag":"\"oLweQuB9Vh7wAB9a0AIHg_K-wsM/EuRsJ-sxI3qstP1T58S5Qnb_NIg\"",
"items":[
{
"id":"ZCJgvabihQ8",
"kind":"youtube#video",
"etag":"\"oLweQuB9Vh7wAB9a0AIHg_K-wsM/CYNTRL05S4okPzkUfE5LbrRKt9g\"",
"snippet":{
"publishedAt":"2013-01-25T13:36:19.000Z",
"channelId":"UCpVm7bg6pXKo1Pr6k5kxG9A",
"title":"Weird Nature: Pink Dolphins?",
"description":"Up to 9 feet long and weighing up to 300 pounds, pink river dolphins are the largest freshwater dolphins in the world.",
"thumbnails":{
"default":{
"url":"https://i.ytimg.com/vi/ZCJgvabihQ8/default.jpg"
},
"medium":{
"url":"https://i.ytimg.com/vi/ZCJgvabihQ8/mqdefault.jpg"
},
"high":{
"url":"https://i.ytimg.com/vi/ZCJgvabihQ8/hqdefault.jpg"
}
},
"categoryId":"24"
}
}
]
}
Is there anything I missed? Many thanks.
recordingDetails is not public for all videos, so if a video doesn't have these attributes available, nothing will be returned. By not returning anything, YouTube saves bandwidth and the response time can be faster.
Original reponse:
This looks like a legitimate bug...
recordingDetails is listed as an option on the videos resource
overview page, but not listed as a valid part in the video list method
here: https://developers.google.com/youtube/v3/docs/videos/list
Normally when you try using an invalid part you get a "400 Bad
Request" error, but recordingDetails still returns a 200 response, so
it looks like it really is supposed to be returned...
You should submit a bug report here:
https://code.google.com/p/gdata-issues/issues/entry?template=YouTube%20(Defect%20Report)
recordingDetails will only be returned for videos that explicitly have either the geolocation or the recording time set.
(Not all videos set recording time; it's a distinct piece of metadata from publication time.)
You can do a list call to retrieve a list of videos based on a location and locationRadius.
Then, with the returned list you can do a videos call including all video IDs in the id parameter of the query. You can then specify recordingDetails as "part" and it will work.
This is how Youtube does it here:
https://github.com/youtube/geo-search-tool