I'm trying to implement Story Mention rendering according to IG messenger graph API.
IG webhooks sends the payload URL of the media as CDN URLs that are extensionless,
which means I can't detect the file type(could be any kind of image or a video file).
The purpose is to render the URL to an HTML element and to prevent saving some file extensions.
Did anybody find out how to get this information?
An example for IG CDN URL
https://lookaside.fbsbx.com/ig_messaging_cdn/?asset_id=17952754300482708&signature=AbxVoHUcW3qKGZvE0FwrbpSEKBqkYGH9wFDUY9xnywlxxek8lWtrTwE173Sxhta9jbp0bgDiL17IpyiI82vqHGNPUD1wdMUZphwQOggW-_877cCI1BxaY_aDUZ8hj5OwmHK9E8OnSybqtMVmGXCX_hBF399t1Hb44zspeL3d9NWb9rib
Python:
import requests
res = requests.head(url)
print res.headers
I was able to retrieve the content type by making a request with node-fetch.
const fetch = require('node-fetch');
const response = await fetch(mediaUrl, { method: 'HEAD' });
const contentType = response.headers.get('Content-Type');
I'm working on a website to load multiple youtube channels live streams. At first i was trying to figure out a way to do this without utilizing youtube's api but have decided to give in.
To find whether a channel is live streaming and to get the live stream links I've been using:
https://www.googleapis.com/youtube/v3/search?part=snippet&channelId={CHANNEL_ID}&eventType=live&maxResults=10&type=video&key={API_KEY}
However with the minimum quota being 10000 and each search being worth 100, Im only able to do about 100 searches before I exceed my quota limit which doesn't help at all. I ended up exceeding the quota limit in about 10 minutes. :(
Does anyone know of a better way to figure out if a channel is currently live streaming and what the live stream links are, using as minimal quota points as possible?
I want to reload youtube data for each user every 3 minutes, save it into a database, and display the information using my own api to save server resources as well as quota points.
Hopefully someone has a good solution to this problem!
If nothing can be done about links just determining if the user is live without using 100 quota points each time would be a big help.
Since the question only specified that Search API quotas should not be used in finding out if the channel is streaming, I thought I would share a sort of work-around method. It might require a bit more work than a simple API call, but it reduces API quota use to practically nothing:
I used a simple Perl GET request to retrieve a Youtube channel's main page. Several unique elements are found in the HTML of a channel page that is streaming live:
The number of live viewers tag, e.g. <li>753 watching</li>. The LIVE NOW
badge tag: <span class="yt-badge yt-badge-live" >Live now</span>.
To ascertain whether a channel is currently streaming live requires a simple match to see if the unique HTML tag is contained in the GET request results. Something like: if ($get_results =~ /$unique_html/) (Perl). Then, an API call can be made only to a channel ID that is actually streaming, in order to obtain the video ID of the stream.
The advantage of this is that you already know the channel is streaming, instead of using thousands of quota points to find out. My test script successfully identifies whether a channel is streaming, by looking in the HTML code for: <span class="yt-badge yt-badge-live" > (note the weird extra spaces in the code from Youtube).
I don't know what language OP is using, or I would help with a basic GET request in that language. I used Perl, and included browser headers, User Agent and cookies, to look like a normal computer visit.
Youtube's robots.txt doesn't seem to forbid crawling a channel's main page, only the community page of a channel.
Let me know what you think about the pros and cons of this method, and please comment with what might be improved rather than disliking if you find a flaw. Thanks, happy coding!
2020 UPDATE
The yt-badge-live seems to have been deprecated, it no longer reliably shows whether the channel is streaming. Instead, I now check the HTML for this string:
{"text":" watching"}
If I get a match, it means the page is streaming. (Non-streaming channels don't contain this string.) Again, note the weird extra whitespace. I also escape all the quotation marks since I'm using Perl.
Here are my two suggestions:
Check my answer where I explain how you can check how retrieve videos from channels who are livesrteaming.
Another option could be use the following URL and somehow make request(s) each time for check if there's a livestreaming.
https://www.youtube.com/channel/<CHANNEL_ID>/live
Where CHANNEL_ID is the channel id you want check if that channel is livestreaming1.
1 Just notice that maybe the URL wont work in all channels (and that depends of the channel itself).
For example, if you check the channel_id UC7_YxT-KID8kRbqZo7MyscQ - link to this channel livestreaming - https://www.youtube.com/channel/UC4nprx9Vd84-ly7N-1Ce6Og/live, this channel will show if he is livestreaming, but, with his channel id UC4nprx9Vd84-ly7N-1Ce6Og - link to this channel livestreaming -, it will show his main page instead.
Adding to the answer by Bman70, I tried eliminating the need of making a costly search request after knowing that the channel is streaming live. I did this using two indicators in the HTML response from channels page who are streaming live.
function findLiveStreamVideoId(channelId, cb){
$.ajax({
url: 'https://www.youtube.com/channel/'+channelId,
type: "GET",
headers: {
'Access-Control-Allow-Origin': '*',
'Accept-Language': 'en-US, en;q=0.5'
}}).done(function(resp) {
//one method to find live video
let n = resp.search(/\{"videoId[\sA-Za-z0-9:"\{\}\]\[,\-_]+BADGE_STYLE_TYPE_LIVE_NOW/i);
//If found
if(n>=0){
let videoId = resp.slice(n+1, resp.indexOf("}",n)-1).split("\":\"")[1]
return cb(videoId);
}
//If not found, then try another method to find live video
n = resp.search(/https:\/\/i.ytimg.com\/vi\/[A-Za-z0-9\-_]+\/hqdefault_live.jpg/i);
if (n >= 0){
let videoId = resp.slice(n,resp.indexOf(".jpg",n)-1).split("/")[4]
return cb(videoId);
}
//No streams found
return cb(null, "No live streams found");
}).fail(function() {
return cb(null, "CORS Request blocked");
});
}
However, there's a tradeoff. This method confuses a recently ended stream with currently live streams. A workaround for this issue is to get status of the videoId returned from Youtube API (costs a single unit from your quota).
I found youtube API to be very restrictive given the cost of search operation. Apparently the accepted answer did not work for me as I found the string on non live streams as well. Web scraping with aiohttp and beautifulsoup was not an option since the better indicators required javascript support. Hence I turned to selenium. I looked for the css selector
#info-text
and then search for the string Started streaming or with watching now in it.
To reduce load on my tiny server that would have otherwise required lot more resources, I moved this test of functionality to a heroku dyno with a small flask app.
# import flask dependencies
import os
from flask import Flask, request, make_response, jsonify
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
base = "https://www.youtube.com/watch?v={0}"
delay = 3
# initialize the flask app
app = Flask(__name__)
# default route
#app.route("/")
def index():
return "Hello World!"
# create a route for webhook
#app.route("/islive", methods=["GET", "POST"])
def is_live():
chrome_options = Options()
chrome_options.binary_location = os.environ.get('GOOGLE_CHROME_BIN')
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
chrome_options.add_argument('--headless')
chrome_options.add_argument('--remote-debugging-port=9222')
driver = webdriver.Chrome(executable_path=os.environ.get('CHROMEDRIVER_PATH'), chrome_options=chrome_options)
url = request.args.get("url")
if "youtube.com" in url:
video_id = url.split("?v=")[-1]
else:
video_id = url
url = base.format(url)
print(url)
response = { "url": url, "is_live": False, "ok": False, "video_id": video_id }
driver.get(url)
try:
element = WebDriverWait(driver, delay).until(EC.presence_of_element_located((By.CSS_SELECTOR, "#info-text")))
result = element.text.lower().find("Started streaming".lower())
if result != -1:
response["is_live"] = True
else:
result = element.text.lower().find("watching now".lower())
if result != -1:
response["is_live"] = True
response["ok"] = True
return jsonify(response)
except Exception as e:
print(e)
return jsonify(response)
finally:
driver.close()
# run the app
if __name__ == "__main__":
app.run()
You'll however need to add the following buildpacks in settings
https://github.com/heroku/heroku-buildpack-google-chrome
https://github.com/heroku/heroku-buildpack-chromedriver
https://github.com/heroku/heroku-buildpack-python
Set the following Config Vars in settings
CHROMEDRIVER_PATH=/app/.chromedriver/bin/chromedriver
GOOGLE_CHROME_BIN=/app/.apt/usr/bin/google-chrome
You can find supported python runtime here but anything below python 3.9 should be good since selenium had problems with improper use of is operator
I hope youtube will provide better alternatives than workarounds.
I know this is a old thread, but i thought i share my way of checking to for example grab the status code to use in an app.
This is for a single Channel, but you could easly do a foreach with it.
<?php
#####
$ytchannelID = "UCd0BTXriKLvOs1ANx3puZ3Q";
#####
$ytliveurl = "https://www.youtube.com/channel/".$ytchannelID."/live";
$ytchannelLIVE = '{"text":" watching now"}';
$contents = file_get_contents($ytliveurl);
if ( strpos($contents, $ytchannelLIVE) !== false ){http_response_code(200);} else {http_response_code(201);}
unset($ytliveurl);
?>
Adding onto the other answers here, I use a GET request to https://www.youtube.com/c/<CHANNEL_NAME>/live and then search for "isLive":true (rather than {"text":" watching"})
As YouTube official documentation about implement and immigrate to API V3, they said:
YouTube Data API (v2) functionality: Retrieve video recommendations
The v3 API does not retrieve a list that only contains videos recommended for the current API user. However, you can use the v3 API to find recommended videos by calling the activities.list method and setting the home parameter value to true.
But now the parameter home has been deprecated too.
Currently, when I set the home parameter to true, I only retrieve the recently uploaded video in the channel: Popular video in YouTube. There are no video with snippet.type=recommendation at all.
I need to show recommended videos of authenticated user in new feed, but seem like this feature is completely deprecated by YouTube.
Anyone has solution for that?
Thanks first!
Unfortunately, I can't find any documentation or example about this feature. It seems that this has been deprecated. However, you may check this documentation with sample JSON structure that shows the format of a activities resource such as recommendation:
"recommendation": {
"resourceId": {
"kind": string,
"videoId": string,
"channelId": string,
},
Hope this helps!
I found this youtubes search api. All we need to do is put a video id in the relatedToVideoId and it'll giveout a list of videos related to it.
The docs for the api include a way to test the request. code samples there show how to set 'mine' for an authenticated request.
youtube activities
This is android sample code. it would need to be in some background thread. The setmine = true on the channelList response is like the home (I think). Was not sure if your implementation was for the web or an app.
this is android code:
YouTube youtube = new YouTube.Builder(transport, jsonFactory,
credential).setApplicationName(getString(R.string.app_name))
.build();
YouTube.Activities.List activities;
ActivityListResponse activityListResponse = null;
List<ActivityData> activitiesData = new ArrayList<ActivityData>();
try {
/*
* Now that the user is authenticated, the app makes a
* channels list request to get the authenticated user's
* channel. Returned with that data is the playlist id for
* the uploaded videos.
* https://developers.google.com/youtube
* /v3/docs/channels/list
*/
ChannelListResponse clr = youtube.channels().list("contentDetails")
.setMine(true).execute();
activities = youtube.activities().list("id,snippet,subscriberSnippet");
activities.setChannelId(clr.getItems().get(0).getId());
activities.setMaxResults((long) 50);
activityListResponse = activities.execute();
ArrayList<String> subscriptionListIdentifier = new ArrayList<String>()
,listTitles = new ArrayList<String>()
,listThumbnails = new ArrayList<String>();
List<Activity> results = activityListResponse.getItems();
for (Activity activity : results) {
listTitles.add(activity.getSnippet().getTitle());
listThumbnails.add(activity.getSnippet().getThumbnails().getDefault().getUrl());
subscriptionListIdentifier.add(activity.getId());
//if ("public".equals(playlist.getStatus()
// .getPrivacyStatus())) {
ActivityData data = new ActivityData();
data.setActivity(activity);
activitiesData.add(data);
//}
}
return activitiesData;
You can retrieve them using the following API call:
GET https://www.googleapis.com/youtube/v3/activitiespart=snippet%2CcontentDetails&channelId={channel—_Id}&maxResults=25®ionCode=tw&key={YOUR_API_KEY}
I'm taking a youtube url as user input.
The logic I have is as follows:
if(URL === link_to_video) then get video
else if( URL == link_to_channel) then get all_videos_of_channel.
I am doing this via JavaScript and using the YouTube API v3.
The problem is, it seems youtube has two types of URLs to youtube channels.
/channel/, eg:
www.youtube.com/channel/UCaHNFIob5Ixv74f5on3lvIw
/user/,
eg: www.youtube.com/user/CalvinHarrisVEVO
both the above links will take you to the same channel, but my current uploader code supports only /user/CalvinHarrisVEVO.
is there any way to make both the URLs behave similarly in terms of obtaining the channel videos?
One solution would be to parse the url and then apply this logic:
if there is `channel` in the url link
call the APIv3 with the **id** of the channel
Ressource : youtube.channels.list
else if there is 'user'
call the APIv3 with the **forUsername** of the channel
Ressource : youtube.channels.list
Check : https://developers.google.com/apis-explorer/#p/youtube/v3/youtube.channels.list
I am trying to get the right redirection URL for my sharepoint documents which then I can use to open documents in WebView of iOS. Currently I am giving the absolute URL for the document where the doc is rendered inside WebView as PDF(Image/Readonly). Whereas I want to redirect to office webapp. Now my issue is I dont know if the URL for office web app is something which I can construct like appending /_layouts/15/WopiFrame.aspx?sourcedoc= or is the URL custom based on installations and we need to call some Sharepoint API which will let us know what is the base URL for Wopi service.
Currently I am passing URL like - https://.sharepoint.com/Shared%20Documents/demo/demo.docx
Whereas I want to pass URL like - https://.sharepoint.com/_layouts/15/WopiFrame.aspx?sourcedoc=/Shared%20Documents/demo/demo.docx
Looking forward for help.
Thanks in advance,
Vishwesh
File f = clientContext.Web.GetFileByServerRelativeUrl("/sites/ /Shared%20Documents/Title.docx");
clientContext.Load(f);
clientContext.ExecuteQuery();
ClientResult<String> result = f.ListItemAllFields.GetWOPIFrameUrl(SPWOPIFrameAction.Edit);
clientContext.Load(f.ListItemAllFields);
clientContext.ExecuteQuery();
result.Value contains a URL, something like this:
http://sharep.xxx:8080/sites/zxxx/_layouts/15/WopiFrame.aspx?sourcedoc=%2Fsites%2Fzxxx%2FShared%20Documents%2FTitle%2Edocx&action=edit
Also you can extract the extract Office Web Apps URL from the above page, if you don't want to hit the sharepoint at all.
using Microsoft.SharePoint.Client;
using Microsoft.SharePoint.Client.Utilities;
// Assume we have these variables:
// ctx: A valid client context
// serverRelativeUrl: the URL of the document
File f = ctx.Web.GetFileByServerRelativeUrl (serverRelativeUrl);
result = f.ListItemAllFields.GetWOPIFrameUrl(SPWOPIFrameAction.Edit);
ctx.Load(f.ListItemAllFields);
ctx.ExecuteQuery();
This builds on the answer from #thebitlic which was the silver bullet for sure! However he or she is doing two calls to the server. Through the wonders of CSOM batching, it's possible to do it in one round trip, and no need to bring back the File object at all.