It is possible to parse master playlist to get and store all the url’s associated with the variants, using libav, to make downloads according to the variant of my choice. Thanks, all help is welcome
In case that someone else need this information, i found it, you just need to do this in your program:
AVFormatContext *fmtctx = NULL;
HLSContext *c = fmtctx -> priv_data;
previously you need to add every structure used in "hls.c" (HLSContext, variant, playlist, rendition, etc)
then you can get access to the variant and his associate data (url, bitrate,etc);
int a;
for(a=0; a < c->n_variants; a++){
av_log(NULL, AV_LOG_INFO, "url = %s \n", c->playlists[a]->url);
} /*for printing url's of the master playlist*//
Related
I just want to fetch all my liked videos ~25k items. as far as my research goes this is not possible via the YouTube v3 API.
I have already found multiple issues (issue, issue) on the same problem, though some claim to have fixed it, but it only works for them as they don't have < 5000 items in their liked video list.
playlistItems list API endpoint with playlist id set to "liked videos" (LL) has a limit of 5000.
videos list API endpoint has a limit of 1000.
Unfortunately those endpoints don't provide me with parameters that I could use to paginate the requests myself (e.g. give me all the liked videos between date x and y), so I'm forced to take the provided order (which I can't get past 5k entries).
Is there any possibility I can fetch all my likes via the API?
more thoughts to the reply from #Yarin_007
if there are deleted videos in the timeline they appear as "Liked https://...url" , the script doesnt like that format and fails as the underlying elements dont have the same structure as existing videos
can be easily fixed with a try catch
function collector(all_cards) {
var liked_videos = {};
all_cards.forEach(card => {
try {
// ignore Dislikes
if (card.innerText.split("\n")[1].startsWith("Liked")) {
....
}
}
catch {
console.log("error, prolly deleted video")
}
})
return liked_videos;
}
to scroll down to the bottom of the page ive used this simple script, no need to spin up something big
var millisecondsToWait = 1000;
setInterval(function() {
window.scrollTo(0, document.body.scrollHeight);
console.log("scrolling")
}, millisecondsToWait);
when more ppl want to retrive this kind of data, one could think about building a proper script that is more convenient to use. If you check the network requests you can find the desired data in the response of requests called batchexecute. One could copy the authentification of one of them provide them to a script that queries those endpoints and prepares the data like the other script i currently manually inject.
Hmm. perhaps Google Takeout?
I have verified the youtube data contains a csv called "liked videos.csv". The header is Video Id,Time Added, and the rows are
dQw4w9WgXcQ,2022-12-18 23:42:19 UTC
prvXCuEA1lw,2022-12-24 13:22:13 UTC
for example.
So you would need to retrieve video metadata per video ID. Not too bad though.
Note: the export could take a while, especially with 25k videos. (select only YouTube data)
I also had an idea that involves scraping the actual liked videos page (which would save you 25k HTTP Requests). But I'm unsure if it breaks with more than 5000 songs. (also, emulating the POST requests on that page may prove quite difficult, albeit not impossible. (they fetch /browse?key=..., and have some kind of obfuscated / encrypted base64 strings in the request-body, among other parameters)
EDIT:
Look. There's probably a normal way to get a complete dump of all you google data. (i mean, other than takeout. Email them? idk.)
anyway, the following is the other idea...
Follow this deep link to your liked videos history.
Scroll to the bottom... maybe with selenium, maybe with autoit, maybe put something on the "end" key of your keyboard until you reach your first liked video.
Hit f12 and run this in the developer console
// https://www.youtube.com/watch?v=eZPXmCIQW5M
// https://myactivity.google.com/page?utm_source=my-activity&hl=en&page=youtube_likes
// go over all "cards" in the activity webpage. (after scrolling down to the absolute bottom of it)
// create a dictionary - the key is the Video ID, the value is a list of the video's properties
function collector(all_cards) {
var liked_videos = {};
all_cards.forEach(card => {
// ignore Dislikes
if (card.innerText.split("\n")[1].startsWith("Liked")) {
// horrible parsing. your mileage may vary. I Tried to avoid using any gibberish class names.
let a_links = card.querySelectorAll("a")
let details = a_links[0];
let url = details.href.split("?v=")[1]
let video_length = a_links[3].innerText;
let time = a_links[2].parentElement.innerText.split(" • ")[0];
let title = details.innerText;
let date = card.closest("[data-date]").getAttribute("data-date")
liked_videos[url] = [title,video_length, date, time];
// console.log(title, video_length, date, time, url);
}
})
return liked_videos;
}
// https://stackoverflow.com/questions/57709550/how-to-download-text-from-javascript-variable-on-all-browsers
function download(filename, text, type = "text/plain") {
// Create an invisible A element
const a = document.createElement("a");
a.style.display = "none";
document.body.appendChild(a);
// Set the HREF to a Blob representation of the data to be downloaded
a.href = window.URL.createObjectURL(
new Blob([text], { type })
);
// Use download attribute to set set desired file name
a.setAttribute("download", filename);
// Trigger the download by simulating click
a.click();
// Cleanup
window.URL.revokeObjectURL(a.href);
document.body.removeChild(a);
}
function main() {
// gather relevant elements
var all_cards = document.querySelectorAll("div[aria-label='Card showing an activity from YouTube']")
var liked_videos = collector(all_cards)
// download json
download("liked_videos.json", JSON.stringify(liked_videos))
}
main()
Basically it gathers all the liked videos' details and creates a key: video_ID - Value: [title,video_length, date, time] object for each liked video.
It then automatically downloads the json as a file.
I've been searching for a couple of days on how to get the album art for a song (or a frame capture of a video) from a file path. All I could find is things related to Mediastore like this answer where it requires getting the album ID of the file.
Cursor cursor = getContentResolver().query(MediaStore.Audio.Albums.EXTERNAL_CONTENT_URI,
new String[] {MediaStore.Audio.Albums._ID, MediaStore.Audio.Albums.ALBUM_ART},
MediaStore.Audio.Albums._ID+ "=?",
new String[] {String.valueOf(albumId)},
null);
if (cursor.moveToFirst()) {
String path = cursor.getString(cursor.getColumnIndex(MediaStore.Audio.Albums.ALBUM_ART));
// do whatever you need to do
}
But I can't find a guid on how it works, how can pass the file to Mediastore or how can I get the album ID of the media or anything... Currently I get the media information using MediaMetadataRetriever but I can't find a way to get the album art or a video thumbnail of a media file using it...
** Update :-
If Mediastore must be used to get the media files in the first place before using it to get their data, I can implement it instead of what I'm currently doing (Currently I iterate the device files to get the supported files) and it can be a better option as Mediastore supports getting data from external storages as well.
Any help is appreciated.
If using MediaMetadataRetriever , you can have a try with follow sample code :
private void loadingCover(string mediaUri)
{
MediaMetadataRetriever mediaMetadataRetriever = new MediaMetadataRetriever();
mediaMetadataRetriever.SetDataSource(mediaUri);
byte[] picture = mediaMetadataRetriever.GetEmbeddedPicture();
Android.Graphics.Bitmap bitmap = BitmapFactory.DecodeByteArray(picture, 0, picture.Length);
musicCover.SetImageBitmap(bitmap);
}
In addition , not forgatting to add permission :
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
Invoke it as follow :
File file = new File("/storage/sdcard/Movies/music1.mp4");
if (file.Exists())
{
loadingCover(file.AbsolutePath);
}
I use the Magento 2 REST API and want to handle errors that get thrown in C++.
Example error response:
{
"message": "Could not save category: %1",
"parameters": [
"URL key for specified store already exists."
]
}
I can retrieve both of these into a String and std::vector which leaves me with the question:
How am I able to return a String that is formatted by filling the placeholders?
With a fixed size, I could do something along the lines of this
char* buffer = new char[200];
String message = "Could not save category: %1";
std::vector<String> parameters = {"URL key for specified store already exists."};
String result = sprintf(buffer,message.c_str(),parameters[0]);
But, alas, I do not know the size beforehand.
How should I go about doing this? Are there stl functions that could help, should I be using self-written templates(no experience with this), can I convert the std::vector to a va_list, or are there other solutions?
Edit: I didn't notice this asks for a C++ dialect and not Standard C++. Leaving it for now, might be of use for other people.
Nothing standard exists that will do that automatically. That being said, if your interpolation format is just %<number> it might be easy enough to write:
string message = "Could not save category: %1";
std::vector<string> parameters = {"URL key for specified store already exists."};
for (size_t i = 0; i < parameters.size(); ++i) {
const auto interp = "%" + std::to_string(i+1);
const size_t pos = message.find(interp);
if (pos != std::string::npos)
message.replace(pos, interp.size(), parameters[i]);
}
This will put the result in the message string. Of course this implementation is rather limited and not particularly efficient, but again, doing this properly requires a library-sized solution, not SO-answer-sized one.
live version.
I'm have domain class with property that represents files uploaded on my GSP. I've defined that file as byte array (byte [] file). When some specific action happens I'm sending mail with attachments from. This is part of my SendMail service:
int i = 1;
[requestInstance.picture1, requestInstance.picture2, requestInstance.picture3].each(){
if(it.length != 0){
DataSource image = new ByteArrayDataSource(it, "image/jpeg");
helper.addAttachment("image" + i + ".jpg", image);
i++;
}
}
This works fine with image files. But now I want to be able to work with all file types and I'm wondering how to implement this. Also, I want to save real file name in database. All help is welcomed.
You can see where the file name and MIME type are specified in your code. It should be straightforward to save and restore that information from your database along with the attachment data.
If you're trying to figure out from the byte array of data what the MIME type is and what a good filename would be, that's a harder problem. Try this.
I want parse pdf for form field names and types. Is it possible? Because when I tried one PDF, it gave me some strange characters e.g.:
...
?õ»â¢_¸ðO´×¢É]Ì|BQÔQClã(¢dVò¶~?ýg?þª í
pÅ2ÞÎÉÍ??Ú?wȳ.?d;k)*lÙ´¸(ò!ú©=ià??d?éPض2Èåäý?»p?nÜÈûÏ??M
õl:`Þ°Ã3£BíTCy5 ?ð?tN¿7fDõK
±¦?i¹vü~»X?s÷A~Ôê±4?ÕµX±¤?
...
Where could be the problem? I used tool http://support.persits.com/pdf/demo_formfields.asp and pdf https://www.drsr.sk//priznania/dpfoa2010.pdf
I want make some parser for iOS. Thanks for answer.
For PDF parsing on iOS, use the Quartz API.
For an example of an app which makes use of this API, see this reader.
To extract the specific information you're interested in, you will need to read the PDF document structure specification and figure out which dictionaries it's in (or, if you're lucky find some sample code).
Ok, so I looked into reference and found something. I was able to open PDF and make some CGPDFDictionaryRef but I am stuck at that point. This is my code:
CFURLRef pdfURL = CFBundleCopyResourceURL(CFBundleGetMainBundle(), CFSTR("simple_form.pdf"), NULL, NULL);
CGPDFDocumentRef myDocument = CGPDFDocumentCreateWithURL((CFURLRef)pdfURL);
//CFRelease(pdfURL);
int k;
CGPDFPageRef myPage;
NSInteger numOfPages = CGPDFDocumentGetNumberOfPages (myDocument);
for (k = 0; k < numOfPages; k++) {
myPage = CGPDFDocumentGetPage (myDocument, k + 1 );
CGPDFDictionaryRef ref = CGPDFPageGetDictionary(myPage); //what at this point?
CGPDFPageRelease (myPage);
}
I`d like to have something similar to Figure 14-1 here