I want to download an AUDIO-ONLY playlist using youtube-dl. I've got the basics down. My version of youtube-dl is up to date, and the command I'm using is:
youtube-dl -x --extract-audio --yes-playlist --playlist-start 1 --playlist-end 18 \
https://www.youtube.com/watch?v=NRHoSXxTpgI&index=1&list=OLAK5uy_lowZyOuAZVs41vEtzV6e0KU8Iue1YQlzg
But it keeps getting stuck on
Deleting original file [filename] (pass -k to keep)
Github doesn't seem to be of any help: https://github.com/rg3/youtube-dl/issues/12570
Any ideas?
The ampersand (&) is a special character in most shells, advising the shell to run the command so far in the background. This should be plainly visible in the first lines after the command execution, which will likely say something like
[1] 17156
[2] 17157
These is your shell telling you the process ID of the new background processes.
You must escape ampersands with a backslash or quotes, like this:
youtube-dl -x --yes-playlist --playlist-start 1 --playlist-end 18 \
'https://www.youtube.com/watch?v=NRHoSXxTpgI&index=1&list=OLAK5uy_lowZyOuAZVs41vEtzV6e0KU8Iue1YQlzg'
--extract-audio is the same as -x, and thus can be deleted.
For more information, see the youtube-dl FAQ.
-x = extract audio only
-i = ignore errors (skip unavailable files)
use only the 'list=...' part, delete the other parameters from copied URL
default is first to last
youtube-dl -ix https://www.youtube.com/watch?list=...
for individual tracks add e.g. --playlist-items 4,7-9
see also: github/ytdl-org: output template examples
Related
I am trying to download videos of a channel which has subtitle. There are more than thousands files, but only a few has subtitle.
youtube-dl --all-subs -ciw -o "./tmp/%(playlist_index)s - %(title)s.%(ext)s" https://www.youtube.com/watch?v=eVW0Xz85qSA&list=PLElG6fwk_0UmBgC02jKJePx
If I can run a command after every url download, it can be good enough. In this case I will check existence of any subtitle file and decide to keep it or remove it.
maybe --exec is good, but it did not work for me as I expected.
This could be probably done in more elegant way.
But it works for me.
First extract urls that have subtitles (replace "playlist_url" with actual url of a playlist, of course)
youtube-dl --write-sub -ij playlist_url | jq -r ".subtitles" \
| grep -Eo "v=[^\&]+" | sort -u > urls.txt \
&& sed -i -e s7^7https:\/\/www\.youtube\.com\/watch\?7 urls.txt
and then download those files with a batch input
youtube-dl -cia urls.txt
* notice that proper playlist_url for channel adress you provided is "https://www.youtube.com/user/SonechkoProject/videos"
Right now i'm using the followin code:
while read num;
do M=$(curl "myurl/$num")
echo "$M"
done < s.txt
where s.txt contains a list (1 per line) of a part of the url.
Is it correct to assume that curl is running sequentially?
Or is it running in thread/jobs/multiple conn at a time?
I've found this online:
parallel -k curl -s "http://example.com/locations/city?limit=100\&offset={}" ::: $(seq 100 100 30000) > out.txt
The problem is that my sequence is coming from a file or from a variable (one element per line) and i can't adapt it to my needs
I've not fully understood how to pass the list to parallel
Should i save all the curl commands in the list and run it with parallel -a ?
Regards,
parallel -j100 -k curl myurl/{} < s.txt
Consider spending an hour walking through man parallel_tutorial. Your command line will love you for it.
I have a external monitor job that I'm pushing the result of another job to it with curl and base on this link :
Monitoring external jobs
After I create the job I just need to run a curl command with the body encoded in HEX to the specified url and then a build will be created and the output will be added to it but what I get instead is part of my output in clear text and the rest in weird characters like so :
Started
Asking akamai to purge this urls:
http://xxx/sites/all/modules/custom/uk.png http://aaaaaasites/all/modules/custom/flags/jp.png
<html><head><title>401 Unauthorized</title> </h�VC��&�G����CV�WF��&��VC�������R&R��BWF��&��VBF�66W72F�B&W6�W&6S�����&�G�����F����F�RW&�F �6�V6�7FGW2�bF�R&WVW7B�2��F�RF��RF�v�B�2��6�Ɩ�r&6�w&�V�B��"F�6�V6�7FGW2�bF�RF�6�W#�v�F��rf�"���F�W&vRF��6O request please keep in mind this is an estimated time
Waiting for another 60 seconds
Asking akamai to purge this urls:
...
..
..
This is how I'm doing it :
export output=`cat msg.out|xxd -c 256 -ps`
curl -k -X POST -d "<run><log encoding=\"hexBinary\">$output</log><result>0</result> <duration>2000</duration></run>" https://$jenkinsuser:$jenkinspass#127.0.0.1/jenkins/job/akamai_purge_results/postBuildResult -H'.crumb:c775f3aa15464563456346e'
If I cat that file is all fine and even if I edit it with vi I can't see any problem with it.
Do you guys have any idea how to fix this ?
Could it be a problem with the hex encoding ? ( I tried hex/enc/dec pages with the result of xxd and they look fine)
Thanks.
I had the same issue, and stumbled across this: http://blog.markfeeney.com/2010/01/hexbinary-encoding.html
From that page, you can get the encoding you need via this command:
echo "Hello world" | hexdump -v -e '1/1 "%02x"'
48656c6c6f20776f726c640a
An excerpt from the explanation:
So what the hell is that? -v means don't suppress any duplicate data
in the output, and -e is the format string. hexdump's very particular
about the formatting of the -e argument; so careful with the quotes.
The 1/1 means for every 1 byte encountered in the input, apply the
following formatting pattern 1 time. Despite this sounding like the
default behaviour in the man page, the 1/1 is not optional. /1 also
works, but the 1/1 is very very slightly more readable, IMO. The
"%02x" is just a standard-issue printf-style format code.
So in your case, you would do this (removing 'export' in favor of inline variable)
OUTPUT=`cat msg.out | hexdump -v -e '1/1 "%02x"'` curl -k -X POST -d "<run><log encoding=\"hexBinary\">$OUTPUT</log><result>0</result> <duration>2000</duration></run>" https://$jenkinsuser:$jenkinspass#127.0.0.1/jenkins/job/akamai_purge_results/postBuildResult -H'.crumb:c775f3aa15464563456346e'
I have installed ffmpeg and mjpeg-streamer. The latter reads a .jpg file from /tmp/stream and outputs it via http onto a website, so I can stream whatever is in that folder through a web browser.
I wrote a bash script that continuously captures a frame from the webcam and puts it in /tmp/stream:
while true
do
ffmpeg -f video4linux2 -i /dev/v4l/by-id/usb-Microsoft_Microsoft_LifeCam_VX-5000-video-index0 -vframes 1 /tmp/stream/pic.jpg
done
This works great, but is very slow (~1 fps). In the hopes of speeding it up, I want to use a single ffmpeg command which continuously updates the .jpg at, let's say 10 fps. What I tried was the following:
ffmpeg -f video4linux2 -r 10 -i /dev/v4l/by-id/usb-Microsoft_Microsoft_LifeCam_VX-5000-video-index0 /tmp/stream/pic.jpg
However this - understandably - results in the error message:
[image2 # 0x1f6c0c0] Could not get frame filename number 2 from pattern '/tmp/stream/pic.jpg'
av_interleaved_write_frame(): Input/output error
...because the output pattern is bad for a continuous stream of images.
Is it possible to stream to just one jpg with ffmpeg?
Thanks...
You can use the -update option:
ffmpeg -y -f v4l2 -i /dev/video0 -update 1 -r 1 output.jpg
From the image2 file muxer documentation:
-update number
If number is nonzero, the filename will always be interpreted as just a
filename, not a pattern, and this file will be continuously overwritten
with new images.
It is possible to achieve what I wanted by using:
./mjpg_streamer -i "input_uvc.so -r 1280×1024 -d /dev/video0 -y" -o "output_http.so -p 8080 -w ./www"
...from within the mjpg_streamer's directory. It will do all the nasty work for you by displaying the stream in the browser when using the address:
http://{IP-OF-THE-SERVER}:8080/
It's also light-weight enough to run on a Raspberry Pi.
Here is a good tutorial for setting it up.
Thanks for the help!
I'm not sure if this is the right place to post this question,I googled a lot about this,but nothing turned up,. for a link of the form
http://www.youtube.com/watch?v=[video_id]
How do i get the link for the data stream?
The following bash script will retrieve youtube streaming url's. I know this is outdated, but maybe this will help someone.
#!/bin/bash
[ -z "$1" ] && printf "usage: `basename $0` <youtube_url>\n" && exit 1
ID="$(echo "$1" | grep -o "v=[^\&]*" | sed 's|v=||')"
URL="$(curl -s "http://www.youtube.com/get_video_info?&video_id=$ID" | sed -e 's|%253A|:|g' -e 's|%252F|/|g' -e 's|%253F|?|g' -e 's|%253D|=|g' -e 's|%2525|%|g' -e 's|%2526|\&|g' -e 's|\%26|\&|g' -e 's|%3D|=|g' -e 's|type=video/[^/]*&sig|signature|g' | grep -o "http://o-o---preferred[^:]*signature=[^\&]*" | head -1)"
[ -z "$URL" ] && printf "Nothing was found...\n" && exit 2
echo "$URL"
Here's a quick lesson in reverse-engineering the YouTube page to extract the stream data.
In the HTML you'll find a <script> tag which defines a variable "swfHTML" - it looks like this: "var swfHTML = (isIE) ? "...
The text in the quotes that follows that snippet is the HTML that displays the Flash object. Note, this text is a set of broken up strings that get concatenated so you'll need to clean it up (i.e. strip instances of '" + "' and and escaping backslashes in order to get the HTML string.)
Once clean you'll need to find the <param> tag with name="flashvars", the value of this tag is an &-delimited URL. Do a split on the & and you'll get your key-value pairs for all the data relating to this video.
The main key you're looking for is "fmt_url_map" and it's an URL Encoded string of Comma-Separated Values starting with "35|" or "34|" or other. (These are defined in another key, "fmt_list" to be files of resolution 854x480 for 35, 640x360 for 34, etc..)
each channel provides rss-data, wich is not updated immediatelly.
Here is a generator for Youtube RSS Files. You should be able to deduce the location of videofiles based on the RSS information. The flv files should be streamable but other formats are also provided.
EDIT:
http://www.referd.info/ is no longer available. It basically was a service where you provided the youtube link and it dereferenced it found all possible downloadsources for that video. I am sure those services are still out there... this one isnt anymore.
You Need Open Link Like This
http://www.youtube.com/get_video_info?&video_id=OjEG808MfF4
And Find Your Stream Your In Return Data