I'm working on a system to convert a bunch of .mov files to H.264 (using HandBrakeCLI) and webm (using ffmpeg) as the .mov files are created. In general, things are going very well. I'm hung up on a bit of error detection. I want to know if one of the encodings failed so that we can investigate, try again, etc.
To test encoding failure, I copied a text file into a file with a .mov extention, and set the programs about trying to encode it. Naturally, they both fail to encode the file (I'm not sure what "success" would mean in this context...) However, while ffmpeg reports this failure by setting its exit code to 1, HandBrakeCLI sets the exit code to 0, because it exited cleanly. This is consistent with the HandBrakeCLI documentation but it leaves me wondering how I can tell if HandBrakeCLI knows if it failed to encode anything. That same documentation page suggests "If you want to monitor HandBrake's process, you should monitor the standard pipes", so I'm now getting the effect that I want by doing something like this:
HandBrakeCLI --preset 'Normal' --input bad.mov --output out.mv4 2>&1 | grep 'Encode done'
grep then sets its exit code to 0 if it found a match, and 1 if it didn't. But, this seems rather barbaric: for instance, the text "Encode done!" could change in a future release of HandBrake.
So, anyone have a better way to tell if HandBrake encoded something or not?
Some edited shell output is included below for reference...
$ ffmpeg -i 'develqueuedir/B_BH_120409.mov' 'develqueuedir/B_BH_120409.webm'
FFmpeg version 0.6.4-4:0.6.4-0ubuntu0.11.04.1, Copyright (c) 2000-2010 the Libav Developers
[snip]
develqueuedir/B_BH_120409.mov: Invalid data found when processing input
$ echo $?
1
$ HandBrakeCLI --preset 'Normal' --maxWidth 720 --optimize --input 'develqueuedir/B_BH_120409.mov' --output 'develqueuedir/B_BH_120409.mv4'
Output format couldn't be guessed from file name, using default.
[11:45:45] hb_init: starting libhb thread
HandBrake 0.9.6 (2012022900) - Linux x86_64 - http://handbrake.fr
Opening develqueuedir/B_BH_120409.mov...
[snip]
[11:45:45] libhb: scan thread found 0 valid title(s)
No title found.
HandBrake has exited.
$ echo $?
0
Short answer is no , you can find detailed explanation at HandBrake forum https://forum.handbrake.fr/viewtopic.php?f=12&t=18559&p=85529&hilit=return+code#p85529
adddition:
I think that there is a patch from user fonkprop that is rejected by developers , if you really need it contact that guy
Good news! It appears that this feature is about to be implemented in HandBrake-CLI 0.10. As you can read on the roadmap for the 0.10 milestone:
Basic support for return codes from the CLI. (0 = No Error, 1 = Cancelled, 2 = Invalid Input, 3 = Initialization error, 4 = Unknown Error")
Related
With a snippet like
# Contents of ./run
my $p = Proc::Async.new: #*ARGS;
react {
whenever Promise.in: 5 { $p.kill }
whenever $p.stdout { say "OUT: { .chomp }" }
whenever $p.ready { say "PID: $_" }
whenever $p.start { say "Done" }
}
executed like
./run raku -e 'react whenever Supply.interval: 1 { .say }'
I expected to see something like
PID: 1234
OUT: 0
OUT: 1
OUT: 2
OUT: 3
OUT: 4
Done
but instead I see
PID: 1234
OUT: 0
Done
I understand that this has to do with buffering: if I change that command into something like
# The $|++ disables buffering
./run perl -E '$|++; while(1) { state $i; say $i++; sleep 1 }'
I get the desired output.
I know that TTY IO::Handle objects are unbuffered, and that in this case the $*OUT of the spawned process is not one. And I've read that IO::Pipe objects are buffered "so that a write without a read doesn't immediately block" (although I cannot say I entirely understand what this means).
But no matter what I've tried, I cannot get the unbuffered output stream of a Proc::Async. How do I do this?
I've tried binding an open IO::Handle using $proc.bind-stdout but I still get the same issue.
Note that doing something like $proc.bind-stdout: $*OUT does work, in the sense that the Proc::Async object no longer buffers, but it's also not a solution to my problem, because I cannot tap into the output before it goes out. It does suggest to me that if I can bind the Proc::Async to an unbuffered handle, it should do the right thing. But I haven't been able to get that to work either.
For clarification: as suggested with the Perl example, I know I can fix this by disabling the buffering on the command I'll be passing as input, but I'm looking for a way to do this from the side that creates the Proc::Async object.
You can set the .out-buffer of a handle (such as $*OUT or $*ERR) to 0:
$ ./run raku -e '$*OUT.out-buffer = 0; react whenever Supply.interval: 1 { .say }'
PID: 11340
OUT: 0
OUT: 1
OUT: 2
OUT: 3
OUT: 4
Done
Proc::Async itself isn't performing buffering on the received data. However, spawned processes may do their own depending on what they are outputting to, and that's what is being observed here.
Many programs make decisions about their output buffering (among other things, such as whether to emit color codes) based on whether the output handle is attached to a TTY (a terminal). The assumption is that a TTY means a human is going to be watching the output, and thus latency is preferable to throughput, so buffering is disabled (or restricted to line buffering). If, on the other hand, the output is going to a pipe or a file, then the assumption is that latency is not so important, and buffering is used to achieve a significant throughput win (a lot less system calls to write data).
When we spawn something with Proc::Async, the standard output of the spawned process is bound to a pipe - which is not a TTY. Thus the invoked program may use this to decide to apply output buffering.
If you're willing to have another dependency, then you can invoke the program via. something that fakes up a TTY, such as unbuffer (part of the expect package, it seems). Here's an example of a program that is suffering from buffering:
my $proc = Proc::Async.new: 'raku', '-e',
'react whenever Supply.interval(1) { .say }';
react whenever $proc.stdout {
.print
}
We only see a 0 and then have to wait a long time for more output. Running it via unbuffer:
my $proc = Proc::Async.new: 'unbuffer', 'raku', '-e',
'react whenever Supply.interval(1) { .say }';
react whenever $proc.stdout {
.print
}
Means that we see a number output every second.
Could Raku provide a built-in solution to this some day? Yes - by doing the "magic" that unbuffer itself does (I presume allocating a pty - kind of a fake TTY). This isn't trivial - although it is being explored by the libuv developers; at least so far as Rakudo on MoarVM goes, the moment there's a libuv release available offering such a feature, we'll work on exposing it.
I am attempting to send a file from IFS to an outq on our AS/400 system. Whenever I do, I get exactly what I send, as well as a line of "#" symbols of varying lengths appended to the end.
Here's the command I'm using:
qsh cmd('cat -c /path/test.txt | Rfile -wbQ -c "ovrprtf file(qprint)
outq(*LIBL/ABCD) devtype(*USERASCII) rplunprt(*no) splfname(test) hold(*no)"
qprint')
The contents of test.txt is just Hello World!
The output I get when I send the command is
Hello World!####################################################################
I have not found any posts online about a similar problem, and have tried changing values and looking for additional switches to get it to work. Nothing I'm doing seems to fix the issue.
Is there a command or switch that I am missing, or is something I have in there already causing this?
EDIT:
I found this documentation which is the first time I've seen this issue mentioned, but it's not very helpful:
“Messages for a Take Action command might consist of a long string of "at" symbols (#) in a pop-up message. (The Reflex automation Take Action command, which is configured in situations, does not have this problem.) A resolution for this problem is under construction. This problem might be resolved by the time of the product release. If you see this problem, contact IBM Software Support.”
The only differences are: 1) this is not a pop-up message, it's printed. 2) I don't believe we use Tivoli Monitoring, although I could be wrong.
Assuming we do use Tivoli Monitoring, what would the solution be? There's no additional documentation past that, and I am not a system administrator, so I can't really make the call to IBM Software Support myself. And assuming we DON'T use it, what else could cause this issue?
I get different results, yet similar. I created a test.txt with Windows Explorer, put in Hello, world!, saved it and tried the script. I got gibberish for the 'Hello, world!' and then the line of # symbols.
My system is 7.3 TR5, CCSID 37 (US English) and my IFS file is CCSID 1252 (Windows English). Results did not change if I used a stream file of CCSID 819 (US ASCII).
I didn't have any luck modifying Rfile switches.
I found that removing devtype(*userascii) produced printed output in plain English without the # symbols. Do you really need *USERASCII? I would think that would be more for a pre-formatted 'print-ready' file like Postscript or the like.
EDIT: some more things to try
I don't understand why *USERASCII is adding those # symbols; it looks like a translation issue.
I tried this and still got the extra ###... You might have to play with the TOCCSID() parameter. Although a failure, it did give me an idea: what if those # symbols are EBCDIC spaces being sent as-is to the *USERASCII print stream? All we'd need is a way to send only the number of bytes in the stream file, without any padding.
CRTPF FILE(QTEMP/PRTSTMF) RCDLEN(132)
CPY OBJ('/path/test.txt') TOOBJ('/qsys.lib/qtemp.lib/prtstmf.file/prtstmf.mbr') replace(*yes)
ovrprtf file(qprint) outq(*LIBL/prt3812) devtype(*USERASCII) rplunprt(*no) splfname(test) hold(*no)
cpyf prtstmf qprint
The data in QTEMP/PRTSTMF is in ASCII; DSPPFM shows that much. It also shows a bunch of spaces: after all, it is a fixed length file. My next step was to write an RPG program to read the stream file and print it, but Scott Klement already did that: http://www.scottklement.com/PrtStmf.zip
This works on my system:
ovrprtf file(qsysprt) outq(*LIBL/abcd) devtype(*USERASCII) rplunprt(*no) splfname(test) hold(*no)
prtstmf stmf('/path/test.txt') outq(abcd)
I have the following piece of code successfully recognizing short (less than 1 min) test audio file, but failing with recognition another long audiofile (1.5h).
from google.cloud import speech
def run_quickstart():
speech_client = speech.Client()
sample = speech_client.sample(source_uri="gs://linear-arena-2109/zoom0070.flac", encoding=speech.Encoding.FLAC)
alternatives = sample.recognize('uk-UA')
for alternative in alternatives:
print(u'Transcript: {}'.format(alternative.transcript))
with open("Output.txt", "w") as text_file:
for alternative in alternatives:
text_file.write(alternative.transcript.encode('utf8'))
if __name__ == '__main__':
run_quickstart()
Both files are uploaded to Google Cloud.
The first one:
https://storage.googleapis.com/linear-arena-2109/sample.flac
The second one:
https://storage.googleapis.com/linear-arena-2109/zoom0070.flac
Both were converted from mp3 with ffmpeg utility:
ffmpeg -i sample.mp3 -ac 1 sample.flac
ffmpeg -i zoom0070.mp3 -ac 1 zoom0070.flac
First file was successfully recognized, but second file outputs the following error:
google.gax.errors.RetryError: GaxError(Exception occurred in retry method that was not classified as transient, caused by <_Rendezvous of RPC that terminated with (StatusCode.INVALID_ARGUMENT, Sync input too long. For audio longer than 1 min use LongRunningRecognize with a 'uri' parameter.)>)
But I have already used uri parameter in my python script. What is wrong?
update
#NieDzejkob helped to understand the error. So, method long_running_recognize should be used instead of recognize. The comprehensive long_running_recognize usage example can be found on the corresponding document page
For any audio file longer than 1 minute, you need to use Asynchronous Speech Recognition and the file has to be uploaded to Google Cloud Storage so that you can pass in a gcs_uri.
In addition, you will need to use the .long_running_recognize method in your script. An example from GCP documentation can be found here.
I realize that OP figured it out but thought it would be useful to provide an answer and generalize it a bit.
I have a school project using Opencv, a Raspberry pi 2, a Raspicam and C++. Our project needs to use a video stream from the camera to detect objects.
That's why I assume to use pipe and fifo by this line :
mkfifo fifo;
raspivid -t 0 -o fifo & ./Detection fifo
As a result I get :
[mp3 # 0x1768ac0] Header missing
But when a well saved video is sent to the program, our project perfectly runs.
Example of good behavioured instructions :
raspivid -t 10000 -o video.h264 ; ./Detection video.h264
Do someone have an idea ?
All result I found where not suitable for our project. I maybe miss some important information.
Thank you
PS : Hope I am understandable, my english is not that good
I'm currently working with tshark using python subprocess and I want to handle errors in the child process currently.
I can get the sub-process exit code via python's subproccess 'returncode' field, but I can't fined any documentation of tshark exit codes.
I already figured out that 0 means success (no surprise) and 2 means corrupt or unsupported file.
But where there is 0 and 2 there is usually a 1, and there should be more codes.
Does anyone know were I can fined a list of thsark's possible error codes and causes?
Ages ago, I seem to remember somebody suggesting an exit status of 1 for command-line syntax errors and 2 for other errors, and I've followed that convention for a lot of code I've written.
So 1 would be returned if you gave an invalid command-line flag or an invalid capture or display filter or something such as that, and 2 would be returned if it couldn't open a capture device or capture file.