I would like to encode a date/time stamp in each frame of a video in a way that it can be easily read back by a computer. On my system the frame rate is variable, so counting frames does not seem like a good solution. I have it displaying the date and time in human readable form (text) on the frame, but reading that back into the computer doesn't appear to be as trivial as I would like. The recorded videos are large (10s of GB) and long so writing a text file also seems to be troublesome besides being one more file to keep track of. Is there a way to store frame-by-frame information in a video?
There are several ways you can do this.
If your compression is not very strong, you may be able to encode the time-stamp in the top or bottom row of your image. These may not contain too much valuable info. You can add some form of error correction (e.g. CRC) to correct any corruptions done by the compressor.
A more general solution (which I used in the past) is to have the video file, e.g. AVI, contain another separate text stream. Besides AVI most formats support multiple streams, since these are used for stereo-audio streams, subs etc. The drawback here is that there aren't many tools that allow you to write these streams, and you'll have to implement this yourself (using the relevant APIs) for each video format you want to support. In a way this is similar to keeping a text file next to your video, only this file content is multiplexed inside the same video file as a separate stream.
Related
I want to read in a video asset on disk and a bunch of processing on it, things like using a CICropFilter on each individual frame and cutting out a mask, splitting up one video into several smaller videos, and removing frames from the original track to "compress" it down and make it more gif-like.
I've come up with a few possible avenues:
AVAssetWriter and AVAssetReader
In this scenario, I would read in the CMSampleBuffers from file, perform my desired manipulations, then write back to a new file using AVAssetWriter.
AVMutableComposition
Here, given a list of CMTimes I can easily cut out frames and rewrite the video or even create multiple compositions for each new video I want to create, then export all of them using AVAssetExportSession.
The metrics I'm concerned about: performance and power. That is to say I'm interested in the method that offers the greatest efficiency in performing my edits while also giving me the flexibility to do what I want. I'd imagine the kind of video editing I'm describing can be done with both approaches but really I want the most performant/with the best capabilities.
In my experience AVAssetExportSession is slightly more performant than using AVAssetReader and AVAssetWriter for a straight forward format A -> format B type conversion, however that said, it's probably not by enough to be too concerned about.
According to Apple's own documentation https://developer.apple.com/library/ios/documentation/AudioVideo/Conceptual/AVFoundationPG/Articles/00_Introduction.html#//apple_ref/doc/uid/TP40010188:
You use an export session to reencode an existing asset into a format
defined by one of a small number of commonly-used presets. If you need
more control over the transformation, in iOS 4.1 and later you can use
an asset reader and asset writer object in tandem to convert an asset
from one representation to another. Using these objects you can, for
example, choose which of the tracks you want to be represented in the
output file, specify your own output format, or modify the asset
during the conversion process.
Given the nature of your question, it seems like you don't have much experience with the AVFoundation framework yet. My advice is to start with AVAssetExportSession and then when you hit a road block, move deeper down the stack into AVAssetReader and AVAssetWriter.
Eventually, depending on how far you take this, you may even want to write your own Custom Compositor.
It's been a while since I first started looking into this and I still haven't found any feasible solutions, here's to hoping someone might have some suggestions/ideas...
The situation: We currently have a couple of live streams streaming mixed source content (some of the streams are being streamed as file playlists that are modified to change the files in the playlist, while others are streamed as live video directly from input). For each new live stream we usually just end up setting up a new streamer... it's feels rather counterproductive and wasteful.
The question: Does there exist a hardware or software solution (LINUX or Windows) that would allow to live stream multiple, for example, two (independent of each other) file playlists and optionally one or two live A/V inputs, from the same encoder?
According to my findings, with the help of FFMPEG library, it is possible to stream multiple live A/V inputs and even stream file playlists ... but it requires too much hacking to get it working and playlists have to be redone by hand and restarted every time changes have been made. This might work for me personally, but this won't do for a less tech-sawy people...
I'm basically looking for a way to reduce the computer hardware instead of allowing it to exponentially grow with each addition of a new live streaming source/destination.
Thank you for all your input and all the posted solutions. By sheer luck I found the solution I was originally looking for.
For anyone else looking for this or similar solution, the combo of systems that can combat our unusual requirements (and that can be integrated into our existing workflow by adjusting the hardware/software to meet our needs instead of us adjusting to hardware/software requirements/limitations) are: Sorenson Squeeze Server 3.0, MediaExcel Hero Live and MediaExcel File
I have a DirectShow application written in Delphi 6 using the DSPACK component library. I want to be able to mix together audio coming from the output pins from multiple Capture Filters that are set to the exact same media format. Is there an open source or "sdk sample" filter that does this?
I know that intelligent mixing is a big deal and that I'd most likely have to buy a commercial library to do that. But all I need is a DirectShow filter that can accept wave audio input from multiple output pins and does a straight addition of the samples received. I know there are Tee Filter's for splitting a single stream into multiple streams (one-to-many), but I need something that does the opposite (many-to-one), preferably with format checking on each input connection attempt so that any attempt to attach an output pin with a different media format than the ones already added is thwarted with an error. Is there anything out there?
Not sure about anything available out of the box, however it would be definitely a third party component.
The complexity of creating this custom filter is not very high (it is not a rocket science in terms of creating such component yourself for specific need). You basically need to have all input audio converted to the same PCM format, match the timestamps, add the data and then deliver via output pin.
Is there a way to get the note data from a MIDI file? That is, I want to break down the MIDI file into its constituent parts so they are in the form of a unique word (or any other data type).
What I want to do in the end is take in a MIDI file and find patterns in the notes. Get in each note, find it's frequency (of being played) and note how likely other notes are to be played after it.
It would be nice to do this in C/C++, but any language would be fine.
Nik Reisman - sorry, but I don't agree with you...parsing midi in C#, C++ is something about 400 rows of code..It's nothing hard and it is nothing difficult.
I will advise you start with this link: https://web.archive.org/web/20141227205754/http://www.sonicspot.com:80/guide/midifiles.html
There is everything you need to know about midi and how to read it..
In the short description how the parser will work:
1)Open midi in byte mode
2)Read the header chunk where there is info about size, number of tracks and IMPORTANT file format!!
- There are 3 types of formats: 0,1,2 (type 2 is really "valuable", there are only few midi files with this type, so you don't need to read the midi if there is type 2)
- if there is not written: "MThd" (0x4D546864), end with error (it's a bad midi file)
3)Read track chunk
- if there is not written: "MTrk" (0x4D54726B) end with error (it's a bad midi file)
4)Read the midi events..
- There are very many events, you can read them all with if-else commands, or you can read only the events what you want to know, for example NOTE ON, NOTE OFF
- Sometimes in some midi files are not NOTE OFF..this event is changed with NOTE ON and velocity 0
On the websites everything is explained really nicely. If you open the midi file in byte mode you will have only a few methods and everything is then only about if-else commands and there you will catch what is stored right now.
It is important to understand VARIABLE LENGTH, but on the websites it is also explained. It's not hard. You can google many sites where VARIABLE LENGTH is explained too, with some images and examples. So I don't think that it is hard to explain it in here.
If you want a bit more advice, write me, I will try it. But parsing midi is not as hard as how it looks. If you have some problems, write me..
Parsing MIDI files by hand is no fun, take my word on this. ;) The format, although well documented, is difficult to deal with since you are always on the raw byte level. Since you are interested in extracting some meaningful information from MIDI files themselves, I'd recommend using a framework such as Juce, which is written in C++ and has support for reading MIDI files.
Juce is pretty large, but the API is good and well-documented. The class for parsing MIDI files, for instance, is pretty straightforward and easy to use.
There is a number of ready made solutions, taking input from MIDI file, generating musical visualization,
so in theory and practice, MIDI parser works fine
I am working on such musical visualizer in HTML5, generating vertical top > down notes timeline live to support handicapped piano players.
DLP projector is great but it seems, I need to install large LCD TV screen, just over the piano keyboard to get visualization to match the notes played
#Brendan Kavanagh is the leader
another key developer is called Stephen Malinowski
just follow my question to get the right web links
How to build MIDI file visualizer to get input from MIDI file and display MIDI timeline over the keyboard to match notes played by a real player
Quicktime has a rich metadata API, allowing one to store all sorts of arbitrary data on a .mov file (or one of its streams). I'm looking for the standard key name and value format for storing the shooting date for a video clip, analogous to EXIF's DateTimeOriginal.
The following discussion at the apple site makes it seem like there may not be one defined by apple, as they don't seem to feel it's very important.
http://discussions.apple.com/message.jspa?messageID=6267622
This is related to How can I get the original capture timestamp from my home movie files:: AVI and MPG4? (which deals with .mp4 and .avi)
I'm afraid there is no standard key for this kind of metadata.
You might try to use a reasonably fitting standard key like
kQTMetaDataCommonKeyInformation,
kQTMetaDataCommonKeyDescription or
kQTMetaDataCommonKeyProducer
although this wouldn't be 'standard' (i.e. would most likely only be processed correctly by your application).
On the question which value format to use this sample code and Q&A article (although it is not exactly fitting) might set you on the right track:
http://developer.apple.com/qa/qa2007/qa1515.html
http://developer.apple.com/samplecode/QTMetaData/listing1.html