I am using mpeg-dash mpd file to stream video using videoJS.
I am trying to display thumbnail of the video while using the seek bar.
The adaptation set for image is received on the manifest file. Now I am trying to parse the mpd file and get segments out of it. How can i achieve this using javascript?
I tried parsing the manifest file using https://www.npmjs.com/package/mpd-parser this plugin but this picks up only segments for Audio, video, subtitle and closed caption.
Is there a plugin which handles the same for image adaptation set?
As I think you know, the images are in a separate adaptation set - from the DASH interoperability spec (https://dashif.org/docs/DASH-IF-IOP-v4.3.pdf):
For providing easily accessible thumbnails with timing, Adaptation Sets with the new #con- tentType="image" may be used in the MPD. A typical use case is for enhancing a scrub bar with visual cues. The actual asset referred to is a rectangular tile of temporally equidistant thumb- nails combined into one jpeg or png image. A tile, therefore is very similar to a video segment from MPD timing point of view, but is typically much longer.
and
It is typically expected that the DASH client is able to process such Adaptation Sets by download- ing the images and using browser-based processing to assign the thumbnails to the Media Presen- tation timeline.
It sounds like you want a tool or some code to allow you to be able to view the thumbnails - some players provide this at a user level, e.g. see TheoPlayer info here:
https://www.theoplayer.com/blog/in-stream-thumbnail-support-dvr-dash-streams
You can also leverage and possibly reuse the parsing that is already built into an open source player - see this discussion in the Shaka Player support issues which provides the method to parse and retrieve thumbnails and also the thumbnail format itself:
https://github.com/google/shaka-player/issues/3371#issuecomment-828819282
The above thread contains some example code to extract images also.
Related
I'm using VLC to playback my H.265 videos
Using Cyberlink PowerDirector to re-encode videos to H.265 from original H.264, saves a bunch of space and i'm the only one viewing the content so not an issue for media platforms.
Currently when running the re-encode the video will come out as expected, except the subtitles no longer exist. This software does have the ability to attach subs via SRT or what have you but I need to first extract the existing subs into a text file as the program doesn't allow that (to my knowledge), VLC apparently does?
PowerDirector does indeed have a built in function to extract subtitles from a video which can be accessed via the timeline when right clicking the video and selecting "Extract Subtitles/English" or whatever language.
This imports the existing data into a locally stored SRT directly in the built in subtitles editor page of the program.
I want to read in a video asset on disk and a bunch of processing on it, things like using a CICropFilter on each individual frame and cutting out a mask, splitting up one video into several smaller videos, and removing frames from the original track to "compress" it down and make it more gif-like.
I've come up with a few possible avenues:
AVAssetWriter and AVAssetReader
In this scenario, I would read in the CMSampleBuffers from file, perform my desired manipulations, then write back to a new file using AVAssetWriter.
AVMutableComposition
Here, given a list of CMTimes I can easily cut out frames and rewrite the video or even create multiple compositions for each new video I want to create, then export all of them using AVAssetExportSession.
The metrics I'm concerned about: performance and power. That is to say I'm interested in the method that offers the greatest efficiency in performing my edits while also giving me the flexibility to do what I want. I'd imagine the kind of video editing I'm describing can be done with both approaches but really I want the most performant/with the best capabilities.
In my experience AVAssetExportSession is slightly more performant than using AVAssetReader and AVAssetWriter for a straight forward format A -> format B type conversion, however that said, it's probably not by enough to be too concerned about.
According to Apple's own documentation https://developer.apple.com/library/ios/documentation/AudioVideo/Conceptual/AVFoundationPG/Articles/00_Introduction.html#//apple_ref/doc/uid/TP40010188:
You use an export session to reencode an existing asset into a format
defined by one of a small number of commonly-used presets. If you need
more control over the transformation, in iOS 4.1 and later you can use
an asset reader and asset writer object in tandem to convert an asset
from one representation to another. Using these objects you can, for
example, choose which of the tracks you want to be represented in the
output file, specify your own output format, or modify the asset
during the conversion process.
Given the nature of your question, it seems like you don't have much experience with the AVFoundation framework yet. My advice is to start with AVAssetExportSession and then when you hit a road block, move deeper down the stack into AVAssetReader and AVAssetWriter.
Eventually, depending on how far you take this, you may even want to write your own Custom Compositor.
I would like to know if there is any way to add a watermark to a video which is currently recording and save it with the watermark. (I know about adding watermarks to video files that are already available in app bundle and exporting it with watermark).
iPhone Watermark on recorded Video.
I checked this link. Accepted answer is not good one. The most voted answer is applicable only if there is already a video file in your bundle. (Please read the answer before suggesting that.)
Thanks in advance
For this purpose its better to use GPUImage library (An open source library available in Git hub), It contains so many filter and its possible add overlay using GPUImageOverlayBlendFilter. That contains sample FilterShowCase that explains lot about using the filters. It uses GPU so that it takes the overhead of processing the image. The full credits goes to #Brad Larson the one who created such a great library.
I would like to encode a date/time stamp in each frame of a video in a way that it can be easily read back by a computer. On my system the frame rate is variable, so counting frames does not seem like a good solution. I have it displaying the date and time in human readable form (text) on the frame, but reading that back into the computer doesn't appear to be as trivial as I would like. The recorded videos are large (10s of GB) and long so writing a text file also seems to be troublesome besides being one more file to keep track of. Is there a way to store frame-by-frame information in a video?
There are several ways you can do this.
If your compression is not very strong, you may be able to encode the time-stamp in the top or bottom row of your image. These may not contain too much valuable info. You can add some form of error correction (e.g. CRC) to correct any corruptions done by the compressor.
A more general solution (which I used in the past) is to have the video file, e.g. AVI, contain another separate text stream. Besides AVI most formats support multiple streams, since these are used for stereo-audio streams, subs etc. The drawback here is that there aren't many tools that allow you to write these streams, and you'll have to implement this yourself (using the relevant APIs) for each video format you want to support. In a way this is similar to keeping a text file next to your video, only this file content is multiplexed inside the same video file as a separate stream.
My research has led me to learn that Apple's media element handler is a singleton, meaning I can't have a video playing while an audio is playing in the background. I'm tasked to build a slideshow presentation framework and the client wants a background audio track, timed audio voice-overs which match bullet points, and variable media which can either be an image or video - or a timed cycle of multiple media elements.
Of course, none of the media works on iOS. Each media element cancels out the previous.
My initial thought is to embed the voice-over audio into the video when there's a video present, but there's an existing Flash version of this setup which depends on existing assets so I pretty much have to use what's delivered.
Is there ANY work-around for this? I'm testing on iOS 4.3.5. The smartest devs in the world are on this site - we've got to be able to come up with something.
EDIT: Updated my iPad to iOS 5.0.1 and the issue remains.
How about do it with CSS to do the trick.
Maybe you know about a company called vdopia that distribute video ad on mobile.
http://mobile.vdopia.com/index.php?page=mobilewebsolutions
They claim to developed what so called vdo video format, that actually just to do a css sprite running on that :D
I mean you could have your "video" as a framed image, then attach html5 audio tag on there.
I would like to know your response
Are you working on a Web App or on a Native Application?
If you are working on a Web App you're in a world of hurt. This is because you simply do not have much control over things that Mobile Safari doesn't provide right away.
If this is the case I would come forth and be honest with the stakeholders.
If you are working on a Native Application you can resort to a mechanism that involves some back and forth communication between UIWebView and ObjC. It's actually doable.
The idea is the following:
Insert special <object> elements in your HTML5 documents, that you handcraft yourself according to your needs, taking special care to maintain the attr-* naming convention for non-standard attributes.
Here you could insert IDs, paths and other control variables in the multimedia artifacts that you want to play.
Then you could actually build some javascript (on top of jQuery,p.e.) that communicates with ObjC through the delegation mechanism on the UIWebView or through HTTP. I'll go over this choice down below.
Say that on $(document).ready() you go through all the objects that have a special class. A class that you carefully choose to identify all the special <object>.
You build a list of such objects and pass them on to the ObjC part of your application. You could easily serialize such list using JSON.
Then in ObjC you can do what you want with them. Play them through AVPlayer or some other framework whenever you want them played (again you would resort to a JS - ObjC bridge to actually signal the native part to play a particular element).
You can "communicate" with ObjC through the delegation pattern in UIWebView or through HTTP.
You would then have a JS - ObjC bridge in place.
The HTTP approach makes sense in some cases but it involves a lot of extra code and is resource hungry.
If you are building an ObjC application and want further details on how to actually build an ObjC - JS bridge that fits these needs get back to us :)
I'm halting this post as of now because it would be nice to know if it is in fact a Native App.
Cheers.
This is currently not possible. As you notice when a video plays it takes up the full screen with quicktime and moves the browser to the background. The only solution at this time is to merge the audio and video together into an mp4 format and play that single item.
If I understand you correctly, you are not able to merge the audio and video together because it relies on flash? Since iOS can't play flash you should merge the audio and video together and use flash as a backup. There are numerous html5 players which use javascript to try and play the html5 video first then fallback to flash for backup.
You mention there is an existing Flash setup of the video - is it a an swf file, could you import it into a video/audio editing software and add an audio track on top?
Something like this: http://www.youtube.com/watch?v=J2vvH7oi8m8&feature=youtube_gdata_player
Also, if it is a Flash file, will you be converting it to an avi or like for iOS? If you'd have to do it anyway, there is your chance for adding an audio track.
Could you use a webservice to merge the streams in real time with FFMpeg and then stream one output to quicktime?
To elaborate maybe a library like http://directshownet.sourceforge.net/about.html could also work. It looks like they have method
DESCombine – A class library that uses DirectShow Editing Services to combine video and audio files (or pieces of files) into a single output file. A help file (DESCombine.chm) is provided for using the class.
This could then be used to return the resulting data as the response to the call and loaded via the HTML5 player.