Determine consecutive video clips - image-processing

I a long video stream, but unfortunately, it's in the form of 1000 15-second long randomly-named clips. I'd like to reconstruct the original video based on some measure of "similarity" of two such 15s clips, something answering the question of "the activity in clip 2 seems like an extension of clip 1". There are small gaps between clips --- a few hundred milliseconds or so each. I can also manually fix up the results if they're sufficiently good, so results needn't be perfect.

A very simplistic approach can be:
(a) Create an automated process to extract the first and last frame of each video-clip in a known image format (e.g. JPG) and name them according to video-clip names, e.g. if you have the video clips:
clipA.avi, clipB.avi, clipC.avi
you may create the following frame-images:
clipA_first.jpg, clipA_last.jpg, clipB_first.jpg, clipB_last.jpg, clipC_first.jpg, clipC_last.jpg
(b) The sorting "algorithm":
1. Create a 'Clips' list of Clip-Records containing each:
(a) clip-name (string)
(b) prev-clip-name (string)
(c) prev-clip-diff (float)
(d) next-clip-name (string)
(e) next-clip-diff (float)
2. Apply the following processing:
for Each ClipX having ClipX.next-clip-name == "" do:
{
ClipX.next-clip-diff = <a big enough number>;
for Each ClipY having ClipY.prev-clip-name == "" do:
{
float ImageDif = ImageDif(ClipX.last-frame.jpg, ClipY.first_frame.jpg);
if (ImageDif < ClipX.next-clip-diff)
{
ClipX.next-clip-name = ClipY.clip-name;
ClipX.next-clip-diff = ImageDif;
}
}
Clips[ClipX.next-clip-name].prev-clip-name = ClipX.clip-name;
Clips[ClipX.next-clip-name].prev-clip-diff = ClipX.next-clip-diff;
}
3. Scan the Clips list to find the record(s) with no <prev-clip-name> or
(if all records have a <prev-clip-name> find the record with the max <prev-clip-dif>.
This is a good candidate(s) to be the first clip in sequence.
4. Begin from the clip(s) found in step (3) and rename the clip-files by adding
a 5 digits number (00001, 00002, etc) at the beginning of its filename and going
from aClip to aClip.next-clip-name and removing the clip from the list.
5. Repeat steps 3,4 until there are no clips in the list.
6. Voila! You have your sorted clips list in the form of sorted video filenames!
...or you may end up with more than one sorted lists (if you have enough
'time-gap' between your video clips).
Very simplistic... but I think it can be effective...
PS1: Regarding the ImageDif() function: You can create a new DifImage, which is the difference of Images ClipX.last-frame.jpg, ClipY.first_frame.jpg and then then sum all pixels of DifImage to a single floating point ImageDif value. You can also optimize the process to abort the difference (or sum process) if your sum is bigger than some limit: You are actually interested in small differences. A ImageDif value which is larger than an (experimental) limit, means that the 2 images differs so much that the 2 clips cannot be one next each other.
PS2: The sorting algorithm order of complexity must be approximately O(n*log(n)), therefore for 1000 video clips it will perform about 3000 image comparisons (or a little more if you optimize the algorithm and you allow it to not find a match for some clips)

Related

Why is this basic MQL4-code taking so much time to load up on my MT4?

I am learning MQL4 language and am using this Code to plot a Simple moving Average, the Code works fine, but when I load it up on my MT4 it takes a lot of time, am I missing something ?
int start() // Special function start()
{
int i, // Bar index
n, // Formal parameter
Counted_bars; // Number of counted bars
// Sum of Low values for period
// --------------------------------------------------------------------
Counted_bars=IndicatorCounted(); // Number of counted bars
i=Bars-Counted_bars-1; // Index of the first uncounted
while(i>=0) // Loop for uncounted bars
{
Buf_0[i]=(iMA(Symbol(),PERIOD_M5,200,i,MODE_EMA,PRICE_HIGH,0);
i--; // Calculating index of the next bar
}
// --------------------------------------------------------------------
return; // Exit the special funct. start()
}
// --------------------------------------------------------------------
Q : am I missing something?
No, this is a standard feature to process all the Bars back, towards the earliest parts of the history.
If your intentions require a minimum setup-time, it is possible to "shorten" the re-painted part of the history to just, say, last week, not going all the way back all the Bars-number of Bars a few years back, as all that data have been stored in the OHLCV-history database.
That "shortened"-part of the history will this way become as long as your needs can handle and not a single bar "longer".
Hooray, The Problem was solved.
BONUS PART :
Given your code instructs to work with EMA, not SMA, there is one more vector of attack onto the shortest possible time.
For EMA, any next Bar value will become a product of alfa * High[next] added to a previously known as EMA[next+1]
Where a constant alfa = 2. / ( N_period + 1 ) is known and constant across the whole run of the history processed.
This approach helped me gain about ~20-30 [us] FASTER processing for a 20-cell Price-vector, when using this algorithmic shortcut on an array of float32-values compared to cell-by-cell processing. Be sure to benchmark the code for your use-case and may polish further tricks with using different call-signatures of iHigh() instead of accessing an array of High[]-s for any potential speedups, if in utmost need to shave-off any further [us] possible.

Lua random image

I have a Lua script I am using in a tabletop game and basically you have a "token" that represents a creature. When it dies, it overlays an image (which I have indicated in an .xml script) with an image of like a blood splat, or tombstone etc.
How do I make it so it would randomize which image gets overlayed?
The Script is here.
The lines below (178-184) are the main section that tells it "put image X over the token". I want it to randomize between say, 5 different images..
if not widgetDeathIndicator then
widgetDeathIndicator = tokenCT.addBitmapWidget("token_dead");
widgetDeathIndicator.setBitmap("token_dead");
widgetDeathIndicator.setName("deathindicator");
widgetDeathIndicator.setTooltipText(sName .. " has fallen, as if dead.");
widgetDeathIndicator.setSize(nWidth-20, nHeight-20);
end
token_dead is the name of the current image being used, which in the .xml directs to a .png
Yes, you can use math.random for this.
local images = {
'token_dead',
'another_image_name',
'yet_another_image_name',
}
local image = images[math.random(#images)]
math.random(n) will return a pseudo-random integer between 1 and n, so if you pass in #images (the length of the images table) you will get a valid pseudo-random table index for images.
To get better randomness you should set math.randomseed before you call math.random. (If you don't set it, then math.random will return the same sequence of "random" numbers each time.)

Simple Babymonitor with Bass.DLL

I am trying to program a simple Babymonitor for Windows (personal use).
The babymonitor should just detect the dB level of the microphone and triggers at a certain volume.
After some research, I found the Bass.dll library and came across it's function BASS_ChannelGetLevel, which is great but seems to have limitations and doesn't fit my needs (Peak equals to a DWORD value).
In the examples I found a livespec example which is "almost" what I need. The example uses BASS_ChannelGetData, but I don't quite know how to handle the returned array...
I want to keep it as simple as possible: Detect the volume from the microphone as dB or any other value (e.g. value 0-MAXINT).
How can this be done with the Bass.dll library?
The BASS_ChannelGetLevel returns the value that is capped to 0dB (return value is 32768 in this case). If you adjust your source level (lower microphone level in sound card settings) then it will work just fine.
Another way, if you want to get uncapped value is to use the BASS_ChannelGetLevelEx function instead: it returns floating point levels, where 1 is maximum (0dB) value that corresponds to BASS_ChannelGetLevel's 32767, but it can exceed 1 to detect sound levels above 0dB which is what you may need.
I also suggest you to monitor sound level for a while: trigger only if certain level exists for 2-3 seconds at least (this way you will exclude false alarms).
Here is how you obtain the db level given an input stream handle (streamHandle):
var peak = (double)Bass.BASS_ChannelGetLevel(streamHandle);
var decibels = 20 * Math.Log10(peak / Int32.MaxValue);
Alternatively, you can use the following to get the RMS (average) peak. To get the RMS value, you have to pass in a sample length into BASS_ChannelGetLevel. I'm using 20 milliseconds here but you can play with the value to see which works best for your needs.
var decibels = 0m;
var channelCount = 2; //Assuming two channels
var sampleLengthMS = 20f;
var rmsLevels = new float[channelCount];
var rmsObtained = Bass.BASS_ChannelGetLevel(streamHandle, rmsLevels, sampleLengthMS / 1000f, BASSLevel.BASS_LEVEL_RMS);
if (rmsObtained)
decibels = 20*Math.Log10(rmsLevels[0]); //using first channel (index 0) but you can get both if needed.
else
Console.WriteLine(Bass.BASS_ErrorGetCode());
Hope this helps.

seed random numbers Objective-C

I'm working on a game in Xcode 6 and need to generate a new random number each time 2 specific objects touch each other. I have tried using srand() at the start of my application but it seems that the values remain the same as if it isn't seeding a new value each time the objects collide.
here is the code
if((CGRectIntersectsRect(Stickman.frame, Box1.frame))) {
xRan = arc4random()%11;
if(xRan<=3){
Spike1 = true;
[self SpikeCall];
}
//Gold
if (xRan==10) {
G1 = true;
}
Box1.center = CGPointMake(0,278);
Box1SideMovement = 5;
}
The problem is that after the Stickman hits the Box1 when it comes back on screen it still holds the same value in xRan except for certain scenarios where it will between 1-3 then it makes Spike1 true. I'd like it to be so that each time the object Box1 intersects with Stickman the xRan seeds a new number between 1-10 so that there is a 1 in 10 chance of G1 becoming true & if xRan is 1-3 it will make Spike1 true.
This is more of a comment than an answer, but it's too long for a comment.
There are a couple of problems with your approach here. First, srand does not seed the arc4random function. It seeds the rand function, which is a different pseudo-random number generator with somewhat worse properties than arc4random. There is no explicit seeding function for arc4random. Second, if you want a random number between 1 and 10 you should not use the % 11 approach. That gives you a random number between 0 and 10 (and I think you don't want zero), but also it probably does not give you a uniform distribution. Even if arc4random is good at providing a uniform distribution in its whole range it may not provide a uniform distribution of the least significant bits.
I suggest you use something like:
arc4random_uniform(10) + 1
arc4random_uniform(10) will return a number between 0 and 9, and will do a good job of providing a uniform distribution in that range. The +1 just shifts your interval so you get numbers between 1 and 10 instead of between 0 and 9.

Xcode iOS: how to find position of one audio file (snippet) inside another?

I have audio files, with different durations. They have common content and unique content. E.g. two files, 70 seconds each, last 10 seconds of the first file is the same as first two seconds of the second file. How can I find the exact position of common content (e.g. 60.0 of the first file)?
Sounds a little bit messy, hope the following image can help https://drive.google.com/file/d/0BzBE2Kfw8uQoUWNTN1RXOEtLVEk/view?usp=sharing
So, I'm looking for the red mark - common content starts at 60.0 sec of the first file.
The problem is that I have files with different durations. Sometimes it's 70 seconds long, sometimes one file is 70 seconds, the other is 80 seconds long, etc. Most likely they have 60.0 seconds of unique content, but I'm not sure (it could be 59.9 of unique content, etc.).
Thus, I assume I need to get a short snippet of the second file from first 10 seconds and find it in the first file:
For example, output: 2.5 sec of the second file = 62.5 from the first file - works for me, as well.
THE MAIN GOAL IS TO PLAY FILE AFTER FILE GAPLESS. If I get the values, I'll be able to do this. Sometimes the values can be: 2.5 = 63.7, that's why I need the exact match.
Can anybody help with the code or at least some information of how to compare two snippets of audio content? Thanks in advance!
Wow, that is quite a problem to solve. And I must confess that i've not done anything exactly like this or have any code based suggestions.
All I will say is that if I were looking to try and solve this problem, then I would try and save the audio file as some kind of uncompressed and fixed size (as in a known number of bytes per second) format.
Then you could take a section of one file and byte match it with another, then you would know how many bytes inwards that snippet occurred. Then, knowing the bytes per ms (sort of frame size), you could work out the exact time position.
It's a bit hair brained, but i've used that technique with images before but at least audio is linear!
Here is an approximate example of how I would go about doing the comparison of a sample within a sound file.
- (int)positionOf:(NSData*)sample inData:(NSData*)soundfile {
// the block size has to be big enough to find something genuinely unique but small enough to ensure it is still fast.
int blockSize = 128;
int position = 0;
int returnPosition = INT32_MAX;
// check to see if the block size exceeds the sample or data file size
if (soundfile.length < blockSize || sample.length < blockSize) {
return returnPosition;
}
// create a byte array of the sample, ready to use to compare with the shifting buffer
char* sampleByteArray = malloc(sample.length);
memcpy(sampleByteArray, sample.bytes, sample.length);
// now loop through the sound file, shifting the window along.
while (position < (soundfile.length - blockSize)) {
char* window = malloc(blockSize);
memcpy(window, soundfile.bytes + position, blockSize);
// check to see if this is a match
if(!memcmp(sampleByteArray, window, blockSize)) {
// these are the same, now to check if the whole sample is the same
if ((position + sample.length) > soundfile.length) {
// the sample won't fit in the remaining soundfile, so it can't be this!
free(window);
break;
}
if(!memcmp(sampleByteArray, soundfile.bytes + position, sample.length)) {
// this is an entire match, position marks the start in bytes of the sample.
free(window);
returnPosition = position;
break;
}
}
free(window);
position++;
}
free(sampleByteArray);
return returnPosition;
}
It compiles, didn't have time to setup the scenario to check your exact case, but i'm quite confident this may help.

Resources