background extraction and update from video using matlab - image-processing

Question

I have a video about traffic scene. Now, I want to calculate the percentage of vehicle area on the road area (or the percentage of foreground area on the background area). The first step for this is background extraction. I have read many document and scientific articles about it, one of them recommends to use the mean filter following this formula:
This is the link of that the article. The results are very good, it is exactly what I want.
I followed his formula and I tried to write my code. But It didn't work! Who can help me and give me some advice.
This is my code:
clc; % Clear the command window.
close all; % Close all figures (except those of imtool.)
imtool close all; % Close all imtool figures.
clear; % Erase all existing variables.
workspace; % Make sure the workspace panel is showing.
fontSize = 14;
%input video;
step = 10;
vob = VideoReader('NKKN.avi');
frame = vob.read(inf);
vidHeight = vob.Height;
vidWidth = vob.Width;
nFrames = vob.NumberOfFrames;
%%% First-iteration background frame
background_frame = double(frame*0);
redbackground_frame = background_frame(:,:,1);
greenbackground_frame = background_frame(:,:,2);
bluebackground_frame = background_frame(:,:,3);
%calculate background
i = 0;
for k = 1:10 %get background from 10 frame (J=10)
thisframe = double(read(vob, k));
%background_frame = background_frame + thisframe;
redbackground_frame = redbackground_frame + thisframe(:,:,1);
greenbackground_frame = greenbackground_frame + thisframe(:,:,2);
bluebackground_frame = bluebackground_frame + thisframe(:,:,3);
i=i+1;
disp(i);
end
A = redbackground_frame/i;
B = greenbackground_frame/i;
C = bluebackground_frame/i;
background = cat(3,A,B,C);
imshow(background);

Answer 1

You can maintain a buffer of B frames for a dynamic estimation of backgound
buff = NaN( [vidHeight, vidWidth, 3, B] ); % allocate room for buffer
% process the video
for fi = 1:nFrames
% read current frame
thisframe = double(read(vob, k)) / 255; % convert to [0..1] range
% update background model
buff(:, :, :, mod( fi, B ) + 1 ) = thisframe;
background_L1 = nanmedian( buff, 4 ); % I think this is better than `mean` - try it!
background_L2 = nanmean( buff, 4 );
% do whatever processing you need with fi-th frame
% and the current background mode...
% ...
end
Note that if fi < B (i.e., you processed less than B frames) the background model is not stable. I am using NaNs as default values for the buffer and these values are ignored when backgound model is estimated -- this is the reason why I use nanmedian and nanmean instead of simply median and mean.

Answer 2

vob = VideoReader('NKKN.avi');
frame = vob.read(inf);
vidHeight = vob.Height;
vidWidth = vob.Width;
nFrames = vob.NumberOfFrames;
%allocate room for buffer of 20 frames
buff = NaN( [vidHeight, vidWidth, 3, 20] ); % allocate room for buffer
for fi = 1:20:nFrames
disp(fi);
% read current frame
thisframe = double(read(vob, fi)) / 255; % convert to [0..1] range
% update background model
buff(:, :, :, mod( fi, 10 ) + 1 ) = thisframe;
background_L1 = nanmedian( buff, 4 );
background_L2 = nanmean( buff, 4 );
hImage = subplot(2, 2, 1);
image(thisframe);
caption = sprintf('thisframe');
title(caption, 'FontSize', fontSize);
drawnow; % Force it to refresh the window.
subplot(2,2,2);
imshow(background_L2);
title('background-L2');
subplot(2,2,3);
imshow(background_L1);
title('background-L1');
end

Answer 3

Extracting the background of this video
https://www.youtube.com/watch?v=URJxS1giCA4&ab_channel=AliShahzil
clear all
close all
reader = VideoReader('C:\Users\Ali Sahzil\Desktop\Media.wmv'); // your video file location
vid = {};
while hasFrame(reader)
vid{end+1} = im2single(readFrame(reader));
end
bg = mean( cat(4, vid{:}), 4);
x =bg;
imshow( bg );

Answer 4

Here is a very simple solution you can build upon. First you will need a sample background image of the scene with no traffic. We will call this 'bg'.
Here is a simple approach in pseudo-code:
load in background image 'bg'
set threshold upper value
set threshold lower value
loop until done for each frame
{
subtract 'bg' image from your first frame
if pixel value of foreground > than threshold upper value
{
set foreground pixel value to 'nan'
}
if pixel value of foreground < than threshold lower value
{
set foreground pixel value to 'nan'
}
if pixel value of foreground == 0
{
set foreground pixel value to 'nan'
}
}
This will bracket your foreground images to only show the parts of the scene you are interested in. Note: this process can be greatly enhanced by using a stereoscopic camera to give you depth perception. However, you should be able to build upon this code to remove unwanted parts of your image.

Answer 5

This is actually based on Shai's and user3725204's answers. I didn't use read and NumberOfFrames which are not recommended. I also adopted user3725204's suggestion, since there's no need to read all frames.
function backGrnd = getBackGrnd(filename, nTest, method)
tic
if nargin < 2, nTest = 20; end
if nargin < 3, method = 'median'; end
v = VideoReader(filename);
nChannel = size(readFrame(v), 3);
tTest = linspace(0, v.Duration-1/v.FrameRate , nTest);
%allocate room for buffer
buff = NaN([v.Height, v.Width, nChannel, nTest]);
for fi = 1:nTest
v.CurrentTime =tTest(fi);
% read current frame and update model
buff(:, :, :, mod(fi, nTest) + 1) = readFrame(v);
end
switch lower(method)
case 'median'
backGrnd = nanmedian(buff, 4);
case 'mean'
backGrnd = nanmean(buff, 4);
end
toc
end
And the result is like this:
subplot(221); imshow(uint8(TrafficVisionLab.getBackGrnd('traffic.avi', 10, 'mean')));
subplot(222); imshow(uint8(TrafficVisionLab.getBackGrnd('traffic.avi', 10, 'median')));
subplot(223); imshow(uint8(TrafficVisionLab.getBackGrnd('traffic.avi', 50, 'mean')));
subplot(224); imshow(uint8(TrafficVisionLab.getBackGrnd('traffic.avi', 50, 'median')));