Using BlobTrackerAuto to track people in computer vision application - opencv

Question

I am currently trying to develop a system that tracks people in a queue using EmguCV (OpenCV Wrapper). I started by running and understanting the VideoSurveilance example that's in Emgu package I downloaded. Here is my code based on the example:
private static void processVideo(string fileName)
{
Capture capture = new Capture(fileName);
MCvFont font = new MCvFont(Emgu.CV.CvEnum.FONT.CV_FONT_HERSHEY_SIMPLEX,
1.0, 1.0);
BlobTrackerAuto<Bgr> tracker = new BlobTrackerAuto<Bgr>();
//I'm using a class that I implemented for foreground segmentation
MyForegroundExtractor fgExtractor = new MyForegroundExtractor();
Image<Bgr, Byte> frame = vVideo.QueryFrame();
fgExtractor.initialize(frame);
while (frame != null)
{
Image<Gray, Byte> foreground = fgExtractor.getForegroundImg(frame);
tracker.Process(frame, foreground);
foreach (MCvBlob blob in tracker)
{
if (isPersonSize(blob))
{
frame.Draw((Rectangle)blob, new Bgr(0, 0, 255), 3);
frame.Draw(blob.ID.ToString(), ref font,
Point.Round(blob.Center), new Bgr(255.0, 255.0, 255.0));
}
}
CvInvoke.cvShowImage("window", frame);
CvInvoke.cvWaitKey(1);
frame = capture.QueryFrame();
}
}
The above code is meant to process each frame of an AVI Video, and show the processed frame with red rectangles around each person in scene. I didn't like the results I was getting using the IBGFGDetector<Bgr> class that is used in VideoSurveilance example, so I am trying to use my own foreground detector, using Emgu's functions such as CvInvoke.cvRunningAvg(), CvInvoke.cvAbsDiff(), CvInvoke.cvThreshold() and cvErode/cvDilate(). I have a few issues:
The video starts with a few people already in the scene. I am not getting the blobs corresponding to the people that are in the scene when the video starts.
Sometimes I "lose" a person for a few frames: I had the red rectangle drawn around a person for several seconds/frames and it disappears and after a while is drawn again with a different ID.
As you can see from the sample code, I check if the blob may be a person checking its height and width (isPersonSize() method), and draw the red rectangle only in the ones that pass in the test. How can I remove the ones that are not person sized?
I want to measure the time a person stays in the scene. What's the best way to know when a blob disappeared? Should I store the IDs of the blobs that I think correspond to people in an array and at each loop check if each one is still there using tracker.GetBlobByID()?
I think I am getting better results if I don't process every frame in the loop. I added a counter variable and an if-statement to process at every 3 frames:
if (i % 3 == 0)
tracker.Process(frame, foreground);
I added the if-statement because the program execution was really slow. But when I did that, I was able to track people that I wasn't able before.
To summarize, I would really appreciate if someone that is more used to OpenCV/EmguCV helped me by saying if it is a good approach to track people using BlobTrackerAuto, and by helping me with the issues above. I get the feeling that I am not taking advantage of the tools EmguCV can provide me.