After I downsample a vector with a constant decimating factor, I want to upsample the vector back to the original sample rate (after performing some analyses). However, I am struggling with the upsampling.
For the downsampling I apply vDSP_desamp from the Accelerate framework, and for the upsampling I tried to apply vDSP_vlint:
// Create some test data for input vector
float inputData[10] = {0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9};
int inputLength = 10;
int decimationFactor = 2; // Downsample by factor 2
int downSampledLength = inputLength/decimationFactor;
// Allocate downsampled output vector
float* downSampledData = malloc(downSampledLength*sizeof(float));
// Create filter (average samples)
float* filter = malloc(decimationFactor*sizeof(float));
for (int i = 0; i < decimationFactor; ++i){
filter[i] = 1.0/decimationFactor;
}
// Downsample and average
vDSP_desamp(inputData,
(vDSP_Stride) decimationFactor,
filter,
downSampledData,
(vDSP_Length) downSampledLength, // Downsample to 5 samples
(vDSP_Length) decimationFactor );
free(filter);
The output of downSampledData using this code is:
0.05, 0.25, 0.45, 0.65, 0.85
To upsample the (processed) data vector back to the original sample rate I use the following code:
// For this example downSampledData is just copied to processedData ...
float* processedData = malloc(downSampledLength*sizeof(float));
processedData = downSampledData;
// Create vector used by vDSP_vlint to indicate interpolation constants.
float* b = malloc(downSampledLength*sizeof(float));
for (int i = 0; i < downSampledLength; i++) {
b[i] = i + 0.5;
}
// Allocate data vector for upsampled data
float* upSampledData = malloc(inputLength*sizeof(float));
// Upsample and interpolate
vDSP_vlint (processedData,
b,
1,
upSampledData,
1,
(vDSP_Length) inputLength, // Resample back to 10 samples
(vDSP_Length) downSampledLength);
However, the output of upSampledData is
0.15, 0.35, 0.55, 0.75, 0.43, 0.05, 0.05, 0.05, 0.08, 0.12
which is not correct, apparently. How should I apply vDSP_vlint? Or should I use other functions for upsampling the data?
Related
I have a disparity image created with a calibrated stereo camera pair and opencv. It looks good, and my calibration data is good.
I need to calculate the real world distance at a pixel.
From other questions on stackoverflow, i see that the approach is:
depth = baseline * focal / disparity
Using the function:
setMouseCallback("disparity", onMouse, &disp);
static void onMouse(int event, int x, int y, int flags, void* param)
{
cv::Mat &xyz = *((cv::Mat*)param); //cast and deref the param
if (event == cv::EVENT_LBUTTONDOWN)
{
unsigned int val = xyz.at<uchar>(y, x);
double depth = (camera_matrixL.at<float>(0, 0)*T.at<float>(0, 0)) / val;
cout << "x= " << x << " y= " << y << " val= " << val << " distance: " << depth<< endl;
}
}
I click on a point that i have measured to be 3 meters away from the stereo camera.
What i get is:
val= 31 distance: 0.590693
The depth mat values are between 0 and 255, the depth mat is of type 0, or CV_8UC1.
The stereo baseline is 0.0643654 (in meters).
The focal length is 284.493
I have also tried:
(from OpenCV - compute real distance from disparity map)
float fMaxDistance = static_cast<float>((1. / T.at<float>(0, 0) * camera_matrixL.at<float>(0, 0)));
//outputDisparityValue is single 16-bit value from disparityMap
float fDisparity = val / (float)cv::StereoMatcher::DISP_SCALE;
float fDistance = fMaxDistance / fDisparity;
which gives me a (closer to truth, if we assume mm units) distance of val= 31 distance: 2281.27
But is still incorrect.
Which of these approaches is correct? And where am i going wrong?
Left, Right, Depth map. (EDIT: this depth map is from a different pair of images)
EDIT: Based on an answer, i am trying this:
`std::vector pointcloud;
float fx = 284.492615;
float fy = 285.683197;
float cx = 424;// 425.807709;
float cy = 400;// 395.494293;
cv::Mat Q = cv::Mat(4,4, CV_32F);
Q.at<float>(0, 0) = 1.0;
Q.at<float>(0, 1) = 0.0;
Q.at<float>(0, 2) = 0.0;
Q.at<float>(0, 3) = -cx; //cx
Q.at<float>(1, 0) = 0.0;
Q.at<float>(1, 1) = 1.0;
Q.at<float>(1, 2) = 0.0;
Q.at<float>(1, 3) = -cy; //cy
Q.at<float>(2, 0) = 0.0;
Q.at<float>(2, 1) = 0.0;
Q.at<float>(2, 2) = 0.0;
Q.at<float>(2, 3) = -fx; //Focal
Q.at<float>(3, 0) = 0.0;
Q.at<float>(3, 1) = 0.0;
Q.at<float>(3, 2) = -1.0 / 6; //1.0/BaseLine
Q.at<float>(3, 3) = 0.0; //cx - cx'
//
cv::Mat XYZcv(depth_image.size(), CV_32FC3);
reprojectImageTo3D(depth_image, XYZcv, Q, false, CV_32F);
for (int y = 0; y < XYZcv.rows; y++)
{
for (int x = 0; x < XYZcv.cols; x++)
{
cv::Point3f pointOcv = XYZcv.at<cv::Point3f>(y, x);
Eigen::Vector4d pointEigen(0, 0, 0, left.at<uchar>(y, x) / 255.0);
pointEigen[0] = pointOcv.x;
pointEigen[1] = pointOcv.y;
pointEigen[2] = pointOcv.z;
pointcloud.push_back(pointEigen);
}
}`
And that gives me a cloud.
I would recommend to use reprojectImageTo3D of OpenCV to reconstruct the distance from the disparity. Note that when using this function you indeed have to divide by 16 the output of StereoSGBM. You should already have all the parameters f, cx, cy, Tx. Take care to give f and Tx in the same units. cx, cy are in pixels.
Since the problem is that you need the Q matrix, I think that this link or this one should help you to build it. If you don't want to use reprojectImageTo3D I strongly recommend the first link!
I hope this helps!
To find the point-based depth of an object from the camera, use the following formula:
Depth = (Baseline x Focallength)/disparity
I hope you are using it correctly as per your question.
Try the below nerian calculator for the therotical error.
https://nerian.com/support/resources/calculator/
Also, use sub-pixel interpolation in your code.
Make sure object you are identifying for depth should have good texture.
The most common problems with depth maps are:
Untextured surfaces (plain object)
Calibration results are bad.
What is the RMS value for your calibration, camera resolution, and lens type(focal
length)? This is important to provide much better data for your program.
let's take this example:
we use X and Y data corresponding to the known polynomial f (x) = 0.25 - x + x2. Using POLY_FIT to compute a second degree polynomial fit returns the exact coefficients (to within machine accuracy).
; Define an 11-element vector of independent variable data:
X = [0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]
; Define an 11-element vector of dependent variable data:
Y = [0.25, 0.16, 0.09, 0.04, 0.01, 0.00, 0.01, 0.04, 0.09, $
0.16, 0.25]
; Define a vector of measurement errors:
measure_errors = REPLICATE(0.01, 11)
; Compute the second degree polynomial fit to the data:
result = POLY_FIT(X, Y, 2, MEASURE_ERRORS=measure_errors, $
SIGMA=sigma)
; Print the coefficients:
PRINT, 'Coefficients: ', result
PRINT, 'Standard errors: ', sigma
this example prints,
Coefficients: 0.250000 -1.00000 1.00000
Standard errors: 0.00761853 0.0354459 0.0341395
that works as expected, but let say I already know the coefficient 0.25, how can I pass to POLY_FIT that coefficient ? or maybe I have to use some other FIT function ?
source: http://www.exelisvis.com/docs/POLY_FIT.html
Have you looked at MPFIT? It has keywords to pass information about the parameters. Docs.
I've built a 3D Histogram in OpenCV from H-S-V samples from an (CV_8UC3) image.
I need to normalize this histogram so that all the values sum to 1.0 (preferrably in a float representation), since it will be used as a probability mass function (pmf) for a lookup table.
I've tried various permutations of built-in OpenCV functions, but none seem to give the desired result.
int histSize[] = {hBins, sBins, vBins};
float hRange[] = {0.0f, (float)H_RANGE};
float sRange[] = {0.0f, (float)S_RANGE};
float vRange[] = {0.0f, (float)V_RANGE};
const float* ranges[] = {hRange, sRange, vRange};
const int channels[] = {0, 1, 2}; // we compute the 3D histogram on all 2 channels (H-S-V)
cv::calcHist(&newBGSamples, 1, channels, cv::Mat(), currentBGColourHist, 3, histSize, ranges, true, false);
// currentBGColourHist /= cv::sum(bgHistoricalColourHist)(0);
cv::normalize(currentBGColourHist, currentBGColourHist, 1.0, 1.0, cv::NORM_L1, CV_32FC3);
// cv::normalize(currentBGColourHist, currentBGColourHist, 1.0, 0, cv::NORM_L2, -1, cv::Mat());
// cv::norm(currentBGColourHist, )
// cv::divide((double)1.0/cv::sum(bgHistoricalColourHist)(0), currentBGColourHist, currentBGColourHist, CV_32FC3);
The commented lines show my rough ideas for the normalisation.
I created three dimension matrix for computing of histogram as follows:
// Histogram of HSV image
int const hue_bins = 180; //
int const sat_bins = 256; //
int const val_bins = 4; // Only four bins for V channel!
float const hue_range[2] = {0, 180};
float const sat_range[2] = {0, 256};
float const val_range[2] = {0, 256};
int const hsv_sizes[] = {hue_bins, sat_bins, val_bins};
cv::Mat1f m_tone_frequences(3, hsv_sizes, 0.);
Then I'm using
cv::calcHist
( &image, 1, channels, mask, histogram
, num_channels, hsv_sizes, ranges);
...
cv::calcBackProject
( &image_f, 1, channels, histogram
, backproject, hsv_sizes, 1.0);
and seems it works fine (code is simplified).
Since the histograms are sampled from a single image, it is possible to run into sampling problems (object of interest has narrow color distribution). So I want to apply Gaussian smoothing to "Value" histogram planes.
I'm tried get histogram rows, but it gives me anothed 3D Mat:
cv::Mat1f hrow = histogram.row(0);
// hrow.dims ==3 && hrow.rows == -1 && hrow.cols == -1
and I don't have ideas about processing of it.
I am at a loss to solve this issue because this action should be very simple to do.
Any advice is greatly appreciated.
I have a piece of code for rotating and translating image:
Point2f pt(0, in.rows);
double angle = atan(trans.c / trans.b) * 180 / M_PI;
Mat r = getRotationMatrix2D(pt, -angle, 1.0);
warpAffine(in, out, r, in.size(), interpolation); /* rotation */
Mat t = (Mat_<double>(2, 3) << 1, 0, trans.a, 0, 1, -trans.d);
warpAffine(out, out, t, in.size(), interpolation); /* translation */
The problem is that I'm doing this in two times. So if I have an angle of 90degree for example, the first "out" variable will be empty because all data are out of bounds. Is there a way to do it in one pass ? In order to avoid loosing my data and having black image.
I think that the best thing would be to combine r and t in one matrix but I'm a little lost.
Best regards,
Here is an example on how to combine 2 homographies by simple multiplication and how to extract an affine transformation from a 3x3 homography.
int main(int argc, char* argv[])
{
cv::Mat input = cv::imread("C:/StackOverflow/Input/Lenna.png");
// create to 3x3 identity homography matrices
cv::Mat homography1 = cv::Mat::eye(3, 3, CV_64FC1);
cv::Mat homography2 = cv::Mat::eye(3, 3, CV_64FC1);
double alpha1 = -13; // degrees
double t1_x = -86; // pixel
double t1_y = -86; // pixel
double alpha2 = 21; // degrees
double t2_x = 86; // pixel
double t2_y = 86; // pixel
// hope there is no error in the signs:
// combine homography1
homography1.at<double>(0, 0) = cos(CV_PI*alpha1 / 180);
homography1.at<double>(0, 1) = -sin(CV_PI*alpha1 / 180);
homography1.at<double>(1, 0) = sin(CV_PI*alpha1 / 180);
homography1.at<double>(1, 1) = cos(CV_PI*alpha1 / 180);
homography1.at<double>(0, 2) = t1_x;
homography1.at<double>(1, 2) = t1_y;
// compose homography2
homography2.at<double>(0, 0) = cos(CV_PI*alpha2 / 180);
homography2.at<double>(0, 1) = -sin(CV_PI*alpha2 / 180);
homography2.at<double>(1, 0) = sin(CV_PI*alpha2 / 180);
homography2.at<double>(1, 1) = cos(CV_PI*alpha2 / 180);
homography2.at<double>(0, 2) = t2_x;
homography2.at<double>(1, 2) = t2_y;
cv::Mat affine1 = homography1(cv::Rect(0, 0, 3, 2));
cv::Mat affine2 = homography2(cv::Rect(0, 0, 3, 2));
cv::Mat dst1;
cv::Mat dst2;
cv::warpAffine(input, dst1, affine1, input.size());
cv::warpAffine(input, dst2, affine2, input.size());
cv::Mat combined_homog = homography1*homography2;
cv::Mat combined_affine = combined_homog(cv::Rect(0, 0, 3, 2));
cv::Mat dst_combined;
cv::warpAffine(input, dst_combined, combined_affine, input.size());
cv::imshow("input", input);
cv::imshow("dst1", dst1);
cv::imshow("dst2", dst2);
cv::imshow("combined", dst_combined);
cv::waitKey(0);
return 0;
}
In this example, an image is first rotated and translated to the left, later to the right. If the two transformations are performed after each other, significant image areas would get lost. Instead if they are combined by homograhy multiplication, it is like the full operation done in a single step without losing image parts in the intemediate step.
input:
if image was first transformed with H1, later with H2:
if the image is transformed with the combination of H1*H2 directly:
One typical application of this homography combination is to first translate the image center to the origin, then rotate, then translate back to original position. This has the effect as if the image was rotated around its center of gravity.