Situation: I am trying to get point cloud with pcl::AdaptiveCostSOStereoMatching, which uses two rectified images (pics are ok).
I used these tutorials to learn how to do this:
First tutorial
Second tutorial
Error: programm crashes in runtime when calling "compute" method of AdaptiveCostSOStereoMatching
Question: how to correctly pass images to "compute" method?
I tried:
1) Images converted by png2pcd
(command line: "png2pcd.exe in.png out.pcd")
2) Images converted with function below from cv::Mat
But no luck.
Function which converts cv::Mat to pcl::PointCloud
void MatToPointCloud(Mat& mat, pcl::PointCloud<RGB>::Ptr cloud)
{
int width = mat.cols;
int height = mat.rows;
pcl::RGB val;
val.r = 0; val.g = 0; val.b = 0;
for (int i = 0; i < mat.rows; i++)
for (int j = 0; j < mat.cols; j++)
{
auto point = mat.at<Vec3b>(i, j);
//std::cout << j << " " << i << "\n";
val.b = point[0];
val.g = point[1];
val.r = point[2];
cloud->at(j, i) = val;
}
}
pcl::AdaptiveCostSOStereoMatching (compute)
// Input
Mat leftMat, rightMat;
leftMat = imread("left.png");
rightMat = imread("right.png");
int width = leftMat.cols;
int height = rightMat.rows;
pcl::RGB val;
val.r = 0; val.g = 0; val.b = 0;
pcl::PointCloud<pcl::RGB>::Ptr left_cloud(new pcl::PointCloud<pcl::RGB>(width, height, val));
pcl::PointCloud<pcl::RGB>::Ptr right_cloud(new pcl::PointCloud<pcl::RGB>(width, height, val));
MatToPointCloud(leftMat, left_cloud);
MatToPointCloud(rightMat, right_cloud);
// Calculation
pcl::AdaptiveCostSOStereoMatching stereo;
stereo.setMaxDisparity(60);
//stereo.setXOffest(0); Почему-то не распознается
stereo.setRadius(5);
stereo.setSmoothWeak(20);
stereo.setSmoothStrong(100);
stereo.setGammaC(25);
stereo.setGammaS(10);
stereo.setRatioFilter(20);
stereo.setPeakFilter(0);
stereo.setLeftRightCheck(true);
stereo.setLeftRightCheckThreshold(1);
stereo.setPreProcessing(true);
stereo.compute(*left_cloud, *right_cloud); // <-- CRASHING THERE
stereo.medianFilter(4);
pcl::PointCloud<pcl::PointXYZRGB>::Ptr out_cloud(new pcl::PointCloud<pcl::PointXYZRGB>);
stereo.getPointCloud(318.11220, 224.334900, 368.534700, 0.8387445, out_cloud, left_cloud);
Error information:
Output log: HEAP[App.exe]:
Heap block at 0000006B0F828460 modified at 0000006B0F8284A8 past requested size of 38
App.exe has triggered a breakpoint.
left_cloud (a right cloud looks like left_cloud)
Mini question: if AdaptiveCostSOStereoMatching really allows build point cloud from 2 images, how ACSSM doing this without insintric and excentic parameters?
Problem: I downloaded and installed old version of PCL without stereo.
After that, I downloaded stereo from other PCL pack and add this library to my PCL pack. And it worked incorrectly.
Solution: I compilled PCL 1.8 and my programm is ok now.
OS: Windows
IDE: MSVS 12 2013 x64
If you will try to compile PCL, these links can help you:
Official-tutorial-1
Official-tutorial-2
Good help with FLANN and VTK
Example to verify installation
Related
Hi i'm new in pointcloud library. I'm trying to show clustering result point on rviz or pcl viewer, and then show nothing. And i realize that my data show nothing too when i subcsribe and cout that. Hopefully can help my problem, thanks
This is my code for clustering and send node
void cloudReceive(const sensor_msgs::PointCloud2ConstPtr& inputMsg){
mutex_lock.lock();
pcl::fromROSMsg(*inputMsg, *inputCloud);
cout<<inputCloud<<endl;
pcl::search::KdTree<pcl::PointXYZRGB>::Ptr tree (new pcl::search::KdTree<pcl::PointXYZRGB>);
tree->setInputCloud(inputCloud);
std::vector<pcl::PointIndices> cluster_indices;
pcl::EuclideanClusterExtraction<pcl::PointXYZRGB> ec;
ec.setClusterTolerance(0.03);//2cm
ec.setMinClusterSize(200);//min points
ec.setMaxClusterSize(1000);//max points
ec.setSearchMethod(tree);
ec.setInputCloud(inputCloud);
ec.extract(cluster_indices);
if(cluster_indices.size() > 0){
std::vector<pcl::PointIndices>::const_iterator it;
int i = 0;
for (it = cluster_indices.begin(); it != cluster_indices.end(); ++it){
if(i >= 10)
break;
cloud_cluster[i]->points.clear();
std::vector<int>::const_iterator idx_it;
for (idx_it = it->indices.begin(); idx_it != it->indices.end(); idx_it++)
cloud_cluster[i]->points.push_back(inputCloud->points[*idx_it]);
cloud_cluster[i]->width = cloud_cluster[i]->points.size();
// cloud_cluster[i]->height = 1;
// cloud_cluster[i]->is_dense = true;
cout<<"PointCloud representing the Cluster: " << cloud_cluster[i]->points.size() << " data points"<<endl;
std::stringstream ss;
ss<<"cobaa_pipecom2_cluster_"<< i << ".pcd";
writer.write<pcl::PointXYZRGB> (ss.str(), *cloud_cluster[i], false);
pcl::toROSMsg(*cloud_cluster[i], outputMsg);
// cout<<"data = "<< outputMsg <<endl;
cloud_cluster[i]->header.frame_id = FRAME_ID;
pclpub[i++].publish(outputMsg);
// i++;
}
}
else
ROS_INFO_STREAM("0 clusters extracted\n");
}
And this one is the main
int main(int argc, char** argv){
for (int z = 0; z < 10; z++) {
// std::cout << " - clustering/" << z << std::endl;
cloud_cluster[z] = pcl::PointCloud<pcl::PointXYZRGB>::Ptr(new pcl::PointCloud<pcl::PointXYZRGB>);
cloud_cluster[z]->height = 1;
cloud_cluster[z]->is_dense = true;
// cloud_cluster[z]->header.frame_id = FRAME_ID;
}
ros::init(argc,argv,"clustering");
ros::NodeHandlePtr nh(new ros::NodeHandle());
pclsub = nh->subscribe("/pclsegmen",1,cloudReceive);
std::string pub_str("clustering/0");
for (int z = 0; z < 10; z++) {
pub_str[11] = z + 48;//48=0(ASCII)
// z++;
pclpub[z] = nh->advertise <sensor_msgs::PointCloud2> (pub_str, 1);
}
// pclpub = nh->advertise<sensor_msgs::PointCloud2>("/pclcluster",1);
ros::spin();
}
This isn't an exact answer, but I think it addresses your issue & may ease your debugging.
RViz can directly subscribe to a published point cloud, the one I'm assuming you're trying to see in the cloud_receive callback. If you set the Frame to whichever frame it's being published at, and add it from the available topics, you should see the points. (Easier than trying to rebroadcast it as different topics).
Also, I recommend looking at the rostopic command line tool. You can do rostopic list to check if it's being published, rostopic bw to see if it's really publishing the expected volume of data (ex bytes vs kilobytes vs megabytes), rostopic hz to see how frequently (if ever) it's publishing, and (briefly) rostopic echo to look at the data itself. (This is me assuming from your question it's more an issue with the data coming into your node).
If you're having trouble, not with data coming into the node, nor with the visualization of pointcloud data in general, but with the transformed data that's supposed to come out of the node, I would check that the clustering worked, & reduce your code moreso to just having 1 publisher publish something. You may be doing something weird. Like messing up your pointers. You could also turn on stronger compilation warnings for your node with -Wall -Wextra -Werror or step through the execution of it via gdb (launch-prefix="xterm -e gdb --args").
The solution is, i change the ASCII number into lexical_cast. Thanks for your response, i hope this can help other
for (int z = 0; z < CLOUD_QTD; z++) {
// pub_str[11] = z + 48;
std::string topicName = "/pclcluster/" + boost::lexical_cast<std::string>(z);
global::pub[z] = n.advertise <sensor_msgs::PointCloud2> (topicName, 1);
}
Scenario:
I am using OpenH264 with my App to encode into a video_file.mp4.
Environment:
Platform : MacOs Sierra
Compiler : Clang++
The code:
Following is the crux of the code I have:
void EncodeVideoFile() {
ISVCEncoder * encoder_;
std:string video_file_name = "/Path/to/some/folder/video_file.mp4";
EncodeFileParam * pEncFileParam;
SEncParamExt * pEnxParamExt;
float frameRate = 1000;
EUsageType usageType = EUsageType::CAMERA_VIDEO_REAL_TIME;
bool denoise = false;
bool lossless = true;
bool enable_ltr = false;
int layers = 1;
bool cabac = false;
int sliceMode = 1;
pEncFileParam = new EncodeFileParam;
pEncFileParam->eUsageType = EUsageType::CAMERA_VIDEO_REAL_TIME;
pEncFileParam->pkcFileName = video_file_name.c_str();
pEncFileParam->iWidth = frame_width;
pEncFileParam->iHeight = frame_height;
pEncFileParam->fFrameRate = frameRate;
pEncFileParam->iLayerNum = layers;
pEncFileParam->bDenoise = denoise;
pEncFileParam->bLossless = lossless;
pEncFileParam->bEnableLtr = enable_ltr;
pEncFileParam->bCabac = cabac;
int rv = WelsCreateSVCEncoder (&encoder_);
pEnxParamExt = new SEncParamExt;
pEnxParamExt->iUsageType = pEncFileParam->eUsageType;
pEnxParamExt->iPicWidth = pEncFileParam->iWidth;
pEnxParamExt->iPicHeight = pEncFileParam->iHeight;
pEnxParamExt->fMaxFrameRate = pEncFileParam->fFrameRate;
pEnxParamExt->iSpatialLayerNum = pEncFileParam->iLayerNum;
pEnxParamExt->bEnableDenoise = pEncFileParam->bDenoise;
pEnxParamExt->bIsLosslessLink = pEncFileParam->bLossless;
pEnxParamExt->bEnableLongTermReference = pEncFileParam->bEnableLtr;
pEnxParamExt->iEntropyCodingModeFlag = pEncFileParam->bCabac ? 1 : 0;
for (int i = 0; i < pEnxParamExt->iSpatialLayerNum; i++) {
pEnxParamExt->sSpatialLayers[i].sSliceArgument.uiSliceMode = pEncFileParam->eSliceMode;
}
encoder_->InitializeExt(pEnxParamExt);
int videoFormat = videoFormatI420;
encoder_->SetOption (ENCODER_OPTION_DATAFORMAT, &videoFormat);
int frameSize = frame_width * frame_height * 3 / 2;
int total_num = 500;
BufferedData buf;
buf.SetLength (frameSize);
// check the buffer before proceeding
if (buf.Length() != (size_t)frameSize) {
CloseEncoder();
return;
}
SFrameBSInfo info;
memset (&info, 0, sizeof (SFrameBSInfo));
SSourcePicture pic;
memset (&pic, 0, sizeof (SSourcePicture));
pic.iPicWidth = frame_width;
pic.iPicHeight = frame_height;
pic.iColorFormat = videoFormatI420;
pic.iStride[0] = pic.iPicWidth;
pic.iStride[1] = pic.iStride[2] = pic.iPicWidth >> 1;
pic.pData[0] = buf.data();
pic.pData[1] = pic.pData[0] + frame_width * frame_height;
pic.pData[2] = pic.pData[1] + (frame_width * frame_height >> 2);
for(int num = 0; num < total_num; num++) {
// try to encode the frame
rv = encoder_->EncodeFrame (&pic, &info);
}
if (encoder_) {
encoder_->Uninitialize();
WelsDestroySVCEncoder (encoder_);
}
}
Above code is something I pulled up from official usage examples of OpenH264 where BufferedData.h is a class I reused from OpenH264 utils
Issue:
But, I am getting the following error:
[OpenH264] this = 0x0x1038bc8c0, Error:ParamValidationExt(), width > 0, height > 0, width * height <= 9437184, invalid 0 x 0 in dependency layer settings!
[OpenH264] this = 0x0x1038bc8c0, Error:WelsInitEncoderExt(), ParamValidationExt failed return 2.
[OpenH264] this = 0x0x1038bc8c0, Error:CWelsH264SVCEncoder::Initialize(), WelsInitEncoderExt failed.
Above does not crash the application but it goes through a blank run without creating the video_file.mp4 with the dummy data that I am trying to write into it.
Question:
There seems to be something wrong with the set up config I applying to pEnxParamExtwhich goes into encoder_->InitializeExt.
What am I doing wrong with the set up of the encoder?
Note:
I am not trying to hook up to any camera device. I am just trying to create a .mp4 video out of some dummy image data.
If you want to get complete and working OpenH264 Encoder Initialization procedure you can click... here.
According to your problem scenario, you are trying to create a video file(.mp4/.avi) from some dummy images. This task can be accomplished using two different libraries: i) Library for Codec, ii) Library for Container.
i) Library for Codec: It's so much easy to use a OpenH264 to compress data. One thing I must mention is that, OpenH264 always works with raw frames e.g. yuv420 data. So, if you want to compress your image data, you have to convert these image data into yuv420 color format. To get OpenH264 click... here
ii) Library for Container: After getting the encoded data you have to use another library to create the container with extension .mp4, .avi, .flv etc. There exists a lot of libraries in github to do that staff like FFmpeg, OpenCV, Bento4, MP4Maker, mp4parser etc. Before using these libraries please check in detail about the license issues. If you use FFmpeg, you will not need to use OpenH264 becuse FFmpeg itself works along with several codecs. You will also find lot more working examples as so many developers are working with video data out there.
Hope it helps. :)
I am using OrbFeaturesFinder to detect keypoints in Images.
Ptr<FeaturesFinder> finder;
finder = makePtr<OrbFeaturesFinder>();
vector<ImageFeatures> features(num_images);
(*finder)(img, features[i]);
I used this code on linux and implemented the same on android, but the results are different sometimes, as in given link
http://imgur.com/a/wQXZx
What can be reason behind this nature of output.
method of accessing images in android
Image is saved in jpeg form, then read[edit] -
for(int i = 0; i < imgNames.size(); i++){
Bitmap bitmap = getThumbnail(imgNames.get(i));
int imageW = bitmap.getWidth();
int imageH = bitmap.getHeight();
byte[] rgb = getByteArray(imageW, imageH, bitmap, "RGB");
bitmap.recycle();
Mat mRgb = new Mat(imageH, imageW, CvType.CV_8UC3);
mRgb.put(0, 0, rgb);
Imgproc.cvtColor(mRgb, mRgb, Imgproc.COLOR_BGR2RGB, 3);
panoImgs.add(mRgb);
}
and sent to jni -
jclass matClass = env->FindClass("org/opencv/core/Mat");
jmethodID getNativeAddr = env->GetMethodID(matClass, "getNativeObjAddr", "()J");
int numImgs = env->GetArrayLength(jInputArray);
vector<Mat> natImgs;
for(int i=0; i < numImgs; ++i) {
natImgs.push_back(
*(Mat*)env->CallLongMethod(
env->GetObjectArrayElement(jInputArray, i),
getNativeAddr
)
);
}
for linux - I am saving the same image in jpeg format and then using imread to access files.
I have recently started porting an application to a new platform which runs OpenCV 2.4.5.
Part of my code which uses OpenCV's implementation of FLANN to do hierarchical clustering no longer compiles.
The code is as follows:
cv::Mat mergedFeatures = cvCreateMat(descriptorTotal, descriptorDims, CV_32F);
int counter = 0;
for (uint j = 0; j < ImageFeatures.size(); j++) {
cv::Mat features = ImageFeatures[j];
for (int k = 0; k < features.rows; k++) {
cv::Mat roi = mergedFeatures.row(counter);
features.row(k).copyTo(roi);
counter++;
}
}
cv::Mat centers = cvCreateMat(1000, descriptorDims, CV_32FC1);
cv::flann::KMeansIndexParams k_params = cv::flann::KMeansIndexParams();
cv::flann::AutotunedIndexParams atp = cv::flann::AutotunedIndexParams();
int numClusters = cv::flann::hierarchicalClustering<float, float>(mergedFeatures, centers, k_params);
The error that I am getting (in Eclipse) is that cv::flann::hierarchicalClustering has invalid arguments and that neither of the candidates for this function are met.
Can someone explain how I suddenly seem to be calling this method incorrectly?
Many thanks for any help.
I fixed the problem myself.
Instead of accepting:
cv::flann::KMeansIndexParams k_params
the hierarchicalClustering function actually needs:
cvflann::KMeansIndexParams k_params
It is rather a confusing namespace convention with the FLANN library in OpenCV and I had just overlooked the namespace difference in what the compiler error was telling me.
It is all working now. The KMeansIndexParams type is present in both namespaces and I guess that the hierarchicalClustering method has changed very slightly from OpenCV 2.3 to 2.4.5.
As a result of a call to estimateRigidTransform() I get a cv::Mat object named "trans". To retrieve its contained matrix I try to access its elements this way:
for (i=0; i<2; i++) for (j=0; j<3; j++)
{
mtx[j][i]=CV_MAT_ELEM(trans,double,i,j);
}
Unfortunately with VS2010 I get a compiler error
error C2228: left of '.ptr' must have class/struct/union
for the line with CV_MAT_ELEM. When I unwrap this macro I find something like
(mat).data.ptr + (size_t)(mat).step*(row) + (pix_size)*(col))
When I remove the ".ptr" behind (mat).data it compiles. But I can't imagine that's the solution (or can't imagine that's a bug and I'm the only one who noticed it). So what could be wrong here really?
Thanks!
You don't access the mat elements like this. For a traversal see my other answer here with sample code:
color matrix traversal
or see the opencv refman for grayscale Mat:
Mat M; // should be grayscale
int cols = M.cols, rows = M.rows;
for(int i = 0; i < rows; i++)
{
const double* Mi = M.ptr<double>(i);
for(int j = 0; j < cols; j++)
{
Mi[j]; // is the matrix element.
}
}
Just an addendum from my side: meanwhile I found CV_MAT_ELEM expects a structure CvMat (OpenCV-C-interface) but not cv::Mat (the C++-interface). That's why I get this funny error. Conversion from cv::Mat to CvMat can be done simply by casting to CvMat. Funny confusion with the C and C++ interfaces in OpenCV...