GSvideo error in Processing 2.03 - opencv

I am working in a Processing project, but when I try to record the sketch with the GSvideo library I get this error:
A library used by this sketch is not installed properly.
GSVideo version: 1.0.0
A library relies on native code that's not available.
Or only works properly when the sketch is run as a 64-bit application.
In my project I'm tracking objects with the HSV space color and the OpenCV for Processing library and I want to record the sketch just so I can show later my work. This is my code:
/**
* HSVColorTracking
* Greg Borenstein
* https://github.com/atduskgreg/opencv-processing-book/blob/master/code/hsv_color_tracking/HSVColorTracking/HSVColorTracking.pde
*
* Modified by Jordi Tost #jorditost (color selection)
* University of Applied Sciences Potsdam, 2014
*
* Modified by Luz Alejandra Magre
* Universidad Tecnológica de Bolívar, 2015
*/
import gab.opencv.*;
import processing.video.*;
import codeanticode.gsvideo.*;
import java.awt.Rectangle;
GSCapture video;
OpenCV opencv;
GSMovieMaker mm;
int fps = 30;
PImage src, colorFilteredImage;
ArrayList<Contour> contours;
// <1> Set the range of Hue values for our filter
int rangeLow = 20;
int rangeHigh = 35;
void setup() {
frameRate(fps);
String[] cameras = GSCapture.list();
size(2*opencv.width, opencv.height, P2D);
if (cameras.length == 0)
{
println("There are no cameras available for capture.");
exit();
}
else {
println("Available cameras:");
for (int i = 0; i < cameras.length; i++) {
println(cameras[i]);
}
video = new GSCapture(this, 640, 480, cameras[0]);
video.start();
opencv = new OpenCV(this, video.width, video.height);
contours = new ArrayList<Contour>();
mm = new GSMovieMaker(this, width, height, "Test.ogg", GSMovieMaker.THEORA, GSMovieMaker.BEST, fps);
mm.setQueueSize(50, 10);
mm.start();
}
}
void draw() {
// Read last captured frame
if (video.available()) {
video.read();
}
// <2> Load the new frame of our movie in to OpenCV
opencv.loadImage(video);
// Tell OpenCV to use color information
opencv.useColor();
src = opencv.getSnapshot();
// <3> Tell OpenCV to work in HSV color space.
opencv.useColor(HSB);
// <4> Copy the Hue channel of our image into
// the gray channel, which we process.
opencv.setGray(opencv.getH().clone());
// <5> Filter the image based on the range of
// hue values that match the object we want to track.
opencv.inRange(rangeLow, rangeHigh);
// <6> Get the processed image for reference.
colorFilteredImage = opencv.getSnapshot();
///////////////////////////////////////////
// We could process our image here!
// See ImageFiltering.pde
///////////////////////////////////////////
// <7> Find contours in our range image.
// Passing 'true' sorts them by descending area.
contours = opencv.findContours(true, true);
// <8> Display background images
image(src, 0, 0);
image(colorFilteredImage, src.width, 0);
// <9> Check to make sure we've found any contours
if (contours.size() > 0) {
// <9> Get the first contour, which will be the largest one
Contour biggestContour = contours.get(0);
// <10> Find the bounding box of the largest contour,
// and hence our object.
Rectangle r = biggestContour.getBoundingBox();
// <11> Draw the bounding box of our object
noFill();
strokeWeight(2);
stroke(255, 0, 0);
rect(r.x, r.y, r.width, r.height);
// <12> Draw a dot in the middle of the bounding box, on the object.
noStroke();
fill(255, 0, 0);
ellipse(r.x + r.width/2, r.y + r.height/2, 30, 30);
text(r.x + r.width/2, 50, 50);
text(r.y + r.height/2, 50, 80);
}
loadPixels();
mm.addFrame(pixels);
saveFrame("frame-######.png");
}
void mousePressed() {
color c = get(mouseX, mouseY);
println("r: " + red(c) + " g: " + green(c) + " b: " + blue(c));
int hue = int(map(hue(c), 0, 255, 0, 180));
println("hue to detect: " + hue);
rangeLow = hue - 5;
rangeHigh = hue + 5;
}
void keyPressed() {
if (key == ' ') {
// Finish the movie if space bar is pressed
mm.finish();
// Quit running the sketch once the file is written
exit();
}
}
I would really appreciate the help on this.

Related

How to use cv::decode (access image) correct?

I need help with the following problem:
Task script:
read in the message sensor_msgs/PointCloud2, display Bird Eye View image and save (png or jpg).
Desired new function:
Send out the displayed images directly as an image message.
Problem:
cv::Mat *bgr is the matrix that contains the image and gives it to a map (for visualisation only).
Solutions by others/so far:
opencv read jpeg image from buffer //
How to use cv::imdecode, if the contents of an image file are in a char array?
Using different member functions, but unsuccessful.
Code reduced to necessary snippets
(complete version here: https://drive.google.com/file/d/1HI3E4nM9mQ--oNh1Q7zfwRFGJB5JRiGD/view?usp=sharing)
// Global Publishers/Subscribers
ros::Subscriber subPointCloud;
ros::Publisher pubPointCloud;
image_transport::Publisher pubImage;
pcl::PointCloud<pcl::PointXYZ>::Ptr cloud (new pcl::PointCloud<pcl::PointXYZ>);
pcl::PointCloud<pcl::PointXYZ>::Ptr cloud_grid (new pcl::PointCloud<pcl::PointXYZ>);
sensor_msgs::PointCloud2 output;
// create Matrix to store pointcloud data
cv::Mat *heightmap, *hsv, *bgr;
std::vector<int> compression_params;
std::vector<String> fn; //filename
cv::Mat image;
// main generation function
void DEM(const sensor_msgs::PointCloud2ConstPtr& pointCloudMsg)
{
ROS_DEBUG("Point Cloud Received");
// clear cloud and height map array
lowest = FLT_MAX;
for(int i = 0; i < IMAGE_HEIGHT; ++i){
for(int j = 0; j < IMAGE_WIDTH; ++j){
heightArray[i][j] = (double)(-FLT_MAX);
}
}
// Convert from ROS message to PCL point cloud
pcl::fromROSMsg(*pointCloudMsg, *cloud);
// Populate the DEM grid by looping through every point
int row, column;
for(size_t j = 0; j < cloud->points.size(); ++j){
// If the point is within the image size bounds
if(map_pc2rc(cloud->points[j].x, cloud->points[j].y, &row, &column) == 1 && row >= 0 && row < IMAGE_HEIGHT && column >=0 && column < IMAGE_WIDTH){
if(cloud->points[j].z > heightArray[row][column]){
heightArray[row][column] = cloud->points[j].z;
}
// Keep track of lowest point in cloud for flood fill
else if(cloud->points[j].z < lowest){
lowest = cloud->points[j].z;
}
}
}
// Create "point cloud" and opencv image to be published for visualization
int index = 0;
double x, y;
for(int i = 0; i < IMAGE_HEIGHT; ++i){
for(int j = 0; j < IMAGE_WIDTH; ++j){
// Add point to cloud
(void)map_rc2pc(&x, &y, i, j);
cloud_grid->points[index].x = x;
cloud_grid->points[index].y = y;
cloud_grid->points[index].z = heightArray[i][j];
++index;
// Add point to image
cv::Vec3b &pixel_hsv = hsv->at<cv::Vec3b>(i,j); // access pixels vector HSV
cv::Vec3b &pixel_bgr = heightmap->at<cv::Vec3b>(i,j); // access pixels vector BGR
if(heightArray[i][j] > -FLT_MAX){
//Coloured Pixel Pointcloud
pixel_hsv[0] = map_m2i(heightArray[i][j]); // H - color value (hue)
pixel_hsv[1] = 255; // S -color saturation
pixel_hsv[2] = 255; // V - brightness
// White Pixel PointCloud
pixel_bgr[0] = map_m2i(heightArray[i][j]); // B
pixel_bgr[1] = map_m2i(heightArray[i][j]); // G
pixel_bgr[2] = map_m2i(heightArray[i][j]); // R
}
else{
// Coloured Pixel Pointcloud
pixel_hsv[0] = 0;
pixel_hsv[1] = 0;
pixel_hsv[2] = 0;
// White Pixel Pointcloud
pixel_bgr[0] = 0;
pixel_bgr[1] = 0;
pixel_bgr[2] = 0; //map_m2i(lowest);
}
}
}
// Display image
cv::cvtColor(*hsv, *bgr, cv::COLOR_HSV2BGR); // HSV matrix (src) to BGR matrix (dst)
// Image denoising (filter strength, pixel size template patch, pixel size window)
//cv::fastNlMeansDenoising(*hsv,*bgr,30 , 7, 11);
// Image denoising (filter strength luminance, same colored, pixel size template patch, pixel size window)
//cv::fastNlMeansDenoisingColored(*hsv,*bgr,30 ,1, 7, 11);
// Plot HSV(colored) and BGR (b/w)
cv::imshow(WIN_NAME, *bgr); // show new HSV matrix
cv::imshow(WIN_NAME2, *heightmap); // show old BGR matrix
// Save image to disk
char filename[100];
// FLAG enable/disable saving function
if (save_to_disk == true)
{
// save JPG format
snprintf(filename, 100, "/home/pkatsoulakos/catkin_ws/images/image_%d.jpg", fnameCounter);
std::cout << filename << std::endl;
// JPG image writing
cv::imwrite(filename, *bgr, compression_params);
/* // generate pathnames matching a pattern
glob("/home/pkatsoulakos/catkin_ws/images/*.jpg",fn); // directory, filter pattern
// range based for loop
for (auto f:fn) // range declaration:range_expression
{
image = cv::imread(f, IMREAD_COLOR);
if (image.empty())
{
std::cout << "!!! Failed imread(): image not found" << std::endl;
}
}*/
// Approach 2
//cv::Mat rawdata(1, bgr,CV_8UC1,(void*)bgr);
image = cv::imdecode(cv::Mat(*bgr, CV_8UC3, CV_AUTO_STEP), IMREAD_COLOR);
//image = cv::imdecode(cv::Mat(*bgr, CV_8UC1), IMREAD_UNCHANGED);
if (image.data == NULL)
{
std::cout << "!!! Failed imread(): image not found" << std::endl;
}
/* // save PNG format
snprintf(filename, 100, "/home/pkatsoulakos/catkin_ws/images/image_%d.png", fnameCounter);
std::cout << filename << std::endl;
// PNG image writing
// cv::imwrite(filename, *heightmap, compression_params);*/
}
++fnameCounter;
// Output height map to point cloud for python node to parse to PNG
pcl::toROSMsg(*cloud_grid, output);
output.header.stamp = ros::Time::now();
output.header.frame_id = "yrl_cloud_id"; // fixed frame (oblique alignment) from LiDAR
pubPointCloud.publish(output);
// Publish bird_view img
cv_bridge::CvImage cv_bird_view;
cv_bird_view.header.stamp = ros::Time::now();
cv_bird_view.header.frame_id = "out_bev_image";
cv_bird_view.encoding = "bgr8";
cv_bird_view.image = image;
pubImage.publish(cv_bird_view.toImageMsg());
// Output Image
//sensor_msgs::ImagePtr msg = cv_bridge::CvImage(std_msgs::Header(), "bgr8", image).toImageMsg();
//pubImage.publish(msg);pubPoin
}
int main(int argc, char** argv)
{
ROS_INFO("Starting LIDAR Node");
ros::init(argc, argv, "lidar_node");
ros::NodeHandle nh;
image_transport::ImageTransport it(nh);
// Setup image
cv::Mat map(IMAGE_HEIGHT, IMAGE_WIDTH, CV_8UC3, cv::Scalar(0, 0, 0));
cv::Mat map2(IMAGE_HEIGHT, IMAGE_WIDTH, CV_8UC3, cv::Scalar(0, 0, 0));
cv::Mat map3(IMAGE_HEIGHT, IMAGE_WIDTH, CV_8UC3, cv::Scalar(0, 0, 0));
// H S V
// image container
heightmap = &map; // default source code (mcshiggings)
hsv = &map2; // added for hSV visualization
bgr = &map3; // for displaying colored Pc
cv::namedWindow(WIN_NAME, WINDOW_AUTOSIZE);
cv::namedWindow(WIN_NAME2, WINDOW_AUTOSIZE);
cv::startWindowThread();
cv::imshow(WIN_NAME, *bgr); // BGR visualization of HSV
cv::imshow(WIN_NAME2, *heightmap); // default visualization
// Setup Image Output Parameters
fnameCounter = 0;
lowest = FLT_MAX;
/* PNG compression param
compression_params.push_back(IMWRITE_PNG_COMPRESSION);
A higher value means a smaller size and longer compression time. Default value is 3.
compression_params.push_back(9); */
// JPG compression param
compression_params.push_back(IMWRITE_JPEG_QUALITY);
// from 0 to 100 (the higher is the better). Default value is 95.
compression_params.push_back(95);
// Setup indicies in point clouds
/*
int index = 0;
double x, y;
for(int i = 0; i < IMAGE_HEIGHT; ++i){
for(int j = 0; j < IMAGE_WIDTH; ++j){
index = i * j;
(void)map_rc2pc(&x, &y, i, j);
cloud_grid->points[index].x = x;
cloud_grid->points[index].y = y;
cloud_grid->points[index].z = (-FLT_MAX)master.log
}
*/
// subscriber and publisher
subPointCloud = nh.subscribe<sensor_msgs::PointCloud2>("/pointcloud", 2, DEM);
pubPointCloud = nh.advertise<sensor_msgs::PointCloud2> ("/heightmap/pointcloud", 1);
pubImage = it.advertise("/out_bev_image",1);
ros::spin();
return 0;
}
Thank you for any advice and suggested solutions.
You can't simply pass the char array to opencv functions to create an image because of how the data is formatted. PointCloud2 data fields are strictly containing information about where a point lives in 3d space(think [x,y,z]); this means nothing in terms of an actual image. Instead you have to first convert the pointcloud into something that better represents an image. Luckily, this already exists. Check out the CloudToImage ROS package.

How to minimize face-detection error

here is the code now I can both detect face and mouth together, and able to roughly measure the distance of its bounding box <--
the problem is the mouth detection seems to detects everything they defines as mouth even it is not
and I want to use the "face" bounding box as a mouth detection region to minimize its error, I don't know if Forloop stacked would work? by put mouth loop inside face loop?? I'm fairly new to coding any help would be appreciated
import gab.opencv.*;
import java.awt.Rectangle;
import processing.video.*;
Capture video;
OpenCV f;
OpenCV m;
void setup() {
size(800, 600);
video = new Capture(this, 800/2, 600/2);
f = new OpenCV(this, 800/2, 600/2);
m = new OpenCV(this, 800/2, 600/2);
video.start();
}
void draw() {
scale(2);
f.loadImage(video);
m.loadImage(video);
f.loadCascade(OpenCV.CASCADE_FRONTALFACE);
m.loadCascade(OpenCV.CASCADE_MOUTH);
image(video, 0, 0 );
noFill();
stroke(0, 255, 0);
strokeWeight(3);
Rectangle[] mouth = m.detect();
Rectangle[] face = f.detect();
println(mouth.length);
strokeWeight(3);
for (int i = 0; i < face.length; i++) {
println(face[i].x + "," + face[i].y);
rect(face[i].x, face[i].y, face[i].width, face[i].height);
}
for (int i = 0; i < mouth.length; i++) {
println(mouth[i].x + "," + mouth[i].y);
rect(mouth[i].x, mouth[i].y, mouth[i].width, mouth[i].height);
}
for (int i = 0; i < mouth.length; i++) {
fill(255, 0, 0);
noStroke();
ellipse((mouth[i].x)+(mouth[i].width/2), mouth[i].y, 5, 5);
ellipse((mouth[i].x)+(mouth[i].width/2), (mouth[i].y)+ (mouth[i].height), 5, 5);
}
for (int i = 0; i < mouth.length; i++) {
int px = (mouth[i].x)+(mouth[i].width/2);
int py = (mouth[i].y)+(mouth[i].height);
int mOpen = int (dist(px, mouth[i].y, px, py));
println(mOpen);
}
}
void captureEvent(Capture d) {
d.read();
}
There are a couple issues:
You shouldn't be loading OpenCV cascades multiple times a second in draw(). You should do it once in setup() and just call detect() in draw()
OpenCV for Processing seems to override the cascade loaded in the second instance with a cascade loaded in the first instance
If accuracy isn't a huge issue, you can get away with a single cascade: the mouth one. Note that there are options/hints you can use for the detect function which may help the detection. For example you can tell the detector to detect largest object only, give it a hint of the smallest and largest bounding boxes the mouth would have with your setup and how much should the results filtered out.
Here's a code sample for the above:
import gab.opencv.*;
import java.awt.Rectangle;
import org.opencv.objdetect.Objdetect;
import processing.video.*;
Capture video;
OpenCV opencv;
//cascade detections parameters - explanations from Mastering OpenCV with Practical Computer Vision Projects
int flags = Objdetect.CASCADE_FIND_BIGGEST_OBJECT;
// Smallest object size.
int minFeatureSize = 20;
int maxFeatureSize = 80;
// How detailed should the search be. Must be larger than 1.0.
float searchScaleFactor = 1.1f;
// How much the detections should be filtered out. This should depend on how bad false detections are to your system.
// minNeighbors=2 means lots of good+bad detections, and minNeighbors=6 means good detections are given but some are missed.
int minNeighbors = 6;
void setup() {
size(320, 240);
noFill();
stroke(0, 192, 0);
strokeWeight(3);
video = new Capture(this,width,height);
video.start();
opencv = new OpenCV(this,320,240);
opencv.loadCascade(OpenCV.CASCADE_MOUTH);
}
void draw() {
//feed cam image to OpenCV, it turns it to grayscale
opencv.loadImage(video);
opencv.equalizeHistogram();
image(opencv.getOutput(), 0, 0 );
Rectangle[] mouths = opencv.detect(searchScaleFactor,minNeighbors,flags,minFeatureSize, maxFeatureSize);
for (int i = 0; i < mouths.length; i++) {
text(mouths[i].x + "," + mouths[i].y + "," + mouths[i].width + "," + mouths[i].height,mouths[i].x, mouths[i].y);
rect(mouths[i].x, mouths[i].y, mouths[i].width, mouths[i].height);
}
}
void captureEvent(Capture c) {
c.read();
}
Note that facial hair can cause false positives.
I have provided more in depth notes in an answer to your previous related question. I recommend focusing on the FaceOSC part as it will be more accurate.

detect paper from background almost same as paper color

I am trying to detect a piece of paper from an image. Well I have had help from posts like this . But one difference is that my background color is almost same as the paper color, and I am getting wrong result.
[edit]
By wrong result I mean the paper outline contour is not detected at all. instead the largest contour covers the whole image.
image1
image2
My code so far (using emgu cv and c#)
MemStorage storage = new MemStorage();
List<Contour<Point>> candidateList = new List<Contour<Point>>();
List<double> areaList = new List<double>();
Image<Bgr, Byte> inputImage = new Image<Bgr, Byte>(image);
//Rectangle roi = new Rectangle(15, 15, image.Width - 15, image.Height - 15);
//inputImage.ROI = roi;
//inputImage = inputImage.Copy();
double threshHeight = inputImage.Size.Height * 0.50;
double threshWidth = inputImage.Size.Width * 0.50;
//std::vector<std::vector<cv::Point> > squares;
//cv::Mat pyr, timg, gray0(_image.size(), CV_8U), gray;
int thresh = 50, N = 5;
//cv::pyrDown(_image, pyr, cv::Size(_image.cols/2, _image.rows/2));
//cv::pyrUp(pyr, timg, _image.size());
//std::vector<std::vector<cv::Point> > contours;
Image<Gray, Byte> [] gray0 = new Image<Gray,byte>[3];
for( int c = 0; c < 3; c++ ) {
try{
int [] ch = {c, 0};
gray0[c] = new Image<Gray,byte>(inputImage.Size);
CvInvoke.cvMixChannels(new IntPtr[] { inputImage }, 1, new IntPtr[]{gray0[c]}, 1, ch, 1);
Image<Gray, Byte> gray = new Image<Gray,byte>(inputImage.Size);
for (int l = 0 ; l < N ; l++){
if (l == 0){
Image<Gray, Byte> cannyImage = gray0[c].Canny(0, thresh, 5);
CvInvoke.cvDilate(cannyImage, gray, IntPtr.Zero, 1);
//CvInvoke.cvShowImage("ch " + c + "-" + l, gray);
}else{
CvInvoke.cvThreshold(gray0[c], gray, (l + 1)*255/N, 255, Emgu.CV.CvEnum.THRESH.CV_THRESH_BINARY);
//CvInvoke.cvShowImage("ch " + c + "-" + l, gray0[c]);
}
//CvInvoke.cvShowImage("image", gray);
for (Contour<Point> contours = gray.FindContours(Emgu.CV.CvEnum.CHAIN_APPROX_METHOD.CV_CHAIN_APPROX_SIMPLE, Emgu.CV.CvEnum.RETR_TYPE.CV_RETR_LIST, storage); contours != null; contours = contours.HNext){
Contour<Point> currentContour = contours.ApproxPoly(contours.Perimeter * 0.02, storage);
if (currentContour.Count() >= 4){
if (currentContour.Area > 3000){
//if (currentContour.BoundingRectangle.Width >= threshWidth && currentContour.BoundingRectangle.Height > threshHeight){
candidateList.Add(currentContour);
areaList.Add(currentContour.Area);
inputImage.Draw(currentContour, new Bgr(255, 0, 0), 1);
}
}
}
}
}catch(Exception ex){
Debug.WriteLine(ex.Message);
}
}
/* finding the biggest one */
double area = -1.0;
Contour<Point> paper = null;
for (int i = 0 ; i < candidateList.Count ; i++){
if (areaList[i] > area){
area = areaList[i];
paper = candidateList[i];
}
}
if (paper != null){
if (paper.BoundingRectangle.Width >= threshWidth && paper.BoundingRectangle.Height > threshHeight){
inputImage.Draw(paper, new Bgr(0, 0, 255), 2);
}
}
return inputImage.ToBitmap();
Please let me know how to process these images.
I did this in matlab (sorry I'm not really proficient at OpenCV) but you should be able to emulate the code. I tried to make it very simple. I noticed that the gradient of the original images really highlights where the paper is. So I used that to make a "rough" outline of the paper. Using the gradient is a good starting point, and maybe you can start from there. I just downsampled, then upsampled the image (emulating morphological operations to clean the image, since you lose all the small details).
You might get better results if you smooth the image out first (with a Gaussian filter maybe) I didn't try it, but maybe you can give it a try. Here is the result
And here is the code for reference
im1 = imread('http://www.imageno.com/image.php?id=ai7b91pm9fcs&kk=1089743759');
im2 = imread('http://www.imageno.com/image.php?id=k99c9xpd6phs&kk=3354581295');
%converts to grayscale
gim1 = rgb2gray(im1);
gim2 = rgb2gray(im2);
%gets size of images
[m1, n1] = size(gim1);
[m2, n2] = size(gim2);
%takes gradient of image
[Gx1, Gy1] = gradient(double(gim1));
[Gx2, Gy2] = gradient(double(gim2));
%takes magnitude of gradient in X and Y direction
Gxy1 = sqrt(Gx1.^2 + Gy1.^2);
Gxy2 = sqrt(Gx2.^2 + Gy2.^2);
%downsamples image (to reduce noise)
scale_factor = 100;
small1 = imresize(Gxy1, [m1/scale_factor n1/scale_factor]);
small2 = imresize(Gxy2, [m2/scale_factor n2/scale_factor]);
%upsamples image (to original size)
big1 = imresize(small1, [m1 n1]);
big2 = imresize(small2, [m2 n2]);
%converts to binary mask
bw1 = (big1 >= 1);
bw2 = (big2 >= 1);
%displays images
figure(1);
subplot(2,4,1);imshow(gim1);title('grayscale 1');
subplot(2,4,5);imshow(gim2);title('grayscale 2');
%these gradients are a little deceiving. In matlab when it sees an image
%of type "double" its maps it so 0=black and 1=white. anything >=1 gets
%clipped to 1
subplot(2,4,2);imshow(Gxy1);title('gradient 1');
subplot(2,4,6);imshow(Gxy2);title('gradient 2');
subplot(2,4,3);imshow(big1);title('down->up sampled 1');
subplot(2,4,7);imshow(big2);title('down->up sampled 2');
%this is just some matlab witchcraft so I can multiply the 2D mask with a
%3D image (r g b) in a very efficient manner
subplot(2,4,4);imshow(bsxfun(#times,im1,uint8(bw1)));title('masked image 1');
subplot(2,4,8);imshow(bsxfun(#times,im2,uint8(bw2)));title('masked image 2');

Image Straightening in iOS using Open CV framework

Code :
cv::Point2f src_vertices[4];
src_vertices[0] = c1[0];
src_vertices[1] = c1[1];
src_vertices[2] = c1[2];
src_vertices[3] = c1[3];
cv::Point2f dst_vertices[4];
dst_vertices[0] = c2[0];
dst_vertices[1] = c2[1];
dst_vertices[2] = c2[2];
dst_vertices[3] = c2[3];
cv::Mat warpMatrix = getPerspectiveTransform(src_vertices,dst_vertices);
cv::Mat output = cv::Mat::zeros(original.cols,original.rows , CV_32FC3);
cv::warpPerspective(original, output, warpMatrix,cv::Size(606,606));
UIImage *_adjustedImage = [MAOpenCV UIImageFromCVMat:output];
Below is the original image
After apply straightening, output is below image
Issue
The output of the image that we are getting after straightening is getting cropped a bit from the corner and the output comes from the Open CV framework itself.
How to resolved this issue. Please let me know if anybody has found the solution. Thank you.
Since this question is asked quite often, I've written a few lines of code which save some time for many others.
try this:
cv::Rect computeWarpedContourRegion(const std::vector<cv::Point> & points, const cv::Mat & homography)
{
std::vector<cv::Point2f> transformed_points(points.size());
for(unsigned int i=0; i<points.size(); ++i)
{
// warp the points
transformed_points[i].x = points[i].x * homography.at<double>(0,0) + points[i].y * homography.at<double>(0,1) + homography.at<double>(0,2) ;
transformed_points[i].y = points[i].x * homography.at<double>(1,0) + points[i].y * homography.at<double>(1,1) + homography.at<double>(1,2) ;
}
// dehomogenization necessary?
if(homography.rows == 3)
{
float homog_comp;
for(unsigned int i=0; i<transformed_points.size(); ++i)
{
homog_comp = points[i].x * homography.at<double>(2,0) + points[i].y * homography.at<double>(2,1) + homography.at<double>(2,2) ;
transformed_points[i].x /= homog_comp;
transformed_points[i].y /= homog_comp;
}
}
// now find the bounding box for these points:
cv::Rect boundingBox = cv::boundingRect(transformed_points);
return boundingBox;
}
cv::Rect computeWarpedImageRegion(const cv::Mat & image, const cv::Mat & homography)
{
std::vector<cv::Point> imageBorder;
imageBorder.push_back(cv::Point(0,0));
imageBorder.push_back(cv::Point(image.cols,0));
imageBorder.push_back(cv::Point(image.cols,image.rows));
imageBorder.push_back(cv::Point(0,image.rows));
return computeWarpedContourRegion(imageBorder, homography);
}
cv::Mat adjustHomography(const cv::Rect & transformedRegion, const cv::Mat & homography)
{
if(homography.rows == 2) throw("homography adjustement for affine matrix not implemented yet");
// unit matrix
cv::Mat correctionHomography = cv::Mat::eye(3,3,CV_64F);
// correction translation
correctionHomography.at<double>(0,2) = -transformedRegion.x;
correctionHomography.at<double>(1,2) = -transformedRegion.y;
return correctionHomography * homography;
}
int main()
{
// straightening algorithm without cropping:
cv::Mat original = cv::imread("straightening_src.png");
cv::Mat output;
cv::Point2f src_vertices[4];
cv::Point2f dst_vertices[4];
// I have to add them manually, you can just use your old code here.
// my result will look different, since I don't use your original point correspondences, but system is the same...
src_vertices[0] = cv::Point2f(108,190);
src_vertices[1] = cv::Point2f(273,178);
src_vertices[2] = cv::Point2f(389,322);
src_vertices[3] = cv::Point2f(183,355);
dst_vertices[0] = cv::Point2f(172,190);
dst_vertices[1] = cv::Point2f(374,193);
dst_vertices[2] = cv::Point2f(380,362);
dst_vertices[3] = cv::Point2f(171,366);
// compute homography
cv::Mat warpMatrix = getPerspectiveTransform(src_vertices,dst_vertices);
// now you have to find out, whether the warped image will fit to the output image or whether it will be cropped.
// if it will be cropped you will most probably have to
// 1. find out how big your output image must be and the coordinates it will be warped to.
// 2. modify your transformation (by a translation) so that the output image will be placed properly inside the output image
// part 1: find the region that will hold the new image.
cv::Rect warpedImageRegion = computeWarpedImageRegion(original, warpMatrix);
// part 2: modify the transformation.
cv::Mat adjustedHomography = adjustHomography(warpedImageRegion, warpMatrix);
cv::Size transformedImageSize = cv::Size(warpedImageRegion.width,warpedImageRegion.height);
cv::warpPerspective(original, output, adjustedHomography, transformedImageSize);
cv::imshow("output", output);
cv::imwrite("straightening_result.png", output);
cv::waitKey(-1);
}
for this input (1) and the given transformation correspondences you will get that result (2)
(1)
(2)
After the image is skewed, it should be possible to remove the black extra part of the image.

Drawing spectrum of an image in C++ (fftw, OpenCV)

I'm trying to create a program that will draw a 2d greyscale spectrum of a given image. I'm using OpenCV and FFTW libraries. By using tips and codes from the internet and modifying them I've managed to load an image, calculate fft of this image and recreate the image from the fft (it's the same). What I'm unable to do is to draw the fourier spectrum itself. Could you please help me?
Here's the code (less important lines removed):
/* Copy input image */
/* Create output image */
/* Allocate input data for FFTW */
in = (fftw_complex*) fftw_malloc(sizeof(fftw_complex) * N);
dft = (fftw_complex*) fftw_malloc(sizeof(fftw_complex) * N);
/* Create plans */
plan_f = fftw_plan_dft_2d(w, h, in, dft, FFTW_FORWARD, FFTW_ESTIMATE);
/* Populate input data in row-major order */
for (i = 0, k = 0; i < h; i++)
{
for (j = 0; j < w; j++, k++)
{
in[k][0] = ((uchar*)(img1->imageData + i * img1->widthStep))[j];
in[k][1] = 0.;
}
}
/* forward DFT */
fftw_execute(plan_f);
/* spectrum */
for (i = 0, k = 0; i < h; i++)
{
for (j = 0; j < w; j++, k++)
((uchar*)(img2->imageData + i * img2->widthStep))[j] = sqrt(pow(dft[k][0],2) + pow(dft[k][1],2));
}
cvShowImage("iplimage_dft(): original", img1);
cvShowImage("iplimage_dft(): result", img2);
cvWaitKey(0);
/* Free memory */
}
The problem is in the "Spectrum" section. Instead of a spectrum I get some noise. What am I doing wrong? I would be grateful for your help.
You need to draw magnitude of spectrum. here is the code.
void ForwardFFT(Mat &Src, Mat *FImg)
{
int M = getOptimalDFTSize( Src.rows );
int N = getOptimalDFTSize( Src.cols );
Mat padded;
copyMakeBorder(Src, padded, 0, M - Src.rows, 0, N - Src.cols, BORDER_CONSTANT, Scalar::all(0));
// Создаем комплексное представление изображения
// planes[0] содержит само изображение, planes[1] его мнимую часть (заполнено нулями)
Mat planes[] = {Mat_<float>(padded), Mat::zeros(padded.size(), CV_32F)};
Mat complexImg;
merge(planes, 2, complexImg);
dft(complexImg, complexImg);
// После преобразования результат так-же состоит из действительной и мнимой части
split(complexImg, planes);
// обрежем спектр, если у него нечетное количество строк или столбцов
planes[0] = planes[0](Rect(0, 0, planes[0].cols & -2, planes[0].rows & -2));
planes[1] = planes[1](Rect(0, 0, planes[1].cols & -2, planes[1].rows & -2));
Recomb(planes[0],planes[0]);
Recomb(planes[1],planes[1]);
// Нормализуем спектр
planes[0]/=float(M*N);
planes[1]/=float(M*N);
FImg[0]=planes[0].clone();
FImg[1]=planes[1].clone();
}
void ForwardFFT_Mag_Phase(Mat &src, Mat &Mag,Mat &Phase)
{
Mat planes[2];
ForwardFFT(src,planes);
Mag.zeros(planes[0].rows,planes[0].cols,CV_32F);
Phase.zeros(planes[0].rows,planes[0].cols,CV_32F);
cv::cartToPolar(planes[0],planes[1],Mag,Phase);
}
Mat LogMag;
LogMag.zeros(Mag.rows,Mag.cols,CV_32F);
LogMag=(Mag+1);
cv::log(LogMag,LogMag);
//---------------------------------------------------
imshow("Логарифм амплитуды", LogMag);
imshow("Фаза", Phase);
imshow("Результат фильтрации", img);
Can you try to do the IFFT step and see if you recover the original image ? then , you can check step by step where is your problem. Another solution to find the problem is to do this process with a small matrix predefined by you ,and calculate it FFT in MATLAB, and check step by step, it worked for me!

Resources