I have an rendered Image. I want to apply radial and tangential distortion coefficients to my image that I got from opencv. Even though there is undistort function, there is no distort function. How can I distort my images with distortion coefficients?
I was also looking for the same type of functionality. I couldn't find it, so I implemented it myself. Here is the C++ code.
First, you need to normalize the image point using the focal length and centers
rpt(0) = (pt_x - cx) / fx
rpt(1) = (pt_y - cy) / fy
then distort the normalized image point
double x = rpt(0), y = rpt(1);
//determining the radial distortion
double r2 = x*x + y*y;
double icdist = 1 / (1 - ((D.at<double>(4) * r2 + D.at<double>(1))*r2 + D.at<double>(0))*r2);
//determining the tangential distortion
double deltaX = 2 * D.at<double>(2) * x*y + D.at<double>(3) * (r2 + 2 * x*x);
double deltaY = D.at<double>(2) * (r2 + 2 * y*y) + 2 * D.at<double>(3) * x*y;
x = (x + deltaX)*icdist;
y = (y + deltaY)*icdist;
then you can translate and scale the point using the center of projection and focal length:
x = x * fx + cx
y = y * fy + cy
I'm training a YOLO model, I have the bounding boxes in this format:-
x1, y1, x2, y2 => ex (100, 100, 200, 200)
I need to convert it to YOLO format to be something like:-
X, Y, W, H => 0.436262 0.474010 0.383663 0.178218
I already calculated the center point X, Y, the height H, and the weight W.
But still need a away to convert them to floating numbers as mentioned.
for those looking for the reverse of the question (yolo format to normal bbox format)
def yolobbox2bbox(x,y,w,h):
x1, y1 = x-w/2, y-h/2
x2, y2 = x+w/2, y+h/2
return x1, y1, x2, y2
Here's code snipet in python to convert x,y coordinates to yolo format
def convert(size, box):
dw = 1./size[0]
dh = 1./size[1]
x = (box[0] + box[1])/2.0
y = (box[2] + box[3])/2.0
w = box[1] - box[0]
h = box[3] - box[2]
x = x*dw
w = w*dw
y = y*dh
h = h*dh
return (x,y,w,h)
im=Image.open(img_path)
w= int(im.size[0])
h= int(im.size[1])
print(xmin, xmax, ymin, ymax) #define your x,y coordinates
b = (xmin, xmax, ymin, ymax)
bb = convert((w,h), b)
Check my sample program to convert from LabelMe annotation tool format to Yolo format https://github.com/ivder/LabelMeYoloConverter
There is a more straight-forward way to do those stuff with pybboxes. Install with,
pip install pybboxes
use it as below,
import pybboxes as pbx
voc_bbox = (100, 100, 200, 200)
W, H = 1000, 1000 # WxH of the image
pbx.convert_bbox(voc_bbox, from_type="voc", to_type="yolo", image_size=(W,H))
>>> (0.15, 0.15, 0.1, 0.1)
Note that, converting to YOLO format requires the image width and height for scaling.
YOLO normalises the image space to run from 0 to 1 in both x and y directions. To convert between your (x, y) coordinates and yolo (u, v) coordinates you need to transform your data as u = x / XMAX and y = y / YMAX where XMAX, YMAX are the maximum coordinates for the image array you are using.
This all depends on the image arrays being oriented the same way.
Here is a C function to perform the conversion
#include <stdlib.h>
#include <stdio.h>
#include <errno.h>
#include <math.h>
struct yolo {
float u;
float v;
};
struct yolo
convert (unsigned int x, unsigned int y, unsigned int XMAX, unsigned int YMAX)
{
struct yolo point;
if (XMAX && YMAX && (x <= XMAX) && (y <= YMAX))
{
point.u = (float)x / (float)XMAX;
point.v = (float)y / (float)YMAX;
}
else
{
point.u = INFINITY;
point.v = INFINITY;
errno = ERANGE;
}
return point;
}/* convert */
int main()
{
struct yolo P;
P = convert (99, 201, 255, 324);
printf ("Yolo coordinate = <%f, %f>\n", P.u, P.v);
exit (EXIT_SUCCESS);
}/* main */
There are two potential solutions. First of all you have to understand if your first bounding box is in the format of Coco or Pascal_VOC. Otherwise you can't do the right math.
Here is the formatting;
Coco Format: [x_min, y_min, width, height]
Pascal_VOC Format: [x_min, y_min, x_max, y_max]
Here are some Python Code how you can do the conversion:
Converting Coco to Yolo
# Convert Coco bb to Yolo
def coco_to_yolo(x1, y1, w, h, image_w, image_h):
return [((2*x1 + w)/(2*image_w)) , ((2*y1 + h)/(2*image_h)), w/image_w, h/image_h]
Converting Pascal_voc to Yolo
# Convert Pascal_Voc bb to Yolo
def pascal_voc_to_yolo(x1, y1, x2, y2, image_w, image_h):
return [((x2 + x1)/(2*image_w)), ((y2 + y1)/(2*image_h)), (x2 - x1)/image_w, (y2 - y1)/image_h]
If need additional conversions you can check my article at Medium: https://christianbernecker.medium.com/convert-bounding-boxes-from-coco-to-pascal-voc-to-yolo-and-back-660dc6178742
For yolo format to x1,y1, x2,y2 format
def yolobbox2bbox(x,y,w,h):
x1 = int((x - w / 2) * dw)
x2 = int((x + w / 2) * dw)
y1 = int((y - h / 2) * dh)
y2 = int((y + h / 2) * dh)
if x1 < 0:
x1 = 0
if x2 > dw - 1:
x2 = dw - 1
if y1 < 0:
y1 = 0
if y2 > dh - 1:
y2 = dh - 1
return x1, y1, x2, y2
There are two things you need to do:
Divide the coordinates by the image size to normalize them to [0..1] range.
Convert (x1, y1, x2, y2) coordinates to (center_x, center_y, width, height).
If you're using PyTorch, Torchvision provides a function that you can use for the conversion:
from torch import tensor
from torchvision.ops import box_convert
image_size = tensor([608, 608])
boxes = tensor([[100, 100, 200, 200], [300, 300, 400, 400]], dtype=float)
boxes[:, :2] /= image_size
boxes[:, 2:] /= image_size
boxes = box_convert(boxes, "xyxy", "cxcywh")
Just reading the answers I am also looking for this but find this more informative to know what happening at the backend.
Form Here: Source
Assuming x/ymin and x/ymax are your bounding corners, top left and bottom right respectively. Then:
x = xmin
y = ymin
w = xmax - xmin
h = ymax - ymin
You then need to normalize these, which means give them as a proportion of the whole image, so simple divide each value by its respective size from the values above:
x = xmin / width
y = ymin / height
w = (xmax - xmin) / width
h = (ymax - ymin) / height
This assumes a top-left origin, you will have to apply a shift factor if this is not the case.
So the answer
I have a fisheye lens:
I would like to undistort it. I apply the FOV model:
rd = 1 / ω * arctan (2 * ru * tan(ω / 2)) //Equation 13
ru = tan(rd * ω) / 2 / tan(ω / 2) //Equation 14
as found in equations (13) and (14) of the INRIA paper "Straight lines have to be straight" https://hal.inria.fr/inria-00267247/document.
The code implementation is the following:
Point2f distortPoint(float w, float h, float cx, float cy, float omega, Point2f input) {
//w = width of the image
//h = height of the image
//cx = center of the lens in the image, aka w/2
//cy = center of the lens in the image, aka h/2
Point2f tmp = new Point2f();
//We normalize the coordinates of the point
tmp.x = input.x / w - cx / w;
tmp.y = input.y / h - cy / h;
//We apply the INRIA key formula (FOV model)
double ru = sqrt(tmp.x * tmp.x + tmp.y * tmp.y);
double rd = 1.0f / omega * atan(2.0f * ru * tan(omega / 2.0f));
tmp.x *= rd / ru;
tmp.y *= rd / ru;
//We "un-normalize" the point
Point2f ret = new Point2f();
ret.x = (tmp.x + cx / w) * w;
ret.y = (tmp.y + cy / h) * h;
return ret;
}
I then used the OpenCV remap function:
//map_x and map_y are computed with distortPoint
remap(img, imgUndistorted, map_x, map_y, INTER_LINEAR, BORDER_CONSTANT, Scalar(0, 0, 0));
I managed to get the distortion model from the lens manufacturer. It's a table of image_height as a function of the field angle:
Field angle(deg) Image Height (mm)
0 0
1 height1
2 height2
3 height3
...
89 height89
90 height90
To give an idea, each height is small and inferior to 2mm.
I found an interesting paper here: https://www.altera.com/content/dam/altera-www/global/en_US/pdfs/literature/wp/wp-01073-flexible-architecture-fisheye-correction-automotive-rear-view-cameras.pdf
How can I modify my pixel-unit undistortion function to factor in the mm-unit table provided by the manufacturer in order to have the most accurate possible undistorted image?
I have an image, and I place a rectangle on the image. Then I rotate the image. How do I get the center of the rectangle on the rotated image?
Or can I rotate a rectangle somehow to put on rotated image? I think in this case rotation must be done along same point with point used to rotate image.
This is the image with a rectangle placed on it.
This is the rotated image.
Here is the code I use to rotate my image:
cv::Mat frame, frameRotated;
frame = cv::imread("lena.png");
cv::Rect rect(225,250,150,150);
cv::rectangle(frame, rect, cv::Scalar(0,0,255),2);
int theta = 30;
double radians = theta * PI / 180.0;
double sin = abs(std::sin(radians));
double cos = abs(std::cos(radians));
int newWidth = (int) (frame.cols * cos + frame.rows * sin);
int newHeight = (int) (frame.cols * sin + frame.rows * cos);
cv::Mat targetMat(cv::Size(newWidth, newHeight), frame.type());
int offsetX = (newWidth - frame.cols) / 2;
int offsetY = (newHeight - frame.rows) / 2;
frame.copyTo(targetMat.rowRange(offsetY, offsetY + frame.rows).colRange(offsetX, offsetX + frame.cols));
cv::Point2f src_center(targetMat.cols/2.0F, targetMat.rows/2.0F);
cv::Mat rot_mat = cv::getRotationMatrix2D(src_center, theta, 1.0);
cv::warpAffine(targetMat, frameRotated, rot_mat, targetMat.size());
imshow("frame", frame);
imshow("rotated frame", frameRotated);
EDIT
Suppose I have a point in the rotated image, how do I get corresponding point in the original image using rotation matrix?
You only need to use rot_mat to transform the original center of the rect. I tested the below and it works:
cv::Rect r(250, 350, 20, 30);
cv::Point2d c(r.x + r.width / 2, r.y + r.height / 2);
// c is center of rect
// get c's location in targetMat when frame is copied
c.x += offsetX;
c.y += offsetY;
int theta = 30;
double radians = theta * M_PI / 180.0;
cv::Point2d src_center(targetMat.cols/2.0F, targetMat.rows/2.0F);
cv::Mat rot_mat = cv::getRotationMatrix2D(src_center, theta, 1.0);
// now transform point using rot_mat
double *x = rot_mat.ptr<double>(0);
double *y = rot_mat.ptr<double>(1);
Point2d dst(x[0] * c.x + x[1] * c.y + x[2],
y[0] * c.x + y[1] * c.y + y[2]);
// dst is center of transformed rect
EDIT
To transform a point from the rotated image you just need to reverse the process:
// undo translation
Point2d dst1(dst.x - x[2], dst.y - y[2]);
// undo rotation
Point2d dst2(x[0] * dst1.x - x[1] * dst1.y, -y[0] * dst1.x + y[1] * dst1.y);
// undo shift
Point2d in_unrotated_image(dst2.x - offsetX, dst2.y - offsetY);
You can translate any point in your source Mat to rotated Mat by multiplying with the Rotation matrix.
If you need to translate X,Y and given T=1, you can do this by Mat multiplication
| cosθ sinθ Tx | | X | | _X |
| | * | Y | = | |
| -sinθ cosθ Ty | | 1 | | _Y |
where Tx and Ty is the translation along the x and y see OpenCV Doc.
Suppose you need to find the centre (cent_x,cent_y) of source Mat in rotated Mat
Mat rot_mat = getRotationMatrix2D(src_center, theta, 1.0); //Find the rotation matrix
Mat co_Ordinate = (Mat_<double>(3,1) << cent_x,cent_y,1); //Create 3x1 matrix with input co-ordinates.
Mat rst=rot_mat*co_Ordinate; // Multiply rotation matrix with input co-ordinate matrix
trans_x=(int)rst.at<double>(0,0); //First row of result will be x
trans_y=(int)rst.at<double>(1,0); //Second row of result will be y.
Hopes these helpful....
I've solved the problem in second question.
I used the same code provided in the accepted answer and create rotation matrix with minus theta.
I better explain my problem with an Image
I have a contour and a line which is passing through that contour.
At the intersection point of contour and line I want to draw a perpendicular line at the intersection point of a line and contour up to a particular distance.
I know the intersection point as well as slope of the line.
For reference I am attaching this Image.
If the blue line in your picture goes from point A to point B, and you want to draw the red line at point B, you can do the following:
Get the direction vector going from A to B. This would be:
v.x = B.x - A.x; v.y = B.y - A.y;
Normalize the vector:
mag = sqrt (v.x*v.x + v.y*v.y); v.x = v.x / mag; v.y = v.y / mag;
Rotate the vector 90 degrees by swapping x and y, and inverting one of them. Note about the rotation direction: In OpenCV and image processing in general x and y axis on the image are not oriented in the Euclidian way, in particular the y axis points down and not up. In Euclidian, inverting the final x (initial y) would rotate counterclockwise (standard for euclidean), and inverting y would rotate clockwise. In OpenCV it's the opposite. So, for example to get clockwise rotation in OpenCV: temp = v.x; v.x = -v.y; v.y = temp;
Create a new line at B pointing in the direction of v:
C.x = B.x + v.x * length; C.y = B.y + v.y * length;
(Note that you can make it extend in both directions by creating a point D in the opposite direction by simply negating length.)
This is my version of the function :
def getPerpCoord(aX, aY, bX, bY, length):
vX = bX-aX
vY = bY-aY
#print(str(vX)+" "+str(vY))
if(vX == 0 or vY == 0):
return 0, 0, 0, 0
mag = math.sqrt(vX*vX + vY*vY)
vX = vX / mag
vY = vY / mag
temp = vX
vX = 0-vY
vY = temp
cX = bX + vX * length
cY = bY + vY * length
dX = bX - vX * length
dY = bY - vY * length
return int(cX), int(cY), int(dX), int(dY)