How to read a frame from YUV file in OpenCV? - image-processing

How to read a frame from YUV file in OpenCV?

I wrote a very simple python code to read YUV NV21 stream from binary file.
import cv2
import numpy as np
class VideoCaptureYUV:
def __init__(self, filename, size):
self.height, self.width = size
self.frame_len = self.width * self.height * 3 / 2
self.f = open(filename, 'rb')
self.shape = (int(self.height*1.5), self.width)
def read_raw(self):
try:
raw = self.f.read(self.frame_len)
yuv = np.frombuffer(raw, dtype=np.uint8)
yuv = yuv.reshape(self.shape)
except Exception as e:
print str(e)
return False, None
return True, yuv
def read(self):
ret, yuv = self.read_raw()
if not ret:
return ret, yuv
bgr = cv2.cvtColor(yuv, cv2.COLOR_YUV2BGR_NV21)
return ret, bgr
if __name__ == "__main__":
#filename = "data/20171214180916RGB.yuv"
filename = "data/20171214180916IR.yuv"
size = (480, 640)
cap = VideoCaptureYUV(filename, size)
while 1:
ret, frame = cap.read()
if ret:
cv2.imshow("frame", frame)
cv2.waitKey(30)
else:
break

As mentioned, there are MANY types of YUV formats:
http://www.fourcc.org/yuv.php
To convert to RGB from a YUV format in OpenCV is very simple:
Create a one-dimensional OpenCV Mat of the appropriate size for that frame data
Create an empty Mat for the RGB data with the desired dimension AND with 3 channels
Finally use cvtColor to convert between the two Mats, using the correct conversion flag enum
Here is an example for a YUV buffer in YV12 format:
Mat mYUV(height + height/2, width, CV_8UC1, (void*) frameData);
Mat mRGB(height, width, CV_8UC3);
cvtColor(mYUV, mRGB, CV_YUV2RGB_YV12, 3);
The key trick is to define the dimensions of your RGB Mat before you convert.

UPDATE there's a newer version of the code here: https://github.com/chelyaev/opencv-yuv
I'm posting some code that will read a single YUV 4:2:0 planar image file. You can directly apply this to most YUV files (just keep reading from the same FILE object). The exception to this is when dealing with YUV files that have a header (typically, they have a *.y4m extension). If you want to deal with such files, you have two options:
Write your own function to consume the header data from the FILE object before using the code below
Strip the headers from *.y4m images (using ffmpeg or similar tool). This is the option I prefer since it's the simplest.
It also will not work for any other form of YUV format (non-planar, different chroma decimation). As #Stephane pointed out, there are many such formats (and most of them don't have any identifying headers), which is probably why OpenCV doesn't support them out of the box.
But working with them is fairly simple:
Start with an image and it's dimensions (this is required when reading a YUV file)
Read luma and chroma into 3 separate images
Upscale chroma images by a factor of 2 to compensation for chroma decimation. Note that there are actually several ways to compensate for chroma decimation. Upsampling is just the simplest
Combine into YUV image. If you want RGB, you can use cvCvtColor.
Finally, the code:
IplImage *
cvLoadImageYUV(FILE *fin, int w, int h)
{
assert(fin);
IplImage *py = cvCreateImage(cvSize(w, h), IPL_DEPTH_8U, 1);
IplImage *pu = cvCreateImage(cvSize(w/2,h/2), IPL_DEPTH_8U, 1);
IplImage *pv = cvCreateImage(cvSize(w/2,h/2), IPL_DEPTH_8U, 1);
IplImage *pu_big = cvCreateImage(cvSize(w, h), IPL_DEPTH_8U, 1);
IplImage *pv_big = cvCreateImage(cvSize(w, h), IPL_DEPTH_8U, 1);
IplImage *image = cvCreateImage(cvSize(w, h), IPL_DEPTH_8U, 3);
IplImage *result = NULL;
assert(py);
assert(pu);
assert(pv);
assert(pu_big);
assert(pv_big);
assert(image);
for (int i = 0; i < w*h; ++i)
{
int j = fgetc(fin);
if (j < 0)
goto cleanup;
py->imageData[i] = (unsigned char) j;
}
for (int i = 0; i < w*h/4; ++i)
{
int j = fgetc(fin);
if (j < 0)
goto cleanup;
pu->imageData[i] = (unsigned char) j;
}
for (int i = 0; i < w*h/4; ++i)
{
int j = fgetc(fin);
if (j < 0)
goto cleanup;
pv->imageData[i] = (unsigned char) j;
}
cvResize(pu, pu_big, CV_INTER_NN);
cvResize(pv, pv_big, CV_INTER_NN);
cvMerge(py, pu_big, pv_big, NULL, image);
result = image;
cleanup:
cvReleaseImage(&pu);
cvReleaseImage(&pv);
cvReleaseImage(&py);
cvReleaseImage(&pu_big);
cvReleaseImage(&pv_big);
if (result == NULL)
cvReleaseImage(&image);
return result;
}

I don't think it is possible to do, at least with the current version. Of course, it wouldn't be that difficult to do, but it is not such an interesting feature, as:
OpenCV usually works on webcam stream, which are in RGB format, or on coded files, which are directly decoded into RGB for display purposes ;
OpenCV is dedicated to Computer Vision, where YUV is a less common format than in the Coding community for example ;
there are a lot of different YUV formats, which would imply a lot of work to implement them.
Conversions are still possible though, using cvCvtColor(), which means that it is of some interest anyway.

I encountered the same problem. My solution is
1. read one yuv frame (such as I420) to a string object "yuv".
2. convert the yuv frame to BGR24 format. I use libyuv to do it. It is easy to write a python wrapper for libyuv functions. now you get another string object "bgr" with BGR24 format.
3. use numpy.fromstring to get image object from the "bgr" string object. you need to change the shape of the image object.
Below is a simple yuv viewer for your reference.
import cv2
# below is the extension wrapper for libyuv
import yuvtorgb
import numpy as np
f = open('i420_cif.yuv', 'rb')
w = 352
h = 288
size = 352*288*3/2
while True:
try:
yuv = f.read(size)
except:
break
if len(yuv) != size:
f.seek(0, 0)
continue
bgr = yuvtorgb.i420_to_bgr24(yuv, w, h)
img = np.fromstring(bgr, dtype=np.uint8)
img.shape = h,w,3
cv2.imshow('img', img)
if cv2.waitKey(50) & 0xFF == ord('q'):
break
cv2.destroyAllWindows()

For future reference: I have converted #xianyanlin's brilliant answer to Python 3. The below code works with videos taken from the Raspberry Pi camera and seems to output correct color and aspect ratio.
Warning: it uses the numpy format for specifying resolution of height * width, e.g. 1080 * 1920, 480 * 640.
class VideoCaptureYUV:
def __init__(self, filename, size):
self.height, self.width = size
self.frame_len = self.width * self.height * 3 // 2
self.f = open(filename, 'rb')
self.shape = (int(self.height*1.5), self.width)
def read_raw(self):
try:
raw = self.f.read(self.frame_len)
yuv = np.frombuffer(raw, dtype=np.uint8)
yuv = yuv.reshape(self.shape)
except Exception as e:
print(str(e))
return False, None
return True, yuv
def read(self):
ret, yuv = self.read_raw()
if not ret:
return ret, yuv
bgr = cv2.cvtColor(yuv, cv2.COLOR_YUV2BGR_I420, 3)
return ret, bgr

Related

Preprocessing for OCR [duplicate]

I've been using tesseract to convert documents into text. The quality of the documents ranges wildly, and I'm looking for tips on what sort of image processing might improve the results. I've noticed that text that is highly pixellated - for example that generated by fax machines - is especially difficult for tesseract to process - presumably all those jagged edges to the characters confound the shape-recognition algorithms.
What sort of image processing techniques would improve the accuracy? I've been using a Gaussian blur to smooth out the pixellated images and seen some small improvement, but I'm hoping that there is a more specific technique that would yield better results. Say a filter that was tuned to black and white images, which would smooth out irregular edges, followed by a filter which would increase the contrast to make the characters more distinct.
Any general tips for someone who is a novice at image processing?
fix DPI (if needed) 300 DPI is minimum
fix text size (e.g. 12 pt should be ok)
try to fix text lines (deskew and dewarp text)
try to fix illumination of image (e.g. no dark part of image)
binarize and de-noise image
There is no universal command line that would fit to all cases (sometimes you need to blur and sharpen image). But you can give a try to TEXTCLEANER from Fred's ImageMagick Scripts.
If you are not fan of command line, maybe you can try to use opensource scantailor.sourceforge.net or commercial bookrestorer.
I am by no means an OCR expert. But I this week had the need to convert text out of a jpg.
I started with a colorized, RGB 445x747 pixel jpg.
I immediately tried tesseract on this, and the program converted almost nothing.
I then went into GIMP and did the following.
image > mode > grayscale
image > scale image > 1191x2000 pixels
filters > enhance > unsharp mask with values of
radius = 6.8, amount = 2.69, threshold = 0
I then saved as a new jpg at 100% quality.
Tesseract then was able to extract all the text into a .txt file
Gimp is your friend.
As a rule of thumb, I usually apply the following image pre-processing techniques using OpenCV library:
Rescaling the image (it's recommended if you’re working with images that have a DPI of less than 300 dpi):
img = cv2.resize(img, None, fx=1.2, fy=1.2, interpolation=cv2.INTER_CUBIC)
Converting image to grayscale:
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
Applying dilation and erosion to remove the noise (you may play with the kernel size depending on your data set):
kernel = np.ones((1, 1), np.uint8)
img = cv2.dilate(img, kernel, iterations=1)
img = cv2.erode(img, kernel, iterations=1)
Applying blur, which can be done by using one of the following lines (each of which has its pros and cons, however, median blur and bilateral filter usually perform better than gaussian blur.):
cv2.threshold(cv2.GaussianBlur(img, (5, 5), 0), 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
cv2.threshold(cv2.bilateralFilter(img, 5, 75, 75), 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
cv2.threshold(cv2.medianBlur(img, 3), 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
cv2.adaptiveThreshold(cv2.GaussianBlur(img, (5, 5), 0), 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 31, 2)
cv2.adaptiveThreshold(cv2.bilateralFilter(img, 9, 75, 75), 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 31, 2)
cv2.adaptiveThreshold(cv2.medianBlur(img, 3), 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 31, 2)
I've recently written a pretty simple guide to Tesseract but it should enable you to write your first OCR script and clear up some hurdles that I experienced when things were less clear than I would have liked in the documentation.
In case you'd like to check them out, here I'm sharing the links with you:
Getting started with Tesseract - Part I: Introduction
Getting started with Tesseract - Part II: Image Pre-processing
Three points to improve the readability of the image:
Resize the image with variable height and width(multiply 0.5 and 1 and 2 with image height and width).
Convert the image to Gray scale format(Black and white).
Remove the noise pixels and make more clear(Filter the image).
Refer below code :
Resize
public Bitmap Resize(Bitmap bmp, int newWidth, int newHeight)
{
Bitmap temp = (Bitmap)bmp;
Bitmap bmap = new Bitmap(newWidth, newHeight, temp.PixelFormat);
double nWidthFactor = (double)temp.Width / (double)newWidth;
double nHeightFactor = (double)temp.Height / (double)newHeight;
double fx, fy, nx, ny;
int cx, cy, fr_x, fr_y;
Color color1 = new Color();
Color color2 = new Color();
Color color3 = new Color();
Color color4 = new Color();
byte nRed, nGreen, nBlue;
byte bp1, bp2;
for (int x = 0; x < bmap.Width; ++x)
{
for (int y = 0; y < bmap.Height; ++y)
{
fr_x = (int)Math.Floor(x * nWidthFactor);
fr_y = (int)Math.Floor(y * nHeightFactor);
cx = fr_x + 1;
if (cx >= temp.Width) cx = fr_x;
cy = fr_y + 1;
if (cy >= temp.Height) cy = fr_y;
fx = x * nWidthFactor - fr_x;
fy = y * nHeightFactor - fr_y;
nx = 1.0 - fx;
ny = 1.0 - fy;
color1 = temp.GetPixel(fr_x, fr_y);
color2 = temp.GetPixel(cx, fr_y);
color3 = temp.GetPixel(fr_x, cy);
color4 = temp.GetPixel(cx, cy);
// Blue
bp1 = (byte)(nx * color1.B + fx * color2.B);
bp2 = (byte)(nx * color3.B + fx * color4.B);
nBlue = (byte)(ny * (double)(bp1) + fy * (double)(bp2));
// Green
bp1 = (byte)(nx * color1.G + fx * color2.G);
bp2 = (byte)(nx * color3.G + fx * color4.G);
nGreen = (byte)(ny * (double)(bp1) + fy * (double)(bp2));
// Red
bp1 = (byte)(nx * color1.R + fx * color2.R);
bp2 = (byte)(nx * color3.R + fx * color4.R);
nRed = (byte)(ny * (double)(bp1) + fy * (double)(bp2));
bmap.SetPixel(x, y, System.Drawing.Color.FromArgb
(255, nRed, nGreen, nBlue));
}
}
bmap = SetGrayscale(bmap);
bmap = RemoveNoise(bmap);
return bmap;
}
SetGrayscale
public Bitmap SetGrayscale(Bitmap img)
{
Bitmap temp = (Bitmap)img;
Bitmap bmap = (Bitmap)temp.Clone();
Color c;
for (int i = 0; i < bmap.Width; i++)
{
for (int j = 0; j < bmap.Height; j++)
{
c = bmap.GetPixel(i, j);
byte gray = (byte)(.299 * c.R + .587 * c.G + .114 * c.B);
bmap.SetPixel(i, j, Color.FromArgb(gray, gray, gray));
}
}
return (Bitmap)bmap.Clone();
}
RemoveNoise
public Bitmap RemoveNoise(Bitmap bmap)
{
for (var x = 0; x < bmap.Width; x++)
{
for (var y = 0; y < bmap.Height; y++)
{
var pixel = bmap.GetPixel(x, y);
if (pixel.R < 162 && pixel.G < 162 && pixel.B < 162)
bmap.SetPixel(x, y, Color.Black);
else if (pixel.R > 162 && pixel.G > 162 && pixel.B > 162)
bmap.SetPixel(x, y, Color.White);
}
}
return bmap;
}
INPUT IMAGE
OUTPUT IMAGE
This is somewhat ago but it still might be useful.
My experience shows that resizing the image in-memory before passing it to tesseract sometimes helps.
Try different modes of interpolation. The post https://stackoverflow.com/a/4756906/146003 helped me a lot.
What was EXTREMLY HELPFUL to me on this way are the source codes for Capture2Text project.
http://sourceforge.net/projects/capture2text/files/Capture2Text/.
BTW: Kudos to it's author for sharing such a painstaking algorithm.
Pay special attention to the file Capture2Text\SourceCode\leptonica_util\leptonica_util.c - that's the essence of image preprocession for this utility.
If you will run the binaries, you can check the image transformation before/after the process in Capture2Text\Output\ folder.
P.S. mentioned solution uses Tesseract for OCR and Leptonica for preprocessing.
Java version for Sathyaraj's code above:
// Resize
public Bitmap resize(Bitmap img, int newWidth, int newHeight) {
Bitmap bmap = img.copy(img.getConfig(), true);
double nWidthFactor = (double) img.getWidth() / (double) newWidth;
double nHeightFactor = (double) img.getHeight() / (double) newHeight;
double fx, fy, nx, ny;
int cx, cy, fr_x, fr_y;
int color1;
int color2;
int color3;
int color4;
byte nRed, nGreen, nBlue;
byte bp1, bp2;
for (int x = 0; x < bmap.getWidth(); ++x) {
for (int y = 0; y < bmap.getHeight(); ++y) {
fr_x = (int) Math.floor(x * nWidthFactor);
fr_y = (int) Math.floor(y * nHeightFactor);
cx = fr_x + 1;
if (cx >= img.getWidth())
cx = fr_x;
cy = fr_y + 1;
if (cy >= img.getHeight())
cy = fr_y;
fx = x * nWidthFactor - fr_x;
fy = y * nHeightFactor - fr_y;
nx = 1.0 - fx;
ny = 1.0 - fy;
color1 = img.getPixel(fr_x, fr_y);
color2 = img.getPixel(cx, fr_y);
color3 = img.getPixel(fr_x, cy);
color4 = img.getPixel(cx, cy);
// Blue
bp1 = (byte) (nx * Color.blue(color1) + fx * Color.blue(color2));
bp2 = (byte) (nx * Color.blue(color3) + fx * Color.blue(color4));
nBlue = (byte) (ny * (double) (bp1) + fy * (double) (bp2));
// Green
bp1 = (byte) (nx * Color.green(color1) + fx * Color.green(color2));
bp2 = (byte) (nx * Color.green(color3) + fx * Color.green(color4));
nGreen = (byte) (ny * (double) (bp1) + fy * (double) (bp2));
// Red
bp1 = (byte) (nx * Color.red(color1) + fx * Color.red(color2));
bp2 = (byte) (nx * Color.red(color3) + fx * Color.red(color4));
nRed = (byte) (ny * (double) (bp1) + fy * (double) (bp2));
bmap.setPixel(x, y, Color.argb(255, nRed, nGreen, nBlue));
}
}
bmap = setGrayscale(bmap);
bmap = removeNoise(bmap);
return bmap;
}
// SetGrayscale
private Bitmap setGrayscale(Bitmap img) {
Bitmap bmap = img.copy(img.getConfig(), true);
int c;
for (int i = 0; i < bmap.getWidth(); i++) {
for (int j = 0; j < bmap.getHeight(); j++) {
c = bmap.getPixel(i, j);
byte gray = (byte) (.299 * Color.red(c) + .587 * Color.green(c)
+ .114 * Color.blue(c));
bmap.setPixel(i, j, Color.argb(255, gray, gray, gray));
}
}
return bmap;
}
// RemoveNoise
private Bitmap removeNoise(Bitmap bmap) {
for (int x = 0; x < bmap.getWidth(); x++) {
for (int y = 0; y < bmap.getHeight(); y++) {
int pixel = bmap.getPixel(x, y);
if (Color.red(pixel) < 162 && Color.green(pixel) < 162 && Color.blue(pixel) < 162) {
bmap.setPixel(x, y, Color.BLACK);
}
}
}
for (int x = 0; x < bmap.getWidth(); x++) {
for (int y = 0; y < bmap.getHeight(); y++) {
int pixel = bmap.getPixel(x, y);
if (Color.red(pixel) > 162 && Color.green(pixel) > 162 && Color.blue(pixel) > 162) {
bmap.setPixel(x, y, Color.WHITE);
}
}
}
return bmap;
}
The Tesseract documentation contains some good details on how to improve the OCR quality via image processing steps.
To some degree, Tesseract automatically applies them. It is also possible to tell Tesseract to write an intermediate image for inspection, i.e. to check how well the internal image processing works (search for tessedit_write_images in the above reference).
More importantly, the new neural network system in Tesseract 4 yields much better OCR results - in general and especially for images with some noise. It is enabled with --oem 1, e.g. as in:
$ tesseract --oem 1 -l deu page.png result pdf
(this example selects the german language)
Thus, it makes sense to test first how far you get with the new Tesseract LSTM mode before applying some custom pre-processing image processing steps.
Adaptive thresholding is important if the lighting is uneven across the image.
My preprocessing using GraphicsMagic is mentioned in this post:
https://groups.google.com/forum/#!topic/tesseract-ocr/jONGSChLRv4
GraphicsMagic also has the -lat feature for Linear time Adaptive Threshold which I will try soon.
Another method of thresholding using OpenCV is described here:
https://docs.opencv.org/4.x/d7/d4d/tutorial_py_thresholding.html
I did these to get good results out of an image which has not very small text.
Apply blur to the original image.
Apply Adaptive Threshold.
Apply Sharpening effect.
And if the still not getting good results, scale the image to 150% or 200%.
Reading text from image documents using any OCR engine have many issues in order get good accuracy. There is no fixed solution to all the cases but here are a few things which should be considered to improve OCR results.
1) Presence of noise due to poor image quality / unwanted elements/blobs in the background region. This requires some pre-processing operations like noise removal which can be easily done using gaussian filter or normal median filter methods. These are also available in OpenCV.
2) Wrong orientation of image: Because of wrong orientation OCR engine fails to segment the lines and words in image correctly which gives the worst accuracy.
3) Presence of lines: While doing word or line segmentation OCR engine sometimes also tries to merge the words and lines together and thus processing wrong content and hence giving wrong results. There are other issues also but these are the basic ones.
This post OCR application is an example case where some image pre-preocessing and post processing on OCR result can be applied to get better OCR accuracy.
Text Recognition depends on a variety of factors to produce a good quality output. OCR output highly depends on the quality of input image. This is why every OCR engine provides guidelines regarding the quality of input image and its size. These guidelines help OCR engine to produce accurate results.
I have written a detailed article on image processing in python. Kindly follow the link below for more explanation. Also added the python source code to implement those process.
Please write a comment if you have a suggestion or better idea on this topic to improve it.
https://medium.com/cashify-engineering/improve-accuracy-of-ocr-using-image-preprocessing-8df29ec3a033
you can do noise reduction and then apply thresholding, but that you can you can play around with the configuration of the OCR by changing the --psm and --oem values
try:
--psm 5
--oem 2
you can also look at the following link for further details
here
So far, I've played a lot with tesseract 3.x, 4.x and 5.0.0.
tesseract 4.x and 5.x seem to yield the exact same accuracy.
Sometimes, I get better results with legacy engine (using --oem 0) and sometimes I get better results with LTSM engine --oem 1.
Generally speaking, I get the best results on upscaled images with LTSM engine. The latter is on par with my earlier engine (ABBYY CLI OCR 11 for Linux).
Of course, the traineddata needs to be downloaded from github, since most linux distros will only provide the fast versions.
The trained data that will work for both legacy and LTSM engines can be downloaded at https://github.com/tesseract-ocr/tessdata with some command like the following. Don't forget to download the OSD trained data too.
curl -L https://github.com/tesseract-ocr/tessdata/blob/main/eng.traineddata?raw=true -o /usr/share/tesseract/tessdata/eng.traineddata
curl -L https://github.com/tesseract-ocr/tessdata/blob/main/eng.traineddata?raw=true -o /usr/share/tesseract/tessdata/osd.traineddata
I've ended up using ImageMagick as my image preprocessor since it's convenient and can easily run scripted. You can install it with yum install ImageMagick or apt install imagemagick depending on your distro flavor.
So here's my oneliner preprocessor that fits most of the stuff I feed to my OCR:
convert my_document.jpg -units PixelsPerInch -respect-parenthesis \( -compress LZW -resample 300 -bordercolor black -border 1 -trim +repage -fill white -draw "color 0,0 floodfill" -alpha off -shave 1x1 \) \( -bordercolor black -border 2 -fill white -draw "color 0,0 floodfill" -alpha off -shave 0x1 -deskew 40 +repage \) -antialias -sharpen 0x3 preprocessed_my_document.tiff
Basically we:
use TIFF format since tesseract likes it more than JPG (decompressor related, who knows)
use lossless LZW TIFF compression
Resample the image to 300dpi
Use some black magic to remove unwanted colors
Try to rotate the page if rotation can be detected
Antialias the image
Sharpen text
The latter image can than be fed to tesseract with:
tesseract -l eng preprocessed_my_document.tiff - --oem 1 -psm 1
Btw, some years ago I wrote the 'poor man's OCR server' which checks for changed files in a given directory and launches OCR operations on all not already OCRed files. pmocr is compatible with tesseract 3.x-5.x and abbyyocr11.
See the pmocr project on github.

how to remove a stamp from an image with opencv

I am working on a OCR project, and in the preprocessing, some RED stamps need to be removed, so that the text near the stamps could be detected. I try a lot of methods(like change the values of pixel, threshold in Red channel) but fail.
Any suggestions are highly appreciated.
Python, C++, Java or what? Since you didn't state the OpenCV implementation you are using, I'm giving my answer in C++.
An option is to use the HSV color space to filter out the range of red values that defines the seal. My approach is to use the CMYK color space to filter everything except the black (or dark) text. It should do a pretty good job on printed media, which is your case.
//read input image:
std::string imageName = "C://opencvImages//seal.png";
cv::Mat imageInput = cv::imread( imageName );
Now, perform the CMYK conversion. OpenCV does not support this operation out of the box, bear with me as I provide the helper function at the end of this post.
//CMYK conversion:
std::vector<cv::Mat> cmyk;
cmyk = rgb2cmyk( imageInput );
//This is the Black channel:
cv::Mat blackChannel = cmyk[3].clone();
This is the image of the black channel; it is nice how everything that is not black (or dark) practically disappears!
Now, optionally, enhance the result applying brightness and contrast adjustment. Just try to separate the text from the background a little bit better; we want some defined pixel distributions to get a nice binary image.
//Brightness and contrast adjustment:
float alpha = 2.0;
float beta = -50.0;
contrastBrightnessAdjustment( blackChannel, alpha, beta );
Again, OpenCV does not offer brightness and contrast adjustment out of the box; however, its implementation is very easy. Hold on a little bit, and let me show you the result of this operation:
Nice. Let's Otsu-threshold this bad boy to get a nice binary image containing the clean text:
cv::threshold( blackChannel, binaryImage ,0, 255, cv::THRESH_OTSU );
This is what you get:
Now, the RGB to CMYK conversion function. I'm using the following implementation. The function receives an RGB image and returns a vector containing each of the CMYK channels
std::vector<cv::Mat> rgb2cmyk( cv::Mat& inputImage ){
std::vector<cv::Mat> cmyk;
for (int i = 0; i < 4; i++) {
cmyk.push_back( cv::Mat( inputImage.size(), CV_8UC1 ) );
}
std::vector<cv::Mat> inputRGB;
cv::split( inputImage, inputRGB );
for (int i = 0; i < inputImage.rows; i++)
{
for (int j = 0; j < inputImage.cols; j++)
{
float r = (int)inputRGB[2].at<uchar>(i, j) / 255.;
float g = (int)inputRGB[1].at<uchar>(i, j) / 255.;
float b = (int)inputRGB[0].at<uchar>(i, j) / 255.;
float k = std::min(std::min(1-r, 1-g), 1-b);
cmyk[0].at<uchar>(i, j) = (1 - r - k) / (1 - k) * 255.;
cmyk[1].at<uchar>(i, j) = (1 - g - k) / (1 - k) * 255.;
cmyk[2].at<uchar>(i, j) = (1 - b - k) / (1 - k) * 255.;
cmyk[3].at<uchar>(i, j) = k * 255.;
}
}
return cmyk;
}
And the contrastBrightnessAdjustment function is this, implemented using pointer arithmetic. The function receives a grayscale image and applies the linear transformation via the alpha and beta parameters:
void contrastBrightnessAdjustment( cv::Mat inputImage, float alpha, int beta ){
cv::MatIterator_<cv::Vec3b> it, end;
for (it = inputImage.begin<cv::Vec3b>(), end = inputImage.end<cv::Vec3b>(); it != end; ++it) {
uchar &pixel = (*it)[0];
pixel = cv::saturate_cast<uchar>(alpha*pixel+beta);
}
}

How to convert an image from CV_8UC1 to CV_32FC1 type in opencv?

I readed an image which type of it is CV_8UC1, and I want to convert it to CV_32FC1.but when I use convertTO() function my image become completely white and I don't know why!
Mat Image(512,512,CV_32FC1);
Image = imread("C:\\MGC.jpg",CV_LOAD_IMAGE_GRAYSCALE);
/// show image
namedWindow("pic");
int x = 0; int y = 0;
imshow("pic", Image);
cout<<Image.type()<<endl;
Image.convertTo(Image,CV_32FC1);
cout<<Image.type()<<endl;
////show converted image
namedWindow("pic1");
imshow("pic1",Image );
Because displayable range for 32FC type element images is [0:1] (for imshow).
Try this.
Image.convertTo(Image,CV_32FC1, 1.0/255.0);
The methods used by Andrey do not seem to exist in current (v3.4) opencv for python.
Here is an alternative solution that uses scikit-image
http://scikit-image.org/docs/dev/user_guide/data_types.html
import cv2
from skimage import img_as_float
cap = cv2.VideoCapture(0)
ret, frame = cap.read()
image1 = cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)
image2 = img_as_float(image1)
cv2.imshow('IMAGE1',image1)
cv2.imshow('IMAGE2', image2)
while(1):
k = cv2.waitKey(100) & 0xff
if k == 27:
break
cap.release()
cv2.destroyAllWindows()
It is a scaling problem. cvtColor is tricky in the ranges of the data type. Read this here.
If you change from CV_8U toCV32F and vs. the ranges does not scale by themselves, You must indicate the scaling measure.

Writing a float image in openCv with pixel values bigger than 1

I am currently working on a program which should take an LDR images and multiply certain pixel in the image, so that their pixel value would exceed the normal 0-255 (0-1) pixel value boundary. The program i have written can do so, but I am not able to write the image file, as the imwrite() in OpenCV clambs the values back in the range of 0-255 (0-1)
if they are bigger than 255.
Is there anybody there who knows how to write a floating point image with pixel values bigger than 255 (1)
My code looks like this
Mat ApplySunValue(Mat InputImg)
{
Mat Image1 = imread("/****/.jpg",CV_LOAD_IMAGE_COLOR);
Mat outPutImage;
Image1.convertTo(Image1, CV_32FC3);
for(int x = 0; x < InputImg.cols; x++){
for(int y = 0; y < InputImg.rows; y++){
float blue = Image1.at<Vec3f>(y,x)[0] /255.0f;
float green = Image1.at<Vec3f>(y,x)[1] /255.0f;
float red = Image1.at<Vec3f>(y,x)[2] /255.0f ;
Image1.at<Vec3f>(y,x)[0] = blue;
Image1.at<Vec3f>(y,x)[1] = green;
Image1.at<Vec3f>(y,x)[2] = red;
int pixelValue = InputImg.at<uchar>(y,x);
if(pixelValue > 254){
Image1.at<Vec3f>(y,x)[0] = blue * SunMultiplyer;
Image1.at<Vec3f>(y,x)[1] = green * SunMultiplyer;
Image1.at<Vec3f>(y,x)[2] = red * SunMultiplyer;
}
}
}
imwrite("/****/Nice.TIFF", Image1 * 255);
namedWindow("Hej",CV_WINDOW_AUTOSIZE);
imshow("hej", Image1);
return InputImg;
}
For storage purposes, the following is more memory efficient than the XML / YAML alternative (due to the use of a binary format):
// Save the image data in binary format
std::ofstream os(<filepath>,std::ios::out|std::ios::trunc|std::ios::binary);
os << (int)image.rows << " " << (int)image.cols << " " << (int)image.type() << " ";
os.write((char*)image.data,image.step.p[0]*image.rows);
os.close();
You can then load the image as follows:
// Load the image data from binary format
std::ifstream is(<filepath>,std::ios::in|std::ios::binary);
if(!is.is_open())
return false;
int rows,cols,type;
is >> rows; is.ignore(1);
is >> cols; is.ignore(1);
is >> type; is.ignore(1);
cv::Mat image;
image.create(rows,cols,type);
is.read((char*)image.data,image.step.p[0]*image.rows);
is.close();
For instance, without compression, a 1920x1200 floating-point three-channel image takes 26 MB when stored in binary format, whereas it takes 129 MB when stored in YML format. This size difference also has an impact on runtime since the number of accesses to the hard drive are very different.
Now, if what you want is to visualize your HDR image, you have no choice but to convert it to LDR. This is called "tone-mapping" (Wikipedia entry).
As far as I know, when opencv writes using imwrite, it writes in the format supported by the image container, and this by default is 255.
However, if you just want to save the data, you might consider writing the Mat object to an xml/yaml file.
//Writing
cv::FileStorage fs;
fs.open(filename, cv::FileStorage::WRITE);
fs<<"Nice"<<Image1;
//Reading
fs.open(filename, cv::FileStorage::READ);
fs["Nice"]>>Image1;
fs.release(); //Very Important

how to obtain a single channel value image from HSV image in opencv 2.1?

I am a beginner in opencv. I am using opencv v2.1. I have converted an RGB image to HSV image. Now I want to obtain single channels Hue, Value and Saturation separately. What should I do? I have seen similar questions here but No-one answered that. Kindly help.
You can access the same way you were accessing for RGB image where 1st channel will be for H, 2nd channel for S and 3rd channel for V.
If you are using OpenCV 2.1, you must be using IplImage then, right?
like if your HSV image is
IplImage *src.
IplImage* h = cvCreateImage( cvGetSize(src), IPL_DEPTH_8U, 1 );
IplImage* s = cvCreateImage( cvGetSize(src), IPL_DEPTH_8U, 1 );
IplImage* v = cvCreateImage( cvGetSize(src), IPL_DEPTH_8U, 1 );
// Split image onto the color planes
cvSplit( src, h, s, v, NULL );
cvSplit function splits a multichannel array into several single channels. Correct me if I am wrong.
I would recommend using OpenCV 2.4. It has structs like cvMat which are very easy to handle just like 2D arrays.
EDIT:
If you are using Mat then you can separate the channels out easily.
Let's say your hsv mat is Mat img_hsv.
Then :
vector<Mat> hsv_planes;
split( img_hsv, hsv_planes );
hsv_planes[0] // H channel
hsv_planes[1] // S channel
hsv_planes[2] // V channel
See if you can work out with this.
Solution for Python:
import cv2
from matplotlib import pyplot as plt
# Read image in BGR
img_path = "test.jpg"
img = cv2.imread(img_path)
# Convert BGR to HSV and parse HSV
hsv_img = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
h, s, v = hsv_img[:, :, 0], hsv_img[:, :, 1], hsv_img[:, :, 2]
# Plot result images
plt.imshow("Original", cv2.cvtColor(img, cv2.COLOR_BGR2RGB))
plt.imshow("HSV", hsv_img)
plt.imshow("H", h)
plt.imshow("S", s)
plt.imshow("V", v)
plt.show()
Here it is for a Mat:
cv::Mat hsv_image = ...;
std::vector<cv::Mat> hsv_channels;
cv::split(hsv_image, hsv_channels);
cv::Mat h_image = hsv_channels[0];
cv::Mat s_image = hsv_channels[1];
cv::Mat v_image = hsv_channels[2];

Resources