I tried many things to make the opencv_createsamples command work, no results. I try to create an OpenCV format training set using photos. I specify files names and rectangles coordinates in the descriptive info file and I always get this parse error. I simplified to one single example, still does not work. Num parameter is there. Windows 7 env.
My command:
C:\lib\opencv\build\x64\vc14\bin>opencv_createsamples -info C:\opencv_ws\info.txt -bg C:\opencv_ws\bg.txt -num 1 -w 360 -h 640 -vec platesCl.vec
My info file:
pos/test_img01.jpg 1 10 10 24 24
I tried using "\", tabs, absolute path, two lines with all ending characters, smaller img dimensions and still stuck.
Maybe there is something I am missing, I don't know - still, seems in accordance with the documentation...
Okay, actually, the opencv_createsamples.exe tool (Windows) does not like Windows End Of File character. I edited the info descriptor (positives samples) on Linux and fed it to the tool on Windows and it worked like a charm.
Related
I'm trying to read(preview) AI (adobe illustrator) file in my web application. my web app is on Linux machine and mainly uses Python.
I couldn't find any native python code that can preview AI file, so I continued to search for solution and found ghostscript, which gives the option to convert AI to JPG/PNG and I these format I have no problem previewing.
The issue I have is that I need the preview to include the whole document and not just the artboard, in illustrator it's possible when removing the checkbox from "use artboards" when saving, see screenshot: https://helpx.adobe.com/content/dam/help/en/illustrator/how-to/export-svg/_jcr_content/main-pars/image0/5286-export-svg-fig1.jpg
but when I try to export from ghostscript, I can't make it work...
from my understanding, it's best to try and first convert to EPS and then from that to JPG/PNG, but I failed doing that as well and the items that are outside the artboard are not showing.
on linux, these are the commands I basically tried , after installing ghostscript:
gs -dNOPAUSE -dBATCH -sDEVICE=eps2write -sOutputFile=out.eps input.ai
gs -dNOPAUSE -dBATCH -sDEVICE=jpeg -r300 -sOutputFile=out.jpeg input.ai
gs -dNOPAUSE -dBATCH -sDEVICE=pngalpha -r300 -sOutputFile=out.png input.ai
if it's not possible with ghostscript and I need imagemagick instead, I don't mind using it... I tried it for 10 minute and just got bunch of errors so I left it....
AI file for example: https://drive.google.com/open?id=1UgyLG_-nEUL5FLTtD3Dl281YVYzv0mUy
Jpeg example of the output I want: https://drive.google.com/open?id=1tLT2Uj1pp1gKRnJ8BojPZJxMFRn6LJoM
Thank you
Some updates on the topic: I've found this:
https://gist.github.com/moluapple/2059569
This is AI PGF extractor which should theoretically help to extract the additional data from the PDF. Currently, it seems quite old and written for win32, so I cannot test it at the moment, but it's at least some kind of lead.
Firstly, Adobe Illustrator native files are not technically supported by Ghostscript at all. They might work, because they are normally either PostScript or PDF files with custom bits that can be ignored for the purposes of drawing the content. But it's not a guarantee.
Secondly; no, do not multiply convert the files! That's a piece of cargo-cult mythology that's been doing the rounds for ages. There are sometimes reasons for doing so but in general this will simply magnify problems, not solve them. Really, don't do that.
You haven't quoted the errors you are getting and you haven't supplied any files to look at, so it's not really possible to tell what your problem is. I have no clue what an 'artboard' is, and a picture of the Illustrator dialog doesn't help.
Perhaps if you could supply an example file, and maybe a picture of what you expect, it might be possible to figure it out. My guess is that your '.ai' file is a PDF file, and that it has a MediaBox (which is what Ghostscript uses by default) and an ArtBox which is what you actually want to use. Or something like that. Hard to say without more information.
Edit
Well, I'm afraid the answer here is that you can't easily get what you want from that file without using Illustrator.
The file is a PDF file (if you rename input.ai to input.pdf then you can open it with a PDF reader). But Illustrator doesn't use most of the PDF file when it opens it. Instead the PDF file contains a '/PieceInfo' key, which is a key in the Page dictionary. That points to a dictionary which has a /Private key, which (finally!) points to a dictionary with a bunch of Illustrator stuff:
52 0 obj
<<
/AIMetaData 53 0 R
/AIPrivateData1 54 0 R
/AIPrivateData10 55 0 R
/AIPrivateData11 56 0 R
/AIPrivateData2 57 0 R
/AIPrivateData3 58 0 R
/AIPrivateData4 59 0 R
/AIPrivateData5 60 0 R
/AIPrivateData6 61 0 R
/AIPrivateData7 62 0 R
/AIPrivateData8 63 0 R
/AIPrivateData9 64 0 R
/ContainerVersion 11
/CreatorVersion 23
/NumBlock 11
/RoundtripStreamType 1
/RoundtripVersion 17
>>
endobj
That's the actual saved file format of the Illustrator file. You can think of the PDF file as a 'preview' wrapped around the Illustrator native file. Illustrator reads the PDF file to find its own data, then throws the PDF file away and uses the native file format stored within it instead.
The problem is that the PDF part of the file simply doesn't contain the content you want to see. That's stored in the Illustrator native data. Ghostscript just renders what's in the PDF file, it doesn't look at the Illustrator native file.
Looking at the Illustrator private data, some of it is uncompressed, but most is compressed, it doesn't say how it is compressed but applying the FlateDecode filter produces a good old-fashioned Illustrator PostScript file, one that will work with Ghostscript.
But you would have to manually parse the PDF file, extract all the compressed AIPrivateData streams, concatenate them together, apply the FlateDecode filter to decompress them, and only then send the resulting output to Ghostscript with the -dEPSCrop switch set. That will result in the output you want.
But neither Ghostscript nor ImageMagick (which generally uses Ghostscript to render PDF files) will do any of that for you, you would have to do it all yourself.
So, I have 20 positive samples and 500 negative samples. I created the .vec file using createsample utility.Now, when i try to train the classifier using the traincascade.exe utility, I run into the following error:
I have looked into many solutions given to people who have faced similar issues, but none of them worked.
Things I tried: 1. Increasing the negative sample size 2. Checking the path of the negative(or background images) stored in the Negative.txt file 3. Varying different parameters.
Here is some information regarding the path: My working directory has the following files: 1. Traincascade.exe 2. Positive image folder 3. NegativeImageFolder 4. vec file 5. Negative.txt (file that has path to images in the negative image folder)
My Negative.txt file has the absolute file path for the images in the negative image folder. I also tried changing the file path to the following format:
NegativeImageFolder\Image1.pgm
but didn't work! I tried both front and backslash too!
I have run out of ways to change the file path or make any modification to make this work!
First of all: is NumStages 1 and maxDepth 1 intentional?
Looking at Opencv's source code (cascadeclassifier.cpp, imagestorage.cpp), the error is thrown when in function
bool CvCascadeClassifier::updateTrainingSet( double& acceptanceRatio)
a number, negCount=500, of negative samples cannot be filled.
Before, everything was ok with positive samples (and the line about pos count that was printed on the screen is a proof of this).
Digging deep into source code negCount cannot be filled when imgReader.getNeg( img ) returns false, this means it cannot provide any image, which in turn happens when the list of source negatives is empty.
So you have to concentrate all your efforts in the direction of providing the algorithm with the correct list of negative images.
There are two ways to solve this: make sure that Negative.txt is read and all paths are regular and that every image in the list can be read regularly.
Is the file name “Negative.txt” or “Negatives.txt”?
Anyway with so few positive and negative samples you won’t train anything functioning, it is only useful to make you understand how the process of training works.
Well I was able to resolve the issue and run the train the classifier successfully. However, I am not 100% sure as to how the change I made helped.
This is what I did:
I was generating the Negative.txt file using Excel. I would enter the file path of one image and increment the image filename (since my images were name image1, image2, image3...). So the format as mentioned earlier would be :
C:\OpenCV-3.0.0\opencv\build\x64\vc12\bin\Negative\Image1.pgm
And finally save the file as a Unicode txt document. However, saving it as a unicode txt document gave me the error stated in the question. I saved it as a Text (tab delimited) file and it worked.
I am relatively new to machine learning/python/ubuntu.
I have a set of images in .jpg format where half contain a feature I want caffe to learn and half don't. I'm having trouble in finding a way to convert them to the required lmdb format.
I have the necessary text input files.
My question is can anyone provide a step by step guide on how to use convert_imageset.cpp in the ubuntu terminal?
Thanks
A quick guide to Caffe's convert_imageset
Build
First thing you must do is build caffe and caffe's tools (convert_imageset is one of these tools).
After installing caffe and makeing it make sure you ran make tools as well.
Verify that a binary file convert_imageset is created in $CAFFE_ROOT/build/tools.
Prepare your data
Images: put all images in a folder (I'll call it here /path/to/jpegs/).
Labels: create a text file (e.g., /path/to/labels/train.txt) with a line per input image . For example:
img_0000.jpeg 1
img_0001.jpeg 0
img_0002.jpeg 0
In this example the first image is labeled 1 while the other two are labeled 0.
Convert the dataset
Run the binary in shell
~$ GLOG_logtostderr=1 $CAFFE_ROOT/build/tools/convert_imageset \
--resize_height=200 --resize_width=200 --shuffle \
/path/to/jpegs/ \
/path/to/labels/train.txt \
/path/to/lmdb/train_lmdb
Command line explained:
GLOG_logtostderr flag is set to 1 before calling convert_imageset indicates the logging mechanism to redirect log messages to stderr.
--resize_height and --resize_width resize all input images to same size 200x200.
--shuffle randomly change the order of images and does not preserve the order in the /path/to/labels/train.txt file.
Following are the path to the images folder, the labels text file and the output name. Note that the output name should not exist prior to calling convert_imageset otherwise you'll get a scary error message.
Other flags that might be useful:
--backend - allows you to choose between an lmdb dataset or levelDB.
--gray - convert all images to gray scale.
--encoded and --encoded_type - keep image data in encoded (jpg/png) compressed form in the database.
--help - shows some help, see all relevant flags under Flags from tools/convert_imageset.cpp
You can check out $CAFFE_ROOT/examples/imagenet/convert_imagenet.sh
for an example how to use convert_imageset.
I'm using ImageMagick (with Wand in Python) to convert images and to get thumbnails from them. However, I noticed that I need to verify whether a file is an image or not ahead of time. Should I do this with Identify?
So I would assume checking the integrity of a file needs the whole file to be read into memory. Is it better to try and convert the file and if there was an error, then we know the file wasn't good.
seems like you answered your own question
$ ls -l *.png
-rw-r--r-- 1 jsp jsp 526254 Jul 20 12:10 image.png
-rw-r--r-- 1 jsp jsp 10000 Jul 20 12:12 image_with_error.png
$ identify image.png &> /dev/null; echo $?
0
$ identify image_with_error.png &> /dev/null; echo $?
0
$ convert image.png /dev/null &> /dev/null ; echo $?
0
$ convert image_with_error.png /dev/null &> /dev/null ; echo $?
1
if you specify the regard-warnings flag with the imagemagick identify tool
magick identify -regard-warnings myimage.jpg
it will throw an error if there are any warnings about the file. This is good for checking images, and seems to be a lot faster than using verbose.
I the case you use Python you can consider also the Pillow module.
In my experiments, I have used both the Pyhton Pillow module (PIL) and the Imagemagick wrapper Wand (for psd, xcf formats) in order to detect broken images, the original answer with code snippets is here.
Update:
I also implemented this solution in my Python script here on GitHub.
I also verified that damaged files (jpg) frequently are not 'broken' images i.e, a damaged picture file sometimes remains a legit picture file, the original image is lost or altered but you are still able to load it.
End Update
I quote the full answer for completeness:
You can use Python Pillow(PIL) module, with most image formats, to check if a file is a valid and intact image file.
In the case you aim at detecting also broken images, #Nadia Alramli correctly suggests the im.verify() method, but this does not detect all the possible image defects, e.g., im.verify does not detect truncated images (that most viewers often load with a greyed area).
Pillow is able to detect these type of defects too, but you have to apply image manipulation or image decode/recode in or to trigger the check. Finally I suggest to use this code:
try:
im = Image.load(filename)
im.verify() #I perform also verify, don't know if he sees other types o defects
im.close() #reload is necessary in my case
im = Image.load(filename)
im.transpose(PIL.Image.FLIP_LEFT_RIGHT)
im.close()
except:
#manage excetions here
In case of image defects this code will raise an exception.
Please consider that im.verify is about 100 times faster than performing the image manipulation (and I think that flip is one of the cheaper transformations).
With this code you are going to verify a set of images at about 10 MBytes/sec (using single thread of a modern 2.5Ghz x86_64 CPU).
For the other formats psd,xcf,.. you can use Imagemagick wrapper Wand, the code is as follows:
im = wand.image.Image(filename=filename)
temp = im.flip;
im.close()
But, from my experiments Wand does not detect truncated images, I think it loads lacking parts as greyed area without prompting.
I red that Imagemagick has an external command identify that could make the job, but I have not found a way to invoke that function programmatically and I have not tested this route.
I suggest to always perform a preliminary check, check the filesize to not be zero (or very small), is a very cheap idea:
statfile = os.stat(filename)
filesize = statfile.st_size
if filesize == 0:
#manage here the 'faulty image' case
Here's another solution using identify, but without convert:
identify -verbose *.png 2>&1 | grep "corrupt image"
identify: corrupt image 'image_with_error.png' # error/png.c/ReadPNGImage/4051.
i use identify:
$ identify image.tif
00000005.tif TIFF 4741x6981 4741x6981+0+0 8-bit DirectClass 4.471MB 0.000u 0:00.010
$ echo $?
My Delphi app has created a squence called frame_001.png to frame_100.png.
I need that to be compiled into a movie clip. I think perhaps the easiest is to call ffmpeg from the command line, according to their documentation:
For creating a video from many images:
ffmpeg -f image2 -i foo-%03d.jpeg -r 12 -s WxH foo.avi
The syntax foo-%03d.jpeg specifies to use a decimal number composed of three digits padded with zeroes to express the sequence number. It is the same syntax supported by the C printf function, but only formats accepting a normal integer are suitable.
From: http://ffmpeg.org/ffmpeg-doc.html#SEC5
However my files are (lossless) png format, so I have to convert using imagemagick first.
My command line is now:
ffmpeg.exe -f image2 -i c:\temp\wentelreader\frame_%05d.jpg -r 12 foo.avi
But then I get the error:
[image2 # 0x133a7d0]Could not find codec parameters (Video: mjpeg)
c:\temp\wentelreader\Frame_C:\VID2EVA\Tools\Mencoder\wentel.bat5d.jpg: could not
find codec parameters
What am I doing wrong?
Alternatively can this be done easily with Delphi?
Not sure if you are interested but there are delphi headers for this # http://ultrastardx.svn.sourceforge.net/viewvc/ultrastardx/trunk/src/lib/ffmpeg/
So you can use the DLL vs command line.
-Brad
Look at the file name in the error message. That can't possibly be right. The percent sign needs to get all the way to the program you're running, but it's being expanded by the batch file instead, where %0 expands to the full name and path of the file. Double the percent sign in the batch file:
ffmpeg.exe -f image2 -i c:\temp\wentelreader\frame_%%05d.jpg -r 12 foo.avi
Also, why do you want five digits when you've already said your files are named like frame_001.png, which has only three digits?
ffmpeg can create a movie from png images, why do you think you have to convert them to jpeg?
Guys in DelphiFFMpeg have been produced a component wrapper for FFMpeg. It's very expensive but it's worth to test it. However what you want to do is very simple and command-line is more than enough for you.