I'm training a simple convolution neural network using pylearn2. I have my RGB image data stored in a npy file. Is there anyway to convert that data directly to grayscale data directly from the npy file?
If this is a standalone file then load the file using numpy.load then convert the content using something like this:
def rgb2gray(rgb):
return np.dot(rgb[...,:3], [0.299, 0.587, 0.144])
If the file is part of a pylearn2 dataset (resulted from use_design_loc()), then load the dataset
from pylearn2.utils import serial
serial.load("file.pkl")
and apply rgb2gray() function to X member (I assume a DenseDesignMatrix).
Related
I have Physiological EEG emotion dataset named "Deap". I want to analyze and visualize the data through MNE but it has its own format.
How can I load my personal data for pre-processing, data format is (.dat)?
import pickle
with open('s01.dat', 'rb') as f:
y = pickle.load(f, encoding='latin1')
This one works for me.
Of course, the ".dat" file is in the same directory as this code.
I want to check the details of some Mat matrices in my OpenCV Codes (within Qt). An easy way, to my knowledge, to check the data matrix is to load it in Matlab. So, I want to save these data into a file that can be loaded in Matlab. Anyone has the experience to do so? A concrete example will be greatly helpful!!
OpenCV provides a straightforward example here using imwrite. And Matlab can then open the jpg files with imread.
The opencv Mat file can be saved to a csv file using cv::format() (writeCSV), which can be read in Matlab using csvread.m.
I would like to generate augmented data for images by Random rotation, shifts, shear and flips.
I have found this keras function.
The function keras.preprocessing.image.ImageDataGenerator But I've seen this being used to directly train networks.
Is there a way to input images and then save the transformed images on HDD instead of how if currently works in examples in this link
Or is there another simple plug and use python package I can use instead of implementing everything with numpy or opencv ?
Basically - this is generator which is infinitely returning a batches of images. One could do the following:
def save_images_from_generator(maximal_nb_of_images, generator):
nb_of_images_processed = 0
for x, _ in generator:
nb_of_images += x.shape[0]
if nb_of_images <= maximal_nb_of_images:
for image_nb in range(x.shape[0]):
your_custom_save(x[image_nb]) # your custom function for saving images
else:
break
to save images from keras image generator.
You can save the images outputted by ImageGenerator to HDD. One option is to use datagen.flow as follows:
for X_batch, y_batch in datagen.flow(X_train, y_train, batch_size=9, save_to_dir='images', save_prefix='aug', save_format='png')
A second option is to manually loop over each image, load it, and apply a random transformation. Once you have instantiated your ImageGenerator, just call:
img_trans = datagen.random_transform(img)
Then, save the transformed image to HDD using PIL etc.
A third option is to manually loop over each image, load it, and apply a random transformation using a third party program. I recommend imgaug, found here.
I'm preparing to train in Caffe using data in a hdf5 file. This file also contains the per-pixel mean data/image of the training set. In the file 'train_val.prototxt' for the input data layer in the section 'transform_params' it is possible to use a mean_file to normalize the data, usually in binaryproto format, for example for the ImageNet Caffe tutorial example:
transform_param {
mirror: true
crop_size: 227
mean_file: "data/ilsvrc12/imagenet_mean.binaryproto"
}
For per-channel normalization one can instead use mean_value instead of mean_file.
But is there any way to use mean image data directly from my database (here hdf5) file?
I have extracted the mean from the hdf5 to a numpy file but not sure if that can be used in the prototxt either or converted. I can't find info about this in the Caffe documentation.
AFAIK, "HDF5Data" layer does not support transformations. You should subtract the mean values yourself when you store the data to HDF5 files.
If you want to save a numpy array in a binaryproto format, you can see this answer for more details.
I have a large image file (single band) that do not fit in my ram.
I wan to read it as numpy array (data) and plot it using matplotlib, possibly using imshow(data). I know how to do it for a small-sized image. But how can I do it for large file? Ofcourse, its okay to resample (possibly scipy zoom) it before plotting. But how can I resample it before reading as numpy arrray because reading of large file into memory is not possible.
maybe it is better to display the tiff with an external viewer https://superuser.com/questions/254677/what-software-works-well-for-viewing-massive-tiff-images-on-windows-7 .
Otherwise you could try to convert the tiff in an HDF5 file first (ftp://ftp.hdfgroup.org/HDF/contrib/salem/tiffutils.c) , and then load only a part of the matrix you want to display.