I am trying to implement the transpose function in Caffe using a Python layer.
Below is the code for the same.
However, it is throwing the error "terminate called after throwing an instance of boost::python::error_already_set at Reshape() method.
Can someone throw some light on what I am doing wrong?
import caffe
import numpy as np
class transpose(caffe.Layer):
def setup(self, bottom, top):
assert len(bottom) == 1, 'requires a single layer.bottom'
assert bottom[0].data.ndim == 2, 'requires matrix data'
assert len(top) == 1, 'requires a single layer.top'
def reshape(self, bottom, top):
top[0].reshape((bottom[0].data.shape[1], bottom[0].data.shape[0]))
def forward(self, bottom, top):
top[0].data = np.transpose(bottom[0].data)
def backward(self, top, propagate_down, bottom):
pass
Thank you,
Vijetha.
I think you are reshapeing incorrectly.
Try:
def reshape(self, bottom, top):
top[0].reshape(bottom[0].data.shape[1], bottom[0].data.shape[0])
shape arguments for Reshape are not given as a tupple but rather as a separate arguments.
Related
torch.matmul doesn't seem to have an nn.Module wrapper to allow the standard forward hook registration by name. In this case, the matrix multiply happens in the middle of a forward() function. I suppose the intermediate result can be returned by forward() in addition to the final result, such as return x, mm_res. But what's a good way to collect these additional outputs?
What are the options for offloading torch.matmul outputs? TIA.
If your primary complaint is the fact that torch.matmul doesn't have a Module wrapper, how about just making one
class Matmul(nn.Module):
def forward(self, *args):
return torch.matmul(*args)
Now you can register a forward hook on a Matmul instance
class Network(nn.Module):
def __init__(self, ...):
self.matmul = Matmul()
self.matmul.register_module_forward_hook(...)
def forward(self, x):
y = ...
z = self.matmul(x, y)
...
Being said that, you must not overlook the warning (in red) in the doc that it should only be used for debugging purpose.
I have to add a k-max pooling layer in CNN model to detect fake reviews. Please can you let me know how to implement it using keras.
I searched the internet but I got no good resources.
As per this paper, k-Max Pooling is a pooling operation that is a generalisation of the max pooling over the time dimension used in the Max-TDNN sentence model
and different from the local max pooling operations applied in a convolutional network for object recognition (LeCun et al., 1998).
The k-max pooling operation makes it possible
to pool the k most active features in p that may be
a number of positions apart; it preserves the order
of the features, but is insensitive to their specific
positions.
There are few resources which show how to implement it in tensorflow or keras:
How to implement K-Max pooling in Tensorflow or Keras?
https://github.com/keras-team/keras/issues/373
New Pooling Layers For Varying-Length Convolutional Networks
Keras implementation of K-Max Pooling with TensorFlow Backend
There seems to be a solution here as #Anubhav_Singh suggested. This response got almost 5 times more thumbs up (24) than thumbs down (5) on the github keras issues link. I am just quoting it as-is here and let people try it out and say whether it worked for them or not.
Original author: arbackus
from keras.engine import Layer, InputSpec
from keras.layers import Flatten
import tensorflow as tf
class KMaxPooling(Layer):
"""
K-max pooling layer that extracts the k-highest activations from a sequence (2nd dimension).
TensorFlow backend.
"""
def __init__(self, k=1, **kwargs):
super().__init__(**kwargs)
self.input_spec = InputSpec(ndim=3)
self.k = k
def compute_output_shape(self, input_shape):
return (input_shape[0], (input_shape[2] * self.k))
def call(self, inputs):
# swap last two dimensions since top_k will be applied along the last dimension
shifted_input = tf.transpose(inputs, [0, 2, 1])
# extract top_k, returns two tensors [values, indices]
top_k = tf.nn.top_k(shifted_input, k=self.k, sorted=True, name=None)[0]
# return flattened output
return Flatten()(top_k)
Note: it was reported to be running very slow (though it worked for people).
Check this out. Not thoroughly tested but works fine for me. Let me know what you think. P.S. Latest tensorflow version.
tf.nn.top_k does not preserve the order of occurrence of values. So, that is the think that need to be worked upon
import tensorflow as tf
from tensorflow.keras import layers
class KMaxPooling(layers.Layer):
"""
K-max pooling layer that extracts the k-highest activations from a sequence (2nd dimension).
TensorFlow backend.
"""
def __init__(self, k=1, axis=1, **kwargs):
super(KMaxPooling, self).__init__(**kwargs)
self.input_spec = layers.InputSpec(ndim=3)
self.k = k
assert axis in [1,2], 'expected dimensions (samples, filters, convolved_values),\
cannot fold along samples dimension or axis not in list [1,2]'
self.axis = axis
# need to switch the axis with the last elemnet
# to perform transpose for tok k elements since top_k works in last axis
self.transpose_perm = [0,1,2] #default
self.transpose_perm[self.axis] = 2
self.transpose_perm[2] = self.axis
def compute_output_shape(self, input_shape):
input_shape_list = list(input_shape)
input_shape_list[self.axis] = self.k
return tuple(input_shape_list)
def call(self, x):
# swap sequence dimension to get top k elements along axis=1
transposed_for_topk = tf.transpose(x, perm=self.transpose_perm)
# extract top_k, returns two tensors [values, indices]
top_k_vals, top_k_indices = tf.math.top_k(transposed_for_topk,
k=self.k, sorted=True,
name=None)
# maintain the order of values as in the paper
# sort indices
sorted_top_k_ind = tf.sort(top_k_indices)
flatten_seq = tf.reshape(transposed_for_topk, (-1,))
shape_seq = tf.shape(transposed_for_topk)
len_seq = tf.shape(flatten_seq)[0]
indices_seq = tf.range(len_seq)
indices_seq = tf.reshape(indices_seq, shape_seq)
indices_gather = tf.gather(indices_seq, 0, axis=-1)
indices_sum = tf.expand_dims(indices_gather, axis=-1)
sorted_top_k_ind += indices_sum
k_max_out = tf.gather(flatten_seq, sorted_top_k_ind)
# return back to normal dimension but now sequence dimension has only k elements
# performing another transpose will get the tensor back to its original shape
# but will have k as its axis_1 size
transposed_back = tf.transpose(k_max_out, perm=self.transpose_perm)
return transposed_back
Here is my implementation of k-max pooling as explained in the comment of #Anubhav Singh above (the order of topk is preserved)
def test60_simple_test(a):
# swap last two dimensions since top_k will be applied along the last dimension
#shifted_input = tf.transpose(a) #[0, 2, 1]
# extract top_k, returns two tensors [values, indices]
res = tf.nn.top_k(a, k=3, sorted=True, name=None)
b = tf.sort(res[1],axis=0,direction='ASCENDING',name=None)
e=tf.gather(a,b)
#e=e[0:3]
return (e)
a = tf.constant([7, 2, 3, 9, 5], dtype = tf.float64)
print('*input:',a)
print('**output', test60_simple_test(a))
The result:
*input: tf.Tensor([7. 2. 3. 9. 5.], shape=(5,), dtype=float64)
**output tf.Tensor([7. 9. 5.], shape=(3,), dtype=float64)
Here is a Pytorch version implementation of k-max pooling:
import torch
def kmax_pooling(x, dim, k):
index = x.topk(k, dim = dim)[1].sort(dim = dim)[0]
return x.gather(dim, index)
Hope it would help.
I'm trying to create an unpooling layer using Keras with the TensorFlow backend. The unpooling operation that I am trying to implement is described in this paper. This is the same unpooling operation that SegNet uses.
Unpooling: In the convnet, the max pooling operation
is non-invertible, however we can obtain an approximate
inverse by recording the locations of the
maxima within each pooling region in a set of switch
variables. In the deconvnet, the unpooling operation
uses these switches to place the reconstructions from
the layer above into appropriate locations, preserving
the structure of the stimulus.
Most of my code is an adaptation of this implementation from an older version of Keras.
So far I have written a custom layer and it is performing the unpooling operation correctly. The problem I am running into is when Keras tries to compute the gradient during the back-prop stage, I get the error:
raise ValueError('An operation has `None` for gradient. '
ValueError: An operation has `None` for gradient. Please make sure that all of your ops have a gradient defined (i.e. are differentiable). Common ops without gradient: K.argmax, K.round, K.eval.
As far as I understand, this error is being caused because I must be using an operation that Keras doesn't know the derivative for.
Here is my code:
from keras import backend as K
from keras.engine.topology import Layer
from keras.layers import UpSampling2D
import numpy as np
import operator
class MyUnpooler(Layer):
# Initialisations
def __init__(self, pool_layer, pool_input, size=(2,2), **kwargs):
self.pool_layer = pool_layer
self.pool_input = pool_input
self.size = size
super(MyUnpooler, self).__init__(**kwargs)
# This method would be used to create weights
# I don't have any trainable weights but this must be implemented
def build(self, input_shape):
super(MyUnpooler, self).build(input_shape)
# This method is for the layers' logic
# x is always input to the layer
def call(self, x):
# Zeros for later
zeros = K.zeros_like(self.pool_input)
# Mask template
upsampled = UpSampling2D(size=self.size)(self.pool_layer)
upsampled_shape = upsampled.get_shape().as_list()[1:]
input_shape = self.pool_input.get_shape().as_list()[1:]
size_diff = map(operator.sub, input_shape, upsampled_shape)
unfiltered_mask = K.spatial_2d_padding(upsampled, padding=((0,size_diff[1]),(0,size_diff[2])))
# Create the mask
self.mask = K.tf.equal(self.pool_input, unfiltered_mask)
assert self.mask.get_shape().as_list() == self.pool_input.get_shape().as_list()
self.unpooled = K.tf.where(self.mask, self.pool_input, zeros)
return K.tf.to_float(self.unpooled)
def compute_output_shape(self, input_shape):
# input_shape is not actually the input shape we need...
# We need to be able to UpSample the layer to calculate the dimensions
upsampled = UpSampling2D(size=self.size)(self.pool_layer)
upsampled_shape = upsampled.get_shape().as_list()[1:]
inp_shape = self.pool_input.get_shape().as_list()[1:]
size_diff = map(operator.sub, inp_shape, upsampled_shape)
unf_mask = K.spatial_2d_padding(upsampled, padding=((0,size_diff[1]),(0,size_diff[2])))
return tuple(unf_mask.get_shape().as_list())
If there is a better way of doing the unpooling operation I am also open to completely disregarding my attempt so far.
If I want after each batch ends to update the output activation of a specific layer, How can I do that in Keras?
For example, the output shape of a specific layer= [9,1500], where 9 is the batch size and 1500 is the output features. How can I update this output after each batch ends ?
This is my model architecture:
from keras.models import Sequential,Model
from keras.layers import Input,Embedding,Conv1D,Multiply,Activation,MaxPooling1D,Dense,Flatten
vocab_size=467
outPut_dimention=8
sequence_length=429278
main_input = Input(shape=(sequence_length,), name='main_input')
embedding_layer=Embedding(vocab_size+1,outPut_dimention,input_length=sequence_length)(main_input)
one_D_conv_layer1=Conv1D(128,32,strides=32 , activation='sigmoid' )(embedding_layer)
one_D_conv_layer2=Conv1D(128,32,strides=32, name="conv1d")(embedding_layer)
merge=Multiply()([one_D_conv_layer1,one_D_conv_layer2])
max_pooling=MaxPooling1D(pool_size=400)(merge)
flat_layer=Flatten()(max_pooling)
fully_connected=Dense(128)(flat_layer)
main_output=Dense(9, activation='softmax')(fully_connected)
model=Model(inputs=[main_input], outputs=[main_output])
model.compile(loss='categorical_crossentropy',
optimizer="Nadam",
metrics=['accuracy'])
history1=model.fit_generator(generator=training_generator,
validation_data=validation_generator,
use_multiprocessing=True,
workers=6,
epochs=1,callbacks=[myCallBack])
What I want to do is that during the training process, I want to update the out put of the fully_connected layer at the end of each batch. For example, if the output of the fully_connected layer is a numpy array of shape (?, 128), where ? is the batch size, then I want to update that numpy array at the end of each batch.
I found on Keras official website link a way to define your costume regularization under the section "Developing new regularizers". Since I need to have an access to layer output (batch_size,n_nodes), then I need to define a custom activity_regularizer.
from keras import backend as K
class Regularizer(object):
"""Regularizer base class.
"""
def __call__(self, x):
return 0.
#classmethod
def from_config(cls, config):
return cls(**config)
class DeCovRegularizer(Regularizer):
# def set_layer(self, layer):
# self.layer = layer
def __call__(self, x):
# print(x.shape)
# x will be the output of the layer: (batch_size,n_nodes)
# means=[u1,u2,u3,u4,....] : where u1 is the (sum of node1 over the batch size)/batch_size
means=(K.sum(x,axis=0)/batch_size) #axis=0 to sum over the column not the rows
sub_mean=x-means # we subtract the mean from the nodes; for Example, node1-u1, node2-u2 ......
node_numbers=x.shape[1] # in our example it will be 128
# Constructing the C matrix; however it will be 1Dim=[c1_2,c1_3....c1_n,c2_1,c2_3......,c2_d.......]
C=K.variable(np.array([0])) # initialize C with 0 --> C=[0]
for i in range (0,node_numbers):
for j in range(0,node_numbers):
if(i !=j):
C_i_j=K.sum(sub_mean[:,i]*sub_mean[:,j],keepdims=True)
C=K.concatenate([C, C_i_j], axis=-1)
DecovLoss=K.sum(C*C)/0.5
return DecovLoss
# def get_config(self):
# return {"name": self.__class__.__name__,
# "p": self.l1}
in my model I added the Decov loss as following :
fully_connected=Dense(128,activity_regularizer=DeCovRegularizer())(flat_layer)
Just to make sure, during the training process the returned value by the call function will be added to the main loss (cross entropy), right? so that the Decov loss will be added automatically to the cross entropy loss, is that right? Am I implementing the DeCov loss correctly ?
I have similar images with similar kind of background noise.
Using OpenCV, is there a way to detect the area (detect contours) of the area covering the label.
Or at least , is it possible to detect "rough" bounding-box covering the label area?
I've tried to accomplish this task, main issue which probably won't allow to generalize this algorithm is how to choose proper contour. I have two values(contour lengths) 3108 and 2855. You may try to fetch all you photos(if they on similar distance from camera) and set threshold for desired contour between 3050 and 2750, but no guaranty that it will work. So this is how i remove background(full code):
import cv2
import numpy as np
image=cv2.imread('C:/Users/srlatch/Desktop/of8cA.png')
img = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
def clear_vertical(img, target):
for i in range(img.shape[1]):
for j in range(img.shape[0]):
if img[j][i]:
break
else:
target[j][i]=[0,0,0]
def clear_horizontal(img, target):
for i in range(img.shape[0]):
for j in range(img.shape[1]):
if img[i][j]:
break
else:
target[i][j]=[0,0,0]
def turn_off(img):
for i in range(img.shape[0]):
for j in range(img.shape[1]):
img[i][j]=0
def turn_on(img,result):
for i in result:
img[i[0][1]][i[0][0]]=255
def f(list):
max=[]
for i in list:
if len(i)>len(max):
max=i
return max
def rem(ls, thresh):
new_c=[]
for i in ls:
if len(i)>thresh:
new_c.append(i)
return new_c
def rn(ls,min,max):
ret=[]
for i in ls:
if len(i)<max and len(i)>min:
print(len(i))
ret.append(i)
return ret
#ret,tresh = cv2.threshold(img,40,255,cv2.THRESH_BINARY)
kernel = np.ones((2,2),np.uint8)
new=cv2.Canny(img,190,1)
dilated=cv2.dilate(new, kernel)
tresh,c,hr=cv2.findContours(dilated,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
c=rn(c, 2600, 4000)
turn_off(new)
turn_on(new,c[0])
clear_horizontal(new,image)
clear_vertical(new,image)
cv2.imwrite('result_image_end.png',image)
cv2.imshow('wnd',image)
cv2.waitKey(100)
I've tried different approaches, but this seems to work better than the others. I believe in opencv exist function which can substitute this clear_horizontally and vertically, but i can't remember it name. Hope it helps!