I am working on CNTK and got following error:
ValueError: 2 unbound Placeholder(s) 'Placeholder('keep', [#, *], [939]), Placeholder('keep', [#, *], [939])' found in the Function. All Placeholders of a Function must be bound (to a variable) before performing a Forward computation.
for i in range(10000):
a1,a2,tar=get_sample(minibatch_size,start)
start=start+int(minibatch_size)
if start>=int(0.8*float(len(lab)))-minibatch_size:
start=0
trainer.train_minibatch({P1: a1, P2: a2, target: tar})
P1 and P2 are defined as C.layers.Input(939)
I was able to figure out the problem in my case. I had to pass the model output instead of the model itself as a parameter to the trainer constructor.
model = cntk.layers.Sequential([l1,l2])
model_output = model(predictor)
Error:
trainer = cntk.train.trainer.Trainer(model,(loss,meas),[learner])
No Error:
trainer = cntk.train.trainer.Trainer(model_output,(loss,meas),[learner])
Related
Creating an LOOCV loop using the randomForest package. I have adapted the following code from this link (https://stats.stackexchange.com/questions/459293/loocv-in-caret-package-randomforest-example-not-unique-results) however I am unable to reproduce a successful code.
Here is the code that I am running but on the iris dataset.
irisdata <- iris[1:150,]
predictionsiris <- 1:150
for (k in 1:150){
set.seed(123)
predictioniris[k] <- predict(randomForest(Petal.Width ~ Sepal.Length, data = irisdata[-k], ntree = 10), newdata = irisdata[k,,drop=F])[2]
}
What I would expect to happen is for it to run the random forest model on all but one row and then use that one row to test the model.
However, when I run this code, I get the following error:
Error in h(simpleError(msg, call)) :
error in evaluating the argument 'object' in selecting a method for function 'predict': object 'Sepal.Length' not found
Any suggestions? I have been messing around with LOOCV code for the past two days including messing with code in this page (Compute Random Forest with a leave one ID out cross validation in R) and running the following:
iris %>%
mutate(ID = 1:516)
loocv <- NULL
for(i in iris$ID){
test[[i]] <- slice(iris, i)
train[[i]] <- slice(iris, i+1:516)
rf <- randomForest(Sepal.Length ~., data = train, ntree = 10, importance = TRUE)
loocv[[i]] <- predict(rf, newdata = test)
}
but I have had no success. Any help would be appreciated.
When I try to use NVIDIA apex.amp O1 to accelerate my training, it report an error in my code logits = einsum('b x y d, r d -> b x y r', q, rel_k):
RuntimeError: RuntimeErrorRuntimeErrorexpected scalar type Half but found Float: :
expected scalar type Half but found Float
It means that rel_kshould be torch.HalfTensor.
rel_k is defined as follow: self.rel_height = nn.Parameter(torch.randn(height * 2 - 1, dim_head) * scale)
But when I specify the type of rel_kto be torch.HalfTensor, it report an error that I should not specify dtype manually
RuntimeErrorRuntimeError: : Found param encoder.layers.0.blocks.0.attn.rel_pos_emb.rel_height with type torch.cuda.HalfTensor, expected torch.cuda.FloatTensor.
When using amp.initialize, you do not need to call .half() on your model
before passing it, no matter what optimization level you choose.Found param encoder.layers.0.blocks.0.attn.rel_pos_emb.rel_height with type torch.cuda.HalfTensor, expected torch.cuda.FloatTensor.
When using amp.initialize, you do not need to call .half() on your model
before passing it, no matter what optimization level you choose.
How should I do to use amp O1 correctly in my code?
def try_and_error(layers, activation):
model = Sequential()
for i, nodes in enumerate(layers):
if i==0:
model.add(Dense(nodes,input_dim=train_X.shape[1]))#input layers
model.add(Activation(activation)) #Activation layer
else:
model.add(Dense(nodes))# Hidden Layers
model.add(Activation(activation))#Activation Layers
model.add(Dense(1)) # output layer
model.compile(optimizer='adam', loss='binary_crossentropy',metrics=['accuracy'])
return model
layers=[[150], [160,100], [140,100,500]]
activations = ['sigmoid', 'relu']
param_grid = dict(layers=layers, activation=activations, batch_size=
[500,800,1000])
grid =
RandomizedSearchCV(
estimator=KerasClassifier(build_fn=try_and_error
,epochs=100,verbose=0),
param_distributions =param_grid)
grid_result= grid.fit(train_X,train_y)}
and this is the error encountered even I have tried this with gridsearchcv result still the same.
RuntimeError: Cannot clone object <keras.wrappers.scikit_learn.KerasClassifier object at 0x7f3d7959c390>, as the constructor either does not set or modifies parameter layers
Try replacing
layers=[[150], [160,100], [140,100,500]]
with
layers=[(150), (160,100), (140,100,500)]
change the layers to this :
layers=[(150,), (160,100), (140,100,500)]
Also , Dont forget to add (,) in (150,) otherwise it will throw an error like : TypeError: 'int' object is not iterable
This is because single tuple without comma(,) is treated as int.
I created a voting regressor from some regressors like
voting_regressor = VotingRegressor(estimators=[('xg',xgbregressor),('gb',gradient_boosting_regressor),('et',extra_trees_regressor),('rf',random_forest_regressor)])
voting_regressor.fit(X_train, y_train)
The regressor predicts well on the test set
y_pred = voting_regressor.predict(X_test)
but when I try to predict for a particular instance
voting_regressor.predict(X_test.iloc[0].values.reshape(1,-1))
it shows following error
ValueError: feature_names mismatch: ['yearpublished', 'minplayers', 'maxplayers', 'playingtime', 'minplaytime', 'maxplaytime', 'minage', 'users_rated', 'total_owners', 'total_traders', 'total_wanters', 'total_wishers', 'total_comments', 'total_weights', 'average_weight'] ['f0', 'f1', 'f2', 'f3', 'f4', 'f5', 'f6', 'f7', 'f8', 'f9', 'f10', 'f11', 'f12', 'f13', 'f14']
expected users_rated, total_wishers, yearpublished, maxplayers, maxplaytime, total_owners, total_weights, average_weight, minplaytime, total_wanters, total_traders, playingtime, minage, total_comments, minplayers in input data
training data did not have the following fields: f9, f3, f13, f0, f8, f4, f14, f5, f2, f6, f12, f11, f7, f10, f1
You are passing pandas.Series instead of pandas.DataFrame when using iloc, when names of columns are required as indicated by error.
If you want to return dataframe with one example you can wrap it with another list like this:
voting_regressor.predict(X_test.iloc[[0]])
This way names of columns are preserved
You could specify many examples as well simply by using [0, 1, 2, 3].
I have a graph as follows, where the input x has two paths to reach y. They are combined with a gModule that uses cMulTable. Now if I do gModule:backward(x,y), I get a table of two values. Do they correspond to the error derivative derived from the two paths?
But since path2 contains other nn layers, I suppose I need to derive the derivates in this path in a stepwise fashion. But why did I get a table of two values for dy/dx?
To make things clearer, code to test this is as follows:
input1 = nn.Identity()()
input2 = nn.Identity()()
score = nn.CAddTable()({nn.Linear(3, 5)(input1),nn.Linear(3, 5)(input2)})
g = nn.gModule({input1, input2}, {score}) #gModule
mlp = nn.Linear(3,3) #path2 layer
x = torch.rand(3,3)
x_p = mlp:forward(x)
result = g:forward({x,x_p})
error = torch.rand(result:size())
gradient1 = g:backward(x, error) #this is a table of 2 tensors
gradient2 = g:backward(x_p, error) #this is also a table of 2 tensors
So what is wrong with my steps?
P.S, perhaps I have found out the reason because g:backward({x,x_p}, error) results in the same table. So I guess the two values stand for dy/dx and dy/dx_p respectively.
I think you simply made a mistake constructing your gModule. gradInput of every nn.Module has to have exactly the same structure as its input - that is the way backprop works.
Here's an example how to create a module like yours using nngraph:
require 'torch'
require 'nn'
require 'nngraph'
function CreateModule(input_size)
local input = nn.Identity()() -- network input
local nn_module_1 = nn.Linear(input_size, 100)(input)
local nn_module_2 = nn.Linear(100, input_size)(nn_module_1)
local output = nn.CMulTable()({input, nn_module_2})
-- pack a graph into a convenient module with standard API (:forward(), :backward())
return nn.gModule({input}, {output})
end
input = torch.rand(30)
my_module = CreateModule(input:size(1))
output = my_module:forward(input)
criterion_err = torch.rand(output:size())
gradInput = my_module:backward(input, criterion_err)
print(gradInput)
UPDATE
As I said, gradInput of every nn.Module has to have exactly the same structure as its input. So, if you define your module as nn.gModule({input1, input2}, {score}), your gradOutput (the result of the backward pass) will be a table of gradients w.r.t. input1 and input2 which in your case are x and x_p.
The only question remains: why on Earth don't you get an error when call:
gradient1 = g:backward(x, error)
gradient2 = g:backward(x_p, error)
An exception must be raised because the first argument must be not a tensor but a table of two tensors. Well, most (perhaps all) of torch modules during calculating :backward(input, gradOutput) don't use input argument (they usually store a copy of input from the last :forward(input) call). In fact, this argument is so useless that modules don't even bother themselves to verify it.