I went through the mmdetection documentation to convert a pytorch model to onnx here link
All installations are correct and i'm using onnxruntime==1.8.1, custom operators for ONNX Runtime MMCV_WITH_OPS.
I'm using the configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.py for faster rcnn link and using R-5-FPN pretrained model link
I used this to convert the pretrained model to onnx and successfully saved an onnx file named fasterrcnn.onnx
python tools/deployment/pytorch2onnx.py \
configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.py \
checkpoints/faster_rcnn/faster_rcnn_r50_fpn_mstrain_3x_coco_20210524_110822-e10bd31c.pth \
--output-file checkpoints/faster_rcnn/fasterrcnn.onnx \
--input-img demo/demo.jpg \
--test-img tests/data/color.jpg \
--shape 608 608 \
--dynamic-export \
--cfg-options \
model.test_cfg.deploy_nms_pre=-1 \
I am using that onnx file to host the model in NVIDIA triton model server.
fasterrcnn_model | 1 | READY
The model summary of the onnx model from Triton is shown below
{
"name": "fasterrcnn_model",
"platform": "onnxruntime_onnx",
"backend": "onnxruntime",
"version_policy": {
"latest": {
"num_versions": 1
}
},
"max_batch_size": 1,
"input": [
{
"name": "input",
"data_type": "TYPE_FP32",
"dims": [
3,
-1,
-1
]
}
],
"output": [
{
"name": "labels",
"data_type": "TYPE_INT64",
"dims": [
-1
]
},
{
"name": "dets",
"data_type": "TYPE_FP32",
"dims": [
-1,
5
]
}
],
"batch_input": [],
"batch_output": [],
"optimization": {
"priority": "PRIORITY_DEFAULT",
"input_pinned_memory": {
"enable": true
},
"output_pinned_memory": {
"enable": true
},
"gather_kernel_buffer_threshold": 0,
"eager_batching": false
},
"instance_group": [
{
"name": "fasterrcnn_model",
"kind": "KIND_CPU",
"count": 1,
"gpus": [],
"profile": []
}
],
"default_model_filename": "model.onnx",
"cc_model_filenames": {},
"metric_tags": {},
"parameters": {},
"model_warmup": []
}
The summary outlines that the output has the categories "labels" and "dets"
After sending an inference request with a sample image to triton I am getting the following responses.
labels
[[ 0. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17.
18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35.
36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. 47. 48. 49. 50. 51. 52. 53.
54. 55. 56. 57. 58. 59. 60. 61. 62. 63. 64. 65. 66. 67. 68. 69. 70. 71.
72. 73. 74. 75. 76. 77. 78. 79. 0. 1. 2. 3. 4. 5. 6. 7. 8. 9.
10. 11. 12. 13. 14. 15. 16. 17. 18. 19.]]
dets
[[[-1.0000e+00 -1.0000e+00 -1.0000e+00 -1.0000e+00 0.0000e+00]
[-3.0000e+02 -3.0000e+02 -3.0000e+02 -3.0000e+02 0.0000e+00]
[-5.9900e+02 -5.9900e+02 -5.9900e+02 -5.9900e+02 0.0000e+00]
[-8.9800e+02 -8.9800e+02 -8.9800e+02 -8.9800e+02 0.0000e+00]
[-1.1970e+03 -1.1970e+03 -1.1970e+03 -1.1970e+03 0.0000e+00]
[-1.4960e+03 -1.4960e+03 -1.4960e+03 -1.4960e+03 0.0000e+00]
[-1.7950e+03 -1.7950e+03 -1.7950e+03 -1.7950e+03 0.0000e+00]
[-2.0940e+03 -2.0940e+03 -2.0940e+03 -2.0940e+03 0.0000e+00]
[-2.3930e+03 -2.3930e+03 -2.3930e+03 -2.3930e+03 0.0000e+00]
[-2.6920e+03 -2.6920e+03 -2.6920e+03 -2.6920e+03 0.0000e+00]
[-2.9910e+03 -2.9910e+03 -2.9910e+03 -2.9910e+03 0.0000e+00]
[-3.2900e+03 -3.2900e+03 -3.2900e+03 -3.2900e+03 0.0000e+00]
[-3.5890e+03 -3.5890e+03 -3.5890e+03 -3.5890e+03 0.0000e+00]
[-3.8880e+03 -3.8880e+03 -3.8880e+03 -3.8880e+03 0.0000e+00]
[-4.1870e+03 -4.1870e+03 -4.1870e+03 -4.1870e+03 0.0000e+00]
[-4.4860e+03 -4.4860e+03 -4.4860e+03 -4.4860e+03 0.0000e+00]
[-4.7850e+03 -4.7850e+03 -4.7850e+03 -4.7850e+03 0.0000e+00]
[-5.0840e+03 -5.0840e+03 -5.0840e+03 -5.0840e+03 0.0000e+00]
[-5.3830e+03 -5.3830e+03 -5.3830e+03 -5.3830e+03 0.0000e+00]
[-5.6820e+03 -5.6820e+03 -5.6820e+03 -5.6820e+03 0.0000e+00]
[-5.9810e+03 -5.9810e+03 -5.9810e+03 -5.9810e+03 0.0000e+00]
[-6.2800e+03 -6.2800e+03 -6.2800e+03 -6.2800e+03 0.0000e+00]
[-6.5790e+03 -6.5790e+03 -6.5790e+03 -6.5790e+03 0.0000e+00]
[-6.8780e+03 -6.8780e+03 -6.8780e+03 -6.8780e+03 0.0000e+00]
[-7.1770e+03 -7.1770e+03 -7.1770e+03 -7.1770e+03 0.0000e+00]
[-7.4760e+03 -7.4760e+03 -7.4760e+03 -7.4760e+03 0.0000e+00]
[-7.7750e+03 -7.7750e+03 -7.7750e+03 -7.7750e+03 0.0000e+00]
[-8.0740e+03 -8.0740e+03 -8.0740e+03 -8.0740e+03 0.0000e+00]
[-8.3730e+03 -8.3730e+03 -8.3730e+03 -8.3730e+03 0.0000e+00]
[-8.6720e+03 -8.6720e+03 -8.6720e+03 -8.6720e+03 0.0000e+00]
[-8.9710e+03 -8.9710e+03 -8.9710e+03 -8.9710e+03 0.0000e+00]
[-9.2700e+03 -9.2700e+03 -9.2700e+03 -9.2700e+03 0.0000e+00]
[-9.5690e+03 -9.5690e+03 -9.5690e+03 -9.5690e+03 0.0000e+00]
[-9.8680e+03 -9.8680e+03 -9.8680e+03 -9.8680e+03 0.0000e+00]
[-1.0167e+04 -1.0167e+04 -1.0167e+04 -1.0167e+04 0.0000e+00]
[-1.0466e+04 -1.0466e+04 -1.0466e+04 -1.0466e+04 0.0000e+00]
[-1.0765e+04 -1.0765e+04 -1.0765e+04 -1.0765e+04 0.0000e+00]
[-1.1064e+04 -1.1064e+04 -1.1064e+04 -1.1064e+04 0.0000e+00]
[-1.1363e+04 -1.1363e+04 -1.1363e+04 -1.1363e+04 0.0000e+00]
[-1.1662e+04 -1.1662e+04 -1.1662e+04 -1.1662e+04 0.0000e+00]
[-1.1961e+04 -1.1961e+04 -1.1961e+04 -1.1961e+04 0.0000e+00]
[-1.2260e+04 -1.2260e+04 -1.2260e+04 -1.2260e+04 0.0000e+00]
[-1.2559e+04 -1.2559e+04 -1.2559e+04 -1.2559e+04 0.0000e+00]
[-1.2858e+04 -1.2858e+04 -1.2858e+04 -1.2858e+04 0.0000e+00]
[-1.3157e+04 -1.3157e+04 -1.3157e+04 -1.3157e+04 0.0000e+00]
[-1.3456e+04 -1.3456e+04 -1.3456e+04 -1.3456e+04 0.0000e+00]
[-1.3755e+04 -1.3755e+04 -1.3755e+04 -1.3755e+04 0.0000e+00]
[-1.4054e+04 -1.4054e+04 -1.4054e+04 -1.4054e+04 0.0000e+00]
[-1.4353e+04 -1.4353e+04 -1.4353e+04 -1.4353e+04 0.0000e+00]
[-1.4652e+04 -1.4652e+04 -1.4652e+04 -1.4652e+04 0.0000e+00]
[-1.4951e+04 -1.4951e+04 -1.4951e+04 -1.4951e+04 0.0000e+00]
[-1.5250e+04 -1.5250e+04 -1.5250e+04 -1.5250e+04 0.0000e+00]
[-1.5549e+04 -1.5549e+04 -1.5549e+04 -1.5549e+04 0.0000e+00]
[-1.5848e+04 -1.5848e+04 -1.5848e+04 -1.5848e+04 0.0000e+00]
[-1.6147e+04 -1.6147e+04 -1.6147e+04 -1.6147e+04 0.0000e+00]
[-1.6446e+04 -1.6446e+04 -1.6446e+04 -1.6446e+04 0.0000e+00]
[-1.6745e+04 -1.6745e+04 -1.6745e+04 -1.6745e+04 0.0000e+00]
[-1.7044e+04 -1.7044e+04 -1.7044e+04 -1.7044e+04 0.0000e+00]
[-1.7343e+04 -1.7343e+04 -1.7343e+04 -1.7343e+04 0.0000e+00]
[-1.7642e+04 -1.7642e+04 -1.7642e+04 -1.7642e+04 0.0000e+00]
[-1.7941e+04 -1.7941e+04 -1.7941e+04 -1.7941e+04 0.0000e+00]
[-1.8240e+04 -1.8240e+04 -1.8240e+04 -1.8240e+04 0.0000e+00]
[-1.8539e+04 -1.8539e+04 -1.8539e+04 -1.8539e+04 0.0000e+00]
[-1.8838e+04 -1.8838e+04 -1.8838e+04 -1.8838e+04 0.0000e+00]
[-1.9137e+04 -1.9137e+04 -1.9137e+04 -1.9137e+04 0.0000e+00]
[-1.9436e+04 -1.9436e+04 -1.9436e+04 -1.9436e+04 0.0000e+00]
[-1.9735e+04 -1.9735e+04 -1.9735e+04 -1.9735e+04 0.0000e+00]
[-2.0034e+04 -2.0034e+04 -2.0034e+04 -2.0034e+04 0.0000e+00]
[-2.0333e+04 -2.0333e+04 -2.0333e+04 -2.0333e+04 0.0000e+00]
[-2.0632e+04 -2.0632e+04 -2.0632e+04 -2.0632e+04 0.0000e+00]
[-2.0931e+04 -2.0931e+04 -2.0931e+04 -2.0931e+04 0.0000e+00]
[-2.1230e+04 -2.1230e+04 -2.1230e+04 -2.1230e+04 0.0000e+00]
[-2.1529e+04 -2.1529e+04 -2.1529e+04 -2.1529e+04 0.0000e+00]
[-2.1828e+04 -2.1828e+04 -2.1828e+04 -2.1828e+04 0.0000e+00]
[-2.2127e+04 -2.2127e+04 -2.2127e+04 -2.2127e+04 0.0000e+00]
[-2.2426e+04 -2.2426e+04 -2.2426e+04 -2.2426e+04 0.0000e+00]
[-2.2725e+04 -2.2725e+04 -2.2725e+04 -2.2725e+04 0.0000e+00]
[-2.3024e+04 -2.3024e+04 -2.3024e+04 -2.3024e+04 0.0000e+00]
[-2.3323e+04 -2.3323e+04 -2.3323e+04 -2.3323e+04 0.0000e+00]
[-2.3622e+04 -2.3622e+04 -2.3622e+04 -2.3622e+04 0.0000e+00]
[-1.0000e+00 -1.0000e+00 -1.0000e+00 -1.0000e+00 0.0000e+00]
[-3.0000e+02 -3.0000e+02 -3.0000e+02 -3.0000e+02 0.0000e+00]
[-5.9900e+02 -5.9900e+02 -5.9900e+02 -5.9900e+02 0.0000e+00]
[-8.9800e+02 -8.9800e+02 -8.9800e+02 -8.9800e+02 0.0000e+00]
[-1.1970e+03 -1.1970e+03 -1.1970e+03 -1.1970e+03 0.0000e+00]
[-1.4960e+03 -1.4960e+03 -1.4960e+03 -1.4960e+03 0.0000e+00]
[-1.7950e+03 -1.7950e+03 -1.7950e+03 -1.7950e+03 0.0000e+00]
[-2.0940e+03 -2.0940e+03 -2.0940e+03 -2.0940e+03 0.0000e+00]
[-2.3930e+03 -2.3930e+03 -2.3930e+03 -2.3930e+03 0.0000e+00]
[-2.6920e+03 -2.6920e+03 -2.6920e+03 -2.6920e+03 0.0000e+00]
[-2.9910e+03 -2.9910e+03 -2.9910e+03 -2.9910e+03 0.0000e+00]
[-3.2900e+03 -3.2900e+03 -3.2900e+03 -3.2900e+03 0.0000e+00]
[-3.5890e+03 -3.5890e+03 -3.5890e+03 -3.5890e+03 0.0000e+00]
[-3.8880e+03 -3.8880e+03 -3.8880e+03 -3.8880e+03 0.0000e+00]
[-4.1870e+03 -4.1870e+03 -4.1870e+03 -4.1870e+03 0.0000e+00]
[-4.4860e+03 -4.4860e+03 -4.4860e+03 -4.4860e+03 0.0000e+00]
[-4.7850e+03 -4.7850e+03 -4.7850e+03 -4.7850e+03 0.0000e+00]
[-5.0840e+03 -5.0840e+03 -5.0840e+03 -5.0840e+03 0.0000e+00]
[-5.3830e+03 -5.3830e+03 -5.3830e+03 -5.3830e+03 0.0000e+00]
[-5.6820e+03 -5.6820e+03 -5.6820e+03 -5.6820e+03 0.0000e+00]]]
The labels response looks like regular COCO classes (80) but I'm having a hard time decoding the dets response. Which looks like bounding boxes coordinates 4 and confidence threshold 1. Making the shape (1,100,5). Any idea on what the dets category is supposed to represent? The output usually depends on the model itself but I think the onnx conversion is changing the output to say labels and dets
Looking at the conversion script seems like dets is a combo of boxes plus score
boxes (Tensor): The bounding boxes of shape [N, num_boxes, 4]
scores (Tensor): The detection scores of shape [N, num_boxes, num_classes]
tuple[Tensor, Tensor]: dets of shape [N, num_det, 5]
and class labels of shape [N, num_det].
dets = torch.cat([boxes, scores], dim=2)
https://github.com/open-mmlab/mmdetection/blob/master/mmdet/core/export/onnx_helper.py#L197
Related
Closed. This question is not about programming or software development. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 6 months ago.
Improve this question
Edit: please share comments as I'm learning to post good questions
I'm trying to train this dataset with IsolationForest(), I need to train this dataset, and use it in another dataset with altered qualities to predict the quality values and fetch all wines with quality 8 and 9.
However I'm having some problems with it. Because the accuracy score is 0.0 from the classification report:
print(classification_report(y_test, prediction))
precision recall f1-score support
-1 0.00 0.00 0.00 0.0
1 0.00 0.00 0.00 0.0
3 0.00 0.00 0.00 866.0
4 0.00 0.00 0.00 829.0
5 0.00 0.00 0.00 841.0
6 0.00 0.00 0.00 861.0
7 0.00 0.00 0.00 822.0
8 0.00 0.00 0.00 886.0
9 0.00 0.00 0.00 851.0
accuracy 0.00 5956.0
macro avg 0.00 0.00 0.00 5956.0
weighted avg 0.00 0.00 0.00 5956.0
I don't know if it's a hyperparameter issue, or if I'm clearing the wrong data or putting wrong parameters, I already tried to use with SMOTE and without SMOTE, I wanted to reach an accuracy of 90% at least.
I'll leave the shared drive link public for dataset verification::
https://drive.google.com/drive/folders/18_sOSIZZw9DCW7ftEKuOG4aIzGXoasFe?usp=sharing
Here's my code:
from sklearn.preprocessing import OrdinalEncoder
from sklearn.ensemble import IsolationForest
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from imblearn.over_sampling import SMOTE
from sklearn.metrics import classification_report,confusion_matrix
df = pd.read_csv('wines.csv')
df.head(5)
ordinalEncoder = OrdinalEncoder(handle_unknown='use_encoded_value', unknown_value=-99).fit(df[['color']])
df[['color']] = ordinalEncoder.transform(df[['color']])
df.info()
df['color'] = df['color'].astype(int)
df.head(3)
stm = SMOTE(k_neighbors=4)
x_smote = df.drop('quality',axis=1)
y_smote = df['quality']
x_smote,y_smote = stm.fit_resample(x_smote,y_smote)
print(x_smote.shape,y_smote.shape)
x_smote.columns
scaler = StandardScaler()
X = scaler.fit_transform(x_smote)
y = y_smote
X.shape, y.shape
x_train, x_test, y_train, y_test = train_test_split(X,y,test_size=0.3)
from sklearn.ensemble import IsolationForest
from sklearn.metrics import hamming_loss
iforest = IsolationForest(n_estimators=200, max_samples=0.1, contamination=0.10, max_features=1.0, bootstrap=False, n_jobs=-1,
random_state=None, verbose=0, warm_start=False)
iforest_fit = iforest.fit(x_train,y_train)
prediction = iforest_fit.predict(x_test)
print (prediction.shape, y_test.shape)
y.value_counts()
prediction
print(confusion_matrix(y_test, prediction))
hamming_loss(y_test, prediction)
from sklearn.metrics import classification_report
print(classification_report(y_test, prediction))
May I know why do you choose Isolation Forest as your model? This article says that Isolation Forest is an unsupervised learning algorithm for anomaly detection.
When I print some samples of the prediction (by Isolation Forest) and samples of actual truth, I get the following results, so you know why the accuracy score is 0.0:
print(list(prediction[0:15]))
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
print(list(y_test[0:15]))
[9, 4, 4, 7, 9, 3, 6, 7, 4, 8, 8, 7, 3, 8, 5]
The wines.csv dataset and your code are both pointing towards a multi-class classification problem. I have chosen RandomForestClassifier() to continue with the second part of your code:
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import hamming_loss
model = RandomForestClassifier()
model.fit(x_train,y_train)
prediction = model.predict(x_test)
print(prediction[0:15]) #see 15 samples of prediction
[3, 9, 5, 5, 7, 9, 7, 6, 9, 8, 5, 9, 8, 3, 3]
print(list(y_test[0:15])) #see 15 samples of actual truth
[3, 9, 5, 6, 6, 9, 7, 5, 9, 8, 5, 9, 8, 3, 3]
print(confusion_matrix(y_test, prediction))
[[842 0 0 0 0 0 0]
[ 2 815 17 8 1 1 0]
[ 8 50 690 130 26 2 0]
[ 2 28 152 531 128 16 0]
[ 4 1 15 66 716 32 3]
[ 0 1 0 4 12 833 0]
[ 0 0 0 0 0 0 820]]
print('hamming_loss =', hamming_loss(y_test, prediction))
hamming_loss = 0.11903962390866353
print(classification_report(y_test, prediction))
precision recall f1-score support
3 0.98 1.00 0.99 842
4 0.91 0.97 0.94 844
5 0.79 0.76 0.78 906
6 0.72 0.62 0.67 857
7 0.81 0.86 0.83 837
8 0.94 0.98 0.96 850
9 1.00 1.00 1.00 820
accuracy 0.88 5956
macro avg 0.88 0.88 0.88 5956
weighted avg 0.88 0.88 0.88 5956
The accuracy is already 0.88 even before tuning hyperparameters.
I created a CNN model for predicting fashions using the mnist fashion dataset. After the model has been trained, I tried predicting one of the test images that are loaded from Keras and another image that is identical but imported from my PC onto my Google Colab notebook, and it turns out, the prediction results are not the same. How can I solve this problem?
This is how I imported the dataset:
import tensorflow as tf
from tensorflow import keras
fashion_mnist = keras.datasets.fashion_mnist
(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()
Data manipulation:
from keras.utils import to_categorical
yTest = to_categorical(y_test)
yTrain = to_categorical(y_train)
xTrain = x_train.reshape((60000, 28, 28, 1))
xTest = x_test.reshape(10000, 28, 28, 1)
Model Setup:
from keras.layers import Dense, Flatten, Conv2D, Dropout, MaxPool2D, BatchNormalization
from keras.callbacks import ModelCheckpoint
model = keras.Sequential()
#Adding the convolutional layer
model.add(Conv2D(50, kernel_size=3, activation='relu',padding = 'same', input_shape = (28, 28, 1)))
model.add(MaxPool2D(pool_size = (2, 2), strides = 1, padding = 'valid'))
model.add(Dropout(0.5))
model.add(Conv2D(40, kernel_size = 3, activation = 'relu', padding = 'same'))
model.add(MaxPool2D(pool_size = (2, 2), strides = 1, padding = 'valid'))
model.add(Dropout(0.5))
model.add(Conv2D(30, kernel_size = 3, activation = 'relu', padding = 'same'))
model.add(MaxPool2D(pool_size = (2, 2), strides = 2, padding = 'valid'))
model.add(Dropout(0.5))
model.add(Conv2D(10, kernel_size = 3, activation = 'relu', padding = 'same'))
model.add(Dropout(0.5))
#Connecting the CNN layers to the ANN
model.add(Flatten())
model.add(Dense(60, activation='relu'))
model.add(Dense(40, activation='relu'))
model.add(Dense(40, activation = 'relu'))
model.add(Dense(10, activation = 'softmax'))
model.load_weights('mnist_fashion.h5')
# Compiling the model
opt = tf.keras.optimizers.Adam(learning_rate=0.0001)
model.compile(optimizer=opt, loss = 'categorical_crossentropy', metrics = ['accuracy']
The model for training:
model = keras.Sequential()
#Adding the convolutional layer
model.add(Conv2D(50, kernel_size=3, activation='relu',padding = 'same', input_shape = (28, 28, 1)))
model.add(MaxPool2D(pool_size = (2, 2), strides = 1, padding = 'valid'))
model.add(Dropout(0.5))
model.add(Conv2D(40, kernel_size = 3, activation = 'relu', padding = 'same'))
model.add(MaxPool2D(pool_size = (2, 2), strides = 1, padding = 'valid'))
model.add(Dropout(0.5))
model.add(Conv2D(30, kernel_size = 3, activation = 'relu', padding = 'same'))
model.add(MaxPool2D(pool_size = (2, 2), strides = 2, padding = 'valid'))
model.add(Dropout(0.5))
model.add(Conv2D(10, kernel_size = 3, activation = 'relu', padding = 'same'))
model.add(Dropout(0.5))
#Connecting the CNN layers to the ANN
model.add(Flatten())
model.add(Dense(60, activation='relu'))
model.add(Dense(40, activation='relu'))
model.add(Dense(40, activation = 'relu'))
model.add(Dense(10, activation = 'softmax'))
The model's Performance:
precision recall f1-score support
0 0.89 0.88 0.88 1000
1 0.99 0.99 0.99 1000
2 0.88 0.89 0.89 1000
3 0.93 0.93 0.93 1000
4 0.87 0.89 0.88 1000
5 0.99 0.98 0.99 1000
6 0.79 0.78 0.78 1000
7 0.97 0.98 0.97 1000
8 0.99 0.98 0.99 1000
9 0.97 0.97 0.97 1000
accuracy 0.93 10000
macro avg 0.93 0.93 0.93 10000
weighted avg 0.93 0.93 0.93 10000
Picture from dataset prediction
#From the dataset
import numpy as np
image = xTrain[0].reshape(1, 28, 28, 1)
prd = model.predict(image)
new_prd = np.argmax(prd, axis = 1)
print(f"Prediction = {new_prd}")
print(f"Full Prediction = {prd}")
print(f"Label = {y_train[0]}")
Dataset Result
Prediction = [9]
Full Prediction = [[1.6268513e-07 2.3548612e-08 1.5456487e-07 8.6898848e-07 1.9692785e-09
4.4544859e-04 6.6932116e-06 1.4004705e-02 4.1784686e-05 9.8550016e-01]]
Label = 9
Imported picture prediction
imported_img = plt.imread("mnist fashion sample.png")
yolo = imported_img.reshape(1, 28, 28, 1)
super_prd = model.predict(yolo)
prediction = np.argmax(super_prd, axis = 1)
print(f"Prediction = {prediction}")
print(f"Full Prediction = {super_prd}")
print(f"Label = {y_train[0]}")
Imported picture prediction result
Prediction = [8]
Full Prediction = [[2.49403762e-04 1.69450897e-04 4.47237398e-04 3.05729372e-05
1.10463676e-04 4.34053177e-03 5.16198808e-04 8.16224664e-02
8.73587310e-01 3.89263593e-02]]
Label = 9
I solved the problem!
What I did wrong was that I did not normalize the pictures before training. This may cause an error because the data pixel range can be too complex for the relu activation function to calculate or predict.
Thank you!!!
how can I get the classification report measures precision, recall, accuracy, and support for 3 class classification and the classes are "positive", "negative" and "neutral". below is the code:
vec_clf = Pipeline([('vectorizer', vec), ('pac', svm_clf)])
print vec_clf.fit(X_train.values.astype('U'),y_train.values.astype('U'))
y_pred = vec_clf.predict(X_test.values.astype('U'))
print "SVM Accuracy-",metrics.accuracy_score(y_test, y_pred)
print "confuson metrics :\n", metrics.confusion_matrix(y_test, y_pred, labels=["positive","negative","neutral"])
print(metrics.classification_report(y_test, y_pred))
and it is giving error as:
SVM Accuracy- 0.850318471338
confuson metrics :
[[206 9 67]
[ 4 373 122]
[ 9 21 756]]
Traceback (most recent call last):
File "<ipython-input-62-e6ab3066790e>", line 1, in <module>
runfile('C:/Users/HP/abc16.py', wdir='C:/Users/HP')
File "C:\ProgramData\Anaconda2\lib\site-packages\spyder\utils\site\sitecustomize.py", line 880, in runfile
execfile(filename, namespace)
File "C:\ProgramData\Anaconda2\lib\site-packages\spyder\utils\site\sitecustomize.py", line 87, in execfile
exec(compile(scripttext, filename, 'exec'), glob, loc)
File "C:/Users/HP/abc16.py", line 133, in <module>
print(metrics.classification_report(y_test, y_pred))
File "C:\ProgramData\Anaconda2\lib\site-packages\sklearn\metrics\classification.py", line 1391, in classification_report
labels = unique_labels(y_true, y_pred)
File "C:\ProgramData\Anaconda2\lib\site-packages\sklearn\utils\multiclass.py", line 104, in unique_labels
raise ValueError("Mix of label input types (string and number)")
ValueError: Mix of label input types (string and number)
please guide me where I am getting wrong
EDIT 1: this is how the y_true and y_pred looks
print "y_true :" ,y_test
print "y_pred :",y_pred
y_true : 5985 neutral
899 positive
2403 neutral
3963 neutral
3457 neutral
5345 neutral
3779 neutral
299 neutral
5712 neutral
5511 neutral
234 neutral
1684 negative
3701 negative
2886 neutral
.
.
.
2623 positive
3549 neutral
4574 neutral
4972 positive
Name: sentiment, Length: 1570, dtype: object
y_pred : [u'neutral' u'positive' u'neutral' ..., u'neutral' u'neutral' u'negative']
EDIT 2: output for type(y_true) and type(y_pred)
type(y_true): <class 'pandas.core.series.Series'>
type(y_pred): <type 'numpy.ndarray'>
Cannot reproduce your error:
import pandas as pd
import numpy as np
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix
# toy data, similar to yours:
data = {'id':[5985,899,2403, 1684], 'sentiment':['neutral', 'positive', 'neutral', 'negative']}
y_true = pd.Series(data['sentiment'], index=data['id'], name='sentiment')
y_true
# 5985 neutral
# 899 positive
# 2403 neutral
# 1684 negative
# Name: sentiment, dtype: object
type(y_true)
# pandas.core.series.Series
y_pred = np.array(['neutral', 'positive', 'negative', 'neutral'])
# all metrics working fine:
accuracy_score(y_true, y_pred)
# 0.5
confusion_matrix(y_true, y_pred)
# array([[0, 1, 0],
# [1, 1, 0],
# [0, 0, 1]], dtype=int64)
classification_report(y_true, y_pred)
# result:
precision recall f1-score support
negative 0.00 0.00 0.00 1
neutral 0.50 0.50 0.50 2
positive 1.00 1.00 1.00 1
total 0.50 0.50 0.50 4
I am trying to create a model to fit data from the cifar-10 dataset. I have a working convolution neural network from an example but when I try to create a multi layer perceptron I keep getting a shape mismatch problem.
#https://gist.github.com/fchollet/0830affa1f7f19fd47b06d4cf89ed44d
#https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Activation, Dropout, Flatten, Dense
from keras import backend as K
from keras.optimizers import RMSprop
# dimensions of our images.
img_width, img_height = 32, 32
train_data_dir = 'pro-data/train'
validation_data_dir = 'pro-data/test'
nb_train_samples = 2000
nb_validation_samples = 800
epochs = 50
batch_size = 16
if K.image_data_format() == 'channels_first':
input_shape = (3, img_width, img_height)
else:
input_shape = (img_width, img_height, 3)
model = Sequential()
model.add(Dense(512, activation='relu', input_shape=input_shape))
model.add(Dropout(0.2))
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(10, activation='softmax'))
model.summary()
model.compile(loss='categorical_crossentropy',
optimizer=RMSprop(),
metrics=['accuracy'])
# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(
rescale=1./255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator()
train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
model.fit_generator(
train_generator,
steps_per_epoch=nb_train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=nb_validation_samples // batch_size)
score = model.evaluate_generator(validation_generator, 1000)
print("Accuracy = ", score[1])
The error I am getting is this:
ValueError: Error when checking target: expected dense_3 to have 4 dimensions, but got array with shape (16, 1)
But if if change the input_shape for the input layer to an incorrect value "(784,)", I get this error:
ValueError: Error when checking input: expected dense_1_input to have 2 dimensions, but got array with shape (16, 32, 32, 3)
This is where I got a working cnn model using flow_from_directory:
https://gist.github.com/fchollet/0830affa1f7f19fd47b06d4cf89ed44d
In case anyone is curious I am getting an accuracy of only 10% for cifar10 using the convolution neural network model. Pretty poor I think.
according to your model, your model summary is
dense_1 (Dense) (None, 32, 32, 512) 2048
dropout_1 (Dropout) (None, 32, 32, 512) 0
dense_2 (Dense) (None, 32, 32, 512) 262656
dropout_2 (Dropout) (None, 32, 32, 512) 0
dense_3 (Dense) (None, 32, 32, 10) 5130
Total params: 269,834
Trainable params: 269,834
Non-trainable params: 0
Your output format is (32,32,10)
In the cifar-10 dataset you want to classify into 10 labels
Try adding
model.add(Flatten())
before your last dense layer.
Now your output layer is
Layer (type) Output Shape Param #
dense_1 (Dense) (None, 32, 32, 512) 2048
dropout_1 (Dropout) (None, 32, 32, 512) 0
dense_2 (Dense) (None, 32, 32, 512) 262656
dropout_2 (Dropout) (None, 32, 32, 512) 0
flatten_1 (Flatten) (None, 524288) 0
dense_3 (Dense) (None, 10) 5242890
Total params: 5,507,594
Trainable params: 5,507,594
Non-trainable params: 0
Also, you've just used the dense and dropout layers in your model. To get better accuracy you should google the various CNN architectures which consists of dense and maxpooling layers.
I want to make k-fold cross-validation with tflearn. Hence I want to reset the network k times. However, I think I need to reset the graph (e.g. tf.reset_default_graph()), but I'm not sure and I don't know how to do this with tflearn.
My code
For the following, you need hasy_tools.py
#!/usr/bin/env python
"""
Trains a simple convnet on the HASY dataset.
Gets to 76.78% test accuracy after 1 epoch.
573 seconds per epoch on a GeForce 940MX GPU.
# WARNING: THIS IS NOT WORKING RIGHT NOW
"""
import os
import hasy_tools as ht
import numpy as np
import tensorflow as tf
import tflearn
from tflearn.layers.core import input_data, fully_connected, dropout
from tflearn.layers.conv import conv_2d, max_pool_2d
from tflearn.layers.estimator import regression
batch_size = 128
nb_epoch = 1
# input image dimensions
img_rows, img_cols = 32, 32
accuracies = []
for fold in range(1, 4):
tf.reset_default_graph()
# Load data
dataset_path = os.path.join(os.path.expanduser("~"), 'hasy')
hasy_data = ht.load_data(fold=fold,
normalize=True,
one_hot=True,
dataset_path=dataset_path,
flatten=False)
train_x = hasy_data['train']['X'][:1000]
train_y = hasy_data['train']['y'][:1000]
test_x = hasy_data['test']['X']
test_y = hasy_data['test']['y']
# Define model
network = input_data(shape=[None, img_rows, img_cols, 1], name='input')
network = conv_2d(network, 32, 3, activation='prelu')
network = conv_2d(network, 64, 3, activation='prelu')
network = max_pool_2d(network, 2)
network = dropout(network, keep_prob=0.25)
network = fully_connected(network, 1024, activation='tanh')
network = dropout(network, keep_prob=0.5)
network = fully_connected(network, 369, activation='softmax')
# Train model
network = regression(network, optimizer='adam', learning_rate=0.001,
loss='categorical_crossentropy', name='target')
model = tflearn.DNN(network, tensorboard_verbose=0)
model.fit({'input': train_x}, {'target': train_y}, n_epoch=nb_epoch,
validation_set=({'input': test_x}, {'target': test_y}),
snapshot_step=100, show_metric=True, run_id='convnet_mnist',
batch_size=batch_size)
# Serialize model
model.save('cv-model-fold-%i.tflearn' % fold)
# Evaluate model
score = model.evaluate(test_x, test_y)
print('Test accuarcy: %0.4f%%' % (score[0] * 100))
accuracies.append(score[0])
accuracies = np.array(accuracies)
print(("CV Accuracy. mean={mean:0.2f}%%\t ({min:0.2f}%% - {max:0.2f}%%)"
).format(mean=accuracies.mean() * 100,
min=accuracies.min() * 100,
max=accuracies.max() * 100))
The error
Running the code with only one fold works fine, but with multiple folds I get:
I tensorflow/core/common_runtime/bfc_allocator.cc:678] Chunk at 0x5556c9100 of size 33554432
I tensorflow/core/common_runtime/bfc_allocator.cc:678] Chunk at 0x5576c9100 of size 66481920
I tensorflow/core/common_runtime/bfc_allocator.cc:687] Free at 0x506d7cc00 of size 256
I tensorflow/core/common_runtime/bfc_allocator.cc:687] Free at 0x506d7d100 of size 62720
I tensorflow/core/common_runtime/bfc_allocator.cc:687] Free at 0x507573700 of size 798208
I tensorflow/core/common_runtime/bfc_allocator.cc:693] Summary of in-use Chunks by size:
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 345 Chunks of size 256 totalling 86.2KiB
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 1 Chunks of size 1024 totalling 1.0KiB
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 23 Chunks of size 1280 totalling 28.8KiB
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 21 Chunks of size 1536 totalling 31.5KiB
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 20 Chunks of size 4096 totalling 80.0KiB
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 16 Chunks of size 73728 totalling 1.12MiB
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 20 Chunks of size 131072 totalling 2.50MiB
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 1 Chunks of size 188928 totalling 184.5KiB
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 17 Chunks of size 262144 totalling 4.25MiB
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 2 Chunks of size 446464 totalling 872.0KiB
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 1 Chunks of size 515584 totalling 503.5KiB
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 1 Chunks of size 524288 totalling 512.0KiB
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 16 Chunks of size 1511424 totalling 23.06MiB
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 6 Chunks of size 16777216 totalling 96.00MiB
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 1 Chunks of size 23026176 totalling 21.96MiB
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 6 Chunks of size 33554432 totalling 192.00MiB
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 1 Chunks of size 66481920 totalling 63.40MiB
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 15 Chunks of size 67108864 totalling 960.00MiB
I tensorflow/core/common_runtime/bfc_allocator.cc:696] 1 Chunks of size 67183872 totalling 64.07MiB
I tensorflow/core/common_runtime/bfc_allocator.cc:700] Sum Total of in-use chunks: 1.40GiB
I tensorflow/core/common_runtime/bfc_allocator.cc:702] Stats:
Limit: 1500971008
InUse: 1500109824
MaxInUse: 1500109824
NumAllocs: 43767
MaxAllocSize: 844062464
W tensorflow/core/common_runtime/bfc_allocator.cc:274] **************************************************************************************************xx
W tensorflow/core/common_runtime/bfc_allocator.cc:275] Ran out of memory trying to allocate 8.00MiB. See logs for memory state.
W tensorflow/core/framework/op_kernel.cc:975] Resource exhausted: OOM when allocating tensor with shape[128,16,16,64]
--
Traceback (most recent call last):
File "tflearn_hasy_cv.py", line 60, in <module>
batch_size=batch_size)
File "/home/moose/GitHub/tflearn/tflearn/models/dnn.py", line 188, in fit
run_id=run_id)
File "/home/moose/GitHub/tflearn/tflearn/helpers/trainer.py", line 277, in fit
show_metric)
File "/home/moose/GitHub/tflearn/tflearn/helpers/trainer.py", line 684, in _train
feed_batch)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 766, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 964, in _run
feed_dict_string, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1014, in _do_run
target_list, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1034, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM when allocating tensor with shape[128,16,16,64]
[[Node: MaxPool2D/MaxPool = MaxPool[T=DT_FLOAT, data_format="NHWC", ksize=[1, 2, 2, 1], padding="SAME", strides=[1, 2, 2, 1], _device="/job:localhost/replica:0/task:0/gpu:0"](Conv2D_1/PReLU/add)]]
[[Node: Crossentropy/Mean/_19 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_920_Crossentropy/Mean", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
Caused by op u'MaxPool2D/MaxPool', defined at:
File "tflearn_hasy_cv.py", line 47, in <module>
network = max_pool_2d(network, 2)
File "/home/moose/GitHub/tflearn/tflearn/layers/conv.py", line 363, in max_pool_2d
inference = tf.nn.max_pool(incoming, kernel, strides, padding)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/nn_ops.py", line 1617, in max_pool
name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_nn_ops.py", line 1598, in _max_pool
data_format=data_format, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 759, in apply_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2240, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1128, in __init__
self._traceback = _extract_stack()
ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[128,16,16,64]
[[Node: MaxPool2D/MaxPool = MaxPool[T=DT_FLOAT, data_format="NHWC", ksize=[1, 2, 2, 1], padding="SAME", strides=[1, 2, 2, 1], _device="/job:localhost/replica:0/task:0/gpu:0"](Conv2D_1/PReLU/add)]]
[[Node: Crossentropy/Mean/_19 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_920_Crossentropy/Mean", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
edit: I get the same problem with Keras.
I put in my code tf.reset_default_graph() right before building the neural network, before the for loop.
You should also define your model outside the for loop, right before it. Only the training should be in the kfolds loop.