Adapting MNIST designed network for a larger dataset - machine-learning

Question

I am attempting to implement this deep clustering algorithm, which was designed to cluster the MNIST dataset. Single Channel 28x28 images.
The images I am trying to use are 416x416 and 3-Channel RGB. The script is initialised with the following functions.
class CachedMNIST(Dataset):
def __init__(self, train, cuda, testing_mode=False):
img_transform = transforms.Compose([transforms.Lambda(self._transformation)])
# img_transform = transforms.Compose([transforms.Resize((28*28)), transforms.ToTensor(), transforms.Grayscale()])
self.ds = torchvision.datasets.ImageFolder(root=train, transform=img_transform)
self.cuda = cuda
self.testing_mode = testing_mode
self._cache = dict()
#staticmethod
def _transformation(img):
return (torch.ByteTensor(torch.ByteStorage.from_buffer(img.tobytes())).float()
* 0.02
)
If the images are left un-altered the resulting tensor shape output from the _transformation function is of size torch.Size{{256,519168]] far too large for the AutoEncoder network to calculate.
Error 1
RuntimeError: mat1 and mat2 shapes cannot be multiplied (128x519168 and 784x500)
When I attempted to resize the images a the result is a 4D Tensor, torch.Size([256,1,784,748]) which even when reducing the Batch Size to minuscule amounts the CUDA will crash as there is not enough memory.
Error 2
RuntimeError: CUDA out of memory.
I'm hoping someone can point me in the right direction to tackle this problem as there must be a more efficient way to adapt the network.
AutoEnocder Model
StackedDenoisingAutoEncoder(
(encoder): Sequential(
(0): Sequential(
(linear): Linear(in_features=784, out_features=500, bias=True)
(activation): ReLU()
)
(1): Sequential(
(linear): Linear(in_features=500, out_features=500, bias=True)
(activation): ReLU()
)
(2): Sequential(
(linear): Linear(in_features=500, out_features=2000, bias=True)
(activation): ReLU()
)
(3): Sequential(
(linear): Linear(in_features=2000, out_features=10, bias=True)
)
)
(decoder): Sequential(
(0): Sequential(
(linear): Linear(in_features=10, out_features=2000, bias=True)
(activation): ReLU()
)
(1): Sequential(
(linear): Linear(in_features=2000, out_features=500, bias=True)
(activation): ReLU()
)
(2): Sequential(
(linear): Linear(in_features=500, out_features=500, bias=True)
(activation): ReLU()
)
(3): Sequential(
(linear): Linear(in_features=500, out_features=784, bias=True)
)
)
)

Answer 1

Error 1 is happening because the first linear layer has in_features=784. That number comes from the 28x28 pixels in the 1-channel MNIST data. Your input data is 416x416x3 = 519168 (different if you resize your inputs). In order to resolve this error, you need to make the in_features in that first linear layer match the number of pixels (times the number of channels) of your input. You can do this by changing that number or resizing your input (or, likely both). Also, note that you will likely have to flatten your input so that it is a vector. Also, note that whatever the in_features becomes (to the encoder) you'll want to make the out_features of the decoder match (otherwise you'll be trying to compare two vectors of different sizes when training).
Error 2 CUDA OOM could happen for lots of reasons (small GPU, too large of network, too large batch size, etc). The network you have doesn't appear to look particularly large. But you could reduce its size by shrinking some of the internal layers (numbers of in_features and out_features). Just be sure that if you adjust these, that you maintain the property that the number of out_features from one layer matches the number of in_features on the next layer. And, in this example, the decoder is a nice mirror of in the encoder (so if you adjust the encoder, make the corresponding mirror-adjustment in the decoder).