Related
Let us assume there are 5-time slots and at each time slot, I have 4 options to choose from, each with a known reward, for eg. rewards = [5, 2, 1, -3]. At every time step, at least 1 of the four options must be selected, with a condition that, if option 3 (with reward -3) is chosen at a time t, then for the remaining time steps, none of the options should be selected. As an example, considering the options are indexed from 0, both [2, 1, 1, 0, 3] and [2, 1, 1, 3, 99] are valid solutions with the second solution having option 3 selected in the 3rd time step and 99 is some random value representing no option was chosen.
The Z3py code I tried is here:
T = 6 #Total time slots
s = Solver()
pick = [[Bool('t%d_ch%d' %(j, i)) for i in range(4)] for j in range(T)]
# Rewards of each option
Rewards = [5, 2, 1, -3]
# Select at most one of the 4 options as True
for i in range(T):
s.add(Or(Not(Or(pick[i][0], pick[i][1], pick[i][2], pick[i][3])),
And(Xor(pick[i][0],pick[i][1]), Not(Or(pick[i][2], pick[i][3]))),
And(Xor(pick[i][2],pick[i][3]), Not(Or(pick[i][0], pick[i][1])))))
# If option 3 is picked, then none of the 4 options should be selected for the future time slots
# else, exactly one should be selected.
for i in range(len(pick)-1):
for j in range(4):
s.add(If(And(j==3,pick[i][j]),
Not(Or(pick[i+1][0], pick[i+1][1], pick[i+1][2], pick[i+1][3])),
Or(And(Xor(pick[i+1][0],pick[i+1][1]), Not(Or(pick[i+1][2], pick[i+1][3]))),
And(Xor(pick[i+1][2],pick[i+1][3]), Not(Or(pick[i+1][0], pick[i+1][1]))))))
if s.check()==False:
print("unsat")
m=s.model()
print(m)
With this implementation, I am not getting solutions such as [2, 1, 1, 3, 99]. All of them either do not have option 3 or have it in the last time slot.
I know there is an error inside the If part but I'm unable to figure it out. Is there a better way to achieve such solutions?
It's hard to decipher what you're trying to do. From a basic reading of your description, I think this might be an instance of the XY problem. See https://xyproblem.info/ for details on that, and try to cast your question in terms of what your original goal is; instead of a particular solution, you're trying to implement. (It seems to me that the solution you came up with is unnecessarily complicated.)
Having said that, you can solve your problem as stated if you get rid of the 99 requirement and simply indicate -3 as the terminator. Once you pick -3, then all the following picks should be -3. This can be coded as follows:
from z3 import *
T = 6
s = Solver()
Rewards = [5, 2, 1, -3]
picks = [Int('pick_%d' % i) for i in range(T)]
def pickReward(p):
return Or([p == r for r in Rewards])
for i in range(T):
if i == 0:
s.add(pickReward(picks[i]))
else:
s.add(If(picks[i-1] == -3, picks[i] == -3, pickReward(picks[i])))
while s.check() == sat:
m = s.model()
picked = []
for i in picks:
picked += [m[i]]
print(picked)
s.add(Or([p != v for p, v in zip(picks, picked)]))
When run, this prints:
[5, -3, -3, -3, -3, -3]
[1, 5, 5, 5, 5, 1]
[1, 2, 5, 5, 5, 1]
[2, 2, 5, 5, 5, 1]
[2, 5, 5, 5, 5, 1]
[2, 1, 5, 5, 5, 1]
[1, 1, 5, 5, 5, 1]
[2, 1, 5, 5, 5, 2]
[2, 5, 5, 5, 5, 2]
[2, 5, 5, 5, 5, 5]
[2, 5, 5, 5, 5, -3]
[2, 1, 5, 5, 5, 5]
...
I interrupted the above as it keeps enumerating all the possible picks. There are a total of 1093 of them in this particular case.
(You can get different answers depending on your version of z3.)
Hope this gets you started. Stating what your original goal is directly is usually much more helpful, should you have further questions.
I am implementing a native webgl context compatible with h5.
Currently I support webgl1.0 APIs.
On iOS I create the EAGLContext with kEAGLRenderingAPIOpenGLES3. Other GL calls works fine, but
glTexImage2D(3553, 0, 6408, 144, 108, 0, 6408, 5126, null), glError()=1282
This call fails.
If I change EAGLContext to opengles2.0, everything works fine.
My question is all parameter values to glTexSubImage2D are the same. Why this call fails if I create the context as es3.0 but succeeds if the context is es2.0.
These are the gl calls dumped. The only difference is that when I create the EAGLContext using GLES3 api level, there is a glError 1282. If the context is created using GLES2 api level, everything works fine.
The first two glTexImage2D use GL_UNSIGNED_BYTE, the failed one uses GL_FLOAT. But es3.0 context should support GL_FLOAT.
17:26:24.683200 Will setup FBOs.
17:26:24.684360 Setup FBOs done.
17:26:24.694778 glCreateTexture()=1
17:26:24.694981 glBindTexture(3553, 1)
17:26:24.695079 glTexParameteri(3553, 10242, 10497)
17:26:24.695142 glTexParameteri(3553, 10243, 10497)
17:26:24.695266 glTexParameteri(3553, 10241, 9985)
17:26:24.695313 glTexParameteri(3553, 10240, 9729)
17:26:24.695414 glTexParameterf(3553, 34046, 1.000000)
17:26:24.695414 glTexImage2D(3553, 0, 6408, 2, 2, 0, 6408, 5121, null)
17:26:24.695414 glTexImage2D(3553, 1, 6408, 1, 1, 0, 6408, 5121, null)
17:26:24.696141 [Buf:GL_UNSIGNED_BYTE:u8] 16, 16, 1
17:26:24.696961 glTexImage2D(3553, 0, 6408, 2, 2, 0, 6408, 5121, [16])
17:26:24.697674 glGenBuffers()=1
17:26:24.697862 glGenBuffers()=2
17:26:24.702478 glGenBuffers()=3
17:26:24.702547 glGenBuffers()=4
17:26:24.702675 glGenBuffers()=5
17:26:24.702734 glGenBuffers()=6
17:26:24.722429 glGenBuffers()=7
17:26:24.722589 glBindBuffer(34962, 7)
17:26:24.722697 glBufferData(34962, [65536], null, 35048)
17:26:24.722758 glGenBuffers()=8
17:26:24.722806 glBindBuffer(34962, 8)
17:26:24.722862 glBufferData(34962, [65536], null, 35048)
17:26:24.723104 createVertexArrayOES(1)
17:26:24.723690 glGenBuffers()=9
17:26:24.723743 glBindBuffer(34962, 9)
17:26:24.723799 glBufferData(34962, [2304000], null, 35048)
17:26:24.723985 glGenBuffers()=10
17:26:24.724068 glBindBuffer(34963, 10)
17:26:24.724120 glBufferData(34963, [64000], null, 35048)
17:26:24.724120 glCreateTexture()=2
17:26:24.747552 glBindTexture(3553, 2)
17:26:24.747625 glTexParameteri(3553, 10242, 33071)
17:26:24.747680 glTexParameteri(3553, 10243, 33071)
17:26:24.747733 glTexParameteri(3553, 10241, 9729)
17:26:24.747778 glTexParameteri(3553, 10240, 9729)
17:26:24.747842 glTexParameterf(3553, 34046, 1.000000)
17:26:24.747842 glTexImage2D(3553, 0, 6408, 144, 108, 0, 6408, 5126, null), glError()=1282
17:26:24.748000 glTexParameteri(3553, 10241, 9728)
17:26:24.748048 glTexParameteri(3553, 10240, 9728)
17:26:24.748120 glTexParameteri(3553, 10242, 33071)
17:26:24.748189 glTexParameteri(3553, 10243, 33071)
17:26:24.748266 glTexParameterf(3553, 34046, 1.000000)+0800
The error is because JS passed invalid combination of internal format/format/type.
glTexImage2D(3553, 0, 6408, 144, 108, 0, 6408, 5126, null), glError()=1282
is actually glTexImage2D(3553, 0, GL_RGBA, 144, 108, 0, GL_RGBA, GL_FLOAT, null)
This combination is not valid according to https://www.khronos.org/registry/OpenGL-Refpages/es3.0/html/glTexImage2D.xhtml.
The interesting thing is that in iOS es2.0 context, this combination is valid.
I want to develop a CNN model to identify 24 hand signs in American Sign Language. I created a custom dataset that contains 3000 images for each hand sign i.e. 72000 images in the entire dataset.
For training the model, I would be using 80-20 dataset split (2400 images/hand sign in the training set and 600 images/hand sign in the validation set).
My question is:
Should I randomly shuffle the images when creating the dataset? And Why?
Based on my previous experience, it led to validation loss being lower than training loss and validation accuracy more than training accuracy. Check this link.
Random shuffling of data is a standard procedure in all machine learning pipelines, and image classification is not an exception; its purpose is to break possible biases during data preparation - e.g. putting all the cat images first and then the dog ones in a cat/dog classification dataset.
Take for example the famous iris dataset:
from sklearn.datasets import load_iris
X, y = load_iris(return_X_y=True)
y
# result:
array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2])
As you can clearly see, the dataset has been prepared in such a way that the first 50 samples are all of label 0, the next 50 of label 1, and the last 50 of label 2. Try to perform a 5-fold cross validation in such a dataset without shuffling and you'll find most of your folds containing only a single label; try a 3-fold CV, and all your folds will include only one label. Bad... BTW, it's not just a theoretical possibility, it has actually happened.
Even if no such bias exists, shuffling never hurts, so we do it always just to be on the safe side (you never know...).
Based on my previous experience, it led to validation loss being lower than training loss and validation accuracy more than training accuracy. Check this link.
As noted in the answer there, it is highly unlikely that this was due to shuffling. Data shuffling is not anything sophisticated - essentially, it is just the equivalent of shuffling a deck of cards; it may have happened once that you insisted on "better" shuffling and subsequently you ended up with a straight flush hand, but obviously this was not due to the "better" shuffling of the cards.
Here is my two cents on the topic.
First of all make sure to extract a test set that has equal number of samples for each hand sign. (hand sign #1 - 500 samples, hand sign #2 - 500 samples and so on)
I think this is referred to as stratified sampling.
When it comes to the training set, there is no huge mistake in shuffling the entire set. However, when splitting the training set into training and validation set make sure that the validation set is good enough to be a representation for the test set.
One of my personal experiences with shuffling:
After splitting the training set into training and validation sets, the validation set turned out to be very easy to predict. Therefore, I saw good learning metric values. However, the performance of the model on the test set was horrible.
I have text data as follows.
X_train_orignal= np.array(['OC(=O)C1=C(Cl)C=CC=C1Cl', 'OC(=O)C1=C(Cl)C=C(Cl)C=C1Cl',
'OC(=O)C1=CC=CC(=C1Cl)Cl', 'OC(=O)C1=CC(=CC=C1Cl)Cl',
'OC1=C(C=C(C=C1)[N+]([O-])=O)[N+]([O-])=O'])
As it is evident that different sequences have different length. How can I zero pad the sequence on both sides of the sequence to some maximum length. And then convert each sequence into one hot encoding based on each characters?
Try:
I used the following keras API but it doesn't work with strings sequence.
keras.preprocessing.sequence.pad_sequences(sequences, maxlen=None, dtype='int32', padding='pre', truncating='pre', value=0.0)
I might need to convert my sequence data into one hot vectors first and then zero pad it. For that I tried to use Tokanizeas follows.
tk = Tokenizer(nb_words=?, split=?)
But then, what should be the split value and nb_words as my sequence data doesn't have any space? How to use it for character based one hot?
MY overall goal is to zero pad my sequences and convert it to one hot before I feed it into RNN.
So i came across a way to do by using Tokenizer first and then pad_sequences to zero pad my sequence in the start as follows.
from keras.preprocessing.text import Tokenizer
tokenizer = Tokenizer(char_level=True)
tokenizer.fit_on_texts(X_train_orignal)
sequence_of_int = tokenizer.texts_to_sequences(X_train_orignal)
This gives me the output as follows.
[[3, 1, 4, 2, 3, 5, 1, 6, 2, 1, 4, 1, 7, 5, 1, 2, 1, 1, 2, 1, 6, 1, 7],
[3,
1,
4,
2,
3,
5,
1,
6,
2,
1,
4,
1,
7,
5,
1,
2,
1,
4,
1,
7,
5,
1,
2,
1,
6,
1,
7],
[3, 1, 4, 2, 3, 5, 1, 6, 2, 1, 1, 2, 1, 1, 4, 2, 1, 6, 1, 7, 5, 1, 7],
[3, 1, 4, 2, 3, 5, 1, 6, 2, 1, 1, 4, 2, 1, 1, 2, 1, 6, 1, 7, 5, 1, 7],
[3,
1,
6,
2,
1,
4,
1,
2,
1,
4,
1,
2,
1,
6,
5,
8,
10,
11,
9,
4,
8,
3,
12,
9,
5,
2,
3,
5,
8,
10,
11,
9,
4,
8,
3,
12,
9,
5,
2,
3]]
Now I do not understand why it is giving sequence_of_int[1], sequence_of_int[4] output in column format?
After getting the tokens, I applied the pad_sequences as follows.
seq=keras.preprocessing.sequence.pad_sequences(sequence_of_int, maxlen=None, dtype='int32', padding='pre', value=0.0)
and it gives me the output as follows.
array([[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 3, 1, 4, 2, 3, 5, 1, 6, 2, 1, 4, 1, 7, 5, 1,
2, 1, 1, 2, 1, 6, 1, 7],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 1, 4,
2, 3, 5, 1, 6, 2, 1, 4, 1, 7, 5, 1, 2, 1, 4, 1,
7, 5, 1, 2, 1, 6, 1, 7],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 3, 1, 4, 2, 3, 5, 1, 6, 2, 1, 1, 2, 1, 1, 4,
2, 1, 6, 1, 7, 5, 1, 7],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 3, 1, 4, 2, 3, 5, 1, 6, 2, 1, 1, 4, 2, 1, 1,
2, 1, 6, 1, 7, 5, 1, 7],
[ 3, 1, 6, 2, 1, 4, 1, 2, 1, 4, 1, 2, 1, 6, 5, 8,
10, 11, 9, 4, 8, 3, 12, 9, 5, 2, 3, 5, 8, 10, 11, 9,
4, 8, 3, 12, 9, 5, 2, 3]], dtype=int32)
Then after that, I converted it into one hot as follows.
one_hot=keras.utils.to_categorical(seq)
How can I read a file like this:
11111
10001
10001
10001
11111
To a bidimensional array like this:
{{1, 1, 1, 1, 1},
{1, 0, 0, 0, 1},
{1, 0, 0, 0, 1},
{1, 0, 0, 0, 1},
{1, 1, 1, 1, 1}}
In Lua?
This is what I thought of:
for i = 1, number_of_lines do
current_line = map_file:read("*line")
character_array = {}
for i = 1, #current_line do
table.insert(character_array, current_line[i])
end
end
However, I don't know how to get 'number_of_lines', this is, the number of lines in a text file with Lua. How can I do it?
Also, if there's some other easier way please tell me about it.
You don't need to get the number of lines. Just keep going until you run out of lines.
local line_data = {}
for line in map_file:lines() do
local character_array = {}
for i = 1, #line do
character_array[#character_array + 1] = line[i];
end
line_data[#line_data + 1] = character_array
end