I would like to implement a bandpass filter using GPUImageColorMatrixFilter. Basically, blue would equal floor(blue - (k*red)) and both red and green would just end up being zero. Where can I find documentation indicating what the columns and rows of the matrix mean?
My intuition would suggest that the 4x4 matrix is following the standard RGBA order and judging by the examples (see for instance GPUImageSepiaFilter) it looks like I'm right.
For instance, this is the identity GPUMatrix4x4
R G B A
| 1 0 0 0 | red
| 0 1 0 0 | green
| 0 0 1 0 | blue
| 0 0 0 1 | alpha
Let's name each coefficient
R G B A
| a b c d | red
| e f g h | green
| i j k l | blue
| m n o p | alpha
Applying the matrix to a RGBA color will result in the following R'G'B'A' color where the components are computed as
R' = a*R + b*G + c*B + d*A
G' = e*R + f*G + g*B + h*A
B' = i*R + j*G + k*B + l*A
A' = m*R + n*G + o*B + p*A
which is nothing but the following matrix multiplication
| a b c d | |R| |R'|
| e f g h | x |G| = |G'|
| i j k l | |B| |B'|
| m n o p | |A| |A'|
Related
There is a multi-classification problem with 27 classes.
y_predict=[0 0 0 20 26 21 21 26 ....]
y_true=[1 10 10 20 26 21 18 26 ...]
A list named "answer_vocabulary" stored the corresponding 27 words to each index.
answer_vocabulary=[0 1 10 11 2 3 agriculture commercial east living north .....]
cm = confusion_matrix(y_true=y_true, y_pred=y_predict)
I'm confused about the order of the confusion matrix. It is in an ascending index order? And if I want to reorder the confusion matrix with a label sequence=[0 1 2 3 10 11 agriculture commercial living east north ...], how can I implement it?
Here is a function I have tried to plot confusion matrix.
def plot_confusion_matrix(cm, classes,
normalize=False,
title='Confusion matrix',
cmap=plt.cm.Blues):
"""
This function prints and plots the confusion matrix.
Normalization can be applied by setting `normalize=True`.
"""
plt.imshow(cm, interpolation='nearest', cmap=cmap)
plt.title(title)
plt.colorbar()
tick_marks = np.arange(len(classes))
plt.xticks(tick_marks, classes, rotation=45)
plt.yticks(tick_marks, classes)
if normalize:
cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]
print("Normalized confusion matrix")
else:
print('Confusion matrix, without normalization')
print(cm)
thresh = cm.max() / 2.
for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
plt.text(j, i, cm[i, j],
horizontalalignment="center",
color="white" if cm[i, j] > thresh else "black")
plt.tight_layout()
plt.ylabel('True label')
plt.xlabel('Predicted label')
The confusion matrices from sklearn don't store information about how the matrix was created (class ordering, and normalization): this means you must use the confusion matrix as soon as you create it or the information will be lost.
By default, sklearn.metrics.confusion_matrix(y_true,y_pred) create the matrix in the order the classes appear in y_true.
If you pass this data to sklearn.metrix.confusion_matrix:
+--------+--------+
| y_true | y_pred |
+--------+--------+
| A | B |
| C | C |
| D | B |
| B | A |
+--------+--------+
Scikit-leart will create this confusion matrix (zeros omited):
+-----------+---+---+---+---+
| true\pred | A | C | D | B |
+-----------+---+---+---+---+
| A | | | | 1 |
| C | | 1 | | |
| D | | | | 1 |
| B | 1 | | | |
+-----------+---+---+---+---+
And it will return this numpy matrix to you:
+---+---+---+---+
| 0 | 0 | 0 | 1 |
| 0 | 0 | 1 | 0 |
| 0 | 0 | 0 | 1 |
| 1 | 0 | 0 | 0 |
+---+---+---+---+
If you want to select classes, or reorder them you can pass the 'labels' argument to confusion_matrix().
For reordering:
labels = ['D','C','B','A']
mat = confusion_matrix(true_y,pred_y, labels=labels)
Or, if you just want to focus on some labels (useful if you have a lot of labels):
labels = ['A','D']
mat = confusion_matrix(true_y,pred_y, labels=labels)
Also,take a look at sklearn.metrics.plot_confusion_matrix. It works very well for small (<100) classes.
If you have >100 classes it will take a white to plot the matrix.
The order of the columns/rows in the resulting confusion matrix is the same as returned by sklearn.utils.unique_labels(), which extracts "an ordered array of unique labels". In the source code of confusion_matrix() (main, git-hash 7e197fd), the lines of interest read as follows
if labels is None:
labels = unique_labels(y_true, y_pred)
else:
labels = np.asarray(labels)
Here, labels is the optional argument of confusion_matrix() to prescribe an ordering/subset of labels yourself:
cm = confusion_matrix(true_y, pred_y, labels=labels)
Therefore, if labels = [0, 10, 3], cm will have shape (3,3), and the rows/columns can be indexed directly with labels. If you know pandas:
import pandas as pd
cm = pd.DataFrame(cm, index=labels, columns=labels)
Note that the docs of unique_labels() state that mixed types of labels (numeric and string) are not supported. In this case, I'd recommend to use a LabelEncoder. This will save you from maintaining your own lookup-table.
from sklearn.preprocessing import LabelEncoder
encoder = LabelEncoder()
y = encoder.fit_transform(y)
# y have now values between 0 and n_labels-1.
# Do some ops here...
...
# To convert back:
y_pred = encoder.inverse_transform(y_pred)
y = encoder.inverse_transform(y)
As the previous answer already mentioned, plot_confusion_matrix() comes in handy to visualize the confusion matrix.
I have 2 tables.
1 table with all possible mistakes, looks like
mistake|description
m1 | a
m2 | b
m3 | c
second table is my data:
n | m1 | m2 | m3
1 | 1 | 0 | 1
2 | 0 | 1 | 1
3 | 1 | 1 | 0
where n is row_num, and for each m I put 1 with mistake, 0 - without.
In total I want to join them showing row_nums (or other info) for each mistake.
Something like:
mistake | n
m1 |1
m1 |3
m2 |2
m2 |3
m3 |1
m3 |2
It looks to me like you are just asking to transpose the data.
data have;
input n m1 m2 m3 ;
cards;
1 1 0 1
2 0 1 1
3 1 1 0
;
proc transpose data=have out=want ;
by n ;
var m1 m2 m3 ;
run;
Let's say I have a database with the schema:
A - String
B - String
C - Int
D - Int
And the database is
A | B | C | D
------------------
'F' | 'a' | 1 | 2
'F' | 'a' | 1 | 4
'F' | 'b' | 2 | 4
'Z' | 'a' | 3 | 7
'Z' | 'b' | 4 | 3
'Z' | 'a' | 6 | 5
And I want something along the lines of
F
a 2 6
b 2 4
Z
a 9 12
b 4 3
So essentially, group by A, then group by B, and SUM(C), SUM(D). How can I do this in ActiveRecord?
Here is an AR solution:
things = Thing.select('a, b, sum(c) as sum_c, sum(d) as sum_d').group(:a, :b)
things.each do |thing|
puts "#{thing.a} / #{thing.b} / #{thing.sum_c} / #{thing.sum_d}"
end
# F / a / 2 / 6
# F / b / 2 / 4
# Z / a / 9 / 12
# Z / b / 4 / 3
You can dynamically create properties in your AR classes (see as sum_c). Depending on what you do this might be easier/nicer with straight SQL.
In neo4j I am querying
MATCH (n)-[t:x{x:"1a"}]->()
WHERE n.a > 1 OR n.b > 1 AND toFloat(n.a) / (n.a+n.b) * 100 < 90
RETURN DISTINCT n, toFloat(n.a) / (n.a + n.b) * 100
ORDER BY toFloat(n.a) / (n.a + n.b) * 100 DESC
LIMIT 10
but I got / by zero error.
Since I declared one of n.a or n.b should be 1, if both zero it should skip that row and I shouldn't get this error. This looks like a logic issue in Neo4j. There is no problem when I delete AND toFloat(n.a)/(n.a+n.b)*100 < 90 from WHERE clause. But I want the results only lower than 90. How can I overcome this?
Can either of n.a or n.b be negative? I was able to reproduce this with:
WITH -2 AS na, 2 AS nb
WHERE (na > 1 OR nb > 1) AND toFloat(na)/(na+nb)*100 < 90
RETURN na, nb
And I get: / by zero
Perhaps try changing your WHERE clause to:
WITH -2 AS na, 2 AS nb
WHERE (na + nb > 0) AND toFloat(na)/(na+nb)*100 < 90
RETURN na, nb
And I get: zero rows.
It seems the second condition, toFloat(na) / (na + nb) * 100 < 90, is tested before the first. Look at the Filter(1) operator in this execution plan:
+--------------+---------------+------+--------+--------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Operator | EstimatedRows | Rows | DbHits | Identifiers | Other |
+--------------+---------------+------+--------+--------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Projection | 1 | 3 | 0 | anon[111], anon[138], n, toFloat(n.a)/(n.a + n.b)* 100 | anon[111]; anon[138] |
| Top | 1 | 3 | 0 | anon[111], anon[138] | { AUTOINT6}; |
| Distinct | 0 | 3 | 24 | anon[111], anon[138] | anon[111], anon[138] |
| Filter(0) | 0 | 3 | 6 | anon[29], n, t | t.x == { AUTOSTRING0} |
| Expand(All) | 1 | 3 | 6 | anon[29], n, t | ( n#7)-[t:x]->() |
| Filter(1) | 1 | 3 | 34 | n | (Ors(List(n#7.a > { AUTOINT1}, Multiply(Divide(ToFloatFunction( n#7.a),Add( n#7.a, n#7.b)),{ AUTOINT3}) < { AUTOINT4})) AND Ors(List( n#7.a > { AUTOINT1}, n.b > { AUTOINT2}))) |
| AllNodesScan | 4 | 4 | 5 | n | |
+--------------+---------------+------+--------+--------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
You can get around this by force breaking the filter into two clauses.
MATCH (n)-[t:x { x:"1a" }]->()
WHERE n.a > 1 OR n.b > 1
WITH n
WHERE toFloat(n.a) / (n.a + n.b) * 100 < 90
RETURN DISTINCT n, toFloat(n.a) / (n.a + n.b) * 100
ORDER BY toFloat(n.a) / (n.a + n.b) * 100 DESC
LIMIT 10
I found this behavior surprising, but as I think about it I suppose it isn't wrong for the execution engine to rearrange the filter in this way. There may be the assumption that the condition will abandon early on failing the first declared condition, but Cypher is exactly that: declarative. So we express the "what", not the "how", and in terms of the "what" A and B is equivalent to B and A.
Here is the query and a sample graph, you can check if it translates to your actual data:
http://console.neo4j.org/r/f6kxi5
S → Y | 1X
X → 1X | 0
Y → Y 0 | 1X1 | 2X2
I do not understand how to do factoring or substitution when there are more than one of the same symbol. Thanking you.
X → 1 X | 0
refers to a 0-or-more 1 followed by 0, so it's equivalent to
X → 1* 0
The same approach can be used to remove your left-recursion. You can rewrite
S → Y | 1 X
X → 1 X | 0
Y → Y 0 | 1 X 1 | 2 X 2
as
S → Y | 1 X
X → 1* 0
Y → ( 1 X 1 | 2 X 2 )* 0
In EBNF:
S → Y | 1 X
X → 1* 0
Y → A* 0
A → 1 X 1 | 2 X 2
In BNF:
S → Y | 1 X
X → 1* 0
Y → A* 0
A → 1 X 1 | 2 X 2
1* → 1 1* | Ɛ
A* → A A* | Ɛ
If all you wanted to do was to eliminated left-recursion, you're done.
If you want to eliminate common prefixes too, you're not done because both sub-rules of S can start with 1 X. To fix this, inline and distribute to obtain the following:
S → 0
| 1 X 1 Y
| 2 X 2 Y
| 1 X
X → 1* 0
Y → A* 0
A → 1 X 1 | 2 X 2
Now, we're finally in a position to factor out the common 1 X.
S → 0
| 1 X ( 1 Y )?
| 2 X 2 Y
X → 1* 0
Y → A* 0
A → 1 X 1 | 2 X 2
In EBNF:
S → 0 | 1 X B? | 2 X 2 Y
X → 1* 0
Y → A* 0
A → 1 X 1 | 2 X 2
B → 1 Y
In BNF:
S → 0 | 1 X B? | 2 X 2 Y
X → 1* 0
Y → A* 0
A → 1 X 1 | 2 X 2
B → 1 Y
B? → B | Ɛ
1* → 1 1* | Ɛ
A* → A A* | Ɛ