I have two metrics with matching labels, both counters:
accounts_created_total{provider="auth0"} 738
accounts_created_total{provider="google} 980
accounts_deleted_total{provider="auth0"} 65
I'd like to calculate the number of existing accounts from those two metrics. I came up with this:
accounts_created_total - accounts_deleted_total
# which results in
{provider="auth0"} 673
# Note the missing provider="Google"
Unfortunately, there's no account_deleted_total for provider="Google", and so I only get the result for provider="auth0".
Is there a way I can tell prometheus to "make up" the missing labels? That would typically be equivalent to a coalesce in SQL.
You can complete a time serie using the OR binary operator:
vector1 or vector2 results in a vector that contains all original elements
(label sets + values) of vector1 and additionally all elements of vector2
which do not have matching label sets in vector1.
Assuming you want to default accounts_deleted_total to 0, the following expression uses accounts_created_total as second vector to extract the labels and multiplying by 0 ensures the value is reset:
accounts_deleted_total OR (accounts_created_total * 0)
In the case of autho0, the label exists in accounts_deleted_total and the second part will not be used ; to the contrary, for google, second part will yield
{provider="google"} 0
Finally, you can use it in your expression:
accounts_created_total - (accounts_deleted_total OR (accounts_created_total * 0))
In your specific case, since you are using the same metric to extract the labels, you can even simplify the expression to:
(accounts_created_total - accounts_deleted_total) OR accounts_created_total
Related
The doc page of TermCriteria says that the MAX_ITER is the same as COUNT and the type can be one of : COUNT, EPS or COUNT + EPS. I am wondering whether there is a difference between COUNT + EPS and MAX_ITER + EPS. I found that in different places, there are these two different styles. Would that lead to different effects while running?
There is no difference. COUNT and MAX_ITER mean the same. They have the same value, hence are indistinguishable.
Well, their meaning depends on what function takes a TermCriteria tuple/struct/object. Still, same value means the identifiers are interchangeable.
Those named constants live in an enum. The values are chosen to be bits in a bit field. So they're actually flags and should, ordinarily, be combined with | (bitwise OR operator).
The + is a funny custom and probably because of the following... if you give two termination criteria, an algorithm terminates if any of them becomes true. So one could say both the one and the other are given... and now people get their brain gyri twisted thinking of "and" and "or". Combining those flags with + sidesteps that nicely.
cv.TermCriteria_COUNT == 1
cv.TermCriteria_MAX_ITER == 1
cv.TermCriteria_EPS == 2
so your choices are:
COUNT (means MAX_ITER)
MAX_ITER (means COUNT)
EPS
COUNT + EPS
MAX_ITER + EPS
Beware that you don't say COUNT + MAX_ITER (wrong!) because that is 1 + 1 = 2 and that is now EPS, which isn't what that expression was supposed to express.
The documentation may not contain all the information, and it is generated from OpenCV public header files (via doxygen and its config file).
Just use an IDE/Editor, browsing the source code, search TermCriteria, and will see MAX_ITER and COUNT enumeration element values. Should be same.
Suppose you have a 2x2 design and you're testing differences between those 4 groups using ANOVA in SPSS.
This is a graph of your data:
After performing ANOVA, there are 6 possible pairwise comparisons between groups that we can perform. These are:
A - C
B - D
A - D
B - C
A - B
C - D
If I want to perform pairwise comparisons, I would usually use this script after the UNIANOVA command:
/EMMEANS=TABLES(Var1*Var2) COMPARE (Var1) ADJ(LSD)
/EMMEANS=TABLES(Var1*Var2) COMPARE (Var2) ADJ(LSD)
However, after running this script, the output only contains 4 of the 6 possible comparisons - there are two pairwise comparisons that are missing, and those are:
A - B
C - D
How can I calculate those comparisons?
EMMEANS in UNIANOVA does not provide all pairwise comparisons among the cells in an interaction like this. There are some other procedures, such as GENLIN, that do offer these, but use large-sample chi-square statistics rather than t or F statistics. In UNIANOVA, you can get these using the LMATRIX subcommand, or you can use some trickery with EMMEANS.
For the trickery with EMMEANS, create a single factor with four levels that index the 2x2 layout of cells, then handle that as a one-way model. The main effect for that is the same as the overall 3 degree of freedom model for the 2x2 layout, and of course EMMEANS with COMPARE works fine on that.
Without creating a new variable, you can use LMATRIX with:
/LMATRIX "(1,1) - (2,2)" var1 1 -1 var2 1 -1 var1*var2 1 0 0 -1
/LMATRIX "(1,2) - (2,1)" var1 1 -1 var1 -1 1 var1*var2 0 1 -1 0
The quoted pieces are labels, indicating the cells in the 2x2 design being compared.
Another trick you can use to make specifying the LMATRIX simpler, but without creating a new variable, is to specify the DESIGN with just the interaction term and suppress the intercept. That makes the parameter estimates just the four cell means:
UNIANOVA Y BY var1 var2
/INTERCEPT=EXCLUDE
/DESIGN var1*var1
/LMATRIX "(1,1) - (2,2)" var1*var2 1 0 0 -1
/LMATRIX "(1,2) - (2,1)" var1*var1 0 1 -1 0.
In this case the one effect shown in the ANOVA table is a 4 df effect testing all means against 0, so it's not of interest, but the comparisons you want are easily obtained. Note that this trick only works with procedures that don't reparameterize to full rank.
I have around 300 variables and I am calculating their Skewness and Kurtosis. Now, I want to create a new varaible which will consist of the list of all those variables whose Skewness and Kurtosis are within a certain range. The idea is to select only those variables which are satisfying a condition and perform normalization on all the other variables.
To calcualte Skewness i am using;
Descriptives A TO Z
/Statistics Skewness.
Execute.
I know this is not a valid Syntax but i Need something like this:
Compute x= if(Skewness(A TO Z)>1)
Please help me out with an SPSS Syntax for this.
There are multiple ways to approach this, so there might be an easier way.
you just need to change the 'var1 TO varN' to your list of variables and whatever criteria you want for Skewness & Kurtosis on the two COMPUTE lines that create the flags, and this will do it for you.
If I were doing this I would go a step further and build the normalization into the syntax using WRITE OUT = ".sps" /CMD. INSERT FILE = ".sps", but that isn't what you asked for.
DATASET DECLARE DistributionSyntax.
OMS
/SELECT TABLES
/IF SUBTYPES=["Descriptives"] INSTANCES=[1]
/DESTINATION FORMAT=SAV OUTFILE = 'DistributionSyntax'.
EXAMINE VARIABLES=var1 TO varN
/PLOT NONE
/STATISTICS DESCRIPTIVES
/CINTERVAL 95
/MISSING PAIRWISE
/NOTOTAL.
OMSEND.
DATASET ACTIVATE DistributionSyntax.
USE ALL.
FILTER OFF.
SELECT IF ANY(Var2,'Skewness','Kurtosis').
EXECUTE.
STRING VarName (A64).
COMPUTE SkewnessFlag = (Var2 = 'Skewness' AND ABS(Statistic) > 2).
COMPUTE KurtosisFlag = (Var2 = 'Kurtosis' AND ABS(Statistic) > 2).
COMPUTE VarName = CHAR.SUBSTR(Var1,1,CHAR.INDEX(Var1,' ')-1).
EXECUTE.
USE ALL.
COMPUTE filter_$=(SkewnessFlag = 1).
VALUE LABELS filter_$ 0 'Not Selected' 1 'Selected'.
FORMATS filter_$ (f1.0).
FILTER BY filter_$.
EXECUTE.
FRE VarName.
USE ALL.
COMPUTE filter_$=(KurtosisFlag= 1).
VALUE LABELS filter_$ 0 'Not Selected' 1 'Selected'.
FORMATS filter_$ (f1.0).
FILTER BY filter_$.
EXECUTE.
FRE VarName.
USE ALL.
FILTER OFF.
EXECUTE.
If you omit the select data blocks after you compute the flags and replace it with this, it will calculate normalized versions of the variables that meet your criteria. This calculates new variables, and you will want to add a file location for the syntax file (replace the "~/" in the WRITE and INSERT commands), and change the name of the dataset referenced as 'RAWDATA' to whatever your dataset name is:
USE ALL.
FILTER OFF.
SELECT IF ANY(1,SkewnessFlag,KurtosisFlag).
EXECUTE.
STRING CMD (A250).
COMPUTE CMD = CONCAT("COMPUTE ",RTRIM(VarName),".Norm = ln(",RTRIM(VarName),").").
EXECUTE.
DATA LIST /CMD 1-250 (A).
BEGIN DATA
EXECUTE.
END DATA.
DATASET NAME EXE WINDOW = FRONT.
DATASET ACTIVATE DistributionSyntax.
ADD FILES /FILE = *
/FILE = 'EXE'.
EXECUTE.
DATASET CLOSE EXE.
DATASET ACTIVATE DistributionSyntax.
WRITE OUT="~\Normalize Variables.sps" /CMD.
DATASET CLOSE DistributionSyntax.
DATASET ACTIVATE RAWDATA.
INSERT FILE="~\Normalize Variables.sps".
I work with SPSS and have difficulty finding/generating a syntax for counting cases.
I have about 120 cases and five variables. I need to know the count /proportion of cases where just one, more than one, or all of the cases have a value of 1 (dichotomous variable). Then I need to compute a new variable that shows the number / proportion of cases which include all of the aforementioned cases (also dichotomous).
For example case number one: var1=1, var2=1, var3=1, var4=0, var5=0 --> newvariable=1.
Case number two: var1=0, var2=0, var3=0, var4=0, var5=0 --> newvariable=1.
And so on...
Can anybody help me with a syntax?
Help would much appreciated!
Here we can use the sum of the variables to determine your conditions. So using a scratch variable that is the sum, we can see if it is equal to 1, more than 1 or 5 in your example.
compute #sum = SUM(var1 to var5).
compute just_one = (#sum = 1).
compute more_one = (#sum > 1).
compute all_one = (#sum = 5).
Similarly, all_one could be computed using the ANY command to evaluate if any zeroes exist, i.e. compute all_one = ANY(0,var1 to var5).. These code snippets assume that var1 to var5 are contiguous in the data frame, if not they just need to be replaced with var1,var2,var3,var4,var5 in all given instances.
You could read up on the logical function ANY in the Command Syntax Reference manual, if you negated a test for ANY with "0", then that is effectively a test for all "1"s. Use of the COUNT command would be another approach.
I'm puzzling over how to map a set of sequences to consecutive integers.
All the sequences follow this rule:
A_0 = 1
A_n >= 1
A_n <= max(A_0 .. A_n-1) + 1
I'm looking for a solution that will be able to, given such a sequence, compute a integer for doing a lookup into a table and given an index into the table, generate the sequence.
Example: for length 3, there are 5 the valid sequences. A fast function for doing the following map (preferably in both direction) would be a good solution
1,1,1 0
1,1,2 1
1,2,1 2
1,2,2 3
1,2,3 4
The point of the exercise is to get a packed table with a 1-1 mapping between valid sequences and cells.
The size of the set in bounded only by the number of unique sequences possible.
I don't know now what the length of the sequence will be but it will be a small, <12, constant known in advance.
I'll get to this sooner or later, but though I'd throw it out for the community to have "fun" with in the meantime.
these are different valid sequences
1,1,2,3,2,1,4
1,1,2,3,1,2,4
1,2,3,4,5,6,7
1,1,1,1,2,3,2
these are not
1,2,2,4
2,
1,1,2,3,5
Related to this
There is a natural sequence indexing, but no so easy to calculate.
Let look for A_n for n>0, since A_0 = 1.
Indexing is done in 2 steps.
Part 1:
Group sequences by places where A_n = max(A_0 .. A_n-1) + 1. Call these places steps.
On steps are consecutive numbers (2,3,4,5,...).
On non-step places we can put numbers from 1 to number of steps with index less than k.
Each group can be represent as binary string where 1 is step and 0 non-step. E.g. 001001010 means group with 112aa3b4c, a<=2, b<=3, c<=4. Because, groups are indexed with binary number there is natural indexing of groups. From 0 to 2^length - 1. Lets call value of group binary representation group order.
Part 2:
Index sequences inside a group. Since groups define step positions, only numbers on non-step positions are variable, and they are variable in defined ranges. With that it is easy to index sequence of given group inside that group, with lexicographical order of variable places.
It is easy to calculate number of sequences in one group. It is number of form 1^i_1 * 2^i_2 * 3^i_3 * ....
Combining:
This gives a 2 part key: <Steps, Group> this then needs to be mapped to the integers. To do that we have to find how many sequences are in groups that have order less than some value. For that, lets first find how many sequences are in groups of given length. That can be computed passing through all groups and summing number of sequences or similar with recurrence. Let T(l, n) be number of sequences of length l (A_0 is omitted ) where maximal value of first element can be n+1. Than holds:
T(l,n) = n*T(l-1,n) + T(l-1,n+1)
T(1,n) = n
Because l + n <= sequence length + 1 there are ~sequence_length^2/2 T(l,n) values, which can be easily calculated.
Next is to calculate number of sequences in groups of order less or equal than given value. That can be done with summing of T(l,n) values. E.g. number of sequences in groups with order <= 1001010 binary, is equal to
T(7,1) + # for 1000000
2^2 * T(4,2) + # for 001000
2^2 * 3 * T(2,3) # for 010
Optimizations:
This will give a mapping but the direct implementation for combining the key parts is >O(1) at best. On the other hand, the Steps portion of the key is small and by computing the range of Groups for each Steps value, a lookup table can reduce this to O(1).
I'm not 100% sure about upper formula, but it should be something like it.
With these remarks and recurrence it is possible to make functions sequence -> index and index -> sequence. But not so trivial :-)
I think hash with out sorting should be the thing.
As A0 always start with 0, may be I think we can think of the sequence as an number with base 12 and use its base 10 as the key for look up. ( Still not sure about this).
This is a python function which can do the job for you assuming you got these values stored in a file and you pass the lines to the function
def valid_lines(lines):
for line in lines:
line = line.split(",")
if line[0] == 1 and line[-1] and line[-1] <= max(line)+1:
yield line
lines = (line for line in open('/tmp/numbers.txt'))
for valid_line in valid_lines(lines):
print valid_line
Given the sequence, I would sort it, then use the hash of the sorted sequence as the index of the table.