Since Tableau does not have a function for P-values(correct me if I'm wrong here) I created a spreadsheet with all possible sample sizes under two different alphas/significance levels and need to connect the appropriate p-value to a calculated field from the main database source (aggregate count of people). I assumed I could easily match numbers with a condition to bring back the p-value in a calculated field yet I'm hitting a brick wall. Biggest issue seems to be that the field I want to join the P-value reference table to is an aggregated integer. Also, I do not have any extensions and my end result needs to be an integer, not a graph.
Any secret tricks here?
Seems I cannot blend the reference table in nor join it to an aggregate?
Thanks!
I found a work around in calculating the critical value for a two tailed t-test in tableau. However, I didn't figure out how to join based on an aggregated calculated field. Work around: I used a conditional statement just copying and pasting about 100 critical values based on (sample size - 2) aka degrees of freedom, into a calculated field. To save time, use excel to pull down the conditions to 120. Worked like a charm!
Here is the conditional logic for alpha = .2 (80%) in two tailed t-test (replace the ## line with about 117 rows):
IF [degrees of freedom] = 1 THEN 3.08
ELSEIF [degrees of freedom] = 2 THEN 1.89
ELSEIF [degrees of freedom] = 3 THEN 1.64
##ELSEIF [...calculate down to 120] = ... then ...
ELSEIF [degrees of freedom] > 121 THEN 1.28
END
Using InfluxDB, I'm trying produce an output that shows cumulative rainfall for a time period, that starts from zero.
The rainfall sensor outputs a cumulative rainfall amount, but resets to zero on power-failure, restart etc.
My first query component uses non_negative_difference() to show the increments.
select
non_negative_difference(rain) as nnd
FROM
weather
WHERE
$time_query
.... yields an increment per raw data point, for example:
2018-06-01T14:21:00.926Z 0
2018-06-01T14:22:02.959Z 0.30000000000000426
2018-06-01T14:23:04.992Z 0.3999999999999986
2018-06-01T14:24:07.024Z 0.10000000000000142
2018-06-01T14:25:09.059Z 0.19999999999999574
2018-06-01T14:26:11.094Z 0
2018-06-01T14:27:13.127Z 0.10000000000000142
2018-06-01T14:28:15.158Z 0.20000000000000284
2018-06-01T14:29:20.027Z 0.09999999999999432
2018-06-01T14:30:22.476Z 0.10000000000000142
2018-06-01T14:30:53.918Z 0.6000000000000014
2018-06-01T14:31:55.968Z 0.5
2018-06-01T14:32:58.007Z 0.5
2018-06-01T14:34:00.046Z 0.20000000000000284
2018-06-01T14:35:02.075Z 0.3999999999999986
2018-06-01T14:36:04.102Z 0.3999999999999986
2018-06-01T14:37:06.136Z 0.20000000000000284
2018-06-01T14:38:08.201Z 0
So far so good.
I'm now trying to stitch these readings back to cumulative total, starting from zero for the intended period.
I can use cumulative_sum() for this, for example:
SELECT
cumulative_sum(nnd)
FROM
(SELECT
non_negative_difference(rain) as nnd
FROM
weather
WHERE
$time_query )
which yields:
2018-06-01T14:21:00.926Z 0
2018-06-01T14:22:02.959Z 0.30000000000000426
2018-06-01T14:23:04.992Z 0.7000000000000028
2018-06-01T14:24:07.024Z 0.8000000000000043
2018-06-01T14:25:09.059Z 1
2018-06-01T14:26:11.094Z 1
2018-06-01T14:27:13.127Z 1.1000000000000014
2018-06-01T14:28:15.158Z 1.3000000000000043
2018-06-01T14:29:20.027Z 1.3999999999999986
2018-06-01T14:30:22.476Z 1.5
2018-06-01T14:30:53.918Z 2.1000000000000014
2018-06-01T14:31:55.968Z 2.6000000000000014
2018-06-01T14:32:58.007Z 3.1000000000000014
2018-06-01T14:34:00.046Z 3.3000000000000043
2018-06-01T14:35:02.075Z 3.700000000000003
2018-06-01T14:36:04.102Z 4.100000000000001
2018-06-01T14:37:06.136Z 4.300000000000004
2018-06-01T14:38:08.201Z 4.300000000000004
Looking good!
Now I'd like to group it up into more distinct time buckets, for nice graphing.
Let's try....
SELECT
cumulative_sum(max(nnd))
FROM (SELECT
non_negative_difference(rain) as nnd
FROM
weather
WHERE
$time_query)
GROUP BY
time(5m)
and I get an error: ERR: aggregate function required inside the call to non_negative_difference
But I cannot find a reasonable way of adding aggregates and groupings to non_negative_difference() that do not affect the accuracy of the differencing function itself.
The only thing I've been able to do is a dummy aggregate SUM() over time groups that are smaller than the sensor period. But this isn't robust enough for my liking - (and i'm still not sure it is 100% correct)
Is it correct that I must have both queries as aggregate queries?
I was trying to do this very thing for my weather station. Instead of having the weather station calculate the cumulative value I wanted Grafana to do it. The solution that worked for me is the advanced syntax Yuri Lachin mentions in his comments.
With InfluxDB you can use CUMULATIVE_SUM(), but the basic syntax doesn't allow you to group by time (only by tag). The "advanced syntax", however, allows you to to have a time series by nesting an aggregate function like MEAN() or SUM().
Here's the function I am using in Grafana to get a cumulative rainfall total for a selected time period:
SELECT CUMULATIVE_SUM(MEAN("rainfall")) FROM "weather" WHERE $timeFilter GROUP BY time(1h) fill(0).
The GROUP BY is, of course, flexible. I was interested in hourly rainfall so I grouped by 1h. You can group by the time period you find most interesting.
Using this query the rainfall will start from zero for period you select in Grafana. In the Seattle area we had measurable rain (I know, shocker) on 8/6/2020 and 8/8/2020. If I set my date range to include both dates the graph shows just under .2mm total rainfall:
If I switch my graph to 8/8 and 8/9 the total is just under 1mm:
Note: I was also interested in seeing the individual bucket tips so included those as bars on the second Y-axis.
For more detail see: https://docs.influxdata.com/influxdb/v1.8/query_language/functions/#advanced-syntax-7
I have pretty simple measurement in influxDB and have default time column and two other columns as shown below,
Select * from measurement
gives me this out put.
time component_id jkey
2016-09-27T02:49:17.837587671Z 3 "timestamp"
2016-09-27T02:49:17.849447239Z 3 "init_time"
2016-09-27T02:49:17.885999439Z 3 "ae_name"
2016-09-27T02:49:17.893056849Z 3 "init_time"
How can i select the last record of this measurement? The record which have maximum time value.
This can be done with last(). See the docs for more information: link. Or take a look at this example from the docs.
SELECT LAST("water_level") FROM "h2o_feet" WHERE "location" = 'santa_monica'
This will return the "newest" entry.
Is it possible to filter values from comlex metric in grafana?
For example:
SELECT sum(one) + sum(two) FROM "table" WHERE $timeFilter GROUP BY time($interval)
I need to show only positive sum sum(one) + sum(two) > 0
In sql I would use alias and HAVING clause like:
SELECT sum(one) + sum(two) AS S FROM "table" WHERE $timeFilter GROUP BY time($interval) HAVING S > 0
However that does not work in grafana.
How can I achieve this result without creating a new sum column in back-end database?
[EDIT]: My grafana GUI looks like this:
After clicking on "pen" button:
As of August 2016, the HAVING clause is not yet available in InfluxDB so finding all points where sum(one) + sum(two) > 0 is not possible directly in InfluxDB without using a continuous query to create an intermediate series.
However, Grafana does allow a minimum y-axis value to be set, which means any negative values will not be shown.
Hope that helps!
This answers just a part of your question but I was able to do the following in grafana+influxdb datasource:
Which selects the sum of one value if >0.
The problem is that it is not possible to select two different values in one query. But maybe you can workaround this problem with two querys in one graph.
I am analyzing an electronic survey I made using Google Forms and I have the following problem.
One of the questions can take multiple answers in the form of Checkboxes as shown in the picture below. The question is in Greek so I have added some Choice1, Choice2, Choice3 etc next to each answer in order to facilitate my question.
In my data when someone chose lets say Choice1 and Choice2,
I will have an answer which is the concatenation of the strings he checked seperated with commas.
In this case it would be:
Choice1, Choice2
If someone else checked Choice1, Choice2 and Choice4
his answer in my data would be:
Choice1, Choice2, Choice4
The problem is SPSS has no way of seperating the substrings (seperated by commas) and understanding which Choices each case has in common. Or maybe there is a way but I don't know it :)
When I, for example, do a simple frequency analysis for this question it produces a table that perceives
Choice1, Choice2
as a completely different case from
Choice1, Choice2, Choice4
Ideally I would like to somehow tell SPSS to count the frequency of each unique Choice (Choice1, Choice2, Choice3 etc etc) rather than each unique combination of those Choices.
Is that possible? And if it is can you point me to the documentation I need to study to make it happen?
Thx a lot!
Imagine you are working with the following data, which is a CSV file you have downloaded from your online form. Copy and paste the text below and save it to a text file named "CourseInterestSurvey.CSV".
Timestamp,Which courses are you interested in?,What software do you use?
12/28/2012 11:57:56,"Research Methods, Data Visualization","Gnumeric, SPSS, R"
12/28/2012 11:58:09,Data Visualization,"SPSS, Stata, R"
12/28/2012 11:59:09,"Research Dissemination, Graphic Design",Adobe InDesign
12/28/2012 11:59:27,"Data Analysis, Data Visualization, Graphic Design","Excel, OpenOffice.org/Libre Office, Stata"
12/28/2012 11:59:44,Data Visualization,"R, Adobe Illustrator"
Read it into SPSS using the following syntax:
GET DATA
/TYPE=TXT
/FILE="path\to\CourseInterestSurvey.CSV"
/DELCASE=LINE
/DELIMITERS=","
/QUALIFIER='"'
/ARRANGEMENT=DELIMITED
/FIRSTCASE=2
/IMPORTCASE=ALL
/VARIABLES=
Timestamp A19
CourseInterest A49
Software A41.
CACHE.
EXECUTE.
DATASET NAME DataSet2 WINDOW=FRONT.
LIST.
It currently looks like the image below--three columns (one timestamp, and two with the data we want):
Working with some syntax from here, we can split the cells up as follows:
* We know the string does not excede 50 characters.
* We got that information while we were reading our data in.
STRING #temp(a50).
* We're going to work on the "CourseInterest" variable.
COMPUTE #temp=CourseInterest.
* We're going to create 3 new variables with the prefix "CourseInterest".
* You should modify this according to the actual number of options your data has
* and the maximum length of one of the strings in your data.
VECTOR CourseInterest(3, a25).
* Here's where the actual variable creation takes place.
LOOP #i = 1 TO 3.
. COMPUTE #index=index(#temp,",").
. DO IF #index GT 0.
. COMPUTE CourseInterest(#i)=LTRIM(substr(#temp,1, #index-1)).
. COMPUTE #temp=substr(#temp, #index+1).
. ELSE.
. COMPUTE CourseInterest(#i)=LTRIM(#temp).
. COMPUTE #temp=''.
. END IF.
END LOOP IF #index EQ 0.
LIST.
The result:
This only addresses one column at a time, and I'm not familiar enough to modify it to work over multiple columns. However, if you were to switch over to R, I already have some readymade functions to help deal with exactly these kinds of situations.
Unfortunately there is no easy "built-in" way to achieve this, but it is certainly achievable with spreadsheet formulae, or Google Apps Script.
Using formulae, assuming your check box question lands in column D, this will produce a "normalised" list:
=ArrayFormula(TRANSPOSE(SPLIT(CONCAENATE(D2:D&",");",")))
and you can turn that into a two-column list and QUERY it to return a table of frequencies:
=ArrayFormula(QUERY(TRANSPOSE(SPLIT(CONCATENATE(D2:D&",");","))&{"",""};"select Col1, count(Col2) group by Col1 label Col1 'Item', count(Col2) 'Frequency'";0))
If your locale uses a comma as a decimal separator, replace {"",""} with {""\""}.
It is easy to split the fields into separate variables as described above. Now define these variables as a multiple response set (Analyze > Tables > Multiple Response Sets), and you can analyze these with the CTABLES or MULT REPONSE procedures and graph them using the Chart Builder