I am splunk noob trying to write a query for a couple of hours but not successful so far.
I want to count the number of times the command 'install' was triggered and the exit code was '0'
Each install command writes log in a new file with format 'install_timestamp' so I am searching for source="install*"
Using 2 source files as example
source1:
event1:command=install
... //a couple of other events
event100:exit_code=0
source2:
event1:command=install -f
... //a couple of other events
event100:exit_code=0
In this case I want the result to be 1. Only 1 occurrence of exit_code=0 when command was 'install' (not -f)
The thing that's confusing me is that the information for command and exit_code is in different events, I can get each of the two events separately but able to figure out how to get the combined result.
Any tips on how can I achieve the result I want ? - Thanks!
It's a little crude but you could do something like this...
("One String" NOT "Bad String") OR "Another String" | stats count by source | where count > 1
It will basically give you a list of files that contain events matching both strings. For your example this would be something like...
("command=install" NOT "-f") OR "exit_code=0" | stats count by source | where count > 1
Related
Working an learning some Redis and I am running into an issue that I can't figure out why it is not working.
It is my understanding from some posts and documentation I read that I can store data with similar keys like this
redis.set("fruit:1", "apple")
# OK
redis.set("fruit:2", "banana")
# OK
and it is my understanding I should be able to get all fruit like this
redis.get("fruit:*")
But I am missing something because this keep returning null and I cannot figure out what I need to do to return something like
=> apple, banana
I was able to figure out, using scan (suggested in answer below as well) how to return all matching fruit keys, but what I need is to search for all matching fruit keys, and then return the values for them.
#redis.scan_each(match: "fruit:*") do |fruit|
Rails.logger.debug('even run?')
fruits << fruit
end
=> fruit:1, fruit:2 # What I need though is apple, banana
What solved it (at least so far) is using mGet
#redis.scan_each(match: "fruit:*") do |fruit|
Rails.logger.debug('even run?')
fruits << #redis.mGet(fruit)
end
As far as I know, you can not do that (it would be cool to be possible).
Instead you have to run a scan SCAN 0 MATCH "fruit:*" COUNT 500 where 500 is number of keys to parse in the first iteration. So you have to run multiple iterations... until you reach the end of your key count.
If you want to get all keys with a single iteration, first you need to know how many keys there are in your DB, so you can run a command to find the total amount of keys:
dbsize which you can use instead of the 500 value in my example.
redis-cli dbsize
or
redis-cli INFO Keyspace | grep ^db0 | cut -f 1 -d "," | cut -f 2 -d"="
To return all values from all keys you can use MGET
https://redis.io/commands/mget
Maybe you can also take a deeper look at data structures
Maybe the solution above is not the correct one, in your case you can also use a set just to add your fruits in there.
https://redis.io/commands/sadd
SCAN can be a very expensive operation as it must traverse all the keys, even if you use a pattern. Because Redis is single-threaded, while that SCAN is happening, nothing else can happen and other requests are blocked. If you only have a few keys, this is fine. But if you have a lot of keys, this can cause serious performance issues.
What you need is an index that contains the keys of all of your fruits. You could use a Set to do this and whenever you add a fruit, you add the key to the Set:
> SET fruit:1 apple
> SADD fruit:index 1
Then, when you need all the fruits, you just get the members of the Set:
> SMEMBERS fruit:index
1) "1"
2) "2"
Then get the values of the fruits:
> GET fruit:1
> GET fruit:2
I have the following situation: a loop (stack data) with only 1 index variable and with multiple items corresponding to the statements, as in the picture below (sorry it is Excel, but is the same as in SPSS):
stack data - cases on multiple lines, but never filling for 1 respondent all the columns
I want to reach to the following situation but without using casestovars to restructure, because that creates a lot of empty variables. I remember for older versions it was a command like Update, which was moving up the cases, to reach the following result:
reducing the cases per respondent
Like starting from this:
ID Index Q1_1 Q1_2 Q1_3 Q1_4 Q1_5 Q1_6
1 1 1 1
1 2 1 1
1 3 1 1
To reach to this:
ID Q1_1 Q1_2 Q1_3 Q1_4 Q1_5 Q1_6
1 1 1 1 1 1 1
But without using casestovars. Is there any command in SPSS syntax for this?
Thank you very much, have a nice day!
Not entirely sure how variable your data structure is likely to be in reality but if as demo'ed where you have only a single response for each q1_1 to q1_6 per respondent ID, then the below would be sufficient:
dataset declare dsAgg.
aggregate outfile="dsAgg" /break=respid /q1_1 to q1_6=max(q1_1 to q1_6).
Also not sure of the significance of duplicate index values within the same respondent IDs, if this was intended or not.
The following syntax could do the job -
* first we'll recreate your example data.
data list list/respid index q1_1 to q1_6.
begin data
1,1,1,,,,,
1,2,,2,,,,
1,3,,,1,,,
1,4,,,,2,,
1,5,,,,,1,
1,6,,,,,,2
2,1,3,,,,,
2,1,,4,,,,
2,2,,,5,,,
2,2,,,,4,,
2,3,,,,,3,
2,3,,,,,,2
end data.
* now to work: first thing is to make sure the data from each ID are together.
sort cases by respid index.
* the loop will fill down the data to the last line of each ID.
do repeat qq=q1_1 to q1_6.
if respid=lag(respid) and missing(qq) qq=lag(qq).
end repeat.
* the following lines will help recognize the last line for each ID and select it.
compute lineNR=$casenum.
aggregate /outfile=* mode=ADDVARIABLES/break=respid/MXlineNR=max(lineNR).
select if lineNR=MXlineNR.
exe.
I have the following monitoring stack:
collecting data with telegraf-0.12
storing in influxdb-0.12
visualisation in grafana (3beta)
I am collecting "system" data from several hosts and I want to create a graph showing the "system.load1" of several host NOT merged. I though I could simply add multiple queries to the graph panel.
When creating my graph panel, I create the first serie and see the result but when I add the second query, I got an error.
Here is the panel creation with 2 queries
Here is the query generated by the panel:
SELECT mean("load1") FROM "system" WHERE "host" = 'xxx' AND time > now() - 24h GROUP BY time(1m) fill(null) SELECT mean("load1") FROM "system" WHERE "host" = 'yyy' AND time > now() - 24h GROUP BY time(1m) fill(null)
And the error:
{
"error": "error parsing query: found SELECT, expected ; at line 2, char 1",
"message": "error parsing query: found SELECT, expected ; at line 2, char 1"
}
So I can see that the generated query is malformed (2 select in one line without even a ';') but I don't know how to use Grafana to achieve what I want.
When I show or hide each query individually I see the corresponding graph.
I have created a similar graph (with multiple series) with chronograf but I would rather use grafana as I have many more control and plugins...
Is there something I am doing wrong here ?
After reading couple of thread in github issues, here is a quick fix.
As mentionned by #schup, the problem and its solution are described here:
https://github.com/grafana/grafana/issues/4533
The binaries are currently not fixed in grafana-3beta (if might in the next weeks). So there are 2 options: fixing the source and compile or patched an existing install.
I actually had to patch my current install:
/usr/share/grafana/public/app/app.<number_might_differ_here>.js
sed --in-place=backup 's/join("\\n");return k=k.replace/join(";\\n");return k=k.replace/;s/.replace(\/%3B\/gi,";").replace/.replace/' app.<number_might_differ_here>.js
Hope this might help (and that it will soon be fixed)
Seems to be an API change in influxdb 0.11
https://github.com/grafana/grafana/issues/4533
I have to look up a list of thousands of gene names (genelist.txt, one column) in a database file called (database.txt, multiple columns). Any lines containing at least one gene name that match the genelist.txt will be extract to output.txt.
I used to do it like this:
findstr /G:genelist.txt database.txt >output.txt
It works well and fast. However, I just found out today that the final output is affected by the gene order in the original genelist.txt. There is one result if using an unsorted gene list, and another result with more lines if sorting the gene list and searching again. But even with the sorted gene list the file output.txt does still not contain all lines as I'm missing some records. I just noticed this after a comparison with the result from
grep -f "genelist.txt" database.txt > output.txt
The results from grep have no difference no matter the gene list is sorted or not, but it's a bit slower than findstr.
I was wondering how this come. Can I add any arguments to findstr to make it return a complete results list?
I have a list of classes that extracts info from the web. Every time each one of them saves something, it sends a different counter to graphite. So, every one of them is a different metric.
How do I know how many of them satisfy a certain condition??
For example, let:
movingAverage(summarize(groupByNode(counters.crawlers.*.saved, 2, "sumSeries), "1hour"), 24)
be the average of content download in past 24 hours. How can i know, at a moment "t", how many of my metrics have this value above 0?
In the rendering endpoint, add format=json. This would return the data-points with corresponding epochs in JSON, which is a breeze to parse. The time-stamps wherein your script sent nothing will be NULL.
[{
"target": "carbon.agents.ip-10-0-0-228-a.metricsReceived",
"datapoints":
[
[912, 1383888170],
[789, 1383888180],
[800, 1383888190],
[null, 1383888200],
[503, 1383888210],
[899, 1383888220]
]
}]
You can use the currentAbove function. Check it out. For example:
currentAbove(stats.route.*.servertime.*, 5)
The above example gets all of the metrics (in the series) that are above 5.
You could then count the number of targets returned, and while Graphite doesn't provide a way to count the "buckets" you should be able to capture it fairly easily.
For example, a way to get a quick count (using curl to pipe to grep and count on the word "target"). Like so:
> curl -silent http://test.net/render?target=currentAbove(stats.cronica.dragnet.messages.*, 5)&format=json \
> | grep -Po "target" | grep -c "target"