How do I escape these kinds of special character in curl. The below is passed in the query part. I am trying to query splunk data and this is reg expression with rex command. Without the below characters it works fine. But I need this to be part of the query.
(?i)^(?:[^-]*-){}\s+\d+
I tried giving -g to stop globbing. But that doesn't work. Is there any simpler way to do this. I am passing --data urlencode in curl so encoding is automatically taken care.
I recently encountered the exact same problem statement. Splunk was having an issue with my regex when it contained [ or ] characters. I was also using curl, and I was sending my search query with curl's -d parameter for post data. I tried several variations of encoding of the brackets (escaping them, percent-encoding them, etc.), but to no avail.
The solution in my case was to use the --data-urlencode parameter instead of the --data parameter.
From the curl help:
$ curl --help | grep "\-\-data"
-d/--data <data> HTTP POST data (H)
--data-ascii <data> HTTP POST ASCII data (H)
--data-binary <data> HTTP POST binary data (H)
--data-urlencode <name=data/name#filename> HTTP POST data url encoded (H)
Related
Hi i have a few archive of FW log and occasionally im required to compare them with a series of IP addresses (thousand of them) to get the date and time if the ip addresses matches. my current script is as follow:
#input the list of ip into array
mapfile -t -O 1 var < ip.txt while true
do
#check array is not null
if [[-n "${var[i]}"]] then
zcat /.../abc.log.gz | grep "${var[i]}"
((i++))
It does work but its way too slow and i would think that grep-ping a line with multiple strings would be faster than zcat on every ip line. So my question is is there a way to generate a 'long grep search string' from the ip.txt? or is there a better way to do this
Sure. One thing is that using cat is usually slightly inefficient. I'd recommend using zgrep here instead. You could generate a regex as follows
IP=`paste -s -d ' ' ip.txt`
zgrep -E "(${IP// /|})" /.../abc.log.gz
The first line loads the IP addresses into IP as a single line. The second line builds up a regex that looks something like (127.0.0.1|8.8.8.8) by replacing spaces with |'s. It then uses zgrep to search through abc.log.gz once, with that -Extended regex.
However, I recommend that you do not do this. Firstly, you should escape strings put into a regex. Even if you know that ip.txt really contains IP addresses (e.g. not controlled by a malicious user), you should still escape the periods. But rather than building up a search string and then escape it, just use the -Fixed strings and -file features of grep. Then you get the simple and fast one-liner:
zgrep -F -f ip.txt /.../abc.log.gz
I'm trying to using grep to try to find things from a given pattern. For instance I have these lines:
A secret word: CoolKapplan
A secret word: Kapplan
A secret word: Bungyjump
So if I get to know the first and last letter of a word. In this example I get 'K' - 'n'.
PATTERN = K.....n
I do this: grep -w -r -H --color=always "^$PATTERN" *
And I except it to only give me the lines containing the patterns that are starting with K. But that command would also include the first line, so the result would be:
A secret word: CoolKapplan
A secret word: Kapplan
How do I make it so it searches for a pattern and not give me the pattern that is included in another word?
After some more trial and error attempts I found out that you have to add '-o' flag for it to work.
In my data set, we have a multitude of emails that must be parsed (alongside a myriad of other unrelated information like phone numbers and addresses and such.)
I am attempting to look for something that meets the criteria of an email, but does not have the proper format of an email. So, I tried using grep's "AND" function, wherein it fits the second parameter but not the first.
grep -E -c -v "^[a-mA-M][a-zA-Z]*\.#[A-Za-z]+\.[A-Za-z]{2,6}"Data.bash | grep # Data.bash
How should I be implementing this? As this just finds anything with an # in it (as the first parameter returns 0 and the second is just finding everything with an # in it).
In short, How do I AND two conditions together in grep?
EDIT: Sample Data
An email address has a user-id and domain names can consist of letters, numbers,
periods, and dashes.
Matches:
saltypickle#gmail.com
saltypickle#g-mail.com
No Match:
saltypickle#g^mail.com
saltypickle#.
#saltyPickle#
saltyPickle#
grep -P '^\w+#[[:alnum:]-.]+.com' inputfile
saltypickle#gmail.com
saltypickle#g-mail.com
This will allow any alpha ,number, - or . as domain name.
Following will print invalid email addresses:
grep -vP '^\w+#[[:alnum:]-.]+.com' inputfile
saltypickle#g^mail.com
saltypickle#.
#saltyPickle#
saltyPickle#
i tried to rendor json format in ruby on rails
My code:(ruby code)
:address=> 'Address:\nshollinganallur,\nchennai.'
Rendered json output:
"address": "Address:\\nshollinganallur,\\nchennai."
I tried with old questions and answer. but nothing happend. Any help?
There are no newlines in this:
:address=> 'Address:\nshollinganallur,\nchennai.'
The \n escape sequence doesn't work in single quoted strings so you're just getting the two characters \ and n.
When that's converted to JSON, the \ has a meaning so it has to be escaped with another \. Hence the output you're seeing.
If you start with:
:address => "Address:\nshollinganallur,\nchennai."
then you'll get your newlines all the way through.
Is there any way to do the opposite of showing only the matching part of strings in grep (the -o flag), that is, show everything except the part that matches the regex?
That is, the -v flag is not the answer, since that would not show files containing the match at all, but I want to show these lines, but not the part of the line that matches.
EDIT: I wanted to use grep over sed, since it can do "only-matching" matches on multi-line, with:
cat file.xml|grep -Pzo "<starttag>.*?(\n.*?)+.*?</starttag>"
This is a rather unusual requirement, I don't think grep would alternate the strings like that. You can achieve this with sed, though:
sed -n 's/$PATTERN//gp' file
EDIT in response to OP's edit:
You can do multiline matching with sed, too, if the file is small enough to load it all into memory:
sed -rn ':r;$!{N;br};s/<starttag>.*?(\n.*?)+.*?<\/starttag>//gp' file.xml
You can do that with a little help from sed:
grep "pattern" input_file | sed 's/pattern//g'
I don't think there is a way in grep.
If you use ack, you could output Perl's special variables $` and $' variables to show everything before and after the match, respectively:
ack string --output="\$`\$'"
Similarly if you wanted to output what did match along with other text, you could use $& which contains the matched string;
ack string --output="Matched: $&"