On mac i can do
sudo filebyproc.d 2>/dev/null | grep " cat "
running
cat balbalbla
i've got
sudo filebyproc.d 2>/dev/null | grep " cat "
Password:
8 178 open:entry cat /dev/dtracehelper
10 964 open_nocancel:entry cat /usr/share/locale/UTF-8/LC_CTYPE
2 178 open:entry cat balbalbla
which is the list of all files open by cat process.
How to do the same (or similar) in QNX?
Use pidin -p <process> fds. <process> may be specified as either a pid or the name of the binary (eg. 'cat').
Ref: http://www.qnx.com/developers/docs/7.1/#com.qnx.doc.neutrino.utilities/topic/p/pidin.html
Related
I have a space-separated file that looks like this:
$ cat in_file
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 Chal_sti_synt_C
GCF_000046845.1_ASM4684v1_protein.faa WP_004927566.1 Chal_sti_synt_C
GCF_000046845.1_ASM4684v1_protein.faa WP_004919950.1 FAD_binding_3
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 FAD_binding_3
I am using the following shell script utilizing grep to search for strings:
$ cat search_script.sh
grep "GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1" Pfam_anntn_temp.txt
grep "GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1" Pfam_anntn_temp.txt
The problem is that I want each grep command to return only the first instance of the string it finds exclusive of the previous identical grep command's output.
I need an output which would look like this:
$ cat out_file
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 Chal_sti_synt_C
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 FAD_binding_3
in which line 1 is exclusively the output of the first grep command and line 2 is exclusively the output of the second grep command. How do I do it?
P.S. I am running this on a big file (>125,000 lines). So, search_script.sh is mostly composed of unique grep commands. It is the identical commands' execution that is messing up my downstream analysis.
I'm assuming you are generating search_script.sh automatically from the contents of in_file. If you can count how many times you'll repeat the same grep command you can just use grep once and use head, for example if you know you'll be using it 2 times:
grep "foo" bar.txt | head -2
Will output the first 2 occurrences of "foo" in bar.txt.
If you have to do the grep commands separately, for example if you have other code in between the grep commands, you can mix head and tail:
grep "foo" bar.txt | head -1 | tail -1
Some other commands...
grep "foo" bar.txt | head -2 | tail -1
head -n displays the first n lines of the input
tail -n displays the last n lines of the input
If you really MUST always use the same command, but ensure that the outputs always differ, the only way I can think of to achieve this is using temporary files and a complex sequence of commands:
cat foo.bar.txt.tmp 2>&1 | xargs -I xx echo "| grep -v \\'xx\\' " | tr '\n' ' ' | xargs -I xx sh -c "grep 'foo' bar.txt xx | head -1 | tee -a foo.bar.txt.tmp"
So to explain this command, given foo as a search string and bar.txt as the filename, then foo.bar.txt.tmp is a unique name for a temporary file. The temporary file will hold the strings that have already been output:
cat foo.bar.txt.tmp 2>&1 : outputs the contents of the temporary file. If none is present, will output an error message to stdout, (important because if the output was empty the rest of the command wouldn't work.)
xargs -I xx echo "| grep -v \\'xx\\' " adds | grep -v to the start of each line in the temporary file, grep -v something excludes lines that include something.
tr '\n' ' ' replaces newlines with spaces, to have on a single string a sequence of grep -vs.
xargs -I xx sh -c "grep 'foo' bar.txt xx | head -1 | tee -a foo.bar.txt.tmp" runs a new command, grep 'foo' bar.txt xx | head -1 | tee -a foo.bar.txt.tmp, replacing xx with the previous output. xx should be the sequence of grep -vs that exclude previous outputs.
head -1 makes sure only one line is output at a time
tee -a foo.bar.txt.tmp appends the new output to the temporary file.
Just be sure to clear the temporary files, rm *.tmp, at the end of your script.
If I am getting question right and you want to remove duplicates based on last field of each line then try following(this should be easy task for awk).
awk '!a[$NF]++' Input_file
I'm using Docker client Version: 18.09.2.
When I run start a container interactively and run a date command, then pipe its output to hexdump for inspection, I'm seeing a trailing \n as expected:
$ docker run --rm -i -t alpine
/ # date | hexdump -c
0000000 T h u M a r 7 0 0 : 1 5
0000010 : 0 6 U T C 2 0 1 9 \n
000001d
However, when I pass the date command as an entrypoint directly and run the container, I get a \r \n every time there's a new line in the output.
$ docker run --rm -i -t --entrypoint=date alpine | hexdump -c
0000000 T h u M a r 7 0 0 : 1 6
0000010 : 1 9 U T C 2 0 1 9 \r \n
000001e
This is weird.
It totally doesn't happen when I omit -t (not allocating any TTY):
docker run --rm -i --entrypoint=date alpine | hexdump -c
0000000 T h u M a r 7 0 0 : 1 7
0000010 : 3 0 U T C 2 0 1 9 \n
000001d
What's happening here?
This sounds dangerous, as I use docker run command in my scripts, and if I forget to omit -t from my scripts, the output I'll collect from docker run command will have invisible/non-printible \r characters which can cause all sorts of issues.
tldr; This is a tty default behaviour and unrelated to docker. Per the ticket filed on github about your exact issue.
Quoting the relevant comments in that ticket:
Looks like this is indeed TTY by default translates newlines to CRLF
$ docker run -t --rm debian sh -c "echo -n '\n'" | od -c
0000000 \r \n
0000002
disabling "translate newline to carriage return-newline" with stty -onlcr correctly gives;
$ docker run -t --rm debian sh -c "stty -onlcr && echo -n '\n'" | od -c
0000000 \n
0000001
Default TTY options seem to be set by the kernel ... On my linux host it contains:
/*
* Defaults on "first" open.
*/
#define TTYDEF_IFLAG (BRKINT | ISTRIP | ICRNL | IMAXBEL | IXON | IXANY)
#define TTYDEF_OFLAG (OPOST | ONLCR | XTABS)
#define TTYDEF_LFLAG (ECHO | ICANON | ISIG | IEXTEN | ECHOE|ECHOKE|ECHOCTL)
#define TTYDEF_CFLAG (CREAD | CS7 | PARENB | HUPCL)
#define TTYDEF_SPEED (B9600)
ONLCR is indeed there.
When we go looking at the ONLCR flag documentation, we can see that:
[-]onlcr: translate newline to carriage return-newline
To again quote the github ticket:
Moral of the story, don't use -t unless you want a TTY.
TTY line endings are CRLF, this is not Docker's doing.
I am trying to filter out lines that don't contain the file names as below using the following command but I am not sure why line with permission denied keeps coming in my result. It should be gone when I have used grep -v "total|denied".
wc -l *.* | egrep -v "total|denied" | sort -nr -k1,1
wc: host.save: Permission denied
33301 apache-maven-3.5.3-bin.tar.gz
14149 jenkins-cli.jar
240 examples.desktop
19 list.py
19 interview_GL.sh
17 lines.txt
7 number.py
Only stdout gets passed to the pipe into grep but those error messages are on stderr
You can either forward stderr to /dev/null or send them to stdout aswell
Send errors to /dev/null:
wc -l * 2>/dev/null
Redirect errors to stdout:
wc -l * 2>&1 | grep -v dir
You obviously aren't allowed to read host.save file's contents, therefore the error coming from the first command.
Have you tried muting the errors instead?
wc -l *.* 2>/dev/null | egrep -v "total|denied" | sort -nr -k1,1
I'm trying to automate a JMX command using a tool called JMXterm.
The script looks like so:
#!/bin/bash
beanstemp="/tmp/jmx_beans"
read -r -p "Enter server name in FQDN " scrapername
bean1="BrokerName=localhost,Connection=Scraper_$scrapername,ConnectorName=openwire,Type=Connection"
echo beans -d $domain | $cmd > $beanstemp
idnum=$(grep "ID_$scrapername" $beanstemp | grep openwire | awk -F- '{print $2"-"$3}')
idname="$scrapername-$idnum"
bean2="BrokerName=localhost,Connection=ID_"$idname"2_0,ConnectorName=openwire,Type=Connection"
domain="org.apache.activemq"
server="scrapermq.sj.peer39.com:1099"
cmd="java -jar Downloads/jmxterm-1.0-alpha-4-uber.jar -l $server"
echo run stop -b $bean1 -d $domain | $cmd -l $server
echo run stop -b $bean2 -d $domain | $cmd -l $server
When I run the script I get the proper response for the first command but the second one fails.
Here's the output of sh -x:
itaig#iganot-lt:~$ sh -x jmx.sh
+ beanstemp=/tmp/jmx_beans
+ read -r -p Enter server name in FQDN scrapername
Enter server name in FQDN selvmnj27.domain.company.com
+ bean1=BrokerName=localhost,Connection=Scraper_selvmnj27.domain.company.com,ConnectorName=openwire,Type=Connection
+ echo beans -d
+
+ + grep openwire
grep ID_selvmnj27.domain.company.com+ awk -F- {print $2"-"$3}
/tmp/jmx_beans
+ idnum=
+ idname=selvmnj27.domain.company.com-
+ bean2=BrokerName=localhost,Connection=ID_selvmnj27.domain.company.com-2_0,ConnectorName=openwire,Type=Connection
+ domain=org.apache.activemq
+ server=scrapermq.sj.peer39.com:1099
+ cmd=java -jar Downloads/jmxterm-1.0-alpha-4-uber.jar -l scrapermq.sj.peer39.com:1099
+ echo+ run stop -b BrokerName=localhost,Connection=Scraper_selvmnj27.domain.company.com,ConnectorName=openwire,Type=Connectionjava -jar -d Downloads/jmxterm-1.0-alpha-4-uber.jar -l org.apache.activemq scrapermq.sj.peer39.com:1099 -l
scrapermq.sj.peer39.com:1099
Welcome to JMX terminal. Type "help" for available commands.
$>run stop -b BrokerName=localhost,Connection=Scraper_selvmnj27.domain.company.com,ConnectorName=openwire,Type=Connection -d org.apache.activemq
#calling operation stop of mbean org.apache.activemq:BrokerName=localhost,Connection=Scraper_selvmnj27.domain.company.com,ConnectorName=openwire,Type=Connection
#operation returns:
null
$>+ + echo runjava stop -b -jar Downloads/jmxterm-1.0-alpha-4-uber.jar BrokerName=localhost,Connection=ID_selvmnj27.domain.company.com-2_0,ConnectorName=openwire,Type=Connection -l scrapermq.sj.peer39.com:1099 -l -d scrapermq.sj.peer39.com:1099
org.apache.activemq
Welcome to JMX terminal. Type "help" for available commands.
$>run stop -b BrokerName=localhost,Connection=ID_selvmnj27.domain.company.com-2_0,ConnectorName=openwire,Type=Connection -d org.apache.activemq
#InstanceNotFoundException: org.apache.activemq:BrokerName=localhost,Connection=ID_selvmnj27.domain.company.com-2_0,ConnectorName=openwire,Type=Connection
$>itaig#iganot-lt:~$
"Operation returns: null" is the response I'm looking for.
The problem begins with this line:
echo beans -d $domain | $cmd > $beanstemp
When it runs the output of the command should be written to the $beanstemp file but the file stays empty.
When I run the command manually from terminal it works:
itaig#iganot-lt:~$ echo beans -d org.apache.activemq | java -jar Downloads/jmxterm-1.0-alpha-4-uber.jar -l scrapermq.domain.company.com:1099 > /tmp/jmx_beans
Welcome to JMX terminal. Type "help" for available commands.
$>beans -d org.apache.activemq
#domain = org.apache.activemq:
$>itaig#iganot-lt:~$ tail -5 /tmp/jmx_beans
org.apache.activemq:BrokerName=localhost,Type=Subscription,clientId=Scraper_selvmnj28.domain.company.com,consumerId=ID_selvmnj28.domain.company.com-45117-1431866382308-0_416_1_1,destinationName=SCRAPER_VIDEO_INPUT,destinationType=Queue,persistentMode=Non-Durable
org.apache.activemq:BrokerName=localhost,Type=Subscription,clientId=Scraper_selvmnj29.domain.company.com,consumerId=ID_selvmnj29.domain.company.com-52961-1431866382264-0_416_-1_1,destinationName=topic_//ActiveMQ.Advisory.TempQueue_topic_//ActiveMQ.Advisory.TempTopic,destinationType=Topic,persistentMode=Non-Durable
org.apache.activemq:BrokerName=localhost,Type=Subscription,clientId=Scraper_selvmnj29.domain.company.com,consumerId=ID_selvmnj29.domain.company.com-52961-1431866382264-0_416_1_1,destinationName=SCRAPER_VIDEO_INPUT,destinationType=Queue,persistentMode=Non-Durable
org.apache.activemq:BrokerName=localhost,Type=Subscription,clientId=Scraper_selvmnj30.domain.company.com,consumerId=ID_selvmnj30.domain.company.com-33036-1431866396456-0_416_-1_1,destinationName=topic_//ActiveMQ.Advisory.TempQueue_topic_//ActiveMQ.Advisory.TempTopic,destinationType=Topic,persistentMode=Non-Durable
org.apache.activemq:BrokerName=localhost,Type=Subscription,clientId=Scraper_selvmnj30.domain.company.com,consumerId=ID_selvmnj30.domain.company.com-33036-1431866396456-0_416_1_1,destinationName=SCRAPER_VIDEO_INPUT,destinationType=Queue,persistentMode=Non-Durable
itaig#iganot-lt:~$
So my question is: What am I doing wrong and why does the command work when running manually from terminal but not from the script?
Thanks in advance
Ok I found the problem.
Certain lines included variables which have not been declared in time.
After re-organizing the order of variable declaration the problem has been solved.
I use grep -L to get a list of files that do not contain a certain string. How can I see the content of those files? Just like:
grep -L "pattern" | cat
You can use xargs:
grep -L "pattern" | xargs cat
As read in man xargs --> build and execute command lines from standard input. So it will cat to those file names that grep -L returns.
You can use cat and use the output of grep -L...
cat $(grep -L "pattern" *.files )