I have multiple files (six files) in csv format. I am trying to compare $3,$4,$5 across multiple files and if match print $6 from all files along with column $2,$3,$4,$5 from file 1.
Input file 1:
Blink,Seeddensity(g/cm^3),1_0002,VU10,37586764,0.458533399568206
Blink,Seeddensity(g/cm^3),1_0004,VU08,37687622,0.548181169267479
Blink,Seeddensity(g/cm^3),1_0006,VU02,6629660,0.553099787284982
Input file 2:
Farmcpu,Seeddensity(g/cm^3),1_0002,VU10,37586764,0.907010463957269
Farmcpu,Seeddensity(g/cm^3),1_0004,VU08,37687622,0.782521980037194
Farmcpu,Seeddensity(g/cm^3),1_0006,VU02,6629660,0.589126094555234
Input file 3:
GLM,Seeddensity(g/cm^3),1_0002,VU10,37586764,0.24089
GLM,Seeddensity(g/cm^3),1_0004,VU08,37687622,0.25771
GLM,Seeddensity(g/cm^3),1_0006,VU02,6629660,0.31282
Desired output:
Trait Marker Chr Pos Blink Farmcpu GLM
Seeddensity(g/cm^3) 2_27144 VU08 36984438 1.7853934213866E-11 0.907010463957269 0.24089
Seeddensity(g/cm^3) 2_13819 VU08 21705264 3.98653459293212E-09 0.782521980037194 0.25771
Seeddensity(g/cm^3) 2_07286 VU01 38953729 3.16663946775461E-07 0.589126094555234 0.31282
I have checked multiple awk commands but this is the closest one do a job across two files:
awk 'NR==FNR{ a[$2,$3,$4,$5]=$1; next } { s=SUBSEP; k=$2 s $3 s $4 s $5 }k in a{ print $0,a[k] }' File1 File2 > output
join <(sort File1) <(sort File2) | join - <(sort File3) | join - <(sort File4) | join - <(sort File5) | join - <(sort File6) > output
I believe join is not working as first column is not same across files so I tried this command:
join -t, -j3 -o 1.2,1.3,1.4,1.5,1.6,2.6,3.6,4.6,5.6,6.6 <(sort -k 3 File1) <(sort -k 3 File2) <(sort -k 3 File3) <(sort -k 3 File4) <(sort -k 3 File5) <(sort -k 3 File6) > output
But I am getting an error msg:
join: invalid file number in field spec: ‘3.6’
For two files the following command works, but I am not sure how to use it for multiple files:
join -t, -j3 -o 1.2,1.3,1.4,1.5,1.6,2.6 <(sort -k 3 File1) <(sort -k 3 File2) > output
Assuming you actually want CSV output then using GNU awk for ARGIND:
$ cat tst.awk
BEGIN { FS=OFS="," }
{ key = $3 FS $4 FS $5 }
ARGIND < (ARGC-1) {
val[key,ARGIND] = $6
next
}
{
sfx = ""
for (i=1; i<ARGIND; i++) {
if ( (key,i) in val ) {
sfx = sfx OFS val[key,i]
}
else {
next
}
}
print $2, $3, $4, $5, $6 sfx
}
$ awk -f tst.awk file2 file3 file1
Seeddensity(g/cm^3),1_0002,VU10,37586764,0.458533399568206,0.907010463957269,0.24089
Seeddensity(g/cm^3),1_0004,VU08,37687622,0.548181169267479,0.782521980037194,0.25771
Seeddensity(g/cm^3),1_0006,VU02,6629660,0.553099787284982,0.589126094555234,0.31282
With any other awk just add a line that's FNR==1 { ARGIND++ } at the start of the script.
I am parsing a text file
Lines File Name Gen LnkLN LINK Time
----- -------------------- ---- ----- ---- ------------------------
00090 TEST1_1519230912 0 00092 .X.X Wed Feb 21 16:35:14 2018
00091 TEST2_1619330534 0 00093 .X.X Wed Feb 21 16:35:14 2018
using code
awk '{if (($1 ~ /^[0-9A-Fa-f]+$/) && (length($1)==5)) {
if (! c[$4]) TLN=TLN $4 ","
c[$4]=$4;
if (! d[$3]) TGN=TGN $3 ","
d[$3]=$3
if (! b[$2]) TLNK=TLNK $2 ","
b[$2]=$2
}
} END {print "TLines="TLN,"TGEN="TGN,"TLink="TLNK}' /var/tmp/slink.jnk
I get O/p
TLines=00092,00093, TGEN=0,0, TLink=TEST1_1519230912,TEST2_1619330534,
I have two questions with this.
First one is I don't understand why value for TGN is being printed twice in the output "0,0,". If file has duplicate value for the field I want only one value in the o/p.
Second, I redirect these o/p into another file and use #source filename.txt command to set these values as environment variables and use them in later part of the script. Is there any better way to use them as variables inside the script rather than creating another file and sourcing it.
Use in to see if a value is being repeated to avoid the case where the value itself evaluates to false. That is what's happening with your 0 value and why it's being repeated in your output.
$ awk '{if (($1 ~ /^[0-9A-Fa-f]+$/) && (length($1)==5)) {
if (!($4 in c)) TLN=TLN $4 ","
c[$4]
if (!($3 in d)) TGN=TGN $3 ","
d[$3]
if (!($2 in b)) TLNK=TLNK $2 ","
b[$2]
}
} END {print "TLines="TLN,"TGEN="TGN,"TLink="TLNK}' f
Output:
TLines=00092,00093, TGEN=0, TLink=TEST1_1519230912,TEST2_1619330534,
EDIT
Above I've kept things close to your original version, but as mentioned in the comments, a more idiomatic and nicer version would be:
$ awk '($1 ~ /^[0-9A-Fa-f]+$/) && (length($1)==5) {
if (!c[$4]++) TLN=TLN $4 ","
if (!d[$3]++) TGN=TGN $3 ","
if (!b[$2]++) TLNK=TLNK $2 ","
} END {print "TLines="TLN,"TGEN="TGN,"TLink="TLNK}' f
END EDIT
For setting the variables, this worked for me (where a.awk contains the awk code, above):
$ eval "$(awk -f a.awk f)"
$ echo $TLines
00092,00093,
$ echo $TGEN
0,
$ echo $TLink
TEST1_1519230912,TEST2_1619330534,
I have created an alias to open an older git version of a file in sublime text.
My alias accepts 2 arguments, $1 is the revision and $2 is the file path,
but I would like to parse the arguments to get the file extension of the file to open it in its correct format.
My existing code is here:
revise=git show $1:$2 > redirected.txt $T C:/PROGRA~1/SUBLIM~2/sublime_text.exe redirected.txt
However I would like to to something like:
revise=git show $1:$2 > redirected.{$2.extesnion} $T C:/PROGRA~1/SUBLIM~2/sublime_text.exe redirected.{$2.extesnion}
environment details:
cmder version: 1.3.15.1010
in cmd.exe session you can read parameters from the command line using the $* placeholder to get everything that comes after the alias, alias example:
vi=vim $*
alias usage:
D:\
λ vi test.txt
Or you can read space-delimited argument using placeholders $1, $2, $n... alias example:
example= echo param one: $1 param two: $2
alias usage:
D:\
λ example hi how are you?
alias result:
param one: hi param two: how
note -are- and -you?- where no echoed
I'm running my program installer done with NSIS but the /D option seems not working (or better overridden inside).
Via command line I trigger:
installer.exe /S /D=C:\Users\Public\installDir
NSIS code is:
InstallDir "C:\Users\Public\${VERSIONSTR}"
Function xxxx
${If} $MultiUser.InstallMode == "AllUsers"
StrCpy $INSTDIR "C:\Users\Public\Dir1"
${EndIf}
IfSilent 0 +20
StrCpy $INSTDIR "C:\Userdata\Dir2"
FunctionEnd
The used installation folder is "C:\Users\Public\Dir2". Even if I comment the IfSilent block the installtion folder will be "C:\Users\Public\Dir1" but never the one passed via command line.
What's wrong in my script?
!include LogicLib.nsh
;InstallDir ; Do not use InstallDir at all so we can detect empty $InstDir
!define DEFDIR_MACHINE "$programfiles\foo"
!define DEFDIR_PERUSER "$localappdata\bar"
Function .onInit
${If} $InstDir == "" ; /D not used
${If} $MultiUser.InstallMode == "AllUsers"
StrCpy $InstDir "${DEFDIR_MACHINE}"
${Else}
StrCpy $InstDir "${DEFDIR_PERUSER}"
${EndIf}
${If} ${Silent}
StrCpy $InstDir "c:\CrazySilentOverride"
${EndIf}
${EndIf}
FunctionEnd
I am having a problem with updating the PATH variable. What I need to do is update the path (which works fine), and then immediately use the new version in the installer. This is not working.
Here is what I'm using to update the path:
!macro ADD_TO_PATH pathAdd
DetailPrint "Adding ${pathAdd} to the system PATH."
ReadRegStr $1 ${WriteEnvStr_RegKey} "PATH"
WriteRegStr ${WriteEnvStr_RegKey} "PATH" "$1;${pathAdd}"
SendMessage ${HWND_BROADCAST} ${WM_WININICHANGE} 0 "STR:Environment" /TIMEOUT=5000
ReadRegStr $1 ${WriteEnvStr_RegKey} "PATH"
MessageBox MB_OK "Path: $1"
!macroend
And here is what I need to do with it:
Section "${PRODUCT_NAME}" SEC_R
DetailPrint 'Installing Python Dateutil...'
!insertmacro EXEC_OUT 'dtutil' 'easy_install python-dateutil' 'DateUtil' 'true'
SectionEnd
Where EXEC_OUT is the following:
; Silent execution of easy_install.
; abrt - is set to 'true', causes Abort on failure.
; name - user-friendly name to print
; package - unique name for labels
; what - full command to execute(ex: "easy_install packageXYZ")
!macro EXEC_OUT package what name abrt
MessageBox MB_OK "what: ${what}"
ExecWait "${what}" $0
${If} $0 == "0"
DetailPrint "${name} module installed successfully."
${Else}
DetailPrint "${name} failed to install: $0"
${If} ${abrt} == "true"
abort "An essential part of the installation, ${name}, failed to install. Aborting installation."
${EndIf}
${EndIf}
!macroend
Why is this not working? Is it not possible for the PATH to be updated before the application exits?
You need to set the environment variable for the running process as well.
http://nsis.sourceforge.net/Setting_Environment_Variables_to_Active_Installer_Process