I have a TXT file:
HISEQ1:105:C0A57ACXX:2:1101:10000:105587/1
HISEQ1:105:C0A57ACXX:2:1101:10000:105587/2
HISEQ1:105:C0A57ACXX:2:1101:10000:121322/1
HISEQ1:105:C0A57ACXX:2:1101:10000:121322/2
HISEQ1:105:C0A57ACXX:2:1101:10000:12798/1
HISEQ1:105:C0A57ACXX:2:1101:10000:12798/2
and a fasta file with sequences:
>HISEQ1:105:C0A57ACXX:2:1101:10000:105587/1
GCACCCTCGGGGGAGCAACGAAGAGGTAGACGAAGGCGATCGCAGCCACCTGCGGCAGTATCCCCAGGAGGTCAAGGTCCTCCTCCCCGCTCACCGTCGCC
>HISEQ1:105:C0A57ACXX:2:1101:10000:105587/2
TTGGTGGCAGGCAACAGCTTTGGACGGCCACCGCCTCATGGCGCCTCCTCCCCGCTGCGTCCTCGCCGCGTCCCTCCCTGCTTCAAGC
>HISEQ1:85:D0C0FABXX:5:1101:1385:36009/1
TTTAGTTCCAGGCCGGCTGAAGACTGTCTTTACAAAAGAAAAGTTTAGCCTAGGAAGCCCATTGTTGTAGGTGTTGTAGTTTTATAGATGTACTTTGGAAA
>HISEQ1:85:D0C0FABXX:5:1101:1385:36009/2
CAGCCAAGTTCGCAGTCTCGATAGTATTGTTTTCATACAGCAGTCTTGACAAACCAAAGTCCGCAACTTTTGGTTCCAGATTATCATCTAGCAATATGTTT
>HISEQ1:105:C0A57ACXX:2:1101:10000:105587/2
TTGGTGGCAGGCAACAGCTTTGGACGGCCACCGCCTCATGGCGCCTCCTCCCCGCTGCGTCCTCGCCGCGTCCCTCCCTGCTTCAAGC
I would like to extract the sequences of these ID's only once from the fasta file and get this output:
>HISEQ1:105:C0A57ACXX:2:1101:10000:105587/1
GCACCCTCGGGGGAGCAACGAAGAGGTAGACGAAGGCGATCGCAGCCACCTGCGGCAGTATCCCCAGGAGGTCAAGGTCCTCCTCCCCGCTCACCGTCGCC
>HISEQ1:105:C0A57ACXX:2:1101:10000:105587/2
TTGGTGGCAGGCAACAGCTTTGGACGGCCACCGCCTCATGGCGCCTCCTCCCCGCTGCGTCCTCGCCGCGTCCCTCCCTGCTTCAAGC
but I get also dublicates.
I tried these:
seqkit grep -f in.txt in.fa > out.fa
seqtk subseq in.fa in.txt > out.fa
How to modify the command line above to get unique sequences?
Try with
seqkit grep -f in.txt in.fa | seqkit rmdup -n -o out.fa
Related
I am using grep to extract lines from file 1 that matches with string in file2. The string in file 2 has both alphabets and numbers. eg;
MSTRG.18691.1
MSTRG.18801.1
I used sed to write word boundaries for all the strings in the file 2.
file 2
\<MSTRG.18691.1\>
\<MSTRG.18801.1\>
and used grep -f file2 file1
but output has
MSTRG.18691.1.2
MSTRG.18801.1.3 also..
I want lines that matches exactly,
MSTRG.18691.1
MSTRG.18801.1
and not,
MSTRG.18691.1.2
MSTRG.18801.1.3
Few lines from my file1
t_name gene_name FPKM TPM
MSTRG.25.1 . 0 0
rna71519 . 93.398872 194.727926057583
gene34024 ND1 2971.72876 6195.77694943117
MSTRG.28.1 . 0 0
MSTRG.28.2 . 0 0
rna71520 . 33.235409 69.2927240732149
Updating the answer
You can use start with ^ and end with $ operator to match start with and begin with. To match exactly MSTRG.18691.1 you can add ^ & $ at both ends and remove the word boundaries, additionally . has special meaning in regex to match exactly . we need to escape that with a backslash \
Example pattern:
^MSTRG\.18691\.1$
^MSTRG\.18801\.1$
file1
MSTRG.18691.1
MSTRG.1311.1
MSTRG.18801.2
MSTRG.18801.3
MSTRG.18801.1.2
MSTRG.18801.1.1
MSTRG.18801.1
PrefixMSTRG.18801.1
Just create a normal file named file1 and paste the above content into it.
file2 (pattern file)
^MSTRG\.18801\.1$
Just create a normal file named file2 and paste the above content into it.
Run the below command from commandline
grep -i --color -f file2 file1
Result:
MSTRG.18801.1
Sed to add changes to the pattern file
Here is the sed command to escape . and add ^ and $ at the beginning and end of the pattern file you already have.
sed -Ee 's/\./\\./g' -e 's/^/\^/g' -e 's/$/\$/g' file2 > file2_updated
-E to support extended regex on BSD sed, you may need to replace -E with -r based on your system's sed
Updated patterns will be saved to file2_updated. Need to use the new pattern file in grep like this
grep -i -f file2_updated file1
The flag you're looking for is -F. From man grep:
-F, --fixed-strings
Interpret PATTERN as a list of fixed strings (instead of regular expressions), separated by newlines, any of which is to be matched.
You can use this quite comfortably in conjunction with -f:
grep -Ff file2 file1
To be clear, this will treat every line of file2 as an exact match against file1.
I have this kind of files:
file1 :
TMCS09g1008676.1
MAMBO.3.3.2.1
TMCS09g1008678.1
TMCS09g1008678.2
CSH.1.2
TMCS09g1008681.3
TMCS09g1008682.1
TMCS09g1008683
file2:
TMCS09g1008676
MAMBO.3.3.2
TMCS09g1008678
TMCS09g1008679
CSH.1
TMCS09g1008681
TMCS09g1008682
TMCS09g1008683
What I want to do is to retrieve only the matching part of file 2 in file 1. Basically only the "red" part of the terminal showing the matching with the command grep -w -F -f file2 file1
Is there a way to achieve this?
thanks in advance
grep -f file.txt input.csv I tee output.csv
This hangs as if it is processessing; however, after a few minutes, it stops. It creates an empty output file.
file.txt is a file with email addresses listed with \n delimiter such as:
Johndoe#yahoo.com
Janedoe#hotmail.com
Manny#gmail.com
This file has about 30,000 lines.
input.csv is a comma delimited file such as:
John,doe,johndoe#yahoo.com,address,dob,etc
...and has about 50,000 lines.
output.csv should only have rows that contain the email addresses listed in file.txt, preserving the contents of each row.
Please help me!
Okay I have a file that contains numbers like this:
L21479
What I am trying to do is use grep (or a similar tool) to find all the strings in a file that have the format:
L#####
The # will be the number. SO an L followed by 5 numbers.
Is this even possible in grep? Should I load the file and perform regex?
You can do this with grep, for example with the following command:
grep -E -o 'L[0-9]{5}' name_of_file
For example, given a file with the text:
kasdhflkashl143112343214L232134614
3L1431413543454L2342L3523269ufoidu
gl9983ugsdu8768IUHI/(JHKJASHD/(888
The command above will output:
L23213
L14314
L35232
If it is just in a single file, you can do something along the lines of:
grep -e 'L[0-9]{5}' filename
If you need to search all files in a directory for these strings:
find . -type f | xargs grep -e 'L[0-9]{5}'
I have a file A with 100 words in it separated by new lines. I would like to search file B to see if ANY of the words in file A occur in it.
I tried the following but does not work to me:
grep -F A B
You need to use the option -f:
$ grep -f A B
The option -F does a fixed string search where as -f is for specifying a file of patterns. You may want both if the file only contains fixed strings and not regexps.
$ grep -Ff A B
You may also want the -w option for matching whole words only:
$ grep -wFf A B
Read man grep for a description of all the possible arguments and what they do.
To find a very long list of words in big files, it can be more efficient to use egrep:
remove the last \n of A
$ tr '\n' '|' < A > A_regex
$ egrep -f A_regex B