Error extension file when use grep to write file [duplicate] - bash

This question already has an answer here:
Why would a correct shell script give a wrapped/truncated/corrupted error message? [duplicate]
(1 answer)
Closed 7 years ago.
I have 100 ".txt" files. Each file contains the data such as
File name Data
1.txt BAP1
2.txt UCHL1
3.txt ABC1234
Now I want to scan content of these files and write to txt file with condition that it contains my input string such as "BAP1". I used below coded but the output files have mistaken such as '1.txt .' I have no idea why the extension file has more dot in last file. Could you help me solve it? I am working in cygwin
#!/bin/sh
grep -w 'BAP1' *.txt>"1.txt"
grep -w 'UCHL1' *.txt>"2.txt"

Run dos2unix on your script, or otherwise tell your editor to save it as a UNIX text file.
Otherwise, your filenames will have carriage returns (aka $'\r') on the end of their names.

Related

writing loop in bash stops after first iteration [duplicate]

This question already has answers here:
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Read a file line by line assigning the value to a variable [duplicate]
(10 answers)
Closed 2 months ago.
New to coding
I have a very simple loop that Im using to rename files, but it stops after the first iteration.
Im running the loop:
for x in `cat list_genomes.txt`
do
mv -v $x"_busco/run_hypocreales_odb10" "/busco_runs/run_"$x
done
and the error I get is:
‘FV25228_busco/run_hypocreales_odb10’ -> ‘busco_runs/run_FV25228’
mv: cannot stat ‘\r_busco/run_hypocreales_odb10’: No such file or directory
mv: cannot stat ‘FV32968\r_busco/run_hypocreales_odb10’: No such file or directory
mv: cannot stat ‘FV34765\r_busco/run_hypocreales_odb10’: No such file or directory
mv: cannot stat ‘FV-NC-01\r_busco/run_hypocreales_odb10’: No such file or directory
so the first line works, but then it cannot read the rest.
list_genomes.txt looks like:
FV25228
FV32968
FV34765
FV-NC-01
Fc25332
Your input file has originated on a system that is not identical to the one where you are running the script.
The "\r" on the 2nd-5th line indicate that the file came from a Windows-based system. The person who created the file should have chosen a format that is compatible with system on which the data was going to be processed, which in this case appears to be Unix/Linux based.
If the creator of your input file doesn't change the way they do that, you need to massage the file before feeding it to your program logic, using the command dos2unix.
The following would likely provide you with the necessary fix if all else is correct:
#!/bin/bash
dos2unix -n list_genomes.txt list_genomes.fix
while read line
do
mv -v "${line}_busco/run_hypocreales_odb10" "/busco_runs/run_${line}"
done < list_genomes.fix

Bash script for moving files: cannot stat [duplicate]

This question already has answers here:
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Closed 1 year ago.
I am trying to write a bash script that takes 2 inputs: a list of IDs and 2) a directory. The idea is that the script will move a subset of files corresponding to the list of IDs into a a new folder.
#! /usr/bin/bash
LIST_FILE=$(cat ${1}) # List of file IDs (example entry: 1.2345)
PATH=${2} # directory name (i.e group_1)
for i in ${LIST_FILE}
do
/usr/bin/mv /tmp/project/project_data/data_all/${i}.annotated.gz /tmp/project/project_data/${PATH}/
done
The script manages to loop ok, however I get the following error for each iteration:
/usr/bin/mv: cannot stat '/tmp/project/project_data/data_all/1.2345'$'\r''.annotated.gz': No such file or directory
It looks like the file name hasn't concatenated properly and I'm not sure why. I've tried researching the problem but I'm also quite new to bash and finding it hard to grasp the concept of the stat error. I appreciate any advice and possible solutions.
Thanks everyone.
I think that the file whose name you pass as the first argument to your script is in dos format instead of unix so you are getting extra \r characters in your file names.
You could change your third line to:
LIST_FILE=$(cat ${1}|tr -d '\r')
Bobby

OSX paste command not pasting whole line [duplicate]

This question already has answers here:
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Closed 1 year ago.
I am trying to paste two files together
file ip.txt
10.32.216.15
10.23.134.8
10.33.2.37
10.33.84.20
10.33.17.38
file obj.txt
obj-10.32.216.15
obj-10.23.134.8
obj-10.33.2.37
obj-10.33.84.20
obj-10.33.17.38
and I use the command like this:
paste ip.txt obj.txt
However I get this truncated output:
10.32.21obj-10.32.216.15
10.23.13obj-10.23.134.8
10.33.2.obj-10.33.2.37
10.33.84obj-10.33.84.20
10.33.17obj-10.33.17.38
The two files are created by grep command in my script earlier. This behavior is also present when I used variables with the command.
Also when I try to specify a delimiter only the second file gets pasted.
Does anyone know what might be the issue? Thanks
The issue was with line endings, for some reason Windows were set up but Unix were needed.

BASH Script to check a folder for a specific file type and then delete the oldest one [duplicate]

This question already has answers here:
Bash script to find and display oldest file
(4 answers)
Closed 2 years ago.
I'm looking for a BASH command (or set of commands) that will look in a specific directory and delete ONLY the single oldest file in that directory. I've looked around, but I can't quite find what I'm looking for. Hopefully someone can help me with this, because it's the last missing piece in my script. Everything else is working perfectly.
One way to delete oldest file ending with .specific :
rm -i $(ls -tr *.specific | sed q)
This is not very reliable if you have spaces in filenames

Permission denied error using sed to change file name [duplicate]

This question already has an answer here:
Permission denied on cat via shell script [duplicate]
(1 answer)
Closed 3 years ago.
I'm trying to take the filename of each file in a directory and 'rename' it to create a respective output file when running through a program. When running the script I get the error Permission denied, for the line that is meant to be doing the renaming.
outputname=basename $file | sed -e "s/_Aligned.sortedByCoord.out.bam/_.gtf/"
For example one file is named 92_Aligned.sortedByCoord.out.bam, I want the output to be 92_.gtf. Another file is called 10.5_rep1_Aligned.sortedByCoord.out.bam, and the output should be called 10.5_rep1_.gtf.
Am I doing it wrong? Not sure if sed is the right way as I'm technically not renaming the file, but creating another file from that name and changing it?
You want to force evaluation on basename-sed pipe. ( with '$(' ) and handle the quoting)
outputname=$(basename $file | sed -e "s/_Aligned.sortedByCoord.out.bam/_.gtf/")
Or using Bash built-in only, which will be much FASTER.
file_base=${file##*/}
outputname=${file_base/_Aligned.sortedByCoord.out.bam/_.gtf}

Resources