Nested commands in bash - bash

I have been trying to truncate a filename, both at start and end.
I could achieve this with the following code:
#!bin/bash
FILENAME="hello123.txt"
NAME=${FILENAME%.t*}
NUMBER=${NAME:5}
DOESNTWORK=${${FILENAME%.t*}:5}
echo "$FILENAME"
echo "$NAME"
echo "$NUMBER"
echo "$DOESNTWORK"
My issue is with the DOESNTWORK line.
I get this error for the line: main.sh: line 7: ${"${FILENAME%.t*}":5}: bad substitution
Can someone please point out the mistake in the line?
Thanks,

If you can use GNU grep why not use below:
DOESNTWORK=$(grep -o -E '[[:digit:]]+' <<< "$FILENAME")
Or with sed:
DOESNTWORK=$(sed 's/[^0-9]*//g' <<< "$FILENAME")

There are (at least) two ways to accomplish this in bash, at least for your specifically cited example. The first removes all characters that are not digits:
echo "${FILENAME//[^0-9]/}"
The second is by requesting a specific substring, which you were already partially using:
echo "${FILENAME:5:3}"
Which if these is most useful to you depends on how your actual use case compares to the example you cited...

Related

Some tips to improve a bash script for count fastq files

Hi guys I got this bash one line that i wish to make a script
for i in 'ls *.fastq.gz'; do echo $(zcat ${i} | wc -l)/4|bc; done
I would like to make it as a script to read from a data dir and print out the result with the name of the file.
I tried to put the dir in front of the 'data/*.fastq.gz' but got am error No such dir exist...
I would like some like this:
name1.fastq.gz 1898516
name2.fastq.gz 2467421
namen.fastq.gz 1234532
I am not experienced in bash.
Could you guys give a help?
Thanks
Take the dir as an argument, but default to the current dir if it's not set.
dir="${1-.}"
Then put it in the glob: "$dir"/*.fastq.gz
As well:
Quote variables and command expansions.
Don't parse ls.
Don't trust echo with arbitrary data (filenames). Use printf instead.
Use an end-of-options flag -- when giving filenames to commands.
I prefer to not have any inline command expansions, but that's just personal preference
Putting it together:
#!/bin/bash
dir="${1-.}"
for file in "$dir"/*.fastq.gz; do
printf '%s ' "$file"
lines="$(zcat -- "$file" | wc -l)"
bc <<< "$lines/4" # Using a here-string (Bash feature)
done
There is no need to escape to bc for integer math (divide by 4), or to use 'ls' to enumerate the files. The original version will do with minor changes:
#!/bin/bash
dir="${1-.}"
for i in "$dir"/*.fastq.gz; do
lines=$(zcat "${i}" | wc -l)
printf '%s %d\n' "$i" "$((lines/4))"
done

Unix Bash content of a file as argument stops at first line

I'm having an issue in something that seems to be a rookie error, but I can't find a way to find a solution.
I have a bash script : log.sh
which is :
#!/bin/bash
echo $1 >> log_out.txt
And with a file made of filenames (taken from the output of "find" which names is filesnames.txt and contains 53 lines of absolute paths) I try :
./log.sh $(cat filenames.txt)
the only output I have in the log_out.txt is the first line.
I need each line to be processed separately as I need to put them in arguments in a pipeline with 2 softwares.
I checked for :
my lines being terminated with /n
using a simple echo without writing to a file
all the sorts of cat filenames.txt or (< filenames.txt) found on internet
I'm sure it's a very dumb thing, but I can't find why I can't iterate more than one line :(
Thanks
It is because your ./log.sh $(cat filenames.txt) is being treated as one argument.
while IFS= read -r line; do
echo "$line";
done < filenames.txt
Edit according to: https://mywiki.wooledge.org/DontReadLinesWithFor
Edit#2:
To preserve leading and trailing whitespace in the result, set IFS to the null string.
You could simplify more and skip using explicit variable and use the default $REPLY
Source: http://wiki.bash-hackers.org/commands/builtin/read
You need to quote the command substitution. Otherwise $1 will just be the first word in the file.
./log.sh "$(cat filenames.txt)"
You should also quote the variable in the script, otherwise all the newlines will be converted to spaces.
echo "$1" >> log_out.txt
If you want to process each word separately, you can leave out the quotes
./log.sh $(cat filenames.txt)
and then use a loop in the script:
#!/bin/bash
for word in "$#"
do
echo "$word"
done >> log_out.txt
Note that this solution only works correctly when the file has one word per line and there are no wildcards in the words. See mywiki.wooledge.org/DontReadLinesWithFor for why this doesn't generalize to more complex lines.
You can iterate with each line.
#!/bin/bash
for i in $*
do
echo $i >> log_out.txt
done

How to format output using sed

I have a shell script and require your expertise on this.
SearchAirline() {
echo "Enter Airline Name:"
read airlineName
if [ $? -eq 0 ];then
echo -e "\t\t\E[43;31;1mFlight Information\E[0m"
echo -e "Departure Time Flight Airlines Vacancy"
echo "__________________________________________________________________________"
#cat flightlist.txt | grep $airlineName flightlist.txt
old_IFS=$IFS
IFS=$'\n'
for LINE in `sed -e '$airlineName' flightlist.txt`
do
print_flight $LINE
done
IFS=$old_IFS
fi
}
It does not work to give me the filtered list. Instead, it prints the entire list.
Change the '$airlineName' to "$airlineName". Variables aren't interpolated when they appear in single quotes.
Change the sed expression to only print lines that match:
sed -n "/$airlineName/p"
Edit: Other answers suggest using other tools such as grep, and they might be right. The only reason my answer relates to sed is that your question asks for it specifically. I'm assuming you're expecting to do more significant processing with sed than what you've described in your question.
Add the -n option to your sed invocation:
-n
Suppress the default output (in which each line, after it is examined for editing, is written to standard output). Only lines explicitly selected for output are written.
Edit: You also have to use the correct quotes around $airlineName. Single quotes disable variable substitution. I credit Martin Ellis for this since I didn't notice it the first time.
BTW - I would highly recommend using awk for this sort of report. It can handle the formatting and selection and it will be a lot faster if you have a large data set.
As #D.Shawley pointed out, this is REALLY a job for awk but that would mean rewriting the print_flight function too so here's a fixed shell script given some assumptions about your input file:
SearchAirline() {
echo "Enter Airline Name:"
read airlineName
if [ $? -eq 0 ];then
echo -e "\t\t\E[43;31;1mFlight Information\E[0m"
echo -e "Departure Time Flight Airlines Vacancy"
echo "__________________________________________________________________________"
grep "$airlineName" flightlist.txt |
while IFS= read -r line
do
print_flight "$line"
done
fi
}
I strongly recommend you rewrite your script in awk though. If you'd like help with that, post another question and show us what print_flight looks like.

Trying to retrieve first 5 characters from string in bash error?

I'm trying to retrieve the first 5 characters from a string and but keep getting a Bad substitution error for the string manipulation line, I have the following lines in my teststring.sh script:
TESTSTRINGONE="MOTEST"
NEWTESTSTRING=${TESTSTRINGONE:0:5}
echo ${NEWTESTSTRING}
I have went over the syntax many times and cant see what im doing wrong
Thanks
Depending on your shell, you may be able to use the following syntax:
expr substr $string $position $length
So for your example:
TESTSTRINGONE="MOTEST"
echo `expr substr ${TESTSTRINGONE} 0 5`
Alternatively,
echo 'MOTEST' | cut -c1-5
or
echo 'MOTEST' | awk '{print substr($0,0,5)}'
echo 'mystring' |cut -c1-5 is an alternative solution to ur problem.
more on unix cut program
Works here:
$ TESTSTRINGONE="MOTEST"
$ NEWTESTSTRING=${TESTSTRINGONE:0:5}
$ echo ${NEWTESTSTRING}
MOTES
What shell are you using?
Substrings with ${variablename:0:5} are a bash feature, not available in basic shells. Are you sure you're running this under bash? Check the shebang line (at the beginning of the script), and make sure it's #!/bin/bash, not #!/bin/sh. And make sure you don't run it with the sh command (i.e. sh scriptname), since that overrides the shebang.
This might work for you:
printf "%.5s" $TESTSTRINGONE
Works in most shells
TESTSTRINGONE="MOTEST"
NEWTESTSTRING=${TESTSTRINGONE%"${TESTSTRINGONE#?????}"}
echo ${NEWTESTSTRING}
# MOTES
echo $TESTSTRINGONE|awk '{print substr($0,0,5)}'
You were so close! Here is the easiest solution: NEWTESTSTRING=$(echo ${TESTSTRINGONE::5})
So for your example:
$ TESTSTRINGONE="MOTEST"
$ NEWTESTSTRING=$(echo ${TESTSTRINGONE::5})
$ echo $NEWTESTSTRING
MOTES
You can try sed if you like -
[jaypal:~/Temp] TESTSTRINGONE="MOTEST"
[jaypal:~/Temp] sed 's/\(.\{5\}\).*/\1/' <<< "$TESTSTRINGONE"
MOTES
That parameter expansion should work (what version of bash do you have?)
Here's another approach:
read -n 5 NEWTESTSTRING <<< "$TESTSTRINGONE"
The original syntax will work with BASH but not with DASH. On debian systems you
might think you are using bash, but maybe dash instead. If /bin/dash/exist then
try temporarily renaming dash to something like no.dash, and then create soft a
link, aka ln -s /bin/bash /bin/dash and see if that fixes the problem.
expr substr $string $position $length
$position starts from 1

echo "-e" doesn't print anything

I'm using GNU bash, version 3.00.15(1)-release (x86_64-redhat-linux-gnu). And this command:
echo "-e"
doesn't print anything. I guess this is because "-e" is one of a valid options of echo command because echo "-n" and echo "-E" (the other two options) also produce empty strings.
The question is how to escape the sequence "-e" for echo to get the natural output ("-e").
The one true way to print any arbitrary string:
printf "%s" "$vars"
This is a tough one ;)
Usually you would use double dashes to tell the command that it should stop interpreting options, but echo will only output those:
$ echo -- -e
-- -e
You can use -e itself to get around the problem:
$ echo -e '\055e'
-e
Also, as others have pointed out, if you don't insist on using the bash builtin echo, your /bin/echo binary might be the GNU version of the tool (check the man page) and thus understand the POSIXLY_CORRECT environment variable:
$ POSIXLY_CORRECT=1 /bin/echo -e
-e
There may be a better way, but this works:
printf -- "-e\n"
You could cheat by doing
echo "-e "
That would be dash, e, space.
Alternatively you can use the more complex, but more precise:
echo -e \\\\x2De
[root#scintia mail]# POSIXLY_CORRECT=1; export POSIXLY_CORRECT
[root#scintia mail]# /bin/echo "-e"
-e
[root#scintia mail]#
Another alternative:
echo x-e | sed 's/^x//'
This is the way recommended by the autoconf manual:
[...] It is often possible to avoid this problem using 'echo "x$word"', taking the 'x' into account later in the pipe.
After paying careful attention to the man page :)
SYSV3=1 /usr/bin/echo -e
works, on Solaris at least
I like that one using a herestring:
cat <<<"-e"
Another way:
echo -e' '
echo -e " \b-e"
/bin/echo -e
works, but why?
[resin#nevada ~]$ which echo
/bin/echo

Resources