I have a shell script and require your expertise on this.
SearchAirline() {
echo "Enter Airline Name:"
read airlineName
if [ $? -eq 0 ];then
echo -e "\t\t\E[43;31;1mFlight Information\E[0m"
echo -e "Departure Time Flight Airlines Vacancy"
echo "__________________________________________________________________________"
#cat flightlist.txt | grep $airlineName flightlist.txt
old_IFS=$IFS
IFS=$'\n'
for LINE in `sed -e '$airlineName' flightlist.txt`
do
print_flight $LINE
done
IFS=$old_IFS
fi
}
It does not work to give me the filtered list. Instead, it prints the entire list.
Change the '$airlineName' to "$airlineName". Variables aren't interpolated when they appear in single quotes.
Change the sed expression to only print lines that match:
sed -n "/$airlineName/p"
Edit: Other answers suggest using other tools such as grep, and they might be right. The only reason my answer relates to sed is that your question asks for it specifically. I'm assuming you're expecting to do more significant processing with sed than what you've described in your question.
Add the -n option to your sed invocation:
-n
Suppress the default output (in which each line, after it is examined for editing, is written to standard output). Only lines explicitly selected for output are written.
Edit: You also have to use the correct quotes around $airlineName. Single quotes disable variable substitution. I credit Martin Ellis for this since I didn't notice it the first time.
BTW - I would highly recommend using awk for this sort of report. It can handle the formatting and selection and it will be a lot faster if you have a large data set.
As #D.Shawley pointed out, this is REALLY a job for awk but that would mean rewriting the print_flight function too so here's a fixed shell script given some assumptions about your input file:
SearchAirline() {
echo "Enter Airline Name:"
read airlineName
if [ $? -eq 0 ];then
echo -e "\t\t\E[43;31;1mFlight Information\E[0m"
echo -e "Departure Time Flight Airlines Vacancy"
echo "__________________________________________________________________________"
grep "$airlineName" flightlist.txt |
while IFS= read -r line
do
print_flight "$line"
done
fi
}
I strongly recommend you rewrite your script in awk though. If you'd like help with that, post another question and show us what print_flight looks like.
Related
I need to add new lines with specific information to one or multiple files at the same time.
I tried to automate this task using the following script:
for i in /apps/data/FILE*
do
echo "nice weather 20190830 friday" >> $i
done
It does the job yet I wish I can automate it more and let the script ask me for to provide the file name and the line I want to add.
I expect the output to be like
enter file name : file01
enter line to add : IWISHIKNOW HOWTODOTHAT
Thank you everyone.
In order to read user input you can use
read user_input_file
read user_input_text
read user_input_line
You can print before the question as you like with echo -n:
echo -n "enter file name : "
read user_input_file
echo -n "enter line to add : "
read user_input_text
echo -n "enter line position : "
read user_input_line
In order to add line at the desired position you can "play" with head and tail
head -n $[$user_input_line - 1] $user_input_file > $new_file
echo $user_input_text >> $new_file
tail -n +$user_input_line $user_input_file >> $new_file
Requiring interactive input is horrible for automation. Make a command which accepts a message and a list of files to append to as command-line arguments instead.
#!/bin/sh
msg="$1"
shift
echo "$msg" | tee -a "$#"
Usage:
scriptname "today is a nice day" file1 file2 file3
The benefits for interactive use are obvious -- you get to use your shell's history mechanism and filename completion (usually bound to tab) but also it's much easier to build more complicated scripts on top of this one further on.
The design to put the message in the first command-line argument is baffling to newcomers, but allows for a very simple overall design where "the other arguments" (zero or more) are the files you want to manipulate. See how grep has this design, and sed, and many many other standard Unix commands.
You can use read statement to prompt for input,
read does make your script generic, but if you wish to automate it then you have to have an accompanying expect script to provide inputs to the read statement.
Instead you can take in arguments to the script which helps you in automation.. No prompting...
#!/usr/bin/env bash
[[ $# -ne 2 ]] && echo "print usage here" && exit 1
file=$1 && shift
con=$1
for i in `ls $file`
do
echo $con >> $i
done
To use:
./script.sh "<filename>" "<content>"
The quotes are important for the content so that the spaces in the content are considered to be part of it. For filenames use quotes so that the shell does not expand them before calling the script.
Example: ./script.sh "file*" "samdhaskdnf asdfjhasdf"
I'm new to UNIX and have this really simple problem:
I have a text-file (input.txt) containing a string in each line. It looks like this:
House
Monkey
Car
And inside my shell script I need to read this input file line by line to get to a variable like this:
things="House,Monkey,Car"
I know this sounds easy, but I just couldnt find any simple solution for this. My closest attempt so far:
#!/bin/sh
things=""
addToString() {
things="${things},$1"
}
while read line; do addToString $line ;done <input.txt
echo $things
But this won't work. Regarding to my google research I thought the while loop would create a new sub shell, but this I was wrong there (see the comment section). Nevertheless the variable "things" was still not available in the echo later on. (I cannot just write the echo inside the while loop, because I need to work with that string later on)
Could you please help me out here? Any help will be appreciated, thank you!
What you proposed works fine! I've only made two changes here: Adding missing quotes, and handling the empty-string case.
things=""
addToString() {
if [ -n "$things" ]; then
things="${things},$1"
else
things="$1"
fi
}
while read -r line; do addToString "$line"; done <input.txt
echo "$things"
If you were piping into while read, this would create a subshell, and that would eat your variables. You aren't piping -- you're doing a <input.txt redirection. No subshell, code works without changes.
That said, there are better ways to read lists of items into shell variables. On any version of bash after 3.0:
IFS=$'\n' read -r -d '' -a things <input.txt # read into an array
printf -v things_str '%s,' "${things[#]}" # write array to a comma-separated string
echo "${things_str%,}" # print that string w/o trailing comma
...on bash 4, that first line can be:
readarray -t things <input.txt # read into an array
This is not a shell solution, but the truth is that solutions in pure shell are often excessively long and verbose. So e.g. to do string processing it is better to use special tools that are part of the “default” Unix environment.
sed ':b;N;$!bb;s/\n/,/g' < input.txt
If you want to omit empty lines, then:
sed ':b;N;$!bb;s/\n\n*/,/g' < input.txt
Speaking about your solution, it should work, but you should really always use quotes where applicable. E.g. this works for me:
things=""
while read line; do things="$things,$line"; done < input.txt
echo "$things"
(Of course, there is an issue with this code, as it outputs a leading comma. If you want to skip empty lines, just add an if check.)
This might/might not work, depending on the shell you are using. On my Ubuntu 14.04/x64, it works with both bash and dash.
To make it more reliable and independent from the shell's behavior, you can try to put the whole block into a subshell explicitly, using the (). For example:
(
things=""
addToString() {
things="${things},$1"
}
while read line; do addToString $line ;done
echo $things
) < input.txt
P.S. You can use something like this to avoid the initial comma. Without bash extensions (using short-circuit logical operators instead of the if for shortness):
test -z "$things" && things="$1" || things="${things},${1}"
Or with bash extensions:
things="${things}${things:+,}${1}"
P.P.S. How I would have done it:
tr '\n' ',' < input.txt | sed 's!,$!\n!'
You can do this too:
#!/bin/bash
while read -r i
do
[[ $things == "" ]] && things="$i" || things="$things","$i"
done < <(grep . input.txt)
echo "$things"
Output:
House,Monkey,Car
N.B:
Used grep to tackle with empty lines and the probability of not having a new line at the end of file. (Normal while read will fail to read the last line if there is no newline at the end of file.)
In a bash script, I have a list of lines in a file I wish to grep and then display on standard out, which is easiest done with a while read:
grep "regex" "filepath" | while read line; do
printf "$line\n"
done
However, I would like to inform the user if no lines were matched by the grep. I know that one can do this by updating a variable inside the loop but it seems like a much more elegant approach (if possible) would be to try to read a line in an until loop, and if there were no output, an error message could be displayed.
This was my first attempt:
grep "regex" "filepath" | until [[ -z ${read line} ]]; do
if [[ -z $input ]]; then
printf "No matches found\n"
break
fi
printf "$line\n"
done
But in this instance the read command is malformed, and I wasn't sure of another way the phrase the query. Is this approach possible, and if not, is there a more suitable solution to the problem?
You don't need a loop at all if you simply want to display a message when there's no match. Instead you can use grep's return code. A simple if statement will suffice:
if ! grep "regex" "filepath"; then
echo "no match" >&2
fi
This will display the results of grep matches (since that's grep's default behavior), and will display the error message if it doesn't.
A popular alternative to if ! is to use the || operator. foo || bar can be read as "do foo or else do bar", or "if not foo then bar".
grep "regex" "filepath" || echo "no match" >&2
John Kugelman's answer is the correct and succinct one and you should accept it. I am addressing your question about syntax here just for completeness.
You cannot use ${read line} to execute read -- the brace syntax actually means (vaguely) that you want the value of a variable whose name contains a space. Perhaps you were shooting for $(read line) but really, the proper way to write your until loop would be more along the lines of
grep "regex" "filepath" | until read line; [[ -z "$line" ]]; do
... but of course, when there is no output, the pipeline will receive no lines, so while and until are both wrong here.
It is worth amphasizing that the reason you need a separate do is that you can have multiple commands in there. Even something like
while output=$(grep "regex filepath"); echo "grep done, please wait ...";
count=$(echo "$output" | wc -l); [[ $count -gt 0 ]]
do ...
although again, that is much more arcane than you would ever really need. (And in this particular case, you would want probably actually want if , not while.)
As others already noted, there is no reason to use a loop like that here, but I wanted to sort out the question about how to write a loop like this for whenever you actually do want one.
As mentioned by #jordanm, there is no need for a loop in the use case you mentioned.
output=$(grep "regex" "file")
if [[ -n $output ]]; then
echo "$output"
else
echo "Sorry, no results..."
fi
If you need to iterate over the results for processing (rather than just displaying to stdout) then you can do something like this:
output=$(grep "regex" "file")
if [[ -n $output ]]; then
while IFS= read -r line; do
# do something with $line
done <<< "$output"
else
echo "Sorry, no results..."
fi
This method avoids using a pipeline or subshell so that any variable assignments made within the loop will be available to the rest of the script.
Also, i'm not sure if this relates to what you are trying to do at all, but grep does have the ability to load patterns from a file (one per line). It is invoked as follows:
grep search_target -f pattern_file.txt
I have been trying to truncate a filename, both at start and end.
I could achieve this with the following code:
#!bin/bash
FILENAME="hello123.txt"
NAME=${FILENAME%.t*}
NUMBER=${NAME:5}
DOESNTWORK=${${FILENAME%.t*}:5}
echo "$FILENAME"
echo "$NAME"
echo "$NUMBER"
echo "$DOESNTWORK"
My issue is with the DOESNTWORK line.
I get this error for the line: main.sh: line 7: ${"${FILENAME%.t*}":5}: bad substitution
Can someone please point out the mistake in the line?
Thanks,
If you can use GNU grep why not use below:
DOESNTWORK=$(grep -o -E '[[:digit:]]+' <<< "$FILENAME")
Or with sed:
DOESNTWORK=$(sed 's/[^0-9]*//g' <<< "$FILENAME")
There are (at least) two ways to accomplish this in bash, at least for your specifically cited example. The first removes all characters that are not digits:
echo "${FILENAME//[^0-9]/}"
The second is by requesting a specific substring, which you were already partially using:
echo "${FILENAME:5:3}"
Which if these is most useful to you depends on how your actual use case compares to the example you cited...
Given a text file with multiple lines, I would like to iterate over each line in a Bash script. I had attempted to use cut, but cut does not accept \n (newline) as a delimiter.
This is an example of the file I am working with:
one
two
three
four
Does anyone know how I can loop through each line of this text file in Bash?
I found myself in the same problem, this works for me:
cat file.cut | cut -d$'\n' -f1
Or:
cut -d$'\n' -f1 file.cut
Use cat for concatenating or displaying. No need for it here.
file="/path/to/file"
while read line; do
echo "${line}"
done < "${file}"
Simply use:
echo -n `cut ...`
This suppresses the \n at the end
cat FILE|while read line; do # 'line' is the variable name
echo "$line" # do something here
done
or (see comment):
while read line; do # 'line' is the variable name
echo "$line" # do something here
done < FILE
So, some really good (possibly better) answers have been provided already. But looking at the phrasing of the original question, in wanting to use a BASH for-loop, it amazed me that nobody mentioned a solution with change of Field Separator IFS. It's a pure bash solution, just like the accepted read line
old_IFS=$IFS
IFS='\n'
for field in $(<filename)
do your_thing;
done
IFS=$old_IFS
If you are sure that the output will always be newline-delimited, use head -n 1 in lieu of cut -f1 (note that you mentioned a for loop in a script and your question was ultimately not script-related).
Many of the other answers, including the accepted one, have multiple lines unnecessarily. No need to do this over multiple lines or changing the default delimiter on the system.
Also, the solution provided by Ivan with -d$'\n' did not work for me either on Mac OSX or CentOS 7. Since his answer is four years old, I assume something must have changed on the logic of the $ character for this situation.
While loop with input redirection and read command.
You should not be using cut to perform a sequential iteration of each line in a file as cut was not designed to do this.
Print selected parts of lines from each FILE to standard output.
— man cut
TL;DR
You should use a while loop with the read -r command and redirect standard input to your file inside a function scope where IFS is set to \n and use -E when using echo.
processFile() { # Function scope to prevent overwriting IFS globally
file="$1" # Any file that exists
local IFS="\n" # Allows spaces and tabs
while read -r line; do # Read exits with 1 when done; -r allows \
echo -E "$line" # -E allows printing of \ instead of gibberish
done < $file # Input redirection allows us to read file from stdin
}
processFile /path/to/file
Iteration
In order to iterate over each line of a file, we can use a while loop. This will let us iterate as many times as we need to.
while <condition>; do
<body>
done
Getting our file ready to read
We can use the read command to store a single line from standard input in a variable. Before we can use that to read a line from our file, we need to redirect standard input to point to our file. We can do this with input redirection. According to the man pages for bash, the syntax for redirection is [fd]<file where fd defaults to standard input (a.k.a file descriptor 0). We can place this before or after our while loop.
while <condition>; do
<body>
done < /path/to/file
# or the non-traditional way
</path/to/file while <condition>; do
<body>
done
Reading the file and ending the loop
Now that our file can be read from standard input, we can use read. The syntax for read in our context is read [-r] var... where -r preserves the \ (backslash) character, instead of using it as an escape sequence character, and var is the name of the variable to store the input in. You can have multiple variables to store pieces of the input in but we only need one to read an entire line. Along with this, to preserve any backslashes in any output from echo you will likely need to use the -E flag to disable the interpretation of backslash escapes. If you have any indentation (spaces or tabs), you will need to temporarily change the IFS (Input Field Separators) variable to only "\n"; normally it is set to " \t\n".
main() {
local IFS="\n"
read -r line
echo -E "$line"
}
main
How do we use read to end our while loop?
There is really only one reliable way, that I know of, to determine when you've finished reading a file with read: check the exit value of read. If the exit value of read is 0 then we successfully read a line, if it is 1 or higher then we reached EOF (end of file). With that in mind, we can place the call to read in our while loop's condition section.
processFile() {
# Could be any file you want hardcoded or dynamic
file="$1"
local IFS="\n"
while read -r line; do
# Process line here
echo -E "$line"
done < $file
}
processFile /path/to/file1
processFile /path/to/file2
A visual breakdown of the above code via Explain Shell.
If I am executing a command and want to cut the output but it has multiple lines I found it helpful to do
echo $([command]) | cut [....]
This puts all the output of [command] on a single line that can be easier to process.
My opinion is that "cut" uses '\n' as its default delimiter.
If you want to use cut, I have two ways:
cut -d^M -f1 file_cut
I make ^M By click Enter After Ctrl+V. Another way is
cut -c 1- file_cut
Does that help?