In a bash script, I see several while statements with those redirect signs when closing the loop.
I know that if I end it with "done < file", I am redirecting the file to the stdin of the command in the while statement. But what the others means?
I would appreciate if someone could give an explanation with examples.
With the file text.txt
1aa
2bb
3cc
Redirection:
$ cat < text.txt
1aa
2bb
3cc
Here document:
$ cat << EOF
> 1AA
> 2BB
> EOF
1AA
2BB
Here string:
$ cat <<< 1aaa
1aaa
The first form, <, is an input redirection. It somewhat different than << and <<< which are two variants of a here document.
The first form, <, is primarily used to redirect the contents of a file to a command or process. It is a named FIFO, and therefor a file that is passed to a command that accepts file arguments.
cmd < file
will open the file named file and create a new file name to open and read. The difference between cmd file and cmd < file is the name passed to cmd in the second case is the name of a named pipe.
You can also do process substitution:
cmd <(process)
An example use would be comparing two directories:
diff <(ls dir1) <(ls dir2)
In this case, the command ls dir1 and ls dir2 has output redirected to a file like stream that is then read by diff as if those were two files.
You can see the name of the file device by passing to echo a process substitution:
$ echo <(ls)
/dev/fd/63
Since echo does not support opening files, it just prints the name of the FIFO.
Here documents are easier to demonstrate. The << form has a 'limit string' that is not included in the output:
$ cat <<HERE
> line 1
> line 2
> line 3
> HERE
line 1
line 2
line 3
The HERE is a unique string that must be on its own line.
The 'here string' or <<< form does not require the delimiting string of the << form and is on a single line:
$ cat <<< 'line 1'
line 1
You can also expand parameters:
$ v="some text"
$ cat <<< "$v"
some text
But not other forms of shell expansions:
Brace expansion:
$ echo a{b,c,d}e
abe ace ade
$ cat <<< a{b,c,d}e
a{b,c,d}e
Given a 'generic' Bash while loop that reads input line by line:
while IFS= read -r line || [[ -n $line ]]; do printf "'%s'\n" "$line"; done
There are several ways that you can feed input into that loop.
First example, you can redirect a file. For demo, create a 6 line file:
$ seq 6 > /tmp/6.txt
Redirect the input of the file into the loop:
while IFS= read -r line || [[ -n $line ]]; do printf "'%s'\n" "$line"; done </tmp/6.txt
'1'
'2'
'3'
'4'
'5'
'6'
Or, second example, you can directly read from the output of seq using redirection:
$ while IFS= read -r line || [[ -n $line ]]; do printf "'%s'\n" "$line"; done < <(seq 3)
'1'
'2'
'3'
(Please note the extra < with a space for this form)
Or, third example, you can use a 'HERE' doc separated by CR:
while IFS= read -r line || [[ -n $line ]]; do printf "'%s'\n" "$line"; done <<HERE
1
2 3
4
HERE
'1 '
'2 3'
' 4'
Going back to diff which will only work on files, you can use process substitution and a HERE doc or process substitution and redirection to use diff on free text or the output of a program.
Given:
$ cat /tmp/f1.txt
line 1
line 2
line 3
Normally you would need to have a second file to compare free text with that file. You can use a HERE doc and process substitution to skip creating a separate file:
$ diff /tmp/f1.txt <(cat <<HERE
line 1
line 2
line 5
HERE
)
3c3
< line 3
---
> line 5
command < foo
Redirect the file foo to the standard input of command.
command << foo
blah 1
blah 2
foo
Here document: send the following lines up to foo to the standard input of command.
command <<< foo
Here-string. The string foo is sent to the standard input of command.
I have a file that contains 10 lines with this sort of content:
aaaa,bbb,132,a.g.n.
I wanna walk throw every line, char by char and put the data before the " , " is met in an output file.
if [ $# -eq 2 ] && [ -f $1 ]
then
echo "Read nr of fields to be saved or nr of commas."
read n
nrLines=$(wc -l < $1)
while $nrLines!="1" read -r line || [[ -n "$line" ]]; do
do
for (( i=1; i<=$n; ++i ))
do
while [ read -r -n1 temp ]
do
if [ temp != "," ]
then
echo $temp > $(result$i)
else
fi
done
paste -d"\n" $2 $(result$i)
done
nrLines=$($nrLines-1)
done
else
echo "File not found!"
fi
}
In parameter $2 I have an empty file in which I will store the data from file $1 after I extract it without the " , " and add a couple of comments.
Example:
My input_file contains:
a.b.c.d,aabb,comp,dddd
My output_file is empty.
I call my script: ./script.sh input_file output_file
After execution the output_file contains:
First line info: a.b.c.d
Second line info: aabb
Third line info: comp
(yes, without the 4th line info)
You can do what you want very simply with parameter-expansion and substring-removal using bash alone. For example, take an example file:
$ cat dat/10lines.txt
aaaa,bbb,132,a.g.n.
aaaa,bbb,133,a.g.n.
aaaa,bbb,134,a.g.n.
aaaa,bbb,135,a.g.n.
aaaa,bbb,136,a.g.n.
aaaa,bbb,137,a.g.n.
aaaa,bbb,138,a.g.n.
aaaa,bbb,139,a.g.n.
aaaa,bbb,140,a.g.n.
aaaa,bbb,141,a.g.n.
A simple one-liner using native bash string handling could simply be the following and give the following results:
$ while read -r line; do echo ${line%,*}; done <dat/10lines.txt
aaaa,bbb,132
aaaa,bbb,133
aaaa,bbb,134
aaaa,bbb,135
aaaa,bbb,136
aaaa,bbb,137
aaaa,bbb,138
aaaa,bbb,139
aaaa,bbb,140
aaaa,bbb,141
Paremeter expansion w/substring removal works as follows:
var=aaaa,bbb,132,a.g.n.
Beginning at the left and removing up to, and including, the first ',' is:
${var#*,} # bbb,132,a.g.n.
Beginning at the left and removing up to, and including, the last ',' is:
${var##*,} # a.g.n.
Beginning at the right and removing up to, and including, the first ',' is:
${var%,*} # aaaa,bbb,132
Beginning at the left and removing up to, and including, the last ',' is:
${var%%,*} # aaaa
Note: the text to remove above is represented with a wildcard '*', but wildcard use is not required. It can be any allowable text. For example, to only remove ,a.g.n where the preceding number is 136, you can do the following:
${var%,136*},136 # aaaa,bbb,136 (all others unchanged)
To print 2016 th line from a file named file.txt u have to run a command like this-
sed -n '2016p' < file.txt
More-
sed -n '2p' < file.txt
will print 2nd line
sed -n '2011p' < file.txt
2011th line
sed -n '10,33p' < file.txt
line 10 up to line 33
sed -n '1p;3p' < file.txt
1st and 3th line
and so on...
For more detail, please have a look in this tutorial and this answer.
In native bash the following should do what you want, assuming you replace the contents of your script.sh with the below:
#!/bin/bash
IN_FILE=${1}
OUT_FILE=${2}
IFS=\,
while read line; do
set -- ${line}
for ((i=1; i<=${#}; i++)); do
((${i}==4)) && continue
((n+=1))
printf '%s\n' "Line ${n} info: ${!i}"
done
done < ${IN_FILE} > ${OUT_FILE}
This will not print the 4th field of each line within the input file, on a new line in the output file (I assume this is your requirement as per your comment?).
[wspace#wspace sandbox]$ awk -F"," 'BEGIN{OFS="\n"}{for(i=1; i<=NF-1; i++){print "line Info: "$i}}' data.txt
line Info: a.b.c.d
line Info: aabb
line Info: comp
This little snippet can ignore the last field.
updated:
#!/usr/bin/env bash
if [ ! -f "$1" -o $# -ne 2 ];then
echo "Usage: $(basename $0) input_file out_file"
exit 127
fi
input_file=$1
output_file=$2
: > $output_file
if [ "$(wc -l < $1)" -ne 0 ];then
while true
do
read -r -n1 char
if [ "$char" == "" ];then
break
elif [ $char != "," ];then
temp=$temp$char
else
echo "line info: $temp" >> $output_file
temp=""
fi
done < $input_file
else
echo "file $1 is empty"
fi
Maybe this is what you want
Did you try
sed "s|,|\n|g" $1 | head -n -1 > $2
I assume that only the last word would not have a comma on its right.
Try this (tested with you sample line) :
#!/bin/bash
# script.sh
echo "Number of fields to save ?"
read nf
while IFS=$',' read -r -a arr; do
newarr=${arr[#]:0:${nf}}
done < "$1"
for i in ${newarr[#]};do
printf "%s\n" $i
done > "$2"
Execute script with :
$ ./script.sh inputfile outputfile
Number of fields ?
3
$ cat outputfile
a.b.c.d
aabb
comp
All words separated with commas are stored into an array $arr
A tmp array $newarr removes last $n element ($n get the read command).
It loops over new array and prints result in $2, the outputfile.
This is my code to read a file line by line:
IFS=$'\n'
myfile="$1"
i=0
while read line; do
echo "Line # $i: '$line'"
let i++
done < "$myfile"
This is the file passed as parameter
Hello
stack
overflow
friends
I execute it like this: test.sh input.txt and I get this result:
'ine # 0: 'Hello
'ine # 1: 'stack
'ine # 2: 'overflow
'ine # 3: 'friends
As you see, The fisrt character is replaced by a quote. And the quote of the final of the line does not appear. Whats going on here? I can't see the mistake? Any idea?
Most likely you have \r before end of line in your input file.
You can test same by using:
cat -vte file
This will show ^M$ in the end of file has dos carriage return \r.
You can use this script to read your file correctly:
i=1
while IFS=$'\r' read -r line; do
echo "Line # $i: '$line'"
let i++
done < "$myfile"
OR else convert your file into unix file using:
dos2unix file
OR If you don't wish to actually save the file stripped off of \r, you can also use:
while read line; do
........# your code as-is
done < <( tr -d '\r' < "$myfile")
I am new to shell scripting. I need to read a file that works in all shells that has variables defined in it. Something like:
variable1=test1
variable2=test2
....
I have to read this file line by line and prepare new string separated by spaces, like:
variable=variable1=test1 variable2=test2 ....
I tried with the below code:
while read LINE
do
$VAR="$VAR $LINE"
done < test.dat
but it's throwing me this error:
command not found Test.sh: line 3: = variable1=test1
The problem with your script is the leading $ before var is initialized, try:
#/bin/bash
while read line; do
var="$var $line"
done < file
echo "$var"
However you can do this with the tr command by substituting the newline character with a space.
$ tr '\n' ' ' < file
variable1=test1 variable2=test2
$ var="$(tr '\n' ' ' < file)"
$ echo "$var"
variable1=test1 variable2=test2
When defining a shell variable you must omit the $. So VAR="bla" is right, $VAR="bla" is wrong. The $ is only necessary for using the variable, as in echo $VAR;
while read LINE
do
VAR="$VAR $LINE"
done < test.dat