varrr=0
while read line
do
if [ $line -gt 500 -a $line -le 600 ]; then # for lines 501-600
echo $line >> 'file_out_${varrr}.ubi'
fi
done << 'file_in_${varrr}.ubi'
file_in_${varrr}.ubi is a text file with around 1000 lines. I want to print lines 501-600 to new file.
Running this code leaves my Ubuntu terminal with a > symbol on a new line, as if I need to type another command to finish the loop. I can' figure out what is wrong with this loop though. Seems like it's complete. See any mistakes I've made? Thanks.
I'm only going to answer your specific question: it's because you used a heredoc << symbol, instead of a redirection <. Your last line should read:
done < 'file_in_${varrr}.ubi'
(observe the single <).
But then you'll realize that you have some quoting problems. So, your last line should read:
done < "file_in_${varrr}.ubi"
(observe the double quotes ").
Similarly, watch out your quotings in line 6. You should have this instead:
echo "$line" >> "file_out_${varrr}.ubi"
(double quotes " for file_out_${varrr}.ubi).
But then, this will not behave as you expect... Maybe this will do:
varrr=0
linenb=0
while IFS= read -r line; do
((++linenb))
if ((linenb>500 && linenb<=600)); then # for lines 501-600
echo "$line" >> "file_out_${varrr}.ubi"
fi
done < "file_in_${varrr}.ubi"
Hope this helps!
If you just want to print lines from 501 to 600, why don't you use the following?
awk 'NR>=501 && NR<=600' file_in > file_out
awk 'NR==n' myfile prints the line n of the file myfile. Then, you can use ranges as I writted above.
You can simply use sed. It's the simplest tool for it and is cleaner and faster than a while loop with tests.
varrr=0
sed -n 501,600p "file_in_${varrr}.ubi" >> "file_out_${varrr}.ubi"
Or
varrr=0
sed -n 501,600p "file_in_${varrr}.ubi" > "file_out_${varrr}.ubi"
If you want to override existing data.
The mistake in your loop by the way is because you're not using a counter and comparing your line number by the line itself instead.
varrr=0
counter=0
while read line; do
(( ++counter ))
[[ counter -gt 500 && counter -le 600 ]] && echo "$line"
done < "file_in_${varrr}.ubi" > "file_out_${varrr}.ubi"
Noticeably you need to use < for input not << and place your variables around double quotes not single quotes.
Related
Hi I am trying to print/echo line numbers that are multiple of 5. I am doing this in shell script. I am getting errors and unable to proceed. below is the script
#!/bin/bash
x=0
y=$wc -l $1
while [ $x -le $y ]
do
sed -n `$x`p $1
x=$(( $x + 5 ))
done
When executing above script i get below errors
#./echo5.sh sample.h
./echo5.sh: line 3: -l: command not found
./echo5.sh: line 4: [: 0: unary operator expected
Please help me with this issue.
For efficiency, you don't want to be invoking sed multiple times on your file just to select a particular line. You want to read through the file once, filtering out the lines you don't want.
#!/bin/bash
i=0
while IFS= read -r line; do
(( ++i % 5 == 0 )) && echo "$line"
done < "$1"
Demo:
$ i=0; while read line; do (( ++i % 5 == 0 )) && echo "$line"; done < <(seq 42)
5
10
15
20
25
30
35
40
A funny pure Bash possibility:
#!/bin/bash
mapfile ary < "$1"
printf "%.0s%.0s%.0s%.0s%s" "${ary[#]}"
This slurps the file into an array ary, which each line of the file in a field of the array. Then printf takes care of printing one every 5 lines: %.0s takes a field, but does nothing, and %s prints the field. Since mapfile is used without the -t option, the newlines are included in the array. Of course this really slurps the file into memory, so it might not be good for huge files. For large files you can use a callback with mapfile:
#!/bin/bash
callback() {
printf '%s' "$2"
ary=()
}
mapfile -c 5 -C callback ary < "$1"
We're removing all the elements of the array during the callback, so that the array doesn't grow too large, and the printing is done on the fly, as the file is read.
Another funny possibility, in the spirit of glenn jackmann's solution, yet without a counter (and still pure Bash):
#!/bin/bash
while read && read && read && read && IFS= read -r line; do
printf '%s\n' "$line"
done < "$1"
Use sed.
sed -n '0~5p' $1
This prints every fifth line in the file starting from 0
Also
y=$wc -l $1
wont work
y=$(wc -l < $1)
You need to create a subshell as bash will see the spaces as the end of the assignment, also if you just want the number its best to redirect the file into wc.
Dont know what you were trying to do with this ?
x=$(( $x + 5 ))
Guessing you were trying to use let, so id suggest looking up the syntax for that command. It would look more like
(( x = x + 5 ))
Hope this helps
There are cleaner ways to do it, but what you're looking for is this.
#!/bin/bash
x=5
y=`wc -l $1`
y=`echo $y | cut -f1 -d\ `
while [ "$y" -gt "$x" ]
do
sed -n "${x}p" "$1"
x=$(( $x + 5 ))
done
Initialize x to 5, since there is no "line zero" in your file $1.
Also, wc -l $1 will display the number of line counts, followed by the name of the file. Use cut to strip the file name out and keep just the first word.
In conditionals, a value of zero can be interpreted as "true" in Bash.
You should not have space between your $x and your p in your sed command. You can put them right next to each other using curly braces.
You can do this quite succinctly using awk:
awk 'NR % 5 == 0' "$1"
NR is the record number (line number in this case). Whenever it is a multiple of 5, the expression is true, so the line is printed.
You might also like the even shorter but slightly less readable:
awk '!(NR%5)' "$1"
which does the same thing.
I have a file which is kind of unformatted, I want to place a new-line after every 100th character and remove any other new lines in it so that file may look with consistent width and readable
This code snippet helps read all the lines
while read LINE
do
len=${#LINE}
echo "Line length is : $len"
done < $file
but how do i do same for characters
Idea is to have something like this : (just an example, it may have syntax errors, not implemented yet)
while read ch #read character
do
chcount++ # increment character count
if [ "$chcount" -eq "100" && "$ch"!="\n" ] #if 100th character and is not a new line
then
echo -e "\n" #echo new line
elif [ "$ch"=="\n" ] #if character is not 100th but new line
then
ch=" " $replace it with space
fi
done < $file
I am learning bash, so please go easy!!
I want to place a new-line after every 100th character and remove any
other new lines in it so that file may look with consistent width and
readable
Unless you have a good reason to write a script, go ahead but you don't need one.
Remove the newline from the input and fold it. Saying:
tr -d '\n' < inputfile | fold -w 100
should achieve the desired result.
bash adds a -n flag to the standard read command to specify a number of characters to read, rather than a full line:
while read -n1 c; do
echo "$c"
done < $file
You can call the function below in any of the following ways:
line_length=100
wrap $line_length <<< "$string"
wrap $line_length < file_name
wrap $line_length < <(command)
command | wrap $line_length
The function reads the input line by line (more efficiently than by character) which essentially eliminates the existing newlines (which are replaced by spaces). The remainder of the previous line is prefixed to the current one and the result is split at the desired line length. The remainder after the split is kept for the next iteration. If the output buffer is full, it is output and cleared otherwise it's kept for the next iteration so more can be added. Once the input has been consumed, there may be additional text in the remainder. The function is called recursively until that is also consumed and output.
wrap () {
local remainder rest part out_buffer line len=$1
while IFS= read -r line
do
line="$remainder$line "
(( part = $len - ${#out_buffer} ))
out_buffer+=${line::$part}
remainder=${line:$part}
if (( ${#out_buffer} >= $len ))
then
printf '%s\n' "$out_buffer"
out_buffer=
fi
done
rest=$remainder
while [[ $rest ]]
do
wrap $len <<< "$rest"
done
if [[ $out_buffer ]]
then
printf '%s\n' "$out_buffer"
out_buffer=
fi
}
#!/bin/bash
w=~/testFile.txt
chcount=0
while read -r word ; do
len=${#word}
for (( i = 0 ; i <= $len - 1 ; ++i )) ; do
let chcount+=1
if [ $chcount -eq 100 ] ; then
printf "\n${word:$i:1}"
let chcount=0
else
printf "${word:$i:1}"
fi
done
done < $w
Are you looking for something like this?
I cannot get this script to work at all. I am just trying to count the number of lines in a file WITHOUT using wc. here is what I have so far
FILE=file.txt
lines=0
while IFS= read -n1 char
do
if [ "$char" == "\n" ]
then
lines=$((lines+1))
fi
done < $FILE
this is just a small part of a bigger script that should count total words, characters and lines in a file. I cannot figure any of it out though. Please help
The problem is the if-statement conditional is never true.. Its as if the program cannot detect what a '\n' is.
declare -i lines=0 words=0 chars=0
while IFS= read -r line; do
((lines++))
array=($line) # don't quote the var to enable word splitting
((words += ${#array[#]}))
((chars += ${#line} + 1)) # add 1 for the newline
done < "$filename"
echo "$lines $words $chars $filename"
You have two problems there. They are fixed in the following:
#!/bin/bash
file=file.txt
lines=0
while IFS= read -rN1 char; do
if [[ "$char" == $'\n' ]]; then
((++lines))
fi
done < "$file"
One problem was the $'\n' in the test, the other one, more subtle, was that you need to use the -N switch, not the -n one in read (help read for more information). Oh, and you also want to use the -r option (check with and without, when you have backslashes in your file).
Minor things I changed: Use more robust [[...]], used lower case variable names (it's considered bad practice to have upper case variable names). Used arithmetic ((++lines)) instead of the silly lines=$((lines+1)).
I'm trying to parse a csv file I made with Google Spreadsheet. It's very simple for testing purposes, and is basically:
1,2
3,4
5,6
The problem is that the csv doesn't end in a newline character so when I cat the file in BASH, I get
MacBook-Pro:Desktop kkSlider$ cat test.csv
1,2
3,4
5,6MacBook-Pro:Desktop kkSlider$
I just want to read line by line in a BASH script using a while loop that every guide suggests, and my script looks like this:
while IFS=',' read -r last first
do
echo "$last $first"
done < test.csv
The output is:
MacBook-Pro:Desktop kkSlider$ ./test.sh
1 2
3 4
Any ideas on how I could have it read that last line and echo it?
Thanks in advance.
You can force the input to your loop to end with a newline thus:
#!/bin/bash
(cat test.csv ; echo) | while IFS=',' read -r last first
do
echo "$last $first"
done
Unfortunately, this may result in an empty line at the end of your output if the input already has a newline at the end. You can fix that with a little addition:
!/bin/bash
(cat test.csv ; echo) | while IFS=',' read -r last first
do
if [[ $last != "" ]] ; then
echo "$last $first"
fi
done
Another method relies on the fact that the values are being placed into the variables by the read but they're just not being output because of the while statement:
#!/bin/bash
while IFS=',' read -r last first
do
echo "$last $first"
done <test.csv
if [[ $last != "" ]] ; then
echo "$last $first"
fi
That one works without creating another subshell to modify the input to the while statement.
Of course, I'm assuming here that you want to do more inside the loop that just output the values with a space rather than a comma. If that's all you wanted to do, there are other tools better suited than a bash read loop, such as:
tr "," " " <test.csv
cat file |sed -e '${/^$/!s/$/\n/;}'| while IFS=',' read -r last first; do echo "$last $first"; done
If the last (unterminated) line needs to be processed differently from the rest, #paxdiablo's version with the extra if statement is the way to go; but if it's going to be handled like all the others, it's cleaner to process it in the main loop.
You can roll the "if there was an unterminated last line" into the main loop condition like this:
while IFS=',' read -r last first || [ -n "$last" ]
do
echo "$last $first"
done < test.csv
I need to to analyze (with grep) and print (with some formatting) the content of an
app's log.
This log contains text data in variable length lines. What I need is, after some grepping, loop each line of this output and print it with a maximum fixed length of 50 characters. If a line is longer than 50 chars, it should print a newline and then continue with the rest in the following line and so on until the line is completed.
I tried to use printf to do this, but it's not working and I don't know why. It just outputs the lines in same fashion of echo, without any consideration about printf formatting, though the \t character (tab) works.
function printContext
{
str="$1"
log="$2"
tmp="/tmp/deluge/$$"
rm -f $tmp
echo ""
echo -e "\tLog entries for $str :"
ln=$(grep -F "$str" "$log" &> "$tmp" ; cat "$tmp" | wc -l)
if [ $ln -gt 0 ];
then
while read line
do
printf "\t%50s\n" "$line"
done < $tmp
fi
}
What's wrong? I Know that I can make a substring routine to accomplish this task, but printf should be handy for stuff like this.
Instead of:
printf "\t%50s\n" "$line"
use
printf "\t%.50s\n" "$line"
to truncate your line to 50 characters only.
I'm not sure about printf but seeing as how perl is installed everywhere, how about a simple 1 liner?
echo $ln | perl -ne ' while( m/.{1,50}/g ){ print "$&\n" } '
Here's a clunky bash-only way to break the string into 50-character chunks
i=0
chars=50
while [[ -n "${y:$((chars*i)):$chars}" ]]; do
printf "\t%s\n" "${y:$((chars*i)):$chars}"
((i++))
done