I have a text file like
line2
line3;
line4
line5
line6
line7;
I need a loop to read the line till ; on each loop.
on the first loop read up to line3; and on the second loop up to line7; and so on.
no need to merge the lines into a single one
Consider telling read to stop on ; instead of on newlines, and instead to use newlines to delimit individual input items (when using -a to read into an array, this makes each line an array element).
When given your input,
while IFS=$'\n' read -r -d ';' -a lines; do
echo "Read group of lines:"
printf ' - %s\n' "${lines[#]};"
done
...emits as output:
Read group of lines:
- line2
- line3;
Read group of lines:
- line4
- line5
- line6
- line7;
You can, if you choose, replace the printf with something like for line in "${lines[#]}"; do to create an inner loop to operate on lines within a group one-by-one.
You can use two loops: one to continue until the end of the file, and an inner loop to read individual lines until you find one that ends with a ;.
For example,
while :; do
lines=()
while IFS= read -r line; do
lines+=( "$line" )
if [[ $line = *; ]]; then
break
fi
done
if (( ${#lines[#]} == 0 )); then
# The previous loop didn't add anything to the array,
# so the last read must have failed, and we've reached
# then end of the file
break
done
# do something with $lines
done < file.txt
Or, use one loop that pauses to use lines when one ending with a ; is found:
lines=()
while IFS= read -r line; do
lines+=("$line")
if [[ $line = *; ]]; then
# do stuff with lines, then clear the array
lines=()
fi
done < file.txt
# If applicable, do something with the last batch of lines
# if the file doesn't end with a ;-terminated line.
Related
I need an output of current count vs total count in single line. I would like to know if this could be done Via Bash using 'for' 'while' loop.
Expecting an output that should only update the count and should not display multiple lines
File Content
$ cat ~/test.rtf
hostname1
hostname2
hostname3
hostname4
#!/bin/sh
j=1
k=$(cat ~/test.rtf | wc -l)
for i in $(cat ~/test.rtf);
do
echo "Working on line ($j/$k)"
echo "$i"
#or any other command for i
j=$((j+1))
done
EX:
Working on line (2/4)
Not like,
Working on line (2/4)
Working on line (3/4)
Assumptions:
OP wants to generate n lines of output that overwrite each other on successive passes through the loop
in OP's sample code there are two echo calls so in this case n=2
General approaches:
issue a clear at the beginning of each pass through the loop so as to clear the current output and reposition the cursor at the 'top' of the console/window
use tput to manage movement of the cursor (and clearing/overwriting of previous output)
Sample input:
$ cat test.rtf
this is line1
then line2
and line3
and last but not least line4
real last line5
clear approach:
j=1
k=$(wc -l < test.rtf)
while read -r line
do
clear
echo "Working on line ($j/$k)"
echo "${line}"
((j++))
done < test.rtf
tput approach:
j=1
k=$(wc -l < test.rtf)
EraseToEOL=$(tput el) # grab terminal specific code for clearing from cursor to EOL
clear # optional: start with a new screen otherwise use current position in console/window for next command ...
tput sc # grab current cursor position
while read -r line
do
tput rc # go (back) to cursor position stored via 'tput sc'
echo "Working on line ($j/$k)"
echo "${line}${EraseToEOL}" # ${EraseToEOL} forces clearing rest of line, effectively erasing a previous line that may have been longer then the current line of output
((j++))
done < test.rtf
Both of these generate the same result:
Something along these lines:
file=~/test.rtf
nl=$(wc -l "$file")
nl=${nl%%[[:blank:]]*}
i=0
while IFS= read -r line; do
i=$((i+1))
echo "Working on line ($i/$nl)"
done < "$file"
Your main question is how to avoid each the counter to be written to new lines. The newlines are \n characters, which is appended by echo. You want \r, like
for ((i=0; i<10; i++)); do
printf "Counter $i\r"
sleep 1
done
echo
When you echo something from the line you are working on, you will use \n again. I will use cut as an example of processing the inputline. Use the output string in the same printf command like
j=1
k=$(cat ~/test.rtf | wc -l)
while IFS= read -r line; do
printf "Working on line (%s): %s\r" "$j/$k" $(cut -c1-10 <<< "${line}")
sleep 1
((j++))
done < ~/test.rft
The problem with the above solution is that you will see output from previous lines when your last output is shorter than the previous one. When you know the maximum length that your processing of the line will show, you can force additional spaces:
j=1
k=$(cat ~/test.rtf | wc -l)
while IFS= read -r line; do
printf "Working on line (%5.5s): %-20s\r" "$j/$k" "$(cut -c1-20 <<< "${line}")";
sleep 1
((j++))
done < ~/test.rft
I have an exercise where I have a file and at the begin of it I have something like
#!usr/bin/bash
# tototata
#tititutu
#ttta
Hello world
Hi
Test test
#zabdazj
#this is it
And I have to take each first line starting with a # until the line where I don't have one and stock it in a variable. In case of a shebang, it has to skip it and if there's blank space between lines, it has to skip them too. We just want the comment between the shebang and the next character.
I'm new to bash and I would like to know if there's a way to do it please ?
Expected output:
# tototata
#tititutu
#ttta
Try in this easy way to better understand.
#!/bin/bash
sed 1d your_input_file | while read line;
do
check=$( echo $line | grep ^"[#;]" )
if ([ ! -z "$check" ] || [ -z "$line" ])
then
echo $line;
else
exit 1;
fi
done
This may be more correct, although your question was unclear about weather the input file had a script shebang, if the shebang had to be skipped to match your sample output, or if the input file shebang was just bogus.
It is also unclear for what to do, if the first lines of the input file are not starting with #.
You should really post your assignment's text as a reference.
Anyway here is a script that does collects first set of consecutive lines starting with a sharp # into the arr array variable.
It may not be an exact solution to your assignment (witch you should be able to solve with what your previous lessons taught you), but will get you some clues and keys to iterate reading lines from a file and testing that lines starts with a #.
#!/usr/bin/env bash
# Our variable to store parsed lines
# Is an array of strings with an entry per line
declare -a arr=()
# Iterate reading lines from the file
# while it matches Regex: ^[#]
# mean while lines starts with a sharp #
while IFS=$'\n' read -r line && [[ "$line" =~ ^[#] ]]; do
# Add line to the arr array variable
arr+=("$line")
done <a.txt
# Print each array entries with a newline
printf '%s\n' "${arr[#]}"
How about this (not tested, so you may have to debug it a bit, but my comments in the code should explain what is going on):
while read line
do
# initial is 1 one the first line, and 0 after this. When the script starts,
# the variable is undefined.
: ${initial:=1}
# Test for lines starting with #. Need to quote the hash
# so that it is not taken as comment.
if [[ $line == '#'* ]]
then
# Test for initial #!
if (( initial == 1 )) && [[ $line == '#!'* ]]
then
: # ignore it
else
echo $line # or do whatever you want to do with it
fi
fi
# stop on non-blank, non-comment line
if [[ $line != *[^\ ]* ]]
then
break
fi
initial=0 # Next line won't be an initial line
done < your_file
I am trying to do some math on 2nd column of a txt file , but some lines are not numbers , i only want to operate on the lines which have numbers .and keep other line unchanged
txt file like below
aaaaa
1 2
3 4
How can I do this?
Doubling the second column in any line that doesn't contain any alphabetic content might look a bit like the following in native bash:
#!/bin/bash
# iterate over lines in input file
while IFS= read -r line; do
if [[ $line = *[[:alpha:]]* ]]; then
# line contains letters; emit unmodified
printf '%s\n' "$line"
else
# break into a variable for the first word, one for the second, one for the rest
read -r first second rest <<<"$line"
if [[ $second ]]; then
# we extracted a second word: emit it, doubled, between the first word and the rest
printf '%s\n' "$first $(( second * 2 )) $rest"
else
# no second word: just emit the whole line unmodified
printf '%s\n' "$line"
fi
fi
done
This reads from stdin and writes to stdout, so usage is something like:
./yourscript <infile >outfile
thanks all ,this is my second time to use this website ,i find it is so helpful that it can get the answer very quickly
I also find a answer below
#!/bin/bash
FILE=$1
while read f1 f2 ;do
if[[$f1 != *[!0-9]*]];then
f2=`echo "$f2 -1"|bc` ;
echo "$f1 $f2"
else
echo "$f1 $f2"
fi
done< %FILE
I'm trying to parse a csv file I made with Google Spreadsheet. It's very simple for testing purposes, and is basically:
1,2
3,4
5,6
The problem is that the csv doesn't end in a newline character so when I cat the file in BASH, I get
MacBook-Pro:Desktop kkSlider$ cat test.csv
1,2
3,4
5,6MacBook-Pro:Desktop kkSlider$
I just want to read line by line in a BASH script using a while loop that every guide suggests, and my script looks like this:
while IFS=',' read -r last first
do
echo "$last $first"
done < test.csv
The output is:
MacBook-Pro:Desktop kkSlider$ ./test.sh
1 2
3 4
Any ideas on how I could have it read that last line and echo it?
Thanks in advance.
You can force the input to your loop to end with a newline thus:
#!/bin/bash
(cat test.csv ; echo) | while IFS=',' read -r last first
do
echo "$last $first"
done
Unfortunately, this may result in an empty line at the end of your output if the input already has a newline at the end. You can fix that with a little addition:
!/bin/bash
(cat test.csv ; echo) | while IFS=',' read -r last first
do
if [[ $last != "" ]] ; then
echo "$last $first"
fi
done
Another method relies on the fact that the values are being placed into the variables by the read but they're just not being output because of the while statement:
#!/bin/bash
while IFS=',' read -r last first
do
echo "$last $first"
done <test.csv
if [[ $last != "" ]] ; then
echo "$last $first"
fi
That one works without creating another subshell to modify the input to the while statement.
Of course, I'm assuming here that you want to do more inside the loop that just output the values with a space rather than a comma. If that's all you wanted to do, there are other tools better suited than a bash read loop, such as:
tr "," " " <test.csv
cat file |sed -e '${/^$/!s/$/\n/;}'| while IFS=',' read -r last first; do echo "$last $first"; done
If the last (unterminated) line needs to be processed differently from the rest, #paxdiablo's version with the extra if statement is the way to go; but if it's going to be handled like all the others, it's cleaner to process it in the main loop.
You can roll the "if there was an unterminated last line" into the main loop condition like this:
while IFS=',' read -r last first || [ -n "$last" ]
do
echo "$last $first"
done < test.csv
I have a file with format like :
[PATTERN]
line1
line2
line3
.
.
.
line
[PATTERN]
line1
line2
line3
.
.
.
line
[PATTERN]
line1
line2
line3
.
.
.
line
I want to extract the following blocks from above file :
[PATTERN]
line1
line2
line3
.
.
.
line
Note: Number of lines between 2 [PATTERN] may varies, so can't rely on number of lines.
Basically, I want to store each pattern and the lines following it to Database, so I wil have to iterate all such blocks in my file.
How do this with Shell Scripting ?
This assumes you are using bash as your shell. For other shells, the actual solution can be different.
Assuming your data is in data:
i=0 ; cat data | while read line ; do \
if [ "$line" == "[PATTERN]" ] ; then \
i=$(($i + 1)) ; touch file.$i ; continue ; \
fi ; echo "$line" >> file.$i ; \
done
Change [PATTERN] by your actual separation pattern.
This will create files file.1, file.2, etc.
Edit: responding to request about an awk solution:
awk '/^\[PATTERN\]$/{close("file"f);f++;next}{print $0 > "file"f}' data
The idea is to open a new file each time the [PATTERN] is found (skipping that line - next command), and writing all successive lines to that file. If you need to include [PATTERN] in your generated files, delete the next command.
Notice the escaping of the [ and ], which have special meaning for regular expressions. If your pattern does not contain those, you do not need the escaping. The ^ and $ are advisable, since they tie your pattern to the beginning and end of line, which you will usually need.
This can be for sure improved, but if you want to store lines in an array here is something I did in past:
#!/bin/bash
file=$1
gp_cnt=-1
i=-1
while read line
do
# Match pattern
if [[ "$line" == "[PATTERN]" ]]; then
let "gp_cnt +=1"
# If this is not the first match process group
if [[ $gp_cnt -gt 0 ]]; then
# Process the group
echo "Processing group #`expr $gp_cnt - 1`"
echo ${parsed[*]}
fi
# Start new group
echo "Pattern #$gp_cnt catched"
i=0
unset parsed
parsed[$i]="$line"
# Other lines (lines before first pattern are not processed)
elif [[ $gp_cnt != -1 ]]; then
let "i +=1"
parsed[$i]="$line"
fi
done < <(cat $file)
# Process last group
echo "Processing group #$gp_cnt"
echo ${parsed[*]}
I don't like the processing of the last group out of the loop...