Let me introduce my loop to you all:
NUM_LINE=0
while read line; do
let NUM_LINE+=1
if [ $NUM_LINE -lt 41 ]; then
echo -e "\t$BLANC|$ORIGINAL $line $BLANC|"
else
echo -e "\n\t$BLANC## "$GRIS"Llista de Nodes sel·leccionats $BLANC############$ORIGINAL\n"
read AUX
NUM_LINE=0
fi
done <$NODES
So that:
$BLANC is \033[1;37m
$GRIS same
$ORIGINAL as well
$NODES is the absolute path of a file containg a lot of lines like:
| 23127 myserver 98.194.263.29 |
The Problem:
The echo inside the else statement it's properly triggered.
But it doesn't happen the same with the read statement
Any suggestion?
The reason that the loop doesn't function properly is because read is reading from stdin in both cases. You need to open an alternate file descriptor to your file and read from the file descriptor.
exec 3<$NODES
NUM_LINE=0
while read -u 3 -r line; do
(( NUM_LINE++ ))
if (( NUM_LINE < 41 )); then
echo -e "\t$BLANC|$ORIGINAL $line $BLANC|"
else
echo -e "\n\t$BLANC## "$GRIS"Llista de Nodes sel·leccionats $BLANC############$ORIGINAL\n"
read AUX
NUM_LINE=0
fi
done
Related
i'm trying to generate a new output file from each existing file in a directory of .txt files. I want to check line by line in each file for two substrings. And append the lines that match that substring to each new output file.
I'm having trouble generating the new files.
This is what i currently have:
#!/bin/sh
# My first Script
success="(Compiling)\s\".*\"\s\-\s(Succeeded)"
failure="(Compiling)\s\".*\"\s\-\s(Failed)"
count_success=0
count_failure=0
for i in ~/Documents/reports/*;
do
while read -r line;
do
if [[$success=~$line]]; then
echo $line >> output_$i
count_success++
elif [[$failure=~$]]; then
echo $line >> output_$i
count_failure++
fi
done
done
echo "$count_success of jobs ran succesfully"
echo "$count_failure of jobs didn't work"
~
Any help would be appreciated, thanks
Please, use https://www.shellcheck.net/ to check your shell scripts.
If you use Visual Studio Code, you could install "ShellCheck" (by Timon Wong) extension.
About your porgram.
Assume bash
Define different extensions for input and output files (really important if there are in the same directory)
Loop on report, input, files only
Clear output file
Read input file
if sequence:
if [[ ... ]] with space after [[ and before ]]
spaces before and after operators (=~)
reverse operands order for operators =~
Prevent globbing with "..."
#! /bin/bash
# Input file extension
declare -r EXT_REPORT=".txt"
# Output file extension
declare -r EXT_OUTPUT=".output"
# RE
declare -r success="(Compiling)\s\".*\"\s\-\s(Succeeded)"
declare -r failure="(Compiling)\s\".*\"\s\-\s(Failed)"
# Counters
declare -i count_success=0
declare -i count_failure=0
for REPORT_FILE in ~/Documents/reports/*"${EXT_REPORT}"; do
# Clear output file
: > "${REPORT_FILE}${EXT_OUTPUT}"
# Read input file (see named file in "done" line)
while read -r line; do
# does the line match the success pattern ?
if [[ $line =~ $success ]]; then
echo "$line" >> "${REPORT_FILE}${EXT_OUTPUT}"
count_success+=1
# does the line match the failure pattern ?
elif [[ $line =~ $failure ]]; then
echo "$line" >> "${REPORT_FILE}${EXT_OUTPUT}"
count_failure+=1
fi
done < "$REPORT_FILE"
done
echo "$count_success of jobs ran succesfully"
echo "$count_failure of jobs didn't work"
What about using grep?
success='Compiling\s".*"\s-\sSucceeded'
failure='Compiling\s".*"\s-\sFailed'
count_success=0
count_failure=0
for i in ~/Documents/reports/*; do
(( count_success += $(grep -E "$success" "$i" | tee "output_$i" | wc -l) ))
(( count_failure += $(grep -E "$failure" "$i" | tee -a "output_$i" | wc -l) ))
done
echo "$count_success of jobs ran succesfully"
echo "$count_failure of jobs didn't work"
I am a beginner and I am wondering if I can get help with this, please. The code should read one line from a file, display it, meet a condition, and then move on to the next line and repeat for as many lines as are in the file. I have been able to come up with a way to do this but this would be better if it was a loop instead. This is what I have so far
output=$(cat urls.txt | sed -n '1p')
read -p "Store $output y or n ?" deci
if [ $deci == "y" ];
then
sed -n '1p' urls.txt >> saved_domains.txt
fi
output=$(cat urls.txt | sed -n '2p')
read -p "Store $output y or n ?" deci
if [ $deci == "y" ];
then
sed -n '2p' urls.txt >> saved_domains.txt
fi
And this goes on, line by line
#!/bin/bash
while read output
do
read -p "Store $output y or n ?" deci < /dev/tty
if [[ "$deci" == "y" ]]
then
echo "$output" >> saved_domains.txt
fi
done < url.txt
You can read the url.txt file into an "output" variable through a while loop and use this to read the output from the user and append to the saved_domains.txt file accordingly.
One thing to note is the use of the < /dev/tty at the end of the read command for user input. This is not usually needed but as we are already reading from standard input through reading from the file, we need to specifically specify that we are reading from the terminal (/dev/tty)
I'm new to bash scripting and I'm asking for a little help !
I've got a little scipt in bash that is not making what I want (but almost) and the behavior of my echo command seems strange to me, look at it :
TST='test'
TEST="${ADDR[3]}"_"$TST"
echo $TEST
#result : _test
echo ${ADDR[3]}
#result : 5
How can you explain these results ? Thanks in advance :)
My ADDR var is defined like this :
#parsing the read line, split on whitespace
IFS=' ' read -ra ADDR <<< "$line"
Here is my complete script :
#!/bin/bash
NUMBER=2
{ read ;
while IFS= read -r line; do
echo "$NUMBER : $line"
IFS=' ' read -ra ADDR <<< "$line"
#If the countdown is set to 0, launch the task ans set it to init value
if [ ${ADDR[0]} == '0' ]; then
#task launching
echo `./${ADDR[1]}.sh ${ADDR[2]} &`
TST='test'
TEST=${ADDR[3]}_$TST
echo $TEST
VAR=$(echo -E "${ADDR[3]}" | tr -d '\n')
#countdown set to init value
sed -i "$NUMBER c $VAR ${ADDR[1]} ${ADDR[2]} ${ADDR[3]}" listing.txt
else
sed -i "$NUMBER c $((ADDR-1)) ${ADDR[1]} ${ADDR[2]} ${ADDR[3]}" listing.txt
fi
((NUMBER++))
done } < listing.txt
Answer: the following is fine,
TEST="${ADDR[3]}"_"$TST"
Although I would recommend.
TEST="${ADDR[3]}_${TST}"
What you need to do is dump ${ADDR[3]} before this statement and confirm that ADDR holds the expected values. You may as well dump the entire array with indexes and confirm all entries
for ((i=0; i<${#ADDR[#]}; i++)); do
printf "ADDR[%3d] %s\n" "$i" "${ADDR[$i]}"
done
This will help isolate the issue. Sorry for the earlier answer. Lesson [sleep 1st: answer 2nd]
I'm trying to write a small script that will count entries in a log file, and I'm incrementing a variable (USCOUNTER) which I'm trying to use after the loop is done.
But at that moment USCOUNTER looks to be 0 instead of the actual value. Any idea what I'm doing wrong? Thanks!
FILE=$1
tail -n10 mylog > $FILE
USCOUNTER=0
cat $FILE | while read line; do
country=$(echo "$line" | cut -d' ' -f1)
if [ "US" = "$country" ]; then
USCOUNTER=`expr $USCOUNTER + 1`
echo "US counter $USCOUNTER"
fi
done
echo "final $USCOUNTER"
It outputs:
US counter 1
US counter 2
US counter 3
..
final 0
You are using USCOUNTER in a subshell, that's why the variable is not showing in the main shell.
Instead of cat FILE | while ..., do just a while ... done < $FILE. This way, you avoid the common problem of I set variables in a loop that's in a pipeline. Why do they disappear after the loop terminates? Or, why can't I pipe data to read?:
while read country _; do
if [ "US" = "$country" ]; then
USCOUNTER=$(expr $USCOUNTER + 1)
echo "US counter $USCOUNTER"
fi
done < "$FILE"
Note I also replaced the `` expression with a $().
I also replaced while read line; do country=$(echo "$line" | cut -d' ' -f1) with while read country _. This allows you to say while read var1 var2 ... varN where var1 contains the first word in the line, $var2 and so on, until $varN containing the remaining content.
Always use -r with read.
There is no need to use cut, you can stick with pure bash solutions.
In this case passing read a 2nd var (_) to catch the additional "fields"
Prefer [[ ]] over [ ].
Use arithmetic expressions.
Do not forget to quote variables! Link includes other pitfalls as well
while read -r country _; do
if [[ $country = 'US' ]]; then
((USCOUNTER++))
echo "US counter $USCOUNTER"
fi
done < "$FILE"
minimalist
counter=0
((counter++))
echo $counter
You're getting final 0 because your while loop is being executed in a sub (shell) process and any changes made there are not reflected in the current (parent) shell.
Correct script:
while read -r country _; do
if [ "US" = "$country" ]; then
((USCOUNTER++))
echo "US counter $USCOUNTER"
fi
done < "$FILE"
I had the same $count variable in a while loop getting lost issue.
#fedorqui's answer (and a few others) are accurate answers to the actual question: the sub-shell is indeed the problem.
But it lead me to another issue: I wasn't piping a file content... but the output of a series of pipes & greps...
my erroring sample code:
count=0
cat /etc/hosts | head | while read line; do
((count++))
echo $count $line
done
echo $count
and my fix thanks to the help of this thread and the process substitution:
count=0
while IFS= read -r line; do
((count++))
echo "$count $line"
done < <(cat /etc/hosts | head)
echo "$count"
USCOUNTER=$(grep -c "^US " "$FILE")
Incrementing a variable can be done like that:
_my_counter=$[$_my_counter + 1]
Counting the number of occurrence of a pattern in a column can be done with grep
grep -cE "^([^ ]* ){2}US"
-c count
([^ ]* ) To detect a colonne
{2} the colonne number
US your pattern
Using the following 1 line command for changing many files name in linux using phrase specificity:
find -type f -name '*.jpg' | rename 's/holiday/honeymoon/'
For all files with the extension ".jpg", if they contain the string "holiday", replace it with "honeymoon". For instance, this command would rename the file "ourholiday001.jpg" to "ourhoneymoon001.jpg".
This example also illustrates how to use the find command to send a list of files (-type f) with the extension .jpg (-name '*.jpg') to rename via a pipe (|). rename then reads its file list from standard input.
I am having a bash script which is something like following,
cat filename | while read line
do
read input;
echo $input;
done
but this is clearly not giving me the right output as when I do read in the while loop it tries to read from the file filename because of the possible I/O redirection.
Any other way of doing the same?
Read from the controlling terminal device:
read input </dev/tty
more info: http://compgroups.net/comp.unix.shell/Fixing-stdin-inside-a-redirected-loop
You can redirect the regular stdin through unit 3 to keep the get it inside the pipeline:
{ cat notify-finished | while read line; do
read -u 3 input
echo "$input"
done; } 3<&0
BTW, if you really are using cat this way, replace it with a redirect and things become even easier:
while read line; do
read -u 3 input
echo "$input"
done 3<&0 <notify-finished
Or, you can swap stdin and unit 3 in that version -- read the file with unit 3, and just leave stdin alone:
while read line <&3; do
# read & use stdin normally inside the loop
read input
echo "$input"
done 3<notify-finished
Try to change the loop like this:
for line in $(cat filename); do
read input
echo $input;
done
Unit test:
for line in $(cat /etc/passwd); do
read input
echo $input;
echo "[$line]"
done
I have found this parameter -u with read.
"-u 1" means "read from stdout"
while read -r newline; do
((i++))
read -u 1 -p "Doing $i""th file, called $newline. Write your answer and press Enter!"
echo "Processing $newline with $REPLY" # united input from two different read commands.
done <<< $(ls)
It looks like you read twice, the read inside the while loop is not needed. Also, you don't need to invoke the cat command:
while read input
do
echo $input
done < filename
echo "Enter the Programs you want to run:"
> ${PROGRAM_LIST}
while read PROGRAM_ENTRY
do
if [ ! -s ${PROGRAM_ENTRY} ]
then
echo ${PROGRAM_ENTRY} >> ${PROGRAM_LIST}
else
break
fi
done