I have written a bash script, but I cannot understand what I am doing wrong. Here it is:
#!/bin/bash
VAR_LIST=('var_A' 'var_B' 'var_C')
FilePrefix='mystring_'
counter=1
for i in "${VAR_LIST[#]}"
do
grep "$i" > "${FilePrefix}${counter}.tab.txt"
counter=$((counter+1))
done <$1
Results: a file is created for each variable, but only the first one has data in it; the other files are empty. I have verified manually that each variable should return a few lines. The strange thing is that if I just do:
for i in "${VAR_LIST[#]}"
do
echo "$i"
done
I get the list of all the variables in VAR_LIST
You're only opening the input file once -- which means that once the first grep has read it, there's nothing left for future ones to read.
Since bash doesn't have a way to call seek(), you'll need to re-open the file once per invocation:
for i in "${VAR_LIST[#]}"
do
grep "$i" >"${FilePrefix}${counter}.tab.txt" <"$1"
counter=$((counter+1))
done
Related
I'm writing a simple Bash script that simply the call of HadnBrakeCli for render videos.
I also implemented a simple queue option: the queue file just store the line-command it has to call to start a render.
So I wrote a while-loop to read one line at time, eval $line and repeat untill file ends.
if [[ ${QUEUE_MODE} = 'RUN' ]]; then
QUEUE_LEN=`cat ${CONFIG_DIR}/queue | wc -l`
QUEUE_POS='1'
printf "Queue lenght:\t ${QUEUE_LEN}\n"
while IFS= read line; do
echo "--Running render ${QUEUE_POS} on ${QUEUE_LEN}..."
echo "++" && echo "$line" && echo "++"
eval "${line}"
tail -n +2 "${CONFIG_DIR}/queue" > "${CONFIG_DIR}/queue.tmp" && mv "${CONFIG_DIR}/queue.tmp" "${CONFIG_DIR}/queue"
echo "--Render ended"
QUEUE_POS=`expr $QUEUE_POS + 1`
done < "${CONFIG_DIR}/queue"
exit 0
The problem is that any command makes the loop to work fine (empty line, echo "test"...), but as soon a proper render is loaded, it is launched and finished correctly, but also the loops exists.
I am a newbie so I tried some minor changes to see what effect I got, but nothing change the result.
I commented the command tail -n +2 "${CONFIG_DIR}/queue" > "${CONFIG_DIR}/queue.tmp" && mv "${CONFIG_DIR}/queue.tmp" "${CONFIG_DIR}/queue" or I added/removed IFS= in the while-loop or removed the -r in read command.
Sorry if the question is trivial, but I'm really missing some major part in how it works, so I have no idea even how to search for the solution.
I'll put a sample of a general render in the queue file.
HandBrakeCLI -i "/home/andrea/Videos/done/Rap dottor male e mini me.mp4" -o "/hdd/Render/Output/Rap dottor male e mini me.mkv" -e x265 -q 23 --encoder-preset faster --all-audio -E av_aac -6 dpl2 --all-subtitles -x pmode:pools='16' --verbose=0 2>/dev/null
HandBrakeCLI reads from standard input, which steals the rest of the queue file before read line can see it. My favorite solution to this is to pass the file over something other than standard input, like file descriptor #3:
...
while IFS= read line <&3; do # The <&3 makes it read from FD #3
...
done 3< "${CONFIG_DIR}/queue" # The 3< redirects the file into FD #3
Another way to avoid the problem is to redirect input to the HandBrakeCLI command:
...
eval "${line}" </dev/null
...
There's some more info about this in BashFAQ #89: I'm reading a file line by line and running ssh or ffmpeg, only the first line gets processed!
Also, I'm not sure I trust the way you're using tail to remove lines from the queue file as they're executed. I'm not sure it's really wrong, it just looks fragile to me. Also, I'd recommend using lower- or mixed-case variable names, since there are a bunch of all-caps names with special meanings, and re-using one of them by mistake can have weird consequences. Finally, I'd recommend running your script through shellcheck.net, as it'll make some other good recommendations.
[BTW, this question is a duplicate of "Bash script do loop exiting early", but that doesn't have any upvoted or accepted answers.]
Im working on a bash script that recive a list of processes and do a bunch of things with them, however when I want to analyze them with a loop this error happens.
Here is some code:
#! /bin/bash
ls /proc | grep ^9 > processes.txt
cat processes.txt
for line in $processes.txt
do
echo "$line"
done
PD: Im preatty new to bash
$ does parameter expansion; it does not expand a file name to the contents of the file.
Use a while read loop instead.
while IFS= read -r line; do
echo "$line"
done < processes.txt
I'm writing a simple Bash script that simply the call of HadnBrakeCli for render videos.
I also implemented a simple queue option: the queue file just store the line-command it has to call to start a render.
So I wrote a while-loop to read one line at time, eval $line and repeat untill file ends.
if [[ ${QUEUE_MODE} = 'RUN' ]]; then
QUEUE_LEN=`cat ${CONFIG_DIR}/queue | wc -l`
QUEUE_POS='1'
printf "Queue lenght:\t ${QUEUE_LEN}\n"
while IFS= read line; do
echo "--Running render ${QUEUE_POS} on ${QUEUE_LEN}..."
echo "++" && echo "$line" && echo "++"
eval "${line}"
tail -n +2 "${CONFIG_DIR}/queue" > "${CONFIG_DIR}/queue.tmp" && mv "${CONFIG_DIR}/queue.tmp" "${CONFIG_DIR}/queue"
echo "--Render ended"
QUEUE_POS=`expr $QUEUE_POS + 1`
done < "${CONFIG_DIR}/queue"
exit 0
The problem is that any command makes the loop to work fine (empty line, echo "test"...), but as soon a proper render is loaded, it is launched and finished correctly, but also the loops exists.
I am a newbie so I tried some minor changes to see what effect I got, but nothing change the result.
I commented the command tail -n +2 "${CONFIG_DIR}/queue" > "${CONFIG_DIR}/queue.tmp" && mv "${CONFIG_DIR}/queue.tmp" "${CONFIG_DIR}/queue" or I added/removed IFS= in the while-loop or removed the -r in read command.
Sorry if the question is trivial, but I'm really missing some major part in how it works, so I have no idea even how to search for the solution.
I'll put a sample of a general render in the queue file.
HandBrakeCLI -i "/home/andrea/Videos/done/Rap dottor male e mini me.mp4" -o "/hdd/Render/Output/Rap dottor male e mini me.mkv" -e x265 -q 23 --encoder-preset faster --all-audio -E av_aac -6 dpl2 --all-subtitles -x pmode:pools='16' --verbose=0 2>/dev/null
HandBrakeCLI reads from standard input, which steals the rest of the queue file before read line can see it. My favorite solution to this is to pass the file over something other than standard input, like file descriptor #3:
...
while IFS= read line <&3; do # The <&3 makes it read from FD #3
...
done 3< "${CONFIG_DIR}/queue" # The 3< redirects the file into FD #3
Another way to avoid the problem is to redirect input to the HandBrakeCLI command:
...
eval "${line}" </dev/null
...
There's some more info about this in BashFAQ #89: I'm reading a file line by line and running ssh or ffmpeg, only the first line gets processed!
Also, I'm not sure I trust the way you're using tail to remove lines from the queue file as they're executed. I'm not sure it's really wrong, it just looks fragile to me. Also, I'd recommend using lower- or mixed-case variable names, since there are a bunch of all-caps names with special meanings, and re-using one of them by mistake can have weird consequences. Finally, I'd recommend running your script through shellcheck.net, as it'll make some other good recommendations.
[BTW, this question is a duplicate of "Bash script do loop exiting early", but that doesn't have any upvoted or accepted answers.]
probably quite easy question for you but has been a while I'm stuck with this issue.
Basically i want to perform some operation in every csv file in a folder and i put together this simple script:
#!/bin/bash
for file in *.csv ;
do
echo $file #check
OLDIFS=$IFS
IFS=","
while read var1
do
echo $var1
done < $file
IFS=$OLDIFS
done
The csv files follow all the same format, 1 line and 12 variables like:
name,1,2,3,4,5,6,7,8,9,11
It seems to me that the while conditions is always false and never run.
Could you tell me where I'm wrong?
To get just the first field, you need to supply two arguments to read; the first for the first field, the second to hold the remaining values after the split.
#!/bin/bash
for file in *.csv; do
echo $file #check
while IFS=, read name rest; do
echo "$name"
done < "$file"
done
Something like this should work. It reads the file into an array, splitting it at every ,. It the iterates over the array and prints out each element on a new line.
#!/bin/bash
for file in *.csv
do
echo "$file" #check
IFS=',' read -a vars < "$file"
for var1 in ${vars[#]}
do
echo "$var1"
done
done
There are 2 pieces of code here, and the value in $1 is the name of a file which contains 3 lines of text.
Now, I have a problem. In the first piece of the code, I can't get the "right" value out of the loop, but in the second piece of the code, I can get the right result. I don't know why.
How can I make the first piece of the code get the right result?
#!/bin/bash
count=0
cat "$1" | while read line
do
count=$[ $count + 1 ]
done
echo "$count line(s) in all."
#-----------------------------------------
count2=0
for var in a b c
do
count2=$[ $count2 + 1 ]
done
echo "$count2 line(s) in all."
This happens because of the pipe before the while loop. It creates a sub-shell, and thus the changes in the variables are not passed to the main script. To overcome this, use process substitution instead:
while read -r line
do
# do some stuff
done < <( some commad)
In version 4.2 or later, you can also set the lastpipe option, and the last command
in the pipeline will run in the current shell, not a subshell.
shopt -s lastpipe
some command | while read -r line; do
# do some stuff
done
In this case, since you are just using the contents of the file, you can use input redirection:
while read -r line
do
# do some stuff
done < "$file"