Using a variable in a bash script for loop - bash

I have a file which has the number of .pdfs in my folder. I assign this number to a variable, fileNum like so:
fileNum=$(ls -l *.pdf | wc -l)
echo $fileNum returns this number without any problem.
Now I need to use fileNum in a for loop and I am having problems with it.
My for loop is:
for i in {1..$fileNum}
do
var=$(awk 'NR=='$i 'pdfs.file')
gs \
-sOutputFile="exgs_"$var \
-sDEVICE=pdfwrite \
-sColorConversionStrategy=Gray \
-dProcessColorModel=/DeviceGray \
-dCompatibilityLevel=1.4 \
-dNOPAUSE \
-dBATCH \
$var
done
The $ at the beginning of fileNum gives me an error message which is:
awk: line 1: syntax error at or near {
Things are fine when I actually use the number itself (which in this case is 17).
Obviously, awk doesn't like this because of ... something.... I don't know what. What should I do about this?
I tried other forms such as $(fileNum) and filenum with the single quotes around it.
Is it something to do with strings?

I'd use bash to read the file instead of running awk for every line.
while read -r file; do
gs -sOutputFile="exgs_$file" \
-sDEVICE=pdfwrite \
-sColorConversionStrategy=Gray \
-dProcessColorModel=/DeviceGray \
-dCompatibilityLevel=1.4 \
-dNOPAUSE \
-dBATCH \
"$file"
done < pdfs.file
See also http://mywiki.wooledge.org/BashFAQ/001
Otherwise, for the general case where you want to iterate from 1 to n, I'd use a C-style for-loop.
n=10
for (( i=1; i <= n; i++)); do
...
done

This is because Bash will do the expansion on the braces before the variable. You need to use eval in this case so that Bash expands the variable first.
for i in $(eval echo {1..$fileNum})
do
var=$(awk 'NR=='$i 'pdfs.file')
gs \
-sOutputFile="exgs_"$var \
-sDEVICE=pdfwrite \
-sColorConversionStrategy=Gray \
-dProcessColorModel=/DeviceGray \
-dCompatibilityLevel=1.4 \
-dNOPAUSE \
-dBATCH \
$var
done

I would rather write:
for i in $(seq $fileNum)
do
....
done

Related

How can I pass a directory argument to this fzf/ripgrep bash script for previewing search results?

fzf/ripgrep can search an alternate directory from the command line with something like.
rg something ./sompath | fzf
The fzf documentation has a nice little bash script for generating a preview of rg's results that I call with a shell alias and then open the file with vim:
#!/usr/bin/env bash
RG_PREFIX="rg --column --line-number --no-heading --color=always --smart-case "
INITIAL_QUERY="${*:-}"
IFS=: read -ra selected < <(
FZF_DEFAULT_COMMAND="$RG_PREFIX $(printf %q "$INITIAL_QUERY")" \
fzf --ansi \
--color "hl:-1:underline,hl+:-1:underline:reverse" \
--disabled --query "$INITIAL_QUERY" \
--bind "change:reload:sleep 0.1; $RG_PREFIX {q} || true" \
--bind "alt-enter:unbind(change,alt-enter)+change-prompt(2. fzf> )+enable-search+clear-query" \
--prompt '1. ripgrep> ' \
--delimiter : \
--preview "$BAT_CMD --color=always {1} --highlight-line {2}" \
--preview-window 'up,60%,border-bottom,+{2}+3/3,~3'
)
[ -n "${selected[0]}" ] && nvim "${selected[0]}" "+${selected[1]}"
It's very cool and works great. Unfortunately, there's no way to pass in a directory argument into this script. So you have to cd into the desired directory, do the search, and cd back.
Instead, I'd like to do something like this:
search_script initial_query ./some_dir
But my bash skills are weak and I'm not really sure what the best approach is for processing an optional directory argument.
The script has to somehow be smart enough to recognize when a directory argument is passed and when it isn't. I'm not sure if some kind of option string like --dir is the best way to go or what. And I'm wondering if I might be missing something really obvious solution, too.
Thanks.
I was able to cargo cult some little bash snippets and piece something that seems to do the trick and detect if last argument is a directory:
RG_PREFIX="rg --column --line-number --no-heading --color=always --smart-case "
INITIAL_QUERY="${*:-}"
last="${#: -1}"
if [[ -d $last ]]; then
INITIAL_QUERY="${#:1:$#-1}";
dir=$last
echo $INITIAL_QUERY
fi
IFS=: read -ra selected < <(
FZF_DEFAULT_COMMAND="$RG_PREFIX $(printf %q "$INITIAL_QUERY") $dir" \
fzf --ansi \
--color "hl:-1:underline,hl+:-1:underline:reverse" \
--disabled --query "$INITIAL_QUERY" \
--bind "change:reload:sleep 0.1; $RG_PREFIX {q} $dir || true" \
--bind "alt-enter:unbind(change,alt-enter)+change-prompt(2. fzf> )+enable-search+clear-query" \
--prompt '1. ripgrep> ' \
--delimiter : \
--preview "bat --color=always {1} --highlight-line {2}" \
--preview-window 'up,60%,border-bottom,+{2}+3/3,~3'
)
[ -n "${selected[0]}" ] && nvim "${selected[0]}" "+${selected[1]}"
If you think there is a simpler way or a better, I'd love to hear.

Add a conditional SORT variable to mongoexport

I'm trying to set up a conditional --sort option in mongoexport but I'm having trouble with the string interpretation of my variable.
Here is the code I'm trying to run :
#!/bin/bash
if [[ $IS_PROD == "true" ]]
then
SORT='--sort "{_id : -1}"'
else
SORT=""
fi
$MONGODB_HOME/bin/mongoexport \
--host=$HOST \
--port=$PORT \
--username=$USER \
--password=$PWD \
--db=$DB \
--limit=$LIMIT \
$SORT \
--collection=my_collection | \
sed 's,\\,\\\\,g' \
> $TMP_FILE
While running this I get the following error error parsing command line options: invalid argument for flag '--sort' (expected string): invalid syntax
I've tried several quotes configuration and still couldn't make it work. Could someone please help me on this one?
Thanks
using bash array
#!/bin/bash
if [[ $IS_PROD == "true" ]]
then
SORT=(--sort "{_id : -1}")
else
SORT=()
fi
$MONGODB_HOME/bin/mongoexport \
--host=$HOST \
--port=$PORT \
--username=$USER \
--password=$PWD \
--db=$DB \
--limit=$LIMIT \
"${SORT[#]}" \
--collection=my_collection | \
sed 's,\\,\\\\,g' \
> $TMP_FILE
Explanation: using single quotes prevent shell expansions and double quotes are literal, but after variable expansion the double quotes are still litteral and expanded string is split by spaces.
Otherwise to work around unbound variable bug
#!/bin/bash
options=(--host=$HOST \
--port=$PORT \
--username=$USER \
--password=$PWD \
--db=$DB \
--limit=$LIMIT)
if [[ $IS_PROD == "true" ]]
then
options+=(--sort "{_id : -1}")
fi
$MONGODB_HOME/bin/mongoexport \
"${options[#]}" \
--collection=my_collection | \
sed 's,\\,\\\\,g' \
> $TMP_FILE

Bash - pass argument from array

I am using bash to call tool written in java (gatk) and I need to pass multiple arguments from array as input arguments. I tried it this way, but it seems not working. Could you please help me, how to solve it?
Code:
java $GATK \
-T GenotypeGVCFs \
-R $ref \
-o output.vcf \
for foo in array
do
--variant $foo \
done
What i want to be called:
java $GATK \
-T GenotypeGVCFs \
-R $ref \
-o output.vcf \
for foo in array
do
--variant file1 \
--variant file2 \
--variant file3 ...etc
done
edit: sorry for misunderstandings
array=("file1","file2","file3"...)
Thanks
I assume that what you actually want is that if array contains a b c, to have the command
java $GATK \
-T GenotypeGVCFs \
-R $ref \
-o output.vcf \
--variant a --variant b --variant c
If that is so, you can prepare a second array:
array=("file 1" "file 2" "file 3")
declare -a fullarray
for i in "${array[#]}"
do
fullarray+=( --variant "$i" )
done
And then
java $GATK \
-T GenotypeGVCFs \
-R $ref \
-o output.vcf \
"${fullarray[#]}"
This will also make sure that if any of the names in array contains a space, it will still be passed as a proper parameter and not split into two (assuming that you didn't mess it up when you added it to the array).
With echo and $():
array=(file1 file2 file3)
java $GATK \
-T GenotypeGVCFs \
-R $ref \
-o output.vcf \
$(for foo in ${array[*]}
do
echo -n " --variant $foo"
done)
You can do this with the following:
java $GATK \
-T GenotypeGVCFs \
-R $ref \
-o output.vcf \
${array[*]/#/ --variant }
#RealSkeptic's answer is the best. I'd write, for readability:
array=( "file 1" "file 2" "file 3" )
args=(
"$GATK"
-T GenotypeGVCFs
-R "$ref"
-o output.vcf
)
for foo in "${array[#]}"; do args+=( --variant "$foo" ); done
java "${args[#]}"

Nested For loop in makefile

I am trying to loop through the .c files in a specific directory through the makefile.
i used the following code, but it seems not working:
DIR= Sources \
Sources_2
#for entry in ${DIR} ; \
do \
#for i in $${entry}/*.c ; \
do \
echo "Processing $${i}"; \
#Building Commands go here
done \
done
I get error: "/bin/sh: -c: line 3: syntax error near unexpected token `do'"
You shouldn't use at sign # near the second for loop. # should be used at the beginning of the whole shell command. The following worked for me:
DIR= Sources \
Sources_2
all:
#for entry in ${DIR}; \
do \
for i in $${entry}/*.c; \
do \
echo "Processing $${i}"; \
done \
done
Change the Makefile as below
#for entry in ${DIR} ; do \
for i in $$entry/*.c ; do \
echo "Processing $$i"; \
#Building Commands go here
done \
done
The reason being wrong syntax usage of for loop.

Unix shell bash script unable to put an echo debug statement

I have the following make file, which i think is a shell script.
I am trying to loop through FILE_DIR to perform some operations. However, i feel that the implementation isn't working as expected. So i am trying to insert some echo breakpoints.
Source:
# Target to recurse through the DIR_LIST and make each makefile found in that DIRS
ALLDIRS:
for se in $(FILE_DIR); do \
if [ -d $se ]; then \
cd $se; \
$(MAKE) -f Makefile.mk all; \
cd ..; \
fi \
done
Running:
$ make -f Makefile.batch
h: syntax error at line 3: `then' unexpected
*** Error code 2
The following command caused the error:
for se in `ls -p /app/internal|grep "/"`; do \
echo "Test" \
if [ -d e ]; then \
cd e; \
/usr/ccs/bin/make -f Makefile.mk all; \
cd ..; \
fi \
done
make: Fatal error: Command failed for target `ALLDIRS'
Can i please get help on this. Would like to insert an echo breakpoint.
One common error in Makefiles is using spaces instead of tabs in a command line. Check the whole for loop and make sure there are only tabs at the beginning of each line
ALLDIRS:
<tab>for se in $(FILE_DIR); do \
<tab><tab>if [ -d $se ]; then \
<tab><tab>cd $se; \
<tab><tab>$(MAKE) -f Makefile.mk all; \
<tab><tab>cd ..; \
<tab><tab>fi \
<tab>done
Another error is the dollar sign $. If you want a dollar sign in the shell command, you must double it in your commands, because otherwise dollar introduces a make variable and will be expanded before the shell sees it.
for se in $(FILE_DIR); do \
if [ -d $$se ]; then \
cd $$se; \
$(MAKE) -f Makefile.mk all; \
cd ..; \
fi \
done
And the final one, echo Test needs a semicolon as well
for se in $(FILE_DIR); do \
if [ -d $$se ]; then \
echo "Test"; \
cd $$se; \
...

Resources