Add a conditional SORT variable to mongoexport - bash

I'm trying to set up a conditional --sort option in mongoexport but I'm having trouble with the string interpretation of my variable.
Here is the code I'm trying to run :
#!/bin/bash
if [[ $IS_PROD == "true" ]]
then
SORT='--sort "{_id : -1}"'
else
SORT=""
fi
$MONGODB_HOME/bin/mongoexport \
--host=$HOST \
--port=$PORT \
--username=$USER \
--password=$PWD \
--db=$DB \
--limit=$LIMIT \
$SORT \
--collection=my_collection | \
sed 's,\\,\\\\,g' \
> $TMP_FILE
While running this I get the following error error parsing command line options: invalid argument for flag '--sort' (expected string): invalid syntax
I've tried several quotes configuration and still couldn't make it work. Could someone please help me on this one?
Thanks

using bash array
#!/bin/bash
if [[ $IS_PROD == "true" ]]
then
SORT=(--sort "{_id : -1}")
else
SORT=()
fi
$MONGODB_HOME/bin/mongoexport \
--host=$HOST \
--port=$PORT \
--username=$USER \
--password=$PWD \
--db=$DB \
--limit=$LIMIT \
"${SORT[#]}" \
--collection=my_collection | \
sed 's,\\,\\\\,g' \
> $TMP_FILE
Explanation: using single quotes prevent shell expansions and double quotes are literal, but after variable expansion the double quotes are still litteral and expanded string is split by spaces.
Otherwise to work around unbound variable bug
#!/bin/bash
options=(--host=$HOST \
--port=$PORT \
--username=$USER \
--password=$PWD \
--db=$DB \
--limit=$LIMIT)
if [[ $IS_PROD == "true" ]]
then
options+=(--sort "{_id : -1}")
fi
$MONGODB_HOME/bin/mongoexport \
"${options[#]}" \
--collection=my_collection | \
sed 's,\\,\\\\,g' \
> $TMP_FILE

Related

query return null in bash for psql

I am tring to get the count for the records in a certain table in bash scripts but the $num is null when returned (should be a number). And the query is correct when I directly run in pgadmin that I can get the number of rows. Any one know what is wrong?
declare -a ROW=($(psql \
-X \
-U $DB_USER \
-h $DB_HOST \
-d $DB_NAME \
-p $DB_PORT \
--set ON_ERROR_STOP=on \
--no-align \
-t \
--field-separator ' ' \
--quiet \
-c "SELECT count(*) as num
FROM table_test)")
)
echo "num_error: $num here"
if [[ $num == 0 ]]; then
echo "no error occur within the past 1 hour"
elif [[ $num == '' ]]; then
echo "return nothing"
else echo "$num"
fi
SQL aliases don't become shell variables, so using AS num in the query will not set $num in the shell.
The output of the query is being put in the ROW array, so you can get the value you want from ${ROW[0]}. There's also no need to use an array if the query just returns a single value. So you could do:
num=$(psql \
-X \
-U $DB_USER \
-h $DB_HOST \
-d $DB_NAME \
-p $DB_PORT \
--set ON_ERROR_STOP=on \
--no-align \
-t \
--field-separator ' ' \
--quiet \
-c "SELECT count(*)
FROM table_test)")
)

Bash - pass argument from array

I am using bash to call tool written in java (gatk) and I need to pass multiple arguments from array as input arguments. I tried it this way, but it seems not working. Could you please help me, how to solve it?
Code:
java $GATK \
-T GenotypeGVCFs \
-R $ref \
-o output.vcf \
for foo in array
do
--variant $foo \
done
What i want to be called:
java $GATK \
-T GenotypeGVCFs \
-R $ref \
-o output.vcf \
for foo in array
do
--variant file1 \
--variant file2 \
--variant file3 ...etc
done
edit: sorry for misunderstandings
array=("file1","file2","file3"...)
Thanks
I assume that what you actually want is that if array contains a b c, to have the command
java $GATK \
-T GenotypeGVCFs \
-R $ref \
-o output.vcf \
--variant a --variant b --variant c
If that is so, you can prepare a second array:
array=("file 1" "file 2" "file 3")
declare -a fullarray
for i in "${array[#]}"
do
fullarray+=( --variant "$i" )
done
And then
java $GATK \
-T GenotypeGVCFs \
-R $ref \
-o output.vcf \
"${fullarray[#]}"
This will also make sure that if any of the names in array contains a space, it will still be passed as a proper parameter and not split into two (assuming that you didn't mess it up when you added it to the array).
With echo and $():
array=(file1 file2 file3)
java $GATK \
-T GenotypeGVCFs \
-R $ref \
-o output.vcf \
$(for foo in ${array[*]}
do
echo -n " --variant $foo"
done)
You can do this with the following:
java $GATK \
-T GenotypeGVCFs \
-R $ref \
-o output.vcf \
${array[*]/#/ --variant }
#RealSkeptic's answer is the best. I'd write, for readability:
array=( "file 1" "file 2" "file 3" )
args=(
"$GATK"
-T GenotypeGVCFs
-R "$ref"
-o output.vcf
)
for foo in "${array[#]}"; do args+=( --variant "$foo" ); done
java "${args[#]}"

Unix shell bash script unable to put an echo debug statement

I have the following make file, which i think is a shell script.
I am trying to loop through FILE_DIR to perform some operations. However, i feel that the implementation isn't working as expected. So i am trying to insert some echo breakpoints.
Source:
# Target to recurse through the DIR_LIST and make each makefile found in that DIRS
ALLDIRS:
for se in $(FILE_DIR); do \
if [ -d $se ]; then \
cd $se; \
$(MAKE) -f Makefile.mk all; \
cd ..; \
fi \
done
Running:
$ make -f Makefile.batch
h: syntax error at line 3: `then' unexpected
*** Error code 2
The following command caused the error:
for se in `ls -p /app/internal|grep "/"`; do \
echo "Test" \
if [ -d e ]; then \
cd e; \
/usr/ccs/bin/make -f Makefile.mk all; \
cd ..; \
fi \
done
make: Fatal error: Command failed for target `ALLDIRS'
Can i please get help on this. Would like to insert an echo breakpoint.
One common error in Makefiles is using spaces instead of tabs in a command line. Check the whole for loop and make sure there are only tabs at the beginning of each line
ALLDIRS:
<tab>for se in $(FILE_DIR); do \
<tab><tab>if [ -d $se ]; then \
<tab><tab>cd $se; \
<tab><tab>$(MAKE) -f Makefile.mk all; \
<tab><tab>cd ..; \
<tab><tab>fi \
<tab>done
Another error is the dollar sign $. If you want a dollar sign in the shell command, you must double it in your commands, because otherwise dollar introduces a make variable and will be expanded before the shell sees it.
for se in $(FILE_DIR); do \
if [ -d $$se ]; then \
cd $$se; \
$(MAKE) -f Makefile.mk all; \
cd ..; \
fi \
done
And the final one, echo Test needs a semicolon as well
for se in $(FILE_DIR); do \
if [ -d $$se ]; then \
echo "Test"; \
cd $$se; \
...

Using a variable in a bash script for loop

I have a file which has the number of .pdfs in my folder. I assign this number to a variable, fileNum like so:
fileNum=$(ls -l *.pdf | wc -l)
echo $fileNum returns this number without any problem.
Now I need to use fileNum in a for loop and I am having problems with it.
My for loop is:
for i in {1..$fileNum}
do
var=$(awk 'NR=='$i 'pdfs.file')
gs \
-sOutputFile="exgs_"$var \
-sDEVICE=pdfwrite \
-sColorConversionStrategy=Gray \
-dProcessColorModel=/DeviceGray \
-dCompatibilityLevel=1.4 \
-dNOPAUSE \
-dBATCH \
$var
done
The $ at the beginning of fileNum gives me an error message which is:
awk: line 1: syntax error at or near {
Things are fine when I actually use the number itself (which in this case is 17).
Obviously, awk doesn't like this because of ... something.... I don't know what. What should I do about this?
I tried other forms such as $(fileNum) and filenum with the single quotes around it.
Is it something to do with strings?
I'd use bash to read the file instead of running awk for every line.
while read -r file; do
gs -sOutputFile="exgs_$file" \
-sDEVICE=pdfwrite \
-sColorConversionStrategy=Gray \
-dProcessColorModel=/DeviceGray \
-dCompatibilityLevel=1.4 \
-dNOPAUSE \
-dBATCH \
"$file"
done < pdfs.file
See also http://mywiki.wooledge.org/BashFAQ/001
Otherwise, for the general case where you want to iterate from 1 to n, I'd use a C-style for-loop.
n=10
for (( i=1; i <= n; i++)); do
...
done
This is because Bash will do the expansion on the braces before the variable. You need to use eval in this case so that Bash expands the variable first.
for i in $(eval echo {1..$fileNum})
do
var=$(awk 'NR=='$i 'pdfs.file')
gs \
-sOutputFile="exgs_"$var \
-sDEVICE=pdfwrite \
-sColorConversionStrategy=Gray \
-dProcessColorModel=/DeviceGray \
-dCompatibilityLevel=1.4 \
-dNOPAUSE \
-dBATCH \
$var
done
I would rather write:
for i in $(seq $fileNum)
do
....
done

Tricky makefile syntax with quotes

I have the following start on a makefile rule (thanks to help from others), but it doesn't quite work yet:
test_svn_version:
#if [ $$(svn --version --quiet \
perl -ne '#a=split(/\./); \
print $$a[0]*10000 + $$a[1]*100 + $$a[2]') \
-lt 10600 ]; \
then \
echo >&2 "Svn version $$(svn --version --quiet) too old; upgrade to v1.6";
false; \
fi
It seems the single quote in the conditional is unmatched.
Please help correct the syntax. I've tried many variants, but none seem correct.
Thanks.
-William
You're missing a pipe | between svn and perl, and you're missing a backslash \ after the echo. This works for me:
test_svn_version:
#if [ $$(svn --version --quiet | \
perl -ne '#a=split(/\./); \
print $$a[0]*10000 + $$a[1]*100 + $$a[2]') \
-lt 10600 ]; \
then \
echo >&2 "Svn version $$(svn --version --quiet) too old; upgrade to v1.6"; \
false; \
fi

Resources