I am using bash to call tool written in java (gatk) and I need to pass multiple arguments from array as input arguments. I tried it this way, but it seems not working. Could you please help me, how to solve it?
Code:
java $GATK \
-T GenotypeGVCFs \
-R $ref \
-o output.vcf \
for foo in array
do
--variant $foo \
done
What i want to be called:
java $GATK \
-T GenotypeGVCFs \
-R $ref \
-o output.vcf \
for foo in array
do
--variant file1 \
--variant file2 \
--variant file3 ...etc
done
edit: sorry for misunderstandings
array=("file1","file2","file3"...)
Thanks
I assume that what you actually want is that if array contains a b c, to have the command
java $GATK \
-T GenotypeGVCFs \
-R $ref \
-o output.vcf \
--variant a --variant b --variant c
If that is so, you can prepare a second array:
array=("file 1" "file 2" "file 3")
declare -a fullarray
for i in "${array[#]}"
do
fullarray+=( --variant "$i" )
done
And then
java $GATK \
-T GenotypeGVCFs \
-R $ref \
-o output.vcf \
"${fullarray[#]}"
This will also make sure that if any of the names in array contains a space, it will still be passed as a proper parameter and not split into two (assuming that you didn't mess it up when you added it to the array).
With echo and $():
array=(file1 file2 file3)
java $GATK \
-T GenotypeGVCFs \
-R $ref \
-o output.vcf \
$(for foo in ${array[*]}
do
echo -n " --variant $foo"
done)
You can do this with the following:
java $GATK \
-T GenotypeGVCFs \
-R $ref \
-o output.vcf \
${array[*]/#/ --variant }
#RealSkeptic's answer is the best. I'd write, for readability:
array=( "file 1" "file 2" "file 3" )
args=(
"$GATK"
-T GenotypeGVCFs
-R "$ref"
-o output.vcf
)
for foo in "${array[#]}"; do args+=( --variant "$foo" ); done
java "${args[#]}"
Related
fzf/ripgrep can search an alternate directory from the command line with something like.
rg something ./sompath | fzf
The fzf documentation has a nice little bash script for generating a preview of rg's results that I call with a shell alias and then open the file with vim:
#!/usr/bin/env bash
RG_PREFIX="rg --column --line-number --no-heading --color=always --smart-case "
INITIAL_QUERY="${*:-}"
IFS=: read -ra selected < <(
FZF_DEFAULT_COMMAND="$RG_PREFIX $(printf %q "$INITIAL_QUERY")" \
fzf --ansi \
--color "hl:-1:underline,hl+:-1:underline:reverse" \
--disabled --query "$INITIAL_QUERY" \
--bind "change:reload:sleep 0.1; $RG_PREFIX {q} || true" \
--bind "alt-enter:unbind(change,alt-enter)+change-prompt(2. fzf> )+enable-search+clear-query" \
--prompt '1. ripgrep> ' \
--delimiter : \
--preview "$BAT_CMD --color=always {1} --highlight-line {2}" \
--preview-window 'up,60%,border-bottom,+{2}+3/3,~3'
)
[ -n "${selected[0]}" ] && nvim "${selected[0]}" "+${selected[1]}"
It's very cool and works great. Unfortunately, there's no way to pass in a directory argument into this script. So you have to cd into the desired directory, do the search, and cd back.
Instead, I'd like to do something like this:
search_script initial_query ./some_dir
But my bash skills are weak and I'm not really sure what the best approach is for processing an optional directory argument.
The script has to somehow be smart enough to recognize when a directory argument is passed and when it isn't. I'm not sure if some kind of option string like --dir is the best way to go or what. And I'm wondering if I might be missing something really obvious solution, too.
Thanks.
I was able to cargo cult some little bash snippets and piece something that seems to do the trick and detect if last argument is a directory:
RG_PREFIX="rg --column --line-number --no-heading --color=always --smart-case "
INITIAL_QUERY="${*:-}"
last="${#: -1}"
if [[ -d $last ]]; then
INITIAL_QUERY="${#:1:$#-1}";
dir=$last
echo $INITIAL_QUERY
fi
IFS=: read -ra selected < <(
FZF_DEFAULT_COMMAND="$RG_PREFIX $(printf %q "$INITIAL_QUERY") $dir" \
fzf --ansi \
--color "hl:-1:underline,hl+:-1:underline:reverse" \
--disabled --query "$INITIAL_QUERY" \
--bind "change:reload:sleep 0.1; $RG_PREFIX {q} $dir || true" \
--bind "alt-enter:unbind(change,alt-enter)+change-prompt(2. fzf> )+enable-search+clear-query" \
--prompt '1. ripgrep> ' \
--delimiter : \
--preview "bat --color=always {1} --highlight-line {2}" \
--preview-window 'up,60%,border-bottom,+{2}+3/3,~3'
)
[ -n "${selected[0]}" ] && nvim "${selected[0]}" "+${selected[1]}"
If you think there is a simpler way or a better, I'd love to hear.
I got this script which reads a delimited part of my .gitignore file and remove all files after the given mark # #:
# https://stackoverflow.com/questions/55527923/how-to-stop-makefile-from-expanding-my-shell-output
RAW_GITIGNORE_CONTENTS := $(shell while read -r line; do printf "$$line "; done < ".gitignore")
GITIGNORE_CONTENTS := $(shell echo "$(RAW_GITIGNORE_CONTENTS)" | sed -E $$'s/[^\#]+\# //g')
# https://stackoverflow.com/questions/4210042/exclude-directory-from-find-command
DIRECTORIES_TO_CLEAN := $(shell /bin/find -not -path "./**.git**" -not -path "./pictures**" -type d)
clean:
# https://stackoverflow.com/questions/10586153/split-string-into-an-array-in-bash
# https://stackoverflow.com/questions/11289551/argument-list-too-long-error-for-rm-cp-mv-commands
readarray -td' ' GARBAGE_DIRECTORIES <<<"$(DIRECTORIES_TO_CLEAN) "; \
unset 'GARBAGE_DIRECTORIES[-1]'; \
declare -p GARBAGE_DIRECTORIES; \
readarray -td' ' GARBAGE_EXTENSIONS <<<"$(GITIGNORE_CONTENTS) "; \
unset 'GARBAGE_EXTENSIONS[-1]'; \
declare -p GARBAGE_EXTENSIONS; \
for filename in "$${GARBAGE_DIRECTORIES[#]}"; \
do \
arraylength="$${#GARBAGE_EXTENSIONS[#]}"; \
printf 'Cleaning %s extensions on %s\n' "$${arraylength}" "$$filename"; \
for extension in "$${GARBAGE_EXTENSIONS[#]}"; \
do \
[[ ! -z "$$filename" ]] || continue; \
[[ ! -z "$$extension" ]] || continue; \
full_expression="$${filename}/$${extension}" ;\
printf '%s\n' "$$full_expression"; \
rm -v "$$full_expression"; \
done; \
done;
Running it with the following .gitignore file:
*.txt
*.var
# Comment #
*.aux
The rm command is not expanding the wildcards and keeps telling me rm: cannot remove './*.aux': No such file or directory and do not remove the *.aux files from the ./ directory.
Update
After asked on a comment by #Beta, I simplified the Makefile to this:
GITIGNORE_CONTENTS := "*.aux" "*.lof"
DIRECTORIES_TO_CLEAN := "./setup/cache" "./setup/cache/chapters"
clean:
readarray -td' ' GARBAGE_DIRECTORIES <<<"$(DIRECTORIES_TO_CLEAN) "; \
unset 'GARBAGE_DIRECTORIES[-1]'; \
declare -p GARBAGE_DIRECTORIES; \
readarray -td' ' GARBAGE_EXTENSIONS <<<"$(GITIGNORE_CONTENTS) "; \
unset 'GARBAGE_EXTENSIONS[-1]'; \
declare -p GARBAGE_EXTENSIONS; \
for filename in "$${GARBAGE_DIRECTORIES[#]}"; \
do \
arraylength="$${#GARBAGE_EXTENSIONS[#]}"; \
printf 'Cleaning %s extensions on %s\n' "$${arraylength}" "$$filename"; \
for extension in "$${GARBAGE_EXTENSIONS[#]}"; \
do \
[[ ! -z "$$filename" ]] || continue; \
[[ ! -z "$$extension" ]] || continue; \
full_expression="$${filename}/$${extension}" ;\
printf '%s\n' "$$full_expression"; \
rm -vf "$$full_expression"; \
done; \
done;
Which results on this output after running it:
$ make
readarray -td' ' GARBAGE_DIRECTORIES <<<""./setup/cache" "./setup/cache/chapters" "; \
unset 'GARBAGE_DIRECTORIES[-1]'; \
declare -p GARBAGE_DIRECTORIES; \
readarray -td' ' GARBAGE_EXTENSIONS <<<""*.aux" "*.lof" "; \
unset 'GARBAGE_EXTENSIONS[-1]'; \
declare -p GARBAGE_EXTENSIONS; \
for filename in "${GARBAGE_DIRECTORIES[#]}"; \
do \
arraylength="${#GARBAGE_EXTENSIONS[#]}"; \
printf 'Cleaning %s extensions on %s\n' "${arraylength}" "$filename"; \
for extension in "${GARBAGE_EXTENSIONS[#]}"; \
do \
[[ ! -z "$filename" ]] || continue; \
[[ ! -z "$extension" ]] || continue; \
full_expression="${filename}/${extension}" ;\
printf '%s\n' "$full_expression"; \
rm -vf "$full_expression"; \
done; \
done;
declare -a GARBAGE_DIRECTORIES=([0]="./setup/cache" [1]="./setup/cache/chapters")
declare -a GARBAGE_EXTENSIONS=([0]="*.aux" [1]="*.lof")
Cleaning 2 extensions on ./setup/cache
./setup/cache/*.aux
./setup/cache/*.lof
Cleaning 2 extensions on ./setup/cache/chapters
./setup/cache/chapters/*.aux
./setup/cache/chapters/*.lof
More simplification
I reduced to the more simple version it could be:
clean:
rm -v "./setup/cache/*.aux";
Running this, also do not remove the files:
$ make
rm -v "./setup/cache/*.aux";
rm: cannot remove './setup/cache/*.aux': No such file or directory
make: *** [Makefile:3: clean] Error 1
$ ls ./setup/cache/*.aux
./setup/cache/main.aux
On above, after running ls, you can see the file still exists and it is there.
I managed to fix it by changing:
rm -vf "$$full_expression"; \
To:
rm -vf $${full_expression}; \
I am trying to do variable interpolation inside a command substitution in a Makefile.
I have this code:
setup:
mkdir -p data_all ; \
for i in $(shell jq -r 'keys | #tsv' assets.json) ; do \
git_url=$(shell jq -r ".$$i" assets.json) ; \
git clone $$git_url data_all/$$i ; \
done
The code is failing, however, because $$i does not expand in the "shell" line that sets git_url.
How do I interpolate the variable $i in the "shell" line that sets git_url?
You mixed up make functions ($(shell ...)) and true shell constructs. When writing a recipe the simplest is to write it first in plain shell:
mkdir -p data_all ; \
for i in $( jq -r 'keys | #tsv' assets.json ) ; do \
git_url=$( jq -r ".$i" assets.json ) ; \
git clone $git_url data_all/$i ; \
done
And then escaping the unwanted $ expansion by make:
mkdir -p data_all ; \
for i in $$( jq -r 'keys | #tsv' assets.json ) ; do \
git_url=$$( jq -r ".$$i" assets.json ) ; \
git clone $$git_url data_all/$$i ; \
done
I'm trying to set up a conditional --sort option in mongoexport but I'm having trouble with the string interpretation of my variable.
Here is the code I'm trying to run :
#!/bin/bash
if [[ $IS_PROD == "true" ]]
then
SORT='--sort "{_id : -1}"'
else
SORT=""
fi
$MONGODB_HOME/bin/mongoexport \
--host=$HOST \
--port=$PORT \
--username=$USER \
--password=$PWD \
--db=$DB \
--limit=$LIMIT \
$SORT \
--collection=my_collection | \
sed 's,\\,\\\\,g' \
> $TMP_FILE
While running this I get the following error error parsing command line options: invalid argument for flag '--sort' (expected string): invalid syntax
I've tried several quotes configuration and still couldn't make it work. Could someone please help me on this one?
Thanks
using bash array
#!/bin/bash
if [[ $IS_PROD == "true" ]]
then
SORT=(--sort "{_id : -1}")
else
SORT=()
fi
$MONGODB_HOME/bin/mongoexport \
--host=$HOST \
--port=$PORT \
--username=$USER \
--password=$PWD \
--db=$DB \
--limit=$LIMIT \
"${SORT[#]}" \
--collection=my_collection | \
sed 's,\\,\\\\,g' \
> $TMP_FILE
Explanation: using single quotes prevent shell expansions and double quotes are literal, but after variable expansion the double quotes are still litteral and expanded string is split by spaces.
Otherwise to work around unbound variable bug
#!/bin/bash
options=(--host=$HOST \
--port=$PORT \
--username=$USER \
--password=$PWD \
--db=$DB \
--limit=$LIMIT)
if [[ $IS_PROD == "true" ]]
then
options+=(--sort "{_id : -1}")
fi
$MONGODB_HOME/bin/mongoexport \
"${options[#]}" \
--collection=my_collection | \
sed 's,\\,\\\\,g' \
> $TMP_FILE
I have some .PHONY targets such as 'clean', 'backup', and 'help'
the rule for some of these targets is very large.
For example:
.PHONY: backup
backup:
#$(GREEN)
#mkdir -p backup/include #make an backup include folder if it doesn't already exist
#mkdir -p backup/src #make a backup src folder if it doesn't already exist
#mkdir -p backup/docs #make a backup docs folder if it doesn't already exist
#total=0; headerCount=0; sourceCount=0; documentCount=0; \
for file in $(HEADER_PATH)*; do \
if ls $$file[~] >/dev/null 2>&1; then \
mv -fu $$file[~] backup/$$file; \
let "headerCount+=1"; \
echo $(DATE)[Backed Up] $$file~ >> $(LOG); \
fi; \
done; \
for file in $(SOURCE_PATH)*; do \
if ls $$file[~] >/dev/null 2>&1; then \
mv -fu $$file[~] backup/$$file; \
let "sourceCount+=1"; \
echo $(DATE)[Backed Up] $$file~ >> $(LOG); \
fi; \
done; \
for file in $(DOC_PATH)*; do \
if ls $$file[~] >/dev/null 2>&1; then \
mv -fu $$file[~] backup/$$file; \
let "documentCount+=1"; \
echo $(DATE)[Backed Up] $$file~ >> $(LOG); \
fi; \
done; \
let "total= headerCount + sourceCount + documentCount"; \
echo -n $(OUTPUT_PROMPT)" "; \
if test $$total -eq 0; then \
echo Nothing To Back up; \
else \
if test $$headerCount -eq $$total; then \
echo -n $$total" "; \
echo -n "Header"; \
if test $$total -ge 2; then \
echo -n "s"; \
fi; \
echo " Backed Up"; \
elif test $$sourceCount -eq $$total; then \
echo -n $$total" "; \
echo -n "Source"; \
if test $$total -ge 2; then \
echo -n "s"; \
fi; \
echo " Backed Up"; \
elif test $$documentCount -eq $$total; then \
echo -n $$total" "; \
echo -n "Document"; \
if test $$total -ge 2; then \
echo -n "s"; \
fi; \
echo " Backed Up"; \
else \
$(UNDERLINE); echo $$total " Files Backed Up"; $(UNUNDERLINE); \
if test $$headerCount -eq 1; then \
echo $(OUTPUT_PROMPT)" "$$headerCount header; \
elif test $$headerCount -ge 2; then \
echo $(OUTPUT_PROMPT)" "$$headerCount headers; \
fi; \
if test $$sourceCount -eq 1; then \
echo $(OUTPUT_PROMPT)" "$$sourceCount source; \
elif test $$sourceCount -ge 2; then \
echo $(OUTPUT_PROMPT)" "$$sourceCount sources; \
fi; \
if test $$documentCount -eq 1; then \
echo $(OUTPUT_PROMPT)" "$$documentCount document; \
elif test $$documentCount -ge 2; then \
echo $(OUTPUT_PROMPT)" "$$documentCount documents; \
fi; \
fi; \
fi;
#$(DEFAULT_TEXT)
what the code does is not important, but I wanted to illustrate that it has macros in which 'make' must expand, and that it also performs shell code (bash), and that some indication on what the script did, is displayed in the terminal.
I want to put this script outside of 'make' in another directory, and turn that code into something like this:
.PHONY: backup
backup:
#run scripts/backup.scr
#or something similar to that
How can I put the rule of my target (which is makefile/bash code) into a separate file, and have make practically paste it in so that it runs how I had it originally?
I thought I might be able to use the "include" command inside 'make'.
It looks like it is used to include other makefiles though..
maybe I should just paste the entire target/rule into another makefile, and then include that makefile into my main one?
Would that be the best way?
In your case, you have quite few output variables. It might be worth the hassle to separate the generation and execution, like:
clean : clean-script
sh clean-script
rm -f clean-script
clean-script : clean-script.in
sed -e 's:[#]HEADER_PATH[#]:$(HEADER_PATH):g' $<.in > $#
And write clean-script.in as a clean sh script with a few substitutions.
If you use GNU make, you can of course build a list of output varables like:
clean-script : clean-script.in
sed $(foreach var,$(SUBSTVARS),-e 's:[#]$(var)[#]:$($(var)):g') $<.in > $#
I don't know if it can help you, but you can run make inside some Makefile usually with a command (inside a rule) e.g.
$(MAKE) subtarget
See the section Recursive use of Make in the GNU make documentation.
I tend to dislike using make for complex projects (but unfortunately, I have to). If you are free to chose some other tool, you might consider omake and many others (cmake, scons, bake, ...)