What is wrong with these lines?
FASTAQ1 = "/home/mdb1c20/my_onw_NGS_pipeline/files/fastq/W2115220_S5_L001_R1_001.fastq"
FASTAQ2 = "/home/mdb1c20/my_onw_NGS_pipeline/files/fastq/W2115220_S5_L001_R2_001.fastq"
DIRECTORY = "/home/mdb1c20/my_onw_NGS_pipeline/scripts_my_first_NGS"
They are in a .conf file with other similar variables. The only difference is that these three are created with printf
printf 'FASTAQ1 = "%s"\n' "$FASTA1" >> "$DIRECTORY/$filename1/scripts/shortcut.config"
printf 'FASTAQ2 = "%s"\n' "$FASTA2" >> "$DIRECTORY/$filename1/scripts/shortcut.config"
printf 'DIRECTORY = "%s"\n' "$DIRECTORY" >> "$DIRECTORY/$filename1/scripts/shortcut.config"
When a script I am using open the .confi file its says that FASTAQ1: command not found
Apart from these three, the rest of variables were created manually in a archive .conf file but the script add these three on the go. The only thing I haven't tried because I don't know how to do that is to remove the white spaces before and after the equal simbol?
In bash, this:
var = value
is not the same as this:
var=value
The first example runs a command named "var" and passes it two arguments "=" and "value".
The second example sets a variable called "var" to "value".
It was hard to find this detail in the Bash manual, but the difference is between simple commands and assigning to variables, or shell parameters.
If you intended to source your configuration file, you should have used printf this way:
printf 'FASTAQ1=%q\n' "$FASTA1" >> "$DIRECTORY/$filename1/scripts/shortcut.config"
This allows you to store the value safely regardless if it has spaces or quotes.
The error was caused by the assignment command being interpretted as a simple command instead because of the spaces around the equal sign.
Alternatively for Bash 4.4+, you can use #Q expansion:
echo "FASTAQ1=${FASTA1#Q}" >> "$DIRECTORY/$filename1/scripts/shortcut.config"
Related
I am using tcsh (contract required, cannot change to bash etc), but am having a problem building up a command based on various conditions for different pieces.
Some names changed to protect the innocent...
If new or old program name, is really chosen earlier on by a preprocessor, and is hardcoded by the time this shell script gets run:
set myCMDline = newProgName
set myCMDlineTmpFile = "/tmp/myCMDlineTmpScriptFile.csh"
set bsubQname = "typical"
set bsubResources = "span[hosts=1]"
set myCMDline = "bsub -q $bsubQname -n 8 -R \"$bsubResources\" $myCMDline"
($myCMDline)
Now, I have tried several variations of the above, all not working for some reason or another. The closest I think I get is a complaint about mismatched double-quotes, even when backspacing them.
When I do an echo of $myCMDline, then that looks OK, but the execution of same must somehow be different...
set bsubResources = '"span[hosts=1]"' #double-quotes inside, single-quotes outside
set myCMDline = "bsub -q $bsubQname -n 8 -R $bsubResources $myCMDline"
.
set bsubResources = "span[hosts=1]" #double-quotes inside, single-quotes outside
set myCMDline = 'bsub -q $bsubQname -n 8 -R "$bsubResources" $myCMDline'
.
set bsubResources = "span[hosts=1]" #double-quotes inside, single-quotes outside
set myCMDline = "bsub -q $bsubQname -n 8 -R '$bsubResources' $myCMDline"
etc.
I have also tried dumping to a separate temp script file to source, but that contains the $variable names, not resolved equivalents as I would prefer, as I am doing set, not setenv, and prefer not to put these into shell vars.
First I could not echo the "#!/bin/csh -f" line, it seems to try and execute that rather than echo redirected into the temp script file, and dies.
rm -f $myCMDlineTmpFile
echo "#!/bin/csh -f > $myCMDlineTmpFile
echo "$myCMDline" >> $myCMDlineTmpFile
($myCMDlineTmpFile)
Then I tried multi-line echo, which is where I am seeing the local variable names go into the file rather than their contents:
/bin/cat > $myCMDlineTmpFile <<EOF
#!/bin/csh -f
$myCMDline
EOF
source $myCMDlineTmpFile
And then I am trying to instead use eval:
eval `echo "$myCMDline &" `
with and without the backticks etc, but complains about unknown variables for the queue name, resources etc.
Adding this echo always looks like what I want to be the commandline, between the >>> and <<<
echo "DEBUG - myCMDline= >>>$myCMDline<<<"
Please help me solve this puzzle...
set myCMDline = "bsub -q $bsubQname -n 8 -R \"$bsubResources\" $myCMDline"
($myCMDline)
This won't work because csh considers this as a single string, so it treats the whole string as one big program name. You have to define an array instead:
set myCMDline = (bsub -q $bsubQname -n 8 -R "$bsubResources" $myCMDline:gaq)
($myCMDline:gaq)
Explanation: The :gaq is a substitution quotes all strings in the list and keeps each list element intact. This is quite similar to "$#" in bash.
This is documented in History Substitution
g Apply the following modifier once to each word.
a (+) Apply the following modifier as many times as possible to a single word. `a' and `g' can be used together to apply a modifier globally. In the current implementation, using the `a' and `s' modifiers together can lead to an infinite loop. For example, `:as/f/ff/' will never terminate. This behavior might change in the future.
q Quote the substituted words, preventing further substitutions.
This is relevant due to the text in variable substitution:
The `:' modifiers described under History substitution, except for `:p', can be applied to the substitutions above. More than one may be used. (+) Braces may be needed to insulate a variable substitution from a literal colon just as with History substitution (q.v.); any modifiers must appear within the braces.
I have some variables in a bash script. The variables comes from an other script.
I have an other variable containing a string some $names within.
How can I replace all of the variables-like names with the real values of the variables in this string? (I don't know all of the possible variables names)
Example:
#From an other place, they are imported by source <( ....)
abc="FOO"
rst="BAR"
format_base='xyz $abc $rst'
# How To Process formate_base?
????
#The Expected Result
echo "$format_base_processed"
#Should Output: xyz FOO BAR
In your example it would be sufficient to use double quotes instead of single quotes. If the variables exist at the time of reading a double quoted string, there is no problem:
abc="FOO"
rst="BAR"
format_base="xyz $abc $rst"
If (for some very complicated reason) you have to specify format_base before the used variables, you can use things like eval or bash -c:
format_base='xyz $abc $rst'
# "load" variables $abc and $rst
format_base_processed=$(eval echo "\"$format_base\"")
Note that the above command has security issues, especially if $format_base is not specified by you. Example:
format_base='$(injectedCommand)'
format_base_processed=$(eval echo "\"$format_base\"")
Here we will execute echo "$(injectedCommand)" where injectedCommand could be anything, for example rm -rf * (do not try this at home).
I am trying to extract the number of lines from a file, and then use it in a variable. However, it keeps passing the file name, not the number of lines. I read through this question, but the examples are not working.
for i in $BASE/$TEMPLATE_DIR-$i$PRUNING_METHOD/*.txt
do
NUM_LINES=$(wc -l < $i)
echo $NUM_LINES
UPLOAD_CMD="sh sshpass_setup_verification.sh $EXP_ID-$i$PRUNING_METHOD__$NUM_LINES__features";
echo "$UPLOAD_CMD"
break 1;
done
Prints:
15 #the correct number of lines
sh sshpass_setup_verification.sh di-60sec-max/TemplateUser1.txt #where TemplateUser1.txt is the name of the file
Should print:
15
sh sshpass_setup_verification.sh di-60sec-max__15__features
A summary of what people are telling you in the comments:
for i in "${BASE}/${TEMPLATE_DIR}-${i}${PRUNING_METHOD}"/*.txt
do
num_lines=$(wc -l < "$i")
echo "$num_lines"
upload_cmd="sh sshpass_setup_verification.sh ${EXP_ID}-${i}${PRUNING_METHOD}__${num_lines}__features"
echo "$upload_cmd"
break
done
The key thing here is using double quotes around your parameter expansions and curly braces to disambiguate in situations where characters such as _ could be interpreted as part of a variable name but shouldn't. I've added them in several places where they aren't strictly needed but they do no harm.
I've also changed your variable names to lowercase. This will help you one day when you decide to have a variable called PATH and suddenly all your commands stop working.
I have a directory config with the following file listing:
$ ls config
file one
file two
file three
I want a bash script that will, when given no arguments, iterate over all those files; when given names of files as arguments, I want it to iterate over the named files.
#!/bin/sh
for file in ${#:-config/*}
do
echo "Processing '$file'"
done
As above, with no quotes around the list term in the for loop, it produces the expected output in the no-argument case, but breaks when you pass an argument (it splits the file names on spaces.) Quoting the list term (for file in "${#:-config/*}") works when I pass file names, but fails to expand the glob if I don't.
Is there a way to get both cases to work?
For a simpler solution, just modify your IFS variable
#!/bin/bash
IFS=''
for file in ${#:-config/*}
do
echo "Processing '$file'"
done
IFS=$' \n\t'
The $IFS is a default shell variable that lists all the separators used by the shell. If you remove the space from this list, the shell won't split on space anymore. You should set it back to its default value after you function so that it doesn't cause other functions to misbehave later in your script
NOTE: This seems to misbehave with dash (I used a debian, and #!/bin/sh links to dash). If you use an empty $IFS, args passed will be returned as only 1 file. However, if you put some random value (i.e. IFS=':'), the behaviour will be the one you wanted (except if there is a : in your files name)
This works fine with #!/bin/bash, though
Set the positional parameters explicitly if none are given; then the for loop is the same for both cases:
[ $# -eq 0 ] && set -- config/*
for file in "$#"; do
echo "Processing '$file'"
done
Put the processing code in a function, and then use different loops to call it:
if [ $# -eq 0 ]
then for file in config/*
do processing_func "$file"
done
else for file in "$#"
do processing_func "$file"
done
fi
So I have .csh script generate.csh
I want to call another .csh script from within it.
I tried
./shell_gen.csh test1 test2.prt
But it runs shell_gen.csh but without command line arguments.
Anyone?
Thanks
generate.csh
#!/bin/csh
./shell_gen.csh test1 test2.prt
shell_gen.csh
#!/bin/csh
set source = output
############################################################
# create pbm-gifs/ directory
############################################################
set destination1 = /scratch/graphics/$1gif
echo making $destination1
if ( ! -e $destination1 ) then
mkdir $destination1
foreach file ( $source/*.pgm )
echo convert $file $destination1/$file:t:r.gif
convert $file $destination1/$file:t:r.gif
end
else
echo "$destination1/ exists"
endif
I would say you probably need to put curly braces around the positional parameter variable when they are concatenated against some other characters:
set destination1 = /scratch/graphics/${1}gif
But you might want a dot in there, too:
set destination1 = "/scratch/graphics/${1}.gif"
and quotes protect against spaces.
$1gif isn't a reference to a positional parameter.
Did you try ${1}gif as Dennis suggested? I think that should work.
ALSO I see no reference to $2 anywhere though your generate clearly sends a second argument.
Lastly the first line should probably be #!/bin/csh -f
Best,
-pbr