Bash - assign variables to yad values - sed usage in for loop - bash

In the code below I am attempting to assign variables to the two yad values Radius and Amount.
This can be done with awk by printing the yad values to file but I want to avoid this if I can.
The string (that is, both yad values) is assigned a variable and trimmed of characters, as required, using sed. However, the script stops at this line;
radius=$(sed 's|[amount*,]||g')
Two questions
is there a better way of tackling this; and
why is the script not completing? I have not been able to figure out the syntax.
EDIT: don't need the loop and working on the sed syntax
#!/bin/bash
#ifs.sh
values=`yad --form --center --width=300 --title="Test" --separator=' ' \
--button=Skip:1 \
--button=Apply:0 \
--field="Radius":NUM \
'0!0..30!1!0' \
--field="Amount":NUM \
'0!0..5!0.01!2'`
radius=$(echo "$values" | sed 's|[amount*,]||g')
amount=$(echo "$values" | sed 's/.a://')
if [ $? = 1 ]; then
echo " " >/dev/null 2>&1; else
echo "Radius = $radius"
echo "Amount = $amount"
fi
exit
Alternatives
# with separator
# radius="${values%????????}"
# amount="${values#????????}"
# without separator
# radius=$(echo "$values" | sed s'/........$//')
# amount=$(echo "$values" | sed 's/^........//')

It's easier than you think:
$ values=( $(echo '7.000000 0.100000 ') )
$ echo "${values[0]}"
7.000000
$ echo "${values[1]}"
0.100000
Replace $(echo '7.000000 0.100000 ') with yad ... so the script would be:
values=( $(yad --form --center --width=300 --title="Test" --separator=' ' \
--button=Skip:1 \
--button=Apply:0 \
--field="Radius":NUM \
'0!0..30!1!0' \
--field="Amount":NUM \
'0!0..5!0.01!2') )
if [ $? -eq 0 ]; then
echo "Radius = ${values[0]}"
echo "Amount = ${values[1]}"
fi

EDIT: Changed answer based on #Ed Morton
#!/bin/bash
#ifs.sh
values=($(yad --form --center --width=300 --title="Test" --separator=' ' \
--button=Skip:1 \
--button=Apply:0 \
--field="Radius":NUM \
'0!0..30!1!0' \
--field="Amount":NUM \
'0!0..5!0.01!2'))
if [ $? -eq 0 ]; then
radius="${values[0]}"
amount="${values[1]}"
fi
exit
bash -x Output
+ '[' 0 -eq 0 ']'
+ radius=7.000000
+ amount=1.000000
+ exit

Related

Makefile cache creating false positive outcome

I have a make target, which i usually need to run twice to get accurate outcome. I.e the 1st run if accurate thenn on the 2nd run, if the variable is changed, it still displays the previous output, which is wrong, is there a way to get rid of cache or clear it in between.
.PHONY:check-tf-lint
check-tf-lint: configure ## TF Linting
$(eval list_of_dir := $(shell cd ${deployment} && ls -ld */ | awk '{print $$NF}'| grep -v 'test_cases'| sed 's|/||g'))
$(shell touch ${quality-metrics}/formatting.txt)
#for i in aws_bot; do \
make set-tf-version -e infra_module_path=$$i; \
terraform fmt -check -list=false ${deployment}/$$i ; \
if [ "$$?" != "0" ]; then \
echo "Need Formatting in $$i" >> ${quality-metrics}/formatting.txt; \
terraform fmt -check ${deployment}/$$i >> ${quality-metrics}/formatting.txt; \
echo "" >> ${quality-metrics}/formatting.txt; \
fi \
done
$(eval TMP := $(shell (cat ${quality-metrics}/formatting.txt | wc -l)))
echo "${TMP}"
#if [ "$(TMP)" = "0" ]; then \
echo "All Good! No Formatting Needed."; \
else \
echo "Kindly Format Below Mentioned code and check in Again"; \
cat ${quality-metrics}/formatting.txt; \
fi
$(shell rm -rf ${quality-metrics}/formatting.txt)
#if [ "$(TMP)" != "0" ]; then \
exit 1; \
fi
Rule of thumb: you should never use eval or shell functions in a make recipe. If you are doing that it's a pretty sure sign that something has gone wrong somewhere.
In your case the reason you see this behavior is that make will expand ALL variables and functions for all lines in a recipe before the first line in the recipe is invoked. So as far as make is concerned your recipe is handled like this:
.PHONY:check-tf-lint
check-tf-lint: configure ## TF Linting
$(eval list_of_dir := $(shell cd ${deployment} && ls -ld */ | awk '{print $$NF}'| grep -v 'test_cases'| sed 's|/||g'))
$(shell touch ${quality-metrics}/formatting.txt)
$(eval TMP := $(shell (cat ${quality-metrics}/formatting.txt | wc -l)))
$(shell rm -rf ${quality-metrics}/formatting.txt)
#for i in aws_bot; do \
make set-tf-version -e infra_module_path=$$i; \
terraform fmt -check -list=false ${deployment}/$$i ; \
if [ "$$?" != "0" ]; then \
echo "Need Formatting in $$i" >> ${quality-metrics}/formatting.txt; \
terraform fmt -check ${deployment}/$$i >> ${quality-metrics}/formatting.txt; \
echo "" >> ${quality-metrics}/formatting.txt; \
fi \
done
echo "${TMP}"
#if [ "$(TMP)" = "0" ]; then \
echo "All Good! No Formatting Needed."; \
else \
echo "Kindly Format Below Mentioned code and check in Again"; \
cat ${quality-metrics}/formatting.txt; \
fi
#if [ "$(TMP)" != "0" ]; then \
exit 1; \
fi
You should always write your recipes using shell facilities and not make facilities. Set shell variables, don't use eval to set make variables, and run shell commands directly (you're in a recipe after all!) rather than using make's shell function.
You may need to put all the lines in a single script (with semicolon / backslash) to allow this to work. Or consider .ONESHELL but that's a much bigger set of changes.
This worked for me !
.PHONY:check-tf-lint
check-tf-lint: ## TF Linting
$(eval list_of_dir := $(shell cd ${deployment} && ls -ld */ | awk '{print $$NF}'| grep -v 'test_cases'| sed 's|/||g'))
#for i in $(list_of_dir); do \
make set-tf-version -e infra_module_path=$$i; \
terraform fmt -check -list=false ${deployment}/$$i ; \
if [ "$$?" != "0" ]; then \
echo "Need Formatting in $$i" >> ${quality-metrics}/formatting.txt; \
terraform fmt -check ${deployment}/$$i >> ${quality-metrics}/formatting.txt; \
echo "" >> ${quality-metrics}/formatting.txt; \
fi \
done
#if [ -e "${quality-metrics}/formatting.txt" ]; then \
export MNC=`cat ${quality-metrics}/formatting.txt | wc -l`; \
if [ "$${MNC}" = "0" ]; then \
echo "All Good! No Formatting Needed."; \
else \
echo ""; \
echo "Kindly Format Below Mentioned code and check in Again"; \
cat ${quality-metrics}/formatting.txt; \
fi; \
rm -rf ${quality-metrics}/formatting.txt; \
if [ "$${MNC}" != "0" ]; then \
exit 1; \
fi \
fi

Makefile: Splitting a string and looping through results

I am trying to write a target in a makefile which will read a variable(having IPs) from one of the .mk file and if a space separated list found split it and take some action.
Issue i am facing that the string do not split and in for loop do not get the value either.
Have tried following
GWTS_FE_IPS=2600:40f0:3e::2 2600:40f0:3e::3 2600:40f0:3e::4 2600:40f0:3e::5
test:
$(eval IPS=$(shell echo "$(GW_IPS)" |awk -F " " '{print NF}'))
if [ ${IPS} -gt 1 ]; then \
echo "Multiple Ips [$(GW_IPS)]"; \
for ip in $(shell echo "${GW_IPS}" | sed -e 's/ /\n/g'); \
do \
echo ".... $(ip) ...."; \
done \
else \
echo "Single IP [$(GW_IPS)]"; \
fi
Result i get is
2600:40f0:3e::2n2600:40f0:3e::3n2600:40f0:3e::4n2600:40f0:3e::5
if [ 4 -gt 1 ]; then \
echo "Multiple Ips [2600:40f0:3e::2 2600:40f0:3e::3 2600:40f0:3e::4 2600:40f0:3e::5]"; \
for ip in 2600:40f0:3e::2n2600:40f0:3e::3n2600:40f0:3e::4n2600:40f0:3e::5; \
do \
echo ".... ...."; \
done \
else \
echo "Single IP [2600:40f0:3e::2 2600:40f0:3e::3 2600:40f0:3e::4 2600:40f0:3e::5]"; \
fi
Multiple Ips [2600:40f0:3e::2 2600:40f0:3e::3 2600:40f0:3e::4 2600:40f0:3e::5]
.... ....
Can any one give some pointers.
You are trying to do too many things at once without testing any of them. When you try new tools, try them one at a time.
GWTS_FE_IPS=2600:40f0:3e::2 2600:40f0:3e::3 2600:40f0:3e::4 2600:40f0:3e::5
IPS := $(words $(GWTS_FE_IPS))
test:
#if [ ${IPS} -gt 1 ]; then \
echo "Multiple Ips [$(GW_IPS)]"; \
for ip in $(GWTS_FE_IPS) ; \
do \
echo ".... $$ip ...."; \
done \
else \
echo "Single IP [$(GW_IPS)]"; \
fi

How can I get arguments containing spaces from file?

If I run:
s.sh A B=" " C
s.sh
#!/bin/bash
count() {
echo $#
echo "$1" "$2" "$3"
}
count "$#"
I get following output:
3
A B= C
which is the expected one. Good!
But if I run:
s2.sh -f file.txt
s2.sh
#!/bin/bash
count() {
echo $#
echo "$1" "$2" "$3"
}
while read line || [[ -n "$line" ]]
do
echo -e "\nLine: $line"
count $line
done < $2
file.txt
A B=" " C
I don't get a similar output. It is:
Line: A B=" " C
4
A B=" "
Actual: 4 arguments
Expected: 3 arguments (the same as the ones mentioned in the example with the arguments provided through the command line)
I've also tried to use the following lines in file.txt:
A B=' ' C
A B=\" \" C
A B= C
A B=\ \ \ \ \ \ \ C
and to use count "$line", but I couldn't achieve the expected behavior.
How can I read those arguments from file?
To process quotes and other metacharacters in the input line, you need to use eval:
eval "count $line"

store txt files separately for each subcategories

I have several experiments. Each experiment has several replicate files. I want to place all these replicate files into one text file in the following way.
Lets say there are 3 experiments and each experiment has 2 replicate files.(Experiment and replicate number can be more than this)
/home/data/study1/EXP1_30/EXP1_replicate_1_30.txt
/home/data/study1/EXP1_30/EXP1_replicate_2_30.txt
/home/data/study1/EXP1_60/EXP1_replicate_1_60.txt
/home/data/study1/EXP1_60/EXP1_replicate_2_60.txt
/home/data/study1/EXP2_30/EXP2_replicate_1_30.txt
/home/data/study1/EXP2_30/EXP2_replicate_2_30.txt
/home/data/study1/EXP2_60/EXP2_replicate_1_60.txt
/home/data/study1/EXP2_60/EXP2_replicate_2_60.txt
/home/data/study1/EXP3_30/EXP3_replicate_1_30.txt
/home/data/study1/EXP3_30/EXP3_replicate_2_30.txt
/home/data/study1/EXP3_60/EXP3_replicate_1_60.txt
/home/data/study1/EXP3_60/EXP3_replicate_2_60.txt
output file1.txt will look like
/home/data/study1/EXP1/EXP1_replicate_1_30.txt,/home/data/study1/EXP1/EXP1_replicate_2_30.txt \
/home/data/study1/EXP2/EXP2_replicate_1_30.txt,/home/data/study1/EXP2/EXP2_replicate_2_30.txt \
/home/data/study1/EXP3/EXP3_replicate_1_30.txt,/home/data/study1/EXP3/EXP3_replicate_2_30.txt
output file2.txt will look like
/home/data/study1/EXP1/EXP1_replicate_1_60.txt,/home/data/study/EXP1/EXP1_replicate_2_60.txt \
/home/data/study1/EXP2/EXP2_replicate_1_60.txt,/home/data/study1/EXP2/EXP2_replicate_2_60.txt \
/home/data/study1/EXP3/EXP3_replicate_1_60.txt,/home/data/study1/EXP3/EXP3_replicate_2_60.txt
....
My code with for loops:
ID=(30 60)
exp=("EXP1" "EXP2" "EXP3")
d=""
for txtfile in /home/data/study1/${exp[0]}/${exp[0]}*_${ID[0]}.txt
do
printf "%s%s" "$d" "$txtfile"
d=","
done
printf " \\"
printf "\n"
d=""
for txtfile in /home/data/study1/${exp[1]}/${exp[1]}*_${ID[0]}.txt
do
printf "%s%s" "$d" "$txtfile"
d=","
done
printf " \\"
printf "\n"
d=""
for txtfile in /home/data/study1/${exp[2]}/${exp[2]}*_${ID[0]}.txt
do
printf "%s%s" "$d" "$txtfile"
d=","
done
I am using for loops with index numbers for each experiment and replicates which is very time consuming. Is there any easy way?
I think that this does what you want:
#!/bin/bash
ids=( 30 60 )
dir=/home/data/study1
# join glob on comma, add slash at end
# modified from http://stackoverflow.com/a/3436177/2088135
join() { local IFS=,; echo "$* "'\'; } #' <- to fix syntax highlighting
i=0
for id in "${ids[#]}"; do
s=$(for exp in "$dir"/EXP*"$id"; do join "$exp/"*"$id".txt; done)
# trim off final slash and output to file
echo "${s%?}" > file$((++i)).txt
done
Output (note that when testing, I set dir=.):
$ cat file1.txt
./EXP1_30/EXP1_replicate_1_30.txt,./EXP1_30/EXP1_replicate_2_30.txt \
./EXP2_30/EXP2_replicate_1_30.txt,./EXP2_30/EXP2_replicate_2_30.txt \
./EXP3_30/EXP3_replicate_1_30.txt,./EXP3_30/EXP3_replicate_2_30.txt
$ cat file2.txt
./EXP1_60/EXP1_replicate_1_60.txt,./EXP1_60/EXP1_replicate_2_60.txt \
./EXP2_60/EXP2_replicate_1_60.txt,./EXP2_60/EXP2_replicate_2_60.txt \
./EXP3_60/EXP3_replicate_1_60.txt,./EXP3_60/EXP3_replicate_2_60.txt
You can use the following bash script:
#!/bin/bash
i=0; n=0; files=""
sort -t_ -k5 files.txt | while read line ; do
files="$files $line"
i=$((i+1))
if [ $((i%6)) -eq 0 ] ; then
n=$((n+1))
cat $files > "$n.txt"
files=""
fi
done
You can also make use of a subshell and do it from the command line (your data in dat/experiment.txt) with:
$ ( first=0; cnt=0; grep 30 dat/experiment.txt | sort | while read line; do \
[ "$first" = 0 ] && first=1 || { [ "$cnt" = 0 ] && echo ' \'; }; echo -n $line; \
((cnt++)); [ "$cnt" = 1 ] && echo -n ","; [ "$cnt" = 2 ] && cnt=0; done; \
echo "" ) >outfile1.txt
$ ( first=0; cnt=0; grep 60 dat/experiment.txt | sort | while read line; do \
[ "$first" = 0 ] && first=1 || { [ "$cnt" = 0 ] && echo ' \'; }; echo -n $line; \
((cnt++)); [ "$cnt" = 1 ] && echo -n ","; [ "$cnt" = 2 ] && cnt=0; done; \
echo "" ) >outfile2.txt
Admittedly, the one liner ended up being longer than originally anticipated to match your line continuations -- exactly. If you omit the line continuations in the outfiles, the line reduces to (e.g.):
$ (cnt=0; grep 30 dat/experiment.txt | sort | while read line; do echo -n $line; \
((cnt++)); [ "$cnt" = 1 ] && echo -n ","; [ "$cnt" = 2 ] && echo "" && cnt=0; \
done ) >outfile1.txt
output:
$ cat outfile1.txt
/home/data/study1/EXP1_30/EXP1_replicate_1_30.txt,/home/data/study1/EXP1_30/EXP1_replicate_2_30.txt \
/home/data/study1/EXP2_30/EXP2_replicate_1_30.txt,/home/data/study1/EXP2_30/EXP2_replicate_2_30.txt \
/home/data/study1/EXP3_30/EXP3_replicate_1_30.txt,/home/data/study1/EXP3_30/EXP3_replicate_2_30.txt \
$ cat outfile2.txt
/home/data/study1/EXP1_60/EXP1_replicate_1_60.txt,/home/data/study1/EXP1_60/EXP1_replicate_2_60.txt \
/home/data/study1/EXP2_60/EXP2_replicate_1_60.txt,/home/data/study1/EXP2_60/EXP2_replicate_2_60.txt \
/home/data/study1/EXP3_60/EXP3_replicate_1_60.txt,/home/data/study1/EXP3_60/EXP3_replicate_2_60.txt \

How to replace or escape <tab> characters with \t in bash script and being able to output single quotes?

In the goal to create a file from a one line (bash) command, the goal is to output the contents of any text file - in this example a bash script - and wrap each line inside a command that is able to output that same line when pasted in a Terminal window.
Example source input file:
Line 1
Line 2
Line 3
Example desired output:
echo 'Line 1';echo 'Line 2';echo 'Line 3';
Note: whether printf, echo or another command is used to create the output, doesn't matter as long as the source is human readable.
One hurdle were the single quotes, that would not be recreated. Therefore use the form $'string', which are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard.
Another requirement is to re-create tab characters from the old file in the new file. Therefore the wish is to replace <\tab> characters with \t.
Our tries to do this with sed or tr fail. How to replace tabs with their escape \t counterpart and still being able to output lines with original quotes?
Input file /Library/Scripts/BootRepairMount.sh contains:
$ cat /Library/Scripts/BootRepairMount.sh
#!/bin/bash
sleep 18
for OUTPUT in $(diskutil list | grep ': Apple_HFS' | awk '{ print $NF }')
do
if [[ -z $(df -lnh | grep /dev/$OUTPUT) ]]; then
echo "$OUTPUT is not mounted, repair and mount"
diskutil repairVolume $OUTPUT
diskutil mount $OUTPUT
fi
done
The best shell one line command we could create is:
$ oldifs=$IFS;printf '\n';printf '{';while IFS= read -r p;do [[ "$p" == *"'"* ]] && echo -n "echo $'$p';" || echo -n "echo '$p';"; done < /Library/Scripts/BootRepairMount.sh | tr '\t' '\134\164';printf '}';printf '\n\n';IFS=$oldifs
Which returns this faulty output:
{echo '#!/bin/bash';echo 'sleep 18';echo $'for OUTPUT in $(diskutil list | grep ': Apple_HFS' | awk '{ print $NF }')';echo 'do';echo '\if [[ -z $(df -lnh | grep /dev/$OUTPUT) ]]; then';echo '\\echo "$OUTPUT is not mounted, repair and mount"';echo '\\diskutil repairVolume $OUTPUT';echo '\\diskutil mount $OUTPUT';echo '\fi';echo 'done';}
Desired output is:
{echo '#!/bin/bash';echo 'sleep 18';echo $'for OUTPUT in $(diskutil list | grep ': Apple_HFS' | awk '{ print $NF }')';echo 'do';echo '\tif [[ -z $(df -lnh | grep /dev/$OUTPUT) ]]; then';echo '\t\techo "$OUTPUT is not mounted, repair and mount"';echo '\t\tdiskutil repairVolume $OUTPUT';echo '\t\tdiskutil mount $OUTPUT';echo '\tfi';echo 'done';}
Bash one line command version 2
$ oldifs=$IFS;printf '\n';printf '{';while IFS= read -r p;do [[ "$p" == *"'"* ]] && printf 'printf $'\''%q'\'';' "$p" || printf 'printf '\''%q'\'';' "$p"; done < /Library/Scripts/BootRepairMount.sh;printf '}';printf '\n\n';IFS=$oldifs
returns output that is heavy escaped:
{printf '\#\!/bin/bash';printf 'sleep\ 18';printf $'for\ OUTPUT\ in\ \$\(diskutil\ list\ \|\ grep\ \':\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Apple_HFS\'\ \|\ awk\ \'\{\ print\ \$NF\ \}\'\)';printf 'do';printf '$'\tif [[ -z $(df -lnh | grep /dev/$OUTPUT) ]]; then'';printf '$'\t\techo "$OUTPUT is not mounted, repair and mount"'';printf '$'\t\tdiskutil repairVolume $OUTPUT'';printf '$'\t\tdiskutil mount $OUTPUT'';printf '$'\tfi'';printf 'done';}
that never gets unescaped back to its original values in Mac OS X 10.7.5.
printf '\#\!/bin/bash';
outputs:
\#\!/bin/bash
As well as:
echo -e '\#\!/bin/bash'
does output the unescaped value
\#\!/bin/bash
-e is not a valid command switch for the Mac OS X 10.7.5 echo command, according to its man page.
bash's builtin command printf has %q format code that handles this:
printf '\n{ '; while IFS= read -r p; do printf "echo %q; " "$p"; done < /Library/Scripts/BootRepairMount.sh; printf '}\n\n'
Unfortunately, it doesn't always choose quoting/escaping modes that're easy to read. Specifically, it tends to prefer escaping individual metacharacters (e.g. spaces) rather than enclosing them in quotes:
{ echo \#\!/bin/bash; echo sleep\ 18; echo for\ OUTPUT\ in\ \$(diskutil\ list\ \|\ grep\ \':\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Apple_HFS\'\ \|\ awk\ \'{\ print\ \$NF\ }\'); echo do; echo $'\tif [[ -z $(df -lnh | grep /dev/$OUTPUT) ]]; then'; echo $'\t\techo "$OUTPUT is not mounted, repair and mount"'; echo $'\t\tdiskutil repairVolume $OUTPUT'; echo $'\t\tdiskutil mount $OUTPUT'; echo $'\tfi'; echo done; }
If I understand right you want paste one long line to the Terminal.app and want get the "source code" of original script. So, need a script what will generate the one-line script.
Maybe a bit unusual solution, but it is easy and simple.
here is the test script called test.sh (instead of your BootReapirMount.sh)
for i in {1..10}
do
date
done
Here is the generator script mkecho.sh
#!/bin/bash
[[ ! -f "$1" ]] && echo "Need filename" && exit 1
asc=$(gzip < "$1" | base64)
echo "base64 -D <<<'$asc'| gzip -d"
Now, run:
bash mkecho.sh test.sh
you will get the next:
base64 -D <<<'H4sIAASwqFEAA0vLL1LIVMjMU6g21NMzNKjlSsnn4kxJLEkFMvJSuQBZFmY0HwAAAA=='| gzip -d
If you copy and paste the above into the terminal, it will will display the original test.sh
Variant2
If you want directly execute the script, you should modify the mkecho.sh to the next mkeval.sh
#!/bin/bash
[[ ! -f "$1" ]] && echo "Need filename" && exit 1
asc=$(gzip < "$1" | base64)
echo -n 'eval "$(base64 -D <<<"'
echo -n $asc
echo -n '" | gzip -d)"'
echo
When run
bash mkeval.sh test.sh
will get
eval "$(base64 -D <<<"H4sIAASwqFEAA0vLL1LIVMjMU6g21NMzNKjlSsnn4kxJLEkFMvJSuQBZFmY0HwAAAA==" | gzip -d)"
and finally when you copy and paste it into the terminal, you run the test.sh and will get:
Fri May 31 16:25:08 CEST 2013
... 8 lined deleted...
Fri May 31 16:25:08 CEST 2013
Warning: because the script is NOT TESTED for every possible conditions, nor for redirects and so on - I really don't recommending using the eval verision.
sed 's/\\t/\\/g'
$ echo 'ffsd \tif [[ -z $' | sed 's/\\t/\\/g'
ffsd \if [[ -z $

Resources