I have this script, and i have multiple results.
How can the multiple results be output in one variable?
Example: results from first loop is English, second loop is Italian.
I need final Results in a Variable : English Italian
for i in $(ls -l $1/$2/);do
if [[ $i =~ .*\.idx$ ]];then
tr -d '\r' < $1/$2/$i > $1/$2/newfile
rm -f $1/$2/$i
mv $1/$2/newfile $1/$2/$i
results=$(cat $1/$2/$i |awk '/^# alt:/ { a[$3] } END { for (l in a) { printf("%s%s", c, l); c = " " } printf("\n") }')
echo "Results for $i : $results"
fi
done
Alternatively, just accumulate your results in another variable:
RESULTS=""
for i in $(ls -l $1/$2/);do
if [[ $i =~ .*\.idx$ ]];then
tr -d '\r' < $1/$2/$i > $1/$2/newfile
rm -f $1/$2/$i
mv $1/$2/newfile $1/$2/$i
results=$(cat $1/$2/$i |awk '/^# alt:/ { a[$3] } END { for (l in a) { printf("%s%s", c, l); c = " " } printf("\n") }')
RESULTS="$RESULTS$results "
fi
done
echo ${RESULTS%" "} #Get rid of the trailing space
You can get all results in a single variable if you use the following syntax:
variable=$(command)
You might find it useful to put your code in a function and then call it like this:
function command() {
# your code from above
}
results=$( command "$#" )
Related
I have a following varaible:
tags = {
environment = "development",
provider = "ServiceOne",
ansible_role = "nfs-role",
comment = "mysql"
}
In my pipeline i need to convert it to the following:
tfh pushvars -overwrite-all -dry-run false -hcl-var "tags={environment=\"development\", provider=\"ServiceOne\", ansible_role=\"nfs-rolep\",comment= \"mysql\"}"
I have tried with SED and AWK but couldn't get any result?
This is where i am standing now:
#!/bin/bash
#[[ -z "$2" ]] && echo "==> Usage: ./transform_tfe_vars.sh <<INPUT_FILE>> <<OUTPUT_FILE>>" && exit 1;
vars_file=${1}
#output_file=${2}
tmp_file=".todelete.tmp"
cmd "$vars_file" | grep -v '^#' | awk '!/^$/' > "$tmp_file"
while read -r p; do
a=$(echo "$p" | awk '{print $1}')
b=$(echo "$p" | awk '{print $3}')
echo "tfh pushvars -overwrite-all -dry-run false -shcl-var \"$a=\\""$b""\""
done <$tmp_file
A shell read loop is always the wrong approach for manipulating text, see why-is-using-a-shell-loop-to-process-text-considered-bad-practice. The guys who invented shell also invented awk for shell to call to manipulate text.
It looks like this might be what you're trying to do:
#!/usr/bin/env bash
(( $# == 2 )) || { echo "==> Usage: ${0##*/} <<INPUT_FILE>> <<OUTPUT_FILE>>"; exit 1; }
vars_file="$1"
output_file="$2"
awk '
BEGIN {
ORS = ""
print "tfh pushvars -overwrite-all -dry-run false -hcl-var \""
}
NF && !/^#/ {
gsub(/[[:space:]]/,"")
gsub(/"/,"\\\\&")
print
}
END {
print "\"\n"
}
' "$vars_file" > "$output_file"
For example, if the input is aabcca, the output needs to be a2b1c2a1 not a3b1c2
I originally wrote this -
echo "aabcca" > file.txt
a=0
b=0
c=0
while IFS= read -r -n1 char
do
[ "$char" == "a" ] && (( a++ ))
[ "$char" == "b" ] && (( b++ ))
[ "$char" == "c" ] && (( c++ ))
done < file.txt
echo "a${a}b${b}c${c}"
But this outputs a3b1c2. I want a2b1c2a1.
Using awk, you may do this:
awk '{
p=c=""
for (i=1; i<=length(); ++i) {
f=substr($0, i, 1)
if (p != "" && f != p) {
printf "%s", p c
c = 0
}
++c
p = f
}
print p c
}' file.txt
a2b1c2a1
How about:
#!/usr/bin/env bash
count=0
read -r -n1 prev_char < file.txt
while IFS= read -r -n1 char
do
if [ "$prev_char" != "$char" ]
then
printf "%c%d" "$prev_char" "$count"
count=0
fi
prev_char="$char"
count=$((count + 1))
done < file.txt
printf "\n"
Here's an one-liner way to do it:
tr '\n' ' ' < file.txt | fold -w1 | uniq -c | awk '$2!=""{printf "%s", $2 $1} END {printf "\n"}'
EDIT: Also if you want to get rid of punctuation characters, just add this to tr:
tr '\n[:punct:]' ' ' < file.txt | fold -w1 | uniq -c | awk '$2!=""{printf "%s", $2 $1} END {printf "\n"}'
I have a folder with the following files:
DA-001-car.jpg
DA-001-dog.jpg
DA-001-coffee.jpg
DA-002-house.jpg
DA-003-coffee.jpg
DA-003-cat.jpg
...
I want to generate this (CSV) output:
SKU, IMAGE
DA-001, "DA-001-car.jpg, DA-001-dog.jpg, DA-001-coffee.jpg"
DA-002, "DA-001-house.jpg"
DA-003, "DA-001-coffee.jpg, DA-001-cat.jpg"
I tried to program this in Bash:
#!/bin/bash
echo "SKU, FILE" >> tmp.csv
for file in /home/calvin/test/*.jpg
do
SKU_NAME="${file##*/}"
echo ${SKU_NAME:0:6}, \"inner for-loop?, ?, ?\" >> tmp.csv
done
uniq tmp.csv output.csv
As you can see I'm a noob as for programming :)
Please help me out, thanks in advance!
This will do the trick. This requires GNU awk to output in ascending order. If you don't care about the order, you can use any old awk and remove the PROCINFO line
#!/bin/bash
awk -F- '
BEGIN{
print "SKU, IMAGE"
}
{
sep=!a[$2]?"":", "
a[$2]=a[$2] sep $0
}
END{
PROCINFO["sorted_in"] = "#ind_str_asc" # GNU only feature
for(i in a){print "DA-" i ", " "\"" a[i] "\""}
}' <(find /home/calvin/test -type f -name "*.jpg" -printf "%f\n") > ./tmp.csv
Example Output
$ cat ./tmp.csv
SKU, IMAGE
DA-001, "DA-001-coffee.jpg, DA-001-car.jpg, DA-001-dog.jpg"
DA-002, "DA-002-house.jpg"
DA-003, "DA-003-coffee.jpg, DA-003-cat.jpg"
If the filenames don't contain spaces, you can use sed instead of an inner loop:
printf '%s\n' *.jpg \
| cut -f1,2 -d- \
| sort -u \
| while IFS= read -r sku ; do
echo "$sku",\"$(echo "$sku"* | sed 's/ /, /')\"
done
With the inner loop, you can switch to printf from echo. Sed is used to remove the trailing comma.
printf '%s\n' *.jpg \
| cut -f1,2 -d- \
| sort -u \
| while IFS= read -r sku ; do
printf %s "$sku, \""
for f in "$sku"* ; do
printf '%s, ' "$f"
done | sed 's/, $//'
printf '"\n'
done
If you don't want to parse the output of ls and run sort, you can store the prefixes in an associative array:
#!/bin/bash
declare -A prefix
for jpg in *.jpg ; do
p1=${jpg%%-*}
jpg=${jpg#*-}
p2=${jpg%%-*}
prefix[$p1-$p2]=1
done
for sku in "${!prefix[#]}" ; do
printf '%s, "' "$sku"
for f in "$sku"* ; do
printf '%s, ' "$f"
done | sed 's/, $//'
printf '"\n'
done
awk '
BEGIN {
OFS = ", "
print "SKU", "IMAGE"
for (i=1; i<ARGC; i++) {
curr = fname = ARGV[i]
sub(/-[^-]+$/,"",curr)
if ( curr != prev ) {
if ( i > 1 ) {
print prev, "\"" fnames "\""
}
prev = curr
fnames = ""
}
fnames = (fnames == "" ? "" : fnames OFS) fname
}
print prev, "\"" fnames "\""
exit
}
' /home/calvin/test/*.jpg
SKU, IMAGE
DA-001, "DA-001-car.jpg, DA-001-coffee.jpg, DA-001-dog.jpg"
DA-002, "DA-002-house.jpg"
DA-003, "DA-003-cat.jpg, DA-003-coffee.jpg"
As result of all the replies and advises I'm using this code to achieve the desired output:
#!/bin/bash
echo "SKU, IMAGES" >> output.csv
ls *.jpg | cut -f1,2 -d- | sort -u | while read SKU
do
echo $SKU, \"$(echo "$SKU"* | sed 's/ /, /g')\" >> output.csv
done
Thanks all!
I have multiple .vcf.gz files that look like this: (and there is 22 of them)
ALL.chr1.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz
ALL.chr2.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz
ALL.chr3.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz
...
ALL.chr22.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz
And I have a script filter.sh which can run on one file that looks like this. How would I loop trough all those 22 files?
filter_and_convert ()
{
echo -ne "varID\t"
bcftools view $1 -S $2 --force-samples -Ou | bcftools query -l | tr '\n' '\t' | sed 's/\t$/\n/'
#The first python inline script will check if a variant is blacklisted
NOW=$(date +%Y-%m-%d/%H:%M:%S)
echo "Starting at $NOW"
bcftools view -S $2 --force-samples $1 -Ou | \
bcftools query -f '%ID[\t%GT]\n' | \
awk '
{
for (i = 1; i <= NF; i++) {
if (substr($i,0,1) == "c") {
printf("%s",$i)
} else if ( substr($i, 0, 1) == ".") {
printf("\tNA")
} else if ($i ~ "[0-9]|[0-9]") {
n = split($i, array, "|")
printf("\t%d",array[1]+array[2])
} else {
#printf("\t%s",$i)
printf("Unexpected: %s",$i)
exit 1
}
}
printf("\n")
}
'
NOW=$(date +%Y-%m-%d/%H:%M:%S)
echo "Ending at $NOW"
}
filter_and_convert ALL.chr22.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz samples.txt
Replace
filter_and_convert ALL.chr22.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz samples.txt
with a for loop that calls the function on all the files that match a wildcard.
for file in ALL.*.vcf.gz; do
filter_and_convert "$file" samples.txt
done
v="ALL.chr"
p=".phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz"
for i in {1..22};
do
file=$v$i$p
bash filter.sh $file sample.txt
done
Use this file variable with your script. It should work. I am assuming first argument to your filer.sh is filename. Rest of the argument you can add
I'm trying to use a command on everyfile .log in a folder but I can't understand how to change the output file for every file in the folder.
#!/bin/bash
N="0"
for i in "*.log"
do
echo "Processing $f file..."
cat $i | grep "test" | awk '/show/ {a = $1} !/show/{print a,$6}' > "log$N.txt"
done
How can i increment the counter for log$n.txt?
It's bad practice to write shell loops just to process text (see https://unix.stackexchange.com/questions/169716/why-is-using-a-shell-loop-to-process-text-considered-bad-practice). This is all you need:
awk '
FNR==1 { print "Processing", FILENAME; a=""; close(out); out="log" fileNr++ ".txt" }
!/test/ { next }
/show/ { a = $1; next }
{ print a, $6 > out }
' *.log
#!/bin/bash
N="0"
for i in *.log
do
echo "Processing $f file..."
cat $i | grep "test" | awk '/show/ {a = $1} !/show/{print a,$6}' > "log$N.txt"
N=$((N+1))
done
You need to increment the variable 'N' over every iteration and remove the double-quotes in the for loop