Debian testing, bash...
I'm trying to load variables from existing programs.
Set Programs Variable
xPROGS="$(echo -e "exiftool\nrsync\nxsel")"
Attempt to create variables with x(program name)
echo "$xPROGS" | while read z; do x$z="$(whereis -b "$z" | awk '{print $2}')" ; done
Errors;
bash: xexiftool=/usr/bin/rsync: No such file or directory
bash: xrsync=/usr/bin/rsync: No such file or directory
bash: xxsel=/usr/bin/rsync: No such file or directory
This works;
$ whereis -b rsync | awk '{print $2}'
I can't achieve varying the variable name successfully.
Could someone please help.
$ cat t.sh
#!/bin/bash
progs=(exiftool rsync xsel)
for prog in "${progs[#]}"; do
read -r _ "x${prog}" _ <<< "$(whereis -b "${prog}")"
done
echo "exiftool: [${xexiftool}]"
echo "rsync: [${xrsync}]"
echo "xsel: [${xxsel}]"
$ ./t.sh
exiftool: []
rsync: [/usr/bin/rsync]
xsel: []
#Etan Reisner provided the link from where the following codes are based:
echo "$xPROGS" | while read z; do IFS= read -r "x$z" <<<$(whereis -b rsync | awk '{print $2}') ; done
But I feel that the rsync command will not change its location in the file system tree
RSYNC=$(whereis -b rsync); RSYNC="${RSYNC#* }"; echo "$xPROGS" | while read z; do IFS= read -r "x$z" <<<"$RSYNC" ; done
Related
I have a file like so:
- ${VAR1}/blah/blah:/blah1
- ${VAR2}/blah/blah:/blah2
- $VAR3:/blah3
I ultimately need to create those three folders.
I am using sed to extract the folder part:
$ cat test.txt | grep -E '^ +- \$.*?:.*?$' | sed 's/.*- \(\$.*\):.*/\1/g'
${VAR1}/blah/blah
${VAR2}/blah/blah
$VAR3
I need to create those folders but I need those shell variables to expand. Right now they don't:
$ cat test.txt | grep -E '^ +- \$.*?:.*?$' | sed 's/.*- \(\$.*\):.*/\1/g' | while read line; do echo "$line"; done
${VAR1}/blah/blah
${VAR2}/blah/blah
$VAR3
Is there a way to get the expanded strings so I can run mkdir instead of echo to make the folders?
You may use this bash script with envsubst:
#!/usr/bin/env bash
export VAR1 VAR2 VAR3
while IFS=' -:' read -r _ d _; do
mkdir -p "$d"
done < <(envsubst < test.txt)
Alternatively use this envsubst + awk + xargs solution:
envsubst < text.txt |
awk -F '[-:[:blank:]]+' -v ORS='\0' '{print $2}' |
xargs -0 mkdir -p
First of all those variables should be exported to be accessible from your script. Then you could just use the cut and tr commands combination to extract dir name in a loop like the following:
#!/bin/bash -eu
while read -r LINE; do
echo "$LINE" | cut -d ':' -f 1 | tr -d ' ' | tr -d '-'
done < test.txt
I have a command and it returns 108 set of week/enumeration:
Command:
impala-shell -B -f query.sql
Results:
20180203 1
20180127 2
20180120 3
...
I parsed the results and read the week and enumeration as two variables. However, I have to use a variable wk to store intermediate results first:
wk="$(impala-shell -B -f query.sql)"
echo "$wk" | while read -r a b; do echo $a--$b; done
I tried to avoid using additional variable wk:
"$(impala-shell -B -f query.sql)" | while read -r a b; do echo $a--$b; done
But it returned:
...
20160213 104
20160206 105
20160130 106
20160123 107
20160116 108: command not found
I understand you can use wk="$(impala-shell -B -f query.sql)" && echo "$wk" | while read -r a b; do echo $a--$b; done but that doesn't avoid using a variable in the middle. How to compose a one-liner without using the variable wk?
or
awk to the rescue!
$ impala-shell -B -f query.sql | awk '{print $1"--"$2}'
You can execute commands first (inline) when using special quotes ``
Try this (untested, as i neither have your shell, nor that script):
`impala-shell -B -f query.sql` | while read -r a b; do echo $a--$b; done
Most elegant answer goes to choroba in the question comments! You just need to remove the quotes!
impala-shell -B -f query.sql | while read -r a b ; do echo $a--$b; done
I am trying to create:
a list containing files present in a but missing in b
a list containing files present in b but missing in a
This is my code so far:
#!/bin/sh
set -e
A_DIR_1=/tmp/1
A_DIR_2=/tmp/2
A_DIR_3=/tmp/3
B_DIR_1=/tmp/1
B_DIR_2=/tmp/2
B_DIR_3=/tmp/3
for i in {1..3}
do
A="A_DIR_$i"
B="B_DIR_$i"
if [ d ${!A} ]; then
bash -c 'diff -u <(find "${!B}" |sed "s:${!B}::") <(find "${!A}" |sed "s:${!A}::") |sed "/^+\//!d; s::${!A}/:"' >> /tmp/fileA
bash -c 'diff -u <(find "${!A}" |sed "s:${!A}::") <(find "${!B}" |sed "s:${!A}::") |sed "/^+\//!d; s::${!B}/:"' >> /tmp/fileB
fi
done
However, when I run it, I get the following error: find: cannot search : No such file or directory'. Why is this? The directories definitely exist.
mm, imho your code is difficult.
If I've understood you clearly, that can help:
Prepare:
mkdir /tmp/dir1
mkdir /tmp/dir2
touch /tmp/dir1/test{1..12}
touch /tmp/dir2/test{1..15}
touch /tmp/dir1/test{22..25}
Code:
#!/bin/bash
dir1=/tmp/dir1
dir2=/tmp/dir2
diff_result=$(diff -u "${dir1}" "${dir2}")
echo "${diff_result}" | grep "${dir1}" | awk '{print $NF}' > /tmp/files_only_in_dir1
echo "${diff_result}" | grep "${dir2}" | awk '{print $NF}' > /tmp/files_only_in_dir2
Output:
$ cat /tmp/files_only_in_dir1
test22
test23
test24
test25
$ cat /tmp/files_only_in_dir2
test13
test14
test15
I need to read a file that has lines like
user=username1
pass=password1
How can I read multiple lines like this into separate variables like username and password?
Would I use awk or grep? I have found ways to read lines into variables with grep but would I need to read the file for each individual item?
The end result is to use these variables to access a database via the command line. So I need to be able to read, store and use these values in other commands.
if the process which generates the file is safe and has shell syntax just source the file.
. ./file
Otherwise the file can be processes before to add quotes
perl -ne 'if (/^([A-Za-z_]\w*)=(.*)/) {$k=$1;$v=$2;$v=~s/\x27/\x27\\\x27\x27/g;print "$k=\x27$v\x27\n";}' <file >file2
. ./file2
If you want to use awk then
Input
$ cat file
user=username1
pass=password1
Reading
$ user=$(awk -F= '$1=="user"{print $2;exit}' file)
$ pass=$(awk -F= '$1=="pass"{print $2;exit}' file)
Output
$ echo $user
username1
$ echo $pass
password1
You could use a loop for your file perhaps, but this is probably the functionality you're looking for.
$ echo 'user=username1' | awk -F= '{print $2}'
username1
Using the -F flag sets the delimiter to = and we select the 2nd item from the row.
file.txt:
user=username1
pass=password1
user=username2
pass=password2
user=username3
pass=password3
Do to avoid browsing several times the file file.txt:
#!/usr/bin/env bash
func () {
echo "user:$1 pass:$2"
}
i=0
while IFS='' read -r line; do
if [ $i -eq 0 ]; then
i=1
user=$(echo ${line} | cut -f2 -d'=')
else
i=0
pass=$(echo ${line} | cut -f2 -d'=')
func "$user" "$pass"
fi
done < file.txt
Output:
user:username1 pass:password1
user:username2 pass:password2
user:username3 pass:password3
I have a problem
I Have a program general like this gene.sh
that for all file (es file: geneX.csv) make a directory with the name of gene (example: Genex/geneX.csv) next this program compile an other program inside gene.sh but this progrm need a varieble and I dont know how do it.
this is the program gene.sh
#!/bin/bash
# Create a dictory for each file *.xls and *.csv
for fname in *.xlsx *csv
do
dname=${fname%.*}
[[ -d $dname ]] || mkdir "$dname"
mv "$fname" "$dname"
done
# For each gene go inside the directory and compile the programs getChromosomicPositions.sh to have the positions, and getHapolotipeStings.sh to have the variants
for geni in */; do
cd $geni
z=$(tail -n 1 *.csv | tr ';' "\n" | wc -l)
cd ..
cp getChromosomicPositions.sh $geni --->
cp getHaplotypeStrings.sh $geni
cd $geni
export z
./getChromosomicPositions.sh *.csv
export z
./getHaplotypeStrings.sh *.csv
cd ..
done
This is the program getChromosomichPositions.sh:
rm chrPosRs.txt
grep '^Haplotype\ ID' $1 | cut -d ";" -f 4-61 | tr ";" "\n" | awk '{print "select chrom,chromStart,chromEnd,name from snp147 where name=\""$1"\";"}' > listOfQuery.txt
while read l; do
echo $l > query.txt
mysql -h genome-mysql.cse.ucsc.edu -u genome -A -D hg38 --skip-column-names < query.txt > queryResult.txt
if [[ "$(cat queryResult.txt)" == "" ]];
then
cat query.txt |
while read line; do
echo $line | awk '$6 ~/rs/ {print $6}' > temp.txt;
if [[ "$(cat temp.txt)" != "" ]];
then cat temp.txt | awk -F'name="' '{print $2}' | sed -e 's/";//g' > temp.txt;
./getHGSVposHG19.sh temp.txt ---> Hear the problem--->
else
echo $line | awk '{num=sub(/.*:g\./,"");num+=sub(/\".*/,"");if(num==2){print};num=""}' > temp2.txt
fi
done
cat query.txt >> varianti.txt
echo "Missing Data" >> chrPosRs.txt
else
cat queryResult.txt >> chrPosRs.txt
fi
done < listOfQuery.txt
rm query*
hear the problem:
I need to enter in the file temp.txt and put automatically at the beginning of the file the variable $geni of the program gene.sh
How can I do that?
Why not pass "$geni" as say the first argument when invoking your script, and treating the rest of the arguments as your expected .csv files.
./getChromosomicPositions.sh "$geni" *.csv
Alternatively, you can set it as environment variable for the script, so that it can be used there (or just export it).
geni="$geni" ./getChromosomicPositions.sh *.csv
In any case, once you have it available in the second script, you can do
if passed as the first argument:
echo "${1}:$(cat temp.txt | awk -F'name="' '{print $2}' | sed -e 's/";//g')
or if passed as environment variable:
echo "${geni}:$(cat temp.txt | awk -F'name="' '{print $2}' | sed -e 's/";//g')