bash script pulling variables from .txt, keeps giving syntax error while trying to use mount command - bash

Ive been trying to get this to work for the last week and cannot figure out why this is not working. I get mixed results typing directly into the terminal, but keep getting syntax error messages when running from the .sh. using ubuntu 11.10
It looks like part of the mount command gets pushed to the next line not allowing it to complete properly.. I have no idea why this is happening or how to prevent it from going to the second line.
i have several lines defined as follows in mounts.txt, that gets read from mount-drives.sh below
I have called it to run using sudo so it shouldnt be a permissions issue.
Thanks for taking a look, let me know if additional info is needed.
mounts.txt
mountname,//server/share$,username,password,
mount-drives.sh ---origional, updated below
#!/bin/bash
while read LINE;
do
# split lines up using , to separate variables
name=$(echo $LINE | cut -d ',' -f 1)
path=$(echo $LINE | cut -d ',' -f 2)
user=$(echo $LINE | cut -d ',' -f 3)
pass=$(echo $LINE | cut -d ',' -f 4)
echo $name
echo $path
echo $user
echo $pass
location="/mnt/test/$name/"
if [ ! -d $location ]
then
mkdir $location
fi
otherstuff="-o rw,uid=1000,gid=1000,file_mode=0777,dir_mode=0777,username=$user,password=$pass"
mount -t cifs $otherstuff $path $location
done < "/path/to/mounts.txt";
mount-drives.sh ---updated
#!/bin/bash
while read LINE
do
name=$(echo $LINE | cut -d ',' -f 1)
path=$(echo $LINE | cut -d ',' -f 2)
user=$(echo $LINE | cut -d ',' -f 3)
pass=$(echo $LINE | cut -d ',' -f 4)
empty=$(echo $LINE | cut -d ',' -f 5)
location="/mount/test/$name/"
if [ ! -d $location ]
then
mkdir $location
fi
mounting="mount -t cifs $path $location -o username=$user,password=$pass,rw,uid=1000,gid=1000,file_mode=0777,dir_mode=0777"
$mounting
echo $mounting >> test.txt
done < "/var/www/MediaCenter/mounts.txt"

Stab in the dark (after reading the comments). The "$pass" is picking up a newline because the mounts.txt was created in windows and has windows line endings. Try changing the echo $pass line to:
echo ---${pass}---
and see if it all shows up correctly.

There's a lot here that could stand improvement. Consider the following -- far more compact, far more correct -- approach:
while IFS=, read -u 3 -r name path user pass empty _; do
mkdir -p "$location"
cmd=( mount \
-t cifs \
-o "rw,uid=1000,gid=1000,file_mode=0777,dir_mode=0777,username=$user,password=$pass" \
"$path" "$location" \
)
printf -v cmd_str '%q ' "${cmd[#]}" # generate a string corresponding with the command
echo "$cmd_str" >>test.txt # append that string to our output file
"${cmd[#]}" # run the command in the array
done 3<mounts.txt
Unlike the original, this will work correctly even if your path or location values contain whitespace.

Related

Shell: Add string to the end of each line, which match the pattern. Filenames are given in another file

I'm still new to the shell and need some help.
I have a file stapel_old.
Also I have in the same directory files like english_old_sync, math_old_sync and vocabulary_old_sync.
The content of stapel_old is:
english
math
vocabulary
The content of e.g. english is:
basic_grammar.md
spelling.md
orthography.md
I want to manipulate all files which are given in stapel_old like in this example:
take the first line of stapel_old 'english', (after that math, and so on)
convert in this case english to english_old_sync, (or after that what is given in second line, e.g. math to math_old_sync)
search in english_old_sync line by line for the pattern '.md'
And append to each line after .md :::#a1
The result should be e.g. of english_old_sync:
basic_grammar.md:::#a1
spelling.md:::#a1
orthography.md:::#a1
of math_old_sync:
geometry.md:::#a1
fractions.md:::#a1
and so on. stapel_old should stay unchanged.
How can I realize that?
I tried with sed -n, while loop (while read -r line), and I'm feeling it's somehow the right way - but I still get errors and not the expected result after 4 hours inspecting and reading.
Thank you!
EDIT
Here is the working code (The files are stored in folder 'olddata'):
clear
echo -e "$(tput setaf 1)$(tput setab 7)Learning directories:$(tput sgr 0)\n"
# put here directories which should not become flashcards, command: | grep -v 'name_of_directory_which_not_to_learn1' | grep -v 'directory2'
ls ../ | grep -v 00_gliederungsverweise | grep -v 0_weiter | grep -v bibliothek | grep -v notizen | grep -v Obsidian | grep -v z_nicht_uni | tee olddata/stapel_old
# count folders
echo -ne "\nHow much different folders: " && wc -l olddata/stapel_old | cut -d' ' -f1 | tee -a olddata/stapel_old
echo -e "Are this learning directories correct? [j ODER y]--> yes; [Other]-->no\n"
read lernvz_korrekt
if [ "$lernvz_korrekt" = j ] || [ "$lernvz_korrekt" = y ];
then
read -n 1 -s -r -p "Learning directories correct. Press any key to continue..."
else
read -n 1 -s -r -p "Learning directories not correct, please change in line 4. Press any key to continue..."
exit
fi
echo -e "\n_____________________________\n$(tput setaf 6)$(tput setab 5)Found cards:$(tput sgr 0)$(tput setaf 6)\n"
#GET && WRITE FOLDER NAMES into olddata/stapel_old
anzahl_zeilen=$(cat olddata/stapel_old |& tail -1)
#GET NAMES of .md files of every stapel and write All to 'stapelname'_old_sync
i=0
name="var_$i"
for (( num=1; num <= $anzahl_zeilen; num++ ))
do
i="$((i + 1))"
name="var_$i"
name=$(cat olddata/stapel_old | sed -n "$num"p)
find ../$name/ -name '*.md' | grep -v trash | grep -v Obsidian | rev | cut -d'/' -f1 | rev | tee olddata/$name"_old_sync"
done
(tput sgr 0)
I tried to add:
input="olddata/stapel_old"
while IFS= read -r line
do
sed -n "$line"p olddata/stapel_old
done < "$input"
The code to change only the english_old_sync is:
lines=$(wc -l olddata/english_old_sync | cut -d' ' -f1)
for ((num=1; num <= $lines; num++))
do
content=$(sed -n "$num"p olddata/english_old_sync)
sed -i "s/"$content"/""$content":::#a1/g"" olddata/english_old_sync
done
So now, this need to be a inner for-loop, of a outer for-loop which holds the variable for english, right?
stapel_old should stay unchanged.
You could try a while + read loop and embed sed inside the loop.
#!/usr/bin/env bash
while IFS= read -r files; do
echo cp -v "$files" "${files}_old_sync" &&
echo sed '/^.*\.md$/s/$/:::#a1/' "${files}_old_sync"
done < olddata/staple_old
convert in this case english to english_old_sync, (or after that what is given in second line, e.g. math to math_old_sync)
cp copies the file with a new name, if the goal is renaming the original file name from the content of the file staple_old then change cp to mv
The -n and -i flag from sed was ommited , include it, if needed.
The script also assumes that there are no empty/blank lines in the content of staple_old file. If in case there are/is add an addition test after the line where the do is.
[[ -n $files ]] || continue
It also assumes that the content of staple_old are existing files. Just in case add an additional test.
[[ -e $files ]] || { printf >&2 '%s no such file or directory.\n' "$files"; continue; }
Or an if statement.
if [[ ! -e $files ]]; then
printf >&2 '%s no such file or directory\n' "$files"
continue
fi
See also help test
See also help continue
Combining them all together should be something like:
#!/usr/bin/env bash
while IFS= read -r files; do
[[ -n $files ]] || continue
[[ -e $files ]] || {
printf >&2 '%s no such file or directory.\n' "$files"
continue
}
echo cp -v "$files" "${files}_old_sync" &&
echo sed '/^.*\.md$/s/$/:::#a1/' "${files}_old_sync"
done < olddata/staple_old
Remove the echo's If you're satisfied with the output so the script could copy/rename and edit the files.

Quoting variable in application call

I'm trying to run an application that needs a parameter passed in single or double quotes:
bwa mem -R '#RG\tID:foo\tLB=foo\tPL=illumina\tPU=1234.1\tSM=bar' ...
I'm trying to run this in a script:
#!/usr/bin/bash
var=foo_bar
first=$(echo $var | cut -d '_' -f 1)
second=$(echo $var | cut -d '_' -f 2)
readgroup="#RG\tID:$first\tLB=HUM\tPL=illumina\tPU=1234.1\tSM=$second"
echo \'$readgroup\'
'#RG\tID:foo\tLB=HUM\tPL=illumina\tPU=1234.1\tSM=bar'
However, when I use this as a parameter in my application call:
var=foo_bar
first=$(echo $var | cut -d '_' -f 1)
second=$(echo $var | cut -d '_' -f 2)
readgroup="#RG\tID:$first\tLB=HUM\tPL=illumina\tPU=1234.1\tSM=$second"
bwa mem -R \'$readgroup\' ...
it's interpreted as
\'#RG\tID:HUM-7\tLB=HUM\tPL=illumina\tPU=1234.1\tSM=HUM-7\'
What am I doing wrong?
The application doesn't care about the quotes; they are only used to protect the string from the shell.
var=foo_bar
IFS=_ read first second <<< "$var"
readgroup="#RG\tID:$first\tLB=HUM\tPL=illumina\tPU=1234.1\tSM=$second"
bwa mem -R "$readgroup"
If the \t are actually supposed to be literal tab characters, consider using printf instead of an assignment statement.
printf -v readgroup '#RG\tID:%s\tLB=HUM\tPL=illumina\tPU=1234.1\tSM=%s' "$first" "$second"
Should be like:
readgroup="'#RG\tID:$first\tLB=HUM\tPL=illumina\tPU=1234.1\tSM=$second'" [with '' wrapping]
readgroup="#RG\tID:$first\tLB=HUM\tPL=illumina\tPU=1234.1\tSM=$second" [without]
Important thing is to use -e option for echo [ See man echo ]
echo -e $readgroup

Execute commands from shell script fails

I'm trying read a file which contains lines like this:
Run COMMAND with options "OPTIONS" and arguments "ARGUMENTS"
Then I want to execute this command with given options and arguments. For example I'd like to execute these commands:
Run pwd with options "" and arguments ""
Run ls with options "-al" and arguments "$HOME"
Run ls with options "-al" and arguments "Example: \"strange folder name\""
This is my code
#!/bin/bash
while read -r line
do
COMMAND=$(echo "$line" | cut -d" " -f 2)
OPTIONS=$(echo "$line" | cut -d" " -f 5 | tr -d '"')
ARGUMENTS=$(echo "$line" | cut -d" " -f 8)
$COMMAND $OPTIONS $ARGUMENTS
done <$1
First example is working as it should, second one is giving me error ls: cannot access $HOME: No such file or directory' and third one is not storing the name of the folder to $ARGUMENTS correctly.
second one is giving me error ls: cannot access $HOME: No such file or directory'
This is because the folder named $HOME does not exist. I am not talking about the value of $HOME variable, but the string literal. The shell does not execute the parameter expansion in your situation.
third one is not storing the name of the folder to $ARGUMENTS correctly
This is because -f 8 only extract column 8, try -f 8- to extract the 8th column and all the others until the end of line.
You can give a try to this version below:
while read -r line; do
COMMAND=$(printf "%s" "${line}" | cut -d" " -f 2)
OPTIONS=$(printf "%s" "${line}" | cut -d" " -f 5 | tr -d '"')
ARGUMENTS=$(printf "%s" "${line}" | cut -d" " -f 8-)
$COMMAND $OPTIONS "$(eval printf \"%s\" "$ARGUMENTS")"
done < "${1}"
The eval is a shell built-in command which is used to enable parameter expansion of ARGUMENTS, if applicable.
I have to warn you that the eval is usualy say risky to use.

A script to find all the users who are executing a specific program

I've written the bash script (searchuser) which should display all the users who are executing a specific program or a script (at least a bash script). But when searching for scripts fails because the command the SO is executing is something like bash scriptname.
This script acts parsing the ps command output, it search for all the occurrences of the specified program name, extracts the user and the program name, verifies if the program name is that we're searching for and if it's it displays the relevant information (in this case the user name and the program name, might be better to output also the PID, but that is quite simple). The verification is accomplished to reject all lines containing program names which contain the name of the program but they're not the program we are searching for; if we're searching gedit we don't desire to find sgedit or gedits.
Other issues I've are:
I would like to avoid the use of a tmp file.
I would like to be not tied to GNU extensions.
The script has to be executed as:
root# searchuser programname <invio>
The script searchuser is the following:
#!/bin/bash
i=0
search=$1
tmp=`mktemp`
ps -aux | tr -s ' ' | grep "$search" > $tmp
while read fileline
do
user=`echo "$fileline" | cut -f1 -d' '`
prg=`echo "$fileline" | cut -f11 -d' '`
prg=`basename "$prg"`
if [ "$prg" = "$search" ]; then
echo "$user - $prg"
i=`expr $i + 1`
fi
done < $tmp
if [ $i = 0 ]; then
echo "No users are executing $search"
fi
rm $tmp
exit $i
Have you suggestion about to solve these issues?
One approach might looks like such:
IFS=$'\n' read -r -d '' -a pids < <(pgrep -x -- "$1"; printf '\0')
if (( ! ${#pids[#]} )); then
echo "No users are executing $1"
fi
for pid in "${pids[#]}"; do
# build a more accurate command line than the one ps emits
args=( )
while IFS= read -r -d '' arg; do
args+=( "$arg" )
done </proc/"$pid"/cmdline
(( ${#args[#]} )) || continue # exited while we were running
printf -v cmdline_str '%q ' "${args[#]}"
user=$(stat --format=%U /proc/"$pid") || continue # exited while we were running
printf '%q - %s\n' "$user" "${cmdline_str% }"
done
Unlike the output from ps, which doesn't distinguish between ./command "some argument" and ./command "some" "argument", this will emit output which correctly shows the arguments run by each user, with quoting which will re-run the given command correctly.
What about:
ps -e -o user,comm | egrep "^[^ ]+ +$1$" | cut -d' ' -f1 | sort -u
* Addendum *
This statement:
ps -e -o user,pid,comm | egrep "^\s*\S+\s+\S+\s*$1$" | while read a b; do echo $a; done | sort | uniq -c
or this one:
ps -e -o user,pid,comm | egrep "^\s*\S+\s+\S+\s*sleep$" | xargs -L1 echo | cut -d ' ' -f1 | sort | uniq -c
shows the number of process instances by user.

How to replace or escape <tab> characters with \t in bash script and being able to output single quotes?

In the goal to create a file from a one line (bash) command, the goal is to output the contents of any text file - in this example a bash script - and wrap each line inside a command that is able to output that same line when pasted in a Terminal window.
Example source input file:
Line 1
Line 2
Line 3
Example desired output:
echo 'Line 1';echo 'Line 2';echo 'Line 3';
Note: whether printf, echo or another command is used to create the output, doesn't matter as long as the source is human readable.
One hurdle were the single quotes, that would not be recreated. Therefore use the form $'string', which are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard.
Another requirement is to re-create tab characters from the old file in the new file. Therefore the wish is to replace <\tab> characters with \t.
Our tries to do this with sed or tr fail. How to replace tabs with their escape \t counterpart and still being able to output lines with original quotes?
Input file /Library/Scripts/BootRepairMount.sh contains:
$ cat /Library/Scripts/BootRepairMount.sh
#!/bin/bash
sleep 18
for OUTPUT in $(diskutil list | grep ': Apple_HFS' | awk '{ print $NF }')
do
if [[ -z $(df -lnh | grep /dev/$OUTPUT) ]]; then
echo "$OUTPUT is not mounted, repair and mount"
diskutil repairVolume $OUTPUT
diskutil mount $OUTPUT
fi
done
The best shell one line command we could create is:
$ oldifs=$IFS;printf '\n';printf '{';while IFS= read -r p;do [[ "$p" == *"'"* ]] && echo -n "echo $'$p';" || echo -n "echo '$p';"; done < /Library/Scripts/BootRepairMount.sh | tr '\t' '\134\164';printf '}';printf '\n\n';IFS=$oldifs
Which returns this faulty output:
{echo '#!/bin/bash';echo 'sleep 18';echo $'for OUTPUT in $(diskutil list | grep ': Apple_HFS' | awk '{ print $NF }')';echo 'do';echo '\if [[ -z $(df -lnh | grep /dev/$OUTPUT) ]]; then';echo '\\echo "$OUTPUT is not mounted, repair and mount"';echo '\\diskutil repairVolume $OUTPUT';echo '\\diskutil mount $OUTPUT';echo '\fi';echo 'done';}
Desired output is:
{echo '#!/bin/bash';echo 'sleep 18';echo $'for OUTPUT in $(diskutil list | grep ': Apple_HFS' | awk '{ print $NF }')';echo 'do';echo '\tif [[ -z $(df -lnh | grep /dev/$OUTPUT) ]]; then';echo '\t\techo "$OUTPUT is not mounted, repair and mount"';echo '\t\tdiskutil repairVolume $OUTPUT';echo '\t\tdiskutil mount $OUTPUT';echo '\tfi';echo 'done';}
Bash one line command version 2
$ oldifs=$IFS;printf '\n';printf '{';while IFS= read -r p;do [[ "$p" == *"'"* ]] && printf 'printf $'\''%q'\'';' "$p" || printf 'printf '\''%q'\'';' "$p"; done < /Library/Scripts/BootRepairMount.sh;printf '}';printf '\n\n';IFS=$oldifs
returns output that is heavy escaped:
{printf '\#\!/bin/bash';printf 'sleep\ 18';printf $'for\ OUTPUT\ in\ \$\(diskutil\ list\ \|\ grep\ \':\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Apple_HFS\'\ \|\ awk\ \'\{\ print\ \$NF\ \}\'\)';printf 'do';printf '$'\tif [[ -z $(df -lnh | grep /dev/$OUTPUT) ]]; then'';printf '$'\t\techo "$OUTPUT is not mounted, repair and mount"'';printf '$'\t\tdiskutil repairVolume $OUTPUT'';printf '$'\t\tdiskutil mount $OUTPUT'';printf '$'\tfi'';printf 'done';}
that never gets unescaped back to its original values in Mac OS X 10.7.5.
printf '\#\!/bin/bash';
outputs:
\#\!/bin/bash
As well as:
echo -e '\#\!/bin/bash'
does output the unescaped value
\#\!/bin/bash
-e is not a valid command switch for the Mac OS X 10.7.5 echo command, according to its man page.
bash's builtin command printf has %q format code that handles this:
printf '\n{ '; while IFS= read -r p; do printf "echo %q; " "$p"; done < /Library/Scripts/BootRepairMount.sh; printf '}\n\n'
Unfortunately, it doesn't always choose quoting/escaping modes that're easy to read. Specifically, it tends to prefer escaping individual metacharacters (e.g. spaces) rather than enclosing them in quotes:
{ echo \#\!/bin/bash; echo sleep\ 18; echo for\ OUTPUT\ in\ \$(diskutil\ list\ \|\ grep\ \':\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Apple_HFS\'\ \|\ awk\ \'{\ print\ \$NF\ }\'); echo do; echo $'\tif [[ -z $(df -lnh | grep /dev/$OUTPUT) ]]; then'; echo $'\t\techo "$OUTPUT is not mounted, repair and mount"'; echo $'\t\tdiskutil repairVolume $OUTPUT'; echo $'\t\tdiskutil mount $OUTPUT'; echo $'\tfi'; echo done; }
If I understand right you want paste one long line to the Terminal.app and want get the "source code" of original script. So, need a script what will generate the one-line script.
Maybe a bit unusual solution, but it is easy and simple.
here is the test script called test.sh (instead of your BootReapirMount.sh)
for i in {1..10}
do
date
done
Here is the generator script mkecho.sh
#!/bin/bash
[[ ! -f "$1" ]] && echo "Need filename" && exit 1
asc=$(gzip < "$1" | base64)
echo "base64 -D <<<'$asc'| gzip -d"
Now, run:
bash mkecho.sh test.sh
you will get the next:
base64 -D <<<'H4sIAASwqFEAA0vLL1LIVMjMU6g21NMzNKjlSsnn4kxJLEkFMvJSuQBZFmY0HwAAAA=='| gzip -d
If you copy and paste the above into the terminal, it will will display the original test.sh
Variant2
If you want directly execute the script, you should modify the mkecho.sh to the next mkeval.sh
#!/bin/bash
[[ ! -f "$1" ]] && echo "Need filename" && exit 1
asc=$(gzip < "$1" | base64)
echo -n 'eval "$(base64 -D <<<"'
echo -n $asc
echo -n '" | gzip -d)"'
echo
When run
bash mkeval.sh test.sh
will get
eval "$(base64 -D <<<"H4sIAASwqFEAA0vLL1LIVMjMU6g21NMzNKjlSsnn4kxJLEkFMvJSuQBZFmY0HwAAAA==" | gzip -d)"
and finally when you copy and paste it into the terminal, you run the test.sh and will get:
Fri May 31 16:25:08 CEST 2013
... 8 lined deleted...
Fri May 31 16:25:08 CEST 2013
Warning: because the script is NOT TESTED for every possible conditions, nor for redirects and so on - I really don't recommending using the eval verision.
sed 's/\\t/\\/g'
$ echo 'ffsd \tif [[ -z $' | sed 's/\\t/\\/g'
ffsd \if [[ -z $

Resources