I want to read in a remote configuration file in a bash script; locally the following works fine:
while IFS="=" read -r name value; do
declare "$name=$value"
done < "$cfg"
I tried to do the same using ssh and cat:
ssh "$hostname" "cat $remote_cfg" |
while IFS="=" read -r name value; do
declare "$name=$value"
echo $name $value
done
But my variables are only declared in scope of the while loop, how can I bring them to the outer scope?
Thanks in advance!
I have figured things out (source: http://mywiki.wooledge.org/BashFAQ/024).
Process substitution works:
while IFS="=" read -r name value; do
declare "$name=$value"
done < <(ssh "$hostname" "cat ~/.cuttleline.cfg")
Related
I've been working through creating a script to move some files from a local machine to a remote server. As part of that process I have a function that can either be called directly or wrapped with 'declare -fp' and sent along to an ssh command. The code I have so far looks like this:
export REMOTE_HOST=myserver
export TMP=eyerep-files
doTest()
{
echo "Test moving files from $TMP with arg $1"
declare -A files=(["abc"]="123" ["xyz"]="789")
echo "Files: ${!files[#]}"
for key in "${!files[#]}"
do
echo "$key => ${files[$key]}"
done
}
moveTest()
{
echo "attempting move with wrapped function"
ssh -t "$REMOTE_HOST" "$(declare -fp doTest|envsubst); doTest ${1#Q}"
}
moveTest $2
If I run the script with something like
./myscript.sh test dev
I get the output
attempting move with wrapped function
Test moving files from eyerep-files with arg dev
Files: abc xyz
bash: line 7: => ${files[]}: bad substitution
It seems like the string expansion for the for loop is not working correctly. Is this expected behaviour? If so, is there an alternative way to loop through an array that would avoid this issue?
If you're confident that your remote account's default shell is bash, this might look like:
moveTest() {
ssh -t "$REMOTE_HOST" "$(declare -f doTest; declare -p $(compgen -e)); doTest ${1#Q}"
}
If you aren't, it might instead be:
moveTest() {
ssh -t "$REMOTE_HOST" 'exec bash -s' <<EOF
set -- ${##Q}
$(declare -f doTest; declare -p $(compgen -e))
doTest \"\$#\"
EOF
}
I managed to find an answer here: https://unix.stackexchange.com/questions/294378/replacing-only-specific-variables-with-envsubst/294400
Since I'm exporting the global variables, I can get a list of them using compgen and use that list with envsubst to specify which variables I want to replace. My finished function ended up looking like:
moveTest()
{
echo "attempting move with wrapped function"
ssh -t "$REMOTE_HOST" "$(declare -fp doTest|envsubst "$(compgen -e | awk '$0="${"$0"}"') '${1}'"); doTest ${1#Q}"
}
I am a beginner and trying to write a script that takes a config file (example below) and sets the rights for the users, if that user or group doesn´t exist, they get added.
For every line in the file, I am cutting out the user or the group and check if they exist.
Right now I only check for users.
#!/bin/bash
function SetRights()
{
if [[ $# -eq 1 && -f $1 ]]
then
for line in $1
do
var1=$(cut -d: -f2 $line)
var2=$(cat /etc/passwd | grep $var1 | wc -l)
if [[ $var2 -eq 0 ]]
then
sudo useradd $var1
else
setfacl -m $line
fi
done
else
echo Enter the correct path of the configuration file.
fi
}
SetRights $1
The config file looks like this:
u:TestUser:- /home/temp
g:TestGroup:rw /home/temp/testFolder
u:TestUser2:r /home/temp/1234.txt
The output:
grep: TestGroup: No such file or directory
grep: TestUser: No such file or directory
"The useradd help menu"
If you could give me a hint what I should look for in my research, I would be very grateful.
Is it possible to reset var1 and var2? Using unset didn´t work for me and I couldn´t find variables could only be set once.
It's not clear how you are looping over the contents of the file -- if $1 contains the file name, you should not be seeing the errors you report.
But anyway, here is a refactored version which hopefully avoids your problems.
# Avoid Bash-only syntax for function definition
SetRights() {
# Indent function body
# Properly quote "$1"
if [[ $# -eq 1 && -f "$1" ]]
then
# Read lines in file
while read -r acl file
do
# Parse out user
user=${acl#*:}
user=${user%:*}
# Avoid useless use of cat
# Anchor regex correctly
if ! grep -q "^$user:" /etc/passwd
then
# Quote user
sudo useradd "$user"
else
setfacl -m "$acl" "$file"
fi
done <"$1"
else
# Error message to stderr
echo Enter the correct path of the configuration file. >&2
# Signal failure to the caller
return 1
fi
}
# Properly quote argument
SetRights "$1"
I want to create a function locally, echo_a in the example, and pass it with to a remote shell through ssh, here with typeset -f. The problem is that function does not have access to the local variables.
export a=1
echo_a() {
echo a: $a
}
bash <<EOF
$(typeset -f echo_a)
echo local heredoc:
echo_a
echo
echo local raw heredoc:
echo a: $a
echo
EOF
ssh localhost bash <<EOF
$(typeset -f echo_a)
echo remote heredoc:
echo_a
echo
echo remote raw heredoc:
echo a: $a
echo
EOF
Assuming the ssh connection is automatic, running the above script gives me as output:
local heredoc:
a: 1
local raw heredoc:
a: 1
remote heredoc:
a:
remote raw heredoc:
a: 1
See how the "remote heredoc" a is empty? What can I do to get 1 there?
I tested adding quotes and backslashes everywhere without success.
What am I missing? Would something else than typeset make this work?
Thanks to #Guy for the hint, it indeed is because ssh disables by default sending the environment variables. In my case, changing the server's setting was not wanted.
Hopefully we can hack around by using compgen, eval and declare.
First we identify added variables generically. Works if variables are created inside a called function too. Using compgen is neat because we don't need to export variables explicitely.
The array diff code comes from https://stackoverflow.com/a/2315459/1013628 and the compgen trick from https://stackoverflow.com/a/16337687/1013628.
# Store in env_before all variables created at this point
IFS=$'\n' read -rd '' -a env_before <<<"$(compgen -v)"
a=1
# Store in env_after all variables created at this point
IFS=$'\n' read -rd '' -a env_after <<<"$(compgen -v)"
# Store in env_added the diff betwen env_after and env_before
env_added=()
for i in "${env_after[#]}"; do
skip=
for j in "${env_before[#]}"; do
[[ $i == $j ]] && { skip=1; break; }
done
if [[ $i == "env_before" || $i == "PIPESTATUS" ]]; then
skip=1
fi
[[ -n $skip ]] || env_added+=("$i")
done
echo_a() {
echo a: $a
}
env_added holds now an array of all names of added variables between the two calls to compgen.
$ echo "${env_added[#]}"
a
I filter out also the variables env_before and PIPESTATUS as they are added automatically by bash.
Then, inside the heredocs, we add eval $(declare -p "${env_added[#]}").
declare -p VAR [VAR ...] prints, for each VAR, the variable name followed by = followed by its value:
$ a = 1
$ b = 2
$ declare -p a b
declare -- a=1
declare -- b=2
And the eval is to actually evaluate the declare lines. The rest of the code looks like:
bash <<EOF
# Eval the variables computed earlier
eval $(declare -p "${env_added[#]}")
$(typeset -f echo_a)
echo local heredoc:
echo_a
echo
echo local raw heredoc:
echo a: $a
echo
EOF
ssh rpi_301 bash <<EOF
# Eval the variables computed earlier
eval $(declare -p "${env_added[#]}")
$(typeset -f echo_a)
echo remote heredoc:
echo_a
echo
echo remote raw heredoc:
echo a: $a
echo
EOF
Finally, running the modified script gives me the wanted behavior:
local heredoc:
a: 1
local raw heredoc:
a: 1
remote heredoc:
a: 1
remote raw heredoc:
a: 1
I am working on a shell script, and I wish to do the following as part of a shell script without declaring a function. Basically, I would like to convert the following code into a shell script simply without declaring a function.
#!/bin/bash
function json2keyvalue {
cat<<EOF | jq -r 'to_entries|map("\(.key)\t\(.value|tostring)")[]'
{
"hello1": "world1",
"testk": "testv"
}
EOF
}
while IFS=$'\t' read -r key value
do
export "$key"="$value"
done < <(json2keyvalue)
into a shell script. I did the following,
values='{"hello1":"world1","hello1.world1.abc1":"hello2.world2.abc2","testk":"testv"}'
while IFS=$'\t' read -r key value
do
export "$key"="$value"
done < < (echo $values | jq -r 'to_entries|map("\(.key)\t\(.value|tostring)")[]')
But this doesn't seem to work. When I run the shell script, it gives the error as follows, where file name is abc.sh.
./abc.sh: line 6: syntax error near unexpected token `<'
./abc.sh: line 6: `done < < (jq -r 'to_entries|map("\(.key)\t\(.value|tostring)")[]' <<<"$values")'
The script takes a JSON like the following, and converts the key-values into environment variables.
{
"hello1": "world1",
"testk": "testv"
}
I suggest:
#!/bin/bash
shopt -s lastpipe
cat <<EOF | jq -r 'to_entries|map("\(.key)\t\(.value|tostring)")[]' | while IFS=$'\t' read -r key value; do export "$key"="$value"; done
{
"hello1": "world1",
"testk": "testv"
}
EOF
lastpipe: If set, and job control is not active, the shell runs the last command of a pipeline not executed in the background in the current shell environment.
Update:
If variable $values contains valid json code, this should work:
#!/bin/bash
shopt -s lastpipe
values='place valid json code here'
echo "$values" | jq -r 'to_entries|map("\(.key)\t\(.value|tostring)")[]' | while IFS=$'\t' read -r key value; do export "$key"="$value"; done
What I am trying to do is having a config file that controls a very basic backup script.
#Credentials
username="backup.user"
password="****"
from="/mnt"
to="/home/backup"
#Mountpoints
n=1
source="//10.X.X.X/Public"
destination="/mnt/Public"
n=2
source="//10.X.X.X/it"
destination="/mnt/it"
Inside the script itself it looks like this:
#!/bin/bash
#getting variables from the external config file
export $(cat config.ini | grep -v ^# | xargs)
#the command I am trying to achieve
mountpoint[$n]="mount -t cifs -o username=$username,password=$password,ro $source $destination"
#mounts the array of mountpoints defined
for mountpoint in "${mountpoint[#]}";
do
${mountpoint}
done
function currentDate () {
date +%Y%m%d
}
if [ ! -d "$to/$(currentDate)" ] ; then
mkdir "$to/$(currentDate)";
cp --verbose -R "$from/." "$to/$(currentDate)" >> $to/$(currentDate)/fileLog.txt
diff -qr $from $to/$(currentDate) >> $to/$(currentDate)/differencesLog.txt
else
exit
fi
umount -a -t cifs -l /mnt/*
done
I am trying to do this:
Have a set of variables in config for each mountpoint.
A for loop for that will echo the last source and destination as normal because it doesn't know when a certain set of variables is done for "n=1".
How will you guys do that?
Thanks a lot!
# create an array
mountpoint=()
# append to the array
mountpoint+=("item 1")
mountpoint+=("item 2")
# iterate the array
for i in "${mountpoint[#]}"; do
echo "${i}"
done