running sed passing a environment variable - shell

i have this sed command which works running from a shell
/bin/sed -i -e '/^\['"9876"'\]/ r /etc/asterisk/tmp_'"9876"'.txt' /etc/asterisk/sip_peers.include
running it from a sh script it does not work:
#!/bin/bash
peers=/etc/asterisk/sip_peers.include
tmp=/etc/asterisk/tmp_$ext.txt
var=`cat < $tmp`
printenv > /etc/asterisk/zprintenv.txt
echo $tmp $ext > /etc/asterisk/zoutput.txt
/bin/sed -i -e '/^\['"$ext"'\]/ r /etc/asterisk/tmp_'"$ext"'.txt' /etc/asterisk/sip_peers.include
$ext is a environment variable and it is still present when i pass printenv to a text file. Also the $ext variable is passed to output.txt for testing purpose.
sip_peers.include, where the values from /etc/asterisk/tmp_$ext.txt will be inserted after []:
[9876]
qualify=yes
nat=yes
call-limit=4
....
output zoutput.txt:
/etc/asterisk/tmp_9876.txt 9876
snippet from zprintenv.txt:
contacts=1
vm=1
ext=9876
emergency_trunk=
callerid_override=
enduser_company_contacts=0
.....
output $tmp:
;mac=e02f6d613554
;model=spa504G
please take a look at it, i can not find the mistake since hours.
thanks a lot

Quoting OP comment:
It is a run time issue.
The script is called from a gui and the file sip_peers.include is stored after the script is executed. So sed cannot process the non-existing file.
(This seems like a use case that could be interesting for others. So I am making a Q/A pair by quoting answer from OPs comment,
matching a proposed edit to the title, for being found by users with similar problem.)

Related

Generating parameters for `docker run` through command expansion from .env files

I'm facing some problems to pass some environment parameters to docker run in a relatively generic way.
Our first iteration was to load a .env file into the environment via these lines:
set -o allexport;
. "${PROJECT_DIR}/.env";
set +o allexport;
And then manually typing the --env VARNAME=$VARNAME as options for the docker run command. But this can be quite annoying when you have dozens of variables.
Then we tried to just pass the file, with --env-file .env, and it seems to work, but it doesn't, because it does not play well with quotes around the variable values.
Here is where I started doing crazy/ugly things. The basic idea was to do something like:
set_docker_parameters()
{
grep -v '^$' "${PROJECT_DIR}/.env" | while IFS= read -r LINE; do
printf " -e %s" "${LINE}"
done
}
docker run $(set_docker_parameters) --rm image:label command
Where the parsed lines are like VARIABLE="value", VARIABLE='value', or VARIABLE=value. Blank lines are discarded by the piped grep.
But docker run complains all the time about not being called properly. When I expand the result of set_docker_parameters I get what I expected, and when I copy its result and replace $(set_docker_parameters), then docker run works as expected too, flawless.
Any idea on what I'm doing wrong here?
Thank you very much!
P.S.: I'm trying to make my script 100% POSIX-compatible, so I'll prefer any solution that does not rely on Bash-specific features.
Based on the comments of #jordanm I devised the following solution:
docker_run_wrapper()
{
# That's not ideal, but in any case it's not directly related to the question.
cmd=$1
set --; # Unset all positional arguments ($# will be emptied)
# We don't have arrays (we want to be POSIX compatible), so we'll
# use $# as a sort of substitute, appending new values to it.
grep -v '^$' "${PROJECT_DIR}/.env" | while IFS= read -r LINE; do
set -- "$#" "--env";
set -- "$#" "${LINE}";
done
# We use $# in a clearly non-standard way, just to expand the values
# coming from the .env file.
docker run "$#" "image:label" /bin/sh -c "${cmd}";
}
Then again, this is not the code I wrote for my particular use case, but a simplification that shows the basic idea. If you can rely on having Bash, then it could be much cleaner, by not overloading $# and using arrays.

Would a "shell function" or "alias" be appropriate for this use

I'm currently trying to create an alias or shell function which I can run to check my battery life, in attempts to familiarize myself with aliases and bash. I have run into a problem where, I'm not receiving any feedback from my command and can not verify if it's working or if there are any steps i have left out that will give me my desired result.
Current .bashrc alias:
alias battery='upower -i $(upower -e | grep -e 'BAT'| grep -E "state|to\ full|percentage")'
Desired use:
b#localhost:~$ battery
Desired result:
state: discharging Time to empty: x.x Hours percentage: xx%
I have read the bash references for something that might help me here. I wasn't able to find anything that I think applies here. Thanks for your consideration!
As #bannji already announced in a comment, he has fixed his command.
Old incorrect alias
'upower -i $(upower -e | grep -e 'BAT'| grep -E "state|to\ full|percentage")'
New correct alias
'upower -i $(upower -e | grep -e "BAT") | grep -E "state|to\ full|percentage"'
Most comments were talking about the interpretation of the quotes. That was not the problem here. The main difference is where the subcommand is closed. In the first case the subcommand is closed after the last grep, su upower -i gets nothing.
In the second command the second grep will filter the output of upower -i.
The difference in quotes is interesting in an other example.
addone() {
((sum=$1+1))
echo "${sum}"
}
i=1
alias battery='addone $(addone $i)'
i=4
battery
# other alias
i=1
alias battery2='addone $(addone '$i')'
i=4
battery2
Both battery commands will try to add 2 to the value of $i, but will give different results.
The command battery will add 2 to the current value 4 of $i, resulting in 6.
The command battery2 will add 2 to the value of $i at the moment that the alias was defined, resulting in 3.
Why?
In battery2 the string $i is surrounded by single quotes, but those single quotes are inside other ones. The result is that $i is evaluated and the alias is defined as
alias battery2='addone $(addone 2)'

How do I use `sed` to alter a variable in a bash script?

I'm trying to use enscript to print PDFs from Mutt, and hitting character encoding issues. One way around them seems to be to just use sed to replace the problem characters: sed -ir 's/[“”]/"/g' {input}
My test input file is this:
“very dirty”
we’re
I'm hoping to get "very dirty" and we're but instead I'm still getting
â\200\234very dirtyâ\200\235
weâ\200\231re
I found a nice little post on printing to PDFs from Mutt that I used as a starting point. I have a bash script that I point to from my .muttrc with set print_command="$HOME/.mutt/print.sh" -- the script currently reads about like this:
#!/bin/bash
input="$1" pdir="$HOME/Desktop" open_pdf=evince
# Straighten out curly quotes
sed -ir 's/[“”]/"/g' $input
sed -ir "s/[’]/'/g" $input
tmpfile="`mktemp $pdir/mutt_XXXXXXXX.pdf`"
enscript --font=Courier8 $input -2r --word-wrap --fancy-header=mutt -p - 2>/dev/null | ps2pdf - $tmpfile
$open_pdf $tmpfile >/dev/null 2>&1 &
sleep 1
rm $tmpfile
It does a fine job of creating a PDF (and works fine if you give it a file as an argument) but I can't figure out how to fix the curly quotes.
I've tried a bunch of variations on the sed line:
input=sed -r 's/[“”]/"/g' $input
$input=sed -ir "s/[’]/'/g" $input
Per the suggestion at Can I use sed to manipulate a variable in bash? I also tried input=$(sed -r 's/[“”]/"/g' <<< $input) and I get an error: "Syntax error: redirection unexpected"
But none manages to actually change $input -- what is the correct syntax to change $input with sed?
Note: I accepted an answer that resolved the question I asked, but as you can see from the comments there are a couple of other issues here. enscript is taking in a whole file as a variable, not just the text of the file. So trying to tweak the text inside the file is going to take a few extra steps. I'm still learning.
On Editing Variables In General
BashFAQ #21 is a comprehensive reference on performing search-and-replace operations in bash, including within variables, and is thus recommended reading. On this particular case:
Use the shell's native string manipulation instead; this is far higher performance than forking off a subshell, launching an external process inside it, and reading that external process's output. BashFAQ #100 covers this topic in detail, and is well worth reading.
Depending on your version of bash and configured locale, it might be possible to use a bracket expression (ie. [“”], as your original code did). However, the most portable thing is to treat “ and ” separately, which will work even without multi-byte character support available.
input='“hello ’cruel’ world”'
input=${input//'“'/'"'}
input=${input//'”'/'"'}
input=${input//'’'/"'"}
printf '%s\n' "$input"
...correctly outputs:
"hello 'cruel' world"
On Using sed
To provide a literal answer -- you almost had a working sed-based approach in your question.
input=$(sed -r 's/[“”]/"/g' <<<"$input")
...adds the missing syntactic double quotes around the parameter expansion of $input, ensuring that it's treated as a single token regardless of how it might be string-split or glob-expanded.
But All That May Not Help...
The below is mentioned because your test script is manipulating content passed on the command line; if that's not the case in production, you can probably disregard the below.
If your script is invoked as ./yourscript “hello * ’cruel’ * world”, then information about exactly what the user entered is lost before the script is started, and nothing you can do here will fix that.
This is because $1, in that scenario, will only contain “hello; ’cruel’ and world” are in their own argv locations, and the *s will have been replaced with lists of files in the current directory (each such file substituted as a separate argument) before the script was even started. Because the shell responsible for parsing the user's command line (which is not the same shell running your script!) did not recognize the quotes as valid at the time when it ran this parsing, by the time the script is running, there's nothing you can do to recover the original data.
Abstract: The way to use sed to change a variable is explored, but what you really need is a way to use and edit a file. It is covered ahead.
Sed
The (two) sed line(s) could be solved with this (note that -i is not used, it is not a file but a value):
input='“very dirty”
we’re'
sed 's/[“”]/\"/g;s/’/'\''/g' <<<"$input"
But it should be faster (for small strings) to use the internals of the shell:
input='“very dirty”
we’re'
input=${input//[“”]/\"}
input=${input//[’]/\'}
printf '%s\n' "$input"
$1
But there is an underlying problem with your script, you are trying to clean an input received from the command line. You are using $1 as the source of the string. Once somebody writes:
./script “very dirty”
we’re
That input is lost. It is broken into shell's tokens and "$1" will be “very only.
But I do not believe that is what you really have.
file
However, you are also saying that the input comes from a file. If that is the case, then read it in with:
input="$(<infile)" # not $1
sed 's/[“”]/\"/g;s/’/'\''/g' <<<"$input"
Or, if you don't mind to edit (change) the file, do this instead:
sed -i 's/[“”]/\"/g;s/’/'\''/g' infile
input="$(<infile)"
Or, if you are clear and certain that what is being given to the script is a filename, like:
./script infile
You can use:
infile="$1"
sed -i 's/[“”]/\"/g;s/’/'\''/g' "$infile"
input="$(<"$infile")"
Other comments:
Then:
Quote your variables.
Do not use the very old `…` syntax, use $(…) instead.
Do not use variables in UPPER case, those are reserved for environment variables.
And (unless you actually meant sh) use a shebang (first line) that targets bash.
The command enscript most definitively requires a file, not a variable.
Maybe you should use evince to open the PS file, there is no need of the step to make a pdf, unless you know you really need it.
I believe that is better use a file to store the output of enscript and ps2pdf.
Do not hide the errors printed by the commands until everything is working as desired, then, just call the script as:
./script infile 2>/dev/null
Or as required to make it less verbose.
Final script.
If you call the script with the name of the file that enscript is going to use, something like:
./script infile
Then, the whole script will look like this (runs both in bash or sh):
#!/usr/bin/env bash
Usage(){ echo "$0; This script require a source file"; exit 1; }
[ $# -lt 1 ] && Usage
[ ! -e $1 ] && Usage
infile="$1"
pdir="$HOME/Desktop"
open_pdf=evince
# Straighten out curly quotes
sed -i 's/[“”]/\"/g;s/’/'\''/g' "$infile"
tmpfile="$(mktemp "$pdir"/mutt_XXXXXXXX.pdf)"
outfile="${tmpfile%.*}.ps"
enscript --font=Courier10 "$infile" -2r \
--word-wrap --fancy-header=mutt -p "$outfile"
ps2pdf "$outfile" "$tmpfile"
"$open_pdf" "$tmpfile" >/dev/null 2>&1 &
sleep 5
rm "$tmpfile" "$outfile"

Sed append to end of line directory

I'm very new to SED and I'm having a hard time trying to append to an end of a directory. What I'm doing involves 2 basic things with sed but for some reason, no changes are made after the script runs. I will show segments of my script
I have a bash script that pulls my home directory from the host and I define the ID variable.
USERNAME="test"
#pull the home directory
dir=$(ssh -n -t $SERVERNAME "echo \$HOME";)
the above example will store /export/home/ID in the dir variable
echo $dir | sed 's/\([/export/home]*\).*/\1/' > olddir
the sed command above stores /export/home/ in the file olddir (takes off the ending)
sed -i 's_/home/$ _\$USERNAME_' olddir
i am now trying to change /export/home/ to /export/home/test using the defined variable with the escaped $.
after the script runs, it still has /export/home/ as the entry in the olddir file.
I'm using the -i to modify the file and I think I'm using the deliminators correctly? what could I be doing wrong? i even took off the $ from the USERNAME variable which didn't do anything. I know I'm missing something small but i just can't figure it out. I really appreciate your time to answer my question.
I think your command line can be modified then redirect output to olddir as follows:
echo $dir | sed 's#\(/export/home\).*#\1#' > olddir
to add the USERNAME VARIABLE
sed -i "s#/export/home#&/$USERNAME#" olddir
After using the provided information and playing around with the code, I found a solution that will work for both /export/home/ID and /home/ID situations.
I used both Xorg and also gniourf_gniourf's suggestions and below is my result.
echo $dir | sed 's_\([/export/home\]*\).*_\1_' > olddir
The above code is the first part of my solution.
I used what Xorg provided for the above code but it looks like i need those [] so i can use the *\ if i have it as echo $dir | sed 's_\(/export/home\).*_\1_' > olddir then the olddir file will contain /export/home/ID/test after the following sed command is used.
Here is the following sed command that I used:
sed -i 's_/home_&/'"$USERNAME"'_' olddir
The above code is the second part of my solution. I figured that I really only need to put focus on /home/ since i'm appending after it. I looked at gniourf_gniourf's comment and used the example where the quotes to isolate $USERNAME with "" that seems to be the way you can tell the script to use $USERNAME as a variable and not just put in the characters $USERNAME.
so after the script is run, depending on the host, I either have export/home/test/ or /home/test/ I can now put either of these into a new variable on the script to use to specify the home directory when creating ID's on remote hosts!
newdir="$(cat olddir)"
Thank you all so much for your help. I wouldn't have been able to figure this out with out your help.
Update: it turns out that the traling / at the end of directory is problem so i found a much easier way to replace the ID in the home directory
newdir="${dir/ID/${USERNAME}}"
i used these if statments for both home directory situations
`if [[ $dir == "/home/dhabinsk" ]]; then
newdir="${dir/dhabinsk/${USERNAME}}"
fi
if [[ $dir == "/export/home/dhabinsk" ]]; then
newdir="${dir/dhabinsk/${USERNAME}}"
fi`
cheers

Is there a way to do multiple shell variable expansions/parameter substitutions?

I have a line in a script that works in zsh but does not work in bash:
SHORTDIR=${${${PWD##*/}//./_dot_}//:/_colon_}
This is basically a short/efficient version of basename $PWD | sed -e 's/\./0/g' -e 's/:/1/g'.
What's the syntax for stringing together variable expansions?
Sadly, the first part of the substitution has to be a parameter name. An alternative sed version would be:
echo $PWD | sed -e 's!.*/!!' -e 'y/.:/01/'
I was hoping that there'd be a better way than
SHORTDIR=${PWD##*/}
SHORTDIR=${SHORTDIR//./_dot_}
SHORTDIR=${SHORTDIR//:/_colon_}
but this is what I'm sticking to.
According to the answers to the question linked by #perreal, bash basically does not allow for expanded variables themselves as a "parameter" and that's why it fails.
Just to throw this out there, many people don't know that multiple assignments can be done in one line. E.g.:
$ a=1234 a=${a:0:3} a=${a/1}
or
SHORTDIR=${PWD##*/} SHORTDIR=${SHORTDIR//./_dot_} SHORTDIR=${SHORTDIR//:/_colon_}

Resources