Sed append to end of line directory - bash

I'm very new to SED and I'm having a hard time trying to append to an end of a directory. What I'm doing involves 2 basic things with sed but for some reason, no changes are made after the script runs. I will show segments of my script
I have a bash script that pulls my home directory from the host and I define the ID variable.
USERNAME="test"
#pull the home directory
dir=$(ssh -n -t $SERVERNAME "echo \$HOME";)
the above example will store /export/home/ID in the dir variable
echo $dir | sed 's/\([/export/home]*\).*/\1/' > olddir
the sed command above stores /export/home/ in the file olddir (takes off the ending)
sed -i 's_/home/$ _\$USERNAME_' olddir
i am now trying to change /export/home/ to /export/home/test using the defined variable with the escaped $.
after the script runs, it still has /export/home/ as the entry in the olddir file.
I'm using the -i to modify the file and I think I'm using the deliminators correctly? what could I be doing wrong? i even took off the $ from the USERNAME variable which didn't do anything. I know I'm missing something small but i just can't figure it out. I really appreciate your time to answer my question.

I think your command line can be modified then redirect output to olddir as follows:
echo $dir | sed 's#\(/export/home\).*#\1#' > olddir
to add the USERNAME VARIABLE
sed -i "s#/export/home#&/$USERNAME#" olddir

After using the provided information and playing around with the code, I found a solution that will work for both /export/home/ID and /home/ID situations.
I used both Xorg and also gniourf_gniourf's suggestions and below is my result.
echo $dir | sed 's_\([/export/home\]*\).*_\1_' > olddir
The above code is the first part of my solution.
I used what Xorg provided for the above code but it looks like i need those [] so i can use the *\ if i have it as echo $dir | sed 's_\(/export/home\).*_\1_' > olddir then the olddir file will contain /export/home/ID/test after the following sed command is used.
Here is the following sed command that I used:
sed -i 's_/home_&/'"$USERNAME"'_' olddir
The above code is the second part of my solution. I figured that I really only need to put focus on /home/ since i'm appending after it. I looked at gniourf_gniourf's comment and used the example where the quotes to isolate $USERNAME with "" that seems to be the way you can tell the script to use $USERNAME as a variable and not just put in the characters $USERNAME.
so after the script is run, depending on the host, I either have export/home/test/ or /home/test/ I can now put either of these into a new variable on the script to use to specify the home directory when creating ID's on remote hosts!
newdir="$(cat olddir)"
Thank you all so much for your help. I wouldn't have been able to figure this out with out your help.
Update: it turns out that the traling / at the end of directory is problem so i found a much easier way to replace the ID in the home directory
newdir="${dir/ID/${USERNAME}}"
i used these if statments for both home directory situations
`if [[ $dir == "/home/dhabinsk" ]]; then
newdir="${dir/dhabinsk/${USERNAME}}"
fi
if [[ $dir == "/export/home/dhabinsk" ]]; then
newdir="${dir/dhabinsk/${USERNAME}}"
fi`
cheers

Related

running sed passing a environment variable

i have this sed command which works running from a shell
/bin/sed -i -e '/^\['"9876"'\]/ r /etc/asterisk/tmp_'"9876"'.txt' /etc/asterisk/sip_peers.include
running it from a sh script it does not work:
#!/bin/bash
peers=/etc/asterisk/sip_peers.include
tmp=/etc/asterisk/tmp_$ext.txt
var=`cat < $tmp`
printenv > /etc/asterisk/zprintenv.txt
echo $tmp $ext > /etc/asterisk/zoutput.txt
/bin/sed -i -e '/^\['"$ext"'\]/ r /etc/asterisk/tmp_'"$ext"'.txt' /etc/asterisk/sip_peers.include
$ext is a environment variable and it is still present when i pass printenv to a text file. Also the $ext variable is passed to output.txt for testing purpose.
sip_peers.include, where the values from /etc/asterisk/tmp_$ext.txt will be inserted after []:
[9876]
qualify=yes
nat=yes
call-limit=4
....
output zoutput.txt:
/etc/asterisk/tmp_9876.txt 9876
snippet from zprintenv.txt:
contacts=1
vm=1
ext=9876
emergency_trunk=
callerid_override=
enduser_company_contacts=0
.....
output $tmp:
;mac=e02f6d613554
;model=spa504G
please take a look at it, i can not find the mistake since hours.
thanks a lot
Quoting OP comment:
It is a run time issue.
The script is called from a gui and the file sip_peers.include is stored after the script is executed. So sed cannot process the non-existing file.
(This seems like a use case that could be interesting for others. So I am making a Q/A pair by quoting answer from OPs comment,
matching a proposed edit to the title, for being found by users with similar problem.)

Bash - extremely simple script redirecting output to file

Disclaimer: I'm very new to bash and for some reason I'm having a very hard time learning this one. The syntax seems very different depending on the website I visit.
I have a simple wrapper script that I want to test if a file is gzipped or not, and if so, to zcat the file to a new temporary file and open it in an editor. Here's part of the script:
if file $FILE | grep -q gzip
then
timestamp=$(date +"%D_%T")
$( zcat $FILE > tmp-$timestamp )
fi
I'm getting an error: "tmp-10/19/15_15:16:41: No such file or directory"
I tried removing the command substitution syntax or putting tmp-$timestamp in double quotes and I get the same error. If I remove the -$timestamp part, then it seems to work fine. Can someone tell me what's going on here? I'm clearing missing something very simple.
tmp-10/19/15_15:16:41 refers to a file named 15_15:16:41 in directory 19 which is a subdirectory of tmp-10. If those directories and subdirectories do not exist, you cannot write to them.
Replace:
timestamp=$(date +"%D_%T")
With:
timestamp=$(date +"%F_%T")
This gives the date without the /.
As an example of this format:
$ date +"%F_%T"
2015-10-19_12:37:05
With %F, the year comes before the month which comes before the day. This means that your files will sort properly. For most people, that is an important advantage over %D.
Revised script
Your script can be simplified to:
if file "$file" | grep -q gzip
then
zcat "$file" > "tmp-$(date +"%F_%T")"
fi
Notes:
It is best practices not to use all caps for your shell variable. The system uses all caps for its variables and you don't want to accidentally overwrite one. Use lower case or mixed case and you'll be safe.
File names, such as $file, should always be in double-quotes. Some day, someone will give you a file name with a space in it and you don't want that to cause your script to fail.
The command substitution $(...) does not belong here. It has been removed.

Bash insert a variable into a backticked command

I need to use this command
/usr/local/bin/mcl find -f .bz2
which returns me this
:???????? '/Cloud Drive/test1.bz2'
:???????? '/Cloud Drive/test2.bz2'
into a BASH script. The problem is that I need the last parameter (.bz2) to be a variable.
I've tried with this
FILENAME=".bz2"
UPLOADED=$(/usr/local/bin/mcl find -f $FILENAME)
# do something with $UPLOADED
But obviously it is not working. After some research on StackOverflow and on the web I have found several ways to do something like that (even using backticks), but still I can't manage to make it work.
What is the correct way to do that?
You mean like this?
uploaded=$(mcl find -f "$FILENAME" | cut -d"'" -f2)
for u in $uploaded; do
echo "$u"
# process "$u"
done
You can try save the following as e.g. ./script.sh
filename="${1:-.bz2}" #<-- your variable as 1st argument, defaults to .bz2
do_my_work() {
local uploaded="$1"
#do whatever you want with the "uploaded"
printf "got:==%s==\n" "$uploaded"
}
while IFS= read -r __mc1path
do
do_my_work "$__mc1path"
done < <(mc1 find -f "$filename" | sed "s/.*'\(.*\)'.*/\1/")
# variable----^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^- keep only stuff inside of quotes
and use it as
./script.sh .bz2 #or anything, defaults to ".bz2"
and will print
got:==/Cloud Drive/test1.bz2==
got:==/Cloud Drive/test2.bz2==
I think you want that :
UPLOADED=`/usr/local/bin/mcl find -f $FILENAME`

Linux shell script delete/restore/trash part 2

Thank you in advance for any help, this is coursework so further reading/ pointers is greatly appreciated.
I asked a question the other day relating to my own delete/trash/restore scripts and I have completed delete and trash as well as giving delete a backup text file for Restore to use later on.
However, instead of giving me errors, the Restore script just kinda stops in the console. Like when I type # ~/Restore -n the cursor skips to the next line without the usual # and I have to close it manually. Likewise without the -n option. The -n option should ask for a new location to restore to, and without it should restore to the files original location.
I'll post my script, see what y'all think.
#!/bin/bash
if [ "$1" == "-n" ]
then cd ~/rubbish
restore= grep $2 ~/store
filename= basename "$restore"
echo "Type the files new location"
read location
location1 = "readlink -f $location"
mv -i $filename "$location1" /$filename
else cd ~/rubbish
restore= grep $2 ~/store
filename= basename "$restore"
mv -i $filename "$location1" $location
fi
so, ~/rubbish is my own created directory to act as a recycle bin and ~/store is my text file which appends the deleted files readlink details on deletion. I can post the whole 3 scripts if necessary?
Many thanks!
If you call ~/Restore -n it will go to the if part and do a grep $2 ~/store. Since there is no parameter $2 it will result in grep ~/store, which tells grep to search for "~/store" in the input coming from standard input.
That's why your script stops and waits for input.
You can either test for a second parameter or enclose $2 in double quotes to make sure grep gets the correct number of parameters. Better yet, do both: 1. test for a second parameter and 2. enclose $2 in double quotes.
Some more points:
Don't put spaces around =
enclose commands in backticks `, if you want to capture the output
And no spaces between directory and filename
So, you should presumably write
restore=`grep "$2" ~/store`
filename=`basename "$restore"`
echo "Type the files new location"
read location
location1=`readlink -f "$location"`
mv -i $filename "$location1/$filename"
I suggest you look at bash info and follow the "Books and Resources".
I wrote one of these quite some time ago which I still use today. I don't have a restore script because I wrote it so that you could open your desktop trash can, right click and select "Restore". In other words it follows the Linux "trash info" standard.
http://wiki.linuxquestions.org/wiki/Scripting#KDE4_Command_Line_Trash_Can

In a small script to monitor a folder for new files, the script seems to be finding the wrong files

I'm using this script to monitor the downloads folder for new .bin files being created. However, it doesn't seem to be working. If I remove the grep, I can make it copy any file created in the Downloads folder, but with the grep it's not working. I suspect the problem is how I'm trying to compare the two values, but I'm really not sure what to do.
#!/bin/sh
downloadDir="$HOME/Downloads/"
mbedDir="/media/mbed"
inotifywait -m --format %f -e create $downloadDir -q | \
while read line; do
if [ $(ls $downloadDir -a1 | grep '[^.].*bin' | head -1) == $line ]; then
cp "$downloadDir/$line" "$mbedDir/$line"
fi
done
The ls $downloadDir -a1 | grep '[^.].*bin' | head -1 is the wrong way to go about this. To see why, suppose you had files named a.txt and b.bin in the download directory, and then c.bin was added. inotifywait would print c.bin, ls would print a.txt\nb.bin\nc.bin (with actual newlines, not \n), grep would thin that to b.bin\nc.bin, head would remove all but the first line leaving b.bin, which would not match c.bin. You need to be checking $line to see if it ends in .bin, not scanning a directory listing. I'll give you three ways to do this:
First option, use grep to check $line, not the listing:
if echo "$line" | grep -q '[.]bin$'; then
Note that I'm using the -q option to supress grep's output, and instead simply letting the if command check its exit status (success if it found a match, failure if not). Also, the RE is anchored to the end of the line, and the period is in brackets so it'll only match an actual period (normally, . in a regular expression matches any single character). \.bin$ would also work here.
Second option, use the shell's ability to edit variable contents to see if $line ends in .bin:
if [ "${line%.bin}" != "$line" ]; then
the "${line%.bin}" part gives the value of $line with .bin trimmed from the end if it's there. If that's not the same as $line itself, then $line must've ended with .bin.
Third option, use bash's [[ ]] expression to do pattern matching directly:
if [[ "$line" == *.bin ]]; then
This is (IMHO) the simplest and clearest of the bunch, but it only works in bash (i.e. you must start the script with #!/bin/bash).
Other notes: to avoid some possible issues with whitespace and backslashes in filenames, use while IFS= read -r line; do and follow #shellter's recommendation about double-quotes religiously.
Also, I'm not very familiar with inotifywait, but AIUI its -e create option will notify you when the file is created, not when its contents are fully written out. Depending on the timing, you may wind up copying partially-written files.
Finally, you don't have any checking for duplicate filenames. What should happen if you download a file named foo.bin, it gets copied, you delete the original, then download a different file named foo.bin. As the script is now, it'll silently overwrite the first foo.bin. If this isn't what you want, you should add something like:
if [ ! -e "$mbedDir/$line" ]; then
cp "$downloadDir/$line" "$mbedDir/$line"
elif ! cmp -s "$downloadDir/$line" "$mbedDir/$line"; then
echo "Eeek, a duplicate filename!" >&2
# or possibly something more constructive than that...
fi

Resources