I am going through a setup script that I am attempting to understand; how the sed line works, in this instance. From my understanding, it is editing the src/conf-cc inline at the first line and appending -include /usr/include/errno.h/ to the last line of input? I have been referencing the sed manual to help me break this sed command down.
#!/usr/bin/env bash
# A script which installs daemontools
#
# Run as root!
#
if [ "$(id -u)" != "0" ]; then
echo "You must be root!" 1>&2
exit 1
fi
mkdir /package
chmod 1755 /package
cd /package
wget http://cr.yp.to/daemontools/daemontools-0.76.tar.gz
tar -xpf daemontools-0.76.tar.gz
rm -f daemontools-0.76.tar.gz
cd admin/daemontools-0.76
sed -i '1s/$/ -include \/usr\/include\/errno.h/' src/conf-cc
package/install
echo -e "start on runlevel [3] \nrespawn \nexec /command/svscanboot" >> /etc/init/svscan.conf
initctl reload-configuration
initctl start svscan
mkdir /var/svc.d
No, it just appends something to the first line. It's a substitution command:
addr s/pattern/replacement/
where addr is 1 (first line), pattern is $ (regex: end of line) and the replacement is the -include ... string. It's not really "replacing" anything as $ has zero width anyway.
Your misunderstanding is interpreting $ as an address instead of a regular expression.
Related
I would like to convert and execute
if egrep -r 'my_pattern' ./template_builder
then exit 1
elif egrep -r 'my_second_pattern' ./template_builder
then exit 1
fi
in a Makefile, without success for now.
To build this:
cd /tmp;
mkdir template_builder;
echo "not_pattern" >> ./template_builder/test.txt
# Do the command at the top, nothing happens
echo "my_pattern" >> ./template_builder/test.txt
# Do the command at the top, terminal stops
touch Makefile
In a Makefile, I thought this would work :
check:
if egrep -r 'my_pattern' ./template_builder
then exit 1
elif egrep -r 'my_second_pattern' ./template_builder
then exit 1
fi
make check
if egrep -r 'my_pattern' ./template_builder
/bin/sh: -c: line 1: syntax error: unexpected end of file
make: *** [template] Error 2
How can I fix this?
Your attempt was not far from working!
Add backslashes at the end of every line, and ;s as explicit command separators (and of course use real tabs instead of the 8-space indents below):
check:
if egrep -r 'my_pattern' ./template_builder; \
then exit 1; \
elif egrep -r 'my_second_pattern' ./template_builder; \
then exit 1; \
fi
If I understand you correctly, if the directory template_builder located in /tmp does not contain a file matching the string 'my_pattern' or 'my_second_pattern', you want to exit from make with an error code.
You can achieve this with this rule in Makefile:
check:
egrep -r -v 'my_pattern' /tmp/template_builder || egrep -r -v 'my_second_pattern' /tmp/template_builder
Explanation: the first egrep is going to return an error in the case he finds a match. Due to the presence of the || operator, the second egrep will be invoked. The result of this second command will be the result that make will see. If it returns an error, the execution of make is aborted, which seems to be the behaviour you are expecting.
Caution: I edited my answer. The right boolean operator is || and not &&.
As others have already noted, make runs each separate line in a recipe in a new shell subprocess. (For the record, it uses sh out of the box, not Bash.) The trivial fix is to add a backslash to escape the newline at the end of each line which should be executed in the same shell as the next one. (You need to add a semicolon as well in some places, like before then and else and fi.) But you really want to refactor to use the facilities and idioms of make.
The default logic of make is to terminate the recipe if any line fails. So, your code can be reduced to simply
check: template_builder
! egrep -r 'my_pattern' $<
! egrep -r 'my_second_pattern' $<
The explicit exit 1 is not necessary here (negating a zero exit code produces exactly that); but if you wanted to force a particular exit code, you could do that with
egrep -r 'my_pattern' $< && exit 123 || true
Modern POSIX prefers grep -E over legacy egrep; of course, with these simple patterns, you can use just grep, or even grep -F (née fgrep).
Moreover, if you want to search for both patterns in the same set of files, it's much more efficient to search for them both at once.
check: template_builder
! egrep -e 'my_pattern' -e 'my_second_pattern' -r $<
... or combine them into a single regex my_(second_)?pattern (which requires egrep / grep -E).
Notice also how I factored out the dependency into $< and made it explicit; but you probably want to make this recipe .PHONY anyway, so that it gets executed even if nothing has changed.
(You can't directly copy/paste this code, because Stack Overflow stupidly renders the literal tabs in the markdown source as spaces.)
Hey guys I'm trying to run this code:
#!/bin/bash
sudo /usr/local/bin/sshpass -p pwd ssh -o stricthostkeychecking=no -p 11022 admin#$1.test.com<<EOI
i=1
while read line
do
location="sudo sed -n ${i}p /Users/Shared/$1.txt"
number="sudo sed -n ${i}p /Users/Shared/$2n.txt"
my_array=("${my_array[i]}" $line)
sudo cp /Applications/TEPS\ OS\ X\ Share\ Folder/MAIN\ IMAGES\ FOLDER\ ƒ/${location}${number} /Users/Shared/FYP/$number
sudo sips -Z 256 /Users/Shared/FYP/$number /Users/Shared/FYP/$number
((i++))
done </Users/Shared/$2.txt
exit
EOI
basically it reads a text file which gives the location of certain images, and will create a thumbnail of those images, which can be downloaded later. The problem is that I need the value of $i to set the values of $location and $number, but when I set the variable within the while loop the variables are not set. I've tried setting it locally and globally with single quotes, double quotes, passing through with the sshpass, exporting it -This works as a test but $i is of course unknown- tried placing brackets, curly braces, parentheses, escaping $, at this point I have exhausted my ideas, it's probably something incredibly simple, but I could use a fresh pair of eyes, any help is greatly appreciated!
EDIT:
Thanks to Charles Duffy for helping me clean it up so this is what I have now:
#!/bin/bash
sudo /usr/local/bin/sshpass -p support ssh -o stricthostkeychecking=no -p 11022 admin#$1.noerrpole.com<<'EOI'
i=1
while read -r line
do
location=sudo sed -n ${i}p "/Users/Shared/$1.txt"
number=sudo sed -n ${i}p "/Users/Shared/$2n.txt"
my_array+=( "$line" )
sudo cp "/Applications/TEPS\ OS\ X\ Share\ Folder/MAIN\ IMAGES\ FOLDER\ ƒ/${location}${number}" "/Users/Shared/FYP/$number"
sudo sips -Z 256 "/Users/Shared/FYP/$number" "/Users/Shared/FYP/$number"
((i++))
exit
done <"/Users/Shared/$2.txt"
EOI
But now $2 isn't getting passed through to the loop here's what I get back
1:bin Photo$ bash -x thumb npco2 20131216154714
+ sudo /usr/local/bin/sshpass -p support ssh -o stricthostkeychecking=no -p 11022 admin#npco2.noerrpole.com
Pseudo-terminal will not be allocated because stdin is not a terminal.
SHPA_12-16-2013/
sed: /Users/Shared/n.txt: No such file or directory
cp: /Applications/TEPS OS X Share Folder/MAIN IMAGES FOLDER ƒ/ is a directory (not copied).
Warning: /Users/Shared/FYP/ not a valid file - skipping
Warning: /Users/Shared/FYP/ not a valid file - skipping
Error 4: no file was specified
Try 'sips --help' for help using this tool
So where $2 should equal 20131216154714 it's returning an empty string like this
sed: /Users/Shared/n.txt: No such file or directory
The correct command would be
sed: /Users/Shared/20131216154714n.txt
The rest is just failing because $2 isn't passed.
Again thanks for the help!
ssh ... <<EOI does expansion on the local end, before starting ssh. Use ssh ... <<'EOI' to do expansions on the remote end.
If you want to pass arguments, use printf '%q ' to quote them so they survive remote unescaping intact:
printf -v quoted_args '%q ' "$one" "$two"
ssh user#host "bash -s - ${quoted_args}" <<<'EOI'
...
EOI
I have a set of scripts that I use to download files via FTP and then delete them from the server.
It works as follows:
for dir in `ls /volume1/auto_downloads/sync-complete`
do
if [ "x$dir" != *"x"* ]
then
echo "DIR: $dir"
echo "Moving out of complete"
# Soft delete from server so they don't get downloaded again
ssh dan#172.19.1.15 mv -v "'/home/dan/Downloads/complete/$dir'" /home/dan/Downloads/downloaded
Now $dir could be "This is a file" which works fine.
The problem I'm having is with special characters eg:
"This is (a) file"
This is a file & stuff"
tend to error:
bash: -c: line 0: syntax error near unexpected token `('
bash: -c: line 0: `mv -v '/home/dan/Downloads/complete/This is (a) file' /home/dan/Downloads/downloaded'
I can't work out how to escape it so both the variable gets evaluated and the command gets escaped properly. I've tried various combinations of escape characters, literal quotes, normal quotes, etc
If both sides are using bash, you can escape the arguments using printf '%q ', eg:
ssh dan#172.19.1.15 "$(printf '%q ' mv -v "/home/dan/Downloads/complete/$dir" /home/dan/Downloads/downloaded)"
You need to quote the whole expression ssh user#host "command":
ssh dan#172.19.1.15 "mv -v /home/dan/Downloads/complete/$dir /home/dan/Downloads/downloaded"
I'm confused, because your code as written works for me:
> dir='foo & bar (and) baz'
> ssh host mv -v "'/home/dan/Downloads/complete/$dir'" /home/dan/Downloads/downloaded
mv: cannot stat `/home/dan/Downloads/complete/foo & bar (and) baz': No such file or directory
For debugging, use set -vx at the top of the script to see what's going on.
Will Palmer's suggestion of using printf is great but I think it makes more sense to put the literal parts in printf's format.
That way, multi-command one-liners are more intuitive to write:
ssh user#host "$(printf 'mkdir -p -- %q && cd -- "$_" && tar -zx' "$DIR")"
One can use python shlex.quote(s) to
Return a shell-escaped version of the string s
docs
I have two scripts (one ksh and other Perl) and one calls the other. I have to handle a scenario when someone accidentally enters a white space in file name and report it as an error (only when the file does not exist). It looks like p.sh which uses $* to pass/forward all arguments to p.pl doesn't handle quoted arguments the way they should be? Any ideas how to fix this? Let's just say one could enter multiple spaces in the filename too.
p.sh:
#!/usr/bin/env ksh
/tmp/p.pl $* 1>/tmp/chk.out 2>&1
print "Script exited with value $?"
print "P.PL OUTPUT:"
cat /tmp/chk.out
exit 0
p.pl:
#!/usr/bin/perl -w
use Getopt::Std;
getopts ("i:", \ %options);
if ($options{i} && -e $options{i}) {
print "File $options{i} Exists!\n";
}
else {
print "File $options{i} DOES NOT exist!\n";
}
Test cases (when there is an actual file '/tmp/a b.txt' (with a space in it) on the system):
[test] /tmp $ p.pl -i /tmp/a b.txt
File /tmp/a DOES NOT exist!
[test] /tmp $ p.pl -i "/tmp/a b.txt"
File /tmp/a b.txt Exists!
[test] /tmp $ ./p.sh -i "/tmp/a b.txt"
Script exited with value 0
P.PL Check OUTPUT:
File /tmp/a DOES NOT exist!
[test] /tmp $ ./p.sh -i "/tmp/ a b.txt"
Script exited with value 0
P.PL Check OUTPUT:
File /tmp/ Exists!
It's the last two scenarios I'm trying to fix. Thank you.
To preserve whitespace that was passed into the script, use the $# parameter:
/tmp/p.pl "$#" 1>/tmp/chk.out 2>&1
The quotation marks are necessary to make sure that quoted whitespace is seen by p.pl.
This work is being done on a test virtualbox machine
In my /root dir, i have created the following:
"/root/foo"
"/root/bar"
"/root/i have multiple words"
Here is the (relevant)code I currently have
if [ ! -z "$BACKUP_EXCLUDE_LIST" ]
then
TEMPIFS=$IFS
IFS=:
for dir in $BACKUP_EXCLUDE_LIST
do
if [ -e "$3/$dir" ] # $3 is the backup source
then
BACKUP_EXCLUDE_PARAMS="$BACKUP_EXCLUDE_PARAMS --exclude='$dir'"
fi
done
IFS=$TEMPIFS
fi
tar $BACKUP_EXCLUDE_PARAMS -cpzf $BACKUP_PATH/$BACKUP_BASENAME.tar.gz -C $BACKUP_SOURCE_DIR $BACKUP_SOURCE_TARGET
This is what happens when I run my script with sh -x
+ IFS=:
+ [ -e /root/foo ]
+ BACKUP_EXCLUDE_PARAMS= --exclude='foo'
+ [ -e /root/bar ]
+ BACKUP_EXCLUDE_PARAMS= --exclude='foo' --exclude='bar'
+ [ -e /root/i have multiple words ]
+ BACKUP_EXCLUDE_PARAMS= --exclude='foo' --exclude='bar' --exclude='i have multiple words'
+ IFS=
# So far so good
+ tar --exclude='foo' --exclude='bar' --exclude='i have multiple words' -cpzf /backup/root/daily/root_20130131.071056.tar.gz -C / root
tar: have: Cannot stat: No such file or directory
tar: multiple: Cannot stat: No such file or directory
tar: words': Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
# WHY? :(
The Check completes sucessfully, but the --exclude='i have multiple words' does not work.
Mind you that it DOES work when i type it in my shell, manually:
tar --exclude='i have multiple words' -cf /somefile.tar.gz /root
I know that this would work in bash when using arrays, but i want this to be POSIX.
Is there a solution to this?
Consider this scripts; ('with whitespace' and 'example.desktop' is sample files)
#!/bin/bash
arr=("with whitespace" "examples.desktop")
for file in ${arr[#]}
do
ls $file
done
This outputs as exactly as yours;
21:06 ~ $ bash test.sh
ls: cannot access with: No such file or directory
ls: cannot access whitespace: No such file or directory
examples.desktop
You can set IFS to '\n' character to escape white spaces on file names.
#!/bin/bash
arr=("with whitespace" "examples.desktop")
(IFS=$'\n';
for file in ${arr[#]}
do
ls $file
done
)
the output of the second version should be;
21:06 ~ $ bash test.sh
with whitespace
examples.desktop
David the H. from the LinuxQuestions forums steered me in the right direction.
First of all, in my question, I did not make use IFS=: all the way through to the tar command
Second of all, I included "set -f" for safety
BACKUP_EXCLUDE_LIST="foo:bar:i have multiple words"
# Grouping our parameters
if [ ! -z "$BACKUP_EXCLUDE_LIST" ]
then
IFS=: # Here we set our temp $IFS
set -f # Disable globbing
for dir in $BACKUP_EXCLUDE_LIST
do
if [ -e "$3/$dir" ] # $3 is the directory that contains the directories defined in $BACKUP_EXCLUDE_LIST
then
BACKUP_EXCLUDE_PARAMS="$BACKUP_EXCLUDE_PARAMS:--exclude=$dir"
fi
done
fi
# We are ready to tar
tar $BACKUP_EXCLUDE_PARAMS \
-cpzf "$BACKUP_PATH/$BACKUP_BASENAME.tar.gz" \
-C "$BACKUP_SOURCE_DIR" \
"$BACKUP_SOURCE_TARGET"
unset IFS # our custom IFS has done it's job. Let's unset it!
set +f # Globbing is back on
I advise against using the TEMPIFS variable, like I did, because that method does not set the IFS back correctly. It's best to unset IFS when you are done with it