ksh: How to pass arguments containing white space between scripts? - shell

I have two scripts (one ksh and other Perl) and one calls the other. I have to handle a scenario when someone accidentally enters a white space in file name and report it as an error (only when the file does not exist). It looks like p.sh which uses $* to pass/forward all arguments to p.pl doesn't handle quoted arguments the way they should be? Any ideas how to fix this? Let's just say one could enter multiple spaces in the filename too.
p.sh:
#!/usr/bin/env ksh
/tmp/p.pl $* 1>/tmp/chk.out 2>&1
print "Script exited with value $?"
print "P.PL OUTPUT:"
cat /tmp/chk.out
exit 0
p.pl:
#!/usr/bin/perl -w
use Getopt::Std;
getopts ("i:", \ %options);
if ($options{i} && -e $options{i}) {
print "File $options{i} Exists!\n";
}
else {
print "File $options{i} DOES NOT exist!\n";
}
Test cases (when there is an actual file '/tmp/a b.txt' (with a space in it) on the system):
[test] /tmp $ p.pl -i /tmp/a b.txt
File /tmp/a DOES NOT exist!
[test] /tmp $ p.pl -i "/tmp/a b.txt"
File /tmp/a b.txt Exists!
[test] /tmp $ ./p.sh -i "/tmp/a b.txt"
Script exited with value 0
P.PL Check OUTPUT:
File /tmp/a DOES NOT exist!
[test] /tmp $ ./p.sh -i "/tmp/ a b.txt"
Script exited with value 0
P.PL Check OUTPUT:
File /tmp/ Exists!
It's the last two scenarios I'm trying to fix. Thank you.

To preserve whitespace that was passed into the script, use the $# parameter:
/tmp/p.pl "$#" 1>/tmp/chk.out 2>&1
The quotation marks are necessary to make sure that quoted whitespace is seen by p.pl.

Related

loop in bash for parameters that uses multiple sources

I have a bash script which works like this;
File structure;
get.sh
loop.sh
config/param1.conf
config/param2.conf
Usage of the main script, get.sh;
./get.sh <param> i.e ./get.sh param1, ./get.sh param2
So when you run the script with specific params it fetches the config files from config/<param>.conf
What I'm trying to do is to run this second script, ./loop.sh so it runs the ./get.sh <param> for you in a loop using the params inside config folder, without .conf extensions.
Here's my loop.sh;
#!/bin/bash
# run the script with the first param you found inside ~/config/
# folder without including it's .conf extension,
# wait for 5 seconds and then do the same with the 2nd param you found
for i in $(find ~/config -name '*.conf'); do
./get.sh $(basename $i) | cut -d'.' -f 1
sleep 5
done
but this one is just displaying the params inside config folder and doing nothing else.
`
Inside of the config/param1.conf;
var=Hello1
Inside of the config/param2.conf;
var=Hello2
Inside of the get.sh;
#!/bin/bash
function testFunction {
echo "$var"
}
cfg_file=$1
if [ -f "$cfg_file" ]; then
. "$cfg_file"
testFunction $1
exit 1
else
echo "$1.conf doesn't exist"
exit 1
fi
So after all, when you run the loop.sh, the expected behavior should be printing the Hello1 and Hello2 strings into shell.
How can I fix this?
In loop (using .sh extensions for bash scripts is not great practice):
#!/bin/bash
for i in ~/config/*.conf; do
i_basename=${i##*/} # change ~/config/foo.conf to just foo.conf
i_basename=${i_basename%.conf} # change foo.conf to just foo
./get "$i_basename"
sleep 5
done
The ${var##prefix} and ${var%suffix} syntax is parameter expansion; with ## it removes the longest match from the beginning (so for */, everything up to the last /); with %, it removes the shortest successful match starting from the end.
In get:
#!/bin/bash
# Using POSIX function syntax; see http://wiki.bash-hackers.org/scripting/obsolete
testFunction() {
echo "$var"
}
cfg_file="config/$1.conf"
if [ -f "$cfg_file" ]; then
. "$cfg_file"
testFunction
else
echo "$cfg_file doesn't exist"
exit 1
fi
With respect to lack of .sh extensions -- UNIX commands don't conventionally have extensions (you run ls, not ls.elf); similarly, when you install a Python module built with setuptools, it doesn't put .py extensions on shims it creates in /usr/local/bin, even if the libraries those executable shims invoke do have such extensions. Moreover, bash and POSIX sh are two different languages: Bash scripts often don't work correctly when started with sh some-bash-only-script.sh (as unlike bash, sh isn't guaranteed to support language features like arrays), but the extension implies that they will.
-name expects the parameter to be just a filename, not a whole pathname. You should use the directory as a regular argument to find, not as part of -name.
for i in $(find ~/config -name '*.conf'); do
Don't use basename, you should pass the entire pathname to script.sh.
Then in script.sh you should should use $1 as the whole path to the config file, rather than concatenating it with a directory prefix.
cfg_file=$1
I don't see any point in this:
case $1 in
$1) cfg=$1 ;;
esac
The case will always be true, how can $1 not match $1? Remove that code.

Does the sed command just append text inline to the first line?

I am going through a setup script that I am attempting to understand; how the sed line works, in this instance. From my understanding, it is editing the src/conf-cc inline at the first line and appending -include /usr/include/errno.h/ to the last line of input? I have been referencing the sed manual to help me break this sed command down.
#!/usr/bin/env bash
# A script which installs daemontools
#
# Run as root!
#
if [ "$(id -u)" != "0" ]; then
echo "You must be root!" 1>&2
exit 1
fi
mkdir /package
chmod 1755 /package
cd /package
wget http://cr.yp.to/daemontools/daemontools-0.76.tar.gz
tar -xpf daemontools-0.76.tar.gz
rm -f daemontools-0.76.tar.gz
cd admin/daemontools-0.76
sed -i '1s/$/ -include \/usr\/include\/errno.h/' src/conf-cc
package/install
echo -e "start on runlevel [3] \nrespawn \nexec /command/svscanboot" >> /etc/init/svscan.conf
initctl reload-configuration
initctl start svscan
mkdir /var/svc.d
No, it just appends something to the first line. It's a substitution command:
addr s/pattern/replacement/
where addr is 1 (first line), pattern is $ (regex: end of line) and the replacement is the -include ... string. It's not really "replacing" anything as $ has zero width anyway.
Your misunderstanding is interpreting $ as an address instead of a regular expression.

Why am I not finding any output files in the desired location?

I am trying to write a processing script and I am stuck at the beginning. It does not seem to be wrong but I cannot simply understand where the error is as it is completing the execution but not giving any output. Any debugging help?
#!/bin/sh
#
# Call with following arguments
# sh test.sh <output_basename> <fastq folder> <output_folder_loc>
#
#
bn=$1
floc=$2
outloc=$3
#Create an output directory
#opdir=$bn"_processed"
mkdir $outloc/$bn"_processed"
echo "output directory for fastq $outloc/$bn"_processed" ..."
fout=$outloc/$bn"_processed"
echo "$fout ..."
echo "performing assembly to create one fastq file for each read mates ..."
zcat $floc/*R1*.fastq.gz > $fout/$bn_R1.fastq
zcat $floc/*R2*.fastq.gz > $fout/$bn_R2.fastq
echo "done"
Run command:
sh test.sh S_13_O1_122 /home/vdas/data/floc/Sample_S_13_O1_122_S12919 /home/vdas/data/OC/RNA-Seq/STAR_run/mut_out
I do not see any wrong in the code and it is also runnning without error but still am not getting any output. Can anyone point me the problem?
First try to change two lines like this:
mkdir -p "$outloc/${bn}_processed"
fout="$outloc/${bn}_processed"
mkdir -p is good when $outloc directory doesn't exist yet.
you could test your arguments (the following may be only in bash, but do work when bash is invoked as /bin/sh)
var=$1
if [ ${#var} -eq 0 ]; then
echo "var is not defined" >&2
exit 1
fi
that will test that the variable has some length, you might want to test other aspects as well, for instance does
ls $floc/*R1*.fastq.gz
produce any output?
#!/bin/sh
#
# Call with following arguments
# sh test.sh <output_basename> <fastq folder> <output_folder_loc>
#
#
bn=$1
floc=$2
outloc=$3
#Create an output directory
#opdir=$bn"_processed"
mkdir $outloc/$bn"_processed"
echo "output directory for fastq $outloc/$bn"_processed" ..."
fout=$outloc/$bn"_processed"
echo "$fout ..."
echo "performing assembly to create one fastq file for each read mates ..."
echo $floc/*R1*.fastq.gz
echo $fout/$bn_R1.fastq
zcat -v $floc/*R1*.fastq.gz > $fout/${bn}_R1.fastq
zcat -v $floc/*R2*.fastq.gz > $fout/${bn}_R2.fastq
echo "done"
`
this may be wath you want,

Passing a path as an argument to a shell script

I've written bash script to open a file passed as an argument and write it into another file. But my script will work properly only if the file is in the current directory. Now I need to open and write the file that is not in the current directory also.
If compile is the name of my script, then ./compile next/123/file.txt should open the file.txt in the passed path. How can I do it?
#!/bin/sh
#FIRST SCRIPT
clear
echo "-----STARTING COMPILATION-----"
#echo $1
name=$1 # Copy the filename to name
find . -iname $name -maxdepth 1 -exec cp {} $name \;
new_file="tempwithfile.adb"
cp $name $new_file #copy the file to new_file
echo "compiling"
dir >filelist.txt
gcc writefile.c
run_file="run_file.txt"
echo $name > $run_file
./a.out
echo ""
echo "cleaning"
echo ""
make clean
make -f makefile
./semantizer -da <withfile.adb
Your code and your question are a bit messy and unclear.
It seems that you intended to find your file, given as a parameter to your script, but failed due to the maxdepth.
If you are given next/123/file.txt as an argument, your find gives you a warning:
find: warning: you have specified the -maxdepth option after a
non-option argument -iname, but options are not positional (-maxdepth
affects tests specified before it as well as those specified after
it). Please specify options before other arguments.
Also -maxdepth gives you the depth find will go to find your file until it quits. next/123/file.txt has a depth of 2 directories.
Also you are trying to copy the given file within find, but also copied it using cp afterwards.
As said, your code is really messy and I don't know what you are trying to do. I will gladly help, if you could elaborate :).
There are some questions that are open:
Why do you have to find the file, if you already know its path? Do you always have the whole path given as an argument? Or only part of the path? Only the basename ?
Do you simply want to copy a file to another location?
What does your writefile.c do? Does it write the content of your file to another? cp does that already.
I also recommend using variables with CAPITALIZED letters and checking the exit status of used commands like cp and find, to check if these failed.
Anyway, here is my script that might help you:
#!/bin/sh
#FIRST SCRIPT
clear
echo "-----STARTING COMPILATION-----"
echo "FILE: $1"
[ $# -ne 1 ] && echo "Usage: $0 <file>" 1>&2 && exit 1
FILE="$1" # Copy the filename to name
FILE_NEW="tempwithfile.adb"
cp "$FILE" "$FILE_NEW" # Copy the file to new_file
[ $? -ne 0 ] && exit 2
echo
echo "----[ COMPILING ]----"
echo
dir &> filelist.txt # list directory contents and write to filelist.txt
gcc writefile.c # ???
FILE_RUN="run_file.txt"
echo "$FILE" > "$FILE_RUN"
./a.out
echo
echo "----[ CLEANING ]----"
echo
make clean
make -f makefile
./semantizer -da < withfile.adb

Dash variable expansion does not work in some cases

This work is being done on a test virtualbox machine
In my /root dir, i have created the following:
"/root/foo"
"/root/bar"
"/root/i have multiple words"
Here is the (relevant)code I currently have
if [ ! -z "$BACKUP_EXCLUDE_LIST" ]
then
TEMPIFS=$IFS
IFS=:
for dir in $BACKUP_EXCLUDE_LIST
do
if [ -e "$3/$dir" ] # $3 is the backup source
then
BACKUP_EXCLUDE_PARAMS="$BACKUP_EXCLUDE_PARAMS --exclude='$dir'"
fi
done
IFS=$TEMPIFS
fi
tar $BACKUP_EXCLUDE_PARAMS -cpzf $BACKUP_PATH/$BACKUP_BASENAME.tar.gz -C $BACKUP_SOURCE_DIR $BACKUP_SOURCE_TARGET
This is what happens when I run my script with sh -x
+ IFS=:
+ [ -e /root/foo ]
+ BACKUP_EXCLUDE_PARAMS= --exclude='foo'
+ [ -e /root/bar ]
+ BACKUP_EXCLUDE_PARAMS= --exclude='foo' --exclude='bar'
+ [ -e /root/i have multiple words ]
+ BACKUP_EXCLUDE_PARAMS= --exclude='foo' --exclude='bar' --exclude='i have multiple words'
+ IFS=
# So far so good
+ tar --exclude='foo' --exclude='bar' --exclude='i have multiple words' -cpzf /backup/root/daily/root_20130131.071056.tar.gz -C / root
tar: have: Cannot stat: No such file or directory
tar: multiple: Cannot stat: No such file or directory
tar: words': Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
# WHY? :(
The Check completes sucessfully, but the --exclude='i have multiple words' does not work.
Mind you that it DOES work when i type it in my shell, manually:
tar --exclude='i have multiple words' -cf /somefile.tar.gz /root
I know that this would work in bash when using arrays, but i want this to be POSIX.
Is there a solution to this?
Consider this scripts; ('with whitespace' and 'example.desktop' is sample files)
#!/bin/bash
arr=("with whitespace" "examples.desktop")
for file in ${arr[#]}
do
ls $file
done
This outputs as exactly as yours;
21:06 ~ $ bash test.sh
ls: cannot access with: No such file or directory
ls: cannot access whitespace: No such file or directory
examples.desktop
You can set IFS to '\n' character to escape white spaces on file names.
#!/bin/bash
arr=("with whitespace" "examples.desktop")
(IFS=$'\n';
for file in ${arr[#]}
do
ls $file
done
)
the output of the second version should be;
21:06 ~ $ bash test.sh
with whitespace
examples.desktop
David the H. from the LinuxQuestions forums steered me in the right direction.
First of all, in my question, I did not make use IFS=: all the way through to the tar command
Second of all, I included "set -f" for safety
BACKUP_EXCLUDE_LIST="foo:bar:i have multiple words"
# Grouping our parameters
if [ ! -z "$BACKUP_EXCLUDE_LIST" ]
then
IFS=: # Here we set our temp $IFS
set -f # Disable globbing
for dir in $BACKUP_EXCLUDE_LIST
do
if [ -e "$3/$dir" ] # $3 is the directory that contains the directories defined in $BACKUP_EXCLUDE_LIST
then
BACKUP_EXCLUDE_PARAMS="$BACKUP_EXCLUDE_PARAMS:--exclude=$dir"
fi
done
fi
# We are ready to tar
tar $BACKUP_EXCLUDE_PARAMS \
-cpzf "$BACKUP_PATH/$BACKUP_BASENAME.tar.gz" \
-C "$BACKUP_SOURCE_DIR" \
"$BACKUP_SOURCE_TARGET"
unset IFS # our custom IFS has done it's job. Let's unset it!
set +f # Globbing is back on
I advise against using the TEMPIFS variable, like I did, because that method does not set the IFS back correctly. It's best to unset IFS when you are done with it

Resources