So I'm writing a simple backup script that when called will either back up a specific file or all the files in my current directory into my backup directory.
This is my script
#!/bin/bash
#verify if $1 is empty, if so, copy all content to backup directory
if [ -z "$1" ]
then
$files=ls
#Looping through files
for file in $files
do
cp $file ../backup/
done
#else copy files specified
else
$files=ls $1
#Looping through files
for file in $files
do
cp $file ../backup/
done
fi
and the only error I'm getting is:
./backup: line 7: =ls: command not found
I'm not sure why the script won't recognize ls as a command. Any ideas?
to assign a variable, you don't need the dollar sign:
files=foo
to save the output of an command to a var, you need do:
files=$(ls)
or
files=$(ls /path/to/some/dir)
I see two mistakes:
When you initialize variable "files" you should not prepend it with "$" symbol
Command ls should be placed in back quotes (`):
This is a short example:
#!/bin/bash
files=`ls`
for file in $files
do
echo "file: $file"
done
You should try putting the ls into ` marks - better, the back quote. Like this:
files=`ls`
Here a little background. Check this page for more information about quotation marks in the shell environment:
The back quote is not used for quoting characters. That character is
used for command substitution, where the characters between them are
executed by the shell and the results is inserted on that line.
Example:
echo the date is `date`
Related
This question already has answers here:
How to loop over files in directory and change path and add suffix to filename
(6 answers)
When to wrap quotes around a shell variable?
(5 answers)
Closed 2 years ago.
I can't find a way to put an entry in read that contains spaces? I want to put directories in the "Enter directory to be cleaned:" read. I think it is reading the spaces as separate variables. Input would be something like /home/user/apple sauce The "cleaning" is just removing special characters from filenames.
#!/bin/bash
read -p "Enter directory to be cleaned: " directory
echo "Current Directory Structure"
/usr/bin/ls -la $directory
read input
if [ "$input" == "y" ]
then
echo "Cleaning files..."
for file in $directory; do mv $file $(echo "$file" | sed -e 's/[^A-Za-z0-9._-]/_/g'); done &
else
stop
fi
Another issue I am facing is the cleanup is repeating the entire directory when it creates the new filename. If I run that for file in *; do mv "$file" $(echo "$file" | sed -e 's/[^A-Za-z0-9._-]/_/g'); done & command in the directory itself, it just creates the new filename. If I specify the directory it writes out the whole directory:
++ sed -e 's/[^A-Za-z0-9._-]/_/g'
++ echo '/home/apples/scratch/test1/test:something?'
+ mv '/home/apples/scratch/test1/test:something?' _home_apples_scratch_test1_test_something_
I want it to just change the filename but having issues. Any help is thankful.
I think it is reading the spaces as separate variables
It does not, as you can easily verify with this:
read -p 'Enter string:' x
echo "Entered: >>>$x<<<"
If you dislike quoting your variables (to avoid word splitting), you may consider switching from bash to Zsh. Where you have to write "$x" in bash, you would simply write $x in Zsh.
Hence, you would have to write
for file in "$directory"
but this would loop just one iteration, with file bound to the content of the variable directory. For looping over the entries in this directory, you would do a
for dirent in "$directory"/*
I'm a newbie in scripting and I have this problem:
I want to use the fuser command to check if a file with that name is present. I want to save the output of the command and later use it to rename the file before loading.
I have tried the following:
#!/bin/bash
file_name='test'
echo 'file_name before: '$file_name
file_name=`fuser FILE_DIR/SD/Test_file_??????????????.dat`
echo 'file_name after: '$file_name'
However, the result of the above code is:
-bash-3.00$ script.sh
file_name before :test
FILE_DIR/SD/Test_file_20180823120345.dat
file_name after:
The output of the command is not getting stored in variable but getting displayed in screen and I can't figure out why!
What I want to do is to store Test_file_20180823120345.dat in the file_name variable, and then remove the timestamp from that and rename the file to Test_file_20180823.dat.
Then after loading the data in the staging table again rename the file to the old file name that we have received and then archive the file with its original name.
Your problems is a missing closing single quote in:
echo 'file_name before: '$file_name
(which is also why you had to unbalance the quotes in echo 'file_name after: '$file_name' on the last line to get the script to run)
Instead, simply double-quote the entire string for echo that will allow expansion of your variable file_name, e.g.
echo "file_name after: $file_name"
Further, avoid using backticks for command substitution, instead use the $(...) form, e.g.
file_name=$(fuser FILE_DIR/SD/Test_file_??????????????.dat)
Or re-written it could be:
#!/bin/bash
file_name='test'
echo "file_name before: $file_name"
file_name=$(fuser FILE_DIR/SD/Test_file_??????????????.dat)
echo "file_name after: $file_name"
(note: your matching with fuser will only match a pattern with 14 characters after Test_file_ followed by .dat -- which may be what you want, but it can probably be written in a cleaner way depending on the possibilities of ??????????????)
While fuser will provide an error, you may also want to get in the habit of validating that your command substitution returned a variable by testing whether file_name is empty after the call to fuser, e.g.
#!/bin/bash
file_name='test'
echo "file_name before: $file_name"
file_name=$(fuser FILE_DIR/SD/Test_file_??????????????.dat)
if [ -z "$file_name" ]; then
echo "error: no file matched pattern" >&2
else
echo "file_name after: $file_name"
fi
Look things over and let me know if you have further questions.
I'm starting to learn how to write shell scripts. I have them all placed in a folder 'personal-scripts' in my home directory. They are starting to add up though. To solve this, I am attempting to create a script that loops over the directory and gives me a brief sentence about what each script does.
For now I can only output the script location and names via:
scriptsinfo
#!/bin/bash
for filename in ~/personal-scripts/*
do
echo $filename
done
Since I don't want to go back and update this file manually, I want to place the about sentence inside each other script either as a comment or string variable that can be evaluated.
How can I read the contents of each other script in the folder and output their string/info about what they do in this script?
You can do that using head command, which prints the first n lines of a file.
test.sh
# this is about line
# date is 14-9-2017
script data
..
..
~# head -n 2 test.sh
# this is about line
# date is 14-9-2017
If you add the description on each second line of your script, (after #!/bin/bash then let use sed -n "2p" $filenamein your script. You can also add a separator between each script:
#!/bin/bash
for filename in ~/personal-scripts/*
do
echo "---------------------"
echo $filename
echo "---------------------"
sed -n "2p" $filename
done
The alternative is to put the description anywhere, in a line starting by e.g # about: then you can grep it:
#!/bin/bash
for filename in ~/personal-scripts/*
do
echo "---------------------"
echo $filename
echo "---------------------"
grep "# about:" $filename | sed 's/# about://'
done
The | sed 's/# about://' is there to keep the description only.
If you have a comment inside all your scripts with a specific pattern such as
#info : This script runs daily
Then you can simply grep for the pattern and append to the name of the script in each line.
for filename in ~/personal-scripts/*
do
echo "$i : $(grep '#info' $i)"
done
I have written a script to change file ownerships based on an input list read in. My script works fine on directories without space in their name. However it fails to change files on directories with space in their name. I also would like to capture the output from the chown command to a file. Could anyone help ?
here is my script in ksh:
#!/usr/bin/ksh
newowner=eg27395
dirname=/home/sas/sastest/
logfile=chowner.log
date > $dir$logfile
command="chown $newowner:$newowner"
for fname in list
do
in="$dirname/$fname"
if [[ -e $in ]]
then
while read line
do
tmp=$(print "$line"|awk '{if (substr($2,1,1) == "/" ) print $2; if (substr($0,1,1) == "/" ) print '})
if [[ -e $tmp ]]
then
eval $command \"$tmp\"
fi
done < $in
else
echo "input file $fname is not present. Check file location in the script."
fi
done
a couple of other errors:
date > $dir$logfile -- no $dir variable defined
to safely read from a file: while IFS= read -r line
But to answer your main concern, don't try to build up the command so dynamically: don't bother with the $command variable, don't use eval, and quote the variable.
chmod "$newowner:$newowner" "$tmp"
The eval is stripping the quotes on this line
command="chown $newowner:$newowner"
In order to get the line to work with spaces you will need to provide backslashed quotes
command="chown \"$newowner:$newowner\""
This way the command that eval actually runs is
chown "$newowner:$newowner"
Also, you probably need quotes around this variable setting, although you'll need to tweak the syntax
tmp="$(print "$line"|awk '{if (substr($2,1,1) == "/" ) print $2; if (substr($0,1,1) == "/" ) print '})"
To capture the output you can add 2>&1 > file.out where file.out is the name of the file ... in order to get it working with eval as you are using it you will need to backslash any special characters much in the same way you need to backslash the double quotes
Your example code suggests that list is a "meta" file: A list of files that each has a list of files to be changed. When you only have one file you can remove the while loop.
When list is a variable with filenames you need echo "${list}"| while ....
It is not completely clear why you sometimes want to start with the third field. It seems that sometimes you have 2 words before the filename and want them to be ignored. Cutting the string on spaces becomes a problem when your filenames have spaces as well. The solution is look for a space followed by a slash: that space is not part of a filename and everything up to that space can be deleted.
newowner=eg27395
# The slash on the end is not really part of the dir name, doesn't matter for most commands
dirname=/home/sas/sastest
logfile=chowner.log
# Add braces, quotes and change dir into dirname
date > "${dirname}/${logfile}"
# Line with command not needed
# Is list an inputfile? It is streamed using "< list" at the end of while []; do .. done
while IFS= read -r fname; do
in="${dirname}/${fname}"
# Quotes are important
if [[ -e "$in" ]]; then
# get the filenames with a sed construction, and give it to chmod with xargs
# The sed construction is made for the situation that in a line with a space followed by a slash
# the filename starts with the slash
# sed is with # to avoid escaping the slashes
# Do not redirect the output here but after the loop.
sed 's#.* /#/#' "${in}" | xargs chmod ${newowner}:${newowner}
else
echo "input file ${fname} is not present. Check file location in the script."
fi
done < list >> "${dirname}/${logfile}"
I am writing a batch-processing script bash that needs to first check to see if a folder exists to know whether or not to run a certain python script that will create and populate the folder. I have done similar things before that do fine with changing the directories and finding directories from a stored variable, but for some reason I am just missing something here.
Here is roughly how the script is working.
if [ -d "$net_output" ]
then
echo "directory exists"
else
echo "directory does not exist"
fi
when I run this script, I usually echo $net_output in the line before to see what it will evaluate to. When the script runs I get my else block of code saying "Directory does not exist", but when I then copy and paste the $net_output directory path that is echoed before into the shell terminal, it changes directories just fine, proving that the directory does in fact exist. I am using Ubuntu 12.04 on a Dell machine.
Thank you in advance for any help that someone can offer. Let me know what additional information I can provide.
The most common cases I've encountered when someone posts a problem like this are the following:
1. The variable contains literal quotes. Bash does not recursively parse quotes, it only parses the "outer" quotes given on the command line.
$ mkdir "/tmp/dir with spaces"
$ var='"/tmp/dir with spaces"'
$ echo "$var"
"/tmp/dir with spaces"
$ [ -d "/tmp/dir with spaces" ]; echo $?
0
$ [ -d "$var" ]; echo $? # equivalent to [ -d '"/tmp/dir with spaces"' ]
1
2. The variable contains a relative path, and the current directory is not what you expected. Check that the value of echo "$PWD" outputs what you expected.
3. The variable was read from a file with dos line endings, CRLF (\r\n). Unix and unix-like systems use just LF (\n) for line endings. If that's the case, the path will contain a CR (\r) at the end. A CR at the end of a line will be "invisible" in a terminal. Check with printf '%q\n' "$var" while debugging the script. See BashFAQ 52 on how to get rid of them.