I need help trying to find a solution to have a bash script be able to read file names with special characters. The user will start the script, but if the folder or the file has special characters, the script will fail or have an error. I have tried several options I found online, but I have not been able to make them work with the script.
The script is set up to take user input with the read command.
read -r -p "Enter directory name : " var1
If the user input is “accoutn&orders,” the script will fail due to the ‘&’ character as it won’t find the directory or file.
When the script looks for the file with specific extensions, the input folder name will be the path to copy the files to a different directory. The issue I am running into is that some of those files or directories have special characters, and the script cannot process the variables and cannot find the file when there are special characters.
The script uses a for loop to check every file in the directory, and if the file's name has a special character, it will fail the loop.
example file name:
file1#depot.rct
file2&logrecord.rct
cd $var1
ls: cannot access '/sharepool/comunityshare//'\''account.&.orders'\''': No such file or directory
line 141: cd: '/sharepool/comunityshare//'\''account.&.orders'\''': No such file or directory
I have tried using single quotes wrapping and bask slashes, but the variable is not readable.
Please note that I am not a coder or developer, I know some basic Linux commands, and I am trying to make this work while a better process is developed. I appreciate your help with this.
I was able to solve the issue using this line.
filename=$(echo "$filename" | sed 's/[&()#+*#!%^'\''^]/\\&/g')
That inserted a backslash if the variable had a special character.
account.&.orders to account.&.orders
Thank you for your help and support.
Related
I am new to UNIX and have a homework assignment that is giving me trouble. I am to write a script that will back up specified files from the current directory into a specified destination directory. This script is to take three arguments.
sourcePath, which is the path to the source files/files being backed up or copied.
backupPath, which is the path to the target directory where the files will be backed up.
filePrefix, which is used to identify which files to backup, specifically only files whose names begin with the given prefix will be copied while others will be ignored. Example would be, if the user enters the letter "d", then all files starting with that letter are to be copied while any other file is to be ignored.
I haven't learned much about scripting/functions in bash so I've tried looking up tutorials which have been helpful but not enough. This script is something I can easily do when just typing out the commands. For instance, I would cd into the target directory that has the files, then using the cp command copy files that begin with the specific prefix to the target directory, but when making a script I am at a dead end.
I feel as though my code is monumentally incorrect and its due to my lack of experience, but nothing online has been of any help. So far my code is
read sourcePath
read backupPath
read filePrefix
grep /export/home/public/"$sourcePath
mkdir -p $backupPath
cp /export/home/public/"$sourcePath"/$filePrefix /home/public/"$backupPath"
So an example execution of the script would be
$ ./script.sh
(sourcePath)HW4testdir (backupPath)backup (filePrefix)d
Output:
backing up: def (example file starting with d)
backing up: dog (example file starting with d)
So far when executing the code, nothing happens. Again, I'm sure most, or even all of the code is wrong and totally off base, but I never learned about scripting. If I did not have to create a script, I could easily achieve this desired outcome.
I suggest with bash:
read -r -p "sourcePath: " sourcePath
read -r -p "backupPath: " backupPath
read -r -p "filePrefix: " filePrefix
mkdir -p /home/public/"$backupPath"
cp /export/home/public/"$sourcePath/$filePrefix"* /home/public/"$backupPath"
Make sure that the used user has the right to create the directory /home/public/"$backupPath".
See: help read
For a start: Your assignment states, that your script should accept arguments.
However your script does not take arguments. It reads the parameters from standard input. Arguments are passed to the script on the command line, and your script would be called as
./script.sh HW4testdir backup d
Hence you can't use read to fetch them. The first argument is available under the name $1, the second argument is $2 and so on. You could write for instance
sourcePath=${1?Parameter missing}
which has the side effect to abort the script with an error message, if the caller forgets to pass the parameter.
Another point: You don't say anywhere that bash should be used to run the script. Since you want the script to be called by
./script.sh ....
and not by
bash ./script.sh ....
you must encode the information, that bash should be used, in your script. Assuming that your bash is located in /usr/bin, you would do this by making the first line of the script
#!/usr/bin/bash
I have a file that I pass to a bash command that will create an output in a loop like so:
for file in /file/list/*
do
command
done
I wish to save the output that would have gone to standard out of each loop to a text file in my working directory. Currently I am trying this:
for file in /file/list/*
do
command | tee "$file_command output.txt"
done
What I expect to see are new files created in my current directory titled file1.txt_commandoutput.txt, file2.txt_commandoutput.txt, etc. The output of the command should be saved as a different file for each file. However I get only one file created and it's called ".txt" and can't be opened by any standard software on Mac. I am new to bash scripting, so help would be much appreciated!
Thanks.
Your problem comes from the variable name you're using:
"$file_command_output.txt" looks for a variable named file_command_output (the dot cannot be in the variable name, but the alphanumerical characters and the underscore all can).
What you're looking for is "${file}_command_output.txt" to make the variable name more explicit.
You have two issues in your script.
First, the wrong parameter/variable is expanded (file_command instead of file) because it's followed by a character that can be interpreted as part of the name (the underscore, _). To fix it, enclose the parameter name in braces, like this: ${file}_command (see Shell Parameter Expansion in bash manual).
Second, even with fixed variable name expansion, the file won't be created in your working directory, because the file holds an absolute pathname (/file/list/name). To fix it, you'll have to strip the directory from the pathname. You can do that with either basename command, or even better with a modified shell parameter expansion that will strip the longest matching prefix, like this: ${file##*/} (again, see Shell Parameter Expansion, section on ${parameter##word}).
All put together, your script now looks like:
#!/bin/bash
for file in /file/list/*
do
command | tee "${file##*/}_command output.txt"
done
Also, to just save the command output to a file, without printing it in terminal, you can use a simple redirection, instead of tee, like this: command > "${file##*/}_com...".
If you are not aware of xargs, try this:
$ ls
file
$ cat > file
one
two
three
$ while read this; do touch $this; done < ./file
$ ls
file one three two
I am writing a script that will symlink all of my dotfiles related to bash into the home directory. I want the script to check if the filenames already exists, so that I I can rename/move them.
For some reason can't get my if test-command to recognize filenames that have an underscore in them.
When testing for files that already exist, this script:
#!/bin/bash
for name in bashrc bash_profile bash_aliases
do
filename=$HOME"/."$name
if [ -e "$filename" ]; then
echo "${filename} exists"
else
echo "${filename} doesn't exist"
fi
done
Outputs:
/home/xavier/.bashrc exists
/home/xavier/.bash_profile doesn't exist
/home/xavier/.bash_aliases doesn't exist
What is it about the underscore that is causing this behavior, and how do I fix it?
The code is correct as posted and underscore is not a problematic character in general.
You mention that you're symlinking the files -- if you're sure the files are there, verify that they are not broken symlinks. -e file will be false if the final target of the link doesn't exist.
Other things that can cause this are:
lacking permissions
invisible unicode characters like a zero-width space
similar-looking unicode characters like bаsh＿profile which has a fullwidth low line instead of an underscore.
running the script in a chroot or sandbox
checking that the file exists in a different terminal than the one used for running the script -- it could be chrooted, SSH'd to another machine or started before a directory was mounted over the dir, and therefore have a different view of the fs
I have a remote Linode, which I am using Cygwin to access. An errant database file, specifically "C:\Users\Blah\Blah\website\blah\sqlite.db" was created. This file was used for local testing on my Windows machine, but was generated due to a mistake on the Linode. Note, this is the full file name inside the Linode, not the location of it. This is Windows syntax, not Unix, which is where I think the problem lies.
Now, I cannot delete it! It says, cannot remove file "file name" where file name does not have any of the original backslashes. This tells me that it cannot recognize that this is an errant windows DB file.
How can I delete this? If I had access to a GUI folder I could use that, but I only have the command line!
Please help!
The backslash and colon are not special characters to the filesystem (which is why you can have a file with those characters in its name), but backslash is a special character to the shell (and : is special in some contexts).
You just have to pass the file's name to the rm command. To do this from the shell, you need to escape the backslash characters.
This should work:
rm C:\\Users\\Blah\\Blah\\website\\blahsqlite.db
For example (I just tried this on my own system):
$ touch C:\\Users\\Blah\\Blah\\website\\blahsqlite.db
$ ls
C:\Users\Blah\Blah\website\blahsqlite.db
$ rm C:\\Users\\Blah\\Blah\\website\\blahsqlite.db
$
And if your shell supports tab completion, then you can probably just type rm Ctab
and, if there are no other files in the current directory whose names start with C, the shell will expand that to (an escaped version of) the file name. (Bash happens to insert a a \ in front of the : as well; this is unnecessary but harmless.)
We have simple Windows batch files that when an error occurs, an "ONCALL.bat" file is run to display support information that is maintained in a separate oncall.txt text file. This is our SOP.
ONCALL.BAT:
set scriptpath=%~dp0
TYPE "%scriptpath%oncall.txt"
I have zero experience with Unix and Shell scripts and I need to quickly provide a shell script equivalent to run in a Unix environment.
Could someone please provide me the .sh equivalent of this code?
Assuming that the help file and the script are in the same directory:
#!/bin/sh
SCRIPTPATH=`dirname "$0"`
cat "$SCRIPTPATH"/oncall.txt
$0 is the file path of the current script; the dirname command extracts the directory part of it. This way you can avoid using a hard-coded path for the help file within the script.
cat oncall.sh
#!/bin/bash
scriptpath=/path/to/scripts
cat ${scriptpath}/oncall.txt
After you create your file, it can't hurt to run
dos2unix oncall.sh
Just to be sure there are no windows Ctrl-M chars that will totally mystify you with the way they can screw up Unix script processing.
THEN
chmod 755 oncall.sh
To make the script executable.
confirm with
ls -l oncall.sh
You should see listing like
-rwxr-xr-x 1 userName grpname 5263 Nov 21 14:44 oncall.sh
Finally, call the script with a full or relative path, i.e.
./oncall.sh
OR
$PWD/oncall.sh
The first line is called the "shebang" line, and when your script is called, the OS reads the first line of the file, to find out what program to run to interpret the rest of the script file.
You may want/need to use as the first line "shebang" one of the following, but bash is a good guess
#!/bin/ksh
#!/bin/sh
#!/bin/ash
#!/bin/dash
#!/bin/zsh
OR you may worst case, your shell lives in a non-standard directory, then you'll have to spell that out, i.e.
#!/usr/bin/ksh
All shell support debugging arguments for trace and variable expansion like
#!/bin/ksh -vx
Or you can wrap just certain lines to turn debugginng on and off like
set -vx
cat ${scriptpath}/oncall.txt
set +vx
Given that
The ~dp special syntax between the % and the 0 basically says to expand the variable %0 to show the drive letter and path, which gives you the current directory containing the batch file!
I think /path/to/scripts is a reasonable substitute, scriptpath=$PWD would be a direct replacement, as there are no drive letters in Unix. The problem there, is that you either rely on unix PATH var to find your script or you cd /path/to/scripts and then run ./oncall.sh using the relative path./ to find the file without naving added a value to PATH.
IHTH.