I am attempting to write a shell script that will take a file name with a wildcard, find all files matching that pattern in current directory, and copy them. My problem is every time I try and use a variable only the first match echo's and thats it.
./copyfiles.ksh cust*.txt
#! /usr/bin/ksh
IN_FILE=${1}
for file in $IN_FILE
do
echo "$file"
done
cust1.txt
This seems to only match the first one even though cust1.txt, cust2.txt, and cust3.txt all exist and when I run it with for file in cust*.txt it works.
The shell expands your argument of "cust*.txt" to a list then passes the list to your script, which then only processes $1 which is cust1.txt.
You want to use $# which will process all arguments passed:
#! /usr/bin/ksh
for file in "$#"
do
echo "$file"
done
I believe there is a limit to how many arguments can be passed this way though. How many files are you having to process? Make sure your version of the shell can handle the number of arguments you are likely to process. If I recall you may need a solution utilizing xargs but I'm a tad rusty to help with that.
In ./copyfiles.ksh cust*.txt the files cust*.txt will be expanded first.
When you do not want to change your copyfiles.ksh script. call it with
./copyfiles.ksh "cust*.txt"
You can also change your script, with something like
IN_FILE="$#" # INFILES would be a better name
Related
I noticed that my script was ignoring my positional arguments in old terminal tabs, but working on recently created ones, so I decided to reduce it to the following:
TAG=test
while getopts 't:' c
do
case $c in
t)
TAG=$OPTARG
;;
esac
done
echo $TAG
And running the script I have:
~ source my_script
test
~ source my_script -t "test2"
test2
~ source my_script -t "test2"
test
I thought it could be that c was an special used variable elsewhere but after changing it to other names I had the exact same problem. I also tried adding a .sh extension to the file to see it that was a problem, but nothing worked.
Am I doing something wrong ? And why does it work the first time, but not the subsequent attempts ?
I am on MacOS and I use zsh.
Thank you very much.
The problem is that you're using source to run the script (the . command does the same thing). This makes it run in your current (interactive) shell (rather than a subprocess, like scripts normally do). This means it uses the same variables as the current shell, which is necessary if you want it to change those variables, but it can also have weird effects if you're not careful.
In this case, the problem is that getopts uses the variable OPTIND to keep track of where it is in the argument list (so it doesn't process the same argument twice). The first time you run the script with -t test2, getopts processes those arguments, and leaves OPTIND set to 3 (meaning that it's already done the first two arguments, "-t" and "test2". The second time you run it with options, it sees that OPTIND is set to 3, so it thinks it's already processed both arguments and just exits the loop.
One option is to add unset OPTIND before the while getopts loop, to reset the count and make it start from the beginning each time.
But unless there's some reason for this script to run in the current shell, it'd be better to make it a standard shell script and have it run as a subprocess. To do this:
Add a "shebang" line as the first line of the script. To make the script run in bash, that'd be either #!/bin/bash or #!/usr/bin/env bash. For zsh, use #!/bin/zsh or #!/usr/bin/env zsh. Since the script runs in a separate shell process, the you can run bash scripts from zsh or zsh scripts from bash, or whatever.
Add execute permission to the script file with chmod -x my_script (or whatever the file's actual name is).
Run the script with ./my_script (note the lack of a space between . and /), or by giving the full path to the script, or by putting the script in some directory in your PATH (the directories that're automatically searched for commands) and just running my_script. Do NOT run it with the bash, sh, zsh etc commands; these override the shebang and therefore can cause confusion.
Note: adding ".sh" to the filename is not recommended; it does nothing useful, and makes the script less convenient to run since you have to type in the extension every time you run it.
Also, a couple of recommendations: there are a bunch of all-caps variable names with special meanings (like PATH and OPTIND), so unless you want one of those special meanings, it's best to use lower- or mixed-case variable names (e.g. tag instead of TAG). Also, double-quoting variable references (e.g. echo "$tag" instead of echo $tag) avoids a lot of weird parsing headaches. Run your scripts through shellcheck.net; it's good at spotting common mistakes like this.
Hello all I have a program running on a linux OS that allows me to call a bash script upon a trigger (such as a file transfer). I will run something like:
/usr/bin/env bash -c "updatelog.sh '${filesize}' '${filename}'"
and the scripts job is to update the log file with the file name and file size. But if I pass in a file name with a single quote in its file name then it will break the script and give an error saying "Unexpected EOF while looking for matching `''"
I realize that a file name with a single quote is making the calling command an invalid one since the single quote is messing with the command itself. However I don't want to sanitize the variables if I can help it cause I would like my log to have the exact file name being displayed to easier cross reference it later. Is this possible or is sanitizing the only option here?
Thanks very much for your time and assistance.
Sanitization is absolutely not needed.
The simplest solution, assuming your script is properly executable (has +x permissions and a valid shebang line), is:
./updatelog.sh "$filesize" "$filename"
If for some reason you must use the bash -c, use single quotes instead of double quotes surrounding your code, and keep your data out-of-band from that code:
bash -c 'updatelog.sh "$#"' 'updatelog' "$filesize" "$filename"
Note that only updatelog.sh "$#" is inside the -c argument and parsed as code, and that this string is in single quotes, passed through without any changes whatsoever.
Following it are your arguments $0, $1 and $2; $0 is used when printing error messages, while $1 and $2 go into the list of arguments -- aka $# -- passed through to updatelog.sh.
I am trying to run the next code on bash. It is suppose to work but it does not.
Can you help me to fix it? I am starting with programming.
The code is this:
for i in {1:5}
do
cd path-to-folder-number/"$i"0/
echo path-to-folder-number/"$i"0/
done
EXAMPLE
I want to iterate over folders which have numbers (10,20..50), and so it changes directory from "path-to-folder-number/10/" to "path-to-folder-number/20/" ..etc
I replace : with .. but it is not working yet. When the script is applied i get:
can't cd to path-to-folder-number/{1..5}0/
I think there are three problems here: you're using the wrong shell, the wrong syntax for a range, and if you solved those problems you may also have trouble with successive cds not doing what you want.
The shell problem is that you're running the script with sh instead of bash. On some systems sh is actually bash (but running in POSIX compatibility mode, with some advanced features turned off), but I think on your system it's a more basic shell that doesn't have any of the bash extensions.
The best way to control which shell a script runs with is to add a "shebang" line at the beginning that says what interpreter to run it with. For bash, that'd be either #!/bin/bash or #!/usr/bin/env bash. Then run the script by either placing it in a directory that's in your $PATH, or explicitly giving the path to the script (e.g. with ./scriptname if you're in the same directory it's in). Do not run it with sh scriptname, because that'll override the shebang and use the basic shell, and it won't work.
(BTW, the name "shebang" comes from the "#!" characters the line starts with -- the "#" character is sometimes called "sharp", and "!" is sometimes called "bang", so it's "sharp-bang", which gets abbreviated to "shebang".)
In bash, the correct syntax for a range in a brace expansion is {1..5}, not {1:5}. Note that brace expansions are a bash extension, which is why getting the right shell matters.
Finally, a problem you haven't actually run into yet, but may when you get the first two problems fixed: you cd to path-to-folder-number/10/, and then path-to-folder-number/20/, etc. You are not cding back to the original directory in between, so if the path-to-folder-number is relative (i.e. doesn't start with "/"), it's going to try to cd to path-to-folder-number/10/path-to-folder-number/20/path-to-folder-number/30/path-to-folder-number/40/path-to-folder-number/50/.
IMO using cd in scripts is generally a bad idea, because there are a number of things that can go wrong. It's easy to lose track of where the script is going to be at which point. If any cd fails for any reason, then the rest of the script will be running in the wrong place. And if you have any files specified by relative paths, those paths become invalid as soon as you cd someplace other than the original directory.
It's much less fragile to just use explicit paths to refer to file locations within the script. So, for example, instead of cd "path-to-folder-number/${i}0/"; ls, use ls "path-to-folder-number/${i}0/".
For up ranges the syntax is:
for i in {1..5}
do
cd path-to-folder-number/"$i"0/
echo $i
done
So replace the : with ..
To get exactly what you want you can use this:
for i in 10 {20..50}
do
echo $i
done
You can also use seq :
for i in $(seq 10 10 50); do
cd path-to-folder-number/$i/
echo path-to-folder-number/$i/
done
I have some files in a folder with content as a 4 digit number in each file. How can I export variable name=filename and its value=filecontent with a loop in bash?
It is quite simple.
#!/bin/bash
dir="."
for file in "$dir"/*
do
[[ -f "$file" ]] || continue
var="${file##*/}"
if
printf -v "$var" "$(<"$file")" 2>/dev/null
then
export "$var"
else
echo "Invalid filename: $file"
fi
done
Not all filenames are valid variable names. You would need to either make 100% sure all files you could ever use your script with have names that are valid variable names, or (preferably) perform some kind of test or error handling. The script above will detect a failed assignment, but will not do anything to clever aside from complaining.
Some explanations...
The loop body is skipped if the file is not a regular file (i.e. directory, special files are skipped).
The code uses the -v option of printf, which causes printf to assign a value whose name is provided instead of printing to stdout. This is safer than, say, improperly using eval, which would open up code injection possibilities, especially considering you are using filenames which the script cannot control.
The "$(<"$file")" statement is a command substitution that outputs the content of the file, like a redirection that produces a string rather than a stream.
Finally, please note that if you want to export the variables in preparation for other things your script will do, you are fine. However, if you want to export these variables to the shell that calls the script, you will need to execute the script with . (or source), because a child process can never export (or make any kind of assignment) to the variables of its parent. Sourcing causes the shell to read the commands from the stated file without starting a child process.
This is one of my homework exercise.
Write a shell program, which will take a directory as an argument.
The script should then print all the regular files in the directory and all
the recursive directories, with the following information n the given order for
each of the files
File name (full name from the specified directory) file size owner
In case the directory argument is not given, the script should assume the
directory to be the current working directory
I am confused about how to approach this problem. For the listing of files part, I tried ls -R | awk ... but i was not able to do it because I was not able to find a suitable field seperator for awk.
I know its unfair to ask for a solution, but please can you guys give me a hint as how to proceed with the problem? Thanks in advance.
You really don't want to use ls and awk for this. Instead you want to check the documentation for find to figure out what string to put in the following script:
find ${1:-.} -type f -printf "format-string-to-be-determined-by-reader\n"
The problem is that parsing the output of ls is complicated at best and dangerous at worst.
What you'll want to do is use find to produce the list of files and a small shell script to produce the output you want. While there are many possible methods to accomplish this I would use the following general form
while read -r file ; do
# retrieve information about $file
# print that information on one line
done < <(find ...)
With a suitable find command to select the files. To retrieve the metadata about the files I would use stat inside the loop, probably multiple times.
I hope that's enough of a hint, but If you want a complete answer I can provide.
awk is fine.. use " +" as separator.
Bah. Where's the challenge in using ls or find? May as well write a one-liner in perl to do all the work, and then just call the one-liner from a script. ;)
You can do your recursive directory traversal in the shell natively, and use stat to get the size and owner. Basically, you write a function to list the directory (for element in *), and have the function change to the directory and call itself if [[ -d $element ]] is true. Something like
do_print "$elem"
if [[ -d "$elem" ]]
then
cd "$elem"
process_dir
cd ..
fi
or something akin to that.
Yeah, you'll have a zillion system calls to stat, but IMHO that's probably preferable to machine-parsing the output of a program whose output is intended to be human-readable. In this case, where performance is not an issue, it's worth it.
For bonus super happy fun times, change the value of IFS to a value which won't appear in a filename so the shell globbing won't get confused by files containing whitespace in its name. I'd suggest either a newline or a slash.
Or take the easy way out and just use find with printf. :)