I'm pretty new to shell scripting, but it's been great in helping me automating cumbersome tasks in OS X.
One of the functions I'm trying to write in a new script needs to find the specific filename in a subdirectory given a regex string. While I do know that the file exists, the version (and therefore filename itself) is being continually updated.
My function is currently as follows:
fname(){
$2=$(find ./Folder1 -name "$1*NEW*")
}
Which I'm then calling later in my script with the following line:
fname Type1 filename1
What I'm hoping to do is save the filename I'm looking for in variable filename1. My find syntax seems to be correct if I run it in Terminal, but I get the following error when I run my script:
./myscript.sh: line 13: filename1=./Folder1/Type1-list-NEW.bin: No such file or directory
I'm not sure why the result of find is not just saving to the variable I've selected. I'd appreciate any help (regardless of trivial this question may end up being). Thanks!
EDIT: I have a bunch of files in the subdirectory, but with the way I'm setting that folder up I know my "find" query will return exactly 1 filename. I just need the specific filename to do various tasks (updating, version checking, etc.)
The syntax for writing output to a file is command > filename. So it should be:
fname() {
find ./Folder1 -name "$1*NEW*" > "$2"
}
= is for assigning to a variable, not saving output in a file.
Are you sure you need to put the output in a file? Why not put it in a variable:
fname() {
find ./Folder1 -name "$1*NEW*"
}
var=$(fname Type1)
If you really want the function to take the variable name as a parameter, you have to use eval
fname() {
eval "$2='$(find ./Folder1 -name "$1*NEW*")'"
}
Okay, so I'm reading this as, you want to take the output of the find and save it in a shell variable given by $2.
You can't actually use a shell variable to dynamically declare the name of a new shell variable to rename, when the shell sees an expansion at the beginning of a line it immediately begins processing the words as arguments and not as an assignment.
There might be some way of pulling this off with declare and export but generally speaking you don't want to use a shell variable to hold n file names, particularly if you're on OS X, those file names probably have whitespaces and you're not protecting for that in the way your find is outputting.
Generally what you do in this case is you take the list of files find is spitting out and you act on them immediately, either with find -exec or as a part of a find . -print0 | xargs -0 pipeline.
Related
I have a requirement where I need to look for the new files in a particular location. Once I see a new file, I need to run other script which will move that file to a
different location. Now I want the first script to look for new files. Once I find any files, based on the filename, I need to trigger the second script by passing the corresponding keyword as parameters.
Suppose the filename is abc_20.txt. In my first script it has to look for abc*.text. Once I find such a file, I need to trigger the second script by passing the parameter as abc.
Second script command: sh 2nd_script.sh abc
Could you please help me how to compare the filename (abc_20.txt) with predefined keyword (abc*.text) and how to pass the corresponding keyword to the second script (abc)?
how to compare the filename (abc_20.txt) with predefined keyword (abc*.text)
The term "keyword' is strangely used here - a string abc*.text is no keyword in shell, it has no special meaning.
The simplest and most portable way is to use case:
case "abc_20.txt" in
abc*.text) echo YES; ;;
*) echo NO; ;;
esac
how to pass the corresponding keyword to the second script(abc)
As with any information passed to any script, the simplest would be to use positional parameters.
corresponding_keyword=something
second_script "$corresponding_keyword"
If there may be 2nd_script.sh in the current directory,
you can execute 2nd_script.sh with the file matches "abc*.text" .
find . -name "abc*.text" -exec sh 2nd_script.sh '{}' \;
I am attempting to create a script that will iteratively run a command against a variable (which is a fully qualified filename) and output the results of that command to an individually named/timestamped file (to %S accuracy). Im not great with this stuff at all
here is what I do:
find /vmfs/volumes/unlistedpathname/unlistedfoldername |
while read list;do
vmkfstools -D "$list" >> duringmigration_10mins_"$list".$(date +"%Y.%m.%d.%H.%M.%S");
done
the output im hoping for is something like
duringmigration_10mins_blahblahblah.vmx.2016.09.25.21.26.35
of course it doesnt work, and im not exactly sure how to solve it. I know the problem outright is $list as the filename variable will reprint the fullpath, so I need some sort of way to tell the loop "hey just use the filename as the variable NOT the full path" but im not sure how to do that in this case. Im also hoping to be able to run this from any location not specific path.
There are two problems preventing the behaviour you are looking for:
As you saw the filenames returned by find include the full path.
Your find command will return all the files and the directory name.
We solve #1 by calling basename on $list in the output filename.
We solve #2 by adding -type f to the find command to only return files and not directories.
find /vmfs/volumes/unlistedpathname/unlistedfoldername -type f |
while read list ; do
vmkfstools -D "${list}" >> "duringmigration_10mins_$(basename "${list}").$(date +"%Y.%m.%d.%H.%M.%S")"
done
Its my first time to use BASH scripting and been looking to some tutorials but cant figure out some codes. I just want to list all the files in a folder, but i cant do it.
Heres my code so far.
#!/bin/bash
# My first script
echo "Printing files..."
FILES="/Bash/sample/*"
for f in $FILES
do
echo "this is $f"
done
and here is my output..
Printing files...
this is /Bash/sample/*
What is wrong with my code?
You misunderstood what bash means by the word "in". The statement for f in $FILES simply iterates over (space-delimited) words in the string $FILES, whose value is "/Bash/sample" (one word). You seemingly want the files that are "in" the named directory, a spatial metaphor that bash's syntax doesn't assume, so you would have to explicitly tell it to list the files.
for f in `ls $FILES` # illustrates the problem - but don't actually do this (see below)
...
might do it. This converts the output of the ls command into a string, "in" which there will be one word per file.
NB: this example is to help understand what "in" means but is not a good general solution. It will run into trouble as soon as one of the files has a space in its name—such files will contribute two or more words to the list, each of which taken alone may not be a valid filename. This highlights (a) that you should always take extra steps to program around the whitespace problem in bash and similar shells, and (b) that you should avoid spaces in your own file and directory names, because you'll come across plenty of otherwise useful third-party scripts and utilities that have not made the effort to comply with (a). Unfortunately, proper compliance can often lead to quite obfuscated syntax in bash.
I think problem in path "/Bash/sample/*".
U need change this location to absolute, for example:
/home/username/Bash/sample/*
Or use relative path, for example:
~/Bash/sample/*
On most systems this is fully equivalent for:
/home/username/Bash/sample/*
Where username is your current username, use whoami to see your current username.
Best place for learning Bash: http://www.tldp.org/LDP/abs/html/index.html
This should work:
echo "Printing files..."
FILES=(/Bash/sample/*) # create an array.
# Works with filenames containing spaces.
# String variable does not work for that case.
for f in "${FILES[#]}" # iterate over the array.
do
echo "this is $f"
done
& you should not parse ls output.
Take a list of your files)
If you want to take list of your files and see them:
ls ###Takes list###
ls -sh ###Takes list + File size###
...
If you want to send list of files to a file to read and check them later:
ls > FileName.Format ###Takes list and sends them to a file###
ls > FileName.Format ###Takes list with file size and sends them to a file###
I'm dealing with a pipeline of predominantly shell and Perl files, all of which pass parameters (paths) to the next. I decided it would be better to use a single file to store all the paths and just call that for every file. The issue is I am using awk to grab the files at the beginning of each file, and it's turning out to be a lot of repetition.
My question is: I do not know if there is a way to store key-value pairs in a file so shell can natively do something with the key and return the value? It needs to access an external file, because the pipeline uses many scripts and a map in a specific file would result in parameters being passed everywhere. Is there some little quirk I do not know of that performs a map function on an external file?
You can make a file of env var assignments and source that file as need, ie.
$ cat myEnvFile
path1=/x/y/z
path2=/w/xy
path3=/r/s/t
otherOpt1="-x"
Inside your script you can source with either . myEnvFile or the more versbose version of the same feature sourc myEnvFile (assuming bash shell) , i.e.
$cat myScript
#!/bin/bash
. /path/to/myEnvFile
# main logic below
....
# references to defined var
if [[ -d $path2 ]] ; then
cd $path2
else
echo "no pa4h2=$path2 found, can't continue" 1>&1
exit 1
fi
Based on how you've described your problem this should work well, and provide a-one-stop-shop for all of your variable settings.
IHTH
In bash, there's mapfile, but that reads the lines of a file into a numerically-indexed array. To read a whitespace-separated file into an associative array, I would
declare -A map
while read key value; do
map[$key]=$value
done < filename
However this sounds like an XY problem. Can you give us an example (in code) of what you're actually doing? When I see long piplines of grep|awk|sed, there's usually a way to simplify. For example, is passing data by parameters better than passing via stdout|stdin?
In other words, I'm questioning your statement "I decided it would be better..."
I am trying to make heads or tails of a shell script. Could someone please explain this line?
$FILEDIR is a directory containing files. F is a marker in an array of files that is returned from this command:
files=$( find $FILEDIR -type f | grep -v .rpmsave\$ | grep -v .swp\$ )
The confusing line is within a for loop.
for f in $files; do
target=${f:${#FILEDIR}}
<<do some more stuff>>
done
I've never seen the colon, and the hash before in a shell script for loop. I haven't been able to find any documentation on them... could someone try and enlighten me? I'd appreciate it.
There are no arrays involved here. POSIX sh doesn't have arrays (assuming you're not using another shell based upon the tags).
The colon indicates a Bash/Ksh substring expansion. These are also not POSIX. The # prefix expands to the number of characters in the parameter. I imagine they intended to chop off the directory part and assign it to target.
To explain the rest of that: first find is run and hilariously piped into two greps which do what could have been done with find alone (except breaking on possible filenames containing newlines), and the output saved into files. This is also something that can't really be done correctly if restricted only to POSIX tools, but there are better ways.
Next, files is expanded unquoted and mutalated by the shell in more ridiculous ways for the for loop to iterate over the meaningless results. If the rest of the script is this bad, probably throw it out and start over. There's no way that will do what's expected.
The colon can be as a substring. So:
A=abcdefg
echo ${A:4}
will print the output:
efg
I'm not sure why they would use a file directory as the 2nd parameter though...
If you are having problems understanding the for loop section, try http://www.dreamsyssoft.com/unix-shell-scripting/loop-tutorial.php