I'm trying to use a bash function from find. I know that I need to export the function. When I do this from the command line it works. When I do it from within a bash script it doesn't. the code is listed below. if you execute it doesn't work. if you source it, then it does. What is interesting, is that once you source it, the script will then work because it's defined at the time of the script invocation.
#!/bin/bash
function doit {
echo args "<$#>"
}
doit a b c
export -f doit
find . -maxdepth 1 -exec sh -c 'doit "$#" ' {} \+
here's the error I see:
> ./x.sh
args <a b c>
.: doit: command not found
You are using bash's export but using plain sh which may not understand it (which may be symlinked to bash, not necessarily true on all platforms).
Use bash to exec. Change:
find . -maxdepth 1 -exec sh -c 'doit "$#" ' {} \+
to
find . -maxdepth 1 -exec bash -c 'doit "$#" ' {} \+
Related
how to write the command??
I tried find . -name test.bash | xargs bash dd, this threw an error bash: xx: No such file or directory
I also tried find . -name test.bash | xargs bash -c, can't work too.
I created a test script, super simple, it just prints the arguments.
#!/bin/bash
echo "$#"
Then I find it in it's directory and call it with arguments:
find . -name test.bash -exec {} arg1 arg2 \;
It runs and outputs "arg1 arg2".
Recently I've came across a strange bash script, which is used to call a custom bash function from inside find -exec. I've developed following simple script to demonstrate the functionality I need to get explained.
In the following example, function foo will be called for each find result.
foo()
{
echo "$#"
}
export -f foo
find . -exec bash -c 'foo "$#"' bash {} \;
Can someone explain how the part after -exec is interpreted?
UPDATE:
To further simplify this, after exporting foo as above, following gets executed for each find result (assume there is a file named my_file).
bash -c 'foo "$#"' bash my_file
And this produces the output myfile. I don't understand how this works. What does the second bash does there? Any detailed explanation is appreciated.
(Please note that this question is not about find command. Also please ignore the functionality of function foo, I just wanted to export some function)
To understand you need to know 4 things:
The find action -exec allows you to apply a command on the found files and directories.
The -c bash option is documented as follows:
BASH(1)
...
OPTIONS
...
-c If the -c option is present, then commands are read from
the first non-option argument command_string.
If there are arguments after the command_string, they
are assigned to the positional parameters, starting with $0.
...
If bash is started with the -c option, then $0 is set to the first
argument after the string to be executed, if one is present.
Otherwise, it is set to the filename used to invoke bash, as given
by argument zero.
In bash, $# expands as all positional parameters ($1, $2...) starting at parameter $1.
In a bash function, the positional parameters are the arguments passed to the function when it is called.
So, in your case, the command executed for each found file or directory is:
bash -c 'foo "$#"' bash <the-file>
The positional parameters are thus set to:
$0 = bash
$1 = <the-file>
and bash is asked to execute 'foo "$#"' in this context. "$#" is first expanded as "<the-file>". So, function foo is called with one single argument: "<the-file>". In the context of function foo the positional parameters are thus:
$1 = "<the-file>"
and echo "$#" expands as echo "<the-file>".
All this just prints the names of all found files or directories. It is almost as if you had any of:
find . -exec echo {} \;
find . -print
find .
find
(for find versions that accept the last one).
Almost as if, only, because if file or directory names contain spaces, depending on your use of find and of quotes, you will get different results. So, if you intend to have a more complex foo function, you should pay attention to the quotes. Examples:
$ touch "filename with spaces" plain
$ ls -1
filename with spaces
plain # 2 files
$ foo() { echo "$#"; } # print arguments
$ find . -type f
./filename with spaces
./plain
$ find . -type f -exec bash -c 'foo "$#"' bash {} \;
./filename with spaces
./plain
$ find . -type f -exec bash -c 'foo $#' bash {} \;
./filename with spaces
./plain
The 3 find commands apparently do the same but:
$ bar() { echo $#; } # print number of arguments
$ wc -w < <(find . -type f)
4 # 4 words
$ find . -type f -exec bash -c 'bar "$#"' bash {} \;
1 # 1 argument
1 # per file
$ find . -type f -exec bash -c 'bar $#' bash {} \;
3 # 3 arguments
1 # 1 argument
With find . -type f -exec bash -c 'bar "$#"' bash {} \;, the first file name is passed to function bar as one single argument, while in all other cases it is considered as 3 separate arguments.
I have a very long command in bash, which I do not want to type all the time, so I put an alias in my .profile
alias foo='...'
Now I want to execute this alias using find -exec
find . -exec foo '{}' \;
but find cannot find foo:
find: foo: No such file or directory
Is it possible to use an alias in find?
find itself doesn't know anything about aliases, but your shell does. If you are using a recent enough version of bash (I think 4.0 added this feature), you can use find . -exec ${BASH_ALIASES[foo]} {} \; to insert the literal content of the alias at that point in the command line.
Nope, find doesn't know anything about your aliases. Aliases are not like environment variables in that they aren't "inherited" by child processes.
You can create a shell script with the same commands, set +x permissions and have it in your path. This will work with find.
Another way of calling an alias when processing the results of find is to use something like this answer
so the following should work:
alias ll="ls -al"
find . -type d | while read folder; do ll $folder; done
I am using the ll commonly know alias for this example but you may use your alias instead, just replace ll in the following line with your alias (foo) and it should work:
find . -exec `alias ll | cut -d"'" -f2` {} \;
your case:
find . -exec `alias foo | cut -d"'" -f2` {} \;
Note it assumes your alias is quoted using the following syntax:
alias foo='your-very-long-command'
It's not possible (or difficult / error-prone) to use aliases in the find command.
An easier way to achieve the desired result is putting the contents of the alias in a shellscript and run that shellscript:
alias foo | sed "s/alias foo='//;s/'$/ \"\$#\"/" > /tmp/foo
find -exec bash /tmp/foo {} \;
The sed command removes the leading alias foo=' and replaces the trailing ' by "$#" which will contain the arguments passed to the script.
You can use the variable instead.
So instead of:
alias foo="echo test"
use:
foo="echo test"
then execute it either by command substitution or eval, for instance:
find . -type f -exec sh -c "eval $foo" \;
or:
find . -type f -exec sh -c "echo `$foo`" \;
Here is real example which is finding all non-binary files:
IS_BINARY='import sys; sys.exit(not b"\x00" in open(sys.argv[1], "rb").read())'
find . -type f -exec bash -c "python -c '$IS_BINARY' {} || echo {}" \;
I ran into the same thing and pretty much implemented skjaidev's solution.
I created a bash script called findVim.sh with the following contents:
[ roach#sepsis:~ ]$ cat findVim.sh #!/bin/bash
find . -iname $1 -exec vim '{}' \;
Then I added the the .bashrc alias as:
[ roach#sepsis:~ ]$ cat ~/.bashrc | grep fvim
alias fvim='sh ~/findVim.sh'
Finally, I reloaded .bashrc with source ~/.bashrc.
Anyways long story short I can edit arbitrary script files slightly faster with:
$ fvim foo.groovy
Is there a way to get find to execute a function I define in the shell?
For example:
dosomething () {
echo "Doing something with $1"
}
find . -exec dosomething {} \;
The result of that is:
find: dosomething: No such file or directory
Is there a way to get find's -exec to see dosomething?
Since only the shell knows how to run shell functions, you have to run a shell to run a function. You also need to mark your function for export with export -f, otherwise the subshell won't inherit them:
export -f dosomething
find . -exec bash -c 'dosomething "$0"' {} \;
find . | while read file; do dosomething "$file"; done
Jac's answer is great, but it has a couple of pitfalls that are easily overcome:
find . -print0 | while IFS= read -r -d '' file; do dosomething "$file"; done
This uses null as a delimiter instead of a linefeed, so filenames with line feeds will work. It also uses the -r flag which disables backslash escaping, and without it backslashes in filenames won't work. It also clears IFS so that potential trailing white spaces in names are not discarded.
Add quotes in {} as shown below:
export -f dosomething
find . -exec bash -c 'dosomething "{}"' \;
This corrects any error due to special characters returned by find,
for example files with parentheses in their name.
Processing results in bulk
For increased efficiency, many people use xargs to process results in bulk, but it is very dangerous. Because of that there was an alternate method introduced into find that executes results in bulk.
Note though that this method might come with some caveats like for example a requirement in POSIX-find to have {} at the end of the command.
export -f dosomething
find . -exec bash -c 'for f; do dosomething "$f"; done' _ {} +
find will pass many results as arguments to a single call of bash and the for-loop iterates through those arguments, executing the function dosomething on each one of those.
The above solution starts arguments at $1, which is why there is a _ (which represents $0).
Processing results one by one
In the same way, I think that the accepted top answer should be corrected to be
export -f dosomething
find . -exec bash -c 'dosomething "$1"' _ {} \;
This is not only more sane, because arguments should always start at $1, but also using $0 could lead to unexpected behavior if the filename returned by find has special meaning to the shell.
Have the script call itself, passing each item found as an argument:
#!/bin/bash
if [ ! $1 == "" ] ; then
echo "doing something with $1"
exit 0
fi
find . -exec $0 {} \;
exit 0
When you run the script by itself, it finds what you are looking for and calls itself passing each find result as the argument. When the script is run with an argument, it executes the commands on the argument and then exits.
Just a warning regaring the accepted answer that is using a shell,
despite it well answer the question, it might not be the most efficient way to exec some code on find results:
Here is a benchmark under bash of all kind of solutions,
including a simple for loop case:
(1465 directories, on a standard hard drive, armv7l GNU/Linux synology_armada38x_ds218j)
dosomething() { echo $1; }
export -f dosomething
time find . -type d -exec bash -c 'dosomething "$0"' {} \;
real 0m16.102s
time while read -d '' filename; do dosomething "${filename}" </dev/null; done < <(find . -type d -print0)
real 0m0.364s
time find . -type d | while read file; do dosomething "$file"; done
real 0m0.340s
time for dir in $(find . -type d); do dosomething $dir; done
real 0m0.337s
"find | while" and "for loop" seems best and similar in speed.
For those of you looking for a Bash function that will execute a given command on all files in current directory, I have compiled one from the above answers:
toall(){
find . -type f | while read file; do "$1" "$file"; done
}
Note that it breaks with file names containing spaces (see below).
As an example, take this function:
world(){
sed -i 's_hello_world_g' "$1"
}
Say I wanted to change all instances of "hello" to "world" in all files in the current directory. I would do:
toall world
To be safe with any symbols in filenames, use:
toall(){
find . -type f -print0 | while IFS= read -r -d '' file; do "$1" "$file"; done
}
(but you need a find that handles -print0 e.g., GNU find).
It is not possible to executable a function that way.
To overcome this you can place your function in a shell script and call that from find
# dosomething.sh
dosomething () {
echo "doing something with $1"
}
dosomething $1
Now use it in find as:
find . -exec dosomething.sh {} \;
To provide additions and clarifications to some of the other answers, if you are using the bulk option for exec or execdir (-exec command {} +), and want to retrieve all the positional arguments, you need to consider the handling of $0 with bash -c.
More concretely, consider the command below, which uses bash -c as suggested above, and simply echoes out file paths ending with '.wav' from each directory it finds:
find "$1" -name '*.wav' -execdir bash -c 'echo "$#"' _ {} +
The Bash manual says:
If the -c option is present, then commands are read from the first non-option argument command_string. If there are arguments after the command_string, they are assigned to positional parameters, starting with $0.
Here, 'echo "$#"' is the command string, and _ {} are the arguments after the command string. Note that $# is a special positional parameter in Bash that expands to all the positional parameters starting from 1. Also note that with the -c option, the first argument is assigned to positional parameter $0.
This means that if you try to access all of the positional parameters with $#, you will only get parameters starting from $1 and up. That is the reason why Dominik's answer has the _, which is a dummy argument to fill parameter $0, so all of the arguments we want are available later if we use $# parameter expansion for instance, or the for loop as in that answer.
Of course, similar to the accepted answer, bash -c 'shell_function "$0" "$#"' would also work by explicitly passing $0, but again, you would have to keep in mind that $# won't work as expected.
Put the function in a separate file and get find to execute that.
Shell functions are internal to the shell they're defined in; find will never be able to see them.
I find the easiest way is as follows, repeating two commands in a single do:
func_one () {
echo "The first thing with $1"
}
func_two () {
echo "The second thing with $1"
}
find . -type f | while read file; do func_one $file; func_two $file; done
Not directly, no. Find is executing in a separate process, not in your shell.
Create a shell script that does the same job as your function and find can -exec that.
I would avoid using -exec altogether. Use xargs:
find . -name <script/command you're searching for> | xargs bash -c
my .bashrc has the following function
function myfile {
file $1
}
export -f myfile
it works fine when i call it directly
rajesh#rajesh-desktop:~$ myfile out.ogv
out.ogv: Ogg data, Skeleton v3.0
it does not work when i try to invoke it through exec
rajesh#rajesh-desktop:~$ find ./ -name *.ogv -exec myfile {} \;
find: `myfile': No such file or directory
is there a way to call bash script functions with exec?
Any help is greatly appreciated.
Update:
Thanks for the response Jim.
But that's exactly what I wanted to avoid in the first place, since I have lot of utility functions defined in my bash scripts, I wanted to use them with other useful commands like find -exec.
I totally see your point though, find can run executables, it has no idea that the argument passed is function defined in a script.
I will get the same error when I try to exec is on bash prompt.
$ exec myfile out.ogv
I was hoping that there may be some neat trick that exec could be given some hypothetical command like "bash -myscriptname -myfunctionname".
I guess I should try to find some way to create a bash script on the fly and run it with exec.
find ./ -name *.ogv -exec bash -c 'myfile {}' \;
I managed to run it perhaps more elegantly as:
function myfile { ... }
export -f myfile
find -name out.ogv -exec bash -c '"$#"' myfile myfile '{}' \;
Notice that myfile is given twice. The first one is the $0 parameter of the script (and in this case it can be basically anything). The second one is the name of the function to run.
$ cat functions.bash
#!/bin/bash
function myecho { echo "$#"; }
function myfile { file "$#"; }
function mycat { cat "$#"; }
myname=`basename $0`
eval ${myname} "$#"
$ ln functions.bash mycat
$ ./mycat /etc/motd
Linux tallguy 2.6.32-22-core2 ...
$ ln functions.bash myfile
$ myfile myfile
myfile: Bourne-Again shell script text executable
$ ln functions.bash myecho
$ myecho does this do what you want\?
does this do what you want?
$
where, of course, the functions can be a tad more complex than my examples.
You can get bash to run a function by putting the command into bash's StdIn:
bash$ find ./ -name *.ogv -exec echo myfile {} \; | bash
The command above will work for your example but you need to take note of the fact that all of the 'myfile...' commands are generated at once and sent to a single bash process.
I don't think find can do this, since it's the find command itself that's executing
the command, and not the shell you're currently running...so bash functions or aliases
won't work there. If you take your function definition and turn it into a separate
bash script called myfile, make it executable, and install it on your path somewhere,
find should do the right thing with it.
even simpiler
function myfile { echo $* ; }
export -f myfile
find . -type f -exec bash -c 'myfile "{}"' \;
Child shell scripts seems to keep the parent functions so you could do a script similar to this one:
'runit.sh'
#! /bin/bash
"$#"
then do find -name out.ogv -exec ./runit.sh myfile '{}' \;
and it works! :)
Thanks Joao. This looks like very clever and elegant solution. Little issue was that I had to source my script first to run myfile function e.g. I borrowed from your suggestion and made my runint.sh as follows
#!/bin/bash
script_name=$1
func_name=$2
func_arg=$3
source $script_name
$func_name $func_arg
Now I can run it as follows
$ find ./ -name *.ogv -exec ./runit.sh ~/.bashrc myfile {} \;
./out.ogv: Ogg data, Skeleton v3.0
Otherwise I was getting
$ find ./ -name *.ogv -exec ./runit.sh myfile {} \;
./runit.sh: 1: myfile: not found
Anyway thanks a lot.