I would like to find txt files with find command and move the directory of the found file, and then apply a command to the file using a bash shell one-liner
For example, this command works, but the acmd is executed in the current directory.
$ find . -name "*.txt" | xargs acmd
I would like to run acmd in the txt file's direcotry.
Does anyone have good idea?
From the find man page:--
-execdir command ;
-execdir command {} +
Like -exec, but the specified command is run from the subdirec‐
tory containing the matched file, which is not normally the
directory in which you started find. This a much more secure
method for invoking commands, as it avoids race conditions dur‐
ing resolution of the paths to the matched files. As with the
-exec action, the `+' form of -execdir will build a command line
to process more than one matched file, but any given invocation
of command will only list files that exist in the same subdirec‐
tory. If you use this option, you must ensure that your $PATH
environment variable does not reference `.'; otherwise, an
attacker can run any commands they like by leaving an appropri‐
ately-named file in a directory in which you will run -execdir.
The same applies to having entries in $PATH which are empty or
which are not absolute directory names. If find encounters an
error, this can sometimes cause an immediate exit, so some pend‐
ing commands may not be run at all. The result of the action
depends on whether the + or the ; variant is being used;
-execdir command {} + always returns true, while -execdir com‐
mand {} ; returns true only if command returns 0.
Just for completeness, the other option would be to do:
$ find . -name \*.txt | xargs -i sh -c 'echo "for file $(basename {}), the directory is $(dirname '{}')"'
for file schedutil.txt, the directory is ./Documentation/scheduler
for file devices.txt, the directory is ./Documentation/admin-guide
for file kernel-parameters.txt, the directory is ./Documentation/admin-guide
for file gdbmacros.txt, the directory is ./Documentation/admin-guide/kdump
...
i.e. have xargs "defer to a shell". In usecases where -execdir suffices, go for it.
Related
I am a bash newbie, and I'm trying to do something that seems fairly straightforward but am having issues.
I am trying to search for a file with a pretty generic but nonunique name (e.g. analysis.uniqueExt, but also maybe sorted_result.uniqueExt) that can be within one specific subdirectory of a directory that was found from a different 'find' query. Then I would like to copy that file to my personal directory whilst also renaming the file to something more descriptive that hints to its origin location.
Here is an example of what I have tried:
case=/home/data/ABC_123 # In reality this is coming from a different query successfully
specific_id=ABC_123 # This was extracted from the previous variable
OUTDIR=/my/personal/directory
mkdir -p $OUT_DIR/$this_folder
find $case/subfolder/ -type f -name "*.uniqueExt" -exec sh -c 'cp "$1" ${OUT_DIR}/${specific_id}/$(basename "$1")' sh {} \;
This doesn't work because OUT_DIR and specific_id are not scoped in the inner shell created by the -exec command.
So I tried to do this another way:
find $case/subfolder/ -type f -name "*.uniqueExt" -exec cp {} ${OUT_DIR}/${specific_id}/$(basename {}) \;
However now I cannot extract the basename of the file found in the 'find' query as I have not invoked a shell to do so.
Is there a way I can either properly scope my variables in example #1 or execute the basename function in example #2 to accomplish this? Or maybe there is a totally different solution (possibly involving multiple -exec calls? Or maybe just piping the find results to xargs?).
Thanks for your help!
You need to export the variables since you're using them in a different shell process than the one you assigned them in.
Exporting variables makes them available in descendant processes.
export specific_id=ABC_123 # This was extracted from the previous variable
export OUTDIR=/my/personal/directory
However, you don't really need to use the shell for this. You can use
find $case/subfolder/ -type f -name "*.uniqueExt" -exec cp -t "$OUTDIR/$specific_id/" {} +
You don't have to call basename yourself, because copying a file to a target directory automatically uses the basename as the destination filename.
In my version, I use the -t option so I can put the destination directory first. This allows it to use the + variant to put all the found filenames in a single command, rather than running cp separately for each file.
Pretty much I want to cd to the output of the find command:
find ~ -name work_project_linux
cd the_output
In general the best way to execute an arbitrary command on the results of find is with find -exec. Curly braces {} are placeholders for the file names it finds, and the entire command ends with + or \;. For example, this will run ls -l on all of the files found:
find ~ -name work_project_linux -exec ls -l {} +
It doesn't work with some special commands like cd, though. -exec runs binaries, such as those found in /usr/bin, and cd isn't a binary. It's a shell builtin, a special type of command that the shell executes directly instead of calling out to some executable on disk. For shell builtins you can use command substitution:
cd "$(find ~ -name work_project_linux)"
This wouldn't work if find finds multiple files. It's only good for a single file name. Command substitution also won't handle some unusual file names correctly, such as those with embedded newlines—unusual, but legal.
I want to copy all the xml files which is having current date as file name from all directories. Below is the script i have written.
#!/bin/bash
CURRENT_DATE=`date +'%d%m%Y'`
Temp_Path=/appinfprd/bi/infogix/IA83/InfogixClient/Scripts/IRP/New_Vendors/
FILE_PATH=/bishare/DLSFTP/DLSTREAM/
FILE_DATE=`date -d "-2 days" +"%Y%m%d"`
cd $FILE_PATH
find . -name '*$FILE_DATE*.xml' -exec cp $Temp_Path
But it is not working.
Your find statement is wrong. You should end it with \; to indicate the end of the exec command and put {} where the name of your file found should come in the command. So, you want :
find . -name "*$FILE_DATE*.xml" -exec cp "{}" "$Temp_Path" \;
Edit
As stated in the comments, there were also a problem in your initial post with your single quotes that should be double quotes. You might be interested by this man page. In particular by these sections :
-exec command ;
Execute command; true if 0 status is returned. All following arguments to find are taken to be arguments to the command until an argument consisting of ; is encountered. The string {} is replaced by the current file name being processed everywhere it occurs in the arguments to the command, not just in arguments where it is alone, as in some versions of find. Both of these constructions might need to be escaped (with a \) or quoted to protect them from expansion by the shell. See the EXAMPLES section for examples of the use of the -exec option. The specified command is run once for each matched file. The command is executed in the starting directory. There are unavoidable security problems surrounding use of the -exec action; you should use the -execdir option instead.
-exec command {} +
This variant of the -exec action runs the specified command on the selected files, but the command line is built by appending each selected file name at the end; the total number of invocations of the command will be much less than the number of matched files. The command line is built in much the same way that xargs builds its command lines. Only one instance of {} is allowed within the command, and (when find is being invoked from a shell) it should be quoted (for example, '{}') to protect it from interpretation by shells. The command is executed in the starting directory. If any invocation returns a non-zero value as exit status, then find returns a non-zero exit status. If find encounters an error, this can sometimes cause an immediate exit, so some pending commands may not be run at all. This variant of -exec always returns true.
In bash I want to copy all .yml.sample files in a Git repository (recursively) and rename them to just have a .yml extension.
Eg. test.yml.sample would be copied to test.yml
Here’s as close as I’ve got, but I'm not clear on how to strip .sample off the end of the file name when I copy.
find . -depth -name "*.yml.sample" -exec sh -c 'cp "$1" "${1%/.sample/}"' _ {} \;
This should work:
find . -depth -name "*.yml.sample" -exec sh -c 'cp -p "$1" "${1%.yml.sample}.yml"' _ {} \;
The first *.yml.sample finds the files via find. Then after the -exec part, the magic happens via cp taking the results of that find via $1 and then the file extension for the copied file is set via ${1%.yml.sample}.yml where .yml.sample is the source extension, and .yml is the new destination extension.
Note I also added the -p attribute to preserve the attributes from the source file to the copied file. You might not need that, but I think it can be helpful when doing copies like this.
And—since this shell logic can be confusing—in terms of the _ {} \;, it breaks down as this:
_ {}: As explained in this answer on the Unix/Linux Stack Exchange site, “The way this works is bash takes the parameters after -c as arguments, _ {} is needed so that the contents of {} is assigned to $1 not l.”
\;: When you run find with a -exec parameter, everything that happens after that is parsed through a new shell. Meaning the main find command runs in one parent shell and stuff after -exec runs in another child shell command. If you run it as _ {} ;, the child shell command would terminate. So instead, you escape it as \; so you get _ {} \; which means only the parent sell find would interpret that ; as a “terminate” and thus the paren find command can successfully run iterative commands via -exec without stopping that child shell command. Read up on -exec command ; here.
I think you can use a tool like mmv, to mass rename all the files you need.
mmv \*.yml.sample \#1.yml
The above line should work... just make sure to test it first. Hope this helps!
Edit: If you want to copy and rename, all in one step, you can use the -c flag. That will preserve the original file, and will make a copy using the rename mask.
mmv -c \*.yml.sample \#1.yml
Apreciate any help and excuse me if my terminology is incorrect.
What I am trying to do is write a scrpit/.bat file that will do the following:
copy 1 directory(and subdirectories) from pointA, to point B.
Then in pointB(and subdirectories) unzip the files which will give *.csv files
Then in pointB(and subdirectories) I want to delete some rows from all these csv files
This unix command, run on cygwin, will copy all the files from /cygdrive/v/pointA/* to the current directory . (i.e. the dot is the current working directory)
cp /cygdrive/v/pointA/* .
This unix command, run on cygwin, will go through all the files in the directory and subdirectories that end with .zip
and unzip them
find -iname *.zip -execdir unzip {} \;
This unix command, run on cygwin, will go through all the files in the directory and subdirectories that end with .csv
For each file it deletes the 1st 6 rows and the last row and that's the returned file.
find ./ -iname '*.csv' -exec sed -i '1,6d;$ d' '{}' ';'
I was looking to do this in one script/bat file but I am having trouble with the first find command
I am having trouble with the find and unzip commands on the one line and am wondering how and if this can be done
chdir C:\pointA
C:\cygwin\bin\cp.exe /cygdrive/v/pointB/* .
::find -iname *.zip -execdir unzip {} \;
::find ./ -iname '*.csv' -exec sed -i '1,6d;$ d' '{}' ';'
I did try something like this:
C:\cygwin\bin\find.exe -iname *.zip -execdir C:\cygwin\bin\unzip.exe {} \;
but I get the following:
/usr/bin/find: missing argument to `-execdir'
Can anyone advise if/how this can be done?
The Cygwin tools use their own kind of paths, e.g. /cygdrive/c/cygwin/bin/unzip.exe though sometimes the Windows paths with backslashes work, the backslashes do tend to confuse the Cygwin tools.
I highly recommend you write your tool in Bash shell script instead of a cmd.exe Windows batch file. In my experience (1) it's much easier to do flow control in bash scripts than in batch files, and (2) the Cygwin environment works better from Bash. You can open a bash shell and run bash yourscript.sh.
Your Bash script might look something like this: (untested)
#!/bin/bash
# This script would be run from a Cygwin Bash shell.
# You can use the Mintty program or run C:\cygwin\bin\bash --login
# to start a bash shell from Windows Command Prompt.
# Configure bash so the script will exit if a command fails.
set -e
cd /cygdrive/c/pointA
cp /cygdrive/v/pointB/* .
# I did try something like this:
# 1. Make sure you quote wildcards so the shell doesn't expand them
# before passing them to the 'find' program.
#
# 2. If you start bash with the --login option, the PATH will be
# configured so that C:\cygwin\bin is in your PATH, and you can
# just call 'find', 'cp' etc. without specifying full path to it.
# This will unzip all .zip files in all subdirectories under this one.
find -iname '*.zip' -execdir unzip {} \;