I am new to Makefiles and was wondering if I could do something like this:
local-lint:
poetry run sqlfluff fix $(shell find . -name "$${ARG}" -not -path "./target/*") --show-lint-violations
then:
make local-lint ARG=myarg
You can, but not quite the way you use it here, but why use $(shell ...)? A recipe is already running in a shell, that's how make works (it invokes a shell to run the recipe). So you don't need to use make's shell function inside a shell script... it just adds confusion (IMO).
The first problem you have is that when you want to expand a make variable, like the ARG variable, you do not want to escape it. So your reference to ARG should just be $(ARG) or ${ARG} (they are equivalent) not $${ARG}.
But a better way to write this is to avoid the shell function, as I mentioned; you can do it like this:
local-lint:
poetry run sqlfluff fix `find . -name ${ARG} -not -path "./target/*"` --show-lint-violations
Or, if you prefer to use the newer shell syntax $(...) rather than backquotes you can but you need to escape the $ because you want the shell to see this:
local-lint:
poetry run sqlfluff fix $$(find . -name ${ARG} -not -path "./target/*") --show-lint-violations
Related
I am a bash newbie, and I'm trying to do something that seems fairly straightforward but am having issues.
I am trying to search for a file with a pretty generic but nonunique name (e.g. analysis.uniqueExt, but also maybe sorted_result.uniqueExt) that can be within one specific subdirectory of a directory that was found from a different 'find' query. Then I would like to copy that file to my personal directory whilst also renaming the file to something more descriptive that hints to its origin location.
Here is an example of what I have tried:
case=/home/data/ABC_123 # In reality this is coming from a different query successfully
specific_id=ABC_123 # This was extracted from the previous variable
OUTDIR=/my/personal/directory
mkdir -p $OUT_DIR/$this_folder
find $case/subfolder/ -type f -name "*.uniqueExt" -exec sh -c 'cp "$1" ${OUT_DIR}/${specific_id}/$(basename "$1")' sh {} \;
This doesn't work because OUT_DIR and specific_id are not scoped in the inner shell created by the -exec command.
So I tried to do this another way:
find $case/subfolder/ -type f -name "*.uniqueExt" -exec cp {} ${OUT_DIR}/${specific_id}/$(basename {}) \;
However now I cannot extract the basename of the file found in the 'find' query as I have not invoked a shell to do so.
Is there a way I can either properly scope my variables in example #1 or execute the basename function in example #2 to accomplish this? Or maybe there is a totally different solution (possibly involving multiple -exec calls? Or maybe just piping the find results to xargs?).
Thanks for your help!
You need to export the variables since you're using them in a different shell process than the one you assigned them in.
Exporting variables makes them available in descendant processes.
export specific_id=ABC_123 # This was extracted from the previous variable
export OUTDIR=/my/personal/directory
However, you don't really need to use the shell for this. You can use
find $case/subfolder/ -type f -name "*.uniqueExt" -exec cp -t "$OUTDIR/$specific_id/" {} +
You don't have to call basename yourself, because copying a file to a target directory automatically uses the basename as the destination filename.
In my version, I use the -t option so I can put the destination directory first. This allows it to use the + variant to put all the found filenames in a single command, rather than running cp separately for each file.
I'm trying to parse the android source directory and i need to extract all the directory names excluding certain patterns. If you notice below., for now i included only 1 directory to the exclude list, but i will be adding more.,
The find command doesn't exclude the directory with name 'docs'.
The commented out line works., but the other one doesn't. For easy debugging, i included the min and maxdepth which i would remove later.
Any comments or hints on why it doesn't work?
#! /bin/bash
ANDROID_PATH=$1
root=/
EXCLUDES=( doc )
cd ${root}
for dir in "${EXCLUDES[#]}"; do
exclude_name_cmd_string=${exclude_name_cmd_string}$(echo \
"-not -name \"${dir}*\" -prune")
done
echo -e ${exclude_name_cmd_string}
custom_find_cmd=$(find ${ANDROID_PATH} -mindepth 1 -maxdepth 1 \
${exclude_name_cmd_string} -type d)
#custom_find_cmd=$(find ${ANDROID_PATH} -mindepth 1 -maxdepth 1 \
# -not -name "doc*" -prune -type d)
echo ${custom_find_cmd}
Building up a command string with possibly-quoted arguments is a bad idea. You get into nested quoting levels and eval and a bunch of other dangerous/confusing syntactic stuff.
Use an array to build the find; you've already got the EXCLUDES in one.
Also, the repeated -not and -prune seems weird to me. I would write your command as something like this:
excludes=()
for dir in "${EXCLUDES[#]}"; do
excludes+=(-name "${dir}*" -prune -o)
done
find "${ANDROID_PATH}" -mindepth 1 -maxdepth 1 "${excludes[#]}" -type d -print
The upshot is, you want the argument to -name to be passed to find as a literal wildcard that find will expand, not a list of files returned by the shell's expansion, nor a string containing literal quotation marks. This is very hard to do if you try to build the command as a string, but trivial if you use an array.
Friends don't let friends build shell commands as strings.
When I run your script (named fin.sh) as:
bash -x fin.sh $HOME/tmp
one of the lines of trace output is:
find /Users/jleffler/tmp -mindepth 1 -maxdepth 1 -not -name '"doc*"' -prune -type d
Do you see the single quotes around the double quotes? That's bash trying to be helpful. I'm guessing that your "doesn't work" problem is that you still get directories under doc* included in the output; other than that, it seems to work for me.
How to fix that?
...it seems you've found a way to fix that...I'm not sure I'd trust it with a Bourne shell (but the Korn shell seems to agree with Bash), but it looks like it might work with Bash. I'm pretty sure this is something that changed during the last 30 years or so, but it is hard to prove that; getting hands on the old code is not easy.
I also wonder whether you need repeated -prune options if you have repeated excluded directories; I'm not sufficiently familiar with -prune to be sure.
Found the problem. Its with the escape sequence in the exclude_name_cmd_string.
Correct syntax should have been
exclude_name_cmd_string=${exclude_name_cmd_string}$(echo \
"-not -name ${dir}* -prune")
I want to make bash function in my .bash_profile that basically does a find ./ -name $1, very simple idea, seems not to work. My tries don't print things the right way i.e.:
find_alias() {
`find ./ -name $1 -print`
}
alias ff='find_alias $1'
The above if I do something like ff *.xml I get the following one liner:
bash: .pom.xml: Permission denied
The following after that:
find_alias() {
echo -e `find ./ -name $1 -print`
}
alias ff='find_alias $1'
does find them all, but puts the output of that onto one massive long line, what am I doing wrong here?
find_alias() {
find ./ -name $1 -print
}
You don't need, nor want, the backticks. That would try to execute what the find command returns.
Backticks make shell treat output of what's inside them as command that should be executed. If you tried ´echo "ls"´ then it would first execute echo "ls", take the output which is text ls and then execute it listing all files.
In your case you are executing textual result of find ./ -name *.xml -print which is a list of matched files. Of course it has no sense because matched file names (in most cases) are not commands.
The output you are getting means two things:
you tried to execute script from pom.xml (like if you typed
./pom.xml) - makes no sense
you don't have execution rights for
that file
So the simple solution for you problem, as #Mat suggested, is to remove backticks and let the output of find be displayed in your terminal.
I usually use like this
$ find -name testname.c
./dir1/dir2/testname.c
$ vi ./dir1/dir2/testname.c
it's to annoying to type file name with location again.
how can I do this with only one step?
I've tried
$ find -name testname.c | xargs vi
but I failed.
Use the -exec parameter to find.
$ find -name testname.c -exec vi {} \;
If your find returns multiple matches though, the files will be opened sequentially. That is, when you close one, it will open the next. You won't get them all queued up in buffers.
To get them all open in buffers, use:
$ vi $(find -name testname.c)
Is this really vi, by the way, and not Vim, to which vi is often aliased nowadays?
You can do it with the following commands in bash:
Either use
vi `find -name testname.c`
Or use
vi $(!!)
if you have already typed find -name testname.c
Edit: possible duplication: bash - automatically capture output of last executed command into a variable
The problem is xargs takes over all of vi's input there (and, having no other recourse, then passes on /dev/null to vi because the alternative is passing the rest of the file list), leaving no way for you to interact with it. You probably want to use a subcommand instead:
$ vi $(find -name testname.c)
Sadly there's no simple fc or r invocation that can do this for you easily after you've run the initial find, although it's easy enough to add the characters to both ends of the command after the fact.
My favorite solution is to use vim itself:
:args `find -name testname.c`
Incidentally, VIM has extended shell globbing builtin, so you can just say
:args **/testname.c
which will find recursively in the sub directory tree.
Not also, that VIM has filename completion on the commandline, so if you know you are really looking for a single file, try
:e **/test
and then press Tab (repeatedly) to cycle between any matchin filenames in the subdirectory tree.
For something a bit more robust than vi $(find -name testname.c) and the like, the following will protect against file names with whitespace and other interpreted shell characters (if you have newlines embedded in your file names, god help you). Inject this function into your shell environment:
# Find a file (or files) by name and open with vi.
function findvi()
{
declare -a fnames=()
readarray -t fnames < <(find . -name "$1" -print)
if [ "${#fnames[#]}" -gt 0 ]; then
vi "${fnames[#]}"
fi
}
Then use like
$ findvi Classname.java
I can't retrieve the way to define a shell alias (in bash) like this one :
alias suppr='/usr/bin/find . -name "*~" | xargs rm -f'
but with "*~" as a parameter of the alias.
I would like to use it like : suppr ".bak" or suppr "*.svn" etc...
(it's just a dummy example here)
Use a function:
suppr() {
/usr/bin/find . -name "$#" | xargs rm -f
}
In general, functions are more flexible and safer to use than aliases. In fact, many people argue that functions should always be used instead of aliases.
why not save your command as a script, and put the script in Path? you can name the script file as anything.