if folders exist - by using wildcard [duplicate] - bash

This question already has answers here:
Test whether a glob has any matches in Bash
(22 answers)
Closed 4 years ago.
or "How to handle prefixed folder names?"
Inside a folder I have two (or more) foo_* folders
foo_0
foo_1
What I'm trying to achieve is to
perform an action if there's 1 or more foo_* folders
Use a wildcard *
Currently I'm doing it this way (going directly to check if directory foo_0 exists):
prefix=foo_
if [ -d "./${prefix}0/" ]; then
printf "foo_0 folder found!"
# delete all foo_* folders
fi
Having directories 0-to-N so the above works, but i'm not sure I'll always have a foo_0 folder...
I'd like to do use a wildcard:
prefix=foo_
if [ -d "./${prefix}*/" ]; then # By using wildcard...
printf "One or more foo_* folders found!" # this never prints
# delete all foo_* folders
fi
I've read that a wildcard * inside quotes loses its powers, but placing it outside quotes throws :
if [ -d "./${prefix}"* ] <<< ERROR: binary operator expected
Or is it possible to use some sort of regex like? ./foo_\d+ ?
The only solution I don't (arguably) like, is by using set
set -- foo_*
if [ -d $1 ]; then
printf "foo_* found!"
fi
but it wipes program arguments.
Is there any other nice solution to this I'm missing?

I think a nice solution for pretty much all such cases is to use ls in the test, in that it often works quite simply:
if [ -n "$(ls -d foo_*)" ]; then ... If you want to do more regexp-like matching you can shopt -s extglob and then match with ls foo_+([0-9]).
There's also an all-bash solution using several shell options, but it's not as easy to remember, so I'll leave that to another poster ;-)
EDIT: As #PesaThe pointed out, using ls foo_* would fail if there's only one empty matching directory, as just the empty contents of that directory would get listed and ls foo_* would not only match directories, so it's preferable to use -d.

Related

How to find a file without knowing its extension? [duplicate]

This question already has answers here:
Test whether a glob has any matches in Bash
(22 answers)
Closed 1 year ago.
Currently I'm writing a script and I need to check if a certain file exists in a directory without knowing the extension of the file. I tried this:
if [[ -f "$2"* ]]
but that didn't work. Does anyone know how I could do this?
-f expects a single argument, and AFIK, you don't get filename expansion in this context anyway.
Although cumbersome, the best I can think of, is to produce an array of all matching filenames, i.e.
shopt -s nullglob
files=( "$2".* )
and test the size of the array. If it is larger than 1, you have more than one candidate., i.e.
if (( ${#files[*]} > 1 ))
then
....
fi
If the size is 1, ${files[0]} gives you the desired one. If the size is 0 (which can only happen if you turn on nullglob), no files are matching.
Don't forget to reset nullglob afterwards, if you don't need it anymore.
In shell, you have to iterate each file globbing pattern's match and test each one individually.
Here is how you could do it with standard POSIX shell syntax:
#!/usr/bin/env sh
# Boolean flag to check if a file match was found
found_flag=0
# Iterate all matches
for match in "$2."*; do
# In shell, when no match is found, the pattern itself is returned.
# To exclude the pattern, check a file of this name actually exists
# and is an actual file
if [ -f "$match" ]; then
found_flag=1
printf 'Found file: %s\n' "$match"
fi
done
if [ $found_flag -eq 0 ]; then
printf 'No file matching: %s.*\n' "$2" >&2
exit 1
fi
You can use find:
find ./ -name "<filename>.*" -exec <do_something> {} \;
<filename> is the filename without extension, do_something the command you want to launch, {} is the placeholder of the filename.

The `ls` command is interpreting my directory with spaces as multiple directories [duplicate]

This question already has answers here:
Why does shell ignore quoting characters in arguments passed to it through variables? [duplicate]
(3 answers)
Closed 3 years ago.
I'm trying to pass a dynamic command that executes ls as a string that lists the files of a directory that contains spaces. However, my ls command always interprets my one directory containing spaces as multiple directories no matter what I do.
Consider the following simplified version of my shell script:
#!/bin/sh
export WORK_DIR="/Users/jthoms/Dropbox (My Company)/backup-jthoms/Work"
echo "WORK_DIR=$WORK_DIR"
export LS_CMD="ls -A \"$WORK_DIR/dependencies/apache-tomcat-8.0.45/logs\""
echo "LS_CMD=$LS_CMD"
if [ -n "$($LS_CMD)" ]
then
echo "### Removing all logs"
sudo rm "$WORK_DIR/dependencies/apache-tomcat-8.0.45/logs/*"
else
echo "### Not removing all logs"
fi
This script results in the following output:
WORK_DIR=/Users/jthoms/Dropbox (My Company)/backup-jthoms/Work
LS_CMD=ls -A "/Users/jthoms/Dropbox (My Company)/backup-jthoms/Work/dependencies/apache-tomcat-8.0.45/logs"
ls: "/Users/jthoms/Dropbox: No such file or directory
ls: (My: No such file or directory
ls: Company)/backup-jthoms/Work/dependencies/apache-tomcat-8.0.45/logs": No such file or directory
### Not removing all logs
How can I correctly escape my shell variables so that the ls command interprets my directory as a single directory containing spaces instead of multiple directories?
I recently changed this script which used to work fine for directories containing no spaces but now doesn't work for this new case. I'm working on Bash on MacOSX. I have tried various forms of escaping, various Google searches and searching for similar questions here on SO but to no avail. Please help.
Variables are for data. Functions are for code.
# There's no apparent need to export this shell variable.
WORK_DIR="/Users/jthoms/Dropbox (My Company)/backup-jthoms/Work"
echo "WORK_DIR=$WORK_DIR"
ls_cmd () {
ls -A "$1"/dependencies/apache-tomcat-8.0.45/logs
}
if [ -n "$(ls_cmd "$WORK_DIR")" ]; then
then
echo "### Removing all logs"
sudo rm "$WORK_DIR/dependencies/apache-tomcat-8.0.45/logs/"*
else
echo "### Not removing all logs"
fi
However, you don't need ls for this at all (and in general, you should avoid parsing the output of ls). For example,
find "$WORK_DIR/dependencies/apache-tomcat-8.0.45/logs/" -type f -exec rm -rf {} +
You could use
# ...
if [ -n "$(eval "$LS_CMD")" ]
# ...
See http://mywiki.wooledge.org/BashFAQ/050
Or even
# ...
if [ -n "$(bash -c "$LS_CMD")" ]
# ...
But you are probably better off using a dedicated function and/or even something more specific to your problem (using find instead of ls is usually a good idea in these cases, see some examples in the answers for this question).
Use arrays, not strings, to store commands:
ls_cmd=(ls -A "$WORK_DIR/dependencies/apache-tomcat-8.0.45/logs")
echo "ls_cmd=${ls_cmd[*]}"
if [ -n "$("${ls_cmd[#]}")" ]; then …
(The syntax highlighting on the last line is incorrect and misleading: we’re not unquoting ${ls_cmd[#]}; in reality, we are using nested quotes via a subshell here.)
That way, word splitting won’t interfere with your command.
Note that you can’t export arrays. But you don’t need to, in your code.
As others have noted, it’s often a better idea to use functions here. More importantly, you shouldn’t parse the output of ls. There are always better alternatives.

Shell Script- Traversing sub directories using shell scripts [duplicate]

This question already has answers here:
Bash: how to traverse directory structure and execute commands?
(6 answers)
Closed 4 years ago.
I am trying to traverse through the sub directories under current directory. There are certain files that i want to access and process inside each sub--directories. Can anyone help how can I access files inside sub directories?
"
for dir in /home/ayushi/perfios/fraud_stmt/*;
do echo $dir;
done;
"
This above script will echo all the sub directories. but instead of echoing I want to go inside the directories and access files that are present inside it.
find /home/ayushi/perfios/fraud_stmt/ -type f | while read fname; do
: do something on $fname here
done
This will search for all files (i.e. not actual directories) from the specified directory downwards. Note that you should enclose "$fname" in double quotes, in case it contains spaces or other "odd" characters.
An example using a recursive function
process_file() {
echo "$1"
}
rec_traverse() {
local file_or_dir
for file_or_dir in "$1"/*; do
[[ -d $file_or_dir ]] && rec_traverse "$file_or_dir"
[[ -f $file_or_dir ]] && process_file "$file_or_dir"
done
}
rec_traverse /home/ayushi/perfios/fraud_stmt
process_file can be changed to do something on file.
"$1"/* may be changed to "$1"/* "$1"/.* to match hidden directories but in this case special hard linked directories . and .. must be filtered to avoid infinite loop.

How to make a vim variable determinable by bash? [duplicate]

This question already has answers here:
How to check if a files exists in a specific directory in a bash script?
(3 answers)
Closed 4 years ago.
I'm not sure how to word my question exactly...
I have the code
if grep "mynamefolder" /vol/Homefs/
then
echo "yup"
else
echo "nope"
fi
which gives me the output
grep: /vol/Homefs/: Is a directory
nope
The sh file containing the code and the directory I'm targeting are not in the same directory (if that makes sense).
I want to find the words myfoldername inside /vol/Homefs/ without going through any subdirectories. Doing grep -d skip, which I hoped would "skip" subdirectories and focus only directories, just gives me nope even though the folder/file/word I'm testing it on does exist.
Edit: I forgot to mention that I would also like mynamefolder to be a variable that I can write in putty, something like
./file spaing and spaing being the replacement for myfoldername.
I'm not sure if I did good enough explaining, let me know!
You just want
if [ -e /vol/Homefs/"$1" ]; then
echo yup
else
echo nope
fi
The [ command, with the -e operator, tests if the named file entry exists.
vim is not involved, and grep is not needed.
If you're insisting on using grep, you should know grep doesn't work on directories. You can convert the directory listing to a string.
echo /vol/Homefs/* | grep mynamefolder

grep files based on name prefixes

I have a question on how to approach a problem I've been trying to tackle at multiple points over the past month. The scenario is like so:
I have a a base directory with multiple sub-directories all following the same sub-directory format:
A/{B1,B2,B3} where all B* have a pipeline/results/ directory structure under them.
All of these results directories have multiple *.xyz files in them. These *.xyz files have a certain hierarchy based on their naming prefixes. The naming prefixes in turn depend on how far they've been processed. They could be, for example, select.xyz, select.copy.xyz, and select.copy.paste.xyz, where the operations are select, copy and paste. What I wish to do is write a ls | grep or a find that picks these files based on their processing levels.
EDIT:
The processing pipeline goes select -> copy -> paste. The "most processed" file would be the one with the most of those stages as prefixes in its filename. i.e. select.copy.paste.xyz is more processed than select.copy, which in turn is more processed than select.xyz
For example, let's say
B1/pipeline/results/ has select.xyz and select.copy.xyz,
B2/pipeline/results/ has select.xyz
B3/pipeline/results/ has select.xyz, select.copy.xyz, and select.copy.paste.xyz
How can I write a ls | grep/find that picks the most processed file from each subdirectory? This should give me B1/pipeline/results/select.copy.xyz, B2/pipeline/results/select.xyz and B3/pipeline/results/select.copy.paste.xyz.
Any pointer on how I can think about an approach would help. Thank you!
For this answer, we will ignore the upper part A/B{1,2,3} of the directory structure. All files in some .../pipeline/results/ directory will be considered, even if the directory is A/B1/doNotIncludeMe/forbidden/pipeline/results. We assume that the file extension xyz is constant.
A simple solution would be to loop over the directories and check whether the files exist from back to front. That is, check if select.copy.paste.xyz exists first. In case the file does not exist, check if select.copy.xyz exists and so on. A script for this could look like the following:
#! /bin/bash
# print paths of the most processed files
shopt -s globstar nullglob
for d in **/pipeline/result; do
if [ -f "$d/select.copy.paste.xyz" ]; then
echo "$d/select.copy.paste.xyz"
elif [ -f "$d/select.copy.xyz" ]; then
echo "$d/select.copy.xyz"
elif [ -f "$d/select.xyz" ]; then
echo "$d/select.xyz"
else
# there is no file at all
fi
done
It does the job, but is not very nice. We can do better!
#! /bin/bash
# print paths of the most processed files
shopt -s globstar nullglob
for dir in **/pipeline/result; do
for file in "$dir"/select{.copy{.paste,},}.xyz; do
[ -f "$file" ] && echo "$file" && break
done
done
The second script does exactly the same thing as the first one, but is easier to maintain, adapt, and so on. Both scripts work with file and directory names that contain spaces or even newlines.
In case you don't have whitespace in your paths, the following (hacky, but loop-free) script can also be used.
#! /bin/bash
# print paths of the most processed files
shopt -s globstar nullglob
files=(**/pipeline/result/select{.copy{.paste,},}.xyz)
printf '%s\n' "${files[#]}" | sed -r 's#(.*/)#\1 #' | sort -usk1,1 | tr -d ' '

Resources