To rephrase - I want to use Bash command substitution and string substitution in the same line.
My actual commands are longer, but the ridiculous use of echo here is just a "substitution" for shortness and acts the same - with same errors ;)
I know we can use a Bash command to produce it's output string as a parameter for another command like this:
echo "$(echo "aahahah</ddd>")"
aahahah</ddd>
I also know we can remove last known part of a string like this:
var="aahahah</ddd>"; echo "${var%</ddd>}"
aahahah
I am trying to write a command where one command gives a string output, where I want to remove last part, which is known.
echo "${$(echo "aahahah</ddd>")%</ddd>}"
-bash: ${$(echo "aahahah</ddd>")%</ddd>}: bad substitution
It might be the order of things happening or substitution only works on variables or hardcoded strings. But I suspect just me missing something and it is possible.
How do I make it work?
Why doesn't it work?
When a dollar sign as in $word or equivalently ${word} is used, it asks for word's content. This is called parameter expansion, as per man bash.
You may write var="aahahah</ddd>"; echo "${var%</ddd>}": That expands var and performs a special suffix operation before returning the value.
However, you may not write echo "${$(echo "aahahah</ddd>")%</ddd>}" because there is nothing to expand once $(echo "aahahah</ddd>") is evaluated.
From man bash (my emphasis):
${parameter%word}
Remove matching suffix pattern. The word is expanded to produce a
pattern just as in pathname expansion. If the pattern
matches a trailing portion of the expanded value of parameter, then
the result of the expansion is the expanded value of parameter
with the shortest matching pattern (the ''%'' case) or the longest matching pattern (the ''%%'' case) deleted.
Combine your commands like this
var=$(echo "aahahah</ddd>")
echo ${var/'</ddd>'}
Related
I have files, for example
- public_00000.jpg
- public_00001.jpg
- ...
- public_00535.jpg
But I want to make these files as
- public_05674.jpg
- public_05675.jpg
- ...
- public_06209.jpg
I mean, I want to increase the number in the filename by +5674 on the whole.
How can I do this by Shell or Command??
Thanks ahead:)
Could you please try following.
for file in *.jpg
do
first_filename_part="${file%_*}"
last_filename_part="${file#*.}"
var="${file#*_}"
count="${var%.*}"
((count = count + 5674))
printf "%s %s %s_%05d.%s\n" "mv" $file $first_filename_part $count $last_filename_part
done
Above will only print the commands on screen like:
mv public_00000.jpg public_05674.jpg
Try running only 1 command First from above printed output on your terminal, once you are Happy with results try following then, since this will rename all the files.
for file in *.jpg
do
first_filename_part="${file%_*}"
last_filename_part="${file#*.}"
var="${file#*_}"
count="${var%.*}"
((count = count + 5674))
printf "%s %s %s_%05d.%s\n" "mv" $file $first_filename_part $count $last_filename_part | bash
done
From man page: I have used parameter expansion mechanism.
${parameter#word}
${parameter##word} Remove matching prefix pattern. The word is
expanded to produce a pattern just as in pathname expansion. If the
pattern matches the beginning of the value of parameter, then the
result of the expansion is the expanded value of parameter with the
shortest matching pattern (the #'' case) or the longest matching
pattern (the##'' case) deleted. If parameter is # or *, the
pattern removal operation is applied to each positional parame- ter in
turn, and the expansion is the resultant list. If parameter is an
array variable subscripted with # or *, the pattern removal
operation is applied to each member of the array in turn, and the
expansion is the resultant list.
${parameter%word}
${parameter%%word} Remove matching suffix pattern. The word is
expanded to produce a pattern just as in pathname expansion. If the
pattern matches a trailing portion of the expanded value of parameter,
then the result of the expansion is the expanded value of parameter
with the shortest matching pattern (the %'' case) or the longest
matching pattern (the%%'' case) deleted. If parameter is # or *,
the pattern removal operation is applied to each positional parameter
in turn, and the expansion is the resultant list. If parameter is an
array variable subscripted with # or *, the pattern removal operation
is applied to each member of the array in turn, and the expansion is
the resultant list.
You can use Perl rename like this to do an "evaluated substitution" - that's the e right at the end:
rename --dry-run 's|(\d+)|sprintf("%05d",$1+5674)|e' pub*jpg
Sample Output
'public_00000.jpg' would be renamed to 'public_05674.jpg'
'public_00001.jpg' would be renamed to 'public_05675.jpg'
In case you are unfamiliar with Perl, the command basically says:
rename "substitute|THIS|with THAT|" IN_THESE_FILENAMES
In your case, THIS is \d+ which means "one or more digits" and that is enclosed within parentheses to make a "capture group". That group can then be referred to in the substitution on the right side by $1 since it is the first capture group.
The THAT in your case is simply a print statement that prints the first capture group $1 incremented by 5674 in a field that is zero-padded to be 5 digits wide using %05d.
Using Perl rename has the benefits that:
you can do a "dry run" to see what it would do without actually doing anything
it will not clobber (overwrite) files without warning
it is fast - it doesn't create a process for sed and another for mv for every single file, it just starts a single Perl interpreter and makes a library call to rename each file
it will automagically create any intermediate directories needed, if you wish
you can use the full power of Perl to do as much substitution or calculation as you wish
Note for macOS users... Perl is installed on macOS by default, so if you use homebrew to install your packages, you just need:
brew install rename
Note for Linux users... there are several rename packages, the one I am referring to is sometimes called prename, or "Perl rename". That means, if you run file on the rename command, it should say it's a Perl script like this:
file $(which rename)
/usr/local/bin/rename: Perl script text executable
I'm trying to figure out what I thought would be a trivial issue in BASH, but I'm having difficulty finding the correct syntax. I want to loop over an array of values, one of them being an asterisk (*), I do not wish to have any wildcard expansion happening during the process.
WHITELIST_DOMAINS="* *.foo.com *.bar.com"
for domain in $WHITELIST_DOMAINS
do
echo "$domain"
done
I have the above, and I'm trying to get the following output:
*
*.foo.com
*.bar.com
Instead of the above, I get a directory listing on the current directory, followed by *.foo.com and *.bar.com
I know I need some escaping or quoting somewhere.. the early morning haze is still thick on my brain.
I've reviewed these questions:
How to escape wildcard expansion in a variable in bash?
Stop shell wildcard character expansion?
Your problem is that you want an array, but you wrote a single string that contains the elements with spaces between them. Use an array instead.
WHITELIST_DOMAINS=('*' '*.foo.com' '*.bar.com')
Always use double quotes around variable substitutions (i.e. "$foo"), otherwise the shell splits the the value of the variable into separate words and treats each word as a filename wildcard pattern. The same goes for command substitution: "$(somecommand)". For an array variable, use "${array[#]}" to expand to the list of the elements of the array.
for domain in "${WHITELIST_DOMAINS[#]}"
do
echo "$domain"
done
For more information, see the bash FAQ about arrays.
You can use array to store them:
array=('*' '*.foo.com' '*.bar.com')
for i in "${array[#]}"
do
echo "$i"
done
I have a Bash script that takes in a directory as a parameter, and after some processing will do some output based on the files in that directory.
The command would be like the following, where dir is a directory with the following structure inside
dir/foo
dir/bob
dir/haha
dir/bar
dir/sub-dir
dir/sub-dir/joe
> myscript ~/files/stuff/dir
After some processing, I'd like the output to be something like this
foo
bar
sub-dir/joe
The code I have to remove the path passed in is the following:
shopt -s extglob
for file in $files ; do
filename=${file#${1}?(/)}
This gets me to the following, but for some reason the optional / is not being taken care of. Thus, my output looks like this:
/foo
/bar
/sub-dir/joe
The reason I'm making it optional is because if the user runs the command
> myscript ~/files/stuff/dir/
I want it to still work. And, as it stands, if I run that command with the trailing slash, it outputs as desired.
So, why does my ?(/) not work? Based on everything I've read, that should be the right syntax, and I've tried a few other variations as well, all to no avail.
Thanks.
that other guy's helpful answer solves your immediate problem, but there are two things worth nothing:
enumerating filenames with an unquoted string variable (for file in $files) is ill-advised, as sjsam's helpful answer points out: it will break with filenames with embedded spaces and filenames that look like globs; as stated, storing filenames in an array is the robust choice.
there is no strict need to change global shell option shopt -s extglob: parameter expansions can be nested, so the following would work without changing shell options:
# Sample values:
file='dir/sub-dir/joe'
set -- 'dir/' # set $1; value 'dir' would have the same effect.
filename=${file#${1%/}} # -> '/sub-dir/joe'
The inner parameter expansion, ${1%/}, removes a trailing (%) / from $1, if any.
I suggested you change files to an array which is a possible workaround for non-standard filenames that may contain spaces.
files=("dir/A/B" "dir/B" "dir/C")
for filename in "${files[#]}"
do
echo ${filename##dir/} #replace dir/ with your param.
done
Output
A/B
B
C
Here's the documentation from man bash under "Parameter Expansion":
${parameter#word}
${parameter##word}
Remove matching prefix pattern. The word is
expanded to produce a pattern just as in pathname
expansion. If the pattern matches the beginning of
the value of parameter, then the result of the
expansion is the expanded value of parameter with
the shortest matching pattern (the ``#'' case) or
the longest matching pattern (the ``##'' case)
deleted.
Since # tries to delete the shortest match, it will never include any trailing optional parts.
You can just use ## instead:
filename=${file##${1}?(/)}
Depending on what your script does and how it works, you can also just rewrite it to cd to the directory to always work with paths relative to .
I have a script which contains the following line:
propFile="${0%/*}/anteater.properties"
What does "${0%/*}" mean?
This command gives a path to the script - but there is a spaces at path and script can't find this file - how to deal with it?
The % operator in variable expansion removes the matching suffix pattern given to it. So ${0%/*} takes the variable $0, and removes all matching /* at the end. This is equivalent to the command dirname, which, when given a path, returns the parent directory of that path.
In order to deal with spaces in bash variable, whenever expanding the variable (i.e. whenever you write $var), you should quote it. In short, always use "$var" instead of just $var.
Consider reading shell parameter expansion and variable quoting in the bash manual to learn more about these two subjects.
strips the suffix matching /*, i.e. everything after last slash including the slash itself.
quote it wherever you use it (cat "$propFile").
A contrived example... given
FOO="/foo/bar/baz"
this works (in bash)
BAR=$(basename $FOO) # result is BAR="baz"
BAZ=${BAR:0:1} # result is BAZ="b"
this doesn't
BAZ=${$(basename $FOO):0:1} # result is bad substitution
My question is which rule causes this [subshell substitution] to evaluate incorrectly? And what is the correct way, if any, to do this in 1 hop?
First off, note that when you say this:
BAR=$(basename $FOO) # result is BAR="baz"
BAZ=${BAR:0:1} # result is BAZ="b"
the first bit in the construct for BAZ is BAR and not the value that you want to take the first character of. So even if bash allowed variable names to contain arbitrary characters your result in the second expression wouldn't be what you want.
However, as to the rule that's preventing this, allow me to quote from the bash man page:
DEFINITIONS
The following definitions are used throughout the rest of this docu‐
ment.
blank A space or tab.
word A sequence of characters considered as a single unit by the
shell. Also known as a token.
name A word consisting only of alphanumeric characters and under‐
scores, and beginning with an alphabetic character or an under‐
score. Also referred to as an identifier.
Then a bit later:
PARAMETERS
A parameter is an entity that stores values. It can be a name, a num‐
ber, or one of the special characters listed below under Special Param‐
eters. A variable is a parameter denoted by a name. A variable has a
value and zero or more attributes. Attributes are assigned using the
declare builtin command (see declare below in SHELL BUILTIN COMMANDS).
And later when it defines the syntax you're asking about:
${parameter:offset:length}
Substring Expansion. Expands to up to length characters of
parameter starting at the character specified by offset.
So the rules as articulated in the manpage say that the ${foo:x:y} construct must have a parameter as the first part, and that a parameter can only be a name, a number, or one of the few special parameter characters. $(basename $FOO) is not one of the allowed possibilities for a parameter.
As for a way to do this in one assignment, use a pipe to other commands as mentioned in other responses.
Modified forms of parameter substitution such as ${parameter#word} can only modify a parameter, not an arbitrary word.
In this case, you might pipe the output of basename to a dd command, like
BAR=$(basename -- "$FOO" | dd bs=1 count=1 2>/dev/null)
(If you want a higher count, increase count and not bs, otherwise you may get fewer bytes than requested.)
In the general case, there is no way to do things like this in one assignment.
It fails because ${BAR:0:1} is a variable expansion. Bash expects to see a variable name after ${, not a value.
I'm not aware of a way to do it in a single expression.
As others have said, the first parameter of ${} needs to be a variable name. But you can use another subshell to approximate what you're trying to do.
Instead of:
BAZ=${$(basename $FOO):0:1} # result is bad substitution
Use:
BAZ=$(_TMP=$(basename $FOO); echo ${_TMP:0:1}) # this works
A contrived solution for your contrived example:
BAZ=$(expr $(basename $FOO) : '\(.\)')
as in
$ FOO=/abc/def/ghi/jkl
$ BAZ=$(expr $(basename $FOO) : '\(.\)')
$ echo $BAZ
j
${string:0:1},string must be a variable name
for example:
FOO="/foo/bar/baz"
baz="foo"
BAZ=eval echo '${'"$(basename $FOO)"':0:1}'
echo $BAZ
the result is 'f'