I have a script which contains the following line:
propFile="${0%/*}/anteater.properties"
What does "${0%/*}" mean?
This command gives a path to the script - but there is a spaces at path and script can't find this file - how to deal with it?
The % operator in variable expansion removes the matching suffix pattern given to it. So ${0%/*} takes the variable $0, and removes all matching /* at the end. This is equivalent to the command dirname, which, when given a path, returns the parent directory of that path.
In order to deal with spaces in bash variable, whenever expanding the variable (i.e. whenever you write $var), you should quote it. In short, always use "$var" instead of just $var.
Consider reading shell parameter expansion and variable quoting in the bash manual to learn more about these two subjects.
strips the suffix matching /*, i.e. everything after last slash including the slash itself.
quote it wherever you use it (cat "$propFile").
Related
I have a bash script that I want to add a backslash in front of all underscores. The script searches for all files in a directory and saves the file name to a variable file. Then in the variable I want to replace every instance of _ with \_.
I have looked at several questions on sed about search and replace as well as how to treat special characters, but none of them seemed to apply correctly to this scenario.
#!/bin/bash
file=some_file_name.f90 # I want this to read as some\_file\_name.f90
# I have tried the following (and some more i didnt keep track of)
fileWithEscapedUnderscores=$(sed 's/_/\\_/g' <<<$file)
fileWithEscapedUnderscores=$(sed 's/_/\_/g' <<<$file)
fileWithEscapedUnderscores=$(sed 's/_/\\\_/g' <<<$file)
fileWithEscapedUnderscores=${file/_/\_/}
It seems like I need to escape the backslash. However, if I do that I can get the backslash but no underscore. I also tried simply inserting a backslash before the underscore, but that also had issues.
The simple and obvious solution is to escape or quote the backslash in the parameter expansion.
You also had the slashes wrong; your attempt would merely replace the first one, with the literal string \_/.
To recap, the syntax is ${variable/pattern/replacement} to replace the first occurrence of pattern, and ${variable//pattern/replacement} to replace all occurrences.
fileWithEscapedUnderscores=${file//_/\\_}
# or
fileWithEscapedUnderscores=${file//_/'\_'}
Your first sed attempt should have worked, too; but avoid calling an external process when you can use shell builtins.
Separately, probably take care to use quotes around variables with file names, though it would not matter in your example; see also When to wrap quotes around a shell variable
To rephrase - I want to use Bash command substitution and string substitution in the same line.
My actual commands are longer, but the ridiculous use of echo here is just a "substitution" for shortness and acts the same - with same errors ;)
I know we can use a Bash command to produce it's output string as a parameter for another command like this:
echo "$(echo "aahahah</ddd>")"
aahahah</ddd>
I also know we can remove last known part of a string like this:
var="aahahah</ddd>"; echo "${var%</ddd>}"
aahahah
I am trying to write a command where one command gives a string output, where I want to remove last part, which is known.
echo "${$(echo "aahahah</ddd>")%</ddd>}"
-bash: ${$(echo "aahahah</ddd>")%</ddd>}: bad substitution
It might be the order of things happening or substitution only works on variables or hardcoded strings. But I suspect just me missing something and it is possible.
How do I make it work?
Why doesn't it work?
When a dollar sign as in $word or equivalently ${word} is used, it asks for word's content. This is called parameter expansion, as per man bash.
You may write var="aahahah</ddd>"; echo "${var%</ddd>}": That expands var and performs a special suffix operation before returning the value.
However, you may not write echo "${$(echo "aahahah</ddd>")%</ddd>}" because there is nothing to expand once $(echo "aahahah</ddd>") is evaluated.
From man bash (my emphasis):
${parameter%word}
Remove matching suffix pattern. The word is expanded to produce a
pattern just as in pathname expansion. If the pattern
matches a trailing portion of the expanded value of parameter, then
the result of the expansion is the expanded value of parameter
with the shortest matching pattern (the ''%'' case) or the longest matching pattern (the ''%%'' case) deleted.
Combine your commands like this
var=$(echo "aahahah</ddd>")
echo ${var/'</ddd>'}
I have files, for example
- public_00000.jpg
- public_00001.jpg
- ...
- public_00535.jpg
But I want to make these files as
- public_05674.jpg
- public_05675.jpg
- ...
- public_06209.jpg
I mean, I want to increase the number in the filename by +5674 on the whole.
How can I do this by Shell or Command??
Thanks ahead:)
Could you please try following.
for file in *.jpg
do
first_filename_part="${file%_*}"
last_filename_part="${file#*.}"
var="${file#*_}"
count="${var%.*}"
((count = count + 5674))
printf "%s %s %s_%05d.%s\n" "mv" $file $first_filename_part $count $last_filename_part
done
Above will only print the commands on screen like:
mv public_00000.jpg public_05674.jpg
Try running only 1 command First from above printed output on your terminal, once you are Happy with results try following then, since this will rename all the files.
for file in *.jpg
do
first_filename_part="${file%_*}"
last_filename_part="${file#*.}"
var="${file#*_}"
count="${var%.*}"
((count = count + 5674))
printf "%s %s %s_%05d.%s\n" "mv" $file $first_filename_part $count $last_filename_part | bash
done
From man page: I have used parameter expansion mechanism.
${parameter#word}
${parameter##word} Remove matching prefix pattern. The word is
expanded to produce a pattern just as in pathname expansion. If the
pattern matches the beginning of the value of parameter, then the
result of the expansion is the expanded value of parameter with the
shortest matching pattern (the #'' case) or the longest matching
pattern (the##'' case) deleted. If parameter is # or *, the
pattern removal operation is applied to each positional parame- ter in
turn, and the expansion is the resultant list. If parameter is an
array variable subscripted with # or *, the pattern removal
operation is applied to each member of the array in turn, and the
expansion is the resultant list.
${parameter%word}
${parameter%%word} Remove matching suffix pattern. The word is
expanded to produce a pattern just as in pathname expansion. If the
pattern matches a trailing portion of the expanded value of parameter,
then the result of the expansion is the expanded value of parameter
with the shortest matching pattern (the %'' case) or the longest
matching pattern (the%%'' case) deleted. If parameter is # or *,
the pattern removal operation is applied to each positional parameter
in turn, and the expansion is the resultant list. If parameter is an
array variable subscripted with # or *, the pattern removal operation
is applied to each member of the array in turn, and the expansion is
the resultant list.
You can use Perl rename like this to do an "evaluated substitution" - that's the e right at the end:
rename --dry-run 's|(\d+)|sprintf("%05d",$1+5674)|e' pub*jpg
Sample Output
'public_00000.jpg' would be renamed to 'public_05674.jpg'
'public_00001.jpg' would be renamed to 'public_05675.jpg'
In case you are unfamiliar with Perl, the command basically says:
rename "substitute|THIS|with THAT|" IN_THESE_FILENAMES
In your case, THIS is \d+ which means "one or more digits" and that is enclosed within parentheses to make a "capture group". That group can then be referred to in the substitution on the right side by $1 since it is the first capture group.
The THAT in your case is simply a print statement that prints the first capture group $1 incremented by 5674 in a field that is zero-padded to be 5 digits wide using %05d.
Using Perl rename has the benefits that:
you can do a "dry run" to see what it would do without actually doing anything
it will not clobber (overwrite) files without warning
it is fast - it doesn't create a process for sed and another for mv for every single file, it just starts a single Perl interpreter and makes a library call to rename each file
it will automagically create any intermediate directories needed, if you wish
you can use the full power of Perl to do as much substitution or calculation as you wish
Note for macOS users... Perl is installed on macOS by default, so if you use homebrew to install your packages, you just need:
brew install rename
Note for Linux users... there are several rename packages, the one I am referring to is sometimes called prename, or "Perl rename". That means, if you run file on the rename command, it should say it's a Perl script like this:
file $(which rename)
/usr/local/bin/rename: Perl script text executable
Here's a sample: ../c/Users/test
I stil have no idea how to achieve this zsh's prompt, if possible also how it will be in bash?
EDIT: the nearest means my pwd will show for i.e: /home/abc/c/Users/test.
If I'm under /home/abc/c or /home/abc or /home, prompt should be /home/abc/c> or /home/abc> or /home>
So only current path that excesses three folders will have .. appended in front and the three nearest folders.
ZSH
In zsh this can be achieved entirely with built-in features. You just have to place
%(4/|../%3d|%d)
in your PROMPT parameter (also known as PS1).
For example:
PROMPT='[%m#%n %(4/|../%3d|%d)]%# '
Would get you something like
[abc#machine ../c/Users/test]%
when the current directory is /home/abc/c/Users/test.
Explanation:
%(x|true-text|false-text): x represents a test, if it evaluates to true it zsh will print what is placed as true-text else it will print the false-text.
4/ is true if the current absolute path has at least 4 elements. For example /home/abc/c/Users/test has 5 elements
So, if the current path has 4 or more elements, the output is ../%3d, where %3d will be replaced with the last 3 elements of the current path. For example ../c/Users/test.
If the current path has less than 4 elements, the output is %d, which will be replaced by the full current path.
BASH
Method 1: simple but not a perfect match
In bash (since version 4) you can achieve very similar results by setting PROMPT_DIRTRIM=3 and placing \w in PS1. This will also only display the last 3 elements of the current path, preceded by either ~/.../ or .../. Which depends on whether the current directory is within the user's home directory.
For example:
PS1='[\u#\h \w]\$ '
PROMPT_DIRTRIM=3
would get you
[abc#machine ~/.../c/Users/test]$
when the current working directory is /home/abc/c/Users/test and
[abc#machine .../share/doc/sometool]$
when the current working directory is /usr/local/share/doc/sometool.
Method 2: complicated but works as asked
For an exact match place the following in your PS1:
$(a=${PWD%/*} a=${a%/*} a=${a%/*}; echo ${PWD/#$a/${a:+..}})
For example
PS1='[\u#\h $(a=${PWD%/*} a=${a%/*} a=${a%/*}; echo ${PWD/#$a/${a:+..}})]\$ '
Important: At least the part that generates the path output needs to be fully quoted, e.g. surrounded by single quotes. Otherwise it would be evaluated at the time of definition and not when the prompt is displayed.
Explanation:
$(command): This is called Command Substitution. It will run command and then be substituted by the resulting output.
The parameter PWD contains the current working directory.
a=${PWD%/*}: The shortest possible match to /* will be removed from the end of $PWD and the resulting value will be assigned to parameter a. That is, the last path element will be removed from $PWD.
a=${PWD%/*} a=${a%/*} a=${a%/*}: this removes the last three path elements from $PWD. If $PWD has three or less elements, then a will be empty at the end. If there are more than three elements, then a contains all element you do not want to be shown, i.e. the ones you want to replace with ...
(Note: While a=${PWD%/*/*/*} also removes the last three path elements, it does not work as intended, if there are less than three elements. In that case the end of $PWD would not match to /*/*/* and nothing would be removed, leaving $a to be identical to $PWD.)
${a:+..}: if a is defined and not null this will be substituted by .., otherwise nothing is substituted. This means if there are path elements to be removed, then ${a:+..} will evaluate to ...
${PWD/#$a/${a:+..}}): if the beginning of $PWD matches $a then it will be replaced by the subtstitution of ${a:+..}. Essentially, if a contains any path elements, than they will be replaced by .., otherwise nothing will be changed.
echo: As this all happens within a Command Substitution, echo is needed in order to output the shortened path.
this seems to do the trick:
PS1='$(pwd|sed -r "sx.+(/[^/]*/[^/]*/[^/]*)\$x..\1x" ):'
eg:
jasen#gonzo:/var/spool/news/comp/lang/c/moderated$ PS1='$(pwd|sed -r "sx.+(/[^/]*/[^/]*/[^/]*)\$x..\1x" ):'
../lang/c/moderated:cd
/home/jasen:
How it works:
PS1 is expanded to produce the prompt.
I use command substitusion $( ... ) to insert output of a command into the prompt.
the command itself is a pipeline
first pwd pints out the current directory
it's output is piped (|) to sed in extended (-r) regular expression mode. sed is given the command.
"sx.+(/[^/]*/[^/]*/[^/]*)\$x..\1x"
this an s substitution command
in this command the symbol that follows the s is the separator here I used x
the phrase [^/]* indicates a seaquence of zero or more non-slashes (like the name of a directory) while the other slashes / represent actual slashes and .+ matches anything at all (but not nothing). and the $ represents end of line.
so starting from dollar it matches lines that end like /name/name/name
the bit after the second X where it says ..\1 is what to replace the match with. in this case .. followed by the bit contained in the bit matched by the parenthesised pattern. ../name/name/name
I have a Bash script that takes in a directory as a parameter, and after some processing will do some output based on the files in that directory.
The command would be like the following, where dir is a directory with the following structure inside
dir/foo
dir/bob
dir/haha
dir/bar
dir/sub-dir
dir/sub-dir/joe
> myscript ~/files/stuff/dir
After some processing, I'd like the output to be something like this
foo
bar
sub-dir/joe
The code I have to remove the path passed in is the following:
shopt -s extglob
for file in $files ; do
filename=${file#${1}?(/)}
This gets me to the following, but for some reason the optional / is not being taken care of. Thus, my output looks like this:
/foo
/bar
/sub-dir/joe
The reason I'm making it optional is because if the user runs the command
> myscript ~/files/stuff/dir/
I want it to still work. And, as it stands, if I run that command with the trailing slash, it outputs as desired.
So, why does my ?(/) not work? Based on everything I've read, that should be the right syntax, and I've tried a few other variations as well, all to no avail.
Thanks.
that other guy's helpful answer solves your immediate problem, but there are two things worth nothing:
enumerating filenames with an unquoted string variable (for file in $files) is ill-advised, as sjsam's helpful answer points out: it will break with filenames with embedded spaces and filenames that look like globs; as stated, storing filenames in an array is the robust choice.
there is no strict need to change global shell option shopt -s extglob: parameter expansions can be nested, so the following would work without changing shell options:
# Sample values:
file='dir/sub-dir/joe'
set -- 'dir/' # set $1; value 'dir' would have the same effect.
filename=${file#${1%/}} # -> '/sub-dir/joe'
The inner parameter expansion, ${1%/}, removes a trailing (%) / from $1, if any.
I suggested you change files to an array which is a possible workaround for non-standard filenames that may contain spaces.
files=("dir/A/B" "dir/B" "dir/C")
for filename in "${files[#]}"
do
echo ${filename##dir/} #replace dir/ with your param.
done
Output
A/B
B
C
Here's the documentation from man bash under "Parameter Expansion":
${parameter#word}
${parameter##word}
Remove matching prefix pattern. The word is
expanded to produce a pattern just as in pathname
expansion. If the pattern matches the beginning of
the value of parameter, then the result of the
expansion is the expanded value of parameter with
the shortest matching pattern (the ``#'' case) or
the longest matching pattern (the ``##'' case)
deleted.
Since # tries to delete the shortest match, it will never include any trailing optional parts.
You can just use ## instead:
filename=${file##${1}?(/)}
Depending on what your script does and how it works, you can also just rewrite it to cd to the directory to always work with paths relative to .