I'm trying to remove everything after a specific_string in a path string in Bash. I've tried using sed to no avail so far.
variable="specific_string"
input_string="/path/to/some/specific_string/specific_string.something/specific_string.something-else"
output=$(sed 's/$variable//' $input_string)
Output should be "/path/to/some/specific_string/"
Would be better if I didn't have to use commands such as sed!
The Problems
There are many problems
Variables are not evaluated inside single quotes. 's/$variable//' will be treated as a literal string, which does not contain specific_string
sed can modify text from files or STDIN, but not text given via parameters. With sed 's/...//' $input_string the /path/to/some/specific_string/.../file is opened and its content is read, instead of the path itself.
s/string// deletes only string, not the words afterwards.
Also remember to double quote your variables. cmd $variable is dangerous if the variable contains spaces. cmd "$variable" is safe.
Sed Solution
output="$(sed "s/$variable.*/$variable/" <<< "$input_string")"
GNU Grep Solution
output="$(grep -Po "^.*?$variable" <<< "$input_string")"
Pure Bash Solution
output="${input_string%%$variable*}$variable"
If you want to remove everything after "specific_string" it will remove the "/" also as it does with the following example:
output=$(echo $input_string|sed "s/${variable}.*$/${variable}/")
try with simple sed:
variable="specific_string"
input_string="/path/to/some/specific_string/specific_string.something/specific_string.something-else"
output=$(echo "$input_string" | sed "s/\(.*$variable\/\).*/\1/")
Output of variable output will be as follows.
echo $output
/path/to/some/specific_string/
Related
I am trying to change the values in a text file using sed in a Bash script with the line,
sed 's/draw($prev_number;n_)/draw($number;n_)/g' file.txt > tmp
This will be in a for loop. Why is it not working?
Variables inside ' don't get substituted in Bash. To get string substitution (or interpolation, if you're familiar with Perl) you would need to change it to use double quotes " instead of the single quotes:
# Enclose the entire expression in double quotes
$ sed "s/draw($prev_number;n_)/draw($number;n_)/g" file.txt > tmp
# Or, concatenate strings with only variables inside double quotes
# This would restrict expansion to the relevant portion
# and prevent accidental expansion for !, backticks, etc.
$ sed 's/draw('"$prev_number"';n_)/draw('"$number"';n_)/g' file.txt > tmp
# A variable cannot contain arbitrary characters
# See link in the further reading section for details
$ a='foo
bar'
$ echo 'baz' | sed 's/baz/'"$a"'/g'
sed: -e expression #1, char 9: unterminated `s' command
Further Reading:
Difference between single and double quotes in Bash
Is it possible to escape regex metacharacters reliably with sed
Using different delimiters for sed substitute command
Unless you need it in a different file you can use the -i flag to change the file in place
Variables within single quotes are not expanded, but within double quotes they are. Use double quotes in this case.
sed "s/draw($prev_number;n_)/draw($number;n_)/g" file.txt > tmp
You could also make it work with eval, but don’t do that!!
This may help:
sed "s/draw($prev_number;n_)/draw($number;n_)/g"
You can use variables like below. Like here, I wanted to replace hostname i.e., a system variable in the file. I am looking for string look.me and replacing that whole line with look.me=<system_name>
sed -i "s/.*look.me.*/look.me=`hostname`/"
You can also store your system value in another variable and can use that variable for substitution.
host_var=`hostname`
sed -i "s/.*look.me.*/look.me=$host_var/"
Input file:
look.me=demonic
Output of file (assuming my system name is prod-cfm-frontend-1-usa-central-1):
look.me=prod-cfm-frontend-1-usa-central-1
I needed to input github tags from my release within github actions. So that on release it will automatically package up and push code to artifactory.
Here is how I did it. :)
- name: Invoke build
run: |
# Gets the Tag number from the release
TAGNUMBER=$(echo $GITHUB_REF | cut -d / -f 3)
# Setups a string to be used by sed
FINDANDREPLACE='s/${GITHUBACTIONSTAG}/'$(echo $TAGNUMBER)/
# Updates the setup.cfg file within version number
sed -i $FINDANDREPLACE setup.cfg
# Installs prerequisites and pushes
pip install -r requirements-dev.txt
invoke build
Retrospectively I wish I did this in python with tests. However it was fun todo some bash.
Another variant, using printf:
SED_EXPR="$(printf -- 's/draw(%s;n_)/draw(%s;n_)/g' $prev_number $number)"
sed "${SED_EXPR}" file.txt
or in one line:
sed "$(printf -- 's/draw(%s;n_)/draw(%s;n_)/g' $prev_number $number)" file.txt
Using printf to build the replacement expression should be safe against all kinds of weird things, which is why I like this variant.
I am trying to use sed to use as input for a variable. The user will choose from a list of files that have numbers before each to identify individual files. Then they choose a number corresponding to a name. I need to get the name of that file. My code is:
for entry in *; do
((i++))
echo "$i) $entry: "
done
echo What file # do you want to choose?:
read filenum
fileName=$(./myscript.sh | sed -n "${filenum}p")
echo $fileName ###this is to see if anything goes into fileName. nothing is ever output
echo What do you want to do with $fileName?
Ideally I would use () instead of the backtick but I can't seem to figure out how. I've looked at the links below, but can't get those ideas to work. I believe a problem may be that I am trying to include the filenum variable inside my sed.
https://www.linuxquestions.org/questions/linux-newbie-8/storing-output-of-sed-in-a-variable-in-shell-script-499997/
Store output of sed into a variable
Don't put backticks around $filenum. That will try to execute the contents of $filenum as a command. Put variables inside double quotes.
And if you do want to nest a backtick expression inside another set of backticks, you have to escape them. That's where $() becomes useful -- they nest without any hassle.
When you use sed -n, you need to use the p command to print the lines that you want to show in the output.
fileName=$(sed -n "${filenum}p" myscript.sh)
This will put the contents of line $filenum of myscript.sh in the variable.
If you actually wanted to execute myscript.sh and print the selected line of its output, you need to pipe to sed:
fileName=$(./myscript.sh | sed -n "${filenum}p")
I am trying to change the values in a text file using sed in a Bash script with the line,
sed 's/draw($prev_number;n_)/draw($number;n_)/g' file.txt > tmp
This will be in a for loop. Why is it not working?
Variables inside ' don't get substituted in Bash. To get string substitution (or interpolation, if you're familiar with Perl) you would need to change it to use double quotes " instead of the single quotes:
# Enclose the entire expression in double quotes
$ sed "s/draw($prev_number;n_)/draw($number;n_)/g" file.txt > tmp
# Or, concatenate strings with only variables inside double quotes
# This would restrict expansion to the relevant portion
# and prevent accidental expansion for !, backticks, etc.
$ sed 's/draw('"$prev_number"';n_)/draw('"$number"';n_)/g' file.txt > tmp
# A variable cannot contain arbitrary characters
# See link in the further reading section for details
$ a='foo
bar'
$ echo 'baz' | sed 's/baz/'"$a"'/g'
sed: -e expression #1, char 9: unterminated `s' command
Further Reading:
Difference between single and double quotes in Bash
Is it possible to escape regex metacharacters reliably with sed
Using different delimiters for sed substitute command
Unless you need it in a different file you can use the -i flag to change the file in place
Variables within single quotes are not expanded, but within double quotes they are. Use double quotes in this case.
sed "s/draw($prev_number;n_)/draw($number;n_)/g" file.txt > tmp
You could also make it work with eval, but don’t do that!!
This may help:
sed "s/draw($prev_number;n_)/draw($number;n_)/g"
You can use variables like below. Like here, I wanted to replace hostname i.e., a system variable in the file. I am looking for string look.me and replacing that whole line with look.me=<system_name>
sed -i "s/.*look.me.*/look.me=`hostname`/"
You can also store your system value in another variable and can use that variable for substitution.
host_var=`hostname`
sed -i "s/.*look.me.*/look.me=$host_var/"
Input file:
look.me=demonic
Output of file (assuming my system name is prod-cfm-frontend-1-usa-central-1):
look.me=prod-cfm-frontend-1-usa-central-1
I needed to input github tags from my release within github actions. So that on release it will automatically package up and push code to artifactory.
Here is how I did it. :)
- name: Invoke build
run: |
# Gets the Tag number from the release
TAGNUMBER=$(echo $GITHUB_REF | cut -d / -f 3)
# Setups a string to be used by sed
FINDANDREPLACE='s/${GITHUBACTIONSTAG}/'$(echo $TAGNUMBER)/
# Updates the setup.cfg file within version number
sed -i $FINDANDREPLACE setup.cfg
# Installs prerequisites and pushes
pip install -r requirements-dev.txt
invoke build
Retrospectively I wish I did this in python with tests. However it was fun todo some bash.
Another variant, using printf:
SED_EXPR="$(printf -- 's/draw(%s;n_)/draw(%s;n_)/g' $prev_number $number)"
sed "${SED_EXPR}" file.txt
or in one line:
sed "$(printf -- 's/draw(%s;n_)/draw(%s;n_)/g' $prev_number $number)" file.txt
Using printf to build the replacement expression should be safe against all kinds of weird things, which is why I like this variant.
I want to issue this command from the bash script
sed -e $beginning,$s/pattern/$variable/ file
but any possible combination of quotes gives me an error, only one that works:
sed -e "$beginning,$"'s/pattern/$variable/' file
also not good, because it do not dereferences the variable.
Does my approach can be implemented with sed?
Feel free to switch the quotes up. The shell can keep things straight.
sed -e "$beginning"',$s/pattern/'"$variable"'/' file
You can try this:
$ sed -e "$beginning,$ s/pattern/$variable/" file
Example
file.txt:
one
two
three
Try:
$ beginning=1
$ variable=ONE
$ sed -e "$beginning,$ s/one/$variable/" file.txt
Output:
ONE
two
three
There are two types of quotes:
Single quotes preserve their contents (> is the prompt):
> var=blah
> echo '$var'
$var
Double quotes allow for parameter expansion:
> var=blah
> echo "$var"
blah
And two types of $ sign:
One to tell the shell that what follows is the name of a parameter to be expanded
One that stands for "last line" in sed.
You have to combine these so
The shell doesn't think sed's $ has anything to do with a parameter
The shell parameters still get expanded (can't be within single quotes)
The whole sed command is quoted.
One possibility would be
sed "$beginning,\$s/pattern/$variable/" file
The whole command is in double quotes, i.e., parameters get expanded ($beginning and $variable). To make sure the shell doesn't try to expand $s, which doesn't exist, the "end of line" $ is escaped so the shell doesn't try anything funny.
Other options are
Double quoting everything but adding a space between $ and s (see Ren's answer)
Mixing quoting types as needed (see Ignacio's answer)
Methods that don't work
sed '$beginning,$s/pattern/$variable/' file
Everything in single quotes: the shell parameters are not expanded (doesn't follow rule 2 above). $beginning is not a valid address, and pattern would be literally replaced by $variable.
sed "$beginning,$s/pattern/$variable/" file
Everything in double qoutes: the parameters are expanded, including $s, which isn't supposed to (doesn't follow rule 1 above).
the following form worked for me from within script
sed $beg,$ -e s/pattern/$variable/ file
the same form will also work if executed from the shell
I am trying to replace a line in a file with the value of a variable (the variable contains a windows path). Although the variable prints correctly to STDOUT, when used in sed to replace the line of interest, the backslashes disappear.
Any idea how to remedy this? Other ideas welcome.
CODE:
WINPATH="\\\\hd-place\\stor1\\fold1\\archive\\$VAR1.$Var2\\Viewer"
print this to screen (as it should): \\hd-place\stor1\fold1\archive\$VAR1.$Var2\Viewer
The SED command I'm using is: cat file.xml | sed "3 c\
<\RunFolder>$WINPATH</RunFolder>" (please excuse the slashes inside the XML tags)
This outputs this:
\hd-placestor1fold1archive$VAR1.$Var2Viewer
Where as I want this:
\\hd-place\stor1\fold1\archive\$VAR1.$Var2\Viewer
You need one additional layer of escapes for the backslashes, as sed also interprets them after the variable has been expanded.