This question already has answers here:
Use sed to replace all backslashes with forward slashes
(9 answers)
Closed 1 year ago.
I've seen a few threads about this but nothing I'm trying seems to be working. I'm using terminal within mac to execute this.
I am looking to change back slashes to forward slashes, and I am looking to do so by using the sed command.
My URL thread is basically
ServerDrive\Folder1\Folder2\Folder3\Folder4\Folder5\Folder6\Folder7\Folder8\Folder9
I believe I could use something like sed -e "s/\\\\/\//"
However I'm not sure how to tie the entire command in, do I need to start with an echo statement of some sort? As when I glue it all together as
ServerDrive\Folder1\Folder2\Folder3\Folder4\Folder5\Folder6\Folder7\Folder8\Folder9 sed -e "s/\\\\/\//"
Terminal yields an error that reads -bash: ServerDrive: command not found.
Yes, you need echo and a pipe. Otherwise, the first thing you type on the line is taken as a command to execute -- that's just basic shell syntax.
You also need the g modifier to make it replace all the slashes in the line, not just the first one.
echo 'ServerDrive\Folder1\Folder2\Folder3\Folder4\Folder5\Folder6\Folder7\Folder8\Folder9' | sed -e 's#\\#/#g'
or you can use a here-string (note that this is a bash extension).
sed -e 's#\\#/#g' <<< 'ServerDrive\Folder1\Folder2\Folder3\Folder4\Folder5\Folder6\Folder7\Folder8\Folder9'
Also, for a replacement of a single character, you can use tr rather than sed:
tr '\' '/' <<< 'ServerDrive\Folder1\Folder2\Folder3\Folder4\Folder5\Folder6\Folder7\Folder8\Folder9'
If the URL is in a variable, use bash's parameter expansion operator for substring replacement.
url='ServerDrive\Folder1\Folder2\Folder3\Folder4\Folder5\Folder6\Folder7\Folder8\Folder9'
echo "${url//\\//}"
You can use tr command (tr manpage) to replace a set of chars to another
echo 'ServerDrive\Folder1\Folder2\Folder3\Folder4\Folder5\Folder6\Folder7\Folder8\Folder9' | tr '\\' '/'
output
➜ ~ echo 'ServerDrive\Folder1\Folder2\Folder3\Folder4\Folder5\Folder6\Folder7\Folder8\Folder9' | tr '\\' '/'
ServerDrive/Folder1/Folder2/Folder3/Folder4/Folder5/Folder6/Folder7/Folder8/Folder9
Related
I wanted to use grep to exclude words from $lastblock by using a pipeline, but I found that grep works only for files, not for stdout output.
So, here is what I'm using:
lastblock="./2.json"
echo $lastblock | sed '1,/firstmatch/d;/.json/,$d'
I want to exclude ./ and .json, keeping only what is between.
This sed command is correct for this purpose, but how to escape the ./ replacing firstmatch so it can work?
Thanks in advance!
Use bash's Parameter Substitution
lastblock="./2.json"
name="${lastblock##*/}" # strips from the beginning until last / -> 2.json
base="${name%.*}" # strips from the last . to the end -> 2
but I found that grep works only for files, not for stdout output.
here it is. (if your grep supports the -P flag.
lastblock="./2.json"
echo "$lastblock" | grep -Po '(?<=\./).*(?=\.)'
but how to escape the ./
With sed(1), escape it using a back slash \
lastblock="./2.json"
echo "$lastblock" | sed 's/^\.\///;s/\..*$//'
Or use a different delimiter like a pipe |
sed 's|^\./||;s|\..*$||'
with awk
lastblock="./2.json"
echo "$lastblock" | awk -F'[./]+' '{print $2}'
Starting from bashv3, regular expression pattern matching is supported using the =~ operator inside the [[ ... ]] keyword.
lastblock="./2.json"
regex='^\./([[:digit:]]+)\.json'
[[ $lastblock =~ $regex ]] && echo "${BASH_REMATCH[1]}"
Although a P.E. should suffice just for this purpose.
I wanted to use grep to exclude words from $lastblock by using a pipeline, but I found that grep works only for files, not for stdout output.
Nonsense. grep works the same for the same input, regardless of whether it is from a file or from the standard input.
So, here is what I'm using:
lastblock="./2.json"
echo $lastblock | sed '1,/firstmatch/d;/.json/,$d'
I want to exclude ./ and .json, keeping only what is between. This sed
command is correct for this purpose,
That sed command is nowhere near correct for the stated purpose. It has this effect:
delete every line from the very first one up to and including the next subsequent one that matches the regular expression /firstmatch/, AND
delete every line from the first one matching the regular expression /.json/ to the last one of the file (and note that . is a regex metacharacter).
To remove part of a line instead of deleting a whole line, use an s/// command instead of a d command. As for escaping, you can escape a character to sed by preceding it with a backslash (\), which itself must be quoted or escaped to protect it from interpretation by the shell. Additionally, most regex metacharacters lose their special significance when they appear inside a character class, which I find to be a more legible way to include them in a pattern as literals. For example:
lastblock="./2.json"
echo "$lastblock" | sed 's/^[.]\///; s/[.]json$//'
That says to remove the literal characters ./ appearing at the beginning of the (any) line, and, separately, to remove the literal characters .json appearing at the end of the line.
Alternatively, if you want to modify only those lines that both start with ./ and end with .json then you can use a single s command with a capturing group and a backreference:
lastblock="./2.json"
echo "$lastblock" | sed 's/^[.]\/\(.*\)[.]json$/\1/'
That says that on lines that start with ./ and end with .json, capture everything between those two and replace the whole line with the captured part alone.
You can use another character like '#' when you want to avoid slashes.
You can remember a part that matches and use it in the replacement.
Use [.] avoiding the dot to be any character.
echo "$lastblock" | sed -r 's#[.]/(.*)[.]json#\1#'
Solution!
Just discovered today the tr command thanks to this legendary, unrelated answer.
When searching all over Google for how to exclude "." and "/", 100% of StackOverflow answers didn't helped.
So, to escape characters from the output of a command, just append this pipe:
| tr -d "{character-emoji-anything-you-want-to-exclude}"
So, a full working and simple sample:
echo "./2.json" | tr -d "/" | tr -d "." | tr -d "json"
And done!
If I run these commands from a script:
#my.sh
PWD=bla
sed 's/xxx/'$PWD'/'
...
$ ./my.sh
xxx
bla
it is fine.
But, if I run:
#my.sh
sed 's/xxx/'$PWD'/'
...
$ ./my.sh
$ sed: -e expression #1, char 8: Unknown option to `s'
I read in tutorials that to substitute environment variables from shell you need to stop, and 'out quote' the $varname part so that it is not substituted directly, which is what I did, and which works only if the variable is defined immediately before.
How can I get sed to recognize a $var as an environment variable as it is defined in the shell?
Your two examples look identical, which makes problems hard to diagnose. Potential problems:
You may need double quotes, as in sed 's/xxx/'"$PWD"'/'
$PWD may contain a slash, in which case you need to find a character not contained in $PWD to use as a delimiter.
To nail both issues at once, perhaps
sed 's#xxx#'"$PWD"'#'
In addition to Norman Ramsey's answer, I'd like to add that you can double-quote the entire string (which may make the statement more readable and less error prone).
So if you want to search for 'foo' and replace it with the content of $BAR, you can enclose the sed command in double-quotes.
sed 's/foo/$BAR/g'
sed "s/foo/$BAR/g"
In the first, $BAR will not expand correctly while in the second $BAR will expand correctly.
Another easy alternative:
Since $PWD will usually contain a slash /, use | instead of / for the sed statement:
sed -e "s|xxx|$PWD|"
You can use other characters besides "/" in substitution:
sed "s#$1#$2#g" -i FILE
一. bad way: change delimiter
sed 's/xxx/'"$PWD"'/'
sed 's:xxx:'"$PWD"':'
sed 's#xxx#'"$PWD"'#'
maybe those not the final answer,
you can not known what character will occur in $PWD, / : OR #.
if delimiter char in $PWD, they will break the expression
the good way is replace(escape) the special character in $PWD.
二. good way: escape delimiter
for example:
try to replace URL as $url (has : / in content)
x.com:80/aa/bb/aa.js
in string $tmp
URL
A. use / as delimiter
escape / as \/ in var (before use in sed expression)
## step 1: try escape
echo ${url//\//\\/}
x.com:80\/aa\/bb\/aa.js #escape fine
echo ${url//\//\/}
x.com:80/aa/bb/aa.js #escape not success
echo "${url//\//\/}"
x.com:80\/aa\/bb\/aa.js #escape fine, notice `"`
## step 2: do sed
echo $tmp | sed "s/URL/${url//\//\\/}/"
URL
echo $tmp | sed "s/URL/${url//\//\/}/"
URL
OR
B. use : as delimiter (more readable than /)
escape : as \: in var (before use in sed expression)
## step 1: try escape
echo ${url//:/\:}
x.com:80/aa/bb/aa.js #escape not success
echo "${url//:/\:}"
x.com\:80/aa/bb/aa.js #escape fine, notice `"`
## step 2: do sed
echo $tmp | sed "s:URL:${url//:/\:}:g"
x.com:80/aa/bb/aa.js
With your question edit, I see your problem. Let's say the current directory is /home/yourname ... in this case, your command below:
sed 's/xxx/'$PWD'/'
will be expanded to
sed `s/xxx//home/yourname//
which is not valid. You need to put a \ character in front of each / in your $PWD if you want to do this.
Actually, the simplest thing (in GNU sed, at least) is to use a different separator for the sed substitution (s) command. So, instead of s/pattern/'$mypath'/ being expanded to s/pattern//my/path/, which will of course confuse the s command, use s!pattern!'$mypath'!, which will be expanded to s!pattern!/my/path!. I’ve used the bang (!) character (or use anything you like) which avoids the usual, but-by-no-means-your-only-choice forward slash as the separator.
Dealing with VARIABLES within sed
[root#gislab00207 ldom]# echo domainname: None > /tmp/1.txt
[root#gislab00207 ldom]# cat /tmp/1.txt
domainname: None
[root#gislab00207 ldom]# echo ${DOMAIN_NAME}
dcsw-79-98vm.us.oracle.com
[root#gislab00207 ldom]# cat /tmp/1.txt | sed -e 's/domainname: None/domainname: ${DOMAIN_NAME}/g'
--- Below is the result -- very funny.
domainname: ${DOMAIN_NAME}
--- You need to single quote your variable like this ...
[root#gislab00207 ldom]# cat /tmp/1.txt | sed -e 's/domainname: None/domainname: '${DOMAIN_NAME}'/g'
--- The right result is below
domainname: dcsw-79-98vm.us.oracle.com
VAR=8675309
echo "abcde:jhdfj$jhbsfiy/.hghi$jh:12345:dgve::" |\
sed 's/:[0-9]*:/:'$VAR':/1'
where VAR contains what you want to replace the field with
I had similar problem, I had a list and I have to build a SQL script based on template (that contained #INPUT# as element to replace):
for i in LIST
do
awk "sub(/\#INPUT\#/,\"${i}\");" template.sql >> output
done
If your replacement string may contain other sed control characters, then a two-step substitution (first escaping the replacement string) may be what you want:
PWD='/a\1&b$_' # these are problematic for sed
PWD_ESC=$(printf '%s\n' "$PWD" | sed -e 's/[\/&]/\\&/g')
echo 'xxx' | sed "s/xxx/$PWD_ESC/" # now this works as expected
for me to replace some text against the value of an environment variable in a file with sed works only with quota as the following:
sed -i 's/original_value/'"$MY_ENVIRNONMENT_VARIABLE"'/g' myfile.txt
BUT when the value of MY_ENVIRONMENT_VARIABLE contains a URL (ie https://andreas.gr) then the above was not working.
THEN use different delimiter:
sed -i "s|original_value|$MY_ENVIRNONMENT_VARIABLE|g" myfile.txt
Cannot seem to find an answer to this one online...
I have a string variable (externally sourced) with new lines "\n" encoded as strings.
I want to replace those strings with actual new line carriage returns. The code below can achieve this...
echo $EXT_DESCR | sed 's/\\n/\n/g'
But when I try to store the result of this into it's own variable, it converts them back to strings
NEW_DESCR=`echo $EXT_DESCR | sed 's/\\n/\n/g'`
How can this be achieved, or what I'm I doing wrong?
Here's my code I've been testing to try get the right results
EXT_DESCR="This is a text\nWith a new line"
echo $EXT_DESCR | sed 's/\\n/\n/g'
NEW_DESCR=`echo $EXT_DESCR | sed 's/\\n/\n/g'`
echo ""
echo "$NEW_DESCR"
No need for sed, using parameter expansion:
$ foo='1\n2\n3'; echo "${foo//'\n'/$'\n'}"
1
2
3
With bash 4.4 or newer, you can use the E operator in ${parameter#operator}:
$ foo='1\n2\n3'; echo "${foo#E}"
1
2
3
Other answers contain alternative solutions. (I especially like the parameter expansion one.)
Here's what's wrong with your attempt:
In
echo $EXT_DESCR | sed 's/\\n/\n/g'
the sed command is in single quotes, so sed gets s/\\n/\n/g as is.
In
NEW_DESCR=`echo $EXT_DESCR | sed 's/\\n/\n/g'`
the whole command is in backticks, so a round of backslash processing is applied. That leads to sed getting the code s/\n/\n/g, which does nothing.
A possible fix for this code:
NEW_DESCR=`echo $EXT_DESCR | sed 's/\\\\n/\\n/g'`
By doubling up the backslashes, we end up with the right command in sed.
Or (easier):
NEW_DESCR=$(echo $EXT_DESCR | sed 's/\\n/\n/g')
Instead of backticks use $( ), which has less esoteric escaping rules.
Note: Don't use ALL_UPPERCASE for your shell variables. UPPERCASE is (informally) reserved for system variables such as HOME and special built-in variables such as IFS or RANDOM.
Depending on what exactly you need it for:
echo -e $EXT_DESCR
might be all you need.
From echo man page:
-e
enable interpretation of backslash escapes
This printf would do the job by interpreting all escaped constructs:
printf -v NEW_DESCR "%b" "$EXT_DESCR"
-v option will store output in a variable so no need to use command substitution here.
Problem with your approach is use of old back-ticks. You could do:
NEW_DESCR=$(echo "$EXT_DESCR" | sed 's/\\n/\n/g')
Assuming you're using gnu sed as BSD sed won't work with this approach.
This question already has answers here:
sed substitution with Bash variables
(6 answers)
Closed 5 years ago.
I have a variable in a config, which I would like to replace with a value.
ROOT="rROOT"
I would like to replace that with
ROOT="$root"
So with the value of $root (Important are the quotation marks).
So far I have tried it that way
sed -i s/'ROOT=rROOT'/'ROOT="$root"'/g $directory/settings.conf
The result is that
ROOT="$root"
But this is stored in the variable $root (It is always a path)
root: /
How can I replace rROOT with the value of $root?
Sed Version: (GNU sed) 4.2.2
Ok, don't like to ruin my scripts for testing:
cat tageslog.sh | sed "s/alt=185094/alt=\${root}/g"
Use double quotes, but mask the dollar sign, so it doesn't get lost and root interpreted while calling sed.
Use ${root} instead of "$root".
sed "s/ROOT=.rRoot./ROOT=\${root}/g" $directory/settings.conf
if this works, use the -i switch:
sed -i "s/ROOT=.rRoot./ROOT=\${root}/g" $directory/settings.conf
You have different problems:
Testing with echo
I think you tested your command with
echo ROOT="rROOT" | sed s/'ROOT=rROOT'/'ROOT="$root"'/g
The double quotes won't appear in the output of the echo, so you will end up with a command working for ROOT=rROOT.
When the inputfile has qouble quotes, you do not have to insert them.
Testing with the double quotes is possible with
echo 'ROOT="rROOT"' | sed s/'ROOT=rROOT'/'ROOT="$root"'/g
Place the double quotes outside the single quotes;
You can test this with echo:
echo 'Showing "$PWD" in double quotes is "'$PWD'"'
echo 'With additional spaces the last part is " '$PWD' " '
Root vaiable has slashes that will confuse sed.
The root variable is replaced before sed tries to understand the command.
You can use another character, like #, in the sed command:
sed 's#before#after#'
When your input has double quotes:
echo 'ROOT="rROOT"' | sed 's#ROOT="rROOT"#ROOT="'$root'"#g'
# or by remembering strings
echo 'ROOT="rROOT"' | sed -r 's#(ROOT=")rROOT(")#\1'$root'\2#g'
Input without double quotes
echo 'ROOT=rROOT' | sed 's#ROOT=rROOT#ROOT="'$root'"#g'
# or by remembering strings
echo 'ROOT=rROOT' | sed -r 's#(ROOT=)rROOT#\1"'$root'"#g'
This question already has answers here:
Replace a word with multiple lines using sed?
(11 answers)
Closed 4 years ago.
I am trying to replace a word with a text which spans multiple lines. I know that I can simply use the newline character \n to solve this problem, but I want to keep the string "clean" of any unwanted formatting.
The below example obviously does not work:
read -r -d '' TEST <<EOI
a
b
c
EOI
sed -e "s/TOREPLACE/${TEST}/" file.txt
Any ideas of how to achieve this WITHOUT modifying the part which starts with read and ends with EOI?
Given that you're using Bash, you can use it to substitute \n for the newlines:
sed -e "s/TOREPLACE/${TEST//$'\n'/\\n}/" file.txt
To be properly robust, you'll want to escape /, & and \, too:
TEST="${TEST//\\/\\\\}"
TEST="${TEST//\//\\/}"
TEST="${TEST//&/\\&}"
TEST="${TEST//$'\n'/\\n}"
sed -e "s/TOREPLACE/$TEST/" file.txt
If your match is for a whole line and you're using GNU sed, then it might be easier to use its r command instead:
sed -e $'/TOREPLACE/{;z;r/dev/stdin\n}' file.txt <<<"$TEST"
You can just write the script as follows:
sed -e 's/TOREPLACE/a\
b\
c\
/g' file.txt
A little cryptic, but it works. Note also that the file won't be modified in place unless you use the -i option.
tricky... but my solution would be :-
read -r -d '' TEST <<EOI
a
b
c
EOI
sed -e "s/TOREPLACE/`echo "$TEST"|awk '{printf("%s\\\\n", $0);}'|sed -e 's/\\\n$//'`/g" file.txt
Important:
Make sure you use the correct backticks, single quotes, double quotes and spaces
else it will not work.
An interesting question..
This may get you closer to a solution for your use case.
read -r -d '' TEST <<EOI
a\\
b\\
c
EOI
echo TOREPLACE | sed -e "s/TOREPLACE/${TEST}/"
a
b
c
I hope this helps.