I would like to substitute a substring that contains an # character with Perl as in the following sed command:
substitution='newusername#anotherwebsite.com'
sed 's/oldusername#website.com/'"${substitution}"'/g' <<< "The current e-mail address is oldusername#website.com"
At present wherever I use Perl instead of sed or awk I first replace \ with \\, / with \/, $ with \$ and # with \#; e.g.
substitution='newusername#anotherwebsite.com'
substitution="${substitution//\\/\\\\}"
substitution="${substitution//\//\\/}"
substitution="${substitution//$/\\$}"
substitution="${substitution//#/\\#}"
perl -pe 's/oldusername\#website.com/'"${substitution}"'/g' <<< "The current e-mail address is oldusername#website.com"
I have read about using single quotation marks (as below based on sed/ perl with special characters (#)) but I was wondering if there is any other way to do this with forward slashes?
substitution='newusername#anotherwebsite.com'
perl -pe "s'oldusername#website.com'"${substitution}"'g" <<< "The current e-mail address is oldusername#website.com"
Also, are there special characters in Perl aside from $, # and % (and why is there no need to escape %)?
The cleanest way is to pass the values to Perl, as it can handle variables in substitution patterns and replacements correctly. Use single quotes so the shell's variable expansion doesn't interfere. You can use the -s option (explained in perlrun).
#!/bin/bash
pattern=oldusername#website.com
substitution=newusername#anotherwebsite.com
perl -spe 's/\Q$pat/$sub/g' -- -pat="$pattern" -sub="$substitution" <<< "The current e-mail address is oldusername#website.com"
or propagate the values to Perl via the environment.
pattern=oldusername#website.com
substitution=newusername#anotherwebsite.com
pat=$pattern sub=$substitution perl -pe 's/\Q$ENV{pat}/$ENV{sub}/g' <<< "The current e-mail address is oldusername#website.com"
Note that you need to assign the values before calling Perl or you need to export them in order to propagate them into the environment.
The \Q applies quotemeta to the pattern, i.e. it escapes all the special characters so that they are interpreted literally.
There's no need to backslash % as hashes aren't interpolated in double quotes or regexes.
Related
Let's say 'pm.max_children = 8' on the file '/usr/local/etc/php-fpm.d/www.conf'
The result of the following is supposed to be 40 but $1 is just ignored.
aaa=5
perl -i.bak -e "s/pm.max_children\s*=\s*\K([0-9]+)/($1 * $aaa)/ge" /usr/local/etc/php-fpm.d/www.conf
But the strange thing is the following is working, in case $aaa is not a variable.
perl -i.bak -e "s/pm.max_children\s*=\s*\K([0-9]+)/($1 * 3)/ge" /usr/local/etc/php-fpm.d/www.conf
The meaning of $1 is different in the shell and in Perl.
In the shell, it means the first positional argument. As double quotes expand variables, $1 in double quotes also means the first positional argument.
In Perl, $1 means the first capture group matched by a regular expression.
But, if you use $1 in double quotes on the shell level, Perl never sees it: the shell expands $1 as the first positional argument and sends the expanded string to Perl.
You can use the %ENV hash in Perl to refer to environment variables:
aaa=5 perl -i.bak -pe 's/pm.max_children\s*=\s*\K([0-9]+)/($1 * $ENV{aaa})/ge' /usr/local/etc/php-fpm.d/www.conf
I have a script to replace a specific email address in various files. The replacement address is the first parameter to the script:
#!/bin/bash
perl -pi -e s/'name\#domain\.org'/$1/ file-list
This doesn't work, as the # character in $1 is substituted by perl. Is there a straightforward fix for this? Running the script as subst foo\#bar.com, subst foo\\#bar.com, subst "foo#bar.com", and so on, doesn't work. Is there a sed script that could handle this more easily?
Instead of expanding a shell variable directly in the perl code you can pass it as an argument by setting the s switch:
#!/usr/bin/env bash
perl -i -spe 's/name\#domain\.org/$replacement/' -- -replacement="$1" file1.txt file2.txt
In perl's s///, without the use of e or ee modifiers, variables in the replacement part are treated as literals so you don't need to escape them.
This works, but needs you to pass the new mail address to the script with the # character preceded by \\:
#!/bin/bash
perl -pi -e "s/name\#domain.org/$1/" file-list
If the script is subst, run as:
subst newname\\#example.com
This is a better alternative, which uses sed to carry out the escaping:
#!/bin/bash
ADR="$(echo "$1" | sed -e 's/#/\\\#/')"
perl -pi -e "s/name\#domain.org/$ADR/" file-list
Of course, in this case it's probably better to use sed to do the whole thing.
I'm trying to replace only single quotes (') with double quotes (") in a file in Unix and not the apostrophe ('). Apostrophe (') must remain as is.
I'm getting the desired output with 3 sed commands executed in sequence. However, I'm unable to address the last line 'fake news'.
sed -i 's/'\''/"/g' test.txt
sed -i 's/"s/'\''s/g' test.txt
sed -i 's/s"/s'\''/g' test.txt
1st sed - converts all single quote to double quote.
2nd sed - converts all double quotes (followed by s) with single quote.
3rd sed - converts all s followed by double quotes with single quote.
Input File -
Hello Sir!
How are you?
How's your health?
All 'good'?
Charles' here.
'fake news'
Expected Output-
Hello Sir!
How are you?
How's your health?
All "good"?
Charles' here.
"fake news"
This does what you ask for with a single substitute command:
$ sed -E "s/'([^']*)'/\"\1\"/g" file
Hello Sir!
How are you?
How's your health?
All "good"?
Charles' here.
"fake news"
The above works by replacing pairs of single quotes with pairs of double quotes.
'([^']*)' matches a single-quote followed by any characters other than a single-quote followed by a single-quote. The characters inside the single-quotes are saved in capture group 1. The replacement, \"\1\", takes the capture group and puts it inside double-quotes.
While this handles the cases that you asked for, it is easy to imagine more complex cases that couldn't be handled without a sophisticated linguistic analysis.
Alternative style
The same command as above can be written in an alternate (but equivalent) shell quoting style:
sed -E 's/'\''([^'\'']*)'\''/"\1"/g' file
You can run all the commands in one sed invocation:
sed 's/'\''/"/g;s/"s/'\''s/g;s/s"/s'\''/g'
or
sed -e 's/'\''/"/g' -e 's/"s/'\''s/g' -e 's/s"/s'\''/g'
To reproduce your three sed, you can use a single perl regex:
perl -p -e 's/([^s])'\''([^s])/$1"$2/g' test.txt
edit: the 'fake news' case:
You can process half of the 'fake news' case with:
perl -p -e 's/([^s]|^)'\''([^s])/$1"$2/g' test.txt
To manage the "news'" case, it requires to define the list of words for which the single quote is acceptable (name and surnames). Else, you can add the following process that change the quote after a word that do not start with a capital:
perl -p -e 's/([^\w][a-z]+)s'\''/$1s"/g' test.txt
In a "single" command (with a pipe):
perl -p -e 's/([^s]|^)'\''([^s])/$1"$2/g' test.txt | perl -p -e 's/([^\w][a-z]+)s'\''/$1s"/g'
that gives:
Hello Sir!
How are you?
How's your health?
All "good"?
Charles' here.
"fake news"
a "fake news"
I'm trying to hash a password using the crypt hash function in Perl. In a Bash script so far I have:
password='Pa$$word'
hashedPassword="$(perl -e "print crypt('$password', 'salt'), \"\n"")"
I then modify/copy /etc/shadow using sed:
sed -e '/^user1:/s_:[^:]*:_:'"$hashedPassword"':_' /etc/shadow > /tmp/shadow
The method works, except when passing a string containing single quotes. How can I handle a password containing ' single quotes? Running Solaris 10 OS.
Your problem is in bash. Trying to set a shell variable containing a single quote by enclosing it in single quotes will not work. Per the man page:
Enclosing characters in single quotes preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.
Supply the password as a parameter to your one-liner instead of interpolating it directly in the code:
#!/usr/bin/env bash
password='Pa$$word'
hashedPassword=$(perl -e 'print crypt($ARGV[0], "salt"), "\n"' $password)
echo "p='$password', h='$hashedPassword'"
Output:
p='Pa$$word', h='saFQXTeqbkiIQ'
As password contains $ and $ are syntactical in perl it is not easy to pass in hard in script, other options are to pass by arguments or by environment.
# by argument
perl -e '($password, $salt)=#ARGV;print crypt($password, $salt), "\n"' "$password" "$salt"
# by environment variable
password=$password salt=$salt perl -e 'print crypt($ENV{password}, $ENV{salt}), "\n"'
I have a bunch of text files (named install*) in which I need to replace the expression curl -L with the expression curl -k -L. I am on OS X 10, Yosemite.
The following attempts don't work:
sed -e "s/'curl -L'/'curl -k -L'/g" install*
sed -i '' -e "s/'curl -L'/'curl -k -L'/g" install*
The contents of the files are shown (as if I had typed cat), but replacement isn't performed.
What am I doing wrong?
Your problem is that you're nesting quoting mechanisms (delimiters) when you shouldn't.
Generally, you should use single quotes to enclose an sed script as a whole:
sed -i '' -e 's/curl -L/curl -k -L/g' install*
By using single quotes, the script is protected from accidental interpretation by the shell.
If you do want the shell to interpret parts of a sed script beforehand, splice those parts in as double-quoted strings.
To sed's s function, it is / that acts as the delimiter - no additional quoting of the strings between / instances should be used, and your use of ' simply causes sed to consider these single quotes part of the regex to match / the replacement string. (As an aside: you're free to choose a delimiter other than /).
That said, if you do need to force interpretation of certain characters as literals, use \-escaping, such as using \. in the regex to represent a literal .
This is the only quoting mechanism that sed itself supports.
However, in your particular case, neither the string used in the regex nor the string used as the replacement string need escaping.
For a generic way to escape strings for use as literals in these situations, see
https://stackoverflow.com/a/29613573/45375.
You need -i.
GNU sed uses -i without arguments to mean "replace in-place".
BSD sed needs an empty argument, so you'll have to use sed -i '' -e '...' install*
There's no portable way to do it with sed...