File Content(file.txt):
table=$table_name
data=$data_name
Shell Script:
name=kush_123
cat file.txt | grep 'table' | sed "s\table_name\$name\g"
Expected output:
table=$kush_123
This gives error
unterminated s command
if the name variable has _ in it.
If you really want to use a backslash as a delimiter, it needs to be escaped itself so that the double-quoted string preserves it before passing to sed:
sed "s\\table_name\\$name\\g"
Otherwise, sed receives the string stable_name$nameg as its script. (\t, \$, and \g expand to t, $, and g, respectively). In this case, the letter t (as it immediately follows the s) is used as the delimiter, and the error results because there aren't enough ts in the result to provide a complete command.
Of course, if you try this, sed should complain that a backslash cannot be used as the delimiter for the s command. Use a different character:
sed "s/table_name/$name/g"
In general, building such scripts dynamically is fragile, because it assumes you know the value of $name doesn't contain your chosen delimiter.
On more investigation i found out the variable name has trailing spaces as the variable value was passed from python code. I trimmed extra spaces and it worked. Thanks :)
Related
I have 2 bash script variables defined:
THELINENUMBER="14" # an arbitrary line number, comes from a separate command
NEWLINE="a line/ with# special! characters<" # arbitrary line with special characters, comes from separate command
I need to use the line number ${THELINENUMBER} to replace a line in a file called after.txt with ${NEWLINE}.
How do I do that?
These are some examples I have tried:
sed -i '${THELINENUMBER}s#.*#"/"${NEWLINE}"/"' after.txt
sed -i "${THELINENUMBER}s#.*#"/"${NEWLINE}"/"" after.txt
sed -i "${THELINENUMBER}s/.*/'${NEWLINE}'" after.txt
sed -i '${THELINENUMBER}s,.*,${NEWLINE}' after.txt
I am told that the delimitter is usually a /, but those are present in my line replacement variable, so I can't use those. I tried with # and , but the desired behavior did not change. I am also told that " and ' are supposed to be used to turn off escaping in text (use literal string), but I have not been able to get that to work either. How do I pass in a string parameter into sed that has special characters? I am wondering if I should pass the variable ${NEWLINE} into another built-in function call to add escape characters or something before passing it into sed. Is sed the right tool for the job? I did not find much helpful information looking at the CLI manpages. I use Ubuntu 18.04.
I have referred to these sources in my internet search:
https://stackoverflow.com/questions/11145270/how-to-replace-an-entire-line-in-a-text-file-by-line-number
https://askubuntu.com/questions/76808/how-do-i-use-variables-in-a-sed-command
https://stackoverflow.com/questions/37372047/find-a-line-with-a-string-and-replace-entire-line-with-another-line
Use the c (change) command.
By the way, the naming convention for regular shell variables is NOT ALLCAPS, as that may result in accidental collisions with special variables like PATH.
sed "$linenumber c\\
$newline" file
Try
sed -i "${THELINENUMBER}s#.*#${NEWLINE}#" after.txt
this works because:
You require " enclosing the entire sed command instead of backtick so that the variables are expanded
No other quotes or backticks are needed to escape " in the variables as there aren't any: there are no literal (escaped) quotes inside the variables
An alternate separator (such as #) is required due to the / inside the NEWLINE variable.
this is perhaps one of the most discussed topics here. I tried almost all the commands and other tweaks found here, but something doesn't seems to be doing well.
i would want to replace all the double quotes in my file with whitespace/blank
I'm seeing the below error when i tried to execute this command.
sed "s/"/ \''/g' x_orbit.txt > new.tx
sed: -e expression #1, char 3: unterminated `s' command
You're close. Just use single quotes, so the shell doesn't try to expand the metacharacters in your sed command:
sed 's/"/ /g' x_orbit.txt > new.txt
You could try tr for example:
tr '"' ' ' < x_orbit.txt > new.txt
The script you provided:
sed "s/"/ \''/g' x_orbit.txt > new.tx
means:
sed # invoke sed to execute the following script:
" # enclose the script in double quotes rather than single so the shell can
# interpret it (e.g. to expand variables like $HOME) before sed gets to
# interpret the result of that expansion
s/ # replace what follows until the next /
" # exit the double quotes so the shell can now not only expand variables
# but can now do globbing and file name expansion on wildcards like foo*
/ # end the definition of the regexp you want to replace so it is null since
# after the shell expansion there was no text for sed to read between
# this / and the previous one (the 2 regexp delimiters)
\' # provide a blank then an escaped single quote for the shell to interpret for some reason
'/g' # enclose the /g in single quotes as all scripts should be quoted by default.
That is so far off the correct syntax it's kinda shocking which is why I dissected it above to try to help you understand what you wrote so you'll see why it doesn't work. Where did you get the idea to write it that way (or to put it another way - what did you think each character in that script meant? I'm asking as it indicates a fundamental misunderstanding of how quoting and escaping works in shell so it'd be good if we could help correct that misunderstanding rather than just correct that script.
When you use any script or string in shell, simply always enclose it in single quotes:
sed 'script' file
var='string'
unless you NEED to use double quotes to let a variable expand and then use double quotes unless you NEED to use no quotes to let globbing and file name expansion happen.
An awk version:
awk '{gsub(/"/," ")}1' file
gsub is used for the replace
1 is always true, so line is printed
I have to variables in a bash script:
$string = "The cat is green.\n"
$line = "Sunny day today.\n"
each of those variables contain "\n" character, how can I use sed to search and replace:
sed 's/$string/$line/g' file.txt
This doesn't seem to work, if I erase the "\n" from the strings sed works properly.
If I had only the text I could escape "\n" by adding a backslash:
sed 's/"The cat is green.\\n"/"Sunny day today.\\n"/g' file.txt
How can I manage to do search/replace when variables contain "\n" in them.
Thank you for the help.
It looks like you are trying to match the two-character sequence \n, as opposed to the single newline character that together they represent in some contexts. There is a tremendous difference between these.
As part of your example, you presented
sed 's/$string/$line/g' file.txt
, but that won't work at all, because variable references are not expanded within single-quoted strings. That has nothing whatever to do with the values of shell variables string and line.
But let's consider those values:
$string="The cat is green.\n"
$line="Sunny day today.\n"
[Extra spaces removed.]
Of course, the problem you're focusing on is that sed recognizes \n as a code for a newline character, but you also have the problem that in a regular expression, the . character matches any character, so if you want it to be treated as a literal then it, too, needs to be escaped (in the pattern, but not in the replacement). If you're trying to support search and replace for arbitrary text, then there are other characters you'll need to escape, too.
Answering the question as posed (escaping only \n sequences) you might do this:
sed "s/${string//\\n/\\\\n}/${line//\\n/\\\\n}/g"
The ${foo//pat/repl} form of parameter expansion performs pattern substitution on the expanded value, but note well that the pattern (pat) is interpreted according to shell globbing rules, not as a regular expression. That specific form replaces every appearance of the pattern; read the bash manual for alternatives that match only the first appearance and/or that match only at the beginning or the end of the parameter's value. Note, too, the extra doubling of the \ characters in the pattern substitution -- they need to be escaped for the shell, too.
Given your variable definitions, that command would be equivalent to this:
sed 's/The cat is green.\\n/Sunny day today.\\n/g'
In other words, exactly what you wanted. Again, however, be warned: that is not a general solution for arbitrary search & replace. If you want that, then you'll want to study the sed manual to determine which characters need to be escaped in the regex, and which need to be escaped in the replacement. Moreover, I don't see a way to do it with just one pattern substitution for each variable.
I'm trying to write a script in bash which extracts a database name from a PHP file. For example I want to copy CRM_123456789 from the below line:
$sugar_config['dbconfig']['db_name'] = 'CRM_123456789';
I have tried using sed, so essentially I want to copy the text between
['db_name'] = '
and
';
sed -n '/['db_name'] = /,/';/p' myfile.php
However this does not return anything. Does anyone know what I'm doing wrong?
Thanks
You cannot nest single quotes. Your expression evaluates to single-quoted /[ next to unquoted db_name where clearly you want to match on a literal single quote.
One workaround is to use double quotes for the outermost quoting, but make sure you make any necessary changes, because double quotes are weaker than single quotes in the shell. In your case, there's nothing to change in that respect, though.
However, you also appear to misunderstand how sed address expressions work. They identify lines, not substrings on a line. So your script would print between a line matching ['db_name'] and a line matching ';. To extract something from within a line, the common idiom is to substitute out the parts you don't want, then print what's left.
Also, because opening square bracket is a metacharacter in sed, you need to backslash-escape it to match it literally.
sed -n "s/.*\['db_name'] = '\([^']*\)'.*/\1/p" myfile.php
This matches up through ['db_name'] = ', then captures whatever is inside the single-quoted string into \1, then matches anything from the next single quote through the end of line, and substitutes it with just the captured string; and prints that line after performing the substitution.
If the config file supports variable whitespace, a useful improvement would be to allow for optional whitespace around the equals sign, and possibly also within the square brackets. [ ]* will match zero or more spaces (the square brackets aren't really necessary around a single space, but I include them here for legibility reasons).
You could try the below sed command.
$ sed -n "s/.*\['db_name'\] = '\([^']*\)';.*/\1/p" file
CRM_123456789
I have a file with a lot of lines, two of them are:
videoId: 'S2Rgr6yuuXQ'
var vid_seq=1;
in a shell script, I have two variables,
for id, the value is always 11 characters/numbers
id='fsafsferii2'
id_seq=80
I want to modify these two lines with id and id_seq
videoId: 'fsafsferii2'
var vid_seq=80;
I used
sed -i 's/\(videoId: \).*\\1'${id}'/\2' file
but there are errors, what is wrong with my script?
thanks
The grep command won't "replace" text, it is for "global regular expression print". But sed will.
sed -i'' '/^videoId: /s/: .*/: '"$id"'/;/^var vid_seq=/s/=.*/='"$id_seq"';/'
I'm not a big fan of inserting variables into sed scripts this way, but sed is simple, and provides no mechanism for actually using actual variables on its own. If you're going to do this, include some format checking for the two variables to make sure they contain the data you want them to contain, before you run this sed script. An accidental / in a variable would cause the sed script to fail.
UPDATE per comments:
Here's a successful test:
$ id=fsafsferii2
$ id_seq=80
$ cat inp686
videoId: 'S2Rgr6yuuXQ'
var vid_seq=1;
$ sed '/^videoId: /s/: .*/: '"$id"'/;/^var vid_seq=/s/=.*/='"$id_seq"';/' < inp686
videoId: fsafsferii2
var vid_seq=80;
$
Of course, you'll need to do some quote magic to get the single quotes into your videoId, but I'm sure you can figure that out yourself.
UPDATE 2
According to sed's man page, the substitute command is in the form:
[2addr]s/regular expression/replacement/flags
The [2addr] means you can specify up to two "addresses", which can be line numbers or regular expressions to match. So the s (substitute) command can take a line, a range, a match, or a span between matches. In our case, we're just using a single match to identify what lines we want to execute the substitution on.
The script above is made up of two sed commands, separated by a semicolon.
/^videoId: / -- Match lines that start with the word videoId:...
s/: .*/: '"$id"'/; -- Substitute all text from the colon to the end of the line with whatever is in the $id environment variable.
/^var vid_seq=/ -- Match lines that ... meh, as above.
s/=.*/='"$id_seq"';/ -- Substitute all text from the equals sign on with $id_seq.
Note that the '"$id"' construct means that we are exiting the single quotes, then immediately entering double quotes for the expansion of the variable ... then exiting the double quotes and going back into a new set of single quotes. Sed scripts are safest inside single quotes because of the frequent use of characters that might be interpreted by a shell.
Note also that because sed's substitute command uses a forward slash as a delimiter, the $id and $id_seq variables may not contain a slash. If they might, you can switch to a different delimiter.
What is wrong with:
sed -i 's/\(videoId: \).*\\1'${id}'/\2' file
Missing the third delimiter (/). Valid syntax is s/regex/replace/
Incorrect regex pattern (let's assume ${id} has been substituted)
\(videoId: \).*\\1fsafsferii2
is telling it to match a string that looks like this:
videoId: anything\1fsafsferii2
(\\ in regex matches literal backslash, so \\1 would match a literal backslash followed by 1 instead of 1st sub-expression)
Replace the matched string with \2
But since there is only one set of parentheses, \2 is actually empty.
Also, since the regex pattern in 2. doesn't match anything, nothing is replaced.
This should work (GNU sed)
sed -i 's/\(videoId: \).*/\1 \x27'${id}'\x27/
s/\(var vid_seq=\).*/\1'${id_seq}'\;/' file
Note:
\x27 is the hexadecimal representation of single quote (to prevent clashing with the other single quote)
\; for literal semicolon. If ; is not escaped, it's interpreted to terminate the s command in sed.