I want to delete in a file each line beginning by '#'. I ran that (I am using osx)
sed -i '' -e 's/#.*/d' file
but I get this error message :
sed: 1: "s/#.*/d
": unescaped newline inside substitute pattern
The s command in sed means "substitute" and it takes two arguments:
s/pattern/replacement/
What you want to do is just to match lines starting with # and delete them, so you need the sed program:
/^#/d
Note that the pattern needs to start with ^ (meaning "start of line") otherwise it will match a # anywhere in the line.
As stated by Gareth Rees above, the correct command is:
sed '/^#/ d' file
This good sed tutorial contains your question as an example:
http://www.grymoire.com/Unix/Sed.html#toc-uh-30
Related
I am trying to comment a line in a file using search pattern, and then insert new line next to it.
search_variable=Dlog4j2.formatMsgNoLookups
new_variable="wrapper.java.additional.47=-Dlog4j2.formatMsg=false"
cat testfile.txt
wrapper.java.additional.47=-Dlog4j2.formatMsgNoLookups=true
This one works, but trying to use variable to comment the line and
sed -i '/Dlog4j2.formatMsgNoLookups/s/^/#/g' testfile.txt
output:
#wrapper.java.additional.47=-Dlog4j2.formatMsgNoLookups=true
Desired output
cat testfile.txt
#wrapper.java.additional.47=-Dlog4j2.formatMsgNoLookups=true
wrapper.java.additional.47=-Dlog4j2.formatMsg=false
With GNU sed:
search_variable="Dlog4j2.formatMsgNoLookups"
new_variable="wrapper.java.additional.47=-Dlog4j2.formatMsg=false"
sed -i "s/.*${search_variable}.*/#&\n${new_variable}/" testfile.txt
Output to testfile.txt:
#wrapper.java.additional.47=-Dlog4j2.formatMsgNoLookups=true
wrapper.java.additional.47=-Dlog4j2.formatMsg=false
For the meaning of & see using sed with ampersand (&).
The curly brackets can also be omitted in this case.
This can also be helpful: Difference between single and double quotes in bash
I have a csv file, with student name and marks. I want to update "marks" of a student with name "jack"(the only person in the csv). the data in csv file looks as below.
student,marks
jack,10
peter,20
rick,10
I found this awk '$1 == "Audrey" {print $2}' numbers.txt, but iam not sure on how to modify the file.
awk 'BEGIN{FS=OFS=","} $1=="jack"{$2=27} 1' foo.csv > tmp && mv tmp foo.csv
It worked for me with
sed -ir "s/^\(jack\),.*/\1,$new_grade/"
input.csv. with argument "r" or else i get the "error sed: 1: "input.csv": command i expects \ followed by text".
ed is usually better for in-place editing of files than sed:
printf "%s\n" "/^jack,/c" "jack,${new_grade}" "." w | ed -s input.csv
or using a heredoc to make it easier to read:
ed -s input.csv <<EOF
/^jack,/c
jack,${new_grade}
.
w
EOF
At the first line starting with jack,, change it to jack,XX where XX is the value of the new_grade variable, and write the new contents of the file.
You could use sed:
new_grade=9
sed -i'' "s/^\(jack\),.*/\1,$new_grade/"
The pattern ^\(jack\),.* matches the beginning of the line ^ followed by jack by a comma and the rest of the line .*. The replacement string \1,$new_mark contains the first captured group \1 (in this case jack) followed by a comma and the new mark.
Alternatively you could loop over the file and use a pattern substitution:
new_grade=9
while read -s line; do
echo ${line/jack,*/jack,$new_grade}
done < grades.txt > grades2.txt
Another approach with sed is to anchor the replacement to the digits at the end of the line with:
sed '/^jack,/s/[0-9][0-9]*$/12/' file
This uses the form sed '/find/s/match/replace' where find locates at the beginning of the line '^' the word "jack," eliminating all ambiguity with, e.g. jackson,33. Then the normal substitution form of 's/match/replace/' where match locates at least one digit at the end of the line (anchored by '$') and replaces it with the 12 (or whatever you choose).
Example Use/Output
With your example file in file, you would have:
$ sed '/^jack,/s/[0-9][0-9]*$/12/' file
student,marks
jack,12
peter,20
rick,10
(note: the POSIX character class of [[:digit:]] is equivalent to [0-9] which is another alternative)
The equivalent expression using the POSIX character class would be:
sed '/^jack,/s/[[:digit:]][[:digit:]]*$/12/' file
You can also use Extended Regular Expression which provides the '+' repetition operator to indicate one-or-more compared to the basic repetition designator of '*' to indicate zero-or-more. The equivalent ERE would be sed -E '/^jack,/s/[0-9]+$/12/' file
You can add the -i option to edit in-place and/or using it as -i.bak to create a backup of the original with the .bak extension before modifying the original.
How can I use a cat and sed to read data from a file and insert it into another file under known line?
For example I have a file named script1.txt that contains a few hundred lines, one of the line has the value "COMMANDS="commands"
If I wanted use sed to insert a line under it, simply I can use sed as the command bellow.
sed -i '/^COMMANDS=.*/a NEW LINE HERE' script1.txt
But if I want to insert a multi lines and these lines inside a file, and these line changes every a few hours.. how can i do that ?
I tried:
DATA=$(cat data.txt)
sed -i '/^COMMANDS=.*/a '$DATA'' script1.txt
I got the error bellow.
sed: -e expression #1, char 1: unknown command: `"'
Is there a way other than sed to insert the data from file under known line with no issues?
This might work for you (GNU sed):
sed -i '/^COMMANDS=/r dataFile' file
This will append the contents of the file dataFile after the line beginning COMMANDS= and update file
If the data you want to append is multi-line, you might want to replace newlines with \n.
#!/bin/sh
DATA="$(awk '{gsub(/[]\/$*.^&[]/, "\\\\&");printf (FNR>1)?"\\n%s":"%s",$0}END{print ""}' data.txt)"
sed -i -e '/^COMMANDS=.*/a\' -e "$DATA" script1.txt
Here the awk command escapes sed special characters (for basic regular expressions), then prints "%s" for the first line, and "\\n%s" for the others. A newline is printed at the end, but it's somewhat pointless as $() strips it anyway.
The sed command is almost the same but multiple expressions are used which is equivalent to a multi-line sed script (The a text sed alternative syntax can act weirdly with leading spaces/backslashes).
This question already has answers here:
Bash - how to put each line within quotation
(8 answers)
Closed 6 years ago.
I have a file with one word on each line, I want to add a " before the word, and a ", after the word with Bash.
Example:
Hello
Sleep
Try
And I want:
"Hello",
"Sleep",
"try",
Hope anyone can help me
sed 's/.*/"&",/' file
In the replacement part of a substitution, & is replaced with whatever matched the original regular expression. .* matches the whole line.
Assuming you've got one word per line, I'd use the following GNU sed command :
sed -i 's/.*/"\0",/' <your_file>
Where you replace <your_file> by the path to your file.
Explanation :
sed means Stream EDitor ; it specialise in filtering and transforming text streams
-i is it's in-place option : it means it will write the result of its transformation in the file it's working on
s/search/replace/ is sed's search & replace command : it search for a pattern written as regular expression, and replace the matched text by something else
.* is a regex pattern that will match as much as it can. Since sed works line by line, it will match a whole line
\0 is a back-reference to the whole matched string
"\0", is the whole matched string, enclosed in quotes, followed by a comma.
s/.*/"\0",/ is a sed search&replace command that will match a whole line and enclose it in quotes, following it by a comma
Already answered here
Using awk
awk '{ print "\""$0"\","}' inputfile
Using pure bash
while read FOO; do
echo -e "\"$FOO\","
done < inputfile
where inputfile would be a file containing the lines without quotes.
If your file has empty lines, awk is definitely the way to go:
awk 'NF { print "\""$0"\","}' inputfile
NF tells awk to only execute the print command when the Number of Fields is more than zero (line is not empty).
sed (the Stream EDitor) is one choice for this manipulation:
sed -e 's/^/"/;s/$/",/' file
There are two parts to the regular expression, separated by ;:
s/^/"/ means add a " to the beginning of each line.
s/$/",/ means add a ", to the end of each line.
The magic being the anchors for start of line (^) and end of line ($). The man page has more details. It's worth learning sed.
Just to be different: perl
perl -lne 'print qq{"$_",}' file
This is a simple one liner:
sed -i 's/^/"/;s/$/",/' file_name
> echo $word
word
> echo "\""$word"\","
"word",
See How to escape double quotes in bash?
"The Unix Programming Environment" states that '$' used in regular expression in sed means end-of-the line which is fine for me, because
cat file | sed 's/$/\n/'
is interpretted as "add newline at the end of each line".
The question arises, when I try to use the command:
cat file | sed '$d'
Shouldn't this line remove each line instead of the last one? In this context, dollar sign means end of the LAST line. What am I getting wrong?
$ is treated as regex anchor when used in pattern in s command e.g.
s/$/\n
However in $d, $ is not a regex anchor, it is address notation that means the last line of the input, which is deleted using the d command.
Also note that cat is unnecessary in your last command. It can be used as:
sed '$d' file
In the second usage, there is no regular expression. The $ there is an address, meaning the last line.
Note that regex in sed must be inside the delimiters(;,:, ~, etc) other than quotes.
/regex/
ex:
sed '/foo/s/bar/bux/g' file
or
~regex~
ex:
sed 's~dd~s~' file
but not 'regex'. So $ in '$d' won't be considered as regex by sed. '$d' acts like an address which points out the last line.