I have a csv file, with student name and marks. I want to update "marks" of a student with name "jack"(the only person in the csv). the data in csv file looks as below.
student,marks
jack,10
peter,20
rick,10
I found this awk '$1 == "Audrey" {print $2}' numbers.txt, but iam not sure on how to modify the file.
awk 'BEGIN{FS=OFS=","} $1=="jack"{$2=27} 1' foo.csv > tmp && mv tmp foo.csv
It worked for me with
sed -ir "s/^\(jack\),.*/\1,$new_grade/"
input.csv. with argument "r" or else i get the "error sed: 1: "input.csv": command i expects \ followed by text".
ed is usually better for in-place editing of files than sed:
printf "%s\n" "/^jack,/c" "jack,${new_grade}" "." w | ed -s input.csv
or using a heredoc to make it easier to read:
ed -s input.csv <<EOF
/^jack,/c
jack,${new_grade}
.
w
EOF
At the first line starting with jack,, change it to jack,XX where XX is the value of the new_grade variable, and write the new contents of the file.
You could use sed:
new_grade=9
sed -i'' "s/^\(jack\),.*/\1,$new_grade/"
The pattern ^\(jack\),.* matches the beginning of the line ^ followed by jack by a comma and the rest of the line .*. The replacement string \1,$new_mark contains the first captured group \1 (in this case jack) followed by a comma and the new mark.
Alternatively you could loop over the file and use a pattern substitution:
new_grade=9
while read -s line; do
echo ${line/jack,*/jack,$new_grade}
done < grades.txt > grades2.txt
Another approach with sed is to anchor the replacement to the digits at the end of the line with:
sed '/^jack,/s/[0-9][0-9]*$/12/' file
This uses the form sed '/find/s/match/replace' where find locates at the beginning of the line '^' the word "jack," eliminating all ambiguity with, e.g. jackson,33. Then the normal substitution form of 's/match/replace/' where match locates at least one digit at the end of the line (anchored by '$') and replaces it with the 12 (or whatever you choose).
Example Use/Output
With your example file in file, you would have:
$ sed '/^jack,/s/[0-9][0-9]*$/12/' file
student,marks
jack,12
peter,20
rick,10
(note: the POSIX character class of [[:digit:]] is equivalent to [0-9] which is another alternative)
The equivalent expression using the POSIX character class would be:
sed '/^jack,/s/[[:digit:]][[:digit:]]*$/12/' file
You can also use Extended Regular Expression which provides the '+' repetition operator to indicate one-or-more compared to the basic repetition designator of '*' to indicate zero-or-more. The equivalent ERE would be sed -E '/^jack,/s/[0-9]+$/12/' file
You can add the -i option to edit in-place and/or using it as -i.bak to create a backup of the original with the .bak extension before modifying the original.
I am grabbing the contents of https://api.wordpress.org/secret-key/1.1/salt/ using curl and asigning it to a bash variable. I want to insert this block of text into a file at a specific line number - 10
#!/bin/bash
salts=$(curl -s https://api.wordpress.org/secret-key/1.1/salt/)
sed -i '10i '"$salts"'' myfile.php
however I keep getting the error
sed: -e expression #1, char 102: extra characters after command
I think there maybe carriage return chars in the returned payload using curl but i'm not sure. I have tried using tr to replace them with \n but am unsure how to use it in this situation.
I have looked at multiple existing questions but I cannot get them to work in my situation. I don't have to use sed for this.
using awk
awk -v "s=$salts" 'NR==10{print s} 1' file
you could also combine head and tail
echo -e "$( head -9 file )\n$salts\n$( tail -n +10 file )"
boh of them seem to work with your variable that contains a lot of special characters.
I defined a marco switch in PublicDefine.h to control the target device in my code.
I meant to replace the target sentence automatically according to the options in my build shell.
Below is a peace of my code, I got an error "sed: -e expression #1, char 5: comments don't accept any addresses".
May somebody tell me how to deal with the #? Or you can just give me another suggestion. Thank you!
nLine=`grep NO_LCD PublicDefine.h -nR | cut -d ":" -f 1`
dfine_nolcd="#define NO_LCD 1"
ndfine_nolcd="#undef NO_LCD "
echo $dfine_nolcd #this is a debugging sentence
echo $ndfine_nolcd #this is a debugging sentence
echo "nLine $nLine"
if [ "$1"x = "NO_LCD"x ]; then
sed -i "${nLine} ${dfine_nolcd}" PublicDefine.h
else
sed -i "${nLine} ${ndfine_nolcd}" PublicDefine.h
fi
Okay, as you noticed, the problem you had was that you gave sed a line number and a string, but not a command. It took the # sign in your string as a command starting a comment.
But finding out the line number is still not necessary, as sed can match lines based on their contents instead of a line number, so you could just use something like these:
Change any line containing NO_LCD with a given string:
sed -e '/NO_LCD/c#define NO_LCD 1' PublicDefine.h
(/regex/ - on lines matching the regex, c - change line to following string.)
Or unconditionally try to do a string replacement over the full line:
sed -e 's/^.*NO_LCD.*$/#define NO_LCD 1/' PublicDefine.h
Assuming PublicDefine.h contains
something
NO_LCD replace me
something
both print
something
#define NO_LCD 1
something
sed -i '2 c xyz' file
can be used to replace the given line of file by "xyz".
The below sentence solved my problem:
sed -i "${nLine} c ${dfine_nolcd}" PublicDefine.h
File.txt
/aaa/bbb/ccc/ddd
/aaa/bbb/ccc/mmm
/aaa/eee/ccc/ddd
if my $(pwd) is /aaa/bbb/ccc
the it should delete only first two
I have tried like sed /^$(pwd)/d but not worked
The problem here is that you are using $(pwd), which tries to execute a command pwd. This result contains slashes, so that the final command is something like:
sed /^/aaa/bbb/ccc/d
Which sed cannot handle and returns an error:
sed: -e expression #1, char 4: extra characters after command
You should instead use another delimiter. For example, _:
sed "\_${PWD}_d"
As 123 comments below, you need to escape the first delimiter if it is not a substitution. I also enclose the var within ${ } to prevent the variable to be considered PWD_ instead of PWD.
You can use awk for a nicer approach:
$ awk -v patt="$PWD" '!($0 ~ patt)' file
/aaa/eee/ccc/ddd
Note $PWD is the same as executing pwd.
grep can also do the job:
grep -v "$(pwd)" file
Just to precise the answer of fedorqui...
In your question there is another problem because you variable $pwd contain special sed symbols (/).
So the sed will not be glad...
Some solution for example could be find here : Replace a string in shell script using a variable
So you could use additional variable to correct this problem.
This work perfectly for your example (I just replace echo $(pwd) by 'echo /aaa/bbb/ccc').
pwd_bis=$( echo $(pwd) | sed 's/[\/]/\\\0/g' )
sed "/^${pwd_bis}/d" File.txt
I have a file with about 1000 lines. I want the part of my file after the line which matches my grep statement.
That is:
cat file | grep 'TERMINATE' # It is found on line 534
So, I want the file from line 535 to line 1000 for further processing.
How can I do that?
The following will print the line matching TERMINATE till the end of the file:
sed -n -e '/TERMINATE/,$p'
Explained: -n disables default behavior of sed of printing each line after executing its script on it, -e indicated a script to sed, /TERMINATE/,$ is an address (line) range selection meaning the first line matching the TERMINATE regular expression (like grep) to the end of the file ($), and p is the print command which prints the current line.
This will print from the line that follows the line matching TERMINATE till the end of the file:
(from AFTER the matching line to EOF, NOT including the matching line)
sed -e '1,/TERMINATE/d'
Explained: 1,/TERMINATE/ is an address (line) range selection meaning the first line for the input to the 1st line matching the TERMINATE regular expression, and d is the delete command which delete the current line and skip to the next line. As sed default behavior is to print the lines, it will print the lines after TERMINATE to the end of input.
If you want the lines before TERMINATE:
sed -e '/TERMINATE/,$d'
And if you want both lines before and after TERMINATE in two different files in a single pass:
sed -e '1,/TERMINATE/w before
/TERMINATE/,$w after' file
The before and after files will contain the line with terminate, so to process each you need to use:
head -n -1 before
tail -n +2 after
IF you do not want to hard code the filenames in the sed script, you can:
before=before.txt
after=after.txt
sed -e "1,/TERMINATE/w $before
/TERMINATE/,\$w $after" file
But then you have to escape the $ meaning the last line so the shell will not try to expand the $w variable (note that we now use double quotes around the script instead of single quotes).
I forgot to tell that the new line is important after the filenames in the script so that sed knows that the filenames end.
How would you replace the hardcoded TERMINATE by a variable?
You would make a variable for the matching text and then do it the same way as the previous example:
matchtext=TERMINATE
before=before.txt
after=after.txt
sed -e "1,/$matchtext/w $before
/$matchtext/,\$w $after" file
to use a variable for the matching text with the previous examples:
## Print the line containing the matching text, till the end of the file:
## (from the matching line to EOF, including the matching line)
matchtext=TERMINATE
sed -n -e "/$matchtext/,\$p"
## Print from the line that follows the line containing the
## matching text, till the end of the file:
## (from AFTER the matching line to EOF, NOT including the matching line)
matchtext=TERMINATE
sed -e "1,/$matchtext/d"
## Print all the lines before the line containing the matching text:
## (from line-1 to BEFORE the matching line, NOT including the matching line)
matchtext=TERMINATE
sed -e "/$matchtext/,\$d"
The important points about replacing text with variables in these cases are:
Variables ($variablename) enclosed in single quotes ['] won't "expand" but variables inside double quotes ["] will. So, you have to change all the single quotes to double quotes if they contain text you want to replace with a variable.
The sed ranges also contain a $ and are immediately followed by a letter like: $p, $d, $w. They will also look like variables to be expanded, so you have to escape those $ characters with a backslash [\] like: \$p, \$d, \$w.
As a simple approximation you could use
grep -A100000 TERMINATE file
which greps for TERMINATE and outputs up to 100,000 lines following that line.
From the man page:
-A NUM, --after-context=NUM
Print NUM lines of trailing context after matching lines.
Places a line containing a group separator (--) between
contiguous groups of matches. With the -o or --only-matching
option, this has no effect and a warning is given.
A tool to use here is AWK:
cat file | awk 'BEGIN{ found=0} /TERMINATE/{found=1} {if (found) print }'
How does this work:
We set the variable 'found' to zero, evaluating false
if a match for 'TERMINATE' is found with the regular expression, we set it to one.
If our 'found' variable evaluates to True, print :)
The other solutions might consume a lot of memory if you use them on very large files.
If I understand your question correctly you do want the lines after TERMINATE, not including the TERMINATE-line. AWK can do this in a simple way:
awk '{if(found) print} /TERMINATE/{found=1}' your_file
Explanation:
Although not best practice, you could rely on the fact that all variables defaults to 0 or the empty string if not defined. So the first expression (if(found) print) will not print anything to start off with.
After the printing is done, we check if this is the starter-line (that should not be included).
This will print all lines after the TERMINATE-line.
Generalization:
You have a file with start- and end-lines and you want the lines between those lines excluding the start- and end-lines.
start- and end-lines could be defined by a regular expression matching the line.
Example:
$ cat ex_file.txt
not this line
second line
START
A good line to include
And this line
Yep
END
Nope more
...
never ever
$ awk '/END/{found=0} {if(found) print} /START/{found=1}' ex_file.txt
A good line to include
And this line
Yep
$
Explanation:
If the end-line is found no printing should be done. Note that this check is done before the actual printing to exclude the end-line from the result.
Print the current line if found is set.
If the start-line is found then set found=1 so that the following lines are printed. Note that this check is done after the actual printing to exclude the start-line from the result.
Notes:
The code rely on the fact that all AWK variables defaults to 0 or the empty string if not defined. This is valid, but it may not be best practice so you could add a BEGIN{found=0} to the start of the AWK expression.
If multiple start-end-blocks are found, they are all printed.
grep -A 10000000 'TERMINATE' file
is much, much faster than sed, especially working on really a big file. It works up to 10M lines (or whatever you put in), so there isn't any harm in making this big enough to handle about anything you hit.
Use Bash parameter expansion like the following:
content=$(cat file)
echo "${content#*TERMINATE}"
There are many ways to do it with sed or awk:
sed -n '/TERMINATE/,$p' file
This looks for TERMINATE in your file and prints from that line up to the end of the file.
awk '/TERMINATE/,0' file
This is exactly the same behaviour as sed.
In case you know the number of the line from which you want to start printing, you can specify it together with NR (number of record, which eventually indicates the number of the line):
awk 'NR>=535' file
Example
$ seq 10 > a #generate a file with one number per line, from 1 to 10
$ sed -n '/7/,$p' a
7
8
9
10
$ awk '/7/,0' a
7
8
9
10
$ awk 'NR>=7' a
7
8
9
10
If for any reason, you want to avoid using sed, the following will print the line matching TERMINATE till the end of the file:
tail -n "+$(grep -n 'TERMINATE' file | head -n 1 | cut -d ":" -f 1)" file
And the following will print from the following line matching TERMINATE till the end of the file:
tail -n "+$(($(grep -n 'TERMINATE' file | head -n 1 | cut -d ":" -f 1)+1))" file
It takes two processes to do what sed can do in one process, and if the file changes between the execution of grep and tail, the result can be incoherent, so I recommend using sed. Moreover, if the file doesn’t not contain TERMINATE, the first command fails.
Alternatives to the excellent sed answer by jfg956, and which don't include the matching line:
awk '/TERMINATE/ {y=1;next} y' (Hai Vu's answer to 'grep +A': print everything after a match)
awk '/TERMINATE/ ? c++ : c' (Steven Penny's answer to 'grep +A': print everything after a match)
perl -ne 'print unless 1 .. /TERMINATE/' (tchrist's answer to 'grep +A': print everything after a match)
This could be one way of doing it. If you know in what line of the file you have your grep word and how many lines you have in your file:
grep -A466 'TERMINATE' file
sed is a much better tool for the job:
sed -n '/re/,$p' file
where re is a regular expression.
Another option is grep's --after-context flag. You need to pass in a number to end at, using wc on the file should give the right value to stop at. Combine this with -n and your match expression.
This will print all lines from the last found line "TERMINATE" till the end of the file:
LINE_NUMBER=`grep -o -n TERMINATE $OSCAM_LOG | tail -n 1 | sed "s/:/ \\'/g" | awk -F" " '{print $1}'`
tail -n +$LINE_NUMBER $YOUR_FILE_NAME