I have a file that looks like this:
[
{
"ncyc" : 28817,
"icels" : 128,
"jcels" : 128,
"t" : 0.185896E-006,
"dt" : 0.955602E-012,
"dtcour" : 0.100000E+021,
"dti" : 0.100000E+021,
"dtc" : 0.262902E-011,
"dtvol" : 0.239735E-010,
"dthall" : 0.100000E+021,
"dtlaser" : -0.925596E+062,
"dtmax" : 0.200000E-009,
}
]
I want to delete the last comma of this file. It appears at the 14th line at position 34. I could do this manually if it was one file but I have to do this for 300 files
sed is your friend:
sed -i.bak '14s/,[[:blank:]]*$//' file ...
This is a bit fragile: it assumes the line to remove is always the 14th, not necessarily the line before the closing brace.
Depending on the platform sed or awk might have varying results, perl might be more flexible:
perl -i.bak -00pe 's/,(?!.*,)//s' file
# , matches a comma.
# (?!.*,) negative lookahead asserts no comma after matched comma.
# s is a DOTALL modifier matching newline characters also.
This is a straightforward ed one-liner:
ed foo.json <<EOF
?,?s/,\([^,]*\)$/\1/
wq
EOF
That line can be broken into an address and a command.
The address is ?,?, namely the previous line matching the regular expression ,.
The command is s/re/replacement/, where the regular expression is ,\([^,]*\)$ (a literal ,, a captured group of zero or more character that are not ,, and the end of the line), and the replacement is \1 (the first captured group).
Technically it's a two-line ed script, wq to save and quit.
You could invoke this in a loop with find, for instance:
find . -name '*.json' | while read name ; do
ed -s $name <<EOF
H
[…ed commands…]
wq
EOF
done
I've also added ed -s to suppress the file size message, and H to output verbose errors instead of the infamous ?.
Thanks for the answers. I was easily able to solve the question myself using Python:
f=open(fjson, 'r')
data= f.readlines()
ndx=len(data)
data[ndx-3]= data[ndx-3].replace(',', '')
Related
I have files that need to be removed from comments and white space until keyword . Line number varies . Is it possible to limit multiple continued sed substitutions based on Keyword ?
This removes all comments and white spaces from file :
sed -i -e 's/#.*$//' -e 's/;.*$//' -e '/^$/d' file
For example something like this :
# string1
# string2
some string
; string3
; string4
####
<Keyword_Keep_this_line_and_comments_white_space_after_this>
# More comments that need to be here
; etc.
sed -i '1,/keyword/{/^[#;]/d;/^$/d;}' file
I would suggest using awk and setting a flag when you reach your keyword:
awk '/Keyword/ { stop = 1 } stop || !/^[[:blank:]]*([;#]|$)/' file
Set stop to true when the line contains Keyword. Do the default action (print the line) when stop is true or when the line doesn't match the regex. The regex matches lines whose first non-blank character is a semicolon or hash, or blank lines. It's slightly different to your condition but I think it does what you want.
The command prints to standard output so you should redirect to a new file and then overwrite the original to achieve an "in-place edit":
awk '...' input > tmp && mv tmp input
Use grep -n keyword to get the line number that contains the keyword.
Use sed -i -e '1,N s/#..., when N is the line number that contains the keyword, to only remove comments on the lines 1 to N.
I have a file that contains data like this
word0:secondword0
word1:secondword1
word2:secondword2
word3:secondword3
word4:secon:word4
I'd like to use sed to split that content to give me only the second word after the first colon.
The end result would look like
secondword0
secondword1
secondword2
secondword3
secon:word4
Notice how the last word has a second colon that is part of the word.
How would I write such a script that splits on only the fist colon but retains the rest?
Following sed could help you in same.
sed 's/\([^:]*\):\(.*\)/\2/' Input_file
Output will be as follows.
secondword0
secondword1
secondword2
secondword3
secon:word4
This can be done with gnu grep
grep -Po ':\K.*' <<END
word0:secondword0
word1:secondword1
word2:secondword2
word3:secondword3
word4:secon:word4
END
: matches the first occurence of : and \K keep : out of match .* matches the rest of the line, -o outputs only match
SET_VALUE(ab.ms.r.gms_dil_cfg.f().gms_dil_mode, dsad_sd );
How can I use sed to replace only from the SET_VALUE until the , with each letter after _ to be upper case?
result:
SET_VALUE(ab.ms.r.gmsDilCfg.f().gmsDilMode, dsad_sd );
For your input string you may apply the following sed expression + bash variable substitution:
s="SET_VALUE(ab.ms.r.gms_dil_cfg.f().gms_dil_mode, dsad sd )"
res=$(sed '1s/_\([a-z]\)/\U\1/g;' <<< "${s%,*}"),${s#*,}
echo "$res"
The output:
SET_VALUE(ab.ms.r.gmsDilCfg.f().gmsDilMode, dsad_sd );
Got distracted while writing this one up so Roman beat me to the punch, but this has a slight variation so figured I'd post it as another option ...
$ s="SET_VALUE(ab.ms.r.gms_dil_cfg.f().gms_dil_mode, dsad_sd );"
$ sed 's/,/,\n/g' <<< "$s" | sed -n '1{s/_\([a-z]\)/\U\1/g;N;s/\n//;p}'
SET_VALUE(ab.ms.r.gmsDilCfg.f().gmsDilMode, dsad_sd );
s/,/,\n/g : break input into separate lines at the comma (leave comma on first line, push rest of input to a second line)
at this point we've broken our input into 2 lines; the second sed invocation will now be working with a 2-line input
sed -n : refrain from printing input lines as they're processed; we'll explicitly print lines when required
1{...} : for the first line, apply the commands inside the braces ...
s/_\([a-z]\)/\U\1/g : for each pattern we find like '_[a-z]', save the [a-z] in buffer #1, and replace the pattern with the upper case of the contents of buffer #1
at this point we've made the desired edits to line #1 (ie, everything before the comma in the original input), now ...
N : read and append the next line into the pattern space
s/\n// : replace the carriage return with a null character
at this point we've pasted lines #1 and #2 together into a single line
p : print the pattern space
It took me a while to figure out how to do this, so posting in case anyone else is looking for the same.
For adding a newline after a pattern, you can also say:
sed '/pattern/{G;}' filename
Quoting GNU sed manual:
G
Append a newline to the contents of the pattern space, and then append the contents of the hold space to that of the pattern space.
EDIT:
Incidentally, this happens to be covered in sed one liners:
# insert a blank line below every line which matches "regex"
sed '/regex/G'
This sed command:
sed -i '' '/pid = run/ a\
\
' file.txt
Finds the line with: pid = run
file.txt before
; Note: the default prefix is /usr/local/var
; Default Value: none
;pid = run/php-fpm.pid
; Error log file
and adds a linebreak after that line inside file.txt
file.txt after
; Note: the default prefix is /usr/local/var
; Default Value: none
;pid = run/php-fpm.pid
; Error log file
Or if you want to add text and a linebreak:
sed -i '/pid = run/ a\
new line of text\
' file.txt
file.txt after
; Note: the default prefix is /usr/local/var
; Default Value: none
;pid = run/php-fpm.pid
new line of text
; Error log file
A simple substitution works well:
sed 's/pattern.*$/&\n/'
Example :
$ printf "Hi\nBye\n" | sed 's/H.*$/&\nJohn/'
Hi
John
Bye
To be standard compliant, replace \n by backslash newline :
$ printf "Hi\nBye\n" | sed 's/H.*$/&\
> John/'
Hi
John
Bye
sed '/pattern/a\\r' file name
It will add a return after the pattern while g will replace the pattern with a blank line.
If a new line (blank) has to be added at end of the file use this:
sed '$a\\r' file name
Another possibility, e.g. if You don't have an empty hold register, could be:
sed '/pattern/{p;s/.*//}' file
Explanation:
/pattern/{...} = apply sequence of commands, if line with pattern found,
p = print the current line,
; = separator between commands,
s/.*// = replace anything with nothing in the pattern register,
then automatically print the empty pattern register as additional line)
The easiest option -->
sed 'i\
' filename
I have a string which i need to insert at a specific position in a file :
The file contains multiple semicolons(;) i need to insert the string just before the last ";"
Is this possible with SED ?
Please do post the explanation with the command as I am new to shell scripting
before :
adad;sfs;sdfsf;fsdfs
string = jjjjj
after
adad;sfs;sdfsf jjjjj;fsdfs
Thanks in advance
This might work for you:
echo 'adad;sfs;sdfsf;fsdfs'| sed 's/\(.*\);/\1 jjjjj;/'
adad;sfs;sdfsf jjjjj;fsdfs
The \(.*\) is greedy and swallows the whole line, the ; makes the regexp backtrack to the last ;. The \(.*\) make s a back reference \1. Put all together in the RHS of the s command means insert jjjjj before the last ;.
sed 's/\([^;]*\)\(;[^;]*;$\)/\1jjjjj\2/' filename
(substitute jjjjj with what you need to insert).
Example:
$ echo 'adad;sfs;sdfsf;fsdfs;' | sed 's/\([^;]*\)\(;[^;]*;$\)/\1jjjjj\2/'
adad;sfs;sdfsfjjjjj;fsdfs;
Explanation:
sed finds the following pattern: \([^;]*\)\(;[^;]*;$\). Escaped round brackets (\(, \)) form numbered groups so we can refer to them later as \1 and \2.
[^;]* is "everything but ;, repeated any number of times.
$ means end of the line.
Then it changes it to \1jjjjj\2.
\1 and \2 are groups matched in first and second round brackets.
For now, the shorter solution using sed : =)
sed -r 's#;([^;]+);$#; jjjjj;\1#' <<< 'adad;sfs;sdfsf;fsdfs;'
-r option stands for extented Regexp
# is the delimiter, the known / separator can be substituted to any other character
we match what's finishing by anything that's not a ; with the ; final one, $ mean end of the line
the last part from my explanation is captured with ()
finally, we substitute the matching part by adding "; jjjj" ans concatenate it with the captured part
Edit: POSIX version (more portable) :
echo 'adad;sfs;sdfsf;fsdfs;' | sed 's#;\([^;]\+\);$#; jjjjj;\1#'
echo 'adad;sfs;sdfsf;fsdfs;' | sed -r 's/(.*);(.*);/\1 jjjj;\2;/'
You don't need the negation of ; because sed is by default greedy, and will pick as much characters as it can.
sed -e 's/\(;[^;]*\)$/ jjjj\1/'
Inserts jjjj before the part where a semicolon is followed by any number of non-semicolons ([^;]*) at the end of the line $. \1 is called a backreference and contains the characters matched between \( and \).
UPDATE: Since the sample input has no longer a ";" at the end.
Something like this may work for you:
echo "adad;sfs;sdfsf;fsdfs"| awk 'BEGIN{FS=OFS=";"} {$(NF-1)=$(NF-1) " jjjjj"; print}'
OUTPUT:
adad;sfs;sdfsf jjjjj;fsdfs
Explanation: awk starts with setting FS (field separator) and OFS (output field separator) as semi colon ;. NF in awk stands for number of fields. $(NF-1) thus means last-1 field. In this awk command {$(NF-1)=$(NF-1) " jjjjj" I am just appending jjjjj to last-1 field.