I have 45 lines of code that need a sed command. Due to the recent change in GNU all my scripts are breaking and need -std=legacy & -fallow-invalid-boz. The only way I know how to do this is with sed. I'm not a computer programmer and sed is simple and easy to understand.
These are a sample of my sed commands.
Is there a way to do all these sed commands in a loop or with sed itself. If there is another editor that makes it easier I can try to learn that too.
I have tried this
for X in [24,28,32,36,40,45,49,53,56,60,64,68,69,73,74,79]
sed -i '$Xs/= /= -std=legacy -fallow-invalid-boz /g' $HOME/WRF/Downloads/NCEPlibs/macros.make.linux.gnu
done
But I get the error:
$ for X in [24,28,32,36,40,45,49,53,56,60,64,68,69,73,74,79] sed -i
'$Xs/= /= -std=legacy -fallow-invalid-boz /g'
$HOME/WRF/Downloads/NCEPlibs/macros.make.linux.gnu done bash: syntax
error near unexpected token sed' bash: syntax error near unexpected token done'
doing it like this is a lot smarter
y="24 28 32 36 40 45 49 53 56 60 64 68 69 73 74 79"
for X in $y; do
sed -i "${X}s/= /= -std=legacy -fallow-invalid-boz /g" $HOME/WRF/Downloads/NCEPlibs/macros.make.linux.gnu
done
First of all, you forgot the do statement in for so the for statement will just fail before it can even execute.
Second of all [24,28,32,36,40,45,49,53,56,60,64,68,69,73,74,79] in not valid as for uses newlines and or white spaces to declare a new value going from left to right.
And last but not least, using $X is not valid in this example as bash reads it as $Xs/ so using ${X} is the correct way and of course using "" instead of using '' so ${X} can actually be used.
Related
I have a file which has 109 lines.
I perform the two operations on the line shown below.
# Delete line 74
sed -i '74d' Test.txt
# Add the entry to line 109
sed -i "109iThis is the string" Test.txt
I see line 74 getting deleted from my Test.txt, but for some reasons, now my Test.txt has only 108 lines, and I don’t see the This is the string being added to line 109.
I am not sure what the error is. How can I fix it?
You may use this POSIX sed command:
sed -i.bak '74d; $ a\
This is the string
' file
This will delete 74th line from file and append a line in the end and will save changes inline.
Note that this will work with gnu-sed as well.
Jonathan already mentioned the potential issues with using sed -i (non-standard, behaves in different ways when supported depending on implementation, etc.). Avoid them by using ed to edit files:
ed -s Test.txt <<EOF
109a
This is the string
.
74d
w
EOF
Note how this appends, and then deletes. Because ed acts on entire files, not a stream of lines, commands to act on specific lines can be in any order.
If you remove a line, the file has only 108 lines left. Correct your second command accordingly:
sed -i "108iThis is the string" Test.txt
Line number 109 does not exist (you removed one, 109-1=108), you must add it before you can enter text into it.
Solution:
sed -i '$ a <text>' Test.txt
The new line will be added with the selected text.
I'm trying to do variable substitution in sed/bash.
I have seen this example available here, but it does not replace the data as required. The value observed in line 20 of $C_CONF is $RAM, when I want it to be 100.
Line 20 of $C_CONF is AllowedRAMSpace=SomeValue and I would like to change that value on-the-fly.
This is the trick I created, but it is a very ugly and inefficient one.
32 for RAM in 100 #80 60 40 20
33 do
34 sudo sed -i -re 's/(AllowedRAMSpace=)[^=]*$/\1"'"$RAM"'"/' $C_CONF
35 sudo sed -i '20s/"//' $C_CONF
36 sudo sed -i '20s/"//' $C_CONF
37 done
What is the other way?
Try using awk instead
awk 'BEGIN{RAM=100}NR==20{print gensub(/(AllowedRAMSpace=)([^=].*$)/,"\\1"RAM,"g", $0)}' filename
I have a script that has a variable that might contain some weird characters: 🍿 ✔. I need to remove them but, honestly, I don't even know where to begin to match those characters. I can't copy and paste them into my script, they just show up as ?? ?. How can a match those characters with sed or awk? I don't have the ability to use perl or php or anything much beyond sed or awk due to system availability.
First, put some flag strings around your special chars and then hexdump -C so you can easily see them. Then use HEX code to write the sed command. For example:
[STEP 118] # cat file
>>>🍿 ✔<<<
[STEP 119] # hexdump -C file
00000000 3e 3e 3e f0 9f 8d bf 20 e2 9c 94 3c 3c 3c 0a |>>>.... ...<<<.|
^^^^^^^^^^^^^^^^^^^^^^^^
[STEP 120] # sed -e $'s/\xf0\x9f\x8d\xbf\x20\xe2\x9c\x94//g' file # need to use the $'...'
>>><<<
[STEP 121] #
Then remove the added flag strings when all is done.
Try this - (file contain some control M and the character that you have mentioned in the question and I am trying to print only the alphanumeric character)
$cat f
hello vipin
street1
pin 12345
🍿 ✔
$awk '/[[:alnum:]]/ {print }' f
hello vipin
street1
pin 12345
Looks like control M character is getting disappeared after saving the input file on SO.
$ cat file
some weird characters: 🍿 ✔. I need to remove
second line of some weird characters: 🍿 ✔. I need to remove
$ tr -c -d '[:print:][:space:]' < file
some weird characters: . I need to remove
second line of some weird characters: . I need to remove
The solution I ended up using was just changing the encoding of the script to UTF-8 instead of ASCII. I did this with notepad++. Then I could work with the character directly instead of some roundabout way of converting to hex (which I couldn't do anyway as the variable is an environmental variable and not from a file) or something else. I also didn't need to use awk or sed as the following was much simpler:
cleaned_var=${environmental_variable//" 🍿 ✔"}
This question already has answers here:
Replace a string in shell script using a variable
(12 answers)
Closed 6 years ago.
I have a file and I want to append the value in a unix variable at the end of each line using SED.
I have already achieved this through AWK but I would like to do it in SED
something like this. I have already tried the below command and it's not working.
sed -i 's/$/"$BATCH_RUN_DATE"/g' data.csv
Error
sed: -e expression #1, char 10: unknown option to `s'
GNU sed version 4.2.1
Regards,
Aswinikumar
here is an example of how you could do it:
% cat subject.txt
Studid StudName Asp.Net DBMS Unix
1 Ani 75 62 80
2 George 90 95 82
3 Jake 45 30 40
4 Dennie 89 92 90
%
% my_var="R2D2"
%
% sed "s/$/${my_var}/" subject.txt
Studid StudName Asp.Net DBMS UnixR2D2
1 Ani 75 62 80R2D2
2 George 90 95 82R2D2
3 Jake 45 30 40R2D2
4 Dennie 89 92 90R2D2
Explanation
The tricky thing about this sed expression, is why don't I need to reinsert the $ newline in the replace part of the substitution?.
I believe this is because when sed loads each line into the pattern space it removes the newline, then after all pattern space operations, it then re-attaches a newline to the contents of the pattern space then prints the pattern space, so I guess the newline is a kind of a "freebie" here.
Then you might ask the question, how does the first $ match if the pattern space has no newline in it ? -- if I had to guess, I might say , sed just knows what you mean when you use the $ meta character, but that is just a guess.
You should put the script itself inside double quotes if you wish bash variable expansion to happen
sed -i "s/$/$BATCH_RUN_DATE/" data.csv
should do the job. This is the most easiest way to do it. Also, note you don't need the global flag as there is only one substitution per line.
Sidenotes
Check if your sed supports inplace edit option
Check if $BATCH_RUN_DATE itself is non-empty.
I want to find top CPU usage pid, my script is here:
#!/bin/sh
ppid=`top -n 1 |sed -n 8p |awk '{print $1}'`
echo $ppid
ps aux|grep $ppid
but I get an error:
grep: Unmatched [ or [^
Why? How can I fix it?
OK! I found it! The problem is that top is including terminal control sequences in its output. So you don't actually see it in the echo, but I noticed because it had put my terminal session into bold output and later I was trying to figure out how it happened and I traced it back to my testing for this question. So, the [ that grep was complaining about is in the escape sequence that ppid gets set to. I got a work around by adding |tr -dc 0-9 after the awk, i.e.
ppid=`top -n 1 |sed -n 8p |awk '{print $1}'|tr -dc 0-9`
That will delete anything that's not a digit at the end. But the sed needs to be adjusted, too, I think. And, I suspect some of the digits may be from the escape sequence, so you need to come up with a cleaner way to excise the escape sequence.
But, in the final analysis this will be pretty useless. The highest user of the CPU every time I ran that pipeline was the top process that's part of it. In retrospect that's probably not surprising.
When all else fails, examine the input:
$ top -n 1 | awk 'NR==8 {print $1 ": " $2}' | hexdump -C
00000000 1b 28 42 1b 5b 6d 1b 5b 31 6d 31 38 37 31 35 3a |.(B.[m.[1m18715:|
00000010 20 6a 6b 6c 6f 77 64 65 6e 0a | jklowden.|
0000001a
(I shortened your command. Nearly every command that combines sed and awk can be better expressed with just awk. Then I added the second field, so we could see what's going on.)
The result is nondeterministic. top will highlight changed lines; to do so, it emits an ANSI escape sequence. If you capture one, you'll capture that sequence -- esc(Besc[mesc[1m -- which will look very weird indeed to ps. The brackets in that sequence doubtless provoked your error message.
To fix that, your top probably has a batch mode. In mine, top -n 1 -b does the trick.
How to use ps to find a command line?
I'm afraid the best answer is RTFM. ps is one of those commands with a lot of variation across systems. My GNU version likes this:
$ ps -c -f -p $(top -n 1 -b | awk 'NR==8 {print $1}')
UID PID PPID CLS PRI STIME TTY TIME CMD
root 1300 1 TS 19 Jun30 ? 00:14:46 /usr/bin/python /usr/bin/la
Other observations:
Prefer $() to backticks
While this kind of thing is fine for learning, look for solutions that don't involve parsing output from interactive utilities. Under account in the manual, you'll find ways to capture much more information than just command line of what happens to be on top at the moment.
HTH.
Because your ppid retrieve null value
change your command to retrieve ppid value as below
top -n 1 | sed -n 8p | awk -F " " '{print$2}'
And other thing for happening this is top process will end immediately
This is in my ubuntu 14.04 os.