I'm trying to do variable substitution in sed/bash.
I have seen this example available here, but it does not replace the data as required. The value observed in line 20 of $C_CONF is $RAM, when I want it to be 100.
Line 20 of $C_CONF is AllowedRAMSpace=SomeValue and I would like to change that value on-the-fly.
This is the trick I created, but it is a very ugly and inefficient one.
32 for RAM in 100 #80 60 40 20
33 do
34 sudo sed -i -re 's/(AllowedRAMSpace=)[^=]*$/\1"'"$RAM"'"/' $C_CONF
35 sudo sed -i '20s/"//' $C_CONF
36 sudo sed -i '20s/"//' $C_CONF
37 done
What is the other way?
Try using awk instead
awk 'BEGIN{RAM=100}NR==20{print gensub(/(AllowedRAMSpace=)([^=].*$)/,"\\1"RAM,"g", $0)}' filename
Related
I have 45 lines of code that need a sed command. Due to the recent change in GNU all my scripts are breaking and need -std=legacy & -fallow-invalid-boz. The only way I know how to do this is with sed. I'm not a computer programmer and sed is simple and easy to understand.
These are a sample of my sed commands.
Is there a way to do all these sed commands in a loop or with sed itself. If there is another editor that makes it easier I can try to learn that too.
I have tried this
for X in [24,28,32,36,40,45,49,53,56,60,64,68,69,73,74,79]
sed -i '$Xs/= /= -std=legacy -fallow-invalid-boz /g' $HOME/WRF/Downloads/NCEPlibs/macros.make.linux.gnu
done
But I get the error:
$ for X in [24,28,32,36,40,45,49,53,56,60,64,68,69,73,74,79] sed -i
'$Xs/= /= -std=legacy -fallow-invalid-boz /g'
$HOME/WRF/Downloads/NCEPlibs/macros.make.linux.gnu done bash: syntax
error near unexpected token sed' bash: syntax error near unexpected token done'
doing it like this is a lot smarter
y="24 28 32 36 40 45 49 53 56 60 64 68 69 73 74 79"
for X in $y; do
sed -i "${X}s/= /= -std=legacy -fallow-invalid-boz /g" $HOME/WRF/Downloads/NCEPlibs/macros.make.linux.gnu
done
First of all, you forgot the do statement in for so the for statement will just fail before it can even execute.
Second of all [24,28,32,36,40,45,49,53,56,60,64,68,69,73,74,79] in not valid as for uses newlines and or white spaces to declare a new value going from left to right.
And last but not least, using $X is not valid in this example as bash reads it as $Xs/ so using ${X} is the correct way and of course using "" instead of using '' so ${X} can actually be used.
I want to get the longest line in /etc/sudoers.
For wc.
sudo wc -L /etc/sudoers
90 /etc/sudoers
For awk
sudo awk '{if(length>L){L=length;s=$0}}END{print NR,L,s}' /etc/sudoers
27 83 Defaults secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
awk treat 8 whitespaces in 27th line as just 1 whitespace,8-1=7,90-83=7.
So the numbers got with wc is 7 bigger than awk.
How to keep the numbers of whitspace in a line unchanged with awk,to make awk treat 8 whitespaces in 27th line unchanged ,instead of 1.
How to fix the command sudo awk '{if(length>L){L=length;s=$0}}END{print NR,L,s}' /etc/sudoers ,to output the following result with awk.
27 90 Defaults secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
This is probably happening because the file contains tabs.
With the --max-line-length' option,wc' prints the length of the
longest line per file, and if there is more than one file it prints
the maximum (not the sum) of those lengths. The line lengths here are
measured in screen columns, according to the current locale and
assuming tab positions in every 8th column.
So wc assumes each tab to be of width 8. We have to emulate this in awk. One special case when the tab only occurs in the beginning of the line is easy to mend.
sudo awk '{sub("\t"," ") ;
if(length>L){L=length;s=$0}}END{print NR,L,s}' /etc/sudoers
The max length of the line in /etc/sudoers contained a tab, as you can see from the hexdump of the file,
$ sudo cat /etc/sudoers | hexdump -C
...
...
00000110 73 73 0a 44 65 66 61 75 6c 74 73 09 73 65 63 75 |ss.Defaults.secu|
...
...
You may see 09 between 2 s (hex: 73). And look it into the ascii table, it would be horizontal tab.
You may see from #Dmitri Chubarov's answer, wc would treat the length of '\t' as 8.
An alternative way to make awk output the same result as wc -L can be,
$ sudo awk '{if(length>L){L=7*(match($0,/\t/)?1:0)+length;s=$0}}END{print NR,L,s}' /etc/sudoers
30 100 Defaults secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin"
The function match($0,/\t/)?1:0 would return 1 any \t existed, add 7 for the length if yes.
This question already has answers here:
Replace a string in shell script using a variable
(12 answers)
Closed 6 years ago.
I have a file and I want to append the value in a unix variable at the end of each line using SED.
I have already achieved this through AWK but I would like to do it in SED
something like this. I have already tried the below command and it's not working.
sed -i 's/$/"$BATCH_RUN_DATE"/g' data.csv
Error
sed: -e expression #1, char 10: unknown option to `s'
GNU sed version 4.2.1
Regards,
Aswinikumar
here is an example of how you could do it:
% cat subject.txt
Studid StudName Asp.Net DBMS Unix
1 Ani 75 62 80
2 George 90 95 82
3 Jake 45 30 40
4 Dennie 89 92 90
%
% my_var="R2D2"
%
% sed "s/$/${my_var}/" subject.txt
Studid StudName Asp.Net DBMS UnixR2D2
1 Ani 75 62 80R2D2
2 George 90 95 82R2D2
3 Jake 45 30 40R2D2
4 Dennie 89 92 90R2D2
Explanation
The tricky thing about this sed expression, is why don't I need to reinsert the $ newline in the replace part of the substitution?.
I believe this is because when sed loads each line into the pattern space it removes the newline, then after all pattern space operations, it then re-attaches a newline to the contents of the pattern space then prints the pattern space, so I guess the newline is a kind of a "freebie" here.
Then you might ask the question, how does the first $ match if the pattern space has no newline in it ? -- if I had to guess, I might say , sed just knows what you mean when you use the $ meta character, but that is just a guess.
You should put the script itself inside double quotes if you wish bash variable expansion to happen
sed -i "s/$/$BATCH_RUN_DATE/" data.csv
should do the job. This is the most easiest way to do it. Also, note you don't need the global flag as there is only one substitution per line.
Sidenotes
Check if your sed supports inplace edit option
Check if $BATCH_RUN_DATE itself is non-empty.
I want to find top CPU usage pid, my script is here:
#!/bin/sh
ppid=`top -n 1 |sed -n 8p |awk '{print $1}'`
echo $ppid
ps aux|grep $ppid
but I get an error:
grep: Unmatched [ or [^
Why? How can I fix it?
OK! I found it! The problem is that top is including terminal control sequences in its output. So you don't actually see it in the echo, but I noticed because it had put my terminal session into bold output and later I was trying to figure out how it happened and I traced it back to my testing for this question. So, the [ that grep was complaining about is in the escape sequence that ppid gets set to. I got a work around by adding |tr -dc 0-9 after the awk, i.e.
ppid=`top -n 1 |sed -n 8p |awk '{print $1}'|tr -dc 0-9`
That will delete anything that's not a digit at the end. But the sed needs to be adjusted, too, I think. And, I suspect some of the digits may be from the escape sequence, so you need to come up with a cleaner way to excise the escape sequence.
But, in the final analysis this will be pretty useless. The highest user of the CPU every time I ran that pipeline was the top process that's part of it. In retrospect that's probably not surprising.
When all else fails, examine the input:
$ top -n 1 | awk 'NR==8 {print $1 ": " $2}' | hexdump -C
00000000 1b 28 42 1b 5b 6d 1b 5b 31 6d 31 38 37 31 35 3a |.(B.[m.[1m18715:|
00000010 20 6a 6b 6c 6f 77 64 65 6e 0a | jklowden.|
0000001a
(I shortened your command. Nearly every command that combines sed and awk can be better expressed with just awk. Then I added the second field, so we could see what's going on.)
The result is nondeterministic. top will highlight changed lines; to do so, it emits an ANSI escape sequence. If you capture one, you'll capture that sequence -- esc(Besc[mesc[1m -- which will look very weird indeed to ps. The brackets in that sequence doubtless provoked your error message.
To fix that, your top probably has a batch mode. In mine, top -n 1 -b does the trick.
How to use ps to find a command line?
I'm afraid the best answer is RTFM. ps is one of those commands with a lot of variation across systems. My GNU version likes this:
$ ps -c -f -p $(top -n 1 -b | awk 'NR==8 {print $1}')
UID PID PPID CLS PRI STIME TTY TIME CMD
root 1300 1 TS 19 Jun30 ? 00:14:46 /usr/bin/python /usr/bin/la
Other observations:
Prefer $() to backticks
While this kind of thing is fine for learning, look for solutions that don't involve parsing output from interactive utilities. Under account in the manual, you'll find ways to capture much more information than just command line of what happens to be on top at the moment.
HTH.
Because your ppid retrieve null value
change your command to retrieve ppid value as below
top -n 1 | sed -n 8p | awk -F " " '{print$2}'
And other thing for happening this is top process will end immediately
This is in my ubuntu 14.04 os.
I have a bash script with that loops through lines of a file (a.txt) containing paths to a list of gzipped files and searches for phrases in each of those files using zgrep.
My bash script is:
for i in $(cat $1); do
echo $i;
echo zgrep -E '"phrase1|phrase2|phrase3|phrase4|phrase5|phrase6"' $i;
zgrep -E '"phrase1|phrase2|phrase3|phrase4|phrase5|phrase6"' $i;
done
which I call by myscript.sh a.txt:
The output is:
zgrep -E "phrase1|phrase2|phrase3|phrase4|phrase5|phrase6" myzippedfile.1.gz
phrase2 48 48.00 48
phrase3 35 35.00 35
phrase4 67 67.00 67
phrase5 99 99.00 99
(repeated for each file listed in a.txt).
However, when I execute the zgrep command that is being executed inside the scripts for loop, I get a different output.
Executing:
zgrep -E "phrase1|phrase2|phrase3|phrase4|phrase5|phrase6" myzippedfile.1.gz
at the command line yields:
phrase1 29 29.00 29
phrase2 48 48.00 48
phrase3 35 35.00 35
phrase4 67 67.00 67
phrase5 99 99.00 99
phrase6 54 54.00 54
This output is correct, whereas the output generated from the bash script for loop is missing the first and last lines. How can this be?
Does anyone see any issues in my bash script? Why would the first and last lines be missing from the output?
The first and last elements in your alternatives contain literal " characters. So you'll only match phrase1 if there's a " before it, and only match phrase6 if there's a " after it. You shouldn't have those double quotes in the pattern, it should be:
zgrep -E 'phrase1|phrase2|phrase3|phrase4|phrase5|phrase6' $i;
You don't have the extra quotes when you execute the zgrep command by hand.