I have shell script (.sh) and I'm trying to insert contents of a file onto another file using the following commands but I it's throwing an error "sed: -e expression #1, char 28: unknown option to `s'":
filename="/home/user1/filename.txt"
contents=$(du -sh /var/log/test.log)
hostname > $filename
sed -i "/test_string/ s/$/, $contents" $filename
I can't seem to figure out where the underlying issue is. Can someone please help?
Example:
filename=/home/user1/filename.txt
hostname = server1.mydomain.com
So the content of $filename is server1.mydomain.com after running hostname > $filename.
The output of du -sh /var/log/test.log command is let say 1.3M /var/log/test.log
So running sed -i "/mydomain.com/ s/$/, $contents" $filename should update the content of the following filename to:
server1.mydomain.com, 1.3M /var/log/test.log
But as I mentioned above, it's throwing an error.
try this
sed -i "s#\$#, ${contents}#g"
Demo :
$cat file.txt
server1.mydomain.com
$echo $contents
1.3M /var/log/test.log
$sed -i "s#\$#, ${contents}#g" file.txt
$cat file.txt
server1.mydomain.com, 1.3M /var/log/test.log
$
sed command usage is s#pattern to search#pattern/String replacement#occurence
This is easier done with perl or another language, to avoid the issues with characters in your variable causing sed parse errors:
contents=$(du -sh /var/log/test.log) perl -pi -e '$_ .= ", $ENV{contents}" if /test_string/' "$filename"
Or using ed to edit the file, and avoiding the $contents variable completely:
ed -s "$filename" <<'EOF'
/test_string/a
,<space>
.
r !du -sh /var/log/test.log
.-2,.j
w
EOF
Replace <space> with a literal space.
This cryptic-at-first set of commands first moves to the first line matching the regular expression test_string, then appends a line ,<space> after it, then reads the output of the du command and inserts it in a line after that, and finally joins those three lines into one and writes the modified file back to disk.
(This assumes that the du invocation will only return one line.)
Related
I want to append bash variable (which has html tags) at the beginning of a file.
INSERTTO=<h2>title</h2>
<li>sdfdsf</li>
Below is the command I am using -
sed -i '1i'$INSERTTO file.html
But i am getting error -
sed: -e expression #1, char 177: unknown command: `<'
Do i need to encode the html tags in INSERTTO variable ?
First of all, use quotes:
INSERTTO='<h2>title</h2>
<li>sdfdsf</li>'
Then try this:
sed "1 i $INSERTTO" file.html
Start with some text in your file, e.g.
$ cat file
some text
Then you need your variable to contain an explicit '\n' character where the line break is, e.g. INSERTTO='<h2>title</h2>\n<li>sdfdsf</li>'. Then you can use the sed expression to place both lines as the beginning lines in the file, e.g.
$ INSERTTO='<h2>title</h2>\n<li>sdfdsf</li>'; sed "1i $INSERTTO" file
<h2>title</h2>
<li>sdfdsf</li>
some text
Now at present, what will be done has only been written to the terminal stdout. To modify the file in place, you will need to add the -i option for sed (or -i.bak to save a backup of the original file with the .bak extension. (however you prefer to do it)
I offer you an ed(1) solution.
INSERTTO=<h2>title</h2>
<li>sdfdsf</li>
printf '%s\n' 1i "$INSERTTO" . w | ed -s file.html
Assuming the file is not empty that will work. otherwise use 0a as the address and command instead of 1i
... Or use the cat(1) and mv(1)
INSERTTO=<h2>title</h2>
<li>sdfdsf</li>
Add the stdin flag and use a herestring works in bash but not in POSIX shells.
cat - file.html <<< "$INSERTTO"
You should see the output to stdout, redirect it to another file and move that file to the original file, something like this.
cat - file.html <<< "$INSERTTO" > tempfile && mv tempfile file.html
However if the html file is a symlink then it is now broken... a work around would be to use another cat.
cat - file.html <<< "$INSERTTO" > tempfile && cat tempfile > file.html && rm tempfile.
tempfile is just an example you should take a loot at How to create a tempfile in a secure manner
I have these files
NotRequired.txt (having lines which need to be remove)
Need2CleanSED.txt (big file , need to clean)
Need2CleanGRP.txt (big file , need to clean)
content:
more NotRequired.txt
[abc-xyz_pqr-pe2_123]
[lon-abc-tkt_1202]
[wat-7600-1_414]
[indo-pak_isu-5_761]
I am reading above file and want to remove lines from Need2Clean???.txt, trying via SED and GREP but no success.
myFile="NotRequired.txt"
while IFS= read -r HKline
do
sed -i '/$HKline/d' Need2CleanSED.txt
done < "$myFile"
myFile="NotRequired.txt"
while IFS= read -r HKline
do
grep -vE \"$HKline\" Need2CleanGRP.txt > Need2CleanGRP.txt
done < "$myFile"
Looks as if the Variable and characters [] making some problem.
What you're doing is extremely inefficient and error prone. Just do this:
grep -vF -f NotRequired.txt Need2CleanGRP.txt > tmp &&
mv tmp Need2CleanGRP.txt
Thanks to grep -F the above treats each line of NotRequired.txt as a string rather than a regexp so you don't have to worry about escaping RE metachars like [ and you don't need to wrap it in a shell loop - that one command will remove all undesirable lines in one execution of grep.
Never do command file > file btw as the shell might decide to execute the > file first and so empty file before command gets a chance to read it! Always do command file > tmp && mv tmp file instead.
Your assumption is correct. The [...] construct looks for any characters in that set, so you have to preface ("escape") them with \. The easiest way is to do that in your original file:
sed -i -e 's:\[:\\[:' -e 's:\]:\\]:' "${myFile}"
If you don't like that, you can probably put the sed command in where you're directing the file in:
done < replace.txt|sed -e 's:\[:\\[:' -e 's:\]:\\]:'
Finally, you can use sed on each HKline variable:
HKline=$( echo $HKline | sed -e 's:\[:\\[:' -e 's:\]:\\]:' )
try gnu sed:
sed -Ez 's/\n/\|/g;s!\[!\\[!g;s!\]!\\]!g; s!(.*).!/\1/d!' NotRequired.txt| sed -Ef - Need2CleanSED.txt
Two sed process are chained into one by shell pipe
NotRequired.txt is 'slurped' by sed -z all at once and substituted its \n and [ meta-char with | and \[ respectively of which the 2nd process uses it as regex script for the input file, ie. Need2CleanSED.txt. 1st process output;
/\[abc-xyz_pqr-pe2_123\]|\[lon-abc-tkt_1202\]|\[wat-7600-1_414\]|\[indo-pak_isu-5_761\]/d
add -u ie. unbuffered, option to evade from batch process, sort of direct i/o
I am trying to insert a line in the beginning of a file using sed.
I tried below commands :
sed -i '1s/^/LINE TO INSERT\n/' test.txt
sed: illegal option -- i --> Error thrown
sed '1i/^/LINE TO INSERT\n/' test.txt
sed: Function 1i/^/LINE TO INSERT\n/ cannot be parsed. --> Error thrown
Both the ways came out to be failed.
Any possible solution to it ? I am using ksh script on HP-UX.
Thanks.
How about good old ed?
printf '%s\n' 1i 'LINE TO INSERT' . w | ed -s file
printf is used to send each command to ed on a separate line.
Alternatively, if you're terrified of ed like me, you can just use a temporary file, as suggested in the comments:
echo 'LINE TO INSERT' > tmp && cat tmp test > new && mv new test && rm tmp
I think you have a typo: you're missing the closing apostrophe from your 1st command. Otherwise it's fine. I.e.:
You have this: sed -i '1s/^/... test.txt
But you need this: sed -i '1s/^/...' test.txt
Putting all together: sed -i '1s/^/LINE TO INSERT\n/' test.txt
Update: if -i is not supported, then you can use a temporary file:
sed '1s/^/LINE TO INSERT\n/' test.txt > /tmp/test.txt.tmp
mv /tmp/test.txt.tmp test.txt
If I want to add content to the 10th line of a file, how do I do it?
This is what I came up with:
sed -ie '' "10s/^/new_content/g" file.txt
I keep getting the message "no such file or directory"
Also, if I want to replace 10 with N+1 and the new_content with a variable $VAR, would the format still be the same?
VAR= $(cat fileA.txt)
sed -ie '' "`expr $N +1`s/^/$VAR/g" fileB.txt
Thanks for your help!!!
Shell is fussy about spacing:
VAR= $(cat fileA.txt)
You do not want the space after the =. Actually, you don't want the assignment either, unless fileA.txt has only one line. You'd have to escape newlines with backslashes, etc, to get it to work, which is both hard and unnecessary.
Unless you're using an antique (non-POSIX) shell, use built-in arithmetic:
sed -i.bak -e "$(($N + 1))r fileA.txt" fileB.txt
The $(($N + 1)) has the shell evaluate the arithmetic expression. The r has sed read the file that is named.
[1addr]r file
Copy the contents of file to the standard output immediately before the next attempt to read a line of input. If file cannot be read for any reason, it is silently ignored and no error condition is set.
It turns out I needed to split -i and -e.
The final result that works was:
sed -i '' -e "10s/^/CONTENT/g" file.txt
Or to increase Ns by +1 each time it loops:
sed -i '' -e "$((N + 1))s/^/CONTENT/g" ~/dance/heatlist/prep.txt
I still have not figured out how to make CONTENT a variable $CONTENT
I am trying to delete a line from a text file that has a matching ID number.
Student id variable: $sid, for example 12345678;
$FILE = student_record
I first tried:
sed -i '/$sid/d' student_record.txt
Which gave me file not found. Next:
sed -i '/$sid/d' $FILE
And I get: sed: 1: "student_record": unterminated substitute in regular expression
sed -i '/12345678/d' $FILE
Same error as above
sed -i '/$sid/ d' student_record.txt
yields:
sed 1: "student_record.txt": bad flag in substitute command: 'x'
If I try without -i,
sed '/$sid/ d' $FILE
It just prints the whole file and doesn't delete any lines.
Advice would be great.
If the file is called student_record as you say for $FILE, you may be making a mistake using student_record.txt which would explain while you get file not found.
For many of the others, if you use single quotes it will not expand variables, so you'll literally be looking for the string "$sid". If you use double quotes it will expand, so try
sed -i "/$sid/d" "$FILE"
assuming you have GNU sed. If you're on something that does not have GNU, you may not have -i or it may require an argument.