i want to insert the first word from file.txt when it matches string in my documents, i have like 500 documents like this so it would be nice if it works
file.txt looks like this:
test1 t1
test2 t2
test3 t3
this is my code
code="t1"
sed -i -e 's/^/Name="$code" /'
this will result in
code="t1"
sed -i -e 's/^/Name="t1" /'
this what i want for final output in all my documents:
document1.txt
code="t1"
sed -i -e 's/^/Name="test1" /'
document2.txt
code="t2"
sed -i -e 's/^/Name="test2" /'
I don't clearly understand your requirement. If you cant to extract the first word depending on the second word and then do some string manipulation, following might help.
file.txt content:
test1 t1
test2 t2
test3 t3
Extract first word, given the second word:
second="t1" # change this for a different match
first=$(sed -n "s/\(.*\) $second/\1/p" file.txt) # now $first is "test1"
echo "first word is ${first} and blah..." # write it to a file or do anything you need here
Related
I figured out how to colorize column 3 in green like this:
green=$'\033[1;32m';off=$'\e[m';echo -e "num co1umn1 column2 column3\n=== === === ===\n1 this is me\n2 that is you"|column -t|sed "s/[^[:blank:]]\{1,\}/$green&$off/3";unset green off
CLI result
How do I need to alter my sed command to colorize the pattern 'is' only within column 3 so that the output becomes:
Wanted result
If you want to color the whole word is, you can use (with GNU sed):
sed "s/\bis\b/$green&$off/"
sed "s/\<is\>/$green&$off/"
Here, \b is a word boundary, \< is a leading word boundary and \> is a trailing word boundary.
Else, you can tell sed to start looking for matches from the third line:
sed "3,$ s/[^[:blank:]]\{1,\}/$green&$off/3"
Output:
One way to do this is to ignore the first two lines of the output in sed:
sed "1,2 ! s/[^[:blank:]]\{1,\}/$green&$off/3";
Using sed
$ ... | sed "/^[[:digit:]]/s/\(\([^ ]* \+\)\{2\}\)\([^ ]*\)/\1$green\3$off/"
Modifying the echo to cover a couple other instances of is ...
$ echo -e "num co1umn1 column2 column3\n=== === === ===\n1 is is me\n2 that isn't you" | column -t
num co1umn1 column2 column3
=== === === ===
1 is is me # only colorize the 2nd occurrence of "is"
2 that isn't you # don't colorize "isn't" in 3rd column
Extending OP's current sed solution:
sed -r "3,$ s/[^[:blank:]]{1,}/XXX&XXX/3; s/XXXisXXX/${green}is${off}/; s/XXX//g"
Where:
3,$ - apply following sed scripts against line numbers 3-to-EOF (ie, skip 1st 2 lines)
first we offset the 3rd column values with XXX bookends (choose a set of characters that you know won't show up anywhere in the data)
then colorize XXXisXXX (removing the XXXs at the same time)
then remove any remaining XXX (from 3rd column in other rows)
This generates:
I have many files named a, b, c and so on. These files contain line like this:-
11.077-105.882
-22.134-302.321
-1.011-201.254
I want to add a space when - sign come in mid of line. I want my output file look like this:-
11.077 -105.882
-22.134 -302.321
-1.011 -201.254
I have tried this command:-
cat a |sed 's/-/ -/g' >out.txt
But it do not give desired result
Require (and capture) a character before each - to replace:
$ sed 's/\(.\)-/\1 -/g' < tmp.txt
11.077 -105.882
-22.134 -302.321
-1.011 -201.254
This will only match a - that is not line-initial, and will include the preceding character in the replacement text.
You could combine 2 sed commands:
$ sed 's/-/ -/g' a | sed 's/^ //'
11.077 -105.882
-22.134 -302.321
-1.011 -201.254
Or, in a single line solution add whitespaces only before - that come after a digit:
$ sed 's,\([0-9]\)-,\1 -,' a
11.077 -105.882
-22.134 -302.321
-1.011 -201.254
I am trying to get a substring between &DEST= and the next & or a line break.
For example :
MYREQUESTISTO8764GETTHIS&DEST=SFO&ORIG=6546
In this I need to extract "SFO"
MYREQUESTISTO8764GETTHIS&DEST=SANFRANSISCO&ORIG=6546
In this I need to extract "SANFRANSISCO"
MYREQUESTISTO8764GETTHISWITH&DEST=SANJOSE
In this I need to extract "SANJOSE"
I am reading a file line by line, and I need to update the text after &DEST= and put it back in the file. The modification of the text is to mask the dest value with X character.
So, SFO should be replaced with XXX.
SANJOSE should be replaced with XXXXXXX.
Output :
MYREQUESTISTO8764GETTHIS&DEST=XXX&ORIG=6546
MYREQUESTISTO8764GETTHIS&DEST=XXXXXXXXXXXX&ORIG=6546
MYREQUESTISTO8764GETTHISWITH&DEST=XXXXXXX
Please let me know how to achieve this in script (Preferably shell or bash script).
Thanks.
$ cat file
MYREQUESTISTO8764GETTHIS&DEST=SFO&ORIG=6546
MYREQUESTISTO8764GETTHIS&DEST=PORTORICA
MYREQUESTISTO8764GETTHIS&DEST=SANFRANSISCO&ORIG=6546
MYREQUESTISTO8764GETTHISWITH&DEST=SANJOSE
$ sed -E 's/^.*&DEST=([^&]*)[&]*.*$/\1/' file
SFO
PORTORICA
SANFRANSISCO
SANJOSE
should do it
Replacing airports with an equal number of Xs
Let's consider this test file:
$ cat file
MYREQUESTISTO8764GETTHIS&DEST=SFO&ORIG=6546
MYREQUESTISTO8764GETTHIS&DEST=SANFRANSISCO&ORIG=6546
MYREQUESTISTO8764GETTHISWITH&DEST=SANJOSE
To replace the strings after &DEST= with an equal length of X and using GNU sed:
$ sed -E ':a; s/(&DEST=X*)[^X&]/\1X/; ta' file
MYREQUESTISTO8764GETTHIS&DEST=XXX&ORIG=6546
MYREQUESTISTO8764GETTHIS&DEST=XXXXXXXXXXXX&ORIG=6546
MYREQUESTISTO8764GETTHISWITH&DEST=XXXXXXX
To replace the file in-place:
sed -i -E ':a; s/(&DEST=X*)[^X&]/\1X/; ta' file
The above was tested with GNU sed. For BSD (OSX) sed, try:
sed -Ee :a -e 's/(&DEST=X*)[^X&]/\1X/' -e ta file
Or, to change in-place with BSD(OSX) sed, try:
sed -i '' -Ee :a -e 's/(&DEST=X*)[^X&]/\1X/' -e ta file
If there is some reason why it is important to use the shell to read the file line-by-line:
while IFS= read -r line
do
echo "$line" | sed -Ee :a -e 's/(&DEST=X*)[^X&]/\1X/' -e ta
done <file
How it works
Let's consider this code:
search_str="&DEST="
newfile=chart.txt
sed -E ':a; s/('"$search_str"'X*)[^X&]/\1X/; ta' "$newfile"
-E
This tells sed to use Extended Regular Expressions (ERE). This has the advantage of requiring fewer backslashes to escape things.
:a
This creates a label a.
s/('"$search_str"'X*)[^X&]/\1X/
This looks for $search_str followed by any number of X followed by any character that is not X or &. Because of the parens, everything except that last character is saved into group 1. This string is replaced by group 1, denoted \1 and an X.
ta
In sed, t is a test command. If the substitution was made (meaning that some character needed to be replaced by X), then the test evaluates to true and, in that case, ta tells sed to jump to label a.
This test-and-jump causes the substitution to be repeated as many times as necessary.
Replacing multiple tags with one sed command
$ name='DEST|ORIG'; sed -E ':a; s/(&('"$name"')=X*)[^X&]/\1X/; ta' file
MYREQUESTISTO8764GETTHIS&DEST=XXX&ORIG=XXXX
MYREQUESTISTO8764GETTHIS&DEST=XXXXXXXXXXXX&ORIG=XXXX
MYREQUESTISTO8764GETTHISWITH&DEST=XXXXXXX
Answer for original question
Using shell
$ s='MYREQUESTISTO8764GETTHIS&DEST=SFO&ORIG=6546'
$ s=${s#*&DEST=}
$ echo ${s%%&*}
SFO
How it works:
${s#*&DEST=} is prefix removal. This removes all text up to and including the first occurrence of &DEST=.
${s%%&*} is suffix removal_. It removes all text from the first & to the end of the string.
Using awk
$ echo 'MYREQUESTISTO8764GETTHIS&DEST=SFO&ORIG=6546' | awk -F'[=\n]' '$1=="DEST"{print $2}' RS='&'
SFO
How it works:
-F'[=\n]'
This tells awk to treat either an equal sign or a newline as the field separator
$1=="DEST"{print $2}
If the first field is DEST, then print the second field.
RS='&'
This sets the record separator to &.
With GNU bash:
while IFS= read -r line; do
[[ $line =~ (.*&DEST=)(.*)((&.*|$)) ]] && echo "${BASH_REMATCH[1]}fooooo${BASH_REMATCH[3]}"
done < file
Output:
MYREQUESTISTO8764GETTHIS&DEST=fooooo&ORIG=6546
MYREQUESTISTO8764GETTHIS&DEST=fooooo&ORIG=6546
MYREQUESTISTO8764GETTHISWITH&DEST=fooooo
Replace the characters between &DEST and & (or EOL) with x's:
awk -F'&DEST=' '{
printf("%s&DEST=", $1);
xlen=index($2,"&");
if ( xlen == 0) xlen=length($2)+1;
for (i=0;i<xlen;i++) printf("%s", "X");
endstr=substr($2,xlen);
printf("%s\n", endstr);
}' file
I am trying to insert a few lines of text before a specific line, but keep getting sed errors when I try to add a new line character. My command looks like:
sed -r -i '/Line to insert after/ i Line one to insert \\
second new line to insert \\
third new line to insert' /etc/directory/somefile.txt
The error that is reported is:
sed: -e expression #1, char 77: unterminated `s' command
I've tried, using \n, \\ (as in the example), no character at all, just moving the second line to the next line. I've also tried something like:
sed -r -i -e '/Line to insert after/ i Line one to insert'
-e 'second new line to insert'
-e 'third new line to insert' /etc/directory/somefile.txt
EDIT!: Apologies, I wanted the text inserted BEFORE the existing, not after!
This should work:
sed -i '/Line to insert after/ i Line one to insert \
second new line to insert \
third new line to insert' file
For anything other than simple substitutions on individual lines, use awk instead of sed for simplicity, clarity, robustness, etc., etc.
To insert before a line:
awk '
/Line to insert before/ {
print "Line one to insert"
print "second new line to insert"
print "third new line to insert"
}
{ print }
' /etc/directory/somefile.txt
To insert after a line:
awk '
{ print }
/Line to insert after/ {
print "Line one to insert"
print "second new line to insert"
print "third new line to insert"
}
' /etc/directory/somefile.txt
On MacOs I needed a few more things.
Double backslash after the i
Empty quotes after the -i to specify no backup file
Leading backslashes to add leading whitespace
Trailing double backslashes to add newlines
This code searches for the first instance of </plugins in pom.xml and inserts another XML object immediately preceding it, separated by a newline character.
sed -i '' "/\<\/plugins/ i \\
\ <plugin>\\
\ <groupId>org.apache.maven.plugins</groupId>\\
\ <artifactId>maven-source-plugin</artifactId>\\
\ <executions>\\
\ <execution>\\
\ <id>attach-sources</id>\\
\ <goals>\\
\ <goal>jar</goal>\\
\ </goals>\\
\ </execution>\\
\ </executions>\\
\ </plugin>\\
" pom.xml
This ll works from the first line.. For eg: If you want to insert from 3rd line of a file, replace "1i" to "3i".
sed -i '1i line1'\\n'line2'\\n'line3' 1.txt
cat 1.txt
line1
line2
line3
Hai
When the lines to be inserted are the result of some command "mycmd" (like cat results.txt or printf "%s\n" line{1..3}), you can do
sed -i 's/Line to insert after/r' <(cmd) file
or
sed -i 's/Line to insert after/echo "&";cmd/e' file
The last command can be simple modified when you want to insert before some match.
sed -i '/Line to insert after/ i\
Line one to insert\
second new line to insert\
third new line to insert' /etc/directory/somefile.txt
This might work for you (GNU sed & Bash):
sed -i $'/Line to insert after/a\line1\\nline2\\nline3' file
To be POSIX compliant and run in OS X, I used the following (single quoted line and empty line are for demonstration purposes):
sed -i "" "/[pattern]/i\\
line 1\\
line 2\\
\'line 3 with single quotes\`
\\
" <filename>
This can be easily done with Perl also
$ cat MeanwhileInHell.txt
Iran|XXXXXX|Iranian
Iraq|YYYYYY|Iraquian
Saudi|ZZZZZ|Saudi is a Rich Country
USA|AAAAAA|USA is United States of America.
India|IIII|India got freedom from British.
Scot|SSSSS|Canada Mexio.
$ perl -pe 'BEGIN {$x="Line one to insert\nLine 2\nLine3\n"} $_=$x.$_ if /USA/ ' MeanwhileInHell.txt
Iran|XXXXXX|Iranian
Iraq|YYYYYY|Iraquian
Saudi|ZZZZZ|Saudi is a Rich Country
Line one to insert
Line 2
Line3
USA|AAAAAA|USA is United States of America.
India|IIII|India got freedom from British.
Scot|SSSSS|Canada Mexio.
$
So I grep for something in some file:
grep "import" test.txt | tail -1
In test.txt there is
import-one
import-two
import-three
some other stuff in the file
This will return the last search result:
import-three
Now how do I add some text -after-- import-three but before "some other stuff in the file". Basically I want to append a line but not at the end of a file but after a search result.
I understand that you want some text after each search result, which would mean after every matching line. So try
grep "import" test.txt | sed '/$/ a\Line to be added'
You can try something like this with sed
sed '/import-three/ a\
> Line to be added' t
Test:
$ sed '/import-three/ a\
> Line to be added' t
import-one
import-two
import-three
Line to be added
some other stuff in the file
One way assuming that you cannot distingish between different "import" sentences. It reverses the file with tac, then find the first match (import-three) with sed, insert a line just before it (i\) and reverse again the file.
The :a ; n ; ba is a loop to avoid processing again the /import/ match.
The command is written throught several lines because the sed insert command is very special with the syntax:
$ tac infile | sed '/import/ { i\
"some text"
:a
n
ba }
' | tac -
It yields:
import-one
import-two
import-three
"some text"
some other stuff in the file
Using ed:
ed test.txt <<END
$
?^import
a
inserted text
.
w
q
END
Meaning: go to the end of the file, search backwards for the first line beginning with import, add the new lines below (insertion ends with a "." line), save and quit