As part of a tiny automation script that I am trying to create, I have one section which filters out the response from an API call and outputs a number such as 12345 with the following:
tidy -xml -i - responseformatting/response.txt > response_tidied.txt | grep externalId response_tidied.txt | awk -F">" '{print $2}' | awk -F"<" '{print $1}' > response_tidied.txt
Is there a way I can pass the output obtained from this into sed, below? If I run this on its own, it works just fine. I just cannot seem to figure out how to use the value from the above tidying process into sed.
sed -i -e 's/\(<td>\).*\(<\/td>\)/<td>OUTPUT_FROM_ABOVE<\/td>/g' new.xml
Any help would be much appreciated :)
Please refer to sed - replace string with file contents, use
sed -i "s/<td>.*<\/td>/<td>$(sed 's:/:\\/:g' response_tidied.txt)<\/td>/g" new.xml
Bear in mind you need to use double quotes round the sed command. The point is that you need to escape all slashes in the replacement string.
Related
Problem
I have this comand:
sed $((SS - default_scripts))!d customScripts.txt
and it gives me Foo Bar.
I want to convert this to lowercase.
Attempt
When I tried using the | awk '{print tolower($0)}' command on it it returned nothing:
$($(sed $((SS - default_scripts))!d customScripts.txt) | awk '{print tolower($0)}')
Final
Please enlighten me on my typo, or recommend me another POSIX way of converting a whole string to lowercase in a compact manner. Thank you!
The pipe to awk should be inside the same command substitution as sed, so that it processes the output of sed.
$(sed $((SS - default_scripts))!d customScripts.txt | awk '{print tolower($0)}')
You don't need another command substitution around both of them.
Your typo was wrapping everything in $(...) and so first trying to execute the output of just the sed part and then trying to execute the output of the sed ... | awk ... pipeline.
You don't need sed commands nor shell arithmetic operations when you're using awk. If I understand what you're trying to do with this:
$(sed $((SS - default_scripts))!d customScripts.txt) | awk '{print tolower($0)}'
correctly then it'd be just this awk command:
awk -v s="$SS" -v d="$default_scripts" 'BEGIN{n=s-d} NR==n{print tolower($0); exit}' customScripts.txt
I have a file with more than 10000 lines. I am trying to search for a string in between particular set of lines, between 2 timestamps.
I am using sed command to achieve this.
sed -n '1,4133p' filename | sed -n '/'2015-08-12'/, /'2015-09-12'/p' filename | grep -i "string"
With the above command I am getting desired result. The above command is considering entire file not the lines I have specified.
Is there is a way to achieve this?.
Please help
I think the problem is here:
sed -n '1,4133p' filename | sed -n '/'2015-08-12'/, /'2015-09-12'/p' filename |
^^^
You want to pipe the output of your first sed command into the second. The way you have this, the output from the first is clobbered and replaced with a re-scan of the file.
Try this:
sed -n '1,4133p' filename | sed -n '/'2015-08-12'/, /'2015-09-12'/p' | grep -i "string"
Any time you find yourself chaining together pipes of seds and greps stop and just use 1 awk command instead:
awk -v IGNORECASE=1 '/2015-08-12/{f=1} f&&/string/; /2015-09-12/||(NR==4133){exit}' file
The above uses GNU awk for IGNORECASE, with other awks you'd just change /string/ to tolower($0)~/string/.
I have the following text file with the following lines:
<test="123">
<test="456">
<test="789">
My aim is to have the above text file to be appended with a keyword "HELLO" after the above numbers, as following:
<test="123.HELLO">
<test="456.HELLO">
<test="789.HELLO">
with the grep command and cut, I manage to get the value between the quotation mark.
grep -o "test=".* test.txt | cut -d \" -f2
I tried to use sed on top of it, with this line
grep -o "test=".* test.txt | cut -d \" -f2 | sed -i -- 's/$/.HELLO/' test.txt
however the closest I manage to get is instead a ".HELLO" which directly appended on the end of the line (and not after the numbers in between the quotes)
<test="123">.HELLO
<test="456">.HELLO
<test="789">.HELLO
How can I fix my sed statement to provide me with the requested line?
You can do it with groups in sed. To create new output, you can do this:
sed 's/\(test="[^"]*\)"/\1.HELLO"/g' test.txt
To modify it in-place, you can use the -i switch:
sed -i 's/\(test="[^"]*\)"/\1.HELLO"/g' test.txt
Explanation:
() is a group. You can refer to it with \1. In sed we have to escape the parentheses: \(\)
[^"]* matches everything that's not a quote. So the match will stop before the quote
In the replacement, you have to add the quote manually, since it's outside of the group. So you can put stuff before the quote.
Try this:
This is how your file looks like.
bash > cat a.txt
<test="123">
<test="456">
<test="789">
Your text piped to SED
bash > cat a.txt |sed 's/">/.HELLO">/g'
<test="123.HELLO">
<test="456.HELLO">
<test="789.HELLO">
bash >
Let me know if this worked out for you.
awk 'sub("[0-9]+","&.HELLO")' file
You can accomplish this with sed directly. Cut should not be necessary:
grep "test=" test.txt | sed 's/"\(.*\)"/"\1.HELLO"/'
The command I'm trying to use is
sed -n 's/'$LASTNAME'/pgIw '$TEMP_FILE2'' < "$TEMP_FILE"
My goal is to get search TEMP_FILE for the value in LASTNAME and write the line containing the match, if there is one, it to TEMP_FILE2. I keep getting that the sed command is garbled. The code above will return
sed: command garbled: s/smith/pgIw /tmp/tmp.aKaGFH
Any help is appreciated! I've been trying to figure this out for hours! This is suppose to be done in the Korn shell in UNIX and I can't use awk or python, it is the stipulations of the homework.
Thank you!
It looks like you have an abundance of quotes that you don't need. Try:
sed -n "/$LASTNAME/p" >$TEMP_FILE2 <$TEMP_FILE
Also, your use of the s sed command seems to be out of place, since you don't actually want to substitute anything.
If you just want to find something and pipe to output, then simply use grep
grep -i "$LASTNAME" "$TEMP_FILE" > "$TEMP_FILE2" # -i case-insensitive
Or Perl
perl -ne "print if /$LAST_NAME/" "$TEMP_FILE" > "$TEMP_FILE2"
[Editorial insertion: Possible duplicate of the same poster's earlier question?]
Hi, I need to extract from the file:
first
second
third
using the grep command, the following line:
second
third
How should the grep command look like?
Instead of grep, you can use pcregrep which supports multiline patterns
pcregrep -M 'second\nthird' file
-M allows the pattern to match more than one line.
Your question abstract "bash grep newline", implies that you would want to match on the second\nthird sequence of characters - i.e. something containing newline within it.
Since the grep works on "lines" and these two are different lines, you would not be able to match it this way.
So, I'd split it into several tasks:
you match the line that contains "second" and output the line that has matched and the subsequent line:
grep -A 1 "second" testfile
you translate every other newline into the sequence that is guaranteed not to occur in the input. I think the simplest way to do that would be using perl:
perl -npe '$x=1-$x; s/\n/##UnUsedSequence##/ if $x;'
you do a grep on these lines, this time searching for string ##UnUsedSequence##third:
grep "##UnUsedSequence##third"
you unwrap the unused sequences back into the newlines, sed might be the simplest:
sed -e 's/##UnUsedSequence##/\n'
So the resulting pipe command to do what you want would look like:
grep -A 1 "second" testfile | perl -npe '$x=1-$x; s/\n/##UnUsedSequence##/ if $x;' | grep "##UnUsedSequence##third" | sed -e 's/##UnUsedSequence##/\n/'
Not the most elegant by far, but should work. I'm curious to know of better approaches, though - there should be some.
I don't think grep is the way to go on this.
If you just want to strip the first line from any file (to generalize your question), I would use sed instead.
sed '1d' INPUT_FILE_NAME
This will send the contents of the file to standard output with the first line deleted.
Then you can redirect the standard output to another file to capture the results.
sed '1d' INPUT_FILE_NAME > OUTPUT_FILE_NAME
That should do it.
If you have to use grep and just don't want to display the line with first on it, then try this:
grep -v first INPUT_FILE_NAME
By passing the -v switch, you are telling grep to show you everything but the expression that you are passing. In effect show me everything but the line(s) with first in them.
However, the downside is that a file with multiple first's in it will not show those other lines either and may not be the behavior that you are expecting.
To shunt the results into a new file, try this:
grep -v first INPUT_FILE_NAME > OUTPUT_FILE_NAME
Hope this helps.
I don't really understand what do you want to match. I would not use grep, but one of the following:
tail -2 file # to get last two lines
head -n +2 file # to get all but first line
sed -e '2,3p;d' file # to get lines from second to third
(not sure how standard it is, it works in GNU tools for sure)
So you just don't want the line containing "first"? -v inverts the grep results.
$ echo -e "first\nsecond\nthird\n" | grep -v first
second
third
Line? Or lines?
Try
grep -E -e '(second|third)' filename
Edit: grep is line oriented. you're going to have to use either Perl, sed or awk to perform the pattern match across lines.
BTW -E tell grep that the regexp is extended RE.
grep -A1 "second" | grep -B1 "third" works nicely, and if you have multiple matches it will even get rid of the original -- match delimiter
grep -E '(second|third)' /path/to/file
egrep -w 'second|third' /path/to/file
you could use
$ grep -1 third filename
this will print a string with match and one string before and after. Since "third" is in the last string you get last two strings.
I like notnoop's answer, but building on AndrewY's answer (which is better for those without pcregrep, but way too complicated), you can just do:
RESULT=`grep -A1 -s -m1 '^\s*second\s*$' file | grep -s -B1 -m1 '^\s*third\s*$'`
grep -v '^first' filename
Where the -v flag inverts the match.