How to ignore case when using awk or sed [duplicate] - macos

sed -i '/first/i This line to be added'
In this case,how to ignore case while searching for pattern =first

You can use the following:
sed 's/[Ff][Ii][Rr][Ss][Tt]/last/g' file
Otherwise, you have the /I and n/i flags:
sed 's/first/last/Ig' file
From man sed:
I
i
The I modifier to regular-expression matching is a GNU extension which
makes sed match regexp in a case-insensitive manner.
Test
$ cat file
first
FiRst
FIRST
fir3st
$ sed 's/[Ff][Ii][Rr][Ss][Tt]/last/g' file
last
last
last
fir3st
$ sed 's/first/last/Ig' file
last
last
last
fir3st

GNU sed
sed '/first/Ii This line to be added' file

You can try
sed 's/first/somethingelse/gI'

if you want to save some typing, try awk. I don't think sed has that option
awk -v IGNORECASE="1" '/first/{your logic}' file

For versions of awk that don't understand the IGNORECASE special variable, you can use something like this:
awk 'toupper($0) ~ /PATTERN/ { print "string to insert" } 1' file
Convert each line to uppercase before testing whether it matches the pattern and if it does, print the string. 1 is the shortest true condition, so awk does the default thing: { print }.
To use a variable, you could go with this:
awk -v var="$foo" 'BEGIN { pattern = toupper(foo) } toupper($0) ~ pattern { print "string to insert" } 1' file
This passes the shell variable $foo and transforms it to uppercase before the file is processed.
Slightly shorter with bash would be to use -v pattern="${foo^^}" and skip the BEGIN block.

Use the following, \b for word boundary
sed 's/\bfirst\b/This line to be added/Ig' file

Related

sed or awk, removing curly brackets but only if there are no commas inside brackets

i have this string:
ex00/{ft_strdup.c} ex04/{ft_convert_base.c,ft_convert_base2.c} ex05/{ft_split.c}
need to remove with sed the curly brackets if there is no comma inside brackets, so desired output:
ex00/ft_strdup.c ex04/{ft_convert_base.c,ft_convert_base2.c} ex05/ft_split.c
Using any sed:
$ sed 's/{\([^,}]*\)}/\1/g' file
ex00/ft_strdup.c ex04/{ft_convert_base.c,ft_convert_base2.c} ex05/ft_split.c
Note that the above will work no matter which characters except ,, {, }, or \n exist in your file names, e.g. these are all valid file names:
$ cat file
ex00/{ft_strdup1.c} ex05/{ft-split.c} ex05/{ft=s&pl#it.c}
$ sed 's/{\([^,}]*\)}/\1/g' file
ex00/ft_strdup1.c ex05/ft-split.c ex05/ft=s&pl#it.c
If your file names can contain any of the characters I mentioned above as excluded then ask a new question including those in your sample input/output.
With your shown samples please try following awk code. Written and tested in GNU awk.
awk -v RS='{[^}]*}' '
RT{
if(!sub(/,/,"&",RT)){ gsub(/^{|}$/,"",RT) }
}
{ ORS=RT }
1
END{ print "" }
' Input_file
Using sed
$ sed -E 's/\{([[:alpha:]_.]+)}/\1/g' input_file
touch ex00/ft_strdup.c ex04/{ft_convert_base.c,ft_convert_base2.c} ex05/ft_split.c

How can I prefix the output of each match in grep with some text?

I have a file with a list of phrases
apples
banananas
oranges
I'm running cat file.txt | xargs -I% sh -c "grep -Eio '(an)' >> output.txt"
What I can't figure out, is that I want the output to contain the original line, for example:
bananas,an
oranges,an
How can I prefix the output of grep to also include the value being piped to it?
This should be a task for awk, could you please try following.
awk '/an/{print $0",an"}' Input_file
This will look for string an in all lines of Input_file and append an in them too.
Solution with sed:
sed '/an/s/$/,an/' intput_file
This finds lines that match the pattern /an/, and appends ,an to the end of the pattern space $.
Use awk instead of grep:
$ awk -v s="an" ' # search string
BEGIN {
OFS="," # separating comma
}
match($0,s) { # when there is a match
print $0,substr($0,RSTART,RLENGTH) # output
}' file
Output:
banananas,an
oranges,an

Find the pattern (YYYY-MM-DD) and replace it with the same value concatenating with apostrophes

I have this kind of data:
1,1990-01-01,2,A,2015-02-09
1,NULL,2,A,2015-02-09
1,1990-01-01,2,A,NULL
And looking for solution which will replace each date in the file with the old value but adding apostrophes. Basically expected result from the example will be:
1,'1990-01-01',2,A,'2015-02-09'
1,NULL,2,A,'2015-02-09'
1,'1990-01-01',2,A,NULL
I have found the way how to find the pattern which match my date, but still can't get with what I can then replace it.
sed 's/[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]/????/' a.txt > b.txt
Catch the date in a group by surrounding the pattern with parentheses (). Then you can use this catched group with \1 (second group would be \2 etc.).
sed "s/\([0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]\)/'\1'/g"
Note the g at the end, which ensures that all matches are replaced (if there are more than one in one line).
If you add -r switch to sed, the awkward backslashes before () can be omitted:
sed -r "s/([0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9])/'\1'/g"
This can be further simplified using quantifiers:
sed -r "s/([0-9]{4}-[0-9]{2}-[0-9]{2})/'\1'/g"
Or even:
sed -r "s/([0-9]{4}-([0-9]{2}){2})/'\1'/g"
As mentioned in the comments: Also, in this particular case, you may use & instead of \1, which matches the whole looked-up expression, and omit the ():
sed -r "s/[0-9]{4}(-[0-9]{2}){2}/'&'/g"
You need to use a capture group, as well as replace all matching occurrences with the g flag.
sed 's/\([0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]\)/'"'"'\1'"'"'/g' a.txt > b.txt
The replacement text is a bit confusing because a single-quoted string in shell cannot contain a single quote, so you have to close the single-quoted string, then use a double-quoted single-quote. Using $'...'-style quoting in bash simplies it a bit, at the cost of needing to escape the backslashes.
sed $'s/\\([0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]\\)/\'\1\'/g' a.txt > b.txt
Or, you can simply double-quote the script, since there's nothing currently in it that is subject to expansion:
sed "s/\([0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]\)/'\1'/g" a.txt > b.txt
There is also the special & replacement text, which expands to whatever the regular expressions matches, so you can avoid an explicit capture group:
sed "s/[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]/'&'/g" a.txt > b.txt
With GNU sed:
sed -E 's/([0-9]{2,4}-?){3}/'\''&'\''/g' file
Depending on your file content, the dates may also be described as 1 or 2 followed by a combination of nine dashes or digits:
sed -E 's/[12][-0-9]{9}/'\''&'\''/g" file
Here is one in awk:
$ awk -v q="'" '
BEGIN { FS=OFS="," } # set selimiters
{
for(i=1;i<=NF;i++) # loop all fields
if($i~/[0-9]{4}-[0-9]{2}-[0-9]{2}/) # if field has a date looking string
$i=q $i q # quote it
}1' file
Output:
1,'1990-01-01',2,A,'2015-02-09'
1,NULL,2,A,'2015-02-09'
1,'1990-01-01',2,A,NULL
Could you please try following.(REGEX mentioned inside match could be written as [0-9]{4}-[0-9]{2}-[0-9]{2} too but since my awk is of old version so couldn't test it, you could try it once)
awk -v s1="'" '
{
while(match($0,/[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]/)){
val=val substr($0,1,RSTART-1) s1 substr($0,RSTART,RLENGTH) s1
$0=substr($0,RSTART+RLENGTH)
}
print val
val=""
}' Input_file
Output will be as follows.
1,'1990-01-01',2,A,'2015-02-09'
1,NULL,2,A,'2015-02-09'
1,'1990-01-01'
With Perl, it is simple
perl -pe ' s/(\d{4}-\d\d-\d\d)/\x27$1\x27/g '
with inputs - \x27 is used for single quotes
$ cat liubo.txt
1,1990-01-01,2,A,2015-02-09
1,NULL,2,A,2015-02-09
1,1990-01-01,2,A,NULL
$ perl -pe ' s/(\d{4}-\d\d-\d\d)/\x27$1\x27/g ' liubo.txt
1,'1990-01-01',2,A,'2015-02-09'
1,NULL,2,A,'2015-02-09'
1,'1990-01-01',2,A,NULL
$
If you want to use single quotes, then escape $ and wrap the command in double quotes
$ perl -pe " s/(\d{4}-\d\d-\d\d)/\'\$1\'/g " liubo.txt
1,'1990-01-01',2,A,'2015-02-09'
1,NULL,2,A,'2015-02-09'
1,'1990-01-01',2,A,NULL
$

How to output only text after a match with sed

I am using sed to find a certain match in a text file and then put this value in to a variable, my problem is that I only want the text after the match, and not the entire line.
Ans=$(sed -n '/^'$1':/,/~/{/:/{p;n};/~/q;p}' $file.txt)
Text File
q1:answer1
~
q2:answer2
~
q3:answer3
~
Actual Output
q1:answer1
Expected Output
answer1
With grep :
Ans=$(grep -oP "^$1:\K.*" file)
or with perl if your grep version doesn't support -P switch :
Ans=$(var=$1 perl -lne '/^$ENV{var}:\K.*/ and print $&' file)
In case a sed solution is needed - e.g., if answers could span multiple lines:
Ans=$(sed -r -n '/^'$1':(.*)/,/^(~)$/ { s//\1/; /^~$/q; p; }' file.txt)
(OSX users: use -E instead of -r).
Uses a backreference (\1) to replace the first matching line with its portion of interest; any other lines between the first matching one and the terminating ~ line are unaffected by the replacement (assuming they don't also start with $1:) and also printed.
Replace q with d if you don't want to quit after the first matching range.
By contrast, if the string of interest is limited to the line starting with $1:, there's no need to also match the ~ line, and the command can be simplified to:
Ans=$(sed -r -n '/^'$1':(.*)/ { s//\1/p; q; }' file.txt)
Remove q; if you don't want to quit after the first match.
However, the single-line case is more easily handled with a grep or awk solution - see #sputnick's and #anubhava's answers. If you wanted those to quit after the first match -- as in the snippets above and the code in the OP -- you'd need to add option -m 1 to the grep solution and ; exit to the awk solution (before the }).
Better use awk for this:
ans=$(awk -F':' -v s='q1' '$1 == s {print $2}' file)

How to add to the end of lines containing a pattern with sed or awk?

Here is example file:
somestuff...
all: thing otherthing
some other stuff
What I want to do is to add to the line that starts with all: like this:
somestuff...
all: thing otherthing anotherthing
some other stuff
This works for me
sed '/^all:/ s/$/ anotherthing/' file
The first part is a pattern to find and the second part is an ordinary sed's substitution using $ for the end of a line.
If you want to change the file during the process, use -i option
sed -i '/^all:/ s/$/ anotherthing/' file
Or you can redirect it to another file
sed '/^all:/ s/$/ anotherthing/' file > output
You can append the text to $0 in awk if it matches the condition:
awk '/^all:/ {$0=$0" anotherthing"} 1' file
Explanation
/patt/ {...} if the line matches the pattern given by patt, then perform the actions described within {}.
In this case: /^all:/ {$0=$0" anotherthing"} if the line starts (represented by ^) with all:, then append anotherthing to the line.
1 as a true condition, triggers the default action of awk: print the current line (print $0). This will happen always, so it will either print the original line or the modified one.
Test
For your given input it returns:
somestuff...
all: thing otherthing anotherthing
some other stuff
Note you could also provide the text to append in a variable:
$ awk -v mytext=" EXTRA TEXT" '/^all:/ {$0=$0mytext} 1' file
somestuff...
all: thing otherthing EXTRA TEXT
some other stuff
This should work for you
sed -e 's_^all: .*_& anotherthing_'
Using s command (substitute) you can search for a line which satisfies a regular expression. In the command above, & stands for the matched string.
Here is another simple solution using sed.
$ sed -i 's/all.*/& anotherthing/g' filename.txt
Explanation:
all.* means all lines started with 'all'.
& represent the match (ie the complete line that starts with 'all')
then sed replace the former with the later and appends the ' anotherthing' word
In bash:
while read -r line ; do
[[ $line == all:* ]] && line+=" anotherthing"
echo "$line"
done < filename
Solution with awk:
awk '{if ($1 ~ /^all/) print $0, "anotherthing"; else print $0}' file
Simply: if the row starts with all print the row plus "anotherthing", else print just the row.

Resources