I am parsing through a log file and I am trying to clean up the output.
Here's a sample input line
2016-04-11 12:45:26 : TEXT TO REMOVE
Here's my current code which removes everything after the first colon.
sed 's/:.*//'
which outputs
2016-04-11 12
I'd like to modify this so that it removes everything after the third colon instead (so I end up with just the date and time).
Here's a sample output I would like:
2016-04-11 12:45:26
That's what cut was invented to do:
$ cut -d':' -f1-3 file
2016-04-11 12:45:26
How about looking for the spaces surrounding the colon?
sed 's/ : .*//'
awk -F ' : ' '{print $1}'
You can use this sed:
str='2016-04-11 12:45:26 : TEXT TO REMOVE'
sed 's/ *:[^:]*$//' <<< "$str"
i.e. use [^:]*$ pattern to make sure we match last segment of line after last :
Output:
2016-04-11 12:45:26
Strictly speaking removing everything after the 3rd : is equivalent to print only the chars that are before it. sed would be easier to use that way.
Give this a try:
sed "s/^\([^:][^:]*:[^:][^:]*:[^:][^:]*\):.*$/\1/"
The same principle can be used to print only date time before ::
sed "s/^\([0-9][0-9]*-[0-9][0-9]-[0-9][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9]\).*$/\1/g"
The chars in between the \( and the \) can be reused in the replacement section with \1.
Related
I have to write a function taking as argument a csv file named players.csv and a number giving the line to print.
Indded, i have to print the nth line 2column and 3rd column with a "is" between. For example Mike is John. column delimeter is ";".
I have the following code which is working :
sed -n "$2p" players.csv | cut -d ";" -f 2,3 --output-delimiter=' is '
However, I have to do the same without using cut. I can only use sed and wc. Do you have any idea what sed command I can use to have the same behavior as with cut.
Thank you for your attention and your help.
You were almost there:
sed -En "$2"'s/[^;]*;([^;]*);([^;]*).*/\1 is \2/p' players.csv
I want to delete three words with a special character on a line such as
Input:
\cf4 \cb6 1749,1789 \cb3 \
Output:
1749,1789
I have tried a couple sed and grep statements but so far none have worked, mainly due to the character \.
My unsuccessful attempt:
sed -i 's/ [.\c ] //g' inputfile.ext >output file.ext
Awk accepts a regex Field Separator (in this case, comma or space):
$ awk -F'[ ,]' '$0 = $3 "." $4' <<< '\cf4 \cb6 1749,1789 \cb3 \'
1749.1789
-F'[ ,]' - Use a single character from the set space/comma as Field Separator
$0 = $3 "." $4 - If we can set the entire line $0 to Field 3 $4 followed by a literal period "." followed by Field 4 $4, do the default behavior (print entire line)
Replace <<< 'input' with file if every line of that file has the same delimeters (spaces/comma) and number of fields. If your input file is more complex than the sample you shared, please edit your question to show actual input.
The backslash is a special meta-character that confuses bash.
We treat it like any other meta-character, by escaping it, with--you guessed it--a backslash!
But first, we need to grep this pattern out of our file
grep '\\... \\... [0-9]+,[0-9]+ \\... \\' our_file # Close enough!
Now, just sed out those pesky backslashes
| sed -e 's/\\//g' # Don't forget the g, otherwise it'll only strip out 1 backlash
Now, finally, sed out the clusters of 2 alpha followed by a number and a space!
| sed -e 's/[a-z][a-z][0-9] //g'
And, finally....
grep '\\... \\... [0-9]+,[0-9]+ \\... \\' our_file | sed -e 's/\\//g' | sed -e 's/[a-z][a-z][0-9] //g'
Output:
1749,1789
My guess is you are having trouble because you have backslashes in input and can't figure out how to get backslashes into your regex. Since backslashes are escape characters to shell and regex you end up having to type four backslashes to get one into your regex.
Ben Van Camp already posted an answer that uses single quotes to make the escaping a little easier; however I shall now post an answer that simply avoids the problem altogether.
grep -o '[0-9]*,[0-9]*' | tr , .
Locks on to the comma and selects the digits on either side and outputs the number. Alternately if comma is not guaranteed we can do it this way:
egrep -o ' [0-9,]*|^[0-9,]*' | tr , . | tr -d ' '
Both of these assume there's only one usable number per line.
$ awk '{sub(/,/,".",$3); print $3}' file
1749.1789
$ sed 's/\([^ ]* \)\{2\}\([^ ]*\).*/\2/; s/,/./' file
1749.1789
I need to delete "" in file
"CITFFUSKD-E0"
I have tried sed 's/\"//.
Result is:
CITFFUSKD-E0"
How I can delete both ?
Also I need to delete everything behind first word but input can be this one:
"CITFFUSKD-E0"
"CITFFUSKD_E0"
"CITFFUSKD E0"
Result I want it:
CITFFUSKD
You may use
sed 's/"//g' file | sed 's/[^[:alnum:]].*//' > newfile
Or, contract the two sed commands into one sed call as #Wiimm suggests:
sed 's/"//g;s/[^[:alnum:]].*//' file > newfile
If you want to replace inline, see sed edit file in place.
Explanation:
sed 's/"//g' file - removes all " chars from the file
sed 's/[^[:alnum:]].*//' > newfile - also removes all chars from a line starting from the first non-alphanumeric char and saves the result into a newfile.
Could you please try following.
awk 'match($0,/[a-zA-Z]+[^a-zA-Z]*/){val=substr($0,RSTART,RLENGTH);gsub(/[^a-zA-Z]+/,"",val);print val}' Input_file
delete everything behind first word
sed 's/^"\([[:alpha:]]*\)[^[:alpha:]]*.*/\1/'
Match the first ". Then match a sequence of alphabetic characters. Match until you find non-alphabetic character ^[:alpha:]. Then match the rest. Substitute it all for \1 - it is a backreference for the part inside \( ... \), ie. the first word.
I need delete two “ ” with sed command
Remove all possible ":
sed 's/"//g'
Extract the string between ":
sed 's/"\([^"]*\)"/\1/'
Remove everything except alphanumeric characters (numbers + a-z + a-Z, ie. [0-9a-zA-z]):
sed 's/[^[:alnum:]]//g'
This should do all in one go, remove the ", print the first part:
awk -F\" '{split($2,a,"-| |_");print a[1]}' file
CITFFUSKD
CITFFUSKD
CITFFUSKD
When you have 1 line, you can use
grep -Eo "(\w)*" file | head -1
For normal files (starting with a double quote on each line)
, try this
tr -c [^[:alnum:]] '"' < file | cut -d'"' -f2
Many legitimate ways to solve this.
I favor using what you know about your data to simplify solutions -- this is usually an option. If everything in your file follows the same pattern, you can simply extract the first set of capitalized letters encountered:
sed 's/"\([A-Z]\+\).*$/\1/' file
awk '{gsub(/^.|....$/,"")}NR==1' file
CITFFUSKD
I have a file with a format like this:
First Last UID
First Middle Last UID
Basically, some names have middle names (and sometimes more than one middle name). I just want a file that only as UIDs.
Is there a sed or awk command I can run that removes everything before the last space?
awk
Print the last field of each line using awk.
The last field is indexed using the NF variable which contains the number of fields for each line. We index it using a dollar sign, the resulting one-liner is easy.
awk '{ print $NF }' file
rs, cat & tail
Another way is to transpose the content of the file, then grab the last line and transpose again (this is fairly easy to see).
The resulting pipe is:
cat file | rs -T | tail -n1 | rs -T
cut & rev
Using cut and rev we could also achieve this goal by reversing the lines, cutting the first field and then reverse it again.
rev file | cut -d ' ' -f1 | rev
sed
Using sed we simply remove all chars until a space is found with the regex ^.* [^ ]*$. This regex means match the beginning of the line ^, followed by any sequence of chars .* and a space . The rest is a sequence of non spaces [^ ]* until the end of the line $. The sed one-liner is:
sed 's/^.* \([^ ]*\)$/\1/' file
Where we capture the last part (in between \( and \)) and sub it back in for the entire line. \1 means the first group caught, which is the last field.
Notes
As Ed Norton cleverly pointed out we could simply not catch the group and remove the former part of the regex. This can be as easily achieved as
sed 's/.* //' file
Which is remarkably less complicated and more elegant.
For more information see man sed and man awk.
Using grep:
$ grep -o '[^[:blank:]]*$' file
UID
UID
-o tells grep to print only the matching part. The regex [^[:blank:]]*$ matches the last word on the line.
I have a file as show below
1.2.3.4.ask
sanma.nam.sam
c.d.b.test
I want to remove the last field from each line, the delimiter is . and the number of fields are not constant.
Can anybody help me with an awk or sed to find out the solution. I can't use perl here.
Both these sed and awk solutions work independent of the number of fields.
Using sed:
$ sed -r 's/(.*)\..*/\1/' file
1.2.3.4
sanma.nam
c.d.b
Note: -r is the flag for extended regexp, it could be -E so check with man sed. If your version of sed doesn't have a flag for this then just escape the brackets:
sed 's/\(.*\)\..*/\1/' file
1.2.3.4
sanma.nam
c.d.b
The sed solution is doing a greedy match up to the last . and capturing everything before it, it replaces the whole line with only the matched part (n-1 fields). Use the -i option if you want the changes to be stored back to the files.
Using awk:
$ awk 'BEGIN{FS=OFS="."}{NF--; print}' file
1.2.3.4
sanma.nam
c.d.b
The awk solution just simply prints n-1 fields, to store the changes back to the file use redirection:
$ awk 'BEGIN{FS=OFS="."}{NF--; print}' file > tmp && mv tmp file
Reverse, cut, reverse back.
rev file | cut -d. -f2- | rev >newfile
Or, replace from last dot to end with nothing:
sed 's/\.[^.]*$//' file >newfile
The regex [^.] matches one character which is not dot (or newline). You need to exclude the dot because the repetition operator * is "greedy"; it will select the leftmost, longest possible match.
With cut on the reversed string
cat youFile | rev |cut -d "." -f 2- | rev
If you want to keep the "." use below:
awk '{gsub(/[^\.]*$/,"");print}' your_file