I have a text file that is basically one giant excel file on one line in a text file. An example would be like this:
Name,Age,Year,Michael,27,2018,Carl,19,2018
I need to change the third occurance of a comma into a new line so that I get
Name,Age,Year
Michael,27,2018
Carl,19,2018
Please let me know if that is too ambiguous and as always thank you in advance for all the help!
With Gnu sed:
sed -E 's/(([^,]*,){2}[^,]*),/\1\n/g'
To change the number of fields per line, change {2} to one less than the number of fields. For example, to change every fifth comma (as in the title of your question), you would use:
sed -E 's/(([^,]*,){4}[^,]*),/\1\n/g'
In the regular expression, [^,]*, is "zero or more characters other than , followed by a ,; in other words, it is a single comma-delimited field. This won't work if the fields are quoted strings with internal commas or newlines.
Regardless of what Linux's man sed says, the -E flag is an extension to Posix sed, which causes sed to use extended regular expressions (EREs) rather than basic regular expressions (see man 7 regex). -E also works on BSD sed, used by default on Mac OS X. (Thanks to #EdMorton for the note.)
With GNU awk for multi-char RS:
$ awk -v RS='[,\n]' '{ORS=(NR%3 ? "," : "\n")} 1' file
Name,Age,Year
Michael,27,2018
Carl,19,2018
With any awk:
$ awk -v RS=',' '{sub(/\n$/,""); ORS=(NR%3 ? "," : "\n")} 1' file
Name,Age,Year
Michael,27,2018
Carl,19,2018
Try this:
$ cat /tmp/22.txt
Name,Age,Year,Michael,27,2018,Carl,19,2018,Nooka,35,1945,Name1,11,19811
$ echo "Name,Age,Year"; grep -o "[a-zA-Z][a-zA-Z0-9]*,[1-9][0-9]*,[1-9][0-9]\{3\}" /tmp/22.txt
Michael,27,2018
Carl,19,2018
Nooka,35,1945
Name1,11,1981
Or, ,[1-9][0-9]\{3\} if you don't want to put [0-9] 3 more times for the YYYY part.
PS: This solution will give you only YYYY for the year (even if the data for YYYY is 19811 (typo mistakes if any), you'll still get 1981
You are looking for 3 fragments, each without a comma and separated by a comma.
The last fields can give problems (not ending with a comma and mayby only two fields.
The next command looks fine.
grep -Eo "([^,]*[,]{0,1}){0,3}" inputfile
This might work for you (GNU sed):
sed 's/,/\n/3;P;D' file
Replace every third , with a newline, print ,delete the first line and repeat.
For satisfying a legacy code i had to add date to a filename like shown below(its definitely needed and cannot modify legacy code :( ). But i need to remove the date within the same command without going to a new line. this command is read from a text file so i should do this within the single command.
$((echo "$file_name".`date +%Y%m%d`| sed 's/^prefix_//')
so here i am removing the prefix from filename and adding a date appended to filename. i also do want to remove the date which i added. for ex: prefix_filename.txt or prefix_filename.zip should give me as below.
Expected output:
filename.txt
filename.zip
Current output:
filename.txt.20161002
filename.zip.20161002
Assumming all the files are formatted as filename.ext.date, You can pipe the output to 'cut' command and get only the 1st and 2nd fields :
~> X=filename.txt.20161002
~> echo $X | cut -d"." -f1,2
filename.txt
I am not sure that I understand your question correctly, but perhaps this does what you want:
$((echo "$file_name".`date +%Y%m%d`| sed -e 's/^prefix_//' -e 's/\.[^.]*$//')
Sample input:
cat sample
prefix_original.txt.log.tgz.10032016
prefix_original.txt.log.10032016
prefix_original.txt.10032016
prefix_one.txt.10032016
prefix.txt.10032016
prefix.10032016
grep from start of the string till a literal dot "." followed by digit.
grep -oP '^.*(?=\.\d)' sample
prefix_original.txt.log.tgz
prefix_original.txt.log
prefix_original.txt
prefix_one.txt
prefix.txt
prefix
perhaps, following should be used:
grep -oP '^.*(?=\.\d)|^.*$' sample
If I understand your question correctly, you want to remove the date part from a variable, AND you already know from the context that the variable DOES contain a date part and that this part comes after the last period in the name.
In this case, the question boils down to removing the last period and what comes after.
This can be done (Posix shell, bash, zsh, ksh) by
filename_without=${filename_with%.*}
assuming that filename_with contains the filename which has the date part in the end.
% cat example
filename.txt.20161002
filename.zip.20161002
% cat example | sed "s/.[0-9]*$//g"
filename.txt
filename.zip
%
I have the following string in a file: "dd/mmm/YYYY:HH:MM:SS -0500"
and would like it to be replaced with an actual date i.e. "17/Mar/2016:18:14:40 -0500"
I tried using sed, but the following doesn't seem to work
dateTemplate="dd/mmm/YYYY:HH:MM:SS -0500"
actualDate="17/Mar/2016:18:14:40 -0500"
sed -i "s#'$dateTemplate'#'$actualDate'#g" tmp.txt
Any help would be appreciated.
Try removing the single quotes(') from the sed program:
dateTemplate="dd/mmm/YYYY:HH:MM:SS -0500"
actualDate="17/Mar/2016:18:14:40 -0500"
sed -i "s#$dateTemplate#$actualDate#g" tmp.txt
Tested with GNU sed version 4.2.1
Before:
1. pre dd/mmm/YYYY:HH:MM:SS -0500 post
2. pre dd/mmm/YYYY:HH:MM:SS -0500 post
After:
1. pre 17/Mar/2016:18:14:40 -0500 post
2. pre 17/Mar/2016:18:14:40 -0500 post
EDIT: I noticed you were using a different delimiter. I can confirm that Michael Jaros' is a more accurate answer.
You will need to escape the slashes:
dateTemplate="dd/mmm/YYYY:HH:MM:SS -0500"
actualDate="17\/Mar\/2016:18:14:40 -0500"
sed -i "s/$dateTemplate/$actualDate/g" tmp.txt
I tested that working. Let me know if it doesn't!
I would automate the process a bit:
dateTemplate="dd/mmm/YYYY:HH:MM:SS -0500"
actualDateTemplate="+$(
sed '
s/\<dd\>/%d/g
s/\<mmm\>/%b/g
s/\<YYYY\>/%Y/g
s/\<HH\>/%H/g
s/\<MM\>/%M/g
s/\<SS\>/%S/g
# add more translations here if desired
' <<<"$dateTemplate"
)"
actualDate=$(date "$actualDateTemplate")
echo "foo $dateTemplate bar" | sed "s#$dateTemplate#$actualDate#g"
foo 17/Mar/2016:20:22:34 -0500 bar
There's always the danger that the dateTemplate variable will contain the character used as the s/// delimiter, no matter what character you use. This process can be done with string operations in the shell, but that's tedious.
My goal is to rename a folder of files of the form 'img_MM-DD-YY_XX.jpg' to the form 'newyears_YYYY-MM-DD_XXX.jpg' by iterating through each filename and using sed to perform substitutions based on character positions. Unfortunately I cannot seem to get the position-based swaps to work.
e.g. s/.\{4\}[0-9][0-9]/.\{10\}[0-9][0-9]/ attempts to replace MM with YY
Here is my attempt (neglecting for now the _XX part):
for filename in images/*
do
newname=$(echo $filename | sed 's/.\{4\}[0-9][0-9]/.\{10\}[0-9][0-9]/;
s/.\{7\}[0-9][0-9]/.\{4\}[0-9][0-9]/;
s/.\{10\}[0-9][0-9]/.\{7\}[0-9][0-9]/;
s/img_/newyears_20/')
mv $filename $newname
done
Any ideas how I can fix this?
$ echo 'img_11-22-14_XX.jpg' | sed -r 's/[^_]*_([0-9]{2})-([0-9]{2})-([0-9]{2})/newyears_20\3-\1-\2/'
newyears_2014-11-22_XX.jpg
The above looks for anything up to and including the first underline followed by a 6-digit date. It replaces the initial part with newyears_ and reformats the date from mm-dd-yy to 20yy-mm-dd.
The two-digit mm, dd, or yy values are matched with ([0-9]{2}). The parentheses indicate that sed should capture the value for later use. The output side of the substitution is _20\3-\1-\2. This restores the underline and adds a 20 to the front of the year. The year was the third captured value so it is denoted \3. Likewise, the month was the first captured value so it is denoted \1 and the day the second so it is \2.
To eliminate some blackslashes, I used the -r option to invoke extended regular expressions. If you are on a Mac or other non-GNU system, use sed -E in place of sed -r. Otherwise, use:
sed 's/[^_]*_\([0-9]\{2\}\)-\([0-9]\{2\}\)-\([0-9]\{2\}\)/newyears_20\3-\1-\2/'
This is simple to do with awk
echo "img_MM-DD-YY_XX.jpg" | awk -F"[_-]" '{print "newyears_20"$4"-"$2"-"$3"_0"$5}'
newyears_20YY-MM-DD_0XX.jpg
Centos / Linux
Bash
I have a log file, which has lots of text in and epoch numbers all over the place. I want to replace all epochs whereever they are into readable date/time.
I've been wanting to this via sed, as that seems the tool for the job. I can't seem to get the replacement part of sed to actually parse the variable(epoch) to it for conversion.
Sample of what I'm working with...
echo "Some stuff 1346474454 And not working" \
| sed 's/1[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]/'"`bpdbm -ctime \&`"'/g'
Some stuff 0 = Thu Jan 1 01:00:00 1970 And not working
The bpdbm part will convert a supplied epoch variable into useful date. Like this..
bpdbm -ctime 1346474454
1346474454 = Sat Sep 1 05:40:54 2012
So how do i get the "found" item to be parsed into a command. As i don't seem to be able to get it to work.
Any help would be lovely. If there is another way, that would be cool...but i suspect sed will be quickest.
Thanks for your time!
that seems the tool for the job
No, it is not. sed can use & only itself, there is no way how to make it an argument to a command. You need something more powerful, e.g. Perl:
perl -pe 'if ( ($t) = /(1[0-9]+)/ ) { s/$t/localtime($t)/e }'
You can do it with GNU sed, the input:
infile
Some stuff 1346474454 And not working
GNU sed supports /e parameter which allows for piping command output into pattern space, one way to take advantage of this with bpdbm:
sed 's/(.*)(1[0-9]{9})(.*)/echo \1 $(bpdbm -ctime \2) \3/e' infile
Or with coreutils date:
sed 's/(.*)(1[0-9]{9})(.*)/echo \1 $(date -d #\2) \3/e' infile
output with date
Some stuff Sat Sep 1 06:40:54 CEST 2012 And not working
To get the same output as with bpdbm:
sed 's/(.*)(1[0-9]{9})(.*)/echo "\1$(date -d #\2 +\"%a %b %_d %T %Y\")\3"/e' infile
output
Some stuff Sat Sep 1 06:40:54 2012 And not working
Note, this only replaces the last epoch found on a line. Re-run if there are more.