I have data like this:
Jul29 16:52
Jul30 19:06
Jul31 17:04
Aug1 17:22
Aug2 18:53
Aug3 21:44
Aug4 22:56
Aug6 17:01
Aug8 02:19
Aug8 16:49
Aug9 16:37
Aug10 21:09
Aug12 05:24
Aug12 17:09
Aug14 16:39
Aug16 16:41
Aug4 22:56
Aug6 17:01
Aug8 02:19
Aug8 16:49
Aug9 16:37
Aug10 21:09
Aug12 05:24
Aug12 17:09
Aug14 16:39
Aug16 16:41
Aug4 22:56
Aug6 17:01
Aug8 02:19
Aug8 16:49
Aug9 16:37
Aug10 21:09
Aug16 20:24
Aug16 19:09
Aug16 18:39
Aug16 16:41
I want to take out the duplicates, sort by the first column, then maintain that order and sort by the second column. Like the following:
Jul01 11:00
Aug01 12:00
Aug02 12:40
Aug03 10:00
Aug03 11:00
Aug03 13:00
I have this code:
cat filename | awk '!a[$0]++'
This only sorts the first column and something random happens to the second column. Any ideas?
When I tried cat ming | sort -k1M -k1d -k2V, I get this:
Jul29 16:52
Jul30 19:06
Jul31 17:04
Aug10 21:09
Aug10 21:09
Aug10 21:09
Aug1 17:22
Aug12 05:24
Aug12 05:24
Aug12 17:09
Aug12 17:09
Aug14 16:39
Aug14 16:39
Aug16 16:41
Aug16 16:41
Aug16 16:41
Aug16 18:39
Aug16 19:09
Aug16 20:24
Aug2 18:53
Aug3 21:44
Aug4 22:56
Aug4 22:56
Aug4 22:56
Aug6 17:01
Aug6 17:01
Aug6 17:01
Aug8 02:19
Aug8 02:19
Aug8 02:19
Aug8 16:49
Aug8 16:49
Aug8 16:49
Aug9 16:37
Aug9 16:37
Aug9 16:37
sort -u -k1.1,1.3M -k1.4n -k2V filename
-u
delete duplicate lines
-k1.1,1.3M
sort each line from word 1, character 1 to word 1, character 3 in month mode
-k1.4n
sort each line from word 1, character 4 until end of word 1 by numeric value
-k2V
sort second word in "version number" mode, which works well for the timestamp
you can use the following:
sort -k1M -k1.4n -k2V abcss | uniq
explanation:
k1M : does a month sort on the 1st column
k1.4n : does an numeric sort to get the columns in order
k2V : does a version sort on the second column to get timestamp right
The output will be:
Jul29 16:52
Jul30 19:06
Jul31 17:04
Aug1 17:22
Aug2 18:53
Aug3 21:44
Aug4 22:56
Aug6 17:01
Aug8 02:19
Aug8 16:49
Aug9 16:37
Aug10 21:09
Aug12 05:24
Aug12 17:09
Aug14 16:39
Aug16 16:41
Aug16 18:39
Aug16 19:09
Aug16 20:24
Related
I have a folder such like:
drwxrwsr-x+ 1 dz33 dcistat 212 Sep 22 13:34 ./
drwxrwsr-x+ 1 dz33 dcistat 46 Sep 7 13:51 ../
-rw-rw---- 1 qg25 dcistat 542 Sep 15 13:55 createsamplelist.R
-rwxrwxr-x 1 dz33 dcistat 3717 Sep 7 14:15 Freedman-HuEx1.0v2-Analysis.Rnw*
drwxrws---+ 1 dz33 dcistat 0 Sep 22 13:34 Gao/
-rw-rw---- 1 qg25 dcistat 530 Sep 14 17:04 .log
-rwxrwxr-x 1 dz33 dcistat 154 Sep 7 13:44 Makefile*
-rwxrwx--x 1 qg25 dcistat 1191 Sep 15 09:04 pacaroma.R*
-rw-rw---- 1 qg25 dcistat 1741 Sep 14 17:23 pacaroma.Rout
-rw-rw---- 1 qg25 dcistat 4426 Sep 15 16:54 pacmeap.R
-rw-rw---- 1 qg25 dcistat 3230 Sep 14 17:15 .RData
-rw-rw---- 1 qg25 dcistat 0 Sep 14 17:04 .txt
My question is how to move all files belong to user qg25 to the directory Gao/.
find -maxdepth 1 -user qg25 -exec mv {} Gao/ \;
find /PATH_NAME -group qg25 -exec mv -t /NEW_PATH_NAME {} +
This should do the trick but I would test it on some dummy data.
Issue : Folders are treated as Files
My Code:
#!/bin/bash
for var in $(ls)
do
echo $var
if [ -e $var ]
then
echo "This is a file"
else
echo "This is not a file"
fi
done
echo All Done
Current Contents of the Root Folder:
-rw-r--r-- 1 stibo stibo 401 Sep 17 2015 id_rsa.pub
drwxrwxr-x 24 stibo stibo 4096 Jul 26 09:25 step
-rwxr-xr-x 1 stibo stibo 51 Jul 27 12:51 test.txt
drwxrwxrwx 2 stibo stibo 4096 Aug 2 10:32 deletionFile
-rwxrw-r-- 1 stibo stibo 225 Aug 2 12:32 deletionScript.vi
-rw-rw-r-- 1 stibo stibo 235 Aug 2 12:33 logdetails.txt
-rw-rw-r-- 1 stibo stibo 123 Aug 2 12:42 path1.txt
-rw-rw-r-- 1 stibo stibo 285 Aug 2 16:18 path2.txt
drwxrwxr-x 2 stibo stibo 4096 Aug 3 10:14 archival
-rw-rw-r-- 1 stibo stibo 0 Aug 3 13:42 ls
-rw-rw-r-- 1 stibo stibo 164732 Aug 3 14:11 messages
-rw-rw-r-- 1 stibo stibo 164732 Aug 3 14:11 wtmp
drwxrwxrwx 2 stibo stibo 4096 Aug 3 14:21 backup
-rwxrwxr-x 1 stibo stibo 160 Aug 4 15:34 newScript.vi
-rw-rw-r-- 1 stibo stibo 160 Aug 4 15:41 Code.txt
-rw-rw-r-- 1 stibo stibo 0 Aug 4 15:43 Details.txt
Where there are 4 folders and 12 files.
But when I run the script, I could see every thing is considered as a file, even if there are folders in it.
Can you please let me know where I am going wrong?
You've tested if a given entry exists with -e. Use -d for testing for directories and -f for files
When I run check script, I get the o/p as below,
-rw-rw-r-- 1 noper sbcprd 9175 Aug 6 03:36 opLogDir
-rw-rw-r-- 1 soper sbcprd 9104 Aug 6 03:04 opLogDir
-rw-rw-r-- 1 moper sbcprd 9561 Aug 6 02:18 opLogDir
-rw-rw-r-- 1 woper sbcprd 9561 Aug 6 05:06 opLogDir
-rw-rw-r-- 1 boper sbcprd 9834 Aug 6 03:34 opLogDir
-rw-rw-r-- 1 xoper sbcprd 9873 Aug 6 00:50 opLogDir
-rw-rw-r-- 1 doper sbcprd 9479 Aug 6 04:12 opLogDir
Now I can select data from it and sort using:
check | awk '{print $3,$8,$6,$7}'| sort
and get the o/p as below,
boper 03:34 Aug 6
doper 04:12 Aug 6
moper 02:18 Aug 6
noper 03:36 Aug 6
soper 03:04 Aug 6
woper 05:06 Aug 6
xoper 00:50 Aug 6
It sorts the o/p by column #1.
How can I sort the o/p according to the timings (column #2)?
Use the sort command. sort -k 2
Although, given this looks like ls output, you could probably change check to have ls -lt to sort by timestamp instead.
If you need something more extensive (e.g. that includes date and time based sorting) then it's harder - you'll need to use something that can parse the timestamp into a unix time.
E.g.:
#!/usr/bin/env perl
use strict;
use warnings;
use Time::Piece;
my $year = Time::Piece->localtime -> year;
for ( <DATA> ) {
my ( $mon, $day, $time ) = (split)[5,6,7];
my $timestamp = Time::Piece->strptime("$mon $day $time $year", '%b %d %H:%M %Y');
print $timestamp -> epoch,"\n";
}
__DATA__
-rw-rw-r-- 1 noper sbcprd 9175 Aug 6 03:36 opLogDir
-rw-rw-r-- 1 soper sbcprd 9104 Aug 6 03:04 opLogDir
-rw-rw-r-- 1 moper sbcprd 9561 Aug 6 02:18 opLogDir
-rw-rw-r-- 1 woper sbcprd 9561 Aug 6 05:06 opLogDir
-rw-rw-r-- 1 boper sbcprd 9834 Aug 6 03:34 opLogDir
-rw-rw-r-- 1 xoper sbcprd 9873 Aug 6 00:50 opLogDir
-rw-rw-r-- 1 doper sbcprd 9479 Aug 6 04:12 opLogDir
Or with the sorting logic built int:
#!/usr/bin/env perl
use strict;
use warnings;
use Time::Piece;
my $year = Time::Piece->localtime -> year;
sub sort_by_timestamp {
my ( $amon, $aday, $atime ) = (split( " ", $a ))[5,6,7];
my ( $bmon, $bday, $btime ) = (split( " ", $b ))[5,6,7];
my $at = Time::Piece->strptime("$amon $aday $atime $year", '%b %d %H:%M %Y');
my $bt = Time::Piece->strptime("$bmon $bday $btime $year", '%b %d %H:%M %Y');
return $at <=> $bt;
}
print sort { sort_by_timestamp } <DATA>;
__DATA__
-rw-rw-r-- 1 noper sbcprd 9175 Aug 6 03:36 opLogDir
-rw-rw-r-- 1 soper sbcprd 9104 Aug 6 03:04 opLogDir
-rw-rw-r-- 1 moper sbcprd 9561 Aug 6 02:18 opLogDir
-rw-rw-r-- 1 woper sbcprd 9561 Aug 6 05:06 opLogDir
-rw-rw-r-- 1 boper sbcprd 9834 Aug 6 03:34 opLogDir
-rw-rw-r-- 1 xoper sbcprd 9873 Aug 6 00:50 opLogDir
-rw-rw-r-- 1 doper sbcprd 9479 Aug 6 04:12 opLogDir
Note - for obvious reasons, this won't work very well when you span a year.
If you don't know -k option, you can still do it by putting $8 at the first placeļ¼
awk '{print $8,$3,$6,$7}'
I want to add a new column at the beggining of every row. Command used:
tree -afispugD --inodes
I want to put a new column which will be the name of the file.
Example:
119801433 -rwxr--r-- u1915811 alum 1252 Apr 1 21:06 ./file
119802284 -rw-r--r-- u1915811 alum 1255 Mar 20 10:25 ./text.txt
119865862 drwxr-xr-x u1915811 alum 4096 Feb 27 10:20 ./S3/folder2
To this:
file 119801433 -rwxr--r-- u1915811 alum 1252 Apr 1 21:06 ./file
text.txt 119802284 -rw-r--r-- u1915811 alum 1255 Mar 20 10:25 ./text.txt
folder2 119865862 drwxr-xr-x u1915811 alum 4096 Feb 27 10:20 ./S3/folder2
PS: I have to do it because tree command doesn't show names :(
All you need is:
$ awk -F'/' '{print $NF,$0}' file
file 119801433 -rwxr--r-- u1915811 alum 1252 Apr 1 21:06 ./file
text.txt 119802284 -rw-r--r-- u1915811 alum 1255 Mar 20 10:25 ./text.txt
folder2 119865862 drwxr-xr-x u1915811 alum 4096 Feb 27 10:20 ./S3/folder2
or if you want to use some specific spacing in the output use printf instead of print:
$ awk -F'/' '{printf "%-10s%s\n",$NF,$0}' file
file 119801433 -rwxr--r-- u1915811 alum 1252 Apr 1 21:06 ./file
text.txt 119802284 -rw-r--r-- u1915811 alum 1255 Mar 20 10:25 ./text.txt
folder2 119865862 drwxr-xr-x u1915811 alum 4096 Feb 27 10:20 ./S3/folder2
or, since this is a simple substitution on a single line, you could use sed instead of awk:
$ sed 's/\(.*\/\(.*\)\)/\2 \1/' file
file 119801433 -rwxr--r-- u1915811 alum 1252 Apr 1 21:06 ./file
text.txt 119802284 -rw-r--r-- u1915811 alum 1255 Mar 20 10:25 ./text.txt
folder2 119865862 drwxr-xr-x u1915811 alum 4096 Feb 27 10:20 ./S3/folder2
This one will also work, if file has whitespaces in its name or if it's a symbolic link
tree -afispugD --inodes | awk '{FS="./"; ORS=""; printf("%-60s%s\n",$NF,$0)}'
Until there are spaces in the filenames, this should work:
tree -afispugD --inodes | awk '{printf("-30s%s\n",$NF,$0}'
Using only grep and sed, is there a way I can tranform the output of ls -l * into this :
-rw-r--r-- agenda.txt
-rw-r--r-- annuaire.txt
Thanks!
seeing that you have already got your "answer", here's one of the simpler solution
ls -l | tr -s " "| cut -d" " -f1,8-
#OP, sed is "powerful", but sometimes, simplicity is more powerful.
Side note: Don't parse file names like that.
ls -l | sed 's/[ ]+//g' | sed 's/ [0-9].*:.[0-9]/ /g'
ls -altrh| sed -E 's/ +.+ / / g'
Or you can go with ssed which supports Perl Regular Expressions.
I solved your problem using the ssed program you can install it in any Posix system, ssed stands for super sed.
so i did a ls -latrh in my home directory.
telsa:~ mahmoh$ ls -altrh
total 136
drwxr-xr-x 5 root admin 170B Jun 24 00:27 ../
drwx------+ 4 mahmoh staff 136B Jun 24 00:27 Pictures/
drwx------+ 3 mahmoh staff 102B Jun 24 00:27 Music/
drwx------+ 3 mahmoh staff 102B Jun 24 00:27 Movies/
drwx------+ 3 mahmoh staff 102B Jun 24 00:27 Desktop/
-rw------- 1 mahmoh staff 3B Jun 24 00:27 .CFUserTextEncoding
drwxr-xr-x+ 5 mahmoh staff 170B Jun 24 00:27 Public/
drwx------+ 5 mahmoh staff 170B Jun 24 02:19 Documents/
-rw-r--r--# 1 mahmoh staff 15K Jun 24 02:19 .DS_Store
drwx------# 36 mahmoh staff 1.2K Jun 24 14:48 Library/
-rw-r--r-- 1 mahmoh staff 279B Jun 24 15:27 .profile~
-rw-r--r--# 1 mahmoh staff 14K Jun 24 15:29 .vimrc
-rw-r--r-- 1 mahmoh staff 279B Jun 24 15:30 .profile
drwx------ 2 mahmoh staff 68B Jun 24 15:46 .Trash/
drwxr-xr-x 3 mahmoh staff 102B Jun 24 20:26 .mplayer/
-rw------- 1 mahmoh staff 3.5K Jun 24 22:11 .bash_history
-rw------- 1 mahmoh staff 42B Jun 24 23:25 .lesshst
-rw-r--r-- 1 mahmoh staff 3.6K Jun 24 23:39 temp
-rw-r--r-- 1 mahmoh staff 3.3K Jun 24 23:43 rtorrent.rc~
drwxr-xr-x 5 mahmoh staff 170B Jun 24 23:52 torrents/
-rw-r--r-- 1 mahmoh staff 3.3K Jun 24 23:56 .rtorrent.rc~
-rw------- 1 mahmoh staff 3.7K Jun 24 23:56 .viminfo
-rw-r--r-- 1 mahmoh staff 3.3K Jun 24 23:56 .rtorrent.rc
drwxr-xr-x+ 25 mahmoh staff 850B Jun 24 23:56 ./
drwx------+ 10 mahmoh staff 340B Jun 24 23:58 Downloads/
Now watch.
telsa:~ mahmoh$ ls -altrh| ssed -R -e 's/ +.+ / / g'
total 136
drwxr-xr-x ../
drwx------+ Pictures/
drwx------+ Music/
drwx------+ Movies/
drwx------+ Desktop/
-rw------- .CFUserTextEncoding
drwxr-xr-x+ Public/
drwx------+ Documents/
-rw-r--r--# .DS_Store
drwx------# Library/
-rw-r--r-- .profile~
-rw-r--r--# .vimrc
-rw-r--r-- .profile
drwx------ .Trash/
drwxr-xr-x .mplayer/
-rw------- .bash_history
-rw------- .lesshst
-rw-r--r-- temp
-rw-r--r-- rtorrent.rc~
drwxr-xr-x torrents/
-rw-r--r-- .rtorrent.rc~
-rw------- .viminfo
-rw-r--r-- .rtorrent.rc
drwxr-xr-x+ ./
drwx------+ Downloads/
ls -l | sed 's/^\([^\t ]\+\)\(.*:.[^ \t]\+\)\(.\+\)/\1 \3/'
Here is a working command. The slightly tricky thing is that ls -l will print the year for files that are older than some time (6 months) and hh:mm for newer files.
ls -l | sed 's/ .*[0-9]* .*[A-Z][a-z][a-z] [ 0-9][0-9] \{1,2\}[0-9][0-9]:*[0-9][0-9] / /'
For the following example
drwxr-xr-x 39 root root 1024 Feb 19 08:58 /
the starting .* will match 39 root root 1024 and then the rest of the regular expression matches month name (so you might restrict a-z to fewer characters) followed by year or hh:mm.
why not use awk instead of sed? awk is built for stuff like this.
see this manual page for more about fields in awk.
Like this?
ls -l | sed 's/ [0-9].*:.[0-9] / /' | less
Transforms
-rw-r--r-- 1 tomislav tomislav 609 2009-11-26 10:32 Test.class
-rw-r--r-- 1 tomislav tomislav 46 2009-12-14 12:16 test.groovy
into
-rw-r--r-- Test.class
-rw-r--r-- test.groovy