Let's assume that command1 is processing something and I am interested in BOTH output of command1 as well as how many lines the output actually had.
$ command1 | wc - l
prints number of lines of output of command1, while
$ command1 | nl
prints something like that:
1 ./PaDe014
2 ./PaDe033
3 ./PaDe001
4 ./PaDe013
5 ./PaDe025
6 ./PaDe028
See How to count lines in a document? for more solutions.
However I am interested to generate output like that:
./PaDe014
./PaDe033
./PaDe001
./PaDe013
./PaDe025
./PaDe028
Total number of files generated: 6
I have a vague feeling that it can be achieved with tee and wc, but cannot figure out how exactly.
What is the simplest way to achieve desired output?
I also tried:
command1 | tee >(wc -l)
but there must be a race condition here, as from time to time I receive strange results. Here is the test output:
pdebski#PaDe:~$ (ls -l ; printf "Total: ") | tee -a >(wc -l)
total 56
drwxr-xr-x 7 pdebski pdebski 4096 cze 22 18:50 Data
drwxr-xr-x 2 pdebski pdebski 4096 cze 22 15:14 Desktop
drwxr-xr-x 2 pdebski pdebski 4096 cze 22 14:50 Documents
drwxr-xr-x 2 pdebski pdebski 4096 cze 16 01:09 Downloads
-rw-r--r-- 1 pdebski pdebski 8980 cze 16 01:04 examples.desktop
drwxr-xr-x 4 pdebski pdebski 4096 cze 17 13:44 Music
drwxr-xr-x 2 pdebski pdebski 4096 cze 22 13:14 Pictures
drwxr-xr-x 2 pdebski pdebski 4096 cze 16 01:09 Public
drwxr-xr-x 2 pdebski pdebski 4096 cze 16 01:09 Templates
drwxrwxr-x 2 pdebski pdebski 4096 cze 22 01:21 test
drwxrwxr-x 2 pdebski pdebski 4096 cze 22 01:21 test2
drwxr-xr-x 2 pdebski pdebski 4096 cze 16 01:09 Videos
Total: pdebski#PaDe:~$ 13
A nice illustration for the need of Dijkstra's Semaphores and Mutexes, is not it? (see https://en.wikipedia.org/wiki/Edsger_W._Dijkstra )
Apparently there should be some wait here.
Same results with:
$ (ls -l && printf "Total: ") | tee -a >(wc -l)
I don't know what you mean by "simplest", but for me the simplest solution is:
command1 | awk '1; END { print "\ntotal number of lines:", NR }'
Using process substitution:
command1 | tee >(wc -l)
Or, for appending the Total number of files generated prefix:
(command1 && printf "Total number of files generated: ") | tee >(wc -l)
Reference:
https://unix.stackexchange.com/questions/28503/how-can-i-send-stdout-to-multiple-commands
Related
Is there any way to take away the owner's permission to read a file in macOS? I know there's no reason to do this but I have to for school and I can't find an answer anywhere. Removing my write permission works fine but when I try to remove my read permission it automatically give me my read and write permissions back. As you can see in the console when I use chmod -v -v (extra verbose) it shows the correct permissions it should be changed to but then when checking afterwards they havent changed into that...
thijs#Thijss-MacBook-Air-2 week6 % ls -l
total 16
-rw----r-- 1 thijs staff 12 Oct 11 21:10 greeting.txt
-rw-r--r-- 1 thijs staff 0 Oct 11 21:10 hello.txt
-rw------- 1 thijs staff 15 Oct 11 21:11 weather.txt
thijs#Thijss-MacBook-Air-2 week6 % chmod -v -v u-w weather.txt
weather.txt: 0100600 [-rw------- ] -> 0100400 [-r-------- ]
thijs#Thijss-MacBook-Air-2 week6 % ls -l
total 16
-rw----r-- 1 thijs staff 12 Oct 11 21:10 greeting.txt
-rw-r--r-- 1 thijs staff 0 Oct 11 21:10 hello.txt
-r-------- 1 thijs staff 15 Oct 11 21:11 weather.txt
thijs#Thijss-MacBook-Air-2 week6 % chmod -v -v u-r weather.txt
weather.txt: 0100400 [-r-------- ] -> 0100000 [---------- ]
thijs#Thijss-MacBook-Air-2 week6 % ls -l
total 16
-rw----r-- 1 thijs staff 12 Oct 11 21:10 greeting.txt
-rw-r--r-- 1 thijs staff 0 Oct 11 21:10 hello.txt
-rw------- 1 thijs staff 15 Oct 11 21:11 weather.txt
When I execute the ls -l -h command, I get an output as show by the image below.
How can the number of the items in a folder be included in the output?
Update
The current output looks like this
total 41M
-rw-r--r-- 1 root root 41M Dec 20 09:56 completed_projects.bson
-rw-r--r-- 1 root root 213 Dec 20 09:57 completed_projects.metadata.json
drwxrwxr-x 2 adipster adipster 4.0K Jun 16 13:22 contents
-rw-rw-r-- 1 adipster adipster 13 Jun 16 13:20 file.py
drwxrwxr-x 4 adipster adipster 4.0K Jun 16 13:22 folder
drwxrwxr-x 2 adipster adipster 4.0K Jun 16 13:21 items
But I'll like to have another column indicating the number of items in a folder like this
total 41M
-rw-r--r-- 1 root root 41M Dec 20 09:56 completed_projects.bson
-rw-r--r-- 1 root root 213 Dec 20 09:57 completed_projects.metadata.json
drwxrwxr-x 2 adipster adipster 4.0K Jun 16 13:22 contents 235
-rw-rw-r-- 1 adipster adipster 13 Jun 16 13:20 file.py
drwxrwxr-x 4 adipster adipster 4.0K Jun 16 13:22 folder 19
drwxrwxr-x 2 adipster adipster 4.0K Jun 16 13:21 items 5
where the numbers at the extreme right represents the number of items in a folder
You can do something like this:
echo -n "Number of files in folder is: " && ls | wc -l && ls -l
ouptut should be something like this:
umber of files in folder is: 3
Total 279K
-rwxr-xr-x 1 user users 19K Jun 16 00:17 a
-rwxr-xr-x 1 user users 5K Jun 16 00:17 b
-rwxr-xr-x 1 user users 255K Jun 16 00:17 c
You can omit echo statement, just as a note -n is no new line flag.
sed has an option to execute the constructed replacement with /e.
We only count subdirs, looking at the first character
ls -l | sed -r 's/d(.*) ([^ ]*)/printf "d%s %-20s%s\n" "\1" \2 $(ls \2| wc -l)/e'
EDIT: Solution for directories with spaces in their name.
Parsing ls should be avoided. When you try to fix above cmmand for directory names with spaces, you might try
# Don't do this
ls -l | sed -r 's/d(.{,48}) (.*)/printf "d%s %-20s%s\n" "\1" "\2" $(ls "\2"| wc -l)/e'
It is time to write a script. Perhaps with find or something like
#/bin/bash
for i in *; do
printf "%-70s %s\n" "$(/bin/ls -ld "$i")" "$(/bin/ls -d "$i"/* 2>/dev/null| wc -l)"
done
The wc in the subdir will count wrong when filenames have newlines.
ls() { command ls "$#" | tee >(echo "$(wc -l) items"); }
That uses an output process substitution to run the little "echo" script on its stdin while also displaying stdin (thanks to tee). This way, you don't have to run ls twice.
Usual caveat: output will be incorrect when there's a file with a newline in the name.
In this directory when I run
ls -l
it prints the following output:
-rw------- 1 csundl dcsugrad 0 Dec 5 13:51 file3
drwx------ 2 csundl dcsugrad 4096 Dec 5 13:51 Photos
drwx------ 2 csundl dcsugrad 4096 Dec 5 13:51 Pron
drwx------ 2 csunfi dcsugrad 4096 Dec 5 13:51 Spreadsheets
drwx------ 3 csundl dcsugrad 4096 Dec 5 15:12 Stuff
-rwx------ 1 csundl dcsugrad 149 Dec 5 15:08 untitled.sh
The bolded values are the byte sizes (?).
However when I run:
ls -l | wc -c
The total byte size comes up as 340. Why is this?
Thanks.
ls command displays size of the file
but this command:
ls -l | wc -c
counts number of characters in the output of ls command.
To count total size of files in a directory use du command:
du -hs mydir
How would you go about changing permissions for a file or in a directory recursively in such a way that group permissions would be copied over to world permissions, with no other changes? For example, to go from this directory listing:
drwxr-x--- 2 septi septi 4096 Jun 29 01:14 example.d
-rw-r----- 1 septi septi 0 Jun 29 01:14 example.r
-rwxr-x--- 1 septi septi 0 Jun 29 01:14 example.x
...to:
drwxr-xr-x 2 septi septi 4096 Jun 29 01:14 example.d
-rw-r--r-- 1 septi septi 0 Jun 29 01:14 example.r
-rwxr-xr-x 1 septi septi 0 Jun 29 01:14 example.x
From the chmod(1) man page (relevant parts extracted):
-R Change the modes of the file hierarchies rooted in the files
instead of just the files themselves.
And:
The symbolic mode is described by the following grammar:
who ::= a | u | g | o
op ::= + | - | =
perm ::= r | s | t | w | x | X | u | g | o
The who symbols "u", "g", and "o" specify the user, group, and
other parts of the mode bits, respectively. The who symbol a is
equivalent to ugo.
The perm symbols represent the portions of the mode bits as follows:
g The group permission bits in the original mode of the file.
So for you:
chmod -R o=g *
Example:
$ ls -l
total 0
drwxr-x--- 2 carl staff 68 Jun 28 10:25 example.d
-rw-r----- 1 carl staff 0 Jun 28 10:25 example.r
-rwxr-x--- 1 carl staff 0 Jun 28 10:25 example.x
$ chmod -R o=g *
$ ls -l
total 0
drwxr-xr-x 2 carl staff 68 Jun 28 10:25 example.d
-rw-r--r-- 1 carl staff 0 Jun 28 10:25 example.r
-rwxr-xr-x 1 carl staff 0 Jun 28 10:25 example.x
Using only grep and sed, is there a way I can tranform the output of ls -l * into this :
-rw-r--r-- agenda.txt
-rw-r--r-- annuaire.txt
Thanks!
seeing that you have already got your "answer", here's one of the simpler solution
ls -l | tr -s " "| cut -d" " -f1,8-
#OP, sed is "powerful", but sometimes, simplicity is more powerful.
Side note: Don't parse file names like that.
ls -l | sed 's/[ ]+//g' | sed 's/ [0-9].*:.[0-9]/ /g'
ls -altrh| sed -E 's/ +.+ / / g'
Or you can go with ssed which supports Perl Regular Expressions.
I solved your problem using the ssed program you can install it in any Posix system, ssed stands for super sed.
so i did a ls -latrh in my home directory.
telsa:~ mahmoh$ ls -altrh
total 136
drwxr-xr-x 5 root admin 170B Jun 24 00:27 ../
drwx------+ 4 mahmoh staff 136B Jun 24 00:27 Pictures/
drwx------+ 3 mahmoh staff 102B Jun 24 00:27 Music/
drwx------+ 3 mahmoh staff 102B Jun 24 00:27 Movies/
drwx------+ 3 mahmoh staff 102B Jun 24 00:27 Desktop/
-rw------- 1 mahmoh staff 3B Jun 24 00:27 .CFUserTextEncoding
drwxr-xr-x+ 5 mahmoh staff 170B Jun 24 00:27 Public/
drwx------+ 5 mahmoh staff 170B Jun 24 02:19 Documents/
-rw-r--r--# 1 mahmoh staff 15K Jun 24 02:19 .DS_Store
drwx------# 36 mahmoh staff 1.2K Jun 24 14:48 Library/
-rw-r--r-- 1 mahmoh staff 279B Jun 24 15:27 .profile~
-rw-r--r--# 1 mahmoh staff 14K Jun 24 15:29 .vimrc
-rw-r--r-- 1 mahmoh staff 279B Jun 24 15:30 .profile
drwx------ 2 mahmoh staff 68B Jun 24 15:46 .Trash/
drwxr-xr-x 3 mahmoh staff 102B Jun 24 20:26 .mplayer/
-rw------- 1 mahmoh staff 3.5K Jun 24 22:11 .bash_history
-rw------- 1 mahmoh staff 42B Jun 24 23:25 .lesshst
-rw-r--r-- 1 mahmoh staff 3.6K Jun 24 23:39 temp
-rw-r--r-- 1 mahmoh staff 3.3K Jun 24 23:43 rtorrent.rc~
drwxr-xr-x 5 mahmoh staff 170B Jun 24 23:52 torrents/
-rw-r--r-- 1 mahmoh staff 3.3K Jun 24 23:56 .rtorrent.rc~
-rw------- 1 mahmoh staff 3.7K Jun 24 23:56 .viminfo
-rw-r--r-- 1 mahmoh staff 3.3K Jun 24 23:56 .rtorrent.rc
drwxr-xr-x+ 25 mahmoh staff 850B Jun 24 23:56 ./
drwx------+ 10 mahmoh staff 340B Jun 24 23:58 Downloads/
Now watch.
telsa:~ mahmoh$ ls -altrh| ssed -R -e 's/ +.+ / / g'
total 136
drwxr-xr-x ../
drwx------+ Pictures/
drwx------+ Music/
drwx------+ Movies/
drwx------+ Desktop/
-rw------- .CFUserTextEncoding
drwxr-xr-x+ Public/
drwx------+ Documents/
-rw-r--r--# .DS_Store
drwx------# Library/
-rw-r--r-- .profile~
-rw-r--r--# .vimrc
-rw-r--r-- .profile
drwx------ .Trash/
drwxr-xr-x .mplayer/
-rw------- .bash_history
-rw------- .lesshst
-rw-r--r-- temp
-rw-r--r-- rtorrent.rc~
drwxr-xr-x torrents/
-rw-r--r-- .rtorrent.rc~
-rw------- .viminfo
-rw-r--r-- .rtorrent.rc
drwxr-xr-x+ ./
drwx------+ Downloads/
ls -l | sed 's/^\([^\t ]\+\)\(.*:.[^ \t]\+\)\(.\+\)/\1 \3/'
Here is a working command. The slightly tricky thing is that ls -l will print the year for files that are older than some time (6 months) and hh:mm for newer files.
ls -l | sed 's/ .*[0-9]* .*[A-Z][a-z][a-z] [ 0-9][0-9] \{1,2\}[0-9][0-9]:*[0-9][0-9] / /'
For the following example
drwxr-xr-x 39 root root 1024 Feb 19 08:58 /
the starting .* will match 39 root root 1024 and then the rest of the regular expression matches month name (so you might restrict a-z to fewer characters) followed by year or hh:mm.
why not use awk instead of sed? awk is built for stuff like this.
see this manual page for more about fields in awk.
Like this?
ls -l | sed 's/ [0-9].*:.[0-9] / /' | less
Transforms
-rw-r--r-- 1 tomislav tomislav 609 2009-11-26 10:32 Test.class
-rw-r--r-- 1 tomislav tomislav 46 2009-12-14 12:16 test.groovy
into
-rw-r--r-- Test.class
-rw-r--r-- test.groovy