When I execute the ls -l -h command, I get an output as show by the image below.
How can the number of the items in a folder be included in the output?
Update
The current output looks like this
total 41M
-rw-r--r-- 1 root root 41M Dec 20 09:56 completed_projects.bson
-rw-r--r-- 1 root root 213 Dec 20 09:57 completed_projects.metadata.json
drwxrwxr-x 2 adipster adipster 4.0K Jun 16 13:22 contents
-rw-rw-r-- 1 adipster adipster 13 Jun 16 13:20 file.py
drwxrwxr-x 4 adipster adipster 4.0K Jun 16 13:22 folder
drwxrwxr-x 2 adipster adipster 4.0K Jun 16 13:21 items
But I'll like to have another column indicating the number of items in a folder like this
total 41M
-rw-r--r-- 1 root root 41M Dec 20 09:56 completed_projects.bson
-rw-r--r-- 1 root root 213 Dec 20 09:57 completed_projects.metadata.json
drwxrwxr-x 2 adipster adipster 4.0K Jun 16 13:22 contents 235
-rw-rw-r-- 1 adipster adipster 13 Jun 16 13:20 file.py
drwxrwxr-x 4 adipster adipster 4.0K Jun 16 13:22 folder 19
drwxrwxr-x 2 adipster adipster 4.0K Jun 16 13:21 items 5
where the numbers at the extreme right represents the number of items in a folder
You can do something like this:
echo -n "Number of files in folder is: " && ls | wc -l && ls -l
ouptut should be something like this:
umber of files in folder is: 3
Total 279K
-rwxr-xr-x 1 user users 19K Jun 16 00:17 a
-rwxr-xr-x 1 user users 5K Jun 16 00:17 b
-rwxr-xr-x 1 user users 255K Jun 16 00:17 c
You can omit echo statement, just as a note -n is no new line flag.
sed has an option to execute the constructed replacement with /e.
We only count subdirs, looking at the first character
ls -l | sed -r 's/d(.*) ([^ ]*)/printf "d%s %-20s%s\n" "\1" \2 $(ls \2| wc -l)/e'
EDIT: Solution for directories with spaces in their name.
Parsing ls should be avoided. When you try to fix above cmmand for directory names with spaces, you might try
# Don't do this
ls -l | sed -r 's/d(.{,48}) (.*)/printf "d%s %-20s%s\n" "\1" "\2" $(ls "\2"| wc -l)/e'
It is time to write a script. Perhaps with find or something like
#/bin/bash
for i in *; do
printf "%-70s %s\n" "$(/bin/ls -ld "$i")" "$(/bin/ls -d "$i"/* 2>/dev/null| wc -l)"
done
The wc in the subdir will count wrong when filenames have newlines.
ls() { command ls "$#" | tee >(echo "$(wc -l) items"); }
That uses an output process substitution to run the little "echo" script on its stdin while also displaying stdin (thanks to tee). This way, you don't have to run ls twice.
Usual caveat: output will be incorrect when there's a file with a newline in the name.
Related
I write below statements in my .bashrc
alias ls='ls -l'
$LS = 'ls'
I expected $LS execute ls -l in command line prompt because ls already modified to ls -l. but $LS execute ls only ls.
ls execute ls -l because alias as below:
> ls
total 28
-rw-r--r-- 1 chlee chlee 82 8월 30 22:07 '#test.py#'
drwxr-xr-x 2 chlee chlee 4096 8월 21 04:09 exer
-rw-r--r-- 1 chlee chlee 64 8월 30 21:49 test.py
-rw-r--r-- 1 chlee chlee 49 8월 30 21:47 test.py~
drwxr-xr-x 29 chlee chlee 12288 8월 21 02:51 vsdbg
$LS execute only ls as below:
> $LS
'#test.py#' exer test.py test.py~ vsdbg
How can I get the shell to take into account aliases when variable evaluation?
This should work as you expect:
#!/bin/bash
alias ls="ls -l"
LS () {
ls
}
echo trying lower
ls
echo Trying upper
LS
The function is called LS including parses aliases.
# source LS.sh
trying lower
total 4
-rwxr-xr-x 1 root root 89 Sep 19 21:50 LS.sh
Trying upper
total 4
-rwxr-xr-x 1 root root 89 Sep 19 21:50 LS.sh
Jetchisel was pointing you to this in the comments.
I want to make my writing to the console output in a nice tidy human readable.
here is how it looks now:
====================== Sat Apr 16 12:57:17 EDT 2022 ======================
==========================================================================
====================== Leopard - Download from S3 ======================
==========================================================================
==========================================================================
====================== Leopard - Decompressing ======================
==========================================================================
total 1349872
drwxr-xr-x 2 root root 12288 Apr 16 12:57 .
drwxrwxrwt. 4 root root 102 Apr 16 12:57 ..
-rw-r--r-- 1 root root 185070885 Apr 16 12:03 asdasdasd.sql.gz
-rw-r--r-- 1 root root 40344632 Apr 16 12:03 asdasdas.sql.gz
-rw-r--r-- 1 root root 26631 Apr 16 12:03 asdad.sql.gz
-rw-r--r-- 1 root root 1679 Apr 16 12:03 asdasd.sql.gz
-rw-r--r-- 1 root root 1237 Apr 16 12:03 asd.sql.gz
-rw-r--r-- 1 root root 5241900 Apr 16 12:03 asdasd.sql.gz
-rw-r--r-- 1 root root 1144 Apr 16 12:03 asdasasd.sql.gz
-rw-r--r-- 1 root root 489312 Apr 16 12:03 asdasd.sql.gz
-rw-r--r-- 1 root root 1138 Apr 16 12:03 asdasdasd.sql.gz
==========================================================================
====================== NewYorkCity - Download from S3 ======================
==========================================================================
==========================================================================
====================== NewYorkCity - Unloading SSL Example ======================
==========================================================================
total 1349872
drwxr-xr-x 2 root root 12288 Apr 16 12:57 .
drwxrwxrwt. 4 root root 102 Apr 16 12:57 ..
-rw-r--r-- 1 root root 185070885 Apr 16 12:03 asdasdasd.sql.gz
-rw-r--r-- 1 root root 40344632 Apr 16 12:03 asdasdas.sql.gz
-rw-r--r-- 1 root root 26631 Apr 16 12:03 asdad.sql.gz
-rw-r--r-- 1 root root 1679 Apr 16 12:03 asdasd.sql.gz
-rw-r--r-- 1 root root 1237 Apr 16 12:03 asd.sql.gz
-rw-r--r-- 1 root root 5241900 Apr 16 12:03 asdasd.sql.gz
-rw-r--r-- 1 root root 1144 Apr 16 12:03 asdasasd.sql.gz
-rw-r--r-- 1 root root 489312 Apr 16 12:03 asdasd.sql.gz
-rw-r--r-- 1 root root 1138 Apr 16 12:03 asdasdasd.sql.gz
I want that all the === line will be in the same length, and the text always in the center with 1 space on each side
Will appreciate assistance here :)
UPDATE / EDIT:
The original script is something like that:
eecho () { echo ==========================================================================; }
echo_stage () {
START=1
END=11
for (( c=$START; c<=$END; c++ ))
do
printf == '-%.0s'
done
echo -n " " $1 " "
for (( c=$START; c<=$END; c++ ))
do
printf == '-%.0s'
done
echo
}
stage() {
eecho
echo_stage "$1" "$2"
eecho
}
print_date () { echo "======================" $(date) "======================"; }
reload_db() {
print_date
rm -rf /var/tmp/db
mkdir -p /var/tmp/db
stage "DB - Download from S3"
aws s3 sync s3://db-backup/latest/ /var/tmp/db --profile=papilon --quiet
stage "DB - Decompressing"
pigz -d /var/tmp/db/*
stage "DB - Restoring Data"
cd /var/tmp/db
stage "DB - Restoring Tables"
for i in `ls -1 *.sql | grep -v "_view.sql"`;do echo $i;mysql db < $i;done
stage "DB - Restoring Views"
for i in `ls -1 *.sql | grep "_view.sql"`;do echo $i;mysql db < $i;done
stage "DB - Clean up"
rm -rf /var/tmp/db
print_date
}
reload_db
This awk filter will size your === padding correctly, and align titles to the center, if you pipe your command output through it:
# cmd |
awk '
BEGIN {a[1] = "="}
NF>1 && $1~/^=+$/ && $NF~/^=+$/ {
sub(/^=+/, "")
sub(/=+$/, "")
title_len=length($0)
pad = ""
for (i=1; i<=(74-title_len)/2; ++i) {
pad=pad"="
}
$0 = pad $0 pad a[title_len%2]
}
1'
This doesn't wrap the ls -l output, only the titles. It assumes 74 is hardcoded as the length of the solid === lines (and doesn't change depending on terminal size). The array a is used to add an extra = when 74 - title_len is an odd number.
Example output:
====================== Sat Apr 16 12:57:17 EDT 2022 ======================
==========================================================================
====================== Leopard - Download from S3 ======================
==========================================================================
==========================================================================
======================= Leopard - Decompressing ========================
==========================================================================
total 1349872
drwxr-xr-x 2 root root 12288 Apr 16 12:57 .
drwxrwxrwt. 4 root root 102 Apr 16 12:57 ..
-rw-r--r-- 1 root root 185070885 Apr 16 12:03 asdasdasd.sql.gz
-rw-r--r-- 1 root root 40344632 Apr 16 12:03 asdasdas.sql.gz
-rw-r--r-- 1 root root 26631 Apr 16 12:03 asdad.sql.gz
-rw-r--r-- 1 root root 1679 Apr 16 12:03 asdasd.sql.gz
-rw-r--r-- 1 root root 1237 Apr 16 12:03 asd.sql.gz
-rw-r--r-- 1 root root 5241900 Apr 16 12:03 asdasd.sql.gz
-rw-r--r-- 1 root root 1144 Apr 16 12:03 asdasasd.sql.gz
-rw-r--r-- 1 root root 489312 Apr 16 12:03 asdasd.sql.gz
-rw-r--r-- 1 root root 1138 Apr 16 12:03 asdasdasd.sql.gz
==========================================================================
==================== NewYorkCity - Download from S3 ====================
==========================================================================
==========================================================================
================= NewYorkCity - Unloading SSL Example ==================
==========================================================================
total 1349872
drwxr-xr-x 2 root root 12288 Apr 16 12:57 .
drwxrwxrwt. 4 root root 102 Apr 16 12:57 ..
-rw-r--r-- 1 root root 185070885 Apr 16 12:03 asdasdasd.sql.gz
-rw-r--r-- 1 root root 40344632 Apr 16 12:03 asdasdas.sql.gz
-rw-r--r-- 1 root root 26631 Apr 16 12:03 asdad.sql.gz
-rw-r--r-- 1 root root 1679 Apr 16 12:03 asdasd.sql.gz
-rw-r--r-- 1 root root 1237 Apr 16 12:03 asd.sql.gz
-rw-r--r-- 1 root root 5241900 Apr 16 12:03 asdasd.sql.gz
-rw-r--r-- 1 root root 1144 Apr 16 12:03 asdasasd.sql.gz
-rw-r--r-- 1 root root 489312 Apr 16 12:03 asdasd.sql.gz
-rw-r--r-- 1 root root 1138 Apr 16 12:03 asdasdasd.sql.gz
Because you have now posted your script, I will add a new answer for pure bash.
title() {
local text pad
(( ${#1} > 70 )) && { echo "$1"; return; }
text=${1:+ }$1${1:+ }
pad=$( eval "printf %.1s ={1..$(( ( 74 - ${#text} ) / 2 ))}" )
echo "$pad$text$pad$( (( ${#text} % 2 )) && printf = )"
}
For title 'foo bar', this function prints foo bar (or any string up to 70 characters) with a space either side, centered and padded to 74 columns with =. If the string is longer than 70 characters it's too long to pad, so it's printed as is.
================================ foo bar =================================
With no argument, or an empty argument, it prints a solid line of 74 =:
==========================================================================
You can swap = for any single ASCII character.
You can swap 74 for any even number. Also for an odd number, if you change && printf to || printf in the last line. (also change the 70 to N - 4)
You can call title once for the date, or three times for the larger three line banners (see banner below).
Explanation:
${#text} is bash for "length of $text".
${1:+ } expands to a space, unless $1 is empty or unset. This allows us to add spaces to either end of the string, or exclude them for an empty string.
={1..10} expands to =1 =2 =3 ... =10, and printf %.1s prints the first character of each string. Combining these allows us to repeat a string (=) N times.
But we can't use a variable (or arithmetic) in {1..10} normally. So we need eval.
You probably heard eval is bad, and a security risk. That's often true, but here we are not passing any unknown data to eval (such as user input), and it's safe from code injection. (${#text} always expands to a single number)
So we make two bars, of length (74 - text-length) / 2, adding another = to the second bar if the text length is an odd number.
I also made a few changes to your script which you might consider. Apart from the date and titles, these have nothing to do with the padding. The title and banner functions will work on your old script.
title() {
local text pad
(( ${#1} > 70 )) && { echo "$1"; return; }
text="${1:+ }$1${1:+ }"
pad=$( eval "printf %.1s ={1..$(( ( 74 - ${#text} ) / 2 ))}" )
echo "$pad$text$pad$( (( ${#text} % 2 )) && printf = )"
}
banner() {
title
title "$1"
title
}
reload_db() {
local i
title "$(date)"
rm -rf /var/tmp/db || exit 1
mkdir -p /var/tmp/db || exit 1
banner 'DB - Download from S3'
aws s3 sync s3://db-backup/latest/ /var/tmp/db --profile=papilon --quiet
banner 'DB - Decompressing'
pigz -d /var/tmp/db/*
banner 'DB - Restoring Data'
cd /var/tmp/db || exit 1
banner 'DB - Restoring Tables'
GLOBIGNORE='*_view.sql'
for i in *.sql; do
echo "$i"
mysql db < "$i"
done
GLOBIGNORE=
banner 'DB - Restoring Views'
for i in *_view.sql; do
echo "$i"
mysql db < "$i"
done
banner 'DB - Clean up'
rm -rf /var/tmp/db
title "$(date)"
}
At the very least, you should exit early if cd, mkdir, or the first rm fail. Also, looping over an unquoted ls command sub is a bad idea. Instead you can use glob expansion (or find).
You could also concatenate the SQL scripts, but this only works if all the commands end with a semicolon (see Run multiple sql files in mysql batch):
big_title 'DB - Restoring Tables'
GLOBIGNORE='*_view.sql'
printf '%s\n' *.sql
cat *.sql | mysql db
GLOBIGNORE=
big_title 'DB - Restoring Views'
printf '%s\n' *_view.sql
cat *_view.sql | mysql db
I am trying to grep some pattern in a file set under a folder like below
Where on the output I have to perform remaining operation.
The output main.log is coming so huge almost 50k lines ,as the files starting with server02.log are almost 30 to 40 in number . The script based on this output is taking forever to complete.
Is there a way that I can only take files name starting with server02.log. and generated between time
20:00:00 and 21:00:00
ls -lrth server02.log.*
-rw-r--r-- 1 user user 1.9M Apr 15 20:20 server02.log.2020
-rw-r--r-- 1 user user 1.7M Apr 15 20:30 server02.log.2030
-rw-r--r-- 1 user user 1.6M Apr 15 20:41 server02.log.2041
-rw-r--r-- 1 user user 1.9M Apr 15 20:50 server02.log.2050
-rw-r--r-- 1 user user 2.1M Apr 15 21:00 server02.log.2100
-rw-r--r-- 1 user user 1.4M Apr 15 21:10 server02.log.2110
-rw-r--r-- 1 user user 1.9M Apr 15 21:20 server02.log.2120
-rw-r--r-- 1 user user 656K Apr 15 21:29 server02.log.2129
-rw-r--r-- 1 user user 4.6M Apr 15 21:40 server02.log.2140
-rw-r--r-- 1 user user 1.9M Apr 15 21:50 server02.log.2150
-rw-r--r-- 1 user user 1.7M Apr 15 21:59 server02.log.2159
-rw-r--r-- 1 user user 724K Apr 15 22:09 server02.log.2209
-rw-r--r-- 1 user user 1.3M Apr 15 22:20 server02.log.2220
-rw-r--r-- 1 user user 1.1M Apr 15 22:29 server02.log.2229
-rw-r--r-- 1 user user 1.7M Apr 15 22:41 server02.log.2241
-rw-r--r-- 1 user user 1.5M Apr 15 22:49 server02.log.2249
-rw-r--r-- 1 user user 2.4M Apr 15 23:01 server02.log.2301
-rw-r--r-- 1 user user 1.4M Apr 15 23:10 server02.log.2310
-rw-r--r-- 1 user user 585K Apr 15 23:19 server02.log.2319
-rw-r--r-- 1 user user 858K Apr 15 23:30 server02.log.2330
-rw-r--r-- 1 user user 892K Apr 15 23:40 server02.log.2340
-rw-r--r-- 1 user user 698K Apr 15 23:49 server02.log.2349
grep -E "###Update |###Initiate |###Re-Initiate " server02.log.* >> main.log
from the comments I made the change to my code as below
#!/bin/bash
DIR="."
d=$(date +%Y-%m-%d);
log_dir="logs/$d"
PREFIX="$log_dir/srv_02.log"
#PREFIX="srv_02.log"
echo "prefix value is $PREFIX"
START_HOUR="06"
for F in "$( find "$DIR" -name "${PREFIX}*" -printf '%Tc %p\n' | grep "\ ${START_HOUR}:" )"; do
echo "F value is $F"
grep -E "###Update |###Initiate |###Re-Initiate" "$F" >> main.log
done
error:
prefix value is logs/2021-04-16/srv_02.log
find: warning: Unix filenames usually don't contain slashes (though pathnames do). That means that '-name `logs/2021-04-16/srv_02.log*'' will probably evaluate to false all the time on this system. You might find the '-wholename' test more useful, or perhaps '-samefile'. Alternatively, if you are using GNU grep, you could use 'find ... -print0 | grep -FzZ `logs/2021-04-16/osbpd_srv_02.log*''.
F value is
grep: : No such file or directory
This solution looks for files in the given directory, created during the specified hour with names matching the given prefix.
#!/bin/bash
d=$(date +%Y-%m-%d)
DIR="logs/$d/$log_dir"
PREFIX="srv_02.log"
#PREFIX=server02.log
echo "prefix value is $PREFIX"
START_HOUR="06"
for F in "$( find "$DIR" -name "${PREFIX}*" -printf '%TY-%Tm-%Td\n' | grep "\ ${START_HOUR}:" )"; do
echo "$F"
# grep -E "###Update |###Initiate |###Re-Initiate Assignment Milestone|###Complete Assignment Milestone|###Cancel Assignment Milestone|###Suspend Assignment Milestone|###Resume Assignment Milestone" "$F" >> main.log
done
In this directory when I run
ls -l
it prints the following output:
-rw------- 1 csundl dcsugrad 0 Dec 5 13:51 file3
drwx------ 2 csundl dcsugrad 4096 Dec 5 13:51 Photos
drwx------ 2 csundl dcsugrad 4096 Dec 5 13:51 Pron
drwx------ 2 csunfi dcsugrad 4096 Dec 5 13:51 Spreadsheets
drwx------ 3 csundl dcsugrad 4096 Dec 5 15:12 Stuff
-rwx------ 1 csundl dcsugrad 149 Dec 5 15:08 untitled.sh
The bolded values are the byte sizes (?).
However when I run:
ls -l | wc -c
The total byte size comes up as 340. Why is this?
Thanks.
ls command displays size of the file
but this command:
ls -l | wc -c
counts number of characters in the output of ls command.
To count total size of files in a directory use du command:
du -hs mydir
I have a sequence of image files that look like this:
image-149454.jpg
image-149455.jpg
I have some other images that I want to append to the end of the sequence where it left off but currently they are numbered from 0 (i.e. image-000000 to image-010000).
What's a script I could use to rename those new images starting from a certain number and going on, in this case 149456 and onwards?
I would try something like this. You just have to adjust the offset and add a string in front of the name, just as you need it.
x=1
for i in *.jpg; do
temp=$(printf "%08d.jpg" ${x}) #padding since you seem to want it
mv ${i} ${temp}
let x=x+1
done
Here's an untested solution in Python. This should change all of the files in a directory to have sequentially higher numbers than the other.
To use it enter: script.py dir1 dir2
Assuming the original files have the higher number (149455) are in dir1 and the new files start from 000000 are in dir2:
import os, sys, re
max_image = 0
# check if current (with higher numbers) and other (with lower) directory is given
if len(sys.argv) == 3:
for files in os.listdir(sys.argv[1]): # first dir is current
if files.endswith(".jpg"):
#for all jpgs get the max
m = re.search("\-(\d+)", files)
number = int(m.group(1))
if max_image < number:
max_image = number
for files in os.listdir(sys.argv[2]): # second dir is other
if files.endswith(".jpg"):
#get current start point
m = re.search("\-(\d+)", files)
number = int(m.group(1))
os.rename(files, "image-" + str(number + max_image)+".jpg") # add the max from current folder
Here's a pure bash solution which uses the last image sequence as your starting point (per your request):
#!/bin/bash
last_seq=$(ls image-* | tail -1 | sort -n | cut -c7-) # grab the sequence number
last_seq=${last_seq%.jpg} # remove the trailing .jpg
if [ -z $last_seq ] ; then
echo "Unable to obtain the last image sequence number"
exit 1
fi
for image in image-0*.jpg ; do
[ -f ${image} ] || break # In case no files match image-0*.jpg
let last_seq=last_seq+1
mv -v ${image} image-$(printf "%06d.jpg" ${last_seq})
done
Here's how it worked locally:
before the run:
~/tmp › ls -l
total 80
-rw-r--r-- 1 dyoung staff 0 Dec 12 10:20 image-00000.jpg
-rw-r--r-- 1 dyoung staff 0 Dec 12 10:20 image-01000.jpg
-rw-r--r-- 1 dyoung staff 0 Dec 12 10:05 image-14908.jpg
-rw-r--r-- 1 dyoung staff 0 Dec 12 10:05 image-14909.jpg
-rw-r--r-- 1 dyoung staff 0 Dec 12 10:05 image-14910.jpg
-rw-r--r-- 1 dyoung staff 418 Dec 12 10:24 testh.sh
during the run:
~/tmp › sh ./testh.sh
image-00000.jpg -> image-14911.jpg
image-01000.jpg -> image-14912.jpg
after the run:
~/tmp › ls -l
total 80
-rw-r--r-- 1 dyoung staff 0 Dec 12 10:05 image-14908.jpg
-rw-r--r-- 1 dyoung staff 0 Dec 12 10:05 image-14909.jpg
-rw-r--r-- 1 dyoung staff 0 Dec 12 10:05 image-14910.jpg
-rw-r--r-- 1 dyoung staff 0 Dec 12 10:20 image-14911.jpg
-rw-r--r-- 1 dyoung staff 0 Dec 12 10:20 image-14912.jpg
-rw-r--r-- 1 dyoung staff 429 Dec 12 12:25 testh.sh