I want to count all files that I have in my directory and put number in front of them, and in a new line, for example :
file.txt nextfile.txt example.txt
and the output to be :
1.file.txt
2.nextfile.txt
3.example.txt
and so on.
i am trying something with : ls -L |
You can do this if you have nl installed:
ls -1 | nl
(Note with modern shells (ls usually a built-in) the -1 part is not needed. And this applies to the below solutions too.)
Or with awk:
ls -1 | awk '{print NR, $0}'
Or with a single awk command:
awk '{c=1 ; for (f in ARGV) {print c, f ; c++ } }' *
Or with cat:
cat -n <(ls -1)
You can do this by using shell built-in printf in a for loop:
n=0
for i in *; do
printf "%d.%s\n" $((n++)) "$i"
done
Related
I have many files on a server which contains many lines:
201701010530.contentState.csv.gz
201701020530.contentState.csv.gz
201701030530.contentState.csv.gz
201701040530.contentState.csv.gz
I would like with one line command this result:
170033|20170101
169865|20170102
170010|20170103
170715|20170104
The goal is to have the number of lines of each file, just by keeping the date which is already in the filename of the file.
I tried this but the result is not in one line but two...
for f in $(ls -1 2017*gz);do zcat $f | wc -l;echo $f | awk '{print substr($0,1,8)}';done
Thanks in advance guys.
Just use zcat file | wc -l to get the number of lines.
For the name, I understand it is enough to extract the first 8 characters:
$ t="201701030530.contentState.csv.gz"
$ echo "${t:0:8}"
20170103
All together:
for file in 2017*gz;
do
lines=$(zcat "$file" | wc -l)
printf "%s|%s\n" "$lines" "${file:0:8}"
done > myresult.csv
Note the usage of for file in 2017*gz; to go through the files matching the 2017*gz pattern: this suffices, no need to parse ls!
Use zgrep -c ^ file to count the lines, here encapsulated in awk:
$ awk 'FNR==1{ "zgrep -c ^ " FILENAME | getline s; print s "|" substr(FILENAME,1,8) }' *.gz
12|20170101
The whole "zgrep -c ^ " FILENAME should probably be in a var (s) and then s | getline s.
I have some directory e.g. /ice/cream which contains some files that I want to want to sort in size, and then find a minimum value in the largest file; however I want to do this from the parent directory /ice.
The bash line I wrote only works within /ice/cream, i'd like to make it work from /ice, I tried
awk 'BEGIN {min = 0} {if($7<min) min=$7} END {print min}' $(ls -lS cream/ | head -n 2 | awk '{print $9}')
which does not work because awk doesnt know the path to the file found by the second $() function; please help! Cheers
A safer way to get the largest file; the call to stat may differ depending on your implementation:
max_file () {
local max_size size
max_size=0
for f in "$1"/*; do
size=$(stat -c %s "$f")
if (( size > max_size )); then
max_file="$f"
max_size="$size"
fi
done
echo "$max_file"
}
awk '...' "$(biggest_file cream/)"
Your ls pipeline is way too complicated, and you need a * after the dir/ to get the relative name output:
awk 'BEGIN {min = 0} {if($7<min) min=$7} END {print min}' $(ls -S cream/* | head -1)
As first answered by #Etan Reisner in comment, the line was missing a *; working code is:
awk 'BEGIN {min = 0} {if($7<min) min=$7} END {print min}' $(ls -lS cream/* | head -n 1 | awk '{print $9}')
Thank you.
I have a list of files and want to sum over their file sizes.
So, I created a (global) variable as counter and are trying to loop over that list, get the file size with ls and cut&add it with
export COUNTER=1
for x in $(cat ./myfiles.lst); do ls -all $x | awk '{COUNTER+=$5}'; done
However, my counter is empty?
> echo $COUNTER
> 1
Does someone has an idea for my, what I am missing here?
Cheers and thanks,
Thomas
OK, I found a way piping the result from the awk pipe into a variable
(which is probably not elegant but working ;) )
for x in $(cat ./myfiles.lst); do a=$(ls -all $x |awk '{print $5}'); COUNTER=$(($COUNTER+$a)) ; done
> echo $COUNTER
> 4793061514
awk is getting called for every file, so in COUNTER you got the last file's size.
A better solution is:
ls -all <myfiles.lst | awk '{COUNTER+=$5} END {print COUNTER}'
But you are reinventing the wheel here. You can do something like
du -s <myfiles.lst
(If you have du installed. Note: see the comments below my answer about du. I had tested this with cygwin and with that it worked like a charm.)
Shorter version of the last:
ls -l | awk '{sum += $5} END {print sum}'
Now, say you want to filter by certain types of files, age, etc... Just throw the ls -l into a find, and you can filter using find's extensive filter parameters:
find . -type f -exec ls -l {} \; | awk '{sum += $5} END {print sum}'
ls -ltS | awk -F " " {'print $5'} | awk '{s+=$1} END {print s}'
At the moment here is what im doing
ret=$(ls -la | awk '{print $3 " " $9}')
usr=$(echo $ret | awk '{print $1}')
fil=$(echo $ret | awk '{print $2}')
The problem is that im not running an ls im running a command that takes time, so you can understand the logic.
Is there a way I can set the return value to set two external values, so something such as
ls -la | awk -r usr=x -r fil=y '{x=$3; y=$9}'
This way the command will be run once and i can minimize it to one line
It's not pretty, but if you really need to do this in one line you can use awk/bash's advanced meta-programming capabilities :)
eval $(ls -la | awk '{usr = $3 " " usr;fil = $9 " " fil} END{print "usr=\""usr"\";fil=\""fil"\""}')
To print:
echo -e $usr
echo -e $fil
Personally, I'd stick with what you have - it's much more readable and performance overhead is tiny compared to the above:
$time <three line approach>
real 0m0.017s
user 0m0.006s
sys 0m0.011s
$time <one line approach>
real 0m0.009s
user 0m0.004s
sys 0m0.007s
A workaround using read
usr=""
fil=""
while read u f; do usr="$usr\n$u"; fil="$fil\n$f"; done < <(ls -la | awk '{print $3 " " $9}')
For performance issue you could use <<<, but avoid it if the returned text is large:
while read u f; do usr="$usr\n$u"; fil="$fil\n$f"; done <<< $(ls -la | awk '{print $3 " " $9}')
A more portable way inspired from #WilliamPursell's answer:
$ usr=""
$ fil=""
$ while read u f; do usr="$usr\n$u"; fil="$fil\n$f"; done << EOF
> $(ls -la | awk '{print $3 " " $9}')
> EOF
What you want to do is capture the output of ls or any other command and then process it later.
ls=$(ls -l)
first=$(echo $ls | awk '{print $1}')
second=$(echo $ls | awk '{print $2}')
Using bash v4 associative array:
unset FILES
declare -A FILES
FILES=( ls -la | awk '{print $9 " " $3}' )
Print the list of owner & file:
for fil in ${!FILES[#]}
do
usr=${FILES["$fil"]}
echo -e "$usr" "\t" "$fil"
done
My apologies, I cannot test on my computer because my bash v3.2 does not support associative array :-(.
Please, report any issue...
The accepted answer uses process substitution, which is a bashism that only works on certain platforms. A more portable solution is to use a heredoc:
read u f << EOF
$( ls ... )
EOF
It is tempting to try:
ls ... | read u f
but the read then runs in a subshell. A common technique is:
ls ... | { read u f; # use $u and $f here; }
but to make the variables available in the remainder of the script, the interpolated heredoc is the most portable approach. Note that it requires the shell to read all of the output of the program into memory, so is not suitable if the output is expected to be large or if the process is long running.
You could use a bash array or the positional parameters as temporary holding place:
ret_ary=( $(command | awk '{print $3, $9}') )
usr=${ret_ary[0]}
fil=${ret_ary[1]}
set -- $(command | awk '{print $3, $9}')
usr=$1
fil=$2
Let's say I have this list of files:
a.1
a.2
a.3
b.1
b.2
Is there a bash one-liner that could find the one 'a' file for which there is no 'b' with the same extension? (i.e. a.3: no match)
I'm sure I could write a short bash/perl script to do this.
But I would like to know if there is any "trick" for this (of course, GNU tools are at my disposal; awk, sed, find ...)
You could try this:
ls -1 [ab].* | sort -t . -k 2 | uniq -u -s2
Here is my bash one-liner
for afile in a*; do bfile=${afile/a/b}; test -f $bfile || echo $afile; done
I like the above solution since it only uses bash and no other tools. However, to showcase the power of Unix tools, the next solution uses 4: ls, sed, sort, and uniq:
ls [ab]* | sed 's/b/a/' | sort | uniq -u
If you can use Perl:
perl -le 'for (<a*>) { /(.*)\.(.*)/; print "$1.$2" if !-e "b.$2"}'
bash version 4 has associative arrays, so you can do this:
declare -A a_files
while read -r filename; do
ext="${filename##*.}"
case "${filename%.*}" in
a) a_files[$ext]="$filename" ;;
b) unset a_files[$ext] ;;
esac
done < <(ls [ab].*)
echo "non-matched 'a' files: ${a_files[#]}"
Or, with awk:
ls [ab].* | awk -F. '
$1 == "a" {a_files[$2] = $0}
$1 == "b" {delete a_files[$2]}
END {for (ext in a_files) print a_files[ext]}
'
Ruby(1.9+)
$ ruby -e 'Dir["a.*"].each{|x|puts x if not File.exist? "b."+x.scan(/a\.(\d+)/)[0][0]}'