I'm looking for help with a bash script to do some renaming of files for me. I don't know much about bash scripting, and what I have read is overwhelming. It's a lot to know/understand for the limited applications I will probably have.
In Dropbox, my media files are named something like:
Photo Jul 04, 5 49 44 PM.jpg
Video Jun 22, 11 21 00 AM.mov
I'd like them to be renamed in the following format: 2015-07-04 1749.ext
Some difficulties:
The script has to determine if AM or PM to put in the correct 24-hour format
The year is not specified; it is safe to assume the current year
The date, minute and second have a leading zero, but the hour does not; therefore the position after the hour is not absolute
Any assistance would be appreciated. FWIW, I'm running MacOS.
Mac OSX
This uses awk to reformat the date string:
for f in *.*
do
new=$(echo "$f" | awk -F'[ .]' '
BEGIN {
split("Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec",month)
for (i in month) {
nums[month[i]]=i
}
}
$(NF-1)=="PM" {$4+=12;}
{printf "%s 2015-%02i-%02i %02i%02i.%s",$1,nums[$2],$3,$4,$5,$8;}
')
mv "$f" "$new"
done
After the above was run, the files are now named:
$ ls -1 *.*
Photo 2015-07-04 1749.jpg
Video 2015-06-22 1121.mov
The above was tested on GNU awk but I don't believe that I have used any GNU-specific features.
GNU/Linux
GNU date has a handy feature for interpreting human-style date strings:
for f in *.*
do
prefix=${f%% *}
ext=${f##*.}
datestr=$(date -d "$(echo "$f" | sed 's/[^ ]* //; s/[.].*//; s/ /:/3; s/ /:/3; s/,//')" '+%F %H%M')
mv "$f" "$prefix $datestr.$ext"
done
Here is an example of the script in operation:
$ ls -1 *.*
Photo Jul 04, 5 49 44 PM.jpg
Video Jun 22, 11 21 00 AM.mov
$ bash script
$ ls -1 *.*
Photo 2015-07-04 1749.jpg
Video 2015-06-22 1121.mov
While not a simple parse and reformat for date, it isn't that difficult. The bash string tools of parameter expansion/substring removal are all you need to parse the pieces of the date into a format that date can use to output a new date string in the format for use in a filename. (see String Manipulation ) date -d is used to generate a new date string based on the contents of the original filename.
Note: the following presumes the dropbox filenames are in the format you have specified. (it doesn't care what the first part of the name or extension is as long as it matches the format you have specified) Here is an example of properly isolating the pieces of the filename needed to generate a date in the format specified)
Further, all spaces have been removed from the filename. While you originally showed a space between the day and hours, I will not provide an example of poor practice by inserting a space in a filename. As such, the spaces have been replaced with '_' and '-':
#!/bin/bash
# Photo Jul 04, 5 49 44 PM.jpg
# Video Jun 22, 11 21 00 AM.mov
# fn="Photo Jul 04, 5 49 44 PM.jpg"
fn="Video Jun 22, 11 21 00 AM.mov"
ext=${fn##*.} # determine extension
prefix=${fn%% *} # determine prefix (Photo or Video)
datestr=${fn%.${ext}} # remove extension from filename
datestr=${datestr#${prefix} } # remove prefix from datestr
day=${datestr%%,*} # isolate Month and date in day
ampm=${datestr##* } # isloate AM/PM in ampm
datestr=${datestr% ${ampm}} # remove ampm from datestr
timestr=${datestr##*, } # isolate time in timestr
timestr=$(tr ' ' ':' <<<"$timestr") # translate spaces to ':' using herestring
cmb="$day $timestr $hr" # create combined date/proper format
## create date/time string for filename
datetm=$(date -d "$cmb" '+%Y%m%d-%H%M')
newfn="${prefix}_${datetm}.${ext}"
## example moving of file to new name
# (assumes you handle the path correctly)
printf "mv '%s' %s\n" "$fn" "$newfn"
# mv "$fn" "$newfn" # uncomemnt to actually use
exit 0
Example/Output
$ bash dateinfname.sh
mv 'Video Jun 22, 11 21 00 AM.mov' Video_20150622-1121.mov
Related
I want to convert this bash command to shell script.
BASH
Input:
date --date="Wed Aug 25 22:37:44 +0900 2021" +"%s"
Output:
1629898664
SHELL
tmp.sh:
function time(a, b, c, d, e) { return date --date="a b c d +0900 e" +"%s" }
{print time($1, $2, $3, $4, $5}
timeline:
Wed Aug 25 22:37:44 2021
Command:
awk -f tmp.sh timeline
Output:
awk: tmp.sh:1: function cvtTime(w) { return date --date="Thu May 14 23:40:52 +0900 2020" +"%s" }
awk: tmp.sh:1: ^ syntax error
What about timeline file has multiple lines? Like:
Wed Aug 25 22:37:44 2021 JACK
Wed Aug 26 22:37:44 2021 EMILY
Wed Aug 27 22:37:44 2021 SAM
I tried:
#!/bin/bash
while read -r line; do
date --date="${1} ${2} ${3} ${4} +0900 ${5}" +"%s"
done
Want:
1629898664 JACK
1629985064 EMILY
1630071464 SAM
But it doesn't work :(
It seems that you want a shell script that is invoked with five command line parameters:
A weekday (in a three-letter format)
A month (in a three-letter format)
Day-of-month
A time expression (HH:MM:SS)
A year (four digits)
(Note that 1. is redundant, it is implied by 2., 3., and 5.)
Hence a somewhat minimal shell script would look sth. like this:
#!/bin/bash
date --date="${1} ${2} ${3} ${4} +0900 ${5}" +"%s"
Of course, this can be greatly improved, e.g., by adding sanity checks for the passed parameters.
In case you want to store the date information in a file so that you can pass a single filename parameter to the script instead (allowing for multiple such lines), the following variation will do:
#!/bin/bash
while read -a i; do
echo $(date --date="${i[0]} ${i[1]} ${i[2]} ${i[3]} +0900 ${i[4]}" +"%s") ${i[5]}
done < ${1}
Note, however, that this version expects an additional name parameter after the date information in each line.
In any event, no need for awk here.
I'm migrating old notes from a SQL database based note taking app to separate text files.
I've managed to export the notes and date codes as separate text files.
The files are ordered like this:
$ ls -1
Note0001.txt
Note0001-date.txt
Note0002.txt
Note0002-date.txt
Note0003.txt
Note0003-date.txt
The contents of the date files looks like this:
$ cat Note0001-date.txt
388766121.742373
$ cat Note0002-date.txt
274605766.273638
$ cat Note0003-date.txt
384996285.436197
The dates are seconds since the epoch 2001-01-01. See other question about the format: What type of date format is this? And how to convert it?.
How do I batch change the accessed and modified date of the notes files, NoteNNNN.txt, to the date in the contents of respective date file, NoteNNNN-date.txt?
How to convert the date to UTC+1? Preferably with consideration of DST (daylight saving time).
I am trying to convert the dates with the method described this question:
https://unix.stackexchange.com/questions/2987/
But it outputs an error message in bash 3.2.57 (macOS):
$ date -d '2001-01-01 UTC+1 + 388766121 seconds'
usage: date [-jnRu] [-d dst] [-r seconds] [-t west] [-v[+|-]val[ymwdHMS]] ...
[-f fmt date | [[[mm]dd]HH]MM[[cc]yy][.ss]] [+format]
I am new to working with the dates and timestamps in the terminal.
Iterate over each file pair, access the timestamp, shift the timestamp so it's something unix tools can understand, then touch files. Ie. big problems are composed of sum of small problems.
# find all files named .txt but not -date.txt
find . -name '*.txt' '!' -name '*-date.txt' |
# remove the .txt suffix
sed 's/\.txt$//' |
{
# the reference point of files content
start=$(date -d "2001-01-01" +%s) # will not work with BSD date
# I guess just precompute the value:
start=978303600
# for each file
while IFS= read -r f; do
# get the timestamp
diff=$(<"$f"-date.txt)
# increment the timestamp to seconds since epoch
ref=$(<<<"scale=6; $start + $diff" bc)
# TODO: use a tool convert the timestamp sinece epoch to BSD touch
# compatible format, ie. to ccyy-mm-ddTHH:MM:SS[.frac][Z]
ref=(TODO "$ref")
# change access and modification times of .txt file
touch -d "#$ref" "$f".txt
done
}
Assuming your OS local timezone is what you want for your output, and you have a version of awk that supports the GNU awk time functions, you could use the
following script. Also:
If the DST daylight-savings flag is positive, the time is assumed to
be daylight savings time; if zero, the time is assumed to be standard
time; and if negative (the default), mktime() attempts to determine
whether daylight savings time is in effect for the specified time.
file tst.awk:
BEGIN {
epoch = mktime("2001 01 01 00 00 00")
}
FNR==1 {
close(out)
out = substr(FILENAME, 1, length(FILENAME)-9) ".txt"
}
{
print strftime("%F %T %Z", epoch+$0) > out
}
Usage:
awk -f tst.awk *-date.txt
Example
Here is an example with the script, without the I/O part, just converting the datetimes.
test file:
> cat file
388766121.742373
274605766.273638
384996285.436197
script tst.awk:
BEGIN { epoch = mktime("2001 01 01 00 00 00") }
{ print strftime("%F %T %Z", epoch+$0) }
Output:
> awk -f tst.awk file
2013-04-27 15:35:21 EEST
2009-09-14 08:22:46 EEST
2013-03-14 23:24:45 EET
The timezone of my box is being used by default (EET). If we 'd like to print to a different timezone, we should define that and set the TZ. Also DST is used by default, notice that some days are printed as EEST (Summer Time).
I'm running into an issue where gawk prints unwanted output. I want to find lines in a file that match an expression, test to see if the information in the line matches a certain condition, and then print the line if it does. I'm getting the output that I want, but gawk is also printing every line that matches the expression rather than just the lines that meet the condition.
I'm trying to search through files containing dates and times for certain actions to be executed. I want to show only lines that contain times in the future. The dates are formatted like so:
text... 2016-01-22 10:03:41 more text...
I tried using sed to just print all lines starting with ones that had the current hour, but there is no guarantee that the file contains a line with that hour, (plus there is no guarantee that the lines all have any particular year, month, day etc.) so I needed something more robust. I decided trying to convert the times into seconds since epoch, and comparing that to the current systime. If the conversion produces a number greater than systime, I want to print that line.
Right now it seems like gawk's mktime() function is the key to this. Unfortunately, it requires input in the following format:
yyyy mm dd hh mm ss
I'm currently searching a test file (called timecomp) for a regular expression matching the date format.
Edit: the test file only contains a date and time on each line, no other text.
I used sed to replace the date separators (i.e. /, -, and :) with a space, and then piped the output to a gawk script called stime using the following statement:
sed -e 's/[-://_]/ /g' timecomp | gawk -f stime
Here is the script
# stime
BEGIN { tsec=systime(); } /.*20[1-9][0-9] [0-1][1-9] [0-3][0-9] [0-2][0-9][0-6][0-9] [0-6][0-9]/ {
if (tsec < mktime($0))
print "\t" $0 # the tab is just to differentiate the desired output from the other lines that are being printed.
} $1
Right now this is getting the basic information that I want, but it is also printing every like that matches the original expression, rather than just the lines containing a time in the future. Sample output:
2016 01 22 13 23 20
2016 01 22 14 56 57
2016 01 22 15 46 46
2016 01 22 16 32 30
2016 01 22 18 56 23
2016 01 22 18 56 23
2016 01 22 22 22 28
2016 01 22 22 22 28
2016 01 22 23 41 06
2016 01 22 23 41 06
2016 01 22 20 32 33
How can I print only the lines in the future?
Note: I'm doing this on a Mac, but I want it to be portable to Linux because I'm ultimately making this for some tasks I have to do at work.
I'd like trying to accomplish this in one script rather than requiring the sed statement to reformat the dates, but I'm running into other issues that probably require a different question, so I'm sticking to this for now.
Any help would be greatly appreciated! Thanks!
Answered: I had a $1 at the last line of my script, and that was the cause of the additional output.
Instead of awk, this is an (almost) pure Bash solution:
#!/bin/bash
# Regex for time string
re='[0-9]{4}-[0-9]{2}-[0-9]{2} ([0-9]{2}:){2}[0-9]{2}'
# Current time, in seconds since epoch
now=$(date +%s)
while IFS= read -r line; do
# Match time string
[[ $line =~ $re ]]
time_string="${BASH_REMATCH[0]}"
# Convert time string to seconds since epoch
time_secs=$(date -d "$time_string" +%s)
# If time is in the future, print line
if (( time_secs > now )); then
echo "$line"
fi
done < <(grep 'pattern' "$1")
This takes advantage of the Coreutils date formatting to convert a date to seconds since epoch for easy comparison of two dates:
$ date
Fri, Jan 22, 2016 11:23:59 PM
$ date +%s
1453523046
And the -d argument to take a string as input:
$ date -d '2016-01-22 10:03:41' +%s
1453475021
The script does the following:
Filter the input file with grep (for lines containing a generic pattern, but could be anything)
Loop over lines containing pattern
Match the line with a regex that matches the date/time string yyyy-mm-dd hh:mm:ss and extract the match
Convert the time string to seconds since epoch
Compare that value to the time in $now, which is the current date/time in seconds since epoch
If the time from the logfile is in the future, print the line
For an example input file like this one
text 2016-01-22 10:03:41 with time in the past
more text 2016-01-22 10:03:41 matching pattern but in the past
other text 2017-01-22 10:03:41 in the future matching pattern
some text 2017-01-23 10:03:41 in the future but not matching
blahblah 2022-02-22 22:22:22 pattern and also in the future
the result is
$ date
Fri, Jan 22, 2016 11:36:54 PM
$ ./future_time logfile
other text 2017-01-22 10:03:41 in the future matching pattern
blahblah 2022-02-22 22:22:22 pattern and also in the future
This is what I have working now. It works for a few different date formats and on the actual files that have more than just the date and time. The default format that it works for is yyyy/mm/dd, but it takes an argument to specify a mm/dd/yyyy format if needed.
BEGIN { tsec=systime(); dtstr=""; dt[1]="" } /.*[0-9][0-9]:[0-9][0-9]:[0-9][0-9]/ {
cur=$0
if ( fm=="mdy" ) {
match($0,/[0-1][1-9][-_\/][0-3][0-9][-_\/]20[1-9][0-9]/) # mm dd yyyy
section=substr($0,RSTART,RLENGTH)
split(section, dt, "[-_//]")
dtstr=dt[3] " " dt[1] " " dt[2]
gsub(/[0-1][1-9][-\/][0-3][0-9][-\/]20[1-9][0-9]/, dtstr, cur)
}
gsub(/[-_:/,]/, " ", cur)
match(cur,/20[1-9][0-9] [0-1][1-9] [0-3][0-9][[:space:] ]*[0-2][0-9] [0-6][0-9] [0-6][0-9]/)
arr=mktime(substr(cur,RSTART,RLENGTH))
if ( tsec < arr)
print $0
}
I'll be adding more format options as I find more formats, but this works for all the different files I've tested so far. If they have a mm/dd/yyyy format, you call it with:
gawk -f stime fm=mdy filename
I plan on adding an option to specify the time window that you want to see, but this is an excellent start. Thank you guys again, this is going to drastically simplify a few tasks at work ( I basically have to retrieve a great deal of data, often under time pressure depending on the situation ).
So basically i want to merge a couple of CSV files. Im using the following script to do that :
paste -d , *.csv > final.txt
However this has worked for me in the past but this time it doesn't work. It appends the data next to each other as opposed to below each other. For instance two files that contain records in the following format
CreatedAt ID
Mon Jul 07 20:43:47 +0000 2014 4.86249E+17
Mon Jul 07 19:58:29 +0000 2014 4.86238E+17
Mon Jul 07 19:42:33 +0000 2014 4.86234E+17
When merged give
CreatedAt ID CreatedAt ID
Mon Jul 07 20:43:47 +0000 2014 4.86249E+17 Mon Jul 07 18:25:53 +0000 2014 4.86215E+17
Mon Jul 07 19:58:29 +0000 2014 4.86238E+17 Mon Jul 07 17:19:18 +0000 2014 4.86198E+17
Mon Jul 07 19:42:33 +0000 2014 4.86234E+17 Mon Jul 07 15:45:13 +0000 2014 4.86174E+17
Mon Jul 07 15:34:13 +0000 2014 4.86176E+17
Would anyone know what the reason behind this is? Or what i can do to force merge below records?
Assuming that all the csv files have the same format and all start with the same header,
you can write a little script as the following to append all files in only one and to take only one time the header.
#!/bin/bash
OutFileName="X.csv" # Fix the output name
i=0 # Reset a counter
for filename in ./*.csv; do
if [ "$filename" != "$OutFileName" ] ; # Avoid recursion
then
if [[ $i -eq 0 ]] ; then
head -1 "$filename" > "$OutFileName" # Copy header if it is the first file
fi
tail -n +2 "$filename" >> "$OutFileName" # Append from the 2nd line each file
i=$(( $i + 1 )) # Increase the counter
fi
done
Notes:
The head -1 or head -n 1 command print the first line of a file (the head).
The tail -n +2 prints the tail of a file starting from the lines number 2 (+2)
Test [ ... ] is used to exclude the output file from the input list.
The output file is rewritten each time.
The command cat a.csv b.csv > X.csv can be simply used to append a.csv and b csv in a single file (but you copy 2 times the header).
The paste command pastes the files one on a side of the other. If a file has white spaces as lines you can obtain the output that you reported above.
The use of -d , asks to paste command to define fields separated by a comma ,, but this is not the case for the format of the files you reported above.
The cat command instead concatenates files and prints on the standard output, that means it writes one file after the other.
Refer to man head or man tail for the syntax of the single options (some version allows head -1 other instead head -n 1)...
Alternative simple answer, this as combine_csv.sh:
#!/bin/bash
{ head -n 1 $1 && tail -q -n +2 $*; }
can be used like this:
pattern="my*filenames*.csv"
combine_csv.sh ${pattern} > result.csv
Thank you so much #wahwahwah.
I used your script to make nautilus-action, but it work correctly only with this changes:
#!/bin/bash
for last; do true; done
OutFileName=$last/RESULT_`date +"%d-%m-%Y"`.csv # Fix the output name
i=0 # Reset a counter
for filename in "$last/"*".csv"; do
if [ "$filename" != "$OutFileName" ] ; # Avoid recursion
then
if [[ $i -eq 0 ]] ; then
head -1 "$filename" > "$OutFileName" # Copy header if it is the first file
fi
tail -n +2 "$filename" >> "$OutFileName" # Append from the 2nd line each file
i=$(( $i + 1 )) # Increase the counter
fi
done
I'm trying to store the files listing into an array and then loop through the array again.
Below is what I get when I run ls -ls command from the console.
total 40
36 -rwxrwxr-x 1 amit amit 36720 2012-03-31 12:19 1.txt
4 -rwxrwxr-x 1 amit amit 1318 2012-03-31 14:49 2.txt
The following bash script I've written to store the above data into a bash array.
i=0
ls -ls | while read line
do
array[ $i ]="$line"
(( i++ ))
done
But when I echo $array, I get nothing!
FYI, I run the script this way: ./bashscript.sh
I'd use
files=(*)
And then if you need data about the file, such as size, use the stat command on each file.
Try with:
#! /bin/bash
i=0
while read line
do
array[ $i ]="$line"
(( i++ ))
done < <(ls -ls)
echo ${array[1]}
In your version, the while runs in a subshell, the environment variables you modify in the loop are not visible outside it.
(Do keep in mind that parsing the output of ls is generally not a good idea at all.)
Here's a variant that lets you use a regex pattern for initial filtering, change the regex to be get the filtering you desire.
files=($(find -E . -type f -regex "^.*$"))
for item in ${files[*]}
do
printf " %s\n" $item
done
This might work for you:
OIFS=$IFS; IFS=$'\n'; array=($(ls -ls)); IFS=$OIFS; echo "${array[1]}"
Running any shell command inside $(...) will help to store the output in a variable. So using that we can convert the files to array with IFS.
IFS=' ' read -r -a array <<< $(ls /path/to/dir)
You may be tempted to use (*) but what if a directory contains the * character? It's very difficult to handle special characters in filenames correctly.
You can use ls -ls. However, it fails to handle newline characters.
# Store la -ls as an array
readarray -t files <<< $(ls -ls)
for (( i=1; i<${#files[#]}; i++ ))
{
# Convert current line to an array
line=(${files[$i]})
# Get the filename, joining it together any spaces
fileName=${line[#]:9}
echo $fileName
}
If all you want is the file name, then just use ls:
for fileName in $(ls); do
echo $fileName
done
See this article or this this post for more information about some of the difficulties of dealing with special characters in file names.
My two cents
The asker wanted to parse output of ls -ls
Below is what I get when I run ls -ls command from the console.
total 40
36 -rwxrwxr-x 1 amit amit 36720 2012-03-31 12:19 1.txt
4 -rwxrwxr-x 1 amit amit 1318 2012-03-31 14:49 2.txt
But there are few answer addressing this parsing operation.
ls's output
Before trying to parse something, we have to ensure command output is consistant, stable and easy to parse as possible
In order to ensure output wont be altered by some alias you may prefer to specify full path of command: /bin/ls.
Avoid variations of output due to locales, prefix your command by LANG=C LC_ALL=C
Use --time-style command switch to use UNIX EPOCH more easier to parse time infos.
Use -b switch for holding special characters
So we will prefer
LANG=C LC_ALL=C /bin/ls -lsb --time-style='+%s.%N'
to just
ls -ls
Full bash sample
#!/bin/bash
declare -a bydate=() bysize=() byname=() details=()
declare -i cnt=0 vtotblk=0 totblk
{
read -r _ totblk # ignore 1st line
while read -r blk perm lnk usr grp sze date file;do
byname[cnt]="${file//\\ / }"
details[cnt]="$blk $perm $lnk $usr $grp $sze $date"
bysize[sze]+="$cnt "
bydate[${date/.}]+="$cnt "
cnt+=1 vtotblk+=blk
done
} < <(LANG=C LC_ALL=C /bin/ls -lsb --time-style='+%s.%N')
From there, you could easily sort by dates, sizes of names (sorted by ls command).
echo "Path '$PWD': Total: $vtotblk, sorted by dates"
for dte in ${!bydate[#]};do
printf -v msec %.3f .${dte: -9}
for idx in ${bydate[dte]};do
read -r blk perm lnk usr grp sze date <<<"${details[idx]}"
printf ' %11d %(%a %d %b %T)T%s %s\n' \
$sze "${date%.*}" ${msec#0} "${byname[idx]}"
done
done
echo "Path '$PWD': Total: $vtotblk, sorted by sizes"
for sze in ${!bysize[#]};do
for idx in ${bysize[sze]};do
read -r blk perm lnk usr grp sze date <<<"${details[idx]}"
printf -v msec %.3f .${date#*.}
printf ' %11d %(%a %d %b %T)T%s %s\n' \
$sze "${date%.*}" ${msec#0} "${byname[idx]}"
done
done
echo "Path '$PWD': Total: $vtotblk, sorted by names"
for((idx=0;idx<cnt;idx++));{
read -r blk perm lnk usr grp sze date <<<"${details[idx]}"
printf -v msec %.3f .${date#*.}
printf ' %11d %(%a %d %b %T)T%s %s\n' \
$sze "${date%.*}" ${msec#0} "${byname[idx]}"
}
( Accessory, you could check if total block printed by ls match total block by lines:
(( vtotblk == totblk )) ||
echo "WARN: Total blocks: $totblk != Block count: $vtotblk" >&2
Of course, this could be inserted before first echo "Path...;)
Here is an output sample. (Note: there is a filename with a newline)
Path '/tmp/so': Total: 16, sorted by dates
0 Sun 04 Sep 10:09:18.221 2.txt
247 Mon 05 Sep 09:11:50.322 Filename with\nsp\303\251cials characters
13 Mon 05 Sep 10:12:24.859 1.txt
1313 Mon 05 Sep 11:01:00.855 parseLs.00
1913 Thu 08 Sep 08:20:20.836 parseLs
Path '/tmp/so': Total: 16, sorted by sizes
0 Sun 04 Sep 10:09:18.221 2.txt
13 Mon 05 Sep 10:12:24.859 1.txt
247 Mon 05 Sep 09:11:50.322 Filename with\nsp\303\251cials characters
1313 Mon 05 Sep 11:01:00.855 parseLs.00
1913 Thu 08 Sep 08:20:20.836 parseLs
Path '/tmp/so': Total: 16, sorted by names
13 Mon 05 Sep 10:12:24.859 1.txt
0 Sun 04 Sep 10:09:18.221 2.txt
247 Mon 05 Sep 09:11:50.322 Filename with\nsp\303\251cials characters
1913 Thu 08 Sep 08:20:20.836 parseLs
1313 Mon 05 Sep 11:01:00.855 parseLs.00
And if you want to format characters (with care: there could be some issues, if you don't know who create content of path). But if folder is your, you could:
echo "Path '$PWD': Total: $vtotblk, sorted by dates, with special chars"
printf -v spaces '%*s' 37 ''
for dte in ${!bydate[#]};do
printf -v msec %.3f .${dte: -9}
for idx in ${bydate[dte]};do
read -r blk perm lnk usr grp sze date <<<"${details[idx]}"
printf ' %11d %(%a %d %b %T)T%s %b\n' $sze \
"${date%.*}" ${msec#0} "${byname[idx]//\\n/\\n$spaces}"
done
done
Could output:
Path '/tmp/so': Total: 16, sorted by dates, with special chars
0 Sun 04 Sep 10:09:18.221 2.txt
247 Mon 05 Sep 09:11:50.322 Filename with
spécials characters
13 Mon 05 Sep 10:12:24.859 1.txt
1313 Mon 05 Sep 11:01:00.855 parseLs.00
1913 Thu 08 Sep 08:20:20.836 parseLs
Isn't these 2 code lines, either using scandir or including the dir pull in the declaration line, supposed to work?
src_dir="/3T/data/MySQL";
# src_ray=scandir($src_dir);
declare -a src_ray ${src_dir/*.sql}
printf ( $src_ray );
In the conversation over at https://stackoverflow.com/a/9954738/11944425
the behavior can be wrapped into a convenience function which applies some action to entries of the directory as string values.
#!/bin/bash
iterfiles() {
i=0
while read filename
do
files[ $i ]="$filename"
(( i++ ))
done < <( ls -l )
for (( idx=0 ; idx<${#files[#]} ; idx++ ))
do
$# "${files[$idx]}" &
wait $!
done
}
where $# is the complete glob of arguments passed to the function! This lets the function have the utility to take an arbitrary command as a partial function of sorts to operate on the filename:
iterfiles head -n 1 | tee -a header_check.out
When a script needs to iterate over files, returning an array of them is not possible. The workaround is to define the array outside of the function scope (and possibly unset it later) — modifying it inside the function's scope. Then, after the function is called by a script, the array variable becomes available. For instance, the mutation on files demonstrates how this could be done.
declare -a files # or just `files= ` (nothing)
iterfiles() {
# ...
files=...
}
Extending the conversation above, #Jean-BaptistePoittevin pointed out a valuable detail.
#!/bin/bash
# Adding a section to unset certain variable names that
# may already be active in the shell.
unset i
unset files
unset omit
i=0
omit='^([\n]+)$'
while read file
do
files[ $i ]="$file"
(( i++ ))
done < <(ls -l | grep -Pov ${omit} )
Note: This can be tested using echo ${files[0]} or for entry in ${files[#]}; do ... ; done
Often times, the circumstance could require an absolute path in double quotes, where the file (or ancestor directories) have spaces or unusual characters in the name. find is one answer here. The simplest usage might look like the above one, except done < <(ls -l ... ) is replaced with:
done < <(find /path/to/directory ! -path /path/to/directory -type d)
Its convenient when you need absolute paths in double quotes as an iterable collection to use a recipe like the one below. When export is not used, the shell does not update the environment namespace to include it in the find subshell:
#!/bin/bash
export DIRECTORY="$PWD" # For example
declare -a files
i=0
while read filename; do
files[ $i ]="$filename"
done < <(find $DIRECTORY ! -path $DIRECTORY -type d)
for (( idx=0; idx<${#files[#]}; idx++ )); do
# Make a templated string for macro script generation
quoted_path="\"${files[$idx]}\""
if [[ "$(echo $quoted_path | grep some_substring | wc -c)" != "0" ]]; then
echo "mv $quoted_path /some/other/watched/folder/" >> run_nightly.sh
fi
done
Upon running this, ./run_nightly.sh will be populated with bulk commands to move a quoted path to /some/other/watched/folder/. This kind of scripting pattern will make it possible to supercharge your scripts.
simply you can use this below for loop (do not forget to quote to handle filenames with spaces)
declare -a arr
arr=()
for file in "*.txt"
do
arr=(${arr[*]} "$file")
done
Run
for file in ${arr[*]}
do
echo "<$file>"
done
to test.