How to grave the Date from this command in bash script - bash

!/bin/bash
# When a match is not found, just present nothing.
shopt -s nullglob
# Match all .wav files containing the date format.
files=(*[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]*.wav)
if [[ ${#files[#]} -eq 0 ]]; then
echo "No match found."
fi
for file in "${files[#]}"; do
# We get the date part by part
file_date=''
# Sleep it to parts.
IFS="-." read -ra parts <<< "$file"
for t in "${parts[#]}"; do
# Break from the loop if a match is found
if [[ $t == [0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] ]]; then
file_date=$t
break
fi
done
# If a value was not assigned, then show an error message and continue to the next file.
# Just making sure there is nothing in Array and date before it moves on
if [[ -z $file_date ]]; then
continue
fi
file_year=${file_date:0:4}
file_month=${file_date:4:2}
mkdir -p "$file_year/$file_month"
# -- is just there to not interpret filenames starting with - as options.
echo "Moving: ./"$file "to: " "./"$file_year"/"$file_month
mv "$file" "$file_year/$file_month"
done
Now there are files I would need to do the date to gra the date and then move it like I do now. for example there is a file called meetme.. its a wav file and I have DIR with YYYY/MM and would like to move thoses files without YYYYMMDD in file name already

If you're writing a program to do something with that information, then you might prefer seconds-since-epoch and use date to get the date in the desired format.
$ date -d #$(stat --format='%Y' testdisk.log) +%Y%m%d
20130422
You can also get the ascii representation, and then manipulate the string.
$ stat --format='%y' testdisk.log
2013-04-22 09:11:39.000000000 -0500
$ date_st=$(stat --format='%y' testdisk.log)
$ date_st=${date_st/ */}
$ date_st=${date_st//-/}
$ echo ${date_st}
20130422

Related

How can I zero pad first instance of a number while preserving the rest of the file name with bash

I have been messing with this for a while and I am stuck. I was spoiled on my laptop because I could use the perl rename and fixing number padding is simple. Now I am on a server and rename is not working. I need a way of adjusting the patting on a file that looks like the following
foo-353-03-53-23.txt
most the examples on the this website and other will destroy all the content in the padding process which is not helpful to me as I have metadata in the filename.
I am looking for the result to produce the following
foo-000353-03-53-23.txt
Everything is preserved just the first number is padded.
Please help preserve my file names and sanity.
Thanks in advance.
Split string and reformat:
#!/usr/bin/env bash
oldname='foo-353-03-53-23.txt'
IFS=- read -r -d '' a n b <<<"$oldname"
printf -v newname '%s-%06d-%s' "$a" "$((10#$n))" "${b%?}"
# Debug dump variables
declare -p oldname newname
Alternate method using Bash's Regex to capture string elmeents:
#!/usr/bin/env bash
oldname='foo-353-03-53-23.txt'
# Capture string elements with Regex
[[ $oldname =~ ([^0-9]+)([0-9]+)(.*) ]]
# Reformat string elements
printf -v newname '%s%06d%s' \
"${BASH_REMATCH[1]}" "$((10#${BASH_REMATCH[2]}))" "${BASH_REMATCH[3]}"
if [ ! -e "$newname" ]; then
echo mv --no-clobber -- "$oldname" "$newname"
else
printf 'Cannot rename %s to %s, because %s already exist!\n' \
"$oldname" "$newname" "$newname" >&2
fi

List the files in Directory and Copy-Replace them into another Directory in Linux

I am trying to automate the below: Any help, please.
We have 2 directories as mentioned below, whenever we get new files in Directory-1, only they should be copied and replaced into Directory-2. How to achieve this in Linux scripting. Filename remains the same but the version will be different.
Directory-1:
FileOne_2.0.0.txt
FileTwo_3.0.0.txt
Directory-2:
FileOne_1.0.0.txt
FileTwo_2.0.0.txt
FileThree_3.0.0.txt
FileFive_5.0.0.txt
Try this code (on a test setup before you trust your real directories and files with it):
#! /bin/bash -p
shopt -s extglob # Enable extended globbing ( +([0-9]) ... )
shopt -s nullglob # Globs that match nothing expand to nothing
shopt -s dotglob # Globs match files with names starting with '.'
srcdir='Directory-1'
destdir='Directory-2'
# A(n extended) glob pattern to match a version string (e.g. '543.21.0')
readonly kVERGLOB='+([0-9]).+([0-9]).+([0-9])'
# shellcheck disable=SC2231 # (Bad warning re. unquoted ${kVERGLOB})
for srcpath in "$srcdir"/*_${kVERGLOB}.txt; do
srcfile=${srcpath##*/} # E.g. 'FileOne_2.0.0.txt'
srcbase=${srcfile%_*} # E.g. 'FileOne'
# Set and check the path that the file will be moved to
destpath=$destdir/$srcfile
if [[ -e $destpath ]]; then
printf "Warning: '%s' already exists. Skipping '%s'.\\n" \
"$destpath" "$srcpath" >&2
continue
fi
# Make a list of the old versions of the file
# shellcheck disable=SC2206 # (Bad warning re. unquoted ${kVERGLOB})
old_destpaths=( "$destdir/$srcbase"_${kVERGLOB}.txt )
# TODO: Add checks that the number of old files (${#old_destpaths[*]})
# is what is expected (exactly one?)
# Move the file
if mv -i -- "$srcpath" "$destpath"; then
printf "Moved '%s' to '%s'\\n" "$srcpath" "$destpath" >&2
else
printf "Warning: Failed to move '%s' to '%s'. Skipping '%s'.\\n" \
"$srcpath" "$destpath" "$srcpath" >&2
continue
fi
# Remove the old version(s) of the file (if any)
for oldpath in "${old_destpaths[#]}"; do
if rm -- "$oldpath"; then
printf "Removed '%s'\\n" "$oldpath" >&2
else
printf "Warning: Failed to remove '%s'.\\n" "$oldpath" >&2
fi
done
done
The code is Shellcheck-clean. Two Shellcheck suppression comments are used because the unquoted expansions are necessary here.
srcdir and destdir are set to constant values. You might want to take them from command line parameters, or set them to different constant values.
The code could be made shorter by removing checks. However, moves and removes are destructive operations that can do a lot of damage if they are done incorrectly. I'd add even more checks if it was my own data.
See glob - Greg's Wiki for an explanation of the "extended globbing" used in the code.
See Parameter expansion [Bash Hackers Wiki] for an explanation of ${srcpath##*/} and ${srcfile%_*}.
mv -i is used as a double protection against overwriting an existing file.
All external commands are invoked with -- to explicitly end options, in case they are ever used with paths that begin with -.
Make sure that you understand the code and test it VERY carefully before using it for real.
source_dir=./files/0
dest_dir=./files/1/
for file in $source_dir/*
do
echo $file
echo "processing"
if [[ "1" == "1" ]]; then
mv $file $dest_dir
fi
done
Where processing and the 1 == 1 is whatever your 'prechecks' are (which you haven't told us)
If your coreutils sort is newer than or equal to v7.0 (2008-10-5) after which sort command
supports -V option (version-sort), would you please try:
declare -A base2ver base2file
# compare versions
# returns 0 if $1 equals to $2
# 1 if $1 is newer than $2
# -1 if $1 is older than $2
vercomp() {
if [[ $1 = $2 ]]; then
echo 0
else
newer=$(echo -e "$1\n$2" | sort -Vr | head -n 1)
if [[ $newer = $1 ]]; then
echo 1
else
echo -1
fi
fi
}
for f in Directory-1/*.txt; do
basename=${f##*/}
version=${basename##*_}
version=${version%.txt} # version number such as "2.0.0"
basename=${basename%_*} # basename such as "FileOne"
base2ver[$basename]=$version # associates basename with version number
base2file[$basename]=$f # associates basename with full filename
done
for f in Directory-2/*.txt; do
basename=${f##*/}
version=${basename##*_}
version=${version%.txt}
basename=${basename%_*}
if [[ -n ${base2ver[$basename]} ]] && (( $(vercomp "${base2ver[$basename]}" "$version") > 0 )); then
# echo "${base2file[$basename]} is newer than $f"
rm -- "$f"
cp -p -- "${base2file[$basename]}" Directory-2
fi
done

Why is "ls -1 $fl | wc -l" not returning value 0 in my for loop?

I am trying to add a condition in a for loop to check the existence of a file as well as check for file size > 0 KB.
Period file contains monthly data:
20180101
20180201
20180301
20180401
20180501
There are individual files created for each month. Suppose a file is not created for one month, (20180201), then the loop below should terminate.
For example:
xxx_20180101.txt
xxx_20180301.txt
xxx_20180401.txt
xxx_20180501.txt
if [[ $STATUS -eq 0 ]]; then
for per in `cat ${PATH}/${PERIOD}.txt | cut -f 1 -d";"`
do
for fl in `ls -1 ${PATH}/${FILE} | grep ${per}`
do
if [[ `ls -1 $fl | wc -l` -eq 0 ]]; then
echo "File not found"
STATUS=1
else
if [[ -s "$fl" ]]; then
echo "$fl contain data.">>/dev/null
else
echo "$fl File size is 0KB"
STATUS=1
fi
fi
done
done
fi
but ls -1 $fl | wc -l is not returning 0 value when the if condition is executed.
The following is a demonstration of what a best-practices rewrite might look like.
Note:
We do not (indeed, must not) use a variable named PATH to store a directory under which we look for data files; doing this overwrites the PATH environment variable used to find programs to execute.
ls is not used anywhere; it is a tool intended to generate output for human consumption, not machines.
Reading through input is accomplished with a while read loop; see BashFAQ #1 for more details. Note that the input source for the loop is established at the very end; see the redirection after the done.
Finding file sizes is done with stat -c here; for more options, portable to platforms where stat -c is not supported, see BashFAQ #87.
Because your filename format is well-formed (with an underscore before the substring from your input file, and a .txt after that substring), we're refining the glob to look only for names matching that restriction. This prevents a search for 001 to find xxx_0015.txt, xxx_5001.txt, etc. as well.
#!/usr/bin/env bash
# ^^^^ -- NOT /bin/sh; this lets us use bash-only syntax
path=/provided/by/your/code # replacing buggy use of PATH in original code
period=likewise # replacing use of PERIOD in original code
shopt -s nullglob # generate a zero-length list for unmatched globs
while IFS=';' read -r per _; do
# populate an array with a list of files with names containing $per
files=( "$path/$period/"*"_${per}.txt" )
# if there aren't any, log a message and proceed
if (( ${#files[#]} == 0 )); then
echo "No files with $per found in $path/$period" >&2
continue
fi
# if they *do* exist, loop over them.
for file in "${files[#]}"; do
if [[ -s "$file" ]]; then
echo "$file contains data" >&2
if (( $(stat -c +%s -- "$file") >= 1024 )); then
echo "$file contains 1kb of data or more" >&2
else
echo "$file is not empty, but is smaller than 1kb" >&2
fi
else
echo "$file is empty" >&2
fi
done
done < "$path/$period.txt"
Here's a refactoring of Mikhail's answer with the standard http://shellcheck.net/ warnings ironed out. I have not been able to understand the actual question well enough to guess whether this actually solves the OP's problem.
while IFS='' read -r per; do
if [ -e "xxx_$per.txt" ]; then
echo "xxx_$per.txt found" >&2
else
echo "xxx_$per.txt not found" >&2
fi
done <periods.txt
You are over engineering here. Just iterate over content of file with periods and search each period in a list of files. Like this:
for per in `cat periods.txt`
do
if ls | grep -q "$per"; then
echo "$per found";
else
echo "$per not found"
fi
done

bash script not filtering

I'm hoping this is a simple question, since I've never done shell scripting before. I'm trying to filter certain files out of a list of results. While the script executes and prints out a list of files, it's not filtering out the ones I don't want. Thanks for any help you can provide!
#!/bin/bash
# Purpose: Identify all *md files in H2 repo where there is no audit date
#
#
#
# Example call: no_audits.sh
#
# If that call doesn't work, try ./no_audits.sh
#
# NOTE: Script assumes you are executing from within the scripts directory of
# your local H2 git repo.
#
# Process:
# 1) Go to H2 repo content directory (assumption is you are in the scripts dir)
# 2) Use for loop to go through all *md files in each content sub dir
# and list all file names and directories where audit date is null
#
#set counter
count=0
# Go to content directory and loop through all 'md' files in sub dirs
cd ../content
FILES=`find . -type f -name '*md' -print`
for f in $FILES
do
if [[ $f == "*all*" ]] || [[ $f == "*index*" ]] ;
then
# code to skip
echo " Skipping file: " $f
continue
else
# find audit_date in file metadata
adate=`grep audit_date $f`
# separate actual dates from rest of the grepped line
aadate=`echo $adate | awk -F\' '{print $2}'`
# if create date is null - proceed
if [[ -z "$aadate" ]] ;
then
# print a list of all files without audit dates
echo "Audit date: " $aadate " " $f;
count=$((count+1));
fi
fi
done
echo $count " files without audit dates "
First, to address the immediate issue:
[[ $f == "*all*" ]]
is only true if the exact contents of f is the string *all* -- with the wildcards as literal characters. If you want to check for a substring, then the asterisks shouldn't be quoted:
[[ $f = *all* ]]
...is a better-practice solution. (Note the use of = rather than == -- this isn't essential, but is a good habit to be in, as the POSIX test command is only specified to permit = as a string comparison operator; if one writes [ "$f" == foo ] by habit, one can get unexpected failures on platforms with a strictly compliant /bin/sh).
That said, a ground-up implementation of this script intended to follow best practices might look more like the following:
#!/usr/bin/env bash
count=0
while IFS= read -r -d '' filename; do
aadate=$(awk -F"'" '/audit_date/ { print $2; exit; }' <"$filename")
if [[ -z $aadate ]]; then
(( ++count ))
printf 'File %q has no audit date\n' "$filename"
else
printf 'File %q has audit date %s\n' "$filename" "$aadate"
fi
done < <(find . -not '(' -name '*all*' -o -name '*index*' ')' -type f -name '*md' -print0)
echo "Found $count files without audit dates" >&2
Note:
An arbitrary list of filenames cannot be stored in a single bash string (because all characters that might otherwise be used to determine where the first name ends and the next name begins could be present in the name itself). Instead, read one NUL-delimited filename at a time -- emitted with find -print0, read with IFS= read -r -d ''; this is discussed in [BashFAQ #1].
Filtering out unwanted names can be done internal to find.
There's no need to preprocess input to awk using grep, as awk is capable of searching through input files itself.
< <(...) is used to avoid the behavior in BashFAQ #24, wherein content piped to a while loop causes variables set or modified within that loop to become unavailable after its exit.
printf '...%q...\n' "$name" is safer than echo "...$name..." when handling unknown filenames, as printf will emit printable content that accurately represents those names even if they contain unprintable characters or characters which, when emitted directly to a terminal, act to modify that terminal's configuration.
Nevermind, I found the answer here:
bash script to check file name begins with expected string
I tried various versions of the wildcard/filename and ended up with:
if [[ "$f" == *all.md ]] || [[ "$f" == *index.md ]] ;
The link above said not to put those in quotes, and removing the quotes did the trick!

Get the substring from a filename and compare to Hour (HH) of current date/timestamp using Shell Script

Say I have a filename ABC.20131212.XX.xml where XX is HH (hour). I need to get the value of XX and compare it to the current hour of the system time. If it's equal, then, i'll rename that file. So if it's 12pm (12:00), I should get the file with ABC.20131212.12.xml and rename it. How can I achieve the comparison in shell script?
Here's how to get started in pure bash without starting external commands:
a=ABC.20131212.XX.xml
# Strip off .xml
b=${a%.xml}
# Extract last 2 characters
xx=${b: -2}
and the hour can be got with
time=`date +'%H'`
Comparison and rename can be done like this
if [[ $time -eq $xx ]]
then
mv something somewhere
fi
Here's how you can do it using Bash built-in regex matching:
file=foobar.11.xml
if [[ $file =~ .*([0-9][0-9])\.xml ]]; then
if [[ "${BASH_REMATCH[1]}" -eq $(date "+%I") ]]; then
# $file has current hour in its name
fi
fi
Use the +%H option with date for comparing to a 24-hour time system.
Here is the Script you can use(Change path accordingly):
`
!/bin/bash
Path=/Your/Path/Here/*
for l in $Path
do
a=$l
b=${a%.xml}
xx=${b: -2}
time=date +"%I"
if [[ $time -eq $xx ]]
then
mv $l /New/file/name/path/newfile.txt
fi
done
exit 0`

Resources