I have some operations to do on files last modified on a specific date. I would like to get the date, stock it in a string, then split it to test if the day corresponds to what I want.
So far, I've been trying things like that:
#!/bin/bash
for i in {45..236}; do
nom=M$i
chem=/Users/nfs/helene/soft/metAMOS-1.5rc3/$nom.fastq/Assemble/out
if [ -e $chem ]; then
IN= $(date -r $chem)
arr=(${IN//\ / })
if [[ ${arr[1]} == 'juin' && ${arr[2]} == '10' ]]; then
echo $nom
#cp $chem/proba.faa /Users/nfs/helene/metagenomes/DB/$nom.faa
fi
fi
done
exit 0
But it seems like the date isn't well stocked in $IN, and I'm not sure about the space-spliting either..
Perhaps the simple mistake is that you didn't place your assignment adjacent to =. It must be:
IN=$(date -r $chem)
And here's a simplified suggestion:
#!/bin/bash
for i in {45..236}; do
nom="M${i}"
chem="/Users/nfs/helene/soft/metAMOS-1.5rc3/${nom}.fastq/Assemble/out"
if [[ -e $chem ]]; then
read month day < <(exec date -r "$chem" '+%b %d')
if [[ $month == 'Jun' && $day == 10 ]]; then
echo "$nom"
# cp "$chem/proba.faa" "/Users/nfs/helene/metagenomes/DB/$nom.faa"
fi
fi
done
exit 0
* See date --help for a list of formats.
* <() is a form of Process Substitution. Check Bash's manual for it.
* Always place your arguments around double quotes when they have variables to avoid word splitting.
Related
I need to write a while loop to check for a file existence.
My requirement is: check for the file only for 5 minutes. If file come in that path within 5 minutes exit the loop and continue rest of the script otherwise exit from the script after 5 minutes with an error 'file not found'. I wrote the code like this :
SOURCEFILE=/path/*file.csv
StartTime=$(date +'%s')
TimeSpan=300
EndTime=$((StartTime + TimeSpan))
while [[ ! -f ${SOURCEFILE} && $(date +'%s') < ${EndTime} ]]
do
echo "inside loop"
sleep 25
done
echo "outside loop"
But with this while loop, even if the file is present in the mentioned path, it is going inside loop and will exit only after 300 seconds. I am beginner in shell scripting and I am not able to understand the issue. I am using ksh.
I could also tell you that it works find with while [ ! -f {SOURCEFILE} ] only. But whenever I add any && condition to while loop , then the -f is not working properly.
The SOURCEFILE=/path/*file.csv is wrong in your case. It can't be evaluated right with the -f flag.
An easy solution would be to use find or ls and count the result:
find /path/ -name "*file.csv" -type f
# then count the result...
Now I think there is a logic issue with the operators precedence. To force evaluation of the ! for the -f only, use parenthesis. Here is what works for me, and you must adapt it a little to match the * before file.csv:
while [[ ( ! -f file.csv ) && $(date +'%s') < ${EndTime} ]]
do
echo "inside loop"
sleep 25
...
There are some more explanation on this answer. The "and" operator precedes the "not", that's why you had the issue.
Your primary issue is getting the asterisk (*) to expand at the 'right time'.
It doesn't help that the [ ] and [[ ]] constructs behave differently, especially when it comes to if/when to expand that asterisk. [You can peruse the google search for 'ksh single bracket vs double bracket' for more details.]
Try running the following to see the differences between single/double brackets and unquoted/single-quoted/double-quoted variable:
SOURCEFILE=/path/*file.csv
set -x
[ ! -f ${SOURCEFILE} ] && echo 'missing'
[ ! -f '${SOURCEFILE}' ] && echo 'missing'
[ ! -f "${SOURCEFILE}" ] && echo 'missing'
[[ ! -f ${SOURCEFILE} ]] && echo 'missing'
[[ ! -f '${SOURCEFILE}' ]] && echo 'missing'
[[ ! -f "${SOURCEFILE}" ]] && echo 'missing'
NOTE: Notice which tests expand the asterisk and which are looking for a (literal) asterisk in the name.
NOTE: Try adding a space to your file name (eg, *file XX.csv) and run the above tests ... tricky, tricky, tricky ...
For this particular case ... asterisk/wildcard in file name, no spaces, ksh ... you'll likely be ok with something like:
while [[ ! -f ${SOURCEFILE} ]] && [[ $(date +'%s') < ${EndTime} ]]
I am looking for a way to test the following korn shell date logic for different days and times. I do not have root access to change the actual date on the system.
CUR_DAY=$(TZ="US/Eastern" date +%a)
typeset -i CUR_HOUR=$(TZ="US/Eastern" date +%H)
# Set the start and end hour given in eastern time
typeset -i START_HOUR=22
typeset -i END_HOUR=7
case $CUR_DAY in
Sun)
if [[ $CUR_HOUR -ge $START_HOUR ]]
then
echo "Send message1"
fi;;
Mon|Tue|Wed|Thu)
if [[ $CUR_HOUR -ge $START_HOUR || $CUR_HOUR -lt $END_HOUR ]]
then
echo "Send message2"
fi;;
Fri)
if [[ "$CUR_HOUR" -lt $END_HOUR ]]
then
echo "Send message3"
fi;;
esac
If you want to test just the logic (and not that date is working correctly), then allow your script to accept CUR_DAY and CUR_HOUR as arguments or via the environment, instead of always running date.
Via the environment (CUR_DAY=9 CUR_HOUR=22 myscript)
: ${CUR_DAY:=$(date +%a)}
: ${CUR_HOUR:=$(date +%H)}
Via arguments (myscript 9 22)
CUR_DAY=${1:-$(date +%a)}
CUR_HOUR=${2:-$(date +%H)}
Either approach will work in any POSIX-compliant shell.
The easiest way to test this is to conditionally add the -d argument to date (assuming the GNU implementation) to pass it a specific date and time for which you intend to test the function's behavior.
The below passes arguments straight through to date, allowing the invocation to be adapted as appropriate:
check_dates() {
typeset cur_day cur_hour start_hour end_hour
# call external date command only once to minimize external process overhead
read -r cur_day cur_hour < <(TZ="US/Eastern" date "$#" +'%a %H')
# trim leading 0s -- otherwise, values will be seen as octal when performing math
cur_day=${cur_day#0}; cur_hour=${cur_hour#0}
start_hour=22
end_hour=8
case $cur_day in
Sun)
if (( cur_hour <= start_hour )); then
echo "Send message1"
fi
;;
Mon|Tue|Wed|Thu)
if (( cur_hour >= start_hour )) || (( cur_hour < end_hour )); then
echo "Send message2"
fi
;;
Fri)
if (( cur_hour < end_hour )); then
echo "Send message3"
fi
;;
esac
}
Thereafter:
check_dates # check for current date and time
check_dates -d '2015-01-06 03:00:00 UTC' # check for a specific date and time
If you're inclined not to use "$#", and don't mind hardcoding behavior specific to GNU date, consider:
check_dates() {
typeset cur_day cur_hour for_date
for_date=$1
read -r cur_day cur_hour < <(TZ="US/Eastern" date ${for_date:+ -d "$for_date"} +'%a %H')
...
}
...which passes -d "$for_date" only if for_date is set to a non-empty value.
I need to write a Bash script that source another script (config script) for hours. If the hour mentioned in config script matches the Linux past hour it needs to print the hour.
$ cat ConfHours.sh
#!/bin/bash --
Hours=(0 1 2 22 23)
$ cat Foo.sh
#!/bin/bash --
source /home/Geo/ConfHours.sh
arrayHours=( ${HOURS} )
for v in "${arrayHours[#]}"
do
HOUR=$(( $(date +%H) -1))
if [ "${HOUR}" == "v" ] ; then
HOUR = ${HOUR}
echo $HOUR
fi
done
When I run Foo.sh, I do not get anything. Could you please correct me where I am wrong?
Some errors:
source /home/Geo/ConfHours.sh
arrayHours=( ${HOURS} )
ConfHours defines a variable named Hours -- different variable
for v in "${arrayHours[#]}"
do
HOUR=$(date -d "1 hour ago" +%H)
You don't need to define this every time through the loop: put it before the for statement
if [ "${HOUR}" == "v" ] ; then
missing $ for the v variable
$HOUR will contain a leading 0 (due to %H)
a better test: if (( 10#$HOUR == 10#$v ))
HOUR = ${HOUR}
No spaces around the = allowed for variable assignment. Why are you trying to redefine the variable to itself?
echo $HOUR
fi
done
A more concise way to test an array contains a value is to take advantage of array string concatenation and pattern matching:
source ConfHours.sh
hour=$(date +%k) # leading space, not leading zero
if [[ " ${Hours[*]} " == *" ${hour# } "* ]]; then
echo "$hour"
fi
All spaces and quotes are required.
Don't use UPPER_CASE_VARS: here's why
Say I have a filename ABC.20131212.XX.xml where XX is HH (hour). I need to get the value of XX and compare it to the current hour of the system time. If it's equal, then, i'll rename that file. So if it's 12pm (12:00), I should get the file with ABC.20131212.12.xml and rename it. How can I achieve the comparison in shell script?
Here's how to get started in pure bash without starting external commands:
a=ABC.20131212.XX.xml
# Strip off .xml
b=${a%.xml}
# Extract last 2 characters
xx=${b: -2}
and the hour can be got with
time=`date +'%H'`
Comparison and rename can be done like this
if [[ $time -eq $xx ]]
then
mv something somewhere
fi
Here's how you can do it using Bash built-in regex matching:
file=foobar.11.xml
if [[ $file =~ .*([0-9][0-9])\.xml ]]; then
if [[ "${BASH_REMATCH[1]}" -eq $(date "+%I") ]]; then
# $file has current hour in its name
fi
fi
Use the +%H option with date for comparing to a 24-hour time system.
Here is the Script you can use(Change path accordingly):
`
!/bin/bash
Path=/Your/Path/Here/*
for l in $Path
do
a=$l
b=${a%.xml}
xx=${b: -2}
time=date +"%I"
if [[ $time -eq $xx ]]
then
mv $l /New/file/name/path/newfile.txt
fi
done
exit 0`
I'm trying to create a script that will loop through files that have their filenames written in the following format: yyyymmdd.hh.filename.
The script is called with:
./loopscript.sh 20091026.00 23
./loopscript.sh 20091026.11 15
./loopscript.sh 20091026.09 20091027.17
The need is for the script to check each hour between those two given dates/hours.
e.g.
cat 20091026.00.filename |more
cat 20091026.01.filename |more
...
cat 20091026.23.filename |more
cat 20091027.01.filename |more
cat 20091027.02.filename |more
...
and so on.
any idea how to go about this? I don't have any difficulty with standard 0 - x loops. or simple for loops. Just not sure how to go about the above.
How about this:
#!/bin/bash
date1=$1
date2=$2
#verify dates
if ! date -d "$date1" 2>&1 > /dev/null ;
then echo "first date is invalid" ; exit 1
fi
if ! date -d "$date2" 2>&1 > /dev/null ;
then echo "second date is invalid" ; exit 1
fi
#set current and end date
current=$(date -d "$date1")
end=$(date -d "$date2 +1 hours")
#loop over all dates
while [ "$end" != "$current" ]
do
file=$(date -d "$current" +%Y%m%d.%H)
cat $file."filename" | more
current=$(date -d "$current +1 hours")
done
To process each file between two given date/hours, you can use the following:
#!/usr/bin/bash
#set -x
usage() {
echo 'Usage: loopscript.sh <from> <to>'
echo ' <from> MUST be yyyymmdd.hh or empty, meaning 00000000.00'
echo ' <to> can be shorter and is affected by <from>'
echo ' e.g., 20091026.00 27.01 becomes'
echo ' 20091026.00 20091027.01'
echo ' If empty, it is set to 99999999.99'
echo 'Arguments were:'
echo " '${from}'"
echo " '${to}'"
}
# Check parameters.
from="00000000.00"
to="99999999.99"
if [[ ! -z "$1" ]] ; then
from=$1
fi
if [[ ! -z "$2" ]] ; then
to=$2
fi
## Insert this to default to rest-of-day when first argument
## but no second argument. Basically just sets second
## argument to 23 so it will be transformed to end-of-day.
#if [[ ! -z "$1"]] ; then
# if [[ -z "$2"]] ; then
# to=23
# fi
#fi
if [[ ${#from} -ne 11 || ${#to} -gt 11 ]] ; then
usage
exit 1
fi
# Sneaky code to modify a short "to" based on the start of "from".
# ${#from} is the length of ${from}.
# $((${#from}-${#to})) is the length difference between ${from} and ${to}
# ${from:0:$((${#from}-${#to}))} is the start of ${from} long enough
# to make ${to} the same length.
# ${from:0:$((${#from}-${#to}))}${to} is that with ${to} appended.
# Voila! Easy, no?
if [[ ${#to} -lt ${#from} ]] ; then
to=${from:0:$((${#from}-${#to}))}${to}
fi
# Process all files, checking that they're inside the range.
echo "From ${from} to ${to}"
for file in [0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9].[0-9][0-9].* ; do
if [[ ! ( ${file:0:11} < ${from} || ${file:0:11} > ${to} ) ]] ; then
echo " ${file}"
fi
done
When you create the files 20091026.00.${RANDOM} through 20091028.23.${RANDOM} inclusive, this is a couple of sample runs:
pax> ./loopscript.sh 20091026.07 9
From 20091026.07 to 20091026.09
20091026.07.21772
20091026.08.31390
20091026.09.9214
pax> ./loopscript.sh 20091027.21 28.02
From 20091027.21 to 20091028.02
20091027.21.22582
20091027.22.30063
20091027.23.29437
20091028.00.14744
20091028.01.6827
20091028.02.10366
pax> ./loopscript.sh 00000000.00 99999999.99 # or just leave off the parameters.
20091026.00.25772
20091026.01.25964
20091026.02.21132
20091026.03.3116
20091026.04.6271
20091026.05.14870
20091026.06.28826
: : :
20091028.17.20089
20091028.18.13816
20091028.19.7650
20091028.20.20927
20091028.21.13248
20091028.22.9125
20091028.23.7870
As you can see, the first argument must be of the correct format yyyymmdd.hh. The second argument can be shorter since it inherits the start of the first argument to make it the correct length.
This only attempts to process files that exist (from ls) and of the correct format, not every date/hour within the range. This will be more efficient if you have sparse files (including at the start and the end of the range) since it doesn't need to check that the files exist.
By the way, this is the command that created the test files, if you're interested:
pax> for dt in 20091026 20091027 20091028 ; do
for tm in 00 01 02 ... you get the idea ... 21 22 23 ; do
touch $dt.$tm.$RANDOM
done
done
Please don't type that in verbatim and then complain that it created files like:
20091026.you.12345
20091028.idea.77
I only trimmed down the line so it fits in the code width. :-)
One possible solution: convert dates into standard Unix representation of "Seconds passed since the epoch" and loop, increasing this number by 3600 (number of seconds in an hour) each iteration. Example:
#!/bin/bash
# Parse your input to date and hour first, so you get:
date_from=20090911
hour_from=10
date_to=20091026
hour_to=01
i=`date --date="$date_from $hour_from:00:00" +%s`
j=`date --date="$date_to $hour_to:00:00" +%s`
while [[ $i < $j ]]; do
date -d "1970-01-01 $i sec" "+%Y%m%d.%H"
i=$[ $i + 3600 ]
done