need to add in the start of script to delete export dump 3 days old and then start export backup,
script should not execute without deleting old backups,
if it fails to delete old backup , script should terminate and send us alert.
# $Header: EXP_TAB_cmprss.sh
# *====================================================================================+
#
# | |
# +====================================================================================+
# |
# | FILENAME
# |
# |
# | DESCRIPTION
# | Daily Export backup script of a list of table
# | PLATFORM
# | Linux/Solaris
# +===========================================================================+
#!/bin/bash
echo Set Oracle Database Env
export ORACLE_SID=$1
export ORACLE_HOME=/oracle/app/oracle/product/12.1.0/dbhome_1
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/usr/lib
export PATH=$ORACLE_HOME/bin:$PATH:/usr/local/bin
export TIMESTAMP=`date +%a%d%b%Y`
export EXP_DIR=/export/home/oracle
echo =======
echo Export command
echo =======
echo $ORACLE_HOME
$ORACLE_HOME/bin/expdp \'/ as sysdba\' directory=DB_DUMPS dumpfile=expdp_tab_${TIMESTAMP}_%U.dmp logfile=expdp_log_${TIMESTAMP}.log
tables=DBATEST.ORDER_TAB PARALLEL=5 COMPRESSION=ALL
echo SEND MAIL TO STAKE HOLDERS
echo =======
mailx -s "$ORACLE_SID $TIMESTAMP Export backup logfile" support#dbclass.com < $EXP_DIR/expdp_log_${TIMESTAMP}.log
echo Export completed at $TIMESTAMP
exit
in EXP_DIR we have below files generated after running script
-rwxrwxrwx 1 tuser tuser 1.4K May 6 12:25 EXP_TAB_cmprss.sh
-rwxrwxrwx 1 t2n5463 odba 3.2G May 6 13:13 expdp_tab_Thu06May2021_05.dmp
-rwxrwxrwx 1 t2n5463 odba 3.1G May 6 13:13 expdp_tab_Thu06May2021_03.dmp
-rwxrwxrwx 1 t2n5463 odba 22G May 6 13:13 expdp_tab_Thu06May2021_02.dmp
-rwxrwxrwx 1 t2n5463 odba 2.9G May 6 13:13 expdp_tab_Thu06May2021_04.dmp
-rwxrwxrwx 1 t2n5463 odba 25G May 6 13:13 expdp_tab_Thu06May2021_01.dmp
-rw-r--r-- 1 t2n5463 odba 8.7K May 6 13:13 expdp_log_Thu06May2021.log
Related
unfortunately I'm quite new at bash, and I want to write a script that will start in a main directory, and check all subdirectories one by one for the presence of certain files, and if those files are present, perform an operation on them. For now, I have written a simplified version to test whether I can do the first part (checking for the files in each directory). This code runs without any errors that I can tell, but it does not echo anything to say that it has successfully found the files which I know are there.
#!/bin/bash
runlist=(1 2 3 4 5 6 7 8 9)
for f in *; do
if [[ -d {$f} ]]; then
#if f is a directory then cd into it
cd "{$f}"
for b in $runlist; do
if [[ -e "{$b}.png" ]]; then
echo "Found {$b}"
#if the file exists then say so
fi
done
cd -
fi
done
'''
Welcome to stackoverflow.
The following will do the trick (a combination of find, array, and if then else):
# list of files we are looking for
runlist=(1 2 4 8 16 32 64 128)
#find each of above anywhere below current directory
# using -maxdepth 1 because, based on on your exam you want to look one level only
# if that's not what you want then take out -maxdepth 1 from the find command
for b in ${runlist[#]}; do
echo
PATH_TO_FOUND_FILE=`find . -name $b.png`
if [ -z "$PATH_TO_FOUND_FILE" ]
then
echo "nothing found" >> /dev/null
else
# You wanted a postive confirmation, so
echo found $b.png
# Now do something with the found file. Let's say ls -l: change that to whatever
ls -l $PATH_TO_FOUND_FILE
fi
done
Here is an example run:
mamuns-mac:stack foo$ ls -lR
total 8
drwxr-xr-x 4 foo 1951595366 128 Apr 11 18:03 dir1
drwxr-xr-x 3 foo 1951595366 96 Apr 11 18:03 dir2
-rwxr--r-- 1 foo 1951595366 652 Apr 11 18:15 find_file_and_do_something.sh
./dir1:
total 0
-rw-r--r-- 1 foo 1951595366 0 Apr 11 17:58 1.png
-rw-r--r-- 1 foo 1951595366 0 Apr 11 17:58 8.png
./dir2:
total 0
-rw-r--r-- 1 foo 1951595366 0 Apr 11 18:03 64.png
mamuns-mac:stack foo$ ./find_file_and_do_something.sh
found 1.png
-rw-r--r-- 1 foo 1951595366 0 Apr 11 17:58 ./dir1/1.png
found 8.png
-rw-r--r-- 1 foo 1951595366 0 Apr 11 17:58 ./dir1/8.png
found 64.png
-rw-r--r-- 1 foo 1951595366 0 Apr 11 18:03 ./dir2/64.png
It only prints the "else" statement for everything but I know for a fact the files exist that it's looking for. I've tried adapting some of the other answers but I thought this should definitely work.
Does anyone know what's wrong with my syntax?
# Contents of script
for ID_SAMPLE in $(cut -f1 metadata.tsv | tail -n +2);
do if [ -f ./output/${ID_SAMPLE} ]; then
echo Skipping ${ID_SAMPLE};
else
echo Processing ${ID_SAMPLE};
fi
done
Additional information
# Output directory
(base) -bash-4.1$ ls -lhS output/
total 170K
drwxr-xr-x 8 jespinoz tigr 185 Jan 3 16:16 ERR1701760
drwxr-xr-x 8 jespinoz tigr 185 Jan 17 18:03 ERR315863
drwxr-xr-x 8 jespinoz tigr 185 Jan 16 23:23 ERR599042
drwxr-xr-x 8 jespinoz tigr 185 Jan 17 00:10 ERR599072
drwxr-xr-x 8 jespinoz tigr 185 Jan 16 13:00 ERR599078
# Example of inputs
(base) -bash-4.1$ cut -f1 metadata.tsv | tail -n +2 | head -n 10
ERR1701760
ERR599078
ERR599079
ERR599070
ERR599071
ERR599072
ERR599073
ERR599074
ERR599075
ERR599076
# Output of script
(base) -bash-4.1$ bash test.sh | head -n 10
Processing ERR1701760
Processing ERR599078
Processing ERR599079
Processing ERR599070
Processing ERR599071
Processing ERR599072
Processing ERR599073
Processing ERR599074
Processing ERR599075
Processing ERR599076
# Checking a directory
(base) -bash-4.1$ ls -l ./output/ERR1701760
total 294
drwxr-xr-x 2 jespinoz tigr 386 Jan 15 21:00 checkpoints
drwxr-xr-x 2 jespinoz tigr 0 Jan 10 01:36 tmp
-f is for checking whether the name is a file, but all your names are directories. Use -d to check that.
if [ -d "./output/$ID_SAMPLE" ]
then
If you want to check whether the name exists with any type, use -e.
So I tried
ncftpls -l
which gives me a list
-rw-r--r-- 1 100 ftpgroup 3817084 Jan 29 15:50 1548773401.tar.gz
-rw-r--r-- 1 100 ftpgroup 3817089 Jan 29 15:51 1548773461.tar.gz
-rw-r--r-- 1 100 ftpgroup 3817083 Jan 29 15:52 1548773521.tar.gz
-rw-r--r-- 1 100 ftpgroup 3817085 Jan 29 15:53 1548773582.tar.gz
-rw-r--r-- 1 100 ftpgroup 3817090 Jan 29 15:54 1548773642.tar.gz
But all I want is to check the timestamp (which is the name of the tar.gz)
How to only get the timestamp list ?
As requested, all I wanted to do is delete old backups, so awk was a good idea (at least it was effective) even it wasn't the right params. My method to delete old backup is probably not the best but it works
ncftpls *authParams* | (awk '{match($9,/^[0-9]+/, a)}{ print a[0] }') | while read fileCreationDate; do
VALIDITY_LIMIT="$((`date +%s`-600))"
a=$VALIDITY_LIMIT
b=$fileCreationDate
if [ $b -lt $a ];then
deleteFtpFile $b
fi
done;
You can use awk to only display the timestamps from the output like so:
ncftpls -l | awk '{ print $5 }'
I have a log table which is maintained for a single day and the data from the table is only present for one day.However, the logs for it is present in the unix directory.
My requirement is to check the logs for the last 10 days and find me the count of records got loaded.
In the log file the pattern is something like this( fastload log of teradata).
**** 13:16:49 END LOADING COMPLETE
Total Records Read = 443303
Total Error Table 1 = 0 ---- Table has been dropped
Total Error Table 2 = 0 ---- Table has been dropped
Total Inserts Applied = 443303
Total Duplicate Rows = 0
I want to the script to be parametrized( parameter will be stage table name) which find the records inserted into table and error tables for the last 10 days.
Is this possible? Can anyone help me build the unix script for this?
There are many logs in the logs directory. what if a want to check only for the below:
bash-3.2$ ls -ltr 2018041*S_EVT_ACT_FLD*
-rw-rw----+ 1 edwops abgrp 52610 Apr 10 17:37 20180410173658_S_EVT_ACT_FLD.log
-rw-rw----+ 1 edwops abgrp 52576 Apr 11 18:12 20180411181205_S_EVT_ACT_FLD.log
-rw-rw----+ 1 edwops abgrp 52646 Apr 13 18:04 20180413180422_S_EVT_ACT_FLD.log
-rw-rw----+ 1 edwops abgrp 52539 Apr 14 16:16 20180414161603_S_EVT_ACT_FLD.log
-rw-rw----+ 1 edwops abgrp 52538 Apr 15 14:15 20180415141523_S_EVT_ACT_FLD.log
-rw-rw----+ 1 edwops abgrp 52576 Apr 16 15:38 20180416153808_S_EVT_ACT_FLD.log
Thanks.
find . -ctime -10 -type f -print|xargs awk -F= '/Total Records Read/ {print $2}'|paste -sd+| bc
find . -ctime -10 -type f -print get the filenames of files 10 days or younger in current working directory. To run on a different directory replace . with the path
awk -F= '/Total Records Read/ {print $2}' using = as a field seperator filter out the second half of any line containing the key phrase
Total Records Read
paste -sd+ add a plus sign
bc evaluate the stream of numbers and operators into a single answer
I could not use find. because the system is Solaris, find doesn't have maxdepth future. I use case to create a FILTER2 and use it to
ls -l --time-style=long-iso FOLDER | grep -E $FILTER.
but I know it's not a good way.
LOCAL_DAY=`date "+%d"`
LOCAL_MONTH=`date "+%Y-%m"`
LASTTENDAYE_MONTH=`date --date='10 days ago' "+%Y-%m"`
case $LOCAL_DAY in
0*)
FILTER2="$LASTTENDAY_MONTH-[2-3][0-9]|$LOCAL_MONTH";;
1*)
FILTER2="$LOCAL_MONTH-0[0-9]|$LOCAL_MONTH-1[0-9]";;
2*)
FILTER2="$LOCAL_MONTH-1[0-9]|$LOCAL_MONTH-2[0-9]";;
esac
I am on Archlinux and I am trying to force a cron application to execute my script. So I installed package fcron and ran it's daemon first and then I added fcrontab which looks like this:
* * * * * /home/ziga/Dropbox/workspace/operacijski/archlinux/hibernate/hibernatescript
Ok so this is where it gets tricky. This script will execute if it's contents looks like this:
#!/bin/bash
PATH=/usr/bin
touch ~/test-002.txt
But it won't execute if it's contents look like this:
#!/bin/bash
PATH=/usr/bin
acpi -b | awk -F'[,:%]' '{print $2, $3}' | {
read -r status capacity
if [ "$status" = Discharging -a "$capacity" -lt 5 ]; then
logger "Critical battery threshold"
systemctl hibernate
fi
}
Why is that? Even the Archlinux Wiki says it should work, but I assume it is wrong - I only changed #!/bin/bash and added PATH=/usr/bin. Script works on its own...
EDIT:
So I edited my script to look like below and it doesn't work.
#!/bin/sh
SHELL=/bin/sh
/usr/bin/acpi -b | /usr/bin/awk -F'[,:%]' '{print $2, $3}' | (
read -r status capacity
if [ "$status" = Discharging ] && [ "$capacity" -lt 50 ]; then
/usr/bin/systemctl hibernate
fi
)
If I run
[ziga#ziga-laptop hibernate]$ run-parts ~/Dropbox/workspace/operacijski/archlinux/hibernate/ -v
run-parts: executing /home/ziga/Dropbox/workspace/operacijski/archlinux/hibernate//hibernatescript
My PC hibernates, but again cron doesn't hibernate my PC using the exact same script. I also tried fcron and it doesn't work although daemon is fully running and fcrontab is set. Again this works for any simpler scripts while it wont work for this one.
Here are some info:
[ziga#ziga-laptop ~]$ systemctl status fcron.service
● fcron.service - fcron periodical command scheduler
Loaded: loaded (/usr/lib/systemd/system/fcron.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2016-06-21 10:39:48 CEST; 58min ago
Process: 419 ExecStart=/usr/bin/fcron (code=exited, status=0/SUCCESS)
Main PID: 423 (fcron)
Tasks: 1 (limit: 512)
CGroup: /system.slice/fcron.service
└─423 /usr/bin/fcron
Jun 21 11:34:00 ziga-laptop fcron[12833]: Job '/home/ziga/Dropbox/workspace/operacijski/archlinux/hibernate/hibernatescrip
Jun 21 11:34:02 ziga-laptop fcron[12833]: Job '/home/ziga/Dropbox/workspace/operacijski/archlinux/hibernate/hibernatescrip
Jun 21 11:35:00 ziga-laptop fcron[13021]: Job '/home/ziga/Dropbox/workspace/operacijski/archlinux/hibernate/hibernatescrip
Jun 21 11:35:02 ziga-laptop fcron[13021]: Job '/home/ziga/Dropbox/workspace/operacijski/archlinux/hibernate/hibernatescrip
Jun 21 11:36:00 ziga-laptop fcron[13210]: Job '/home/ziga/Dropbox/workspace/operacijski/archlinux/hibernate/hibernatescrip
Jun 21 11:36:02 ziga-laptop fcron[13210]: Job '/home/ziga/Dropbox/workspace/operacijski/archlinux/hibernate/hibernatescrip
Jun 21 11:37:00 ziga-laptop fcron[13407]: Job '/home/ziga/Dropbox/workspace/operacijski/archlinux/hibernate/hibernatescrip
Jun 21 11:37:02 ziga-laptop fcron[13407]: Job '/home/ziga/Dropbox/workspace/operacijski/archlinux/hibernate/hibernatescrip
Jun 21 11:38:00 ziga-laptop fcron[13599]: Job '/home/ziga/Dropbox/workspace/operacijski/archlinux/hibernate/hibernatescrip
Jun 21 11:38:02 ziga-laptop fcron[13599]: Job '/home/ziga/Dropbox/workspace/operacijski/archlinux/hibernate/hibernatescrip
[ziga#ziga-laptop ~]$ fcrontab -l
2016-06-21 11:38:48 INFO listing ziga's fcrontab
* * * * * /home/ziga/Dropbox/workspace/operacijski/archlinux/hibernate/hibernatescript
[ziga#ziga-laptop ~]$ cat /home/ziga/Dropbox/workspace/operacijski/archlinux/hibernate/hibernatescript
#!/bin/sh
SHELL=/bin/sh
/usr/bin/acpi -b | /usr/bin/awk -F'[,:%]' '{print $2, $3}' | (
read -r status capacity
if [ "$status" = Discharging ] && [ "$capacity" -lt 50 ]; then
/usr/bin/systemctl hibernate
fi
)
[ziga#ziga-laptop ~]$ ls -l /home/ziga/Dropbox/workspace/operacijski/archlinux/hibernate/
total 16
-rwxrwxrwx 1 ziga users 214 Jun 21 08:46 hibernatescript
-rw-rw-rw- 1 ziga users 2832 Jun 21 00:24 README.txt
[ziga#ziga-laptop ~]$ groups ziga
lp audio bumblebee users
[ziga#ziga-laptop ~]$