I tried searching online but was unable to find anything pertaining to my requirements.
I am new to Teradata.
In our team Teradata jobs are used to call the ksh which in turn calls the procedure to run at a scheduled time.
I want to understand how exactly does this calling works? How does a job call a KSH and then how does a KSH call a procedure in turn.
Your help would be much appreciated.
At a very basic level UNIX has a scheduler mechanism called cron. Users with sufficient privilege on the UNIX server can use cron to run jobs at a scheduled time by defining a crontab. Your crontab can call UNIX commands or in many cases a shell script (ksh in your example) to perform a complex set of operations. In many production environments jobs may be scheduled using an enterprise platform instead of many independent crontab files across many users and many servers in the data center.
As this pertains to Teradata, the ksh is likely invoking a Teradata utility such as BTEQ to logon to the database and execute a stored procedure, macro, or set of SQL statements contained within the BTEQ script. Once the BTEQ script has completed a return code is sent to the ksh script to account for any error handling should an error occur within the BTEQ script or an unhandled/handled error within the stored procedure.
You can use your search engine of choice to read up on how to develop UNIX shell scripts (Korn, Bash, etc.) and how Teradata utilities such as BTEQ work. If you have a more discrete question about something in your environment feel free to post a separate question here with the appropriate tags in the question to target the audience who can best help you.
Related
In Oracle or other DBs, we have a concept of PL/SQL package where we can package multiple queries/procedures and call them inside a UNIX script. In case of Hive queries, what's the process used to package and automate the query processing in actual production environments.
If you are looking to automate the execution of numerous Hive queries, the hive or beeline CLI (think sqlplus with Oracle) allows you to pass a file containing one or more commands such as multiple inserts, select, create tables, etc. The contents of said file can be created programmatically using your favorite scripting language like python or shell.
See the "-i" option in this documentation: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Cli
In terms of a procedural language, please see:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=59690156
HPL/SQL does have a Create Package option but if whatever you are trying to achieve is scripted outside of HPL/SQL (e.g. python, shell), you can 'package' your application in accordance with scripting best practices of your selected language.
To run mutilpe queries simply write it down one after another in a file (say 'hivescript.hql') and then it can be run from bash by simply calling it through beeline or hive shell
beeline -u "jdbc:hive2://HOST_NAME:10000/DB" -f hivescript.hql
I have bunch of SQL queries that run against an Oracle database. Is there a way to schedule these .sql files using UNIX Based SAS, so they can execute one after another at certain time of day?
If they are .sql files, why do you want to schedule them using SAS? Are they SAS programs? If not, I would do one of three things, depending on my constraints:
1) Convert the .sql files to stored procedures and call them from DBMS_SCHEDULER within Oracle, since Oracle has a fantastic job scheduling subsystem (actually multiple variants) that protects against duplicate jobs among other issues, and you get transactional control, auditing and logging. http://docs.oracle.com/cd/B19306_01/appdev.102/b14258/d_sched.htm
2) If converting them to stored procs is too much, then call the .sql scripts directly from DBMS_SCHEDULER with DBMS_SCHEDULER.CREATE_PROGRAM() and then schedule that program with DBMS_SCHEDULER.CREATE_JOB.
3) Use cron or atrun to schedule batch / shell script wrappers that call sqlplus to run .sql files.
If the question is specifically how to do this with SAS, then DBMS_SCHEDULER can still execute external SAS programs using option (2) above.
Say I have a SQL script physically stored on the database server. Is there a SQL command I can send Oracle from an application to tell it to execute that script?
(Yes, I know this sounds ridiculous. I'm considering it as part of a work around of a very nasty problem that "shouldn't happen" but does.)
The easiest option would generally be to use the dbms_scheduler package to run an external job. This would let you invoke a shell script that started SQL*Plus, connected to the database, and ran your .sql script.
It would also be possible to create a Java stored procedure that uses Java's ability to call out to the operating system to run the same shell script. That tends to be a bit more of a security issue, though, since you're ending up granting the owner of this procedure privileges to run any command on the database server as the oracle user. That would include things like connecting to the database as SYSDBA or corrupting the database (accidentally or intentionally) so it's something that auditors would generally frown upon.
I'm using oracle 11g and I'm loading data into the database using the sql loader which are invoked through the unix scripts. I want to select some rows of data and write the data into the file using shell scripts. Is it possible to write a shell script for the same.
Here is an excellent tutorial which clearly explain how to execute a query from UNIX
Ultimately what he does is login into Oracle i.e., setup a session for Oracle and read a query that needs to be executed and execute that query and do what ever operation needed.
Instead of reading the query from the user we could read it from a file or even hardcode it there itself as per our need.
Blog which explain about usage of Oracle query in Shell script
Here's a good one for any Oracle gurus out there. I'm working on a web page that dynamically configures Oracle DB backup settings in a closed environment. Right now, I have everything set up to generate scheduled jobs that run pre-determined RMAN scripts that already exist on the Database server's disk. This works, but I want to go a step further.
Is there any way to create jobs with the scheduler that will run RMAN scripts which haven't first been written to disk? For example, is it possible to fire off an RMAN backup script directly from the scheduler by using a pipe of some sort? I've found some vague information on the RMAN Pipe Interface, but I can't see how I could create a private pipe, pack it with RMAN commands, and then feed it to RMAN all in one job run... Any thoughts would be very much appreciated.
In anything related to backup/restore of the database, I advise you to prefer OS's means to execute scheduled jobs (cron/at on unix, Scheduled tasks on Windows). The advantage is that they are independent from oracle instance and you can better handle cases when oracle instance is down or malfunctioning. The "RMAN pipe interface" is meant to be used together with operating system's shell, as well.
However, executing scripts directly from database is also possible: AskTom
If you want to use DBMS_SCHEDULER then the script has to reside on the database server.
But if you install an Oracle client on the web server you can run RMAN from there and connect to the TARGET database. E.g.:
rman 'usr/pwd#conn_str AS SYSDBA' CMDFILE /home/www/db/backup-full.rman
In this case the script can reside on the web server.
Hope this helps.