Our database is shared between few teams. Since a few day ago, our database generate a lots of archive log, making our server run out of space. There a day with over 60gb, ~1gb each file each 30minute.
I have asked other team if they have any process which make a lot of change to database but they all denied.
So i want to check the root cause of this, find out which user, table, process or query which generated these log. Any1 know a way to check it?
Related
We are trying to use the Oracle Debezium connector (1.9b) on top of Kafka.
We tried to use 2 things regarding snapsot_mode: schema_only and initial.
We use "log.mining.strategy":"online_catalog" (should be the default)
We are using a PDB/CDB Oracle instance on Oracle 19c.
My understanding is that;
The connector create a session to the PDB
It add a shared lock to ensure the structure will not change (shared) for a short duration
the DDL structure is retrieved from the PDB
It create a session to the CDB
It retrieve the last LSN event from CDB
if snapshot == initial, it will use a "JDBC query to retrieve the whole data" from PDB
it does NOT seems to release the initiated session (or rather process) to the PDB
it continues to mines new events from CDB
x. ... it seems to work for a couple of minutes
After a couple of minutes, the number of process increase drastically
The Oracle database freeze, due to an excess of process (that you can follow using v$process)
We had a lot of errors message; like:
A. Failed to resolve Oracle database
B. IO Error: Got minus one from a read call
C. ORA-04025: maximum allowed library object lock allocated
D. None of log files contains offset SCN: xxxxxxx
The message in point D. said it tries to use a offset which was part of "an old" archived log.
Every 30min (or before, it we have more activity), the log is switched from a file to another.
And a backup is occuring every 30minutes which will read the logs, backup it and then: delete it .
It seems to me that Debezium tried to reach past archived log whose was deleted by the backup process.
The process of "deleting previous archived logs" seems "correct" to me, isn't it ?
Why Debezium tries to pass through the archived logs ? because when snapshot==schema_only it should only catch the news events, therefore why using the archived one ?
How can we manage it ?
I hope that if this point is resolved in my use-case, the debezium will stop to "loop" creating new process and ultimately will stop blocking the Oracle DB.
If you have any clues or opinions, don't hesitate to share it. Many thanks
We try to use shared lock and none
We try to limite the number of tables in the scope
I cannot ask to stop the backup, in production it's not a good idea and in test, it seems that the backup is only there to clean the archived logs and avoid ending with completely used storage.
I am developing an H2 (TCP), based application and I created a feature that takes a backup file when a button is clicked.
I used the
BACKUP TO '<FILENAME>'
command to take online backups. I had tested it and it works, but for very rare instances, the backup is corrupted.
Is there any API in H2 to check if the backup file is corrupted? I am thinking I would load that backup zip file, then do count queries on all tables and display it on the screen.
As Thomas (creator of H2) citiated before fastest way to verify is getting connection. If you want to learn if the data is corrupted, Backup and Restore is the another encouraged way. Currently There isn't any api or tool now.
From Thomas:
There is a small risk that the database file exists, but not fully initialized. If that is the case,
then some of the tables don't exist. The standard way to verify all tables exists is to us
DatabaseMetaData.getTables.
I have a WPF application with back-end as Oracle11gR2. We need to enable our application to work in both online and offline(disconnected) mode. We are using Oracle standard edition(with single instance) as client database. I am using Sequnece Numbers for Primary Key Columns. Is there anyway to sync my client and server database without any issues in Sequence number columns. Please note that we will restrict creation of basic(master) data to be created only in server.
There are a couple of approaches to take here.
1- Write the sync process to rebuild the server tables (on the client) each time with a SELECT INTO. Once complete, RENAME the current table to a "temp" table, and RENAME the newly created table with the proper name. The sync process should DROP the temp table as one of its first steps. Finally, recreate the indexes and you should be good-to-go.
2- Create a backup of the server-side database, write a shell script to copy it down and restore it on the client.
Each of these options will preserve your sequence numbers. Which one you choose really depends on your skills. If you're more of a developer, you can make #1 work. If you've got some Oracle DBA skills you should be able to make #2 work.
Since you're on 11g, there might be a cleaner way to do this using Data Pump.
Can any body give me an idea about Redo logs? An example would be most appreciated.
As Oracle changes data in a datafile, it writes out information to the redo log. In the event of a database failure, you can use this information to get the database back to the point it was before the database failure.
In a disaster recovery scenario, you could restore your last full database backup, and then apply the redo logs taken since that last backup to get the database recovered. Without those redo logs, you could only recover to the last full backup, and changes made since then would be lost.
In Oracle, you can also run in "no archive log mode", which basically means, "redo logs can be overwritten without being saved". This is generally only acceptable for a development database where you don't care about losing data since the last backup. You wouldn't typically run in this mode in a production environment, as it could be disastrous.
Here's a reference link with more info, and also an example of how you can find out the amount of generated redo.
http://www.adp-gmbh.ch/ora/concepts/redo_log.html
A definitive answer from the documentation: http://download.oracle.com/docs/cd/B19306_01/server.102/b14231/onlineredo.htm#sthref850
To expand on #dcp's answer: Technically, #dcp is referring to archived redo logs. These are optional, and as stated are only produced when running the database in archivelog mode. Every Oracle database has at least two mandatory online redo log files. These track all changes to the database. They are essential for recovery if the database crashes unexpectedly, whereas archived logs are not. Oracle uses the online redo log files to transparently bring the database back to the most recently committed state in the event of a system crash. Archived logs are used during recovery from a backup - the backup is restored, then archived logs are applied to the backup to bring the database back to it's current state or some prior point in time.
The online logs are written to in circular fashion - as one fills the next one is "swtiched" to. If archive log mode is set, then these older logs are written to the archive log destination(s). If not, they are overwritten as needed, once the changes they track are written to the datafiles.
This overview of backup and recovery at Oracle's site is pretty good to give one an idea of how the whole thing is put together.
My team inherited an Oracle-based web application and they are fairly inexperienced with Oracle database servers.
The Oracle 10g server is running on a Windows 2003 Server with plenty of disk space and from time to time, all connectivity is lost, the application stops working, not even SQL Plus is able to connect to the database server.
But when we check the Windows Service manager, it says that the service is up and running. A restart usually fixes the problem, but we need to properly troubleshoot it so we know what's causing it and so we can avoid it to happen anymore.
Where should we start looking for clues? What are the criticial log files we should be investigating?
On the server you should have an environment variable called ORACLE_HOME which indicate the root of the Oracle install. Most likely the Oracle trace/dump folders will be under there. Search for a folder called "bdump" (background dump). That's where the main log file, knows as the alert log, will be, as well as trace files generated by background processes. There will be an adjacent file called "udump" which will contain any trace files generated by user processes.
However, my real advice is that you should either hire someone who knows Oracle or get Oracle Support involved.
The alert log would be the first file to check.
It will probably be in $ORACLE_HOME/admin/bdump and (probably) called alert_DATABASE-SID.log
It contains most of the important actions that the database does, as well as any important errors that occur.
I have to agree with cagcowboy. Check your alert logs for errors. If no errors then maintain a sysdba login into the database and when it hangs, attempt to do a hang analysis. See metalink note 215858.1 on hanganalyze.
Have you tried tnsping? We've occasionally run into problems with the listener that requires an assist from our DBA. tnsping is the diagnostic tool we use to do triage.
I would recommend hiring an experienced Oracle DBA if at all possible.
check the alert log to see how the Db is structured. sometimes badly set parameters make hangs or slow performance. or you can shutdown and start in mount mode, then check the v$parameter values for problems. setting total memory is very important.