I've got a load-balanced (not using Session state) ASP.Net 2.0 app on IIS5 running back to a single Oracle 10g server, using version 10.1.0.301 of the ODAC/ODP.Net drivers. After a long period of inactivity (a few hours), the application, seemingly randomly, will throw an Oracle exception:
Exception: ORA-03113: end-of-file on communication channel at
Oracle.DataAccess.Client.OracleException.HandleErrorHelper(Int32
errCode, OracleConnection conn, IntPtr opsErrCtx, OpoSqlValCtx*
pOpoSqlValCtx, Object src, String procedure) at
Oracle.DataAccess.Client.OracleCommand.ExecuteReader(Boolean requery,
Boolean fillRequest, CommandBehavior behavior) at
Oracle.DataAccess.Client.OracleCommand.System.Data.IDbCommand.ExecuteReader()
...Oracle portion of the stack ends here...
We are creating new connections on every request, have the open & close wrapped in a try/catch/finally to ensure proper connection closure, and the whole thing is wrapped in a using (OracleConnection yadayada) {...} block. This problem does not appear linked to the restart of the ASP.Net application after being spun down for inactivity.
We have yet to reproduce the problem ourselves. Thoughts, prayers, help?
More: Checked with IT, the firewall isn't set to kill connections between those servers.
ORA-03113: end-of-file on communication channel
Is the database letting you know that the network connection is no more. This could be because:
A network issue - faulty connection, or firewall issue
The server process on the database that is servicing you died unexpectedly.
For 1) (firewall) search tahiti.oracle.com for SQLNET.EXPIRE_TIME. This is a sqlnet.ora parameter that will regularly send a network packet at a configurable interval ie: setting this will make the firewall believe that the connection is live.
For 1) (network) speak to your network admin (connection could be unreliable)
For 2) Check the alert.log for errors. If the server process failed there will be an error message. Also a trace file will have been written to enable support to identify the issue. The error message will reference the trace file.
Support issues can be raised at metalink.oracle.com with a suitable Customer Service Identifier (CSI)
Add Validate Connection=true to your connection string.
Look at this blog to find more about.
DETAILS:
After OracleConnection.Close() the real database connection does not terminate. The connection object is put back in connection pool. The use of connection pool is implicit by ODP.NET. If you create a new connection you get one of the pool. If this connection is "yet open" the OracleConnection.Open() method does not really creates a new connection. If the real connection is broken (for any reason) you get a failure on first select, update, insert or delete.
With Validate Connection the real connection is validated in Open() method.
Check that there isn't a firewall that is ending the connection after certain period of time (this was the cause of a similar problem we had)
end-of-file on communication channel:
One of the course of this error is due to database fail to write the log when its in the stage of opening;
Solution check the database if its running in ARCHIVELOG or NOARCHIVELOG
to check use
select log_mode from v$database;
if its on ARCHIVELOG try to change into NOARCHIVELOG
by using sqlplus
startup mount
alter database noarchivelog;
alter database open;
if it works for this
Then you can adjust your flashrecovery area its possibly that your flashrecovery area is full
-> then after confirm that your flashrecovery area has the space you can alter your database into the ARCHIVELOG
This error message can be thrown in the application logs when the actual issue is that the oracle database server ran out of space.
After correcting the space issue, this particular error message disappeared.
You could try this registry hack:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters]
"DeadGWDetectDefault"=dword:00000001
"KeepAliveTime"=dword:00120000
If it works, just keep increasing the KeepAliveTime. It is currently set for 2 minutes.
The article previously mentioned is good. http://forums.oracle.com/forums/thread.jspa?threadID=191750 (as far as it goes)
If this is not something that runs frequently (don't do it on your home page), you can turn off connection pooling.
There is one other "gotcha" that is not mentioned in the article. If the first thing you try to do with the connection is call a stored procedure, ODP will HANG!!!! You will not get back an error condition to manage, just a full bore HANG! The only way to fix it is to turn OFF connection pooling. Once we did that, all issues went away.
Pooling is good in some situations, but at the cost of increased complexity around the first statement of every connection.
If the error handling approach is so good, why don't they make it an option for ODP to handle it for us????
//First start the database in mount mode
startup mount
//Disable archivelog
alter database noarchivelog
//Then put db in open
alter database open
Related
Is there an option in Oracle SQL Developer (I use version 4.1.5) to make it reconnect to the DB when needed?
For example, if I start an SQL script but the DB was disconnected, then it should reconnect with minimum hassle and execute the script instead of giving a SQL Error: Closed Connection message.
Note: I'm not interested in keep-alive (what most related questions discuss). If I don't use it today, then it is OK to disconnect. I just want it to reconnect next time I use it.
No.
Auto-connecting dropped connections sounds good in theory, but the 'auto' part is where the trouble begins.
I have a standing ER to add the reconnect button to the dropped connection ERROR message.
In at least SQL Developer version 20.2 this is implemented:
when running an SQL operation when the DB was disconnected after a while, the connection is restored and the operation is executed (there is also a dialog shown telling about the fact).
I'm currently troubleshooting a VB6 application that sporadically comes up with the following error:
[Oracle][ODBC][Ora]ORA-01013: user requested cancel of current operation
All of the research I've done on this error states that it is either an actual request for cancellation by the user or a timeout. It can't be a request for cancellation because the input is coming in from an automated source, so it must be a timeout. One thing I read online was to un-check the query timeout checkbox in the DSN configuration box but my program uses a DSN-less connection to the database, which is an Oracle 10g database.
There are several queries in this program but it always fails on one query in particular, however I can't reproduce the error in a test environment using all of the same input to the program that caused the error in the first place.
A co-worker of mine suggested doing a rollback after each query even though the queries are read only because some kind of buffer might be getting filled up or something of the like, but this didn't work. At this point I don't even know how to continue troubleshooting it because I can't reproduce the error. If someone could give me any idea of what is going on and how to fix the problem I'd greatly appreciate it. Thanks in advance!
All of the options that you can choose when setting up a DSN can be specified in the connection string if you are using a DSN-less connection. If you want to disable query timeouts, you would add
QTO=F
to the connection string. So your new connection string would be something like
DRIVER={Oracle ODBC Driver};UID=Kotzwinkle;PWD=whatever;DBQ=instl_alias;QTO=F;
Do you know any reason to get the following error when I'm taking a few Oracle Reports?
"FRM-40735: ON-ERROR trigger raised unhandled exception ORA-03114"
This happens some times to a few users.
ORA-03114: not connected to ORACLE is an error with several possible causes. As it suggests, it means your client (Forms in this case) has been disconnected from the database.
It's possible that the database or the listener has shut down. Or perhaps a problem with the client has caused the database to disconnect you. There may well be some messages in the database alert log.
ORA-03114 is "Not Connected to Oracle" - i would begin by troubleshooting this issues first.
Either you got disconnect in some way (perhaps an idle connection timeout)
or someone shut down the DB while you were connected. Either way, it means you are disconnected from the database.
My app to recovers automatically from failures. I test it as follows:
Start app
In the middle of processing, kill the application server host (shutdown -r -f)
On host reboot, application server restarts (as a windows service)
Application restarts
Application tries to process, but is blocked by incomplete 2-phase commit transaction in Oracle DB from previous session.
Somewhere between 10 and 30 minutes later the DB resolves the prior txn and processing continues OK.
I need it to continue processing faster than this. My DBA advises that I should prefix my statement with
ALTER SESSION ADVISE COMMIT;
But he can't give me guarantees or details about the potential for data loss doing this.
Luckily the statement in question is simply updating a datetime value to SYSDATE every second or so, so if there was some data corruption it would last < 1 second before it was overwritten.
But, to my question. What exactly does the statement above do? How does Oracle resolve data synchronisation issues when it is used?
Can you clarify the role of the 'local' and 'remote' databases in your scenario.
Generally a multi-db transaction does the following
Starts the transaction
Makes a change on on database
Makes a change on the other database
Gets the other database to 'promise to commit'
Commits locally
Gets the remote db to commit
In doubt transactions happen if step 4 is completed and then something fails. The general practice is to get the remote database back up and confirm if it committed. If so, step (5) goes ahead. If the remote component of the transaction can't be committed, the local component is rolled back.
Your description seems to refer to an app server failure which is a different kettle of fish. In your case, I think the scenario is as follows :
App server takes a connection and starts a transaction
App server dies without committing
App server restarts and make a new database connection
App server starts a new transaction on the new connection
New transaction get 'stuck' waiting for a lock held by the old connection/transaction
After 20 minutes, dead connection is terminated and transaction rolled back
New transaction then continues
In which case the solution is to kill off the old connection quicker, with a shorter timeout (eg SQLNET_EXPIRE_TIME in the sqlnet.ora of the server) or a manual ALTER SYSTEM KILL SESSION.
Im having some trouble with an oracle database. Every time i try to connect, i get this message.
ORA-01033: ORACLE initialization or
shutdown in progress
I searched the web, and found that the solution is to execute an alter database open command, but what I dont understand is where should i execute if, since I cant connect to the database.
Am I missing something?
Thanks in advance
You should connect AS SYSDBA:
sqlplus "sys/pwd AS SYSDBA"
An ORA-01033 would also be thrown if the connection attempt were made against a mounted standby database (Oracle Data Guard environment) by a non-sysdba user. Maybe a database role change (switchover or failover) has occurred since your last connection attempt.
Which OS are you using? The database is still shutting down - check the alert log as to the status of where it is at. Sometimes there are sessions hanging around that need to manually be killed off, there should be an indication of this in the alert log. It also depends how the database was shutdown, NORMAL, TRANSACTIONAL, IMMEDIATE. Even with a SHUTDOWN IMMEDIATE the sessions hanging around may still happen. To find the sessions on UNIX use the 'ps' to list all processes on the server ('ps -eaf' on Solaris) command and 'grep' for the ORACLE_SID name.