While testing a new application, we came across an issue that sometimes a stored proc takes over 1 minute to execute and causes a time out. It was not 1 stored proc in particulary, it could be any.
Trying to reproduce the issue I've created a small (local) testapp that calls the same stored proc in different threads (code below).
Now it seems that the Oracle-sessions are still there. Inactive. And the CPU of the Oracle-server hits 100%.
I use the System.Data.OracleClient
I'm not sure if one is related to the other, but it slows down the time needed to get an answer from the database.
for (int index = 0; index < 1000; ++index)
{
ThreadPool.QueueUserWorkItem(GetStreet, index);
_runningThreads++;
WriteThreadnumber(_runningThreads);
}
private void GetStreet(object nr)
{
const string procName = "SPCK_ISU.GETPREMISESBYSTREET";
DataTable dataTable = null;
var connectionstring = ConfigurationManager.ConnectionStrings["CupolaDB"].ToString();
try
{
using (var connection = new OracleConnection(connectionstring))
{
connection.Open();
using (var command = new OracleCommand(procName, connection))
{
//Fill parameters
using (var oracleDataAdapter = new OracleDataAdapter(command))
{
//Fill datatable
}
}
}
}
finally
{
if (dataTable != null)
dataTable.Dispose();
}
}
EDIT:
I just let the dba make a count of the open sessions and there are 105 sessions that stay open-inactive. After closing my application, the sessions are removed.
Problem is solved.
We hired an Oracle-expert to take a look at this and the problem was caused due to some underlying stored procedures that took a while to execute and consumed a lot of CPU.
After the necessary tuning, everything runs smoothly.
Related
I have some kind of promblem with BulkInsert on my OracleDB. I need to insert couple of thousand objects so I decided to use EF.BulkInsert.Oracle added by Nuget which is extension of EF6.BulkInsert for Oracle.
private IOracleDbContext _context;//Class property
//method body:
EF6.BulkInsert.ProviderFactory.Register<EF6.BulkInsert.Providers.OracleBulkInsertProvider>("BulkInsertProvider");
using (var context = (OracleDbContext)_context)
{
using (var dbContextTransaction = context.Database.BeginTransaction())
{
try
{
//Preparing list of objects
var opt = new EF6.BulkInsert.BulkInsertOptions();
opt.Connection = context.Database.Connection;
await context.BulkInsertAsync<ObjectType>(ObjectList,opt);
await context.SaveChangesAsync();
dbContextTransaction.Commit();
stopwatch.Stop();
}
catch (Exception ex)
{
dbContextTransaction.Rollback();
throw ex;
}
}
}
Without adding opt (BulkInsertOptions object) as parameter of BulkInsert it is trying to connect with SQLServer (which don't exist so I get connection failure). After add this BulkOptions with connection I get exception that connection is already part of transaction :/
Traditional way (_context.TableName.Add() ) of course works but It takes unacceptable amount of time.
Any idea what I did wrong here?
I found better way (BulkInsert still do not cooperate). I used Array Binding
mentioned here
It reduced insert time from ~6 minutes to ~1-1.5 seconds :D (7770 records)
The Domino interop API which is included with Lotus Notes causes an out of memory exception in .NET when the NotesDXLExporter class based object fails to export the 390th record, which is a big document, after exporting 389 records (which are smaller documents).
Here is a code snippet:
I initialize the NotesDXLExporter class.
NotesDXLExporter dxl1 = null;
I then configure the NotesDXLExported object as shown below:
dxl1 = notesSession.CreateDXLExporter();
dxl1.ExitOnFirstFatalError = false;
dxl1.ConvertNotesbitmapsToGIF = true;
dxl1.OutputDOCTYPE = false;
I then perform a for a loop shown below in reading documents using the dxl1 class (line on which exception occurs is indicated below).
NotesView vincr = database.GetView(#"(AllIssuesView)"); //view from an NSF file
for (int i = 1; i < vincr.EntryCount; i++)
{
try
{
vincrdoc = vincr.GetNthDocument(i);
System.IO.File.WriteAllText(#"C:\Temp\" + i + #".txt", dxl1.Export(vincrdoc)); //OUT OF MEMORY EXCEPTION HAPPENS HERE WHEN READING A BIG DOCUMENT.
}
catch(Exception ex)
{
Console.WriteLine(ex);
}
I have tried using a different version of the Interop domino dll and had had no success.
As I understand this, I see an API issue but I dont know if I am missing something?
Can you please shed some light on this?
Thanks in advance.
Subbu
You haven't said what version of the Lotus Notes you are working with. Given the history of DXL, I would say that you should try your code on the latest version of Notes that you possibly can.
But also, I don't see any calls to recycle(). Failure to call recycle() for Domino objects causes memory to leak from the Domino back end classes, and since you are running out of memory it could be contributing to your problem. You should also not use a for loop and getNthDocument. You should use getFirstDocument and a while loop with getNextDocument. You'll get much better performance. Putting these two things together leads you to the common pattern of using a temporary document to hold the result of getNextDocument, allowing you to recycle the current document, and then assign the temp document to the current, which would be something like this (not error-checked!)
NotesView vincr = database.GetView(#"(AllIssuesView)"); //view from an NSF file
vincrdoc = vincr.getFirstDocument();
while (vincrdoc != null)
{
try {
System.IO.File.WriteAllText(#"C:\Temp\" + i + #".txt", dxl1.Export(vincrdoc));
}
catch(Exception ex)
{
Console.WriteLine(ex);
}
Document nextDoc = vincr.getNextDocument(vincrdoc);
vincrdoc.recycle();
vincrdoc = nextDoc;
}
I am having a problem with SysCache/SysCache2 on my MVC application. My configuration seems to be correct. I have set it up just like countless examples on the web.
On my class I have put: Cache.Region("LongTerm").NonStrictReadWrite().IncludeAll();
Here is a Test I made for the application cache.
[Test]
public void cache()
{
using (var session = sessionFactory.OpenSession())
using (var tx = session.BeginTransaction())
{
var acc = session.QueryOver<Log>().Cacheable().List();
tx.Commit();
}
var test = sessionFactory.Statistics.SecondLevelCacheHitCount;
using (var session = sessionFactory.OpenSession())
{
var acc = session.QueryOver<Log>().List();
}
var test1 = sessionFactory.Statistics.SecondLevelCacheHitCount;
}
The first statement is cached as I see in the session factory statistics (for example 230 records).
If i understand it right second statement that is below shouldnt hit the db but the Cache.
Problem here is that it goes to DB anyway. Checked with profiler to be 100% sure.
I don't know what am I doing wrong here. Anyone has an idea?
I have managed to solve this problem. It had to do with my session creation. I didn't use session per request which triggered not going to cache. I created transaction on begining and it lasted through entire session. I managed to trigger entering cache if i opened the session again within using mark like: using(var sess = session.SessionFactory.OpenSession()) but this solution was only a workaround which didn't suit me so I changed how I created sessions in the first place and it works fine now! :)
When data is entered, it ultimately needs to be saved remotely on a server. I do want the app to work if there is no data connection at the time also, so I need to save everything locally on the phone too. The app can then sync with the server when it gets a connection.
This brings up a little issue. I'm used to saving everything on the server and then getting the records back with id's generated from the server for them. If there is no connection, the app will save locally to the phone but not the server. When syncing with the server, I don't see a way for the phone to know when a record comes back which locally record it's associated with. There isn't enough unique data to figure this out.
What is the best way to handle this?
One way I've been thinking is to change the id of the records to a GUID and let the phone set the id. This way, all records will have an id locally, and when saving to the server, it should still be a unique id.
I'd like to know what other people have been doing, and what works and what doesn't from experience.
This is how we done with a first windows phone 7 app finished few days ago with my friend.
It might not be the best solution but 'till additional refactoring it works just fine.
It's an application for a web app like a mint.com called slamarica.
If we have feature like save transaction, we first check if we have connection to internet.
// Check if application is in online or in offline mode
if (NetworkDetector.IsOnline)
{
// Save through REST API
_transactionBl.AddTransaction(_currentTransaction);
}
else
{
// Save to phone database
SaveTransactionToPhone(_currentTransaction);
}
If transaction is successfully saved via REST, it responses with transaction object and than we save it to local database. If REST save failed we save data to local database.
private void OnTransactionSaveCompleted(bool isSuccessful, string message, Transaction savedTransaction)
{
MessageBox.Show(message);
if(isSuccessful)
{
// save new transaction to local database
DatabaseBl.Save(savedTransaction);
// save to observable collection Transactions in MainViewModel
App.ViewModel.Transactions.Add(App.ViewModel.TransactionToTransactionViewModel(savedTransaction));
App.ViewModel.SortTransactionList();
// Go back to Transaction List
NavigationService.GoBack();
}
else
{
// if REST is failed save unsent transaction to Phone database
SaveTransactionToPhone(_currentTransaction);
// save to observable collection Transactions in MainViewModel
App.ViewModel.Transactions.Add(App.ViewModel.TransactionToTransactionViewModel(_currentTransaction));
App.ViewModel.SortTransactionList();
}
}
Every Transaction object has IsInSync property. It is set to false by default until we got confirmation from REST API that it's saved successful on the server.
User has ability to refresh transactions. User can click on a button Refresh to fetch new data from the server. We do the syncing in the background like this:
private void RefreshTransactions(object sender, RoutedEventArgs e)
{
if (NetworkDetector.IsOnline)
{
var notSyncTransactions = DatabaseBl.GetData<Transaction>().Where(x => x.IsInSync == false).ToList();
if(notSyncTransactions.Count > 0)
{
// we must Sync all transactions
_isAllInSync = true;
_transactionSyncCount = notSyncTransactions.Count;
_transactionBl.AddTransactionCompleted += OnSyncTransactionCompleted;
if (_progress == null)
{
_progress = new ProgressIndicator();
}
foreach (var notSyncTransaction in notSyncTransactions)
{
_transactionBl.AddTransaction(notSyncTransaction);
}
_progress.Show();
}
else
{
// just refresh transactions
DoTransactionRefresh();
}
}
else
{
MessageBox.Show(ApplicationStrings.NETWORK_OFFLINE);
}
}
private void DoTransactionRefresh()
{
if (_progress == null)
{
_progress = new ProgressIndicator();
}
// after all data is sent do full reload
App.ViewModel.LoadMore = true;
App.ViewModel.ShowButton = false;
ApplicationBl<Transaction>.GetDataLoadingCompleted += OnTransactionsRefreshCompleted;
ApplicationBl<Transaction>.GetData(0, 10);
_progress.Show();
}
OnTransactionRefreshCompleted we delete all transaction data in local database and get the latest 10 transactions. We don't need all the data, and this way user have synced data. He can always load more data by taping load more at the end of transaction list. It's something similar like those twitter apps.
private void OnTransactionsRefreshCompleted(object entities)
{
if (entities is IList<Transaction>)
{
// save transactions
var transactions = (IList<Transaction>)entities;
DatabaseBl.TruncateTable<Transaction>();
DatabaseBl.Save(transactions);
((MainViewModel) DataContext).Transactions.Clear();
//reset offset
_offset = 1;
//update list with new transactions
App.ViewModel.LoadDataForTransactions(transactions);
App.ViewModel.LoadMore = false;
App.ViewModel.ShowButton = true;
}
if (entities == null)
{
App.ViewModel.ShowButton = false;
App.ViewModel.LoadMore = false;
}
// hide progress
_progress.Hide();
// remove event handler
ApplicationBl<Transaction>.GetDataLoadingCompleted -= OnTransactionsRefreshCompleted;
}
Caveat - I haven't tried this with windows phone development but use of GUID identities is something I usually do when faced with similar situations - eg creating records when I only have a one-way connection to the database - such as via a message bus or queue.
It works fine, albeit with a minor penalty in record sizes, and can also cause less performant indexes. I suggest you just give it a shot.
I'm working with the Beta 2 version of Visual Studio 2010 to get some advanced learning using WF4. I've been working with the SqlTracking Sample in the WF_WCF_Samples SDK, and have gotten a pretty good understanding of how to emit and store tracking data in a SQL Database, but haven't seen anything on how to query the data when needed. Does anyone know if there are any .Net classes that are to be used for querying the tracking data, and if so are there any known samples, tutorials, or articles that describe how to query the tracking data?
According to Matt Winkler, from the Microsoft WF4 Team, there isn't any built in API for querying the tracking data, the developer must write his/her own.
These can help:
WorkflowInstanceQuery Class
Workflow Tracking and Tracing
Tracking Participants in .NET 4 Beta 1
Old question, I know, but there is actually a more or less official API in AppFabric: Windows Server AppFabric Class Library
You'll have to find the actual DLL's in %SystemRoot%\AppFabric (after installing AppFabric, of course). Pretty weird place to put it.
The key classes to look are at are SqlInstanceQueryProvider, InstanceQueryExecuteArgs. The query API is asynchronous and can be used something like this (C#):
public InstanceInfo GetWorkflowInstanceInformation(Guid workflowInstanceId, string connectionString)
{
var instanceQueryProvider = new SqlInstanceQueryProvider();
// Connection string to the instance store needs to be set like this:
var parameters = new NameValueCollection()
{
{"connectionString", connectionString}
};
instanceQueryProvider.Initialize("Provider", parameters);
var queryArgs = new InstanceQueryExecuteArgs()
{
InstanceId = new List<Guid>() { workflowInstanceId }
};
// Total ruin the asynchronous advantages and use a Mutex to lock on.
var waitEvent = new ManualResetEvent(false);
IEnumerable<InstanceInfo> retrievedInstanceInfos = null;
var query = instanceQueryProvider.CreateInstanceQuery();
query.BeginExecuteQuery(
queryArgs,
TimeSpan.FromSeconds(10),
ar =>
{
lock (synchronizer)
{
retrievedInstanceInfos = query.EndExecuteQuery(ar).ToList();
}
waitEvent.Set();
},
null);
var waitResult = waitEvent.WaitOne(5000);
if (waitResult)
{
List<InstanceInfo> instances = null;
lock (synchronizer)
{
if (retrievedInstanceInfos != null)
{
instances = retrievedInstanceInfos.ToList();
}
}
if (instances != null)
{
if (instances.Count() == 1)
{
return instances.Single();
}
if (!instances.Any())
{
Log.Warning("Request for non-existing WorkflowInstanceInfo: {0}.", workflowInstanceId);
return null;
}
Log.Error("More than one(!) WorkflowInstanceInfo for id: {0}.", workflowInstanceId);
}
}
Log.Error("Time out retrieving information for id: {0}.", workflowInstanceId);
return null;
}
And just to clarify - this does NOT give you access to the tracking data, which are stored in the Monitoring Database. This API is only for the Persistence Database.