I need to have multiple logins and query executions into an Oracle db, 10 users per process, 10 processes per PC.
I was thinking that I would create 10 threads, one thread per user login.
Is this feasible? Any advice is appreciated.
Very new to threads.
Update:
Thanks for all the comments and answers.
Here are some additional details:
Using Oracle 10.2, Delphi XE, and dbExpress components created on the fly.
Our design is to run 10 processes per machine and simulate 10 user-logins per process. Each login is within its own thread (actually I need to have two logins in each thread, so I am actually creating 200 sessions per machine).
For this simulation exercise, after establishing a connection, each thread retrieves a bunch of data by calling several stored procedures within a loop. For each stored procedure I create a TSQLProcedure object on the fly and close and then free it after using it. Now I am getting ORA1000 Max Cursors exceeded, which I don't understand since I close and free each sp object.
Changing the settings on the server side is out of the question. I saw some documentation that says that on the application side you can set RELEASE_CURSOR=YES. I am guessing that it's an option set at the procedure level.
Yes, it is feasible. You may need a thread for each session you need (see here for an explanation), and you have to ensure OCI is called in a thread safe way, how to do it depends on the library you use to call OCI, if you don't call OCI directly.
Yes it is feasible. Remember that the UI runs on its own thread and can't be accessed directly by the other threads. Also remember you can't share state between threads unless you secure it. This is a start. Here an example on using threads with databases and the dbGo library. I suggest you give it a try and come back if you have specific questions.
Related
Is there any problem on using many open connections at the same time from different threads?
From what I've read it's thread safe by default, but, can this be hurting performance rather than improving it?
Having multiple connection is not a problem, the only thing to keep in mind is that SQLite does not support concurrency of multiple write transactions. From the SQlite site:
SQLite supports an unlimited number of simultaneous readers, but it will only allow one writer at any instant in time. For many situations, this is not a problem. Writer queue up. Each application does its database work quickly and moves on, and no lock lasts for more than a few dozen milliseconds. But there are some applications that require more concurrency, and those applications may need to seek a different solution.
SQLite is an "untypical" database management system: in practice it is a library that offers SQL as language to access a simple "database-in-a-file", and a few other functionalities of DBMSs. For instance, it has no real concurrency control (it uses the Operating Systems functions to lock the db file).
So, if you need concurrent insertions into a database, you should use something else, for instance PostgreSQL.
The documentation say:
A connection can only be used from within the thread that created it.
Moving connections between threads or creating queries from a
different thread is not supported.
In addition, the third party libraries used by the QSqlDrivers can
impose further restrictions on using the SQL Module in a multithreaded
program. Consult the manual of your database client for more
information.
It is mean you have to create connection to database which will be linking with parent thread. At docs of QSqlDatabase class you can see description:
The QSqlDatabase class represents a connection to a database.
The QSqlDatabase class provides an interface for accessing a database
through a connection. An instance of QSqlDatabase represents the
connection. The connection provides access to the database via one of
the supported database drivers, which are derived from QSqlDriver.
Create a connection (i.e., an instance of QSqlDatabase) by calling one
of the static addDatabase() functions, where you specify the driver or
type of driver to use (i.e., what kind of database will you access?)
and a connection name.
Using static addDatabase() function is way to create connection.
But as Renzo said SQLite does not support multiple write transactions at the same time. So you need some mechanisms(wrapper) for synchronizing threads like task queue using low-level mutex or something like that. More information you can see at docs.
I am experiencing very low performance in my web application in which trivial HTTP requests take dozens of seconds to be processed. Tracing down the application code I discovered the majority of time is spent executing the first DB query, even if it is as simple as a SELECT on a single row-single column table. This happens for every HTTP request, independently from the query performed. After this first pathological DB interaction the remaining queries go smoothly.
I am using Hibernate on top of an Oracle DB (using jdbc).
It is not a problem of connection pool since I am successfully using Hibernate-c3p0, neither it seems to be related to Oracle itself, because all query returns immediately if performed directly on DB.
Furthermore, Hibernate SessionFactory is correctly created only once, at application start up time and concurrency is not a problem at all since tests have been done with single user.
Finally, my DB IP address is correctly resolved in my application server /etc/hosts so that even DNS related issues can be discarded (I am using two distinct virtual machines, DB and APP server).
I do not know what to look for, any help?
This sounds like your session factory object is being spun up on the first query. Generally I try to initialize the session factory on application startup to avoid this when issuing the first query because generally the user can see this slowdown. When doing it up front in application startup you will avoid this.
I am in process of developing an MFC based Windows based application, using PostgreSQL which would perform
Fetches information from the UI
Performs some logic and store related information to the database
The stored information has to be send immediately OR at schedule interval (ex. at 5:00 on xyz date) over the network
Currently, we have developed a dispacher mechanism (thread ), which constantly polls the database for new information inserted in the database. The thread fetches the information and send to the network module.
But, I feel this is not the correct approach as
Polling every time is a overhead. There can be times when there is nothing to execute
It is not real time , because we poll after every 5 seconds
So
Is there a way to send a trigger to my network module as soon as information is updated in database?
Or any better way to achieve this task?
Thanks in advance.
You can use the listen/notify feature of PostgreSQL for this.
http://www.postgresql.org/docs/current/static/sql-listen.html
http://www.postgresql.org/docs/current/static/sql-notify.html
The clients interested in the messages would execute a listen statement and the trigger would then notify them.
I don't use C# so, but according to the manual you can retrieve the messages in an asynchronous manner - which still involves some "lightweight" polling as the notification message is only sent as part of the answer of the server. The manual claims that running an "empty" statement (such as ;) will be enough. Using Java/JDBC I used a simple select 42 which doesn't impose a big workload on the server as no tables are touched.
This polling is defintely faster and more scalable than actually retrieving the table's data.
Yes you are right #RDX, you shouldnt poll it every time rather you could write a trigger in Postgres and from that trigger try calling a java program which could be seen in the below thread.
Calling java pgm from Postgres trigger
Here: http://download.oracle.com/docs/html/A95907_01/diff_uni.htm#1077398
I found that on Windows Oracle is thread based, while on Unix this is process based. Why it is like that?
What's more, there are many Oracle processes http://www.adp-gmbh.ch/ora/concepts/processes/index.html regardless the system.
Why log writer and db writer are implemented as processes... and the query execution is done using threads (windows) or processes (unix).
Oracle makes use of a SGA shared memory area to store information that is (and has to be) accessible to all sessions/transactions. For example, when a row is locked, that lock is in memory (as an attribute of the row) and all the other transactions need to see it is locked.
In windows a thread cannot access another process's memory
threads cannot access memory that
belongs to another process, which
protects a process from being
corrupted by another process.
As such, in Windows Oracle must be a single process with multiple threads.
On OS's supporting the sharing of memory between processes then it is less work for Oracle to work as a multi-process architecture and leave the process management to the OS.
Oracle runs a number of background threads/processes to do work that is (or can be) asynchronous to the other processes. That way those can continue even when other processes/threads are blocked or busy.
See this answer I posted earlier on in similar vein to this question 'What is process and thread?'. Windows makes extensive use of threads in this fashion. Unlike *nix/Linux based systems which are based on threads. And see here also, this link is a direct link(which is embedded in the first link I have given) to the explanation I gave on how Linux time divisions threads and processes.
Hope this helps,
Best regards,
Tom.
is there any way i can make my records in the database to be automatic. e.g i want a message to be sent to helpdesk if a requested service is not attended within 24 hours, without clicking anything.
technically it depends on the database you are using. if the database supports it, you could set up a scheduled job to scan the records and identify late services and email the helpdesk.
if the database doesn't support scheduled tasks then you could set up a client job on a timer to do the same thing.
This is what application software is for.
When the application saves to the database, the application also sends an email.
The traditional approach to this is to schedule a job (there are too many ways[1] to do that for me to go into details without knowing your server operating system, DBMS, and how much control you have to install or schedule programs on the server).
Your scheduled job would regularly check the database for records that have not been attended, and then take the appropriate action such as emailing the support team.
[1] Just so that this is not left completely unanswered; some DBMS (ex. SQL Server) have built in job scheduling facilities. You could run a Windows service on the server to do this. If not, you might consider running a Windows Service on one of your own servers to access the website (a great way to waste bandwidth).
Use a scheduler like this one, found on rufus site. You could program it to run, for instance, every hour, and make it do the job without human interaction.
I am a Java shop myself and I've been using quartz. It is quite good and usable if you can adjust to jruby.
I've never liked database or operating system based solutions, since you might not control them and often get asked to run on different environments.
Here's a very simple background job handler for Ruby:
codeforpeople.rubyforge.org/svn/bj/trunk/README
Easy to install and use. Fairly lightweight. It uses a SQL backend for managing concurrency. Runs on multiple machines simultaneously if you need it to.