The command to enable versioning (part of what Oracle calls Workspace Management) in Oracle (DBMS_WM.EnableVersioning) creates non-materialized views, which cannot be indexed. Will this kill performance, or will the indexes for the _AUX, _LT, and _LCK tables be used when the views are queried?
Are there significant performance issues in addition to indexing when enabling versioning?
I am using Oracle 11g.
As with most things, it depends.
Do you have queries today that need to do table scans to fetch all their data? Or is everything going to go through an index?
What is the use case? Are you using Workspace Manager to support long-running transactions? Or to maintain history data in a single LIVE workspace?
How frequently do rows change? How many versions of a row are you planning to keep?
If you have existing queries that will do table scans, the table is rebuilt every night, and you plan on keeping history data forever, you're likely going to have major performance issues. If all your queries use indexes to access data, rows change infrequently, and you just intend to retain a few versions of history, the indexes on the underlying tables should be sufficient.
We've used Workspace Manager to maintain history on relatively slowly changing tables forever as well as relatively fast changing tables for a month. And we've used it to maintain discrete savepoints across tables in a few applications so that users can permanently save the state of application data at interesting points in time. In general, we've been satisfied with performance though complex queries will occasionally go off into the weeds when the optimizer gets confused.
Since you're on 11g, you may also consider Total Recall. It's an extra-cost option on top of the enterprise license but it provides a much more efficient architecture for tracking changes to data over time assuming that you intend to store all changes for a fixed period of time. On the other hand, you're more limited in the DDL you can issue without causing history to be discarded which tends to be a rather serious constrain in the applications I've worked on.
Related
I have one table which are number of rows '7515966' and this table depend on another tables. We create View for generating SSRS reports.
Now size of View is increase so that performance issue occur on report.
We start archiving data for large table. but i can't understand which methodology use please guide us..
Thank you...
Table partitioning in 2012 is only available in Enterprise Edition. See https://msdn.microsoft.com/en-us/library/cc645993(v=sql.110).aspx for details on what's available for each edition.
7million rows is not a lot of rows for SQL Server, we routinely deal with billions of rows. However, as your rows get into the 10s of millions range, you'll probably expose various performance gaps in your system. E.g. are your queries efficiently written so they only touch the rows they need, do you have the right indexes, are statistics up to date, is tempdb optimized, etc...
One common weak link in 9 out of 10 databases (regardless of make) I've worked with is the storage subsystem. Is yours able to keep up with the large data set you need to work with. Storage for databases should be designed and configured based on throughput, concurrency and latency requirements first. Space generally the last thing to worry about once the other requirements, including HA/DR, are met.
If you have deficiencies in your current system, you can pay for the expensive enterprise edition and implement table partitioning but you will likely still suffer performance problems soon after, if not immediately.
Using Oracle 11.2.
Is there a standard approach to build an execution plan history like how an execution plan may change ofer time? I thought about tracking changes in dba_hist_sql_plan (plans are kept there for 10 days in our env.). But are all plans stored in dba_hist_sql_plan or does that depent on the awr config? Or is there a standard approach for doing that (may without dba_hist_sql_plan).
I'm rather new to Oracle so every idea is welcomed.
Not all plans are stored in the AWR. Generally, though, all the plans that you're interested in are. Few people are interested in tracking changes to query plans for queries that don't consume a lot of resources in a particular snapshot window. They're generally only interested in the behavior of the top resource consumers. You can adjust the N in the top N queries that AWR captures with each snapshot.
If you're looking at a single database, the simplest option is to keep data in the AWR as long as you're likely to need it. If you want data back beyond 10 days, the simplest option is to increase your AWR retention. AWR data doesn't consume that much disk space so it's generally easier to let Oracle retain more data than to write anything of your own.
Sometimes, people configure replication processes that take AWR data from many individual databases and write them to a single central database. That requires a decent amount of setup but it makes it much easier to compare information across databases (either to compare data among dev/ test/ staging/ production environments or to compare across multiple client-specific instances).
Oracle states that 'Invisible indexes are especially useful for testing the removal of an index before dropping it or using indexes temporarily without affecting the overall application.'
I don't understand why visibility is 'especially' useful for this, wouldn't making an index unusable be especially useful since DML operations are not maintained therefor it resembles dropping an index more so than making it simply invisible. I've never actually worked with this, I'm guessing that making an index invisible/visible is easier than making it usable/unusable because you have to rebuild an index somehow when you make it usable?
It is referring to the impact on your queries via statistics and the optimizer.
Many Oracle databases have complex schemas, user bases, as well as really large tables and indexes. Some even have very controlled schema statistics. Dropping a big index can be an expensive step (time-wise).
The statistics gatherer collects statistics to populate the data dictionary, which is used for the optimizer. Index stats are one of the key inputs to the Cost Based Optimizer. "Faking" the drop will cause the optimizer to act as if the index is gone, and you can then see the impact on the query plans. If you find that the drop wasn't such a good idea, you can immediately revert it. On the other hand, some indexes take hours to build, so you can see how it is valuable to be able to test it out first.
I have an app using an Oracle 11g database. I have a fairly large table (~50k rows) which I query thus:
SELECT omg, ponies FROM table WHERE x = 4
Field x was not indexed, I discovered. This query happens a lot, but the thing is that the performance wasn't too bad. Adding an index on x did make the queries approximately twice as fast, which is far less than I expected. On, say, MySQL, it would've made the query ten times faster, at the very least. (Edit: I did test this on MySQL, and there saw a huge difference.)
I'm suspecting Oracle adds some kind of automatic index when it detects that I query a non-indexed field often. Am I correct? I can find nothing even implying this in the docs.
As has already been indicated, Oracle11g does NOT dynamically build indexes based on prior experience. It is certainly possible and indeed happens often that adding an index under the right conditions will produce the order of magnitude improvement you note.
But as has also already been noted, 50K (seemingly short?) rows is nothing to Oracle. The Oracle database in fact has a great deal of intelligence that allows it to scan data without indexes most efficiently. Every new release of the Oracle RDBMS gets better at moving large amounts of data. I would suggest to you that the reason Oracle was so close to its "best" timing even without the index as compared to MySQL is that Oracle is just a more intelligent database under the covers.
However, the Oracle RDBMS does have many features that touch upon the subject area you have opened. For example:
10g introduced a feature called AUTOMATIC SQL TUNING which is exposed via a gui known as the SQL TUNING ADVISOR. This feature is intended to analyze queries on its own, in depth and includes the ability to do WHAT-IF analysis of alternative query plans. This includes simulation of indexes which do not actually exist. However, this would not explain any performance differences you have seen because the feature needs to be turned on and it does not actually build any indexes, it only makes recommendations for the DBA to make indexes, among other things.
11g includes AUTOMATIC STATISTICS GATHERING which when enabled will automatically collect statistics on database objects as it deems necessary based on activity on those objects.
Thus the Oracle RDBMS is doing what you have suggested, dynamically altering its environment on its own based on its experience with your workload over time in order to improve performance. Creating indexes on the fly is just not one of the things is does yet. As an aside, this has been hinted to by Oracle in private sevearl times so I figure it is in the works for some future release.
Does Oracle 11g automatically index fields frequently used for full table scans?
No.
In regards the MySQL issue, what storage engine you use can make a difference.
"MyISAM relies on the operating system for caching reads and writes to the data rows while InnoDB does this within the engine itself"
Oracle will cache the table/data rows, so it won't need to hit the disk. depending on the OS and hardware, there's a chance that MySQL MyISAM had to physically read the data off the disk each time.
~50K rows, depending greatly on how big each row is, could conceivably be stored in under 1000 blocks, which could be quickly read into the buffer cache by a full table scan (FTS) in under 50 multi-block reads.
Adding appropriate index(es) will allow queries on the table to scale smoothly as the data volume and/or access frequency goes up.
"Adding an index on x did make the
queries approximately twice as fast,
which is far less than I expected. On,
say, MySQL, it would've made the query
ten times faster, at the very least."
How many distinct values of X are there? Are they clustered in one part of the table or spread evenly throughout it?
Indexes are not some voodoo device: they must obey the laws of physics.
edit
"Duplicates could appear, but as it
is, there are none."
If that column has neither a unique constraint nor a unique index the optimizer will choose an execution path on the basis that there could be duplicate values in that column. This is the value of declaring the data model as accuratley as possible: the provision of metadata to the optimizer. Keeping the statistics up to date is also very useful in this regard.
You should have a look at the estimated execution plan for your query, before and after the index has been created. (Also, make sure that the statistics are up-to-date on your table.) That will tell you what exactly is happening and why performance is what it is.
50k rows is not that big of a table, so I wouldn't be surprised if the performance was decent even without the index. Thus adding the index to equation can't really bring much improvement to query execution speed.
Oracle seems to have a reputation for being more scalable than other RDBMSes. After working with it a bit, I can say that it's more complex than other RDBMSes, but I haven't really seen anything that makes it more scalable than other RDBMSes. But then again, I haven't really worked on it in a whole lot of depth.
What features does Oracle have that are more scalable?
Oracle's RAC architecture is what makes it scalable where it can load balance across nodes and parallel queries can be split up and pushed to other nodes for processing.
Some of the tricks like loading blocks from another node's buffer cache instead of going to disc make performance a lot more scalable.
Also, the maintainability of RAC with rolling upgrades help make the operation of a large system more sane.
There is also a different aspect of scalability - storage scalability. ASM makes increasing the storage capacity very straightforward. A well designed ASM based solution, should scale past the 100s of terabyte size without needing to do anything very special.
Whether these make Oracle more scalable than other RDBMSs, I don't know. But I think I would feel less happy about trying to scale up a non-Oracle database.
Cursor sharing is (or was) a big advantage over the competition.
Basically, the same query plan is used for matching queries. An application will have a standard set of queries it issue (eg get the orders for this customer id). The simple way is to treat every query individually, so if you see 'SELECT * FROM ORDERS WHERE CUSTOMER_ID = :b1', you look at whether table ORDERS has an index on CUSTOMER_ID etc. As a result, you can spend as much work looking up meta data to get a query plan as actually retrieving the data. With simple keyed lookups, a query plan is easy. Complex queries with multiple tables joined on skewed columns are harder.
Oracle has a cache of query plans, and older/less used plans are aged out as new ones are required.
If you don't cache query plans, there's a limit to how smart you can make your optimizer as the more smarts you code into it, the bigger impact you have on each query processed. Caching queries means you only incur that overhead the first time you see the query.
The 'downside' is that for cursor sharing to be effective you need to use bind variables. Some programmers don't realise that and write code that doesn't get shared and then complain that Oracle isn't as fast as mySQL.
Another advantage of Oracle is the UNDO log. As a change is done, the 'old version' of the data is written to an undo log. Other database keep old versions of the record in the same place as the record. This requires VACUUM style cleanup operations or you bump into space and organisation issues. This is most relevant in databases with high update or delete activity.
Also Oracle doesn't have a central lock registry. A lock bit is stored on each individual data record. SELECT doesn't take a lock. In databases where SELECT locks, you could have multiple users reading data and locking each other or preventing updates, introducing scalability limits. Other databases would lock a record when a SELECT was done to ensure that no-one else could change that data item (so it would be consistent if the same query or transaction looked at the table again). Oracle uses UNDO for its read consistency model (ie looking up the data as it appeared at a specific point in time).
Tom Kyte's "Expert Oracle Database Architecture" from Apress does a good job of describing Oracle's architecture, with some comparisons with other rDBMSs. Worth reading.