In Toad for Oracle 12:
I make various queries on-the-fly to analyze data. For example:
select
worktype,
count(*) as count
from
maximo.workorder
group by
worktype
In the query resultset, it would be helpful if I could toggle a totals row on/off to quickly analyze the results.
Example: get the grand total of the COUNT column. Or, in other queries, play around with count, max, standard deviation, variance, etc.
It would be the equivalent to the MS Access totals tool (not to be confused with GROUP BY):
Is there an way to do that using the UI in Toad 12?
You can, I believe. I don't have TOAD 12 any more, but - give it a try:
select all cells you're interested in
right-click
from the menu, select "Calculate selected cells"
check the result
note that - if result list contains a lot of columns and you're calculating the ones far on the right, you'll have to scroll left to see the result
Related
In a foxpro table, I have 4 fields: ordernum, orderdate, ordertotal, custpk.
What is the foxpro select statement to return all of those fields from the most recent order for each custpk?
Your table structure and what you want is not very clear. How do you define the "most recent order" for a given customer? Let's assume orderNum is primary key in that table and you decide "most recent order" for a given customer by the row having highest orderNum per customer. Then it is easy and a single VFP select command is sufficient (actually there are many SQL select variations that would return you the result you need. Below is one of those that would be most efficient):
select * from myTable ;
where orderNum in ;
(select max(orderNum) from myTable group by custPK)
BTW, VFP's SQL is close to ANSI-SQL 92.
I need a foxpro command, not SQL database
Let's start by saying that you will not get what you want with a single Foxpro command line.
Instead the most expedient manner to get this in Foxpro is to use Foxpro's SQL Query commands run on a FP/VFP Data Table and get the resultant data into a data table/cursor.
SELECT CustPk, OrderNum, OrderDate, OrderTotal FROM MyDataTable ORDER BY CustPk, OrderDate Desc INTO CURSOR OrderedData READWRITE
* --- Now ordered data is in memory cursor, use it however you need ---
* --- Each Customer's Most Recent Order will be first record ---
SELECT OrderedData
* --- You might just want to BROWSE the results to understand how the data appears ---
* --- Or you can do anything else with the data ---
< do whatever >
There are ways to get ONLY the most recent order per customer, but start here and see if that gets you what you need.
BTW: you might want to spend some time looking over the Free, On-line VFP tutorial videos at:
Free Visual Foxpro Videos
Good Luck
I would like to find how to do it with just the sql query command
Well I did it with just a SQL Query command, but you had to follow up with some other code.
EDIT: Oops: I forgot that Custpk is integer.
I am modifying the SQL Query syntax to take that into account.
Although it is still not a ONE LINE method, another way might be:
* --- Get reference records of only customer's last order date ---
SELECT STR(CustPk) AS cCustPk, MAX(OrderDate) AS LstOrdrDt FROM MyDataTable GROUP BY CustPk INTO CURSOR LstOrds READWRITE
* --- Now use that to get the complete last order for each customer ---
SELECT * FROM MyDataTable WHERE (STR(CustPk) + DTOS(OrderDate)) IN (SELECT cCustPk + DTOS(LstOrdrDt) AS KeyExp FROM LstOrds) INTO CURSOR OnlyLstOrds READWRITE
* --- Now you have ONLY the last orders ---
SELECT OnlyLstOrds
< do whatever >
NOTE the above code is off-the-cuff and not tested, so there might be a typo in it which you can fix.
Also note that one of the Free, On-line VFP Tutorial Videos at that site is named: FoxPro and the SQL Language
Good Luck
I’m a longtime MSSQL developer who finds himself back in PL/SQL for the first time since Oracle 7. I’m looking for some tuning advice re a large export stored procedure, which is sporadically and not very reproducably running slow at certain points. This happens around some static working tables which it truncates, fills and uses as part of the export. The code in outline typically looks like this:
create or replace Procedure BigMultiPurposeExport as (
-- about 2000 lines of other code
INSERT WORK_TABLE_5 SELECT WHATEVER1 FROM WHEREVER1;
INSERT WORK_TABLE_5 SELECT WHATEVER2 FROM WHEREVER2;
INSERT WORK_TABLE_5 SELECT WHATEVER3 FROM WHEREVER3;
INSERT WORK_TABLE_5 SELECT WHATEVER4 FROM WHEREVER4;
-- WORK_TABLE_5 now has 0 to ~500k rows whose content can vary drastically from run to run
-- e.g. one hourly run exports 3 whale sightings, next exports all tourist visits to Kenya this decade
-- about 1000 lines of other code
INSERT OUTPUT_TABLE_3
SELECT THIS, THAT, THE_OTHER
FROM BUSINESS_TABLE_1 BT1
INNER JOIN BUSINESS_TABLE_2 ON etc -- typical join on indexed columns
INNER JOIN BUSINESS_TABLE_3 ON etc -- typical join on indexed columns
INNER JOIN BUSINESS_TABLE_4 ON etc -- typical join on indexed columns
LEFT OUTER JOIN WORK_TABLE_1 ON etc -- typical join on indexed columns
LEFT OUTER JOIN WORK_TABLE_2 ON etc -- typical join on indexed columns
LEFT OUTER JOIN WORK_TABLE_3 ON etc -- typical join on indexed columns
LEFT OUTER JOIN WORK_TABLE_4 ON etc -- typical join on indexed columns
LEFT OUTER JOIN WORK_TABLE_5 WT5 ON BT1.ID = WT5.BT1_ID AND WT5.RECORD_TYPE = 21
-- join above is now supported by indexes on BUSINESS_TABLE_1 (ID) and WORK_TABLE_5 (BT1_ID, RECORD_TYPE), originally wasn't
LEFT OUTER JOIN WORK_TABLE_6 ON etc -- typical join on indexed columns
LEFT OUTER JOIN WORK_TABLE_7 ON etc -- typical join on indexed columns
-- about 4000 lines of other code
)
That final insert into OUTPUT_TABLE_3 usually runs in under 10 seconds, but once in a while on certain customer servers it times out at our default 99 minutes. Then we have them take the tiemout off and run it on Friday night, and it finishes but takes 16 hours.
I narrowed the problem down to the join to WORK_TABLE_5, which had no index support, and put an index on the join terms. The next run took 4 seconds. But success has been intermittent, the customer occasionally gets some slow runs when they drastically change their export selection (i.e. drastically change the data in WORK_TABLE_5). And if we update statistics and rebuild indexes after a timed out export, it runs fine at the next attempt.
So, I am wondering about how best to handle truncating/filling static work tables with static indexes, statistics updated overnight, and a stored procedure compiled when the statistics are nothing like runtime.
I have a few general questions about things I'd like to understand better:
Is the nature of the data in the work table going to substantially effect the query plan? Does Oracle form its query plan when you compile the stored procedure? Could we get a highly inappropriate query plan if we compile the stored procedure with the table empty then use a table with 500k rows at runtime?
I expect that if this were an ad-hoc script then updating statistics on the problem table just before selecting from it would eliminate the sporadic slowdowns. But what if I were to update statistics inside the stored procedure, which is compiled with different statistics from runtime?
Anything else you'd like to add...
Thanks for any advice. I hope my MSSQL preconceptions haven't made me too far off base.
This is happening in Oracle 11g, but the code is deployed to assorted customers using Oracle 10 through 12 and I'd like to cater to all of those if possible.
-- Joel
Huge differences in table or index sizes can most definitely cause performance problems. The solution is to add statistics gathering to the procedure instead of relying on the default statistics jobs.
If you've been away from Oracle since version 7, the most important new feature is the Cost Based Optimizer. Oracle now builds query execution plans based on the optimizer statistics of tables, indexes, columns, expressions, system statistics, outlines, directives, dynamic sampling, etc. If you're a full time Oracle developer you should probably spend a day reading about optimizer statistics. Start with Managing Optimizer Statistics and DBMS_STATS in the official documentation.
Eventually the stored procedure should look like this:
--1: Insert into working tables.
insert into work_table...
--2: Gather statistics on working tables.
dbms_stats.gather_table_stats('SCHEMA_NAME', 'WORK_TABLE', ...);
--3: Use working tables.
insert into other_table select * from work_table...
There are so many statistics features it's hard to know exactly what parameters to use in that second step above. Here are some guesses about some features you might find useful:
DEGREE - One reason people avoid gathering statistics inside a process is the time is takes. You can significantly improve the run time by setting the degree. Although this also uses significantly more resources.
NO_INVALIDATE - It can be tricky to know when exactly are the statistics "set" for a query. Gathering statistics usually quickly invalidates execution plans that were based on old statistics. But not always. If you want to be 100% sure that the next query is using the latest statistics you want to set NO_INVALIDATE=>FALSE.
ESTIMATE_PERCENT In 11g and above you definitely want to use the default, which uses a faster algorithm. In 10g and below you may need to set the value to something low to make the gathering fast enough.
Although Oracle 10g and above comes with default statistics gathering jobs you cannot rely on them for a few reasons:
They are scheduled and may not run at the right time. If a process significantly changes the data then new stats are needed right away, not at 10 PM. If there are a lot of tables that need to be analyzed the job may not get to them all in one day.
Many DBAs disable the jobs. This is ridiculous and almost always a mistake. But you'll find many DBAs that disabled the job because they think they can do it better. Instead of working with the auto tasks, and settings preferences, many DBAs like to throw the whole thing out and replace it with a custom procedure that rots over time.
I have the following query (generated by Entity Framework with standard paging. This is the inner query and I added the TOP 438 part):
SELECT TOP 438 [Extent1].[Id] AS [Id],
[Extent1].[MemberType] AS [MemberType],
[Extent1].[FullName] AS [FullName],
[Extent1].[Image] AS [Image],
row_number() OVER (ORDER BY [Extent1].[FullName] ASC) AS [row_number]
FROM [dbo].[ShowMembers] AS [Extent1]
WHERE 3 = CAST( [Extent1].[MemberType] AS int)
ShowMembers table has about 11K rows, but only 438 with MemberType == 3. The 'Image' column is of type nvarchar(2000) that holds the URL to the image on a CDN. If I include this column in the query (only in SELECT part), the query chokes up somehow and generates result in a range between 2-30 seconds (it differs in different runs). If I comment out that column, query runs fast as expected. If I include the 'Image' column, but comment out the row_number column, query also runs fast as expected.
Obviously, I've been too liberal with the size of the URL, so I started playing around with the size. I found out that if I set the Image column to nvarchar(884), then the query runs fast as expected. If I set it up to 885 it's slow again.
This is not bound to one column, but to the size of all columns in the SELECT statement. If I just increase the size by one, performance differences are obvious.
I am not a DB expert, so any advice is welcomed.
PS In local SQL Server 2012 Express there are no performance issues.
PPS Running the query with OFFSET 0 ROWS FETCH NEXT 438 ROWS ONLY (without the row_count column of course) is also slow.
Row_number has to sort all the rows to get you things in the order you want. Adding a larger column into the result set implies that it all get sorted and thus is much slower/does more IO. You can see this, btw, if you enable "set statistics io on" and "set statistics time on" in SSMS when debugging problems like this. It will give you some insight into the number of IOs and other operations happening at runtime in the query:
https://learn.microsoft.com/en-us/sql/t-sql/statements/set-statistics-io-transact-sql?view=sql-server-2017
In terms of what you can do to make this query go faster, I encourage you to think about some things that may change the design of your database schema a bit. First, consider whether you actually need the rows sorted in a specific order at all. If you don't need things in order, it will be cheaper to iterate over them without the row_number (by a measurable amount). So, if you just want to conceptually iterate over each entry once, you can do that by doing an order by something more static that is still monotonic such as the identity column. Second, if you do need to have things in sorted order, then consider whether they are changing frequently/infrequently. If it is infrequent, it may be possible to just compute and persist a column value into each row that has the relative order that you want (and update it each time you modify the table). In this model, you could index the new column and then request things in that order (in the top-level order by in the query - row_number not needed). If you do need things dynamically computed like you are doing and you need things in an exact order all the time, your final option is to move the URL to a second table and join with it after the row_number. This will avoid the sort being "wide" in the computation of row_number.
Best of luck to you
I have a dataset that has about 1,100,000 rows.
When I load this into my jqGrid, SQL Profiler tells me it takes 29.7 seconds just to return the count of records and then a further 29.8 seconds to return the data to display in the grid.
Please see below the SQL that does the row count against my SQL Server table.
SELECT
[GroupBy1].[A1] AS [C1]
FROM ( SELECT
COUNT(1) AS [A1]
FROM [dbo].[vw_ProductSearch_FULL] AS [Extent1]
) AS [GroupBy1]
Can anyone suggest how to improve the performance of this "count" query that is generated by jqGrid?
We need more information about your database in order to recommend improvements to your query. But as Oleg said, you may not need to query for the count.
As to the data in the grid, you have seen that having ~1 million rows in the grid just does not work well. I suggest you either use Pagination or True Scrolling Rows to only load a small subset of the rows at any given time. This should get your performance back up to an acceptable level.
We have a table with a 'price' field of type NUMBER(20,7).. In TOAD I do this:
update mytable set price = 1234567890123.1234567;
Then I do this select:
select price, to_char(price) from mytable
PRICE TO_CHAR(PRICE)
1234567890123.12 "1234567890123.1234567"
Question is, why does TOAD truncate the result when displaying the NUMBER(20,7) field? The data is obviously there as it prints out with to_char.
??
Toad limits numbers in the data grid to 15 digits. I believe this is because excel limits numbers to 15 digits as well (or used to limit them). You can turn on "scientific notation" in options -> data grids -> data, check box Display large numbers in scientific notation. That won't probably help, either.
-> Right click on the data grid
-> Select "Grid Options..."
-> Under "Data Grids" chose "Data"
-> Uncheck "Display large numbers in Scientific Notation"
After doing these you need to close and open TOAD again. This worked for me in TOAD Version 8.6.0.38
I had a similar problem, Toad was truncating/rounding my numbers when being displayed in grid view.
I found out that by using the Execute Statement button or by pressing F9 to run the query then the truncation/rounding would not be performed.
I hope that helps.