What are the differences between Connection#exec_params and Connection#exec_prepared? It seems like they both require the same kinds of input and they both perform the same action. Why would someone choose one versus the other?
You can use exec_prepared to optimize SQL query, see documentation.
exec_params lets you execute specified SQL query with binded parameters. exec_prepared lets you execute prepared (parsed, optimized etc) SQL query specified by its string identificator/name which you have got earlier.
If you make a lot of similar SQL select queries and difference is parameter values, you can make it effectively by preparing SQL statement once (you receive it identifier) and execute it with different parameters multiple times.
Related
Good morning.
I am making a function that connects to Oracle from JDBC and executes a query.
The number of query statements to be executed is about 50.
I looked for several examples, but most of them are examples that execute only one, and even this is only executed by putting the query in a variable.
I think it is impossible to put all 50 queries in a variable.
Among these, there is an option called allowMultiQueries=true in mysql, but it does not appear to be in oracle.
';', which means the end of the query syntax. Is there any other way to perform a multi-line query other than parsing based on?
I have using the google-cloud-bigquery gem (version 0.20.2, can't upgrade at the moment).
I have an audit dataset which contains many tables of the following format:
audit.some_table20170101
audit.some_table20170102
audit.some_table20170103
etc..
I am trying to run a query which will scan all of these tables, and give me the last value of field some_field.
What I was going for is using the tables wildcard:
FROM audit.some_table*
and to hopefully
SELECT LAST(some_field) AS last_some_field
While using Bigquery web console, I was able to do so by
using backticks (FROM `audit.some_table*`), but doing the same programmatically with the gem causes a Google::Cloud::InvalidArgumentError: invalid: Invalid table name: `audit.some_table*`
Even in the web console, when I try to use the LAST command it requires using legacy SQL, which then gives an error due to the backticks of the previous section. If I disable legacy sql, LAST is no available anymore (unfamiliar command) and then I have to order by a timestamp column descending and limit 1.
Any ideas how to solve these problems and to be able to query using the above mention gem and version?
LAST is only meaningful when there is an order. Tables in BigQuery do not have inherit ordering, and if you run SELECT * FROM table you may get results in different order every time. Therefore the right thing to do it is to use ORDER BY some_value DESC LIMIT 1 construct.
The wildcard tables are indeed only available in Standard SQL, to get similar functionality with Legacy SQL you can use TABLE_DATE_RANGE function in FROM clause.
I want to use the count(*) result from a Hive Query as input for a second hive query. The query is simplified as:
set LIM = SELECT count(*) from default.mytable* 0.8;
select * from default.mytable LIMIT ${hiveconf:LIM};
The above code will lead to an error as the first query does not get executed and there the LIM variable will not get substituted with a numeric value.
Is there a way to force Hive to substitute the variable LIM so that I have a numeric value in the second query?
## WARNING - verbose explanation follows; the short answer is "no way" ##
In terms of IT Architecture, this kind of tricks is not done in the database tier but in the application tier.
Since I don't know nuthin' about your Teradata stack (fondly nicknamed "taratata" by some of your French-speaking colleagues) I'll take the Oracle stack as an example.
A. Inside a PL/SQL block, you can retrieve the (scalar) result of a query into a variable, and use it later -- as an input bind variable in a prepared statement, or as a way to build dynamically a string to be parsed dynamically as a SQL query. That PL/SQL block an "application", with application logic of arbitrary complexity; it just happens to run inside an Oracle session, on the same host that also runs the database tier.
B. Inside the SQL*Plus client (and maybe compatible tools e.g. SQL Developer) you can use a weird syntax to retrieve a value in a kind of macro-variable, that can be used to stuff the value as-is in further SQL queries. That trick allows some crude "application" logic to be applied to an otherwise static SQL script, client-side. But that is clearly a non-portable trick.
Bottom line - since Hive has no procedural language, and will probably (hopefully) never have one, the best way to do what you want would be to develop your own custom Hive client all by yourself, with whatever business logic you want. After all, there must be thousands of people around the world who are developing Java code to access Hive with JDBC, so you would not be alone...
Well.. you can do that if you are comfy with writing a shell script.
Take the query output and store it into a variable & use the variable for your second query.
I have an Oracle bind query that is extremely slow (about 2 minutes) when it executes in my C# program but runs very quickly in SQL Developer. It has two parameters that hit the tables index:
select t.Field1, t.Field2
from theTable t
where t.key1=:key1
and t.key2=:key2
Also, if I remove the bind variables and create dynamic sql, it runs just like it does in SQL Developer.
Any suggestion?
BTW, I'm using ODP.
If you are replacing the bind variables with static varibles in sql developer, then you're not really running the same test. Make sure you use the bind varibles, and if it's also slow you're just getting bit by a bad cached execution plan. Updating the stats on that table should resolve it.
However if you are actually using bind variables in sql developers then keep reading. The TLDR version is that parameters that ODP.net run under sometimes cause a slightly more pessimistic approach. Start with updating the stats, but have your dba capture the execution plan under both scenarios and compare to confirm.
I'm reposting my answer from here: https://stackoverflow.com/a/14712992/852208
I considered flagging yours as a duplicate but your title is a little more concise since it identifies the query does run fast in sql developer. I'll welcome advice on handling in another manner.
Adding the following to your config will send odp.net tracing info to a log file:
This will probably only be helpful if you can find a large gap in time. Chances are rows are actually coming in, just at a slower pace.
Try adding "enlist=false" to your connection string. I don't consider this a solution since it effecitively disables distributed transactions but it should help you isolate the issue. You can get a little bit more information from an oracle forumns post:
From an ODP perspective, all we can really point out is that the
behavior occurs when OCI_ATR_EXTERNAL_NAME and OCI_ATR_INTERNAL_NAME
are set on the underlying OCI connection (which is what happens when
distrib tx support is enabled).
I'd guess what you're not seeing is that the execution plan is actually different (meaning the actual performance hit is actually occuring on the server) between the odp.net call and the sql developer call. Have your dba trace the connection and obtain execution plans from both the odp.net call and the call straight from SQL Developer (or with the enlist=false parameter).
If you confirm different execution plans or if you want to take a preemptive shot in the dark, update the statistics on the related tables. In my case this corrected the issue, indicating that execution plan generation doesn't really follow different rules for the different types of connections but that the cost analysis is just slighly more pesimistic when a distributed transaction might be involved. Query hints to force an execution plan are also an option but only as a last resort.
Finally, it could be a network issue. If your odp.net install is using a fresh oracle home (which I would expect unless you did some post-install configuring) then the tnsnames.ora could be different. Host names in tnsnams might not be fully qualified, creating more delays resolving the server. I'd only expect the first attempt (and not subsequent attempts) to be slow in this case so I don't think it's the issue but I thought it should be mentioned.
Are the parameters bound to the correct data type in C#? Are the columns key1 and key2 numbers, but the parameters :key1 and :key2 are strings? If so, the query may return the correct results but will require implicit conversion. That implicit conversion is like using a function to_char(key1), which prevents an index from being used.
Please also check what is the number of rows returned by the query. If the number is big then possibly C# is fetching all rows and the other tool first pocket only. Fetching all rows may require many more disk reads in that case, which is slower. To check this try to run in SQL Developer:
SELECT COUNT(*) FROM (
select t.Field1, t.Field2
from theTable t
where t.key1=:key1
and t.key2=:key2
)
The above query should fetch the maximum number of database blocks.
Nice tool in such cases is tkprof utility which shows SQL execution plan which may be different in cases above (however it should not be).
It is also possible that you have accidentally connected to different databases. In such cases it is nice to compare results of queries.
Since you are raising "Bind is slow" I assume you have checked the SQL without binds and it was fast. In 99% using binds makes things better. Please check if query with constants will run fast. If yes than problem may be implicit conversion of key1 or key2 column (ex. t.key1 is a number and :key1 is a string).
I have a vey huge query. It is rather large, so i will not post it here(it has 6 levels of nested queries with ordering and grouping). Query has 2 parameters that are passed to it via PreparedStatement.setString(index, value). When I execute my query through SQL Developer(replacing query parameters to actual values before it by hand) the query runs about 10 seconds and return approximately 15000 rows. But when I try to run it through java program using PreparedStament with varibales it fails with ORA-01652(unable to extend temp segment). I have tried to use simple Statement from java program - it works fine. Also when I use preparedStatement without variables(don't use setString(), but specify parameters by hand) it works fine too.
So, I suspect that problem is in PreparedStatemnt parameters.
How does the mechanism of that parameters work? Why simple statement works fine but prepared one fails?
You're probably running into issues with bind variable peeking.
For the same query, the best plan can be significantly different depending on the actual bind variables. In 10g, Oracle builds the execution plan based on the first set of bind variables used. 11g mostly fixed this problem with adaptive cursor sharing, a feature that creates multiple plans for different bind variables.
Here are some ideas for solving this problem:
Use literals This isn't always as bad as people assume. If the good version of your query runs in 10 seconds, the overhead of hard-parsing the query will be negligible. But you may need to be careful to avoid SQL injection.
Force a hard-parse There are a few ways to force Oracle to hard-parse every query. One method is to call DBMS_STATS with NO_INVALIDATE=>FALSE on one of the tables in the query.
Disable bind-variable peeking / hints You can do this by removing the relevant histograms, or using one of the parameters in the link provided by OldProgrammer. This will stabilize your plan, but will not necessarily pick the correct plan. You may also need to use hints to pick the right plan. But then you may not have the right plan for every combination of inputs.
Upgrade to 11g This may not be an option, but this issue is another good reason to start planning an upgrade.