I am looking arround for prepared statements in Jython with JDBC (zxJDBC) / SQLite.
All I find is examples like this (book, Jython documentation):
qry = "select continent from country where name = ?"
cursor.executemany(qry,['Austria'])
However I always understood prepared statements as a way to let the Database optimize a statement without knowing the values inserted for the placeholders. One then receives a handle to this statement wich has been passed through the optimizer and can execute this with different values, saving the query optimization effort, thus I would expect that the following approach is better:
qry = cursor.prepare("SELECT * FROM `table` WHERE `field`=? AND `field2`=?") #usually mor complex queries
qry.execute(cursor, ?, (val1, val2)??) #here is the problem
unfortunately I can't find examples of how to use the execute and the documentation is lacking.
PyStatement.execute(cursor, params, bindings)
Could you explain me what params is for and if my guess at how to use bindings is correct. My best guess for params would be, that they are the first parameter to the setInt-Method (or the like) from Javas JDBC, put that would be rather surprising. I expected to be Jython more short handed and for it to simply expect the parameters in the right(front to back) order.
Bonus: Is there a way to free a resultset? (Besides closing the cursor)
It looks like this extension uses the normal Python Database Specification v2
For execute it says:
A reference to the operation will be retained by the cursor. If the same operation object is passed in again, then the cursor can optimize its behavior. This is most effective for algorithms where the same operation is used, but different parameters are bound to it (many times).
And for executemany it says:
The same comments as for .execute() also apply accordingly to this method.
So if this drivers actually implements these suggested semantics, then just passing in the same string into execute or executemany should reuse the preparedstatement, however you might want to check the implementation (or ask the implementer) to be sure
Related
I know that this question is all over..
But Im really struggling to understand,
I see everywhere that functions cannot be used to perform crud operations on the db ( such as update statement ) which is not true.
Other than that, Basically the MAIN difference between them is that a procedure can have in and out parameters 0->n but a function has a return, and does not store the value in an out parameter..
Yea there are some small differences like the way you invoke it, if Im not wrong a function can be called within a select statement whereas a procedure cannot.
So to be honest, I dont really see any difference bewteen those two.
What should I answer if Im asked " Why would you choose function over procedure" ( or the opposite ). THANKS
Well, that's true (except for a typo, here):
function can be called within a select statement whereas a function cannot (bold part should be "procedure")
You'd choose function when there's something you want to return to caller. Use a procedure when you want to process something.
Functions - as you said - can be used in a select statement, which can be used in both SQL and PL/SQL. Procedures, on the other hand, require PL/SQL. It is way simpler to call a function than a procedure (presume f_today and p_today return sysdate; function as return value, procedure via its out parameter), e.g.
select f_today from dual;
than
declare
l_today date;
begin
p_today(l_today);
dbms_output.put_line(l_today);
end;
/
Functions can execute DML operations, but only if they are autonomous transactions. That's not what you'd always want to do. If you want to perform DML, you - usually - pick a procedure.
Although it is possible to do probably everything in both of them, choose the one that is most appropriate for what you are currently doing. Sometimes it is a function, another time it is a procedure (and sometimes you create a function which is then called by the procedure).
It just depends.
Because procedures can have OUT parameters, they too can return a value to the caller. And because functions are internally PL/SQL blocks that can do everything procedures can do, they also process things. And DML is just as possible in either of them.
I think the real difference is that a function must have one and only one main return value (passed back by the RETURN clause), whereas a procedure does not have to return anything, and indeed cannot do so with a RETURN clause. But either can use OUT parameters to pass back information.
Of course only functions can be called in SQL (as opposed to PL/SQL). But besides that, you can pretty much do anything you need with either a function or a procedure. I think it makes good programming sense to use a function when the main point of it is to retrieve a single atomic value, and to use a procedure when the main point is to make changes to the database or there are multiple things that are to be passed back and no one item is conceptually the "main" thing.
However, let me further qualify. In my PL/SQL programming, I tend to use functions more often than procedures even if the main point isn't to get something back, because I nearly always want a return/status code back (success vs. failure), so status is a good "main return value" for just about anything, making functions a good fit. This style of programming handles exceptions close to their source and returns statuses from named functions rather than raising exceptions out of them. But this is only one way to do it. Anyway, I hope this helps.
Why is it that CURRENT_DATE, CURRENT_TIMESTAMP, SYSDATE, and SYSTIMESTAMP are called without parentheses. I understand that they take no parameters, but in other languages, you would still call the functions using parentheses. Is it the case in Oracle that any function that never takes parameters cannot be called with parentheses?
If someone could point me to the documentation on this, I'd appreciate it.
Oracle is weird in many ways. It plays fast and loose with lots of things: the meaning of NULL, implicit data type conversions, and a whole number of other things.
Among them, as you noticed, is their inconsistent syntax for calling functions with no parameters. (For declaring such functions, too - see below.)
Native functions like sysdate and current_timestamp, which do not take arguments, must be written without parentheses. You ask for documentation... the most direct (and yet not entirely satisfactory) pointer is to the documentation of each function, where the syntax is shown very clearly without parentheses. What are you looking for - a separate mention in the documentation, where they state this explicitly?
Compare this with analytic functions like rownumber(), for example, which also do not take arguments. You must write them with empty parentheses!
Worse: For functions you write yourself in PL/SQL, and call from SQL statements: if the function takes no arguments, then it must be defined without parentheses. However, when you invoke it (in a select statement, for example) you can call it with or without (empty) parentheses - both syntaxes are valid. Not so, alas, with the native functions, like sysdate. Why? Again, a good question to ask Oracle.
It doesn't end there, either. connect_by_root is a "hierarchical function" (a function that can be used in hierarchical queries). It takes an argument - which can be given in parentheses (as in any normal function) or without parentheses! Go figure.
If you ask WHY??? - you are not alone. I have no clue either.
To further confuse things, some SQL functions have corresponding PL/SQL functions defined in the "STANDARD" PL/SQL package. This allows functions like SYSDATE to be invoked on the right hand side of an assignment statement in a PL/SQL program block. Because the function in the "STANDARD" package is a PL/SQL function (and not an SQL one), it can be invoked with or without parentheses.
DECLARE
x DATE;
BEGIN
x:= SYSDATE(); -- Valid
x:= SYSDATE; -- Also valid
END;
Those are all Pseudocolumns which are used in SQL or PL/SQL statements like columns but they aren't really stored on the disk. They can be thought of as special-purpose data elements within the SQL statements just as if they were part of the table.
A DML statement neither be applied on Pseudocolumns, nor user-defined they are, so they do not need to be considered as standard functions or procedures, and the style is conventionally defined by Oracle, itself.
By the way, this syntax(without parentheses) is valid even for a function in Oracle provided that no needs to have any parameter as
seems more logical rather than what other systems do.
Oracle says : Parameter declarations are optional. Functions that take no parameters are written without parentheses
I'm looking for confirmation/clarification with these LINQ expressions:
var context = new SomeCustomDbContext()
// LINQ to Entities?
var items = context.CustomItems.OrderBy(i => i.Property).ToList();
// LINQ to Objects?
var items2 = context.CustomItems.ToList().OrderBy(i => i.Property);
Am I correct in thinking the first method is LINQ to Entities where EF builds a more specific SQL statement to pass on, putting the ordering effort on on the database?
Is the second method LINQ to Objects where LINQ drags the whole collection into memory (the ToList() enumeration?) before ordering thus leaving the burden on the server side (the web server in this case)?
If this is the case, I can quickly see situations where L2E would be advantageous (ex. filtering/trimming collections before pulling them into memory).
But are there any other details/trade-offs I should be aware of, or times when "method 2" might be advantageous over the first method?
UPDATE:
Let's say we are not using EntityFramework, this is still true so long as the underlying repository/data source implements IQueryable<T> right? And if it doesn't both these statements result in LINQ to Objects operations in memory?
Yes.
Yes.
Yes.
You are correct that calling ToList() forces linq-to-entities to evaluate and return the results as a list. As you suspect, this can have huge performance implications.
There are cases where linq-to-entities cannot figure out how to parse what looks like a perfectly simple query (like Where(x => SomeFunction(x))). In these cases you often have no choice but to call ToList() and operate on the collection in memory.
In response to your update:
ToList() always forces everything ahead of it to evaluate immediately, as opposed to deferred execution. Take this example:
someEnumerable.Take(10).ToList();
vs
someEnumerable.ToList().Take(10);
In the second example, any deferred work on someEnumerable must be executed before taking the first 10 elements. If someEnumerable is doing something labor intensive (like reading files from the disk using Directory.EnumerateFiles()), this could have very real performance implications.
Am I correct in thinking the first method is LINQ to Entities where EF builds a more specific SQL statement to pass on, putting the ordering effort on on the database?
Yes
Is the second method LINQ to Objects where LINQ drags the whole collection into memory ... before ordering thus leaving the burden on the server side ...?
Yes
But are there any other details/trade-offs I should be aware of, or times when "method 2" might be advantageous over the first method?
There will be many times where Method 1 is not possible - usually when you have a complex filter or sort order that can't be directly translated to SQL (or more appropriately where EF does not support a direct SQL translation). Also since you can't transmit lazy-loaded IQueryables over-the-wire, any time you have to serialize a result you're going to have to materialize it first with ToList() or something similar.
The other thing to be aware of is that IQueryable makes no guarantees on either (a) the semantic reasoning of the underlying provider, or (b) how much of the set of IQueryable methods are implemented by the provider.
For example: -
EF does not support Last().
Nor does it support time-part comparisons of DateTimes into valid T-SQL.
It doesn't support FirstOrDefault() in subqueries.
In such circumstances you need to bring data back to the client and then perform further evaluation client-side.
You also need to have an understanding of "how" it parses the LINQ pipeline to generate (in the case of EF) T-SQL. So you sometimes have to think carefully about how you construct your LINQ queries in order to generate effective T-SQL.
Having said all that, IQueryable<> is an extremely powerful tool in the .NET framework and well worth getting more familiar with.
In situations where we cannot use bind variables, such as when our dynamic queries have to execute ddl statements, is the following list of defenses enough?
Never use anonymous blocks in dynamic queries so that only one statement can be executed by execute immediate. This stops code injection attacks.
Escape all single quotes using replace function. This stops statement modification attacks.
What characters other than single quote can be used for quoting and how can they be escaped?
How to prevent statement modification through AND, UNION etc attacks?
How to prevent function calling attacks so that user cannot call built-in functions? Every user has rights of calling those functions and calling of those functions can cause denial of service and buffer over flow attacks. How to save from that?
I prefer allowing gui to take single quote character means not check that in client side and server side validation in a web application. This is to allow names like O'Brian. At the database level, just before the execute immediate statement escape the single quotes. Do you know of any better approach?
Solution to any other vulnerabilities not listed above.
Note: I have already gone through about a dozen questions related to SQL injection on this site. I still posted this question because:
It's specific to oracle. Most questions I found on this site on the topic are related to MySQL, SQL server etc.
It's specific to situations where bind variables cannot be used. If one can use bind variables then that is enough and no other defense is needed.
It's better to list down all needed methods at one place.
Some advanced techniques of SQL injection like function calling are not discussed in detail and I cannot find any solution against that.
Edit:
Following may be a viable solution.
I think I have a solution. Its in addition of using usual defenses such as static statements, bind variables etc. Its particularly useful in situations where the usual defenses cannot be used. Note that the only situation where bind variables cannot be used is ddl statements. For such statements:
Verify existence of database object using static sql. This solves half of the problem.
The other half is related to the new value we want to put in the database object. For example when changing password of a user: the first half is username, the second half is password. One should encrypt the new value at front-end and save encrypted value in database. The encrypted value encrypted as sql code cannot do any damage to database (cannot call any functions for example).
Never change user input because of various reasons such as confusing user for example in passwords, the value may be valid in some situations such as it can be a valid html etc. It means let the ['], [\'], [#] pass through all validations unchanged. It's the static SQL or the encryption that is supposed to handle it.
Is there any situation where we cannot encrypt the new value?
Thanks for your help as I bootstrap my way into Ruby and Rails.
In the Rails API for ActiveRecord::Base, there's a section on Conditions that's meant to simply cover syntax for interactions with ActiveRecord. But the example they've used includes a very interesting (to me) primer on input-sanitization in Ruby/Rails:
class User < ActiveRecord::Base
def self.authenticate_unsafely(user_name, password)
where("user_name = '#{user_name}' AND password = '#{password}'").first
end
def self.authenticate_safely(user_name, password)
where("user_name = ? AND password = ?", user_name, password).first
end
def self.authenticate_safely_simply(user_name, password)
where(:user_name => user_name, :password => password).first
end
end
Following this example code, they explain that:
"The authenticate_safely and authenticate_safely_simply both will sanitize the user_name and password before inserting them in the query, which will ensure that an attacker can’t escape the query and fake the login (or worse)."
I totally get how this sanitization of inputs is a Good Thing in preventing injection attacks. What I do not understand is where and how this implicit sanitization is happening, given that there are no special methods being called to pre-process the input data. The various example methods appear to have nearly identical semantics and yet the variations in form have huge effects on safety because of the way that they're parsed. I'm assuming that these variations in form are similar in effect to the difference between using single-quotes and double-quotes around a string containing escape characters. But can anybody help me get smarter, faster, by understanding in general terms (or rather: at the logical level, rather than inside the interpreter) what's actually happening under the hood to make this so?
Also, to what extent are any of these differences dependent on Rails-specific constructs, rather than the underlying Ruby?
Thank you!
The input sanitization is a feature provided by Active Record known as prepared statements or parameterized statements.
Most database access libraries provide this natively, however, Active Record chooses to emulate this feature using string-mangling mechanisms. See build_where in active_record/relation/query_methods.rb and backtrack your way to sanitize_sql_for_conditions (via its sanitize_sql alias) in active_record/base.rb. (Many thanks to mu is too short for the research.)
Instead of the old-style practice of building query strings as strings in the application, you have a static template query with parameters. When you call the query, you supply the parameters and Active Record will construct a safe query for the SQL engine to execute.
You could do this task yourself -- myriad PHP programmers choose to eschew their similar PDO database access layer. However, many programmers cannot get this correct: there are roughly 1500 SQL Injection flaws discovered in the last three years and my local CVE database shows over 5000 instances of programs vulnerable to SQL injection attacks since MITRE started keeping track. Nearly all of these were due entirely to people who chose to write their own SQL code by hand and did not properly sanitize their input. (I didn't personally inspect each one but I can't recall seeing any flaws in the ORM or database access layers that allow SQL injection attacks through. But to be on the conservative side, I chose "Nearly all".)
Fellow stacker Jeff Atwood equates old-style SQL queries as the goto of database programming:
Non-parameterized SQL is the GoTo statement of database programming. Don't do it, and make sure your coworkers don't either.