I am encountering an interesting issue with an application that was migrated from Oracle 9i to 10g.
Previously, we had a problem when a field contained double quotes since Oracle recordsets encapsulated fields in double quotes.
Example:
"field1"||"field2"||"field "Y" 3"||"field4"
Since the move to 10g, I believe that the Oracle client-side driver is parsing the double quotes and replacing them with "
Unfortunately I don't have an old 9i environment to test my theory.
Have you seen similar behavior or can someone validate if my theory is true?
It's an estrange behavior. Can you check your NLS_xx environment variables? Some NLS_xx variables can translate chars, for instance NLS_LANG can translate acutes from a latin charset on client to a us2ascii charset on server.
To be sure about what is going, try to trace Net connections. You'll see everything moving from a side to other. Be careful...
Related
I'm trying to hit Oracle from Ruby and getting an error on the first line. (I'm actually doing this in pry, but that probably doesn't matter.)
[1] pry(main)> require 'oci8'
RuntimeError: Invalid NLS_LANG format: AMERICAN
What's the problem and how do I fix it?
Googling the error message didn't turn up anything promising. (It now turns up this question.) The only other question resembling this one on stackoverflow is dealing with a different problem (the variable not having any value at all even though the user set one) and the answer there did not work for me (the value proposed is also invalid, and $LANG is not set in my environment, so setting it to that did not work.)
NLS_LANG should have the format <language>_<territory>.<characterset>
Straight from the doc there is an example corresponding to your exact use case:
The NLS_LANG environment variable is set as a local environment variable for the shell on all UNIX-based platforms. For example, if the operating system locale setting is en_US.UTF-8, then the corresponding NLS_LANG environment variable should be set to AMERICAN_AMERICA.AL32UTF8.
Please note the AL32UTF8 is a superset of UTF8 (without hyphen) accepting all Unicode characters. UTF8 only supports Unicode 3.1 and earlier. I would strongly recommend using AL32UTF8 as your default "UTF-8" character set unless you have very specific needs.
In Oracle 12.1, AL32UTF8 supports Unicode up to 6.1. One advantage is AL32UTF8 has support for supplementary characters introduced by Unicode 4.0 (code points from U+10000 to U+10FFFF)
I don't know where the value "AMERICAN" came from, but it turns out a better option, which the ruby-oci8 gem will accept, is NLS_LANG=AMERICAN_AMERICA.UTF8.
The HSQLDB documentation says something to the effect of:
Special care may be needed w.r.t. file path specifications containing whitespace, mixed-case, special characters and/or reserved file names. Please read your OS file system documentation.
However, they provide ZERO documentation or examples on the syntax for actually using file names with reserved characters or spaces. When I use a file name with a space in it, HSQLDB appears to simply hang my entire application.
I've tried escaping with URL syntax ("%20") which does not work because HSQLDB interprets that literally. I've tried surrounding various portions of the JDBC URL with single and double quotes. Most result in a failure to access the database error. At least they don't simply hang.
Does anyone know the proper way to reference an embedded HSQLDB file when the file path has spaces?
When I use a file name with a space in it, HSQLDB appears to simply hang my entire application.
For what it's worth, my Windows (Vista) test box has no problems with spaces in the database path or the database name
String connStr = "jdbc:hsqldb:file:C:/Users/Public/test/HSQLDB test/my db";
try (Connection con = DriverManager.getConnection(connStr, "sa", "")) {
results in this
Tested using hsqldb-2.3.1.
Thats awful, I can't find where I can turn if off!
The prolem is that PhpStorm detectes quote as Bad Token and Underwaves it.
The backticks delimiters for table and column names are not SQL compliant, they're MySQL, hence it's giving an error.
To fix the problem, inject the "MySQL" language into the string. The problem should be resolved.
The question mark "?" appears only in the front of the first field of the first row to insert.
For once, I changed the ftp upload file type to text/ascii (rather than binary) and it seemed resolve the problem. But later it came back.
The server OS is aix5.3.
DataStage is 7.5x2.
Oracle is 11g.
I used ue to save the file to utf-8, using unix end mark.
Has anyone got this thing before?
The question mark itself doesn't mean much as it could be only a "mask" for some special character which is not recognized by the database. You didn't provide any details about your environment, so my opinions here are only a guess. I hope it can give you a little of a light.
How is the text file created? If it's a file created in a windows environment you're very likely to have character like this due brake lines {CR}{LF} characters.
What is the datatype for the oracle table?
Char datatype will "fill" every position according to the size of the field, I'd recommend to use varchar instead on this case.
If it's not the case, I would edit the file in Hex mode and check for the Ascii code for this specific character then use a TRIM (if parallel) or Convert(if server) to replace the character.
The convert function would be something like this:
Convert(Char([ascii_char_number]),'',[your_string])
Alternatively you can use the Trim function if your job is a parallel job
Trim([your_string],[ascii_char_number],'L')
The option "L" will remove all leading characters. You might need to adapt this function to suit your needs. If you're not familiar with the TRIM function you can find more details at the datastage online documentation.
The only warning I'd give when doing this, is that you'll be deleting data from your original source of data, so make sure you're not deleting any valid information when manipulating a file like this as this is not a very recommended practice between the ETL gurus out there.
Any questions, give me a shout. Happy to help if I can.
Cheers
I had a similar issue where unprintable characters were being displayed as '?' and datastage was throwing a warning when processing these records. It was ok for me to not display those unprintable characters, so I used the function ICONV which converts those characters into printable ones. There are multiple options, I chose the one which will convert them to '.' which worked for me. More details are available in the IBM pages below:
https://www-01.ibm.com/support/knowledgecenter/SSZJPZ_11.3.0/com.ibm.swg.im.iis.ds.parjob.dev.doc/topics/r_deeref_String_Functions.html
http://docs.intersystems.com/ens201317/csp/docbook/DocBook.UI.Page.cls?KEY=RVBS_foconv
The conversion I used:
ICONV(column_name,"MCP")
I'm doing an audit of a system, which the developers insist is SQL injection proof. This they achieve by stripping out the single-quotes in the login form - but the code behind is not parameterized; it's still using literal SQL like so:
username = username.Replace("'", "");
var sql = "select * from user where username = '" + username + "'";
Is this really secure? Is there another way of inserting a single quote, perhaps by using an escape character? The DB in use is Oracle 10g.
Maybe you can also fail them because not using bind variables will have a very negative impact on performance.
A few tips:
1- It is not necessarily the ' character that can be used as a quote. Try this:
select q'#Oracle's quote operator#' from dual;
2- Another tip from "Innocent Code" book says: Don't massage invalid input to make it valid (by escaping or removing). Read the relevant section of the book for some very interesting examples. Summary of rules are here.
Have a look at the testing guide here: http://www.owasp.org/index.php/Main_Page That should give you more devious test scenarios, perhaps enough to prompt a reassessment of the SQL coding strategy :-)
No, it is not secure. SQL injection doesn't require a single-quote character to succeed. You can use AND, OR, JOIN, etc to make it happen. For example, suppose a web application having a URL like this: http://www.example.com/news.php?id=100.
You can do many things if the ID parameter is not properly validated. For example, if its type is not checked, you could simply use this: ?id=100 AND INSERT INTO NEWS (id, ...) VALUES (...). The same is valid for JOIN, etc. I won't teach how to explore it because not all readers have good intentions like you appear to have. So, for those planning to use a simple REPLACE, be aware that this WILL NOT prevent an attack.
So, no one can have a name like O'Brian in their system?
The single quote check won't help if the parameter is numeric - then 1; DROP TABLE user;-- would cause some trouble =)
I wonder how they handle dates...
If the mechanism for executing queries got smart like PHP, and limited queries to only ever run one query, then there shouldn't be an issue with injection attacks...
What is the client language ? That is, we'd have to be sure exactly what datatype of username is and what the Replace method does in regard to that datatype. Also how the actual concatenation works for that datatype. There may be some character set translation that would translate some quote-like character in UTF-8 to a "regular" quote.
For the very simple example you show it should just work, but the performance will be awful (as per Thilo's comment). You'd need to look at the options for cursor_sharing
For this SQL
select * from user where username = '[blah]'
As long as [blah] didn't include a single quote, it should be interpreted as single CHAR value. If the string was more than 4000 bytes, it would raise an error and I'd be interested to see how that was handled. Similarly an empty string or one consisting solely of single quotes. Control characters (end-of-file, for example) might also give it some issues but that might depend on whether they can be entered at the front-end.
For a username, it would be legitimate to limit the characterset to alphanumerics, and possibly a limited set of punctuation (dot and underscore perhaps). So if you did take a character filtering approach, I'd prefer to see a whitelist of acceptable characters rather than blacklisting single quotes, control characters etc.
In summary, as a general approach it is poor security, and negatively impacts performance. In particular cases, it might (probably?) won't expose any vulnerabilities. But you'd want to do a LOT of testing to be sure it doesn't.