When trying to insert into the database a string such as :
"セブンゴースト; 神幻拍挡07;"
there is no error but it is read as nil after a select.
Exemple :
string = "セブンゴースト; 神幻拍挡07;"
db = SQLite3::Database.new "randomfilename"
prep = db.prepare "INSERT INTO test_db VALUES (NULL, ?)"
prep.bind_param 1, string
prep.execute
prep2 = db.prepare "SELECT * FROM test_db"
ret = prep2.execute
p ret
this will display something like this : [[0, nil]]
( assuming that the table has a primary int key as first value and a TEXT for second )
It is possible that my database does not have the good encoding but if it is the case, how do I change it without loosing everything ?
found the answer : I needed a NTEXT field, not a TEXT.
the NTEXT supports UTF-8 encoding
http://www.w3schools.com/sql/sql_datatypes.asp
Related
Reading value of fixed CHAR string from table, using Linq2db Oracle provider:
CREATE TABLE mytable
(pk NUMBER(15,0) NOT NULL,
fixed_data CHAR(20) DEFAULT ' ' NOT NULL)
Although in database, length of FIXED_DATA filed is 20,
SELECT LENGTH(fixed_data) FROM mytable WHERE pk = 1
-- result is 20
When same field is read using Linq2Db, value gets truncated to empty string:
var row = (from row in database.mytable where row.pk == 1 select row).ToList()[0];
Console.WriteLine(row.fixed_data.Length);
// result is zero
This causes problem when record is updated using Linq2Db, Oracle converts empty string to NULL, and UPDATE fails:
database.Update(row);
// Oracle.ManagedDataAccess.Client.OracleException: 'ORA-01407: cannot update ("MYSCHEMA"."MYTABLE"."FIXED_DATA") to NULL
Is there any setting in Linq2Db for read->update cycle to work with CHAR type and NOT NULL constraint?
Found a solution, thanks to source code openly available. By default, Linq2Db calls expression IDataReader.GetString(int).TrimEnd(' ') on every CHAR and NCHAR column. However this can be easily customized, by implementing custom provider, which overrides field value retrieval expression, with one that does not trim:
class MyOracleProvider : OracleDataProvider
{
public MyOracleProvider(string name)
: base(name)
{
// original is SetCharField("Char", (r,i) => r.GetString(i).TrimEnd(' '));
SetCharField("Char", (r, i) => r.GetString(i));
// original is SetCharField("NChar", (r,i) => r.GetString(i).TrimEnd(' '));
SetCharField("NChar", (r, i) => r.GetString(i));
}
}
I'm trying to insert CLOBs into a database (see related question). I can't quite figure out what's wrong. I have a list of about 85 clobs I want to insert into a table. Even when inserting only the first clob I get ORA-00911: invalid character. I can't figure out how to get the statement out of the PreparedStatement before it executes, so I can't be 100% certain that it's right, but if I got it right, then it should look exactly like this:
insert all
into domo_queries values ('select
substr(to_char(max_data),1,4) as year,
substr(to_char(max_data),5,6) as month,
max_data
from dss_fin_user.acq_dashboard_src_load_success
where source = ''CHQ PeopleSoft FS''')
select * from dual;
Ultimately, this insert all statement would have a lot of into's, which is why I just don't do a regular insert statement. I don't see an invalid character in there, do you? (Oh, and that code above runs fine when I run it in my sql developer tool.) And I if I remove the semi-colon in the PreparedStatement, it throws an ORA-00933: SQL command not properly ended error.
In any case, here's my code for executing the query (and the values of the variables for the example above).
public ResultSet executeQuery(String connection, String query, QueryParameter... params) throws DataException, SQLException {
// query at this point = "insert all
//into domo_queries values (?)
//select * from dual;"
Connection conn = ConnectionPool.getInstance().get(connection);
PreparedStatement pstmt = conn.prepareStatement(query);
for (int i = 1; i <= params.length; i++) {
QueryParameter param = params[i - 1];
switch (param.getType()) { //The type in the example is QueryParameter.CLOB
case QueryParameter.CLOB:
Clob clob = CLOB.createTemporary(conn, false, oracle.sql.CLOB.DURATION_SESSION);
clob.setString(i, "'" + param.getValue() + "'");
//the value of param.getValue() at this point is:
/*
* select
* substr(to_char(max_data),1,4) as year,
* substr(to_char(max_data),5,6) as month,
* max_data
* from dss_fin_user.acq_dashboard_src_load_success
* where source = ''CHQ PeopleSoft FS''
*/
pstmt.setClob(i, clob);
break;
case QueryParameter.STRING:
pstmt.setString(i, "'" + param.getValue() + "'");
break;
}
}
ResultSet rs = pstmt.executeQuery(); //Obviously, this is where the error is thrown
conn.commit();
ConnectionPool.getInstance().release(conn);
return rs;
}
Is there anything I'm just missing big time?
If you use the string literal exactly as you have shown us, the problem is the ; character at the end. You may not include that in the query string in the JDBC calls.
As you are inserting only a single row, a regular INSERT should be just fine even when inserting multiple rows. Using a batched statement is probable more efficient anywy. No need for INSERT ALL. Additionally you don't need the temporary clob and all that. You can simplify your method to something like this (assuming I got the parameters right):
String query1 = "select substr(to_char(max_data),1,4) as year, " +
"substr(to_char(max_data),5,6) as month, max_data " +
"from dss_fin_user.acq_dashboard_src_load_success " +
"where source = 'CHQ PeopleSoft FS'";
String query2 = ".....";
String sql = "insert into domo_queries (clob_column) values (?)";
PreparedStatement pstmt = con.prepareStatement(sql);
StringReader reader = new StringReader(query1);
pstmt.setCharacterStream(1, reader, query1.length());
pstmt.addBatch();
reader = new StringReader(query2);
pstmt.setCharacterStream(1, reader, query2.length());
pstmt.addBatch();
pstmt.executeBatch();
con.commit();
Of the top of my head, can you try to use the 'q' operator for the string literal
something like
insert all
into domo_queries values (q'[select
substr(to_char(max_data),1,4) as year,
substr(to_char(max_data),5,6) as month,
max_data
from dss_fin_user.acq_dashboard_src_load_success
where source = 'CHQ PeopleSoft FS']')
select * from dual;
Note that the single quotes of your predicate are not escaped, and the string sits between q'[...]'.
One of the reason may be if any one of table column have an underscore(_) in its name . That is considered as invalid characters by the JDBC . Rename the column by a ALTER Command and change in your code SQL , that will fix .
Oracle provide some explanation for ORA-00911. You can got this explanation after executing SQL request in Oracle SQL Developer.
ORA-00911. 00000 - "invalid character"
*Cause: identifiers may not start with any ASCII character other than
letters and numbers. $#_ are also allowed after the first
character. Identifiers enclosed by doublequotes may contain
any character other than a doublequote. Alternative quotes
(q'#...#') cannot use spaces, tabs, or carriage returns as
delimiters. For all other contexts, consult the SQL Language
Reference Manual
But in your case it seems to be double ' character
I have a SQL query with a WHERE clause that typically has values including a dash as stored in the database as a CHAR(10). When I explicitly call it like in the following:
$sth = $dbh->prepare("SELECT STATUS_CODE FROM MyTable WHERE ACC_TYPE = 'A-50C'");
It works and properly returns my 1 row; however if I do the following:
my $code = 'A-50C';
$sth = $dbh->prepare("SELECT STATUS_CODE FROM MyTable WHERE ACC_TYPE = ?");
$sth->execute($code);
or I do:
my $code = 'A-50C';
$sth = $dbh->prepare("SELECT STATUS_CODE FROM MyTable WHERE ACC_TYPE = ?");
$sth->bind_param(1, $code);
$sth->execute();
The query completes, but I get no results. I suspect it has to do with the dash being interpretted incorrectly, but I can't link it to a Perl issue as I have printed my $code variable using print "My Content: $code\n"; so I can confirm its not being strangely converted. I also tried including a third value for bind_param and if I specify something like ORA_VARCHAR2, SQL_VARCHAR (tried all possibilities) I still get no results. If I change it to the long form i.e. { TYPE => SQL_VARCHAR } it gives me an error of
DBI::st=HASH<0x232a210>->bind_param(...): attribute parameter
'SQL_VARCHAR' is not a hash ref
Lastly, I tried single and double quotes in different ways as well as back ticks to escape the values, but nothing got me the 1 row, only 0. Any ideas? Haven't found anything in documentation or searching. This is oracle for reference.
Code with error checking:
my $dbh = DBI->connect($dsn, $user, $pw, {PrintError => 0, RaiseError => 0})
or die "$DBI::errstr\n";
# my $dbh = DBI->connect(); # connect
my $code = 'A-50C';
print "My Content: $code\n";
$sth = $dbh->prepare( "SELECT COUNT(*) FROM MyTable WHERE CODE = ?" )
or die "Can't prepare SQL statement: $DBI::errstr\n";
$sth->bind_param(1, $code);
$sth->execute() or die "Can't execute SQL statement: $DBI::errstr\n";
my $outfile = 'output.txt';
open OUTFILE, '>', $outfile or die "Unable to open $outfile: $!";
while(my #re = $sth->fetchrow_array) {
print OUTFILE #re,"\n";
}
warn "Data fetching terminated early by error: $DBI::errstr\n"
if $DBI::err;
close OUTFILE;
$sth->finish();
$dbh->disconnect();
I ran a trace and got back:
-> bind_param for DBD::Oracle::st (DBI::st=HASH(0x22fbcc0)~0x3bcf48 2 'A-50C' HASH(0x22fbac8)) thr#3b66c8
dbd_bind_ph(1): bind :p2 <== 'A-50C' (type 0 (DEFAULT (varchar)), attribs: HASH(0x22fbac8))
dbd_rebind_ph_char() (1): bind :p2 <== 'A-50C' (size 5/16/0, ptype 4(VARCHAR), otype 1 )
dbd_rebind_ph_char() (2): bind :p2 <== ''A-50' (size 5/16, otype 1(VARCHAR), indp 0, at_exec 1)
bind :p2 as ftype 1 (VARCHAR)
dbd_rebind_ph(): bind :p2 <== 'A-50C' (in, not-utf8, csid 178->0->178, ftype 1 (VARCHAR), csform 0(0)->0(0), maxlen 16, maxdata_size 0)
Your problem is likely a result of comparing CHAR and VARCHAR data together.
The CHAR data type is notorious (and should be avoided), because it stores data in fixed-length format. It should never be used for holding varying-length data. In your case, data stored in the ACC_TYPE column will always take up 10 characters of storage. When you store a value whose length is less than the size of the column, like A-50C, the database will implicitly pad the string up to 10 characters, so the actual value stored becomes A-50C_____ (where _ represents a whitespace).
Your first query works because when you use a hard-code literal, Oracle will automatically right-pad the value for you (A-50C -> A-50C_____). However, in your second query where you use bind variables, you're comparing a VARCHAR against a CHAR and no auto-padding will happen.
As a quick fix to the problem, you could add right-padding to the query:
SELECT STATUS_CODE FROM MyTable WHERE ACC_TYPE = rpad(?, 10)
A long-term solution would be to avoid using the CHAR data type in your table definitions and switch to VARCHAR2 instead.
As your DBI_TRACE revealed, the ACC_TYPE column is CHAR(10) but the bound parameter is understood as VARCHAR.
When comparing CHAR, NCHAR, or literal strings to one another, trailing blanks are effectively ignored. (Remember, CHAR(10) means that ACC_TYPE values are padded to 10 characters long.) Thus, 'A ' and 'A', as CHARs, compare equal. When comparing with the VARCHAR family, however, trailing blanks become significant, and 'A ' no longer equals 'A' if one is a VARCHAR variant.
You can confirm in sqlplus or via quick DBI query:
SELECT COUNT(1) FROM DUAL WHERE CAST('A' AS CHAR(2)) = 'A'; -- or CAST AS CHAR(whatever)
SELECT COUNT(1) FROM DUAL WHERE CAST('A' AS CHAR(2)) = CAST('A' AS VARCHAR(1));
(Oracle terms these blank-padded and nonpadded comparison semantics, since the actual behavior per the ANSI-92 spec is to pad the shorter CHAR or literal to the length of the longer and then compare. The effective behavior, whatever the name, is that one ignores trailing blanks and the other does not.)
As #MickMnemonic suggested, RPAD()ing the bound value will work, or, better, altering the column type to VARCHAR. You could also CAST(? AS CHAR(10)). TRIM(TRAILING FROM ACC_TYPE) would work, too, at the cost of ignoring any index on that column.
Thanks in advance for any help on this one.
I have a model in rails that includes a postgresql text column.
I want to append (i.e. mycolumn = mycolumn || newdata) data to the existing column. The sql I want to generate would look like:
update MyOjbs set mycolumn = mycolumn || newdata where id = 12;
I would rather not select the data, update the attribute and then write the new data back to the database. The text column could grow relatively large and I'd rather not read that data if I don't need to.
I DO NOT want to do this:
#myinstvar = MyObj.select(:mycolumn).find(12)
newdata = #myinstvar.mycolumn.to_s + newdata
#myinstvar.update_attribute(:mycolumn, newdata)
Do I need to do a raw sql transaction to accomplish this?
I think you could solve this problem directly writing your query using the arel gem, that's already provided with rails.
Given that you have these values:
column_id = 12
newdata = "a custom string"
you can update the table this way:
# Initialize the Table and UpdateManager objects
table = MyOjbs.arel_table
update_manager = Arel::UpdateManager.new Arel::Table.engine
update_manager.table(table)
# Compose the concat() function
concat = Arel::Nodes::NamedFunction.new 'concat', [table[:mycolumn], new_data]
concat_sql = Arel::Nodes::SqlLiteral.new concat.to_sql
# Set up the update manager
update_manager.set(
[[table[:mycolumn], concat_sql]]
).where(
table[:id].eq(column_id)
)
# Execute the update
ActiveRecord::Base.connection.execute update_manager.to_sql
This will generate a SQL string like this one:
UPDATE "MyObjs" SET "mycolumn" = concat("MyObjs"."mycolumn", 'a custom string') WHERE "MyObjs"."id" = 12"
I have a tricky issue with the Oracle JDBC driver's handling of CHAR data types. Let's take this simple table:
create table x (c char(4));
insert into x (c) values ('a'); -- inserts 'a '
So when I insert something into CHAR(4), the string is always filled with whitespace. This is also done when I execute queries like this:
select * from x where c = 'a'; -- selects 1 record
select * from x where c = 'a '; -- selects 1 record
select * from x where c = 'a '; -- selects 1 record
Here, the constant 'a' is filled with whitespace as well. That's why the record is always returned. This holds true when these queries are executed using a JDBC PreparedStatement as well. Now the tricky thing is when I want to use a bind variable:
PreparedStatement stmt =
conn.prepareStatement("select * from x where c = ?");
stmt.setString(1, "a"); // This won't return any records
stmt.setString(1, "a "); // This will return a record
stmt.executeQuery();
This is a workaround:
PreparedStatement stmt =
conn.prepareStatement("select * from x where trim(c) = trim(?)");
stmt.setString(1, "a"); // This will return a record
stmt.setString(1, "a "); // This will return a record
stmt.executeQuery();
EDIT: Now these are the constraints:
The above workaround is not desireable as it modifies both the contents of c and ?, AND it makes using indexes on c quite hard.
Moving the column from CHAR to VARCHAR (which it should be, of course) is not possible
EDIT: The reasons for these constraints is because I ask this question from the point of view of the developer of jOOQ, a database abstraction library. So my requirements are to provide a very generic solution that doesn't break anything in jOOQ's client code. That is why I'm not really a big fan of the workaround. And that's why I don't have access to that CHAR column's declaration. But still, I want to be able to handle this case.
What would you do instead? What's a good practice for handling CHAR data types when I want to ignore trailing whitespace?
If you want
stmt.setString(1, "a"); // This won't return any records
to return a record, try
conn.prepareStatement("select * from x where c = cast(? as char(4))")
I don't see any reason to use CHAR datatype even if it is char(1) in Oracle. Can you change the datatype instead?
Gary's solution works well. Here's an alternative.
If you are using an Oracle JDBC driver, the call to prepareStatement() will actually return an OraclePreparedStatement, which has a setFixedCHAR() method that automatically pads your inputs with whitespace.
String sql = "select * from x where c = ?";
OraclePreparedStatement stmt = (OraclePreparedStatement) conn.prepareStatement(sql);
stmt.setFixedCHAR(1, "a");
...
Obviously, the cast is only safe if you are using the Oracle driver.
The only reason I would suggest that you use this over Gary's answer is that you can change your column sizes without having to modify your JDBC code. The driver pads the correct number of spaces without the developer needing to know/manage the column size.
I have nice fix for this. You have to add one property while getting connection from database.
NLS_LANG=american_america.AL32UTF8
or in Java connection you can use below code:
java.util.Properties info = new java.util.Properties();
info.put ("user", user);
info.put ("password",password);
info.put("fixedString","TRUE");
info.put("NLS_LANG","american_america.AL32UTF8");
info.put("SetBigStringTryClob","TRUE");
String url="jdbc:oracle:thin:#"+serverName;
log.debug("url="+url);
log.debug("info="+info);
Class.forName("oracle.jdbc.OracleDriver");
conn = DriverManager.getConnection(url,info);
the other way is modify your sql as
select * from x where NVL(TRIM(c),' ') = NVL(TRIM('a'),' ')
Simply add RTRIM() to the column name(which is defimed) in the update query.