This is a general question and not the usual StackOverflow "can someone please correct my syntax mistake..."
SQL is the most wonderfully condensed language, so in one line I can save a record. e.g.
INSERT INTO database VALUES (4,'Nilsen', 'Johan', 'Bakken 2', 'Stavanger')
But if I want to save/insert a record in Cocoa with SQLite I'll end up writing closer to 14, e.g.
NSString *insertSQL = [NSString stringWithFormat: #"INSERT INTO CONTACTS (name, address, phone) VALUES (\"%#\", \"%#\", \"%#\")", name.text, address.text, phone.text];
const char *insert_stmt = [insertSQL UTF8String];
sqlite3_prepare_v2(contactDB,insert_stmt, -1, &statement, NULL);
if (sqlite3_step(statement) == SQLITE_DONE) {
self.status.text = #"Contact added";
self.name.text = #"";
self.address.text = #"";
self.phone.text = #"";
} else {
self.status.text = #"Failed to add contact";
}
sqlite3_finalize(statement);
sqlite3_close(contactDB);
 }
}
I understand what each line is doing but as a Cocoa/SQLite newbie all I want to double check is if there is a simpler way or if I just have to regurgitate this level of verbosity every time I want to do the tritest SQL command...
Many thanks
Use FMDB (an excellent SQLite wrapper).
As an alternative, you could create a function that accepts both the SQL query and the database name. One great use of functions is to simplify the process of writing code.
Related
Our project recently updated to the newer Oracle.ManagedDataAccess DLL's (v 4.121.2.0) and this error has been cropping up intermittently. We've fixed it a few times without really knowing what we did to fix it.
I'm fairly certain it's caused by CLOB fields being mapped to strings in Entity Framework, and then being selected in LINQ statements that pull entire entities instead of just a limited set of properties.
Error:
Value cannot be null.
Parameter name: byteArray
Stack Trace:
at System.BitConverter.ToString(Byte[] value, Int32 startIndex, Int32 length)
at OracleInternal.TTC.TTCLob.GetLobIdString(Byte[] lobLocator)
at OracleInternal.ServiceObjects.OracleDataReaderImpl.CollectTempLOBsToBeFreed(Int32 rowNumber)
at Oracle.ManagedDataAccess.Client.OracleDataReader.ProcessAnyTempLOBs(Int32 rowNumber)
at Oracle.ManagedDataAccess.Client.OracleDataReader.Read()
at System.Data.Entity.Core.Common.Internal.Materialization.Shaper`1.StoreRead()
Suspect Entity Properties:
'Mapped to Oracle CLOB Column'
<Column("LARGEFIELD")>
Public Property LargeField As String
But I'm confident this is the proper way to map the fields per Oracle's matrix:
ODP.NET Types Overview
There is nothing obviously wrong with the generated SQL statement either:
SELECT
...
"Extent1"."LARGEFIELD" AS "LARGEFIELD",
...
FROM ... "Extent1"
WHERE ...
I have also tried this Fluent code per Ozkan's suggestion, but it does not seem to affect my case.
modelBuilder.Entity(Of [CLASS])().Property(
Function(x) x.LargeField
).IsOptional()
Troubleshooting Update:
After extensive testing, we are quite certain this is actually a bug, not a configuration problem. It appears to be the contents of the CLOB that cause the problem under a very specific set of circumstances. I've cross-posted this on the Oracle Forums, hoping for more information.
After installation of Oracle12 client we ran into the same problem.
In machine.config (C:\Windows\Microsoft.NET\Framework\v4.0.30319\Config) I removed all entries with Oracle.ManagedDataAccess.
In directory C:\Windows\Microsoft.NET\assembly\GAC_MSIL I removed both Oracle.ManagedDataAccess and Policy.4.121.Oracle.ManagedDataAccess.
Then my C# program started working as usually, using the Oracle.ManagedDataAccess dll in it's own directory.
We met this problem in our project an hour ago and found a solution. It is generating this error because of null values in CLOB caolumn. We have a CLOB column and it is Nullable in database. In EntityFramework model it is String but not Nullable. We changed column's Nullable property to True in EF model and it fixed problem.
We have this problem as well on some computers, and we are running the latest Oracle.ManagedDataAccess.dll (4.121.2.20150926 ODAC RELEASE 4).
We found a solution to our problem, and I just wanted to share.
This was our problem that occurred some computers.
Using connection As New OracleConnection(yourConnectionString)
Dim command As New OracleCommand(yourQuery, connection)
connection.Open()
Using reader As OracleDataReader = command.ExecuteReader()
Dim clobField As String = CStr(reader.Item("CLOB_FIELD"))
End Using
connection.Close()
End Using
And here's the solution that made it work on all computers.
Using connection As New OracleConnection(yourConnectionString)
Dim command As New OracleCommand(yourQuery, connection)
connection.Open()
Using reader As OracleDataReader = command.ExecuteReader()
Dim clobField As String = reader.GetOracleClob(0).Value
End Using
connection.Close()
End Using
I spent a lot of time trying to decipher this and found bits of this and that here and there on the internet, but no where had everything in one place, so I'd like to post what I've learned, and how I resolved it, which is much like Ragowit's answer, but I've got the C# code for it.
Background
The error: So I had this error on my while (dr.Read()) line:
Value cannot be null. \r\nParmeter name: byteArray
I ran into very little on the internet about this, except that it was an error with the CLOB field when it was null, and was supposedly fixed in the latest ODAC release, according to this: https://community.oracle.com/thread/3944924
My take on this -- NOT TRUE! It hasn't been updated since October 5, 2015 (http://www.oracle.com/technetwork/topics/dotnet/utilsoft-086879.html), and the 12c package I'm using was downloaded in April 2016.
Full stack trace by someone else with the error that pretty much mirrored mine: http://pastebin.com/24AfFDnq
Value cannot be null.
Parameter name: byteArray
at System.BitConverter.ToString(Byte[] value, Int32 startIndex, Int32 length)
at OracleInternal.TTC.TTCLob.GetLobIdString(Byte[] lobLocator)
at OracleInternal.ServiceObjects.OracleDataReaderImpl.CollectTempLOBsToBeFreed(Int32 rowNumber)
at Oracle.ManagedDataAccess.Client.OracleDataReader.ProcessAnyTempLOBs(Int32 rowNumber)
at Oracle.ManagedDataAccess.Client.OracleDataReader.Read()
at System.Data.Entity.Core.Common.Internal.Materialization.Shaper`1.StoreRead()
'Mapped to Oracle CLOB Column'
<Column("LARGEFIELD")>
Public Property LargeField As String
'Mapped to Oracle BLOB Column'
<Column("IMAGE")>
Public Property FileContents As Byte()
How I encountered it: It was while reading an 11-column table of about 3000 rows. One of the columns was actually an NCLOB (so apparently this is just as susceptible as CLOB), which allowed nulls in the database, and some of its values were empty - it was an optional "Notes" field, after all. It's funny that I didn't get this error on the first or even second row that had an empty Notes field. It didn't error until row 768 finished and it was about to start row 769, according to an int counter variable that started at 0 that I set up and saw after checking how many rows my DataTable had thus far. I found I got the error if I used:
DataSet ds = new DataSet();
OracleDataAdapter adapter = new OracleDataAdapter(cmd);
adapter.Fill(ds);
as well if I used:
DataTable dt = new DataTable();
OracleDataReader dr = cmd.ExecuteReader();
dt.Load(dr);
or if I used:
OracleDataReader dr = cmd.ExecuteReader();
if (dr.HasRows)
{
while (dr.Read())
{
....
}
}
where cmd is the OracleCommand, so it made no difference.
Resolution
The following is basically the code I used to parse through an OracleDataReader values in order to assign them to a DataTable. It's actually not as refined as it could be - I am using it to just return dr[i] to datarow in all cases except when the value is null, and when it is the eleventh column (index = 10, because it starts at 0) and a particular query has been executed so that I know where my NCLOB column is.
public static DataTable GetDataTableManually(string query)
{
OracleConnection conn = null;
try
{
string connString = ConfigurationManager.ConnectionStrings["MyConn"].ConnectionString;
conn = new OracleConnection(connString);
OracleCommand cmd = new OracleCommand(query, conn);
conn.Open();
OracleDataReader dr = cmd.ExecuteReader(CommandBehavior.CloseConnection);
DataTable dtSchema = dr.GetSchemaTable();
DataTable dt = new DataTable();
List<DataColumn> listCols = new List<DataColumn>();
List<DataColumn> listTypes = new List<DataColumn>();
if (dtSchema != null)
{
foreach (DataRow drow in dtSchema.Rows)
{
string columnName = System.Convert.ToString(drow["ColumnName"]);
DataColumn column = new DataColumn(columnName, (Type)(drow["DataType"]));
listCols.Add(column);
listTypes.Add(drow["DataType"].ToString()); // necessary in order to record nulls
dt.Columns.Add(column);
}
}
// Read rows from DataReader and populate the DataTable
if (dr.HasRows)
{
int rowCount = 0;
while (dr.Read())
{
string fieldType = String.Empty;
DataRow dataRow = dt.NewRow();
for (int i = 0; i < dr.FieldCount; i++)
{
if (!dr.IsDBNull[i])
{
fieldType = dr.GetFieldType(i).ToString(); // example only, this is the same as listTypes[i], and neither help us distinguish NCLOB from NVARCHAR2 - both will say System.String
// This is the magic
if (query == "SELECT * FROM Orders" && i == 10)
dataRow[((DataColumn)listCols[i])] = dr.GetOracleClob(i); // <-- our new check!!!!
// Found if you have null Decimal fields, this is
// also needed, and GetOracleDecimal and GetDecimal
// will not help you - only GetFloat does
else if (listTypes[i] == "System.Decimal")
dataRow[((DataColumn)listCols[i])] = dr.GetFloat(i);
else
dataRow[((DataColumn)listCols[i])] = dr[i];
}
else // value was null; we can't always assign dr[i] if DBNull, such as when it is a number or decimal field
{
byte[] nullArray = new byte[0];
switch (listTypes[i])
{
case "System.String": // includes NVARCHAR2, CLOB, NCLOB, etc.
dataRow[((DataColumn)listCols[i])] = String.Empty;
break;
case "System.Decimal":
case "System.Int16": // Boolean
case "System.Int32": // Number
dataRow[((DataColumn)listCols[i])] = 0;
break;
case "System.DateTime":
dataRow[((DataColumn)listCols[i])] = DBNull.Value;
break;
case "System.Byte[]": // Blob
dataRow[((DataColumn)listCols[i])] = nullArray;
break;
default:
dataRow[((DataColumn)listCols[i])] = String.Empty;
break;
}
}
}
dt.Rows.Add(dataRow);
}
ds.Tables.Add(dt);
}
}
catch (Exception ex)
{
// handle error
}
finally
{
conn.Close();
}
// After everything is closed
if (ds.Tables.Count > 0)
return ds.Tables[0]; // there should only be one table if we got results
else
return null;
}
In the same way that I have it assigning specific types of null based on the column type found in the schema table loop, you could add the conditions to the "not null" side of the if...then and do various GetOracle... statements there. I found it was only necessary for this NCLOB instance, though.
To give credit where credit is due, the original codebase is based on the answer given by sarathkumar at Populate data table from data reader .
For me it was simple! I had this error with odac v 4.121.1.0. I have just updated Oracle.ManagedDataAccess to 4.121.2.0 with Nuget and now it is working.
Have you have tried to uninstall and reinstall Oracle.ManagedDataAccess with Nugget?
upgrade Oracle.ManagedDataAccess.dll to version 4.122.1.0 solved.
If you are using vs 2017, we can update via NuGet.
Using Notes 9 and extension library's data controls...
I want to use the #JDBCDbColumn() to get the type ahead values from an Oracle table.
Is that possible at all? Does it work the same way the standard DBColumn in the type ahead?
Note that the DBColumn is looking in a huge table, but I specify a where clause that should filter what the JDBCDBColumn returns.
Somehow, nothing happens. Work sfine with Notes data though, but I need to get this working with the Oracle data.
Thanks!
Update 1: Code I have...
#JdbcExecuteQuery(
"oracle",
"select distinct postal_code from cifadmin.postal_codes where postal_code like '"
+ getComponent("CodePostal").getValue() + "%'")
Update 2: Here is the code I have right now, but it doesn't return anything:
var CodePostal = getComponent("rsSearchQuery").getValue();
if(!!CodePostal) {
var params = [CodePostal];
var a = #JdbcDbColumn("oracle", "postal_codes", "postal_code", "postal_code like ?", params);
return #Unique(a);
} else {
return "--";
}
OK, I got it working thanks to all your comments and a bit of intuition!!!
Here is the working code:
var sql = "SELECT DISTINCT POSTAL_CODE FROM cifadmin.POSTAL_CODES WHERE POSTAL_CODE LIKE'" + getComponent("PostalCode").getValue() +"%' ORDER BY POSTAL_CODE";
var res = #JdbcExecuteQuery("oracle", sql);
var values = new Array();
while (res.next()) {
values.push(res.getString("POSTAL_CODE"));
}
return values;
The thing that got it working was when I have forced 'values' as an array. Without that, it just doesn't work.
Picky and sensitive, XPages are!!!
Thanks to all for your help ;)
Just tested it the first time ever: works fine for me except I didn't use a WHERE clause in my sample and another type of DB here.
<xp:inputText id="inputText1" styleClass="form-control">
<xp:typeAhead mode="partial" minChars="1" ignoreCase="true">
<xp:this.valueList><![CDATA[#{javascript:#JdbcDbColumn("postgres", "xpagesdemo.names", "lastname")}]]></xp:this.valueList>
</xp:typeAhead>
</xp:inputText>
See live demo here: http://www.notesx.net/postgres.nsf/typeahead.xsp
Ben, I think you mean you want to put the wildcard to the end of the parameter? Not sure of your data and if you are entering an exact match? Did you try:
var params = [CodePostal+ "%"];
Also, not sure if you need to enter "cifadmin.postal_codes" for your table as you did in your sql?
Howard
I'm trying to create a datatable in microsoft ax x++.
I need to return some information back to a webservice. Problem is i'm having trouble getting this code to work. it runs to the point where i try to add a column because apparently that function doesn't work....
Does anyone have the code in full to create a datatable in x++. this code works up until the column code.. because basically what i'm doing is query some information and returning it back to a webservice.. unless theres another way to return multiple information back to a webservice that uses c#
System.Data.DataTable dt = new System.Data.DataTable("MyTable");
System.Data.DataColumnCollection columns = dt.get_Columns();
System.Data.DataColumn ProductName;
System.Data.DataColumn QtyOrdered;
System.Data.DataColumn ProductID;
System.Data.DataRow row;
ProductID = new System.Data.DataColumn("ProductID", System.Type::GetType("System.Int32"));
ProductName = new System.Data.DataColumn("ProductName", System.Type::GetType("System.String"));
QtyOrdered = new System.Data.DataColumn("QtyOrdered", System.Type::GetType("System.String"));
// dt.Columns.Add(ProductID);
// dt.Columns.Add(ProductName);
// dt.Columns.Add(QtyOrdered);
row=dt.NewRow();
Did you try replacing lines
// dt.Columns.Add(ProductID);
// dt.Columns.Add(ProductName);
// dt.Columns.Add(QtyOrdered);
with
columns.Add(ProductID);
columns.Add(ProductName);
columns.Add(QtyOrdered);
?
P.S. To populate the row:
System.Data.DataRowCollection rows = dt.get_Rows();
...
row = dt.NewRow();
row.set_Item("ProductID", 1);
row.set_Item("ProductName", "PN");
row.set_Item("QtyOrdered", "QO");
rows.Add(row);
I suggest you to read some MSDN articles such as .NET Interop from X++, How To Use X++ Syntax for CLR Arrays, etc. - it should help you understand how to use .NET assemblies from X++.
This is a bit of subjective question about a specific situation. Main goal for this question for me is to remind my self to code up the solution. However if there is already a solution, or an alternate approach, I would like to know it.
I'm working on a project and I'm using Entity Framework 4 for database access. The database design is something that I don't have control over. The database was designed many years ago, and in my opinion the database design does not fit for the current database purposes. This results in very complicated queries.
This is the first time I'm using Entity Framework in a project, but I have extensive experience in development against MS SQL Server.
What I found myself doing again and again is this:
I write a complex L2E query. The query either slow or returns wrong results
I'm looking at my L2E query and I have absolutely no idea how to improve it
I fire up SQL Profiler and capture the SQL that EF generated from my query
I want to execute part of that sql to identify the part of the query that is giving problems
The query comes through as sp_executesql with a dozen of parameters, because if a parameter is used 3 times in a query, L2E creates 3 parameters and passes to all of them the same value. Same deal with every parameter.
Now I have to extract the SQL from sp_executesql, unescape all escaped apostrophes, and substitute every parameter in the query with its value
After this is done I finally can run parts of the query and pin-point the problem.
I go back to my L2E code, change it to fix the problem I found and the cycle repeats.
To be honest, I'm starting thinking that one should not use an ORM if you don't own database design.
This aside, the process of unescaping the sql and substituting the parameters is the one that I want to automate. The goal is to get 'naked', de-parametrized sql, that I can run in SSMS.
This a very simple example of what I see in the profile and what I want to get in result. My real cases are many times more complex.
The capture:
exec sp_executesql N'SELECT
[Extent1].[ProductName] AS [ProductName]
FROM [dbo].[Products] AS [Extent1]
INNER JOIN [dbo].[Categories] AS [Extent2] ON [Extent1].[CategoryID] = [Extent2].[CategoryID]
WHERE ([Extent1].[UnitPrice] > #p__linq__0) AND ([Extent2].[CategoryName] = #p__linq__1) AND (N''Chang'' <> [Extent1].[ProductName])',N'#p__linq__0 decimal(1,0),#p__linq__1 nvarchar(4000)',#p__linq__0=1,#p__linq__1=N'Beverages'
Desired result:
SELECT
[Extent1].[ProductName] AS [ProductName]
FROM [dbo].[Products] AS [Extent1]
INNER JOIN [dbo].[Categories] AS [Extent2] ON [Extent1].[CategoryID] = [Extent2].[CategoryID]
WHERE ([Extent1].[UnitPrice] > 1) AND ([Extent2].[CategoryName] = N'Beverages') AND (N'Chang' <> [Extent1].[ProductName])
I'm just going to write code to convert the likes of first to the likes of second if there is nothing better, I'll post the solution here. But maybe it's already done by someone? Or maybe there is a profiler or something, that can give me sql code I can execute partially in SSMS?
So here is what I ended up with. A couple of notes:
This won't work in 100% of cases, but this is good enough for me
There is a lot to improve in terms of usability. Currently I put a shortcut to the compiled binary on the desktop, cut the text to convert to clipboard, double-click the shortcut and paste the result.
using System;
using System.Text.RegularExpressions;
using System.Windows.Forms;
namespace EFC
{
static class Program
{
[STAThread]
static void Main()
{
try
{
string input = Clipboard.GetText();
const string header = "exec sp_executesql N'";
CheckValidInput(input.StartsWith(header), "Input does not start with {0}", header);
// Find part of the statement that constitutes whatever sp_executesql has to execute
int bodyStartIndex = header.Length;
int bodyEndIndex = FindClosingApostroph(input, bodyStartIndex);
CheckValidInput(bodyEndIndex > 0, "Unable to find closing \"'\" in the body");
string body = input.Substring(bodyStartIndex, bodyEndIndex - bodyStartIndex);
// Unescape 's
body = body.Replace("''", "'");
// Work out where the paramters are
int blobEndIndex = FindClosingApostroph(input, bodyEndIndex + 4);
CheckValidInput(bodyEndIndex > 0, "Unable to find closing \"'\" in the params");
string ps = input.Substring(blobEndIndex);
// Reverse, so that P__linq_2 does not get substituted in p__linq_20
Regex regexEf = new Regex(#"(?<name>#p__linq__(?:\d+))=(?<value>(?:.+?)((?=,#p)|($)))", RegexOptions.RightToLeft);
Regex regexLinqToSql = new Regex(#"(?<name>#p(?:\d+))=(?<value>(?:.+?)((?=,#p)|($)))", RegexOptions.RightToLeft);
MatchCollection mcEf = regexEf.Matches(ps);
MatchCollection mcLinqToSql = regexLinqToSql.Matches(ps);
MatchCollection mc = mcEf.Count > 0 ? mcEf : mcLinqToSql;
// substitutes parameters in the statement with their values
foreach (Match m in mc)
{
string name = m.Groups["name"].Value;
string value = m.Groups["value"].Value;
body = body.Replace(name, value);
}
Clipboard.SetText(body);
MessageBox.Show("Done!", "CEF");
}
catch (ApplicationException ex)
{
MessageBox.Show(ex.Message, "Error");
}
catch (Exception ex)
{
MessageBox.Show(ex.Message, "Error");
MessageBox.Show(ex.StackTrace, "Error");
}
}
static int FindClosingApostroph(string input, int bodyStartIndex)
{
for (int i = bodyStartIndex; i < input.Length; i++)
{
if (input[i] == '\'' && i + 1 < input.Length)
{
if (input[i + 1] != '\'')
{
return i;
}
i++;
}
}
return -1;
}
static void CheckValidInput(bool isValid, string message, params object[] args)
{
if (!isValid)
{
throw new ApplicationException(string.Format(message, args));
}
}
}
}
Well,may be this will be helpfull. MSVS 2010 has IntelliTrace. Every time when EF make a query there is an ADO.Net Event with a query
Execute Reader "SELECT TOP (1)
[Extent1].[id] AS [id],
[Extent1].[Sid] AS [Sid],
[Extent1].[Queue] AS [Queue],
[Extent1].[Extension] AS [Extension]
FROM [dbo].[Operators] AS [Extent1]
WHERE [Extent1].[Sid] = #p__linq__0" Command Text = "SELECT TOP (1) \r\n[Extent1].[id] AS [id], \r\n[Extent1].[Sid] AS [Sid], \r\n[Extent1].[Queue] AS [Queue], \r\n[Extent1].[Extension] AS [Extension]\r\nFROM [dbo].[Operators] AS [Extent1]\r\nWHERE [Extent1].[Sid] = #p__linq__0",
Connection String = "Data Source=paris;Initial Catalog=telephony;Integrated Security=True;MultipleActiveResultSets=True"
Actually I am not sure there is a problem but, I have some data and I want to insert it to database.
Code seems to works well, every statement works, there are no errors shown but I cant see the data in database. It doesn't insert the data to database but in code it is shown as data is inserted.
Is there a setting related with database? I am using Sqlite Firefox Add-on.
When selecting records there are no problems. I checked the .sqlite file's permissions but it is setted to read & write.
Sorry, I forgot to add code snippet;
NSString *stmt=[NSString stringWithFormat:#"INSERT INTO t_exam_applies(user_id,exam_id, apply_correct, apply_wrong, apply_empty, apply_start, apply_end) VALUES(%d, %d, %d, %d, %d, %d, %d)", 0,exam_id,0,0,0,0,0];
[stmt UTF8String];
const char *sql=(const char *) [stmt UTF8String];
//const char *sql= "INSERT INTO t_exam_applies(user_id,exam_id, apply_correct, apply_wrong, apply_empty, apply_start, apply_end) VALUES(?,?,?,?,?,?,?)";
sqlite3_stmt *statement;
sqlite3_prepare_v2(database, sql, -1, &statement, NULL);
if (sqlite3_step(statement) == SQLITE_DONE)
NSLog(#"SQLITE statement executed");
else
NSLog(#"ERROR!!");
sqlite3_finalize(statement);
Given that this is tagged for iPhone dev I would recommend using an objective c wrapper for sqlite like FMDB. FMDB is very lightweight and widely used (check it out on github: https://github.com/ccgus/fmdb/) and will simplify this whole process for you.
Here is an example of a simple insert with FMDB:
self.db = [FMDatabase databaseWithPath:fullPath];
if (![db open]) {
NSLog(#"DB Open Failed");
return NO;
}
[db executeUpdate:#"INSERT INTO TestTable (id, value) VALUES (?,?)", [NSNumber numberWithInt:1], #"SomeString"];
[db close];
Maybe you are messaging nil when adding the data (assuming you use Obj-C)? That's not an error, but it also doesn't do anything. More info or code snippet needed for more specific answer.