Dynamic Linq to Datatable Derived Field - linq

Is it possible to use Dynamic Linq to run a query similar to:
Select a, b, a + b as c
from MyDataTable
I have an application where the user can enter SQL statements, the results of these statements are then assigned to a DataTable. There is also the option to derive a field based on other fields. (e.g. user can say field C = a + b, or field D = A*B+10 etc).
Ideally I would like to do something similar to:
string myCalc = "Convert.ToDouble(r.ItemArray[14])+Convert.ToDouble(r.ItemArray[45])";
var parameters = from r in dt.AsEnumerable()
select (myCalc);
What I want to do in this example is add the value of column 14 to column 45 and return it. It's up to the user to decide what expression to use so the text in the select needs to be from a string, I cannot hard code the expression. The string myCalc is purely for demonstration purposes.

You could do that using a Dictionary, and a DataReader and Dynamic Queries. Here is an example based in part in Rob Connery's Massive ORM RecordToExpando:
void Main()
{
string connString = "your connection string";
System.Data.SqlClient.SqlConnection conn = new SqlConnection(connString);
string statement = "SUM = EstimatedEffort + OriginalEstimate, Original = OriginalEstimate";
// Note: You should parse the statement so it doesn't have any updates or inserts in it.
string sql = "SELECT " + statement +" FROM Activities";
List<IDictionary<string, object>> results = new List<IDictionary<string, object>>();
conn.Open();
using(conn)
{
var cmd = new SqlCommand(sql, conn);
var reader = cmd.ExecuteReader();
while (reader.Read())
{
var dic = new Dictionary<string, object>();
for (int i = 0; i < reader.FieldCount; i++)
{
dic.Add(
reader.GetName(i),
DBNull.Value.Equals(reader[i]) ? null : reader[i]);
}
results.Add(dic);
}
}
foreach (var dicRow in results)
{
foreach (string key in dicRow.Keys)
{
Console.Write("Key: " + key + " Value: " + dicRow[key]);
}
Console.WriteLine();
}
}

Something like this:
void Main()
{
var dataTable = new DataTable();
dataTable.Columns.Add("a", typeof(double));
dataTable.Columns.Add("b", typeof(double));
dataTable.Rows.Add(new object[] { 10, 20 });
dataTable.Rows.Add(new object[] { 30, 40 });
string myCalc = "Convert.ToDouble(ItemArray[0]) + Convert.ToDouble(ItemArray[1])";
var query = dataTable.AsEnumerable().AsQueryable();
var result = query.Select(myCalc);
foreach (Double c in result)
{
System.Console.WriteLine(c);
}
}

Related

'Argument expression is not valid' when creating anonymous types dynamically

I'm creating an expression tree builder to return custom anonymous types. I tried it first with discrete types and it works ok, but using TypeBuilder to build types at runtime and pass that type to the expression tree fail with this error
'Argument expression is not valid'
here is the code I use:
this method I use to create the anonymous type
private Type CreateAnonymousType(Dictionary<string, Type> properties)
{
AssemblyName dynamicAssemblyName = new AssemblyName("MyAssembly");
AssemblyBuilder dynamicAssembly = AssemblyBuilder.DefineDynamicAssembly(dynamicAssemblyName, AssemblyBuilderAccess.Run);
ModuleBuilder dynamicModule = dynamicAssembly.DefineDynamicModule("MyAssembly");
TypeBuilder dynamicAnonymousType = dynamicModule.DefineType("ReturnType", TypeAttributes.Public | TypeAttributes.Class | TypeAttributes.AutoClass);
foreach (var p in properties)
{
dynamicAnonymousType.DefineField(p.Key, p.Value, FieldAttributes.Public);
}
return dynamicAnonymousType.CreateType();
}
and here is how I create the expression tree
var cars = new List<Car>();
for (int i = 0; i < 10; i++)
{
cars.Add(new Car { Id = i, Name = "Car " + i, Age = 2010 + i });
}
IQueryable<Car> allCars = cars.AsQueryable();
var properties = new Dictionary<string, Type>
{
{ "Id", typeof(int) },
{ "Name", typeof(string) }
};
ParameterExpression x = Expression.Parameter(typeof(Car), "x");
var listMembers = properties.Select(p => Expression.Property(x, p.Key));
var returnType = CreateAnonymousType(properties);
object destObject = Activator.CreateInstance(returnType);
var listBind = listMembers.Select(p => Expression.Bind(returnType.GetField(p.Member.Name), p));
var result = Expression.New(returnType);
var initExp = Expression.MemberInit(result, listBind.ToArray());
var call = Expression.Call(typeof(Queryable), "Select",
new Type[] {
typeof(Car),
returnType
}
, Expression.Constant(allCars)
, Expression.Lambda(initExp, x));
var qResult = allCars.Provider.CreateQuery<IdName>(call);
foreach (var car in qResult)
{
Console.WriteLine(car.Id + " - " + car.Name);
}
the error happened while CreateQuery method executes
This is because call returns dynamically created ReturnType not IdName thus the exception. Additionally you cannot put such dynamic types like ReturnType as generic type parameters because compiler knows nothing about them so you should use dynamic instead so the type will be resolved at runtime:
var qResult = allCars.Provider.CreateQuery<dynamic>(call);

How can improve this Linq query expressions performance?

public bool SaveValidTicketNos(string id,string[] ticketNos, string checkType, string checkMan)
{
bool result = false;
List<Carstartlistticket>enties=new List<Carstartlistticket>();
using (var context = new MiniSysDataContext())
{
try
{
foreach (var ticketNo in ticketNos)
{
Orderticket temp = context.Orderticket.ByTicketNo(ticketNo).SingleOrDefault();
if (temp != null)
{
Ticketline ticketline= temp.Ticketline;
string currencyType = temp.CurrencyType;
float personAllowance=GetPersonCountAllowance(context,ticketline, currencyType);
Carstartlistticket carstartlistticket = new Carstartlistticket()
{
CsltId = Guid.NewGuid().ToString(),
Carstartlist = new Carstartlist(){CslId = id},
LeaveDate = temp.LeaveDate,
OnPointName = temp.OnpointName,
OffPointName = temp.OffpointName,
OutTicketMan = temp.OutBy,
TicketNo = temp.TicketNo,
ChekMan = checkMan,
Type = string.IsNullOrEmpty(checkType)?(short?)null:Convert.ToInt16(checkType),
CreatedOn = DateTime.Now,
CreatedBy = checkMan,
NumbserAllowance = personAllowance
};
enties.Add(carstartlistticket);
}
}
context.BeginTransaction();
context.Carstartlistticket.InsertAllOnSubmit(enties);
context.SubmitChanges();
bool changeStateResult=ChangeTicketState(context, ticketNos,checkMan);
if(changeStateResult)
{
context.CommitTransaction();
result = true;
}
else
{
context.RollbackTransaction();
}
}
catch (Exception e)
{
LogHelper.WriteLog(string.Format("CarstartlistService.SaveValidTicketNos({0},{1},{2},{3})",id,ticketNos,checkType,checkMan),e);
context.RollbackTransaction();
}
}
return result;
}
My code is above. I doubt these code have terrible poor performance. The poor performance in the point
Orderticket temp = context.Orderticket.ByTicketNo(ticketNo).SingleOrDefault();
,actually, I got an string array through the method args,then I want to get all data by ticketNos from database, here i use a loop,I know if i write my code like that ,there will be cause performance problem and it will lead one more time database access,how can avoid this problem and improve the code performance,for example ,geting all data by only on databse access
I forget to tell you the ORM I use ,en ,the ORM is PlinqO based NHibernate
i am looking forward to having your every answer,thank you
using plain NHibernate
var tickets = session.QueryOver<OrderTicket>()
.WhereRestrictionOn(x => x.TicketNo).IsIn(ticketNos)
.List();
short? type = null;
short typeValue;
if (!string.IsNullOrEmpty(checkType) && short.TryParse(checkType, out typeValue))
type = typeValue;
var entitiesToSave = tickets.Select(ticket => new Carstartlistticket
{
CsltId = Guid.NewGuid().ToString(),
Carstartlist = new Carstartlist() { CslId = id },
LeaveDate = ticket.LeaveDate,
OnPointName = ticket.OnpointName,
OffPointName = ticket.OffpointName,
OutTicketMan = ticket.OutBy,
TicketNo = ticket.TicketNo,
ChekMan = checkMan,
CreatedOn = DateTime.Now,
CreatedBy = checkMan,
Type = type,
NumbserAllowance = GetPersonCountAllowance(context, ticket.Ticketline, ticket.CurrencyType)
});
foreach (var entity in entitiesToSave)
{
session.Save(entity);
}
to enhance this further try to preload all needed PersonCountAllowances

syntax to return a number of results per page

I have this linq statement that returns result and i wanted to add the specifications of how many items are displayed per page. I know the default is 10 per page how can i change it to 40?
var _roster = DataCBase.StoredProcedures.GetUser<Users>(userID, r => new Users
{
Name = RosterColumnMap.Name(r),
Email = RosterColumnMap.Email(r)
});
Get User
public virtual IEnumerable<T> GetUser<T>(int userId, Func<IDataRecord, T> modelBinder, int resultsPerPage = 10, int pageNumber = 1)
{
if (userId < 1)
throw new NullReferenceException("The sectionId cannot be null, when retreiving an element");
if (resultsPerPage < 1)
resultsPerPage = 1; // enforce bare minimum result set
if (pageNumber < 1)
pageNumber = 1; // enforce one-based page numbering
SqlCommand _command = new SqlCommand("dbo.GetUser");
_command.CommandType = CommandType.StoredProcedure;
_command.Parameters.Add(new SqlParameter { ParameterName = "userId", SqlDbType = SqlDbType.Int, Value = userId });
_command.Parameters.Add(new SqlParameter { ParameterName = "resultsPerPage", SqlDbType = SqlDbType.Int, Value = resultsPerPage });
_command.Parameters.Add(new SqlParameter { ParameterName = "pageNumber", SqlDbType = SqlDbType.Int, Value = pageNumber });
return DbInstance.ExecuteAs<T>(_command, modelBinder);
}
Neither Linq nor entity framework have any default number of records 'per page'. But since your GetUser function includes a resultsPerPage parameter, you can just do this:
var _roster = DataCBase.StoredProcedures.GetUser<Users>(userID, r => new Users
{
Name = RosterColumnMap.Name(r),
Email = RosterColumnMap.Email(r)
}, 40);
To limit the number of results in Linq use the the Enumerable.Take method:
var _roster = DataCBase.StoredProcedures.GetUser<Users>(userID, r => new Users
{
Name = RosterColumnMap.Name(r),
Email = RosterColumnMap.Email(r)
}).Take(40);

Comparing very large List<string> with database table via LINQ to Entities

I have a very large list of strings containing GUIDs and a 100,000+ record table in a database with an Entity Framework model in my app.
What' is the most efficient approach to finding all records in a particular dataset where the GUID List is not present?
The following is performing very slowly:
var list= new List<string> { "1", "2", "3" };
return (from t1 in db.Items
where (!list.Contains(t1.GUID))
When I have a lot of parameters (several hundreds or more) to a query I use bulk insert to temporary table and then join it from main table.
My code looks something like this:
private static DataTable FillDataTable(IEnumerable<int> keys) {
var dataTable = new DataTable("Stage");
dataTable.Locale = CultureInfo.CurrentCulture;
dataTable.Columns.Add("Key", typeof(int));
foreach (var key in keys) {
var row = dataTable.NewRow();
row[0] = key;
dataTable.Rows.Add(row);
}
return dataTable;
}
private static void CreateStageTable(SqlConnection connection, string tableName, DataTable dataTable) {
var sql = new StringBuilder();
sql.AppendLine("CREATE TABLE {StageTableName} ( ");
sql.AppendLine(" Key INT NOT NULL ");
sql.AppendLine(") ");
sql.Replace("{StageTableName}", SqlUtilities.QuoteName(tableName));
using (var command = connection.CreateCommand()) {
command.CommandText = sql.ToString();
command.CommandType = CommandType.Text;
command.ExecuteNonQuery();
}
using (var bulkcopy = new SqlBulkCopy(connection)) {
bulkcopy.DestinationTableName = tableName;
bulkcopy.WriteToServer(dataTable);
}
}
public void DoQuery(IEnumerable<int> keys) {
var dataTable = FillDataTable(keys);
using (var connection = new SqlConnection(_connectionString)) {
connection.Open();
CreateStageTable(connection, "#Stage", dataTable);
string sql = "SELECT x " +
"FROM tbl " +
" LEFT JOIN {StageTableName} AS Stage " +
" ON x.Key = Stage.Key "
"WHERE Stage.Key IS NULL";
...
}
}
Don't use a List, use a HashSet<string>, this will give you O(1) lookup ups instead of O(n).

Multiple result set from SQL Via LiNQ

I have a stored procedure as
pr___GetArchiveData
Select * from TABLE1
SELECT * FROM TABLE2
SELECT * FROM TABLE 3
I want to get this result set into a dataset. Or access the values of three select queries!!
I have a DBML file in which when i drag and drop the stored procedure generates a code as follows:-
global::System.Data.Linq.Mapping.FunctionAttribute(Name="dbo.pr___GetArchiveData")]
public ISingleResult<pr___GetArchiveDataResult> pr___GetArchiveData([global::System.Data.Linq.Mapping.ParameterAttribute(DbType="UniqueIdentifier")] System.Nullable<System.Guid> projectID)
{
IExecuteResult result = this.ExecuteMethodCall(this, ((MethodInfo)(MethodInfo.GetCurrentMethod())), projectID);
return ((ISingleResult<pr__Project_pr___GetArchiveData>)(result.ReturnValue));
}
In the code MVC3 Architecture + LINQ i have written a code to get the result set as follows :-
using (HBDataContext hb = new HBDataContext())
{
System.Data.DataSet ds = new System.Data.DataSet();
String connString = connString;
var conn = new System.Data.SqlClient.SqlConnection(connString);
var cmd = conn.CreateCommand();
cmd.CommandType = System.Data.CommandType.StoredProcedure;
cmd.CommandText = "pr__GetArchiveData";
cmd.Connection.Open();
var mReader = cmd.ExecuteReader(System.Data.CommandBehavior.SequentialAccess);
//var reader = cmd.ExecuteReader();
//using (System.Data.SqlClient.SqlDataReader mReader = cmd.ExecuteReader())
//{
// while (mReader.Read())
//{
// mReader.Read();
var tbl1 = hb.Translate<tbl1 >(mReader).ToList();
// mReader = cmd.ExecuteReader();
mReader.NextResult();
var tbl2 = hb.Translate<tbl2 >(mReader).ToList();
mReader.NextResult();
var tbl3 = hb.Translate<tbl3>(mReader).ToList();
// }
// }
}
But while running it throws error as -
"Invalid attempt to call NextResult when reader is closed."
I am not sure where i am wrong!!
I have tried using it as while
(mReader.Read())
Kindly suggest!!!!
The code gen for ISingleResult won't provide multiple result sets
Try adding your own IMultipleResults wrapper - see the tutorial in http://blogs.msdn.com/b/dditweb/archive/2008/05/06/linq-to-sql-and-multiple-result-sets-in-stored-procedures.aspx
In .dbml file for the stored procedure the code will be like
[global::System.Data.Linq.Mapping.FunctionAttribute(Name="dbo.pr__Home_GetArchiveData")]
public ISingleResult<pr__Home_GetArchiveData> pr__Home_GetArchiveData([global::System.Data.Linq.Mapping.ParameterAttribute(Name="AlbumID", DbType="UniqueIdentifier")] System.Nullable<System.Guid> albumID)
{
IExecuteResult result = this.ExecuteMethodCall(this, ((MethodInfo)(MethodInfo.GetCurrentMethod())), albumID);
return ((ISingleResult<pr__Album_GetAlbumsFilesResult>)(result.ReturnValue));
}
Replace it with IMultipleResult as below `
[global::System.Data.Linq.Mapping.FunctionAttribute(Name = "dbo.pr__Home_GetArchiveData")]
[ResultType(typeof(tbl1))]
[ResultType(typeof(tbl2))]
[ResultType(typeof(tbl3))]
[ResultType(typeof(tbl4))]
public IMultipleResults pr__Home_GetArchiveData([global::System.Data.Linq.Mapping.ParameterAttribute(Name = "HOMEID", DbType = "UniqueIdentifier")] System.Nullable hOMEID)
{
IExecuteResult result = this.ExecuteMethodCall(this, ((MethodInfo)(MethodInfo.GetCurrentMethod())), hOMEID);
return ((IMultipleResults)result.ReturnValue);
}
in the code .
using (HBDataContext hb = new HBDataContext())
{
using (System.Data.Linq.IMultipleResults _results = hb.pr__Home_GetArchiveData(model.HomeID))
{
List<tbl1> _tbl1= _results.GetResult<tbl1>().ToList();
List<tbl2> _tbl2= _results.GetResult<tbl2>().ToList();
List<tbl3> _tbl3= _results.GetResult<tbl3>().ToList();
List<tbl4> _tbl4= _results.GetResult<tbl4>().ToList();}}
You will get the values of the Select queries from theStoredProcedure ...

Resources