What is the faster way to access a DataTable/DataRowsCollection? - performance

I have a datatable with 100,000+ DataRow. Which method is faster to access the collection?
Is there any faster way to process the rows collection ?
Method 1:
var rows= dsDataSet.Tables["dtTableName"].Rows;
int rowCount = dsDataSet.Tables["dtTableName"].Rows.Count;
for (int c = 0; c < rowCount; c++)
{
var theRow = rows[c];
//process the dataRow
}
Method 2:
for (int c = 0; c < dsDataSet.Tables["dtTableName"].Rows.Count; c++)
{
var theRow = dsDataSet.Tables["dtTableName"].Rows[c];
//process the dataRow
}

It is worth noting the most direct way to access cells is via the DataColumn indexer; the data is actually stored in the columns, not the rows (no: really).
So something like:
var table = dataSet.Tables["dtTableName"];
// HERE: fetch the DataColumn of those you need, for example:
var idCol = table.Columns["Id"];
var nameCol = table.Columns["Name"];
// now loop
foreach(DataRow row in table.Rows)
{
var id = (int)row[idCol];
var name = (string)row[nameCol];
// ...
}
However, frankly if you want the best performance, I would start by saying "don't use DataSet / DataTable". That is actually a very complicated model designed to be all kinds of flexible, with change tracking, rule enforcement, etc. If you want fast, I'd use a POCO and something like "dapper", for example:
public class Foo {
public int Id {get;set;}
public string Name {get;set;}
}
...
string region = "North";
foreach(var row in conn.Query<Foo>("select * from [Foo] where Region = #region",
new { region })) // <=== simple but correct parameterisation
{
// TODO: do something with row.Id and row.Name, which are direct
// properties of the Foo row returned
var id = row.Id;
var name = row.Name;
// ...
}
or even skip the type via dynamic:
string region = "North";
foreach(var row in conn.Query("select * from [Foo] where Region = #region",
new { region })) // ^^^ note no <Foo> here
{
// here "row" is dynamic, but still works; not quite as direct as a
// POCO object, though
int id = row.Id; // <=== note we can't use `var` here or the
string name = row.Name; // variables would themselves be "dynamic"
// ...
}

Related

ASP.NET Core 3.1 / EF Core - Search table column by string name

Currently using:
ASP.NET Core 3.1 / EF Core
C#
Code-first approach
Postgres database
I'm building a method to support column searching on a table. I need to feed the column name to be searched by string value and build a query / lambda that can search the right column. I suspect I need to build some sort of expression and search on the expression but am having trouble with the syntax.
Here's the base code:
string search = "Search Value";
string givenColumn = "search_column";
IQueryable<MyModel> data = _dbContext.table;
data = data.Where(data => data.givenColumn.Contains(search));
I'd like to feed the column name in givenColumn and be able to build a query that searches the right column. At first I thought I wanted reflection but I'm looking to build a SQL query based off of a string, so I think I want to build an expression?
TIA!
Here is some sample code for a runtime WhereContains that operates on string columns:
public static class IQueryableExt {
// String.Contains(string)
static MethodInfo containsMI = typeof(string).GetMethod("Contains", 0, new[] { typeof(string) });
// generate r => r.{columnname}.Contains(value)
static Expression<Func<T, bool>> WhereContainsExpr<T>(string columnname, string value) {
// (T r)
var rParm = Expression.Parameter(typeof(T), "r");
// r.{columnname}
var rColExpr = Expression.Property(rParm, columnname);
// r.{columnname}.Contains(value)
var bodyExpr = Expression.Call(rColExpr, containsMI, Expression.Constant(value));
return Expression.Lambda<Func<T,bool>>(bodyExpr, rParm);
}
public static IQueryable<T> WhereContains<T>(this IQueryable<T> src, string columname, string value) => src.Where(WhereContainsExpr<T>(columname, value));
}
Just pass HTML Table id as a parameter onkeyup method of input field. HTML Code:
<input type="text" id="myInput" class="form-control search-input" onkeyup="searchData('myTable')" placeholder="Search...">
Javascript Code for exact match of any column:
function searchData(tableId) {
// Declare variables
var input, filter, table, tr, i, j, column_length, count_td;
column_length = document.getElementById(tableId).rows[0].cells.length;
input = document.getElementById("myInput");
filter = input.value.toUpperCase();
table = document.getElementById(tableId);
tr = table.getElementsByTagName("tr");
if (filter != "") {
for (i = 1; i < tr.length; i++) { // except first(heading) row
count_td = 0;
for (j = 1; j < column_length - 1; j++) { // except first column
td = tr[i].getElementsByTagName("td")[j];
/* ADD columns here that you want you to filter to be used on */
if (td) {
if (td.innerHTML.toUpperCase() === filter) {
count_td++;
}
}
}
if (count_td > 0) {
tr[i].style.display = "";
} else {
tr[i].style.display = "none";
}
}
}
else {
for (i = 1; i < tr.length; i++) {
tr[i].style.display = "";
}
}
}

Dynamically choose which properties to get using Linq

I have an MVC application with a dynamic table on one of the pages, which the users defines how many columns the table has, the columns order and where to get the data from for each field.
I have written some very bad code in order to keep it dynamic and now I would like it to be more efficient.
My problem is that I don't know how to define the columns I should get back into my IEnumerable on runtime. My main issue is that I don't know how many columns I might have.
I have a reference to a class which gets the field's text. I also have a dictionary of each field's order with the exact property It should get the data from.
My code should look something like that:
var docsRes3 = from d in docs
select new[]
{
for (int i=0; i<numOfCols; i++)
{
gen.getFieldText(d, res.FieldSourceDic[i]);
}
};
where:
docs = List from which I would like to get only specific fields
res.FieldSourceDic = Dictionary in which the key is the order of the column and the value is the property
gen.getFieldText = The function which gets the entity and the property and returns the value
Obviously, it doesn't work.
I also tried
StringBuilder fieldsSB = new StringBuilder();
for (int i = 0; i < numOfCols; i++)
{
string field = "d." + res.FieldSourceDic[i] + ".ToString()";
if (!string.IsNullOrEmpty(fieldsSB.ToString()))
{
fieldsSB.Append(",");
}
fieldsSB.Append(field);
}
var docsRes2 = from d in docs
select new[] { fieldsSB.ToString() };
It also didn't work.
The only thing that worked for me so far was:
List<string[]> docsRes = new List<string[]>();
foreach (NewOriginDocumentManagment d in docs)
{
string[] row = new string[numOfCols];
for (int i = 0; i < numOfCols; i++)
{
row[i] = gen.getFieldText(d, res.FieldSourceDic[i]);
}
docsRes.Add(row);
}
Any idea how can I pass the linq the list of fields and it'll cut the needed data out of it efficiently?
Thanks, Hoe I was clear about what I need....
Try following:
var docsRes3 = from d in docs
select (
from k in res.FieldSourceDic.Keys.Take(numOfCols)
select gen.getFieldText(d, res.FieldSourceDic[k]));
I got my answer with some help from the following link:
http://www.codeproject.com/Questions/141367/Dynamic-Columns-from-List-using-LINQ
First I created a string array of all properties:
//Creats a string of all properties as defined in the XML
//Columns order must be started at 0. No skips are allowed
StringBuilder fieldsSB = new StringBuilder();
for (int i = 0; i < numOfCols; i++)
{
string field = res.FieldSourceDic[i];
if (!string.IsNullOrEmpty(fieldsSB.ToString()))
{
fieldsSB.Append(",");
}
fieldsSB.Append(field);
}
var cols = fieldsSB.ToString().Split(',');
//Gets the data for each row dynamically
var docsRes = docs.Select(d => GetProps(d, cols));
than I created the GetProps function, which is using my own function as described in the question:
private static dynamic GetProps(object d, IEnumerable<string> props)
{
if (d == null)
{
return null;
}
DynamicGridGenerator gen = new DynamicGridGenerator();
List<string> res = new List<string>();
foreach (var p in props)
{
res.Add(gen.getFieldText(d, p));
}
return res;
}

LINQ - How to read ID which is Autonumber after submitting the data

I am working on LINQ. and in one transaction I have to update three different tables.
e.g A(A1,A2,A3) B(B1,B2) AB(A1,B1)
here B1 is autonumber in my database. I want to submit all the changes together. so in one data context I wrote
using (BBBDataContext DC= new BBBDataContext())
{
A tba = new A()
{
A1 = this.A1,
A2 = this.A2,
A3 = this.A3,
};
DC.A.InsertOnSubmit(tba);
B tbb= new B()
{
B2 = this.B2,
};
DC.B.InsertOnSubmit(tbb);
// NOW i WONT B1(WHICH IS AUTONUMBER) FROM B SO THAT I USE IT IN MY NEXT TABLE.
AB tbab = new AB()
{
A1 = this.A1,
B1 = ??????,
};
DC.AB.InsertOnSubmit(tbab);
//FINALLY I WILL WRITE SUBMIT CHANGES SO MY ALL TABLES GET CHANGES SAME TIME
DC.SubmitChanges();
}
Que: what should I write # the place of ?????. for B1 in AB table??
Short answer: B
Bit longer answer:
When you create the object and add it you won't get the ID before you commit. So what you do is reference the object and not the ID property of the object.
class A {
public int Id { get;set;}
public ICollection <B> Bees { get;set;}
}
class B {
public int Id { get;set;}
public int InstanceOfAId { get;set;}
public A InstanceOfA { get;set;}
}
var a = new A();
var b = new B();
b.InstanceOfA = a;
Depending on your model you will define a relationship. In code first you could do it this way:
EntityTypeConfiguration<B> cfgB = new EntityTypeConfiguration<B>();
cfgB.HasRequired(p => p.InstanceOfA).WithMany(p => p.Bees)
.HasForeignKey(p => p.InstanceOfAId);
In EDMX you can do the same in the designer.
You will use InstanceOfA to assign to when inserting, after that InstanceOfAId will always hold the Id value. When retrieving data InstanceOfA will only be filled when requested.
You can use it like this:
var x = DC.B.Include(p=>p.A);
or
var x = DC.B.Select(p=> new { Id = p.Id, A_Id = p.A.Id});
or
var x = DC.A.Select(p=> new { Id = p.Id, BeeCount = p.Bees.Count()});

EF single entity problem

I need to return a single instance of my viewmodel class from my repository in order to feed this into a strongly-typed view
In my repository, this works fine for a collection of viewmodel instances:
IEnumerable<PAWeb.Domain.Entities.Section> ISectionsRepository.GetSectionsByArea(int AreaId)
{
var _sections = from s in DataContext.Sections where s.AreaId == AreaId orderby s.Ordinal ascending select s;
return _sections.Select(x => new PAWeb.Domain.Entities.Section()
{
SectionId = x.SectionId,
Title = x.Title,
UrlTitle = x.UrlTitle,
NavTitle = x.NavTitle,
AreaId = x.AreaId,
Ordinal = x.Ordinal
}
);
}
But when I attempt to obtain a single entity, like this:
public PAWeb.Domain.Entities.Section GetSection(int SectionId)
{
var _section = from s in DataContext.Sections where s.SectionId == SectionId select s;
return _section.Select(x => new PAWeb.Domain.Entities.Section()
{
SectionId = x.SectionId,
Title = x.Title,
UrlTitle = x.UrlTitle,
NavTitle = x.NavTitle,
AreaId = x.AreaId,
Ordinal = x.Ordinal
}
);
}
I get
Error 1 Cannot implicitly convert type
'System.Linq.IQueryable<PAWeb.Domain.Entities.Section>' to
'PAWeb.Domain.Entities.Section'. An explicit conversion exists
(are you missing a cast?)"
This has got to be simple, but I'm new to c#, and I can't figure out the casting. I tried (PAWeb.Domain.Entities.Section) in various places, but no success. Can anyone help??
Your query is returning an IQueryable, which could have several items. For example, think of the difference between an Array or List of objects and a single object. It doesn't know how to convert the List to a single object, which one should it take? The first? The last?
You need to tell it specifically to only take one item.
e.g.
public PAWeb.Domain.Entities.Section GetSection(int SectionId)
{
var _section = from s in DataContext.Sections where s.SectionId == SectionId select s;
return _section.Select(x => new PAWeb.Domain.Entities.Section()
{
SectionId = x.SectionId,
Title = x.Title,
UrlTitle = x.UrlTitle,
NavTitle = x.NavTitle,
AreaId = x.AreaId,
Ordinal = x.Ordinal
}
).FirstOrDefault();
}
This will either return the first item, or null if there are no items that match your query. In your case that won't happen unless the table is empty since you don't have a where clause.

Iterating tables in a context and the properties of those tables

I'm iterating the tables of a context and then the properties of those tables to eager load all columns in a context. I received some help via another question, but I don't seem to be able to figure out how to iterate the column properties of the actual table.
Final working code:
public static void DisableLazyLoading(this DataContext context)
{
DataLoadOptions options = new DataLoadOptions();
var contextTables = context.GetType().GetProperties().Where(n => n.PropertyType.Name == "Table`1");
foreach (var contextTable in contextTables)
{
var tableType = contextTable.GetValue(context, null).GetType().GetGenericArguments()[0];
var tableProperties = tableType.GetProperties().Where(n => n.PropertyType.Name != "EntitySet`1");
foreach (var tableProperty in tableProperties)
{
ParameterExpression paramExp = Expression.Parameter(tableType, "s");
Expression expr = Expression.Property(paramExp, tableProperty.Name);
options.LoadWith(Expression.Lambda(expr, paramExp));
}
}
context.LoadOptions = options;
}
You're only getting the ProperInfos. You need to get the values from the PropertyInfos:
var tablePropertInfos = context.GetType().GetProperties().Where(
n => n.PropertyType.Name == "Table`1");
foreach (var tablePropertyInfo in tablePropertInfos)
{
// Get the actual table
var table = tablePropertyInfo.GetValue(context, null);
// Do the same for the actual table properties
}
Once you have the PropertyInfo class, you need to get the value using the GetValue method.

Resources