How easy would it be to write a dumb LINQ provider that can just use my class definitions (which don't have any object references as properties) and give me the translated SQL. It can assume the name of the properties and the columns to be same as well as the names of the classes and the underlying tables. Can you please give me some pointers.?
It took me about 4 months of fulltime work (8 hours a day) to build a stable, working provider that implements the entire spec of linq. I would say I had a very simple, buggy and unstable version after about three weeks, so if you're just looking for something rough I would say you're probably looking at anything from a week up to two months depending on how good you are and what types of requiements you have.
I must point you to the Wayward blog for this, Matt has written a really good walkthrough on how to implement a linq provider, and even if you're probably not going to be able to copy and paste, it will help you to get to grips with how to think when working. You can find Matt´s walkthrough here: http://blogs.msdn.com/mattwar/archive/2007/07/30/linq-building-an-iqueryable-provider-part-i.aspx . I recommend you go about it the same way Matt does, and extend the expression tree visitor Matt includes in the second part of his tutorial.
Also, when I began working with this, I had so much help from the expression tree visualizer, it really made parsing a whole lot easier once you could see how linq parsed to queries.
Building a provider is really a lot of fun, even if a bit frustrating at times. I wish you all the best of luck!
Give a look to the LINQExtender project, is a toolkit for creating custom LINQ providers.
Another option for giving you a leg up seems to be re-linq which is a framework for creating custom LINQ providers.
Here's the Source code and a nice overview (pdf) of what's involved in writing one.
I’ve written a tutorial series on my blog base on my experience developing a LINQ-to-SQL provider from scratch, starting with the expression tree composition stage (calling the LINQ methods), continuing with the expression visitor, breaking down of the query into components, parsing the where clause, generating the text and parameter and, eventually, compiling the whole thing into IL using the .NET expression namespace.
I’ve seen many incomplete posts that promised to explain how to write a provider, falling very short of the mark, barely scratching the surface and not actually delivering anything remotely executable.
The blog series I’ve written based on my experience has a sample project available for download with the simple provider that covers only the functionality required by the tutorial example. However, it also includes the production version supporting a number of operations (where, join, first, count, top, etc.), subqueries, nested statements, and etc. Additionally, it produces a cleaner SQL than a lot of what I’ve seen from Entities and LINQ-to-SQL. There’s no unnecessary/redundant nesting, wrapping everything in brackets and etc.
For anyone with a good level of abstract thinking, developing such a provider isn’t such a difficult task many set it out to be. I’ve developed one that’s used in production environment in about 3 months of part time work (meaning some evenings and weekends). From the get go it was aimed with performance and tidy SQL in mind – a goal it achieved.
It was a little hard to find the time to publish this material, but I thought – if it may help someone out there, there’s no reason for this experience to go to waste:
How to write a LINQ to SQL provider in C# Part 1 - Introduction
How to write a LINQ to SQL provider in C# Part 2 - Expression Visitor
How to write a LINQ to SQL provider in C# Part 3 - Where Clause Visitor
How to write a LINQ to SQL provider in C# Part 4 - Compiling Expression Trees
I have created a project 'LinqToAnything' which is designed to make it very very easy to implement a (simple) Linq provider.
Related
I'm looking for an application to display what a linq expression would do, in particular regarding the usage of multiple access to the same list in a query.
Better yet, the tool would tell if the linq query is good.
I used the expression tree visualizer in the past to at least help decode what is inside of an expression tree. It aids in figureing out the parts of the tree and how gives each part is related.
Well, to begin with, I could easily foresee a tool that would pick a query apart and detect that the Where-clause is the standard runtime implementation, and thus not examine that method, but "know" what the execution plan for that method would be, and could thus piece together a plan for the whole query.
Right up until the point where you introduce a custom Linq provider, where the only way to figure out what it will be doing would be to read the code.
So I daresay there is no such tool, and making one would be very hard.
Would be fun to try though, at least for standard classes, would be a handy debugging visualizer for Visual Studio.
What about making the tool yourself?! ;)
Take a look at Expression trees, I believe they might be useful
So - highly hypothetical question and more like discussion about your coding style and practice you use daily.
I will take as example: CodeGear RAD Studio 2009 (sorry to all D7 fans, but Unicode rules).
I have capability to expand/collapse functions/procedures/records and few other complex data structures, but what if code is lengthy?
What makes the task and its accomplishment efficient - the time required to add comments (its req actually) and expand/collapse necessary area or use OMT offered possibilities?
To give example input from myself - I have small app, about 1,5k lines and I do not use Modeling. Is it smart enough or do I lose a lot of time if I need to find some simple references or (event) calls?
If I understand your question correctly, it is a bout finding your way into code (yours or someone elses').
I use Model Maker Code Explorer for browsing through source code (and for refactoring existing code, and creating new code). At EUR 99, it is dead cheap for what it does.
It usually gives me a perfect overview of what I need, and has a nice 'search' interface as well.
If I need more complex searches, I usually use the GExperts (grep) search function: it is blazingly fast, and with good naming of your identifiers, it is usually a breeze to find stuff.
If I understand your question correctly, you want to know what is more efficient:
Use comments and expandable sections.
Use moddeling techniques.
I think it depends on personal style. Modeling can be great, but has dangers of spending too much time creating nice pictures.
We have a large app 500k+ lines. We do not use collapsable sections because we keep our file size acceptable and we have a good file organisation structure. We sometimes use modeling if complex parts are added (class diagrams and state diagrams). And we use lots of comment to explain difficult parts.
If you have Delphi 2009 you can use also the Delphi Class Explorer (in the View menu) in order to see your classes. It seems a little bit cryptic but only for the first 5 minutes. After this you will get used with it.
Also you can use CnPack a very impressive package in order to help you manage your project. Basically, in the IDE appears a new menu called 'CnPack' which has a bunch of wizards to help you find the way out in the source. Some examples:
Uses Cleaner
Procedure List (it gives you the incremental search capability for your procedures - very neat)
Bookmark Browser
etc.
I am developing my first ASP.NET MVC application using the Entity Framework. I have no previous .NET experience, although I'm fluent in other OO languages. I am having trouble understanding Linq to Entity query expressions, more specifically its syntax and how to render the results on an ASP page.
For example, I am joining three tables (entities) together and trying to iterate over the results, but I get all sorts of type errors. Every example I find on Google, SO, or MSFT hasn't been straightforward and each site takes their own approach. I looked at the book Programming Entity Framework, but this too seems to be over my head at the moment.
I am looking for clear, concise examples of executing the equivalent of complex multiple joins and iterating over the results.
Thanks
I've just started this myself, and I bought both the wrox book and the o'reilly one.
I've found the wrox one a little more inaccessible - the language used in it is a bit heavier, and the layout/organisation isn't quite so 'flowy' (is that a word? hehe).
The o'reilly one on the other hand is a cracking read, and really does lead you nicely through it. It's available on safaribooks too if you have a sub, I got it from Amazon who seemed one of the few places to have it in stock. Very much recommended.
take a look at these samples - http://blogs.msdn.com/adonet/archive/2007/03/13/101-linq-samples-updated.aspx
i picked up the concept from these examples itself
Scott Guthrie's blog has also really good explanation & samples
Sounds to me like you should get yourself a good book on the subject. Since you have no previous experience in .NET a general book that covers LINQ might be a start and then a ASP.NET-MVC specific one.
Perhaps the reason why being fluent in "other" OO languages isn't helping is that LINQ isn't an OO language, its much more functional and therefore takes a different way of reasoning to understand well.
What are the drawbacks of linq in general.
Can be hard to understand when you first start out with it
Deferred execution can separate errors from their causes (in terms of time)
Out-of-process LINQ (e.g. LINQ to SQL) will always be a somewhat leaky abstraction - you need to know what works and what doesn't, essentially
I still love LINQ massively though :)
EDIT: Having written this short list, I remembered that I've got an answer to a very similar question...
The biggest pain with LINQ is that (with database backends) you can't use it over a repository interface without it being a leaky abstraction.
LINQ is fantastic within a layer (especially the DAL etc), but since different providers support different things, you can't rely on Expression<Func<...>> or IQueryable<T> features working the same for different implementations.
As examples, between LINQ-to-SQL and Entity Framework:
EF doesn't support Single()
EF will error if you Skip/Take/First without an explicit OrderBy
EF doesn't support UDFs
etc. The LINQ provider for ADO.NET Data Services supports different combinations. This makes mocking and other abstractions unsafe.
But: for in-memory (LINQ-to-Objects), or in a single layer/implementation... fantastic.
Some more thoughts here: Pragmatic LINQ.
Like any abstraction in programming, it is vulnerable to a misunderstanding: "If I just understand this abstraction, I don't need to understand what's happening under the covers."
The truth is, if you do understand what's happening under the covers, you'll get much better value out of the abstraction, because you'll understand where it ceases to be applicable, so you'll be able to apply it with greater confidence of success where it is appropriate.
This is true of all abstractions, and applies to Linq in bucketfuls. To understand Linq to Objects, the best thing to do is to learn how to write Select, Where, Aggregate, etc. in C# with yield return. And then figure out how yield return replaces a lot of hand-written code by writing it all with classes. Then you'll be able to use it with an appreciation of the effort it is saving you, and it will no longer seem like magic, so you'll understand the limitations.
Same for the variants of Linq where the predicates are captured as expressions and transported off to another environment to be executed. You have to understand how it works in order to safely use it.
So the number 1 drawback of Linq is: the simple examples look deceptively short and simple. The problem is, how did the author of the sample know what to write? Because they knew how to write it all out in long form, and they knew how pieces of Linq could be used as abreviations, and so they arrived at the nice short version.
As I say, not really specific to Linq, but highly relevant to it anyway.
Anonymous types. Proper ORM should always return objects of 'your' type (partial class, with possiblity of adding my methods, overriding etc.). There are doezne of tutorials and examples of different complex queries using linq but non of them care to explain the advantage of returning a 'bag of properties' (return new { .........} ). How am I supposed to work with anonymous type, wrap it in another class again?
Actually I can´t think of any drawbacks. It makes programming life a lot simpler because a lot of things can be written in a more compact but still better readable way.
But having said this, I must also agree with Jon that you should have some idea what you´re doing (but that holds for all technological advances).
the only drawback which it has is its performance see this article
I was thinking about making something like Linq for Lua, and I have a general idea how Linq works, but was wondering if there was a good article or if someone could explain how C# makes Linq possible
Note: I mean behind the scenes, like how it generates code bindings and all that, not end user syntax.
It's hard to answer the question because LINQ is so many different things. For instance, sticking to C#, the following things are involved:
Query expressions are "pre-processed" into "C# without query expressions" which is then compiled normally. The query expression part of the spec is really short - it's basically a mechanical translation which doesn't assume anything about the real meaning of the query, beyond "order by is translated into OrderBy/ThenBy/etc".
Delegates are used to represent arbitrary actions with a particular signature, as executable code.
Expression trees are used to represent the same thing, but as data (which can be examined and translated into a different form, e.g. SQL)
Lambda expressions are used to convert source code into either delegates or expression trees.
Extension methods are used by most LINQ providers to chain together static method calls. This allows a simple interface (e.g. IEnumerable<T>) to effectively gain a lot more power.
Anonymous types are used for projections - where you have some disparate collection of data, and you want bits of each of the aspects of that data, an anonymous type allows you to gather them together.
Implicitly typed local variables (var) are used primarily when working with anonymous types, to maintain a statically typed language where you may not be able to "speak" the name of the type explicitly.
Iterator blocks are usually used to implement in-process querying, e.g. for LINQ to Objects.
Type inference is used to make the whole thing a lot smoother - there are a lot of generic methods in LINQ, and without type inference it would be really painful.
Code generation is used to turn a model (e.g. DBML) into code
Partial types are used to provide extensibility to generated code
Attributes are used to provide metadata to LINQ providers
Obviously a lot of these aren't only used by LINQ, but different LINQ technologies will depend on them.
If you can give more indication of what aspects you're interested in, we may be able to provide more detail.
If you're interested in effectively implementing LINQ to Objects, you might be interested in a talk I gave at DDD in Reading a couple of weeks ago - basically implementing as much of LINQ to Objects as possible in an hour. We were far from complete by the end of it, but it should give a pretty good idea of the kind of thing you need to do (and buffering/streaming, iterator blocks, query expression translation etc). The videos aren't up yet (and I haven't put the code up for download yet) but if you're interested, drop me a mail at skeet#pobox.com and I'll let you know when they're up. (I'll probably blog about it too.)
Mono (partially?) implements LINQ, and is opensource. Maybe you could look into their implementation?
Read this article:
Learn how to create custom LINQ providers
Perhaps my LINQ for R6RS Scheme will provide some insights.
It is 100% semantically, and almost 100% syntactically the same as LINQ, with the noted exception of additional sort parameters using 'then' instead of ','.
Some rules/assumptions:
Only dealing with lists, no query providers.
Not lazy, but eager comprehension.
No static types, as Scheme does not use them.
My implementation depends on a few core procedures:
map - used for 'Select'
filter - used for 'Where'
flatten - used for 'SelectMany'
sort - a multi-key sorting procedure
groupby - for grouping constructs
The rest of the structure is all built up using a macro.
Bindings are stored in a list that is tagged with bound identifiers to ensure hygiene. The binding are extracted and rebound locally where ever an expression occurs.
I did track the progress on my blog, that may provide some insight to possible issues.
For design ideas, take a look at c omega, the research project that birthed Linq. Linq is a more pragmatic or watered down version of c omega, depending on your perspective.
Matt Warren's blog has all the answers (and a sample IQueryable provider implementation to give you a headstart):
http://blogs.msdn.com/mattwar/