This question pertains to writing a massively scalable & efficient customer sessionstate class library in C# and I am in desperate need of sage counsel.
Original thread that has been guiding me based upon:
I just discovered why all ASP.Net websites are slow, and I am trying to work out what to do about it
Hope this thread can be called back from the netherworld as I have been spending the past 48 hours (with a few power naps) trying to implement the ideas elucidated here and I must admit I am somewhat lost in the solution space and would be grateful for some clarity.
Allow me to explain where I am at & what is confusing me & what I am trying to accomplish. The back & forth here with James & Joel raises great points but concrete implementations beyond the links leave me somewhat in the dark.
I'm trying to create a class library/dll I can use in several of my projects to bypass the default sessionstate options of inproc, sql & stateserver. This is somewhat new to me so please excuse my ignorance. Further I am somewhat of a down & dirty asp.net / C# coder and some of the more subtle nuances of generics, thread safety, locking, and serialization do tend to hurt my little head :))
I had started by creating a custom sesion state provider based upon a sample I found on MSDN that writes to access. Then I had found this page and Pandora's box was wide open and now I cannot get the genie back in the bottle. I dug more and I found a template for a custom sessionstate module and I added that too to my class library. SO nowwwww... I have both a provider and module of my own and then after reading Joel's many tips I ended up creating a custom session state items collection based upon yet another sample I found on MSDN.
That was originally based upon a hashlist, somehow I ended up turning it into a sortedlist, and then I seem to have followed Joel's steps one by one as pertains to the following:
lock(typeOf) rewritten in all .cs files to use lock(thisLock) where thisLock is a private static object as per the Rico article mentioned here ... I do understand the "concept" of deadlocks but this is my first attempt to scale an asp.net site and I am not feeling like Joe Architect quite yet. I cannot afford to be lazy here because I do not wish to build upon a shaky foundation only to get a bunch of hard to figure out bugs further on up the road.
Then I followed Joel's advice about the "slim" version of the locks and got that working in my classlib as well. And yet still my spider-sense is tingling telling me I basically still am in obedience mode rather than enlightenment mode, i.e. I don't have the NEO style vision of 1s & 0s dancing in my perceptual field.
So I keep coming back to this page like a refugee from a Dan Brown novel and now I am looking at the final admonition by Joel advising that until the sessionstateitem collection is made threadsafe no good can be assured in the multiverse. So I figure it is time to start brushing up on threadsafe collections.
That's when I start reading an article by MSFT on http://msdn.microsoft.com/en-us/library/dd997305.aspx SortedLists & other oldies but apparently not so goodies being deprecated in favor of generics. Now I have actually created entire business layers and linq layers that are serializable and are used for shopping carts and stored in an offline database.
But the truth is sessionstate, threadsafety, using BOTH module & provider, and the lack of any samples here to debug has my cerebral drive continuously redlining and the heat sink is faltering.
MSFT says in above link, "The collection classes introduced in the .NET Framework 2.0 are found in the System.Collections.Generic namespace. These include List, Dictionary, and so on. These classes provide improved type safety and performance compared to the .NET Framework 1.0 classes. However, the .NET Framework 2.0 collection classes do not provide any thread synchronization; user code must provide all synchronization when items are added or removed on multiple threads concurrently.
We recommend the concurrent collections classes in the .NET Framework 4 because they provide not only the type safety of the .NET Framework 2.0 collection classes, but also more efficient and more complete thread safety than the .NET Framework 1.0 collections provide.."
So James sorta bolted thru the exit saying I got it, and that's the end of the thread but I have experienced some very peculiar behavior while debugging.
I am using a harness website to test my session lib. If I specify BOTH the httpmodule and the custom provider only the module runs in the harness/testing site I whipped up. BUT- if I modify the web.config of a real mature codebase it opts for the provider.
I did realize I could use one at a time to get the harness site pointing to the provider by not using the module.
That is fine for now but then I found myself saying to myself, "self.... you have a custom collection but is it threadsafe, are we still locking it, should we be locking it from outside the collection, i.e. in module or provider? My brain is saying is "thread-safety" encapsulated in the collection itself.
And how do I really know that all these slow page.aspx/longpage.aspx samples where one must hang out waiting for a thread to finish, even if user has click another link in frustration (what frustration? on a web site? perish de thought!)...
So basically, like Bugs Bunny, hangin' from the smithereen remnants of la bella luna I beseech thee, "GET ME OUTTA HERE..."
I thank you in advance for your timely consideration in this matter & others of monumental importance.
Best,
Cary Abramoff, MCSD.NET (Ten years of .net & I still wish it were all JQUERY baby!)
Sounds like what you really need to do next is create a testing program that calls multiple pages in your harness site simultaneously. This will simulate the situation that locking is meant to benefit and it will also help you determine if your implementation is logically correct. Without that I think you're sort of working at this blindly.
Hope that helps.
On a side note: I am currently working on a custom session state provider backed by Redis. My plan is to have an implementation that allows the locking to be turned off. I will likely get open sourced on Github once it is stable.
Related
Has anyone created an empty solution stack for .NET (C#) that incorporates an IoC framework for DI using multiple projects?
I've struggled for months to create a good reusable stack that has:
MVC UI web app
Empty BLL project (will add real entities later)
Empty DAL project (will add real daab classes later)
Reference/Search data tier
Includes an IoC framework
Sample usage of DI in a Home controller that can reach all the way to the DAL thru the entity layer or to ref/search tier all thru interfaces
Must NOT set a hard reference of any concrete classes at the UI layer
I've attempted this a few times but I always get hung up at #6 and I'm missing something basic in the structure of the stack. Has anyone managed to do this and have a sample solution to show how it's structured? I can create stacks all day long and add a IoC framework, but completely fail at getting it structured so that no concrete references are added to the UI layer. How else can the interface/concrete resolution of objects take place?
Surely some of you scholars have nipped this in the bud, please share some of this enlightenment with me :-)
ps - I've read Mark Seeman's book more than once.. I understand the concept of Composition Root... but have never seen one in use in an NTier solution and have been unable to implement the theory successfully
What I am looking for is a fleshed out solution stack of multiple projects that can be used as a base to start from. One that implements the composition root successfully and can be used to teach the SOLID principles by doing instead of telling. A solution that brings it all to life. See this question for reference.
My Shuttle Wiki FOSS has some elements of what you are after:
http://shuttlewiki.codeplex.com/
Although not every concern is in its own proejct/assembly I have found that it is not worth the effort to tease them apart unless you are really going to use the relevant assembly somewhere else. That being said, it is still very easy to split them out as care has been taken to keep the concerns decoupled.
Scanned through some of the comments. My opinion is that no project structure or technique should be used to try and prevent or protect other developers from using certain classes. Junior developers have to learn the ropes at some stage and some quick code walkthroughs would pick up coding not consistent with what you are trying to achieve.
Take a look at mine:
using repository pattern, ninject, entity framework 6, ...
https://github.com/mesuttalebi/NTierCSharpExample
I’m developing a part of an application, named A. The application I want to plug my DLL into, called application B is in vb 6, and my code is in vb.net. (Application B will in time be converted to vb.net) My main question i, how is the best way for me to organize my code (application A)?
I want to split application A into layers (Service, Business, Data access), so it will be easy to integrate application A into B when B is converted to vb.net. I also want to learn about all the topics like layered architecture, patterns, inversion of dependency, entity framework and so on. Although my application (A) is small I want to organize my code in the best way.
The application I’m working with (A) is using web services for authenticating users and for sending schema to an organization. The user of application B is selecting a menu point in application B and then some functions in my application A is called.
In application A I have an auto generated schema class from an xsd schema. I fill this schema object with data and serialize the object to a memory string (is it a good solution to use memory string, I don’t have to save the data yet), wrap the xml inside a CDATA block and return the CDATA block as a string and assign the CDATA block to a string property of a web service.
I am also using Entity framework for database communication (to learn how this is done for the future work with application B). I have two entities in my .edmx, User and Payer.
I also want to use the repository pattern (is this a good choice?) to make a façade between the DAL and the BLL.
My application has functions for GeneratingSchema (filling the schema object with data), GetSchemaContent, GetSchemaInformation, GenerateCDATABlock, WriteToTextFile, MemoryStreamToString, EncryptData and some functions that uses web services, like SendShema, AuthenticateUser, GetAvalibelServises and so on.
I’m not sure where I should put it all?
I think I have to have some Interfaces like IRepository, ISchema (contract for the auto generated schema class, how can I do this?) ICryptoManager, IFileManager and so on, and classes that implements the interfaces.
My DAL will be the Entity framework. And I want a repository façade in my BLL (IRepository, UserRepository, PayerRepository) and classes for management (like the classes I have mention above) holding functions like WriteToFile, EncryptData …..
Is this a good solution (do I need a service layer, all my GUI is in application B) and how can I organize my layers, interfaces, classes an functions in Visual Studio?
Thanks in advance.
This is one heck of a question, thought I might try to chip away at a few parts for you so there's less for the next guy to answer...
For application B (VB6) to call application/assemblies A, I'm going to assume you're exposing the relevant parts of App A as COM Components, using ComVisibleAttributes and similar, much like described in this artcle. I only know of one other way (WCF over COM) but I've never tried it myself.
Splitting your solution(s) into various tiers and layers is a very subjective/debatable topic, and will always come down to a combination of personal preference, business requirements, time available, etc. However, regardless of the depth of your tiers and layers, it is good to understand the how and the why.
To get you started, here's a couple articles:
Wikipedia's general overview on "Multitier Architectures"
MSDN's very own "Building an N-Tier Application in .Net"
Inversion of Control is also a very good pattern to get into right now, with ever increasing (and brilliant!) resources becoming available to the .Net platform, it's definitely worth infesting some time to learn.
Although I haven't explored the full extent of IoC, I do love dependency injection(a type of IoC if I understand correctly though people seem to muddle the IoC/DI terms quite a lot). My personal preference for DI right now is the open source Ninject project, which has plenty of resources online and a reasonable wiki section talking you through the various aspects.
There are many more takes on DI and IoC, so I don't want to even attempt to provide you a comprehensive list for fear of being flamed for missing out somebody's favourite. Just have a search, see which you like the look of and have a play with it. Make sure to try a couple if you have the time.
Again, the Repository Pattern - often complemented well by the Unit of Work Pattern are also great topics to mull over for hours. I've seen a lot of good examples out on the inter-webs, and as many bad examples. My only advice here is to try it for yourself... see what works for you, develop a version of the patterns that suits you best and try to keep things consistent for maintainability.
For organising all these tiers and layers in VS, I recommend trying to keep all your independent tiers/layers in their own Solution Folders (r-click the Solution, Add New Solution Folder), or in some cases (larger projects) there own solutions and preferably an automated build service to update dependent projects with up to date assemblies as required. Again, a broad subject and totally down to personal preference. Just keep an eye out when designing your application for potential upcoming Circular References.
So, I'm afraid that doesn't even slightly answer your question, but hopefully provides you with some resources to check out and a few hours of reading.
Good luck!
All,
I'm currently revamping an ancient IVR written using Classic ASP with VXML 2.0. Believe me, it was a mess, largely due to the mixing of routing logic between the ASP code and the VXML logic, featuring multiple postbacks a la ASP.NET. Not fun to debug.
So we're starting fresh with MVC 3 and Razor and so far so good. I've succeeded in moving pretty much all the processing logic to the controller and just letting most of the VXML be just voicing a prompt and waiting for a DTMF reply.
But, looking at a lot of sample VXML code, it's beginning to look like it might actually be simpler to do basic routing using multiple on a page and VXML's built-in DTMF processing and . More complex decision-making and database/server access would call the controller as it does now.
I'm torn between the desire to be strict about where the logic is, versus what might actually be simpler code. My VXML chops are not terribly advanced (I know enough to be dangerous), so I'm soliciting input. Have others used multiple forms on a page? Better or worse?
Thanks
Jim Stanley
Blackboard Connect Inc.
Choosing to use simple VoiceXML and moving the logic server side is a fairly common practice. Pros/Cons below.
Server-side logic
Often difficult to get retry counters to perform the way you want if you are also performing input validation (valid for grammar, but not for host or other validation logic)
Better programming language/toolkits for making logical descriptions (I'm not a fan of JavaScript, but even if you like JavaScript, you tend to have to create a lot of forms to get the flow control you want).
Usually easier to debug. Step through logical decisions and access to logging tools.
Usually easier to create reusable components that use parameters to alter component behavior.
Client side logic
Usually more scalable. VoiceXML browsers tend to use a large amount of their resources compiling and processing pages. One larger page will typically do better than a variety of smaller pages. However, platforms vary significantly and your size may make this negligible.
Better chance of using static pages. Many platforms have highly optimized caches (more than just fetched data). Like above may only matter if you have 100s of ports per device or 1000s of ports hitting a server.
Mixing and matching isn't bad until somebody requests some sort of global behavior change. You may be making the change in multiple places. Debugging techniques will also vary so it may complicate your support paths (e.g. looking in browser logs versus server logs to see what happened on a call).
Our current framework currently uses a mix of server and client. All our logic is in the VoiceXML, and the server is used for state saving and generating recognition components. Unfortunately as all our logic is in the voicexml, it makes it harder to unit test.
Rather than creating a large voicexml page that subdialogs to each question and all the routing done on the clientside, postback to the server after each collection, then work out where to go now. Obviously this has it's pros/cons as Jim pointed out, but the hope is to abstract some of the IVR/callflow from the VoiceXML and reduce the dependency on skilling up developers in VoiceXML.
I'm looking at redeveloping using MVC3, creating different views based on base IVR functions, which can then be modified based on the hosting VoiceXML platform:
Recognition
Prompts
Transfer
CTI Get/Set
Disconnect
What I'm still working out is how to create reusable components within the MVC. Whether to create something we subdialog to and return back the result (similar to how we currently do it), or redirect to a generic controller, and then redirect to the "Completed" action once the controller is done.
Jim Rush provides a pretty good overview of the pros and cons of server side versus client side logic and is pretty consistent with my discussion on this topic in my blog post "Client-side versus Server-side Development of VoiceXML Applications". I believe the pros of putting the logic on the server far outweigh putting it on the client. The VoiceXML User Group is moving towards removing most of this logic from VoiceXML in version 3.0 and suggesting using a new standard called State Chart XML (SCXML) to handle control of the voice application. I have started an open source project to make it easier to develop VoiceXML applications using ASP.NET MVC 3.0 which can be found on CodePlex and is called VoiceModel. There is an example application in this project which will demonstrate a method for keeping the logic server side, which I believe greatly improves reuse of voice objects.
I have recently designed a web application that I would like to write in Ruby. Coming from a ASP background I designed it with method and fields and linked them together (in my diagram and UML) like I would do it in C#.
However, now that I've moved from a single app to MVC I have no idea where my code goes or how the pieces are linked.
For example, my application basically collects information from various sources for users, and when they log in the information is presented to them with "new" information (information collected since last login) is tagged specially in the interface.
In C# I would have a main loop that waits let's say 5 minutes and does the collection, then when a client tries to connect it would spawn a new thread that generates the page with the new information. Now that I'm moving to Ruby I'm not sure how to achieve the same result.
I understand that the controller connects the model to the view and I thus assume this is where my code goes yet I've haven't seen a tutorial that talks about doing what I've mentioned. If someone could point me to one or tell me precisely what I need to do to turn my pseudocode into production code I'd be extremely grateful and probably will still have hair: D
EDIT: Somehow I forgot to mention that I'll be using the Rails framework. I don't really like Ruby but RoR is so nice together that I think I can put up with it.
The part of your application that is retrieving the data at certain interval shouldn't be, strictly speaking, part of the web application. In Unix world (including Rails), it would be implemented either as a daemon process, or a cron job. On Windows, I presume that Windows service is the right tool.
Regarding C# -> Ruby transition, if that's purely for Rails, I'd listen to the George's advice and give ASP.NET MVC a shot, as it resembles Rails logic pretty closely (some would call it a ripoff, I guess ;)). However, learning a new language, especially so different than C# as Ruby is, is always a good idea and a way to improve yourself as a developer.
I realize you want to move to Ruby; but you may want to give ASP.NET MVC a shot. It's the MVC framework on the ASP.NET platform.
Coming from ASP, you're going to have to do a lot of conversion to change your code to become more modular. Much more than any one post on Stack Overflow will do justice.
MVC is made up into 'tiers':
Model - Your Data
View - What the user Sees
Controller - Handles requests and communicates with the View and Model.
Pick up a book on ASP.NET MVC 1.0, and do some research on the MVC pattern. It's worth it.
Whatever Ruby web framework you plan to use (Rails, merb, Sinatra), it sounds like the portion that collects this data would typically be handled by a background task. Your models would be representations of this data, and the rest of your web app would be pretty standard.
There are some good Railscast episodes on performing tasks in the background:
Rake in Background
Starling and Workling
Custom Daemon
Delayed Job
There are other options for performing tasks in the background (such as using a message queue and the ActiveMessaging plugin) but these screen casts will at least give you a feel for how background jobs are generally approached in Rails.
If you perform these tasks on a regular schedule, there are tools for that as well.
I hope this is of some help.
Check out Rails for .NET Developers. I've heard good things about this book and it sounds like it's just what you're looking for.
I've been reading through a couple of questions on here and various articles on MVC and can see how it can even be applied to GUI event intensive applications like a paint app.
Can anyone cite a situation where MVC might be a bad thing and its use ill-advised?
EDIT: I'm specifically talking about GUI applications here!
I tried MVC in my network kernel driver. The patch was rejected.
I think you're looking at it kind of backwards. The point is not to see where you can apply a pattern like MVC, the point is to learn the patterns and recognize when the problem you are trying to solve can naturally be solved by applying the pattern. So if your problem space can be naturally divided into model, view and controller then it is a good candidate for MVC. If you can't easily see which parts of your design fall into the three categories, it may not be the appropriate pattern.
MVC makes sense for web applications.
In web applications, you process some data (on SA: writing questions, adding comments, changing user info), you have state (logged in user), you don't have many different pages, but a lot of different content to fit into those pages. One Question page vs. a million questions.
For making CMS, for example, MVC is useless. You don't have any models, no controllers, just a pages of text with decorations and menus. The problem is no longer processing data - the problem now is serving that text content properly.
Tho, CMS Admin would build on top of MVC just fine, it's just user part that wouldn't.
For web services, you'd better use REST which, I believe, is a distinct paradigm.
WebDAV application wouldn't benefit greatly from MVC, either.
The caveat on Ruby for Web programming is that Rails is better suited for building Web applications. I’ve seen many projects attempt to create a WebDAV server or a content management system CMS with Rails and fail miserably. While you can do a CMS in Rails, there are much more efficient technologies for the task, such as Drupal and Django. In fact, I’d say if you’re looking at a Java Portal development effort, you should evaluate Drupal and Django for the task instead.
Anything where you want to drop in 3rd party components will make it tough to work in the MVC pattern. A good example of this is a CMS.
Each component you get will have their "own" controller objects and you won't be able to share "control" of model -> ui passing.
I don't necessarily know that MVC is ever really a bad idea for a GUI app. But there are alternatives that are arguably better (and also arguably worse depending on whose opinion you're asking). The most common is MVP. See here for an explanation: Everything You Wanted To Know About MVC and MVP But Were Afraid To Ask.
Although I suppose it might be a bad idea to use MVC if you're using a framework or otherwise interacting with software that wasn't designed with MVC in mind.
In other words, it's a lot like comparing programming languages. There's usually not many tasks that one can say that one is better than the other for. It usually boils down to programmer preference, availability of libraries, and the team's experience.
MVC shouldn't be used in applications where performance is critical. I don't know if this still applys with the increase of computing power but one example is a call center application. If you can save .5 seconds per call entering and updating information those savings add up over time. To get the last bit of performance out of your app you should use a desktop app instead of a web app and have it talk directly to the database.
When is it a bad thing? Where ever there is another code-structure that would better fit your project.
There's countless projects where MVC wouldn't "fit", but I don't see how a list of them would be of any benefit..
If MVC fits, use it, if not, use something else..
MVC and ORM are a joke....they are only appropriate when your app is not a database app, or when you want to keep the app database agnostic. If you're using an RDBMS that supports stored procedures, then that's the only way to go. Stored procs are the preferred approach for experienced application developers. MVC and ORM are only promoted by companies trying to sell products or services related to those technologies (e.g. Microsoft trying to sell VS). Stop wasting your time learning Java and C#, focus instead on what really matters, Javascript and SQL.