Tuesday 4 September 2007

AutoEventWireup is evil!

Summary



AutoEvenWireup is evil, especially if you need performance and scalability.
Setting it to true will add hundreds of function calls at runtime to the code which is running you Asp.Net page (probably thousands if there have many controls), function calls that could be safely avoided manually plugging the event handlers you need to deal with.


Investigating AutoEventWireup



A few years ago, I read in a Microsoft document that the AutoEventWireup attribute of the Asp.Net @ Page directive, although handy and probably even cherished by some, especially those from a VB background, is in truth only suitable for prototyping and only whenever performance and scalability are not an issue. Well, if you wonder when performance and scalability are not an issue, you should probably read on!



Let's imagine to have an ASP.Net project called Riolo.Investigation.Web, and to put inside a folder called AutoEventWireup an ASP.Net page called True.aspx, with the corresponding True.aspx.cs code behind file.



This is the ASP.NET page source code:


<%@ Page
Language = "C#"
AutoEventWireup = "true"
Inherits = "Riolo.Investigation.Web.AutoEventWireup.True"
ValidateRequest = "false"
EnableSessionState = "false"
CodeFile = "True.aspx.cs"
%>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>True</title>

<meta http-equiv="content-type" content="text/html; charset=utf-8" />
<meta http-equiv="CACHE-CONTROL" content="NO-CACHE" />
<meta http-equiv="PRAGMA" content="NO-CACHE" />

<link href="Riolo.Investigation.Web.AutoEventWireup.css" type="text/css" rel="stylesheet" />
</head>
<body>
<% = _message %>
</body>
</html>



This is the c# code behind file source code:


using System.Web.UI;

namespace Riolo.Investigation.Web.AutoEventWireup
{
public partial class True : Page
{
protected string _message;

private void Page_Load(object sender, System.EventArgs e)
{
if(!IsPostBack)
{
_message = string.Format
("SupportAutoEvents is {0}", base.SupportAutoEvents);
}
}
}
}



After compiling the ASP.Net project, after I get the True.aspx page from an ASP.Net enabled web server, I can see the following text on my browser:


SupportAutoEvents is True



When I take a look at the underlying HTML, I can see the following resulting source code:


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>True</title>

<meta http-equiv="content-type" content="text/html; charset=utf-8" />
<meta http-equiv="CACHE-CONTROL" content="NO-CACHE" />
<meta http-equiv="PRAGMA" content="NO-CACHE" />

<link href="Riolo.Investigation.Web.AutoEventWireup.css" type="text/css" rel="stylesheet" />
</head>
<body>
SupportAutoEvents is True
</body>
</html>


What is happening?



What is happening is that setting the AutoEventWireup to true is telling the ASP.Net runtime engine to generate some delegates that enables us to use the methods with the following signatures to intercept the corresponding events of the ASP.Net page lifecycle:


  • Page_PreInit

  • Page_PreLoad

  • Page_LoadComplete

  • Page_PreRenderComplete

  • Page_InitComplete

  • Page_SaveStateComplete

  • Page_Init

  • Page_Load

  • Page_DataBind

  • Page_PreRender

  • Page_Unload

  • Page_Error

  • Page_AbortTransaction

  • OnTransactionAbort

  • Page_CommitTransaction

  • OnTransactionCommit



In the example before I used the Page_Load method to get the value of the SupportAutoEvent property and to put that in a nicely formatted string, which will be then printed inside the body of the resulting HTML.
Relying on the ASP.Net runtime to generate these delegates for us has a price, anyway, and this price is going to get quite expensive if our ASP.NET pages will become popular.


The price to pay



First of all, let's precise that this is an ASP.Net implementation issue, that is you get it on the platform targeted by my investigation (.Net 2.0), but some smarter future implementation could easily make it cheaper, pretty cheaper indeed. For the time being, anyway, the price is there to pay.
How high is this price, anyway? The key is an internal method of the abstract class System.Web.UI.TemplateControl called HookUpAutomaticHandlers().



There are various ways that are leading our pages and controls to HookUpAutomaticHandlers, the following is probably one of the more common:


System.Web.UI.TemplateControl.GetDelegateInformationWithNoAssert(IDictionary) : Void
System.Web.UI.TemplateControl.GetDelegateInformation(IDictionary) : Void
System.Web.UI.TemplateControl.HookUpAutomaticHandlers() : Void
System.Web.UI.Page.SetIntrinsics(HttpContext, Boolean) : Void
System.Web.UI.Page.SetIntrinsics(HttpContext) : Void
System.Web.UI.Page.ProcessRequestWithNoAssert(HttpContext) : Void
System.Web.UI.Page.ProcessRequest(HttpContext) : Void
ASP.autoeventwireup_true_aspx.ProcessRequest(HttpContext) : Void



If we take a look to HookUpAutomaticHandlers (thanks to Lutz Roeder's Reflector), we will notice that its code is executed only if the SupportAutoEvents is set to true:


internal void HookUpAutomaticHandlers()
{
if (this.SupportAutoEvents)
{
object obj2 = _eventListCache[base.GetType()];
IDictionary dictionary = null;
if (obj2 == null)
{
lock (_lockObject)
{
obj2 = _eventListCache[base.GetType()];
if (obj2 == null)
{
dictionary = new ListDictionary();
this.GetDelegateInformation(dictionary);
if (dictionary.Count == 0)
{
obj2 = _emptyEventSingleton;
}
else
{
obj2 = dictionary;
}
_eventListCache[base.GetType()] = obj2;
}
}
}
if (obj2 != _emptyEventSingleton)
{
dictionary = (IDictionary) obj2;
foreach (string text in dictionary.Keys)
{
EventMethodInfo info = (EventMethodInfo) dictionary[text];
bool flag = false;
MethodInfo methodInfo = info.MethodInfo;
Delegate delegate2 = base.Events[_eventObjects[text]];
if (delegate2 != null)
{
foreach (Delegate delegate3 in delegate2.GetInvocationList())
{
if (delegate3.Method.Equals(methodInfo))
{
flag = true;
break;
}
}
}
if (!flag)
{
IntPtr functionPointer = methodInfo.MethodHandle.GetFunctionPointer();
EventHandler handler = new CalliEventHandlerDelegateProxy(this, functionPointer, info.IsArgless).Handler;
base.Events.AddHandler(_eventObjects[text], handler);
}
}
}
}
}



That's it, setting AutoEventWireup to true will set SupportAutoEvents to true, and once HookUpAutomaticHandlers will be called, and it will be called for any page and all the controls in the page, will cause all of that code in red, using foreach, reflection, locks and especially calling GetDelegateInformation(IDictionary), which in turn will call GetDelegateInformationWithNoAssert(IDictionary dictionary). Using Reflector, let's take a look at that:


private void GetDelegateInformationWithNoAssert(IDictionary dictionary)
{
if (this is Page)
{
this.GetDelegateInformationFromMethod("Page_PreInit", dictionary);
this.GetDelegateInformationFromMethod("Page_PreLoad", dictionary);
this.GetDelegateInformationFromMethod("Page_LoadComplete", dictionary);
this.GetDelegateInformationFromMethod("Page_PreRenderComplete", dictionary);
this.GetDelegateInformationFromMethod("Page_InitComplete", dictionary);
this.GetDelegateInformationFromMethod("Page_SaveStateComplete", dictionary);
}
this.GetDelegateInformationFromMethod("Page_Init", dictionary);
this.GetDelegateInformationFromMethod("Page_Load", dictionary);
this.GetDelegateInformationFromMethod("Page_DataBind", dictionary);
this.GetDelegateInformationFromMethod("Page_PreRender", dictionary);
this.GetDelegateInformationFromMethod("Page_Unload", dictionary);
this.GetDelegateInformationFromMethod("Page_Error", dictionary);
if (!this.GetDelegateInformationFromMethod("Page_AbortTransaction", dictionary))
{
this.GetDelegateInformationFromMethod("OnTransactionAbort", dictionary);
}
if (!this.GetDelegateInformationFromMethod("Page_CommitTransaction", dictionary))
{
this.GetDelegateInformationFromMethod("OnTransactionCommit", dictionary);
}
}



As you can see, GetDelegateInformationFromMethod(..) is called a minimum 14 times from each page and a minimum of 8 time from each control in the page! GetDelegateInformationFromMethod(..) will then call System.Delegate.CreateDelegate(..) and the .Net trail will end when that will call the extern method BindToMethodInfo(..).
It is quite impressive that all of this is basically for nothing! If you think about it a few seconds, you will realize that you may save your web pages hundreds if not probably thousands of function calls just setting AutoEventWireup to false. We may then follow others strategies to handle events happening during the lifecycle of the Asp.Net pages and controls, but before introducing those, let's take a look at what happens when we set AutoEventWireup to false.


Setting AutoEventWireup to false



I created a new Asp.Net page in the AutoEventWireup folder, and I called that False.aspx, and then I created a new code behind c# file, and I called that False.aspx.cs, and then I copied inside them the code from True.aspx and from True.aspx.cs respectively. Togheter with AutoEventWireup I had to change also the values of the CodeFile and Inherits attributes of the @ Page directive.
If we compile the web site, and we ask False.aspx from an Asp.Net enabled web server, we will quickly notice that the code inside the Page_Load(..) method is not running anymore.
This is happening because at compile time the Asp.Net compiler is overriding the SupportAutoEvents property, which will now returns false, so the code inside HookUpAutomaticHandlers() is not running anymore, so no precooked delegates will be created for us at runtime.
You may confirm that going to the Temporary ASP.NET Files folder (C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\ in my machine) and drilling down to the folder containing false.aspx.[x].compiled ([x] was ef11d7f0 in my case) file. If you open that file with a text editor, you will notice it is an xml file, and if you check the assembly attribute of the preserve element, you will read the name of the assembly where the Asp.Net page has been compiled in (in my case, it was App_Web_rzqguydd.dll).
You may choose to investigate that assembly with Reflector, but you may notice that there are a set of *.cs file starting with the same of the assembly. If you open these, if you will find one of them is containing a public class named autoeventwireup_false_aspx, which is where SupportAutoEvents get overridden and set to false.


Alternative Strategies



The alternative strategy is basically that of manually create the delegate and plug the event handler, as in the following version of False.aspx.cs:


using System.Web.UI;

namespace Riolo.Investigation.Web.AutoEventWireup
{
public partial class False : Page
{
protected string _message;

protected override void OnInit(System.EventArgs e)
{
base.OnInit(e);
this.Load += new System.EventHandler(Page_Load);
}

private void Page_Load(object sender, System.EventArgs e)
{
if(!IsPostBack)
{
_message = string.Format
("SupportAutoEvents is {0}", base.SupportAutoEvents);
}
}
}
}



Instead of plugging the event handler inside the OnInit(..) method, you may plug it inside a constructor, but another alternative is that of getting rid of Page_Load(..) and using OnInit(..) directly.

Thursday 30 August 2007

How to pass parameters set at runtime to the log4net configuration

Today I learnt that it is possible to pass parameters set at runtime to the log4net configuration sections.
We could in example put the following appender in our log4net configuration section:

<appender name="SystemAppender" type="log4net.Appender.RollingFileAppender">
<file type="log4net.Util.PatternString" value="C:\Logs\%property{LogFileName}.log" />
<appendToFile value="true" />
<maximumFileSize value="1000KB"/>
<maxSizeRollBackups value="2"/>
<staticLogFileName value="false" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date %level %thread %logger - %message%newline"/>
</layout>
</appender>

Did you noticed the attributes of the file element? Togheter with the usual value, we got also a type, set to log4net.Util.PatternString, and if you double check, you will notice also the value is a bit unusual: part of the string is a %property{LogFileName} substring.
Now, that substring is telling to the log4net to look for a property called LogFileName in its ThreadContext. If you set that property before the call to XmlConfigurator.Configure, as in the following snippet, you will be able to programmatically set it at runtime:

log4net.ThreadContext.Properties["LogFileName"] = "MyApplication";
log4net.Config.XmlConfigurator.Configure();

This will set your log file to C:\Logs\MyApplication.log.
This approach can be useful in many scenarios, but probably not so efficient if you wish to multiplex the log entries to different files, because everytime you set the LogFileName property, you need to call again XmlConfigurator.Configure().

Thanks to Max Leifer to show us this technic :)

Monday 13 August 2007

Installing a missing component on Microsoft SQL Server 2005 sp2

Today to set up a Team Foundation Server I noticed an instance of SQL Server 2005 was already installed on the machine. Unfortunately it was missing the Reporting Service component, which is required from TFS, so I decided to install it.
I thought it should be an easy exercise. I thought so.
First of all, I quickly discovered the service pack 2 had been installed in this machine, but quite irregurarly: that is, some components were on sp2, some others on sp2 pre-5th March, some other one on sp2 post-5th March (if you are not an insider, Bow Ward as a quite revealing explanation of the mess that was sp2).
Following an officemate's advice (cheers Alex!), I decided first to bring everythign at GDR2 (that is 9.00.3054), then I tried to add the missing components (once there, I decided to add also NS, even if I am going to stop it straight away!).
Here is where I discovered (or rediscovered? I am pretty familiar with this argument, but I can't remember to have met this issue before) that SKUUPGRADE is not the same than skuupgrade ..
That said, this is what I got to see when I finally get to choose the components to add:



Isn't that misleading? I mean, what I wish to achieve is to add some component not previously installed (by rule, if not otherwise required by security policies or other issues, I do ever install all the available components, and then disable the services I don't need). This interface clearly leading (me at last) to believe that the Database Services and the other components paired with the red x will be uninstalled, which will not happen. What this is trying to communicate is that the components I have installed on my machine are not the components on the install file, but if so, would be easier to just show that (i.e. showing Database Services as installed, but with the version number in braces).

Yep, all of this was a rant, but when people (well, me ..) have to waste 3 hours of their lives to add a component to an already installed software, there is definitively something smelling bad there ..

Monday 7 May 2007

De Santis, Inter and Milan

Massimo De Santis used to be a thriving international football referee. He had been selected to represent Italy at the 2006 FIFA World Cup, and slightly more than one year ago he was busy preparing to establish himself an apt recipient of Pierluigi Collina’s mountainous legacy when the storm of the so-called Calciopoli, the infamous Italian football scandal, shattered his dreams.
He was barred from the approaching World Cup, and just after the end of the tournament he was banned until 2011. His career as a professional referee was completely ruined.
Monday 30th April, De Santis, a police constable, was guest of a TV program for a north Italian broadcaster, Antenna Tre Lombardia. During the program, he declared that he never talked on the phone with the former Juventus director Luciano Moggi, who was present, and the he also confessed that during his not so short career as a Serie A referee he received a lot of calls from former Inter president, the late Giacinto Facchetti, and former Milan “liaisons with referees” manager Luca Meani.
He also confessed to be an Inter supporter, and that he used to enjoy a very fine relationship with the late Facchetti, marred only from some occasional requests that “some time went also beyond the legal”.
Italian media basically skipped about this story, when they reported it they were mostly commenting about De Santis’ bad taste on talking about some already dead fellow.
Mainstream Italian media (apart a few lines http://qn.quotidiano.net/2007/05/04/9333-santis_sentito_moggi_facchetti.shtml) and English media basically ignored De Santis’ comments so far.

Friday 13 April 2007

Improving build performances: assemblies on the GAC

Working with a Monorail based web project, to improve build performances, I added some of the Castle Project assemblies to the GAC.

Obviously, there is a time when you need a fast build, and there is a time when you need to be sure that all the assemblies your application is using are in the /bin folder, so I wrote 2 batch files to cache and uncache the assemblies listed in 2 text files (some example code at the end of the article).

I tested on my machine with a sizable web application project the effect of having or not having those assemblies on the GAC on the build time, and this are the rounded average results:

Solution
GAC 10"
No-GAC 27"

Web
GAC 5"
No-GAC 22"

That is, on my machine when I put the files on the GAC, when building I gain 17".

[Note: The web build is normally 5" faster than the Solution build because in the solution there are project not directly needed by the Web (In primis the tests, then some data migration and seeding stuff, some productivity measurament tool, and so on).]

Why it happens? My best answer is that having the Castle project assemblies in the GAC means that during the build these are not copied anymore in the various /bin folders.

I do advice you to try to test this in your machines as well, anyway, please read carefully and make sure you understand the next points before moving ahead:

- the build phase generate assemblies from the source code. Those assemblies are bytecode, not executable programs, and if not opportunely precompiled, they will be compiled at the startup of the .Net application. This means that after building, when you will refresh the web page, this will take exactly the same time to answer you as it was taking before putting the Castle Project assemblies on the GAC.

- I had to modify the web.config, because the assembly Castle.ActiveRecord has to be explicitly added this way.

- You may notice I also added Castle.Monorail.Framework and Castle.Monorail.Views.Brail. Unfortunately it seems IIS is requiring those two files before reading all the web.config, so I add to change the property Copy Local to true (so in the /bin folder of the web application project you still can find these two assemblies and their related files). This means that you need to slightly the web application project as well to get those latest needed changes.

- I didn't test the web with a web site project configuration, if you happen to use such, you will probably get an error due missing assemblies. I do not advice to use such project option for an unending list of reasons, but if you really wish to persist on your own way, you should solve that just making sure the appropriate assemblies are copied in the /bin folder of the web application. Manually (I mean with some pre o post build event) copying files is a bad practice, because it will worsen the performances of the build, so exercise with care.

- You may be tempted to modify the list of the assemblies to include in the GAC. Please exercise care when doing so, because you can only add strongly typed assemblies this way. In example Newtonsoft.Json is not in that list because it is not a strongly typed assembly.

- Microsoft doesn't recommed to put in the GAC the assemblies you are developing. That is, MyDomainModel.dll or MyInitialisation.dll have still to be keept outside the GAC. If in the future you stabilize one of those libraries, you may decide to sign it (to make it strongly typed) and that to add it to the GAC.

I tested all of this on this Monorail based web project, but its results will probably be more or less similar with normal ASP.NET projects.

Batch file to cache assemblies

pause
"C:\Program Files\Microsoft.NET\SDK\v2.0 64bit\Bin\gacutil" /f /il .\toCache.txt
pause


Batch file to uncache assemblies

pause
"C:\Program Files\Microsoft.NET\SDK\v2.0 64bit\Bin\gacutil" /f /ul .\toUncache.txt
pause


ToCache.txt (list of file to cache, note the relative path to the working directory of the batch file)

..\..\Resources\bin\anrControls.Markdown.NET.dll
..\..\Resources\bin\Boo.Lang.Compiler.dll
..\..\Resources\bin\Boo.Lang.dll
..\..\Resources\bin\Boo.Lang.Parser.dll
..\..\Resources\bin\Castle.ActiveRecord.dll
..\..\Resources\bin\Castle.Components.Binder.dll
..\..\Resources\bin\Castle.Components.Common.EmailSender.dll
..\..\Resources\bin\Castle.Components.Common.EmailSender.SmtpEmailSender.dll
..\..\Resources\bin\Castle.Core.dll
..\..\Resources\bin\Castle.DynamicProxy.dll
..\..\Resources\bin\Castle.Monorail.ActiveRecordSupport.dll
..\..\Resources\bin\Castle.Monorail.Framework.dll
..\..\Resources\bin\Castle.Monorail.Views.Brail.dll
..\..\Resources\bin\Castle.Services.Logging.Log4netIntegration.dll
..\..\Resources\bin\Iesi.Collections.dll
..\..\Resources\bin\log4net.dll
..\..\Resources\bin\NHibernate.dll
..\..\Resources\bin\NHibernate.Caches.SysCache.dll
..\..\Resources\bin\NHibernate.Generics.dll
..\..\Resources\bin\Nullables.dll
..\..\Resources\bin\Nullables.NHibernate.dll


Batch file to uncache assemblies

anrControls.Markdown.NET
Boo.Lang.Compiler
Boo.Lang
Boo.Lang.Parser
Castle.ActiveRecord
Castle.Components.Binder
Castle.Components.Common.EmailSender
Castle.Components.Common.EmailSender.SmtpEmailSender
Castle.Core
Castle.DynamicProxy
Castle.Monorail.ActiveRecordSupport
Castle.Monorail.Framework
Castle.Monorail.Views.Brail
Castle.Services.Logging.Log4netIntegration
Iesi.Collections
log4net
NHibernate
NHibernate.Caches.SysCache
NHibernate.Generics
Nullables
Nullables.NHibernate

Thursday 12 April 2007

Web Site Projects vs Web Application Projects

For web development, Visual Studio 2005 (VS 2005) supports two project options: Web Site Projects (WSPs) and Web Application Projects (WAPs). The earlier was built-in with the initial release of VS 2005, the latter were introduced later and are included on the VS 2005 SP1.
WSPs are leveraging the very same dynamic compilation system used by the ASP.NET 2.0 at runtime; when using this option, you don't need a project file (you just need to follow some easy and intuitive folder conventions).
WAPs are leveraging the MSBuild buld system, and when using them, all the code in a project is builded in a single assembly; when using this option, you do need a project file.
There are various pro and cons for each of those models, but if we focalize just on the speed of the build, the WAP option wins hand down every time you build from scratch or rebuild.
Why?
The main reason, as explained from Scott Guthrie, is that with the WAP, only the code-behind and the /app_code/* classes are built, while with the WSP the ASP.NET runtime spend its time analyzing and compiling each ASP.NET related code (content/controls/inline code).
Monorail case:
First and foremost, having a MSBuild capable project, as in the WAP option, helps when you have to integrate your projects with tools like CruiseControl.Net. Now, it is possible to do that also for the WSP option, but it is not as neat.
Second, using Monorail, the amount of ASP.NET pages in the web projects is probable negligible (usually just 1, quite trivial in every chances, which is left there to "fire" the build on certain circumstances), so there is not really any reason to use the deep verification features of the WSP model: there are not ASP.NET related file to build.
From experience and from investigation, I can add that on the web there are various tips and tricks to speed up the WSP, but the point is that as much as you can speed up a WSP build, the main target of this kind of exercise is to make it as fast as a WAP build.
Conclusion:
On non trivial web projects, and when you don't need deep verification of ASP.NET related files, use ever the WAP option.

p.s.: I shouldn't add this, but really, don't use the /app_code folder under almost any circumstance. Put your logic in different layers (VS class projects), and if really you need to use the /app_code folder, leave there only very stable classes.

Wednesday 4 April 2007

Transaction and Concurrency on SQL Server 2005

Ayende recently posted an article about Transaction and Concurrency, and yeah, I read his blog very often, and if you are a .Net developer you should do it as well.
He had noticed that using the ReadCommitted isolation level something was not working as he had expected. I have to confess that when I first read his post and some of the comments, I had superficially concluded that could be related to the emergence of the Phantom phenomenon.

The SQL standard (at last SQL-92 and SQL-99) is defining the Phantom phenomenon as such:

P3 (‘‘Phantom’’): SQL-transaction T1 reads the set of rows N that satisfy some <search>.
SQL-transaction T2 then executes SQL-statements that generate one or more rows that satisfy the <search> used by SQL-transaction T1. If SQL-transaction T1 then repeats the initial read with the same <search>, it obtains a different collection of rows.


Today I tried to test Ayende's code, and I finally understood the issue is far more interesting than what I was supposing (lecture 1: run the code you are trying to understand).

Under the hypothesis of Ayende (which are far from being exotic, but for a little point), the ReadCommitted isolation level doesn't work as expected on Microsoft SQL Server 2005! Kudos to Stuart Carnie, who had grokked this well before me, and pointed out a very interesting article from Tony Rogerson.

Rogerson's conclusions are that on Microsoft SQL Server 2005 the ReadCommitted isolation level (and its ReadCommittedSnapshot sibling) "does not give a point in time view of your data", so when the need arises, the safe way to go may be to use the Snapshot isolation level (which incidentally seems to have been a playtoy of Microsoft Research since 10 years).

Today I played a bit with Ayende's code, and I wrote a little console application I called TransactionsAndConcurrency.exe:
TransactionsAndConcurrency mode ilp ilc [iterations [records]]
The mode argument can be one of "aye", "aye-run" or "ale", ilp and ilc one of "ch", "rc", "rr", "ru", "se", "sn" (respectively Chaos, Read Committed, Repeatable Read, Read Uncommitted, Serializable and Snapshot) while iterations and records should be self-explanative ints.
In example you may call this little console application as such:
TransactionsAndConcurrency aye rc rc
or as such:
TransactionsAndConcurrency ale rc rc 20 500
You will notice that when called with aye, this application is probably working as Ayende's code, with aye-run with a slightly different behaviour (doesn't stop the first time the consumer fetch an "unexpected" amount of rows) instead with ale unexpectedly works (I tested it until with rc and ru until 5000 various times, not a single glitch).
What is the difference? The exotic point on Ayende's hypothesis, that is that its table doesn't have a primary key. If the same exercise is done with a table with a primary key (using a surrogate key through the T-SQL Identity column) everything goes fine.

I have to confess that I am still wondering if this is a bug of Microsoft SQL Server 2005, because those are obviously uncommitted phantom rows that are showing up because something in the range lock of the tables without primary keys is obviously not working as most of us would expect.

If you wish to play as well, here is the code:
using System;
using System.Data.SqlClient;
using System.Threading;
using System.Data;

namespace TransactionsAndConcurrency
{
public class Program
{
#region Vars and Consts
private const int DEFAULT_ITERATIONS = 20;
private const int DEFAULT_RECORDS = 500;

private static int records = DEFAULT_RECORDS;
private static int iterations = DEFAULT_ITERATIONS;
private static IsolationLevel isolationLevelProducer = IsolationLevel.Unspecified;
private static IsolationLevel isolationLevelConsumer = IsolationLevel.Unspecified;
private static string mode;

private static string connectionString = "Data Source=myDB;Initial Catalog=test;User=sa;Pwd=ICannotSay1!;";
#endregion

#region Setup
private static void Setup()
{
SqlConnection connection = new SqlConnection(connectionString);
connection.Open();
SqlTransaction sqlTransaction = connection.BeginTransaction();
for (int i = 0; i < records; i++)
{
SqlCommand sqlCommand = connection.CreateCommand();
sqlCommand.Transaction = sqlTransaction;
if (mode != "ale")
{
sqlCommand.CommandText = "IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[t]') AND type in (N'U')) DROP TABLE [dbo].[t]";
sqlCommand.ExecuteNonQuery();
string create = "CREATE TABLE [dbo].[t]("
+ " [id] [int] NOT NULL"
+ " ) ON [PRIMARY]";
sqlCommand.CommandText = create;
sqlCommand.ExecuteNonQuery();
}
else
{
sqlCommand.CommandText = "IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[s]') AND type in (N'U')) DROP TABLE [dbo].[s]";
sqlCommand.ExecuteNonQuery();
string create = "CREATE TABLE [dbo].[s]("
+ " [id] [int] IDENTITY(1,1) NOT NULL,"
+ " [field] [int] NULL,"
+ " CONSTRAINT [PK_s] PRIMARY KEY CLUSTERED"
+ " ([id] ASC) WITH (PAD_INDEX = OFF, IGNORE_DUP_KEY = OFF) ON [PRIMARY]"
+ ") ON [PRIMARY]";
sqlCommand.CommandText = create;
sqlCommand.ExecuteNonQuery();
}
sqlCommand.Dispose();
}

sqlTransaction.Commit();
Console.WriteLine("Create table");
connection.Close();
}
#endregion

#region TearDown
private static void TearDown()
{
SqlConnection connection = new SqlConnection(connectionString);
connection.Open();
SqlTransaction sqlTransaction = connection.BeginTransaction();
for (int i = 0; i < records; i++)
{
SqlCommand sqlCommand = connection.CreateCommand();
sqlCommand.Transaction = sqlTransaction;
if (mode != "ale")
{
sqlCommand.CommandText = "IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[t]') AND type in (N'U')) DROP TABLE [dbo].[t]";
sqlCommand.ExecuteNonQuery();
}
else
{
sqlCommand.CommandText = "IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[s]') AND type in (N'U')) DROP TABLE [dbo].[s]";
sqlCommand.ExecuteNonQuery();
}
sqlCommand.Dispose();
}

sqlTransaction.Commit();
Console.WriteLine("Drop table");
connection.Close();
}
#endregion

#region Producer
private static void Producer()
{

SqlConnection connection = new SqlConnection(connectionString);
int iteration = 0;
while (true)
{
connection.Open();
SqlTransaction sqlTransaction = connection.BeginTransaction(isolationLevelProducer);
for (int i = 0; i < records; i++)
{
SqlCommand sqlCommand = connection.CreateCommand();
sqlCommand.Transaction = sqlTransaction;
if (mode != "ale")
sqlCommand.CommandText = "INSERT INTO t (Id) VALUES(@p1)";
else
sqlCommand.CommandText = "INSERT INTO s (field) VALUES(@p1)";
sqlCommand.Parameters.AddWithValue("@p1", iteration);
sqlCommand.ExecuteNonQuery();
sqlCommand.Dispose();
}

sqlTransaction.Commit();
Console.WriteLine("Wrote {0} records in iteration {1}", records, iteration+1);
iteration += 1;
connection.Close();
if (iteration == iterations)
return;
}
}
#endregion

#region Consumer
private static void Consumer()
{

SqlConnection connection = new SqlConnection(connectionString);
int iteration = 0;
while (true)
{
connection.Open();
SqlTransaction sqlTransaction = connection.BeginTransaction(isolationLevelConsumer);
SqlCommand sqlCommand = connection.CreateCommand();
sqlCommand.Transaction = sqlTransaction;
if (mode != "ale")
sqlCommand.CommandText = "SELECT COUNT(*) FROM t GROUP BY id ORDER BY id ASC";
else
sqlCommand.CommandText = "SELECT COUNT(*) FROM s GROUP BY field";
SqlDataReader sqlDataReader = sqlCommand.ExecuteReader();
if (sqlDataReader.RecordsAffected != -1)
Console.WriteLine("Read: {0}", sqlDataReader.RecordsAffected);
while (sqlDataReader.Read())
{
int count = sqlDataReader.GetInt32(0);
if (mode != "ale")
Console.WriteLine("Count = {0} in {1} iteration", count, iteration+1);
if (count != records)
{
if (mode == "ale")
Console.WriteLine("Count = {0} in {1} iteration", count, iteration+1);
if (!(mode == "aye-run"))
Environment.Exit(1);
}
}

sqlDataReader.Dispose();
sqlCommand.Dispose();
sqlTransaction.Commit();
iteration += 1;
connection.Close();
if (iteration == iterations)
return;
}
}
#endregion

#region Delete
private static void Delete()
{
SqlConnection connection = new SqlConnection(connectionString);
connection.Open();
SqlTransaction sqlTransaction = connection.BeginTransaction();
for (int i = 0; i < records; i++)
{
SqlCommand sqlCommand = connection.CreateCommand();
sqlCommand.Transaction = sqlTransaction;
if (mode != "ale")
{
sqlCommand.CommandText = "DELETE FROM t";
sqlCommand.ExecuteNonQuery();
}
else
{
sqlCommand.CommandText = "DELETE FROM s";
sqlCommand.ExecuteNonQuery();
}
sqlCommand.Dispose();
}

sqlTransaction.Commit();
Console.WriteLine("Delete data from table");
connection.Close();
}
#endregion

#region Main
private static void Main(string[] args)
{
// string describing the isolation level of the producer
string ilp = string.Empty;
// string describing the isolation level of the consumer
string ilc = string.Empty;

if ((args.Length > 2) && (args.Length < 6))
{
mode = args[0];
if ((mode != "aye") &&amp;amp;amp;amp; (mode != "aye-run") && (mode != "ale"))
Environment.Exit(2);
ilp = args[1];
ilc = args[2];
}
else
Environment.Exit(3);
if (args.Length > 3)
int.TryParse(args[3], out iterations);
if (args.Length == 5)
int.TryParse(args[4], out records);

isolationLevelProducer = getIsolationLevel(ilp);
isolationLevelConsumer = getIsolationLevel(ilc);

try
{
Setup();
Delete();
Thread p = new Thread(Producer);
Thread c = new Thread(Consumer);
p.Start();
c.Start();
while ((p.IsAlive) || c.IsAlive)
{ }
}
finally
{
TearDown();
}
}
#endregion

#region Utils
private static IsolationLevel getIsolationLevel(string isolationLevel)
{
IsolationLevel il = IsolationLevel.Unspecified;

switch (isolationLevel)
{
case "ch":
il = IsolationLevel.Chaos;
break;
case "rc":
il = IsolationLevel.ReadCommitted;
break;
case "ru":
il = IsolationLevel.ReadUncommitted;
break;
case "rr":
il = IsolationLevel.RepeatableRead;
break;
case "se":
il = IsolationLevel.Serializable;
break;
case "sn":
il = IsolationLevel.Snapshot;
break;
default:
il = IsolationLevel.Unspecified;
break;
}
return il;
}
#endregion
}
}



Tuesday 20 March 2007

LLBLGen Pro to support NH, GN, AR: +1

I just read a post from Ayende about the sad state of NHibernate code generation. Between the comments, I spotted a few ones from Frans Bouma.

Exactly one year ago (well, minus a couple of days really), after a thoroughly careful evaluation, I decided to purchase LLBLGen Pro's license. I liked the tool since I first used it and in a very short time it let me to become very productive. I think I was a quite advanced user, I wrote new templates, inserted base classes and stuffs like that.

Problem was, a couple of people in the team never really grokked it, and one of them was the manager. I was sincerely not understanding why they were not able to be as productive as me, so I did some research looking for something to help them to become more proficient, and I found a very clear article written by Frans Bouma. I referred that to my team mates, and after reading that, my manager and some other team member found themselves more in camp 3 (the domain model approach) than 2 (the entity approach). At the time, after evaluating what the market was offering, we switched to ActiveRecord + NHibernate + NHibernate.Generics (incidentally, I was totally against using NHibernate without ActiveRecord due some previous hellish experiences with older versions of Hibernate). One of the alternatives we tested was Genome, which I remember to be quite promising, but which at the time was having very sloppy performances (but it was by no means the worst between the ones we evalueted).

I found that personally I feel comfortable with both the approaches (as long as people don't commit repository breaking code, I am comfortable with almost anything), but some people seems not to be so flexible, so I do sincerely believe it could be a good strategy for LLBLGen Pro to start supporting NHibernate templates (and, even better, ActiveRecord and Gentle.Net templates as well).

Monday 12 March 2007

Mutex could not be created

Working with ASP.Net, have you ever got a:

Server Error in '/xxx' Application. -------------------------------------------------------------------------------- Mutex could not be created.

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. bla bla bla ..

I have solved it in various way, but so far the best workaround I have got (thanks mostly to a post of a chap called Joao Morais) is the following:
  • - If you have visual studio 2005 is open, close it
  • - Go to the ASP.NET temporary folder for v2.0 of the framework
  • \Microsoft.Net\Framework\v2.0\Temporary ASP.NET pages
  • - Remove the folder for your application (or all of them)
  • - Reset IIS (on a command line window, >iisreset) [not always needed, but I had to use it sometimes]
  • - First Browse your page from IE (http://localhost/your app)
  • - Then reopen Visual studio
I can only add that a aspnet_regiis -ga \ before resetting IIS, is usually helping, as in 64 bit environments to try to enable/disable the Enable32bitAppOnWin64 IIS Metabase properties.

cscript %SystemDrive%\inetpub\AdminScripts\adsutil.vbs set w3svc/AppPools/Enable32bitAppOnWin64 0
%SYSTEMROOT%\Microsoft.NET\Framework64\v2.0.50727\aspnet_regiis.exe -i


cscript %SystemDrive%\inetpub\AdminScripts\adsutil.vbs set w3svc/AppPools/Enable32bitAppOnWin64 1
%SYSTEMROOT%\Microsoft.NET\Framework\v2.0.50727\aspnet_regiis.exe -i


Warning: this is a brute force approach, a bit like formatting an hard disk because the operating system got some cold, use with discrection!

Monday 5 February 2007

CruiseControl.Net and SVN over SSL

On Microsoft Windows (2003 and XP) CruiseControl.Net is usually running as a service under the Local System Account (LSA), but, to update the working copy used from the integration service using a SVN repository contacted through SSL, we have to accept permanently the server certificate.
To achieve that, we have to log as Local System, which is accomplished using the command at with the interactive switch. In example, let's suppose that the time is 16:59, running the following command we'll open a Command Prompt running as LSA:
at 17:00 /interactive cmd.exe
Once logged in as LSA, it is just a matter of launching a svn up (or another command which is permitting us to save the server certificate) from the desidered working copy folder
, and to accept permanently the server certificate.
The server certificate will be then saved on the $UserProfile\Application Data\Subversion\auth\svn.ssl.server folder (where in the case of the LSA $UserProfile is usually something as C:\Documents and Settings\LocalService\).
Warning: no, you cannot log as LSA using the abovementioned trick from a terminal server. Use the console!

Tuesday 30 January 2007

How to install Subversion 1.4.x with Apache 2.2.x and mod_ssl on Microsoft Windows

General guidelines to install Subversion 1.4.x with Apache 2.2.x and mod_ssl on Microsoft Windows. Use the following instructions at your own risk.

Subversion

The home page of the Subversion project is http://subversion.tigris.org.
The files we need to install Subversion 1.4.x with Apache 2.2.x are currently on this folder, and as for the 25th January 2007, the needed binary package is svn-win32-1.4.3.zip (this file has to be the one builded for Apache 2.2.x).
To install Subversion 1.4.x take the following steps:
  • Unzip the svn-win32-1.4.3.zip file then move its content on a directory as %ProgramFiles%\Subversion (or %ProgramFiles(x86)%\Subversion on X64 systems);
  • Add %ProgramFiles%\Subversion\bin to the environment variable path.
  • Optionally, look for the hot-backup.py script on the source code of Subversion (ideally, get it from the same SVN branch/tag from which the previous binary package was builded) and save it in the %ProgramFiles%\Subversion\tools\backup directory (create it if needed).
  • Create the repository container directory (i.e. e:\repositories);
  • Create the configuration directory (i.e. e:\repositories\etc);
  • Create the backup directory (i.e. e:\repositories\backup);
  • Create a svnserver.conf file inside the configuration directory (i.e. e:\repositories\etc\svserver.conf);
The svnserver.conf file may look like this:
<location>
DAV svn
SVNPath e:/repositories/myProject
SVNIndexXSLT /svnindex.xsl
AuthName "Subversion Authentication"
AuthType SSPI
SSPIAuth On
SSPIAuthoritative On
SSPIDomain INTERNAL
SSPIOfferBasic On
Require valid-user
AuthzSVNAccessFile "e:/repositories/myProject/authorization.conf"
</location>
Note that is just an Apache directive, that obviously you may use more than one such directive if you plan to manage more than one SVN repository, and that this is how it would look like in the case of Active Directory authentication. To get an introduction to the Authentication, read the Authentication Options section of the SVN Book.

Apache

The home page of the Apache HTTP Server project is http://httpd.apache.org, but because we wish to use mod_ssl, and because for various reasons the Apache community doesn't provide a binary package for Microsoft Windows with such a module, another place to look for those binaries is Steffen Land's Apache Lounge.
The files we need to install Apache 2.2.4 are currently on this folder, and as for the 25th January 2007, the needed binary package is httpd-2.2.4-win32-x86-ssl.zip. Given that this binary is build with Microsoft Visual C++ 2005 SP 1, you may need to install the latest Visual C++ 2005 Redistributable Package (i.e. from here).
Another quite useful resource that provides binary packages for Apache is the Unofficial Apache webserver binaries / module binaries, where we can find a graciously packaged mod_auth_sspi. To install Apache 2.2.x take the following steps:
  • If you have IIS installed on same machine, stop it now. If you wish to use Apache and IIS on same machine using port 80 on different IP addresses, you should understand what Socket Pooling is, and how to disable it (in IIS 5 and IIS 6);
  • Unzip the httpd-2.2.4-win32-x86-ssl.zip file then move its content on a directory as c:/Apache2;
  • Copy the unzipped mod_auth_sspi.so in the c:/Apache2/modules directory;
  • Copy the mod_dav_svn.so and the mod_authz_svn.so modules from the %ProgramFiles%\Subversion\bin directory in the c:/Apache2/modules directory;
  • Tweak with the c:/Apache2/conf/httpd.conf configuration file until you will not get Apache working. How to tweak that configuration file to get Apache working is out of the scope of this document, but usually it involves fixing a few file system paths, IP numbers, DNS names and port numbers, and testing these fixes invoking Apache with the -t switch (i.e. as in C:\Apache2\bin\httpd.exe -w -t -f "C:\Apache2\conf\httpd.conf" -d "C:\Apache2\.") until there are not configuration error reported. Once Apache is properly configured, backup the httpd.conf file, and then start editing that:
    • In the section entitled # Dynamic Shared Object (DSO) Support, after the line starting with #LoadModule auth_digest_module modules/mod_auth_digest.so, add the LoadModule directive to load the module to support the Active Directory:
      LoadModule sspi_auth_module modules/mod_auth_sspi.so
      To get more informations about this module, check its home page;
    • In the same section, uncomment the lines to load the modules to support WebDAV:
      LoadModule dav_module modules/mod_dav.so
      LoadModule dav_fs_module modules/mod_dav_fs.so
      To read more about those modules, read the Apache documentation about mod_dav and mod_dav_fs;
    • After the latter lines, add the following lines to load the modules to integrate Subversion:
      LoadModule dav_svn_module modules/mod_dav_svn.so
      LoadModule authz_svn_module modules/mod_authz_svn.so
      To read more about those modules, read the sections of the SVN Book about mod_dav_svn and mod_authz_svn;
    • Uncomment the module to support ssl:
      LoadModule ssl_module modules/mod_ssl.so
      To read more about this module, check its home page;
    • Uncomment the following Include directive:
      Include conf/extra/httpd-ssl.conf
  • Start editing the httpd-ssl.conf file:
    • Set the SSLCertificateFile Directive, as in:
      SSLCertificateFile c:/Apache2/conf/ssl/yourwebsitecertificate.crt
    • Set the SSLCertificateKeyFile, as in:
      SSLCertificateKeyFile c:/Apache2/conf/ssl/yourprivatekey.key
    • Add the following line, before closing the VirtualHost directive:
      Include c:/repositories/etc/svnserver.conf
    Obviously, how to generate the private key, its correspondent .csr and the web certificate its widely outside of the scope of the present document, anyway documents as The Apache + SSL on Win32 HOWTO, the Generating an SSL Certificate with Apache+mod_ssl, the mod_ssl FAQ List (especially this entry) should be quite useful, as well the CAcert documents and services.

Friday 26 January 2007

TSQL Script to get the space used by every table in a SQL Server 2005 database

Here is a TSQL script (developed, tested and used only on Microsoft SQL Server 2005) to get the space used by the tables of a database.
declare @sql nvarchar(MAX)
create table #usedSpace (
name nvarchar
(128),
rows varchar(11),
reserved varchar(18),
data varchar(18),
index_size varchar(18),
unused varchar(18))

declare @table nvarchar(MAX)
declare tabcur cursor for
select s.[name] +'.'+ t.[name] as [name]
from
sys.tables t inner join sys.schemas s on t.schema_id = s.schema_id
order by
s.[name], t.[name];

open tabcur;
fetch next from tabcur into @table;
while @@FETCH_STATUS = 0
begin
fetch
next from tabcur into @table;

select @sql = 'sp_executesql N''insert #usedSpace exec sp_spaceused ''''' + @table + ''''''''
-- print @sql -- uncomment if you wish to read the sql statements
exec (@sql)
end;
close tabcur;
deallocate tabcur;
-- select * from #usedSpace order by cast((left(Data,len(Data)-3)) as int) desc -- uncomment if you wish to get a recordset
drop table #usedSpace


Use it at your own risk, uncomment the select (or substitute that with your more appropriate log strategy) and read about sp_spaceused and above all dbcc updateusage before bitching about the quality of the results.