De Santis, Inter and Milan

Massimo De Santis used to be a thriving international football referee. He had been selected to represent Italy at the 2006 FIFA World Cup, and slightly more than one year ago he was busy preparing to establish himself an apt recipient of Pierluigi Collina’s mountainous legacy when the storm of the so-called Calciopoli, the infamous Italian football scandal, shattered his dreams.
He was barred from the approaching World Cup, and just after the end of the tournament he was banned until 2011. His career as a professional referee was completely ruined.
Monday 30th April, De Santis, a police constable, was guest of a TV program for a north Italian broadcaster, Antenna Tre Lombardia. During the program, he declared that he never talked on the phone with the former Juventus director Luciano Moggi, who was present, and the he also confessed that during his not so short career as a Serie A referee he received a lot of calls from former Inter president, the late Giacinto Facchetti, and former Milan “liaisons with referees” manager Luca Meani.
He also confessed to be an Inter supporter, and that he used to enjoy a very fine relationship with the late Facchetti, marred only from some occasional requests that “some time went also beyond the legal”.
Italian media basically skipped about this story, when they reported it they were mostly commenting about De Santis’ bad taste on talking about some already dead fellow.
Mainstream Italian media (apart a few lines http://qn.quotidiano.net/2007/05/04/9333-santis_sentito_moggi_facchetti.shtml) and English media basically ignored De Santis’ comments so far.

Improving build performances: assemblies on the GAC

Working with a Monorail based web project, to improve build performances, I added some of the Castle Project assemblies to the GAC.

Obviously, there is a time when you need a fast build, and there is a time when you need to be sure that all the assemblies your application is using are in the /bin folder, so I wrote 2 batch files to cache and uncache the assemblies listed in 2 text files (some example code at the end of the article).

I tested on my machine with a sizable web application project the effect of having or not having those assemblies on the GAC on the build time, and this are the rounded average results:

Solution
GAC 10"
No-GAC 27"

Web
GAC 5"
No-GAC 22"

That is, on my machine when I put the files on the GAC, when building I gain 17".

[Note: The web build is normally 5" faster than the Solution build because in the solution there are project not directly needed by the Web (In primis the tests, then some data migration and seeding stuff, some productivity measurament tool, and so on).]

Why it happens? My best answer is that having the Castle project assemblies in the GAC means that during the build these are not copied anymore in the various /bin folders.

I do advice you to try to test this in your machines as well, anyway, please read carefully and make sure you understand the next points before moving ahead:

- the build phase generate assemblies from the source code. Those assemblies are bytecode, not executable programs, and if not opportunely precompiled, they will be compiled at the startup of the .Net application. This means that after building, when you will refresh the web page, this will take exactly the same time to answer you as it was taking before putting the Castle Project assemblies on the GAC.

- I had to modify the web.config, because the assembly Castle.ActiveRecord has to be explicitly added this way.

- You may notice I also added Castle.Monorail.Framework and Castle.Monorail.Views.Brail. Unfortunately it seems IIS is requiring those two files before reading all the web.config, so I add to change the property Copy Local to true (so in the /bin folder of the web application project you still can find these two assemblies and their related files). This means that you need to slightly the web application project as well to get those latest needed changes.

- I didn't test the web with a web site project configuration, if you happen to use such, you will probably get an error due missing assemblies. I do not advice to use such project option for an unending list of reasons, but if you really wish to persist on your own way, you should solve that just making sure the appropriate assemblies are copied in the /bin folder of the web application. Manually (I mean with some pre o post build event) copying files is a bad practice, because it will worsen the performances of the build, so exercise with care.

- You may be tempted to modify the list of the assemblies to include in the GAC. Please exercise care when doing so, because you can only add strongly typed assemblies this way. In example Newtonsoft.Json is not in that list because it is not a strongly typed assembly.

- Microsoft doesn't recommed to put in the GAC the assemblies you are developing. That is, MyDomainModel.dll or MyInitialisation.dll have still to be keept outside the GAC. If in the future you stabilize one of those libraries, you may decide to sign it (to make it strongly typed) and that to add it to the GAC.

I tested all of this on this Monorail based web project, but its results will probably be more or less similar with normal ASP.NET projects.

Batch file to cache assemblies

pause
"C:\Program Files\Microsoft.NET\SDK\v2.0 64bit\Bin\gacutil" /f /il .\toCache.txt
pause


Batch file to uncache assemblies

pause
"C:\Program Files\Microsoft.NET\SDK\v2.0 64bit\Bin\gacutil" /f /ul .\toUncache.txt
pause


ToCache.txt (list of file to cache, note the relative path to the working directory of the batch file)

..\..\Resources\bin\anrControls.Markdown.NET.dll
..\..\Resources\bin\Boo.Lang.Compiler.dll
..\..\Resources\bin\Boo.Lang.dll
..\..\Resources\bin\Boo.Lang.Parser.dll
..\..\Resources\bin\Castle.ActiveRecord.dll
..\..\Resources\bin\Castle.Components.Binder.dll
..\..\Resources\bin\Castle.Components.Common.EmailSender.dll
..\..\Resources\bin\Castle.Components.Common.EmailSender.SmtpEmailSender.dll
..\..\Resources\bin\Castle.Core.dll
..\..\Resources\bin\Castle.DynamicProxy.dll
..\..\Resources\bin\Castle.Monorail.ActiveRecordSupport.dll
..\..\Resources\bin\Castle.Monorail.Framework.dll
..\..\Resources\bin\Castle.Monorail.Views.Brail.dll
..\..\Resources\bin\Castle.Services.Logging.Log4netIntegration.dll
..\..\Resources\bin\Iesi.Collections.dll
..\..\Resources\bin\log4net.dll
..\..\Resources\bin\NHibernate.dll
..\..\Resources\bin\NHibernate.Caches.SysCache.dll
..\..\Resources\bin\NHibernate.Generics.dll
..\..\Resources\bin\Nullables.dll
..\..\Resources\bin\Nullables.NHibernate.dll


Batch file to uncache assemblies

anrControls.Markdown.NET
Boo.Lang.Compiler
Boo.Lang
Boo.Lang.Parser
Castle.ActiveRecord
Castle.Components.Binder
Castle.Components.Common.EmailSender
Castle.Components.Common.EmailSender.SmtpEmailSender
Castle.Core
Castle.DynamicProxy
Castle.Monorail.ActiveRecordSupport
Castle.Monorail.Framework
Castle.Monorail.Views.Brail
Castle.Services.Logging.Log4netIntegration
Iesi.Collections
log4net
NHibernate
NHibernate.Caches.SysCache
NHibernate.Generics
Nullables
Nullables.NHibernate


Web Site Projects vs Web Application Projects

For web development, Visual Studio 2005 (VS 2005) supports two project options: Web Site Projects (WSPs) and Web Application Projects (WAPs). The earlier was built-in with the initial release of VS 2005, the latter were introduced later and are included on the VS 2005 SP1.

WSPs are leveraging the very same dynamic compilation system used by the ASP.NET 2.0 at runtime; when using this option, you don't need a project file (you just need to follow some easy and intuitive folder conventions).

WAPs are leveraging the MSBuild buld system, and when using them, all the code in a project is builded in a single assembly; when using this option, you do need a project file.

There are various pro and cons for each of those models, but if we focalize just on the speed of the build, the WAP option wins hand down every time you build from scratch or rebuild.

Why?

The main reason, as explained from Scott Guthrie, is that with the WAP, only the code-behind and the /app_code/* classes are built, while with the WSP the ASP.NET runtime spend its time analyzing and compiling each ASP.NET related code (content/controls/inline code).

Monorail case:

First and foremost, having a MSBuild capable project, as in the WAP option, helps when you have to integrate your projects with tools like CruiseControl.Net. Now, it is possible to do that also for the WSP option, but it is not as neat.

Second, using Monorail, the amount of ASP.NET pages in the web projects is probable negligible (usually just 1, quite trivial in every chances, which is left there to "fire" the build on certain circumstances), so there is not really any reason to use the deep verification features of the WSP model: there are not ASP.NET related file to build.

From experience and from investigation, I can add that on the web there are various tips and tricks to speed up the WSP, but the point is that as much as you can speed up a WSP build, the main target of this kind of exercise is to make it as fast as a WAP build.

Conclusion:

On non trivial web projects, and when you don't need deep verification of ASP.NET related files, use ever the WAP option.

p.s.: I shouldn't add this, but really, don't use the /app_code folder under almost any circumstance. Put your logic in different layers (VS class projects), and if really you need to use the /app_code folder, leave there only very stable classes.

Transaction and Concurrency on SQL Server 2005

Ayende recently posted an article about Transaction and Concurrency, and yeah, I read his blog very often, and if you are a .Net developer you should do it as well.
He had noticed that using the ReadCommitted isolation level something was not working as he had expected. I have to confess that when I first read his post and some of the comments, I had superficially concluded that could be related to the emergence of the Phantom phenomenon.

The SQL standard (at last SQL-92 and SQL-99) is defining the Phantom phenomenon as such:

P3 (‘‘Phantom’’): SQL-transaction T1 reads the set of rows N that satisfy some <search>.
SQL-transaction T2 then executes SQL-statements that generate one or more rows that satisfy the <search> used by SQL-transaction T1. If SQL-transaction T1 then repeats the initial read with the same <search>, it obtains a different collection of rows.

Today I tried to test Ayende's code, and I finally understood the issue is far more interesting than what I was supposing (lecture 1: run the code you are trying to understand).

Under the hypothesis of Ayende (which are far from being exotic, but for a little point), the ReadCommitted isolation level doesn't work as expected on Microsoft SQL Server 2005! Kudos to Stuart Carnie, who had grokked this well before me, and pointed out a very interesting article from Tony Rogerson.

Rogerson's conclusions are that on Microsoft SQL Server 2005 the ReadCommitted isolation level (and its ReadCommittedSnapshot sibling) "does not give a point in time view of your data", so when the need arises, the safe way to go may be to use the Snapshot isolation level (which incidentally seems to have been a playtoy of Microsoft Research since 10 years).

Today I played a bit with Ayende's code, and I wrote a little console application I called TransactionsAndConcurrency.exe:
TransactionsAndConcurrency mode ilp ilc [iterations [records]]
The mode argument can be one of "aye", "aye-run" or "ale", ilp and ilc one of "ch", "rc", "rr", "ru", "se", "sn" (respectively Chaos, Read Committed, Repeatable Read, Read Uncommitted, Serializable and Snapshot) while iterations and records should be self-explanative ints.
In example you may call this little console application as such:
TransactionsAndConcurrency aye rc rc
or as such:
TransactionsAndConcurrency ale rc rc 20 500
You will notice that when called with aye, this application is probably working as Ayende's code, with aye-run with a slightly different behaviour (doesn't stop the first time the consumer fetch an "unexpected" amount of rows) instead with ale unexpectedly works (I tested it until with rc and ru until 5000 various times, not a single glitch).
What is the difference? The exotic point on Ayende's hypothesis, that is that its table doesn't have a primary key. If the same exercise is done with a table with a primary key (using a surrogate key through the T-SQL Identity column) everything goes fine.

I have to confess that I am still wondering if this is a bug of Microsoft SQL Server 2005, because those are obviously uncommitted phantom rows that are showing up because something in the range lock of the tables without primary keys is obviously not working as most of us would expect.

If you wish to play as well, here is the code:

using System;
using System.Data.SqlClient;
using System.Threading;
using System.Data;

namespace TransactionsAndConcurrency
{
public class Program
{
   #region Vars and Consts
   private const int DEFAULT_ITERATIONS = 20;
   private const int DEFAULT_RECORDS = 500;

   private static int records = DEFAULT_RECORDS;
   private static int iterations = DEFAULT_ITERATIONS;
   private static IsolationLevel isolationLevelProducer = IsolationLevel.Unspecified;
   private static IsolationLevel isolationLevelConsumer = IsolationLevel.Unspecified;
   private static string mode;

   private static string connectionString = "Data Source=myDB;Initial Catalog=test;User=sa;Pwd=ICannotSay1!;";
   #endregion

   #region Setup
   private static void Setup()
   {
       SqlConnection connection = new SqlConnection(connectionString);
       connection.Open();
       SqlTransaction sqlTransaction = connection.BeginTransaction();
       for (int i = 0; i < records; i++)
       {
           SqlCommand sqlCommand = connection.CreateCommand();
           sqlCommand.Transaction = sqlTransaction;
           if (mode != "ale")
           {
               sqlCommand.CommandText = "IF  EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[t]') AND type in (N'U')) DROP TABLE [dbo].[t]";
               sqlCommand.ExecuteNonQuery();
               string create = "CREATE TABLE [dbo].[t]("
                   + " [id] [int] NOT NULL"
                   + " ) ON [PRIMARY]";
               sqlCommand.CommandText = create;
               sqlCommand.ExecuteNonQuery();
           }
           else
           {
               sqlCommand.CommandText = "IF  EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[s]') AND type in (N'U')) DROP TABLE [dbo].[s]";
               sqlCommand.ExecuteNonQuery();
               string create = "CREATE TABLE [dbo].[s]("
                   + " [id] [int] IDENTITY(1,1) NOT NULL,"
                   + " [field] [int] NULL,"
                   + " CONSTRAINT [PK_s] PRIMARY KEY CLUSTERED"
                   + " ([id] ASC) WITH (PAD_INDEX  = OFF, IGNORE_DUP_KEY = OFF) ON [PRIMARY]"
                   + ") ON [PRIMARY]";
               sqlCommand.CommandText = create;
               sqlCommand.ExecuteNonQuery();
           }
           sqlCommand.Dispose();
       }

       sqlTransaction.Commit();
       Console.WriteLine("Create table");
       connection.Close();
   }
   #endregion

   #region TearDown
   private static void TearDown()
   {
       SqlConnection connection = new SqlConnection(connectionString);
       connection.Open();
       SqlTransaction sqlTransaction = connection.BeginTransaction();
       for (int i = 0; i < records; i++)
       {
           SqlCommand sqlCommand = connection.CreateCommand();
           sqlCommand.Transaction = sqlTransaction;
           if (mode != "ale")
           {
               sqlCommand.CommandText = "IF  EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[t]') AND type in (N'U')) DROP TABLE [dbo].[t]";
               sqlCommand.ExecuteNonQuery();
           }
           else
           {
               sqlCommand.CommandText = "IF  EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[s]') AND type in (N'U')) DROP TABLE [dbo].[s]";
               sqlCommand.ExecuteNonQuery();
           }
           sqlCommand.Dispose();
       }

       sqlTransaction.Commit();
       Console.WriteLine("Drop table");
       connection.Close();
   }
   #endregion

   #region Producer
   private static void Producer()
   {

       SqlConnection connection = new SqlConnection(connectionString);
       int iteration = 0;
       while (true)
       {
           connection.Open();
           SqlTransaction sqlTransaction = connection.BeginTransaction(isolationLevelProducer);
           for (int i = 0; i < records; i++)
           {
               SqlCommand sqlCommand = connection.CreateCommand();
               sqlCommand.Transaction = sqlTransaction;
               if (mode != "ale")
                   sqlCommand.CommandText = "INSERT INTO t (Id) VALUES(@p1)";
               else
                   sqlCommand.CommandText = "INSERT INTO s (field) VALUES(@p1)";
               sqlCommand.Parameters.AddWithValue("@p1", iteration);
               sqlCommand.ExecuteNonQuery();
               sqlCommand.Dispose();
           }

           sqlTransaction.Commit();
           Console.WriteLine("Wrote {0} records in iteration {1}", records, iteration+1);
           iteration += 1;
           connection.Close();
           if (iteration == iterations)
               return;
       }
   }
   #endregion

   #region Consumer
   private static void Consumer()
   {

       SqlConnection connection = new SqlConnection(connectionString);
       int iteration = 0;
       while (true)
       {
           connection.Open();
           SqlTransaction sqlTransaction = connection.BeginTransaction(isolationLevelConsumer);
           SqlCommand sqlCommand = connection.CreateCommand();
           sqlCommand.Transaction = sqlTransaction;
           if (mode != "ale")
               sqlCommand.CommandText = "SELECT COUNT(*) FROM t GROUP BY id ORDER BY id ASC";
           else
               sqlCommand.CommandText = "SELECT COUNT(*) FROM s GROUP BY field";
           SqlDataReader sqlDataReader = sqlCommand.ExecuteReader();
           if (sqlDataReader.RecordsAffected != -1)
               Console.WriteLine("Read: {0}", sqlDataReader.RecordsAffected);
           while (sqlDataReader.Read())
           {
               int count = sqlDataReader.GetInt32(0);
               if (mode != "ale")
                   Console.WriteLine("Count = {0} in {1} iteration", count, iteration+1);
               if (count != records)
               {
                   if (mode == "ale")
                       Console.WriteLine("Count = {0} in {1} iteration", count, iteration+1);
                   if (!(mode == "aye-run"))
                       Environment.Exit(1);
               }
           }

           sqlDataReader.Dispose();
           sqlCommand.Dispose();
           sqlTransaction.Commit();
           iteration += 1;
           connection.Close();
           if (iteration == iterations)
               return;
       }
   }
   #endregion

   #region Delete
   private static void Delete()
   {
       SqlConnection connection = new SqlConnection(connectionString);
       connection.Open();
       SqlTransaction sqlTransaction = connection.BeginTransaction();
       for (int i = 0; i < records; i++)
       {
           SqlCommand sqlCommand = connection.CreateCommand();
           sqlCommand.Transaction = sqlTransaction;
           if (mode != "ale")
           {
               sqlCommand.CommandText = "DELETE FROM t";
               sqlCommand.ExecuteNonQuery();
           }
           else
           {
               sqlCommand.CommandText = "DELETE FROM s";
               sqlCommand.ExecuteNonQuery();
           }
           sqlCommand.Dispose();
       }

       sqlTransaction.Commit();
       Console.WriteLine("Delete data from table");
       connection.Close();
   }
   #endregion

   #region Main
   private static void Main(string[] args)
   {
       // string describing the isolation level of the producer
       string ilp = string.Empty;
       // string describing the isolation level of the consumer
       string ilc = string.Empty;

       if ((args.Length > 2) && (args.Length < 6))
       {
           mode = args[0];
           if ((mode != "aye") &&amp;amp;amp;amp; (mode != "aye-run") && (mode != "ale"))
               Environment.Exit(2);
           ilp = args[1];
           ilc = args[2];
       }
       else
           Environment.Exit(3);
       if (args.Length > 3)
           int.TryParse(args[3], out iterations);
       if (args.Length == 5)
           int.TryParse(args[4], out records);

       isolationLevelProducer = getIsolationLevel(ilp);
       isolationLevelConsumer = getIsolationLevel(ilc);

       try
       {
           Setup();
           Delete();
           Thread p = new Thread(Producer);
           Thread c = new Thread(Consumer);
           p.Start();
           c.Start();
           while ((p.IsAlive) || c.IsAlive)
           { }
       }
       finally
       {
           TearDown();
       }
   }
   #endregion

   #region Utils
   private static IsolationLevel getIsolationLevel(string isolationLevel)
   {
       IsolationLevel il = IsolationLevel.Unspecified;

       switch (isolationLevel)
       {
           case "ch":
               il = IsolationLevel.Chaos;
               break;
           case "rc":
               il = IsolationLevel.ReadCommitted;
               break;
           case "ru":
               il = IsolationLevel.ReadUncommitted;
               break;
           case "rr":
               il = IsolationLevel.RepeatableRead;
               break;
           case "se":
               il = IsolationLevel.Serializable;
               break;
           case "sn":
               il = IsolationLevel.Snapshot;
               break;
           default:
               il = IsolationLevel.Unspecified;
               break;
       }
       return il;
   }
   #endregion
}
}

LLBLGen Pro to support NH, GN, AR: +1

I just read a post from Ayende about the sad state of NHibernate code generation. Between the comments, I spotted a few ones from Frans Bouma.

Exactly one year ago (well, minus a couple of days really), after a thoroughly careful evaluation, I decided to purchase LLBLGen Pro's license. I liked the tool since I first used it and in a very short time it let me to become very productive. I think I was a quite advanced user, I wrote new templates, inserted base classes and stuffs like that.

Problem was, a couple of people in the team never really grokked it, and one of them was the manager. I was sincerely not understanding why they were not able to be as productive as me, so I did some research looking for something to help them to become more proficient, and I found a very clear article written by Frans Bouma. I referred that to my team mates, and after reading that, my manager and some other team member found themselves more in camp 3 (the domain model approach) than 2 (the entity approach). At the time, after evaluating what the market was offering, we switched to ActiveRecord + NHibernate + NHibernate.Generics (incidentally, I was totally against using NHibernate without ActiveRecord due some previous hellish experiences with older versions of Hibernate). One of the alternatives we tested was Genome, which I remember to be quite promising, but which at the time was having very sloppy performances (but it was by no means the worst between the ones we evalueted).

I found that personally I feel comfortable with both the approaches (as long as people don't commit repository breaking code, I am comfortable with almost anything), but some people seems not to be so flexible, so I do sincerely believe it could be a good strategy for LLBLGen Pro to start supporting NHibernate templates (and, even better, ActiveRecord and Gentle.Net templates as well).

Mutex could not be created

Working with ASP.Net, have you ever got a:

Server Error in '/xxx' Application. -------------------------------------------------------------------------------- Mutex could not be created.

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. bla bla bla ..

I have solved it in various way, but so far the best workaround I have got (thanks mostly to a post of a chap called Joao Morais) is the following:

• - If you have visual studio 2005 is open, close it
• - Go to the ASP.NET temporary folder for v2.0 of the framework
• \Microsoft.Net\Framework\v2.0\Temporary ASP.NET pages
• - Remove the folder for your application (or all of them)
• - Reset IIS (on a command line window, >iisreset) [not always needed, but I had to use it sometimes]
• - First Browse your page from IE (http://localhost/your app)
• - Then reopen Visual studio

I can only add that a aspnet_regiis -ga \ before resetting IIS, is usually helping, as in 64 bit environments to try to enable/disable the Enable32bitAppOnWin64 IIS Metabase properties.

cscript %SystemDrive%\inetpub\AdminScripts\adsutil.vbs set w3svc/AppPools/Enable32bitAppOnWin64 0
%SYSTEMROOT%\Microsoft.NET\Framework64\v2.0.50727\aspnet_regiis.exe -i

cscript %SystemDrive%\inetpub\AdminScripts\adsutil.vbs set w3svc/AppPools/Enable32bitAppOnWin64 1
%SYSTEMROOT%\Microsoft.NET\Framework\v2.0.50727\aspnet_regiis.exe -i

Warning: this is a brute force approach, a bit like formatting an hard disk because the operating system got some cold, use with discrection!

CruiseControl.Net and SVN over SSL

On Microsoft Windows (2003 and XP) CruiseControl.Net is usually running as a service under the Local System Account (LSA), but, to update the working copy used from the integration service using a SVN repository contacted through SSL, we have to accept permanently the server certificate.
To achieve that, we have to log as Local System, which is accomplished using the command at with the interactive switch. In example, let's suppose that the time is 16:59, running the following command we'll open a Command Prompt running as LSA:
at 17:00 /interactive cmd.exe
Once logged in as LSA, it is just a matter of launching a svn up (or another command which is permitting us to save the server certificate) from the desidered working copy folder, and to accept permanently the server certificate.
The server certificate will be then saved on the $UserProfile\Application Data\Subversion\auth\svn.ssl.server folder (where in the case of the LSA $UserProfile is usually something as C:\Documents and Settings\LocalService\).
Warning: no, you cannot log as LSA using the abovementioned trick from a terminal server. Use the console!