I’ve just published – and created an initial release for – a set of Visual Studio project and item templates for StreamInsight. The release dropped yesterday is a beta primarily for feedback on the templates, particularly the naming conventions, comments and sample boilerplate code. The items are placed in a “StreamInsight” grouping in the Add New dialogues so they’re very easy to find. There are four project templates: A console application project template that creates a basic app based on the Simple StreamInsight Application that I posted about 2 months ago. A StreamInsight library project template that creates a blank DLL project with the StreamInsight references added. Unlike the built-in DLL project template, this template does not include “Class1.cs” … which has always gotten on my nerves. Item templates for both input and output adapters, typed and untyped. These are multi-file item templates and add the factory, a configuration class and adapters for edge, point and interval events. The typed adapters add a basic/simple type class as well. Each adapter is placed in its own folder and has its own namespace. Because of how the replacement parameters work with the templates, the file names have the format [BaseItemName][Direction].[ItemType]. The base item name will be a short name for the source/destination for the adapter. For example, a point adapter for, say, WCF input adapter for edge events would be WcfInput.Edge. When naming this in the Add New Item dialogue, you would use “Wcf” only. Project Templates: Item Templates: New Item in Solution Explorer: In the near future, I’m planning on adding item templates for the various StreamInsight specific extension points such as UDA’s and, when version 1.2 is released, UDSO’s. In the longer term, I’d like to make this a wizard rather than simple item templates. A wizard may allow me to get away from the “.” in the file names and it will allow users to select existing classes for the typed input adapters rather than creating a “phony” one, thereby creating a better user experience. Once I do that, the VSIX will need to be installed via MSI, which would also allow for the addition of Visual Studio snippets for common query patterns and other code chunks. Ambitious goals, perhaps, but those are the only ones worth having, right? Creating the templates themselves was a royal pain in the a$$. Not so much because of the actual process but because the documentation for creating templates sucks so badly. Some of the documented replacement parameters documented are simply wrong – as in, they don’t get replaced!. Others aren’t documented at all, which makes discovery of these things far more difficult. Considering the number of templates that the Microsoft folks create (and especially the Visual Studio group), one would hope that the documentation would be better that the suckage that exists on MSDN. I wound up digging around in the Visual Studio templates installed on my machine than to figure out some of this stuff. Of course, because I am a big fan of .NET Open Source software, it is available on CodePlex … and you, dear reader, are welcome to sign up if you’d like to help!
Well, I wanted to provide a little update on my previous discussion on the my performance testing methodology; I've refined it a bit while getting ready for the Austin Code Camp.
Of course, GC.Collect() is still very important ... but I must correct myself in the previous post. It's called before each test method run. This ensures that the garbage collector is all cleaned up and collected before the test run even starts executing.
Now, on the calculations. I still do a normalized (or perhaps weighted, but we're getting into semantics here) average. But ... I've altered the equation a bit to subtract the overhead associated with the profiler probe. These were, surprisingly, pretty different across the board with the different test methods. It really is appropriate to discount these from the overall results as they do impact the overall numbers. And, considering the differences between them in the various methods (in one set of tests, it ranged from .1 msec to 2.54 msec), they really needed to be removed from the results.
The final tweak was to make a call to each of the test methods before I went into the actual test. This was done in a separate Initialize method. This ensures that all of the classes being used (as was mentioned in the previous post) are loaded into memory and initialized. It also ensures that the methods themselves are JIT'd before the test runs begin as well; again, this is something that we need to take out of the final equation.
Here's the latest on Brian Harry's blog: http://blogs.msdn.com/bharry/archive/2008/01/09/jan-08-devdiv-dogfood-statistics.aspx. Keep in mind that this is just for DevDiv (Developer Division) ... so we're talking about the developer tools (i.e. Visual Studio, etc). There are other groups using TFS inside of Microsoft. Looking at this ... wow ... it gives you an appreciation of the amount of development that is done here. Speaking of TFS, I heard a customer story about TFS recently that really speaks to the value. So ... a remote dev center was coding away on an application, but Central IT noticed that they had a lot of bugs being re-opened and that overall, things didn't seem to be going too well. Of course, re-opened bugs ... well ... that's definitely a problem. This was exposed to the Central IT team through the OLAP and reporting capabilities of TFS. But the data and processes ... well, it's easy and natural for devs to work with so they tend to actually do it. Having the working items completely integrated into VS and into your check-in process was a huge thing for me. So ... Central IT was very concerned and they had the remote team come down to see them. They went over what was going on. As it turned out, the remote team didn't have a clear idea of the vision of the project and this was leading to a lot of issues. They got this resolved ... and the team got back to work. This time, though, the stats were better. It took the business another 3 weeks to notice this; by that time, the underlying issue had been corrected. Three weeks of development time that would have been ... ummm ... less than productive had already become very productive. It saved 3 weeks on a time-critical project. And, of course, the largest cost of a software project is the people ... the developers' time. Well, I thought it was cool.