Working on F# with NDepend

Following up from the earlier post here, looking more in depth at some of the results out of NDepend for my own little code quality project, and in addition to the results noted there

First out of the gate, for sanity's sake, when working with F#, it would make sense to add !HasAttribute OPTIONAL:System.Runtime.CompilerServices.CompilerGeneratedAttribute onto every rule. That way, you are not left to contemplate how and why the generated CompareTo methods on a simple record type

can have an IL Cyclomatic complexity of 43.

A similar blanket exclusion should be applied to types whose names contain the sigil @ marking them as generated function objects. Unfortunately, compiler inserted write-once local variables do not have any such distinctive sigils by which they can be filtered, just unexciting normal names like str, str2.

It would be nice if the HasAttribute condition also allowed you to filter on properties of the attributes like !HasAttribute OPTIONAL:Microsoft.FSharp.Core.CompilationMappingAttribute WITH (SourceConstructFlags & 31 == 1) OR (SourceConstructFlags & 31 == 2) -- to filter record or sum types, which are rich in generated structure, where appropriate. For example, though the sum type

may be implemented into the CLR by making Either<'a, 'b> an abstract type with concrete inner types Either<'a, 'b>+Left and Either<'a, 'b>+Right, the headline Either<'a, 'b> type is not an abstract base class in the usual sense, that user written code will be expected to extend the type.

Similarly, while it is arguably an oversight of the F# code generation that none of the inner types within a sum type are marked as sealed -- there is no good case for extending them -- it would also be nice to be able to exclude all (implicitly, generated) types that derived from (or are an inner type of) a sum or record type from analysis; as well as all the content of types attributed with [CompilationMapping(SourceConstructFlags.NonpublicRepresentation | ...)].

Allowing names of the form |ActivePattern|_| is a trivial extension of the "names should be upper case" rule.

There is what looks like a bug in NDepend's analysis as it manages to make all the types I have derived from Microsoft.FxCop.Sdk.BaseIntrospectionRule pass the filter

DepthOfInheritance == 1 // Must derive directly from System.Object

The containing rule for that test Classes that are candidate to be turned into Structures could probably also benefit from having AND !IsStatic added, so as to exempt functional modules.

F# under the covers XI -- Literal expressions that aren't; attributes that don't

The problem with being unable to attribute just the getter or setter of a property is resolved at the 1.9.9.9 release (February 2010 CTP).

Consider the following simple C# property

This compiles in debug mode to

which assigns the literal result -- 6 -- of the expression to the temporary that is then returned. The analogous F# method

compiles to IL which preserves the expression to execute only at run time:

which makes static analysis of the code in the more usually analysed state a rather more complicated business.

Another complicating factor is that while this syntax

builds, it attaches the attribute (with Method and Constructor usage only) to the property as a whole; and not the generated get_Options method, as in

What I'd like to express, and what I've not found a way to express, is

and MSDN remains opaque on the subject as well.

This is something I could code around, but I can't yet see a clean way of doing to allow separate attributes on the getters and the setters.

Later: The intended syntax for this option is, much as you might expect, this:

with the necessary insets as shown (to avoid error FS0010: Incomplete structured construct at or before this point in pattern), however on filing a bug report I got confirmation that it is a known issue with the Beta 2 release (aka October 2009 CTP), that this still decorates the property and not just the getter.

Links for 24-Nov

Simon Willison is excited by Node.js

PPK remarks that Apple is not evil -- and may be surprisingly cunning.

Luke Hoban’s F# for Parallel and Asynchronous Programming (PDC video)

Bart Czernicki’s Silverlight 3 and F# Support in Visual Studio 2010

Pense-moi's Immutable queue library for F#

F# under the covers X -- the curious case of record types

This is unchanged in the February 2010 CTP (1.9.9.9) release.

They're coming thick and fast now...

Define a record type such as

The class that results looks like

where the source context for each method is the same -- the range containing just the type name.

Now, you wouldn't expect anything to be seriously unusual about this class in terms of its implementation, but there is.

FxCop reminds you about the IComparable should-haves that can't be enforced through interface constraints:

[Location not stored in Pdb] : warning : CA1036 : Microsoft.Design : 'Context' should define operator '!=' since it implements IComparable.
[Location not stored in Pdb] : warning : CA1036 : Microsoft.Design : 'Context' should define operator '<' since it implements IComparable.
[Location not stored in Pdb] : warning : CA1036 : Microsoft.Design : 'Context' should define operator '==' since it implements IComparable.
[Location not stored in Pdb] : warning : CA1036 : Microsoft.Design : 'Context' should define operator '>' since it implements IComparable.

but which aren't there; and Reflector's decompilation to C# is stymied by the highlighted CompareTo overloads -- the simple one is implemented as

which fortunately doesn't give much scope for things to go wrong.

In this and the previous case, the offending instruction that Reflector balks at is a simple branch such as indicated in this example

It's quite clear at every turn that F# is coming at the problem of code generation from a very different direction to the well explored parts of the phase space of valid IL that C# and VB.net dabble in. And clear, too, that this will present a significant challenge to all writers of tools to work with the language -- it takes us well out of our old comfort zone.

F# under the covers IX -- the case of the missing coverage

As I'm driving my little set of FxCop rules and associated helpers to a usable state, I'm looking in more detail at how the code coverage in the automated system/integration test is going, trying to drive that towards 100%. And as usual, the F# code generation is throwing up some interesting results.

Take this method (lightly refactored from last time) which starts at line 167 in the actual source:

Although my tests give a match for every case in the pattern -- and nCover shows a visit for every line from 168 to 176 (2 to 9), and the expression body of each branch of the if/else, it also claims that a code point that spans from from line 168 col 5 to line 177 col 24 (everything from if to then inclusive) is not visited.

Unfortunately, the IL generated for this method is not back-compilable to C# in Reflector (yes, another problem report submitted), so I can't use that to analyse what is going on in this particular case : I shall have to inject some diagnostic code into the rule itself to see what FxCop thinks is going on as a statement which spans those lines.

But this isn't the only mysterious bit of uncoverage I've found in F# code -- in the previous state of the method (as showcased here), it was the identifier result in let result = match… that got the uncoverage, despite the named value being used as the final expression in the method.

In most cases, it is possible to refactor away such temporaries, and clear the spurious uncoverages that they give rise to; but, as this example shows, it is not always possible.

Later

By recursively dumping Statements in FxCop for this method (which inserted 2 lines in a routine above this one, thus displacing the line numbers), I get a lot of multiple hits on statements (same type, same source context range), but the only ones with the appropriate source context are

Block -> from 170 : 5 to 179 : 24
Nop -> from 170 : 5 to 179 : 24
Nop -> from 170 : 5 to 179 : 24
AssignmentStatement -> from 170 : 9 to 170 : 22
AssignmentStatement -> from 170 : 9 to 170 : 22
Branch -> from 170 : 9 to 170 : 22
Block -> from 170 : 9 to 170 : 22
Branch -> from 170 : 9 to 170 : 22
Block -> from 170 : 9 to 170 : 22
Branch -> from 170 : 9 to 170 : 22
Block -> from 171 : 27 to 171 : 35
ExpressionStatement -> from 171 : 27 to 171 : 35
Nop -> from 171 : 27 to 171 : 35
…

So it seems that first Nop pair, with no other statement occupying the the same range, may be what it being detected as uncovered (unlike the second Nop, which overlaps the immediately preceding ExpressionStatement exactly).

The equivalent IL looks like

which does indeed have a pair of Nop opcodes at the start (lines 21 and 22).

So, that looks like another heuristic to add - removing unvisited code point records that correspond to nothing but a Nop.

The February 2010 CTP generates identical code here.

NDepend -- a belated "kicking the tyres" review

While there may be subtle details changed by the February 2010 CTP, the broad thrust of how F# looks to a C# directed tool remains. The mass of compiler inserted locals which skews the analysis certainly containues to hold.

First off, a little confession -- while I'm a C# coder by day, by night I use a whole variety of other languages instead; so I don't have a large corpus of my own code to use it against; and having installed my gift copy of NDepend v2.12.1.3123 (Pro Edition) on my personal development machine at home, there are certain ethical issues involved in getting it to run over code from the day job as-is.

I do have an on-going project in F#, which at least the tool will consume after a fashion (as the tool doesn't know .fsproj files from Greek, analysis has to proceed by selecting a number of assemblies to work with), so my experiences are based on that.

First impressions

At one extreme, tools such as FxCop or StyleCop provide a report that indicates a number of point changes to make to code with pretty much zero configuration to the tools; at the other extreme, nUnit and nCover are things that require you to configure (code) pretty much everything, and responding to the output is a matter of judgement. NDepend is somewhere in the middle -- you can run it with default settings and get some analysis of your code, but the settings are completely malleable; and the response can in places be a judgement call too.

There are some rules in the default set which overlap, and in places contradict, guidance from FxCop or StyleCop -- especially in matters like naming conventions. This latter is the stuff of which religious arguments are made (my own choice being to leave policing of names to the MSFT tools). Similarly, it invents a method-level NDepend.CQL.GeneratedAttribute to mark generated code when we already have System.Runtime.CompilerServices.CompilerGeneratedAttribute and the class-applicable System.CodeDom.Compiler.GeneratedCodeAttribute

Some of the conditions tested (setting aside the false positives out of F#) are useful complements to FxCop, such as flagging places where code visibility could beneficially be adjusted to better encapsulate code, and are essentially resolved by point fixes in many cases.

The real value comes in the rules that examine the structure of the code (even if, again, the precise thresholds used are potentially subjects for debate).

Live test

I ran NDepend against the latest drop of my FxCop rules project; this is possibly a small sample of code, and in some respects an edge case (with strong coupling to framework or FxCop classes, and very low abstraction).

Being F#, I have to just supply the assemblies, and cannot get source level analysis (such as cyclomatic complexity or comment density metrics).

Aside from the language difficulty the one place I have had trouble is making sense of the coverage file integration -- the nCover output I supply seems to get ignored; and the linked-to tutorial refers to such an earlier version that the UI doesn't match at all.

Interpretation (and F# peculiarities)

With no interfaces defined in the project, I seem to trigger a fencepost error in the report for class statistics

Stat	# Occurrences	Avg	StdDev	Max
Properties on Interfaces	0 Interfaces	0	0	-1 properties on
Methods on Interfaces	0 Interfaces	0	0	-1 methods on
Arguments on Methods on Interfaces	0 Methods	0	0	-1 arguments on

and there are some places where the names of F# methods may be problematic -- for a number of rules I get a "WARNING: The following CQL constraint is not satisfied. 1 methods on 358 tested match the condition.", but the name of the offending method I have to find for myself:

methods	# lines of code (LOC)	Full Name
Sum:	2
Average:	2
Minimum:	2
Maximum:	2
Standard deviation:	0
Variance:	0

As I've been posting in my "F# under the covers" series of posts, the IL that gets generated is, shall we say, interesting; and that is one of the things that gets shown up by rules tuned to C#. Indeed, the very first run I made was what provoked a one-shot into an on-going series. For example, this method

has 11 variables and a cyclomatic complexity (in IL) of 12

That method -- which is used in

where >?> and >+?? are combinators, gets detected as unused ("No Afferent Coupling -> The method is not used in the context of this application.")

A number of rules -- such as "A stateless type might be turned into a static type" -- show up quite how many compiler generated classes there are; both the ones with @-line-number names (e.g. Patterns+MapToTypes@41) for F# funs, and simple nested classes for algebraic types. In production use of NDepend on F#, these rules would have to be extended so such types could be filtered automatically.

For other false positives, such as the spurious numbers of locals, it would be beneficial to filter on a suppression attribute (along the lines of System.Diagnostics.CodeAnalysis.SuppressMessage) with a string property that the rule could match on.

Overall

Another useful and interesting tool in the general .net developer's armoury. Even out of the box, I could see that it would provide useful automation to supplement a manual code review process, though it would require a degree of customization to fit into an established build and coding standards regime.

Links for 19-Nov

New from the PDC:

• Windows Identity Framework RTM.
• Silverlight 4 beta.
• Reactive Extensions release (Introduction parts 3,4, another introduction).
• IronScheme 1.0 RC1.

WPF - from C# to F#.

Automating signing of PoSh scripts.

Prefer structured lifetimes – local, nested, bounded, deterministic.

Embedding IronPython in Silverlight.

Mono tools for Visual Studio.

Moonlight futures.

Google's 'go' -- is it the new VB?

Nature notes

Mild wet and very windy up from the south, so mild that, when I went downstairs to get a bedtime drink last night, I found that the cats had brought a frog in, and were just sitting around looking at it.

And while a few clear cool nights have just about ended the tomatoes for the year, it was only a couple of days ago I picked the last couple of properly ripe fruit off the Alicante in the greenhouse. The chilis are still going wild, and we're just getting the first capsicums ripening.

And of course even the tender plants like the fuchsia are going strong.

Alpha-encoding file versions

When building installers the UpgradeVersion must have a unique property value that is an installer public property (upper-case alpha). So, what better way of adding uniqueness than making it have the form "product name + product version" with the version suitably encoded...

So, a script for turning a file version (4 x 16bit ints) encoded as a System.Version into a short alpha string, assuming that Major and Minor will be small, and that common approaches are to step Build, to use a stepped Build plus date-stamped Revision, or a timestamp Build and Revision --

where the first two facets are encoded as telescoped base-13 (with a bit to say "more to come"), and the second two are encoded as pairs of bytes -- Z for a zero byte or as a 2-character base-25 representation if non-zero, with a zero Revision being dropped. This gives 10 characters in a plausible worst case, or as low as 5 in some conventions (stepped build numbers only); as opposed to the naive 64-bit as all-base-26 which would give 11 characters always.

“Hello GTK#” from the latest IronPython and F#

A little post to record a short bit of spiking with GTK# as UI toolkit, porting the simple C# examples from here to my preferred .net languages. Neither of these explorations are totally novel -- there are examples in either language to Google, but not all of them recorded all the details of the slight rough spots that needed a little working (and they were often not at all recent, either).

For IronPython 2.6 latest with GTK# 2.12.9-2, running the programs as ipy Program.py, the code looks like

where the dynamic nature of the language means we can lose a lot of the declaration clutter. We just have to explicitly reference the main GTK# assembly (which is GAC'd by the GTK# installer), and away we go.

F# was almost, but not quite, as smooth. You have to add references to additional assemblies atk-sharp and glib-sharp, and the types are a little more explicit:

With the project built as a Windows Application, the Console output doesn't show (even to the Output window in Visual Studio), so the code has been changed to update the button caption after clicking. Apart from that, GTK# follows the WinForms eventing model, so wiring up the events is just a matter of adding the appropriate handler functions in the same way as you would normally -- including the effect that the function value OnDelete needs to be coerced to a delegate type of the same signature such as via a wrapper fun as shown, and can't just be added directly.

This program is also not FxCop clean, but there's nothing GTK# specific about the tidying operations required.

As expected, these work unchanged with the 1.9.9.9 CTP

Links for 9-Nov

Windows Identity Framework ("Geneva") hits RC.

Trampoline recursion in C#.

Cross-browser XUL and SVG library.

CComPtr<T> type safety issue.

F#:

• Updated language spec.
• Equality and comparison constraints (Motivation).
• Reactive  framework (part 2).
• Multiple outputs from T4 made easy.

F# under the covers VIII

I'm in the tidying up stages for the little project I've been working on lately, a set of FxCop rules, mainly aimed at providing some complementary features for nCover-style code coverage, including static analysis for trivial methods -- or compiler generated ones. I've done the red -- add a set of methods that should give expected static analysis results -- and green -- write the simplest implementation that works when given the code base to analyse as a post-build activity. And now I'm on the refactor leg, abstracting out the common activities, taming sprawling methods into tighter ones, and reducing the level of imperative features (perhaps also making the code more idiomatic), even at times managing to delete whole methods.

And while I'm doing so, I'm running various tools over the code; FxCop and nCover over a small nUnit run every build, of course, but in turns Reflector and NDepend -- the results of which have been the inspiration for this series of posts.

And as we have seen, the code emitted by the F# compiler in its various iterations is unlike that coming from the more traditional .net langauges, in a way that doesn't always play nice with tools developed against C# or VB.net (and perhaps C++/CLI).

NDepend will deserve an essay of its own in due course -- part of the quid pro quo for the copy that Patrick Smacchia has generously donated -- but today's point of interest is that some IL code generated from F# cannot (at least with current Reflector builds) be easily folded back into C# source.

Take this active pattern used to match a property getter method that just returns a backing field with a name related to the property name, which is at a current intermediate stage of refactoring

where MapToTypes turns an FxCop StatementCollection into a list of their corresponding NodeTypes, and @ wraps String.StartsWith with ordinal comparison.

Confronted with a request to decompile into C#, Reflector 5.1.6.0 asks me to file a bug report (which I have done) for an exception "Invalid branching statement for condition expression with target offset 002A."

Now, as far as I can tell from the IL, this is part of line 2 where it is balking

Presumably -- I would have to build some equivalent C# code to validate -- the brfalse.s is not one that C# (or any of the other languages Reflector knows of) would actually emit. Certainly it seems that the new F# CTP compiler is fond of emitting it, and it breaks Reflector every time it does.

Another good reason for avoiding too many imperative constructs in your F# code, it would seem.

---

Later: this C# code that approximates the above F#

compiles to

which indeed, as I suspected, uses a brtrue.s for the final branch out of the if statement (though there is a brfalse.s to perform the short-circuiting.

Code generation here is unchanged in 1.9.9.9 from the previous CTP.

F# under the covers VII

Working with nCover to examine the number of code-points represented by any given method, the October CTP made a large number of changes -- some up, some down -- to the code generated by the compiler.

Most interesting is the change to attribute setters, which used to be NOP;assign;return and are now just assign;return. Also the name of the implicit backing field has changed to be the property name with an appended '@', rather than being a case-mangled version of the property name.

(As always, talking about Debug builds).

Later ... This one is a bit too cryptic for me to decipher what I was writing about. Code like this:

Has internal fields like Line@ still at 1.9.9.9; and property setters as in

compile to the NOP-bearing

with the Feb 2010 CTP.

Links for 3-Nov

Vista presentation mode tips and enabling applications.

Attaching a debugger on process start.

Simplicity is complicated.

F# immutable queue and range.

Validating JavaScript syntax with a .net wrapper.

Underscore -- a functional programming utility library for JavaScript

Nature notes

Driving home, after dining with colleagues, in the hard bright moonlight of a Hunter's Moon, I actually saw a live badger ambling along the verge at one point, a rare spotting of some of the larger wildlife around here.

Quite how starkly bright a winter full moon can be always comes as a surprise.

Also, still harvesting a last few tomatoes from both outdoor and greenhouse plants.