Saturday, November 21, 2009

F# under the covers IX -- the case of the missing coverage

As I'm driving my little set of FxCop rules and associated helpers to a usable state, I'm looking in more detail at how the code coverage in the automated system/integration test is going, trying to drive that towards 100%. And as usual, the F# code generation is throwing up some interesting results.

Take this method (lightly refactored from last time) which starts at line 167 in the actual source:


Although my tests give a match for every case in the pattern -- and nCover shows a visit for every line from 168 to 176 (2 to 9), and the expression body of each branch of the if/else, it also claims that a code point that spans from from line 168 col 5 to line 177 col 24 (everything from if to then inclusive) is not visited.

Unfortunately, the IL generated for this method is not back-compilable to C# in Reflector (yes, another problem report submitted), so I can't use that to analyse what is going on in this particular case : I shall have to inject some diagnostic code into the rule itself to see what FxCop thinks is going on as a statement which spans those lines.

But this isn't the only mysterious bit of uncoverage I've found in F# code -- in the previous state of the method (as showcased here), it was the identifier result in let result = match… that got the uncoverage, despite the named value being used as the final expression in the method.

In most cases, it is possible to refactor away such temporaries, and clear the spurious uncoverages that they give rise to; but, as this example shows, it is not always possible.

Later

By recursively dumping Statements in FxCop for this method (which inserted 2 lines in a routine above this one, thus displacing the line numbers), I get a lot of multiple hits on statements (same type, same source context range), but the only ones with the appropriate source context are

Block -> from 170 : 5 to 179 : 24
Nop -> from 170 : 5 to 179 : 24
Nop -> from 170 : 5 to 179 : 24
AssignmentStatement -> from 170 : 9 to 170 : 22
AssignmentStatement -> from 170 : 9 to 170 : 22
Branch -> from 170 : 9 to 170 : 22
Block -> from 170 : 9 to 170 : 22
Branch -> from 170 : 9 to 170 : 22
Block -> from 170 : 9 to 170 : 22
Branch -> from 170 : 9 to 170 : 22
Block -> from 171 : 27 to 171 : 35
ExpressionStatement -> from 171 : 27 to 171 : 35
Nop -> from 171 : 27 to 171 : 35

So it seems that first Nop pair, with no other statement occupying the the same range, may be what it being detected as uncovered (unlike the second Nop, which overlaps the immediately preceding ExpressionStatement exactly).

The equivalent IL looks like

which does indeed have a pair of Nop opcodes at the start (lines 21 and 22).

So, that looks like another heuristic to add - removing unvisited code point records that correspond to nothing but a Nop.

The February 2010 CTP generates identical code here.

3 comments :

Stephen Ward said...

which version of NCover were you using to generate coverage? the ppl @ ncover.com are usually pretty happy to investigate and fix bad instrumentation.

Steve Gilham said...

The last free one i.e. 1.5.8 -- which is the same version as we use in the day-job.

Steve Gilham said...

NCover version aside, the point of interest is that this spurious NOP-statement behaviour is something I've not observed with C#.

The code generation in F# is something alien that the existing generations of .net tools are not going to be at all at ease with, and when I find something anomalous, I'll log it so maybe it will help other tool writers when they start working on tools that accommodate the newlanguage.