Should.BeAvailableInF#

Since I first heard of it, I've found the Should Assertions library convenient for the driver programs that stand in for more formal unit tests for code fragments that I want to blog. However, such main programs have to be written in C#, because you can't access C# extension methods on generic open types from F# -- which is exactly what the general run of Should extensions are.

So, you could write F# code like

but that is pretty ugly. And F# type extensions are defined against a specific class, so won't be any use. Which leaves us with generic functions as the best way to wrap the C# extension methods, but

isn't much of an improvement.

The approach that brings us closer to the C# feel would be to make these methods infix operators.

In an ideal world, we'd use ⊦ (U+22A6 ASSERTION) as a lead character, but that's not yet available without forking the compiler, so I chose to write the functions in the form |-{op}, doubling the first character of the {op} for the versions that take custom comparison objects, adding a trailing % for variants taking a message, and using !- as an introducer for unary operations like !-/, where / alludes to ∅ (U+2205 EMPTY SET) for ShouldBeNull.

Apart from the object/generic assertion, the other type that needs transformation across the language barrier is the ShouldThrow<T> extension on Action, which can be wrapped as a generic function taking a unit -> unit, which uses that to create a delegate for the C# world.

Putting it all together we have module fhould, a pun on f for F# and for the old-fashioned cursive long 's' as in ſhould:

Note that there is no order checking in the range tuples (they are just passed as (low, high) to the corresponding arguments of the underlying code); and that 3-ary methods (custom comparers or message strings) have to be invoked in one or other of these styles to get the association correct

and F# syntax doesn't permit us to define unary/generic ShouldBeType or ShouldImplement operators in the style of (!-@)<'a>.

FxCop Rule CA1021:AvoidOutParameters — Not just for Morts

When you really have to return two values (like a return state with various values for success and failure, and some more substantive sought object on success

), resorting to an out parameter seems so obvious, and the relevant static analysis rule does say

It is safe to suppress a warning from this rule. However, this design could cause usability issues.

So "real programmers" will reach for that, regarding the rule as one to mollycoddle junior developers; and will shy from the clunkiness of an anonymous tuple type (even now we have them in core .net languages). But there is still a point where they cause pain, and that is when the parameter is on an interface method, and you want to mock that method.

At this point you have to individually specify each invocation of the mocked method and its pre-computed out parameters -- even though with modern mocking frameworks like Moq, you can use a lambda to compute the return value based on arbitrary inputs, and make a normal mocked method algorithmic over expected inputs.

rather than

Upshot -- the usability issue is not that the caller has to look at two values like a success state, and then if relevant a different value, rather than look at fields in some carrying type; it's that your customers (in the Total Quality sense) will end up with rather more coupled and brittle unit tests when they try to abstract your component away through its interface.

FsCheck is great for for native code too!

I've just had call to write some good old-fashioned 'C' code recently, for use in an environment where my access to anything fancy like external libraries is limited. And a one part needs a simple transcoding function to take data in and manipulate it from the form convenient to provide into a form that's actually useful.

But of course, I want to test that bit before I start wiring it into place. There are, of course a whole slew of 'C' unit test frameworks out there, none of which I've used before, but really I wanted to just soak-test the algorithm quickly and simply, with a maximum of coverage and a minimum of distractions, so, I actually wanted a QuickCheck-equivalent tool that would "just work".

And this is where F#'s easy P/Invoke syntax (almost as simple as cutting and pasting out of the 'C' header file) and FsCheck were just the tools I was looking for -- write a property (or, in fact, a series of properties) that asserted that an input mapped by my function was the same as obtained using an equivalent .net library function already to hand as reference, invoke them with FsCheck.Check.Quick, and then TDD away, starting with the rough outlines as asserted by the early properties, and then filling in details as asserted by the later ones.

C# under the covers

More of what the compiler does without you realising it, only this time in a C# sample.

The absolutely simplest sort of method, which you can clearly completely cover by calling it once. Right?

That's what I thought until I tried running OpenCover over code rather like this, and it told me I had only covered one of two possible branches.

So naturally, I go "WTF?", and wonder whether it's the return or throw alternatives out of .First that it's alluding to and to be certain, crack open ILSpy, and see that first line expands to

which actually involves first caching the delegate corresponding to the lambda, if it hasn't been already; or using it on subsequent calls.

The release build is much the same; it lacks the initial nop, and the meaningless jump-to-next-instruction at offset 0x23.

Move the constant string out into the method, and make the lambda close over it

and the caching goes away : a new instance of ClassLibrary2.Class1/'<>c__DisplayClass1' gets created every time, debug or release. And with it goes the branch.

Links for 8-Nov

Asynchronous programming in .net

• Async programming in F#2 and the PDC announced C#5 compared.
• The F# model in detail. 
• CTP downloads.
• Jon Skeet plays around with the internals of the CTP release here and here.
• Related : continuation passing style and asynchrony i.e. what the compiler is doing for you with await.

Also from the PDC -- what next for Silverlight?

Rx Design Guidelines.

SSL/TLS isn't computationally expensive.

NuGet -- gems for .net has a new name.

The Should Assertion Library.

Sandcastle MSBuild integration.

Links for 12-Aug

Using the Reactive Extensions:

• to simplify tests of asynchronous operations.
• monkey-patching with IronRuby (parts 1, 2).

F# Silverlight template.

PartCover patched to avoid writing to HKLM (my first patch to someone else's FOSS project!)

Untangling expression trees.

Using Code coverage -- did the right tests get written?

How we approach unfamiliar code (study).

Cosmos (C# OS kit) reaches milestone 5 with Visual Studio integration.

Links for 30-Jun

WCF dev-testing without admin privilege.

Understanding Monads (Part1, Part2).

Using .net code to interact with USB devices.

F#:

• FsCheck 0.7 release.
• PowerPack Commandline parser -- even if it didn't get into .net 4 core, it's there and usable if you can't or don't want to use Mono.Options
• Event handling -- convenience vs performance.

Links for 28-Apr

StyleCop now open-sourced.

Mono's C# compiler-as-service -- now works in .net too!

IronRuby+Rails+Rack+IIS7 -- between this stack and the same for JRuby in Tomcat. cross platform web-apps have become fun!

No Bugs (e-book).

Unit testing FxCop rules.

Axum (Concurrency focussed language on .net) for VS2010

Snap! -- Aspect-oriented support library for .net

Double-dispatch as a code smell.

F#

• Mailbox processors and WebSockets.
• Invoking from PoSh sessions.

Iron* console for VS2010.

coverage.exe and instrumenting F# files

I observed yesterday that coverage.exe balked in PdbReader.GetSegmentsByMethod() when trying to instrument F#-derived assemblies.

By checking to see whether a new CodeSegment entry would provoke the exception by having an identical offset to a pre-existing one, and dumping both old and new to the console, I observed that at least one, and often both, had associated line number 0x00Feefee -- i.e. were compiler generated code with no source reference.

And in the method Executor.ProcessMethod(), which is the only place to call PdbReader.GetSegmentsByMethod(), entries with this line number are discarded before beginning to instrument the IL code -- so it would seem to make perfect sense to discard these CodeSegments before putting them in the dictionary. With that change, instrumentation completes cleanly.

Of course, having built coverage.exe with C#4 for .net 4, the helper DLL it built is compiled for .net 4 as well, which has stymied my first test attempt. Oh well, enough for today.

Getting coverage.exe (trunk) to work with nUnit and .net 4

Following up from yesterday, about an instrumenting coverage tool that I'd spotted on Googlecode (Apr 2015 : rescued to GitHub), what I needed to do to get the instrumenting coverage tool working under .net 3.5sp1 with nUnit 2.5.2 driving unit tests.

First, the command line looks like

[path to]\ClassLibrary1.dll [path to]\UnitTestClassLibrary1.dll /x coverage.xml /r /exe [tool path to]\nunit-console.exe [path to]\UnitTestClassLibrary1.dll

where the /r (backup and replace) flag is essential; so you need to work on a copy directory with your assemblies and their .pdb files.

Second, you need to make the following change to the coverage main program in Runner.cs, line 92 from

to be

The hard-coded relative path could be added as yet another argument instead; and I've not played around with the shadow-copy parameter.

But that at least -- with a totally trivial test set -- provided me with a clean run and non-empty coverage data.

So the next thing to do will be to port the whole lot to .net 4 and see what gives. Also, it will be worth trying the teamcity branch (faking NCover 1.x) to see whether the different AppDomain usage there means we don't have to resort to this trick.

Later: doing a rebuild of Coverage and the sample test under .net 4, with command line

[path to]\ClassLibrary1.dll [path to]\UnitTestClassLibrary1.dll /x coverage.xml /r /exe [tool path to nunint 2.5.4]\nunit-console.exe /framework=net-4.0.30319 [path to]\UnitTestClassLibrary1.dll

where the nunit-console.exe.config has been adjusted appropriately, I get

Copyright (C) 2002-2004 James W. Newkirk, Michael C. Two, lexei A. Vorontsov.
Copyright (C) 2000-2002 Philip Craig.
All Rights Reserved.

Runtime Environment -
   OS Version: Microsoft Windows NT 6.0.6002 Service Pack 2
  CLR Version: 2.0.50727.4200 ( Net 2.0 )

ProcessModel: Default    DomainUsage: Multiple
Execution Runtime: net-4.0.30319
Unhandled Exception:
System.NullReferenceException: Object reference not set to an instance of an object.

Server stack trace:
   at NUnit.Util.ProcessRunner.Load(TestPackage package)
   at NUnit.Core.ProxyTestRunner.Load(TestPackage package)
   at NUnit.Util.RemoteTestAgent.AgentRunner.Load(TestPackage package)
   at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Int32 methodPtr, Boolean fExecuteInContext, Object[]& outArgs)
   at System.Runtime.Remoting.Messaging.StackBuilderSink.PrivateProcessMessage(RuntimeMethodHandle md, Object[] args, Object server, Int32 methodPtr, Boolean fExecuteInContext, Object[]& outArgs)
   at System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(IMessage msg, Int32 methodPtr, Boolean fExecuteInContext)

Exception rethrown at [0]:
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at NUnit.Core.TestRunner.Load(TestPackage package)
   at NUnit.Core.ProxyTestRunner.Load(TestPackage package)
   at NUnit.Util.ProcessRunner.Load(TestPackage package)
   at NUnit.ConsoleRunner.ConsoleUi.Execute(ConsoleOptions options)
   at NUnit.ConsoleRunner.Runner.Main(String[] args)

However, if I do this as a two-stage operation

[path to]\ClassLibrary1.dll [path to]\UnitTestClassLibrary1.dll /x coverage.xml /r

to instrument the code and create a coverage file with zero visit counts throughout -- and then run nunit separately with rest of the command line

[tool path to nunint 2.5.4]\nunit-console.exe /framework=net-4.0.30319 [path to]\UnitTestClassLibrary1.dll

this then fills in the visit counts as expected.

We have code coverage for .net 4 -- at least under some conditions! And without having to rebuild any tools apart from coverage.exe!

Later yet (refrigerator logic) -- of course when working coverage in two passes, I don't need to do the fiddle to change the settings for the AppDomain. And I don't need to have coverage built in the same .net version as I want to run nUnit in, either. I can use the code as synched from Googlecode built under .net 3.5 even for .net 4 codebases; all I have to do now is fix the problems with running it over F# code.

Links for March

Programmatically detecting screen-readers.

SSL Man-in-the-Middle certificates.

.net 4:

• Changes to events.
• Workflow migration tool.

Reactive Framework:

• Drag and Drop using Rx and WPF in F#.
• Introduction in Javascript.
• JQuery integration.

Continuation passing in C#.

A fluent wrapper for System.IO.File/Path/Directory types.

TypeMock test lint.

Links for 9-Oct

Exploding Software-Engineering Myths.

Dark Corners of IronPython

Gtk# for .Net developers

Tell don't ask.

Smart Stubs with Rhino Mocks.

Functional

• Matthew Podwysocki’s Generically Constraining F# Parts One, Two and Three
• M<’a> Lib (F# and C# LINQ) Monads Library
• Functional Programming for Everyday .NET Development.

Web

• Free CSS Toolbox - Free CSS Validator, CSS Formatter, CSS Compressor
• Starting with @font-face.

C# under the covers

In an earlier exploration of debug build code patterns I unearthed bits of debug assistance built into simple field-backed getter properties in C#.

That same pattern -- evaluate to a temporary, and then unconditionally branch to a return -- also shows up in non-property methods that return a constant literal or a field. With purely empty methods, there is no such intermediate value to expose to the debugger, so the method just looks like

i.e. a block with a Nop and a Return (with no expression).

The canonical stub left by Visual Studio implementing an interface looks like

which resolves to

i.e. a block with a Nop and a Throw (with a Construct expression).

Nop and a Throw alone will catch any method that simply throws as its one and only operation (even if the exception is passed in from somewhere else); if you want to determine what is being thrown, then dissecting the expression would be required.

In writing a simple static analysis tool, identifying anything that just throws -- and has no decision making logic -- is sufficient for my purposes; for anyone tempted to put massive lambdas in the throw expression, testing for the Construct may be worthwhile.

F# -- separating coverage sheep from goats

The recent flurry of F# related posts have achieved a first little goal -- the FxCop rule to separate out developer written code truly uncovered by unit tests from the extras that the compiler adds in that I was mentioning earlier is now roughly working.

More details on my projects blog.

Links for 7-Sep

F#

• Composing functions.
• Events and asynchronous workflows (series).
• Custom monads.
• Delayed Maybe monad.
  

Silverlight/Web

• Silverlight <canvas> emulator (follow-up).
• SVG for IE via Flash.
• Silverlight L10n.

PowerShell

• Exploring WMI.
• Formatted dumps of byte arrays.

Misc.

• Ultimate tool list for Windows.
• Exploit analysis plug-in for WinDbg post-mortem work.
• Winforms & WPF data UI without code or XAML.
• Visual Studio ignores some MSBuild conditional imports.
• Unit tests -- definition and best practice.

F# algebraic types under the covers

Consider the simple type

Innocuous, right? Well look at what Reflector has to say about the May 2009 CTP version...

Most of this is tagged as [CompilerGenerated] -- with some surprising omissions, like

which makes filtering out what is written vs what is generated for coverage analysis purposes rather more tedious than it might be. Arguably, in a unit test using the type, the latter two should get exercised anyway, but having the __DebugDisplay() method left over seems like a bit of an oversight.

---

By way of comparison, the Feb 2010 CTP yields

which is broadly similar, but differs in detail (such as the Tags values and the DebuggerDisplay attributes -- and the bug about __DebugDisplay not having the CompilerGenerated attribute is fixed.

F# unit testing and code coverage

And you thought it was going to be easy to get full coverage of functional code... Try this little example, assuming a Haskell-like Either<'a,'b>:

Feed it "dummy.txt" for the happy case, and something like "!!" for the bad case -- and you have an uncovered code-point for the untaken branch of the pattern -- i.e. rethrow any other exception type.

Then take that second case

from which NCover reveals we have hidden classes like <StartupCode$Tinesware-InfrastructureTests>.$Tests+ExpandFileWithJunkShouldFail@24 and <StartupCode$Tinesware-InfrastructureTests>.$Tests+ExpandFileWithJunkShouldFail@24-1 where the module is Tinesware.InfrastructureTests, the file Tests.fs and the Assert call is on line 24 of the file. And the first is 100% covered and the second is 0% covered.

The Feb 2010 CTP only generates 1 indirecting class on that line, according to Reflector, the @24 one and not the @24-1 one.

Code coverage measure for F# unit tests

The Feb 2010 CTP (1.9.9.9) has resolved this issue. I never looked at whether 1.9.7.8 did.

This one bit me the other day. Doing the usual post-build step for the unit test assembly

Ncover.console.exe nunit-console.exe $(TargetFileName) //reg

left me with no coverage data for my F# and a lot of "Failed to load symbols" messages in the log file for the F# assemblies in nUnit's shadow copy cache.

Turns out that whereas C# assemblies have absolute paths to the .pdb baked in, F# (May CTP) has project-relative paths; so the association was being lost. Short of running the F# code through an ILDASM/ILASM cycle, the quickest fix is to turn off shadowing:

Ncover.console.exe nunit-console.exe $(TargetFileName) /noshadow //reg

and NCover 1.5.8 will give you coverage metrics for F# too.

Links for 17-Aug

C# -- Four switch oddities.

F# -- Creating WPF dependency properties.

Compiling Python packages into Assemblies.

Better unit tests with Test.Assert(expression).

Functional thinking, unit testing, and the hole in the middle II

Following on from this post...

Summer, the vacation season, and time when things need to get checked in to "shelf" status while the developer goes on leave, and I get a massive review and patching up of bleeding edges to do, the latest of which has been an interesting experience.

The code I was presented with is tidily written, well commented, and is what a few years ago I would have considered to be good code -- the dev who wrote it is a competent engineer, but just not close to the leading edge. So there was an apologetic note about not having been able to test a layer of the code directly above the database abstraction interface (beneath which code is all generated and subject to separate testing).

Showing how to inject a mock of the interface was easy -- but that alone would have left a mountain of unit tests to write for anything approaching good coverage. The problem was that a lot of code was repeating the same mantra

Is the record set empty?  If so return an empty array
Create a new typed collection
For each record
  create new typed object
  initialize a type specific list of properties from fields in the record
  add object to collection
Create a new typed array of sufficient size
Copy from collection to array
Return array

which is just crying out to be factored into the generic recipe as above, with the initialization line passed in as a lambda or delegate. And that factoring alone should give an order of magnitude reduction in the amount and complexity of unit tests required, which should result in a less traumatic item in the in-tray ready for the return from vacation.

Hopefully this will also be a dramatic enough object lesson in the sheer niftiness of the functional hole-in-the-middle pattern.