The joys of RAII

A topic which, as Michael Caine might put it, "Not a lot of people know that". Alas.

One idiom which is almost unique to C++ amongst the languages in common use (for practical purposes this is defined as 'C' and its descendants -- C++, Java, C#; though it equally applies to Ruby or Python) is the concept of Resource Aquisition Is Initialisation (RAII). That is to say, when a resource -- whatever its type -- is acquired, it should be part of the initialisation of an object. Then, using the stack-based scope for variables, we can make the corresponding release operation happen naturally on exiting that scope, even during the stack unwinding following an exception*.

Managed languages (all those named above other than 'C') have separated out heap memory as a special (albeit frequent) case of resource allocation, handed the problem of tidying up to a garbage collector running inside the VM on which the code executes, and makes the deallocation of heap objects a non-deterministic process. For other types of resource, there are special idioms -- like IDisposable and the using construct in C# (or, if all else fails, try/finally).

In C++ all resources are treated equally, and all have deterministic points where clean-up can happen -- at the end of a {} delimited scope.

• Need to release memory -- allocate it as a std::vector<> (for arrays) or std::auto_ptr<> (for a singly owned single object) or similar smart pointer in the appropriate scope
• Need to release a COM object -- wrap it as a CComPtr<class T> template (ATL has a lot of these useful features kicking around for general Win32 native C++ programming). (Just be careful to release COM itself at the end of an enclosing scope, as destructors fire in arbitrary order).
• Need to release some other resource (HANDLE, HINTERNET, GDI Object...) -- find (or, if you have to, write) another class to contain the resource and whose destructor frees it.

The classic is, of course, the MFC CWaitCursor -- create one at the start of a long-running block of code, and the cursor will show an hourglass over your application main window, until it is restored in the destructor.

If you are in a position of having to write native C++, then rather than considering it the equivalent of working in the Dark Ages of manual memory management, look to the power that the language offers you to solve the more general resource management problem at a single point.

*Related to this is the reason why destructors must never throw -- throwing during a stack unwinding leads to program termination, no saving throw; only the chance to to some last ditch tidying up.

Responsibility Driven Design

The classical approach to object based analysis and design is the process of taking your problem or solution statement and picking out the nouns (objects) and verbs (methods). The objects that emerge are considered as data, with a bunch of associated methods to operate on those data members.

As a method of organising your problem into code this works; but it can make the code more cumbersome that it could be. A common point at which this sort of problem arises is the point where two or more objects interact through a given verb, and it's not obvious where to put the verb. And then when it comes to implement, side effects or necessary pre-conditions for doing whatever verb have to be considered.

And finally, when we are coding a verb -- an algorithm -- associated with one object, we are working at a level below the object design, where things start to become purely procedural again -- this can lead to code (which I am sure that we have all seen examples of) such as :

class ObjectUser {
    void doSomethingWithData() { // return and argument lists immaterial
        // do some stuff
        bool switch = someObject->getSomeState();
        if(switch)
            // do something not involving this
        else
            // do something else also not involving this
        // do some more
    }
}

where we inspect the state of one object to do something without making any reference to the calling object's state.

This sort of code, once it exists, is amenable to refactoring, moving the if/else code into a new method on the class to which someObject belongs. You may even find that after doing this, the whole need for the public getSomeState() method may vanish. Even if someObject is of a class we don't own, specialising or wrapping it can still be a good move. We have a whole book (and web-site) on how to do that with code.

But, wouldn't it be nice to get the same effect ahead of time, during the analysis and design stages?

Responsibility driven design is one technique that can help with this.  Rather than the historic data+algorithm approach for looking at objects, the approach is to look at objects for their roles within the system and their responsibilities in making it work. It also helps us focus not on the static anatomy of the system (such as inheritance paths), but rather on the dynamic behaviour of the objects in their context, and emphasises the encapsulation part of object design.

In this approach an object is defined by what it knows, and what services it performs. Those services may be performed by passing parts of the tasks to other objects that this first object knows about; but the clients need not know about that. All they need to see is an interface, describing the services.

Thinking about or objects this way, we can take a rough partition of the system into objects -- perhaps a noun and verb model -- and start to assign roles : does this object know things? coordinate activities? perform services? interface with things beyond the system? Then we can perform what is in effect a dry run through the model, based upon known use cases. This will show up those cases where an object may actually need to know things not initially available to it.  It will also reveal where objects are being asked to do too many different things (and occasionally, those doing too little to justify themselves).

Objects that have more than one focus of responsibility are candidates for being broken up. Tasks that are not strictly related to their responsibility should be moved to objects for whom they are better suited.

After a couple of iterations -- which may be done on the fly, while running through the use cases -- the changes should settle down; and the web of interactions simplified.

With your objects defined by their roles (interfaces), and the unnecessary interactions untangled, the design is also readied for test driven development -- you have interfaces for mock objects to follow, and have reduced the amount of work that a mock will have to perform.

Related links

http://www.wirfs-brock.com/Design.html -- Rebecca Wirfs-Brock's site, pretty much the last word on the subject

http://www.cs.colorado.edu/~kena/classes/6448/s05/lectures/lecture04.pdf and http://www.cs.colorado.edu/~kena/classes/6448/s05/lectures/lecture05.pdf -- two lectures (presentations in .pdf form) as a high-level overview of Responsibility Driven Design

AD FS + Windows Integrated Authentication = Trap for the unwary

There is a quirk in the use of AD FS in its default intranet mode that may come as a surprise to the unwary user.

By default, AD FS sets up the Federation Server to take Windows Integrated authentication; out of the box, it also installs client certificate authentication, but you have to actively enable that by editing the root logon page at  /adfs/ls/clientlogon.aspx

<%@ Page language="c#" AutoEventWireup="false" ValidateRequest="false" %>
<%@ OutputCache Location="None" %>
<% Context.Response.Redirect("auth/integrated/"+Context.Request.Url.Query); %>

to do something other than redirect to the Integrated authentication page.

Now, the interesting thing about Windows Integrated authentication is that having authenticated explictly, your browser is already doing single sign-on (SSO) on your behalf every time you touch an authenticated URL. 

So, having authenticated once to the Federation Service, you are henceforth silently re-authenticated for the rest of the browser session.

"What has this got to do with AD FS?" you may be asking.

Well, AD FS is not only an SSO mechanism; for applications spread across many hosts, it also offers a single sign-off capability.

So let's talk our way through what goes on when all of these are happening at once

1. You go to an AD FS protected application
2. It redirects you to the Federation Server
3. That does integrated authentication, popping up a dialog box at the browser
4. Having logged on, you're redirected back to the application
5. Later, you log off that application -- AD FS directs you back to the Federation Server
6. The Federation Server sends you a page that removes its session cookies, and contains images whose URLs are part of AD FS on the application, so they can remove the session cookies
7. Now you go back to the application page, and--
8. AD FS redirects you to the Federation server which--
9. Silently reauthenticates you and sends you back to the application
   

Step 9 may come as a surprise to the unwary -- unless you have some obvious indication in the web application (such as a last sign-on time), the log-off appears not to have happened.

This is not a bug -- it is an inevitable consequence of an old SSO model with no sign-out capability being used to bootstrap the new federation system; and will happen with any silent re-authentication scheme (such as the alternative authentication mechanism provided in the install, which uses client certificates).

When re-authentication is not silent -- explicit form driven authentication at a Federation Service Proxy, for example -- step 9 is blatantly obvious. This may lull the user into complacency.

Links for 19-May

More catch-up…

JavaScript -- the Big Divide -- a presentation

Remove Duplicate Rows From A Text File Using Powershell

Code Access Security and Bitfrost -- can Code Access security be made usable enough to work?

Powershell on Rails

Six cool things you can build with OpenID

Learn C# 3.0 - the easy way

SQL Server 2005 Security Best Practices

Punching holes into HTTP.SYS

Top 6 List of Programming Top 10 Lists

Our Dirty Little Secret

Wonder of the When-Be-Splat -- very nice compact Ruby idiom

Higher-order Messaging -- a the next logical step after higher-order functions

Cross-browser scripting with importNode() -- abstracting away browser dependencies for AJAX style applications

Yet Another JavaScript Library Without Documentation™ -- From the man who brought us the ie7scripts for CSS in IE5 & 6 -- a patch for current browsers subtly different DOM implementations.

Decrypting CardSpace Tokens in partial trust

Reuse is not Usable

Going Commando -- Put down that mouse

Software Development as a Collaborative Game -- remembering the fun

Software Projects as Rock Climbing -- that sort of collaborative game

Machine tags and ISBNs  -- interesting application of folksonomy

Lightweight Unit Testing for C, C++ and C++/CLR

Probably the simplest tool for this purpose is MiniCppUnit -- no install needed, 2 source files, ~ 500 lines and all in cross platform C++. This tool was in fact generated as a response to quite how heavyweight the more familiar CppUnit is.

The portability of the framework is extremely useful when code is being developed for a non-Windows platform, but you would like the platform independent body of the code in a unified Windows build for monitoring, as well as on the other platform. The one downside as it comes out the box, though, is that it doesn't build under C++/CLR due to its use of native C++ exception behaviour (i.e. uses the fact that in C++ you can throw anything).  This wrinkle is simply fixed, however.  

Guarded by an appropriate #ifdef/#else/#endif, do the following

in the .hxx file,

incoporate the .Net framework with

#using <mscorlib.dll>

provide an alternative definition of the TestFailedException class:-

public ref class TestFailedException : public System::ApplicationException

and catch System::Exception^ rather than TestFailedException&

in the .cxx file

throw gcnew TestFailedException();

rather than just TestFailedException(); when building as C++/CLI .

Links for 18-May

A bit of a catch-up:

CardSpace and decrypting Tokens

An Approach to Composing Domain-Specific Languages in Ruby

Giles Bowkett: The Business Case For Firefox

Bitwise Magazine:: What’s Wrong With Ruby?

New mailing list: HTML 5 Help

swfIR: swf Image Replacement

Five Principles to Design By

Debugging a Service on Windows Vista

The One Important Factor of programming languages

Lazylist implementation for Ruby

What Colour do you like your Objects? Pink or Blue?

Actions, Not Words

Are Web Interfaces "Good Enough"?

Your Code: OOP or POO? 

Creating User Friendly 404 Pages

Graphic designers misunderstanding Web standards

XSS bestiary

Remove empty lines from a file using Powershell.

Why <video>?

W3C's new HTML blog

Web Typography Sucks -- presentation and links to web typography resources.

Alpha release of Adobe's Apollo cross-platform runtime.

Keep your cookies straight when using ADFS

Is Your Software Team Sticky? -- do they share a coherent vision?

Primary Keys: IDs versus GUIDs

IE 7 does not resize text sized in pixels -- or any other absolute size *sigh* : Squelching what might become an urban myth

The hidden delights of Unit testing

One of the things that has left this blog light of content of late is having been provided with internal blogging, wherein I try to enlighten my colleagues. Often the entries on that blog are collations of links to and through other blogs I read. Some are from experience. Like this one.

One part of the current development project has brought home to me quite how much how serious unit testing results in cleaner code -- and that the closer you strive for 100% coverage in the testing, the more incentive there is to write that clean code.

The closer to 100% you strive to get, especially with a coverage tool (such as gcov) that does branch coverage rather than just line coverage, the more it squeezes your code. At the most brutal, the more code you have, that means the more tests there are to write to cover it all -- the incentive is there to make the code tighter, just to reduce the amount of work to do for completion.

Much of the 'C' code being written contains routines that are explicitly each a little state machine. As such the structure of a routine is along the lines of

• check preconditions
• determine "state"
• "switch" on the current state
• tidy
• return outcome

Some of the precondition checks are assert() but others cannot be (so will include an exit-on-failure); and the switch may not be a simple flat one -- some cases may have sub-cases; and some might overlap, in a structure like

where a and b are independent.

However, even if you're not explicitly thinking of the code as a state machine as such, the routine structure is still quite generic.

Coverage testing

A set of unit tests can make sure that expected inputs map to expected outputs, both positive and negative; coverage helps tell you if you have "enough" tests. It answers the questions "Has all that code been exercised, yet?" (if not the related "if it doesn't get used, why did you write it in the first place?").

The first thing that a set of obvious positive tests will show are the bits that are difficult to reach. The obvious one is handling exception states -- and here automation and good mocks in the test framework, or your own wrapper to it, are essential. After all, exceptions are meant to be, well, exceptional, but here need to be generated on demand.

With those out of the way, the real difficult-to-reach corner-cases of the logic stand out -- and with only those to concentrate on, you're either faced with writing a lot of tests to reach them; or figuring a way to simplify the code so you don't have to.

It is often tempting to write code like this:

but, damn it, if arranging the case a and c is hard work, you don't want to go through the slog with b as well. Factoring out the special case goes from being something you could do, if you had the enthusiasm, to something you want to do, because it's less work than writing the extra tests. "Don't Repeat Yourself" becomes positively encouraged.

Coverage types

The code metric you use is important in how much benefit you can derive. For a first pass, NCover isn't too bad. But it only counts line visits, being as it is an instance of the profiling API for .Net. In particular if you have code like--

NCover will never show you that you're missing the case of zero and negative values of a. One of the up-sides of working in 'C' on a *nix platform is that that has meant that gcov is available. And that will take code like--

and distinguish between whether a or b triggered the do something -- 100% in NCover usually isn't more than 90-odd% in gcov

Squeezing out the logic

Here's a real example of code improvement in making the last step to 100% branch coverage

I had 100% in NCover; but gcov reminds me that I don't cover all the bases -- because have_token and need_token aren't independent variables : if you don't need the token, you should never have one. So, what to do when aiming for the 100% mark?

The routine here started in a state where I just enumerated all possible cases (there are more than just these), handling them individually in some sort of logical order. Now, the unit tests I already have provide me a framework to check that the code is still doing what I mean it to do when I refactor; so I can look at the code and see that what I have is actually of the form

or, more simply

refactor, re-run the tests and see that the simpler code is still right.

Similarly code guarded by an if clause, where the else is never executed under any input you can generate, perhaps because the assert() defined contract of the method or its callers enforces the constraint, can be simplified to and assert() of the condition and an unconditional block. And you get that better code because you've made yourself go the last little bit.

In the case above, user input could have, but not need, the cookie; we can't assert -- but we don't need to write (though we can) another test case to prove that is harmless, because that is just another flavour of the "else".

Decisions, decisions

When I'm doing stuff for my own interest and enjoyment, it's almost certainly client-side applications. And unless it's something that is very definitely targeted at a particular set of OS specific functionality, I like it to be cross-platform and easy for a naive end-user to run. This was a particular pull of Java — though it was soon shown to be very definitely “debug everywhere” when the two main browser JVMs (Microsoft and Netscape) interpreted the thumb-width of a scrollbar differently (inclusive or exclusive of the range — or “lol, box-model”). And the API, even for AWT, was far nicer than any of raw Win32/Win16, MFC or OWL, back in the day.

So then the progression went AWT to GWT (a light-weight widget set that had all the missing widgets from AWT) to Swing… and there it stayed. But that did mean programming in Java, which, over the years, I've come to find lives at the wrong level of abstraction : it doesn't afford the ruthless power that C++ does (templates, multiple inheritance), but without giving very much in return (generics are latecomers, and comparatively weaksauce; there's nothing like mix-ins; and everywhere a lot of the same scaffolding that C++ would require).

Growing dissatisfaction pushed me to look at native (as opposed to bytecode) toolkits, where I could get to use C++, eventually settling on FOX, over wxWidgets. The downside of FOX is that it was not built for localisation (too many strings baked in), and the event handling model seemed to fight the language compared with Java's X-style callback registration (wxWidgets lost on having what felt like a more opaque layout model). But than meant compiling on Windows and Linux, and having to tweak platform dependent system header files (and hoping that other *nix-like platforms would work).

The moving target that is C# provided a bit of a distraction — I'll learn it for work, but are all new languages going to be just warmed-over Java? It didn't matter that Mono was providing portable CLR GUI by stages, when the language (C#) had little appeal in and of itself. So it wasn't until stumbling into Ruby and Python in the last few months that I discovered some more shiny in terms of language.

But what to do for client-side applications? Python with wxWidgets? Ruby with FOX? and what about naive users? Jython 2.2 in stand-alone mode? Pity about JRuby's sprawling nature. But what about AllInOneRuby and its friends? I had just about come down on standalone Jython, when the whole silverlight business blew up.

So now I think I know my GUI toolkit. And while we only have IronPython out (and ported to Mono) at the moment, that both Python and Ruby should soon co-exist in the DLR means that it should be possible to do what works best for the application at hand. When an object is an object is an object, much fun should be available.

IronPython and PyFit

Problem — PyFit uses the parser module.

Workround —

• remove redundant import compiler from fit\TypeAdapter.py
• remove import compiler from fit\taBase.py
• remove seq_types, map_types, oper_types from fit\taBase.py
• remove _safeAssemble from fit\taBase.py
• remove _safeEval from fit\taBase.py and lose the ability to input non-scalar types (string being scalar); or replace its body with return eval(s) and lose the protection against wild input (sorta-OK if your FitNesse wiki is on an internal network).

Either approach breaks various of the PyFit unit tests, but does permit you to run IronPython 1.x directly, so long as you keep your cell types under control.

Special case - IronPython 2.0alpha1

This currently has problems with the type() function used in CellHandlers.isValid(). Unroll the in test as follows

which passes the unit tests.

I believe that the type() problem is known in IronPython 2.0α1, but for the record, a minimal test case, in case it persists into beta

yields

C:\Documents and Settings\Steve\My Documents\code\python>\IronPython-2.0A1\ipy.exe ipy2.py
Traceback (most recent call last):
  File ipy2.py, line 5, in Initialize
  File , line 0, in _stub_##14
  File ipy2.py, line 3, in inspect
  File , line 0, in ContainsValueWrapper##18
  File , line 0, in _stub_##19
SystemError: Object reference not set to an instance of an object.

More IronPython

Since my earlier post on the topic, I find that I've been picked up by the nifty aggregator site IronPython URL's.

At that point, I'd just started using the implementation as part of the build process for the project I'm working on at the moment — while the bulk of the choreography is handled by running devenv against the main solution, the interstitial work is done through the pre- and post-build events. In most cases, .BAT-file behaviour is enough to just invoke an executable or two with a simple command line, but anything with string handling or iteration now gets pushed into IronPython.

Why IronPython in the build? Well, given that we're building with DevStudio 2005, .Net 2.0 is already installed, so IronPython's xcopy-style install means that we can put into a project's tools folder without having to perform any change-of-enviroment on the build machines (a big no-no).

What sort of tasks are we doing?

Quite a variety —

• Doing everything that needs version stamping from creating AssemblyInfo.cs files from templates, through web.config references to the generated assemblies to the installers and merge modules.
• Doing pretty much everything in managing installer creation from driving Wix to create the initial single language installers, version stamping them, and then extracting the language transforms and embedding them into the master installer

From that base, I've now moved into using IronPython as a tool for helping our system test team put scripts together. Unfortunately even 2.0Alpha1 doesn't have an implementation of parser, so it can't be directly used with FitNesse and PyFit; but even so, it still makes a great utility for CPython scripts to invoke to get at awkward or tedious bits of the Win32 APIs like

• ACLs on files and registry keys
• the current thread identity
• Assembly full-names

or even less esoteric things like DLL versions (even though pulling those out of PE format directly isn't that difficult) -- and without needing to check that all the necessary CPython Win32 add-on libraries are present.

It has also proved useful for quickly putting together scripts to drive web applications, by giving simple script-level access to all the System.Net and System.Xml facilities.

 

tl;dr — IronPython for the win for its xcopy-style install and out-of-the-box access to Windows APIs.

Film — El Topo

Having heard about it first many years ago in the pages of A&E, I finally got a chance to see Jodorowsky's enigmatic film of the Weird West.

Starting off as what seems to be an ultra-violent Clint Eastwood style spaghetti western, it turns into Zen quest, a tale of redemption, and then starts the cycle off all over again.

My main thoughts were to wryly observe where sensibilities have shifted in the last 35 years; and that I'm not sure what the two women with male voices were meant to represent.

Film — Curse of the Golden Flower

Third of the sequence of visually stunning tragedies that started with Hero and continued with House of Flying Daggers.

Plot and counterplot amongst the Imperial Family wind up to a climax on the night of the Chrysanthemum Festival — the eponymous golden flower — to yield something like a Jacobean revenge tragedy with added ninja.

All style, very little substance — but what style it is.

[Now playing - Planet Rock]

Nature notes

Sitting out in the garden blogging, after having done another batch of garden work, and tidying up the garage. Still a lot of stuff for the tip, and more for jumble. Cherry blossom is falling in the breeze, carpeting the lawn like snow.

There's a lot of tidying to do after last year's total neglect, but when there are wanted plants in amongst the weeds, simple clear to the surface and suppress won't do, so it goes more slowly. And unexpected tasks like removing the suddenly expired clematis add to the work. At least there's a natural limit — stop when the green bin is full.

The frogs are appearing in the pond again, lurking at the surface; and there's at least one newt as well.

[Now playing - Planet Rock]

Anime — Hataraki Man

A rare thing, a josei anime, i.e. aimed at the 20-something woman.

Hiro is an editor at a weekly magazine that does news, comment, investigative journalism, and serial novels; possibly a little bit too masculine in her drive for her job for her colleagues : she gets called a “Hataraki man” — a working man — and her managing editor, on hearing she has date, wonders if it's with another woman.

But with all this work, she also has a problem with her lifestyle. She and her boyfriend never seem to have gaps in their schedules at the same time.

I just wonder if Shinji is a name like Kevin, with associations of being feeble; whether Ikari-kun has had a pervasive influence; or if this wimpish Shinji is just a coincidence.

In the usual way, to be thought half as good as a man, she had to do twice as well. And again, for her, this is not difficult…

This, in 11 episodes, is a slice of her life, its ups, downs, praise from and friction with, her colleagues.

Surprisingly compulsive viewing, for all that it is entirely mundane. — the Man Switch gag being a one-off.

And then there was one…

Friday night, Penny was walking a bit stiffly — she didn't seem to find any of my prodding her painful, so I didn't think it could have been serious damage. Perhaps another case of a bit of a muscle strain from getting a claw caught in the throws over the furniture. That had happened in the autumn, when I'd come hom to find her hanging like a bat from the side of one chair, and going into all sorts of muscle spasm when unhooked.

Saturday morning, I came downstairs soon after seven to see her floundering about on the kitchen floor, trying to escape from the fetters that were making her hindquarters useless, and front legs almost so. Unlike Lady May, who had just sat herself into a tidy little bundle and purred as she faded, after she had a stroke, Penny was clearly going to fight with all the strength left to her.

So, the unpleasant task of bundling her into a towel, into the cat-box, for the one-way journey to the vet on an emergency call-out.

And after, cycle into town for the shopping I was going to do anyway, and stop at Grantchester on the way back to get hammered.

More about Penny.