Playing with (almost) the latest C++ features -- groundwork

Having had my interest in playing with native code reawoken by the new C++11 features, the first thing I went to look at was portability. One of the advantages of managed code -- JVM or CLR -- is that the VM handles portability for you and the code can be built pretty much anywhere; with native code we have to see what the compilers have in common.

I've been using VC++2010 on Windows as having many of the "big rocks" for the new standard, while being backward compatible onto OS versions before Win7 (unlike the VC++11 compiler and its runtime); and for *nix-like systems, I have cygwin and debian squeeze... Well the distro support for these is a bit behind-hand (gcc versions 4.5 and 4.4 respectively), whereas gcc 4.7 is now out with quite a broad coverage of the new standard. While 4.5 has a good chunk of the new stuff, 4.4 doesn't -- in particular, it doesn't have the new lambda syntax. So, it's build from source time to get an upgrade to a sensible version there, which means I might as well go to the latest and greatest on both platforms...

Building gcc 4.7 from source

debian

Fortunately there are some handy instructions out there which can be used as a baseline for debian. Following them, I found that I needed to tweak how I built GMP to fit PPL's requirements, by modifying the configure step to be:

CPPFLAGS=-fexceptions ../../sources/gmp-5.0.4/configure --prefix=$PROJECT_DIR/output/ --enable-cxx

before PPL would go through happily. The --enable-cxx is required for the PPL ./configure stage to run through, the CPPFLAGS=-fexceptions is optional, but it avoids a make-time warning about possible unwanted runtime behaviours if you don't.

ClooG also needed a CPPFLAGS=-I$PROJECT_DIR/output/include CFLAGS=-L$PROJECT_DIR/output/lib on the configure line to find GMP.

In the gcc build, as well as pointing at ../../sources/gcc-4.7.0/configure it's also worth taking the advice from the MacOS build instructions and only selecting languages of interest to you i.e. to play with new C/C++ there's no need for Java or Fortran, at a considerable saving in time.

Then it's a matter of just adding soft links from whichever g*-4.7 files to the unadorned versions, and

export LD_LIBRARY_PATH=~/gcc-4.7/output/lib/
export PATH=~/gcc-4.7/output/bin:$PATH

to your .bashrc or equivalent

cygwin

Cygwin is more fun -- I've not yet managed to get that to build all the way through with the loop optimization libraries. When you get to PPL, you find you also need to go back and re-configure GMP with --disable-static --enable-shared, as explained in the friendly manual, to build the shared library version. However then when building gcc, we get a mismatch with the earlier libraries in the configure stage, where it just stops:

checking for the correct version of gmp.h... yes
checking for the correct version of mpfr.h... yes
checking for the correct version of mpc.h... yes
checking for the correct version of the gmp/mpfr/mpc libraries... no

It is possible that if you start by building PPL and CLooG with shared library GMP in a first pass, then build the rest starting with reconfiguring and building a static GMP it will work, but life is too short. The MacOS build instructions didn't use the PPL/ClooG/graphite libraries either, so we can do this to configure gcc instead:

$ ../../sources/gcc-4.7.0/configure          \
>     --prefix=$PROJECT_DIR/output/    \
>     --with-gmp=$PROJECT_DIR/output/  \
>     --with-mpfr=$PROJECT_DIR/output/ \
>     --with-mpc=$PROJECT_DIR/output/  \
>     --program-suffix=-4.7            \
>     --without-ppl --without-cloog    \
>     --enable-languages=c,c++

which sits and cooks for quite some time to get you the new compiler build.

Linkbait: fixing cygwin "mkdir foo mkdir: cannot create directory `foo': Permission denied"

I got into a state where I had this error, which other people have seen, after having tried to trash the build output of one of the failed PPL/CLooG attempts from Windows Explorer, where anywhere under in my home directory and down it was rejecting mkdir with "mkdir foo mkdir: cannot create directory `foo': Permission denied". Having spotted a tangentially related mailing list message about this sort of problem happening on network shares and there being ACL related, I tried the following and it worked to clear things up:

1. Start a PowerShell console as Administrator window
2. Run Get-Acl a folder (like /tmp) which you can mkdir in in cygwin (this will be %cygwin_root%\tmp where %cygwin_root% is where you installed cygwin)
   $acl = Get-Acl C:\cygwin\tmp
3. In Windows Explorer set yourself Full Control on all the affected folders -- %cygwin_root%\home and %cygwin_root%\home\%USERNAME% at least
4. In the PowerShell, Set-Acl on each folder you've just frobbed with the saved ACL object
   Set-Acl C:\cygwin\home $acl
   Note that the Set-Acl call may take considerable time (tens of seconds) to execute when it gets to the really problematic node and has to roll permissions down.
   

SCons -- Death to makefiles

Since I last did native code seriously at home (c. year 2000), I had discovered the very handy MiniCppUnit tool as a nice light-weight unit testing framework, so of course I went and fetched a copy to be going on with. This time, curiosity prompted me to wonder "WTF is this SConstruct file anyway?" and now when I opened it, I immediately recognised that it was some form of Python script, and wondered what sort of Python based make system this might be.

It was simple enough to find where it came from -- http://www.scons.org/ -- and looking at the user guide, I felt that it is much more intuitive system than makefiles (admittedly there's not a high barrier there), and far less cluttered than declarative XML based systems like Ant or MSBuild; so I'll be using that for my *nix builds -- it works very nicely for doing things like running unit tests as part of the build e.g.

MiniCppUnit -- Building it under modern C++

Just like I had to patch it to build in C++/CLI, I needed to make some changes to MiniCppUnit to get it to build clean under VC++2010, the out-of-the-box gcc 4.5 on cygwin 1.7.x and gcc 4.7 debian squeeze with -Wall -std=gnu++0x or -Wall -std=c++11 on respectively. First MiniCppUnit.hxx:

92c92
< #if _MSC_VER < 1300
---
> #if defined(_MSC_VER) && _MSC_VER < 1300
93a94
> /* Without the "defined(_MSC_VER) &&" this code gets included when building on cygwin 1.7.x with gcc 4.5.3 at least */
204c205
<  static void assertTrue(char* strExpression, bool expression,
---
>     static void assertTrue(const char* strExpression, bool expression,
207c208
<  static void assertTrueMissatge(char* strExpression, bool expression, 
---
>     static void assertTrueMissatge(const char* strExpression, bool expression, 
304c305
<    catch ( TestFailedException& failure) //just for skiping current test case
---
>    catch ( TestFailedException& /*failure*/) //just for skiping current test case

and the corresponding signature change in the .cxx file:

108c108
< void Assert::assertTrue(char* strExpression, bool expression,
---
> void Assert::assertTrue(const char* strExpression, bool expression,
122c122
< void Assert::assertTrueMissatge(char* strExpression, bool expression, 
---
> void Assert::assertTrueMissatge(const char* strExpression, bool expression, 

Of course there may be other things lurking to be scared out when I ramp up my standard warning levels to beyond the misleadingly named -Wall (when you have -Wextra, formerly -W, provided to switch on a whole bunch more including spotting signed/unsigned comparisons, before getting onto the really specialized ones) and switch on -Werror to force everything to be really clean.

3471.5

So In the first quarter, I managed 269 miles on my own bike -- a lot more cycle commutes in March than I had expected -- plus 75 on the hire bike for holiday -- putting me 1/3 the way to my target of 1000 for the half-year.

Early cycling holiday

To use up some carry-over leave from last year, I booked the last possible use-by dates, and decided on the cyclebreaks Tudor Treat package, so that even if the weather was awful, I could at least be pampered. As it was, having been their last customer out last year, I was their first out this -- and doing so, I caught the last day of the glorious early summer, before a return to more normal conditions. So on the first morning in glorious sunshine I set off, and let the miles roll past:

View 29-Mar-2012 in a larger map

Setiing out about 09:30, I didn't decide on what route I'd take until getting to the Red Lion at Martlesham, where I turned right, and made the same sort of loop around Ipswich as I did last time I was there -- only this time, I paid more attention to the cycle-route signs when having to clip the corner of the built-up area (where motorists were queueing to full up cars and petrol cans), and then down the only really busy piece of road, the B1113 through Sproughton, which combined up-hill with constant traffic both ways.

I went through the old village at Copdock, rather than slogging up the old dual carriageway, then through the obvious route to the White Horse at Tattingstone, getting there just after midday; by which time it was warm enough to have taken off the brushed cotton shirt I'd started with. And after a pint -- too early yet for lunch (despite the tempting menu) after a huge breakfast -- warm enough to strip down to shorts as well. From there, the cycle-path around Alton Water began shortly after, then to the Suffolk Food Hall for a light lunch.

The return route was pretty much forced into backtracking some of the way in until turning north at Belstead; then having made mistakes in the in-bound route last time, I followed the more direct way to the city centre and then followed the cycle-route signs out East -- no long spells of pushing, only the first little bit around that multiple roundabout -- and out along the country lanes to the hotel.

Sum total, 45 miles; so I was a trifle creaky when having freshened up, I went down for a much needed pint at the hotel bar.

View 30-Mar-2012 in a larger map

Friday opened sunny, and was forecast warm, so I started on the loop down to Felixstowe, to look over at Harwich; but cloud soon bubbled up and socked in completely, with a raw northerly wind, so by the time I got onto the unpleasantly busy road south of Kirton, I thought this was no longer a day for going to the coast, and took the only way out that wasn't back, aiming for the Dog at Grundisburgh for a light lunch; and then took a rather indirect amble back to the hotel, for just over 30 miles in total, before setting off home to beat the evening rush.

Although the second day was more the weather I had been expecting, cold northerly aside, it was really nice to be out and enjoying the countryside in spring -- cowslips, daffodils and primroses running riot in the verges, and daffs in most patches of woodland.

C++ then and now

A long while ago (turn of the century) I wrote a little Win32 utility to drag and drop files to get their MD5 and SHA-1 hashes -- useful for verifying downloads. And when I composed the hex-string hashes into a std::string, it looked almost 'C' like, thus (repeated once per hash):

with <string> being the only bit of C++ in sight, in fact

After the recent Going Native C++11 bash I decided to dust it off and revise it using Boost and the TR1 features in VC++ 2010 (which has most of the C++11 goodness all by itself). So now that section looks like:

which gets called in another lambda that is for_eaching over a vector of tuples containing hash contexts.

The code size has gone up (down if you count discarding hand-written MD5 and SHA1 for the CryptoAPI -- which adds its own error-handling bloat); but the increase includes half a dozen little RAII classes, and code to discover a better monospace font than SYSTEM_FIXED_FONT, albeit from a hard-coded list of Inconsolata, Consolas, Lucida Console and finally Courier New.

My immediate reaction is that the provision of lambdas now makes the standard algorithms actually usable -- no more functor classes remote from the point of use -- and I can program it in the more functional style that I'm now used to.

Summer's lease

Today, even before the clocks go forward for summer time, it was warm enough to sit out in the garden until the sun got too low, at about 3pm -- or 4pm in tomorrow's money.

Still, there's bound to be rain at Easter.

C# under the covers

More of what the compiler does without you realising it, only this time in a C# sample.

The absolutely simplest sort of method, which you can clearly completely cover by calling it once. Right?

That's what I thought until I tried running OpenCover over code rather like this, and it told me I had only covered one of two possible branches.

So naturally, I go "WTF?", and wonder whether it's the return or throw alternatives out of .First that it's alluding to and to be certain, crack open ILSpy, and see that first line expands to

which actually involves first caching the delegate corresponding to the lambda, if it hasn't been already; or using it on subsequent calls.

The release build is much the same; it lacks the initial nop, and the meaningless jump-to-next-instruction at offset 0x23.

Move the constant string out into the method, and make the lambda close over it

and the caching goes away : a new instance of ClassLibrary2.Class1/'<>c__DisplayClass1' gets created every time, debug or release. And with it goes the branch.

Spring keeps springing

As usual at this time of year, high pressure usually means socked in 8/8 cloud, or fog after a clear night; and with sunset around 6pm, a prompt return from work and/or some remote working after dinner when cycling. Despite that, a clear night before the full moon meant good conditions for cycling to and from dinner; and since then I've managed three days (out of seven possible) cycling to work when it wasn't foggy and/or drizzling.

The garden continues to progress - primroses now all in flower, not just the ones in the sunny places that have been going a month or more; viburnum still, a solitary daffodil, can crocuses and snowdrops giving way to the early dwarf tulips. I've even had the mower out a couple of times now.

Still a lot of April chores ahead -- lawn work, clearing weeds from the various beds; but I've made a start on some of that already, mainly last Sunday when we had the first warm day where I didn't need a jacket going to the garden centre for supplies.

Spring has sprung

After a foggy start, yesterday was bright and mild; a perfect day for spring to open. Let us hope that the month does not exit in too leonine a fashion.

All winter, when the cats have complained about being kept on strictly measured rations, as the vet says they could do to shed a few ounces, I've told them to go and catch something if they want more to eat.

So yesterday, they celebrated spring by leaving a pile of entrails on the kitchen floor, as token that one at least had found a reasonably sized rodent to supplement the kibble.

My reactions to Stroustrup's keynote on C++11 style

I finally got around to watching this on channel9. Now, I don't do much C++ myself these days, since most of the code at work is in C#, and the language is rather high in ceremony (e.g. header files) for out of hours playtime, compared with the broad spectrum of widely available and popular languages there to choose from these days; but I do from time to time, so it's always good to see where things are going.

Interestingly, for a C++11 talk, a good 60% of it was about how we could drag the usual run of C-with-a-cpp-file extension code kicking and screaming out of the 1980s and into the early 21st century. In-language lambdas, async/futures and move constructors were touched upon, but where real world examples were talked about, it was showing how actual live code wasn't taking advantage of features that have been available in compilers for the best part of a decade, or longer, to make code simpler, more maintainable, and faster -- often considerably faster. And when I have to do some C++ and have been greeted by files full of almost-C-in-files-with-cpp-extensions, written by people who are writing native most of the time, I know that feel, bro :

>2012
>Not using RAII
ISHYGDDT

The "a lot of people don't know that" moment -- when doing random access insertion or deletion from a sequence, the random access part dominates the insertion/deletion reshuffling to such an extent that whatever the collection size, use a vector rather than a list.

The big takeaway -- despite the folklore, going to low-level 'C' style constructs is premature optimization. The machine code generated by the higher level constructs is often exactly the same as for the hand-crafted lower level code; and at times can be better, as the compiler has more context for optimizations.

Heh, almost makes me want to do something with the language again, if only to keep my hand in.

Signs of the times

A few weeks ago, we went to the Arts Theatre to see Cheek by Jowl's production of the overwrought Jacobean tragedy 'Tis Pity She's a Whore. Entering the auditorium, we were presented with a sign warning that the production contained nudity and bloody violence. Fair enough, given that these days when a character murders and dissects his sister, that sort of thing naturally occasions buckets of the old Kensington Gore for audiences oft rendered blasé by the various species of torture porn that runs through cinemas without any shrieking of "Look -- a video nasty!!".

Today, we went to see the Marlowe Society's production of A Midsummer Night's Dream; and the warning this time was that there would be smoking on stage.

Is this what we have descended to, that when a production decides that Oberon shall open the second half lazily waving a cigarette as he observes the chaos he has occasioned, that we have to warn the sensitive of the fact?

Some people need to grow the hell up.

Dear Diary,...

January was a rubbish month for cycling -- a total of 15.2 miles logged. And February being winter, maybe not so much better.

After having passed 4000 miles on the way back from last holiday, the car went past 5000 coming home from work last Thursday (on a day when it was -4 all journey in, and still sub-zero for going home time).

Cold weather at the end of last week came of a sudden, and I'd forgotten to replace the bubble-wrap over the pond; so Saturday morning that was 2" thick with ice, humped under the plank that was supposed to protect against gentle frost, more than enough to take my weight, but I managed to cut a belated air-hole -- most of the way through with a pruning saw, then hot water for the last bit. That let water gush out from below, and then I could chip away until there was a reasonably sized hole.

I also harvested some sprouting broccoli ahead of the snow that fell after dark as a warm front came over.

Sunday morning was snow-clearing so that Karen's morning call had somewhere dry to park -- but even then the snow was starting to thaw on the car. The amount of snow meant that it didn't shift in a hurry -- the close was quite deep in slush this morning; and I was glad I packed a shovel, as I needed to dig myself clear several times in the car-park at work.

Coming home early to work remotely for the afternoon, I didn't need to dig past what I'd already cleared, but the traction warning was going all the time I was in the car-park. The rest of the drive was just wet -- much black ice if it frosts tonight -- and the close almost clear.

Forecasts this morning were also rubbish -- the Met Office said it was foggy; The Weather Outlook (Forecast issued: 06/02/2012 08:06:10) that it was -5C when Cambridge airport latest METAR reported

Time: 07:50 UTC
Visibility: 8000 m
Clouds: Broken sky, at 800 feet above aerodrome level
Temperature: 1C

HTML reporting for StyleCop

Based on the outputs from the earlier script; and using an appearance inspired by the output from the XSL report approach from codecampserver; but with the source in-lined into the report, and violation messages inserted after the affected lines. Each file (and the nested violation messages) are hidden by default, and can be expanded by clicking the file header or the source marked as violation (where the hand cursor shows):

N.B. Defined as it is within a PowerShell here-string, the jQuery script needs its $ symbols to be escaped to keep the PowerShell parser happy.

You will need to adjust the script to find the appropriate StyleCop output files from your build process, and create suitably named/located reports based on those files.

Garden rubbish

Apart from a brief spell around last weekend, and into the start of the past week, where temperatures driving to work were in the -3C to -5C range, winter hasn't really arrived -- we've just had an extended autumn.

The cold finally did for the the antirrhinum flowers, and the last signs of life in the sunflowers, which are still serving as bird-feeders; but we still have overwintering escholzias; self-sown poached-egg plants from last summer's flowering already in bloom, primroses since before the end of last year, and now snowdrops and the first of the early crocuses.

I have at least been able to tend to a number of the tidy-up chores : pruning roses, fuchsias and apple trees; clearing out the straw from various dead annuals, and the last year's growth from the crocosmias and lemon balm.

And of course there are weeds already -- including a fine old crop of winter wheat from the porridge-like mass of spilt grain out of the bird feeder, as well as the usual stubborn perennials, misplaced self-seedings and the like. Which all means that the green bin is continually being put out full, even in the putative low season.

If it were drier, there'd even be grass clippings, as the mild wet weather is encouraging the lawn already!

Adding the missing pieces to the Standalone StyleCop script

Rather than post the whole ~200 lines, just the salient bits so that the original can be tidied up (but without obscuring everything with obsessive error handling). Start by giving it parameters like

Find StyleCop in the default install location by

if (-not $StyleCopFolder) {
    $styleCopFolder =  (dir "$($env:programFiles)*\stylecop*" | Sort-object LastWriteTimeUtc | Select-Object -Last 1).FullName
}

Get the list of projects by either just the project, or using a regex on the project file from here: http://bytes.com/topic/c-sharp/answers/483959-enumerate-projects-solution

Find a Settings.StyleCop file by looking at the project folder and up, defaulting to one in the StyleCop folder; and then scan the C# files of interest by

and customise the output file as

Standalone StyleCop : a rough draft

Unlike FxCop, the StyleCop tool out of the box only comes with Visual Studio and MSBuild integration; there is no stand-alone command-line tool. Still, as it's now open-source, we can look at the workings of the MSBuild task and see how to drive it directly.

Here is a proof-of-concept PowerShell script which can be used as the basis for a proper command-line tool or script; and as a preliminary "how to" to write unit tests for your rules. Well, they aren't going to be proper unit tests, since they'd have to touch the file system, but they can crawl over test source files in a dummy project in the StyleCop rule solution.

And that's all there is to it, really.