Sunday, August 16, 2015

"Getting" Functional Programming

We all know there's something to "get" -- but is it one massive "Eureka!" moment, or is it a series of smaller "Aha!" steps? I think that it's the latter, but that most functional languages drop them in your lap in one undigested lump, because I had an odd introduction to the style.

Back around the turn of the century, I encountered the famous Why Functional Programming Matters paper; and while I’m a mathematician by background, that was always in Applied -- crunching differential equations or re-casting multiple integrals as Fourier transforms -- and I simply didn't get it. How could you do anything in your programs if your variables didn't vary? Then after some years, I encountered an article -- no longer on the web, alas -- suggesting that Erlang was well placed to be the "next Java", and my curiosity was piqued.

Some while later, I spotted Joe Armstrong's Programming Erlang on the shelves of the local Borders, so picked it up; and though the syntax and conventions of the language were bizarre and alien -- very "spiky" looking -- the mystery of how you did things with immutables was dispelled in the first couple of chapters. And it was revealed to be quite mundane -- you keep your scopes small (general good practise, anyway) to avoid running out of sensible write-once names, and you recurse or use callbacks rather than iterate.

So, despite its unrelentingly immutable nature (process dictionary aside), it turned out that Erlang had a very low barrier to the program after “Hello world!” Writing non-trivial programs (a few hundred lines of networking code) that accomplished and outperformed what had taken me thousands of lines in ‘C’ was easy. Yes, there were compilation errors along the way, but those were mostly simple syntax (putting the correct punctuation marks at the ends of lines), and I debugged at run-time as usual.

Really, despite the alien syntax and strict immutability, this felt like OO in the abstract -- you send an object process a message, and it responds in a Smalltalk-esque (or even Win32 message loop-alike) fashion. First class functions were just familiar callbacks with a different name, even in cases like a transformative map over a list. Indeed so OO is the paradigm that people talked about implementing Ruby in an "object → process" style on the Erlang VM, and then actually implemented such a near-Ruby.

Then I found out that this F# language was not Fortran.net, but a functional language on the CLR and gave it a whirl. And I felt like an utter dunce who “didn’t get it” for months; not helped by the lack of good introductory material -- like the original Foundations of F# book, which for the newcomer was frankly terrible on account of the volume of assumed knowledge it brought along.

In hindsight, the big shift was between Erlang's dynamic type model, and the static, inferred, typing of F#; the feature that means that a program that compiles will probably also work as well. Unfortunately, that also means that you have in a sense to debug your program at this stage, which is the massively unfamiliar bit. And it's not helped when your program fails to compile with a message that about as comprehensible as a C++ template failure -- it wasn't until F# for Scientists that I found my Wittgenstein's Ladder, a book that actually explained enough of the the types and type notations, and suddenly made sense of the bizarre arrow-filled compile failures (and API definitions).

Then, and only then, did the list-processing nature of LISP, and Haskell's mysterious monads, all suddenly become things that fall naturally out of the new style of thought. The extra constraint of type exactness is indeed what makes operations like bind to transform types a meaningful and necessary concept rather than being approached ad hoc.

The other shift, related to the debugging at compile time is that, like template metaprograming in C++, suddenly computation can happen as compilation -- a pure function with fixed compile time inputs can in principle be evaluated at that point. And where your program does take inputs, the way of looking at it approaches being an aggregation of functions where input comes in and results come out through a transformative process built of little stateless sub-transformations; rather than as a collection of stateful objects talking to each other about the input.

And at that point, I think what is being practised is functional programming.

Thursday, August 13, 2015

A cycling anniversary

Today concludes my first year logging my journeys on the Cambridgeshire Cycle Challenge website, over which span I've logged 601 journeys covering 4204 miles; supposedly 1372.0 kg CO2 saved (ignoring my increased respiration when burning the 155,532 calories it tells me I have spent doing so) and saving £989.69 on fuel (ignoring the costs of fueling me or running through consumables like chains).

The serious cases are probably all on Strava, because that performance puts me in the top few of those who use the site.

But then it's better to be a big fish in a small pond, than continuously broadcast one's recent whereabouts to the world.

Saturday, August 01, 2015

July cycling

Usual cycling round-up, with the end of the month numbers 12261.5 and 1932.6, for 162.5 + 252.2 or 414.7 miles total for the month, and 2458 year to date -- so down on last month due to generally miserable weather, and having to take advantage of what good weather there was at weekends to keep the garden under control.

Even so, all it took was a run of a few wet days for the caterpillars to devastate the new broccoli plants, some down to the stalks, while I wasn't watching.



Wednesday, July 01, 2015

June Cycling -- On the move II

Usual cycling round-up, with the end of the month numbers 12099.0 and 1680.4, for 148.5 + 310.4 or 458.9 miles total for the month, and 2044 year to date, only just short of a hoped for 2100. And that without doing any particular long excursions.

OTOH, work now having moved to the Science Park, the nicer -- as in less encounters with other road users -- routes to the office are between 1 and 5 miles longer than before, so there's been a systematic increase in my weekly mileage.



Sunday, May 31, 2015

Late May holiday cycling

We took the Bank Holiday week to go down to Netley Waterside again, after a couple of year gap, and for the first time, taking the folding bike I got last spring for holiday purposes, which meant that in just an initial quick ramble I could go further in either direction than the long walks I did back then.


The weather was fine, except for a wet Friday, and generally pleasant to be out in, even if at times through an uneasy balance of sultry heat and strong breezes, as for the expedition to the Sir Harold Hillier Gardens on the Bank Holiday Monday. In fact, the weather usually exceeded expectations set by the forecasts, such that I ended up acquiring more of a tan than I'd expected.

Having forgotten to pack a map, I ended up following the National Cycle Network 2 signs a lot, filling in the gaps by dead reckoning, especially in the places where the paved surface suddenly stopped, but being hemmed in between coast and motorway, that didn't leave too much scope for getting lost -- just scope for somewhat more ups-and-downs than I needed, grinding in bottom gear, as in the sub-optimal routes through the back doubles in Bursledon, and too much main A-road for crossing the Hamble. Next time, having discovered the foot ferry on the last way back, I might be able to cut out much of the main road and hills.

Hovercraft Museum, Lee-on-Solent

Hovercraft Museum, Lee-on-Solent

Also, lacking a map, I didn't realise quite how close I came to running out of road entirely, not realising that Portsmouth Harbour was that close.

D-Day plaque

D-Day plaque at the Rising Sun, Warsash

Many of the pubs I passed on my wanderings were rather sea-sidey, but the Rising Sun at Warsash was a more traditional rural pub, that overcame the handicap of being a Greene King house, and served most excellent sandwiches. Their take on chicken bacon mayo was not the vaguely unpleasant mush by the same name that were offered as packed lunches by the Revitalise team, but a coarsely sliced chicken breast, a couple of rashers, and a pot of mayo to add to taste on doorstops from a fresh soft bloomer loaf -- washed down with a pint of Rsing Sun bitter, an excellent lunch!

On the move I

Usual cycling round-up, with the end of the month numbers 11950.5 and 1370.4, with the belated arrival of the rains eating into my cycling days, for a disappointing 50.4 + 168.2 + 78 off-meter or 296.7 miles total for the month, and 1585 year to date, making 2000 for the half year a stretch.

Also, on Saturday 16th, I put my car past the 11,000 mark, so 1000 miles in 52 weeks, and under 2^10 in the year.


Thursday, May 14, 2015

F# under the covers XV -- functions, types, and what you see isn't what you get

Consider this innocent example


When run in a new interactive session, it yields (after trimming the SOAP baggage, and noting that _x002B_ would be an encoding of + and _x0040_ of @)

FSI_0002+clo@20
<SOAP-ENV:..>
<SOAP-ENV:Body>
<a1:FSI_0002_x002B_clo_x0040_20 id="ref-1" xmlns:a1=...>
</a1:FSI_0002_x002B_clo_x0040_20>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

FSI_0002+clo@20-1
<SOAP-ENV:...>
<SOAP-ENV:Body>
<a1:FSI_0002_x002B_clo_x0040_20-1 id="ref-1" xmlns:a1=...>
</a1:FSI_0002_x002B_clo_x0040_20-1>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

FSI_0002+add2@11
<SOAP-ENV:...>
<SOAP-ENV:Body>
<a1:FSI_0002_x002B_add2_x0040_11 id="ref-1" xmlns:a1=...>
</a1:FSI_0002_x002B_add2_x0040_11>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

FSI_0002+clo@20-2
<SOAP-ENV:..>
<SOAP-ENV:Body>
<a1:FSI_0002_x002B_clo_x0040_20-2 id="ref-1" xmlns:a1=...>
</a1:FSI_0002_x002B_clo_x0040_20-2>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

-------------------
FSI_0002+it@25
<SOAP-ENV:...>
<SOAP-ENV:Body>
<a1:FSI_0002_x002B_it_x0040_25 id="ref-1" xmlns:a1=...>
</a1:FSI_0002_x002B_it_x0040_25>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

FSI_0002+it@25-1
<SOAP-ENV:...>
<SOAP-ENV:Body>
<a1:FSI_0002_x002B_it_x0040_25-1 id="ref-1" xmlns:a1=...>
</a1:FSI_0002_x002B_it_x0040_25-1>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

FSI_0002+add2@11
<SOAP-ENV:...>
<SOAP-ENV:Body>
<a1:FSI_0002_x002B_add2_x0040_11 id="ref-1" xmlns:a1=..>
</a1:FSI_0002_x002B_add2_x0040_11>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

FSI_0002+it@25-2
<SOAP-ENV:...>
<SOAP-ENV:Body>
<a1:FSI_0002_x002B_it_x0040_25-2 id="ref-1" xmlns:a1=...>
</a1:FSI_0002_x002B_it_x0040_25-2>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>


val mhead : m:System.Object list -> System.Object list
val addN : n:int -> x:int -> int
val add2 : (int -> int)
val emitString : x:'a -> unit
val it : unit = ()

Where we observe that of the user input names, only add2 survives as part of the serialized name; and that the second instance of what looked like the same function is actually an instance of a different type.

Running the same code as a compiled .exe, we get

Program+clo@20
<SOAP-ENV:...>
<SOAP-ENV:Body>
<a1:Program_x002B_clo_x0040_20 id="ref-1" xmlns:a1=...>
</a1:Program_x002B_clo_x0040_20>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Program+clo@20-1
<SOAP-ENV:...>
<SOAP-ENV:Body>
<a1:Program_x002B_clo_x0040_20-1 id="ref-1" xmlns:a1=...>
</a1:Program_x002B_clo_x0040_20-1>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Program+add2@11
<SOAP-ENV:...>
<SOAP-ENV:Body>
<a1:Program_x002B_add2_x0040_11 id="ref-1" xmlns:a1=...>
<n>2</n>
</a1:Program_x002B_add2_x0040_11>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Program+clo@20-2
<SOAP-ENV:...>
<SOAP-ENV:Body>
<a1:Program_x002B_clo_x0040_20-2 id="ref-1" xmlns:a1=...>
</a1:Program_x002B_clo_x0040_20-2>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

-------------------
Program+clo@25-4
<SOAP-ENV:...>
<SOAP-ENV:Body>
<a1:Program_x002B_clo_x0040_25-4 id="ref-1" xmlns:a1=...>
</a1:Program_x002B_clo_x0040_25-4>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Program+clo@25-5
<SOAP-ENV:...>
<SOAP-ENV:Body>
<a1:Program_x002B_clo_x0040_25-5 id="ref-1" xmlns:a1=...>
</a1:Program_x002B_clo_x0040_25-5>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Program+add2@11
<SOAP-ENV:...>
<SOAP-ENV:Body>
<a1:Program_x002B_add2_x0040_11 id="ref-1" xmlns:a1=...>
<n>2</n>
</a1:Program_x002B_add2_x0040_11>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Program+clo@25-6
<SOAP-ENV:...>
<SOAP-ENV:Body>
<a1:Program_x002B_clo_x0040_25-6 id="ref-1" xmlns:a1=...>
</a1:Program_x002B_clo_x0040_25-6>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

where the names are similar -- and now the closure add2 actually contains that 2 as part of its body.

So, what's going on? Well, let's look at what we get when we decompile.


This is the debug version, the release version just moves the main program locals to be static members of the Program class, with no other relevant changes.

This shows us that our named functions are compiled as normal functions as we expect in C#, but the first-class function objects we pass around are actually indirections to those functions; and different ones at each site, even if they close over nothing, and so must be identical, even in the release build.

So, what does this mean?

Well, for one thing, type-needy serialization methods like DataContract aren't going to work well with functions (including lambdas inside of methods of innocent looking types), because the types are instance-unique. And deserialization is only going to be possible in the same binary as the serialization came from (or at least have the same functions on the same types, methods and line numbers), unless you do a fair bit of decompilation to provide a translation binding.

The latter is a potential pitfall -- if you implement a Windows Workflow in F#, and have objects persisted, things could work just fine within the first working version of your program; but processes that are hibernating mid-task while you're rolling out a new version can break when they wake because these secret types are no longer there by the same names.


Friday, May 08, 2015

Getting the types in an F# union type II -- the perils of relying on undocumented implementation details

Recalling this example --


It was adequate for the use case I had of it at the time; but then I tried a different union type, at which point a bug showed up



yields

FSI_0003+Card
System.NullReferenceException: Object reference not set to an instance of an object.
   at FSI_0005.it@24-1.Invoke(Type x)
   at Microsoft.FSharp.Collections.SeqModule.Iterate[T](FSharpFunc`2 action, IEnumerable`1 source)
   at .$FSI_0005.main@()
Stopped due to error

And, of course, that is exactly what I deserved for relying on the undocumented internals.

A quick decompilation reveals this to be because union cases that carry no data are actually concrete base type instances, bearing the tag number as data, like --



rather than being redundant empty subtypes.

So we need to tweak our function to be




and all is well. For the moment. Until the representation changes.

Thursday, May 07, 2015

Getting the types in an F# union type

This arose in the context of performing data contract serialization of a union type, and needing to feed the concrete types in as known types. As the FSharp.Reflection facilities only let you at the names of the union cases, it's necessary to rely on the secret knowledge that the union cases are implemented as nested subtypes of the abstract union type



Then we can serialize objects of the union type like



where we feed the list of types to be known to the constructor for the serializer.


Thursday, April 30, 2015

April cycling

Readings of 11900.1 and 1202.2 mean 144.7 + 262.2, a total of 406.9 miles, 1288 YTD, having cycled at least a kilometer every day this month -- #30daysofbiking. And it was only today that we really had April showers to dampen the excitement.


Friday, April 24, 2015

It's that time again, the simplistic "Who should you vote for?" quiz. At least it's not quite so out of whack as it was a decade ago, or even last time around.

Your UK Election 2015 quiz results


UK Independence
 
53
Conservative
 
26
Liberal Democrat
 
10
Labour -30
 
Green -48
 

You expected: UKIP

Your result: UK Independence


Tuesday, March 31, 2015

March cycling &c.

Readings of 11,755.4 and 939.98 mean 287.9 + 49.25 + 97 off meter or 434.1 miles this month, or 861 miles year to date; just over a hundred miles more than last year, though well less than a 1000 mile stretch goal.


Those 97 miles as above on the last weekend of the month -- the first day was pouring rain after Mickfield, and the last was into the wind most of the way, but at least I had a long ride on the middle day, and good hearty pub meals in between.

In flower this timeWinter lingers

This year, and 2 years ago at the same time

The month started and ended not very warm, but wet and windy; with some bursts of warm sun in between -- enough to sit outside at lunch on the 20th, after a very grey start that hid the eclipse. And enough to coax the last of last year's tomatoes to ripen as well. With tomatoes and leaf beet from the greenhouse, and then broccoli and romanescu from the vegetable patch, plenty of home harvest, along with the regular spring flowers -- and flowers on the October sowing of broad beans in patio planters as well.

Saturday, February 28, 2015

February Cycling and other notes

A short month, with snow, rain and other reasons to not stir from the house, so only up to 11,467.5, meaning 171.7 miles for the month, which is a dozen or so behind last February; but the 426.8 miles for the year to date is still 30 more than same time last year.

The slim crescent moon I saw at 06:40 on the 17th was actually less than two days before the new moon (23:49 on the 18th).

The greenhouse yielded a few belated ripe cherry-plum tomatoes, and some salad leaves, also from last year's crop.

Tuesday, February 03, 2015

January Cycling and other notes

Up to 11,295.8, meaning 255.1 miles for the month, in generally dry weather; a better total than last year's 213. February has started less promisingly; so I shall just have to see if I can stay ahead.

The mud and grime have chewed through the chain I put on at the start of December, so I got that replaced; probably need a thorough overhaul come Easter.

We finished the last of the apples from the garden by the last week of the month.


Saturday, January 24, 2015

Anime — 2014 in review

Slightly earlier in the year than the previous retrospective; and slightly more to talk about.

Of the carry-over series, Gatchaman Crowds and House of Five Leaves remained stalled; but we did finish Chihayafuru 2; meanwhile the continued adaptation of Mushi-shi is one we carry forwards to this year, being about 2/3 the way through.

Chihayafuru 2 continued from where the previous series left off, through the next national tournament, but this time, rather than being a cut-off, there were a number of other plot strands introduced that left the series ending feeling rather ragged. Very much, if you liked the previous series, you'll like this.

Crunchyroll also picked up

JoJo's Bizarre Adventure

which so far (the first 2.5 parts) has been a very competent adaptation of the manga -- apart from certain issues of nomenclature. For example:

... clearly pronounced "Santana".

With JoJo providing Saturday morning cartoons for much of the year, I usually wanted a B-feature, so picked at a number of other series -- Saki: the Nationals which lacks the charm of the previous series (or even the Achiga side story), starting a bit too fan-servicey, and not having anything near the level of magical girl SFX in the games. It's not helped by bringing on a whole parade of essentially disposable opponents who get partly fleshed out, but not enough to care, possibly due to the 1 cour running time. Again, like Chihayafuru, this one ended a little more raggedly than the previous series, in the gap between the preliminary stages and the semi-finals, rather than at the end of a tournament. Verdict: Mostly harmless.

I also started Hoozuki no Reitetsu, which for a Japanese comedy skit series actually manages a level of dry wit; that will probably carry on through the year as the back-up feature for the next season JoJo.

And then I also watched 2013's

Sunday Without God

a very pretty series, with an intriguing premise -- God quit fifteen years before, and the whole process of birth and death broke down, leaving the dead walking around unless settled by specially designated Gravekeepers. Ai, the main character, is one of these -- but she's unusual in that she was born after the Departure.

So there's an introductory arc, involving Ai's long lost father, and his friend Yuri Dmitriyevich (Romanized, inevitably as Julie!) that starts to explore the workings of this strange new world, followed by an arc, where they are all aboard the Mystery Machine -- the intrusion of a Yuri's VW camper van into the bucolic setting is a bit jarring -- and visit a town of the deceased, where the last human infant is kept in suspension.

So, threads have been laid and things may be happening when BAM! Japanese high-school out of nowhere (two different schools at that). And it all started so well!

I suppose, like Sora no Woto, despite the written French, the Roman aqueduct and similar furnishings, it was set in Japan all along (with an L/R misspelling being a plot point and July 14th just another school-day, it really couldn't be Provence). Such a shame, because it had had a strong and novel start, before it fell into the pit of
>school
>slice-of-(after)life

I also picked up Q1's Wizard Barristers, which had a reasonably strong beginning with a (MC excepted) adult cast, being a sort of magical Perry Mason set-up. Alas, it turned out less like the promise of the intro and opening few minutes, and instead of staying episodic, grew a plot, which in the next to last episode seemed to have been purchased at the expense of the entire animation budget. Notable for the lamest magical circle appearing over a city ever, compensated only a little by the sexiest diabolus ex machina (sasuga, Japan!)

JoJo took a break for Q4, so at the end of the year I was casting around for something more to watch in the dark evenings. I tried a bit of Hanasku Iroha, which generally got praise, but found myself more irritated than amused by the teenagers and the quirky cast of adults, and of the few other series that had received positive word of mouth, nothing really motivated me. So I settled for some trashy magical girl action in the form of

Fate/kaleid liner Prisma☆Illya

and Fate/kaleid liner Prisma☆Illya 2wei. Now, I'd become aware of the whole Type-Moon Fate/Whatever franchise since I started browsing /a/ about a decade ago, but it had always struck me as a handful of neat ideas, buried in a whole welter of tedious chuuni nonsense; Prisma☆Illya takes the setting, sifts out the teen angst and hilariously bad ero-scenes (for the purposes of mana transfer, you understand), and replaces it with an irreverant and self-aware (Illya gets her magical girl training from watching anime) mahou shoujo set-up clearly in the post-Nanoha otaku-oriented mode.

As a consequence it veers from the light-hearted (in the day-to-day scenes) to being exactly as serious as it needs to be (in the main fights), but without damaging the suspension of disbelief. Entirely brain-off mode fluff, but entertaining fluff.


And then for Q4, the anime of the year out of nowhere, having flown stealth, by advertising itself as
>school
>slice-of-life
and getting reactions accordingly:

so it wasn't until mid to late season that I picked up on the "and now the suffering begins" chatter (as well as the wheelchair moe), and got into

Yūki Yūna wa Yūsha de Aru

The eponymous Yūna-chan is a member of her school Hero Club (Yūsha-bu); and for half an episode, it is all about the ostensible slice-of-life business of the club -- things like entertaining small kids and finding homes for kittens, just like in the preview

until the special club app on their phones goes off, and suddenly they're projected into the Otherworld to fight existential threats, whose nature and magnitude only slowly become apparent.

If Prisma☆Illya was a post-Nanoha magical girl, then Yūki Yūna is a post-Madoka one, where the business of even being a magical girl is a fraught one; the Heroes of the Yūsha-bu are more like the Heroes of Glorantha, but in a system where mystical powers are purchased by sacrifice of more than just nebulous points of POW that can be earned back in the next quest. Indeed, I would characterise the series, in D&D terms, as clerical girl rather than magical girl, which is a refreshing change from the more usual "arcane science" or rootless "magic; I don't have to explain jack".

And it's here that the school/slice-of-life/"cute girls doing cute things" false-flagging actually benefits the series; the cute and comfy bits of normal life provide the necessary contrast for the cumulating horrors of their secret roles -- in its sentimentality, this series works, in a way that the rather mannered and sterile presentation of Puella Magi Madoka Magica failed to do, to make me care for the characters and their plight (I've also probably read too much Greg Egan to get worked up about characters executing out of an Ndoli jewel or equivalent).

The difference of approach goes even further than that -- PMMM put its Easter Eggs in the form of a substitution cipher, an abstract puzzle that mostly served to reveal the names of the Witches; YūYūYū saves its Easter Eggs for showing (sometimes in a "blink and you'll miss it" manner), not telling, what is going on and how things work, all the way up until the epilogue -- touches like one of the girls stopping to don a hachimaki in addition to her magical garb before going into final battle; things which add to the story rather than distancing the viewer from it to solve a puzzle.

Verdict -- you should watch it, you should buy it : Studio Gokumi seem to be getting an unexpected but well deserved hit out of this one, and original series (rather than yet another harem variation adaptation) like this should be encouraged.