Monday, December 01, 2014

PowerShell -- dynamically typed, except when it isn't

Consider

$x = [xml]"<xml>text</xml>"
[xml]$y = "<xml>text</xml>"

$x.GetType().FullName
$x = 23
$y.GetType().FullName
$y = 23

PowerShell is dynamically typed, so it should just work, right?"

Wrong. The output goes

System.Xml.XmlDocument
System.Xml.XmlDocument
Cannot convert value "23" to type "System.Xml.XmlDocument". Error: "Data at the root level is invalid. Line 1, position 1."
At line:5 char:3
+ $y <<<<  = 23
    + CategoryInfo          : MetadataError: (:) [], ArgumentTransformationMetadataException
    + FullyQualifiedErrorId : RuntimeException

So you can make static typed values, but can't even tell by inspecting the object whether it's static type or not. Combine this with the loose scoping that makes separate scopes nigh impossible (unless you want to write a proper closure, when it doesn't work), this makes little local scratch variables a lurking menace in any non-trivial script.

If only more people would get with the program and use F# as their .net scripting language.

November cycling

The numbers for the month are 10749.7/745.6, plus 3 miles where I noticed the odo drop out or +145.0/+145.1 or 293 miles, so 3921 miles year-to-date, lower than it might have been due to a couple of days where I needed to work from home, and with most weekends being at home too; but starting to be affected by bad weather too.

So, 4000 will happen; but with more home-working on the cards, 4200 will depend on how much I manage to get out for rides in the country at the end of the month.

Saturday, November 29, 2014

Powershell -- cascading exit codes through nested shells

Finally resolved why I couldn't repro the issue in this cut down case; so, for future reference, just the real problem, and none of the dead ends:

I have a problem. I want to run a set of PowerShell scripts from an orchestrating PowerShell script, each in their own process so that I can relinquish assemblies that they've Add-Typed quickly, and thus allow them to be updated when I re-deploy the whole system. And those scripts can potentially fail for some reasons, and the failure can be soft (retry with different parameters) or hard (abort entirely).

Plus, I don't want to capture the (write-)output of the inner scripts as I want to watch their progress; which leaves me with the exit code as mechanism, which is enough for my need.

We can test this mechanism with a simple script that we can make fail on demand:

#InnerScript.ps1
param (
  [int]$code = 0
)

$here = Get-Location
try {

Write-Output "output"
Write-Host "host"
[DateTime]::Now

exit $code
Write-Host "wtf??"
}
finally {
  Set-Location $here
  Write-Output "Inner script done"
}

Write-Output "Inner script REALLY done"
exit 0

And drive it like

#OuterScript.ps1
$failWith = 23
$succedWith = 0

& powershell -NoProfile -File .\InnerScript.ps1 -Code $succedWith
$code = $LASTEXITCODE
Write-Host "Got file code $code"
if ($code) { exit $code }

& powershell -NoProfile -File .\InnerScript.ps1 -Code $failWith
$code = $LASTEXITCODE
Write-Host "Got file code $code"
if ($code) { exit $code }

& powershell -NoProfile -File .\InnerScript.ps1 -Code $succedWith
$code = $LASTEXITCODE
Write-Host "Got file code $code"
if ($code) { exit $code }

This results in

PS> $LASTEXITCODE
0
PS> .\OuterScript.ps1
output
host

01 December 2014 17:34:50
Inner script done
Got file code 0
output
host

01 December 2014 17:34:52
Inner script done
Got file code 23
PS> $LASTEXITCODE
23

However, if I add in one line (the one with the comment):

#OuterScript.ps1
$failWith = 23
$succedWith = 0
$LASTEXITCODE = 0 # Make sure we start clean

& powershell -NoProfile -File .\InnerScript.ps1 -Code $succedWith
$code = $LASTEXITCODE
Write-Host "Got file code $code"
if ($code) { exit $code }

& powershell -NoProfile -File .\InnerScript.ps1 -Code $failWith
$code = $LASTEXITCODE
Write-Host "Got file code $code"
if ($code) { exit $code }

& powershell -NoProfile -File .\InnerScript.ps1 -Code $succedWith
$code = $LASTEXITCODE
Write-Host "Got file code $code"
if ($code) { exit $code }

We get

PS> $LASTEXITCODE
23
PS> $LASTEXITCODE = 0
PS> $LASTEXITCODE
0
PS> .\OuterScript.ps1
output
host

01 December 2014 17:36:14
Inner script done
Got file code 0
output
host

01 December 2014 17:36:15
Inner script done
Got file code 0
output
host

01 December 2014 17:36:17
Inner script done
Got file code 0
PS> $LASTEXITCODE
0
PS> 

we get bitten by PowerShell's odd behaviour regarding automatic variables, which makes the local use of the name somehow refer to a different (and locally overriding) thing to what gets set by process exit -- another variation on the gotcha I hit a couple of years ago.

What I'd been hitting was just that explicit zeroing of the exit code (in a dense block of initialisations, where I'd not spotted it) had been happening, before a process launch and completion had created the "real" $LASTEXITCODE. Remove that line, leave the value unset on start, and it all just works.

Friday, October 31, 2014

October cycling

The numbers for the month are 10604.7/745.6, plus 95.14 miles on cycling holiday or +171.8/+146.0 or 412.9 miles, so 3628 miles year-to-date, passing last year's total on the morning of the 19th, in continuing mild and dry weather, and with 4200 now looking like a good stretch goal for the year.

The numbers were slightly helped by the incentive of the Cambridge Cycle Challenge, in which I came second for the district as an intermediate rider, and was also in second place in the work team. Never before had I been paid for making that extra ride to the pub! But it was also helped by continuing balmy weather -- T-shirt and shorts even in the middle of the month.


That included the first day of an autumn cycling holiday, which was another Constable Country pub-crawl, ambling to the Tattingstone White Horse (under new management, so a bit less looking like a bikers' pub) for lunch, the Bakers' Arms at Harkstead for a quick one, then the Sun Inn at Dedham to sample some of their wares (not so impressed by the Lushingtons) before settling down for the night.

Monday the weather was not so nice, but I decided to take a different tack this time, stopping at the Red Rose in Lindsey Tye (which has a cycle stand!), then across to the Maglia Rosso (which I'd not tried back in July) for some cake and hot chocolate, en route to which, the front dérailleur jammed in middle gear; before making the usual detour to the Cherry Tree in Glemsford (sampling the rather potent Redemption Brewery Urban Dusk), then a different route to Lavenham (avoiding Shimpling in favour of Long Melford, via a B road that emerges just before the 30 limit).

Evening sun

Evening sunshine, Sunday

Offton, village sign

Village Sign

Bizarre house sign

How Bizarre!

Between mechanical problems, and the remnants of Hurricane Gonzalo, bringing strong winds and heavy showers, I then wimped out, and got taken back to base rather than spend a few hours being driven before the wind.

Thursday, October 02, 2014

Configuring Jenkins with PowerShell

No sooner do I say I have nothing technical to write up of general interest, than I spend a day stitching together pieces across the internet, because most Jenkins examples tend to be written to *nix or the JVM, and I'm on Windows where the admin tool of choice is PowerShell.

So, start on the Jenkins wiki page for Authenticating scripted clients, which tells you how to get your API key -- visit $(JENKINS_URL)/me/configure in your browser and look for the API token so you don't have to script your password (especially if you're using AD authentication on the server). Setting username/API Token as a Credentials object on a WebClient will just get you 403 errors, until you notice that the Groovy script example sets pre-emptive auth on its web client. The secret to HTTP Authorization and .NET WebRequest, WebClient Classes needs to be sought separately.

At this point you can GET from $(JENKINS_URL)/job/[job name]/config.xml, with DownloadString and cast to [xml] PowerShell style to read and modify.

Then you have to POST the modified XML back; but if you just do that with the Basic auth header, suddenly more 403 out of nowhere, until you read the small print about the Jenkins Remote access API about CSRF protection. When you get that and add it to your headers, it's now just a case of using UploadString to push the xml as xml.

Putting it all together we get

# Create client with pre-emptive authentication
$webClient = new-object System.Net.WebClient
$webclient.Headers.Add("Authorization","Basic "+
  [System.Convert]::ToBase64String(
  [System.Text.Encoding]::ASCII.GetBytes("$($UserName):$JenkinsAPIToken")))

# fetch CSRF token as authenticated user
$crumbURL = $JENKINS_URL + "/crumbIssuer/api/xml"
$crumbs = [xml]$webClient.DownloadString($crumbURL)

# set the CSRF token in the headers
$webclient.Headers.Add($crumbs.defaultCrumbIssuer.crumbRequestField, $crumbs.defaultCrumbIssuer.crumb)

# GET the job configuration (you don't actually need the CSRF token for this
# but it's better to get that token once at the top in case you want to do multiple
# operations e.g. setting parameters on many jobs, in this script)
$configURL = $JOB_URL + "config.xml"
$config = [xml]$webClient.DownloadString($configURL)

# do whatever transformation you need to the XML

# POST back
try {
        $webClient.Encoding = [System.Text.Encoding]::UTF8
        $webclient.Headers.Add([System.Net.HttpRequestHeader]::ContentType,"application/xml")
        $webclient.UploadString($configURL, $config.OuterXml)
} 
finally ## reset headers for client re-use if you're processing many jobs
        ## (or .Dispose() the client if you're done)
{
      $webClient.Headers.Remove([System.Net.HttpRequestHeader]::ContentType)
}

This appears to correctly preserve line endings as is, so you don't need to do anything non-default, equivalent to the --data-binary as you need for scripting with curl.

Tuesday, September 30, 2014

September Cycling

Well, this wasn't supposed to be just a cycling mileage log, but for the past few months, when it's not been keeping house and garden on an even keel, my side-project time has been eaten up with at-work side projects, variously involving Perforce streams, Jenkins, Windows Workflow 4, and a heavy dose of PowerShell to glue it all together -- nothing that has turned up any real revelations in and of themselves, and the results are automation tools mainly of interest to immediate colleagues, rather than of the "did you realise that you could...?" that I've hit in the past.

So, meanwhile, on two wheels, the numbers are 10432.9/599.6 or +162.1/+250.3 or 412.4 miles, in what was a dry month, continuing warm to the end, and would have been more had I not been down with a cold in the middle of last week, so 3215 for the first three quarters of the year. That makes it likely that I'll pass last year's total of 3441 miles sometime around mid October, and 4000 still a reasonable end-of-year target.

I got the new bike serviced at 550 miles on the odo, when I was able to get away with just a new chain this time; so time to start checking for wear about the 1250 mile mark and weekly thereafter.

Sunday, August 31, 2014

Southport holiday

We went back to Vitalise Sandpipers again (missing the Flower Show, as there weren't any twin rooms available that week when we went to book).

Sandpipers from the Marine Way bridge

And of course I took my bike -- both for little shopping trips and for longer ambles.


For a change, on Tuesday, I took the train for the first few miles -- buying a day-return ticket for Blundellsands and Crosby for the princely sum of £4.08 (for which I thank the local ratepayers!), getting off at Freshfields, and allowing myself a choice of stopping points on the way back.

The new furthest point of my cycling expeditions

Starting by picking up the trans-Pennine trail, which was at worst narrow packed gravel with a few puddles, I picked up NCR 62 until it reached the canal, and then went the other way, until the point where the cyclepath switched banks made a sensible place to tend back to base. Taking the main drag back for the first mile or so was no problem -- the very wide street and lack of serious traffic all helping. Then a little track through the park to join up with the way I'd come last time.

Actual art

This time, the tide was out, and I actually noticed the Crosby beach attraction


Improv

Modern art committing slow suicide

and then headed back along the coast path until I was back at Freshfields, which in terms of convenience was the obvious place to get the train back from.

On the way, the strong wind had knocked my bike over when I'd propped it up at one stop, and it had snapped the shaft of the clip-on bar-end mirror I'd been using; so it was fortunate that on the train back, as we pulled out of Ainsdail, I noticed the Mecycle shop next to the station.

So, there being a dearth of obvious cycle shops in Southport, I headed there on Thursday for a replacement, or as near as possible, and they were ever so helpful fitting a different, smaller, but actually more convenient little mirror. Then I went on for a bit more of an amble -- encountering as it happened another couple of cycle stores, whose stock of mirrors was even more meagre -- until the rain started, just a drizzle as I headed back from Formby, but starting to come down heavily as I reached Ainsdale.

So I stopped at Mecycle again, this time for the cafe, and a lunch of lentil stew, until the rain went over, and clear blue sky took its place.

Despite a forecast that had painted Monday as the only wet day of the week (there were actually just a few sprinkles of wet, at least locally -- there were substantial puddles on the way on the southward loop on Tuesday), the real rain waited until Friday, when I didn't get any miles in at all, because of it. Wednesday was a no-miles day too, because in the good weather, we made the organised trip to the Lakes (two hours each way, and a couple of hours there).








Views from Lake Windermere

August cycling

With 81.9 miles off meter; odos 10270.8/349.3 gives 157.2 + 136 -- a disappointing 375.1 miles for the month, and a total of 2803 for the year. Looks like I shall be passing the 3000 miles mark (last year's stretch goal) around mid-September.

Wednesday, August 13, 2014

Almost breaking even

So after 4 years of owning my new Smart, it's on 10151 miles, while the WAV is on 10365 (since March '10); meanwhile my old bike is on 10231 (since fitting the odo in early Jul '10), and the new one at 256 miles after the reset.

So that's 1243 and 2515 miles driven respectively, a total of 3758; adding up the miles for the bikes, and counting all the off-meter miles (other bikes and various meter failures), that comes to 3747 miles pedalled, which is only about an average day's worth of miles less.

It's been true a while now that in terms of hours, cycling is my primary mode of transport; now we're getting to the stage where it'll be true in terms of miles as well.

Later: Recalling another 7.5 miles done off-meter puts the cycling total to 3754 -- which is within margin of error the same as driving.

Saturday, August 02, 2014

The Outer Limits

Following on from the previous post -- having done the Ely to St. Ives run, I filled in the last big arc on this map


showing the extreme points I've reached on day cycling trips from home, essentially all of it during the last five years.

The limitations of terrain, including the limited crossing points of rivers and major roads, mean that there aren't really any more significant big targets to aim for now :(

Update 7-Sep-2014: what is probably the last long ride of the season, to close that wedge to the east; while doing so, I found the junction at Brinkley and going past the pub that is now the Brinkley Lion familiar; just that it was the first time that the pub had been open as I went past. It is possible that in the past I went a little further that the new line on the map, but probably not too much.

Thursday, July 31, 2014

July cycling

So 10113.6/213.3 + 84 miles cycling holiday + 15 other miles or 468 for the month, and 2428 for year to date.

While Le Tour was circling Leeds, I did a circuit of the Fens (Ely and St. Ives, for a 67 mile run) to put the odo on the old bike over the 10000 mile mark by the end of the 4th year of use.

Reach village green

Reach village green

Hitching post again

Hitching post again

Adventurers' Fen
10,000 miles

Round the clock at Wicken

Hundred foot river at Earith Bridge

Earith Bridge

Busway terminus

Start of the good bit

Windmill Bridge

Windmill Bridge

That was somewhat hard work, in a bit over 7 hours including lunch and tea breaks -- but without stopping at any of the pubs. The cycleway from St. Ives was welcome at that point of the run, smooth as well as level, so I could open up to a good cruising speed despite the legs starting to complain.

Then in not so nice weather in mid month, a quick break based in Bury


Stanton Mill

Stanton Mill -- a working post-mill

Bardwell Mill

Bardwell Mill

Pakenham Mill

Pakenham Mill

where it was almost but not quite raining for both days, so deciding me against going to the Anglo-Saxon village at West Stow, or stopping at the cycling cafe at Hawstead Green (maybe next time I do the Constable Country run).

And apart from that, one bike service and no days rained off when I wasn't on holiday, so petty much as many miles as reasonably fit in a summer month.

Monday, June 30, 2014

June cycling

The headline numbers are 9957.5 and an estimated 480 (as the new odo factory reset itself after rain on Friday, basing on 16 commutes and a couple of side-jaunts during the month on the new bike) for a total of 454 miles for the month and 1960 year to date.

As a stretch goal, 4000 for the year seems more achievable than it did earlier in the year.


Sunday, June 01, 2014

Cycling provision fail

Southport is generally well furnished for cyclists -- but there are exceptions:


View Larger Map

The only reason for the gutter would be for a short-cut for bikes -- but there's that sharp-bend sign in the way!

Cycle provision fail

FAIL!

Sufficiently awful that I stopped and posed the bike to show the corner of the sign just above where the stem meets the handlebars!


May Cycling

Recording the numbers gets complicated now, as I have a new commuter bike for getting to work (with the old one relegated to shopping use) -- so 9838.7/144.98; or 313 miles around the local area (including a spin out to Sandy on the early Bank Holiday). Then add 68 miles around Southport and a couple more unmetered, for 383 miles for the month, and 1506 miles year to date.

The Bank Holiday run

The tower at Sandy

The tower at Sandy

The ford at Sutton

The ford at Sutton

The Southport cycling



From Crosby looking to Bootle

From Crosby looking to Bootle

Turning point

Turning point

And to show how technology has moved on in the 15 years since I last bought a bike, the new one has an aluminium frame (so is lighter even with a chunkier frame) and hydraulic disk brakes -- and has cut about 5min off the commute each way.

Just need to remember to look at the chain on the old bike around the 11,000 mark.

Saturday, May 17, 2014

Some numbers

This is for my benefit more than the world's...

The car dash was showing 10000 on the odo and 10:00 on the clock this morning as I drove to collect my bike from being serviced.

And the large circuit (partly off-road) of the village is 1.67 miles.