Snagging just the Windows Spotlight Lock-screen images

One of the annoying things about Windows Spotlight is that at times, it includes advertising junk in the screen text. Somewhat more annoying is when a picture comes up, you wonder what it's a picture of, but the screen text never shows.

There are plenty of articles out there which tell you how to capture all the OS image assets and leave you to manually sort through them. Noting that the lock screen images will be of recent date, it's actually much easier to automate the entire process in a few lines of PowerShell --

which pulls just the phone and desktop images into the current directory for you; this means you can then use the standard means of identifying images to attempt to answer questions like "Spain or old California?", "England or New England?"

Powershell Transcript cmdlets and secondary runspaces gotcha

So, I had some fun this past week, with a piece of code that, stripped to its essentials, looked like

which yields

Stop-Transcript : An error occurred stopping transcription: The host is not currently transcribing.
At line:1 char:1
+ Stop-Transcript
+ ~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Stop-Transcript], PSInvalidOperationException
    + FullyQualifiedErrorId : InvalidOperation,Microsoft.PowerShell.Commands.StopTranscriptCommand

with a transcript (again, stripped to its essentials) of

Transcript started, output file is ...
PS>$pool = [System.Management.Automation.Runspaces.RunspaceFactory]::CreateRunspacePool(1,1)
PS>$pool.Open()
PS># do stuff...
PS>$pool.Close()
**********************
Windows PowerShell transcript end

with nothing after the Close() showing up.

It turns out that, however you create a runspace, even if using an instance of a custom PSHost subclass and explicitly minting one through CreateRunspace(), the runspace will still get attached to an internal PSHost subtype, which couples it to a whole web of other internal and/or sealed types, eventually linking it to the transcription state of the overall PowerShell session. And when a runspace closes, it closes all open transcripts attached to it.

WTF FAIL!

Fortunately, there is one public API available to us that can sever this link, and one that makes a perverse sort of sense, after you've run through all the plumbing:

which finishes with

Transcript stopped, output file is...

and a transcript that looks like

Transcript started, output file is ...

PS>$pool = [System.Management.Automation.Runspaces.RunspaceFactory]::CreateRunspacePool(1,1)
PS>$save = [System.Management.Automation.Runspaces.Runspace]::DefaultRunspace
PS>try {
    [System.Management.Automation.Runspaces.Runspace]::DefaultRunspace = $null
    $pool.Open()
    # do stuff...
    $pool.Close()
    # do more stuff...
}
finally {
    [System.Management.Automation.Runspaces.Runspace]::DefaultRunspace = $save
}
PS>Stop-Transcript
**********************
Windows PowerShell transcript end

because it turns out that, via a long chain of indirections, it is the -- fortunately thread-static -- global default runspace which is the thing that contaminates our intended-to-be-isolated worker environment.

Now, it would be understandable if runspace construction were to directly use the default runspace as a prototype, but it's nothing so obvious. It actually comes in via the UI object that is tenuously attached to the runspace reaching out to the default runspace. That's not so good, and speaks of excessive internal coupling.

Checking metrics on the assembly System.Management.Automation 3.0.0.0, we can see that it is indeed highly internally coupled, with a relational cohesion of 7.22, which is a level that is not so much coherent as positively incestuous. So while I didn't spot any other obvious booby-traps waiting to be sprung, I'm sure there are others that will rise up and bite the occasional edge-case.

PowerShell -- dynamically typed, except when it isn't

Consider

PowerShell is dynamically typed, so it should just work, right?"

Wrong. The output goes

System.Xml.XmlDocument
System.Xml.XmlDocument
Cannot convert value "23" to type "System.Xml.XmlDocument". Error: "Data at the root level is invalid. Line 1, position 1."
At line:5 char:3
+ $y <<<<  = 23
    + CategoryInfo          : MetadataError: (:) [], ArgumentTransformationMetadataException
    + FullyQualifiedErrorId : RuntimeException

So you can make static typed values, but can't even tell by inspecting the object whether it's static type or not. Combine this with the loose scoping that makes separate scopes nigh impossible (unless you want to write a proper closure, when it doesn't work), this makes little local scratch variables a lurking menace in any non-trivial script.

If only more people would get with the program and use F# as their .net scripting language.

Powershell -- cascading exit codes through nested shells

Finally resolved why I couldn't repro the issue in this cut down case; so, for future reference, just the real problem, and none of the dead ends:

I have a problem. I want to run a set of PowerShell scripts from an orchestrating PowerShell script, each in their own process so that I can relinquish assemblies that they've Add-Typed quickly, and thus allow them to be updated when I re-deploy the whole system. And those scripts can potentially fail for some reasons, and the failure can be soft (retry with different parameters) or hard (abort entirely).

Plus, I don't want to capture the (write-)output of the inner scripts as I want to watch their progress; which leaves me with the exit code as mechanism, which is enough for my need.

We can test this mechanism with a simple script that we can make fail on demand:

And drive it like

This results in

PS> $LASTEXITCODE
0
PS> .\OuterScript.ps1
output
host

01 December 2014 17:34:50
Inner script done
Got file code 0
output
host

01 December 2014 17:34:52
Inner script done
Got file code 23
PS> $LASTEXITCODE
23

However, if I add in one line (the one with the comment):

We get

PS> $LASTEXITCODE
23
PS> $LASTEXITCODE = 0
PS> $LASTEXITCODE
0
PS> .\OuterScript.ps1
output
host

01 December 2014 17:36:14
Inner script done
Got file code 0
output
host

01 December 2014 17:36:15
Inner script done
Got file code 0
output
host

01 December 2014 17:36:17
Inner script done
Got file code 0
PS> $LASTEXITCODE
0
PS> 

we get bitten by PowerShell's odd behaviour regarding automatic variables, which makes the local use of the name somehow refer to a different (and locally overriding) thing to what gets set by process exit -- another variation on the gotcha I hit a couple of years ago.

What I'd been hitting was just that explicit zeroing of the exit code (in a dense block of initialisations, where I'd not spotted it) had been happening, before a process launch and completion had created the "real" $LASTEXITCODE. Remove that line, leave the value unset on start, and it all just works.

Configuring Jenkins with PowerShell

No sooner do I say I have nothing technical to write up of general interest, than I spend a day stitching together pieces across the internet, because most Jenkins examples tend to be written to *nix or the JVM, and I'm on Windows where the admin tool of choice is PowerShell.

So, start on the Jenkins wiki page for Authenticating scripted clients, which tells you how to get your API key -- visit $(JENKINS_URL)/me/configure in your browser and look for the API token so you don't have to script your password (especially if you're using AD authentication on the server). Setting username/API Token as a Credentials object on a WebClient will just get you 403 errors, until you notice that the Groovy script example sets pre-emptive auth on its web client. The secret to HTTP Authorization and .NET WebRequest, WebClient Classes needs to be sought separately.

At this point you can GET from $(JENKINS_URL)/job/[job name]/config.xml, with DownloadString and cast to [xml] PowerShell style to read and modify.

Then you have to POST the modified XML back; but if you just do that with the Basic auth header, suddenly more 403 out of nowhere, until you read the small print about the Jenkins Remote access API about CSRF protection. When you get that and add it to your headers, it's now just a case of using UploadString to push the xml as xml.

Putting it all together we get

This appears to correctly preserve line endings as is, so you don't need to do anything non-default, equivalent to the --data-binary as you need for scripting with curl.

PowerShell 3 Start-Process MSBuild.exe -Wait hang

Having recently upgraded a Win7 machine that I had previously been using to check PoSh 2 compatibility (and had forgotten was not upgraded) to PoSh3, a build script launched from a PoSh prompt that did Start-Process MSBuild.exe -Wait -PassThru - ArgumentList ... started hanging on the wait more often than not, with Process Explorer showing the MSBuildTaskHost executable still lingering outside of the PowerShell process tree.

Killing that MSBuildTaskHost process usually unwedged the process, so it could go on to do the post build analyses in the rest of the script. Running with DISABLEOUTOFPROCTASKHOST=1 to avoid the separate process was worse -- there was no process around to kill and maybe release the PowerShell session when it hung.

Replacing the cmlet with a direct invocation & MSBuild.exe ..., and using $LastExitCode rather than the process object ExitCode property value seems to avoid the hang so far.

I don't know whether it's just MSBuild that's affected by this hang on wait, but I've not seen this behaviour elsewhere yet.

Five finger exercise -- a more complex cmdlet in C++/CLI

Moving on from the previous post, porting the main cmdlet example to C++/CLI -- along with adapting it to later PowerShell behaviour (the original code assumed that the provider path resolution would not throw on non-existent items, so some exception handling needed to be added ahead of the fallback check for the file existing), making it pass the MSFT FxCop rules for PowerShell (fixing the verb, class and namespace in the main), and allowing it to be localized (through resource files as well as through the PowerShell look-up mechanism), and so forth...

I let Visual Studio generate the class outline, so there's a slightly pointless split into header file and implementation, but that aside, it's only the little bits of baroque syntax that really distinguish it from the C# equivalent. So perhaps I shouldn't forget about the language quite so much.

Five finger exercise -- simple cmdlet in C++/CLI

I tend to forget that there are other .net languages than F# (oh, and the one that pays my salary); in particular, after some traumatic encounters with the original Managed C++, that C++/CLI is there and isn't too bad, especially when significant native interop is required (where even F#'s P/Invoke starts to drown in attributes). But for hardcore managed stuff...?

So I've been playing with some native code again, and was thinking to surface the functionality in PowerShell, since GUI programming is pretty tedious (I've set projects aside for half a year or more when the next step is to include a tree view control). So my first thought was to do that in F#, as I did that ages ago, back when you needed snap-ins and there were glitches in how F# compiled against that particular object inheritance tree.

But then, I thought, there's no need to be so polyglot, even if C++/CLI is a bit like Geordie when compared to Standard C++'s RP. So, to practice with the somewhat distractingly different keyword placement, here's the very first example from the old (and out of print) Wrox book

and, yes, snap-ins are old-tech, but they provide examples of how to override property definitions (in the simplest form, without mapping names). This has also been tweaked to be FxCop clean, including for the MSFT powershell rules.

Of course this was the point where I discovered that C++/CLI in VS2010 will only target .net 4, and I was writing this on my old Vista laptop, without space for back-version installs, and had to use the old familiar hack. But, that aside, it all went through most gratifyingly.

Next, to port one of the real examples, to see what other joy is involved.

Powershell 3 -- automatic variable gotcha

PowerShell automatic variables are useful for giving a way in to system state, but oh, how I wish that they had been sigilled rather than looking like user-definable names that one might actually wish to define. And worse when the behaviour of the variable name changes between revisions.

Here's an example that bit me the other day, in the wake of WinRM 3 being pushed out on automatic update into an environment that had previously been .net 4, PowerShell 2 only. It was a script a bit like this (but without the $host reference):

In Powershell 2

Major  Minor  Build  Revision
-----  -----  -----  --------
2      0      -1     -1
System.Xml.XmlDocument

xml            : version="1.0" encoding="utf-8"
xml-stylesheet : type="text/xsl" href="path\to\microsoft fxcop
                 1.36\Xml\FxCopReport.xsl"
FxCopReport    : FxCopReport

But in PowerShell 3

Major  Minor  Build  Revision
-----  -----  -----  --------
3      0      -1     -1      
System.Object[]
<?xml version="1.0" encoding="utf-8"?>
<?xml-stylesheet type="text/xsl" href="path\to\microsoft fxcop 1.36\Xml\FxCopReport.xsl"?>
<FxCopReport ...

All because $input is an automatic variable rather than a user definable name -- and, worse, one that has had some semantic change between revisions. Now, if it had been $$input with the extra $ being a reserved character, this sort of unwitting name clash could never have happened. And there's no real penalty in having to type $$host to get the PowerShell host data, or similar.

Updating the last vsvars32.ps1 you'll ever need for x64

I've been using Chris Tavares' ultimate vsvars32.ps1 in my powershell start-up script pretty much since it was first posted. But of course, it shows its age in that it assumes you're running 32-bit, given the registry path it uses.

Now, you can hard-code based on your hardware, or assume that anything you're using nowadays is 64-bit (except, of course, when you explicitly run the x86 version of powershell); and there are various ways of detecting current bit-ness. But the simplest way of doing things is to take the behaviour-driven style of detection used as standard in JavaScript and just write

or reversing the order of the two registry keys to taste.

There are of course alternative ways of getting the path we're eventually aiming at e.g. via environment variable like VS###COMNTOOLS where the ### is a number like 90, 100 or, nowadays, 110, depending on VS version.

HTML reporting for StyleCop

Based on the outputs from the earlier script; and using an appearance inspired by the output from the XSL report approach from codecampserver; but with the source in-lined into the report, and violation messages inserted after the affected lines. Each file (and the nested violation messages) are hidden by default, and can be expanded by clicking the file header or the source marked as violation (where the hand cursor shows):

N.B. Defined as it is within a PowerShell here-string, the jQuery script needs its $ symbols to be escaped to keep the PowerShell parser happy.

You will need to adjust the script to find the appropriate StyleCop output files from your build process, and create suitably named/located reports based on those files.

Adding the missing pieces to the Standalone StyleCop script

Rather than post the whole ~200 lines, just the salient bits so that the original can be tidied up (but without obscuring everything with obsessive error handling). Start by giving it parameters like

Find StyleCop in the default install location by

if (-not $StyleCopFolder) {
    $styleCopFolder =  (dir "$($env:programFiles)*\stylecop*" | Sort-object LastWriteTimeUtc | Select-Object -Last 1).FullName
}

Get the list of projects by either just the project, or using a regex on the project file from here: http://bytes.com/topic/c-sharp/answers/483959-enumerate-projects-solution

Find a Settings.StyleCop file by looking at the project folder and up, defaulting to one in the StyleCop folder; and then scan the C# files of interest by

and customise the output file as

Standalone StyleCop : a rough draft

Unlike FxCop, the StyleCop tool out of the box only comes with Visual Studio and MSBuild integration; there is no stand-alone command-line tool. Still, as it's now open-source, we can look at the workings of the MSBuild task and see how to drive it directly.

Here is a proof-of-concept PowerShell script which can be used as the basis for a proper command-line tool or script; and as a preliminary "how to" to write unit tests for your rules. Well, they aren't going to be proper unit tests, since they'd have to touch the file system, but they can crawl over test source files in a dummy project in the StyleCop rule solution.

And that's all there is to it, really.

Deleting files, keeping a few

Inspired by this post, how it works in PowerShell:

will delete all but the most recent 3 files ending in ".log"; and it would be the same sort of thing in any scripting language.

PowerShell and GTK-Server

Another little five-finger exercise, porting the example VBScript stdin driver to PowerShell. The only annoying thing is that the Start-Process cmdlet doesn't give you direct access to the streams, so we have to drop into .net to fire up the GTK server process.

Another PowerShell egg-timer (using events)

Recording a learning exercise for how PowerShell handles events, and how to communicate into event handlers

Note that every registered event must be unregistered (otherwise running the script leaves droppings in your PowerShell session in the form of those claims on events). The events that are handled -- the timer 250ms wake-ups -- are consumed by being handled; but the one that is merely waited on remains latched after triggering the wait to release and must be explicitly cleared.

Additional AV frills would be the same as any of the polling loop timer examples out there.

Computing cyclomatic complexity with PowerShell and FxCop

A measure of the cyclomatic complexity of a .net method can be gauged by counting the number of IL branch instructions that do nor branch to the next instruction, and which have a distinct target from any other branch -- this is essentially the algorithm used in NDepend 1.x to make the computation. The introspection mechanism of FxCop provides enough decompilation of the IL that getting the instruction type and offset, and the target offset of a branch. An actual FxCop rule, though feasible to write, however, would be less useful than one could hope.

For one thing, FxCop itself doesn't provide any convenient way of passing parameters to custom rules (to e.g. set a trigger threshold); and for another, any analysis is likely to be run over a debug build (DEBUG and CODE_ANALYSIS variables defined there to not contaminate the released code with [SuppressMessage] annotations), which is likely to have a different underlying complexity to a release build (this is especially true in F# debug builds where there are a lot of sequences that look like

which can be replaced by

which, as it turns out, is pretty much what the release build does.

A more configurable and flexible way of performing the operation is to drive the FxCop facilities from a script -- these days I'm doing a lot of my .net scripting in PowerShell, but it could equally well be done with IronPython or similar .net scripting language. The result looks like this:

and running it over a set of F# code shows that the reported complexity of the release build of a method halves (or more) what is reported for a debug build. I haven't made a comparison over an significant amount of C# as yet, to see how much complexity the compiler removes between the two configurations.

Harnessing the PowerShell command line parser in .net

One of the annoying niggles about the .net framework is the lack of an intrinsic command line parser -- yes, there's Mono.Options or the F# power-pack, but they're not always just there to hand. PowerShell is there on Win7 and up, and is likely to be there on machines with older OS versions in a suitably technical environment, So if you just want to whip up a simple command line tool, and you're not writing it in PowerShell, why not at least borrow its features?

First, we want to get the argument list as provided -- for this we have to get the semi-raw line via Win32 (semi-raw as the shell will already have processed escapes -- in PowerShell that means that backticks, semi-colons and double-dashes are significant):

This uses the PowerShell tokenizer to strip out the executable name. We also want to be able to describe what the command line parameters are like, and other usage text

Then we can build a PowerShell function that takes the parameters we've defined, and just places them in a hash : this is the meat of the exercise

For producing a usage description, we can ask PowerShell for one, flatten it to a sequence of strings, and then strip out the unwanted PowerShell-isms

Finally, we can parse a supplied command-line, writing a failure reason and usage info if things fail

A simple example driver would be

There are some quirks -- Mandatory positional parameters and arbitrary nameless options don't mix : the current example program if given a command line "-Val hello -N 23 -S a b c" will yield

"$args" -> [|"a"; "b"; "c"|]
"ValueArg" -> "hello"
"SwitchArg" -> True
"NextArg" -> 23

Make NextArg mandatory, and instead it goes

A positional parameter cannot be found that accepts argument 'a'.

NAME
    ConsoleApplication1.exe

SYNOPSIS
    This is the synopsis

SYNTAX
    ConsoleApplication1.exe [[-ValueArg] <string>] [-NextArg] <int32> [-SwitchArg]...

So, perhaps not industrial strength; but suited to gentle use when there's nothing else to hand.

What you see vs what you get : matching source and IL with Mono.Cecil and PowerShell

Following on from yesterday, looking at how Mono.Cecil pairs up debug information in the .pdb file with just the sequence points the IL -- which follows the approach taken by coverage tools such as dot-net-coverage. The code is much the same as before:

with the addition of the line/column data in SequencePoint ("null" if the values are not given, "0xfeefee" for the well-known compiler generated fake line number).

Launching PowerShell 2.0 into .net4 via F# Interactive or IronPython 2.7

With .net4 being 18 months old, and the Win8/PowerShell 3.0 being both in CTP and only for Win7+, it's a drag having to do things with .net 4 code when you really could do them faster and better in PowerShell. So, why not bootstrap ourselves from out of the box PowerShell 2.0 into the .net 4 world with a little help from another scripting language which is .net 4 aware? This saves messing about with environment activation variables, centralized registry settings or application config files, and takes advantage of the fact that PowerShell, like other scripting languages, can be hosted by another .net process.

The recipe is based around Bart de Smet's Option 2 – Hosting Windows PowerShell yourself.

Via F# interactive:

Via IronPython 2.7 (augmented to take command line arguments via the -Options parameter):

Since we can't pipe into Start-Process, the inner script is written to a temporary file; and then the initial PowerShell process waits so that it can safely delete it. If you would rather, it's reasonable to save the script to a constant .fsx or .py, replace $source with the path to that file, then the -Wait and Remove-Item -Force $source can be removed.

Replacing the fixed command line arguments for the F# interactive launched PowerShell with ones input to the script as per IronPython follows the same pattern. In either case, care has to be taken with getting your quotation marks right, and the IPy example is rather rough-and-ready. Besides, doing appropriate quotation parsing and escaping would obscure the real point of the exercise.

Note: Launching PowerShell directly from the .net4 command line launches PowerShell in the plain old .net 2 configuration; it is, alas, not sticky.

Note: These scripts are intended to give you a .net4 based interactive session; an alternative approach aimed at running individual commands can be found on Jason Stangroome's GitHub.