Wednesday, July 08, 2009

IronPython for build scripting

Following on from the earlier post with the snippet about generating GUIDs -- and covering a good chunk of what's been occupying me since...

I have ended up in charge of the build system for the current project at work. This started out with one framework that used a number of custom projects inside a solution to perform unit test, FxCop and coverage analysis, with a lot of magic happening in post-build steps, including direct calls to Wix command-line utilities. Another team had developed a better separated MSBuild-based system, which split out things like the analysis and installer building from the assembly-building solution. We can argue the merits of taking the unit tests out of every checking compile; but separating out the installer build does have a significant benefit in terms of cycle time for a recompilation.

Frankensteining the two together was an interesting task; and IronPython has been a valuable component of the mix. As I noted quite a while ago, the convenience of an XCOPY install on a machine with a current .net installation (any build or dev machine), and the access to the full APIs makes for a powerful tool during a build -- it's not just the fact that you get better string manipulation than a batch file, or can easily spawn off a call to source control to get a synch-level value to stamp an assembly with.

In the current context, there are a number of components being built with a common architecture, so there are plenty of opportunities to DRY the system out.

  • There are repetitive pieces of code (declaring concrete subtypes of shared base classes, to inject component specific information) which can be run by having a couple of .py files (just containing a single map initialization with common keys and component or project specific values) to define the files affected and the component-specific substitutions to make (including in some cases stable but component specific GUIDs, that can be keyed off the component specific names)
  • The shared architecture makes the MSBuild .proj file just as valid for such String.Format based substitution
  • Wix source files are just XML documents -- they can be generated programmatically from XML fragments, inspecting the solution output and filling in the appropriate entities.
  • So are Wix project files (or any MSBuild project for that matter) -- a project to build a 64-bit installer can be derived from a 32-bit installer project by a similar set of XML manipulations.

Taking the latter as an example

## many imports omitted

def LoadXml(name, namespace):
  xml = XmlDocument()
  xml.Load(name)
  manager = XmlNamespaceManager(xml.NameTable)
  manager.AddNamespace('ns', namespace)
  return (xml, manager) 

def LoadWixProject(name):
  return LoadXml(name, 'http://schemas.microsoft.com/developer/msbuild/2003')
  

sigil = '_x64'
(xml, manager) = LoadWixProject(sys.argv[1])
guid = xml.SelectSingleNode('//ns:ProjectGuid', manager)
# Use the previous example to make a new stable project GUID
guid.InnerText = '{'+GuidFromHash(guid.InnerText)+'}'

# put the 64-bit .wixobj files somewhere separate
# have to scan for all build configurations  
for intermediate in xml.SelectNodes('//ns:IntermediateOutputPath', manager):
    objdir = intermediate.InnerText
    parent = Directory.GetParent(objdir)
    base = DirectoryInfo(objdir).Name
    intermediate.InnerText = Path.Combine(parent.FullName, base + sigil).FullName + '\\'
  
# give the output a separate name
output = xml.SelectSingleNode('//ns:OutputName', manager)
output.InnerText = output.InnerText + sigil

# Require the base project to define a X64=no variable via CompilerAdditionalOptions 
for node in xml.SelectNodes('//ns:CompilerAdditionalOptions', manager):
    node.InnerText ='-dX64=yes'

# save out new project
outfile = sys.argv[1].Replace('.wixproj', '_64.wixproj')
xml.Save(outfile)

Of course, assumes that the .wxs files have Win64='$(var.X64)' attributes sprinkled appropriately.

Post a Comment