IronPython for build scripting
Following on from the earlier post with the snippet about generating GUIDs -- and covering a good chunk of what's been occupying me since...
I have ended up in charge of the build system for the current project at work. This started out with one framework that used a number of custom projects inside a solution to perform unit test, FxCop and coverage analysis, with a lot of magic happening in post-build steps, including direct calls to Wix command-line utilities. Another team had developed a better separated MSBuild-based system, which split out things like the analysis and installer building from the assembly-building solution. We can argue the merits of taking the unit tests out of every checking compile; but separating out the installer build does have a significant benefit in terms of cycle time for a recompilation.
Frankensteining the two together was an interesting task; and IronPython has been a valuable component of the mix. As I noted quite a while ago, the convenience of an XCOPY install on a machine with a current .net installation (any build or dev machine), and the access to the full APIs makes for a powerful tool during a build -- it's not just the fact that you get better string manipulation than a batch file, or can easily spawn off a call to source control to get a synch-level value to stamp an assembly with.
In the current context, there are a number of components being built with a common architecture, so there are plenty of opportunities to DRY the system out.
- There are repetitive pieces of code (declaring concrete subtypes of shared base classes, to inject component specific information) which can be run by having a couple of .py files (just containing a single map initialization with common keys and component or project specific values) to define the files affected and the component-specific substitutions to make (including in some cases stable but component specific GUIDs, that can be keyed off the component specific names)
- The shared architecture makes the MSBuild .proj file just as valid for such
String.Formatbased substitution - Wix source files are just XML documents -- they can be generated programmatically from XML fragments, inspecting the solution output and filling in the appropriate entities.
- So are Wix project files (or any MSBuild project for that matter) -- a project to build a 64-bit installer can be derived from a 32-bit installer project by a similar set of XML manipulations.
Taking the latter as an example
Of course, assumes that the .wxs files have Win64='$(var.X64)' attributes sprinkled appropriately.
Posts
2 comments :
Ah, yes. Using Python as a build tool brings back good memories.
Back in the day, before MSBuild and IronPython, I wrote a build script in regular Python. It was my team's first and crude attempt at continuous integration:
http://www.bugfree.dk/blog/2006/01/04/being-a-functional-pythonian
My Python build system replaced a series of batch files that did looping, called out to external commands, and did their best at checking for errors along the way. While the batch files worked most of the time, they were in trouble whenever a command didn't signal success or failure using the proper exit codes. Not to mention that large batch files are a nightmare to debug.
But using Python I could parse stdout/stderr, look at the file system, or whatever to assert success.
There's just something more appealing to Python code over the angle brackets of MSBuild or NAnt configuration files, and defining custom build tasks in .Net.
The Ruby guys have long used Rake to express makefiles in Ruby, but I don't believe Python have something equally popular.
"There's just something more appealing to Python code over the angle brackets of MSBuild"
Exactly. The Python code is imperative rather than declarative -- and a lot of the time that imperative nature is exactly what you want.
Being able to script with a rich set XML handling APIs to hand means that even where you absolutely have to use the declarative XML forms (MSBuild and Wix, rather than Ant, in my case) for building, you can treat them as data in the steps before you actually use them.
You can turn the Python scripts into a semi-declarative style backed by generative code as well. The declarative parts are project specific modules that just consist of lists or dicts defining data that are then imported by a reusable generator script, e.g. along the lines of
"for each file (key) in imported dict X of deliverable files, generate a Wix component entity, and append it to the directory node with which it is associated by the mapping"
Post a Comment