Software Factories: better automating software development

I said that the next article would look at Simon Phipp’s ideas about the open source cycle in more detail, but instead there’s a small interlude because last night I heard Steve Cook from Microsoft talk at one of a series of British Computer Society Software Practice Advancement meetings in Cambridge.

Steve is one of the software architects working on Visual Studio. His work concerns Software Factories and his talk, which I thought was spectacularly good, was titled ‘Software Factories: Assembling Applications with Patterns, Models, Frameworks and Tools’.

Like all good talks his was absolutely jam-packed with ideas. A sort of brain-dump. He has obviously been thinking a great deal about the area for years, and it showed. Unfortunately I was so captivated by what he was saying that I didn’t keep good notes, so what follows is from memory. At some point I may have the slides from the talk, and I’ll update this. In the meantime, Steve, I apologize if I’ve mis-represented anything you said or missed out some important points.

He started with the thesis that software development needs to (and is) moving away from artisanship towards something better. He spoke a bit about the phases that other industries have moved through (artisanship, mass production, continuous improvement and mass customisation) and tried to draw parallels with the software industry. It’s an idea I agree with in part, although I thought the analogies he used were misleading; the software industry spends most of its time and effort in the product development phase of a product lifecycle, in contrast to most other industries where production is the part that takes most effort. Despite this, I agree that there’s a clear trend towards using higher and higher abstractions in order to allow us to accomplish more. And Steve’s ideas are all about the ways in which we can accomplish this; through Software Factories.

Next he looked at what a Software Factory has to accomplish. He presented a convincing case that in order to effectively use knowledge it had to be readily available at the point of application. Many (possibly most) re-use programs fail because finding and re-using the knowledge is harder than rebuilding the component, for instance. Corporate standards often fail because the the group mandating the standard has no power or mechanism by which to lever the standard at the time and in the place when the standard would be beneficial; the first thing that developers inside corporations when presented with a new project do is not to open their book of corporate rules. They open an editor. Steve convincingly showed that for re-usable knowledge to be successful in practice, tools that understood the context of your work were necessary. You only want GUI design guidelines to be on hand when you are designing a user interface. You only want coding guidelines when you are writing code. Your viewpoint provides the context.

Next he looked at how we could provide knowledge, and here we have to take a little leap-of-faith: Steve thinks that little domain-specific languages are the way that we can codify and re-apply knowledge. He gave an example of a domain specific language which modelled phone billing. He showed a model for the process, and demonstrated conceptually how a domain specific language could be used to capture the points of variability between several product lines, for instance. To produce each application, you would vary the input to your domain-specific language model, use this to generate code, and the code would plug into the framework for your application, the framework being the part that doesn’t vary. For the phone example, using the domain language you could generate many billing systems – each of which would be suitable for type of customer using a particular phone payment plan.

Of course you can use more than one domain specific language in the same product line, each controlling a particular aspect of the product line.

In the last part of his talk, Steve looked at a tool that his group has built to help with all of this. They have an add-on to Visual Studio which helps you to design domain specific languages and matched code generator, suitable to your application. One lovely feature of their tool is that it’s the embodiment of a domain specific language used to describe domain specific languages; they have bootstrapped themselves.

In the closing minutes of his talk, Steve walked through some examples of their tool in action. I think that one of the examples was that if you worked in a bank you might use this tool to capture the knowledge of the variability between account types. This would allow people building applications to embed this knowledge in their work; building applications which automatically correctly manipulated the state of a bank account, and knowing with more certainty that the operation was correct because the code was generated using a proven domain language for describing bank accounts and transactions.

If you use Visual Studio, you can download a beta of their tool here. There’s also wealth of other information about the ideas.

I’ve seen the tools listed on the VS site, and before now and had a quick look. I wrote them off as uninteresting. I was wrong. I misunderstood the potential power that they have to help companies with particular types of projects.

I came away from the talk quite elated; the ideas seem to me to be one (meta) step beyond object libraries, and potentially very useful for certain classes of applications. I think that they represent an important step towards being able to automate the repetitive parts of software development, and an excellent way to allow you to capture knowledge, and reuse it.

In the ending moments of Steve’s talk, I was struck by the smartness of the ideas. It occurred to me that this is the sort of thing that open source doesn’t often produce, but which it should. I think that the reasons have to do with economics. More about this next time…