Agile Build, CI and Testing Automation
9 Feb
One of OpenMake Software’s product strategies is to keep things simple. Build management is one of the most complex operations in all of the IT world, and one of our key benefits is to simplify, organize and automate the build process for development, testing and production.
We’ve seen a trend among our customers to simplify their build management infrastructure by going to fewer build machines with more CPU cores. Builds in particular use relatively more CPU resources than other resources as code is interpreted and compiled in memory and then finally written to disk. By reducing the total number of machines, rack space, procurement, administration and other IT overhead costs are reduced at great cost savings per machine eliminated.
Recently, I was at one of the big chip makers where they used dual quad-core CPU Linux machines for their development and builds. They had two machines and were able to control access to allow separate areas for development, testing and release builds in keeping with best practices. Having all the horsepower of 8 CPU cores on a single machine kept them from needing more machines.
Another customer does 6000 builds per month with Meister on just two build machines.
IBM, when selling BuildForge, likes to talk about big build server farms, because their tool does remote execution on multiple machines, as does Meister. However, BuildForge does not do builds at all. It can remotely execute your existing build scripts, but there is little real value add to that. BuildForge is also famously expensive. What happens over the next few years to the high investment in multi-machine remote execution software as the number of machines declines, perhaps dramatically?
A similar argument can be for Electric Cloud’s Electric Accelerator product. It’s possible in some cases, for C/C++ builds to gain an edge by pushing a compile operation to another machine, and then bringing it back. You would only do this to gain access to additional CPU resources. In the past, you might have 8 build machines that Electric Accelerator would farm operations out to. Now, you can pull all those operations into a single machine and there is no need for that functionality. Also, you are stuck with converting your GNU makefiles into other GNU makefiles.
Meister is optimized for multi-core CPU build machines and offers multi-threaded capability to both build events and non-build workflow events. You know where your build is and there are fewer dependencies on network resources. Both BuildForge and Electric Accelerator add additional overhead to build administration to coordinate across multiple machines – a dying practice, that no organization wants to invest in. Meister is the best bet for a future with fewer build machines with more horsepower.
5 Sep
I think it is funny that Electric Cloud is saying that Perl is and old, outdated language (presumably to position themselves against OpenMake and Meister) when they are based on make, which truly was designed for C and is not very extensible (but works for ad hoc build scripting). I can add FTP or email support to a Meister build method in less than 10 lines of Perl code – not so easy to do with make alone. I can even update my Facebook page with the latest build result through a Perl API.
Most developers know Perl has a recognized place in most companies. There are trends in software languages, and Perl may not be the latest, coolest thing on the street (like Ruby or Python) but Perl is incredibly dynamic and robust. It is possibly the greatest single example of successful open source development. Neither Ruby, Python nor anything else besides the open source Java community even comes close to offering the breadth of functionality that Perl does. Perl even does automatic release dependency management – something unique for its scope in the open source world, although the Maven project is attempting to do something similar with Java.
Java developers often think Java is the only thing in the world, but in comparing Maven (an all Java build tool for Java builds only) and OpenMake, it took 60 pages of Java code to do the same thing (see my previous blog on XMLBeans) as 1.5 pages of perl code as used in OpenMake.
Perl works great at moving files around and working with the operating system – Java is not so good at that. Meister and Mojo use Java for the server which works well, but Meister uses Perl where it can be used best (parsing texts and interacting with the operating system) and C is in turn used where is best (C can’t be beat for speed). We also use make for what it is does well – the rules engine and dependency ordering. Combining the C based make engine with Perl for executing commands really gives enormous power to Meister.