I’ve worked out a good system putting Meister build management metadata under version control. Why put it under version control when you can do a backup, you may ask? Well, there are times when you may want to develop a reusable workflow or update a Java build method script and test in an isolated test environment. For this you would use a separate test instance of the Meister server. Putting at least some of these files under control will help ensure that you move the known version of your tested server metadata file into your production environment.

There are a few other challenges to be aware of. You may want projects and dependency directories to be able to be created an edited in production while you simultaneously modify a workflow, for example. You don’t want to overwrite a dependency directory definition in production with an older one you are migrating from the test environment.

I’ve got a few other challenges as I’m using CA Harvest for version control and its excellent control over changes extends version locking to the file system by removing write access to files that are not locked by someone for change. Meister requires that most metadata files be writable, so if you check in files to CA Harvest directly, you will leave Meister metadata files read-only and that will cause problems.

So, here are a few tips from my now slick Meister metadata version control system:

• Use the …/meister/kbserver/tomcat/webapps/openmake.ear/openmake.war/ directory as the root of the file tree under version control. Don’t version anything outside of that tree, but do version everything under it for a simple boundary to your project.
• Before checking in/committing changes, copy files from the Meister runtime directory to an alternate workspace or temporary directory tree. Check in/commit from there, not the Meister runtime directory. This ensures that you don’t write into the runtime environment, possibly corrupting something.
• Before taking updates from version control, check them out to an alternate workspace, reference directory or other temporary directory. This is again to avoid corrupting the runtime environment. For CA Harvest, you can use the automated reference directories to copy from. Be careful not to preserve the read-only attribute when you copy from there. I used File::Copy::Recursive and its option not to preserve permissions for a simple, streamlined copy.
• Manage the check in’s and deploys (copy’s) according to the major subdivisions of the Meister metadata. My check in and deploy commands work by taking a directory relative to the openmake.war/ directory as an argument.
• Use the awesome Perl SCM project by yours truly to reduce your coding by 90% if you are using CA Harvest. CA Harvest also requires a number of other steps, such as creating a change package, locking files you plan to change to the package before checking them in, and breaking up the locking and check in commands into manageable chunks. The Perl SCM project helps with all that.
• If you can manage your changes incrementally, all the better. OpenMake Software’s HarRefresh and CA Harvest’s HRefresh manage reference directories with incremental changes. This gives you a workspace or reference directory with only the changes relative to a baseline, not the full baseline. This means if you changed only one workflow, then it is easy to copy just the one changed file to the production environment, instead of the whole baseline with only one changed file. You will have a very simple and transparent production change procedure with this – very nice.

I’ve found the multi-threaded capabilities of Mojo and Meister workflows to be very valuable for builds and deployment. The chief benefit I’ve received is in saving time as you might expect. I’ve been working with a workflow that deploys a Java application to up to 24 servers. Let’s ignore the sequential part of the workflow and examine the time difference of running parallel deployments versus one where each of the 24 machines is updated in sequence. The deployment process takes about 5 seconds per machine. Sequentially, that’s 24 x 5 seconds, or 2 minutes. In parallel, well it’s not quite 5 seconds, but closer to about 20 seconds because of limitations of the Linux machine it is running on. Still, that’s a tremendous 100 second savings.

In addition to using the parallel workflow to cater to impatience and improve productivity, I want the Java application to hit all of the servers in the cluster close to the same time. In this particular strategy, only 3 machines out of the 24 are in the cluster. The rest are to support dynamic resource allocation and disaster recovery. Running the deploys in parallel allows me to hit all machines, and therefore all the machines in a cluster at close to the same time without having to figure out some ordering so that the cluster servers are hit first and then the rest. This ends up saving a lot of coding, testing and possibly debugging. Great stuff.