Best Practices and Technology in Software Delivery
26 May
I’m now getting about 100 spam comments a week, and so I have to require registration in order to leave comments. A few of you have asked questions to me and this will ensure I can get back to you as well.
So, until I find a new spam filter plug-in for Word Press (and figure out how to use Word Press) I’ll have to require registration.
Thanks,
Sean
26 May
Here comes buildr: yet another Java build tool. Hopefully I, or one of my other cohorts will check this out in detail soon. But, with my experience working with all manner of build tools, with 100 companies and many more development teams, I can already make a few observations.
First of all, why another build tool for Java? I am occasionally told that Maven or Ant is a perfect tool, but clearly the people behind buildr don’t think so. The choice of JRuby as the vehicle for delivering this tool, I think is probably a good one. JRuby is a scripting language in the same vain as Perl, which is used by Meister.
Doing software builds is an ugly business involving lots of file and operating system interaction. This is not where Java shines, but scripting languages can. As long as operating systems are written in C and not Java, C-like tools will be better and faster at interacting with them. Plain Ruby itself is C-based, and JRuby no doubt inherits C-like operating traits. Calling out to a Java compiler from Perl or JRuby, though it has its own JVM, does not represent a significant overhead compared with the file system operations and the compilation/translation itself.
Both Maven and Ant are relatively difficult to extend compared most other build tools, and I’ll be buildr beats them here. If you have all the Maven plug-ins and Ant tasks you need, then good for you. If not, then you have to start developing in Java and it becomes too much of an investment to sink into a build system. It is much cheaper to extend in JRuby or Perl. My frequently cited example is the XMLBeans compile step in Meister, written in Perl, which is only 40 lines of real code. The Maven plug-in is 60 pages of Java code and no one can tell me really what it is doing (I asked on all the forums). Less code is usually more transparent, which is also good for build audits.
I am a little disappointed to see them try to placate the Maven and Ant users by promising it is a drop-in replacement for Maven and they have all the Ant tasks covered. Both tools have their drawbacks and I don’t want to see another tool with the same deficiencies. They should have the cajones (or coñejos) to apply all their resources to what they think is a better tool (with its own unique benefits and deficiencies). I imagine offering Ant task equivalents is pretty easy because of the ease of coding in JRuby compared with Java.
They also don’t mention who is supposed to use the tool. Is it for individuals, small development teams, the enterprise? Maven falls short because it is only appropriate for development teams and not for stable, controlled, enterprise builds. Ant is not even tool, but a means to create some tools for small teams. I don’t think Meister will fear buildr either.
Well, since buildr is only in incubation status with Apache, I’m not sure how much time I’ll be able to spend on it, but I am curious and I’ll let you know if I find out more.
9 May
I’ve been using and experimenting with Facebook, LinkedIn, All About Wine, Ancestry.com and Plaxo online social networking. I am confident I will be a user of online social networks as long as I can type, and hopefully longer. Here are the things I like best about using them:
This is especially true with people that you would not socialize with frequently such as people you don’t see in the office every day. My social interaction with these people went from ‘never’ to ’sometimes’. This includes those of my friends and relatives who are a great physical distance away. As people move, my laptops explode, and email clients change, I lose track of people’s email addresses, so I miss out on sending a holiday greeting or the latest round of baby pictures.
Between Facebook and LinkedIn, I pretty much have all of my high school and college friends somewhere and that can be beneficial professionally as well as socially.
Some of my friends say that Facebook uses up too much of their time, but I find that Facebook in particular saves me a lot of time. I can click on anyone’s face in the Entourage application and send them a quick note. Or, I can take advantage of Facebook’s slick auto-complete where I only need to type one or two characters to uniquely identify someone. Time savings would be #1 on my list if it were not for the fact that this greater facilitation leads me to slightly more interaction. I do not consider the greater interaction to be a distraction, but it in fact slightly improves my overall quality of life.
LinkedIn also saves me time professionally. Forwarding someone’s LinkedIn profile is much faster than attaching a resume document and writing an introductory email.
Facebook also helps me share my interest in less popular topics with like-minded enthusiasts. I can easily send soccer highlight videos or punk rock news to those few friends of mine who are interested. While I could create an email list to do the same, Facebook makes it far simpler and much faster to manage these types of lists. Or, I can post something, and those who spot the post on their mini-feed and are interested can click on it to learn more.
I suppose baby pictures fall into this category. Facebook in particular has awesome picture sharing functionality. The ability to ‘tag’ people in photos and have Facebook collect them is priceless. For example, I get can all the pictures taken by many different people where I am tagged and Facebook will collect them for me into a single slide show.
9 May
The first rule for Bash/C/Korn shell scripts in a Perl program environment is to re-write them all in Perl. If your Perl environment has any sophistication, you will have common code, standardized logging (perhaps with Log::Log4perl), testing with Test::More, etc. and your shell scripts just can’t keep pace.
If you share the environment with any non-Perl applications, however, you will still have to deal with the environment profile(s). I also have some legacy shell scripts that we can’t justify converting to Perl unless they have another reason to change. (Don’t change tested code in my house ~~ head bobble + finger wave ~~, nuh-uh!)
There are two ways I know of that you can extend the benefits of your Perl implementation towards your legacy and profile shell scripts. The first is through Bahut’s excellent tip on embedding POD documentation in shell script. This solves my problem of generating HTML documentation from POD in Perl scripts and having upsetting holes where the shell scripts are. I also have some controls for the Perl scripts that run podchecker before committing to version control, which fails if no documentation is found. Now, I can extend this control to the shell scripts.
The second Perl tool you can extend is the testing functionality. I’ve found the functionality in Test::More to be useful for validating that the changes to the shell environment profiles are correct and do not introduce defects. Profiles can be notoriously tricky to change when they get fat and you have variables depending on other variables. Mostly the profiles in my case are used to set environment variables that control the version control and build system, and these can be easily validated in a test script called profiles.t via checks like:
ok( $ENV{CODE_ROOT} eq ‘/opt/code’, “CODE_ROOT set to ‘/opt/code’”);
You then just rattle off tests for all the variables that are set and you have a great way to validate that everything will still work after the profile change. For a legacy script, you may not be able to have a crack at the internals, but you can at least check the return code and maybe some external effect it has somewhere, such as a file timestamp change.
eval { `legacy_script.sh`};
ok( !$?, “legacy_script”); #– $? is zero if script executes successfully
Profile.t and any other test scripts used to test legacy shell code can be bundled with all the other Perl tests via Test::Harness for a single test suite that really tests everything shell and Perl.
9 May
First, let me say how nice it is to have the Mojo workflow engine that allows us to manage the compliance checks, deploy to multiple machines in parallel and validate deployment. This makes our lives a lot easier and provides clear benefits for deployment via the parallelization, dependency management, scalability, logging and reporting. Underneath the covers, and for those of you who don’t have the luxury to use this almost-free product, there are some important low-level tools that are critical to the development, testing and operation of the Mojo JBoss deployment system on Linux.
With the most important listed first, they are:
JBoss support wins hands down due to the number of bugs and critically important undocumented features. On a scale of 1 to 10 where 10 is the best documentation, I give JBoss about a 3 or 4. Googling doesn’t even help that much for deployment issues.
You may be surprised at the prominence of Perl, but if you think about what you are really doing and what the best tool for the job is, it makes sense. You are really moving an archive (a ZIP format file), copying XML files, creating directories, changing permissions, extracting the archive to the file system perhaps. Where did I mention Java? Nowhere. The twiddle.sh command comes in handy if you get the secret commands from JBoss support that tell you if the application you deployed has actually started correctly. Notice that this is a shell script suggesting we’re not the first to use non-Java tools to manage deployment.
Particularly on the testing side, I can’t think of a viable alternative to Perl testing. We need to test that we created this directory, changed that permission, updated that file timestamp, etc. We have about 300 test cases encoded in Perl that are run with every change to the deployment system. It takes about 20 seconds to write and run a simple test case in Perl.
Lessons? Use JBoss support early and often and use Perl.