I’ve recently been learning the Ruby on Rails framework for web development. It’s become a quite popular framework for getting database connected websites up and running relatively quickly. One way it is easier to get a site started than with other frameworks is because of the Convention over Configuration mantra that it lives by. Instead of requiring loads of configuration files to build a basic site (that can grow and become quite complex by the way) it has a feature called scaffolding which automatically builds your Model, View and Controller classes based on tables it finds in the database. It can do this by making assumptions based on standard conventions about interacting with a database from a website and naming and using classes in a standard way.

Although I am still a rookie when it comes to understanding the many facets of Ruby on Rails, I have really been trying to emulate the Convention over Configuration way of doing things in my various build/release projects at customer sites. One problem I inevitably encounter in most organizations is that the development of build and release methodologies has been left to the various development teams and not been thought about holistically using a centrally managed approach - this leads to little to no standards and Convention over Configuration is chucked aside. Not only is this inefficient from an organizational standpoint - why reinvent the wheel over and over again for each team when they are essentially tackling the same sets of problems, but it also makes for a nightmarish audit trail that could get you into trouble.

One reason this happens so frequently is that the managers that are supposed to be in charge of standards for building and releasing applications are often not privy to the kind of technical requirements that the various development units have when it comes to putting together and delivering their applications. And when developers try to explain the requirements, the standards people may get lost because they can’t possibly understand the nuts and bolts of every application.

To pull off real centralized management of builds and deployments, the standards people need to take a deep breath and rethink their objectives - start looking for the commonalities, not the differences between applications. In doing this, they will find that that problem that that developer told you was so unique and must be solved a certain way is probably very similar to the problem the other developer told you about last week - or just look on the web and see how many thousands of external developers have this same “unique” problem. It turns out that most applications can be constructed in the same type of way. Just because the source files are different between applications doesn’t mean that the paths they take to their target executable, dll, Jar, War or Ear file is very different at all. And when those paths are essentially the same - create a reusable process that the various teams can share. Use Convention over Configuration as your guide - standardize and centralize the common processes and externalize the technical specifications using highly modularized control files.

Here’s a simple task for you to try. This assumes all of your application teams have their code checked into a central repository - if not, you have bigger problems than standardizing builds and releases and should address those first. Look at your various technologies, whether it’s .Net, Java or some other and try to identify where the code tree’s start under the root of the project. You’d be amazed at how many teams check their .Net solutions into different levels of a code tree for no good reason, or Java teams that have their their source packages buried some place in the code tree. Next, look for the common root starting point for all these application types and try to come up with a simple standard based on this information. Finally, notify those that are not following that standard that you would like to move their code up and over to this new location (its usually up and not down) - it should actually be pretty easy to do. After this has been done, you can now have all build and deploy scripts use a standard root variable to find dependencies (think something like SOURCE_ROOT).

It always amazes me how many teams don’t standardize simple things like code tree start points in their source projects - it equally amazes me how much mileage you can get just out of making simple path standardization adjustments. After you’ve worked on the source tree, try doing the same with your common libraries. This isn’t rocket science - just remember, Convention over Configuration makes everything easier.

Adam

Recently on an assignment I come across a problem that seemingly should have been easy to solve but turned out to be a bit of a stumper. I was trying to execute a remote process using an agent that had to start as a service under the local system account. My requirement was that the process started by that service had to execute a build using a specific user’s profile. It was not enough just start the build process as another user, because I needed all of the user’s environment variables, such as APPDATA and USERPROFILE and its registry settings - the equivalent of logging in that user. In Unix, I would just ’su’ to the new user and source the profile - not so easy, as it turned out, on Windows.

After a bit of research I learned about the Windows RunAs program that allows you to do essentially what I needed. The only problem was that it couldn’t be executed in batch mode since it opened another command prompt to ask for the user’s password. Looking online, I found some people who had written all sorts of wrapper processes to try to get RunAs to work programatically. This included such circuitous methods as using a VB Script wrapper that detected typed keystrokes to pass them to the password login command prompt. Not a glamorous solution to be sure and not one that I could trust to be reliable for widespread enterprise use. I knew there had to be a Windows API for this - after some trial and error I finally found the perfect one to do what I wanted and its supposed to be compatible with 2000 - Vista versions of Windows. It involved using the C# ProcessStartInfo class under the System.Diagnostics package. And, like anytime you find the right tool for the job, once I figured out how to use it, the problem actually became a very quick fix. The class requires five arguments to Start a new process: executable file name, arguments to executable, user name, domain name and password. Note, the password needs to be converted into a  SecureString for PSI to work. For my purposes, I didn’t want to pass in an encrypted password, so I just converted it in my script.

Here is the critical code snippet to handle command line arguments passed into the Main method.

ProcessStartInfo psi = new ProcessStartInfo();
psi.FileName = args[0];
Console.WriteLine(”Execution File: ” + args[0]);
psi.Arguments = args[1];
Console.WriteLine(”Execution Arguments: ” + args[1]);
psi.UseShellExecute = false;
psi.Domain = args[2];
Console.WriteLine(”Domain: ” + args[2]);
psi.UserName = args[3];
Console.WriteLine(”User: ” + args[3]);
String pw = args[4];
Char[] pwChars = pw.ToCharArray();
SecureString ss = new SecureString();
foreach (Char pwChar in pwChars)
{
ss.AppendChar(pwChar);
}
psi.Password = ss;
psi.WorkingDirectory = Directory.GetCurrentDirectory(); //start the process in the current working dir
psi.LoadUserProfile = true;
Process p = new Process();
p.StartInfo = psi;
p.Start();

That’s it! I actually wrapped the Start() method in a try, catch and attempted to login the user name to a local machine account of the same name in cases where the domain login failed in order to handle some machines that were not on the same domain, but you probably won’t need that. Hope this helps!

Adam

As a follow up to my article on automating XML updates, I’d like to report that I did use Excel and Perl’s XML::Twig to successfully generate XML descriptors for my web service consumer, and it was a lot easier than I thought. I’m using XFire 1.2.6 web services stack running under JBoss and using MyEclipse IDE 5.0. I’m happy to say I went from blank spreadsheet and no plan to generated XML files from spreadsheet values in one and half hours. The implementation is of course expandable and reusable. This implementation should work for WebSphere and .NET as well.

I needed to create different configurations for my web application so that the service request went to different endpoints for different environments. The endpoint is at an enterprise service bus (ESB) and there is a different ESB for each environment. I need to have my ‘dev’ instance of the consumer hit the ‘dev’ instance of the ESB, the ‘qa’ instance of my web app hit the ‘qa’ instance of the ESB, etc. We’ve set up Meister to pick up the correct XML file for the target environment for the build of the WAR.

I started by setting up the spreadsheet as follows. I had an unnecessary column for Host indicating JBoss, but I hope to include WebSphere and maybe .NET as well some day. My web app actually connects to two services a.k.a. providers, so there is a column there. And, next is the configuration label for my web app with the name corresponding to the environment it is designed for. So, the first three columns of the spreadsheet look like:

 

Then I needed a way to indicate the resource that would change. Right now I only have XML files, but I chose to stick with a generic URL for that. Unlike Maven or Ant generators, we start with an XML file that actually works and has been tested - not some hacked up parameterized version that takes additional effort to create. The fourth column of the spreadsheet looks like the following (with repeated entries omitted):

 

Next, I needed a way to specify a target location to change within the XML file. Now, I know I’m going to use XPath, but I’ll want this to one day work for properties files as well, so I came up with a URL-like thing called a Universal Datum Locator (UDL) which pre-pends the method of locating the datum to change on to a method-specific locator. It could be a property name, an XPath or a Perl regex, for example. In this case it is XPath and then the last column contains the replacement value for the datum indicated by the UDL. XPath is also very intuitive and easier to construct than it may look.

The value for the UDL looks like:

xpath://beans/bean[@factory-bean='xfireProxyFactory']/ constructor-arg[@index='1']/value

So the fifth column contains the UDL’s, which in my case is always the same XPath expression. The final column of the spreadsheet contains the replacement value of the datum indicated by the UDL:

 

My nifty Perl script is only about 80 lines of real code and because XML::Twig is nearly the best thing in the world, I pass the entire XPath in as a hash key to modify the source XML file:

my $twig = XML::Twig->new(

pretty_print => ‘indented’,

twig_handlers => {

“$xpath” => sub {

$_->set_text($new_datum);

}

}

);

Here, “$xpath” is directly from the “UDL” column of the spreadsheet with only the ‘xpath://’ stripped off and “$new_datum” is directly from the “Value” column. That’s a pretty useful one line subroutine if you ask me. I had the new XML files each generated into a different folder (dev/,int/, etc). Then, I checked them into version control (CA Harvest) and built each of them with Meister. If you want the full code, let me know and I’ll post it somewhere.

I did find working with the Excel 2003 XML Spreadsheet format a tiny bit awkward. You have to keep track of the column and row indices, but not bad other than that. I see Microsoft Word 2007 allows you to save as an XML document directly, but you apparently have to define bindings. I’ll have to check that out.