Simplifying Rational Build Forge Projects With an External Control File
By Li Luo, José De Jesús, Derrick Wong
IBM® Rational® Build Forge® is a web-based tool for automating build and deployment tasks. Build Forge projects that require too many steps, checkpoints, and exceptions can become tedious to create and difficult to maintain. This article explains how to simplify those types of Build Forge projects by using an external control file. We begin with a brief explanation of how Build Forge projects work and then present this solution as an easy way to simplify things.
Overview of Rational Build Forge components
IBM® Rational® Build Forge® is a web-based tool that is used for running, tracking, scheduling, and distributing processes to various computers. Build Forge helps with automating change and release management activities.
Build Forge consists of a server component that communicates with agents running on target systems. The server oversees the activities on the managed target systems, organizes the projects, and manages user access, permissions, logging, and audit trails. Any system that takes part in a build, deployment, and test process managed by Build Forge requires a Build Forge agent.
The Build Forge management console is the main graphical user interface (GUI) for the system. It is the interface that Build Forge administrators use to manage users, configure resources, organize commands into projects, and manage the server and environment resources.
The Build Forge agent is a small process that does three things:
- Listens for instructions from the management server
- Executes those instructions (project steps)
- Sends results from the instructions back to the management server in the form of output or logs.
Build Forge also includes adapters for communicating with version control and defect-tracking systems, as well as with test systems. It supports the Rational Automation Framework for WebSphere, which can help encapsulate the configuration and deployment of applications.
Build Forge projects
When a Build Forge agent is installed and running on a target system, the Build Forge server can connect to the target system and execute the requested commands by running a Build Forge project.
You can create, schedule for execution, and execute a Build Forge project through the management console. The next six figures explain the general steps involved
Server authentication is configured in the first step to set up a Build Forge project.
A target system runs a Build Forge agent that can execute the requested commands and return results to the Build Forge server. To configure the target system, use the credentials that are normally used to log in to it. Configuring the target system’s server authentication is the first step. The connection can be tested, and it must pass the tests before you continue to the next step.
A selector is configured so that a Build Forge project will run on the target system.
The next step is to configure a selector. A selector is a dynamically chosen server or system on which the project or part of the project will run. In the figure below, BF_NAME is an example of a selector variable with a value that is DS_DEV2_BHD2DS02_Portsmouth:
The next figure shows the third step, which involves defining your environment:
An environment is a set of variables that are defined for a project or part of a project before its execution. You can use environment variables to abstract common parts of code, such as paths to binary files and version numbers.
In Step 4, you create the Build Forge project and add the related steps. Each project step consists of one or more commands that you can include in the Details section for the step. Here is an example:
A project contains a sequence of steps to run on specified selectors and with specified environments. Each step is composed of one or more commands that can be run from a command line. You can add, edit, or delete the steps and change their sequences. You can also specify the selector and the environment for every step or for the whole project.
You can execute a Build Forge project after it is created.
Step 5, shown in the figure below, is where you can execute the project. You can do so by clicking the blue Play button next to the project name or by clicking the Execute button on the project page. You can view the log and the results of the run from the Jobs menu on the left.
You can also create a schedule of execution for your projects through the Schedules menu (see below). You can use that menu to plan the execution of a project that is based on a specific time and other criteria.
In general, this is the effort that is required to create and run a Build Forge project.
A more complex example
Now, assume that we have a more complex example project, with the following characteristics:
- An application that is Microsoft Windows-based and consists of multiple application servers and a database server.
- The Build Forge project needs to incorporate dozens of application-specific projects, each of which consists of several packages.
- Each package, in turn, consists of several deployment artifacts.
- Both packages and artifacts must be built and executed in a specific order and on a specified server.
- The deployment artifacts can be .msi files, .sql files, .dll files, and application configuration files.
- Depending on the build and deployment, certain files must be deployed in a certain way and to specific servers.
- Before deployment, in some cases, special strings in the configuration files must be replaced before running the corresponding application.
- The project is programmed to know what to do, depending on the artifact type.
- This behavior should be extensible for new artifact types introduced in future builds.
- The list of artifacts to be deployed and their order will change frequently.
Using an external control file
These requirements are much more elaborate and would make it difficult to implement a solution with just Rational Build Forge. It would involve hundreds of Build Forge steps and either too many scripts or a very complex script to handle the different possibilities. Maintenance would also be problematic.
The solution that we present here uses a generic Java-based script that can process a .properties file (which we call a control file) that contains “instructions” on what to do and when. The Build Forge project would call this Java-based script as one of its steps.
These instructions are merely a list of package names and artifacts listed as key- value pairs in the order in which they should be deployed (a typical .properties file). The Java program that processes these instructions is the key. It recognizes exactly how to handle each type of artifact listed. The complete source code for the Java- based script and a sample control file is provided with this article in the Download section.
Let’s take a look at the general flow:
The figure above shows what happens at the level of the Build Forge project: the steps in the project perform the initial setup work, such as gathering all of the necessary artifacts from IBM® Rational® ClearCase® software and moving them to the correct target systems. The Java-based script then reads the control file to perform the corresponding build and deploy operations. When the Java-based script runs, it reads the control file to determine which packages and files need to run and in what order. The script then executes them one by one according to their file types.
This is a sample control file:
The key PackageDeploymentOrder specifies two packages, Package1 and Package2, to run in the order listed. The dynamically created keys, Package1.RunOrder and Package2.RunOrder, specify the files to build for each package. This script is also capable of editing a specific configuration file according to the find-and-replace key- value pairs in the control file. The figure above also shows two keys that are used to replace a string in a configuration file:
- PackageName.FileName_Find_Lines
- PackageName.FileName_Replace_Lines
In some cases, the build or deployment process might need to switch between different target systems. In that case, the Java-based script uses Java Remote Method Invocation (RMI) to reach the remote environment and deploy the necessary code onto it. For example, as Figure 8 shows, some of the artifacts that get built
or extracted should be executed on the application servers, but others run on the database server. SQL files in the control file and all other data related deployments go to the database server, while the application servers receive other types of artifacts.
Benefits
By abstracting your build and deployment tasks with an external control file and a generic script that can act upon its content, you can simplify what would otherwise be an incredibly complex Build Forge project. Using a control file decouples some of the invocations from Build Forge and makes creating and maintaining the application much easier. Of course, what is presented here is only an initial iteration of the solution. It can be improved in many ways. Here are a few ideas:
- You can configure the control file either to update what changed over the previous deployment or to run a new deployment.
- A check-before-deploy feature or flag can be implemented that would involve the user as little as possible.
- The deployment operations could be generalized and encapsulated in a plug- in class that would define generic operations for new plug-ins. This would make the code more reusable and make it easier to address the specific build and deployment processes of other applications.
Using other tools
With this type of approach, you would still use Build Forge as the main platform to automate and streamline your software build and deployment process.
The control file approach works well with other tools too. For example, it does not intend to replace the functions that are provided by Apache Ant, which is an open source, Java-based build tool to automate software builds, similar in many ways to a “make” utility. In this case, you can use Ant, for example, to transfer the artifacts from ClearCase to the target system and use the control file and the Java-based script to the build and deployment parts.
Ant is based on Java and can be easily extended with Java classes. So it is also possible to replace some of the script’s functionality with Ant. In some cases, however, because Ant works from an XML file, users might find it difficult to use for very complex build and deployment processes, as in our example. One reason is that Ant and XML is not as convenient to use to depict the complex logic.
In this case, although XML is human-readable, it is still more difficult to maintain and validate than a simple .properties file. For example, defining common XML elements and reusing them for other applications can be tedious. Also, as mentioned earlier, you can easily extend the functionality of the control file with separate Java classes that implement specific interfaces.
Ultimately, it depends on design philosophies, but the idea is to simplify things and make your build and deployment tasks more reliable and much easier to maintain. Knowing which tools to use and when depends on the specific needs of your project.
Real-life results
We used an external control file to complement our Build Forge projects on an internal IBM project. This approach has worked quite well and enabled our Build Forge projects to handle very complex processes for some specific applications. We hope that sharing the ideas presented in this article with you will shed some light and give you ideas for your own projects.
