SOA Job Scheduling - Out of the Dark Ages

Classical batch job scheduling is still caught in the dark ages (or more specifically the 1980s). Many organizations still run jobs in arcane cron driven types of environments and manage their jobs in ad hoc ways with hard to track/report and administer systems. There are expensive mainframe and client/server job management platforms out there but they are out dated, for the most part, and are complex to manage and not very open or developer friendly.

Batch job scheduling and workflow processing in general is missing the SOA bandwagon and leaving the developer and the enterprise with a patch work of ad hoc solutions to deal with. To solve this challenge several issues need to be addressed. One, there is a need for a standard way to build server-side processing tasks that are modular/reusable and can be composed into more complex jobs that can be wired together at design time. Second, server-end tasks need convenient integration with ESBs and other web services type of infrastructure to make it easy to build jobs that can access the evolving SOA environments emerging in many organizations. Lastly what is needed a is to put a nice face on your hard working blue collar batch jobs and tasks. Wouldn't be nice if you could configure a job/task from an AJAX web application to quickly customize the job's properties? And wouldn't be nice if you could do this using SOA services available in your environment and be able to build, package and deploy these batch jobs and their supporting GUI applications (property customizers) quickly and easily?

There is no sliver bullet, but what is needed is an open standard for building batch tasks/jobs. Well as you may have guessed we have the answer. SOAFaces is just such a framework. It is built with all these challenges in mind and with goal to make building modular batch jobs and SOA applications modular and reusable.

SOAFaces provides a unique approach to the solution. Here is how it works. A job can be composed of one or more tasklets. A tasklet is a rather simple server-side interface that once implemented does your bidding. Tasklets can be chained together to form more complex jobs (tasklets in a job need no prior knowledge of each other). A tasklet can optionally have one input JavaBean and one output JavaBean. The input JavaBean can be customized (its properties can be edited) by something called a weblet. This is a GWT driven modular web component that functions as the GUI customizer for the tasklet's input JavaBean. Note that the input JavaBean and weblet are optional. You can build for example a simple or just hardwired tasklet with no customizer inputs at all. So many of the features are optional to help keep simple things easy and hard things possible. The attached diagram show how tasklets and weblets work together to build SOA powered jobs.

This diagram shows the three states that a job can be in. First is the design time, when the job is being configured and wired together. This is when the weblet can be used to customize the input JavaBeans used by the tasklets among other things. Second is the processing/runtime mode when the tasklets are run as part of a job. In this stage the tasklets can consume an input JavaBean and produce an output JavaBean. The post runtime mode, defines the period after which the tasklet/job has run and end users can view the logging history of the tasklet/job and what input/output JavaBeans where used by it. This is very critical for auditing and reporting.

With the SOAFaces framework, both the weblet and the tasklet also have direct access to web services so resources across the enterprise can be accessed and used to both configure the job at design time or execute the job at runtime and with convient access to SOA service like Mule and other web services.

The final piece of the SOAFaces framework is the packaging of tasklets and weblets into bundles. A tasklet and corresponding weblet can be packaged into a JAR called a SOAFaces Bundle (aka SFB). This is a modular and self contained package that can be dropped into any job processing container that supports SOAFaces to allow developers and end users to quickly plug in new components to build jobs.

For a complete example of how SOAFaces can be used to build a robust and extensible job scheduling and management platform check out JobServer. JobServer supports and implements the SOAFaces framework giving developers and end users the features and capabilities discussed.