As most readers are probably aware, the Web Services-Business Process Execution Language (WS-BPEL) provides a broadly adopted process orchestration standard supported by many vendors today and used to define business processes that orchestrate services, systems, and people into end-to-end business processes and composite applications. However, in many ways BPEL's adoption has gotten ahead of the formal standardization process.
The BPEL4WS 1.1 specification was submitted to OASIS back in 2004 and after three years of work by one of the largest technical committees at OASIS, WS-BPEL 2.0 finally became an OASIS standard on April 12, 2007. While adoption of the BPEL language has not been gated by the 2.0 standard or the OASIS stamp of approval - there are thousands of successful BPEL projects and deployments today - the formal publication of the standard is an important milestone and will further accelerate BPEL's adoption and vendor support.
A lot has been already written about the new features in WS-BPEL 2.0 on various blogs, Web sites, and magazine articles, including the article "BPEL Grows Up" (http://soa.sys-con.com/read/346372.htm). In this article, we'll drill down into the next level of detail regarding the new features in WS-BPEL 2.0 using concrete examples wherever possible. Throughout this article, we abbreviate BPEL4WS 1.1 as BPEL 1.1 and WS-BPEL 2.0 as BPEL 2.0.
BPEL 2.0 Overview
At a high level BPEL is an XML language that provides a rich set of activities to describe an executable business process. The processes and activities can be synchronous or asynchronous, short-lived or long-running; BPEL provides a sophisticated language for defining the process flow, system interactions, data manipulation, exception handling, compensation rules, etc. First, we will briefly summarize the important features of the BPEL standard, explicitly calling out what is new or changed in BPEL 2.0: (Figure 1)
• Service interaction activities: A BPEL process is automatically a Web Service and receives inputs via <receive> or <pick> activities. A process can send back a synchronous Web Service response using <reply>. The <invoke> activity is available to invoke an external service, as described below, but also to respond asynchronously to a client.
• Event handling constructs: A process can get input requests at non-deterministic points during process execution with <eventHandlers> using <onEvent> for messages (new in BPEL - replaces <onMessage> from BPEL 1.1) or <onAlarm> for time-triggered events. <wait> can wait for a specified time or until a deadline is reached and <receive> can wait for events at pre-determined points in the process.
• Back-end system interactions: Interactions with external services are represented as <partnerLinks>. Asynchronous conversational interactions can be correlated using <correlationSet> or the WS-Addressing standard. A process maintains its state using <variables> that can be defined at global or local scope. BPEL 2.0 makes it easier to map process variables to WSDL message variables. It also provides the new <messageExchange> activity to distinguish instances of similar conversations (request/response pairs).
• Data manipulation activities: BPEL 2.0 adds a new simplified XPath notation ($variableName) replacing the getVariableData() function. Besides the existing <assign> activity to map data between variables, BPEL 2.0 provides a doXSLTransform() function to natively support XSL Transformations. A <validate> activity has been added for schema validations. These additions have already been time tested, having been implemented as extensions in vendor implementations of BPEL for quite some time now.
• Process structural flow related activities: BPEL includes basic structural activities similar to other workflow or programming languages for sequencing, iteration, and branching. BPEL 1.1 supported <sequence> for sequential execution, <flow> for parallel branches, and <while> for looping. BPEL 2.0 adds <if> / <else>, <repeatUntil> and <forEach> for richer flow control syntax. In particular, the new <forEach> construct now supports dynamic parallelism (executing N activities in parallel, when the value N is not known until execution time). This was not supported in BPEL 1.1 except through vendor extensions.
• Exception handling and recovery constructs: Exceptions, represented as faults, are propagated using <throw>, and BPEL 2.0 adds <rethrow> to provide more explicit control over exception management patterns. In <faultHandlers>, faults can be detected using <catch> and <catchAll>. A process can undo completed work through <compensationHandlers> and the <compensate> activity; BPEL 2.0 adds <compensateScope> to clarify the syntax of BPEL 1.1's overloading of the <compensate> activity. BPEL 2.0 also adds <terminationHandlers> to enable processes to specify the logic to be invoked just prior to the termination of a process.
• Extensibility: BPEL 2.0 adds <extensionAssignOperation> to extend the standard <assign> activity; it also provides <extensionActivity> to add new activity types. This is another area where the 2.0 standard now explicitly covers things that vendor implementations were already doing. BPEL 2.0 also now supports <import> and <documentation>.
Kitchens Online Use Case
To illustrate some of the BPEL features, we will use the example of Kitchens Online, a fictitious Internet-based kitchen-remodeling solution. Kitchens Online provides a Web site where customers can select appliances and cabinets and schedule delivery and installation. Kitchens Online doesn't carry any stock and sources the different components from various vendors; however it still wants to keep inventory information as up-to-date as possible. Complicating matters, the vendors Kitchens Online's suppliers don't always have automated inventory systems - in many instances, a person needs to check manually whether a given item is available. When a customer places an order, Kitchens Online tries to reserve all the needed components from its vendors. If it can't reserve a piece, the customer is notified and can then change or cancel the order.
An overview of Kitchens Online's business process is shown in Figure 2.
This example highlights the following advanced requirements:
- Dynamic Parallelism - handling of different components (line items) of an Order in parallel
- Change handling (out-of-band events) - a delivery schedule may change while an Order is in process
- Compensation - reversing credit card charges and releasing reserved inventory when a component is out-of-stock
Since this article is focused on the BPEL 2.0 standard, and BPEL is an XML language, we'll examine the BPEL "source code" in XML format. Of course most developers won't hand-code BPEL in the XML format - they'll use visual tools built on top of the standard. Several vendors and open source implementations of such tools exist, including some that enable developers to switch back and forth between a visual model of a BPEL process and the underlying BPEL source. Here we'll examine the underlying XML source, so get ready!
Dynamic Parallelism
The Kitchens Online requirement to process the multiple items of an Order in parallel is a very common pattern. The <flow> activity introduced in BPEL 1.1 supported parallelism, but only when the number of parallel branches was known at design time. Here, an Order may have a variable number of items and therefore the number of parallel branches won't be known until runtime. What's needed is the equivalent of a typical programming language "for" loop where all the iterations of the loop are executed in parallel.
To address this pattern, BPEL 2.0 introduces the <forEach> activity. As the name indicates, this activity causes the enclosed scope to iterate between the <startCounterValue> and <finalCounterValue> inclusive (i.e., N+1 times, where N is the difference between the two). The <forEach> activity has an attribute parallel that when set to yes causes all the branches to be executed in parallel and a counterName attribute that specifies an integer variable that will hold the counter value for each particular iteration (1 for the first iteration in the example below, 2 for the second, etc.).
Listing 1 illustrates how Kitchens Online can use the parallel <forEach> activity to process all items in the Order in parallel. Notice the use of a local <partnerLink>, vendorPL, in the <scope> reserveInventoryItem; BPEL 2.0 introduced this notion of a local partnerLink to create a different service reference dynamically for each branch. Also notice how this <partnerLink> is being initialized - sref:service-ref is another concept introduced by BPEL 2.0. It defines a way to make a service reference in a process. Typically, local variables are used within <scope> to hold request and response data, in this case, reservationResult for each parallel iteration.
The <forEach> activity also supports the specification of a <completionCondition> to complete the loop when the specified condition is met without waiting for all branches to complete. This enables the "N out of M" join pattern where a process would dynamically poll a number of services but wants to continue without waiting for all responses to be received. We have often seen this in cases where a minimum of two price quotes are required or where a minimum of two human approvals are needed.
Change Management with Out-of-Band Events
Kitchens Online takes pride in its world-class customer service and a key business principle is to be as flexible as possible in every customer interaction. What this implies for our ordering process is that customers should be allowed to change their delivery schedule at any time before Kitchens Online actually initiates shipping. This is a very common requirement, not only in order processing but also in any business process where the state of the business environment can change at any time.
Kitchens Online uses <eventHandlers> to implement this requirement. Listing 2 illustrates the use of <eventHandlers> in the Kitchens Online scenario.
BPEL lets <eventHandlers> be specified for different scopes, allowing an event to be handled differently based on the state of the business process when it is received. For example, Kitchens Online's process includes the steps after shipping is initiated - shipping, delivery, and installation - in a separate scope from the rest of the process; the event handler is specified only in the scope containing the process prior to shipping (i.e., the <scope> main in Listing 2). Therefore, once the process reaches the shipping step, the event handler will be no longer listen to the modifyDeliverySchedule request. In this use case, a different event handler would handle delivery change requests that are received after shipping has been initiated - perhaps by tasking a customer service rep to call the customer and inform them that it's too late to change the delivery schedule without incurring some cost.
Besides out-of-band event handling as illustrated in the above example, another common scenario enabled by <eventHandlers> is where a process exposes a query or cancel operation that may be received at any time during the execution of a process. BPEL also supports waiting for events at specific points in a process using <receive> and <pick>.
In this example, when the BPEL engine receives the modifyDeliverySchedule event, many instances of the process may be active; how does it know which in-flight process instance the event should be sent to? This is where the <correlations> feature of BPEL comes in, which specifies the attributes of the event that are used to correlate it to a specific process instance. In the example above, we used the orderId and customerId attributes for correlation. In addition to event handling, <correlations> can be used for other asynchronous conversational patterns where the correlation of messages needs to be done based on the content of the message. Besides this concept of payload-based correlation, WS-Addressing has emerged as a standard for correlation in the WS stack; it's complementary to the BPEL standard and commonly supported by BPEL and other Web Services engine implementations.
Readers familiar with BPEL 1.1 may have noticed the use of <onEvent> in place of <onMessage>. This is a syntactic difference in BPEL 2.0 - use of <onMessage> is limited in BPEL 2.0 to <pick>. BPEL 2.0 also includes some clarifications on how variables, partnerLinks, and correlation sets are resolved for <onEvent>.
Exception Handling & Compensation
Although Kitchens Online tries to keep inventory information from its suppliers up-to-date, it can run into situations where a vendor is unable to supply an item in time to meet a committed delivery window. It's Kitchens Online practice to notify the customer of its inability to fulfill an Order within a 24-hour window.
Kitchens Online's business process is first to reserve inventory from all suppliers. If one or more suppliers come back with an out-of-stock notification or fails to confirm the reservation within 24 hours, Kitchens Online must release the inventory reserved from other suppliers and notify the customer.
Kitchens Online uses BPEL's <faultHandlers> and <compensationHandler> feature in conjunction with <onAlarm> to achieve the desired behavior as shown in Listing 3.
Some of the interesting aspects of the example above are:
• Asynchronous fault handling - Typically, we think of exceptions as faults and catch them as such. However, in this example, as in many other asynchronous interaction scenarios, the suppliers come back with their responses asynchronously any time within the 24-hour period. However, compensation in BPEL can only be invoked in certain activities such as <faultHandlers>. Therefore, after getting an asynchronous response, if needed, we do a <throw> to generate a fault. Timeout is handled similarly by doing a <throw> from within the <onAlarm>.
• Fault propagation - Notice that the cannotReserveInventory fault is thrown from several scopes but we catch it in the main scope. BPEL automatically propagates uncaught faults up to enclosing scopes, similar to how try-catch works in Java. BPEL 2.0 also features <rethrow> to propagate caught faults. The example above uses <rethrow> to enable other parts of the process to be terminated gracefully. (Note: the <exit> activity, previously known as <terminate> in BPEL 1.1, is typically not a good way to stop a process since it "kills" the process immediately without letting it do any recovery work or release any allocated resources.)
• Scope Snapshot - When a scope (such as processPayment) completes its work, all the local variables (such as chargeResponse) of the scope are saved in case of the need for compensation. Whenever the compensation of a scope happens, the saved state of local variables will be made available to the compensation logic (such as the reverseCharge service invocation).
If one supplier responds with an out-of-stock message, we release inventory from all suppliers without checking whether they had actually reserved it or not. Some readers may wonder if this would lead to incorrect inventory accounting on the vendor side. Such a check isn't needed because of the way compensation works in BPEL - a scope in BPEL is compensated only if it has successfully completed; therefore, scopes waiting to for inventoryReserved won't be compensated.
What's Next
The aforementioned article "BPEL's Growing Up" highlighted some of the interesting developments in the BPEL world, specifically around human workflow and process model round-tripping. By the time this article gets into print, we expect there to have been some significant announcements around human workflow. Another interesting development is the Service Component Architecture (SCA) standard. While most WS-* specifications have been focusing on the protocol and interoperability aspects of Service Oriented Architecture (SOA), SCA standardizes the deployment and assembly of SOA components, including BPEL. This will further help enable the portability of BPEL processes, and the composite applications that leverage them, across different vendor implementations.
Summary
Even prior to the publication of the BPEL 2.0 standard, BPEL had become the de facto standard for process orchestration. Now with the formal standardization process complete, we can only expect the adoption to accelerate.
The efforts of the BPEL committee over a three-year period have enhanced BPEL 2.0 with important features and clarified the specification in several areas. In this article we covered some of the most interesting new features in depth, however not every aspect of BPEL 2.0 could be covered. Readers interested in learning more about BPEL and BPEL 2.0 should refer to the WS-BPEL 2.0 primer (www.oasis-open.org/apps/org/workgroup/wsbpel/download.php/23974/wsbpel-v2.0-primer.pdf) and the WS-BPEL 2.0 specification (http://docs.oasis-open.org/wsbpel/2.0/OS/wsbpel-v2.0-OS.pdf). The primer is a good introduction to BPEL concepts, while the specification lets advanced readers reference detailed and precise semantics of the language.