COMBINING BUSINESS ACTIVITY MONITORING WITH THE
DATA WAREHOUSE FOR EVENT-CONTEXT CORRELATION
Examining the Practical Applicability of this BAM Approach
Gabriel Cavalheiro, Ajantha Dahanayake
Faculty of Technology, Policy and Management, Delft University of Technology, Jaffalaan 5, Delft, The Netherlands
Richard Welke
Center for Process Innovation, Georgia State University, Atlanta, GA, USA
Keywords: Business Activity Monitoring, Data Warehouse, Real-Time Event, Historical Data, Push-Based,
Performance-Indicators.
Abstract: Business Activity Monitoring (BAM) is a term introduced by Gartner, Inc to define systems that serve to
provide real-time access to critical business performance indicators to improve speed and effectiveness of
business operations. Despite the emphasis of BAM on the provision of low latency views on enterprise
performance, literature on BAM also indicates the technical feasibility of a BAM approach, which adds
context from historical information stored in a data warehouse to real-time events detected by a BAM
system so as to help enterprises improving understanding of current monitoring scenarios. However, at this
point, there is a lack of studies that discuss the use of this approach to tackle real-world business problems.
To improve practical understanding of the potential applicability of this BAM approach, this paper will
present a synthesis of existing research on BAM and data warehouse to provide an objective basis for
proposing feasible business scenarios for applying the combination of both technologies. This study reveals
that the noted BAM approach empowers operational managers to respond in a more precise manner to the
occurrence of events by enabling a better understanding of the nature of the detected event.
1 INTRODUCTION
Enterprises are currently exposed to rapidly
changing market conditions and increasingly
competitive business environments (Luckham,
2002). To ensure competitiveness, enterprises need
to be able to maximize revenue generation and cost
savings when it comes to the execution of their
business processes. Since the occurrence of
exception events of interest in the transactional
systems, such as unusual large orders, delays,
situations of high risk, or unavailability of resources,
have a significant economic impact on the
performance of business processes, enterprises need
to respond adequately to such events as they occur.
However, this requires the availability of an
infrastructure that provides real-time visibility into
the business operations of enterprises.
To enable enterprises minimizing response times
to events, software vendors started developing a new
set of functions. Gartner, Inc introduced the term
Business Activity Monitoring (BAM) to define this
new set of functions (Cavalheiro, 2005).
Fundamentally, BAM systems are push-based
systems that translate input events into real-time
analysis that is pushed on recipients for immediate
reaction upon the occurrence of events (Govekar et
al, 2002). The provision of real-time access to
business performance indicators requires gathering
and analysing business events from multiple and
heterogeneous data sources to detect exception
conditions and generate low latency alerts to enable
business managers responsible for business
processes to make well-informed decisions quickly
(DeFee and Harmon, 2004).
Although the concept behind BAM systems
emphasizes the provision of real-time operational
insights, some studies acknowledge that the
combination of real-time events with historical data
can help enterprises diagnose problems in current
monitoring scenarios (McCoy, 2002). Nonetheless,
at present time, there is a lack of studies addressing
the practical applicability of this business
monitoring approach. The purpose of this paper is
thus to contribute to fill this gap in the literature by
263
Cavalheiro G., Dahanayake A. and Welke R. (2006).
COMBINING BUSINESS ACTIVITY MONITORING WITH THE DATA WAREHOUSE FOR EVENT-CONTEXT CORRELATION - Examining the Practical
Applicability of this BAM Approach.
In Proceedings of the Eighth International Conference on Enterprise Information Systems - DISI, pages 263-268
DOI: 10.5220/0002449202630268
Copyright
c
SciTePress
presenting a synthesis of existing research on BAM
and data warehouse to provide an objective basis for
proposing the application of this approach to address
real-world business scenarios. This is accomplished
by examining technical characteristics of both
concepts and by proposing practical combinations
BAM and data warehouse functions.
This paper utilizes the insights obtained by
conducting a graduation project, which involved
collaboration between the faculty of Technology,
Policy and Management at Delft University of
Technology, Delft, The Netherlands, and the Centre
for Process Innovation (CEPRIN) at Georgia State
University, Atlanta, USA.
2 TECHNICAL
CONSIDERATIONS ABOUT
BAM
Essentially, BAM systems analyse real-time events
to identify problems, diagnose them, and generate
alerts to recommend managerial action. BAM
systems can be incorporated into the IT
infrastructure of enterprises as an event analytics
layer at the top of existing middleware infrastructure
(Govekar et al, 2002). In practice, BAM systems can
be deemed as rule-based systems that detect
significant real-time events. These real-time events
can be characterized as instantaneous state changes
in a condition from one value to another occurring in
transactional systems (Delgado et al, 2004). BAM
systems serve as a platform upon which event-driven
applications can be developed by defining patterns
to represent events with significant economic impact
on business processes. To support the development
of event-driven applications, BAM systems are
likely to include event-modelling functions to define
and validate event patterns of interest (Gassman,
2004). This makes BAM systems highly adaptable,
as new event-driven applications can be developed
rapidly to address new or changing business
problems.
The investment committed by enterprises on the
adoption of middleware technologies makes it
possible that, in BAM systems, events come from
multiple and heterogeneous data sources, which
include both internal sources as well as external
sources such as the Internet (McCoy et al, 2001).
Essentially, BAM performs multi-application event
analysis by correlating events originating from
multiple and independent sources (McCoy, 2004).
To represent BAM systems, Gartner, Inc has
proposed a three-layer model, which is comprised of
an Event Absorption Layer, an Event Processing and
Filtering Layer, and an Event Delivery and Display
Layer (Govekar et al, 2002). In this model, the
border of a BAM system is the interface with the
recipients of BAM alerts. Figure 1 provides an
illustration of each layer and the recipient. In the
following we briefly describe each layer.
Figure 1: Layers of a BAM system.
2.1 Event Absorption Layer
Events are fed into the event absorption layer by an
event acquisition tool. The source of event messages
for BAM will most often be business or process-
related, however, technical events that occur during
the operations of the IT infrastructure may also be
collected (Gassman, 2004). BAM systems rely on
event gathering mechanisms to gather event
information, in real-time, directly from transactional
systems. A BAM system should have a mechanism
in place to gather events occurring concurrently in
different transactional systems. This is enabled by
the middleware layer that has been placed over the
last decade to support interoperability among
disparate transactional systems.
2.2 Event Processing and Filtering
Basically, after gathering real-time events, BAM
systems must be able to process and correlate
multiple sources of independent data (McCoy,
2004). To this end, a filtering system must be in
place to draw data from a wide range of sources and
then compare those real-time events against rules
that when met generate alerts (DeFee and Harmon,
2001). These alerts should contain information about
nature of the problem associated with the occurrence
of the event in order to empower the recipient of the
alert, the BAM recipient, to initiate an immediate
and appropriate response (Hellinger and Fingerhut,
2002).
2.3 Event Delivery and Display
The alerts issued by a BAM system can be sent to
diverse parties that have real-time decision support
needs. The alerts that are raised can populate a
ICEIS 2006 - DATABASES AND INFORMATION SYSTEMS INTEGRATION
264
display or trigger an action (Gassman, 2004). In this
way, alerts that are used to populate a display, are
often delivered via graphical displays
(“Dashboards”) that are customized for use in
different parts of the enterprise and for different
audiences (McCoy et al, 2001). Another option is to
forward the alerts to BAM recipients through other
existing mechanisms such as batch office staff,
emails, pagers, PDAs, and other systems to ensure
that someone or something can react (McCoy,
2003).
The alert can also be used to trigger an action
executed by a Business Process Management (BPM)
system. In this respect, controlling the reaction cycle
will probably be the role of BPM tools (McCoy,
2003). Here, BAM can be part of a BPM, where it
generates an input for the BPM system that triggers
a workflow corresponding to a predefined sequence
of events (Gassman, 2004). The BPM tool can react
to the BAM alert by running a chain of tasks that
alter a running business process (McCoy, 2004).
2.4 BAM Recipients
The alerts issued by a BAM system can be sent to
diverse BAM recipients (Govekar et al, 2002). The
alerts that are delivered by a BAM system can
populate a graphical display or trigger an action
(Gassman, 2004). In this way, the alerts that are used
to populate a display are often delivered via
“Dashboards” that are customized for use in
different parts of the enterprise and for different
audience (McCoy et al, 2001). Another option is to
forward the alerts to BAM recipients through other
existing mechanisms such as batch office staff,
emails, pagers, PDAs, and other systems such that
someone or something can react (McCoy, 2003).
Additionally, the alert can also be used to trigger
an action executed by a BPM system. In this respect,
controlling the reaction cycle will probably be a role
of BPM tools (McCoy, 2003). Here, BAM can be
part of a BPM, where it generates an input for the
BPM system that triggers a workflow corresponding
to a predefined sequence of events (Gassman, 2004).
As such, the BPM tool can react to the BAM alert by
running a selected set of processes that alter a
running business process (McCoy, 2004).
3 APPLICATIONS OF BAM
BAM systems are applicable to monitoring
situations that require very low latency operational
insights on the execution of business processes.
Essentially, in BAM systems, the emphasis is on
responding to events within short time-windows of
opportunity (Chandy and McGoveran, 2004).
Consequently, BAM systems must generate alerts in
real-time, otherwise the value of the system is lost
(McCoy, 2003). DeFee and Harmon (2004) define
effective BAM systems as those systems that
provide managers with sufficient operational
insights close to real-time so managers can take
decisions in time to affect the ongoing performance
of the process flow. A number of examples of the
need for real-time operational insights is provided by
literature. Table 1 shows a set of representative
examples of such needs.
Table 1: Examples of the need for real-time Operational
Insights.
Real-Time Operational Insights Source
A retailer that needs to monitor
hourly sales levels
(Hackthorn,
2004)
A financial system BAM tool issued
an alert on a stock volume increase
or decrease beyond a threshold level
(McCoy, 2003)
Provide a real-time view on supply
chain metrics
(McCoy et al,
2001)
Monitor service level agreements
(SLAs)
(Luckham,
2002)
4 TECHNICAL
CONSIDERATIONS ABOUT
THE DATA WAREHOUSE
In practice, the data warehouse can be deemed as a
workflow that involves periodically gathering
disparate data and cleansing, transforming, and
integrating that data according to business rules store
in metadata repository. Then, the data is loaded into
persistent data structures where it can be analyzed
(Inmon et al, 2001).
The data stored in a data warehouse represents
historical data in the sense that it represents events
now passed (Inmon et al, 2001). The data warehouse
is designed to store fact for each time period,
thereby creating a historical perspective on
performance (Tanler, 1997). It can create a single
subject-oriented collection of information by
assembling the data from heterogeneous databases
(Thuraisingham, 1999). This allows business
analysts to query only a single point consisting of a
repository from multiple sources. In order to provide
a single version of the enterprise operational data,
the data warehouse need to integrate data from
multiple operational systems of the enterprise and,
as data warehouse environments mature, it brings in
data from an ever-expanding set of sources (Inmon
et al, 1999). To do this, the data warehouse typically
relies on a software to carry out the extract,
COMBINING BUSINESS ACTIVITY MONITORING WITH THE DATA WAREHOUSE FOR EVENT-CONTEXT
CORRELATION - Examining the Practical Applicability of this BAM Approach
265
transform, and load (ETL) process. New or changed
transactions (fact records) are moved and
dimensions are captured as point-in-time snapshots
for each load (Kimball and Caserta, 2004). In fact, a
new snapshot is created whenever a change needs to
be reflected in the data warehouse (Inmon et al,
2001). A fundamental consideration about the ETL
process, though, is that it depends on the availability
of windows of acceptable downtime for the
transactional systems, because it places an additional
load on the transactional systems during the loading
process. In this way, the latency associated with the
data warehouse makes it unsuitable to provide real-
time operational insights.
The snapshots captured by the ETL software are,
then, sent to the operational data store (ODS). The
ODS was originally defined as a place where data
was integrated and fed to a downstream data
warehouse (Kimball and Caserta, 2004). The ODS
environment has a time period identical to that of the
application. The difference between the ODS and
the application is that the ODS contains integrated
corporate data, and the applications do not (Inmon et
al, 2001).
A data mart is a collection of data that is
adequate to fulfil the decision support needs of a
particular department. It is a subset of a data
warehouse that that is generally customized to fit the
needs of a department (Inmon et al, 2001). Data
Marts can be a subset of the enterprise data
warehouse (Brown and Hill, 2000). The data marts
that house data for various departments contain
different combination and selections of the same
detailed data found at the data warehouse (Sousa,
1999). They provide detailed information focused on
a single area, such as marketing, sales, production,
or finance (Brown and Hill, 2000). From the data
warehouse, atomic data flows to various departments
for their customized usage. These departmental
databases are called data marts. A data mart is a
body of data for a department that has an
architectural foundation of a data warehouse (Sousa,
1999).
5 APPLICATIONS OF THE DATA
WAREHOUSE
Typically, the main purpose of the data warehouse is
to supply business analysts, such as managers and
decision-makers, with information they need, so as
not to disturb transactional systems processing
transactions rapidly and reliably (Abramowicz et al,
2002). Specifically, data warehouses are generally
highly adequate to fulfil decision support needs
when there is a clear need for long-term trend
reports. A data warehouse is designed to support
exploratory analysis through adhoc data analysis,
inquiry and reporting by end users. The integrated,
detailed data and the robust amount of history found
at the data warehouse are ideal for this
comprehensive business and data analysis (Inmon et
al, 2001).
A great number of examples of the exploratory
analyzes based on data warehouse is presented by
scientific literature. Table 2 lists some of the most
interesting applications of the data warehouse.
Table 2: Examples of exploratory analyzes supported by
the data warehouse
.
Example of Analyzes Based on a
Data Warehouse
Source
Managers need to view sales by
product and region and make
correlations with advertising
campaigns and marketing promotion.
(Brown and
Hill, 2000)
Answering questions such as,
• What type of customer is the
most profitable for our
business?
• Over the years, how has
transaction activity changed?
• Where has sales activity
been highest in the
springtime for the past three
years?
• When we change prices, how
much elasticity is there in the
market place?
(Inmon et al,
2001)
The most frequent use of a data
warehouse is to analyze historical data,
discover trends and correlations and
project events forward into the future.
(Meltzer,
1999)
Use data warehouse to analyze
business data historically by focusing
on planning such as trends in daily
sales levels as compared with previous
months.
(Hackathorn,
2004)
6 LINKING BAM AND THE DATA
WAREHOUSE: EVENT-
CONTEXT CORRELATION
Besides analyzing real-time operational information,
BAM systems can also support event-context
correlation. This is an important aspect of BAM
systems because the addition of contextual
information to an event provides additional
information about the nature of the situation that is
being monitored. For example, the definition of a
certain filtering criteria to detect exceptional events
can be seen as a basic form of adding context to
BAM alerts, which are defined by Gassman (2004)
ICEIS 2006 - DATABASES AND INFORMATION SYSTEMS INTEGRATION
266
as events with context. So, by specifying a threshold
for a certain performance-indicator, which is
represented by an event property, the value of the
threshold can be regarded as contextual information.
However, more sophisticated ways of adding context
to events are usually required. Since the decision
support need of the BAM recipient may require the
diagnose of problems to respond to events with high
accuracy or event forecast data to guide anticipated
response to future problems, just raising alerts
indicating the occurrence of exceptional situations
can be, in many monitoring situations, insufficient to
ensure a proper response.
Data warehouses can be employed to add context
from a historical store for business activity to a real-
time alert. This can be done by building models that
correlate patterns of real-time events with previous
occurrences of similar events representing both
operational problems and opportunities. For
example, an hourly trend of expected order volume
may be updated nightly by a data warehouse and
used as a reference against real-time orders to detect
exceptional variations in order volumes (Gassman,
2004). Additionally, BAM systems can improve
long-term analysis by adding the context of real-time
feeds into an Operational Data Store (ODS)
(Hackthorn, 2004). This enables a more complete
understanding of trends, because it supports an
analysis of trends by providing understanding of
current states.
7 APPLICATION OF THE
PROPOSED BAM APPROACH
After understanding differences and similarities
between BAM systems and data warehouse, we
highlight a set of generic monitoring situations that
can be addressed through the combination of real-
time events with historical data. These monitoring
situations were identified in the course of a
graduation project. Table 3 illustrates the problems
and analyses of historical data and real-time event
information.
Table 3: Monitoring situations that can be addressed by the proposed BAM approach.
Brief Problem Description Data Warehouse BAM
Benefit from opportunities in Customer Relationship
Management (CRM) systems by selecting customers for
preferred treatment based on purchase records of
customers and provide a higher level of service to the
type of customer that is most profitable for the business.
Identify what type of
customer is the most
profitable for the business.
Generate alert to notify the
need to provide service
privileges to the selected type
of customer.
Health surveillance and disease control can be
empowered to detect early signs of emerging epidemics.
In this way, patient care decisions can be adjusted when
patterns of previous epidemics are detected.
Identify patterns of patient
admissions in previous
epidemics
Integrate patient admission
data from several hospitals
and searching for pattern that
matches a historical pattern of
previous epidemics.
Refine sales forecast models by adjusting forecasts
taking exception events into account.
Extrapolate sales levels
from historical data.
Produce new forecast when an
exceptional high order is
placed that affects the forecast.
Support investor’s trade decisions to buy and sell shares
in companies.
Calculate historical prices
of shares.
Generate alerts when prices
are above or below historical
levels to indicate the need to
buy or sell. Include news and
forecast in analysis.
Optimize purchase of products in a supply chain. Calculate historical market
prices of products.
Generate alerts when current
prices approximate historical
limits.
Support retailer’s inventory management.
Calculate historical
averages for demand for
products and services
Identify peaks in demand for
products or sales from
historical data.
COMBINING BUSINESS ACTIVITY MONITORING WITH THE DATA WAREHOUSE FOR EVENT-CONTEXT
CORRELATION - Examining the Practical Applicability of this BAM Approach
267
8 CONCLUSIONS AND FUTURE
WORK
We have presented an assessment of the practical
applicability of a BAM systems and data warehouse.
This paper provides substantial evidence that the
combination of real-time events with historical data
can help improving understanding of the nature of
the current problems, which leads to a better support
for rapid and adequate response. Additionally, we
could identify opportunities for further research. We
believe that a very important direction is the need for
a thorough search for industry specific problems that
are likely to require the noted BAM approach.
Another area that requires further research concerns
the quantitative measurement of the benefits
generated by this BAM approach. Although the
benefits that can be obtained through this BAM
approach may seem evident, it is necessary to
generate grounds for conducting cost benefit
analyses.
REFERENCES
Abramowicz, W., Kalczynski, P., and Wecel, K., 2002.
Filtering the Web to Feed Data Warehouses, Springer,
London.
Brown, J., and Hill, P., 2000. Data Marts; Key to Reviving
the Enterprise Data Warehouse. In SCN Education,
2001. Data Warehousing: The Ultimate Guide to
Building Corporate Business Intelligence, SCN
Education, Venendaal.
Cavalheiro, G.M.C., 2005. Defining Business Activity
Monitoring: Understanding a Real-Time Event-Driven
Infrastructure, MSc Thesis, Delft University of
Technology.
Chandy, M., and McGoveran, D., 2004. The Role of
BAM. Business Integration Journal, Retrieved
Septembre 03, 2005 from:
http,//www.bijonline.com/PDF/chandy%20role%20of%20
bam%20april.pdf
Defee, J.M. and Harmon, P., 2004. Business Activity
Monitoring and Simulation, Business Process Trends,
White paper.
Delgado, N., Gates, A.Q., and Roach, S, 2004. A
Taxonomy and Catalog of Runtime Software Fault
Monitoring Tools. IEEE Transactions on Software
Engineering, Volume 30, Issue 12.
Gassman, B., 2004. How the Pieces in a BAM
Architecture Work, Gartner Document, TU-22-3754.
Govekar, M., McCoy, D., Dresner, H., and Correia, J.,
2002. Turning the Theory of BAM into a Working
Reality, Gartner Document, COM-14-9785.
Hackathorn, R.D., 2004. The BI Watch: Who’s on First.
Issue of the DM Review. Retrieved July 12, 2005
from: http://www.bolder.com/pubs/DMR200403-
Who%20is%20on%20First.pdf
Hellinger, M., and Fingerhut, S., 2002. Business Activity
Monitoring: EAI Meets Data Warehousing. Business
Integration Journal. Retrived August 06, 2005 from:
http://www.bijonline.com/pdf/BAMFingerhut.pdf
Inmon, W.H., Imhoff, C., and Sousa, R., 2001. Corporate
Information Factory, Wiley, New York, 2
nd
edition.
Inmon, W.H., Rudin, K., Buss, C.K., Sousa, R., 1999.
Data Warehouse Performance, Wiley, New York.
Kimball, R., and Caserta, J., 2004. The Data Warehouse
ETL Toolkit: Practical Techniques for Extracting,
Cleaning, Conforming and Delivering Data, Wiley,
New York.
Luckham, D., 2002. The Power of Events: An Introduction
to Complex Event Processing and Distributed
Enterprise Systems, Addison Wesley, San Francisco.
McCoy, D., 2003. Blending Business Process
Management and Business Activity Monitoring,
Gartner- Strategic Planning, RU.
McCoy, D., 2002. Business Activity Monitoring: Calm
Before the Storm. Gartner, LE-IS-9724.
McCoy, D., 2002. Business Activity Monitoring; The
Merchant’s Tale, Gartner- Case Studies, CS-16-1780.
McCoy, D., 2004. The Convergence of BPM and BAM,
Gartner, SPA-20-6074.
McCoy, D., 2003. Blending Business Process
Management and Business Activity Monitoring,
Gartner- Strategic Planning, RU.
McCoy, D., Schulte, R., Buytendijk, F., Rayner, N., and
Tiedricht, A. 2001. Business Activity Monitoring: The
Promise and Reality, Gartner, COM-13-9992.
Meltzer, M., 1999. Getting Started; Building the Scalable
Warehouse the Right Way. In SCN Education, 2001.
Data Warehousing: The Ultimate Guide to Building
Corporate Business Intelligence, SCN Education,
Venendaal.
Sousa, R., 1999. Data Warehouse Performance, Wiley,
New York.
Tanler, R., 1997. The Intranet Data Warehouse: Tools and
Techniques for Building an Intranet Enabled Data
Warehouse, Wiley, New York.
Thuraisingham, B., 1999. Data Mining; Technologies,
Tools and Trends, CRC Press, New York.
ICEIS 2006 - DATABASES AND INFORMATION SYSTEMS INTEGRATION
268
