Business Activity Monitoring (BAM) with Complex Business Event Processing (BEP/CEP)

Business Activity Monitoring (BAM) has been for a while. However, it has been limited to real-time dashboard applications. Basically, it displays visual real-time graphs and charts based on KPIs or Key Performance Indicators. It lacks Complex Event Processing (CEP) capabilities that businesses require. Dashboards are just a tiny segment of BAM. Current BAM cannot meet the business requirements. For example, dashboards show only snapshots of information. It does not indicate what transitions have happened. In addition, managers do not keep watching them all the time. Vital events can be missed. Obviously, better framework is BAM with CEP.

Complex Business Activity Monitoring with Event Processing (CAMEP) combines the two as a single frame work. Rosella Analytic Platform is an integrated end-to-end developer environment which provides all the needed ingredients for CAMEP. It incorporates the followings;

  • BI: Statistics, KPIs, DW/DM, GIS, reporting, charting,...
  • Time-series modeling: Capture complex time-series events.
  • Expert systems: User-friendly SQL-based RME-EP rule specification language.
  • Predictive modeling: Neural network, decision tree, regression, time-series analysis, ...
  • Persistent fact management: Fast, full ACID transaction support, Real-time cross-platform data replication, ...
  • Event handling: Extensible event handlers can meet any business requirements, ...

BAM feeders collect information from operational databases, data warehouses, data marts, and from other sources, and feed it into RME-EP expert systems. RME-EP engine evaluates rules and invokes event handlers to generate business events. Rules are developed by business users and analysts.

RME-EP – Expert Systems Engine

The core of CAMEP is the RME-EP engine. Rete engine is a de facto industry standard technique for rule-based expert systems. It is also known as expert system shells. The problem with conventional implementations of Rete engines is that they are based on pattern-matching algorithms which are not only un-intuitive but also difficult to learn, especially, by non-IT users such as business analysts, medical doctors, and domain experts. As a consequence, rules have to be extracted from domain experts and then translated by IT knowledge engineers. This process not only takes time to develop but also makes it difficult to extract domain knowledge. The worst part is that detection of errors can be extremely difficult since domain experts cannot verify what the written rules exactly mean.

To alleviate this problem, RME-EP (Rule-based Model Evaluation - Rete) has been developed to use SQL-like language which can be learnt by business users and domain experts very easily and quickly. As a matter of facts, users already familiar with SQL can learn it within an hour! Non-IT professional users can write RME-EP expert system rules by themselves without IT professional help. RME-EP opens a new era for business users and domain experts.

SQL-like language for Business Activity Monitoring

The popularity of SQL as the preferred database query language has been largely due to the natural intuitiveness of the language as well as the richness of logical expressions. SQL is very easy to learn. Expressions written in SQL is very easy to understand and hence has been widely used by many non-IT professional such as business analysts and users, as well as by IT professionals. RME-EP is based on SQL-99 syntax supporting most logical and arithmetic language features of the language.

The following shows examples of RME-EP rules. These rules show how newly developing sales/market trends can be detected automatically and generate notification messages;

// SALES TREND MONITORING;
RULE 1: // if sales dropped over 5% with correlation coefficient 0.3 and over;
IF TIMESERIES(RR, LINEAR, 1, Month1, Month2, Month3, Month4)) >= 0.3 
   AND TIMESERIES(GROWTH_RATE, LINEAR, 1, Month1, Month2, Month3, Month4) < -0.05 
   THEN SET sales.trend AS 'declining' END;

RULE 2: // if sales increased over 15% with correlation coefficient 0.3 and over;
IF TIMESERIES(RR, LINEAR, 1, Month1, Month2, Month3, Month4)) >= 0.3 
   AND TIMESERIES(GROWTH_RATE, LINEAR, 1, Month1, Month2, Month3, Month4) > 0.15 
   THEN SET sales.trend AS 'increasing' END;

RULE 3: // alert if something detected;
IF sales.trend IS NOT NULL 
   THEN THROWEVENT('alert', Region, Channels, sales.trend) END;

The following rule shows how RME-EP rules can be used in medical diagnosis;

RULE 4:  // GENETIC DIAGNOSIS;
IF patient.gene.k12h2 PLIKE '*-C-*(2,4)-C-*(3)-[LIVMFVWC]-*(8)-H-*(3,5)-H-*'
       AND PREDICT("model1") USING(patient.age, patient.gender, ...) > 0.34
       THEN SET patient.diagnosis AS 'has cancer gene' END;

For more, please read RME-EP Rule Specification Language.

Event Handling

RME-EP engine evaluates rules and responds by generating events to event handlers. For example, when there is a violation of regulations, users may write rules that inform to relevant business managers. Event handlers are normally user provided functions. They may include;

  • Send email messages.
  • Send SMS text messages.
  • Log messages into files.
  • Log messages into databases.
  • Call other services.
  • and so on.

These functions can be easily implemented using extensible APIs.

Predictive Modeling

The core concept of RME-EP is Rule-based Model Evaluation (RME) of predictive models. RME-EP incorporates predictive models with many logical and mathematical expressions of SQL-99. By incorporating advanced predictive models into Rete engine, RME-EP provides a superb platform for advanced rule-based expert systems. RME-EP supports the following predictive models;

  • Neural network: Neural network is a very powerful predictive modeling technique. CMSR/RME-EP provides neural modeling tools which are very robust and intuitive but still very easy to use. Neural network can be used to predict numerical scores as well as classifications.
  • Decision tree: Decision tree is primarily used for classification task. In addition, CMSR decision trees can also predict statistical probability of events.
  • Regression: Regression is widely used technique by statisticians. It can predict numerical scores. RME-EP can include regression models in two ways: as mathematical functions and as a CMSR model.
  • Time-series regression and analysis: RME-EP has built-in time-series regression functions. It can explore various ways and find best regression functions based on a set of RME-EP facts.

For more, please read Predictive Modeling Software.

High Performance!

RME-EP uses compile-time analysis to build efficient run-time rule activation and evaluation environment. Rules are activated very efficiently. Facts are also managed and fetched using fast binary search techniques. Note that with binary search techniques, fetching a fact out of 1 million facts takes 20 only comparative steps at the worst case. High performance is the theme of RME-EP!

Applications of BEP BAM

BEP BAM or CAMEP powered by RME-EP is ideally suited for many mission-critical real-time tasks. The robustness of the rule specification language combined with advanced predictive modeling provides a superb platform for RTE applications. Real-time applications need high processing ratios. Embedded deployment of applications will provide the needed speed, if SOAP/SOA based deployment cannot meet! Applications of RME-EP may include;

  • Automated data mining.
  • Customer-churn detection.
  • Cross-sell advisor.
  • Algorithmic share trading.
  • Real-time transaction audit.
  • Account auditing.
  • Intrusion detection.
  • Risk management.
  • Fraud detection: insurance, healthcare.
  • Medical diagnostic expert systems.
  • and so on.