elemental links

Streambase continues to stack the deck with CEP Pioneers

If you are of the opinion that organizations with the most talent tend to win the day, and you have a business need that could benefit from Complex Event Processing (event-driven architecture, business event processing), you should keep an eye on Streambase.  In January, Colin Clark of Kaskad joined Streambase as VP for Customers, and this week, Mark Palmer of Apama joined Streambase as President and COO. 

For those that follow the CEP space, you'll find these hires interesting because both Colin and Mark are (colorfully) on the public record stating an SQL approach is not the only way to do CEP.  However, an SQL approach is at the core of Streambase.  So, obviously something compelled Colin and Mark to Streambase.  In my recent conversations with Colin, he has called out Streambase's customer base and his self-made mandate to demonstrate value every month of 2008.  Perhaps Colin will invite Mark to join our April call.  I'm definitely intrigued...

[Disclosure: Neither Streambase or Progress Apama are clients of my company, Elemental Links.  Kaskad was a client of Elemental Links]

ebizQ: The Role of Event Processing in Modern Business

Dr. K. Mani Chandy and Roy Schulte recently published an article on ebizQ entitled The Role of Event Processing in Modern Business.  I liked this article because it focuses on event processing (EDA, ESP and CEP) in the context of business.

The article describes three capabilities of an adaptive enterprise: situational awareness, sense and respond and track and trace:

"Situational awareness implies having an up-to-the-minute understanding of all critical aspects of your internal operations and the external environment. It is "knowing what is going on so you can figure out what to do" (Adam 1993). Sense-and-respond has a similar flavor, and in fact, the term is sometimes used interchangeably with situational awareness. However sense-and-respond emphasizes timely reaction to specified opportunities and threats whereas situational awareness connotes having a holistic view of many factors.

Track-and-trace is a bit different. It focuses on recording the status of identified items as they move between physical locations or through the steps of a business process. For example, logistic systems track-and-trace movements of physical goods in supply chains. They tell users where a shipment has been and where it is now, predict when it will arrive at a destination and can prove that it was delivered. The idea of track-and-trace can also apply to the movement of insurance claims, customer orders and other information-based items as they go through their life cycles."

From this business context, the article proceeds to discuss event processing definitions, EDA design patterns, patterns of response and the relationship of event processing and business intelligence, via BAM.

I found the article largely consistent with my own views and writing.  Of course, terminology varies.  But, that is something the event processing community is working together to address. 

The article is a precursor (promotion) for Gartner's Event Processing Summit, but a good read nonetheless.

[Disclosures: Elemental Links has a business relationship with ebizQ, none with Gartner.  Although I'd be happy to receive a press pass to the Event Processing Summit :) ]

Event-Driven Architecture Survey at ebizQ

The folks over at ebizQ are conducting a survey on Event-Driven Architecture.  The 15-question survey is designed to "assess the current state of the event processing market, and discover organizations current and future plans for employing EDA."

I'm working with ebizQ to analyze the survey results. The survey is open through August 19.   I believe the final report will be published in late September - but, don't quote me!

If event-driven architecture is something you are interested in, or pursuing, check out the survey.  Oh, speaking of surveys, the results of the Elemental Links/SOASTA testing survey will be published in August.  Those findings are interesting.  One, took me completely by surprise... 

[Disclosure: Elemental Links has a business relationship with ebizQ].

Event Processing Conversation Shifts from Research to Practitioners

[Update March 26, 2008 September 8, 2007 August 19, 2007 August 6, 2007 May 15, 2007 - updated blog list]

In March, as I was sending through a bunch of event processing (EDA, ESP, CEP) related links , JT posted that he sensed CEP was due for an inflection point.  I concur.  While event-driven architectures and event engines (streaming, complex) are not exactly new, the discussions around them became more prevalent and business/practitioner focused after Gartner placed EDA in the August 2006  hype cycle. (EDA is plotted on the "technology trigger" upslope, with a 5 - 10 year mainstream adoption.)

Instead of just streaming event processing related links through, I thought I'd take a minute to post on a collection of event processing related blogs that have good content for practitioners.  Some, like Marco on ESP, have existed for awhile. Others are new.  Here's my first pass at a list:

Aleri CEP Blog - Scott Groenendal, Jack Rusher, Jeff Wootton and friends  

Apama Blog  - John Bates, Chris Martins, and Mark Palmer of Progress' Apama

CEP Technology Blog  - Mark Tsimelzon of Coral8

Complex Event Processing - David Luckham, Father of CEP

Marco on ESP - Marco Seiriö of ruleCore

SOA and EDA - Jack Van Hoof, Enterprise Integration Architect

StreamBlog - Barry Morris of Streambase

Tibco Complex Event Processing - Tim Bass (and others) of Tibco

Complex Event Processing - Tim Bass of Tibco SilkRoad

Database Management Services and Text Technologies by Curt Monash

Alexandre Alve's Blog - Architect for BEA WebLogic Event Server

Event Processing Thinking - Opher Etzion, IBM and Event Processing Technical Society (EPTS)

Please share other blogs that should be listed here. 

Patent-ly Incorrect: Microsoft’s Event Stream Processing Patent Application

Yesterday, Microsoft’s patent application for Event Stream Processing was published.  This surprised me for a couple of reasons.  One, I had no idea Microsoft was working on ESP.  Two, the listed invention of a “Real time event stream processor to ensure up-to-date and accurate result” appears to be a patent of the fundamental concepts of event stream processing.  From the abstract:

“Identifying a transaction from a real time event stream having latency. A method of the invention receives events from the real time event stream where events define a plurality of transactions to be identified. Each of the transactions includes a first event and a second event; the first event has a first latency relative to the real time event stream and the second event has a second latency relative to the real time event stream. The first event is identified from the received real time event stream, and a time is record when the first event is received. The method determines a time period during which the second event occurs as a function of the first latency, the recorded time of the first event, and the second latency. The transaction having the first and second events is identified based on the determined time period of the second event.”

Ok, I can hear the cynics amongst you saying “and that surprises you, how?”  So, I guess what really surprises/confounds me, is that something like this, that has been worked on for years in research projects, and is implemented in production products, can be an invention of Microsoft, as of the June 9, 2005 application date.  Reminds me of the Y2K windowing patent, and of course, 1-click shopping.  This leads me to second (or 100^th) the recent question on techdirt But What are Software Patents Good For?

Anyway, I welcome insights from those better informed on software patents, and I encourage the Event Processing Community/Technical Society to discuss/challenge this patent.

OOPSLA Event-Driven Architecture Workshop - 10.22.2006 in the "other" Portland

Last week, during dinner with friends, I was asked about my upcoming travel.  I told them I was off to Portland next week, which got me a round of grief for traveling less than 30 miles from home.  I then had to explain, "the other Portland", the one on the left coast.  Although, there is no reason why a big technology conference shouldn't come here.

Anyway, next week I'll be attending OOPSLA, to participate in the Event-Driven Architecture Workshop on Sunday, 10.22.2006 and then attend some sessions on Tuesday 10.24.2006.  A session that looks like a must is Design Patterns: Beginnings and Futures -- a panel discussion featuring the surviving members of the GOF (Erich Gamma , Richard Helm, Ralph Johnson), in honor of John Vlissides.

The way the conference is setup, for the first time ever, I'll have a free day to explore another city.  Or at least write in a location more interesting than the hotel.  If you're attending OOPSLA and want to meet up, let me know: bda at elementallinks dot com.

OMG's Event-Driven Architecture DRAFT RFI available for public review

Updated 8.23.2006 at 11:02 to insert "DRAFT" into the post title, and add the following clarifying note:

The linked document is a "work in progress", and not (yet) an official RFI.  In other words, please review the RFI for content (additions, changes, deletions), not for submitting a response.  Thanks.

[The original post starts here]

The Object Management Group’s SOA SIG is looking for community (practitioner, vendor, researcher, observer) feedback on a draft request for information (RFI) on event-driven architecture (EDA) and its relationship with service-oriented architecture (SOA) & business-process management (BPM).

Here’s the RFI Summary:

The EDA Sub-group of the OMG SOA SIG seeks information from members of the EDA, BPM and SOA community as well as anyone interested in promoting standards in this area. Requested information will be evaluated by the EDA Sub-group, resulting in the development of Requests for Proposal(s) (RFP) for standardization of Event definition, relationship between EDA, BPM and SOA that will ultimately allow development of standards for Complete Life Cycle of Events -Ontology of Events, Sense and Respond Services, Events Metrics and processing of complex events. Please note that it is our intent to develop modeling standards for the EDA/SOA and EDA-Business Process interaction and provide standards for the implementation of that interaction as well.

For those (like me) not completely familiar with the OMG process, here’s the description of the RFI process taken from section 8 of The OMG Hitchhiker’s Guide, V7.3:

The intent of the Request for Information (RFI) is to gather information for the purpose of guiding a subgroup in its efforts to provide solutions to industry problems. The RFI is an optional process used by a subgroup to canvass a targeted industry segment for one or more of the following purposes:

•                
  • Soliciting assistance in identifying potential technology sources.
                 
  • Soliciting input to and validate a subgroup’s roadmap.
              

  • Acquiring general or specific information about industry requirements.    

Generally speaking, the RFI process determines which Request For Proposals (RFPs) get issued (and, based on negative feedback, which don't) or influences the way a particular RFP is constructed.

There are no restrictions on who may receive or respond to a RFI. Both OMG Members and non-members can respond. RFI Responses may include information about relevant technologies, products, standards, research, requirements, and other guidance for the subgroup.

If you are interested in EDA (simple, stream, complex), take a look at the RFI and provide comments to Dr. Harsh Sharma. The plan calls for a final review of the RFI at the OMG’s September Technical Meeting in Anaheim.

I suppose this is where I say “I’m going to Disneyland!” I am a guest contributor (no $ exchanged either way) to the EDA subgroup of the OMG SOA SIG.

David Luckham on Complex Event Processing (CEP) and Event Stream Processing (ESP)

David Luckham, the father of CEP, just published an article What's the Difference Between ESP and CEP?   In the article, Professor Luckham discusses the origins of CEP and ESP, the differences between streams and clouds, where ESP engines are today, how CEP and ESP will be less different over time, and the challenges going forward. 

The article is a must read for anyone interested in event processing (simple, stream, complex).  But before you go over, I have a quick thought on the challenge of “New Horizons”, described by Professor Luckham as follows:

New Horizons: The first challenge is to expand the areas to which event processing is being applied. We have to educate the IT community, and a lot of other communities, about its potentials, and that will involve a lot of proof of concept work. So far, the early adopters have been people who already know they need real-time event processing and their problems are usually well formulated. It is true that new applications are appearing all the time – in areas involving RFID, eRetailing and so on. But there are other areas where event processing could be applied…

…There are huge clouds of events from multiple disparate sources in Homeland Security², Epidemiology, Global Warming and the Environment, just to mention three areas. We need to demonstrate that event processing can be applied to challenging problems in these areas. For example, could Homeland Security use telephone surveillance data to enhance monitoring bank transfer events on SWIFT networks for money laundering? Another example, there are some very imaginative experiments going on in medical epidemiology. It turns out you may be able to predict ‘flu outbreaks earlier by monitoring over-the-counter medication sales than by monitoring doctor’s reports. What about analyzing a lot of event sources for early prediction of epidemic outbreaks? And the world itself is becoming an event generating globe with sensors and probes for deep ocean pressure monitoring for Tsunami warning, fault monitoring for earthquake studies, forestry monitoring, etc. All of these events are available via satellite. What are the possibilities for event processing here?

The big question is what problems (business, social, government, science, technology) can/should event processing solve?  But to answer that, we first need to understand the possibilities.  Not only what events are swirling around in clouds, and floating in streams, but how those events connect, and what are appropriate responses. 

This will require tapping into folks with deep domain knowledge, a knack for ‘pattern recognition’ and ‘connecting the dots’, and the ability to influence change.  My early thinking is this person is a true ‘business architect’.  A business person who architects (designs, creates) business models, processes, information flows, rules and policies. 

If you are an IT person, considering event processing, find a ‘business architect type’ partner to explore the new horizons.  Event processing isn’t just an IT thing.  Read the article.

StreamBase's Da Vinci Coder Contest

Looking for an interesting way to learn more about event stream processing?  Could you use a laugh after a long day of trying to get something done between meetings and email?  Then, you should check out StreamBase's Da Vinci Coder Contest, and the related short film, a parody on the Da Vinci Code.

The first part of the contest is a series of weekly "Jousts".  To complete each joust, you need to discover secrets within StreamBase developer edition, the video, or "other" (easy to find) resources.  This week is joust #2.  Each weekly joust awards a prize ($1,000 range).  If you're so inclined, you can choose a charitable donation.

The second part of the contest, starting August 14, is the "Grand Tournament".  This is a StreamBase application coding contest to win the coveted (?) title of "Da Vinci Coder" and a bigger prize ($10,000 range).

StreamBase, if you don't know, is Mike Stonebraker's latest company.  No surprise, StreamBase takes a database approach to the event/information stream processing problem.  StreamBase uses an SQL derivative (StreamSQL) to perform in-stream complex event processing.  The StreamBase product is based on Mike Stonebraker's Aurora project.

I had a chance to meet with the StreamBase team earlier this year, and was definitely impressed.  This paper does a good job explaining the requirements for an event stream processing engine.

OMG SOASIG Meeting: Quick Chat with Rhysome's Bob Covington - Event Sensors

During the afternoon caffeine break of the OMG SOASIG meeting, I sat down with Bob Covington, CTO and co-founder of Rhysome, a provider of sense and respond (event-driven) technology.  In our quick conversation, I learned about Rhysome’s founders (former enterprise IT practitioners), how Rhysome’s products evolved, and what (in my view) makes them unique.

Rhysome’s original mission was to create a real-time reporting tool.  However, the team quickly recognized a gap – sensing and delivering relevant real-time information.  This led the team to the broader event-driven space: event detection, collection and processing. 

What’s unique about Rhysome is the focus isn’t solely on event processing.  Rhysome starts at the source – event detection and collection.  See left hand side of illustration copied from Rhysome’s site.

Rhysome describes the “smart cyber-sensors^tm” as follows:

• Detects and collects very small changes in both data and metadata from each information source in an enterprise, in real-time, using a native Event-Driven Architecture.

• Supports a wide range of information sources such as files, database transactions and schemas, logs (device, system application, and transactions), emails (both enterprise and web mail) instant messages (IM), SOAP messages, messages queues, HTTP, and Web content.

• Smart Cyber-Sensors run in the background with negligible or no impact on client or system performance, and no use of invasive database triggers.

• Dynamic filtering techniques remove non-relevant information at the source, significantly reducing implementation requirements and system overhead.

While designed as part of the broader Rhysome solution, the cyber-sensors can (and have) been incorporated in applications that require non-invasive tracking (think security and compliance) without the downstream complex event processing. 

When you start to think of the possibilities, mixing and matching sensors, services, events, business processes, information sources, and user interfaces, things get interesting.  With standards, to ease the interactions, it is even probable. 

For more on Rhysome, see their site.  [Shaded lenses recommended. :)]

[Note: This post was republished (unchanged) on July 13, 2006 at 3:22 due to Typepad system problems]