ApacheCon NA 2013

Portland, Oregon

February 26th – 28th, 2013

Register Now!

Thursday 10:15 a.m.–11:15 a.m.

Wengines, Workflows, and 2 years of advanced data processing in Apache OODT.

Chris Mattmann

Track:
Apache in Science
Audience level:
Intermediate

Description

  • Discuss OODT-215 and OODT-491 and its amazing new capabilities and features

  • Live demo of WEngine capabilities working for Snow Hydrology in the Western US project for the U.S. National Climate Asssessment

  • Discuss next steps in Workflow processing with Apache OODT

Abstract

With the advent of OODT-215 and OODT-491, there has been a tremendous amount of work to port our next generation Workflow Management system (cutely dubbed "WEngine" for "workflow engine") from an isolated branch into the mainline trunk.

The WEngine system brings amazing advantages including explicit support for branch and bounds in workflow models; prioritized thread pooling and queueing on a per task, and per workflow level; global workflow level conditions (pre and post); condition and workflow timeouts, and an entirely new and more descriptive state model complete with failure codes, and with checkpointing.

WEngine is currently processing the NPOESS Preparatory Project (NPP) PEATE testbed and its thousands of jobs per day, and is being slowly introduced into processing of an entire snow and ice climatology for the Western US and Alaska for the U.S. National Climate Assessment (NCA), working with the world's best snow hydrologists and snow scientists.

With all of those new features, what's an Apache OODT user and fan to do? How can you use WEngine in your system? How does it work today? How will it work tomorrow? We'll answer those questions and more in this fly-by-the-seat-of-your-pants exciting super talk!