CIMSS Seminar Series:
12 June 2007
Tying together large datasets, computing resources, and visualization with Origami
Maciek Smuga-Otto
In 2006, the SSEC demonstrated a lightweight web-oriented system for processing, management and visualization of hyperspectral data. This system, nicknamed Origami, integrated existing technologies that were heretofore not being fully utilized to handle the volumes associated with modern meteorological remote sensing instruments. The system took on-demand processing requests from end-users and distributed these requests across a compute cluster, automatically manageing data transfers to and from a centrally located Storage Area Network (SAN). Upon completion of the task, a status webpage (as well as notification email sent to the end-user) allows visualization of the end-product on the Unidata Integrated Data Viewer (IDV) through a single click, by means of the Java WebStart technology.
The original Origami demonstration was meant to showcase rapid distributed software development techniques that bind together existing resources. As a demonstration, it provided only one kind of processing task - namely, the computation of relative humidity fields from temperature and water vapor fields. This served to demonstrate an otherwise cumbersome task of comparing derived humidities from a weather model against humidities derived from a retrieval based on computed Top of Atmosphere (ToA) radiances.
The simple workflow and considerable data-management capacity demonstrated by the Origami prototype caught the interest of scientists wanting to see their own product generation tasks integrated into such a system. Since Origami was purposely designed to be as loosely coupled as possible, the addition of such new tasks is a natural fit for its architecture.
This presentation will give a general overview of Origami and the technologies it depends on, as well as an update on the tasks that have been successfully integrated into it.
