UIMA and the Semantic Search

IBM’s Unstructured Information Management Architecture (UIMA) was released to the open source community in early 2006 when the entire source code was made available on Sourceforge. After spending more than a year at Sourceforge, UIMA is now a part of the Apache Incubator.

UIMA is pitched to become the first and only open-standard for unstructured information management. In very short, UIMA is a framework for building analytics solutions for the new world of structured-unstructured information sharing. Other frameworks like CALAIS are narrowly focussed on the Semantic Web technologies rather than providing a framework for building rich Text Analytics applications.

UIMA allows developers to build applications around technologies and chain the processes through its framework. Each component in the framework is an annotator. Consider for example an application that identifies person names in a text document. The algorithm can be implemented as an annotator that implements the UIMA interface (jCAS if you are using apache UIMA) for common analysis systems(CAS). A CAS is a general representation schema and can store arbitrary data structures for the analysis of documents. Using CAS, the span of annotation can be represented easily. The data can be passed through several Analysis Engines (AE) so far as each of them comply to the descriptor. Details on using UIMA and how to build Aggregate Analysis Engines are available here.

One of the most exciting engagement will be between UIMA and the Semantic Search. Semantic Search is the next generation of Search Technology using metadata (read information) created through Advanced Text Analytics and enabling ‘contextual’ search. The underlying technologies from NLP, Machine Learning, Statistics have existed for decades and explored to finer details by the research community. With the increasing adoption of enabling frameworks like UIMA, it is now easy to develop scalable solutions using Advanced Research Tools.

Some useful links to learn how UIMA can be used for building advanced text analytics solutions:
1. Background information on UIMA
2. UIMA and Semantic Search

Undercover information: When at IBM, I was part of the gang that developed ProAct – A UIMA based Customer Satisfaction Analysis technology.

Age of the Wikimania: the importance of online collaboration

Wikinomics

The Web 2.0 and the easy access to bandwidth has ushered in some new paradigms in collaboration. The rise of the wikipedia (in general the open source wiki frameworks such as mediawiki) and P2P frameworks have transformed knowledge sharing in the open-source (or should i say open-knowledge) world.

Continue reading