Information Warehouse 1.0 - Part 2

It all begin with their inability to understand human-friendly or unstructured data. Yes. I am talking about computers. The position of data becomes more important than the content or the context of the data in order to use it effectively with the computer. Rows and Columns become the norm of the day. A tremendous effort has been put in organising data in a structured way by companies wishing to utilize the value of their data. Despite spending enormous amount of money in converting data to a structured format, data, in its original format – unstructured / semi structured is still useful. A lot of companies still have the non-structured data that they simply do not have the time or money to convert to a structured format.

And then the question of looking at the data arises. Data itself becomes meaningless unless they are presented in a context understandable by humans, as information. Data stored, structured or unstructured has to be retrieved and shown with context as information to be of any value. Companies, having invested heavily in Business Intelligence or custom built applications, still cannot open up their data completely or efficiently as they wish to. The turn-around-time for a typical BI report is still measured in minutes, for instance.

Why does the traditional warehouse not fix the problem entirely? First, it has a hard (and expensive) time getting access to unstructured and semi structured data, and then it has limited ways of relating it to existing structured data. Add the limitations of the BI tool to this. And if you manage to do all of it, the query powering the reports at the front-end that show the information you are looking for, still takes a considerable amount of time doing it. Why the discrepancy in the query time? When, for instance Google can return results searching in a dataset considerably larger than any company can own, in less than seconds, why does the typical BI tool take much longer?

Lets see why.