How do we generate insight and understanding from a multitude of market data?

Drawing conclusions from market data is difficult. We think we have the tiger by the tail! There is a crush of data, over many products, and many exchanges dispersed over time. It is easy to drown in this data. With strategy and planning, we tamed the data by dividing the effort into two problems: storage and calculation.

At the outset our aim was a system where database and calculation requirements are discretely seperated. This allows us to approach both problems with full flexibility. Here are some important design features we took into account in building