Wednesday, 3 July 2013

MetaNet - Coping With Large Data and Ecosystems Using Meta-Computing


Rating and analyzing a single company is easy. A typical Balance Sheet or Cash Flow statement can have around 100 parameters, sometimes a bit more. Running an analysis of a small "ecosystem" of 10-20 companies is also feasible. Our quarterly analyses of EU economy takes into account the entire EU, with its 27 member states and 24 macro-economic variables per country. The EU ecosystem is therefore an ensemble of 648 variables with a very large number of interactions - in the order of 100000 or more.  Such analyses are performed using our QCM Engine OntoNet™ with runtimes typically under 1-2 hours. 

But what about larger ecosystems? Suppose the idea is to analyze all the listed companies on, say the Shanghai Shenzhen market which has approximately 900 listed companies. Imagine that we focus only on the balance statements of each company which contains around 100 entries. This leads to approximately 90000 variables and, potentially, 4 billion relationships. Not only runtimes would be astronomical, also memory requirements would make the whole thing impractical. How can you deal with that?

Ontonix has recently developed an application specifically for treating large ecosystems. It is a meta-application because it makes use of an application - OntoNet™. The tool is called MetaNet™ to reflect the fact that it is based on the concept of meta-computing.

MetaNet™ is written in a high-level language and reads thousands of input files which embrace the ecosystem in question. The application is based on an approximate algorithm which partitions the system into smaller chunks. Differences in final results, when compared to the "exact" method are in the order of 5-10%. This is perfectly acceptable, considering that Balance Sheet data is rarely "precise". 

The applications of MetaNet™ are numerous. However, our focus will be on analyzing stock markets. The objective is to perform the analysis of all the companies belonging to a particular stock market. The number of companies in the World's major markets are indicated in the tables below:




The largest markets contain approximately 5000 companies. Considering around 100 entries in a Balance Sheet, yields 500000 variables. The goal behind such analyses is to provide a systems perspective of a given market, pinpointing fragilities, sources of instability and establishing the footprint of a company on a market from a non-traditional point of view. The users of such information are large institutional investors and (large) investment funds. In a turbulent  economy new approaches are needed if one is to understand better why the global financial system is so fragile and so unstable.

The global financial system - if we reflect it in the World's major stock exchanges - looks more or less like this:

Considering that each of the nodes in the above Complexity Map is itself a system of many of companies, gives an idea of the magnitude of the problem. The total number of listed companies, based on the above tables, is approximately 46700. Again, considering on average 100 Balance Sheet entries per company, yields a staggering 4.6 million variables. A problem of such dimension requires a supercomputer in order to complete processing in a reasonable time-frame. However, it is our objective to establish the capability to perform such an analysis on a quarterly basis. More soon.