In a world where data is growing exponentially, increasing complexity has the potential to overwhelm those who don’t have the technology to try to make sense of it.
When I started trading, more than 20 years ago, I had just one alpha and I guarded it with my life. As I write this, my firm will soon pass 6 million alphas — i.e., algorithms or models that seek to predict the prices of securities — and I no longer have to worry about any of them like I used to.
The exponential growth in the number of alphas at WorldQuant is representative of a much larger trend. Each day the world gives birth to 2.5 quintillion bytes of data, created by everything from satellites and social media to credit cards and smartphones. Research firm IDC forecasts that the total volume of data will grow from 44 zettabytes in 2020 to 180 zettabytes by 2025.
In a 2012 report titled “Big Data, Big Impact: New Possibilities for International Development,” the World Economic Forum outlined how government organizations and companies could turn data into an economic development tool. Five years later Big Data is old news. What’s new is prediction.
For anything you’re trying to predict — securities prices, retail sales, unemployment, the weather — the amount of data available is growing exponentially. As a result, following certain mathematical laws, you get a linear increase in predictability as long as the data continues to grow at a nonlinear rate. All things held constant, everything will become more and more predictable from the perspective of the data.
But there are countervailing forces at work. The exponential growth in data is making the world increasingly complex, as the frequency of events accelerates and as the world gets more and more chaotic geopolitically. Even though people can predict events with greater accuracy, everything is getting more complicated. Unless you’re able to predict things better, the complexity is going to overwhelm you.
Technology is critical for success in the age of prediction. Organizations that can effectively employ machine learning and artificial intelligence to analyze all this data will have an advantage over their less sophisticated peers. People will be vital to the process. You need lots of people with lots of different ideas and opinions, because the key to better prediction is combining many different models. Once you have the people, you need the ability to test ideas quickly and easily. Here simulation technology plays a key role. Machine learning and AI can help develop more models, amplifying the humans at the center of the process. Instead of producing one model, a human aided by technology can produce thousands of them.
The age of prediction is not some far-off idea. It’s already having a big impact. For companies, it is improving manufacturing efficiency. For WorldQuant, it enhances our ability to deliver quality returns to investors. For predictive medicine, like the research initiative we recently launched in partnership with Weill Cornell Medicine, it is saving lives.
Welcome to the future.
Igor Tulchinsky is Founder, Chairman and CEO of WorldQuant.
