Before diving into the meat and potatoes of this blog, I’d like to take a (perhaps humorous) detour to recognize the kind of problems that led to the creation of Cloudmetrx. The reality is, the systems and methodologies in place at a lot of hedge funds are rather archaic; investment professionals need to perform certain tasks to do their job correctly, and presently, nearly all of those tasks are performed via some inelegant combination of macro-heavy spreadsheets. This turns what should be a sound investments team into essentially an IT sweatshop chaotically trying to maintain a poorly-written, poorly-designed legacy software system — a job traders are neither qualified for nor interested in.
And it’s not their fault — there is no immediately obvious alternative. The software companies who operate in the space enforce such rigorous mediocrity standards on their employees, it would be almost impossible for them to add value. The consultants and analysts who work in the space are motivated mainly by their personal ambitions, making them unsuited, as well.
For my first “tale from the desk,” we need to go back to a simpler time: 2008. I was an analyst at a large Wall Street investment bank, and Planet Earth was coming apart at the seams. Morning bloodshed was commonplace — the whole floor would applaud MDs who were terminated, since the rumors always spread before the official word was delivered.
Although I was initially assigned to U.S. rates, due to the crisis I was temporarily moved to the agency MBS strategy desk, which was charged with analyzing the crisis as it unfolded. We were constantly working on research projects, and to do so, we needed to tap into the Great Data Source — a report comprised of data on the GSEs, the mortgage markets, and some summary statistics that were run at the end.
The way the Great Data Source worked was this: you opened up a spreadsheet, clicked a button, the spreadsheet would flick and flash for about ten minutes, and if you were lucky, it would finish. If any step along the way failed — if any one point of data was missing, for example — the whole process would halt and throw up an error. You had to try to fix it, restart the whole thing, and wait for another ten minutes. Usually, you had a trader breathing down your neck, demanding some piece of information contained in the Great Data Source.
I insisted that I could make this process better, and got the go-ahead to investigate. It turned out the data was being delivered via UNIX machines that were about thirty years old; it was delivered (and permanently stored) in CSV format. The spreadsheets, then, were downloading these spreadsheets and loading them into an Excel workbook, forcing all the stats and analytics and graphs to recalc.
I had a very simple solution: I wrote a twenty-line program that loaded the data (when received) into a MySQL database maintained by our desk — in computer terms, this is the Simplest Solution Possible to the problem of storing and using data. I then set up everyone on the desk with a little plugin capable of accessing the data in the database.
These people were shocked. Suddenly the Great Data Source was not only unnecessary, but seemed laughably out-of-date. That’s the strange thing about innovation — people don’t see how bad their current solution is until they have a new one!
At any rate, the Great Data Source was defeated, and the desk forever was able to run their statistics by accessing a single database. The workload for the desk, I’d estimate, was approximately cut in half.
I later added more stuff — analytics engines and so forth, to allow them to analyze the data more efficiently — but the real point is, they had a process that was broken, and until presented with an obviously better solution, they didn’t even notice. Cloudmetrx does 1000x what I wrote for that desk in 2008, but at the end of the day, the point is the same: to make traders more efficient in what they do.