Please note: Part 1 of this essay appeared in the Feb 2015 edition of Contributoria. It is published and modified here under a Creative Commons license.
PART 1 (3500 Words)
A 900 million microsecond primer on high-frequency trading
In the time it takes you to read this sentence, a high-frequency trading (HFT) algorithm, connected to a stock exchange via “low latency” trading infrastructure, could make, perhaps, 1,000 trades.
I say 'perhaps', because it really depends on how long you pause on those commas I put in the sentence. If you’re an individual with great respect for commas you might give the algorithm a chance to throw in a few hundred more orders.
Let’s just clarify this. That means computers owned (or leased) by a firm somewhere can 1) suck in data from a stock exchange, 2) process it through a coded step-by-step rule system (algorithm) to make a decision about whether to trade or not, 3) send a message back to the exchange with an order for shares of ownership in a company – for example, a company that makes children’s toys – 4) get the order executed and confirmed, and 5) repeat this maybe 250 times a second.
Well, it could be more or less than that, too, and to be honest, few people seem to actually know how fast these algorithmic engines trade. But even if it’s only trading 50 times a second, or even a mere 10 times a second, it’s still inhumanly fast.
Having worked in financial trading markets – albeit in much slower over-the-counter swaps markets – and having worked on a variety of advocacy campaigns related to financial trading, this is a subject that fascinates me. The purpose of this piece, though, is not necessarily to convince you on whether or not HFT is a good or bad thing. Rather, it is to provide some frames through which to look at the phenomenon, and through which to understand the debates and news stories that will undoubtedly continue to be written about it in the years ahead.
1.1: Putting HFT in context
 |
| HUMANS: SO SLOOOWWWW |
There was a time, in the distant past of the 1970s, when trades on stock exchanges were basically the exclusive domain of human actors. Whether it was the prudent, long-term investor buying a portfolio of stocks for a retirement fund, or the cowboy speculator buying and selling in rapid succession, the process was always limited by the speed of the human mind, and the time taken to actually pick up a phone and put an order through. Even the fastest speculator would still take a number of minutes to complete trades.
Nowadays, this is no longer the case. The confluence of computer technology, coding techniques and communications infrastructure have made it possible for traders to automate human thought processes by turning them into algorithms that can be executed using beams of light in fibre optic cables. The time taken to complete a trade has dipped into the realm of milliseconds and even microseconds, mere thousandths and millionths of seconds.
This has brought to life the surreal realm of high-frequency trading. It hasn’t come out of nowhere – it’s been a long time developing, since the early days of “program trading” in the 80s, gradually getting faster and faster – but it is only in recent years that people have started to take notice. In particular, it came to the fore during the Flash Crash of 2010, when the US stock market inexplicably crashed and then righted itself within a few short minutes, an event many attributed to HFT algorithms going haywire.
I do not presume to know how you should feel about this. People are routinely worried about harmless things, and routinely completely unworried about incredibly harmful things. What we can say, though, is that to many ordinary people going about day-to-day work involving actual labour of some sort, the concept of a robot trader making 100 trades in the time it takes them to sip a cup of tea makes them feel uneasy. The practice may just seem unnatural, or complex, or out of control, or just weird.
Even if it seems to do no harm, it’s hard to even conceptualise what it is. I mean, aerospace engineers do something that’s pretty complex and I can’t tell you how they technically do it, but I nevertheless understand what they do in principle: They design flying machines that enable people to travel long distances. A high-frequency trading algorithm designer, on the other hand, does what exactly?
Well, we know they make money, but normally people make money by doing something that has some use value to society, like fitting pipes into your toilet or designing your business card or slaughtering cattle to make hamburgers. If we had to ask "what is the purpose of HFT?" on the other hand, people would probably pause for a while before trying to answer. The obscurity of the technique and the goal naturally raises the suspicion that this is just another scheme by bloated financial elites to extract more from society.
Or course, to those financial professionals who work in HFT, people who are freaked out about it might be viewed a bit like superstitious, ignorant peasants who don’t understand markets. They want people to override that intuitive sense that there is something alien about HFT, and to just chill out: “We are scientists, hard-nosed rationalists, stop your unfounded waffling about this. It’s perfectly natural. We wouldn’t make money if our service wasn’t ‘demanded’… right?”
They find allies with certain market economists and frequently go on to add an explicitly moral edge of indignation: “We are helping markets by offering a valuable service of liquidity and price discovery. If you stop us, all of you will suffer.”
Let’s take a step back, and try put this activity into context. Financial markets like a stock-market facilitate the buying and selling of financial instruments, which are contracts that give you rights to receive returns over time. They tend to host different players with different time horizons. On the outer rings you get the huge institutional investors such as pension funds. They arrive in the market occasionally and make big investments, buying up large numbers of shares, often with a view to holding them for a number of years. Then, in the inner rings, you get faster, more fickle, players – we might call them traders – who make money by jumping in and out of markets, like nimble sharks swimming between the slower pods of huge whales.
Not all trading is the same though. If you want to conceptualise the road to high-frequency trading...
1.2: How should I feel about this?
 |
| MIXED FEELINGS? |
Even if it seems to do no harm, it’s hard to even conceptualise what it is. I mean, aerospace engineers do something that’s pretty complex and I can’t tell you how they technically do it, but I nevertheless understand what they do in principle: They design flying machines that enable people to travel long distances. A high-frequency trading algorithm designer, on the other hand, does what exactly?
Well, we know they make money, but normally people make money by doing something that has some use value to society, like fitting pipes into your toilet or designing your business card or slaughtering cattle to make hamburgers. If we had to ask "what is the purpose of HFT?" on the other hand, people would probably pause for a while before trying to answer. The obscurity of the technique and the goal naturally raises the suspicion that this is just another scheme by bloated financial elites to extract more from society.
Or course, to those financial professionals who work in HFT, people who are freaked out about it might be viewed a bit like superstitious, ignorant peasants who don’t understand markets. They want people to override that intuitive sense that there is something alien about HFT, and to just chill out: “We are scientists, hard-nosed rationalists, stop your unfounded waffling about this. It’s perfectly natural. We wouldn’t make money if our service wasn’t ‘demanded’… right?”
They find allies with certain market economists and frequently go on to add an explicitly moral edge of indignation: “We are helping markets by offering a valuable service of liquidity and price discovery. If you stop us, all of you will suffer.”
1.3: Trading, to technical trading, to algo trading, to HFT
Let’s take a step back, and try put this activity into context. Financial markets like a stock-market facilitate the buying and selling of financial instruments, which are contracts that give you rights to receive returns over time. They tend to host different players with different time horizons. On the outer rings you get the huge institutional investors such as pension funds. They arrive in the market occasionally and make big investments, buying up large numbers of shares, often with a view to holding them for a number of years. Then, in the inner rings, you get faster, more fickle, players – we might call them traders – who make money by jumping in and out of markets, like nimble sharks swimming between the slower pods of huge whales.
Not all trading is the same though. If you want to conceptualise the road to high-frequency trading...
- Start by understanding the general concept of trading: Financial traders buy and sell financial instruments, such as shares in companies. They hope to buy at a lower price than they sell at, thereby making a profit.
- Now understand Technical Trading: Traders have different techniques of speculation. They may, for example, spend hours researching the records of a particular company to make assessments, a practice called fundamental trading. Alternatively, they may analyse the activities of other traders in a market to make decisions. This 'technical analysis' of price, order and volume data generated by other traders leads to technical trading.
- Now imagine that automated into Algorithmic Trading: One might decide to automate the process of technical trading, such that an algorithm analyses an incoming stream of price, order and volume data and makes trades under certain conditions. We call this algorithmic trading. (note: it's possible to make a distinction between algorithmic and automated trading, but for ease let's just assume these are the same)
- Then imagine that sped up into High-Frequency Trading: If you accelerate that process of automated algorithmic trading to extreme speeds, you are doing high-frequency trading.
NOTE: If you have enjoyed this so far, you might like my book
1.4: How to set up an HFT firm
 |
| "MUST. BE. FASTEST!" |
Different trading organisations might have slightly different reasons to engage in HFT. Some big banks, for example, use it as a tool to take a big order and fragment it into lots of small orders, like using a dispersion nozzle to turn a fire-hose jet into a fine market mist that people don’t readily notice. Many HFT players, though, are pure short-term speculators, specialist proprietary trading firms and hedge funds. If you wanted to set one of these up, here are some things you’d do.
Firstly, get some start-up money - either your own or from some really rich people. Secondly, incorporate and capitalise a company (maybe set up a management company in London, where you actually sit and work, and then create a separate firm in the Cayman Islands that actually holds the money, and then draw up a contract that says that the London one works for the Cayman one).
Then you hire some people, perhaps through a specialist recruitment agency, or perhaps by popping onto LinkedIn to search for HFT professionals. Maybe you’d like to hire Steve, who knows how to make expensive HFT hardware work for you, or Fabio who can write you C++ code and build your software architecture. Mark here has almost nothing written on his profile, which suggests he works exclusively through headhunters, typical of ex-Cambridge, ex-Goldman Sachs employees.
Offer this prime talent lots of money, and get them to design some algorithms. Start with the conceptual design and then get your C++ guy to code it for you. You might even want to patent your algorithms, or the systems architecture you’ve designed.
This isn’t a pedantic technical article about the exact nature of HFT technology though. There are huge amounts of jargon-laden bumf and geeky discussions on the internet if you’re really interested in the tech, but the essence of what you have to do is this: you must cut some kind of a deal with a brokerage firm and a stock exchange to get your awesome algorithm as close to the stock exchange as possible! You must minimise the physical distance between the computer your algorithm is in, and the computer that the exchange’s order-matching system is in, so that the two can enter into an intense, light-speed dialogue with each other.
There is a whole arcane technical sub-field around such low-latency direct market access infrastructure. Normally, if a person wants to buy or sell shares, they’d have to go via a broker who is a member of the exchange they want to buy or sell shares on. That takes waaaaaaay too long for an HFT trader though. Screw that, you need to go directly into the heart of the exchange without passing through normal brokerage processes. Ideally, you want to find a way to directly co-locate with an exchange, which is a fancy way of saying you need to literally set up your computer in the room next to their computers.
To sort out co-location, check out the services offered by NYSE, Nasdaq, London Stock Exchange, Eurex, CME and even the Johannesburg Stock Exchange. Here is the Toyko Stock Exchange describing the difference between its co-location area and its proximity area (both give you a 100-200v power source, but the co-location area has a greater cooling capacity of 8kVa, so you might want to use that if your algorithm is likely to make the computer melt). Here is a promotional video they made about it. The exchanges have a whole raft of “connectivity” services. Maybe this involves giving you nice high-spec cable and cooling systems, whilst also setting you up with premium data feeds.
Needless to say, the whole array comes with an (initially) baffling array of jargon - a lot of it associated with the tech stack - but in the end it comes down to a pretty simple formula: You lease a computer next to the exchange. You install your algorithms into it. The exchange then sends your algorithms a big data feed through a cable, your algorithms process it and shoot orders back through the cable. And you try to design your rig so that it does this all faster than anyone else. Maybe you’ll sit in an office a few kilometres away monitoring it all, building a newer version of your system.
If you need help setting all this up, you can pick up some low-latency trading infrastructure support and consultancy from the likes of Sungard, Cisco, Algospan, Interactive Data, and Lato Networks. Otherwise, learn from the existing masters, the actual HFT firms that have already got this stuff down. Like most powerful, behind-the-scenes institutions, these firms often have obscure, unrecognised names and uninformative, slightly vague websites. Check out, for example, Virtu, ATD, KCG, Tradebot, Tradeworx, Liquid Capital, Chopper Trading, Citadel’s Tactical Trading Fund, Tower Research and RGM.
1.5: Perfect the electronic Art of War
 |
| PIMP MY ALGORITHM |
Now, it’s not like these firms all use the same strategies. Some use statistical analysis and arbitrage of various sorts, while others operate exclusively in “market microstructure” strategies, which seem to involve knowing the intimate electronic guts of the exchange systems and how they can be, um, taken advantage of. One might engage in flash trading, which some argue is a form of legalised front-running. You might bludgeon markets with orders through ”order stuffing″ (what HFT whistle-blower Dave Lauer calls a financial DDOS attack).
You may layer orders across a market like fairy dust, perhaps trying to incite outbreaks of ”momentum ignition“, which appears to be a form of subtle market manipulation. Some have aggressive trading strategies aimed at proactively following trends and taking opportunities, while others might be more passive, like electronic Aikido-bots using minimal exertion of energy. It’s worth taking a read of this piece by Irene Aldridge if you’re interested in some of the strategies. This introduction here is also useful.
As an aside, if you wish to get a feel for the language and spirit of the scene, it’s always worth browsing the techie discussions of the professional finance quants on places like the Wilmott Forums. Such people are immersed in the nitty gritty of day-to-day finance and generally have a decent knowledge of this stuff. If you’re game for grappling with jargon, check out a user like Quantumar, who likes to lay down a stream of financial cowboy speak (It doesn’t matter if you don’t understand it, but it’s useful to mine these conversations for clues):
“Most of [HFT] is a very simple speed game of arbitrage. They either arb cash vs futures markets or in equities they get hit/taken in one ECN and sell/buy on somewhere else, either all or most of the money is made from a fraction of rebates in market making equities. Some few firms do milliseconds momentum trading, they realise someone is coming in with orders and they jump ahead of the orders (because they are faster to reach the market) they push the market one cent and sell back to the original buyer… They also use flash orders to jump ahead of big orders. Some also look into depth of book and try to trade as well. There are a few more strategies they use in equities. Also some firms look at options markets and arb the delta hedgers... Most of the strategies are not mathematical but related to microstructure of the markets… These shops are ultra high frequency shops, there could be up to millions of orders a day depending on how many markets and how actively they trade. They require mostly really good C++ skill sets, API connectivity knowledge on the software side. Hardware side they require really low level hardware knowledge such as bypassing the stack and tricking kernels. They also look for lan/wan guys who can push data a few microseconds faster in the network. They use very expensive and specialised equipment. A simple switch that is decently fast costs 50K… All the data that is available to HF groups is available to all traders, the difference is they trade on that information before you can even receive it in your computer. How fast they can get it and react in the market is the difference. They are dealing with single-digit microsecond latencies in their networks and computers, not milliseconds.”
1.6: The (narrow) academic debate
Away from all the Youtube video explanations, journalistic reporting and forum discussions on HFT, there is obviously also a body of academic research. If you’re looking for robust arguments rather than Quantumar’s gunslinging “’let me tell you how it is” street-smarts, the research-oriented individual might browse the academic journals. There is research emerging from finance and economics departments, unsurprisingly, but also from a few other disciplines.
A friend of mine who teaches university-level finance noted that a potential problem in HFT research is that researchers rely on HFT firms to give them data and hence are always at risk of being intellectually captured by the firms they rely upon, perhaps even engaging in forms of self-censorship. To add to that, the research often seems to try be as dry and technical as possible; it sounds like it emerges in a world without politics, culture or history, or, for that matter, actual people, making it deathly dull and hard to read. The research questions are narrow, with an obsessive focus on questions like HFT’s impact on liquidity and price discovery.
In essence, liquidity refers to how easy it is to trade. If I arrive in a market and I’m immediately able to sell or buy, there is high liquidity. If, on the other hand, it takes me a long time to buy or sell, there is low liquidity. Some of the debate around HFT and liquidity concerns whether HFT adds to, or just absorbs, liquidity. In other words, do HFT firms, on net, help other participants to trade more easily, or do they get in the way? This debate includes questions of whether the liquidity they might offer is real or not. For example, the robot traders may constantly signal that they’re willing to trade, and then run away.
'Price discovery' is a somewhat fetishised term for the process whereby the apparently correct price for something is figured out through a group of market participants 1) reacting to information by 2) placing buy and sell orders that are 3) mediated through some market infrastructure. So if it’s announced that a firm is about to go bankrupt and the stock price suddenly rockets upwards, it’s likely that something has gone wrong with the price discovery process. The question is, does HFT help reveal the true sentiment in a market, or does it just cause instability and weird anomalies like the Flash Crash?
There is also an emergent body of research on whether what HFT firms do is legal, or constitutes some form of market manipulation or “front-running” at the expense of other market participants. And, finally, we are starting to see a trickle of articles about the human dimensions of HFT, the actual people who run these operations, the politics of it all, the anthropology and how it reflects the broader trajectory of the global economy. [at a later date I will hopefully update this with a proper database of this research!]
1.7: Research, and lobbying, informs a regulatory debate
 |
| "PRESENT YOUR IMPARTIAL EVIDENCE" |
Much of the news on HFT is about the political battles and the threats of regulators to clamp down on it. Theoretically, the regulatory debate is supposed to be informed by the academic research, but of course we might also suspect that the regulatory debates are equally informed by lobbying.
Lobbying itself often takes the form of groups picking particular academic research pieces to showcase to regulators. The Modern Markets Initiative, for example, has curated a heartwarming selection of friendly research articles to back up its claim that HFT creates a market utopia that “saves individual investors’ money by lowering the cost of trades” and that it “democratises today’s marketplace”.
When not getting spammed by such transparently self-serving groups, the regulatory bodies have been putting a fair amount of research into this themselves, churning out papers and briefings. They might also receive submissions from those firms and reforms groups that are on the warpath against HFT. This includes groups like Themis Trading, who, in the words of author Michael Lewis, have “done more than anyone to explain and publicise the predation in the new stock market” (see their extensive collection of critical HFT research). Other critics include data provider Nanex and the aforementioned David Lauer. There is also a whole raft of renegade financial pundits from the financial blogosphere who speak out against it.
Types of regulations that are being suggested include taxation of HFT (something along the lines of a financial transaction tax), and regulations concerning the speed of trading, the order size, and order-to-trade ratio (how many orders a trader can put in, relative to how many times they actually trade). These debates are at various stages in the US, the EU and Asia. Take a look, for example, at the German High-Frequency Trading Act.
Incidentally, it’s worth looking at the dynamics of similar regulatory battles over commodity market speculation. Commodity exchanges like CME Group pointed to a single study by a dude at the University of Illinois to argue for why speculation didn’t negatively destabilise commodity prices, despite the fact that many other studies argued it did. HFT firms, like commodity trading firms, take advantage of the complexity of the situation and slowness of regulators. They implicitly take the position that “until it’s proven wrong, it’s right”, rather than “until it’s proven right, we should take precautions”.
NOTE: If you'd like to support my ongoing Creative Commons writing, please consider buying me a virtual beer
PART 2 (4500 words)
Five frames through which to view HFT
 |
| WHAT DO YOU WANT TO SEE? |
But, let’s for a moment imagine that market infrastructures do generate some kind of mysterious holy force that always makes society better off. What exactly does HFT do to help this? Well, the refrain from the proponents is that HFT facilitates liquidity in markets for financial instruments. This statement comes complemented by wrath-of-god like warnings about what will happen if this liquidity is reduced. If you allow liquidity to go down, your grandmother's pension will suffer!
Goddamn LIQUIDITY is always trotted out like it’s the greatest service to humankind since fire was invented. Now, I don’t like to be (too) judgmental, but in the grand scale of societal injustices, ‘slightly lower liquidity in Microsoft shares’ barely ranks, and in the grand scale of human achievements, ‘slightly higher liquidity in Microsoft shares’ barely ranks either. It’s not like the creator of the algorithm has invented a new way to harvest energy from lightning bolts. And certainly, using your university degree in advanced computing to contribute to microscopically more accurate ‘price discovery’ doesn’t mean you get to go down in the grand book of human virtue.
But of course, this whole appeal to morality from the lobbyists is obviously completely disingenuous. It’s not like people are getting jobs in HFT firms because they're possessed with an evangelical desire to improve liquidity to help pensioners. Creating an epic light-speed infrastructure array to exploit microscopic price discrepancies has got about as much to do with helping pensioners as Formula 1 racing has to do with improving transportation for the elderly.
2.1: The unstoppable progress of rational agents without agency
 |
| WE'RE INDEPENDENT AGENTS, RESPONDING RATIONALLY TO INCENTIVES! |
Regardless of appeals to the morality of HFT, there is another more subtle line. Note the name of the aforementioned pro-HFT lobby group – The Modern Markets Initiative. The name is a deliberate attempt to paint anyone who is concerned about HFT as an enemy of modern progress, standing in the way of the inevitable triumph of a more efficient, rational world. The tech-as-progress dogma is widely entrenched in our society, like a hard-to-remove piece of social malware that disarms people's critical impulses. The 'luddite' impulse is ridiculed, rather than celebrated as a healthy skepticism towards tools of the powerful.
But this tech fetishism becomes even more entrenched when it meets with the mainstream economics belief in the virtue and inevitability of rational economic agents pursuing self-interest. It’s here that the dual utopian visions of tech-as-unstoppable-progress and markets-as-unstoppable-progress merge into one stream. If the technology can be built, and provided that some kind of profit can be made by entrepreneurs taking the short-term opportunity to build it, an inertia sets in, an imagined inability to stop the ‘progress’, regardless of whether it is actually useful in the long term or not.
Attempting to stand in the way of such a stream of individual actions is seen as futile, and even unjust, like trying to stop a river flowing down a hill. Indeed, this is part of the implicit background thinking that leads to terms like ‘arms race’ being used to describe the development of HFT (and other technologies). If one entrepreneur doesn’t do it, another will. Ever heard a tech person saying 'you cannot stop technology'?
Attempting to stand in the way of such a stream of individual actions is seen as futile, and even unjust, like trying to stop a river flowing down a hill. Indeed, this is part of the implicit background thinking that leads to terms like ‘arms race’ being used to describe the development of HFT (and other technologies). If one entrepreneur doesn’t do it, another will. Ever heard a tech person saying 'you cannot stop technology'?
There is a deep irony to this vision. Above all, there is a distinct lack of agency projected when people insist that ‘this cannot be stopped’, but it gets coupled with a vision of thousands of entrepreneurs all individually impelled through ‘agency’ towards something that will occur regardless of whether they choose to make it occur or not. In other words, a kind of agency towards executing a preordained plan.
This vision of the rational-agent-without-agency is something that plagues much mainstream thinking on economics, a strange blend of extolling the virtue of the risk-taking individual whilst simultaneously asserting that they’re irrelevant, mere puppets acting out the will of ‘the market’.
Here, for example, is big-shot venture capitalist Mark Cuban explaining his take on HFT: ‘"If you know the game is rigged and that it is legal to participate in this rigged game, would you do everything possible to participate if you could? Of course you would". It's a statement dripping in contradictory ambiguity, a vision of independent agents acting like pre-programmed robots that have to abide by some imagined law of economics. MUST.PARTICIPATE.IN.RIGGED.GAME.
In reply I'd say: "No Mark, I am not a character out of an Econ 101 textbook. I am perfectly capable of overriding the impulse to play the rigged game, and to decide to not play it. In saying it is inevitable, you’re just trying to justify your own inability to do that." The problem, though, is that provided enough people think like Mark, the inertia continues to be presented as natural, a collective action problem portrayed as a liberator of previously unrealised human potential.
2.2: Automation: People create robots to crunch (big) data
 |
| GOOD DOG, GO FETCH! |
To be one of the aforementioned economic agents in the HFT space, you need to master three things. Firstly, you need to master the physical hardware: the actual wires, cables and microwave towers. Then, you must be able to master the data streams travelling through those wires, to collect it and arrange it in an efficient manner. Then you must be master of the algorithm that knows what to do based on that data. The algorithm is your automated avatar in the marketplace, ‘thinking’ and acting on your behalf.
Your algos must work with lots of data, but it’s worth noting that not everyone perceives HFT as a realm of ‘big data’. The hype around 'big data', to some extent, concerns the growing capability to do real-time processing of huge dams of data (like modelling of climate on supercomputers), but ‘real-time’ doesn’t necessarily mean microsecond-level speed. It makes no difference whether it takes you 5 minutes or 20 microseconds to know a hurricane is forming. Much HFT, on the other hand, is more about brute reaction time to a high pressure hose of data, rather than a dam.
That said, being able to react at microsecond speed to colossal dams of data is emerging. The HFT company Tradebot (based in this building in Kansas City), has been known to trade a billion shares in one day, making millions of individual trades. In their own words, “Market data changes trigger our system to produce new orders in a few hundred nanoseconds. We collect and analyze billions of data rows to find the edge. Our Hadoop cluster is over two petabytes.” A Hadoop cluster is an array for holding massive amounts of data, and two petabytes is 2 million gigabytes. How many gigabytes is your computer?
Regardless of whether all HFT strategies should be considered a realm of Big Data, HFT is a subset of the broader realm of algorithmic trading, which is on the cutting edge of financial data science more generally. To create financial algorithms often first involves ‘back-testing’ potential algorithms on huge banks of historical market data, essentially engaging in “what if I’d done this between 1980 and the present” time-travel exercises. If you find an algo that seems to work on past data, you can crystalise it, then send it to work on real-time data in the present.
John Fawcett of Quantopian notes with a certain amount of joy how automated algorithms “remove human emotion and bias from trading decisions”, opening up a brave new world of emotionless finance. You use statistical back-testing to find the most ‘rational’ strategy, then lock it in a hard-coded shell that “never falls prey to sentimental pitfalls”. You too can now isolate and strip away your emotion from your rationality, automating your rational self in the form of your very own algorithm that you can keep like a pet, or a slave, to do things for you.
2.3: Automation: Robots create (big) data to crunch people
| NETWORK, OR SPIDER'S WEB? |
We tend to understand the concept of actively using technology to achieve certain ends (exercising agency), but we find it harder to conceptualise the potential loss of agency that technology can bring. It’s a phenomenon perhaps best demonstrated with email: I can use email to exercise my agency in this world, to send messages that make things happen. At the same time, it’s not like I truly have the option to not use email. In fact, if I did not have an email account, I would be severely disabled. There is a contradiction at play: The email empowers me, whilst simultaneously threatening me with disempowerment if I refuse to use it.
In HFT and algorithmic trading more generally, we have a range of disparate players each individually working on building little pieces of the infrastructure and single algorithms that they control. When we zoom out though, we might see the outlines of something bigger. While individual algorithms appear as isolated, individual slaves to creative masters, the collective array of algos can begin to seem like a spiders web displaying emergent properties that are not under the control of any particular human master.
Or, let's put it this way: Traditional sci-fi depictions of 'artificial intelligence' always show an individual mad genius building and unleashing an overlord computer that then kind of behaves like a hyper-powerful human. In reality, if an overlord technological system was to be built, it would not be a single computer, and neither would it be built by a single mad genius, and it wouldn't really look or feel anything like a human. It would be an interwoven mesh of technology, brought to life by individuals who never explicitly designed it, with no obvious human face or interface.
In reality we already see these emergent forms all around us, but are not well trained to recognise them. They emerge whenever humans ‘lock themselves in’ to reliance on a technological infrastructure, and lock themselves in to a point where they cannot pull back out due to the interconnections and dependencies that subsequently emerge. Those infrastructures then, have a certain power over society, even though their individual nodes may be under the control of particular people.
I have previously referred to this concept – albeit in a different context – as the Techno-Leviathan, technological infrastructures that seem passive and neutral but that contain a kind of latent organising force over the people who seemingly contract into using them.
So, the question to ponder is whether, when viewed collectively, we might begin to imagine the high speed mesh of individual algorithms as resembling one a giant robot, brought to life by hapless human agents-without-agency, all believing themselves to be shit-hot independent gunslingers of the market, but actually just a disconnected workforce for an emergent AI. Or at least that’s what Stephen Hawking might argue.
Indeed, the FT’s Sally Davies notes that “GFT, which works with big global investment banks, has partnered with Massive Analytic, a big data start-up, to develop trading software based on “artificial precognition”. Even if it doesn't end up in the realm of Minority Report, it stands to be a total mindfuck for regulators.
2.4: Disconnected boys with dangerous toys
 |
| NERD CULTURE MEETS JOCK CULTURE |
Having detoured into the possibilities for an emergent rise of the machines, we might go back down to earth and look into the human world of HFT. Who are the individual people involved, and what are the cultural dynamics?
Obviously there are many different types of people involved in HFT. I'm sure many of them are lovely, but the last algo trader-boy I met was a guy from Ronin Capital who happened to be one of the most condescending assholes I’ve experienced in a while, coming packaged with one of those heavy-set wrist-watches and a shirt with expensive fibres, both marking out a rising member of the financial elite.
He spent a lot of time in the gym, because you don't develop big muscles from sitting behind a computer. This seemed to fit quite well with the pseudo Samurai aesthetic of his firm. If you enter Ronin’s website, there are gong sounds and pictures of swords, as if the traders behind the interface of the computers want to imagine themselves engaged in hand-to-hand combat with a vicious opponent that could actually kill them. Of course, given that they are probably educated at elite universities, it is unlikely that they’ve ever had any exposure to actual bodily harm, and probably never will.
This is a dynamic in the financial sector more broadly: highly educated people, frequently male, induced into believing they’re engaged in some kind of mortal combat, despite the fact that they’re surrounded with abundant opportunities and money, and despite the fact that they’re sitting in an air-conditioned office at a computer engaged in nothing remotely like combat or physical hardship. This pseudo-battle is perhaps best exemplified by TradeBot, who without a trace of irony state that:
The stock market is tough. It owes us nothing. It punishes our mistakes. Others have more money, more power, more connections. We are underdogs. We keep learning. We innovate. Every day is a new fight. Technology is our weapon. We make millions of small trades. We cut losses. We identify opportunities. We focus. The market can be beaten. We love the game.
When assessing these banal market-as-mortal-combat statements, I like to use the WWII Grandfather Test, which involves me asking myself what my grandfather would say about it. He was a bomber pilot during WWII and got shot down over Germany, crash-landing a flaming heap of metal on the coastline after probably killing a lot of people with incendiary bombs. Ask your granddad: what do you think about Tradebot’s battle with the ‘the market’?
I don’t know about your granddad, but I like to think mine would have said, ‘I have no bloody idea what they are doing, but I know it has no connection to real people living in real places’. Seriously Tradebot, if you really think you’re so tough, go do some shipbreaking in Bangladesh, and you’ll quickly discover that an actual battle isn’t a ‘game’.
Indeed, you can always sense something is a realm of cushioned elites when the language is all about ‘players’ jostling with each other. Sports and games are simulacrums of combat, not actual combat. You only perceive the real world as a ‘game’ when you’re in certain types of environments that provide a big cushion to protect you – like an elite, global city, for example. There is a particular urban geography to these infantile computer games. When one is sitting in a nice modern city full of other fairly superficial activities, things like HFT gain a certain legitimacy. They are creatures of urban, tech-centric society, where couches, excel-spreadsheets and lattes abound.
It’s only in such a setting that you can imagine grown adults bickering with each other over the meterage of cable connecting them to a stock exchange. Imagine the furious exec shouting at the co-location manager, ‘Our cable is a metre longer than Tradebot’s cable, why the fuck did you allow that! We’ve lost a nanosecond!’
It’s kind of embarrassingly juvenile when you stand back a moment, look through granddad’s eyes, and watch a serious-faced discussion about whether C++ or Java will achieve the holy grail of zero latency. We’re not talking about old-school realpolitik here, where some tycoon is battling another tycoon for control of some vast mining territory. Regardless of whether HFT is damaging or not, it's just kind of... um... lame.
It’s kind of embarrassingly juvenile when you stand back a moment, look through granddad’s eyes, and watch a serious-faced discussion about whether C++ or Java will achieve the holy grail of zero latency. We’re not talking about old-school realpolitik here, where some tycoon is battling another tycoon for control of some vast mining territory. Regardless of whether HFT is damaging or not, it's just kind of... um... lame.
2.5: HFT and the financialisation of meaningless noise
 |
| AS YOU CAN SEE, OUR PRODUCT CAPTURES MEANINGFUL MARKET MOVEMENTS |
‘Financialisation’ is a term laden with various interpretations, but it tends to refer to the increasing importance of the financial sector in overall economic life, the infusion of financial sector norms and morality into everyday culture, and the process by which previously uncommoditised things get turned into financial products that can be traded on financial markets.
That’s a pretty broad description, so I prefer to initially think of financialisation as the end result of things being made 1) ownable 2) investable and 3) tradable. The greater the intensity and extent of these elements, the greater the degree of financialisation of that thing.
- Ownable means the thing can be claimed by someone, and that they can exclude others from its use. ‘Ownability’ relies on being able to isolate and separate something off from things around it. For example, the enclosure movement involved turning land into demarcated parcels that could be separated from each other and privately owned
- Investable means turning the thing owned into an asset that delivers returns over time. While a piece of land might be something that you can own, and have an emotional connection too, you might begin to view it as an 'asset' when it is used to produce yields over time. It might be perceived as a generic 'investment', rather than a piece of land with a particular history and life
- Tradable means that asset can be passed on to others
Still, moving land from person to person is slow and personal. Land only really gets financialised with it is turned into a generic ‘asset class’ that disconnected investors can quickly buy into or out of. So, imagine a financialisation process in this sequence:
- I own a farm. I can use it to make food
- I own a share in my neighbour’s farm. I can claim a portion of the produce
- I own a share in a small private farming company. I get annual monetary dividends and read the reports
- I own a share in a large publicly traded farming corporation. I get monetary dividends and can sell my shares to others at any point on the stock exchange
- I own a share in a huge agriculture exchange-traded fund (ETF) that owns shares of farming corporations all over the world
- I own a share in a hedge fund that rapidly trades a portfolio of such ETFs, and bets on such ETFs via derivatives
We might say that financialisation is the creeping process by which new frontiers of ownership are isolated, and turned into investable products that a wide, disconnected range of dispassionate investors can emotionlessly slide into and trade with each other. The more distant you are from the thing you’re invested in, and the easier it is to trade, and the faster the trading, the more disconnection you can experience.
But, there is a point when the speed of trading hits a tipping point, and takes you into a realm that is no longer about the farm, or anything real for that matter, at all.
But, there is a point when the speed of trading hits a tipping point, and takes you into a realm that is no longer about the farm, or anything real for that matter, at all.
This is where HFT take us. While it ostensibly seems to be about the trading of shares on stock-markets (and other things like currencies), in reality HFT has nothing to do with shares. The ‘thing’, or object that is being traded is not actually [a financial instrument], but rather it is [the microscopic tremblings of a financial instrument].
This is a subtle point to convey. Much normal speculative trading is done fast, with a trader quickly buying something and then trying to sell it to someone else. Nevertheless, there is always a sense of a 'thing' being manipulated in some way. Just like when you are flipping a hot potato, there is always a brief moment of being invested in the heat of the real world, even if fleeting, and there is always some residual awareness that there is some 'reality' to the thing. In the case of a BP share, for example, the share has a reality based the fact that it is a legal claim upon what BP owns. It is thus directly connected to the real world outlook of those oil fields and pipelines.
We call traders who make assessments of that reality 'fundamental traders': They might say "I think OPEC is going to decrease supply and thereby boost the price of oil, and thereby boost BPs profit. I will therefore buy this BP share that allows me to benefit from any perceived increase in the value of BP’s collective assets."
The actions of such fundamental traders give rise to a second-degree reality that is exploited by traders who watch the data they generate. We call this technical trading. Such traders may say "Market data suggests that a lot of people are currently buying BP shares. I am going to ride with this sentiment."
This is a subtle point to convey. Much normal speculative trading is done fast, with a trader quickly buying something and then trying to sell it to someone else. Nevertheless, there is always a sense of a 'thing' being manipulated in some way. Just like when you are flipping a hot potato, there is always a brief moment of being invested in the heat of the real world, even if fleeting, and there is always some residual awareness that there is some 'reality' to the thing. In the case of a BP share, for example, the share has a reality based the fact that it is a legal claim upon what BP owns. It is thus directly connected to the real world outlook of those oil fields and pipelines.
We call traders who make assessments of that reality 'fundamental traders': They might say "I think OPEC is going to decrease supply and thereby boost the price of oil, and thereby boost BPs profit. I will therefore buy this BP share that allows me to benefit from any perceived increase in the value of BP’s collective assets."
The actions of such fundamental traders give rise to a second-degree reality that is exploited by traders who watch the data they generate. We call this technical trading. Such traders may say "Market data suggests that a lot of people are currently buying BP shares. I am going to ride with this sentiment."
Both of these techniques rely on a type of sentience, an awareness of some external reality and an ability to reason about it. In the case of fundamental trading, it's the awareness about some new development in the world of oil. In the case of technical trading, it's the awareness of some new trend that is developing among other traders.
For something to contain 'meaning', in the human sense of the word, it should be something that is open to human experience. There are many things that are not open to human experience - for example, perceiving radio waves - and in a sense that takes them out of the realm of meaning. Sure, we can use instruments to detect radiowaves, and try make meaning out of the resultant observations, but radio waves cannot ever really mean anything to us in their raw state.
The key thing about a radiowave though, is that it's existence does not depend upon human observation. It exists regardless of whether you can perceive it or not. A share is nothing like this. A share, by definition, is a politically constructed claim on a politically constructed company that is run by humans, doing things that are perceivable by humans. It's value does not exist outside of human assessment of how well that is being done, and there is no 'hidden reality' to a company that operates outside the realm of human experience. We cannot say something like "well, we cannot see BP, but we know it exists through experiments at CERN". BP, unlike radio-waves, has no microsecond reality. In other words, nothing can meaningfully change in such a legally constructed entity in the imperceptible space of microseconds.
Thus, when you in fact do dip into the realm of microseconds, it is highly implausible that an automated trading algorithm is actually being exposed to external ‘outside information’ that has anything to do with either BPs operations, or observation of an emergent trend in people trading BP shares. At that level, all you're doing is highly precise arbitrage activities in microscopic inconsistencies in people's perceptions, or perceptions of perceptions, of reality. The activity going on at the molecular microsecond level is by definition, not about the thing being traded. The sheer emotional disconnection engendered by the technological medium, combined with the sheer speed means that this certainly cannot be thought of as trading in 'things' at all. This is the isolation of, and subsequent trading of microscopic, subconscious instability.
It is the financialisation of meaningless noise, something that previously wasn't subjected to commodification. The algos have an internal world, like the internal world we see in those electron microscope pictures where tiny, imperceptible flakes of dust appears as a whole landscape with valleys and hills. From the perspective of an atom, that world means a lot, but from the perspective of humans, the internal contours of a speck of dust are irrelevant and meaningless. Likewise, at microsecond level, you’re trading meaninglessness.
It is the financialisation of meaningless noise, something that previously wasn't subjected to commodification. The algos have an internal world, like the internal world we see in those electron microscope pictures where tiny, imperceptible flakes of dust appears as a whole landscape with valleys and hills. From the perspective of an atom, that world means a lot, but from the perspective of humans, the internal contours of a speck of dust are irrelevant and meaningless. Likewise, at microsecond level, you’re trading meaninglessness.
In closing: Parasitic algorithmic surrealism
One common problem in thinking about HFT, though, is that people’s minds run away with them. They feel panicked by how alien it seems, having visions of extreme market meltdown as rogue algorithms run everything in a giant psychedelic orgy of routers.
It's worth taking a breath before stressing out too much. While it’s true that the algorithms might exist in serene, unreal bubbles, at some point they are constrained by the reality of the world. Take, for example, the Flash Crash. It was a momentary breach where rogue algorithms painted a warped picture of reality, but minutes later the real world kicked back in and the algos had their collective wills bent back. If Wallmart goes bankrupt, the value of a Wallmart share will tank, and if an algorithm says otherwise, it will be crushed by the legal reality that the shareholders of Wallmart are going to get blasted out of the water.
There are probably limits on how much HFT can proliferate. I mean, a parasite relies upon an ecosystem to survive, and in the end, HFT algos have to feed off something. In this case, it’s probably the big institutional investors - the whales that make up the baseload order flow of the market - that 'host' the HFT parasite. The question is not so much whether HFTs can ‘take over’ a market, but rather whether they disrupt it, exert a new cost on it, or otherwise cause a nuisance. (of course, if you're an industry lobbyist, you might alternatively suggest that they ‘offer useful services’ and improve the ecosystem)
To me, though, the really interesting question about HFT is not this banal fixation on whether it disrupts markets or not. It's the cultural and political elements. It's how such a ridiculous thing can be viewed as legitimate. And, it's the sheer physicality of it, the fact that it appears 'ephermeral' yet relies upon huge real world infrastructure to engage in the essentially meaningless activity. And, it is the geography. It's a technology set that attempts to eliminate distance and time, but perception of distance and time are two main components of a sense of difference between places. Eliminate the sense of distance and the time it takes to get there, and you can create the homogenising illusion of being in many places at once simultaneously. The computer interface at a global HFT firm, presiding over multiple global markets, is an agent of bland homogenisation.
Above all, though, HFT is an agent of financial surrealism. We make electricity by burning real fossil fuels dredged out of the Niger Delta, and then waste that running servers doing something that cannot even be represented. Serious-faced men have serious-faced meetings about it, but they might just as well be wearing pink unicorn outfits in a Neverland dream. Seriously, WTF are you doing?
Further reading: People to follow for up-to-date HFT info
- Alexandre Laumonier (@SniperInMahwah): Website Sniper in Mahwah
- Eric Scott Hunsader (@nanexllc): Website Nanex
- Sal Arnuk (@ThemisSal): Website Themis Trading
- Joe Saluzzi (@JoeSalluzi): Website Themis Trading
- Irene Aldridge (@irenealdridge)
- David Lauer (@DLauer)
- Haim Bodek (@HaimBodek)
And finally...
These pieces take me ages to write, so if you'd like to support my ongoing Creative Commons writing, please consider buying me a virtual beer. Cheers!
