This is the title of a report published by the Bank of International Settlements (which serves central banks around the world) in September 2011. As a Forex trader myself, I of course peruse it with great interest hoping to glimpse whatever is the state-of-the-art. Here are a few interesting nuggets, together with my commentary:
1) FX HFT operate with a latency of less than 1 ms, while most of us mere algorithmic traders typically suffer a latency of at least 10ms. For example, Interactive Brokers does not yet provide collocation facilities for its customers, so the best we can do is to place our trading servers on the internet backbone close to its Stamford, CT, location. The best round-trip ping time is 10ms. Those who trade with FXCM may have a better chance for lower latency, as they provide free collocation to their clients. Those who trade on the ECN FXall can collocate at their Equinix data center, while FCM360 provides collocation service to EBS traders. I cannot find any collocation service for Hotspot FX or Currenex. If you know of such services, or FX brokers who provide collocation, do leave a comment!
2) HFT typically operate in markets with high liquidity and low volatility. The former is not surprising, since markets with low liquidity has few counter-parties to take advantage of. The latter requires a bit of nuance. I think most HFT would benefit from high volatility in a mean-reverting market, but unfortunately high volatility is usually correlated with market in a free fall. So don't be surprised if you find that HFT-provided liquidity suddenly disappears when the market is in stress, though the BIS report stated that they are also quick to re-enter the market once the turmoil is over.
3) As a corollary of 2), HFT mostly trade in the major currency pairs. But increasingly, NZD and MXN have drawn many automated and HF traders.
4) Almost by definition, the bid/ask quotes placed by HFT tend to remain on the book for a very short time, measured in ms, unless forced by the exchange to stay longer. EBS and Reuters both has minimum quote life or minimum fill ratio. One exchange that does not have such minimums is Currenex, which is therefore particularly attractive to HF trading. Hence if you are not a HF player, and do not wish to be taken advantage of by a HF player, be wary of Currenex!
5) Two of the favourite categories of HFT strategies: triangle arbitrage and liquidity-redistribution (taking advantage of pricing discrepancies across different trading platforms.) Despite the bad reputation HFTers have been acquiring in the last few years, I think they do provide a useful service to other algo traders like myself via these 2 strategies. It is a hassle to keep looking for a better broker/prices for your strategy!
56 comments:
hi Ernie, interesting article.
couple of points:
1. Co-location; I wouldn't trust a broker who offers co-location with my strategies; unless you own the hardware; but still; even that they still require access to the box to setup the networks and stuff.
2. Many brokers such as FXCM, Currenex, Hotspot are the counter-party of your trades; hence HFT does not really work with these types. On the other side; IB is suitable for HFT because they are an ECN.
Issy
I don't compete in the ultra low latency arena (at least not yet). So for the co-location, I'm more concerned with the fail-over issue. I only use Amazon EC2 for that matter.
But as a momentum trader, I really enjoy the lower bid/ask spread and higher liquidity HFT'ers are providing us. It help to lower my transaction costs and make some exotic instruments now more suitable for my short term strategies.
Hi Issy,
1) You can just upload the executable codes and not the source codes to your server. Your broker will be none the wiser with your strategy by just getting hold of your executables.
2) Currenex and Hotspot are not brokers. They are ECN's. According to the BIS report, most HFT occur on these ECN's. I don't believe IB can be used for HFT because a) lack of collocation facility, b) a delay in trade confirmations of up to 6 seconds, and c) according to many knowledgeable sources, their price feeds are "filtered". That is, they don't display all the quotes from the bank dealers, possibly due to internal risk management reasons. I also find that IB's price feeds in the equities market to be quite noisy, filled with erroneous ticks. There is some evidence that the same noise is present in their FX feeds as well.
Ernie
Soom,
Do you find that HFT are providing more liquidity outside of G9 (AUD, CAD, JPY, NZD, NOK, SEK, CHF, GBP, EUR) + MXN + SGD?
Ernie
hi Ernie, interesting to know about IB quotes.
I have been trading with them for a while. will surely try Currenex and Hotspot. However just by observing FX quotes on Currenex and Hotspot; i concluded that IB FX quotes are superior. Take for example EURUSD; the spread is often 0.5 pips in IB; but more then that on Hotspot.
Have you observed otherwise?
Issy
Hi Issy,
People who are familiar with Hotspot and Currenex told me that the quotes you see on these ECN's depend on which dealers you have established relationships with, or which prime broker you use. So you and I may see two different spreads on the same currency pair. I personally have not traded on the ECN's directly.
Ernie
I love forex trading with IB. Their liquidity is world class and the commissions are very fair. My biggest gripes with IB are:
1) Their connection drops constantly. I've tried multiple VPS vendors with the same problem.
2) Their non-FIX API makes you log out every day. It's a real pain in the rear when you have strategies that don't require daily babysitting - and I hate those card codes you have to use on every log in.
3) Using NinjaTrader for execution with them seems to compound their inherent trade confirmations and account reporting. It seems a lot slower than it should be.
Shaun,
1) By "constantly" do you mean once a day? Or multiple times a day? I only observe that their connection ALWAYS drops between 12:00 am ET-12:30 am ET, but otherwise it only drops occasionally (maybe once a week?) at other times at random.
2) You can use IBGateway for the same API. It won't log off automatically ever.
Ernie
Ernie,
interesting to know about the different spreads you get dependent who you are. When I worked for a prop firm; we used Goldman Sachs FX dealing; and the spreads was not that attractive. Partly because the strategies was trading equities; and hedging with FX one in a while. In other world the spread cost was not directly related to the profitability of the strategies.
Shaun,
regarding TWS shutting down once day; there is a workaround; which am using personally. I don't have to shutdown and restart at all.
the FREE tool is called IBController (http://ibcontroller.sourceforge.net/); it's a java app that monitors TWS; and make it stay connected forever.
Issy.
Hi Ernie,
Alas, I have no actual data to back my assumptions concerning the effect of HFT on liquidity.
My observations only show me a better execution with almost no slippage and good size on bid/ask. However, I have to admit that I don't trade yet exotic pairs (only planning) so I can't answer your question. Sorry.
Hi all,
Hotspot FX just told me that they provide collocation facilities to institutional customers at Equinix’s NY4 datacenter.
Ernie
Here is an interesting review of HotspotFX
http://www.trade2win.com/boards/forex-brokers/77082-hotspot-fxi-true-e-c-n.html
The discussions confirms that the spread you get; depends on who you are. Also the minimum lots are quite high compared to IB
Issy
Hi Issy,
That's a useful link - thanks!
Ernie
Hi all,
You can also see answers to the question about FX ECN I posted here:
https://www.gplus.com/foreign-currency/discussion/what-are-the-advantages-of-directly-accessing-a-fx-ecn-such-as-hotspot-or-currenex
Ernie
Hi Ernie,
I have been measuring execution statistics for IB fx (USDCAD, AUDUSD, USDCHF) for few months now, and the best execution time I ever observed is around 400ms (i.e., time from submitting the order till time of receiving a filled message) for 125000USD orders. I am still working on the simulation account, as I cannot really get my strategy to work well with such latency. I wonder if this is what other users are observing and whether perhaps the latency gets better on the real-money account?
Hi Puzzled,
Does the 400ms really measure the time between your submission of the order and the time of reception of the fill message, or is it really the time between submission and the actual execution time of the order? The latter should be shorter than the former.
Ernie
Hi Ernie,
It is the former - the program can mark when it has submitted an order, and it can mark when it receives the fill message. The actual execution time is hidden and only IB can know it.
400ms still seems a lot of time to me - and that is the shortest round trip ever observed. If one waits for 2 sec to receive the fill message and cancels otherwise, then only around 1/3 of the limit orders are executed. This is rather horrible.
Hi puzzled,
As I mentioned in one of the comments, I have experienced delay of up to 6s between submission and reception of fill message. So 400ms is pretty good compared to my experience! IB is not designed for HFT. In equities, Lime Brokerage is much better in this respect. I haven't yet tried another broker/ECN in the FX space.
Ernie
Hi,
Also, MB Trading provides colocation service next to their execution orders servers in California.
http://www.mbtrading.com/developersMain.aspx?page=vps
Hi Ernie,
I observe filling after 6 sec as well, even when cancelling after 2 sec. Actually, most of the orders are filled after the 2sec despite continues cancel attempts starting with 2sec. Go figure.
Reading the above links re FX ECN, it sounds somehow like the wild west. I was wondering how much and which type of work is involved in building the "relationships" with the banks, and whether these can be revoked and what does it take to maintain them. It sounds strange that there is a network, but one still needs to build 1:1 relationships in extra.
Hi, Eernie: Sorry for asking a question which is a bit off the topic... I am using Matlab and Quant2IB API to perform some intraday trading strategies. I attempted to run multiple strategies simultaneously (coded in separate scripts) within the same Matlab session. I tried using the parallel computing tool but I failed. Is there a good way of handling this issue? Thanks!
Hi Anon,
Why do you have to run different strategies in one Matlab session? You just run multiple Matlabs, each with a different client id, with the same TWS.
Ernie
Hi Ernie and everyone else,
Have any of you used IG Markets as your FX broker?
Is there a place to get free FX tic data?
Hi Anon,
No, I haven't tried IG Markets.
You can get FX tick data from http://ratedata.gaincapital.com/
Ernie
Hi Ernie
With http://ratedata.gaincapital.com/, do you know what field "ITid" represents, and what "RateDateTime' field is relative to? Thanks
Hi Anon,
I don't know what Tid is, but RateDateTime refers to the time that a bid or ask quote is updated. It is in ET.
Ernie
Hi Ernie,
Which broker do you use for FX trading?
For a retail brokerage account is it better to open one with a Bank?
Hi Ex-Hedge fund,
I am currently with Interactive Brokers. If you intend to directly access the ECN's, then you need to open an account with a prime broker.
Ernie
Hi Ernie,
Slightly off-topic, but are planning to come back on your "what worked in 2011" post as you mentioned? Found it a very interesting topic and would be curious to read what worked for you.
Hi Anon,
Yes, I will come back to that topic, but it will likely be on what didn't work for me in 2011.
Ernie
I would not trust GainCapital data though. the broker changes the historical data for their own purposes. They have been fined for that. Google them to find out more.
http://www.youtube.com/watch?v=Ie9gbZjMqxs
Anon,
Thanks for the very valuable information that you provided! But while the NFA ruling said that GainCapital manipulated spreads, slippage, margin, and execution delays to the customer's disadvantage in real time, it didn't say anything about the historical data that it provides. Did I miss some parts of the ruling?
Ernie
Hi all,
I can recommend high quality tick data from Dukascopy. It can be downloaded from jforex platform, although it takes a while. The broker is a good one, but not for hft.
Hi Ernie,
although HFT is quite an interesting area, i have my doubts about it. As we are dealing with very small time frames (as you said 1-10ms), the algorithms have to be simple, fast and not very sophisticated. This leads to three problems:
1.) If there is something like an unusual situation or a market anomaly, these "dump" algorithms may take wrong decisions. And as they work in high frequencies, there is a high risk of unforseen consequences and chain reactions. See the flash crash (http://en.wikipedia.org/wiki/2010_Flash_Crash).
2.) As the algorithms need to be fast, they are definitely very static. This means they can not adapt to different market situations. Again, in a crisis or during a market anomaly this lack of flexibility could be fatal.
3.) The algorithms in use are pretty standard (you mentioned triangle arbitrage and liquidity-redistribution), simply because there are not to many different approaches to make ways to make profits in 1ms. If everybody uses the same (or similar) algorithms at the same time, nobody can stand out from the crowd. As a result, everybody will make the same (meaning no) profit. The only way to be better than the others is speed (this was already discussed).
For these reasons i think HFT can be a dangerous toy. In my opinion, more sophisticated approaches are the future. An example could be transparent self-learning systems (no neural nets as they are not transparent at all). They are highly dynamic in adapting to the market and they dont rely on standard procedures found by humans. Of course, this comes with a higher processing time, but also with more safety.
Timo
(see http://thekairosaiproject.blogspot.com)
Hi Timo,
The points you made are valid, though I think almost every strategy suffers to various degree from increasing competition and diminishing returns. But even if we are engaging in trading with holding periods measured in minutes, not milliseconds, we still need to find ways to lower our latency by collocation and better broker system. When you see a buy opportunity, you don't want to always pay a few more cents than other low-latency traders, even if you intend to hold that position for, say, 1 hour.
Ernie
Hello Ernie,
are you still teaching course at Singapore (NTU-SGX centre)?
best regards.
-guan
Hi Guan,
Yes, the next NTU-SGX course will take place July 6-8. You can contact Michelle at nscfe@ntu.edu.sg to register.
Ernie
@PASS AI,
"As we are dealing with very small time frames (as you said 1-10ms), the algorithms have to be simple, fast and not very sophisticated."
You would be amazed at what you can to in under a ms in the HFT space, including processing neural networks, machine learning and such. I've worked in FX HFT for the past 5 years, and all true HFT shops are processing in under 100 microseconds, some MUCH faster than that.
And there are many many different ways to profit in that timeframe, whether you're providing or removing liquidity. It's just that most 'normal' traders don't think on that scale and it's hard to adjust (also hard to go the other way as I've discovered from time to time when trying to increase my trading time horizons).
Anon,
Thanks for your insight!
Do you see diminishing profits in the HFT space in the last year or so due to competition or decreasing volume/volatility?
Ernie
Yes, profitability in the HFT has been on a decline for many firms since the peak in volatility in 2008/2009. This YTD has been particularly rough for most players. In the latter half of 2011, while passive strategies (Market Making) were getting whipped around, at least the aggressive strategies (Market Taking) were doing well on the increased volatility. This year however, with the dramatic decrease in volatility and volume, both sides are hurting and looking to expand into new markets, particularly overseas.
Hi Anon,
Thank you for your candid assessment of the state of HFT ... unfortunately, my models suffer from exactly the same problems you described YTD, even though they are not exactly HF! My way to deal with the situation is to trade longer term strategies.
Ernie
Great post totally appreciate!
The report examines the facts about high-frequency trading (HFT) in foreign exchange (FX), including its definition, effect on other market participants, behaviour in normal and stressed times, and key differences compared with HFT in equities. It also identifies areas that may warrant further investigation.
Hi Ernie
You said if we intend to directly access the ECN's, then we need to open an account with a prime broker, but what prime broker are the best? Because the biggest forex dealers as Deutsche Bank, Citi Bank, UBS in their sites they show a really high spread, such 2 pips for EUR/USD!!! Or if am I big they will offer a much lower condition? How good they can get if think? 0.2 pip?
Thank for answering.
Hi Felipe,
The biggest FX prime broker typically requires a $10M account.
Also, the displayed spreads are often not necessarily smaller that those on IB, but your limit orders can be filled much quicker and at better prices.
Ernie
Accompanying the currency pair is the quota, or bid/ask price. This is expressed in the following format: EUR/USD : 1.2836 1.2839. The first number in the series represents the bid price, the cost of selling the Euro against the Dollar, or going ‘short' on the Euro. The second number is the ask price, the cost of buying the Euro against the dollar, or going ‘long’ on the Euro. The difference between the bid/ask price is called the pip spread.
ecn forex broker
Oh, 10M is a lot.
So their differential is not a lower spread but a better execution (lower slippage)? Is that correct?
But can they provide at least 0.5 pip spread?
Could you give an example of a good prime broker?
Thanks
Felipe,
Yes, better execution.
You can probably get 0.5pips for EURUSD at many ECN's such HotspotFX.
I heard that Boston Prime accepts $100K accounts.
Ernie
Hi Ernie,
Thank you for a great article. How about armada markets? Have you heared that broker? It said offering Lowest Forex Spreads in the World.
Hi Bob,
No, I haven't heard of Armada markets.
As I said, smallest spread is not my major concern, as we can often make markets ourselves.
The longevity and financial soundness of the brokerage is more important to me.
Ernie
Thanks, I really learning a lot from your article.
Hi, thank you so much for your post! Recently I've been struggling with my master thesis. It's on FX HFT strategies. I have the data (milliseconds) of the best bid and ask prices or the deal price, but I really have no idea about how to start. Coz it's so difficult to find models for FX HFT. Do you have any recommending articles or websites? Many thanks..
Hi Anon,
HFT can be mostly highly profitable trading strategies, so no surprise few people are willing to tell you their strategies!
However, check out the book "Broken Markets" on my Recommended Book list on the right sidebar of my blog. You might get some inspirations.
Ernie
http://nuclearphynance.com/Show%20Post.aspx?PostIDKey=152633
Thanks, Anon. Very relevant link.
Ernie
Thanks for the link, M.
We can easily get order book info in FX. It is trade prices that are hard to get (at least within 1 ms). I guess for this paper, trade prices are used only for model backtest and validation, but not for live signal generation?
Ernie
Post a Comment