Introduction to FX Data Mining

Let's make a simple and quick introduction to one of the most interesting fields today - Data Mining. There is a wide range of Data Mining applications. We should integrate Data Mining in our FX trading.

FX, FOREX or the Foreign Exchange
FX is the biggest market in terms of daily traded volume. It has three main levels of participants: the big boys, the intermediate level and simple traders as you and me. It has a speculative nature, which means most of the time we do not exchange goods. We care only for the difference and wish to buy low and sell high or sell high and buy low. By short or long operations we can gain pips. Depending on your trading volume, pip value can range from one cent to 10$ and more. This is the major way to make money in the FX market (alongside with Carry Trade, Brokering, Arbitrage and more). Notice that the FX market is huge but is suitable for all levels of players. Think of the FX market as an infinite supermarket with infinite number of products and customers, but it also has an infinite number of cashiers. Meaning there is an equal amount of opportunities for all.

Data Mining and Machine Learning
Data Mining is a mature sub field of Computer Science. It's about a lot of data and non trivial extraction of usable knowledge from massive amounts of data. It's done by Intelligent data processing using Machine Learning algorithms. Data Mining is not just CRUD (Create, Read, Update and Delete). We have several Data Mining methods. Hereby the methods and some applications.

• Classification - classifying email as spam, classifying a transaction as fraud.
• Association - YouTube suggests us new videos based on our history. Amazon suggests us more items during checkout.
• Clustering - analysis of unstructured data such as economic news and opinions to find common groups.
• Process Mining - examine logs of call operators in order to find inefficient operations.
• Text Mining - mining news or technical analysis for pattern recognition.

Algorithmic Trading
Algorithmic Trading is an automated execution of a trading algorithm. In our case, the trading algorithm comes from the mining. The automated trading is done by some king of programming language. Speed and robustness are key points here: human trader cannot beat the computer program regarding those attributes. It could be HFT (High Frequency Trading) and low level programming (as C++) or long term trading and high level programming (as Java).

Mix Algorithmic Trading with Data Mining
Mixing Data Mining in Algorithmic Trading is important. The most important thing is data. A simple principle states that if your data is not good enough, your models will not be good enough (GIGO). It is all about creating a model, implementing it and testing it (as always). Currently this flow is mostly manual.

Data Mining Software
There are many open source software options in the field of Data Mining. WEKA is a Data Mining framework originated in the University of Waikato, Hamilton, New Zealand. WEKA is written in Java and has a great API. Also you have implementations for most of the well known Machine Learning algorithms. 

Summary
Mixture of good tools is vital. There are too many possible trading models. Tossing a coin is a stupid trading system but it’s a trading system. We need Data Mining to find the gold. Good tools are easy to get so good luck with the mining.
​
Next Station
If you are looking for more info about scientific FX trading your next step is exploring Data Mining tools and historical data.