The Monte Carlo, filled with a lot of mystery is defined by Anderson et al (1999) as the art of approximating an expectation by the sample mean of a function of simulated variables. Used as a code word between Stan Ulam and John von Neumann for the stochastic simulations they applied to building better atomic bombs (Anderson, 1999), the term Monte Carlo evolved into a method used in a variety discipline including physic, finance, mechanics and even in areas such as town planning and demographic studies.
Monte Carlo methods are very different from deterministic methods (McLeish, 2004). In the case of a deterministic model the value of the dependent variable, given the explanatory variables, can only be unique value as given by a mathematical formula. This type of model contains no random components (Rotelli, 2015). In contrast, Monte Carlo does not solve an explicit equation, but rather obtains answers by simulating individual particles and recording some aspects (tallies) of their average behavior (Briesmeister, 2000). Given the broad applications and matters involving Monte Carlo Methods we will split this article into three parts to allow for a clear understanding.
First I give a brief history behind Monte Carlo methods then highlight some of its uses by taking an example in physics and showing its necessity in finance, then conclude with what is known as importance sampling for those statistically driven.
In later writings we will delve more on the technical side of things giving strict definitions of significant concepts then progress to discuss Markov Chain Monte Carlo (MCMC) as they form a great role in recent MC computations. In my last offering I will combine all these aspect to form a solid and holistic understanding of the “Black-Box” called Monte Carlo Methodologies with the spotlight on the Metropolis-Hasting algorithm and explore future possibilities within Ulan and von Neumann’s creation.
Monte Carlo’s History
Just as an apple landed on Newton’s head ignited a new stream of science, Stan Ulan had to be sick for him to discover what developed to be an answer for a vast number of scientific problems. Though Ulam and van Neumann formalized and coined the term Monte Carlo, earlier evidence of the approximation method exists. Most notable is the Buffon’s needle experiment. This experiment is as follows:
“A needle of length L is thrown in a random fashion onto a smooth table ruled with parallel lines separated by a distance of 2L. An observer records whether or not the needle intersects a ruled line. From the experiment it is deduced that as the sample increases the probability of the needle crossing the line tends to 1/pi.”(Schuster, 1974).
Ulan in 1964 while in his sick bed wondered if a Canfield Solitaire laid out with 52 cards will successfully be observed. After some thoughts and using pure mathematical methods he played the game 100 times and noted all the successful plays and the proportion of the wins reflected the odds of getting a winning hand. He obtained his results and soon enough he postulated problems on most mathematical physics and on differential equations that would benefit from this practical way of calculation. Later that year he conferred the idea to a John von Neumann (Eckhardt, 1987)with whom they started more complex computational problems and later working on nuclear weapons.
The name Monte Carlo is also inspired by the game of roulette played in Monaco which was a game involving simple random number generation. Another somewhat comical motive was because Ulan’s uncle enjoyed visiting the Monte Carlo to play this roulette as such it was in his honor.
Applications of Monte Carlo
As mentioned above Monte Carlo methods can be applied in many fields of studies. The very first applications where done by the inventors where they used MC methods in solving problems of neutron diffusion and multiplication problems in fission devices using Electronic Numeric Integrator and Calculator(ENIAC). This later developed into the MCNP
Image of the ENIAC
Application in Physical Sciences
Monte Carlo N–Particle (MCNP) is a general-purpose, continuous-energy, generalized-geometry, time-dependent, coupled neutron/photon/electron Monte Carlo transport code. It can be used in several transport modes: neutron only, photon only, electron only, combined neutron/photon transport where the photons are produced by neutron interactions, neutron/photon/electron, photon/electron, or electron/photon.
Though the jargon can be confusing, assuming very little knowledge of the Monte Carlo method and no experience with MCNP Judith F. Briesmeister maintains that even novice may grasp the concepts (Briesmeister, 2000), with practice of course.
Application in Finance
The MC approach to has proved to be a valuable and flexible computational tool in modern finance. With a focus on asset pricing: basic securities and their underlying state variables are often modeled as continuous-time stochastic processes. It was only a matter of time before Monte Carlo methods where used in the evaluation of security prices represented as expectation (Boyle, Broadie, & Glasserman, 1997) . We can clearly see the need for MC methods in pricing derivatives and other financial products due to its flexibility in handling increasingly complex financial instruments. A detailed example on the use of Monte Carlo methods in finance is provided in the third part of this article.
Importance sampling is choosing a good distribution from which to simulate random variables (Anderson E. C., 1999). It makes intuitive sense that we must obtain a sample from important regions so as to obtain accurate results. This is done by over weighing important regions within the sample; hence the name importance sampling. Contrary to its name importance sampling is not sampling per say but rather an approximation method. Let us at this point explore importance sampling in an intuitive manner:
Assuming we wish perform an analysis on a factor but do not have the relevant data to which we can perform the analysis or the data we have does not offer sufficient results. We then generate a random sample which complies with the following properties:
Let g(x) be the original sample distribution (if it exists) and h(x) be the proposed sample distribution.
- h(x) should be close to being proportional to |g(x)|
- it should be easy to simulate values from h(x)
- It should easy to compute the density h(x) for any value x that may be realized.
Adhering to these requirements may be very difficult require sufficient time investment but proves to be effective in dealing with the two problems mentioned above. I chose to include importance sampling at this stage as it is edifying for later discussions more so with regards Monte Carlo improvement techniques. We will discover that there is an important relation between importance sampling and Markov Chains.
We have defined a Monte Carlo as an approximation technique which has vast usage and learned of it’s intriguing and somewhat throbbing history (throbbing because name’s nature of being associated with bomb constructions). Having developed from using ENIAC by von Neumann to being used by modern computers, MC play a large role in the simulation and manipulation of data/processes which has had tremendous effects in the way we live and how interact with our world.
In the next article we go more in-depth and explore other components inherent in the MC and discuss the Metropolis-Hasting Algorithm as a special case of Monte Carlo Markov Chain process.
Anderson, E. (1999). Monte Carlo Method and Importance sampling.
Anderson, E. C. (1999). Monte Carlo Methods and Importance Sampling. Statistical Genetics.
Boyle, P., Broadie, M., & Glasserman, P. (1997). Monte Carlo Methods for Security Pricing. Journal of Economics and Control, 1267-1321.
Briesmeister, J. F. (2000). MCNP- A General Monte Carlo N-Particle Transport Code.
Eckhardt, R. (1987). Stan Ulam, John von Neumann, and the Monte Carlo Method. Obtained from ScienceMadness.org.
McLeish, D. L. (2004). Monte Carlo Simulation and Finance.
Rotelli, F. (2015). Stochastic Processes. Pretoria.
Schuster, E. F. (1974). Buffon’s Needle Experiment. Mathematical Association of America.