Question

In the usual statistical softwares we have implemented several methods to generate (pseudo) random numbers according to a continuous uniform distribution in the interval [0, 1]. The method of the integral transformation of probability can be used to transform a random sample of such distribution into a random sample of essentially any other univariate distribution.

b) Lecture about some difficulties of the integral transformation of probability, in particular arguing that it is not applicable (nor can be easily adapted) to generate numbers according to a multivariate distribution.

Answer 1

This is a question related to understanding the difficulties /challenges of practicing /implementing the integral tranformation of probability which is used to transform a random sample of such distribution into a random sample of essentially any other univariate distribution.

We need to discuss the difficulties in its implementation particularly for multi variate distributions

But what is an integral probability transformation.

The probability integral transform states that if is a continuous random variable with cumulative distribution function , then the random variable has a uniform distribution on [0, 1].

The Probability integral transform (PIT) implies that the data values that are modelled as being random variables from any given continuous distribution can be converted to random variables having a standard uniform distribution

This holds exactly provided that the distribution being used is the true distribution of the random variables; if the distribution is one fitted to the data, the result will hold approximately in large samples.

So why is it important?

PIT helps to provide the basis for testing whether a set of observations can reasonably be modelled as arising from a specified distribution.

Let us take an example carry an exponential function .

If X has an exponential distribution with unit mean, then its CDF is

and the immediate result of the probability integral transform is that

has a uniform distribution.

The symmetry of the uniform distribution can then be used to show that

also has a uniform distribution.

Thus,

Suppose that a random variable X has a continuous distribution for which the cumulative distribution function (CDF) is F_X. Then the random variable Y defined as

has a uniform distribution.

Now, how does one go about generating uniform random numbers? The short answer is that it’s impossible. This should not surprising as truly random uniform numbers from [0, 1] will take on irrational values with probability one and it is impossible to precisely represent irrational numbers.

However, one can approximate random numbers, or in other words, one can generate sets of rational numbers which in many ways appear to behave as if they were random uniform draws. (called as congruential pseudo-random number generation)

The intuition behind the congruential rule is that when one divides strings of very large numbers and then discards the integer part of the quotient, the result is a set of uniform-like numbers. More specifically, the rule generates a set of numbers

xi = (α + βxi−1) mod l, i = 1, 2, . . . ,

where x0, l, α and β < l are known beforehand, and mod denotes the modulus operator.

But this has a drawback .

One drawback is that it has a period of at most l, or in other words, it can generate at most l numbers before arriving back at x0.

The remedy to it can be to choosing l prime which helps to maximize the period of the algorithm.

A larger prime l also lengthens the period and generates a better approximation to true randomness

Another difficulty is that  The random uniform distribution created  is different for distinct values of the range, but the same range of random numbers  will repeatedly produce the same number. This effect multiplies as the application of the random distribution is increased in the process.