Maximum likelihood estimation applies the concept of the likelihood function - an equation that considers the probability of a set of recorded observations given one or more unknown parameters.  The maximum value that this function attains when mapped over the unknown parameter is of particular importance, as it indicates the "most likely" value for that parameter.

The following is an example problem that demonstrates a method for finding the maximum likelihood estimate of the scale parameter c in a Pareto distribution.

Generate a sample from the Pareto distribution that will act as the provided data.

Compute the likelihood function of a Pareto distribution with unknown scale parameter, assuming only one sample has been provided.

Compute the log likelihood function, assuming four samples have been provided.

Compute the score (derivative of the log likelihood function), under the same considerations.

Compute the score, instead applying the generated sample data.

Solve for c, such that the score is equal to zero.

Calculate the maximum likelihood estimate (this should give us the same result).

Compute the likelihood ratio statistic.

The likelihood ratio statistic is generally distributed as a ChiSquare random variable with 1 degree of freedom. Using this and the fact that the confidence interval bounds the maximum likelihood estimate, we can determine an approximate 95% confidence interval for the maximum likelihood estimate.

The method of moments is used to determine the parameters of a probability distribution function by comparing the moments of a sample to the moments of a probability distribution. In order to apply the method of moments, apply the following steps.

1.  Calculate the first n moments of the fitting distribution, where n is the number of free indeterminates in the fitting distribution. For example, to perform a fit using the Normal distribution, which has parameters mu and sigma, calculate the first 2 moments.

2.  Calculate the first n moments of the data sample.

3.  Equate the equations generated by steps 1 and 2 and solve for the indeterminates.

Assuming that the equations created in step 3 can be solved, these steps provide you with an estimate to the values of the parameters of the fitting distribution. As an example, consider an estimate of the parameters of a Beta distribution, with parameters $\mathrm{\ν}$ and $\mathrm{\ω}$:

Generate a sample from a Beta distribution to act as the provided data.

1.  Calculate the first and second moments of the Beta distribution with free parameters $\mathrm{\ν}$ and $\mathrm{\ω}$.

2.  Calculate the first and second moments of the provided data.

3.  Equate the moments and solve for $\mathrm{\ν}$ and $\mathrm{\ω}$.