If the number of parameters is less than six, gcdex applies the extended Euclidean algorithm to compute unique polynomials s, t and g in x such that  where g is the monic GCD (Greatest Common Divisor) of A and B. The results computed satisfy  and . The GCD g is returned as the function value.

In the case of six parameters, gcdex solves the polynomial Diophantine equation  for polynomials s and t in x. Let g be the GCD of A and B. The input polynomial C must be divisible by g. The polynomial s computed satisfies . If  then the polynomial t will satisfy . The NULL value is returned as the function value.

Note that if the input polynomials are multivariate then, in general, s and t will be rational functions in variables other than x.