The GegenbauerC(n, a, x) function computes the nth Gegenbauer polynomial - see Abramowitz and Stegun, Handbook of Mathematical Functions, Chap. 22.

When all of  are not a negative integer or zero, the Gegenbauer polynomials satisfy:

GegenbauerC(n,a,z) = 'piecewise'(n::negint,0, n=0, 1,convert(GegenbauerC(n,a,z),hypergeom));

and are orthogonal on the interval  with respect to the weight function :

Int(w(z)* GegenbauerC(m, a, z) * GegenbauerC(n, a, z), z=-1..1) = 'piecewise'(n=m, Pi*2^(1-2*a)*GAMMA(n+2*a)/(n!*(n+a)*GAMMA(a)^2),0);

When any of  is a negative integer or zero, the Gegenbauer polynomials are computed using the following identity:

GegenbauerC(n,a,z) = (2*a*z*(1+2*a)*GegenbauerC(n-1,1+a,z) + 4*(-1+z^2)*a*(1+a)*GegenbauerC(n-2,a+2,z)) / ((n+2*a)*n);

which in turn can be derived from the differential equation with respect to z satisfied by this function:

f(z) = GegenbauerC(a,b,z);

diff(f(z),z,z) = (-1-2*b)*z/(-1+z^2)*diff(f(z),z)+a*(2*b+a)/(-1+z^2)*f(z);

For n::posint and n > 1 and a <> 0, the Gegenbauer polynomials satisfy the following recurrence relations:

GegenbauerC(0,a,z) = 1:

GegenbauerC(1,a,z) = 2*a*z:

GegenbauerC(n,a,z) = 2*(n+a-1)/n*z*GegenbauerC(n-1,a,z) - (n+2*a-2)/n*GegenbauerC(n-2,a,z):

and for a = 0, they are related to the ChebyshevT polynomials:

GegenbauerC(n,0,z) = 2/n*ChebyshevT(n,z):