Drop Missing Values - Maple Help

DataSeries/FillMissing

fill missing values in a DataSeries

DataSeries/DropMissing

drop missing values in a DataSeries

 Calling Sequence FillMissing(ds, fill, missopt, methopt) DropMissing(ds, missopt)

Parameters

 ds - a DataSeries object fill - (optional) a value to replace missing values missopt - (optional) equation of the form missing = m1, where m1 can be any expression methopt - (optional) equation of the form method = m2, where m2 can be forward or backward

Description

 • The FillMissing command creates a copy of a DataSeries where missing values are replaced with a value of your choice.
 • The DropMissing command creates a copy of a DataSeries where missing values are removed.
 • For both commands, by default, what is a missing value is determined by the data type of the DataSeries:
 – For floating point data types, the missing value is the appropriate version of undefined. For example, DataSeries objects with data type float[8] use Float(undefined).
 – For hardware integer data types integer[$k$], where $k$ is 1, 2, 4, or 8, the missing value is 0. (Such DataSeries cannot store non-integer values, so one cannot use a version of undefined here.)
 – For string DataSeries objects, the empty string is the missing value.
 – For DataSeries of type truefalseFAIL and boolean_constant, the missing value is FAIL.
 – For all other data types, the missing value is undefined.
 • In order to use a different value as the missing value, you can use the option missing = m1. If you supply this option, then any occurrence of m1 will be considered missing.
 • For the FillMissing command, by default, the value used to replace a missing value depends on the data type of the column it occurs in:
 – For all numeric data types, including floating point and integer, the default value is 0.
 – For the data type string, the default value is the empty string.
 – For the data types truefalse, truefalseFAIL, boolean, and boolean_constant, the default value is false.
 • The fill argument, if it is specified, overrides the value used to replace a missing value.
 • If the method option is used, then any missing values will be replaced either by the last non-missing value before it (with method = forward or method = "forward"), or the first non-missing value after it (with method = backward or method = "backward"). If no such value is available (for example, if the first value is missing and method = forward is specified), then the value used is determined in the same way as if the method option were not specified.
 • For DataSeries objects of hardware integer type and string columns, the default missing value and the default fill value are the same. Using the FillMissing command on such DataSeries has no effect, unless one or more of the fill, missopt, and methopt arguments are specified.

Examples

 > $\mathrm{ds1}≔\mathrm{DataSeries}\left(\left[8,9,0\right]\right)$
 ${\mathrm{ds1}}{≔}\left[\begin{array}{cc}{1}& {8}\\ {2}& {9}\\ {3}& {0}\end{array}\right]$ (1)
 > $\mathrm{ds2}≔\mathrm{DataSeries}\left(\left[8,9,0\right],\mathrm{datatype}=\mathrm{integer}\left[4\right]\right)$
 ${\mathrm{ds2}}{≔}\left[\begin{array}{cc}{1}& {8}\\ {2}& {9}\\ {3}& {0}\end{array}\right]$ (2)
 > $\mathrm{ds3}≔\mathrm{DataSeries}\left(\left[8.,9.,Float\left(\mathrm{undefined}\right)\right],\mathrm{datatype}=\mathrm{float}\left[8\right]\right)$
 ${\mathrm{ds3}}{≔}\left[\begin{array}{cc}{1}& {8.}\\ {2}& {9.}\\ {3}& {Float}{}\left({\mathrm{undefined}}\right)\end{array}\right]$ (3)

DataSeries $\mathrm{ds1}$ has declared type anything, and $\mathrm{ds2}$ has type integer[4]. This means that, for a value in $\mathrm{ds1}$ to be considered missing, it would have to be undefined; in $\mathrm{ds2}$, the value considered missing is 0. Consequently, the DropMissing command will remove a value from $\mathrm{ds2}$, but not from $\mathrm{ds1}$. DataSeries object $\mathrm{ds3}$ also contains the default missing value for its data type, float[8], and consequently it is also removed.

 > $\mathrm{DropMissing}\left(\mathrm{ds1}\right)$
 $\left[\begin{array}{cc}{1}& {8}\\ {2}& {9}\\ {3}& {0}\end{array}\right]$ (4)
 > $\mathrm{DropMissing}\left(\mathrm{ds2}\right)$
 $\left[\begin{array}{cc}{1}& {8}\\ {2}& {9}\end{array}\right]$ (5)
 > $\mathrm{DropMissing}\left(\mathrm{ds3}\right)$
 $\left[\begin{array}{cc}{1}& {8.}\\ {2}& {9.}\end{array}\right]$ (6)

With FillMissing, one can only see a change in $\mathrm{ds3}$ by default.

 > $\mathrm{FillMissing}\left(\mathrm{ds1}\right)$
 $\left[\begin{array}{cc}{1}& {8}\\ {2}& {9}\\ {3}& {0}\end{array}\right]$ (7)
 > $\mathrm{FillMissing}\left(\mathrm{ds2}\right)$
 $\left[\begin{array}{cc}{1}& {8}\\ {2}& {9}\\ {3}& {0}\end{array}\right]$ (8)
 > $\mathrm{FillMissing}\left(\mathrm{ds3}\right)$
 $\left[\begin{array}{cc}{1}& {8.}\\ {2}& {9.}\\ {3}& {0.}\end{array}\right]$ (9)

If we specify a missing value manually, for example, 8, then DropMissing removes that exact value. This applies to $\mathrm{ds1}$ and $\mathrm{ds2}$, but not to $\mathrm{ds3}$, which contains 8. but not 8.

 > $\mathrm{DropMissing}\left(\mathrm{ds1},\mathrm{missing}=8\right)$
 $\left[\begin{array}{cc}{2}& {9}\\ {3}& {0}\end{array}\right]$ (10)
 > $\mathrm{DropMissing}\left(\mathrm{ds2},\mathrm{missing}=8\right)$
 $\left[\begin{array}{cc}{2}& {9}\\ {3}& {0}\end{array}\right]$ (11)
 > $\mathrm{DropMissing}\left(\mathrm{ds3},\mathrm{missing}=8\right)$
 $\left[\begin{array}{cc}{1}& {8.}\\ {2}& {9.}\\ {3}& {Float}{}\left({\mathrm{undefined}}\right)\end{array}\right]$ (12)

For FillMissing, we can specify the value to be used for replacing missing values. This is the fill argument. In the following example, we specify the value 6. This is stored in the last entries of $\mathrm{ds2}$ and $\mathrm{ds3}$; in $\mathrm{ds3}$, it is automatically changed to the floating point value 6., because of the data type of that DataSeries object.

 > $\mathrm{FillMissing}\left(\mathrm{ds1},6\right)$
 $\left[\begin{array}{cc}{1}& {8}\\ {2}& {9}\\ {3}& {0}\end{array}\right]$ (13)
 > $\mathrm{FillMissing}\left(\mathrm{ds2},6\right)$
 $\left[\begin{array}{cc}{1}& {8}\\ {2}& {9}\\ {3}& {6}\end{array}\right]$ (14)
 > $\mathrm{FillMissing}\left(\mathrm{ds3},6\right)$
 $\left[\begin{array}{cc}{1}& {8.}\\ {2}& {9.}\\ {3}& {6.}\end{array}\right]$ (15)

If we specify the option method = backward, then missing values are replaced with later values.

 > $\mathrm{FillMissing}\left(\mathrm{ds1},\mathrm{missing}=9,\mathrm{method}=\mathrm{backward}\right)$
 $\left[\begin{array}{cc}{1}& {8}\\ {2}& {0}\\ {3}& {0}\end{array}\right]$ (16)
 > $\mathrm{FillMissing}\left(\mathrm{ds2},\mathrm{missing}=9,\mathrm{method}=\mathrm{backward}\right)$
 $\left[\begin{array}{cc}{1}& {8}\\ {2}& {0}\\ {3}& {0}\end{array}\right]$ (17)
 > $\mathrm{FillMissing}\left(\mathrm{ds3},\mathrm{missing}=9,\mathrm{method}=\mathrm{backward}\right)$
 $\left[\begin{array}{cc}{1}& {8.}\\ {2}& {Float}{}\left({\mathrm{undefined}}\right)\\ {3}& {Float}{}\left({\mathrm{undefined}}\right)\end{array}\right]$ (18)

Compatibility

 • The DataSeries/FillMissing and DataSeries/DropMissing commands were introduced in Maple 2016.