culebra.tools.Dataset class

class Dataset(inputs: _Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes], outputs: _Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes])

Bases: Base

Create a dataset from input and output data.

Both inputs and outputs must be array-like objects, such as NumPy arrays, lists of lists, tuples of tuples, pandas Series or pandas DataFrame objects.

inputs must represent a two-dimensional structure where each row is a sample and each column is a feature.

outputs may be either one-dimensional or two-dimensional. When a two-dimensional structure is provided, only its first column is used, assuming a single output value per sample.

Parameters:

  • inputs (numpy.typing.ArrayLike) – Input samples. Rows correspond to samples and columns correspond to features.

  • outputs (numpy.typing.ArrayLike) – Output values associated with the input samples. It may be one-dimensional or two-dimensional. If it is two-dimensional, only the first column is considered.

Raises:

  • ValueError – If inputs and outputs do not contain the same number of samples.

  • ValueError – If no input samples are provided.

  • ValueError – If no output values are provided.

Class methods

classmethod Dataset.load(filename: str) → Base

Load a serialized object from a file.

Parameters:

filename (str) – The file name.

Returns:

The loaded object

Raises:
classmethod Dataset.from_mat(src: str | PathLike[str] | BytesIO, inputs_key: str = 'X', outputs_key: str = 'Y') → None

Load a dataset from a mat file.

Parameters:

  • src (str | PathLike[str] | io.BytesIO) – Source

  • inputs_key (str) – Key to access the dataset inputs. Defaults to ‘X’

  • outputs_key (str) – Key to access the dataset oututs. Defaults to ‘Y’

Raises:

  • ValueError – If src is not a valid source

  • ValueError – If either inputs_key or outputs_key is not a valid key

Returns:

The dataset

Return type:

Dataset

classmethod Dataset.from_text(*files: tuple[str | PathLike[str] | TextIOBase], output_index: int | None = None, sep: str = '\\s+') → None

Load a dataset from one or two text files.

Datasets can be organized in only one file or in two files. If only one file is used, then output_index must be used to indicate which column stores the output values. If output_index is omitted, it will be assumed that the dataset is composed by two consecutive files, the first one containing the input columns and the second one storing the output column. Only the first column of the second file will be loaded in this case (just one output value per sample).

Parameters:

  • files (tuple[str | PathLike[str] | TextIOBase]) – Files containing the dataset. If output_index is omitted, two files are necessary, the first one containing the input columns and the second one containing the output column. Otherwise, only one file will be used to access to the whole dataset (input and output columns)

  • output_index (int) – If the dataset is provided with only one file, this parameter indicates which column in the file does contain the output values. Otherwise this parameter must be omitted (set to None) to express that inputs and ouputs are stored in two different files. Its default value is None

  • sep (str) – Column separator used within the files. Defaults to DEFAULT_SEP

Raises:

  • ValueError – If files is empty

  • TypeError – If output_index is not None or int

  • TypeError – If sep is not a string

  • IndexError – If output_index is out of range

  • RuntimeError – If output_index is None and only one file is provided

  • RuntimeError – When loading a dataset composed of two files, if the file containing the input columns and the file containing the output column do not have the same number of rows.

  • RuntimeError – If any file is empty

Returns:

The dataset

Return type:

Dataset

classmethod Dataset.from_uci(name: str | None = None, id_number: int | None = None) → Dataset

Load a dataset from the UCI ML repository.

The dataset can be identified by either its id_number or its name, but only one of these should be provided.

If the dataset has more than one output column, only the first column is considered.

Parameters:

  • name (str) – Dataset name, or substring of name, optional

  • id_number (int) – Dataset ID for UCI ML Repository, optional

Raises:

RuntimeError – If the dataset can not be loaded

Returns:

The dataset

Return type:

Dataset

Properties

property Dataset.inputs: ndarray

Input data of the dataset.

Return type:

ndarray

property Dataset.num_feats: int

Number of features in the dataset.

Return type:

int

property Dataset.outputs: ndarray

Output data of the dataset.

Return type:

ndarray

property Dataset.size: int

Number of samples in the dataset.

Return type:

int

Methods

Dataset.append_random_features(num_feats: int, random_seed: int | None = None) → Dataset

Return a new dataset with some random features appended.

Parameters:

  • num_feats (int) – Number of random features to be appended (greater than 0)

  • random_seed (int) – Random seed for the random generator, defaults to None

Raises:

  • TypeError – If the number of random features is not an integer

  • ValueError – If the number of random features not greater than 0

Returns:

The new dataset

Return type:

Dataset

Dataset.drop_missing() → Dataset

Drop samples with missing values.

Returns:

A clean dataset

Return type:

Dataset

Dataset.dump(filename: str) → None

Serialize this object and save it to a file.

Parameters:

filename (str) – The file name.

Raises:
Dataset.normalize() → Dataset

Normalize the dataset between 0 and 1.

Returns:

A normalized dataset

Return type:

Dataset

Dataset.oversample(n_neighbors: int = 5, random_seed: int | None = None) → Dataset

Oversample all classes but the majority class.

All classes but the majority class are oversampled to equal the number of samples of the majority class. SMOTE is used for oversampling, but if any class has less than n_neighbors samples, RandomOverSampler is first applied

Parameters:
Returns:

An oversampled dataset

Return type:

Dataset

Dataset.remove_outliers(prop: float = 0.05, random_seed: int | None = None) → Dataset

Remove the outliers.

Parameters:
Returns:

A clean dataset

Return type:

Dataset

Dataset.scale() → Dataset

Scale features robust to outliers.

Returns:

A scaled dataset

Return type:

Dataset

Dataset.select_features(feats: Sequence[int]) → Dataset

Return a new dataset only with some selected features.

Parameters:

feats (Sequence[int]) – Indices of the selected features

Returns:

The new dataset

Return type:

Dataset

Dataset.split(test_prop: float, random_seed: int | None = None) → tuple[Dataset, Dataset]

Split the dataset.

Parameters:

  • test_prop (float) – Proportion of the dataset used as test data. The remaining samples will be returned as training data

  • random_seed (int) – Random seed for the random generator, defaults to None

Raises:
Returns:

The training and test datasets

Return type:

tuple[Dataset]

Dataset.to_text(filename: str | PathLike[str], sep: str = '\\s+') → None

Save the dataset to a text file.

Parameters:

  • filename (PathLike[str]) – Destination file name

  • sep (str) – Column separator used within the files. Defaults to DEFAULT_SEP

Private methods

Dataset._get_repr_properties() → dict[str, object]

Return the subset of properties used for __repr__.

Filters and evaluates all class-level @property attributes, returning only those intended for representation purposes. Private properties (names starting with _) are excluded.

Returns:

Mapping of property names to their corresponding values.

Return type:

dict[str, object]