culebra.fitness_function.abc.ClassificationScorer class

class ClassificationScorer(training_data: Dataset, test_data: Dataset | None = None, test_prop: float | None = None, cv_folds: int | None = None, classifier: ClassifierMixin | None = None)

Construct a fitness function.

If test_data are provided, the whole training_data are used to train. Otherwise, if test_prop is provided, training_data are split (stratified) into training and test data each time evaluate() is called and a Monte Carlo cross validation is applied. Finally, if both test_data and test_prop are omitted, a k-fold cross-validation is applied.

Parameters:

Class attributes

class culebra.fitness_function.abc.ClassificationScorer.Fitness

Handles the values returned by the evaluate() method within a Solution.

This class must be implemented within all the ClassificationScorer subclasses, as a subclass of the Fitness class, to define its three class attributes ( weights, names, and thresholds) according to the fitness function.

Class methods

classmethod ClassificationScorer.load_pickle(filename: str) → Base

Load a pickled object from a file.

Parameters:

filename (str) – The file name.

Raises:
classmethod ClassificationScorer.set_fitness_thresholds(thresholds: float | Sequence[float]) → None

Set new fitness thresholds.

Modifies the thresholds of the Fitness objects generated by this fitness function.

Parameters:

thresholds (float or Sequence of float) – The new thresholds. If only a single value is provided, the same threshold will be used for all the objectives. Different thresholds can be provided in a Sequence

Raises:
classmethod ClassificationScorer.get_fitness_objective_threshold(obj_name: str) → None

Get the similarity threshold for the given objective.

Parameters:

obj_name (str) – Objective name whose threshold is returned

Raises:

  • TypeError – If obj_name isn’t a string

  • ValueError – If value isn’t a valid objective name

classmethod ClassificationScorer.set_fitness_objective_threshold(obj_name: str, value: float) → None

Set a similarity threshold for the given fitness objective.

Parameters:

  • obj_name (str) – Objective name whose threshold is modified

  • value (float) – New value for the similarity threshold.

Raises:

  • TypeError – If obj_name isn’t a string or value isn’t a real number

  • ValueError – If obj_name isn’t a valid objective name or value is lower than 0

Properties

property ClassificationScorer.is_noisy: int

Return True if the fitness function is noisy.

property ClassificationScorer.num_obj: int

Get the number of objectives.

Type:

int

property ClassificationScorer.num_nodes: int | None

Return the problem graph’s number of nodes for ACO-based trainers.

Subclasses solvable with ACO-based approaches should override this property to return the problem graph’s number of nodes. Otherwise, None is returned

Returns:

The problem graph’s number of nodes if an ACO-based approach is applicable or None otherwise

Return type:

int

property ClassificationScorer.training_data: Dataset

Get and set the training dataset.

Getter:

Return the training dataset

Setter:

Set a new training dataset

Type:

Dataset

Raises:

TypeError – If set to an invalid dataset

property ClassificationScorer.test_data: Dataset | None

Get and set the test dataset.

Getter:

Return the test dataset

Setter:

Set a new test dataset

Type:

Dataset

Raises:

TypeError – If set to an invalid dataset

property ClassificationScorer.test_prop: float | None

Get and set the proportion of data used to test.

Getter:

Return the test data proportion

Setter:

Set a new value for the test data porportion. A real value in (0, 1) or None is expected

Type:

float

Raises:

  • TypeError – If set to a value which is not a real number

  • ValueError – If set to a value which is not in (0, 1)

property ClassificationScorer.classifier: ClassifierMixin

Get and set the classifier applied within this fitness function.

Getter:

Return the classifier

Setter:

Set a new classifier

Type:

ClassifierMixin

Raises:

TypeError – If set to a value which is not a classifier

Private properties

abstract property ClassificationScorer._worst_score: float

Worst achievable score.

This property must be overridden by subclasses to return a correct value.

Raises:

NotImplementedError – if has not been overridden

Type:

float

Methods

ClassificationScorer.save_pickle(filename: str) → None

Pickle this object and save it to a file.

Parameters:

filename (str) – The file name.

Raises:
ClassificationScorer.heuristic(species: Species) → Sequence[ndarray, ...] | None

Get the heuristic matrices for ACO-based trainers.

Subclasses solvable with ACO-based approaches should override this method. Otherwise, None is returned

Parameters:

species (Species) – Species constraining the problem solutions

Returns:

A sequence of heuristic matrices if an ACO-based approach is applicable or None otherwise

Return type:

Sequence of ndarray

ClassificationScorer.evaluate(sol: Solution, index: int | None = None, representatives: Sequence[Solution] | None = None) → Tuple[float, ...]

Evaluate a solution.

Parameters:

  • sol (Solution) – Solution to be evaluated.

  • index (int, optional) – Index where sol should be inserted in the representatives sequence to form a complete solution for the problem. Only used by cooperative problems

  • representatives (A Sequence containing instances of Solution, optional) – Representative solutions of each species being optimized. Only used by cooperative problems

Returns:

The fitness values for sol

Return type:

tuple of float

Private methods

abstract static ClassificationScorer._score(outputs: Sequence[float], outputs_pred: Sequence[float], **kwargs: Any) → float

Score function to be used in the evaluation.

This method must be overridden by subclasses to return a correct value.

ClassificationScorer._final_training_test_data(sol: Solution) → Tuple[Dataset, Dataset]

Get the final training and test data.

Parameters:

sol (Solution) – Solution to be evaluated. It may influence the final datasets

Returns:

The final training and test datasets

Return type:

tuple of Dataset

ClassificationScorer._evaluate_train_test(training_data: Dataset, test_data: Dataset) → Tuple[float, ...]

Evaluate a solution.

This method must be overridden by subclasses to return a correct value.

Parameters:

  • training_data (Dataset) – The training dataset

  • test_data (Dataset) – The test dataset

Returns:

The fitness values for sol

Return type:

tuple of float

ClassificationScorer._evaluate_mccv(training_data: Dataset) → Tuple[float, ...]

Evaluate a solution.

The training_data are split (stratified) into training and test data according to test_prop each time the solution is evalueted and a Monte Carlo cross-validation is applied.

Parameters:

training_data (Dataset) – The training dataset

Returns:

The fitness values for sol

Return type:

tuple of float

ClassificationScorer._evaluate_kfcv(training_data: Dataset) → Tuple[float, ...]

Evaluate a solution.

A k-fold cross-validation is applied using the training_data with cv_folds folds.

Parameters:

training_data (Dataset) – The training dataset

Returns:

The fitness values for sol

Return type:

tuple of float