culebra.tools module

Tools to automate the execution of experiments.

This module is composed by:

  • The abc sub-module, where some abstract base classes are defined to support the evaluation of trainers and decision managers that let select an adequate solution

  • The evaluation sub-module, providing several classes to make automated experimentation easier.

  • The decision_manager sub-module, which offers several decision managers.

  • The Dataset class to hold and manage the datasets.

  • The EffectSize class, to keep the outcome of an effect size estimation of several batches results

  • The Results class, to manage the results provided by the evaluation of any Trainer

  • The ResultsAnalyzer class, to perform statistical analysis over the results of several experimtent batchs

  • The ResultsComparison class, to keep the outcome of a comparison of several batches results

  • The TestOutcome class, to keep the outcome of a statistical test

Attributes

DEFAULT_ALPHA = 0.05

Default significance level for statistical tests.

DEFAULT_CONFIG_SCRIPT_FILENAME = 'config.py'

Default file name for configuration files.

DEFAULT_EXCEL_FILE_EXTENSION = '.xlsx'

File extension for Excel datasheets.

DEFAULT_HOMOSCEDASTICITY_TEST = <function bartlett>

Default homoscedasticity test.

DEFAULT_OUTLIER_PROPORTION = 0.05

Expected outlier proportion por class.

DEFAULT_NORMALITY_TEST = <function shapiro>

Default normality test.

DEFAULT_NUM_EXPERIMENTS = 1

Default number of experiments in the batch.

DEFAULT_P_ADJUST = 'fdr_tsbky'

Default method for adjusting the p-values with the Dunn’s test.

DEFAULT_RESULTS_BASE_FILENAME = 'results'

Default base name for results files.

DEFAULT_RUN_SCRIPT_FILENAME = 'run.py'

Default file name for the script to run an evaluation.

DEFAULT_SCRIPT_FILE_EXTENSION = '.py'

Default file extension for python scripts.

DEFAULT_SEP = '\\s+'

Default column separator used within dataset files.

DEFAULT_SMOTE_NUM_NEIGHBORS = 5

Default number of neighbors for SMOTE.