skika.hyper_parameter_tuning.drift_detectors.evaluate_drift_detection_experiment

Classes

EvaluateDriftDetection(...)

Prequential evaluation method with adaptive tuning of hyper-parameters for drift detector tuning.

class skika.hyper_parameter_tuning.drift_detectors.evaluate_drift_detection_experiment.EvaluateDriftDetection(list_drifts_detectors, list_names_drifts_detectors, adapt, k_base, dict_k_base, win_adapt_size, stream, n_runs, name_file)

Prequential evaluation method with adaptive tuning of hyper-parameters for drift detector tuning.

Description :

Prequential evaluation method with adaptive tuning of hyper-parameters for drift detector tuning. This class enable to evaluate the performance of an adaptive tuning of drift detectors based on a meta-knowledge base built from results from class evaluate_drift_detection_knowledge.

Parameters :

list_drifts_detectors: list of drift detector object

List of drift detectors to evaluate. Each detector is used for warning and drift detection. If the detector doesn’t give both directly, one should pass two detectors, the first one for the warning detection and the second one for the drift detection.

list_names_drifts_detectors: list of drift detectors names to evaluate.

kBase:

Knowledge base containing meta-features values matched with drift detectors best configurations.

dict_k_base: dict

Dict linking the name of the configurations with the warning and drift detectors

adapt: list of int

List of int to indicate if the drift detector configuration should be :

• 0 : Not adapted

• 1 : Adapted given knowledge with meta-learning

Must be the same size as list_drifts_detectors.

win_adapt_size: list

Length of the sliding window to store and compare meta-features with knowledge.

stream: stream object

Stream on which the detectors are evaluated (Bernoulli stream).

n_runs: int

Number of runs to process The results will be given as mean and std over n_runs.

name_file: str

Name of the file to save the results.

Output:

Csv files containing the performance results.

Example

See https://github.com/scikit-ika/hyper-param-tuning-examples

evaluate()

Evaluate the detectors on the stream

property mean_n_FP

Retrieve the mean number of False Positives :returns: The number of False Positives. :rtype: float

property mean_n_TP

Retrieve the mean number of true Positives :returns: The number of True Positives. :rtype: float

prepare_evaluation()

Prepare variables and stream for drift detection and meta-features extraction