skika.hyper_parameter_tuning.drift_detectors.evaluate_drift_detection_knowledge

Classes

EvaluateDriftDetection([...])

Prequential evaluation method to collect knowledge for drift detector tuning.

class skika.hyper_parameter_tuning.drift_detectors.evaluate_drift_detection_knowledge.EvaluateDriftDetection(list_drifts_detectors=[[ADWIN(delta=0.5), ADWIN(delta=0.05)], [DDM(min_num_instances=None, out_control_level=3.0, warning_level=2.0)], [PageHinkley(alpha=0.9999, delta=0.05, min_instances=100, threshold=50), PageHinkley(alpha=0.9999, delta=0.05, min_instances=100, threshold=100)]], list_names_drifts_detectors=['ADWIN', 'DDM', 'PH'], stream=<skika.data.bernoulli_stream.BernoulliStream object>, n_runs=1, name_file=None)

Prequential evaluation method to collect knowledge for drift detector tuning.

Description :

Class to evaluate the performance of the drift detection for the knowledge computation. Performance is evaluate with the numbers of TP and FP detections.

Parameters :

list_drifts_detectorslist of drift detector object

List of drift detectors to evaluate. Each detector is used for warning and drift detection. If the detector doesn’t handle both directly, one should pass two detectors, the first one for the warning detection and the second one for the drift detection.

list_names_drifts_detectors: list of str.

List of drift detectors names to evaluate.

stream: stream object

Stream on which the detectors are evaluated.

n_runs: int

Number of runs to process. The results will be given as mean and std over n_runs.

name_file: str

Name of the file to save the results.

Output:

Csv files containing the performance results to be exploited to build the knowledge base.

Example

See https://github.com/scikit-ika/hyper-param-tuning-examples

evaluate()

Evaluate the detectors on the stream

interval_measure(detecId)

Measure the interval between 2 drifts.

magnitude_measure(detecId, p, q)

Extraction of the magnitude of a single drift

Attributes : detecId : ID of the drift detector (int) p : distribution before drift (array) q : distribution after drift (array)

prepare_evaluation()

Prepare variables and stream for drift detection and meta-features extraction

severity_measure(detecId)

Measure severity of the drift. The severity is defined as the number of instances in the warning interval.