mltk.core.EvaluateAutoEncoderMixin

class EvaluateAutoEncoderMixin[source]

Provides evaluation properties and methods to the base MltkModel

Note

This mixin is specific to “auto-encoder” models

Refer to the Model Evaluation guide for more details.

Properties

eval_augment

Enable random augmentations during evaluation Default: False NOTE: This is only used if the DataGeneratorDatasetMixin or sub-class is used by the MltkModel

eval_classes

List if classes to use for evaluation.

eval_custom_function

Custom evaluation callback

eval_max_samples_per_class

The maximum number of samples for a given class to use during evaluation.

eval_shuffle

Shuffle data during evaluation Default: False

eval_steps_per_epoch

Total number of steps (batches of samples) before declaring the prediction round finished.

scoring_function

The auto-encoder scoring function to use during evaluation

Methods

__init__

get_scoring_function

Return the scoring function used during evaluation

property scoring_function

The auto-encoder scoring function to use during evaluation

If None, then use the mltk_model.loss function

Default: None

property eval_classes

List if classes to use for evaluation. The first element should be considered the ‘normal’ class, every other class is considered abnormal and compared independently. This is used if the –classes argument is not supplied to the eval command.

Default: [normal, abnormal]

property eval_augment

Enable random augmentations during evaluation Default: False NOTE: This is only used if the DataGeneratorDatasetMixin or sub-class is used by the MltkModel

property eval_custom_function

Custom evaluation callback

This is invoked during the mltk.core.evaluate_model() API.

The given function should have the following signature:

my_custom_eval_function(my_model:MyModel, built_model: Union[KerasModel, TfliteModel]) -> EvaluationResults:
    results = EvaluationResults(name=my_model.name)

    if isinstance(built_model, KerasModel):
        results['overall_accuracy] = calculate_accuracy(built_model)
    return results
property eval_max_samples_per_class

The maximum number of samples for a given class to use during evaluation. If -1 then use all available samples Default: -1

property eval_shuffle

Shuffle data during evaluation Default: False

property eval_steps_per_epoch

Total number of steps (batches of samples) before declaring the prediction round finished. Ignored with the default value of None. If x is a tf.data dataset and steps is None, predict will run until the input dataset is exhausted.

get_scoring_function()[source]

Return the scoring function used during evaluation

Return type:

Callable