Validation#
Benchmark#
mlprodict.onnxrt.validate.validate_difference.measure_relative_difference (skl_pred, ort_pred, batch = True, abs_diff = False)
Measures the relative difference between predictions between two ways of computing them. The functions returns nan if shapes are different.
mlprodict.onnxrt.validate.enumerate_benchmark_replay (folder, runtime = ‘python’, time_kwargs = None, skip_long_test = True, time_kwargs_fact = None, time_limit = 4, verbose = 1, fLOG = None)
Replays a benchmark stored with function
enumerate_validated_operator_opsetsor command line validate_runtime. Enumerates the results.
mlprodict.onnxrt.validate.enumerate_validated_operator_opsets (verbose = 0, opset_min = -1, opset_max = -1, check_runtime = True, debug = False, runtime = ‘python’, models = None, dump_folder = None, store_models = False, benchmark = False, skip_models = None, assume_finite = True, node_time = False, fLOG = <built-in function print>, filter_exp = None, versions = False, extended_list = False, time_kwargs = None, dump_all = False, n_features = None, skip_long_test = True, fail_bad_results = False, filter_scenario = None, time_kwargs_fact = None, time_limit = 4, n_jobs = None)
Tests all possible configurations for all possible operators and returns the results.
mlprodict.onnxrt.validate.validate_helper.measure_time (stmt, x, repeat = 10, number = 50, div_by_number = False, first_run = True, max_time = None)
Measures a statement and returns the results as a dictionary.
mlprodict.onnxrt.validate.sklearn_operators (subfolder = None, extended = False, experimental = True)
Builds the list of operators from scikit-learn. The function goes through the list of submodule and get the list of class which inherit from :epkg:`scikit-learn:base:BaseEstimator`.
mlprodict.onnxrt.validate.summary_report (df, add_cols = None, add_index = None)
Finalizes the results computed by function
enumerate_validated_operator_opsets.
mlprodict.onnxrt.validate.validate.latency