pdmlabs.evaluation.evaluation

pdmlabs.evaluation.evaluation#

Functions

AUCPR(predictions[, Failuretype, ...])

AUCPR_new(predictions[, Failuretype, ...])

AUCPR_ranges_new(predictions, anomalyranges, ...)

Episode_Borders(tempdatesofscores, ...)

Episodes_AUCPR(dictresults[, ids, plotThem, ...])

Gather_Episodes(score, timescore, thresholds)

breakIntoEpisodes(alarms, failuredates[, ...])

breakIntoEpisodesWithCodes(alarms, ...)

calculatePHandLead(PH, lead)

calculate_AD_levels(anomalyranges, ...)

calculate_ts_recall(anomalyranges, ...)

episodeMyPresicionTPFP(episodealarms, ...)

evaluate_episodes(dictresults[, ids, ...])

extract_anomaly_ranges(maintenances, ...)

formulatedataForEvaluation(predictions, ...)

ignore(valuedate, tupleIngoredaes)

ingnorecounter(valuedate, tupleIngoredaes)

integrate(x, y)

myeval(predictions, threshold[, ...])

This method is used to perform PdM evaluation of Run-to-Failures examples.

myeval_multiPH(predictions, Failuretype, ...)

This method is used to perform PdM evaluation of Run-to-Failures examples.

pdm_eval_multi_PH(predictions, threshold[, ...])

This method is used to perform PdM evaluation of Run-to-Failures examples.

plotforevaluationNonFailure(timestyle, ...)

plotforevalurion(timestyle, ignoredates, ...)

pdmlabs.evaluation.evaluation.AUCPR(predictions, Failuretype=None, datesofscores=[], maintenances=None, isfailure=[], PH='100', lead='20', plotThem=True, ignoredates=[], beta=1, resolution=100, slidingWindow_vus=0)#
pdmlabs.evaluation.evaluation.AUCPR_new(predictions, Failuretype=None, datesofscores=[], maintenances=None, isfailure=[], PH='100', lead='20', plotThem=True, ignoredates=[], beta=1, resolution=100, slidingWindow_vus=0)#
pdmlabs.evaluation.evaluation.AUCPR_ranges_new(predictions, anomalyranges, leadranges, beta=1, resolution=100, slidingWindow_vus=0)#
pdmlabs.evaluation.evaluation.Episode_Borders(tempdatesofscores, PredictiveHorizon, leadtime, timestyle, timestylelead)#
pdmlabs.evaluation.evaluation.Episodes_AUCPR(dictresults, ids=None, plotThem=False, PH='14 days', lead='1 hours', beta=2, phmapping=None)#
pdmlabs.evaluation.evaluation.Gather_Episodes(score, timescore, thresholds, failures=[], failurecodes=[], dictresults={}, id='id', description='')#
Parameters:

  • score (the produced score of a technique)

  • timescore (the correspoiding timestamps (pandas.datetime) for the produced score)

  • thresholds (a list with a threshold value for each score value.)

  • failures (dates of failures)

  • failurecodes (the type of failure)

  • dictresults (the dictionary we want to store the parameters for evaluation.)

  • id (the name under whihch the parameters will be stored)

  • description (a string for description of experiment)

pdmlabs.evaluation.evaluation.breakIntoEpisodes(alarms, failuredates, thresholds=None, dates=None)#
pdmlabs.evaluation.evaluation.breakIntoEpisodesWithCodes(alarms, failuredates, failurecodes, thresholds, dates)#
pdmlabs.evaluation.evaluation.calculatePHandLead(PH, lead)#
pdmlabs.evaluation.evaluation.calculate_AD_levels(anomalyranges, leadranges, predictions, ignore_range, threshold, beta)#
pdmlabs.evaluation.evaluation.calculate_ts_recall(anomalyranges, predictionsforrecall)#
pdmlabs.evaluation.evaluation.episodeMyPresicionTPFP(episodealarms, tempdatesofscores, PredictiveHorizon, leadtime, timestyle, timestylelead, ignoredates)#
pdmlabs.evaluation.evaluation.evaluate_episodes(dictresults, ids=None, plotThem=False, PH='14 days', lead='1 hours', beta=2, phmapping=None)#
Parameters:

  • dictresults (is a dictionary produced from Gather_Episodes, containing a dictionary for each evaluation with keys:) – isfailure, episodescores,episodesdates,episodesthresholds,failuretypes,failuretypes

  • ids (The ids in form of list of the dictionary which is going to be evaluated)

  • PH (is the predictive horizon used for recall, can be set in time domain using one from accepted time spans:) – [“days”, “seconds”, “microseconds”, “milliseconds”, “minutes”, “hours”, “weeks”] using a space after the number e.g. PH=”8 hours” in case of single number then a counter related predictive horizon is used (e.g. PH=”100” indicates that the last 100 values are used for predictive horizon It is possible to accept differenc PH of different failure codes.

  • lead (similar to PH for lead time.)

  • beta (the beta value to calculate f-beta score)

  • plotThem (True in case we want to plot episodes.)

pdmlabs.evaluation.evaluation.extract_anomaly_ranges(maintenances, PHS_leads, isfailure, datesofscores)#
pdmlabs.evaluation.evaluation.formulatedataForEvaluation(predictions, threshold, datesofscores, isfailure, maintenances)#
pdmlabs.evaluation.evaluation.ignore(valuedate, tupleIngoredaes)#
pdmlabs.evaluation.evaluation.ingnorecounter(valuedate, tupleIngoredaes)#
pdmlabs.evaluation.evaluation.integrate(x, y)#
pdmlabs.evaluation.evaluation.myeval(predictions, threshold, datesofscores=[], maintenances=None, isfailure=[], PH='100', lead='20', plotThem=True, ignoredates=[], beta=1)#

This method is used to perform PdM evaluation of Run-to-Failures examples.

Parameters:

  • predictions (Either a flatted list of all predictions from all episodes or) – list with a list of prediction for each of episodes

  • datesofscores (Either a flatted list of all indexes (timestamps) from all episodes or) – list with a list of indexes (timestamps) for each of episodes If it is empty list then aritificially indexes are gereated

  • threshold (can be either a list of thresholds (equal size to all predictions), a list with size equal to number of episodes, a single number.)

  • maintenances (is used in case the predictions are passed as flatten array (default None)) – list of ints which indicate the time of maintenance (the position in predictions where a new episode begins) or the end of the episode.

  • isfailure (a binary array which is used in case we want to pass episode which end with no failure, and thus don't contribute) – to recall calculation. For example isfailure=[1,1,0,1] indicates that the third episode end with no failure, while the others end with a failure. default value is empty list which indicate that all episodes end with failure.

  • PH (is the predictive horizon used for recall, can be set in time domain using one from accepted time spans:) – [“days”, “seconds”, “microseconds”, “milliseconds”, “minutes”, “hours”, “weeks”] using a space after the number e.g. PH=”8 hours” in case of single number then a counter related predictive horizon is used (e.g. PH=”100” indicates that the last 100 values are used for predictive horizon

  • lead (represent the lead time (the time to ignore in last part of episode when we want to test the predict capabilities of algorithm)) – same rules as the PH are applied.

  • ignoredates (a list with tuples in form (begin,end) which indicate periods to be ignored in the calculation of recall and precision) – begin and end values must be same type with datesofscores instances (pd.datetime or int)

  • beta=1. (beta is used to calculate fbeta score deafult)

pdmlabs.evaluation.evaluation.myeval_multiPH(predictions, Failuretype, threshold, datesofscores=[], maintenances=None, isfailure=[], PH=[('type 1', '100')], lead=[('type 1', '10')], plotThem=True, ignoredates=[], beta=1)#

This method is used to perform PdM evaluation of Run-to-Failures examples.

Parameters:

  • predictions (Either a flatted list of all predictions from all episodes or) – list with a list of prediction for each of episodes

  • datesofscores (Either a flatted list of all indexes (timestamps) from all episodes or) – list with a list of indexes (timestamps) for each of episodes If it is empty list then aritificially indexes are gereated

  • threshold (can be either a list of thresholds (equal size to all predictions), a list with size equal to number of episodes, a single number.)

  • maintenances (is used in case the predictions are passed as flatten array (default None)) – list of ints which indicate the time of maintenance (the position in predictions where a new episode begins) or the end of the episode.

  • isfailure (a binary array which is used in case we want to pass episode which end with no failure, and thus don't contribute) – to recall calculation. For example isfailure=[1,1,0,1] indicates that the third episode end with no failure, while the others end with a failure. default value is empty list which indicate that all episodes end with failure.

  • PH (is the predictive horizon used for recall, can be set in time domain using one from accepted time spans:) – [“days”, “seconds”, “microseconds”, “milliseconds”, “minutes”, “hours”, “weeks”] using a space after the number e.g. PH=”8 hours” in case of single number then a counter related predictive horizon is used (e.g. PH=”100” indicates that the last 100 values are used for predictive horizon

  • lead (represent the lead time (the time to ignore in last part of episode when we want to test the predict capabilities of algorithm)) – same rules as the PH are applied.

  • ignoredates (a list with tuples in form (begin,end) which indicate periods to be ignored in the calculation of recall and precision) – begin and end values must be same type with datesofscores instances (pd.datetime or int)

  • beta=1. (beta is used to calculate fbeta score deafult)

pdmlabs.evaluation.evaluation.pdm_eval_multi_PH(predictions, threshold, datesofscores=[], maintenances=None, isfailure=[], PH=[('type 1', '100')], lead=[('type 1', '10')], Failuretype=[], plotThem=True, ignoredates=[], beta=1)#

This method is used to perform PdM evaluation of Run-to-Failures examples.

Parameters:

  • predictions (Either a flatted list of all predictions from all episodes or) – list with a list of prediction for each of episodes

  • datesofscores (Either a flatted list of all indexes (timestamps) from all episodes or) – list with a list of indexes (timestamps) for each of episodes If it is empty list then aritificially indexes are gereated

  • threshold (can be either a list of thresholds (equal size to all predictions), a list with size equal to number of episodes, a single number.)

  • maintenances (is used in case the predictions are passed as flatten array (default None)) – list of ints which indicate the time of maintenance (the position in predictions where a new episode begins) or the end of the episode.

  • isfailure (a binary array which is used in case we want to pass episode which end with no failure, and thus don't contribute) – to recall calculation. For example isfailure=[1,1,0,1] indicates that the third episode end with no failure, while the others end with a failure. default value is empty list which indicate that all episodes end with failure.

  • PH (is the predictive horizon used for recall, can be set in time domain using one from accepted time spans:) – [“days”, “seconds”, “microseconds”, “milliseconds”, “minutes”, “hours”, “weeks”] using a space after the number e.g. PH=”8 hours” in case of single number then a counter related predictive horizon is used (e.g. PH=”100” indicates that the last 100 values are used for predictive horizon

  • lead (represent the lead time (the time to ignore in last part of episode when we want to test the predict capabilities of algorithm)) – same rules as the PH are applied.

  • ignoredates (a list with tuples in form (begin,end) which indicate periods to be ignored in the calculation of recall and precision) – begin and end values must be same type with datesofscores instances (pd.datetime or int)

  • beta=1. (beta is used to calculate fbeta score deafult)

pdmlabs.evaluation.evaluation.plotforevaluationNonFailure(timestyle, ignoredates, tempdatesofscores, episodethreshold, episodePred, countaxes, axes)#
pdmlabs.evaluation.evaluation.plotforevalurion(timestyle, ignoredates, tempdatesofscores, episodethreshold, episodePred, countaxes, axes, borderph, border1)#
Contents