pyepo.metric

Performance evaluation

Submodules

• pyepo.metric.metrics
• pyepo.metric.mse
• pyepo.metric.regret
• pyepo.metric.unambregret

Functions

MSE(predmodel, dataloader)	A function to evaluate model performance with MSE
calRegret(optmodel, pred_cost, true_cost, true_obj)	A function to calculate normalized true regret for a batch
regret(predmodel, optmodel, dataloader)	A function to evaluate model performance with normalized true regret
calUnambRegret(optmodel, pred_cost, true_cost, true_obj)	A function to calculate normalized unambiguous regret for a batch
unambRegret(predmodel, optmodel, dataloader[, tolerance])	A function to evaluate model performance with normalized unambiguous regret
SPOError(pred_cost, true_cost, model_type, args)	A function to calculate normalized true regret
makeSkScorer(optmodel)	A function to create sklearn scorer
makeAutoSkScorer(optmodel)	A function to create Auto-SKlearn scorer

Package Contents

pyepo.metric.MSE(predmodel, dataloader)

A function to evaluate model performance with MSE

Parameters:

• predmodel (nn) – a regression neural network for cost prediction

• dataloader (DataLoader) – Torch dataloader from optDataSet

Returns:

MSE loss

Return type:

float

pyepo.metric.calRegret(optmodel, pred_cost, true_cost, true_obj)

A function to calculate normalized true regret for a batch

Parameters:

• optmodel (optModel) – optimization model

• pred_cost (torch.tensor) – predicted costs

• true_cost (torch.tensor) – true costs

• true_obj (torch.tensor) – true optimal objective values

Returns:predmodel

float: true regret losses

pyepo.metric.regret(predmodel, optmodel, dataloader)

A function to evaluate model performance with normalized true regret

Parameters:

• predmodel (nn) – a regression neural network for cost prediction

• optmodel (optModel) – an PyEPO optimization model

• dataloader (DataLoader) – Torch dataloader from optDataSet

Returns:

true regret loss

Return type:

float

pyepo.metric.calUnambRegret(optmodel, pred_cost, true_cost, true_obj, tolerance=1e-05)

A function to calculate normalized unambiguous regret for a batch

Parameters:

• optmodel (optModel) – optimization model

• pred_cost (torch.tensor) – predicted costs

• true_cost (torch.tensor) – true costs

• true_obj (torch.tensor) – true optimal objective values

Returns:

unambiguous regret losses

Return type:

float

pyepo.metric.unambRegret(predmodel, optmodel, dataloader, tolerance=1e-05)

A function to evaluate model performance with normalized unambiguous regret

Parameters:

• predmodel (nn) – a regression neural network for cost prediction

• optmodel (optModel) – an PyEPO optimization model

• dataloader (DataLoader) – Torch dataloader from optDataSet

Returns:

unambiguous regret loss

Return type:

float

pyepo.metric.SPOError(pred_cost, true_cost, model_type, args)

A function to calculate normalized true regret

Parameters:

• pred_cost (numpy.array) – predicted costs

• true_cost (numpy.array) – true costs

• model_type (ABCMeta) – optModel class type

• args (dict) – optModel args

Returns:

regret loss

Return type:

float

pyepo.metric.makeSkScorer(optmodel)

A function to create sklearn scorer

Parameters:

optmodel (optModel) – optimization model

Returns:

callable object that returns a scalar score; less is better.

Return type:

scorer

pyepo.metric.makeAutoSkScorer(optmodel)

A function to create Auto-SKlearn scorer

Parameters:

optmodel (optModel) – optimization model

Returns:

callable object that returns a scalar score; less is better.

Return type:

scorer