Support sensi_orders in gradient checks

[dilpath]

While debugging another model, the issue arose again that sometimes Objective(x, sensi_orders=(0,)) provides a different fval than Objective(x, sensi_orders=(0,1)). Tolerances etc. can be adjusted until these two are similar; however, pyPESTO's check_grad might incorrectly report that gradients are not close, because it computes fval with sensis

pyPESTO/pypesto/objective/base.py

Line 535 in ead29b3

fval, grad = self(x, (0 + order, 1 + order), mode)

but fval_p and fval_m (for the plus/minus steps used in finite difference approximations) without sensis

pyPESTO/pypesto/objective/base.py

Line 556 in ead29b3

fval_p = self(x_p, (0,), mode)

pyPESTO/pypesto/objective/base.py

Line 561 in ead29b3

fval_m = self(x_m, (0,), mode)

This could be resolved by using the same sensi_orders in all function calls, or otherwise providing some way for the user to specify this.

[dweindl]

This could be resolved by using the same sensi_orders in all function calls, or otherwise providing some way for the user to specify this.

It might make sense to address that independently of gradient checks, since this will also be a problem for any optimizer that calls the objective with varying sensi_orders. Maybe a simple wrapper around Objective that always passes a certain sensi_orders internally and then filters the result to match the externally requested sensi_orders.