v0.3.0
The interface update
This version greatly improves the interface of backward
and mtl_backward
, at the cost of some easy-to-fix breaking changes (some parameters of these functions have been renamed, or their order has been swapped due to becoming optional).
Downstream changes to make to keep using backward
and mtl_backward
:
- Rename
A
toaggregator
or pass it as a positional argument. - For
backward
, unless you specifically want to avoid differentiating with respect to some parameters, you can now simply use the default value of theinputs
argument. - For
mtl_backward
, unless you want to customize which params should be updated with a step of JD and which should be updated with a step of GD, you can now simply use the default value of theshared_params
and of thetasks_params
arguments. - If you keep providing the
inputs
or theshared_params
ortasks_params
arguments as positional arguments, you should provide them after the aggregator.
For instance,
backward(tensors, inputs, A=aggregator)
should become
backward(tensors, aggregator)
and
mtl_backward(losses, features, tasks_params, shared_params, A=aggregator)
should become
mtl_backward(losses, features, aggregator)
We thank @raeudigerRaeffi for sharing his idea of having default values for the tensors with respect to which the differentiation should be made in backward
and mtl_backward
, and for implementing the first working version of the function that automatically finds these parameters from the autograd graph.
Changelog
Added
- Added a default value to the
inputs
parameter ofbackward
. If not provided, theinputs
will
default to all leaf tensors that were used to compute thetensors
parameter. This is in line
with the behavior of
torch.autograd.backward. - Added a default value to the
shared_params
and to thetasks_params
arguments of
mtl_backward
. If not provided, theshared_params
will default to all leaf tensors that were
used to compute thefeatures
, and thetasks_params
will default to all leaf tensors that were
used to compute each of thelosses
, excluding those used to compute thefeatures
. - Note in the documentation about the incompatibility of
backward
andmtl_backward
with tensors
that retain grad.
Changed
- BREAKING: Changed the name of the parameter
A
toaggregator
inbackward
and
mtl_backward
. - BREAKING: Changed the order of the parameters of
backward
andmtl_backward
to make it
possible to have a default value forinputs
and forshared_params
andtasks_params
,
respectively. Usages ofbackward
andmtl_backward
that rely on the order between arguments
must be updated. - Switched to the PEP 735 dependency groups format in
pyproject.toml
(from a[tool.pdm.dev-dependencies]
to a[dependency-groups]
section). This
should only affect development dependencies.
Fixed
- BREAKING: Added a check in
mtl_backward
to ensure thattasks_params
andshared_params
have no overlap. Previously, the behavior in this scenario was quite arbitrary.