adam_optimiser_type

adam_optimiser_type(
   learning_rate=0.01,
   beta1=0.9,
   beta2=0.999,
   epsilon=.false.,
   num_params=1,
   regulariser=None,
   clip_dict,
   lr_decay
)

The adam_optimiser_type derived type provides a data structure that contains all optimisation/learning parameters for a network model. Most simply, it defines the learning rate for the model.

This type provides an implementation of the Adaptive Momentum estimation optimisation (Adam) method.

Arguments

• learning_rate: A real scalar. The rate of learning applied to the weights.
• beta1: A real scalar (between 0 and 1). Initial decay rate of the first moment of the gradient.
• beta2: A real scalar (between 0 and 1). Initial decay rate of the second moment of the gradient.
• epsilon: A small real scalar. Used for zero division handling.
• num_params: An integer scalar. The number of learnable parameters in the model.
• regulariser: A derived type extended from the base_regulariser_type derived type.
• clip_dict: A derived data type defining weight clipping parameters. These nested parameters can be set using the set_clip procedure.
• lr_decay: A derived type extended from the base_lr_decay_type derived type.