tenzr::optim::GradientTransformation¶

Abstract gradient transformation defining Init and Update for an optimizer.

#include <transform.h>

Inherited by tenzr::optim::Adam, tenzr::optim::Lion, tenzr::optim::SGD

Public Functions¶

Public Functions Documentation¶

function ~GradientTransformation¶

virtual ~GradientTransformation() =default

function Init¶

virtual OptState Init(
    const std::unordered_map< std::string, Tensor > & params
) =0

Parameters:

• params Parameters to optimize, keyed by name.

Return: Fresh optimizer state.

Reimplemented by: tenzr::optim::Adam::Init, tenzr::optim::Lion::Init, tenzr::optim::SGD::Init

Allocate zero-initialized state for params.

function Update¶

virtual std::unordered_map< std::string, Tensor > Update(
    const std::unordered_map< std::string, Tensor > & grads,
    OptState & state,
    const std::unordered_map< std::string, Tensor > & params
) =0

Parameters:

• grads Gradients keyed by parameter name.
• state Optimizer state, updated in place.
• params Current parameters keyed by name.

Return: Update tensors keyed by parameter name.

Reimplemented by: tenzr::optim::Adam::Update, tenzr::optim::Lion::Update, tenzr::optim::SGD::Update

Compute parameter updates for one step.