tenzr::optim::ExponentialMovingAverage¶

Maintains an exponential moving average of a parameter set.

#include <ema.h>

Public Functions¶

Public Functions Documentation¶

function ExponentialMovingAverage¶

explicit ExponentialMovingAverage(
    const std::unordered_map< std::string, Tensor > & params,
    double decay =0.999
)

Parameters:

• params Parameters to track.
• decay Decay factor, in [0, 1).

Initialize shadow copies of params for exponential moving averaging.

function Update¶

void Update(
    const std::unordered_map< std::string, Tensor > & params
)

Parameters:

• params Current parameters keyed by name.
• params Current parameter values.

Fold the current parameters into the shadow average. Fold the current parameter values into the shadow copies.

function ApplyShadow¶

void ApplyShadow(
    std::unordered_map< std::string, Tensor > & params
)

Parameters:

• params Parameters overwritten in place.
• params Parameters to overwrite in place.

Replace the live parameters with the shadow average. Copy the shadow values onto params, backing up the originals for restore.

function Restore¶

void Restore(
    std::unordered_map< std::string, Tensor > & params
)

Parameters:

• params Parameters restored in place.
• params Parameters to restore in place.

Restore the parameters saved by the last apply_shadow. Restore the parameter values saved by the last apply_shadow.

function ShadowParams¶

inline const std::unordered_map< std::string, Tensor > & ShadowParams() const

Return: Const reference to the shadow parameters.

function Decay¶

inline double Decay() const

Return: EMA decay factor.