tenzr::optim::AdamW¶
Adam variant that applies decoupled weight decay.
#include <adam.h>
Inherits from tenzr::optim::Adam, tenzr::optim::GradientTransformation
Public Functions¶
| Name | |
|---|---|
| AdamW(double lr =1.0e-3, double beta1 =0.9, double beta2 =0.999, double eps =1.0e-8, double weight_decay =1.0e-2, bool amsgrad =false, ScheduleFn schedule =nullptr) |
Additional inherited members¶
Public Functions inherited from tenzr::optim::Adam
| Name | |
|---|---|
| Adam(double lr =1.0e-3, double beta1 =0.9, double beta2 =0.999, double eps =1.0e-8, double weight_decay =0.0, bool amsgrad =false, bool decoupled_weight_decay =false, ScheduleFn schedule =nullptr) | |
| virtual OptState | Init(const std::unordered_map< std::string, Tensor > & params) override |
| virtual std::unordered_map< std::string, Tensor > | Update(const std::unordered_map< std::string, Tensor > & grads, OptState & state, const std::unordered_map< std::string, Tensor > & params) override |
| double | Lr() const |
| double | Beta1() const |
| double | Beta2() const |
| double | Eps() const |
| double | WeightDecay() const |
| bool | Amsgrad() const |
Protected Attributes inherited from tenzr::optim::Adam
| Name | |
|---|---|
| double | lr_ |
| double | beta1_ |
| double | beta2_ |
| double | eps_ |
| double | weight_decay_ |
| bool | amsgrad_ |
| bool | decoupled_weight_decay_ |
| ScheduleFn | schedule_ |
Public Functions inherited from tenzr::optim::GradientTransformation
| Name | |
|---|---|
| virtual | ~GradientTransformation() =default |
| virtual OptState | Init(const std::unordered_map< std::string, Tensor > & params) =0 |
| virtual std::unordered_map< std::string, Tensor > | Update(const std::unordered_map< std::string, Tensor > & grads, OptState & state, const std::unordered_map< std::string, Tensor > & params) =0 |