CLS Token for FT-Transformer

Concatenates a CLS token to the input as the last feature. The input shape is expected to be (batch, n_features, d_token) and the output shape is (batch, n_features + 1, d_token).

This is used in the LearnerTorchFTTransformer.

nn_module

Calls nn_ft_cls() when trained.

State

The state is the value calculated by the public method $shapes_out().

Super classes

mlr3pipelines::PipeOp -> PipeOpTorch -> PipeOpTorchFTCLS

Methods

Public methods

• PipeOpTorchFTCLS$new()

• PipeOpTorchFTCLS$clone()

Inherited methods

• mlr3pipelines::PipeOp$help()
• mlr3pipelines::PipeOp$predict()
• mlr3pipelines::PipeOp$print()
• mlr3pipelines::PipeOp$train()
• PipeOpTorch$shapes_out()

---

PipeOpTorchFTCLS$new()

Creates a new instance of this R6 class.

Usage

PipeOpTorchFTCLS$new(id = "nn_ft_cls", param_vals = list())

Arguments

id

(character(1))
Identifier of the resulting object.

param_vals

(list())
List of hyperparameter settings, overwriting the hyperparameter settings that would otherwise be set during construction.

---

PipeOpTorchFTCLS$clone()

The objects of this class are cloneable with this method.

Usage

PipeOpTorchFTCLS$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

# Construct the PipeOp
pipeop = nn("ft_cls")
pipeop
#> 
#> ── PipeOp <ft_cls>: not trained ────────────────────────────────────────────────
#> Values: initialization=uniform
#> 
#> ── Input channels: 
#>    name           train predict
#>  <char>          <char>  <char>
#>   input ModelDescriptor    Task
#> 
#> ── Output channels: 
#>    name           train predict
#>  <char>          <char>  <char>
#>  output ModelDescriptor    Task
# The available parameters
pipeop$param_set
#> <ParamSet(1)>
#>                id    class lower upper nlevels        default   value
#>            <char>   <char> <num> <num>   <num>         <list>  <list>
#> 1: initialization ParamFct    NA    NA       2 <NoDefault[0]> uniform