Simple Recurrent Layer — mlr_pipeops_nn

Applies a multi-layer Elman RNN with $\tanh$ or $\mbox{ReLU}$ non-linearity to an input sequence.

Tensor Layout

Input and output are (batch, sequence, feature), i.e. the batch_first layout of torch::nn_rnn(), which is fixed and not a hyperparameter. torch defaults to (sequence, batch, feature), but the first dimension of every shape has to be the batch dimension here.

The final hidden state is (layers * directions, batch, hidden_size) in torch, even in the batch-first layout. It is transposed to (batch, layers * directions, hidden_size) here, so that the batch dimension comes first for it as well.

nn_module

Calls torch::nn_rnn() when trained, where the parameter input_size is inferred as the last dimension of the input tensor and batch_first is always TRUE, see section Tensor Layout.

Parameters

hidden_size :: integer(1)
The number of features of the hidden state.
num_layers :: integer(1)
The number of stacked recurrent layers. Default is 1.
bias :: logical(1)
Whether to use bias weights. Default is TRUE.
dropout :: numeric(1)
Dropout probability on the output of each layer but the last. Only has an effect when num_layers is greater than 1. Default is 0.
bidirectional :: logical(1)
Whether to run the sequence in both directions and concatenate the results, which doubles the size of the feature dimension of the output. Default is FALSE.
nonlinearity :: character(1)
The activation function, either "tanh" or "relu". Default is "tanh".

Note that input_size is not a parameter, as it is inferred from the shape of the input tensor, and that batch_first is not a parameter either, as it is fixed to TRUE, see section Tensor Layout.

Input and Output Channels

There is one input channel "input", the sequence to run over.

The module has two outputs: "output", the output sequence of shape (batch, sequence, hidden_size * directions), and "h_n", the final hidden state of shape (batch, num_layers * directions, hidden_size). Only "output" is an output channel by default. Set $outputs to also (or only) get the final state, e.g. outputs = c("output", "h_n").

For an explanation see PipeOpTorch.

State

The state is the value calculated by the public method $shapes_out().

Super classes

mlr3pipelines::PipeOp -> PipeOpTorch -> PipeOpTorchRecurrent -> PipeOpTorchRNN

Methods

Inherited methods

`PipeOpTorchRNN$new()`

Creates a new instance of this R6 class.

Usage

PipeOpTorchRNN$new(id = "nn_rnn", param_vals = list())

Arguments

id: (character(1))
Identifier of the resulting object.
param_vals: (list())
List of hyperparameter settings, overwriting the hyperparameter settings that would otherwise be set during construction.

`PipeOpTorchRNN$clone()`

The objects of this class are cloneable with this method.

Usage

PipeOpTorchRNN$clone(deep = FALSE)

Arguments

deep: Whether to make a deep clone.

Examples

# Construct the PipeOp
pipeop = nn("rnn", hidden_size = 10)
pipeop
#> 
#> ── PipeOp <rnn>: not trained ───────────────────────────────────────────────────
#> Values: hidden_size=10
#> 
#> ── Input channels: 
#>    name           train predict
#>  <char>          <char>  <char>
#>   input ModelDescriptor    Task
#> 
#> ── Output channels: 
#>    name           train predict
#>  <char>          <char>  <char>
#>  output ModelDescriptor    Task
# The available parameters
pipeop$param_set
#> <ParamSet(6)>
#>               id    class lower upper nlevels        default  value
#>           <char>   <char> <num> <num>   <num>         <list> <list>
#> 1:   hidden_size ParamInt     1   Inf     Inf <NoDefault[0]>     10
#> 2:    num_layers ParamInt     1   Inf     Inf              1 [NULL]
#> 3:          bias ParamLgl    NA    NA       2           TRUE [NULL]
#> 4:       dropout ParamDbl     0     1     Inf              0 [NULL]
#> 5: bidirectional ParamLgl    NA    NA       2          FALSE [NULL]
#> 6:  nonlinearity ParamFct    NA    NA       2           tanh [NULL]