Linear

Quelle: nvidia.com

A linear layer, often referred to as a fully connected layer, dense layer or inner product layer, is a fundamental component in machine learning and deep learning models. It is utilized to transform the input data using learned weights and biases.

In a linear layer, each input node (or neuron) is connected to each output node. The layer applies a linear transformation to the incoming data: it multiplies the input by a weight matrix (), and adds a bias vector (). This operation can be mathematically represented as , where is the input data, is the output data, represents weights and represents bias.

The weights and biases in the linear layer are learnable parameters. During training of the model, these parameters are updated using methods such as gradient descent to minimize the error between predicted output and actual output.

Linear layers are often followed by non-linear activation functions like ReLU or Sigmoid to introduce non-linearity into the model. This allows deep learning models to learn complex patterns from data.