Autoencoders

An autoencoder is a special type of feedforward neural network where the input is as same the output. It compresses the input into a lower dimension and then reconstructs the output from this representation. It is an unsupervised algorithm that applies backpropagation to set the target value to be equal to the input.

Autoencoder has 3 layers

Encodes the input into a hidden representation (h).

Here,

we do a linear transformation followed by a nonlinear transformation.

Hidden layer captures everything that is required to reconstruct the original input.

Decodes the input again from the hidden representation(h).

Model will be trained to minimize a certain loss function which ensures that output is close to input.

We had two activation functions (f and g) in our architecture, right? So what will be the choice of activation function?

Choice of activation function depends on the input data.

2. If the input data is real number i.e., ∈ R, then

The objective of autoencoders is to reconstruct output to be as close as input data.

Choice of the loss function in autoencoders also depends on the input data.

References

Related posts:

Regression analysis is a set of statistical operations to estimate the relationships between an independent variable (X) and a dependent variables (Y). The most Simple form of regression analysis is…