Class PretrainedMLP

Parameters:

• nUnits (int list) - The sizes of the (input, hidden* , output) representations.
• outputActivation (class derived from Activation) - The type of activation for the output layer.
• nInputLayers (int) - Number of layers starting from input to be stacked with AE
• nOutputLayers (int) - Number of layers starting from output to be stacked with AE
• InputAutoEncoderClass (AutoEncoder sub class) - Class to be used for Input Auto Encoders
• OutputAutoEncoderClass (OutputAutoEncoder sub class) - Class to be used for Output Auto Encoders

Overrides: module.Module.__init__

Overrides: module.Module.prepareParams

(inherited documentation)

Performs the unsupervised learning step of the input autoencoders, using a batch-gradient backpropagation algorithm.

Classically, in a DeepNeuralNetwork, only the input autoencoders are pretrained. The data used for this pretraining step can be the input training dataset used for the supervised learning (see finetune), or a subset of this dataset, or else a specially crafted input pretraining dataset.

Once an AutoEncoder is learned, the projection (encoding) layer is kept and used to initialize the network layers. The backprojection (decoding) part is not useful anymore.

Parameters:

• data (SharedVariable from ndarray) - The training data (typically example features).
• params (dict) - The learning parameters, encoded in a dictionary, that are used in the finetune method.

Performs the unsupervised learning step of the output autoencoders, using a batch-gradient backpropagation algorithm.

The InputOutputDeepArchitecture pretrains the output autoencoders, in the same way the DeepNeuralNetwork does for input autoencoders. In this case, the given training data are the labels (y) and not the examples (x) (i.e. the labels that the network must predict).

Once an AutoEncoder is learned, the backprojection layer (decoding) is kept and used to initialize the network layers. The projection (encoding) part is not useful anymore.

Parameters:

• data (SharedVariable from ndarray) - The training data (typically example labels).
• params (dict) - The learning parameters, encoded in a dictionary, that are used in the finetune method.

Parameters:

• x_train (ndarray) - The training examples.
• y_train (ndarray) - The training labels.

• params (dict) - The learning parameters, encoded in a dictionary, that are used during the autoencoders pretraining (pretrainInputAutoEncoders, pretrainOutputAutoEncoders), the link layer pretraining, and the final learning (finetune) steps.

  Possible keys: batch_size, learning_rate, epochs, growth_factor, growth_threshold, badmove_threshold, verbose, input_pretraining_params, output_pretraining_params, link_pretraining, link_pretraining_params.

  The link_pretraining parameter controls whether the link layer is pretrained or not (default: False).

  The input_pretraining_params, output_pretraining_params and link_pretraining_params parameters are themselves dictionaries containing the training parameters for each pretraining step.

Returns:

elapsed time, in deltatime.

Overrides: MultiLayerPerceptron.train