MPSNNOptimizerStochasticGradientDescent(3)

NAME
MPSNNOptimizerStochasticGradientDescent

SYNOPSIS

#import <MPSNNOptimizers.h>

Inherits MPSNNOptimizer.

  Instance Methods
(nonnull instancetype) - initWithDevice:
(nonnull instancetype) - initWithDevice:learningRate:
(nonnull instancetype) - initWithDevice:momentumScale:useNestrovMomentum:optimizerDescriptor:
(void) - encodeToCommandBuffer:inputGradientVector:inputValuesVector:inputMomentumVector:resultValuesVector:
(void) - encodeToCommandBuffer:convolutionGradientState:convolutionSourceState:inputMomentumVectors:resultState:
(void) - encodeToCommandBuffer:batchNormalizationState:inputMomentumVectors:resultState:
(void) - encodeToCommandBuffer:batchNormalizationGradientState:batchNormalizationSourceState:inputMomentumVectors:resultState:

  Properties
float momentumScale
BOOL useNestrovMomentum

  Additional Inherited Members

Detailed Description

The MPSNNOptimizerStochasticGradientDescent performs a gradient descent with an optional momentum Update RMSProp is also known as root mean square propagation.

useNestrov == NO: m[t] = momentumScale * m[t-1] + learningRate * g variable = variable - m[t]

useNestrov == YES: m[t] = momentumScale * m[t-1] + g variable = variable - (learningRate * (g + m[t] * momentumScale))

        where,
          g    is gradient of error wrt variable
          m[t] is momentum of gradients it is a state we keep updating every update iteration.fi

Method Documentation

  - (void) encodeToCommandBuffer: (__nonnull id< MTLCommandBuffer >) commandBuffer(MPSCNNBatchNormalizationState *__nonnull) batchNormalizationGradientState(MPSCNNBatchNormalizationState *__nonnull) batchNormalizationSourceState(nullable NSArray< MPSVector * > *) inputMomentumVectors(nonnull MPSCNNNormalizationGammaAndBetaState *) resultState
Encode an MPSNNOptimizerStochasticGradientDescent object to a command buffer to perform out of place update

Parameters:
    commandBuffer A valid MTLCommandBuffer to receive the encoded kernel.
    batchNormalizationGradientState A valid MPSCNNBatchNormalizationState object which specifies the input state with gradients for this update.
    batchNormalizationSourceState A valid MPSCNNBatchNormalizationState object which specifies the input state with original gamma/beta for this update.
    inputMomentumVectors An array MPSVector object which specifies the gradient momentum vectors which will be updated and overwritten. The index 0 corresponds to gamma, index 1 corresponds to beta, array can be of size 1 in which case beta won't be updated
    resultState A valid MPSCNNNormalizationGammaAndBetaState object which specifies the resultValues state which will be updated and overwritten.

The following operations would be applied

        useNestrov == NO:
            m[t]     = momentumScale * m[t-1] + learningRate * g
            variable = variable - m[t]
        useNestrov == YES:
            m[t]     = momentumScale * m[t-1] + g
            variable = variable - (learningRate * (g + m[t] * momentumScale))
        inputMomentumVector == nil
            variable = variable - (learningRate * g)
        where,
          g    is gradient of error wrt variable
          m[t] is momentum of gradients it is a state we keep updating every update iteration.fi

  - (void) encodeToCommandBuffer: (__nonnull id< MTLCommandBuffer >) commandBuffer(MPSCNNBatchNormalizationState *__nonnull) batchNormalizationState(nullable NSArray< MPSVector * > *) inputMomentumVectors(nonnull MPSCNNNormalizationGammaAndBetaState *) resultState
Encode an MPSNNOptimizerStochasticGradientDescent object to a command buffer to perform out of place update

Parameters:
    commandBuffer A valid MTLCommandBuffer to receive the encoded kernel.
    batchNormalizationState A valid MPSCNNBatchNormalizationState object which specifies the input state with gradients and original gamma/beta for this update.
    inputMomentumVectors An array MPSVector object which specifies the gradient momentum vectors which will be updated and overwritten. The index 0 corresponds to gamma, index 1 corresponds to beta, array can be of size 1 in which case beta won't be updated
    resultState A valid MPSCNNNormalizationGammaAndBetaState object which specifies the resultValues state which will be updated and overwritten.

The following operations would be applied

        useNestrov == NO:
            m[t]     = momentumScale * m[t-1] + learningRate * g
            variable = variable - m[t]
        useNestrov == YES:
            m[t]     = momentumScale * m[t-1] + g
            variable = variable - (learningRate * (g + m[t] * momentumScale))
        inputMomentumVector == nil
            variable = variable - (learningRate * g)
        where,
          g    is gradient of error wrt variable
          m[t] is momentum of gradients it is a state we keep updating every update iteration.fi

  - (void) encodeToCommandBuffer: (__nonnull id< MTLCommandBuffer >) commandBuffer(MPSCNNConvolutionGradientState *__nonnull) convolutionGradientState(MPSCNNConvolutionWeightsAndBiasesState *__nonnull) convolutionSourceState(nullable NSArray< MPSVector * > *) inputMomentumVectors(nonnull MPSCNNConvolutionWeightsAndBiasesState *) resultState
Encode an MPSNNOptimizerStochasticGradientDescent object to a command buffer to perform out of place update

Parameters:
    commandBuffer A valid MTLCommandBuffer to receive the encoded kernel.
    convolutionGradientState A valid MPSCNNConvolutionGradientState object which specifies the input state with gradients for this update.
    convolutionSourceState A valid MPSCNNConvolutionWeightsAndBiasesState object which specifies the input state with values to be updated.
    inputMomentumVectors An array MPSVector object which specifies the gradient momentum vectors which will be updated and overwritten. The index 0 corresponds to weights, index 1 corresponds to biases, array can be of size 1 in which case biases won't be updated
    resultState A valid MPSCNNConvolutionWeightsAndBiasesState object which specifies the resultValues state which will be updated and overwritten.

The following operations would be applied

        useNestrov == NO:
            m[t]     = momentumScale * m[t-1] + learningRate * g
            variable = variable - m[t]
        useNestrov == YES:
            m[t]     = momentumScale * m[t-1] + g
            variable = variable - (learningRate * (g + m[t] * momentumScale))
        inputMomentumVector == nil
            variable = variable - (learningRate * g)
        where,
          g    is gradient of error wrt variable
          m[t] is momentum of gradients it is a state we keep updating every update iteration.fi

  - (void) encodeToCommandBuffer: (nonnull id< MTLCommandBuffer >) commandBuffer(nonnull MPSVector *) inputGradientVector(nonnull MPSVector *) inputValuesVector(nullable MPSVector *) inputMomentumVector(nonnull MPSVector *) resultValuesVector
Encode an MPSNNOptimizerStochasticGradientDescent object to a command buffer to perform out of place update

Parameters:
    commandBuffer A valid MTLCommandBuffer to receive the encoded kernel.
    inputGradientVector A valid MPSVector object which specifies the input vector of gradients for this update.
    inputValuesVector A valid MPSVector object which specifies the input vector of values to be updated.
    inputMomentumVector A valid MPSVector object which specifies the gradient momentum vector which will be updated and overwritten.
    resultValuesVector A valid MPSVector object which specifies the resultValues vector which will be updated and overwritten.

The following operations would be applied

        useNestrov == NO:
            m[t]     = momentumScale * m[t-1] + learningRate * g
            variable = variable - m[t]
        useNestrov == YES:
            m[t]     = momentumScale * m[t-1] + g
            variable = variable - (learningRate * (g + m[t] * momentumScale))
        inputMomentumVector == nil
            variable = variable - (learningRate * g)
        where,
          g    is gradient of error wrt variable
          m[t] is momentum of gradients it is a state we keep updating every update iteration.fi

  - (nonnull instancetype) initWithDevice: (nonnull id< MTLDevice >) device
Standard init with default properties per filter type

Parameters:
    device The device that the filter will be used on. May not be NULL.

Returns:
    a pointer to the newly initialized object. This will fail, returning nil if the device is not supported. Devices must be MTLFeatureSet_iOS_GPUFamily2_v1 or later.

Reimplemented from MPSNNOptimizer.

  - (nonnull instancetype) initWithDevice: (nonnull id< MTLDevice >) device(float) learningRate
Convenience initialization for the momentum update

Parameters:
    device The device on which the kernel will execute.
    learningRate The learningRate which will be applied

Returns:
    A valid MPSNNOptimizerStochasticGradientDescent object or nil, if failure.

  - (nonnull instancetype) initWithDevice: (nonnull id< MTLDevice >) device(float) momentumScale(BOOL) useNestrovMomentum(nonnull MPSNNOptimizerDescriptor *) optimizerDescriptor
Full initialization for the momentum update

Parameters:
    device The device on which the kernel will execute.
    momentumScale The momentumScale to update momentum for values array
    useNestrovMomentum Use the Nestrov style momentum update
    optimizerDescriptor The optimizerDescriptor which will have a bunch of properties to be applied

Returns:
    A valid MPSNNOptimizerMomentum object or nil, if failure.

Property Documentation

  - momentumScale [read], [nonatomic], [assign]
The momentumScale at which we update momentum for values array Default value is 0.0

  - useNestrovMomentum [read], [nonatomic], [assign]
Nestrov momentum is considered an improvement on the usual momentum update Default value is NO

Author

Generated automatically by Doxygen for MetalPerformanceShaders.framework from the source code.