MPSKernel(3)

NAME
MPSKernel -

SYNOPSIS

#import <MPSKernel.h>

Inherits NSObject, and <NSCopying>.

Inherited by MPSBinaryImageKernel, MPSCNNKernel, MPSImageHistogram, MPSMatrixMultiplication, and MPSUnaryImageKernel.

  Instance Methods
(nonnull instancetype) - initWithDevice:
(nonnull instancetype) - copyWithZone:device:
() - MPSCopyAllocator

  Public Attributes
const MTLRegion MPSRectNoClip

  Properties
MPSKernelOptions options
id< MTLDevice > device
NSString * label

Detailed Description

This depends on Metal.framework The MPSKernel class is the base class for all MPS objects. It defines a standard interface for MPS kernels. You should not use the MPSKernel class directly. Instead, a number of MPSKernel subclasses are available in MetalPerformanceShaders.framework that define specific high-performance image processing operations.

The basic sequence for applying a MPSKernel to an image is as follows:

1.  Create a MPSKernel corresponding to the operation you wish to perform:

MPSImageSobel *sobel = [[MPSImageSobel alloc] initWithDevice: mtlDevice];

2.  Encode the filter into a command buffer:

sobel.offset = ...;
sobel.clipRect = ...;
sobel.options = ...;
[sobel encodeToCommandBuffer: commandBuffer
               sourceTexture: inputImage
          destinationTexture: resultImage ];
if( returnVal < 0 )
    MyShowError( returnVal );

 Encoding the kernel merely encodes the operation into a MTLCommandBuffer. It does not modify any pixels, yet. All MPSKernel state has been copied to the command buffer. MPSKernels may be reused. If the texture was previously operated on by another command encoder (e.g. MTLRenderCommandEncoder), you should call -endEncoding on the other encoder before encoding the filter.

Some MPS filters work in place (inputImage = resultImage) even in situations where Metal might not normally allow in place operation on textures. If in-place operation is desired, you may attempt to call [MPSKernel encodeKernelInPlace...]. If the operation can not be completed in place, then NO will be returned and you will have to create a new result texture and try again. To make an in-place image filter reliable, pass a fallback MPSCopyAllocator to the method to create a new texture to write to in the event that a filter can not operate in place.

(Repeat steps 2 for more filters, as desired.)

It should be self evident that step 2 may not be thread safe. That is, you can not have
multiple threads manipulating the same properties on the same MPSKernel object at the
same time and achieve coherent output. In common usage, the MPSKernel properties don't
often need to be changed from their default values, but if you need to apply the same
filter to multiple images on multiple threads with cropping / tiling, make additional
MPSKernel objects per thread. They are cheap. You can use multiple MPSKernel objects on
multiple threads, as long as only one thread is operating on any particular MPSKernel
object at a time.

3.  After encoding any additional work to the command buffer using other encoders, submit the MTLCommandBuffer to your MTLCommandQueue, using:

[mtlCommandBuffer commit];

Method Documentation

  - (nonnull instancetype) copyWithZone: (nullable NSZone *) zone(nullable id< MTLDevice >) device
Make a copy of this MPSKernel for a new device -copyWithZone: will call this API to make a copy of the MPSKernel on the same device. This interface may also be called directly to make a copy of the MPSKernel on a new device. Typically, the same MPSKernels should not be used to encode kernels on multiple command buffers from multiple threads. Many MPSKernels have mutable properties that might be changed by the other thread while this one is trying to encode. If you need to use a MPSKernel from multiple threads make a copy of it for each additional thread using -copyWithZone: or -copyWithZone:device:

Parameters:
    zone The NSZone in which to allocate the object
    device The device for the new MPSKernel. If nil, then use self.device.

Returns:
    a pointer to a copy of this MPSKernel. This will fail, returning nil if the device is not supported. Devices must be MTLFeatureSet_iOS_GPUFamily2_v1 or later.

  - (nonnull instancetype) initWithDevice: (nonnull id< MTLDevice >) NS_DESIGNATED_INITIALIZER
Standard init with default properties per filter type

Parameters:
    device The device that the filter will be used on. May not be NULL.

Returns:
    a pointer to the newly initialized object. This will fail, returning nil if the device is not supported. Devices must be MTLFeatureSet_iOS_GPUFamily2_v1 or later.

Reimplemented in MPSCNNCrossChannelNormalization, MPSCNNLocalContrastNormalization, MPSCNNSpatialNormalization, MPSCNNPooling, MPSCNNConvolution, MPSCNNFullyConnected, MPSImagePyramid, MPSCNNNeuronTanH, MPSCNNNeuronAbsolute, MPSImageSobel, MPSCNNNeuronReLU, MPSCNNNeuronSigmoid, MPSCNNNeuronLinear, MPSImageThresholdToZeroInverse, MPSImageBox, MPSImageGaussianBlur, MPSImageThresholdToZero, MPSImageThresholdTruncate, MPSImageDilate, MPSImageThresholdBinary, MPSMatrixMultiplication, MPSImageThresholdBinaryInverse, MPSImageMedian, and MPSImageAreaMax.

  - MPSCopyAllocator
A block to make a copy of sourceTexture for MPSKernels that can only execute out of place. Some MPSKernel objects may not be able to operate in place. When that occurs, and in-place operation is requested, MPS will call back to this block to get a new texture to return instead. To avoid spending long periods of time allocating pages to back the MTLTexture, the block should attempt to reuse textures. The texture returned from the MPSCopyAllocator will be returned instead of the sourceTexture from the MPSKernel method on return.

// A MPSCopyAllocator to handle cases where in-place operation fails.
MPSCopyAllocator myAllocator = ^id <MTLTexture>( MPSKernel * __nonnull filter,
                                                __nonnull id <MTLCommandBuffer> cmdBuf,
                                                __nonnull id <MTLTexture> sourceTexture)
{
    MTLPixelFormat format = sourceTexture.pixelFormat;  // FIXME: is this format writable?
    MTLTextureDescriptor *d = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat: format
                                 width: sourceTexture.width
                                height: sourceTexture.height
                             mipmapped: NO];
    d.usage = MTLTextureUsageShaderRead | MTLTextureUsageShaderWrite;
    //FIXME: Allocating a new texture each time is slow. They take up to 1 ms each.
    //       There are not too many milliseconds in a video frame! You can recycle
    //       old textures (or MTLBuffers and make textures from them) and reuse
    //       the memory here.
    id <MTLTexture> result = [cmdBuf.device newTextureWithDescriptor: d];
    // FIXME: If there is any metadata associated with sourceTexture such as colorspace
    //        information, MTLResource.label, MTLResource.cpuCacheMode mode,
    //        MTLResource.MTLPurgeableState, etc., it may need to be similarly associated
    //        with the new texture to avoid losing your metadata.
    // FIXME: If filter.clipRect doesn't cover the entire image, you may need to copy
    //        pixels from sourceTexture to the new texture or regions of the new texture
    //        will be uninitialized. You can make a MTLCommandEncoder to encode work on
    //        the MTLCommandBuffer here to do that work, if necessary. It will be
    //        scheduled to run immediately before the MPSKernel work. Do not call
    //        [MTLCommandBuffer enqueue/commit/waitUntilCompleted/waitUntilScheduled]
    //        in the MPSCopyAllocator block. Make sure to call -endEncoding on the
    //        MTLCommandEncoder so that the MTLCommandBuffer has no active encoder
    //        before returning.
    // CAUTION: The next command placed on the MTLCommandBuffer after the MPSCopyAllocator
    //          returns is almost assuredly going to be encoded with a MTLComputeCommandEncoder.
    //          Creating any other type of encoder in the MPSCopyAllocator will probably cost
    //          an additional 0.5 ms of both CPU _AND_ GPU time (or more!) due to a double
    //          mode switch penalty.
    // CAUTION: If other objects (in addition to the caller of -encodeToCommandBuffer:inPlaceTexture:...)
    //          own a reference to sourceTexture, they may need to be notified that
    //          sourceTexture has been replaced so that they can release that resource
    //          and adopt the new texture.
    //          The reference to sourceTexture owned by the caller of
    //          -encodeToCommandBuffer:inPlaceTexture... will be released by
    //          -encodeToCommandBuffer:inPlaceTexture:... after the kernel is encoded if
    //          and only if the MPSCopyAllocator is called, and the operation is successfully
    //          encoded out of place.
    return result;
    // d is autoreleased
};

 If nil is returned by the allocator, NO will be returned by the calling function.

When the MPSCopyAllocator is called, no MTLCommandEncoder is active on the commandBuffer. You may create a MTLCommandEncoder in the block to initialize the texture. Make sure to call -endEncoding on it before returning, if you do.

Parameters:
    filter A valid pointer to the MPSKernel that is calling the MPSCopyAllocator. From it you can get the clipRect of the intended operation.
    commandBuffer A valid MTLCommandBuffer. It can be used to obtain the device against which to allocate the new texture. You may also enqueue operations on the commandBuffer to initialize the texture on a encoder allocated in the block. You may not submit, enqueue or wait for scheduling/completion of the command buffer.
    sourceTexture The texture that is providing the source image for the filter. You may wish to use its size and MTLPixelFormat for the new texture, but it is not requred.

Returns:
    A new valid MTLTexture to use as the destination for the MPSKernel. If the calling function succeeds, its texture parameter will be overwritten with a pointer to this texture. If the calling function fails (highly unlikely, except for user error) then the texture will be released before the calling function returns.

Member Data Documentation

  - (const MTLRegion) MPSRectNoClip
MPSRectNoClip This is a special constant to indicate no clipping is to be done. The entire image will be used. This is the default clipping rectangle or the input extent for MPSKernels.

Property Documentation

  - device [read], [nonatomic], [retain]
The device on which the kernel will be used

  - label [read], [write], [atomic], [copy]
A string to help identify this object.

  - options [read], [write], [nonatomic], [assign]
The set of options used to run the kernel. MPSKernelOptions

Author

Generated automatically by Doxygen for MetalPerformanceShaders.framework from the source code.