ios – Security of MTLTexture reuse and interactions with AVAssetWriter

I’ve a number of doubts about Metallic rendering close to each correctness and effectivity. The next is an outline of the processing pipeline in query:

Author aspect, pushed by AVCaptureOutput

1. A CVPixelBuffer is acquired from captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection)
2. A MTLTexture textureA backed by this pixel buffer is created utilizing CVMetalTextureCacheCreateTextureFromImage
3. A command buffer is created from command queue #1. Quite a few “filter” shaders are encoded into this command buffer, every of which reads from textureA, performs some calculations, and writes again to textureA in collection. Certainly one of these filters is reliant on an auxiliary MTLTexture to retailer intermediate outcomes, which is created as soon as at startup and reused.
4. This command buffer can also be used to write down the contents of textureA into one other MTLTexture, textureB. This texture isn’t backed by a pixel buffer, so it may be stored round so long as obligatory with out holding onto the offered pixel buffer (to keep away from inflicting AVCaptureOutput to drop frames). The reference to this texture is saved in a shared location, and a marker is about to suggest that new knowledge is obtainable to be accessed by the “reader” (described beneath)
5. The unique CVPixelBuffer (modified by means of textureA) is handed to AVAssetWriterInputPixelBufferAdaptor.append() for writing to a video file

Reader aspect, pushed by MTKViewDelegate.draw(in:)

1. Examine if an up to date textureB is obtainable, and in that case use it within the following steps. Additionally mark that this knowledge has been dealt with so we do not render it once more within the subsequent draw invocation if no new knowledge has been offered by the author within the interim. In any other case do nothing
2. Utilizing a command buffer created from a distinct command queue #2, encode a number of extra filters that modify this texture
3. Utilizing the identical command buffer, current the outcomes to the view’s drawable if accessible

This construction was chosen to account for the truth that the MTKView‘s body price could also be both slower or quicker than the speed at which pixel buffers are dealt with and delivered by the author. It additionally seeks to maintain as a lot of the “unimportant” reader steps away from the far more vital video processing steps carried out by the author.

My questions are as follows:

1. Ought to textureB be created anew with every repetition of the method? That definitely works, however it appears wasteful. Possibly it needs to be triple-buffered? On the opposite excessive finish, what could be the implications of all the time utilizing the identical texture for textureB? I can solely assume that might be dangerous, however would it not simply result in pointless bottlenecks through which the author’s GPU instructions can’t proceed whereas the reader is accessing the shared texture, or would there be points with correctness? This additionally results in the following query:
2. Will Metallic’s built-in hazard monitoring nonetheless be sure that studying and modifying textureB from the reader (command queue #2) solely occurs when modification by the author (command queue #1) isn’t in progress, or do I want to make use of one thing like addCompletedHandler to make sure this? Conceptually just like this unanswered query
3. If not, is there a greater strategy to attain the same consequence? Am I higher off simply utilizing one command queue for each the reader and author ends?
4. The only reused auxiliary texture in step (3) of the author aspect should imply that every new body should await the earlier body’s instructions to complete, appropriate? All the pieces at present occurs quick sufficient that this should not matter, however generally, is it good observe to triple-buffer these helper textures?
5. Are there some other basic misunderstandings or points evident on this setup?

Bonus query: is AVAssetWriter (or AVAssetWriterInputPixelBufferAdaptor) doing one thing internally to attend for the GPU to complete modifying any textures backed by the pixel buffer I am feeding it? It appears as if it is likely to be, until I’m lacking some aspect impact of one other a part of my render pipeline. Even when I forego a waitUntilCompleted or addCompletedHandler earlier than step (5) within the author, I do not get unmodified (or partially modified) pixel buffers written to the output video file as one may anticipate. To confirm, I added a vastly costly step to one of many shaders that modifies every incoming pixel buffer, and arrange log entries to document when the pixel buffer is appended and when the command buffer modifying that pixel buffer really completes. The log seems to be one thing like this, exhibiting that obtained pixel buffers are appended to the AVAssetWriter properly earlier than their respective GPU instructions end, but the output file has all of the rendering results.

Appending <CVPixelBuffer 0x282b6acb0> // <--- Occurs virtually instantly after pixel buffer is obtained
Completed rendering <CVPixelBuffer [some other address]>
Completed rendering <CVPixelBuffer [some other address]>
Appending <CVPixelBuffer [some other address]>
Completed rendering <CVPixelBuffer [some other address]>
Appending <CVPixelBuffer [some other address]>
Completed rendering <CVPixelBuffer [some other address]>
Completed rendering <CVPixelBuffer 0x282b6acb0> // <--- Occurs virtually a half second later, as soon as the extraordinarily gradual shader finishes executing, but the output file incorporates the outcomes of this rendering, not the unmodified pixel buffer!

Thanks prematurely to anybody who reads and helps with any a part of this. Joyful to supply clarification on any factors.