MediaPipe allows graphs to run OpenGL in multiple GL contexts. OpenGL ES 3.1 or greater is required (on Android/Linux systems) for running machine learning inference calculators and graphs. In addition, MediaPipe also supports Metal on iOS. MediaPipe supports OpenGL ES up to version 3.2 on Android/Linux and up to ES 3.0 on iOS.

A calculator should be allowed maximum flexibility in using the GPU for all or part of its operation, combining it with the CPU if necessary.

Because different platforms may require different techniques for best performance, the API should allow flexibility in the way things are implemented behind the scenes.

Transfer of frame data between CPU and GPU should be as efficient as the platform allows.

Transfer of frame data from one GPU-based calculator to another should be fast, and not incur expensive copy operations.

GPU-based calculators should be able to occur anywhere in the graph, and not necessarily be used for on-screen rendering.

Video rendering and effects, not just analysisīelow are the design principles for GPU support in MediaPipe.

On-device real-time processing, not just batch processing.

MediaPipe enables developers to write GPU compatible calculators that support the use of GPU for: GPU support is essential for good performance on mobile platforms, especially for real-time video. Individual nodes can be written using different APIs, allowing them to take advantage of platform specific features when needed. MediaPipe does not attempt to offer a single cross-API GPU abstraction. There exist several GPU APIs on mobile platforms (eg, OpenGL ES, Metal and Vulkan). MediaPipe supports calculator nodes for GPU compute and rendering, and allows combining multiple GPU nodes, as well as mixing them with CPU based calculator nodes. This site uses Just the Docs, a documentation theme for Jekyll.

YouTube-8M Feature Extraction and Model Inference.

AutoFlip (Saliency-aware Video Cropping).

KNIFT (Template-based Feature Matching).

Tell if gpu supports opengl 4.3