Optimizations

Baseline is Out-of-the-Box (OOTB). Cached Models - model server loads ML model once on startup per thread. Inference uses cached model GPU - runs cached models on A100 GPU in AWS

tl;dr

Yella's cloned detectors process audio, pics in real-time (15 ms for pics, 81 ms for audio). They consume < 1GB RAM. And scale linearly as the batch size increases.

Optimizations

Baseline is Out-of-the-Box (OOTB). Cached Models - model server loads ML model once on startup per thread. Inference uses cached model GPU - runs cached models on A100 GPU in AWS

tl;dr

Yella's cloned detectors process audio, pics in real-time (15 ms for pics, 81 ms for audio). They consume < 1GB RAM. And scale linearly as the batch size increases.

Optimizations

Baseline is Out-of-the-Box (OOTB). Cached Models - model server loads ML model once on startup per thread. Inference uses cached model GPU - runs cached models on A100 GPU in AWS

tl;dr

Yella's cloned detectors process audio, pics in real-time (15 ms for pics, 81 ms for audio). They consume < 1GB RAM. And scale linearly as the batch size increases.