Our WOPR inference server allows teams to deploy any AI model from multiple deep learning and machine learning frameworks, such as TensorRT, PyTorch, ONNX, and many others. Our inference server supports inference in the cloud, data centers, and embedded NVIDIA GPU systems. It offers optimized performance for many query types, including real-time, batch, ensemble, and video streaming. The figure below illustrates the general architecture of the WOPR inference server. The model repository is a file system containing the models that WOPR makes available for inference. Inference requests reach the server via HTTP/REST, gRPC, or the C API and are then routed to the appropriate scheduler for each model. WOPR implements several scheduling and batch processing algorithms, configurable for each model.

The scheduler for each model can optionally group inference queries into batches and then forward them to the backend corresponding to the model type. The backend performs inference on the input data provided in the grouped queries to produce the expected results. These results are then returned.

Models served by WOPR can be queried and controlled via a dedicated model management API, available via HTTP/REST or GRPC, or via the C API.

Availability and operational metrics, as well as usage, throughput, and latency metrics, facilitate WOPR integration into deployment frameworks such as Kubernetes.

Network Video Recorder / IP is a real-time media server. It's a versatile tool that allows you to publish, play, record, and redistribute video streams over a network. It's often described as a "media router" that automatically converts streams between different protocols. Its main strength is its support for numerous protocols, which facilitates integration with a wide variety of hardware and software.

It notably supports:

- RTSP (port 8554): for IP cameras, VLC, etc.
- RTMP (port 1935): for OBS Studio, Facebook Live, etc.
- HLS (port 8888): for streaming via a web browser.
- WebRTC (ports 8889/8189): for low latency in the browser.
- SRT (port 8890): for reliable streaming over unstable networks.
- MPEG-TS: for UDP streams.

Detection C++ (Edge) is a complete AI pipeline for real-time video analysis, from video stream acquisition to intelligent information extraction.

From data acquisition (files, RTSP, cameras) to decoding, image preprocessing, neural network inference via TensorRT, and finally object tracking, this pipeline generates metadata (such as bounding boxes or unique identifiers) that can be visualized, saved, or sent to cloud systems via protocols like Kafka. Detection C++ (Edge) is optimized to run on a wide range of hardware, from NVIDIA Jetson embedded devices to data center GPUs, and allows for application development in C/C++ or Python.

VisionLab-AI Detection NetCore offers a rich and powerful ecosystem for building computer vision pipelines which are automated processing chains where visual data (images or video streams) are captured, transformed and analyzed successively to extract information.

Our Network Video Recorder/IP media server manages the acquisition of video streams, for example, from IP cameras via RTSP, and converts them into an accessible format. VisionLab-IA Detection NetCore can then, using dedicated libraries, perform object detection, which involves locating and classifying specific elements in an image, leveraging accelerators like ONNX Runtime for fast inferences, including real-time inferences on video streams.

In parallel, high-performance image processing libraries are used in the pipeline for the preprocessing steps (resizing, cropping, normalization) that are crucial for optimizing the accuracy of the detection models, while efficiently managing system resources.