VisionLab-IA-Drone
How can a drone reliably identify a person in mid-flight, in the middle of a crowd
or over a wide area? Thanks to a unique combination of computer vision, biometric models,
and optimized flight paths, our solution first detects human presence, isolates the face,
and then performs a reliable identification in seconds.
or over a wide area? Thanks to a unique combination of computer vision, biometric models,
and optimized flight paths, our solution first detects human presence, isolates the face,
and then performs a reliable identification in seconds.
WYATT (Indore) Autonomous UAV VSLAM
An autonomous drone using VSLAM (Visual Simultaneous Localization and Mapping) is a system that creates a
map of its environment in real time while simultaneously locating itself within it, solely using its cameras,
most often in the absence of GPS.
This technology is particularly valuable in areas where the satellite signal is weak or nonexistent, such as inside buildings, tunnels, or dense urban environments.
In the civilian sector, this technology is used for infrastructure inspections (bridges, dams, industrial facilities), 3D mapping of construction sites or historical monuments, and for rescue missions in confined spaces (building collapses, mines, or caves).
In the military sector, VSLAM equips reconnaissance drones in urban or underground environments for missions. Surveillance, intelligence gathering, or assessment of hostile areas without revealing their position via GPS transmission, while maintaining a discreet link via NAVlink to transmit real-time tactical information.
This technology is particularly valuable in areas where the satellite signal is weak or nonexistent, such as inside buildings, tunnels, or dense urban environments.
In the civilian sector, this technology is used for infrastructure inspections (bridges, dams, industrial facilities), 3D mapping of construction sites or historical monuments, and for rescue missions in confined spaces (building collapses, mines, or caves).
In the military sector, VSLAM equips reconnaissance drones in urban or underground environments for missions. Surveillance, intelligence gathering, or assessment of hostile areas without revealing their position via GPS transmission, while maintaining a discreet link via NAVlink to transmit real-time tactical information.
The workings of the VSLAM system
The operation of a VSLAM drone relies on a real-time image processing loop, generally broken down into three main stages:
- Feature Perception and Extraction: The drone's camera captures the video stream. The imaging system, or front-end, analyzes each image to extract "points of interest," or distinctive visual features (corners, edges, textures), using algorithms such as ORB or SIFT. These points serve as unique reference points that the drone can recognize from one image to the next.
- Location and Trajectory Tracking: By comparing the position of these feature points between two successive images, the system estimates its own movement. This process, called visual odometry, allows the drone to deduce how it has moved in space. To refine this estimation and make it more robust, the camera data is often combined with data from an inertial measurement unit (IMU) that measures the drone's accelerations and rotations. This is called VIO (Visual-Inertial Odometry)
- Mapping and Optimization: The drone selects certain images, called keyframes, to integrate into its map as it is being built. This back-end of the system creates a 3D map (often a point cloud) of the environment, which allows the position of obstacles to be determined. It also incorporates a loop closure mechanism: when the drone recognizes a previously visited location, it can "close the loop" and correct accumulated positioning errors, making the map and location more accurate.
Our system combines:
1. Intel D435i camera and NVIDIA Jetson Orin Nano embedded computer
The Intel D435i camera is a stereoscopic camera that provides both a standard video feed and a real-time depth map. It also features an integrated inertial measurement unit (IMU), enabling simultaneous measurement of the drone's movements and accelerations.
All this data is transmitted to the onboard computer, an NVIDIA Jetson Orin Nano, which acts as the visual "brain." Thanks to its AI-optimized computing power, it executes VSLAM algorithms in real time: point-of-interest detection, image matching, map construction, and trajectory estimation. This onboard architecture allows for on-the-fly processing without relying on a connection to an external server.
2. Pixhawk flight controller
The Pixhawk is an open-source autopilot widely used in the drone world. It continuously receives position and attitude estimates from the VSLAM system and translates them into flight commands: adjusting pitch, engine power, direction, and altitude.
It also manages flight modes (hover, move, return to home) and incorporates safety mechanisms: in case of loss of visual references or inconsistent data, it can switch to a degraded mode or initiate an emergency landing procedure.
3. Extended Kalman Filter (EKF) for visuo-inertial fusion
The Extended Kalman Filter (EKF) is the algorithm that merges information from the camera (relative position with respect to the map) and the inertial measurement unit (IMU) (accelerations and angular velocities). Each data source has its own strengths and weaknesses—the camera is precise but can be affected by sudden movements or a lack of texture, while the IMU is continuous but drifts over time—the EKF intelligently combines the two to obtain a stable, smooth, and reliable position and orientation estimate. This fusion allows the drone to maintain accurate navigation even in the event of a temporary loss of visual reference points, relying on inertia to "fill in the gaps" until the camera finds reference points again.
Essential technical points
- A stereoscopic system (two cameras) or RGB-D (camera with depth sensor) can directly measure depth, providing more precise localization.
- To conserve battery power, the VSLAM's heavy processing can be offloaded to a ground computer (edge computing) that communicates with the drone, leaving only essential control tasks on board.
- VSLAM is coupled with artificial intelligence for advanced tasks, such as obstacle avoidance in real-world conditions.
