Mobile drone targeting brings object recognition and motion control together in a compact cyber-physical system. In the approach described here, a smart sensor unit (SSU) processes video, identifies a moving object, estimates its position relative to the camera view and sends commands to guide the drone. The key design choice is to perform recognition and tracking on a mobile device rather than depend on a remote computing system.
Mobile drone targeting as a cyber-physical system task
A cyber-physical system combines physical processes with computing and measurement capabilities. Its smart sensor units interact with the real world: they collect information, process it and can help apply actions to a physical system. For a drone, the SSU’s targeting task is to interpret camera frames and use the observed object position to inform movement.
Remote or cloud-based processing can support tracking applications, but it also makes the system reliant on network communication. The described design instead uses an iOS mobile device to handle the video stream and recognition, while communicating with the drone over Wi-Fi using UDP commands. In the tested configuration, the drone supplies the video and receives movement instructions; the mobile device performs the main computations.
How the targeting algorithm estimates object position
The algorithm uses two measurements from each recognition result: the object’s bounding-box center relative to the center of the image, and the bounding box’s size compared with the image area. Together, these measurements provide information about where the target appears and how much of the frame it occupies.
Position in the camera frame
The difference between the image center and the bounding-box center gives a displacement vector. Its horizontal and vertical components describe how far the recognized object is from the desired center point. The system can also calculate an angle from this displacement, helping determine the direction in which the drone should move or turn.
Rather than react to a single detection, the tracking logic stores observations in a batch. It compares the angle in a new observation with the previous one and retains observations when the difference is below a configured limit. Repeated indications in a similar direction can then inform a movement command. The paper’s example describes a 20-degree comparison limit and a history of up to 30 observations before movement is performed. This type of history-based decision helps make control depend on a sequence of detections, not just one frame.
Bounding-box area as a distance cue
The algorithm calculates the bounding box’s area as a percentage of the full frame. This ratio acts as a distance coefficient: a smaller box occupies less of the observed image, while a larger one occupies more. The method uses that value to choose whether to move forward, backward or remain still.
There is an important implementation detail in the published threshold rules: they specify moving forward below 25%, moving backward above 25%, and standing still between 25% and 50%. Those conditions overlap, so they do not define an unambiguous decision for every value. Anyone implementing the method should resolve the thresholds and their boundaries before using them in control logic; the article’s description does not establish a non-overlapping rule set.
YOLOv3 object detection on a mobile device
The recognition module uses YOLOv3, a convolutional neural network architecture for object detection. The model processes an image and produces candidate bounding boxes and associated confidence and class information. The system applies filters to improve the usefulness of recognition results, including confidence-based minimization and smoothing based on overlap between detections.
In the described model, an input image is processed at multiple grid scales. For a 416-by-416-pixel input, the grid dimensions are 52 by 52, 26 by 26 and 13 by 13. The network’s outputs are processed to estimate box position and size, while a SoftMax function is used to derive class probabilities. These steps turn raw model output into detections that the tracking module can use.
Core ML handles inference on the iOS device, and frame processing runs asynchronously so that recognition can be performed alongside video handling. This division of work is central to the design: the drone provides sensing and movement, while the mobile device carries out the more demanding image-processing tasks.
Integrating vision with drone control
The mobile application receives the video stream and passes frames to the recognition and tracking modules. It then sends commands through the drone’s Wi-Fi access point. The command set described includes takeoff, landing, starting or stopping the stream, directional movement and clockwise or counterclockwise rotation.
When the system recognizes multiple object paths, it selects the one with the highest probability. The tracking algorithm then uses the selected detection’s position and history to decide which movement commands to issue. In the reported demonstration, commands were used to move the drone closer to bringing the recognized object toward the center of the information frame.
Performance considerations for real-time tracking
Network latency can reduce tracking quality, particularly over larger distances. Separating video-stream parsing into its own program thread is one measure described to keep stream handling from blocking other processing. The authors also identify a more powerful network adapter or implementing the algorithm directly on the drone as possible ways to address latency, without presenting these as tested improvements.
The work was evaluated in simulation and on a physical mobile platform. The paper reports approximately 70–80% correct operation under normal conditions, with the conclusion describing accuracy of about 80%. These figures are specific to the reported system and conditions; they should not be treated as a general performance guarantee for other cameras, targets, networks or operating environments.
Practical implications for automation teams
This design illustrates how a smart sensor unit can combine embedded vision, object detection and motion logic in a mobile system. For engineers and integrators, the main takeaway is architectural: recognition, tracking and command generation need to work as a coordinated pipeline, while video transport and inference must be managed so neither stalls the other.
Before adapting the approach, teams should validate recognition quality for the intended objects, define clear and non-overlapping movement thresholds, and assess the effects of communication latency. Mobile drone targeting can bring real-time object tracking closer to the edge, but reliable operation depends on carefully connecting the vision output to well-defined control decisions.
Access Official Technical PDF