Abstract A novel technique will be presented to test the flight of an unmanned aerial vehicle autonomously in a real-world scenario using a data-driven technique without intervening with its onboard software. With the growing applications of such vehicles, testing of autonomous flight is a very important task for rapid deployment. There are different tools for modeling and simulating unmanned vehicles in virtual worlds such as Gazebo, MATLAB, Simulink, and Webots to name a few. None of these simulation tools are able to model all possible physical parameters of a real-world environment. Hence, the flight controller or mission planning software has to be tested in the physical world in the presence of an expert before deployment for a specific task. A Perception Inference Engine evaluation tool is developed that can infer internal states of the autonomous system from external observations only. The Gazebo simulation platform is used to collect data to develop the perception model. For real-time data collection, a VICON motion capture system is used to observe the autonomous flight of a small unmanned aerial vehicle. A state-of-the-art decision tree algorithm is used to implement the data-driven approach. The technique was tested using simulation data and verified with real-time data from Intel Aero Ready to Fly and Parrot AR. 2.0 drones. Moreover, we analyzed the robustness of the proposed system by introducing noise in sensor measurement and ambiguity in the testing scenario. It is shown that the developed system can be used for the performance evaluation of a UAV operating in the physical world by significantly reducing the uncertainty in mission failure due to environmental parameters.