This paper describes a system for direct position and orientation control of remotely piloted aerial vehicles using haptic force feedback. Motion capture hardware is used to provide real-time position and orientation information to the closed-loop flight control algorithm. A 3D simulation environment running in parallel with the vehicle control system provides real-time proximity/collision detection and flight volume constraints. Virtual coupling elements are used to connect a haptic user interface, the simulation environment, and the vehicle controller. An implementation of the system using an electrical radio controlled (RC) helicopter in an indoor environment is presented.