Autonomous Cinematography:

Your Virtual Film Crew

Getting your video from a camera that is free to move around a scene and which films from the best viewing angles is a compelling proposition. In practice, however, controlling a drone to generate video is often far too daunting for the average user. There are simply too many things to worry about: how to move the drone, how much to rotate it, panning the camera down, framing the shot, and making sure the drone doesn’t crash into anything, and all while the scene itself is changing.

Compounding this difficulty, unreliable GPS measurements and noisy sensor readings mean the drone’s own positioning measurements are often inaccurate. Even after filming, it can be tedious to access the video output, which must then be edited to produce presentable final footage. As a whole, it adds up to a cumbersome user experience, which makes it difficult to reliably capture great video of your scene.

We set out to bridge the gap between this challenge of reliably producing great drone video and the vision of what a truly intelligent flying camera can be. Leveraging decades of combined expertise in the fields of machine learning, computer vision, graphics, and animation, we create innovative, practical and intuitive solutions. Our goal is to enable anyone to capture great video of the moments in their life, using readily-available, consumer drones along with the ClearVuze app.

Our solution begins by analyzing a scene, drawing on advanced techniques in the field of computer vision to scan and map the 3-D structure of the scene, and then characterizing the objects and people within it. Users are guided through a simple and intuitive set-up process so that even novices are able to quickly begin filming. The scene data acquired during this set-up phase is used to accurately localize the drone in 3-D space, dynamically plan complex flight trajectories for the video capture, and reliably generate engaging video footage of the scene.

Controlling a drone by explicitly setting its direction, speed and orientation (such as with joysticks) sets a high bar that requires training and many hours of experience to master. ClearVuze’s patented Fly-By-Touch control paradigm allows users to simply indicate a region of the scene to film (much like looking through the viewport of any camera), and the software does the rest, calculating a flight trajectory to guide the drone through the scene and film the indicated region.

As users continue to capture video of the scene, AI algorithms learn from their interactions with the app to adapt the generated flight paths to the user’s preferences, improving the cinematic quality of the video, while also further streamlining the overall experience. We do this by applying advanced techniques in deep learning that have been pre-trained on a large repository of drone video data. In the future, our AI engine will even generate fully autonomous flight policies, customized for user preferences and individual scenes.

Obviating the need for a skilled pilot to explicitly control the drone is important for another reason: once the ability to capture engaging video is as simple as indicating a part of the scene to film, any user – not just the “pilot” — can connect to the drone via the ClearVuze Cloud and “order” their own customized videos.

The final step in making drone cinematography easy is to be able to get your video fast, and to get it how you want it. To this end, we live stream the drone video to the ClearVuze Cloud, from where it is clipped, processed, and delivered directly to users’ phones – only several seconds after they first request the shot.

And that is our vision: to build an intuitive, hassle-free way to get great video from an intelligent flying camera.