Much of my work at 3DR during 2016 has been on Site Scan - 3DR’s enterprise drone software product. Site Scan enables customers to collect, process, and analyze data captured with drones. Site Scan is used by customers in a number of industries but is primarily targeted at customers in the construction industry where timely drone data is extremely valuable.
Site Scan Android
The first version of Site Scan was launched on Android in March of this year. The Android product provided a number of flight modes that were natively supported in the APM autopilot project that powers 3DR’s Solo drone. These modes allowed customers not only to create 2d maps, but also to scan 3D objects and freely fly Solo to inspect equipment and structures.
Sony R10C camera
The need for a new camera
The Android app integrated with 3DR’s Solo quadcopter and the GoPro camera natively supported by Solo. Although this was fine for early experiments, it quickly became clear that enterprise customers demanded higher quality images. To satisfy this customer demand, 3DR developed and released integrated hardware and software support for the enterprise-grade Sony R10C camera.
The R10C continues to leverage Solo-3DR’s drone platform–although enhanced motor controllers and other hardware upgrades make it a robust enterprise drone solution. The R10C provides customers with distortion-free 20MP photos. Integrating new hardware is never easy. In this case, we decided to take on the engineering challenge because the results from the R10C camera (compared to the GoPro) provided dramatically better outcomes for our customers.
Camera integration software
With the GoPro camera, we quickly discovered that attempting to work with tiny microSD cards on dusty job sites was unpleasant at best. One of the key requirements of the R10C camera was therefore to enable wireless image transfer so users would no longer have to extract microSD cards from the vehicle to access their data. The team at 3DR wrote software to run on the Linux computer on board Solo (affectionately known as the “companion” computer because it accompanies the autopilot itself).
This software serves several purposes. It provides an API to the camera for system components to integrate with, it handles the process of appending geolocation data to the images, and it provides a server to wirelessly transfer images off of the system. Interestingly this software is written in Rust, a relatively new systems programming language.
The new camera software accompanies other code running on the companion computer to facilitate flight planning and execution. Messaging between different components in the system is conducted via Mavlink over UDP, Capnproto, and HTTP. The entire system looks something like this:
Creating Site Scan on iOS
We quickly discovered that Site Scan customers loved Site Scan, but they wanted to run it on iOS. This is due to both the quality of iOS hardware and the fact that iOS devices are the standard mobile devices deployed in industry.
New challenges
Building on what we learned from the Android product, we set out to build an iOS app that was simple, reliable, and fully integrated with the new R10C camera. This presented a few obvious challenges. The first main challenge was the need to redesign the Site Scan experience from scratch. We also needed to decide how to implement the flight control on the drone.
We resolved the design issue by focusing on the user and creating a simple and clean user interface. For example, because the application primarily allows users to create maps with drones, the application launches to a map view where the user can start flying with a few simple button presses instead of navigating extensive menus and options to set up a drone flight.
We decided to write new flight controls on the drone’s companion computer (reference the diagram above) instead of using APM’s flight modes. This provided considerable flexibility - we were able to customize the flight mode precisely and create an clean but extensive vehicle control API for the app to use. This decision has paid dividends, as implementing new flight behavior enhancements in Python on the companion computer is much faster than modifying the autopilot code.
The simple interface of the Site Scan iOS app conceals considerable complexity: the app asynchronously exchanges data with 3DR Cloud, communicates with the Solo drone over a variety of protocols, and transfers massive data sets wirelessly between the drone and an iPad. Despite this complexity, we were able to launch Site Scan 1.0 on iOS on September 1, 2016, only 5 months after starting the project. Several follow-up releases have added more functionality. The app remains focused on allowing users without any experience with drones to quickly acquire aerial data with just a few button presses.
Swift 3
The Site Scan app on iOS was written completely in Swift. Swift is rapidly becoming the standard for new iOS apps as the language and ecosystem continue to stabilize. Compared to Obj-C, writing the app in Swift made it easier to develop (particularly for new iOS developers like myself) and easier to maintain.
We leaned on a several awesome libraries and frameworks to make development easier and faster on iOS. The following are some of the more important ones:
- RxSwift
- RxSwift provides a Swift implementation of ReactiveX API. This simplifies asynchronous programming by translating asynchronous events into observable event sequences. We use observables across the iOS app to more easily handle asynchrony.
- Realm
- Realm is rapidly becoming one of the best persistence platforms for iOS. It’s easier to use in many ways than Core Data. The Site Scan app leverages Realm for persistence.
- Punic
- The Site Scan app relies on a number of internal and 3rd party dependencies. After getting frustrated with the poor performance of Carthage, one of 3DR’s developers re-implemented an Xcode build system using Python. This project has matured into a reliable build system for iOS and is much faster than Carthage, saving our team a ton of time.
Site Scan web application
Finally, Site Scan now includes a web application. The web app allows users to process and analyze data they captured with the Solo drone. This includes processing data through 3DR’s integration with Autodesk’s Recap photogrammetry engine, adding ground control points to make two dimensional maps more accurate, and exporting data for analysis in other programs.
What’s next for Site Scan
I’m extremely proud of what we accomplished with Site Scan. In under a year, our team managed to create a new software and hardware platform that is in many ways the best offering in the industry. Site Scan is getting some great new features in the near future. Over the long term, Site Scan will continue to evolve to serve customers in a number of industries. I’m excited to see how the Site Scan product continues to evolve.