I’m working on an open-source image stitching pipeline written mostly in python and opencv. This morning I generated my first stitched imagery. There are some great commercial tools that do image stitching, but I’m exploring what is possible when I have good flight data and good knowledge of the aircraft (and camera) pose when each image is snapped. This is part of a larger University of Minnesota precision agriculture research project.
Nice, is it all 2D or is any 3D information reconstructed too?
Nice work!!!
Hi Oliver, the rendering I showed is 2d, but the tools do generate a best 3d fit of all the feature points. You can see some of the matching got a little wonky in the trees so the 3d result is a bit “noisy” but it’s in the ball park of plausible if you compare against a DEM (SRTM) and also just inspect the result visually and see the tree tops are above the crop field and the crop field is pretty flat.
Here’s some dense structure from motion I’ve been working on that might interest you https://www.youtube.com/watch?v=ktHjT0Ir0Bk
The structure from motion stuff is really fascinating. The push for our project is (a) agricultural environments where it can be hard to find good matches (tree canopies, crops, noisy, unstable feature space) (b) leverage good quality camera pose information and tight camera/flight computer integration, and © explore ideas of how good camera pose information known ahead of time can improve the feature matching and stitching process.
Part of my secret plan to take over the world is to release this work open-source (soon) and gather some feedback and ideas … so far it’s been an in-house effort but we think an open-source image stitching pipeline could be useful to the community (something with a uav focus, written in python/numpy/opencv, where researchers can dive in and make changes and test their own ideas.) Of course there are some great commercial packages and some free “cloud” services. There’s no intent to match them for final rendering quality or speed, but for those that like to dig under the hood and test their own ideas, there doesn’t seem to be a lot of options available.
@Oliver_Hamilton are you using camera or Lidar? Is this real-time?
@Ruwan_Janapriya just a single moving camera. It’s not real-time just yet, that sequence probably only took about 20 seconds to capture and 35 minutes to process on a single core i7 so there’s lots of potential for optimisation.
Nice! I myself started working on something similar to capture structure from motion.
I see you are using an approach based on optical flow… I was mainly researching on MonoSLAM. Your results are great!!