0

I need to stitch images of 5 cameras together. The cameras are mounted on the sides of an underwater remote operated vehicle. It's basicly a cupoid with sizes of 2.1m x 1.3m x 1.85m. The cameras are mounted at the sides of this cupoid. So one camera at the front, one at the back, one at the right side, one at the left side and one at the bottom pointing downwards. All of the cameras have fisheye lenses with a viewing angle of 180° or more and have a 4k resolution. The goal is to stitch the images in realtime and to view the result with a VR headset.

I've tried a pure geometric approach where I only used the positions and parameters of the cameras. This worked ok, but the seam between the images is only valid for a very certain viewing distance. Otherwise the seam is very visible and I only managed to stitch the horizontal cameras and not the bottom one.

I've also tried vrWorks 360 from NVidia but this only works for horizontal aligned cameras so the bottom camera is ignored. Also the stitching result for the horizontal cameras was really bad.

I also stumbled across Parallax-tolerant Image Stitching with this implementation. But it also only works for horizontal aligned images and doesn't allow fisheye lenses.

My last approach was to use the calibration of PTGui and use this calibration in stitchEm which used to be a commercial stitching software but is now open source. This delivered the best results so far. But there are still some alignment errors and it only works okay if there are little to no near objects.

I also tried to use openCV directly but this was too overwhelming for me, since there are so many options to use the api and so many modules to configure.

Has anyone an idea how I could stitch these images efficiently together? I'd prefer an free open source solution but a payed closed source application is also acceptable if it gets the job done.

Grimkin
  • 1
  • 1
  • You might be better off over on https://photo.stackexchange.com/ for this one – spikey_richie Jul 20 '20 at 07:31
  • Thanks, for the tip. I also posted this now on photo.stackexchange.com, but I'll also leave it here, since it's also kind of a software problem – Grimkin Jul 22 '20 at 10:07

1 Answers1

0

I guess that when you talk about "real-time", you refer to fast image processing when stitching. If so, take a look to this open-source solution, with a lot of control, options, GPU processing and able to create panoramas with botton images:

If you refer to a real "real-time", probably you need to go for an adhoc system using OpenCV, and probably the real-time processing is going to depend on a speed-resolution trade-off. Find below the link to a basic example:

Cheers!

Arritmic
  • 136
  • 2
  • Thanks for your answer. With real time I meant live stitching. I recieve 5 video network streams and I have to display the stitched result each frame. So I have around 33ms for each frame. I also discovered Hugin, but I couldn't get it to produce a stitched result with my data. I also saw the OpenCV tutorial but the simple API is not able to produce a result and I couldn't quite figure out the advanced API – Grimkin Aug 04 '20 at 07:16
  • Okey, then you are talking about real "real-time" :D, so apart from thee OpenCV option, I do not know any other option. Sorry and good luck. – Arritmic Aug 05 '20 at 04:05