openCV image Stitching wide angle 160 degrees - opencv

Question

I'm trying to Stitching image wide angle 160.5 degree but the result is not a good
i'm using OpenCV 4 and ffmpeg to get frames from video
ffmpeg command to get 15 frame per sec :
ffmpeg -i first.mp4 -vf fps=15 preview%05d.jpg
OpenCV Stitching code
import cv2
import numpy as np
images = []
for i in range(70):
name = ('preview%05d.jpg' % (i+1))
print(name)
images.append(cv2.imread(name , cv2.IMREAD_COLOR))
print("start ")
stitcher = cv2.Stitcher_create()
ret, pano = stitcher.stitch(images)
if ret == cv2.STITCHER_OK:
cv2.imshow('panorama', pano)
cv2.waitKey()
cv2.destroyAllWindows()
else:
print(cv2.STITCHER_ERR_NEED_MORE_IMGS)
print(cv2.STITCHER_ERR_HOMOGRAPHY_EST_FAIL)
print(cv2.STITCHER_ERR_CAMERA_PARAMS_ADJUST_FAIL)
print(ret)
print('Error during stiching')
actual result :
expected result :

Answer 1

Before the code line stitcher = cv2.Stitcher_create() you have to append some more algorithms that transform your trapezoid image view into a rectangle image view via the homography method.
use: cv2.findHomography(srcPoints, dstPoints[, method[, ransacReprojThreshold[, mask]]])
srcPoints – Coordinates of the points in the original plane, a matrix of the type CV_32FC2 or vector .
dstPoints – Coordinates of the points in the target plane, a matrix of the type CV_32FC2 or a vector .
See also here for findHomography at OpenCV.
In particular: in your case the base (bottom side of the image) shows most information whereas topside has more non relevant information. Here you should keep the aspect ratio topside the same and narrow the bottom. This should be done for every image. Once done you can try stitching them again.
Approach example to transform Trapezium based image information in e.g. square image:
(information ratio x)
----+++++++---- (1)
---+++++++++--- (1)
--+++++++++++-- (1)
-+++++++++++++- (1)
+++++++++++++++ (1)
into Squared image information:
(information ratio x)
----+++++++---- (1)
----+++++++---- (1.1)
----+++++++---- (1.2)
----+++++++---- (1.3)
----+++++++---- (1.4; most compressed information ratio)
Once this is done you can stitch it. Don't forget to post the result ;-)
Another approach is to treat the camera as a line-inspector. This method you use when you take information from each image for lets say line y1060 to 1080 (e.g. image size 1920x1080px) and then fill a new array with the information from those 20 lines in ascending order.
Update Jan 2019:
As homography appears not to do 100% the job due to the steep 60 degree angle you can try to correct the angle by performing the PerspectiveTransform first.
# you can add a for-loop + image counter here to perform action on all images taken from
# the movie-file. Then its easily replacing numbers in the next part for the name
# of the image.
scr_array = [] # source e.g. pts1 = np.float32([[56,65],[368,52],[28,387],[389,390]])
dest_array = [] # destination e.g. pts2 = np.float32([[0,0],[300,0],[0,300],[300,300]])
Matrix1 = cv2.getPerspectiveTransform(scr_array,dest_array)
dst = cv2.warpPerspective(image, Matrix1 , (cols, rows))
label = 'CarImage1' # use ('CarImage%s' % labelnr) here for automated annotation.
# cv2.imshow(label , dst) # check the image
# cv2.imwrite(('%s.jpg' % label), dst)
See also the docs here on PerspectiveTransform.

Answer 2

Stither is expecting images that have similar parts (up to some perspective transformation). It performs pairwise image registration to find this perspective transform. In your case it won't be able to find it because it simply does not exists.
Additinal step that you must perform prior to stitcher - rectify each image to correct wide angle disportion. To find rectification parameters you will need to do some camera calibrations with calibration targets.