Halcon图像拼接实例讲解

[复制链接]
查看67250 | 回复0 | 2024-5-11 13:39:58 | 显示全部楼层 |阅读模式
点击下方卡片,关注“机器视觉与深度学习”

视觉/图像重磅干货,第一时间送达!

作者:DoubleLi

来源:https://www.cnblogs.com/lidabo/p/15931138.html

1. 为什么要拼接

如果你的被射物足够小;
如果你的镜头视野足够大;
如果你的银子足够多,可以买更牛的相机,更牛的镜头。。。

如果你没有那么多的如果,项目多了,图像拼接在所难免。

2. 效果是啥

借助Halcon自带的例子,就是将下面两张图像,拼接为一个更宽的图像。

图像1:



图像2:



拼接后的图像:



有没有变得更宽?

3. 拼接步骤



    读取图像

    提取特征点

    计算变换矩阵

    拼接

参考Halcon例程proj_match_points_distortion_ransac.hdev,逐步分析。该例程是基于特征点来拼接图像的。



3.1. 读取图像并显示图像

代码:
read_image (Image1, 'building_01')
read_image (Image2, 'building_02')
get_image_size (Image1, Width, Height)
dev_close_window ()
dev_open_window (0, 0, Width, Height, 'white', WindowHandle)
set_display_font (WindowHandle, 16, 'mono', 'true', 'false')
dev_display (Image1)
disp_message (WindowHandle, 'Image 1 to be matched', 'image', -1, -1, 'black', 'true')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
dev_display (Image2)
disp_message (WindowHandle, 'Image 2 to be matched', 'image', -1, -1, 'black', 'true')
disp_continue_message (WindowHandle, 'black', 'true')
效果:





3.2. 获取特征点

该例程是基于图像的特征点进行拼接图像的,需要获取两张图像的特征点。
何为特征点?根据算子points_foerstner的解释,特征点有两类,一类名为交点特征点,指的是那些图像边沿的点,另一类称作区域特征点,如,图像的颜色和亮度与周围不同的点。

代码:
points_foerstner (Image1, 1, 2, 3, 50, 0.1, 'gauss', 'true', Rows1, Columns1, CoRRJunctions, CoRCJunctions, CoCCJunctions, RowArea, ColumnArea, CoRRArea, CoRCArea, CoCCArea)
points_foerstner (Image2, 1, 2, 3, 50, 0.1, 'gauss', 'true', Rows2, Columns2, CoRRJunctions, CoRCJunctions, CoCCJunctions, RowArea, ColumnArea, CoRRArea, CoRCArea, CoCCArea)3.3. 计算仿射变换矩阵

根据两幅图像的特征点,计算出仿射变换矩阵。

代码:
proj_match_points_ransac (Image1, Image2, Rows1, Columns1, Rows2, Columns2, 'ncc', 10, 0, 0, Height, Width, 0, 0.5, 'gold_standard', 2, 42, HomMat2DUnrectified, Points1Unrectified, Points2Unrectified)3.3. 拼接

根据仿射变换矩阵进行拼接。

代码:
concat_obj (Image1, Image2, Images)
gen_projective_mosaic (Images, MosaicImageUnrectified, 1, 1, 2, HomMat2DUnrectified, 'default', 'false', MosaicMatrices2DUnrectified)
效果:



显示接缝代码:
projective_trans_pixel (MosaicMatrices2DUnrectified[9:17], [0,493], [0,0], RowTrans, ColumnTrans)
gen_contour_polygon_xld (Contour, RowTrans, ColumnTrans)
接缝效果:



仔细观察图像拼接的接缝处,发现拼接的效果并不理想,接缝是错开的。原因是两张图像的径向畸变造成的。何为径向畸变?这是镜头固有的,当焦距很大或很小时,拍出的图像尤其明显,图像的边缘处向前凹,或者向里凸的效果,仔细观察原来的两张图像,边缘处是向里凸进去的。
课外知识

相机内参数:
f:相机的主矩,即焦距
k:径向扭曲的大小,即径向畸变,一般不考虑切向畸变
sx,sy:图像传感器在水平和垂直方向上相邻像素之间的距离
cx,cy: 投影中心在成像平面的垂直投影
相机外参数:
平移向量X,Y,Z
旋转向量X,Y,Z
透视矫正
相机的内外参数是相机标定的重点。

因此,例程的后半部分就是消除这种径向畸变对拼接的影响,Halcon中有对应的算子,使用起来很方便。

4. 消除径向畸变

4.1. 读出图像

同上
4.2. 计算仿射变换矩阵

注意,使用了消除径向畸变的算子。代码:
proj_match_points_distortion_ransac (Image1, Image2, Rows1, Columns1, Rows2, Columns2, 'ncc', 10, 0, 0, Height, Width, 0, 0.5, 'gold_standard', 1, 42, HomMat2D, Kappa, Error, Points1, Points2)4.3. 预处理

消除图像中的镜像畸变。

代码:
CamParDist := [0.0,Kappa,1.0,1.0,0.5 * (Width - 1),0.5 * (Height - 1),Width,Height]
change_radial_distortion_cam_par ('fixed', CamParDist, 0, CamPar)
change_radial_distortion_image (Image1, Image1, Image1Rect, CamParDist, CamPar)
change_radial_distortion_image (Image2, Image2, Image2Rect, CamParDist, CamPar)
效果:

图像1(原图)



图像1(去除径向畸变后)



图像2(原图)



图像2(去除径向畸变后)



第一组图效果不是很明显,仔细观察第二组图,两个边缘是不是被拉平了?
4.4. 图像拼接

同上。

最终的效果:



仔细观察接缝处,这次图像拼接的很好。

5. 代码

完整代码如下:
* This example shows how to use proj_match_points_distortion_ransac to
* match two images in a mosaicking application.
* 该例子说明在拼接应用中,如何使用proj_match_points_distortion_ransac算子
* 拼接两张图片(基于特征点匹配拼接图像)
*
* Initialization
* 初始化
dev_update_off ()
* Read and display the images
* 读取并显示图像
read_image (Image1, 'building_01')
read_image (Image2, 'building_02')
get_image_size (Image1, Width, Height)
dev_close_window ()
dev_open_window (0, 0, Width, Height, 'white', WindowHandle)
set_display_font (WindowHandle, 16, 'mono', 'true', 'false')
dev_display (Image1)
disp_message (WindowHandle, 'Image 1 to be matched', 'image', -1, -1, 'black', 'true')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
dev_display (Image2)
disp_message (WindowHandle, 'Image 2 to be matched', 'image', -1, -1, 'black', 'true')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
*
* Extract points to be matched from the images
* 获取特征点
points_foerstner (Image1, 1, 2, 3, 50, 0.1, 'gauss', 'true', Rows1, Columns1, CoRRJunctions, CoRCJunctions, CoCCJunctions, RowArea, ColumnArea, CoRRArea, CoRCArea, CoCCArea)
points_foerstner (Image2, 1, 2, 3, 50, 0.1, 'gauss', 'true', Rows2, Columns2, CoRRJunctions, CoRCJunctions, CoCCJunctions, RowArea, ColumnArea, CoRRArea, CoRCArea, CoCCArea)
*
* We will first perform a normal projective matching that does not take
* the radial distortions into account to show the errors that are caused
* by neglecting the radial distortions.
* 首先,不考虑径向畸变的情况下,执行图像拼接,我们会看到由于径向畸变的影响
* 接缝处的拼接效果并不理想
proj_match_points_ransac (Image1, Image2, Rows1, Columns1, Rows2, Columns2, 'ncc', 10, 0, 0, Height, Width, 0, 0.5, 'gold_standard', 2, 42, HomMat2DUnrectified, Points1Unrectified, Points2Unrectified)
* Construct a projective mosaic from the two unrectified images.
* 将两个未修改(有径向畸变)的图像构造为一个投影拼接(projective mosaic)
concat_obj (Image1, Image2, Images)
gen_projective_mosaic (Images, MosaicImageUnrectified, 1, 1, 2, HomMat2DUnrectified, 'default', 'false', MosaicMatrices2DUnrectified)
*
* Display unrectified results
* 显示结果
get_image_size (MosaicImageUnrectified, Width, Height)
dev_set_window_extents (-1, -1, Width, Height)
dev_clear_window ()
dev_display (MosaicImageUnrectified)
* Display seam line
* 显示拼接缝隙
projective_trans_pixel (MosaicMatrices2DUnrectified[9:17], [0,493], [0,0], RowTrans, ColumnTrans)
gen_contour_polygon_xld (Contour, RowTrans, ColumnTrans)
set_line_style (WindowHandle, [1,5])
dev_set_line_width (1)
dev_set_color ('yellow')
dev_display (Contour)
set_line_style (WindowHandle, [])
dev_set_draw ('margin')
dev_set_color ('red')
dev_set_line_width (3)
gen_circle (Circle, [82,402], [228,223], [15,15])
dev_display (Circle)
* 从结果看,不考虑径向畸变的情况下,接缝处的拼接效果并不理想,接缝处是错开的
Message := 'The mosaic image does not fit'
Message[1] := 'perfectly, if radial distortions'
Message[2] := 'are not taken into account.'
disp_message (WindowHandle, Message, 'image', 200, 300, 'black', 'true')
disp_continue_message (WindowHandle, 'black', 'true')
stop ()
*
* Now, we will perform a projective matching that takes the radial
* distortions into account.
* 这次,去除径向畸变的影响,再次执行拼接
get_image_size (Image1, Width, Height)
proj_match_points_distortion_ransac (Image1, Image2, Rows1, Columns1, Rows2, Columns2, 'ncc', 10, 0, 0, Height, Width, 0, 0.5, 'gold_standard', 1, 42, HomMat2D, Kappa, Error, Points1, Points2)
* Construct camera parameters for the purpose of rectifying the images,
* i.e., to remove the radial distortions.
* 为了修改图像,构造相机参数
* 如,去除径向畸变
CamParDist := [0.0,Kappa,1.0,1.0,0.5 * (Width - 1),0.5 * (Height - 1),Width,Height]
* Remove the radial distortions from the images.
* 去除图像中的径向畸变
change_radial_distortion_cam_par ('fixed', CamParDist, 0, CamPar)
change_radial_distortion_image (Image1, Image1, Image1Rect, CamParDist, CamPar)
change_radial_distortion_image (Image2, Image2, Image2Rect, CamParDist, CamPar)
* Construct a mosaic from the two rectified images.  Note that the images
* fit together perfectly.
* 使用去除了径向畸变的图像构造拼接图像
* 可以看到图像拼接的很好
concat_obj (Image1Rect, Image2Rect, ImagesRect)
gen_projective_mosaic (ImagesRect, MosaicImage, 1, 1, 2, HomMat2D, 'default', 'false', MosaicMatrices2D)
*
* Display rectified results
* 显示修改后的结果
get_image_size (MosaicImage, Width, Height)
dev_set_window_extents (-1, -1, Width, Height)
dev_clear_window ()
dev_display (MosaicImage)
* Display seam line
* 显示接缝
projective_trans_pixel (MosaicMatrices2D[9:17], [0,493], [0,0], RowTrans, ColumnTrans)
gen_contour_polygon_xld (Contour2, RowTrans, ColumnTrans)
set_line_style (WindowHandle, [1,5])
dev_set_line_width (1)
dev_set_color ('yellow')
dev_display (Contour2)
set_line_style (WindowHandle, [])
dev_set_draw ('margin')
dev_set_color ('green')
dev_set_line_width (3)
gen_circle (Circle, [124,496], [244,239], [15,15])
dev_display (Circle)
Message := 'The mosaic image fits perfectly,'
Message[1] := 'if radial distortions are taken'
Message[2] := 'into account.'
disp_message (WindowHandle, Message, 'image', 200, 300, 'black', 'true')

* 输出拼接后的图像
write_image(MosaicImage, 'bmp', 0,'result.bmp')
资料


    Halcon例程和帮助文档
    利用halcon进行图像拼接的基本教程

本帖子中包含更多资源

您需要 登录 才可以下载或查看,没有账号?注册哦

x
您需要登录后才可以回帖 登录 | 注册哦

本版积分规则