3D Reconstruction Methods

兩個3D坐標系的位置關系：旋轉分量矩陣的三個列向量為新坐標系的三個軸向相對于舊坐標系的坐標。

Coordinate System Rotation Chains

? When reading a chain from the left to right, rotations are performed around the “new” axes.
? When reading a chain from the right to left, rotations are performed around the “old” axes.

The camera coordinate system is defined so that its x and y axes are parallel to the column and row axes of the image respectively, and the z axis is perpendicular to the image plane.

Measurement in a specified plane with a single camera

In halcon,the measurement plane is defined as the plane z=0 with respect to the world coordinate system.

確定測量平面位姿方法：

其中一個標定板位置位于測量平面上

get_calib_data (CalibDataID, 'calib_obj_pose', [0,1], 'pose', Pose) 　　//[CalibObjIdx=0, CalibObjPoseIdx=1]

To compensate for the thickness of the calibration plate, the pose is moved by the corresponding value.

set_origin_pose( : : PoseIn, DX, DY, DZ : PoseNewOrigin) 　　//the translation is performed relative to the local coordinate system of the pose itself

set_origin_pose（: :PoseIn, 0, 0, d: PoseNewOrigin）//d為標定板厚度，PoseNewOrigin為真實測量平面姿態

標定板不位于測量平面上

首先對相機進行標定，標定時相機可以先不固定?？梢詷硕ㄍ瓿珊髮⑾鄼C安裝到測量位置，標定板放置到測量平面上，相機拍攝板定板圖像，從而獲取測量平面位姿。通過get_calib_data_observ_points（）獲取位姿。

使用3D點和對應的圖像點坐標點對

vector_to_pose( : : WorldX, WorldY, WorldZ, ImageRow, ImageColumn, CameraParam, Method, QualityType : Pose, Quality)

Use halcon calibration plate,the whole process of finding the calibration plate,extract the marks and determine the corresponddence between the marks and the respective 3D world coordinates can be carried automatically.Even more important, these calibration plates are highly accurate, down to 1 μm or below, which is aprerequisite for high accuracy applications.

Calibration plates with hexagonally arranged marks and calibration marks with rectangularly arranged marks are available. The calibration plates with hexagonally arranged marks are introduced with HALCON 12 and are recommended for most applications, as they provide the following advantages compared to the calibration plates with rectangularly arranged marks:

? The calibration plates with rectangularly arranged marks should fill a fourth of the image area. To
cover the whole field of view including different tilts, many images (at least 10 to 20) are needed.
As calibration plates with hexagonally arranged marks contain a significantly larger number of
calibration marks and therefore can and should cover the whole image area with a single image,
less images (6 to 7) are needed to get a comparable calibration result.
? The calibration plates with rectangularly arranged marks must be completely visible in the im-
ages whereas plates with hexagonally arranged marks may protude beyond the rim of the image.
Thus, with the latter less care must be taken when placing the calibration plate in the images.
The acquisition of the calibration images becomes faster and more convenient without a loss of
robustness.

標定板描述文件

The description files for calibration plates with hexagonally arranged marks have the file extension ’.cpd’ and those of the calibration plates with rectangularly arranged marks have the file extension ’.descr’.

Image coordinate --> World coordinate

In halcon,the region cannot be transformed directly into the world coordinate system.you should first convert them into XLD contours by gen_contour_region_xld(),then use the operator contour_to_world_plane_xld().

World coordinate--> Image coordinate

pose_to_hom_mat3d()　　affine_trans_point_3d()　　project_3d_point()

3D Position Recognition of Known Object

3D-2D points pair(mono 3D)

Shape-based 3D matching

Surface-based 3D matching (Volume matching)

3D Primitives Fitting

Perspective matching (calibratedperspective deformable matching,calibrated descriptor-based matching)

circle/rectangle pose estimation

Pose Estimation from Points

disp_3d_coord_system( : : WindowHandle, CamParam, Pose, CoordAxesLength : ) //顯示3D坐標系

vector_to_pose( : : WorldX, WorldY, WorldZ, ImageRow, ImageColumn, CameraParam, Method, QualityType : Pose, Quality) //通過點對計算姿態

find_caltab(Image : CalPlate : CalPlateDescr, SizeGauss, MarkThresh, MinDiamMarks : ) //從圖片中找出標定板區域

find_marks_and_pose() //Extract rectangularly arranged 2D calibration marks from the image and calculate initial values for the external camera parameters.

Pose Estimation Using Shape-based 3D Matching

　　The 3D shape model is generated by computing different views of the 3D object model within a user-specified pose range. The views are obtained by placing virtual cameras around the object model and projecting the 3D object model into the image plane of each camera position. The resulting 2D shape representations of all views are stored in the 3D shape model.

　　An important task is to specify the pose range. To ease this task, imagine a sphere that surrounds the object. On the surface of the sphere, a camera is placed that looks at the object. Now, the pose range can be defined by restricting the position of the camera to a part of the sphere’s surface. Additionally, the
minimum and maximum distance of the camera to the object, i.e., the radii of different spheres, must be specified.

物體坐標系的xz面定義為赤道面，N極為負y軸方向，零經線與赤道交點位于z軸上。

read_shape_model_3d（），create_shape_model_3d（），find_shape_model_3d（）

Pose Estimation Using Surface-Based 3D Matching

The surface model consists of a set of 3D points and the points' normal vectors.In contrast to shape-based matching,the instances of object are not located in images but in a 3D scene.

create_surface_model(),find_surface_model(),get_surface_matching_result()

Pose Estimation Using Deformable Surface-Based 3D Matching

create_deformable_surface_model()

find_deformable_surface_model()

add_deformable_surface_model_reference_point() //定義參考點，可位于任何位置，可用于確定抓取物體時的抓取點

get_deformable_surface_matching_result()

Pose Estimation Using 3D Primitives Fitting

fit_primitives_object_model_3d()

segment_object_model_3d()

Pose Estimation Using Calibrated Perspective Deformable Matching

create_planar_calib_deformable_model()

find_planar_calib_deformable_model()

Pose Estimation Using Calibrated Descriptor-Based Matching

Note that the calibrated descriptor-based matching is suitable mainly to determine the 3D pose of planar objects with characteristic texture and distinctive object points. For low-textured objects with rounded edges you should select one of the other pose estimation approaches. Further, thedescriptor-basedmatch-
ing is less accurate than the perspective deformable matching. But on the other hand, it is significantly faster if a large search space, e.g., caused by a large scale range, is used.

Pose Estimation for Cirlces

It is based on the known geometrical behavior of perspectively distorted circles. In particular, 3D circles are represented as ellipses in the image. Using the extracted 2D ellipse of a 3D circle together with the internal camera parameters and the known radius of the circle, the two possible 3D poses of the circle (having the same position but opposite orientations) can be obtained easily using the operator get_circle_pose.

Pose Estimation for Rectangles

In particular, a contour is segmentedinto four line segments and their intersections are considered as the corners of a quadrangular contour.Using the extracted 2D quadrangle of the 3D rectangle together with the internal camera parametersand the known size of the rectangle, the four (or eight in case of a square) possible 3D poses of therectangle can be obtained easily using the operator get_rectangle_pose.

3D Vison with a Stereo System

雙目

立體匹配方法有：

correlation-basedstereo matching：fast,can be parallelized and is invariant against gray-value chages,works good only for significantly textured areas.

multigrid stereo matching:interpolates 3D information for areas without texture based on the surrounding areas,in particular for edges, the accuracy in general is higher than for correlation-based stereo, andthe resolution is higher than for correlation-based stereo, i.e., smaller objects can be reconstructed.

multi-scanline stereo matching:determines 3D information for areas with little texture,runtime increases significantly with image size and disparity search range andhigh memory consumption.

計算視差：binocular_disparity(),binocular_disparity_mg(),binocular_disparity_ms(),計算結果以第一個rectified image為參考，the disparity for thepoint (r,c) of the first rectified image is the gray value at the position (r,c) of the disparity image.

計算距離：The distance of an object point from the stereo camera system is defined as its distance from the x-y-plane of the coordinate system of the first rectified camera.

binoclular_distance(),binocular_distance_mg(),binocular_distance_ms(),disparity_to_distance(),

計算3D坐標：disparity_to_point_3d(),disparity_image_to_xyz()

Laser Triangulation with Sheet ofLight

A sheet of light can be calibrated in two different ways:using a standard HALCON calibration plate or using a special 3D calibration object.

The calibration of the sheet-of-light setup is applied to get the internal and external camera parameters,the orientation of the light plane in the WCS, and the relative movement of the object between two successive measurements.

Depth from Focus

The setup requires only a single camera, therefore, it is possibly more compact than, e.g., a stereo setup. DFF requires, however, cameras with telecentric or microscope lenses in order to achieve a (nearly) parallel projection. Therefore, DFF is only suitable for small objects.

圖像銳度評價：hdevelopApplicationsGeneraldetermine_sharpness.hdev

Robot Vision

典型的關節型機器人有6個自由度，SCARA機器人有4個自由度（3平移+1旋轉）

For articulated robots（關節型機器人） that are used together with one camera, it is possible to calibrate the camera together with the hand-eye setup. For SCARA robots and all systems that use stereo or multi-view setups, the camera(s) must be calibrated in advance.

對于關節型機器人，如果手眼標定前，相機已經經過標定，則相機內參參數應該從優化參數中排除：

set_calib_data (CalibDataID, 'camera', 'general', 'excluded_settings','params')

對于SCARA機器人，相機需要提前經過標定，相機內參參數會自動從待優化參數中排除，無需再手動設置。

手眼標定時，如果使用相機+標定板時，create_calib_data（）中，NumCameras和NumCalibObjects參數指定為1，使用3D傳感器時，這兩個參數指定為0.

Please note that for the hand-eye calibration we stronglyrecommend to use the calibration plate with hexagonally arranged calibration marks.

使用3D sensor時，直接將姿態設置到標定數據模型中：set_calib_data_observ_pose (CalibDataID, 0, 0, I, ObjInCamPose)

Determine Translation in Z Direction for SCARA Robots

　　When calibrating SCARA robots, it is not possible to determine the Z translation of ’obj_in_base_pose’ (moving camera) or ’obj_in_tool_pose’ (stationary camera). To eliminate this ambiguity the Z translation is internally set to 0.0 in these poses and the poses ’tool_in_cam_pose’ (moving camera) and
’base_in_cam_pose’ (stationary camera), respectively, are calculated accordingly. It is necessary to determine the true translation in Z after the calibration by moving the robot to a pose of known height in the camera coordinate system. For this, the following approach can be applied：

總結

以上是生活随笔為你收集整理的Halcon 3D Vision的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。