3D Computer Vision - Efficient Methods and Applications
von: Christian Wöhler
Springer-Verlag, 2012
ISBN: 9781447141501
Sprache: Englisch
382 Seiten, Download: 11139 KB
Format: PDF, auch als Online-Lesen
geeignet für:
| Preface | 6 | ||
| Acknowledgements | 9 | ||
| Contents | 11 | ||
| Part I: Methods of 3D Computer Vision | 16 | ||
| Chapter 1: Triangulation-Based Approaches to Three-Dimensional Scene Reconstruction | 17 | ||
| 1.1 The Pinhole Model | 17 | ||
| 1.2 Geometric Aspects of Stereo Image Analysis | 20 | ||
| 1.2.1 Euclidean Formulation of Stereo Image Analysis | 20 | ||
| 1.2.2 Stereo Image Analysis in Terms of Projective Geometry | 22 | ||
| 1.2.2.1 De nition of Coordinates and Camera Properties | 22 | ||
| 1.2.2.2 The Essential Matrix | 23 | ||
| 1.2.2.3 The Fundamental Matrix | 24 | ||
| 1.2.2.4 Projective Reconstruction of the Scene | 25 | ||
| 1.3 The Bundle Adjustment Approach | 28 | ||
| 1.4 Geometric Calibration of Single and Multiple Cameras | 29 | ||
| 1.4.1 Methods for Intrinsic Camera Calibration | 29 | ||
| 1.4.2 The Direct Linear Transform (DLT) Method | 30 | ||
| 1.4.3 The Camera Calibration Method by Tsai (1987) | 33 | ||
| 1.4.4 The Camera Calibration Method by Zhang (1999a) | 34 | ||
| 1.4.5 The Camera Calibration Toolbox by Bouguet (2007) | 37 | ||
| 1.4.6 Self-calibration of Camera Systems from Multiple Views of a Static Scene | 37 | ||
| 1.4.6.1 Projective Reconstruction: Determination of the Fundamental Matrix | 37 | ||
| 1.4.6.2 Metric Self-calibration | 40 | ||
| The Basic Equations for Self-calibration and Methods for Their Solution | 41 | ||
| 1.4.6.3 Self-calibration Based on Vanishing Points | 43 | ||
| 1.4.7 Semi-automatic Calibration of Multiocular Camera Systems | 44 | ||
| 1.4.7.1 The Calibration Rig | 45 | ||
| 1.4.7.2 Existing Algorithms for Extracting the Calibration Rig | 46 | ||
| 1.4.7.3 A Graph-Based Rig Extraction Algorithm | 47 | ||
| Outline of the Rig Finding Algorithm | 47 | ||
| De nition of the Graph | 49 | ||
| Extraction of Corner Candidates | 49 | ||
| Candidate Filter and Graph Construction | 50 | ||
| Non-bidirectional Edge Elimination | 50 | ||
| Edge Circle Filter | 51 | ||
| Edge Length Filter | 51 | ||
| Corner Enumeration | 52 | ||
| Notch Direction Detector | 52 | ||
| Rig Direction | 52 | ||
| 1.4.7.4 Discussion | 52 | ||
| 1.4.8 Accurate Localisation of Chequerboard Corners | 53 | ||
| 1.4.8.1 Different Types of Calibration Targets and Their Localisationin Images | 54 | ||
| 1.4.8.2 A Model-Based Method for Chequerboard Corner Localisation | 57 | ||
| 1.4.8.3 Experimental Evaluation | 60 | ||
| 1.4.8.4 Discussion | 65 | ||
| 1.5 Stereo Image Analysis in Standard Geometry | 66 | ||
| 1.5.1 Image Recti cation According to Standard Geometry | 66 | ||
| 1.5.2 The Determination of Corresponding Points | 69 | ||
| 1.5.2.1 Correlation-Based Blockmatching Stereo Vision Algorithms | 70 | ||
| 1.5.2.2 Feature-Based Stereo Vision Algorithms | 71 | ||
| General Overview | 71 | ||
| A Contour-Based Stereo Vision Algorithm | 73 | ||
| 1.5.2.3 Dense Stereo Vision Algorithms | 79 | ||
| 1.5.2.4 Model-Based Stereo Vision Algorithms | 80 | ||
| 1.5.2.5 Spacetime Stereo Vision and Scene Flow Algorithms | 81 | ||
| General Overview | 81 | ||
| Local Intensity Modelling | 83 | ||
| 1.6 Resolving Stereo Matching Errors due to Repetitive Structures Using Model Information | 88 | ||
| 1.6.1 Plane Model | 90 | ||
| 1.6.1.1 Detection and Characterisation of Repetitive Structures | 90 | ||
| 1.6.1.2 Determination of Model Parameters | 91 | ||
| 1.6.2 Multiple-plane Hand-Arm Model | 93 | ||
| 1.6.3 Decision Feedback | 93 | ||
| 1.6.4 Experimental Evaluation | 95 | ||
| 1.6.5 Discussion | 101 | ||
| Chapter 2: Three-Dimensional Pose Estimation and Segmentation Methods | 102 | ||
| 2.1 Pose Estimation of Rigid Objects | 102 | ||
| 2.1.1 General Overview | 103 | ||
| 2.1.1.1 Pose Estimation Methods Based on Explicit Feature Matching | 103 | ||
| 2.1.1.2 Appearance-Based Pose Estimation Methods | 104 | ||
| Methods Based on Monocular Image Data | 105 | ||
| Methods Based on Multiocular Image Data | 106 | ||
| 2.1.2 Template-Based Pose Estimation | 107 | ||
| 2.2 Pose Estimation of Non-rigid and Articulated Objects | 110 | ||
| 2.2.1 General Overview | 110 | ||
| 2.2.1.1 Non-rigid Objects | 110 | ||
| 2.2.1.2 Articulated Objects | 112 | ||
| 2.2.2 Three-Dimensional Active Contours | 117 | ||
| 2.2.2.1 Active Contours | 117 | ||
| 2.2.2.2 Three-Dimensional Multiple-View Active Contours | 118 | ||
| 2.2.2.3 Experimental Results on Synthetic Image Data | 120 | ||
| 2.2.3 Three-Dimensional Spatio-Temporal Curve Fitting | 122 | ||
| 2.2.3.1 Modelling the Hand-Forearm Limb | 122 | ||
| 2.2.3.2 Principles and Extensions of the CCD Algorithm | 124 | ||
| Step 1: Learning Local Probability Distributions | 125 | ||
| Step 2: Re nement of the Estimate (MAP Estimation) | 127 | ||
| 2.2.3.3 The Multiocular Extension of the CCD Algorithm | 129 | ||
| Step 1: Extraction and Projection of the Three-Dimensional Model | 129 | ||
| Step 2: Learning Local Probability Distributions from all Nc Images | 129 | ||
| Step 3: Re nement of the Estimate (MAP Estimation) | 129 | ||
| 2.2.3.4 The Shape Flow Algorithm | 130 | ||
| Step 1: Projection of the Spatio-Temporal Three-Dimensional Contour Model | 131 | ||
| Step 2: Learn Local Probability Distributions from all Nc Images | 132 | ||
| Step 3: Re ne the Estimate (MAP Estimation) | 132 | ||
| 2.2.3.5 Veri cation and Recovery of the Pose Estimation Results | 133 | ||
| Pose Veri cation | 133 | ||
| Pose Recovery on Loss of Object | 134 | ||
| 2.3 Point Cloud Segmentation Approaches | 135 | ||
| 2.3.1 General Overview | 136 | ||
| 2.3.1.1 The k-Means Clustering Algorithm | 136 | ||
| 2.3.1.2 Agglomerative Clustering | 136 | ||
| 2.3.1.3 Mean-Shift Clustering | 137 | ||
| 2.3.1.4 Graph Cut and Spectral Clustering | 137 | ||
| 2.3.1.5 The ICP Algorithm | 138 | ||
| 2.3.1.6 Photogrammetric Approaches | 139 | ||
| 2.3.2 Mean-Shift Tracking of Human Body Parts | 139 | ||
| 2.3.2.1 Clustering and Object Detection | 139 | ||
| 2.3.2.2 Target Model | 140 | ||
| 2.3.2.3 Image-Based Mean-Shift | 141 | ||
| 2.3.2.4 Point Cloud-Based Mean-Shift | 141 | ||
| 2.3.3 Segmentation and Spatio-Temporal Pose Estimation | 142 | ||
| 2.3.3.1 Scene Clustering and Model-Based Pose Estimation | 143 | ||
| 2.3.3.2 Estimation of the Temporal Pose Derivatives | 144 | ||
| 2.3.4 Object Detection and Tracking in Point Clouds | 147 | ||
| 2.3.4.1 Motion-Attributed Point Cloud | 147 | ||
| 2.3.4.2 Over-Segmentation for Motion-Attributed Clusters | 148 | ||
| 2.3.4.3 Generation and Tracking of Object Hypotheses | 149 | ||
| Chapter 3: Intensity-Based and Polarisation-Based Approaches to Three-Dimensional Scene Reconstruction | 151 | ||
| 3.1 Shape from Shadow | 151 | ||
| 3.1.1 Extraction of Shadows from Image Pairs | 152 | ||
| 3.1.2 Shadow-Based Surface Reconstruction from Dense Sets of Images | 154 | ||
| 3.2 Shape from Shading | 155 | ||
| 3.2.1 The Bidirectional Re ectance Distribution Function (BRDF) | 156 | ||
| 3.2.2 Determination of Surface Gradients | 160 | ||
| 3.2.2.1 Photoclinometry | 160 | ||
| 3.2.2.2 Single-Image Approaches with Regularisation Constraints | 162 | ||
| 3.2.3 Reconstruction of Height from Gradients | 165 | ||
| 3.2.4 Surface Reconstruction Based on Partial Differential Equations | 167 | ||
| 3.3 Photometric Stereo | 170 | ||
| 3.3.1 Photometric Stereo: Principle and Extensions | 170 | ||
| 3.3.2 Photometric Stereo Approaches Based on Ratio Images | 172 | ||
| 3.3.2.1 Ratio-Based Photoclinometry of Surfaces with Non-uniform Albedo | 173 | ||
| 3.3.2.2 Ratio-Based Variational Photometric Stereo Approach | 174 | ||
| 3.4 Shape from Polarisation | 175 | ||
| 3.4.1 Surface Orientation from Dielectric Polarisation Models | 175 | ||
| 3.4.2 Determination of Polarimetric Properties of Rough Metallic Surfaces for Three-Dimensional Reconstruction Purposes | 178 | ||
| Chapter 4: Point Spread Function-Based Approaches to Three-Dimensional Scene Reconstruction | 183 | ||
| 4.1 The Point Spread Function | 183 | ||
| 4.2 Reconstruction of Depth from Defocus | 184 | ||
| 4.2.1 Basic Principles | 184 | ||
| 4.2.2 Determination of Small Depth Differences | 188 | ||
| 4.2.3 Determination of Absolute Depth Across Broad Ranges | 191 | ||
| 4.2.3.1 De nition of the Depth-Defocus Function | 192 | ||
| 4.2.3.2 Calibration of the Depth-Defocus Function | 192 | ||
| Stationary Camera | 192 | ||
| Moving Camera | 193 | ||
| 4.2.3.3 Determination of the Depth Map | 194 | ||
| Stationary Camera | 194 | ||
| Moving Camera | 195 | ||
| 4.2.3.4 Estimation of the Useful Depth Range | 197 | ||
| 4.3 Reconstruction of Depth from Focus | 198 | ||
| Chapter 5: Integrated Frameworks for Three-Dimensional Scene Reconstruction | 200 | ||
| 5.1 Monocular Three-Dimensional Scene Reconstruction at Absolute Scale | 201 | ||
| 5.1.1 Combining Motion, Structure, and Defocus | 202 | ||
| 5.1.2 Online Version of the Algorithm | 203 | ||
| 5.1.3 Experimental Evaluation Based on Tabletop Scenes | 203 | ||
| 5.1.3.1 Evaluation of the Of ine Algorithm | 204 | ||
| Cuboid Sequence | 207 | ||
| Bottle Sequence | 207 | ||
| Lava Stone Sequence | 208 | ||
| 5.1.3.2 Evaluation of the Online Algorithm | 209 | ||
| 5.1.3.3 Random Errors vs. Systematic Deviations | 210 | ||
| 5.1.4 Discussion | 212 | ||
| 5.2 Self-consistent Combination of Shadow and Shading Features | 213 | ||
| 5.2.1 Selection of a Shape from Shading Solution Based on Shadow Analysis | 214 | ||
| 5.2.2 Accounting for the Detailed Shadow Structure in the Shape from Shading Formalism | 217 | ||
| 5.2.3 Initialisation of the Shape from Shading Algorithm Based on Shadow Analysis | 218 | ||
| 5.2.4 Experimental Evaluation Based on Synthetic Data | 220 | ||
| 5.2.5 Discussion | 221 | ||
| 5.3 Shape from Photopolarimetric Re ectance and Depth | 222 | ||
| 5.3.1 Shape from Photopolarimetric Re ectance | 224 | ||
| 5.3.1.1 Global Optimisation Scheme | 225 | ||
| 5.3.1.2 Local Optimisation Scheme | 227 | ||
| 5.3.2 Estimation of the Surface Albedo | 228 | ||
| 5.3.3 Integration of Depth Information | 229 | ||
| 5.3.3.1 Fusion of SfPR with Depth from Defocus | 230 | ||
| 5.3.3.2 Integration of Accurate but Sparse Depth Information | 231 | ||
| 5.3.4 Experimental Evaluation Based on Synthetic Data | 233 | ||
| 5.3.5 Discussion | 238 | ||
| 5.4 Stereo Image Analysis of Non-Lambertian Surfaces | 239 | ||
| 5.4.1 Iterative Scheme for Disparity Estimation | 242 | ||
| 5.4.2 Qualitative Behaviour of the Specular Stereo Algorithm | 245 | ||
| 5.5 Combination of Shape from Shading and Active Range Scanning Data | 246 | ||
| 5.6 Three-Dimensional Pose Estimation Based on Combinations of Monocular Cues | 249 | ||
| 5.6.1 Photometric and Polarimetric Information | 250 | ||
| 5.6.2 Edge Information | 251 | ||
| 5.6.3 Defocus Information | 252 | ||
| 5.6.4 Total Error Optimisation | 252 | ||
| 5.6.5 Experimental Evaluation Based on a Simple Real-World Object | 253 | ||
| 5.6.6 Discussion | 255 | ||
| Part II: Application Scenarios | 256 | ||
| Chapter 6: Applications to Industrial Quality Inspection | 257 | ||
| 6.1 Inspection of Rigid Parts | 258 | ||
| 6.1.1 Object Detection by Pose Estimation | 258 | ||
| Comparison with Other Pose Estimation Methods | 260 | ||
| 6.1.2 Pose Re nement | 262 | ||
| Comparison with Other Pose Re nement Methods | 266 | ||
| 6.2 Inspection of Non-rigid Parts | 267 | ||
| 6.3 Inspection of Metallic Surfaces | 270 | ||
| 6.3.1 Inspection Based on Integration of Shadow and Shading Features | 271 | ||
| 6.3.2 Inspection of Surfaces with Non-uniform Albedo | 271 | ||
| 6.3.3 Inspection Based on SfPR and SfPRD | 273 | ||
| 6.3.3.1 Results Obtained with the SfPR Technique | 274 | ||
| 6.3.3.2 Results Obtained with the SfPRD Technique | 277 | ||
| 6.3.4 Inspection Based on Specular Stereo | 280 | ||
| 6.3.4.1 Qualitative Discussion of the Three-Dimensional Reconstruction Results | 280 | ||
| 6.3.4.2 Comparison to Ground Truth Data | 282 | ||
| 6.3.4.3 Self-consistency Measures for Three-Dimensional Reconstruction Accuracy | 283 | ||
| 6.3.4.4 Consequences of Poorly Known Re ectance Parameters | 285 | ||
| 6.3.5 Inspection Based on Integration of Photometric Image Information and Active Range Scanning Data | 287 | ||
| 6.3.6 Discussion | 289 | ||
| Chapter 7: Applications to Safe Human-Robot Interaction | 291 | ||
| 7.1 Vision-Based Human-Robot Interaction | 291 | ||
| 7.1.1 Vision-Based Safe Human-Robot Interaction | 292 | ||
| 7.1.2 Pose Estimation of Articulated Objects in the Context of Human-Robot Interaction | 295 | ||
| 7.1.2.1 The Role of Gestures in Human-Robot Interaction | 296 | ||
| 7.1.2.2 Recognition of Gestures | 296 | ||
| 7.1.2.3 Including Context Information: Pointing Gestures and Interactions with Objects | 297 | ||
| 7.1.2.4 Discussion in the Context of Industrial Safety Systems | 298 | ||
| 7.2 Object Detection and Tracking in Three-Dimensional Point Clouds | 299 | ||
| 7.3 Detection and Spatio-Temporal Pose Estimation of Human Body Parts | 301 | ||
| 7.4 Three-Dimensional Tracking of Human Body Parts | 304 | ||
| 7.4.1 Image Acquisition | 304 | ||
| 7.4.2 Data Set Used for Evaluation | 305 | ||
| 7.4.3 Fusion of the ICP and MOCCD Poses | 307 | ||
| 7.4.4 System Con gurations Regarded for Evaluation | 309 | ||
| Con guration 1: Tracking Based on the MOCCD | 309 | ||
| Con guration 2: Tracking Based on the Shape Flow Method | 309 | ||
| Con guration 3: ICP-Based Tracking | 309 | ||
| Con guration 4: Fusion of ICP and MOCCD | 310 | ||
| Con guration 5: Fusion of ICP, MOCCD, and SF | 310 | ||
| 7.4.5 Evaluation Results | 310 | ||
| 7.4.6 Comparison with Other Methods | 314 | ||
| 7.4.7 Evaluation of the Three-Dimensional Mean-Shift Tracking Stage | 316 | ||
| 7.4.8 Discussion | 318 | ||
| 7.5 Recognition of Working Actions in an Industrial Environment | 318 | ||
| Chapter 8: Applications to Lunar Remote Sensing | 321 | ||
| 8.1 Three-Dimensional Surface Reconstruction Methodsfor Planetary Remote Sensing | 321 | ||
| 8.1.1 Topographic Mapping of the Terrestrial Planets | 321 | ||
| 8.1.1.1 Active Methods | 321 | ||
| 8.1.1.2 Shadow Length Measurements | 322 | ||
| 8.1.1.3 Stereo and Multi-image Photogrammetry | 323 | ||
| 8.1.1.4 Photoclinometry and Shape from Shading | 324 | ||
| 8.1.2 Re ectance Behaviour of Planetary Regolith Surfaces | 325 | ||
| 8.2 Three-Dimensional Reconstruction of Lunar Impact Craters | 328 | ||
| 8.2.1 Shadow-Based Measurement of Crater Depth | 328 | ||
| 8.2.2 Three-Dimensional Reconstruction of Lunar Impact Craters at High Resolution | 329 | ||
| 8.2.3 Discussion | 339 | ||
| 8.3 Three-Dimensional Reconstructionof Lunar Wrinkle Ridges and Faults | 340 | ||
| 8.4 Three-Dimensional Reconstruction of Lunar Domes | 343 | ||
| 8.4.1 General Overview of Lunar Domes | 343 | ||
| 8.4.2 Observations of Lunar Domes | 344 | ||
| 8.4.2.1 Spacecraft Observations of Lunar Mare Domes | 344 | ||
| 8.4.2.2 Telescopic CCD Imagery | 348 | ||
| 8.4.3 Image-Based Determination of Morphometric Data | 349 | ||
| 8.4.3.1 Construction of DEMs | 349 | ||
| 8.4.3.2 Error Estimation | 358 | ||
| 8.4.3.3 Comparison to Other Height Measurements | 360 | ||
| 8.4.4 Discussion | 363 | ||
| Chapter 9: Conclusion | 366 | ||
| References | 372 |