Computer Vision & Graphics Projects
Point Cloud Registration of Objects with Changing Geometries
Jakub Kaminski, Thomas Dieterle
Project summary: This project was used for testing a new product line of Intel RealSense SR300 cameras. Additionally, we aimed to ignite interests in engineering of youth visiting Technion Israel Institute of Technology by providing hands-on gaming experience: a sculpting contest scored using 3D scanning techniques. We developed an application that demonstrates capturing and processing of 3D data and sculpture quality assessment by gamifying user interaction. The program uses point cloud preprocessing and ICP-based matching of 3D scans with reference models and data visualization in MATLAB. In the beginning, the user selects a 3D model that he or she will reproduce. The user manipulates the model on the screen to learn about its geometry features, to later reproduce it using Play-Doh. The user places the model on a rotary table observed by a 3D camera and the 3D scan is acquired. We then used mesh processing techniques to remove the noise and ultimately perform rigid point-cloud registration and sculpting performance assessment. The results achieved ranged typically from 3mm to 15mm RMS error depending on the user skills, model geometry, material color, and lighting conditions. Qualitative registration results were presented to users with visualizations of the magnitude of relative mesh registration errors. We concluded that the scanning quality of the camera prototype was suboptimal to accurately reproduce small, feature-rich objects (target size of < 80mm), however it showed RMS errors as small as 1mm (~70mm height simple convex model) for registration of simple ground-truth 3D printed objects with their respective 3D models.
Point Cloud Registration of Objects with Changing Geometries
Jakub Kaminski
Project Summary: In this project I detected and classified different assemblies of LEGO bricks differentiated by shape and color. The objects of interest were distributed over a flat surface together with other bricks which should be ignored. I used classic computer vision techniques to achieve the goal. The background was removed using a combination of masks including color-based filtering and adaptive thresholding (preceded with Gaussian filtering). The remaining objects were subject to segmentation in the HSV space combined with canny-edge detection. Convex hulls of objects were found and used to encapsulate each of the the ROI in rectangular boundaries. The original, unprocessed objects presented in the rectangular boundaries were scaled to a normalized size and further evaluated in terms of color uniformity and the space occupied by a brick to determine the assembly type and color. The results were compared with the ground truth data and demonstrated over 85% detection accuracy. The classification consisted of 31 brick assembly types with shape and color variations.
Simple OpenGL Game with Vision-based User Input
Jakub Kaminski
Project Summary: In this project I developed a C++, OpenGL-based video game featuring a flying object that shoots bullets at emerging obstacles in the sky. I created basic game logic including checking for collisions between objects and bullets with targets. I used two red markers detected by OpenCV with a simple, color-based segmentation to determine the horizontal and vertical motion. The motion direction was defined by a vector from the image center to the centroid point of two markers. Additionally, the velocity of the spaceship was defined by the distance between two markers. The firing procedure was triggered by pressing one of the markers, therefore causing occlusion. This approach allowed for very simple game control, yet, it was prone to false positive firing due to occasional occlusions caused by the motion blur and limitations due to the lighting.