Shape Analysis With Crowdsourced Data

Abstract

With explosion of 3D geometric data and its increasing usage in many applications, ranging

from computer-aided design to medicine to paleontology, shape analysis is becoming an

important research field. Common to many shape analysis tasks are two sub-problems: 1)

segmenting a shape into meaningful parts and 2) identifying important/salient points on

a shape. Both are easier for humans than for computers. In this thesis, we revisit these

problems, from the angle of using crowdsourced data to learn from humans.

We first investigate the problem of mesh segmentation. By constructing a benchmark of

4300 manually generated segmentations for 380 surface meshes of 19 different object categories,

and developing software to analyze 11 geometric properties of segmentations and

to compute 4 quantitative metrics for comparison of segmentations, we are able to quantitatively

answer "How do people decompose shapes into meaningful parts" and "How

algorithms do in comparison with humans". The benchmark is widely adopted for evaluating

new segmentation algorithms and prompts emergence of learning-based algorithms for

mesh segmentation and labeling.

We then visit the problem of identifying salient points on meshes. Rather than defining

saliency bottom-up from low-level geometric features, we start with the social/psychological

essence of saliency by investigating Schelling points [1] on 3D meshes. We designed an

online pure coordination game that asked people to select points on 3D surfaces that they

expect will be selected by other people. We then analyzed properties of the selected points,

finding that Schelling point sets are usually highly symmetric, that local curvature properties

proposed in previous work are most helpful for identifying obvious Schelling points

while global properties (e.g., segment centeredness, proximity to symmetry axis, etc.) are

required to explain more subtle features. Based on these observations, we use regression to

combine multiple properties into an analytical model that predicts where Schelling points

are likely to be on new meshes.

Shared by these two problems are the desire to transfer properties and distributions collected

for known meshes to unseen ones. Finally, we propose MeshMatch, a mesh-based

analogy of the image-based PatchMatch [2] algorithm and a non-parametric multi-resolution

approach to surface property transfer based on this algorithm. This method offers benefits

of both parametrization and texture synthesis approaches to preserving both large- and finescale

patterns of the source properties. We show its applications in texture transfer, detail

transfer, texture painting, and transferring Schelling distributions.