Learning a crowd-powered perceptual distance metric for facial blendshapes


Creative Commons License

ÇİPİLOĞLU YILDIZ Z.

Eurasip Journal on Image and Video Processing, cilt.2023, sa.1, 2023 (SCI-Expanded) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 2023 Sayı: 1
  • Basım Tarihi: 2023
  • Doi Numarası: 10.1186/s13640-023-00609-w
  • Dergi Adı: Eurasip Journal on Image and Video Processing
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, Aerospace Database, Communication & Mass Media Index, Communication Abstracts, Compendex, INSPEC, Metadex, Directory of Open Access Journals, Civil Engineering Abstracts
  • Anahtar Kelimeler: Blendshapes, Animation, Facial expressions, Visual perception, Crowdsourcing, Metric learning
  • Manisa Celal Bayar Üniversitesi Adresli: Evet

Özet

It is known that purely geometric distance metrics cannot reflect the human perception of facial expressions. A novel perceptually based distance metric designed for 3D facial blendshape models is proposed in this paper. To develop this metric, comparative evaluations of facial expressions were collected from a crowdsourcing experiment. Then, the weights of a distance metric, based on descriptive features of the models, were optimized to match the results with crowdsourced data, through a metric learning process. The method incorporates perceptual properties such as curvature and visual saliency. A formal analysis of the results proves the high correlation between the metric output and human perception. The effectiveness and success of the proposed metric were also compared to other distance alternatives. The proposed metric will enable intelligent processing of 3D facial blendshapes data in several ways. It will be possible to generate perceptually valid clustering and visualization of 3D facial blendshapes. It will help reduce storage and computational requirements by removing redundant expressions that are perceptually identical from the overall dataset. It can also be used to assist novice animators while creating plausible and expressive facial animations.