From Pixels
to Polygons

A survey of deep learning approaches for medical image-to-mesh reconstruction.

Fengming Lin, Arezoo Zakeri, Yidan Xue, Michael MacRaild, Haoran Dou, Zherui Zhou, Ziwei Zou, Ali Sarrami-Foroushani, Jinming Duan & Alejandro F. Frangi

The University of Manchester · KU Leuven

Read the paper Watch overview

One field. Four complementary perspectives.

Template Statistical shape Generative Implicit

Why geometry matters.

Medical images describe anatomy. Meshes make its geometry explicit.

From patient-specific reconstruction to virtual populations and numerical simulation, anatomical geometry connects imaging with computational medicine. The survey maps the methods, data and evaluation choices behind this transition. A reconstructed surface alone does not establish simulation readiness or clinical benefit.

Two in-silico trial examples: cerebral aneurysm flow diversion and transcatheter aortic valve treatment, each spanning imaging, reconstruction, generation and simulation. — Imaging → Reconstruction → Generation → SimulationOriginal figure

Four ways to learn anatomy.

Explore the survey taxonomy: four families, twelve subcategories.

Template models

Deform or register a reference mesh to the target anatomy. Controlled connectivity and correspondence depend on the chosen template.

Conditional deformation: image features guide deformation of a template mesh.

The trade-off Stable mesh connectivity; limited flexibility when anatomy differs from the template.

Statistical shape models

Learn anatomical variation through shape priors. Prior constraints can support consistent reconstruction while limiting fine detail.

Linear statistical shape model reconstruction pipeline.

The trade-off Anatomical regularity; a balance between the learned prior and patient-specific variation.

Generative models

Learn distributions of anatomical shape through latent representations, adversarial learning, interpolation or diffusion.

Variational autoencoder: encoding and decoding anatomical shape.

The trade-off Flexible shape synthesis; training stability, data needs and anatomical validity remain important.

Implicit models

Represent geometry through continuous functions. Detailed surfaces emerge from distance fields, neural dynamics, occupancy or radiance representations.

Signed distance function reconstruction pipeline.

The trade-off Flexible surface representation; optimisation, topology control and mesh extraction need care.

Complete Image2Mesh overview connecting CT, MR and ultrasound inputs to template, statistical shape, generative and implicit models, their output representations, losses and evaluation, with all twelve method subcategories. — The full picture: inputs, methods, representations and evaluation.Original PNG

Read the field,
not a leaderboard.

A quantitative view of the literature, with the context kept in view.

Brain MRI cross-study summary of reconstruction accuracy, geometric validity and related evaluation measures. — Brain MRI · manuscript cross-study synthesis

Cardiac MRI cross-study summary of reported reconstruction accuracy and geometric evaluation. — Cardiac MRI · manuscript cross-study synthesis

How the literature was organised Study selection

PRISMA-style flow diagram of literature identification, screening and inclusion reported by the survey. — Literature identification and selection, as reported in the manuscript.

See the bigger picture.

The full survey overview, from images to anatomical geometry.

Survey overview / 6 min 36 secOpen original video

What comes next?

Better geometry is only part of the challenge.

Geometry

Topology, watertightness, manifold structure and fidelity in thin or highly curved anatomy.

Robustness

Reliable reconstruction from noisy, sparse, incomplete and multimodal observations.

Simulation readiness

Surface accuracy is a starting point. Volume meshing and downstream validation require their own checks.

Survey illustration of the key challenges facing anatomical mesh reconstruction.

Original project figure collection All 12 original figures, preserved

The figures from the earlier project page remain available below, including its earlier cross-study chart versions. The evidence section above uses the updated charts referenced by the supplied manuscript.

Original imaging-to-simulation pipeline — Imaging to simulation

Original project taxonomy figure — Survey taxonomy

Original conditional deformation diagram — Conditional deformation

Original registration diagram — Registration

Original linear statistical shape model — Linear SSM

Original non-linear statistical shape model — Non-linear SSM

Original generative completion diagram — Generative completion

Original neural ODE model diagram — Neural ODE

Earlier brain MRI cross-study chart — Brain MRI · earlier chart

Earlier cardiac MRI cross-study chart — Cardiac MRI · earlier chart

Original literature selection flow — Study selection

Original reconstruction challenges figure — Open challenges

Build on this survey.

Read the paper for the full taxonomy, datasets, losses, evaluation measures and study-level discussion.

Read the paper

University research record

@misc{lin2025pixels,
  title = {From Pixels to Polygons: A Survey of Deep Learning Approaches for Medical Image-to-Mesh Reconstruction},
  author = {Fengming Lin and Arezoo Zakeri and Yidan Xue and Michael MacRaild and Haoran Dou and Zherui Zhou and Ziwei Zou and Ali Sarrami-Foroushani and Jinming Duan and Alejandro F. Frangi},
  year = {2025},
  eprint = {2505.03599},
  archivePrefix = {arXiv},
  primaryClass = {cs.CV},
  doi = {10.48550/arXiv.2505.03599},
  url = {https://arxiv.org/abs/2505.03599}
}