Task operation
An operation says what is created, changed or recovered. The six rows separate new-instance sampling from identity-preserving translation, local edits, missing-content recovery, measurement inversion and sequence generation. Their conditions and acceptable errors differ even when one model family supports several operations.
ML–CV–medical correspondence
01 · Synthesis
Synthesis samples an instance rather than recovers a specific observed person. A text-to-image model creates a scene compatible with a description; a medical volume generator creates a possible anatomy under supplied conditions. The input can be noise alone or a combination of noise and metadata, while the output may be pixels, voxels, geometry or texture. Latent diffusion is a representative visual implementation [7]. In medicine, MAISI conditions volumetric generation on body region and spacing and provides mask-guided or tumor-filling paths [9]. Texture is another object coordinate: Text2Tex generates surface appearance on an already supplied mesh, so it does not demonstrate joint geometry recovery [8]. The elementary question is which attributes the conditions control and which variations remain free, not simply whether the rendered result looks realistic.
Synthesis is useful when its variation supports a real downstream need. Training augmentation requires improvement on held-out real cases, not performance on generated labels alone; cohort construction requires representative coverage, not sample count alone. MAISI includes external autoPET image-distribution evaluation, but that is not an independent diagnostic endpoint or privacy audit [9]. MAISI-v2 combines rectified flow, region-specific losses and changed data/training conditions, so differences cannot all be attributed to one objective [27]. Conditional anatomy and rare congenital topology also have existing implementations [28, 29]. A responsible comparison measures conditional accuracy, tail support, memorization risk and real-task usefulness at matched resources. If resampling or a statistical shape model already supplies the required variation, a larger learned generator must justify its extra assumptions and costs.
Figure 3 places 20 representative synthesis publications in the common 2018–2026* window; the arrows resolve to the bibliography [30, 31, 32, 33, 34, 35, 36, 37, 38, 7, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48]. The common selection and date conventions are defined in Section 1.
Representative publication timeline20 papers
02 · Translation
Translation changes the domain while trying to preserve the relevant object. Turning a label map into a photograph changes appearance; turning MRI into synthetic CT changes an imaging representation; virtual staining maps an unstained measurement to stain-like contrast. Paired translation learns from aligned examples, whereas unpaired translation learns from marginal domain samples with additional constraints [10, 11]. The distinction is consequential: cycle consistency can support an invertible mapping without uniquely identifying the medically correct subject-wise mapping. Virtual staining demonstrates a genuine medical translation task, with registration and tissue preparation affecting the reference pairs [13]. The output should therefore be described as a conditional estimate under a specific acquisition and training protocol, not a new measurement of every biological quantity that the target modality might reveal.
The preservation contract determines the evaluation. For synthetic CT, density and geometric errors matter through dose calculation and positioning, while for staining they matter through structures and diagnostic observations. AortaDiff already couples contrast translation with segmentation and clinically interpretable measurements, so “task-aware joint translation” is not an untouched idea [49]. Its preprint evaluation also has ROI, alignment and derived-reference limitations that should remain visible. The SynthRAD2025 report separates image, geometric and dose criteria and demonstrates why rankings depend on the endpoint [50]. Unpaired realism or average similarity is insufficient when a rare structure changes treatment planning. Translation should be compared against the current clinical workflow, including non-generative alternatives, and failures should trigger an explicit fallback rather than be hidden in a favorable mean score.
Figure 4 places 20 representative translation publications in the common 2018–2026* window; the arrows resolve to the bibliography [51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70]. The common selection and date conventions are defined in Section 1.
Representative publication timeline20 papers
03 · Editing
Editing asks for a controlled difference from an existing instance. An instruction such as changing an object's color is not unconditional sampling; the original image supplies identity and non-target attributes. InstructPix2Pix learns from generated instruction-edit triplets and reports difficulties with unwanted changes, isolation and viewpoints [14]. In medicine, digital siblings already create local anatomic variations and assess morphology and topology [15]. Here, the useful output is often a family of controlled alternatives for sensitivity analysis rather than a clinically asserted future patient. One must specify the editable region, required invariants, permitted topology changes and the meanings of edit magnitude. A visually small change can still be large for a downstream measurement, especially near a lesion, valve or vessel boundary.
A counterfactual image is not automatically a causal treatment counterfactual. An edited anatomy can support a computational experiment if the simulator and intervention are explicitly defined, but visual plausibility alone does not identify how a patient would respond to treatment. Distributional editing also cannot establish that all non-target physiology is preserved. Appropriate tests compare independent measurements before and after editing, stress boundary cases, and quantify collateral changes outside the requested region. Classical parameterized deformation or resampling can be a strong comparator when the desired variation is simple. The contribution of a learned editor should be a newly supported, validated class of variation or a better error–cost trade-off, not the relabeling of existing anatomy manipulation. This distinction keeps controlled sensitivity studies useful without overstating biological causality.
Figure 5 places 20 representative editing publications in the common 2018–2026* window; the arrows resolve to the bibliography [71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 14, 81, 82, 83, 84, 85, 86, 87, 88, 48]. The common selection and date conventions are defined in Section 1.
Representative publication timeline20 papers
04 · Restoration / fill
Restoration retains an observed scene while compensating for degradation. Denoising, deblurring, colorization and filling missing regions differ in how much of the answer is identifiable. Palette uses a shared image-conditioned diffusion formulation across several such tasks [16]; multiple conditions do not alone establish multitask learning without shared task training and evaluation. DDRM reuses an image prior across linear inverse problems through operator structure [17]. Medical examples include microscopy restoration and MAISI tumor filling [18, 9]. A filled region may be a plausible simulation object rather than evidence of the true hidden anatomy. The task description must say whether the aim is retrospective recovery, prospective diagnosis, synthetic augmentation or experimental perturbation, because identical-looking outputs can have very different acceptable failure modes.
The unobserved region is where a strong prior can become a liability. A small residual on visible pixels says little about missing structures; several plausible samples do not automatically quantify calibrated uncertainty. Existing work already separates prior-induced hallucinations from ordinary reconstruction error and develops distribution-referenced hallucination metrics [89, 18]. The latter depends on a defined reference and approximation assumptions, not a universal truth oracle. Evaluation should therefore distinguish measurement-supported detail, ambiguous completion and wrong but convincing structures. For a clinical use, the question is whether the ambiguity changes a diagnosis or quantitative endpoint and whether the system can decline to fill it. Deterministic restoration remains an essential comparator: this operation-based taxonomy includes such comparators without claiming that every denoiser is a stochastic generative model.
Figure 6 places 20 representative restoration / fill publications in the common 2018–2026* window; the arrows resolve to the bibliography [90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 16, 100, 101, 102, 103, 104, 105, 106, 107, 108]. The common selection and date conventions are defined in Section 1.
Representative publication timeline20 papers
05 · Measurement inverse
Reconstruction uses a specified forward model and a particular measurement. CT projection data and MRI k-space constrain a latent object through Equation 2; they are not merely prompts for a similar-looking image. Generative compressed sensing makes representation error explicit, while DPS approximates a noisy likelihood using denoised estimates [5, 19]. Medical score-based reconstruction demonstrates reuse of an unconditional prior under different sampling operators [20]. In the inspected study, CT projections and single-coil MRI measurements are simulated from image data. Those experiments probe operator variation, but they do not constitute prospective scanner or diagnostic validation. A precise report separates real source images, simulated measurements, real acquisition tests, external patients and clinical readers rather than collapsing all of them into “real medical data.”
Faithfulness requires more than exact data fitting. Noise and model mismatch can make strict fitting undesirable, while a small data residual cannot certify components poorly constrained by the operator [89]. Generator-range bias can persist even with more observations [5], and posterior-sample diversity can mix model, numerical and observation effects. A stronger experiment fixes the raw-data split, compares a competitive non-generative method and a generative baseline, and measures a prespecified lesion, quantitative or reader endpoint at a realistic compute budget. Active acquisition is already explored with diffusion posterior samples [109]; novelty must lie beyond merely choosing the next view. Uncertainty becomes practically useful only when it supports a calibrated decision, rejection or additional measurement with a favorable total burden, including failed cases and reacquisition time.
Figure 7 places 20 representative measurement inverse publications in the common 2018–2026* window; the arrows resolve to the bibliography [110, 111, 112, 113, 114, 115, 20, 17, 116, 19, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126]. The common selection and date conventions are defined in Section 1.
Representative publication timeline20 papers
06 · Temporal generation
Time introduces several distinct problems. Unconditional video creates a sequence; interpolation fills missing times; prediction conditions on a past; action-conditioned generation also receives an intervention on the acquisition or environment. Video diffusion and Genie exemplify joint visual sequences and interactive dynamics [21, 22]; 4D-fy adds a dynamic 3D representation with complementary image and video supervision [23]. Medical cases include CHeart's within-cycle anatomy, Cardiac Mesh Flow's dynamic meshes and ultrasound views under probe movement [24, 25, 26]. A cardiac cycle is not longitudinal disease progression, and an autoregressive “causal” architecture is not a treatment-effect model. Similarly, a smooth rendered mesh is not automatically a measured displacement field. Each temporal use must specify its conditioning history, horizon, action semantics and independent reference.
Temporal validity needs negative controls that defeat superficial metrics. Cardiac Mesh Flow's ablation shows that static hearts can attain strong periodic consistency, making endpoint agreement an insufficient motion validator [25]. CHeart's dense temporal anatomy relies on processed segmentation labels, not independent local motion truth [24]. SonoWorld already grounds ultrasound generation in metric poses, while EchoWorld studies predictive representations for probe guidance [26, 127]; neither inspected evaluation establishes an independent real-robot clinical policy-transfer trial. Contact force, deformation, physiology and scanner settings can all break a pose-only observation model. Tests should include static outputs, condition shuffling, non-replayed actions and local motion or control endpoints. The important advance is not longer or smoother video, but a demonstrably correct relation between time, action and the downstream quantity.
Figure 8 places 20 representative temporal generation publications in the common 2018–2026* window; the arrows resolve to the bibliography [128, 129, 130, 131, 21, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146]. The common selection and date conventions are defined in Section 1.