At an academic or tech conference, several attendees gather around a well-lit poster presentation titled "Diffusion Handles: Enabling 3D Edits for Diffusion Models by Lifting Activations to 3D." The research, a collaboration among top institutions including Adobe, University of Toronto, and UCL, is prominently displayed under a "Highlight" banner. The poster delves into technical details about 3D transformations and experimental evaluations, aiming to innovate in the field of 3D model edits and diffusion. One presenter is actively explaining the content, pointing towards specific sections of the poster, while the engaged audience attentively listens and examines the presented materials. The setting suggests a vibrant exchange of knowledge and an enthusiasm for breakthrough advancements in technology and artificial intelligence.
Text transcribed from the image:
280
Highlight
28
*UCL
3D-
UNIVERSITY OF
TORONTO
Adobe
Diffusion Handles: Enabling 3D Edits for Diffusion Models by Lifting Activations to 3D
KARRAN PANDEY, PAUL GUERRERO, MATHEUS GADELHA, YANNICK HOLD-GEOffROY, KARAN SINGH, NILOY J. MITRA
DiffUSION MODELS DATA-FREE TRAINING-FREE 3D TRANSFORMATIONS
enabling 3D-
Key insight. Lift diffusion activations for an object to 3D using
a proxy depth, 3D-transform the depth and associated activations,
and project back to image space.
EXPERIMENTAL EVALUATION
Our
pres
ch improves upon prior art in terms of identity
n, edit adherence and plausibility.
Target Edit
Obj Stach
Input Ima
In contra
control, o
Pixel-Space Edit
age-space identity
edits on images.
Step 1: Lift
Step 2: Transform Step 3: Project
Using the 3D transformed V' as guidance in a diffusion model
allows us to generate an edited image that adheres to the 3D
edit while preserving identity and realism of the input ima
Inversion
0. y-a car on the beach"
Depth2lmg
0, y "a car on the beach
++ Depth21mg
Activations
nce is applied by biasing the diffusion steps using
a guidance energy G comprising a foreground ter
and a background term to ensure scene consister
Microsoft
//ASG
Our 3D-aware guidance results in consistent edit trajectories.
OXEL
VOXELS1