ByteDance Seed · Diffusion Training

DiffusionOPSD

On-policy self-distillation for reward-guided diffusion training. Instead of supervising only the final image, it converts image-level reward into bounded positive and negative targets at intermediate denoising steps, refreshing those targets as the policy updates via an exponential moving average — attacking the sparse-supervision problem where endpoint rewards alone had to guide every intermediate prediction. Reports the best score in all 20 reward-matched comparisons across ten metrics against ReFL, FlowGRPO and DiffusionNFT, a 40% training-compute cut on SD3.5-M versus DiffusionNFT, and 63% fewer GPU-hours on a step-distilled Z-Image-Turbo variant. Code released.