Synthetic Data Generation: 从稀缺到丰富

How frontier models create training data at scale, and the critical failure modes that threaten generalization.

为什么合成数据如此关键

Real-world labeled data is expensive, slow, and fragmented. Collecting high-quality examples for specialized domains—medical imaging, legal documents, scientific papers—requires expert annotation costing thousands per sample. Meanwhile, privacy regulations (GDPR, HIPAA) restrict reuse of sensitive data. Synthetic data sidesteps these constraints: a capable model can generate unlimited task-specific examples, domain-adapted to your exact needs, without privacy leaks. This is why every frontier lab now runs synthetic data pipelines.

核心生成技术

Prompt-Based Generation

The simplest approach: use a capable model (GPT-4, Claude) to generate examples given a prompt template. E.g., "Generate 10 customer support questions about billing" → model outputs diverse questions. Quick and cheap, but quality is bounded by the prompt precision and model instruction-following.

Retrieval-Augmented Synthesis (RAS)

Ground generation in real documents. Retrieve relevant passages, then instruct the model: "Based on this document, generate a Q&A pair." This anchors synthesis to domain facts, reducing hallucination. Essential for technical domains where factual grounding matters.

Iterative Self-Refinement

Multiple passes improve quality. Generate → score → filter → regenerate weak examples. Each iteration tightens distribution toward desired quality threshold. Works especially well with weak supervision signals (e.g., reward models, classifier feedback).

Evol-Instruct (WizardLM, 2023)

Evolve instructions into progressively harder versions. Start with simple prompt → model generates response → "Make this 20% harder" → recurse. This creates curriculum-like synthetic data where complexity gradates naturally. WizardLM achieved competitive results with >90% synthetic training data.

Self-Instruct (Alpaca, Stanford 2023)

Bootstrap from a seed set of hand-written examples. Iteratively: (1) sample seed instructions, (2) prompt model to generate new instructions and outputs, (3) filter low-quality pairs, (4) add to training set. Alpaca's 52K examples cost ~$500 to generate using GPT-3.5, competitive with supervised fine-tuning.

RAFT: Retrieval-Augmented Fine-Tuning

RAFT (2024) marries retrieval-augmented generation with fine-tuning: (1) retrieve relevant documents, (2) use RAG to generate synthetic examples grounded in those documents, (3) fine-tune on synthetic + real data. Result: models that answer questions in-domain, citing sources, resistant to distribution shift. Shows 5-15% accuracy gains over vanilla fine-tuning on specialized tasks.

Model Collapse: 合成数据的致命陷阱

The Critical Risk: Train a model on synthetic data → use that model to generate more synthetic data → train the next model on that → repeat. Each generation, the distribution narrows: outliers are trimmed, variance collapses, modes concentrate. Mathematically:

After k generations, the support of the distribution is drastically reduced. Specific failure modes:

• Loss of diversity: Models repeat top-k common patterns, losing tail behaviors.
• Language degradation: Repeated paraphrasing introduces grammatical artifacts (e.g., increased use of "undoubtedly," "in conclusion").
• Fact erosion: Hallucinations compound; false patterns get reinforced as "facts."
• Catastrophic forgetting: Minority classes vanish; marginal domains become extinct in data.

Mitigation Strategies:

Scaling Laws for Synthetic Data

Early research (2023-2024) suggested synthetic data obeys different scaling laws than real data. Recent findings:

• Quality Threshold: Synthetic data below ~80% human-equivalent quality hurts more than it helps. Beyond 90%, gains plateau.
• Task Dependency: Simple tasks (classification) tolerate 100% synthetic; complex reasoning needs 40%+ real examples.
• Scale Curve: Synthetic data enables 2-4x larger effective dataset size before hitting diminishing returns, vs. real-only baseline.
• Diversity Cost: To match coverage of N real examples, need 2-3x more synthetic examples due to mode concentration.

Llama 3 & Frontier Labs: Synthetic in Practice

Meta disclosed that Llama 3 (400B tokens) mixed:

• ~60% public web data (original real)
• ~25% synthetic code/reasoning examples (generated by internal models)
• ~15% high-quality curated real data (research papers, books, forums)

GPT-4o similarly uses synthetic data for chain-of-thought reasoning and multi-modal alignment. OpenAI has hinted at iterative refinement loops where weak synthetic examples are fixed by human feedback, then reused.

Cost-Benefit Analysis

Cost per 1M tokens of synthetic data: ~$100-500 using a capable model (GPT-4 API). Comparison:

• Human annotation: $2000-5000 per 1M tokens (slower, higher quality).
• Weak supervision: $500-1000 per 1M tokens (mixed quality, requires validation).
• Synthetic (high-quality filter): $100-300 per 1M tokens (fast, needs model investment).

Model Merging: 无需重训的模型融合

Combining task-specific fine-tuned models through parameter manipulation, without retraining. A breakthrough for scalable multi-task serving.

问题：孤立的任务特定模型

Fine-tune a base model on Task A → get a specialized model with SOTA performance. Fine-tune the same base on Task B → another SOTA model. But you have two separate models. Want both in one? Naive averaging of weights produces catastrophic interference: performance on both tasks drops 20-40%. You need either (a) ensemble all models at inference (expensive), or (b) retrain from scratch with multi-task loss (slow, needs careful tuning).

Model merging solves this: combine task-specific checkpoints into a single unified model that retains performance on all tasks, without retraining. This unlocked the "model marketplace" where community members fine-tune and merge models freely (e.g., Open LLM Leaderboard merged models achieving SOTA with zero additional training).

核心融合技术

Model Soup (Wortsman et al., 2022)

The simplest merge: uniform averaging of fine-tuned checkpoints. Given base model θ₀ and n task-specific models θ₁, θ₂, ..., θₙ, the soup is:

Why it works: Fine-tuning on different tasks explores different regions of parameter space, but many regions are "compatible"—averaging them sums the task-specific improvements. Works best when tasks are diverse and base model is strong.

• Pros: Dead simple; no hyperparameters; embarrassingly parallel.
• Cons: Naive averaging doesn't resolve task interference (conflicting gradients); performance lags single-task models by 5-15%.

Task Arithmetic (Ilharco et al., 2023)

Instead of merging raw weights, merge task vectors—the difference between fine-tuned and base model. Given base θ₀ and task-specific θᵢ, the task vector is τᵢ = θᵢ - θ₀. Then:

where λᵢ are scalar weights controlling each task's contribution. Intuition: Task vectors isolate task-specific signal, removing the base model noise. Merging them preserves shared knowledge (θ₀) while adding task-specific deltas.

• Pros: More interpretable; task vectors are sparse (few non-zero elements), so interference is reduced.
• Cons: Still naive—conflicting deltas cause cancellation or amplification; needs manual tuning of λᵢ.

TIES-Merging (Trim, Elect Sign, Merge) - Wang et al., 2023

A principled solution to task interference via a three-step algorithm:

Impact: TIES-merging reduced interference dramatically. On 8-task merges, TIES achieved 96% of average single-task performance vs. 78% for naive soup. This became the de-facto standard for community model merging.

DARE: Drop And REscale (Yu et al., 2023)

A probabilistic complement to TIES. Instead of keeping top-k%, randomly drop each delta parameter with probability p, then rescale survivors by 1/(1-p):

Why rescale? To maintain expected value; dropping 90% of params requires scaling survivors 10x to compensate, preventing collapse.

• Key finding: Works surprisingly well even at p=0.9-0.99 (dropping 90-99% of deltas!). Most task-specific info concentrates in a sparse subset.
• Intuition: Task vectors are redundant; many params are noise. Stochastic sparsity discovers the signal.

DARE-TIES Combination (2024)

Recent work combines DARE's sparsity with TIES' sign-election. (1) Apply DARE to sparsify task vectors, (2) apply TIES to resolve conflicts on surviving params. Result: better generalization across diverse task sets, ~3-5% improvement over either alone.

Differentiable DARE-TIES (2024-2025)

Rather than fixed sparsity/sign rules, learn optimal merge via gradient descent. Treat dropout probabilities {p_i} and combination weights {λᵢ} as learnable parameters. Optimize on a held-out validation set (e.g., MMLU subset). Result: task-aware, adaptive merges that outperform manual tuning by 2-3%. Compute cost: ~1-2 GPU hours per merge task, still trivial vs. retraining.

任务干扰：为什么朴素平均失败

When fine-tuning on Task A, the optimizer updates weights to reduce Task A loss. Task B fine-tuning updates those same weights for Task B. If the updates conflict (e.g., increase weight w for A, decrease for B), naïve averaging cancels both changes, leaving w near its base value. Neither task benefits.

Mathematical formulation: suppose Task A loss ∂L_A/∂w > 0 (increase w) and Task B loss ∂L_B/∂w < 0 (decrease w). Merging both models gives a w between them—suboptimal for both. TIES resolves this by dropping conflicts (keeping only params where signs agree), and DARE by discovering sparsity (assuming signal concentrates in non-conflicting regions).

实际应用：多任务服务与社区融合

Scenario 1: LoRA Merging — Fine-tune a base model with 5 different LoRA adapters (one per task). Each LoRA adds ~0.1% to base param count. Rather than load all LoRAs at inference, merge them into base via TIES. Single inference path, negligible overhead, near-lossless performance on all tasks.

Scenario 2: Open LLM Leaderboard — Community merges fine-tuned models (Mistral, Llama) into "super-models" (e.g., Mistral-7B-Merge-v1). These merged models sometimes outperform their constituent models on averaged benchmarks. Model merging democratized "research-grade" model optimization—any practitioner can now combine models without access to training infrastructure.

Scenario 3: Cost Reduction for Multi-Task APIs — Instead of running 10 separate models (10x VRAM, 10x latency), merge into 1. Trade-off: single-task performance drops 2-5%, but total cost drops 10x. Favorable for SaaS providers.

When Merging Helps vs. Hurts

Diffusion Transformers: U-Net的终结

Why Vision Transformers replaced U-Net in diffusion models, and how Sora, Flux, and Stable Diffusion 3 define the new generation.

背景：扩散模型的传统架构

For the first ~5 years of diffusion models (2020-2023), the standard architecture was U-Net: encoder-decoder with skip connections, learned via DDPM and refined via DDIM. U-Net works—it powers Stable Diffusion 1.x, DALL-E 2, and countless community models.

But U-Net has fundamental limitations: (1) Fixed compute graph— depth and width are hardcoded; scaling is awkward. (2) CNN-biased design— prioritizes local, shift-invariant patterns; long-range dependencies are harder. (3) Ecosystem mismatch— optimizations for Transformers (Flash Attention, DDP, quantization) don't transfer.

In 2023, researchers realized: diffusion is just next-token prediction with visual tokens. If Transformers scale better for language, why not vision? Enter Diffusion Transformers (DiT).

为什么DiT超越U-Net

DiT Architecture详解

Input Pipeline:

• Take noisy image latent z_t (from VAE encoder, ~8x compressed).
• Patchify: split into patches (e.g., 2×2 stride), linearize → sequence.
• Embed: learnable patch embedding → token dimension (e.g., 768).
• Add positional embeddings: absolute position IDs or rotary embeddings (RoPE).

Conditioning: Timestep + Class via Adaptive Layer Norm (adaLN-Zero)

Classical approach: concatenate timestep & class labels to the sequence. Problem: breaks Transformer symmetry; adds extra tokens.

Better approach (DiT): Adaptive Layer Norm (adaLN): use timestep/class to compute layer norm affine params (γ, β). For each block:

This modulates the representation without adding tokens. adaLN-Zero variant initializes γ, β → 0, so pre-trained models are "conditioned downstream" (can plug in new timesteps/classes at test time).

Core: Standard Transformer Blocks

Repeat N times (e.g., N=28 for DiT-XL):

• Multi-head self-attention (e.g., 8 heads, head_dim=64).
• Residual connection + LayerNorm.
• Feed-forward (MLP with hidden_dim=4×embed_dim).
• Residual connection + LayerNorm.

Output: Noise or Velocity

Final layer projects each patch token to per-pixel predictions (e.g., 4 channels for 2×2 patch = 16 values). Predict either:

• ε (noise): the added Gaussian noise → standard DDPM loss.
• v (velocity): interpolation velocity between x_0 and x_T → often converges faster.

DiffiT: NVIDIA的时间敏感注意力

NVIDIA's DiffiT (2024) extends DiT with Time-dependent Multihead Self Attention (TMSA): rather than using a fixed positional embedding, each diffusion timestep gets its own positional encoding. The intuition: early denoising steps focus on structure (low freq), late steps on details (high freq). TMSA adapts receptive fields per timestep.

Results: on ImageNet 256×256, DiffiT achieves FID 1.73 (SOTA at the time). Simple change, big gains. Shows that conditioning information can be baked directly into attention geometry.

Scaling: DiT-XL/2与缩放规律

DiT scaling experiments (from the original paper):

• DiT-S/2: 60M params, FID ~5.3.
• DiT-B/2: 300M params, FID ~3.1.
• DiT-L/2: 750M params, FID ~2.6.
• DiT-XL/2: 675M params, FID ~2.27 (optimal efficiency).

Sora的背后：视频生成的DiT

OpenAI's Sora (2024) applies DiT to video generation. Key insight: video is just image sequences. Extend patchification to 3D:

• Spacetime patches: (t, h, w) → 1D sequence (e.g., 2×2 spatial, 1 frame chunks).
• Positional embeddings: now encode frame number + spatial location.
• Self-attention: over all spacetime tokens, enabling temporal coherence.

Sora can generate videos of arbitrary duration (beyond training) and resolution, because the attention operates on patch sequences—no fixed image size constraint. This is a major capability leap over earlier video diffusion models.

Flux: 开源DiT的成功

Black Forest Labs' Flux (2024) is a DiT-based open-source image model that commands ~40% of the image generation market (by inference volume on Replicate). Key features:

• 12B params, trained on public data.
• DiT architecture with joint spatial-temporal attention.
• FID ~11 on standard benchmarks, competitive with Stable Diffusion 3.
• Fast inference: ~1 second for 1024×1024 image on H100.

Flux's success validated two ideas: (1) DiT is the right architecture choice. (2) Open-source diffusion models can compete with commercial ones if scaled properly.

Stable Diffusion 3: 多模态DiT

Stability AI's Stable Diffusion 3 (2024) introduces MM-DiT (Multimodal Diffusion Transformer): a single Transformer handles both image tokens and text tokens. Instead of separate encoders, the Transformer attends across all modalities:

• Text tokens from a frozen language model (e.g., T5).
• Image patch tokens.
• Cross-attention: unified attention over both modalities.

Benefit: native multimodal alignment—the Transformer learns how image structure relates to text semantics directly, not through separate alignment losses. SD3 shows improved text rendering and concept consistency vs. Stable Diffusion 2.

动态DiT变体

D2iT (Dynamic Diffusion Transformer): Adaptively compute attention based on input complexity. Simple images → fewer layers/heads. Complex images → full model. Reduces latency by 20-30% with minimal quality loss.

DyDiT (Dynamic Depth-Wise DiT): Prune layers based on timestep. Early denoising steps (high noise) don't need deep networks; later steps do. Skip layers early → faster inference without sacrificing quality.

Video Diffusion & Temporal Attention

Extending DiT to video requires capturing temporal dynamics. Standard approaches:

• Spacetime attention: single attention over (T, H, W) tokens (Sora approach, expensive).
• Separable attention: spatial attention per frame, then temporal attention across frames (cheaper, less coherent).
• 3D patches: group (t, h, w) voxels into tokens (reduces sequence length, better scaling).

Recent models (Runway, Stability) opt for hybrid: 3D patches + some cross-frame attention. Provides good coherence without prohibitive compute.

世界模型与具体化AI：学习环境动力学

Internal environment simulators that enable prediction, planning, and counterfactual reasoning in embodied systems.

什么是世界模型

A world model is a learned, compact representation of how an environment evolves. Given current state + action, predict the next state. Given observations, infer hidden state. Given a plan, imagine future trajectories.

Unlike end-to-end RL (state → action mapping), world models decouple understanding (what will happen next) from planning (which action is best). This decomposition is powerful: a model trained on observation video can enable planning without action labels, or transfer to new tasks unseen during training.

Three canonical representations:

核心管道：观察→编码→预测→规划→行动

Observe: Camera/sensor input (e.g., image, point cloud).
Encode: Compress into latent representation (VAE bottleneck, embedding layer).
Predict: RNN or Transformer forecasts k steps ahead.
Plan: Optimize action sequence under learned model (CEM, MPPI, gradient-based).
Act: Execute highest-value action; observe result; loop.

Key advantage: planning happens in latent space (10-100 dims), not pixel space (millions of dims). This makes imagining long trajectories tractable.

应用领域

Autonomous Driving: Predict pedestrian trajectories, vehicle behavior, lane evolution 5+ seconds ahead. World models enable risk-aware planning: if pedestrian crosses, steer; otherwise, maintain speed.

Robotics: Pre-train world models on unlabeled video (robot reaching, manipulation in diverse scenes). Fine-tune with small labeled action data. Examples: diffusion models for robot arm trajectory planning (Diffusion Policy), video prediction for pick-and-place.

Game AI: MuZero (DeepMind, 2020) learns a world model of Atari games without knowing rules. Plans by rolling out the learned model; achieves superhuman play. Key insight: you don't need to understand the game rules to plan in the learned latent dynamics.

Video Generation & Understanding: World models naturally extend to unconditional generation (sample trajectories) or video understanding (infer hidden causes from observations).

三种架构范式详解

RSSM-Based (DreamerV3, 2023)

Architecture: VAE encoder → latent z_t → RNN → predict z_{t+1}. During training, use latent loss + reconstruction loss. During inference, plan by sampling action sequences and scoring under the learned model.

• Pros: Efficient (small latent space); interpretable (z_t is a bottleneck).
• Cons: Lossy compression; hallucinations compound over long horizons.

JEPA-Based (Yann LeCun's Vision, adopted by Meta AI)

Joint-Embedding Predictive Architecture: learn encoders f (for current obs) and g (for future obs) such that f(x_t) ≈ g(x_{t+1}). Minimize ||f(x_t) - g(x_{t+1})||² but with stop-gradients to prevent collapse.

• Pros: Non-contrastive (no negatives needed); learns high-level structure; invariant to pixel details.
• Cons: Doesn't directly predict observations; harder to visualize what model learns.

Transformer-Based Token Prediction

Tokenize observations (VQ-VAE, image-to-token models like VQGAN). Train a Transformer to predict next tokens: p(x_{t+1} | x_1,...,x_t). Enables scaling to billions of parameters; benefits from all Transformer optimizations.

Example: Genie (DeepMind, 2024) uses a Transformer to predict interactive environment tokens from video. Given a text prompt ("move left"), Genie generates plausible next frames. Works on diverse games without action labels.

MuZero: 无规则的游戏规划

MuZero (2020) is a landmark result: an RL algorithm that learns a compact world model—not of the full environment, but of the "value-relevant" dynamics. The model predicts:

• s_{t+1} (next abstract state)
• r_t (expected immediate reward)
• v_t (expected future return)

Why effective: By predicting only reward-relevant features, the model is smaller, faster to compute, and generalizes better than predicting full observables. MuZero achieved SOTA on Atari, Go, and chess with a single algorithm.

Genie: DeepMind 2024的交互生成

Genie (2024) is a generative interactive environment model: given a video clip from any game, Genie learns to simulate it interactively. User provides text prompts or keyboard input; Genie generates next frames.

Architecture: Transformer-based world model (token predictor). Training:

• Tokenize video frames into a sequence of codes (VQ tokens).
• Train Transformer to predict next tokens conditioned on action tokens (learned embeddings of up/down/left/right).
• Inference: iteratively sample tokens, decode back to pixels, repeat.

4D Embodied World Models (ICCV 2025)

Emerging frontier: 4D models that capture spatial layout + temporal evolution + camera motion. Instead of predicting 2D pixels, predict a 4D representation (3D voxels over time). Enables:

• View synthesis from novel viewpoints (camera motion prediction).
• Simulation from multiple simultaneous viewpoints.
• Better robotics transfer (3D understanding generalizes across camera heights/angles).

Early results show 4D models are more data-efficient and robust than 2D token models. This likely becomes standard for embodied AI in 2025-2026.

Mixture of World Models (MoWM)

Recent work (2024-2025): combine multiple world models (one for each "mode" of environment behavior). Mixture-of-Experts-style: given observations, learn which model best explains current state, then use that for planning.

Benefit: handles multimodal futures gracefully. Standard models average over modes (blurry predictions). MoWM can maintain multiple hypotheses. Essential for long-horizon planning where uncertainty compounds.

世界模型与强化学习

Model-based RL: use world model to generate synthetic trajectories ("dream" into the future). Train policy on these dream-generated samples, then deploy on real environment.

• Sample efficiency: generate unlimited synthetic experience; real environment interactions are minimized.
• Off-policy learning: world model enables learning from past data; no need to collect new trajectories for each policy version.
• Transfer: policy trained in latent space can transfer across tasks if world model is task-agnostic.

DreamerV3 (DeepMind, 2023) unified world models + policy learning: single loss optimizes both model accuracy and policy reward. Achieves competitive RL performance on Atari and control benchmarks with <10% of environment interactions compared to model-free RL.

开放问题与挑战

Sim-to-Real Transfer: World models trained in simulation often fail on real robots due to domain gap. Techniques: domain randomization, adversarial training, but still imperfect. A key research frontier.

Long-Horizon Prediction Accuracy: Errors compound exponentially. Predicting 100 frames accurately is hard; predicting 1000 frames nearly impossible. Active research: ensemble methods, uncertainty quantification, latent diffusion models.

Multimodal Futures: In stochastic environments (humans in room), many futures are plausible. Model must maintain multi-hypothesis beliefs. Standard single-mode prediction fails.

Computational Cost: Planning via world models is expensive: ~1000 forward passes per action. Techniques: distillation (student policy learns from teacher world model planning), but still nascent.