Evaluating flow-matching policies with the energy score

Many VLA models today generate actions with a diffusion or flow-matching action head, and they are trained on demonstration data that is inherently multi-modal: given the same observation, different operators (or the same operator on different days) may take very different but equally valid actions. To monitor such a model during training, a common practice is to hold out a validation set, sample an action (chunk) from the policy for every validation observation, and compute its mean squared error against the ground-truth action. Another natural metric is the training objective itself evaluated on the validation set, i.e., the velocity (or noise) prediction loss.

In practice, these two metrics sometimes go in opposite directions: the action MSE keeps decreasing while the velocity loss starts to increase. Which one should we trust? In this post I’ll argue that, for a policy trained on multi-modal data, the action MSE is the misleading one, and introduce the energy score, a sample-based metric that is suitable for evaluating a stochastic policy.

A 1D toy example

Consider a 1D action whose ground-truth distribution (for some fixed observation) has two equally likely, narrow modes: a Gaussian at $-1$ and a Gaussian at $+1$, each with a standard deviation of 0.15. For example, a robot can go around an obstacle either from the left ($-1$) or from the right ($+1$), but definitely not straight ahead ($0$). A flow-matching policy is trained to predict the velocity that carries a noise sample to a data sample along the straight line between them. To generate an action, it starts from Gaussian noise and follows the learned velocity field.

To mimic what can happen to a model during training, consider a family of policies whose two modes are gradually pulled towards each other: they sit at $\pm(1-\lambda)$ with the same width. $\lambda=0$ is a perfect policy, and $\lambda=1$ completely collapses the two modes into a single one at the mean action $0$. For each policy, we use the exact velocity field that generates it (which has a closed form for a Gaussian mixture), and evaluate it against the ground-truth data with two metrics:

  • Velocity MSE loss: the flow-matching training loss, computed on ground-truth data.
  • Action MSE loss: the squared error between a sampled action and a ground-truth action.

Drag the slider below to see how the two metrics change as the modes collapse. The demo also shows the energy score, which we will introduce later, and the reverse KL divergence used by GEN-0, which we discuss in Related work.

At $\lambda=0$, the sampling trajectories split into the two ground-truth modes, and the velocity loss is at its minimum. (It is not zero, because the same noisy intermediate point can lie on the path of many different noise–data pairs, so the regression target is inherently ambiguous.) Yet the action MSE of this perfect policy is about 2.05. As $\lambda$ increases, the action MSE decreases monotonically and is halved at $\lambda=1$, where every sample is (roughly) the invalid “go straight” action. Meanwhile, the velocity loss increases by more than 4x.

This is easy to see. The perfect policy picks left or right at random, independently of which way the demonstration went. Half of the time it picks the other side and is off by about 2 (a squared error of about 4), and the other half it is nearly right, so its action MSE is about 2. The collapsed policy always goes straight, which is off by about 1 from either mode, so its squared error is always about 1. It never matches the demonstration, but it is never very wrong either, and that is all the action MSE cares about.

More generally, let $X$ be a sampled action and $Y$ the ground-truth action. Since the two are independent given the observation, the action MSE is the sum of three things: the spread of the policy’s samples, the spread of the ground-truth actions, and the squared gap between their averages:

$$ \mathbb{E}(X-Y)^2 = \text{Var}(X) + \text{Var}(Y) + \left(\mathbb{E}X - \mathbb{E}Y\right)^2. $$

The data spread $\text{Var}(Y)$ is fixed, so the best a policy can do is to have zero spread and always output the average action, i.e., a deterministic policy. A perfect policy pays the data spread twice ($2\,\text{Var}(Y)$), once for the data and once for itself, so it scores twice as badly as the collapsed policy. In other words, the problem is not that the action MSE is noisy; its optimum is simply the wrong distribution. The action MSE is the right metric for a deterministic regression policy (that is what it was trained to minimize), but for a generative policy trained on multi-modal data it actively rewards mode collapse.

So why not just use the velocity loss? It is indeed a faithful metric for a single flow-matching model, but it has some limitations:

  1. Its value depends on the parameterization (velocity, noise, or clean action prediction), the interpolation path, and the time sampling distribution and weighting. So it cannot compare models trained with different recipes, let alone policies that are not flow-matching models (e.g., regression heads or autoregressive action tokens).
  2. It evaluates the velocity field at noisy intermediate points instead of the actions that the robot actually executes. Things like the number of integration steps or other inference-time tricks are not reflected.

What we want is a metric that, like the action MSE, is computed from sampled actions, but that, unlike the action MSE, is minimized only when the sampled action distribution matches the ground-truth one. The energy score is such a metric.

The energy score

Computing the energy score

For each validation observation, draw $S\ge 2$ action samples $x_1,\ldots,x_S$ from the policy and compare them with the single ground-truth action $y$:

$$ \widehat{\text{ES}} = \frac{1}{S}\sum_{s} d(x_s, y)\; -\; \frac{1}{2}\,\frac{1}{S(S-1)}\sum_{s\neq s'} d(x_s, x_{s'}),\qquad d(x,y)=\tfrac{1}{\sqrt{D}}\|x-y\|_2, $$

where $d$ is the root mean square (RMS) distance between two actions $x,y\in\mathbb{R}^D$ (an action chunk is flattened into one vector). The first term is the average distance from the samples to the ground truth, and the second is half the average distance between all distinct pairs of samples. The final metric is the average of $\widehat{\text{ES}}$ over all validation observations, and lower is better.

In other words, the energy score is the action error, measured with a non-squared distance, minus a bonus for sample diversity. A batched PyTorch implementation is only a few lines. It uses the plain Euclidean norm. Dividing the result by $\sqrt{D}$ gives the RMS version above, which only rescales the score.

import torch


def energy_score(samples, gt):
    r"""Energy score of policy samples against ground-truth actions.

    For each observation, with M samples a_1..a_M from the policy and
    ground-truth action a, the unbiased estimator is:

        ES = (1/M) * sum_i ||a_i - a||
             - 1/(2*M*(M-1)) * sum_{i != j} ||a_i - a_j||

    The returned value is the mean of ES over the batch. Lower is better.
    The norm is the (unsquared) Euclidean norm. The score is minimized only
    when the policy's action distribution matches the data distribution.

    Args:
        samples: (B, M, D) policy samples, M >= 2 per observation.
            Flatten action chunks to D = horizon * action_dim.
        gt: (B, D) ground-truth actions.

    Returns:
        Scalar tensor: energy score averaged over the batch.
    """
    M = samples.shape[1]
    acc = (samples - gt[:, None]).norm(dim=-1).mean(dim=1)
    pair = torch.cdist(samples, samples).sum(dim=(1, 2)) / (M * (M - 1))
    return (acc - 0.5 * pair).mean()

Note that torch.cdist also returns the diagonal $i=j$, but those distances are zero, so summing over all pairs equals the sum over $i\neq j$.

The rest of this section explains why this simple recipe is the right one.

Energy distance

The energy score comes from a distance between two distributions. Let $P$ be the predicted action distribution and $Q$ the ground-truth action distribution for an observation. Draw $X,X'\overset{\text{iid}}{\sim}P$ and $Y,Y'\overset{\text{iid}}{\sim}Q$, all independent. Using the same RMS distance $d$ as above, the energy distance [1] is

$$ D_E(P,Q) = 2\,\mathbb{E}\,d(X,Y) - \mathbb{E}\,d(X,X') - \mathbb{E}\,d(Y,Y'). $$

Intuition. $D_E$ compares how far apart points are across the two distributions with how far apart points are within each one:

  • $\mathbb{E}\,d(X,Y)$: the typical distance from a predicted action to a ground-truth action.
  • $\mathbb{E}\,d(X,X')$ and $\mathbb{E}\,d(Y,Y')$: the typical spread of each distribution.

If $P=Q$, a predicted action is no farther from a ground-truth action than two ground-truth actions are from each other, so the cross term equals the within terms and $D_E=0$. Any mismatch in location, spread, or shape (for example, a missing mode) makes the cross term larger, so $D_E>0$. Formally, $D_E(P,Q)\ge 0$, with equality iff $P=Q$ (proof in the Appendix).

It’s worth noting that the distance must not be squared. With squared distances, the same decomposition as in the action MSE applies: the spread terms cancel out exactly, and all that remains is the gap between the averages of the two distributions. A collapsed policy with the right average action would then get a perfect score. The non-squared distance is what makes the energy distance sensitive to the full shape of a distribution.

From distance to score

We cannot evaluate $D_E$ directly: each observation has only one ground-truth action $y$, so $\mathbb{E}\,d(Y,Y')$ cannot be estimated. That term does not depend on $P$, though, so we drop it and halve the rest. This gives the energy score [2]:

$$ \text{ES}(P,y) = \mathbb{E}\,d(X,y) - \tfrac{1}{2}\,\mathbb{E}\,d(X,X'). $$

The $\widehat{\text{ES}}$ that we compute at the start of this section is exactly this with each expectation replaced by an average over samples: the first term averages over samples, and the second averages over all distinct pairs of samples. Both averages are unbiased, so $\widehat{\text{ES}}$ is correct on average even with only two samples per observation.

Because the dropped term does not depend on the policy, minimizing the energy score averaged over the data is the same as minimizing the energy distance, so the minimum is reached only when the policy matches the data. The diversity bonus gives back the error that a well-spread, correct policy pays simply for being spread out.

In the toy example above, the purple curve shows the energy score. The perfect policy has an error term of about 1.08, but its samples are also about 1.08 apart from each other, so half of that comes back as the diversity bonus and its score is about 0.54. The collapsed policy has a slightly smaller error term (about 1.0) but almost no diversity bonus, so its score is about 0.92. Unlike the action MSE, the energy score is minimized at $\lambda=0$ and increases as the modes collapse, agreeing with the velocity loss.

Why the samples are independent

The energy distance assumes that, for a given observation, the sampled actions and the ground-truth action are independent draws. A flow-matching policy produces a sample by integrating its learned ODE from a fresh Gaussian noise draw. Given the observation and the trained weights, the integration is deterministic, so all of the randomness in a sample comes from that noise.

  • Samples vs. the ground truth: the noise is drawn at inference time and knows nothing about the demonstrated action. On held-out validation data, the weights were not fit to that action either.
  • Samples vs. each other: different samples come from independent noise draws.

Similar is not dependent. Training makes the policy’s distribution close to the data distribution, but that is a statement about distributions, not about individual draws. The model learned the distribution from other demonstrations. A held-out action is a fresh draw that the model never saw, so knowing it tells you nothing about what the policy will sample beyond what the observation already does. Independence breaks only when the model was fit to that specific action: on training data, or when validation frames leak from training episodes. In those cases the samples land unusually close to the ground truth and the score is optimistic, as with any in-sample metric.

Real evaluation curves

Finally, below are the validation curves of the three metrics from one of our VLA training runs with a flow-matching action head. All three metrics drop quickly in the first 10k steps, so we only show steps after that. The three metrics have different units and scales, so each curve is min–max normalized (on a log scale) over the steps shown. Thin lines are the raw values and thick lines are smoothed.

After roughly 25k steps, the velocity MSE loss and the energy score reach their minima and start to creep up, while the action MSE loss keeps decreasing until the end of training. Based on the toy example, this is the signature of the sampled action distribution starting to contract: the action MSE rewards it, while the other two metrics penalize it. If we selected the checkpoint by the action MSE, we would pick the last one, while both the velocity loss and the energy score would prefer a checkpoint around 25k–30k steps. Note that the energy score agrees with the velocity loss even though, just like the action MSE, it is computed purely from sampled actions.

To summarize: when a policy is trained on multi-modal data, don’t select checkpoints or compare models by the sampled action MSE. The velocity loss on validation data is a good sanity check within a single flow-matching model, and the energy score is a sample-based metric that works across different models and action heads.

Related work

Uses of the energy score

The energy score already has several applications in modern generative modeling, mostly as a training objective for models that generate samples without a tractable likelihood: speech synthesis [3], continuous autoregressive image generation [4], and, closest to our setting, a robot policy that predicts multi-modal actions in a single forward pass [5]. In probabilistic forecasting (e.g., weather), it and its 1D special case, the CRPS [6], are also standard evaluation metrics. However, to our knowledge it has not been used to evaluate generative robot policies, where the sampled action MSE remains the common choice.

Relation to the reverse KL used by GEN-0

The GEN-0 release blog [7] by Generalist AI reports a reverse KL divergence alongside the validation MSE, “which better measures mode-seeking behavior”. Its purpose there is different from ours. Together with the MSE, it characterizes how a model’s samples are distributed, rather than scoring how well they match the data distribution. They found that models with both low prediction error and low reverse KL tend to perform better with supervised fine-tuning, while models with high prediction error but low reverse KL tend to be more multi-modal and benefit more from reinforcement learning. Given $M$ policy samples $\hat{\mathbf{a}}_1,\ldots,\hat{\mathbf{a}}_M$ and the ground-truth action $\mathbf{a}^\star$, they smooth the policy samples into a density by placing a unit-variance Gaussian on each one, model the data density as a unit-variance Gaussian on the ground-truth action, and estimate the KL from the policy to the data by averaging the log density ratio at the policy samples. Plugging in the Gaussian densities, the estimator becomes

$$ \widehat{D}_{\text{KL}} = \underbrace{\frac{1}{2}\cdot\frac{1}{M}\sum_{m}\|\hat{\mathbf{a}}_m-\mathbf{a}^\star\|^2}_{\text{half the sampled action MSE}}\ +\ \underbrace{\frac{1}{M}\sum_{m}\log\frac{1}{M}\sum_{m'}\exp\Big(-\tfrac{1}{2}\|\hat{\mathbf{a}}_m-\hat{\mathbf{a}}_{m'}\|^2\Big)}_{\text{negative kernel entropy of the samples}}. $$

Interestingly, it has the same structure as the energy score: an error term to the ground truth, plus a term that rewards diverse samples (the second term becomes more negative as the samples spread out). The difference is in how the two terms are measured. The energy score uses the same distance in both terms, and that is exactly what makes them balance so that the score is minimized only when the policy matches the data. In the reverse KL estimator, the error term is a squared distance while the diversity term is a log-sum-exp over a Gaussian kernel with a fixed unit bandwidth. These two do not balance in general. This is not an issue when reading the reverse KL next to the MSE: since it is roughly half the MSE minus a sample-diversity bonus, the pair separates prediction error from spread, consistent with how GEN-0 interprets it. But it means that, on its own, the reverse KL is not minimized by the ground-truth distribution, and which distribution it prefers depends on the kernel bandwidth relative to the action scale.

What if we use the reverse KL for checkpoint selection?

Suppose we took the reverse KL out of its diagnostic role and used it as a single score to select checkpoints. We can check what it would prefer on the toy example. If we change the width (bandwidth) $h$ of the Gaussian kernels from 1 to other values and use infinitely many samples, the collapse level $\lambda$ that minimizes the expected reverse KL is:

Kernel bandwidth $h$2.01.0 (as in GEN-0)0.50.30.1
Preferred $\lambda$0.000.260.640.811.00

This is the green dotted curve in the interactive demo above, where you can switch the bandwidth and watch its minimum (the hollow circle) move.

The toy actions have an overall standard deviation of about 1, so $h=1$ corresponds to a unit-variance kernel on normalized actions. With it, the reverse KL prefers a policy whose two modes are already partially collapsed. With a smaller bandwidth, it prefers an almost completely collapsed policy, just like the action MSE. In contrast, the energy score always prefers $\lambda=0$. It has no bandwidth to tune, and rescaling the actions simply rescales the score. Moreover, with a finite number of samples the reverse KL estimator is biased (each sample’s density includes its own kernel), whereas the energy score estimator is unbiased with as few as two samples.

None of this is a problem for how GEN-0 uses the reverse KL. It is a caution against repurposing a mode-seeking diagnostic as a selection metric. If the goal is to select checkpoints or compare models by how well the sampled actions match the demonstration distribution, we want a metric whose optimum is that distribution, and the energy score is one.

Appendix: proof that the energy distance is non-negative

Show the proof

The factor $\frac{1}{\sqrt{D}}$ only rescales $D_E$, so we use $\|\cdot\|_2$. Assume $\mathbb{E}\|X\|,\mathbb{E}\|Y\|<\infty$.

Step 1: one dimension. Let $F,G$ be the CDFs of scalar $X,Y$. For any $a,b\in\mathbb{R}$, the integrand below is 1 exactly when $t$ lies between $a$ and $b$, and 0 otherwise. Its integral is therefore the length of that interval:

$$ |a-b| = \int\big(\mathbf{1}[a\le t]-\mathbf{1}[b\le t]\big)^2\,dt = \int\big(\mathbf{1}[a\le t]\,\mathbf{1}[b>t] + \mathbf{1}[b\le t]\,\mathbf{1}[a>t]\big)\,dt, $$

where the second equality holds because the square of a difference of two indicators is 1 exactly when one is 1 and the other is 0. Substituting $a=X$, $b=Y$ and taking expectations, we swap $\mathbb{E}$ and $\int$ (the integrand is non-negative). Independence of $X$ and $Y$ then factors each term:

$$ \mathbb{E}\big[\mathbf{1}[X\le t]\,\mathbf{1}[Y>t]\big] = \Pr(X\le t)\,\Pr(Y>t) = F(t)\big(1-G(t)\big), $$

and likewise for the second term, which gives $G(t)\big(1-F(t)\big)$. Hence

$$ \mathbb{E}|X-Y| = \int\big[F(1-G)+G(1-F)\big]\,dt. $$

Setting $G=F$ (for $X,X'$) and $F=G$ (for $Y,Y'$) in the same formula gives

$$ \mathbb{E}|X-X'| = \int 2F(1-F)\,dt,\qquad \mathbb{E}|Y-Y'| = \int 2G(1-G)\,dt. $$

Therefore

$$ D_E(P,Q) = 2\int\big(F(t)-G(t)\big)^2\,dt\ \ge\ 0, $$

with equality iff $F=G$.

Step 2: $D$ dimensions. The idea is to write the $D$-dimensional norm as an average of 1D absolute values, then apply Step 1 to each one.

(a) The norm as an average of projections. Let $\theta$ be a random unit vector, uniform on the sphere, and define $f(x)=\mathbb{E}_\theta|\theta^\top x|$, the average length of the “shadow” of $x$ on a random line. This function has two properties:

  • Rotation invariant: for any rotation $R$, $f(Rx)=\mathbb{E}_\theta|(R^\top\theta)^\top x| = f(x)$, because $R^\top\theta$ is also uniform on the sphere. So $f(x)$ depends only on $\|x\|$.
  • Scales linearly: $f(cx)=c\,f(x)$ for $c\ge 0$.

Hence $f(x)=f(e)\,\|x\|$ for any unit vector $e$, and

$$ \|x\| = k_D\,\mathbb{E}_\theta|\theta^\top x|,\qquad k_D = 1/f(e) > 0. $$

For example, in $D=2$, $f(e)=\mathbb{E}|\cos\phi| = 2/\pi$, so $\|x\|=\frac{\pi}{2}\mathbb{E}_\theta|\theta^\top x|$.

(b) Apply it to each term. Write $X_\theta=\theta^\top X$ and $Y_\theta=\theta^\top Y$. These are scalars with 1D distributions $P_\theta$ and $Q_\theta$, and they are still independent. Take $x=X-Y$, so that $\theta^\top x = X_\theta - Y_\theta$. Taking $\mathbb{E}$ over $X,Y$ and swapping it with $\mathbb{E}_\theta$ (the integrand is non-negative):

$$ \mathbb{E}\|X-Y\| = k_D\,\mathbb{E}_\theta\,\mathbb{E}|X_\theta-Y_\theta|. $$

In the same way, $\mathbb{E}\|X-X'\| = k_D\,\mathbb{E}_\theta\,\mathbb{E}|X_\theta-X'_\theta|$ and $\mathbb{E}\|Y-Y'\| = k_D\,\mathbb{E}_\theta\,\mathbb{E}|Y_\theta-Y'_\theta|$.

(c) Combine. Substituting into the definition of $D_E$:

$$ D_E(P,Q) = k_D\,\mathbb{E}_\theta\Big[2\,\mathbb{E}|X_\theta-Y_\theta| - \mathbb{E}|X_\theta-X'_\theta| - \mathbb{E}|Y_\theta-Y'_\theta|\Big] = k_D\,\mathbb{E}_\theta\,D_E(P_\theta,Q_\theta). $$

The bracket is exactly the 1D energy distance between $P_\theta$ and $Q_\theta$, which is $\ge 0$ by Step 1. An average of non-negative numbers is non-negative, so $D_E(P,Q)\ge 0$.

(d) Equality. If $D_E(P,Q)=0$, then $D_E(P_\theta,Q_\theta)=0$ for almost every $\theta$, so $P_\theta=Q_\theta$ by Step 1. A distribution is uniquely determined by its 1D projections (Cramér–Wold), so $P=Q$. $\blacksquare$

References

[1] G. J. Székely and M. L. Rizzo. Energy statistics: A class of statistics based on distances. Journal of Statistical Planning and Inference, 143(8):1249–1272, 2013.

[2] T. Gneiting and A. E. Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):359–378, 2007.

[3] A. A. Gritsenko, T. Salimans, R. van den Berg, J. Snoek, and N. Kalchbrenner. A spectral energy distance for parallel speech synthesis. Advances in Neural Information Processing Systems (NeurIPS), 2020.

[4] C. Shao, F. Meng, and J. Zhou. Continuous visual autoregressive generation via score maximization. International Conference on Machine Learning (ICML), 2025.

[5] J. Jia, T. Yang, X. Chen, C. Liu, and W. Zhang. Fast visuomotor policy for robotic manipulation. arXiv preprint arXiv:2510.12483, 2025.

[6] T. Gneiting, L. I. Stanberry, E. P. Grimit, L. Held, and N. A. Johnson. Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds. TEST, 17(2):211–235, 2008.

[7] Generalist Team. GEN-0: Embodied foundation models that scale with physical interaction. Generalist AI Blog, November 4, 2025. https://generalistai.com/blog/gen-0

Haonan Yu
Haonan Yu
Researcher & Engineer

Personal page