Action Map Policy Paper

Robotics · Manipulation Policy Learning

Action Map Policy: Learning 3D Closed-loop Manipulation via Pixel Classification

Recasting 3D closed-loop manipulation as a dense classification problem on the image plane: single-pass, multi-modal, and grounded in the pixel space.

Haojie Huang1,∗, Zhang Ye1, Linfeng Zhao2, Boce Hu1, Mingxi Jia3, Yu Qi1, Ahmed Agha1, Dian Wang2, Robert Platt1,†, Robin Walters1,† 1Northeastern University  ·  2Stanford University  ·  3Brown University  ·  ∗Corresponding author  ·  †Equal advising
Paper arXiv Code Video
argmax · k₁ kₜ
Per-step Gaussian heatmaps → argmax keypoints along the trajectory ●  full horizon · single forward pass

A new path toward learning language, vision, and action jointly. The success of large language models has revealed a clear scaling law for one-dimensional sequence data, driven by autoregressive training under a cross-entropy objective. Extending this recipe directly to robot learning is not straightforward: it is far from obvious how to autoregressively infer the next action distribution over a temporal horizon. The core difficulty is that actions are inherently high-dimensional — at least 6 or 7 degrees of freedom (including the gripper) at every step of the horizon. Action Map Policy tackles this by recasting 3D closed-loop manipulation as a dense classification problem, yielding a method whose training objective is naturally compatible with that of LLMs and VLMs.

Overview

Why this policy is different

Action Map Policy (AMP) casts 3D closed-loop manipulation as pixel classification. Rather than regressing a single action or running an iterative denoiser, AMP predicts a dense heatmap over several calibrated views and reads actions directly off the activated pixels.

Comparison of policy-learning frameworks: regression, diffusion, and Action Map Policy.
assets/intro.png figure appears here
Comparison of different policy-learning frameworks. Regression methods can easily collapse to the mean of multiple modes. Diffusion methods can, in principle, capture multi-modal distributions, but require multiple denoising steps to generate a single sample and sampling-based procedures to recover the full distribution. In contrast, our method supports single-pass inference and captures the full action distribution using a dense map that represents actions geometrically.

Experiments · Real robot

General policy-learning performance

We evaluate AMP on three long-horizon, high-precision, contact-rich tasks and compare against strong regression and diffusion baselines.

amp_make_coffee_1.mp4
videos/amp_make_coffee_1.mp4video appears here
amp_make_coffee_2.mp4
videos/amp_make_coffee_2.mp4video appears here
make-coffee — the robot opens the coffee machine lid, inserts a coffee capsule, places the mug, and presses the button.
amp_toast_bread_1.mp4
videos/amp_toast_bread_1.mp4video appears here
amp_toast_bread_2.mp4
videos/amp_toast_bread_2.mp4video appears here
toast-bread — the robot places each bread slice into the toaster slot and presses the lever down.
amp_steam_egg_1.mp4
videos/amp_steam_egg_1.mp4video appears here
amp_steam_egg_2.mp4
videos/amp_steam_egg_2.mp4video appears here
steam-egg — the robot iteratively places three eggs into the slots and sets the lid onto the egg steamer.
Method # Demos Speed Coffee Toast Egg
DiffPo DDIM 70 93.53 ms 25% (5/20) 40% (8/20) 25% (5/20)
ACT 70 7.16 ms 15% (3/20) 35% (7/20) 15% (3/20)
Action Map Policy 70 13.80 ms 80% (16/20) 90% (18/20) 85% (17/20)

Success over 20 real-robot trials. Bold marks the best result in each column; speed is end-to-end latency, detailed below.

On inference speed

AMP runs at 13.80 ms end-to-end, timed from receiving the observation to emitting the full trajectory command, not just the model forward pass, on a single NVIDIA RTX 3090. The SVD-based triangulation step adds non-negligible overhead and can be optimized further.

Action representation

The new robot data representation and model paradigm: a dense action map for closed-loop 3D learning

How actions are represented decides how a policy learns from the physical world. Action Map Policy introduces a new paradigm: a dense action representation that casts closed-loop 3D manipulation as heatmaps over the image plane, turning policy learning into classification.

Heatmap-prediction visualization on the make-coffee task across calibrated side views.
assets/dense_action_repr.png figure appears here
Heatmap-prediction visualization on the make-coffee task, evaluated at the midpoint of a downward grasp toward the coffee pod. Each row corresponds to one calibrated side view. The columns show the dense prediction process: the raw decoder output, cross-entropy supervision, spatial softmax, argmax pixel selection, and the final horizon-wide trajectory. Columns (1)–(2) summarize the full 5×12 keypoint-step output of the policy, while Columns (3)–(7) visualize the image-space distribution of a single keypoint.
make-coffee · trajectory · across two side views
videos/amp_coffee_vis.mp4video appears here
make-coffee — pixel trajectory prediction rolled out over the episode.
toast-bread · trajectory · across two side views
videos/amp_toast_vis.mp4video appears here
toast-bread — pixel trajectory prediction rolled out over the episode.
steam-egg · trajectory · across two side views
videos/amp_egg_vis.mp4video appears here
steam-egg — pixel trajectory prediction rolled out over the episode.

Spatial reasoning

Grounded in pixels, responsive to small cues

Because each input pixel maps to an action prediction, the policy stays grounded in the spatial structure of the scene and can exploit fine-grained, pixel-level cues.

We test this with several identical objects where only a laser pointer marks the target. AMP follows the cue reliably while a diffusion policy ignores it, so the policy can act on small but task-critical signals.

AMP (Ours)
videos/amp_cup.mp4video appears here
Diffpo
videos/dp_cup.mp4video appears here
ACT
videos/act_cup.mp4video appears here
Several identical cups; the target is selected only by a laser pointer. AMP picks the indicated cup, while the diffusion policy and ACT do not reliably follow the cue.
AMP (Ours)
videos/amp_cube.mp4video appears here
Diffpo
videos/dp_cube.mp4video appears here
ACT
videos/act_cube.mp4video appears here
Several identical cubes; the target is selected only by a laser pointer. AMP picks the indicated cube, while the diffusion policy and ACT do not reliably follow the cue.
Method # Demos stack-target-cup stack-target-block
DiffPo DDIM 40 25% (5/20) 15% (3/20)
ACT 40 75% (15/20) 65% (13/20)
Action Map Policy 40 100% (20/20) 100% (20/20)

Success over 20 real-robot trials. Bold marks the best result in each column.

Reference

Citation

BibTeX 
@article{huang2026action,
  title={Action Map Policy: Learning 3D Closed-loop Manipulation via Pixel Classification},
  author={Huang, Haojie and Ye, Zhang and Zhao, Linfeng and Hu, Boce and Jia, Mingxi and Qi, Yu and Agha, Ahmed and Wang, Dian and Platt, Robert and Walters, Robin},
  journal={arXiv preprint},
  year={2026}
}