ActPrune: Action-Grounded Visual Token Pruning for Efficient Vision-Language-Action Models

Moon, Sungwon; Kim, Minji; Boo, Hyemin; Song, Daeun; Lee, Jiyoung

ActPrune

Action-Grounded Visual Token Pruning for Efficient Vision-Language-Action Models

Sungwon Moon¹ Minji Kim² Hyemin Boo² Daeun Song^2† Jiyoung Lee^1,2†

¹Human-Centered AI Research Institute, Ewha Womans University ²Division of AI & Software, Ewha Womans University

^†Corresponding Author

arXiv CodeSOON Video BibTeX

Video Overview

Abstract

Vision-language-action (VLA) models perform versatile manipulation tasks from language instructions, but their inference cost limits how fast a robot can respond in closed-loop control. Visual token pruning reduces this cost; however, prior methods estimate token redundancy from encoded visual embeddings or attention, only after the visual encoder has processed the entire observation. We propose ActPrune, a training-free token pruning method that moves the pruning decision ahead of the vision encoder by grounding it in a geometric prior derived from the robot's own action. The action chunk predicted at the previous control step is already available before the current observation is encoded, and it specifies the end-effector and camera motion between consecutive observations. The same action affects the two standard camera views asymmetrically: in the fixed third-person view, ActPrune retains patches along the projected end-effector trajectory together with semantic anchors, while in the wrist view, it prunes patches that remain unchanged after compensating for the action-induced camera motion. Extensive experiments on two simulation environments (LIBERO and VLABench) and four real-robot tasks demonstrate that ActPrune achieves up to 1.61× speedup and 59.5% FLOPs reduction with negligible degradation in success rates.

The action reveals visual redundancy before encoding

ActPrune concept — **Overview.** ActPrune uses the action chunk predicted at the previous step to prune visual tokens before the vision encoder, with view-specific strategies for the third-person and wrist cameras.

1

The action is available before the observation

The action chunk predicted at the previous step is available before the current observation is processed in the visual encoder. Unlike visual features or attention scores, it needs no encoding first.

2

The effect of the action depends on the viewpoint

The redundancy pattern depends on whether the camera moves. A fixed third-person camera sees a static scene where only the manipulator region changes. A wrist camera moves with the end-effector, so the whole image shifts with the action.

3

Pruning with view-specific geometric priors

Before encoding, ActPrune turns the action into a geometric prior for each view and prunes patches accordingly. After encoding, encoder attention ranks the retained tokens and prunes the rest within the budget.

Method

Before encoding, the previous action guides patch-level pruning in each view. After encoding, encoder attention guides embedding-level pruning of the remaining tokens.

ActPrune overview — **Model.** The overall framework of ActPrune. The action chunk predicted at the previous step derives view-specific pruning signals before visual encoding. Pruned patches are compressed into context tokens, and encoder attention applies the remaining budget.

STAGE 1-1. Patch-level

Third-person view: follow the trajectory

The end-effector trajectory from the previous action is projected onto the image plane, and patches around it are retained, with a larger neighborhood when the gripper is closed. Semantic anchors, extracted once per episode, preserve task-relevant regions that remain static.

STAGE 1-2. Patch-level

Wrist view: compensate the camera motion

Patches from the previous observation are reprojected into the current view using the action-induced camera motion and wrist depth. Patches whose features remain unchanged carry no new information, so they are merged into coarse context tokens instead of being encoded.

STAGE 2. Embedding-level

Rank by attention

After encoding, encoder attention captures the relative importance of patches within the current observation. The retained tokens are ranked by attention and pruned with the remaining budget, so only a small fraction of visual tokens reaches the LLM.

Results

Simulation Experiments

LIBERO results at 50%, 25%, and 12.5% visual token retention. Bold: best, underline: second best within each retention ratio.

OpenVLA-OFT on LIBERO

Method	Spatial	Object	Goal	Long	Avg. (%) ↑	FLOPs (T) ↓	Latency (ms) ↓	Speedup ↑
OpenVLA-OFT (100%)	99.0	97.0	97.4	92.6	96.5	4.739	86.17	1.00×
Retained 50% (50%↓)
+FastV	98.6	97.6	97.2	91.2	96.2	3.123	66.19	1.30×
+SparseVLM	98.0	96.2	97.2	80.8	93.1	3.388	71.89	1.20×
+DivPrune	97.8	98.2	97.2	92.8	96.5	3.015	72.33	1.19×
+VLA-Cache	97.0	96.2	97.1	90.6	95.2	3.509	78.72	1.09×
+VLA-Pruner	97.6	97.4	96.4	92.6	96.0	3.025	73.49	1.17×
+SpecPrune-VLA	95.2	98.4	97.4	48.0	84.8	3.069	72.05	1.20×
+ActPrune	98.6	98.4	97.2	93.4	96.9	2.814	65.04	1.33×
Retained 25% (75%↓)
+FastV	98.4	97.0	96.4	89.8	95.4	2.325	54.62	1.58×
+SparseVLM	97.8	97.2	97.6	74.4	91.8	2.666	61.74	1.40×
+DivPrune	98.6	97.8	95.4	88.8	95.2	2.172	55.23	1.56×
+VLA-Cache	98.0	86.0	95.4	79.2	89.7	2.869	63.20	1.36×
+VLA-Pruner	98.0	97.6	97.0	89.8	95.6	2.240	63.11	1.37×
+SpecPrune-VLA	94.4	98.4	94.8	44.0	82.9	2.283	54.37	1.58×
+ActPrune	98.0	98.8	97.4	90.0	96.1	1.921	53.61	1.61×
Retained 12.5% (87.5%↓)
+FastV	98.0	90.8	94.4	64.2	86.9	1.941	52.91	1.63×
+SparseVLM	97.2	94.6	92.6	65.6	87.5	2.340	59.94	1.44×
+DivPrune	93.6	90.0	85.6	53.0	80.6	1.750	52.51	1.64×
+VLA-Cache	92.2	88.2	92.4	56.4	82.3	2.517	59.08	1.46×
+VLA-Pruner	96.8	95.2	91.8	68.8	88.2	1.844	65.28	1.32×
+SpecPrune-VLA	94.0	97.4	94.0	40.8	81.6	2.018	54.35	1.59×
+ActPrune	96.0	96.0	91.0	71.2	88.6	1.474	52.02	1.66×

π_0.5 on LIBERO

Method	Spatial	Object	Goal	Long	Avg. (%) ↑	FLOPs (T) ↓	Latency (ms) ↓	Speedup ↑
π_0.5 (100%)	98.0	98.0	99.4	90.8	96.6	4.975	178.3	1.00×
Retained 50% (50%↓)
+FastV	97.4	98.6	96.2	89.6	95.5	3.952	178.1	1.00×
+SparseVLM	98.4	98.2	96.2	91.2	96.0	4.143	177.1	1.01×
+DivPrune	98.2	96.6	96.8	88.8	95.1	3.825	180.1	0.99×
+VLA-Cache	98.0	97.0	95.2	90.2	95.1	4.091	181.4	0.98×
+VLA-Pruner	98.0	97.8	97.0	89.0	95.5	3.895	180.3	0.99×
+ActPrune	98.0	98.4	98.4	90.6	96.4	3.695	175.5	1.02×
Retained 25% (75%↓)
+FastV	88.4	97.8	87.0	83.6	89.2	3.447	173.8	1.03×
+SparseVLM	89.6	97.8	86.4	84.6	89.6	3.632	175.0	1.02×
+DivPrune	93.6	95.0	91.2	85.8	91.4	3.257	175.8	1.01×
+VLA-Cache	91.0	94.8	92.0	83.0	90.2	3.781	177.4	1.01×
+VLA-Pruner	90.4	97.6	88.8	83.4	90.1	3.357	176.6	1.01×
+ActPrune	92.0	99.0	91.4	84.6	91.8	3.063	172.9	1.03×
Retained 12.5% (87.5%↓)
+FastV	68.4	89.6	64.0	50.4	68.1	3.196	172.4	1.03×
+SparseVLM	67.6	85.8	63.2	54.2	67.7	3.357	173.3	1.03×
+DivPrune	72.2	87.2	70.2	58.4	72.0	2.975	175.2	1.02×
+VLA-Cache	71.4	85.8	69.6	51.4	69.6	3.627	176.3	1.01×
+VLA-Pruner	70.2	86.8	73.2	55.2	71.4	3.075	175.9	1.01×
+ActPrune	74.0	90.2	72.4	59.4	74.0	2.734	171.7	1.04×

Real-world Experiments

Real-robot results of π_0.5 on a 7-DoF Franka Research 3 at 50%, 25%, and 12.5% visual token retention.

Method	Success (%) ↑				FLOPs (T) ↓	Latency (ms) ↓
Method	Tool	Doll	Towel	Banana	FLOPs (T) ↓	Latency (ms) ↓
π_0.5 (100%)	70	60	60	50	5.021	174.8
Retained 50% (50%↓)
+FastV	60	50	50	30	3.996	167.6
+SparseVLM	60	50	40	40	4.187	169.3
+DivPrune	60	40	40	20	3.869	172.3
+VLA-Cache	50	60	50	30	4.036	169.7
+VLA-Pruner	50	50	50	20	3.899	169.8
+ActPrune	60	60	50	30	3.710	166.4
Retained 25% (75%↓)
+FastV	50	40	30	10	3.490	163.4
+SparseVLM	30	40	30	20	3.675	164.5
+DivPrune	40	30	20	10	3.299	163.7
+VLA-Cache	40	50	40	20	3.799	168.8
+VLA-Pruner	40	40	30	10	3.400	167.5
+ActPrune	50	50	40	10	3.066	163.2
Retained 12.5% (87.5%↓)
+FastV	10	20	10	10	3.239	159.1
+SparseVLM	10	10	0	10	3.538	160.5
+DivPrune	20	10	0	0	3.017	159.6
+VLA-Cache	10	20	10	10	3.676	162.0
+VLA-Pruner	10	10	10	0	3.117	161.4
+ActPrune	20	20	10	10	2.744	158.3

BibTeX

@article{moon2026actprune,
  title   = {{ActPrune}: Action-Grounded Visual Token Pruning
             for Efficient Vision-Language-Action Models},
  author  = {Moon, Sungwon and Kim, Minji and Boo, Hyemin and
             Song, Daeun and Lee, Jiyoung},
  journal = {arXiv preprint arXiv:2611.08827},
  year    = {2026}
}