About this work
Neural Control: Adjoint Learning Through Equilibrium Constraints is a boundary-control framework for long-horizon optimization through implicit equilibria. At each step of a physical manipulation task, boundary controls are applied and the resulting configuration is obtained by solving an equilibrium problem. Neural Control propagates gradients through branch-dependent sequences of such equilibrium solves via an proxy-adjoint formulation, avoiding solver unrolling entirely. Combined with receding-horizon continuation (RHC), it re-anchors optimization to realized equilibria at each segment, mitigating unintended switches between equilibrium branches in multistable regimes. The method is validated on simulated and real deformable linear object (DLO) manipulation, outperforming SPSA and iCEM baselines by up to 195× in time while reaching substantially lower loss under the same update budget.
Problem Setting
We study systems where the state \(\mathbf{x}(\lambda)\) at each continuation step \(\lambda \in [0,1]\) is defined implicitly by an equilibrium residual
\(\mathcal{G}(\mathbf{x}(\lambda),\,\mathbf{z}(\lambda)) = \mathbf{0},\)
Here \(\mathbf{z}(\lambda)\) denotes controllable quantities such as boundary conditions or loads, driven by a neural controller \(\mathbf{u}(\lambda) = \mathbf{u}_\Theta(\lambda)\) through known continuation dynamics \(d\mathbf{z}/d\lambda = f(\mathbf{z}, \mathbf{u})\). In mechanics this represents quasi-static force balance: \(\mathcal{G}(\mathbf{x},\mathbf{z}) = \nabla_\mathbf{x} E(\mathbf{x},\mathbf{z}) - \mathbf{F}^{\text{ext}}\).
This sequential implicit setting introduces two compounding difficulties beyond differentiating a single fixed point:
- Multistability and branch dependence. For the same control input \(\mathbf{z}\), multiple stable equilibria may coexist. The realized state is selected by the previous configuration and the basin of attraction of the warm-started solver. Small boundary changes can therefore move the system to a different equilibrium branch, making long-horizon credit assignment brittle.
- Cost of backpropagation through iterative solvers. Each state requires iterative solution. Naively differentiating through every inner iteration across a long horizon is memory-intensive and numerically fragile near instabilities.
A canonical example is robotic manipulation of deformable linear objects (DLOs) such as elastic strips, where geometrically nonlinear deformation and multistable equilibria make even seemingly simple shaping tasks highly nontrivial. DLO manipulation serves as the primary testbed throughout this work.
Method
The method has two components. First, proxy-adjoint gradients differentiate through the equilibrium constraint \(\mathcal{G}(\mathbf{x},\mathbf{z})=\mathbf{0}\) via implicit differentiation, computing \(\mathbf{S}^\top\mathbf{v}\) matrix-free through one adjoint linear solve per step, without unrolling the inner Newton–CG iterations. Along the realized branch, the proxy sensitivity dynamics are
\(\dfrac{d\mathbf{x}}{d\lambda} = \mathbf{S}(\lambda)\,f(\mathbf{z}(\lambda),\mathbf{u}(\lambda)), \qquad \mathbf{S} := \partial\mathbf{x}/\partial\mathbf{z},\quad \mathbf{G}_x\mathbf{S} = -\mathbf{G}_z.\)
The backward pass freezes \(\mathbf{S}(\lambda)\) as locally constant on each continuation interval (frozen-tangent approximation), giving a proxy gradient that is first-order accurate in \(\Delta\lambda\) under local regularity. Second, receding-horizon continuation (RHC) re-anchors optimization to the realized equilibrium at the end of each short segment of \(H\) steps, limiting frozen-tangent error accumulation and mitigating unintended basin switches over long horizons.
For the full derivation, discrete backward recursions, pseudocode (Algorithm 1), and the elastic strip energy model, see Section 3 and Appendices A–B of the paper.
Tasks
All three tasks share the same forward simulator. Boundary DoFs define \(\mathbf{z}\); free DoFs \(\mathbf{x}\) are solved implicitly and used to evaluate task losses. The neural controller \(\mathbf{u}_\Theta(\lambda)\) is a lightweight MLP (hidden layers \([64,64]\) or \([64,64,64]\), ReLU activations, tanh-bounded output) taking the scalar \(\lambda\) as input.
- Task 1: Point-to-point node targeting. A selected strip node \(i\) must reach a target position \(\mathbf{p}_i^\star\) at the end of continuation. Loss: \(\mathcal{L}_{\text{pos}} = \tfrac{1}{2}\|\mathbf{p}_i(1) - \mathbf{p}_i^\star\|_2^2\). Both clamped ends are actuated in the transverse (\(x\)) direction; \(\mathbf{u}(\lambda) \in \mathbb{R}^2\). Uses 200 optimizer updates.
- Task 2: Midpoint trajectory tracking. The middle node must follow a prescribed planar path \(\mathbf{p}^{\star}_{\text{mid}}(\lambda)\) (sinusoidal, circular, square-wave, or triangular). Loss: \(\mathcal{L}_{\text{traj}} = \int_0^1 \tfrac{1}{2}\|\mathbf{p}_{\text{mid}}(\lambda) - \mathbf{p}^{\star}_{\text{mid}}(\lambda)\|_2^2\,d\lambda\). Same two-end transverse actuation as Task 1. Uses 500 optimizer updates.
- Task 3: Multistable shape formation. One clamped end is manipulated (\(\mathbf{u}(\lambda) \in \mathbb{R}^3\): \(x,y\) displacement rates and in-plane rotation rate) while the other end is passively clamped. For the same boundary condition, multiple stable equilibria coexist (mirror-symmetric C-shapes, U/M-like shapes). The goal is to reach the target branch, measured by curvature matching: \(\mathcal{L}_\kappa = \int_0^L (\kappa(s;1) - \kappa^\star(s))^2\,ds\). Uses 500 optimizer updates.
Results
We compare four methods — SPSA, iCEM, Adjoint-only (full-horizon adjoint without RHC), and Adjoint+RHC (our full method) — all sharing the same forward equilibrium simulator and controller parameterization \(\mathbf{u}_\Theta(\lambda)\). They differ only in how \(\Theta\) is updated and whether receding-horizon execution is used. CEM results (Appendix E of the paper) follow the same trend.
On Task 1, all methods reduce loss early; both adjoint variants continue improving under the fixed budget while gradient-free baselines plateau. Adjoint-only converges fastest in this short-horizon regime. On Task 2, Adjoint-only succeeds on smooth references but struggles on non-smooth targets where long-horizon error accumulates; Adjoint+RHC achieves consistent convergence across all trajectory types. On Task 3 (the hardest regime), SPSA and iCEM stagnate, Adjoint-only frequently locks into an incorrect branch, and Adjoint+RHC consistently reaches the lowest curvature loss and correctly selects the target equilibrium branch. Real-robot experiments on a 6-DoF arm confirm qualitative sim-to-real transfer under gravity and experimentally identified material parameters.
| Method | Time / update | Memory / update | Task 1 (200 updates) | Task 2 (500 updates) | Task 3 (500 updates) | |||
|---|---|---|---|---|---|---|---|---|
| Time (s) ↓ | Best Loss ↓ | Time (s) ↓ | Best Loss ↓ | Time (s) ↓ | Best Loss ↓ | |||
| SPSA | \(\mathcal{O}(2KC_{\text{eq}})\) | \(\mathcal{O}(n_\Theta)\) | 727.3±108.8 | 2.0e−6±1.8e−6 | 3612.9±1231.9 | 5.4e−3±7.7e−3 | 3967.3±417.9 | 2.8e−5±1.5e−5 |
| iCEM | \(\mathcal{O}(PKC_{\text{eq}})\) | \(\mathcal{O}(Pn_\Theta)\) | 2332.2±126.7 | 1.5e−2±1.5e−2 | 3208.0±13.7 | 9.8e−2±3.6e−2 | 9907.3±59.2 | 1.2e−2±7.3e−3 |
| Adjoint-only | \(\mathcal{O}(K(C_{\text{eq}}+C_{\text{lin}}))\) | \(\mathcal{O}(K(n_x+n_z)+n_\Theta)\) | 176.0±53.1 | 3.2e−7±5.6e−7 | 1111.2±83.1 | 4.3e−4±5.4e−4 | 935.2±164.7 | 8.4e−3±1.2e−2 |
| Adjoint + RHC | \(\mathcal{O}(H(C_{\text{eq}}+C_{\text{lin}}))\) | \(\mathcal{O}(H(n_x+n_z)+n_\Theta)\) | 16.1±2.8 | 2.3e−7±3.0e−7 | 186.9±24.8 | 3.6e−8±3.6e−8 | 50.9±6.1 | 3.8e−8±7.4e−9 |
\(K\): continuation steps in a full-horizon rollout; \(H\): steps per RHC segment; \(P\): iCEM population size; \(C_{\text{eq}}\): cost of one converged equilibrium solve; \(C_{\text{lin}}\): cost of one adjoint/tangent linear solve; \(n_x, n_z, n_\Theta\): dimensions of state, control, and controller parameters. Wall-clock time sums all segment optimizations over the executed trajectory. Mean ± std over four cases.
We additionally validate on a learned DEQ-style implicit equilibrium model trained from experimental force–strain data of a slinky: Neural Control successfully tracks a sinusoidal strain reference with segment losses on the order of \(10^{-8}\)–\(10^{-11}\), confirming the framework applies to any differentiable implicit model beyond analytical simulators.
TODO
Several directions remain open and are planned for future investigation:
- Beyond elastic strips: 2D/3D deformable bodies. Extend the framework to shells and volumetric solids, where equilibrium solves are more expensive.
- Addressing the frozen-tangent approximation near bifurcations. The first-order accuracy guarantee breaks down when \(\mathbf{G}_x\) becomes ill-conditioned near bifurcations or high-curvature branches; improving gradient reliability in these regimes will be investigated in future work.
- Unmodeled non-conservative effects. Friction, hysteresis, and contact are currently unmodeled and limit sim-to-real transfer.
- More diverse manipulation scenarios. Future case studies will include knot tying, cable routing, and multi-arm cooperative manipulation.