Agent RL
Reward-based training from verified task outcomes, expert judgments and calibrated agent feedback.
Agent RL trains models on multi-step tool-use trajectories, state recovery, and long-horizon workflows using deterministic verifiers and calibrated rubrics.
SFT can also learn multi-turn conversations and tool-use traces. Agent RL differs through reward-based policy optimization across an execution trace: which tool to invoke, how to recover from a malformed query or sandbox error, and when to request human approval.
Choose the learning method#
Choose methods based on the task, available feedback and checkpoint eligibility. Establish a measured baseline and independent evaluation. SFT and DPO are not prerequisites for RL; methods can be combined.
| Input | Method | When to use | Output artifact |
|---|---|---|---|
| Baseline | Context & Harness Engineering | Instructions, tool schemas, retrieval, and verification gates solve the task without weight updates. | Versioned agent harness + eval suite on Agentic OS |
| Demonstrations | Supervised Fine-Tuning (SFT / LoRA) | You have strong reference outputs or tool traces and want lower latency or higher format reliability. | Private LoRA adapter or merged checkpoint on Training |
| Preferences | Preference Optimization (DPO / ORPO) | You have paired preferred vs. rejected completions for tone, structure, or policy adherence. | Preference-aligned checkpoint on Training |
| Rewards | Trajectory Agent RL | The task requires multi-step tool use, recovery from intermediate errors, and optimization against a verifiable outcome reward. | RL-optimized policy checkpoint on Agent RL |
Environment, trajectory, and verifier contracts#
An Agent RL pilot scopes four interfaces before training compute begins:
- Trajectory schema: Multi-turn execution logs from Agentic OS (Forward HIVE) or your own harness, including prompt state, tool calls, tool responses, approval interrupts, and final deliverables.
- Deterministic verifiers: Code execution checks, JSON schema validators, citation verification gates, and state-diff assertions paired with rubric evaluators.
- Rollout loop ownership: Decide whether Forward operates the policy optimization loop with your team (Scoped Agent RL) or whether your ML team keeps its own trainer and uses Grid strictly for high-concurrency sampling (Own-Trainer RL Rollouts).
- Promotion gate: Held-out task completion rate, tool-call precision, latency, and cost per accepted run before promoting the checkpoint to Grid Dedicated or Private serving.
Evidence before a training commitment#
Start with a fixed held-out evaluation and a strong baseline: improve prompts, retrieval and tools; compare a capable frontier API, an untuned open checkpoint and a supervised checkpoint where suitable. Apply the same tools, data access, task set and scoring rules. Keep evaluation tasks out of training and reward development.
Measure accepted task completion, policy violations, unsupported citations, tool failures, latency and total cost per accepted task. Include training amortization, verifier and rollout costs. Use independent scoring to check for reward exploitation and regressions.
A software integration or CPU smoke test does not demonstrate GPU training quality or an improvement in customer outcomes. A scoped pilot needs compatible checkpoints, rights to traces and tool outputs, a verifiable reward, compute availability and an explicit acceptance decision. Export and serving compatibility are reviewed separately. Own-trainer sampling is an operating model, not another adaptation method.
Agent feedback, rewards and real policy updates#
Agent Learning uses authorized task outcomes, expert judgments and calibrated agent critiques or scores to construct rewards for agents interacting with tools and environments. Calibrate agent judges against expert labels and observable outcomes. Review disagreement and reward exploitation; a self-score is not independent evidence.
A training run updates eligible model or policy weights. Retaining memory, changing prompts or exporting traces alone does not constitute reinforcement learning. Checkpoint licenses, training APIs and the action-token representation must permit the intended update. Calling a closed model API does not grant access to its weights.
For on-policy training, qualify the exact model inputs, sampled action tokens, masks, tool observations, terminal status, policy version and required sampling metadata. Reconstructed narrative logs cannot substitute for the trainer's actual episode contract.
Execution permission and training permission are separate. Confirm data rights, tenant boundaries, retention, permitted feedback use and compute budget before a learning run. Keep governance and authorization outside the reward objective. Learning cannot broaden agent permissions.
Controlled release#
Freeze a held-out evaluation outside reward development. Compare accepted outcomes, policy violations, recovery, latency and complete cost, including rollout, verifier and training amortization. Record checkpoint lineage and export/reload evidence. The accountable release owner approves promotion and rollback under the agreed policy. Continuous improvement does not promise a gain from every training run.
Agent governance describes operating and release boundaries. Forward-deployed delivery covers integration and operating ownership. Agent Learning is also available as an independently scoped engagement for qualified ML teams.