Agent Learning / Reward-based Agent RL
Turn task outcomes, expert judgments and calibrated agent feedback into reward signals. Train eligible agent policies in your environment, compare them on independent evaluations and deploy the versions that meet your acceptance criteria.
Agent Learning can use an admitted customer harness without an enterprise Agentic OS deployment. Environment, model, data rights and acceptance are qualified separately.
Scoped by invitation · Custom environment & verifier contract · Quoted per engagement
Continuous improvement / Controlled learning
Build a loop from business execution to measured improvement. Agent feedback helps propose and score changes. Verified outcomes and independent evaluation determine whether they work.
A defined job and acceptance criteria.
Authorized context, tools and action limits.
Outcomes, expert review and calibrated agent judgments.
Authorized signals update an eligible policy.
Held-out quality, policy, latency and complete cost.
An accountable owner or pre-authorized policy governs promotion and rollback.
Illustrative learning lifecycle. Execution permission does not grant training permission. Training changes eligible model or policy weights; prompt and tool changes have their own evaluation path. Learning cannot expand agent permissions. Deployment, training and release gates are agreed for each engagement.
01 / Learning methods
Demonstrations support SFT. Preferences support DPO or ORPO. Executable environments and scorable outcomes support RL. Establish a baseline, then choose or combine methods for your task. Prior SFT or DPO is not required for an eligible RL engagement.
Establish a measured baseline with context, retrieval, tools and approval policies. Evaluate configuration changes alongside any eligible training method.
Required inputs
20–50 representative test cases + explicit acceptance rubric.
Output artifact
System instructions, retrieval chunking, tool schemas, approval gates, and citation rules (weights unchanged).
Use when curated demonstrations, including multi-turn conversations and tool traces, can teach the required behavior on an eligible checkpoint.
Required inputs
Curated demonstrations with data rights, representative task coverage and an independent evaluation set.
Output artifact
An eligible adapter or checkpoint is optimized against approved demonstrations; these can cover multi-step tasks.
Use when authorized preferred and rejected responses or trajectories express the desired behavior.
Required inputs
Curated preferred and rejected responses or trajectories with data rights and a held-out evaluation set.
Output artifact
An eligible adapter or checkpoint is optimized from the approved preference pairs.
Use when an executable environment and scorable task outcomes support reward-based optimization of multi-step decisions. Prior SFT or DPO is not required.
Required inputs
Executable environments, eligible checkpoints, authorized trajectories, outcome checks and calibrated expert or agent feedback, plus a measured baseline.
Output artifact
End-to-end policy optimization across multi-turn tool calls, environment state transitions, and final task verification.
02 / Contracts
An Agent RL pilot scopes four concrete interfaces before training compute starts: the trajectory log format, the deterministic verifier suite, the rollout loop, and the promotion gate.
Collect multi-turn execution traces from Forward HIVE or your own harness. Each trajectory captures prompt context, intermediate tool invocations, sandbox outputs, approval checkpoints, and terminal deliverables.
Use observable outcome checks and expert judgments to calibrate agent critiques and rubric scores. Measure agreement, audit disagreements and check for reward exploitation. An agent judging its own result is not an independent acceptance test.
Run the full trajectory sampling and policy optimization loop with Forward, or keep your own trainer (GRPO, PPO, REINFORCE) and use Grid purely as the high-concurrency rollout fleet.
Every candidate checkpoint is benchmarked against held-out tasks for task completion, tool-call precision, latency, and cost per accepted run before promotion to production serving.
03 / Ownership
Choose whether Forward operates the trajectory optimization loop with your team, or whether your ML engineers keep the trainer and use Grid strictly for high-concurrency rollout sampling.
| Loop stage | Agent RL (Scoped with Forward) | Own-Trainer RL Rollouts |
|---|---|---|
| 1. Environment & Tool Sandbox | Customer: Defines domain tasks, target tool schemas, and safety boundaries. Forward: Qualifies the agreed environment and access boundary before live sampling. | Customer: Executes tool calls, code sandboxes, or environment state transitions inside customer infrastructure. Forward: Serves high-throughput rollout completions (`n` samples per prompt, logprobs, stop tokens) on Forward Grid. |
| 2. Trajectory Generation & Sampling | Customer: Provides approved tasks or traces with explicit training rights. Forward: Orchestrates parallel multi-turn trajectories across tool calls, retries, and branch explorations. | Customer: Dispatches rollout batches from VeRL / OpenRLHF / custom trainer and manages episode loops. Forward: Qualifies sampling throughput, checkpoint freshness and latency for the quoted window. |
| 3. Verifier, Rubric & Reward Scoring | Customer: Approves business acceptance criteria and domain edge-case rules. Forward: Builds and runs deterministic verifiers, citation gates, and step-level process reward harnesses. | Customer: Computes scalar rewards, unit-test pass rates, or judge scores inside the customer training loop. Forward: Optionally serves fast judge/reward models or OpenDetector checks via Grid endpoints. |
| 4. Gradient Update & Optimizer Step | Customer: Reviews learning curves, held-out benchmark gains, and regression reports. Forward: Executes the post-training optimization loop and checkpoint selection on Forward infrastructure. | Customer: Runs backward pass, KL penalty, and optimizer step (GRPO / PPO / GSPO) on customer trainer GPUs. Forward: Runs the qualified sampling service; customer trainer code and optimizer state remain customer-owned. |
| 5. Weight Sync & Production Serving | Customer: Signs off on staging evaluation and promotes the versioned agent policy. Forward: Qualifies export and serving reload, then deploys only the accepted checkpoint under the agreed release scope. | Customer: Pushes updated LoRA adapters or checkpoint deltas at each rollout step. Forward: Refreshes rollout workers under the qualified contract. Production serving is a separate accepted deployment. |
A qualified training run updates eligible model or agent policy weights from rewards across tool and environment interactions. Execution logs, memory and prompt edits alone are not reinforcement learning. Checkpoint rights, model license and training interfaces must permit the update; calling a closed model API does not provide access to its weights.
Authorized feedback can feed a recurring improvement pipeline. Independent evaluation and the agreed release policy govern promotion and rollback. Governance and access policies stay outside reward optimization. Execution access does not authorize training, and learned behavior cannot expand permissions.
Choose based on the feedback and task. Use SFT for demonstrations, DPO or ORPO for preferences, and RL for an executable environment with scorable outcomes. SFT can also teach multi-step tool use. There is no required SFT-to-DPO-to-RL sequence; compare methods against a strong baseline and held-out acceptance criteria.
No. Multi-step reinforcement learning requires a verified environment and reward contract. Every Agent RL engagement is scoped by invitation and quoted around your checkpoint, environment harness, and serving target.
Yes. If your ML team already runs GRPO, PPO, or custom RL loops, use Own-Trainer RL Rollouts (/training/rl-rollouts) for reserved high-concurrency trajectory sampling and weight synchronization.
Scope the trajectory dataset, reward environment, candidate checkpoint, and serving plan with our team.