Keep your trainer
Own-Trainer RL Rollouts
You run GRPO, PPO, or custom policy updates on your own cluster. Grid serves multi-turn trajectory samples on quoted reserved capacity.
/training/rl-rollouts
Keep your trainer.
Scale the rollouts.
Run rollout sampling on Grid while your team owns the trainer and algorithm. Scope a pilot, then quote the reserved capacity.
Quoted capacity · No public rate card · Not self-serve
What to bring to the pilot.
Read the rollout guide- Checkpoint and trainer
- Candidate checkpoint, precision, framework, algorithm and reward or evaluator. Confirm each method and refresh requirement.
- Sampling workload
- Context and output lengths, samples per prompt, concurrency and the useful sample target.
- Capacity window
- Desired region, start, duration and checkpoint-refresh pattern. The quote confirms resources and effective rates.
- Acceptance
- Compare useful samples, quality, checkpoint freshness, throughput and cost against your baseline.
- 01Your checkpoint
- 02Grid sampling
- 03Your trainer
Choose your RL ownership boundary.
Compare with Agent RLJoint trajectory & verifier loop
Scoped Agent RL
Forward scopes the trajectory dataset, deterministic verifiers, and training loop with your team. Scoped by invitation and quoted per engagement.
| Loop stage | Own-Trainer RL Rollouts | Scoped Agent RL |
|---|---|---|
| 1. Environment & Tool Sandbox | Customer: Executes tool calls, code sandboxes, or environment state transitions inside customer infrastructure. Forward: Serves high-throughput rollout completions (`n` samples per prompt, logprobs, stop tokens) on Forward Grid. | Customer: Defines domain tasks, target tool schemas, and safety boundaries. Forward: Qualifies the agreed environment and access boundary before live sampling. |
| 2. Trajectory Generation & Sampling | Customer: Dispatches rollout batches from VeRL / OpenRLHF / custom trainer and manages episode loops. Forward: Qualifies sampling throughput, checkpoint freshness and latency for the quoted window. | Customer: Provides approved tasks or traces with explicit training rights. Forward: Orchestrates parallel multi-turn trajectories across tool calls, retries, and branch explorations. |
| 3. Verifier, Rubric & Reward Scoring | Customer: Computes scalar rewards, unit-test pass rates, or judge scores inside the customer training loop. Forward: Optionally serves fast judge/reward models or OpenDetector checks via Grid endpoints. | Customer: Approves business acceptance criteria and domain edge-case rules. Forward: Builds and runs deterministic verifiers, citation gates, and step-level process reward harnesses. |
| 4. Gradient Update & Optimizer Step | Customer: Runs backward pass, KL penalty, and optimizer step (GRPO / PPO / GSPO) on customer trainer GPUs. Forward: Runs the qualified sampling service; customer trainer code and optimizer state remain customer-owned. | Customer: Reviews learning curves, held-out benchmark gains, and regression reports. Forward: Executes the post-training optimization loop and checkpoint selection on Forward infrastructure. |
| 5. Weight Sync & Production Serving | Customer: Pushes updated LoRA adapters or checkpoint deltas at each rollout step. Forward: Refreshes rollout workers under the qualified contract. Production serving is a separate accepted deployment. | Customer: Signs off on staging evaluation and promotes the versioned agent policy. Forward: Qualifies export and serving reload, then deploys only the accepted checkpoint under the agreed release scope. |
Capacity and sampling details
Keep the trainer
Your loop stays where it is. Grid is the sampling fleet: OpenAI-compatible completions, sticky sessions for multi-turn trajectories, and a weight-update path we size with you.
Same inference stack
Rollouts run on the factory that already serves Grid. You do not stand up a second engine beside production. Capacity can move between product inference and rollout windows.
Sampler, not full-loop Agent RL
Scoped Agent RL engagements design the verifier environment and run policy updates with you. RL rollouts are the sampling layer underneath for teams that already own their trainer and do not want to migrate it.
Quoted capacity
Rollouts run on reserved capacity, not on anonymous Serverless. We start with a proof of concept, then size regions. There is no public rate card.
Questions
Can I keep my own trainer?
Yes. That is the product. Upload checkpoints to object storage we agree on, sample through Grid, and push weight updates on a schedule we set with you.
How is this different from Agent RL?
In an Agent RL pilot (/training/agent-rl), Forward scopes the trajectory schema, deterministic verifiers, and policy optimization loop with your team. RL rollouts provide sampling and fleet orchestration only when the trainer is already yours.
Is this live self-serve?
No. It is reserved capacity, by conversation. Talk to us to size a proof of concept.