AWS Puts Reward Functions at the Center of Nova Forge Training
AWS has outlined how teams can design custom reward functions for multi-turn reinforcement learning with Amazon Nova Forge. The post focuses on the Bring Your Own Orchestration path, where customers run reward logic in their own environment while Nova Forge coordinates rollouts and training. The central message is practical: a reward can look healthy in aggregate while silently teaching the wrong behavior.Why reward design matters in multi-turn RFT
Amazon’s post frames reinforcement fine-tuning as a different customization route from supervised fine-tuning. Instead of learning from curated examples with annotated reasoning paths, RFT scores the model’s own outputs and adjusts weights so higher-reward completions become more likely.That distinction matters more in multi-turn tasks because the model is not being graded on a single answer. It is being scored across a trajectory that may include tool use, code execution, clarifying questions or recovery from mistakes. AWS says Nova Forge uses Group Relative Policy Optimization, or GRPO, where multiple rollouts for the same conversation are ranked by a reward function and training uses the normalized reward advantage within the group. If a reward component produces the same value for every rollout, it creates no useful difference for GRPO to learn from.
How BYOO changes the reward loop
For single-turn RFT, AWS says a reward can be registered as an AWS Lambda function. Multi-turn work can exceed a single Lambda invocation because conversation state and longer-running scoring may go past the 15-minute limit, so the post uses Nova Forge’s Bring Your Own Orchestration capability.Under that approach, teams set Nova Forge to delegate rollouts to a customer-managed environment container, such as one running on Amazon ECS. The container manages the interaction, conversation state, user simulator, code execution and verifier, then returns an aggregate reward score with optional per-component metrics. AWS also notes that Nova Forge offers a serverless multi-turn RL option that is generally available, but the worked example in the post follows the BYOO path.
The coding task example
AWS describes a collaborative-coding experiment built around 500 unique programming tasks and Amazon Nova Lite 2.0. The task is intentionally under-specified: the model sees a brief coding request, while a user simulator privately holds the full specification and reveals details only when the model asks.Each turn allows the model either to ask a clarifying question or commit code. If it asks, the simulator answers and the episode continues. If it commits code, the rollout ends and the reward handler runs the submitted code against hidden unit tests. The intended behavior is clear but not trivial: guessing should usually produce wrong code, while asking before coding should expose the missing requirement and improve the chance of a correct result.
Composite rewards reduce sparse-signal risk
AWS argues that a single scalar reward is often easy to game and that a terminal-only reward may be too sparse for multi-turn training. The post separates reward signals into outcome rewards, behavioral rewards and penalties.Outcome rewards measure whether the final artifact satisfied the goal, such as passing tests. Behavioral rewards measure intermediate actions, such as asking before acting or using the right tool. Penalties discourage failure modes such as guessing, repeating or stalling. In the coding example, AWS says the reward was designed so asking before coding could be credited independently, while immediate guessing was penalized. The implication is that the optimizer needs visible differences between strategies, not just a final pass or fail.
Code execution needs containment
The AWS example includes a correctness component that executes model-generated code against tests. The post treats that output as unvalidated and warns that reinforcement learning encourages exploration, which can produce unexpected behavior.AWS recommends running generated code without credentials or network access, applying resource limits, using a temporary directory and adding verifier protections such as a per-run random sentinel. It also says teams should validate the number of tests actually run against the expected count, so a model cannot dilute scoring with its own trivially passing tests. For live reward environments, those safeguards are presented as required implementation measures rather than optional hardening.
Failure modes can hide behind healthy curves
The most important warning in the post is that reward collapse can remain invisible in aggregate metrics. AWS distinguishes reward hacking, training instability and reward collapse, then focuses on the last category because aggregate reward, loss and completion-length curves may still look normal.In one earlier version of the coding reward, AWS says the asking bonus was gated behind correctness and an efficiency term rewarded shorter conversations. Training collapsed toward guessing on the first turn. Because correctness was near zero on hard tasks, the asking bonus rarely fired, while the shorter-conversation incentive pushed the model toward a degenerate strategy. AWS says a separate issue also appeared when a correctness scorer returned the same value on every rollout, causing the model’s clarifying-question rate to rise from roughly 34 percent to 96 percent while code correctness barely moved.
Instrumentation becomes the operational lesson
AWS’s practical recommendation is to monitor each reward component’s contribution to learning, not just the aggregate score. The post says teams should report each component through metrics, track its mean and within-group standard deviation, and flag components whose variance sits near zero.The same logic applies to debugging transcripts. AWS advises reading examples sorted by the component being tested rather than by total reward, because total reward can hide a dead channel behind active ones. It also recommends ablation: if removing a component changes nothing, the component probably was not doing work. For AI teams building agentic systems, that turns reward design into an observability problem as much as a modeling problem.
Conclusion
The Nova Forge post is less a product announcement than a technical warning about where multi-turn reinforcement learning can fail. AWS is saying that the reward function is the part customers control, and therefore the part most likely to encode subtle mistakes.For teams experimenting with RFT, the takeaway is direct. Multi-turn agents need rewards that separate good and bad strategies within the same prompt group, not only final scores that look reasonable on a chart. Composite rewards, safe execution controls and per-component variance monitoring are the mechanisms AWS highlights for making that signal visible before a training run teaches the wrong behavior.
Sources
Editorial Team - CoinBotLab