Most labs answer "build" by default. It feels like the right instinct: you control the environment, own the training data, and stay off a vendor's roadmap. That instinct holds in the domains where your team already knows what correct looks like. It breaks for most enterprise workflow domains — which happen to be exactly where agentic AI is being deployed at scale.
The build vs. buy question in RL environment construction isn't a single decision. It's a domain-by-domain judgment, and the answer depends on one thing your engineering headcount doesn't fix: whether anyone on your team understands the work well enough to define correctness for a reward function. That's where the ceiling is. Not compute, not reinforcement learning algorithms, not scaffolding — the domain knowledge behind the reward signal.
Understanding where that ceiling sits, and what it costs to ignore it, is what this piece is about.
Strip away the framing and building a production-grade RL environment requires two things: the infrastructure and the domain knowledge. Your team almost certainly has the first. The second is where the problems start.
The infrastructure layer is tractable. Any competent engineering team can formulate a Markov decision process (MDP), define an action space, run rollouts through a Docker container-based sandbox, and wire up a scaffolding layer that connects your policy to a task sequence. OpenAI Gym and its successors have commoditized this. The plumbing is not the problem.
What isn't commoditized is everything built on top of the plumbing. Reward function design — translating "correct behavior in this workflow" into a training signal — requires someone who knows what correct actually looks like. Not in documentation. In the workflow itself, including the edge cases, the places where experienced practitioners disagree, and the failure modes the model needs to learn to avoid. An action space defined from documentation is an approximation. An action space defined by someone who has navigated the workflow is grounded.
Verifiable rewards raise that bar further. RLVR-based training requires a reward signal where correctness can be independently confirmed — the code executes and passes the test suite, the math answer is numerically right, the legal clause is jurisdictionally accurate. Writing a verifier for a new domain requires domain expertise the ML engineering team doesn't have. A verifier that wasn't validated against expert judgment trains your model toward the wrong target. You won't see that in your rollout metrics. You'll see it when the model hits production.
Reward hacking is the downstream cost of that imprecision — and why the grader is where most RL environments fail in practice. When a training agent finds a trajectory that satisfies the verifier without doing the actual work, the reward signal is compromised from that point forward. Sparse rewards — common in long-horizon enterprise workflows where intermediate steps aren't naturally verifiable — make it easier to exploit and harder to design against. The cumulative reward curve from a well-calibrated environment tells a different story from one where the reward function was written quickly: progressive improvement over training cycles versus early saturation and flat production performance.
Your team has a genuine advantage in the domains where ML engineers are also practitioners. Coding agents are the clearest case. The people building the environment understand software development well enough to design verifiable rewards that reflect genuine code quality — not just test passage, but structure and edge-case handling. Mathematical reasoning environments, the kind that drive performance on benchmarks like GSM8K, sit in the same category. The domain is the one your team lives in. Failure modes are familiar. Verifiers are calibrated against genuine expertise because the expertise is already in the room.
Computer use environments — UI gyms where agents navigate browsers, interact with applications, and complete multi-step tool use sequences — sit in a middle ground. The technical layer is tractable in-house. What your team may not understand is the specific enterprise workflow the agent is learning: the insurance adjudication process, the fund accounting reconciliation, the contract review sequence. Build the UI gym in-house. Reconsider the domain it's simulating.
Simulation environments for abstract tasks — grid worlds, constrained planning problems, synthetic multi-step games — are also solidly in the build column. No specialist domain knowledge required. These have real value for studying agent capabilities and generalization, and for stress-testing new training algorithms before you scale them to real-world domains. The constraint is that what you learn in a grid world doesn't transfer cleanly to the enterprise workflow you care about. They're useful research environments. They're not production AI training environments for the workflows agentic AI will actually be deployed in.
The through-line: build where your team already knows what correct looks like. The rest is the question.
The domains generating the most enterprise AI value — financial analysis, legal and compliance operations, healthcare workflows, enterprise software automation — are precisely the domains where ML engineering skill doesn't overlap with domain expertise. Your reinforcement learning team knows how to build coding agents and the AI agents that handle technical tasks. It doesn't know what a good fund accounting reconciliation looks like. It can write a verifier. It cannot write one that a senior finance controller would trust.
This is the ceiling. The reward function for healthcare prior authorization is only as good as the medical and billing expertise behind it. The action space for M&A due diligence document review is only as good as the legal knowledge that defined what correct extraction means across each document type. LLMs trained against environments where that expertise was approximated don't just perform worse on benchmarks — they perform inconsistently in ways that are hard to diagnose, because the reward signal was shaped by a guess rather than ground truth. LLM performance in enterprise workflow domains traces directly back to the domain grounding in the environments they were trained on.
Rollout metrics from a weakly grounded environment look fine. You see improvement curves, pass rates climbing, benchmark scores rising. The model learns to satisfy the verifier, which was built from documentation and engineering intuition rather than from the people who actually do the work. That gap shows up in production, not in training.
Labs that have tried to hire past this constraint find the same thing every time. The intersection of senior domain expertise and RL environment construction skill is small outside of technical fields. The candidate either knows the domain and needs to learn environment construction, or knows environment construction and approximates the domain. Neither produces a production-grade environment on the first build cycle. Neither produces one you can maintain as model capabilities improve.
The shift from RLHF and supervised fine-tuning toward GRPO and RLVR reflects a broader recognition that pre-training alone is no longer sufficient for frontier capability gains — and it changes what your environments need to do.
GRPO-based training generates rollouts in groups and computes relative rewards within each group, using within-group variance as the learning signal. That efficiency depends on reward signal calibration. If rewards cluster too tightly across rollouts, the within-group variance collapses and the training signal degrades. If they spread too wide, you're introducing noise that destabilizes learning. Getting that calibration right requires understanding the difficulty distribution of your task set — which requires understanding the domain.
RLVR makes verifiability non-negotiable. If the reward signal can't be independently confirmed, you're back to the approximation problem. This is tractable for math and code — answers are checkable. It's tractable for well-structured enterprise workflows where correctness criteria can be specified precisely. It isn't tractable for domains where "correct" requires expert judgment unless you've built that judgment into your verification layer.
Reward models — learned verifiers trained on expert preferences — don't escape this requirement. A reward model for a new enterprise domain means expert preference data for that domain. That puts you at the same constraint: the expert supply chain. Whether you're building rule-based verifiers or training reward models, the quality ceiling is set by the domain expertise behind the verification layer.
Machine learning infrastructure scales with spend. RL environment quality scales with the domain expertise you can access. These are different problems, and the second one doesn't get easier as your AI training budget grows.
Not infrastructure. Your team can build the infrastructure. What you're buying is the environments hub built around genuine domain expertise — tasks designed by senior practitioners who understand the target workflow, reward functions and verifiable rewards calibrated against expert judgment rather than written from first principles, and rollout pass rates calibrated against actual model behavior.
The pass rate calibration matters more than it looks. A production-grade environment for AI training targets a minimum rollout pass rate around 2–3%, calibrated against observed model behavior, not written from intuition about task difficulty. Too low and the model isn't getting positive signal to learn from. Too high early in training and the environment saturates before model capabilities have developed. Getting that calibration right requires domain expertise and iterative observation — it can't be set once at build time and left alone.
It also means environments that evolve. A static environment has a ceiling. As training agents improve and approach saturation on the existing task distribution, a well-maintained environment introduces new complexity, updates tasks to reflect distribution shift in the real world, and recalibrates pass rates as the model's capability profile changes. That ongoing work requires sustained domain expert engagement — not a build handoff followed by six months of silence.
The vendors doing this correctly separate themselves on one question: who built the reward functions and verifiers, and what was their role in the calibration process? Understanding how domain expertise actually gets translated into a production-grade training environment is the due diligence step most lab buyers skip until after their first training cycle.
The moat isn't the scaffolding infrastructure, the rollout pipeline, or the sandboxing layer. Those are solved problems. The moat is the expert network — the vetted senior professionals across enterprise workflow domains who understand the work well enough to define correctness for a reward function — and the process that connects that expertise to environment construction at scale. That's what you can't replicate by standing up another agentic AI team internally.
The decision maps onto a few practical questions you can answer about each domain on your roadmap.
Does your team have genuine practitioners in the target domain? If the engineers building the environment have done the work themselves — written the code, solved the math, navigated the enterprise software in a professional context — build. The expertise required to design verifiable rewards and calibrate rollouts is already present. If not, the honest answer is that you're building approximations.
Is verifiable reward design tractable without domain knowledge? Code passes or fails tests. Math answers are numerically correct or not. For these, write the verifier in-house. For domains where "correct" requires expert judgment — most enterprise workflow domains — that judgment needs to come from somewhere. Building it in-house means hiring for it or accepting approximations. Outsourcing it means buying the expert supply chain.
How central is this domain to your commercial roadmap? An exclusive environment in a domain where you have a strategic interest in being the best model gives you a capability moat. A non-exclusive environment where coverage matters more than competitive differentiation doesn't require the same investment. Most labs end up with both: in-house environments for the technical domains they know, outsourced environments for the enterprise workflow domains where the next wave of agentic AI value sits.
What is your team's current build capacity? An in-house team running at capacity on coding and math environments isn't the right team to simultaneously build healthcare authorization and fund accounting environments. The opportunity cost of misallocated engineering effort compounds over training cycles. The decision isn't just "can we build this?" It's "what don't we build if we spend that capacity here?"
The build vs. buy decision in RL environments is the same decision it is everywhere else in production AI infrastructure. Build what you can build well, with genuine expertise behind it. Buy what requires an expert supply chain you don't have. The labs that get this right compound the capability advantage; the ones that build approximations across every domain find out at the benchmark stage.
Ready to build RL environments that train against real-world domain expertise? Get started with Invisible.
Supervised fine-tuning adjusts a model by training on labeled examples of correct outputs. Building an RL environment means creating the training arena itself — defining the action space, designing reward signals, and calibrating verifiers around what correct behavior looks like in the domain. The two approaches complement each other: many labs run SFT to establish baseline behavior, then use RL environments to push capability further.
In-house builds give you full control over training data, which matters for proprietary workflows and regulated industries. That is a real advantage. The question is whether it outweighs the domain expertise constraint. For most enterprise workflow domains, the answer is to outsource environment construction while keeping training data on-prem — you get verifiable rewards calibrated against genuine domain knowledge without moving sensitive data off your infrastructure.
Reward hacking occurs when a training agent satisfies the verifier's conditions without completing the intended task — it found a shortcut the reward function didn't anticipate. It signals that verifiable rewards aren't capturing genuine correctness in the domain. High-quality environments prevent it through precise reward design, expert-validated verifiers, and ongoing calibration. An environment that produced a reward-hacking model was built with insufficient domain grounding.
Ask about the expert engagement model: who designed the reward functions, what their professional background is, and how they validated the verifiers. Ask how rollouts are calibrated against actual model behavior and what the update process looks like after six months of training. Vendors whose answer to that last question is rebuild — not recalibrate — are selling a different product from vendors whose environment evolves with model capability.
Almost always. Coding agents and mathematical reasoning are the domains where ML engineering skill and domain expertise overlap directly — your engineers are the domain experts. Verifiable rewards are tractable: code passes or fails, math answers are correct or not. Where in-house coding environments fall short is simulating a specific enterprise software context or codebase domain your team doesn't work in day to day.
A minimum pass rate around 2-3% on rollouts is the production standard for a calibrated environment. Below that, the model isn't getting enough positive signal. Higher than that early in training means tasks are too easy and the environment will saturate before model capabilities develop. The pass rate must be calibrated against observed model behavior — set from intuition about task difficulty, it produces a flat training curve.
