Build vs Buy: When Does a Custom API Orchestration Layer Make Sense?

The build vs buy decision for API orchestration infrastructure comes up in almost every engineering organisation that has graduated from a monolith to multiple services. The conversation usually starts with a specific pain point: frontend teams complaining about waterfall API calls, a checkout flow that times out because five upstream services are called sequentially, or an on-call rotation drowning in cross-service debugging. Someone proposes a BFF layer or API gateway. And then the question surfaces: do we build it, or do we use a platform?

The instinct to build is understandable. Engineering teams are good at building things. The problem space seems tractable — it is just HTTP calls, after all. But the gap between a prototype that fans out to three services and a production system that handles thousands of requests per second with consistent sub-100ms latency, complete observability, credential rotation, configurable retry policies, schema validation, partial failure handling, and alerting is enormous. And that gap is what this post is about.

What You Are Actually Building When You Build It Yourself

The conversation about building an API orchestration layer often focuses on the happy path: receive a request, fan out to upstream services, merge responses, return a result. In a weekend prototype, this can be done in a few hundred lines of code. What that prototype does not include — and what takes months to build and years to maintain — is the production infrastructure around the happy path.

Here is what a production-grade orchestration layer actually requires:

• Connection pooling and keep-alive management: Opening a new TCP connection for every upstream call adds 20–100ms of overhead. A production orchestrator maintains persistent connection pools per upstream service with configurable pool sizes, idle timeouts, and health checks.
• Configurable retry logic with backoff strategies: Exponential backoff, jitter, maximum retry counts, and per-status-code retry policies. Without this, retries from a high-traffic orchestrator can create thundering herd problems that amplify failures rather than absorbing them.
• Circuit breaking: Detecting when an upstream service is unhealthy and stopping calls to it until it recovers, rather than continuing to time out on every request.
• Response caching with TTL management: A cache layer with per-route TTL configuration, cache invalidation hooks, stale-while-revalidate semantics, and cache storage that is consistent across multiple orchestrator instances.
• Credential management and secret rotation: Storing API keys, OAuth tokens, and service account credentials securely, with automatic rotation and no credential leakage into logs.
• Request and response schema validation: Validating inbound requests against expected schemas and validating upstream responses before transforming them, so schema drift in upstream services surfaces as a clear error rather than silent data corruption.
• Distributed tracing and execution logs: Producing structured execution traces for every request — which upstream services were called, in what order, how long each took, what was returned — in a format that integrates with your observability stack.
• Alerting and on-call integration: Error rate thresholds, latency SLO alerts, and integration with PagerDuty, OpsGenie, or your on-call tooling.
• Horizontal scaling and high availability: The orchestration layer becomes a critical path component. It needs redundant deployment, graceful shutdown, load balancing, and health check endpoints.
• A workflow authoring interface: Some mechanism for teams to define, test, and deploy new orchestration workflows without modifying the core orchestrator code and triggering a full deployment cycle.

None of these are optional in production. And each one is a non-trivial engineering problem. Teams that start with the happy-path prototype typically spend 6–12 months building the production infrastructure before the orchestration layer is reliable enough to put on the critical path.

The True Total Cost of Ownership

TCO for a build decision has three components: initial build cost, ongoing maintenance cost, and opportunity cost. The initial build cost is what most teams calculate. The other two are where build decisions typically surprise.

Initial Build Cost

A realistic estimate for building a production-grade orchestration layer from scratch, using a senior engineering team that has done it before, is 3–6 months of 2–3 senior engineer time. This covers the happy path, the production infrastructure listed above, basic observability, and enough testing to deploy with confidence. At a fully-loaded cost of $200–350k per senior engineer per year, that is $100–525k in initial investment before a single business workflow runs on it.

Teams that underestimate typically discover the hidden work during the build: the retry logic that creates thundering herds in load testing, the connection pool that leaks under sustained traffic, the cache that serves stale data after upstream schema changes, the credential rotation that requires a deployment to take effect. These are not edge cases — they are the normal engineering work of building reliable infrastructure.

Ongoing Maintenance Cost

Maintenance cost is often the most underestimated component. A production orchestration layer requires ongoing investment in:

A realistic ongoing maintenance allocation for a production orchestration layer is 20–30% of one senior engineer's time — roughly one day per week — once it reaches a stable state. In the first year, when the layer is still maturing, this allocation is often higher.

Opportunity Cost

Opportunity cost is the hardest component to quantify but often the most significant. When 2–3 senior engineers spend 6 months building orchestration infrastructure, they are not building product features, improving customer experience, or reducing technical debt in your core business logic. If those engineers are your best backend engineers — which they typically are, because infrastructure work requires deep expertise — the opportunity cost is even higher.

For most companies, the right question is not whether they could build an orchestration layer — they can. The question is whether building it is the best use of senior engineering capacity relative to what else those engineers could build in the same time.

When Building Makes Sense

There are scenarios where building a custom orchestration layer is genuinely the right choice. Understanding these scenarios helps teams make the decision clearly rather than defaulting to build out of habit or buy out of convenience.

• Highly specialised protocol requirements: If your upstream services use uncommon protocols (binary formats, proprietary messaging systems, extremely custom auth schemes) that no managed platform supports, building a custom layer may be the only option.
• Regulatory data residency constraints: Some industries have strict requirements about where data is processed and stored. If managed platforms cannot meet your data residency requirements, a self-hosted solution may be required.
• Your orchestration IS the product: If orchestration logic is your core business differentiator — you are building a platform that sells orchestration capabilities to others — then building it is investing in your product, not infrastructure.
• Extreme scale with highly specific performance profiles: At very high traffic volumes (millions of requests per second) with extremely specific latency requirements, custom-built infrastructure can be optimised for your exact traffic shape in ways that generalised platforms cannot.

If your situation does not match one of these scenarios, the case for building is usually based on habit, the attraction of full control, or an underestimate of the real build cost — not a genuine technical or business requirement.

When Buying Makes Sense

Buying a managed orchestration platform makes sense in the much larger set of scenarios where the orchestration layer is necessary infrastructure, not a competitive differentiator. This includes:

• Speed to production: A managed platform can have your first orchestration workflow in production in days, not months. The production infrastructure — connection pooling, retries, caching, observability, credential management — is already built and battle-tested.
• Operational burden transfer: Infrastructure reliability, security patches, dependency updates, and scaling become the platform vendor's problem. Your on-call rotation covers workflows and business logic, not the orchestration infrastructure itself.
• Predictable cost model: Platform pricing is typically volume-based and predictable. The hidden maintenance costs of a build decision are harder to forecast and often grow unexpectedly as traffic scales.
• Evolving requirements: Managed platforms continuously add capabilities — new connector types, new transformation functions, new observability features. Building custom infrastructure means you only get what you build.
• Team focus on product: Every sprint your engineers spend on orchestration infrastructure is a sprint not spent on the product that generates revenue. For most companies, product velocity is the scarcer resource.

A Practical Decision Framework

Use this framework to structure the build vs buy conversation with your team and stakeholders. Answer each question honestly — the answers usually point clearly to one choice.

The “Build Then Buy” Trap

One of the most common and costly patterns in engineering organisations is building a custom solution, running it for 18–24 months, accumulating maintenance debt and operational burden, and then switching to a managed platform — paying the build cost and the platform cost rather than just the platform cost.

This happens because the initial build decision was made without an honest TCO analysis. The team underestimated maintenance cost and opportunity cost. By the time the full cost is clear, the custom solution is on the critical path with real traffic running through it, and the migration to a managed platform requires a careful, staged cutover.

The way to avoid this trap is to be rigorous about the build vs buy analysis before writing the first line of code. If the honest analysis points to buy, commit to it early — the earlier you start on a managed platform, the less migration cost you accumulate.

Evaluating Managed Orchestration Platforms

If the analysis points to buying, the next question is which platform. The capabilities that differentiate orchestration platforms in production are not always the ones that feature prominently in marketing materials. Here is what to evaluate in a proof of concept:

• Latency overhead: How much latency does the orchestration layer add above the raw upstream call time? A well-implemented platform adds 5–15ms. Platforms with high overhead negate some of the parallel execution benefit.
• Execution observability: Can you see a complete execution trace for every request? Can you filter by upstream service, by error type, by latency percentile? Debugging ability in production is more important than any feature in development.
• Workflow authoring experience: How quickly can a new engineer define and deploy a workflow? How are workflows tested before production? Is there a mock/sandbox mode?
• Connector ecosystem: Does the platform have pre-built connectors for the services you use most often? Pre-built connectors for Stripe, Salesforce, Shopify, and major commerce platforms can save weeks of integration work.
• Failure handling configuration: How granular is the retry and failure policy configuration? Can you set different policies per upstream service? Per status code? Per workflow step?
• Security model: How are upstream credentials stored and rotated? Is there credential masking in logs? What is the authentication model for the orchestration layer itself?

The platform that performs best on these criteria in your specific context — with your upstream services, your traffic patterns, and your team's workflow — is the right choice. A proof of concept that exercises your real use cases is worth more than any feature comparison spreadsheet.