Applied artificial intelligence

Applied AI, engineered for production.

Koi Consulting delivers inference engineering, model adaptation, routing, retrieval and machine learning beyond language models, together with the governed runtime that supports them. As a principal co-founder of Orchard AI, we deploy the platform within your own infrastructure.

The platform

The operating system for an enterprise that runs on AI.

Sovereign AI Infrastructure

Orchard AI provides a governed runtime in which models, memory, tools and agents are components that your organization owns and can replace or combine as its needs change. The platform runs on your infrastructure, operates under your policies and builds knowledge of your business over time.

Corporate rulesets are enforced at the point of action, without waiting for a software release. Solutions that your organization has already developed become available to everyone in it. Frontier models and local models operate within a single platform, so data that must remain on your premises can do so.

An organization that owns its AI platform builds a lasting asset as its capability grows. This conviction led us to co-found Orchard AI and to develop the platform ourselves.

Research & engineering

Engineering depth across the AI stack

Koi Consulting’s AI practice extends well beyond the integration of a vendor API. Our engineers work on the layers that determine how AI systems perform in production, including inference, model adaptation, routing, retrieval and evaluation, as well as the machine learning methods that are better suited than language models to many enterprise problems.

Because we develop the platform these capabilities run on, we are able to adapt it when a client requires something that is not yet commercially available.

01

Inference engineering

The throughput of autoregressive decoding is limited by memory bandwidth more than by compute. We address this with speculative decoding, in which candidate tokens are drafted at low cost, verified in a single pass of the target model and retained when they are accepted.

Our implementation is a multi-head arrangement derived from the published Medusa method. Additional decoding heads predict several positions ahead, candidate continuations are arranged as a tree, and verification takes place in one batched forward pass under a modified attention mask. The published method is a starting point. The achievable speed-up depends on the number and depth of heads relative to the target model, the pruning of the candidate tree, the acceptance criterion, and the training of the heads on the workload the deployment serves. That tuning is proprietary to Koi Consulting.

We measure performance in accepted tokens per forward pass, which is a more reliable indicator than raw tokens per second.

02

Parameter-efficient model adaptation

Full fine-tuning is costly, can reduce the general capability of the base model and becomes outdated quickly. We use parameter-efficient adaptation layers for each user, team or domain, which are applied to the base model when it is loaded.

Each layer is versioned, inspectable and reversible. A trading desk, a claims unit and a platform team can each work with a system adapted to their vocabulary, risk posture and tools, without separate deployments or separate training budgets. When the base model is replaced, the adaptation layers are refitted. Adaptation remains an engineering artifact under change control, and it can be audited.

Every adaptation can be compared with its predecessor, rolled back and documented.

03

Model routing

We apply mixture-of-experts selection at the request boundary. A classifier selects the smallest model that is sufficient for each request, whether local or frontier, based on cost, latency, sensitivity and jurisdiction.

A significant share of enterprise AI spending goes to frontier models answering questions that a smaller model running on the organization’s own hardware could answer correctly. Routing recovers that cost while preserving capability for complex work. It is also the point at which data sovereignty is enforced, because a request involving regulated material is routed within the perimeter as a matter of policy and without a code change.

In the estates we have instrumented, approximately half of frontier model spending did not require a frontier model.

04

Structured retrieval

Similarity search over text passages returns content that resembles a question, which is not always the content that answers it. In a codebase, this approach often performs poorly.

We operate a structural index alongside the embedding store. The index is a graph of code, documents and decisions that is queried by relationship as well as by keyword or distance. Graph traversal answers dependency questions accurately because it follows actual relationships, while embeddings handle less precise queries. The combination maintains retrieval quality as the body of content grows.

Relationship queries outperform similarity queries on structured content such as source code.

05

Machine learning beyond language models

Language models are not the most suitable tool for a large share of enterprise problems. Much of our applied work uses other methods that are more accurate and more economical for the task.

This work includes time-series forecasting and anomaly detection on operational and financial telemetry, gradient-boosted trees for tabular classification, computer vision and optical character recognition for document and forensic work, and signal processing across sensor networks, including indoor location using three-axis Delaunay triangulation on commodity hardware. It also includes graph algorithms for dependency and fraud analysis, and federated approaches where training data cannot legally be centralized.

We establish the purpose of the model before selecting the model, which narrows the options considerably.

06

Local inference

Sovereign deployment depends on hardware as much as on policy. Achieving useful inference on on-premises equipment requires careful work on quantization, key-value cache management, batching strategy and measurement.

We quantize models and measure the resulting effect on quality, size the cache for actual concurrency, and tune batching to the request pattern of the deployment. We then compare the results with a frontier baseline on the client’s own tasks. In some cases local inference meets the requirement. In others a portion of requests must be routed externally, and policy is adjusted to reflect that. Both conclusions are based on measurement.

The effect of quantization varies by task, so we measure it on the client’s own workload.

07

Evaluation and regression testing

Improvement depends on measurement, and public benchmarks do not reflect an individual organization’s work. Each deployment receives an evaluation harness built on the client’s own tasks and its own definition of a correct result.

Regression suites run whenever a model, adapter, routing rule or prompt changes, because each of these can alter behavior without warning. The harness identifies behavioral drift before users encounter it. This discipline is among the most valuable elements of an enterprise AI program, and it is frequently overlooked.

A regression suite gives every model change a clear signal for acceptance or rollback.

We reproduce published research

We read, implement and measure published methods against our own workloads before applying them to a client system. Results reported for one model family do not always transfer to another, and reproduction is the reliable way to confirm them.

We operate the platform we research on

Our research is applied directly to Orchard AI, which we co-founded and operate. Research and production share the same codebase, the same runtime and the same environments.

We report negative results

Not every technique improves a given workload. We report those findings as well, so that clients can direct their investment to the methods that will benefit them.

Our principles

Six principles for enterprise AI

01

Sovereign deployment

The runtime, the memory store, the orchestration layer and the knowledge graph are deployed inside your perimeter. Model weights, embeddings, recall indices and audit trails remain within your infrastructure, so your proprietary context stays under your control.

02

Persistent learning

Every correction, constraint and lesson is committed to a persistent recall layer and enforced by policy at the agent loop. As a result, the system does not repeat an error once it has been identified and corrected.

03

Governance at the speed of policy

Rules are authored centrally, versioned and enforced deterministically at the point of action. A policy decision takes effect when it is published, without waiting for a development cycle.

04

Shared institutional knowledge

A semantic index that spans teams, repositories and business units identifies existing solutions, together with their authors, their constraints and the reasoning behind them. Work completed in one part of the organization becomes available to every other part.

05

Model independence

A gateway places frontier APIs and local inference behind a single interface, so that decisions about cost, latency, sensitivity and sovereignty are made for each request. Because memory resides in the platform, a model can be replaced without losing accumulated context.

06

Protected intellectual property

Contractors and offshore teams are granted a scoped interface to your assets. They can query, build, test and deliver against your code and data while the underlying intellectual property remains in your environment, and access ends when the contract does.

Platform architecture

Fourteen layers in a single platform

The function of each layer and the benefit it provides to the enterprise.

LayerFunctionEnterprise effect
RuntimeENGINEFault-tolerant concurrent runtime supervising agents, sessions and events across the estate.Long-running autonomous work continues through failures and restarts.
MemoryTRELLISDurable recall aligned to your retrieval patterns, with secrets sealed and excluded from the index.Information is derived once and reused, which reduces repeated processing cost.
KnowledgeGRAPHStructural index across code, documents and decisions, queried by relationship as well as keyword.Answers reflect the actual dependencies between systems.
RoutingGATEWAYA single interface to frontier and local models, with policy determining where each request runs.Sensitive workloads remain inside the perimeter without code changes.
CoordinationMESHCross-machine dispatch, a shared event bus and conflict detection between concurrent agents.Multiple agents work together without conflict.
GovernanceHOOKSDeterministic policy applied at agent decision points.Controls remain effective during autonomous operation and produce evidence for audit.
ComplianceRULESETSCorporate, regulatory and internal controls compiled to enforcement points and managed under version control.Policy changes take effect when published, without a software release.
DistributionSEEDPACKSVersioned capability bundles published once and applied across the estate.Proven methods from one team become available to every team.
DiscoveryINDEXSemantic index across every team’s code, decisions and post-mortems.Existing solutions are identified before work is duplicated.
AdaptationGRAFTSUsage patterns compiled into inspectable, reversible specialization layers for each user, team or domain.A single platform serves divisions with different requirements.
EfficiencyMOE ROUTINGMixture-of-experts selection sends each request to the smallest sufficient model, local or frontier.Unit costs decline while capability is preserved for complex work.
AssuranceFINGERPRINTBoundary classification and content fingerprinting of regulated, privileged and customer data.Sensitive material is identified and contained by design.
ContainmentGATESEvery egress path from the owner’s infrastructure is an explicit, logged policy decision.Data residency is enforced by the architecture.
MediationSCOPED ACCESSThird parties work with assets through permissioned interfaces, without receiving copies of code or data.Outsourced delivery proceeds while intellectual property remains protected.

Measured economics

Measured business results

Improvements in time, efficiency, resources and quality, measured on instrumented workloads.

~50%+
Token spend reduction

Stored recall reduces repeated processing, and local routing handles requests that do not require a frontier model.

65–75%
Output compression

Response shaping reduces generated tokens while preserving technical accuracy.

0
Egress required

Sovereign deployment keeps regulated workloads inside the perimeter by design.

0
Dev cycles per policy change

Rulesets are published without a software release, and enforcement takes effect across the estate as soon as a control changes.

Time

Context is prepared before it is requested, and results that have already been derived are recalled without being recomputed.

Efficiency

Mixture-of-experts routing matches each request to the smallest sufficient model, whether local or frontier.

Resources

Duplicate effort is identified early, and existing internal solutions are found before new work is commissioned.

Quality

Standards are enforced deterministically at the point of action, so conformance is consistent across the organization.

One week of training, and a lifetime of capability.

Adoption is measured in days because the platform learns how your enterprise works. The conventions, controls and solutions it acquires in the first week are retained and extended for as long as the platform is in use.

Economics reflect platform-instrumented savings measured against the same workloads run without recall, routing and response-shaping enabled. Realized figures vary with workload mix, model selection and deployment topology.

Our engagements

How we work with clients

Our AI engagements draw on our experience as a principal co-founder of Orchard AI and take the following forms.

01

AI readiness and architecture

An objective assessment of where AI will deliver value in your organization, what your data can support, and the cost of operating in production.

  • Estate and data-flow assessment
  • Model strategy: frontier, local, hybrid
  • Unit-economics modeling before commitment
  • Regulatory and residency constraint mapping
02

Sovereign deployment

Installation of a governed AI runtime on-premises, in a private cloud or in an air-gapped environment, so that regulated and privileged information remains on infrastructure you own.

  • On-prem, private cloud or air-gapped
  • Multi-machine mesh with cross-node dispatch
  • Local inference for data that cannot leave
  • Gated, logged egress by architecture
03

Governance and audit

Corporate standards and regulatory obligations implemented as controls that apply at the point of action, supported by a durable record of each decision, approval, escalation and override.

  • Rulesets authored once, enforced estate-wide
  • Production gates and destructive-action confirmation
  • Complete, searchable session audit trail
  • Per-jurisdiction and per-division policy scoping
04

Agent workflow engineering

Production workloads delivered through coordinated multi-agent execution, with persistent memory, conflict detection and structured hand-off between agents.

  • Multi-agent dispatch and coordination gates
  • Persistent institutional recall across sessions
  • Capability packs distributed estate-wide
  • Durable watchers and long-running autonomous work
05

Data intelligence

Analytics, data warehousing and pattern analysis focused on business decisions, including trend identification, early warning and reconciliation.

  • Data warehousing and analytics platforms
  • Trend and anomaly identification
  • Validation and reconciliation engines
  • Boundary classification and content fingerprinting
06

Mediated delivery

Contractors and offshore teams work with your code and data through a scoped interface. The underlying intellectual property remains in your environment, and access ends when the engagement does.

  • Scoped interface to assets, with no copies transferred
  • Per-engagement permission scope
  • Attributable action-level audit
  • Access terminates with the engagement

Common questions

Frequently asked questions

What is sovereign AI infrastructure?

Sovereign AI infrastructure means that the AI runtime, memory store, orchestration layer and knowledge graph are deployed within the organization that owns the data, and not within a vendor’s environment. Model weights, embeddings, recall indices and audit trails remain inside the organization, so data residency is enforced by the architecture. Koi Consulting deploys this model on-premises, in a private cloud or in an air-gapped environment.

What is Orchard AI?

Orchard AI is a sovereign AI infrastructure platform positioned as the operating system for an enterprise that runs on AI. It provides a governed runtime in which models, memory, tools and agents are components that the enterprise owns and can replace or combine. Koi Consulting is a principal co-founder of Orchard AI.

How does Koi Consulting reduce AI running costs?

Koi Consulting reduces AI operating costs through three mechanisms. Persistent recall stores results so that information is computed once and reused. Mixture-of-experts routing sends each request to the smallest sufficient model, whether local or frontier. Response shaping reduces generated output while preserving technical accuracy. Platform-instrumented measurements show a token spend reduction of approximately fifty percent or more and output compression of sixty-five to seventy-five percent, compared with the same workloads run without these features.

How is AI governance enforced in practice?

Corporate standards and regulatory obligations are compiled into policy controls that apply at agent decision points. Prohibited actions are blocked before they are executed, which keeps controls effective when agents operate autonomously. Rulesets are versioned and take effect when published, so a policy decision is enforced without a development cycle, and every decision, approval, escalation and override is captured in a durable audit record.

Can contractors work on our systems without receiving our intellectual property?

Yes. A contractor is granted a scoped interface to the asset and does not receive a copy of it. The contractor can query, build, test and deliver against the code and data while the underlying intellectual property remains in the client’s environment. Permissions are scoped to each engagement, every action is attributable, and access ends when the contract does.

Next step

Own your AI platform.

If your organization needs clear oversight and control of its AI agents, we would welcome the opportunity to discuss your requirements.