Core Abstractions

Prompt Store’s philosophy is that framework services are more effectively applied as a service - wrapping the foundation model APIs instead of embedding the service logic into client application code. The application code can and should focus on the UX and workflow, with AI services such as document indexing and retrieval, or model guardrails, applied transparently at the services layer. The AI should almost be invisible, available as functions that can be called like other functions. Yes, these functions have some unique operational characteristics, but these are best managed so multiple roles - product, operations, governance - have visibility.

In addition, the AI services will evolve independently from the application logic. Tight coupling of these concerns is likely to hold back adoption of new AI services as business applications don’t have the luxury of upgrading unless it coincides with an existing priority.

Prompt Store’s strength is externalising model orchestration internals such as prompts so they can be more visible and enable more operational controls.

The following doesn’t include agent-specific functions.

SemanticFunction and SemanticFunctionImplementation operate as a pair. SemanticFunction is the logical representation of a function. SemanticFunctionImplementation is the physical implementation. The main reason for the difference is so that clients can call SemanticFunction, and behind the scenes, the implementation can be swapped. One reason to swap the implementation is to trial a different prompt or model. The framework can enable blue green deployment.

Blue green deployment is an application release that gradually transfers user traffic from a previous version to a nearly identical new release—both of which are running in production. The old version is called the blue environment while the new version is known as the green environment. Once production traffic is fully transferred from blue to green, blue can standby in case of rollback or be pulled from production.

Blue green deployment is a safe way to roll out a new configuration.

Using a similar approach, we can conduct A/B testing of a new prompt or model to see which version performs better in practice.

Separating interface from implementation allows us to achieve the above without requiring maintenance or upgrade of the client application, which also means we can make and test changes without downtime.