Model Management

Prompt Store supports a “Model Zoo” including Large Language Models (LLMs) from multiple vendors, public and private, task specific language models from Hugging Face, and custom models.

Define Models

To view the list of models or create a new model, click on “Models” under “Model Execution” in the left side menu.

Model List

Click “Create” in the top Nav bar to define a new model.

New Semantic Function Form

To find a model, other than by ID, we can attach a key, then lookup the model by key.

Model Form

Model Types

Four model types are currently supported.

LLM Completion: Given a single prompt, the model will return one or more predicted completions. This is the classis LLM interface. Many open source models only support completion requests. (Conversational interfaces can be simulated however by including chat history in the prompt.)
LLM Chat: The Chat model accepts a list of messages comprising a conversation, and will return a response. A message may play one of several roles - system messages provide upfront instructions, user messages are from the user or are used to simulate user requests when providing few-shot learning examples, assistant messages are from the model in response to a user message, and function messages (OpenAI only) are requests from the model to call an external function.
Hugging Face: Models available from Hugging Face including a range of LLM and task-specific natural language processing (NLP) models.
Custom: Custom models that provide an HTTP endpoint. Both single request and batch endpoints can be specified.