Semantic Functions

A Semantic Function wraps a prompt to use it as a function call to perform some task.

Why can’t we use just use prompts directly?

A prompt needs a model to create a response. Essentially, a semantic function links a model instance to a prompt.

Instead of assuming that the model will always be, save GPT 3.5-turbo, a semantic function chooses from a number of models including GPT-4 and non-GPT models such as from Hugging Face.

Also, a Semantic Function is more flexible than always assuming that the input to a model is just text. Non GPT models, for example, may expect a structured JSON document. Semantic function inputs may include both text and variables to be injected into the prompt prior to calling the model.

Similarly, outputs may be JSON instead of plain text, requiring the output to be validated, fixed if necessary, or regenerated if the output is wrong or incorrectly formatted.

These are all the concerns of a semantic function.

What exactly is a Semantic Function?

The following example from the Marvin library demonstrates a semantic function within the context of LLMs.

from marvin import ai_fn

@ai_fn
def my_function(input: Type) -> ReturnType:
    """ 
    A docstring that describes the function's purpose and behavior.
    """

# call the function
my_function(input="my input")

Don’t worry about the specific details as this is just one approach to implementation. We’re using it to demonstrate the concept.

Essentially, a semantic function is like any other function - producing a transformed output for a given set of inputs, which define the function’s signature. The crucial difference is that the body of the function is implemented by the Large Language Model using a natural language input, which may be a question or set of instructions.

The input to the semantic function may be structured. If so, it is converted into a text value first; for example, interpolating structured values into a prompt string.

In the example below, the task of creating the list of fruits is performed by the LLM using the prompt “Generate a list of n fruits”. We declare what we want instead of specifying how to product it.

@ai_fn
def list_fruits(n: int) -> List[str]
    """Generate a list of n fruits"""


list_fruits(3)  # ["apple", "banana", "orange"]

This is potentially more powerful than simply saving coding effort. The LLM was able to generate values, resolving the semantic meaning of “fruit”.

The alternative code would look something like:

def list_fruits(n: int) -> List[str]
    """Generate a list of n fruits"""
    fruits = []
    for i in range(n):
        ##
        fruit = get_next_entry_from_fruits_db()
        # or equivalent data service
        ##
        fruits.append(fruit)
    return fruits

list_fruits(3)  # ["apple", "banana", "orange"]

Sounds too good to be true. What’s the catch?

Nothing comes for free. I wouldn’t trust this approach just yet to functions that involve math or expert knowledge without augmentation (see Agents). For problems within the domain of language, such as sentiment analysis, topic extraction, etc., using an LLM will work better than imperative code.

Then there is everything in the middle. What works best, depends. Choosing the right use case requires skill. For example, knowledge retrieval is potentially problematic because of the tendency of LLMs to “hallucinate” answers it doesn’t know. It doesn’t know what it doesn’t know. However, this issue can be mitigated by augmenting prompts with additional relevant context. (See Knowledge Doping.) Therefore, knowledge retrieval could be a great use case.

Other qualities of a potentially good use case include:

• Context heavy questions
• Fuzzy suggestions
• Summaries
• Explanations
• First drafts

Enhancing Semantic Functions

The example above is overly simplistic. To make semantic functions work in practice, we also need:

• Validation of inputs to avoid “garbage in, garbage out” (GIGO)
• The ability to define the expected inputs
• Validation of outputs
• Conversion of outputs into structured formats
• Fixing output formats where possible
• Regenerating outputs that cannot be fixed
• The ability to use different prompt versions
• The ability to inject context (knowledge) into prompts to mitigate the hallucination issue
• The ability to inject current or personalized data into prompts
• The ability to compose higher order functions out of atomic functions
• The ability to apply guardrails such as testing inputs for PII or Prompt Injection attacks

Prompt Store addresses each of these issues to create a production-ready application.