iostack.ai Portal

A Custom Content or RAG Retriever Function can be used to retrieve content that will be directly injected into the LLM Prompt.

Once defined, a Custom Content Retriever Function can be utilised in multiple Stages or in the Common Stage Settings and will be called after the user's utterance is processed and just before prompting the LLM for a response at each turn.

Defining a Custom Content Retriever Function

To fully define a Custom Content Retriever Function, you must specify:

A Function Interface - define each input and secret parameter required.
A Function Implementation - define the Javascript code that uses the parameters and implements the logic of the function.

Note: Other arguments passed to a Custom RAG/Content Retrieval function are specified where the definition get's used, not where it is defined.

Terminology

The discussion that follows uses specific terms to specifically differentiate:

Agent Variables - the variables defined in the agent
Parameters - the input parameters of the function.

At runtime, agent variables are mapped to input parameters according to the mappings specified where the custom function definition is used in Stages and/or Common Stage Settings.

Input Parameters

For each input parameter, specify a Javascript parameter name that will be used by the function logic to access that parameter.
In addition, to aid in the identification of each input parameter when the function actually gets used in Stages or Common Stage Settings, specify a description for each.

Secrets

For each secret used in the function, specify a Javascript parameter name that will be used by the function logic to access that secret.
In addition, select the secret that will be mapped to that parameter.

Implementing a Function

Execution Environment

The function execution environment is a V8 Javascript environment supporting ECMAScript 2022.

fetch() and GraphQL Support

The environment has embedded support for fetch() and GraphQL for interacting with external endpoints. For more information, refer to the documentation on their use in Custom Functions.

Function Definition

The function definition must take the following specific form:

async function main({ query, topK, metadataFields, inputParams, secrets }) {
    return {
        docs: [],
    };
}

Note:

The function must be async.
The function must be named main.
The function must take a single argument object containing correct parameters (see below)
The function must return a single object containing a docs array element.

Passing data in to the function

All input data is passed to the Javascript Function as a single argument object.

The argument object contains 5 specific parameters:

Input Parameters

Note

With the exception of the inputParams and secrets, the following arguments are not specified as part of the Integration definition. Instead they are configured where the RAG/Content Retrieval is used in Stages or Common Stage Settings, enabling specialization by context.

Parameter	Details
query	The user utterance - when this integrations is used, if it is configured to resolve ambiguous language, this will be the resolved version of the user utterance
topK	The maximum number of docs to retrieve
metadataFields	The configured list of additional metadata fields to include in the metadata object
inputParams	As in a Custom JS Integration, these are additional agent variables configured in the function definition
secrets	As in a Custom JS Integration, these are any secrets configured in the function definition

Returning documents from the function

All documents are returned from the Javascript Function as a single return object with a single docs member.

    // Retrieve the result
    ...
 
    return {
        docs: [],
    };
};

Return type

Each entry of the returned docs array must conform to the following shape:

{
    // 'id' field - optional.
    // ----------------------
    // If none is provided, one will be generated. This field is merged with
    // the metadata collected in the 'metadata' field (see below) and can be
    // included in reference metadata for use by custom front end clients to
    // refer to entities in an external system.
    id:       string,
 
    // 'text' - Text that will be injected directly into the LLM prompt.
    text:     string,
 
    // 'distance' and 'score' fields
    // -----------------------------
    // 1. The following 2 fields are optional but also mutually exclusive ie.
    // if you are going to specify one, do NOT specify the other.
    //
    // 2. If either are used, ALL documents in the returned array must use the
    // same field.
    //
    // 3. Both are used as sort values to order documents according to their
    // relevance.
    //
    // 4. If your retrieved data has no inherent ranking, omit these fields.
    //
    distance: number,   // Lower values === higher relevance
    score: number,      // Higher values === higher relevance
 
    // 'metadata' field
    // ----------------
    // 1. The 'metadata' field holds any metadata associated with this content.
    //
    // 2. Depending on how this integration is configured and specialised where
    //    it's used,metadata fields included here can be configured to be:
    //    a) Shown to the LLM
    //    or
    //    b) Mined as references, enabling the LLM to emit contextual
    //       information as it talks about the subject of this content.
    //
    // 3. For more information, refer to the documentation on 'Using a Custom
    //    Content Retrieval Definition'.
    //
    // 4. Use the metaDataFields input parameter to filter and select only
    //    nominated fields (see example below).
    metadata: {
        <field_1>: <value>,
        <field_2>: <another_value>
        ...
    }
}

Example

async function main({
    query,          // The resolved user utterance.
    topK,           // The max number of docs to retrieve.
    metadataFields, // A list of metadata fields to include in the metadata field of each
                    // returned document.
    inputParams,    // Input parameters containing values from any required agent variables
    secrets         // Secrets requred by the function.
}) {
 
  // Retrieve raw documents
  const results = ...
 
  // Massage the raw documents into an array of the required return type.
  const docs = results.docs.map(r => {
    return {
      id: r.chunk_id,
      distance: r.distance,
      text: r.text_chunk,
      // Filter the metadata to only include configured key/value pairs
      metadata: Object.fromEntries(
        Object.entries(r.metadata).filter([key]) => metadataFields.includes(key)
      )
    }
  })
 
  return {
    docs
  };
 
}

Defining a Custom Content Retriever

On this page