Cortex COMPLETE Structured Outputs¶

Using COMPLETE Structured Outputs¶

To obtain a response in a structured format, specify a JSON schema as the response_format argument. The supplied JSON schema object defines the structure, data types, and constraints that the generated text must conform to, including required fields. For simple task,s you don’t need to specify any details of the output format, or even instruct the model to “respond in JSON.” For more complex tasks, prompting the model to respond in JSON can improve accuracy; see Optimizing JSON adherence accuracy.

The schema must be specified as a sub-object within the options argument, as shown here, not as a string. This requires the use of single quotes for strings, not the double quotes used in JSON, as this is a SQL object. The response is a JSON object.

options: {
    ...
    response_format: {
        'type': 'json',
        'schema': {
            'type': 'object',
            'properties': {
                'property_name': {
                    'type': 'string'
                },
                ...
            },
            'required': ['property_name', ...]
        }
    }

SQL example¶

The following example demonstrates how to use the response_format argument to specify a JSON schema for the response.

SELECT SNOWFLAKE.CORTEX.COMPLETE('mistral-large2', [
        {
        'role': 'user',
        'content': 'Return the customer sentiment for the following review: New kid on the block, this pizza joint! The pie arrived neither in a flash nor a snail\'s pace, but the taste? Divine! Like a symphony of Italian flavors, it was a party in my mouth. But alas, the party was a tad pricey for my humble abode\'s standards. A mixed bag, I\'d say!'
            }
    ],
    {
        'temperature': 0,
        'max_tokens': 1000,
        'response_format':{
            'type':'json',
            'schema':{'type' : 'object','properties' : {'sentiment_categories':{'type':'array','items':{'type':'object','properties':
            {'food_quality' : {'type' : 'string'},'food_taste': {'type':'string'}, 'wait_time': {'type':'string'}, 'food_cost': {'type':'string'}},'required':['food_quality','food_taste' ,'wait_time','food_cost']}}}}
            }
    }
);

Response:

{
    "created": 1738683744,
    "model": "mistral-large2",
    "structured_output": [
        {
        "raw_message": {
            "sentiment_categories": [
            {
                "food_cost": "negative",
                "food_quality": "positive",
                "food_taste": "positive",
                "wait_time": "neutral"
            }
            ]
        },
        "type": "json"
        }
    ],
    "usage": {
        "completion_tokens": 60,
        "prompt_tokens": 94,
        "total_tokens": 154
    }
}

Python example¶

The following example demonstrates how to use the response_format argument to specify a JSON schema for the response.

from snowflake.cortex import complete, CompleteOptions

response_format = {
    "type": "json",
    "schema": {
        "type": "object",
        "properties": {
            "people": {
                "type": "array",
                "items": {
                    "type": "object",
                    "properties": {
                        "name": {"type": "string"},
                        "age": {"type": "number"},
                    },
                    "required": ["name", "age"],
                },
            }
        },
        "required": ["people"],
    },
}
prompt = [{
    "role": "user",
    "content": "Please prepare me a data set of 5 ppl and their age",
}]

options = CompleteOptions(
        max_tokens=4096,
        temperature=0.7,
        top_p=1,
        guardrails=False,
        response_format=response_format
    )


result = complete(
model="claude-3-5-sonnet",
prompt=prompt,
session={session_object}, # session created via connector
stream=True,
options=options,
)

output = "".join(result)
print(output)

Response:

{"people": [{"name":"John Smith","age":32},{"name":"Sarah Johnson","age":28},
{"name":"Michael Chen","age":45},{"name":"Emily Davis","age":19},{"name":"Robert Wilson","age":56}]}

REST API example¶

You can use the Snowflake Cortex LLM REST API to invoke COMPLETE with the LLM of your choice. Below is an example supplying a schema using the Cortex LLM REST API:

curl --location --request POST 'https://<account_identifier>.snowflakecomputing.com/api/v2/cortex/inference:complete'
--header 'Authorization: Bearer <jwt>' \
--header 'Accept: application/json, text/event-stream' \
--header 'Content-Type: application/json' \
--data-raw '{
    "model": "claude-3-5-sonnet",
    "messages": [{
        "role": "user",
        "content": "Order a pizza for a hungry space traveler heading to the planet Zorgon. Make sure to include a special instruction to avoid any intergalactic allergens."
    }],
    "max_tokens": 1000,
    "response_format": {
            "type": "json",
            "schema":
            {
                "type": "object",
                "properties":
                {
                    "crust":
                    {
                        "type": "string",
                        "enum":
                        [
                            "thin",
                            "thick",
                            "gluten-free",
                            "Rigellian fungus-based"
                        ]
                    },
                    "toppings":
                    {
                        "type": "array",
                        "items":
                        {
                            "type": "string",
                            "enum":
                            [
                                "Gnorchian sausage",
                                "Andromedian mushrooms",
                                "Quasar cheese"
                            ]
                        }
                    },
                    "delivery_planet":
                    {
                        "type": "string"
                    },
                    "special_instructions":
                    {
                        "type": "string"
                    }
                },
                "required":
                [
                    "crust",
                    "toppings",
                    "delivery_planet"
                ]
            }
        }
    }
}'

Response:

data: {"id":"4d62e41a-d2d7-4568-871a-48de1463ed2a","model":"claude-3-5-sonnet","choices":[{"delta":{"content":"{\"crust\":","content_list":[{"type":"text","text":"{\"crust\":"}]}}],"usage":{}}

data: {"id":"4d62e41a-d2d7-4568-871a-48de1463ed2a","model":"claude-3-5-sonnet","choices":[{"delta":{"content":" \"thin\"","content_list":[{"type":"text","text":" \"thin\""}]}}],"usage":{}}

data: {"id":"4d62e41a-d2d7-4568-871a-48de1463ed2a","model":"claude-3-5-sonnet","choices":[{"delta":{"content":", \"topping","content_list":[{"type":"text","text":", \"topping"}]}}],"usage":{}}

data: {"id":"4d62e41a-d2d7-4568-871a-48de1463ed2a","model":"claude-3-5-sonnet","choices":[{"delta":{"content":"s\": [\"Quasar","content_list":[{"type":"text","text":"s\": [\"Quasar"}]}}],"usage":{}}

Create a JSON schema definition¶

To get the best accuracy from COMPLETE Structured Outputs, follow these guidelines:

• Use the “required” field in the schema to specify required fields. COMPLETE raises an error if a required field cannot be extracted.

  In the following example, the schema directs COMPLETE to find people mentioned in the document. The people field is marked as required to make sure people are identified.

  {
      'type': 'object',
      'properties': {
          'dataset_name': {
              'type': 'string'
          },
          'created_at': {
              'type': 'string'
          },
          'people': {
              'type': 'array',
              'items': {
                  'type': 'object',
                  'properties': {
                      'name': {
                          'type': 'string'
                      },
                      'age': {
                          'type': 'number'
                      },
                      'isAdult': {
                          'type': 'boolean'
                      }
                  }
              }
          }
      },
      'required': [
          'dataset_name',
          'created_at',
          'people'
      ]
  }
  

  Copy

  Response:

  {
      "dataset_name": "name",
      "created_at": "date",
      "people": [
          {
              "name": "Andrew",
              "isAdult": true
          }
      ]
  }
  

• Provide detailed descriptions of the fields to be extracted so that the model can more accurately identify them. For example, the following schema includes a description of each of the fields of people: name, age, and isAdult.

  {
      'type': 'object',
      'properties': {
          'dataset_name': {
              'type': 'string'
          },
          'created_at': {
              'type': 'string'
          },
          'people': {
              'type': 'array',
              'items': {
                  'type': 'object',
                  'properties': {
                      'name': {
                          'type': 'string',
                          'description': 'name should be between 9 to 10 characters'
                      },
                      'age': {
                          'type': 'number',
                          'description': 'Should be a value between 0 and 200'
                      },
                      'isAdult': {
                          'type': 'boolean',
                          'description': 'Persons is older than 18'
                      }
                  }
              }
          }
      }
  }
  

  Copy

Optimizing JSON adherence accuracy¶

COMPLETE Structured Outputs does not usually require a prompt; it already understands that its response should conform to the schema you specify. However, task complexity can significantly influence the ability of LLMs to follow a JSON response format. The more complex the task, the more you can improve the accuracy of results by specifying a prompt.

• Simple tasks such as text classification, entity extraction, paraphrasing, and summarization tasks that don’t require complex reasoning generally do not require additional prompting. For smaller models of lower intelligence, just using Structured Outputs significantly improves JSON adherence accuracy, as it ignores any text the model provides unrelated to the supplied schema.

• Medium-complexity tasks include any simple task in which the model is asked for additional reasoning, such as providing its rationale for a classification decision. For these use cases, we recommend adding “Respond in JSON” in the prompt to optimize performance.

• Complex reasoning tasks prompt models to perform more open-ended ambiguous tasks, such as assessing and scoring the quality of a call based on the relevance, professionalism, and faithfulness of answers. For these use cases, we recommend using the most powerful models like Anthropic’s claude-3-5-sonnet or Mistral AI’s mistral-large2 and adding “Respond in JSON”, and details about the schema you want to generate in the prompt.

For the most consistent results, set the temperature option to 0 when you call COMPLETE, regardless of the task or model.

Cost considerations¶

Cortex COMPLETE Structured Outputs incurs compute cost based on the number of tokens processed, but does not incur additional compute cost for the overhead of verifying each token against the supplied JSON schema. However, the number of tokens processed (and billed) increases with schema complexity. In general, the larger and more complex the supplied schema is, the more input and output tokens are consumed. Highly-structured responses with deep nesting (e.g., hierarchical data) consume a larger number of tokens than simpler schemas.

Limitations¶

• You cannot use spaces in the keys of the schema.

• The characters allowed for property names are letters, digits, hyphen, underscore. Names may be a maximum of 64 characters long.

• You cannot address external schemas using $ref or $dynamicRef.

The following constraint keywords are not supported. The use of an unsupported constraint keyword results in an error.

These limitations might be addressed in future releases.

Error conditions¶