Extract metadata from file (structured)
Extract metadata from file (structured)
With Box AI API, you can extract metadata from the provided file
and get the result in the form of key-value pairs.
As input, you can either create a structure using the fields
parameter, or use an already defined metadata template.
To learn more about creating templates, see Creating metadata templates in the Admin Console or use the metadata template API.
Send a request
To send a request, use the
POST /2.0/ai/extract_structured
endpoint.
Make sure you have generated the developer token to authorize your app. See prerequisites for using Box AI for details.
curl -i -L 'https://api.box.com/2.0/ai/extract_structured' \
-H 'content-type: application/json' \
-H 'authorization: Bearer <ACCESS_TOKEN>' \
-d '{
"items": [
{
"id": "12345678",
"type": "file",
"content": "This is file content."
}
],
"metadata_template": {
"template_key": "",
"type": "metadata_template",
"scope": ""
},
"fields": [
{
"key": "name",
"description": "The name of the person.",
"displayName": "Name",
"prompt": "The name is the first and last name from the email address.",
"type": "string",
"options": [
{
"key": "First Name"
},
{
"key": "Last Name"
}
]
}
],
"ai_agent": {
"type": "ai_agent_extract",
"long_text": {
"model": "azure__openai__gpt_3_5_turbo_16k",
"system_message": "You are a helpful travel assistant specialized in budget travel",
"prompt_template": "It is `{current_date}`, and I have $8000 and want to spend a week in the Azores. What should I see?",
"num_tokens_for_completion": 8400,
"llm_endpoint_params": {
"type": "openai_params",
"temperature": 0,
"top_p": 1,
"frequency_penalty": 1.5,
"presence_penalty": 1.5,
"stop": "<|im_end|>"
},
"embeddings": {
"model": "openai__text_embedding_ada_002",
"strategy": {
"id": "basic",
"num_tokens_per_chunk": 64
}
}
},
"basic_text": {
"model": "azure__openai__gpt_3_5_turbo_16k",
"system_message": "You are a helpful travel assistant specialized in budget travel",
"prompt_template": "It is `{current_date}`, and I have $8000 and want to spend a week in the Azores. What should I see?",
"num_tokens_for_completion": 8400,
"llm_endpoint_params": {
"type": "openai_params",
"temperature": 0,
"top_p": 1,
"frequency_penalty": 1.5,
"presence_penalty": 1.5,
"stop": "<|im_end|>"
}
}
}
}'
await client.ai.createAiExtractStructured({
metadataTemplate: {
templateKey: templateKey,
scope: 'enterprise',
} satisfies AiExtractStructuredMetadataTemplateField,
items: [new AiItemBase({ id: file.id })],
} satisfies AiExtractStructured);
client.ai.create_ai_extract_structured(
[AiItemBase(id=file.id)],
metadata_template=CreateAiExtractStructuredMetadataTemplate(
template_key=template_key, scope="enterprise"
),
)
await client.Ai.CreateAiExtractStructuredAsync(requestBody: new AiExtractStructured(items: Array.AsReadOnly(new [] {new AiItemBase(id: file.Id)})) { MetadataTemplate = new AiExtractStructuredMetadataTemplateField() { TemplateKey = templateKey, Scope = "enterprise" } });
BoxAIExtractMetadataTemplate template = new BoxAIExtractMetadataTemplate("templateKey", "enterprise");
BoxAIExtractStructuredResponse result = BoxAI.extractMetadataStructured(
api,
Collections.singletonList(new BoxAIItem("123456", BoxAIItem.Type.FILE)),
template
);
JsonObject sourceJson = result.getSourceJson();
Parameters
To make a call, you must pass the following parameters. Mandatory parameters are in bold.
Parameter | Description | Example |
---|---|---|
metadata_template | The metadata template containing the fields to extract. For your request to work, you must provide either metadata_template or fields , but not both. | |
metadata_template.type | The type of metadata template. | metadata_template |
metadata_template.scope | The scope of the metadata template that can either be global or enterprise . Global templates are those available to any Box enterprise, whereas enterprise templates are bound to a specific enterprise. | metadata_template |
metadata_template.template_key | The name of your metadata template. | invoice |
items.id | Box file ID of the document. The ID must reference an actual file with an extension. | 1233039227512 |
items.type | The type of the supplied input. | file |
items.content | The content of the item, often the text representation. | This article is about Box AI . |
fields.type | The type of the field. It include but is not limited to string , float , date , enum , and multiSelect . | string |
fields.description | A description of the field. | The person's name. |
fields.displayName | The display name of the field. | Name |
fields.key | A unique identifier for the field. | name |
fields.options | A list of options for this field. This is most often used in combination with the enum and multiSelect field types. | [{"key":"First Name"},{"key":"Last Name"}] |
fields.options.key | A unique identifier for the field. | First Name |
fields.prompt | Additional context about the key (identifier) that may include how to find and format it. | Name is the first and last name from the email address |
ai_agent | The AI agent used to override the default agent configuration. This parameter allows you to, for example, replace the default LLM with a custom one using the model parameter, tweak the base prompt to allow for a more customized user experience, or change an LLM parameter, such as temperature , to make the results more or less creative. Before you use the ai_agent parameter, you can get the default configuration using the GET 2.0/ai_agent_default request. For specific use cases, see the AI model overrides tutorial. |
Use case
Let's assume you want to extract the vendor name, invoice number, and a few more details from the following sample invoice:
Create the request
To get the response from Box AI, call POST /2.0/ai/extract
endpoint with the following parameters:
items.type
anditems.id
to specify the file to extract the data from.fields
to specify the data that you want to extract from the given file.metadata_template
to supply an already existing metadata template.
Using fields
parameter
The fields
parameter allows you to specify the data you want to extract. Each fields
object has a subset of parameters you can use to add more information about the searched data.
For example, you can add the field type, description, or even a prompt with some additional context.
curl --location 'https://api.box.com/2.0/ai/extract_structured' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <ACCESS_TOKEN>'' \
--data '{
"items": [
{
"id": "1517628697289",
"type": "file"
}
],
"fields": [
{
"key": "document_type",
"type": "enum",
"prompt": "what type of document is this?",
"options": [
{
"key": "Invoice"
},
{
"key": "Purchase Order"
},
{
"key": "Unknown"
}
]
},
{
"key": "document_date",
"type": "date"
},
{
"key": "vendor",
"description": "The name of the entity.",
"prompt": "Which vendor is sending this document.",
"type": "string"
},
{
"key": "document_total",
"type": "float"
}
]
}'
The response lists the specified fields and their values:
{
"document_date": "2024-02-13",
"vendor": "Quasar Innovations",
"document_total": $1050,
"document_type": "Purchase Order"
}
Using metadata template
If you prefer to use a metadata template, you can provide its template_key
, type
, and scope
.
curl --location 'https://api.box.com/2.0/ai/extract_structured' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <ACCESS_TOKEN>' \
--data '{
"items": [
{
"id": "1517628697289",
"type": "file"
}
],
"metadata_template": {
"template_key": "rbInvoicePO",
"type": "metadata_template",
"scope": "enterprise_1134207681"
}
}'
The response lists the fields included in the metadata template and their values:
{
"documentDate": "February 13, 2024",
"total": "$1050",
"documentType": "Purchase Order",
"vendor": "Quasar Innovations",
"purchaseOrderNumber": "003"
}