🎨 Data Designer Tutorial: Structured Outputs and Jinja Expressions¶

📚 What you'll learn¶

In this notebook, we will continue our exploration of Data Designer, demonstrating more advanced data generation using structured outputs and Jinja expressions.

If this is your first time using Data Designer, we recommend starting with the first notebook in this tutorial series.

📦 Import the essentials¶

The essentials module provides quick access to the most commonly used objects.

In [1]:

Copied!





from data_designer.essentials import (
    CategorySamplerParams,
    DataDesigner,
    DataDesignerConfigBuilder,
    ExpressionColumnConfig,
    InferenceParameters,
    LLMStructuredColumnConfig,
    ModelConfig,
    PersonFromFakerSamplerParams,
    SamplerColumnConfig,
    SamplerType,
    SubcategorySamplerParams,
)
from data_designer.essentials import (
    CategorySamplerParams,
    DataDesigner,
    DataDesignerConfigBuilder,
    ExpressionColumnConfig,
    InferenceParameters,
    LLMStructuredColumnConfig,
    ModelConfig,
    PersonFromFakerSamplerParams,
    SamplerColumnConfig,
    SamplerType,
    SubcategorySamplerParams,
)

⚙️ Initialize the Data Designer interface¶

DataDesigner is the main object that is used to interface with the library.
When initialized without arguments, the default model providers are used.

In [2]:

Copied!

data_designer_client = DataDesigner()
data_designer_client = DataDesigner()

🎛️ Define model configurations¶

Each ModelConfig defines a model that can be used during the generation process.
The "model alias" is used to reference the model in the Data Designer config (as we will see below).
The "model provider" is the external service that hosts the model (see the model config docs for more details).
By default, we use build.nvidia.com as the model provider.

In [3]:

Copied!





# This name is set in the model provider configuration.
MODEL_PROVIDER = "nvidia"

# The model ID is from build.nvidia.com.
MODEL_ID = "nvidia/nvidia-nemotron-nano-9b-v2"

# We choose this alias to be descriptive for our use case.
MODEL_ALIAS = "nemotron-nano-v2"

# This sets reasoning to False for the nemotron-nano-v2 model.
SYSTEM_PROMPT = "/no_think"

model_configs = [
    ModelConfig(
        alias=MODEL_ALIAS,
        model=MODEL_ID,
        provider=MODEL_PROVIDER,
        inference_parameters=InferenceParameters(
            temperature=0.5,
            top_p=1.0,
            max_tokens=1024,
        ),
    )
]
# This name is set in the model provider configuration.
MODEL_PROVIDER = "nvidia"

# The model ID is from build.nvidia.com.
MODEL_ID = "nvidia/nvidia-nemotron-nano-9b-v2"

# We choose this alias to be descriptive for our use case.
MODEL_ALIAS = "nemotron-nano-v2"

# This sets reasoning to False for the nemotron-nano-v2 model.
SYSTEM_PROMPT = "/no_think"

model_configs = [
    ModelConfig(
        alias=MODEL_ALIAS,
        model=MODEL_ID,
        provider=MODEL_PROVIDER,
        inference_parameters=InferenceParameters(
            temperature=0.5,
            top_p=1.0,
            max_tokens=1024,
        ),
    )
]

🏗️ Initialize the Data Designer Config Builder¶

The Data Designer config defines the dataset schema and generation process.
The config builder provides an intuitive interface for building this configuration.
The list of model configs is provided to the builder at initialization.

In [4]:

Copied!

config_builder = DataDesignerConfigBuilder(model_configs=model_configs)
config_builder = DataDesignerConfigBuilder(model_configs=model_configs)

🧑‍🎨 Designing our data¶

We will again create a product review dataset, but this time we will use structured outputs and Jinja expressions.
Structured outputs let you specify the exact schema of the data you want to generate.
Data Designer supports schemas specified using either json schema or Pydantic data models (recommended).

We'll define our structured outputs using Pydantic data models

💡 Why Pydantic?

Pydantic models provide better IDE support and type validation.

They are more Pythonic than raw JSON schemas.

They integrate seamlessly with Data Designer's structured output system.

In [5]:

Copied!





from decimal import Decimal
from typing import Literal

from pydantic import BaseModel, Field


# We define a Product schema so that the name, description, and price are generated
# in one go, with the types and constraints specified.
class Product(BaseModel):
    name: str = Field(description="The name of the product")
    description: str = Field(description="A description of the product")
    price: Decimal = Field(description="The price of the product", ge=10, le=1000, decimal_places=2)


class ProductReview(BaseModel):
    rating: int = Field(description="The rating of the product", ge=1, le=5)
    customer_mood: Literal["irritated", "mad", "happy", "neutral", "excited"] = Field(
        description="The mood of the customer"
    )
    review: str = Field(description="A review of the product")
from decimal import Decimal
from typing import Literal

from pydantic import BaseModel, Field


# We define a Product schema so that the name, description, and price are generated
# in one go, with the types and constraints specified.
class Product(BaseModel):
    name: str = Field(description="The name of the product")
    description: str = Field(description="A description of the product")
    price: Decimal = Field(description="The price of the product", ge=10, le=1000, decimal_places=2)


class ProductReview(BaseModel):
    rating: int = Field(description="The rating of the product", ge=1, le=5)
    customer_mood: Literal["irritated", "mad", "happy", "neutral", "excited"] = Field(
        description="The mood of the customer"
    )
    review: str = Field(description="A review of the product")

Next, let's design our product review dataset using a few more tricks compared to the previous notebook.

In [6]:

Copied!





# Since we often only want a few attributes from Person objects, we can
# set drop=True in the column config to drop the column from the final dataset.
config_builder.add_column(
    SamplerColumnConfig(
        name="customer",
        sampler_type=SamplerType.PERSON_FROM_FAKER,
        params=PersonFromFakerSamplerParams(),
        drop=True,
    )
)

config_builder.add_column(
    SamplerColumnConfig(
        name="product_category",
        sampler_type=SamplerType.CATEGORY,
        params=CategorySamplerParams(
            values=[
                "Electronics",
                "Clothing",
                "Home & Kitchen",
                "Books",
                "Home Office",
            ],
        ),
    )
)

config_builder.add_column(
    SamplerColumnConfig(
        name="product_subcategory",
        sampler_type=SamplerType.SUBCATEGORY,
        params=SubcategorySamplerParams(
            category="product_category",
            values={
                "Electronics": [
                    "Smartphones",
                    "Laptops",
                    "Headphones",
                    "Cameras",
                    "Accessories",
                ],
                "Clothing": [
                    "Men's Clothing",
                    "Women's Clothing",
                    "Winter Coats",
                    "Activewear",
                    "Accessories",
                ],
                "Home & Kitchen": [
                    "Appliances",
                    "Cookware",
                    "Furniture",
                    "Decor",
                    "Organization",
                ],
                "Books": [
                    "Fiction",
                    "Non-Fiction",
                    "Self-Help",
                    "Textbooks",
                    "Classics",
                ],
                "Home Office": [
                    "Desks",
                    "Chairs",
                    "Storage",
                    "Office Supplies",
                    "Lighting",
                ],
            },
        ),
    )
)

config_builder.add_column(
    SamplerColumnConfig(
        name="target_age_range",
        sampler_type=SamplerType.CATEGORY,
        params=CategorySamplerParams(values=["18-25", "25-35", "35-50", "50-65", "65+"]),
    )
)

# Sampler columns support conditional params, which are used if the condition is met.
# In this example, we set the review style to rambling if the target age range is 18-25.
# Note conditional parameters are only supported for Sampler column types.
config_builder.add_column(
    SamplerColumnConfig(
        name="review_style",
        sampler_type=SamplerType.CATEGORY,
        params=CategorySamplerParams(
            values=["rambling", "brief", "detailed", "structured with bullet points"],
            weights=[1, 2, 2, 1],
        ),
        conditional_params={
            "target_age_range == '18-25'": CategorySamplerParams(values=["rambling"]),
        },
    )
)

# Optionally validate that the columns are configured correctly.
config_builder.validate()
# Since we often only want a few attributes from Person objects, we can
# set drop=True in the column config to drop the column from the final dataset.
config_builder.add_column(
    SamplerColumnConfig(
        name="customer",
        sampler_type=SamplerType.PERSON_FROM_FAKER,
        params=PersonFromFakerSamplerParams(),
        drop=True,
    )
)

config_builder.add_column(
    SamplerColumnConfig(
        name="product_category",
        sampler_type=SamplerType.CATEGORY,
        params=CategorySamplerParams(
            values=[
                "Electronics",
                "Clothing",
                "Home & Kitchen",
                "Books",
                "Home Office",
            ],
        ),
    )
)

config_builder.add_column(
    SamplerColumnConfig(
        name="product_subcategory",
        sampler_type=SamplerType.SUBCATEGORY,
        params=SubcategorySamplerParams(
            category="product_category",
            values={
                "Electronics": [
                    "Smartphones",
                    "Laptops",
                    "Headphones",
                    "Cameras",
                    "Accessories",
                ],
                "Clothing": [
                    "Men's Clothing",
                    "Women's Clothing",
                    "Winter Coats",
                    "Activewear",
                    "Accessories",
                ],
                "Home & Kitchen": [
                    "Appliances",
                    "Cookware",
                    "Furniture",
                    "Decor",
                    "Organization",
                ],
                "Books": [
                    "Fiction",
                    "Non-Fiction",
                    "Self-Help",
                    "Textbooks",
                    "Classics",
                ],
                "Home Office": [
                    "Desks",
                    "Chairs",
                    "Storage",
                    "Office Supplies",
                    "Lighting",
                ],
            },
        ),
    )
)

config_builder.add_column(
    SamplerColumnConfig(
        name="target_age_range",
        sampler_type=SamplerType.CATEGORY,
        params=CategorySamplerParams(values=["18-25", "25-35", "35-50", "50-65", "65+"]),
    )
)

# Sampler columns support conditional params, which are used if the condition is met.
# In this example, we set the review style to rambling if the target age range is 18-25.
# Note conditional parameters are only supported for Sampler column types.
config_builder.add_column(
    SamplerColumnConfig(
        name="review_style",
        sampler_type=SamplerType.CATEGORY,
        params=CategorySamplerParams(
            values=["rambling", "brief", "detailed", "structured with bullet points"],
            weights=[1, 2, 2, 1],
        ),
        conditional_params={
            "target_age_range == '18-25'": CategorySamplerParams(values=["rambling"]),
        },
    )
)

# Optionally validate that the columns are configured correctly.
config_builder.validate()

[02:31:58] [INFO] ✅ Validation passed

Out[6]:

DataDesignerConfigBuilder(
    sampler_columns: [
        "customer",
        "product_category",
        "product_subcategory",
        "target_age_range",
        "review_style"
    ]
)

Next, we will use more advanced Jinja expressions to create new columns.

Jinja expressions let you:

Access nested attributes: {{ customer.first_name }}
Combine values: {{ customer.first_name }} {{ customer.last_name }}
Use conditional logic: {% if condition %}...{% endif %}

In [7]:

Copied!





# We can create new columns using Jinja expressions that reference
# existing columns, including attributes of nested objects.
config_builder.add_column(
    ExpressionColumnConfig(name="customer_name", expr="{{ customer.first_name }} {{ customer.last_name }}")
)

config_builder.add_column(ExpressionColumnConfig(name="customer_age", expr="{{ customer.age }}"))

config_builder.add_column(
    LLMStructuredColumnConfig(
        name="product",
        prompt=(
            "Create a product in the '{{ product_category }}' category, focusing on products  "
            "related to '{{ product_subcategory }}'. The target age range of the ideal customer is "
            "{{ target_age_range }} years old. The product should be priced between $10 and $1000."
        ),
        system_prompt=SYSTEM_PROMPT,
        output_format=Product,
        model_alias=MODEL_ALIAS,
    )
)

# We can even use if/else logic in our Jinja expressions to create more complex prompt patterns.
config_builder.add_column(
    LLMStructuredColumnConfig(
        name="customer_review",
        prompt=(
            "Your task is to write a review for the following product:\n\n"
            "Product Name: {{ product.name }}\n"
            "Product Description: {{ product.description }}\n"
            "Price: {{ product.price }}\n\n"
            "Imagine your name is {{ customer_name }} and you are from {{ customer.city }}, {{ customer.state }}. "
            "Write the review in a style that is '{{ review_style }}'."
            "{% if target_age_range == '18-25' %}"
            "Make sure the review is more informal and conversational."
            "{% else %}"
            "Make sure the review is more formal and structured."
            "{% endif %}"
        ),
        system_prompt=SYSTEM_PROMPT,
        output_format=ProductReview,
        model_alias=MODEL_ALIAS,
    )
)

config_builder.validate()
# We can create new columns using Jinja expressions that reference
# existing columns, including attributes of nested objects.
config_builder.add_column(
    ExpressionColumnConfig(name="customer_name", expr="{{ customer.first_name }} {{ customer.last_name }}")
)

config_builder.add_column(ExpressionColumnConfig(name="customer_age", expr="{{ customer.age }}"))

config_builder.add_column(
    LLMStructuredColumnConfig(
        name="product",
        prompt=(
            "Create a product in the '{{ product_category }}' category, focusing on products  "
            "related to '{{ product_subcategory }}'. The target age range of the ideal customer is "
            "{{ target_age_range }} years old. The product should be priced between $10 and $1000."
        ),
        system_prompt=SYSTEM_PROMPT,
        output_format=Product,
        model_alias=MODEL_ALIAS,
    )
)

# We can even use if/else logic in our Jinja expressions to create more complex prompt patterns.
config_builder.add_column(
    LLMStructuredColumnConfig(
        name="customer_review",
        prompt=(
            "Your task is to write a review for the following product:\n\n"
            "Product Name: {{ product.name }}\n"
            "Product Description: {{ product.description }}\n"
            "Price: {{ product.price }}\n\n"
            "Imagine your name is {{ customer_name }} and you are from {{ customer.city }}, {{ customer.state }}. "
            "Write the review in a style that is '{{ review_style }}'."
            "{% if target_age_range == '18-25' %}"
            "Make sure the review is more informal and conversational."
            "{% else %}"
            "Make sure the review is more formal and structured."
            "{% endif %}"
        ),
        system_prompt=SYSTEM_PROMPT,
        output_format=ProductReview,
        model_alias=MODEL_ALIAS,
    )
)

config_builder.validate()

[02:31:58] [INFO] ✅ Validation passed

Out[7]:

DataDesignerConfigBuilder(
    sampler_columns: [
        "customer",
        "product_category",
        "product_subcategory",
        "target_age_range",
        "review_style"
    ]
    llm_structured_columns: ['product', 'customer_review']
    expression_columns: ['customer_name', 'customer_age']
)

🔁 Iteration is key – preview the dataset!¶

Use the preview method to generate a sample of records quickly.
Inspect the results for quality and format issues.
Adjust column configurations, prompts, or parameters as needed.
Re-run the preview until satisfied.

In [8]:

Copied!

preview = data_designer_client.preview(config_builder, num_records=2)
preview = data_designer_client.preview(config_builder, num_records=2)

[02:31:58] [INFO] 📸 Preview generation in progress

[02:31:58] [INFO] ✅ Validation passed

[02:31:58] [INFO] ⛓️ Sorting column configs into a Directed Acyclic Graph

[02:31:58] [INFO] 🩺 Running health checks for models...

[02:31:58] [INFO]   |-- 👀 Checking 'nvidia/nvidia-nemotron-nano-9b-v2' in provider named 'nvidia' for model alias 'nemotron-nano-v2'...

[02:32:01] [INFO]   |-- ✅ Passed!

[02:32:01] [INFO] 🎲 Preparing samplers to generate 2 records across 5 columns

[02:32:01] [INFO] 🧩 Generating column `customer_name` from expression

[02:32:01] [INFO] 🧩 Generating column `customer_age` from expression

[02:32:01] [INFO] 🗂️ Preparing llm-structured column generation

[02:32:01] [INFO]   |-- column name: 'product'

[02:32:01] [INFO]   |-- model config:
{
    "alias": "nemotron-nano-v2",
    "model": "nvidia/nvidia-nemotron-nano-9b-v2",
    "inference_parameters": {
        "temperature": 0.5,
        "top_p": 1.0,
        "max_tokens": 1024,
        "max_parallel_requests": 4,
        "timeout": null,
        "extra_body": null
    },
    "provider": "nvidia"
}

[02:32:01] [INFO] 🐙 Processing llm-structured column 'product' with 4 concurrent workers

[02:32:02] [INFO] 🗂️ Preparing llm-structured column generation

[02:32:02] [INFO]   |-- column name: 'customer_review'

[02:32:02] [INFO]   |-- model config:
{
    "alias": "nemotron-nano-v2",
    "model": "nvidia/nvidia-nemotron-nano-9b-v2",
    "inference_parameters": {
        "temperature": 0.5,
        "top_p": 1.0,
        "max_tokens": 1024,
        "max_parallel_requests": 4,
        "timeout": null,
        "extra_body": null
    },
    "provider": "nvidia"
}

[02:32:02] [INFO] 🐙 Processing llm-structured column 'customer_review' with 4 concurrent workers

[02:32:05] [INFO] 📊 Model usage summary:
{
    "nvidia/nvidia-nemotron-nano-9b-v2": {
        "token_usage": {
            "prompt_tokens": 1253,
            "completion_tokens": 586,
            "total_tokens": 1839
        },
        "request_usage": {
            "successful_requests": 4,
            "failed_requests": 0,
            "total_requests": 4
        },
        "tokens_per_second": 435,
        "requests_per_minute": 56
    }
}

[02:32:05] [INFO] 🙈 Dropping columns: ['customer']

[02:32:05] [INFO] 📐 Measuring dataset column statistics:

[02:32:05] [INFO]   |-- 🎲 column: 'product_category'

[02:32:05] [INFO]   |-- 🎲 column: 'product_subcategory'

[02:32:05] [INFO]   |-- 🎲 column: 'target_age_range'

[02:32:05] [INFO]   |-- 🎲 column: 'review_style'

[02:32:05] [INFO]   |-- 🧩 column: 'customer_name'

[02:32:05] [INFO]   |-- 🧩 column: 'customer_age'

[02:32:05] [INFO]   |-- 🗂️ column: 'product'

[02:32:05] [INFO]   |-- 🗂️ column: 'customer_review'

[02:32:05] [INFO] ✅ Preview complete!

In [9]:

Copied!

# Run this cell multiple times to cycle through the 2 preview records.
preview.display_sample_record()
# Run this cell multiple times to cycle through the 2 preview records.
preview.display_sample_record()

                                                                                                                   
                                                 Generated Columns                                                 
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Name                ┃ Value                                                                                     ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ product_category    │ Clothing                                                                                  │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────┤
│ product_subcategory │ Winter Coats                                                                              │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────┤
│ target_age_range    │ 65+                                                                                       │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────┤
│ review_style        │ brief                                                                                     │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────┤
│ customer_name       │ Peter Holland                                                                             │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────┤
│ customer_age        │ 64                                                                                        │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────┤
│ product             │ {                                                                                         │
│                     │     'name': 'Winter Insulated Coat for Seniors',                                          │
│                     │     'description': 'A warm, durable, and stylish winter coat designed for individuals     │
│                     │ aged 65 and above. Features include a thick insulated lining, water-resistant fabric,     │
│                     │ adjustable cuffs, and a comfortable fit to ensure ease of movement and maximum warmth     │
│                     │ during cold seasons.',                                                                    │
│                     │     'price': 89.99                                                                        │
│                     │ }                                                                                         │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────┤
│ customer_review     │ {                                                                                         │
│                     │     'rating': 5,                                                                          │
│                     │     'customer_mood': 'happy',                                                             │
│                     │     'review': 'The Winter Insulated Coat for Seniors is an excellent purchase. Its thick  │
│                     │ insulation, water-resistant fabric, and comfortable fit make it both practical and        │
│                     │ stylish for winter use. Highly recommended for seniors seeking warmth and durability.'    │
│                     │ }                                                                                         │
└─────────────────────┴───────────────────────────────────────────────────────────────────────────────────────────┘
                                                                                                                   
                                                    [index: 0]

In [10]:

Copied!

# The preview dataset is available as a pandas DataFrame.
preview.dataset
# The preview dataset is available as a pandas DataFrame.
preview.dataset

Out[10]:

	product_category	product_subcategory	target_age_range	review_style	customer_name	customer_age	product	customer_review
0	Clothing	Winter Coats	65+	brief	Peter Holland	64	{'name': 'Winter Insulated Coat for Seniors', ...	{'rating': 5, 'customer_mood': 'happy', 'revie...
1	Home & Kitchen	Appliances	50-65	detailed	Peggy Turner	30	{'name': 'Energy-Efficient Induction Cooktop',...	{'rating': 5, 'customer_mood': 'happy', 'revie...

📊 Analyze the generated data¶

Data Designer automatically generates a basic statistical analysis of the generated data.
This analysis is available via the analysis property of generation result objects.

In [11]:

Copied!

# Print the analysis as a table.
preview.analysis.to_report()
# Print the analysis as a table.
preview.analysis.to_report()

──────────────────────────────────────── 🎨 Data Designer Dataset Profile ─────────────────────────────────────────

                                                                                                                   
                                                 Dataset Overview                                                  
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ number of records               ┃ number of columns               ┃ percent complete records                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 2                               │ 8                               │ 100.0%                                      │
└─────────────────────────────────┴─────────────────────────────────┴─────────────────────────────────────────────┘
                                                                                                                   
                                                                                                                   
                                                🎲 Sampler Columns                                                 
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓
┃ column name                      ┃        data type ┃               number unique values ┃         sampler type ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩
│ product_category                 │           string │                         2 (100.0%) │             category │
├──────────────────────────────────┼──────────────────┼────────────────────────────────────┼──────────────────────┤
│ product_subcategory              │           string │                         2 (100.0%) │          subcategory │
├──────────────────────────────────┼──────────────────┼────────────────────────────────────┼──────────────────────┤
│ target_age_range                 │           string │                         2 (100.0%) │             category │
├──────────────────────────────────┼──────────────────┼────────────────────────────────────┼──────────────────────┤
│ review_style                     │           string │                         2 (100.0%) │             category │
└──────────────────────────────────┴──────────────────┴────────────────────────────────────┴──────────────────────┘
                                                                                                                   
                                                                                                                   
                                             🗂️ LLM-Structured Columns                                              
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┓
┃                       ┃               ┃                            ┃     prompt tokens ┃      completion tokens ┃
┃ column name           ┃     data type ┃       number unique values ┃        per record ┃             per record ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩
│ product               │          dict │                 2 (100.0%) │     269.0 +/- 1.0 │           73.5 +/- 2.1 │
├───────────────────────┼───────────────┼────────────────────────────┼───────────────────┼────────────────────────┤
│ customer_review       │          dict │                 2 (100.0%) │     317.5 +/- 2.5 │        188.0 +/- 178.2 │
└───────────────────────┴───────────────┴────────────────────────────┴───────────────────┴────────────────────────┘
                                                                                                                   
                                                                                                                   
                                               🧩 Expression Columns                                               
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ column name                       ┃                data type ┃                             number unique values ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ customer_name                     │                   string │                                       2 (100.0%) │
├───────────────────────────────────┼──────────────────────────┼──────────────────────────────────────────────────┤
│ customer_age                      │                   string │                                       2 (100.0%) │
└───────────────────────────────────┴──────────────────────────┴──────────────────────────────────────────────────┘
                                                                                                                   
                                                                                                                   
╭────────────────────────────────────────────────── Table Notes ──────────────────────────────────────────────────╮
│                                                                                                                 │
│  1. All token statistics are based on a sample of max(1000, len(dataset)) records.                              │
│  2. Tokens are calculated using tiktoken's cl100k_base tokenizer.                                               │
│                                                                                                                 │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
                                                                                                                   
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────

🆙 Scale up!¶

Happy with your preview data?
Use the create method to submit larger Data Designer generation jobs.

In [12]:

Copied!

job_results = data_designer_client.create(config_builder, num_records=10)
job_results = data_designer_client.create(config_builder, num_records=10)

[02:32:05] [INFO] 🎨 Creating Data Designer dataset

[02:32:05] [INFO] ✅ Validation passed

[02:32:05] [INFO] ⛓️ Sorting column configs into a Directed Acyclic Graph

[02:32:05] [INFO] 📂 Dataset path '/home/runner/work/DataDesigner/DataDesigner/docs/notebook_source/artifacts/dataset' already exists. Dataset from this session
		     will be saved to '/home/runner/work/DataDesigner/DataDesigner/docs/notebook_source/artifacts/dataset_12-11-2025_023205' instead.

[02:32:05] [INFO] 🩺 Running health checks for models...

[02:32:05] [INFO]   |-- 👀 Checking 'nvidia/nvidia-nemotron-nano-9b-v2' in provider named 'nvidia' for model alias 'nemotron-nano-v2'...

[02:32:06] [INFO]   |-- ✅ Passed!

[02:32:06] [INFO] ⏳ Processing batch 1 of 1

[02:32:06] [INFO] 🎲 Preparing samplers to generate 10 records across 5 columns

[02:32:06] [INFO] 🧩 Generating column `customer_name` from expression

[02:32:06] [INFO] 🧩 Generating column `customer_age` from expression

[02:32:06] [INFO] 🗂️ Preparing llm-structured column generation

[02:32:06] [INFO]   |-- column name: 'product'

[02:32:06] [INFO]   |-- model config:
{
    "alias": "nemotron-nano-v2",
    "model": "nvidia/nvidia-nemotron-nano-9b-v2",
    "inference_parameters": {
        "temperature": 0.5,
        "top_p": 1.0,
        "max_tokens": 1024,
        "max_parallel_requests": 4,
        "timeout": null,
        "extra_body": null
    },
    "provider": "nvidia"
}

[02:32:06] [INFO] 🐙 Processing llm-structured column 'product' with 4 concurrent workers

[02:32:13] [INFO] 🗂️ Preparing llm-structured column generation

[02:32:13] [INFO]   |-- column name: 'customer_review'

[02:32:13] [INFO]   |-- model config:
{
    "alias": "nemotron-nano-v2",
    "model": "nvidia/nvidia-nemotron-nano-9b-v2",
    "inference_parameters": {
        "temperature": 0.5,
        "top_p": 1.0,
        "max_tokens": 1024,
        "max_parallel_requests": 4,
        "timeout": null,
        "extra_body": null
    },
    "provider": "nvidia"
}

[02:32:13] [INFO] 🐙 Processing llm-structured column 'customer_review' with 4 concurrent workers

[02:32:22] [INFO] 🙈 Dropping columns: ['customer']

[02:32:22] [INFO] 📊 Model usage summary:
{
    "nvidia/nvidia-nemotron-nano-9b-v2": {
        "token_usage": {
            "prompt_tokens": 6247,
            "completion_tokens": 3275,
            "total_tokens": 9522
        },
        "request_usage": {
            "successful_requests": 20,
            "failed_requests": 0,
            "total_requests": 20
        },
        "tokens_per_second": 607,
        "requests_per_minute": 76
    }
}

[02:32:22] [INFO] 📐 Measuring dataset column statistics:

[02:32:22] [INFO]   |-- 🎲 column: 'product_category'

[02:32:22] [INFO]   |-- 🎲 column: 'product_subcategory'

[02:32:22] [INFO]   |-- 🎲 column: 'target_age_range'

[02:32:22] [INFO]   |-- 🎲 column: 'review_style'

[02:32:22] [INFO]   |-- 🧩 column: 'customer_name'

[02:32:22] [INFO]   |-- 🧩 column: 'customer_age'

[02:32:22] [INFO]   |-- 🗂️ column: 'product'

[02:32:22] [INFO]   |-- 🗂️ column: 'customer_review'

In [13]:

Copied!

# Load the generated dataset as a pandas DataFrame.
dataset = job_results.load_dataset()

dataset.head()
# Load the generated dataset as a pandas DataFrame.
dataset = job_results.load_dataset()

dataset.head()

Out[13]:

	product_category	product_subcategory	target_age_range	review_style	customer_name	customer_age	product	customer_review
0	Electronics	Accessories	18-25	rambling	Kelly Jones	39	{'description': 'High-quality wireless earbuds...	{'customer_mood': 'happy', 'rating': 4, 'revie...
1	Home Office	Chairs	65+	detailed	Donna Hernandez	66	{'description': 'A supportive and adjustable c...	{'customer_mood': 'happy', 'rating': 5, 'revie...
2	Clothing	Men's Clothing	25-35	detailed	Xavier Jones	63	{'description': "A modern, versatile pair of m...	{'customer_mood': 'happy', 'rating': 5, 'revie...
3	Clothing	Women's Clothing	50-65	brief	Bryan Burch	69	{'description': "A sophisticated women's dress...	{'customer_mood': 'happy', 'rating': 5, 'revie...
4	Clothing	Men's Clothing	25-35	detailed	Brian Howard	81	{'description': 'A high-quality, breathable dr...	{'customer_mood': 'happy', 'rating': 5, 'revie...

In [14]:

Copied!

# Load the analysis results into memory.
analysis = job_results.load_analysis()

analysis.to_report()
# Load the analysis results into memory.
analysis = job_results.load_analysis()

analysis.to_report()

──────────────────────────────────────── 🎨 Data Designer Dataset Profile ─────────────────────────────────────────

                                                                                                                   
                                                 Dataset Overview                                                  
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ number of records               ┃ number of columns               ┃ percent complete records                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 10                              │ 8                               │ 100.0%                                      │
└─────────────────────────────────┴─────────────────────────────────┴─────────────────────────────────────────────┘
                                                                                                                   
                                                                                                                   
                                                🎲 Sampler Columns                                                 
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓
┃ column name                      ┃        data type ┃               number unique values ┃         sampler type ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩
│ product_category                 │           string │                          5 (50.0%) │             category │
├──────────────────────────────────┼──────────────────┼────────────────────────────────────┼──────────────────────┤
│ product_subcategory              │           string │                          9 (90.0%) │          subcategory │
├──────────────────────────────────┼──────────────────┼────────────────────────────────────┼──────────────────────┤
│ target_age_range                 │           string │                          5 (50.0%) │             category │
├──────────────────────────────────┼──────────────────┼────────────────────────────────────┼──────────────────────┤
│ review_style                     │           string │                          3 (30.0%) │             category │
└──────────────────────────────────┴──────────────────┴────────────────────────────────────┴──────────────────────┘
                                                                                                                   
                                                                                                                   
                                             🗂️ LLM-Structured Columns                                              
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┓
┃                       ┃               ┃                            ┃     prompt tokens ┃      completion tokens ┃
┃ column name           ┃     data type ┃       number unique values ┃        per record ┃             per record ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩
│ product               │          dict │                10 (100.0%) │     268.5 +/- 0.8 │          70.0 +/- 10.2 │
├───────────────────────┼───────────────┼────────────────────────────┼───────────────────┼────────────────────────┤
│ customer_review       │          dict │                10 (100.0%) │     315.5 +/- 9.9 │         222.0 +/- 98.6 │
└───────────────────────┴───────────────┴────────────────────────────┴───────────────────┴────────────────────────┘
                                                                                                                   
                                                                                                                   
                                               🧩 Expression Columns                                               
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ column name                       ┃                data type ┃                             number unique values ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ customer_name                     │                   string │                                      10 (100.0%) │
├───────────────────────────────────┼──────────────────────────┼──────────────────────────────────────────────────┤
│ customer_age                      │                   string │                                      10 (100.0%) │
└───────────────────────────────────┴──────────────────────────┴──────────────────────────────────────────────────┘
                                                                                                                   
                                                                                                                   
╭────────────────────────────────────────────────── Table Notes ──────────────────────────────────────────────────╮
│                                                                                                                 │
│  1. All token statistics are based on a sample of max(1000, len(dataset)) records.                              │
│  2. Tokens are calculated using tiktoken's cl100k_base tokenizer.                                               │
│                                                                                                                 │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
                                                                                                                   
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────

⏭️ Next Steps¶

Check out the following notebook to learn more about:

Seeding synthetic data generation with an external dataset