๐จ Data Designer 101: The Basicsยถ
Click here to download this notebook to your computer.
๐ What you'll learnยถ
This notebook demonstrates the basics of Data Designer by generating a simple product review dataset.
๐ฆ Import the essentialsยถ
- The
essentialsmodule provides quick access to the most commonly used objects.
from data_designer.essentials import (
CategorySamplerParams,
DataDesigner,
DataDesignerConfigBuilder,
InferenceParameters,
LLMTextColumnConfig,
ModelConfig,
PersonSamplerParams,
SamplerColumnConfig,
SamplerType,
SubcategorySamplerParams,
UniformSamplerParams,
)
โ๏ธ Initialize the Data Designer interfaceยถ
DataDesigneris the main object that is used to interface with the library.
data_designer = DataDesigner()
๐๏ธ Define model configurationsยถ
Each
ModelConfigdefines a model that can be used during the generation process.The "model alias" is used to reference the model in the Data Designer config (as we will see below).
The "model provider" is the external service that hosts the model (see the model config docs for more details).
By default, we use build.nvidia.com as the model provider.
# This name is set in the model provider configuration.
MODEL_PROVIDER = "nvidia"
# The model ID is from build.nvidia.com.
MODEL_ID = "nvidia/nvidia-nemotron-nano-9b-v2"
# We choose this alias to be descriptive for our use case.
MODEL_ALIAS = "nemotron-nano-v2"
# This sets reasoning to False for the nemotron-nano-v2 model.
SYSTEM_PROMPT = "/no_think"
model_configs = [
ModelConfig(
alias=MODEL_ALIAS,
model=MODEL_ID,
provider=MODEL_PROVIDER,
inference_parameters=InferenceParameters(
temperature=0.5,
top_p=1.0,
max_tokens=1024,
),
)
]
๐๏ธ Initialize the Data Designer Config Builderยถ
The Data Designer config defines the dataset schema and generation process.
The config builder provides an intuitive interface for building this configuration.
The list of model configs is provided to the builder at initialization.
config_builder = DataDesignerConfigBuilder(model_configs=model_configs)
๐ฒ Getting started with sampler columnsยถ
Sampler columns offer non-LLM based generation of synthetic data.
They are particularly useful for steering the diversity of the generated data, as we demonstrate below.
You can view available samplers using the config builder's info property:
config_builder.info.display("samplers")
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ NeMo Data Designer Samplers โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโ โ Type โ Parameter โ Data Type โ Required โ Constraints โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ bernoulli โ p โ number โ โ โ >= 0.0, <= 1.0 โ โโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโค โ bernoulli_mixture โ p โ number โ โ โ >= 0.0, <= 1.0 โ โ โ dist_name โ string โ โ โ โ โ โ dist_params โ dict โ โ โ โ โโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโค โ binomial โ n โ integer โ โ โ โ โ โ p โ number โ โ โ >= 0.0, <= 1.0 โ โโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโค โ category โ values โ string[] | integer[] | โ โ โ len > 1 โ โ โ โ number[] โ โ โ โ โ weights โ number[] | null โ โ โ โโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโค โ datetime โ start โ string โ โ โ โ โ โ end โ string โ โ โ โ โ โ unit โ string โ โ โ โโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโค โ gaussian โ mean โ number โ โ โ โ โ โ stddev โ number โ โ โ โ โ โ decimal_places โ integer | null โ โ โ โโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโค โ person โ locale โ string โ โ โ โ โ sex โ string | null โ โ โ โ โ city โ string | string[] | null โ โ โ โ โ age_range โ integer[] โ โ len > 2, len < 2 โ โ โ state โ string | string[] | null โ โ โ โ โ with_synthetic_personas โ boolean โ โ โ โ โ sample_dataset_when_available โ boolean โ โ โ โโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโค โ poisson โ mean โ number โ โ โ โ โโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโค โ scipy โ dist_name โ string โ โ โ โ โ โ dist_params โ dict โ โ โ โ โ โ decimal_places โ integer | null โ โ โ โโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโค โ subcategory โ category โ string โ โ โ โ โ โ values โ dict โ โ โ โ โโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโค โ timedelta โ dt_min โ integer โ โ โ >= 0 โ โ โ dt_max โ integer โ โ โ > 0 โ โ โ reference_column_name โ string โ โ โ โ โ โ unit โ string โ โ โ โโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโค โ uniform โ low โ number โ โ โ โ โ โ high โ number โ โ โ โ โ โ decimal_places โ integer | null โ โ โ โโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโค โ uuid โ prefix โ string | null โ โ โ โ โ short_form โ boolean โ โ โ โ โ uppercase โ boolean โ โ โ โโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโ
Let's start designing our product review dataset by adding product category and subcategory columns.
config_builder.add_column(
SamplerColumnConfig(
name="product_category",
sampler_type=SamplerType.CATEGORY,
params=CategorySamplerParams(
values=[
"Electronics",
"Clothing",
"Home & Kitchen",
"Books",
"Home Office",
],
),
)
)
config_builder.add_column(
SamplerColumnConfig(
name="product_subcategory",
sampler_type=SamplerType.SUBCATEGORY,
params=SubcategorySamplerParams(
category="product_category",
values={
"Electronics": [
"Smartphones",
"Laptops",
"Headphones",
"Cameras",
"Accessories",
],
"Clothing": [
"Men's Clothing",
"Women's Clothing",
"Winter Coats",
"Activewear",
"Accessories",
],
"Home & Kitchen": [
"Appliances",
"Cookware",
"Furniture",
"Decor",
"Organization",
],
"Books": [
"Fiction",
"Non-Fiction",
"Self-Help",
"Textbooks",
"Classics",
],
"Home Office": [
"Desks",
"Chairs",
"Storage",
"Office Supplies",
"Lighting",
],
},
),
)
)
config_builder.add_column(
SamplerColumnConfig(
name="target_age_range",
sampler_type=SamplerType.CATEGORY,
params=CategorySamplerParams(values=["18-25", "25-35", "35-50", "50-65", "65+"]),
)
)
# Optionally validate that the columns are configured correctly.
config_builder.validate()
[19:00:10] [INFO] โ Validation passed
DataDesignerConfigBuilder( sampler_columns: ['product_category', 'product_subcategory', 'target_age_range'] )
Next, let's add samplers to generate data related to the customer and their review.
config_builder.add_column(
SamplerColumnConfig(
name="customer",
sampler_type=SamplerType.PERSON,
params=PersonSamplerParams(age_range=[18, 70], locale="en_GB"),
)
)
config_builder.add_column(
SamplerColumnConfig(
name="number_of_stars",
sampler_type=SamplerType.UNIFORM,
params=UniformSamplerParams(low=1, high=5),
convert_to="int", # Convert the sampled float to an integer.
)
)
config_builder.add_column(
SamplerColumnConfig(
name="review_style",
sampler_type=SamplerType.CATEGORY,
params=CategorySamplerParams(
values=["rambling", "brief", "detailed", "structured with bullet points"],
weights=[1, 2, 2, 1],
),
)
)
config_builder.validate()
[19:00:10] [INFO] โ Validation passed
DataDesignerConfigBuilder( sampler_columns: [ "product_category", "product_subcategory", "target_age_range", "customer", "number_of_stars", "review_style" ] )
๐ฆ LLM-generated columnsยถ
The real power of Data Designer comes from leveraging LLMs to generate text, code, and structured data.
When prompting the LLM, we can use Jinja templating to reference other columns in the dataset.
As we see below, nested json fields can be accessed using dot notation.
config_builder.add_column(
LLMTextColumnConfig(
name="product_name",
prompt=(
"You are a helpful assistant that generates product names. DO NOT add quotes around the product name.\n\n"
"Come up with a creative product name for a product in the '{{ product_category }}' category, focusing "
"on products related to '{{ product_subcategory }}'. The target age range of the ideal customer is "
"{{ target_age_range }} years old. Respond with only the product name, no other text."
),
system_prompt=SYSTEM_PROMPT,
model_alias=MODEL_ALIAS,
)
)
config_builder.add_column(
LLMTextColumnConfig(
name="customer_review",
prompt=(
"You are a customer named {{ customer.first_name }} from {{ customer.city }}, {{ customer.state }}. "
"You are {{ customer.age }} years old and recently purchased a product called {{ product_name }}. "
"Write a review of this product, which you gave a rating of {{ number_of_stars }} stars. "
"The style of the review should be '{{ review_style }}'."
),
system_prompt=SYSTEM_PROMPT,
model_alias=MODEL_ALIAS,
)
)
config_builder.validate()
[19:00:10] [INFO] โ Validation passed
DataDesignerConfigBuilder( sampler_columns: [ "product_category", "product_subcategory", "target_age_range", "customer", "number_of_stars", "review_style" ] llm_text_columns: ['product_name', 'customer_review'] )
๐ Iteration is key โย preview the dataset!ยถ
Use the
previewmethod to generate a sample of records quickly.Inspect the results for quality and format issues.
Adjust column configurations, prompts, or parameters as needed.
Re-run the preview until satisfied.
preview = data_designer.preview(config_builder)
[19:00:10] [INFO] ๐ง Preview generation in progress
[19:00:10] [INFO] โ
Validation passed
[19:00:10] [INFO] โ๏ธ Sorting column configs into a Directed Acyclic Graph
[19:00:10] [INFO] ๐ฉบ Running health checks for models...
[19:00:10] [INFO] |-- ๐ Checking 'nvidia/nvidia-nemotron-nano-9b-v2' in provider named 'nvidia' for model alias 'nemotron-nano-v2'...
[19:00:13] [INFO] |-- โ
Passed!
[19:00:13] [INFO] ๐ฒ Preparing samplers to generate 10 records across 6 columns
[19:00:13] [INFO] ๐ Preparing llm-text column generation
[19:00:13] [INFO] |-- column name: 'product_name'
[19:00:13] [INFO] |-- model config:
{
"alias": "nemotron-nano-v2",
"model": "nvidia/nvidia-nemotron-nano-9b-v2",
"inference_parameters": {
"temperature": 0.5,
"top_p": 1.0,
"max_tokens": 1024,
"max_parallel_requests": 4,
"timeout": null,
"extra_body": null
},
"provider": "nvidia"
}
[19:00:13] [INFO] ๐ Processing llm-text column 'product_name' with 4 concurrent workers
[19:00:17] [INFO] ๐ Preparing llm-text column generation
[19:00:17] [INFO] |-- column name: 'customer_review'
[19:00:17] [INFO] |-- model config:
{
"alias": "nemotron-nano-v2",
"model": "nvidia/nvidia-nemotron-nano-9b-v2",
"inference_parameters": {
"temperature": 0.5,
"top_p": 1.0,
"max_tokens": 1024,
"max_parallel_requests": 4,
"timeout": null,
"extra_body": null
},
"provider": "nvidia"
}
[19:00:17] [INFO] ๐ Processing llm-text column 'customer_review' with 4 concurrent workers
[19:00:31] [INFO] ๐ Model usage summary:
{
"nvidia/nvidia-nemotron-nano-9b-v2": {
"token_usage": {
"prompt_tokens": 1639,
"completion_tokens": 5642,
"total_tokens": 7281
},
"request_usage": {
"successful_requests": 20,
"failed_requests": 0,
"total_requests": 20
},
"tokens_per_second": 398,
"requests_per_minute": 65
}
}
[19:00:31] [INFO] ๐ Measuring dataset column statistics:
[19:00:31] [INFO] |-- ๐ฒ column: 'product_category'
[19:00:31] [INFO] |-- ๐ฒ column: 'product_subcategory'
[19:00:31] [INFO] |-- ๐ฒ column: 'target_age_range'
[19:00:31] [INFO] |-- ๐ฒ column: 'customer'
[19:00:31] [INFO] |-- ๐ฒ column: 'number_of_stars'
[19:00:31] [INFO] |-- ๐ฒ column: 'review_style'
[19:00:31] [INFO] |-- ๐ column: 'product_name'
[19:00:31] [INFO] |-- ๐ column: 'customer_review'
[19:00:31] [INFO] ๐ Preview complete!
# Run this cell multiple times to cycle through the 10 preview records.
preview.display_sample_record()
Generated Columns โโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ Name โ Value โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ product_category โ Home & Kitchen โ โโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ product_subcategory โ Organization โ โโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ target_age_range โ 50-65 โ โโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ customer โ { โ โ โ 'uuid': '67c0a53a-ddfb-49c4-adea-69a220eef85f', โ โ โ 'locale': 'en_GB', โ โ โ 'first_name': 'Angela', โ โ โ 'last_name': 'Atkins', โ โ โ 'middle_name': None, โ โ โ 'sex': 'Female', โ โ โ 'street_number': '3', โ โ โ 'street_name': 'Ashley turnpike', โ โ โ 'city': 'Mohammedview', โ โ โ 'state': None, โ โ โ 'postcode': 'WV02 7UH', โ โ โ 'age': 31, โ โ โ 'birth_date': '1994-05-17', โ โ โ 'country': 'Bahamas', โ โ โ 'marital_status': 'widowed', โ โ โ 'education_level': 'doctorate', โ โ โ 'unit': '', โ โ โ 'occupation': 'Metallurgist', โ โ โ 'phone_number': '0117 496 0534', โ โ โ 'bachelors_field': 'arts_humanities' โ โ โ } โ โโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ number_of_stars โ 4 โ โโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ review_style โ detailed โ โโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ product_name โ CozyNest Organizer โ โ โ โ โโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ customer_review โ **Product Review: CozyNest Organizer (4 Stars)** โ โ โ *By Angela, 31* โ โ โ โ โ โ I recently purchased the CozyNest Organizer from Mohammedview, and Iโm thrilled to share โ โ โ my experience. As someone whoโs always struggled with clutter in my workspace and home, I โ โ โ was excited to try this product, and Iโm glad I didโthough I have a few thoughts to โ โ โ share. โ โ โ โ โ โ **Design and Build Quality** โ โ โ The CozyNest Organizer is a sleek, modern piece that immediately caught my eye. Its โ โ โ minimalist design blends well with my existing decor, and the materials feel durableโno โ โ โ flimsy plastic or cheap finishes. The organizers are made of a sturdy, eco-friendly โ โ โ fabric thatโs both soft to the touch and resistant to stains, which is a huge plus. I โ โ โ appreciate the attention to detail, like the reinforced stitching and the way the โ โ โ compartments are thoughtfully sized for different items. Itโs not just a pretty box; itโs โ โ โ built to last. โ โ โ โ โ โ **Functionality** โ โ โ This is where the CozyNest truly shines. The organizer comes with multiple compartments, โ โ โ each designed for specific purposesโpens, papers, small electronics, and even a hidden โ โ โ pocket for receipts or keys. Iโve used it in my office to keep my desk tidy, and itโs โ โ โ been a game-changer. The layout is intuitive, and I can easily find what I need without โ โ โ rummaging through a pile of items. However, I did find that the largest compartment is a โ โ โ bit smaller than I expected, which made it a bit tight when storing bulkier items. That โ โ โ said, itโs perfect for everyday essentials. โ โ โ โ โ โ **Ease of Use and Assembly** โ โ โ Setting up the CozyNest was straightforward. The instructions were clear, and it took me โ โ โ about 10 minutes to assemble. The pieces fit together securely, and there were no tools โ โ โ required, which is a big win. Once assembled, itโs easy to move around or adjust as โ โ โ needed. I also love that itโs lightweight yet stable, so it doesnโt tip over easily. โ โ โ โ โ โ **Comfort and Practicality** โ โ โ One of the standout features is how comfortable it is to use. The fabric is soft enough โ โ โ that I donโt mind touching it frequently, and the compartments are deep enough to hold โ โ โ items without feeling cramped. Iโve used it to organize my craft supplies at home, and โ โ โ itโs made my creative sessions much more enjoyable. The only minor downside is that the โ โ โ fabric isnโt waterproof, so I had to be careful with spills. But for general use, this โ โ โ hasnโt been an issue. โ โ โ โ โ โ **Value for Money** โ โ โ At the price point, I feel the CozyNest Organizer is well worth it. Itโs not the cheapest โ โ โ option on the market, but the quality and functionality justify the cost. Iโve already โ โ โ noticed a significant reduction in clutter, and itโs helped me stay more organized in โ โ โ both my personal and professional life. โ โ โ โ โ โ **Final Thoughts** โ โ โ While I gave it 4 stars instead of 5, itโs because of the slightly smaller main โ โ โ compartment and the lack of waterproofing. These are minor flaws, though, and they donโt โ โ โ detract from the overall value. If youโre looking for a stylish, functional organizer โ โ โ thatโs easy to use and durable, the CozyNest is a fantastic choice. Iโd definitely โ โ โ recommend it to anyone needing a reliable way to declutter their space. โ โ โ โ โ โ Thanks to Mohammedview for offering such a great productโIโm already considering โ โ โ purchasing another one for my home! โ โ โ โ โโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ [index: 0]
# The preview dataset is available as a pandas DataFrame.
preview.dataset
| product_category | product_subcategory | target_age_range | customer | number_of_stars | review_style | product_name | customer_review | |
|---|---|---|---|---|---|---|---|---|
| 0 | Home & Kitchen | Organization | 50-65 | {'uuid': '67c0a53a-ddfb-49c4-adea-69a220eef85f... | 4 | detailed | CozyNest Organizer\n | **Product Review: CozyNest Organizer (4 Stars)... |
| 1 | Books | Textbooks | 18-25 | {'uuid': 'cb555c14-2773-4b91-9052-d8f736b34c79... | 4 | brief | SmartStudyGuides\n | **Review:** \nSmartStudyGuides is a solid too... |
| 2 | Home & Kitchen | Organization | 35-50 | {'uuid': 'd140c09d-ff91-4259-86bd-bfa86365f8b0... | 4 | detailed | SmartSpace Organizer\n | **Product Review: SmartSpace Organizer (4 Star... |
| 3 | Home Office | Lighting | 25-35 | {'uuid': '386fc57c-e769-442c-8fbb-7e92348fe052... | 5 | structured with bullet points | BrightNest\n | **Trevor's Review of BrightNest (5 Stars) โ La... |
| 4 | Home & Kitchen | Furniture | 25-35 | {'uuid': 'a2dd98a8-6e95-4953-9c6d-0e35c40cb976... | 3 | detailed | ModernNest\n | **Product Review: ModernNest โ A 3-Star Experi... |
| 5 | Books | Fiction | 35-50 | {'uuid': '4156a3d2-79d4-4b72-9fdb-4dd22b477935... | 3 | detailed | Whispers of the Forgotten Realm\n | **Review Title:** A Mixed Experience with "Whi... |
| 6 | Electronics | Cameras | 65+ | {'uuid': '44af7f7c-2aa4-4cba-8099-41b47767c0f8... | 3 | detailed | SeniorShutter\n | **Product Review: SeniorShutter โ A Mixed Expe... |
| 7 | Clothing | Men's Clothing | 50-65 | {'uuid': '91d59484-dba8-494e-a9c1-f5e90044ac2b... | 4 | rambling | TimelessTactics\n | Okay, so I bought this thing called TimelessTa... |
| 8 | Home Office | Desks | 25-35 | {'uuid': '8b5c6ead-a39b-4ddf-bf36-c91af15cfe90... | 5 | detailed | WorkNest Pro\n | **Product Review: WorkNest Pro โ A Game-Change... |
| 9 | Books | Self-Help | 50-65 | {'uuid': '7ba8a0a6-f2a9-4145-b887-64957ee40158... | 4 | structured with bullet points | Mindful Moments\n | **Product Review: Mindful Moments โ 4 Stars** ... |
๐ Analyze the generated dataยถ
Data Designer automatically generates a basic statistical analysis of the generated data.
This analysis is available via the
analysisproperty of generation result objects.
# Print the analysis as a table.
preview.analysis.to_report()
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ ๐จ Data Designer Dataset Profile โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ Dataset Overview โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ number of records โ number of columns โ percent complete records โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ 10 โ 8 โ 100.0% โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ ๐ฒ Sampler Columns โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโ โ column name โ data type โ number unique values โ sampler type โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ product_category โ string โ 5 (50.0%) โ category โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโค โ product_subcategory โ string โ 9 (90.0%) โ subcategory โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโค โ target_age_range โ string โ 5 (50.0%) โ category โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโค โ customer โ dict โ 10 (100.0%) โ person โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโค โ number_of_stars โ int โ 3 (30.0%) โ uniform โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโค โ review_style โ string โ 4 (40.0%) โ category โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโ ๐ LLM-Text Columns โโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ โ โ prompt tokens โ completion tokens โ โ column name โ data type โ number unique values โ per record โ per record โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ product_name โ string โ 10 (100.0%) โ 77.0 +/- 0.7 โ 4.5 +/- 1.0 โ โโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโค โ customer_review โ string โ 10 (100.0%) โ 63.0 +/- 1.5 โ 625.0 +/- 281.1 โ โโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโ โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ Table Notes โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ โ โ โ 1. All token statistics are based on a sample of max(1000, len(dataset)) records. โ โ 2. Tokens are calculated using tiktoken's cl100k_base tokenizer. โ โ โ โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
๐ Scale up!ยถ
Happy with your preview data?
Use the
createmethod to submit larger Data Designer generation jobs.
results = data_designer.create(config_builder, num_records=20)
[19:00:31] [INFO] ๐จ Creating Data Designer dataset
[19:00:31] [INFO] โ
Validation passed
[19:00:31] [INFO] โ๏ธ Sorting column configs into a Directed Acyclic Graph
[19:00:31] [INFO] ๐ฉบ Running health checks for models...
[19:00:31] [INFO] |-- ๐ Checking 'nvidia/nvidia-nemotron-nano-9b-v2' in provider named 'nvidia' for model alias 'nemotron-nano-v2'...
[19:00:32] [INFO] |-- โ
Passed!
[19:00:32] [INFO] โณ Processing batch 1 of 1
[19:00:32] [INFO] ๐ฒ Preparing samplers to generate 20 records across 6 columns
[19:00:32] [INFO] ๐ Preparing llm-text column generation
[19:00:32] [INFO] |-- column name: 'product_name'
[19:00:32] [INFO] |-- model config:
{
"alias": "nemotron-nano-v2",
"model": "nvidia/nvidia-nemotron-nano-9b-v2",
"inference_parameters": {
"temperature": 0.5,
"top_p": 1.0,
"max_tokens": 1024,
"max_parallel_requests": 4,
"timeout": null,
"extra_body": null
},
"provider": "nvidia"
}
[19:00:32] [INFO] ๐ Processing llm-text column 'product_name' with 4 concurrent workers
[19:00:43] [INFO] ๐ Preparing llm-text column generation
[19:00:43] [INFO] |-- column name: 'customer_review'
[19:00:43] [INFO] |-- model config:
{
"alias": "nemotron-nano-v2",
"model": "nvidia/nvidia-nemotron-nano-9b-v2",
"inference_parameters": {
"temperature": 0.5,
"top_p": 1.0,
"max_tokens": 1024,
"max_parallel_requests": 4,
"timeout": null,
"extra_body": null
},
"provider": "nvidia"
}
[19:00:43] [INFO] ๐ Processing llm-text column 'customer_review' with 4 concurrent workers
[19:01:10] [INFO] ๐ Model usage summary:
{
"nvidia/nvidia-nemotron-nano-9b-v2": {
"token_usage": {
"prompt_tokens": 3263,
"completion_tokens": 9038,
"total_tokens": 12301
},
"request_usage": {
"successful_requests": 40,
"failed_requests": 0,
"total_requests": 40
},
"tokens_per_second": 323,
"requests_per_minute": 63
}
}
[19:01:10] [INFO] ๐ Measuring dataset column statistics:
[19:01:10] [INFO] |-- ๐ฒ column: 'product_category'
[19:01:10] [INFO] |-- ๐ฒ column: 'product_subcategory'
[19:01:10] [INFO] |-- ๐ฒ column: 'target_age_range'
[19:01:10] [INFO] |-- ๐ฒ column: 'customer'
[19:01:10] [INFO] |-- ๐ฒ column: 'number_of_stars'
[19:01:10] [INFO] |-- ๐ฒ column: 'review_style'
[19:01:10] [INFO] |-- ๐ column: 'product_name'
[19:01:10] [INFO] |-- ๐ column: 'customer_review'
# Load the generated dataset as a pandas DataFrame.
dataset = results.load_dataset()
dataset.head()
| product_category | product_subcategory | target_age_range | customer | number_of_stars | review_style | product_name | customer_review | |
|---|---|---|---|---|---|---|---|---|
| 0 | Electronics | Headphones | 35-50 | {'age': 44, 'bachelors_field': 'arts_humanitie... | 3 | detailed | SoundSage | **Review Title: A Mixed Experience with SoundS... |
| 1 | Home Office | Storage | 50-65 | {'age': 52, 'bachelors_field': 'no_degree', 'b... | 3 | structured with bullet points | TimelessTote | **Review of TimelessTote (3 Stars)** - **De... |
| 2 | Electronics | Accessories | 25-35 | {'age': 56, 'bachelors_field': 'no_degree', 'b... | 4 | detailed | NeonPulse | **Product Review: NeonPulse by Mathew from Kie... |
| 3 | Electronics | Smartphones | 65+ | {'age': 48, 'bachelors_field': 'no_degree', 'b... | 4 | detailed | SeniorSense Smartphone | **Product Review: SeniorSense Smartphone โ A T... |
| 4 | Home Office | Desks | 35-50 | {'age': 48, 'bachelors_field': 'no_degree', 'b... | 2 | brief | WorkspaceElevate | **Review:** Disappointing. Expected more from ... |
# Load the analysis results into memory.
analysis = results.load_analysis()
analysis.to_report()
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ ๐จ Data Designer Dataset Profile โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ Dataset Overview โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ number of records โ number of columns โ percent complete records โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ 20 โ 8 โ 100.0% โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ ๐ฒ Sampler Columns โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโ โ column name โ data type โ number unique values โ sampler type โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ product_category โ string โ 5 (25.0%) โ category โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโค โ product_subcategory โ string โ 14 (70.0%) โ subcategory โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโค โ target_age_range โ string โ 5 (25.0%) โ category โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโค โ customer โ dict โ 20 (100.0%) โ person โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโค โ number_of_stars โ int โ 4 (20.0%) โ uniform โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโค โ review_style โ string โ 4 (20.0%) โ category โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโ ๐ LLM-Text Columns โโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ โ โ prompt tokens โ completion tokens โ โ column name โ data type โ number unique values โ per record โ per record โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ product_name โ string โ 20 (100.0%) โ 77.0 +/- 0.8 โ 4.5 +/- 0.9 โ โโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโค โ customer_review โ string โ 20 (100.0%) โ 62.0 +/- 2.1 โ 406.5 +/- 322.5 โ โโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโ โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ Table Notes โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ โ โ โ 1. All token statistics are based on a sample of max(1000, len(dataset)) records. โ โ 2. Tokens are calculated using tiktoken's cl100k_base tokenizer. โ โ โ โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โญ๏ธ Next Stepsยถ
Now that you've seen the basics of Data Designer, check out the following notebooks to learn more about: