๐จ Data Designer Tutorial: Seeding Synthetic Data Generation with an External Datasetยถ
๐ What you'll learnยถ
In this notebook, we will demonstrate how to seed synthetic data generation in Data Designer with an external dataset.
If this is your first time using Data Designer, we recommend starting with the first notebook in this tutorial series.
๐ฆ Import Data Designerยถ
data_designer.configprovides access to the configuration API.DataDesigneris the main interface for data generation.
Inย [1]:
Copied!
import data_designer.config as dd
from data_designer.interface import DataDesigner
import data_designer.config as dd
from data_designer.interface import DataDesigner
โ๏ธ Initialize the Data Designer interfaceยถ
DataDesigneris the main object responsible for managing the data generation process.When initialized without arguments, the default model providers are used.
Inย [2]:
Copied!
data_designer = DataDesigner()
data_designer = DataDesigner()
๐๏ธ Define model configurationsยถ
Each
ModelConfigdefines a model that can be used during the generation process.The "model alias" is used to reference the model in the Data Designer config (as we will see below).
The "model provider" is the external service that hosts the model (see the model config docs for more details).
By default, we use build.nvidia.com as the model provider.
Inย [3]:
Copied!
# This name is set in the model provider configuration.
MODEL_PROVIDER = "nvidia"
# The model ID is from build.nvidia.com.
MODEL_ID = "nvidia/nemotron-3-nano-30b-a3b"
# We choose this alias to be descriptive for our use case.
MODEL_ALIAS = "nemotron-nano-v3"
model_configs = [
dd.ModelConfig(
alias=MODEL_ALIAS,
model=MODEL_ID,
provider=MODEL_PROVIDER,
inference_parameters=dd.ChatCompletionInferenceParams(
temperature=1.0,
top_p=1.0,
max_tokens=2048,
extra_body={"chat_template_kwargs": {"enable_thinking": False}},
),
)
]
# This name is set in the model provider configuration.
MODEL_PROVIDER = "nvidia"
# The model ID is from build.nvidia.com.
MODEL_ID = "nvidia/nemotron-3-nano-30b-a3b"
# We choose this alias to be descriptive for our use case.
MODEL_ALIAS = "nemotron-nano-v3"
model_configs = [
dd.ModelConfig(
alias=MODEL_ALIAS,
model=MODEL_ID,
provider=MODEL_PROVIDER,
inference_parameters=dd.ChatCompletionInferenceParams(
temperature=1.0,
top_p=1.0,
max_tokens=2048,
extra_body={"chat_template_kwargs": {"enable_thinking": False}},
),
)
]
๐๏ธ Initialize the Data Designer Config Builderยถ
The Data Designer config defines the dataset schema and generation process.
The config builder provides an intuitive interface for building this configuration.
The list of model configs is provided to the builder at initialization.
Inย [4]:
Copied!
config_builder = dd.DataDesignerConfigBuilder(model_configs=model_configs)
config_builder = dd.DataDesignerConfigBuilder(model_configs=model_configs)
๐ฅ Prepare a seed datasetยถ
For this notebook, we'll create a synthetic dataset of patient notes.
We will seed the generation process with a symptom-to-diagnosis dataset.
We already have the dataset downloaded in the data directory of this repository.
๐ฑ Why use a seed dataset?
Seed datasets let you steer the generation process by providing context that is specific to your use case.
Seed datasets are also an excellent way to inject real-world diversity into your synthetic data.
During generation, prompt templates can reference any of the seed dataset fields.
Inย [5]:
Copied!
# Download sample dataset from Github
import urllib.request
url = "https://raw.githubusercontent.com/NVIDIA/GenerativeAIExamples/refs/heads/main/nemo/NeMo-Data-Designer/data/gretelai_symptom_to_diagnosis.csv"
local_filename, _ = urllib.request.urlretrieve(url, "gretelai_symptom_to_diagnosis.csv")
# Seed datasets are passed as reference objects to the config builder.
seed_source = dd.LocalFileSeedSource(path=local_filename)
config_builder.with_seed_dataset(seed_source)
# Download sample dataset from Github
import urllib.request
url = "https://raw.githubusercontent.com/NVIDIA/GenerativeAIExamples/refs/heads/main/nemo/NeMo-Data-Designer/data/gretelai_symptom_to_diagnosis.csv"
local_filename, _ = urllib.request.urlretrieve(url, "gretelai_symptom_to_diagnosis.csv")
# Seed datasets are passed as reference objects to the config builder.
seed_source = dd.LocalFileSeedSource(path=local_filename)
config_builder.with_seed_dataset(seed_source)
Out[5]:
DataDesignerConfigBuilder()
๐จ Designing our synthetic patient notes datasetยถ
- The prompt template can reference fields from our seed dataset:
{{ diagnosis }}- the medical diagnosis from the seed data{{ patient_summary }}- the symptom description from the seed data
Inย [6]:
Copied!
config_builder.add_column(
dd.SamplerColumnConfig(
name="patient_sampler",
sampler_type=dd.SamplerType.PERSON_FROM_FAKER,
params=dd.PersonFromFakerSamplerParams(),
)
)
config_builder.add_column(
dd.SamplerColumnConfig(
name="doctor_sampler",
sampler_type=dd.SamplerType.PERSON_FROM_FAKER,
params=dd.PersonFromFakerSamplerParams(),
)
)
config_builder.add_column(
dd.SamplerColumnConfig(
name="patient_id",
sampler_type=dd.SamplerType.UUID,
params=dd.UUIDSamplerParams(
prefix="PT-",
short_form=True,
uppercase=True,
),
)
)
config_builder.add_column(dd.ExpressionColumnConfig(name="first_name", expr="{{ patient_sampler.first_name }}"))
config_builder.add_column(dd.ExpressionColumnConfig(name="last_name", expr="{{ patient_sampler.last_name }}"))
config_builder.add_column(dd.ExpressionColumnConfig(name="dob", expr="{{ patient_sampler.birth_date }}"))
config_builder.add_column(
dd.SamplerColumnConfig(
name="symptom_onset_date",
sampler_type=dd.SamplerType.DATETIME,
params=dd.DatetimeSamplerParams(start="2024-01-01", end="2024-12-31"),
)
)
config_builder.add_column(
dd.SamplerColumnConfig(
name="date_of_visit",
sampler_type=dd.SamplerType.TIMEDELTA,
params=dd.TimeDeltaSamplerParams(dt_min=1, dt_max=30, reference_column_name="symptom_onset_date"),
)
)
config_builder.add_column(dd.ExpressionColumnConfig(name="physician", expr="Dr. {{ doctor_sampler.last_name }}"))
config_builder.add_column(
dd.LLMTextColumnConfig(
name="physician_notes",
prompt="""\
You are a primary-care physician who just had an appointment with {{ first_name }} {{ last_name }},
who has been struggling with symptoms from {{ diagnosis }} since {{ symptom_onset_date }}.
The date of today's visit is {{ date_of_visit }}.
{{ patient_summary }}
Write careful notes about your visit with {{ first_name }},
as Dr. {{ doctor_sampler.first_name }} {{ doctor_sampler.last_name }}.
Format the notes as a busy doctor might.
Respond with only the notes, no other text.
""",
model_alias=MODEL_ALIAS,
)
)
data_designer.validate(config_builder)
config_builder.add_column(
dd.SamplerColumnConfig(
name="patient_sampler",
sampler_type=dd.SamplerType.PERSON_FROM_FAKER,
params=dd.PersonFromFakerSamplerParams(),
)
)
config_builder.add_column(
dd.SamplerColumnConfig(
name="doctor_sampler",
sampler_type=dd.SamplerType.PERSON_FROM_FAKER,
params=dd.PersonFromFakerSamplerParams(),
)
)
config_builder.add_column(
dd.SamplerColumnConfig(
name="patient_id",
sampler_type=dd.SamplerType.UUID,
params=dd.UUIDSamplerParams(
prefix="PT-",
short_form=True,
uppercase=True,
),
)
)
config_builder.add_column(dd.ExpressionColumnConfig(name="first_name", expr="{{ patient_sampler.first_name }}"))
config_builder.add_column(dd.ExpressionColumnConfig(name="last_name", expr="{{ patient_sampler.last_name }}"))
config_builder.add_column(dd.ExpressionColumnConfig(name="dob", expr="{{ patient_sampler.birth_date }}"))
config_builder.add_column(
dd.SamplerColumnConfig(
name="symptom_onset_date",
sampler_type=dd.SamplerType.DATETIME,
params=dd.DatetimeSamplerParams(start="2024-01-01", end="2024-12-31"),
)
)
config_builder.add_column(
dd.SamplerColumnConfig(
name="date_of_visit",
sampler_type=dd.SamplerType.TIMEDELTA,
params=dd.TimeDeltaSamplerParams(dt_min=1, dt_max=30, reference_column_name="symptom_onset_date"),
)
)
config_builder.add_column(dd.ExpressionColumnConfig(name="physician", expr="Dr. {{ doctor_sampler.last_name }}"))
config_builder.add_column(
dd.LLMTextColumnConfig(
name="physician_notes",
prompt="""\
You are a primary-care physician who just had an appointment with {{ first_name }} {{ last_name }},
who has been struggling with symptoms from {{ diagnosis }} since {{ symptom_onset_date }}.
The date of today's visit is {{ date_of_visit }}.
{{ patient_summary }}
Write careful notes about your visit with {{ first_name }},
as Dr. {{ doctor_sampler.first_name }} {{ doctor_sampler.last_name }}.
Format the notes as a busy doctor might.
Respond with only the notes, no other text.
""",
model_alias=MODEL_ALIAS,
)
)
data_designer.validate(config_builder)
[12:08:06] [INFO] โ Validation passed
๐ Iteration is key โย preview the dataset!ยถ
Use the
previewmethod to generate a sample of records quickly.Inspect the results for quality and format issues.
Adjust column configurations, prompts, or parameters as needed.
Re-run the preview until satisfied.
Inย [7]:
Copied!
preview = data_designer.preview(config_builder, num_records=2)
preview = data_designer.preview(config_builder, num_records=2)
[12:08:06] [INFO] ๐ Preview generation in progress
[12:08:06] [INFO] โ Validation passed
[12:08:06] [INFO] โ๏ธ Sorting column configs into a Directed Acyclic Graph
[12:08:06] [INFO] ๐ฉบ Running health checks for models...
[12:08:06] [INFO] |-- ๐ Checking 'nvidia/nemotron-3-nano-30b-a3b' in provider named 'nvidia' for model alias 'nemotron-nano-v3'...
[12:08:07] [INFO] |-- โ Passed!
[12:08:07] [INFO] ๐ฑ Sampling 2 records from seed dataset
[12:08:07] [INFO] |-- seed dataset size: 820 records
[12:08:07] [INFO] |-- sampling strategy: ordered
[12:08:07] [INFO] ๐ฒ Preparing samplers to generate 2 records across 5 columns
[12:08:07] [INFO] (๐พ + ๐พ) Concatenating 2 datasets
[12:08:07] [INFO] ๐งฉ Generating column `first_name` from expression
[12:08:07] [INFO] ๐งฉ Generating column `last_name` from expression
[12:08:07] [INFO] ๐งฉ Generating column `dob` from expression
[12:08:07] [INFO] ๐งฉ Generating column `physician` from expression
[12:08:07] [INFO] ๐ llm-text model config for column 'physician_notes'
[12:08:07] [INFO] |-- model: 'nvidia/nemotron-3-nano-30b-a3b'
[12:08:07] [INFO] |-- model alias: 'nemotron-nano-v3'
[12:08:07] [INFO] |-- model provider: 'nvidia'
[12:08:07] [INFO] |-- inference parameters:
[12:08:07] [INFO] | |-- generation_type=chat-completion
[12:08:07] [INFO] | |-- max_parallel_requests=4
[12:08:07] [INFO] | |-- extra_body={'chat_template_kwargs': {'enable_thinking': False}}
[12:08:07] [INFO] | |-- temperature=1.00
[12:08:07] [INFO] | |-- top_p=1.00
[12:08:07] [INFO] | |-- max_tokens=2048
[12:08:07] [INFO] โก๏ธ Processing llm-text column 'physician_notes' with 4 concurrent workers
[12:08:07] [INFO] โฑ๏ธ llm-text column 'physician_notes' will report progress after each record
[12:08:14] [INFO] |-- ๐ธ llm-text column 'physician_notes' progress: 1/2 (50%) complete, 1 ok, 0 failed, 0.13 rec/s, eta 7.7s
[12:08:16] [INFO] |-- ๐ฆ llm-text column 'physician_notes' progress: 2/2 (100%) complete, 2 ok, 0 failed, 0.20 rec/s, eta 0.0s
[12:08:17] [INFO] ๐ Model usage summary:
[12:08:17] [INFO] |-- model: nvidia/nemotron-3-nano-30b-a3b
[12:08:17] [INFO] |-- tokens: input=292, output=2334, total=2626, tps=256
[12:08:17] [INFO] |-- requests: success=2, failed=0, total=2, rpm=11
[12:08:17] [INFO] ๐ Measuring dataset column statistics:
[12:08:17] [INFO] |-- ๐ฒ column: 'patient_sampler'
[12:08:17] [INFO] |-- ๐ฒ column: 'doctor_sampler'
[12:08:17] [INFO] |-- ๐ฒ column: 'patient_id'
[12:08:17] [INFO] |-- ๐งฉ column: 'first_name'
[12:08:17] [INFO] |-- ๐งฉ column: 'last_name'
[12:08:17] [INFO] |-- ๐งฉ column: 'dob'
[12:08:17] [INFO] |-- ๐ฒ column: 'symptom_onset_date'
[12:08:17] [INFO] |-- ๐ฒ column: 'date_of_visit'
[12:08:17] [INFO] |-- ๐งฉ column: 'physician'
[12:08:17] [INFO] |-- ๐ column: 'physician_notes'
[12:08:17] [INFO] โ๏ธ Preview complete!
Inย [8]:
Copied!
# Run this cell multiple times to cycle through the 2 preview records.
preview.display_sample_record()
# Run this cell multiple times to cycle through the 2 preview records.
preview.display_sample_record()
Seed Columns โโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ Name โ Value โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ diagnosis โ cervical spondylosis โ โโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ patient_summary โ I've been having a lot of pain in my neck and back. I've also been having trouble with โ โ โ my balance and coordination. I've been coughing a lot and my limbs feel weak. โ โโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ Generated Columns โโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ Name โ Value โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ patient_sampler โ { โ โ โ 'uuid': 'e5e0be22-7f5e-42a8-8b7e-d5b1a6ae30d6', โ โ โ 'locale': 'en_US', โ โ โ 'first_name': 'James', โ โ โ 'last_name': 'Duncan', โ โ โ 'middle_name': None, โ โ โ 'sex': 'Male', โ โ โ 'street_number': '841', โ โ โ 'street_name': 'Willie Haven', โ โ โ 'city': 'North Steven', โ โ โ 'state': 'Rhode Island', โ โ โ 'postcode': '03246', โ โ โ 'age': 105, โ โ โ 'birth_date': '1920-03-06', โ โ โ 'country': 'Guernsey', โ โ โ 'marital_status': 'widowed', โ โ โ 'education_level': 'associates', โ โ โ 'unit': '', โ โ โ 'occupation': 'Bonds trader', โ โ โ 'phone_number': '001-500-310-4423', โ โ โ 'bachelors_field': 'no_degree' โ โ โ } โ โโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ doctor_sampler โ { โ โ โ 'uuid': '4ab78e72-2150-434c-8842-6ced4510158f', โ โ โ 'locale': 'en_US', โ โ โ 'first_name': 'Kim', โ โ โ 'last_name': 'Stevenson', โ โ โ 'middle_name': None, โ โ โ 'sex': 'Female', โ โ โ 'street_number': '659', โ โ โ 'street_name': 'Hannah Rapids', โ โ โ 'city': 'Lake Karen', โ โ โ 'state': 'North Carolina', โ โ โ 'postcode': '67107', โ โ โ 'age': 100, โ โ โ 'birth_date': '1925-09-01', โ โ โ 'country': 'Northern Mariana Islands', โ โ โ 'marital_status': 'divorced', โ โ โ 'education_level': 'graduate', โ โ โ 'unit': '', โ โ โ 'occupation': 'Garment/textile technologist', โ โ โ 'phone_number': '001-230-953-0462x8385', โ โ โ 'bachelors_field': 'stem_related' โ โ โ } โ โโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ patient_id โ PT-E350F177 โ โโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ symptom_onset_date โ 2024-02-14 โ โโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ date_of_visit โ 2024-03-14 โ โโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ first_name โ James โ โโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ last_name โ Duncan โ โโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ dob โ 1920-03-06 โ โโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ physician โ Dr. Stevenson โ โโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ physician_notes โ **Visit Summary โ Dr. Kim Stevenson** โ โ โ **Patient:** James Duncan โ 68 yo M โ โ โ **Date:** 2024โ03โ14 โ โ โ **Chief Complaint:** Newโonset neck/back pain, worsening balance, frequent cough, โ โ โ generalized weakness โ โ โ โ โ โ --- โ โ โ โ โ โ ### HPI โ โ โ - **Onset:** 2024โ02โ14 (โ1โฏmo after initial symptoms) โ progressive worsening. โ โ โ - **Location:** Anterior neck, midโcervical region; radiating to upper back. โ โ โ - **Character:** Dullโachy pain, 6/10 at rest, 8โ9/10 with movement; constant. โ โ โ - **Associated symptoms:** โ โ โ - **Neurologic:** Frequent falls, unsteady gait, difficulty negotiating stairs; โ โ โ sensation of โnumbnessโ in hands; occasional tingling in fingers. โ โ โ - **Respiratory:** Chronic cough (dry, nonโproductive) โ no fever, no sputum. โ โ โ - **Motor:** Weakness noted in both arms and legs (estimated 4/5) especially โ โ โ proximal proximal muscles; trouble lifting objects. โ โ โ - **General:** Mild fatigue; denies fever, weight loss, recent infections. โ โ โ - **Exacerbating factors:** Poor posture (desk work), neck flexion/extension, โ โ โ standing up from seated position. โ โ โ - **Alleviating factors:** Short rest, analgesics (acetaminophen) modest relief; heat โ โ โ application helps briefly. โ โ โ - **Medications:** OTC acetaminophen PRN, previous physical therapy (completed 2023) โ โ โ with minimal benefit. โ โ โ - **Past medical history:** Hypertension, prediabetes, mild osteoarthritis of knees. โ โ โ - **Social:** Former smoker (10 packโyr, quit 2005); lives alone; active gardening โ โ โ but limited now. โ โ โ - **Family:** No known neurologic or connective tissue disease. โ โ โ โ โ โ ### ROS (pertinent) โ โ โ - **Neck/Spine:** Chronic pain, stiffness; limited range of motion. โ โ โ - **Neurologic:** Episodes of dizziness; occasional bladder urgency (no โ โ โ incontinence). โ โ โ - **Respiratory:** Persistent cough; denies wheeze or dyspnea at rest. โ โ โ - **Skin:** No rashes. โ โ โ - **GI/ GU:** Normal. โ โ โ โ โ โ ### PE โ โ โ - **Vitals:** BP 138/84, HR 78, RR 16, Temp 98.4ยฐF, SpOโ 98% RA. โ โ โ - **General:** Alert, oriented ร3, mild discomfort on movement. โ โ โ - **Neck:** Limited flexion/extension; tenderness at C3โC5; no crepitus on passive โ โ โ motion. โ โ โ - **Upper Extremities:** Strength 4/5 proximally, 5/5 distally; decreased grip. โ โ โ Reflexes 2+ throughout; mild hyperreflexia in biceps. No clonus. โ โ โ - **Lower Extremities:** Strength 4/5; gait unsteady, uses cane; tandem walk positive โ โ โ (loss of balance). Reflexes 2+; Babinski negative. โ โ โ - **Respiratory:** Clear breath sounds bilaterally; no wheezes. โ โ โ - **Cardiac:** Regular rate/rhythm; no murmurs. โ โ โ - **Neurologic:** Positive Romberg; gait ataxic with narrow base; proprioception โ โ โ mildly impaired in lower extremities. โ โ โ โ โ โ ### Assessment โ โ โ 1. **Cervical spondylosis with progressive myelopathic signs** โ cervical โ โ โ radiculopathy extending to myelopathy (balance, gait, upper/lower limb weakness). โ โ โ 2. **Chronic cough likely secondary to postโnasal drip / GERD vs. early respiratory โ โ โ pathology** โ not currently dominant but warrants evaluation. โ โ โ 3. **Ageโrelated musculoskeletal pain** (neck/back) with functional limitation. โ โ โ 4. **Hypertension & prediabetes** โ stable, but monitor given new medication changes. โ โ โ โ โ โ ### Plan โ โ โ | Item | Details | โ โ โ |------|---------| โ โ โ | **Imaging** | Order cervical spine MRI (cervical canal, cord, neural foramina) to โ โ โ assess degree of spondylotic compression. | โ โ โ | **Neurology consult** | For formal assessment of myelopathy; consider EMG/NCS if โ โ โ peripheral neuropathy suspected. | โ โ โ | **Physical therapy** | Referral to PT specializing in cervical traction, posture โ โ โ training, and gait stabilization. Initiate home exercise program (neck ROM, core โ โ โ strengthening). | โ โ โ | **Pain management** | Continue acetaminophen PRN; consider topical NSAID gel for โ โ โ cervical pain; avoid longโterm opioids. | โ โ โ | **Cough evaluation** | Begin trial of protonโpump inhibitor (omeprazole 20โฏmg โ โ โ daily) for 4โ6โฏweeks; consider ENT referral if symptoms persist. | โ โ โ | **Medication review** | Review current meds for antihypertensives that may cause โ โ โ dizziness; adjust if needed. | โ โ โ | **Fall risk assessment** | Home safety evaluation; consider occupational therapy โ โ โ for adaptive devices (assistive cane, bathroom grab bars). | โ โ โ | **Followโup** | Return in 2โฏweeks to review MRI results and PT progress; sooner if โ โ โ new neurologic deficits or falls. | โ โ โ | **Lifestyle** | Encourage ergonomic workstation modifications; limit prolonged neck โ โ โ flexion (e.g., phone use). | โ โ โ | **Vaccinations** | Ensure flu and COVIDโ19 boosters upโtoโdate (patient had booster โ โ โ 2023). | โ โ โ โ โ โ **Patient Education:** โ โ โ - Discussed the natural history of cervical spondylosis and importance of early โ โ โ intervention to prevent irreversible cord damage. โ โ โ - Reviewed signs of worsening myelopathy (new weakness, gait deterioration, hand โ โ โ clumsiness) and instructed to call clinic immediately if they develop. โ โ โ - Emphasized posture, activity pacing, and use of supportive devices. โ โ โ โ โ โ **Signature:** โ โ โ Dr. Kim Stevenson, MD โ Primary Care Physician โ โ โ **Contact:** (555) 987โ6543 โ โ โ โ โ โ --- โ โ โ โ โ โ *End of note.* โ โโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ [index: 0]
Inย [9]:
Copied!
# The preview dataset is available as a pandas DataFrame.
preview.dataset
# The preview dataset is available as a pandas DataFrame.
preview.dataset
Out[9]:
| diagnosis | patient_summary | patient_sampler | doctor_sampler | patient_id | symptom_onset_date | date_of_visit | first_name | last_name | dob | physician | physician_notes | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | cervical spondylosis | I've been having a lot of pain in my neck and ... | {'uuid': 'e5e0be22-7f5e-42a8-8b7e-d5b1a6ae30d6... | {'uuid': '4ab78e72-2150-434c-8842-6ced4510158f... | PT-E350F177 | 2024-02-14 | 2024-03-14 | James | Duncan | 1920-03-06 | Dr. Stevenson | **Visit Summary โ Dr. Kim Stevenson** \n**Pat... |
| 1 | impetigo | I have a rash on my face that is getting worse... | {'uuid': '5f0de54c-39d0-49d8-8689-78fea12861aa... | {'uuid': '7af64b76-486a-4cbd-8a20-3a2d8c8b93f8... | PT-A30580AF | 2024-03-15 | 2024-04-12 | Melissa | Bryant | 1979-08-03 | Dr. Thompson | **Patient:** Melissa Bryant \n**DOB:** 04/03/... |
๐ Analyze the generated dataยถ
Data Designer automatically generates a basic statistical analysis of the generated data.
This analysis is available via the
analysisproperty of generation result objects.
Inย [10]:
Copied!
# Print the analysis as a table.
preview.analysis.to_report()
# Print the analysis as a table.
preview.analysis.to_report()
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ ๐จ Data Designer Dataset Profile โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ Dataset Overview โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ number of records โ number of columns โ percent complete records โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ 2 โ 10 โ 100.0% โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ ๐ฒ Sampler Columns โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ column name โ data type โ number unique values โ sampler type โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ patient_sampler โ dict โ 2 (100.0%) โ person_from_faker โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ doctor_sampler โ dict โ 2 (100.0%) โ person_from_faker โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ patient_id โ string โ 2 (100.0%) โ uuid โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ symptom_onset_date โ string โ 2 (100.0%) โ datetime โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ date_of_visit โ string โ 2 (100.0%) โ timedelta โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ ๐ LLM-Text Columns โโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ โ โ prompt tokens โ completion tokens โ โ column name โ data type โ number unique values โ per record โ per record โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ physician_notes โ string โ 2 (100.0%) โ 122.5 +/- 3.5 โ 1087.0 +/- 264.5 โ โโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโ ๐งฉ Expression Columns โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ column name โ data type โ number unique values โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ first_name โ string โ 2 (100.0%) โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ last_name โ string โ 2 (100.0%) โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ dob โ string โ 2 (100.0%) โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ physician โ string โ 2 (100.0%) โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ Table Notes โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ โ โ โ 1. All token statistics are based on a sample of max(1000, len(dataset)) records. โ โ 2. Tokens are calculated using tiktoken's cl100k_base tokenizer. โ โ โ โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
๐ Scale up!ยถ
Happy with your preview data?
Use the
createmethod to submit larger Data Designer generation jobs.
Inย [11]:
Copied!
results = data_designer.create(config_builder, num_records=10, dataset_name="tutorial-3")
results = data_designer.create(config_builder, num_records=10, dataset_name="tutorial-3")
[12:08:17] [INFO] ๐จ Creating Data Designer dataset
[12:08:17] [INFO] โ Validation passed
[12:08:17] [INFO] โ๏ธ Sorting column configs into a Directed Acyclic Graph
[12:08:17] [INFO] ๐ฉบ Running health checks for models...
[12:08:17] [INFO] |-- ๐ Checking 'nvidia/nemotron-3-nano-30b-a3b' in provider named 'nvidia' for model alias 'nemotron-nano-v3'...
[12:08:17] [INFO] |-- โ Passed!
[12:08:17] [INFO] โณ Processing batch 1 of 1
[12:08:17] [INFO] ๐ฑ Sampling 10 records from seed dataset
[12:08:17] [INFO] |-- seed dataset size: 820 records
[12:08:17] [INFO] |-- sampling strategy: ordered
[12:08:18] [INFO] ๐ฒ Preparing samplers to generate 10 records across 5 columns
[12:08:18] [INFO] (๐พ + ๐พ) Concatenating 2 datasets
[12:08:18] [INFO] ๐งฉ Generating column `first_name` from expression
[12:08:18] [INFO] ๐งฉ Generating column `last_name` from expression
[12:08:18] [INFO] ๐งฉ Generating column `dob` from expression
[12:08:18] [INFO] ๐งฉ Generating column `physician` from expression
[12:08:18] [INFO] ๐ llm-text model config for column 'physician_notes'
[12:08:18] [INFO] |-- model: 'nvidia/nemotron-3-nano-30b-a3b'
[12:08:18] [INFO] |-- model alias: 'nemotron-nano-v3'
[12:08:18] [INFO] |-- model provider: 'nvidia'
[12:08:18] [INFO] |-- inference parameters:
[12:08:18] [INFO] | |-- generation_type=chat-completion
[12:08:18] [INFO] | |-- max_parallel_requests=4
[12:08:18] [INFO] | |-- extra_body={'chat_template_kwargs': {'enable_thinking': False}}
[12:08:18] [INFO] | |-- temperature=1.00
[12:08:18] [INFO] | |-- top_p=1.00
[12:08:18] [INFO] | |-- max_tokens=2048
[12:08:18] [INFO] โก๏ธ Processing llm-text column 'physician_notes' with 4 concurrent workers
[12:08:18] [INFO] โฑ๏ธ llm-text column 'physician_notes' will report progress after each record
[12:08:24] [INFO] |-- ๐ถ llm-text column 'physician_notes' progress: 1/10 (10%) complete, 1 ok, 0 failed, 0.15 rec/s, eta 58.4s
[12:08:24] [INFO] |-- ๐ถ llm-text column 'physician_notes' progress: 2/10 (20%) complete, 2 ok, 0 failed, 0.29 rec/s, eta 27.1s
[12:08:25] [INFO] |-- ๐ด llm-text column 'physician_notes' progress: 3/10 (30%) complete, 3 ok, 0 failed, 0.40 rec/s, eta 17.6s
[12:08:26] [INFO] |-- ๐ด llm-text column 'physician_notes' progress: 4/10 (40%) complete, 4 ok, 0 failed, 0.46 rec/s, eta 13.0s
[12:08:30] [INFO] |-- ๐ llm-text column 'physician_notes' progress: 5/10 (50%) complete, 5 ok, 0 failed, 0.41 rec/s, eta 12.1s
[12:08:31] [INFO] |-- ๐ llm-text column 'physician_notes' progress: 6/10 (60%) complete, 6 ok, 0 failed, 0.46 rec/s, eta 8.7s
[12:08:32] [INFO] |-- ๐ llm-text column 'physician_notes' progress: 7/10 (70%) complete, 7 ok, 0 failed, 0.48 rec/s, eta 6.3s
[12:08:34] [INFO] |-- โ๏ธ llm-text column 'physician_notes' progress: 8/10 (80%) complete, 8 ok, 0 failed, 0.50 rec/s, eta 4.0s
[12:08:34] [INFO] |-- โ๏ธ llm-text column 'physician_notes' progress: 9/10 (90%) complete, 9 ok, 0 failed, 0.54 rec/s, eta 1.8s
[12:08:38] [INFO] |-- ๐ llm-text column 'physician_notes' progress: 10/10 (100%) complete, 10 ok, 0 failed, 0.50 rec/s, eta 0.0s
[12:08:38] [INFO] ๐ Model usage summary:
[12:08:38] [INFO] |-- model: nvidia/nemotron-3-nano-30b-a3b
[12:08:38] [INFO] |-- tokens: input=1438, output=9721, total=11159, tps=546
[12:08:38] [INFO] |-- requests: success=10, failed=0, total=10, rpm=29
[12:08:38] [INFO] ๐ Measuring dataset column statistics:
[12:08:38] [INFO] |-- ๐ฒ column: 'patient_sampler'
[12:08:38] [INFO] |-- ๐ฒ column: 'doctor_sampler'
[12:08:38] [INFO] |-- ๐ฒ column: 'patient_id'
[12:08:38] [INFO] |-- ๐งฉ column: 'first_name'
[12:08:38] [INFO] |-- ๐งฉ column: 'last_name'
[12:08:38] [INFO] |-- ๐งฉ column: 'dob'
[12:08:38] [INFO] |-- ๐ฒ column: 'symptom_onset_date'
[12:08:38] [INFO] |-- ๐ฒ column: 'date_of_visit'
[12:08:38] [INFO] |-- ๐งฉ column: 'physician'
[12:08:38] [INFO] |-- ๐ column: 'physician_notes'
Inย [12]:
Copied!
# Load the generated dataset as a pandas DataFrame.
dataset = results.load_dataset()
dataset.head()
# Load the generated dataset as a pandas DataFrame.
dataset = results.load_dataset()
dataset.head()
Out[12]:
| diagnosis | patient_summary | patient_sampler | doctor_sampler | patient_id | symptom_onset_date | date_of_visit | first_name | last_name | dob | physician | physician_notes | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | cervical spondylosis | I've been having a lot of pain in my neck and ... | {'age': 19, 'bachelors_field': 'business', 'bi... | {'age': 54, 'bachelors_field': 'no_degree', 'b... | PT-DA9579A9 | 2024-02-07 | 2024-02-28 | Frank | David | 2007-02-10 | Dr. Gonzalez | **Visit Note โ 2024-02-28 โ Frank David** **... |
| 1 | impetigo | I have a rash on my face that is getting worse... | {'age': 54, 'bachelors_field': 'no_degree', 'b... | {'age': 108, 'bachelors_field': 'stem_related'... | PT-D6657F28 | 2024-05-16 | 2024-05-22 | Diane | Copeland | 1971-04-07 | Dr. Haney | **Patient:** Diane Copeland **DOB:** 1987-04... |
| 2 | urinary tract infection | I have been urinating blood. I sometimes feel ... | {'age': 88, 'bachelors_field': 'stem_related',... | {'age': 59, 'bachelors_field': 'no_degree', 'b... | PT-B5670318 | 2024-07-08 | 2024-08-01 | Faith | Nguyen | 1937-04-15 | Dr. Bullock | **Encounter: Faith Nguyen โ 8/1/24** - **CC:*... |
| 3 | arthritis | I have been having trouble with my muscles and... | {'age': 50, 'bachelors_field': 'no_degree', 'b... | {'age': 105, 'bachelors_field': 'no_degree', '... | PT-390E173A | 2024-03-18 | 2024-03-21 | Robert | Patel | 1975-10-24 | Dr. Gallagher | **Visit Note - Robert Patel** **Date:** 2024... |
| 4 | dengue | I have been feeling really sick. My body hurts... | {'age': 27, 'bachelors_field': 'business', 'bi... | {'age': 31, 'bachelors_field': 'no_degree', 'b... | PT-19013167 | 2024-09-21 | 2024-10-01 | Maria | Flores | 1998-07-06 | Dr. Fields | **Patient:** Maria Flores **DOB:** 1998-03-1... |
Inย [13]:
Copied!
# Load the analysis results into memory.
analysis = results.load_analysis()
analysis.to_report()
# Load the analysis results into memory.
analysis = results.load_analysis()
analysis.to_report()
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ ๐จ Data Designer Dataset Profile โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ Dataset Overview โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ number of records โ number of columns โ percent complete records โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ 10 โ 10 โ 100.0% โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ ๐ฒ Sampler Columns โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ column name โ data type โ number unique values โ sampler type โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ patient_sampler โ dict โ 10 (100.0%) โ person_from_faker โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ doctor_sampler โ dict โ 10 (100.0%) โ person_from_faker โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ patient_id โ string โ 10 (100.0%) โ uuid โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ symptom_onset_date โ string โ 10 (100.0%) โ datetime โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ date_of_visit โ string โ 10 (100.0%) โ timedelta โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ ๐ LLM-Text Columns โโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ โ โ prompt tokens โ completion tokens โ โ column name โ data type โ number unique values โ per record โ per record โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ physician_notes โ string โ 10 (100.0%) โ 120.5 +/- 4.9 โ 962.0 +/- 314.9 โ โโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโ ๐งฉ Expression Columns โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ column name โ data type โ number unique values โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ first_name โ string โ 10 (100.0%) โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ last_name โ string โ 10 (100.0%) โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ dob โ string โ 10 (100.0%) โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค โ physician โ string โ 10 (100.0%) โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ Table Notes โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ โ โ โ 1. All token statistics are based on a sample of max(1000, len(dataset)) records. โ โ 2. Tokens are calculated using tiktoken's cl100k_base tokenizer. โ โ โ โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โญ๏ธ Next Stepsยถ
Check out the following notebook to learn more about: