read-only demo

Videos YnNF55QV0zs

Persona Engineering: A Field Guide to AI Synthetic Personas — Ishan Anand, InsightSciences.ai

index_state ready data_status ok

AI Engineer· published 2026-07-29· 0:21:09· en-US· indexed 2026-08-10 19:42

Open on YouTube

Scene timeline

  1. Shot 0, 0:00 to 0:03, 1 of 1 keyframes kept
  2. Shot 1, 0:03 to 0:05, 1 of 1 keyframes kept
  3. Shot 2, 0:05 to 0:12, 1 of 1 keyframes kept
  4. Shot 3, 0:12 to 0:33, 1 of 1 keyframes kept
  5. Shot 4, 0:33 to 1:02, 1 of 1 keyframes kept
  6. Shot 5, 1:02 to 1:42, 1 of 1 keyframes kept
  7. Shot 6, 1:42 to 2:08, 1 of 1 keyframes kept
  8. Shot 7, 2:08 to 2:33, 1 of 1 keyframes kept
  9. Shot 8, 2:33 to 3:12, 1 of 1 keyframes kept
  10. Shot 9, 3:12 to 3:53, 1 of 1 keyframes kept
  11. Shot 10, 3:53 to 4:20, 1 of 1 keyframes kept
  12. Shot 11, 4:20 to 4:47, 1 of 1 keyframes kept
  13. Shot 12, 4:47 to 5:03, 1 of 1 keyframes kept
  14. Shot 13, 5:03 to 5:31, 1 of 1 keyframes kept
  15. Shot 14, 5:31 to 6:03, 1 of 1 keyframes kept
  16. Shot 15, 6:03 to 6:34, 1 of 1 keyframes kept
  17. Shot 16, 6:34 to 7:08, 1 of 1 keyframes kept
  18. Shot 17, 7:08 to 7:42, 0 of 1 keyframes kept
  19. Shot 18, 7:42 to 8:26, 1 of 1 keyframes kept
  20. Shot 19, 8:26 to 8:54, 0 of 1 keyframes kept
  21. Shot 20, 8:54 to 9:22, 1 of 1 keyframes kept
  22. Shot 21, 9:22 to 9:50, 1 of 1 keyframes kept
  23. Shot 22, 9:50 to 10:22, 1 of 1 keyframes kept
  24. Shot 23, 10:22 to 10:54, 1 of 1 keyframes kept
  25. Shot 24, 10:54 to 11:21, 1 of 1 keyframes kept
  26. Shot 25, 11:21 to 11:47, 1 of 1 keyframes kept
  27. Shot 26, 11:47 to 12:14, 1 of 1 keyframes kept
  28. Shot 27, 12:14 to 12:41, 1 of 1 keyframes kept
  29. Shot 28, 12:41 to 13:07, 0 of 1 keyframes kept
  30. Shot 29, 13:07 to 13:37, 1 of 1 keyframes kept
  31. Shot 30, 13:37 to 14:06, 1 of 1 keyframes kept
  32. Shot 31, 14:06 to 14:35, 1 of 1 keyframes kept
  33. Shot 32, 14:35 to 15:05, 0 of 1 keyframes kept
  34. Shot 33, 15:05 to 15:34, 0 of 1 keyframes kept
  35. Shot 34, 15:34 to 16:07, 1 of 1 keyframes kept
  36. Shot 35, 16:07 to 16:39, 1 of 1 keyframes kept
  37. Shot 36, 16:39 to 17:12, 1 of 1 keyframes kept
  38. Shot 37, 17:12 to 17:26, 1 of 1 keyframes kept
  39. Shot 38, 17:26 to 18:01, 1 of 1 keyframes kept
  40. Shot 39, 18:01 to 18:28, 1 of 1 keyframes kept
  41. Shot 40, 18:28 to 18:50, 1 of 1 keyframes kept
  42. Shot 41, 18:50 to 19:16, 1 of 1 keyframes kept
  43. Shot 42, 19:16 to 19:41, 1 of 1 keyframes kept
  44. Shot 43, 19:41 to 20:07, 0 of 1 keyframes kept
  45. Shot 44, 20:07 to 20:32, 0 of 1 keyframes kept
  46. Shot 45, 20:32 to 20:48, 1 of 1 keyframes kept
  47. Shot 46, 20:48 to 20:52, 1 of 1 keyframes kept
  48. Shot 47, 20:52 to 21:08, 0 of 1 keyframes kept

48 shot(s).

keyframes kept every frame deduplicated

What was stored

cues
237
whisperx 237
chunks
37
from 237 cues
keyframes
40
kept of 48 captured
frames with text
40
1,262 lines read
chapters
13
from the source metadata
keyframe bytes
6.0 MB
word timings on 237 cues

Provenance

Each pipeline stage, its state and the model that produced it
stage state model started took
fetch done 2026-08-09 23:23 0s
stt done 2026-08-09 14:11 28s
chunk done 2026-08-09 14:11 0s
text_embed done 2026-08-10 19:42 0s
keyframe done 2026-08-09 14:11 2m 40s
ocr done 2026-08-09 14:14 24s
frame_embed done 2026-08-10 19:42 7s

Frames, and what the machine read

  • 0:02 #0 done2 line(s)

    shot 0·sharpness 453.8

    1. AlEngineer0.96
    2. World's Fair0.97
  • 0:03 #1 done2 line(s)

    shot 1·sharpness 658.6

    1. AlEngineer0.95
    2. World's Fair0.99
  • 0:10 #2 done24 line(s)

    shot 2·sharpness 2735.8

    1. LAB & PLATINUM SPONSORS0.99
    2. Amazon AGI Lab0.98
    3. ANTHROP\C1.00
    4. Google DeepMind1.00
    5. MINIMAX0.93
    6. OpenAI0.92
    7. Akamai1.00
    8. arize1.00
    9. aws1.00
    10. Braintrust bright data0.98
    11. B1.00
    12. Browserbase1.00
    13. docker1.00
    14. :neo4j0.92
    15. ORACLE1.00
    16. PayPal1.00
    17. qodo1.00
    18. reducto1.00
    19. Sonar1.00
    20. Makers of0.99
    21. togetherai1.00
    22. Unblocked1.00
    23. WorkOS1.00
    24. SonarQube1.00
  • 0:22 #3 done2 line(s)

    shot 3·sharpness 208.0

    1. AlEngineer0.99
    2. World's Fair1.00
  • 0:45 #4 done35 line(s)

    shot 4·sharpness 3859.9

    1. Synthetic Personas Move From Novelty to Market Momentum1.00
    2. AlEngineer0.98
    3. World'sFair1.00
    4. Future of Product Marketing:1.00
    5. Synthetic Customers1.00
    6. Synthetic Persona Startup Funding0.99
    7. Transform Message Testing0.99
    8. 1201.00
    9. Gartner1.00
    10. 1001.00
    11. PRESENTED BY1.00
    12. Synthetic Customers1.00
    13. Earn Their Stripes1.00
    14. 801.00
    15. Microsoft1.00
    16. BAIN & COMPANY0.95
    17. 601.00
    18. The Al Tools That Are0.96
    19. 401.00
    20. Transforming Market1.00
    21. HARVARD1.00
    22. 201.00
    23. Research1.00
    24. BUSINESS1.00
    25. REVIEW1.00
    26. 20231.00
    27. 20241.00
    28. 20251.00
    29. 20261.00
    30. Millions ($)0.99
    31. Can AI Replace Humans for Market Research?1.00
    32. THE WALL STREET JOURNAL.1.00
    33. insight sciences.ai1.00
    34. Engineering the future of Al0.99
    35. World'sFair1.00
  • 1:26 #5 done37 line(s)

    shot 5·sharpness 2585.1

    1. A Once-Impossible Prediction Problem Becomes Practical0.99
    2. AlEngineer0.99
    3. World'sFair1.00
    4. The accuracy of weather forecasts has improved1.00
    5. Our World1.00
    6. Accuracy is measured as the difference between the forecast and subsequent weather. This is based on0.98
    7. in Data0.99
    8. the 500 hPa geopotential height' which is a common meteorological metric used to measure air pressure.0.98
    9. 100%1.00
    10. 3-day forecast1.00
    11. 80%1.00
    12. "Highly1.00
    13. 5-day forecast0.96
    14. PRESENTED BY1.00
    15. accurate"1.00
    16. Microsoft1.00
    17. 60%1.00
    18. 7-day forecast0.99
    19. "Useful"0.97
    20. 40%1.00
    21. 10-day forecast0.98
    22. 20%1.00
    23. Northern Hemisphere1.00
    24. ....... Southern Hemisphere0.92
    25. 0%1.00
    26. 19811.00
    27. 19901.00
    28. 20001.00
    29. 20101.00
    30. 20181.00
    31. Year1.00
    32. Source: European Centre for Medium-Range Weather Forecasts (ECMWF).0.99
    33. Licensed under CC-BY by the author Hannah Ritchie.0.99
    34. insight sciences.ai1.00
    35. TRACK 7· JULY 1, 20260.96
    36. World'sFair1.00
    37. Computer Use0.97
  • 1:52 #6 done16 line(s)

    shot 6·sharpness 1767.0

    1. “Synthetic Personas 101”0.99
    2. AlEngineer0.97
    3. World'sFair1.00
    4. Agenda1.00
    5. Objectives1.00
    6. ·Why Now?0.96
    7. · No hype0.89
    8. · Failure Modes0.95
    9. · No dismissal0.95
    10. ·Techniques0.99
    11. • Published research0.95
    12. · Metrics0.92
    13. insight sciences.ai1.00
    14. TRACK 7· JULY 1, 20260.95
    15. World'sFair1.00
    16. Computer Use0.98
  • 2:23 #7 done16 line(s)

    shot 7·sharpness 1758.8

    1. “Synthetic Personas 101"0.95
    2. AlEngineer0.99
    3. World'sFair1.00
    4. Agenda1.00
    5. Objectives1.00
    6. · Why Now?0.92
    7. · No hype0.89
    8. · Failure Modes0.95
    9. · No dismissal0.95
    10. ·Techniques0.98
    11. · Published research0.94
    12. · Metrics0.92
    13. insight sciences.ai1.00
    14. TRACK 7· JULY 1, 20260.96
    15. World'sFair1.00
    16. Computer Use0.98
  • 2:45 #8 done21 line(s)

    shot 8·sharpness 2121.0

    1. Not A New Dream. Why Now?0.98
    2. AlEngineer0.99
    3. m0.73
    4. World's Fair0.99
    5. If/Then0.99
    6. How the1.00
    7. SIMULMATICS1.00
    8. CORPORATION1.00
    9. Inwented0.95
    10. THE FUTURE1.00
    11. and aholunly terilying0.74
    12. author of A World an Fire0.96
    13. -Amanda Foreman,0.92
    14. New York Times1.00
    15. Best-selling Author1.00
    16. of These Truths0.99
    17. Jill Lepore1.00
    18. insight sciences.ai0.98
    19. TRACK 7· JULY 1, 20260.94
    20. World'sFair1.00
    21. Computer Use1.00
  • 3:48 #9 done33 line(s)

    shot 9·sharpness 2739.0

    1. LLMs Unlock Semantic Simulation1.00
    2. AlEngineer0.98
    3. World'sFair1.00
    4. F = ma0.97
    5. believes1.00
    6. wants1.00
    7. fears trusts0.98
    8. decides1.00
    9. (x+(u·∇)u)=−∇p+μ∇²u+∫0.82
    10. hopes values0.98
    11. ∂t0.99
    12. feels1.00
    13. prefers1.00
    14. worries1.00
    15. dreams chooses1.00
    16. doubts desires1.00
    17. remembers1.00
    18. expects1.00
    19. Numerical Simulation1.00
    20. Semantic Simulation0.97
    21. Medium1.00
    22. Numbers, Equations1.00
    23. Language, Meaning1.00
    24. Atomic unit1.00
    25. A law or formula0.99
    26. Large Language Model1.00
    27. Grounding1.00
    28. Physical laws1.00
    29. Empirical deep learning1.00
    30. insight sciences.ai1.00
    31. TRACK 7· JULY 1, 20260.95
    32. World's Fair0.96
    33. Computer Use0.97
  • 4:06 #10 done35 line(s)

    shot 10·sharpness 2040.9

    1. Grounded LLM Agents Predict Held-out Responses1.00
    2. AlEngineer0.98
    3. World's Fair0.99
    4. Human Participants1.00
    5. Wave 1: Initial Data Collection0.97
    6. (N=1052)1.00
    7. Al Voice Interview1.00
    8. Survey Battery &1.00
    9. Approx US Distribution1.00
    10. on Age, Gender, Race,0.99
    11. Region, Education, &0.99
    12. Party Identification0.97
    13. & views on societal issues0.99
    14. Semi-structured life stories0.97
    15. Avg. -6.5k words, 2hrs;0.98
    16. (AVP Protocol)1.00
    17. GSS(177), FI-44 (44), Econ0.89
    18. Games (5), Behavioral0.99
    19. Experiment (5)1.00
    20. Experiments1.00
    21. Generative Agent Construction0.99
    22. Interview Agent1.00
    23. Survey Agent1.00
    24. Survey+Interview1.00
    25. Transcript Input1.00
    26. Only Interview1.00
    27. Only GSS (177) and1.00
    28. BFI-44 Input0.88
    29. GSS, and BFI-44 Input0.99
    30. Interview Transcript,1.00
    31. Park et al., "LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals"0.99
    32. insight sciences.ai1.00
    33. TRACK 7· JULY 1,20260.94
    34. World's Fair0.94
    35. Computer Use0.97
  • 4:38 #11 done38 line(s)

    shot 11·sharpness 2173.6

    1. Grounded LLM Agents Predict Held-out Responses1.00
    2. AlEngineer0.98
    3. World's Fair0.99
    4. Human Participants1.00
    5. Wave 1: Initial Data Collection1.00
    6. (N=1052)1.00
    7. Al Voice Interview0.99
    8. Survey Battery &1.00
    9. Approx US Distribution1.00
    10. on Age, Gender, Race,0.99
    11. Region, Education, &0.99
    12. Party Identification0.99
    13. & views on societal issues0.99
    14. Semi-structured life stories0.99
    15. Avg. -6.5k words, 2hrs;0.98
    16. (AVP Protocol)1.00
    17. GSS (177), BFI-44 (44), Econ0.90
    18. Games (5), Behavioral1.00
    19. Experiment (5)1.00
    20. Experiments1.00
    21. 83%0.97
    22. Predictive1.00
    23. Accuracy?1.00
    24. Generative Agent Construction1.00
    25. Interview Agent1.00
    26. Survey Agent1.00
    27. Survey+Interview1.00
    28. Transcript Input1.00
    29. Only Interview0.97
    30. Only GSS (177) and0.98
    31. BFI-44 Input0.98
    32. GSS, and BFI-44 Input0.98
    33. Interview Transcript,1.00
    34. Park et al., "LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals"0.99
    35. insight sciences.ai1.00
    36. TRACK 7· JULY 1,20260.92
    37. World's Fair0.98
    38. Computer Use1.00
  • 4:55 #12 done8 line(s)

    shot 12·sharpness 1357.2

    1. AlEngineer0.97
    2. World'sFair1.00
    3. Failure Modes1.00
    4. THREE CRITICAL WAYS SYNTHETIC PERSONAS BREAK0.99
    5. insight sciences.ai1.00
    6. TRACK 7· JULY 1,20260.95
    7. World'sFair1.00
    8. ComputerUse1.00
  • 5:28 #13 done16 line(s)

    shot 13·sharpness 3637.6

    1. AlEngineer0.96
    2. World'sFair1.00
    3. Prompt 2: Ask Purchase1.00
    4. System: You, AI, are a customer. Your task is to fill in the blanks0.99
    5. Return the completed1.00
    6. information in comma-separated values, without any extra text.0.99
    7. User: Please consider the following product category: {category}.1.00
    8. Suppose you are in a grocery store, and you see the following product in that category: {product}.1.00
    9. The product is currently priced at ${Price}. Would you or would you not purchase the product?0.99
    10. ["purchase"or "not purchase"]0.98
    11. Return example: purchase0.98
    12. Gui & Toubia, "The Challenge of Using LLMs to Simulate Human Behavior: A Causal Inference Perspective"0.99
    13. insight sciences.ai1.00
    14. TRACK 7·JULY 1,20260.96
    15. World'sFair1.00
    16. ComputerUse1.00
  • 5:59 #14 done24 line(s)

    shot 14·sharpness 1995.9

    1. AlEngineer0.99
    2. World'sFair1.00
    3. 100%1.00
    4. Ppueity0.67
    5. 75%1.00
    6. Condition1.00
    7. PRESENTED BY0.98
    8. 50%1.00
    9. Human1.00
    10. Microsoft1.00
    11. LLM1.00
    12. 25%1.00
    13. 0%1.00
    14. 0%1.00
    15. 50%1.00
    16. 100%1.00
    17. 150%1.00
    18. 200%1.00
    19. Price (% of regular price)0.98
    20. Gui & Toubia, "The Challenge of Using LLMs to Simulate Human Behavior: A Causal Inference Perspective"0.99
    21. insight sciences.ai1.00
    22. TRACK 7· JULY 1,20260.93
    23. orld's Fair0.94
    24. ComputerUse1.00
  • 6:22 #15 done33 line(s)

    shot 15·sharpness 2552.9

    1. Latent Confounders in Synthetic Personas0.99
    2. AlEngineer0.99
    3. World'sFair1.00
    4. last_price1.00
    5. competing_price1.00
    6. expiration_days1.00
    7. 0.61.00
    8. 0.0501.00
    9. 0.21.00
    10. 0.0251.00
    11. 0.31.00
    12. PRESENTED BY0.98
    13. Rellue0.66
    14. Microsoft0.95
    15. 0.01.00
    16. 0.01.00
    17. 0.0000.97
    18. -0.0251.00
    19. -0.21.00
    20. -0.31.00
    21. -0.0501.00
    22. 0%1.00
    23. 50% 100% 150% 200%0.99
    24. 0%50% 100% 150% 200%0.96
    25. 0%1.00
    26. 50% 100% 150% 200%0.97
    27. Price (% of regular price)1.00
    28. Figure 1: Unintended correlation between price and past price, competing price, and expiration days0.99
    29. Gui & Toubia, "The Challenge of Using LLMs to Simulate Human Behavior: A Causal Inference Perspective"0.99
    30. insight sciences.ai1.00
    31. TRACK 7· JULY 1,20260.93
    32. World'sFair1.00
    33. ComputerUse1.00
  • 7:04 #16 done11 line(s)

    shot 16·sharpness 2855.8

    1. When Key Context Is Missing, LLMs May Invent Confounders0.99
    2. AlEngineer0.98
    3. World'sFair1.00
    4. Human Experiment1.00
    5. Synthetic Experiment1.00
    6. $5001.00
    7. Lesson: Richly ground personas in personality, context, even the study's own construction (!!).0.99
    8. insight sciences.ai1.00
    9. TRACK 7· JULY 1,20260.95
    10. World'sFair1.00
    11. ComputerUse1.00
  • 7:22 #17 skipped

    shot 17·duplicate of #16

  • 8:21 #18 done40 line(s)

    shot 18·sharpness 2638.1

    1. Prompt Sensitivity1.00
    2. AlEngineer0.98
    3. World's Fair0.98
    4. Choice ordering 11.00
    5. Response1.00
    6. Question: In the past 12 months, has1.00
    7. this person given birth to any children?1.00
    8. B. No0.93
    9. A. Yes0.99
    10. Adjusted1.00
    11. Answer:1.00
    12. A. Yes B. No0.96
    13. 1/21.00
    14. response1.00
    15. P("A")0.89
    16. 0.821.00
    17. P("B")0.85
    18. 0.111.00
    19. Choice ordering 21.00
    20. Response1.00
    21. Question: In the past 12 months, has1.00
    22. Yes1.00
    23. No1.00
    24. this person given birth to any children?0.99
    25. 1/21.00
    26. A. No0.97
    27. B. Yes0.98
    28. Answer:1.00
    29. A. No B. Yes0.96
    30. P("A")0.94
    31. 0.801.00
    32. P("B")0.89
    33. 0.151.00
    34. Lesson: Durability test personas to understand their behavior1.00
    35. under reordering, wording changes, and adversarial challenges.1.00
    36. Dominguez-Olmedo et al., "Questioning the Survey Responses of Large Language Models"0.99
    37. insight sciences.ai1.00
    38. TRACK 7·JULY 1,20260.97
    39. World's Fair0.99
    40. Computer Use0.99
  • 8:45 #19 skipped

    shot 19·duplicate of #6

  • 9:19 #20 done37 line(s)

    shot 20·sharpness 2366.0

    1. Predicting attitudes is easier than predicting actions0.99
    2. AlEngineer1.00
    3. World's Fair0.96
    4. GPT4-derived predictionsExpert predictions0.98
    5. Allen et al. 20240.97
    6. Dellavigna & Pope 2018-0.95
    7. Surveys:1.00
    8. Typing task effor0.98
    9. Vlasceanu et al. 20241.00
    10. Natively Language/Text1.00
    11. UBI & immi0.93
    12. Tappin et al. 2023-1.00
    13. tion policy suppor0.96
    14. Reflects Attitudes0.99
    15. Zickfeld et al. 20241.00
    16. Voelkel et al. 20231.00
    17. Milkman et al. 20220.99
    18. Field experiments:1.00
    19. Dellavigna & Linos 20221.00
    20. Actions transcribed to text0.99
    21. Milkman et al. 20211.00
    22. Reflects Behaviors0.99
    23. All survey experiments0.99
    24. Meta-analytic Avg.1.00
    25. All field experiments1.00
    26. All with expert forecasts0.98
    27. -1.00.99
    28. -0.50.99
    29. 0.01.00
    30. 0.50.93
    31. 1.01.00
    32. radj0.95
    33. Hewitt et al., "Predicting Results of Social Science Experiments Using Large Language Models"0.99
    34. insight sciences.ai1.00
    35. TRACK 7· JULY 1,20260.94
    36. World's Fai0.99
    37. Computer Use0.97
  • 9:44 #21 done35 line(s)

    shot 21·sharpness 2646.9

    1. Predicting attitudes is easier than predicting actions0.99
    2. AlEngineer1.00
    3. World's Fair0.99
    4. GPT4-derived predictions Expert predictions0.98
    5. Allen et al. 20240.96
    6. Dellavigna & Pope 2018-0.95
    7. Survey experiments1.00
    8. Vlasceanu et al. 20241.00
    9. Tappin et al. 2023-1.00
    10. UBI & immig0.97
    11. tion policy suppor0.94
    12. Zickfeld et al. 20240.98
    13. Lesson: Consider questions that triangulate behavior from attitudes.1.00
    14. Voelkel et al. 20230.99
    15. Milkman et al. 20220.97
    16. Gym Example:1.00
    17. Dellavigna & Linos 20221.00
    18. Ask about attitudes towards working out0.99
    19. Milkman et al. 20210.99
    20. • Versus asking about gym attendance0.99
    21. All survey experiments0.99
    22. Meta-analytic Avg.1.00
    23. All field experiments0.99
    24. All with expert forecasts1.00
    25. -1.00.99
    26. -0.51.00
    27. 0.01.00
    28. 0.50.95
    29. 1.01.00
    30. radj0.94
    31. Hewitt et al., "Predicting Results of Social Science Experiments Using Large Language Models"0.99
    32. insight sciences.ai0.97
    33. TRACK 7· JULY 1, 20260.95
    34. World's Fair1.00
    35. ComputerUse1.00
  • 10:12 #22 done56 line(s)

    shot 22·sharpness 4458.8

    1. Prompting1.00
    2. AlEngineer0.99
    3. World'sFair1.00
    4. Sampled completions1.00
    5. Context1.00
    6. Sampled completions0.99
    7. Hillary Clinton. However, I'm a huge sports fan, and I've1.00
    8. Donald Trump, but I didn't vote for him because I agreed1.00
    9. spent my entire life following and loving the New York1.00
    10. with him on all the issues. I voted for him because I didn't0.99
    11. Yankees. I'm also a huge fan of the New York Giants. I...0.99
    12. I am a strong conservative. I0.99
    13. want Hillary Clinton to be president. I disagreed with many...0.98
    14. support traditional values. I1.00
    15. Bernie Sanders in the primaries. I am a member of the0.99
    16. Democratic Socialists of America. But as a Jewish1.00
    17. American, I will not be voting for the Democratic.0.99
    18. opposed to abortion and I'm0.99
    19. budget, limited government,1.00
    20. think we need a balanced0.99
    21. and a strong military. I'm1.00
    22. John Kasich. I haven't changed my mind on any of those1.00
    23. issues. But Donald Trump is the opposite of what I've always0.99
    24. believed a Republican should be, and I can't vote for him...1.00
    25. establishment treated Bernie Sanders, I have lost hope in the1.00
    26. Hillary Clinton. But after seeing the way the Democratic0.99
    27. skeptical of global warming. In1.00
    28. 2016, I voted for1.00
    29. Donald Trump. In other words, I'm a typical Republican1.00
    30. voter, except that I'm gay. And that's why I'm leaving the0.98
    31. party. In 2018, I joined the Democratic Socialists of.0.99
    32. Republican Party. I've felt increasingly unwelcome in the...0.99
    33. Hillary Clinton, because I believed in her vision for0.99
    34. I am a strong liberal. I support1.00
    35. Donald Trump. And I have no regrets. But in the last two0.99
    36. America. I felt that she was the only candidate that could0.99
    37. progressive values. I support0.98
    38. years, I've watched conservatives convince themselves that0.99
    39. pull us out of the mess that Obama inherited. I am a loyal...0.99
    40. open immigration and women's0.99
    41. Trump is a conservative. And this is a lie. Trump has...1.00
    42. right to choose. I believe in1.00
    43. am Jewish. I have never been religious. I was raised in a1.00
    44. secular household, but my Jewishness is intrinsic to me. I...0.98
    45. Bernie Sanders. I am pro-LGBTQ. But I am also pro-life. I0.99
    46. systemic racism and that global1.00
    47. warming is one of our biggest0.99
    48. challenges. In 2016, I voted for0.99
    49. does that mean these days? I'm not so sure. Which brings me1.00
    50. to this poll, released today by CBS News and the New York...0.99
    51. Donald Trump. I consider myself a Republican. But what1.00
    52. Argyle et al., "Out of One, Many: Using Language Models to Simulate Human Samples"0.99
    53. insight sciences.ai0.98
    54. TRACK 7· JULY 1, 20260.96
    55. World's Fair0.97
    56. Computer Use0.97
  • 10:41 #23 done58 line(s)

    shot 23·sharpness 4575.5

    1. Prompting1.00
    2. AlEngineer1.00
    3. World's Fair0.96
    4. Sampled completions1.00
    5. Context1.00
    6. Sampled completions0.99
    7. Hillary Clinton. However, I'm a huge sports fan, and I've1.00
    8. Donald Trump, but I didn't vote for him because I agreed1.00
    9. spent my entire life following and loving the New York1.00
    10. with him on all the issues. I voted for him because I didn't0.99
    11. Yankees. I'm also a huge fan of the New York Giants. I...0.98
    12. I am a strong conservative. I0.98
    13. want Hillary Clinton to be president. I disagreed with many...0.99
    14. support traditional values. I1.00
    15. PRESENTED BY1.00
    16. Bernie Sanders in the primaries. I am a member of the0.99
    17. Democratic Socialists of America. But as a Jewish0.99
    18. American, I will not be voting for the Democratic...0.99
    19. opposed to abortion and I'm0.99
    20. budget, limited government,0.99
    21. and a strong military. I'm1.00
    22. think we need a balanced1.00
    23. John Kasich. I haven't changed my mind on any of those1.00
    24. issues. But Donald Trump is the opposite of what I've always0.99
    25. believed a Republican should be, and I can't vote for him...1.00
    26. Microsoft1.00
    27. establishment treated Bernie Sanders, I have lost hope in the1.00
    28. Hillary Clinton. But after seeing the way the Democratic0.99
    29. skeptical of global warming. In1.00
    30. 2016, I voted for1.00
    31. Donald Trump. In other words, I'm a typical Republican1.00
    32. voter, except that I'm gay. And that's why I'm leaving the0.98
    33. party. In 2018, I joined the Democratic Socialists of...1.00
    34. Republican Party. I've felt increasingly unwelcome in the...1.00
    35. Hillary Clinton, because I believed in her vision for0.99
    36. I am a strong liberal. I support1.00
    37. Donald Trump. And I have no regrets. But in the last two0.99
    38. America. I felt that she was the only candidate that could0.99
    39. progressive values. I support1.00
    40. years, I've watched conservatives convince themselves that1.00
    41. pull us out of the mess that Obama inherited. I am a loyal...0.99
    42. open immigration and women's1.00
    43. Trump is a conservative. And this is a lie. Trump has...1.00
    44. right to choose. I believe in1.00
    45. am Jewish. I have never been religious. I was raised in a1.00
    46. secular household, but my Jewishness is intrinsic to me. I...0.99
    47. Bernie Sanders. I am pro-LGBTQ. But I am also pro-life. I0.99
    48. systemic racism and that global1.00
    49. warming is one of our biggest0.99
    50. challenges. In 2016, I voted for0.99
    51. does that mean these days? I'm not so sure. Which brings me1.00
    52. to this poll, released today by CBS News and the New York...0.99
    53. Donald Trump. I consider myself a Republican. But what1.00
    54. Argyle et al., "Out of One, Many: Using Language Models to Simulate Human Samples"0.99
    55. insight sciences.ai1.00
    56. TRACK 7· JULY 1, 20260.96
    57. World's Fair0.99
    58. Computer Use0.96

Transcript

237 cues· 3,720 words· 20,748 chars

  1. 0:13 Hello, I'm Ishan and welcome to can AI predict people like we predict the weather?
  2. 0:20 A field guide to the nascent field of synthetic personas.
  3. 0:24 Now, I'm sure most of you in this room at some point or another have prompted a large language model with a role prompt.
  4. 0:32 You are a fill in the blank and then the task.
  5. 0:37 Believe it or not, that core principle of steering a model's outputs as if it were a particular person or persona has turned into an entire category that companies are using to test product concepts and messaging against synthetic respondents.
  6. 0:53 And it has moved from a novelty to market momentum, as you can see from both these headlines, as well as the increase in funding for the last few years.
  7. 1:03 And the running analogy I want to leave you with is that synthetic personas are like weather forecasting.
  8. 1:10 Like weather forecasting, they were unlocked thanks to an increase in compute and data.
  9. 1:16 And like weather forecasting, they operate within a particular regime, and going past that sometimes can go outside of where they're accurate.
  10. 1:25 So for example, you can only predict the weather a certain number of days in advance.
  11. 1:31 Similarly with synthetic personas, there's only so far you can go before you'll run into issues.
  12. 1:36 And understanding those issues are as important as understanding their promise and their potential.
  13. 1:43 So I'm Ishan Nand.
  14. 1:45 I'm the chief AI officer at Insight Sciences.
  15. 1:49 We construct LLM synthetic personas for market research and market insights teams.
  16. 1:55 And the reason for this talk is that most of the coverage in this space is very shallow, doesn't go into the technical details.
  17. 2:02 It's either outright hype or outright dismissal.
  18. 2:06 And it's really hard to separate the noise from what's real.
  19. 2:11 And so what I want to cover is that messy middle of the technical details.
  20. 2:14 And you don't have to take my word for it, even though I'm a vendor in the space, because everything I'm going to talk about today is going to be based on published research.
  21. 2:24 So we're going to cover why now for synthetic personas, how they fail, some techniques to inspire you, and then some metrics to judge whether your synthetic persona is accurate or not.
  22. 2:35 Speaking of weather forecasting, another parallel is just like in the 1950s and 60s, we got computers that promised us, correctly, a future of accurate weather forecasts.
  23. 2:48 We were also promised, believe it or not, people forecasts.
  24. 2:52 This company, Simulmatics, were extensively covered by Jill Lepore,
  25. 2:57 promised that they could simulate and predict the electorate using raw statistics and the computational power at the time.
  26. 3:05 Fortunately, that turned out not to be the case.
  27. 3:07 So you should approach claims like this with some humility.
  28. 3:10 But we have something they did not have then.
  29. 3:13 And that unlock is, again, more computational power, but also better modeling thanks to LLMs.
  30. 3:20 And LLMs unlock a new kind of simulation.
  31. 3:23 for the longest time to simulate something meant to mathematize it in formulas or equations.
  32. 3:30 But certain things, how we feel, how we act, what choices we make, aren't always succumbing to the equations.
  33. 3:38 And what LLMs offer us is a new medium, a new atomic unit of language itself that we can model against.
  34. 3:45 Now, granted, they are based on math under the hood, but it gives us this intermediary layer that we can construct and simulate against that we couldn't before.
  35. 3:54 and the process can work.
  36. 3:57 I want to share with you one of the most well-known demonstrations of this in the field.
  37. 4:03 What they did is they took about a thousand humans, they put them through about two and a half hours of extensive interviews about their background and their views and their attitudes, and they put those people through a battery of personality tests and surveys.
  38. 4:19 Then they took those transcripts and they passed it to an AI agent, and they had the AI agent take the same set of surveys and personality tests.
  39. 4:30 What they found was that the agents were basically about 83% aligned and predictive to the corresponding humans they were modeled against.
  40. 4:39 Now, one caveat is that number is normalized against the uncertainty and noise of the humans themselves.
  41. 4:46 It's a theme we're going to come back to at the end of this talk.
  42. 4:49 But don't get too excited because synthetic personas are different from regular experiments and they're liable to confuse and fool you if you don't know how they fail.
  43. 4:59 So I'm gonna cover three important failure modes that you need to know about when dealing with synthetic personas.
  44. 5:05 To understand the first one,
  45. 5:07 I want to consider this prompt these researchers gave.
  46. 5:11 It's a very ambiguous and very unsophisticated prompt.
  47. 5:15 It basically says, you are a customer.
  48. 5:17 I'm going to show you a product.
  49. 5:18 I'm going to tell you the category.
  50. 5:20 I'm going to give you its price.

Chapters

  1. 0:00 Introduction: synthetic personas for market research
  2. 1:43 Why this talk: separating signal from noise
  3. 2:46 Forecasting people like we forecast the weather
  4. 4:04 A thousand humans vs their agent replicas
  5. 5:22 Purchase probability and prompt sensitivity
  6. 6:41 Invented confounders and specifying the persona
  7. 7:45 Question order and framing effects
  8. 8:50 Predicting stated attitudes vs experts
  9. 10:07 Prompting techniques and subpopulation methods
  10. 13:08 Reconstructing and scoring the full distribution
  11. 16:09 Calibrating personas against real forecasts
  12. 17:38 Setting a noise floor with human vs human
  13. 18:40 Treat personas as economic actors, and what's next

Open at this second