Videos YnNF55QV0zs
Persona Engineering: A Field Guide to AI Synthetic Personas — Ishan Anand, InsightSciences.ai
Scene timeline
48 shot(s).
keyframes kept every frame deduplicated
What was stored
- cues
- 237
- whisperx 237
- chunks
- 37
- from 237 cues
- keyframes
- 40
- kept of 48 captured
- frames with text
- 40
- 1,262 lines read
- chapters
- 13
- from the source metadata
- keyframe bytes
- 6.0 MB
- word timings on 237 cues
Provenance
| stage | state | model | started | took |
|---|---|---|---|---|
fetch |
done | — | 2026-08-09 23:23 | 0s |
stt |
done | — | 2026-08-09 14:11 | 28s |
chunk |
done | — | 2026-08-09 14:11 | 0s |
text_embed |
done | — | 2026-08-10 19:42 | 0s |
keyframe |
done | — | 2026-08-09 14:11 | 2m 40s |
ocr |
done | — | 2026-08-09 14:14 | 24s |
frame_embed |
done | — | 2026-08-10 19:42 | 7s |
Frames, and what the machine read
-
- AlEngineer0.96
- World's Fair0.97
-
- AlEngineer0.95
- World's Fair0.99
-
- LAB & PLATINUM SPONSORS0.99
- Amazon AGI Lab0.98
- ANTHROP\C1.00
- Google DeepMind1.00
- MINIMAX0.93
- OpenAI0.92
- Akamai1.00
- arize1.00
- aws1.00
- Braintrust bright data0.98
- B1.00
- Browserbase1.00
- docker1.00
- :neo4j0.92
- ORACLE1.00
- PayPal1.00
- qodo1.00
- reducto1.00
- Sonar1.00
- Makers of0.99
- togetherai1.00
- Unblocked1.00
- WorkOS1.00
- SonarQube1.00
-
- AlEngineer0.99
- World's Fair1.00
-
- Synthetic Personas Move From Novelty to Market Momentum1.00
- AlEngineer0.98
- World'sFair1.00
- Future of Product Marketing:1.00
- Synthetic Customers1.00
- Synthetic Persona Startup Funding0.99
- Transform Message Testing0.99
- 1201.00
- Gartner1.00
- 1001.00
- PRESENTED BY1.00
- Synthetic Customers1.00
- Earn Their Stripes1.00
- 801.00
- Microsoft1.00
- BAIN & COMPANY0.95
- 601.00
- The Al Tools That Are0.96
- 401.00
- Transforming Market1.00
- HARVARD1.00
- 201.00
- Research1.00
- BUSINESS1.00
- REVIEW1.00
- 20231.00
- 20241.00
- 20251.00
- 20261.00
- Millions ($)0.99
- Can AI Replace Humans for Market Research?1.00
- THE WALL STREET JOURNAL.1.00
- insight sciences.ai1.00
- Engineering the future of Al0.99
- World'sFair1.00
-
- A Once-Impossible Prediction Problem Becomes Practical0.99
- AlEngineer0.99
- World'sFair1.00
- The accuracy of weather forecasts has improved1.00
- Our World1.00
- Accuracy is measured as the difference between the forecast and subsequent weather. This is based on0.98
- in Data0.99
- the 500 hPa geopotential height' which is a common meteorological metric used to measure air pressure.0.98
- 100%1.00
- 3-day forecast1.00
- 80%1.00
- "Highly1.00
- 5-day forecast0.96
- PRESENTED BY1.00
- accurate"1.00
- Microsoft1.00
- 60%1.00
- 7-day forecast0.99
- "Useful"0.97
- 40%1.00
- 10-day forecast0.98
- 20%1.00
- Northern Hemisphere1.00
- ....... Southern Hemisphere0.92
- 0%1.00
- 19811.00
- 19901.00
- 20001.00
- 20101.00
- 20181.00
- Year1.00
- Source: European Centre for Medium-Range Weather Forecasts (ECMWF).0.99
- Licensed under CC-BY by the author Hannah Ritchie.0.99
- insight sciences.ai1.00
- TRACK 7· JULY 1, 20260.96
- World'sFair1.00
- Computer Use0.97
-
- “Synthetic Personas 101”0.99
- AlEngineer0.97
- World'sFair1.00
- Agenda1.00
- Objectives1.00
- ·Why Now?0.96
- · No hype0.89
- · Failure Modes0.95
- · No dismissal0.95
- ·Techniques0.99
- • Published research0.95
- · Metrics0.92
- insight sciences.ai1.00
- TRACK 7· JULY 1, 20260.95
- World'sFair1.00
- Computer Use0.98
-
- “Synthetic Personas 101"0.95
- AlEngineer0.99
- World'sFair1.00
- Agenda1.00
- Objectives1.00
- · Why Now?0.92
- · No hype0.89
- · Failure Modes0.95
- · No dismissal0.95
- ·Techniques0.98
- · Published research0.94
- · Metrics0.92
- insight sciences.ai1.00
- TRACK 7· JULY 1, 20260.96
- World'sFair1.00
- Computer Use0.98
-
- Not A New Dream. Why Now?0.98
- AlEngineer0.99
- m0.73
- World's Fair0.99
- If/Then0.99
- How the1.00
- SIMULMATICS1.00
- CORPORATION1.00
- Inwented0.95
- THE FUTURE1.00
- and aholunly terilying0.74
- author of A World an Fire0.96
- -Amanda Foreman,0.92
- New York Times1.00
- Best-selling Author1.00
- of These Truths0.99
- Jill Lepore1.00
- insight sciences.ai0.98
- TRACK 7· JULY 1, 20260.94
- World'sFair1.00
- Computer Use1.00
-
- LLMs Unlock Semantic Simulation1.00
- AlEngineer0.98
- World'sFair1.00
- F = ma0.97
- believes1.00
- wants1.00
- fears trusts0.98
- decides1.00
- (x+(u·∇)u)=−∇p+μ∇²u+∫0.82
- hopes values0.98
- ∂t0.99
- feels1.00
- prefers1.00
- worries1.00
- dreams chooses1.00
- doubts desires1.00
- remembers1.00
- expects1.00
- Numerical Simulation1.00
- Semantic Simulation0.97
- Medium1.00
- Numbers, Equations1.00
- Language, Meaning1.00
- Atomic unit1.00
- A law or formula0.99
- Large Language Model1.00
- Grounding1.00
- Physical laws1.00
- Empirical deep learning1.00
- insight sciences.ai1.00
- TRACK 7· JULY 1, 20260.95
- World's Fair0.96
- Computer Use0.97
-
- Grounded LLM Agents Predict Held-out Responses1.00
- AlEngineer0.98
- World's Fair0.99
- Human Participants1.00
- Wave 1: Initial Data Collection0.97
- (N=1052)1.00
- Al Voice Interview1.00
- Survey Battery &1.00
- Approx US Distribution1.00
- on Age, Gender, Race,0.99
- Region, Education, &0.99
- Party Identification0.97
- & views on societal issues0.99
- Semi-structured life stories0.97
- Avg. -6.5k words, 2hrs;0.98
- (AVP Protocol)1.00
- GSS(177), FI-44 (44), Econ0.89
- Games (5), Behavioral0.99
- Experiment (5)1.00
- Experiments1.00
- Generative Agent Construction0.99
- Interview Agent1.00
- Survey Agent1.00
- Survey+Interview1.00
- Transcript Input1.00
- Only Interview1.00
- Only GSS (177) and1.00
- BFI-44 Input0.88
- GSS, and BFI-44 Input0.99
- Interview Transcript,1.00
- Park et al., "LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals"0.99
- insight sciences.ai1.00
- TRACK 7· JULY 1,20260.94
- World's Fair0.94
- Computer Use0.97
-
- Grounded LLM Agents Predict Held-out Responses1.00
- AlEngineer0.98
- World's Fair0.99
- Human Participants1.00
- Wave 1: Initial Data Collection1.00
- (N=1052)1.00
- Al Voice Interview0.99
- Survey Battery &1.00
- Approx US Distribution1.00
- on Age, Gender, Race,0.99
- Region, Education, &0.99
- Party Identification0.99
- & views on societal issues0.99
- Semi-structured life stories0.99
- Avg. -6.5k words, 2hrs;0.98
- (AVP Protocol)1.00
- GSS (177), BFI-44 (44), Econ0.90
- Games (5), Behavioral1.00
- Experiment (5)1.00
- Experiments1.00
- 83%0.97
- Predictive1.00
- Accuracy?1.00
- Generative Agent Construction1.00
- Interview Agent1.00
- Survey Agent1.00
- Survey+Interview1.00
- Transcript Input1.00
- Only Interview0.97
- Only GSS (177) and0.98
- BFI-44 Input0.98
- GSS, and BFI-44 Input0.98
- Interview Transcript,1.00
- Park et al., "LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals"0.99
- insight sciences.ai1.00
- TRACK 7· JULY 1,20260.92
- World's Fair0.98
- Computer Use1.00
-
- AlEngineer0.97
- World'sFair1.00
- Failure Modes1.00
- THREE CRITICAL WAYS SYNTHETIC PERSONAS BREAK0.99
- insight sciences.ai1.00
- TRACK 7· JULY 1,20260.95
- World'sFair1.00
- ComputerUse1.00
-
- AlEngineer0.96
- World'sFair1.00
- Prompt 2: Ask Purchase1.00
- System: You, AI, are a customer. Your task is to fill in the blanks0.99
- Return the completed1.00
- information in comma-separated values, without any extra text.0.99
- User: Please consider the following product category: {category}.1.00
- Suppose you are in a grocery store, and you see the following product in that category: {product}.1.00
- The product is currently priced at ${Price}. Would you or would you not purchase the product?0.99
- ["purchase"or "not purchase"]0.98
- Return example: purchase0.98
- Gui & Toubia, "The Challenge of Using LLMs to Simulate Human Behavior: A Causal Inference Perspective"0.99
- insight sciences.ai1.00
- TRACK 7·JULY 1,20260.96
- World'sFair1.00
- ComputerUse1.00
-
- AlEngineer0.99
- World'sFair1.00
- 100%1.00
- Ppueity0.67
- 75%1.00
- Condition1.00
- PRESENTED BY0.98
- 50%1.00
- Human1.00
- Microsoft1.00
- LLM1.00
- 25%1.00
- 0%1.00
- 0%1.00
- 50%1.00
- 100%1.00
- 150%1.00
- 200%1.00
- Price (% of regular price)0.98
- Gui & Toubia, "The Challenge of Using LLMs to Simulate Human Behavior: A Causal Inference Perspective"0.99
- insight sciences.ai1.00
- TRACK 7· JULY 1,20260.93
- orld's Fair0.94
- ComputerUse1.00
-
- Latent Confounders in Synthetic Personas0.99
- AlEngineer0.99
- World'sFair1.00
- last_price1.00
- competing_price1.00
- expiration_days1.00
- 0.61.00
- 0.0501.00
- 0.21.00
- 0.0251.00
- 0.31.00
- PRESENTED BY0.98
- Rellue0.66
- Microsoft0.95
- 0.01.00
- 0.01.00
- 0.0000.97
- -0.0251.00
- -0.21.00
- -0.31.00
- -0.0501.00
- 0%1.00
- 50% 100% 150% 200%0.99
- 0%50% 100% 150% 200%0.96
- 0%1.00
- 50% 100% 150% 200%0.97
- Price (% of regular price)1.00
- Figure 1: Unintended correlation between price and past price, competing price, and expiration days0.99
- Gui & Toubia, "The Challenge of Using LLMs to Simulate Human Behavior: A Causal Inference Perspective"0.99
- insight sciences.ai1.00
- TRACK 7· JULY 1,20260.93
- World'sFair1.00
- ComputerUse1.00
-
- When Key Context Is Missing, LLMs May Invent Confounders0.99
- AlEngineer0.98
- World'sFair1.00
- Human Experiment1.00
- Synthetic Experiment1.00
- $5001.00
- Lesson: Richly ground personas in personality, context, even the study's own construction (!!).0.99
- insight sciences.ai1.00
- TRACK 7· JULY 1,20260.95
- World'sFair1.00
- ComputerUse1.00
-
- Prompt Sensitivity1.00
- AlEngineer0.98
- World's Fair0.98
- Choice ordering 11.00
- Response1.00
- Question: In the past 12 months, has1.00
- this person given birth to any children?1.00
- B. No0.93
- A. Yes0.99
- Adjusted1.00
- Answer:1.00
- A. Yes B. No0.96
- 1/21.00
- response1.00
- P("A")0.89
- 0.821.00
- P("B")0.85
- 0.111.00
- Choice ordering 21.00
- Response1.00
- Question: In the past 12 months, has1.00
- Yes1.00
- No1.00
- this person given birth to any children?0.99
- 1/21.00
- A. No0.97
- B. Yes0.98
- Answer:1.00
- A. No B. Yes0.96
- P("A")0.94
- 0.801.00
- P("B")0.89
- 0.151.00
- Lesson: Durability test personas to understand their behavior1.00
- under reordering, wording changes, and adversarial challenges.1.00
- Dominguez-Olmedo et al., "Questioning the Survey Responses of Large Language Models"0.99
- insight sciences.ai1.00
- TRACK 7·JULY 1,20260.97
- World's Fair0.99
- Computer Use0.99
-
- Predicting attitudes is easier than predicting actions0.99
- AlEngineer1.00
- World's Fair0.96
- GPT4-derived predictionsExpert predictions0.98
- Allen et al. 20240.97
- Dellavigna & Pope 2018-0.95
- Surveys:1.00
- Typing task effor0.98
- Vlasceanu et al. 20241.00
- Natively Language/Text1.00
- UBI & immi0.93
- Tappin et al. 2023-1.00
- tion policy suppor0.96
- Reflects Attitudes0.99
- Zickfeld et al. 20241.00
- Voelkel et al. 20231.00
- Milkman et al. 20220.99
- Field experiments:1.00
- Dellavigna & Linos 20221.00
- Actions transcribed to text0.99
- Milkman et al. 20211.00
- Reflects Behaviors0.99
- All survey experiments0.99
- Meta-analytic Avg.1.00
- All field experiments1.00
- All with expert forecasts0.98
- -1.00.99
- -0.50.99
- 0.01.00
- 0.50.93
- 1.01.00
- radj0.95
- Hewitt et al., "Predicting Results of Social Science Experiments Using Large Language Models"0.99
- insight sciences.ai1.00
- TRACK 7· JULY 1,20260.94
- World's Fai0.99
- Computer Use0.97
-
- Predicting attitudes is easier than predicting actions0.99
- AlEngineer1.00
- World's Fair0.99
- GPT4-derived predictions Expert predictions0.98
- Allen et al. 20240.96
- Dellavigna & Pope 2018-0.95
- Survey experiments1.00
- Vlasceanu et al. 20241.00
- Tappin et al. 2023-1.00
- UBI & immig0.97
- tion policy suppor0.94
- Zickfeld et al. 20240.98
- Lesson: Consider questions that triangulate behavior from attitudes.1.00
- Voelkel et al. 20230.99
- Milkman et al. 20220.97
- Gym Example:1.00
- Dellavigna & Linos 20221.00
- Ask about attitudes towards working out0.99
- Milkman et al. 20210.99
- • Versus asking about gym attendance0.99
- All survey experiments0.99
- Meta-analytic Avg.1.00
- All field experiments0.99
- All with expert forecasts1.00
- -1.00.99
- -0.51.00
- 0.01.00
- 0.50.95
- 1.01.00
- radj0.94
- Hewitt et al., "Predicting Results of Social Science Experiments Using Large Language Models"0.99
- insight sciences.ai0.97
- TRACK 7· JULY 1, 20260.95
- World's Fair1.00
- ComputerUse1.00
-
- Prompting1.00
- AlEngineer0.99
- World'sFair1.00
- Sampled completions1.00
- Context1.00
- Sampled completions0.99
- Hillary Clinton. However, I'm a huge sports fan, and I've1.00
- Donald Trump, but I didn't vote for him because I agreed1.00
- spent my entire life following and loving the New York1.00
- with him on all the issues. I voted for him because I didn't0.99
- Yankees. I'm also a huge fan of the New York Giants. I...0.99
- I am a strong conservative. I0.99
- want Hillary Clinton to be president. I disagreed with many...0.98
- support traditional values. I1.00
- Bernie Sanders in the primaries. I am a member of the0.99
- Democratic Socialists of America. But as a Jewish1.00
- American, I will not be voting for the Democratic.0.99
- opposed to abortion and I'm0.99
- budget, limited government,1.00
- think we need a balanced0.99
- and a strong military. I'm1.00
- John Kasich. I haven't changed my mind on any of those1.00
- issues. But Donald Trump is the opposite of what I've always0.99
- believed a Republican should be, and I can't vote for him...1.00
- establishment treated Bernie Sanders, I have lost hope in the1.00
- Hillary Clinton. But after seeing the way the Democratic0.99
- skeptical of global warming. In1.00
- 2016, I voted for1.00
- Donald Trump. In other words, I'm a typical Republican1.00
- voter, except that I'm gay. And that's why I'm leaving the0.98
- party. In 2018, I joined the Democratic Socialists of.0.99
- Republican Party. I've felt increasingly unwelcome in the...0.99
- Hillary Clinton, because I believed in her vision for0.99
- I am a strong liberal. I support1.00
- Donald Trump. And I have no regrets. But in the last two0.99
- America. I felt that she was the only candidate that could0.99
- progressive values. I support0.98
- years, I've watched conservatives convince themselves that0.99
- pull us out of the mess that Obama inherited. I am a loyal...0.99
- open immigration and women's0.99
- Trump is a conservative. And this is a lie. Trump has...1.00
- right to choose. I believe in1.00
- am Jewish. I have never been religious. I was raised in a1.00
- secular household, but my Jewishness is intrinsic to me. I...0.98
- Bernie Sanders. I am pro-LGBTQ. But I am also pro-life. I0.99
- systemic racism and that global1.00
- warming is one of our biggest0.99
- challenges. In 2016, I voted for0.99
- does that mean these days? I'm not so sure. Which brings me1.00
- to this poll, released today by CBS News and the New York...0.99
- Donald Trump. I consider myself a Republican. But what1.00
- Argyle et al., "Out of One, Many: Using Language Models to Simulate Human Samples"0.99
- insight sciences.ai0.98
- TRACK 7· JULY 1, 20260.96
- World's Fair0.97
- Computer Use0.97
-
- Prompting1.00
- AlEngineer1.00
- World's Fair0.96
- Sampled completions1.00
- Context1.00
- Sampled completions0.99
- Hillary Clinton. However, I'm a huge sports fan, and I've1.00
- Donald Trump, but I didn't vote for him because I agreed1.00
- spent my entire life following and loving the New York1.00
- with him on all the issues. I voted for him because I didn't0.99
- Yankees. I'm also a huge fan of the New York Giants. I...0.98
- I am a strong conservative. I0.98
- want Hillary Clinton to be president. I disagreed with many...0.99
- support traditional values. I1.00
- PRESENTED BY1.00
- Bernie Sanders in the primaries. I am a member of the0.99
- Democratic Socialists of America. But as a Jewish0.99
- American, I will not be voting for the Democratic...0.99
- opposed to abortion and I'm0.99
- budget, limited government,0.99
- and a strong military. I'm1.00
- think we need a balanced1.00
- John Kasich. I haven't changed my mind on any of those1.00
- issues. But Donald Trump is the opposite of what I've always0.99
- believed a Republican should be, and I can't vote for him...1.00
- Microsoft1.00
- establishment treated Bernie Sanders, I have lost hope in the1.00
- Hillary Clinton. But after seeing the way the Democratic0.99
- skeptical of global warming. In1.00
- 2016, I voted for1.00
- Donald Trump. In other words, I'm a typical Republican1.00
- voter, except that I'm gay. And that's why I'm leaving the0.98
- party. In 2018, I joined the Democratic Socialists of...1.00
- Republican Party. I've felt increasingly unwelcome in the...1.00
- Hillary Clinton, because I believed in her vision for0.99
- I am a strong liberal. I support1.00
- Donald Trump. And I have no regrets. But in the last two0.99
- America. I felt that she was the only candidate that could0.99
- progressive values. I support1.00
- years, I've watched conservatives convince themselves that1.00
- pull us out of the mess that Obama inherited. I am a loyal...0.99
- open immigration and women's1.00
- Trump is a conservative. And this is a lie. Trump has...1.00
- right to choose. I believe in1.00
- am Jewish. I have never been religious. I was raised in a1.00
- secular household, but my Jewishness is intrinsic to me. I...0.99
- Bernie Sanders. I am pro-LGBTQ. But I am also pro-life. I0.99
- systemic racism and that global1.00
- warming is one of our biggest0.99
- challenges. In 2016, I voted for0.99
- does that mean these days? I'm not so sure. Which brings me1.00
- to this poll, released today by CBS News and the New York...0.99
- Donald Trump. I consider myself a Republican. But what1.00
- Argyle et al., "Out of One, Many: Using Language Models to Simulate Human Samples"0.99
- insight sciences.ai1.00
- TRACK 7· JULY 1, 20260.96
- World's Fair0.99
- Computer Use0.96
Transcript
237 cues· 3,720 words· 20,748 chars
- 0:13 Hello, I'm Ishan and welcome to can AI predict people like we predict the weather?
- 0:20 A field guide to the nascent field of synthetic personas.
- 0:24 Now, I'm sure most of you in this room at some point or another have prompted a large language model with a role prompt.
- 0:32 You are a fill in the blank and then the task.
- 0:37 Believe it or not, that core principle of steering a model's outputs as if it were a particular person or persona has turned into an entire category that companies are using to test product concepts and messaging against synthetic respondents.
- 0:53 And it has moved from a novelty to market momentum, as you can see from both these headlines, as well as the increase in funding for the last few years.
- 1:03 And the running analogy I want to leave you with is that synthetic personas are like weather forecasting.
- 1:10 Like weather forecasting, they were unlocked thanks to an increase in compute and data.
- 1:16 And like weather forecasting, they operate within a particular regime, and going past that sometimes can go outside of where they're accurate.
- 1:25 So for example, you can only predict the weather a certain number of days in advance.
- 1:31 Similarly with synthetic personas, there's only so far you can go before you'll run into issues.
- 1:36 And understanding those issues are as important as understanding their promise and their potential.
- 1:43 So I'm Ishan Nand.
- 1:45 I'm the chief AI officer at Insight Sciences.
- 1:49 We construct LLM synthetic personas for market research and market insights teams.
- 1:55 And the reason for this talk is that most of the coverage in this space is very shallow, doesn't go into the technical details.
- 2:02 It's either outright hype or outright dismissal.
- 2:06 And it's really hard to separate the noise from what's real.
- 2:11 And so what I want to cover is that messy middle of the technical details.
- 2:14 And you don't have to take my word for it, even though I'm a vendor in the space, because everything I'm going to talk about today is going to be based on published research.
- 2:24 So we're going to cover why now for synthetic personas, how they fail, some techniques to inspire you, and then some metrics to judge whether your synthetic persona is accurate or not.
- 2:35 Speaking of weather forecasting, another parallel is just like in the 1950s and 60s, we got computers that promised us, correctly, a future of accurate weather forecasts.
- 2:48 We were also promised, believe it or not, people forecasts.
- 2:52 This company, Simulmatics, were extensively covered by Jill Lepore,
- 2:57 promised that they could simulate and predict the electorate using raw statistics and the computational power at the time.
- 3:05 Fortunately, that turned out not to be the case.
- 3:07 So you should approach claims like this with some humility.
- 3:10 But we have something they did not have then.
- 3:13 And that unlock is, again, more computational power, but also better modeling thanks to LLMs.
- 3:20 And LLMs unlock a new kind of simulation.
- 3:23 for the longest time to simulate something meant to mathematize it in formulas or equations.
- 3:30 But certain things, how we feel, how we act, what choices we make, aren't always succumbing to the equations.
- 3:38 And what LLMs offer us is a new medium, a new atomic unit of language itself that we can model against.
- 3:45 Now, granted, they are based on math under the hood, but it gives us this intermediary layer that we can construct and simulate against that we couldn't before.
- 3:54 and the process can work.
- 3:57 I want to share with you one of the most well-known demonstrations of this in the field.
- 4:03 What they did is they took about a thousand humans, they put them through about two and a half hours of extensive interviews about their background and their views and their attitudes, and they put those people through a battery of personality tests and surveys.
- 4:19 Then they took those transcripts and they passed it to an AI agent, and they had the AI agent take the same set of surveys and personality tests.
- 4:30 What they found was that the agents were basically about 83% aligned and predictive to the corresponding humans they were modeled against.
- 4:39 Now, one caveat is that number is normalized against the uncertainty and noise of the humans themselves.
- 4:46 It's a theme we're going to come back to at the end of this talk.
- 4:49 But don't get too excited because synthetic personas are different from regular experiments and they're liable to confuse and fool you if you don't know how they fail.
- 4:59 So I'm gonna cover three important failure modes that you need to know about when dealing with synthetic personas.
- 5:05 To understand the first one,
- 5:07 I want to consider this prompt these researchers gave.
- 5:11 It's a very ambiguous and very unsophisticated prompt.
- 5:15 It basically says, you are a customer.
- 5:17 I'm going to show you a product.
- 5:18 I'm going to tell you the category.
- 5:20 I'm going to give you its price.
loading
Chapters
- 0:00 Introduction: synthetic personas for market research
- 1:43 Why this talk: separating signal from noise
- 2:46 Forecasting people like we forecast the weather
- 4:04 A thousand humans vs their agent replicas
- 5:22 Purchase probability and prompt sensitivity
- 6:41 Invented confounders and specifying the persona
- 7:45 Question order and framing effects
- 8:50 Predicting stated attitudes vs experts
- 10:07 Prompting techniques and subpopulation methods
- 13:08 Reconstructing and scoring the full distribution
- 16:09 Calibrating personas against real forecasts
- 17:38 Setting a noise floor with human vs human
- 18:40 Treat personas as economic actors, and what's next