vidtheque.

read-only demo

Videos zMiSRliEzv4

Self Driving Products: Product Signals to Pull Requests — Joshua Snyder, PostHog

index_state ready data_status ok

AI Engineer· published 2026-06-10· 0:15:39· en-US· indexed 2026-08-11 10:52

Open on YouTube

Scene timeline

  1. Shot 0, 0:00 to 0:05, 1 of 1 keyframes kept
  2. Shot 1, 0:05 to 0:09, 1 of 1 keyframes kept
  3. Shot 2, 0:09 to 0:14, 1 of 1 keyframes kept
  4. Shot 3, 0:14 to 0:44, 1 of 1 keyframes kept
  5. Shot 4, 0:44 to 1:12, 1 of 1 keyframes kept
  6. Shot 5, 1:12 to 1:43, 1 of 1 keyframes kept
  7. Shot 6, 1:43 to 2:13, 1 of 1 keyframes kept
  8. Shot 7, 2:13 to 2:44, 0 of 1 keyframes kept
  9. Shot 8, 2:44 to 3:10, 1 of 1 keyframes kept
  10. Shot 9, 3:10 to 3:37, 0 of 1 keyframes kept
  11. Shot 10, 3:37 to 4:03, 1 of 1 keyframes kept
  12. Shot 11, 4:03 to 4:30, 0 of 1 keyframes kept
  13. Shot 12, 4:30 to 4:57, 0 of 1 keyframes kept
  14. Shot 13, 4:57 to 5:23, 1 of 1 keyframes kept
  15. Shot 14, 5:23 to 5:50, 0 of 1 keyframes kept
  16. Shot 15, 5:50 to 6:15, 1 of 1 keyframes kept
  17. Shot 16, 6:15 to 6:41, 0 of 1 keyframes kept
  18. Shot 17, 6:41 to 7:07, 1 of 1 keyframes kept
  19. Shot 18, 7:07 to 7:33, 0 of 1 keyframes kept
  20. Shot 19, 7:33 to 7:59, 1 of 1 keyframes kept
  21. Shot 20, 7:59 to 8:24, 1 of 1 keyframes kept
  22. Shot 21, 8:24 to 8:50, 0 of 1 keyframes kept
  23. Shot 22, 8:50 to 9:16, 1 of 1 keyframes kept
  24. Shot 23, 9:16 to 9:42, 0 of 1 keyframes kept
  25. Shot 24, 9:42 to 10:08, 0 of 1 keyframes kept
  26. Shot 25, 10:08 to 10:34, 0 of 1 keyframes kept
  27. Shot 26, 10:34 to 10:59, 0 of 1 keyframes kept
  28. Shot 27, 10:59 to 11:25, 1 of 1 keyframes kept
  29. Shot 28, 11:25 to 11:51, 1 of 1 keyframes kept
  30. Shot 29, 11:51 to 12:17, 0 of 1 keyframes kept
  31. Shot 30, 12:17 to 12:43, 0 of 1 keyframes kept
  32. Shot 31, 12:43 to 13:09, 0 of 1 keyframes kept
  33. Shot 32, 13:09 to 13:35, 0 of 1 keyframes kept
  34. Shot 33, 13:35 to 14:00, 0 of 1 keyframes kept
  35. Shot 34, 14:00 to 14:26, 0 of 1 keyframes kept
  36. Shot 35, 14:26 to 14:51, 0 of 1 keyframes kept
  37. Shot 36, 14:51 to 15:17, 1 of 1 keyframes kept
  38. Shot 37, 15:17 to 15:21, 1 of 1 keyframes kept
  39. Shot 38, 15:21 to 15:23, 1 of 1 keyframes kept
  40. Shot 39, 15:23 to 15:38, 1 of 1 keyframes kept

40 shot(s).

keyframes kept every frame deduplicated

What was stored

cues
192
whisperx 192
chunks
27
from 192 cues
keyframes
21
kept of 40 captured
frames with text
20
482 lines read
chapters
0
from the source metadata
keyframe bytes
4.5 MB
word timings on 192 cues

Provenance

Each pipeline stage, its state and the model that produced it
stage state model started took
fetch done — 2026-08-11 10:49 1m 20s
stt done — 2026-08-11 10:50 17s
chunk done — 2026-08-11 10:51 0s
text_embed done — 2026-08-11 10:51 0s
keyframe done — 2026-08-11 10:51 1m 19s
ocr done — 2026-08-11 10:52 8s
frame_embed done — 2026-08-11 10:52 3s

Frames, and what the machine read

  • 0:02 #0 done2 line(s)

    shot 0·sharpness 668.7

    1. Al Engineer0.93
    2. EUROPE1.00

  • 0:06 #1 done2 line(s)

    shot 1·sharpness 819.1

    1. PRESENTING SPONSOR0.99
    2. Google DeepMind1.00

  • 0:11 #2 done3 line(s)

    shot 2·sharpness 939.2

    1. PLATINUM SPONSORS0.98
    2. # Braintrust0.95
    3. WorkOS OpenAI0.96

  • 0:18 #3 done5 line(s)

    shot 3·sharpness 503.8

    1. PostHog1.00
    2. What if your product built itself?0.99
    3. Engineering the pipeline from product1.00
    4. signals to pull requests1.00
    5. Josh Snyder1.00

  • 1:06 #4 done24 line(s)

    shot 4·sharpness 2627.0

    1. PostHog1.00
    2. _ □ ×0.83
    3. The single platform to analyze, test,0.99
    4. observe, and delpoy new features1.00
    5. AIE1.00
    6. 山0.99
    7. Posthog1.00
    8. Session1.00
    9. Web1.00
    10. Product1.00
    11. Error1.00
    12. AI1.00
    13. replay1.00
    14. analytics1.00
    15. analytics1.00
    16. tracking1.00
    17. X0.59
    18. Experiments1.00
    19. Feature1.00
    20. flags1.00
    21. Logs1.00
    22. CDP1.00
    23. Workflows1.00
    24. Engineering the future of Al1.00

  • 1:27 #5 done23 line(s)

    shot 5·sharpness 3350.6

    1. How is observability changing?1.00
    2. Today1.00
    3. Tomorrow1.00
    4. AIE1.00
    5. ★1.00
    6. Signal happens1.00
    7. Signal happens1.00
    8. ★1.00
    9. ★1.00
    10. ★1.00
    11. → Metric changes in dashboard1.00
    12. → Background agent detects a change1.00
    13. → Human notices (hours/days later)1.00
    14. → Agent researches problem1.00
    15. → Human investigates in PostHog1.00
    16. → Agent creates a PR in the background0.99
    17. → Human writes a linear issue0.99
    18. → Ship feature behind a feature flag1.00
    19. → Human creates a PR0.99
    20. → Human reviews1.00
    21. → ship0.96
    22. Braintrust1.00
    23. WorkOS OpenAI0.94

  • 2:07 #6 done22 line(s)

    shot 6·sharpness 3252.2

    1. How is observability changing?1.00
    2. Today1.00
    3. Tomorrow1.00
    4. AIE1.00
    5. ★1.00
    6. Signal happens1.00
    7. Signal happens1.00
    8. ★1.00
    9. ★1.00
    10. → Metric changes in dashboard1.00
    11. → Background agent detects a change1.00
    12. → Human notices (hours/days later)1.00
    13. → Agent researches problem1.00
    14. → Human investigates in PostHog1.00
    15. → Agent creates a PR in the background0.99
    16. → Human writes a linear issue0.97
    17. → Ship feature behind a feature flag1.00
    18. → Human creates a PR1.00
    19. → Human reviews1.00
    20. → ship0.96
    21. AlEngineer0.97
    22. EUROPE1.00

  • 2:37 #7 skipped

    shot 7·duplicate of #6

  • 3:05 #8 done28 line(s)

    shot 8·sharpness 1772.2

    1. PostHog1.00
    2. _ □ ×0.81
    3. The pipeline1.00
    4. AIE1.00
    5. Ingest1.00
    6. Group1.00
    7. Research1.00
    8. Assess1.00
    9. Execute1.00
    10. ★1.00
    11. ★1.00
    12. Filter & standardize1.00
    13. Cluster related1.00
    14. What repo?1.00
    15. Is this problem actually1.00
    16. Write code,0.95
    17. PR1.00
    18. signals1.00
    19. signals into reports0.99
    20. What's the problem?1.00
    21. fixable?1.00
    22. test, ship1.00
    23. STREAMING1.00
    24. CLUSTERING1.00
    25. AGENTIC RESEARCH1.00
    26. LLMREASONING1.00
    27. SANDBOXED AGENT0.97
    28. Engineering the future of Al1.00

  • 3:31 #9 skipped

    shot 9·duplicate of #8

  • 3:58 #10 done35 line(s)

    shot 10·sharpness 2122.5

    1. PostHog0.94
    2. _ □ ×0.71
    3. Ingestion: filtering & normalization1.00
    4. Error tracking1.00
    5. ★1.00
    6. ★0.97
    7. stack traces, error spikes1.00
    8. ★1.00
    9. AIE1.00
    10. Safety filter0.98
    11. ★1.00
    12. ★1.00
    13. Session replay1.00
    14. LLM classifier drops injection attempts0.99
    15. Normalized signal1.00
    16. ★1.00
    17. ★1.00
    18. rage clicks, broken flows1.00
    19. source product1.00
    20. Experiments1.00
    21. Normalize1.00
    22. ● signal type0.90
    23. metric drops, significance1.00
    24. Common shape: source, type, content1.00
    25. content1.00
    26. Analytics1.00
    27. vector embedding1.00
    28. funnels, trend anomalies1.00
    29. Embed1.00
    30. weight1.00
    31. Vector embedding for cross-type search0.99
    32. Slack/external1.00
    33. complaints, discussions1.00
    34. AlEngineer0.97
    35. EUROPE1.00

  • 4:24 #11 skipped

    shot 11·duplicate of #10

  • 4:46 #12 skipped

    shot 12·duplicate of #8

  • 5:05 #13 done31 line(s)

    shot 13·sharpness 2141.2

    1. PostHog0.94
    2. _ □ ×0.78
    3. Ingestion: filtering & normalization1.00
    4. Error tracking1.00
    5. stack traces, error spikes0.99
    6. ★1.00
    7. AIE1.00
    8. Safety filter0.98
    9. ★1.00
    10. Session replay0.99
    11. LLM classifier drops injection attempts0.99
    12. Normalized signal1.00
    13. ★1.00
    14. ★1.00
    15. rage clicks, broken flows1.00
    16. source product0.97
    17. Experiments1.00
    18. Normalize1.00
    19. signal type1.00
    20. metric drops, significance1.00
    21. Common shape: source, type, content1.00
    22. content1.00
    23. Analytics1.00
    24. vector embedding1.00
    25. funnels, trend anomalies1.00
    26. Embed1.00
    27. weight1.00
    28. Vector embedding for cross-type search1.00
    29. Slack/external1.00
    30. complaints, discussions1.00
    31. Engineering the future of Al1.00

  • 5:29 #14 skipped

    shot 14·duplicate of #8

  • 5:55 #15 done40 line(s)

    shot 15·sharpness 2391.9

    1. PostHog1.00
    2. _ □ ×0.75
    3. Rewrite the query, don't search with raw embeddings1.00
    4. NEW SIGNAL1.00
    5. Error tracking1.00
    6. NullPointerException in CheckoutService.java:1420.99
    7. ★1.00
    8. ★1.00
    9. ★0.95
    10. AIE1.00
    11. Naive: search with raw embedding0.99
    12. Better: search with LLM-generated queries1.00
    13. ★1.00
    14. ★1.00
    15. Embed the signal directly, find nearest neighbors0.99
    16. LLM rewrites signal into cross-type search queries1.00
    17. ★1.00
    18. ★1.00
    19. TOP MATCHES1.00
    20. GENERATED QUERIES0.99
    21. Error tracking0.99
    22. "checkout flow errors and failures"0.97
    23. IndexOutOfBoundsException in CartService.java:891.00
    24. "user-facing issues on the payment page"0.99
    25. Error tracking1.00
    26. NullPointerException in PaymentService.java:2030.98
    27. "checkout conversion drops"0.99
    28. Error tracking1.00
    29. TOP MATCHES0.98
    30. ClassCastException in OrderService.java:570.98
    31. Analytics1.00
    32. Error rate 3x in /api/checkout0.99
    33. Misses the Slack complaint and funnel drop entirely.0.99
    34. Groups by type, not by issue.0.99
    35. "hey checkout is broken for me"0.99
    36. Slack1.00
    37. Analytics1.00
    38. Checkout funnel conversion -18% since Tuesday1.00
    39. AlEngineer0.96
    40. EUROPE1.00

  • 6:38 #16 skipped

    shot 16·duplicate of #15

  • 7:01 #17 done38 line(s)

    shot 17·sharpness 2404.0

    1. PostHog1.00
    2. _ □ ×0.84
    3. Rewrite the query, don't search with raw embeddings1.00
    4. NEW SIGNAL1.00
    5. NullPointerException in CheckoutService.java:1420.99
    6. Error tracking1.00
    7. ★1.00
    8. ★1.00
    9. AIE1.00
    10. Naive: search with raw embedding0.99
    11. Better: search with LLM-generated queries0.99
    12. ★1.00
    13. ★1.00
    14. Embed the signal directly, find nearest neighbors1.00
    15. LLM rewrites signal into cross-type search queries1.00
    16. ★1.00
    17. ★1.00
    18. TOP MATCHES0.99
    19. GENERATED QUERIES0.99
    20. Error tracking1.00
    21. "checkout flow errors and failures"0.99
    22. IndexOutOfBoundsException in CartService.java:891.00
    23. "user-facing issues on the payment page"0.99
    24. Error tracking1.00
    25. NullPointerException in PaymentService.java:2030.99
    26. "checkout conversion drops"1.00
    27. Error tracking1.00
    28. TOP MATCHES0.98
    29. ClassCastException in OrderService.java:570.99
    30. Analytics1.00
    31. Error rate 3x in /apl/checkout0.99
    32. Misses the Slack complaint and funnel drop entirely.0.99
    33. Groups by type, not by issue.1.00
    34. "hey checkout is broken for me"1.00
    35. Slack1.00
    36. Analytics1.00
    37. Checkout funnel conversion -18% since Tuesday1.00
    38. Engineering the future of Al1.00

  • 7:10 #18 skipped

    shot 18·duplicate of #17

  • 7:36 #19 done50 line(s)

    shot 19·sharpness 2122.0

    1. PostHog1.00
    2. _ □ ×0.79
    3. Research & Actionability0.98
    4. Research agent1.00
    5. Repo selection1.00
    6. Multi-turn investigation with full context0.98
    7. Agent picks the right codebase via GitHub0.99
    8. TOOLS1.00
    9. Promoted report1.00
    10. PostHog MCP1.00
    11. Codebase1.00
    12. ★1.00
    13. weight above threshold1.00
    14. analytics, errors, replays1.00
    15. read files, search code0.96
    16. AIE1.00
    17. NulPointorExoeption in Checkout.0.94
    18. Error tracking1.00
    19. PRs, commits, deploys0.97
    20. GitHub API1.00
    21. Linear, Notion, etc.0.98
    22. External MCPs1.00
    23. ★1.00
    24. ★1.00
    25. ★1.00
    26. Emor rate 3x in /aplicheckout0.91
    27. "checkout is broken for me"0.97
    28. Analytics1.00
    29. Slack1.00
    30. PRODUCES1.00
    31. human-readable0.97
    32. Summary1.00
    33. Priority1.00
    34. PO - P40.82
    35. from git blame0.98
    36. Reviewers1.00
    37. Actionability judgment1.00
    38. Immediately actionable1.00
    39. Needs human input1.00
    40. Not actionable1.00
    41. Create task, agent writes the fix1.00
    42. Surface report to tagged owners0.99
    43. Reset weight, keep listening1.00
    44. Clear stack trace, obvious code fix,1.00
    45. Product decision, ambiguous scope,1.00
    46. No useful fix derivable,0.98
    47. high confidence0.99
    48. needs judgment call1.00
    49. may become actionable later1.00
    50. Engineering the future of Al1.00

  • 8:21 #20 done52 line(s)

    shot 20·sharpness 2204.8

    1. PostHog1.00
    2. _ □ ×0.75
    3. Research & Actionability1.00
    4. Research agent1.00
    5. Repo selection1.00
    6. Multi-turn investigation with full context0.99
    7. Agent picks the right codebase via GitHub0.98
    8. TOOLS1.00
    9. Promoted report1.00
    10. PostHog MCP1.00
    11. Codebase1.00
    12. ★1.00
    13. weight above threshold0.99
    14. analytics, errors, replays0.96
    15. read files, search code0.97
    16. AIE1.00
    17. NulPointerException in Checkout.0.85
    18. Error tracking1.00
    19. PRs, commits, deploys0.97
    20. GitHub API1.00
    21. Linear, Notion, etc.0.99
    22. External MCPs1.00
    23. ★1.00
    24. ★1.00
    25. ★1.00
    26. ★1.00
    27. Emor rate 3x in lapüicheckout0.91
    28. "checkout is broken for me"0.96
    29. Analytics1.00
    30. Slack1.00
    31. PRODUCES1.00
    32. uman-readabie0.95
    33. Summary1.00
    34. Priority0.99
    35. PO -P40.81
    36. Reviewers1.00
    37. from git blame0.98
    38. Actionability judgment0.99
    39. Immediately actionable1.00
    40. Needs human input1.00
    41. Not actionable1.00
    42. Create task, agent writes the fix0.98
    43. Surface report to tagged owners1.00
    44. Reset weight, keep listening0.99
    45. Clear stack trace, obvious code fix,1.00
    46. Product decision, ambiguous scope,0.99
    47. No useful fix derivable,0.99
    48. high confidence1.00
    49. needs judgment call0.98
    50. may become actionable later1.00
    51. Braintrust1.00
    52. WorkOS OpenAI0.96

  • 8:35 #21 skipped

    shot 21·duplicate of #20

  • 9:13 #22 done52 line(s)

    shot 22·sharpness 2124.7

    1. PostHog1.00
    2. _ □ ×0.80
    3. Research & Actionability0.98
    4. Research agent1.00
    5. Repo selection1.00
    6. Multi-turn investigation with full context0.96
    7. Agent picks the right codebase via GitHub0.99
    8. TOOLS1.00
    9. Promoted report1.00
    10. PostHog MCP1.00
    11. Codebase1.00
    12. ★1.00
    13. weight above threshold1.00
    14. analytics, errors, replays1.00
    15. read files, search code0.98
    16. AIE1.00
    17. NulPointerException in Checkout.0.91
    18. Error tracking0.96
    19. PRs, commis, deploys0.96
    20. GitHub API1.00
    21. Linear, Notion, etc.0.97
    22. External MCPs1.00
    23. ★1.00
    24. ★1.00
    25. ★1.00
    26. ★1.00
    27. ★1.00
    28. Emor rate 3x in /aplicheckout0.90
    29. "checkout is broken for ma"0.93
    30. Analytics1.00
    31. PRODUCES1.00
    32. tuman-readable0.89
    33. Summary1.00
    34. Priority0.99
    35. PO- P40.92
    36. from git blame0.97
    37. Reviewers1.00
    38. Actionability judgment1.00
    39. Immediately actionable1.00
    40. Needs human input1.00
    41. Not actionable1.00
    42. Create task, agent writes the fix0.99
    43. Surface report to tagged owners1.00
    44. Reset weight, keep listening1.00
    45. Clear stack trace, obvious code fix,1.00
    46. Product decision, ambiguous scope,1.00
    47. No useful fix derivable,0.97
    48. high confidence1.00
    49. needs judgment call1.00
    50. may become actionable later0.99
    51. AlEngineer0.96
    52. EUROPE1.00

  • 9:34 #23 skipped

    shot 23·duplicate of #22

Transcript

192 cues· 2,756 words· 14,574 chars

  1. 0:15 So I'm Josh.
  2. 0:15 I'm from Postog.
  3. 0:17 If you haven't heard of us, you might know us because of some hedgehogs.
  4. 0:20 Or you might have seen our founder, James, posting some funny things on LinkedIn.
  5. 0:25 He's quite popular.
  6. 0:27 I'm going to be talking today about what if your product built itself.
  7. 0:31 And the pipeline that we're currently working on, which we're trying to turn our observability data, instead of something that you read and that you interpret based on dashboards, we're trying to turn that into something that submits pull requests for you.
  8. 0:45 Cool, yeah, so quick background on PostHog.
  9. 0:47 We've got a bunch of tools.
  10. 0:49 We started out as a product analytics company.
  11. 0:51 We now have session replay, web analytics, error tracking, experiments.
  12. 0:55 This isn't a pitch that you should use PostHog.
  13. 0:58 This is just to say that we've got a lot of data about your product.
  14. 1:01 So if you connect PostHog to your product, we're collecting a huge amount of data from various different sources that we then show to you so that you can explore that data yourself.
  15. 1:13 But right now, how observability is working in PostHog, you're collecting all this data for your product.
  16. 1:20 And then you're going to a PostHog dashboard to figure out what's going on.
  17. 1:25 And we think this is super slow and that we should change that.
  18. 1:28 So right now, something happens in your products.
  19. 1:30 We call this a signal.
  20. 1:31 That changes a metric on one of your dashboards.
  21. 1:34 And then you might log into PostHog a few hours or maybe some days later, and you notice a change in that dashboard.
  22. 1:41 And you investigate a problem.
  23. 1:42 And then maybe the problem's not that important.
  24. 1:45 So instead of tackling it right now, you're going to put it in a linear issue or whatever.
  25. 1:48 A few days later, you try and create a PR for this problem.
  26. 1:52 Then you review it, and you ship it.
  27. 1:54 You get the message.
  28. 1:55 This is a pretty slow process.
  29. 1:57 From start to finish, this is going to take anywhere from a few hours to a few days.
  30. 2:01 And it's not very interesting, but it represents a lot of your work as a software engineer.
  31. 2:06 So what we want to do tomorrow, what we're working on right now, is that a product signal happens.
  32. 2:13 And instead of waiting to see that in your dashboard, we want to run a background agent to figure out what's going wrong.
  33. 2:20 And then once they've figured that out, we just want to create a PR for you automatically.
  34. 2:25 So instead of ever looking at your analytics dashboard or your errors or your logs, we just want you to look at PRs that are ready for you in GitHub.
  35. 2:35 And if we create the PR, maybe you want to review that.
  36. 2:38 Or maybe we can just ship that immediately behind a feature flag if it's not a risky change.
  37. 2:45 Cool.
  38. 2:45 So I'm going to go over the pipeline that we've built to do this.
  39. 2:49 And just whilst I go over that, I'm going to share a few tips, lessons that we've learned, things that were hard about building this pipeline.
  40. 2:57 So the pipeline has a few key steps.
  41. 3:00 At first, we're ingesting a lot of signals.
  42. 3:02 In post-hug, we have a huge amount of events.
  43. 3:05 We're ingesting trillions of events a month.
  44. 3:10 this pipeline needs to handle a lot of noise.
  45. 3:13 And then once we've ingested those events, we need to group them.
  46. 3:16 So if you think of an error tracking issue and then a session recording, those are two completely different things, but they might be representing the same problem in your product.
  47. 3:26 So once we've ingested them, we group them.
  48. 3:29 Then we're going to be running a research agent on them.
  49. 3:33 This specific issue, what is actually the problem that is causing the error spike or causing the issue that the user faced in the replay?
  50. 3:41 And what repo does this belong to?

Open at this second