read-only demo

Videos 4uFVSLgD2Q4

Agents in Production: How OpenGov Built and Scaled OG Assist - Gabe De Mesa, OpenGov

index_state ready data_status ok

AI Engineer· published 2026-06-26· 0:18:29· en-US· indexed 2026-08-11 08:31

Open on YouTube

Scene timeline

  1. Shot 0, 0:00 to 0:27, 1 of 1 keyframes kept
  2. Shot 1, 0:27 to 0:54, 0 of 1 keyframes kept
  3. Shot 2, 0:54 to 1:22, 1 of 1 keyframes kept
  4. Shot 3, 1:22 to 1:50, 0 of 1 keyframes kept
  5. Shot 4, 1:50 to 2:07, 1 of 1 keyframes kept
  6. Shot 5, 2:07 to 2:32, 1 of 1 keyframes kept
  7. Shot 6, 2:32 to 2:37, 1 of 1 keyframes kept
  8. Shot 7, 2:37 to 2:49, 0 of 1 keyframes kept
  9. Shot 8, 2:49 to 2:52, 1 of 1 keyframes kept
  10. Shot 9, 2:52 to 2:58, 0 of 1 keyframes kept
  11. Shot 10, 2:58 to 3:07, 0 of 1 keyframes kept
  12. Shot 11, 3:07 to 3:25, 0 of 1 keyframes kept
  13. Shot 12, 3:25 to 3:55, 1 of 1 keyframes kept
  14. Shot 13, 3:55 to 4:25, 1 of 1 keyframes kept
  15. Shot 14, 4:25 to 4:33, 1 of 1 keyframes kept
  16. Shot 15, 4:33 to 4:59, 1 of 1 keyframes kept
  17. Shot 16, 4:59 to 5:24, 0 of 1 keyframes kept
  18. Shot 17, 5:24 to 5:49, 0 of 1 keyframes kept
  19. Shot 18, 5:49 to 6:18, 1 of 1 keyframes kept
  20. Shot 19, 6:18 to 6:47, 0 of 1 keyframes kept
  21. Shot 20, 6:47 to 7:16, 0 of 1 keyframes kept
  22. Shot 21, 7:16 to 7:45, 0 of 1 keyframes kept
  23. Shot 22, 7:45 to 8:15, 1 of 1 keyframes kept
  24. Shot 23, 8:15 to 8:45, 0 of 1 keyframes kept
  25. Shot 24, 8:45 to 9:15, 0 of 1 keyframes kept
  26. Shot 25, 9:15 to 9:40, 1 of 1 keyframes kept
  27. Shot 26, 9:40 to 10:05, 0 of 1 keyframes kept
  28. Shot 27, 10:05 to 10:30, 0 of 1 keyframes kept
  29. Shot 28, 10:30 to 11:13, 1 of 1 keyframes kept
  30. Shot 29, 11:13 to 11:43, 1 of 1 keyframes kept
  31. Shot 30, 11:43 to 12:12, 0 of 1 keyframes kept
  32. Shot 31, 12:12 to 12:25, 1 of 1 keyframes kept
  33. Shot 32, 12:25 to 12:51, 1 of 1 keyframes kept
  34. Shot 33, 12:51 to 13:17, 1 of 1 keyframes kept
  35. Shot 34, 13:17 to 13:44, 0 of 1 keyframes kept
  36. Shot 35, 13:44 to 14:10, 0 of 1 keyframes kept
  37. Shot 36, 14:10 to 14:52, 1 of 1 keyframes kept
  38. Shot 37, 14:52 to 15:21, 1 of 1 keyframes kept
  39. Shot 38, 15:21 to 15:50, 0 of 1 keyframes kept
  40. Shot 39, 15:50 to 16:19, 0 of 1 keyframes kept
  41. Shot 40, 16:19 to 16:45, 1 of 1 keyframes kept
  42. Shot 41, 16:45 to 17:11, 0 of 1 keyframes kept
  43. Shot 42, 17:11 to 17:37, 0 of 1 keyframes kept
  44. Shot 43, 17:37 to 18:21, 1 of 1 keyframes kept
  45. Shot 44, 18:21 to 18:29, 1 of 1 keyframes kept

45 shot(s).

keyframes kept every frame deduplicated

What was stored

cues
152
whisperx 152
chunks
33
from 152 cues
keyframes
23
kept of 45 captured
frames with text
23
687 lines read
chapters
0
from the source metadata
keyframe bytes
4.6 MB
word timings on 152 cues

Provenance

Each pipeline stage, its state and the model that produced it
stage state model started took
fetch done 2026-08-11 08:29 1m 06s
stt done 2026-08-11 08:30 20s
chunk done 2026-08-11 08:30 0s
text_embed done 2026-08-11 08:30 0s
keyframe done 2026-08-11 08:30 56s
ocr done 2026-08-11 08:31 13s
frame_embed done 2026-08-11 08:31 4s

Frames, and what the machine read

  • 0:05 #0 done5 line(s)

    shot 0·sharpness 833.5

    1. OpenGov70.94
    2. Al Engineer Conference 20261.00
    3. Agents in Production: How OpenGov1.00
    4. Built and Scaled OG Assist1.00
    5. OG Assist0.94
  • 0:33 #1 skipped

    shot 1·duplicate of #0

  • 1:00 #2 done18 line(s)

    shot 2·sharpness 1964.7

    1. OpenGov70.96
    2. Agenda1.00
    3. 1.0.93
    4. Meet OG Assist0.98
    5. 2.0.90
    6. The Origin Story1.00
    7. 3.0.94
    8. Betting on Effect0.98
    9. 4.0.95
    10. The Core Agent Loop0.99
    11. 5.0.97
    12. A2A, Evals & Sandboxing1.00
    13. 6.0.98
    14. Long Context Handling1.00
    15. 7.0.94
    16. Monitoring & Observability1.00
    17. 8.0.97
    18. Tools, Skills & Dev Workflows1.00
  • 1:34 #3 skipped

    shot 3·duplicate of #2

  • 1:55 #4 done5 line(s)

    shot 4·sharpness 571.1

    1. OpenGovT0.93
    2. Your host1.00
    3. Gabe1.00
    4. De Mesa0.99
    5. Software Engineer III @ OpenGov0.96
  • 2:27 #5 done9 line(s)

    shot 5·sharpness 1690.9

    1. OpenGov70.98
    2. About OpenGov1.00
    3. Software for1.00
    4. more effective,0.99
    5. accountable1.00
    6. government.1.00
    7. Founded over 14 years ago, OpenGov builds ERP, budgeting,1.00
    8. asset management, and permitting software for state and local1.00
    9. government and OG Assist now connects across all of it.1.00
  • 2:36 #6 done46 line(s)

    shot 6·sharpness 2155.1

    1. OpenGov70.96
    2. About OpenGov1.00
    3. OpenGov Utility Billing Faworites0.87
    4. Workflows0.93
    5. + OG Assist0.95
    6. QSearch0.97
    7. Cloud City, USA1.00
    8. Software for1.00
    9. Prepare Bil Run0.91
    10. View Accounts0.99
    11. Apply Penalty Charges0.96
    12. ProcessPayments0.99
    13. ④ Create a Deposit0.95
    14. more effective,1.00
    15. Workspace1.00
    16. Days Sales Outstanding1.00
    17. Total Amount Owed0.97
    18. Total Payments Taken Today0.98
    19. 00.61
    20. 6 lar moth0.62
    21. $691.00
    22. accountable1.00
    23. Target: -30 days0.95
    24. As of Jume 18. 2026 2:22 am0.97
    25. As of June 18. 2026 2:22 am0.93
    26. Receipts by Payment Method1.00
    27. Jrme 20260.79
    28. government.1.00
    29. @Cash $0.000.96
    30. 0.0%1.00
    31. Check $0.000.99
    32. 0.0%1.00
    33. Bank Draft $0.001.00
    34. 0.0%1.00
    35. Money Order $0.000.99
    36. 0.0%1.00
    37. Your billing breakdown awaits!0.98
    38. ACH $0.000.97
    39. 0.0%1.00
    40. $ ACV $0.000.95
    41. 0.0%0.91
    42. Founded over 14 years ago, OpenGov builds ERP, budgeting,0.99
    43. Dabit/Credit Card s0.000.90
    44. 0.0%0.99
    45. asset management, and permitting software for state and local0.99
    46. government and OG Assist now connects across all of it.1.00
  • 2:47 #7 skipped

    shot 7·duplicate of #6

  • 2:50 #8 done56 line(s)

    shot 8·sharpness 2114.5

    1. OpenGov70.94
    2. About OpenGov1.00
    3. OpenGov Utility Billing Faworites0.91
    4. Cash Receipts0.97
    5. Workflows0.94
    6. +:0Assist0.89
    7. Q Search0.94
    8. + Create Custon0.94
    9. Cloud City, USA1.00
    10. OG Assist0.90
    11. Software for1.00
    12. Prepare Biit Run0.90
    13. View Accounts1.00
    14. Apply Penalty1.00
    15. Charges1.00
    16. 0.88
    17. Process1.00
    18. Payments1.00
    19. Croate a Deposis0.90
    20. Welcome Gabe!0.99
    21. more effective,1.00
    22. Summarize what happened in this workspace0.99
    23. Workspace1.00
    24. What should I tackle first today?0.97
    25. Draft a response I can sond to my teans0.97
    26. Days Sales Outstanding0.99
    27. 00.50
    28. • 8t lat mnth0.66
    29. $691.00
    30. Total Amount Owed0.98
    31. Total Payments Taken Today0.99
    32. Create a quick action plen from my current0.99
    33. tasks0.96
    34. accountable1.00
    35. Target: 30 days0.98
    36. As of June 18, 2026 2:22 am0.99
    37. As of June 18, 2026 2:22.am0.97
    38. Receipts by Payment Method1.00
    39. Ane. 20280.80
    40. government.1.00
    41. @ Cash $0.000.96
    42. 0.0%1.00
    43. Check $0.000.96
    44. 0.0%1.00
    45. Bank Draft $0.000.99
    46. 0.0%1.00
    47. Maney Order 50.000.95
    48. 0.0%0.99
    49. ur biing breakdown awaits!0.94
    50. $0.000.91
    51. 0:01/0:221.00
    52. $ ACV $0.000.82
    53. 0.0%0.99
    54. Founded over 14 years ago, OpenGov builds ERP, budgeting,0.99
    55. asset management, and permitting software for state and local0.99
    56. government and OG Assist now connects across all of it.1.00
  • 2:57 #9 skipped

    shot 9·duplicate of #6

  • 3:06 #10 skipped

    shot 10·duplicate of #6

  • 3:18 #11 skipped

    shot 11·duplicate of #6

  • 3:43 #12 done61 line(s)

    shot 12·sharpness 1647.0

    1. OpenGov70.98
    2. Home0.91
    3. Cash Receipts0.99
    4. Management Center0.99
    5. Workflows0.96
    6. +: OG Assist0.93
    7. QSearch1.00
    8. + Create Customer0.97
    9. Origin Story1.00
    10. Cloua0.99
    11. OG Assist1.00
    12. ×0.96
    13. One bet on1.00
    14. Prepare Bill Run0.96
    15. View Accounts0.99
    16. Charges1.00
    17. 0.78
    18. Payments1.00
    19. Process1.00
    20. ④ Create a Deposit0.94
    21. Welcome Gabe!1.00
    22. Summarize what happened in this workspace.1.00
    23. agents, one0.97
    24. Workspace1.00
    25. What sthould I tackle first today?0.96
    26. Days Sales Outstanding1.00
    27. 00.52
    28. + 68 lnet month0.88
    29. $691.00
    30. Total Amount Owed0.97
    31. Total Payments Taken Today1.00
    32. Create a quick action plan from my current0.99
    33. Draft a response I can send to my team.0.99
    34. tasks.1.00
    35. immediate yes.1.00
    36. Target: 30 days0.99
    37. As of June 18, 2026 2:22 am1.00
    38. As of June 18, 2026 2:22 am0.97
    39. Receipts by Payment Method1.00
    40. June 20280.69
    41. @ Cash $0.000.96
    42. 0.0%1.00
    43. Check $0.001.00
    44. 0.0%1.00
    45. Bank Draft $0.000.98
    46. 0.0%1.00
    47. Money Order $0.000.98
    48. 0.0%0.94
    49. Your billing breakdown awaits!1.00
    50. ACH $0.000.97
    51. 0.0%1.00
    52. Type your message.0.98
    53. A principal engineer spun up a team focused on Al agents and1.00
    54. Once you start biling services youll see beautiful reakdown right hereere0.92
    55. $ ACV $0.000.92
    56. 0.0%1.00
    57. asked me to join. OG Assist grew from there. We started to1.00
    58. Debit/Credit Card $0.000.96
    59. 0.0%1.00
    60. integrate deeply with our products through frontend and0.99
    61. backend tools.1.00
  • 4:22 #13 done67 line(s)

    shot 13·sharpness 1730.5

    1. OpenGov70.95
    2. A Home0.85
    3. Cash Receipts0.99
    4. Management Center0.99
    5. Workflows0.98
    6. +: OG Assist0.91
    7. Q Search0.95
    8. + Create Customer0.96
    9. Origin Story1.00
    10. Cloua0.99
    11. OG Assist1.00
    12. ×0.72
    13. One bet on1.00
    14. What can you see on the screen? Highight0.99
    15. a next step0.99
    16. Prepare Bill Run0.96
    17. Charges1.00
    18. 0.57
    19. Process1.00
    20. Payments1.00
    21. ④ Create a Deposit0.92
    22. Current screen1.00
    23. agents, one0.97
    24. Workspace1.00
    25. You're on the OpenGov Uility Blling Home page lor Cloud0.94
    26. City, USA0.99
    27. I can see:0.97
    28. Days Sales Outstanding1.00
    29. Total Amount Owed1.00
    30. Total Payments Taken Today0.99
    31. Main actions: Prepar Bil Run, View Accounts. Apply0.96
    32. 00.53
    33. • 68 lest month0.76
    34. $691.00
    35. 00.51
    36. • Workspaco metrics like Days Sales Outstanding. Total0.95
    37. Penalty Charges. Process Payments. Create a Deposit0.99
    38. immediate yes.0.99
    39. Target: <30 days0.94
    40. As of June 18, 2026 2.22 am0.98
    41. As of June 18, 2026 2:22 am0.99
    42. Amount Owed and Total Payments Taken Today0.98
    43. • Sections for Workflows in Progress and Sessions0.96
    44. • Navigation for Cash Receipts.0.95
    45. Receipts by Payment Method1.00
    46. Aune 20280.76
    47. @Cash $0.000.99
    48. 0.0%1.00
    49. Check $0.001.00
    50. 0.0%1.00
    51. Bank Draft $0.000.99
    52. 0.0%1.00
    53. Money Order $0.001.00
    54. 0.0%1.00
    55. Your billing breakdown awaits!1.00
    56. ACH $0.000.99
    57. 0.0%1.00
    58. Type your message..0.97
    59. A principal engineer spun up a team focused on Al agents and1.00
    60. Once you start biing services youl ee beautitulbreakdow right here0.91
    61. $ ACV $0.000.95
    62. 0.0%1.00
    63. asked me to join. OG Assist grew from there. We started to1.00
    64. Debit/Credit Card $0.000.98
    65. 0.0%1.00
    66. integrate deeply with our products through frontend and0.99
    67. backend tools.1.00
  • 4:29 #14 done9 line(s)

    shot 14·sharpness 1562.5

    1. OpenGov70.99
    2. Origin Story1.00
    3. One bet on0.99
    4. agents,one1.00
    5. immediate yes.1.00
    6. A principal engineer spun up a team focused on Al agents and1.00
    7. asked me to join. OG Assist grew from there. We started to1.00
    8. integrate deeply with our products through frontend and0.99
    9. backend tools.0.99
  • 4:44 #15 done2 line(s)

    shot 15·sharpness 1201.1

    1. OpenGov1.00
    2. Effect1.00
  • 5:21 #16 skipped

    shot 16·duplicate of #15

  • 5:46 #17 skipped

    shot 17·duplicate of #15

  • 6:09 #18 done22 line(s)

    shot 18·sharpness 1455.3

    1. OpenGov70.96
    2. Example1.00
    3. The Harness1.00
    4. Our agent loop,1.00
    5. import { Chat, LanguageModel } from "@effect/ai"1.00
    6. import { Effect, Stream } from "effect"0.99
    7. rebuilt1.00
    8. // Streaming chat with tool support1.00
    9. const streamingChat = Effect.gen(function* () {0.98
    10. const chat = yield* Chat.empty1.00
    11. Effect-native.1.00
    12. yield* chat0.96
    13. .streamText({0.99
    14. prompt: "Generate a creative story"1.00
    15. })0.82
    16. .pipe(Stream.runForEach((part) => Effect.sync(() => console.log(part))))0.97
    17. })0.67
    18. The core loop started on LangGraph. It now runs fully0.99
    19. Effect-native typed control flow, structured concurrency, and1.00
    20. resource safety end to end. Also allows more granular control to0.99
    21. the agent loop.1.00
    22. Source: https://effect-ts.github.io/effect/ai/ai/Chat.ts.html0.99
  • 6:29 #19 skipped

    shot 19·duplicate of #18

  • 6:50 #20 skipped

    shot 20·duplicate of #18

  • 7:36 #21 skipped

    shot 21·duplicate of #18

  • 7:57 #22 done48 line(s)

    shot 22·sharpness 1488.2

    1. OpenGov1.00
    2. interface AgentCard {1.00
    3. // Human-readable name for the Agent (e.g., "Recipe Agent")0.99
    4. name: string;1.00
    5. // Human-readable description of the Agent's function1.00
    6. description: string;1.00
    7. // URL address where the Agent is hosted0.99
    8. url: string;1.00
    9. // Service provider information for the Agent1.00
    10. provider?: {0.99
    11. Agent-to-Agent Protocol1.00
    12. organization: string;1.00
    13. url: string;1.00
    14. };0.71
    15. // Version of the Agent (format defined by provider, e.g., "1.0.0")0.98
    16. version: string;1.00
    17. // URL for the Agent's documentation1.00
    18. documentationUrl?: string;1.00
    19. Built Off The1.00
    20. // Optional capabilities supported by the Agent1.00
    21. capabilities: {0.97
    22. streaming?: boolean; // If the Agent supports SSE1.00
    23. pushNotifications?: boolean; // If the Agent can push update notifications to the client1.00
    24. stateTransitionHistory?: boolean; // If the Agent exposes task state change history1.00
    25. A2A Protocol1.00
    26. };0.87
    27. // Authentication requirements for the Agent (intended to match OpenAPI auth structure)0.99
    28. authentication: {0.99
    29. schemes: string[]; // e.g., Basic, Bearer0.97
    30. credentials?: string; // Credentials for the client to use for private Cards0.99
    31. };0.78
    32. // Default interaction modes supported by the Agent across all skills1.00
    33. defaultInputModes: string[]; // Supported input MIME types0.99
    34. defaultOutputModes: string[]; // Supported output MIME types0.98
    35. // Collection of capability units the Agent can perform1.00
    36. skills: {0.97
    37. id: string; // Unique identifier for the skill1.00
    38. The A2A protocol allows OG Assist to build off a supported1.00
    39. name: string; // Human-readable name for the skill0.98
    40. description: string; // Skill description0.99
    41. spec, rather than from the ground up.1.00
    42. tags: string[]; // Tags describing the skill's capability category (e.g., "cooking", "cu:0.98
    43. examples?: string[]; // Example scenarios or prompts the skill can execute (e.g., "I nee0.98
    44. inputModes?: string[l: // Input MIME types supported by the skill (if different from def0.98
    45. outputModes?: string[]: // Output MIME types supported by the skill (if different from d0.99
    46. }];0.92
    47. }0.98
    48. Source: https://agent2agent.info/docs/concepts/agentcard/1.00
  • 8:35 #23 skipped

    shot 23·duplicate of #22

Transcript

152 cues· 2,865 words· 14,961 chars

  1. 0:01 Hi, everyone.
  2. 0:01 My name is Gabe DeMesa.
  3. 0:03 I'm an engineer here at OpenGov, and today we're going to be talking about agents in production, specifically how OpenGov built and scaled OG Assist.
  4. 0:13 So this presentation is going to be jam-packed with just so much good stuff.
  5. 0:21 We're going to talk about AI agents.
  6. 0:23 We're going to talk about our harness.
  7. 0:25 We're going to talk about
  8. 0:27 evals, observability, traces.
  9. 0:30 We're going to talk about tools and skills.
  10. 0:35 There's going to be a lot of good stuff in here.
  11. 0:38 We're going to talk to you guys about what we do at OpenGov and how we operate at the scale that we operate at in production.
  12. 0:46 So you'll be able to see a real use case and workload with AI agents.
  13. 0:52 So without further ado, let's get started.
  14. 0:56 okay agenda so just really quickly going to go through uh high level what we're going to talk about today i'm going to tell you guys a little bit about og assist and what uh opengov is i'm going to tell you guys the origin story of how this all kind of came to be uh we're going to talk about og assist's uh big bet on effect
  15. 1:17 a little bit into our core agent loop.
  16. 1:20 We're going to talk about the A2A protocol, evals, and sandboxing.
  17. 1:23 We're going to talk about how we manage long context.
  18. 1:27 We're going to talk about monitoring observability, how we collect feedback.
  19. 1:33 and how we iterate on that feedback.
  20. 1:35 We're going to lastly also talk about tools and skills and how at OpenGov we use AI not only externally that we serve to customers, but also internally to improve our development workflows.
  21. 1:51 Just a little bit about me before we go any further.
  22. 1:54 My name is Gabe.
  23. 1:55 I'm a software engineer here at OpenGov.
  24. 1:57 I work on the AI agents team, and I'm one of the folks that helped build OG Assist and some of the systems that you guys will be seeing today.
  25. 2:07 So a little bit about OpenGov.
  26. 2:09 OpenGov is a software company on a mission to power more effective and accountable government.
  27. 2:16 So OpenGov sells ERP software.
  28. 2:18 That's things like budgeting, procurement, asset management and permitting.
  29. 2:22 And we were founded about 14 years ago.
  30. 2:25 And what's cool is
  31. 2:28 we have this thing called OG Assist.
  32. 2:31 OG Assist is this little button on the top of all of our products in the navigation bar.
  33. 2:38 What's cool is all of our product suites and product teams,
  34. 2:44 have built tools and skills in order to power this button.
  35. 2:49 For example, if I click this button and I open up OG Assist, it says, hey, I'm going to ask about rate codes, which is very specific to Utility Billing, the current product that I'm in.
  36. 3:03 You can see that inside of this chat interface, I'm able to speak to an agent and the agent is able to make tool calls in order to
  37. 3:12 look up information against data inside of that suite.
  38. 3:15 So it's really cool to be able to kind of first party create these experiences through the capability that we've built called OG Assist.
  39. 3:27 okay so just a quick story about how this all came to be so um a little while back we we we saw that ai was really starting to take off and a principal i spun up this new team called the ai agents team and asked me to join and um instantly i said yes and ogs has started to to grow and we started to integrate
  40. 3:49 uh og assistant to all our products and not only our back-end capabilities but also our front-end capabilities as well so you'll see that one of the capabilities that we give the agent is it's able to um see what's on the screen and and see
  41. 4:05 and take action on what's on the page.
  42. 4:08 So you could see that I'm asking the agent here, hey, hey, what's the screen?
  43. 4:12 Can you maybe highlight some of the next steps that I could take?
  44. 4:16 So you can see that the agent here is thinking, it's saying, okay, what tools do I have available to use?
  45. 4:21 And hey, let me go and highlight something that you could actually click on and tell you more about it.
  46. 4:26 So just another capability of OG Assist and just a little short story about how this all came to be.
  47. 4:35 So the big bet on effect.
  48. 4:37 So I really wanted to include this slide because here on the agents team, we made a huge bet to bet on effect and suffice to say it's paid off in dividends.
  49. 4:52 We write effect so effect is this library for TypeScript.
  50. 4:57 It's open source and it helps you write better TypeScript code.

Open at this second