Prompted Lines

Depth 2 of 6 · Working vocabulary · Everyone · ~10 min

The terms you'll hear, an honest account of strengths and weaknesses, and where the value is for insurers.

Terminology, one line each

Honest pros and cons

What LLMs are genuinely good at

• Reading and summarizing large volumes of documents quickly
• Drafting: they produce a strong first draft; humans produce the final
• Extracting structured data from unstructured text
• Explaining, translating, and reformatting information
• Writing and reviewing code

Where they fail

• They fabricate. Fluency is not accuracy. Error rates are low but never zero, and errors look identical to correct output.
• They don't reliably do arithmetic unaided (they can, however, write and run code that does).
• They are non-deterministic (the same question can get slightly different answers), which complicates reproducibility and audit.
• They reflect biases in training data, a direct concern under unfair-discrimination law.
• They can be manipulated ("prompt injection": malicious instructions hidden in documents the AI reads).

Where the value is for insurers

Highest-confidence use cases, roughly in order of value ÷ risk. Examples lean commercial P&C, but each maps directly to other lines (swap "broker submission" for "application" and "loss run" for "claims history"):

1. Submission intake and triage: extract named insured, operations, exposures, and requested coverages from broker submissions; flag out-of-appetite risks early. Directly attacks the "quote what you see first" bottleneck.
2. Document summarization for underwriting: condense loss runs, engineering reports, financials, and prior policies into underwriter-ready briefs.
3. Policy wording and contract comparison: compare endorsements, spot deviations from standard forms, first-pass review of manuscript wordings (with expert review).
4. Claims file summarization: synthesize adjuster notes, medical and legal correspondence, and coverage positions for faster reserving and severity triage.
5. Knowledge access (RAG): a searchable assistant over underwriting guidelines and appetite guides ("What's our appetite for habitational in coastal Florida?").
6. Actuarial and analytics acceleration: code assistance, data-quality checks, documentation drafting, filing prep. This is where actuaries typically start; see Level 4.
7. Internal drafting: broker communications, meeting summaries, first-draft reports.

Depth 3 of 6 · Hands-on skills · Everyone · ~15 min

Most disappointing AI output traces back to a prompt that didn't give the model enough to work with. This level is the practical skill: what to put in a prompt, ten techniques that do most of the work, copy-paste templates for common insurance tasks, and the habits that separate effective users from frustrated ones.

Anatomy of a good prompt

Five ingredients, most of which people skip:

Before: "Summarize this loss run."

After: "You are a commercial lines underwriter preparing for renewal. Summarize the attached 5-year loss run for a $2M GL account: total incurred by year, the top 3 claims with one-line descriptions, frequency and severity trends, and any open large losses. Format as a short brief I can paste into the underwriting file. Flag data gaps rather than guessing."

Same document, radically different output.

Ten techniques that do most of the work

1. Paste the source material. Never ask the model to recall a policy form, regulation, or account from memory; give it the document and ask it to work from that. This is the difference between grounded output and confident fiction.
2. Assign a role. "You are a skeptical reinsurance treaty reviewer" produces different (better) output than no framing.
3. Specify the output format. Table vs. prose vs. bullet brief; word limits; audience ("explain for a board member" vs. "for an actuary").
4. Show an example. If you want a specific style (a triage note, a file summary format), paste one good example and say "match this format." The highest-leverage trick almost nobody uses.
5. Ask for reasoning first on hard problems. "Think through the coverage question step by step before giving your conclusion." For genuinely hard analysis, use the tool's extended-thinking mode if it has one.
6. Iterate. The first output is a draft, not a verdict. "Shorter." "More formal." "You missed the 2023 claim; redo with that included." Follow-ups are cheap.
7. Use it as a critic, not just a drafter. Paste your own memo or analysis and ask for a skeptical peer review. One of the highest-value, lowest-risk uses that exists.
8. Break big tasks into steps. "First extract the claims, then we'll analyze trends" beats one giant do-everything prompt: you catch errors at each stage.
9. Tell it what to do when unsure. Add "If the information isn't in the document, say 'not stated'; do not guess." This one sentence dramatically cuts fabrication in extraction tasks.
10. Start fresh chats for new topics. Long conversations accumulate stale context and the model starts blending earlier instructions into new answers.

Copy-paste templates, insurance flavored

Examples use P&C documents; adapt the nouns to your line of business. The structure is what matters.

Summarize a claim file

You are a senior commercial claims adjuster. Below is a claim file. Summarize it as: (1) a 3-sentence overview, (2) coverage issues, (3) current reserves vs. realistic exposure, (4) open questions, (5) recommended next actions. Flag anything unusual. If something isn't in the file, say so rather than assuming.

[paste file]

Extract a loss run to a table

Extract every claim from the loss run below into a table with columns: claim number, loss date, line of business, cause of loss, status, paid, incurred. Use "not stated" for missing values; do not guess or infer. After the table, list any entries you were uncertain about and why.

[paste loss run]

Compare policy wordings

Compare the two endorsement wordings below. List every substantive difference in coverage grants, exclusions, conditions, and definitions in a table. Ignore formatting and ordering differences. Then explain which differences would matter most for [describe the risk] and why.

[paste wording A] / [paste wording B]

Draft a communication

Draft an email to a retail broker declining the [risk type] submission for [named insured] because [reasons]. Professional and warm, under 150 words, keep the door open for other business, and do not promise to reconsider this risk.

Red-team your own work

Below is my draft analysis. Act as a skeptical peer reviewer: list the strongest objections, the weakest assumptions, anything missing, and alternative interpretations of the data. Rank by importance. Do not compliment the work.

[paste analysis]

Understand or write code (actuaries/analysts)

Explain what this [R/Python/SQL/VBA] code does, section by section, then flag any bugs, silent assumptions, or edge cases that could produce wrong numbers.

[paste code]

Turn a meeting into actions

From the meeting notes below, produce: decisions made, action items (owner + due date if stated), and open questions. Don't invent owners or dates that weren't stated.

[paste notes]

Habits of effective users

• Verify anything that leaves your hands: numbers, quotes, citations, policy-language references. LLMs fabricate plausible-looking citations; check every one against the source.
• Numbers → code, not mental math. For anything quantitative, ask the model to write the calculation (Excel formula, Python, SQL) rather than compute it in prose. The code is checkable; the mental math often isn't right.
• Drafts, not decisions. Great for the first 80%; you own the last 20% and the signature.
• Re-ask important questions in a fresh chat. If the answer changes materially, the model is guessing; dig deeper before relying on it.
• Build a prompt library. When a prompt works well, save it. Teams that share proven prompts get consistent quality; individuals reinventing prompts get lottery results.
• Give it your standards. Paste your team's checklist, style guide, or a model example of "good"; the model will hold work to it.
• Use it to learn. "Explain ceded reinsurance accounting like I'm a new analyst, then quiz me" is a legitimately excellent use with essentially zero risk.
• Respect the data rules. Sanctioned tools only; no client or confidential data in personal accounts. No exceptions. See the policy template in Level 6.

Common mistakes and their fixes

Depth 4 of 6 · Technical · Actuaries & analysts · ~40 min

For readers who will build with these tools, not just chat with them. Assumes statistics and code comfort (GLMs, R/Python) but zero LLM background.

1. What an LLM actually is, for a statistical audience

An LLM is a very large neural network (a transformer) trained to predict the next token (~¾ of a word) given all preceding tokens, over trillions of words of text. Post-training steps (instruction tuning, reinforcement learning from feedback) shape this raw predictor into an assistant that follows instructions.

Three consequences follow directly from this design, and they explain almost every strength and weakness you'll observe:

1. It is a conditional distribution over text, not a knowledge base. Output is sampled, so it is stochastic: the same prompt can yield different answers. A temperature parameter controls sampling dispersion. Even at temperature 0, exact reproducibility across model versions is not guaranteed, a material point for actuarial documentation (§6).
2. Fluency and truth are different objectives. The model was optimized to produce plausible text. When it lacks knowledge, it produces plausible text anyway: a hallucination. Frequency varies by task: low when summarizing a document it's been given, higher when recalling specific facts (citations, table values, policy language) from memory.
3. Parameters encode compressed knowledge with a training cutoff. The model knows nothing after its cutoff and nothing about your company unless you put that information in the prompt. This is the single most important practical insight: an LLM is only as good as the context you give it.

2. Core concepts, precisely

• Context window: the maximum tokens processed per request (currently 200K–1M+, i.e., hundreds of pages). Everything the model "knows" about your problem lives here. Long-context recall is good but degrades subtly; don't assume perfect retrieval from page 400.
• System prompt: standing instructions that frame every request (role, constraints, output format). In applications, this is where you encode business rules.
• RAG: instead of hoping the model memorized your underwriting guidelines, you (a) chunk and index documents as embeddings (dense vectors where semantic similarity ≈ geometric proximity), (b) retrieve the most relevant chunks per query, and (c) paste them into the prompt with instructions to answer only from them, with citations. The workhorse architecture for grounded internal Q&A. Its failure modes include retrieval failures, not just generation failures; evaluate both stages.
• Fine-tuning: further gradient training on your examples. Good for style and format consistency at scale; generally the wrong tool for injecting knowledge (RAG wins there), and it raises governance burden. Most insurance teams need it rarely or never. Start with prompting + RAG.
• Structured output: modern APIs can force output to conform to a JSON schema. This is how you turn "read this loss run" into reliable columns: {claim_date, cause, incurred, paid, open_flag}. For extraction pipelines, always use schema-constrained output rather than parsing prose.
• Tool use / agents: the model can be given tools (code execution, search, database queries) and can loop: plan → act → observe → act. Crucial implication: never let the LLM do arithmetic in its head; let it write and execute code. An LLM asked to sum a loss triangle will often be slightly wrong; an LLM asked to write pandas code to sum it will almost always be right, and the code is auditable.
• Reasoning models: variants that generate extended internal chains of thought before answering. Materially better on multi-step quantitative problems; slower and costlier. Use them for hard analysis, standard models for bulk extraction and summarization.

3. Good at / bad at, calibrated for actuarial work

Strong today: use freely, with review

• Code generation and review (R, Python, SQL, VBA translation): for most actuaries the single highest-ROI use, including explaining inherited legacy code
• Summarizing and comparing documents: reinsurance treaties, policy forms, regulatory correspondence, prior analyses
• Extracting structured data from unstructured sources: loss runs in inconsistent broker formats, exposure schedules, contracts
• Drafting: report language, documentation, memo sections, filing narrative (ASOP 41 review still applies; see §6)
• Regulation orientation: "explain this state's rate filing requirements" as a starting point to verify, never a citation source

Weak or dangerous: control tightly

• Mental arithmetic and multi-step numeric manipulation without code execution
• Factual recall of specific numbers, citations, ASOP text, or policy language from memory; always retrieve the source document instead
• Anything requiring guaranteed reproducibility without an eval harness around it
• Judgment calls that are actually yours: assumption setting, method selection, reserve picks. The model can argue positions usefully (ask it to critique your assumptions, which is genuinely valuable) but it cannot own them

4. A hands-on learning path (4–6 weeks of spare-time effort)

Stage 1: Fluent prompting (week 1). Everything in Level 3, practiced daily on real work. Deliberately try to make the model hallucinate: ask for citations to actuarial papers, then check them. Calibrating your skepticism is the point.

Stage 2: API basics (weeks 2–3). Call a model from Python/R rather than a chat window: system prompts, temperature, structured JSON output against a schema. First project: extract a standardized claims table from three differently-formatted loss-run PDFs, with schema-constrained output, and reconcile totals against the source in code.

Stage 3: Evaluation discipline (weeks 3–4). Before trusting any pipeline, build an eval set: 30–100 examples with known correct answers, a scoring script, and a tracked accuracy metric. This is the LLM analogue of a holdout sample, and the skill that separates production work from demos (§5).

Stage 4: RAG and agents (weeks 5–6). Build a small RAG prototype over a document set you know well (e.g., your own past reports), so you can judge retrieval quality personally. Then experiment with tool use: an assistant that answers loss-development questions by writing and running code against a triangle you supply.

Stage 5: ongoing. SOA/CAS materials (see Resources), and pick one real workflow to instrument end-to-end with a colleague as reviewer.

5. Validation: treat LLM pipelines like models, because they are

For any LLM use beyond personal drafting assistance, apply model-risk thinking:

1. Define the task narrowly and the failure cost. "Summarize claim files" is not evaluable; "extract these 8 fields from loss runs with ≥98% field-level accuracy" is.
2. Build a labeled eval set from real data (with permissions). Measure before deploying, and re-measure on every model version change, prompt change, or vendor update. Model upgrades silently change behavior; pin model versions and re-run evals on upgrade.
3. Measure the right failure modes: field-level accuracy for extraction; faithfulness (does every claim in the summary trace to the source?) for summarization; retrieval hit rate and answer groundedness for RAG.
4. LLM-as-judge (using a second model to grade outputs) scales evaluation cheaply but inherits its own biases; calibrate it against human grading on a subsample first.
5. Keep humans in the loop proportionate to consequence, and monitor for automation bias: spot-audit accepted outputs, not just rejected ones.
6. Log everything: prompts, model version, outputs, reviewer decisions. This is your exam-readiness evidence under the NAIC bulletin regime (Level 6).

6. Professional standards: how this touches your credential

Nothing about the ASOPs is suspended because a model drafted the text or the code:

• ASOP 56 (Modeling): an LLM pipeline used for actuarial work is a model; you're responsible for understanding its intended purpose and limitations, and for appropriate validation and reliance. Reliance on an LLM is reliance; document it as such.
• ASOP 23 (Data Quality): LLM-extracted data is processed data; you must consider its quality. Your eval-set accuracy measurements (§5) are exactly the documentation this calls for.
• ASOP 41 (Actuarial Communications): you own every word in an actuarial communication regardless of who (or what) drafted it. Consider disclosure of material AI reliance in your documentation.
• Reproducibility: outputs are stochastic and vendors update models, so archive prompts, model versions, and outputs used in any work product supporting an actuarial opinion. Where feasible, have the LLM produce code (deterministic, re-runnable) rather than answers.

7. Data handling rules that are never optional

• Use only company-sanctioned tools and APIs (enterprise terms: no training on your data, defined retention). Never paste policyholder, claimant, or confidential company data into consumer AI tools.
• Prefer de-identified or minimal-necessary data in prompts even on sanctioned tools.
• Treat model outputs derived from confidential inputs as confidential.
• Documents from outside the company (broker submissions, claimant correspondence) can carry prompt-injection payloads: hidden instructions to the model. Pipelines that act on external documents need injection-aware design and constrained tool permissions.

Full rules are in the policy template at Level 6; these are the ones analysts hit daily.

Depth 5 of 6 · Agentic work · Everyone · ~15 min

Chat is a conversation; an agentic workflow is a delegated job. The model plans, calls tools (search, extraction, code, your systems), runs unattended, and produces artifacts you can check. This level covers the three shapes those jobs take, when to fan work out across parallel agents, and the supervision rules that keep the output honest.

Chat versus delegated work

In a chat, you are in the loop of every step: you ask, it answers, you steer. In an agentic workflow you delegate the whole job and review the output. That shift, from conversation to delegation, is what "agentic" means in practice. It changes both the payoff (work happens unattended, at any hour, at fleet scale) and the risk (errors compound across steps you did not watch). Managing that tradeoff is the skill of this level.

The three shapes of agentic work

Nearly every agentic workflow in production is one of these shapes, or a combination of them. Learn to name the shape before you build or buy anything.

Fan-out: when parallel agents pay, and when they don't

Fan-out is the highest-leverage agentic pattern and the most misused. Telling agents to work in parallel pays when:

• The units are independent. Forty submissions, ten state filings, three candidate wordings. If each unit stands alone, ten workers finish in the time of one, and a human reviews the merged queue instead of running forty chats.
• The queue is the bottleneck. CAT-event claim notices, renewal-season bordereaux: throughput is the win, and identical workers beat a heroic individual.
• Redundancy buys quality. Ask for three independent answers to a hard question, then a fourth pass that compares them and explains the disagreements. Cheap, and measurably better than one confident pass.

It fails when:

• Steps are coupled. If step two changes what step three should do, parallel workers diverge and the merge is mush.
• Workers share mutable state. Several agents editing one document or one shared context produce collisions, not collaboration.
• You wanted personas. Parallel agents pay because each gets isolated context, not because one of them plays "the skeptic." Role-playing subagents measurably underperform; quarantined context is the entire point (see Cognition and Anthropic's multi-agent research write-up).

Cost discipline: fan-out multiplies token spend roughly linearly, and orchestrated teams have been reported at around 15x the token cost of a single agent. Spend parallelism where volume or quality justifies it, and measure the merge quality, not just the speed.

Supervision rules that keep fan-out honest

1. One context spec for every worker. Same guidelines, same exemplars, same constraints. Divergence in output should reflect the work, not the briefing.
2. Limits in code, not prompts. Authority caps, PII blocks, and field validators enforced by hooks. Instructions inform; hooks enforce.
3. Verify at the merge. Reconcile counts, check that citations resolve, and route worker disagreements to a human instead of silently averaging them.
4. Human gate on consequential outputs. Quotes, filings, and customer-facing messages get a named reviewer. The NAIC bulletin expects documented human oversight exactly here (Level 6).
5. Log everything. Inputs, outputs, hook firings, overrides. Fan-out without logs is unaccountable slop at scale.

Where to go from here

The practice ladder carries both next steps: levels 5 and 6 are the personal-practice version (agent boss, fleet orchestrator), and its harness section is the build manual (context specs, hooks, golden datasets, eval gates). The AI roadmap's Phase 3 is the organizational version. Level 6 of this guide covers the governance expectations that apply to all of it.

Depth 6 of 6 · Governance · Leaders & policy owners · ~20 min

The regulatory landscape, the questions leadership should be asking, a pragmatic 90-day posture, and an adaptable responsible-use policy template.

The regulatory and risk landscape

• NAIC Model Bulletin on the Use of AI Systems by Insurers (Dec 2023): adopted by 24+ states plus D.C. as of early 2026 (adoption map). It expects every insurer using AI to maintain a written AI governance program covering inventory, risk assessment, human oversight, testing, and vendor-AI diligence. Regulators can request this documentation during market conduct exams.
• NAIC AI Systems Evaluation Tool: a standardized examination framework being piloted by twelve states January–September 2026 (pilot summary). Translation: AI governance is moving from "expectation" to "exam item."
• Existing law already applies. Unfair trade practices, unfair discrimination, and rate regulation apply to AI-assisted decisions exactly as to human ones. "The model did it" is not a defense.
• Colorado remains the bellwether for algorithm and data-governance requirements (the SB 21-169 regime, currently life-focused but widely expected to expand by line).
• NIST AI Risk Management Framework (plus its Generative AI Profile) is the de facto scaffolding for a defensible governance program, and the one the NAIC bulletin echoes.
• EU AI Act: relevant only with EU business or operations; high-risk obligations phase in through August 2026.

Top risks to manage, in order

1. Confidentiality: employees pasting policyholder or company data into consumer AI tools. Solved with enterprise agreements plus policy; fix this first.
2. Accuracy / hallucination in outputs that reach customers, regulators, or decisions.
3. Unfair discrimination / bias if AI touches risk selection or pricing.
4. Vendor risk: AI is arriving embedded in software insurers already buy; know where.
5. Over-reliance: staff accepting AI output without review ("automation bias").

Questions leaders should be asking

Use these in management meetings; they map to what examiners will ask:

1. Do we have a written AI governance policy and an inventory of where AI (including vendor-embedded AI) is used today?
2. Do employees have a sanctioned, enterprise-grade AI tool, and a clear rule about consumer tools?
3. For each use case: what is the human review step, and who is accountable for the output?
4. How would we answer a market conduct exam question about AI in underwriting or claims, today?
5. Which vendors have added AI features to products we already license, and what data do they see?
6. What is our measurement plan: are we tracking time saved, error rates, and adoption, or just launching pilots?
7. Who owns AI governance? (Common answer: a small cross-functional group of data science, legal/compliance, IT security, and a business sponsor.)

A pragmatic 90-day posture

1. Weeks 1–4: adopt an interim acceptable-use policy (template below); procure enterprise AI access with zero-data-retention / no-training terms; brief all staff.
2. Weeks 4–8: stand up the governance group; inventory current AI use including vendor tools; select 2–3 pilot use cases with named owners and success metrics.
3. Weeks 8–13: run pilots with human-in-the-loop review; measure; report results and a scale/kill decision to the executive team.

The goal is governed momentum: moving fast enough to learn, with guardrails proportionate to risk. The two failure modes are symmetric: banning AI (staff will use personal accounts invisibly, so-called "shadow AI") and ungoverned enthusiasm (which regulators are now actively examining for).

For the multi-year strategic view (where the industry stands, what competitors have deployed, and a phased 36-month progression with governance gates and economics) see the companion AI Progression Roadmap.

---

Responsible-use policy template

1. Scope

These guidelines apply to all employees and contractors using: general-purpose AI assistants; AI features embedded in vendor software (including underwriting, claims, and productivity platforms); and internally built AI/LLM applications. Traditional predictive models remain governed by existing model-governance policy; where an AI system feeds a regulated decision, both policies apply.

2. Sanctioned tools: the bright line

1. Use only company-approved AI tools [list; e.g., enterprise instances under company agreements with no-training and retention terms].
2. Never enter company, policyholder, claimant, broker, or employee information into personal or consumer AI accounts. This includes "just this once," and it includes screenshots.
3. Requests for new tools or AI-enabled vendor features go to [AI governance group] before use.

3. Data rules

Outputs derived from confidential inputs inherit the input's classification.

4. Human accountability

1. You own what you ship. AI output that you send, file, or act on is your work product. Review it as you would a junior colleague's draft.
2. Consequential decisions require human review. No AI output may, without documented human review, determine or effectively determine: risk selection or declination, pricing or rating, claim acceptance/denial or reserve values, coverage interpretations communicated externally, or personnel decisions.
3. Verify facts, numbers, and citations. Any figure, quotation, legal or regulatory citation, or policy-language reference must be checked against the source before use.
4. Disclosure: [company position; recommended minimum: disclose AI assistance within work products supporting actuarial opinions and regulatory filings; customer-facing disclosure per applicable state law].

5. Use-case risk tiers

6. Governance structure

• AI Governance Group: [named members; recommended: data science lead (chair), Legal/Compliance, IT Security, business-unit sponsor]. Owns this policy, the AI inventory, tool approvals, and tier classification.
• AI inventory: a living register of every AI system in use (internal, vendor-embedded, and experimental) with owner, tier, data touched, and evaluation status. This inventory is the first thing a market-conduct examiner will ask for.
• Vendor AI diligence: procurement and renewals must ask: Does this product use AI? On what data? Can it be disabled? What are the provider's training and retention terms?
• Incident handling: suspected AI-caused errors reaching customers, regulators, or financials are reported to [channel] within [24 hours]; treat like any other E&O-relevant incident.
• Records: for Medium/High-tier systems, retain prompts, configurations, model versions, evaluation results, and review decisions per [retention schedule].

7. Security notes

• AI systems that read external documents (submissions, emails, claims correspondence) are exposed to prompt injection: adversarial instructions embedded in those documents. Such systems must be designed with least-privilege tool access and no unreviewed external actions; Medium tier minimum.
• Report suspected AI-related phishing or deepfake contact (voice or video impersonation of executives, brokers, or claimants is now a standard fraud vector) to IT Security immediately.

8. Training requirement

All staff complete [AI awareness briefing] before tool access; Medium/High-tier system owners complete [role-specific training]. Re-certification [annually]. Review cycle for this document: [quarterly] by the AI Governance Group.

Reference · Curated external resources · Verified August 2026

Annotated and prioritized. Items marked Start here are the recommended first read for that audience. Regulatory items should be re-verified quarterly.

For senior managers

For actuaries: structured learning

For actuaries: hands-on reference

Standing intelligence: assign an owner, check quarterly