A demand is due Friday. Six providers sent records, one of them a 300-page hospital chart, and your paralegal has a new AI tool that turns the stack into a summary in four minutes. She pastes the chart into the browser window, reads the output, and it looks clean. Nothing about that moment feels like a legal question.
It is one. The upload may have left your control, the summary may have dropped a visit, and the signature at the bottom of the demand is still yours. Ask what is AI law in a personal injury practice and the honest answer is narrower and more useful than the headlines suggest: it's the set of rules that decides how much of that risk lands on the firm.
Most of the exposure doesn't arrive as a dramatic hallucination. It arrives as an ordinary confidentiality habit or a review nobody can prove happened. The duties were yours before the tool existed, and the tool didn't take them.
What AI Law Is and the Layers That Govern It
A building code doesn't care what a wall is made of. It cares what the building is used for. A hospital and a storage shed can rise from the same lumber, and only one of them needs fire separations, wider egress, and sprinkler coverage. AI law works the same way. The rules follow the purpose of the system and the people it touches, not the model's brand name.

AI law is the body of legislation, regulation, and legal doctrine that governs how artificial-intelligence systems are designed, deployed, and used. That definition is tidy. The practice isn't. No single federal AI statute governs a personal injury firm in the United States, and no single reading session gets you compliant. What you have instead is four layers of rules running at once, each with a different author and a different enforcement mechanism.
The Four Layers That Reach a Personal Injury File
The outermost layer is binding AI legislation. The clearest example is the European Union's AI Act, Regulation (EU) 2024/1689, which assigns duties by risk category rather than by technology type. A firm with no European clients, data, or vendors can often treat it as background noise. A firm that markets to clients with EU residence, or buys a tool from a vendor with an EU deployment, cannot.
The second layer is sector law you already follow. The HIPAA Privacy and Security Rules govern protected health information, and most of what sits in a personal injury file is exactly that. Nothing about AI suspends those rules or shifts the obligation to your software vendor.
The third layer is professional and court rules. State bar guidance on competence and confidentiality applies to AI-assisted work the same way it applies to a scanner. Individual judges add standing orders requiring disclosure or verification of AI use, and those orders differ from courtroom to courtroom.
The fourth layer is voluntary standards. The NIST AI Risk Management Framework binds nobody, but it supplies the vocabulary that insurers, clients, and reviewing courts use to ask whether you tested a system before you relied on it.
Governments aren't waiting for a uniform model. The Stanford 2025 AI Index report counted 204 AI-related laws passed by countries in total, with the number passed in a single year rising from 30 in 2023 to 40 in 2024. The direction is one-way: from principles to enforceable duties that follow a system through purchase, deployment, monitoring, and retirement.
Six frameworks come up most often in conversations with personal injury firms. Read the table against your own facts rather than as a checklist to finish.
| Framework | Scope | Binding or voluntary | Core focus | Relevance to PI firms |
|---|---|---|---|---|
| EU AI Act (Regulation (EU) 2024/1689) | European Union | Binding | Risk tiers and lifecycle duties | Applies when EU clients, data, or vendors fall in scope |
| NIST AI Risk Management Framework | United States, sector-neutral | Voluntary | Govern, Map, Measure, Manage | Practical baseline for documenting AI use and vendor review |
| HIPAA Privacy and Security Rules | United States, federal | Binding | Protected health information safeguards | Governs medical records inside AI tools; business associate agreements required with vendors |
| Colorado AI Act | Colorado | Binding | High-risk AI in consequential decisions, including employment | Matters for employment-related and claims-adjacent tools; confirm the current effective date |
| California proposed AI practice rules | California | Pending, reported for 2027 | Limits on delegating legal work, accuracy checks, disclosure in filings | Monitor; would change how AI appears in filings |
| State bar guidance and court standing orders | State bars and individual courts | Binding in many courts | Verification and disclosure of AI use | Check local rules before filing AI-assisted documents |
Read the third column first. It answers the question lawyers have, which is whether a framework can reach them.
Binding Rules and Standards That Only Look Optional
The voluntary row deserves more weight than its label suggests. When a malpractice carrier, an opposing expert, or a client's general counsel asks how you validated a summarization tool, the NIST structure is the shape of answer they expect: named owners, documented context, test results, and an incident process. A firm that can produce those four things looks careful. A firm that says the vendor handles it looks like it never asked.
Practical rule: Before you sign a vendor contract, write three facts on one page. Where your clients live, where the records are stored, and where the vendor is incorporated. Those three answers decide which rows above apply to you.
Where the EU Act does reach, the stakes are large. The European Commission's enforcement guidance sets penalties for prohibited AI practices at up to €35 million or 7% of an organization's total worldwide annual turnover, whichever is higher. Lower ceilings apply to other breaches. For a US firm the point is jurisdictional rather than arithmetic: the Act can follow a system's output into Europe even when the provider sits in Ohio.
State legislatures move fastest where AI touches employment and consumer decisions, which is why hiring tools and claims-scoring products attract rules a records summarizer never does. Court rules travel on a separate track. Judges have issued AI disclosure orders with different triggers, different wording, and different consequences for a filing that omits the disclosure, so two firms in the same city can face different obligations on the same document.
EU AI Act Timeline and the Four Risk Tiers
A personal injury firm uses a tool to summarize medical records and build an internal chronology. If the same tool is later used to rank claims for settlement decisions, its purpose and risk profile may change. The EU AI Act helps explain why: it groups AI uses by risk, rather than treating every tool as if it creates the same obligations.
The European Commission describes Regulation (EU) 2024/1689 as a legal framework for AI. It entered into force on 1 August 2024 (European Commission AI Act framework).

Its risk-based structure offers a useful way for firms to assess their own AI use, even when the Act does not apply to them.
Four Tiers and One Principle
The Act distinguishes unacceptable risk, high risk, transparency or limited risk, and minimal or no risk. Unacceptable-risk practices are prohibited. Examples include certain forms of harmful manipulation, exploitation of vulnerable people, government social scoring, criminal-risk prediction, facial-recognition scraping, biometric categorization, and real-time remote biometric identification.
A system that decides who receives a job or credit is an example of high-risk AI, subject to requirements such as documentation, human oversight, and data-quality controls. A customer-facing chatbot falls under transparency requirements, including telling people they are interacting with a machine. A spam filter is a lower-risk example.
The principle applies beyond Europe: duties depend on purpose, foreseeable impact, the people affected, and the level of risk, not merely on a tool's marketing label. For a personal injury firm, an internal record summary and a system that influences settlement decisions are different uses, even if they rely on the same technology.
The Act's obligations take effect in stages:
| Date | What begins to apply | What it means for PI firms |
|---|---|---|
| 1 August 2024 | Act enters into force | Starting point for the compliance timeline |
| February 2025 | Prohibited practices and AI-literacy requirements | Staff need to understand the AI tools they use, and prohibited uses must stop |
| August 2025 | Governance and obligations for general-purpose AI models | Relevant when a firm relies on large foundation-model vendors |
| August 2026 | Most remaining provisions, including transparency duties | In effect; disclose AI interaction where the rules require it |
| High-risk system deadlines vary | Certain high-risk system obligations | Requirements depend on the system and its classification |
| Regulated-product deadlines vary | AI embedded in regulated products | Requirements depend on the product and applicable rules |
Classification is a fact question, not a product question. A records-summary tool used to prepare an internal chronology may present a different risk from a system used to score claims for settlement authority or screen applicants for a paralegal role. A change in purpose can change the duties that apply, so firms should assess the actual use with counsel.
For day-to-day governance, the NIST AI Risk Management Framework organizes controls into four functions: Govern, Map, Measure, and Manage. For a medical-record summary, that can mean checking whether each statement traces back to a source, testing for omissions or fabricated facts, limiting access to sensitive records, and recording who reviewed and approved the work product.
EU transparency requirements apply where the relevant rules reach a firm or its AI use. Check the current requirements for the specific system and context rather than assuming that one disclosure rule covers every interaction.
Privacy, HIPAA and Privilege for Medical Records
Medical records are the most sensitive documents a personal injury firm holds, and they arrive with conditions attached. Protected health information is individually identifiable health data handled by covered entities and their business associates. A firm receives it through client authorization, a provider's release, or discovery, and whether the firm is a covered entity, a business associate, or neither depends on how the records arrived. Confirm that status with counsel rather than assuming.
Where the Records Come From
The arrival route sets the rules. Records produced under a signed authorization carry the scope of that authorization. Records produced in discovery carry protective order terms that may restrict who can view them and where they can be stored. Records a client uploads through a portal carry whatever the client agreed to there, which is rarely the same thing. An AI tool sits downstream of all three and inherits the strictest condition in the file.
The risk that catches firms first isn't a wrong diagnosis in a summary. It's the upload itself. Information entered into open or public AI tools may enter the public domain and compromise confidentiality or legal privilege, and courts have found that AI-generated documents were not necessarily privileged (DLA Piper on AI and legal privilege).
Say a case manager drops a full chart and a defense expert's report into a free summarizer on a Friday to see whether it saves time. By Monday the file has a retention question, a possible reportable event, and a privilege argument the other side can raise in a motion. None of it required the tool to get a fact wrong.
Seven Controls That Hold Up
- Retention and training. Get written confirmation on whether prompts and uploads are retained and whether they're used for model training. A vendor that won't answer in writing is a vendor you don't upload into.
- Vendor access. Limit who at the vendor can reach client records, and ask for the subcontractor list rather than waiting to be told.
- Business associate agreements. Execute one with any vendor handling protected health information, including subcontractor terms. Ares executes Business Associate Agreements with customers who are covered entities or business associates, and maintains them with every subcontractor and service provider that may access PHI.
- Data-processing terms. Document them, and file the signed version where you can find it when a carrier or client asks.
- Role-based access. A paralegal working one case shouldn't open another's records, and the tool should enforce that rather than trusting the honor system.
- Deletion and audit. Know how to have records deleted, and keep an audit trail of who opened what.
- Separate test data. Run de-identified records through a new tool before you run a live file through it.
A vendor's confidentiality clause protects the vendor's interests. Your duty of confidentiality runs to the client, and it doesn't transfer when the records leave your server.
If you're mapping these controls onto your own office, this walkthrough of HIPAA compliance for law firms covers the record-handling side in more detail. Firms that hand record organization to a law firm virtual assistant need the same controls written into that relationship, because the assistant's access is your access.
Settle one question before an incident rather than during it: who decides whether an event is reportable, and how fast. Name that person, name the backup, and put the escalation path in the policy instead of in someone's memory.
Liability, Evidence and Who Answers for AI Errors
An AI summary of a two-year treatment history leaves out a single urgent-care visit from March 2024. The demand goes out showing what looks like a long gap in care. The adjuster finds the visit in the provider records, notices the omission, and discounts the demand. Nothing dramatic happened. A line was missing.

The rule that decides the outcome here isn't a new AI statute. It's professional responsibility applied to AI-assisted work, and it has been settling in one direction for several years.
The Layers of Liability
The software provider may owe you something under its contract if a tool drops a line. Read the terms anyway, because most of them disclaim that outputs are legal advice and confirm that the firm retains responsibility for review. A contract claim against a vendor doesn't answer a malpractice claim from a client.
The firm carries the malpractice exposure, and the supervising attorney owes the duties of competence and supervision. Neither shrinks because software produced the first draft. The decision to file, to demand a number, or to settle stays with the attorney, and no vendor term moves it.
The recurring errors look like this:
- Missed treatment dates that create apparent gaps in care
- Omitted providers or facilities, especially ones outside the main hospital system
- Causation narratives stitched together from partial records
- Damages figures that don't trace to a billing line or a diagnosis
What Counts as Evidence
The medical records are the evidence. A summary is a working tool, and the moment it's attached as an exhibit or read into testimony, opposing counsel can attack its reliability and demand the source pages behind it. Keep the trail from summary line back to the page that supports it and you can answer in an afternoon. Lose it and you're rebuilding a chronology under a deadline.
Regulators and courts are tightening the verification expectation rather than relaxing it. California legislation reported for 2027 would bar lawyers from delegating legal practice to generative AI, require reasonable accuracy checks, restrict confidential inputs, and require disclosure of AI use in court filings, and federal judicial guidance places responsibility for verification on the attorney rather than the technology vendor (Legal 500 on state bar scrutiny of AI misuse).
Documentation separates review from a claim of review. A record of who checked what, and when, is the difference between an assertion and evidence. Ares doesn't provide legal advice, and the customer is solely responsible for all legal decisions, client communications, and filings made using service outputs. That division matches the rules: the tool organizes records, the lawyer answers for the filing.
The uncomfortable part: human-in-the-loop is not a defense by itself. Review has to be proportionate to the risk in the file and demonstrable afterward, through source-linked checks, version history, and a written path for escalating records that don't add up.
Intellectual Property, Bias and Employment Exposure
Ownership of AI-generated drafts is unsettled. Vendor terms vary on whether you own the output, whether the vendor may reuse it, and whether outputs might reproduce third-party text. Read them before you put a copyrighted expert report or another firm's demand into a tool without a license. The practical worry for a firm is narrower than a law review article suggests: a draft you can't cleanly claim as your own work product is a draft with an asterisk.
Bias Shows Up in Triage, Not in Summaries
A summarizer produces a chronology. A tool that triages claims, ranks cases, or estimates settlement value makes a judgment, and judgments trained on skewed historical outcomes can treat groups differently. NIST treats harmful bias as a fairness property to be managed rather than a defect to be denied, which means testing and documenting rather than asserting.
- Sample closed files. Pull cases across claimant demographics and compare the tool's valuation notes against the attorneys' final figures.
- Log the outliers. Write down every case where the model's number landed far from the settled number, and ask why.
- Test the edges. Check cases with limited English proficiency, no health insurance, or long treatment gaps, because that's where a model's assumptions show.
Employment Tools You May Already Own
AI that screens paralegal applicants, schedules staff, or monitors productivity falls under employment rules, not legal-services rules. Colorado's AI Act places duties on employers using high-risk AI in consequential employment decisions, including risk management and notice to affected individuals, and other states and the EU are moving in the same direction. Pull a list of anything touching hiring, scheduling, or performance review, and read the notices you've given staff and applicants.
Most firms find three or four employment-adjacent tools they never classified as AI. A scheduling add-on that ranks applicants by availability is one. A writing assistant that flags "tone" in internal reviews is another.
Firm Compliance Steps From Inventory to Sign-Off
Seven steps take a firm from scattered tool use to something defensible. None needs a budget line, and each exists because of a specific failure it prevents.

The Seven Steps
- Inventory every AI tool. Name the tool, its purpose, the data types it touches, and whether protected health information can land in it. Include the browser extensions and the free summarizer someone found useful in March. You can't classify what you haven't listed.
- Classify each use by risk and sensitivity. Run it through the EU tiers and your HIPAA exposure at the same time. A deposition transcription tool and a settlement-value estimator don't belong in the same bucket, even if one vendor sells both.
- Ask every vendor the same six questions. Retention period, training use, business associate agreement, subcontractor list, breach notice timeline, deletion on request. Same questions, same format, written answers, so the comparison means something. Our answers are on the record: Ares maintains zero-data-retention agreements with all AI infrastructure providers, so customer data isn't stored after the processing request completes, and we don't use customer content to train AI models.
- Name the attorney who signs off. One name per AI-assisted demand or filing, written on the file. Responsibility shared across a team becomes responsibility nobody holds.
- Set disclosure rules by court. Collect the standing orders for every judge your firm appears before, and address AI use in the engagement letter when it's likely on the matter.
- Test on a schedule. Sample a fixed number of summaries each month against source records, log every omission and fabrication, and walk the log with the attorneys who rely on those summaries.
- Keep the record. Version history, reviewer identity, and source links for each final work product. Six months from now, that record is what answers a challenge.
Those steps map onto NIST's four functions without much translation. Steps one and four assign owners, which is the governance work. Step two records the intended context, step six tests accuracy, and step seven plus your incident process handle what goes wrong. The vocabulary earns its keep because vendors, insurers, and opposing counsel already use it.
The scaling rule: a solo practice meets the minimum with steps one, three, four, and seven, plus a monthly sample of summaries against the records. Skip the sampling under deadline pressure and you're relying on the vendor's quality claims rather than your own evidence.
A two-attorney office can finish the inventory and the vendor questions in a week if one person owns the task and the calendar holds the time. Assign it to a named person with a date, not to "the team."
Adopt a Written AI Use Policy Before Your Next Case
A medical-record summary can save review time, but the policy should make clear who checks it before an attorney relies on it. Written rules turn that expectation into a consistent process for everyone handling client files.
In Omdia's 2024 survey of 478 enterprises, fewer than 50% reported being compliant with existing AI regulations or actively working toward compliance (Omdia on AI governance and compliance).
What Goes Into the Policy
- Purpose and scope. Specify which offices, roles, and systems the policy covers.
- Approved tools list. Name permitted tools and identify any versions staff may not use for client work.
- Protected health information. State which tools are approved for PHI and prohibit entering it into unapproved tools, including for quick tests.
- Review standard. Require a named attorney to check AI-assisted demands and filings against their source materials before they leave the office.
- Disclosure. Explain how staff check applicable court requirements and where they can find them.
- Vendor requirements. Set expectations for business associate agreements, use of client data, subcontractors, and breach notification.
- Training and AI literacy. Describe what staff must complete before using AI on client-facing work.
- Incident reporting. Name the person to contact, the reporting timeframe, and the records to preserve.
- Retention and deletion. Set out how long outputs are kept and how they are deleted.
Where to Source the Rules
Build the policy from primary materials: the EU AI Act, the NIST AI Risk Management Framework, HHS guidance on the HIPAA Privacy Rule, state bar guidance, and standing orders from the courts where the firm practices. Use those sources to set rules that fit the firm's cases and obligations. A policy template can provide a starting structure, but the firm's own requirements must determine what it says. For a reference point on policy structure, see this AI policy template.
Ares provides an AI-powered litigation support platform with tools for medical record summarization and demand drafting. A firm evaluating any platform should compare its capabilities and data practices with the requirements in its policy. Ares does not use customer content to train AI models, maintains zero-data-retention agreements with AI infrastructure providers, and executes Business Associate Agreements with customers who are Covered Entities or Business Associates.
Adopt the policy, circulate a one-page version to everyone who handles client files, and schedule regular reviews so the written rules stay aligned with the firm's practices. The policy sets the process; the lawyer remains responsible for legal decisions, client communications, and filings made using AI outputs.



