A practical guide to choosing the right AI tools for your law firm: Inside the framework
min readHow We Compared 5 AI Vendors in Just Two Weeks
Five vendors. Forty-six lawyers. Two weeks. Here’s how we made it work.
We invited five shortlisted AI vendors into a head-to-head challenge. Lawyers tested each product in real practice contexts.
How did we do this? We looked at how our lawyers had been using AI and grouped the most common tasks into five categories:
- Answering legal questions (like “what does this term mean?”)
- Summarising long documents or email threads
- Drafting text (clauses, statements, reports)
- Translating documents
- Reviewing contracts and creating reports
Each vendor gave us multiple 90-minute sessions, both in person and online. Lawyers tested the tools using real (non-confidential) examples from their day-to-day work. No fake scenarios, no gimmicks.
Everyone tested the same tasks across all five products so we could compare apples to apples. We collected structured feedback after every session, mirroring RFP questions to validate vendor claims. This helped us see where vendor promises matched reality, and where they didn’t.
The outcome? Comparable data across products, persona-based insights, and a clear winner - without months of disruption.
The full report can be found here: A practical guide to choosing the right AI tools for your firm.
👉 Next post: The five insights that surprised us most.
