Bench IQ
Judicial intelligence platform that characterises how individual federal judges reason, built to reach the rulings conventional research cannot. The premise is a gap in the public record: judges issue written opinions for only around three per cent of their rulings, so a lawyer preparing a motion or an oral argument has little to work from on the other ninety-seven. Bench IQ has assembled a proprietary dataset covering those unwritten decisions, including rulings delivered orally from the bench, and applies large language models and AI agents to identify the patterns in a judge's reasoning. The output is explanatory rather than statistical. Where litigation analytics report how often a judge grants a given motion, Bench IQ sets out when and why a particular judge reached a particular conclusion, so counsel can shape an argument to the person deciding it; the vendor illustrates this with a bankruptcy example, pinpointing when and why a presiding judge has approved above-market deal protections. Its founders draw the distinction sharply, saying the aim is not to predict the outcome of a case but to help lawyers shape it. Coverage is the United States federal judiciary, opening in commercial bankruptcy, with state courts and other common law jurisdictions named as the expansion path. The buyer is a litigator at a large firm, and the vendor says four of the top five AmLaw firms use it. The method behind the dataset is not published, which the company attributes to pending patents. Bench IQ, Inc. is independent and based in Toronto, founded in 2023 by the two co-founders of ROSS Intelligence and a former Kirkland & Ellis partner, and has raised 7.4 million dollars across two rounds, with the law firms Cooley, Fenwick & West and Wilson Sonsini among its investors.
Capability grades
All 15 axes, graded from public sources on the date shown. Hover a grade to see what the letter means on that axis.
AI Centrality
How much of the product is actually AI. Whether the machine learning is the mechanism the buyer is paying for or a feature layered onto conventional software, and whether the vendor is specific about which is which.
The models are not layered on a product, they constitute it, and they also constitute the corpus. What Bench IQ sells is a characterisation of how a named federal judge reasons, derived by large language models and AI agents from rulings that were never issued in writing. Remove the models and nothing remains that could be sold: there is no repository, no workflow, no document store, no search index over published opinions, and no dataset either, because the dataset is itself the product of machine extraction from oral rulings and other unwritten indicators rather than a collection that existed before the models were pointed at it. The privacy policy names large language models as the mechanism for analysing rulings, and the vendor describes fine-tuning commercial models on the dataset it built. This is the second A on this axis in the pull, alongside Pre/Dicta, and on the constitutive test it is the stronger of the two: Pre/Dicta applies a model to federal dockets that exist independently of it, while here the underlying record is assembled by the model in the first place.
Citation Accuracy and Hallucination Disclosure
Whether the vendor publishes measured accuracy on citations and assertions, grounds output to primary sources, and says plainly what its system does when it does not know. Legal has a documented public record of fabricated citations reaching filed briefs, so an untested claim of accuracy is not evidence.
Nothing is published on accuracy or grounding for a product that produces assertions about what a named judge thinks. The site is five pages and the home page is the product page: three FAQ blocks, a social proof claim and a demo request. Checked those, the about, press, privacy policy and responsible disclosure pages on 31 Aug 2026, and there is no accuracy figure, no test set, no evaluation, no error rate, no statement of failure modes, and no hallucination disclosure of any kind. The verification question is sharper here than on any other record in this lane, and it follows from the product's own premise. Bench IQ's value proposition is that it reaches the roughly ninety-seven per cent of rulings that produce no written opinion, which means the source material behind an insight is by definition material a reader cannot pull up in a conventional research tool and check. Nothing published states whether an insight links back to the ruling it came from, whether a user can inspect that source, or what the system does when the record is thin. Two limbs of this axis do not apply rather than being failed: a citator or good-law check is not in scope for a product that characterises judicial reasoning rather than validating authority, and it is neither credited nor penalised.
Autonomy and Oversight Model
What the system decides on its own, what a lawyer must approve, and whether the vendor documents where the review point sits. A tool that drafts under review and a tool that files without one are different products and different risks.
No oversight structure is published for a system whose output feeds live litigation strategy. Searched the home page, about page, press page, terms of service, privacy policy and responsible disclosure page on 31 Aug 2026: nothing describes what the models decide unattended, no review surface is documented, no confidence or abstention behaviour is described, no statement tells a lawyer what weight an insight should carry, and no route exists to report or correct an insight that proves wrong. The vendor is publicly careful about the framing, its chief executive saying the aim is not to predict the outcome of a case but to help lawyers shape it, which locates judgement with the lawyer. That distinction is real and is recorded here, but it was made in an interview rather than in product or policy material, and a framing is not a control.
Operational and Outcome Evidence
Named, dated evidence that the product works in production at real firms or legal departments. Case studies with figures and identified customers count. Unattributed testimonials and launch announcements do not.
Adoption counts and anonymous testimonials stand in for evidence. The vendor publishes two figures, both self-reported and neither naming anyone: four of the top five AmLaw firms are said to use the platform, and an earlier announcement put twelve of the hundred largest US firms as pilot customers actively using the product on live cases. Three testimonials appear on the home page attributed only by seniority and firm rank. No customer is named anywhere, no case study exists, and no figure is published for what changed at any organisation. One conflation to guard against, because the same names circulate in both roles: Cooley, Fenwick & West, Wilson Sonsini and several Kirkland & Ellis partners are investors in Bench IQ, not disclosed customers. Both bands have some purchase here, since an adoption count is closer to real deployment evidence than a logo strip, and the lower is taken because the count is unverifiable, unattributed and silent on outcome. Checked the home, about and press pages on 31 Aug 2026.
Privilege and Confidentiality Posture
How client confidences are handled: attorney client privilege and work product treatment, segregation of one client matter from another, whether client data trains any model, and what the vendor commits to in writing rather than in marketing.
The published policy is a website and application privacy notice rather than a customer data commitment, and the distinction governs this grade. Effective 20 February 2025, it covers visitors to the site and users of the application, and what it addresses it addresses well: personal data limited to name, job title, employer, work email and work phone, with the vendor noting that this information is already public through firm websites, state regulators and PACER; no sale or rental of personal information; thirteen subprocessors named outright; storage and processing in the United States only; and a named security officer, the co-founder and chief technology officer. Several limbs of this axis do not apply to this product rather than being failed by it, and are neither credited nor penalised: the platform ingests judicial records rather than client documents, so there is no client matter material to segregate, no ethical wall to inherit and no privilege or work product question about ingested content. What does bite is unanswered. A litigator researching a named judge on a named issue discloses the matter they are working on, and nothing published states whether that query history is retained, analysed, used to improve the models or disclosed, and no customer-facing confidentiality commitment of any kind was located on 31 Aug 2026.
UPL and Professional Responsibility Posture
Whether the vendor is clear that it supplies a tool rather than legal advice, who its audience is, and how it addresses unauthorized practice of law, competence and supervision duties, and jurisdiction limits. ABA Formal Opinion 512 is the reference point.
Nothing published on the advice line, and no page exists that would ordinarily carry it. The site inventory was established directly rather than assumed: navigation of login, about us, careers and press, with privacy policy, responsible disclosure, security and terms of service in the footer. There is no disclaimer page and no ethics or professional responsibility page, and none of the pages read on 31 Aug 2026 states that an insight is not legal advice, that no attorney-client relationship arises, or that a lawyer remains responsible for the argument they build on it. No bar or ethics guidance is named, including ABA Formal Opinion 512, and no jurisdiction or practice limit is stated. The exposure is not incidental: the product exists to shape how counsel argues to a specific judge in a live matter. One limit to state, because it is the rebuttal route: the terms of service could be read only in fragments recovered through the search index, so a professional advice disclaimer could sit in the unread portion.
AI Governance and Bias Disclosure
Published governance over model behaviour: who owns it inside the vendor, what is tested before release, and what is disclosed about disparate output across matter types, parties, or populations.
No governance position is published for a system that models the reasoning of named federal judges. Searched the home, about, press, privacy policy, responsible disclosure and terms pages on 31 Aug 2026: no responsible AI statement, no accountable owner for model behaviour, no pre-release testing regime, no fairness or uneven-output evaluation, and nothing addressing what it means to characterise how identifiable individuals on the federal bench think. The method is expressly withheld, which the vendor attributes to pending patents, so a buyer cannot examine either the governance or the technique. This is the third vendor in this category with the same gap and each takes a different form, which is what makes it a pattern rather than three coincidences: Trellis displays judges' political affiliation as a data field, Pre/Dicta uses political affiliation and net worth as model inputs, and Bench IQ reconstructs judicial reasoning by large language model from material that was never published and cannot be independently checked. That last is the furthest from a verifiable public record and carries the least published governance of the three.
AI Safety and Data Stewardship
Retention, deletion, access control, and what happens to prompts and documents after they are processed. Whether the vendor states its subprocessors and its incident practice, or leaves the buyer to assume.
Substantive published policy covering most of the ground, and one element better than anything else in this pull. The privacy notice effective 20 February 2025 names thirteen subprocessors outright, including Amazon Web Services, Heroku, Segment, Mixpanel, FullStory, Intercom, HubSpot, Notion, Slack, Sentry and Datadog. No other vendor across pull 2 publishes a list of that specificity without an access request. Alongside it: storage and processing stated as United States only; retention stated as the duration of the relationship plus a further period for the vendor's own operations and archiving; a deletion route by email; data processing agreements and model clauses with vendors described as entered into where feasible and appropriate; essential cookies only in the application; a named security officer, the co-founder and chief technology officer; and a published responsible disclosure programme with an in-scope list, a reporting address, a discretionary reward, a commitment not to pursue good-faith researchers, and a requirement to avoid privacy violations and disruption during testing. What holds this short of the top band: the post-relationship retention period is not quantified, no access control model is described, and no breach notification practice was located.
AI Liability and Recourse
What the vendor stands behind contractually when its output is wrong. Indemnities, caps, carve outs, insurance, and whether any of it is published or only reachable through a negotiated agreement.
What is published disclaims the exposure and allocates nothing toward the buyer. The terms of service provide the services on an as is, with all faults and as available basis, and disclaim all warranties express or implied including merchantability, fitness for a particular purpose and non-infringement. The surviving provisions are named and the direction is one way: the customer's obligation to indemnify Bench IQ survives termination, as do limitations on Bench IQ's liability. Bench IQ may terminate without advance notice at any time and for any reason. No indemnity running to the customer was located, no warranty attaches to any insight, and no insurance position appears. Two limits belong in the record rather than in the grade. No liability cap figure was recovered, so whether one exists is unknown. And the recovered text reads as an evaluation agreement rather than a subscription agreement, framing permitted use as testing and evaluating the services and providing feedback, which raises the question of whether a separate commercial agreement governs paying customers. Both follow from the terms being readable only in fragments through the search index on 31 Aug 2026, and both are the rebuttal route if a fuller reading changes the picture.
Practice Systems Integration Depth
How deeply the product reaches into the systems legal work already lives in: document management such as iManage and NetDocuments, Word and Outlook, contract lifecycle management, matter management, e-billing, and court filing systems.
No integration into any practice system exists, and this is an absence rather than a gap in the reading. The site inventory was established directly from the navigation and footer: login, about us, careers, press, privacy policy, responsible disclosure, security and terms of service. There is no integrations page, no developer or API documentation, and no product page on which an integration might be described, because the home page is the product page. Nothing names a document management system, matter management, e-billing, court filing, Word or Outlook, and no export or data feed is described. The product is reached through a browser at its own application and stands alone. Checked 31 Aug 2026.
Deployment Model and Data Residency
Where the software runs and where the data sits. Multi tenant cloud, single tenant, private deployment, on premises, and whether region of residence is a published option or an enterprise conversation.
The residency answer is published plainly, which is more than either incumbent in this category manages. The privacy notice states that personal information is stored and processed in the United States only, so a buyer knows where the data sits and where it is handled, and the two are not left to be inferred from one another. The subprocessor list makes the hosting identifiable rather than implied, naming Amazon Web Services and Heroku. What is not published: no tenancy model, no statement of whether customer environments are separated, no region options and no enterprise or private deployment tier, and nothing describing where inference runs as distinct from where records are stored. That is what holds it below the top band rather than any vagueness about location. Worth recording for calibration: a five-page site from a company three years old states its data residency clearly, while Docket Alarm and Bloomberg Law both sit a band lower on this axis, one of them offering two different answers across its own documents.
Security Certifications and Trust Center
Independent attestation a buyer can pull without a sales call: SOC 2, ISO 27001, penetration test summaries, a trust center with current reports and named scope rather than a badge image.
A real, named, third-party trust centre exists and is linked from every page of the site, hosted on Drata at a public trust URL. That structure is genuine and is why this is not the floor: the vendor has stood up an independent compliance portal and points to it prominently, which is the connector that parent-level evidence elsewhere in this category lacks. What the site itself states about certification is nothing. No standard is named, no auditor identified, no scope described and no coverage period given on any page read on 31 Aug 2026. The portal's contents could not be read: it refused automated access to two independent fetchers on that date, which is recorded as a retrieval limit on the index's side under the standing rule that a page which will not extract is not an empty page, and nothing about what it contains is inferred here. This grade therefore rests only on what the vendor's own pages state, which is that a trust centre exists. The band fits imperfectly and the note should say so, since the wording at this level describes badges displayed on a site and there are none here. A proposed band amendment is logged, the second on this axis.
Model Supply Chain Disclosure
Which models sit underneath, whose they are, where they run, and whether the vendor commits to telling customers when that changes. A legal buyer inherits every dependency it cannot see.
The vendor identifies the class of model and never says whose. Large language models are named in the privacy notice as the mechanism by which rulings are analysed, and the company has described fine-tuning commercial large language models on its own dataset, so a buyer knows that third-party foundation models sit underneath. No provider is named, no model or version is identified, no processing location for inference is given as distinct from the stated United States storage, and no commitment to notify customers when the model or its training data changes was located on 31 Aug 2026. What makes the omission notable rather than routine is the company it keeps: the same privacy notice names thirteen subprocessors outright, covering hosting, analytics, session recording, support, marketing and observability, and not one of them is a model provider. A vendor that lists Sentry and Datadog by name and omits the systems that generate its core output has drawn a line somewhere, and nothing published explains where or why.
Commercial Transparency
Whether a buyer can learn what this costs without entering a sales process: published rates, the unit being charged, what sits behind an enterprise tier, and what implementation adds.
No pricing information is published at any level, including the unit of charge, and no pricing page exists to carry it. The site inventory was established directly rather than assumed: the navigation is login, about us, careers and press, and the footer adds privacy policy, responsible disclosure, security and terms of service. There is no pricing page, no plan or tier names, no feature split, no per-seat, per-judge or per-report unit, no term length and no statement of what implementation adds. The single call to action on the home page is a demo request. The only adjacent statement located is a 2024 interview in which the chief executive referred to flexible pricing models and said the product is designed for firms of all sizes, which names neither a figure nor a unit and does not appear on any vendor page. Checked 31 Aug 2026.
Firm and Practice Coverage
Who the product is actually built for. AmLaw, midlaw, small firm and solo, in house departments, government and courts, and which practice areas are supported rather than merely claimed.
The claim on the product surface is total and the actual boundary appears only elsewhere. The home page describes a service covering all of a judge's rulings rather than the roughly three per cent producing written opinions, which reads as complete coverage, and it identifies the audience as attorneys and the buyer as a large litigation firm. The real limits are stated in funding announcements rather than on any product page: coverage is the United States federal judiciary only, the platform opened in commercial bankruptcy, and state courts, where the vendor notes the number of judges is orders of magnitude larger, are an expansion target along with other common law jurisdictions. A buyer reading only the site would not learn that their state court judges are absent. No practice area list appears anywhere, no court-by-court coverage statement exists, and nothing identifies which judges are covered or how far back. Checked the home, about and press pages and the vendor's own funding releases on 31 Aug 2026.
Legal Signals
What each signal meansA signal records what public sources say on the date shown. It is not a grade and it is not a recommendation. Where a signal reads Not addressed, it means the index did not locate the material in public sources on that date, which is a statement about disclosure rather than about the product.
Client Data in Training
Can material a lawyer puts into this product be used to train a model?
No located term or policy addresses the question either way.
Searched the home page, about page, press page, privacy notice, responsible disclosure page and the terms of service fragments recoverable through the search index on 31 Aug 2026. No located term or policy states whether anything a customer puts into the product is used to train or refine the models, either way. The exposure differs from a document platform, since the corpus is judicial records rather than client files, but it is not absent: what a litigator asks about, meaning which judge and which issue, is customer-specific information, and the privacy notice lists improving and enhancing the services among its purposes without saying whether that extends to model training on user activity. The notice governs website and application users rather than customer data under a subscription, so the question a general counsel would ask is not addressed by the document that exists.
Prompt and Output Retention
How long does the product keep what a lawyer typed, and can that be set to zero?
Retention is acknowledged in public materials with no stated period.
Retention is acknowledged and not quantified. The privacy notice states that information is kept only as long as necessary, and elsewhere frames the period as the duration of the relationship plus a further period for the vendor's own operations and archiving, with a deletion route offered by email. No number attaches to either limb. Nothing separately addresses queries put to the platform or the insights returned: no retention period for them, no customer-configurable setting and no zero-retention option. Read 31 Aug 2026 alongside the terms of service fragments, which carry no retention provision in the portion recoverable.
Ethical Walls and Matter Segregation
Does retrieval respect the firm’s ethical walls, or can the model read across them?
No located public material addresses walls or matter level segregation.
Nothing addresses separation, and much of what this signal asks does not apply to this product. Bench IQ ingests judicial records rather than client documents, so there is no firm corpus to index, no document management access model to inherit and no matter-level material that a retrieval layer could read across. Recorded as not addressed because that is the honest value, with the reason stated: this is a question the product class does not raise in its usual form rather than one the vendor has neglected. What would apply and is unanswered is narrower: nothing published states whether one user's research activity is visible to colleagues inside a subscribing firm, which matters where two teams in the same firm are on opposite sides of a matter before the same judge. Searched the privacy notice, terms fragments, home, about and responsible disclosure pages on 31 Aug 2026.
Third Party Request and Subpoena Notice
If someone subpoenas the vendor for a firm’s data, does the firm hear about it first?
No located term or policy addresses third party requests for customer data.
An unusual and creditable disclosure that does not answer the question this signal asks. The privacy notice states that since it was founded the company has received zero government requests for information, which is a transparency report in miniature and the only published count of its kind located across pull 2. What it does not do is commit to anything: nothing states what would happen if a request arrived, whether the customer would be told, whether the vendor would seek a waiver where notice is prohibited, or whether it would resist. Recorded as not addressed on that basis, since the value set turns on notice rather than on history, and the disclosure is carried here so the record does not read as though nothing were published. Checked the privacy notice, the terms fragments and the responsible disclosure page on 31 Aug 2026.
Primary Law Corpus Provenance
Where does the law in this product come from, and does the vendor have the right to use it?
Coverage is described by jurisdiction with no identification of the underlying corpus.
Coverage is described by jurisdiction and the corpus behind it is deliberately undisclosed. What is published: the United States federal judiciary, opening in commercial bankruptcy, with the material said to span the roughly ninety-seven per cent of rulings that produce no written opinion, including rulings delivered orally from the bench and other indicators. What is not published is how any of that was obtained. The vendor states that details of the dataset and the technique behind it are confidential because patents are pending, so no source is named, no licence or public-record basis is given, no supplier is identified and no update cadence is stated. The privacy notice mentions PACER, but as a public source of attorney contact details rather than as a corpus source. This is the sharpest instance of the provenance question in the pull, because the corpus is both the entire differentiator and the one thing the vendor has decided not to describe. Checked 31 Aug 2026.
Good Law Verification
Does the product tell you when the authority it just cited has been overruled?
No located public material addresses whether authority is checked for subsequent history.
Searched the home, about, press, privacy notice and terms surfaces on 31 Aug 2026. No citator, treatment signal or currency check is claimed, and none applies: the product characterises how a judge has reasoned rather than retrieving authority whose standing needs validating, so subsequent history is not a question it raises. Recorded as not addressed because that is the honest value, with the reason stated rather than left to look like neglect. One adjacent currency question does bite and is unanswered: nothing published states how far back the judicial record runs, how often it is refreshed, or what happens to a judge's profile when they move court or leave the bench.
Refusal and Uncertainty Behaviour
What does the product do when the answer is not in the corpus?
No located public material addresses what the product does when it cannot ground an answer.
Searched the home page, which is also the product page, along with the about, press, privacy notice, responsible disclosure and terms surfaces on 31 Aug 2026. Nothing documents what the platform does when the record on a judge is thin, when a judge is newly appointed, or when an issue has not come before them. No abstention path is described, no confidence or coverage indicator is mentioned, and no statement addresses how a user would know the difference between a well-evidenced characterisation and a sparse one. The question has unusual weight on this product: its premise is that it reaches material nobody else has, so a user has no independent way to gauge how much sat behind a given insight.
Fabricated Citation Record
Does a public court record exist involving output from this product?
No court order, opinion or disciplinary record naming this product has been located as of the date shown. This is a statement about the public record, not a finding about the product.
No court order, opinion or disciplinary record naming this product has been located as of 31 Aug 2026. Instrument searched: the AI Hallucination Cases database maintained by Damien Charlotin, which tracks decisions worldwide where a court addressed hallucinated AI content and records the tool implicated where known, searched on the product name and the unspaced variant, alongside 2026 sanctions trackers and law firm commentary indexes. This is a statement about the public record on the date shown rather than a clearance, and it is bounded by what that database covers. The failure mode this signal tracks fits the product only loosely: Bench IQ characterises judicial reasoning rather than generating citations to authority, so a fabricated citation reaching a filing would have to originate elsewhere, though an inaccurate characterisation of a judge is a distinct risk this signal does not capture.
Bar Guidance Alignment
Has the vendor engaged in public with the ethics opinions its buyers are bound by?
No located public material engages with bar or ethics guidance.
Searched the full site inventory on 31 Aug 2026, established directly from the navigation and footer rather than assumed: home, about us, careers, press, privacy notice, responsible disclosure, security and terms of service. No bar or ethics authority is engaged with anywhere, including ABA Formal Opinion 512 and any state bar guidance, and no page maps the product to a professional conduct obligation. There is no disclaimer page and no ethics page in the inventory at all, so the absence is structural rather than an omission from a page that exists. Nothing addresses the questions a risk committee would raise about a tool that profiles the judge before whom the firm appears.
Billing and Fee Posture
Does the vendor address what happens to the bill when the work takes an hour instead of six?
No located public material addresses billing, fee or disclosure treatment.
Searched the home, about, press, privacy notice and terms surfaces on 31 Aug 2026. No per matter record of AI-assisted work exists, no guidance on billing, fee or client disclosure treatment is published, and no time-saving claim was located either, which is why this records as not addressed rather than as savings claims only. The vendor's public framing runs to outcome rather than efficiency: its chief executive describes litigators crafting smarter strategies and delivering better results clients will pay for. That is a claim about the value of the work rather than about the hours behind it, and it engages neither limb of this signal.
Outside Counsel Guideline Readiness
Can a firm get this vendor through a client’s AI clause without a bespoke negotiation?
A current subprocessor or model provider list is published.
A current subprocessor list is published openly, with no agreement or access request required, and it is the most specific in pull 2. The privacy notice names thirteen subprocessors outright, among them Amazon Web Services, Heroku, Segment, Mixpanel, FullStory, Intercom, HubSpot, Notion, Slack, Sentry and Datadog, and states that data processing agreements and model clauses are entered into with vendors where feasible and appropriate. A firm asked by a client which third parties sit behind the tool has a real answer to forward, which almost nothing else in this category offers without a portal request. Two gaps keep it from the top value. No model provider appears on the list, so the question of which system generates the insights is unanswered by the very document that answers everything else. And no client-facing disclosure or consent material is published to accompany it.
Court Disclosure Support
If a judge’s standing order requires an AI disclosure, can the product produce one?
No located public material addresses court disclosure or verification certification.
Searched the home page, about page, press page, privacy notice, responsible disclosure page and terms surfaces on 31 Aug 2026. Nothing addresses court disclosure or verification certification. No model is identified or versioned, so which system produced a given characterisation cannot be established; no record of sources behind an insight is described, and the underlying rulings are by the product's own premise mostly unwritten, so there is no opinion a lawyer could attach; nothing records who reviewed an insight; and no export of any kind is offered. A lawyer under a standing order requiring disclosure of AI use, or certification that outputs were checked, would find nothing here to build the answer from.