AI is making safety decisions in hospitals. Nobody independent is checking if it works. Posognos publishes PsiBench, the first safety scorecard for clinical AI, built on the standards 2,000+ hospitals already trust.
AI is replacing the safety systems hospitals have used for decades. These new tools make calls that affect patient outcomes. We are measuring whether they actually work.
Clinicians override the large majority of drug-safety alerts because most are not relevant. AI that cannot distinguish a fatal order from a routine one makes alert fatigue worse, not better.
Estimated annual U.S. cost of adverse drug events. Independent, continuously updated evaluation is the missing infrastructure for measuring whether AI moves that number.
PsiBench translates the clinical safety standards hospitals already trust into automated evaluation scenarios, runs them against AI models independently, and publishes the results.
Clinical pharmacology experts translate established medication-safety standards into validated benchmark scenarios. Every scenario is grounded in the criteria the industry already audits against, and reviewed by named clinical authorities.
Posognos evaluates clinical AI models through EHR test environments and API endpoints using synthetic patient scenarios. No protected health information is accessed, generated, or stored.
Aggregate scores are published on the PsiBench scorecard, freely available to the public. Detailed failure analysis, expert annotations, and remediation guidance are available to subscribers.
In our methods paper, Posognos evaluated 40 frontier language models from 10 providers against 492 expert-authored medication-safety scenarios, across 59,040 independent evaluations. The headline number hides the variation that matters.
Operational balanced accuracy ≥ 80%, response time ≤ 15s, attribution match ≥ 80%. Thirty-one of forty fall short on at least one.
A model can detect a hazard and still attribute it to the wrong clinical category, roughly one in five times, across the field.
On a single number the field looks comparable. At the tier level, models reaching 100% sensitivity drop below 25% specificity, statistically the same as “alert on everything.”
The criteria PsiBench encodes are the criteria the industry already uses. We do not invent metrics. We make the existing ones measurable for AI.
Whether you build clinical AI, deploy it, or set the standards for evaluating it, PsiBench gives you independent safety data you can act on.
Hospital systems are starting to require independent safety validation for clinical AI. The public score is the baseline; subscribers get the expert-validated intelligence that shows exactly what to fix.
Clinical AI vendors make safety claims you cannot independently verify. Compare products against the standards your organization already reports on, without building the testing infrastructure yourself.
PsiBench is built by the experts who write the standards. We are growing the validation network with individual thought leaders contributing scenarios across pharmacy, pharmacovigilance, quality, and safety. Named authorship. A small, elite group. Compatible with institutional positions.
We do not invent safety metrics. We operationalize the clinical safety standards the industry already uses, so evaluation results are immediately meaningful to the organizations that rely on them.
Credible safety evaluation requires independence from the organizations being evaluated, deep clinical expertise, and access to the standards the industry already trusts. Posognos was built on all three.
Posognos' founding experts co-created the national medication-safety evaluation used to audit 2,000+ U.S. hospitals. They bring decades of domain authority and direct relationships with the bodies that define clinical safety.
Every PsiBench scenario is built and peer-reviewed by named domain experts, clinical pharmacists, informaticists, and safety leaders from top U.S. and international institutions.
Posognos is not funded by EHR vendors or AI labs. We do not consult for the entities we evaluate. Evaluation results are published independently. The integrity of the benchmark depends on it.
Posognos /poh-SOH-noss/ · the G is silent, from the Greek posos, "how much," and gnosis, "knowing." To know how much: the third clinical knowing, after diagnosis and prognosis.
Whether you build clinical AI, buy it for a hospital, or hold the expertise that should shape how it's tested, we want to hear from you.