Policy Pulse

Here are 3 quick bytes of info to keep you up to speed on the latest news, spotlighting Algorithms and their impact:

1. In response to the rise of AI, U.S. senators have introduced two bipartisan bills, one mandating transparency and appeals processes for AI decisions in government interactions, and the other proposing an Office of Global Competition Analysis to maintain U.S. competitiveness in AI and strategic technologies.

2. The Justice Department has unsealed a 37-count federal indictment against former President Donald Trump, alleging he retained and mishandled classified documents after leaving office, with charges ranging from violation of the Espionage Act to obstruction of justice​​.

3. Yesterday (Monday June 12) was the deadline for public comments on the federal register concerning the use case algorithm discussed below; Click here to read the submitted comments to gain insight into other viewpoints on this database.

The Algorithmic Bias in the Prescription Drug Monitoring Program

AI Foundations - Algorithms in Medicine

Algorithm Use Case - PDMP and Bias

Enough teasing. As promised, it is time to share the critical use case that illustrates how biased algorithms can cause harm to individuals. Enter the PDMP by NarxCare: PMDP = Prescription Drug Monitoring Program; NarxCare = the main software company responsible for managing this massive database for individual states, the Veterans Administration, and others. Prior to joining OPAC, a large portion of my efforts as an activist were dedicated to addressing this and related issues, and I trust my passion for resolving healthcare disparities will shine through my writing. While the bias in this case is likely unintentional, it underscores how easily well-intentioned initiatives can inadvertently lead to adverse outcomes.

Prescription Drug Monitoring Programs (PDMPs) like NarxCare are tools designed to aid in the prevention of prescription drug misuse and overdose. They work by collecting, analyzing, and reporting data on the prescribing and dispensing of controlled substances, with the aim of supporting public health initiatives and clinical decision-making. However, while these tools can be beneficial, it's crucial to understand that they are not infallible and their use has already been shown to lead to unintended consequences due to inherent biases in the algorithms that drive them. In this comprehensive analysis titled 'Dosing Discrimination: Regulating PDMP Risk Scores', Jennifer D. Oliva, a professor at UC Law, San Francisco, digs deep into the potential biases and discriminatory practices in Prescription Drug Monitoring Program (PDMP) predictive surveillance platforms.

The risk scores generated by tools like NarxCare are driven by an algorithm, the misuse of which can lead to the introduction of bias and potentially harmful outcomes. Let's unpack how this can unfold:

• Data Bias: The algorithm is only as good as the data it's trained on. If the data used to train the algorithm is biased, the algorithm will also be biased. For instance, if the data disproportionately represents certain groups of people over others, the algorithm will be more accurate for those groups and less accurate for others. In the case of NarxCare, the algorithm uses data from state registries and potentially other sources like medical claims data, electronic health records, EMS data, and criminal justice data. We know that some of those data sources contain biases, therefore those biases are reflected in the algorithm's output.

• Lack of Contextual Understanding: Algorithms don't understand context in the way humans do. In the case of Kathryn from an article by the brilliant Maia Szalavitz in Wired, the algorithm didn't understand that the multiple prescriptions under her name were for her pets, not for her. This lack of contextual understanding can lead to false positives and negatives, which can have serious consequences for patients.

• Opaque Scoring Mechanism: The scoring mechanism used by these algorithms is proprietary and not transparent. This means that while healthcare providers and even law enforcement are expected to rely on this data to make decisions about a patients care or potential charges on a prescribing healthcare providers, they don't have a clear understanding of how the scores are calculated. This lack of transparency makes it difficult to identify and correct biases in the algorithm and has led to tragic outcomes for both patients and prescribers.

• Overreliance on Algorithmic Output: There's a risk that healthcare providers may over-rely on the algorithm's output when making decisions about patient care. This can lead to decisions that are not in the best interest of the patient. For instance, a high risk score might lead a provider to deny necessary treatment to a patient, even if the high score is due to a bias in the algorithm. This is becoming the case more often than anyone wants to admit, as many prescribers now admit they decide if and what to prescribe based on their risk of law enforcement misusing the PDMP to bring criminal charges versus what they know is medically correct for the patient.

• Lack of Redress Mechanisms: If a patient is adversely affected by a biased algorithm, there is no clear mechanism for them to challenge the decision or to have the decision reviewed. This lack of redress mechanisms can exacerbate the harm caused by algorithmic bias. In addition to harms to the patient, careers have been ruined when prescribers are investigated and/or publicly raided and charged based on reports created by the database. Even if they are later found to have done nothing wrong, they have likely lost their medical license, their reputations are ruined, and their patients have been forced to suffer needlessly.

While algorithms can be powerful tools in healthcare, it's crucial to be aware of their limitations and potential for bias. It's important to use these tools as part of a holistic approach to patient care, and to continually review and refine the algorithms to ensure they are as fair and accurate as possible.

A challenge for you:

In the Policy Pulse section, we discussed the introduction of a new bill in Congress concerning algorithm transparency. This is a step in the right direction, but there is still more work to be done. I challenge you to advocate for greater transparency in how tools such as the PDMP operate. This could involve supporting the new bill and pushing for additional legislation that requires companies like NarxCare to disclose how their algorithms are calculating risk scores when those score hold the potential to cause harm or perpetuate existing biases.

Additionally, I urge you to engage with healthcare providers in your community. Listen to their valuable perspective and insights on this and other ways AI and algorithms are being threaded into medicine by policy at all levels. By understanding their experiences, you can better advocate for policies that ensure these tools are used responsibly and effectively.

Artificial Intelligence Illustrated