‘Being Human’ in a Technological World

Karola, your career has taken a somewhat unexpected path: you started with acting and playwriting and eventually found your way into bioethics. How did your interests develop along the way?

Karola Kreitmair: I originally went to the United States from Germany to attend drama school in New York City. For two years, I studied acting and eventually playwriting. I realised I wanted a more traditional university foundation, so I matriculated at Brown University, where I studied philosophy and playwriting.

I took a course in epistemology and discovered that there was a language for the kinds of questions I had been interested in. A course in existentialism confirmed that philosophy was the right path for me. After my bachelor’s at Brown and a master’s at the University of Edinburgh, I earned my PhD in philosophy from Stanford University. The decisive turn came through a Clinical Ethics Fellowship at the Stanford Center for Biomedical Ethics. I immersed myself in bioethics and realised I felt very much at home at the interface between analytic philosophy and medical ethics.

As a clinical ethicist and Associate Professor of Bioethics, you move between the clinic and academic research. What does your everyday work involve?

About a quarter to a third of my work is clinical ethics at University of Wisconsin Health. When I am on call, I am available 24/7 for clinical ethics consultation. I may be contacted by a physician who is unsure how to navigate an advance directive stating a patient does not want intubation, while the clinical team believes intubation is necessary to achieve the patient’s medical goals. Sometimes I am at the patient’s bedside, speaking with family members who disagree with the team’s approach to treatment. I help clinicians and families manage that conflict in an ethical and compassionate manner. It is visceral, real-life work.

When I am off call, I conduct mainly conceptual and philosophical work. I think through difficult questions, develop arguments, and engage in academic literary discourse with other scholars. At times, my conceptual work intersects with clinical ethics by addressing broader questions that arise in medical practice. For example, during the COVID-19 pandemic, clinical ethics had to be applied to decisions affecting entire hospitals and the wider health care system.

The pandemic placed medical ethicists in an extraordinary position. How did you experience that period?

In March 2020, it became clear that hospitals would likely face a shortage of ventilators and critical care resources. Very quickly, medical ethicists and clinicians had to develop protocols to decide who would receive resources when there were not enough for everyone. I worked on guidelines for the University of Wisconsin’s hospitals and clinics, and for the state of Wisconsin. We initially followed the principle of “save the most lives”, which essentially meant allocating resources to those most likely to survive. Soon, however, we learned that COVID-19 disproportionately affected Native American and Black patients, meaning they were often least likely to survive. As a result, we incorporated justice considerations into our protocols and did something similar when vaccines became available. Unsurprisingly, we faced some political backlash.

It was an extraordinary period of crisis. I had a baby at home and no childcare, while dealing with deeply traumatic issues. Getting through it while also working towards tenure was a real test. I remember thinking: if I can manage this, I can probably handle anything.

You have written books and papers about wearables and brain organoids and are now working on artificial intelligence in medicine. What is your interest in emerging technologies?

The technologies themselves are not my primary concern. I am more interested in what it means ‘to be human’ and in how this is changed, preserved or perhaps even improved in a technological world.

When I worked on mHealth (editorial note: mobile health technologies such as fitness trackers), I was interested in what these technologies do to our embodied and embedded existence: If you take part in the quantified-self movement, are you becoming alienated from your own body? Are you experiencing your body directly, or are you relating to it mainly through data?

With human brain organoids, which are laboratory-grown models of the human brain, the technology forces us to ask fundamental philosophical questions about moral status. Human beings have moral status, but what grounds it? Many believe that if an entity has a capacity for consciousness, it has at least some moral status. We currently have no good evidence that brain organoids are conscious, and almost nobody believes they are. But even if we did have a way of determining that an organoid was conscious and, say, capable of experiencing pain, that would not settle questions about whether or not research with organoids should stop. We would still have to settle the question of the degree of moral status of an organoid, and how this stacks up against the benefits organoid research might enable.

Brain organoids may sound like science fiction to many people. Artificial intelligence, by contrast, has already become part of health care. How do you assess AI-related technologies?

There are many tasks AI is able to do well. It is useful in medical imaging, for example in detecting abnormalities in mammograms. I also think that AI for clinical documentation is useful, provided that clear constraints are in place. But currently, we’re seeing widespread efforts to integrate AI into all areas of health care, without a well-reasoned argument for why AI is better than what we’re currently doing. This seems backwards to me.

I am particularly interested in what happens when clinical encounters are offloaded onto AI agents. The encounter between a patient and a clinician is deeply vulnerable and involves a significant power imbalance. It is also a setting in which important existential and normative work takes place, quite independently of the medical information being exchanged. It is a fiduciary relationship of enormous normative value.

Your current research project at FRIAS focuses on “empathetic AI” in clinical encounters. What are you investigating?

‘Empathetic AI’ refers to AI systems that can produce language that appears understanding, caring or emotionally attuned. Some argue that AI can reduce clinicians’ burnout. But that would only be true if it was not used to add to clinicians’ workload. If AI-generated chart notes simply enable clinicians to see more patients, it may increase burnout rather than reduce it. More worryingly, if AI floods clinicians and patients with text, insights, and recommendations, which clinicians then have to assess for accuracy, they may become more like proofreaders than physicians, and become alienated from the actual work of medicine.

On the patient side, AI agents may give the illusion of being seen. But we need to distinguish a subjective experience from actually being seen. AI cannot be empathetic. It can produce a somewhat convincing imitation of empathy, but it cannot offer the possibility of genuine human connection. That matters because the clinical encounter is not simply about using the right words. It is about two embodied human beings with lived experience meeting in a deeply vulnerable situation. This encounter is central to the traditional conception of medicine, and, I argue, should remain central in the future. In this context, moral distress shows us the ethically problematic situations, the poor communication between teams, or the problematic absence of agency. This is what I am examining more closely in my current project.

You first came to FRIAS as a Marie Skłodowska-Curie Fellow in 2023 and have returned several times as an alumna since then. What keeps bringing you back?

FRIAS is a fantastic place because it brings together people doing top-notch research in vastly different areas. Even when I present my work to scholars doing research in ecology or on communist furniture, it can have a real impact on my research. Sometimes you do not see the forest for the trees, and feedback from people who are not experts in your field can be particularly valuable.

I also appreciate the balance between community and intellectual freedom. There is enough structure for exchange, but still enough independence to organise one’s time. The city of Freiburg itself contributes to that atmosphere. It is small enough not to pull you in a hundred directions, but large enough not to feel limited. The proximity to the Black Forest is important to me as well. It encourages you to step back from the daily routine and ask more fundamental questions: What actually matters? And how can I serve that through my research? That is very much in the spirit of an Institute for Advanced Studies.