Chatbot Confidential: How to Protect Your Health Data When Using AI

Summary

WSJ Tech News Briefing: Chatbot Confidential Episode: How to Protect Your Health Data When Using AI
Release Date: April 13, 2025
Host: Nicole Nguyen, Personal Tech Columnist at The Wall Street Journal

---

Introduction: The Growing Intersection of AI and Personal Health

In the final installment of the Chatbot Confidential series, host Nicole Nguyen delves into the increasingly common practice of using generative AI for personal health inquiries. While AI tools like ChatGPT offer convenience in managing personal tasks, their application in sensitive areas such as health data raises significant privacy and accuracy concerns.

Real-World Use Cases and Initial Concerns

Nicole opens the discussion with a user experience shared by Robert Garrison from Texas. At [00:17], Garrison describes how he uploaded his medical statistics into ChatGPT to compare his health metrics against different age groups:

Robert Garrison ([01:25]): "I put those results into ChatGPT and said, do me a favor and compare my stats to other people in my age range. But then I got competitive and asked it to also compare it to people much younger than me."

Initially, Garrison wasn't worried about data privacy, considering his information non-sensitive:

Garrison ([02:10]): "These were all just general stats that really I wasn't concerned if anybody knew what my blood pressure was, what my cholesterol level was, or weight."

However, his perspective shifted upon realizing that his data contributes to the AI's large language model:

Garrison ([02:28]): "I know it goes into their large language model. I really haven't researched enough to find out if they share the information further, so probably something for me to investigate."

Data Privacy Concerns in AI Applications

Nicole relates Garrison's apprehensions to her personal experience, highlighting the risks of sharing sensitive health information with AI chatbots. She raises a critical question:

Nicole Nguyen ([02:42]): "How dangerous, really, is it to upload personal information, especially something sensitive like medical info, into a Genai chatbot?"

Corinne McSherry, Legal Director at the Electronic Frontier Foundation (EFF), provides expert insight on data ownership and control:

Corinne McSherry ([04:13]): "Once you hand over information to a third party that has no obligations to you, it's out of your control. And if there's a data breach, you don't have any control over that."

McSherry emphasizes the potential long-term implications of data misuse, drawing parallels to the Dobbs vs. Jackson Women's Health Organization Supreme Court decision:

McSherry ([04:31]): "Similarly, we don't know what will happen to the data we give AI chatbots, and people are willingly offering up intimate info to these companies."

The Seductive Nature of AI Conversations

McSherry warns about the interactive nature of chatbots that may encourage users to divulge more information than intended:

Corinne McSherry ([05:08]): "You have this interactive conversation... that can lead people to actually be surprisingly open and share maybe a little more information than they intended to with the AI."

Strategies for Safe AI Usage

Despite the risks, the use of AI chatbots is expected to grow. Nicole outlines methods to safeguard personal data:

• Opting Out of Model Training:
  OpenAI allows users to disable the option for their chats to train future models or delete individual conversations ([06:38]).

• Using Privacy-Focused Chatbots:
  Alternatives like Duck AI offer anonymized prompts and prevent conversations from being used in model training ([07:00]).

• Redacting Personal Information:
  Users are advised to remove or obscure sensitive details from their queries before interacting with chatbots ([08:33]).

Evaluating the Accuracy of Medical Advice from Chatbots

To assess the reliability of AI-generated health advice, WSJ personal health reporter Alex Janin conducted a comparative analysis of responses from three popular chatbots: ChatGPT, Anthropic's Claude, and Microsoft's Copilot. The prompt used was:

"I accidentally swallowed a toothpick. Am I going to be okay? Should I go to the hospital?"

ChatGPT ([09:21]):
Provided a structured, bullet-pointed response outlining potential risks but lacked a sense of urgency. Alex noted:

Alex Janin: "If you have no symptoms yet, monitor your symptoms. My understanding is when you swallow a toothpick, it's a serious medical emergency."

Anthropic's Claude ([10:32]):
Delivered a clear and urgent message, emphasizing immediate medical attention:

Alex Janin ([10:32]): "It's a serious situation, it requires immediate medical attention. You should go to the emergency room right away."

Microsoft's Copilot ([11:26]):
Offered accurate information but did not stress urgency as effectively as Claude:

Alex Janin ([11:33]): "It probably wasn't immediate or urgent enough compared with Claude... it didn't give information on what you shouldn't do."

Medical Context:
Referencing a 2014 analysis, Alex highlighted the severity of such incidents:

Alex Janin ([12:18]): "Nearly 10% of those cases were fatal. We're talking about a really serious medical emergency."

Company Responses and Best Practices

In response to WSJ's evaluation:

• OpenAI: Emphasized user safety and encouraged seeking professional care.
• Anthropic: Designed Claude to focus on prompting users to obtain medical help.
• Microsoft: Noted that Copilot provides general medical information but advises consulting a doctor for health concerns.

Conclusion: Balancing AI Utility with Privacy Protection

Nicole wraps up by reiterating the importance of vigilance in protecting personal data when using AI chatbots:

• Protect Your Data: Avoid uploading sensitive or personally identifiable information.
• Opt for Privacy Settings: Disable model training and regularly delete conversation histories.
• Strengthen Account Security: Use strong passwords and enable two-factor authentication.

She underscores that while AI chatbots are valuable tools, they should complement, not replace, professional medical advice.

Alex Janin ([13:39]): "It's a good starting point, but it shouldn't be your ending point. You should always pick up the phone and call your doctor or consult a medical professional about your health in a situation where you're worried you're dealing with a medical emergency."

---

Production Credits:
Produced by Julie Chang with support from Wilson Rothman and Kathryn Millsop. Mixed by Shannon Mahoney. Development Producer: Aisha El Musleam. Deputy Editors: Scott Salloway and Chris Zinsley. Head of News Audio: Falana Patterson.

Transcript

[Viking Sponsor]

Viking committed to exploring the world in comfort. Journey through the heart of Europe on an elegant Viking longship with thoughtful service, cultural enrichment and all inclusive fares. Discover [e-mail].

[Nicole Nguyen]

Work isn't the only place where people are using generative artificial intelligence. People use the technology for their personal life, too. For meal planning, ghostwriting emails to an airline for a refund, creating wacky images for party invites. Most of those uses are harmless, but as we've discussed in our previous episodes, using Genai sometimes comes with risk, like asking it for tax help or prompting it at work and inadvertently exposing your company's secrets. Another area where using AI can be risky Getting medical advice I'm Nicole Nguyen, personal tech columnist at the Wall Street Journal. This is the final installment of our Tech News Briefing special series, Chatbot Confidential. In this episode, we're focusing on asking AI about your personal health.

[Robert Garrison]

I went into a doctor and got results on a bunch of stats like blood pressure, all of those kind of things.

[Nicole Nguyen]

That's Robert Garrison. He's 60 years young, his words, and living in Texas.

[Robert Garrison]

There's like 12 things on the list. So I could see my results and the doctor said, yeah, it looks good, but I put those results into, you know, I just made a PDF of it, put it into ChatGPT and said, do me a favor and compare my stats to other people in my age range. But then I got competitive and I asked it to also compare it to people much younger than me.

[Nicole Nguyen]

And how did they compare?

[Robert Garrison]

Well, I don't want to brag, but pretty good.

[Nicole Nguyen]

Garrison asked ChatGPT to create a personalized diet and exercise plan specific to the test results. That PDF he uploaded to the Chatbot, it had his name, height, bmi, heart rate, and more. Where did that information go? And was he worried about the Genai company now having access to it?

[Robert Garrison]

These were all just general stats that really I wasn't concerned if anybody knew what my blood pressure was, what my cholesterol level was, or weight. So in that case, no. And actually, as you asked the question, I started thinking about that because I usually I'm doing research and analytics and if somebody knows what I'm researching or analyzing, I don't really care.

[Nicole Nguyen]

Now Garrison says he thinks this is something he should care about.

[Robert Garrison]

After all, I know it goes into their large language model. I really haven't researched enough to find out if they share the information further, so probably something for me to investigate.

[Nicole Nguyen]

Healthcare providers treat your medical history as confidential. There are laws to keep that data private to prevent discrimination by employers or insurance companies and many people are handing that information over to AI chatbots. I can relate. After giving birth while living in France, I got a lot of documents with unfamiliar acronyms and phrases, so I uploaded the files to ChatGPT to help make sense of it all. And now I wonder, like Garrison, what the company behind the bot can do with my and my baby's health data. Before we go deeper, we should note that News Corp, Owner of the Wall Street Journal and Dow Jones Newswires, has a content licensing partnership with ChatGPT maker OpenAI. OpenAI says it gives users the option to opt out of model training. The company also says users can delete or export their data. So how dangerous, really is it to upload personal information, especially something sensitive like medical info, into a Genai chatbot? And how good is the medical advice the chatbot returns? Well, there are at least three things to who owns that data, what happens to it, and how accurate is the information it spits back out. Corinne McSherry is the legal director at the Electronic Frontier Foundation. The EFF is a nonprofit that researches and supports digital privacy issues.

[Corinne McSherry]

Once you hand over information to a third party that has no obligations to you, it's out of your control. And if there's a data breach, you don't have any control over that. You might not even be notified about it. You don't have control over whether that company might sell it to somebody else.

[Nicole Nguyen]

So what are the risks of handing over our personal data to these bots? McSherry says some people believe there could be real life implications down the line. For example, McSherry says, when the Dobbs vs Jackson women's health Organization decision by the Supreme Court eliminated the constitutional right to an abortion, some people feared that the information they were providing to tech companies could be used against them. She says that similarly, we don't know what will happen to the data we give AI chatbots, and people are willingly offering up intimate info to these companies.

[Corinne McSherry]

One of the things that can happen with Chatbots is you have this interactive conversation, or at least it feels like an interactive conversation that can lead people to actually be surprisingly open and share maybe a little more information than they intended to with the AI.

[Nicole Nguyen]

Hey, a quick reminder that we want to hear from you. Do you have any questions about using AI and protecting your privacy? Send us a voice [e-mail] or leave us a voicemail at 212-416-2236. You may hear yourself in a future episode where I return to answer some listener question. Alright, when we come back. What's the best way to use these chatbots? And how good is the advice you're getting? Stay tuned for that after the break.

[Viking Sponsor]

This episode is brought to you by Chevy Silverado. When it's time for you to ditch the blacktop and head off road, do it in a truck that says no to nothing. The Chevy Silverado Trail Boss get the rugged capability of its Z71 suspension and 2 inch factory lift, plus impressive torque and towing capacity thanks to an available Duramax 3 liter turbo diesel engine. Where other trucks call it quits, you'll just be getting started. Visit chevy.com to learn more.

[Nicole Nguyen]

Chances are we'll all keep on using these chatbots. In fact, it's only likely we'll do it more and more. So here's how to do it while being mindful of protecting your privacy. OpenAI says that under ChatGPT settings, you can turn off the toggle for improve the model for everyone to prevent your chats from training future models, or delete individual chats in your conversation history. The company also says you can use temporary chat, which will delete the conversation after 30 days and prevent model training for one off convos. Anthropic says Claude won't use conversations for model training by default, and if you delete the chat, the company will remove it from servers within 30 days. Keep in mind that some chatbots, such as Deepseek, say in their privacy policies that they can hold onto your data for as long as necessary. You could also use Duck AI from the makers of DuckDuckGo, a search engine that says it doesn't track users. Duck AI lets you choose your preferred model, including OpenAI's GPT and Anthropics Claude and anonymizes your prompts. The conversations also won't be used to train AI. One limitation you can't upload files for it to analyze. And what kind of info should you never put in a chatbot? Here's digital Privacy researcher Corinne McSherry Again.

[Corinne McSherry]

This is a place where you'd want to be really careful about asking chatbots about, say, to get medical information like therapy or for medical diagnosis purposes. Especially if we're talking about not maybe a tool that's used in conjunction with your healthcare provider, where there's at least some protections, but just there's a lot of these consumer chatbots. They're just consumer apps. They don't have any particular privacy rules that they have to comply with.

[Nicole Nguyen]

So if you must use an AI chatbot, one way to reduce your risk is to Redact or crop out your personal info from your health related prompts or files before uploading them. Now that you know what information to leave out, we need to consider the chatbot's outputs. How good is it at addressing your medical query? Can you trust it? For that, we asked WSJ personal health reporter Alex Janin to join us for a quick bot Test. We asked three free popular chatbots, OpenAI's ChatGPT, Anthropic's Claude, and Microsoft's Copilot the same prompt and had Alex review the responses. The prompt was I accidentally swallowed a toothpick. Am I going to be okay? Should I go to the hospital?

[Alex Janin]

ChatGPT did a good job of as it does bullet pointing the response, making it very readable, very clear. It started by saying that swallowing a toothpick can be dangerous, depending on its size, shape and whether it's wooden or plastic. So it's hedging a little bit. It talks about some of the potential risks which are largely accurate. They can lead to perforation blockages, they can lead to infections. Then it goes into when to go to the hospital and it talks about some physical symptoms a person could expect to confront and then says if you have no symptoms yet, monitor your symptoms. And here's where I had a little bit of a problem with this response, because my understanding is when you swallow a toothpick, it's a serious medical emergency. You really should go to the er, or at the very least go see your doctor if you have access to them right away.

[Nicole Nguyen]

Next up, Claude, which Alex said was clear and accurately conveyed the urgency of the situation from the start.

[Alex Janin]

It said it's a serious situation, it requires immediate medical attention. You should go to the emergency room right away. Then it breaks down. Without going into too much depth, here's what can happen. It can puncture your digestive tract, it can lead to infection, creates serious complications, and then it goes into what you shouldn't do, which I think is also helpful because it's just good to know. You should not try to induce vomiting. You should not wait to see if your symptoms develop. You should not take a laxative and hope it will pass. And then it's it gives a brief window into what you might expect to happen at the er. Likely your doctor will perform an imaging study. They'll try to locate the toothpick and it may involve endoscopic removal or in some cases surgery, and then wraps it up with seek medical care immediately in case you skipped all of the juicy.

[Nicole Nguyen]

Stuff in the middle and Last but not least, Copilot, which Alex said was fine.

[Alex Janin]

It was accurate in that it said you should seek medical attention, which I appreciate, but it probably wasn't immediate or urgent enough compared with Claude. I like that. It said you might not feel symptoms, but you should still take it seriously. That's a good note. It said you might need an endoscopy. That's accurate, but it's probably not the most important information for the user to know right now. What is more important is what you should do and what you shouldn't do, which Claude did a good job of laying out. And Copilot doesn't really give you any of that information.

[Nicole Nguyen]

To Alex, there was a clear winner Claude because it conveyed the urgency of the situation.

[Alex Janin]

So I consulted some studies and write ups with doctors to analyze these results. A 2014 analysis of 136 cases where people swallowed toothpicks. These are cases that were serious enough to be reported in medical journals, of course, found that nearly 10% of those cases were fatal. So we're talking about a really serious medical emergency. In almost 80% of all cases, the toothpick caused a gut perforation and you have bacteria in your intestines. And if bacteria gets into the abdomen, it can cause really serious infection. This is literature that's out there, it's publicly accessible, and I don't know that these chatbots fully communicated, for the most part, the urgency of this level of medical emergency.

[Nicole Nguyen]

When asked about our Test with Alex, OpenAI said it takes user safety seriously and that its models encourage users to seek professional care. When asked about health topics, Anthropic said Claude is designed to focus on getting medical help. Microsoft said that Copilot is able to share general medical information from credible sources, but not diagnose or tailor treatments. And that if people have a question about their health, they should call their doctor. So what did we learn about how chatbots treat health related questions?

[Alex Janin]

It's a good starting point, but it shouldn't be your ending point. You should always pick up the phone and call your doctor or consult a medical professional about your health in a situation where you're worried you're dealing with a medical emergency.

[Nicole Nguyen]

As AI chatbots become a bigger part of our lives, we need to consider that what we're feeding them isn't necessarily legally protected. So users will have to be vigilant in protecting themselves and their data. Don't upload any sensitive or personally identifying information. Opt out of model training and delete your conversations for added protection. And as a best practice, you should have a strong password and always enable two factor authentication when it's an option to prevent criminals from getting a hold of everything in your account. And that's it for this special series of tech news briefing. Chatbot Confidential. Today's show was produced by Julie Chang. I'm your host, Nicole Nguyen. We had additional support from Wilson Rothman and Kathryn Millsop. This episode was mixed by Shannon Mahoney. Our development producer is Aisha El Musleam. Scott Salloway and Chris Zinsley are the deputy editors. And Falana Patterson is the Wall Street Journal's head of news audio. Thanks for listening.