Chatbot Confidential: When AI at Work Is Risky Business

Summary

WSJ Tech News Briefing: Chatbot Confidential – When AI at Work Is Risky Business

Introduction

In the April 6, 2025 episode of WSJ Tech News Briefing, hosted by Nicole Nguyen, The Wall Street Journal delves into the increasing integration of generative artificial intelligence (GenAI) chatbots in the workplace. Titled "Chatbot Confidential: When AI at Work Is Risky Business," the episode explores the delicate balance between leveraging AI for productivity and safeguarding company privacy and data integrity. This detailed summary captures the key discussions, insights, and conclusions presented in the episode.

The Rise of AI in the Workplace

Nicole Nguyen opens the discussion by highlighting the transformative impact of GenAI chatbots like ChatGPT and Claude on modern work environments. These tools have become indispensable for tasks ranging from research and drafting emails to creating presentations. The proliferation of AI usage is underscored by statistics from the Pew Research Center, indicating that by 2024, one in five U.S. workers utilized ChatGPT for work purposes—a figure that has more than doubled since the previous year (00:29).

Real-World Applications and User Experiences

Listeners Ashraf Zaid and Ian Yang share their personal experiences with AI in their professional lives. Ashraf Zaid emphasizes the versatility of AI, stating, "I use it on a daily basis for engineering and for modeling and simulation... the AI is my supporter because... I can handle juggling like more than eight organizations, three-year leadership roles and now I'm building my own venture company" (00:29). These testimonies illustrate how AI supports complex, multitasking roles, enhancing efficiency and creativity.

Privacy and Security Concerns

The episode shifts focus to the significant risks associated with AI integration, particularly concerning privacy and cybersecurity. Stephen Rosenbush, Chief of the Enterprise Technology Bureau at WSJ Pro, elaborates on the dual nature of risks presented by large language models (LLMs). He notes, "Companies are very familiar with a certain kind of LLM risk... but they're not too focused on this idea that the LLM could present an actual cybersecurity threat" (03:22).

Nicole Nguyen further categorizes these threats into outbound and inbound risks. Outbound threats involve data leaks, either intentional or accidental, while inbound threats pertain to the potential of AI generating compromised code or suggesting malicious software (03:47). Rosenbush adds that outbound threats are becoming increasingly prominent, citing a 2023 ChatGPT bug where users' personal information was exposed, including names and payment details (04:10).

Case Studies: Corporate Responses to AI Risks

The episode examines how major corporations are responding to these AI-related threats. Bloomberg reports that Samsung banned ChatGPT after an engineer inadvertently leaked sensitive internal source code to the chatbot. Similarly, the Wall Street Journal reveals that Apple has restricted external AI tools for certain employees as it develops its proprietary technology, concerned about the inadvertent release of confidential data (06:37).

Kathy K., CIO of Principal Financial Group, discusses her company's proactive measures. "We actually have locked down any of the public chatbots... we have a whole workflow that will say what's your business rationale? And then there's an approval. They have to take a quick training, their leader has to take a training" (07:37). Principal Financial Group has also developed its own enterprise chatbot, ensuring data protection by controlling access and monitoring interactions with external bots (08:04).

Mitigation Strategies and Best Practices

Nicole Nguyen emphasizes the importance of comprehensive training and robust policies to mitigate AI-related risks. Kathy K. advocates for creating a safe environment where employees can explore and utilize AI tools without compromising security: "Trust employees with any new technology. You have to find ways for safely allowing employees to try these things... my philosophy is, how do I make a safe environment for employees to try these things such that they're learning" (09:11). This approach aims to prevent employees from circumventing security measures, which could lead to data leaks.

Stephen Rosenbush draws parallels to the early days of cloud computing, suggesting that a shared responsibility model will eventually emerge for AI tool integration. "Right now, let's say that the dial that the share that falls on the company itself is pretty close to 100%" (05:00). This indicates that, at present, companies bear the brunt of ensuring AI tool security, but this responsibility may become more distributed as the technology matures.

Future Outlook and Recommendations

As GenAI tools continue to permeate the workplace, the episode underscores the necessity for businesses to stay ahead of potential threats. With rapid technological advancements outpacing governmental policy-making, companies must independently devise strategies to protect their data and privacy. Kathy K. advises continuous training and fostering a culture of safe AI usage to empower employees while safeguarding company interests.

Conclusion

Nicole Nguyen wraps up the episode by previewing the next topic, which will explore the use of chatbots in personal life, particularly in health-related contexts, and how to maintain privacy in such interactions. The episode, produced by Julie Chang with support from Wilson Rothman and Katharine Millsop, offers a comprehensive look into the evolving landscape of AI in the workplace, balancing innovation with vigilance.

Key Takeaways:

• Increased AI Adoption: A significant rise in the use of GenAI chatbots in workplaces for various tasks.
• Privacy and Security Risks: Outbound and inbound threats pose substantial risks to corporate data integrity.
• Corporate Responses: Leading companies like Samsung and Apple are implementing strict AI usage policies to prevent data leaks.
• Mitigation Strategies: Comprehensive training, approval workflows, and the development of proprietary AI tools are essential for managing risks.
• Future Implications: A shared responsibility model for AI security is anticipated, mirroring the evolution seen in cloud computing.

By thoughtfully integrating AI tools and prioritizing security measures, businesses can harness the benefits of GenAI while minimizing potential risks.

Transcript

[Comcast Business]

With leading networking and connectivity, advanced cybersecurity and expert partnership. Comcast Business helps turn today's enterprises into engines of modern business. Powering the engine of modern business powering possibilities Restrictions apply.

[Nicole Nguyen]

It's safe to say work will never be the same again now that generative artificial intelligence is in the picture, people have used AI chatbots for all sorts of tasks.

[Ashraf Zaid]

I use it on a daily basis for engineering and for modeling and simulation, which is really my job sometimes for fun stuff like oh, tell my son a story while we were driving in the car to keep him busy. I'm pretty busy on myself. The AI is my supporter because because of the AI now I can handle juggling like more than eight organization, three year leadership role and now I'm building my own venture company.

[Nicole Nguyen]

Those were Wall Street Journal readers Ashraf Zaid and Ian Yang. And I'm personal tech columnist Nicole Nguyen. Today we bring you the second installment of our special tech news briefing series Chatbot Confidential, where we look at whether it's possible to protect your privacy and keep your personal data safe when using generative AI chatbots like ChatGPT and Claude. In this episode, we dive into an area where the temptation to tap gen AI tools is very strong work. We'll give you the lowdown on the risks your new helper brings and how not to give away company secrets while using them. Before we dive in real quick, a reminder that we want to hear from you. Do you have questions about using AI and privacy? Send us a voice [e-mail] or leave us a voicemail at 212-441-62236. One more time. That's 212-416-2236. I'll be back in a future episode to answer your questions. Alright, back to the show. After ChatGPT first came onto the scene, many companies were quick to ban the chatbot. Still, one in five U.S. workers said they used ChatGPT for work in 2024, according to the Pew Research Center. That's more than double compared to the year before. It's easy to understand why chatbots can take on some of your work, saving you time. The most common use cases research, writing first draft emails, and creating presentations before we get into it. News Corp. Owner of the Wall Street Journal and Dow Jones Newswires, has a content licensing partnership with ChatGPT maker OpenAI. Another Pew survey found that about 16% of respondents say they do at least some of their work with AI, and a quarter say while they're not using it much now at least some of their work can be done with AI. So with AI use growing in the workplace, what are some risks employees and their employers should keep in mind when it comes to these large language models, or LLMs? Stephen Rosenbush is the chief of the Enterprise Technology Bureau at WSJ Pro.

[Stephen Rosenbush]

Companies are very familiar with a certain kind of LLM risk. They're familiar with this idea that the LLMs might make poor decisions in a very convincing way, that they might hallucinate that they might be biased in some way. But they're not too focused on this idea that the LLM could present an actual cybersecurity threat.

[Nicole Nguyen]

And security pros have two names for that threat outbound as in a data leak, and inbound, as in generating compromised code or recommending malicious software, Steven explains.

[Stephen Rosenbush]

The outbound is somewhat more familiar. This is a cybersecurity threat in which there's a risk of data exposure, either intentionally or unintentionally.

[Nicole Nguyen]

In March 2023, a bug in ChatGPT allowed some users to see what other people initially typed in their chats. OpenAI also said the user's first and last names, email addresses, and payment information were exposed. OpenAI said it is committed to user privacy and keeping its data safe.

[Stephen Rosenbush]

But there's also an inbound threat in which companies could be at risk of importing not just compromised data, but actual compromised software through an LLM.

[Nicole Nguyen]

Steven says such threats are bound to multiply, especially as more Genai tools flood the market. As the tech is still so new and technology advances at a much faster clip than the government's ability to enact policy, most companies are on their own, at least for now.

[Stephen Rosenbush]

It reminds me of the early days of cloud computing, when many companies were moving to the cloud and they didn't really fully appreciate the risks that were sort of hidden in the system. There was so much technical work to be done, they didn't have real visibility and they didn't really understand what the cloud providers were responsible for, what they were responsible for, and make sure that everyone was living up to that bargain. So I think that over time we'll see a similar shared responsibility model take shape when it comes to OEMs. Right now, let's say that the dial that the share that falls on the company itself is pretty close to 100%.

[Nicole Nguyen]

And amplifying the risk. Companies are made up of hundreds, sometimes thousands of people, and with that, points of potential failure abound. So who's responsible for making sure a company isn't at risk when employees engage with new online tools. When we come back, we'll hear from a chief information officer on how she's handling the use of gen AI in her workplace. That's after the break.

[Comcast Business]

With leading networking and connectivity, advanced cybersecurity and expert partnership Comcast Business helps turn today's enterprises into engines of modern business Powering the engine of modern business powering possibilities Restrictions apply.

[Nicole Nguyen]

Since the advent of ChatGPT and other gen AI tools, security chiefs at companies have had to figure out how to mitigate risks. And it's not just cyber breaches they have to worry about. Generative AI brings with it a unique challenge. It's easy for employees to inadvertently spill company secrets like confidential or proprietary information, and this has happened already. According to Bloomberg, Samsung banned the use of ChatGPT and other AI powered chatbots after sensitive internal source code was accidentally leaked to ChatGPT by an engineer. And the Wall Street Journal reported that Apple has restricted external AI tools for some employees as it develops its own similar technology. Documents viewed by the Journal show that the iPhone maker is concerned workers could release confidential data. So how are company leaders addressing this? Kathy K. Is the CIO of the global financial company Principal Financial Group.

[Kathy K.]

We actually have locked down any of the public chatbots. If somebody wants to use them. We have a whole workflow that will say what's your business rationale? And then there's an approval. They have to take a quick training, their leader has to take a training, kay says.

[Nicole Nguyen]

They've also signed agreements for enterprise technology to use at the company, like their own chatbot, we call it page that.

[Kathy K.]

People can use that provides a lot of protections around making sure that they're the only ones who are leveraging the data, that they have access to things like that. For those that do go outside, we do track the interactions they're having with the external bots.

[Nicole Nguyen]

So some bosses can see everything you've typed in on a company device. But do they look? That's a discussion for another time. Uploading a client contract, composing an internal email, generating a chart with undisclosed financial data. Getting an unauthorized bot to do any of that could land you and your company in hot water if that data is leaked or absorbed as a part of the model's training data set. Kay says if company secrets do get out, there's a system in place to deal with the fallout.

[Kathy K.]

We have a whole playbook of who do we immediately include? How do we assess the impact of that? Were customers impacted?

[Nicole Nguyen]

But the best fail safe for companies, she says, is to work with the new tech train up their staff and.

[Kathy K.]

Trust employees with any new technology. You have to find ways for safely allowing employees to try these things. Right. Because if not, if you make it so hard for them to try these things, they're going to make mistakes going around all the blockage. Right. And so my philosophy is, how do I make a safe environment for for employees to try these things such that they're learning? We're coming up with new ways of using it.

[Nicole Nguyen]

As Kay suggests, people will keep coming up with new ways to use these tools, like getting medical advice. Next week, we'll tell you about using chatbots in your personal life, specifically health, and how to do it without compromising your privacy. And that's it for this episode of Tech News Briefings special series Chatbot Confidential. Today's show was produced by Julie Chang. I'm your host, Nicole Nguyen. We had additional support from Wilson Rothman and Katharine Millsop. Shannon Mahoney mixed this episode. Our development producer is Aisha Al Muslim. Scott Salloway and Chris Zinsley are the deputy editors. And Falana Patterson is the Wall Street Journal's head of news audio. Thanks for listening.

[Comcast Business]

With leading networking and connectivity, advanced cybersecurity and expert partnership, Comcast business helps turn today's enterprises into engines of modern business. Powering the engine of modern business powering possibilities. Restrictions apply.