SPECIAL FEATURE: ‘With AIs Wide Open’ from IRL: Online Life is Real Life

Summary

Summary of "Click Here" Podcast Episode: SPECIAL FEATURE: ‘With AIs Wide Open’ from IRL: Online Life is Real Life

Introduction

In the special feature episode titled "With AIs Wide Open" from the podcast IRL: Online Life is Real Life, hosted by Bridget Todd and produced by Mozilla, the discussion delves into the intricate world of Large Language Models (LLMs) like ChatGPT. The episode explores both the transformative potential and the significant risks associated with these advanced AI systems. Dina Temple-Raston of Click Here by Recorded Future News provides an overview, setting the stage for an in-depth exploration of the ethical, societal, and technical dimensions of LLMs.

The Rise of Large Language Models

Dina Temple-Raston introduces the concept of LLMs, highlighting their ability to process vast amounts of text data to generate human-like responses. She references a joint venture named Stargate, involving OpenAI, SoftBank, and Oracle, aimed at investing $100 billion in AI infrastructure (00:02). This ambitious project underscores the rapid advancement and the high stakes associated with AI development.

Bridget Todd elaborates on LLMs, explaining their foundational role in applications like ChatGPT and their expanding influence across various industries, including healthcare, finance, and customer service (01:48). The discussion emphasizes how LLMs can enhance productivity through applications such as virtual assistants and automated drafting of communications.

Risks of Large Language Models

David Evan Harris, a former Meta responsible AI researcher, voices concerns about the potential misuse of LLMs. At (02:35), he warns, “You could use them to really tear apart the civic fabric of a country,” highlighting the threats of disinformation and hate speech proliferation.

Harris further explains the dangers of open-source LLMs, specifically referencing Meta's LLaMA model. He states at (03:24), “I just think the bigger danger that I keep coming back to... is misinformation and the idea that a system like Llama2 could be really effectively abused in a large influence operation campaign by what we call in the industry a sophisticated threat actor.” His insights reveal the vulnerabilities inherent in widely accessible AI technologies.

The Open Source Debate

Abeba Berhani, a Cognitive Scientist and Mozilla advisor, critiques the notion of open-source LLMs. At (08:24), she remarks, “So I've followed these models very closely, and I know every time they are real, I know there is some element of deception,” pointing to the lack of transparency in dataset disclosures. Berhani underscores the ethical implications of incomplete data transparency, arguing that without full disclosure, the integrity and safety of AI models remain questionable.

Sasha Luccioni, a researcher at Hugging Face, presents a nuanced perspective on openness. She challenges the binary view of open versus closed source, advocating for a spectrum-based approach. At (16:08), Luccioni states, “Currently there's been a lot of kind of like polarizing discourse about open versus closed source, as if those were the only two choices. But they aren't the only two choices. It's kind of like more productive, more forward thinking to acknowledge the fact that it's a gradient, it's a spectrum.” This approach allows for tailored openness levels based on specific model requirements and use cases.

Data Quality and the Concept of 'Data Swamps'

Abeba Berhani introduces the term "data swamps" to describe the vast, uncurated datasets used to train LLMs. At (10:31), she explains, “Data swamp is an attempt to kind of express how such a huge dump like the Common Crawl or even large scale datasets now... represent not only the good and the healthy of humanity, but also the nasty and ugly of humanity.” This metaphor highlights the contamination of training data with harmful and biased content, which poses significant challenges for creating fair and unbiased AI systems.

Berhani emphasizes the importance of auditing datasets to mitigate biases and prevent the reinforcement of stereotypes. She shares her personal motivation, stating at (11:34), “...if you don't say anything, if you don't do anything about it, nobody else is going to.”

Privacy and Accessibility Concerns

Andre Molyar, co-founder of Nomic, discusses the privacy implications of LLMs like GPT for All. At (20:46), he notes, “One of the core reasons behind why we even built GPT for All... was because of all these large sort of like issues and concerns about privacy with people using OpenAI's models.” GPT for All offers a privacy-preserving alternative by allowing users to run models offline, thereby preventing data from being used to further train or potentially leak personal information.

Molyar also addresses the customization of LLMs, highlighting both the benefits and risks. At (22:35), he articulates a critical perspective: “The reality is like this technology isn't going away... if we're going to live in this inevitable world where we're surrounded by machines that can generate synthesized versions of information... we want to make sure that these generative AI models... are built with everyone's view... not just a couple of organizations behind closed doors with unlimited resources.”

Community Initiatives and Big Science

Sasha Luccioni highlights community-driven projects like Big Science, a collaborative initiative supported by Hugging Face. At (18:37), she describes it as a global effort involving 1,000 researchers from 60 countries to develop an open LLM called Bloom. This project exemplifies how collective efforts can democratize AI development, making advanced technologies accessible beyond the confines of large corporations.

Luccioni further emphasizes the sustainability and collaborative benefits of open-source AI. At (17:43), she compares open-source development to recycling, advocating for resource efficiency and collective problem-solving to reduce wasted effort and promote inclusive technological advancements.

Regulation and Transparency

Abeba Berhani calls for robust regulatory frameworks to ensure transparency in AI development. At (12:10), she asserts, “We need regulation to make companies more transparent about the data they use and where it came from.” Berhani argues that transparency is crucial for auditing and scrutinizing AI systems to prevent misuse and ensure ethical standards are upheld.

The discussion underscores the necessity of balancing openness with safety, proposing that regulated transparency can empower independent researchers and foster ethical AI development without stifling innovation.

Future Outlook and Conclusions

The episode concludes with a consensus on the inevitability of LLMs and the imperative to navigate their development responsibly. Andre Molyar reflects on the dual nature of AI advancements, recognizing the positive potentials while wary of the negative repercussions. At (23:21), he contemplates the societal impact, stating, “The biggest thing is we need to learn how to live with it and how to be able to cope with the side effects that emerge from it.”

Bridget Todd summarizes the necessity for open-source communities to lead the way in ethical AI development, balancing innovation with responsible practices. The episode emphasizes that the future of AI hinges on collaborative efforts, transparent practices, and thoughtful regulation to harness the benefits of LLMs while mitigating their risks.

Notable Quotes

• Dina Temple-Raston (00:02): "They call the venture Stargate. Like the movie. Your job here is to realign the Stargate."

• David Evan Harris (03:24): "I just think the bigger danger that I keep coming back to... is misinformation and the idea that a system like Llama2 could be really effectively abused in a large influence operation campaign by what we call in the industry a sophisticated threat actor."

• Abeba Berhani (08:34): "Llama, for example, was introduced as, oh, an open source large language model. And I went into the paper hoping to find information, detailed information... it was just one tiny, small paragraph in that giant paper."

• Sasha Luccioni (16:08): "It's not like a two camp situation. It's really like let's pick what works for each model."

• Andre Molyar (22:35): "We want to make sure that these generative AI models... are built with everyone's view into how the models are being created, not just a couple of organizations behind closed doors with unlimited resources."

Conclusion

The Click Here podcast episode featuring IRL: Online Life is Real Life offers a comprehensive exploration of Large Language Models, balancing their transformative potential with the pressing ethical and societal challenges they pose. Through expert insights and critical discussions, the episode underscores the importance of responsible AI development, transparency, and collaborative efforts to ensure that LLMs serve the collective good while minimizing their inherent risks.

Transcript

[Dina Templrest]

From Recorded Future News and prx, this is Click here. Hey there, it's Dina. Back in January, President Trump announced a new joint venture between OpenAI, SoftBank, and Oracle. The three companies have banded together to create at least $100 billion in additional computing infrastructure they say will power AI future. They call the venture Stargate. Like the movie. Your job here is to realign the Stargate.

[David Evan Harris]

Can you do that or not?

[Sasha Luccioni]

I can't.

[Dina Templrest]

You can't or you won't? And Sam Altman, the CEO of OpenAI, says there's a lot at stake.

[Andre Molyar]

I believe that as this technology progresses.

[Bridget Todd]

We will see diseases get cured at an unprecedented rate. We will be amazed at how quickly we're curing this cancer and that one and heart disease.

[Dina Templrest]

AI became a household word about three years ago when OpenAI introduced ChatGPT to the world. And what makes ChatGPT so remarkable is something called an LLM. Large language model. LLMs are like these giant storytelling machines. They train on mountains of text, tease out patterns and connections, and then spit out something that feels almost human. One of our favorite podcasts took a deep dive into this world of LLMs back in 2023. It's called IRL online life is real life. It's produced by Mozilla and distributed by our friends at PRX. Here's IRL's host, Bridget Todd.

[Bridget Todd]

This is IRL, an original podcast for Mozilla, the nonprofit behind Firefox. I'm Bridget Todd. In this episode, we get into the risks and rewards of the tech that makes ChatGPT talk. We're talking about large language models, LLMs for short, and the controversy over suddenly giving the whole world access to build with them. But chatbots are only one example of what powerful LLMs can do. Imagine video games where characters can chat with you more. Or virtual assistants that can draft emails for you at work. Banks, insurance companies, travel agencies, everyone is thinking about how to use this technology to increase productivity and more. But there's also a lot of talk about the risks.

[David Evan Harris]

I think a lot of people don't understand the detailed capabilities of large language models. You could use them to really tear apart the civic fabric of a country.

[Bridget Todd]

That's David Evan Harris. Over five years, he managed teams that kept harmful content off Facebook and later also researched responsible AI for Meta. Today, he's worried that LLMs can be used to generate disinformation and hate speech on a greater scale than ever. Like other big tech companies, Meta develops its own LLMs. And now they're urging people to use them and Tweak them with few strings attached. Meta's LLMs are called llama. They might have a cute name, but David says there's a potentially ugly side to Meta's open LLM.

[David Evan Harris]

I have a long history with open source and a big passion for it, but thinking about large language models and Llama and whether or not these things are safe to be open source has been a real turning point for me. I remember more than a decade ago having some conversations with a friend at MIT about the possibility of open source licenses that don't allow for military use. We love making open source software, but what if our open source software is being used to make bombs and kill people? We don't want to do that. Now that connects to this question of what's the threshold for something that we're not comfortable having open source? I just think the bigger danger that I keep coming back to, and maybe not bigger, but the very important danger is misinformation and is the idea that a system like llama2 could be really effectively abused in a large influence operation campaign by what we call in the industry a sophisticated threat actor. And that basically means like a intelligence agency that probably has great hardware and big budgets and well trained engineers.

[Bridget Todd]

David's argument, echoed by many in the industry, is that we don't really know how LLMs of today or tomorrow could be harmful in the long term. But he's also focused on the harms of the here and now and how these disproportionately affect people who are already at risk of exclusion and discrimination. So here's how I think about LLMs. Put on your chef's hat for a moment and imagine you're baking a delicious cake, a layer cake. The foundation or bottom layer of that cake is a large language model. It's made out of lots of Internet data. Now, some of these ingredients aren't the best quality, but with additional layers, coloring, icing and sprinkles, you can fine tune your system to make a chatbot. You fine tune an LLM with data of people chatting. To make a safer chatbot, you train it with data that shows what prompts should trigger safety replies. Whenever you're building software with LLMs like Llama, GPT4 or Falcon, that's just part of what goes into the cake. So there are a lot of options that go into creating an AI system, even when the so called foundational models are the same.

[David Evan Harris]

When you're using AI in a hiring system or in an applicant tracking system that's sorting through thousands and thousands of resumes. You don't need an LLM for that, but you could use LLMs for that kind of thing. You could use LLMs to give you analysis of different candidates. And there may be situations where LLMs demonstrate bias. I say this because, you know, banks are using LLMs, too. If a bank is using an LLM as part of their processes to evaluate loans, and nobody has noticed yet because that LLM has never been systematically tested for bias, maybe that's introducing bias into that bank system. So I think there's some danger there. And a lot of people think, oh, danger, that's not danger. And, you know, if you're getting denied a mortgage because of your race, that's danger to me.

[Bridget Todd]

David feels the industry as a whole is rushing development. At the same time, responsible AI teams have been downsized at several companies. David himself was laid off from Meta's responsible AI team in 2022.

[David Evan Harris]

As a company that's using AI, or even as a government that's using AI, or a nonprofit organization that's using AI, you need to create robust processes to figure out how and when it's appropriate to use AI systems. And you need to have people who are not interested parties. And in the case of a company, an interested party might be just the engineer who wants to ship the damn thing and get the feature running with the AI. And you need to have someone who does not have an incentive to ship products in the loop there, who, who can say, hold on, we might need another month of testing of this. Hold on, we might need to find a way to get someone out from outside the company to really give us an opinion about if this is a fair AI system or if this is safe.

[Bridget Todd]

The reason so many LLMs are at our fingertips now is that investors with deep pockets, Google, Microsoft, Meta, Elon Musk, and others have been pouring money into AI research and powerful supercomputers. Some companies will bake LLMs into their own products. Others will make money by licensing access to them. Everyone is competing for influence and for engineering talent that can help them go faster. Openness can be a strategic move to get ahead by attracting more developers. But often companies also exaggerate how open they are, since it's not always possible to see their data or methods.

[Abeba Berhani]

So I've followed these models very closely, and I know every time they are real, I know there is some element of deception.

[Bridget Todd]

That's Abeba Berhani. Time magazine just named her one of the 100 most influential people in AI. She's a Mozilla advisor and a Cognitive scientist from Ethiopia working at Trinity College in Dublin, Ireland.

[Abeba Berhani]

I mean, llama, for example, was introduced as, oh, an open source large language model. And I went into the paper hoping to find information, detailed information, because I work with data sets. I went immediately into the data sets section and it was just one tiny, small paragraph in that giant paper.

[Bridget Todd]

Abeba wants to know what's inside the datasets for AI, because systems trained on them mimic their biases. Just a handful of data sets get used repeatedly across most LLMs, and these usually include massive amounts of Internet content from an open data set called Common Crawl.

[Abeba Berhani]

The Internet can be a really toxic place. It holds everything from the world's beauty to its ugliness and everything in between. For example, during our audits, we found content such as child abuse or genocide or a lot of explicit pornographic images. You also have to make sure that personal, sensitive information that could be used to identify individuals. You have to make sure things like, like this are not included in datasets. That's one of the reasons why we need to audit the datasets we are using to train models.

[Bridget Todd]

Decades of research show the Internet has never been representative of all the world's people or languages. But in generative AI, it becomes the ground truth. Abeba and her colleagues have coined a term to highlight the problem they see. Abeba, I noticed in one of your papers that y'all actually use the term data swamps, not data sets. Where did that term come from? Like why data swamps?

[Abeba Berhani]

Data swamp is an attempt to kind of express how such a huge dump like the Common Crawl or even large scale datasets now, how they represent not only the good and the healthy of humanity, but also the nasty and ugly of humanity, because you find all kinds of horrible, hateful, degrading texts, especially towards minoritized communities, and you find all kinds of images that is really disturbing to the human eye.

[Bridget Todd]

Even when these enormous data sets are open, it can be too difficult and costly for independent researchers to audit because they're too big. But even using smaller samples of data sets, Abeba and her colleagues have uncovered a ton of problems. In the past, their audits of a leading image data set for AI documented so much racism and sexism that it was decommissioned after decades of use. So, Abeba, is it personal for you, the motivation to keep going?

[Abeba Berhani]

Yeah, it is a bit personal. When I go into data sets, for example, you know, the first thing I query is around, you know, how black women are represented, how Africa as a continent is represented, and so on. So when I see all the negative images or extreme negative stereotypical caricatures or completely inaccurate, false, misleading informations, you feel like if you don't say anything, if you don't do anything about it, nobody else is going to.

[Bridget Todd]

Abeba says we need regulation to make companies more transparent about the data they use and where it came from. She says if companies can hide this information, they can include data they don't actually have permission to use.

[Abeba Berhani]

These artifacts are not something that just remain in the labs of big corporations. These are tools that infiltrate into every social spheres, what information goes into them, what kind of data set that is used to train them, where the data set is sourced and the quality of the data set itself, and how the models were built and any other important information should be open for auditing and for scrutiny. Given that they are almost treated as social goals that are supposed to serve everybody. So some level of openness is really important in terms of making them entirely open. Some people have raised the issue of if they can be accessed by everybody, bad actors can download them and use them for problematic applications. There is always a balance that we have to keep working around. We have to always try and find that is between open and closed.

[Bridget Todd]

It's because LLMs and their data sets can be problematic that we need independent scrutiny of them. Could regulation empower people to work together to improve these systems?

[Dina Templrest]

You're listening to an episode of IRL Online Life is Real Life, a tech series from Mozilla and prx. Just ahead, tackling climate concerns with large language models. This is Qlik here. This episode is brought to you by Progressive Insurance. Fiscally responsible financial geniuses, monetary magicians. These are things people say about drivers who switch their car insurance to Progressive and save hundreds. Visit progressive.com to see if you could save Progressive Casualty Insurance Company and affiliates. Potential savings will vary. Not available in all states or situations. Support for Click Here comes from Curiosity Weekly, a podcast from Discovery. Each episode unpacks breaking science and tech news with experts who can make sense of it all. Recent episodes have covered what neuroscientists have learned from TikTok, how AI manages to read hieroglyphics, and whether today's robots can actually feel pain. All things I've wondered about on Curiosity Weekly from Discovery makes sense of some of the biggest questions and ideas shaping our world. You can listen wherever you get your podcasts.

[Andre Molyar]

When was the last time you said, hmm, I never thought about it that way. The current aims to give you that moment every single day. Hello, I'm Matt Galloway and our award winning team brings you stories and conversations to expand your worldview. Sometimes they connect to the news of the day, sometimes to the issues of our time. And you'll hear all kinds of people on the current, from bestselling authors to maybe your neighbor. Find us wherever you get your podcasts now, including YouTube. We'll talk to you soon.

[Dina Templrest]

Hey, it's Dena. This week on Click Here, we're featuring an episode from IRL Online Life is Real Life. It's hosted by Bridget Todd, and they're taking a deep dive into the risks and rewards of the tech that makes ChatGPT. Well, ChatGPT and how open source communities are developing not just ethical but also sustainable machine learning. Take a listen.

[Sasha Luccioni]

Currently there's been a lot of kind of like polarizing discourse about open versus closed source, as if those were the only two choices. But they aren't the only two choices. It's kind of like more productive, more forward thinking to acknowledge the fact that it's a gradient, it's a spectrum.

[Bridget Todd]

That's Sasha Luccioni, a leading researcher at a startup called Hugging Face. They run an online platform for testing and developing AI. It's so popular that they've been valued at $4.5 billion. Sasha and her colleagues have a fresh take on the open source debate.

[Sasha Luccioni]

What point in the spectrum can I pick for this and this model? And I think it's important especially for policymakers to understand that, that it's not an us versus them. It's not like a two camp situation. It's really like let's pick what works for each model. And also there's no one size fits all solution. Depending on the model, depending on the data, depending on the usage, some point in that gradient is more or less.

[Bridget Todd]

Fitting the spectrum of openness Sasha talks about. It's not just for a model's code or the data sets. It can be for a lot more. Like the documentation and the so called weights that determine how it works. These are all decision points on openness along with the usage terms. Sasha's research at Hugging Face depends on openness. That's because it's all about how to measure and lower the environmental impact of language models, she says training the LLM GPT3 emitted as much carbon as 500 transatlantic flights. And she says open source technology helps with sustainability in other ways too.

[Sasha Luccioni]

Definitely one of the reasons I joined Hugging Face was because I truly believe that by helping open source AI research we can help the sustainability, the energy side of things, but also in terms of democratization, like giving more people Access to models that they can both use out of the box or they can fine tune them in order to fit their context better. I think that's like a net positive for everyone. And for me it's kind of like recycling or thrifting or, or, you know, buying something used and then, you know, patching it up or changing it a little bit to work with what you need it for. And I mean, I thrift like 95% of my clothes, so that's definitely a philosophy I'm really on board with. And for me, open source is definitely much more sustainable in the long run because you're not constantly starting from scratch. And also people can work together and so you have less wasted effort.

[Bridget Todd]

Sasha says a community initiative called Big Science is an example of this. About two years ago, Hucking Face backed 1,000 people from 60 countries in a collaboration to develop an open LLM called Bloom.

[Sasha Luccioni]

It was literally 1000 researchers and volunteers from all over the world who were like, hey, let's train a large language model together. Because we don't have the resources to do it, like each one of us separately. And it was great because we had people who were lawyers, we had people who were like specialists in archival studies to help get data from different places. I mean, we had all sorts of people from all over the world and people who don't necessarily have like a supercomputer on premise, who don't work in a big tech company that can give them access to some kind of computes to train these models.

[Bridget Todd]

Open communities like this one could be directly affected by policies that either limit or encourage important research for alternatives.

[Sasha Luccioni]

During the Big Science project, I joined Hugging Face because I was like, yeah, this is the kind of work I want to do. I don't want to have to be secretive about what I'm doing. I want to do it in an open source way. And I want to help other people who don't necessarily have the means to train these kinds of models. I want to help them also benefit from this technology. The fact that we had all these people involved in Big Science made the whole project and the ensuing model much more representative of society, I feel. And that's important because when these models get used in downstream models or downstream tools or systems, then any kind of information that's implicitly encoded in the model will bubble up to the surface.

[Bridget Todd]

So with all these gradients of openness, it's not only the biggest AI companies developing LLMs, and that can be a good thing. There's an open source alternative to ChatGPT called GPT4All. Amazingly, it works without an Internet connection and the LLMs are compressed so much that you can download them to any regular personal computer. GPT4All was launched by a New York startup called Nomic earlier this year as a privacy preserving alternative to ChatGPT. Tens of thousands of people flock to it. Here's Nomics co founder Andre Molyar.

[Andre Molyar]

One of the biggest focuses that we have around GPT for All is making sure that privacy is the first thing we think about. In some sense. One of the core reasons behind why we even built GPT for All and the ecosystem of models that came in with it was because of all these large sort of like issues and concerns about privacy with people using OpenAI's models.

[Bridget Todd]

You may not know this, but when you type prompts into ChatGPT, OpenAI can use whatever you type to further train their models. There have even been numerous privacy leaks because of it, both corporate and personal.

[Andre Molyar]

The privacy angle that we focus on specifically is making sure that the application in its open source form, you can see all of the code. So we start out from that, that makes it safe. We make sure that everything's audited by the community and the next thing is that we make sure we align by all laws and regulations across Europe and across the us. We don't gather user specific data whenever they use, for instance, the models. And we make sure that the models can run without access to any Internet. So you can go in. Once you download the models to your computer, you can turn off your Internet. If you're stuck in the jungle and you don't have access to Internet, you can ask it for help.

[Bridget Todd]

Nomic's mission is to improve the explainability and accessibility of AI. Their main software product is a data exploration tool for massive data sets called Atlas. But Andre believes GPT for all is important for them to devote resources to as a company.

[Andre Molyar]

When you run a business, there are certain things you get the opportunity to do that you wouldn't be able to do if you weren't running a business. One of those is you have access to capital to be able to work on risky projects like GPT for All purely because you want to, not because you know there's some direct revenue driving source of it.

[Bridget Todd]

Mainly, Andre says he's motivated by a wish to see AI developed by more than just a handful of companies. But he also raises a question of values and who decides how LLMs behave?

[Andre Molyar]

So biases aren't always bad. So an example of a bias could be the model. Always you know prefers to greet you with a salutation before giving you a response. That's a bias that might not be bad, but obviously there's biases that could be bad, right? And one of these sort of important things with large language models is the fact that you can actually go in and customize this. So if you have your own examples of data that you would like your model to be able to output, you can actually change that by training the model.

[Bridget Todd]

Andre offers the example of OpenAI training ChatGPT not to output hateful statements. Today, GPT4All gives access to models fine tuned not to offend as well as some that aren't. Andre says they've had some backlash from people criticizing them for giving more people access to LLMs that could be used for harm.

[Andre Molyar]

The reality is like this technology isn't going away. The biggest thing is we need to learn how to live with it and how to be able to cope with the side effects that emerge from it. A lot of them will be positive, some of them are going to be negative. Like one of the things that I guess I think about quite a bit is like what happens in the 2024 election in the United States. You can go in and pick 10,000 people, get their Facebook profile and customize a chatbot that pretends to be a human to convince them to think one way or the other. And you can do that for no cost at all. I guess the thing that keeps me awake at night is if we're going to live in this inevitable world where we're surrounded by machines that can generate synthesized versions of information and all that information is being piped from one or two company servers, if there's a world where someone like OpenAI owns all the pipes for the information flow and then they get the chance to manipulate that however they want. This is like why we do what we do. We want to make sure that these generative AI models that exist and persist through the world are built with everyone's view into how the models are being created, not just a couple of organizations behind closed doors with unlimited resources.

[Bridget Todd]

LLMs are here. Open source communities that do put people ahead of profits are crucial to unlocking the positive potential of generative AI. The challenge for builders and regulators is to find that balance. On the one hand, so generative AI isn't developed or deployed in harmful ways, and on the other, to empower independent researchers to contribute to healthcare systems work.

[Dina Templrest]

That was Bridget Todd, host of IRL Online Life is Real Life. It's produced by Mozilla and distributed by PRX. Be sure to check out their new season coming February 20th, on the Tech alternatives that are changing how we work, communicate, and even fall in love. I'm Dina Templrest and and this is Klik here from Recorded Future News. We'll be back on Friday with Mic Drop. Have a great week.

[Bridget Todd]

If you're looking for a daily guide to cybersecurity news and policy, sign up for the Cyber Daily from Recorded Future News. It serves up today's most interesting and important cyber stories from our sister publication the Record, and then aggregates all of the big cyber stories you might have missed from news outlets around the world. Just go to the Record Media and click on Cyber Daily to get all you need to know about the world of cybersecurity right in your inbox.