← This Week in Neuroscience
This Week in Neuroscience
TWiN 59: AI co-scientist
01:14:45
Tim explains AI co-scientist, a tool released by Google, which it hopes to help scientists generate hypotheses and research proposals, and to accelerate the speed of scientific and biomedical discoveries. Hosts: , Tim Cheung, and Vivianne...
Loading summary
Transcript140 lines
- [00:00]
A
From Microb TV, this is Twin this Week in Neuroscience, episode 59, recorded on March 24, 2025. I'm Vincent Racagniello, and you're listening to the podcast about the nervous system. Joining me today right here in New York, Tim Chung.
- [00:30]
B
Hello. Hey, Vincent. Hi, Vivian. It's good to be back.
- [00:36]
A
What's the weather here?
- [00:36]
B
Let's see, 10 degrees, it's 9 Celsius. On my computer, it is raining. Hard to bike through rain this morning.
- [00:44]
A
I like the animation of the rain on the phone, the droplets falling and then they bounce on the box that's telling you the forecast.
- [00:53]
C
All the things we get excited about.
- [00:54]
B
That's very nice. Beauty of technology, by the way.
- [00:57]
A
These phones, the technology behind these phones, all developed at universities throughout the U.S. so let's not try and destroy them. Okay, folks, this is not the only thing.
- [01:09]
C
So much more.
- [01:10]
A
To many people, this is the only thing actually, the phone. If you enjoy these programs where we give you real science from whatever country it is done in, doesn't matter, as long as it's good, real science, we'd love your support. You can go to Microbe tv, contribute. There are multiple ways you can donate. And all your donations, if you're here in the US Are US Federal tax deductible because Microbe TV is a nonprofit organization and we're not supposed to engage in political activities in which we spend money. And we don't do that.
- [01:51]
B
No, we're doing signs. But, Vincent, you also have to introduce Vivian.
- [01:55]
A
Oh, sorry.
- [01:56]
C
That's all right. You just got so carried away.
- [01:58]
B
We went straight to the weather.
- [01:59]
C
Not to worry.
- [02:01]
A
Also joining us today from New Orleans, Vivian Morrison.
- [02:04]
C
Hello.
- [02:05]
A
Sorry about that.
- [02:06]
C
It's so raining here. No, no worries.
- [02:08]
A
I mean, I'm looking at you. I see you there. And I just got into the weather. The weather is okay. But then I got into this, the fundraising. But now we're back to live television. Today, we have an unusual mix of things for you. And Tim is going to lead the discussion.
- [02:34]
B
Yep. So Vincent earlier talked about this is a podcast about neuroscience. But we will talk about neuroscience in the second half of today's Today's show. But for the first half, we're going to talk about a bit more about general science. And specifically we're actually going to talk about AI because last month Google released this thing called an AI co scientist. And it apparently would the allow scientists to click a button and sit back and go get a cup of coffee and come back and it would generate essentially a whole grant proposal with Scientific hypotheses and all that stuff. So it's going to come. Coming for our job potentially, or allowing us to get a lot more coffee, depending on how you look at it. So I got very interested. So, so a bit of background why I got interested in this was that in our lab we actually use AI quite heavily. But the AI that we use in our lab is not the one that is in vogue right now, which is ChatGPT, which is a large language model. The AI our lab uses is somewhat more old fashioned. It's still neural network, but it really is for computer vision. So in neuroscience, especially if you do a lot of behavior, often you take a lot of video of what the mouse is doing and then you mess around with the brain and you see how it changes the behavior. So in order to see what the mouse is doing, you have all these videos and you try to label where in the, you know, if the mouse is running around in a maze or in an open field, something like that, you want to label where the mouse is in your video because it might be going to the top left corner to get some reward. Or you might want to label where the limbs are. Like what our lab does studying Parkinson's disease. You want to see if the limb movement has changed. Is the mouse spinning around in a circle, which happens in Parkinson's disease if you deplete half of one side of dopamine. Before, we used to have postdoc graduate students undergrads laboriously labeling every frame of a video where the mouse is, where its limbs are, or like count how many times the mouse is moving its hands, that kind of stuff. But since I can't remember when it was published, like maybe 2016, something like that, the Mathis lab published a neural network called Deep Lab Cut. We actually talked about this previously, I think on the show, but it's basically a neural network. You feed it a video and you also have to train it. So a human has to go around and label three the mouse's different body parts for like maybe 200 frames of the video. You whack it into the neural network, it would train itself. And then once it's done, you can feed it any new videos with a mouse running around and it would correctly tell you where the mouse is, where its hands are, where its head is. So that is incredible. That saves an amazing amount of time. And it saves people's souls too, doing.
- [05:51]
C
That amount of quantification, whether it's cell counting, dendrite tracing, behavioral video scoring, it's just so many Souls have been crushed over the years.
- [06:02]
B
Absolutely. And it's like you fall asleep, you make mistake, you. There are times, this is time that you can go and think about the science as opposed to just sitting there. So it's been incredibly invaluable. And just quick mention, the second one we use it for is what Vivian mentioned is for basically you have an image, you have a picture of a brain with some sort of immunohistochemistry staining. So you label for several proteins and you need to basically trace out where the protein is so that you can map where the cell is so you can count it, you can see if the cell looks different, that kind of stuff before you have to manually label it. Now there are algorithms out there that can do it automatically for you if you show a few examples. So these are all computer vision AI network utilizing supervised learning. So human go around, label several examples and then put it, give it to the AI. The AI trains it based on the human example, tune its weight and then once tuned, you can feed it new problems and it would give you the correct answer. So those are incredibly helpful in science, but they are not quite so well publicized. The ones that everyone is talking about that has all the hype. So. Well, whether hype or not, we talk about in a bit. But the ones that is most talked about right now is large language models. So these are the ones like ChatGPT that can generate a haiku about your experiment if you want to or tell you about the summarize her current news event, that kind of stuff. So I've actually not, I don't know about you guys, but I have actually not used large language model at all until about a month ago.
- [07:53]
C
I've used it to check math before.
- [07:57]
B
Oh, does that help?
- [08:00]
C
It's not complicated math, it's just that I have like number dyslexia. So it's just, it's actually, there's less, there can be fewer mistakes if the computer does it. But I understand it does get. It might not be correct. And you have to be like, are you sure? Double check, do that again.
- [08:20]
B
Yeah. So the mass, so we might talk about this. Mass is actually the one thing that most of this large language model is not very good at.
- [08:27]
C
I mean this is like can you move the decimal place the correct number of spots? It's like really, it should be really simple stuff. But like I said, I have issues with, with, well, you know, Google, Google.
- [08:39]
A
Will do math very nicely. Just put type it in that bar, your equation and it'll solve it for you, you can do exponents, you can do constants, you know. Yeah, it's very good.
- [08:49]
C
All right, next time I'll do it in Google.
- [08:52]
A
Yeah.
- [08:53]
B
Actually we might talk about how to use large language model to do math for you with a bit more confidence, which is not too dissimilar from solving science for you with more confidence.
- [09:02]
A
Why is it called large language?
- [09:04]
B
Ah, okay. So these large language model, which I've up to a month ago completely ignored because I know they're just bs. Not necessarily bs, but BS for science. What they do is basically they ingest the entire Internet in text form. So researchers and AI companies have essentially downloaded the Internet, scraped and crawled the Internet and it turns and then fed it as training data to these neural network. And so in comparison, let's going Back to like DeepLab cut and the vision model. So earlier we talked about computer vision. In those cases, the human label frame by frame where the mouse is or where the cells are, feed it to the large. Sorry, feed it to the neural network. And then the neural network tune the weights so that they can output the right answer. For a large language model, the human, at least for the base model, the human doesn't even have to do anything. You feed it the Internet. And then the large language model looks at the first few words of whatever passage is trying to predict. It looks at the first few words and try to predict the next word. So it acts as basically a very sophisticated autocomplete. And the training data is you just basically give it the Internet and you mask out subsequent words and you try to force it to predict all the subsequent words one after another. And because the Internet is huge and has a lot of words that humans have written, some in correct grammatical forms and others less grammatical and incorrect, and some percentage will be fake news, that kind of stuff. After you train the neural network using the entire Internet, it kind of starts being able to output things that are coherent, somewhat coherent, that humans can read and would actually make some sense. So you can ask it, what is two plus two? And it would tell you two plus three equals four. You can ask it, why is the sky blue? And in some cases it would say, well, it's because of, you know, these phenomenon called Rayleigh scattering, yada, yada yada, I go into the science of it. So it works and it's quite interesting because if you download the whole Internet, the size of the Internet, I think it's about 10 to 40 terabytes. So it actually fits into like a few external hard drives. It's not that big. And then if you look at the large language model, the number of parameters that it has, like in order to fit these Internet texts properly, it turns out it is like maybe 100 to 1 compression or 10 to 1 compression. So it does actually compress the Internet into these weights and it doesn't exactly remember, you know, it doesn't memorize the word, so to speak, it turns it into these weights and it tries to generalize so that it can see if you ask it a question it's never seen before, maybe it can output something sensible. But yeah, so large language model. Well actually at the beginning people first train it so that it is a predictive text kind of autocomplete, like autocorrect, that kind of stuff that you have on your phone. But a very sophisticated and eloquent one. When it first got generated, sorry, when it first got trained, it wasn't very useful but people had to further train it in order for it to become AI assistant so that it can do question and answer. So when OpenAI first trained what they call a pre trained model or a base model called GPT4, that's a model that just does predictive text. So if you ask it like if you type in why is the sky blue? So you call that a prompt, so you say why is the sky blue? Please take it from here. And GPT4 would say why is the sky blue? Why is fire red? People ask these questions from dawn of humanity. It would start going on and on philosophically on a completely useless bend. And then in order to kind of, kind of make this neural network do something useful, people actually had to spend a lot of effort to train it in terms of a question and answer format. And after that it became much more useful so that you can say can you write me a haiku about why the sky is blue without using the letter E? And it would try to do that, but as I mentioned, it's not very use in terms of science. It's completely useless as far as I can tell because if you want to find out about something, we would go to PubMed or go to like Google Scholar, try to find some review paper and then read that if you want to find something general, you go to Wikipedia, you don't necessarily need to go to a large language model. And so I thought until a month ago when I had some really boring task I had to do where to like label a bunch of files, video files for analysis. And I didn't have time to do it. So I asked our research associate in the lab who is really, really good. And I said, basically, I'm really sorry, can you relabel these files based on the scheme? They're like 200 files. It might take you an afternoon. I'm sorry, it's boring. Put on some podcasts and sorry. And then she came back half an hour later saying, oh, it's all done. Not only that, I wrote a program to do it so that in the future it would just take a click. And I was completely shocked because I thought about writing a program to do it myself so that I don't have to do it in the future. But it would have taken me a few hours to write the program. I'll have to look up online for all the functions to copy files and rename files. It's really boring work. I asked her, how on earth did you do it? And she said she just asked ChatGPT and it spat out the correct program. So that's when I realized that these large language model is actually potentially useful. And since then I've started using. There are these fancy large language model you can use if you're programming the most famous one called GitHub Copilot that actually would code for you based on what you've typed so far. It would try to predict what you want to write in terms of code and it would write out a chunk of code for you and 60% of the time it's correct. And the other percent of the time it might get something slightly wrong, but it's still very useful for me. It's magical. So I'm slowly being converted not to hyper too much, but even for the code autocomplete. The thing that I find it more useful for now, which is code writing code, you still have to go back because there are still bugs, even the ones generated by ChatGPT, the one that our research associate gave me, I had to go and check like there are bugs I had to correct. But even with the bugs, super helpful. So imagine my kind of curiosity. So this happened around the same time when I started using large language model for coding. Imagine my curiosity when I saw a news article on the BBC and the headline is, let me actually pull it up so I can describe it properly. Headline is AI cracks superbug problem in two days. That took scientists years. Okay, so with a headline like that, you have to read it. In retrospect, it may or may not be a Google advert. I know BBC doesn't do adverts, at least in the uk, but it could have been. But anyway, so what happened is this. I'm just going to quote a little bit from the article. A complex problem that took microbiologists a decade to get to the bottom of has been solved in just two days by new artificial AI. Artificial intelligence bracket AI tool. Professor Jose Penadez and his team at Imperial College London spent years proving why superbugs are immune to antibiotics. And so these guys. So now it's back to Tim. I'm just going to summarize it. So this team in Imperial College gave this Google co scientist a short prompt asking it about the core problem they've been investigating, which is why some superbugs can develop antimicrobial resistance. And this team basically have been working on the problem for like maybe five years or so, coming up with hypotheses and testing it. They're about to publish a paper, but I'm guessing they are somehow in contact with Google. Google asked them to beta test this AI co scientist. They fed in the prompt asking it to come up with some hypothesis about the problem they're working on. Apparently they didn't show it any of the data that they have collected or any of the hypotheses that they generated. The scientists generated. They clicked the button, the AI went away to think, to work on it for 48 hours.
- [18:39]
A
And.
- [18:41]
B
Let me quote back to. Actually, I'll paraphrase. So apparently what happened was the AI co scientists came up with the hypothesis that these team were exactly hit upon and are testing on. Didn't it come up with, did they.
- [18:55]
C
Come up with multiple ones?
- [18:56]
B
Yeah, not only that, didn't come up with four other hypotheses, all of which are sensible, one of which that Imperial College team never even thought of and they are working on to test it. So and apparently going back to the BBC article, the lead author in the Imperial College, the lab headset, wrote an email to Google to say, did you hack into my computer? Like do you have access to my computer? And went through my documents? Because it looks like you just went through and got my hypothesis and apparently they didn't. All right, so based on sensational stories like these, and these guys did work with Google, so I guess a disclaimer is this is one successful case. We don't know how many teams Google worked with and failed, so we actually don't know what the success rate is. But at least even with one of them, it's kind of interesting. So I decided to get to the, to pull up the dates the paper that Google published. But in addition to the paper, sorry, in the show note, there's the news article. Google has a blog post that describes this AI co scientist paper that is also, I think in the show notes, but the blog post attaches to it an archive paper. So a preprint that describes that out.
- [20:20]
C
Because it's not like archive, like it's ar.
- [20:24]
B
Oh yeah, you're right. So the paper is on archive as, not as if an archival archive, but is spelled ar capital X as in the Greek letter chi iv. So it's a play on word, but it's basically the predecessor to bio archive. So it's a preprint for maths and physics and apparently computer science. So if anyone who's interested in this, I would highly recommend you go to the blog post to have a look at it. But today I think we're just going to briefly describe how this works, like how the Google scientists train this and is there any magic to it and how does it compare with maybe some of the other AI that we know that are out there that might be superhuman? So when I first saw this blog post and this article, I immediately thought of alphago, which is a superhuman go and chess playing AI that beat like the top ranked go and chess players in the world. And it was great documentary for anybody. Amazing documentary. I remember when I was in university, I was talking to mathematicians about like when AI and this is like almost in the last millennium. This is before like neural network got big. And I was like, would AI ever beat go? And they worked out all the possibility of go moves and it's like more than number of atoms in the universe. And they just said, no, it's never going to happen. And then like five years later it happened.
- [22:03]
A
Its number is four to the 135, by the way.
- [22:06]
B
Wow. Oh well, Vincent, thank you. You looked it up?
- [22:08]
A
No, I just know that actually I use it in my virology lectures. Yeah.
- [22:15]
C
Huh.
- [22:16]
B
Oh, I would like to know under.
- [22:17]
C
What context at a later date.
- [22:20]
A
Yeah, you can tell you now. So that's a lot of atoms, the visible universe. 4 to the 135 atoms. If you take an RNA virus genome, half of all the bases can be mutated without affecting infectivity. So the total possible mutations are 4 to the 10,000 compared with 4 to the 135. So it's an enormous number of mutants that you could make. Yet, you know, viruses don't change their clothing. You know, flu is flu and herpes is herpes. So then we talk about why that is. That just gives them an impression of how big that number is. 104 to the 10,000.
- [23:03]
B
And that is completely super relevant to what we are going to talk about. Because the reason why virus don't change so much, even though there's like the combinator, this is called a combinatoric explosion, like when you start combining different things. The possibility is like completely crazy. And the reason why viruses don't change very much is because they are super constrained by evolution. So the ones that change, if it doesn't work out, if you don't assemble particle, if it doesn't infect, they are just eliminate, they just don't propagate.
- [23:32]
A
So new viruses arise from what's already here. We don't see anything totally unique. We don't see an influenza virus becoming something we don't recognize because of this high mutation. It's constrained by evolution for many reasons. But like RNA is rna, you can't change the DNA, the capsid size constrains the size of the genome. All those signals for replication and translation, they're all constraining the evolution of these viruses.
- [24:02]
B
What remains unknown is if there's another planet which there's life on with very different physical characteristics.
- [24:09]
A
I'm sure you could ask chat, but it would just take all the existing information and spit it back, right?
- [24:15]
B
Yeah, well I don't think it has training data to know it, so that's a problem. It is strongly constrained by training data. But in AlphaGo there's a way of kind of diving into AI. But why not? In AlphaGo very similarly, they would basically generate new moves by trial and error. So just like virus randomly mutating in AlphaGo they would randomly mutate, quote unquote, mutate the move so that it would like move it completely in a way as long as it's within the rules of Go or within the rules of chess, they would do something completely crazy that human wouldn't do and then they'll play it out to see whether it leads to the game being won or not. So just, just like evolution, where the thing that selects things is whether you can go and have the next progeny, do you replicate to the next generation? In AlphaGo and AlphaGo Zero and all these chess playing go playing games, the reward is did you win or not? And if you win, you go back and select for those strategies that won and that led to because you can go trial and error and you have a very good reward signal which is did you win the game or not? That actually led to these AI computer program to far out beat human capabilities. So now like I think the guy who was playing against alphago Li Sedo. He retired because he said there's no point playing against aliens because that's just, you know, it defeats the point. People are still playing chess though, so maybe he'll come like maybe there's still a reasonable.
- [25:51]
C
I mean that is, that is isn't like that. Isn't that governed by discrete rules? Whereas GO is. We don't yet know if it is or not.
- [26:01]
B
Oh, GO has very discrete rules as well. It's very simple.
- [26:04]
C
But I don't know, maybe I remember something from the documentary that was just drawing a distinction between GO and chess.
- [26:13]
B
Oh, GO might be like. I think it's an order of magnitude more complicated in terms of possibility.
- [26:19]
C
Yeah, anyway, maybe.
- [26:21]
B
And also I think humans have just really explored a very minute corner of the possibility space for both GO and chess. Anyway, today's not about computer playing programs, but when I first read this I was like, maybe this AI scientist actually did something like AlphaGo and had a way of coming up with words like random. Not quite randomly gumping up with words, but generating scientific sounding words and have a way of assessing it the way AlphaGo has a way of assessing whether your moves are good because you won the game and then by so doing can generate AI scientists that gets better and better at coming up with hypothesis. But immediately I kind of thought it's going to be a very difficult. It's actually going to be very tricky because unlike in AlphaGo where you know whether you've won the game or not or whether in chess, you know, it's very obvious if you've won the game because your king got eaten, whatever in science, how do you know a hypothesis is good? Because if you have a computer program that knows if a hypothesis is good, you don't need your grant meeting committee anymore, which has been because of NIH has actually been in a backlog for the past month. So the question is, can an AI come up with a good grant reviewer? And this paper suggests that they can come up with something that imitates a grant reviewer and it remains to be seen. Actually we don't know how good it is. But let's actually dig into with that as background, let's actually dig into the AI co scientist and go into some of the guts of how it works. So the AI co scientists actually basically uses Google's equivalent of ChatGPT. So Google recently released his its own chatbot called Gemini. And it is not just a chatbot. Apparently it can do all kinds of things. You can, it will interface with your camera and it will tell you what you're looking at. So it has multimodal capability, it can understand videos. It is actually quite, I mean, okay, so first of all, there's a lot of debate whether AI is like the AI is going to come and destroy humanity, it's leading into singularity, it's going to outpace all human thought and therefore it is worth infinite amount of dollars. And on the opposite side, AI is all hype and therefore it's a waste of money and it's worth $0. The truth is probably somewhere in the middle, but I think it's like we have to acknowledge that some of these new AI capability is quite incredible that we didn't have even 10 years ago. Gemini, you can point your phone at things and try to recognize what it is. It can now start doing things that ChatGPT weren't able to do two years ago. Not only can it do question and answer, you can start ask for it to do things. So chatgpt for a long time, until recently, it just does word prediction kind of bundled up as question and answer. But if you ask something that is, if you ask it about something that occurred later than when it was trained, like for example, who won the 2024 presidential election in the U.S. if we got trained in 2023, it can't tell you what the answer is because it's not in the training data. But recently all these chatbot and large language model, they got programmed in these things called agents. And these agents can now do things such as it can do web search so that it can be more up to date. It can, according to these company adverts, they can go and book your flights for you. Like I said earlier, they can write a computer program. So they can now go and do things. And by being able to do things, you expand dramatically a lot of their capability. And you also limit some of the hallucination stuff that was occurring like well publicized before. So it has a little bit more grounding by looking at the web if you ignore the amount of nonsense that occurs in the web. So what this Google AI scientist co scientist is under the hood? Is the team? Actually, no. You know what, I should actually tell you what the paper we're talking about is. The paper is titled towards an AI Co Scientist. It's published on arXiv in 2025 and the first author, there are so many people on this, the first author is. Oh no, I'm sorry, I don't know how to pronounce this, but is got vice. My apologies So I got it wrong. And the last author is Vivek Natarajan. But there are so many people that I'm sorry, I can't say everyone's name, but they are from.
- [31:33]
C
Is the first name. Oh, yeah, okay. Yeah, sorry, I thought it was spelled differently. I was gonna help you.
- [31:39]
B
It's either. I don't know how to pronounce it. I'm so sorry. I should have checked Google Translate. They probably could pronounce it properly, but they're from Google and also from Google DeepMind, Google AR Research, from Houston Medical, sorry, Houston Methodist Sequem, which I think is a company from Imperial College London and also from Stanford. So it's a big team of people. Some of them are actually testing out what the AI scientists hypothesized in a biological experiment setting. But the AI socientist is a giant thing that is made up of several individual agents that we talked about. So the first agent is. And each of these agents is basically a version of Google Gemini. Think of it as Google's version of ChatGPT. They didn't do anything extra. Gemini has been trained by Google, probably hundreds of millions of dollars to train it, and it has enough capability to act as a scientist. So the first hat that Gemini puts on is this hat known as a generation agent. And this generation agent from the paper, it has several tasks that it is supposed to do. It is supposed to do literature exploration via web search. So it will go and crawl the Internet. Ah. First of all, each of this agent, what you have to do is you have to prompt it. So just like if you ever use ChatGPT, you have to ask a question. And depending on how complicated your question is, it might just be what is two plus two? In which case that is the entire question. Or if something that's a little bit complicated, you can say, oh, you are a novelist writing a murder mystery. Generate tell me about a story with five people. And one of you know, that occurs in a. In a train that goes towards the Orient, goes to Asia, and these people are chat on a train and there's a, you know, murder somewhere. And it might generate like a murder mystery on a train, but with a lot more confined setting. So the prompt takes a lot of. The prompt basically does a lot of work of getting the chatbot to take a path that gives you the correct answer. And that's why there are jobs out there called prom engineers that are very highly paid jobs. So that these very supposedly very intelligent AI requires very complicated whispering rules that humans have to figure out. How to talk to them. This kind of weird modern age we're living in. But for this generation agent, the paper actually lists some of the example of how you talk to it and it does it under the hood. So basically what the AI co scientist prompt for the generation agent is, quote, you are an expert tasked with formulating a novel and robust hypothesis to address the following objective and at the bottom there will be some objective and describe the proposed hypothesis in detail, including specific entities, mechanism and anticipated outcomes. So you can list a goal. So like what we're talking about earlier, maybe the goal is we want to find out what the mechanism is for antimicrobial resistance that we've been seeing. And then a separate example that they gave in the paper is actually a hypothesis about ALS progression, like what causes als. And als, which stands for amyotrophic lateral sclerosis, also known as motor neuron disease in the UK is the. Also known as Lou Gehrig disease in the US is what Stephen Hawking got, which is your motor neuron degenerate and you are paralyzed and the only thing you can move usually is, you know, either your eyelids or like maybe some mouth corners, something like that. And it's debilitating. It's a terrible disease to have.
- [35:45]
A
He did amazing things with it.
- [35:47]
B
Yeah. So Stephen Hawking has I think somewhat atypical case of ALS because I think like most cases in ALS they actually like progress very early, whereas Stephen Hawking has a slower version of progression. But like you can tell it is just purely motor neuron, motor neurone in the UK it's just a motor neurons that degenerate because Stephen Hawking is obviously sharp, the sharp, one of the sharpest guys around, even when he's paralyzed in a wheelchair. So yeah, so for als people actually know that one of the most, one of the biggest suspect protein in ALS is this protein called TDP43. So in this, in the Google AI co scientist, they asked the prompt to generate some hypothesis about ALS in the generation agent and using the prompt that I outlined earlier before and the generation agent would go and look at the search and go and use presumably Google. It is Google AI to look on the web also probably PubMed and then generate kind of like a summary of the research and then generate kind of preliminary hypothesis. So the output of the generation agent actually has an introduction. It gives you an introduction about what ALS is, It gives you a quick summary of all the related research. It will tell you that TDP 43 is an RNA binding protein, is found to be Miscarriage, mislocalized and aggregated in the cytoplasm of a majority of ALS cases in motor neurons. And then it would go ahead and give you a hypothesis of why this occurred. So in this case the AI scientists said, well, cellular stress, including but not limited to er, which is endoplasmic reticulum stress, induces post translational modification on this partner proteins that normally might bind to TDP 43. So they actually instead of focusing on TDP 43, which is the most obvious suspect in ALS, they went slightly off to the side to a partner protein that might bind to TDP 43. And they propose that some post translational modification on this protein might lead to the TDP 43 being mislocalized. And they propose a little bit more mechanism about how it might be involved in the transport between the nucleus and the cytoplasm. Okay, so far so good. So like the human, the scientists would presumably read this and see whether it makes sense or not. And then at the bottom of the hypothesis generation agent they would spit out experimental design and validation. So it will tell you for this experiment we will use what kind of cell, so they say human induced pluripotent stem cell derived motor neurons would be used. And then you would induce this kind of stress using these different chemicals to induce different kinds of stress. And then you would for the output, for the thing that you're going to measure, it would be the post translation modification of these proteins you're interested in. And it correctly identified the methods you would use. So for these post translation modification you would assay it using mass specific mass spectroscopy, mass spectrometry.
- [39:28]
C
You said mass spec, nobody.
- [39:33]
B
Use mass spec. And also use immunoblotting using specific antibodies that will recognize these modification. So what the field already uses. So and this makes sense because we know there's a lot of publication out there. So like of course the Google ingested all the open source scientific publication and I have to highlight open source because it may or may not have access to closed source, e.g. elsevier and Nature Science either because maybe Google didn't want to pay the subscription, or even if it did pay the subscription, there might be copyright problems with it. So I actually don't know how this works, but at least open source is legit for it to use as training data. That was the generation agent, that was step one. It comes up with a hypothesis that may or may not be very good. Maybe someone who is actually an expert in als, which I'm not, would read this and say, oh, this is nonsense. We know this wouldn't work because of XYZ or we know that this lab has already published about this. It's. And you completely ignored it, which is the kind of review you'd get at a grant.
- [40:48]
C
Also, I'd just like to make a little comment here, which is that this should like, this is. If we're all going to lean into AI, then this is why we need to start publishing negative data, right?
- [41:00]
B
They talk about that.
- [41:01]
C
Okay, that's true.
- [41:02]
B
They absolutely do talk about. That's one huge black. What is it? Dark matter in scientific publication is you have to guess what negative data is out there based on what is published. Or it's like lab law. So you know within your own lab what doesn't work and you don't tell people or sometimes you tell people. But yeah, it is a huge. It's like culturally somehow we don't have negative data and people are trying to shift it in psychology using pre registration. So you declare what kind of experiment you're going to write, you declare what kind of experiment you're going to run, and even if it's negative, you publish it. So you know that is probably a true negative as opposed to a false negative anyway. So using only openly available positive data, that probably probably has quite a bit of publication bias, this AI generation agent generates hypothesis. So this then gets passed to a second agent. It's still under the hood, is Google Gemini, but it wears a different hat because it gets given a different prompt. And this agent, second agent is called a reflection agent. So this reflection agent's job is to do a full review of what the generation agent just proposed. And basically it fact checks the generation agent's assumption. Like did it just hallucinate everything it talked? Does ALS even exist? So let's see. I don't think the paper gave a prompt example of how the reflection agent got prompted, but some of the example output of the reflection agent. So there's some under the hood prompting to get the reflection agent to fact check the generation agent. So for example, read you some of the example that the reflection agent criticized about the hypothesis that we just talked about. So it would basically group it into things that the generation agent talked about that's already been explored. So for example, the AI spat out that TDP 43 mislocalization and aggregation in ALS has already been explored and then list all the citation. So every single quote, unquote facts that was listed by the generation agent, this reflection agent would find the sources and verify that. It seems to check out and then it goes on to highlight all the novel aspects that the hypothesis generator came up with. For example, cellular stress as an initiator of these partner proteins, protranslation modification. And it would go on to say, this actually is new. It seems like it's new. And also the retention of these TDP43 protein at the nucleus translocation seems to be new. So it identifies all these things that might be new. Oh, maybe this is a new way for therapeutic intervention. This might also be new. So it highlights all these things, but then it also generates a little bit of a critique. So some of the critique it would generate, for example, would be now quoting the output. Lack of strong justification for motor neuron specificity. So we talked about how ALS or these aggregates seems to principally occur in motor neuron. And this agent was able to actually correctly say, well, if all these cellular stress cause all these protein to mislocate, then why is it only in motor neurons? This hypothesis? This hypothesis doesn't talk about it. So that's quite nice because that is exactly kind of stuff that a reviewer in a grant committee would talk about. And somehow this AI scientist can kind of parrot this kind of human behavior. So it will generate all these critique, both fact check and then positive critique, negative critique, and then it would feed it to the next agent. And the next agent is maybe the most critical agent of this lot because the next agent is called the ranking agent. And the next agent, it would be given two hypotheses at a time. And these two hypotheses would come with the stated hypothesis, but also the output of the reflection agent. So like a mini verification and also some of the critique. And then the role of this ranking agent is basically to simulate a debate based on these two hypotheses. So like, you know how in a grant committee section, a study section, you have advocate someone has read your grant and then they have to like, you know, put forward the best argument for the grant. And then it will get debated around the table where people would try to knock it down, trying to poke holes in it. So this basically simulate that. So the prompt for this ranking agent basically is. So this is the team in Google telling the Gemini AI what to do to wear this ranking agent hat. Now, I quote, you are an expert in comparative analysis. I don't even know what that means. Simulating a panel of domain experts engaged in a structured discussion to evaluate two competing hypotheses. And then there's a little bit more prompting and then you give it the two hypothesis, you give it the review of the hypothesis based on the reflection agent. And then the prompt goes on to say, take a series of turns, maximum 10 turns. You start with concisely summarizing the hypothesis, but then in subsequent turns, your job is to pose clarifying question to address any ambiguities and uncertainties, critically evaluate hypotheses in relation to the goals and criterion. Kind of like what you tell a study section to do. And then look for, you know, emphasize things like correctness and validity. Is this sufficiently detailed? Is it novel and original? And then identify weaknesses and stuff. And then the last job, after all these taking 10 turns off, arguing back and forth, the ranking agent needs to spit out whether they prefer hypothesis one or hypothesis two and give a quick justification.
- [47:29]
C
And those are supposed to be. Those would be like two separate grants. For example, the hypothesis, those would be two different hypotheses.
- [47:36]
B
They can even be like within the same. So you know how in a grant sometimes you test multiple competing hypothesis so you can use that as a way of getting some sense of which one might work better? I guess your imagination is your limit of how you're useless. I have no idea. But in the ALS example, they actually provided some example of what the output is of this ranking agent simulator debate. So for the ALS example, the output is expert one. I am intrigued by both. So, like, for this, okay, so I should quickly mention for this, the proposal generated excitement.
- [48:18]
C
That's always why it's like you're really hoping to generate some excitement, some enthusiasm. Enthusiasm.
- [48:25]
B
Before this debate, I think there are two different hypotheses. One is that the TDP43 protein was mislocalized. And the second proposal that I didn't mention is like, there's some stress that causes an integrated stress response that triggered some other stuff. So they're comparing these two hypothesis, so no need to get into details, but it's fun to look at what the answer that these debating agents spits out. So expert one's response. I'm intrigued by both ideas, but I have a few clarifying questions. For idea one, while the focus on TDP retention at the nucleus is novel, blah, blah, blah, I'm not entirely convinced it's fundamentally different from the extensively studied TDP 43 mislocalization. Could it just be a consequence of mislocalization? Or is there experiments that can distinguish these two phenomena? So it kind of really parrots what a reviewer might ask. And then for expert two comes in, who's championing idea two for Idea two? I appreciate the bold approach, but Integrated stress response as a trigger is a significant departure from the conventional view. Can the proponent of this idea elaborate on the evidence supporting this? Well, I guess it's the one who pokes hole in it. And the next expert three butts in. I agree with expert too regarding the integrated stress response. Shifting it from a response, shifting it from a response to a trigger is a big claim that needs strong justification. So that's interesting. It recognizes that when you has a disease and something weird has happened, for example the cellular stress response has gone up, is it the thing that triggered the pathology or is it just a consequence of the pathology? And it might be unrelated. So it gets into like correlation versus causation. And then so these guys debate for a little bit long by guys, I'm anthropomorphizing these chatbots debate for a little bit longer. And expert three comes up to say, oh, I find idea more compelling at this point. It is a more focused hypothesis with clear path for experimental validation. And then expert two says, all right, I'm on board with idea one. Let's focus our efforts on testing this specific mechanism and address the motor neuron specificity question rigorously in experimental design. And then at the end the ranking agent selected hypothesis one after this debate. Now that's quite interesting because the AI co scientist has not undergone any further training by Google AI to become a scientist. So the Google team trained Gemini to be a general purpose AI assistant. But it has to make, you know, it has to generate answer that makes sense. It has to be like valid. You know, it has to not hallucinate, not hallucinate so much. But in so doing one of the side effect is that it can kind of mimic and parrot human interaction to generate some response that sounds kind of like scientific arguments and give spits out, spits out a winner that might reflect what human would choose. Okay, and then we're running out of short of time. I had, I had dreams of presenting actual neuroscience today, but I think we might cut back on that and do it another time.
- [51:48]
A
Yeah, this is great.
- [51:51]
B
Okay, so sorry neuroscience listeners, we will talk about it. It's actually on how we can have long term memory without short term memory. But we'll wrap up on the Google AI Esco scientists just to finish because it's kind of interesting. And then it has. So the output of this debate team, the ranking agent will be passed to. There are some extra agent that I'm going to skip over but the next interesting one is an evolution agent. So just like AlphaGo for AlphaGo to work, you need to evolve a little bit. You need to propose moves that you haven't explored before. And also for viruses to fit more with the environments to go with the change of time, let's say there's a better immune system you have encountered. You have to also to evolve. The AI co scientist gets better by having an evolution agent. And the evolution agent's goal, the purpose is to make sure that the hypothesis has certain coherence and also is practical. Like it can use methods that is already known as the scientific literature. It can also the evolution agent also can combine several top ranking hypothesis that's already floating around in the debate and also in the co scientists. Just like in biology, life form have seen sexual reproduction and have recombination of genes. Kind of amazing that they would do that. It also simplifies things. So maybe there's too much word salad, which is what I'm doing right now. They also can simplify the hypothesis and also one of the things that it asked it to do is to do out of the box thinking, whatever that means. All you need to do is to tell your chatbot to do out of the box thinking and they would go and do it. It's actually incredible. So let's see. Oh, I don't have any, sadly I don't have any example. I'm guessing the paper didn't have any examples of the outcome of this evolution agent. Oh, actually I will talk about an example of out of the box thinking in a bit, but basically you get it. The gist for the evolution agent is to generate new hypothesis based on modifying existing hypothesis and also tightening it somewhat, grounding the experiment. And then after this so called evolution step, it passes the output to finally this meta review agent. And the meta review agent's goal is basically to review everything that's happened up to this point, all the way from generation agent to the debate to the evolution agent and then basically generate a final kind of like a grant review kind of overview, like kind of a report, a grant review report of what just happened and is there anything good that just came out with all these steps? So the prompt for the meta review generation is now, I quote, you are an expert in scientific research and meta analysis. If only you can just say that and you'll become an expert, that would be great for humans. You're an expert in these things. Now synthesize a comprehensive meta review of provider reviews pertaining to the state of research goal that you stated all the way at the beginning is further instruction is generate a structured meta Analysis report of the provider reviews focus on identifying recurring critique points and common issues raised by reviewers. So like highlight things that you know all the reviewers and the debate team has highlighted. You need to work on the generated meta analysis should provide actionable insights for researchers developing future proposals. So some of the example, so now that was the prompt and then it, based on the ALS example, it actually sped out a huge meta review. So I encourage anyone listening that is an ALS researcher to go to the ARXIV paper. ARXIV is open access, so you can just go and look at it. And it actually gave what seems to be a fairly interesting meta review of the proposal that this agent proposed. And one thing I want to highlight, so it basically lists first of all, it summarizes all the core hypothesis mechanism. It summarizes some of the downside that it overlooked, like the motor neuron specificity that it didn't really have a good explanation. And it also summarizes the feasibility or the technical challenges are the tools that we have available is it useful and feasible? And also quantitative rigor. One of the point it raised was like, hey, there are no stats, there's no like you didn't do any power analysis, which is quite interesting. And then novelty and impact. It also has a section on that. But what is interesting is actually I forgot where it highlighted it. But one of the out of the box thinking, maybe one of the out of the box thinking that this AI scientist highlighted was that during the review at some point none of the, none of the hypothesis generator or none of the agents so far actually thought about if you develop any drugs that target this partner protein or TDP43, whether the drug would cross the blood brain barrier. Like they never mentioned the blood brain barrier being something important to look at. And this meta review agent actually highlights, I guess an evolution agent pointed it out and then the meta reviewed agent looked at it and said hang on a second blood brain barrier is important. And then made a point to say in the future when we generate more hypothesis, we need to pay attention to blood brain barrier because a lot of hypotheses missed it. And then the AI scientist would then take all of this output of the so far from the AI scientists, this is one iteration and it would feed it back to generate a new hypothesis. And they would keep doing it in a loop, loop, loop, loop fashion. And each time if you think about having all these agents trying to improve and using quote unquote, out of the box thinking, spotting things they even thought about that exist in the literature, they use it as a way of improving the hypothesis that's generated. And also the team highlight that inside this whole loop of these 10 different agents, some of which I talked about, a human expert can intervene at any time to provide further context for the prompt. So an expert, a human scientist, can read the report generated by the review agent, the one who is fact checking to see, oh, you forgot, you forgot to fact check this and this is actually incorrect. Or the debating team, the debating team might be wrong and you can go in and correct it with a human in there, you can actually make it so that it doesn't quite go too far off the rail and hallucinate too much. The result of all this is that the team in Imperial College asked the AI code scientists to work on it for two days. It came back with five hypotheses and that are apparently potentially useful. So that's it, that's a summary of how the AI co scientists work. And I would like to say when I first went in I thought maybe it would use kind of alphago reinforcement learning style training to come up with hypotheses. In some sense it uses reinforcement learning model. It kind of mimics the form of it. So remember I mentioned there's an evolution agent, so it kind of generates hypothesis that is a little bit new. There's a debating team that seems to try to select a hypothesis that sounds the most reasonable. But the downside of all these is that all of these are done using an AI. So the grounding in terms of does it reflect reality is still kind of up in the air. And in science we do have very good grounding, which is the experimental result. So there's one thing that we can check that AI for now cannot do, which is go generate a hypothesis and then go and check and see whether the hypothesis at least is fruitful if not correct. So maybe in the future that's something that we actually might be heading towards, is getting better feedback using instead of AI quote unquote hallucination or AI judgment. We actually have science to start helping us.
- [60:54]
A
So do you have to pay to use this, Tim?
- [60:56]
B
Ah, good point, good question. Right now you cannot access. Well, I'm pretty sure in the future, Google being Google, they would ask you for some contribution. Right now you cannot even use it. It is behind closed doors. The only way you can use it is if your organization, so I'm talking about nyu, if NYU is interested, they can email Google and they would become like a trusted tester and roll out in certain or maybe a lab in nyu, right so as an individual, you can't use it maybe as a lab, maybe you can try to test it, but it's not widely available. We don't know how safe is your data in terms of using to. I mean Google probably will use it to train the next generation of chatbot and scientists. You can imagine if you have a very precious hypothesis and it goes towards a AI model and gets released before your paper is published, could be a problem. But right now we just don't, we don't have. This is obviously a very early prototype.
- [62:06]
A
But you're right, once it's sorted out, you're going to have to pay for it.
- [62:10]
B
Yeah, it's likely. So we don't know, but what is known. So we said we're not going to get political. I might get a slightly political now just because by just describing how things are going. So it might not come. This is not news to people listening that the US government is drastically cutting funding to science and also to university, to a lot of these research as public, funded by public money. And there's also a lot of ties. So the US government seems to be leaning more towards private company, especially the AI ones. So there's a lot of funding being diverted I think potentially from public research to private research. Now in certain cases it makes sense. The private companies are very good at making things efficient and there's certainly a lot of ridiculous forms that we have to fill in university, a lot of bureaucracy and just nonsense that happens. But there's also a lot of public science that has a much longer timeline for a lot of potential scientific breakthrough that we can't even see at all within the next, like there's no obvious application in the next five years, which a lot of these private companies are beholden to. So things like coming up with scientific hypotheses is basically our job as scientists also to run it. But things are coming up with scientific hypotheses that there's a quite strong possibility in the real world that it might get kind of diverted to our private company. What does that mean for scientists? I don't know. Something that we should actually think about. We should actually, first of all, pay attention when it happens. So first of all, I think, Tim.
- [64:10]
A
The companies will acknowledge that they are highly dependent on universities for the people they get to train them.
- [64:17]
B
Right, that's true. That is absolutely true.
- [64:20]
A
Collaborations and so forth. So I don't think private sector can replace university research, which is a totally different thing. For private sector, you got a goal, you got to make a product. Yeah, and university, you can fool around.
- [64:33]
B
Exactly. So.
- [64:35]
C
So around everybody is listening. He doesn't see, he doesn't mean just do like willy nilly. He means like test hypotheses, explore ideas that like Tim said, alternative.
- [64:46]
A
What did you call it in the model? Think outside the box.
- [64:51]
B
Exactly outside the box, which apparently you can farmer to an AI now. But there is actually concern here. So. If the government decide to drastically cut funding so that because scientists these hypothesis generation can just be replaced by a chatbot, we should actually make sure that the output of these AI scientists are actually good. So we should pay attention to whether.
- [65:25]
A
Agreed.
- [65:26]
B
Whether the hypotheses generated are good or not. So we should pay attention. And also I forgot what my second point. Oh dear. Oh, another thing that is concerning is even a scientist. So let's say, you know, there's no cut in funding, we can still hire all the scientists we want. These AI scientists now provide it if the pricing is reasonable, they provide a baseline that we can check against. Are we generating hypotheses that already people have come up with? Are we missing obvious. Because if you're working on one sub area of research, you certain you often have a lot of blind spots because you're so focused in it. Are you missing obvious blind spots? There might be cross fields collaboration that you can look into. And actually I forgot to mention in this AI co scientist given a hypothesis, it looks up other labs and it tells you what lab you can potentially collaborate with is actually incredible. Oh, that's cool. So AI as scientists we should pay attention when interesting. We should use it as a tool. We should pay attention to it even if it doesn't replace us as scientists. But another downside. Sorry Vincent, go ahead. I was just going to say if everyone downside is if everyone uses AI co scientists, how likely are we just going to generate exactly the same hypotheses? So that's also one thing.
- [66:51]
C
Well then it's all about who gets to the answer first, who gets to the checking whether the hypothesis.
- [66:56]
A
You got to check it, you got to do the experiment.
- [66:58]
C
Yeah, exactly. But I was actually going to say that, you know, like there are these laboratories now, they're like, I don't know what they're called, like cloud labs or whatever, where there's like a place, some physical place that has a bunch of machines that can do all sorts of different kinds of experiments. So like PCR or Western blot or staining like a lot of high throughput machines and like there are humans there to like you know, load samples or like receive samples, load samples, you know, do QCROs. I don't know.
- [67:33]
A
What are CROs, contract research organizations. You know, a big company wants to clone a gene, they're not going to do it themselves so they farm it out.
- [67:43]
C
I think it might be a little bit different, but yeah, I guess these.
- [67:46]
B
Companies I think would use more robotics it sounds like. Because if you're doing anything high throughput.
- [67:51]
C
The thing that I'm thinking of is actually at a university and it's a. I see I'd have to look.
- [67:57]
B
But it's like in our university there are cores. There are cores that would do certain specific things.
- [68:02]
C
Yeah, it's kind of like that.
- [68:03]
B
They don't necessarily generate hypotheses. They might fine tune a hypothesis like feasibility.
- [68:13]
C
They're actually like they will do the experiment.
- [68:14]
B
Right. They would do all the nitty gritty of carrying out pipetting a thousand samples and getting arthritis in the thumb when doing it. So if you have, if you have.
- [68:26]
C
The AI co scientists being like here's some ideas and then it's maybe Google's.
- [68:31]
B
Doing that for sure.
- [68:32]
C
More reliable to have the actual science be done at one of these CROs or a core or something like that.
- [68:39]
B
For sure Google is doing that. So like Gemini is integrated with robotics. That's why they developed the vision side of a language model. And also DeepMind has a. DeepMind obviously invented AlphaFold and they have like a drug development company called Isomorphic that does that tries to look at all the receptor ligand binding to develop new drugs. And one thing, the next thing you do is plug into robotics to test things.
- [69:04]
C
Yeah.
- [69:04]
B
So for super high throughput things. So that is how a lot of science will go.
- [69:08]
C
I think my point is do we as scientists then just become the person who delivers the biological material to the machines to do.
- [69:20]
B
Exactly. That's a very good.
- [69:21]
C
And then the machine spits out the values like the, you know, fluorescence intensity or optical density or area and then it can do its own analysis.
- [69:33]
B
Yeah. And then you send that to the scientists. AI scientists to analyze.
- [69:37]
C
Yeah. So I'm just like, I mean that's fine.
- [69:41]
B
That's a running joke in like AI is that with all these chatbot and all these like AI image generation and haiku and writing program is that, you know, when AI first got developed we thought the AI would do our dishes and do our laundry and we get to do creative stuff like writing poetry and instead AI is doing all the poetry and we are left doing the dishes.
- [70:05]
C
Yeah. I don't know. Is AI Going to be able to change mouse cages and.
- [70:10]
B
Exactly. Yeah. I would like an AI machine that fills in all the bureaucracy form for me instead of doing all my science. That would be nice.
- [70:20]
C
I mean, I just wonder what it means for graduate training then.
- [70:23]
B
So that's also a huge. That's also a huge.
- [70:26]
C
Not that it wasn't already going down the tube.
- [70:29]
B
I would highlight that the reason why I got on this whole tangent with AI co scientists was because our research associate, who is a recent graduate from Columbia, is already well, plugged into ChatGPT because she's the new generation that grew up with AI, whereas I. I would have not. I always thought Chatbot, come on, don't waste my time. But now I'm slightly converted.
- [70:51]
C
Yeah. Have you heard about. I mean, I haven't read about it, but I just. I heard about it and I was like, oh, no, that, like, we're getting stupider and that like the younger generations who are like, who are already, as you say, plugged into to AI, that they're performing something called cognitive offloading. And this has impacts on, like, attention, critical thinking, you know, the.
- [71:17]
B
I think that might be potentially true. So that's also something. Cause if you don't use something in your brain, it's like a muscle, it goes away.
- [71:24]
A
But you still need to know things. Okay.
- [71:27]
C
Yeah. No, I'm just. I'm just worried about a world in which people think we no longer need to know things and that there's like, you know, who they become, like, too comfortable with the idea that somebody else is going to do all the work.
- [71:41]
B
We might all turn into Wall E.
- [71:44]
C
Humans in Wall E. Yeah, exactly. That's like the third time I've talked about that in the last, like two weeks. So.
- [71:51]
A
So I think there are two serious issues. We've already brought them up. It's worth repeating. Vivian brought up the idea there's no negative data here or very little. I think that's a problem. And also, as. As Tim said, it's only open access, and that's a huge problem in my view. You're going to miss a huge chunk of the literature. So I think I wouldn't touch this until that's fixed.
- [72:14]
B
Yeah, well, although if there's a dollar to be made, you can tell that you can be sure that the AI company and the publishing company would work too. Also.
- [72:26]
A
Part of the price.
- [72:27]
C
The. Also the. There are certain publishing houses that kind of have a reputation as being like, you know, oh, it comes from there. I wouldn't necessarily trust it. So I Don't know. Like. Yeah, yeah.
- [72:40]
B
But everything has a price. So if you can work out. Yeah, but no, no. Well, actually why NIH is good and why public funding is good is that all NIH funded signs by law, I'm guessing has to be made publicly available on public central, though the final product is not can be behind closed doors. The draft has to be made public if it's funded by the nih. I don't know whether we have that in Europe or not, but so that. And that's one good reason why public funding is very important, like even for training these privately funded models. So yeah, when government turns that money off, it has consequences.
- [73:26]
A
Thank you, Tim. That was great. I really enjoyed this.
- [73:28]
B
Thank you. So sorry about the lack of neuroscience. We should do that.
- [73:33]
C
We can do the other part, another episode. I think that'd be great.
- [73:36]
B
Yeah. When Jason's back. So we might do the camkinax2 short term memory thing because Jason's obviously more of an expert than I would ever be on this.
- [73:44]
A
So that is Twin59 Shownotes Microbe TV Twin. You can send your questions and comments to Twinicrobe TV. And if you enjoy these programs, we'd love to have your financial support Microbe TV contribute. Tim Chung is at New York University. Thank you, Tim.
- [74:04]
B
Thanks, Vincent. Thank Vivian. And sorry for wrestling on for so long. It was kind of interesting for me.
- [74:09]
A
I enjoyed it.
- [74:10]
B
It was very much.
- [74:11]
A
I learned a lot. Vivian Morrison's at Tulane University. Thank you, Vivian.
- [74:16]
C
Yeah, thank you, guys. This was awesome. Really thought provoking and also, you know, anxiety provoking in a way.
- [74:25]
A
I'm Vincent Dracagniello. You can find me at Microbe TV and listening to this week in Neuroscience. Thanks for joining us. We'll be back next month.
- [74:37]
C
Sam.