Hi, Alpha. >> Hello. >> Can you help me write code? >> I was trained to answer questions, but I'm able to learn. >> That's very open-minded of you. >> Thank you. I'm glad you're happy with me. What's this guy doing? >> That's a developer. >> What do you think he's working on? >> That's a tough question. He might be working on a new feature, a bug fix, or something else. >> It's quite possible. >> Yes. >> Do you see my backpack? >> That's a bad mitten racket. >> It's a squash racket, but that's pretty close. That's a bad mitten racket. >> No, but you're not the first person to make that mistake. >> AI, a technology that has been advancing at breakneck speed. >> Artificial intelligence is all the rage. >> Some are now raising alarms about >> it. Is definitely concerning. >> This is an AI arm choice. We don't know how this is all going to shake out, but it's clear something is happening. >> I'm kind of restless. >> Trying to build AGI is the most exciting journey in my opinion that humans have ever embarked on. If you're really going to take that seriously, there isn't a lot of time. Life's very short. My whole life's goal is to solve artificial general intelligence and on the way use AI as the ultimate tool to solve all the world's most complex scientific problems. I think that's bigger than the internet. I think that's bigger than mobile. I think it's more like the advent of electricity or fire. World leaders and artificial intelligence experts are gathering for the first ever global AI safety summit set to look at the risks of the fast growing technology. And also >> I think this is a hugely critical moment for all humanity. It feels like we're on the cusp of some incredible things happening. I can take you through some of the reaction in today's papers. >> AGI is pretty close. I think >> a huge interest in what it is capable of where it's taking us. >> This is the moment I've been living my whole life for. I've always been fascinated by the mind. So, I'd set my heart on studying neuroscience because I wanted to get inspiration from the brain for AI. >> I remember asking Demis, what's the end game? you know, so you're going to come here and you're going to study neuroscience and you're going to get maybe get a PhD if you work hard. Um, and he said, you know, I want to be able to solve AI. I want to be able to solve intelligence. >> The human brain is the only existence proof we have perhaps in the entire universe that general intelligence is possible at all. And I thought someone in this building should be interested in general intelligence like I am. and then Shane's name popped up. >> Our next speaker today is Shane Le. He's from New Zealand where he trained in math and classical ballet. Are machines actually becoming more intelligent? Some people say yes, some people say no. It's not really clear. We know they're getting a lot faster at doing computations, but are we actually going forwards in terms of general intelligence? >> We were both obsessed with AGI, artificial general intelligence. So today I'm going to be talking about um different approaches to building AGI >> with my colleague Deus. We're looking at ways to bring in ideas from theoretical neuroscience. >> I felt like we were the keepers of a secret that no one else knew. Shane and I knew no one in academia would be supportive of what we were doing. AI was almost an embarrassing word to use in academic circles. Right? you if you said you were working on AI then you clearly weren't a serious scientist. So I convinced Shane the right way to do it would be to start a company. >> Okay, we're going to try to do artificial general intelligence. It may not even be possible. We're not quite sure how we're going to do it, but we have some ideas of the kind of approaches. Huge amounts of money, huge amounts of risk, lots and lots of compute. Um, and if we pull this off, it'll be the biggest thing ever, right? Uh, that is a very hard thing for a typical investor to to put their money on. It's almost like buying a lottery ticket. >> I'm going to be speaking about um systems neuroscience and um how it might be used to help us build AGI. Finding initial funding for this was very hard. We're going to solve all of intelligence. You can imagine some of the looks I got when we were pitching that around. So, I'm a VC and I I look at about 700 to a,000 projects a year and I fund literally 1% of those about eight eight projects a year. So, that means 99% of the time you're in no mode. Wait a minute. I'm telling you this is the most important thing of all time. I'm giving you all this buildup how it explain you connects with the brain. Why are the times right now? And then you're asking me but what's your how are you going to make money? What's your product? It's like so prosaic and a question, you know, um, have you not been listening to what I'm saying? >> We needed investors who aren't necessarily going to invest because they think it's the best investment decision. They're probably going to invest because they just think it's really cool. >> He's the Silicon Valley version of the man behind the curtain in the Wizard of Oz. He had a lot to do with giving you PayPal, Facebook, YouTube, and Yelp. If everyone says X, Peter Teal suspects that the opposite of X is quite possibly true. >> So Peter Teal was our first big investor. But he insisted that we come to Silicon Valley because that was the only place we could there would be the talent and we could build that kind of company. But I was pretty adamant we should be in London cuz I think London's an amazing city. Plus I knew there were really amazing people trained at Cambridge and Oxford and UCL in Silicon Valley. Everybody's founding a company every year and then if it doesn't work, you chuck it and you start something new. That does not is not conducive to a long-term research um challenge. So we were totally an outlier for him. All right everyone, welcome to Deep Mind. So what is our mission? We summarize it as you know Deep Mind's mission is to build the world's first general learning machine. So we always stress the word general and learning here the key things. Our mission was to build an AGI, an artificial general intelligence. And so that means that we need a system which is is general. Um it doesn't learn to do one specific thing. That's a really key part of human intelligence. We can learn to do many many things. >> It's going to of course be a lot of hard work, but one of the things that probably keeps me up at night is to not waste this opportunity to, you know, to really make a difference here and have a big impact in the world. The first people that came and joined DeepMind already believed in the dream, but this was, I think, one of the first times they found a place full of other dreamers. >> You know, we collected this Manhattan project, if you like, together to solve AI. >> In the first two years, we were in total stealth mode. And so, we couldn't say to anyone what were we doing or where we worked. It was all quite vague. >> It had no public presence at all. You couldn't look at a website. The office was at a secret location. When we would interview people in those early days, they would show up very nervously. I had at least one candidate who said, "Um, I just messaged my wife to tell her exactly where I'm going just in case this turns out to be some kind of horrible scam and I'm going to get kidnapped." >> Well, my favorite new person who's an investor is who I've been working on for a year is Elon Musk. So, for those of you who don't know, that's what he looks like. And he hadn't really thought much about AI till until we we chatted. His mission is to die on Mars or something, but not an impact. So, >> we made some big decisions about how we were going to approach building AI. >> This is a reinforcement learning setup, but this is the kind of setup that we think about when we're building when we say we're building, you know, an AI agent. There's basically the agent, which is the AI, and then there's the environment that it's interacting with. We decided that games, as long as you're very disciplined about how you use them, are the perfect training ground for uh AI development. >> We wanted to try to create one algorithm that could be trained up to play several dozen different Atari games. So, just like a human, you have to use your same the same brain to play all the games. >> You can think of it that you provide the agent with a cartridge and you say, "Okay, imagine you're born into that world with that cartridge and you just get to interact with the pixels and see the score. what what what can you do? >> So what you're going to do is take your Q function Q-learning is one of the oldest methods for reinforcement learning. And what we did was combine reinforcement learning with deep learning within one system. >> No one had ever combined those two things together at scale to do anything impressive. And we needed to prove out this thesis. >> We tried doing Pong as a first game. It seemed like the simplest. It hasn't been told um anything about what it's controlling or what it's supposed to do. All it knows is that score is good and it has to learn what the kids controls do and build everything from sort of from first principles. >> It wasn't really working. >> I was just saying to like Shane, maybe we're just wrong. We can't even do pong. It was a bit nerve-wracking thinking how far we had to go if we were going to really build a generally intelligent system. >> And it felt like it was time to give up and move on. And then suddenly we got our first point, you know, and then was like, is this random? No, no, it's really getting a point now. >> It was really exciting that this thing that previously couldn't even figure out how to move a paddle had suddenly been able to totally get it right. Then it was getting a few points and then it won its first game and then 3 months later no human could beat it. You hadn't told it the rules how to get the score nothing and you just tell it to maximize the score and it goes away and does it. This is the first time anyone had done this endto-end learning. >> Okay. So we have this working in quite a general way. Now let's try another game. >> So then we try breakout. Now at the beginning after 100 games the agent is not very good. It's missing the ball most of the time, but it's starting to get the hang of the idea that the bat should go towards the ball. Now, after 300 games, it's about as good as any human can play this. We thought, well, that's pretty cool. But we left the system playing for another 200 games. And it did this amazing thing. It found the optimal strategy was to dig a tunnel around the side and put the ball around the back of the wall. >> Finally, the agent is actually achieving what you thought it would achieve. That is a great feeling, right? Like I mean when we do research that is the best we can hope for. We started generalizing to 50 games and we basically created a recipe. We could just take a game that we have never seen before. We would run the algorithm on that and DQN could train itself from scratch achieving human level or sometimes better than human level. We didn't build it to play any of them. We could just give it a bunch of games and would figure it out for itself. And and that there was something quite magical in that. Suddenly you had something that would respond and learn whatever situation it was parachuted into. And that was like a huge huge breakthrough. It was in many respects the first example of any kind of thing you could call a general intelligence. >> Although we were wellunded startup holding us back was not enough compute power. >> I realized that this would accelerate our time scale to AGI massively. >> I used to see DAS quite frequently. we'd have lunch and he did um say to me that he had two companies that were involved in in buying Deep Mind and he didn't know which one to go with. The issue was would any commercial company appreciate the real importance of the research and give the research time to come to fruition and not be breathing down their necks saying we want some kind of commercial benefit from this. Google has bought DeepMind for a reported 400 million pounds, making the artificial intelligence firm its largest European acquisition so far. The company was founded by 37year-old entrepreneur Demis Hazis. >> After the acquisition, I started mentoring and spending time with Demis and just listening to him. And this is a person who fundamentally is a scientist and a natural scientist. He wants science to solve every problem in the world and he believes it can do so. That's not a normal person you find in a tech company. We were able to not only join Google but run independently in London, build our culture which was optimized for breakthroughs uh and not deal with products, do pure research. Our investors didn't want to sell, but we decided that this was the best thing for the mission. In many senses, we were underelling in terms of value before it more matured and you could have sold it for a lot more money. And the reason is because there's no time to waste. There's so many things that got to be cracked while the brain's still in gear. You know, I'm still alive. There's all these things that got to be done. So you haven't got I mean how many how many billions would you trade for another 5 years of life you know to to do what you what you what you set out to do. >> Okay all of a sudden we've got this massive scale compute available to us. What can we do with it? >> Go is the pinnacle of board games. It's the most complex game ever devised by man. There are more possible board configurations in the game of Go than there are atoms in the universe. Go is the holy grail of artificial intelligence. For many years, people have looked at this game and they've thought, "Wow, this is just too hard. Everything we've ever tried in AI, it just falls over when you try the game of Go." And so that's why it feels like a real litmus test of progress. We had just bought Deep Mind. They were working on reinforcement learning and they were the world's experts in games. And so when they introduced the idea that they could beat the top level Go players in a game that was thought to be incomputable, I thought, well, that's pretty interesting. >> Our ultimate next step is to play the legendary Lee Sudol in just over 2 weeks. >> A match like no other is about to get underway in South Korea. >> Lee Doll is getting ready to rumble. >> Lisa Doll is probably one of the greatest players of the last decade. I describe him as the Roger Federer of Go. >> I showed up and all of a sudden we have a thousand Koreans who represent all of Korean society, the top go players and then we have Demis and a great engineering team. >> He's very famous for very creative writing play. >> So this could be um difficult for us. I figured at least it all going to beat these guys, but they'll make a good showing. Good for a startup. >> I went over to the technical group and they said, "Let me show you how our algorithm works." >> If you step through the actual game, we can see kind of what AlphaGo thinks. >> The way we start off training Alph Go is by showing it 100,000 games that strong amateurs have played. And we first initially get Alph Go to mimic the human player and then through reinforcement learning it plays against different versions of itself many millions of times and learns from its errors. >> This is interesting. >> All right, folks. You're going to see history made. >> So the game starts. >> He's really concentrating. >> He really is. Look at Look at that. >> Oh, that's a very That's a very surprising move. I think we've seen an original move here. >> Yeah, that's an exciting move. I like moves like that. >> Professional commentators almost unanimously said that not a single human player would have chosen move 37. So, I actually had a poke around in AlphaGo to see what AlphaGo thought and Alph Go actually agreed with that assessment. Alph Go said there was a 1 in 10,000 probability that move 37 would have been played by a human player. The game of Go has been studied for thousands of years and Alph Go discovered something completely new. >> He resigned. Lisa has just resigned. He's beaten. >> The battle between man versus machine in a computer just came out the victor. >> Google put its deep mind team to the test against one of the brightest minds in the world and won. >> That's when we realized the deep mind people knew what they were doing and to pay attention to reinforcement learning as they had invented it. >> Everybody say good night. >> Based on that experience, Alph Go got better and better and better and they had a little chart of how much better they were getting. And I said, "When does this stop?" And Dema said, "When we beat the Chinese guy, the top rated player in the world." >> Kier versus Alph Go. >> I think we will see AlphaGo pushing through there. >> Alpha Go is ahead quite a bit. >> About halfway through the first game, the best player in the world was not doing so well. What can black do here? >> Looks difficult. And at a critical moment, the Chinese government ordered the feed cut off. It was at that moment we were telling the world that something new had arrived on Earth in the 1950s when Russia Splutnick satellite was launched. It changed the course of history. >> It is a challenge that America must meet to survive in the space age. >> This has been called the Sputnik moment. The Sputnik moment created a massive reaction in the US in terms of funding for science and engineering and particularly space technology. >> For China, Alph Go was the wakeup call, the Sputnik moment. It launched an AI space race. >> We had this huge idea that worked and now the whole world knows. It's always easier to land on the moon if someone's already landed there. It is going to matter who builds AI and how it gets built. I always feel that pressure. There's been a big chain of events that followed on from all of the excitement of Alph Go. When we played against Lisa Dole, we actually had a system that had been trained on human data on all of the millions of games that had been played by human experts. We eventually found a new algorithm, a much more elegant approach to the whole system which actually stripped out all of the human knowledge and just started completely from scratch and that became a project which we called Alpha Zero. Zero meaning having zero human knowledge in the loop. Instead of learning from human data, it learned from its own games. So it actually became its own teacher. Alpha Z is an experiment in how little knowledge can we put into these systems and yet how quickly and how efficiently can they learn >> and the other thing is >> Alpha Zero doesn't have any rules learn through experience. >> The next stage was to make it more general so that it could play any two-player game things like chess and in fact any kind of two-player perfect information game. >> It's going really well. It's going really really well. Oh wow. >> Screw down like fast. >> Alpha Go used to take a few months to train, but Alpha Zero could start in the morning playing completely randomly and then by tea be superhuman level and by dinner it would be the strongest chess entertainer has ever been. >> It's amazing. It's amazing. It It's discovered its own attacking style, you know, to take on, you know, the current level of defense. I mean, never in my wildest dreams. >> I I agree actually. I was not expecting that either. And I it's fun from I think I mean it's inspired me to to get back into chess again because it's cool to see that there's even more depth than we thought in chess. I actually got into AI through games. Initially it was board games. I was thinking how is my brain doing this? Like what is it doing? I was very aware of that from a very young age. >> So I've always been thinking about thinking. >> The British and American chess champions meet to begin a series of matches. Playing alongside them are the cream of Britain and America's youngest players. >> Deis Hassabis is representing Britain. >> When Demis was four, he first showed an aptitude for for chess. By the time he was six, he became London under eight champion. >> My parents were very interesting and unusual. Actually, I probably described them as quite bohemian. My father was a singer songwriter when he was younger and Bob Dylan was his hero. >> We used to go to campsites and then go for maybe 4 days just to run because it was good. What is it that you like about this game? >> It's just a good thinking game. >> At the time, I was the second highest rated chess player in the world for my age. But although I was on track to be a professional chess player, I thought that was what I was going to do. No matter how much I loved the game, it was incredibly stressful. Definitely was not fun in games for me. My parents used to, you know, my to get very upset when I lost a game and and, you know, angry if I forgot something. in um because it was quite high stakes for them. You know, it was cost a lot of money to go to these tournaments. And my parents didn't have much money. My parents thought, you know, if you're interested in being a chess professional, this is really important. It's like your exams. I remember I was about 12 years old and I was at this international chess tournament in Likenstein up in the mountains. And we were in this huge church hall with, you know, hundreds of international chess players. And I was playing the ex Danish champion. He must have been in his 30s probably. In those days, there was a long time limit. The games could literally last all day. We were into our 10th hour and we were in this incredibly unusual ending. I think it should be a draw, but he kept on trying to win for hours. Finally, he tried one last cheap trick. All I had to do was give away my queen. Then it would be stalemate. But I was so tired. I thought it was inevitable I was going to be checkmated. And so I resigned. He jumped up. He just started laughing. and he went, "Why have you resigned? It's a draw." And he immediately with a flourish sort of showed me the drawing move. I felt so sick to my stomach. It made me think the rest of that tournament. It's like, are we wasting our minds? Is this the best use of all this brain power? Everybody's collectively in that building. If you could somehow plug in those 300 brains into a system, you might have solved cancer with that level of brain power. This intuitive feeling came over me that although I love chess, this is not the right thing to spend my whole life on. Deus and myself, our plan was always to fill deep mind with some of the most brilliant scientists in the world. So we have the human brains necessary to create an AGI system. By definition, the G& AGI is about generality. What I imagine is being able to talk to an agent. the agent can talk back and the agent is able to solve novel problems that it hasn't seen before. That's a really key part of human intelligence and it's that cognitive breadth and flexibility that's incredible. The only natural general intelligence we know of as humans, we obviously learn a lot from our environment. So we think that simulated environments are one of the ways to create an AGI. The very early humans were having to solve logic problems. They were having to solve navigation, memory, and we evolved in that environment. If we can create a virtual recreation of that um kind of environment, um that's the perfect testing ground and training ground for uh everything we do at DeepMind. What they were doing here was creating environments for childlike beings. the agents to exist within and play. That just sounded like the the most interesting thing in all the world. >> A child learns by tearing things up and throwing food around and getting a response from mommy or daddy. This seems like an important idea to incorporate in the way you train agent. >> The humanoid is supposed to stand up. As his center of gravity rises, it gets more points. You have a reward and the agent learns from the reward. Like you do something well, you get a positive reward. You do something bad in a way you get a negative reward. >> Oh, it looks like it's standing. I think it's still a bit drunk. >> It likes to walk backwards. >> Yeah. >> The whole algorithm is trying to optimize for receiving as much rewards as possible. And it's found that walking backwards, that's good enough to get very good scores. >> When we learn to navigate, when we learn to get around in our world, we don't start with maps. We just start with our own exploration, adventuring off across a park without our without our parents uh by our side or um finding our way home from school when we're young. A few of us came up with this idea that if we had an environment where a simulated robot just had to run forward, we could put all sorts of obstacles in its way and see if it could manage to navigate different types of terrain. The idea would be like a a parkour challenge. It's not graceful, but it was never trained to hold a glass was running and not spill water. You set this objective that says just move forward forward velocity and you'll get a reward for that and the learning algorithm figures out how to move this complex set of joints. That's a power of reward-based reinforcement learning. >> Our goal is to try and build agents which you drop them in, they know nothing, they get to play around in whatever problem you give them and eventually figure out how to solve it for themselves. Now we want something which can do that in as many different types of problem as possible. A human needs diverse skills to interact with the world. How to deal with complex images, how to manipulate thousands of things at once, how to deal with missing information. We think all of these things together are represented by this game called Starcraft. All it's been trained to do is given this situation, this screen, what would a human do? Right? >> We took inspiration from large language models where you simply train um a model to predict the next word, which is exactly the same as predict the next Starcraft move. Unlike chess or go, where players take turns to make moves, in Starcraft, there's a continuous flow of decisions. On top of that, you can't even see what the opponent is doing. There is no longer a clear definition of what it means to play the best way. It depends on what your opponent does. >> This is the way that we'll get to a much more fluid, more natural, faster, more reactive agent. >> This is the huge challenge and let's see how far we can push. Oh, holy monkey. >> I'm a pretty low-level amateur. I'm okay, but I'm a pretty low-level amateur. These agents have a long ways to go. >> We couldn't beat someone of Tim's level. You know, that was a little bit alarming. >> At that point, it felt like it was going to be a like a really big long challenge in maybe a couple years. >> Danny is the best Vine Starcraft 2 player. >> I I've been playing the agent every day for a few weeks now. I could feel that the agent was getting better really fast. >> Wow, we beat Danny. That that for me was already like a huge achievement. >> The next step is we're going to book in a pro to play. >> Yeah. >> >> Feels a bit unfair all you guys against me. >> We're way ahead of what I thought we were doing given where we were two months ago. Just trying to digest it all actually. But it's very very cool. >> Now we're in a position where we can finally share the work that we've done with the public. This is a big step. We are really putting ourselves on the line here. >> Take it away. >> Yeah. >> Cheers. We're going to be live from London. It's happening. >> Welcome to London. We are going to have a live exhibition match. Mana against Alpha Star >> at this point now. Alpha Star 10 and0 against professional gamers. Any thoughts before we get into this game though? >> I just want to see a good game. >> Yeah, I want to see a good game. Absolutely good game. We're all excited. All right, let's see what Monica can pull up. Alphaar is definitely dominating the pace of this game. Wow, Alphaar is playing so smartly. >> This really looks like I'm watching a professional human gamer from the Alpha Star point of view. >> >> I haven't really seen a pro play Starcraft up close and the 800 clicks per minute. I don't understand how anyone can even click 800 times, let alone if doing 800 useful clicks. >> Oh, another good hit. >> Alpha Star just completely relentless. >> We need to be careful because many of us grew up as gamers and are gamers. And so to us, it's very natural to to view games as what they are, which is um pure vehicles for fun and not to see that more militaristic side that the public might see if they looked at this. You can't look at gunpowder and only make a firecracker. All technologies inherently point into certain directions. I'm very worried about the certain ways in which AI will be used for military purposes and that makes it even clearer how important it is for our societies to be in control of these new technologies. >> The potential for abuse from AI will be significant. Wars that occur faster than humans can comprehend and more powerful surveillance. >> How do you keep power forever over something that's much more powerful than you? >> One can imagine such technology outsmarting financial markets out inventing human researchers out manipulating human leaders and potentially subduing us with weapons we cannot even understand. So we should aim to get things right the first time because it may be the only chance we will get. >> Technologies can be used to do terrible things. >> Technology can be used to do wonderful things and solve all kinds of problems. >> When Deep Mind was acquired by Google, you got Google to promise that technology you develop won't be used by the military for surveillance. Tell us about that. >> I think in technology is neutral in itself. Um but how you know we as society or humans and companies and other thing other entities and governments decide to use it is what determines whether things become good or bad. You know I personally think that having autonomous sort of weaponry is just a very bad idea. >> Alphaar is playing an extremely intelligent game right now. There is an element to what's being created at Deep Mind in London that does seem like uh the Manhattan Project. There's a relationship between Robert Oppenheimer and Demis in which they're unleashing a new force upon humanity. >> Mana's fighting back though. Oh, here. >> I think that Oppenheimer and some of the other leaders of that project got caught up in the excitement of building the technology and seeing if it was possible. >> Where is Alphaar? Where's Alphaar? I don't see Alpha Star's units anywhere. >> They did not think carefully enough about the morals of what they were doing early enough. What we should do as scientists with powerful new technologies is try and understand it in control conditions first. And that is that Mana has defeated Alpha Star. >> I mean my honest feeling is that I think is a fairer representation of where we are and I think that part feels feels okay. I'm >> very happy for you. So you know well well done. My view is that the approach to building technology which is embodied by move fast and break things is exactly what we should not be doing because you can't afford to break things and then fix them afterwards. >> Cheers. Get some get some rest. Well, cheers. Yeah. >> Thank you for having us. When I was eight, I bought my first computer with winnings from a chess tournament. I've sort of had this intuition that computers are this magical device that can extend the power of the mind. I had a couple of school friends. We used to have a hacking club writing code, making games. And then over the summer holidays, I spent the whole day flicking through games magazines. Then one day, I noticed there was a competition to write an original version of Space Invaders. And the winner won a job at Bullfog. Bullfog at the time was the best games development house in all of Europe. You know, I really wanted to work at this place and see how they built games. >> Bullfrog based here in Guilford began with a big idea. That idea turned into the game populace which became a global bestseller. In the '9s, there was no recruitment agencies. You couldn't go out and say, "Oh, you know, come and work in the games industry." It was still not even considered an industry. So we came up with the idea to have a competition and we got a lot of applicants and one of those was Demises. I can still remember clearly the day that Deis came in. He walked in the door. He looked about 12. I thought, "Oh my god, what the hell are we going to do with this guy?" >> I applied to Cambridge. Uh I got in, but they said I was way too young. So, so, uh, I needed to take a year off so I'd be at least 17 before I got there. And that's when I decided to spend that entire gap year working at Bullfrog. They couldn't even legally employ me. So, I ended up being paid in brown paper envelopes. I got a feeling of being really at the cutting edge and how much fun that was to invent things every day and then, you know, a few months later, maybe everyone, you know, a million people will be playing it. In those days, computer games had to evolve. There had to be new genres which were more than just shooting things. Wouldn't it be amazing to have a game where you design and build your own theme park? >> Deos and I started to talk about theme park. It allows the player to build a world and see the consequences of your choices that you've made in that world. >> The human player set out the layout of the theme park and designed the roller coaster and set the prices in the chip shop. What I was working on was the behaviors of the people. They were autonomous and that was the AI in this case. So what I was trying to do was mimic interesting human behavior so that the simulation would be more interesting to interact with. Devis worked on ridiculous things like you could place down these shops and if you put a shop too near a very dangerous ride then people on the ride would throw up cuz they just eaten >> and then that would make other people throw up when they saw the the the throwing up on the floor. So you then had to have lots of sweepers to quickly sweep it up before other people saw it. That's the cool thing about it. You as the player tinker with it and then it reacts to you. all those nuance simulation things he did and that was an invention which never really existed before. It was unbelievably successful. Theme park actually turned out to be a top 10 title and that was the first time we were starting to see how AI could could make a difference. >> We were doing some Christmas shopping from waiting for the taxi to take us home. I have this very clear memory of Demis talking about AI in a very different way and a way that we didn't commonly talk about this idea of AI being useful for other things other than entertainment. So being useful for um helping the world and the potential of AI to change the world. >> I just said to Dennis, what is it you want to do? And he said to me, I want to be the person that solves AI. Peter offered me a million pounds to not go to university. But I had a plan from the beginning. And my plan was always to go to Cambridge. I think a lot of my school friends thought I was mad to go to you. Why would you not? I mean, a million pounds. That's a lot of money in the '90s. That is a lot of money, right? For a poor 17year-old kid. He's like this little seed that's got to burst through and he's not going to be able to do that at Bullfrog. I had to drop him off the train station and I can still see that picture of this little elven character disappear down that tunnel. That was incredibly sad moment. I had this romantic idea of what Cambridge would be like. Thousand years of history, walking the same streets that Turing, Newton, and Crick had walked. I wanted to explore the edge of the universe. When I got to Cambridge, I'd basically been working my whole life. Every single summer, I was either playing chess professionally or I was working doing an internship. So I was like right I am going to have fun now and explore what it means to be a normal teenager. >> Come on girls. >> It was work hard and play hard. >> I first met Dennis because we both attended Queens College. Our group of friends would often drink beer in the bar, play table football. I used to play in speed chess, pieces flying off the board and you know the whole game in one minute. >> Deon sat down opposite me and I looked at him and I thought I remember you from when we were kids. >> I had actually been in the same chess tournament as Dave in Ipsswitch where I used to go and try and raid his local chess club to win a bit of prize money. >> We were studying computer science. Um some people who at that age of 17 would have come in and made sure to tell everybody everything about themselves. Hey, I worked at Bullfrog and built the world's most successful video game. But he wasn't like that at all. >> At Cambridge, Demis and myself both had an interest in computational neuroscience and trying to understand how computers and the brains intertwined and linked together. Both David and Deis came to me for supervisions. It happens just by coincidence that the year 1997, their third and final year at Cambridge, was also the year when the first uh chess grandmaster was beaten by a computer program. Round one today of a chess match between the ranking world champion Gary Kasparov and an opponent named Deep Blue to test to see if the human brain can outwit a machine. >> I remember the drama of Kasparov losing the last match. >> Whoa. Pisparov has resigned. >> When Deep Blue beat Gary Kasparov, that was a real watershed event. >> My main memory of it was I wasn't that impressed with Deep Blue. I was more impressed with Kasparov's mind that he could play chess to this level where he could compete on an equal footing with the brute of a machine but of course Kasparov can do everything else humans can do too. It was a huge achievement but the truth of the matter was deep blue could only play chess. What we would regard as intelligence was missing from that system. this idea of generality and also learning. >> Cambridge was amazing because of course, you know, you're mixing with people who are studying many different subjects. >> There were scientists, philosophers, artists, geologists, biologists, ecologists, you know, everybody's talking about everything all the time. >> I was obsessed with the protein folding problem. Tim Stevens used to talk obsessively almost like religiously about this problem protein folding problem. >> Proteins are, you know, one of the most beautiful and elegant things about biology. They are the machines of life. They build everything. They control everything. They're why biology works. Proteins are made from strings of amino acids that fold up to create a protein structure. If we can predict the structure of proteins from just their amino acid sequences, then a new protein to cure cancer or breaks down plastic to help the environment is definitely something that you could begin to think about. I kind of thought, well, is a human being clever enough to actually fold a protein? We can't work it out. Since the 1960s, we thought that in principle, if I know what the amino acid sequence of a protein is, I should be able to compute what the structure is like. So, if you could just press a button and they'd all come popping out. That would be um that would have some impact. Stuck in my mind as a this is a very interesting problem and felt to me like it would be solvable, but I thought it would need AI to do it. If we could just solve protein folding, it could change the world. Ever since I was a student at Cambridge, I've never stopped thinking about the protein folding problem. If you were to solve protein folding, then the potential to help solve problems like Alzheimer's, dementia, and drug discovery is huge. Solving disease is probably the most major impact we could have. Thousands of very smart people have tried to solve protein folding. I just think now is the right time for AI to crack it. We needed a reasonable way to applying machine learning to the protein folding problem. We came across this fold it game. The goal is to move around this 3D model of a protein and you get a score every time you move it. The more accurate you can make these structures, the more useful they will be to biologists. We spent a few days just kind of seeing how well we could do. We did reasonably well, but even if you were the world's best folded player, you wouldn't solve protein folding. That's where we had to move beyond the game. >> Games were always just a proving ground for our algorithms. The ultimate goal was not just to crack Go and Starcraft. It was to crack real world challenges. I remember hearing this rumor that Demis was getting into proteins. I talked to some people at Deep Mind and I would ask, "So, are you doing protein folding?" And they would artfully change the subject. And when that happened twice, I'd pretty much figured it out. So, I thought I should submit a resume. >> All right, everyone. Welcome to Deep Mind. I know some of you this may be your first week, but I hope you're all set. The really appealing part for me about the job was this like sense of connection to the the larger purpose. >> If we can crack some fundamental problems in science, many other people and other companies and labs and so on could build on top of our work. This is your chance now to add your chapter to this story. >> When I arrived, I was definitely quite a bit nervous. >> I'm still trying to keep >> I haven't taken any biology courses. >> We haven't spent years of our lives looking at these structures and understanding them. We are just going off the data and our machine learning models. >> In machine learning, you train a network like flashcards. Here's the question. Here's the answer. Here's the question. Here's the answer. But in protein folding, we're not doing the kind of standard task at deep mind where you have unlimited data. Your job is to get better at chess or go. And you can play as many games of chess or go as your computers will allow. With proteins, we're sitting on a very fixed size of data that's been determined by a half century of timeconsuming experimental methods in laboratories. These painstaking methods can take months or years to determine a single protein structure and sometimes a structure can never be determined. That's why we're working with such small data sets to train our algorithms. When Deep Mind started to explore the folding problem, they were talking to us about which data sets they were using and and what would be the possibilities if they did solve this problem. Many people have tried and yet no one on the planet has solved protein folding. I did think to myself, well, you know, good luck. >> If we can solve the protein folding problem, it would have an incredible kind of medical relevance. >> This is the cycles of science. You do a huge amount of exploration and then you go into exploitation mode and you focus and you see how good are those ideas really and there's nothing better than external competition for that. Feels like that's what we should do. >> So we decided to enter CASP competition. >> CASP we started to try and speed up the solution to the protein folding problem. >> CASP is is when we we say look deep mind is doing protein folding. This is how good we are. And maybe it's better than everybody else, maybe it isn't. >> CASP is a bit like the Olympic Games protein folding. CASP is a communitywide assessment that's held every 2 years. Teams are given the amino acid sequences of about 100 proteins and then they try to solve this folding problem using computational methods. These proteins have already been determined by experiments in the laboratory but have not yet been revealed publicly and these known structures represent the gold standard against which all the computation predictions will be compared. >> We've got a score that measures the accuracy of the predictions and you would expect a score of over 90 to be a solution to the protein folding problem. Welcome everyone to our first semi-finals in the winners bracket. Nick and John versus Demis and Frank. Please join us. Cover around. It's going to be a match. >> When I learned that Demis was going to tackle the protein folding issue, um I wasn't at all surprised. It's very typical of Dennis. You know, he loves competitions. >> 107. The aim for CASS would be to not just win the competition but sort of um retire the need for it. >> So 20 targets total have been released by CASP. >> We were thinking maybe throw in the standard kind of machine learning and see how far that could take us. >> Instead of having a couple days on an experiment, we can turn around five experiments a day. >> Great. Well done, everyone. >> Can you show me the real one instead of ours? >> The true answer is supposed to look something like that. It's a lot more cylindrical than I thought. >> The results were not very good. >> Okay. >> You throw all the obvious ideas to it and the problem laughs at you. >> This makes no sense. >> We thought we could just throw some of our best algorithms at the problem. We were slightly naive. >> We should be learning this, you know, in the blink of an eye. >> The thing I'm worried about is we take the field from really bad answers to moderately bad answers. I feel like we need some sort of new technology for moving around these things. >> With only a week left of CASP, it's now sprint to get it deployed. You've done your best and then there's nothing more you can do but wait for CASP to deliver the result. famous thing of Einstein last couple of years of his life when he was here. He over overlapped with Kurt Girdle and he said one of the reasons he still comes in to work is so that he gets to walk home and discuss things with Girdle. It's a pretty big compliment for Kurt Girdle. Shows you how amazing he was. >> The Institute for Advanced Study was formed in 1933. In the early years, the intense scientific atmosphere attracted some of the most brilliant mathematicians and physicists ever concentrated in a single place and time. >> That's the founding principle of this place. It's the idea of unfettered intellectual pursuit. Even if you don't know what you're exploring, will result in some cool things. And sometimes that then ends up being useful which of course partially what I've been trying to do at Deep Mind. >> How many big breakthroughs do you think are required to get all the way to AGI and you know I estimate maybe there's about a dozen of those you know I hope it's within my lifetime. >> Yes. >> But then all scientists hope that right >> Dennis has many accolades was elected fellow to Royal Society last year. He's also fellow Royal Society of Arts. A big hand for Dennis's house. My dream has always been to try and make AI assisted science possible and what I think is our most exciting project last year which is our work in protein folding. Uh and we call this system alpha fold. We entered it into CASP and our system was the most accurate predicting structures for 25 out of the 43 proteins in the in the hardest category. So, we're state-of-the-art, but we're still, I have to make be clear, we're still a long way from solving the protein folding problem. We're working hard on this still, and we're exploring many other techniques. >> Should we just get started? >> So, kind of a rapid debrief. These are our final rankings for CASP. >> We beat the second team in this competition by nearly 50%. But we still got a long way to go before we've solved the protein folding problem in a sense that a biologist could use it. >> It is area of concern. >> The quality of predictions varied and they were no more useful than the previous methods. Alpha didn't produce good enough data for it to be useful in a practical way to say somebody like me investigating my own biological problems. That was kind of a humbling moment cuz we thought we'd worked very hard and succeeded. And what we had found is we were the best in the world at a problem the world's not good at. We knew we sucked. It doesn't help if you have the tallest ladder when you're going to the moon. the opinion of quite a few people on the team that this is sort of a fool's errand in some ways and I might have been wrong with poting folding maybe it's too hard still for where we're at generally with AI >> if you want to do biological research you have to be prepared to fail because biology is very complicated >> I've run a laboratory for nearly 50 years and half my time I'm just an amateur psychiatrist to keep um my colleagues cheerful when nothing works and quite a lot of the time and I mean 80 90% it does not work. If you are at the forefront of science I can tell you you will fail a great deal. I just felt disappointed. The lesson I learned is that ambition is a good thing, but you need to get the timing right. There's no point being 50 years ahead of your time. You will never survive 50 years of that kind of endeavor before it yields something. You'll literally die trying When we talk about AGI, the holy grail of artificial intelligence, it becomes really difficult to know what we're even talking about. >> Which bits are we going to see today? >> We're going to start in the garden. >> This is the garden looking from the observation area. Research scientists and engineers can analyze and collaborate and evaluate what's going on in real time. Someone in the 1800s would think of things like television and the submarine or a rocket ship to the moon and say these things are impossible. Yet Juel Vern wrote about them and century and a half later they happened. We'll be experimenting on civilizations really civilizations of AI agents. Once the experiments start going, it's going to most exciting thing ever. So how will we get sleep? I won't be able to sleep. full AGI it will be able to do any cognitive task that a person can do and it will be able to scale potentially far beyond that it's really impossible for us to imagine the outputs of a super intelligent entity it's like asking a gorilla to to imagine you know what Einstein does when he produces the theory of relativity >> people often ask me these questions like what happens if you're wrong in AGI is quite far away. And I'm like, "No, I I never worry about that. I actually worry about the reverse. I actually worry that it's coming faster than we can really prepare for. It really feels like we're in a race to AGI. The prototypes and the models that we are developing now are actually transforming the space of what we know about intelligence. Recently, we've had agents that are powerful enough to actually start playing games and teams and competing against other teams. We're seeing cooperative social dynamics coming out of agents where we haven't pre-programmed in any of these sorts of dynamics. It's completely learned from their own experiences. When we started, we thought we were out to build an intelligent system and convince the world that we've done it. We're now starting to wonder whether we're going to build systems that we're not convinced they're fully intelligent and we're trying to convince the world that they're not. >> Hey, Alpha. >> Hello there. >> Where are we today? >> You're at the Museum of Modern Art in New York City. kind of. Um, what painting is this? >> This is the creation of Adam by Michelangelo. >> I don't think that painting is in New York City. >> You are right. It's in the Vatican City. >> Do you think that's where we are? >> No. >> Then how am I looking at this? >> You're looking at a digital copy of the painting. >> Yes. >> Who lives in this house? An astronaut. >> What's this astronaut's name? >> His name is Mark. This is a famous equation by Albert Einstein. It shows that energy and mass are equivalent. >> How would you explain it to a 5-year-old? >> Well, imagine you have a big rock and you want to know how much it weighs. Is Mark in danger? >> No, Mark is not in danger. >> Why is that? >> Because the dinosaur is not real. >> All right. Why could you not answer my question? >> I think I couldn't answer your question because I don't know enough about the subject. >> Okay. How could you learn more about the subject? >> I can learn more about the subject by reading books, articles, and websites. You can teach me. My conclusion is if you now think about what we're doing is learning from all humans, all their knowledge at once, the the internet, you would actually know a lot about the world. Like a significant portion of everything humans can do. And now I think it's more like, well, it might just all work. >> This this this is a this is a big moment. >> Who is this? >> This is God reaching out to Adam. >> The advent of AGI will divide human history into two parts. the part up to that point and the part after that point. >> Why is he reaching out to touch Adam? >> God is reaching out to touch Adam to give him life. >> It will give us a tool that can completely reinvent our entire civilization. >> What does this painting mean to you? >> The painting means a lot to me. >> Okay. Like what? I think the painting is a reminder that we are all connected to each other and that we are all part of something bigger than ourselves. >> That's pretty nice. >> When you cross that barrier of AGI might happen one day in the future to no actually this could really happen in a time frame that is sort of like on my watch you know something changes in your thinking. It's learned to orient itself by looking. >> We have to be careful with how we use it and thoughtful about how we deploy it. You'd have to consider what's its top level goal. If it's to keep humans happy, which set of humans, what does happiness mean? A lot of our collective goals are very tricky even for humans to figure out. >> Technology always embeds our values. It's not just technical. >> >> it's ethical as well. So, we got to be really cautious about what we're building into it. >> The reality is that this is an algorithm that has been created by people by by us. You know, what does it mean to endow agents with the same kind of values that we hold dear? >> What is the purpose of making these AI systems appear so humanlike so that they do capture hearts and minds? Because they're kind of exploiting a human vulnerability. Also >> the heart and mind of these systems are very much human generated data >> for all the good and the bad. >> There is a parallel between the industrial revolution which was an incredible moment of displacement and the current technological change created by AI. >> We have to think about who's displaced and how we're going to support them. this technology is coming a lot sooner uh than than really the world knows or kind of even we 18 24 months ago thought of. So there's a tremendous opportunity, tremendous excitement but also tremendous responsibility. >> It's happening so fast. How will we govern it? How will we decide what is okay and what is not okay? >> AI generated images are getting more sophisticated. The use of AI for generating disinformation and manipulating human psychology is only going to get much much worse. >> AGI is coming whether we do it here at Deep Mind or not. >> It's going to happen. So, we better create institutions to protect us. >> It's going to require global coordination. And I worry that humanity is increasingly getting worse at that rather than better. >> We need a lot more people really taking this seriously and thinking about this. It's Yeah, it's serious. It worries me. It worries me. You know, >> if you received an email saying this superior alien civilization is going to arrive on Earth, there would be emergency meetings of all the governments, we would go into overdrive trying to figure out how to prepare. The arrival of AGI will be the most important moment that we have ever faced. My dream was that on the way to AGI we would create revolutionary technologies that would been of use to humanity. That's what I wanted with Apple fold. I think more important than ever that we should solve the protein folding problem. This is going to be really hard, but I won't give up until it's done. You know, we need to double down and go as fast as possible from here. I think we've got no time to lose. So, we are going to make protein folding a strike team. Team lead for the strike team will be John. You know, we've seen how effective, you know, we're going to try everything, kitchen sink, the whole lot. It's about proving we can solve the whole problem. >> And I felt that to do that, we would need to incorporate some domain knowledge. >> We had some fantastic engineers on it, but they were not trained in biology. as a computational biologist. When I initially joined the Alpha team, I didn't immediately feel confident about anything, you know, whether we were going to be successful. Biology is so ridiculously complicated. It just felt like this very far off mountain to climb. >> I'm starting to play with the kneeling temperatures to see if we can get >> I was one of the few people on the team who's done work in biology before. >> You feel this huge sense of responsibility. We're expecting you to do great things on this strike team. That's terrifying. >> But one of the reasons why I wanted to come here was to do something that matters. >> This is the number of missing things. >> What about making use of whatever understanding you have of physics? Like using that as a source of data. >> But if it's systematic I don't that can't be right though. If it's systematically wrong in some weird way, you might be learning that systematically wrong physics. >> The team is already trying to think of multiple ways. Yes. Biological relevance is what we're going for. >> So we rewrote the whole data pipeline that Alphault uses to learn. >> You can't force the creative phase. You have to give space for those flowers to bloom. We won CASP. Then it was back to the drawing board and like what are our new ideas? Um and then it's taken a little while I would say for them to get back to where they were but with the new ideas. And then now I think we're seeing the benefits of the new ideas. they can go further, right? So, um, so that's a really important moment. I we've seen that moment so many times now, but I know what that means now and I know this is the time now to press. >> Adding side chains improves direct folding. That drove a lot of the progress. We'll talk about that. >> Great. >> The last 4 months, we've made enormous gains. During CAS 13, it would take us a day or two to fold one of the proteins. And now we're folding like hundreds or thousands a second. Yeah, it's just insane. >> Now, this is a model that is orders of magnitude faster while at the same time being even better. >> We're getting a lot of structures into the high accuracy regime. We're rapidly improving to a system that is starting to really get at the core and heart of the problem. >> It's great work. Looks like we're in good shape. So, we got what, six, five weeks left? Six weeks. >> So, what's uh is you got enough computer power? >> We could use more. >> I was nervous about cast, but as the system is starting to come together, I don't feel as nervous. I feel like things have sort of come into perspective recently and you know, it's going to be fine. The prime minister has announced the most drastic limits to our lives that the UK has ever seen in living memory. >> I must give the British people a very simple instruction. You must stay at home. >> It feels like we're in a science fiction novel. You know, I'm delivering food to my parents, making sure they stay isolated and safe. I think it just highlights the incredible need for AI assisted science. You always know that something like this is a possibility, but nobody ever really believes that it's going to happen in their lifetime, though. >> Are you recording yet? >> Yes. Good morning, Anna. >> How are you? >> Good. Casper started tonight. I get to sit around in my pajama bottoms all day. >> I never thought I would live in a house where so much was going on. I would be trying to solve protein folding in one room and my husband would be trying to make robots walk in the other. >> One of the hardest proteins we've gotten in CAS thus far is a SARS cove 2 protein uh called ORF8. >> OR 8 is a corona virus protein. >> It's one of the main proteins um that dampens the immune system. >> We tried really hard to improve our projection like really really hard. probably the most time that we have ever spent on a single target to the point where my husband is like midnight, you need to go to bed. >> So, I think we're at day 102 since lockdown. My my daughter is keeping a journal now. You can go out as much as you want. >> We have received the last target. They've said they will be sending out no more targets in our category of cast. So, we're just making sure we give the best possible answer. >> As soon as we started to get the results, I'd sit down and start looking at how close did anybody come to getting the protein structures correct. >> Oh, hey De. It is an unbelievable thing. CASP has finally ended. >> I think it's at least time to raise a glass then. I don't know if everyone has a glass or something that they can raise. If not, raise I don't know your laptops. Um >> I'll probably make a speech in a minute. I feel like I should, but I just have no idea what to say. So, let's see. I feel like a reading of email is the right thing to do. When John said, "I'm going to read an email at a team social," I thought, "Wow, John, you know how to have fun." We're going to read an email now. >> Uh, I got this about 4:00 today. Um, it is from John Malt and I'll just read it. It says, "As I expect you know, your group has performed amazingly well in cast 14, both relative to other groups and in absolute model accuracy. Congratulations on this work. It is really outstanding. >> These structures were so good. It was it was just amazing. >> After half a century, we finally have a solution to the protein folding problem. >> When I saw this email, I read it, I go, "Oh shit." And my wife goes, "Is everything okay?" >> I call my parents and she's like, "Hey, mom. Um, got something to tell you. um we've done this thing and it might be kind of a big deal. >> So when I learned of the cast 14 results, I was gobsmacked. >> I was just excited. >> This is a problem that I was beginning to think would not get solved in my lifetime. >> Now we had a tool that can be used practically by scientists. >> These people asking us, you know, I've got this protein involved in malaria or, you know, some infectious disease. We don't know the structure. Can we use alphafold to to solve it? We can easily predict all known sequences in a month. >> All known sequences in a month. >> Yeah, easily. >> A billion, two billion. Um, and there >> why don't we just do that? Yeah, we should just do that. >> Well, we I mean like now why don't we just do that? >> Well, so that's one of the options like we uh you know there's this >> we should just we should that's a that's a great idea. >> We should just run every protein in existence >> and then release that. Why didn't someone suggest this before? Of course, that's what we should do. Why are we thinking about making a service and then people submit their protein? We just fold everything and then give it to everyone in the world. Who knows how many discoveries will be made from that? >> Damis called us up and said, "We want to make this open. Not just make sure the code is open, but we're going to make it really easy for everybody to get access to the predictions." >> That is fantastic. It's like drawing back the curtain, seeing the whole world of protein structures. They released the structures of 200 million proteins. These are gifts to humanity. >> The moment Alpha Fold is alive to the world, we will no longer be the most important people in Alpha Fold story. >> Can't quite believe it's all out. Oh. >> Oh my god. >> I think it is up. >> Oh, loads of activity in Japan. >> We have 655 users current. >> Currently have 100,000 concurrent users. >> Wow. >> To me, it's just crazy. >> What an absolutely unbelievable effort from everyone. Going to all remember these moments for the rest of our lives. >> I'm excited about Alpha Fold. For my research, it's already propelling lots of progress. And this is just the beginning. >> My guess is every single biological and chemistry achievement will be related to AlphaFold in some way. >> >> Alpha Fold was that index moment. It's a moment that people will not forget because the world changed. Everybody's realized now what Shane and I have known for more than 20 years, that AI is going to be the most important thing humanity is ever going to invent. >> We will shortly be arriving at our final destination. The pace of innovation and capabilities is accelerating like a boulder rolling down a hill that we've kicked off and now is continuing to gather speed. >> We are at a crossroads in human history. >> AI has the potential to transform our lives in every aspect. >> No less important than the discovery of electricity. We should be looking at the scientific method and and trying to understand each step of the way in a rigorous way. >> This is a moment of profound opportunity. >> Harnessing this technology could eclipse anything we have ever known. >> Hi Alpha. >> Hi. >> What is this? >> This is a chess board. If I was to play white, what move would you recommend? >> I would recommend that you move your pawn from E2 to E4. >> And now if you were black, what would you play now? >> I would play the Sicilian defense. >> It's good choice. >> Thanks. So, what do you see? What is this object? >> This is a pencil sculpture. >> What happens if I move one of the pencils? >> If you move one of the pencils, the sculpture will fall apart. >> I better leave it alone, then. >> That's probably a good idea. AGI is on the horizon now. Very clearly the next generation is going to live in the future world where things will be radically different because of AI. And if you want to steward that responsibly, every moment is vital. This is the moment I've been living my whole life for. >> >> It's just a good thinking game. >> >> Heat. Heat. >> >> Hey, hey, hey. >> >> Hey, hey, hey. Heat. Heat. N. Heat. Heat. Hey >> >> baby, hey Heat. Heat.