Neuralink Show and Tell 30 ноября 2022 г. Официальная осенняя выставка и рассказ Neuralink 2022, где Маск и команда представляют прогресс интерфейса мозг-компьютер и планы на человеческие испытания.
Официальная осенняя выставка и рассказ Neuralink 2022, где Маск и команда представляют прогресс интерфейса мозг-компьютер и планы на человеческие испытания.
Расшифровка Sa. Welcome to the Neuralink show and tell. So we've got an amazing amount of new developments to share with you that I think are incredibly exciting, as well as tell you about the future of what we're planning to do here. Now, this is meant to be a technical podcast, sort of like I'm going to provide an overall summary and then we're going to have a number of members of the Neuralink team come in and give a deep technical overview of the various areas. So, yeah, so let me move forward with the overall summary. Now. Some of the things I'm going to say are things you've. Well, if you've been following Neuralink, you've already heard before, but for a lot of people out there, they've no idea what Neuralink does. And so I will be a little bit repetitive of things you may already know, but that others do not. So the overarching goal of Neuralink is to create ultimately a whole brain interface, so a generalized input output to device that in the long term, literally could interface with every aspect of your brain, and in the short term can interface with any given section of your brain and solve a tremendous number of things that cause debilitating issues for people. So, you know, so our long term is like, I mean, I'll talk a little bit about a long term goal. It's going to sound a little esoteric, but it's the. It was actually the sort of my prime motivation, which was, you know, kind of, what, what do we do about AI? Like, what do we do about artificial general intelligence? If we have digital superintelligence that's just much smarter than any human, how do we mitigate that risk? At a species level, how do we mitigate that risk? And then even in a benign scenario where the AI is very, very benevolent, then how do we even go along for the.
Dan (Neuralink)
Go along for the ride?
How do we participate? And the conclusion, the thing that, the biggest limitation in going along for the ride and in aligning AI, I think, is the bandwidth, how quickly you can interact with the computer. So we are all already cyborgs in a way, in that you're. Your phone and your computer are extensions of yourself. And if you, I'm sure you found, like, if you leave your phone behind, you end up tapping your pockets. And it's like having missing limb syndrome, like where you know the phone is. It is leaving your phone behind is kind of like a missing limb. At this point, you're so used to interfacing with it, you're so used to being a de facto cyborg, but. So what's the limitation on a phone or a laptop? Limitation is the rate at which you can receive and send information, especially the speed with which you can send information. So if you're interacting with a phone, it's limited by the speed at which you can move your thumbs or speed at which you can talk into your phone. This is an extremely low data rate. Maybe it's like 10, optimistically 100 bits per second, but a computer can communicate at gigabits terabits per second. So this is the fundamental limitation that I think we need to address to mitigate the long term risk of artificial intelligence and also just go along for the ride. And. Yeah, so, but like I said, that's an esoteric explanation that I think will appeal to a niche audience, some of whom may be here. But. And that's a very difficult problem. So even if we do not succeed with that problem, I think we are confident at this point that we will succeed at solving many brain injury issues, spine injury issues along the way. So. Yeah, so anyways, so actually we have Justin Roiland in the audience. Hi, Justin. So it's a little Rick and Morty reference here, the great Rick and Morty episode about intelligence enhancement of your dog and what's the worst that can happen? So anyway, Rick and Morty, I recommend it. So for. So you want to be able to read the signals from the brain, you want to be able to write the
you want to be able to ultimately do that for the entire brain and then also extend that to communicating to the rest of your nervous system. If there's a, if you have sort of a severed spinal cord or neck. So now this video is now 18 months old. So this is Pager, who is playing monkey mind pong. So this is Pager has a neural link implant in this video. And the thing that's interesting is that you can't even see the neural implant. So we've miniaturized the neural implant to the point where it matches the thickness of the skull that is removed. So essentially it's sort of like having an Apple watch or a Fitbit replacing a piece of skull with like a, you know, a smartwatch, for lack of a better analogy. So
You really can't. He looks pretty, is normal. And I think that's pretty important. If you have a neuralink device, like I could have a neuralink device implanted right now and you wouldn't, you wouldn't even know. I mean, hypothetically, Maybe One of these demos.
one of these demos. I will. Yeah. So, yeah. Anyway, so here's. First of all, it's kind of wild. Hey, monkeys can play Pong. Like they can actually play Pong if you give them a joystick. So Pedro first learned to play Pong with a joystick. So I'm like, that was novel. It's like, I didn't know monkeys could play Pong, but they can. And then, so we first trained Pedro to play Pong with a joystick. Then we took the joystick away and have the neural link. And now this is. He's playing Telepath. It's a telepathic video games, essentially. So what we've been doing since then is we've been on the very difficult journey from prototype to product. And I've often said that prototypes are easy, production is hard. It's really, I'd say 100 to 1,000 times harder to go from a prototype to a device that is safe, reliable, works under a wide range of circumstances, is affordable, and done at scale, insanely difficult. I mean, there's an old saying that it's 1% inspiration, 99% perspiration, but I think it might be 99%, 99.9% perspiration. The best example I could give of an idea being easy, but the execution being hard is going to the moon. The idea of going to the moon, easy, going to the moon, very hard. And we've been working hard to be ready for our first human. And obviously we want to be extremely careful and certain that it will work well before putting a device in a human. But we've submitted, I think, most of our paperwork to the fda and we think probably in about six months we should be able to have a first neural link in a human. But as I said, we do everything we possibly can to test the devices before not even going into a human, before even going into an animal. So we do bench top testing. We do accelerate accelerated life testing. We have a fake brain simulator that has the texture and it's like emulating a brain, but it's sort of rubber. And so any before we would even think of putting a device in an animal, we do everything we possibly can with rigorous bench top testing. So we're not cavalier in putting devices into animals. We're extremely careful. And we always want the device, whenever we do the implant, if it's in a sheep or a pig or monkey, to be confirmatory, not exploratory. So that we've done everything we possibly can with bench Top testing. And only then would we consider putting a device in an animal. And yeah, we'll actually show you a demo later today in a few hours, really, of implanting in a brain proxy. And if anyone in the audience wants to volunteer, we have the robot right there. So, Lithien, since the pager demo, we've expanded to work with a troupe of six monkeys. We've actually upgraded pager. They do varied tasks, and we do everything possible to ensure that things are stable and replicable and that the device lasts for a long time without degradation. So, and what you're seeing there is it looks like the Matrix, but that's actually, that's a real output of neural signals. So that's not a simulation or just a screensaver or something. Those are actual neurons firing. That is what one of the readouts looks like. And here you can see Sake, it's one of other monkeys typing on a keyboard. Now, this is telepathic typing. So to be clear, this is the. He's not actually using a keyboard. He's moving the cursor with his mind to the highlighted key. Now, technically, we can't actually spell, and so I don't want to oversell this thing because that's. That's the next version.
But what's really cool here is Sakethemonkey is moving the mouse cursor using just his mind, moving the cursor around to the highlighted key and then spelling out what we. What we want, whatever we want to spell. But. And then. So this, this is something that could be used for somebody who's, say, quadriplegic or tetraplegic human. Even before we make the spinal cord stuff work, is being able to control a mouse cursor, control a phone. And we're confident that someone who has basically no other interface, the outside world, would be able to control their phone better than someone who has working hands. I mentioned upgradability. Upgradability is very important because our first production device will be much like an iPhone one. And I'm pretty sure you would not want an iPhone one stuck in your head if the iPhone 14 is available. So it's going to be
demonstrate full reversibility and upgradability. So you can remove a device and replace it with the latest version, or if it stopped working for any reason, replace it. That's a fundamental requirement for the device of neuralink. And I should say both Saki and pager were upgraded to our latest and greatest implants. So that's been really over a year and a half. Now that pager's had the first implant and then the upgraded implant, so this is a very good sign that it lasts for a long time with no observed ill effect. I think it's also important to show that Sake actually likes doing the demo and is not like strapped to the chair or anything. So it's. Yeah, so the monkeys actually enjoy doing the demos and they get the banana smoothie and it's kind of a fun game. So I guess what I'm trying to make is like, we care a great deal about animal welfare and I'm pretty sure, like, our monkeys are pretty happy, you know, so as you can see, quick decision maker on the fruit front. So for our, the first two applications we're going to aim for in humans are restoring vision. And I think this is like, notable in that even if someone has never had vision ever, like they were born blind, we believe we can still restore vision. So because the visual part of the visual part of the cortex is still there. So, yeah, even if they've never seen before, we're confident that they could, they could see. And then the, the other application being in the motor cortex, where we would initially enable someone who has no ability, almost no ability to operate their, their muscles, you know, sort of like a sort of Stephen Hawking type situation, and enable them to operate their phone faster than someone who has working hands. But then even, obviously, even better than that would be to bridge the connection. So take the signals from the motor cortex and let's say somebody's got a broken neck, then bridging those signals to neural link devices located in the spinal cord. So we're confident there are no physical limitations to enabling full body functionality. So, I mean, as miraculous as it may sound, we're confident that it is possible to restore full body functionality to someone who has a severed spinal cord. So, yeah. So, yeah. All right. And then I want to emphasize again that the primary purpose of this update is recruiting. A lot of times people think that they couldn't really work at neuralink because they don't know anything about biology or how brains work. And the thing that we really want to emphasize here is that you don't need to, because when you break down the skills that are needed to make neuralink work, it's actually many of the same skills that are required to make a smartwatch or modern phone work. So it's sort of, you know, software, batteries, radios, inductive charging, and, you know, as well as things that are specific to us like animal care and clinical and regulatory matters, obviously, machine Learning that phrase is used a lot. But we obviously need to interpret the signals from the brain, which is a biological neural net. And the best thing to interpret a biological neural net is a digital neural net. So this is. If there's one message I want to convey, it is that if you have expertise in creating advanced devices like watches and phones, computers, then your capabilities would be of great use in solving these important problems. That's more than anything the message I want to convey. So, see, yeah, so with that, I guess dj. So. DJ was on the founding team of Neuralink and just made immense contributions to the company as of many of the others who will present. But I want just to thank DJ for his immense contribution to Neuralink and.
Sam (Neuralink)
Thanks, Elon.
DJ (Neuralink)
When I moved from South Korea at age 13 and needed to learn a new language to communicate, I wondered whether there are better and more effective means of communicating my thoughts to the outside world. And watching Neo learn Kung Fu in the Matrix, I remember thinking, wow, I want to work on making that possible. And today I believe that this is a tractable engineering challenge, since everything about your intentions, your thoughts and your experiences are all in your brain, encoded as firing statistics of action potentials. If you're able to put electrodes in the right places with the right sensing and stimulation capabilities, this and many other applications that Elon talked about possible, and we can help a lot of people. I'm incredibly excited to be working on this ambitious, yet important mission to make that future a reality here at Neuralink. And I'm also incredibly honored to be working with some of the brilliant colleagues, scientists and engineers across many engineering disciplines to work on this intersection of biology and technology. You'll hear from several of them today to learn about the breadth of technical challenges we face and our progress in the last year. And I think you'll find that for most of these challenges, as Elon mentioned, you don't need a prior understanding of how the brain works, and that a lot of what we do is applying engineering first principles to biology. So how do you create a high bandwidth generalized interface to the brain? From day one, we focused on a set of foundational technologies that are safe, scalable, and capable of accessing all areas of the brain. These three axes, safety, scalability, and access to brain regions, really form the basis for how we engineer products here at neuralink. Safety, because we want to make our devices, as well as the installation as safe as possible so that we can drive the adoption of this technology and scalability, because as we make our devices Safer and more useful. More people will want it. And with scale, we also want to make it more affordable and access to brain regions so that we can expand the functionalities of our technologies. So our first steps along these dimensions for our device is what we call the N1 implant. It's a size of about a quarter and it has over 1000 channels that are capable of recording and stimulating. It's microfabricated on a flexible thin film arrays that we call threads. It's fully implantable and wireless, so no wires. And after the surgery, the implant is under the skin and it is invisible. It also has a battery that you can charge wirelessly and you can use it at home. So similarly, for implanting our device safely into the brain, we built a surgical robot that we call the R1 robot. It's capable of maneuvering these tiny threads that are only on the order of few red blood cells wide and inserting them reliably into a moving brain while avoiding vasculature. It's quite good at doing this reliably. And in fact, because we've never shown an end to end insertion of a robot in action, we're going to do a live demo of the robot doing surgery in our brain proxy. So who wants to see some insertions? So here it is. That's our R1 robot with our patient alpha, who is lying comfortably on the patient bed. This is what we call the targeting view. So what you're seeing is this is a picture of our brain proxy. And the pink represents the cortical surface that we want to insert our electrodes into. And the black represents the vasculatures that we want to avoid. And what you're seeing is these hash mark with numbers that represents where we intend to put each of our threads. So should we see some insertions? So this is another view real quick. On the left is the view of the insertion area. And on the right, what the robot's gonna do is it's going to peel the array, the threads, one by one from its silicon backing and, and insert it into the targets that we predetermined in the targeting view.
DJ (Neuralink)
There you go, that's the first insertion. So we're going to see a couple more insertions. The whole process of inserting about 64 threads in our first product is going to be around 15 minutes for this robot. So, yeah, there's a second one that went in and we're going to do a third one. There you go. And then that's going to go in the background and we'll come back to it in the later part of the presentation. And as Elon mentioned, we've been working very hard to go from prototype to building product as part of this. One of the things that we did is to move our device manufacturing to a dedicated facility in Austin for scale up manufacturing. And what's important to highlight and is evident in this clip is that it's very typical for us to have our engineers who design also work on the physical manufacturing line to build and debug. And this has been extremely, extremely critical in reducing our iteration cycle time. And we've also scaled up our surgery so we now have a dedicated, our own or in fact a double or in Austin. And this is just a stepping stone before we eventually build our own neuralink clinic. So with this product, N1 and R1, our initial goal is to help people with paralysis from complete spinal cord injury regain their digital freedom by enabling them to use their devices as good as, if not better than they could before the injury. And as Elon mentioned, over the last year this has been the central focus of the company and we've been working very closely with the FDA to get approval and to launch our first inhuman clinical trial in the US hopefully in the next six months. So hopefully this gives you a good overview of our product. For the next hour, we're going to go through a deep technical dive on these topics to tell you about our technical challenges, share some of our progress and preview what's coming next. So with that, over to NIR from my team who's going to talk to you about neural decoding.
Julian (Neuralink)
Thank you DJ
My name is Niravan Chen and I'm the head of Brain Interfaces Applications. Our goal is to enable someone with paralysis control a computer as well as me. Or even better, we'd like to provide fast and accurate control with all the functionality of computers that works anytime, any, anywhere. So I'm very excited to show you how we are using the N1 device with our software and algorithms to achieve this. Last year we shared with you a video of Pedro the Monkey controlling computer cursor with his brain. So how do we do that? Just a brief reminder. First, we record his neural activity from the motor cortex. Using the N1 device. We can record from over thousands of channels while he's playing with the joystick. Then we can train a neural net that predicts the cursor velocity from the patterns of his neural activity. With this decoder, he can then control a cursor just by thinking about it without even moving the joystick. He can play with this decoder a variety of games. Also a grid task where he's moving the white dot towards the yellow target. Every time he gets one, he receives a drop of his favorite smoothie. And he chooses to play this game every day. Here you can see his performance from early 2021, around the time we released the previous demo. It's quite accurate, but it's a bit slower than what we would like. And cursor control is the foundation for interacting with most computer applications. So since then we've been working to improve cursor speed and accuracy. As you can see, it's much, much faster, almost twice as fast. However, it's still a bit slower than what I can do. So we are working on creative ways to improve that. Now, speed is not enough. You want a full set of functionalities. And for decades most software was built for mouse and keyboard control. And it doesn't make sense to reinvent this entire ecosystem for brain control, at least for now. So we are working and we are designing mouse and keyboard interfaces for the brain. The way we do that is by training Pedro and his friends on a variety of computer tasks and then designing algorithms to predict the behavior. Here you can see a few examples of tasks in different phases of monkey training. For example, left and right click, click and drag, cursor typing, sprite typing, handwriting and even hand gestures. Now, interacting with computer is bi directional and feedback is very important. I like when I click on a button and I can physically feel the button being pressed. When a potential N1 user will attempt to click, they won't be able to feel it. An example of how we are addressing that is by providing a real time visual feedback that represents the strength of the neural click by changing the color of the cursor. Just by typing on a physical keyboard is much faster and easier than typing on an iPad keyboard. This will make the brain control much faster and easier to use typing one of the most important functionalities. So you already seen this message and I want to show you the behind the scene of how this message was created. And here you can see again sake using the virtual keyboard. Typing this message. This virtual keyboard is similar to the one I use on my phone. And with the speed and accuracy that we achieved so far, typing on a virtual keyboard is already fast and easy. However, I never use a virtual keyboard when I type on my computer because it covers my screen and it's also much slower than what I can do with my ten fingers. We can do better. For example, a group from Stanford asks a person to imagine handwriting letters. Then they decoded the letters from his brain activity. Using this approach, they were able to speed up the typing rate. We start this project with our monkeys, but of course they don't know how to write. So to mimic writing, we train Ranger, one of our favorite monkeys, to trace digits on an iPad. Here you can see him tracing the digit 5 and the digit 2. Then we recorded his neural activity with the N1 device. But now instead of recording the cursor velocity, we decode in real time the digit that he's tracing on the screen. We had two main takeaways from this project. One, that monkeys are awesome and can learn very, very complex tasks. The second one, that although it can increase the typing rate, it requires hundreds of examples and samples of each of the digits and the characters we wanted to classify. This will not scale the way we are solving that is by indirection. Instead of decoding directly the digits, we first decode the hand trajectory on the screen. And then when we decoded the hand trajectory, we can use any off the shelf handwriting classifier to to predict the digits and the characters. For example, classicals that are trained on an MLIST data set. Why it's so important it's important because now we can potentially decode any character in any language with only one neural decoder. For hand trajectory, it means that you can write in English, Hebrew, Mandarin or even monkey language. And we can understand you want a banana? So there are many challenges ahead of us to improve functionality and speed. And I want to hand it off to Bliss to talk about the third part, how we are making our brain interfaces work anytime, anywhere.
Hello everyone, my name is Bliss and I'm a software engineer here at neuralink. When I use my computer, my mouse and keyboard work how I intend them to, at least like 99.9999% of the time. My goal is to enable a user with paralysis to control their computer as reliably as I can. Here's what we want that experience to feel like. In this video you can see Saki walking over to his MacBook and choosing to work on his typing task. The entire decoding system works out of the box and it feels totally plug and play. The first step to achieving this kind of high reliability is to test extensively offline. A typical flow for using the N1 link is to connect over Bluetooth, stream out neural activity from the brain, and then use that neural activity to train decoders and do real time inference. We've built a simulation for exactly this sequence. But instead of using A monkey with an implant. We use a simulated brain that injects synthetic neural activity into an implant sitting in a server rack. From the point of view of that implant, it's in a real brain. This simulation runs on every code commit to validate that from the hardware all the way up through to the neural decoders, our entire stack can achieve state of the art performance. However, while this kind of simulation is great for integration testing of software and hardware, it's not yet detailed enough to guarantee high reliability in the real world. In the real world, the underlying signals we're trying to decode actually change day to day. In this plot, you can see the average firing rate detected on a representative channel of psaki's implant. Each bar represents one day. And you can see that each day has a different average firing rate than the previous. This presents us with a very interesting problem for how to make our decoders robust day to day. It can actually happen that if you train a neural decoder on one day of data and then try to use it on the next, the average firing rates can actually shift enough to cause a bias in the output of the model. Here on the right, you can see that this bias is making it hard for the cursor to move to the upper right corner. You see it's struggling here to make it up to the upper right, and then it moves much more effortlessly down to the bottom left. We're trying many approaches to mitigate this problem. Some examples include building models on large data sets of many days of data to try to find patterns of neural activity that are stable across days. Another approach we're trying is to continuously sample statistics of neural activity on the implant and use the latest estimates to pre process the data before feeding it into the model. This is really an active area of research for the team and it's a critical problem to solve if we want to enable someone with paralysis to control their computer as well as I can. Another big problem we have is to minimize the time it takes for a spike in the brain to impact the movement of the cursor on the screen. If you have lag or jitter in this control loop, the cursor becomes hard to control, leading to the kinds of overshoots that you can see here on the right. One big improvement we've made towards in this direction is called phase lock. Phase lock aligns the edge of each packet that we send off the implant to the exact moment that the Bluetooth radio is going to wake up. This minimizes the time it takes for a spike in the brain to be incorporated into the prediction of our neural network. Here you can see the latency distribution after phaselock. Not only has the mean been greatly reduced, but the variance has been reduced as well. This makes it easier for the user to predict the behavior of their cursor. Over the last year, we've made tremendous improvements to the stability and reliability of our system and we've been able to demonstrate consistent high performance across many sessions and many months. However, there's still a long road ahead of us before this system will truly feel plug and play. So if solving the hard problems required to ship this technology is exciting to you, you should consider applying to join the team. Now I'm going to hand it over to Avinash to to talk about how our custom low power ASIC detects spikes in the brain.
Avinash (Neuralink)
Hi, I'm Avinash, one of the engineers on the ASIC team. We designed the custom neural sensors which include both analog and digital circuitry to record and stimulate across 1024 independent channels. We face challenges across all three major performance, power and area. Not only do we have to fit all 1024 channels into a single quarter sized implant, but we also have to measure spiking activity less than 20 microvolts in amplitude. And today I'd like to focus on the last challenge I mentioned. Power. Power consumption is important to us because we want to give future users a full day of use of their implant without any interruption for charging. Back in 2018, we were sending every sample from every channel off the device for processing, which burned a ton of power. In 2020, we brought Spike detection onto the chip. As you may know, neurons transmit information by firing, so simply monitoring for these spikes and only sending these spike events off the implant acts as a very efficient form of compression. And over the past two years we've continued to make optimizations within the ASIC, dropping the total system power consumption down to just 32 milliwatts and doubling battery life. Let's take a look at our on chip spike detection algorithm which makes our battery powered implants possible. We first start by applying a 500Hz to 5kHz bandpass filter to remove noise that's out of band. Next, we use an estimate of the noise floor to generate an adaptive threshold per channel. And finally, our spike detector module identifies three key points of a spike. Identifying three points allows us to detect not just the presence of a spike, but the shape of a spike as well. This can be extremely important for distinguishing between multiple neurons adjacent to a single channel. Today I'd like To focus on one of the many optimizations that we've made in our latest chip, this one specifically cutting system power by 15%. Note that neurons spike relatively infrequently, which means that our spike detector spends a lot of time searching for the first point of a spike and very little time searching for the other two points of a spike that only occur after the threshold is correct crossed. We can use this characteristic of the input waveform to reduce memory accesses within the chip by 30%. Let's take a look at how that works. Our spike detector is implemented as a single functional unit that's shared across all channels with an SRAM to buffer the state of each channel. As a sample comes in, its channel state is read from SRAM and incremental spike spike detection step is run and then the updated state is written back to SRAM. Since this is happening 20 million times per second across the implant, each of these accesses add up quite quickly. In our latest chip, we split the state into two parts, A hot state and a cold state. The hot state is accessed on every cycle, while the cold state is only accessed once the threshold is crossed, Reducing the average access width and saving power. We're also working on a next generation stimulation focused chip with 4096 channels still within the footprint of our current chips. In addition to increasing the channel count, we're also increasing the drive voltage so we can get better activation per channel. And to support this higher channel count, as well as a broad range of future applications that you'll soon hear about, we're adding an arm core onto the chip. And finally, since these chips are the same size as our current chips, we can still put four of them together into a single implant for a total of 16,000 channels. Still within the size of a quarter. As you can see, we've been working very hard to improve the power consumption within the implant. But we've also been working very hard to improve the charging experience of the implant, which Matt will talk about. But first, the robot has just completed inserting all 64 threads, so let's take a look. This is a view of the insertion site similar to the one that DJ showed you earlier. But instead of the targeting reticles, if you look closely, you can see that all 64 threads, each carrying 16 electrodes, have been inserted into the brain proxy while avoiding vasculature. And all just within the past 20 minutes. Let's hand it over to Matt now
Sam (Neuralink)
to continue the technical deep dive.
Matt (Neuralink)
Hi, I'm Matt, head of Brain Interfaces Electrical engineering. Our fully implantable N1 device depends on a battery for continuous operation. When that battery is running, low charging is accomplished through wireless power transfer. However, unlike many consumer electronic devices which can simply offer a physical connector, charging, a fully implantable device poses several unique challenges. First, the system must operate over a wide charging volume without relying on magnets for perfect alignment. The system must be robust to disturbance and complete quickly so as not to be overly burdensome. However, most important is safety in contact with brain tissue. The outer surface of the implant must not rise more than 2 degrees C. In pursuit of these goals, our charging system has gone through several engineering iterations. The first, if you watched our pig demo in August of 2020, Gertrude was implanted with a version of the N1 charged with our first generation charger. This device was implemented in a small puck package and later separated into a remote coil and battery base. This charger was challenging to use. However, we learned a lot through its implementation. Our current production charger, which charges our current generation of implants, is implemented in an aluminum battery base which also includes the drive circuitry. A remote coil four times the size of our original device. Also disconnectable, This remote coil has increased switching frequency driving improved coil coupling. This charger is in use today, including several applications within our engineering and animal test facilities. I'd like to show you one of these applications here. With a device we call our simple charger and the coil has been embedded into the habitat. With the addition of one new outer control loop plus a banana smoothie pump, the troop has been trained to charge themselves. So let's see how Pager charges his implant. On the right, we're streaming real time diagnostics from Pager's N1. When he climbs up and sits below the coil, you can see the charger automatically detect his presence and transition from searching to charging charging. We see the regulated power output on a scale of 0 to 1 and the current driven into this battery. I mentioned earlier that we improved the coil coupling. However, the high quality factor coils exhibit good charging performance over relatively larger distances. But as they're brought closer to the implant, what you see is a peak splitting effect where the best, highest efficiency power transfer is pushed up into higher frequencies outside of the ISM band required for compliance with regulated radiated emissions. In our next generation charger. We address this problem by the introduction of dynamic tuning shown on the right. This allows us to in real time adjust the resonant frequency of the transmit and receive coils so that we can change their properties just ahead of degraded performance. The electrical engineering Team is currently engaged in developing a third generation charter charger. Notable improvements include bi directional near field communication. This has allowed us to reduce the control latency and improve the thermal regulation. Improve thermal regulation results in faster charge times. And now Julian will tell us about how we test the N1.
Julian (Neuralink)
Thank you very much, Matt. My name is Julian and I lead the embedded software group on the Brain Interfaces team. So when we started building implants, we had a small manufacturing line and to collect data from an implant, you would manually walk over with your laptop, you would connect and collect the data of interest. But our goal is to make an ultra safe and ultra reliable implant. And so to do this, we scaled up the manufacturing line, our testing, throughput and data collection capabilities. So firstly, we added a large suite of acceptance tests to the manufacturing line. These test the functionality of each component and the final assembly. Implants coming off the line are then subjected to benchtop testing, accelerated lifetime and animal models. We then collect data from these implants round the clock. This data is processed by a series of cloud workers and displayed in an aggregate manner. And then finally, all of this information feeds back into our design process and empowers our engineers to answer any question about any implant at any time. I'm now going to walk you through different parts of this infrastructure, starting off with firmware testing. So the implant contains a small microprocessor running firmware to manage a whole bunch of its operations. And before we release a firmware update, we want to rigorously test it with both unit and hardware in the loop tests, also known as hill tests. So to do a hill test, what you do is you instrument the battery, you instrument the power rails, the microprocessor, and then we connect to each device with a Bluetooth client, and then we walk the devices through various scenarios to test things like power consumption, real time performance, security systems, fault recovery mechanisms, a lot of different things. In our original implementation of these systems, we used off the shelf components to start automating tests quickly. However, these systems were constructed in a relatively artisan fashion and were very difficult to maintain. And this meant that testing quickly became the bottleneck for development. So to alleviate this, the hardware and software teams developed a new system which integrates all the required components onto a single baseboard. We can then put the charger and implant hardware on individual modules that plug into this baseboard, including one board with opposing coils, so that we can test charging performance. This architecture allows us to rapidly iterate different hardware prototypes, because we can simply drop them into the system and reuse all the testing infrastructure. Additionally, we can host the current and next generation of our neural asics onto FPGAs and plug those into this board as well. And that allows us to test a whole extra layer altogether. So that's how we generated this rather inceptive image here on the right. What you're looking at is spiking activity emitted from some of our simulated neural sensors, streamed through the entire system over Bluetooth and then displayed on a phone. This allows us to test everything in one system from chip to cloud. This system is 1/5 the cost, 1/5 the volume, and is very easy to manufacture. This allows every developer to have a personal unit on their desk, and it also allows us to to shard the entire test suite over a large number of these units mounted into a rack. All of this has greatly accelerated our rate of development. Let's look next at how we monitor the implant's electronics, the battery and the enclosure. So the implant will periodically capture all of its vital signs and commit those to flash. And then upon next connection with one of our recording stations, it will stream that data off. So for instance, if we look at humidity, we can get an understanding of the integrity of the implant's enclosure. And by looking at battery voltage and power measurements, we can gauge battery health. All of this is done automatically, without any intervention, giving us 24,7 visibility into the quality of every single device. Additionally, we can use this infrastructure to request high fidelity information on demand, so that we can investigate different anomalous situations. So, for instance, in this particular scenario, we were trying to track down the source of some spurious spikes that we were observing on different channels. And so we requested raw wave samples directly from those channels. Capturing good quality neural signals requires intact low impedance electrodes. And so this is also something we monitor very closely with dedicated circuitry on the neural sensor. So how do we do this? We do this by first using an onboard DAC to play a test tone on a single channel. And then we record, using our ADCs simultaneously, we record the response signal on both that channel and physically adjacent channels. Not only can we measure the impedance of every channel with this, but we can also map different physical phenomena to different characteristic signatures. So, for instance, an open channel will, will appear as a very large response on the channel, and shorter channels will appear as a large response on neighboring channels. By looking at the purity of the signal coming back, we can also validate that the analog front end of the neural sensor itself is operational. In our original implementation of doing these impedance scans, it took four hours to get through all 1,000 channels. But by paralyzing the tests, down, sampling, filtering, and then reducing the amount of information we have to stream off the device, by moving a lot of the calculation to the firmware side, we're now able to scan all 1,000 channels in just 20 seconds. This means that we can run impedance on every implant every day. And then our internal dashboards can play back a history of this impedance so that we can get a really good quantitative insight into that interface between biology electronics. Now that you have an idea about how we test and monitor our implants, I'm going to hand it off to Josh, who's going to tell you about how we get feedback even faster by accelerating our implants to failure.
Hello, my name is Joshua Hess and I'm an engineer on the Brain interfaces team. We are responsible for the implant system design as well as many of the manufacturing and testing tools. Julian, just talked to you a little bit about some of the ways in which we test our implant electronics, hardware and software. But what about the entire system as it relates to longevity in tissue? One of the ways we've addressed this is with the development of our in house accelerated lifetime testing system. The the system allows us to expedite and capture long duration implant failure modes at scale to rapidly increase our pace of iteration. Even better, the system also significantly reduces the amount of tests which require animal models, both for implant prototypes and of course, longevity testing. So how does the system work? At a very basic level, it comes down to three things. First, we want to mimic the internal chemistry of tissue. Next, we want to accelerate these chemical interactions as well as diffusion with our implant materials. And finally, we want to aggressively cycle the internal electronics of our implant with these things, Primarily the first two, we have achieved a conservative 4x acceleration factor by the Arrhenius relationship. In other words, every day our implants spend in our accelerated system is equivalent to at least four days spent in vivo. Historically, one of our greatest challenges has been the battle against moisture ingress into our implants. So we continuously monitor the internal humidity to watch for abnormal rise. Here in white, you can see some internal humidity data from implants and some of our animals for the duration of over one year. As you can see, our internal humidity sensing is so sensitive, it can even detect the very small and slow humidity rise just from diffusion through our implant materials. Now, in blue, you can see that same internal humidity data, but from devices in our accelerated system. Now, if we adjust this data for our acceleration factor, you can begin to see not only the agreement in this data, but also just how far into the future the data extends. Now, in red you can see a device which has failed in our accelerated system. This device showed an abnormal increase in humidity over the duration of many months before implant electronic failures occurred. So how do we build this system? Well, we started building the first system prototype just after the COVID shutdown had begun into early 2020. So we had to get a little creative. As you can see, our first system prototype was a little scrappy and operated out of one of our apartments as indicated by the carpeting. Although scrappy, the system allowed us the fastest path to start testing our devices, tuning our working fluid chemistry and checking our constraints. We also immediately started root causing observed failures in early implant prototypes, fed that information into the next prototype designs and literally rinsed and repeated over the duration of just a few months. The system was built out totally custom and highly iterated with two system versions and countless minor iterations leading us to our currently operated third generation system which achieves high density testing with automatic in vessel charging as well as automatic data collection. The system also features an implant sled assembly which accepts brain proxy material such that the implant can be installed and inserted by the surgical robot just like you saw a few minutes ago. We also integrated the system into a high density rack mount form factor along with a centralized fluid management system, both for chemical uniformity across vessels and also reduced operational maintenance. The system has been in operation for the last year and a half and has had its fair share of challenges. Since the system itself is undergoing the same accelerated abuse as the implants within it, it has been extremely challenging to design, build and maintain a system of this scale while keeping it robust even against itself. So what comes next? Well, we have started work on our fourth generation system and have totally redesigned it from the ground up to be a hot swappable single implant per vessel design, partly inspired by high density compute servers. With this new system, we will achieve a whole new level of density, robustness and scale. We also intend to have many of these systems operational in the pursuit of capturing even the lowest frequency edge case failure modes. With this, we will have thousands of implants testing in pursuit of these goals. We've already started work building out the system, but there is still a lot left to do. There are also many exciting challenges ahead of us such as introducing mechanical stressing, brain proxy micromotion and EFIN replicating tissue growth around the threads for more complete and representative accelerated testing. So now that you've heard some of the ways in which we rigorously test our implant designs before production for surgery. Christine is now going to take you through a detailed look at our surgical process.
Christine (Neuralink)
Thanks, Josh. Hi, everyone. I'm Christine, lead of the surgery engineering team. To get an N1 device, it's essentially these steps. Targeting and the incision, drill the craniectomy, remove the tough outer meningeal layer called the dura, then insert the thin, flexible threads of electrodes, place the implant into the hole we created, and then that's it. You've got an implant under the skin. Look, Ma, no wires. Just kidding. I mean, seriously, no wires. But I don't actually have one. The surgical robot does the thread insertion part of the surgery. This is because it would be very difficult to do manually. Imagine taking a hair from your head and trying to stick it into jello covered by saran wrap and doing this at a precise depth and position and doing this 64 times within a reasonable amount of time. And not. Neurosurgeon would probably not like it very much if we asked them to do this for the surgery. So we have the robot that you saw doing its tiny dance. I sort of wanted to call it Tiny Dancer, but it's called R1, which is also great. The rest of the surgery is done by the neurosurgeon. In order for us to make an accessible and affordable procedure, we need to revisit this. I'll tell you why. When I was in school, my dad lost the ability to walk and to use his arms and even to speak. He was diagnosed with als. We would look on the Internet and you could see maybe one person here or there who had some cool custom robotic assistive device. But it was deeply frustrating. How limited were the options available to him? And there's hundreds of thousands of people with paresis, not even counting people with other conditions that our device might be able to help. Meanwhile, there's not that many neurosurgeons, maybe about 10 per million people. And it takes about a decade or more to train neurosurgeons, and they're already generally very busy. And as you can imagine, the time is very expensive. So in order for us to do the most good and have an affordable and accessible procedure. Procedure. We need to figure out how one neurosurgeon could oversee many procedures at the same time. This might sound sort of crazy, but probably so did laser eye surgery before Lasik made it normal. Lasik's been around for about 30 years and counting. In the beginning, the laser robot did just the most fundamental core part that it had to do and the surgeon did the rest. And, and over the iterations, the surgeon has to do less and less and the laser robot does most of it. And it's a highly compelling procedure, takes just a handful of minutes and often gives life changing results. Since I joined in 2017, we've also done a handful of iterations to optimize the thread insertions of the robot. One of the challenges that we've had to face has to do with the optomechanical packaging. So, as you can see here, there's about three primary optical paths that are really valuable for us to have reliable thread insertions. One is the visible imaging of the needle inserting a thread. And then another is the laser interferometry system called octical coherence tomography. That gives us the precise position of the brain while it's moving in real time. And then also we have to provide lighting and illumination to see what's going on in the visible light camera. And doing all this, where the needle is at the bottom of the craniectomy, especially when it's close to the skull wall, can be pretty difficult to fit everything and be able to see it. So the way that the team solved this is by putting all three of these optical paths into one optical stack using photon magic, or polarization, whatever you want to call it. And that enables us to do vessel avoidance in real time. So as I mentioned, the brain is moving and where we place targets in the beginning may not be where you want to insert at the moment the needle is going down there. So the robot can actually detect the vessels and then determine if we're going to insert onto a vessel or not, if it's safe to insert. And then that way we can avoid inserting onto major vessels. And that brings us to the robot that we have here today. There's still a lot for us to do to get to that procedure where we reduce the role of the neurosurgeon and make it affordable and accessible. The primary the two elements of the surgery that demand the most skills from the neurosurgeon are the craniectomy and the durectomy. Alex and Sam are going to tell you a bit more about how we think we can get rid of the durectomy step. So that leaves the craniectomy. In neurosurgery, if your craniectomy is small enough, you can use a standard tool called a perforator, which makes quick work of this shape job. But for a larger craniectomy, the surgeon has to rely on their skill in order to accommodate the variability, patient to patient, in skull thickness, skull hardness, even within the same patient in the same craniectomy, you can have different skull thicknesses, for example. In addition, if we can make something that has a very high precision craniectomy, we can open the design space for future ways of mounting the implant to the skull. So. So I'll show you a few of our prototypes. Ultrasonic cutters like what's on the screen and oscillating cutters have the benefit of not cutting soft tissue. You can cut the bone and not the brain, but however, as you can see here, our ultrasonic cutter prototype created quite a bit of heat to cut at the rate that we wanted. So onto the oscillating saw here we designed a blade to minimize cut time and also conducted sound and also heating. And as you can see, you can cut through hard things like bone, but not soft things like skin. It's simple and it works. However, if you wanted to cut an arbitrary depth or arbitrary shape, the oscillating saw just won't cut it. I was afraid no one would get it. You guys are smart. So there's a time tested solution for drilling arbitrary shapes, which is a CNC drill. The challenge with us doing this on a person is that we need to make sure it cuts reliably every single time, doesn't cut too deep. And a few ways that we're using feedback to make sure we don't cut through the brain are force feedback and also impedance. And if I could get a volunteer. No, just kidding. Maybe next time. But yeah. So this is some insight into some of the things we're working on to make an accessible and affordable procedure. And now Alex is going to tell you a bit about our next generation developments.
Alex (Neuralink)
Thanks, Christine. I'm Alex. I'm a mechanical engineer here on the robotics team. Now that we've covered the technology and surgical process for a current device, we'd like to cover some of our next generation development projects. I and the next couple speakers would like to talk about one of those projects, which is enabling device upgradability. You've gotten to hear about the advancements we've made over the past year. We've improved implant robustness, battery and charging performance, Bluetooth usability. Realistically, every new device version is going to be significantly better. It'll be more functional, it'll last longer. We need to keep this new technology accessible for our early adopters. This means that we need a solution to make device upgrade or replacement just as easy as it is to initially install, as many medical device companies have found, this is a challenging problem. The body's healing response doesn't make this easy. So this isn't solved yet. But we've made significant progress towards enabling this that we'd like to cover today. Now we'll have to start with some background as to what makes device upgrade challenging. And we'll start with the anatomy. Under the skin, you have the skull. Below that, the dura, a tough membrane that separates the bone from the brain. And between the dura and the brain, you have the pia arachnoid complex, a fluid filled suspension for the brain. To install the device, the surgeon removes a disc of skull and dura to expose the brain surface. The device then replaces the removed material. The challenge is here at this interface. Over months, all empty volume is filled by tissue encapsulating the device and the threads. The device would be trivially easy to remove because of the threads small size, they would have slipped right out of the brain. It's the tissue layer that forms above the surface that makes removable challenging. We built tools in house to study this response and characterize it, such as histology and micro ct. In these images, you can see that layer of tissue that has formed above the surface, encapsulating the threads and adhering to the surrounding tissue. We've explored many different avenues for designing around this healing process and finding a solution to make device upgrades seamless. Our best successes have come from making the procedure less invasive. Instead of directly exposing the brain surface, we instead keep the dura in place, maintaining the body's natural protective barrier. This prevents encapsulation at the brain surface. And really this is actually a huge win for making the surgery simpler and safer. As Christine alluded to, however, this doesn't come for free. The dura is a very tough opaque membrane. As you can see in these SEM images. It's composed of a dense network of of collagen fibers. These offer an array of technical challenges for inserting our electrodes. One of those challenges is imaging through the dura. As you can see on the left, our current custom optical systems offer pretty incredible capabilities for imaging the exposed brain surface. However, as you can see on the right, once the dura is in place, you can't see the dense vasculature at the brain surface. The dirt is in the way. There's simply too much attenuation. To solve this problem, we're developing a new optical system that uses medical standard fluorescent dye to image vessels underneath the tissue. Here you can see that dye perfusing through the vessels, highlighting them. There's still a lot of engineering work to go to prove accuracy and repeatability of this system, but once that's done, this will allow us to target and avoid blood vessels underneath the dura. We're also exploring applying our laser imaging system to deeper tissue structures. In the bottom left, you can see a section of the tissue layers underneath the dura. This image is compiled from multiple volumes from our optical coherence tomography system. You can see the collage of those volumes of above. In the future, these new systems, when combined with correlation to pre op imaging such as mri, will enable precise targeting without directly exposing the brain surface. Now, imaging isn't the only challenge that comes with a tough dural anatomy. Now I'd like to hand it over to Sam to talk about some of the challenges of inserting our electrodes through this membrane.
Sam (Neuralink)
Thanks, Alex. Hey, I'm Sam and I lead the needle manufacturing and design team. So, as Alex mentioned, the same properties of the dura that make it a good protector of the brain also make it really difficult for us to insert the threads into. In humans, the dura can be over a millimeter in thickness, which doesn't sound like a lot, but compared to our 40 micron needles, it actually is a lot. For example, if you scaled up the needles to the size of a pencil, the dura would scale to over 4 inches in thickness. Take a look at how far you have to zoom in to even see it. By the time the features of the needle come into frame, you could see individual red blood cells in the same frame. This is, this is just.
Sam (Neuralink)
This is a real life SEM image of our latest design. On the left there, you can see the end of the thread. In the middle is the needle, and on the right is actually a piece of my hair. So, yeah, it's extremely small. And besides being really small, there's a lot of other challenges associated with designing this. One challenge is that we have to use the needle and the protective cannula that it sits in to grab onto the thread and to hold it while we peel it from this protective silicon backing. And then we have to keep holding it while we bring it over to the surface and then release it from the cannula during insertions. Another challenge is that the brain is really soft beneath the tuff dura. And so if the needle isn't sharp enough, it'll just keep dimpling the surface without puncturing. And if this free length gets too long, it can actually Just buckle the needle like this. Another challenge is that we don't just have to get the needle through, we have to get the thread through as well. So we really have to focus on optimizing the combined profile of the needle and thread together. These are just some of the challenges associated with designing something like this. And so far, we found that the key to solving this problem has been improving on our speed of iteration. But let's look at how we make these things in the first place. So we start with a length of 40 micron wire made out of tungsten and alloyed with a little bit of radium for added ductility. We designed this femtosecond laser mill in house to cut the features of the needle and cannula, and it can do this with sub micron precision. We spent a lot of time this year turning this thing from a science project into an industrial system. Just a couple months ago, it took a skilled operator 22 minutes to make a needle, and even a skilled operator could only get about 58% yield. Today, that same process takes just six minutes, and anyone can get 91% yield with just a few minutes of training. With only one click, the mill cuts and measures the needle and cannula and uploads the measurements to our lim system so that the robots can use the exact dimensions for each needle that it uses. Now, this is all for our current design, though, and we've had a couple years to optimize the manufacturing process of it. The current design has served us well so far, but it doesn't quite protect the thread well enough to get through the tuft. So, like I said, we had to come up with something new, and we needed to be able to iterate on designs quickly. Unsurprisingly, there's no page in machinery's handbook for this kind of thing. So we dug into the science of femtosecond laser ablation and figured out a workflow that allows us to use our laser mill much like a CNC mill. This allows us to iterate several times. This allows us to iterate in under an hour for new designs, allowing several iterations per day when we're really on a roll. As a result, the latest design still seen on the right can actually insert through nine layers of durock, totaling three millimeters on the bench top. This is far more than we could ever expect in a human with significant margin. The needle isn't the only part of the puzzle, though. As you can imagine, all these designs here work with different threads. So we need a way to iterate on that as well. And we do this by having our microfabrication process here in house. This summer, we completely rebuilt our clean room in about nine weeks, which, among other things, greatly reduced particulate counts, which allows yield and throughput to greatly increase. This, combined with all the other great improvements the microfab team has made, Allows us to iterate on new designs in just a matter of days. The last piece of the puzzle, though, is testing. We can come up with as many new designs as we want, but unless we have a way to actually test them in the right conditions, we won't know what to tweak. Or even worse, we'll spend time optimizing for the wrong things. Take this failure mode, for example. A few months ago, we got to the point where we could pretty reliably insert through the dura. But when we took the proxies and put them in our micro CT imaging, We realized that our hold on the end of the threads was actually too strong, and we were pulling them out just a little bit underneath the stick surface. By the time we solved the problem, we realized that this issue was very sensitive to the properties of the surrounding material or tissue. We could make a proxy where this never happens, and we can make another proxy where this happened every single time. And this highlights why it's crucial that we spend time making our benchtop tests match tissue as accurately as possible. And I'm going to pass it off to Leslie now, who's going to talk about how we've been doing that. Thanks.
Leslie (Neuralink)
Hi, I'm Leslie and I lead microfabrication R and D. And part of what we're interested in is understanding the biological environment Our implant and threads experience Once they're fully installed in the body. Learning directly from biology, though, is inherently slow. So in order to move fast, we're developing synthetic materials that mimic the biological environment. This allows us to learn as much as we can on benchtop and start taking steps away from the industry standard of animal testing. Developing accurate proxies, though, is challenging. The implant environment is made up of many anatomical layers that all have unique properties. And as time goes on and the implant site heals, New tissue forms filling any available space. In addition to that, motion related to cardiovascular activity and head movement introduce added complexity. So to start addressing some of these challenges, we're engineering materials using feedback from biology. This may involve mechanical characterization of tissue or analysis of interactions at thread tissue interfaces. Much of this characterization is even done during surgery itself by using custom hardware and software that modifies our surgical Robot to double up as a sensitive characterization tool. We then use the data collected and feed it back into optimizing our materials so that they behave mechanically, chemically and as shown here, structurally, just like biology. We've come a long way from our humble first brain proxy, shown here sitting on a plate and consisting of agar and a parafilm sheet. And while simple, it allowed us to perfect robot insertions through countless bench shop tests. Today, our proxy is slightly more complex where we've upgraded to a composite hydrogel based brain proxy that better mimics the modulus of real human brain. We've also incorporated a duraproxy and developed an injectable soft tissue proxy that so far has allowed us to perform bench top mock X plant testing. We have a super long wish list for our proxy of the future, but some of those items include a surgery proxy with integrated soft tissue, brain, bone, skin or even a whole body. A brain proxy that simulates motion, vascular and electrophysiological activity, and a biological proxy to test biocompatibility and electrical stimulation. There's a ton of ongoing work getting us closer to our proxy of the future, including work on lab grown cerebral organoids as shown here. And all of this will get us closer to a future where we learn more and iterate faster on benchtop and reduce our reliance on animal models or even one day replace them completely. And with that, I'll hand it over to Dan, who will be presenting a very exciting next generation application. Thank you.
Dan (Neuralink)
Thank you, Lesley. My name's Dan and I came to work at neuralink after following a career in visual neuroscience research.
Dan (Neuralink)
I was inspired to join this company because I saw in our device the potential to restore vision to people rendered blind by eye injury or disease. There are a number of particular characteristics of our device that make it uniquely suited to this application. Firstly, as well as being able to record from every channel, we can stimulate neural activity in the brain by injecting current through every channel. This is important because it allows us to bypass the eye and generate a visual image in the brain directly. Secondly, our device can have an enormous number of electrodes for a visual prosthesis. This is important because the more electrodes you can have, the higher density of an image you can create in the brain. Thirdly, thanks to our robot, we can insert these electrodes deeply into the brain. Now, this is an important thing for a visual prosthesis because the human visual cortex is buried deeply in a fold in the medial face of the brain called the calcarine sulcus. In this image I've highlighted the calcarine sulcus in red in an mri. It contains a map of the visual world, visual field. It's about surface area, equal to a credit card on each side. And if you unfold it and flatten it, you see that the image is inverted, it's upside down, but more interestingly, it's distorted, so that the central part of the visual field, the fixation point, is greatly magnified. So, for example, if you look at this image of Lincoln, if you look directly into his right eye, everything to the left of that fixation point is directed to your right visual cortex, and everything to the right goes to your left visual cortex. His eye, even though it's very small in the image, is magnified in the brain to occupy nearly a quarter of the surface area of the visual cortex. Over the last half century, visual neuroscientists have developed a profound understanding of visual processing in the brain. What's driven most of this research is recording from single cells in the cortex, usually of macaque monkeys. One of the seminal discoveries was that every cell in the visual cortex represents only a tiny part of the visual field. Your perception is made up of a mosaic of tiny receptive fields, each belonging to a single cell in your visual cortex. So if you record from one of these cells in a monkey, say in this location, you can find a very tiny region of the screen where a light stimulus will cause modulation of that neuron. Another location in visual cortex will have a location elsewhere on the screen. In this case, in the lower visual field. These regions are called receptive fields. We've inserted our device into the visual cortex of two rhesus monkeys whose names are code and dash. That means we can record activity from their visual cortex generated by their normal home environment as they roam around. But as we all know, monkeys love banana smoothie. That means we can easily teach them to fixate points on a screen and reward them. We can reward them very precisely because we can track the location of their eye using an infrared camera. One of the things this allows us to do is to plot the receptive fields for every neuron that we can record with a single device. Now, we do this by showing the animal a movie of random checkerboards whilst he fixates steadily on the screen. Then we take only the frames of the movie that generated a response in the cell and average them all together. This is a technique known as reverse correlation. It's generally used quite widely in visual neuroscience for this purpose. And this is an example of a receptive field plotted with this technique, the central cross is the fixation point, and you can see the little red and blue regions of excitatory and inhibitory receptive field. These regions give cortical cells some of their characteristic properties. So we can record all the receptive fields from all the electrodes at the same time. And if we take all these receptive fields and accumulate them together, overlap them and place them on a, on a computer monitor for scale at a typical viewing distance, you begin to get an idea of how much of the visual field we can cover. With this preliminary device, many of the receptive fields are close to the fovea,
Alex (Neuralink)
close to the fixation point.
Dan (Neuralink)
That's partly due to the magnification that I talked about of the fovea, but there's also a scattering of fields in the periphery. These are from recording sites deeper in the brain in the calcarine sulcus. So far, I've only talked about recording information from the cortex. But to produce a visual prosthesis, we need to stimulate. So if we stimulated the cells whose receptive fields are in this location, we would produce a perception of a flash in that location that only the monkey can see. How do we know that the monkey sees it? How do we know what it looks like? Well, unfortunately, we can't ask them what they see, but we can train them to tell us something about that phosphine. We start by training the monkey to fixate a central point on the screen, like this white dot. And we start by presenting real visual stimuli on the screen and rewarding the monkey for making eye movements toward those stimuli. So here we flash a white dot and the monkey makes an eye movement towards it, symbolized by the green arrow. We then choose another random location and reward the monkey for making an eye movement towards it. Once he's got good at this task, we can begin to interleave these real stimuli with electrical stimulation of electrodes and produce a phosphine. The monkey sees the flash and naturally makes a saccade towards it. This tells us not only where in the visual field the flash occurred, but we can also change the current that we inject in that electrode to see how often he makes that saccade, how noticeable, or how big perhaps the stimulation phosphine is that we're producing. Let's look at code performing this task. I want to show you first, at 1/4 speed, there's a visual flash and he makes an eye movement towards it. The monkey can only see what is white on this screen. He can't see his own eye movement, and he certainly can't see when we stimulate, but here we stimulate, and he makes the same saccade to the same location. Because we stimulated the same electrode, nothing appears on the screen at that time, and he has no other cue to make that eye movement. Let me show you this in real time. You can see monkeys like to work very quickly. And when we stimulate, he makes that saccade in real time. And looks like he's had enough. So what I've shown you is a way to produce a phosphine in the visual field. This is not something new in visual neuroscience, but if you think about that phosphine as a single pixel in a visual image, all we need to do is scale up and produce a great many more pixels and have them covering the visual field. This is a schematic of what a visual prosthesis using our N1 device might look like. A camera. The output from a camera would be processed by an iPhone, for example, which would then stream the data to the device. And the image would be converted into a pattern of stimulation of the electrodes into visual cortex. With 1,000 electrodes, we might be able to produce an image resembling something that you see there on the right. But as Avinash told you, our next generation of the device will have 16,000 electrodes. If you put a device on both sides of your visual cortex, that would give you 32,000 points of light to make an image in someone who's blind. Our goal will be to turn the lights on for someone who's spent decades living in the dark. Thanks very much. I'll pass you over to Joey, who's now going to talk about another very exciting application of our device.
Matt (Neuralink)
Thank you, Dan.
Joey (Neuralink)
So, my name is Joey. I'm a neuroengineer, and I'm the head for the next gen team at Neuralink. So, for persons with spinal cord injury, the connection between the brain and the body is severed. The brain continues functioning normally, but it's unable to communicate with the outside world. You've already heard about how we can use the N1 link as a communication prosthesis to help someone with spinal cord injury control a computer or a phone. But it can also be used to reanimate the body. Let me show you how. First, a little neuroanatomy. Movement intentions arise in motor cortex and are sent down long nerve fibers through the spinal cord. These are upper motor neurons in the spinal cord. They synapse, that is, make a connection with another motor neuron, a lower motor neuron, which sends these movement intentions to the muscles which contract and in turn, you have movement. While, of course, there are many other circuits involved in voluntary movement. You can think about the spinal cord as as many pairs of these two connections. And in spinal cord injury, one of these connections is severed, unable to make the muscles contract. Let's zoom a little bit further. So here you can see on the left, a cross section of the spinal cord with a fiber coming down schematically. This travels through the white matter tracts. This is the upper motor neuron, and then it synapses within this butterfly shaped region of gray matter in what's known
Audience Member
as a motor pool.
Joey (Neuralink)
In the motor pool, the lower motor neuron descends out the ventral roots to the muscles, which contract, and then the sensory consequences of those movements, for example, the touch of your hand against an object, return to the spinal cord through the dorsal roots and ascend the spinal cord up into the sensory regions of the brain. Again, in spinal cord injury, this connection is severed. If we could place electrodes into the spinal cord, say, in a motor pool adjacent to lower motor neurons, we could stimulate those neurons, activating them and in turn causing the muscle to contract and movement to occur. But this is very hard to do. The spinal cord is quite delicate, and it moves significantly within the bony spinal canal. This could cause damage to the electrode, it could cause damage to tissue, or both. But our electrodes are small and flexible, and our robot is able to insert them deep into tissue, perhaps all the way down into the ventral horn of the spinal cord. And so we have done just that. Here you can see a view from the R1 robot. It's a targeting view. And we've placed electrodes across many millimeters of the spinal cord. And the R1 robot is able to insert those electrodes deep into the ventral horn, into motor pools in very close proximity to lower motor neurons. This is important because it allows them to have a localized connection to those neurons and activate very precise movements. Now, to track movement, it's very common to use motion capture markers like you might see in the production of a movie. These can be placed with a light adhesive, and you can see me placing these on my hand. We're going to use these markers to let us zoom in on movement in the next couple of slides. Okay, so here's a pig walking on a treadmill. And you may have seen something like this before in a previous neuralink presentation. But unlike before, this pig has more than one neuralink device. There's a device in the brain, but there's also one in the spinal cord. And we can stream neural data from this device, these devices in real time and use them to do things like, like decode the movement of the joints of the pig. So here you can see on the left a time series of the hip, knee and ankle. And we're decoding those movements. So this is super cool, but that's actually not what we want to do. We want to go in the other direction. We would like to stimulate the spinal cord and cause movement to occur. Okay, so let's do that. So here's a pig, a happy and healthy pig, doing what pigs like to do, which is root around for food and snacks. And as you'll see on the floor, there's a blue screen. This is a voluntary engagement zone where the pig places itself, indicating that it's comfortable to receive stimulation. When it's in the zone, we stimulate. And if the pig leaves the zone, we'll stop stimulating. And as before, you can see we're able to track the position of the joints and also stream neural data as well. Okay, so let's stimulate an electrode. So here's one electrode on one thread that when we stimulate clot causes a flexion movement of the leg. So on the left you can see the movement of the joints and you can also see the time series of the stimulation pattern in yellow. So the leg is moving up. Here's another electrode which when we stimulate, causes an extensor movement. This is actually a little harder to see because the leg is straightening and the hips are shifting. But if you look carefully, you can see how this is. The leg is moving. We can stimulate on a great variety of threads and produce different movements and, and actually sequence them spatial temporally to provide patterns. So on the left you can see a time series of different stimulation on different electrodes. You can see the movements of the joints. And on the right, we're zooming in on muscle activity. That gives us an idea of the kind of strength and power and specificity of those movements as well. So in addition to doing sequences, we can also achieve sustained movement. These are powerful muscle contractions of the sort that you might need for standing or other load bearing activities and are really crucial for interacting through the world. Okay, so stimulating the spinal cord is only one piece of the story. You also have to get like command signals for the stimulation on the spinal cord. Unfortunately, we have a way to do that. We have the M1 link that you've already heard about, placed in motor cortex.
Joey (Neuralink)
So we place threads in motor cortex and record spikes. These spikes would be wirelessly transmitted, emitted in real Time and decoded into patterns of stimulation. Stimulation would then be delivered to the ventral horn of the spinal cord, to the appropriate motor pool for the muscles that we like to activate. We then stimulate activate those lower motor neurons, which causes the muscles to contract and movement to occur. Now, of course, movement without sensation is actually kind of difficult. Just think about what it would be like to try to move your limbs if they're numb. But we can also get sensory information as well. So the sensory consequences of your movement can be recorded in the dorsal horn of the spinal cord in the form of spikes. For example, here, a feather touching the hand. These spikes can in turn be decoded in real time, sent to patterns of stimulation to either the same N1 device in the brain or perhaps a different one in a sensory area. Stimulation of that part of the brain would cause percepts of touch and proprioception, closing the loop. So putting those two loops together, we have motor intentions decoded from the brain used to stimulate the spinal cord, causing movement, and then the sensory consequences of those actions being recorded in the spinal cord to stimulate the brain, causing perception. Now, we have a lot of work to do to achieve this full vision, but I hope you can see how the pieces are all there to achieve this. And if you find this prospect as exciting to you as it is to me, I hope you'll consider joining us here at neuralink.
Leslie (Neuralink)
Thank you.
Sam (Neuralink)
Thank you.
Dan (Neuralink)
This is obviously amazing and has clear therapeutic potential. It would also be great for the
Audience Member
scientific neuroscience community to access some of these tools.
Dan (Neuralink)
Do you have any plans to make these available to neuroscientists?
Yes, yes, we do. So it's a great question. I think there's probably a lot that can be figured out if we provide the surgical robot and devices to neuroscience research departments at universities and hospitals. So I think at the point at which we have. We need to be in production with the machines and obviously have the FDA approvals, but I think it would make a lot of sense to provide this to research universities and hospitals. The question is, of the data that we.
Leslie (Neuralink)
Of the data sets that you've collected, are there any that you plan to open source source for the scientific community?
Yeah, I think that would be. That would be fine, I think. Yeah, sure, absolutely.
Leslie (Neuralink)
Because I think it could be really interesting for people working in AI research to build upon that and build foundation models for the brain?
Yeah, it's a good point. Yeah. I actually get no problem with just publishing it on our website. Use it if you want.
Leslie (Neuralink)
Looking forward.
Sam (Neuralink)
Thank you.
Audience Member
Thank you for the Very wonderful presentation. So I have one question. So as we all know, for implantable electrode, either for stimulation or recording, after
we implant the electrode, the scar tissue
Audience Member
will grow around the electrode and especially for recording, the signal we get will become smaller and smaller after long term implant. How do you solve this issue?
Zach (Neuralink)
So for context, I'm Zach, I lead the microfabrication team on brain interfaces. I don't think we can see solve it specifically, but one thing, one advantage we have is both the flexibility and the small size of our threads to try to limit that scar tissue and that damage. And some future work that we have started working on, that we'll continue working on is pushing the size of the threads down just to try to limit the immune response and really limit that scar tissue growth.
Audience Member
Actually, I want to follow up.
So do you think it will be
Audience Member
helpful to actually load some drag on the surface of your electrode or some other way?
Well, I think like maybe the just the question is like what sort of signal degradation have we seen over time? And you know, basically does it work a year later, does it work two years later? It does.
Zach (Neuralink)
So yeah, yeah, so that's a good point. So in terms of thread longevity specifically, really the gold standard that we can use to assess is the data we have from our animal participants. And so for that I'm not sure if it was mentioned before, but the longest data we have right now is for an animal participant who has 600, went 600 days with useful functioning channels where we were doing something useful with the signals for bci. And then with the newest version of our device, we have sort of a collection of participants who are at or near one year of data and completely useful functioning BCI from that as well.
Joey (Neuralink)
Thank you.
Audience Member
If I may add one more thing. So you mentioned potentially having drugs to kind of reduce inflammation. So one of the things that we are actually actively working on is having some sort of biological coding to either reduce inflammatory inflammation or make them slippery. So you know, you mentioned, you heard from the presentation that one of the challenges that we have is removing the threads from these neomembrane tissue that are formed after implantation. So there are programs like that where we're really looking at kind of incorporating some of the learnings from biology and these coatings into our threads so that we can hopefully reduce inflammation as well as make it easier to extract.
Also continuing to reduce the size of the electrode. So when the electrode gets really small, the sort of inflammation response of scar tissue becomes minuscule. So it's like a very tiny electrode the body basically ignores. This is really impressive.
Audience Member
Congrats to the whole team. So, as you of course know, one of the problems with current electronics electrodes is they're rigid and they move around.
So you have these neural non stationarities
and I think many of us had hoped that with these very thin threads
Audience Member
they would maybe move more with the
brain and you wouldn't see that.
Audience Member
But from the data we showed over many hundreds of days, there was a lot of variability. So can you speak to how much
do they move and do you have any idea of like, why does it move?
Can you stop it from moving?
How stable are the signals hour to hour and day to day?
Hi, I'm Bliss, I'm one of the leads of software groups in the Brain Interfaces team. In the particular plot you were mentioning before, what we were showing was the average firing rate recorded per day on a particular channel. It's, as you well know, pretty complicated to understand. If you're recording from the exact same neuron day after day after day. It could be, for example, that you're actually picking up a different neuron day to day and that's why you get the change in firing rate. We don't think this is at least the majority cause of the situation here. The reason is that if you look at sort of the spike shapes day to day, even when the average firing rate is shifting a lot, you still see sort of stable spike shapes. That's obviously not a fully bulletproof story, but at least gives some confidence that it's not actually different neurons you're picking up. However, there still is very much a chance that that could be the case in at least some part of the robustness, non stationarity story. Yeah, cool, thanks. Yep, thanks for the question.
Yeah, to be clear that like this electrode position is actually fairly stable because you've got these very tiny, basically very tiny wires with. And there's some play in the, like you've got, you've got the device attached to the skull originally, but then you've got this long, so tiny wire with kind of a coiled section. So it's, it does tend to basically stay in the same place.
Leslie (Neuralink)
We also asked for questions on Neuralink's Twitter, so we'll be interleaving some of those. Supe wants to ask, what could Neuralink help people with that most people don't realize?
Well, I mean, once you're in there, you know, there's a lot you could do. So, you know, you can obviously measure temperature, so you could do very early detection of a fever you could not measure pressure. I think you probably detect that
the very early, the very beginnings of a stroke because you can see sort of like electrical signals starting to go sort of haywire. So there's actually probably a lot of just general health monitoring that you could do once you're in there, you know, and with very simple sensors.
You guys all did a great job of distilling a lot of complex engineering
Joey (Neuralink)
and science and making it wonderfully clear.
So great job. I wanted to ask a little bit about the simulation. I guess for the phosphide and for the evoked movement. Are you think is this more like local stimulation? Is it juxtocellular? Are you steering current around? How many cells are you activating? How much current are you using? I'm just curious what the scale of this is and whether you have a lot of precision or a lot of, you know, you have pretty profound behavioral effects too.
Dan (Neuralink)
Hi, yeah, I'm Dan. And how many cells you still stimulate with a single electrode is dependent on the impedance of the electrode, size of the conductive pad, how much current you deliver, the frequency, all these factors. So there's a great deal of variability that we can use to customize the shape of a phosphine or the shape necessarily, but maybe the intensity of a phosphine. We think with our current current electrodes, at least in code, back of the envelope calculation would be something like about a 50 to 100 micron diameter sphere of cells are being stimulated in a visual system. The smaller that sphere, the smaller and more specific you can make a particular phosphine. Basically the smaller the pixel in the
Dan (Neuralink)
So there's plenty of scope for customization of that.
This actually also it's possible to get to a much higher like effective pixel count by controlling the field, electric field between the electrodes. So it's not necessarily, just not a one to one relationship. You could actually dynamically adjust the field and simulate far have a, have a very high neuron to electrode ratio. So try like, could you get like you know, maybe 10 to 1, 100 to 1 potentially. So a megapixel type basically. Can you see normally? I think people would want to know that. I think that is one of the possible outcomes.
Zach (Neuralink)
Hi Lon. This is amazing.
Can you talk about the longevity of the implants itself?
Audience Member
Also how would the material of the implant would react with the brain tissue
Alex (Neuralink)
or density of the bone or bone structure?
Audience Member
Yeah, happy to talk about this. I'm Jeremy, engineer on the Brain Interfaces team and I think it's good to start with data. So like Zach mentioned, we have an implant that was, you know, a Monkey was performed BCI for 617 days, that was pager before being upgraded to the laser device. For our current version of the device, it's lasted for almost a year. And then for accelerated lifetime tester that Josh kind of talked about, we have data from our implants, from the previous version, eight years of accelerated time, and from the current version, four years of accelerated time and counting. So that's kind of starting with the data. Those devices are still lasting and still going theoretically. There are kind of three fundamental factors that contribute to the longevity of the device. One is going to be the seal, that hermetic enclosure of the device. Two is going to be the battery and internal electronics. And then three is going to be the threads that Zach talked about a little bit and the channels being able to functionally record signals from the brain. The seal, we think will far outlast the other two in terms of the bottlenecks. So the seal, just theoretically, I think Josh mentioned that it is a thermoplastic polymer material. So there's going to be a very small amount of moisture that diffuses through it over time. And we think that that will last, you know, 20 plus years easily in terms of just that property. And like I said, we have not seen our seals fail with our current version of the device yet. So we haven't really pushed the limits here for the battery and internal electronics. That's really based on usage and how much runtime you want. And we are working currently on getting data to project out even farther. But right now we believe that we can, you know, achieve 80% runtime at the three year time point, which would be about, you know, three and a half hours for a four hour runtime. But we're, like Avinash mentioned, we're, we're doubling that very soon and quadrupling as well. We have plans to do that. So internal electronics really aren't the bottleneck either. And so really we're attacking the threads themselves and longevity of those channels that Zach, Zach can kind of talk about some of the improvements that we're doing to increase that longevity.
Zach (Neuralink)
Thanks. Yeah. So as sort of mentioned before, we don't necessarily have an endpoint, as Jeremy said, for the testing of the threads. That being said, we are focusing on longevity because we think this is an important issue to solve. So one thing that we're doing in parallel with the current device is aggressively pursuing amorphous silicon carbide insulation of the threads, which we believe will take us well beyond five years of longevity, but of course still to be tested. And in parallel with that, we're just starting to look at atomic layer deposition, which we think could even push longevity of the threads much further and deposit very thin layers to keep the flexibility of the threads and that advantage there. So along with that, we're also of course having to design and validate very robust bench top testing to model really in vivo conditions and look at channel degradation. So that's what we're looking at for longevity of the thread threads. And then I think you asked about BioComp and I think for Biocomp, essentially all the materials we're using right now I can say are at least bio stable. And we send out testing for biocompatibility very often. And essentially what we're doing is we're using in many cases known materials from literature that academic labs have already started to look at and sort of jumping on that and using that as a starting point.
Thanks for answering that.
So we have another question from Twitter.
Alex (Neuralink)
This is from David and he asked the team, what are the biggest lessons you learned since the previous presentation?
It's been about two years. I'm sure there was a lot of engineering done. So yeah. Anyone want to answer what we learned in the last two years?
Christine (Neuralink)
So one thing that we've learned in the last couple years is, is how much the brain moves on the human scale compared to when you start small, when you make brain proxies and a lot of research starts with rodents, the brain does not move that much and you get a human and the brain can move like hundreds of microns or more. And when our threads and needles are so small that motion when you zoom in looks like a mile.
Leslie (Neuralink)
I think to add to that one thing is how I guess dynamic the implant environment actually is. So we've talked about like when this implant site heals scar or new tissue might grow and fill in the space and that'll affect like how our threads might interact in that space. So that's why we've emphasized so heavily the importance of designing accurate proxies. So instead of having to wait months for an implant site to heal, you can hopefully learn that information in hours.
Alex (Neuralink)
I'm Alex on the robotics team. I think one of the things we've definitely learned within the engineering teams is the importance of really continuous validation and testing. Where we're building say motion systems that are precise to single digit microns, we need validation and test systems that we trust even More than that, to prove that they work reliably. And putting just as much focus into those validation and test systems and designing those alongside our products, I think, is one thing we've definitely learned.
Another thing that we learned, I think, as part of BCI or the Bend Control and Algorithm, is that again, building a prototype and making it work with only one monkey, one pager was a great, maybe a success, but also relatively easy to making it work every day for all the other monkeys. So actually making it a product is something that it's not easy, but we are learning how to do it.
I mean, I've learned that the brain is really squishy, like way squishier than you think. It's not like, you know, cauliflower or broccoli or something like that. It's more like a water balloon and it's moving in your skull, like a lot. So you got a squishy water balloon in a coconut is maybe a good way to think of it.
Zach (Neuralink)
Given Bluetooth's bandwidth limitations, have you considered other technologies for wireless communication?
Hey, yeah, I can take the first part of this question and then I'll let Matt answer the second part of it. It's a great question, especially as you think about how to increase and scale the number of channels that we want to record from. This becomes increasingly a bottleneck for the kinds of work that we want to do. We're thinking about this in a couple ways. One is just directly improving the underlying radio interfaces, and I'll let Matt talk about that in a second. The other way we're thinking about this is how can you be more efficient with the data you send off the implant? And I think the first version of that is compression. So just taking your data, looking at the characteristics of it, find out a way to represent it more efficiently and just send off that compressed stream. So for reference, right now our Bluetooth bandwidth is around 150 kilobytes per second. The compressed stream of data that we send off the implant is around 50 kilobytes per second. So we're doing fairly well there so far. But when you start thinking about 16,000 channel devices, that won't get you all the way there. So some other things that can help on the compression side are to actually just send out the output of the machine learning model rather than the input required to actually run it. So one thing we've been trying in the background here is called Decode on Head, which is essentially taking the machine learning models that right now we're running on MacBooks that our monkeys are gaming on and moving those to actually run on the implant. And this is the like a super cool engineering problem. If you want to talk about how to make complex neural networks run on what is the equivalent of a garage door opener, come talk to me. It's fun. Yeah. So that's another way to solve this problem is to basically do the computationally intensive work to just get the raw signal that you actually care to use to control something and then send that thing out of the implant. On the radio side, I'll hand it over to Matt.
Matt (Neuralink)
Yes. So to answer your question, we are looking at other radio technologies. Technologies. One in particular is 500 MHz band with ultra wideband at a couple different frequencies. So this has an advantage in terms of the bit rate that you can achieve. It's on the order of 6 to 8 to 10 megabit. There's also a latency improvement that's quite substantial. And there's also another wireless technology that we're looking at and W band.
Leslie (Neuralink)
Hi, thank you all for really clear and compelling presentations. Something that struck me in one of the earliest talks, I think it might have been DJs was this vision for the ability to acquire new complex skills via these BCIs, like the ability to perform Kung Fu. And that reflects the fact that the brain is fundamentally a learning machine. And yet many of the technical solutions presented later framed were framed in such a way as to try to correct for the way the brain changes over time over longer time scales, drift over the course of days, or the way that the tissue might heal over time. I was curious what you your vision collectively was for developing out this technology that interfaces with a fundamentally plastic system that changes in complex ways over a variety of time scales. Days, months, years.
It's a tough question. I think it will be kind of maybe bidirectional learning in some way in there. Sometimes scores that we will might fix our algorithms and we prefer to have like more stable kind of performance. But of course, if the over time the person in the brain will learn how to use better the bci. We'll need to update our models. So there will be kind of in an interactive kind of relationship in some way to learn even new tasks. These probably will be something over time we'll need to learn what the person kind of learn how to interact with the computer and then build the appropriate interface UX and also the UI and build algorithms that will help him to control what we want.
Audience Member
Just one thing I'd like to add on to what Nir said Yeah, it actually is an advantage in some ways that the brain is plastic and learns, and that can help us because we actually have to do less work and the human in the loop will actually learn how to use our device better. But one of the advantages of our particular approach and device is that we are trying to do an extremely high channel count device so we can, you know, uniformly distribute electrodes over a functional region. And then it doesn't matter so much whether things move or shift over time. We can offload that to software and so we can build algorithms that change over time as well. And so both those things are actually, I think, advantages to our particular approach.
Leslie (Neuralink)
We have another question from Twitter. Juan wants to know, what career path do you suggest for somebody that is just getting out of high school if they want to work at neuralink in the future?
It's really any of the skills that we described. So we're developing new chips. There's material science, there's software, obviously animal care. It's really all the things that we listed in the neuralink careers that would
Audience Member
Yeah, I'm actually very fond of saying when you flip through any college booklets and look through all the majors, I think you can pull point to every single one of those majors. And there's someone at this company who either is an expert or, you know, have majored in that. So it really is truly, truly multidisciplinary endeavor. And I think, you know, just focus on whatever you're, you know, passionate about or whatever you're talented at, and then just, you know, pursue that as deeply as you can. And then there's definitely going to be a place for you in your own building neural interfaces.
Leslie (Neuralink)
Hey, we got to see the monkeys doing telepathy. But could you say a little bit more about the animal behavioral training kind of their lives and day to day processes? Sure. I'm Autumn. I am head of research services, which includes our animal care program. And as an animal welfare scientist, this is a topic that I'm deeply interested in. So our training program is outfitted mostly with behavior analysts who help us think about how to remove any of the potential aversives or frustrations from our training. We think about conditioning as the primary, which includes positive reinforcement as the primary way to train.
Christine (Neuralink)
Let's see, what else can I share with you?
Leslie (Neuralink)
Yeah, yeah. I mean, that may not be part of the behavioral training itself, but we think of animal welfare assessment in the framework of the three Rs, which is referring to refinement, replacement and reduction. And so when we think about refinement, behavioral training does apply in that way. And where we want to remove, specifically in research, restraint is one of the things we make a very top goal to remove. So you saw a lot of videos today where animals were walking up to their stations because we worked really hard to, to remove any requirement to restrain the animal. Anything else?
Audience Member
Well, just on top of the last point, you said just as an engineer here, one of the things that is really inspiring and really cool about this place is that we do get to work on a lot of technological innovations that directly translate to greater independence for the animals when they're engaging in these tasks. So as you saw monkeys charge just by voluntary walking up to a branch, they play games in their home habitat with a laptop computer voluntarily. And the fully implantable, fully wireless device, the inductive charger, all these things enable that kind of experience. And so this is one of the very cool parts about working here is we do get to innovate on things like that.
Leslie (Neuralink)
Definitely helps to work with a group of engineers who can like really make cool stuff for monkeys to be able to do easier behavioral training.
Audience Member
So I guess to answer the previous question about what you can study to be part of neuralink, I guess monkey engineering, you can add to that monkey business.
My question is on upgradability, which you guys mentioned quite a bit. So in that procedure, in there's some kind of explant procedure and then you're going to put in a new set of implants. So could you talk about the damage possible, if any, tissue damage from the explant procedure? How long you have to wait? Do you implant the same areas and what's your like brain scanning for the implant procedure in terms of upgrading it? I don't know how many questions I
Alex (Neuralink)
so I can start to speak to some of those. So I work a lot on upgradability and those explant processes and designing those to be better. The goal that we're working towards is that as I mentioned in the presentation, it's really just as easy to upgrade an implant as it is to initially install. We didn't, we didn't show many of those explant examples today, but we've come pretty close to just popping out an implant and reinstalling another one in the exact same location. Definitely, definitely the goal. We are installing the implant in primary motor cortex, which is a valuable area for interacting with a device like this. And so we. The goal is to implant in the same location. Maybe if you expand out to other applications Then you'd be interested in moving somewhere else. But we definitely want to be able to insert into the same area. In terms of damage, the. I think that the damage that we care most about is damage within the brain. And what we found, and we talked about that, that challenge of the tissue layer on top of the brain. And we're well on our way towards figuring that out. But because of the thread's small size, the sort of scar capsule within the brain is so minimal that they are actually removed quite easily. And so we see useful signals even on the second or third time that you've placed an implant. And I think some of our BCI folks probably speak to that. We do have monkey participants working with their second devices and really making use of those.
Audience Member
So one, two questions. One was somebody had asked the question about the plastic. Have you noticed any plasticity from a behavior perspective from any of the monkeys? Or is it too soon to tell? Or there haven't been any observations.
From the monkey behavior. We see that it takes them a while to learn how to, of course, to train on the test, but also when they are implanted. And it's relatively quickly for them to ramp up and get to a high performance of brain control. With Pedro, for example, after a few days he was able to, like, three days already able to learn very quickly to use the device. He was trained on the task, of course, from his previous implant, but with the new one, he was, after three, four days, he was able to control to close the performance of he had with the previous implant.
Audience Member
But have you noticed anything on the advance, which means the brain has outpaced the neural network that you're running?
It's hard to say. No, not really.
Audience Member
Okay, so I have another question, which is more about the electrical side. So you talked about 10, 24 channels being recording. Are you transmitting the raw signal or was it only the three spike events that. That you were talking about, the low, mid and the high, or is it the raw, entire raw form waveform that you transmit?
Julian (Neuralink)
Hi, I'm Julian. I can speak a bit about this and maybe Avinash wants to contribute. But our chips see the raw signals, but the one we transmit out typically spikes, and we detect those spikes in real time on the chip. This massively compresses the data, I guess. Yeah, moving. We're making improvements to that, but we can request. We can request raw samples. Sometimes we also process particular statistics or other data directly on the chip and then send out the calculated values. So there are many ways to sort of play with the Data.
Audience Member
Yeah. So at least with the current N1 system that we have, which relies on BLE radio, there is a bandwidth limitation. So you can't actually stream raw data from all 10, 20, 24 channels. But just kind of to give you a little bit of a history of how our compression algorithm, the spike detection algorithm was developed, we did have sort of a wire system. There was a paper that we published with the USB C connector that you know, streams all those signals through a high bandwidth wire connection. So we did have kind of those development platform to be able to see the raw signals and know we, which set of information that we want to extract that are, you know, going to fit within the bandwidth of the radio as well as is useful for BCI control. And you know, also just sending data wirelessly does cost a lot of energy. So there's any opportunities we have to reduce that burden. You know, we try to do basically have all that compression closer to where the electrodes are as possible.
One thing that isn't obvious is that the actual bit rate that you need to control a phone or a computer is actually very low. So I think we might have the record for bit rate, is that correct? We think we do maybe so on the order of 10 bits per second. So that's super slow. But if you think like when you're inputting data into a phone, like how fast your thumb's moving, how many thumb, what's your thumb taps per second. Pretty, pretty low. And I mean basically our thumbs are like two slow moving meat sticks that we, you know, do this and it's like there's really a load, it's like a low bar is what I'm saying. So for at least for output it's, it's a, you're getting, get 10 bits per second, you're holding ass. So and that's, you don't need Bluetooth anything. So you could practically send it out with beeps and bops, you know. So it's not, if you go, if you're going like a high bandwidth visual now you're, you know, maybe going to megabit plus. But it's, it's all well within Bluetooth or anyway it's just that is what I'm saying is that's not a constraint. The data rate. One other sort of like maybe notable item which we talked about in the presentation, but we think we can probably solve for doing the implant without cutting the dura. We can just do basically a bunch of holes through the dura, which is like, the dura is like the Big thick, orange rindy thing that contains the, that's up against the skull. If you don't pierce the dura, you know, if you don't cut the dura away and instead you have a bunch of tiny holes and insert the electrodes through the tiny holes into the brain, then the recovery time is ridiculously fast. You know, you're not really losing much in the way of cerebrospinal fluid. It's, it's, you could, in theory, I mean, this could be like a, the whole thing could be a 10 minute operation like Lasik. Like, it's fast. It's not like a big laborious thing. It's super fast.
Sam (Neuralink)
Just going back to the long term
use, I'm wondering if you have any pathology looking at scar tissue from many animals that have had long term implants. And along that lines it seems like there might be a little bit of a gap between use in medical conditions and healthy individuals from a safety perspective.
Audience Member
I didn't quite catch the last question, but I'll hear the first one and I'll ask you to repeat the second one. So the first one is, do we have pathology from long term use animals? We absolutely do. We don't have any pathology from our monkeys, which we upgrade and you know, are still going. We have other studies that are primarily to determine safety. And so we do have histopathologic endpoints that we determine. The scar tissue formation around the threads themselves in the brain is typically negligible. Like it barely reacts to the threads at all. So that's very promising in terms of the scar tissue formation over the cortex. So this neomembrane growth that fills in the, the areas that Elon and Alex were mentioning, that we remove with our current operation. Those we, we do, you know, evaluate that scar tissue, but it isn't, it doesn't pose a problem in any way. It's not a continuous reaction to a foreign body. It's just filling in tissue that was removed. And if you could repeat the, the second question.
Dan (Neuralink)
I didn't hear that.
Sam (Neuralink)
Yeah, the second question, really following up
on that, seems like there might be a little bit of a gap in use in healthy individuals from a safety perspective. You know, I think people mentioned that they might be interested in trying prototypes, but just wondering what your perspective is on trying to lower the safety risks.
Audience Member
Yeah, it's a great question. So in terms of, really it's about the long term use of the device. So, you know, we have devices that have been implanted, like I said, in monkeys, where you know for many years where we see no behavioral deficit at all. So then this first is a question of how you evaluate safety. So you have histopathologic endpoints you can evaluate, but we're also looking for cognitive deficits or behavioral deficits as well. And we don't see any of those in our animals, which is an important point. In terms of the histopathologic endpoints, they look really, really great. The challenge is one of explanting the device, which is why we're putting so much effort into the reversibility efforts and our through dura insertions. So when removing the device, that's when you potentially, potentially could cause damage. And so we are doing, we have a lot of ongoing studies right now to really minimize the risk of that, but we don't think it's a substantial risk with our current approach. And like I said, pager was upgraded with the previous surgical approach and is doing great. So clearly it is, you know, can be perfectly safe. But proving that beyond a shadow of a doubt for humans is something that we're still working to do rigorously. Did that answer your question?
Dan (Neuralink)
Thank you. So thank you for a very deep
dive on many of the different aspects
Dan (Neuralink)
of the device and the system. It's very impressive to see all the
engineering work that's gone into it. You just mentioned about bitrate.
Dan (Neuralink)
As the prior bitrate holder, I can
confirm you have indeed shattered my record. So congratulations on. I think I saw a peak of
Dan (Neuralink)
7.4 bits per second. Well done. My question is actually around clinical trials and the fda, to the extent that you can share, I gather that device removal or maybe electrode removal is one
of the concerns that the FDA highlighted. Is there anything else you can tell
Dan (Neuralink)
us about what the FDA was concerned
about or had questions about with respect to your IDE submission?
Audience Member
Yeah, I mean, we can probably talk a little bit. I mean it's. These are really challenges that we have broadly so exploitation, safety, Proving that right rigorously for humans is something that we definitely is one challenge and was something that the FDA commented on. Other things that they do ask some really great questions. So other things involve things like the thermal bench top testing of our implants. So obviously it's important that our implant doesn't damage the tissue by overheating. So having really rigorous and valid bench top testing for that is very important. It's actually something that we'll redesign to be even more accurate. Now it's also the case that, you know, they ask a lot of very hard questions on Biocompatibility chemical characterization. So we've done very rigorous testing for that. But you know, they, they do ask a lot of questions about getting into the weeds of the data and making sure that there really is no chance for any toxic chemicals or bio incompatible materials to be in the brain. So these are all things we're working with, you know, to, to just prove again, above and beyond, beyond a shadow of a doubt. One thing that's maybe worth mentioning here is that it can be difficult to appreciate the novelty of our product. So the surgical robot and the thin film array in particular are quite new and unlike existing devices. And this means that we can't rely heavily on literature to support the safety and efficacy of the device. So we do spend a ton of effort in designing and performing testing on our devices so that we can rigorously prove the safety of them and we can't rely just on another product or on some paper. And that's something that we're not willing to compromise for our first human participant and working very hard to do.
I think if you ask a question like in my opinion, would I be comfortable implanting this in someone, one of my kids or something like that at this point, if they're in a serious, like, let's say if they broke their neck, would I feel comfortable right now doing it? I would, I would say we're at the point where, at least in my opinion, it would not be dangerous.
Christine (Neuralink)
Hi, thank you for the presentation. So I have a non technical question.
Leslie (Neuralink)
Are you collaborating with people with motor disabilities?
Christine (Neuralink)
And if so, have you shared any ideas of applications that they would be excited about?
I can take the first part of this. I'm not the best person to speak to this, to be honest, but there is a consumer advisory board we have made up of a number of people that have various conditions, including tetraplegia, and they give advice to us on a number of topics. Just as an anecdote. Someone came to the office maybe six months ago and they were telling me what they most wanted to do with their neuralink device. And there were two things that they said. One was they wanted to be able to trade stocks day to day, to be able to beat their brother. And the second one was they wanted to be able to play shooter games. So I think what was most shocking to me about that encounter was the normalcy of that. And I found that conversation truly inspiring. So you know who you are? The person who came and talked with me. Have a great day. Yeah,
You know, something that's we've talked about, but it's maybe should be reemphasized. We are doing, we're building up a production system for the devices. So we're building up, bringing up the production line, making large numbers of devices. We want to make thousands, ultimately tens of thousands, then millions of devices. So the progress at first, particularly as it applies to humans, will seem perhaps agonizingly slow. But we're doing all of the things necessary to bring it to scale in parallel. So in theory it should progress should be exponential. So thank you, that was a very cool presentation. So one of the stated goals was recording from everywhere in the the brain, being able to record from and perturb any location. So it seems like currently it's all cortical.
Zach (Neuralink)
And I'm curious, with the current device,
is it, is there any sort of long term goal or idea as to extending it into going deeper in the brain? I mean for neuropsychiatric disorders, for memory, all these things are much deeper, several centimeters. So I'm wondering what's the time scale? If you were to give a very rough estimate of when I can expect to see an erlink product that goes that deep.
So I mean the fundamentals of the device in the skull will stay essentially the same because the, as I said earlier, the device in the skull is very much like a smartwatch. Essentially it's got, it's a battery, radio, inductive charger, computer, and then you've got the little wires and so you need to make the wires longer and you'd have to have a deeper insertion needle for the robot. But this really is intended to be a generalized I O device. So apart from the tiny wires being longer and the surgical robot needing a longer needle, in theory you should be able to go anywhere because it seems to me that part of the robot is trying to detect where the blood vessels are and then avoid them. Correct. Would that be possible at that scale? I mean, certainly not just visually, but maybe there's some other way of detecting it. Is that a current goal and do
Dan (Neuralink)
you expect that within, I suppose, the next decade?
Definitely, yes. I'm Ian, I run the robotics and surgery engineering team here. Like of the three axes that DJ mentioned, one of them is, you know,
Alex (Neuralink)
access to more areas of the brain.
So the robot team thinks about this a ton.
In terms of what sensors do, you need to essentially go past the surface. And so in this case you're right
Alex (Neuralink)
that right now we can really only
see down maximum about a millimeter. I think within the team there's questions of what's Best to use next. But like ultrasound and photoacoustic tomography are two that come to mind as things that can get centimeters deep, essentially. But it's a super interesting problem. You sort of need deep imaging and some ability to steer to at least avoid large vasculature deep down. Yeah.
Christine (Neuralink)
Or if we can make our needles and threads small enough in a way that we can still be precise and accurate at a deep depth, then maybe you don't cause a bleed if you hit a vessel.
Yeah, I think that's really the ideal situation. If the threads are really tiny, they can actually go through a blood vessel. And it's okay if they're tiny enough so we wouldn't need the blood vessel imaging in that case. I actually am slightly optimistic that that is achievable.
Christine (Neuralink)
Matt, you could probably speak more to this, but with DBS currently, it's kind of just like send it.
The current approach involves a wire that you blindly pass in.
Julian (Neuralink)
That's massive compared to our threads, orders of magnitude bigger.
Matt (Neuralink)
And so that's a low bar for
us to clear as well. I guess people don't realize, like, for the deep. Right. Simulation. Just how big the hole is. It's a. I mean, what is it like? I mean, basically, in current deep brain stimulation, how much of a borehole is drilled in the brain? Yeah.
Matt (Neuralink)
You're drilling a 14 millimeter borehole and then passing a 2 millimeter wire 6
centimeters, 8 centimeters deep into the brain.
Matt (Neuralink)
So all blindly hoping that you don't hit a blood vessel, Telling the patient
up front, this might be good for
Matt (Neuralink)
you, and there's a 1% chance your
brain is going to bleed in a
That is current technology that is happening right now. So doing better than that is. We can definitely do way better than that.
Christine (Neuralink)
Our needle is 40 microns.
Thanks again for the phenomenal presentation. I thought it was fascinating how rapidly you could test all of these electrodes, but it begs the question about, like, what your fault tolerance is. If you run these diagnostics and it comes back that you have something that's either shorted or high z, how many of those before you get degraded performance? And the second question is, when you're actually inserting this device, we saw examples of the electrode going in and then, like, looping back on itself. But it looked like that was something that was assessed basically by slicing the synthetic material. I'm curious what you're doing to validate the insertion of all these electrodes sort of in vivo. How do we know that that's not happening on an actual patient.
Sam (Neuralink)
Yeah, I can answer that sec the second one. So like I mentioned, we can. So we weren't actually sectioning in that case. We have a really cool micro ct. So I mean it's essentially like a CT scanner. So that's just in intact proxy that we put in this machine and we can, you know, take a picture all the way through it. And like I mentioned before, like we can make a proxy where it happens, you know, that that looping back happens every single time. And then we can make one where it never happens. And we've pinpointed roughly now where actual tissue falls in there. And so our current plan for, you know, validating and confirming that is making proxies where it, you know, happens really easily, much worse case than any, you know, any tissue could possibly be, and then designing it such that it never happens in that scenario. And that'll give us the doing that enough times and with a weak enough proxy that'll give us the confidence that this isn't actually happening.
This is the next gen needle.
Sam (Neuralink)
Yeah. And this is the next gen needle. We don't see this problem at all
Alex (Neuralink)
with the current generation.
Julian (Neuralink)
And I'll take a stab at your first question. So to clarify, you're asking what happens if there's a fault on a particular channel or something?
Julian (Neuralink)
Yeah. So the nominal scenario is that basically the impedance will stabilize pretty quickly within the brain. And even at that level we can record great signals. We see lots of spikes and we can use that for BCI because we have so many channels, like 1000 now, 16,000 later, we can actually run our models with far less channels than we actually have. So it doesn't matter if one channel dies here or there, we can still do really good decode. I'm not sure if we have official numbers on how many channels we need, but it's like we have an order of magnitude more and the more we have about it, we can already do a lot with what we have.
Maybe just one or two more questions.
Leslie (Neuralink)
Yeah, I have a question about your very, very long term inspiration to have this high bandwidth communication with advanced AIs. So it seems like the advanced AI would need to understand the human's most complex thoughts and emotions. And that's what neuroscientists are trying to do. So do you have any ambitions to tackle neuroscience beyond neuroengineering?
Well, I mean, I think we're going to make the input output device and the software interface with it and I think probably suggestion earlier we'll try to open source as much as possible so people can take a look at it. And I think there will be a lot of others that build upon the work that we're doing. You know, the same way that if you make a microprocessor or CPU or computer that people will write lots of software that runs on that computer. So but if you don't have the computer, the software's moot. So we're making the input output device with the computer and then I think probably there will probably be a lot of other organizations, companies that build upon that foundation. So yeah, I mean, one of the things that I sometimes wonder is that if you do have a whole brain interface and you can record memories,
getting into black mirror stuff here, but this could be one of them.
Audience Member
I also think it's worth mentioning an important point which is that neuralink didn't come out of nothing. There's decades and decades of research in the medical academic field that has really set the foundation for what is possible by putting these electrodes in parts of the brain and being able to read those signals, decode it for mapping it to some application. And you know, being in academia before coming to neuralink, you know, I do think that there's a lot of opportunities for kind of the field to advance at a much rapid rate by having just better tools for observing the dynamics that are happening and then engaging with it in a seamless way. And I think it was Ian who sort of mentioned that, you know, it's almost as if like we're kind of building an oscilloscope for the brain, which I think is like kind of a beautiful analogy of just giving us a bit more abilities into peering into the dynamics and using those information. Learn that to, I don't know, hopefully understand like what makes us and how the brain works and you know, the whole shebang.
Julian (Neuralink)
The presentation covered keyboard and handwriting based input methods. How do you plan to develop an input model that will achieve much higher bandwidths for complex tasks in humans?
This is a tough question and we start exploring this with monkeys. As you saw, we have like a multiple. We train many monkeys on very different tasks. It's still an open question that we are after. I think hopefully once we get to our first participant it will be easier to investigate. One of the options we are exploring, as we showed, is to decode handwriting directly. This is one a work that started at Stanford and we are exploring here and trying to expand. There's also a different, in addition to just Decoding different things from the brain. We also try to provide the user different maybe user like interfaces. For example, we show different type of keyboards. Maybe also swipe and other things that can help increase the communication rate. So we are kind of tackling those in two dimensions.
Just one other thing to add in that direction. As pointed out by many people here so far, this is a general I O system that you can sort of plug and play in different places of the brain. There's other areas of the brain that can help increase bandwidth. For example, language or speech centers that can help you much more seamlessly communicate. For example text, if that's your main thing that you're trying to do.
Audience Member
Yeah, just.
I think just having this general input operate device will just so gigantically improve our understanding of the brain. It's hard to. The words can barely express. Like, you know, right now we're just guessing a lot of what's going on in the brain. But if you have direct IO, it's not. No more guessing. What we'd learn about the. What we will learn about the brain with such a device in wide use is absolutely many orders of magnitude more than we currently understand. So I guess on that note, thank you for coming and thank you for watching online.