Hey everyone, Ernis here, and welcome back to PaperLedge! Today we're diving into a fascinating paper that tackles a huge problem in the world of AI: How do we make these massive language models, like GPT-3, actually usable without breaking the bank?
Think of it this way: Imagine you have this incredibly smart, super-general AI, trained on the entire internet. It's like a genius who knows a little about everything. Now, you want to teach it a specific skill, like writing marketing copy or summarizing legal documents. Traditionally, you'd have to retrain everything it knows, which is incredibly expensive and time-consuming. It’s like re-educating that genius on everything just to get them to focus on writing catchy slogans.
This paper introduces a clever solution called LoRA, short for Low-Rank Adaptation. The core idea is brilliant: instead of retraining the entire massive model, LoRA freezes the main part of the model, which is like preserving all that general knowledge our genius has. Then, it adds a small, trainable "add-on" to each layer of the model. These add-ons are like giving our genius a set of specialized tools and a quick training course specifically for the task at hand.
Here's the real kicker: these "add-ons" are tiny compared to the original model. The paper claims that LoRA can reduce the number of trainable parameters by ten thousand times compared to retraining the whole thing! And it also reduces the GPU memory needed by three times! That's a massive saving in computational resources, making these powerful models accessible to more people and organizations.
But does it work? The answer is a resounding yes! The researchers tested LoRA on several popular language models like RoBERTa, DeBERTa, GPT-2, and even the behemoth GPT-3. And guess what? LoRA performed just as well, and in some cases even better, than retraining the entire model. Plus, it's faster to train and doesn't slow things down when you're actually using the model, which is a common issue with other approaches.
To put it in perspective, it’s like having your genius retain all their existing knowledge while quickly mastering a new skill – without any performance hit. The authors also explored why this approach works so well. They found that when adapting a language model to a new task, only a small part of the model's knowledge actually needs to be changed. This is why these tiny "add-ons" can be so effective.
Why does this matter?

For AI researchers, LoRA offers a way to experiment with and fine-tune these massive models without needing a supercomputer.


For businesses, it means being able to leverage the power of large language models for specific tasks without the prohibitive costs of full fine-tuning. Imagine tailoring customer service chatbots or creating marketing campaigns more efficiently.


For developers, the research team released their code and model checkpoints, making it easy to integrate LoRA into existing projects.

Key Takeaways:
"LoRA allows us to adapt gigantic language models to specific tasks with a fraction of the computational resources, making AI more accessible and practical."

LoRA dramatically reduces the number of trainable parameters when adapting large language models.


It performs on par with or better than full fine-tuning, while being faster and more efficient.


The researchers provide code and models to help others use LoRA.

Questions that pop into my head:

How does LoRA compare to other parameter-efficient fine-tuning methods in different scenarios?


Could LoRA be used to adapt models to multiple tasks simultaneously?


What are the potential limitations of LoRA, and are there tasks where full fine-tuning is still necessary?

So there you have it! LoRA: a simple yet powerful technique for making large language models more practical and accessible. I think this is a really exciting development, and I'm curious to see how it will be used in the future. What do you all think? Let me know in the comments!Credit to Paper authors: Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen

Alright learning crew, Ernis here, and buckle up because today we're diving into some seriously cool research that's making AI more accessible to everyone!
Imagine you're trying to teach a super-smart AI, like a giant language model with billions of parameters, new tricks. Normally, this is incredibly expensive, requiring tons of powerful computers and a small fortune in electricity. It's like trying to teach an elephant ballet – impressive, but not exactly practical for your average Joe.
Well, some brilliant folks came up with a clever solution called QLoRA (pronounced "kew-lora"). Think of it as a way to teach that elephant ballet with a tiny, super-efficient training program. This research is all about how to fine-tune these massive AI models using way less computing power. The headline? They managed to fine-tune a 65-billion parameter model – that's HUGE – on a single, relatively affordable GPU! This previously would have been completely out of reach for many people.
So, how did they pull this off? Here's the breakdown:
4-bit NormalFloat (NF4): They created a new way to represent the AI's knowledge using only 4 bits per piece of information. It’s like compressing a huge music library into a format that takes up way less space without losing the overall sound quality. They specifically optimized this compression for the kind of data these language models use, making it super effective.
Double Quantization: They even compressed the compression information! It's like zipping a zipped file – squeezing every last bit of efficiency out of the process. By quantizing the constants used in the initial quantization, they further reduced the memory footprint.
Paged Optimizers: Imagine a video game console that only loads parts of the game level as you need them. That's what paged optimizers do for AI training. They cleverly manage memory spikes, preventing crashes and keeping everything running smoothly.
The result of all this cleverness is a model family they call Guanaco. Get this: Guanaco actually outperforms many other openly available models on a standard benchmark. And get this – it even reaches 99.3% of ChatGPT's performance, all while being trained on a single GPU in just 24 hours!
"Our results show that QLoRA finetuning on a small high-quality dataset leads to state-of-the-art results, even when using smaller models than the previous SoTA."
But it doesn't stop there. The researchers trained over 1,000 models using QLoRA, analyzing how well they followed instructions and performed as chatbots. This massive experiment showed that QLoRA really shines when trained on high-quality data, even with smaller models. They also dug into how well GPT-4 can evaluate chatbots, finding it's a pretty good and cheap alternative to expensive human evaluations. They also found that current chatbot benchmarks aren't always reliable.
So, why does all this matter?
For researchers: QLoRA opens the door to exploring even bigger and better AI models without breaking the bank. It allows for faster experimentation and development.
For businesses: This means more affordable and accessible AI solutions, potentially leading to better customer service, more efficient operations, and new product innovations.
For everyone else: It democratizes access to powerful AI, potentially leading to more personalized learning experiences, improved healthcare, and a wider range of creative tools.
They even released all their models and code, including the special CUDA kernels for 4-bit training. This is a huge win for open-source AI!
This paper feels like a turning point. It's not just about making AI bigger, it's about making it smarter and more accessible. It's about leveling the playing field so that everyone can participate in the AI revolution.
Now, a few things that popped into my head while reading this paper:
How far can we push this 4-bit quantization technique? Are there even more efficient ways to represent AI knowledge?
Could QLoRA be adapted for other types of AI models, like those used in image recognition or robotics?
If GPT-4 is a good evaluator, does this mean that AI could eventually evaluate AI better than humans? What are the implications of that?
What do you think, learning crew? Let me know your thoughts in the comments!Credit to Paper authors: Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer

Hey PaperLedge learning crew, Ernis here, ready to dive into another fascinating piece of research! Today, we're talking about something we all use, sometimes without even realizing it: text-to-speech, or TTS.
Think about Siri, Alexa, Google Assistant – all those voices bringing our devices to life. TTS has come a long way, but a big question has always been: can we make these digital voices truly sound like a real human? And if so, how do we even measure that?
Well, that's exactly what the researchers behind this paper tackled. They asked three crucial questions: Can TTS reach human-level quality? How do we define and judge that quality? And how do we actually get there?
And guess what? They think they've cracked the code, at least on one popular benchmark dataset! They've developed a TTS system called NaturalSpeech, and they're claiming it's the first to achieve human-level quality when it comes to sounding natural!
So, how did they do it? This is where it gets a little techy, but I'll break it down. Imagine you're trying to teach a computer to draw. You could give it a bunch of finished drawings, but it might not understand the underlying principles.
Instead, these researchers used something called a Variational Autoencoder (VAE). Think of it like this: the VAE is like a super-smart student who learns to both encode text into a set of instructions, and then decode those instructions back into realistic-sounding speech. It's an end-to-end system, meaning it goes straight from text to waveform (the actual sound wave).
Now, to make their VAE even better, they added a few key ingredients:
Phoneme pre-training: Like giving the student a lesson in the alphabet before asking them to write a novel. This helps the system understand the basic sounds of language.
Differentiable duration modeling: This helps the system figure out how long to hold each sound, making the speech sound more natural and less robotic. Think about how we naturally vary the length of words when we speak.
Bidirectional prior/posterior modeling: This sounds complex, but it basically means the system looks at both the text before and the speech after to make better predictions. It's like looking at the context of a sentence to understand its meaning.
A memory mechanism in VAE: This lets the system remember important information from earlier in the text, helping it maintain a consistent tone and style throughout the speech.
Now, for the really exciting part: the results! They tested NaturalSpeech on the LJSpeech dataset, which is a standard collection of recordings used to train and evaluate TTS systems. They had people listen to both human recordings and the output from NaturalSpeech, and then rate how natural they sounded.
The result? NaturalSpeech scored so close to human recordings that there was no statistically significant difference! In other words, listeners couldn't reliably tell the difference between the AI and a real person.
"Our proposed NaturalSpeech achieves -0.01 CMOS (comparative mean opinion score) to human recordings... which demonstrates no statistically significant difference from human recordings for the first time on this dataset."
That's a huge breakthrough!
So, why does this matter? Well, for starters, it opens up all sorts of possibilities. Imagine:
More natural-sounding virtual assistants: Chatting with Siri could feel a lot more like talking to a friend.
Improved accessibility for people with disabilities: TTS could become even more effective at helping people with visual impairments access information.
More engaging educational tools: Learning could be more fun and immersive with realistic, expressive voices.
Potential for creating personalized voices: Imagine having a TTS system that sounds exactly like you!
But it also raises some interesting questions:
If we can't tell the difference between a real voice and an AI, what are the ethical implications? Could this technology be used to create convincing fake audio?
How generalizable is this result? Does NaturalSpeech perform equally well on different datasets or with different languages?
Now that we've achieved human-level quality in terms of naturalness, what other aspects of speech can we focus on improving, like expressiveness and emotion?
This is a fascinating area of research, and I'm excited to see where it goes next. What do you think, learning crew? Let me know your thoughts in the comments below!Credit to Paper authors: Xu Tan, Jiawei Chen, Haohe Liu, Jian Cong, Chen Zhang, Yanqing Liu, Xi Wang, Yichong Leng, Yuanhao Yi, Lei He, Frank Soong, Tao Qin, Sheng Zhao, Tie-Yan Liu

Alright learning crew, Ernis here, ready to dive into some seriously cool tech that's blurring the lines between what we hear and what we say! Today, we're unpacking a research paper about something called AudioPaLM.
Now, that might sound like something out of a sci-fi movie, but trust me, it's real, and it's fascinating. Think of it as a super-smart AI that can understand and generate both text and speech. It's like teaching a computer to not only read and write but also to listen and speak fluently. It's all developed by the clever folks over at Google.
So, how does it work? Well, imagine you have two brilliant specialists: one is a word whiz (PaLM-2), amazing at understanding and creating text, and the other (AudioLM) is a sound guru, able to mimic voices and capture the nuances of speech, like intonation and even who's speaking. AudioPaLM is like fusing these two specialists together into one super-powered entity.
The really clever bit is how they built it. They started with the word whiz, PaLM-2, which has been trained on tons of text data. This is like giving it a massive library of information. Then, they carefully added the speech skills of AudioLM. This means AudioPaLM doesn't just understand the words; it also understands how they're spoken, capturing things like emotion and identity.
"AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation...and the linguistic knowledge present only in text large language models."
Think of it like this: imagine you're learning a new language. You can read the textbooks (like PaLM-2), but you really start to understand when you hear native speakers and pick up on their accent and tone (that's AudioLM's influence). AudioPaLM does both at the same time!
So, why is this important? Well, the researchers found that by giving AudioPaLM that head start with all that text data, it became much better at understanding and translating speech. In fact, it outperformed existing systems, especially when it came to speech translation.
Here's where it gets really mind-blowing: AudioPaLM can even do what they call "zero-shot" translation. That means it can translate speech between languages it wasn't specifically trained on. It's like being able to understand snippets of a language you've never formally studied just because you've learned so many other similar languages. That's incredible!
But wait, there's more! Remember how AudioLM could mimic voices? AudioPaLM can do that too, even across different languages. So, you could potentially have it translate your voice into another language, sounding like you!
Here are some of the potential applications:

For travelers: Imagine having a real-time translator that not only understands the words but also conveys the nuances of the speaker's intent.


For people learning new languages: This could be a powerful tool for practicing pronunciation and understanding spoken language in a more natural way.


For accessibility: This technology could help bridge communication gaps for people with hearing or speech impairments.


Now, this raises some interesting questions, doesn't it?
How far can we push the boundaries of voice cloning, and what are the ethical implications of being able to replicate someone's voice so accurately?
Could this technology eventually lead to a universal translator that breaks down all language barriers, or will there always be something lost in translation?
As AI becomes more adept at understanding and generating human language, how will this impact the way we communicate and interact with each other?
Lots to ponder, learning crew! You can find examples of AudioPaLM's capabilities at the link in the show notes. Go check it out and let me know what you think. Until next time, keep those neurons firing!Credit to Paper authors: Paul K. Rubenstein, Chulayuth Asawaroengchai, Duc Dung Nguyen, Ankur Bapna, Zalán Borsos, Félix de Chaumont Quitry, Peter Chen, Dalia El Badawy, Wei Han, Eugene Kharitonov, Hannah Muckenhirn, Dirk Padfield, James Qin, Danny Rozenberg, Tara Sainath, Johan Schalkwyk, Matt Sharifi, Michelle Tadmor Ramanovich, Marco Tagliasacchi, Alexandru Tudor, Mihajlo Velimirović, Damien Vincent, Jiahui Yu, Yongqiang Wang, Vicky Zayats, Neil Zeghidour, Yu Zhang, Zhishuai Zhang, Lukas Zilka, Christian Frank

Hey PaperLedge learning crew, Ernis here, ready to dive into some fascinating research! Today, we're looking at a paper about teaching computers to understand speech, but with a really cool twist.
Imagine you're trying to learn a new language. The traditional way is to take classes, do exercises, and maybe even spend time in a country where it's spoken. But what if you could just... soak it in? Like, listen to thousands of hours of conversations, radio shows, and podcasts? That's kind of what these researchers did with their speech processing system.
They basically fed their system a massive amount of audio – a whopping 680,000 hours worth! And not just in one language, but multiple languages, from all sorts of different sources they found on the internet. Think of it like giving the computer access to the entire Library of Alexandria of spoken word!
So, what did the system learn? Well, the really amazing thing is that it became incredibly good at understanding speech, even speech it had never "officially" been trained on. It's like learning Spanish and then being able to understand a surprising amount of Italian without ever studying it directly. This is called zero-shot transfer.
Zero-shot transfer is key here. The system wasn't fine-tuned for specific tasks or accents. It just listened to a ton of stuff and figured it out. The results? The system performed really well on standard speech recognition tests, often matching or even beating systems that had been specifically trained for those tests. And get this, they even approached human levels of accuracy and robustness.
Think of those times you're trying to understand someone speaking on a bad phone line, or with a really strong accent. Humans are surprisingly good at filling in the gaps and figuring out what's being said. This system is starting to show that same ability.
Now, why does this matter? Well, a few reasons:
For the tech enthusiasts: This shows the power of "unsupervised learning" and how much we can achieve by simply feeding AI systems large amounts of data. It could revolutionize how we build speech recognition systems in the future.
For the global citizens: Multilingual capabilities are HUGE. Imagine a world where language barriers are drastically reduced, making communication and collaboration easier than ever.
For everyone: More robust speech recognition means better voice assistants, more accurate transcriptions, and improved accessibility for people with disabilities.
The researchers are even releasing their models and code, which is fantastic! This means other researchers and developers can build on their work and push the field even further.
"We are releasing models and inference code to serve as a foundation for further work on robust speech processing."
This is a really exciting development, and it highlights the potential of large-scale, unsupervised learning in the field of speech processing.
So, what do you think, learning crew? Here are a couple of questions that popped into my head:
If we can achieve this level of accuracy with just raw audio data, what other areas of AI could benefit from a similar approach?
What are the ethical implications of training AI systems on such large amounts of publicly available data? Are there privacy concerns we need to consider?
Let me know your thoughts in the comments! Until next time, keep learning!Credit to Paper authors: Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, Ilya Sutskever

Hey PaperLedge learning crew, Ernis here, ready to dive into some fascinating research! Today, we're talking about something that's changing the game in AI: Large Language Models, or LLMs.
Now, you might be thinking, "LLMs? Sounds complicated!" But trust me, it's cooler than it sounds. Think of LLMs like super-smart parrots that have read everything and can now mimic human language incredibly well. They're used for all sorts of things, like writing articles, translating languages, and even generating code! And the key to making these parrots smart? Data, data, and more data!
That's where today's paper comes in. These researchers have built something called The Stack. Imagine a giant digital library filled with 3.1 terabytes of source code – that’s code from over 30 programming languages! It's like a massive cookbook for computers, showing them how to do everything from building websites to running complex simulations.
So, what's so special about The Stack? Well, a couple of things. First, it's all permissively licensed. Think of it like this: the creators of the code are giving you permission to use it, learn from it, and even build on top of it. This is a big deal because it allows researchers to freely explore how LLMs can understand and generate code without worrying about copyright issues.
Second, the researchers have thought really carefully about data governance. That means they have a plan in place to make sure the data is used responsibly. They even created a tool called "Am I in The Stack?" where developers can search to see if their code is included and request removal if needed. It's like a digital neighborhood watch, ensuring everyone feels comfortable with how their code is being used.
It's like giving LLMs a masterclass in computer programming!

The researchers then used The Stack to train their own LLMs to write code, specifically in Python. And guess what? They found that by cleaning up the data – removing duplicates, for example – the LLMs got way better at writing code. In fact, they were able to match the performance of other LLMs that were trained on data that wasn't as carefully curated or permissively licensed. That's a huge win for open and responsible AI research!
Near-deduplication matters: Removing duplicate code significantly improves performance.
Permissively licensed data is powerful: High performance can be achieved without relying on restricted data.

So, why does this matter to you? Well:
For developers: The Stack provides a valuable resource for learning new programming languages and improving your coding skills. Plus, the "Am I in The Stack?" tool gives you control over your code.
For researchers: The Stack offers a massive, permissively licensed dataset for training and evaluating LLMs for code.
For everyone else: This research is helping to build more powerful and accessible AI tools that can automate tasks, solve problems, and even create new technologies.
This research really pushes the boundaries of what's possible with AI and code. It makes you wonder:
Could LLMs eventually replace human programmers entirely?
What other creative applications can we unlock by giving AI access to massive amounts of code?
How can we ensure that these powerful tools are used ethically and responsibly?
Definitely some food for thought! You can check out the dataset at https://hf.co/BigCode if you're curious to learn more. That's all for this episode, learning crew. Until next time, stay curious!Credit to Paper authors: Denis Kocetkov, Raymond Li, Loubna Ben Allal, Jia Li, Chenghao Mou, Carlos Muñoz Ferrandis, Yacine Jernite, Margaret Mitchell, Sean Hughes, Thomas Wolf, Dzmitry Bahdanau, Leandro von Werra, Harm de Vries

Hey learning crew, Ernis here, ready to dive into some seriously cool AI research! Today, we're talking about a problem that's been bugging even the smartest large language models (LLMs), like the ones powering your favorite chatbots: their memory is kinda short.
Think of it like this: imagine trying to write a novel, but you can only remember the last page you wrote. Tough, right? That's what LLMs face when dealing with long conversations or analyzing massive documents. They have a limited "context window," which is basically how much information they can actively process at once.
So, how do we give these AI brains a better memory? Well, the researchers behind this paper took inspiration from something we've been using in computers for ages: how operating systems manage memory. It's all about creating the illusion of a giant memory, even when the physical memory is limited.
They introduce MemGPT, which stands for Memory-GPT. Think of MemGPT as a super-efficient librarian for the LLM. It's built to manage different "tiers" of memory, like:
Immediate Memory: This is the LLM's short-term memory, the stuff it's actively working with.
Main Memory: Think of this as a slightly longer-term memory, holding important information that's frequently needed.
External Memory: This is the deep storage, like a hard drive, where everything else is kept.
MemGPT intelligently shuffles information between these tiers, keeping the most relevant stuff readily available for the LLM. It's like strategically placing books on your desk versus storing them in boxes in the attic.
But here's the really clever part: MemGPT also uses something called "interrupts." Imagine you're reading a book, and suddenly the doorbell rings. You pause your reading, deal with the interruption, and then go back to your book. MemGPT uses interrupts to manage the flow of information between itself and the user, allowing it to handle requests and update its memory efficiently.
So, why does this matter? Well, the researchers tested MemGPT in two key areas:
Document Analysis: Imagine summarizing a 500-page book. Normally, an LLM would choke on that! But MemGPT allowed it to analyze documents far exceeding the LLM's normal limits.
Multi-Session Chat: Ever wish your chatbot remembered your previous conversations? MemGPT enables conversational agents that can actually remember, reflect on past interactions, and evolve over time. It's like having a digital friend who actually learns about you.
"MemGPT...effectively provide[s] extended context within the LLM's limited context window..."
This isn't just about making chatbots better. It opens up possibilities for:
Personalized Learning: AI tutors that remember your learning style and progress.
Enhanced Research: AI assistants that can analyze vast amounts of data and synthesize insights.
Improved Customer Service: Chatbots that can actually understand and resolve complex issues.
The researchers have even released the MemGPT code and data, which you can find at https://memgpt.ai, so others can build on their work. It's a big step towards more capable and useful AI.
This got me thinking: If AI can now have extended memories, how will that change our interactions with technology? And, ethically speaking, what responsibilities do we have when AI can remember everything we tell it?
And finally, could this approach be applied to other AI models beyond LLMs, maybe even to robotics or computer vision? The possibilities are pretty mind-blowing!Credit to Paper authors: Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, Joseph E. Gonzalez

Hey PaperLedge crew, Ernis here, ready to dive into some fascinating research! Today, we're tackling a paper that's all about making AI smarter, specifically when it comes to complex problem-solving – think of it like teaching a robot to not just memorize answers, but to actually understand how to get there.
So, we all know those AI models, the large language models, that are getting pretty good at doing complex things. They can write stories, answer questions, even try to solve math problems. But here's the thing: even the best ones still make silly mistakes, like getting basic logic wrong. It's like that friend who's generally brilliant but occasionally puts their shoes on the wrong feet!
Now, how do we fix this? Well, the researchers behind this paper looked at two main ways to train these models:
Outcome Supervision: This is like giving a student a grade only on their final exam. You tell them if the answer is right or wrong, but you don't give them feedback on how they got there.
Process Supervision: This is like a teacher going through each step of a student's work, pointing out where they went wrong and why. You give feedback on each intermediate step, not just the final answer.
Think of it like learning to bake a cake. Outcome supervision is like tasting the finished cake and saying "too sweet!" Process supervision is like someone watching you add ingredients, saying, "Whoa, hold on! That's way too much sugar for this recipe!"
The researchers wanted to figure out which method works best, especially since getting feedback from humans (that process supervision part) can be really expensive and time-consuming. Previous studies have scratched the surface, but this paper goes deeper.
And guess what? They found that process supervision wins, big time! They trained models to solve problems from a really tough math dataset called MATH. The model trained with process supervision aced a whopping 78% of the problems in a test set. That's a huge jump!
"Process supervision significantly outperforms outcome supervision for training models to solve problems from the challenging MATH dataset."
But it doesn't stop there! They also looked at something called active learning. This is like letting the AI model choose which problems it wants to be trained on. The model basically says, "Hey, I'm really struggling with this type of problem, can you give me some extra feedback on that?" Turns out, active learning makes process supervision even more effective!
To help other researchers, they're releasing a massive dataset of human feedback labels – 800,000 of them! It's called PRM800K, and it's a treasure trove for anyone working on improving AI reasoning.
So, why does all this matter? Well, better AI reasoning has implications for everything from medical diagnosis to financial modeling. Imagine AI that can reliably solve complex problems in healthcare, leading to more accurate diagnoses and personalized treatments. Or AI that can make smarter financial decisions, helping people manage their money more effectively.
Here are a few things I was pondering as I read this:
If process supervision is so much better, why aren't we using it all the time? Is the cost of human feedback truly the only barrier?
Could we develop AI tools to automatically provide process supervision, reducing the need for expensive human input?
Beyond math, what other domains could benefit most from this type of process-supervised AI training?
This research is a big step forward in building more reliable and trustworthy AI. It's exciting to think about the possibilities! What do you guys think? Let me know your thoughts in the comments!Credit to Paper authors: Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, Karl Cobbe