--- title: "Exploring Karpathy's New NanoChat Project" type: post url: "https://robotpaper.ai/exploring-karpathys-new-nanochat-project/" canonical: "https://robotpaper.ai/exploring-karpathys-new-nanochat-project/" markdown_url: "https://robotpaper.ai/exploring-karpathys-new-nanochat-project.md" published: "2025-10-13T21:02:48.000+00:00" updated: "2026-10-22T21:57:55.000+00:00" tags: ["Models"] author: "Roy Osherove" author_links: ["https://robotpaper.ai", "https://x.com/royosherove", "https://www.linkedin.com/in/osherove/", "https://forms.gle/jQSx5KoXvrVA1JCK7", "https://robotpaper.ai/about-roy.md"] description: "Andrej Karpathy released nanochat, a minimal from-scratch full-stack ChatGPT clone training pipeline. I'm training it on Lambda and digging into how it works." usage: "Licensed under CC BY 4.0 — reuse, quote and adapt with attribution (Roy Osherove, link to the original post)." license: "https://creativecommons.org/licenses/by/4.0/" --- # Exploring Karpathy's New NanoChat Project OK. This is Big. If you don't know who [Andrej Karpathy](https://karpathy.ai/) is, please get to know him and all of his amazing writing and projects. 💡 If you're a hands-on developer, I recommend Karpathy's "[Technical Track" set of youtube videos](https://www.youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ) where he basically builds an LLM from absolute scratch, including tokenization. Just be warned - it's about as deep as a rabbit hole goes. Recently, Andrej released a [new repository project called NanoChat](https://github.com/karpathy/nanochat) - > "A minimal, from scratch, full stack training/inference pipeline of a simple chatGPT clone. " You can read all about it in his own words: > Excited to release new repo: nanochat! > (it's among the most unhinged I've written). > > Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single,… [pic.twitter.com/LLhbLCoZFt](https://t.co/LLhbLCoZFt) > > — Andrej Karpathy (@karpathy) [October 13, 2025](https://twitter.com/karpathy/status/1977755427569111362?ref_src=twsrc%5Etfw) # Now What? My background is software development & engineering, and less research, and I've been slowly getting into the "deep" side of LLMs and how models work under the covers. So I'll be walking through the code , run the training myself and document any insights or findings here on this blog. I'll be [starting from this page](https://github.com/karpathy/nanochat/discussions/1), and as we speak I'm running the training on lambda.ai. But my goal is to understand all the steps in a deep way. Today I understand them in a very surface level way, and have been working to change that. Will report back as I progress. ![Terminal output of nanochat training steps showing loss, learning rate and tokens per second](https://robotpaper.ai/content/images/2025/10/image.png) *Training in progress* --- ## About the author **Roy Osherove** - Senior Solutions Architect at AWS working on AI for production and AI-driven development; author of The Art of Unit Testing and Elastic Leadership, with about 20 years of consulting and training on unit testing, TDD, team leadership and continuous delivery. ### Sites - [RobotPaper](https://robotpaper.ai/): This blog: AI patterns and anti-patterns, AI and coding agents, and engineering leadership. - [osherove.com](https://osherove.com): Roy's personal site. Author of The Art of Unit Testing and Elastic Leadership; consulting and training on unit testing, TDD, team leadership and continuous delivery, plus about 20 years of blog posts. - [The Art of Unit Testing](https://www.artofunittesting.com): The Art of Unit Testing, 3rd Edition (Manning): readable, maintainable, trustworthy unit tests, plus TDD workshops in C#, JavaScript, Java and Kotlin. - [5whys.com](https://www.5whys.com): Roy's writing on software team leadership: self-organizing teams, surviving chaos, learning mode and the Elastic Leadership model. - [Pipeline Driven](https://pipelinedriven.org): Continuous delivery and DevOps practices: build patterns, pipeline-friendly testing, DevOps metrics and enterprise DevOps. - [Elastic Leadership](https://www.elasticleadership.com): Elastic Leadership, Roy's framework for software team leaders (survival, learning and self-organizing modes). ### Social - X: [https://x.com/royosherove](https://x.com/royosherove) - LinkedIn: [https://www.linkedin.com/in/osherove/](https://www.linkedin.com/in/osherove/) ### Contact - Contact form: [https://forms.gle/jQSx5KoXvrVA1JCK7](https://forms.gle/jQSx5KoXvrVA1JCK7) ### License Content on robotpaper.ai is licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/): reuse, quote and adapt with attribution (Roy Osherove, link to the original post). More about Roy and his other sites and projects: [https://robotpaper.ai/about-roy.md](https://robotpaper.ai/about-roy.md)