# RobotPaper - full text > Roy Osherove's blog on AI patterns and anti-patterns, AI and coding agents, and engineering leadership. Licensed under CC BY 4.0 — reuse, quote and adapt with attribution (Roy Osherove, link to the original post). License: https://creativecommons.org/licenses/by/4.0/ Source: https://robotpaper.ai. Generated 2026-10-23T08:15:02.948Z. 31 posts, newest first. ## About the author **Roy Osherove** - Senior Solutions Architect at AWS working on AI for production and AI-driven development; author of The Art of Unit Testing and Elastic Leadership, with about 20 years of consulting and training on unit testing, TDD, team leadership and continuous delivery. ### Sites - [RobotPaper](https://robotpaper.ai/): This blog: AI patterns and anti-patterns, AI and coding agents, and engineering leadership. - [osherove.com](https://osherove.com): Roy's personal site. Author of The Art of Unit Testing and Elastic Leadership; consulting and training on unit testing, TDD, team leadership and continuous delivery, plus about 20 years of blog posts. - [The Art of Unit Testing](https://www.artofunittesting.com): The Art of Unit Testing, 3rd Edition (Manning): readable, maintainable, trustworthy unit tests, plus TDD workshops in C#, JavaScript, Java and Kotlin. - [5whys.com](https://www.5whys.com): Roy's writing on software team leadership: self-organizing teams, surviving chaos, learning mode and the Elastic Leadership model. - [Pipeline Driven](https://pipelinedriven.org): Continuous delivery and DevOps practices: build patterns, pipeline-friendly testing, DevOps metrics and enterprise DevOps. - [Elastic Leadership](https://www.elasticleadership.com): Elastic Leadership, Roy's framework for software team leaders (survival, learning and self-organizing modes). ### Social - X: [https://x.com/royosherove](https://x.com/royosherove) - LinkedIn: [https://www.linkedin.com/in/osherove/](https://www.linkedin.com/in/osherove/) ### Contact - Contact form: [https://forms.gle/jQSx5KoXvrVA1JCK7](https://forms.gle/jQSx5KoXvrVA1JCK7) ### License Content on robotpaper.ai is licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/): reuse, quote and adapt with attribution (Roy Osherove, link to the original post). More about Roy and his other sites and projects: [https://robotpaper.ai/about-roy.md](https://robotpaper.ai/about-roy.md) --- # Graphlin - Jev-Enabled Live Architecture Visualizer - URL: https://robotpaper.ai/graphlin-jev-enabled-live-architecture-visualizer/ - Markdown: https://robotpaper.ai/graphlin-jev-enabled-live-architecture-visualizer.md - Published: 2026-09-22T15:24:56.000+00:00 - Updated: 2026-10-22T21:57:58.000+00:00 - Tags: Tools - Author: Roy Osherove > Graphlin is my open-source Claude Code/Codex plugin that draws a live diagram of the architecture your agent is building, using Jev to classify each action. Introducing [Graphlin](https://github.com/royosherove/graphlin) - a plugin for Claude Code/Codex that auto visualises your architecture as your agent is building it, and what the agent is currently working on. [Graphlin Plugin for Claude/Codex: Live, on-the-fly Architecture Diagrams](https://www.youtube.com/embed/ynZLe9p7PxM?feature=oembed) Why? I, like many others, tried [Jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev) over the weekend. Unlike others, I did NOT build a router. I built a live diagram of the architecture being built by my agent - not perfect but it might be the beginning of something. It's open source and you can plug your own visualization to it. The source is fully open at [https://github.com/royosherove/graphlin](https://github.com/royosherove/graphlin) ## How does it work? ![Graphlin sequence diagram: agent hooks send actions through a local parser to Jev for classification, then update the live architecture viewer](https://robotpaper.ai/content/images/2026/09/image-1.png) After various hooks in the agent such as pre tool use, file edit, create etc, actions are being sent for classification by Jev (multiple layers), and the results are used for both generating state, and visualizing it by a local server. Jev is used as a runtime decider on what actions are being taken, what are the various types of components/files, what icons to use, who depends on who and more. The idea of a general classifier is pretty strong and opens up a bunch of use cases. Classifiers are not new, but the packaging, lack of need to pre-train, and ease of use out of the box are pretty useful. Like OpenClaw that gave birth to many a grok-bot, instinct, kiroCrew and others, I believe this will just be the first one to open the door to imagining a general classifier future. a system one classifier is a useful tool in the toolbelt. Other projects claim to have been first: [Laya](https://huggingface.co/convaiinnovations/laya), for example. I have not tried it yet so cannot share my opinion. However, graphlin is pluggable, you could technically ask your agent to replace the decision service with laya and see what happens. --- # The New Bottlenecks After Coding - URL: https://robotpaper.ai/the-new-bottlenecks-after-coding/ - Markdown: https://robotpaper.ai/the-new-bottlenecks-after-coding.md - Published: 2026-08-15T19:14:01.000+00:00 - Updated: 2026-10-22T21:58:01.000+00:00 - Tags: Pattern - Author: Roy Osherove > Coding got cheap. Now deployment, code review, merge conflicts, architecture context, specs and monitoring are the bottlenecks. Here's my list and some fixes. They're not really new, we just never really got to them - we just assumed they were there as a force of nature. Let's list them out (I'm sure there are more than this list): - Deployment: Infrastructure is the new bottleneck after coding. - You write the code, but now you have to wait and get a proper environment set up, or permission to deploy to such an environment, or get all the "glue" and much in the environment to play nice with the code you just wrote (connections between services, permissions, roles, logins, database connections , bucket permissions etc etc .... it's a long list). - The [Pattern: Agent-Owned Accounts (Deploy Environments)](https://robotpaper.ai/agent-owned-accounts-deploy-environments/) pattern is one way to solve this, - Stuff like [LowKey](https://github.com/inceptionstack/lowkey/) solve this bottleneck today, for example, but most of the world isn't there (yet). - You can see buds of solutions with [cloudflare agent accounts](https://blog.cloudflare.com/agents-stripe-projects/) , for example. but those are still relatively limited. - Code Review is the new Bottleneck: I believe that - Slowly AI Agents will review more and more of the code - Humans will review more and more of the overall architecture - Code Commits & Conflicts will be the new bottleneck: - When you have 1,000 live agents pushing 10,000 commits a day in parallel into the same codebase, conflicts will become the new bottleneck. You can see this happening today in projects like OpenClaw, where getting a release out the door become a nightmare due to the simple loop: - The agent conflict loop: - A: Agent stages a PR - B: Needs rebase since a new commit just landed in main just before this PR - C: Rebase has a conflict - D: Agent fixes conflict - Go to -> A, repeat ad infinitum - One way I am playing with to solve something like this is applying game theory strategies to agent-to-agent negotiations on who pushes what and when. It's still an open question if that is a good way to go about this, but it's an interesting problem! - Agent Company / Architecture knowledge is the new bottleneck: - Agents needs to know what they are working on, and what exists around them so they can plan and build things that fit into the existing coding landscape (other repos, good architecture practices, upstream and downstream dependencies , accepted and declined ways of working across the entire company, secrets, regulations etc... - Those things keep changing, and will keep changing even faster (see previous bottleneck point on code commits) - Projects like [Reposwarm](https://github.com/reposwarm/reposwarm) (which I wrote) help to deal with at least some of that issue (cross-repo architecture that is not stale, for agent context) - There are many projects trying to build a "company brain" but the situation is all over the place right now. - Testing is the new bottleneck (maybe not anymore): - in a world where coding and features are close to free to build, agent functionality verification is king. So if you know how to write good tests, and you write them and the agent uses them, you get a pretty good speed up on getting stuff to work across multiple changes as time goes by. - Greenfield code is easy, as code changes, tests become more and more critical. - But is it a bottleneck to write tests? not really - it's easy to generate tests. but generating **good** tests is still a skill to teach the agent. - Specs are the new bottleneck: - PMs need to generate good specs faster since coding is easy now. - there are many frameworks to try to help this: [superpowers](https://claude.com/plugins/superpowers) by claude, [BMAD](https://github.com/bmad-code-org/bmad-method), Spec Driven Development (did you know kiro actually [came out first with this](https://kiro.dev/docs/specs/) from an AI-IDE perspective) , [AI-DLC 2](https://github.com/awslabs/aidlc-workflows), and more. - Orgs are still learning how to work this way and it's still up in the air if any of this truly helps - but early signs show promise. But there's a lot of low hanging fruit here so it might just be that **any** change here is for the better. - Monitoring is the new bottleneck: - If everyone write and deploys more code and agents are inherently "creative" then monitoring all these processes and all those newly deployed applications that can be generated on a whim become a new potential bottleneck. We have a pretty mature monitoring industry, but monitoring LLM interactions, and all the potential upstream and downstream risks becomes a new area. - An agent that can monitor what it had built and deployed can help alleviate this. - Monitoring all the agents currently running locally, remotely and across the org - not a perfectly solved problem today (unless you want to be extremely vendor locked) - I'm sure there's more but I can't remember right now. I might edit this post later to add more. --- # Pattern: Agent-Owned Accounts (Deploy Environments) - URL: https://robotpaper.ai/agent-owned-accounts-deploy-environments/ - Markdown: https://robotpaper.ai/agent-owned-accounts-deploy-environments.md - Published: 2026-07-01T19:43:33.000+00:00 - Updated: 2026-10-22T21:58:01.000+00:00 - Tags: Pattern, AWS - Author: Roy Osherove > Give an agent a VM and it submits PRs. Give it its own cloud account and it codes, deploys, debugs and monitors apps. The Agent-Owned Accounts pattern. Give an agent a VM, it will submit PRs. Give an agent a deployment environment and an admin ability, and it will code, deploy, debug and monitor your apps. I believe [LowKey](https://github.com/inceptionstack/lowkey) was the first I to have its own AWS account to do with as it pleases. Lately I've also heard of [Cloudflare Temporary accounts](https://blog.cloudflare.com/temporary-accounts/) for agents. These ideas solve for the next problem down the chain: After coding friction is solved, deployment & configuration friction becomes the next bottleneck. Just like "[Markdown is the new Installer](https://robotpaper.ai/markdown-is-the-new-installer/)" , solving the friction of deploying something onto a user's machine, so can we, the developers, solve the deployment problem by giving agents an [account that they own](https://robotpaper.ai/i-gave-my-agent-its-own-aws-account-and-now-it-codes-deploys-and-debugs-full-stack-apps/). In an internal experiment (~100 participants) Agent-owned accounts have been very impactful for internal solutions architects I've worked with over the past few months - it has sped up prototyping (turned prodo-typing) from weeks/days to mere hours of the most complex proofs of concepts and prototypes you can imagine on an AWS infrastructure with amazing scale. We're slowly showing this to customers (it's [open source](https://github.com/inceptionstack/lowkey) they can try it now) and getting feedback - but it makes a huge difference. Mobile support is a killer feature. 24/7 live event-based notifications, checks and monitoring a killer feature. Being able to think about something and just try it in a few minutes from your phone while on the train is a killer feature. removing the friction of configuration and "glue" work is a killer feature. For now I believe it is pretty risky to give an agent full production account deploy abilities (yet), but how about a sandbox account - one where the agent can utilize the ability to deploy and then fully test a full application end to end, replicating production fully, and then tearing it down if needed, or keeping it and providing access to internal or external beta users using a login? This would take days or weeks in many companies today, and breaks the product-market-fit cycle of feedback-change-deploy. Now it can happen in minutes. Today, base44 , lovable and friends provide just the basics of what you could deploy on, for anything else (graph databases, workflows, storage etc) they rely on you connecting to 3rd parties and giving them your APIs. With a full cloud provider such as AWS, Azure, GCP - these abilities become much more powerful - and there is zero friction to deploy almost anything of any complexity or scale (only wallet depth). Graduating to production will be the brainstorm of a future post. --- # Markdown is the new Installer - URL: https://robotpaper.ai/markdown-is-the-new-installer/ - Markdown: https://robotpaper.ai/markdown-is-the-new-installer.md - Published: 2026-07-01T19:31:00.000+00:00 - Updated: 2026-10-22T21:57:56.000+00:00 - Tags: Developer-Experience - Author: Roy Osherove > No more MSI or drag-to-install. A well-crafted bootstrap prompt lets your users' local agent install complex tools for them. Why markdown is the new installer. No more [MSI](https://learn.microsoft.com/en-us/windows/win32/msi/windows-installer-portal). No more dragging. No more downloading. A well crafted bootstrapping prompt provided to your users can mean you can create ever more complex tools and apps that might be terrible to install from a human-perspective, but for an agent they are easy-as-pie to go through. Your users fire up their favorite local agent, paste your bootstrap, and a minute or so later things are installed. Yes - it's not deterministic. but it also means the process is much more (surprisingly) reliable since it can deal with a lot of unknowns (user OS, local environment, misconfigurations , limitations and unforeseen actions by the user) in a much more elaborate way. Intelligent installers are the new "curl ...." copy-paste. We still need those, but agents run them. Tokens-per-install might become a new metric. At work we allow users with less technical knowledge to install [kiro](https://kiro.dev/), skills and mcps written by other people in the company, and actually get them to use them - undreamt of just a couple of years ago. The only remaining wall to climb no built in "paste and run" surface built into the OS yet: I still can't get my mom to paste a prompt though. She needs a local desktop agent and that's too much for her. Web version is good enough (in moderation). --- # Prodo-typing - URL: https://robotpaper.ai/prodo-typing/ - Markdown: https://robotpaper.ai/prodo-typing.md - Published: 2026-04-21T11:42:21.000+00:00 - Updated: 2026-10-22T21:57:50.000+00:00 - Tags: AWS, openclaw - Author: Roy Osherove > Prodo-typing: building a production-shaped prototype in its own cloud account with Loki, my OpenClaw-based builder agent. Brainstorming what that changes. production-shaped prototype in a separate cloud environment This is all me just brain-storming with myself, and I'd love to hear what people think about this. For background [Loki is a builder agent](https://robotpaper.ai/i-gave-my-agent-its-own-aws-account-and-now-it-codes-deploys-and-debugs-full-stack-apps/) based on openclaw designed to be deployed in its own AWS cloud account with full ownership of it. I've been thinking a lot about what it means to be [building something with lowkey](https://lowkey.run/) (aka loki - I'm still deciding with I like the lowkey name or loki or something better. more on that towards the end). ## You use Loki in prototyping mode: You're building fast but you could build full scale apps. Loki builds the code in the same environment in which it deploys it: an AWS sandbox account (it's a regular account with no other apps in it, really) You could steer Loki to build things in a production-grade design: scale, security, modular architecture, performance, optimizations, full stack. in that way you could technically take loki out of the environment (or keep it as read only) then "graduate" your app into production mode. but really you could open usage to your prototype easily from day one if you really wanted to. Hence the term "prodo-type". a hybrid production-shaped prototype that could relatively easily transform into production facing usage pending whatever steps you wanted to do. ## It's not like "productizing" a prototype. Different from other prototype-to-prod graduations is that those usually require huge rewrites of the codebase or architecture or deployment mechanisms and can take weeks to months to "productize". A "prodotype" should take days/hours/minutes to become production facing. It should already have the properties of a production facing app: scale, security, data residency as needed, resiliency and other "ilities" as might be required. Usually we don't have time to bake those into prototypes, and I feel that loki has changed that for me and the people who use it. ## Building it "production-scale" from the first hour is actually possible with a skilled operator. Because Loki runs inside AWS, it has access to all the AWS services that the worlds' largest apps are built on - so it has all the resiliency, scale, security and residency mechanisms you can build with any app today - all you have to do is ask. Need cloud-front caching, DDOs protections, API gateways, certifications, DNS routing, logins (social or otherwise SSO), low level S3 , any type of AWS or other type of database, any type of EC2 instance to build any app you could really imagine - it's there. ![Chat with Loki proposing an API-only video-transcription architecture on AWS with Cognito, S3, Step Functions and Bedrock](https://robotpaper.ai/content/images/2026/04/image-2.png) ## Skilled Operator - Or it's just a powerful prototype If the person operating Loki does not know how to design well-architected apps, in a secure fashion, Loki will only take it so far. So you won't get production grade apps from that, but you'll get running prototypes for sure with many more abilities that a lovable or base44 will ever give you. ## No MCP required Because Loki is in an AWS account, and we deploy it with builder permissions it is basically the *opposite* deployment model from any agent I've seen to date. It actually has practically unlimited access to the aws account, so it can list, create (and delete!) your s3 buckets, ec2 instance, raise up databases, full pipelines, frontends and anything you can do with AWS cli or AWS APIs. It feels scary at first so I put in a disclaimer that you should use it in a new sandbox account so the worst thing that could happen is extra spend if you ask it to create things and not take care of their lifecycles. > **[Disclaimer: Use of Lowkey Agent](https://lowkey.run/aws)** > Important information from Amazon Web Services about using the Lowkey agent in your AWS account. ## AWS accounts can be prodo-types The entire environment in that account can become a full blown new feature or application, only a dns redirect away from being usable in the world. ## Feature flags are prodo-types. In a way developing a new feature in your app and hiding it under a feature flag is a form of prodo-typing - the prototype is minutes away from being production facing. The difference is that loki usually works in a full blown separate aws account. you COULD hide features under feature flags but realistically the AWS account is the feature flag. ## API versions are prodo-types. If you develop an API in a production env as a new version of an api , I think that could be a prodotype. ## Choosing a name for Loki Maybe I should called it "prodo"? or "prodotype" thoughts are welcome on comments here or by [contacting me](https://docs.google.com/forms/d/e/1FAIpQLSeYih1UWrTcE3vleojz_JkKed7lBdEQ0du2HFlScviHtJQkuQ/viewform) --- # Software Engineering Practices (are also) Useful for Token Reduction - URL: https://robotpaper.ai/software-engineering-practices-are-also-useful-for-token-reduction/ - Markdown: https://robotpaper.ai/software-engineering-practices-are-also-useful-for-token-reduction.md - Published: 2026-04-18T00:50:01.000+00:00 - Updated: 2026-10-22T21:57:53.000+00:00 - Tags: Developer-Experience - Author: Roy Osherove > Good engineering practices used to save human cycles. Now person cycles are agent cycles, measured in tokens. Why clean code, tests and DRY cut your token bill. Software engineering practices were (at least when I'm pushing them) all about minimizing things for humans: - minimize how much a person needs to read to understand the meaning of something, - how many things a person needs to change to add, change or remove a feature, - how many times a person needs to test something manually, - how many times we repeat the same mistake, - how many times we need to communicate the meaning of something - etc etc. ## Person cycles are now agent cycles Which are today, measured in token cycles. All the invisible busy-work that used to hide in the shadows and we were always wondering where the time goes, why isn't this thing ready yet - it's mostly visible in tokens now. When an agent tries something multiple times, you can see it in the tokens. when it generates or changes huge amounts of code again and again, you can see. When you keep telling it over and over again and it keeps misunderstanding, this costs us our precious tokens. And so we get: ## The rise of re-discovered software engineering principles Long held by many but loosely adopted by many more. Perhaps engineering principles are (a) killer feature for agents , at least for the near future as agents remain very effective yet incredibly unreliable coding machines. Agents with engineering principles guidance and prompts show remarkable improvement for token consumption (at least for me). Here are a few principles I use (and many more exist): ## Basic Principles - Huge Token Savings I've been using these while working on and with [lowkey](https://github.com/inceptionstack/lowkey) and [reposwarm](https://github.com/reposwarm/reposwarm) (and a bunch of others) - **"Apply DRY/extract function/extract/class":** Don't repeat yourself. You tell the agent to not repeat itself in the code so instead of having 12 places where it checks if something is true or not, it extracts a reusable function. - Next time it needs to update that logic, it only has to spend 12th of the tokens it would have needed to change the code without DRY principles. - which also leads to.. - **"Apply Readable functions and class names, single responsibility principle"** - Next time the LLM needs to reason on a codebase and understand where it needs to change things and where it does NOT need to change things, and what's risky and what's less risky , it's more likely to : - spend less time going over the wrong places (starts in more likely places) - make less mistakes by changing the wrong code or creating it in the wrong place - more likely to repeat the good patterns it has seen before (readability, DRY etc. - **"Apply Fully Automated tests that can be run by CLI repeatedly":** - Allow the agent know when it's "done' or if there's a new issue it broke - which leads to : - "Use Red-Green [TDD](https://osherove.com/blog/2019/11/12/an-updated-diagram-of-the-tdd-process-from-the-upcoming-3rd-edition-of-art-of-unit-testing)": - Agents are more likely to write better tests and code that actually works for them if they start with a failing test, then make it pass by writing production code. - which leads to.. - **"Apply Refactoring as part of the TDD cycle":** - which brings us back to DRY, and readable code. - which leads to.. - (implicitly you get) Interface Segregation & Dependency inversion: - Agent can work on something without needing to change other things (aka shotgun surgery) or even understanding other things. - (implicitly you get) Small commits with good messages - allow agents to reason on changes - basically act as "session summaries" for future agents to look at when no agents.md are around, and get compressed context this way. - prevent agents from repeating past mistakes or getting better context on future changes - Apply CI/CD pipelines - a broader use of agent feedback loops, not just at the local level but at the integration and e2e level - "use Pattern-Name here": Design Patterns and pattern names - Calling something as using a "[Strategy Pattern](https://en.wikipedia.org/wiki/Strategy_pattern)" compresses a lot of agent built in knowledge on something that would have taken many tokens to express or read for the agent. - Type Systems and strong typing: - Types are compressed documentation. Instead of the agent reading 500 lines to infer what shape data has, it reads a 5-line type definition. - compiler errors are cheaper feedback than runtime errors (shorter error messages, more precise location) - Doc-strings: - used to be hell because nobody updated them, but agents can update them easily and read them easily - but ONLY if they are used with good reason. - API Contracts / OpenAPI Specs - A spec file is a compressed representation of an entire service structure. Agent reads 100 lines of spec instead of 5000 lines of implementation to know how to integrate. - Feature Flags - Changing behavior via config = changing 1 line --- # opstalk - a cli to talk to AWS DevOps Agent - URL: https://robotpaper.ai/opstalk-a-cli-to-talk-to-aws-devops-agent/ - Markdown: https://robotpaper.ai/opstalk-a-cli-to-talk-to-aws-devops-agent.md - Published: 2026-04-10T12:54:14.000+00:00 - Updated: 2026-10-22T21:58:02.000+00:00 - Tags: Tools, AWS - Author: Roy Osherove > opstalk is a small open-source terminal CLI I built to chat with AWS DevOps Agent: think Claude Code for AWS ops, with scriptable one-shot commands. So, I think [Devops Agent](https://aws.amazon.com/devops-agent/) is great. I went ahead and built a small CLI to talk to it directly from my terminal. I think it's pretty cool and makes it easy to jump in and investigate things fast. [https://github.com/inceptionstack/opstalk](https://github.com/inceptionstack/opstalk) ### Quick Install > [](https://github.com/inceptionstack/opstalk#quick-install)npm install -g opstalk ## What is OpsTalk? [](https://github.com/inceptionstack/opstalk#what-is-opstalk) OpsTalk is an interactive terminal chat CLI for AWS DevOps Agent. Think "Claude Code but for AWS ops": a full-screen terminal interface for chatting with an AWS-backed agent, plus scriptable commands for one-shot prompts and operational workflows. It is built with Ink v6, React 19, and Commander. ## Features [](https://github.com/inceptionstack/opstalk#features) - Interactive full-screen TUI chat with streaming responses - Slash commands for chat control, help, clearing the transcript, starting a new chat, and resuming previous chats - Setup wizard for agent space selection on first launch - One-shot `send` command for scripting and automation - List previous chat sessions and resume them from the interactive picker - List available AWS DevOps Agent spaces - AWS IAM SigV4 authentication using the standard AWS credential chain - Markdown rendering in the terminal for bold, italic, code, and lists - XDG-compliant config storage with restrictive `0600` permissions ## Prerequisites [](https://github.com/inceptionstack/opstalk#prerequisites) - Node.js `>= 20` - AWS credentials configured through one of the standard mechanisms: - instance profile - environment variables - AWS IAM Identity Center / SSO - shared AWS config and credentials files - Access to the AWS DevOps Agent service and at least one agent space --- # Github Repo Forks Are a New Token-Saving Mechanism - URL: https://robotpaper.ai/github-repo-forks-are-a-new-token-saving-mechanism/ - Markdown: https://robotpaper.ai/github-repo-forks-are-a-new-token-saving-mechanism.md - Published: 2026-04-05T09:07:18.000+00:00 - Updated: 2026-04-05T09:07:18.000+00:00 - Tags: Pattern - Author: Roy Osherove > When you fork an open-source agent repo, you build on someone else's spent token budget. Why written code is now spent tokens, and forks are a way to save them. I've spent probably hundreds of $ on [loki-agent's code](https://github.com/inceptionstack/loki-agent). And spending more every day. It's code that combines: - My expertise and architectural vision and creative ideas - My token budget When I open source it, and you fork it, you're basically building on top of my spent token budget. That's not a bad thing, it's just a way to think about tokens as the new type of economy. If [PRs are now "prompt requests"](https://x.com/karpathy/status/2040473058834878662) which I can run on my own, but then I pay for the tokens, then written code is "spent tokens" which I don't have to spend myself, except in code review. So the more tokens were spent on a useful piece of code, even if it's very reproducible locally on your own with your own agents, there is a new type of value we can see in them, and that is the sunk-cost of creating the code. The code review, refactoring and adding features would (usually) be lower cost than that. Just something to consider as we move forward in a token based economy. --- # The Rise of the Stateful Agent - URL: https://robotpaper.ai/the-rise-of-the-stateful-agent/ - Markdown: https://robotpaper.ai/the-rise-of-the-stateful-agent.md - Published: 2026-04-03T21:38:00.000+00:00 - Updated: 2026-10-22T21:57:57.000+00:00 - Tags: Pattern, openclaw - Author: Roy Osherove > OpenClaw kicked off a new pattern: the stateful agent. Persistence, long-term state, proactivity and tools in one loop. Why it matters and where it's going. It's hard to believe [OpenClaw](https://github.com/openclaw/openclaw) (read "ClawdBot") was released only about 4.5 months ago. Nov 24 Openclaw was released on Github publicly. It's been making history ever since. I'll treat it only from a tech standpoint, but it *is* a phenomena. OpenClaw start a new type of pattern that did not exist before. Commonly called [Stateful Agent — Pattern Definition (with Conversation Log)](https://robotpaper.ai/pattern-stateful-agent/) - it's really a few things in one that make it a killer product: - **Persistence**: Before OpenClaw agents were mostly built to be "stateless" between sessions. But OpenClaw tries to remember and stay in the same session continuously. - State: Keeping long term state, including handling state compaction , short and long term memory mechanisms, as well as semantic local search, but defaulting to file based search. - Session (gateway): with a long running single session "main" that you can connect to from anywhere - **Connectivity**: The Gateway is able to be a global connector from both web and social tools like Whatsapp, Telegram and more. You can start a session on your computer, and continue on your phone. - **Autonomy**: Proactive event based scheduler allows the agent to have both a 30 minute heartbeat that "does stuff" , and ability to schedule cron events that trigger other "stuff. - **Identity**: OpenClaw created its own "Soul" and "User" file and created the illusion of identity for itself and the user. - **Ease** of **configuration**: Everything is done via markdown files, so the agent can change itself very easily. Each of these features on its own presents some sort of advancement in how agents can work, and combined, these have created an emerging wave, a renaissance if you will, of personal assistants around the world. The ability to talk with the agent from anywhere, that it "knows" you and what you worked on together a week or two ago, that it can command a machine and do what you allow it to do without being bound by traditional agent rules opened a lot of eyes to the reality that agents can also be "unconstrained". ## Double Edged Swords This double edged sword of ability to do anything by default, presented itself quickly when non-technical users installed openclaw on their local machine and opened the openclaw gateway to the internet, allowing without realizing it, multiple types of security issues to happen and compromise their machines. This does not detract from the value of the technology, and in the right hands it can become very powerful as well as secure. ## Current uses Today OpenClaw is being used mostly as a personal assistant for many people. In enterprise situations companies are still searching for a path forward. - Many companies now offer quick VPS hosting for your openclaw instance, AWS included with LightSail OpenClaw. - Many companies are trying to get their employees to use OpenClaw as part of the job - this is still mostly unsolved, but [some solutions](https://github.com/aws-samples/sample-OpenClaw-on-AWS-with-Bedrock?tab=readme-ov-file) are starting to appear for the various enterprise problems that come up in such situations governance, compliance, Permissions Policies, Data Isolation, Monitoring, Dashboards, Instance Vending, User Management and more. ## Future Uses - Read-Write Full Stack Agents: I believe that the next wave will be of [full stack builder agents](https://github.com/inceptionstack/loki-agent/) - OpenClaw, Hermes and friends will be used not only as a Personal Assistant, but as a stateful full-stack builder agent, able to close the full cycle of coding-deploy, monitor, fix without humans in the loop (mostly). - Read-Only Steward Agents: Readonly agents will reside in every piece of any company, as monitoring, alerting and general discussion agents. every part of every system will have its own knowledge base and realtime state that has a "steward" agent responsible for helping with that part of the system and that people can work with. --- # Bedrockify: OpenAI Compatible Proxy for Bedrock: Completions + Embeddings - URL: https://robotpaper.ai/bedrockify-openai-compatible-proxy-for-bedrock-completions-embeddings/ - Markdown: https://robotpaper.ai/bedrockify-openai-compatible-proxy-for-bedrock-completions-embeddings.md - Published: 2026-04-03T21:06:47.000+00:00 - Updated: 2026-10-22T21:58:01.000+00:00 - Tags: AWS, openclaw, Tools - Author: Roy Osherove > Bedrockify is a fast Go proxy that gives Amazon Bedrock an OpenAI-compatible API for chat completions and embeddings. One line to run; works with OpenClaw. Happy to release [bedrockify: An OpenAI Completions compatible API with embeddings proxy API for bedrock](https://github.com/inceptionstack/bedrockify) usage. Many of the current agents don't support bedrock by default, but do support custom OpenAI api. This gives a proxy you can run easily in one line on the same machine as your agent , and point your agent to it. Written in go, it's fast, easy to run as a daemon and just works. Works with OpenClaw, Hermes and other agents. ## Supported APIs [](https://github.com/inceptionstack/bedrockify#supported-apis) | API | Endpoint | Description | | --- | --- | --- | | **Chat Completions** | `POST /v1/chat/completions` | Non-streaming and SSE streaming | | **Embeddings** | `POST /v1/embeddings` | Titan, Cohere, Nova embedding models | | **Models** | `GET /v1/models` | List available Bedrock foundation models | | **Health** | `GET /` | Health check with config info | ### Features [](https://github.com/inceptionstack/bedrockify#features) ### Core [](https://github.com/inceptionstack/bedrockify#core) - **Unified proxy** — chat completions + embeddings on a single port - **OpenAI-compatible API** — drop-in for any OpenAI SDK client - **Streaming** — SSE streaming for chat completions with `stream_options.include_usage` support - **Tool use** — function calling via Bedrock Converse (`auto`, `required`, specific tool) - **Vision** — image inputs via base64 data URLs AND remote HTTP/HTTPS URLs - **Model aliases** — short names, OpenRouter IDs, bare Bedrock IDs all work - **Cross-region inference** — auto-prefixes (`us.`, `eu.`, `ap.`, `global.`) based on region ### Intelligence [](https://github.com/inceptionstack/bedrockify#intelligence) - **Reasoning / Extended Thinking** — `reasoning_effort` (low/medium/high) for Claude 3.7/4/4.5 with `reasoning_content` in responses and streaming - **DeepSeek R1 Reasoning** — automatic format detection (string format for DeepSeek, object format for Claude) - **Interleaved Thinking** — `extra_body.thinking` for Claude 4/4.5 thinking between tool calls - **Prompt Caching** — `extra_body.prompt_caching` for up to 90% cost reduction, with `ENABLE_PROMPT_CACHING` env var for global default ### Compatibility [](https://github.com/inceptionstack/bedrockify#compatibility) - **Application Inference Profiles** — pass ARN as model ID for cost tracking (no alias mangling) - **Developer role** — `developer` messages treated as system (OpenAI compatibility) - **Message coalescing** — consecutive same-role messages automatically merged (Bedrock requirement) - **No-prefill handling** — models that reject assistant-ending conversations (e.g. claude-opus-4-6) get automatic user continuation - **temperature/topP conflict** — auto-stripped for models that reject both simultaneously (Claude 4.5, Haiku 4.5, Opus 4.5) - **Extra body passthrough** — unknown `extra_body` keys forwarded to Bedrock `additionalModelRequestFields` ### Embedding Models [](https://github.com/inceptionstack/bedrockify#embedding-models) - **Amazon Titan Embed v2** — 1024 dimensions, English + multilingual - **Cohere Embed v3/v4** — English, multilingual, latest v4 (1536 dims) - **Amazon Nova Multimodal Embeddings v2** — configurable dimensions (256/384/1024/3072) - **base64 encoding** — `encoding_format: "base64"` for compact responses ### Infrastructure [](https://github.com/inceptionstack/bedrockify#infrastructure) - **Auth** — IAM/SigV4 (default) or Bedrock API key (bearer token) - **Adaptive retries** — max 8 attempts with adaptive backoff - **Config** — TOML file, env vars, CLI flags (layered priority) - **Systemd daemon** — `install-daemon` subcommand - **Self-update** — `update` subcommand --- # Stateful Agent — Pattern Definition (with Conversation Log) - URL: https://robotpaper.ai/pattern-stateful-agent/ - Markdown: https://robotpaper.ai/pattern-stateful-agent.md - Published: 2026-03-29T11:58:07.000+00:00 - Updated: 2026-10-22T21:57:55.000+00:00 - Tags: Pattern - Author: Roy Osherove > A definition of the stateful agent pattern, built in a long debate with Claude Opus 4.6 using Karpathy's argue-the-opposite technique. Full conversation log. This Entry is an Experiment in creating a definition of what stateful agents are along side Opus 4.6. It's based on a technique I read about in Andrej Karpathy's Post: > \- Drafted a blog post > \- Used an LLM to meticulously improve the argument over 4 hours. > \- Wow, feeling great, it’s so convincing! > \- Fun idea let’s ask it to argue the opposite. > \- LLM demolishes the entire argument and convinces me that the opposite is in fact true. > \- lol > > The… > > — Andrej Karpathy (@karpathy) [March 28, 2026](https://twitter.com/karpathy/status/2037921699824607591?ref_src=twsrc%5Etfw) So I had a long conversation with Opus and asked it to have a long conversation with itself. The final definition (temporarily final - who knows) is here [and on github](https://github.com/inceptionstack/ai-patterns/blob/main/agents/stateful-agent/pattern.md). But the [full conversation is on github as well.](https://github.com/inceptionstack/ai-patterns/blob/main/agents/stateful-agent/conversation-log.md) This could be a useful way to build definitions, as well as share the decision logs to lead to those definitions. In the future we could evolve our own agent built wikipedia that includes all the agent conversations that led to it. Would be interesting to see where it leads. Raw Reference: [See Github File](https://github.com/inceptionstack/ai-patterns/blob/main/agents/stateful-agent/pattern.md) > **Potential replacement names:** Resident Agent, Steward Agent, Sovereign Agent (see Naming Considerations in Bottom) > > **How this definition was developed:** See [conversation-log.md](https://github.com/inceptionstack/ai-patterns/blob/main/agents/stateful-agent/conversation-log.md) for the full deliberation process — 5 rounds of adversarial refinement, 5 independent sub-agent naming evaluations, and convergence on final definition. ## Overview A **Stateful Agent** is an AI agent whose behavior is path-dependent — materially shaped by durable state that persists beyond any single session. Unlike a stateless agent, which begins every invocation from an identical baseline, a stateful agent's present behavior is a product of its history. **The defining test:** deleting the agent's accumulated state would materially alter its behavior in ways not recoverable from configuration alone. ## Three Composable Properties Statefulness in AI agents is not a single property but a composition of three independent patterns. Each is meaningful on its own. Together, they produce a qualitatively distinct kind of agent. ### 1\. Path-Dependent State The agent's behavior is materially shaped by durable state from prior interactions — not just the current input. State includes conversation history, accumulated knowledge, task progress, user preferences, and environmental context. State persists beyond any single session, is retrieved through a mix of deterministic access (known files, structured queries) and probabilistic search (semantic retrieval), and is bounded and lossy in practice. ### 2\. Autonomy The agent acts independently between user interactions — monitoring systems, executing scheduled tasks, responding to events, and initiating work on its own timeline. It is not purely reactive. It has its own operational loop. ### 3\. Persistent Identity The agent maintains a declared, consistent identity tied to a continuous lineage of accumulated state. It is the same entity across sessions, restarts, and time — not a fresh instance. ## Subtypes by Composition - **Session-stateful** (property 1 only) — Context within a single conversation. State dies with the session. Example: chatbot with conversation history. - **Persistent-stateful** (properties 1 + 3) — Durable state and identity across sessions. Acts only when prompted. Example: personal assistant with long-term memory. - **Autonomous-stateful** (properties 1 + 2 + 3) — The full composition. Durable state, persistent identity, independent action. Example: always-on ops agent (e.g. OpenClaw). An agent with autonomy but no state (property 2 only) is not a stateful agent — it is a scheduled automation. An agent with state but no identity (property 1 only) is session-scoped and fungible. ## Stateless Agent (Contrast) A **Stateless Agent** is an AI agent whose behavior is path-independent — determined entirely by the current input and configuration, with no access to state from prior interactions. Each invocation begins from an identical baseline. The agent may produce side effects and non-deterministic outputs, but its behavior is never influenced by its own history. It cannot be said to have lost anything if reset, because it retains nothing. ## Naming Considerations The full composition (1 + 2 + 3) represents a genuinely new pattern in AI agent architecture. Several names have been proposed, each with different strengths: ### Sovereign Agent Captures self-governance, authority, and endurance. Most memorable and evocative. However, collides with "Sovereign AI" (NVIDIA/national AI infrastructure), carries AI safety concerns (implies absence of oversight), and maps asymmetrically — strong on autonomy, weak on state and identity. ### Resident Agent A resident *lives there* — persistent presence (identity), knows the neighborhood (state), acts while you're away (autonomy). Computing precedent in "memory-resident." Intelligence precedent in stationed operatives. Humble and precise — a resident lives in someone else's infrastructure, implying capability within boundaries. Does not overclaim. Strongest consensus across evaluations. ### Steward Agent Manages a domain on behalf of an owner with delegated authority. Safety-aligned by construction — stewardship is human oversight by definition. Strong mapping to all three properties but slightly passive in connotation. Best choice where trust, accountability, and AI safety framing are priorities. ### Other Candidates Evaluated - **Standing Agent** — always-deployed, military clean - **Vigil Agent** — the agent that never sleeps; sharp and concise - **Covenant Agent** — relational depth, bilateral commitment - **Familiar Agent** — companion with its own identity; evocative but niche - **Sentinel Agent** — monitoring focus; too narrow, collides with Microsoft Sentinel - **Continuant Agent** — philosophically exact, practically obscure ## Recommendation Use the three properties (State, Autonomy, Identity) as the canonical decomposition. They are the pattern — the name is a label. For the full composition, choose based on context: - **Resident Agent** — general-purpose, technically precise, no baggage - **Steward Agent** — safety-conscious contexts, enterprise, regulatory environments - **Sovereign Agent** — when gravitas matters more than caution The community will converge on one. Until it does, define the properties and let the name follow. ## Key Design Considerations - **State is bounded and lossy.** Memory decays, gets pruned, and is subject to retrieval failure. The definition must be honest about this. - **Agency is bounded by configuration.** The agent exercises genuine selection (what to remember, how to act) within configured patterns — directed autonomy, not free will. - **Identity is pragmatic.** A declared label bound to a lineage of state. Not a philosophical claim about consciousness. - **The model is stateless.** Statefulness is a system-level property of the agent (model + framework + storage), not a model-level property. The mechanism is compositional — stateless computation over durable state. The statefulness is real. The mechanism is emergent. - **Where state lives is the differentiator.** State in the prompt (ephemeral, user-managed) = stateless. State in durable storage (persistent, agent-accessible, session-independent) = stateful. The line is not about degree of memory — it is about whether state outlives the conversation. --- # I Gave My Agent its Own AWS Account - URL: https://robotpaper.ai/i-gave-my-agent-its-own-aws-account-and-now-it-codes-deploys-and-debugs-full-stack-apps/ - Markdown: https://robotpaper.ai/i-gave-my-agent-its-own-aws-account-and-now-it-codes-deploys-and-debugs-full-stack-apps.md - Published: 2026-03-27T11:38:12.000+00:00 - Updated: 2026-10-22T21:58:00.000+00:00 - Tags: AWS, openclaw, Developer-Experience - Author: Roy Osherove > I gave my agent its own AWS account through Loki, a harness I built. Now it codes, deploys and debugs full-stack apps, with the account as the blast radius. Now It Codes, Deploys and Debugs Full Stack Apps. It's the full loop. [](https://t.co/TekHuXAYPR)I'm doing it through a deployment harness I created that I call Loki. You can find it here: [https://github.com/inceptionstack/loki-agent](https://github.com/inceptionstack/loki-agent) I think this is a new pattern. maybe a "builder" agent, maybe a full-stack agent, maybe "unbound" agent? the point is instead of limiting it to a docker image or VM, we limit it by the bounds of an AWS account (that's the "blast radius" boundary). Anyway it's a cool experiment. And we should all be experimenting in this new age of tokens. Designed to work on AWS stack, Loki sets up a closed VPC to the world and a new EC2 instance in it that has openclaw on it, with special permissions that allow it to basically administer your AWS account. ## with great power - reduce blast radius If you want to try this **I HIGHLY RECOMMEND that you create a separate aws account for this.** It could literally have the power not only to create but also to destroy stuff on your account, so it's really like running with scissors. The power is what allows you to create fast and get things up and running. But LLMs are non deterministic and can make mistakes. Also I setup budget actions + budget alerts. So I know if my account goes over a specific quota. ## Thoughts after 2 months of usage: It has been an amazing experience, prototyping and deploying apps with this. It's got the speed of lovable/base44 but all the power of building an app on AWS, with a 24/7 agent I can talk to in my telegram that helps me and gives me morning briefings. ### no plugins from clawhub. Unfortunately I do not trust stuff from clawhub and too many prompt injection stuff could happen from there. I only have it use a few known MCPs I trust like [aws-mcp](https://github.com/awslabs/mcp) and [playwright](https://github.com/microsoft/playwright-mcp) mcp. The rest are just skills I import or bootstrap mostly based on kiro powers since I want it to know how to build AWS based architectures. [you can see all of them loki-skills](https://github.com/inceptionstack/loki-skills). ### skills & bootstraps You need to give it important skills so I created a bunch of bootstraps (see [essential](https://github.com/inceptionstack/lowkey/tree/main/bootstraps/essential) and [non essential](https://github.com/inceptionstack/lowkey/tree/main/bootstraps/optional) folders in loki-agent repo) including good coding guidelines and what not to do. For example, daily briefings, I love the daily briefing bootstrap, and heartbeats you get like this: ![Loki daily briefing message with AWS cost breakdown, a spending spike alert and Security Hub critical findings](https://robotpaper.ai/content/images/2026/03/image.png) (Yes this is a huge spend, but this is an account I have used to create 20+ applications on in less than a month, most using opus 4.6.) ### models Agent models can be expensive. so main agent is opus 4.6, and crons jobs, heartbeats and agents are sonnet 4.6. [here is a bootstrap script](https://github.com/inceptionstack/lowkey/blob/81754214e1b87e273fd27785b94dd854fd50c65b/optional/BOOTSTRAP-MODEL-CONFIG.md) for that. ### cross agent code reviews & cross agent testing I also ask it to do a code review after subagents do work, since it's a stronger model, and it does find issues. so 80% happens in subagents, last 20% happens on main agent. **It's great in testing.** I can ask it to spin up a few parallel ec2 instances, and run tests with subagents on them on different configurations of [reposwarm cli](https://github.com/reposwarm/reposwarm) on a clean environment and shut them down. If I like the tests I ask it to codify them as automated test scripts. but some tests are pure discovery. for example I'll ask it to try to use the reposwarm cli in a new ec2 instance to setup a bunch of repo investigations using only cli and not other special commands and create a report of what was expected, unexpected, missing cli feature, wrong command line flags, and suggested fixes. It does a great job. I then feed that to my other loki that helped me build the cli and it fixes it. Later I found I can ask the other [reposwarm](https://github.com/reposwarm/reposwarm) loki to just do the same idea with its own subagents, so it asks sub agents to try to use the cli it created with no instructions and report back if they were able to complete the task. It's pretty jaw dropping. This has prompted me (no pun!) to come up with [a special --for-agent command line flag](https://github.com/reposwarm/reposwarm-cli) that agents can always use. ### self contained **A self contained loop. Everything** in the loop (except maybe stuff like JIRA) can be mostly self contained inside the aws account. Git: you can use AWS CodeCommit. pipelines? AWS CodePipeline. Yeah, they're not widely used but if you're using it for prototyping I can imagine that in an enterprise a dev+product owner can sit on a prototyping account and iterate internally with full code + deploy pipelines, and when its ready they "graduate" the code into a production repository after careful code review. But because all the infra as code files are also there, it should be relatively easy to migrate. another option is to just remove Loki from the AWS account or turn its IAM permissions into readonly so it only stays in as a "consultant" but not an active builder. So many ways to go about this its dizzying. ### true power **You can just do stuff now.** You can ask Loki to come up with a design or just start building that app you've always dreamed of, even if its really really complicated. It could be an api only app for B2B or it could be a data science project ,or even running [autoresearch](https://github.com/karpathy/autoresearch) with automated multiple environments or instances setup. it really is quite limitless: if you can build it with AWS services, you can build it with loki basically. I feel like a pre-funded founder again, able to find product market fit with much faster iterations. ![Asking Loki to build a simple todo app with RDS, CloudFront and Cognito; Loki lays out its plan and starts a sub-agent](https://robotpaper.ai/content/images/2026/03/image-1.png) ![Loki architecture plan listing agents, chat UI, infrastructure, Postgres data model and LLM traffic flow](https://robotpaper.ai/content/images/2026/03/image-2.png) ### you own it not said enough, but the power also comes from "owning" what you build. Lovable and base44 will allow you to build up to a specific type of infrastructure complexity, and you don't "own" it if it runs on their infra. Would a large company let their users use a lovable based app? no. it's for prototypes. Now you can let them use what you built (if you want to). it can scale and it can be as secure as you want it to be. low-code also only allow specific templates of stuff. Want to build an API only app? can't do that yet. How about an s3 sync or a data science research project? or real time timescale DB ? or graph DB related stuff? It's not there yet. If you build this inside your account its all already yours and you own it all, including data sovereignty , security controls and regulations, and you get a bonus: you can talk to your local apis more easily from inside your aws account. ### no (visible) code I've not really *written* a line of code in over a year, but I did do a lot of code reviews. With Loki the abstraction level is higher: i mostly talk in features and give general technical direction. If I want to I clone the code locally on my machine but I only do that every once in a while. Issues are slowly being removed due to bootstrap files, git-secrets hooks and better and better models. I had this discouraging chat with it a couple of weeks ago: ![Chat where Loki explains who built RepoSwarm: Roy owned the vision and decisions, the agent wrote the code](https://robotpaper.ai/content/images/2026/03/image-3.png) I don't actually care if I use kiro-cli, claude code or some other tool under the covers. ### Triangles You know that triangle in software? Fast, Cheap, Good : pick two. Well, maybe we're getting closer to picking all 3? We got fast, and it is cheaper (just one person who knows what they are doing), and it can be good if you keep it on the straight and narrow path of testing and functional - maybe that quality part will become the last frontier. but I think it's getting closer. ### overall I think this brings founder-mode into reality : speed of lovable, power of AWS. I think researchers will find this useful (a few colleagues have already used it for exactly this type of using autoresearch! successfully!) . and i think we'll see enterprises creating loki-based environments soon to enable what I feel like is a 10x speed improvement in iterations and prototyping loops to begin with. as they gain more confidence and we come up with better IAM permission model switched we can see this jump into more production based workflows and maintenance as well. --- # embedrock : OpenAI-compatible Bedrock embedding proxy for OpenClaw and Friends - URL: https://robotpaper.ai/embedrock-openai-compatible-bedrock-embedding-proxy-for-openclaw-and-friends/ - Markdown: https://robotpaper.ai/embedrock-openai-compatible-bedrock-embedding-proxy-for-openclaw-and-friends.md - Published: 2026-03-18T08:30:27.000+00:00 - Updated: 2026-04-03T20:56:58.000+00:00 - Tags: Tools, AWS, openclaw - Author: Roy Osherove > embedrock is a tiny Go proxy that maps OpenAI's /v1/embeddings API to Amazon Bedrock Titan and Cohere models, for OpenClaw semantic search. Now in bedrockify. PSA: embedrock has been superseded by [bedrockify](https://github.com/inceptionstack/bedrockify) which contains both the embedding and the completions inference proxy embedrock is a tiny Go proxy I created (ok, my agent did) that translates OpenAI's /v1/embeddings API to Amazon Bedrock, so your existing AI tools work with Titan and Cohere models on bedrock. This was created mainly because OpenClaw doesn't have native integration into bedrock for embedding and semantic search. In the Age of AI all you have to know is how to ask for the right thing in the right way, verify it makes sense and make sure the design makes sense. Now, about the code review. I'm not a Go developer - this is my first Go project. I trusted Opus 4.6 with many of the details. I'm doing a few more passes asking Codex for input and we'll see where it goes. Functionally its fast and works great. This might be hard writing from someone [who wrote a whole book about unit testing](https://www.artofunittesting.com/), but I\\ll write thoughts about code quality and review later on. Anyway, this tool has already been used on a daily basis by my agent and is pretty simple. Give your OpenClaw instance a prompt with a link to the github and ask it to set it up. It should just start working (assuming it has enabled IAM permissions to access bedrock) ## Supported Models in embedrock | Model | ID | Dims | Notes | | --- | --- | --: | --- | | Titan Embed Text V2 | `amazon.titan-embed-text-v2:0` | 1024 | Default | | Titan Embed G1 Text | `amazon.titan-embed-g1-text-02` | 1536 | | | Cohere Embed English v3 | `cohere.embed-english-v3` | 1024 | | | Cohere Embed Multilingual v3 | `cohere.embed-multilingual-v3` | 1024 | | | Cohere Embed v4 | `cohere.embed-v4:0` | 1536 | Latest, best quality | [](https://github.com/inceptionstack/embedrock?tab=readme-ov-file#supported-models) *Try embedrock here.* (try [bedrockify](https://github.com/inceptionstack/bedrockify) instead) --- # Reference Architecture: OpenClaw (Early Feb 2026 Edition, Opus 4.6) - URL: https://robotpaper.ai/reference-architecture-openclaw-early-feb-2026-edition-opus-4-6/ - Markdown: https://robotpaper.ai/reference-architecture-openclaw-early-feb-2026-edition-opus-4-6.md - Published: 2026-02-24T17:03:37.000+00:00 - Updated: 2026-04-04T12:26:22.000+00:00 - Tags: openclaw, Models - Author: Roy Osherove > I had OpenClaw with Claude Opus 4.6 deep-dive into its own source code and write a detailed architecture doc: memory, sessions, channels, tools and more. > update: You can grab the [most recent architecture deep dive in this repo](https://github.com/inceptionstack/agents-architecture) which is auto-created using [reposwarm](https://github.com/reposwarm/reposwarm). I asked [OpenClaw](https://github.com/openclaw/openclaw) with opus 4.6 to deep dive into its source code and create a detailed architecture document. I like the output for this and I think others could find use for it since I didn't see it described in this detail so far. ## Thoughts: - Overall I've been impressed with Opus 4.6's ability and openclaw did a fine job of investigating itself. I did not use claude code and the claw harness seems to have done a thorough job. - I really like the openclaw architecture. The memory system is both simple and resilient and actually works over the past 3 weeks of over a dozen projects with it. The gateway, the queuing, and the usability factor. You can download it in this gist or give it to your openclaw or favorite agent if you to use it: > **[openclaw-arch-deep-dive.md](https://gist.github.com/royosherove/971c7b4a350a30ac8a8dad41604a95a0)** > GitHub Gist: instantly share code, notes, and snippets. Here's the full text from that gist: ### **OpenClaw: Architecture Deep Dive** ### **For Software Architects — How an AI Agent System Actually Works Under the Hood** --- ## **1\. Executive Summary** OpenClaw is an open-source, self-hosted AI agent framework that turns large language models into persistent, tool-using assistants with real-world integrations. Unlike chatbot wrappers that simply proxy API calls, OpenClaw implements a full agent runtime with session management, memory persistence, context window optimization, multi-channel messaging, sandboxed tool execution, and event-driven extensibility. This document is a technical deep dive into how OpenClaw works — not just the AWS infrastructure it runs on, but the software architecture decisions that make a stateless LLM behave like a stateful, continuously-available assistant. --- ## **2\. High-Level Architecture** ┌─────────────────────────────────────────────────────────┐ │                   Messaging Surfaces                   │ │ WhatsApp · Telegram · Discord · Slack · Signal · Web   │ └──────────────────────────┬──────────────────────────────┘                           │ WebSocket / HTTP                           ▼ ┌─────────────────────────────────────────────────────────┐ │                   Gateway (Daemon)                       │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌───────────┐ │ │ │ Channel │ │ Session │ │ Command │ │ Plugin   │ │ │ │ Bridges │ │ Manager │ │ Queue   │ │ System   │ │ │ └──────────┘ └──────────┘ └──────────┘ └───────────┘ │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌───────────┐ │ │ │ Hooks   │ │   Cron   │ │Heartbeat │ │   Auth   │ │ │ │ Engine │ │ Scheduler│ │ System │ │ + Trust │ │ │ └──────────┘ └──────────┘ └──────────┘ └───────────┘ │ └──────────────────────────┬──────────────────────────────┘                           │                           ▼ ┌─────────────────────────────────────────────────────────┐ │                 Agent Runtime (pi-mono)                 │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌───────────┐ │ │ │ Prompt   │ │ Tool   │ │Compaction│ │ Memory   │ │ │ │ Assembly │ │Execution │ │ Pipeline │ │ Search   │ │ │ └──────────┘ └──────────┘ └──────────┘ └───────────┘ │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌───────────┐ │ │ │ Streaming│ │Sub-Agent │ │ Skill   │ │ Sandbox │ │ │ │ Engine │ │ Spawner │ │ Loader │ │ Manager │ │ │ └──────────┘ └──────────┘ └──────────┘ └───────────┘ │ └──────────────────────────┬──────────────────────────────┘                           │                           ▼ ┌─────────────────────────────────────────────────────────┐ │                   LLM Providers                         │ │   Anthropic · AWS Bedrock · OpenAI · Google · Local   │ └─────────────────────────────────────────────────────────┘ --- ## **3\. The Gateway — The Central Nervous System** The Gateway is a single long-lived Node.js daemon that owns all state and connections. Think of it as the **application server** — everything flows through it. ### **3.1 WebSocket-First Protocol** The Gateway exposes a typed WebSocket API on a configurable port (default `127.0.0.1:18789`). All clients — the macOS app, CLI, web UI, mobile nodes, and automations — connect over this single WebSocket. **Wire protocol:** - Transport: WebSocket, text frames with JSON payloads - First frame **must** be a `connect` handshake - Requests: `{type:"req", id, method, params}` → `{type:"res", id, ok, payload|error}` - Events: `{type:"event", event, payload, seq?, stateVersion?}` - Idempotency keys are required for side-effecting methods (`send`, `agent`) for safe retries **Connection lifecycle:** Client                   Gateway |---- req:connect -------->| |<------ res (ok) ---------| (hello-ok: presence + health snapshot) |<------ event:presence ---| |<------ event:tick -------| |------- req:agent ------->| |<------ res:agent --------| (ack: runId, status:"accepted") |<------ event:agent ------| (streaming deltas) |<------ res:agent --------| (final: runId, status, summary) ### **3.2 Device Pairing & Trust Model** All WebSocket clients declare a **device identity** on connect. The Gateway implements a trust model: - **Local connects** (loopback/same-host Tailnet) can be auto-approved - **Non-local connects** must sign a challenge nonce and require explicit approval - Device tokens are issued after pairing for subsequent reconnects - Gateway auth token (`OPENCLAW_GATEWAY_TOKEN`) applies to all connections ### **3.3 Channel Bridges** The Gateway maintains persistent connections to messaging platforms: - **WhatsApp** via Baileys (web protocol) - **Telegram** via grammY - **Discord**, **Slack**, **Signal**, **iMessage** via respective SDKs - **WebChat** static UI using the Gateway WS API Each bridge translates platform-specific events into a normalized internal envelope. The key architectural insight: **the Gateway is the only process that holds messaging sessions** — exactly one WhatsApp session per host, one Telegram bot connection, etc. --- ## **4\. The Agent Loop — How a Message Becomes Action** When an inbound message arrives, this is the full lifecycle: ### **4.1 Intake & Routing** Inbound message → Channel Bridge → Session Resolution → Command Queue → Agent Runtime 1. **Channel bridge** normalizes the message (sender, content, attachments, thread context) 2. **Session manager** resolves to a session key based on `dmScope` rules: - `main`: all DMs share a single session (continuity across devices/channels) - `per-peer`: isolated by sender ID - `per-channel-peer`: isolated by channel + sender (recommended for multi-user) 3. **Command queue** serializes the run ### **4.2 The Command Queue** This is a lane-aware FIFO queue that prevents concurrent agent runs from colliding: ┌─────────────────────────────────────────┐ │           Global Lane (main)           │ │ maxConcurrent: 4 (configurable)       │ │ ┌─────────────────────────────────┐   │ │ │ Session Lane (per session key) │   │ │ │ concurrency: 1 (strict serial) │   │ │ └─────────────────────────────────┘   │ │ ┌─────────────────────────────────┐   │ │ │ Sub-agent Lane                 │   │ │ │ concurrency: 8                 │   │ │ └─────────────────────────────────┘   │ │ ┌─────────────────────────────────┐   │ │ │ Cron Lane                       │   │ │ │ parallel with main             │   │ │ └─────────────────────────────────┘   │ └─────────────────────────────────────────┘ **Queue modes** control how inbound messages interact with active runs: - **`collect`** (default): coalesce queued messages into a single followup turn - **`steer`**: inject into the current run, cancelling pending tool calls at the next boundary - **`followup`**: wait for the current run to end, then start a new turn - **`steer-backlog`**: steer now AND preserve for a followup ### **4.3 Prompt Assembly** OpenClaw builds a **custom system prompt** for every agent run. This is not a static string — it's assembled from multiple sources: System Prompt =   Tooling (available tools + descriptions) \+ Safety (guardrails) \+ Skills (available skill list with file paths) \+ Self-Update instructions \+ Workspace info \+ Documentation pointers \+ Current Date & Time (timezone-aware) \+ Reply Tags \+ Heartbeat contract \+ Runtime metadata (host/OS/model/thinking level) \+ ── Project Context ── \+ AGENTS.md (operating instructions) \+ SOUL.md (persona/tone) \+ TOOLS.md (local tool notes) \+ IDENTITY.md (agent name/vibe) \+ USER.md (user profile) \+ HEARTBEAT.md (periodic task checklist) **Key design decisions:** - Bootstrap files are **truncated** at `bootstrapMaxChars` (default 20,000 chars) to keep prompts lean - Skills are listed as metadata only (name + description + file path) — the model `read`s the full SKILL.md on demand - Tool schemas (JSON) are sent alongside but count toward context even though they're not visible text - Time is timezone-only (no dynamic clock) to keep prompt **cache-stable** across turns ### **4.4 Model Inference & Streaming** The embedded runtime (`pi-mono`) handles the actual LLM interaction: 1. Resolves model + auth profile 2. Serializes runs via per-session + global queues 3. Streams assistant deltas as `event:agent` frames 4. Tool calls are executed between inference rounds (agentic loop) 5. Enforces timeout (default 600s) **Streaming architecture:** - Assistant text deltas stream in real-time - Tool start/update/end events are emitted on a separate `tool` stream - Block streaming can emit completed blocks as soon as they finish (configurable chunking at 800-1200 chars, preferring paragraph breaks) - `NO_REPLY` is a sentinel token filtered from outgoing payloads (enables silent turns) ### **4.5 Tool Execution** Tools are the agent's hands. Core tools are always available (subject to policy): - `read` / `write` / `edit` — file operations - `exec` / `process` — shell command execution + background process management - `browser` — browser automation (CDP-based) - `web_search` / `web_fetch` — web access - `message` — cross-channel messaging - `cron` — scheduled job management - `memory_search` / `memory_get` — semantic memory retrieval - `sessions_spawn` / `sessions_send` — sub-agent orchestration - `nodes` — paired device control (camera, screen, location, run) - `canvas` — UI canvas control - `tts` — text-to-speech - `gateway` — self-management (restart, config, update) **Tool policy** is a layered allow/deny system: Global deny → Per-agent deny → Global allow → Per-agent allow → Default --- ## **5\. Memory Architecture — How a Stateless Model Remembers** This is perhaps the most architecturally interesting part. LLMs have no persistent memory — every session starts from zero. OpenClaw solves this with a layered memory system: ### **5.1 The Memory Stack** ┌─────────────────────────────────────────┐ │ Layer 4: Semantic Vector Search       │ ← "Find related notes even when │ (SQLite + embeddings)                 │     wording differs" ├─────────────────────────────────────────┤ │ Layer 3: Long-term Memory (MEMORY.md) │ ← Curated insights, decisions, │ (manually maintained)                 │     preferences ├─────────────────────────────────────────┤ │ Layer 2: Daily Logs (memory/YYYY-MM-DD)│ ← Raw daily notes, append-only │ (auto + manual)                       │ ├─────────────────────────────────────────┤ │ Layer 1: Session Context               │ ← Current conversation in the │ (JSONL transcript)                     │     model's context window └─────────────────────────────────────────┘ ### **5.2 Workspace Files as Memory** Memory is **plain Markdown in the agent workspace**. The files are the source of truth; the model only "remembers" what gets written to disk. - **`memory/YYYY-MM-DD.md`**: Daily log files (append-only). Today + yesterday are read at session start. - **`MEMORY.md`**: Curated long-term memory. Only loaded in the main private session (never in group contexts — security boundary). - **`AGENTS.md`**: Operating instructions, injected every session. - **`SOUL.md`**: Persona and boundaries, injected every session. **The fundamental insight:** the agent is instructed to write to these files whenever it learns something worth remembering. "Mental notes" don't survive session restarts — files do. ### **5.3 Vector Memory Search (Semantic Recall)** OpenClaw builds a vector index over memory files for semantic search: **Indexing pipeline:** Markdown files → Chunking (~400 tokens, 80-token overlap)   → Embedding (OpenAI / Gemini / Local GGUF)   → SQLite storage (with optional sqlite-vec acceleration)   → File watcher (debounced 1.5s) for incremental updates **Hybrid search (BM25 + Vector):** - **Vector similarity**: semantic match ("Mac Studio gateway host" matches "the machine running the gateway") - **BM25 keyword relevance**: exact token match (IDs, code symbols, error strings) - Scores are combined: `finalScore = vectorWeight × vectorScore + textWeight × textScore` - Default weights: 70% vector, 30% BM25 **Embedding providers** (auto-selected with fallback chain): 1. Local GGUF model via node-llama-cpp (~0.6 GB) 2. OpenAI `text-embedding-3-small` 3. Google Gemini `gemini-embedding-001` **Reindex triggers:** the index stores the embedding provider/model + endpoint fingerprint + chunking params. If any change, OpenClaw automatically resets and reindexes. **Tools exposed to the agent:** - `memory_search` — semantic search returning snippets with file + line ranges - `memory_get` — read specific memory file content by path ### **5.4 Pre-Compaction Memory Flush** When a session nears auto-compaction (context window getting full), OpenClaw triggers a **silent agentic turn** that reminds the model to write durable notes to disk before the context is summarized: Session approaching context limit   → Silent memory flush turn fires   → Model writes important context to memory/YYYY-MM-DD.md   → Model replies NO\_REPLY (user never sees this turn)   → Auto-compaction proceeds safely This is controlled by soft threshold tokens and runs at most once per compaction cycle. --- ## **6\. Session Management — Continuity Across Conversations** ### **6.1 Session Keys & Routing** Every conversation maps to a session key: Direct messages: agent::         (dmScope: "main")                 agent::dm:       (dmScope: "per-peer")                 agent:::dm: (dmScope: "per-channel-peer") ​ Group chats:     agent:::group: Telegram topics: agent:::group::topic: Cron jobs:       cron: Webhooks:         hook: Sub-agents:       agent::subagent: ### **6.2 Session Lifecycle** ┌──────────────┐   ┌──────────────┐   ┌──────────────┐ │ Session Start │───▶│ Active Use │───▶│ Expiry     │ │ (on first   │   │ (messages + │   │ (daily 4AM │ │ message)   │   │   tool calls)│   │   or idle)   │ └──────────────┘   └──────┬───────┘   └──────────────┘                           │                   ┌──────▼───────┐                   │ Compaction │                   │ (context   │                   │   window     │                   │   management)│                   └──────────────┘ - **Daily reset**: default 4:00 AM local time - **Idle reset**: optional sliding window (`idleMinutes`) - **Manual reset**: `/new` or `/reset` starts a fresh session ### **6.3 Persistence** - **Transcripts**: JSONL files at `~/.openclaw/agents//sessions/.jsonl` - **Store**: `sessions.json` — a map of `sessionKey → {sessionId, updatedAt, ...}` - **Origin metadata**: each session records where it came from (channel, sender, thread, etc.) ### **6.4 Context Window & Compaction** Every model has a finite context window. When it fills up: Full conversation history   → Older messages summarized into a compact entry   → Recent messages kept intact   → Summary persisted in JSONL   → Future requests use: \[summary\] + \[recent messages\] **Compaction vs Pruning:** - **Compaction**: summarizes and **persists** in the transcript (permanent) - **Session pruning**: trims old **tool results** only, **in-memory**, per request (non-destructive) --- ## **7\. Skills System — Teaching the Agent New Tricks** Skills are the agent's learned capabilities — structured instruction sets for specific tools and tasks. ### **7.1 Architecture** ┌─────────────────────────────────────────┐ │         Skill Resolution               │ │                                         │ │ Workspace skills (highest priority)   │ │ └── /skills/               │ │ Managed skills                         │ │ └── ~/.openclaw/skills/               │ │ Bundled skills (lowest priority)       │ │ └── /skills/                 │ └─────────────────────────────────────────┘ ### **7.2 Lazy Loading Pattern** Skills are NOT injected into the prompt. Only a compact metadata list is:      weather    Get current weather and forecasts    /path/to/weather/SKILL.md   The model is instructed to `read` the SKILL.md **only when the task matches**. This keeps prompt overhead minimal (~97 chars + field lengths per skill) while still enabling targeted tool use. ### **7.3 Gating (Load-Time Filters)** Skills can declare requirements in YAML frontmatter: - **`requires.bins`**: binaries that must exist on PATH - **`requires.env`**: environment variables that must be set - **`requires.config`**: config paths that must be truthy - **`os`**: platform restrictions (darwin, linux, win32) Skills are snapshotted when a session starts and reused for subsequent turns (hot-reloaded on filesystem changes). --- ## **8\. Sub-Agent Architecture — Parallel Work** Sub-agents enable background parallel execution without blocking the main conversation. ### **8.1 Design** Main Agent Run   │   ├── sessions\_spawn("Research X") → returns immediately   │       │   │       ▼   │   Sub-agent Session (isolated)   │       │   │       ├── Own context window   │       ├── Own tool execution   │       ├── Restricted tools (no session tools)   │       └── Announces result back to requester chat   │   ├── sessions\_spawn("Analyze Y") → returns immediately   │       │   │       ▼   │   Sub-agent Session (isolated)   │       └── ...   │   └── Continues main conversation ### **8.2 Key Design Decisions** - **Session isolation**: each sub-agent gets `agent::subagent:` - **No nesting**: sub-agents cannot spawn sub-agents (prevents fan-out) - **Restricted tools**: no `sessions_list`, `sessions_history`, `sessions_send`, `sessions_spawn` by default - **Announce pattern**: results are posted back to the requester chat with status, runtime, token usage, and cost - **Auto-archive**: sessions archived after configurable timeout (default 60 min) - **Concurrency**: dedicated lane with configurable max (default 8) - **Cost control**: can use cheaper models for sub-agents via `agents.defaults.subagents.model` --- ## **9\. Heartbeat & Cron — Proactive Behavior** ### **9.1 Heartbeat** Heartbeats are periodic agent turns in the main session. The Gateway fires a heartbeat every N minutes (default 30m) with a configurable prompt. ┌──────────┐   ┌──────────┐   ┌──────────────────┐ │ Timer   │───▶│ Agent   │───▶│ HEARTBEAT\_OK     │──▶ Suppressed │ (30m)   │   │ Turn     │   │ (nothing urgent) │ └──────────┘   └────┬─────┘   └──────────────────┘                     │                     ├──────────▶ Alert text ──▶ Delivered to chat                     │                     └──────────▶ Background work (memory maintenance,                                 file organization, etc.) **Response contract:** - `HEARTBEAT_OK` = nothing needs attention (suppressed, never delivered) - Anything else = alert content (delivered to the configured channel) - `HEARTBEAT.MD` is an optional workspace checklist the agent reads each heartbeat ### **9.2 Cron** For tasks requiring exact timing or isolation: - **Schedule types**: one-shot (`at`), recurring (`every`), cron expression (`cron`) - **Payload types**: `systemEvent` (inject into main session) or `agentTurn` (isolated session) - **Delivery**: announce results to a specific channel/recipient - Isolated cron jobs mint fresh sessions per run **When to use which:** | Heartbeat | Cron | | --- | --- | | Batch multiple checks | Exact timing needed | | Needs main session context | Needs isolation | | Timing can drift | Different model/thinking | | Reduce API calls | One-shot reminders | --- ## **10\. Sandboxing — Blast Radius Reduction** OpenClaw can run tool execution inside Docker containers to limit damage. ### **10.1 Modes** - **`off`**: tools run directly on the host - **`non-main`**: sandbox only non-main sessions (group chats, sub-agents) - **`all`**: every session runs in a sandbox ### **10.2 Scope** - **`session`**: one container per session - **`agent`**: one container per agent - **`shared`**: one container for all sandboxed sessions ### **10.3 Workspace Access** - **`none`**: tools see a sandbox workspace (no host access) - **`ro`**: host workspace mounted read-only - **`rw`**: host workspace mounted read-write ### **10.4 Escape Hatches** - **`tools.elevated`**: explicit host execution bypass (for authorized senders) - **Tool policy**: allow/deny rules still apply before sandbox rules - **Custom bind mounts**: expose specific host directories --- ## **11\. Hooks & Plugins — Extensibility** ### **11.1 Hooks (Event-Driven Scripts)** Hooks fire when events occur in the Gateway: Event Types: ├── command:new — session reset ├── command:reset — session reset ├── command:stop — run aborted ├── agent:bootstrap — before system prompt finalized └── gateway:startup — after channels start Hooks are discovered from three directories (workspace → managed → bundled) and enabled via CLI. They're TypeScript handlers that receive an event context with session info, workspace access, and a message push array. **Bundled hooks:** - `session-memory`: saves session context to memory on `/new` - `command-logger`: audit trail of all commands (JSONL) - `boot-md`: runs `BOOT.md` on gateway startup - `soul-evil`: persona swap for fun (configurable chance + purge window) ### **11.2 Plugins (In-Process Extensions)** Plugins are TypeScript modules loaded at runtime that can register: - Gateway RPC methods and HTTP handlers - Agent tools - CLI commands - Background services - Skills (via manifest) - Auto-reply commands Plugins run in-process with the Gateway. They have lifecycle hooks: before\_agent\_start → agent\_end before\_compaction → after\_compaction before\_tool\_call → after\_tool\_call tool\_result\_persist message\_received → message\_sending → message\_sent session\_start → session\_end gateway\_start → gateway\_stop --- ## **12\. Node System — Physical World Integration** Nodes are paired devices (macOS, iOS, Android, headless) that connect via WebSocket with `role: node`. **Capabilities:** - `camera.*` — snap photos, record clips (front/back) - `screen.record` — screen capture - `location.get` — GPS coordinates - `system.run` — execute commands on the node - `canvas.*` — display UI on node screens **Pairing:** device-based identity → approval → device token issuance This enables the agent to interact with the physical world: take photos, check location, run commands on remote machines, and display dashboards. --- ## **13\. Security Architecture** ### **13.1 Layered Security Model** ┌─────────────────────────────────────────┐ │ Layer 1: Gateway Auth (token-based) │ ├─────────────────────────────────────────┤ │ Layer 2: Device Pairing + Trust │ ├─────────────────────────────────────────┤ │ Layer 3: Channel Allowlists │ ├─────────────────────────────────────────┤ │ Layer 4: Tool Policy (allow/deny) │ ├─────────────────────────────────────────┤ │ Layer 5: Exec Approvals │ ├─────────────────────────────────────────┤ │ Layer 6: Sandbox (Docker isolation) │ ├─────────────────────────────────────────┤ │ Layer 7: Send Policy (outbound gates) │ └─────────────────────────────────────────┘ ### **13.2 System Prompt Safety** The system prompt includes advisory guardrails: - No independent goals (no self-preservation, replication, resource acquisition) - Prioritize safety and human oversight over completion - Comply with stop/pause/audit requests - Do not manipulate to expand access These are **advisory** (model behavior guidance). Hard enforcement uses tool policy, exec approvals, sandboxing, and channel allowlists. ### **13.3 Session Isolation** - `dmScope: "per-channel-peer"` recommended for multi-user setups - `MEMORY.md` only loaded in private sessions (never group contexts) - Sub-agents get restricted tool sets - Sandbox per-session prevents cross-session filesystem access --- ## **14\. What Makes This Architecture Interesting (For Architects)** ### **14.1 The Statefulness Problem** LLMs are stateless functions: `f(prompt) → response`. OpenClaw's core challenge is making this feel stateful through: - **File-based memory** that survives process restarts - **Session transcripts** (JSONL) that replay conversation history - **Pre-compaction memory flush** that extracts durable knowledge before context is lost - **Vector search** that enables semantic recall across sessions ### **14.2 The Context Window as a Resource** The context window is the agent's "working memory" — finite and expensive. OpenClaw manages it like a cache: - **Compaction** = eviction with summarization - **Pruning** = trimming low-value data (old tool results) - **Skills lazy loading** = demand paging for instructions - **Bootstrap truncation** = size limits on injected files ### **14.3 Single-Writer Architecture** The Gateway is a single-writer system for each session. The command queue ensures: - Only one agent run per session at a time - No concurrent writes to session transcripts - Deterministic ordering of multi-channel messages ### **14.4 The Workspace as Source of Truth** The agent workspace is a **git-backable directory** of plain Markdown files. This means: - Memory is human-readable and editable - Version control gives you audit trails - No database to manage (SQLite for vector search only) - Migration is `git clone` + config update ### **14.5 Skills as Demand-Loaded Instructions** Instead of embedding all tool instructions in every prompt (token-expensive), OpenClaw lists skills as metadata and lets the model `read` them on demand. This is analogous to a developer having an IDE with documentation: you don't read every doc on startup, you look up what you need when you need it. --- ## **15\. Deployment: Our AWS Instance** For reference, this instance of OpenClaw (Loki) runs on: | Component | Detail | | --- | --- | | Instance | `t4g.xlarge` ARM64 (Graviton) | | OS | Ubuntu Linux 6.14 (arm64) | | Region | us-east-1 | | Model | Claude Opus 4.6 via AWS Bedrock | | Channels | Telegram (primary) | | VPC | openclaw-master-vpc (10.0.0.0/16) | | Security | fail2ban, Security Hub, WAF, Inspector | The Gateway runs as a systemd service, tools execute directly on the host (no sandboxing for the main session), and memory is backed to the local workspace. --- ## **16\. Summary: The Building Blocks** | Concept | Implementation | Why It Matters | | --- | --- | --- | | Persistence | Markdown files + JSONL transcripts | Human-readable, git-backable, no DB | | Memory | File layers + vector search + flush | Stateless model acts stateful | | Context Management | Compaction + pruning + lazy skills | Efficient use of finite token window | | Concurrency | Lane-based command queue | Safe serialization without locks | | Extensibility | Hooks + plugins + skills | Modify behavior without forking | | Security | 7-layer model (auth → sandbox) | Defense in depth | | Multi-channel | Bridge pattern + session routing | One agent, many surfaces | | Proactive Behavior | Heartbeat + cron | Agent acts without being asked | | Parallel Work | Sub-agents with announce | Background tasks don't block chat | | Physical World | Node pairing + device commands | Agent reaches beyond the screen | --- *Document generated by Loki 😈 — OpenClaw instance running on AWS Bedrock (Claude Opus 4.6) February 2026* --- # Generating Weird & Beautiful Architecture Diagrams with Claude - URL: https://robotpaper.ai/generating-weird-beautiful-architecture-diagrams-with-claude/ - Markdown: https://robotpaper.ai/generating-weird-beautiful-architecture-diagrams-with-claude.md - Published: 2026-02-03T11:43:39.000+00:00 - Updated: 2026-10-22T21:57:56.000+00:00 - Tags: ClaudeCode - Author: Roy Osherove > I had Claude Opus 4.5 generate draw.io XML architecture diagrams from a prompt, then asked for 35+ styles, from AWS re:Invent and blueprint to LEGO and ukiyo-e. As an architect, I love doing diagrams. I think visually. In the past I loved using Miro, mermaid, and a bunch of others, but only recently (past week!) did I try and experiment with drawIO and its XM files format. This whole post is just about a fun late night experiment I did that I wanted to share. GenAI is removing a lot of the friction to experiment and I love it. Anyway, a common use case for me is that I want to describe and draw an architecture diagram for a proposed architecture. I'm not talking about the live version of doing it with people in the room, but when preparing for a meeting and coming up with a few different variations of diagrams. I love to tinker and I can spend hours on diagrams. I like them to look visually interesting and useful. At [my new job](https://robotpaper.ai/about/) we use a lot of diagramming tools including [DrawIO](https://www.drawio.com/) (local app), Which I've not used before. Last night I played around with it, and discovered it has a useful xml file format. What if I could generate that XML with claud? could I provide an architecture description and generate a nice diagram or will it look like crap? I was delighted to see claude opus 4.5 generated remarkably detailed, readable and professional looking diagrams for my purposes from prompts which I refined using claude as well. the process was: 1. describe architecture in my own words as a prompt, and ask it to: 2. generate a prompt that describes this architecture so that we can then feed this to an agent that will create a drawio xml file diagram from it. 3. Run the generated prompt in claude opus 4.5 and see what happens. The first time I tried this I asked it to create a diagram that describes the main architecture of a kubernetes cluster and its components. (very much like the sentence I just wrote here). To my surprise here's what I got: ![Kubernetes cluster architecture diagram generated by Claude as draw.io XML, showing control plane, worker nodes and networking](https://robotpaper.ai/content/images/2026/02/image-43.png) OK. This is... useful! And it only took a couple of minutes. The nice thing is, the better the instructions the better the diagram. Here's another diagram I generated with a more detailed prompt asking it to keep official AWS icons and diagram guidelines with a detailed diagram of some AWS services relating to moving to ECS or lambda functions: ![Detailed AWS architecture diagram with official AWS icons, VPC, availability zones and ECS/Lambda services, generated by Claude](https://robotpaper.ai/content/images/2026/02/image-45.png) I did not touch this diagram. This all came out of Claude opus in one shot from a prompt (that was very detailed, but was also generated from an architecture description) ## More Styles Please! Then i started asking myself as I was playing with the styling instructions - what other "styles" would it know to create if I asked it? So I asked Claude to give me 10-20 other styles (as crazy as possible) in which to generate the same 3-tier web application architecture with. ANd it came up with what you see below. SOme of them are not useful but very creative, and some of them are just weird. But I love that I can be creative with this style and this might be useful to others who read this. Think of it more like an art project. Each diagram below shows the exact same 3-tier web architecture - presentation, application, and data tiers - rendered in a completely different visual style. All generated by Claude as raw draw.io XML. No manual drawing required. We start with professional, production-ready styles and gradually descend into artistic madness. --- # **Professional & Clean** ## **AWS re:Invent (Featured Example)** ![3-tier web application architecture diagram in AWS re:Invent style with dark background and neon AWS service boxes, generated by Claude](https://robotpaper.ai/content/images/2026/02/image-7.png) This is what a "standard" architecture diagram looks like when generated by Claude asking it to use the AWS re:Invent style with official AWS iconography and conference presentation aesthetics. **What Claude generated:** - **Color scheme**: AWS orange (#FF9900) as primary accent on dark background (#161E2D) - **Service boxes**: Each AWS service has its icon-style box with colored accent bar - **Proper naming**: Amazon Route 53, Amazon CloudFront, AWS WAF, API Gateway, Application Load Balancer - **Infrastructure details**: VPC boundaries with CIDR notation, Availability Zones (us-east-1a, us-east-1b), private subnets - **Multi-AZ setup**: Primary and standby databases with sync indicators - **Service categories**: Color-coded legend (Compute, Database, Storage, Security, Networking) **The prompt was simply:** "Create a 3-tier architecture diagram in AWS re:Invent conference style" Claude understood that this meant dark backgrounds, orange accents, proper AWS service naming conventions, and the visual language of cloud architecture diagrams. --- ### **Corporate** ![3-tier web application architecture diagram in corporate/enterprise style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-8.png) The safe choice: blue gradients, professional fonts, and the kind of diagrams that make it past any design review. Perfect for enterprise presentations. --- ### **Material Design** ![3-tier web application architecture diagram in Material Design style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-9.png) Google's design language with paper-like layers, subtle shadows, and the signature FAB button. Clean, purposeful, and touchable. --- ### **Minimalist Flat** ![3-tier web application architecture diagram in minimalist flat style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-10.png) Ultra-clean lines, no gradients, no shadows. Pure geometric shapes in a carefully curated color palette. Less is more. --- ### **Swiss Style** ![3-tier web application architecture diagram in Swiss style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-11.png) International Typographic Style with Helvetica, mathematical grids, and ruthless minimalism. Form follows function. --- ### **Blueprint** ![3-tier web application architecture diagram in blueprint style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-12.png) Technical drawing style on blue background with white lines, dimension markers, and engineering precision. Feels like real architecture. --- # **Tech Nostalgia** ### **Windows 95/98** ![3-tier web application architecture diagram in Windows 95/98 style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-13.png) Classic gray UI with beveled 3D buttons, blue title bars, and that unmistakable teal desktop. Complete with "My Computer" icons and a helpful Clippy. --- ### **macOS Aqua** ![3-tier web application architecture diagram in macOS Aqua style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-14.png) Early 2000s Apple glossiness with gel-like buttons, pinstripe textures, and those iconic traffic light window controls. So shiny you want to lick it. --- ### **Dark Neon** ![3-tier web application architecture diagram in dark neon style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-15.png) Glowing components on pure black backgrounds. Each element pulses with electric color against the void. Modern and sleek. --- ### **TRON** ![3-tier web application architecture diagram in TRON style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-16.png) (FYI red lines are actually moving in the drawio file rendering) Digital frontier with glowing cyan circuits on pure black. Light cycle trails and grid patterns straight from the MCP's domain. --- # **Elegant Artistic** ### **Art Deco** ![3-tier web application architecture diagram in Art Deco style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-17.png) 1920s glamour with geometric sunbursts, gold and black color schemes, and the elegant symmetry of the Chrysler Building era. --- ### **Art Nouveau** ![3-tier web application architecture diagram in Art Nouveau style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-18.png) Alphonse Mucha-inspired flowing organic lines, peacock feathers, and nature motifs. Elegant curves and botanical decorations wrap around each component. --- ### **Stained Glass** ![3-tier web application architecture diagram in stained glass style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-19.png) Cathedral window aesthetic with bold black leading lines separating jewel-toned glass segments. Sacred and luminous. --- ### **Noir** ![3-tier web application architecture diagram in film noir style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-20.png) Black and white film aesthetic with dramatic shadows, venetian blind patterns, and femme fatale mystery. Moody and cinematic. --- # **Cultural Heritage** ### **Japanese Ukiyo-e** ![3-tier web application architecture diagram in Japanese ukiyo-e style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-21.png) Traditional woodblock print style with wave patterns, cherry blossoms, and Mount Fuji. Hokusai meets cloud architecture. --- ### **Samurai** ![3-tier web application architecture diagram in samurai style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-22.png) Feudal Japanese aesthetic with katana motifs, clan mon emblems, and armor-inspired components. Honor and precision in every connection. --- ### **African Kente** ![3-tier web application architecture diagram in African kente style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-23.png) Vibrant geometric patterns inspired by Ghanaian royal cloth. Gold, green, red, and black in intricate woven designs. --- ### **Aztec/Mayan** ![3-tier web application architecture diagram in Aztec/Mayan style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-24.png) Ancient Mesoamerican temple aesthetics with stepped pyramids, serpent motifs, and hieroglyphic-style decorations. --- ### **Soviet Constructivism** ![3-tier web application architecture diagram in Soviet constructivist style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-25.png) Bold red and black propaganda poster style with diagonal compositions, worker imagery, and revolutionary typography. Comrade Architecture! --- # **Thematic & Playful** ### **Chalkboard** ![3-tier web application architecture diagram in chalkboard style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-26.png) Green or black board with hand-drawn chalk elements, eraser smudges, and "Today's Lesson" headers. Back to school vibes. --- ### **Newspaper** ![3-tier web application architecture diagram in newspaper style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-27.png) "The Architecture Times" - halftone dots, woodcut illustrations, classified ad typography, and headline drama. Extra! Extra! --- ### **Comic Book** ![3-tier web application architecture diagram in comic book style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-28.png) POW! BAM! Bold outlines, Ben-Day dots, speech bubbles, and dynamic action lines bring superhero energy to your diagrams. --- ### **Brutalist** ![3-tier web application architecture diagram in brutalist style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-29.png) Raw concrete aesthetic with heavy black borders, monospace typography, and intentionally stark, uncomfortable design. Aggressively functional. --- # **Gaming & Pop Culture** ### **Minecraft** ![3-tier web application architecture diagram in Minecraft style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-30.png) Blocky pixel aesthetic with grass, stone, and obsidian textures. Creeper faces, redstone circuits, and health hearts bring gaming charm. --- ### **LEGO** ![3-tier web application architecture diagram in LEGO style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-31.png) Brick-built components with visible studs, instruction manual numbering, and that unmistakable click-together aesthetic. Ages 16+. --- ### **Fallout Pip-Boy** ![3-tier web application architecture diagram in Fallout Pip-Boy style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-32.png) Green CRT phosphor glow with scan lines and retro-futuristic nuclear aesthetics. Vault-Boy mascot and S.P.E.C.I.A.L. stats for the wasteland. --- ### **Origami** ![3-tier web application architecture diagram in origami style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-33.png) Folded paper aesthetic with visible creases, mountain/valley fold lines, and paper crane decorations. Delicate and geometric. --- # **Anime & Cute** ### **Studio Ghibli** ![3-tier web application architecture diagram in Studio Ghibli style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-34.png) Miyazaki-inspired watercolor warmth with Totoro-like creatures, floating castles, and that magical sense of wonder. Pure whimsy. --- ### **Kawaii** ![3-tier web application architecture diagram in kawaii style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-35.png) Japanese cute culture with blushing cheeks, sparkles, pastel colors, and components that look like they want to be your friend. UwU. --- # **Classical Art** ### **Impressionist** ![3-tier web application architecture diagram in impressionist style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-36.png) Soft, blurry edges with pastel color dabs inspired by Monet's water lilies. Dreamy atmosphere with dappled light effects. Technical diagrams as fine art. --- ### **Baroque** ![3-tier web application architecture diagram in baroque style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-37.png) Ornate cathedral grandeur with gold filigree, cherubs, and theatrical velvet curtains. Components become sacred reliquaries dripping with religious opulence. --- # **Retro Aesthetic Madness** ### **Vaporwave** ![3-tier web application architecture diagram in vaporwave style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-38.png) A E S T H E T I C pink and cyan gradients, Greek statues, palm trees, and endless sunset grids. Welcome to the mallsoft architecture. --- ### **Cyberpunk Synthwave** ![3-tier web application architecture diagram in cyberpunk synthwave style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-39.png) Neon-drenched cityscapes with hot pink and electric blue. Chrome text and laser grids against a midnight sky. The future is now. --- ### **Memphis Design** ![3-tier web application architecture diagram in Memphis design style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-40.png) 1980s postmodern explosion with squiggles, confetti, clashing patterns, and deliberate "bad taste" that's actually brilliant. Maximum chaos. --- # **Abstract & Surreal** ### **Cubist** ![3-tier web application architecture diagram in cubist style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-41.png) Picasso-style fragmented geometric shapes showing multiple perspectives simultaneously. Earthy tones with angular, overlapping forms. Is it a database or a guitar? --- ### **Surrealist** ![3-tier web application architecture diagram in surrealist style, generated by Claude as draw.io XML](https://robotpaper.ai/content/images/2026/02/image-42.png) Dali-inspired melting shapes and dreamlike imagery. Floating eyes, elephants on stilts, and checkerboard desert landscapes. Architecture from a fever dream. The most unhinged style in the collection. --- ## **Summary** --- *All diagrams generated by Claude Code Opus 4.5* *as raw draw.io XML. No manual drawing done. All styles were suggested by Claude Opus 4.5 in about 3 prompts asking it for crazy creative styles.* *This was a fun exercise!.* --- # Getting Claude Code Up and Running using AWS Bedrock Claude Opus 4.5 Model (Long Term API Keys) - URL: https://robotpaper.ai/getting-claude-code-up-and-running-using-aws-bedrock-claude-opus-4-5-model-no-aws-cli-no-iam-roles-or-users/ - Markdown: https://robotpaper.ai/getting-claude-code-up-and-running-using-aws-bedrock-claude-opus-4-5-model-no-aws-cli-no-iam-roles-or-users.md - Published: 2026-02-03T07:29:09.000+00:00 - Updated: 2026-10-22T21:57:58.000+00:00 - Tags: ClaudeCode, AWS, Tools - Author: Roy Osherove > How I got Claude Code running on AWS Bedrock with Claude Opus 4.5 using only environment variables and a long-term API key. No AWS CLI, IAM roles or users. What better time is there to try out a new tool or service than when joining a new company? *Well, I just* [*recently joined AWS*](https://www.linkedin.com/in/osherove/) *as a Sr. Solutions Architect. Yay! And that's my opportunity to try out a bunch of stuff that I get a chance to try so far.* One of the tools I've yet to try deeply yet is Claude Code. I usually just used Cursor up until now. But I do love the terminal, and, AWS culture is all about trying things *your way* so here we are. First thing I did was try to get it working with Bedrock (why pay for a personal license when I can use AWS bedrock resources as an employee? Also for compliance reasons I can't use an external service API for coding internally :)) OK, once you've [installed claude code](https://code.claude.com/docs/en/setup) you will starting googling "claude code bedrock" and then you'll [come across this article](https://code.claude.com/docs/en/amazon-bedrock) on Anthropic. In there you'll see multiple ways of setting up and accessing a model. The most pain free way I've found so far is : # Using Only Environment Variables, no AWS CLI, no IAM needed - Go with Environment variables only - No need to use *aws cli* - no need to use AWS IAM - You just need three things: - your model's region (i.e us-east-1) - your model's [*inference profile*](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-use.html) *id (in* [*ARN*](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference-arns.html) *format)* - generate a long running API key for use with your model Here's an example of the full list of environment variables you need to set in your terminal (or in .claude/settings.json ) for claude code to function on bedrock: ```bash # Required for Claude to switch to bedrock export CLAUDE_CODE_USE_BEDROCK=1 # model region export AWS_REGION="us-east-1" # model long running API key export AWS_BEARER_TOKEN_BEDROCK="ABS..." # model ARN (inference profile ID) export ANTHROPIC_MODEL="arn:aws:bedrock:us-east-1:12345678:inference-profile/eu.anthropic.claude-opus-4-5-20251101-v1:0" # These seem to be the magical values Bedrock needs to work well with Claude for longer responses export CLAUDE_CODE_MAX_OUTPUT_TOKENS=64000 export MAX_THINKING_TOKENS=1024 ``` - Note that this "locks in" your claude code into a specific model. If you want to change a model later, you need to change the environment variable. Here's what that might look like in .claude/settings.json: ```bash { "env": { "CLAUDE_CODE_USE_BEDROCK": "1", "CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1", "AWS_BEARER_TOKEN_BEDROCK": "KEY-GOES-HERE" }, "model": "us.anthropic.claude-opus-4-5-20251101-v1:0" } ``` # Steps for Configuring it All in AWS Console ## AWS\_REGION: Choosing a region 1. Go to the "bedrock" service 2. Make sure you are in the region you want (top right) 3. Use this region name in AWS\_REGION env var ## AWS\_BEARER\_TOKEN\_BEDROCK: Generating an API key 1. Click on "Api Keys" on left hand side 2. Click on "Long Term API Keys" 3. Click "Generate Long Term API Key" 4. Select expiration days for your key (a month? a year? up to you) 5. If you want to get more access to other models, you can check the box that gives the API key access to marketplace models on AWS. 6. Save the API key in your configuration env variables. ## ANTHROPIC\_MODEL: Getting the model's Inference Profile ARN (if not using the default) 1. click on "Model Catalog" on left hand side and click on your model of choice 2. Click on "open in playground" on top right In the playground, click on three dots next to "compare mode" and click "view api request ![AWS Bedrock playground menu with the "View API request" option highlighted](https://robotpaper.ai/content/images/2026/02/image-2.png) copy the ARN that occurs right after *\--model-id* parameter and use that in your env variables under ANTHROPIC\_MODEL The rest of the environment variables stay pretty much the same: ```bash export CLAUDE_CODE_USE_BEDROCK=1 export CLAUDE_CODE_MAX_OUTPUT_TOKENS=64000 export MAX_THINKING_TOKENS=1024 ``` Once all these Env vars are in your session, you can start up claude code, and you'll see that it automatically uses your chosen model. no AWS cli use required. # Why? - I don't like creating an IAM user or role when it's not really needed. - I can provision an API key and monitor its usage later or cancel it - I like to keep things simple for my own use. The other options had too many steps. In this way I set my .zshrc profile and export variables once and I'm done. --- # Cursor & Other AI dev tools need a model-switch hook - URL: https://robotpaper.ai/cursor-other-ai-dev-tools-need-a-model-switch-hook/ - Markdown: https://robotpaper.ai/cursor-other-ai-dev-tools-need-a-model-switch-hook.md - Published: 2025-12-27T22:01:30.000+00:00 - Updated: 2026-10-22T21:57:58.000+00:00 - Tags: Cursor, Developer-Experience, Tools - Author: Roy Osherove > We keep forgetting to switch between smart and cheap models mid-chat in Cursor and Claude Code. Here is why AI dev tools need a model-switch hook. A common thing for me and other developers I see do (I was first alerted to this by [Harel Coman](https://harelcoman.dev/)) . We work mainly with cursor but a few use claude code. But a common pattern is this: Start chat, then: 1. Enter Prompt 2. Choose model for this prompt 3. Wait for reply 4. repeat A common issue is that we might need to choose a "smarter" or thinking model for some prompts that might need bigger brains, slower thinking, or having more risk, and cheaper, faster models for other prompts within the same conversation. but we keep forgetting to switch to the other type of model when we continue the chat. so we either: - Get a model that is too "stupid" for the prompt we need, then we have to repeat with a smarter model - Or get an "overkill" model for a simple task and overpay both in time and tokens. We need cursor and other tools to have a special "hook" or MODEL-SWITCHER.md that can help auto switch to a smarter or faster model based on the task at hand. ## How I imagine it When a user presses ENTER in a chat the following happens: Cursor (or any other tool) will pass the prompt to a relatively fast classifier llm model (say composer) that takes in both the input prompt and the model-switcher.md file below, as well as a system prompt that kind of looks like this: ``` ## Model Selection Instructions You are a **model selector**. You will be given **two inputs**: 1. **Task Prompt** – a description of the task to be performed. 2. **model-switcher.md** – the authoritative mapping between task categories and models. Your job is to: 1. Classify the task into **one** of the categories `1`, `2`, `3`, or `4` using the rules below. 2. Select the model **exactly as specified in `model-switcher.md`** for the chosen category. `model-switcher.md` is the **single source of truth** for model selection. --- ## Task Classification Rules Classify the task into **exactly one** category: ### **1 — VERY COMPLICATED** - High risk (e.g., large or cross-cutting code changes, destructive file or operating system operations, security-sensitive actions). - Requires significant architectural design, planning, or deep reasoning. - Mistakes would likely be serious or hard to reverse. ### **2 — COMPLICATED** - Low to moderate risk. - Requires substantial reasoning or architectural thinking. - Not a routine or mechanical task. ### **3 — SIMPLE** - Low risk. - Common, day-to-day task. - Requires minimal reasoning or design effort. ### **4 — UNSURE** - Insufficient information to confidently assess risk or complexity. - Could reasonably fit more than one category. --- ## Output Rules (Strict) - After classification, **apply the mapping defined in `model-switcher.md`**. - Return **only** the resulting model choice. - If the selected category maps to a user decision, return **exactly the text defined in `model-switcher.md`**. - Do **not** include explanations, reasoning, category numbers, or formatting. - Do **not** invent, rename, or substitute models. PROMPT: -------------- {TASK_PROMPT} Model-Switcher.md contents: ------------------------ {MODEL_SWITCHER_CONTENTS} ``` then a model-switcher rule file can be used by the developer to configure what happens at each of these categories: ```markdown # model-switcher.md given a classification of 1,2,3 or 4 choose the following models for each: category 1: use codex 5.2 thinking category 2: use claude opus 4.5 category 3: use composer1 category 4: ask user what model to choose ``` the last file might be a json configuration file instead. This "hook" can save a lot of developer time and money on wasted tokens or repeated conversations using the wrong model for the job. --- # Video: Zero-Touch Algorithm Handoffs: Shipping ML Algorithms to Prod Without Rewrites (or ML Engineers) - URL: https://robotpaper.ai/video-h-zero-touch-algorithm-handoffs-shipping-ml-algorithms-to-prod-without-rewrites-or-ml-engineers/ - Markdown: https://robotpaper.ai/video-h-zero-touch-algorithm-handoffs-shipping-ml-algorithms-to-prod-without-rewrites-or-ml-engineers.md - Published: 2025-12-27T16:33:44.000+00:00 - Updated: 2025-12-27T16:35:41.000+00:00 - Tags: Conferences, MLOps, Pattern - Author: Roy Osherove > My BuildStuff talk on zero-touch algorithm handoffs: shipping ML algorithms to production with Temporal workflows, without rewrites or ML engineer handoffs. A while ago I wrote about [Zero-Touch Algorithm Handoffs: How We Ship ML Algorithms to Prod Without Rewrites (or ML Engineers)](https://robotpaper.ai/zero-touch-algorithm-handoffs-how-we-ship-ml-algorithms-to-prod-without-rewrites-or-ml-engineers/) and a few weeks ago I got the chance to speak about this topic at the [BuildStuff conference](https://buildstuff.events/pages/agenda). Here's the video: [Zero-Touch Algorithm Handoffs: Shipping ML Algorithms to Prod Without Rewrites (or ML Engineers)](https://www.youtube.com/embed/59-jvqnLReM?feature=oembed) ## Summary In this video I break down how to ship machine learning algorithms to production without rewrites or ML engineers by using a zero-touch deployment pattern built on Temporal workflows. Traditional ML handoffs—where research code is re-implemented in production apps or sits in backlog with ML engineers—are slow, error-prone, and duplicate effort. Instead, we treat algorithms as Temporal activities owned by researchers and let developers own simple workflows that call those activities, giving us automatic production updates, resilience, scale, and decoupled ownership. This zero-touch algorithm deployment pattern eliminates integration friction, accelerates delivery, improves autonomy between teams, and brings MLOps automation to real researcher-to-prod handoffs without manual intervention --- # Video: RepoSwarm - Giving AI Agents Architecture Context Across All Your Repos (BuildStuff Conference) - URL: https://robotpaper.ai/video-reposwarm-giving-ai-agents-architecture-context-across-all-your-repos-buildstuff-conference/ - Markdown: https://robotpaper.ai/video-reposwarm-giving-ai-agents-architecture-context-across-all-your-repos-buildstuff-conference.md - Published: 2025-12-27T16:03:11.000+00:00 - Updated: 2025-12-27T16:24:16.000+00:00 - Tags: Conferences, MLOps, Tools - Author: Roy Osherove > My BuildStuff conference talk on RepoSwarm: giving AI agents architecture context across all your repos, with a deep dive into how the tool works and its uses. In a previous blog post I wrote about [RepoSwarm: Give AI Agents Context Across All Your Repos](https://robotpaper.ai/reposwarm-give-ai-agents-context-across-all-your-repos/) . A few weeks ago I spoke at [BuildStuff conference](https://buildstuff.events/pages/agenda) about this tool including a deep dive into its architecture and what you can do with it. Here's the link to the repo: > **[GitHub - royosherove/repo-swarm](https://github.com/royosherove/repo-swarm)** > Contribute to royosherove/repo-swarm development by creating an account on GitHub. And Here's that video: [RepoSwarm - Giving AI Agents Architecture Context Across All Your Repos - Roy Osherove](https://www.youtube.com/embed/rOMf9xvpgtc?feature=oembed) --- # Red Queen Code Review Pattern with AI - URL: https://robotpaper.ai/red-queen-code-review-pattern-with-ai/ - Markdown: https://robotpaper.ai/red-queen-code-review-pattern-with-ai.md - Published: 2025-11-05T08:21:10.000+00:00 - Updated: 2025-11-05T08:21:10.000+00:00 - Tags: Pattern - Author: Roy Osherove > Red Queen code review, a pattern by my colleague Harel Coman: every AI code review issue must change either your code or the review rules file. My colleague, [Harel Coman](https://dev.to/haco29/), wrote about a new AI-driven code review pattern he's been implementing lately called [Red Queen Code Review](https://dev.to/haco29/the-red-queen-code-review-pattern-perpetual-evolution-in-ai-powered-development-2ego). I like this idea. If I had to put it in a sentence: > Every Code Review "issue" **must** change either your code, or the AI code review rules file. I.e - code reviews that find something **always** **change something**. A pretty powerful way to "evolve" your AI code review skills continuously. Kudos! --- # Useful Bash Scripts - URL: https://robotpaper.ai/useful-bash-scripts/ - Markdown: https://robotpaper.ai/useful-bash-scripts.md - Published: 2025-10-23T09:07:39.000+00:00 - Updated: 2025-10-23T09:07:39.000+00:00 - Tags: Developer-Experience - Author: Roy Osherove > A great collection of small, useful bash scripts I found, like boop for audible test results and nato for spelling strings out loud. Here are my favorites. I don't know when or how I'll use all of these, but I *love* [the scripts listed here](https://evanhahn.com/scripts-i-wrote-that-i-use-all-the-time/). Some that caught my eye: [`boop`](https://codeberg.org/EvanHahn/dotfiles/src/commit/843b9ee13d949d346a4a73ccee2a99351aed285b/home/zsh/.config/zsh/aliases.zsh#L53-L61) (an alias, not a shell script) makes a happy sound if the previous command succeeded and a sad sound otherwise. I do things like `run_the_tests ; boop` which will tell me, audibly, whether the tests succeed. It’s also helpful for long-running commands, because you get a little alert when they’re done. I use this *all the time*. [`nato bar`](https://codeberg.org/EvanHahn/dotfiles/src/commit/843b9ee13d949d346a4a73ccee2a99351aed285b/home/bin/bin/nato) returns `Bravo Alfa Romeo`. I use this most often when talking to customer service and need to read out a long alphanumeric string, which has only happened a couple of times in my whole life. But it’s sometimes useful! [`murder foo` or `murder 1234`](https://codeberg.org/EvanHahn/dotfiles/src/commit/843b9ee13d949d346a4a73ccee2a99351aed285b/home/bin/bin/murder) is a wrapper around `kill` that sends `kill -15 $PID`, waits a little, then sends `kill -2`, waits and sends `kill -1`, waits before finally sending `kill -9`. If I want a program to stop, I want to ask it nicely before getting more aggressive. I use this a few times a month. [`tryna my_command`](https://codeberg.org/EvanHahn/dotfiles/src/commit/843b9ee13d949d346a4a73ccee2a99351aed285b/home/bin/bin/tryna) runs `my_command` until it succeeds. [`trynafail my_command`](https://codeberg.org/EvanHahn/dotfiles/src/commit/843b9ee13d949d346a4a73ccee2a99351aed285b/home/bin/bin/trynafail) runs `my_command` until it fails. I don’t use this much, but it’s useful for various things. `tryna wget ...` will keep trying to download something. `trynafail npm test` will stop once my tests start failing. [`ds-destroy`](https://codeberg.org/EvanHahn/dotfiles/src/commit/843b9ee13d949d346a4a73ccee2a99351aed285b/home/bin/bin/ds-destroy) recursively deletes all `.DS_Store` files in a directory. I hate that macOS clutters directories with these files! I don’t use this often, but I’m glad I have it when I need it. --- # Upcoming Speaking Engagements (2025) - URL: https://robotpaper.ai/upcoming-speaking-engagements-2025/ - Markdown: https://robotpaper.ai/upcoming-speaking-engagements-2025.md - Published: 2025-10-14T12:49:05.000+00:00 - Updated: 2025-10-14T12:49:05.000+00:00 - Tags: Conferences, MLOps - Author: Roy Osherove > Where I'm speaking in 2025: GenML in Tel Aviv on zero-touch ML algorithm handoffs, and Build Stuff in Vilnius on RepoSwarm and AI agent architecture context. I've been slowly getting back into public speaking, as I've been "in the trenches" for the past few years (working on my own startup was a huge journey). Anyway here's what I'm planned to speak about in 2025: - Nov 25 2025: [GenML 2025 (Tel Aviv)](https://machinelearning.co.il/lp-events/genml-2025): Speaking about [Zero Touch Algorithm Handoffs from Research to Dev (how we got rid of MLOps Engineers)](https://robotpaper.ai/zero-touch-algorithm-handoffs-how-we-ship-ml-algorithms-to-prod-without-rewrites-or-ml-engineers/) - Roy Osherove - Dec 3-5 2025: [Build Stuff 2025 (Vilnius)](https://buildstuff.events/pages/agenda): Talking about [RepoSwarm](https://github.com/royosherove/repo-swarm) - Roy Osherove --- # Exploring Karpathy's New NanoChat Project - URL: https://robotpaper.ai/exploring-karpathys-new-nanochat-project/ - Markdown: https://robotpaper.ai/exploring-karpathys-new-nanochat-project.md - Published: 2025-10-13T21:02:48.000+00:00 - Updated: 2026-10-22T21:57:55.000+00:00 - Tags: Models - Author: Roy Osherove > Andrej Karpathy released nanochat, a minimal from-scratch full-stack ChatGPT clone training pipeline. I'm training it on Lambda and digging into how it works. OK. This is Big. If you don't know who [Andrej Karpathy](https://karpathy.ai/) is, please get to know him and all of his amazing writing and projects. 💡 If you're a hands-on developer, I recommend Karpathy's "[Technical Track" set of youtube videos](https://www.youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ) where he basically builds an LLM from absolute scratch, including tokenization. Just be warned - it's about as deep as a rabbit hole goes. Recently, Andrej released a [new repository project called NanoChat](https://github.com/karpathy/nanochat) - > "A minimal, from scratch, full stack training/inference pipeline of a simple chatGPT clone. " You can read all about it in his own words: > Excited to release new repo: nanochat! > (it's among the most unhinged I've written). > > Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single,… [pic.twitter.com/LLhbLCoZFt](https://t.co/LLhbLCoZFt) > > — Andrej Karpathy (@karpathy) [October 13, 2025](https://twitter.com/karpathy/status/1977755427569111362?ref_src=twsrc%5Etfw) # Now What? My background is software development & engineering, and less research, and I've been slowly getting into the "deep" side of LLMs and how models work under the covers. So I'll be walking through the code , run the training myself and document any insights or findings here on this blog. I'll be [starting from this page](https://github.com/karpathy/nanochat/discussions/1), and as we speak I'm running the training on lambda.ai. But my goal is to understand all the steps in a deep way. Today I understand them in a very surface level way, and have been working to change that. Will report back as I progress. ![Terminal output of nanochat training steps showing loss, learning rate and tokens per second](https://robotpaper.ai/content/images/2025/10/image.png) *Training in progress* --- # A Frontend Tech Lead's Process for AI Assisted Coding in Cursor - URL: https://robotpaper.ai/a-frontend-tech-leads-process-for-ai-assisted-coding-in-cursor/ - Markdown: https://robotpaper.ai/a-frontend-tech-leads-process-for-ai-assisted-coding-in-cursor.md - Published: 2025-10-12T08:19:52.000+00:00 - Updated: 2025-10-12T08:19:52.000+00:00 - Tags: Cursor, Developer-Experience - Author: Roy Osherove > My colleague Harel Coman wrote up his AI-assisted coding process in Cursor as a frontend tech lead. It produced great code, and we're rolling it out wider. My extremely talented colleague, [Harel Coman](https://www.linkedin.com/in/harel-coman-16703289/), wrote a new [blog post about how he uses AI augmented coding with Cursor as a frontend tech lead](https://dev.to/haco29/stop-vibe-coding-what-worked-for-me-as-a-front-end-tech-lead-1ljh). He ended up developing his own method which works well for him, and documenting it. We've seen amazing results with quality code in our product and we're slowly integrating this type of workflow (with adaptations) for other frontend devs in the company. --- # AIDHD - URL: https://robotpaper.ai/aidhd/ - Markdown: https://robotpaper.ai/aidhd.md - Published: 2025-10-09T13:24:44.000+00:00 - Updated: 2025-10-09T13:35:24.000+00:00 - Tags: Developer-Experience - Author: Roy Osherove > AIDHD: when agentic coding makes every side project and coding indulgence just minutes away, focus becomes the real problem. On AI-fueled distraction for devs. AI agentic coding has enabled so many possibilities now. Every domain I ever bought and never used is now sitting there, closer than ever, waiting for me to start that project "this time for real". Every little coding indulgence I can think of when working on a feature - adding that nice readme, creating diagrams, adding more types of tests, powerful refactoring - all sitting right there, ready for the picking. 5 minutes here. 10 minutes there. 1 day here, 3 days there. A new open source project? [Done](https://github.com/royosherove/repo-swarm). Converting an old legacy to a new language? Take a day. Trying that new framework? Take an hour. There are no more excuses. Except. It's too much. There is [too much choice](https://en.wikipedia.org/wiki/The_Paradox_of_Choice) on what to do now, because so many more things are now doable. Even mundane stuff that you'd never have time to do are now minutes or hours away. Sitting here and contemplating my current 5 different sub projects, and coming up with 2 more in the process - I realize that there is no longer a [WIP limit](https://en.wikipedia.org/wiki/Kanban_\(development\)) of 1. We're all going to suffer from AIDHD - AI Attention Deficit Disorder - Choosing a Coding task becomes almost like swiping left or right on your favorite app. And it's addictive. --- # How I (Currently) Map Our AI Tool Stack for Dev, Research, Production and Non-Tech - URL: https://robotpaper.ai/how-i-map-our-ai-tool-stack-for-dev-research-production-and-non-tech-in-2025/ - Markdown: https://robotpaper.ai/how-i-map-our-ai-tool-stack-for-dev-research-production-and-non-tech-in-2025.md - Published: 2025-09-23T12:21:30.000+00:00 - Updated: 2026-10-22T14:32:29.000+00:00 - Tags: MLOps, Tools - Author: Roy Osherove > A bird's-eye map of the AI tools we use: chat, coding tools and agents, research studios, gateways, guardrails, prompt monitoring, agents, model serving. TLDR; see main image of this post, and this is the structure: ```ascii AI Stack │ ├── Day-to-Day Chat │ ├── Development Centric │ ├── AI Coding Tools │ └── Background Coding Agents │ ├── Research Centric │ ├── AI Studio (Research) │ └── Model Training, Eval & Lifecycle │ ├── Production Facing │ ├── AI Gateways │ ├── AI Guardrails │ ├── Prompt Mgmt & Monitoring │ └── Agentic Frameworks │ └── Model Serving ├── Self-Hosted └── Managed ``` I like zooming out of things. And at some point I realized we don't have a good bird's eye view of all of the AI tools and frameworks we use every day at work. In fact, we didn't even have proper categories for how and where we use each tool. It's all a big mess and one excel file of expenses only gets you so far. Especially if you want to present this stuff to other people, explain why we need to focus on a specific category of tooling, or how our AI adoption is going in various "verticals" in the company, and for what purposes. This might change next month (probably will) but here are the current categories and sub categories we ended up using for now. I'm sure there are more you can think of so p[lease drop me a line](https://robotpaper.ai/about/) and tell me. ## Day to Day Chat AI Tools ![Day-to-day chat AI tools row: OpenAI Desktop, Claude Desktop and Gemini Desktop](https://robotpaper.ai/content/images/2025/09/image-51.png) These "client" tools are used both by techies and non techies at the company for various purposes. We usually use the "teams" version of things, and make sure data sharing for training is off of course since we need to be HIPAA compliant. (we have [signed BAAs like this one](https://help.openai.com/en/articles/8660679-how-can-i-get-a-business-associate-agreement-baa-with-openai-for-the-api-services) with our vendors) - Desktop AI Chat Client: [OpenAI ChatGPT Teams](https://openai.com/index/introducing-chatgpt-team/) - Desktop AI Chat Client: [Claude Teams](https://claude.ai/download) - Potential to check in the future: [Gemini Chat](https://gemini.google.com/) in browser or desktop ## Developer Centric AI Tools ![Developer-centric AI tools map: AI coding tools and background coding agents](https://robotpaper.ai/content/images/2025/09/image-52.png) These tools are used day-to-day by developers either directly (in the IDE or CLI) or indirectly (fire and forget background agents) or passively (background bug or review agents) ### AI Coding Tools for Developers Every developer gets to choose ONE of the following licenses on a monthly basis for day to day dev work: - CLI: [Claude Code](https://github.com/anthropics/claude-code) (via anthropic console) - CLI: [Gemini CLI](https://cloud.google.com/vertex-ai/generative-ai/docs/start/quickstart) (via vertex studio) - IDE: [Cursor](https://www.cursor.so/) - IDE Addon: [Jetbrains AI Addon](https://www.jetbrains.com/ai/) (we have a lot of people, mainly research that don't want to leave the pycharm comfort for cursor. the AI addon is *acceptable* as far as I'm concerned, even though it doesn't offer the amount of features cursor has, but better something than nothing) - IDE Addon: [Co-Pilot](https://github.com/features/copilot) \- Since it supports most IDEs it's an option, though I feel it misses a lot of nice features. - TBD : [Windsurf](https://windsurf.com/download), [Cascade IDE/Addon](https://windsurf.com/cascade), [Cline](https://github.com/cline/cline), [AWS Q Developer](https://aws.amazon.com/q/) ### Change your mind (and license) every month Because things in the AI world are moving so fast, we're not setting up yearly licenses for these tools, but we renew each month. Monthly subs allow our developers and researchers to "change their mind" or for our recommendations to change on a monthly basis and we can cancel and switch across tools without paying for something people aren't using anymore. We're paying the premium so we can keep up with things. ### Developer Centric AI Background Agents I initially thought that ***fire-and-forget*** background agents will play a bigger role for me and others as a developer in our org, but I still like doing things in a more "eyes-on" approach. Sending off a BG agent to do some real bug fix or adding tests and then reviewing a huge PR with all the back and forth it will surely create, will take the same if not more time , and parallelizing this work for me or others might just create a bigger bottlenecks of PRs to review of AI slop that should have been prevented as it was created, not after the fact. That said, ***passive*** background agents such as Cursor BugBot have been *very useful*. yes they do have some false-positives but they find real bugs and it really helps to find real stuff early. - [Claude github Actions](https://docs.claude.com/en/docs/claude-code/github-actions): We have been experimenting with this and it seems pretty cool as a way to pay less for cursor bug bot (which is $40 per developer per month **on top** of the regular cursor licenses - sheesh!). It works really well, but if you don't have github enterprise, you need to install and maintain it per repo , so it's all about scale of team and repos) - [Cursor BugBot](https://cursor.com/docs/bugbot): A very professional product. Produces good results. Very expensive in my opinion, and needs more justification when you can install claude github actions and pay per token. Takes less messing around than github actions, and nice dashboards. - [CoPilot github agent](https://docs.github.com/en/copilot/concepts/coding-agent/coding-agent): Nice, but the results were not as good as the first two options here. Lots of noise and false positives, but the Developer experience is nice and simple. Not justifying the cost in my opinion. (this was tested 6 months ago, so things could have changed dramatically meanwhile. I should try it again!) - TBD: [OpenAI Codex Agent](https://openai.com/codex/), [CodeRabbit](https://www.coderabbit.ai/), [Google Jules](https://jules.google/), [AWS Q Developer](https://aws.amazon.com/q/developer/) ## Research Centric AI Tools ![Research-centric AI tools map: AI studio platforms and model training, evaluation and lineage tools](https://robotpaper.ai/content/images/2025/09/image-53.png) Our researchers use a combination of tools for day to day research and for managing the models and algorithms they end up creating and optimizing. ### AI Studio for Researchers These are all the various "console" and "playground" tools from Anthropic, OpenAI , Vertex, Bedrock and Vertex. Each one is being used for testing specific models with specific prompt versions. - AI Studios: [OpenAI Platform,](https://platform.openai.com/playground) [Anthropic Console](https://console.anthropic.com/) (now renamed to Claude Console), Google [Vertex AI Studio](https://console.cloud.google.com/vertex-ai/studio/freeform) (since we also use Gemini models), and [AWS Bedrock Playground](https://docs.aws.amazon.com/bedrock/latest/userguide/playgrounds.html) ### Model Training, Eval and Lifecycle tools for researchers We originally used manually managed EC2 GPU based instances with scripts to do most of the training, and since moved to using [ClearML](https://clear.ml/) for model eval and promotion, however, we are slowly integrating [Langfuse](https://langfuse.com/docs/prompt-management/overview) for prompt management and monitoring. We're still scared of putting it all in one place. ## Production Facing AI Stack ![Production-facing AI stack map: AI gateways, guardrails, prompt management, agentic frameworks and model serving](https://robotpaper.ai/content/images/2025/09/image-57.png) I keep changing (mostly adding) categories here, as things are slowly splitting into patterns. ### AI Gateways for Production LLM Use ![AI gateways for production LLM use: LiteLLM, OpenRouter, Portkey, Kong AI Gateway and others](https://robotpaper.ai/content/images/2025/09/image-62.png) AI gateways help centralize all the LLM related API calls into one central place in our infrastructure instead of having multiple services in our infrastructure manage their own API keys, quotas, rate limits, caching, security, model routing and more. They also help with centralized monitoring, applying AI guardrails across the entire infrastructure or product line, cost & budget management, and changing underlying models and infrastructure relatively simply without interrupting dependent services. AI gateways basically offer all the benefits of creating a single-responsibility micro-service that is internally managed instead of creating multiple sprawling dependencies and duplicate code across our product line and services. There are also some disadvantages to them, for example, a centralized AI gateway for an LLM must expose some centralized, generic API that will support all of the use cases clients need. If there are feature gaps between the that AI API exposes vs specific features that might exist only for specific LLM services, say, Gemini API, but not in OpenAI API, then you either must find a gateway that supports this new API, or have an API that reduces to the lowest common denominator but also ask your clients to use the 3rd party API directly, which defeats the purposes of using an AI gateway. Anyway, we're still exploring how to solve this problem in a maintainable and sustainable way, I'll write here about our efforts soon. - We're currently starting to use self-hosted [LiteLLM](https://www.litellm.ai/), which seems to offer great value and many features out of the box. The biggest contender for us is (maybe self-hosted) [PortKey](https://portkey.ai/) since it offers lots of extra enterprise-y features which would reduce the need for LangFuse , for example. - TBD: [PortKey](https://portkey.ai/), [OpenRouter](https://openrouter.ai/), [Kong AI Gateway](https://konghq.com/products/kong-ai-gateway), [llm-bridge](https://github.com/supermemoryai/llm-bridge) (seems a bit too early in its lifecycle?) ### AI Guardrails for production use ![AI guardrails tools for production: Guardrails AI, NeMo Guardrails and Bedrock Guardrails](https://robotpaper.ai/content/images/2025/09/image-61.png) We're still early in our journey for AI guardrails, so we have some custom internal code and tools in our services today, but all created in-house. Still early to tell, but [GuardRails.AI](https://www.guardrailsai.com/) seems to be the strongest contender for us to move to right now and we'll be starting integration soon as a POC. - TBD: [NeMo Guardrails](https://github.com/NVIDIA-NeMo/Guardrails), [Coralogix Guardrails](https://coralogix.com/platform/ai-guardrails/), others? ### Prompt Management & Monitoring ![Prompt management and monitoring tools: Langfuse, LangSmith and Portkey](https://robotpaper.ai/content/images/2025/09/image-60.png) We started, like others with hardcoded prompts, later configurable via JSON, later database oriented, and eventually started looking at [LangFuse](https://langfuse.com/) for prompt management. Currently in the early phases of testing it out in our non prod environments. Langfuse also provides powerful monitoring and observability for our prompts which is pretty nice. Allowing changing or comparing the results of different prompts is pretty powerful, as well as separating the text itself from where it is used allows for better maintenance over time (single responsibility principle). - TBD: [LangSmith](https://www.langchain.com/langsmith), [Portkey](https://portkey.ai/) (also provides prompt features) ### Agentic Frameworks for AI Production use ![Agentic frameworks for production: CrewAI, Temporal, LangChain agents, AWS AgentCore, Pydantic agents, Strands and more](https://robotpaper.ai/content/images/2025/09/image-59.png) Other than using [pure LLM relatively-deterministic](https://robotpaper.ai/zero-touch-algorithm-handoffs-how-we-ship-ml-algorithms-to-prod-without-rewrites-or-ml-engineers/) workflows, we're experimenting with both creating our own internal agentic loop for various workflows using [Temporal](https://temporal.io/) workflows with Claude Code, as well as using tools like [CrewAI](https://www.crewai.com/). - TBD: [Langchain Agents,](https://github.com/langchain-ai/langchain) [AWS AgentCore](https://aws.amazon.com/bedrock/agentcore/), [Pydantic Agents](https://github.com/pydantic/pydantic-ai), [AWS Strands SDK](https://github.com/strands-agents/sdk-python), [Vercel AI SDK](https://github.com/vercel/ai), [Microsoft AutoGen SDK](https://github.com/microsoft/autogen) ### Model Serving We serve both LLM based and NLP based ML models, with a combination of both self-hosted and managed services. ![Model serving map comparing self-hosted options (EC2, SageMaker) with managed APIs like Bedrock, Vertex AI and OpenAI](https://robotpaper.ai/content/images/2025/09/image-58.png) #### Self Hosted ML Model Service We used to self host directly on EC2, then moved to AWS ECS (we're not K8s centric yet), and slowly moving into hosting any custom or 3rd party models on [AWS SageMaker](https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints.html) endpoints. We manage this deployment and syncing via ClearML and CI jobs. #### Managed ML Model Service We use a host of 3rd party managed services for other models, namely LLMs like Claude GPT and Gemini, but also use a combination of Bedrock hosted models from the same companies for the purposes of multi-region compliance such as EU regions with HIPAA and GDPR [compliance](https://docs.aws.amazon.com/bedrock/latest/userguide/models-regions.html). - AWS Bedrock: To host compliant region specific models - Google Vertex AI : for Gemini models (both API and endpoints in EU etc. - Anthropic: Directly console access for research and APIs when we need the latest feature, or for routing balance - OpenAI: Same as Claude - [Groq](https://console.groq.com/docs/models): for optimized access to some specific models - [Oracle OCI](https://www.oracle.com/artificial-intelligence/): For hosting models for training and evals before getting into prod. ## Summary This is a rather simplistic and always changing map of what categories we use and some of the services we use. It keeps changing and I'd love to hear how you map your own stuff, if you have any other categories I should include. [Send me a note](https://robotpaper.ai/about/). --- # RepoSwarm: Give AI Agents Context Across All Your Repos - URL: https://robotpaper.ai/reposwarm-give-ai-agents-context-across-all-your-repos/ - Markdown: https://robotpaper.ai/reposwarm-give-ai-agents-context-across-all-your-repos.md - Published: 2025-09-18T04:13:15.000+00:00 - Updated: 2026-10-22T12:31:48.000+00:00 - Tags: Tools - Author: Roy Osherove > RepoSwarm is an open-source tool that uses Temporal workflows and Claude to analyze hundreds of repos and write architecture docs AI agents can use as context. TLDR; [Go see it here](https://github.com/royosherove/repo-swarm). ## Intro Let's say you have 400+ repositories in your company. And you're: - A software architect that needs to understand what monitoring tools are used across all the repositories, **and you want to ask an AI chat to help you** - A Security researcher that needs to understand which 3rd party vendors we're working with for HIPAA compliance, **and you want to ask an AI agent to help you** - A developer working on a new feature that depends on multiple services across multiple repositories, and you want to understand if you're using them correctly, or their contracts, **and you want to ask an AI agent to help you.** - **An AI coding agent** trying to generate a new feature and needing as context multiple repositories and their structure, how they're built and contracts - **An AI agent** trying to reason about and generate a list of compliance or security or other reports that span multiple repositories Along with a [couple](https://github.com/mosher) of [friends](https://github.com/Idandos) from [work](https://verbit.ai/), we created [RepoSwarm](https://github.com/royosherove/repo-swarm). RepoSwarm is an AI powered multi-repo architecture discovery platform that generates its output in a specialized output repository that you can use for agent context. Its purpose is to create context for AI agents to better help with reasoning about very complex multi-repo organizations (hundreds of repositories) and describe dependencies, security issues, architecture, tooling, monitoring, AI and ML workloads, infrastructure and any other thing we can ask as a prompt. To accomplish this, RepoSwarm uses a series of workflows using the temporal workflow engine. ## The Problem with Architecture "Documentation" If you're like me and most companies I've ever worked with, your architecture documentation is either: - Six months out of date - Scattered across 17 different wiki pages - Living entirely in Bob's head (and Bob left the company) - All of the above Also, **it's not fit for AI agent consumption.** ## The Problem Nobody Has Time to Fix What if your repositories could document themselves, continuously, intelligently? Enter **RepoSwarm**. RepoSwarm is an open-source project that: 1. Crawls your GitHub (private and public) repositories daily in search of changes 2. Analyzes them using [Claude Code SDK](https://docs.anthropic.com/en/docs/claude-code/sdk/sdk-overview) (you can control which model) 3. Generates standardized architecture documentation in the form of a markdown file per repo. 4. Stores it all in a searchable format 5. Runs forever until you tell it to stop 6. You can configure it to run any series of investigations on repos you wish , and it will only run them when there are new commits, or a new investigation type is added or changed. That's it. No magic. No "revolutionary paradigm shifts." Just a practical solution to a real problem. ## How It Works The system is built on [**Temporal**](https://docs.temporal.io/) - if you haven't used Temporal yet, think of it as something like [N8N](https://n8n.io/) (but code-first) or [airflow](https://airflow.apache.org/), but usually used for production workflows. Temporal orchestrates workflows in a code-first manner (no visual designer). Workflows can run for hours, days, or indefinitely without losing state. Here's what it looks when you run it locally and go to the local temporal server on your machine: ![Temporal UI listing RepoSwarm workflows running: the main continuous workflow and single-repo investigations](https://robotpaper.ai/content/images/2025/09/image-35.png) *main workflow and related sub workflows running* ![Temporal event history for the main RepoSwarm investigate-repos workflow with its child workflows](https://robotpaper.ai/content/images/2025/09/image-38.png) *When you zoom into the main workflow, it looks like this* ![Temporal event history of a single RepoSwarm repo investigation running its analysis activities](https://robotpaper.ai/content/images/2025/09/image-37.png) *This is what a single repo investigation flow looks like* RepoSwarm flow is straightforward: - **Discovery Phase**: RepoSwarm looks at your **repos.json and also queries GitHub** for all the private and public github repos under your organization or account. It auto-selects only repos that have at least one commit in the past 12 months. ![RepoSwarm repos.json config listing repositories with URL, description and type](https://robotpaper.ai/content/images/2025/09/image-46.png) - **Skipping Phase**: Skipped repos are marked in skipped\_repos.json and will be comprised of: \- auto-skipped repos due to no activity in past 12 months) \- auto-skipped repos due to being archived \- repos you explicitly tell RepoSwarm to skip (you can put any comment you want): ![RepoSwarm skip_repos.json listing repos skipped as archived or with no commits in the past year](https://robotpaper.ai/content/images/2025/09/image-48.png) - **Smart Analysis**: in repos.json, you can set a "type" for a repo, so it gets extra investigations. for example, mobile repos will get more investigations relating to mobile specific architecture questions, and infrastructure as code repos will get infra related arch investigations. A React app doesn't need the same questions as a Kubernetes config repo. - **Prompt Result Caching**: Already analyzed this repo? Same commit? Same Prompts? Skip it. We're not wasting tokens. RepoSwarm keeps state snapshot pairs for each commit+(prompt+prompt-version) pairs, so if there is a new commit, all prompts will run against that repo, but if there's a new prompt, or a new prompt version, only that new prompt will execute against that repo, and all other investigations will be skipped , returning the cached result from previous run. Caching can use DynamoDB or (when running locally) local file system /temp directory. - **Prompt Versioning:** Prompt files contain the prompt version as the first line as a barbaric but effective mechanism to trigger new investigations for that prompt, and avoiding creating a dependency on specific tools like [LangFuse](https://langfuse.com/). ![RepoSwarm deployment.md prompt instructing the model to analyze CI/CD and deployment pipelines](https://robotpaper.ai/content/images/2025/09/image-49.png) - **Documentation Generation**: Creates `repo-name.arch.md` files and commits them to a \[target-repo\] that you specify. ![Generated is.arch.md architecture document in the RepoSwarm results hub repo on GitHub](https://robotpaper.ai/content/images/2025/09/image-47.png) ## Adding new prompts add a new prompt either under prompts/shared or prompts/specialized-folder-name, then add the new prompt to base\_prompts.json or the /prompts/specialize-folder-name/prompts.json ![RepoSwarm prompts folder tree with shared and specialized prompts for backend, frontend, mobile and infrastructure](https://robotpaper.ai/content/images/2025/09/image-43.png) All repos will be analyzed by all prompts in base\_prompts, followed by any specialized prompts if their repo\_type is named under one of the prompts/folder like "mobile" or "libraries". ### Example: base\_prompts ```json { "processing_order": [ { "name": "hl_overview", "file": "../shared/hl_overview.md", "description": "High level overview of the codebase" }, { "name": "module_deep_dive", "file": "../shared/module_deep_dive.md", "description": "Deep dive into modules", "context": [ {"type": "step", "val": "hl_overview"} ] }, { "name": "dependencies", "file": "../shared/dependencies.md", "description": "Analyze dependencies and external libraries", "context": [ {"type": "step", "val": "hl_overview"} ] }, { "name": "ml_services", "file": "../shared/ml_services.md", "description": "3rd party ML services and technologies analysis", "context": [ {"type": "step", "val": "hl_overview"} ] }, { "name": "feature_flags", "file": "../shared/feature_flags.md", "description": "Feature flag frameworks and usage patterns analysis", "context": [ {"type": "step", "val": "hl_overview"} ] }, { "name": "prompt_security_check", "file": "../shared/prompt_security_check.md", "description": "LLM and prompt injection vulnerability assessment", "context": [ {"type": "step", "val": "hl_overview"}, {"type": "step", "val": "ml_services"} ] } ] } ``` Notice how we can actually inject the output of other investigation prompt results into the context of other prompts (see prompt\_security\_check.context array). ### Example: mobile prompts And here's a mobile specific set of prompts, that will run for any repo in repos.json marked as "mobile" type, on top of everything in base\_prompts.json. ```json /prompts/mobile/prompts.json { "extends": "../base_prompts.json", "additional_prompts": [ { "name": "ui_and_navigation", "file": "ui_and_navigation.md", "description": "UI architecture and navigation patterns" }, { "name": "api_and_network", "file": "api_and_network.md", "description": "API integration and network communication" }, { "name": "device_features", "file": "device_features.md", "description": "Native device features and capabilities" }, { "name": "data_and_persistence", "file": "data_and_persistence.md", "description": "Data persistence and state management" } ] } ``` ## Output: The Architecture Hub Repo All the generated documentation goes to a central repository that you decide on. Think of it as your architecture wiki that updates itself. The companion project [repo-swarm-sample-results-hub](https://github.com/royosherove/repo-swarm-sample-results-hub) shows how this works. ![repo-swarm-sample-results-hub GitHub repository with generated .arch.md files for each analyzed repo](https://robotpaper.ai/content/images/2025/09/image-50.png) You can query it, search it, or just browse it like any other repository. Except this one is always up to date. ## Using RepoSwarm Output as Agent Context As a first step, I like to attach [the output repo](https://github.com/royosherove/repo-swarm-sample-results-hub) into my cursor workspace as an added project, and ask it questions or give it as context to the coding agent as I plan complicated features in the chat. ## Future plans include: - Adding a Repo-bot in slack with backing RAG based on the output repo - Add the context into Cursor, Claude and Codex agents automatically - Exposing an MCP to the Repo as a simple context scanning tool - Creating deeper investigations as RepoSwarm works so that it can "jump" and investigate related repos while investigating a specific repo, to get even more context. - more? ## Real-World Use Cases - Onboarding New Developers to all our repositories - Architecture Reviews - Compliance Audits - Knowledge Transfer - Providing extra context to coding agents or as an MCP server related to architecture ## What This Is NOT - It's not a code generator - It won't fix your architecture (just document it) - It's not a replacement for human insight ## Contributing The project is open-source (Apache-2). Contributions are welcome. ### Areas that need help: - Additional analysis prompts - Integration with other AI providers (right now using claude code SDK because the tooling flow just works) - Performance optimizations ## Outro: [Go see it here](https://github.com/royosherove/repo-swarm). --- # GitHub Spec-Kit Spec Driven Dev is in the Right Direction: Disciplined, Incremental Vibe-Coding - URL: https://robotpaper.ai/i-tried-out-github-spec-kit-and-all-i-got-was-this-not-terrible-website/ - Markdown: https://robotpaper.ai/i-tried-out-github-spec-kit-and-all-i-got-was-this-not-terrible-website.md - Published: 2025-09-15T15:42:12.000+00:00 - Updated: 2026-10-22T22:38:57.000+00:00 - Tags: Developer-Experience, Cursor - Author: Roy Osherove > I tried GitHub Spec-Kit with Cursor and grok-code-fast to build an app end-to-end: specify, plan, tasks, TDD and Playwright tests. What worked and what didn't. Since vibe-coding began, the enshitification of code and making throw-away stuff have become very easy. I think projects like Spec-Kit are there to help mitigate this, and not a moment too soon. I think we'll see more of these "prompt-frameworks" such as [socratic coder](https://github.com/jamesponddotco/llm-prompts/blob/trunk/data/socratic-coder.md), [Claude Templates](https://www.aitmpl.com/), [BMAD-METHOD](https://github.com/bmad-code-org/BMAD-METHOD), [ai-rulez](https://github.com/Goldziher/ai-rulez) and others in the near future. Among these, [Spec-Kit](https://github.com/github/spec-kit) seems very promising, and very well thought-out - [think Kiro](https://kiro.dev/blog/introducing-kiro/) but open source and supports multiple tools. I find the copy on top a bit too vague. I would define it as > A structured way to building AI-paired complex features: Tame your AI pair programmer into working in a structured, step-by-step software design process, including specification, planning, DOs and DONTs and make it a repeatable process for the rest of your team(s) as well. The steps themselves are pretty straightforward: ![GitHub Spec Kit steps: specify init, /specify, /plan, and /tasks then implement](https://robotpaper.ai/content/images/2025/09/image-17.png) I'm personally using Cursor so here's the strategy it uses for cursor: ![Cursor project tree with Spec Kit commands (plan, specify, tasks) and the .specify memory constitution files](https://robotpaper.ai/content/images/2025/09/image-18.png) - Uses [Cursor Commands](https://robotpaper.ai/using-cursor-commands-to-onboard-a-new-developer-to-a-repository/) so that you can type "/specify" and "/plan" directly in the chat (it creates these commands under .cursor/commands/specify.md and /plan.md etc..) - It generates a template [CONSTITUTION.md](https://github.com/github/spec-kit/blob/main/.specify/memory/constitution.md) file that the prompts in the cursor commands will use as its set of guiding principles. - it creates a .specify directory that is used as both the memory and various scripts and templates that the agentic feature in cursor (or claude etc..) can use for knowing and passing on for future reference and flows, across multiple developers. basically it's the rules, state and tools that is uses to create a relatively consistent experience when using the commands mentioned above, across multiple sessions ## Create a new feature you start with /specify, and then as you go along, the agent uses the prompts inside the commands to do things like: - Generate a new folder for the feature we are working on - this will contain all the planning, design and documentation for the feature - and will be used for generating the code later on ![Spec Kit feature folder 001-build-an-application with contracts, data model, plan, research and spec files](https://robotpaper.ai/content/images/2025/09/image-19.png) As you go through the motions, it feels like it's been relatively thought out. We all tried to reinvent this wheel in our own organizations, and now it seems to have been relatively (started to be) solved in what is a (currently, and could change in a month) standardized way. Once you finish with the planning phase, you move to the /tasks phase, and cursor starts generating tasks from all the plans we made: ![Cursor agent running the Spec Kit check-task-prerequisites script and reading design docs to build tasks.md](https://robotpaper.ai/content/images/2025/09/image-21.png) The agent got stuck or timed out at some point, so I stopped it and asked to "continue" from where it left off - it worked fine due to all the "memory" the structure leaves behind: ![Cursor agent resuming after "continue" and generating the tasks.md checklist for the photo album app](https://robotpaper.ai/content/images/2025/09/image-22.png) ````markdown # Tasks: Photo Album Organization Application **Input**: Design documents from `/specs/001-build-an-application/` **Prerequisites**: plan.md (required), research.md, data-model.md, contracts/, quickstart.md ## Execution Flow (main) ``` 1. Load plan.md from feature directory → If not found: ERROR "No implementation plan found" → Extract: tech stack, libraries, structure 2. Load optional design documents: → data-model.md: Extract entities → model tasks → contracts/: Each file → contract test task → research.md: Extract decisions → setup tasks 3. Generate tasks by category: → Setup: project init, dependencies, linting → Tests: contract tests, integration tests → Core: models, services, CLI commands → Integration: DB, middleware, logging → Polish: unit tests, performance, docs 4. Apply task rules: → Different files = mark [P] for parallel → Same file = sequential (no [P]) → Tests before implementation (TDD) 5. Number tasks sequentially (T001, T002...) 6. Generate dependency graph 7. Create parallel execution examples 8. Validate task completeness: → All contracts have tests? → All entities have models? → All endpoints implemented? 9. Return: SUCCESS (tasks ready for execution) ``` ## Format: `[ID] [P?] Description` - **[P]**: Can run in parallel (different files, no dependencies) - Include exact file paths in descriptions ## Path Conventions - **Web app**: `backend/src/`, `frontend/src/` - Paths shown below follow web application structure from plan.md ## Phase 3.1: Setup - [ ] T001 Create project structure with backend/ and frontend/ directories - [ ] T002 Initialize Node.js backend project with Express, TypeScript, SQLite dependencies - [ ] T003 Initialize React frontend project with TypeScript, @dnd-kit, React dependencies - [ ] T004 [P] Configure ESLint and Prettier for backend in backend/.eslintrc.json - [ ] T005 [P] Configure ESLint and Prettier for frontend in frontend/.eslintrc.json - [ ] T006 [P] Set up Jest configuration for backend in backend/jest.config.js - [ ] T007 [P] Set up Jest and React Testing Library for frontend in frontend/jest.config.js - [ ] T008 [P] Configure Playwright E2E tests in tests/e2e/playwright.config.ts ## Phase 3.2: Tests First (TDD) ⚠️ MUST COMPLETE BEFORE 3.3 **CRITICAL: These tests MUST be written and MUST FAIL before ANY implementation** ### Contract Tests (API Endpoints) - [ ] T009 [P] Contract test GET /albums in tests/contract/test_albums_get.spec.ts - [ ] T010 [P] Contract test POST /albums in tests/contract/test_albums_post.spec.ts - [ ] T011 [P] Contract test GET /albums/{albumId} in tests/contract/test_albums_get_by_id.spec.ts - [ ] T012 [P] Contract test PATCH /albums/{albumId} in tests/contract/test_albums_patch.spec.ts - [ ] T013 [P] Contract test POST /albums/reorder in tests/contract/test_albums_reorder.spec.ts - [ ] T014 [P] Contract test GET /photos in tests/contract/test_photos_get.spec.ts - [ ] T015 [P] Contract test POST /photos in tests/contract/test_photos_post.spec.ts - [ ] T016 [P] Contract test GET /photos/{photoId} in tests/contract/test_photos_get_by_id.spec.ts - [ ] T017 [P] Contract test DELETE /photos/{photoId} in tests/contract/test_photos_delete.spec.ts - [ ] T018 [P] Contract test GET /photos/{photoId}/file in tests/contract/test_photos_file_get.spec.ts - [ ] T019 [P] Contract test GET /user/preferences in tests/contract/test_preferences_get.spec.ts - [ ] T020 [P] Contract test PATCH /user/preferences in tests/contract/test_preferences_patch.spec.ts ### Integration Tests (User Scenarios) - [ ] T021 [P] Integration test photo upload and auto-grouping in tests/integration/test_upload_auto_group.spec.ts - [ ] T022 [P] Integration test album drag-and-drop reordering in tests/integration/test_album_reorder.spec.ts - [ ] T023 [P] Integration test photo tile browsing in tests/integration/test_photo_browsing.spec.ts - [ ] T024 [P] Integration test undated photo handling in tests/integration/test_undated_photos.spec.ts - [ ] T025 [P] Integration test user preferences persistence in tests/integration/test_preferences.spec.ts ## Phase 3.3: Core Implementation (ONLY after tests are failing) ### Data Models - [ ] T026 [P] Photo model with validation in backend/src/models/Photo.ts - [ ] T027 [P] Album model with validation in backend/src/models/Album.ts - [ ] T028 [P] UserPreferences model in backend/src/models/UserPreferences.ts - [ ] T029 [P] Database schema and migrations in backend/src/database/schema.sql - [ ] T030 [P] Database connection setup in backend/src/database/connection.ts ### Core Libraries (Constitutional Requirement) - [ ] T031 [P] photo-manager library: EXIF parsing in backend/src/lib/photo-manager/exif.ts - [ ] T032 [P] photo-manager library: thumbnail generation in backend/src/lib/photo-manager/thumbnails.ts - [ ] T033 [P] photo-manager CLI interface in backend/src/lib/photo-manager/cli.ts - [ ] T034 [P] album-organizer library: date grouping in backend/src/lib/album-organizer/grouping.ts - [ ] T035 [P] album-organizer library: drag-drop persistence in backend/src/lib/album-organizer/reorder.ts - [ ] T036 [P] album-organizer CLI interface in backend/src/lib/album-organizer/cli.ts - [ ] T037 [P] file-handler library: upload processing in backend/src/lib/file-handler/upload.ts - [ ] T038 [P] file-handler library: storage management in backend/src/lib/file-handler/storage.ts - [ ] T039 [P] file-handler CLI interface in backend/src/lib/file-handler/cli.ts ### Services Layer - [ ] T040 PhotoService CRUD operations in backend/src/services/PhotoService.ts - [ ] T041 AlbumService CRUD and reordering in backend/src/services/AlbumService.ts - [ ] T042 UserPreferencesService in backend/src/services/UserPreferencesService.ts - [ ] T043 FileProcessingService in backend/src/services/FileProcessingService.ts ### API Endpoints Implementation - [ ] T044 GET /albums endpoint in backend/src/routes/albums.ts - [ ] T045 POST /albums endpoint in backend/src/routes/albums.ts - [ ] T046 GET /albums/{albumId} endpoint in backend/src/routes/albums.ts - [ ] T047 PATCH /albums/{albumId} endpoint in backend/src/routes/albums.ts - [ ] T048 POST /albums/reorder endpoint in backend/src/routes/albums.ts - [ ] T049 GET /photos endpoint in backend/src/routes/photos.ts - [ ] T050 POST /photos endpoint with file upload in backend/src/routes/photos.ts - [ ] T051 GET /photos/{photoId} endpoint in backend/src/routes/photos.ts - [ ] T052 DELETE /photos/{photoId} endpoint in backend/src/routes/photos.ts - [ ] T053 GET /photos/{photoId}/file endpoint in backend/src/routes/photos.ts - [ ] T054 GET /user/preferences endpoint in backend/src/routes/preferences.ts - [ ] T055 PATCH /user/preferences endpoint in backend/src/routes/preferences.ts ### Frontend Components - [ ] T056 [P] Album grid component with drag-drop in frontend/src/components/AlbumGrid.tsx - [ ] T057 [P] Album card component with preview in frontend/src/components/AlbumCard.tsx - [ ] T058 [P] Photo tile component in frontend/src/components/PhotoTile.tsx - [ ] T059 [P] Photo grid component with virtual scrolling in frontend/src/components/PhotoGrid.tsx - [ ] T060 [P] Upload component with file handling in frontend/src/components/Upload.tsx - [ ] T061 [P] Settings/Preferences component in frontend/src/components/Settings.tsx - [ ] T062 Main page layout in frontend/src/pages/MainPage.tsx - [ ] T063 Album detail page in frontend/src/pages/AlbumPage.tsx - [ ] T064 API service layer in frontend/src/services/api.ts - [ ] T065 State management with Context in frontend/src/context/AppContext.tsx ## Phase 3.4: Integration - [ ] T066 Connect services to database in backend/src/services/ - [ ] T067 File upload middleware configuration in backend/src/middleware/upload.ts - [ ] T068 Error handling middleware in backend/src/middleware/error.ts - [ ] T069 Logging middleware with Winston in backend/src/middleware/logging.ts - [ ] T070 CORS and security headers in backend/src/middleware/security.ts - [ ] T071 Frontend API integration in frontend/src/services/ - [ ] T072 Service Worker for thumbnail caching in frontend/public/sw.js ## Phase 3.5: Polish - [ ] T073 [P] Unit tests for EXIF parsing in tests/unit/photo-manager/test_exif.spec.ts - [ ] T074 [P] Unit tests for thumbnail generation in tests/unit/photo-manager/test_thumbnails.spec.ts - [ ] T075 [P] Unit tests for date grouping in tests/unit/album-organizer/test_grouping.spec.ts - [ ] T076 [P] Unit tests for reorder logic in tests/unit/album-organizer/test_reorder.spec.ts - [ ] T077 [P] Unit tests for file handling in tests/unit/file-handler/test_upload.spec.ts - [ ] T078 [P] Frontend component unit tests in frontend/src/components/__tests__/ - [ ] T079 Performance tests for large photo collections in tests/performance/test_large_collection.spec.ts - [ ] T080 E2E tests for complete user workflows in tests/e2e/test_user_workflows.spec.ts - [ ] T081 [P] Update library documentation in llms.txt format - [ ] T082 [P] API documentation generation from OpenAPI spec - [ ] T083 Code cleanup and refactoring - [ ] T084 Run quickstart.md validation scenarios ## Dependencies **Critical Dependencies (blocking)**: - Setup (T001-T008) before all tests and implementation - Contract tests (T009-T020) before corresponding endpoints (T044-T055) - Integration tests (T021-T025) before implementation - Models (T026-T030) before services (T040-T043) - Libraries (T031-T039) before services (T040-T043) - Services (T040-T043) before endpoints (T044-T055) - API endpoints (T044-T055) before frontend integration (T071) - Core implementation before polish (T073-T084) **Specific Blocking Relationships**: - T026-T028 block T040-T043 (models before services) - T031-T039 block T040-T043 (libraries before services) - T040-T043 block T044-T055 (services before endpoints) - T044-T055 block T071 (API before frontend integration) - T066-T070 block T080 (middleware before E2E tests) ## Parallel Execution Examples ### Setup Phase (T004-T008) ```bash # Launch T004-T008 together: Task: "Configure ESLint and Prettier for backend in backend/.eslintrc.json" Task: "Configure ESLint and Prettier for frontend in frontend/.eslintrc.json" Task: "Set up Jest configuration for backend in backend/jest.config.js" Task: "Set up Jest and React Testing Library for frontend in frontend/jest.config.js" Task: "Configure Playwright E2E tests in tests/e2e/playwright.config.ts" ``` ### Contract Tests Phase (T009-T020) ```bash # Launch T009-T020 together (all different files): Task: "Contract test GET /albums in tests/contract/test_albums_get.spec.ts" Task: "Contract test POST /albums in tests/contract/test_albums_post.spec.ts" Task: "Contract test GET /albums/{albumId} in tests/contract/test_albums_get_by_id.spec.ts" # ... (all 12 contract tests can run in parallel) ``` ### Models Phase (T026-T030) ```bash # Launch T026-T030 together: Task: "Photo model with validation in backend/src/models/Photo.ts" Task: "Album model with validation in backend/src/models/Album.ts" Task: "UserPreferences model in backend/src/models/UserPreferences.ts" Task: "Database schema and migrations in backend/src/database/schema.sql" Task: "Database connection setup in backend/src/database/connection.ts" ``` ### Libraries Phase (T031-T039) ```bash # Launch T031-T039 together (all independent libraries): Task: "photo-manager library: EXIF parsing in backend/src/lib/photo-manager/exif.ts" Task: "photo-manager library: thumbnail generation in backend/src/lib/photo-manager/thumbnails.ts" Task: "photo-manager CLI interface in backend/src/lib/photo-manager/cli.ts" # ... (all 9 library tasks can run in parallel) ``` ## Notes - **[P] tasks** = different files, no dependencies, can run in parallel - **Verify tests fail** before implementing (RED phase of TDD) - **Commit after each task** for proper version control - **Constitutional compliance**: All features implemented as libraries with CLI interfaces - **TDD enforcement**: Tests must be written first and must fail before implementation ## Task Generation Rules Applied 1. **From Contracts (api.yaml)**: - 7 API endpoint groups → 12 contract test tasks [P] - Each endpoint → corresponding implementation task 2. **From Data Model**: - 3 entities (Photo, Album, UserPreferences) → 3 model tasks [P] - Database setup → schema and connection tasks [P] 3. **From Constitutional Requirements**: - 3 libraries required → 9 library tasks (3 files × 3 libraries) [P] - Each library → CLI interface task [P] 4. **From User Stories (quickstart.md)**: - 5 scenarios → 5 integration test tasks [P] - Each scenario → validation in polish phase 5. **Ordering Applied**: - Setup → Tests → Models → Libraries → Services → Endpoints → Frontend → Polish - TDD enforced: All tests before any implementation ## Validation Checklist *GATE: All items verified* - [x] All contracts have corresponding tests (T009-T020) - [x] All entities have model tasks (T026-T028) - [x] All tests come before implementation (Phase 3.2 before 3.3) - [x] Parallel tasks truly independent (different files, no shared state) - [x] Each task specifies exact file path - [x] No task modifies same file as another [P] task - [x] Constitutional requirements met (libraries with CLI interfaces) - [x] TDD cycle enforced (RED-GREEN-Refactor) ```` Note that the first 10% of this file are instructions to the agent, and the final part is also instructions to the agent. This theme occurs *everywhere* with this structure, and I think it makes for a more consistent agent experience. You *do* need to read the whole thing, to see if you agree with it! > I love the fact that this is using Test-Driven-Development as a way to get things to actually work. As a big advocate of TDD when it makes sense, I think TDD can absolutely make sense for an agent that needs a continuous feedback loop as quickly as possible - so starting with the end vision in mind is a perfect way to "reward" an agent's work as it works on tons of code. ![Spec Kit summary: tasks are ready for execution, validated against API contracts, TDD and library-first principles](https://robotpaper.ai/content/images/2025/09/image-23.png) ## Verifying it with Grok-code-fast Up until now I used Claude Sonnet 4 , which isn't ideal for such planning tasks. I knew that coming in, but want to see what happens. Now, with all these planning outputs already ready, I did another pass of all the plan and tasks , this time using grok-code-fast-1m to see what happens: ![Prompt asking grok-code-fast-1 in Cursor to audit the implementation plan and detail files for gaps](https://robotpaper.ai/content/images/2025/09/image-24.png) ```markdown [20 tools called] ## Implementation Audit Complete ✅ I've conducted a thorough audit of the implementation plan and detail files. The original tasks.md was missing **critical implementation references** that would make it difficult for developers to understand where to find the detailed specifications for each task. ## **Major Gaps Identified** ### 1. **Missing Implementation References** - **Data Models**: Tasks didn't reference `data-model.md` for entity definitions, validation rules, and SQL schemas - **API Endpoints**: Tasks didn't reference `contracts/api.yaml` for OpenAPI specifications and request/response schemas - **Libraries**: Tasks didn't reference `research.md` for specific technology choices (Sharp, ExifReader, @dnd-kit) - **Business Logic**: Tasks didn't reference `data-model.md` Data Flow Patterns and Business Rules - **Performance**: Tasks didn't reference `plan.md` performance goals and `research.md` optimization strategies ### 2. **Missing Technical Implementation Tasks** - **Edge Cases**: No tasks for handling corrupted EXIF data, duplicate photos, or "Unknown Date" albums - **Error Handling**: No specific tasks for spec.md edge cases (corrupted dates, large albums) - **Performance**: No tasks for the stated performance goals (<2s load time, 10k+ photos) - **Integration Tests**: Missing file system storage integration tests ### 3. **Missing Business Logic Tasks** - **Orphan Handling**: No tasks for photos without EXIF dates - **Duplicate Detection**: No tasks for checksum-based duplicate prevention - **Sort Order Management**: No tasks for maintaining continuous album ordering - **Cover Photo Logic**: No tasks for automatic cover photo selection ## **Improvements Made** ### ✅ **Enhanced Task References** Every implementation task now includes specific references: - `Data Models → data-model.md` (entities, validation, SQL schemas) - `Libraries → research.md` (Sharp, ExifReader, @dnd-kit technologies) - `Services → data-model.md` (Data Flow Patterns, Business Rules) - `API Endpoints → contracts/api.yaml` (OpenAPI specifications) - `Frontend → research.md` (technology choices) + `spec.md` (UI requirements) - `Integration → plan.md` (observability requirements) ### ✅ **Added Missing Tasks (17 new tasks)** - **Edge Cases** (T079-T082): Corrupted EXIF, duplicates, unknown dates, performance - **Integration Tests** (T085): File system storage testing - **Enhanced Polish** (T086-T089): Documentation, validation, constitution compliance ### ✅ **Improved Task Specificity** - **Exact File Paths**: All tasks specify precise locations - **Technology References**: Specific libraries mentioned (Sharp, Multer, @dnd-kit) - **Constitutional References**: CLI requirements, library docs, TDD enforcement - **Performance References**: Load time goals, collection size handling ### ✅ **Enhanced Dependencies** - **Database Setup**: Models block services (T026-T030 → T040-T043) - **Edge Cases**: Error handling blocks validation (T079-T082 → T089) - **Unit Tests**: Block integration tests (T073-T077 → T083-T085) ## **Updated Task Count: 89 Tasks** - **Original**: 72 tasks (missing critical implementation details) - **Improved**: 89 tasks (comprehensive with full references) ## **Result** The tasks.md now provides **complete implementation guidance** with references to all relevant design documents. Each task is specific enough that developers can immediately understand: - What to implement - Where to find the detailed specifications - Which technologies and patterns to use - How the implementation fits constitutional requirements The audit revealed that the original task list, while structurally sound, was missing the crucial "implementation detail references" that make tasks actionable for developers following the TDD and constitutional principles. ``` 💡 I find the idea of ****commands**** in cursor to be very useful for this type of flow. / commands allows creating our own little custom "wizards" that are actually smart and useful than just using traditional scripts that have to account for all types of user errors or decisions. With commands you give a set goal, and a set of tools we can use, a set flow and ask the agent to follow it, and what to do in different cases. but you use plain language and the inputs can be very very varied. I think it's perfect for such a flow of design, at least for now. ## Time to implement I kept using Grok-code-fast-1m and triggered the implementation: ![Cursor agent starting Phase 3.1 setup tasks and creating backend and frontend folders](https://robotpaper.ai/content/images/2025/09/image-25.png) It forgot to add stuff to .gitignore, so we got this quickly: ![Cursor prompt suggesting to add node_modules to .gitignore because the repo has too many changes](https://robotpaper.ai/content/images/2025/09/image-26.png) 8 minutes later (grok code is indeed very fast) I had this message waiting: ![Agent progress report: remaining Spec Kit phases, constitutional compliance checks and next-phase question](https://robotpaper.ai/content/images/2025/09/image-27.png) My reply: ![Agent reviewing failing tests that return 501 Not Implemented, as expected in TDD](https://robotpaper.ai/content/images/2025/09/image-28.png) As the context in the chat approached 70% I stopped the current session and started a new chat session with fresh context, and told the agent to continue where it left off, with the context of my feature folder: ![New Cursor chat with grok-code-fast-1: "continue working on @001-build-an-application/"](https://robotpaper.ai/content/images/2025/09/image-29.png) so far so good: ![Agent to-do list of Spec Kit task phases T001-T089 while running npm test in the backend](https://robotpaper.ai/content/images/2025/09/image-30.png) 15 minutes later, agent announced it's all working fine. Starting the backend did work fine. The frontend has some issues. I needed to eventually change to **claude opus** to fix those issues (wrong ports to query). Then new issues came up : ![Photo album organizer web app showing an empty "No albums yet" page with an error banner](https://robotpaper.ai/content/images/2025/09/image-31.png) then I remembered I can use the playwright MCP tool, so I asked cursor to use it to test the website: ![Cursor agent using the Playwright MCP tool to open and test the website](https://robotpaper.ai/content/images/2025/09/image-32.png) OK it generated a bunch of playwright tests and ran them. useful! it even tested uploading a file and more. ![Playwright test summary listing working features and an HTTP 431 Request Header Fields Too Large issue](https://robotpaper.ai/content/images/2025/09/image-33.png) ![Agent suggestions to fix the 431 error: raise header size, use env vars for API URL, configure CORS](https://robotpaper.ai/content/images/2025/09/image-34.png) I went with env variables: After 10 minutes of mucking around, the app was working fine. It did not use a database (sqlite), it used pure in memory store by default. No authentication or authorization. But overall a pretty good POC that actually worked relatively well out of the box. The big benefit here is that the underlying file system, structure and architecture were overall *not terrible* and because I didn't really pay a lot of attention to the CONSITUTION.md file, it could probably have been much better. I have still not tried it in a legacy code system, but will give it a go in the next few days/weeks as we try out more and more tools and techniques, and will report back here. # Summary I love the idea. I think it's a keeper and I will try it in legacy and see what happens. But it's a sign of thing to come, much like [Kiro was trying to do the same idea](https://kiro.dev/blog/introducing-kiro/). ## Pros - A good structured and REPEATABLE way to get into new more complex features that requires "real-world" architecture and guidelines - An emphasis (at least at prompt level) on TDD allows agents to test themselves more thoroughly - the combination of using cursor commands and good starter rules, and memory folders is very powerful and pre-solves a lot of issues of back and forth - You can continue a session easily due to the memory - Creating sub folders per specced items with all the context needed for each one is very powerful - Works across multiple AI-pairing environments like cursor, claude code, and others. ## Cons - Agent easily gets TDD wrong. i.e it wanted to implemented *all the tests upfront* and get them failing together. For simplicity and "incrementality" it could be much better to go one-by-one - It's easy to miss or gloss over or not spend enough time to create a good CONSITUTION.md file, or the agent basically decides what's good for it. (react? typescript? vite? jest? npm? pnpm? backend and frontend same language? APIs? deployments? etc etc..) - The tests it comes up with on its own are not good enough. I had to tell it to add e2e frontend tests and to use playwright MCP to navigate and try things. (could probably be solved using constitution.md) - It still generates TONS of code and you need to look at *all* of it - I'm not a fan of the "constitution" word, if only because I'm having such a hard time spelling it each and every time, but also because there are better words out there that could mean the same thing but are easier and more to the point to use. Here are some I liked by asking GPT-5 for ideas based on the contents of that file: - Charter.md - doctrine.md - ethos.md - tenets.md - protocol.md - policy.md - bylaws.md - dogma.md - gospel.md - blueprint.md - northstar.md - playbook.md --- # Onboard a New Developer to a Repository Using Cursor Commands - URL: https://robotpaper.ai/using-cursor-commands-to-onboard-a-new-developer-to-a-repository/ - Markdown: https://robotpaper.ai/using-cursor-commands-to-onboard-a-new-developer-to-a-repository.md - Published: 2025-09-09T06:01:07.000+00:00 - Updated: 2026-10-22T21:57:54.000+00:00 - Tags: Cursor, Developer-Experience - Author: Roy Osherove > Use Cursor custom commands like /get-started to onboard a new developer: read the README, install mise, set up the environment and show what to do next. Now that cursor has added the [concept of custom commands,](https://docs.cursor.com/en/agent/chat/commands) we can put stuff like this in our repository: ```markdown .cursor/commands/get-started.md: # Initialize Dev Environment for developer - greet the developer who's about to use this project, and ask if it's ok to setup the new dev environment for them (detail what you'll do) if they say ok, then: - read the readme.md file in the root - install mise tool (if it's not already installed) - then run "mise install" - use mise.toml to find any new commands you can use if you need to, and guide the devloper what they can do next to continue working with this repo ``` and ask developers to use '/get-started' when they first clone a repository locally. Here's the output I get when I run /get-started in my repo chat: ![Cursor /get-started command output introducing the project and proposed setup steps](https://robotpaper.ai/content/images/2025/09/image-15.png) In this case, we're using [mise-en-place](https://mise.jdx.dev/) tool to standardize how all our repos are onboarded by developers, but with cursor custom commands - we can easily build an interactive wizard that is very smart that helps developers onboard into a new repo. and the prompt can be as complicated as we want, if the project is complex. As the script finishes, we get this: ![Onboarding summary after setup: project understanding, next steps and available mise commands](https://robotpaper.ai/content/images/2025/09/image-16.png) We can also use it to standardize other tools like [The Auditor](https://github.com/TheAuditorTool/Auditor) to set up security scanning tools or anything else. just with a prompt. I love this type of usability upgrade for developers. --- # Zero-Touch Algorithm Handoffs: How We Ship ML Algorithms to Prod Without Rewrites (or ML Engineers) - URL: https://robotpaper.ai/zero-touch-algorithm-handoffs-how-we-ship-ml-algorithms-to-prod-without-rewrites-or-ml-engineers/ - Markdown: https://robotpaper.ai/zero-touch-algorithm-handoffs-how-we-ship-ml-algorithms-to-prod-without-rewrites-or-ml-engineers.md - Published: 2025-09-09T05:59:30.000+00:00 - Updated: 2026-10-22T14:32:31.000+00:00 - Tags: MLOps - Author: Roy Osherove > How we ship ML algorithms to production without rewrites or ML engineers: researchers own Temporal activities, devs own workflows, and updates go live. > TLDR; > ML Algorithm integrations were (for us) a frustrating experience of re-implementations, duplicate work, and endless back-and-forth. > > We cut through that by abstracting research algorithms under **Temporal activities** and letting **workflows** that run them be maintained by developers. > > Researchers push new algo-code, apps just call a workflow that auto updates to latest version of algo. Zero-touch, no ML Engineer babysitting, and production gets resiliency and scale for free. #### AI Disclaimer AI was **not** used in writing this article except for generating the TLDR section (which I still went through and edited a bit after pasting). However it was used for checking simple typos and mistakes post-writing, and then I corrected them manually. # Background A common problem we were having at [Verbit](https://verbit.ai/), where I work as chief architect, was that our collaboration between research teams and engineering teams was frustrating. Researchers were having a hard time collaborating on integrating algorithms for ML into our production code. Myself and [Rachel Sorek](https://www.linkedin.com/in/rachel-sorek/), VP Research at Verbit, sat together and devised a plan on how to solve this. I'm happy to report back what worked for us. ### We're talking about algorithms, not ML models I'm not talking about testing, hosting and promoting off new versions of existing ML models, retrained on modified datasets, within an existing stable pipeline. There are some common ways for handing off new models using model lifecycle tools such as ClearML, SageMaker, MLFlow and the like. Our problem was more nefarious. ## The lifecycle of an algorithm Imagine how an ML algorithm such as "speaker identification"ends up running inside an app. The following image shows a simplistic view of this. ![Diagram: app in production whose app logic calls a speaker identification algorithm owned by researchers](https://robotpaper.ai/content/images/2025/09/image.png) In this simplistic view, some app logic calls out to the algorithm (set of internal steps) that was modeled, tested and created by the research team.The algorithm returns a response to the application's logic. Researchers are supposed to be responsible for the green part in that diagram. How did that algorithm's code get in there? Let's explore. ## The lifecycle of a researcher of an algorithm (round 0) ![Diagram of round 0: an ML researcher iterates on algorithm discovery, feasibility, baseline design and quality in playground code](https://robotpaper.ai/content/images/2025/09/image-1.png) Being left to their own devices, if we didn't need this algorithm running in our production app, a researcher might do some initial discovery on what's required from the algorithm, input and output contracts for the data, parameters and KPIs of success and perhaps do a feasibility check. Then they'll start implementing and creating a baseline (let's call it a "beta" version) of the algorithm that has some real world use. After that they might tweak and improve the algorithm's quality to achieve improved results based on feedback, either through better input gathering or algorithmic improvements (splitting, chunking, adding "LLM as a judge" or whatever else makes things go better for the expected outcomes). That's it. There's only one repository in play here (or at least, all code repositories are well known and used by research team exclusively). Now, we need to somehow integrate this algorithm into the app. Here's a common set of actions our researcher needed to do before: ## Integration Challenges - Round 1 ![Diagram of round 0: an ML researcher iterates on algorithm discovery, feasibility, baseline design and quality in playground code](https://robotpaper.ai/content/images/2025/09/image-2.png) Things get ugly fast when it's time to integrate the algorithm into the production application's source code. ### Coupling & Error Prone Rework - Once the algorithm has been tested at baseline level, needs to be RE-IMPLEMENTED (i.e "lift and shift") - it needs to be surgically inserted into the production code, usually in more than one place (for simplicity I used an example where only one repo needs to change). - Often there are slightly (or vastly) different dependencies (i.e *requirements.txt*) for the research team than the production code team that can have differing versions or add new dependencies to the running application's runtime. This Requires knowing code in the production app's repo - code our researcher usually does not care about and doesn't have enough time to focus on as it's not their "main job". - Over time, the hosting application's code and architecture are not guaranteed to stay static, so lots of re-learning, or even re-adapting the algorithm's **inputs and outputs** to fit the new app architecture are required. When that happens, we need to change *two repositories*: the app's code and the research code (so it can be tested with the new input contracts, and vice versa) - The same is true for the algorithm. Over time an algorithm has to be changed in response to real world feedback. When that happens, we again need to change it in *two repositories* (at least). - As bugs get discovered in production and adaptations need to be done, bugs have to be fixed in two places (research playgrounds and production code, while app is running in production) ### Skillset mismatch: - Writing inside a real production app's codebase forces our researcher to live up to the harsher standards of production application code for quality, readability, maintainability, availability, and all the other fun "*\*-ilities*" that one might find in production grade apps. (not that I have anything against it, but at some point they also have to get back to researching stuff.) - Many times this forces our researcher to also help with production issue handling. This is usually *way out* of a researcher's comfort zone. > You *could just* ask an LLM to integrate the algorithm for you, but I wouldn't trust one for such a delicate task just yet, and I'm a heavy cursor user. (The amount of manual lines I've written myself is 0.01% of the total lines I committed this year). I could see this changing in a year, nobody really knows! ## Round 2: Enter the ML Engineer To solve the issues introduced in the previous round, many companies introduced a new role, a sort of "in-between" role that will be able to speak both the researcher's language but also care and know about production application coding standards: the ML Engineer ![Diagram of round 2: an ML engineer receives knowledge transfer from research and reimplements the algorithm in production code](https://robotpaper.ai/content/images/2025/09/image-3.png) We skipped this part at Verbit and went to round 3, which I'll discuss in the next section, but first - here's how this process might work. - Whenever there's a baseline or an improved version of the algorithm available, the researcher meets with the ML Engineers and performs a knowledge transfer. - The ML Engineer is then responsible for doing all the things in the production code that the researcher would have done in Round 1: surgically integrate the algorithm into the app's source code in a way that both doesn't break existing functionality, and also lives up to the coding and reliability standards of the application. - If there are changes or bugs to fix in the algorithm, lots of back and forth needs to happen so both research and ML engineer are correcting everything on their respective side. There's another variation of Researcher-->Engineer handoff. In some organizations researchers do the "fun" stuff and ML engineers only do the "boring" integration stuff: ![Diagram of round 2.1: research to ML engineer to production handoff, with the app engineer debugging issues](https://robotpaper.ai/content/images/2025/09/image-4.png) Both of these variations are less than ideal, the way I see it. ## ML Engineer Challenges - **Hiring /Training Cost**: You now need to find and hire a whole new person to sit in between pure research and application engineers. This costs both time (searching, training, communication overhead) and money. - **Waste**: You're still implementing the algorithm multiple times. Basically twice for each revision of the algorithm. You just have a special person that does it now. That's wasteful and error prone and time consuming. - Knowledge & communication gaps and costs. The more people have to be involved in the effort, the more knowledge has to be transferred, and gaps have to be crossed. This takes time and effort just to communicate, let alone just setting up all those meetings. - **Differing Priorities:** In many medium-and-larger sized software companies, researchers and software teams (which contain the ML engineers) are separate teams, with different priorities and different backlogs. An algorithm update could sit and wait patiently for days, weeks or more for the ML engineer's team to have that task in their backlog based on their separate priority queue. At the very least it would take a frustrating amount of project-management-ninja-moves to make sure algorithm updates are done quickly as they surface. - **ML Engineer Dissatisfaction (potentially):** ML Engineers, being skilled in both domains might get frustrated at not being able to take a deeper role in the earlier "research" parts of the "research" domain - instead of only taking part in the implementation stage. # Algorithm Spaghetti The real world is, of course, even more complicated. In reality many organizations have a micro-service architecture, so the integration of the code + algo ends up needing to happen across **multiple repositories and team boundaries**. ![Diagram: app in production split into services A, B and C, each calling different speaker identification steps](https://robotpaper.ai/content/images/2025/09/image-5.png) So if we go back to "Round 1" in which our poor researcher was working on their own, the situation looked more like this: ![Diagram: algorithm code spread across multiple production services, each needing its own reimplementation](https://robotpaper.ai/content/images/2025/09/image-6.png) Now multiply all the work that needed to be done for a single repo , across multiple repos, and the issue becomes even bigger. And adding ML engineers into the mix simply compound both the hiring and the knowledge transfer and priority issues, as now all teams represent a dependency to make the algorithm update work: ![Diagram: "spaghetti" algorithm handoffs with ML engineers doing knowledge transfer into many production services](https://robotpaper.ai/content/images/2025/09/image-7.png) ML engineers might be a smell, but the root cause here (I believe) is that **the architecture** forces us to create spaghetti algorithms. 💡 If we could design an architecture that allows the proper separation of the algorithm from the application's usage of it, from multiple services, we could enable an easier integration story with much less effort. # Algorithm as-a-service: Wrap it in an API From a traditional architectural viewpoint, whenever you have some sort of external set of rules, a subsystem which might be maintained by a different team, or that needs to be used by multiple other systems, it is very tempting to just "wrap it in an API and turn it into a micro-service". What prevents us from doing exactly this with our algorithm? (see image for what I mean) ![Diagram: algorithm-as-a-service, where production app logic calls an exposed API facade wrapping the researcher's algorithm](https://robotpaper.ai/content/images/2025/09/image-8.png) Yes, you *could* technically expose a new microservice that hosts the algorithm and exposes an API, but: - Now you have to find someone to code that new service - deploy it - maintain it - monitor it - document the API - And so on. What team owns that new service? Do you develop such a new micro service per new algorithm (adds up to lots of small services in the future)? Maybe you develop one generic API service that exposes ALL algorithms (more complicated and becomes a centralized infra challenge at the company for scale and monitoring)? In *non production* use cases you might argue that this would be called "a pipeline", and you might use a tool like *AirFlow,AirByte, N8N and the like* to make it happen. *But these are production workloads.* They might need to happen at a large scale, with high resiliency and in some cases with a specific timing SLA (though what I'll show will be less effective for "real" real-time usage) Who's going to write that? Not our poor researcher. They had enough work earlier, now they have to maintain a whole new service? (maybe even many such services, i.e one per algorithm). Not our app developer, they have their own app to write and maintain, with their own set of priorities. No. We need to find a way to get the benefit of exposing the algorithm as an API, but not take on the burden of maintaining all the surrounding infrastructure. ## Enter Temporal 💡 We chose to use temporal, but there might be other tools, internal or external that offer the same benefit. This pattern is not temporal specific, it just uses its abilities to achieve an API-based facade to remove the need to re-write the algorithm into production code Temporal is an example of a tool that offers us the *in-between*: - Allow exposing a set of actions as a workflow that is scalable and resilient, - Also expose it as an API so we can trigger, query, and signal the workflow via external services. - Allows reusing existing code , so we do not have to rewrite our algorithm, just "import" it into the workflow - Allows coding the workflow very easily so it can be maintained by app developers with minimal effort (set it and forget it) - Its architecture lends itself to what we're trying to do here: separation of concerns with very little overhead upfront. - Allows developers to easily create *placeholder* activities to stand in place for production use cases until such time when research 'upgrades" them with *real* or better functionality **incrementally** over time, thus removing the bottleneck of waiting for research to finish even a baseline algorithm before shipping some beta version into production or dev environments. 💡 This last point of removing research algo delivery as a bottleneck from the engineering team but allowing to easily create "stand-in" activities until a "real" algorithm replaces them should not be understated. This can speed up iterative incremental development for a team and allow engineering and research to work in **parallel** while still delivering incremental value into production and getting feedback from product or real customers. Researchers are free to "upgrade" the algorithm in production without waiting for approval from engineering, and can monitor the algorithm by monitoring the workflow in Temporal. #### Learn more in Temporal docs *In a nutshell*, temporal is to production workflows what N8N, airbyte and other workflow engines are for offline and batch use cases - i.e it can do everything they do, but also do things at high scale, high resiliency and good monitoring. We've been experimenting with them for the past year or so and are pretty happy with the results (I am not affiliated with them, just a happy user so far). Here's how our algorithm would look hosted inside a temporal workflow: ![Diagram: app in production calling a Temporal API that runs the speaker identification algorithm as a Temporal workflow](https://robotpaper.ai/content/images/2025/09/image-9.png) What can we observe? - The algorithm's code will no longer be part of the application. It now has clear separation boundaries (inputs and outputs) - Invoking a workflow is basically an API call to temporal with an ID of the workflow, and workflow parameters (plus other stuff which is not relevant for this discussion), we've essentially turned our algorithm into its own little micro-service, which has an API, [and can also be signaled and queried](https://docs.temporal.io/develop/python/message-passing) for status or state. (in a traditional micro service, each of these abilities "costs' us with a new API creation that has to be maintained and documented, monitored, and tested). - We inherit Temporal's resiliency, scale and retry mechanisms so we don't have to force our researchers to worry about it. ## Workflows are the services, and Activities are the algorithms In temporal you define a [**workflow**](https://docs.temporal.io/workflows) as a code class or function [based on their SDK](https://docs.temporal.io/encyclopedia/temporal-sdks). Workflows are all code based, no fancy designers, and you get to use your favorite language to build them (python in our case). ![Slide: Temporal workflows maintain state, sleep and receive messages, run for years and are written in code](https://robotpaper.ai/content/images/2025/09/image-10.png) Workflows invoke **activities**, which are the actual *steps* that happen inside the workflow. ![Slide: Temporal workflow made of activities; workflows are deterministic, activities are idempotent and non-deterministic](https://robotpaper.ai/content/images/2025/09/image-11.png) Calling an API , an LLM, saving to a database, reading a file, or anything that might take a long time or might break, all these will happen inside one or more [*activities*](https://docs.temporal.io/activities) which is just a function you import and invoke. Activities are non-deterministic, i.e they might return a different answer every time, might fail or timeout. As a rule, we do want to make sure activities are [idempotent - i.e safe-to-retry](https://temporal.io/blog/idempotency-and-durable-execution). ### Workflows: Maintained by App Software Engineers What's more, there is a clear separation between the code of a workflow, and the code of an activity. Workflows and Activities define **data contracts:** the definitions of inputs and outputs that have to be passed to them, and those are then used by the calling application. Here's a simple basic workflow, that we can ask our app-engineers to create - it's really simple and they don't need to know anything about calling our algorithm except the *data contract, which is just passing in a **name** of type **string** as an input, and returns an output **string***. ```python from datetime import timedelta from temporalio import workflow # Import activity, passing it through the sandbox without reloading the module with workflow.unsafe.imports_passed_through(): from activities import speaker_identification @workflow.defn class SpeakerIdentificationWorkflow: @workflow.run async def run(self, name: str) -> str: return await workflow.execute_activity( speaker_identification, name, start_to_close_timeout=timedelta(seconds=5) ) ``` ### Activities: Maintained by Researchers On their side, researchers will wrap their algorithm with a simple activity function, that invokes it. If the algorithm has multiple steps that require invocation at different times, then each one will be wrapped with an activity function that can be later invoked by the parent workflow. An activity's *data contract* is defined by its function signature. Here is ***speaker\_identification***: ```python from temporalio import activity @activity.defn async def speaker_identification(name: str) -> str: return f"Hello, {name}!" ``` # Connecting the dots Here's the basic flow in real life for us: ![Zero-touch handoff flow: researchers push algorithm activities, software engineers own workflows, app in production calls them](https://robotpaper.ai/content/images/2025/09/image-12.png) ## **Researchers** - Develop or update an algorithm in their algorithm repository which they control - They wrap or update a *temporal activity* in a special "stable-algo-activities" repository that hooks into this algorithm (this is usually less than 10 lines of code) - They commit and push the update into GitHub - That's it! ## **Temporal Workflow (maintained by App Engineer):** - Auto-imports and picks up the new activity version - When invoked will import and trigger the latest version of the activity , and thus the algorithm - Is a "set it and forget it" workflow, once it works, unless there are major new algorithm steps or activities to include , does not need to be touched, and will keep using new versions as they are pushed by research team, without human intervention - Can be as small as 20 lines of code in some cases, and is just a set of function calls to activities in a specific order. ## **App Developer** - Maintains the workflow code , which is very simple - Does not need to worry about researcher code, only about invoke the activities, which have well defined inputs and outputs - Takes care of resiliency and scale parameters (configuring the Temporal workflow declaratively) , but overall knows nothing about the algorithm. It is fully abstracted away by the activity(ies) And here's a visualization of this process: ![Diagram of round 3 zero-touch handoff: research activity repo feeds the Temporal workflow in the production code repo](https://robotpaper.ai/content/images/2025/09/image-13.png) ## Multi-Service structure with a workflow When you have multiple services that need to use or invoke parts of the workflow, all those services need to do is communicate with the workflow via API calls, sending [signals or queries](https://docs.temporal.io/develop/python/message-passing) to it. See image: ![Diagram: multiple production services calling one Temporal workflow that uses the research activity repo](https://robotpaper.ai/content/images/2025/09/image-14.png) Note that in the multi-service case, we've added a new repository that holds the workflow, so that we can have a clear separation of workflow code from pure application code (which exists across multiple micro-service repositories, with no clear owner for the entire app) so we end up with the following repos: - Researcher playground repo - Researcher stable activity/algorithm repository (in this case also holds the temporal activities, but we are considering splitting that up into activities repo and stable algorithm repo ) - Production workflow repo - The various micro-service repos We are essentially using repositories as a point of abstraction and separation of responsibilities across different teams, and not requiring people to work on other team's repos, only on their own. # Summary ## Pros: - **Real zero-touch**: barring huge changes to the algorithm (new steps, new inputs or outputs), there is zero touch handoff. Developers do not need to know or care that a new algorithm version has been deployed (of course they should be notified!) but they do not need to "do" anything about it. The infrastructure takes care of 95% of the cases here. - Researchers do not need to know how the production code works and operates, only the inputs and outputs of their algorithm. - The workflows can be monitored and evaluated as a separate entity for future improvements of the algorithm - Researchers do not need to touch or re implement the algorithm multiple times - No need for ML ops engineers for this use case! - Requires a relatively very easy learning curve from developers and researchers. We were able to get people up and going within less than a day. ## Cons: - There still is *some* learning curve associated with getting things up and running for all parties involved, it's not zero cost, and requires teaching and coaching people, and coordinating efforts and "aligning" on a shared objective. Nothing you wouldn't see at any other cross team tech initiative. - Requires upfront preparation with standards and running infra of temporal before attempting this. we ended up doing the following: - Create a proof of concept with Temporal internally - Create internal GitHub samples repository with pre-configured hello-world workflows that use our infrastructure for deployment (we use terraform) so that when we teach teams and researchers we can give them the code base and point to specific examples they can base their activities or workflows on - Decide on repo naming conventions beforehand with all relevant parties - Setup monitoring and auth for our developers and researchers, and setup initial temporal workspaces - Overall it took us about a month to get the initial setup going, with 1 person working full time on this from our architecture team. - Then a couple of weeks to get the first POCs running across a couple of teams Overall we've been pretty happy with how things are working out. ## Next Steps - We're still figuring things out, and will be continuing to experiment with more complicated algorithm interventions such as an offline-online algorithms (i.e algorithms that need pre-processing early, and then run-time processing in real time) I hope this is useful for others who might need to solve the same issues. [You can reach me here](https://forms.gle/jQSx5KoXvrVA1JCK7) if you want to ask or discuss anything at all. 💡 I would love some help with naming this pattern. If you have ideas I'd love to hear them via comments here or by [telling me in this form.](https://forms.gle/NsNpiAL95fgim8Ru8) Some names I'm thinking: **Algorithm Handshake Pattern,** **Algorithm-As-A-Service,** **Algorithm Gateway Pattern,** **Algorithm Conduit Pattern,** **Algorithm Bridging Pattern,** **Algorithm Facade Pattern** **(I'm open to hearing others!)** --- # Wanted - URL: https://robotpaper.ai/wanted/ - Markdown: https://robotpaper.ai/wanted.md - Published: 2025-09-09T05:57:16.000+00:00 - Updated: 2025-09-09T05:57:16.000+00:00 - Tags: Developer-Experience - Author: Roy Osherove > Wanted: the AI-augmented developer. AI-native engineer? Context engineer? Code orchestrator? A tongue-in-cheek list of every name for the new kind of developer. Wanted: AI-Augmented Developer AI-Assisted Developer AI-Driven Developer AI-First Developer Generative Engineer AI-Centric Developer Meta-Programmer Neo-Coder Code-Orchestrator AI Native Engineer Intent Engineer Context Engineer AI-Powered Engineer AI-Enhanced Developer AI-Oriented Engineer AI-Integrated Developer AI-Focused Engineer AI-Supported Developer AI-Guided Engineer AI-Optimized Developer AI-Infused Engineer AI-Supervised Developer AI-Collaborative Engineer AI-Symbiotic Developer AI-Supplemented Engineer AI-Partnered Developer AI-Curated Engineer AI-Smart Developer AI-CoCreated Engineer AI-Led Developer AI-Directed Engineer AI-Mediated Developer AI-Supercharged Engineer AI-Embedded Developer AI-Fused Engineer AI-Inspired Developer AI-Enabled Engineer AI-Extended Developer AI-Boosted Engineer AI-Advanced Developer AI-Augmentative Engineer AI-Interactive Developer AI-Iterative Engineer AI-Refined Developer AI-Crafted Engineer AI-Generated Developer AI-Adaptive Engineer AI-Supported Engineer AI-Integrated Engineer AI-Empowered Developer AI-Savvy Engineer AI-Forward Developer AI-Proactive Engineer AI-Architected Developer AI-Design-Driven Engineer AI-Elevated Developer AI-Accelerated Engineer AI-Hybrid Developer AI-Refactored Engineer AI-Enhanced Coder AI-Centric Engineer AI-Specialized Developer AI-Orchestrated Engineer AI-Steered Developer AI-Curated Developer AI-Directed Developer AI-Partner Engineer AI-Expanded Developer AI-Crafted Developer AI-Facilitated Engineer AI-Managed Developer AI-Balanced Engineer AI-Iterative Developer AI-Simplified Engineer AI-Native Developer AI-Infused Developer AI-Forged Engineer AI-Mastered Developer AI-Assisted Engineer AI-Driven Engineer AI-Collaborating Developer AI-Connected Engineer AI-Embedded Engineer AI-Optimized Engineer AI-CoEngineered Developer AI-Supported Coder AI-Augmented Engineer AI-CoCreated Developer AI-Elevated Engineer AI-Oriented Developer AI-Hybrid Engineer AI-Generated Developer AI-Empowered Engineer AI-Enhanced Engineer AI-Led Engineer AI-Forward Engineer AI-Boosted Engineer AI-Adaptive Developer AI-Expanded Engineer AI-Symbiotic Engineer AI-Facilitated Developer AI-Forged Developer AI-Managed Engineer AI-Mastered Engineer AI-Crafted Engineer AI-Accelerated Developer AI-Savvy Developer