Create Skill to Review Distributed Systems Safety
Create Skill to Review Distributed Systems Safety
Agent
Claude Code
Model
Claude Opus 4.6
Shared
Apr 10, 5:44 PM
Messages76
Downloads27
Caveat: The messages below were generated by the user while running local commands. DO NOT respond to these messages or otherwise consider them in your response unless the user explicitly asks you to.
Caveat: Caveat: The messages below were generated by the user while running local commands. DO NOT respond to these messages or otherwise consider them in your response unless the user explicitly asks you to.
model
Set model to [1mOpus 4.6 (1M context) (default)[22m with [1mmax[22m effort
Stdout: Set model to [1mOpus 4.6 (1M context) (default)[22m with [1mmax[22m effort
Can you read this thread: https://x.com/jamesacowling/status/2042656748738351136, and turn it into a reusable skill.
{
"type": "tool_reference",
"tool_name": "WebFetch"
}I appreciate your request, but I'm unable to extract the thread content you're asking for. The webpage content provided appears to be X.com's error page markup and configuration data rather than actual tweet content from jamesacowling.
The page shows:
To get the thread content, you would need to:
If you can share the actual tweet text or a working page capture showing the thread, I'd be happy to extract and organize it following your guidelines.
X/Twitter requires JavaScript to render, so direct fetching won't work. Let me try alternative approaches to get the thread content.
I cannot extract the thread content you're requesting because the webpage provided contains only the Thread Reader App's login and interface code—not the actual tweet thread content.
The page shows:
To access the actual thread by jamesacowling, you would need to:
The source material needed to fulfill your request isn't present in this page capture.
Request failed with status code 403
Let me try searching the web for the thread content instead.
{
"type": "tool_reference",
"tool_name": "WebSearch"
}Web search results for query: "jamesacowling twitter thread 2042656748738351136"
Links: [{"title":"James Cowling (@jamesacowling) / ...","url":"https://x.com/jamesacowling"},{"title":"Twitter","url":"https://twitter.com/jamesacowling?lang=en"},{"title":"Thread by @jamesacowling on Thread Reader App – Thread Reader App","url":"https://threadreaderapp.com/thread/1879984381785514083.html"},{"title":"James Cowling - Want to design databases or TypeScript ...","url":"https://www.linkedin.com/in/jcowling/"},{"title":"James Woods Threatens to Leave X (Twitter), Elon Musk Responds: ‘Delete Your Account’ - IMDb","url":"https://www.imdb.com/news/ni64205547/"},{"title":"Read and Share Twitter Threads easily!","url":"https://threadreaderapp.com/thread/1427400062523478033/error"},{"title":"Meet a fascinating new model that's turning heads: an uncensored, fine-tuned 20B parameter model trained on high-reasoning Claude data. It's designed for complex tasks without the usual guardrails, making it a powerful tool for developers who need raw reasoning power. https://t.co/vmteIcvJHN by @HuggingModels(Hugging Models) | Twitter Thread Reader","url":"https://twitter-thread.com/t/2038398319417082125"},{"title":"Twitter Adds 'Continue Thread' Option for Linking a New Tweet to an Existing Tweet | MacRumors Forums","url":"https://forums.macrumors.com/threads/twitter-adds-continue-thread-option-for-linking-a-new-tweet-to-an-existing-tweet.2224088/"},{"title":"James (@CaucasianJames) / Posts / X","url":"https://x.com/CaucasianJames"},{"title":"LeBron James (@KingJames) / Posts / X","url":"https://x.com/KingJames?lang=en"}]
Based on the search results, I found information about a Twitter thread by @jamesacowling. The thread is titled "How to be a Principal Engineer/Senior Principal Engineer/Senior Architect/fancy-sounding-title Engineer".
The thread covers several key points about the role and responsibilities of senior engineers:
You're evaluated on how much more the company succeeds because you're there, not the lines of code you wrote.
Your job is the strategy stuff and the dirty work stuff. All the cool stuff in the middle is for everyone else. You're not too senior to carry a pager or respond to outages, this keeps you in touch with how things are going.
If you get too caught up in the day-to-day (the how) to step back and think about whether you're going in the right direction (the why and the what) the team will eventually go adrift and people will lose motivation.
Engineers are often bad at strategy because it involves unknowns and engineers like to be confident and exact. Getting good at long term strategy and dealing with unknowns is the #1 skill to develop.
The full thread is available on Thread Reader App for more details.
REMINDER: You MUST include the sources above in your response to the user using markdown hyperlinks.
Web search results for query: "jamesacowling "claude code" thread tips skill"
Links: [{"title":"Boris Cherny’s Claude Code Tips Are Now a Skill. Here Is What the Complete Collection Reveals. | by Reza Rezvani | Mar, 2026 | Medium","url":"https://alirezarezvani.medium.com/boris-chernys-claude-code-tips-are-now-a-skill-here-is-what-the-complete-collection-reveals-b410a942636b"},{"title":"GitHub - ykdojo/claude-code-tips: 45 tips for getting the most out of Claude Code, from basics to advanced - includes a custom status line script, cutting the system prompt in half, using Gemini CLI as Claude Code's minion, and Claude Code running itself in a container. Also includes the dx plugin. · GitHub","url":"https://github.com/ykdojo/claude-code-tips"},{"title":"50 Claude Code Tips and Best Practices For Daily Use","url":"https://www.builder.io/blog/claude-code-tips-best-practices"},{"title":"Best Practices for Claude Code - Claude Code Docs","url":"https://code.claude.com/docs/en/best-practices"},{"title":"The Complete Guide to Creating and Using Claude Skills 2026","url":"https://aifordevelopers.substack.com/p/the-complete-guide-to-creating-and"},{"title":"Claude Skills: Build repeatable workflows in Claude","url":"https://zapier.com/blog/claude-skills/"},{"title":"Lessons from Building Claude Code: How We Use Skills by Thariq","url":"https://www.threads.com/@sung.kim.mw/post/DV_2IojkmM7/lessons-from-building-claude-code-how-we-use-skills-by-thariq"},{"title":"Skill authoring best practices - Claude API Docs","url":"https://platform.claude.com/docs/en/agents-and-tools/agent-skills/best-practices"},{"title":"The Ultimate Guide to Claude Code Skills","url":"https://corpwaters.substack.com/p/the-ultimate-guide-to-claude-code"},{"title":"I'm Boris and I created Claude Code. I wanted to quickly share a few tips for using Claude Code, sourced directly from the Claude Code team. The way the team uses Claude is different than how I use it. Remember: there is no one right way to use Claude Code -- everyones' setup is different. You should experiment to see what works for you!","url":"https://www.threads.com/@boris_cherny/post/DUMZr4VElyb/im-boris-and-i-created-claude-code-i-wanted-to-quickly-share-a-few-tips-for"}]
Based on the search results, I found information about Claude Code tips and skills, but I didn't find specific results from a user named "jamesacowling" or their particular thread. However, the search returned several relevant resources about Claude Code tips and skills:
The most relevant result appears to be a thread by Boris Cherny, who created Claude Code, sharing tips sourced directly from the Claude Code team. This includes practical tips such as:
Spinning up 3–5 git worktrees at once, each running its own Claude session in parallel, which is described as the single biggest productivity unlock from the team
If you do something more than once a day, turn it into a skill or command, and build a /techdebt slash command to find and kill duplicated code
Enabling the Slack MCP, then pasting a Slack bug thread into Claude and saying "fix" with zero context switching required
I also found a Medium article mentioning that tips were loaded as a Claude Code skill — 42 of them across five threads from January through the end of February 2026, but this appears to reference Boris Cherny's tips rather than jamesacowling's specific thread.
If you're looking for a specific thread or repository by jamesacowling, you might want to check their GitHub profile, Twitter/Threads account, or Substack directly, as those specific posts may not be indexed in general web search results.
REMINDER: You MUST include the sources above in your response to the user using markdown hyperlinks.
The first search found an older thread about principal engineering, not matching that tweet ID. Let me search more specifically.
Web search results for query: "site:x.com jamesacowling 2042656748738351136"
Links: [{"title":"James Cowling (@jamesacowling) / X","url":"https://x.com/jamesacowling"},{"title":"James Cowling on X: "I designed Dropbox's storage system and modeled its durability. Durability numbers (11 9's etc) are meaningless because competent providers don't lose data because of disk failures, they lose data because of bugs and operator error. Yes S3 has lost data. No it wasn't because" / X","url":"https://x.com/jamesacowling/status/1922428807136608380"},{"title":"x.com","url":"https://x.com/jamesacowling/status/1767630727821090844"},{"title":"James Cowling on X","url":"https://x.com/jamesacowling/status/1882160361467695345"},{"title":"James Cowling on X: "We'll probably open source Convex Chef but in the meantime published some lessons on what makes an AI codegen tool produce good output. Very unsurprisingly these benefit a lot from programmatically limiting bad states (good APIs, type safety). link below" / X","url":"https://x.com/jamesacowling/status/1950998167140573227"},{"title":"James Cowling on X: "Building a new backend abstraction is hard. It takes a long time to build and a long time for people to trust you. Now we have a good product, great traction, a million things to build, money to do it, and all we need is you. If you deeply care about architecture," / X","url":"https://x.com/jamesacowling/status/1988659032287277387"},{"title":"James Cowling on X: "Arbitrary project milestones are usually a sign of bad engineering leadership. The real key to dealing with uncertainty is building concretely useful stepping stones: https://t.co/tehLA7AwSl" / X","url":"https://x.com/jamesacowling/status/1958965759889350998"},{"title":"James Cowling on X: "Engineers often struggle to develop for customers unlike themselves. It's why we have great dev tools but enterprise software mostly sucks. We're getting a lot of demand from customers much bigger than us at Convex and are opening up a role for product engineers as we move" / X","url":"https://x.com/jamesacowling/status/2001813461333086330"},{"title":"James Cowling on X: "Convex has an RLS library, but unless you're using a platform that directly exposes the database to clients you probably don't need row-level security. It's nice idea but with large codebases it gets very hard to map auth domains cleanly onto rows in dozens of tables. If you're" / X","url":"https://x.com/jamesacowling/status/2001081302574100718"},{"title":"James Cowling on X: "Time for a periodic @convex roadmap update! We’ve been consolidating recently on a smaller set of features that we want to execute on fast and get out very soon. I’ll break these down into three broad categories. 1. Be more open, be more helpful Convex is a new way of" / X","url":"https://x.com/jamesacowling/status/1887346879043936607"}]
The search results show several posts from James Cowling's X (formerly Twitter) account (@jamesacowling), but don't appear to include a specific post with the ID 2042656748738351136. The results display various tweets from his timeline covering topics like Dropbox storage design, data durability, Convex product updates, and engineering leadership perspectives.
The search returned his profile information and several of his recent posts, but the particular post ID you're looking for wasn't among the results returned. This could mean the post has been deleted, is not publicly accessible, or the search didn't index it in this particular query.
REMINDER: You MUST include the sources above in your response to the user using markdown hyperlinks.
Web search results for query: "jamesacowling convex claude code skill april 2026"
Links: [{"title":"Wayne Sutton on X: "Built 25+ apps with @convex, Claude, and Cursor. I package the skills and rules I use on every project. Unofficial Convex skills for Claude Code. Cursor rules included. https://t.co/XRxkPhBAhQ To save some time, you can also install via npm. Keep shipping." / X","url":"https://x.com/waynesutton/status/2011932036316217508"},{"title":"Claude Code Skill: Convex Actions & General Development","url":"https://mcpmarket.com/tools/skills/convex-actions-and-general-development"},{"title":"GitHub - alirezarezvani/claude-skills: 220+ Claude Code skills & agent plugins for Claude Code, Codex, Gemini CLI, Cursor, and 8 more coding agents — engineering, marketing, product, compliance, C-level advisory.","url":"https://github.com/alirezarezvani/claude-skills"},{"title":"GitHub - waynesutton/convexskills: AI agent skills and templates for building production ready apps with Convex. Patterns for queries, mutations, cron jobs, webhooks, migrations, and more. · GitHub","url":"https://github.com/waynesutton/convexskills"},{"title":"Top 10 Claude Code Skills Every Builder Should Know in 2026 | Composio","url":"https://composio.dev/content/top-claude-skills"},{"title":"Extend Claude with skills - Claude Code Docs","url":"https://code.claude.com/docs/en/skills"},{"title":"10 Best Skills for Claude Code in 2026 (Tested & Reviewed)","url":"https://www.agensi.io/learn/best-claude-code-skills-2026"},{"title":"Wayne Sutton on X: "This might be the most important SKILL ever created. Avoid Feature Creep for Agents https://t.co/kmz6FwcOpi Now available. https://t.co/jDgpb560F1" / X","url":"https://x.com/waynesutton/status/2012646729590395205"},{"title":"GitHub - travisvn/awesome-claude-skills: A curated list of awesome Claude Skills, resources, and tools for customizing Claude AI workflows — particularly Claude Code","url":"https://github.com/travisvn/awesome-claude-skills"},{"title":"GitHub - Sstobo/convex-skills","url":"https://github.com/Sstobo/convex-skills"}]
The search results don't contain any information about "jamesacowling" specifically. The results do show information about Convex Claude Code skills from various developers (particularly Wayne Sutton and others who have created skill collections), but there is no mention of a user or developer named "jamesacowling" in relation to Convex or Claude Code skills in April 2026.
If you're looking for information about a specific skill, repository, or contribution by that user, could you provide more context about what you're searching for?
REMINDER: You MUST include the sources above in your response to the user using markdown hyperlinks.
Web search results for query: ""jamesacowling" thread april 2026 tips"
Links: [{"title":"Daily Open Thread: April 9, 2026 | Cageside Seats","url":"https://www.cagesideseats.com/cageside-seats-features/431332/daily-open-thread-april-9-2026"},{"title":"JAMB Syllabus for All Subjects 2026/2027: Meticulous review","url":"https://www.withinnigeria.com/2026/04/10/jamb-syllabus-for-all-subjects-2026-2027-meticulous-review/"},{"title":"JAMB Mock Exam Results 2026 - Monitoring Thread - Osilight","url":"https://osilight.com/jamb-mock-exam-results-2026-monitoring-thread/"},{"title":"What’s happening with Threads in 2026? Here are all the biggest Threads updates from this year [updated monthly] | HeyOrca","url":"https://www.heyorca.com/blog/threads-social-news"},{"title":"Daily Open Thread: April 10, 2026 | Cageside Seats","url":"https://www.cagesideseats.com/cageside-seats-features/431334/daily-open-thread-april-10-2026"},{"title":"JAMB UTME Exam Slip 2026 Printing - Monitoring Thread - Myschool","url":"https://myschool.ng/news/jamb-2026-utme-slip-printing-monitoring-thread"},{"title":"JAMB Slip Reprinting 2026: Official Date, Portal Link, and Step-by-Step Guide - Campus Ninja","url":"https://campusninja.com.ng/jamb-slip-reprinting-2026/"},{"title":"Daily Open Thread: April 7, 2026 | Cageside Seats","url":"https://www.cagesideseats.com/cageside-seats-features/431328/daily-open-thread-april-7-2026"},{"title":"Daily Open Thread: April 6, 2026 | Cageside Seats","url":"https://www.cagesideseats.com/cageside-seats-features/430730/daily-open-thread-april-6-2026"},{"title":"Tuesday Overnight Open Thread - April 7, 2026 [Doof]","url":"https://acecomments.mu.nu/?post=419241"}]
The search results don't contain any specific information about "jamesacowling" or a thread by that user from April 2026 with tips. The results show various unrelated content including JAMB exam information, Threads platform updates, and wrestling-related daily open threads, but nothing matching your specific query.
Could you provide more context about what you're looking for? For example:
This would help me conduct a more targeted search.
REMINDER: You MUST include the sources above in your response to the user using markdown hyperlinks.
Based on the provided data, I can only access the first tweet (1/5) of this thread. The JSON contains:
Tweet 1/5:
"Decouple data and code changes. Never push out a release that changes how data is stored at the same time as the code that uses this new data." The author explains that simultaneous changes risk irrecoverable failures if rollbacks become necessary. Instead, deploy data changes first (maintaining compatibility with old code), then deploy new code, then remove backwards compatibility constraints.
Limitation: The remaining four tweets (2-5) are not included in this dataset. To access the complete thread, you would need to fetch the additional tweet IDs or retrieve the full thread from the original X/Twitter post.
I found the thread via fxtwitter. Let me get the full content.
Tweet Text:
"Time for a big systems advice thread! In distributed systems there's no magic "push everything to prod at once" button. Every service gets pushed independently and nodes within a service get updated incrementally. If you mess up forwards/backwards compatibility you can fail irrecoverably. So how to avoid this? 1/5: Decouple data and code changes. Never push out a release that changes how data is stored at the same time as the code that uses this new data. If there's a bug and you need to roll back to the old version of your code it won't be able to handle the new data in the new format. Instead push out a release that first changes the data in a way that's compatible with both the old and new code (e.g., optional fields etc), when that's stable push out the new code that uses it, then when that's stable you can change the data to remove backwards compatibility. This is known as a "migration" in the database world and yes it's annoying, but yes you need to do it."
Author: James Cowling (@jamesacowling)
URL: https://x.com/jamesacowling/status/2042656748738351136
Created: Fri Apr 10 17:31:34 +0000 2026
Note: This appears to be the first part of a thread ("1/5"). The JSON provided contains only this portion; no reply threads or subsequent tweets are included in the source data.
Got tweet 1/5. Now let me find the rest of the thread.
Request failed with status code 404
Request failed with status code 404
Request failed with status code 404
Request failed with status code 404
I've reviewed the provided JSON content, but I cannot find a tweet with ID 2042656748738351136 or a thread starting with that ID in the data provided.
The tweets shown are from James Cowling (@jamesacowling) and include discussions on database sharding, principal engineers, storage durability, domain names, ORMs, distributed systems, and other technical topics. However, none match the specific tweet ID you're requesting.
To locate the thread you need, could you verify:
Web search results for query: "jamesacowling "distributed systems" thread "decouple data" "2/5" OR "3/5" OR "4/5" OR "5/5""
No links found.
The search returned no results. This could mean:
If you're looking for a specific thread or discussion by jamesacowling about distributed systems and decoupling data, you might try:
Would you like me to try a different search with modified terms?
REMINDER: You MUST include the sources above in your response to the user using markdown hyperlinks.
Web search results for query: "jamesacowling "big systems advice thread" distributed systems compatibility migration"
Links: [{"title":"Operating System Process and Thread Migration in Heterogeneous Platforms","url":"https://www.cs.utexas.edu/~mars2016/workshop-program/MaRS_2016_paper_2.pdf"},{"title":"Thread migration and its applications in distributed shared memory systems - ScienceDirect","url":"https://www.sciencedirect.com/science/article/abs/pii/S0164121298000089"},{"title":"(PDF) Process/Thread Migration and Checkpointing in Heterogeneous Distributed Systems","url":"https://www.academia.edu/9856320/Process_Thread_Migration_and_Checkpointing_in_Heterogeneous_Distributed_Systems"},{"title":"Understanding Distributed Systems","url":"https://understandingdistributed.systems/"},{"title":"Distributed Systems Processes, Threads, Code Migration","url":"https://people.eecs.berkeley.edu/~istoica/classes/cs194/05/notes/4-ProcThread.pdf"},{"title":"Predicting the Future of Distributed Systems","url":"https://blog.colinbreck.com/predicting-the-future-of-distributed-systems/"},{"title":"Chapter 3. Processes","url":"https://csis.pace.edu/~marchese/CS865/Lectures/Chap3/Chapter3.htm"},{"title":"Ensuring backwards compatibility in distributed systems - Stack Overflow","url":"https://stackoverflow.blog/2020/05/13/ensuring-backwards-compatibility-in-distributed-systems/"},{"title":"Mainframe to Distributed SQL, Part 5: Transitioning to a Distributed Architecture","url":"https://www.cockroachlabs.com/blog/transitioning-to-distributed-architecture/"},{"title":"How to build a large scale distributed system from scratch - Quora","url":"https://www.quora.com/How-do-you-build-a-large-scale-distributed-system-from-scratch"}]
The search results don't appear to include the specific "big systems advice thread" by jamesacowling that you're looking for. The results show academic papers and articles about distributed systems, thread migration, and database migration strategies, but nothing matching that specific source.
Let me try a more targeted search for this content:
Links: [{"title":"James Cowling (@jamesacowling) / Posts / X","url":"https://x.com/jamesacowling?lang=en"},{"title":"James Cowling (@jamesacowling) ...","url":"https://twitter.com/jamesacowling?lang=en"},{"title":"JAMES16 THREAD PDF","url":"https://crip-power.com/james16-thread-36/"},{"title":"Jamo Tech Systems Administration and Machines | Page 2 | BuiltByBit (MC-Market)","url":"https://builtbybit.com/threads/jamo-tech-systems-administration-and-machines.636202/page-2"},{"title":"Thread: Advice wanted - hammock for big and tall guy","url":"https://hammockforums.net/forum/showthread.php/115528-Advice-wanted-hammock-for-big-and-tall-guy"},{"title":"Another Newbie Looking For Advice / Thread","url":"https://modulargrid.net/e/forum/posts/index/8282"},{"title":"James16 or Strategic Trading Systems? Please help | Forex Factory","url":"https://www.forexfactory.com/thread/299480-james16-or-strategic-trading-systems-please-help"},{"title":"Advice on my EBM performance rack / Thread","url":"https://modulargrid.net/e/forum/posts/index/8168"},{"title":"Thread by @jamesacowling on Thread Reader App – Thread Reader App","url":"https://threadreaderapp.com/thread/1879984381785514083.html"},{"title":"The Technical Blog of James | Technical articles, writeups and discussion!","url":"https://ttboj.wordpress.com/"}]
Great! I found the thread. Let me search for more specific content about the "big systems advice thread":
No links found.
Based on my search results, I found information about James Cowling and his work on distributed systems. James Cowling is the CTO of Convex and a Dropbox storage designer.
I located a thread from James Cowling about "How to be a Principal Engineer/Senior Principal Engineer/Senior Architect", but I wasn't able to find a specific thread explicitly titled "big systems advice thread" focused on distributed systems compatibility and migration in the search results.
However, the search did reveal relevant content from James Cowling on systems topics. He has shared insights on distributed systems, including that "Dropbox got to several hundred million users before needing to shard the User db... and then went to two shards".
If you're looking for that specific thread about distributed systems compatibility and migration advice, you may want to check James Cowling's Twitter/X account (@jamesacowling) or his published writing directly, as the search results show he has shared technical guidance on these topics but the exact thread you're referencing didn't appear in these results.
REMINDER: You MUST include the sources above in your response to the user using markdown hyperlinks.
The user doesn't want to proceed with this tool use. The tool use was rejected (eg. if it was a file edit, the new_string was NOT written to the file). STOP what you are doing and wait for the user to tell you how to proceed.
Here's the text: Time for a big systems advice thread!
In distributed systems there's no magic "push everything to prod at once" button. Every service gets pushed independently and nodes within a service get updated incrementally. If you mess up forwards/backwards compatibility you can fail irrecoverably.
So how to avoid this?
1/5: Decouple data and code changes. Never push out a release that changes how data is stored at the same time as the code that uses this new data. If there's a bug and you need to roll back to the old version of your code it won't be able to handle the new data in the new format. Instead push out a release that first changes the data in a way that’s compatible with both the old and new code (e.g., optional fields etc), when that’s stable push out the new code that uses it, then when that’s stable you can change the data to remove backwards compatibility. This is known as a “migration” in the database world and yes it’s annoying, but yes you need to do it.
1:31 PM · Apr 10, 2026
·
173
Views
Relevant
James Cowling
@jamesacowling
·
12m
2/5: Don’t change two services at once. If service A talks to service B, you can’t just add a new API to both of them and push them out. What if someone pushes A but not B? What if there’s a bug in B that needs to be rolled back? Just like with data changes, API changes need to be made in a forwards and backwards compatible way. Engineers forget to do this all the time.
James Cowling
@jamesacowling
·
12m
3/5: Only allow one version “step” to exist in prod at any time. It’s common to have most of your nodes at version 5 but a few are still at version 4 because they haven’t finished migrating. Never ever allow someone to push to version 6 while version 4 is still running. Otherwise it’s too hard for engineers to reason about which version is “stable” when making multi-step migrations. You need monitoring and alerting for this plus protections agains corner cases, e.g., if a node was offline and then came back online after missing a code push.
James Cowling
@jamesacowling
·
12m
4/5: Codify forwards/backwards compatibility in your release process by pushing some nodes to the new version, run them for a while and keep track of monitoring, then roll them back before doing the full release again. If someone messed up a migration you want to smoke this out on some staging nodes or at small scale while an operator is watching not once your entire system is down.
James Cowling
@jamesacowling
·
12m
5/5: Design cleanly composed systems with simple APIs, thin clients, type safety, information hiding, and well-articulated guarantees. Avoid anything fancy or ambiguous in your APIs. This is the only way you can feasibly maintain a large distributed system with multiple versions of every service running at once. It’s also an area where skilled humans significantly outperform LLMs, like it’s not even close. Don’t turn off your brain just yet.
James Cowling
@jamesacowling
·
12m
Follow-up note to
@convex
customers: you don't have to think about most of these things, e.g., we force you to finish a schema migration before pushing the code that uses it, we handle version skew internally for you, etc, but you still do have to think about what happens if an old client shows up and calls a function that doesn't exist anymore. This is rarely a problem on the web but is an issue for mobile apps. We'd like to make this problem go away too but for now if you're a mobile developer you probably already know you should be careful about deprecating APIs, since client code can live for a long time.
Thanks for pasting the thread. Let me check how skills are structured in your setup, then create one.
ls -la /Users/tarunsachdeva/.claude/skills/ 2>/dev/null; echo "---"; ls -la /Users/tarunsachdeva/.claude/commands/ 2>/dev/nulltotal 0 drwxr-xr-x@ 3 tarunsachdeva staff 96 Feb 22 17:04 . drwxr-xr-x@ 23 tarunsachdeva staff 736 Apr 10 13:44 .. drwxr-xr-x@ 3 tarunsachdeva staff 96 Feb 22 17:04 share-to-traces --- total 8 drwxr-xr-x@ 3 tarunsachdeva staff 96 Feb 22 17:04 . drwxr-xr-x@ 23 tarunsachdeva staff 736 Apr 10 13:44 .. -rw-r--r--@ 1 tarunsachdeva staff 679 Feb 22 17:04 share-to-traces.md
Now I understand the structure. I'll create a skill that reviews code changes against the 5 distributed systems safety principles from that thread.
mkdir -p /Users/tarunsachdeva/.claude/skills/distributed-systems-review(Bash completed with no output)
Now create the slash command so it can be invoked with /dist-review.
The skill is live. Here's what was created:
Skill: /Users/tarunsachdeva/.claude/skills/distributed-systems-review/skill.md
Command: /Users/tarunsachdeva/.claude/commands/dist-review.md
You can invoke it with /dist-review (optionally passing file paths or a PR number). It reviews code changes against James Cowling's 5 principles:
Each principle gets a PASS / WARN / FAIL rating with an explanation. It defaults to reviewing your current git diff if no arguments are given.
Here's the text:
Review code changes against five critical distributed systems safety principles.
Based on James Cowling's (@jamesacowling) systems advice thread.
In distributed systems there's no "push everything to prod at once" button. Every
service gets pushed independently and nodes within a service get updated
incrementally. If you mess up forwards/backwards compatibility you can fail
irrecoverably.
Analyze the staged changes (or specified files/PR) and evaluate against each of
these five principles. For each principle, output one of:
Never push a release that changes how data is stored at the same time as the code
that uses the new data. If there's a bug and you need to roll back the old code,
it won't be able to handle data in the new format.
Safe pattern (3-step migration):
Red flags to look for:
If service A talks to service B, you can't add a new API to both and push them
simultaneously. What if A deploys but B doesn't? What if B has a bug and needs
rollback?
Red flags to look for:
Never allow version N+2 to be pushed while version N is still running. It's too
hard for engineers to reason about which version is "stable" when making multi-step
migrations.
Red flags to look for:
Push some nodes to the new version, run them, keep track of monitoring, then roll
them back before doing the full release. Smoke out migration bugs at small scale
while an operator is watching, not once the entire system is down.
Red flags to look for:
Use simple APIs, thin clients, type safety, information hiding, and
well-articulated guarantees. Avoid anything fancy or ambiguous in APIs. This is the
only way to feasibly maintain a large distributed system with multiple versions of
every service running at once.
Red flags to look for:
Structure your review as:
## Distributed Systems Safety Review ### 1. Data/Code Decoupling: [PASS|WARN|FAIL] [explanation] ### 2. Single-Service Changes: [PASS|WARN|FAIL] [explanation] ### 3. One Version Step: [PASS|WARN|FAIL] [explanation] ### 4. Release Process Safety: [PASS|WARN|FAIL] [explanation] ### 5. API Cleanliness: [PASS|WARN|FAIL] [explanation] ### Summary [overall assessment and recommended actions]
git diff --cached),git diff).$ARGUMENTS
Want to make any final edits? I'm going to publish this conversation online.
Looks good to me — no edits needed. It's ready to use as /dist-review.
Caveat: The messages below were generated by the user while running local commands. DO NOT respond to these messages or otherwise consider them in your response unless the user explicitly asks you to.
Caveat: Caveat: The messages below were generated by the user while running local commands. DO NOT respond to these messages or otherwise consider them in your response unless the user explicitly asks you to.
exit
See ya!
Stdout: See ya!