I watched a talk by Matt Pocock about writing great skills for AI agents. He could not make it to the AI Engineer World's Fair in person, so he recorded the talk and shared it online. It is called The Missing Manual: How to Write Great Skills. Source: The Missing Manual: How to Write Great Skills on YouTube He starts with a problem he calls skill hell. There are many skills available now. You can download them, write your own, or copy from community repos. But it is hard to tell a good skill from a bad one. People try to stitch too many skills together and do not get the results the skills promise. His checklist has four parts: the trigger, the structure, the steering, and the pruning. 1. The trigger A skill can be invoked in two ways. User invoked: the user tells the agent to use it. Model invoked: the agent decides on its own to use it, based on the skill's description. Model invoked skills sound better because the agent can use them automatically. But every model invoked skill adds load to the agent's context. If you have a hundred of them, the agent has to...
I saw a thread by Mitchell Hashimoto, co-founder of HashiCorp. He says there are entire companies right now under what he calls AI psychosis. He cannot name them because they include personal friends he deeply respects. He is worried about how it plays out. Source: Mitchell Hashimoto on X/Twitter · Screenshot of the thread below. The old lesson coming back Hashimoto lived through the MTBF vs MTTR reckoning in infrastructure during the move to cloud and cloud automation. That argument is spreading beyond ops teams into software development more generally. MTBF means Mean Time Between Failures. It measures how rarely things break. MTTR means Mean Time To Recovery. It measures how fast you recover when things break. Both matter. The problem is when people decide MTTR is all you need. The "MTTR is all you need" mindset Hashimoto describes the psychosis as the belief that fast recovery is enough. The line he keeps hearing is something like this: "It's fine to ship bugs because the agents will fix them so quickly, at a scale humans can't do." Infrastructure already learned that MTTR is great, but you cannot yeet resilient systems entirely. Fast recovery does not remove the cost of breaking...
I worked at two companies recently. Both are decent size. Both have real engineering teams that ship to production every week. They take very different approaches to one specific question: how does an engineer get access to the cloud to do their job? I am not going to name them. For this post, it is Company A and Company B. The point is the approaches, not the brands. I also want to make clear that I am not here to say one is right and one is wrong. Both teams I worked with shipped good software. Both teams had engineers who cared. The different access model changes how the work feels, and what kind of mistakes are easy to make. That is the part I want to write about. The two setups Company A: team owns the account, GitOps decides who has access At Company A, each team has their own AWS accounts. Both the production one and the staging one. The accounts are genuinely owned by the team. Production is a bit more careful: nobody has write access by default. If I need to make a change, I have to ask a teammate to escalate me from read-only to...
I use AI to help me write. I also use it to help me code. I wrote about that last week. This post is the other side. The part I do not say as much about. The LLM often sounds confident. Whether the answer is right, half right, or completely made up, it usually comes out the same. Same tone, same formatting, same "here you go" energy. That is the trap. Not that the LLM is bad. The trap is that it does not know when it does not know, and neither do you, until something breaks. Two kinds of "I do not know" There is a difference between these two situations: I know the topic. I use the LLM to draft, to format, to skip the typing. I am the source of truth. The LLM is a faster pen. I do not know the topic. I use the LLM to learn, to summarise, to answer. The LLM is my only source of truth. The first case is what I wrote about in the "I use AI to write" post. The risk is small there. If the draft is wrong, I can tell, because I already know the answer. The...
← All tags
