Writing · Nº 041065 essays
The writing
Ideas from inside the work. Explore technology, entrepreneurship, and the practice of building things that last.
24 essays found · Page 2 of 2
Clear filtersStrategyHow to Evaluate a Voting Platform: A ChecklistEvaluate a voting platform on verifiability, auditability, ballot secrecy, and resolution rules, not on feature lists. Here is the checklist I would use.StrategyQuestions to Ask Before Buying an All-in-One PlatformThe questions to ask before buying an all-in-one business platform: data model, migration, exit terms, and depth per function. Ask these before you sign.EngineeringLLM as Judge: When to Trust It for AI EvaluationUsing an LLM as judge to score AI output is fast and scalable, but it is biased and gameable. When to trust LLM-as-judge for evaluation and when to bring in a human.StrategyHow to Choose an AI Agency Operating SystemA practical checklist for choosing an AI agency operating system: what to test, what to ignore, and the questions that separate real platforms from demos.AIHow to Evaluate a Self Assessment ToolA checklist for evaluating a self assessment tool: specificity, honesty, whether it updates, and whether the output ends in action. Most fail at least two.StrategyHow to Evaluate a Modern Marketing AgencyA checklist for evaluating a modern marketing agency: look for a system, accountability to outcomes, and real tooling, not a timesheet with an AI sticker on it.EngineeringOffline Evals vs Production Monitoring for AIOffline evals and production monitoring measure different things for an AI feature. You need both. Here is what each catches, what each misses, and how they work together.AIAI Automation Platform Evaluation ChecklistA practical checklist for evaluating an AI automation platform: orchestration, failure handling, audit trails, connectors, and the exit path before you commit.EngineeringHow to Build a Golden Eval Set for an AI FeatureA golden eval set is the fixed, labeled test cases that tell you if an AI feature got better or worse. Here is how to build one before you tune a single prompt.AIHow to Evaluate an AI Agent Before You Deploy ItA practical checklist to evaluate an AI agent before deployment: reliability, guardrails, audit, tool scope, and fallback behavior, not just the demo.AIHow to Evaluate AI Bookkeeping Software Before You SwitchA practical checklist for evaluating AI bookkeeping software: audit trails, source documents, accuracy proof, and the export test that reveals lock-in.ThesisSeven Signs a Product Is Just AI Bolted OnHow to spot AI bolted on to old software before you buy. Seven concrete signs a product wrapped a model instead of building AI-native around it.