Writing · Nº 041065 essays
The writing
Ideas from inside the work. Explore technology, entrepreneurship, and the practice of building things that last.
43 essays found · Page 2 of 4
Clear filtersEngineeringIs Web Scraping Legal? What Actually MattersIs web scraping legal? The honest answer for operators: public data, terms of service, personal data, and rate limits matter more than the yes-or-no question.EngineeringWhat Running Your Own Server Actually CostsWhat does running your own server cost? Here are the real numbers, dollars and hours, of self-hosting a VPS versus a managed platform at real scale.EngineeringHow to Keep Scrapers Reliable at ScaleHow to keep scrapers reliable at scale: design for failure, add backpressure, isolate targets, and measure real success so one bad target cannot sink the job.EngineeringSelf Host vs Managed Cloud: How to DecideA grounded decision guide for owning your stack versus renting it: cost, control, lock-in, on-call burden, and when managed services become a liability.EngineeringWhen Managed Hosting Is Actually the Right CallSelf-hosting is not always the answer. Here is when managed hosting is the right call, when it is not, and how to keep the exit cheap either way.EngineeringHow to Evaluate a Web Scraping ProviderA checklist for evaluating a web scraping provider: real success rates, target coverage, cost at scale, control, and what happens when a target fights back.StrategyHow to Meet Data Residency Requirements as an AI VendorEnterprise and regulated buyers demand data residency: their data stays in a region. Here is what data residency means for AI vendors and how to actually deliver it.EngineeringSelf-Hosting Mistakes That Burn People and How to Avoid ThemMost self-hosting horror stories come from a few avoidable mistakes. Here are the self-hosting mistakes that burn people and how to skip every one.EngineeringSelf-Host Your Job Queue vs Renting SQSSelf-host your job queue vs renting SQS: when a Redis or Postgres-backed queue on your own box beats a managed queue, and when the managed one earns its keep.EngineeringBuild vs Buy Real-Time InfrastructureShould you build or buy real-time infrastructure for websockets and presence? Here is the honest cost of each and the point where buying clearly wins.EngineeringHow to Design a Data Pipeline for Scraped DataHow to design a data pipeline for scraped data: separate collection from processing, stage raw HTML, validate before storage, and make reprocessing cheap.StrategyEvery Vendor Needs an Exit Plan Before You Depend on ItA vendor exit plan is the switching cost you can actually pay. Here is how to build one before you depend on a host, and why data portability comes first.