The show returns after a month-long break to announce the launch of VarOps (varops.com) and introduces its premise via a short documentary about enterprises losing critical, undocumented decision knowledge when key employees are unavailable.

Ran argues organizations adopt AI incorrectly by dumping documents into RAG systems, relying on prompts for security, and trying to mold long-established processes around new tools instead of building AI around the business. He describes VarOps’ approach – “Resident AI” owned by the company – supported by a published paper on a governed operational model and a “large organization model” with version-controlled decisions and deterministic access control. Benchmarks show local models plus frontier reasoning can reach ~96–99% of frontier performance at low cost, motivating model and data sovereignty.

Ran also discusses new frontier releases and Jev, a fast classification-focused model, emphasizing models are not interchangeable and small accuracy differences matter in mission-critical enterprise settings.

00:00 Welcome Back VarOps Launch
01:17 Resident AI Mini Documentary
04:22 Why AI Adoption Fails
09:02 Acropolis Research Findings
11:37 New Model Wave Pricing
15:06 System Architecture Varys
18:40 Local Models Sovereignty
21:19 VarOps Site Bot Magazine
23:27 Jev System One Classifier
29:08 Model Fit Error Rates
30:48 Wrap Up Newsletter Invite

Host Ran Aroussi (Old School / New Tech) shares a recent “war story” from his agency Automaze: a hard-to-reproduce mobile bug where calls were sometimes dropped only on a client’s devices, which the team couldn’t reproduce for 2–3 weeks. After two long sessions using a Droid Factory setup with agents and a “small council” debate between models (e.g., Fable and Sol), he got the team unstuck, found the issue, and then turned the session logs into an internal guide and public article on his methodology. Key practices include distrusting agent conclusions (treat “preexisting/flaky/unrelated” as hypotheses), demanding proof (including his Proof library requiring video evidence), using extensive unit and end-to-end tests, asking for a numeric confidence level before production, controlling environments, handling merge conflicts with full review/testing, switching models for independent review, and re-spec’ing from scratch to detect drift from the implementation plan.

00:00 Show Begins
00:58 The Sneaky Mobile Bug
02:46 Agents Find The Fix
05:23 Never Trust Preexisting
08:29 Proof Over Promises
10:14 Confidence Level Gate
12:47 Model Debate Reviews
17:08 Merge Conflicts Standards
21:23 Spec From Scratch
25:49 Ten Rules Summary
31:30 Token Usage And Wrap Up

-----

Old School / New Tech is produced by VarOps.

• Subscribe to the newsletter – the written version of this show, weekly: varops.com
• Read the books – Company-Scale Agentic AI (for operators and the C-suite) and Production-Grade Agentic AI (for engineers): varops.com/books
• Work with us – VarOps installs resident AI inside mid-market companies. It runs where your work already happens, sees the whole operation, and what it learns belongs to you: varops.com/transform

Why Software Factories Still Need Humans in the Loop

Ran Aroussi discusses “software factories” as automated systems that manage the full software development and deployment cycle, arguing the industry is moving from a single workstation to virtual, always-on cloud environments with agents running off-machine. Prompted by Dex (HumanLayer) describing a fully automated “lights-out” factory that produced “slop,” and Uncle Bob’s claim he doesn’t read agent-written code when surrounded by extreme constraints, Ran explains why he doesn’t believe in lights-out automation. He outlines Automaze’s internal factory, Cloop: tickets/PRDs are created and grounded in an indexed knowledge graph of the codebase; multiple models generate and critique PRDs; a human approves; agents implement in parallel, run unit/e2e/quality tests, simplify and profile code, then perform code and security review, looping up to five times before escalating to a human. Humans remain essential for alignment, enforcement, and final validation via PR review and temporary deployments, avoiding false positives, runaway costs, and untrustworthy results.

-----

Old School / New Tech is produced by VarOps.

• Subscribe to the newsletter – the written version of this show, weekly: varops.com
• Read the books – Company-Scale Agentic AI (for operators and the C-suite) and Production-Grade Agentic AI (for engineers): varops.com/books
• Work with us – VarOps installs resident AI inside mid-market companies. It runs where your work already happens, sees the whole operation, and what it learns belongs to you: varops.com/transform