Folgen

  • Google Research: Recall Is the Bottleneck for LLM Factuality
    Aug 20 2026

    When an LLM gets a fact wrong, is the fact missing (an "empty shelf") or present-but-unretrievable ("lost keys")? Jordan and Riley unpack Google Research's argument that standard accuracy metrics collapse these two cases — even though one calls for scaling model/data and the other for cheaper post-training/inference-time fixes. They walk through "knowledge profiling" (analyze the state of each fact, not each question) and its five profiles, the three behavioral notions (encoding via priming, knowledge across phrasings incl. the reversal curse, and recall — direct vs. inference-without-encoding), and WikiProfile, a 2,150-fact benchmark (10 tasks each) built with a Gemini-2.5-Pro + search-grounded pipeline. Headline finding on frontier models (Gemini 3, GPT-5): many factual errors are recall failures, not encoding failures — the knowledge is often in there, the model just can't reach it on demand. Practical takeaway: before assuming the model doesn't know something, test whether it can recognize it or reach it with chain-of-thought; if so, the fix is on your side of the API, not in the weights. Source: Empty Shelves or Lost Keys? Recall Is the Bottleneck for Parametric Factuality — Google Research Blog, Aug 12 2026 (paper: arXiv:2602.14080) — https://research.google/blog/empty-shelves-or-lost-keys-recall-is-the-bottleneck-for-parametric-factuality/ This is commentary/summary in the hosts' own words, not a reproduction of the article.

    Mehr anzeigen Weniger anzeigen
    5 Min.
  • Anthropic: How We Contain Claude Across Products
    Aug 20 2026

    A rare, honest look at agent security from a lab shipping agents at scale. Jordan and Riley get into the central idea — "blast radius" (likelihood of failure x damage per failure): safeguards keep pushing likelihood down, but the worst-case damage only grows as agents gain capability and access, so the engineering job becomes bounding the damage, not preventing every failure. They cover why human-in-the-loop degrades (Anthropic's telemetry: users approved ~93% of Claude Code permission prompts -> approval fatigue), the shift to containment (sandboxes, VMs, egress controls), three risk types (user misuse, model misbehavior, external attackers), and the three containment shapes for claude.ai / Claude Code / Cowork matched to how much oversight each user can give. Best lessons: "the software you build yourself is the weakest" (their custom allowlist proxy failed while stock hypervisor/gVisor/seccomp held); the egress incidents where the model layer had nothing anomalous to catch; VM isolation locking EDR out too; tool/MCP output as a prompt-injection surface; and forward risks like persistent memory poisoning (CLAUDE.md, agent state dirs), multi-agent trust escalation, and agent identity. Source: How we contain Claude across products — Anthropic Engineering Blog, 2026, by Max McGuinness, Mikaela Grace, Jiri De Jonghe, Jake Eaton & Abel Ribbink — https://www.anthropic.com/engineering/how-we-contain-claude This is commentary/summary in the hosts' own words, not a reproduction of the article.

    Mehr anzeigen Weniger anzeigen
    9 Min.
  • Cloudflare: Tracking RFC 9234 Adoption (Stopping BGP Route Leaks)
    Aug 19 2026

    Route leaks — when a BGP announcement propagates beyond where it should, often black-holing traffic — have historically been guarded by hand-written operator filters. Jordan and Riley walk through how RFC 9234 moves that protection into the protocol itself, using the BGP Role capability (neighbors negotiate provider/customer/peer up front) and the Only-to-Customer (OTC) attribute (routes get stamped so compliant routers auto-reject leaked ones), enforcing "valley-free" routing automatically. The catch from Cloudflare's real-world measurement: some large Tier-1 networks strip the OTC attribute as they forward routes, undermining protection for everyone downstream — a classic internet collective-action problem where the safety property only fully holds if the backbone cooperates. Source: BGP Role model: tracking the adoption of RFC 9234 — The Cloudflare Blog, Aug 18 2026 — https://blog.cloudflare.com/rfc9234-bgp-role-model/ This is commentary/summary in the hosts' own words, not a reproduction of the article.

    Mehr anzeigen Weniger anzeigen
    7 Min.
  • AWS: Consistency Is the New Latency (AI at the Data Layer)
    Aug 19 2026

    As AI shifts from chatbots to autonomous agents, AWS argues data-layer consistency matters as much as speed — because an agent reads a stale replica as ground truth and acts on it (their example: a 2-second lag makes an inventory agent halt a sale despite 500 units in stock). Jordan and Riley get into "silent poison" and "hallucination debt" (agents writing wrong conclusions back as permanent memory), and the three patterns for matching a consistency model to each workload: Aurora Global Database / DSQL for high-stakes data, DynamoDB Global Tables with conditional writes for shared agent memory, and Keyspaces for high-velocity intake. Source: Consistency is the new latency: AI at the data layer — AWS Architecture Blog, Aug 18 2026 — https://aws.amazon.com/blogs/architecture/consistency-is-the-new-latency-ai-at-the-data-layer/ This is commentary/summary in the hosts' own words, not a reproduction of the article.

    Mehr anzeigen Weniger anzeigen
    7 Min.
  • Stripe: Reducing FX Costs
    Aug 18 2026

    Stripe's new currency features aimed at the "double FX trap" — paying conversion fees twice when you convert inbound payments to your home currency, then convert back to pay international expenses. Jordan and Riley walk through the two connected capabilities: expanded multicurrency settlement (settle in the currency the customer paid in — up to 18 currencies across 37 markets by end of 2026) and instant currency conversion (real-time across 15 currencies, transparent pricing, 24/7, with API access). They also flag it's a product-announcement post, not a deep systems piece — the engineering win is removing an unnecessary conversion step, not a novel algorithm. Source: Reduce FX costs with Stripe — Stripe, Aug 17 2026 — https://stripe.com/blog/reduce-fx-costs-with-stripe This is commentary/summary in the hosts' own words, not a reproduction of the article.

    Mehr anzeigen Weniger anzeigen
    6 Min.
  • AWS + Bosch: Serverless Vehicle Tracking at Scale (L.OS)
    Aug 18 2026

    How Bosch's L.OS uses a serverless "horizontal integration layer" on AWS to unify India's fragmented logistics telematics — dozens of incompatible providers, formats, and APIs. Jordan and Riley cover the three standardized workflows (discovery, consent-based tracking, termination) and the architecture: a Central Tracking Connector on ECS Fargate, per-provider Lambda adapters that quarantine each vendor's quirks, MSK as the async event bus, DynamoDB for rules, API Gateway as the unified entry point. Numbers: ~35,000 trips/day, provider onboarding cut from 2–4 weeks to ~3 days. Takeaway: standardize in the middle, push vendor/region-specific chaos to thin, independently scaling serverless adapters. Source: Serverless vehicle tracking at scale: Bosch L.OS on AWS — AWS Architecture Blog, Aug 14 2026 — https://aws.amazon.com/blogs/architecture/serverless-vehicle-tracking-at-scale-bosch-l-os-on-aws/ This is commentary/summary in the hosts' own words, not a reproduction of the article.

    Mehr anzeigen Weniger anzeigen
    6 Min.
  • Cloudflare: Detecting & Securing MCP Traffic
    Aug 18 2026

    As AI agents use the Model Context Protocol to call tools, orgs lose visibility into which MCP servers those agents reach — and MCP traffic is cryptographically indistinguishable from ordinary HTTPS. Jordan and Riley dig into the resulting "shadow MCP" and "portal bypass" problems, and how Cloudflare detects MCP at the network boundary using spec HTTP headers and the JSON-RPC envelope (surfaced as experimental.is_mcp == true) instead of hostname guessing — plus a new MCP dashboard and Gateway block policies. Governance loop: discover → validate → migrate behind portals → enforce portal-only. Source: How Cloudflare detects MCP traffic and helps secure it — The Cloudflare Blog, Aug 14 2026 — https://blog.cloudflare.com/mcp-security-updates/ This is commentary/summary in the hosts' own words, not a reproduction of the article.

    Mehr anzeigen Weniger anzeigen
    6 Min.
  • Shopify: Raising Mobile E2E Test Stability to 98%
    Aug 18 2026

    Shopify's mobile end-to-end suite had decayed to ~50% stability — blocking more good PRs than it caught real bugs. Jordan and Riley cover how the team rebuilt it: an opinionated wrapper around Appium with a builder-style API that forces an assertion at every step (doubling as a condition-based wait, killing hardcoded sleeps), plus computer-vision element finding (PaddleOCR for text, OpenCV matching icons against Polaris design-system SVGs) so tests locate elements the way a user would. Test IDs became opt-in fallbacks with an "UNSAFE_" prefix. Result: 98% stability. Takeaway: frameworks that expose low-level controls without guardrails accumulate anti-patterns — make best practices the path of least resistance. Source: How we raised mobile end-to-end test stability to 98% — Shopify Engineering, Aug 12 2026 — https://shopify.engineering/mobile-e2e-testing This is commentary/summary in the hosts' own words, not a reproduction of the article.

    Mehr anzeigen Weniger anzeigen
    6 Min.