AWS Applied AI Chief Colleen Aubrey Urges Enterprises to Ditch Token Metrics for Business Fundamentals

Enterprises attempting to transition generative artificial intelligence initiatives out of pilot testing should stop evaluating projects through custom metrics such as token consumption or prompt counts and instead assess them against established balance-sheet performance indicators, according to Colleen Aubrey, senior vice president of Applied AI Solutions at Amazon Web Services Inc.

Speaking in an interview with industry analyst Zeus Kerravala published by SiliconANGLE ahead of AWS re:Invent, Aubrey argued that organizations frequently trap AI projects in experimental limbo by treating computational volume as a proxy for business value. Aubrey, who sits on Amazon's senior leadership executive team known as the S-team, said that any initiative unable to show movement in metrics already monitored by a chief financial officer should be treated as an experiment rather than an enterprise investment.

Aubrey linked that perspective to her overhaul of AWS's application-layer portfolio, which contained 35 distinct programs when she took leadership of the group. Analyzing past performance, Aubrey pointed to the Amazon Connect contact center platform as the unit's clearest operational success. Rather than pursuing comprehensive feature parity with legacy call-center vendors, Connect prioritized foundational voice quality over the AWS network and addressed a narrow set of acute user workflows. AWS has since adopted that narrow-wedge methodology across a broader suite of applications, including Connect Decisions, Connect Customer, Connect Health, and Connect Talent, which launched into general availability in September.

That strategy prompted a fundamental reboot of AWS Supply Chain into Connect Decisions. Aubrey noted that initial efforts to build five distinct supply chain modules stalled because engineering teams focused on matching enterprise resource planning incumbents that possessed decades-long development leads. Connect Decisions narrowed that scope specifically to inventory planners who were overwhelmed by disconnected signal streams. In one deployment cited by AWS, automotive supplier Wells Vehicle Electronics reduced inventory levels by 7% within three months while growing its monitored stock-keeping units from 20 to 2,000, focusing on working capital rather than compute usage. Industrial manufacturer TE Connectivity has similarly rolled out Connect Decisions to 2,000 planners.

Aubrey also warned that rolling out AI tools inside technical organizations does not eliminate operational bottlenecks so much as expose downstream constraints. Within her own engineering organization, providing developers with AI coding tools initially produced experimental chaos rather than immediate throughput gains. Real velocity gains emerged only after teams systematically resolved the underlying test coverage and continuous delivery pipeline deficiencies that the coding assistants uncovered. Aubrey reported that those downstream infrastructure fixes ultimately yielded a twofold increase in the velocity of production code pushes across her unit.

To prevent software churn, AWS is structuring its application design around an approach Aubrey terms "humorphism"—designing AI capabilities to integrate into established worker routines instead of imposing command-line prompts or detached dashboards. Internally, that transition has driven Amazon to reshape team structures, merging user experience design with engineering execution and introducing hybrid technical roles evaluated on concrete business outcomes rather than task completion.

The operational blueprint outlines AWS's enterprise software thesis heading into re:Invent, scheduled for Nov. 30 through Dec. 4, where cloud providers face heightened enterprise scrutiny over the balance between capital expenditure on AI infrastructure and demonstrable balance-sheet productivity.