US Army hits AI token limits

US Army staff using generative AI tools were told in mid-June that their “unlimited” token pool had been exhausted, forcing new limits on usage, according to WIRED, as cited by Ars Technica. The email went to personnel at the Army’s Combat Capabilities Development Command (DEVCOM) and referenced the Army chief information officer’s token pool. The restriction landed weeks after the Department of Defense said nearly half of its roughly 3.5 million employees were using AI at work.

The episode is a mundane accounting problem with operational consequences. The Army is using Ask Sage, a multimodal platform that can run large language models including Alphabet’s Gemini, Meta’s Llama and OpenAI’s ChatGPT, and the service is accredited for Controlled Unclassified Information, according to the reporting. Tokens are the metered unit of output for such systems; Ask Sage equates one token to about 3.7 characters. Ars Technica reports that the Army had access to 100 million tokens annually as part of an enterprise subscription pack, with employees allocated at least 200,000 tokens per month and receiving automatic top-ups if they used their allotment.

When a central office sells “unlimited” access inside a bureaucracy, the bill still arrives somewhere, and the fastest users tend to be the least constrained. Ars Technica cites an anonymous Army employee saying the entire Army used up a whole year’s worth of tokens for just one service. The same story notes similar patterns in the private sector: Meta encouraged staff to maximize token use before removing a leaderboard tracking consumption, and Uber engineers reportedly burned through a year’s worth of tokens in four months. The common thread is that token budgets are easiest to spend when the costs are pooled and the benefits are local.

The timing also collides with the Pentagon’s push to expand AI use. Ars Technica reports that the Defense Department cut staff at the Civilian Protection Center of Excellence, which worked to prevent civilian casualties, while developing an AI tool to speed up assessments previously done by that team. Whether or not such tools ultimately reduce workload, a token cap turns “use AI everywhere” into a queue: the most politically protected projects get renewed access first, while routine users are told to ration prompts.

The Army chose to renew token usage at current levels, but Ars Technica reports it was unclear whether the CIO token pool would be renewed after October 1. In the meantime, the anonymous employee quoted in the story described the tools as not particularly reliable for their work, including one case where a model falsely claimed to have completed a task it had not done.

The Army’s enterprise AI workspace is marketed as always-on, but the limit arrived via an internal email telling people to use it less. The metering unit is small enough to count characters, and large enough to decide which parts of a bureaucracy get to automate first.