Some companies are encouraging AI tokenmaxxing, but this is likely to backfire.
getty
In today’s column, I examine a new trend about generative AI and large language models (LLMs) that seems to be causing quite a bit of eyerolling. It has to do with the use of AI tokens, which, at a 30,000-foot level, generally refers to how many words you use in your prompts and AI-generated responses (I’ll explain more momentarily).
Here’s the backstory. Many companies are encouraging their employees to use AI. One way to do so is to tell the employees that the company is going to keep track of the number of tokens that each employee consumes (i.e., essentially the number of words in prompts and responses). The more tokens you consume, the more AI that you are seemingly utilizing.
Firms have even set up internal leaderboards that showcase for each employee the number of tokens they have used per day, per week, per month, and so on. Employees are proud to get their name at the top of the leaderboard. By using AI as much as possible, their token count gets higher and higher. Some companies give prizes, monetary bonuses, a pat on the back, and other heralded rewards for the highest token counts.
The rub is this. Just because you consume a lot of tokens doesn’t mean you are productively using AI. You can sneakily tell AI to do dumb and useless things; meanwhile, your token counts keep mounting. Employees realize that companies are creating a stupid game of sorts. If your employer wants you to use up tokens, by gosh, you’ll readily find a means to do so. Crank out all sorts of AI slop that has no value and no significance to the company. The gist is that you can claim to heroically be using AI and maximizing your token count.
This nutty race to maximize tokens is often known as tokenmaxxing. Some people brag openly about their tokenmaxxing. On a private basis, sure, have fun and spin the wheels. But are companies shooting their own foot by inspiring employees to foolishly waste expensive and limited AI resources merely to become the king or queen of tokenmaxxing?
Let’s talk about it.
This analysis of AI breakthroughs is part of my ongoing Forbes column coverage on the latest in AI, including identifying and explaining various impactful AI complexities (see the link here).
Tokenization Is Crucial In AI
The technical crux of the matter entails the tokenization aspects of modern-era generative AI and LLMs. I’ve covered the nitty-gritty details of tokenization at the link here. I will provide a quick overview to get you up to speed.
When you enter a prompt into generative AI, the text gets converted into various numbers. Each word or part of a word is given an assigned number, known as a token. Those numbers are then dealt with throughout the rest of the processing of your prompt. Once the AI has arrived at an answer, the answer is still in a numeric format of tokens and needs to be converted back into text, so it is readable by the user. The AI proceeds to convert the generated tokens into words and displays the response to you accordingly.
That whole process is known as tokenization.
Everyday users are generally blissfully unaware of the tokenization process. It is akin to plumbing that takes place under the hood of AI. The reason that you might know about tokens is that many of the AI makers charge users based on the number of tokens that they consume. The AI keeps track of how many tokens are in your prompts, along with how many tokens the answer contains, and tacks on some additional token counts arising during the processing of your prompt. For the details on counting tokens, including the use of so-called thinking tokens, see my coverage at the link here.
People are usually billed or invoiced based on the number of tokens that they use during an AI session. If you aren’t paying for AI usage, you aren’t likely to ever see the bills. For those who are paying for their AI usage, they tend to become acutely aware of tokens since that’s the core metric used to determine how much they will be billed.
People who care about tokens as a real-world cost are often going to try to find ways to reduce how many tokens they consume. They want to pay the least they can for their AI usage. The hope is to stretch every dollar and consume only the least number of tokens absolutely necessary to get the job done. AI researchers are constantly finding new methods to minimize the number of tokens required to undertake various activities inside AI and make LLMs as efficient as possible.
Tokenmaxxing Becomes A Thing
Shifting gears, let’s discuss a rapidly expanding fad known as tokenmaxxing.
Tokenmaxxing entails the act of maximizing the number of tokens that you use during an AI session. This seems quite contrary to the idea of trying to minimize the number of tokens that you consume. Why would anybody opt to increase the number of tokens rather than decrease them?
One aspect is that you can brag to your friends and colleagues that you are a true tokenmaxxer. You’ve probably heard about a similar trend known as looksmaxxing, whereby you do whatever you can to increase your good looks. I previously identified that some people are AI-maxxing, meaning they try to use AI in everything they undertake, see the link here.
In that sense, tokenmaxxing ties into a fast-moving trend or perhaps a fad of attempting to maximize one thing or another. An AI user will get the AI to do all sorts of useless tasks so that the number of tokens gets higher and higher. Suppose you use generative AI, and it ordinarily consumes 1,000 tokens to do a given task. You realize this. Aha, you decide that you want to force the AI to use more tokens. Easy-peasy, just bloat the prompt or give the AI superfluous duties that will get the token consumption to 2,000 tokens or possibly 10,000 tokens.
Boom, drop the mic – you are tokenmaxxing.
Criticisms Of Tokenmaxxing
There is a growing backlash toward tokenmaxxing. Some are concerned that precious and costly AI resources are being used to merely make a silly claim. Suppose those AI resources could have gone towards curing cancer? Instead, they were wasted needlessly.
A retort is that if someone is willing to pay for the tokens, so be it. They can choose whether they want AI to proceed down a minimization route or a maximization path. It’s on their dime. Stop worrying about what other people choose to do.
The counterargument is that AI uses up electricity. Excessive electricity consumption produces excessive pollution. Also, water is typically used to cool the data centers that house AI servers. You are using up water to simply brag about your tokenmaxxing. It is an environmental and ecological concern.
Some say you shouldn’t place every tokenmaxxer into the same bucket. Suppose a user is genuinely trying to find a breakthrough in medical science and happens to consume a massive number of tokens doing so. They are essentially a tokenmaxxer, but they have their heart in the right place. Be fair and divide users into those that are tokenmaxxing for the sake of tokenmaxxing, versus those that perchance reach high counts of tokens while having AI do constructive tasks.
Companies On The AI Bandwagon
Into this murky brew come companies that are desirous of using AI and finding a means to stir their employees towards becoming avid users of AI.
As an executive, what levers or approaches would you take to get your employees to lean into AI at their jobs?
Assume that as an executive, you truly believe that your employees ought to be using AI. You are of the mind that AI will make them more productive. Meanwhile, you suspect that some or many of your employees are reluctant to use AI. Perhaps they fear that their job will be handed over to AI if they show how much AI can do. Or they simply don’t know what AI can do. Etc.
One angle would be to gamify the use of AI. Tell your employees that the more they use AI, the more they will be rewarded in some fashion. It could be just getting your name on a leaderboard. It might be a bonus or a financial reward. The aim is to get the employees into a positive mindset of leveraging AI in the workplace.
Besides rewards, there is the other side of the coin that can be made use of. Employees who rank at the bottom of the AI leaderboard could be jeopardizing their employment. Or they might be in trouble and stalling their career at the company. Executives can use both the carrot and the stick in their efforts to motivate staff.
A Bridge Too Far
Like any effort by management, if the approach isn’t well devised, there are bound to be problems.
As a consultant to businesses, one aspect that used to be common and made me shake my head had to do with how the customer service function was being managed. Firms would tell their customer service reps that they would be rewarded for the number of calls they handled per hour, per day, per week, and so on.
This seems sensible on the surface. You are presumably maximizing the productivity of the customer server agent. Get as many calls under their belt as possible. An agent doing 10 calls per hour is seemingly 5x as productive as an agent doing only 2 calls per hour.
The rub was this. Customer service agents realized they could force more calls by purposely shortening a given call. For example, a customer calls in and wants to know how to do something. The agent gives them a quick explanation. The agent knows that there is more to it, but they silently hold back those additional steps. In any case, the call is considered ended and closed.
Sure enough, the customer calls back later because they are stuck again and need the additional instructions. This is wonderful for the agent because they generated another call. Nice! If they keep doing this, they can max their calls.
The downside is that customer satisfaction plummeted. Customers hated having to make several calls rather than just one. But the company didn’t care about customer satisfaction. They weren’t paying attention to that metric. The metric that the executives cared about was the number of calls per agent. Period, end of story.
History Repeats Itself Repeatedly
The same issue is facing those companies that are currently clamoring for their employees to use AI.
Rather than gauging whether the AI usage is making a significant difference in job performance, these firms are focusing solely on maximizing tokens. The assumption is that if an employee is extensively using AI and generating lots of tokens, by gosh, they must be doing something fruitful. It’s a fragile assumption.
Employees have figured out how to game the system. They use AI to the maximum extent possible. Some of the employees will sincerely attempt to use the AI for sensible purposes. Others will try to get tokens produced even if the work for the AI is marginally useful. And there is a contingent that doesn’t care what the AI does, as long as it generates an enormous number of tokens.
Rethinking Tokenmaxxing At Work
In my example about the customer service agents, I had the firms change their discernment about measuring agents by calls alone. I created a more balanced set of metrics. Besides tracking the number of calls, customer satisfaction was equally tracked and given due weight, and included other important metrics. This made it much tougher to game things the way that some employees were doing so.
The same sense of balance should be used by companies toying with tokenmaxxing.
Stop being unduly fixated on the number of tokens consumed by your employees. Create additional measures that aptly align with productivity. Are they using AI in a manner that aids their work? You can then compare the tokens consumed against the impacts on productivity.
I realize that some executives will insist that the best way forward is to first get their employees excited about AI usage – thus, tokenmaxxing is perfectly fine. Once the employees are actively using AI, you can then turn towards the useful use of AI. Just get them to use AI. Make it fun. If tokenmaxxing achieves this, so be it. You can later turn the ship toward more advantageous uses of AI.
In my experience, that type of tomfoolery isn’t going to attain what the executives think it will attain. Employees will realize that their executives seem clueless about AI. When those same executives later try to shift the usage toward being useful, a lot of eyerolling is going to occur. Those executives have lost the faith of the employees. The employees will undoubtedly instantly look for ways to game whatever the next step is.
The World We Are In
Consider rewarding employees for finding useful ways to use AI. Do this at the get-go. The singular focus of tokenmaxxing is not the way to get there. It is myopic, wastes company resources, and signals to the employees that management is not serious about AI. If anything, it makes the executive seem like a dullard who is falling for yet another popular fad.
AI researchers might have a bona fide reason to seek tokenmaxxing if they are studying how internal mechanisms of modern-era AI can be refined and advanced. That’s a narrow realm. This doesn’t and shouldn’t open the floodgates throughout the rest of a firm.
Is tokenmaxxing a harmless and idle form of horseplay?
You will find ardent arguments on either side of the controversy. One side insists that it is fun and no one is harmed in the process. The other side contends that you are undermining all of us by needlessly consuming computer cycles and running up the undue consumption of world resources such as electricity and water.
A final thought for now. Businesses often float from one fad to another. Once a fad has run its course, the executives find a new one to jump on board with. If your firm is doing tokenmaxxing, be wary that once the rizz has receded, the odds are that something else will take hold.
As per the insightful words of Mark Twain: “Every time I reform in one direction, I go overboard in another.” Don’t be that kind of an executive.