Saturday 24 October 2026 05:46:46 GMT+02:00

Netcrook

HomeManifesto
News
Techcrook
Geocrook
WikicrookTeamAppContact
EnglishItaliano

WIKICROOK

Tokens

Small consumption units used to measure how much model processing an AI system performs.

In AI systems, tokens are small units of text that a model reads and generates. A token may be a whole word, part of a word, punctuation, or another piece produced by the model’s tokenizer. Models do not process raw text character by character; they work in token counts, which makes tokens the basic unit for measuring input size, output size, and compute usage.

Tokens matter in cyber security because they shape cost, rate limits, and abuse detection. An attacker can force high token consumption with long prompts, repeated retries, or agent loops, creating denial-of-service-like cost spikes. Defenders monitor token usage to spot abnormal behavior, enforce quotas, and audit which workflows consumed model capacity. In AI billing systems, token logs also help reconcile charges and detect tampering or duplicate metering. This term is different from authentication tokens, which are credentials used to prove identity.

← WIKICROOK index