Rate limit: A rate limit is a cap on how much you can use an AI service in a given period, such as messages per hour on a chat plan or requests and tokens per minute on an API, set to protect the provider's capacity and to tier pricing.
On consumer chat plans, rate limits show up as usage caps: a certain number of messages on the strongest model in a window of hours, after which you are switched to a lighter model or asked to wait. Paid tiers raise the caps. On APIs, limits are expressed as requests per minute, tokens per minute, and sometimes tokens per day, and they usually increase as your account establishes a payment history.
When you hit a limit, the service refuses the request, often with an error code such as 429, and tells you when to retry. For a person in a chat window, that is an annoyance. For an automation processing hundreds of records, it is a failure mode you must plan for: the workflow needs to slow down, retry after a delay, or queue work rather than crash halfway through a batch.
Practical steps: know the limits of the plan you are on, watch the usage dashboard, and design batch jobs to run at a steady pace instead of all at once. For team rollouts, remember that shared accounts hit limits faster; individual seats on a team plan keep one person's heavy use from blocking everyone.
Example at work
An HR team runs an automation that screens 500 applications against a rubric on a Monday morning. It stops after 60 with a rate-limit error. The fix is to process applications a few at a time with a short pause between calls, which finishes in under an hour without any errors.
Why it matters
Rate limits are the most common reason an AI workflow that worked in testing fails in production. Knowing they exist, and building in pacing and retries, keeps a batch job from stopping halfway through on the morning you needed it.
Related terms
- API (application programming interface)An API (application programming interface) is a way for one piece of software to talk to another. An AI API lets your own tools, spreadsheets, or automations send a prompt to a model and receive the response, without a person typing in a chat window.
- TokenA token is the unit of text a language model reads and writes, usually a word or a piece of a word. Models measure input limits, output limits, and pricing in tokens; as a rough rule, 1,000 tokens is about 750 words of English.
- LatencyLatency is the delay between sending a request to an AI system and receiving its response. For language models it is usually measured as time to the first word and then words per second as the answer streams in.
- Workflow automationWorkflow automation connects apps and steps so that a trigger, such as a new form response or an incoming email, automatically runs a sequence of actions, increasingly with an AI step that reads, decides, or writes along the way.
- InferenceInference is the act of running a trained AI model to produce an output, such as answering a prompt or classifying an image. Training happens once; inference happens every time anyone uses the model, and it is what you pay for as a user.