hikari.impl.buckets#
Rate-limit extensions for RESTful bucketed endpoints.
Provides implementations for the complex rate limiting mechanisms that Discord requires for rate limit handling that conforms to the passed bucket headers correctly.
This was initially a bit of a headache for me to understand, personally, since there is a lot of "implicit detail" that is easy to miss from the documentation.
In an attempt to make this somewhat understandable by anyone else, I have tried to document the theory of how this is handled here.
What is the theory behind this implementation?
In this module, we refer to a CompiledRoute as a definition
of a route with specific major parameter values included (e.g.
POST /channels/123/messages), and a Route as a definition of a route
without specific parameter values included
(e.g. POST /channels/{channel}/messages). We can create a CompiledRoute
from a Route by providing the corresponding parameters as kwargs, as you
may already know.
In this module, a "bucket" is an internal data structure that tracks and
enforces the rate limit state for a specific CompiledRoute,
and can manage delaying tasks in the event that we begin to get rate limited.
It also supports providing in-order execution of queued tasks.
Discord allocates types of buckets to routes. If you are making a request and
there is a valid rate limit on the route you hit, you should receive an
X-RateLimit-Bucket header from the server in your response. This is a hash
that identifies a route based on internal criteria that does not include major
parameters. This X-RateLimitBucket is known in this module as an "bucket hash".
This means that generally, the route POST /channels/123/messages and
POST /channels/456/messages will usually sit in the same bucket, but
GET /channels/123/messages/789 and PATCH /channels/123/messages/789 will
usually not share the same bucket. Discord may or may not change this at any
time, so hard coding this logic is not a useful thing to be doing.
Rate limits, on the other hand, apply to a bucket and are specific to the major
parameters of the compiled route. This means that POST /channels/123/messages
and POST /channels/456/messages do not share the same real bucket, despite
Discord providing the same bucket hash. A real bucket hash is the string hash of
the bucket that Discord sends us in a response concatenated to the corresponding
major parameters. This is used for quick bucket indexing internally in this
module.
One issue that occurs from this is that we cannot effectively hash a
CompiledRoute that has not yet been hit, meaning that
until we receive a response from this endpoint, we have no idea what our rate
limits could be, nor the bucket that they sit in. This is usually not
problematic, as the first request to an endpoint should never be rate limited
unless you are hitting it from elsewhere in the same time window outside your
hikari.applications. To manage this situation, unknown endpoints are allocated to
a special unlimited bucket until they have an initial bucket hash code allocated
from a response. Once this happens, the route is reallocated a dedicated bucket.
Unknown buckets have a hardcoded initial hash code internally.
Initially acquiring time on a bucket
Each time you call hikari.impl.buckets.RESTBucket.acquire a request
timeslice for a given Route, several things happen. The first is that we
attempt to find the existing bucket for that route, if there is one, or get an
unknown bucket otherwise. This is done by creating a real bucket hash from the
compiled route. The initial hash is calculated using a lookup table that maps
CompiledRoute objects to their corresponding initial hash
codes, or to the unknown bucket hash code if not yet known. This initial hash is
processed by the CompiledRoute to provide the real bucket
hash we need to get the route's bucket object internally.
The hikari.impl.buckets.RESTBucket.acquire method will take the
bucket and acquire a new timeslice on it. This takes the form of a
asyncio.Future that is awaited and will complete once the caller is allowed
to make a request. Most of the time, this is done instantly, but if the bucket
has an active rate limit preventing requests being sent, then the future will
be paused until the rate limit is over. This may be longer than the rate limit
period if you have queued a large number of requests during this limit, as it
is first-come-first-served.
Acquiring a rate limited bucket will start a bucket-wide task (if not already running) that will wait until the rate limit has completed before allowing more futures to complete. This is done while observing the rate limits again, so can easily begin to re-ratelimit itself if needed. Once the task is complete, it tidies itself up and disposes of itself. This task will complete once the queue becomes empty.
The result of hikari.impl.buckets.RESTBucketManager.acquire_bucket is an async
context manager that must be acquired during the entirety of the request and
released once it is done (in reality, it is just a
hikari.impl.buckets.RESTBucket, but we want the ratelimit update to be
forced through hikari.impl.buckets.RESTBucketManager.update_rate_limits
to keep proper state)
Handling the rate limit headers of a response
Once you have received your response, you are expected to extract the values of the vital rate limit headers manually and parse them to the correct data types. These headers are:
X-RateLimit-Limit: anintdescribing the max requests in the bucket from empty to being rate limited.X-RateLimit-Remaining: anintdescribing the remaining number of requests before rate limiting occurs in the current window.X-RateLimit-Bucket: astrcontaining the initial bucket hash.X-RateLimit-Reset-After: afloatcontaining the number of seconds when the current rate limit bucket will reset with decimal millisecond precision.
Each of the above values should be passed to the
hikari.impl.buckets.RESTBucketManager.update_rate_limits method to
ensure that the bucket you acquired time from is correctly updated should
Discord decide to alter their ratelimits on the fly without warning (including
timings and the bucket).
This method will manage creating new buckets as needed and resetting vital information in each bucket you use.
Body-field-specific rate limiting
As of the start of June, 2020, Discord appears to be enforcing another layer of rate limiting logic to their HTTP APIs which is field-specific. This means that special rate limits will also exist on some endpoints that limit based on what attributes you send in a JSON or form data payload.
No information is sent in headers about these specific limits. You will only
be made aware that they exist once you get ratelimited. In the 429 ratelimited
response, you will have the "global" attribute set to False, and a
"reset_after" attribute that differs entirely to the X-RateLimit-Reset-After
header. Thus, it is important to not assume the value in the 429 response
for the reset time is the same as the one in the bucket headers. hikari's
hikari.api.rest.RESTClient implementation specifically uses the value
furthest in the future when working out which bucket to adhere to.
It is worth remembering that there is an API limit to the number of 401s, 403s, and 429s you receive, which is around 10,000 per 15 minutes. Passing this limit results in a soft ban of your account.
The true nature of these limits are not known and Discord staff have repeatedly pointed to them never being documented for the sake of system integrity. These special ratelimits are not something a normal user should encounter unless they are calling a single route multiple times with the end goal of editing a single attribute in quick succession. It is up to Discord's discretion on what is considered as "spammy" behaviour and one they would not like to allow on their API.
These ratelimits should not be "properly" handled and instead be avoided completely by the end developer (similar to Cloudflare 429s).
module-attribute
#
The hash used for an unknown bucket that has not yet been resolved.
RESTBucket(
name: str,
compiled_route: CompiledRoute,
global_ratelimit: ManualRateLimiter,
max_rate_limit: float,
)
Bases: WindowedBurstRateLimiter
Represents a rate limit for an HTTP endpoint.
Component to represent an active rate limit bucket on a specific HTTP route with a specific major parameter combo.
This is somewhat similar to the hikari.impl.rate_limits.WindowedBurstRateLimiter in how it
works.
This algorithm will use fixed-period time windows that have a given limit (capacity). Each time a task requests processing time, it will drip another unit into the bucket. Once the bucket has reached its limit, nothing can drip and new tasks will be queued until the time window finishes.
Once the time window finishes, the bucket will empty, returning the current capacity to zero, and tasks that are queued will start being able to drip again.
Additional logic is provided by the hikari.impl.buckets.RESTBucket.update_rate_limit call
which allows dynamically changing the enforced rate limits at any time.
instance-attribute
#
The maximum number of hikari.impl.rate_limits.WindowedBurstRateLimiter.acquire's
allowed in this time window.
instance-attribute
#
How long the window lasts for from the start in seconds.
instance-attribute
#
The queue of any futures under a rate limit.
instance-attribute
#
remaining: int = 0
The number of hikari.impl.rate_limits.WindowedBurstRateLimiter.acquire's
left in this window before you will get rate limited.
instance-attribute
#
reset_at: float = 0.0
The time.monotonic that the limit window ends at.
async
#
Acquire time and the lock on this bucket.
Note
You should afterwards invoke hikari.impl.buckets.RESTBucket.update_rate_limit to
update any rate limit information you are made aware of and
hikari.impl.buckets.RESTBucket.release to release the lock.
| RAISES | DESCRIPTION |
|---|---|
RateLimitTooLongError
|
If the rate limit is longer than |
Determine how long until the current rate limit is reset.
Warning
Invoking this method will update the internal state if we were previously rate limited, but at the given time are no longer under that limit. This makes it imperative that you only pass the current timestamp to this function, and not past or future timestamps. The effects of doing the latter are undefined behaviour.
| PARAMETER | DESCRIPTION |
|---|---|
now |
The monotonic
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
float
|
The time left to sleep before the rate limit is reset. If no rate limit
is in effect, then this will return |
Determine if we are under a rate limit at the given time.
Warning
Invoking this method will update the internal state if we were previously rate limited, but at the given time are no longer under that limit. This makes it imperative that you only pass the current timestamp to this function, and not past or future timestamps. The effects of doing the latter are undefined behaviour.
| PARAMETER | DESCRIPTION |
|---|---|
now |
The monotonic
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
bool
|
Whether the bucket is ratelimited. |
resolve(real_bucket_hash: str) -> None
Resolve an unknown bucket.
| PARAMETER | DESCRIPTION |
|---|---|
real_bucket_hash |
The real bucket hash for this bucket.
TYPE:
|
| RAISES | DESCRIPTION |
|---|---|
RuntimeError
|
If the hash of the bucket is already known. |
async
#
Perform the throttling rate limiter logic.
Iterates repeatedly while the queue is not empty, adhering to any rate limits that occur in the mean time.
Note
You should usually not need to invoke this directly, but if you do,
ensure to call it using asyncio.create_task, and store the
task immediately in
hikari.impl.rate_limits.WindowedBurstRateLimiter.throttle_task.
When this coroutine function completes, it will set the
hikari.impl.rate_limits.WindowedBurstRateLimiter.throttle_task
to None. This means you can check if throttling
is occurring by checking if it is not None.
Update the rate limit information.
Note
The reset_at epoch is expected to be a time.monotonic
monotonic epoch, rather than a time.time date-based epoch.
| PARAMETER | DESCRIPTION |
|---|---|
remaining |
The calls remaining in this time window.
TYPE:
|
limit |
The total calls allowed in this time window.
TYPE:
|
reset_at |
The epoch at which to reset the limit.
TYPE:
|
RESTBucketManager(max_rate_limit: float)
The main rate limiter implementation for HTTP clients.
This is designed to provide bucketed rate limiting for Discord HTTP
endpoints that respects the X-RateLimit-Bucket rate limit header. To do
this, it makes the assumption that any limit can change at any time.
| PARAMETER | DESCRIPTION |
|---|---|
max_rate_limit |
The max number of seconds to backoff for when rate limited. Anything greater than this will instead raise an error.
TYPE:
|
acquire_bucket(
compiled_route: CompiledRoute, authentication: Optional[str]
) -> AsyncContextManager[None]
Acquire a bucket for the given route.
Note
You MUST keep the context manager acquired during the full duration
of the request: from making the request until calling hikari.impl.buckets.RESTBucket.update_rate_limit.
| PARAMETER | DESCRIPTION |
|---|---|
compiled_route |
The route to get the bucket for.
TYPE:
|
authentication |
The authentication that will be used in the request. |
| RETURNS | DESCRIPTION |
|---|---|
AsyncContextManager
|
The context manager to use during the duration of the request. |
Start this ratelimiter up.
This spins up internal garbage collection logic in the background to keep memory usage to an optimal level as old routes and bucket hashes get discarded and replaced.
| PARAMETER | DESCRIPTION |
|---|---|
poll_period |
Period to poll the garbage collector at in seconds.
TYPE:
|
expire_after |
Time after which the last
TYPE:
|
throttle(retry_after: float) -> None
Throttle the global ratelimit for the buckets.
| PARAMETER | DESCRIPTION |
|---|---|
retry_after |
How long to throttle for.
TYPE:
|
update_rate_limits(
compiled_route: CompiledRoute,
authentication: Optional[str],
bucket_header: str,
remaining_header: int,
limit_header: int,
reset_after: float,
) -> None
Update the rate limits for a bucket using info from a response.
| PARAMETER | DESCRIPTION |
|---|---|
compiled_route |
The compiled route to get the bucket for.
TYPE:
|
authentication |
The authentication that was used in the request. |
bucket_header |
The
TYPE:
|
remaining_header |
The
TYPE:
|
limit_header |
The
TYPE:
|
reset_after |
The
TYPE:
|