Interface IRateLimiter
Namespace: Oracle.NoSQL.SDK
Assembly: Oracle.NoSQL.SDK.dll
Syntax
public interface IRateLimiterRemarks
Rate limiting is used to provide efficient access to Oracle NoSQL Database Cloud Service, maximize throughput and protect from resource starvation. The SDK provides built-in rate limiting functionality for data operations on tables such as GetAsync(String, MapValue, GetOptions, CancellationToken), PutAsync(String, MapValue, PutOptions, CancellationToken), DeleteAsync(String, MapValue, DeleteOptions, CancellationToken), DeleteRangeAsync, WriteManyAsync, PrepareAsync(String, PrepareOptions, CancellationToken), QueryAsync and their variants. Rate limiting in the driver allows spreading the operations over time thus maximizing throughput by avoiding costly throttling exceptions (see ReadThrottlingException and WriteThrottlingException) and the associated retry handling.
Each operation listed above does reads and/or writes on a table and thus consumes certain number of read and write units (see ConsumedCapacity). Each table in NoSQL Database Cloud Service defines limits on the maximum number of read and write units that can be consumed by all clients accessing the table, per 1 second time. For provisioned tables (see Provisioned) these limits are defined in TableLimits, for on-demand tables (see OnDemand), higher predefined limits are used. Thus, the total rate of operations on a given table is limited. If a client tries to perform the operations at faster rate, the service will respond with a read or write throttling exception, in which case the client can retry the operation after certain time (this behavior may be defined by NoSQLRetryHandler). Handling throttling exceptions and their retries is costly both for the service and the client application since the application has to wait before an operation can be retried (retrying immediately or not waiting enough will only result in another throttling exception and more load on the service). A much better strategy for a client is to spread out the operations over time to an extent that the table limits for a given table are not exceeded and thus avoiding throttling errors and their retries. Rate limiting built in to the SDK provides this functionality.
Enable rate limiting by setting the property
RateLimitingEnabled of the initial
configuration to true.
An instance of IRateLimiter is used to enforce one table limit. Thus the driver will instantiate two instances of IRateLimiter for each table in use, one for read limit and another for write limit. You have a choice of using a default implementation of rate limiter provided by the SDK or providing a custom implementation of IRateLimiter interface. See NoSQLRateLimiter for details on default rate limiter used by the SDK. To use custom rate limiter implementation, set property RateLimiterCreator of the initial configuration, which is factory function to create your custom instances of IRateLimiter.
In order to create pair of rate limiters for a table, the driver will need to know its table limits. The table limits are already known if one of table APIs have been called, such as ExecuteTableDDLAsync, SetTableLimitsAsync(String, TableLimits, TableDDLOptions, CancellationToken), GetTableAsync, WaitForCompletionAsync(Nullable<TimeSpan>, Nullable<TimeSpan>, CancellationToken) or any of their variants. Otherwise, the driver will call GetTableAsync(String, GetTableOptions, CancellationToken) in the background to obtain the table limits as soon as any data operation is issued for that table. This means that enabling rate limiting for a table may be delayed until its table limits are obtained successfully in the background.
The main operation of rate limiting in the driver is to call ConsumeUnitsAsync(Int32, TimeSpan, Boolean, CancellationToken) to consume a number of (read or write) units for a given operation. Depending on rate limiting implementation, its current state and the number of units to consume, this call may asynchronously block (sleep) for certain amount of time (and return Task returning this amount of time) before letting the operation proceed. This API also needs to correctly operate in the presence of timeout set by the caller. Note that it may be possible to consume certain amount of units without blocking (e.g. if there has been no or very little recent activity). In this state the rate limiter is said to be under its limit. Conversely, even consuming 0 units may block as a result of consuming units for recent past operations. In this state the rate limiter is said to be over its limit.
Rate limiting works best when we know in advance how many units each operation will consume, which would allow to know precisely how long to wait before issuing each operation. Unfortunately, we don't know how many units an operation requires until we get the result back, with this information returned as part of ConsumedCapacity. It may be difficult or impossible to estimate number of units required before the operation completes. The approach is taken where ConsumeUnitsAsync(Int32, TimeSpan, Boolean, CancellationToken) is called twice, once before the operation passing 0 units to (possibly) wait until the rate limiter is under its limit and then after the operation passing the number of units returned as part of ConsumedCapacity of the result. This will allow to delay subsequent operations and stagger subsequent concurrent operations over time.
Note that by default the rate limiting only works as expected when used within one NoSQLClient instance. When using multiple NoSQLClient instances against the same service, whether in the same process, different process or even running on different machines, rate limiters in one instance will not be aware of operations performed by other instances and thus will not correctly rate limit the operations. If multiple concurrent NoSQLClient instances are required, you can set RateLimiterPercent property of the initial configuration to allocate only a percentage of each table's limits to each NoSQLClient instance. Although not optimal (not accounting for under-utilization at some instances), this will allow to rate limit operations on multiple concurrent instances.
Unfortunately, there is no perfect driver-side rate limiting strategy so it is still possible for throttling exceptions to occur. IRateLimiter provides HandleThrottlingException(RetryableException) method that the driver will call when an operation results in throttling error. When creating custom rate limiter, implementing this method will allow you to adjust the rate limiter's state to account for this information.
Instances of IRateLimiter are used by multiple concurrent threads and thus must be thread-safe.
Methods
| Name | Description |
|---|---|
| ConsumeUnitsAsync(Int32, TimeSpan, Boolean, CancellationToken) | Consumes the specified number of units and blocks (sleeps) asynchronously if required before an operation can proceed. |
| HandleThrottlingException(RetryableException) | Defines the behavior of the rate limiter when throttling exception occurs. |
| SetLimit(Double) | Configures rate limiter by setting its limit in units per second. |