Package oracle.nosql.driver

Interface RateLimiter

public interface RateLimiter

RateLimiter interface provides default methods that all rate limiters must implement.

In NoSQL Cloud, rate limiters are used internally in NoSQLHandle operations when enabled using NoSQLHandleConfig#setRateLimitingEnabled}.

Typical usage: The simplest use of the rate limiter is to consume a number of units, blocking until they are successfully consumed:

    int delayMs = rateLimiter.consumeUnits(units);
    // delayMs indicates how long the consume delayed
 

To poll a limiter to see if it is currently over the limit:

    if (rateLimiter.tryConsumeUnits(0)) {
        // limiter is below its limit
    }
 

To attempt to consume units, only if they can be immediately consumed without waiting:

    if (ratelimiter.tryConsumeUnits(units)) {
        // successful consume
    } else {
        // could not consume units without waiting
    }
 

Usages that involve waiting with timeouts:

In cases where the number of units an operation will consume is already known before the operation, a simple one-shot method can be used:

    long units = (known units the operation will use)
    try {
        boolean alwaysConsume=false; // don't consume if we time out
        int delayMs = rateLimiter.consumeUnitsWithTimeout(units,
                      timeoutMs, alwaysConsume);
        // we waited delayMs for the units to be consumed, and the
        // consume was successful
        ...do operation...
    } catch (TimeoutException e) {
        // we could not do the operation within the given timeframe.
        ...skip operation...
    }
 

In cases where the number of units an operation will consume is not known before the operation, typically two rate limiter calls would be used: one to wait till the limiter is below it limit, and a second to update the limiter with used units:

    // wait until we're under the limit
    try {
        // note here we don't consume units if we time out
        int delayMs = rateLimiter.consumeUnitsWithTimeout(0,
                      timeoutMs, false);
    } catch (TimeoutException e) {
        // we could not go below the limit within the given timeframe.
        ...skip operation...
    }
    // we waited delayMs to be under the limit, and were successful
    long units = ...do operation, get number of units used...
    // update rate limiter with consumed units. Next operation on this
    // limiter may delay due to it being over its limit.
    rateLimiter.consumeUnits(units);
 

Alternately, the operation could be always performed, and then the limiter could try to wait for the units to be consumed:

    long units = ...do operation, get number of units used...
    try {
        boolean alwaysConsume=true; // consume, even if we time out
        int delayMs = rateLimiter.consumeUnitsWithTimeout(units,
                      timeoutMs, alwaysConsume);
        // we waited delayMs for the units to be consumed, and the
        // consume was successful
    } catch (TimeoutException e) {
        // the rate limiter could not consume the units and go
        // below the limit in the given timeframe. Units are
        // consumed anyway, and the next call to this limiter
        // will likely delay
    }
 

Limiter duration: Implementing rate limiters should support a configurable "duration". This is sometimes referred to as a "burst mode", or a "window time", or "burst duration". This will allow consumes of units that were not consumed in the recent past. For example, if a limiter allows for 100 units per second, and is not used for 5 seconds, it should allow an immediate consume of 500 units with no delay upon a consume call, assuming that the limiter's duration is set to at least 5 seconds. The maximum length of time for this duration should be configurable. In all cases a limiter should set a reasonable minimum duration, such that a call to

tryConsumeUnits(1)

has a chance of succeeding. It is up to the limiter implementation to determine if units from the past before the limiter was created or reset are available for use. If a limiter implemetation does not allow setting a duration, it must throw an UnsupportedOperationException when its setDuration() method is called.

Method Summary

Modifier and Type	Method	Description
int	consumeUnits(long units)	Consumes a number of units, blocking until the units are available.
void	consumeUnitsUnconditionally(long units)	Consumes units without checking or waiting.
int	consumeUnitsWithTimeout(long units, int timeoutMs, boolean alwaysConsume)	Attempts to consume a number of units, blocking until the units are available or the specified timeout expires.
double	getCurrentRate()	Returns the current rate as a percentage of current limit.
default double	getDuration()	Returns the duration configured for this rate limiter instance
double	getLimitPerSecond()	Returns the number of units configured for this rate limiter instance
void	reset()	Resets the rate limiter as if it was newly constructed.
void	setCurrentRate(double rateToSet)	Sets the current rate as a percentage of current limit.
default void	setDuration(double durationSecs)	Sets the duration for this rate limiter instance.
void	setLimitPerSecond(double rateLimitPerSecond)	Sets a new limit (units per second) on the limiter.
boolean	tryConsumeUnits(long units)	Consumes the specified number of units if they can be returned immediately without waiting.

Method Details

consumeUnits

int consumeUnits(long units)

Consumes a number of units, blocking until the units are available.

Parameters:

units - the number of units to consume. This can be a negative value to "give back" units.

Returns:

the amount of time blocked in milliseconds. If not blocked, 0 is returned.

tryConsumeUnits

boolean tryConsumeUnits(long units)

Consumes the specified number of units if they can be returned immediately without waiting.

Parameters:

units - The number of units to consume. Pass zero to poll if the limiter is currently over its limit. Pass negative values to "give back" units (same as calling consumeUnits(long) with a negative value).

Returns:

true if the units were consumed, false if they were not immediately available. If units was zero, true means the limiter is currently below its limit.

consumeUnitsWithTimeout

int consumeUnitsWithTimeout(long units,
 int timeoutMs,
 boolean alwaysConsume)
                     throws TimeoutException,
IllegalArgumentException

Attempts to consume a number of units, blocking until the units are available or the specified timeout expires.

Parameters:

units - the number of units to consume. This can be a negative value to "give back" units.

timeoutMs - the timeout in milliseconds. Pass 0 to block indefinitely. To poll if the limiter is currently over its limit, use tryConsumeUnits(long) instead.

alwaysConsume - if true, consume units even on timeout

Returns:

the amount of time blocked in milliseconds. If not blocked, 0 is returned.

Throws:

TimeoutException - if the timeout expires before the units can be acquired by the limiter.

IllegalArgumentException - if the timeout is negative

getLimitPerSecond

double getLimitPerSecond()

Returns the number of units configured for this rate limiter instance

Returns:

the max number of units per second this limiter allows

setDuration

default void setDuration(double durationSecs)

Sets the duration for this rate limiter instance.

The duration specifies how far back in time the limiter will go to consume previously unused units.

For example, if a limiter had a limit of 1000 units and a duration of 5 seconds, and had no units consumed for at least 5 seconds, a call to

tryConsumeUnits(5000)

will succeed immediately with no waiting.

Parameters:

durationSecs - the duration in seconds

Throws:

UnsupportedOperationException - if duration is not supported by this limiter.

getDuration

default double getDuration()

Returns the duration configured for this rate limiter instance

Returns:

the duration in seconds

Throws:

UnsupportedOperationException - if duration is not supported by this limiter.

reset

void reset()

Resets the rate limiter as if it was newly constructed.

Allows reuse.

setLimitPerSecond

void setLimitPerSecond(double rateLimitPerSecond)

Sets a new limit (units per second) on the limiter.

Note that implementing rate limiters should fully support non-integer (double) values internally, to avoid issues when the limits are set very low. Changing the limit may lead to unexpected spiky behavior, and may affect other threads currently operating on the same limiter instance.

Parameters:

rateLimitPerSecond - the new number of units to allow

consumeUnitsUnconditionally

void consumeUnitsUnconditionally(long units)

Consumes units without checking or waiting. the internal amount of units consumed will be updated by the given amount, regardless of its current over/under limit state.

Parameters:

units - the number of units to consume (may be negative to give back units)

getCurrentRate

double getCurrentRate()

Returns the current rate as a percentage of current limit.

Returns:

the rate as of this instant in time.

setCurrentRate

void setCurrentRate(double rateToSet)

Sets the current rate as a percentage of current limit.

This modifies the internal limiter values; it does not modify the rate limit.

Parameters:

rateToSet - percentage to set the current rate to. This may be greater than 100.0 to set the limiter to "over its limit".