util.throttle

This is a small helper library to implement a throttle/limiter on some action over time. That is, a rate limiter. For example, this could be used to limit:

the number of messages per second, minute, etc.
the amount of traffic (in bytes)
the number of times a user performs some action (adding a user to their contact list, etc.)

Because util.throttle is flexible in what it can limit, you have to decide on a 'cost' for every action. If you are limiting based on traffic, for example, the cost of a packet of data would be the size of that packet in bytes. If you are limiting some action, you can simply use a cost of 1.

The library is an implementation of the token bucket algorithm.

Usage

This small example shows a fictional 'check_message' that ensures received messages are allowed by the throttles.

    local throttle = require "util.throttle";
 
    -- Create a throttle to allow only 5 messages to be sent in a minute (each message will have a cost of 1)
    local message_throttle = throttle.create(5, 60);
 
    -- Create a throttle to allow only 20 KB/s of messages through the server
    local traffic_throttle = throttle.create(20 * 1024, 1);
 
    function check_message(message)
        if not message_throttle:poll(1) then
          -- Reject this message, we have received too many, too fast
          return false;
        end
        if not traffic_throttle:poll(message.size_in_bytes) then
          -- Reject this message, because we are over the traffic limit
          return false;
        end
        -- The message fits within the limits of both throttles, let it through
        return true;
    end

In a real life system, sometimes you will want to reject the event (e.g. responding to a message with an error), and sometimes you will want to defer/delay the event while temporarily not accepting any further input (e.g. to smooth out traffic flow, with a traffic-based throttle). The extra return values from throttle:poll() can be used to determine how long to wait until the throttle is ready to allow events again.

Reference

create(max, period)

Creates a new throttle object. For 'max', specify the number of events or maximum total cost to allow over the given time 'period' (in seconds). Both parameters must be positive numbers, they need not be whole numbers.

Tip: Often you may want a "softer" limit, where temporarily going over the rate limit is allowed for short periods. This can be achieved by multiplying both 'max' and 'period' by the 'burst' factor. E.g. multiplying both by '2' will keep the overall average the same, but temporarily allow up to 2x the normal rate.

throttle:poll(cost)

If it would not take the throttle over the limit, deducts 'cost' and returns true, followed by the remaining maximum cost in the current time period.

If the cost would take the throttle over the limit, does nothing and returns false, followed by the remaining maximum cost in the current time period.

throttle:poll(cost, split)

This version of the 'poll' method is intended for events that can be split, for example if part of a packet can fit within the limit.

If it would not take the throttle over the limit, deducts 'cost' and returns true, followed by the remaining maximum cost in the current time period.

If the cost would take the throttle over the limit, and 'split' is true, deducts as much of 'cost' as would be allowed without going over the limit. Returns three results: true, the remaining balance (typically 0), and the outstanding cost (i.e. the cost that couldn't be satisfied, and should not be let through).

throttle:peek(cost)

Tests to see whether a given cost could be allowed, without actually counting it against the rate limit. Returns true if it would be allowed, false otherwise.

throttle:update()

Forces the throttle to update. This should not normally be called.