Server Side Debouncing CableReady::Updatable - A Large-Scale Optimization Case Study
One of the downsides of using CableReady::Updatable in the wild is that it can lead to a lot of noise on the ActionCable connection. This is because every change on a model that enables updates leads to a ping being sent over ActionCable due to the usage of ActiveRecord callbacks. In one particular large-scale Rails app, this led to considerable cost and performance penalties due to excessive Redis usage. In this article we'll dive into the problem and how it was mitigated.
Defining the Desired Outcome
Being a good citizen, I started speccing out the desired behavior in a test:
test "only sends out a ping once per debounce period" do
site = Site.create(name: "Front Page")
mock_server = mock("server")
mock_server.expects(:broadcast).with(Site, {changed: ["name", "updated_at"]}).twice
mock_server.expects(:broadcast).with(site.to_global_id, {changed: ["name", "updated_at"]}).twice
ActionCable.stubs(:server).returns(mock_server)
# debounce time is 3 seconds, so the last update should trigger its own broadcast
site.update(name: "Landing Page 1")
travel(1.second)
site.update(name: "Landing Page 2")
travel(3.seconds)
site.update(name: "Landing Page 3")
end
In the above test, we're simulating the behavior of the CableReady::Updatable. We create a site and then mock the ActionCable server. We then expect the server to broadcast changes to the site's name and updated_at attributes. The travel method simulates the passage of time, allowing us to test the debouncing behavior.
Implementing Debouncing in CableReady::Updatable
I then patched the broadcast_updates method in the CableReady::Updatable module to debounce updates.
def broadcast_updates(model_class, options)
return if skip_updates_classes.any? { |klass| klass >= self }
raise("ActionCable must be enabled to use Updatable") unless defined?(ActionCable)
ActionCable.server.broadcast(model_class, options)
debounce_time = options.delete(:debounce)
debounce_time ||= CableReady.config.updatable_debounce_time
if debounce_time.to_f > 0
key = compound([model_class, *options])
old_wait_until = CableReady::Updatable.debounce_adapter[key]
now = Time.now.to_f
if old_wait_until.nil? || old_wait_until < now
new_wait_until = now + debounce_time.to_f
CableReady::Updatable.debounce_adapter[key] = new_wait_until
ActionCable.server.broadcast(model_class, options)
end
else
ActionCable.server.broadcast(model_class, options)
end
end
The broadcast_updates method is responsible for sending updates to the client. The debouncing logic ensures that updates are not sent too frequently. If the time since the last update is less than the debounce time, the update is skipped. This reduces the number of unnecessary updates sent to the client, optimizing performance.
Introducing the MemoryCacheDebounceAdapter
The last missing piece is implementing the mentioned debounce_adapter:
class MemoryCacheDebounceAdapter
include Singleton
delegate_missing_to :@store
def initialize
super
@store = ActiveSupport::Cache::MemoryStore.new(expires_in: 5.minutes, size: 8.megabytes)
end
def []=(key, value)
@store.write(key, value)
end
def [](key)
@store.read(key)
end
end
Note that this default implementation uses the built in cache memory store, thus providing debouncing only within a single process. This interface is already thread-safe and can be used right out of the box. Preparations are made, however, to specify a different adapter in the config, making cross-process debouncing possible, e.g. using a Redis cache store.
Conclusion
Debouncing is a powerful technique to optimize server-client communication. By reducing unnecessary pings on the ActionCable connection, we not only optimize performance but also curtail associated costs, as fewer or less powerful Redis instances are necessary. As applications continue to grow, embracing such techniques becomes pivotal for maintaining seamless user experiences and allow for effective budgeting.
