[Bro-Dev] Broker raw throughput
vallentin at icir.org
Tue Mar 8 12:54:17 PST 2016
> You could additionally try to tweak
> caf#middleman.max_consecutive_reads, which configures how many
> new_data_msg messages a broker receives from the backend in a single
> shot. It makes sense to have the two separated, because one configures
> fairness in the scheduling and the other fairness of connection
Good to know about this tuning knob. I played with a few values, from 1
to 1K, but could not find an improvement by tweaking this value alone.
Have you already performed some measurements to find the optimal
combination of parameters?
> The 27.9% is accumulating all load down the path, isn't it?
Yeah, right, I must have confused the absolute vs. cumulative numbers in
this case. :-/
> By using many small messages, you basically maximize the messaging
Exactly. That is the worse-case scenario I'm trying to benchmark :-).
> Tackling the "many small messages problem" isn't going to be easy. CAF
> could try to wrap multiple messages from the network into a single
> heap-allocated storage that is then shipped to an actor as a whole,
> but this optimization would have a high complexity.
A common strategy to reduce high heap pressure involves custom
allocators, and memory pools in particular. Assuming that a single actor
produces a fixed number of message types (e.g., <= 10), one could create
one memory pool for each message type. What do you think about such a
More information about the bro-dev