[Bro-Dev] Broker raw throughput

[Dominik Charousset]

With most noise like serialization etc. out of the way, this is what I measured on Linux:

native sender -> native receiver
567520085 Bytes/s

CAF sender -> native receiver
511333973 Bytes/s

native sender -> CAF receiver
229689173 Bytes/s

CAF sender -> CAF receiver
222102755 Bytes/s

Send performance is OK, but performance drops significantly once CAF is used at the receiver. The profiler output (attached) doesn't point to a particular function that consumes an inappropriate amount of time. So it's either the sum of the many little functions called for each received chunk or the epoll_wait loop itself.

I have created a ticket for further progress tracking / discussion [1] as this is clearly not a Bro/Broker problem. Thank you all for reporting this and all the input you have provided.

@Matthias: FYI, I have used a new feature in CAF that allows senders to get feedback from the I/O layer for not overloading it. This allows the sender to adapt to the send rate of the network.

    Dominik


[1] https://github.com/actor-framework/actor-framework/issues/432 <https://github.com/actor-framework/actor-framework/issues/432>


> On Mar 1, 2016, at 16:05, Dominik Charousset <Dominik.Charousset at haw-hamburg.de> wrote:
> 
> Thanks for providing build scripts and sharing results.
> 
> Just a quick heads-up from me: I have implemented a simple sender/receiver pair using C sockets as well as CAF brokers (attached, but works only with the current actor-system topic branch). Both sending and receiving are slower with CAF (as expected), although the performance is slightly better when using the ASIO backend [1]. I'm still investigating and hopefully come back to you guys later this week.
> 
>     Dominik
> 
> [1] e.g. ./caf_impl --caf#middleman.network-backend=asio -s
> 
> > On Feb 25, 2016, at 17:19, Matthias Vallentin <vallentin at icir.org> wrote:
> > 
> > For better reproducibility, here's the Makefile that I used to drive the
> > experiments:
> > 
> >    CC = cc
> >    CXX = c++
> >    FLAGS = -O3 -g -std=c++11 -stdlib=libc++
> >    LIBS = -lcaf_core -lcaf_io -ltcmalloc -lprofiler
> > 
> >    caf-client: caf-client.cpp
> >        $(CXX) $(FLAGS) $< -o $@ $(LIBS)
> > 
> >    caf-server: caf-server.cpp
> >        $(CXX) $(FLAGS) $< -o $@ $(LIBS)
> > 
> >    bench-caf-client:
> >        CPUPROFILE=caf-client.prof ./caf-client 1000
> > 
> >    bench-caf-server:
> >        CPUPROFILE=caf-server.prof ./caf-server 10
> > 
> >    bench-caf-pprof: caf-client.prof caf-server.prof 
> >        pprof --pdf caf-client caf-client.prof > caf-client.pdf
> >        pprof --pdf caf-server caf-server.prof > caf-server.pdf
> > 
> > On my FreeBSD box, I had to add /usr/local/include to -I and -L, because
> > I installed CAF and gperftools via ports. Since it's a headless machine
> > without ps2pdf, we need extra level of indirection:
> > 
> >    (1) pprof --raw caf-client caf-client.prof > caf-client.raw
> >    (2) copy raw profile to desktop
> >    (3) pprof --pdf caf-client.raw > caf-client.pdf
> > 
> > Hope this helps,
> > 
> >    Matthias
> 
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev <http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev>
> <caf_impl.cpp><Makefile><native_impl.cpp>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20160302/cc67db69/attachment-0003.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: caf-client.pdf
Type: application/pdf
Size: 24820 bytes
Desc: not available
Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20160302/cc67db69/attachment-0002.pdf 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20160302/cc67db69/attachment-0004.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: caf-server.pdf
Type: application/pdf
Size: 25593 bytes
Desc: not available
Url : http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20160302/cc67db69/attachment-0003.pdf 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20160302/cc67db69/attachment-0005.html