<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<br class="">
<blockquote type="cite" class="">On Mar 8, 2018, at 2:51 PM, Jon Siwek <<a href="mailto:jsiwek@corelight.com" class="">jsiwek@corelight.com</a>> wrote:<br class="">
<br class="">
On Thu, Mar 8, 2018 at 2:18 PM, Azoff, Justin S <<a href="mailto:jazoff@illinois.edu" class="">jazoff@illinois.edu</a>> wrote:<br class="">
<br class="">
<blockquote type="cite" class="">
<blockquote type="cite" class="">(1) &synchronized, &mergeable, &persistent. Seems fine to deprecate these now.<br class="">
</blockquote>
<br class="">
I'm fine with it going away, but there needs to be some sort of replacement for &synchronized,<br class="">
minimally a how-to for porting existing scripts to something else.<br class="">
</blockquote>
<br class="">
The answer is generally "use the new store API" to replace<br class="">
&synchronized or &persistent. It's difficult to give a canonical<br class="">
porting strategy as it depends on how the data is being<br class="">
accessed/modified, though you can find a preview of the how-to port<br class="">
document at [1]. Links and stuff will get rendered more nicely when<br class="">
it's live on <a href="http://bro.org" class="">bro.org</a>.<br class="">
<br class="">
- Jon<br class="">
<br class="">
[1] <a href="https://github.com/bro/bro/blob/topic/actor-system/doc/frameworks/broker.rst" class="">
https://github.com/bro/bro/blob/topic/actor-system/doc/frameworks/broker.rst</a><br class="">
</blockquote>
<div class=""><br class="">
</div>
The most common use of &synchronized here is for loading files into tables using input+reread. a cron job updates the table on the manager
<div class="">which updates the in memory table and syncs it over to all the workers.</div>
<div class=""><br class="">
</div>
<div class="">These are read only tables, so I think the best replacement for those may be to switch to the new configuration framework, which would probably</div>
<div class="">mean the deletion of a ton of verbose Input::add_table statements.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">I got the actor-system branch running on a small test cluster.. manager+logger+proxy on one box and 32 workers on 2 physical boxes.</div>
<div class=""><br class="">
</div>
<div class="">It seems to be working for the most part. known hosts and friends are working great now with no duplicate entries at startup.</div>
<div class=""><br class="">
</div>
<div class="">One thing I notice is that the traffic to/from the manager box and cpu has increased quite a bit.</div>
<div class=""><br class="">
</div>
<div class="">Ignoring the large CPU spikes from building the new branch just before the switch at 15:30, the overall cpu load on the manager box is about 3x higher.</div>
<div class="">The bandwidth isn't terribly excessive, but it increased from about 16mbit to 40mbit (stupid graph is in mebibytes).</div>
<div class=""><br class="">
</div>
<div class="">Based on some capstats runs, it's all going to the logger port 47761. I have another graph that shows the total log volume being written to disk and that hasn't changed.</div>
<div class="">So it looks like this branch uses 2x the cpu and bandwidth to write the same volume of logs.</div>
<div class=""><br class="">
</div>
<div class=""><img apple-inline="yes" id="D335F81A-3227-4FA0-BB11-CCBE22DD53FA" width="714" height="327" src="cid:498F993F-1122-4661-B343-AB3683018F09" class=""></div>
<div class=""><img apple-inline="yes" id="3F8A2185-86DE-408C-BEBB-C941112E7329" width="819" height="381" src="cid:D6C31831-533C-40AB-995B-B025A0237E2C" class=""></div>
<div class=""><br class="">
</div>
<div class="">— <br class="">
Justin Azoff<br class="">
<br class="">
</div>
</body>
</html>