[Bro] /misc/capture_loss percent_lost vs /misc/stats pkts dropped and missed bytes in bro_conn

[Federico Foschini]

Hi, I’m using af_packet. This is my broctl.cfg file:

LogRotationInterval = 3600
LogExpireInterval = 5day
StatsLogEnable = 1
StatsLogExpireInterval = 14
StatusCmdShowAll = 0
CrashExpireInterval = 0
SitePolicyScripts = local.bro
LogDir = /var/log/bro/logs
SpoolDir = /var/log/bro/spool
CfgDir = /opt/bro/etc
lb_custom.InterfacePrefix=af_packet::

And this is my node.cfg file:

[manager]
type=manager
host=localhost

[proxy-1]
type=proxy
host=localhost

[worker-1]
type=worker
host=localhost
interface=enp2s0f1
lb_method=custom
lb_procs=2
af_packet_fanout_id=21
af_packet_fanout_mode=AF_Packet::FANOUT_HASH
af_packet_buffer_size=128*1024*1024

I hope this helps. Thanks for your help!

Il giorno mar 16 ott 2018 alle ore 19:29 Michał Purzyński <
michalpurzynski1 at gmail.com> ha scritto:

> Tell us what kind of capture method you use and we will take it from here.
>
>
> On Oct 16, 2018, at 2:55 AM, Federico Foschini <undicizeri at gmail.com>
> wrote:
>
> Hello,
>
> In one of our bro deployments we are logging some missed byets on
> bro_conn logs. This is an example of a conn log with missing bytes:
>
> "local_resp": false,
>     "tunnel_parents": [],
>     "local_orig": true,
>     "dst_addr": "211.115.118.190",
>     "src_port": 57786,
>     "dst_port": 443,
>     "service": "ssl",
>     "duration": 0.717725,
>     "resp_pkts": 28,
>     "src_addr": "10.16.0.115",
>     "uid": "C7H1Jb1qJhHVg05wq8",
>     "history": "ShADadfF",
>     "orig_pkts": 16,
>     "host": "logstash",
>     "conn_state": "SF",
>     "orig_bytes": 2883,
>     "path": "/var/log/bro/logs/current/conn.log",
>     "@timestamp": "2018-10-16T09:42:14.074Z",
>     "times_created": "2018-10-16T09:42:13.357Z",
>     "tags": [
>       "bro",
>       "bro_conn"
>     ],
>     "proto": "tcp",
>     "@version": "1",
>     "resp_ip_bytes": 23649,
>     "orig_ip_bytes": 3535,
>     "missed_bytes": 2920,
>     "resp_bytes": 22517,
>     "resp_cc": "IT"
>   }
>
> I’m running both /policy/misc/capture_loss and /policy/misc/stats scripts
> and this is the result:
> /misc/stats:
>
> "_source": {
>     "files": 40386,
>     "mem": 820,
>     "active_icmp_conns": 341,
>     "dns_requests": 0,
>     "active_tcp_conns": 6641,
>     "timers": 542182,
>     "peer": "worker-1-1",
>     "reassem_file_size": 1040104,
>     "events_proc": 2285899,
>     "active_timers": 33245,
>     "host": "logstash",
>     "reassem_frag_size": 10528,
>     "active_files": 208,
>     "icmp_conns": 877,
>     "events_queued": 2285898,
>     "pkts_dropped": 0,
>     "pkts_proc": 10232397,
>     "path": "/var/log/bro/logs/current/stats.log",
>     "pkts_link": 10232664,
>     "udp_conns": 21084,
>     "reassem_unknown_size": 0,
>     "@timestamp": "2018-10-16T09:15:32.648Z",
>     "pkt_lag": 0.007681,
>     "active_dns_requests": 0,
>     "reassem_tcp_size": 863992,
>     "tags": [
>       "bro",
>       "bro_stats"
>     ],
>     "active_udp_conns": 2207,
>     "tcp_conns": 27070,
>     "@version": "1",
>     "bytes_recv": 6580937768
>   }
>
> /misc/capture_loss:
>
> "_source": {
>     "gaps": 92247,
>     "peer": "worker-1-1",
>     "path": "/var/log/bro/logs/current/capture_loss.log",
>     "ts_delta": 900.000031,
>     "@timestamp": "2018-10-16T09:15:32.632Z",
>     "percent_lost": 2.053046,
>     "tags": [
>       "bro",
>       "bro_stats",
>       "bro_capture_loss"
>     ],
>     "@version": "1",
>     "host": "logstash",
>     "acks": 4493178
>   }
>
> By reading the documentation It looks like the switch SPAN port or the
> network interface is dropping packets since bro stats doesn’t register any
> packet drops.
> I’ve checked on the switch and it doesn’t report any dropped traffic.
>
> Is this possible that the network interface of our server is dropping? Is
> there a way to analyze the problem further?
> --
> Federico Foschini.
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>

-- 
Federico Foschini.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181017/831c3d35/attachment-0001.html