[Bro] /misc/capture_loss percent_lost vs /misc/stats pkts dropped and missed bytes in bro_conn
Michał Purzyński
michalpurzynski1 at gmail.com
Wed Oct 17 01:40:57 PDT 2018
Excellent choice!
We created (with Peter Manev, a Suricata developer) a tuning guide that also applies to Bro.
Pay special attention to these two sections
- life of a packet
- packet drops (near the end)
There are a few places packets can be dropped, so it’s important to know all of them.
https://github.com/pevma/SEPTun
> On Oct 16, 2018, at 11:49 PM, Federico Foschini <undicizeri at gmail.com> wrote:
>
> Hi, I’m using af_packet. This is my broctl.cfg file:
>
> LogRotationInterval = 3600
> LogExpireInterval = 5day
> StatsLogEnable = 1
> StatsLogExpireInterval = 14
> StatusCmdShowAll = 0
> CrashExpireInterval = 0
> SitePolicyScripts = local.bro
> LogDir = /var/log/bro/logs
> SpoolDir = /var/log/bro/spool
> CfgDir = /opt/bro/etc
> lb_custom.InterfacePrefix=af_packet::
> And this is my node.cfg file:
>
> [manager]
> type=manager
> host=localhost
>
> [proxy-1]
> type=proxy
> host=localhost
>
> [worker-1]
> type=worker
> host=localhost
> interface=enp2s0f1
> lb_method=custom
> lb_procs=2
> af_packet_fanout_id=21
> af_packet_fanout_mode=AF_Packet::FANOUT_HASH
> af_packet_buffer_size=128*1024*1024
> I hope this helps. Thanks for your help!
>
>
>> Il giorno mar 16 ott 2018 alle ore 19:29 Michał Purzyński <michalpurzynski1 at gmail.com> ha scritto:
>> Tell us what kind of capture method you use and we will take it from here.
>>
>>
>>> On Oct 16, 2018, at 2:55 AM, Federico Foschini <undicizeri at gmail.com> wrote:
>>>
>>> Hello,
>>>
>>> In one of our bro deployments we are logging some missed byets on bro_conn logs. This is an example of a conn log with missing bytes:
>>>
>>> "local_resp": false,
>>> "tunnel_parents": [],
>>> "local_orig": true,
>>> "dst_addr": "211.115.118.190",
>>> "src_port": 57786,
>>> "dst_port": 443,
>>> "service": "ssl",
>>> "duration": 0.717725,
>>> "resp_pkts": 28,
>>> "src_addr": "10.16.0.115",
>>> "uid": "C7H1Jb1qJhHVg05wq8",
>>> "history": "ShADadfF",
>>> "orig_pkts": 16,
>>> "host": "logstash",
>>> "conn_state": "SF",
>>> "orig_bytes": 2883,
>>> "path": "/var/log/bro/logs/current/conn.log",
>>> "@timestamp": "2018-10-16T09:42:14.074Z",
>>> "times_created": "2018-10-16T09:42:13.357Z",
>>> "tags": [
>>> "bro",
>>> "bro_conn"
>>> ],
>>> "proto": "tcp",
>>> "@version": "1",
>>> "resp_ip_bytes": 23649,
>>> "orig_ip_bytes": 3535,
>>> "missed_bytes": 2920,
>>> "resp_bytes": 22517,
>>> "resp_cc": "IT"
>>> }
>>> I’m running both /policy/misc/capture_loss and /policy/misc/stats scripts and this is the result:
>>> /misc/stats:
>>>
>>> "_source": {
>>> "files": 40386,
>>> "mem": 820,
>>> "active_icmp_conns": 341,
>>> "dns_requests": 0,
>>> "active_tcp_conns": 6641,
>>> "timers": 542182,
>>> "peer": "worker-1-1",
>>> "reassem_file_size": 1040104,
>>> "events_proc": 2285899,
>>> "active_timers": 33245,
>>> "host": "logstash",
>>> "reassem_frag_size": 10528,
>>> "active_files": 208,
>>> "icmp_conns": 877,
>>> "events_queued": 2285898,
>>> "pkts_dropped": 0,
>>> "pkts_proc": 10232397,
>>> "path": "/var/log/bro/logs/current/stats.log",
>>> "pkts_link": 10232664,
>>> "udp_conns": 21084,
>>> "reassem_unknown_size": 0,
>>> "@timestamp": "2018-10-16T09:15:32.648Z",
>>> "pkt_lag": 0.007681,
>>> "active_dns_requests": 0,
>>> "reassem_tcp_size": 863992,
>>> "tags": [
>>> "bro",
>>> "bro_stats"
>>> ],
>>> "active_udp_conns": 2207,
>>> "tcp_conns": 27070,
>>> "@version": "1",
>>> "bytes_recv": 6580937768
>>> }
>>> /misc/capture_loss:
>>>
>>> "_source": {
>>> "gaps": 92247,
>>> "peer": "worker-1-1",
>>> "path": "/var/log/bro/logs/current/capture_loss.log",
>>> "ts_delta": 900.000031,
>>> "@timestamp": "2018-10-16T09:15:32.632Z",
>>> "percent_lost": 2.053046,
>>> "tags": [
>>> "bro",
>>> "bro_stats",
>>> "bro_capture_loss"
>>> ],
>>> "@version": "1",
>>> "host": "logstash",
>>> "acks": 4493178
>>> }
>>> By reading the documentation It looks like the switch SPAN port or the network interface is dropping packets since bro stats doesn’t register any packet drops.
>>> I’ve checked on the switch and it doesn’t report any dropped traffic.
>>>
>>> Is this possible that the network interface of our server is dropping? Is there a way to analyze the problem further?
>>>
>>> --
>>> Federico Foschini.
>>> _______________________________________________
>>> Bro mailing list
>>> bro at bro-ids.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
> --
> Federico Foschini.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20181017/e042107f/attachment.html
More information about the Bro
mailing list