[Bro] Bro Digest, Vol 127, Issue 2
John Edwards
jedwards2728 at gmail.com
Tue Nov 1 12:03:41 PDT 2016
Ive just configured it so I will see how the logging performs now. I was
basing my information of saving space from here
https://github.com/jahshuah/splunk-ta-bro-json/blob/master/README.md
Cheers
John
On Wednesday, 2 November 2016, <bro-request at bro.org> wrote:
> Send Bro mailing list submissions to
> bro at bro.org <javascript:;>
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> or, via email, send a message with subject or body 'help' to
> bro-request at bro.org <javascript:;>
>
> You can reach the person managing the list at
> bro-owner at bro.org <javascript:;>
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Bro digest..."
>
>
> Today's Topics:
>
> 1. Re: Bro Digest, Vol 126, Issue 56 (Azoff, Justin S)
> 2. Re: extract smtp objects (Seth Hall)
> 3. Re: extract smtp objects (erik clark)
> 4. Re: Have a cluster infrastructure read pcaps (Seth Hall)
> 5. Re: extract smtp objects (Seth Hall)
> 6. Convert integer to string (Chen Xu)
> 7. accept failed, Too many open files 24 (Matt Clemons)
> 8. Re: Convert integer to string (Troy Jordan)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 1 Nov 2016 13:25:02 +0000
> From: "Azoff, Justin S" <jazoff at illinois.edu <javascript:;>>
> Subject: Re: [Bro] Bro Digest, Vol 126, Issue 56
> To: John Edwards <jedwards2728 at gmail.com <javascript:;>>
> Cc: "bro at bro.org <javascript:;>" <bro at bro.org <javascript:;>>
> Message-ID: <867626FB-6BC7-4339-A883-333A0DAA9D89 at illinois.edu
> <javascript:;>>
> Content-Type: text/plain; charset="us-ascii"
>
>
> > On Nov 1, 2016, at 6:49 AM, John Edwards <jedwards2728 at gmail.com
> <javascript:;>> wrote:
> >
> > ASCII had a 3:1 compression ratio of inflation! so json is much more
> efficient use of space
>
> The json log entries need to include the field names in every record.
> There is no possible way that the json logs are more space efficient.
>
> --
> - Justin Azoff
>
>
>
>
> ------------------------------
>
> Message: 2
> Date: Tue, 1 Nov 2016 10:31:35 -0400
> From: Seth Hall <seth at icir.org <javascript:;>>
> Subject: Re: [Bro] extract smtp objects
> To: erik clark <philosnef at gmail.com <javascript:;>>
> Cc: bro at bro.org <javascript:;>
> Message-ID: <95CD8A71-1E53-4C1B-9A08-ED7901426194 at icir.org <javascript:;>>
> Content-Type: text/plain; charset=us-ascii
>
>
> > On Oct 28, 2016, at 11:25 AM, erik clark <philosnef at gmail.com
> <javascript:;>> wrote:
> >
> > Sorry for the clutter. I did this a different way with extract from file
> analyzer. I will just script some glue with conn.log, smtp.log, and fuid. I
> had originally wanted to scrap the data out of the raw smtp message (and
> would still prefer to do that) with other tools entirely, so if someone has
> a way to do that, that would be fantastic. :)
>
> You are hinting towards a design change that I've wanted to see for quite
> a while where the MIME analyzer would turn into a file analyzer and the
> MIME content carried over SMTP would be fed into the MIME file analyzer.
> This would have the nice side effect of making it simple to extract the
> full MIME message through the normal file extraction channels.
>
> Unfortunately this design change hasn't happened yet and isn't slated for
> the near term.
>
> .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
>
>
> ------------------------------
>
> Message: 3
> Date: Tue, 1 Nov 2016 10:33:48 -0400
> From: erik clark <philosnef at gmail.com <javascript:;>>
> Subject: Re: [Bro] extract smtp objects
> To: Seth Hall <seth at icir.org <javascript:;>>
> Cc: bro at bro.org <javascript:;>
> Message-ID:
> <CAK6atxqBBY4xb57jeM6=
> sz2w5v7qnJVcXPfGd5yJv_u+9TSVbA at mail.gmail.com <javascript:;>>
> Content-Type: text/plain; charset="utf-8"
>
> How difficult would this be to do?
>
> On Tue, Nov 1, 2016 at 10:31 AM, Seth Hall <seth at icir.org <javascript:;>>
> wrote:
>
> >
> > > On Oct 28, 2016, at 11:25 AM, erik clark <philosnef at gmail.com
> <javascript:;>> wrote:
> > >
> > > Sorry for the clutter. I did this a different way with extract from
> file
> > analyzer. I will just script some glue with conn.log, smtp.log, and
> fuid. I
> > had originally wanted to scrap the data out of the raw smtp message (and
> > would still prefer to do that) with other tools entirely, so if someone
> has
> > a way to do that, that would be fantastic. :)
> >
> > You are hinting towards a design change that I've wanted to see for quite
> > a while where the MIME analyzer would turn into a file analyzer and the
> > MIME content carried over SMTP would be fed into the MIME file analyzer.
> > This would have the nice side effect of making it simple to extract the
> > full MIME message through the normal file extraction channels.
> >
> > Unfortunately this design change hasn't happened yet and isn't slated for
> > the near term.
> >
> > .Seth
> >
> > --
> > Seth Hall
> > International Computer Science Institute
> > (Bro) because everyone has a network
> > http://www.bro.org/
> >
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/
> 20161101/370453f2/attachment-0001.html
>
> ------------------------------
>
> Message: 4
> Date: Tue, 1 Nov 2016 10:35:32 -0400
> From: Seth Hall <seth at icir.org <javascript:;>>
> Subject: Re: [Bro] Have a cluster infrastructure read pcaps
> To: william de ping <bill.de.ping at gmail.com <javascript:;>>
> Cc: erik clark <philosnef at gmail.com <javascript:;>>, bro at bro.org
> <javascript:;>
> Message-ID: <F70B1A9D-4106-42C8-99E8-AA21B4453C11 at icir.org <javascript:;>>
> Content-Type: text/plain; charset=us-ascii
>
>
> > On Oct 31, 2016, at 7:34 AM, william de ping <bill.de.ping at gmail.com
> <javascript:;>> wrote:
> >
> > I was hoping for some solution that will keep bro process loaded and
> running and feeding it with pcaps.
> > This way I can at least skip the reoccurring loading process.
>
> You are going to have trouble keeping the logs with the original pcap in
> this case. You could have sessions that cross the pcaps like this....
>
> PCAP 1 -> TCP session establishment
> PCAP 2 -> lots of session data
> PCAP 3 -> TCP session teardown - The conn log entry will be written here!
>
> Your logs won't match up as closely as you'd like and could become very
> confusing. I would argue that this offline packet loading situation is a
> situation that you want to avoid at all costs, but if you have to live
> within that situation, I would argue that you want to keep the Bro
> processes up and treat the sequential files as a stream and don't try to
> tie logs to a particular file.
>
> .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
>
>
> ------------------------------
>
> Message: 5
> Date: Tue, 1 Nov 2016 10:38:58 -0400
> From: Seth Hall <seth at icir.org <javascript:;>>
> Subject: Re: [Bro] extract smtp objects
> To: erik clark <philosnef at gmail.com <javascript:;>>
> Cc: bro at bro.org <javascript:;>
> Message-ID: <ED5B6C66-9613-4150-8915-6B794ABCE07A at icir.org <javascript:;>>
> Content-Type: text/plain; charset=us-ascii
>
>
> > On Nov 1, 2016, at 10:33 AM, erik clark <philosnef at gmail.com
> <javascript:;>> wrote:
> >
> > How difficult would this be to do?
>
> Probably quite a bit of work and maybe 80-90% of it would be in the
> analyzer which is hand written in C++.
>
> .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
>
>
> ------------------------------
>
> Message: 6
> Date: Tue, 1 Nov 2016 11:03:02 -0400
> From: Chen Xu <xuchen890530 at gmail.com <javascript:;>>
> Subject: [Bro] Convert integer to string
> To: bro at bro.org <javascript:;>
> Message-ID:
> <CAJ5Y-2M4rLLH3R3Aq8Yd3C8GfweSwX7Ed=kex+rvOOL6O_=
> 0Yg at mail.gmail.com <javascript:;>>
> Content-Type: text/plain; charset="utf-8"
>
> Hello all,
>
> I am new to bro. I have a simple question. Is there any function which can
> convert integer to string?
>
> Thanks,
>
> Chen
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/
> 20161101/2c41e582/attachment-0001.html
>
> ------------------------------
>
> Message: 7
> Date: Tue, 1 Nov 2016 10:06:09 -0500
> From: Matt Clemons <matt.clemons at gmail.com <javascript:;>>
> Subject: [Bro] accept failed, Too many open files 24
> To: "bro at bro.org <javascript:;>" <bro at bro.org <javascript:;>>
> Message-ID:
> <
> CANiyPJazU9SQP2HED5aSFAP5M-wH8F2QEX25tOdAvoOyCFV38g at mail.gmail.com
> <javascript:;>>
> Content-Type: text/plain; charset="utf-8"
>
> Lo All,
>
> Started receiving this error after adding a worker yesterday. If I remove
> the worker and deploy, no issues.
>
> Communications.log: "accept failed, Too many open files 24"
>
>
> Running CentOS6. Bro 2.4.1. 17 Physical worker systems. 150 total worker
> processes.
>
> When adding the 18th worker (6 additional worker processes) logs slow to a
> crawl and the communications log is filled with the failure message.
>
> I've experimented with limits.conf and set a high soft and hard limits of
> open files. Also tried doubling the defaults, and many different
> combinations to no avail. Most of these caused bro to hang and stop
> logging. Others had no affect on the problem.
>
> Has anyone had to deal with this issue or have some ideas? Is there some
> hidden setting in bro where I can set open file limits?
>
> --
> Regards,
>
> Matt Clemons
> (816) 200-0789
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/
> 20161101/25788555/attachment-0001.html
>
> ------------------------------
>
> Message: 8
> Date: Tue, 1 Nov 2016 11:08:22 -0400
> From: Troy Jordan <troyj at maine.edu <javascript:;>>
> Subject: Re: [Bro] Convert integer to string
> To: bro at bro.org <javascript:;>
> Message-ID: <1e92e26d-0985-6054-aa2e-e8749df92dd9 at maine.edu <javascript:;>
> >
> Content-Type: text/plain; charset=windows-1252
>
> Chen,
>
> There is a fmt function for formatting strings that will handle signed
> and unsigned integers:
>
> https://www.bro.org/sphinx/scripts/base/bif/bro.bif.bro.html#id-fmt
>
> - Troy
>
>
>
> On 11/1/2016 11:03 AM, Chen Xu wrote:
> > Hello all,
> >
> > I am new to bro. I have a simple question. Is there any function which
> > can convert integer to string?
> >
> > Thanks,
> >
> > Chen
> >
> >
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org <javascript:;>
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> --
>
>
> Troy Jordan
> t r o y j @ m a i n e . e d u
> GIAC GCIH,GCIA
> ------------------------------------------------------------
> Network Systems Security Analyst
> Information Technology Security Office
> University of Maine System
> ------------------------------------------------------------
> 233 Science Building | voice: 207.561.3590
> Portland, ME 04103 | fax: 509.351.3650
>
>
>
> "As you all know, Security Is Mortals chiefest Enemy"
> William Shakespeare, Macbeth
>
>
>
> ------------------------------
>
> _______________________________________________
> Bro mailing list
> Bro at bro.org <javascript:;>
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
> End of Bro Digest, Vol 127, Issue 2
> ***********************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20161102/6e364c91/attachment-0001.html
More information about the Bro
mailing list