From bro at pingtrip.com  Wed Feb  1 05:05:04 2017
From: bro at pingtrip.com (Dave Crawford)
Date: Wed, 1 Feb 2017 08:05:04 -0500
Subject: [Bro] Converting Notice::Info to JSON
In-Reply-To: <93400B68-43ED-4070-9181-C1677EB6C2A1@pingtrip.com>
References: <8084C305-D1BE-4E68-8A21-684F7A4E50A5@pingtrip.com>
	<B40655FC-046C-42CD-A2F2-27F363C21A62@illinois.edu>
	<93400B68-43ED-4070-9181-C1677EB6C2A1@pingtrip.com>
Message-ID: <C526C6BB-9CD3-4632-A374-57E6394A6522@pingtrip.com>


> On Jan 31, 2017, at 3:39 PM, Dave Crawford <bro at pingtrip.com> wrote:
> 
> 
>> On Jan 31, 2017, at 11:16 AM, Azoff, Justin S <jazoff at illinois.edu> wrote:
>> 
>> You could probably avoid the whole issue by using to_json like this:
>> 
>>   to_json(note, T);
>> 
>> to set the only_loggable option to true which should cause it to ignore fields that aren't normally logged in the first place.
>> 
>> -- 
>> - Justin Azoff
>> 
> 

A follow-up question on to_json() is if the function is always producing valid JSON? As an example:

to_json(n, T)

Produces a few field values that aren?t properly quoted, or in the case of Booleans, not converting T/F to true/false: 

{"proto": tcp, "peer_descr": "bro", "id": {"resp_h": "199.192.156.134", "resp_p": 443, "orig_h": "10.0.2.15", "orig_p": 1381}, "dst": "199.192.156.134", "p": 443, "sub": "POST /bbs/info.asp HTTP/1.1\\x0d\\x0aHost: 199.192.156.134:443\\x0d\\x0aContent-Length: 165\\x0d\\x0aConnection: Keep-Alive\\x0d\\x0aCache-Control: no-cache\\x0d\\x0a\\x0d\\x0a3D333531501A...", "suppress_for": "", "src": "10.0.2.15", "msg": "10.0.2.15: ATTACK-RESPONSES Microsoft cmd.exe banner (reverse-shell originator)", "note": Signatures::Sensitive_Signature, "ts": 1485952936.47094, "uid": "CZwEv13Gadjmnaf6W6", "dropped": F, "actions": [Notice::ACTION_LOG, Phantom::ACTION_TEST]}

-Dave


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170201/4f466084/attachment-0001.html 

From robin at icir.org  Wed Feb  1 07:51:57 2017
From: robin at icir.org (Robin Sommer)
Date: Wed, 1 Feb 2017 07:51:57 -0800
Subject: [Bro] Converting Notice::Info to JSON
In-Reply-To: <C526C6BB-9CD3-4632-A374-57E6394A6522@pingtrip.com>
References: <8084C305-D1BE-4E68-8A21-684F7A4E50A5@pingtrip.com>
	<B40655FC-046C-42CD-A2F2-27F363C21A62@illinois.edu>
	<93400B68-43ED-4070-9181-C1677EB6C2A1@pingtrip.com>
	<C526C6BB-9CD3-4632-A374-57E6394A6522@pingtrip.com>
Message-ID: <20170201155157.GE783@icir.org>



On Wed, Feb 01, 2017 at 08:05 -0500, Dave Crawford wrote:

> to_json(n, T)

> Produces a few field values that aren?t properly quoted, or in the
> case of Booleans, not converting T/F to true/false:

That sounds like a bug to me: if it says JSON, it should be JSON.
Please file a ticket for that.

Robin


-- 
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin

From jazoff at illinois.edu  Wed Feb  1 08:29:26 2017
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Wed, 1 Feb 2017 16:29:26 +0000
Subject: [Bro] Logging and memory leak
In-Reply-To: <CAJ19UZDJwoGq9RGKYZ+OftCeS+AOqxyfmAbV_3PHmghfWUEFFA@mail.gmail.com>
References: <CAJ19UZBxxOEZqmT2MYMBVsjno3mimNjN6egRWv0h5A-Sp_bd_Q@mail.gmail.com>
	<A7503EE4-D454-4404-9239-DA3C8A6D559A@illinois.edu>
	<CAJ19UZDJwoGq9RGKYZ+OftCeS+AOqxyfmAbV_3PHmghfWUEFFA@mail.gmail.com>
Message-ID: <F1308A75-9240-4DEB-88C4-B1ED8F9CD5F5@illinois.edu>


> On Jan 31, 2017, at 7:36 PM, Hovsep Levi <hovsep.sanjay.levi at gmail.com> wrote:
> 
> No, both are disabled.
> 

Do you have any other custom scripts loaded that are using sumstats?

With a dedicated logger process the manager doesn't really do anything other than sumstats.

Look in your cluster-layout.bro to see what port your manager process is assigned.. with 4 loggers I'd imagine it is around 47765/tcp

Then, run this command on the manager, on the interface that it talks to workers:

    tcpdump  -n -i em1 port 47765 -A | egrep -io '[A-Za-z_:-]{10,}'

That will output the names of the events that are bouncing between the workers and the manager

And see what you see..  It SHOULD be almost nothing, maybe a trickle of events.



-- 
- Justin Azoff



From philosnef at gmail.com  Thu Feb  2 05:20:03 2017
From: philosnef at gmail.com (erik clark)
Date: Thu, 2 Feb 2017 08:20:03 -0500
Subject: [Bro] branching Bro
Message-ID: <CAK6atxrnAt173uOSSidbGM5b_nNqg4aRE=ctJKVhD6e1hUs=tA@mail.gmail.com>

We need to branch Bro due to FIPS non-compliance. Is there any thorough
documentation as to what parts are FIPS non-compliant, other than the md5
analyzer, and is there any documentation on the md5 analyzer hooks (built
in stock scripts, so on so forth) so we can begin attempting to branch this
into a FIPS compliant tool?

Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170202/47792c99/attachment.html 

From jazoff at illinois.edu  Thu Feb  2 05:51:47 2017
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Thu, 2 Feb 2017 13:51:47 +0000
Subject: [Bro] branching Bro
In-Reply-To: <CAK6atxrnAt173uOSSidbGM5b_nNqg4aRE=ctJKVhD6e1hUs=tA@mail.gmail.com>
References: <CAK6atxrnAt173uOSSidbGM5b_nNqg4aRE=ctJKVhD6e1hUs=tA@mail.gmail.com>
Message-ID: <878EBD62-C217-4550-AD73-752112A62CA3@illinois.edu>


> On Feb 2, 2017, at 8:20 AM, erik clark <philosnef at gmail.com> wrote:
> 
> We need to branch Bro due to FIPS non-compliance. Is there any thorough documentation as to what parts are FIPS non-compliant, other than the md5 analyzer, and is there any documentation on the md5 analyzer hooks (built in stock scripts, so on so forth) so we can begin attempting to branch this into a FIPS compliant tool?
> 
> Thanks.

I hereby grant you an exception to FIPS compliance that allows you to use Bro as it is intended and to its full capabilities.

Note: This exception is equally as meaningless as FIPS compliance.

-- 
- Justin Azoff



From philosnef at gmail.com  Thu Feb  2 05:53:55 2017
From: philosnef at gmail.com (erik clark)
Date: Thu, 2 Feb 2017 08:53:55 -0500
Subject: [Bro] branching Bro
In-Reply-To: <878EBD62-C217-4550-AD73-752112A62CA3@illinois.edu>
References: <CAK6atxrnAt173uOSSidbGM5b_nNqg4aRE=ctJKVhD6e1hUs=tA@mail.gmail.com>
	<878EBD62-C217-4550-AD73-752112A62CA3@illinois.edu>
Message-ID: <CAK6atxo7RqrcokSp-fFNe1kVSX0O_niLKnaUFsh9ZYKX3wxpag@mail.gmail.com>

Sadly, in the federal world, FIPS compliance isn't meaningless. There is a
real need for it.

On Thu, Feb 2, 2017 at 8:51 AM, Azoff, Justin S <jazoff at illinois.edu> wrote:

>
> > On Feb 2, 2017, at 8:20 AM, erik clark <philosnef at gmail.com> wrote:
> >
> > We need to branch Bro due to FIPS non-compliance. Is there any thorough
> documentation as to what parts are FIPS non-compliant, other than the md5
> analyzer, and is there any documentation on the md5 analyzer hooks (built
> in stock scripts, so on so forth) so we can begin attempting to branch this
> into a FIPS compliant tool?
> >
> > Thanks.
>
> I hereby grant you an exception to FIPS compliance that allows you to use
> Bro as it is intended and to its full capabilities.
>
> Note: This exception is equally as meaningless as FIPS compliance.
>
> --
> - Justin Azoff
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170202/f6f75e17/attachment.html 

From jazoff at illinois.edu  Thu Feb  2 05:59:36 2017
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Thu, 2 Feb 2017 13:59:36 +0000
Subject: [Bro] branching Bro
In-Reply-To: <CAK6atxo7RqrcokSp-fFNe1kVSX0O_niLKnaUFsh9ZYKX3wxpag@mail.gmail.com>
References: <CAK6atxrnAt173uOSSidbGM5b_nNqg4aRE=ctJKVhD6e1hUs=tA@mail.gmail.com>
	<878EBD62-C217-4550-AD73-752112A62CA3@illinois.edu>
	<CAK6atxo7RqrcokSp-fFNe1kVSX0O_niLKnaUFsh9ZYKX3wxpag@mail.gmail.com>
Message-ID: <9263D96B-A693-4EE5-9263-75BE625665BC@illinois.edu>


> On Feb 2, 2017, at 8:53 AM, erik clark <philosnef at gmail.com> wrote:
> 
> Sadly, in the federal world, FIPS compliance isn't meaningless. There is a real need for it. 

And handicapping the best tool you'd have to detect noncompliant certificates is extremely misguided and counterproductive.

It's like if you had a tool that could scan for use of 512bit key SSL certificates, and someone prevented you from using it because it "supports" 512bit certificates and 512bit certificates are not FIPS compliant.

-- 
- Justin Azoff



From hosom at battelle.org  Thu Feb  2 06:30:42 2017
From: hosom at battelle.org (Hosom, Stephen M)
Date: Thu, 2 Feb 2017 14:30:42 +0000
Subject: [Bro] branching Bro
In-Reply-To: <CAK6atxo7RqrcokSp-fFNe1kVSX0O_niLKnaUFsh9ZYKX3wxpag@mail.gmail.com>
References: <CAK6atxrnAt173uOSSidbGM5b_nNqg4aRE=ctJKVhD6e1hUs=tA@mail.gmail.com>
	<878EBD62-C217-4550-AD73-752112A62CA3@illinois.edu>
	<CAK6atxo7RqrcokSp-fFNe1kVSX0O_niLKnaUFsh9ZYKX3wxpag@mail.gmail.com>
Message-ID: <E5A2071C6FFFF640B714C224139C9B3327E2CE72@WP-MBX2B.milky-way.battelle.org>

FIPS compliance isn?t an application ?thing?. FIPS compliance is an implementation ?thing?.

FIPS does not apply to the md5 file hash analyzer because in this case no cryptography is being performed.  FIPS also does not apply to the calls in the SSL analysis stuff to md5 because once again, no cryptography occurs with the calculation of that MD5 hash. I don?t remember which portion of Bro ultimately caused this issue for you, but whether it was the file analyzer or the SSL analyzer?I am not aware of segments of either of those analyzers performing any cryptography.

FIPS is a process by which the government assesses implementations of cryptography. Is your question ?Where is cryptography implemented in Bro so that I review those sections to determine whether or not they would pass a FIPS validation assessment?? To the best of my knowledge, cryptography isn?t implemented anywhere in Bro.

I suspect that what you really want is for the communication that occurs when network traffic leaves the server to be encrypted in a manner that would pass FIPS 140 validation tests. My recommendation for that would be to run the cluster traffic in an isolated network and document that in your system security plan. Not only is this the best scenario for you, this is almost always how the most active people on this mailing list would deploy Bro. If what is really going on here is that a STIG check is ?requiring? you to run everything in FIPS mode, then documenting an exception is usually alright.

If it is unacceptable for your audit that you document an exception (which would be a new one for me since the government that you are working with generally only wants data encrypted when it leaves the system), then you should follow Vlad?s advice about setting up tunnels between cluster nodes and the master. This implementation (assuming that you perform the configuration right) has the ability to pass FIPS-140 validation.

Branching Bro and rewriting all of its cluster communications to use encryption that you write yourself from scratch is almost certainly going to be a bad idea. You?re unlikely to pass FIPS validation writing something yourself. While I don?t doubt your skills, FIPS validation is something that development teams strive for and a developer would likely struggle with. You will also be introducing an immense amount of risk. You have the risk that you?ll almost certainly never be able to pull down updates from the origin?since you?re working with completely different clustering code. You?re going to have the risk that your version of Bro?which I?m assuming will be closed source?will now only ever be reviewed by you or your team. Your implementation will likely be significantly weaker because of this.

For those watching: please feel free to correct any information you feel that I have misrepresented. I would normally include sources and double check everything in an email this lengthy, but I have to rush off and want to make an attempt at being helpful here.

From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of erik clark
Sent: Thursday, February 2, 2017 8:54 AM
To: Azoff, Justin S <jazoff at illinois.edu>
Cc: Bro-IDS <bro at bro.org>
Subject: Re: [Bro] branching Bro

Sadly, in the federal world, FIPS compliance isn't meaningless. There is a real need for it.

On Thu, Feb 2, 2017 at 8:51 AM, Azoff, Justin S <jazoff at illinois.edu<mailto:jazoff at illinois.edu>> wrote:

> On Feb 2, 2017, at 8:20 AM, erik clark <philosnef at gmail.com<mailto:philosnef at gmail.com>> wrote:
>
> We need to branch Bro due to FIPS non-compliance. Is there any thorough documentation as to what parts are FIPS non-compliant, other than the md5 analyzer, and is there any documentation on the md5 analyzer hooks (built in stock scripts, so on so forth) so we can begin attempting to branch this into a FIPS compliant tool?
>
> Thanks.

I hereby grant you an exception to FIPS compliance that allows you to use Bro as it is intended and to its full capabilities.

Note: This exception is equally as meaningless as FIPS compliance.

--
- Justin Azoff

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170202/01abeb14/attachment-0001.html 

From mfernandez at mitre.org  Thu Feb  2 06:37:35 2017
From: mfernandez at mitre.org (Fernandez, Mark I)
Date: Thu, 2 Feb 2017 14:37:35 +0000
Subject: [Bro] branching Bro
In-Reply-To: <9263D96B-A693-4EE5-9263-75BE625665BC@illinois.edu>
References: <CAK6atxrnAt173uOSSidbGM5b_nNqg4aRE=ctJKVhD6e1hUs=tA@mail.gmail.com>
	<878EBD62-C217-4550-AD73-752112A62CA3@illinois.edu>
	<CAK6atxo7RqrcokSp-fFNe1kVSX0O_niLKnaUFsh9ZYKX3wxpag@mail.gmail.com>
	<9263D96B-A693-4EE5-9263-75BE625665BC@illinois.edu>
Message-ID: <MWHPR09MB1533B4525E5F07A0E8AE6AC5CF4C0@MWHPR09MB1533.namprd09.prod.outlook.com>

Erik, Justin -

You both have good points.  For Erik, I think you have solid ground on which to stand if you make the following distinctions:

	(a) Bro is capable of inspecting/monitoring/detecting FIPS non-compliant encryption; this is a valid and necessary capability for the defense and security of your network; and

	(b) Is Bro being used to PROTECT federal information (whether in transit or at rest)?  If not, then no worries, argument alleviated.  But if so, then is Bro able to implement a FIPS-compliant encryption to do so?

As long as Bro uses FIPS-compliant encryption to PROTECT information (or if you can come up with an appropriate mitigation), then I believe you can make a reasonable case to your certification and accreditation folks to allow Bro to also continue monitoring for non-compliance.

Cheers!
Mark Fernandez


-----Original Message-----
From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of Azoff, Justin S
Sent: Thursday, February 02, 2017 9:00 AM
To: erik clark <philosnef at gmail.com>
Cc: Bro-IDS <bro at bro.org>
Subject: Re: [Bro] branching Bro


> On Feb 2, 2017, at 8:53 AM, erik clark <philosnef at gmail.com> wrote:
> 
> Sadly, in the federal world, FIPS compliance isn't meaningless. There is a real need for it. 

And handicapping the best tool you'd have to detect noncompliant certificates is extremely misguided and counterproductive.

It's like if you had a tool that could scan for use of 512bit key SSL certificates, and someone prevented you from using it because it "supports" 512bit certificates and 512bit certificates are not FIPS compliant.

-- 
- Justin Azoff


_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From philosnef at gmail.com  Thu Feb  2 06:46:48 2017
From: philosnef at gmail.com (erik clark)
Date: Thu, 2 Feb 2017 09:46:48 -0500
Subject: [Bro] branching Bro
In-Reply-To: <MWHPR09MB1533B4525E5F07A0E8AE6AC5CF4C0@MWHPR09MB1533.namprd09.prod.outlook.com>
References: <CAK6atxrnAt173uOSSidbGM5b_nNqg4aRE=ctJKVhD6e1hUs=tA@mail.gmail.com>
	<878EBD62-C217-4550-AD73-752112A62CA3@illinois.edu>
	<CAK6atxo7RqrcokSp-fFNe1kVSX0O_niLKnaUFsh9ZYKX3wxpag@mail.gmail.com>
	<9263D96B-A693-4EE5-9263-75BE625665BC@illinois.edu>
	<MWHPR09MB1533B4525E5F07A0E8AE6AC5CF4C0@MWHPR09MB1533.namprd09.prod.outlook.com>
Message-ID: <CAK6atxoa187jEhyhfBXWG4sP4i=ie4+C43sowBkiG1Jt=RR_iQ@mail.gmail.com>

Stephen, Bro flat out does not run if your kernel is in fips mode. You
specifically get:

ValueError: error:060800A3: digitial envelope
routines:EVP_DigestInit_ex:disabled for fips


I brought up the cross network logging encryption issue previously. This is
very specifically an issue where you can not run Bro at all with a FIPS
compliant kernel. Getting someone to sign off on an exception not only for
Bro, but the kernel as well, is unlikely. The issue with the md5 crypto
libs in Bro causing it to simply not run with a FIPS kernel was already
brought up in the list as well, by Gary.


On Thu, Feb 2, 2017 at 9:37 AM, Fernandez, Mark I <mfernandez at mitre.org>
wrote:

> Erik, Justin -
>
> You both have good points.  For Erik, I think you have solid ground on
> which to stand if you make the following distinctions:
>
>         (a) Bro is capable of inspecting/monitoring/detecting FIPS
> non-compliant encryption; this is a valid and necessary capability for the
> defense and security of your network; and
>
>         (b) Is Bro being used to PROTECT federal information (whether in
> transit or at rest)?  If not, then no worries, argument alleviated.  But if
> so, then is Bro able to implement a FIPS-compliant encryption to do so?
>
> As long as Bro uses FIPS-compliant encryption to PROTECT information (or
> if you can come up with an appropriate mitigation), then I believe you can
> make a reasonable case to your certification and accreditation folks to
> allow Bro to also continue monitoring for non-compliance.
>
> Cheers!
> Mark Fernandez
>
>
> -----Original Message-----
> From: bro-bounces at bro.org [mailto:bro-bounces at bro.org] On Behalf Of
> Azoff, Justin S
> Sent: Thursday, February 02, 2017 9:00 AM
> To: erik clark <philosnef at gmail.com>
> Cc: Bro-IDS <bro at bro.org>
> Subject: Re: [Bro] branching Bro
>
>
> > On Feb 2, 2017, at 8:53 AM, erik clark <philosnef at gmail.com> wrote:
> >
> > Sadly, in the federal world, FIPS compliance isn't meaningless. There is
> a real need for it.
>
> And handicapping the best tool you'd have to detect noncompliant
> certificates is extremely misguided and counterproductive.
>
> It's like if you had a tool that could scan for use of 512bit key SSL
> certificates, and someone prevented you from using it because it "supports"
> 512bit certificates and 512bit certificates are not FIPS compliant.
>
> --
> - Justin Azoff
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170202/b13b2917/attachment.html 

From bro at pingtrip.com  Thu Feb  2 09:09:10 2017
From: bro at pingtrip.com (Dave Crawford)
Date: Thu, 2 Feb 2017 12:09:10 -0500
Subject: [Bro] Converting Notice::Info to JSON
In-Reply-To: <20170201155157.GE783@icir.org>
References: <8084C305-D1BE-4E68-8A21-684F7A4E50A5@pingtrip.com>
	<B40655FC-046C-42CD-A2F2-27F363C21A62@illinois.edu>
	<93400B68-43ED-4070-9181-C1677EB6C2A1@pingtrip.com>
	<C526C6BB-9CD3-4632-A374-57E6394A6522@pingtrip.com>
	<20170201155157.GE783@icir.org>
Message-ID: <F44F55AE-D8B8-4B3C-BDAC-64BE97CADE3B@pingtrip.com>


> On Feb 1, 2017, at 10:51 AM, Robin Sommer <robin at icir.org> wrote:
> 
> 
> 
> On Wed, Feb 01, 2017 at 08:05 -0500, Dave Crawford wrote:
> 
>> to_json(n, T)
> 
>> Produces a few field values that aren?t properly quoted, or in the
>> case of Booleans, not converting T/F to true/false:
> 
> That sounds like a bug to me: if it says JSON, it should be JSON.
> Please file a ticket for that.
> 
> Robin
> 
> 
> -- 
> Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin



As requested. BIT-1789 <https://bro-tracker.atlassian.net/browse/BIT-1789>

-Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170202/906cd322/attachment.html 

From dnthayer at illinois.edu  Thu Feb  2 12:09:01 2017
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Thu, 2 Feb 2017 14:09:01 -0600
Subject: [Bro] Converting Notice::Info to JSON
In-Reply-To: <F44F55AE-D8B8-4B3C-BDAC-64BE97CADE3B@pingtrip.com>
References: <8084C305-D1BE-4E68-8A21-684F7A4E50A5@pingtrip.com>
	<B40655FC-046C-42CD-A2F2-27F363C21A62@illinois.edu>
	<93400B68-43ED-4070-9181-C1677EB6C2A1@pingtrip.com>
	<C526C6BB-9CD3-4632-A374-57E6394A6522@pingtrip.com>
	<20170201155157.GE783@icir.org>
	<F44F55AE-D8B8-4B3C-BDAC-64BE97CADE3B@pingtrip.com>
Message-ID: <af235709-f995-7177-498d-88d4175a2f0b@illinois.edu>

On 2/2/17 11:09 AM, Dave Crawford wrote:
>
>> On Feb 1, 2017, at 10:51 AM, Robin Sommer <robin at icir.org
>> <mailto:robin at icir.org>> wrote:
>>
>>
>>
>> On Wed, Feb 01, 2017 at 08:05 -0500, Dave Crawford wrote:
>>
>>> to_json(n, T)
>>
>>> Produces a few field values that aren?t properly quoted, or in the
>>> case of Booleans, not converting T/F to true/false:
>>
>> That sounds like a bug to me: if it says JSON, it should be JSON.
>> Please file a ticket for that.
>>
>> Robin
>>
>>
>
> As requested. BIT-1789
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__bro-2Dtracker.atlassian.net_browse_BIT-2D1789&d=DwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=Bi5qPBnY0NmYPqnRTPj_AfXQKpfQTZUpCzpfFBcawv0&m=Os4JPZ3kmh0fE2dy0BXrW0ApmeHRvm2oWS5FfNv50Kw&s=lVNlWmu5PlaLPWBPJ6pCK4br6xQhzxyk5BmngcyPZw8&e=>
>
> -Dave

I've fixed this issue (see 
https://bro-tracker.atlassian.net/browse/BIT-1788).

-Daniel


From bro at pingtrip.com  Thu Feb  2 12:17:37 2017
From: bro at pingtrip.com (Dave Crawford)
Date: Thu, 2 Feb 2017 15:17:37 -0500
Subject: [Bro] Converting Notice::Info to JSON
In-Reply-To: <af235709-f995-7177-498d-88d4175a2f0b@illinois.edu>
References: <8084C305-D1BE-4E68-8A21-684F7A4E50A5@pingtrip.com>
	<B40655FC-046C-42CD-A2F2-27F363C21A62@illinois.edu>
	<93400B68-43ED-4070-9181-C1677EB6C2A1@pingtrip.com>
	<C526C6BB-9CD3-4632-A374-57E6394A6522@pingtrip.com>
	<20170201155157.GE783@icir.org>
	<F44F55AE-D8B8-4B3C-BDAC-64BE97CADE3B@pingtrip.com>
	<af235709-f995-7177-498d-88d4175a2f0b@illinois.edu>
Message-ID: <FEB8143F-D370-499A-9CC6-121EF696BBD1@pingtrip.com>


> On Feb 2, 2017, at 3:09 PM, Daniel Thayer <dnthayer at illinois.edu> wrote:
> 
> On 2/2/17 11:09 AM, Dave Crawford wrote:
>> 
>>> On Feb 1, 2017, at 10:51 AM, Robin Sommer <robin at icir.org
>>> <mailto:robin at icir.org>> wrote:
>>> 
>>> 
>>> 
>>> On Wed, Feb 01, 2017 at 08:05 -0500, Dave Crawford wrote:
>>> 
>>>> to_json(n, T)
>>> 
>>>> Produces a few field values that aren?t properly quoted, or in the
>>>> case of Booleans, not converting T/F to true/false:
>>> 
>>> That sounds like a bug to me: if it says JSON, it should be JSON.
>>> Please file a ticket for that.
>>> 
>>> Robin
>>> 
>>> 
>> 
>> As requested. BIT-1789
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__bro-2Dtracker.atlassian.net_browse_BIT-2D1789&d=DwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=Bi5qPBnY0NmYPqnRTPj_AfXQKpfQTZUpCzpfFBcawv0&m=Os4JPZ3kmh0fE2dy0BXrW0ApmeHRvm2oWS5FfNv50Kw&s=lVNlWmu5PlaLPWBPJ6pCK4br6xQhzxyk5BmngcyPZw8&e=>
>> 
>> -Dave
> 
> I've fixed this issue (see https://bro-tracker.atlassian.net/browse/BIT-1788).
> 
> -Daniel
> 

Awesome, thanks Daniel. I closed my ticket as a duplicate and referenced your ticket.

-Dave

From albertociolini92 at gmail.com  Fri Feb  3 06:14:20 2017
From: albertociolini92 at gmail.com (Alberto Ciolini)
Date: Fri, 3 Feb 2017 15:14:20 +0100
Subject: [Bro]  How to stop script and send email
Message-ID: <CA+wdK6v-pjdWjDQFZobsf3TQyJ7F_tvvgM+zNi4ey1i4BE-g1A@mail.gmail.com>

Hi everyone.
I wrote a simple script Bro that print a log file after read event from
broccoli. I would like that the script stop automatically at the end and
not with ctrl-c on keyboard. Is there a way for this?
And another simple question: how send email with bro?

Thanks,
Alberto.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170203/661db8f6/attachment.html 

From twaller at bivio.net  Fri Feb  3 12:27:25 2017
From: twaller at bivio.net (Tony Waller)
Date: Fri, 3 Feb 2017 20:27:25 +0000
Subject: [Bro] Information on OCSP and CRL
Message-ID: <EE2D32CF-B9DE-433F-9A01-D79001D1B33B@bivio.net>

I am looking for additional information on utilizing OCSP and CRL in Bro NSM. I would like to know if certificates from clients can be checked in real-time or near real-time against a CRL? Also, can Bro NSM perform a OCSP request to a RA and check a certificate to determine if it is valid? If this is the case where in Bro NSM do you set the address for the RA or CRL responder?

Sincerely,

Tony


Tony Waller, CISSP
Director, Systems Engineering
Bivio Networks, Inc.
?Powering Advanced Cyber Operations? (TM)
Mobile (443) 994-0936

[cid:image001.png at 01D27E32.049899E0]

*Note: The information contained in this email confidential. This information is intended only for the individual, individuals or entity to whom it is addressed. If you are not the intended recipient(s), the employee or agent responsible for delivering it to the intended recipient(s), you are hereby notified that any use, dissemination, distribution or copying of this communication is strictly prohibited. If you have received this email in error, please return the original message to us by email and delete or destroy any copies. Please note any views or opinions expressed or presented in this email are solely those of the author and do not necessarily represent those of Bivio Networks, Inc. The recipient should check this email or any attachments for the presence of viruses or malware. Bivio Networks, Inc. accepts no responsibility for any damage caused by any virus or malware transmitted by this email. Thank you.

Think Green when printing

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170203/535bf03b/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 8127 bytes
Desc: image001.png
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170203/535bf03b/attachment.bin 

From hovsep.sanjay.levi at gmail.com  Fri Feb  3 16:48:39 2017
From: hovsep.sanjay.levi at gmail.com (Hovsep Levi)
Date: Sat, 4 Feb 2017 00:48:39 +0000
Subject: [Bro] Logging and memory leak
In-Reply-To: <F1308A75-9240-4DEB-88C4-B1ED8F9CD5F5@illinois.edu>
References: <CAJ19UZBxxOEZqmT2MYMBVsjno3mimNjN6egRWv0h5A-Sp_bd_Q@mail.gmail.com>
	<A7503EE4-D454-4404-9239-DA3C8A6D559A@illinois.edu>
	<CAJ19UZDJwoGq9RGKYZ+OftCeS+AOqxyfmAbV_3PHmghfWUEFFA@mail.gmail.com>
	<F1308A75-9240-4DEB-88C4-B1ED8F9CD5F5@illinois.edu>
Message-ID: <CAJ19UZD+0ZR3W2WYMPc8ELLTTKrkYPtyJ0J4EnWunMLeF6bhdg@mail.gmail.com>

No, no custom scripts.  I found the cluster overwhelmed again today with
massive virtual memory usage and an hour after restarting it the same
condition returns.

I'm using a single logger since the last time.  It seems when using a Kafka
only export a single logger works fine as the event timestamps arriving at
Kafka are near realtime.  (the "ts" for conn, http, etc.)

The logger is using 47761, manager is 47762.  I took a sample of 5000
packets for each during the high memory usage and it looks like the manager
is still receiving logs of some sort dealing with x509 certificates.


tcpdump -A -r Bro__Manager_port_47662.pcap | egrep -io '[A-Za-z_:-]{10,}' |
sort | uniq -c | sort -rn | head -10
reading from file Bro__Manager_port_47662.pcap, link-type EN10MB (Ethernet)
2852 certificate
2064 Notice::begin_suppressionA
1800 Conn::IN_ORIG
 739 Authentication
 685 Identifier
 681 Encipherment
 660 validation
 646 Corporation



Config:

event bro_init()
        Analyzer::disable_analyzer(Analyzer::ANALYZER_SYSLOG);
@load misc/loaded-scripts
@load tuning/defaults
@load misc/stats
@load misc/capture-loss
@load frameworks/software/vulnerable
@load frameworks/software/version-changes
@load-sigs frameworks/signatures/detect-windows-shells
@load protocols/ftp/software
@load protocols/smtp/software
@load protocols/ssh/software
@load protocols/http/software
@load protocols/dns/detect-external-names
@load protocols/ftp/detect
@load protocols/conn/known-hosts
@load protocols/conn/known-services
@load protocols/ssl/known-certs
@load protocols/ssl/validate-certs
@load protocols/ssl/log-hostcerts-only
@load protocols/ssh/geo-data
@load protocols/ssh/detect-bruteforcing
@load protocols/ssh/interesting-hostnames
@load protocols/http/detect-sqli
@load frameworks/files/hash-all-files
@load frameworks/files/detect-MHR
@load frameworks/intel/seen
@load frameworks/intel/do_notice
@load local-intel.bro
@load Bro/Kafka/logs-to-kafka.bro






Snapshots before the first restart at


@ Wed Feb  1 19:57:28 UTC 2017

[bro at mgr /opt/bro]$ bin/broctl top manager logger-1 && echo "" && date
Name         Type    Host             Pid     Proc    VSize  Rss  Cpu   Cmd
logger-1     logger  10.1.1.1   25469   parent  849M   330M 171%  bro
logger-1     logger  10.1.1.1   25523   child   458M    69M  38%  bro
manager      manager 10.1.1.1   25685   child   494M   261M 100%  bro
manager      manager 10.1.1.1   25543   parent    9G     1G  27%  bro


@ Fri Feb  3 04:05:04 UTC 2017

[bro at mgr /opt/bro]$ bin/broctl top manager logger-1 && echo "" && date
Name         Type    Host             Pid     Proc    VSize  Rss  Cpu   Cmd
logger-1     logger  10.1.1.1   25469   parent  793M   284M  99%  bro
logger-1     logger  10.1.1.1   25523   child   466M    80M  25%  bro
manager      manager 10.1.1.1   25685   child   494M   261M 100%  bro
manager      manager 10.1.1.1   25543   parent    9G     1G  35%  bro


@ Fri Feb  3 21:15:51 UTC 2017

[bro at mgr /opt/bro]$ bin/broctl top manager logger-1 && echo "" && date
Name         Type    Host             Pid     Proc    VSize  Rss  Cpu   Cmd
logger-1     logger  10.1.1.1   25469   parent  813M   316M 178%  bro
logger-1     logger  10.1.1.1   25523   child   466M    83M  38%  bro
manager      manager 10.1.1.1   25685   child     8G     8G 100%  bro
manager      manager 10.1.1.1   25543   parent  1222G   87G  99%  bro





last pid: 33713;  load averages:  7.35,  6.19,
5.69
up 6+20:45:54  21:32:43
49 processes:  4 running, 45 sleeping
CPU:  8.0% user,  0.3% nice,  5.7% system,  0.2% interrupt, 85.8% idle
Mem: 31G Active, 69G Inact, 20G Wired, 1884K Cache, 5167M Free
ARC: 14G Total, 2327M MFU, 11G MRU, 155K Anon, 58M Header, 555M Other
Swap: 12G Total, 17M Used, 12G Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU
COMMAND
25469 bro         172  20    0  1089M   591M select 11 118.3H 258.45% bro
25685 bro           1 108    5  9174M  9035M CPU22  22  63.0H 100.00% bro
25543 bro           7  20    0  1275G 93626M uwait   2  21.1H 100.00% bro
25523 bro           1  92    5   466M 85900K CPU3    3  18.0H  51.27% bro
33713 bro           1  78    0 97192K 17284K CPU32  32   0:02  21.09%
python2.7
33709 bro           1  52    0 97192K 17300K piperd 31   0:01   2.69%
python2.7





I restarted the cluster at 21:42 and see the same behavior almost
immediately, within an hour:

@ Fri Feb  3 22:27:38 UTC 2017

[bro at mgr /opt/bro]$ bin/broctl top manager logger-1 && echo "" && date
Name         Type    Host             Pid     Proc    VSize  Rss  Cpu   Cmd
logger-1     logger  10.1.1.1   37525   parent  647M   124M  15%  bro
logger-1     logger  10.1.1.1   37576   child   458M    65M   2%  bro
manager      manager 10.1.1.1   37653   child   506M   199M 100%  bro
manager      manager 10.1.1.1   37600   parent  250G    18G 100%  bro



@ Sat Feb  4 00:22:01 UTC 2017

[bro at mgr /opt/bro]$ bin/broctl top manager logger-1 && echo "" && date
Name         Type    Host             Pid     Proc    VSize  Rss  Cpu   Cmd
logger-1     logger  10.1.1.1   37525   parent  663M   134M  14%  bro
logger-1     logger  10.1.1.1   37576   child   458M    65M   2%  bro
manager      manager 10.1.1.1   37653   child   506M   237M 100%  bro
manager      manager 10.1.1.1   37600   parent  640G    46G  38%  bro



last pid: 75833;  load averages:  2.31,  2.47,
2.57
up 6+23:49:38  00:36:27
47 processes:  2 running, 45 sleeping
CPU:  1.2% user,  0.2% nice,  2.4% system,  0.1% interrupt, 96.1% idle
Mem: 20G Active, 26G Inact, 22G Wired, 1752K Cache, 57G Free
ARC: 8159M Total, 2242M MFU, 5323M MRU, 64K Anon, 49M Header, 545M Other
Swap: 12G Total, 16M Used, 12G Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU
COMMAND
37653 bro           1 108    5   506M   237M CPU33  33 172:00 100.00% bro
37600 bro           7  20    0   640G 47116M uwait  25 143:23  27.88% bro
37525 bro         162  20    0   663M   134M select 45 106:45  12.60% bro
37576 bro           1  25    5   458M 67404K select  6   5:03   1.46% bro



@ Sat Feb  4 00:38:50 UTC 2017

Name         Type    Host             Pid     Proc    VSize  Rss  Cpu   Cmd
logger-1     logger  10.1.1.1   37525   parent  663M   134M  13%  bro
logger-1     logger  10.1.1.1   37576   child   458M    65M   1%  bro
manager      manager 10.1.1.1   37653   child   506M   237M  99%  bro
manager      manager 10.1.1.1   37600   parent  640G    46G  37%  bro




With pure Kafka export there's very few logs on the manager filesystem.  I
don't think this is related to the manager issue but fwiw there's a steady
stream of kafka errors even though logging seems to be working:

>From /opt/bro_data/logs/current/stderr.log -> (/opt/bro_data/spool/logger-1)

%3|1486168762.913|ERROR|rdkafka#producer-15| 10.1.1.5:9092/bootstrap:
Receive failed: Disconnected
%3|1486168773.922|FAIL|rdkafka#producer-18| 10.1.1.5:9092/bootstrap:
Receive failed: Disconnected
%3|1486168773.922|ERROR|rdkafka#producer-18| 10.1.1.5:9092/bootstrap:
Receive failed: Disconnected
%3|1486168775.662|FAIL|rdkafka#producer-19| 10.1.1.5:9092/bootstrap:
Receive failed: Disconnected
%3|1486168775.663|ERROR|rdkafka#producer-19| 10.1.1.5:9092/bootstrap:
Receive failed: Disconnected
%3|1486168778.818|FAIL|rdkafka#producer-21| 10.1.1.5:9092/bootstrap:
Receive failed: Disconnected
%3|1486168778.819|ERROR|rdkafka#producer-21| 10.1.1.5:9092/bootstrap:
Receive failed: Disconnected
(...)
%3|1486169053.779|ERROR|rdkafka#producer-8| 10.1.1.5:9092/bootstrap:
Receive failed: Disconnected
%3|1486169082.658|FAIL|rdkafka#producer-23| 10.1.1.5:9092/bootstrap:
Receive failed: Disconnected
%3|1486169082.659|ERROR|rdkafka#producer-23| 10.1.1.5:9092/bootstrap:
Receive failed: Disconnected

[bro at mgr /opt/bro_data/logs]$ date -r 1486169082
Sat Feb  4 00:44:42 UTC 2017

[bro at mgr /opt/bro_data/logs]$ date
Sat Feb  4 00:45:08 UTC 2017
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170204/21b01db4/attachment-0001.html 

From jazoff at illinois.edu  Fri Feb  3 20:00:39 2017
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Sat, 4 Feb 2017 04:00:39 +0000
Subject: [Bro] Logging and memory leak
In-Reply-To: <CAJ19UZD+0ZR3W2WYMPc8ELLTTKrkYPtyJ0J4EnWunMLeF6bhdg@mail.gmail.com>
References: <CAJ19UZBxxOEZqmT2MYMBVsjno3mimNjN6egRWv0h5A-Sp_bd_Q@mail.gmail.com>
	<A7503EE4-D454-4404-9239-DA3C8A6D559A@illinois.edu>
	<CAJ19UZDJwoGq9RGKYZ+OftCeS+AOqxyfmAbV_3PHmghfWUEFFA@mail.gmail.com>
	<F1308A75-9240-4DEB-88C4-B1ED8F9CD5F5@illinois.edu>
	<CAJ19UZD+0ZR3W2WYMPc8ELLTTKrkYPtyJ0J4EnWunMLeF6bhdg@mail.gmail.com>
Message-ID: <6B204A64-5F4C-4230-8741-9CDD550EC433@illinois.edu>


> On Feb 3, 2017, at 7:48 PM, Hovsep Levi <hovsep.sanjay.levi at gmail.com> wrote:
> 
> No, no custom scripts.  I found the cluster overwhelmed again today with massive virtual memory usage and an hour after restarting it the same condition returns.
> 
> I'm using a single logger since the last time.  It seems when using a Kafka only export a single logger works fine as the event timestamps arriving at Kafka are near realtime.  (the "ts" for conn, http, etc.) 
> 
> The logger is using 47761, manager is 47762.  I took a sample of 5000 packets for each during the high memory usage and it looks like the manager is still receiving logs of some sort dealing with x509 certificates.
> 
> 
> tcpdump -A -r Bro__Manager_port_47662.pcap | egrep -io '[A-Za-z_:-]{10,}' | sort | uniq -c | sort -rn | head -10
> reading from file Bro__Manager_port_47662.pcap, link-type EN10MB (Ethernet)
> 2852 certificate
> 2064 Notice::begin_suppressionA
> 1800 Conn::IN_ORIG
>  739 Authentication
>  685 Identifier
>  681 Encipherment
>  660 validation
>  646 Corporation
> 

What timeframe was that pcap for?  If the pcap was from an hour or so, it's probably nothing.. but if that was from a few seconds you could have a problem there.

The 2000+ things are definitely related to SSL, as well as the other strings in there.. if you look at the raw tcpdump output those would make more sense in the normal order..  The numbers are a little inflated because when the manager sends out something like the Notice::begin_suppression event, it has to send it once to each worker (which is also something that needs to be addressed for better scaling bro).  ~2064 events would have been sent out for only ~14 notices events if you had 150 workers.

What does your notice.log contain related to SSL.  Do you have a TON of notices for Invalid_Server_Cert or something like it?  Is your known_certs.log file growing rapidly?

using a larger cutoff for head may have shown SSL::Invalid_Server_Cert.

The 1800 Conn::IN_ORIG are from workers -> manager from policy/frameworks/intel/seen/conn-established.bro


One thing you could try is commenting out anything in your config related to ssl or intel, and see if that's stable.  That would help narrow down what the problem is.

In general, the manager just isn't doing much anymore, so for it to be using that much ram that fast, it would have to be doing something extremely frequently.  That's why knowing the timeframe is really important :-)

If your cluster is doing something like generating Invalid_Server_Cert notices at an extremely high rate, then it's possible that the manager parent is trying to tell all the workers about it and the manager child is not able to keep up.  That kind of fits with this output:

manager      manager 10.1.1.1   37653   child   506M   237M 100%  bro
manager      manager 10.1.1.1   37600   parent  640G    46G  38%  bro

-- 
- Justin Azoff



From john at isfaster.com  Sat Feb  4 19:20:34 2017
From: john at isfaster.com (John Brown (isFaster))
Date: Sat, 4 Feb 2017 20:20:34 -0700
Subject: [Bro] new to bro, a few questions
Message-ID: <CAAdi25_braqHc06MiVhYctViJXZ4KigECL4e-ASJ-zQ7VDs7DQ@mail.gmail.com>

Hi,  I'm new to Bro and I'm wondering how I can do a couple of things:

1.  I'd like to basically disable all of the various rules and detection
stuff.
2. I'd like to create a simple rule that detects say DNS packets with
cpsc.gov in the query or answer

Figure it would be best to start simple and then build up rules (either my
own, or others) as I need them.   Sort of a K&R "Hello World" approach..

Any specifics would be much appreciated.


Thank you
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170204/ecb0ab9b/attachment.html 

From pyrodie18 at gmail.com  Sun Feb  5 12:09:23 2017
From: pyrodie18 at gmail.com (Troy Ward)
Date: Sun, 5 Feb 2017 15:09:23 -0500
Subject: [Bro] new to bro, a few questions
Message-ID: <CAKjK3aTtj7SKQ_ik-gm=yaXK_sKHBvWygpDo3_MQZSRa_r8Xag@mail.gmail.com>

Not sure that bro is the best choice for what you're looking for.  Bro is
capable of doing what you're asking but this sounds like it may be better
to try out SNORT.  Bro is much more usefully for getting a wide variety of
statistics for a wide variety of packets, not just a single DNS packet.

Troy



> Hi,  I'm new to Bro and I'm wondering how I can do a couple of things:
>
> 1.  I'd like to basically disable all of the various rules and detection
> stuff.
> 2. I'd like to create a simple rule that detects say DNS packets with
> cpsc.gov in the query or answer
>
> Figure it would be best to start simple and then build up rules (either my
> own, or others) as I need them.   Sort of a K&R "Hello World" approach..
>
> Any specifics would be much appreciated.
>
>
> Thank you
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/
> 20170204/ecb0ab9b/attachment-0001.html
>
> ------------------------------
>
> _______________________________________________
> Bro mailing list
> Bro at bro.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
> End of Bro Digest, Vol 130, Issue 8
> ***********************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170205/f4a4c6f0/attachment.html 

From cgaylord at vt.edu  Sun Feb  5 12:21:41 2017
From: cgaylord at vt.edu (Clark Gaylord)
Date: Sun, 5 Feb 2017 15:21:41 -0500
Subject: [Bro] new to bro, a few questions
In-Reply-To: <CAKjK3aTtj7SKQ_ik-gm=yaXK_sKHBvWygpDo3_MQZSRa_r8Xag@mail.gmail.com>
References: <CAKjK3aTtj7SKQ_ik-gm=yaXK_sKHBvWygpDo3_MQZSRa_r8Xag@mail.gmail.com>
Message-ID: <CADzU5g6nD_VgcYysmfNx80k_2mgz7L5qMRC65jKBJ3PUx6xr4Q@mail.gmail.com>

Though if you're thinking the eventuality of more of bro's functionality is
possibly in your future, there's something to be said for that. You could
retain logs for a couple days or a week, say, and use grep, etc for
retaining your query of interest longer. That's probably easier than going
into the config and turning off the default reports, though as Troy points
out you can do that.

I've never known anyone to say "I wish I didn't have these data" (though
perhaps "I wish they didn't take up the space")...

--
Clark Gaylord
cgaylord at vt.edu
... autocorrect may have improved this message
    brevity should not imply curtness ...

On Feb 5, 2017 15:11, "Troy Ward" <pyrodie18 at gmail.com> wrote:

Not sure that bro is the best choice for what you're looking for.  Bro is
capable of doing what you're asking but this sounds like it may be better
to try out SNORT.  Bro is much more usefully for getting a wide variety of
statistics for a wide variety of packets, not just a single DNS packet.

Troy



> Hi,  I'm new to Bro and I'm wondering how I can do a couple of things:
>
> 1.  I'd like to basically disable all of the various rules and detection
> stuff.
> 2. I'd like to create a simple rule that detects say DNS packets with
> cpsc.gov in the query or answer
>
> Figure it would be best to start simple and then build up rules (either my
> own, or others) as I need them.   Sort of a K&R "Hello World" approach..
>
> Any specifics would be much appreciated.
>
>
> Thank you
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/2
> 0170204/ecb0ab9b/attachment-0001.html
>
> ------------------------------
>
> _______________________________________________
> Bro mailing list
> Bro at bro.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
> End of Bro Digest, Vol 130, Issue 8
> ***********************************
>


_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro




On Feb 5, 2017 15:11, "Troy Ward" <pyrodie18 at gmail.com> wrote:

Not sure that bro is the best choice for what you're looking for.  Bro is
capable of doing what you're asking but this sounds like it may be better
to try out SNORT.  Bro is much more usefully for getting a wide variety of
statistics for a wide variety of packets, not just a single DNS packet.

Troy



> Hi,  I'm new to Bro and I'm wondering how I can do a couple of things:
>
> 1.  I'd like to basically disable all of the various rules and detection
> stuff.
> 2. I'd like to create a simple rule that detects say DNS packets with
> cpsc.gov in the query or answer
>
> Figure it would be best to start simple and then build up rules (either my
> own, or others) as I need them.   Sort of a K&R "Hello World" approach..
>
> Any specifics would be much appreciated.
>
>
> Thank you
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/2
> 0170204/ecb0ab9b/attachment-0001.html
>
> ------------------------------
>
> _______________________________________________
> Bro mailing list
> Bro at bro.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
> End of Bro Digest, Vol 130, Issue 8
> ***********************************
>


_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170205/bae92b5a/attachment.html 

From anthony.kasza at gmail.com  Sun Feb  5 14:03:55 2017
From: anthony.kasza at gmail.com (anthony kasza)
Date: Sun, 5 Feb 2017 15:03:55 -0700
Subject: [Bro] new to bro, a few questions
In-Reply-To: <CAAdi25_braqHc06MiVhYctViJXZ4KigECL4e-ASJ-zQ7VDs7DQ@mail.gmail.com>
References: <CAAdi25_braqHc06MiVhYctViJXZ4KigECL4e-ASJ-zQ7VDs7DQ@mail.gmail.com>
Message-ID: <CAEZw2bzg8-N0D++hMENip1x3NQnTGJhsUSqb-HOLOd27-kKYxQ@mail.gmail.com>

You may want to look at Bro's "bare mode". It starts Bro without many of
Bro's features.

-AK

On Feb 4, 2017 8:23 PM, "John Brown (isFaster)" <john at isfaster.com> wrote:

> Hi,  I'm new to Bro and I'm wondering how I can do a couple of things:
>
> 1.  I'd like to basically disable all of the various rules and detection
> stuff.
> 2. I'd like to create a simple rule that detects say DNS packets with
> cpsc.gov in the query or answer
>
> Figure it would be best to start simple and then build up rules (either my
> own, or others) as I need them.   Sort of a K&R "Hello World" approach..
>
> Any specifics would be much appreciated.
>
>
> Thank you
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170205/2692f167/attachment.html 

From darkheaven1983 at gmail.com  Mon Feb  6 01:13:38 2017
From: darkheaven1983 at gmail.com (duhang)
Date: Mon, 6 Feb 2017 17:13:38 +0800
Subject: [Bro] Content gap breaks application layer analysis
Message-ID: <CAG+yijOVf4q2pB1mOTG+YCGb7Yjm6rgySW6TgH2rvbzksJuprg@mail.gmail.com>

Hi,

I'm using Bro which listens to the nic card connects to a mirror port from
a switch to dump http request/response and smtp email for further analysis.
The packets that received from the mirror port are massively
disordered(Unseen ACKed in wireshark). I saw a lot of content gap events
which skips the following packets received. A lot of uncompleted http/smtp
logs exist which relatively means high packet loss rate from appliance
layer's perspective. Is there any workaround/solution to have
bi-directional reassembly in this case?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170206/a0b6040b/attachment-0001.html 

From hosom at battelle.org  Mon Feb  6 07:31:23 2017
From: hosom at battelle.org (Hosom, Stephen M)
Date: Mon, 6 Feb 2017 15:31:23 +0000
Subject: [Bro] Remember to double check your DNS resolver configuration
Message-ID: <E5A2071C6FFFF640B714C224139C9B3327E2E30D@WP-MBX2B.milky-way.battelle.org>

I've been troubleshooting an issue where a single node would have all of its workers grow in memory until they would be OOM killed. The troubleshooting process spanned multiple days and I only happened to come across this with some help from Justin combined with a thread on the issue tracker (https://bro-tracker.atlassian.net/browse/BIT-1482).

Keep in mind that when you are using the MHR script (enabled by default) or the notary script, your Bro workers are performing a LOT of DNS. In my case I was using both. Since lookup_host_txt and lookup_host never return if the worker node doesn't reach a DNS server, this results in what would appear to be a new thread for each new DNS query when your DNS resolvers are misconfigured.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170206/f1c4d3de/attachment.html 

From jlay at slave-tothe-box.net  Mon Feb  6 09:32:55 2017
From: jlay at slave-tothe-box.net (James Lay)
Date: Mon, 06 Feb 2017 10:32:55 -0700
Subject: [Bro] Extracted files don't rotate
Message-ID: <7072c554eb4d393e84f5f5ae8a37440e@localhost>

Hey all,

So I recently changed the way I run bro at a site.  Originally this was 
run via command line, now I have the below:

[logger]
type=logger
host=localhost

[manager]
type=manager
host=localhost

[proxy-1]
type=proxy
host=localhost

[worker-1]
type=worker
host=localhost
interface=ethx

[worker-2]
type=worker
host=localhost
interface=ethx

extract_files shows up in worker-2.  Here's the extract-files script:

global ext_map: table[string] of string = {
         ["application/x-dosexec"] = "exe",
         ["application/zip"] = "zip",
         ["application/msword"] = "xls",
         
["application/vnd.openxmlformats-officedocument.wordprocessingml.document"] 
= "docx",
         
["application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"]  = 
"xlsx",
         
["application/vnd.openxmlformats-officedocument.presentationml.presentation"] 
= "pptx"

};

event file_sniff(f: fa_file, meta: fa_metadata)
         {
         if ( f$source != "SMTP" )
                 return;

         if ( ! meta?$mime_type || meta$mime_type !in ext_map )
                 return;

         local ext = "";

         if ( meta?$mime_type )
                 ext = ext_map[meta$mime_type];

         local fname = fmt("%s-%s.%s", f$source, f$id, ext);
         Files::add_analyzer(f, Files::ANALYZER_EXTRACT, 
[$extract_filename=fname]);
}


In looking, I see that the files are accumulating and not rotating out.  
Anything I can do to troubleshoot this?  Thank you.

James

From hovsep.sanjay.levi at gmail.com  Mon Feb  6 11:14:28 2017
From: hovsep.sanjay.levi at gmail.com (Hovsep Levi)
Date: Mon, 6 Feb 2017 19:14:28 +0000
Subject: [Bro] Logging and memory leak
In-Reply-To: <6B204A64-5F4C-4230-8741-9CDD550EC433@illinois.edu>
References: <CAJ19UZBxxOEZqmT2MYMBVsjno3mimNjN6egRWv0h5A-Sp_bd_Q@mail.gmail.com>
	<A7503EE4-D454-4404-9239-DA3C8A6D559A@illinois.edu>
	<CAJ19UZDJwoGq9RGKYZ+OftCeS+AOqxyfmAbV_3PHmghfWUEFFA@mail.gmail.com>
	<F1308A75-9240-4DEB-88C4-B1ED8F9CD5F5@illinois.edu>
	<CAJ19UZD+0ZR3W2WYMPc8ELLTTKrkYPtyJ0J4EnWunMLeF6bhdg@mail.gmail.com>
	<6B204A64-5F4C-4230-8741-9CDD550EC433@illinois.edu>
Message-ID: <CAJ19UZCQ-iuHNco0EiPFdRPgQ-Xd_x5B84q0QniN+8QpxXOeQA@mail.gmail.com>

On Sat, Feb 4, 2017 at 4:00 AM, Azoff, Justin S <jazoff at illinois.edu> wrote:

What timeframe was that pcap for?  If the pcap was from an hour or so, it's
> probably nothing.. but if that was from a few seconds you could have a
> problem there.
>

The pcap is 9 seconds during the problem timeframe at the peak of system
usage.



> The 2000+ things are definitely related to SSL, as well as the other
> strings in there.. if you look at the raw tcpdump output those would make
> more sense in the normal order..  The numbers are a little inflated because
> when the manager sends out something like the Notice::begin_suppression
> event, it has to send it once to each worker (which is also something that
> needs to be addressed for better scaling bro).  ~2064 events would have
> been sent out for only ~14 notices events if you had 150 workers.
>
>
Good to know.  132 workers.


> What does your notice.log contain related to SSL.  Do you have a TON of
> notices for Invalid_Server_Cert or something like it?  Is your
> known_certs.log file growing rapidly?
>


Not sure at the moment.



> using a larger cutoff for head may have shown SSL::Invalid_Server_Cert.
>
>
The 1800 Conn::IN_ORIG are from workers -> manager from
> policy/frameworks/intel/seen/conn-established.bro
>
>
It did not but I think that type of message would have been sent to the
Logger on a different port.

2064 Notice::begin_suppression
1800 Conn::IN_ORIG
 396 Notice::cluster_notice
  26 SumStats::cluster_key_intermediate_response
   1 Intel::match_no_items
   1 Conn::Info



> One thing you could try is commenting out anything in your config related
> to ssl or intel, and see if that's stable.  That would help narrow down
> what the problem is.
>
> In general, the manager just isn't doing much anymore, so for it to be
> using that much ram that fast, it would have to be doing something
> extremely frequently.  That's why knowing the timeframe is really important
> :-)
>
> If your cluster is doing something like generating Invalid_Server_Cert
> notices at an extremely high rate, then it's possible that the manager
> parent is trying to tell all the workers about it and the manager child is
> not able to keep up.  That kind of fits with this output:
>
>

I'll try that.  Seems like I'll be able to narrow down the issue this
week.  There's a weekly pattern to the failure starting late Thursday and
continuing most of the day Friday so I'm guessing either a researcher or an
automated scan is the cause.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170206/154f6f8a/attachment.html 

From lc.taylor at protonmail.com  Mon Feb  6 17:16:52 2017
From: lc.taylor at protonmail.com (Lincy Taylor)
Date: Mon, 06 Feb 2017 20:16:52 -0500
Subject: [Bro] Run Bro with inspecting specific protocol only
Message-ID: <Tsj4Ac_yMyFLq9P8xHh3ELjQB7FpEsrqDWG-PW9tzTVhH1XIFWBS60nL3yeDbyV_Awk9jpq9fI5ui9GWh-Zip16FYpZ4mTEDTkdYedDbyTU=@protonmail.com>

Hello all:

How to run bro with only necessary module and specific protocol analyzers enabled? I am trying to use Bro to detect huge amount of malicious DNS queries and found the packet dropping rate is higher than 50% in bro with PF_RING enabled. I was thinking if there's any method to speed up Bro by disabling unnecessary modules and protocol analyzers. Another problem I am having is I implemented an event handler for 'log_dns' event in my work and i will get no event logs if I removed the default built-in log stream of DNS with "Log::remove_stream(DNS::LOG)". Can anyone share with me your experiences? thanks.



Sent with [ProtonMail](https://protonmail.com) Secure Email.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170206/743f8b35/attachment.html 

From albertociolini92 at gmail.com  Tue Feb  7 08:07:26 2017
From: albertociolini92 at gmail.com (Alberto Ciolini)
Date: Tue, 7 Feb 2017 17:07:26 +0100
Subject: [Bro]  Send mail
Message-ID: <CA+wdK6tJQ+WF4cKUGU96R+sXCxNnoCUgojnzq_vqvFNA+0_4LQ@mail.gmail.com>

Hi everyone,
i have a little question. How to send email with a simple text message in
Bro?

Thanks a lot!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170207/6cd9d2d9/attachment.html 

From jdopheid at illinois.edu  Tue Feb  7 11:48:25 2017
From: jdopheid at illinois.edu (Dopheide, Jeannette M)
Date: Tue, 7 Feb 2017 19:48:25 +0000
Subject: [Bro] Bro4Pros slides
Message-ID: <A22D84E1-F620-4889-AF72-F4B37B8C31C7@illinois.edu>

We?ve posted the slideshow presentations for Bro4Pros, you can find them here: 

https://www.bro.org/community/bro4pros2017.html#agenda 

Thanks again to the presenters and all of you who were able to join us. 


------
Jeannette Dopheide
Training and Outreach Coordinator
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
 



From jdopheid at illinois.edu  Tue Feb  7 13:17:48 2017
From: jdopheid at illinois.edu (Dopheide, Jeannette M)
Date: Tue, 7 Feb 2017 21:17:48 +0000
Subject: [Bro] 7-question survey about professional Bro Training
Message-ID: <42F0BB6F-4F77-4F81-9898-682D5C89D81B@illinois.edu>

Hello Bro Community,

We are attempting to gauge interest in professional Bro training, potentially with corresponding certification officially approved by the Bro Project. Let us know about your needs, or the needs of your staff, by filling out this 7-question survey:

https://goo.gl/forms/Cm2MZRYexAzYAQrk2

Thank you,


The Bro Project

------
Jeannette Dopheide
Training and Outreach Coordinator
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
 



From bro at pingtrip.com  Tue Feb  7 16:09:25 2017
From: bro at pingtrip.com (Dave Crawford)
Date: Tue, 7 Feb 2017 19:09:25 -0500
Subject: [Bro] PF_Ring
Message-ID: <44F81CFD-BC3C-46C1-8D40-9985B3E4CDAA@pingtrip.com>

Are there any performance benefits over compiling Bro with pf_ring (--with-pcap=*) versus using the Bro pf_ring plugin? 

Additionally, if I?m using the ZC drivers (with zbalance_ipc clusters) is the plugin compatible or do I still need to compile Bro with the ?with-pcap option?

Also, this documentation is a tad outdated at this point:

https://www.bro.org/sphinx-git/configuration/index.html <https://www.bro.org/sphinx-git/configuration/index.html>

Thanks!
-Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170207/54547dc6/attachment.html 

From bro at pingtrip.com  Wed Feb  8 04:52:37 2017
From: bro at pingtrip.com (Dave Crawford)
Date: Wed, 8 Feb 2017 07:52:37 -0500
Subject: [Bro] Netmap
Message-ID: <9E525AE1-D515-463B-B815-6C5A0D83CD6C@pingtrip.com>

Related to my other question on PF_Ring? I just read through Seth?s slides on Netmap (https://www.bro.org//bro4pros2017/Hall_Netmap_Bro4Pros2017.pdf <https://www.bro.org//bro4pros2017/Hall_Netmap_Bro4Pros2017.pdf>) and I?m curious if it's currently a viable solution for production? From the slides it appears to be a much simpler implementation than PF_Ring, while providing the same performance as PF_Ring ZC?

-Dave 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170208/5a663820/attachment.html 

From seth at icir.org  Wed Feb  8 07:33:35 2017
From: seth at icir.org (Seth Hall)
Date: Wed, 8 Feb 2017 10:33:35 -0500
Subject: [Bro] Netmap
In-Reply-To: <9E525AE1-D515-463B-B815-6C5A0D83CD6C@pingtrip.com>
References: <9E525AE1-D515-463B-B815-6C5A0D83CD6C@pingtrip.com>
Message-ID: <64519771-0846-4E0A-91D0-03C187E512FB@icir.org>


> On Feb 8, 2017, at 7:52 AM, Dave Crawford <bro at pingtrip.com> wrote:
> 
> Related to my other question on PF_Ring? I just read through Seth?s slides on Netmap (https://www.bro.org//bro4pros2017/Hall_Netmap_Bro4Pros2017.pdf) and I?m curious if it's currently a viable solution for production? From the slides it appears to be a much simpler implementation than PF_Ring, while providing the same performance as PF_Ring ZC?

It is viable with the caveat that it's still a bit in development.  I've used it quite a bit and it's been very stable for me.   Let me know if you run into any trouble if you try it!

 .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From shirkdog.bsd at gmail.com  Wed Feb  8 07:50:17 2017
From: shirkdog.bsd at gmail.com (Michael Shirk)
Date: Wed, 8 Feb 2017 10:50:17 -0500
Subject: [Bro] Netmap
In-Reply-To: <64519771-0846-4E0A-91D0-03C187E512FB@icir.org>
References: <9E525AE1-D515-463B-B815-6C5A0D83CD6C@pingtrip.com>
	<64519771-0846-4E0A-91D0-03C187E512FB@icir.org>
Message-ID: <CAL8PkUVASf0jOiS8HVP09EStLgU9xeYo19nuSK7PmPh8AU6a3w@mail.gmail.com>

In the FreeBSD sense, 12-CURRENT has a recent check-in of netmap code into
the OS/Kernel, which is usable by default. You have to compile lb from the
netmap github repo.

lb compiles and works with 11-STABLE but YMMV with the use of it with the
netmap code.

--
Michael Shirk
Daemon Security, Inc.
http://www.daemon-security.com

On Feb 8, 2017 10:42 AM, "Seth Hall" <seth at icir.org> wrote:

>
> > On Feb 8, 2017, at 7:52 AM, Dave Crawford <bro at pingtrip.com> wrote:
> >
> > Related to my other question on PF_Ring? I just read through Seth?s
> slides on Netmap (https://www.bro.org//bro4pros2017/Hall_Netmap_
> Bro4Pros2017.pdf) and I?m curious if it's currently a viable solution for
> production? From the slides it appears to be a much simpler implementation
> than PF_Ring, while providing the same performance as PF_Ring ZC?
>
> It is viable with the caveat that it's still a bit in development.  I've
> used it quite a bit and it's been very stable for me.   Let me know if you
> run into any trouble if you try it!
>
>  .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170208/095aaf14/attachment.html 

From seth at icir.org  Wed Feb  8 08:18:45 2017
From: seth at icir.org (Seth Hall)
Date: Wed, 8 Feb 2017 11:18:45 -0500
Subject: [Bro] Netmap
In-Reply-To: <CAL8PkUVASf0jOiS8HVP09EStLgU9xeYo19nuSK7PmPh8AU6a3w@mail.gmail.com>
References: <9E525AE1-D515-463B-B815-6C5A0D83CD6C@pingtrip.com>
	<64519771-0846-4E0A-91D0-03C187E512FB@icir.org>
	<CAL8PkUVASf0jOiS8HVP09EStLgU9xeYo19nuSK7PmPh8AU6a3w@mail.gmail.com>
Message-ID: <8F0EE4A6-D31E-46B3-AEF0-519F22F1061A@icir.org>

Nice!  Let me know if you have any problems.

  .Seth

> On Feb 8, 2017, at 10:50 AM, Michael Shirk <shirkdog.bsd at gmail.com> wrote:
> 
> In the FreeBSD sense, 12-CURRENT has a recent check-in of netmap code into the OS/Kernel, which is usable by default. You have to compile lb from the netmap github repo.
> 
> lb compiles and works with 11-STABLE but YMMV with the use of it with the netmap code.
> 
> --
> Michael Shirk
> Daemon Security, Inc.
> http://www.daemon-security.com
> 
> On Feb 8, 2017 10:42 AM, "Seth Hall" <seth at icir.org> wrote:
> 
> > On Feb 8, 2017, at 7:52 AM, Dave Crawford <bro at pingtrip.com> wrote:
> >
> > Related to my other question on PF_Ring? I just read through Seth?s slides on Netmap (https://www.bro.org//bro4pros2017/Hall_Netmap_Bro4Pros2017.pdf) and I?m curious if it's currently a viable solution for production? From the slides it appears to be a much simpler implementation than PF_Ring, while providing the same performance as PF_Ring ZC?
> 
> It is viable with the caveat that it's still a bit in development.  I've used it quite a bit and it's been very stable for me.   Let me know if you run into any trouble if you try it!
> 
>  .Seth
> 
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From jdopheid at illinois.edu  Wed Feb  8 13:27:41 2017
From: jdopheid at illinois.edu (Dopheide, Jeannette M)
Date: Wed, 8 Feb 2017 21:27:41 +0000
Subject: [Bro] 7-question survey about professional Bro Training
Message-ID: <D8DC4ECF-6DD7-4E4F-9E93-71EC2097F66F@illinois.edu>

Hello, 

Pinging this thread again to remind folks to fill out this survey. We?re getting some really great feedback already. 

------
Jeannette Dopheide
Training and Outreach Coordinator
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
 

On 2/7/17, 3:17 PM, "bro-bounces at bro.org on behalf of Dopheide, Jeannette M" <bro-bounces at bro.org on behalf of jdopheid at illinois.edu> wrote:

Hello Bro Community,

We are attempting to gauge interest in professional Bro training, potentially with corresponding certification officially approved by the Bro Project. Let us know about your needs, or the needs of your staff, by filling out this 7-question survey:

https://goo.gl/forms/Cm2MZRYexAzYAQrk2

Thank you,


The Bro Project

------
Jeannette Dopheide
Training and Outreach Coordinator
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
 


_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro




From jlay at slave-tothe-box.net  Wed Feb  8 14:01:43 2017
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 08 Feb 2017 15:01:43 -0700
Subject: [Bro] Extracted files don't rotate
In-Reply-To: <7072c554eb4d393e84f5f5ae8a37440e@localhost>
References: <7072c554eb4d393e84f5f5ae8a37440e@localhost>
Message-ID: <9739c7e8dd7e8de7f2b6161a05ea2079@localhost>

Any takers on the below?  Thank you.

James

On 2017-02-06 10:32, James Lay wrote:
> Hey all,
> 
> So I recently changed the way I run bro at a site.  Originally this was
> run via command line, now I have the below:
> 
> [logger]
> type=logger
> host=localhost
> 
> [manager]
> type=manager
> host=localhost
> 
> [proxy-1]
> type=proxy
> host=localhost
> 
> [worker-1]
> type=worker
> host=localhost
> interface=ethx
> 
> [worker-2]
> type=worker
> host=localhost
> interface=ethx
> 
> extract_files shows up in worker-2.  Here's the extract-files script:
> 
> global ext_map: table[string] of string = {
>          ["application/x-dosexec"] = "exe",
>          ["application/zip"] = "zip",
>          ["application/msword"] = "xls",
> 
> ["application/vnd.openxmlformats-officedocument.wordprocessingml.document"]
> = "docx",
> 
> ["application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"]  
> =
> "xlsx",
> 
> ["application/vnd.openxmlformats-officedocument.presentationml.presentation"]
> = "pptx"
> 
> };
> 
> event file_sniff(f: fa_file, meta: fa_metadata)
>          {
>          if ( f$source != "SMTP" )
>                  return;
> 
>          if ( ! meta?$mime_type || meta$mime_type !in ext_map )
>                  return;
> 
>          local ext = "";
> 
>          if ( meta?$mime_type )
>                  ext = ext_map[meta$mime_type];
> 
>          local fname = fmt("%s-%s.%s", f$source, f$id, ext);
>          Files::add_analyzer(f, Files::ANALYZER_EXTRACT,
> [$extract_filename=fname]);
> }
> 
> 
> In looking, I see that the files are accumulating and not rotating out.
> Anything I can do to troubleshoot this?  Thank you.
> 
> James
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

From jazoff at illinois.edu  Wed Feb  8 14:10:54 2017
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Wed, 8 Feb 2017 22:10:54 +0000
Subject: [Bro] Extracted files don't rotate
In-Reply-To: <7072c554eb4d393e84f5f5ae8a37440e@localhost>
References: <7072c554eb4d393e84f5f5ae8a37440e@localhost>
Message-ID: <CE59E941-B85C-4751-9724-BA3DA1BE7BE4@illinois.edu>


> On Feb 6, 2017, at 12:32 PM, James Lay <jlay at slave-tothe-box.net> wrote:
> 
> Hey all,
> In looking, I see that the files are accumulating and not rotating out.  
> Anything I can do to troubleshoot this?  Thank you.

Ah yes.. Extracted files aren't managed by anything.  If you want files archived in a specific way you need to extract them to the full path that you want or setup a cron job to move+compress them periodically.

If you had something specific in mind I could probably whip up an example script for you.


-- 
- Justin Azoff



From jlay at slave-tothe-box.net  Wed Feb  8 14:17:49 2017
From: jlay at slave-tothe-box.net (James Lay)
Date: Wed, 08 Feb 2017 15:17:49 -0700
Subject: [Bro] Extracted files don't rotate
In-Reply-To: <CE59E941-B85C-4751-9724-BA3DA1BE7BE4@illinois.edu>
References: <7072c554eb4d393e84f5f5ae8a37440e@localhost>
	<CE59E941-B85C-4751-9724-BA3DA1BE7BE4@illinois.edu>
Message-ID: <8ef08f1c450bfd0fbf71c49fe853abf5@localhost>

On 2017-02-08 15:10, Azoff, Justin S wrote:
>> On Feb 6, 2017, at 12:32 PM, James Lay <jlay at slave-tothe-box.net> 
>> wrote:
>> 
>> Hey all,
>> In looking, I see that the files are accumulating and not rotating 
>> out.
>> Anything I can do to troubleshoot this?  Thank you.
> 
> Ah yes.. Extracted files aren't managed by anything.  If you want
> files archived in a specific way you need to extract them to the full
> path that you want or setup a cron job to move+compress them
> periodically.
> 
> If you had something specific in mind I could probably whip up an
> example script for you.

Thanks Justin that's helpful.  So as I look at my old setup I see that I 
indeed had move and compress manually on a cron job, so I'll just do 
that for the extract_files dir.  Maybe a feature request down the road 
would be (maybe in broctl.conf) to be able to add "pre" rotate and 
"post" rotate scripts.  Just a thought.

James

From deshmukh at slac.stanford.edu  Wed Feb  8 14:55:11 2017
From: deshmukh at slac.stanford.edu (Deshmukh, Andy)
Date: Wed, 8 Feb 2017 22:55:11 +0000
Subject: [Bro] Netmap plugin issue
Message-ID: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>

Hello,

I am trying to build netmap plugin with Bro 2.5 and having the following error:

[root at bro  netmap]# ./configure --bro-dist=/home/bro/bro-2.5 --install-root=/var/bro --with-netmap=/home/bro/netmap-corelight_updates
Build Directory        : build
Bro Source Directory   : /home/bro/bro-2.5
-- The C compiler identification is GNU 4.8.5
-- The CXX compiler identification is GNU 4.8.5
-- Check for working C compiler: /bin/cc
-- Check for working C compiler: /bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /bin/c++
-- Check for working CXX compiler: /bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Performing Test cxx11_header_works
-- Performing Test cxx11_header_works - Success
-- Bro executable      : /home/bro/bro-2.5/build/src/bro
-- Bro source          : /home/bro/bro-2.5
-- Bro build           : /home/bro/bro-2.5/build
-- Bro install prefix  : /var/bro
-- Bro plugin directory: /var/bro
-- Bro debug mode      : false
-- Could NOT find Netmap (missing:  NETMAP_INCLUDE_DIR)
CMake Error at CMakeLists.txt:20 (message):
  Netmap headers not found.


-- Configuring incomplete, errors occurred!
See also "/home/bro/bro-2.5/aux/plugins/netmap/build/CMakeFiles/CMakeOutput.log".

Any suggestions?

Thanks,
Andy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170208/928b8bb6/attachment.html 

From seth at icir.org  Wed Feb  8 19:15:54 2017
From: seth at icir.org (Seth Hall)
Date: Wed, 8 Feb 2017 22:15:54 -0500
Subject: [Bro] Extracted files don't rotate
In-Reply-To: <8ef08f1c450bfd0fbf71c49fe853abf5@localhost>
References: <7072c554eb4d393e84f5f5ae8a37440e@localhost>
	<CE59E941-B85C-4751-9724-BA3DA1BE7BE4@illinois.edu>
	<8ef08f1c450bfd0fbf71c49fe853abf5@localhost>
Message-ID: <A0F8DFD9-1CF8-4B81-88AF-BD558A8F4D09@icir.org>


> On Feb 8, 2017, at 5:17 PM, James Lay <jlay at slave-tothe-box.net> wrote:
> 
> Thanks Justin that's helpful.  So as I look at my old setup I see that I 
> indeed had move and compress manually on a cron job, so I'll just do 
> that for the extract_files dir.  Maybe a feature request down the road 
> would be (maybe in broctl.conf) to be able to add "pre" rotate and 
> "post" rotate scripts.  Just a thought.

This has been a bit of a sticking point for quite a while.  Part of the issue is the diversity in how clusters are run and managed.  It's hard to create one solution which works for everyone's deployment.

I've been hoping to spend some time rejiggering how file extraction happens a little bit this year but I'd be glad to see anyone else beat me to it.  It's a deceptively sneaky issue.

 .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Wed Feb  8 19:17:23 2017
From: seth at icir.org (Seth Hall)
Date: Wed, 8 Feb 2017 22:17:23 -0500
Subject: [Bro] Netmap plugin issue
In-Reply-To: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
Message-ID: <237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>


> On Feb 8, 2017, at 5:55 PM, Deshmukh, Andy <deshmukh at slac.stanford.edu> wrote:
> 
> -- Could NOT find Netmap (missing:  NETMAP_INCLUDE_DIR)
> CMake Error at CMakeLists.txt:20 (message):
>   Netmap headers not found.

Unfortunately in the plugin that comes with 2.5, you need netmap installed on the system you're building on.  We're going to be making changes for the 2.6 release so that nothing is required from netmap at build time and it is just built by default and included in Bro.

If you install netmap, does the problem go away?

  .Seth


--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From jlay at slave-tothe-box.net  Thu Feb  9 05:09:48 2017
From: jlay at slave-tothe-box.net (James Lay)
Date: Thu, 09 Feb 2017 06:09:48 -0700
Subject: [Bro] Extracted files don't rotate
In-Reply-To: <A0F8DFD9-1CF8-4B81-88AF-BD558A8F4D09@icir.org>
References: <7072c554eb4d393e84f5f5ae8a37440e@localhost>
	<CE59E941-B85C-4751-9724-BA3DA1BE7BE4@illinois.edu>
	<8ef08f1c450bfd0fbf71c49fe853abf5@localhost>
	<A0F8DFD9-1CF8-4B81-88AF-BD558A8F4D09@icir.org>
Message-ID: <1486645788.2453.5.camel@slave-tothe-box.net>

On Wed, 2017-02-08 at 22:15 -0500, Seth Hall wrote:
> > 
> > On Feb 8, 2017, at 5:17 PM, James Lay <jlay at slave-tothe-box.net>
> > wrote:
> > 
> > Thanks Justin that's helpful.??So as I look at my old setup I see
> > that I?
> > indeed had move and compress manually on a cron job, so I'll just
> > do?
> > that for the extract_files dir.??Maybe a feature request down the
> > road?
> > would be (maybe in broctl.conf) to be able to add "pre" rotate and?
> > "post" rotate scripts.??Just a thought.
> This has been a bit of a sticking point for quite a while.??Part of
> the issue is the diversity in how clusters are run and managed.??It's
> hard to create one solution which works for everyone's deployment.
> 
> I've been hoping to spend some time rejiggering how file extraction
> happens a little bit this year but I'd be glad to see anyone else
> beat me to it.??It's a deceptively sneaky issue.
> 
> ?.Seth
> 
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
Appreciate the feedback all.
James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170209/fec0b557/attachment.html 

From jdopheid at illinois.edu  Thu Feb  9 06:39:41 2017
From: jdopheid at illinois.edu (Dopheide, Jeannette M)
Date: Thu, 9 Feb 2017 14:39:41 +0000
Subject: [Bro] Bro Blog: Software Freedom Conservancy fund drive
Message-ID: <E5DDC587-4A8D-49DA-AA95-851A18283E15@illinois.edu>

[https://2.bp.blogspot.com/-CJKHQRmOhJA/VgGwoP4dcBI/AAAAAAAAADg/6UBXnHvLg2c/s200/sfc_logo.png]<https://sfconservancy.org/donate/>



In October of 2015 we announced<http://blog.bro.org/2015/10/bro-software-freedom-conservancy.html> that the Bro Project joined Software Freedom Conservancy. Conservancy is a not-for-profit organization that helps promote, improve, develop, and defend Free, Libre, and Open Source Software (FLOSS) projects. You are likely familiar with many of its member projects; including Git, BusyBox, Samba, and PyPy.

We chose to join Conservancy for several reasons: it builds community transparency and trust, provides legal protection for contributors, clarifies intellectual property, and signifies longevity for the project. They leave the technical and artistic control of the project to the contributors and community.

With the guidance of Conservancy we have formed a leadership team, applied for a trademark, and created a donation and sponsorship framework for community members to give back to the project. This may seem like stuffy paperwork but it is the real work necessary for maintaining a sustainable open-source project.

Now it is our turn to help bring the Bro Community to the aid of Conservancy. Conservancy funds its organization by taking a 10% share of donations to member projects, however that is nowhere near enough to fully fund its staff and services. They also rely on donations from people and organizations that are passionate about supporting FLOSS projects.

An anonymous donor has challenged Software Freedom Conservancy with the task of signing up 150 supporters in one week, the deadline is this Monday (February 13th).



Donate here<https://sfconservancy.org/donate/>

If you support the Bro Project, thank you. Please consider including Conservancy in the support network necessary for keeping the Bro Project running.



(Blog originally posted here<http://blog.bro.org/2017/02/software-freedom-conservancy-fund-drive.html>.)

------

Jeannette Dopheide

Training and Outreach Coordinator

National Center for Supercomputing Applications

University of Illinois at Urbana-Champaign
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170209/52749b79/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 4684 bytes
Desc: image001.png
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170209/52749b79/attachment.bin 

From seth at icir.org  Thu Feb  9 10:22:01 2017
From: seth at icir.org (Seth Hall)
Date: Thu, 9 Feb 2017 13:22:01 -0500
Subject: [Bro] Remember to double check your DNS resolver configuration
In-Reply-To: <E5A2071C6FFFF640B714C224139C9B3327E2E30D@WP-MBX2B.milky-way.battelle.org>
References: <E5A2071C6FFFF640B714C224139C9B3327E2E30D@WP-MBX2B.milky-way.battelle.org>
Message-ID: <3913C8F3-D62E-45FA-8D31-7EC1302DB86D@icir.org>

Did evidence of this show up in stats.log?  There are some fields that track the amount of DNS actively being performed by Bro in there.

  .Seth

> On Feb 6, 2017, at 10:31 AM, Hosom, Stephen M <hosom at battelle.org> wrote:
> 
> I?ve been troubleshooting an issue where a single node would have all of its workers grow in memory until they would be OOM killed. The troubleshooting process spanned multiple days and I only happened to come across this with some help from Justin combined with a thread on the issue tracker (https://bro-tracker.atlassian.net/browse/BIT-1482).
>  
> Keep in mind that when you are using the MHR script (enabled by default) or the notary script, your Bro workers are performing a LOT of DNS. In my case I was using both. Since lookup_host_txt and lookup_host never return if the worker node doesn?t reach a DNS server, this results in what would appear to be a new thread for each new DNS query when your DNS resolvers are misconfigured.
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From vladg at illinois.edu  Thu Feb  9 11:37:44 2017
From: vladg at illinois.edu (Vlad Grigorescu)
Date: Thu, 09 Feb 2017 13:37:44 -0600
Subject: [Bro] Information on OCSP and CRL
In-Reply-To: <EE2D32CF-B9DE-433F-9A01-D79001D1B33B@bivio.net>
References: <EE2D32CF-B9DE-433F-9A01-D79001D1B33B@bivio.net>
Message-ID: <m0efz76tpz.fsf@vladg.net>

Tony,

There's an optional script for OCSP validation:
https://github.com/bro/bro/blob/v2.5/scripts/policy/protocols/ssl/validate-ocsp.bro

To use:
> @load protocols/ssl/validate-ocsp

  --Vlad

Tony Waller <twaller at bivio.net> writes:

> I am looking for additional information on utilizing OCSP and CRL in Bro NSM. I would like to know if certificates from clients can be checked in real-time or near real-time against a CRL? Also, can Bro NSM perform a OCSP request to a RA and check a certificate to determine if it is valid? If this is the case where in Bro NSM do you set the address for the RA or CRL responder?
>
> Sincerely,
>
> Tony
>
>
> Tony Waller, CISSP
> Director, Systems Engineering
> Bivio Networks, Inc.
> ?Powering Advanced Cyber Operations? (TM)
> Mobile (443) 994-0936
>
> [cid:image001.png at 01D27E32.049899E0]
>
> *Note: The information contained in this email confidential. This information is intended only for the individual, individuals or entity to whom it is addressed. If you are not the intended recipient(s), the employee or agent responsible for delivering it to the intended recipient(s), you are hereby notified that any use, dissemination, distribution or copying of this communication is strictly prohibited. If you have received this email in error, please return the original message to us by email and delete or destroy any copies. Please note any views or opinions expressed or presented in this email are solely those of the author and do not necessarily represent those of Bivio Networks, Inc. The recipient should check this email or any attachments for the presence of viruses or malware. Bivio Networks, Inc. accepts no responsibility for any damage caused by any virus or malware transmitted by this email. Thank you.
>
> Think Green when printing
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 800 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170209/418d68fa/attachment-0001.bin 

From jdopheid at illinois.edu  Thu Feb  9 13:19:03 2017
From: jdopheid at illinois.edu (Dopheide, Jeannette M)
Date: Thu, 9 Feb 2017 21:19:03 +0000
Subject: [Bro] 7-question survey about professional Bro Training
In-Reply-To: <D8DC4ECF-6DD7-4E4F-9E93-71EC2097F66F@illinois.edu>
References: <D8DC4ECF-6DD7-4E4F-9E93-71EC2097F66F@illinois.edu>
Message-ID: <E1DD9B3F-0B12-45B9-A9C3-042C5EF7279D@illinois.edu>

Hello again, last call for filling out this survey, we?ll close it tomorrow (Friday) at noon CST. 

------
Jeannette Dopheide
Training and Outreach Coordinator
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
 

On 2/8/17, 3:27 PM, "bro-bounces at bro.org on behalf of Dopheide, Jeannette M" <bro-bounces at bro.org on behalf of jdopheid at illinois.edu> wrote:

Hello, 

Pinging this thread again to remind folks to fill out this survey. We?re getting some really great feedback already. 

------
Jeannette Dopheide
Training and Outreach Coordinator
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
 

On 2/7/17, 3:17 PM, "bro-bounces at bro.org on behalf of Dopheide, Jeannette M" <bro-bounces at bro.org on behalf of jdopheid at illinois.edu> wrote:

Hello Bro Community,

We are attempting to gauge interest in professional Bro training, potentially with corresponding certification officially approved by the Bro Project. Let us know about your needs, or the needs of your staff, by filling out this 7-question survey:

https://goo.gl/forms/Cm2MZRYexAzYAQrk2

Thank you,


The Bro Project

------
Jeannette Dopheide
Training and Outreach Coordinator
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
 


_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



_______________________________________________
Bro mailing list
bro at bro-ids.org
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From wren3 at illinois.edu  Thu Feb  9 13:20:59 2017
From: wren3 at illinois.edu (Ren, Wenyu)
Date: Thu, 9 Feb 2017 21:20:59 +0000
Subject: [Bro] Question about installing broker
Message-ID: <D1DD2D198D059443B177F461727B015953E23529@CHIMBX6.ad.uillinois.edu>

Dear all,

I have problem installing Broker. I installed CAF from the source. When I tried to install Broker, I got the following error.

Build Directory : build
Source Directory: /home/rwy/Downloads/broker
-- Found CAF: /usr/local/lib/libcaf_core.so;/usr/local/lib/libcaf_io.so  found components:  core io 
CMake Error at CMakeLists.txt:21 (message):
  Broker requires CAF version 0.14, older and newer versions are not
  supported.  Detected version: 0.15.3


-- Configuring incomplete, errors occurred!
See also "/home/rwy/Downloads/broker/build/CMakeFiles/CMakeOutput.log".

It seems that Broker only support CAF version 0.14 but not any higher version. Is there a workaround for this? Or I need to find a lower version of CAF to install?

And all the examples in Broker's documents are in C. Are there any python examples to show how to use Broker in python?

Thanks a lot.

Wenyu

From espressobeanies at gmail.com  Thu Feb  9 13:26:01 2017
From: espressobeanies at gmail.com (Espresso Beanies)
Date: Thu, 9 Feb 2017 16:26:01 -0500
Subject: [Bro] Question on redefining Bro variables
Message-ID: <CAOeD16-CMpUfiKnGbZJrgy9yaRZF7tNkR3Usdw9c=WRks85J_A@mail.gmail.com>

Good afternoon,

Where would it be best to redefine Bro variables? Should they all be in the
local.bro file or individual main.bro files for each module?

Thanks in advance,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170209/033f2d28/attachment.html 

From anthony.kasza at gmail.com  Thu Feb  9 13:50:06 2017
From: anthony.kasza at gmail.com (anthony kasza)
Date: Thu, 9 Feb 2017 14:50:06 -0700
Subject: [Bro] Question on redefining Bro variables
In-Reply-To: <CAOeD16-CMpUfiKnGbZJrgy9yaRZF7tNkR3Usdw9c=WRks85J_A@mail.gmail.com>
References: <CAOeD16-CMpUfiKnGbZJrgy9yaRZF7tNkR3Usdw9c=WRks85J_A@mail.gmail.com>
Message-ID: <CAEZw2bygADw9nEAQ1jCn2NwbFAjM3f-duZMWixBbXWBL0qWGEg@mail.gmail.com>

Personally, I make my own directory of scripts and modules and @load them
from the local.bro file.

-AK

On Feb 9, 2017 2:33 PM, "Espresso Beanies" <espressobeanies at gmail.com>
wrote:

> Good afternoon,
>
> Where would it be best to redefine Bro variables? Should they all be in
> the local.bro file or individual main.bro files for each module?
>
> Thanks in advance,
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170209/041de9ff/attachment.html 

From dnthayer at illinois.edu  Thu Feb  9 13:59:09 2017
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Thu, 9 Feb 2017 15:59:09 -0600
Subject: [Bro] Question about installing broker
In-Reply-To: <D1DD2D198D059443B177F461727B015953E23529@CHIMBX6.ad.uillinois.edu>
References: <D1DD2D198D059443B177F461727B015953E23529@CHIMBX6.ad.uillinois.edu>
Message-ID: <6c8eef0e-982a-1d9f-0119-c6dce1807962@illinois.edu>

You can download older versions of CAF from this page:
https://github.com/actor-framework/actor-framework/releases



On 2/9/17 3:20 PM, Ren, Wenyu wrote:
> Dear all,
>
> I have problem installing Broker. I installed CAF from the source. When I tried to install Broker, I got the following error.
>
> Build Directory : build
> Source Directory: /home/rwy/Downloads/broker
> -- Found CAF: /usr/local/lib/libcaf_core.so;/usr/local/lib/libcaf_io.so  found components:  core io
> CMake Error at CMakeLists.txt:21 (message):
>   Broker requires CAF version 0.14, older and newer versions are not
>   supported.  Detected version: 0.15.3
>
>
> -- Configuring incomplete, errors occurred!
> See also "/home/rwy/Downloads/broker/build/CMakeFiles/CMakeOutput.log".
>
> It seems that Broker only support CAF version 0.14 but not any higher version. Is there a workaround for this? Or I need to find a lower version of CAF to install?
>
> And all the examples in Broker's documents are in C. Are there any python examples to show how to use Broker in python?
>
> Thanks a lot.
>
> Wenyu


From deshmukh at slac.stanford.edu  Fri Feb 10 10:29:47 2017
From: deshmukh at slac.stanford.edu (Deshmukh, Andy)
Date: Fri, 10 Feb 2017 18:29:47 +0000
Subject: [Bro] Netmap plugin issue
In-Reply-To: <237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
	<237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
Message-ID: <7e14f24d17134bdbb13a4e41a8ec9e87@exch13-mail04.win.slac.stanford.edu>

Yes, the issue was resolved and was able to install the plugin. 
However, we have two 10 gig NICs on the Bro worker node and netmap cannot allocate memory for the second interface. 

[root at sec-bro04 ]# lb -i em1 -B 1024 -p broem1:10 
881.534306 main [637] interface is em1
881.534384 main [658] requested 1024 extra buffers
883.371747 main [772] successfully opened netmap:em1 (tx rings: 512)
883.371818 main [783] obtained 1024 extra buffers

[root at sec-bro04 ]# lb -i em2 -B 1024 -p broem2:10
014.454468 main [637] interface is em2
014.454555 main [658] requested 1024 extra buffers
014.501811 nm_open [920] NIOCREGIF failed: Cannot allocate memory em2
014.501828 main [768] cannot open netmap:em2

-----Original Message-----
From: Seth Hall [mailto:seth at icir.org] 
Sent: Wednesday, February 8, 2017 7:17 PM
To: Deshmukh, Andy <deshmukh at slac.stanford.edu>
Cc: bro at bro.org
Subject: Re: [Bro] Netmap plugin issue


> On Feb 8, 2017, at 5:55 PM, Deshmukh, Andy <deshmukh at slac.stanford.edu> wrote:
> 
> -- Could NOT find Netmap (missing:  NETMAP_INCLUDE_DIR) CMake Error at 
> CMakeLists.txt:20 (message):
>   Netmap headers not found.

Unfortunately in the plugin that comes with 2.5, you need netmap installed on the system you're building on.  We're going to be making changes for the 2.6 release so that nothing is required from netmap at build time and it is just built by default and included in Bro.

If you install netmap, does the problem go away?

  .Seth


--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From deshmukh at slac.stanford.edu  Fri Feb 10 15:43:25 2017
From: deshmukh at slac.stanford.edu (Deshmukh, Andy)
Date: Fri, 10 Feb 2017 23:43:25 +0000
Subject: [Bro] Netmap plugin issue
In-Reply-To: <F91195A4-A35D-48B8-B893-63E66E8821A4@pingtrip.com>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
	<237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
	<7e14f24d17134bdbb13a4e41a8ec9e87@exch13-mail04.win.slac.stanford.edu>
	<F91195A4-A35D-48B8-B893-63E66E8821A4@pingtrip.com>
Message-ID: <6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>

In my case only 5 instances are running per NIC; cannot run 10/NIC as it crashes. I modified the lb_procs to 5 in node.cfg.
However, I am not seeing any packet forwarded or dropped. Do you see that on the running instances ?

From: Dave Crawford [mailto:dave at pingtrip.com]
Sent: Friday, February 10, 2017 3:20 PM
To: Deshmukh, Andy <deshmukh at slac.stanford.edu>
Cc: Seth Hall <seth at icir.org>; bro at bro.org
Subject: Re: [Bro] Netmap plugin issue


On Feb 10, 2017, at 1:29 PM, Deshmukh, Andy <deshmukh at slac.stanford.edu<mailto:deshmukh at slac.stanford.edu>> wrote:

Yes, the issue was resolved and was able to install the plugin.
However, we have two 10 gig NICs on the Bro worker node and netmap cannot allocate memory for the second interface.

[root at sec-bro04 ]# lb -i em1 -B 1024 -p broem1:10
881.534306 main [637] interface is em1
881.534384 main [658] requested 1024 extra buffers
883.371747 main [772] successfully opened netmap:em1 (tx rings: 512)
883.371818 main [783] obtained 1024 extra buffers

[root at sec-bro04 ]# lb -i em2 -B 1024 -p broem2:10
014.454468 main [637] interface is em2
014.454555 main [658] requested 1024 extra buffers
014.501811 nm_open [920] NIOCREGIF failed: Cannot allocate memory em2
014.501828 main [768] cannot open netmap:em2

I just ran into the same issue. I have an IGB and IXGBE NIC and some of the workers monitoring the IXGBE nic fail to start. From broctl drag:

113.929963 nm_open [920] NIOCREGIF failed: Cannot allocate memory bro}1
fatal error: problem with interface netmap::bro}1 (Cannot allocate memory)

MID_GLR is the IGB and 5 workers started without issue. Only 3 of 10 on the IXGBE interface started:

Name         Type    Host             Status    Pid    Started
MGR_INT      manager x.y.217.48    running   7697   10 Feb 18:08:29
MID_INT_PXY_1 proxy   x.y.5.149     running   7086   10 Feb 18:08:31
MID_GLR-1    worker  x.y.5.149     running   7265   10 Feb 18:08:32
MID_GLR-2    worker  x.y.5.149     running   7270   10 Feb 18:08:32
MID_GLR-3    worker  x.y.5.149     running   7296   10 Feb 18:08:32
MID_GLR-4    worker  x.y.5.149     running   7292   10 Feb 18:08:32
MID_GLR-5    worker  x.y.5.149     running   7302   10 Feb 18:08:32
MID_INT-1    worker  x.y.5.149     crashed
MID_INT-10   worker  x.y.5.149     crashed
MID_INT-2    worker  x.y.5.149     crashed
MID_INT-3    worker  x.y.5.149     running   7331   10 Feb 18:08:32
MID_INT-4    worker  x.y.5.149     crashed
MID_INT-5    worker  x.y.5.149     running   7339   10 Feb 18:08:32
MID_INT-6    worker  x.y.5.149     running   7345   10 Feb 18:08:32
MID_INT-7    worker  x.y.5.149     crashed
MID_INT-8    worker  x.y.5.149     crashed
MID_INT-9    worker  x.y.5.149     crashed


-Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170210/61a6da34/attachment.html 

From bro at pingtrip.com  Fri Feb 10 15:47:59 2017
From: bro at pingtrip.com (Dave Crawford)
Date: Fri, 10 Feb 2017 18:47:59 -0500
Subject: [Bro] Netmap plugin issue
In-Reply-To: <6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
	<237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
	<7e14f24d17134bdbb13a4e41a8ec9e87@exch13-mail04.win.slac.stanford.edu>
	<F91195A4-A35D-48B8-B893-63E66E8821A4@pingtrip.com>
	<6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>
Message-ID: <A736CBF1-2D4B-4E0D-9E1B-FE35C4017FE0@pingtrip.com>


> On Feb 10, 2017, at 6:43 PM, Deshmukh, Andy <deshmukh at slac.stanford.edu> wrote:
> 
> In my case only 5 instances are running per NIC; cannot run 10/NIC as it crashes. I modified the lb_procs to 5 in node.cfg.
> However, I am not seeing any packet forwarded or dropped. Do you see that on the running instances ?

I can get 10 instances to run on the x520 if I comment out the IGB worker in node.cfg. I?m wondering if the issue has to due with memory allocations done when the netmap kernel module loads. Do we need to tweak them in modprobe.d to account for two instances requesting X number of buffers?

"data_forward_rate_Mbps": 330.0390,
 "data_drop_rate_Mbps": 0.0000,
 "packet_forward_rate_kpps": 71.5200,
 "packet_drop_rate_kpps": 0.0000,

-Dave 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170210/b858bd49/attachment.html 

From deshmukh at slac.stanford.edu  Fri Feb 10 16:05:41 2017
From: deshmukh at slac.stanford.edu (Deshmukh, Andy)
Date: Sat, 11 Feb 2017 00:05:41 +0000
Subject: [Bro] Netmap plugin issue
In-Reply-To: <A736CBF1-2D4B-4E0D-9E1B-FE35C4017FE0@pingtrip.com>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
	<237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
	<7e14f24d17134bdbb13a4e41a8ec9e87@exch13-mail04.win.slac.stanford.edu>
	<F91195A4-A35D-48B8-B893-63E66E8821A4@pingtrip.com>
	<6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>
	<A736CBF1-2D4B-4E0D-9E1B-FE35C4017FE0@pingtrip.com>
Message-ID: <e02cb86282cf40f786817ca0572c2559@exch13-mail04.win.slac.stanford.edu>

I did tweak it by putting this in /etc/modprobe.d/netmap.conf :
options netmap default_pipes=1000
options netmap ring_num=1024
options netmap buf_num=655360
options netmap if_num=1024
options netmap ring_size=100000
options netmap buf_size=4096
options netmap if_size=4096

However, when I run:
[sec-bro04 ~]$ lb -i em1 -B 1000 -p broem1:5 &
[sec-bro04 ~]$ lb -i em2 -B 1000 -p broem2:5 &

Ring stats :
Feb 10 16:03:54 sec-bro04 lb:
{"ts":1486771434.006337,"input_interface":"netmap:em1","output_interface":"netmap:broem1{0/xT at 1","packets_forwarded":0,"packets_dropped":0,"data_forward_rate_Mbps":0.0000,
"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":0.0000,"packet_drop_rate_kpps":0.0000,"overflow_queue_size":0}
Feb 10 16:03:54 sec-bro04 lb: {"ts":1486771434.668234,"input_interface":"netmap:em2","output_interface":"netmap:broem2{0/xT at 1","packets_forwarded":0,"packets_dropped":0,"data_forward_rate_Mbps":0.0000,
"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":0.0000,"packet_drop_rate_kpps":0.0000,"overflow_queue_size":0}

From: Dave Crawford [mailto:bro at pingtrip.com]
Sent: Friday, February 10, 2017 3:48 PM
To: Deshmukh, Andy <deshmukh at slac.stanford.edu>
Cc: Seth Hall <seth at icir.org>; bro at bro.org
Subject: Re: [Bro] Netmap plugin issue


On Feb 10, 2017, at 6:43 PM, Deshmukh, Andy <deshmukh at slac.stanford.edu<mailto:deshmukh at slac.stanford.edu>> wrote:

In my case only 5 instances are running per NIC; cannot run 10/NIC as it crashes. I modified the lb_procs to 5 in node.cfg.
However, I am not seeing any packet forwarded or dropped. Do you see that on the running instances ?

I can get 10 instances to run on the x520 if I comment out the IGB worker in node.cfg. I?m wondering if the issue has to due with memory allocations done when the netmap kernel module loads. Do we need to tweak them in modprobe.d to account for two instances requesting X number of buffers?

"data_forward_rate_Mbps": 330.0390,
 "data_drop_rate_Mbps": 0.0000,
 "packet_forward_rate_kpps": 71.5200,
 "packet_drop_rate_kpps": 0.0000,

-Dave

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170211/f4fb2ca6/attachment-0001.html 

From seth at icir.org  Fri Feb 10 18:53:49 2017
From: seth at icir.org (Seth Hall)
Date: Fri, 10 Feb 2017 21:53:49 -0500
Subject: [Bro] Netmap plugin issue
In-Reply-To: <6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
	<237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
	<7e14f24d17134bdbb13a4e41a8ec9e87@exch13-mail04.win.slac.stanford.edu>
	<F91195A4-A35D-48B8-B893-63E66E8821A4@pingtrip.com>
	<6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>
Message-ID: <35224580-AE4F-45AE-8046-303758FF9A9E@icir.org>


> On Feb 10, 2017, at 6:43 PM, Deshmukh, Andy <deshmukh at slac.stanford.edu> wrote:
> 
> In my case only 5 instances are running per NIC; cannot run 10/NIC as it crashes. I modified the lb_procs to 5 in node.cfg.
> However, I am not seeing any packet forwarded or dropped. Do you see that on the running instances ?

Netmap doesn't currently mark interfaces as promiscuous when it connects.  If you manually mark the interface promisc, do you get packets?

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From deshmukh at slac.stanford.edu  Fri Feb 10 19:07:00 2017
From: deshmukh at slac.stanford.edu (Deshmukh, Andy)
Date: Sat, 11 Feb 2017 03:07:00 +0000
Subject: [Bro] Netmap plugin issue
In-Reply-To: <35224580-AE4F-45AE-8046-303758FF9A9E@icir.org>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
	<237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
	<7e14f24d17134bdbb13a4e41a8ec9e87@exch13-mail04.win.slac.stanford.edu>
	<F91195A4-A35D-48B8-B893-63E66E8821A4@pingtrip.com>
	<6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>
	<35224580-AE4F-45AE-8046-303758FF9A9E@icir.org>
Message-ID: <95315aa48ece4e8696e229da0879d445@exch13-mail04.win.slac.stanford.edu>

Yea!! I totally forgot about that... 
But the packet_drop is very high! Is it cause of the tweaks in modprobe.d ? 

Feb 10 19:04:59 sec-bro04 lb: {"ts":1486782299.253931,"input_interface":"netmap:em1","output_interface":"netmap:broem1{0/xT at 1","packets_forwarded":66870,"packets_dropped":112000,"data_forward_rate_Mbps":9.5147,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":0.9570,"packet_drop_rate_kpps":0.0000,"overflow_queue_size":0}
Feb 10 19:04:59 sec-bro04 lb: {"ts":1486782299.253931,"input_interface":"netmap:em1","output_interface":"netmap:broem1{1/xT at 1","packets_forwarded":56280,"packets_dropped":90700,"data_forward_rate_Mbps":11.9930,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":1.5820,"packet_drop_rate_kpps":0.0000,"overflow_queue_size":0}
Feb 10 19:04:59 sec-bro04 lb: {"ts":1486782299.253931,"input_interface":"netmap:em1","output_interface":"netmap:broem1{2/xT at 1","packets_forwarded":51795,"packets_dropped":83500,"data_forward_rate_Mbps":10.2431,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":0.8180,"packet_drop_rate_kpps":0.0000,"overflow_queue_size":0}
Feb 10 19:04:59 sec-bro04 lb: {"ts":1486782299.253931,"input_interface":"netmap:em1","output_interface":"netmap:broem1{3/xT at 1","packets_forwarded":535398,"packets_dropped":924500,"data_forward_rate_Mbps":175.7472,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":13.4440,"packet_drop_rate_kpps":0.0000,"overflow_queue_size":0}
Feb 10 19:04:59 sec-bro04 lb: {"ts":1486782299.253931,"input_interface":"netmap:em1","output_interface":"netmap:broem1{4/xT at 1","packets_forwarded":228411,"packets_dropped":370900,"data_forward_rate_Mbps":13.0369,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":2.8260,"packet_drop_rate_kpps":0.0000,"overflow_queue_size":0}
Feb 10 19:04:59 sec-bro04 lb: {"ts":1486782299.253931,"interface":"netmap:em1","packets_received":2522287,"packets_forwarded":938754,"packets_dropped":1581600,"non_ip_packets":54442,"data_forward_rate_Mbps":220.5349,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":19.6260,"packet_drop_rate_kpps":0.0000,"free_buffer_slots":1000}
Feb 10 19:04:59 sec-bro04 lb: {"ts":1486782299.255037,"input_interface":"netmap:em2","output_interface":"netmap:broem2{0/xT at 1","packets_forwarded":200176,"packets_dropped":297700,"data_forward_rate_Mbps":3.0338,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":2.2230,"packet_drop_rate_kpps":0.0000,"overflow_queue_size":0}
Feb 10 19:04:59 sec-bro04 lb: {"ts":1486782299.255037,"input_interface":"netmap:em2","output_interface":"netmap:broem2{1/xT at 1","packets_forwarded":18104,"packets_dropped":46000,"data_forward_rate_Mbps":0.1089,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":0.0960,"packet_drop_rate_kpps":0.0000,"overflow_queue_size":0}
Feb 10 19:04:59 sec-bro04 lb: {"ts":1486782299.255037,"input_interface":"netmap:em2","output_interface":"netmap:broem2{2/xT at 1","packets_forwarded":18740,"packets_dropped":20400,"data_forward_rate_Mbps":0.5375,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":0.4710,"packet_drop_rate_kpps":0.0000,"overflow_queue_size":0}
Feb 10 19:04:59 sec-bro04 lb: {"ts":1486782299.255037,"input_interface":"netmap:em2","output_interface":"netmap:broem2{3/xT at 1","packets_forwarded":26775,"packets_dropped":31500,"data_forward_rate_Mbps":0.0908,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":0.0700,"packet_drop_rate_kpps":0.0000,"overflow_queue_size":0}
Feb 10 19:04:59 sec-bro04 lb: {"ts":1486782299.255037,"input_interface":"netmap:em2","output_interface":"netmap:broem2{4/xT at 1","packets_forwarded":235348,"packets_dropped":358200,"data_forward_rate_Mbps":3.4327,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":2.6540,"packet_drop_rate_kpps":0.0000,"overflow_queue_size":0}
Feb 10 19:04:59 sec-bro04 lb: {"ts":1486782299.255037,"interface":"netmap:em2","packets_received":1255048,"packets_forwarded":499143,"packets_dropped":753800,"non_ip_packets":47454,"data_forward_rate_Mbps":7.2036,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":5.5130,"packet_drop_rate_kpps":0.0000,"free_buffer_slots":1000}

-----Original Message-----
From: Seth Hall [mailto:seth at icir.org] 
Sent: Friday, February 10, 2017 6:54 PM
To: Deshmukh, Andy <deshmukh at slac.stanford.edu>
Cc: Dave Crawford <dave at pingtrip.com>; bro at bro.org
Subject: Re: [Bro] Netmap plugin issue


> On Feb 10, 2017, at 6:43 PM, Deshmukh, Andy <deshmukh at slac.stanford.edu> wrote:
> 
> In my case only 5 instances are running per NIC; cannot run 10/NIC as it crashes. I modified the lb_procs to 5 in node.cfg.
> However, I am not seeing any packet forwarded or dropped. Do you see that on the running instances ?

Netmap doesn't currently mark interfaces as promiscuous when it connects.  If you manually mark the interface promisc, do you get packets?

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From bro at pingtrip.com  Fri Feb 10 20:24:51 2017
From: bro at pingtrip.com (Dave Crawford)
Date: Fri, 10 Feb 2017 23:24:51 -0500
Subject: [Bro] Netmap plugin issue
In-Reply-To: <95315aa48ece4e8696e229da0879d445@exch13-mail04.win.slac.stanford.edu>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
	<237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
	<7e14f24d17134bdbb13a4e41a8ec9e87@exch13-mail04.win.slac.stanford.edu>
	<F91195A4-A35D-48B8-B893-63E66E8821A4@pingtrip.com>
	<6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>
	<35224580-AE4F-45AE-8046-303758FF9A9E@icir.org>
	<95315aa48ece4e8696e229da0879d445@exch13-mail04.win.slac.stanford.edu>
Message-ID: <34442F24-0A8B-4EE7-A835-52707EA20478@pingtrip.com>


> On Feb 10, 2017, at 10:07 PM, Deshmukh, Andy <deshmukh at slac.stanford.edu> wrote:
> 
> Yea!! I totally forgot about that... 
> But the packet_drop is very high! Is it cause of the tweaks in modprobe.d ? 

I have the same observations Andy.

{"ts":1486786393.408004,"interface":"netmap:eth6","packets_received":3816916,"packets_forwarded":2495606,"packets_dropped":1213100,"non_ip_packets":2014,"data_forward_rate_Mbps":93.6190,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":29.4030,"packet_drop_rate_kpps":0.0000,"free_buffer_slots":100000}

Wouldn?t a 30% packet loss result in a high number of weird.log messages as well as a high capture_loss? Bro is reporting under 1% for that worker.

1486786916.930127       600.000012      MID_INT-9       21      120263  0.017462
1486786916.899102       600.000025      MID_INT-2       26      113792  0.022849
1486786916.913207       600.000046      MID_INT-5       17      114020  0.01491
1486786917.062056       600.000040      MID_INT-1       37      122988  0.030084
1486786916.899164       600.000046      MID_INT-6       20      117978  0.016952
1486786916.898184       600.000043      MID_INT-8       10      117535  0.008508
1486786916.899106       600.000023      MID_INT-10      31      135819  0.022824
1486786916.899611       600.000023      MID_INT-3       19      130912  0.014514
1486786916.902911       600.000014      MID_INT-7       24      144454  0.016614
1486786916.897984       600.000029      MID_INT-4       25      106400  0.023496

-Dave



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170210/b2a5cb66/attachment.html 

From deshmukh at slac.stanford.edu  Sat Feb 11 10:35:16 2017
From: deshmukh at slac.stanford.edu (Deshmukh, Andy)
Date: Sat, 11 Feb 2017 18:35:16 +0000
Subject: [Bro] Netmap plugin issue
In-Reply-To: <34442F24-0A8B-4EE7-A835-52707EA20478@pingtrip.com>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
	<237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
	<7e14f24d17134bdbb13a4e41a8ec9e87@exch13-mail04.win.slac.stanford.edu>
	<F91195A4-A35D-48B8-B893-63E66E8821A4@pingtrip.com>
	<6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>
	<35224580-AE4F-45AE-8046-303758FF9A9E@icir.org>
	<95315aa48ece4e8696e229da0879d445@exch13-mail04.win.slac.stanford.edu>
	<34442F24-0A8B-4EE7-A835-52707EA20478@pingtrip.com>
Message-ID: <fb7aab1d41564392b45268b7c3594163@exch13-mail04.win.slac.stanford.edu>

Hmm, that?s interesting. For me Bro is reporting the capture loss which kind of matches the overall netmap stats and it is very high:

1486814690.289371       900.000065      sec-bro04-1-1   4983       28153   17.699712
1486814690.296848       900.000132      sec-bro04-1-5   29050     69353   41.887157
1486814690.283136       900.000080      sec-bro04-1-4   221424  242109 91.456328
1486814690.315410       900.000052      sec-bro04-1-2   26613     65599   40.569216
1486814690.300392       900.000025      sec-bro04-1-3   7591       34491   22.00864
1486815590.289398       900.000027      sec-bro04-1-1   15437     42078   36.68663
1486815590.315530       900.000120      sec-bro04-1-2   913          9650      9.46114
1486815590.283290       900.000154      sec-bro04-1-4   40906     49390   82.822434

From: Dave Crawford [mailto:bro at pingtrip.com]
Sent: Friday, February 10, 2017 8:25 PM
To: Deshmukh, Andy <deshmukh at slac.stanford.edu>
Cc: Seth Hall <seth at icir.org>; bro at bro.org
Subject: Re: [Bro] Netmap plugin issue


On Feb 10, 2017, at 10:07 PM, Deshmukh, Andy <deshmukh at slac.stanford.edu<mailto:deshmukh at slac.stanford.edu>> wrote:

Yea!! I totally forgot about that...
But the packet_drop is very high! Is it cause of the tweaks in modprobe.d ?

I have the same observations Andy.

{"ts":1486786393.408004,"interface":"netmap:eth6","packets_received":3816916,"packets_forwarded":2495606,"packets_dropped":1213100,"non_ip_packets":2014,"data_forward_rate_Mbps":93.6190,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":29.4030,"packet_drop_rate_kpps":0.0000,"free_buffer_slots":100000}

Wouldn?t a 30% packet loss result in a high number of weird.log messages as well as a high capture_loss? Bro is reporting under 1% for that worker.

1486786916.930127       600.000012      MID_INT-9       21      120263  0.017462
1486786916.899102       600.000025      MID_INT-2       26      113792  0.022849
1486786916.913207       600.000046      MID_INT-5       17      114020  0.01491
1486786917.062056       600.000040      MID_INT-1       37      122988  0.030084
1486786916.899164       600.000046      MID_INT-6       20      117978  0.016952
1486786916.898184       600.000043      MID_INT-8       10      117535  0.008508
1486786916.899106       600.000023      MID_INT-10      31      135819  0.022824
1486786916.899611       600.000023      MID_INT-3       19      130912  0.014514
1486786916.902911       600.000014      MID_INT-7       24      144454  0.016614
1486786916.897984       600.000029      MID_INT-4       25      106400  0.023496

-Dave



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170211/6e196507/attachment-0001.html 

From seth at icir.org  Sat Feb 11 18:12:04 2017
From: seth at icir.org (Seth Hall)
Date: Sat, 11 Feb 2017 21:12:04 -0500
Subject: [Bro] Netmap plugin issue
In-Reply-To: <34442F24-0A8B-4EE7-A835-52707EA20478@pingtrip.com>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
	<237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
	<7e14f24d17134bdbb13a4e41a8ec9e87@exch13-mail04.win.slac.stanford.edu>
	<F91195A4-A35D-48B8-B893-63E66E8821A4@pingtrip.com>
	<6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>
	<35224580-AE4F-45AE-8046-303758FF9A9E@icir.org>
	<95315aa48ece4e8696e229da0879d445@exch13-mail04.win.slac.stanford.edu>
	<34442F24-0A8B-4EE7-A835-52707EA20478@pingtrip.com>
Message-ID: <34BAF267-69FE-4DF9-8516-B4689DCB3819@icir.org>


> On Feb 10, 2017, at 11:24 PM, Dave Crawford <bro at pingtrip.com> wrote:
> 
> {"ts":1486786393.408004,"interface":"netmap:eth6","packets_received":3816916,"packets_forwarded":2495606,"packets_dropped":1213100,"non_ip_packets":2014,"data_forward_rate_Mbps":93.6190,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":29.4030,"packet_drop_rate_kpps":0.0000,"free_buffer_slots":100000}

What's more interesting for me here is that the packet_drop_rate_kpps is 0.  What could have happened is that the packets were lost because Bro processes hadn't been started yet.  It seems to me from this line is that ~100Mbps of traffic is flowing with no packets being lost.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Sat Feb 11 18:13:29 2017
From: seth at icir.org (Seth Hall)
Date: Sat, 11 Feb 2017 21:13:29 -0500
Subject: [Bro] Netmap plugin issue
In-Reply-To: <fb7aab1d41564392b45268b7c3594163@exch13-mail04.win.slac.stanford.edu>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
	<237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
	<7e14f24d17134bdbb13a4e41a8ec9e87@exch13-mail04.win.slac.stanford.edu>
	<F91195A4-A35D-48B8-B893-63E66E8821A4@pingtrip.com>
	<6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>
	<35224580-AE4F-45AE-8046-303758FF9A9E@icir.org>
	<95315aa48ece4e8696e229da0879d445@exch13-mail04.win.slac.stanford.edu>
	<34442F24-0A8B-4EE7-A835-52707EA20478@pingtrip.com>
	<fb7aab1d41564392b45268b7c3594163@exch13-mail04.win.slac.stanford.edu>
Message-ID: <03503572-1528-40F0-9A0F-E686F0AFA67F@icir.org>


> On Feb 11, 2017, at 1:35 PM, Deshmukh, Andy <deshmukh at slac.stanford.edu> wrote:
> 
> Hmm, that?s interesting. For me Bro is reporting the capture loss which kind of matches the overall netmap stats and it is very high:
>  
> 1486814690.289371       900.000065      sec-bro04-1-1   4983       28153   17.699712
> 1486814690.296848       900.000132      sec-bro04-1-5   29050     69353   41.887157
> 1486814690.283136       900.000080      sec-bro04-1-4   221424  242109 91.456328
> 1486814690.315410       900.000052      sec-bro04-1-2   26613     65599   40.569216
> 1486814690.300392       900.000025      sec-bro04-1-3   7591       34491   22.00864
> 1486815590.289398       900.000027      sec-bro04-1-1   15437     42078   36.68663
> 1486815590.315530       900.000120      sec-bro04-1-2   913          9650      9.46114
> 1486815590.283290       900.000154      sec-bro04-1-4   40906     49390   82.822434

During the time where the data was collected for this capture-loss log, what was the output of lb showing?  Did it show any bursts of loss?

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From deshmukh at slac.stanford.edu  Sat Feb 11 21:58:04 2017
From: deshmukh at slac.stanford.edu (Deshmukh, Andy)
Date: Sun, 12 Feb 2017 05:58:04 +0000
Subject: [Bro] Netmap plugin issue
In-Reply-To: <03503572-1528-40F0-9A0F-E686F0AFA67F@icir.org>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
	<237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
	<7e14f24d17134bdbb13a4e41a8ec9e87@exch13-mail04.win.slac.stanford.edu>
	<F91195A4-A35D-48B8-B893-63E66E8821A4@pingtrip.com>
	<6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>
	<35224580-AE4F-45AE-8046-303758FF9A9E@icir.org>
	<95315aa48ece4e8696e229da0879d445@exch13-mail04.win.slac.stanford.edu>
	<34442F24-0A8B-4EE7-A835-52707EA20478@pingtrip.com>
	<fb7aab1d41564392b45268b7c3594163@exch13-mail04.win.slac.stanford.edu>
	<03503572-1528-40F0-9A0F-E686F0AFA67F@icir.org>
Message-ID: <6165d3b4ddbc483996206c2dfc2f17ad@exch13-mail04.win.slac.stanford.edu>

Capture_loss event: 
1486803890.282458	900.000048	sec-bro04-1-4	28622	35953	79.60949 

Lb logs: 
Feb 11 01:04:50 sec-bro04 lb: {"ts":1486803890.238885,"input_interface":"netmap:em1","output_interface":"netmap:broem1{4/xT at 1","packets_forwarded":6664358,"packets_dropped":177568,"data_forward_rate_Mbps":3.1411,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":2.0600,"packet_drop_rate_kpps":0.0000,"overflow_queue_size":0}

On other interfaces it does show some loss but nothing substantial. 

-----Original Message-----
From: Seth Hall [mailto:seth at icir.org] 
Sent: Saturday, February 11, 2017 6:13 PM
To: Deshmukh, Andy <deshmukh at slac.stanford.edu>
Cc: Dave Crawford <bro at pingtrip.com>; bro at bro.org
Subject: Re: [Bro] Netmap plugin issue


> On Feb 11, 2017, at 1:35 PM, Deshmukh, Andy <deshmukh at slac.stanford.edu> wrote:
> 
> Hmm, that?s interesting. For me Bro is reporting the capture loss which kind of matches the overall netmap stats and it is very high:
>  
> 1486814690.289371       900.000065      sec-bro04-1-1   4983       28153   17.699712
> 1486814690.296848       900.000132      sec-bro04-1-5   29050     69353   41.887157
> 1486814690.283136       900.000080      sec-bro04-1-4   221424  242109 91.456328
> 1486814690.315410       900.000052      sec-bro04-1-2   26613     65599   40.569216
> 1486814690.300392       900.000025      sec-bro04-1-3   7591       34491   22.00864
> 1486815590.289398       900.000027      sec-bro04-1-1   15437     42078   36.68663
> 1486815590.315530       900.000120      sec-bro04-1-2   913          9650      9.46114
> 1486815590.283290       900.000154      sec-bro04-1-4   40906     49390   82.822434

During the time where the data was collected for this capture-loss log, what was the output of lb showing?  Did it show any bursts of loss?

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From pssunu6 at gmail.com  Sun Feb 12 01:31:21 2017
From: pssunu6 at gmail.com (ps sunu)
Date: Sun, 12 Feb 2017 15:01:21 +0530
Subject: [Bro] bro_intel feeds as csv file
Message-ID: <CALGAZaCnRTXenOVND3mrS3knS1k8JxaRJ2YRVOOv7BmfL3nhfA@mail.gmail.com>

Hi all ,
                        Is possible to generate bro_intel  txt files as a
.CSV format ?

my format

#fields indicator indicator_type meta.source meta.url meta.do_notice
meta.if_in meta.whitelist
# Intel::ADDR binarydefense-ip - T - -
# Intel::ADDR binarydefense-ip - T - -
# Intel::ADDR binarydefense-ip - T - -
# Intel::ADDR binarydefense-ip - T - -
# Intel::ADDR binarydefense-ip - T - -
# Intel::ADDR binarydefense-ip - T - -
# Intel::ADDR binarydefense-ip - T - -


Need to change this feeds as a csv format


Regards,
Sunu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170212/5c993440/attachment.html 

From bro at pingtrip.com  Sun Feb 12 04:15:56 2017
From: bro at pingtrip.com (Dave Crawford)
Date: Sun, 12 Feb 2017 07:15:56 -0500
Subject: [Bro] Netmap plugin issue
In-Reply-To: <6165d3b4ddbc483996206c2dfc2f17ad@exch13-mail04.win.slac.stanford.edu>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
	<237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
	<7e14f24d17134bdbb13a4e41a8ec9e87@exch13-mail04.win.slac.stanford.edu>
	<F91195A4-A35D-48B8-B893-63E66E8821A4@pingtrip.com>
	<6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>
	<35224580-AE4F-45AE-8046-303758FF9A9E@icir.org>
	<95315aa48ece4e8696e229da0879d445@exch13-mail04.win.slac.stanford.edu>
	<34442F24-0A8B-4EE7-A835-52707EA20478@pingtrip.com>
	<fb7aab1d41564392b45268b7c3594163@exch13-mail04.win.slac.stanford.edu>
	<03503572-1528-40F0-9A0F-E686F0AFA67F@icir.org>
	<6165d3b4ddbc483996206c2dfc2f17ad@exch13-mail04.win.slac.stanford.edu>
Message-ID: <DEC916FE-77F1-4035-919C-3E36FF78346A@pingtrip.com>

My LB stats are similar, but Bro isn?t reflecting a loss:

Feb 11 04:58:02 mid-csignsm-01 lb[3144]: {"ts":1486807082.815681,"interface":"netmap:eth6","packets_received":739258266,"packets_forwarded":737718676,"packets_dropped":1213100,"non_ip_packets":371283,"data_forward_rate_Mbps":758.6328,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":102.7250,"packet_drop_rate_kpps":0.0000,"free_buffer_slots":100000}

The next cycle of capture_loss shows less than 1% loss for the 10 workers:

2017-02-11T05:01:56-0500        600.000119      MID_INT-2       5       102927  0.004858
2017-02-11T05:01:56-0500        600.000015      MID_INT-4       7       107577  0.006507
2017-02-11T05:01:56-0500        600.000053      MID_INT-9       6       101979  0.005884
2017-02-11T05:01:56-0500        600.000016      MID_INT-3       1       304887  0.000328
2017-02-11T05:01:56-0500        600.000543      MID_INT-10      5       186552  0.00268
2017-02-11T05:01:56-0500        600.000009      MID_INT-5       6       101433  0.005915
2017-02-11T05:01:56-0500        600.000005      MID_INT-6       5       110256  0.004535
2017-02-11T05:01:56-0500        600.000085      MID_INT-7       1       98164   0.001019
2017-02-11T05:01:56-0500        600.000041      MID_INT-8       4       99979   0.004001
2017-02-11T05:01:57-0500        600.000047      MID_INT-1       3       90591   0.003312

I also noticed that Andy?s LB output is slightly different. His displays the free buffers as ?overflow_queue_size? where my output is ?free_buffer_slots?.

Also Andy, your overflow queue size is ?0?, did you define one and its been depleted? Creating one, or increasing the size may help with the Bro dropped packets.

-Dave

> On Feb 12, 2017, at 12:58 AM, Deshmukh, Andy <deshmukh at slac.stanford.edu> wrote:
> 
> Capture_loss event: 
> 1486803890.282458	900.000048	sec-bro04-1-4	28622	35953	79.60949 
> 
> Lb logs: 
> Feb 11 01:04:50 sec-bro04 lb: {"ts":1486803890.238885,"input_interface":"netmap:em1","output_interface":"netmap:broem1{4/xT at 1","packets_forwarded":6664358,"packets_dropped":177568,"data_forward_rate_Mbps":3.1411,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":2.0600,"packet_drop_rate_kpps":0.0000,"overflow_queue_size":0}
> 
> On other interfaces it does show some loss but nothing substantial. 
> 
> -----Original Message-----
> From: Seth Hall [mailto:seth at icir.org] 
> Sent: Saturday, February 11, 2017 6:13 PM
> To: Deshmukh, Andy <deshmukh at slac.stanford.edu>
> Cc: Dave Crawford <bro at pingtrip.com>; bro at bro.org
> Subject: Re: [Bro] Netmap plugin issue
> 
> 
>> On Feb 11, 2017, at 1:35 PM, Deshmukh, Andy <deshmukh at slac.stanford.edu> wrote:
>> 
>> Hmm, that?s interesting. For me Bro is reporting the capture loss which kind of matches the overall netmap stats and it is very high:
>> 
>> 1486814690.289371       900.000065      sec-bro04-1-1   4983       28153   17.699712
>> 1486814690.296848       900.000132      sec-bro04-1-5   29050     69353   41.887157
>> 1486814690.283136       900.000080      sec-bro04-1-4   221424  242109 91.456328
>> 1486814690.315410       900.000052      sec-bro04-1-2   26613     65599   40.569216
>> 1486814690.300392       900.000025      sec-bro04-1-3   7591       34491   22.00864
>> 1486815590.289398       900.000027      sec-bro04-1-1   15437     42078   36.68663
>> 1486815590.315530       900.000120      sec-bro04-1-2   913          9650      9.46114
>> 1486815590.283290       900.000154      sec-bro04-1-4   40906     49390   82.822434
> 
> During the time where the data was collected for this capture-loss log, what was the output of lb showing?  Did it show any bursts of loss?
> 
>  .Seth
> 
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170212/743834da/attachment-0001.html 

From randy at psg.com  Sun Feb 12 05:45:26 2017
From: randy at psg.com (Randy Bush)
Date: Sun, 12 Feb 2017 22:45:26 +0900
Subject: [Bro] Netmap plugin issue
In-Reply-To: <DEC916FE-77F1-4035-919C-3E36FF78346A@pingtrip.com>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
	<237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
	<7e14f24d17134bdbb13a4e41a8ec9e87@exch13-mail04.win.slac.stanford.edu>
	<F91195A4-A35D-48B8-B893-63E66E8821A4@pingtrip.com>
	<6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>
	<35224580-AE4F-45AE-8046-303758FF9A9E@icir.org>
	<95315aa48ece4e8696e229da0879d445@exch13-mail04.win.slac.stanford.edu>
	<34442F24-0A8B-4EE7-A835-52707EA20478@pingtrip.com>
	<fb7aab1d41564392b45268b7c3594163@exch13-mail04.win.slac.stanford.edu>
	<03503572-1528-40F0-9A0F-E686F0AFA67F@icir.org>
	<6165d3b4ddbc483996206c2dfc2f17ad@exch13-mail04.win.slac.stanford.edu>
	<DEC916FE-77F1-4035-919C-3E36FF78346A@pingtrip.com>
Message-ID: <m2wpcvo749.wl-randy@psg.com>

[ n00b.  well i ran bro over a decade ago. ]

a ganeti cluster running ganeti 2.15 on deb8 and ubuntu16

i run bro in a vm on one of the nodes.  as it is on the bridged lan, it
sees all the traffic to all vms whose primary is on the same node.
this is sweet.  but i want to see the traffic to the vms whose primary
are on the other nodes.

so what is the minimial hack i can run on other nodes to stream pcaps
to that bro instance so that the whole cluster is feeding to one bro
instance?  i would prefer a simple hack to run on the host opsys, but
could create more guest vms iff i had to.

the cluster has a second inter-node lan i could use to avoid pcapping
the pcap transport.

[ no, i prefer not to mirror off the switch ]

randy

From randy at psg.com  Sun Feb 12 05:46:22 2017
From: randy at psg.com (Randy Bush)
Date: Sun, 12 Feb 2017 22:46:22 +0900
Subject: [Bro] apologies for coopting another thread
Message-ID: <m2vasfo72p.wl-randy@psg.com>

On Sun, 12 Feb 2017 22:45:26 +0900,
Randy Bush wrote:
> 
> [ n00b.  well i ran bro over a decade ago. ]

...

From randy at psg.com  Sun Feb 12 19:56:51 2017
From: randy at psg.com (Randy Bush)
Date: Mon, 13 Feb 2017 12:56:51 +0900
Subject: [Bro] bro access to ether on ubuntu
Message-ID: <m2tw7yn3p8.wl-randy@psg.com>

ok, i give. i realize that i need to have a bro vm on each of the
physical nodes in a three-node ganeti cluster and run a bro cluster.

can i do this with only three vms for a three-node cluster, i.e. one of
the bro vms be both a worker and the central manager with broctl?  or
does the manager need to be a fourth vm?

randy

From randy at psg.com  Sun Feb 12 20:22:54 2017
From: randy at psg.com (Randy Bush)
Date: Mon, 13 Feb 2017 13:22:54 +0900
Subject: [Bro] ganeti cluster with bro cluster
In-Reply-To: <m2tw7yn3p8.wl-randy@psg.com>
References: <m2tw7yn3p8.wl-randy@psg.com>
Message-ID: <m2shnin2ht.wl-randy@psg.com>

ok, to put it directly, is it reasonable, presuming i can give the bro
vms whatever cpus they need, and the load is not heavy, to do the
following:

    [logger]
    type=logger
    host=localhost
    #
    [manager]
    type=manager
    host=localhost
    #
    #[proxy-1]
    #type=proxy
    #host=localhost
    #
    [worker-0]
    type=worker
    host=localhost
    interface=eth0
    #
    [worker-1]
    type=worker
    host=bro1.sea.rg.net
    interface=eth0
    #
    [worker-2]
    type=worker
    host=bro2.sea.rg.net
    interface=eth0

randy

From randy at psg.com  Sun Feb 12 23:04:40 2017
From: randy at psg.com (Randy Bush)
Date: Mon, 13 Feb 2017 16:04:40 +0900
Subject: [Bro] ganeti cluster with bro cluster
In-Reply-To: <m2shnin2ht.wl-randy@psg.com>
References: <m2tw7yn3p8.wl-randy@psg.com>
	<m2shnin2ht.wl-randy@psg.com>
Message-ID: <m2d1emmv07.wl-randy@psg.com>

[ ubuntu 16.04 on ganeti cluster ]

so i figured the config out

    [logger]
    type=logger
    host=bro0.sea.rg.net
    #
    [manager]
    type=manager
    host=bro0.sea.rg.net
    #
    [proxy-1]
    type=proxy
    host=bro0.sea.rg.net
    #
    [worker-0]
    type=worker
    host=bro0.sea.rg.net
    interface=eth0
    #
    [worker-1]
    type=worker
    host=bro1.sea.rg.net
    interface=eth0
    #
    [worker-2]
    type=worker
    host=bro2.sea.rg.net
    interface=eth0

and i got the worker-0 node to be able to pcap its eth0 by

    sudo setcap cap_net_raw,cap_net_admin=eip /usr/local/bro/bin/bro

although i ran the same on worker-1 and worker-2, they fail with

    worker-2 terminated immediately after starting; check output with "diag"
    worker-1 terminated immediately after starting; check output with "diag"

and the logs say

    fatal error: problem with interface eth0 (pcap_error: socket: Operation not permitted (pcap_activate))

i suspected that when `broctl deploy` copies over
/usr/local/bro/bin/bro, the copies do not inherit the capabilities.  but
i did

   broctl deploy
   <it failed with the pcap_error>
   <did `sudo setcap cap_net_raw,cap_net_admin=eip /usr/local/bro/bin/bro` on all workers>
   broctl start

and the same result, pcap_error on workers 1 and 2, not on 0.

---

i also get

   Error: error occurred while trying to send mail: send-mail: SENDMAIL-NOTFOUND not found

despite

    $ which sendmail
    /usr/sbin/sendmail

---

clue bat, please

randy

From Izik.Birka at hot.net.il  Mon Feb 13 00:34:29 2017
From: Izik.Birka at hot.net.il (Izik Birka)
Date: Mon, 13 Feb 2017 08:34:29 +0000
Subject: [Bro] SMB
Message-ID: <592228F4D0C8504187F2F76658040CB6DFE23DFF@HOT-MAILBOX-02.HOT.NET.IL>

Hi
Is there any logs that contains SMB stats ? why conn.log doesn't contains SMB connection ?

I have bro 2.5

Thanks
Izik Birka


This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain materials protected by copyright or information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or agreement.

If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication by error, notify the sender immediately and delete this message immediately.

Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170213/a95aad8d/attachment.html 

From bro at pingtrip.com  Mon Feb 13 05:43:16 2017
From: bro at pingtrip.com (Dave Crawford)
Date: Mon, 13 Feb 2017 08:43:16 -0500
Subject: [Bro] ganeti cluster with bro cluster
In-Reply-To: <m2d1emmv07.wl-randy@psg.com>
References: <m2tw7yn3p8.wl-randy@psg.com> <m2shnin2ht.wl-randy@psg.com>
	<m2d1emmv07.wl-randy@psg.com>
Message-ID: <5ACD12BB-B9E0-4684-9017-03C911CF9A1C@pingtrip.com>

Hi Randy,

I wrote a Bro plugin that takes care of the ?setcap? tasks after each deploy. Just adjusts the paths to reflect your Bro instal. Let me know if you run into any issues with it.

https://github.com/PingTrip/broctl-setcap <https://github.com/PingTrip/broctl-setcap>

-Dave
 
> 
> and i got the worker-0 node to be able to pcap its eth0 by
> 
>    sudo setcap cap_net_raw,cap_net_admin=eip /usr/local/bro/bin/bro
> 
> although i ran the same on worker-1 and worker-2, they fail with
> 
>    worker-2 terminated immediately after starting; check output with "diag"
>    worker-1 terminated immediately after starting; check output with "diag"
> 
> and the logs say
> 
>    fatal error: problem with interface eth0 (pcap_error: socket: Operation not permitted (pcap_activate))

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170213/ca593dba/attachment.html 

From bro at pingtrip.com  Mon Feb 13 10:07:52 2017
From: bro at pingtrip.com (Dave Crawford)
Date: Mon, 13 Feb 2017 13:07:52 -0500
Subject: [Bro] Netmap plugin issue
In-Reply-To: <DEC916FE-77F1-4035-919C-3E36FF78346A@pingtrip.com>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
	<237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
	<7e14f24d17134bdbb13a4e41a8ec9e87@exch13-mail04.win.slac.stanford.edu>
	<F91195A4-A35D-48B8-B893-63E66E8821A4@pingtrip.com>
	<6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>
	<35224580-AE4F-45AE-8046-303758FF9A9E@icir.org>
	<95315aa48ece4e8696e229da0879d445@exch13-mail04.win.slac.stanford.edu>
	<34442F24-0A8B-4EE7-A835-52707EA20478@pingtrip.com>
	<fb7aab1d41564392b45268b7c3594163@exch13-mail04.win.slac.stanford.edu>
	<03503572-1528-40F0-9A0F-E686F0AFA67F@icir.org>
	<6165d3b4ddbc483996206c2dfc2f17ad@exch13-mail04.win.slac.stanford.edu>
	<DEC916FE-77F1-4035-919C-3E36FF78346A@pingtrip.com>
Message-ID: <E0011940-43FC-427E-9F4A-0FAFA99D21CB@pingtrip.com>

Continuing to see impressive performance with Bro+Netmap:

data_forward_rate_Mbps":1484.1698"
data_drop_rate_Mbps":0.0000"

And of the 10 workers, the greatest capture_loss reported by Bro is well under 1%:

2017-02-13T09:01:56-0500        600.000013      MID_INT-8       1221    3368533 0.036247

-Dave

> On Feb 12, 2017, at 7:15 AM, Dave Crawford <bro at pingtrip.com> wrote:
> 
> My LB stats are similar, but Bro isn?t reflecting a loss:
> 
> Feb 11 04:58:02 mid-csignsm-01 lb[3144]: {"ts":1486807082.815681,"interface":"netmap:eth6","packets_received":739258266,"packets_forwarded":737718676,"packets_dropped":1213100,"non_ip_packets":371283,"data_forward_rate_Mbps":758.6328,"data_drop_rate_Mbps":0.0000,"packet_forward_rate_kpps":102.7250,"packet_drop_rate_kpps":0.0000,"free_buffer_slots":100000}
> 
> The next cycle of capture_loss shows less than 1% loss for the 10 workers:
> 
> 2017-02-11T05:01:56-0500        600.000119      MID_INT-2       5       102927  0.004858
> 2017-02-11T05:01:56-0500        600.000015      MID_INT-4       7       107577  0.006507
> 2017-02-11T05:01:56-0500        600.000053      MID_INT-9       6       101979  0.005884
> 2017-02-11T05:01:56-0500        600.000016      MID_INT-3       1       304887  0.000328
> 2017-02-11T05:01:56-0500        600.000543      MID_INT-10      5       186552  0.00268
> 2017-02-11T05:01:56-0500        600.000009      MID_INT-5       6       101433  0.005915
> 2017-02-11T05:01:56-0500        600.000005      MID_INT-6       5       110256  0.004535
> 2017-02-11T05:01:56-0500        600.000085      MID_INT-7       1       98164   0.001019
> 2017-02-11T05:01:56-0500        600.000041      MID_INT-8       4       99979   0.004001
> 2017-02-11T05:01:57-0500        600.000047      MID_INT-1       3       90591   0.003312
> 
> I also noticed that Andy?s LB output is slightly different. His displays the free buffers as ?overflow_queue_size? where my output is ?free_buffer_slots?.
> 
> Also Andy, your overflow queue size is ?0?, did you define one and its been depleted? Creating one, or increasing the size may help with the Bro dropped packets.
> 
> -Dave

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170213/dfc68497/attachment-0001.html 

From espressobeanies at gmail.com  Mon Feb 13 10:34:47 2017
From: espressobeanies at gmail.com (Espresso Beanies)
Date: Mon, 13 Feb 2017 13:34:47 -0500
Subject: [Bro] Question on redefining Bro variables
In-Reply-To: <CAEZw2bygADw9nEAQ1jCn2NwbFAjM3f-duZMWixBbXWBL0qWGEg@mail.gmail.com>
References: <CAOeD16-CMpUfiKnGbZJrgy9yaRZF7tNkR3Usdw9c=WRks85J_A@mail.gmail.com>
	<CAEZw2bygADw9nEAQ1jCn2NwbFAjM3f-duZMWixBbXWBL0qWGEg@mail.gmail.com>
Message-ID: <CAOeD169txOp8LSN49qcqKQ+0PBbNhTk+EY92wMWDfN3LSB9qrg@mail.gmail.com>

Got it. Thanks.

On Thu, Feb 9, 2017 at 4:50 PM, anthony kasza <anthony.kasza at gmail.com>
wrote:

> Personally, I make my own directory of scripts and modules and @load them
> from the local.bro file.
>
> -AK
>
> On Feb 9, 2017 2:33 PM, "Espresso Beanies" <espressobeanies at gmail.com>
> wrote:
>
>> Good afternoon,
>>
>> Where would it be best to redefine Bro variables? Should they all be in
>> the local.bro file or individual main.bro files for each module?
>>
>> Thanks in advance,
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170213/fded579f/attachment.html 

From randy at psg.com  Mon Feb 13 11:39:12 2017
From: randy at psg.com (Randy Bush)
Date: Tue, 14 Feb 2017 04:39:12 +0900
Subject: [Bro] ganeti cluster with bro cluster
In-Reply-To: <5ACD12BB-B9E0-4684-9017-03C911CF9A1C@pingtrip.com>
References: <m2tw7yn3p8.wl-randy@psg.com> <m2shnin2ht.wl-randy@psg.com>
	<m2d1emmv07.wl-randy@psg.com>
	<5ACD12BB-B9E0-4684-9017-03C911CF9A1C@pingtrip.com>
Message-ID: <m2poillw2n.wl-randy@psg.com>

hi dave,

aha!

bro0.sea.rg.net:/usr/local/bro> broctl install
removing old policies in /usr/local/bro/spool/installed-scripts-do-not-touch/site ...
removing old policies in /usr/local/bro/spool/installed-scripts-do-not-touch/auto ...
creating policy directories ...
installing site policies ...
generating cluster-layout.bro ...
generating local-networks.bro ...
generating broctl-config.bro ...
generating broctl-config.sh ...
updating nodes ...
setcap plugin: executing setcap on each node:
bro2.sea.rg.net - Executing setcap: SUCCESS
bro0.sea.rg.net - Executing setcap: SUCCESS
bro1.sea.rg.net - Executing setcap: SUCCESS

thank you!!

randy

From pierre.gaulon at transfer-to.com  Mon Feb 13 18:18:06 2017
From: pierre.gaulon at transfer-to.com (Pierre Gaulon)
Date: Tue, 14 Feb 2017 10:18:06 +0800
Subject: [Bro] ganeti cluster with bro cluster
In-Reply-To: <5ACD12BB-B9E0-4684-9017-03C911CF9A1C@pingtrip.com>
References: <m2tw7yn3p8.wl-randy@psg.com> <m2shnin2ht.wl-randy@psg.com>
	<m2d1emmv07.wl-randy@psg.com>
	<5ACD12BB-B9E0-4684-9017-03C911CF9A1C@pingtrip.com>
Message-ID: <2c41c7ba-fa91-d40b-1894-4b1c2d3cd298@transfer-to.com>

Hi Dave,

First of all, thank you for your plugin, it is very useful! I am using
since a while.

Last time I used it I ran into an error and had to change the code to
see it.
In the last line of the plugin I appended the reason of the FAIL when it
fails:

self.message("{0} - Executing setcap: {1}".format(n.host, 'SUCCESS' if
success else 'FAIL ' + output[0]))

The reason was:
sorry, you must have a tty to run sudo

This is linked to the sudoers setting:
Defaults    requiretty

In order to fix it I couldn't find a workaround using the command given
to the plugin (using -tt for instance). I just commented the line using
visudo.
Any ideas to make both compatible are still welcome!

Hope it helps!
Best regards,
Pierre Gaulon.

On 13/2/17 21:43, Dave Crawford wrote:
> Hi Randy,
>
> I wrote a Bro plugin that takes care of the ?setcap? tasks after each
> deploy. Just adjusts the paths to reflect your Bro instal. Let me know
> if you run into any issues with it.
>
> https://github.com/PingTrip/broctl-setcap
>
> -Dave
>  
>>
>> and i got the worker-0 node to be able to pcap its eth0 by
>>
>>    sudo setcap cap_net_raw,cap_net_admin=eip /usr/local/bro/bin/bro
>>
>> although i ran the same on worker-1 and worker-2, they fail with
>>
>>    worker-2 terminated immediately after starting; check output with
>> "diag"
>>    worker-1 terminated immediately after starting; check output with
>> "diag"
>>
>> and the logs say
>>
>>    fatal error: problem with interface eth0 (pcap_error: socket:
>> Operation not permitted (pcap_activate))
>
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170214/f5b7f900/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170214/f5b7f900/attachment.bin 

From bro at pingtrip.com  Mon Feb 13 18:26:24 2017
From: bro at pingtrip.com (Dave Crawford)
Date: Mon, 13 Feb 2017 21:26:24 -0500
Subject: [Bro] ganeti cluster with bro cluster
In-Reply-To: <2c41c7ba-fa91-d40b-1894-4b1c2d3cd298@transfer-to.com>
References: <m2tw7yn3p8.wl-randy@psg.com> <m2shnin2ht.wl-randy@psg.com>
	<m2d1emmv07.wl-randy@psg.com>
	<5ACD12BB-B9E0-4684-9017-03C911CF9A1C@pingtrip.com>
	<2c41c7ba-fa91-d40b-1894-4b1c2d3cd298@transfer-to.com>
Message-ID: <C17CC453-F816-4D3A-ADF9-745E0BC6AF47@pingtrip.com>

Thanks for the feedback Pierre, what OS and version are you running Bro on?

Modify the sudoers file to add an additional line to disable the tty requirement specifically for setcap.

bro ALL=NOPASSWD: /sbin/setcap
Defaults!/sbin/setcap !requiretty

Let me know if that takes care of the issue and I?ll get my documentation updated.

-Dave

> On Feb 13, 2017, at 9:18 PM, Pierre Gaulon <pierre.gaulon at transfer-to.com> wrote:
> 
> Hi Dave,
> 
> First of all, thank you for your plugin, it is very useful! I am using since a while.
> 
> Last time I used it I ran into an error and had to change the code to see it.
> In the last line of the plugin I appended the reason of the FAIL when it fails:
> 
> self.message("{0} - Executing setcap: {1}".format(n.host, 'SUCCESS' if success else 'FAIL ' + output[0]))
> 
> The reason was:
> sorry, you must have a tty to run sudo
> 
> This is linked to the sudoers setting:
> Defaults    requiretty
> 
> In order to fix it I couldn't find a workaround using the command given to the plugin (using -tt for instance). I just commented the line using visudo.
> Any ideas to make both compatible are still welcome!
> 
> Hope it helps!
> Best regards,
> Pierre Gaulon.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170213/4d2e91bb/attachment.html 

From pierre.gaulon at transfer-to.com  Mon Feb 13 19:40:20 2017
From: pierre.gaulon at transfer-to.com (Pierre Gaulon)
Date: Tue, 14 Feb 2017 11:40:20 +0800
Subject: [Bro] ganeti cluster with bro cluster
In-Reply-To: <C17CC453-F816-4D3A-ADF9-745E0BC6AF47@pingtrip.com>
References: <m2tw7yn3p8.wl-randy@psg.com> <m2shnin2ht.wl-randy@psg.com>
	<m2d1emmv07.wl-randy@psg.com>
	<5ACD12BB-B9E0-4684-9017-03C911CF9A1C@pingtrip.com>
	<2c41c7ba-fa91-d40b-1894-4b1c2d3cd298@transfer-to.com>
	<C17CC453-F816-4D3A-ADF9-745E0BC6AF47@pingtrip.com>
Message-ID: <93fe0f98-c0b2-748b-4bcb-1edad56c15a7@transfer-to.com>

The workers are running on CentOS Linux release 7.2.1511 (Core).

The Defaults!/sbin/setcap !requiretty fixed the problem!

Thanks for your help!
Best regards,
Pierre Gaulon.

On 14/2/17 10:26, Dave Crawford wrote:
> Thanks for the feedback Pierre, what OS and version are you running
> Bro on?
>
> Modify the sudoers file to add an additional line to disable the tty
> requirement specifically for setcap.
>
> bro ALL=NOPASSWD: /sbin/setcap
> Defaults!/sbin/setcap !requiretty
>
> Let me know if that takes care of the issue and I?ll get my
> documentation updated.
>
> -Dave
>
>> On Feb 13, 2017, at 9:18 PM, Pierre Gaulon
>> <pierre.gaulon at transfer-to.com
>> <mailto:pierre.gaulon at transfer-to.com>> wrote:
>>
>> Hi Dave,
>>
>> First of all, thank you for your plugin, it is very useful! I am
>> using since a while.
>>
>> Last time I used it I ran into an error and had to change the code to
>> see it.
>> In the last line of the plugin I appended the reason of the FAIL when
>> it fails:
>>
>> self.message("{0} - Executing setcap: {1}".format(n.host, 'SUCCESS'
>> if success else 'FAIL ' + output[0]))
>>
>> The reason was:
>> sorry, you must have a tty to run sudo
>>
>> This is linked to the sudoers setting:
>> Defaults    requiretty
>>
>> In order to fix it I couldn't find a workaround using the command
>> given to the plugin (using -tt for instance). I just commented the
>> line using visudo.
>> Any ideas to make both compatible are still welcome!
>>
>> Hope it helps!
>> Best regards,
>> Pierre Gaulon.
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170214/2fafe1a4/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170214/2fafe1a4/attachment.bin 

From randy at psg.com  Mon Feb 13 19:44:58 2017
From: randy at psg.com (Randy Bush)
Date: Tue, 14 Feb 2017 12:44:58 +0900
Subject: [Bro] ganeti cluster with bro cluster
In-Reply-To: <93fe0f98-c0b2-748b-4bcb-1edad56c15a7@transfer-to.com>
References: <m2tw7yn3p8.wl-randy@psg.com> <m2shnin2ht.wl-randy@psg.com>
	<m2d1emmv07.wl-randy@psg.com>
	<5ACD12BB-B9E0-4684-9017-03C911CF9A1C@pingtrip.com>
	<2c41c7ba-fa91-d40b-1894-4b1c2d3cd298@transfer-to.com>
	<C17CC453-F816-4D3A-ADF9-745E0BC6AF47@pingtrip.com>
	<93fe0f98-c0b2-748b-4bcb-1edad56c15a7@transfer-to.com>
Message-ID: <m2bmu5jv0l.wl-randy@psg.com>

fwiw, at my age, i leave a docco trail because i will forget what the
heck i did a day later.  so the hack i finally used is available at
https://git.rg.net/randy/randy/src/master/bro-cluster.md

randy

From Izik.Birka at hot.net.il  Mon Feb 13 23:36:24 2017
From: Izik.Birka at hot.net.il (Izik Birka)
Date: Tue, 14 Feb 2017 07:36:24 +0000
Subject: [Bro] SMB Language
Message-ID: <592228F4D0C8504187F2F76658040CB6DFE2B50A@HOT-MAILBOX-02.HOT.NET.IL>

Hi
Just enable SMB analyzer , works great

I have a problem with the Hebrew language , it's looks like it's not supported , I'm getting  this files name in log file :

Test\hello\\\xd7\x92\xd7\x99\xd7\x95\xd7\x9c \xd7\x9b\xd7\x9e\xd7\x95\xd7\xaa\xd7\x99.csv

The \\\xd7\x92\xd7\x99\xd7\x95\xd7\x9c<file:///\\\xd7\x92\xd7\x99\xd7\x95\xd7\x9c> - is Hebrew words

is there a way to fix it ?



This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain materials protected by copyright or information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or agreement.

If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication by error, notify the sender immediately and delete this message immediately.

Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170214/08549660/attachment.html 

From jazoff at illinois.edu  Tue Feb 14 06:15:59 2017
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Tue, 14 Feb 2017 14:15:59 +0000
Subject: [Bro] SMB Language
In-Reply-To: <592228F4D0C8504187F2F76658040CB6DFE2B50A@HOT-MAILBOX-02.HOT.NET.IL>
References: <592228F4D0C8504187F2F76658040CB6DFE2B50A@HOT-MAILBOX-02.HOT.NET.IL>
Message-ID: <ACD07475-0FEE-4946-9068-39271F6845E9@illinois.edu>


> On Feb 14, 2017, at 2:36 AM, Izik Birka <Izik.Birka at hot.net.il> wrote:
> 
> Hi
> Just enable SMB analyzer , works great
>  
> I have a problem with the Hebrew language , it's looks like it's not supported , I'm getting  this files name in log file :
>  
> Test\hello\\\xd7\x92\xd7\x99\xd7\x95\xd7\x9c \xd7\x9b\xd7\x9e\xd7\x95\xd7\xaa\xd7\x99.csv
>  
> The \\\xd7\x92\xd7\x99\xd7\x95\xd7\x9c - is Hebrew words 
>  
> is there a way to fix it ?

That's just an escaped utf-8 string:

>>> s='Test\hello\\\xd7\x92\xd7\x99\xd7\x95\xd7\x9c \xd7\x9b\xd7\x9e\xd7\x95\xd7\xaa\xd7\x99.csv'
>>> print s
Test\hello\???? ?????.csv

(or in python3)

>>> s=b'Test\hello\\\xd7\x92\xd7\x99\xd7\x95\xd7\x9c \xd7\x9b\xd7\x9e\xd7\x95\xd7\xaa\xd7\x99.csv'
>>> print(s.decode('utf-8'))
Test\hello\???? ?????.csv



-- 
- Justin Azoff



From andrew.dellana at bayer.com  Tue Feb 14 06:52:34 2017
From: andrew.dellana at bayer.com (Andrew Dellana)
Date: Tue, 14 Feb 2017 14:52:34 +0000
Subject: [Bro] Connection summary <???> values
Message-ID: <40f9b6e86ef34a8bb8046a97fcc822c9@moxde9.na.bayer.cnb>

Hello,

For the connection summary is there a way to reformat the way the notification looks - Like put each of these on their own line?  Also is there way to make the <???> value show its domain name?  The <???> value shows up in both the incoming, outgoing and total sections.



        #1=<???>  #2=(redacted)  #3=<???>

        #4= (redacted) #5= (redacted) #6= (redacted)

        #7= (redacted) #8=<???>  #9=<???>

        #10= (redacted) #11=<???>  #12= (redacted)

        #13=<???>  #14= (redacted) #15=<???>

        #16= (redacted) #17=<???>  #18=<???>

        #19=<???>  #20= (redacted)




Thanks,

Andrew Dellana
Intern
________________________

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170214/a1ab1920/attachment.html 

From vladg at illinois.edu  Tue Feb 14 07:05:02 2017
From: vladg at illinois.edu (Vlad Grigorescu)
Date: Tue, 14 Feb 2017 09:05:02 -0600
Subject: [Bro] Connection summary <???> values
In-Reply-To: <40f9b6e86ef34a8bb8046a97fcc822c9@moxde9.na.bayer.cnb>
References: <40f9b6e86ef34a8bb8046a97fcc822c9@moxde9.na.bayer.cnb>
Message-ID: <m0wpcs6cf5.fsf@vladg.net>

<???> simply means that DNS resolution failed.

Andrew Dellana <andrew.dellana at bayer.com> writes:

> Hello,
>
> For the connection summary is there a way to reformat the way the notification looks - Like put each of these on their own line?  Also is there way to make the <???> value show its domain name?  The <???> value shows up in both the incoming, outgoing and total sections.
>
>
>
>         #1=<???>  #2=(redacted)  #3=<???>
>
>         #4= (redacted) #5= (redacted) #6= (redacted)
>
>         #7= (redacted) #8=<???>  #9=<???>
>
>         #10= (redacted) #11=<???>  #12= (redacted)
>
>         #13=<???>  #14= (redacted) #15=<???>
>
>         #16= (redacted) #17=<???>  #18=<???>
>
>         #19=<???>  #20= (redacted)
>
>
>
>
> Thanks,
>
> Andrew Dellana
> Intern
> ________________________
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 800 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170214/d2661b25/attachment.bin 

From Izik.Birka at hot.net.il  Tue Feb 14 07:11:39 2017
From: Izik.Birka at hot.net.il (Izik Birka)
Date: Tue, 14 Feb 2017 15:11:39 +0000
Subject: [Bro] SMB Language
In-Reply-To: <ACD07475-0FEE-4946-9068-39271F6845E9@illinois.edu>
References: <592228F4D0C8504187F2F76658040CB6DFE2B50A@HOT-MAILBOX-02.HOT.NET.IL>
	<ACD07475-0FEE-4946-9068-39271F6845E9@illinois.edu>
Message-ID: <592228F4D0C8504187F2F76658040CB6DFE2EB4A@HOT-MAILBOX-02.HOT.NET.IL>

Can I add utf-8 encoding to bro ?
Is there other option ?

 I'm forward the log file to SIEM system , and it's not readable in that way

Thanks



-----Original Message-----
From: Azoff, Justin S [mailto:jazoff at illinois.edu] 
Sent: Tuesday, February 14, 2017 4:16 PM
To: Izik Birka <Izik.Birka at hot.net.il>
Cc: bro at bro.org
Subject: Re: [Bro] SMB Language


> On Feb 14, 2017, at 2:36 AM, Izik Birka <Izik.Birka at hot.net.il> wrote:
> 
> Hi
> Just enable SMB analyzer , works great
>  
> I have a problem with the Hebrew language , it's looks like it's not supported , I'm getting  this files name in log file :
>  
> Test\hello\\\xd7\x92\xd7\x99\xd7\x95\xd7\x9c \xd7\x9b\xd7\x9e\xd7\x95\xd7\xaa\xd7\x99.csv
>  
> The \\\xd7\x92\xd7\x99\xd7\x95\xd7\x9c - is Hebrew words 
>  
> is there a way to fix it ?

That's just an escaped utf-8 string:

>>> s='Test\hello\\\xd7\x92\xd7\x99\xd7\x95\xd7\x9c \xd7\x9b\xd7\x9e\xd7\x95\xd7\xaa\xd7\x99.csv'
>>> print s
Test\hello\???? ?????.csv

(or in python3)

>>> s=b'Test\hello\\\xd7\x92\xd7\x99\xd7\x95\xd7\x9c \xd7\x9b\xd7\x9e\xd7\x95\xd7\xaa\xd7\x99.csv'
>>> print(s.decode('utf-8'))
Test\hello\???? ?????.csv



-- 
- Justin Azoff


This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain materials protected by copyright or information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or agreement.

If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication by error, notify the sender immediately and delete this message immediately.

Thank you.


From seth at icir.org  Tue Feb 14 08:37:17 2017
From: seth at icir.org (Seth Hall)
Date: Tue, 14 Feb 2017 08:37:17 -0800
Subject: [Bro] Netmap plugin issue
In-Reply-To: <E0011940-43FC-427E-9F4A-0FAFA99D21CB@pingtrip.com>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
	<237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
	<7e14f24d17134bdbb13a4e41a8ec9e87@exch13-mail04.win.slac.stanford.edu>
	<F91195A4-A35D-48B8-B893-63E66E8821A4@pingtrip.com>
	<6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>
	<35224580-AE4F-45AE-8046-303758FF9A9E@icir.org>
	<95315aa48ece4e8696e229da0879d445@exch13-mail04.win.slac.stanford.edu>
	<34442F24-0A8B-4EE7-A835-52707EA20478@pingtrip.com>
	<fb7aab1d41564392b45268b7c3594163@exch13-mail04.win.slac.stanford.edu>
	<03503572-1528-40F0-9A0F-E686F0AFA67F@icir.org>
	<6165d3b4ddbc483996206c2dfc2f17ad@exch13-mail04.win.slac.stanford.edu>
	<DEC916FE-77F1-4035-919C-3E36FF78346A@pingtrip.com>
	<E0011940-43FC-427E-9F4A-0FAFA99D21CB@pingtrip.com>
Message-ID: <3B5C35C1-76E1-415D-9B27-16A738DEBCE3@icir.org>


> On Feb 13, 2017, at 10:07 AM, Dave Crawford <bro at pingtrip.com> wrote:
> 
> Continuing to see impressive performance with Bro+Netmap:
> 
> data_forward_rate_Mbps":1484.1698"
> data_drop_rate_Mbps":0.0000"
> 
> And of the 10 workers, the greatest capture_loss reported by Bro is well under 1%:

Yay, that's great!  At least now you can feel more certain that the capture loss is either misreported or you have a SPAN port or packet broker that is having trouble.  You could also check the interface hardware counters here and there to see if you are having any loss on the NIC. (ethtool -S)

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From seth at icir.org  Tue Feb 14 08:49:55 2017
From: seth at icir.org (Seth Hall)
Date: Tue, 14 Feb 2017 08:49:55 -0800
Subject: [Bro] Netmap plugin issue
In-Reply-To: <DEC916FE-77F1-4035-919C-3E36FF78346A@pingtrip.com>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
	<237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
	<7e14f24d17134bdbb13a4e41a8ec9e87@exch13-mail04.win.slac.stanford.edu>
	<F91195A4-A35D-48B8-B893-63E66E8821A4@pingtrip.com>
	<6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>
	<35224580-AE4F-45AE-8046-303758FF9A9E@icir.org>
	<95315aa48ece4e8696e229da0879d445@exch13-mail04.win.slac.stanford.edu>
	<34442F24-0A8B-4EE7-A835-52707EA20478@pingtrip.com>
	<fb7aab1d41564392b45268b7c3594163@exch13-mail04.win.slac.stanford.edu>
	<03503572-1528-40F0-9A0F-E686F0AFA67F@icir.org>
	<6165d3b4ddbc483996206c2dfc2f17ad@exch13-mail04.win.slac.stanford.edu>
	<DEC916FE-77F1-4035-919C-3E36FF78346A@pingtrip.com>
Message-ID: <088F7D1E-14E0-48CF-B169-232272DF32C4@icir.org>


> On Feb 12, 2017, at 4:15 AM, Dave Crawford <bro at pingtrip.com> wrote:
> 
> I also noticed that Andy?s LB output is slightly different. His displays the free buffers as ?overflow_queue_size? where my output is ?free_buffer_slots?.

Those are different lb log lines.  The lines with overflow_queue_size are regarding the output pipes that send packets off the Bro (or other) processes.  The line that has free_buffer_slots is regarding the interface being sniffed and it means that those are buffers (each buffer holds a single packet) that can be used if a pipe isn't being flushed quickly enough.  If you have free buffers and packets begin to get backed up, the free_buffer_slots number on the physical interface will begin to go down and the overflow_queue_size on the pipe or pipes getting backed up will begin to rise.

I'm planning on writing a more extensive guide on all of this soon.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From bro at pingtrip.com  Tue Feb 14 10:10:44 2017
From: bro at pingtrip.com (Dave Crawford)
Date: Tue, 14 Feb 2017 13:10:44 -0500
Subject: [Bro] Netmap plugin issue
In-Reply-To: <3B5C35C1-76E1-415D-9B27-16A738DEBCE3@icir.org>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
	<237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
	<7e14f24d17134bdbb13a4e41a8ec9e87@exch13-mail04.win.slac.stanford.edu>
	<F91195A4-A35D-48B8-B893-63E66E8821A4@pingtrip.com>
	<6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>
	<35224580-AE4F-45AE-8046-303758FF9A9E@icir.org>
	<95315aa48ece4e8696e229da0879d445@exch13-mail04.win.slac.stanford.edu>
	<34442F24-0A8B-4EE7-A835-52707EA20478@pingtrip.com>
	<fb7aab1d41564392b45268b7c3594163@exch13-mail04.win.slac.stanford.edu>
	<03503572-1528-40F0-9A0F-E686F0AFA67F@icir.org>
	<6165d3b4ddbc483996206c2dfc2f17ad@exch13-mail04.win.slac.stanford.edu>
	<DEC916FE-77F1-4035-919C-3E36FF78346A@pingtrip.com>
	<E0011940-43FC-427E-9F4A-0FAFA99D21CB@pingtrip.com>
	<3B5C35C1-76E1-415D-9B27-16A738DEBCE3@icir.org>
Message-ID: <ECCADAC6-F02E-4645-91E9-40BE5FB99966@pingtrip.com>


> 
> Yay, that's great!  At least now you can feel more certain that the capture loss is either misreported or you have a SPAN port or packet broker that is having trouble.  You could also check the interface hardware counters here and there to see if you are having any loss on the NIC. (ethtool -S)
> 
>  .Seth
> 

Seth, so are you thinking that LB is mis-reporting packet loss? This is the ethtool stats for the capture NIC:

NIC statistics:
     rx_packets: 55681830
     tx_packets: 0
     rx_bytes: 35600025423
     tx_bytes: 0
     rx_errors: 0
     tx_errors: 0
     rx_dropped: 0
     tx_dropped: 0
     multicast: 43367780
     collisions: 0
     rx_over_errors: 0
     rx_crc_errors: 0
     rx_frame_errors: 0
     rx_fifo_errors: 0
     rx_missed_errors: 193958
     tx_aborted_errors: 0
     tx_carrier_errors: 0
     tx_fifo_errors: 0
     tx_heartbeat_errors: 0
     rx_pkts_nic: 18460871971
     tx_pkts_nic: 0
     rx_bytes_nic: 10234659126400
     tx_bytes_nic: 0
     lsc_int: 9
     tx_busy: 0
     non_eop_descs: 0
     broadcast: 277894
     rx_no_buffer_count: 0
     tx_timeout_count: 0
     tx_restart_queue: 0
     rx_long_length_errors: 0
     rx_short_length_errors: 0
     tx_flow_control_xon: 0
     rx_flow_control_xon: 0
     tx_flow_control_xoff: 0
     rx_flow_control_xoff: 0
     rx_csum_offload_errors: 0


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170214/b7ff4a74/attachment.html 

From espressobeanies at gmail.com  Tue Feb 14 11:52:14 2017
From: espressobeanies at gmail.com (Espresso Beanies)
Date: Tue, 14 Feb 2017 14:52:14 -0500
Subject: [Bro] Conceptual question on main.bro files
Message-ID: <CAOeD16_-M7ohetowev1KP4JXe2CAgsPJzxEZ7oWZr_UJpod3BQ@mail.gmail.com>

Hi,

I'm trying to better understand Bro's architecture and what is the
significance of the "main.bro" files in relation to the other .bro files?
I'm guessing some heirarchal purpose, but I don't see a "main.bro" file in
every folder that contains a .bro file itself. Is someone able to better
explain?

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170214/0ae605c2/attachment.html 

From jdopheid at illinois.edu  Tue Feb 14 12:09:44 2017
From: jdopheid at illinois.edu (Dopheide, Jeannette M)
Date: Tue, 14 Feb 2017 20:09:44 +0000
Subject: [Bro] =?utf-8?q?Save_the_date=3A_BroCon_=E2=80=9917=2C_Sept_12_-_?=
 =?utf-8?q?14_in_Urbana=2C_IL?=
Message-ID: <CAEB54FF-82AC-469C-B1ED-EAAE67B1461D@illinois.edu>

Save the date! BroCon ?17 will occur Tuesday September 12th ? Thursday September 14th in at the National Center for Supercomputing Applications in Urbana, IL. 

We are wrapping up registration and hotel info so be on the lookout for more complete information. 
 

------
Jeannette Dopheide
Training and Outreach Coordinator
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
 



From zeolla at gmail.com  Tue Feb 14 12:12:13 2017
From: zeolla at gmail.com (Zeolla@GMail.com)
Date: Tue, 14 Feb 2017 20:12:13 +0000
Subject: [Bro] Conceptual question on main.bro files
In-Reply-To: <CAOeD16_-M7ohetowev1KP4JXe2CAgsPJzxEZ7oWZr_UJpod3BQ@mail.gmail.com>
References: <CAOeD16_-M7ohetowev1KP4JXe2CAgsPJzxEZ7oWZr_UJpod3BQ@mail.gmail.com>
Message-ID: <CAM1mTCqPW_bcG=pnZByq=o8zTOdf-MoDnth4QQono=k9haWHrw@mail.gmail.com>

As far as I'm aware, main.bro isn't actually special.  It's just a
pseudo-standard (maybe a real standard, even) for a main/primary bro script
for a folder/organizational area.  What is special is __load__.bro.  If you
use @load to load a directory, it will look there for __local__.bro, and
then follow whatever instructions it finds (@load or @load-sigs, for
instance).  You will sometimes find that __load__.bro has a `@load
./main.bro` statement in it.  For instance:
https://github.com/bro/bro/blob/master/scripts/base/frameworks/files/__load__.bro

So, for instance, if you go to local.bro
<local.brohttps://github.com/bro/bro/blob/master/scripts/site/local.bro>,
you will find `@load tuning/defaults`, then if you go to the tuning/defaults
<https://github.com/bro/bro/tree/master/scripts/policy/tuning/defaults> folder,
you find a __load__.bro
<__load__.brohttps://github.com/bro/bro/blob/master/scripts/policy/tuning/defaults/__load__.bro>,
which will be followed to load some bro scripts which are /not/ main.bro.
In this situation, main.bro doesn't exist for tuning/defaults, and that's
fine.

Another example is, go to local.bro, and find `@load
misc/detect-traceroute` (commented out by default).  But if you follow what
would happen if this was uncommented, it would go to misc/detect-traceroute
<https://github.com/bro/bro/tree/master/scripts/policy/misc/detect-traceroute>,
load __load__.bro due to convention, and then the relative main.bro
<main.brohttps://github.com/bro/bro/blob/master/scripts/policy/misc/detect-traceroute/main.bro>
/is/
loaded because it's specified in __load__.bro.

Hope that helps - also, please correct me if there is an actual main.bro
convention anywhere that I'm not aware of.

Jon

On Tue, Feb 14, 2017 at 2:54 PM Espresso Beanies <espressobeanies at gmail.com>
wrote:

> Hi,
>
> I'm trying to better understand Bro's architecture and what is the
> significance of the "main.bro" files in relation to the other .bro files?
> I'm guessing some heirarchal purpose, but I don't see a "main.bro" file in
> every folder that contains a .bro file itself. Is someone able to better
> explain?
>
> Thanks!
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-- 

Jon

Sent from my mobile device
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170214/eb95e7d3/attachment.html 

From johanna at icir.org  Tue Feb 14 13:12:46 2017
From: johanna at icir.org (Johanna Amann)
Date: Tue, 14 Feb 2017 13:12:46 -0800
Subject: [Bro] bro_intel feeds as csv file
In-Reply-To: <CALGAZaCnRTXenOVND3mrS3knS1k8JxaRJ2YRVOOv7BmfL3nhfA@mail.gmail.com>
References: <CALGAZaCnRTXenOVND3mrS3knS1k8JxaRJ2YRVOOv7BmfL3nhfA@mail.gmail.com>
Message-ID: <20170214211246.4z5lk4bp2d7apncq@wifi107.sys.ICSI.Berkeley.EDU>

Hello Sunu,

no, it is not trivially possible to get the intel framework to read csv
files. The easiest way is to convert your format into the Bro syntax.

Alternatively, you can use the input framework yourself, specify the
delimiters that you need and manually call Intel::insert.

Johanna

On Sun, Feb 12, 2017 at 03:01:21PM +0530, ps sunu wrote:
> Hi all ,
>                         Is possible to generate bro_intel  txt files as a
> .CSV format ?
> 
> my format
> 
> #fields indicator indicator_type meta.source meta.url meta.do_notice
> meta.if_in meta.whitelist
> # Intel::ADDR binarydefense-ip - T - -
> # Intel::ADDR binarydefense-ip - T - -
> # Intel::ADDR binarydefense-ip - T - -
> # Intel::ADDR binarydefense-ip - T - -
> # Intel::ADDR binarydefense-ip - T - -
> # Intel::ADDR binarydefense-ip - T - -
> # Intel::ADDR binarydefense-ip - T - -
> 
> 
> Need to change this feeds as a csv format
> 
> 
> Regards,
> Sunu

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From johanna at icir.org  Tue Feb 14 13:15:25 2017
From: johanna at icir.org (Johanna Amann)
Date: Tue, 14 Feb 2017 13:15:25 -0800
Subject: [Bro] Information on OCSP and CRL
In-Reply-To: <m0efz76tpz.fsf@vladg.net>
References: <EE2D32CF-B9DE-433F-9A01-D79001D1B33B@bivio.net>
	<m0efz76tpz.fsf@vladg.net>
Message-ID: <20170214211525.jsbcvteffga5mlpb@wifi107.sys.ICSI.Berkeley.EDU>

To expand a bit on this...

there is a policy script to perform certificate validation, just load
policy/protocols/ssl/validate-certs.bro. Note that this uses plain OpenSSL
for validation and might not be able to validate a few cases which
browsers can validate.

For OCSP, the situation is a bit less optimal. The script that Vlad
mentioned performs OCSP validation, but _only_ in case that the OCSP
response is sent stapled in the TLS handshake - which is rather rare.

Outside of this, Bro currently does not support OCSP validatiob.

Johanna

On Thu, Feb 09, 2017 at 01:37:44PM -0600, Vlad Grigorescu wrote:
> Tony,
> 
> There's an optional script for OCSP validation:
> https://github.com/bro/bro/blob/v2.5/scripts/policy/protocols/ssl/validate-ocsp.bro
> 
> To use:
> > @load protocols/ssl/validate-ocsp
> 
>   --Vlad
> 
> Tony Waller <twaller at bivio.net> writes:
> 
> > I am looking for additional information on utilizing OCSP and CRL in Bro NSM. I would like to know if certificates from clients can be checked in real-time or near real-time against a CRL? Also, can Bro NSM perform a OCSP request to a RA and check a certificate to determine if it is valid? If this is the case where in Bro NSM do you set the address for the RA or CRL responder?
> >
> > Sincerely,
> >
> > Tony
> >
> >
> > Tony Waller, CISSP
> > Director, Systems Engineering
> > Bivio Networks, Inc.
> > ?Powering Advanced Cyber Operations? (TM)
> > Mobile (443) 994-0936
> >
> > [cid:image001.png at 01D27E32.049899E0]
> >
> > *Note: The information contained in this email confidential. This information is intended only for the individual, individuals or entity to whom it is addressed. If you are not the intended recipient(s), the employee or agent responsible for delivering it to the intended recipient(s), you are hereby notified that any use, dissemination, distribution or copying of this communication is strictly prohibited. If you have received this email in error, please return the original message to us by email and delete or destroy any copies. Please note any views or opinions expressed or presented in this email are solely those of the author and do not necessarily represent those of Bivio Networks, Inc. The recipient should check this email or any attachments for the presence of viruses or malware. Bivio Networks, Inc. accepts no responsibility for any damage caused by any virus or malware transmitted by this email. Thank you.
> >
> > Think Green when printing
> >
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From johanna at icir.org  Tue Feb 14 13:17:19 2017
From: johanna at icir.org (Johanna Amann)
Date: Tue, 14 Feb 2017 13:17:19 -0800
Subject: [Bro] Content gap breaks application layer analysis
In-Reply-To: <CAG+yijOVf4q2pB1mOTG+YCGb7Yjm6rgySW6TgH2rvbzksJuprg@mail.gmail.com>
References: <CAG+yijOVf4q2pB1mOTG+YCGb7Yjm6rgySW6TgH2rvbzksJuprg@mail.gmail.com>
Message-ID: <20170214211719.sekdzvip4g2b6t7l@wifi107.sys.ICSI.Berkeley.EDU>

Hi,

Bro does not deal well with disordered packets. There currently is no
workaround for that.

Johanna

On Mon, Feb 06, 2017 at 05:13:38PM +0800, duhang wrote:
> Hi,
> 
> I'm using Bro which listens to the nic card connects to a mirror port from
> a switch to dump http request/response and smtp email for further analysis.
> The packets that received from the mirror port are massively
> disordered(Unseen ACKed in wireshark). I saw a lot of content gap events
> which skips the following packets received. A lot of uncompleted http/smtp
> logs exist which relatively means high packet loss rate from appliance
> layer's perspective. Is there any workaround/solution to have
> bi-directional reassembly in this case?

> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From johanna at icir.org  Tue Feb 14 13:20:49 2017
From: johanna at icir.org (Johanna Amann)
Date: Tue, 14 Feb 2017 13:20:49 -0800
Subject: [Bro] Getting SSL events into Python
In-Reply-To: <181a0dd9-aec3-700f-92df-6dda30da1e66@babioch.de>
References: <181a0dd9-aec3-700f-92df-6dda30da1e66@babioch.de>
Message-ID: <20170214212049.3or3xiyaxkg736nf@wifi107.sys.ICSI.Berkeley.EDU>

Hello Karol,

by default, events that are handled locally inside of Bro are not sent
anywhere else; for core-raised events, you will have to re-raise them
(with a different name to not cause any issues), and set your
communication preferences so that the raised event will be forwarded to
broccoli (like done in broping.bro).

I hope this helps,
 Johanna

On Mon, Jan 30, 2017 at 03:34:53PM +0100, Karol Babioch wrote:
> Hi,
> 
> I'm currently researching SSL/TLS handshakes and want to process several
> events Bro provides with the SSL plugin. I've installed Bro along with
> broccoli and broccoli-python and the "broping" example (from the test
> directory) is working just fine. For each "ping" event I sent to Bro, a
> "pong" is received and processed in my Python script.
> 
> However, in case of the SSL my callbacks are never executed. The most
> simplified version looks something like this:
> 
> > #! /usr/bin/env python
> > 
> > from broccoli import *
> > 
> > @event
> > def ssl_established(c):
> >     print('established')
> > 
> > bc = Connection("127.0.0.1:47760")
> > 
> > while True:
> >     bc.processInput()
> 
> To my understanding I don't even have to load the SSL plugin, since it
> resides within "base", but nevertheless my local.bro contains the following:
> 
> > @load broping
> > @load base/protocols/ssl
> 
> When starting Bro and executing the Python script mentioned above,
> nothing happens, even if SSL traffic is going through the interface
> (and/or coming from a recorded pcap). I've also tried to register
> callbacks for various other SSL related events (ssl_client_hello,
> ssl_server_hello, etc.), but in no case were my callbacks invoked.
> 
> The only difference to the "broping.py" from the examples, is that I'm
> not sending any events, but just want to receive them (hence I'm calling
> processInput() regularly).
> 
> What am I missing here? Do I somehow need to enable the SSL
> functionality within Bro? How can I further debug the problem?
> 
> Any help is very much appreciated, since I've spent a fair amount of
> time on this already, with no real progress.
> 
> Thank you very much!
> 
> Best regards,
> Karol Babioch
> 




> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From karol at babioch.de  Tue Feb 14 14:26:25 2017
From: karol at babioch.de (Karol Babioch)
Date: Tue, 14 Feb 2017 23:26:25 +0100
Subject: [Bro] Getting SSL events into Python
In-Reply-To: <20170214212049.3or3xiyaxkg736nf@wifi107.sys.ICSI.Berkeley.EDU>
References: <181a0dd9-aec3-700f-92df-6dda30da1e66@babioch.de>
	<20170214212049.3or3xiyaxkg736nf@wifi107.sys.ICSI.Berkeley.EDU>
Message-ID: <1a4fa2d6-7bbe-75f5-05c8-51d7e08b7429@babioch.de>

Hello Johanna,

Am 14.02.2017 um 22:20 schrieb Johanna Amann:
> by default, events that are handled locally inside of Bro are not sent
> anywhere else; for core-raised events, you will have to re-raise them
> (with a different name to not cause any issues), and set your
> communication preferences so that the raised event will be forwarded to
> broccoli (like done in broping.bro).

thank you very much for your feedback. It is very much appreciated and
definitely helpful :-). I've worked around this problem for the time
being, but will revisit this now again :-).

Best regards,
Karol Babioch

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170214/e2d6289c/attachment-0001.bin 

From cherish_139 at foxmail.com  Tue Feb 14 18:28:31 2017
From: cherish_139 at foxmail.com (=?ISO-8859-1?B?Y2hlcmlzaCA=?=)
Date: Wed, 15 Feb 2017 10:28:31 +0800
Subject: [Bro] Passive DNS
Message-ID: <tencent_0AB7338C4946E96B1D136EA2@qq.com>

hi,i'm learning "Using Bro for Building Passive DNS Data", and i encounter the error(Permission denied) in executing these two long command.


 BRO_PDNS_DB=mysql://pdns:pdns at localhost/pdns /nsm/bro/share/bro/site/bro-pdns/bro_pdns.py serve
 BRO_PDNS_DB=mysql://pdns:pdns at localhost/pdns /nsm/bro/share/bro/site/bro-pdns/bro_pdns.py process /nsm/bro/logs/current/dns.log


Thanks for your reading ,would you please give me some advice ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170215/8013e694/attachment.html 

From twaller at bivio.net  Tue Feb 14 19:50:33 2017
From: twaller at bivio.net (Tony Waller)
Date: Wed, 15 Feb 2017 03:50:33 +0000
Subject: [Bro] Information on OCSP and CRL
In-Reply-To: <20170214211525.jsbcvteffga5mlpb@wifi107.sys.ICSI.Berkeley.EDU>
References: <EE2D32CF-B9DE-433F-9A01-D79001D1B33B@bivio.net>
	<m0efz76tpz.fsf@vladg.net>
	<20170214211525.jsbcvteffga5mlpb@wifi107.sys.ICSI.Berkeley.EDU>
Message-ID: <336B81D1-3EC0-4B9C-BEA6-470DD7CDFD0D@bivio.net>

Johanna,

Thank you for the update. Do you see any plans for Bro to be able to leverage a OCSP client soon? If not, is this something that could be added to the code-base by the community?  

Sincerely,

Tony


Tony Waller, CPO (USN-Ret.)
CISSP
Director, Systems Engineering
Bivio Networks, Inc.
?Powering Advanced Cyber Operations? (TM)
Mobile (443) 994-0936



*Note: The information contained in this email confidential. This information is intended only for the individual, individuals or entity to whom it is addressed. If you are not the intended recipient(s), the employee or agent responsible for delivering it to the intended recipient(s), you are hereby notified that any use, dissemination, distribution or copying of this communication is strictly prohibited. If you have received this email in error, please return the original message to us by email and delete or destroy any copies. Please note any views or opinions expressed or presented in this email are solely those of the author and do not necessarily represent those of Bivio Networks, Inc. The recipient should check this email or any attachments for the presence of viruses or malware. Bivio Networks, Inc. accepts no responsibility for any damage caused by any virus or malware transmitted by this email. Thank you.

Think Green when printing  


On 2/14/17, 4:15 PM, "Johanna Amann" <johanna at icir.org> wrote:

    To expand a bit on this...
    
    there is a policy script to perform certificate validation, just load
    policy/protocols/ssl/validate-certs.bro. Note that this uses plain OpenSSL
    for validation and might not be able to validate a few cases which
    browsers can validate.
    
    For OCSP, the situation is a bit less optimal. The script that Vlad
    mentioned performs OCSP validation, but _only_ in case that the OCSP
    response is sent stapled in the TLS handshake - which is rather rare.
    
    Outside of this, Bro currently does not support OCSP validatiob.
    
    Johanna
    
    On Thu, Feb 09, 2017 at 01:37:44PM -0600, Vlad Grigorescu wrote:
    > Tony,
    > 
    > There's an optional script for OCSP validation:
    > https://github.com/bro/bro/blob/v2.5/scripts/policy/protocols/ssl/validate-ocsp.bro
    > 
    > To use:
    > > @load protocols/ssl/validate-ocsp
    > 
    >   --Vlad
    > 
    > Tony Waller <twaller at bivio.net> writes:
    > 
    > > I am looking for additional information on utilizing OCSP and CRL in Bro NSM. I would like to know if certificates from clients can be checked in real-time or near real-time against a CRL? Also, can Bro NSM perform a OCSP request to a RA and check a certificate to determine if it is valid? If this is the case where in Bro NSM do you set the address for the RA or CRL responder?
    > >
    > > Sincerely,
    > >
    > > Tony
    > >
    > >
    > > Tony Waller, CISSP
    > > Director, Systems Engineering
    > > Bivio Networks, Inc.
    > > ?Powering Advanced Cyber Operations? (TM)
    > > Mobile (443) 994-0936
    > >
    > > [cid:image001.png at 01D27E32.049899E0]
    > >
    > > *Note: The information contained in this email confidential. This information is intended only for the individual, individuals or entity to whom it is addressed. If you are not the intended recipient(s), the employee or agent responsible for delivering it to the intended recipient(s), you are hereby notified that any use, dissemination, distribution or copying of this communication is strictly prohibited. If you have received this email in error, please return the original message to us by email and delete or destroy any copies. Please note any views or opinions expressed or presented in this email are solely those of the author and do not necessarily represent those of Bivio Networks, Inc. The recipient should check this email or any attachments for the presence of viruses or malware. Bivio Networks, Inc. accepts no responsibility for any damage caused by any virus or malware transmitted by this email. Thank you.
    > >
    > > Think Green when printing
    > >
    > > _______________________________________________
    > > Bro mailing list
    > > bro at bro-ids.org
    > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
    
    
    
    > _______________________________________________
    > Bro mailing list
    > bro at bro-ids.org
    > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
    
    



From hartley.87 at osu.edu  Tue Feb 14 21:18:27 2017
From: hartley.87 at osu.edu (Hartley, Christopher J.)
Date: Wed, 15 Feb 2017 05:18:27 +0000
Subject: [Bro] Passive DNS
In-Reply-To: <tencent_0AB7338C4946E96B1D136EA2@qq.com>
References: <tencent_0AB7338C4946E96B1D136EA2@qq.com>
Message-ID: <19609E39-CF29-4F48-BCD9-B7DE552DE270@osu.edu>

Do you have execute privileges for the python script?

Probably:
chmod u+x /nsm/bro/share/bro/site/bro-pdns/bro_pdns.py

Apologies if that?s not the case - I would expect a more descriptive error message or stack trace were that not the cause.

Chris

On Feb 14, 2017, at 9:28 PM, cherish <cherish_139 at foxmail.com<mailto:cherish_139 at foxmail.com>> wrote:

hi,i'm learning "Using Bro for Building Passive DNS Data", and i encounter the error(Permission denied) in executing these two long command.

 BRO_PDNS_DB=mysql://pdns:pdns at localhost/pdns /nsm/bro/share/bro/site/bro-pdns/bro_pdns.py serve
 BRO_PDNS_DB=mysql://pdns:pdns at localhost/pdns /nsm/bro/share/bro/site/bro-pdns/bro_pdns.py process /nsm/bro/logs/current/dns.log

Thanks for your reading ,would you please give me some advice ?
_______________________________________________
Bro mailing list
bro at bro-ids.org<mailto:bro at bro-ids.org>
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170215/0393e119/attachment-0001.html 

From Mirko.Sailio at vtt.fi  Wed Feb 15 02:03:47 2017
From: Mirko.Sailio at vtt.fi (Sailio Mirko)
Date: Wed, 15 Feb 2017 10:03:47 +0000
Subject: [Bro] bro_init asyncronicity problem
Message-ID: <EF7382457046724387424FDB89A723969A6BD42E@VTTMAIL3.ad.vtt.fi>

Hi

Does bro_init event stop before the other events are started? I have a init script, which removes some of ip addresses from monitoring, but am still getting detections for them (which is bad). :)
The datastructure is checked in "event new_connections()" for matches. The unwanted detection events only seems to happen in the very first moments after starting Bro, so I'm assuming that the problem occours because the rules are not yet in my datastructure, but could of course be wrong.

If bro_init does not (as default)  finish before other events are accepted, is there a way to force bro_init to finish first?

Thanks for any help,
Mirko

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170215/74a39a05/attachment.html 

From jazoff at illinois.edu  Wed Feb 15 05:35:03 2017
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Wed, 15 Feb 2017 13:35:03 +0000
Subject: [Bro] bro_init asyncronicity problem
In-Reply-To: <EF7382457046724387424FDB89A723969A6BD42E@VTTMAIL3.ad.vtt.fi>
References: <EF7382457046724387424FDB89A723969A6BD42E@VTTMAIL3.ad.vtt.fi>
Message-ID: <C5A655CF-3FFC-4830-BE3D-BC5C1AE18156@illinois.edu>


> On Feb 15, 2017, at 5:03 AM, Sailio Mirko <Mirko.Sailio at vtt.fi> wrote:
> 
>  I have a init script, which removes some of ip addresses from monitoring,

How are you doing this exactly?

-- 
- Justin Azoff



From johanna at icir.org  Wed Feb 15 07:38:18 2017
From: johanna at icir.org (Johanna Amann)
Date: Wed, 15 Feb 2017 07:38:18 -0800
Subject: [Bro] Information on OCSP and CRL
In-Reply-To: <336B81D1-3EC0-4B9C-BEA6-470DD7CDFD0D@bivio.net>
References: <EE2D32CF-B9DE-433F-9A01-D79001D1B33B@bivio.net>
	<m0efz76tpz.fsf@vladg.net>
	<20170214211525.jsbcvteffga5mlpb@wifi107.sys.ICSI.Berkeley.EDU>
	<336B81D1-3EC0-4B9C-BEA6-470DD7CDFD0D@bivio.net>
Message-ID: <86F86059-4E50-433E-8D3C-1B206CD97EE5@icir.org>

Hello Tony,

Bro will get the capability to parse OCSP requests and replies that are 
being sent over the network soon (as in - the ones that you see in HTTP, 
not just the ones in the stapled ocsp extension of the TLS handshake). 
You should be able to map that to certificates seen in the network 
traffic, and to validate them; however, this only will be possible for 
certificates for which network clients perform OCSP requests by 
themselves.

Apart from that, there are currently no plans to add any OCSP support to 
Bro; there especially are no plans to make Bro able to perform OCSP 
queries itself.

If you want to add that - you should be able to just write a couple of 
bifs and add them as a package :)

Johanna

On 14 Feb 2017, at 19:50, Tony Waller wrote:

> Johanna,
>
> Thank you for the update. Do you see any plans for Bro to be able to 
> leverage a OCSP client soon? If not, is this something that could be 
> added to the code-base by the community?
>
> Sincerely,
>
> Tony
>
>
> Tony Waller, CPO (USN-Ret.)
> CISSP
> Director, Systems Engineering
> Bivio Networks, Inc.
> ?Powering Advanced Cyber Operations? (TM)
> Mobile (443) 994-0936
>
>
>
> *Note: The information contained in this email confidential. This 
> information is intended only for the individual, individuals or entity 
> to whom it is addressed. If you are not the intended recipient(s), the 
> employee or agent responsible for delivering it to the intended 
> recipient(s), you are hereby notified that any use, dissemination, 
> distribution or copying of this communication is strictly prohibited. 
> If you have received this email in error, please return the original 
> message to us by email and delete or destroy any copies. Please note 
> any views or opinions expressed or presented in this email are solely 
> those of the author and do not necessarily represent those of Bivio 
> Networks, Inc. The recipient should check this email or any 
> attachments for the presence of viruses or malware. Bivio Networks, 
> Inc. accepts no responsibility for any damage caused by any virus or 
> malware transmitted by this email. Thank you.
>
> Think Green when printing
>
>
> On 2/14/17, 4:15 PM, "Johanna Amann" <johanna at icir.org> wrote:
>
>     To expand a bit on this...
>
>     there is a policy script to perform certificate validation, just 
> load
>     policy/protocols/ssl/validate-certs.bro. Note that this uses plain 
> OpenSSL
>     for validation and might not be able to validate a few cases which
>     browsers can validate.
>
>     For OCSP, the situation is a bit less optimal. The script that 
> Vlad
>     mentioned performs OCSP validation, but _only_ in case that the 
> OCSP
>     response is sent stapled in the TLS handshake - which is rather 
> rare.
>
>     Outside of this, Bro currently does not support OCSP validatiob.
>
>     Johanna
>
>     On Thu, Feb 09, 2017 at 01:37:44PM -0600, Vlad Grigorescu wrote:
>     > Tony,
>     >
>     > There's an optional script for OCSP validation:
>     > 
> https://github.com/bro/bro/blob/v2.5/scripts/policy/protocols/ssl/validate-ocsp.bro
>     >
>     > To use:
>     > > @load protocols/ssl/validate-ocsp
>     >
>     >   --Vlad
>     >
>     > Tony Waller <twaller at bivio.net> writes:
>     >
>     > > I am looking for additional information on utilizing OCSP and 
> CRL in Bro NSM. I would like to know if certificates from clients can 
> be checked in real-time or near real-time against a CRL? Also, can Bro 
> NSM perform a OCSP request to a RA and check a certificate to 
> determine if it is valid? If this is the case where in Bro NSM do you 
> set the address for the RA or CRL responder?
>     > >
>     > > Sincerely,
>     > >
>     > > Tony
>     > >
>     > >
>     > > Tony Waller, CISSP
>     > > Director, Systems Engineering
>     > > Bivio Networks, Inc.
>     > > ?Powering Advanced Cyber Operations? (TM)
>     > > Mobile (443) 994-0936
>     > >
>     > > [cid:image001.png at 01D27E32.049899E0]
>     > >
>     > > *Note: The information contained in this email confidential. 
> This information is intended only for the individual, individuals or 
> entity to whom it is addressed. If you are not the intended 
> recipient(s), the employee or agent responsible for delivering it to 
> the intended recipient(s), you are hereby notified that any use, 
> dissemination, distribution or copying of this communication is 
> strictly prohibited. If you have received this email in error, please 
> return the original message to us by email and delete or destroy any 
> copies. Please note any views or opinions expressed or presented in 
> this email are solely those of the author and do not necessarily 
> represent those of Bivio Networks, Inc. The recipient should check 
> this email or any attachments for the presence of viruses or malware. 
> Bivio Networks, Inc. accepts no responsibility for any damage caused 
> by any virus or malware transmitted by this email. Thank you.
>     > >
>     > > Think Green when printing
>     > >
>     > > _______________________________________________
>     > > Bro mailing list
>     > > bro at bro-ids.org
>     > > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
>
>     > _______________________________________________
>     > Bro mailing list
>     > bro at bro-ids.org
>     > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From john.b.althouse at gmail.com  Wed Feb 15 10:23:23 2017
From: john.b.althouse at gmail.com (John B. Althouse)
Date: Wed, 15 Feb 2017 13:23:23 -0500
Subject: [Bro] Function for Decimal to Hex
Message-ID: <CALW-b1zU6HZL0DMFcMxAGtfKVapkRgG6htGv9z9NLo=97j_VGA@mail.gmail.com>

Is there a function within Bro that will allow me to turn decimal into hex?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170215/9f5d22ea/attachment.html 

From dnthayer at illinois.edu  Wed Feb 15 10:44:21 2017
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Wed, 15 Feb 2017 12:44:21 -0600
Subject: [Bro] Function for Decimal to Hex
In-Reply-To: <CALW-b1zU6HZL0DMFcMxAGtfKVapkRgG6htGv9z9NLo=97j_VGA@mail.gmail.com>
References: <CALW-b1zU6HZL0DMFcMxAGtfKVapkRgG6htGv9z9NLo=97j_VGA@mail.gmail.com>
Message-ID: <ec46b959-8336-13c8-7732-353673aee655@illinois.edu>

You could do something like this ("d" is a count or int value):

print fmt("%x", d);



On 2/15/17 12:23 PM, John B. Althouse wrote:
>
> Is there a function within Bro that will allow me to turn decimal into hex?
>
>

From dopheide at gmail.com  Wed Feb 15 10:52:01 2017
From: dopheide at gmail.com (Mike Dopheide)
Date: Wed, 15 Feb 2017 12:52:01 -0600
Subject: [Bro] Function for Decimal to Hex
In-Reply-To: <ec46b959-8336-13c8-7732-353673aee655@illinois.edu>
References: <CALW-b1zU6HZL0DMFcMxAGtfKVapkRgG6htGv9z9NLo=97j_VGA@mail.gmail.com>
	<ec46b959-8336-13c8-7732-353673aee655@illinois.edu>
Message-ID: <CAPy2kFY9ZKRjBVDoDYOdTbkpXQ2_DuTuGp9zPr+bwmjPwO8J=g@mail.gmail.com>

Or, if your decimal is already a string, there's always this:

https://www.bro.org/sphinx/scripts/base/bif/bro.bif.bro.html#id-bytestring_to_hexstr

-Dop

On Wed, Feb 15, 2017 at 12:44 PM, Daniel Thayer <dnthayer at illinois.edu>
wrote:

> You could do something like this ("d" is a count or int value):
>
> print fmt("%x", d);
>
>
>
> On 2/15/17 12:23 PM, John B. Althouse wrote:
> >
> > Is there a function within Bro that will allow me to turn decimal into
> hex?
> >
> >
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170215/8a8a3238/attachment-0001.html 

From john.b.althouse at gmail.com  Wed Feb 15 11:06:43 2017
From: john.b.althouse at gmail.com (John B. Althouse)
Date: Wed, 15 Feb 2017 14:06:43 -0500
Subject: [Bro] Function for Decimal to Hex
In-Reply-To: <CAPy2kFY9ZKRjBVDoDYOdTbkpXQ2_DuTuGp9zPr+bwmjPwO8J=g@mail.gmail.com>
References: <CALW-b1zU6HZL0DMFcMxAGtfKVapkRgG6htGv9z9NLo=97j_VGA@mail.gmail.com>
	<ec46b959-8336-13c8-7732-353673aee655@illinois.edu>
	<CAPy2kFY9ZKRjBVDoDYOdTbkpXQ2_DuTuGp9zPr+bwmjPwO8J=g@mail.gmail.com>
Message-ID: <CALW-b1z=fY1SDgrvRp4-O=71GUq93WeZKvFm_yPhS98dcuyBJA@mail.gmail.com>

Sorry, what I want to do is take an integer like:
32154
and convert it to hexidecimal:
7D9A

On Wed, Feb 15, 2017 at 1:52 PM, Mike Dopheide <dopheide at gmail.com> wrote:

> Or, if your decimal is already a string, there's always this:
>
> https://www.bro.org/sphinx/scripts/base/bif/bro.bif.bro.html
> #id-bytestring_to_hexstr
>
> -Dop
>
> On Wed, Feb 15, 2017 at 12:44 PM, Daniel Thayer <dnthayer at illinois.edu>
> wrote:
>
>> You could do something like this ("d" is a count or int value):
>>
>> print fmt("%x", d);
>>
>>
>>
>> On 2/15/17 12:23 PM, John B. Althouse wrote:
>> >
>> > Is there a function within Bro that will allow me to turn decimal into
>> hex?
>> >
>> >
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170215/19b096b0/attachment.html 

From dopheide at gmail.com  Wed Feb 15 11:41:07 2017
From: dopheide at gmail.com (Mike Dopheide)
Date: Wed, 15 Feb 2017 13:41:07 -0600
Subject: [Bro] Function for Decimal to Hex
In-Reply-To: <CALW-b1z=fY1SDgrvRp4-O=71GUq93WeZKvFm_yPhS98dcuyBJA@mail.gmail.com>
References: <CALW-b1zU6HZL0DMFcMxAGtfKVapkRgG6htGv9z9NLo=97j_VGA@mail.gmail.com>
	<ec46b959-8336-13c8-7732-353673aee655@illinois.edu>
	<CAPy2kFY9ZKRjBVDoDYOdTbkpXQ2_DuTuGp9zPr+bwmjPwO8J=g@mail.gmail.com>
	<CALW-b1z=fY1SDgrvRp4-O=71GUq93WeZKvFm_yPhS98dcuyBJA@mail.gmail.com>
Message-ID: <CAPy2kFaQUWBahCi=C1-g_RWfs-Qu=vuA7S63YrJ8kCxdoPScrA@mail.gmail.com>

Then what Daniel said is your way forward, if you need capital letters,
something like this would work:

        local num: count = 32154;
        print fmt("%s",to_upper(fmt("%x",num)));

-Dop

On Wed, Feb 15, 2017 at 1:06 PM, John B. Althouse <john.b.althouse at gmail.com
> wrote:

> Sorry, what I want to do is take an integer like:
> 32154
> and convert it to hexidecimal:
> 7D9A
>
> On Wed, Feb 15, 2017 at 1:52 PM, Mike Dopheide <dopheide at gmail.com> wrote:
>
>> Or, if your decimal is already a string, there's always this:
>>
>> https://www.bro.org/sphinx/scripts/base/bif/bro.bif.bro.html
>> #id-bytestring_to_hexstr
>>
>> -Dop
>>
>> On Wed, Feb 15, 2017 at 12:44 PM, Daniel Thayer <dnthayer at illinois.edu>
>> wrote:
>>
>>> You could do something like this ("d" is a count or int value):
>>>
>>> print fmt("%x", d);
>>>
>>>
>>>
>>> On 2/15/17 12:23 PM, John B. Althouse wrote:
>>> >
>>> > Is there a function within Bro that will allow me to turn decimal into
>>> hex?
>>> >
>>> >
>>> _______________________________________________
>>> Bro mailing list
>>> bro at bro-ids.org
>>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170215/5666096e/attachment.html 

From dnthayer at illinois.edu  Wed Feb 15 11:52:22 2017
From: dnthayer at illinois.edu (Daniel Thayer)
Date: Wed, 15 Feb 2017 13:52:22 -0600
Subject: [Bro] Function for Decimal to Hex
In-Reply-To: <CAPy2kFaQUWBahCi=C1-g_RWfs-Qu=vuA7S63YrJ8kCxdoPScrA@mail.gmail.com>
References: <CALW-b1zU6HZL0DMFcMxAGtfKVapkRgG6htGv9z9NLo=97j_VGA@mail.gmail.com>
	<ec46b959-8336-13c8-7732-353673aee655@illinois.edu>
	<CAPy2kFY9ZKRjBVDoDYOdTbkpXQ2_DuTuGp9zPr+bwmjPwO8J=g@mail.gmail.com>
	<CALW-b1z=fY1SDgrvRp4-O=71GUq93WeZKvFm_yPhS98dcuyBJA@mail.gmail.com>
	<CAPy2kFaQUWBahCi=C1-g_RWfs-Qu=vuA7S63YrJ8kCxdoPScrA@mail.gmail.com>
Message-ID: <46828cb3-4f77-767e-653b-9ab665646c44@illinois.edu>

The to_upper() function returns a string, so your example can be simplified:

   print to_upper(fmt("%x",num));


On 2/15/17 1:41 PM, Mike Dopheide wrote:
> Then what Daniel said is your way forward, if you need capital letters,
> something like this would work:
>
>         local num: count = 32154;
>         print fmt("%s",to_upper(fmt("%x",num)));
>
> -Dop
>

From john.b.althouse at gmail.com  Wed Feb 15 12:57:57 2017
From: john.b.althouse at gmail.com (John B. Althouse)
Date: Wed, 15 Feb 2017 15:57:57 -0500
Subject: [Bro] Function for Decimal to Hex
In-Reply-To: <46828cb3-4f77-767e-653b-9ab665646c44@illinois.edu>
References: <CALW-b1zU6HZL0DMFcMxAGtfKVapkRgG6htGv9z9NLo=97j_VGA@mail.gmail.com>
	<ec46b959-8336-13c8-7732-353673aee655@illinois.edu>
	<CAPy2kFY9ZKRjBVDoDYOdTbkpXQ2_DuTuGp9zPr+bwmjPwO8J=g@mail.gmail.com>
	<CALW-b1z=fY1SDgrvRp4-O=71GUq93WeZKvFm_yPhS98dcuyBJA@mail.gmail.com>
	<CAPy2kFaQUWBahCi=C1-g_RWfs-Qu=vuA7S63YrJ8kCxdoPScrA@mail.gmail.com>
	<46828cb3-4f77-767e-653b-9ab665646c44@illinois.edu>
Message-ID: <CALW-b1xDeR99cYbpAKDmGoAaD+mHf_5=bbsFG6TYk8OcKwy1Jw@mail.gmail.com>

Awesome, works! Thank you guys!

On Wed, Feb 15, 2017 at 2:52 PM, Daniel Thayer <dnthayer at illinois.edu>
wrote:

> The to_upper() function returns a string, so your example can be
> simplified:
>
>    print to_upper(fmt("%x",num));
>
>
> On 2/15/17 1:41 PM, Mike Dopheide wrote:
> > Then what Daniel said is your way forward, if you need capital letters,
> > something like this would work:
> >
> >         local num: count = 32154;
> >         print fmt("%s",to_upper(fmt("%x",num)));
> >
> > -Dop
> >
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170215/351d584c/attachment.html 

From jim.simpson.work at gmail.com  Wed Feb 15 13:14:21 2017
From: jim.simpson.work at gmail.com (Jim Simpson)
Date: Wed, 15 Feb 2017 16:14:21 -0500
Subject: [Bro] Getting flow stats from Bro
Message-ID: <CAJKYhs-Tcs90yfwTWnRe54UF5X-2GQcFRGCzwTiZCgdVVAFWFA@mail.gmail.com>

Is there an existing set of scripts for Bro to get flow stats?

I'm looking for counts, avg, and std dev on small packets, large packets,
nonempty packets, interarrival times, etc, similar to what YAF gives with
the `--flow-stats` option. I'm also interested in the Shannon entropy of
the payload, similar to what YAF gives with the `--entropy` option.
https://tools.netsa.cert.org/yaf/yaf.html

- Jim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170215/a60a0f61/attachment.html 

From fatema.bannatwala at gmail.com  Wed Feb 15 13:37:18 2017
From: fatema.bannatwala at gmail.com (fatema bannatwala)
Date: Wed, 15 Feb 2017 16:37:18 -0500
Subject: [Bro] Bro seg faults when used with gperftools monitoring.
Message-ID: <CACX0rURDmxA_=y6C3yhNNBK3ZVda3wsPMDcGaN9yEn-UvOY5tQ@mail.gmail.com>

So, I wanted to find out whether there are any memory leaks in any of the
code/scripts,
that I am loading with Bro, therefore I installed gperftools and compiled
Bro 2.5 on a VM
with 5GB of memory and centos 7.2.

I loaded only the default scripts to start with, i.e didn't include any
custom scripts in local.bro.
I ran Bro as default on a pcap (~11G) and it terminated normally producing
output log files.

When I ran Bro with Perftools heap leak checker active on the same pcap, it
seg faulted
after some time.
I was watching memory usage during the run and it didn't seem to be using
100% of available
memory.

Hence wanted to ask Is there any reason Bro would seg fault while run with
HEAPCHECK enabled?

My goal was to load custom scripts one by one and everytime run bro with
heap check
to see what can cause a memory leak. But stumbled upon this at the very
beginning,
so until I find out why it seg faulted on default configs, I can't really
check the custom scripts/code
for memory leak :( :(

Including the coredump file.

Thanks,
Fatema.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170215/4fc0b1f6/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dump
Type: application/octet-stream
Size: 32024 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170215/4fc0b1f6/attachment-0001.obj 

From Izik.Birka at hot.net.il  Wed Feb 15 23:35:58 2017
From: Izik.Birka at hot.net.il (Izik Birka)
Date: Thu, 16 Feb 2017 07:35:58 +0000
Subject: [Bro] SMB
References: <592228F4D0C8504187F2F76658040CB6DFE23DFF@HOT-MAILBOX-02.HOT.NET.IL>
	<DM5PR0101MB3068AEF447F6C29DC081F05BFE590@DM5PR0101MB3068.prod.exchangelabs.com>
Message-ID: <592228F4D0C8504187F2F76658040CB6DFE39D1A@HOT-MAILBOX-02.HOT.NET.IL>

Hi
Any idea ?











From: Izik Birka
Sent: Tuesday, February 14, 2017 9:15 AM
To: 'Martin, Eric J' <ejmartin2 at wpi.edu>
Subject: RE: SMB

Hi
I enable them and it's great but I'm looking for SMB bytes statistics , like in conn.log file
For example if someone downloaded 300 MB with SMB protocol (form network share) , is there any file that hold this statistics  ?

with http protocol , I can find it in conn.log file


thanks



From: Martin, Eric J [mailto:ejmartin2 at wpi.edu]
Sent: Tuesday, February 14, 2017 12:09 AM
To: Izik Birka <Izik.Birka at hot.net.il<mailto:Izik.Birka at hot.net.il>>
Subject: Re: SMB


There's smb_files and smb_mappings that need to be enabled.  When you say 'stats', what are you looking for?


--

Eric Martin

ejmartin2 at wpi.edu<mailto:ejmartin2 at wpi.edu>


Information Security Analyst

Office: (508) 831-6070


Worcester Polytechnic Institute

www.wpi.edu<http://www.wpi.edu>

PGP: C74F 1EBF 2E80 7984 8CB5  064E BF17 D34C C704 B30F
For security purposes, this message has been double ROT13 encoded

________________________________
From: bro-bounces at bro.org<mailto:bro-bounces at bro.org> <bro-bounces at bro.org<mailto:bro-bounces at bro.org>> on behalf of Izik Birka <Izik.Birka at hot.net.il<mailto:Izik.Birka at hot.net.il>>
Sent: Monday, February 13, 2017 3:34:29 AM
To: bro at bro.org<mailto:bro at bro.org>
Subject: [Bro] SMB

Hi
Is there any logs that contains SMB stats ? why conn.log doesn't contains SMB connection ?

I have bro 2.5

Thanks
Izik Birka


This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain materials protected by copyright or information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or agreement. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication by error, notify the sender immediately and delete this message immediately. Thank you.

This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain materials protected by copyright or information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or agreement.

If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication by error, notify the sender immediately and delete this message immediately.

Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170216/933ba89f/attachment.html 

From Izik.Birka at hot.net.il  Wed Feb 15 23:36:34 2017
From: Izik.Birka at hot.net.il (Izik Birka)
Date: Thu, 16 Feb 2017 07:36:34 +0000
Subject: [Bro] SMB Language
References: <592228F4D0C8504187F2F76658040CB6DFE2B50A@HOT-MAILBOX-02.HOT.NET.IL>
	<ACD07475-0FEE-4946-9068-39271F6845E9@illinois.edu> 
Message-ID: <592228F4D0C8504187F2F76658040CB6DFE39D45@HOT-MAILBOX-02.HOT.NET.IL>

Hi

Any help ?

Thanks
Izik
-----Original Message-----
From: Izik Birka 
Sent: Tuesday, February 14, 2017 5:12 PM
To: 'Azoff, Justin S' <jazoff at illinois.edu>
Cc: bro at bro.org
Subject: RE: [Bro] SMB Language

Can I add utf-8 encoding to bro ?
Is there other option ?

 I'm forward the log file to SIEM system , and it's not readable in that way

Thanks



-----Original Message-----
From: Azoff, Justin S [mailto:jazoff at illinois.edu] 
Sent: Tuesday, February 14, 2017 4:16 PM
To: Izik Birka <Izik.Birka at hot.net.il>
Cc: bro at bro.org
Subject: Re: [Bro] SMB Language


> On Feb 14, 2017, at 2:36 AM, Izik Birka <Izik.Birka at hot.net.il> wrote:
> 
> Hi
> Just enable SMB analyzer , works great
>  
> I have a problem with the Hebrew language , it's looks like it's not supported , I'm getting  this files name in log file :
>  
> Test\hello\\\xd7\x92\xd7\x99\xd7\x95\xd7\x9c \xd7\x9b\xd7\x9e\xd7\x95\xd7\xaa\xd7\x99.csv
>  
> The \\\xd7\x92\xd7\x99\xd7\x95\xd7\x9c - is Hebrew words 
>  
> is there a way to fix it ?

That's just an escaped utf-8 string:

>>> s='Test\hello\\\xd7\x92\xd7\x99\xd7\x95\xd7\x9c \xd7\x9b\xd7\x9e\xd7\x95\xd7\xaa\xd7\x99.csv'
>>> print s
Test\hello\???? ?????.csv

(or in python3)

>>> s=b'Test\hello\\\xd7\x92\xd7\x99\xd7\x95\xd7\x9c \xd7\x9b\xd7\x9e\xd7\x95\xd7\xaa\xd7\x99.csv'
>>> print(s.decode('utf-8'))
Test\hello\???? ?????.csv



-- 
- Justin Azoff


This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain materials protected by copyright or information that is non-public, proprietary, privileged, confidential, and exempt from disclosure under applicable law or agreement.

If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication by error, notify the sender immediately and delete this message immediately.

Thank you.


From andrew.dellana at bayer.com  Thu Feb 16 07:40:23 2017
From: andrew.dellana at bayer.com (Andrew Dellana)
Date: Thu, 16 Feb 2017 15:40:23 +0000
Subject: [Bro] Connection summary <???> values
In-Reply-To: <m0wpcs6cf5.fsf@vladg.net>
References: <40f9b6e86ef34a8bb8046a97fcc822c9@moxde9.na.bayer.cnb>
	<m0wpcs6cf5.fsf@vladg.net>
Message-ID: <32339691ef1544028846b16f410b8dac@moxde9.na.bayer.cnb>

Thanks! Is there a way to resolve the DNS connection issues?

Freundliche Gr??e / Best regards,

Andrew Dellana
Intern
________________________

-----Original Message-----
From: Vlad Grigorescu [mailto:vladg at illinois.edu] 
Sent: Tuesday, February 14, 2017 10:05 AM
To: Andrew Dellana; bro at bro.org
Subject: Re: [Bro] Connection summary <???> values

<???> simply means that DNS resolution failed.

Andrew Dellana <andrew.dellana at bayer.com> writes:

> Hello,
>
> For the connection summary is there a way to reformat the way the notification looks - Like put each of these on their own line?  Also is there way to make the <???> value show its domain name?  The <???> value shows up in both the incoming, outgoing and total sections.
>
>
>
>         #1=<???>  #2=(redacted)  #3=<???>
>
>         #4= (redacted) #5= (redacted) #6= (redacted)
>
>         #7= (redacted) #8=<???>  #9=<???>
>
>         #10= (redacted) #11=<???>  #12= (redacted)
>
>         #13=<???>  #14= (redacted) #15=<???>
>
>         #16= (redacted) #17=<???>  #18=<???>
>
>         #19=<???>  #20= (redacted)
>
>
>
>
> Thanks,
>
> Andrew Dellana
> Intern
> ________________________
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From vladg at illinois.edu  Thu Feb 16 08:14:10 2017
From: vladg at illinois.edu (Vlad Grigorescu)
Date: Thu, 16 Feb 2017 10:14:10 -0600
Subject: [Bro] Bro seg faults when used with gperftools monitoring.
In-Reply-To: <CACX0rURDmxA_=y6C3yhNNBK3ZVda3wsPMDcGaN9yEn-UvOY5tQ@mail.gmail.com>
References: <CACX0rURDmxA_=y6C3yhNNBK3ZVda3wsPMDcGaN9yEn-UvOY5tQ@mail.gmail.com>
Message-ID: <m0k28q5d0t.fsf@vladg.net>

That's a common config that we test with which I would expect to work.
Can you provide some details on how exactly you built and ran Bro?

  --Vlad

fatema bannatwala <fatema.bannatwala at gmail.com> writes:

> So, I wanted to find out whether there are any memory leaks in any of the
> code/scripts,
> that I am loading with Bro, therefore I installed gperftools and compiled
> Bro 2.5 on a VM
> with 5GB of memory and centos 7.2.
>
> I loaded only the default scripts to start with, i.e didn't include any
> custom scripts in local.bro.
> I ran Bro as default on a pcap (~11G) and it terminated normally producing
> output log files.
>
> When I ran Bro with Perftools heap leak checker active on the same pcap, it
> seg faulted
> after some time.
> I was watching memory usage during the run and it didn't seem to be using
> 100% of available
> memory.
>
> Hence wanted to ask Is there any reason Bro would seg fault while run with
> HEAPCHECK enabled?
>
> My goal was to load custom scripts one by one and everytime run bro with
> heap check
> to see what can cause a memory leak. But stumbled upon this at the very
> beginning,
> so until I find out why it seg faulted on default configs, I can't really
> check the custom scripts/code
> for memory leak :( :(
>
> Including the coredump file.
>
> Thanks,
> Fatema.
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 800 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170216/06f7cf2d/attachment.bin 

From vladg at illinois.edu  Thu Feb 16 08:15:35 2017
From: vladg at illinois.edu (Vlad Grigorescu)
Date: Thu, 16 Feb 2017 10:15:35 -0600
Subject: [Bro] Getting flow stats from Bro
In-Reply-To: <CAJKYhs-Tcs90yfwTWnRe54UF5X-2GQcFRGCzwTiZCgdVVAFWFA@mail.gmail.com>
References: <CAJKYhs-Tcs90yfwTWnRe54UF5X-2GQcFRGCzwTiZCgdVVAFWFA@mail.gmail.com>
Message-ID: <m0h93u5cyg.fsf@vladg.net>

Not set of scripts for this that I'm aware of. The closest thing I'm
aware of is this script for computing PCR, which might be a good
jumping-off point at least:

https://github.com/reservoirlabs/bro-producer-consumer-ratio

  --Vlad

Jim Simpson <jim.simpson.work at gmail.com> writes:

> Is there an existing set of scripts for Bro to get flow stats?
>
> I'm looking for counts, avg, and std dev on small packets, large packets,
> nonempty packets, interarrival times, etc, similar to what YAF gives with
> the `--flow-stats` option. I'm also interested in the Shannon entropy of
> the payload, similar to what YAF gives with the `--entropy` option.
> https://tools.netsa.cert.org/yaf/yaf.html
>
> - Jim
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 800 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170216/f46acdad/attachment.bin 

From espressobeanies at gmail.com  Thu Feb 16 11:05:19 2017
From: espressobeanies at gmail.com (Espresso Beanies)
Date: Thu, 16 Feb 2017 14:05:19 -0500
Subject: [Bro] Question on PacketFilter::DroppedPackets
Message-ID: <CAOeD169CdYpZ8j4kVKhFtbqwFFt0mkH2dWYC_AUmGgQ8z_cuFw@mail.gmail.com>

Hi,

It seems no matter what I do, I still get these notices
"PacketFilter::DroppedPackets". I created more workers, but I have a
question about creating new workers via using an existing worker to capture
on the same interface using the "lb_procs" method to up the number of
"sub-threads?" for multi-CPU processing. What advantage does a new worker
give me over "lb_procs"?

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170216/0315361a/attachment.html 

From seth at icir.org  Thu Feb 16 11:20:27 2017
From: seth at icir.org (Seth Hall)
Date: Thu, 16 Feb 2017 14:20:27 -0500
Subject: [Bro] SMB Language
In-Reply-To: <592228F4D0C8504187F2F76658040CB6DFE39D45@HOT-MAILBOX-02.HOT.NET.IL>
References: <592228F4D0C8504187F2F76658040CB6DFE2B50A@HOT-MAILBOX-02.HOT.NET.IL>
	<ACD07475-0FEE-4946-9068-39271F6845E9@illinois.edu>
	<592228F4D0C8504187F2F76658040CB6DFE39D45@HOT-MAILBOX-02.HOT.NET.IL>
Message-ID: <25F2BF5F-A053-42A0-9D48-E69ECB886EC0@icir.org>


> On Feb 16, 2017, at 2:36 AM, Izik Birka <Izik.Birka at hot.net.il> wrote:
> 
> Any help ?

Bro's strings are purely ascii at this time.  You would have to interpret the string as UTF8 in whatever you have receiving logs with.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From fatema.bannatwala at gmail.com  Thu Feb 16 11:45:35 2017
From: fatema.bannatwala at gmail.com (fatema bannatwala)
Date: Thu, 16 Feb 2017 14:45:35 -0500
Subject: [Bro] Bro seg faults when used with gperftools monitoring.
In-Reply-To: <m0k28q5d0t.fsf@vladg.net>
References: <CACX0rURDmxA_=y6C3yhNNBK3ZVda3wsPMDcGaN9yEn-UvOY5tQ@mail.gmail.com>
	<m0k28q5d0t.fsf@vladg.net>
Message-ID: <CACX0rUReia-OhM6taL4FpemY91ocUSxQMkcSb=bJwU2oDWR2cw@mail.gmail.com>

Hi Vlad,

I installed Bro from source by getting the tar of v2.5 and performing
following to compile and install it:

$ sudo yum install cmake make libpcap-devel openssl-devel python-devel swig
zlib-devel
$ tar -xvfz bro-2.5.tar.gz
$ cd bro-2.5
$./configure --enable-debug --enable-perftools --enable-perftools-debug
$ make
$ sudo make install

And before installing Bro, I installed gperftools according to the
directions mentioned by the Project Readme:

$ wget
http://download.savannah.gnu.org/releases/libunwind/libunwind-0.99-beta.tar.gz
$ tar -xvzf libunwind-0.99-beta.tar.gz
$ cd libunwind-0.99-beta
$./configure
$ make
$ sudo  make install

$ sudo yum groupinstall 'Development Tools'
$ unzip gperftools-master.zip
$ cd gperftools-master
$ ./autogen.sh
$ ./configure
$ make
$ sudo make install

I also ran bro with a small pcap trace (~319MB) capture from one of the Bro
sensors, but it seg faulted.

# bro -m -r /home/fatemabw/bro-test-vvvsmall.pcap
WARNING: Perftools heap leak checker is active -- Performance may suffer
Segmentation fault (core dumped)


On Thu, Feb 16, 2017 at 11:14 AM, Vlad Grigorescu <vladg at illinois.edu>
wrote:

> That's a common config that we test with which I would expect to work.
> Can you provide some details on how exactly you built and ran Bro?
>
>   --Vlad
>
> fatema bannatwala <fatema.bannatwala at gmail.com> writes:
>
> > So, I wanted to find out whether there are any memory leaks in any of the
> > code/scripts,
> > that I am loading with Bro, therefore I installed gperftools and compiled
> > Bro 2.5 on a VM
> > with 5GB of memory and centos 7.2.
> >
> > I loaded only the default scripts to start with, i.e didn't include any
> > custom scripts in local.bro.
> > I ran Bro as default on a pcap (~11G) and it terminated normally
> producing
> > output log files.
> >
> > When I ran Bro with Perftools heap leak checker active on the same pcap,
> it
> > seg faulted
> > after some time.
> > I was watching memory usage during the run and it didn't seem to be using
> > 100% of available
> > memory.
> >
> > Hence wanted to ask Is there any reason Bro would seg fault while run
> with
> > HEAPCHECK enabled?
> >
> > My goal was to load custom scripts one by one and everytime run bro with
> > heap check
> > to see what can cause a memory leak. But stumbled upon this at the very
> > beginning,
> > so until I find out why it seg faulted on default configs, I can't really
> > check the custom scripts/code
> > for memory leak :( :(
> >
> > Including the coredump file.
> >
> > Thanks,
> > Fatema.
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170216/4680b732/attachment.html 

From seth at icir.org  Thu Feb 16 13:06:32 2017
From: seth at icir.org (Seth Hall)
Date: Thu, 16 Feb 2017 16:06:32 -0500
Subject: [Bro] Conceptual question on main.bro files
In-Reply-To: <CAM1mTCqPW_bcG=pnZByq=o8zTOdf-MoDnth4QQono=k9haWHrw@mail.gmail.com>
References: <CAOeD16_-M7ohetowev1KP4JXe2CAgsPJzxEZ7oWZr_UJpod3BQ@mail.gmail.com>
	<CAM1mTCqPW_bcG=pnZByq=o8zTOdf-MoDnth4QQono=k9haWHrw@mail.gmail.com>
Message-ID: <928BA6E5-65BA-4D62-9AC3-783A2BEF224D@icir.org>


> On Feb 14, 2017, at 3:12 PM, Zeolla at GMail.com <zeolla at gmail.com> wrote:
> 
> Hope that helps - also, please correct me if there is an actual main.bro convention anywhere that I'm not aware of.

That was a great description, and no, there is no formal standard around that.  It's just a convention we started using a long time ago.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From bro at pingtrip.com  Thu Feb 16 18:50:52 2017
From: bro at pingtrip.com (Dave Crawford)
Date: Thu, 16 Feb 2017 21:50:52 -0500
Subject: [Bro] Netmap Seg faults
Message-ID: <E61E0444-2CA3-45FF-A478-668FB11C8F85@pingtrip.com>

Has anyone experienced segfaults with Bro + Netmap when executing a ?broctrl stop'?

1487299650.431866 818913 packets received on interface bro}3, 0 dropped
/opt/bro/share/broctl/scripts/run-bro: line 107:  4821 Segmentation fault      nohup "$mybro" "$@"


-Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170216/18304182/attachment.html 

From bro at pingtrip.com  Thu Feb 16 20:13:41 2017
From: bro at pingtrip.com (Dave Crawford)
Date: Thu, 16 Feb 2017 23:13:41 -0500
Subject: [Bro] Netmap Seg faults
In-Reply-To: <E61E0444-2CA3-45FF-A478-668FB11C8F85@pingtrip.com>
References: <E61E0444-2CA3-45FF-A478-668FB11C8F85@pingtrip.com>
Message-ID: <0555C89E-4F7B-4FE0-AAEC-5EFBF06C7FCF@pingtrip.com>


> On Feb 16, 2017, at 9:50 PM, Dave Crawford <bro at pingtrip.com> wrote:
> 
> Has anyone experienced segfaults with Bro + Netmap when executing a ?broctrl stop'?
> 
> 1487299650.431866 818913 packets received on interface bro}3, 0 dropped
> /opt/bro/share/broctl/scripts/run-bro: line 107:  4821 Segmentation fault      nohup "$mybro" "$@"
> 
> 

Also seeing these messages in dmesg:

[ 8113.725495] bro[2098]: segfault at 0 ip 00007f7695f360f7 sp 00007fffe8969360 error 4 in libtcmalloc.so.4.2.2[7f7695eeb000+98000]
[ 8113.766085] bro[2088]: segfault at 0 ip 00007f6584ff40f7 sp 00007ffc40fcf7f0 error 4 in libtcmalloc.so.4.2.2[7f6584fa9000+98000]
[29084.773876] bro[4823]: segfault at 0 ip 00007f116704d0f7 sp 00007ffe93723430 error 4 in libtcmalloc.so.4.2.2[7f1167002000+98000]
[29084.787171] bro[4821]: segfault at 0 ip 00007f013f9610f7 sp 00007ffe0d0a9950 error 4 in libtcmalloc.so.4.2.2[7f013f916000+98000]

-Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170216/62293022/attachment.html 

From seth at icir.org  Fri Feb 17 07:53:01 2017
From: seth at icir.org (Seth Hall)
Date: Fri, 17 Feb 2017 10:53:01 -0500
Subject: [Bro] Netmap plugin issue
In-Reply-To: <ECCADAC6-F02E-4645-91E9-40BE5FB99966@pingtrip.com>
References: <9496c58b8b164eaba586a276eb56cb89@exch13-mail04.win.slac.stanford.edu>
	<237C84FB-3F36-4C72-BB55-ED91615D934C@icir.org>
	<7e14f24d17134bdbb13a4e41a8ec9e87@exch13-mail04.win.slac.stanford.edu>
	<F91195A4-A35D-48B8-B893-63E66E8821A4@pingtrip.com>
	<6dcdf9f8999249ef9018e84c53c897fd@exch13-mail04.win.slac.stanford.edu>
	<35224580-AE4F-45AE-8046-303758FF9A9E@icir.org>
	<95315aa48ece4e8696e229da0879d445@exch13-mail04.win.slac.stanford.edu>
	<34442F24-0A8B-4EE7-A835-52707EA20478@pingtrip.com>
	<fb7aab1d41564392b45268b7c3594163@exch13-mail04.win.slac.stanford.edu>
	<03503572-1528-40F0-9A0F-E686F0AFA67F@icir.org>
	<6165d3b4ddbc483996206c2dfc2f17ad@exch13-mail04.win.slac.stanford.edu>
	<DEC916FE-77F1-4035-919C-3E36FF78346A@pingtrip.com>
	<E0011940-43FC-427E-9F4A-0FAFA99D21CB@pingtrip.com>
	<3B5C35C1-76E1-415D-9B27-16A738DEBCE3@icir.org>
	<ECCADAC6-F02E-4645-91E9-40BE5FB99966@pingtrip.com>
Message-ID: <E5A7D7AC-7C7D-4877-9E24-078C79467EC5@icir.org>


> On Feb 14, 2017, at 1:10 PM, Dave Crawford <bro at pingtrip.com> wrote:
> 
>      rx_missed_errors: 193958

This number isn't really that high compared to how many packets came into the nic, but you may want to play with pinning the lb process to a core and changing it's nice level.  That might improve how many packets are lost on the nic.

lb shouldn't be misreporting the number of packets that it is accepting and forwarding along to the netmap pipes.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From gary.w.weasel2.civ at mail.mil  Fri Feb 17 09:34:39 2017
From: gary.w.weasel2.civ at mail.mil (Weasel, Gary W Jr CIV DISA RE (US))
Date: Fri, 17 Feb 2017 17:34:39 +0000
Subject: [Bro] Bro Script not working
Message-ID: <0C34D9CA9B9DBB45B1C51871C177B4B285DA676D@UMECHPA68.easf.csd.disa.mil>

Greetings all,

I've written a custom Bro script that supposed to output a new log, but for some reason it simply refuses to create new output log.  I developed and wrote my script using http://try.bro.org, and it works beautifully there.  But for some reason when I move it over to an actual Bro instance, it just doesn't produce anything.  I get no errors, no warnings, and no outputs, and I'm certain the script is getting loaded.

Can anyone provide any input into any pitfalls I may be hitting here?

v/r
Gary



From pyrodie18 at gmail.com  Fri Feb 17 12:40:17 2017
From: pyrodie18 at gmail.com (Troy Ward)
Date: Fri, 17 Feb 2017 12:40:17 -0800
Subject: [Bro] Bro Script not working
Message-ID: <CAKjK3aSoLTVjY4f3EANbqHD3=So-CG3jaed3vCmFNu=2Z-1V0A@mail.gmail.com>

>
> When you do broctl, have you deployed it?  Any errors?  After that still
> within broctl give the "scripts" command and make sure that your script is
> being loaded.
>

--Troy



> Message: 5
> Date: Fri, 17 Feb 2017 17:34:39 +0000
> From: "Weasel, Gary W Jr CIV DISA RE (US)"
>         <gary.w.weasel2.civ at mail.mil>
> Subject: [Bro] Bro Script not working
> To: "'bro at bro.org'" <bro at bro.org>
> Message-ID:
>         <0C34D9CA9B9DBB45B1C51871C177B4B285DA676D at UMECHPA68.easf.
> csd.disa.mil>
> Content-Type: text/plain; charset="us-ascii"
>
> Greetings all,
>
> I've written a custom Bro script that supposed to output a new log, but
> for some reason it simply refuses to create new output log.  I developed
> and wrote my script using http://try.bro.org, and it works beautifully
> there.  But for some reason when I move it over to an actual Bro instance,
> it just doesn't produce anything.  I get no errors, no warnings, and no
> outputs, and I'm certain the script is getting loaded.
>
> Can anyone provide any input into any pitfalls I may be hitting here?
>
> v/r
> Gary
>
>
>
>
> ------------------------------
>
> _______________________________________________
> Bro mailing list
> Bro at bro.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
> End of Bro Digest, Vol 130, Issue 32
> ************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170217/9884d5dd/attachment.html 

From gfaulkner.nsm at gmail.com  Fri Feb 17 12:54:47 2017
From: gfaulkner.nsm at gmail.com (Gary Faulkner)
Date: Fri, 17 Feb 2017 14:54:47 -0600
Subject: [Bro] af_packet comparison to PF_RING ZC/DNA for Bro (in light of
 recent Suricata tuning paper)
Message-ID: <ebcb9ee8-6976-b8dc-a5b9-769edcc02165@gmail.com>

After reading over the paper Michal and others worked on concerning 
tuning Suricata for best performance with AF_Packet I'm wondering how 
af_packet performance compares to pf_ring DNA/ZC (with the commercially 
licensed drivers, not just vanilla) especially when it comes to Bro.

Is af_packet generally sufficient for Bro when it comes to monitoring 
100G+ networks using a cluster of commodity servers with Intel X520 NICs?

Is the distro shipped driver for something like an up-to-date Ubuntu 
16.04 (4.4 kernel) server sufficient or do you really need to compile 
the driver from source to enable some extended features, or to get a 
properly patched driver etc? I could see some benefits to just using the 
distro packaged driver and not having to compile the driver from scratch 
or rely on dkms when patching sensors. I've had this go very wrong a few 
times.

Are there any gotchas where running one or the other might be the better 
way to go? Examples (want to use some bro feature such as capstats, or 
want to see VLAN tags in Bro logs, something else is broken or not 
performing as expected)

Does af_packet or the Bro plugin for it have a way to deal with multiple 
NICS (one per numa node), sort of like how pf_ring has dnacluster and 
zbalance_ipc?

Feel free to share any other relevant considerations. I'm especially 
interested in things such as ease of management, performance, 
compatibility etc.

~Gary

From jan.grashoefer at gmail.com  Sat Feb 18 02:11:39 2017
From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=)
Date: Sat, 18 Feb 2017 11:11:39 +0100
Subject: [Bro] af_packet comparison to PF_RING ZC/DNA for Bro (in light
 of recent Suricata tuning paper)
In-Reply-To: <ebcb9ee8-6976-b8dc-a5b9-769edcc02165@gmail.com>
References: <ebcb9ee8-6976-b8dc-a5b9-769edcc02165@gmail.com>
Message-ID: <dad40440-ffa6-bede-4863-8ac9f78951ba@gmail.com>

Hi Gary,

> After reading over the paper Michal and others worked on concerning 
> tuning Suricata for best performance with AF_Packet I'm wondering how 
> af_packet performance compares to pf_ring DNA/ZC (with the commercially 
> licensed drivers, not just vanilla) especially when it comes to Bro.

unfortunately I cannot provide any numbers. My main motivation for using
AF_Packet with Bro was the ease of use. Especially the PF_RING ZC
drivers caused issues in my environment, which I struggled to debug.
Given the extra cost of building this and that myself I chose AF_Packet.

> Is af_packet generally sufficient for Bro when it comes to monitoring 
> 100G+ networks using a cluster of commodity servers with Intel X520 NICs?

Good question. Someone should test this :)

> Is the distro shipped driver for something like an up-to-date Ubuntu 
> 16.04 (4.4 kernel) server sufficient or do you really need to compile 
> the driver from source to enable some extended features, or to get a 
> properly patched driver etc? I could see some benefits to just using the 
> distro packaged driver and not having to compile the driver from scratch 
> or rely on dkms when patching sensors. I've had this go very wrong a few 
> times.

For me (CentOS 7) the packaged driver worked well with AF_Packet. But if
you want to tune things for maximal performance, I would recommend using
the latest drivers. E.g., from time to time looking at the code might
help in this case to understand what's going on.

> Are there any gotchas where running one or the other might be the better 
> way to go? Examples (want to use some bro feature such as capstats, or 
> want to see VLAN tags in Bro logs, something else is broken or not 
> performing as expected)

I haven't used capstats but if I remember correctly, it is kind of
deprecated as it relies on libpcap. One should be able to obtain the
same information from other sources.

VLAN tags are indeed an issue using AF_Packet. For consistency reasons,
the kernel extracts VLAN tags even if there is no hardware VLAN
offloading (in contrast to Bro, Suricata can handle this due to its
monolithic structure). Actually that's something on my list.

Finally, one has to be careful regarding the kernel used. There is a bug
concerning AF-Packet's symmetric hashing that has been fixed in recent
kernels
(https://bro-tracker.atlassian.net/browse/BIT-1575?focusedCommentId=29627#comment-29627).

> Does af_packet or the Bro plugin for it have a way to deal with multiple 
> NICS (one per numa node), sort of like how pf_ring has dnacluster and 
> zbalance_ipc?

In theory configuring a set of workers per NUMA node using separate NICs
shouldn't be an issue. The only thing is that you won't get load
balancing across the NICs. I am not sure how well this works with
PF_RING, though.

> Feel free to share any other relevant considerations.

In addition to the VLAN stuff I have a couple of other things on my
list, which might allow some tuning. Unfortunately this list hasn't seen
much progress lately as I don't have access to a test setup. So there
might be room for improvement.

Jan

From obdnanr at gmail.com  Sat Feb 18 04:57:33 2017
From: obdnanr at gmail.com (Obndnar smith)
Date: Sat, 18 Feb 2017 12:57:33 +0000
Subject: [Bro] Passive DNS IOC hunting script
Message-ID: <CALCRDTTBBOPBp9479V+csW2RePHxguw3bj4vUsetwmh+sFhaSw@mail.gmail.com>

I've created a script that uses Justin Azoff's bro-pdns-go-rewrite script
to search the passive DNS database for IOC hits from a text file hosted on
a webserver; we're using CRITS.  You can cron both scripts, but I can't
figure out how to get it to send one email alert per run of the script, so
don't set it to every 5 minutes.  You may need to touch some of the csvs if
it complains they aren't there.  You'll need to enter the full path name on
the sortuniqe.sh script also.

I can't find Justin's Github for the go-rewrite, so maybe he can chime in
with those details.

https://github.com/obdnanr/bro-pdns-ioc-search-alert

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170218/e3d9e8f9/attachment.html 

From jazoff at illinois.edu  Sat Feb 18 07:09:09 2017
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Sat, 18 Feb 2017 15:09:09 +0000
Subject: [Bro] Passive DNS IOC hunting script
In-Reply-To: <CALCRDTTBBOPBp9479V+csW2RePHxguw3bj4vUsetwmh+sFhaSw@mail.gmail.com>
References: <CALCRDTTBBOPBp9479V+csW2RePHxguw3bj4vUsetwmh+sFhaSw@mail.gmail.com>
Message-ID: <A88BCAC3-B9F9-4B4E-B7B8-85625B6DDA12@illinois.edu>


> On Feb 18, 2017, at 7:57 AM, Obndnar smith <obdnanr at gmail.com> wrote:
> 
> I can't find Justin's Github for the go-rewrite, so maybe he can chime in with those details. 
> 

It's in a branch:

https://github.com/JustinAzoff/bro-pdns/tree/go-rewrite

-- 
- Justin Azoff



From carlopmart at gmail.com  Mon Feb 20 01:25:18 2017
From: carlopmart at gmail.com (C. L. Martinez)
Date: Mon, 20 Feb 2017 09:25:18 +0000
Subject: [Bro] Splunk or ELK to parse Bro logs
Message-ID: <20170220092518.metxkrmiezhaizo4@scotland.uxdom.org>

Hi all,

 I would like to do some tests and deploy rules using Bro under my laptop test lab. Due to limited resources, I would like to install some log parser tool for Bro like Splunk or ELK.

 In the past, I have used Splunk and goes very well for my needs. But doing some searches, I am finding more docs about using ELK with Bro than using Splunk. 

 But I don't see how can I limit resources assigned to an ELK infrastructure to suit my needs (I can't assign more than 2.5 GB of RAM to vm where I will install splunk or elk or another solution).

 I don't expect a lot of logs per day or hour from Bro's side (in fact, I expect very few), but i don't see pretty clear what solution to use.

 What are your opinions or recommendations?

Many thanks to all.

-- 
Greetings,
C. L. Martinez

From blackhole.em at gmail.com  Mon Feb 20 05:43:59 2017
From: blackhole.em at gmail.com (Joe Blow)
Date: Mon, 20 Feb 2017 08:43:59 -0500
Subject: [Bro] Splunk or ELK to parse Bro logs
In-Reply-To: <20170220092518.metxkrmiezhaizo4@scotland.uxdom.org>
References: <20170220092518.metxkrmiezhaizo4@scotland.uxdom.org>
Message-ID: <CAMCSCoXap63XbY3Ojdz74m5Hzf_eQfReFAxyY69v4SZR7ov61g@mail.gmail.com>

You could just change the JVM you're using elasticsearch/logstash on to
only allocate 1GB of RAM.  On that VM if you give it 2.5GB of RAM, then
only 1GB of it will be used by your Elasticsearch install.  The rest will
be used by the OS (disk cache) and logstash.

In CentOS land, you'd make your /etc/sysconfig/elasticsearch file say this:

ES_HEAP_SIZE=1g

Cheers,

JB

On Mon, Feb 20, 2017 at 4:25 AM, C. L. Martinez <carlopmart at gmail.com>
wrote:

> Hi all,
>
>  I would like to do some tests and deploy rules using Bro under my laptop
> test lab. Due to limited resources, I would like to install some log parser
> tool for Bro like Splunk or ELK.
>
>  In the past, I have used Splunk and goes very well for my needs. But
> doing some searches, I am finding more docs about using ELK with Bro than
> using Splunk.
>
>  But I don't see how can I limit resources assigned to an ELK
> infrastructure to suit my needs (I can't assign more than 2.5 GB of RAM to
> vm where I will install splunk or elk or another solution).
>
>  I don't expect a lot of logs per day or hour from Bro's side (in fact, I
> expect very few), but i don't see pretty clear what solution to use.
>
>  What are your opinions or recommendations?
>
> Many thanks to all.
>
> --
> Greetings,
> C. L. Martinez
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170220/9fe96c26/attachment.html 

From jwc3f at virginia.edu  Mon Feb 20 07:50:57 2017
From: jwc3f at virginia.edu (Collyer, Jeffrey W. (jwc3f))
Date: Mon, 20 Feb 2017 15:50:57 +0000
Subject: [Bro] Splunk or ELK to parse Bro logs
In-Reply-To: <20170220092518.metxkrmiezhaizo4@scotland.uxdom.org>
References: <20170220092518.metxkrmiezhaizo4@scotland.uxdom.org>
Message-ID: <AC784053-0CAC-4175-8800-67A4F684DCBA@virginia.edu>

For Splunk I?ve had good luck with logging to JSON format and using this TA

https://github.com/jahshuah/splunk-ta-bro-json <https://github.com/jahshuah/splunk-ta-bro-json>




Jeffrey Collyer



> On Feb 20, 2017, at 4:41 AM, C. L. Martinez <carlopmart at gmail.com> wrote:
> 
> Hi all,
> 
> I would like to do some tests and deploy rules using Bro under my laptop test lab. Due to limited resources, I would like to install some log parser tool for Bro like Splunk or ELK.
> 
> In the past, I have used Splunk and goes very well for my needs. But doing some searches, I am finding more docs about using ELK with Bro than using Splunk. 
> 
> But I don't see how can I limit resources assigned to an ELK infrastructure to suit my needs (I can't assign more than 2.5 GB of RAM to vm where I will install splunk or elk or another solution).
> 
> I don't expect a lot of logs per day or hour from Bro's side (in fact, I expect very few), but i don't see pretty clear what solution to use.
> 
> What are your opinions or recommendations?
> 
> Many thanks to all.
> 
> -- 
> Greetings,
> C. L. Martinez
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170220/d54d8db5/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4939 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170220/d54d8db5/attachment.bin 

From ahmadjd94 at gmail.com  Mon Feb 20 09:24:06 2017
From: ahmadjd94 at gmail.com (ahmad dana)
Date: Mon, 20 Feb 2017 19:24:06 +0200
Subject: [Bro] Python visualizing tool (BILA)
Message-ID: <CAKMhW2vJ7VrOeWXRpAhDC94Sk83m=MKRSmhbOYcPDEF52Xjsbg@mail.gmail.com>

I have been working on python3 tool that makes use of the the generated
logs out of BRO, it uses plotly-library to make interactive plots and chart
that would ease the process of analyzing the alerts.
it would be a pleasure if anyone is interested in further development of
the code, or even suggest features to make the program more usable.

you can check the code on GitHub repositiory :
https://github.com/ahmadjd94/Bro-ID-Log-Analyzer

Thank you
-- 
Ahmad Da'na
Linkedin profile <https://www.linkedin.com/in/ahmadjd>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170220/ed3d4ba9/attachment.html 

From fatema.bannatwala at gmail.com  Tue Feb 21 05:40:50 2017
From: fatema.bannatwala at gmail.com (fatema bannatwala)
Date: Tue, 21 Feb 2017 08:40:50 -0500
Subject: [Bro] Any thoughts on "Microsoft-CryptoAPI/10.0" user-agent?
Message-ID: <CACX0rUT4FRHSBo96fxzHK=d6ThDGymHKs3_2WEU6sz8=QreKAw@mail.gmail.com>

Hello,

I am trying to figure out what Windows operating system version have
the user agent "Microsoft-CryptoAPI/10.0" when it accesses Microsoft
Certificate Revocation List (CRL).

I am seeing good amount of these in software.log, where it ends up being
"Unknown CryptoAPI Version" as the windows-version-detection.bro script
doesn't have a mapping for that CryptoAPI.

Therefore was thinking if anyone knows more about this user agent and what
information we can
infer about the OS from it.

Appreciate the help.

Thanks,
Fatema.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170221/a7c24509/attachment.html 

From seth at icir.org  Tue Feb 21 06:26:27 2017
From: seth at icir.org (Seth Hall)
Date: Tue, 21 Feb 2017 09:26:27 -0500
Subject: [Bro] Any thoughts on "Microsoft-CryptoAPI/10.0" user-agent?
In-Reply-To: <CACX0rUT4FRHSBo96fxzHK=d6ThDGymHKs3_2WEU6sz8=QreKAw@mail.gmail.com>
References: <CACX0rUT4FRHSBo96fxzHK=d6ThDGymHKs3_2WEU6sz8=QreKAw@mail.gmail.com>
Message-ID: <10FE168F-C9A8-48B2-8A9F-FBBF0726F135@icir.org>


> On Feb 21, 2017, at 8:40 AM, fatema bannatwala <fatema.bannatwala at gmail.com> wrote:
> 
> I am trying to figure out what Windows operating system version have
> the user agent "Microsoft-CryptoAPI/10.0" when it accesses Microsoft Certificate Revocation List (CRL).
> 
> I am seeing good amount of these in software.log, where it ends up being "Unknown CryptoAPI Version" as the windows-version-detection.bro script doesn't have a mapping for that CryptoAPI.

I suspect this is Windows 10.  Can someone out there validate that suspicion so we can add that to the windows version detection script?

  .Set

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From klehigh at iu.edu  Tue Feb 21 06:40:40 2017
From: klehigh at iu.edu (Keith Lehigh)
Date: Tue, 21 Feb 2017 09:40:40 -0500
Subject: [Bro] Any thoughts on "Microsoft-CryptoAPI/10.0" user-agent?
In-Reply-To: <10FE168F-C9A8-48B2-8A9F-FBBF0726F135@icir.org>
References: <CACX0rUT4FRHSBo96fxzHK=d6ThDGymHKs3_2WEU6sz8=QreKAw@mail.gmail.com>
	<10FE168F-C9A8-48B2-8A9F-FBBF0726F135@icir.org>
Message-ID: <DE3435A3-E2EF-4B77-AB6D-D05768B79CE2@iu.edu>

Confirmed with a virtual machine I have running Windows 10.

- Keith

> On Feb 21, 2017, at 09:26, Seth Hall <seth at icir.org> wrote:
> 
> 
>> On Feb 21, 2017, at 8:40 AM, fatema bannatwala <fatema.bannatwala at gmail.com> wrote:
>> 
>> I am trying to figure out what Windows operating system version have
>> the user agent "Microsoft-CryptoAPI/10.0" when it accesses Microsoft Certificate Revocation List (CRL).
>> 
>> I am seeing good amount of these in software.log, where it ends up being "Unknown CryptoAPI Version" as the windows-version-detection.bro script doesn't have a mapping for that CryptoAPI.
> 
> I suspect this is Windows 10.  Can someone out there validate that suspicion so we can add that to the windows version detection script?
> 
>  .Set
> 
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3569 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170221/b7fe7f02/attachment.bin 

From mike.patterson at uwaterloo.ca  Tue Feb 21 06:41:37 2017
From: mike.patterson at uwaterloo.ca (Mike Patterson)
Date: Tue, 21 Feb 2017 14:41:37 +0000
Subject: [Bro] Any thoughts on "Microsoft-CryptoAPI/10.0" user-agent?
In-Reply-To: <10FE168F-C9A8-48B2-8A9F-FBBF0726F135@icir.org>
References: <CACX0rUT4FRHSBo96fxzHK=d6ThDGymHKs3_2WEU6sz8=QreKAw@mail.gmail.com>
	<10FE168F-C9A8-48B2-8A9F-FBBF0726F135@icir.org>
Message-ID: <E8D4A4CE-D38E-4F72-AA74-51E5AED98AF8@uwaterloo.ca>


> On Feb 21, 2017, at 09:26, Seth Hall <seth at icir.org> wrote:
> 
> 
>> On Feb 21, 2017, at 8:40 AM, fatema bannatwala <fatema.bannatwala at gmail.com> wrote:
>> 
>> I am trying to figure out what Windows operating system version have
>> the user agent "Microsoft-CryptoAPI/10.0" when it accesses Microsoft Certificate Revocation List (CRL).
>> 
>> I am seeing good amount of these in software.log, where it ends up being "Unknown CryptoAPI Version" as the windows-version-detection.bro script doesn't have a mapping for that CryptoAPI.
> 
> I suspect this is Windows 10.  Can someone out there validate that suspicion so we can add that to the windows version detection script?

I have Win10 down for CryptoAPI 6.4, along with Server 2016, but my notes there are pretty old (like, Win10 GA timeframe).

I'm now seeing CryptoAPI 10.0 as well, confirmed on several hosts as being Win10. Maybe there's a difference between editions? I can't easily find out what versions ours are.


>  .Set

+h? ;)

Mike



From fatema.bannatwala at gmail.com  Tue Feb 21 07:01:55 2017
From: fatema.bannatwala at gmail.com (fatema bannatwala)
Date: Tue, 21 Feb 2017 10:01:55 -0500
Subject: [Bro] Any thoughts on "Microsoft-CryptoAPI/10.0" user-agent?
In-Reply-To: <E8D4A4CE-D38E-4F72-AA74-51E5AED98AF8@uwaterloo.ca>
References: <CACX0rUT4FRHSBo96fxzHK=d6ThDGymHKs3_2WEU6sz8=QreKAw@mail.gmail.com>
	<10FE168F-C9A8-48B2-8A9F-FBBF0726F135@icir.org>
	<E8D4A4CE-D38E-4F72-AA74-51E5AED98AF8@uwaterloo.ca>
Message-ID: <CACX0rUThMpmwbRf4dWkVeA=if=2CPDrzU1q6=2URx7v7t4C3dA@mail.gmail.com>

Thanks Seth, Mike and Keith for the confirmation, will update the script to
log it as win10 system! :)

Regards,
Fatema.

On Tue, Feb 21, 2017 at 9:41 AM, Mike Patterson <mike.patterson at uwaterloo.ca
> wrote:

>
> > On Feb 21, 2017, at 09:26, Seth Hall <seth at icir.org> wrote:
> >
> >
> >> On Feb 21, 2017, at 8:40 AM, fatema bannatwala <
> fatema.bannatwala at gmail.com> wrote:
> >>
> >> I am trying to figure out what Windows operating system version have
> >> the user agent "Microsoft-CryptoAPI/10.0" when it accesses Microsoft
> Certificate Revocation List (CRL).
> >>
> >> I am seeing good amount of these in software.log, where it ends up
> being "Unknown CryptoAPI Version" as the windows-version-detection.bro
> script doesn't have a mapping for that CryptoAPI.
> >
> > I suspect this is Windows 10.  Can someone out there validate that
> suspicion so we can add that to the windows version detection script?
>
> I have Win10 down for CryptoAPI 6.4, along with Server 2016, but my notes
> there are pretty old (like, Win10 GA timeframe).
>
> I'm now seeing CryptoAPI 10.0 as well, confirmed on several hosts as being
> Win10. Maybe there's a difference between editions? I can't easily find out
> what versions ours are.
>
>
> >  .Set
>
> +h? ;)
>
> Mike
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170221/f9fadb54/attachment.html 

From fatema.bannatwala at gmail.com  Tue Feb 21 09:50:32 2017
From: fatema.bannatwala at gmail.com (fatema bannatwala)
Date: Tue, 21 Feb 2017 12:50:32 -0500
Subject: [Bro] software/version-changes.bro comparison between the two
	versions.
Message-ID: <CACX0rUTULRQhi3WG1fUnHk6V22wnz4TsPGAeCWnJ7+Jk2sJPTQ@mail.gmail.com>

I was going through the version-changes.bro script, thinking of adding some
software
to track the version changes, but realized that there is no comparison done
between the
old version tracked and the version detected in "rec: Info" of log_software
event.

Hence, was thinking to add a condition to check it before the notice is
raised for the version
change, like following:
( or I might be missing something regarding the functionality of the
script. :/)

event log_software(rec: Info)
{
local ts = tracked[rec$host];

if ( rec$name in ts )
{
local old = ts[rec$name];

# Is it a potentially interesting version change?
if ( rec$name in interesting_version_changes )
{

  *if (software_fmt_version(old$version) !=
software_fmt_version(rec$version))*
             { local msg = fmt("%.6f %s switched from %s to %s (%s)",
network_time(), rec$software_type,
software_fmt_version(old$version),
software_fmt(rec), rec$software_type);
       NOTICE([$note=Software_Version_Change, $src=rec$host,
       $msg=msg, $sub=software_fmt(rec)]);
             }
}
}
}

Any thoughts? anybody using this script to track software changes?

Thanks,
Fatema.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170221/24545fb3/attachment-0001.html 

From jazoff at illinois.edu  Tue Feb 21 10:55:48 2017
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Tue, 21 Feb 2017 18:55:48 +0000
Subject: [Bro] software/version-changes.bro comparison between the
	two	versions.
In-Reply-To: <CACX0rUTULRQhi3WG1fUnHk6V22wnz4TsPGAeCWnJ7+Jk2sJPTQ@mail.gmail.com>
References: <CACX0rUTULRQhi3WG1fUnHk6V22wnz4TsPGAeCWnJ7+Jk2sJPTQ@mail.gmail.com>
Message-ID: <BB07A34F-9EBE-45F6-8599-C6BB4759E180@illinois.edu>

It looks like that script is broken :-(  The main software script that logs new software versions does:

    ts[info$name] = info;
    Log::write(Software::LOG, info);

and then the version changes script is doing

    local old = ts[rec$name]

But at that point old and rec are the same exact thing.  It's possible to fix this, it just can't use the log_software event because at that point the "old" version has already been overwritten.

Another issue with the script is that the 'tracked' variable has a create expire of only 24h, so if the host is only seen every 48 hours, or if bro is restarted it won't know the version changed.

Newer features in Broker should allow interesting version changes to be tracked using persistent data stores.  That would really fix the issue.  There are similar things that need to be re-written for better tracking known hosts/known services/known certs.

I added this info to the existing ticket I had for this:

https://bro-tracker.atlassian.net/browse/BIT-1521



-- 
- Justin Azoff

> On Feb 21, 2017, at 12:50 PM, fatema bannatwala <fatema.bannatwala at gmail.com> wrote:
> 
> I was going through the version-changes.bro script, thinking of adding some software
> to track the version changes, but realized that there is no comparison done between the
> old version tracked and the version detected in "rec: Info" of log_software event.
> 
> Hence, was thinking to add a condition to check it before the notice is raised for the version
> change, like following:
> ( or I might be missing something regarding the functionality of the script. :/)
> 
> event log_software(rec: Info)
> 	{
> 	local ts = tracked[rec$host];
> 	
> 	if ( rec$name in ts )
> 		{
> 		local old = ts[rec$name];
> 	
> 		# Is it a potentially interesting version change?
> 		if ( rec$name in interesting_version_changes )
> 			{  
> 			   
> 			   if (software_fmt_version(old$version) != software_fmt_version(rec$version))
> 		             {	local msg = fmt("%.6f %s switched from %s to %s (%s)",
> 					network_time(), rec$software_type,
> 					software_fmt_version(old$version),
> 					software_fmt(rec), rec$software_type);
> 		    	    NOTICE([$note=Software_Version_Change, $src=rec$host,
> 			        $msg=msg, $sub=software_fmt(rec)]);
> 		             }
> 			}
> 		}
> 	}
> 
> Any thoughts? anybody using this script to track software changes?
> 
> Thanks,
> Fatema.
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From fatema.bannatwala at gmail.com  Tue Feb 21 11:32:52 2017
From: fatema.bannatwala at gmail.com (fatema bannatwala)
Date: Tue, 21 Feb 2017 14:32:52 -0500
Subject: [Bro] software/version-changes.bro comparison between the two
	versions.
In-Reply-To: <BB07A34F-9EBE-45F6-8599-C6BB4759E180@illinois.edu>
References: <CACX0rUTULRQhi3WG1fUnHk6V22wnz4TsPGAeCWnJ7+Jk2sJPTQ@mail.gmail.com>
	<BB07A34F-9EBE-45F6-8599-C6BB4759E180@illinois.edu>
Message-ID: <CACX0rUSvuETL6-FdpB0skGS6zT=5Uihu4FoRaXdyOkS9yOq+1g@mail.gmail.com>

Thanks Justin for an update, will be awaiting the Broker integration for
the new features/scripts.

On Tue, Feb 21, 2017 at 1:55 PM, Azoff, Justin S <jazoff at illinois.edu>
wrote:

> It looks like that script is broken :-(  The main software script that
> logs new software versions does:
>
>     ts[info$name] = info;
>     Log::write(Software::LOG, info);
>
> and then the version changes script is doing
>
>     local old = ts[rec$name]
>
> But at that point old and rec are the same exact thing.  It's possible to
> fix this, it just can't use the log_software event because at that point
> the "old" version has already been overwritten.
>
> Another issue with the script is that the 'tracked' variable has a create
> expire of only 24h, so if the host is only seen every 48 hours, or if bro
> is restarted it won't know the version changed.
>
> Newer features in Broker should allow interesting version changes to be
> tracked using persistent data stores.  That would really fix the issue.
> There are similar things that need to be re-written for better tracking
> known hosts/known services/known certs.
>
> I added this info to the existing ticket I had for this:
>
> https://bro-tracker.atlassian.net/browse/BIT-1521
>
>
>
> --
> - Justin Azoff
>
> > On Feb 21, 2017, at 12:50 PM, fatema bannatwala <
> fatema.bannatwala at gmail.com> wrote:
> >
> > I was going through the version-changes.bro script, thinking of adding
> some software
> > to track the version changes, but realized that there is no comparison
> done between the
> > old version tracked and the version detected in "rec: Info" of
> log_software event.
> >
> > Hence, was thinking to add a condition to check it before the notice is
> raised for the version
> > change, like following:
> > ( or I might be missing something regarding the functionality of the
> script. :/)
> >
> > event log_software(rec: Info)
> >       {
> >       local ts = tracked[rec$host];
> >
> >       if ( rec$name in ts )
> >               {
> >               local old = ts[rec$name];
> >
> >               # Is it a potentially interesting version change?
> >               if ( rec$name in interesting_version_changes )
> >                       {
> >
> >                          if (software_fmt_version(old$version) !=
> software_fmt_version(rec$version))
> >                            {  local msg = fmt("%.6f %s switched from %s
> to %s (%s)",
> >                                       network_time(), rec$software_type,
> >                                       software_fmt_version(old$version),
> >                                       software_fmt(rec),
> rec$software_type);
> >                           NOTICE([$note=Software_Version_Change,
> $src=rec$host,
> >                               $msg=msg, $sub=software_fmt(rec)]);
> >                            }
> >                       }
> >               }
> >       }
> >
> > Any thoughts? anybody using this script to track software changes?
> >
> > Thanks,
> > Fatema.
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170221/8919813c/attachment.html 

From seth at icir.org  Tue Feb 21 12:07:24 2017
From: seth at icir.org (Seth Hall)
Date: Tue, 21 Feb 2017 15:07:24 -0500
Subject: [Bro] Any thoughts on "Microsoft-CryptoAPI/10.0" user-agent?
In-Reply-To: <CACX0rUThMpmwbRf4dWkVeA=if=2CPDrzU1q6=2URx7v7t4C3dA@mail.gmail.com>
References: <CACX0rUT4FRHSBo96fxzHK=d6ThDGymHKs3_2WEU6sz8=QreKAw@mail.gmail.com>
	<10FE168F-C9A8-48B2-8A9F-FBBF0726F135@icir.org>
	<E8D4A4CE-D38E-4F72-AA74-51E5AED98AF8@uwaterloo.ca>
	<CACX0rUThMpmwbRf4dWkVeA=if=2CPDrzU1q6=2URx7v7t4C3dA@mail.gmail.com>
Message-ID: <E2881396-7C56-475B-9B18-85F05FF84B77@icir.org>


> On Feb 21, 2017, at 10:01 AM, fatema bannatwala <fatema.bannatwala at gmail.com> wrote:
> 
> Thanks Seth, Mike and Keith for the confirmation, will update the script to log it as win10 system! :)

Thanks everyone.  It's in master now.
	https://github.com/bro/bro/commit/205a28bad8714a19b37080f069034868ee6dda9e

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From pssunu6 at gmail.com  Tue Feb 21 17:12:48 2017
From: pssunu6 at gmail.com (ps sunu)
Date: Wed, 22 Feb 2017 06:42:48 +0530
Subject: [Bro] Detect tor
Message-ID: <CALGAZaAc+6f30WyxBhqvOpHVVNdj4EcrEewAPjsrBasVUwWmOw@mail.gmail.com>

Hi,
                Which is the best TOR detection script in bro ? below on is
good , or any other script there

https://raw.githubusercontent.com/sethhall/bro-junk-drawer/master/detect-tor.bro


Regards,
Sunu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170222/ead53616/attachment.html 

From carlopmart at gmail.com  Wed Feb 22 00:32:11 2017
From: carlopmart at gmail.com (C. L. Martinez)
Date: Wed, 22 Feb 2017 08:32:11 +0000
Subject: [Bro] Splunk or ELK to parse Bro logs
In-Reply-To: <CAMCSCoXap63XbY3Ojdz74m5Hzf_eQfReFAxyY69v4SZR7ov61g@mail.gmail.com>
References: <20170220092518.metxkrmiezhaizo4@scotland.uxdom.org>
	<CAMCSCoXap63XbY3Ojdz74m5Hzf_eQfReFAxyY69v4SZR7ov61g@mail.gmail.com>
Message-ID: <20170222083211.3tgtotzfwhvcihlv@scotland.uxdom.org>

On Mon, Feb 20, 2017 at 08:43:59AM -0500, Joe Blow wrote:
> You could just change the JVM you're using elasticsearch/logstash on to
> only allocate 1GB of RAM.  On that VM if you give it 2.5GB of RAM, then
> only 1GB of it will be used by your Elasticsearch install.  The rest will
> be used by the OS (disk cache) and logstash.
> 
> In CentOS land, you'd make your /etc/sysconfig/elasticsearch file say this:
> 
> ES_HEAP_SIZE=1g
> 
> Cheers,
> 
> JB
> 

Many thanks to all for your inputs. Regarding to use ELK, is it safe to use latest versions of Logstash, Elasticsearch adn Kibana? What version do you recommend?

-- 
Greetings,
C. L. Martinez

From pssunu6 at gmail.com  Wed Feb 22 01:50:27 2017
From: pssunu6 at gmail.com (ps sunu)
Date: Wed, 22 Feb 2017 15:20:27 +0530
Subject: [Bro] Fwd: Detect tor
In-Reply-To: <CALGAZaAc+6f30WyxBhqvOpHVVNdj4EcrEewAPjsrBasVUwWmOw@mail.gmail.com>
References: <CALGAZaAc+6f30WyxBhqvOpHVVNdj4EcrEewAPjsrBasVUwWmOw@mail.gmail.com>
Message-ID: <CALGAZaCTH-LX0qSmShOtj14xwFK3pGzerK57z4zOWz0ELUVLEw@mail.gmail.com>

-----
Hi,
                Which is the best TOR detection script in bro ? below one
is good , or any other script there ?

https://raw.githubusercontent.com/sethhall/bro-junk-drawer/
master/detect-tor.bro


Regards,
Sunu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170222/6eb430dd/attachment.html 

From fatema.bannatwala at gmail.com  Wed Feb 22 05:21:32 2017
From: fatema.bannatwala at gmail.com (fatema bannatwala)
Date: Wed, 22 Feb 2017 08:21:32 -0500
Subject: [Bro] Detect tor
In-Reply-To: <CALGAZaCTH-LX0qSmShOtj14xwFK3pGzerK57z4zOWz0ELUVLEw@mail.gmail.com>
References: <CALGAZaAc+6f30WyxBhqvOpHVVNdj4EcrEewAPjsrBasVUwWmOw@mail.gmail.com>
	<CALGAZaCTH-LX0qSmShOtj14xwFK3pGzerK57z4zOWz0ELUVLEw@mail.gmail.com>
Message-ID: <CACX0rURxCpBadbFcCYkid3U4j5j45AVGygBizZ-p53cwnOahZQ@mail.gmail.com>

Another thing you could try is, if you use intel framework, then you can
feed the intel FW with
the IOCs data for TOR, and load it in Intel, so that you will get logs in
intel.log, whenever there's
a hit on TOR IPs in your network traffic.

Thanks,
Fatema.

On Wed, Feb 22, 2017 at 4:50 AM, ps sunu <pssunu6 at gmail.com> wrote:

>
> -----
> Hi,
>                 Which is the best TOR detection script in bro ? below one
> is good , or any other script there ?
>
> https://raw.githubusercontent.com/sethhall/bro-junk-drawer/m
> aster/detect-tor.bro
>
>
> Regards,
> Sunu
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170222/d33a5a6f/attachment.html 

From pssunu6 at gmail.com  Wed Feb 22 07:26:36 2017
From: pssunu6 at gmail.com (ps sunu)
Date: Wed, 22 Feb 2017 20:56:36 +0530
Subject: [Bro] Detect tor
In-Reply-To: <CACX0rURxCpBadbFcCYkid3U4j5j45AVGygBizZ-p53cwnOahZQ@mail.gmail.com>
References: <CALGAZaAc+6f30WyxBhqvOpHVVNdj4EcrEewAPjsrBasVUwWmOw@mail.gmail.com>
	<CALGAZaCTH-LX0qSmShOtj14xwFK3pGzerK57z4zOWz0ELUVLEw@mail.gmail.com>
	<CACX0rURxCpBadbFcCYkid3U4j5j45AVGygBizZ-p53cwnOahZQ@mail.gmail.com>
Message-ID: <CALGAZaDaiNTZeV==ZKKC5GJ=wZnAC1wYZGYyQms=SL421=_HTA@mail.gmail.com>

ok thanks for your info

On Wed, Feb 22, 2017 at 6:51 PM, fatema bannatwala <
fatema.bannatwala at gmail.com> wrote:

> Another thing you could try is, if you use intel framework, then you can
> feed the intel FW with
> the IOCs data for TOR, and load it in Intel, so that you will get logs in
> intel.log, whenever there's
> a hit on TOR IPs in your network traffic.
>
> Thanks,
> Fatema.
>
> On Wed, Feb 22, 2017 at 4:50 AM, ps sunu <pssunu6 at gmail.com> wrote:
>
>>
>> -----
>> Hi,
>>                 Which is the best TOR detection script in bro ? below one
>> is good , or any other script there ?
>>
>> https://raw.githubusercontent.com/sethhall/bro-junk-drawer/m
>> aster/detect-tor.bro
>>
>>
>> Regards,
>> Sunu
>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170222/cfde4331/attachment.html 

From jwclark at ucar.edu  Wed Feb 22 12:08:51 2017
From: jwclark at ucar.edu (John Clark)
Date: Wed, 22 Feb 2017 13:08:51 -0700
Subject: [Bro] Detect tor (j. clark)
In-Reply-To: <mailman.1.1487793601.1776.bro@bro.org>
References: <mailman.1.1487793601.1776.bro@bro.org>
Message-ID: <242646e0-ef81-87de-6cbb-e7dee20fa4ce@ucar.edu>


I did some general research into this about a year ago and discovered
that the Cert used to encrypt tor changes about every half hour.  So if
you can detect repeated changes in the cert with a particular IP it
might be a good IoC.

From fatema.bannatwala at gmail.com  Wed Feb 22 12:37:09 2017
From: fatema.bannatwala at gmail.com (fatema bannatwala)
Date: Wed, 22 Feb 2017 15:37:09 -0500
Subject: [Bro] Splunk or ELK to parse Bro logs
Message-ID: <CACX0rUSPy-L_a1bZV9mo6FFpv=sdLrdw5Pq3S_Ds5pkOQTnKig@mail.gmail.com>

>Many thanks to all for your inputs. Regarding to use ELK, is it safe to
use latest versions of Logstash, Elasticsearch adn Kibana? What version do
you recommend?

We have been forwarding some of our bro logs to ELK stack, and recently
upgraded to the latest ELK versions [5.2.0].
It works just fine.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170222/8a0d8948/attachment.html 

From gfaulkner.nsm at gmail.com  Wed Feb 22 15:29:40 2017
From: gfaulkner.nsm at gmail.com (Gary Faulkner)
Date: Wed, 22 Feb 2017 17:29:40 -0600
Subject: [Bro] AF_PACKET on Ubuntu 16.04
In-Reply-To: <dad40440-ffa6-bede-4863-8ac9f78951ba@gmail.com>
References: <ebcb9ee8-6976-b8dc-a5b9-769edcc02165@gmail.com>
	<dad40440-ffa6-bede-4863-8ac9f78951ba@gmail.com>
Message-ID: <4ce67c1f-849c-a1ca-4543-379e538ecf75@gmail.com>

I'm trying AF_PACKET with Bro, but seem to be running a kernel and 
driver combo that doesn't appear to properly support symmetric hashing. 
I'm on Ubuntu 16.04 with kernel 4.4.0-59-generic. From what I can tell 
the patches should have been added around kernel 4.4.0-39 or so, but 
Justin's verification tool and Bro both seem to agree that it is broken 
on my system. I've tried with the OS supplied IXGBE driver (4.2.1-k) as 
well as compiling from scratch using a recent IXGBE directly from Intel 
(5.0.4). Is there a known working kernel and driver combo for Ubuntu 
16.04, or are the necessary patches still not pushed into 16.04?

Thanks,

Gary


From michalpurzynski1 at gmail.com  Wed Feb 22 15:51:47 2017
From: michalpurzynski1 at gmail.com (=?utf-8?Q?Micha=C5=82_Purzy=C5=84ski?=)
Date: Wed, 22 Feb 2017 15:51:47 -0800
Subject: [Bro] AF_PACKET on Ubuntu 16.04
In-Reply-To: <4ce67c1f-849c-a1ca-4543-379e538ecf75@gmail.com>
References: <ebcb9ee8-6976-b8dc-a5b9-769edcc02165@gmail.com>
	<dad40440-ffa6-bede-4863-8ac9f78951ba@gmail.com>
	<4ce67c1f-849c-a1ca-4543-379e538ecf75@gmail.com>
Message-ID: <0B18A1BF-FB8D-41AC-9181-BA6BAA5CE973@gmail.com>

Have you disabled hardware hash with ethtool? By default kernel will use the card hash which is asymmetric.

You can verify it with ethtool -k <card>

Look for rxhash - should be disabled.

https://github.com/pevma/SEPTun

Should show you how to prepare your system, you can ignore the core isolation and affinity for bro.

> On Feb 22, 2017, at 3:29 PM, Gary Faulkner <gfaulkner.nsm at gmail.com> wrote:
> 
> I'm trying AF_PACKET with Bro, but seem to be running a kernel and 
> driver combo that doesn't appear to properly support symmetric hashing. 
> I'm on Ubuntu 16.04 with kernel 4.4.0-59-generic. From what I can tell 
> the patches should have been added around kernel 4.4.0-39 or so, but 
> Justin's verification tool and Bro both seem to agree that it is broken 
> on my system. I've tried with the OS supplied IXGBE driver (4.2.1-k) as 
> well as compiling from scratch using a recent IXGBE directly from Intel 
> (5.0.4). Is there a known working kernel and driver combo for Ubuntu 
> 16.04, or are the necessary patches still not pushed into 16.04?
> 
> Thanks,
> 
> Gary
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170222/d1a1b78d/attachment.html 

From gfaulkner.nsm at gmail.com  Wed Feb 22 16:47:51 2017
From: gfaulkner.nsm at gmail.com (Gary Faulkner)
Date: Wed, 22 Feb 2017 18:47:51 -0600
Subject: [Bro] AF_PACKET on Ubuntu 16.04
In-Reply-To: <0B18A1BF-FB8D-41AC-9181-BA6BAA5CE973@gmail.com>
References: <ebcb9ee8-6976-b8dc-a5b9-769edcc02165@gmail.com>
	<dad40440-ffa6-bede-4863-8ac9f78951ba@gmail.com>
	<4ce67c1f-849c-a1ca-4543-379e538ecf75@gmail.com>
	<0B18A1BF-FB8D-41AC-9181-BA6BAA5CE973@gmail.com>
Message-ID: <dd52a62e-a2b0-e448-d207-c4e4a234e315@gmail.com>

I tried to follow the guide fairly closely and adapt for Bro with the
exception of BIOS level tuning (which I plan to investigate later).
rxhash is set to off. I was cpu pinning bro before, so I am continuing
to do so. Settings are below as well as a rough script I am tweaking to
load them.

For troubleshooting purposes I decided not to simplify the script with a
loop as I was running into some issues with command order (especially
with set_irq_affinity placement) as well as a couple unsupported options:

Features for eth4:

rx-checksumming: off
tx-checksumming: off
        tx-checksum-ipv4: off
        tx-checksum-ip-generic: off [fixed]
        tx-checksum-ipv6: off
        tx-checksum-fcoe-crc: off [fixed]
        tx-checksum-sctp: off [fixed]
scatter-gather: off
        tx-scatter-gather: off
        tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
        tx-tcp-segmentation: off
        tx-tcp-ecn-segmentation: off [fixed]
        tx-tcp6-segmentation: off
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
rx-vlan-offload: off
tx-vlan-offload: off
ntuple-filters: off
receive-hashing: off
highdma: on [fixed]
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
busy-poll: on [fixed]
hw-tc-offload: off [fixed]


#!/bin/bash

#Unload any existing module and load with new parameters
rmmod ixgbe
modprobe ixgbe MQ=0,0,0,0 RSS=1,1,1,1 VMDQ=0,0,0,0
InterruptThrottleRate=12500,12500,12500,12500 FCoE=0,0,0,0 LRO=0,0,0,0
vxlan_rx=0,0,0,0
sleep 1

#Disable irqbalance to stop bouncing interrupts between cores
killall irqbalance
sleep 1

#Enable interfaces in promisc mode
ip link set eth4 promisc on arp off up
ip link set eth6 promisc on arp off up
sleep 1

#Disable IPv6 on interfaces
echo 1 > /proc/sys/net/ipv6/conf/eth4/disable_ipv6
echo 1 > /proc/sys/net/ipv6/conf/eth6/disable_ipv6

#Enable Jumbo Frames (MTU of 9216 used on routers)
ip link set dev eth4 mtu 9216
ip link set dev eth6 mtu 9216

#Enforce a single RX queue
ethtool -L eth4 combined 1
ethtool -L eth6 combined 1

#Manage interrupts
ethtool -C eth4 adaptive-rx on rx-usecs 100
ethtool -C eth6 adaptive-rx on rx-usecs 100

#Lower the NIC ring descriptor size
ethtool -G eth4 rx 512
ethtool -G eth6 rx 512

#Disable pause frames
#ethtool -A eth4 autoneg off
#ethtool -A eth6 autoneg off
ethtool -A eth4 rx off tx off
ethtool -A eth6 rx off tx off

#Disable offloading features
ethtool -K eth4 rx off
ethtool -K eth4 tx off
ethtool -K eth4 tso off
ethtool -K eth4 ufo off
ethtool -K eth4 gso off
ethtool -K eth4 gro off
ethtool -K eth4 lro off
ethtool -K eth4 tx-nocache-copy off
ethtool -K eth4 rxhash off
ethtool -K eth4 ntuple off
ethtool -K eth4 sg off
ethtool -K eth4 txvlan off
ethtool -K eth4 rxvlan off
ethtool -K eth6 rx off
ethtool -K eth6 tx off
ethtool -K eth6 tso off
ethtool -K eth6 ufo off
ethtool -K eth6 gso off
ethtool -K eth6 gro off
ethtool -K eth6 lro off
ethtool -K eth6 tx-nocache-copy off
ethtool -K eth6 rxhash off
ethtool -K eth6 ntuple off
ethtool -K eth6 sg off
ethtool -K eth6 txvlan off
ethtool -K eth6 rxvlan off

#Set irq affinity
/bin/bash ./set_irq_affinity 2 eth4
/bin/bash ./set_irq_affinity 3 eth6

On 2/22/2017 5:51 PM, Micha? Purzy?ski wrote:
> Have you disabled hardware hash with ethtool? By default kernel will use the card hash which is asymmetric.
>
> You can verify it with ethtool -k <card>
>
> Look for rxhash - should be disabled.
>
> https://github.com/pevma/SEPTun
>
> Should show you how to prepare your system, you can ignore the core isolation and affinity for bro.
>
>> On Feb 22, 2017, at 3:29 PM, Gary Faulkner <gfaulkner.nsm at gmail.com> wrote:
>>
>> I'm trying AF_PACKET with Bro, but seem to be running a kernel and 
>> driver combo that doesn't appear to properly support symmetric hashing. 
>> I'm on Ubuntu 16.04 with kernel 4.4.0-59-generic. From what I can tell 
>> the patches should have been added around kernel 4.4.0-39 or so, but 
>> Justin's verification tool and Bro both seem to agree that it is broken 
>> on my system. I've tried with the OS supplied IXGBE driver (4.2.1-k) as 
>> well as compiling from scratch using a recent IXGBE directly from Intel 
>> (5.0.4). Is there a known working kernel and driver combo for Ubuntu 
>> 16.04, or are the necessary patches still not pushed into 16.04?
>>
>> Thanks,
>>
>> Gary
>>
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From rdump at river.com  Wed Feb 22 20:20:01 2017
From: rdump at river.com (Richard Johnson)
Date: Wed, 22 Feb 2017 21:20:01 -0700
Subject: [Bro] Detect tor
In-Reply-To: <CALGAZaDaiNTZeV==ZKKC5GJ=wZnAC1wYZGYyQms=SL421=_HTA@mail.gmail.com>
References: <CALGAZaAc+6f30WyxBhqvOpHVVNdj4EcrEewAPjsrBasVUwWmOw@mail.gmail.com>
	<CALGAZaCTH-LX0qSmShOtj14xwFK3pGzerK57z4zOWz0ELUVLEw@mail.gmail.com>
	<CACX0rURxCpBadbFcCYkid3U4j5j45AVGygBizZ-p53cwnOahZQ@mail.gmail.com>
	<CALGAZaDaiNTZeV==ZKKC5GJ=wZnAC1wYZGYyQms=SL421=_HTA@mail.gmail.com>
Message-ID: <ab2e4953-9711-b6cf-0d7c-8a4ec2e719f7@river.com>

If you want valid, low false positive, detection of the public Tor (not TOR)
network use, you can look at the descriptors of the public relays. Get them
from any Tor node you run, or download from the Tor Project site. That will
give you IP addresses and ports over time. A connection to those is very
probably Tor user->network traffic.

A connection to a Tor node's IP on a port that isn't listed as a Tor port at
the time of interest is much less likely to be Tor traffic. That's one of the
failings of intel feeds listing only IPs, as almost all do when it comes to Tor.

Bridges complicate the picture, as they're handed only to a limited subset of
users. There, you may want to consider active measures--connect to the same
port yourself, see if you can evoke a Tor handshake. China's delay on active
probing of the ports was on the order of hours to days when this was most
popular; they may have gotten faster since.

Trying to ID Tor traffic characteristics is not as easy as it used to be. DPI
vendors can often keep up, but it's unlikely they'll share the competitive
advantage.

Further along the arms race, bridges using pluggable transports like obfs4, or
connections using domain fronting are not going to be easily detected, even by
active probing.


Richard

On 2017-02-22 08:26, ps sunu wrote:
> ok thanks for your info
> 
> On Wed, Feb 22, 2017 at 6:51 PM, fatema bannatwala <
> fatema.bannatwala at gmail.com> wrote:
> 
>> Another thing you could try is, if you use intel framework, then you can
>> feed the intel FW with
>> the IOCs data for TOR, and load it in Intel, so that you will get logs in
>> intel.log, whenever there's
>> a hit on TOR IPs in your network traffic.
>>
>> Thanks,
>> Fatema.
>>
>> On Wed, Feb 22, 2017 at 4:50 AM, ps sunu <pssunu6 at gmail.com> wrote:
>>
>>>
>>> -----
>>> Hi,
>>>                 Which is the best TOR detection script in bro ? below one
>>> is good , or any other script there ?
>>>
>>> https://raw.githubusercontent.com/sethhall/bro-junk-drawer/m
>>> aster/detect-tor.bro
>>>
>>>
>>> Regards,
>>> Sunu
>>>
>>>
>>>
>>>
>>
> 
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 


From daniel.guerra69 at gmail.com  Thu Feb 23 01:52:04 2017
From: daniel.guerra69 at gmail.com (Daniel Guerra)
Date: Thu, 23 Feb 2017 10:52:04 +0100
Subject: [Bro] Detect tor
In-Reply-To: <ab2e4953-9711-b6cf-0d7c-8a4ec2e719f7@river.com>
References: <CALGAZaAc+6f30WyxBhqvOpHVVNdj4EcrEewAPjsrBasVUwWmOw@mail.gmail.com>
	<CALGAZaCTH-LX0qSmShOtj14xwFK3pGzerK57z4zOWz0ELUVLEw@mail.gmail.com>
	<CACX0rURxCpBadbFcCYkid3U4j5j45AVGygBizZ-p53cwnOahZQ@mail.gmail.com>
	<CALGAZaDaiNTZeV==ZKKC5GJ=wZnAC1wYZGYyQms=SL421=_HTA@mail.gmail.com>
	<ab2e4953-9711-b6cf-0d7c-8a4ec2e719f7@river.com>
Message-ID: <A9794BBA-2866-41FE-A0A1-96C6D2FD0E88@gmail.com>

Tor can be detected by looking at the ssl certificates. Because the certificates
are generated by tor, the subject issuer or ssl_hostname can be used to detect
it.

This example matches only if subject and issuer match. I have seen tor connections
module DetectTor;

event ssl_established(c: connection ) &priority=6
        {
                if ( c$ssl?$subject && /^CN=www.[0-9a-zA-Z]+.(net|com)$/  == c$ssl$subject && c$ssl?$issuer && /^CN=www.[0-9a-zA-Z]+.(com|net)$/ == c$ssl$issuer || (c$ssl?$server_name && /^CN=www.[0-9a-zA-Z]+.(net|com)$/  == c$ssl?$server_name  )
                        {
                                add c$service["tor"];
                        }
        }

Regards,

Daniel

> On 23 Feb 2017, at 05:20, Richard Johnson <rdump at river.com> wrote:
> 
> If you want valid, low false positive, detection of the public Tor (not TOR)
> network use, you can look at the descriptors of the public relays. Get them
> from any Tor node you run, or download from the Tor Project site. That will
> give you IP addresses and ports over time. A connection to those is very
> probably Tor user->network traffic.
> 
> A connection to a Tor node's IP on a port that isn't listed as a Tor port at
> the time of interest is much less likely to be Tor traffic. That's one of the
> failings of intel feeds listing only IPs, as almost all do when it comes to Tor.
> 
> Bridges complicate the picture, as they're handed only to a limited subset of
> users. There, you may want to consider active measures--connect to the same
> port yourself, see if you can evoke a Tor handshake. China's delay on active
> probing of the ports was on the order of hours to days when this was most
> popular; they may have gotten faster since.
> 
> Trying to ID Tor traffic characteristics is not as easy as it used to be. DPI
> vendors can often keep up, but it's unlikely they'll share the competitive
> advantage.
> 
> Further along the arms race, bridges using pluggable transports like obfs4, or
> connections using domain fronting are not going to be easily detected, even by
> active probing.
> 
> 
> Richard
> 
> On 2017-02-22 08:26, ps sunu wrote:
>> ok thanks for your info
>> 
>> On Wed, Feb 22, 2017 at 6:51 PM, fatema bannatwala <
>> fatema.bannatwala at gmail.com> wrote:
>> 
>>> Another thing you could try is, if you use intel framework, then you can
>>> feed the intel FW with
>>> the IOCs data for TOR, and load it in Intel, so that you will get logs in
>>> intel.log, whenever there's
>>> a hit on TOR IPs in your network traffic.
>>> 
>>> Thanks,
>>> Fatema.
>>> 
>>> On Wed, Feb 22, 2017 at 4:50 AM, ps sunu <pssunu6 at gmail.com> wrote:
>>> 
>>>> 
>>>> -----
>>>> Hi,
>>>>                Which is the best TOR detection script in bro ? below one
>>>> is good , or any other script there ?
>>>> 
>>>> https://raw.githubusercontent.com/sethhall/bro-junk-drawer/m
>>>> aster/detect-tor.bro
>>>> 
>>>> 
>>>> Regards,
>>>> Sunu
>>>> 
>>>> 
>>>> 
>>>> 
>>> 
>> 
>> 
>> 
>> _______________________________________________
>> Bro mailing list
>> bro at bro-ids.org
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro



From giedrius.ramas at gmail.com  Thu Feb 23 03:21:49 2017
From: giedrius.ramas at gmail.com (Giedrius Ramas)
Date: Thu, 23 Feb 2017 13:21:49 +0200
Subject: [Bro] bro 2.5 . How to get meta fields on intel.log
Message-ID: <CAHr6V5rr=nU_hFj2ADKZsqoTp=TBcwpA_ynX=O+ZcrFFDAFrqA@mail.gmail.com>

Hi Seth,
How can we get working those bro extensions for Bro 2.4 on Bro 2.5
Currently I get errors:
error in /opt/bro/share/bro/base/frameworks/intel/./main.bro, line 155:
already defined (Intel::extend_match)
internal warning in
/opt/bro/share/bro/my_scripts/intel-ext/./scripts/main.bro, line 20:
Duplicate identifier documentation: Intel::extend_match
proxy scripts failed.
error in /opt/bro/share/bro/base/frameworks/intel/./main.bro, line 155:
already defined (Intel::extend_match)
internal warning in
/opt/bro/share/bro/my_scripts/intel-ext/./scripts/main.bro, line 20:
Duplicate identifier documentation: Intel::extend_match
ids-nksc004-eth1-1 scripts failed.
error in /opt/bro/share/bro/base/frameworks/intel/./main.bro, line 155:
already defined (Intel::extend_match)
internal warning in
/opt/bro/share/bro/my_scripts/intel-ext/./scripts/main.bro, line 20:
Duplicate identifier documentation: Intel::extend_match


Our intel data have following format :

#fields indicator       indicator_type  meta.desc       meta.cif_confidence
    meta.source

And we need to have these meta's: meta.desc, meta.cif_confidence,
meta.source on bro.intel log as previously had with bro extensions for Bro
2.4 found on https://github.com/sethhall/intel-ext.


Or question is how to get meta fields on bro intel.log.?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170223/ebec5873/attachment.html 

From giedrius.ramas at gmail.com  Thu Feb 23 04:18:08 2017
From: giedrius.ramas at gmail.com (Giedrius Ramas)
Date: Thu, 23 Feb 2017 14:18:08 +0200
Subject: [Bro] bro 2.5 . How to get meta fields on intel.log
Message-ID: <CAHr6V5pDdVqn-OL1QqTfU5_Em7zU3XaurFC0QRPS9U=0pdn1Eg@mail.gmail.com>

Hi ,

How can we get working those bro extensions for Bro 2.4 on Bro 2.5
Currently I get errors:
error in /opt/bro/share/bro/base/frameworks/intel/./main.bro, line 155:
already defined (Intel::extend_match)
internal warning in /opt/bro/share/bro/my_scripts/intel-ext/./scripts/main.bro,
line 20: Duplicate identifier documentation: Intel::extend_match
proxy scripts failed.
error in /opt/bro/share/bro/base/frameworks/intel/./main.bro, line 155:
already defined (Intel::extend_match)
internal warning in /opt/bro/share/bro/my_scripts/intel-ext/./scripts/main.bro,
line 20: Duplicate identifier documentation: Intel::extend_match
ids-nksc004-eth1-1 scripts failed.
error in /opt/bro/share/bro/base/frameworks/intel/./main.bro, line 155:
already defined (Intel::extend_match)
internal warning in /opt/bro/share/bro/my_scripts/intel-ext/./scripts/main.bro,
line 20: Duplicate identifier documentation: Intel::extend_match


Our intel data have following format :

#fields indicator       indicator_type  meta.desc       meta.cif_confidence
    meta.source

And we need to have these meta's: meta.desc, meta.cif_confidence,
meta.source on bro.intel log as previously had with bro extensions for Bro
2.4 found on https://github.com/sethhall/intel-ext
.


Or question is how to get meta fields on bro intel.log.?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170223/eb27cf4b/attachment.html 

From jan.grashoefer at gmail.com  Thu Feb 23 05:12:38 2017
From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=)
Date: Thu, 23 Feb 2017 14:12:38 +0100
Subject: [Bro] bro 2.5 . How to get meta fields on intel.log
In-Reply-To: <CAHr6V5pDdVqn-OL1QqTfU5_Em7zU3XaurFC0QRPS9U=0pdn1Eg@mail.gmail.com>
References: <CAHr6V5pDdVqn-OL1QqTfU5_Em7zU3XaurFC0QRPS9U=0pdn1Eg@mail.gmail.com>
Message-ID: <a3eb6968-f44b-7040-3721-d03a839fbd9b@gmail.com>

Hi,

> How can we get working those bro extensions for Bro 2.4 on Bro 2.5
> Currently I get errors:
> ...
> line 20: Duplicate identifier documentation: Intel::extend_match

the intel framework has been reworked for 2.5 and includes a similar
extension mechanism (a hook instead of an event). The following blog
entry goes into details:
http://blog.bro.org/2016/12/the-intelligence-framework-update.html

> Or question is how to get meta fields on bro intel.log.?

You can use the extension mechanisms included but keep in mind that each
hit might be associated with multiple indicators and each indicator
might be associated with multiple meta data records.

Jan

From andrew.dellana at bayer.com  Thu Feb 23 06:20:37 2017
From: andrew.dellana at bayer.com (Andrew Dellana)
Date: Thu, 23 Feb 2017 14:20:37 +0000
Subject: [Bro] Bro Detections and Compliance Questions
Message-ID: <ecffc231622447e8b92f3530e6883a13@moxde9.na.bayer.cnb>

Hello,

When a bro script detects something, how can you go about resolving the issues that caused it (assuming it wasn't noise that caused it)? Is there something that I change in Bro or is this something that would be covered in the corporate compliance / security?

Following up with that what is the best practice to analyze the packet captures from Bro to determine if there is an actual issue? I am currently looking into Splunk as a log parser.




Best regards,

Andrew Dellana
Intern
________________________

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170223/501af6c8/attachment-0001.html 

From giedrius.ramas at gmail.com  Thu Feb 23 06:34:55 2017
From: giedrius.ramas at gmail.com (Giedrius Ramas)
Date: Thu, 23 Feb 2017 16:34:55 +0200
Subject: [Bro] bro 2.5 . How to get meta fields on intel.log
In-Reply-To: <a3eb6968-f44b-7040-3721-d03a839fbd9b@gmail.com>
References: <CAHr6V5pDdVqn-OL1QqTfU5_Em7zU3XaurFC0QRPS9U=0pdn1Eg@mail.gmail.com>
	<a3eb6968-f44b-7040-3721-d03a839fbd9b@gmail.com>
Message-ID: <CAHr6V5o619tqqJrufW1HnWaa8aPdzHRA=7zFAzkoqtPsu=_1CQ@mail.gmail.com>

Thanks, Jan
Got it working .

On Thu, Feb 23, 2017 at 3:12 PM, Jan Grash?fer <jan.grashoefer at gmail.com>
wrote:

> Hi,
>
> > How can we get working those bro extensions for Bro 2.4 on Bro 2.5
> > Currently I get errors:
> > ...
> > line 20: Duplicate identifier documentation: Intel::extend_match
>
> the intel framework has been reworked for 2.5 and includes a similar
> extension mechanism (a hook instead of an event). The following blog
> entry goes into details:
> http://blog.bro.org/2016/12/the-intelligence-framework-update.html
>
> > Or question is how to get meta fields on bro intel.log.?
>
> You can use the extension mechanisms included but keep in mind that each
> hit might be associated with multiple indicators and each indicator
> might be associated with multiple meta data records.
>
> Jan
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170223/ad6db5a0/attachment.html 

From seth at icir.org  Thu Feb 23 06:40:52 2017
From: seth at icir.org (Seth Hall)
Date: Thu, 23 Feb 2017 09:40:52 -0500
Subject: [Bro] bro 2.5 . How to get meta fields on intel.log
In-Reply-To: <CAHr6V5pDdVqn-OL1QqTfU5_Em7zU3XaurFC0QRPS9U=0pdn1Eg@mail.gmail.com>
References: <CAHr6V5pDdVqn-OL1QqTfU5_Em7zU3XaurFC0QRPS9U=0pdn1Eg@mail.gmail.com>
Message-ID: <FDAE498F-AD83-419C-A92C-944D161F98CE@icir.org>


> On Feb 23, 2017, at 7:18 AM, Giedrius Ramas <giedrius.ramas at gmail.com> wrote:
> 
> And we need to have these meta's: meta.desc, meta.cif_confidence, meta.source on bro.intel log as previously had with bro extensions for Bro 2.4 found on https://github.com/sethhall/intel-ext.

Sorry about the confusion.  I'll put a note on that repository that the feature is now built into Bro and point to Jan's blog post.

  .Seth


--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From bro at pingtrip.com  Thu Feb 23 09:47:19 2017
From: bro at pingtrip.com (Dave Crawford)
Date: Thu, 23 Feb 2017 12:47:19 -0500
Subject: [Bro] Debugging high-cpu on a single worker
Message-ID: <8D4AB0F0-15D1-4206-B01C-1BE9F14208DB@pingtrip.com>

I?m seeing a repeatable issue where the first worker of ten always has a higher CPU usage.

 11  3014 bro        20   0 2078M 1250M  502M R 97.4  1.0 18h34:48 /opt/bro/bin/bro -i netmap::bro}0 -U .status -p broctl -p broctl-live -p
 14  3005 bro        20   0 1885M 1056M  501M R 83.5  0.8 13h09:44 /opt/bro/bin/bro -i netmap::bro}4 -U .status -p broctl -p broctl-live -p
 18  3013 bro        20   0 1746M  983M  501M R 67.3  0.8 13h25:41 /opt/bro/bin/bro -i netmap::bro}8 -U .status -p broctl -p broctl-live -p
 17  3009 bro        20   0 2139M  977M  502M S 63.5  0.8 13h11:37 /opt/bro/bin/bro -i netmap::bro}7 -U .status -p broctl -p broctl-live -p
 19  3008 bro        20   0 2168M 1333M  501M S 63.0  1.0 13h30:12 /opt/bro/bin/bro -i netmap::bro}9 -U .status -p broctl -p broctl-live -p
 13  3006 bro        20   0 2164M 1329M  502M S 62.5  1.0 13h16:05 /opt/bro/bin/bro -i netmap::bro}3 -U .status -p broctl -p broctl-live -p
 12  3012 bro        20   0 2194M 1287M  501M R 60.6  1.0 13h15:14 /opt/bro/bin/bro -i netmap::bro}2 -U .status -p broctl -p broctl-live -p
 15  3011 bro        20   0 2177M  982M  502M R 60.6  0.8 13h03:26 /opt/bro/bin/bro -i netmap::bro}5 -U .status -p broctl -p broctl-live -p
 16  3007 bro        20   0 2317M 1237M  502M R 60.1  1.0 13h12:35 /opt/bro/bin/bro -i netmap::bro}6 -U .status -p broctl -p broctl-live -p
 20  3010 bro        20   0 2328M  991M  501M R 58.2  0.8 13h14:40 /opt/bro/bin/bro -i netmap::bro}1 -U .status -p broctl -p broctl-live -p

Even after a fresh restart the ?bro}0? worker is about double the CPU usage compared to the other workers:

 11  1799 bro        20   0 1422M  668M  501M S 73.5  0.5  0:12.28 /opt/bro/bin/bro -i netmap::bro}0 -U .status -p broctl -p broctl-live -p
 15  1813 bro        20   0 1420M  530M  364M S 37.9  0.4  0:07.04 /opt/bro/bin/bro -i netmap::bro}5 -U .status -p broctl -p broctl-live -p
 17  1831 bro        20   0 1420M  527M  362M R 37.5  0.4  0:06.92 /opt/bro/bin/bro -i netmap::bro}7 -U .status -p broctl -p broctl-live -p
 12  1805 bro        20   0 1424M  525M  356M R 36.5  0.4  0:07.04 /opt/bro/bin/bro -i netmap::bro}2 -U .status -p broctl -p broctl-live -p
 18  1830 bro        20   0 1420M  514M  350M R 35.1  0.4  0:06.67 /opt/bro/bin/bro -i netmap::bro}8 -U .status -p broctl -p broctl-live -p
 19  1826 bro        20   0 1421M  533M  367M S 34.6  0.4  0:07.15 /opt/bro/bin/bro -i netmap::bro}9 -U .status -p broctl -p broctl-live -p
 20  1832 bro        20   0 1421M  529M  363M R 34.6  0.4  0:07.06 /opt/bro/bin/bro -i netmap::bro}1 -U .status -p broctl -p broctl-live -p
 14  1827 bro        20   0 1420M  530M  365M S 34.6  0.4  0:06.87 /opt/bro/bin/bro -i netmap::bro}4 -U .status -p broctl -p broctl-live -p
 13  1802 bro        20   0 1418M  520M  356M S 34.6  0.4  0:06.79 /opt/bro/bin/bro -i netmap::bro}3 -U .status -p broctl -p broctl-live -p
 16  1824 bro        20   0 1417M  502M  339M R 33.1  0.4  0:06.60 /opt/bro/bin/bro -i netmap::bro}6 -U .status -p broctl -p broctl-live -p


A quick spot check of an hours worth of conn logs shows the traffic is balanced as expected:

MID_EXT-1 = 397479
MID_EXT-2 = 409771
...
MID_EXT-10 = 408699


The real issue is that its the only worker thats reporting capture_loss, which I assume is due to hitting 100% CPU usage:

1487866049.759038       600.000033      MID_EXT-2       4713    4710856 0.100046
1487866049.759333       600.000015      MID_EXT-9       5427    5193821 0.10449
1487866049.760596       600.000068      MID_EXT-5       5768    5548436 0.103957
1487866049.762628       600.000107      MID_EXT-8       5190    5224691 0.099336
1487866049.764048       600.000125      MID_EXT-3       5036    4947221 0.101795
1487866111.312071       600.000666      MID_EXT-6       8007    5959706 0.134352
1487866111.313803       600.000803      MID_EXT-4       6581    4981572 0.132107
1487866111.314788       600.000126      MID_EXT-7       6579    4905261 0.134121
1487866111.327390       600.000047      MID_EXT-10      7693    6369836 0.120772
1487866111.584908       600.000566      MID_EXT-1       823616  4227324 19.483153

Any tips on how to debug/profile further would be greatly appreciated.

Thanks!
-Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170223/7ee2d280/attachment.html 

From jan.grashoefer at gmail.com  Thu Feb 23 09:57:57 2017
From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=)
Date: Thu, 23 Feb 2017 18:57:57 +0100
Subject: [Bro] Debugging high-cpu on a single worker
In-Reply-To: <8D4AB0F0-15D1-4206-B01C-1BE9F14208DB@pingtrip.com>
References: <8D4AB0F0-15D1-4206-B01C-1BE9F14208DB@pingtrip.com>
Message-ID: <343def51-09ae-63b6-252f-c58f42111696@gmail.com>

Hi Dave,

> Any tips on how to debug/profile further would be greatly appreciated.

using https://github.com/0xxon/bro-scripts/blob/master/conn-workers.bro
and looking at the type etc. of connections that are handled by the
worker might help.

Jan

From jazoff at illinois.edu  Thu Feb 23 10:37:09 2017
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Thu, 23 Feb 2017 18:37:09 +0000
Subject: [Bro] Debugging high-cpu on a single worker
In-Reply-To: <8D4AB0F0-15D1-4206-B01C-1BE9F14208DB@pingtrip.com>
References: <8D4AB0F0-15D1-4206-B01C-1BE9F14208DB@pingtrip.com>
Message-ID: <EE843F7B-26B0-4914-A759-E4E1B4E4E000@illinois.edu>


> On Feb 23, 2017, at 12:47 PM, Dave Crawford <bro at pingtrip.com> wrote:
> 
> I?m seeing a repeatable issue where the first worker of ten always has a higher CPU usage.
> 
>  11  3014 bro        20   0 2078M 1250M  502M R 97.4  1.0 18h34:48 /opt/bro/bin/bro -i netmap::bro}0 -U .status -p broctl -p broctl-live -p

I'd guess it is one of two things:

The pin cpu setting that is pinning that worker to the same cpu that is being used for things like interrupts

The fact that it is the first worker on netmap may mean it receives all of a certain kind of traffic that can't otherwise be load balanced to the other workers.

What pin_cpus setting are you using and what does /proc/interrupts contain?

-- 
- Justin Azoff



From bro at pingtrip.com  Thu Feb 23 14:12:53 2017
From: bro at pingtrip.com (Dave Crawford)
Date: Thu, 23 Feb 2017 17:12:53 -0500
Subject: [Bro] Debugging high-cpu on a single worker
In-Reply-To: <EE843F7B-26B0-4914-A759-E4E1B4E4E000@illinois.edu>
References: <8D4AB0F0-15D1-4206-B01C-1BE9F14208DB@pingtrip.com>
	<EE843F7B-26B0-4914-A759-E4E1B4E4E000@illinois.edu>
Message-ID: <4FB1E387-6DCA-4ADD-B342-07A90AE6CAA2@pingtrip.com>


> I'd guess it is one of two things:
> 
> The pin cpu setting that is pinning that worker to the same cpu that is being used for things like interrupts
> 
> The fact that it is the first worker on netmap may mean it receives all of a certain kind of traffic that can't otherwise be load balanced to the other workers.
> 
> What pin_cpus setting are you using and what does /proc/interrupts contain?
> 
> -- 
> - Justin Azoff
> 

You nailed it, I missed the IRQ triage and that NIC is pinned to the first CPU in its numa node, which matches with the first one Bro is pinned to.

So now the dilemma is that hardware only has 10 cores per CPU. So either I reduce the number of Bro workers to 9, and leave a core dedicated to NIC interrupts, or I pin the NIC to a core on the other CPU and incur a latency hit due to it being on a different numa node.

-Dave

From robin at icir.org  Thu Feb 23 15:29:40 2017
From: robin at icir.org (Robin Sommer)
Date: Thu, 23 Feb 2017 15:29:40 -0800
Subject: [Bro] osquery integration
Message-ID: <20170223232939.GL52561@icir.org>

I wanted to send a pointer to a new integration project that Steffen
Haas presented at Bro4Pros recently: He wrote an osquery
(https://osquery.io) extension that connects the host monitor to Bro,
through Broker. The extension, along with a corresponding Bro script
framework, allows turning osquery's SQL-style queries into real-time
Bro event streams reflecting host-level activity. As an example, one
can ask osquery to send an event every time there's a new USB device
on a host, and then write a Bro script that processes these events as
they come in, just as any other Bro events. One can turn such host
activity into Bro log files for example, or correlate with the network
activity Bro is seeing.

The extension is currently in prototype state, we're working with the
osquery team to integrate it directly into their distribution---a
corresponding PR is open. In the meantime, we'd appreciate feedback,
either here on the list or through GitHub. You can find the project at
https://github.com/bro/bro-osquery .

For more context, see the Steffen's Bro4Pro slides:
https://www.bro.org//bro4pros2017/Haas_OSquery_Bro4Pros2017.pdf

Robin

-- 
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin

From hovsep.sanjay.levi at gmail.com  Fri Feb 24 16:38:30 2017
From: hovsep.sanjay.levi at gmail.com (Hovsep Levi)
Date: Sat, 25 Feb 2017 00:38:30 +0000
Subject: [Bro] Logging and memory leak
In-Reply-To: <CAJ19UZCQ-iuHNco0EiPFdRPgQ-Xd_x5B84q0QniN+8QpxXOeQA@mail.gmail.com>
References: <CAJ19UZBxxOEZqmT2MYMBVsjno3mimNjN6egRWv0h5A-Sp_bd_Q@mail.gmail.com>
	<A7503EE4-D454-4404-9239-DA3C8A6D559A@illinois.edu>
	<CAJ19UZDJwoGq9RGKYZ+OftCeS+AOqxyfmAbV_3PHmghfWUEFFA@mail.gmail.com>
	<F1308A75-9240-4DEB-88C4-B1ED8F9CD5F5@illinois.edu>
	<CAJ19UZD+0ZR3W2WYMPc8ELLTTKrkYPtyJ0J4EnWunMLeF6bhdg@mail.gmail.com>
	<6B204A64-5F4C-4230-8741-9CDD550EC433@illinois.edu>
	<CAJ19UZCQ-iuHNco0EiPFdRPgQ-Xd_x5B84q0QniN+8QpxXOeQA@mail.gmail.com>
Message-ID: <CAJ19UZD-SzuyFAchbFEdHZeL_2iZaTNjei=hGxf_z=wdK_vGFQ@mail.gmail.com>

...The saga continues...

Disabling protocols/ssl/validate-certs relieved the rapid memory leak but
the problem still exists over long periods of time.

Implemented the change on Feb 6th and by Feb 18th the VZSIZE / RSS was 32G
/ 2G.  By Feb 23rd it was 337G / 24G.

FWIW I'm considering migrating the cluster to Intel CPUs but they are the
same speed so I'm not sure it's worth the trouble since the worker count
will remain the same.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170225/806cf488/attachment.html 

From remi.jullian at ssi.gouv.fr  Mon Feb 27 03:12:38 2017
From: remi.jullian at ssi.gouv.fr (Jullian Remi)
Date: Mon, 27 Feb 2017 12:12:38 +0100
Subject: [Bro] Using native PF_RING plugin with broctl
Message-ID: <58B409A6.3010805@ssi.gouv.fr>

Hi all,

I am trying to use Bro's PF_RING plugin with broctl, using a simple bro
cluster on a single host.

Here is an extract of my 'node.cfg' file:

[worker]
type=worker
host=localhost
interface=pf_ring::eth0
lb_method=pf_ring
lb_procs=8
pin_cpus=0,1,2,3,4,5,6,7

When I used the deploy command, I got the following error: "fatal error:
type of packet source 'pf_ring' no recognized, or mode not supported"

Here is the output of the deploy command:

[BroControl] > deploy
...
starting ...
starting manager ...
starting proxy ...
starting worker-1
...
starting worker-8
worker-1 terminated immediately after starting; check output with "diag"
...
worker-8 terminated immediately after starting; check output with "diag"

And when running "diag":

[BroControl] > diag

==== stderr.log
fatal error: type of packet source 'pf_ring' no recognized, or mode not
supported


However I do not have any problem running bro as a standalone process
with local commands such as:

$/usr/local/bro/bin/bro -i pf_ring::eth0
listening on eth0

and:

$/usr/local/bro/bin/bro -N | grep PF
Bro::PF_RING - Packet acquisition via PF_RING (dynamic, version 1.0)

This tends to prove Bro plugin has been installed and works.

I think Broctl is launching Bro binary without the right settings for
the plugin to be found/to work correctly. Am I missing something with
configuration files ?
May be the environment variables aren't properly set?

Does anyone use bro's PF_RING plugin with a cluster configuration
without issues?

Thanks,

R?mi


From landy-bible at utulsa.edu  Mon Feb 27 04:02:29 2017
From: landy-bible at utulsa.edu (Landy Bible)
Date: Mon, 27 Feb 2017 12:02:29 +0000
Subject: [Bro] Using native PF_RING plugin with broctl
In-Reply-To: <58B409A6.3010805@ssi.gouv.fr>
References: <58B409A6.3010805@ssi.gouv.fr>
Message-ID: <CADtxXLm7G8FALyakZRvgbNzB9goUSZSHHB29mEC9vU0dV+2Daw@mail.gmail.com>

I think you just need "interface=eth0". It knows to use pf_ring because of
the next line.

On Mon, Feb 27, 2017, 05:14 Jullian Remi <remi.jullian at ssi.gouv.fr> wrote:

> Hi all,
>
> I am trying to use Bro's PF_RING plugin with broctl, using a simple bro
> cluster on a single host.
>
> Here is an extract of my 'node.cfg' file:
>
> [worker]
> type=worker
> host=localhost
> interface=pf_ring::eth0
> lb_method=pf_ring
> lb_procs=8
> pin_cpus=0,1,2,3,4,5,6,7
>
> When I used the deploy command, I got the following error: "fatal error:
> type of packet source 'pf_ring' no recognized, or mode not supported"
>
> Here is the output of the deploy command:
>
> [BroControl] > deploy
> ...
> starting ...
> starting manager ...
> starting proxy ...
> starting worker-1
> ...
> starting worker-8
> worker-1 terminated immediately after starting; check output with "diag"
> ...
> worker-8 terminated immediately after starting; check output with "diag"
>
> And when running "diag":
>
> [BroControl] > diag
>
> ==== stderr.log
> fatal error: type of packet source 'pf_ring' no recognized, or mode not
> supported
>
>
> However I do not have any problem running bro as a standalone process
> with local commands such as:
>
> $/usr/local/bro/bin/bro -i pf_ring::eth0
> listening on eth0
>
> and:
>
> $/usr/local/bro/bin/bro -N | grep PF
> Bro::PF_RING - Packet acquisition via PF_RING (dynamic, version 1.0)
>
> This tends to prove Bro plugin has been installed and works.
>
> I think Broctl is launching Bro binary without the right settings for
> the plugin to be found/to work correctly. Am I missing something with
> configuration files ?
> May be the environment variables aren't properly set?
>
> Does anyone use bro's PF_RING plugin with a cluster configuration
> without issues?
>
> Thanks,
>
> R?mi
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170227/2c81af2c/attachment.html 

From remi.jullian at ssi.gouv.fr  Mon Feb 27 09:10:47 2017
From: remi.jullian at ssi.gouv.fr (Jullian Remi)
Date: Mon, 27 Feb 2017 18:10:47 +0100
Subject: [Bro] Using native PF_RING plugin with broctl
In-Reply-To: <CADtxXLm7G8FALyakZRvgbNzB9goUSZSHHB29mEC9vU0dV+2Daw@mail.gmail.com>
References: <58B409A6.3010805@ssi.gouv.fr>
	<CADtxXLm7G8FALyakZRvgbNzB9goUSZSHHB29mEC9vU0dV+2Daw@mail.gmail.com>
Message-ID: <58B45D97.8040204@ssi.gouv.fr>

When I set 'interface=eth0' and 'lb_method=pf_ring', the broctl deploy
command works, bro starts, but PF_RING is not used. Indeed, all workers
receive the same packets (i.e no load-balancing is performed).

When I cat the file /proc/net/pf_ring/info, the total number of rings
used is 0. Moreover, when I put a breakpoint within Source.cc:192
(PcapSource::ExtractNextPacket), I can see the call to the libpcap
function pcap_next(), which should never be called.

> I think you just need "interface=eth0". It knows to use pf_ring because
> of the next line.
> 
> 
> On Mon, Feb 27, 2017, 05:14 Jullian Remi <remi.jullian at ssi.gouv.fr
> <mailto:remi.jullian at ssi.gouv.fr>> wrote:
> 
>     Hi all,
> 
>     I am trying to use Bro's PF_RING plugin with broctl, using a simple bro
>     cluster on a single host.
> 
>     Here is an extract of my 'node.cfg' file:
> 
>     [worker]
>     type=worker
>     host=localhost
>     interface=pf_ring::eth0
>     lb_method=pf_ring
>     lb_procs=8
>     pin_cpus=0,1,2,3,4,5,6,7
> 
>     When I used the deploy command, I got the following error: "fatal error:
>     type of packet source 'pf_ring' no recognized, or mode not supported"
> 
>     Here is the output of the deploy command:
> 
>     [BroControl] > deploy
>     ...
>     starting ...
>     starting manager ...
>     starting proxy ...
>     starting worker-1
>     ...
>     starting worker-8
>     worker-1 terminated immediately after starting; check output with "diag"
>     ...
>     worker-8 terminated immediately after starting; check output with "diag"
> 
>     And when running "diag":
> 
>     [BroControl] > diag
> 
>     ==== stderr.log
>     fatal error: type of packet source 'pf_ring' no recognized, or mode not
>     supported
> 
> 
>     However I do not have any problem running bro as a standalone process
>     with local commands such as:
> 
>     $/usr/local/bro/bin/bro -i pf_ring::eth0
>     listening on eth0
> 
>     and:
> 
>     $/usr/local/bro/bin/bro -N | grep PF
>     Bro::PF_RING - Packet acquisition via PF_RING (dynamic, version 1.0)
> 
>     This tends to prove Bro plugin has been installed and works.
> 
>     I think Broctl is launching Bro binary without the right settings for
>     the plugin to be found/to work correctly. Am I missing something with
>     configuration files ?
>     May be the environment variables aren't properly set?
> 
>     Does anyone use bro's PF_RING plugin with a cluster configuration
>     without issues?
> 
>     Thanks,
> 
>     R?mi
> 
>     _______________________________________________
>     Bro mailing list
>     bro at bro-ids.org <mailto:bro at bro-ids.org>
>     http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
> 


From jazoff at illinois.edu  Mon Feb 27 09:30:20 2017
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Mon, 27 Feb 2017 17:30:20 +0000
Subject: [Bro] Using native PF_RING plugin with broctl
In-Reply-To: <58B409A6.3010805@ssi.gouv.fr>
References: <58B409A6.3010805@ssi.gouv.fr>
Message-ID: <FFC279B5-C7AF-4519-8EBF-E60735DCCBCD@illinois.edu>


> On Feb 27, 2017, at 6:12 AM, Jullian Remi <remi.jullian at ssi.gouv.fr> wrote:
> 
> 
> 
> However I do not have any problem running bro as a standalone process
> with local commands such as:
> 
> $/usr/local/bro/bin/bro -i pf_ring::eth0
> listening on eth0
> 
> and:
> 
> $/usr/local/bro/bin/bro -N | grep PF
> Bro::PF_RING - Packet acquisition via PF_RING (dynamic, version 1.0)
> 
> This tends to prove Bro plugin has been installed and works.
> 
> I think Broctl is launching Bro binary without the right settings for
> the plugin to be found/to work correctly. Am I missing something with
> configuration files ?
> May be the environment variables aren't properly set?

How did you install bro?  Do you have more than one version of bro installed?

What environment variables are you referring to?



-- 
- Justin Azoff



From remi.jullian at ssi.gouv.fr  Mon Feb 27 10:51:08 2017
From: remi.jullian at ssi.gouv.fr (Jullian Remi)
Date: Mon, 27 Feb 2017 19:51:08 +0100
Subject: [Bro] Using native PF_RING plugin with broctl
In-Reply-To: <FFC279B5-C7AF-4519-8EBF-E60735DCCBCD@illinois.edu>
References: <58B409A6.3010805@ssi.gouv.fr>
	<FFC279B5-C7AF-4519-8EBF-E60735DCCBCD@illinois.edu>
Message-ID: <58B4751C.3030903@ssi.gouv.fr>

I installed bro using the following commands:

./configure --prefix=/usr/local/bro/
make
sudo make install

Then, bro is started using broctl install, followed by broctl start.

I have only one version of bro installed, the stable version 2.5
(Released Nov 16, 2016).

I was referring to the environment variables such as PATH, BROPATH or
CLUSTER_NODE, contained in ${PREFIX}/spool/worker-X/.env_vars, and
generated by the wrapper script ${PREFIX}/share/broctl/scripts/run-bro.

I forgot to mention that without using the native PF_RING plugin, I am
able to use PF_RING with the dedicated libpcap, such as explained here:
https://www.bro.org/sphinx/configuration/index.html#pf-ring-cluster-configuration.


Therefore, I don't think that this issue is related to the pf_ring
network driver or something like that, but rather to bro or broctl that
does not set the right configuration to enable the plugin.

R?mi
> How did you install bro?  Do you have more than one version of bro installed?
> 
> What environment variables are you referring to?


From gfaulkner.nsm at gmail.com  Mon Feb 27 11:19:31 2017
From: gfaulkner.nsm at gmail.com (Gary Faulkner)
Date: Mon, 27 Feb 2017 13:19:31 -0600
Subject: [Bro] Using native PF_RING plugin with broctl
In-Reply-To: <58B4751C.3030903@ssi.gouv.fr>
References: <58B409A6.3010805@ssi.gouv.fr>
	<FFC279B5-C7AF-4519-8EBF-E60735DCCBCD@illinois.edu>
	<58B4751C.3030903@ssi.gouv.fr>
Message-ID: <33024fd3-7373-568c-4cbc-a0e9524faba9@gmail.com>

When you built Bro did you also configure/make/make install the pf_ring 
plugin? My recollection is that the plugins are not automatically built 
when you build bro. They should be in the 
<path-to-bro-source>/aux/plugins/ in the source tree. They typically 
install into <path-to-bro>/lib/bro/plugins/.

~Gary

On 2/27/17 12:51 PM, Jullian Remi wrote:
> I installed bro using the following commands:
>
> ./configure --prefix=/usr/local/bro/
> make
> sudo make install
>
> Then, bro is started using broctl install, followed by broctl start.
>
> I have only one version of bro installed, the stable version 2.5
> (Released Nov 16, 2016).
>
> I was referring to the environment variables such as PATH, BROPATH or
> CLUSTER_NODE, contained in ${PREFIX}/spool/worker-X/.env_vars, and
> generated by the wrapper script ${PREFIX}/share/broctl/scripts/run-bro.
>
> I forgot to mention that without using the native PF_RING plugin, I am
> able to use PF_RING with the dedicated libpcap, such as explained here:
> https://www.bro.org/sphinx/configuration/index.html#pf-ring-cluster-configuration.
>
>
> Therefore, I don't think that this issue is related to the pf_ring
> network driver or something like that, but rather to bro or broctl that
> does not set the right configuration to enable the plugin.
>
> R?mi
>> How did you install bro?  Do you have more than one version of bro installed?
>>
>> What environment variables are you referring to?
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


From mabuchan at gmail.com  Mon Feb 27 11:41:50 2017
From: mabuchan at gmail.com (Mark Buchanan)
Date: Mon, 27 Feb 2017 13:41:50 -0600
Subject: [Bro] Using native PF_RING plugin with broctl
In-Reply-To: <33024fd3-7373-568c-4cbc-a0e9524faba9@gmail.com>
References: <58B409A6.3010805@ssi.gouv.fr>
	<FFC279B5-C7AF-4519-8EBF-E60735DCCBCD@illinois.edu>
	<58B4751C.3030903@ssi.gouv.fr>
	<33024fd3-7373-568c-4cbc-a0e9524faba9@gmail.com>
Message-ID: <CANQDeBMmrzN035mM02N7CkxKLTMuWiiU79WLejSrBG8W4K50=Q@mail.gmail.com>

There is a glitch that Justin and I worked through a few weeks ago with the
PF_RING (native) support in Bro 2.5.  Try adding these two items to your
/opt/bro/etc/broctl.conf file:

PFRINGClusterID = 21
PFRINGClusterType = 6-tuple

The issue is there is some broken login in a file that defaults the
ClusterID to zero (0) and at the same time that doesn't split the packets
out of the PF_RING interface as it should, which results in all workers
getting all packets (as you described).  So long as the PFRINGClusterID is
anything aside from zero (0), I believe it will fix the issue.  I add the
ClusterType just as a safety net to ensure you get decent distribution of
packets.  Other modes (5-tuple, 4-tuple or 2-tuple) should also be valid.
FYI - the 6th tuple is VLAN, so it may be more benificial to use 5-tuple in
some environments.

If those items are in  your broctl.conf file, then I'm a bit lost, but I've
been fighting with PF_RING over the past few weeks and this has allowed for
a repeatable process for myself.

This is also assuming you use in the node.cfg:
interface = eth0
lb_method = pf_ring

Additionally, the PF_RING module (new in 2.5) I believe suffers from the
same glitch or possibly another one.  I know roughly how to fix it, but
have time to validate and push back upstream.

Mark

On Mon, Feb 27, 2017 at 1:19 PM, Gary Faulkner <gfaulkner.nsm at gmail.com>
wrote:

> When you built Bro did you also configure/make/make install the pf_ring
> plugin? My recollection is that the plugins are not automatically built
> when you build bro. They should be in the
> <path-to-bro-source>/aux/plugins/ in the source tree. They typically
> install into <path-to-bro>/lib/bro/plugins/.
>
> ~Gary
>
> On 2/27/17 12:51 PM, Jullian Remi wrote:
> > I installed bro using the following commands:
> >
> > ./configure --prefix=/usr/local/bro/
> > make
> > sudo make install
> >
> > Then, bro is started using broctl install, followed by broctl start.
> >
> > I have only one version of bro installed, the stable version 2.5
> > (Released Nov 16, 2016).
> >
> > I was referring to the environment variables such as PATH, BROPATH or
> > CLUSTER_NODE, contained in ${PREFIX}/spool/worker-X/.env_vars, and
> > generated by the wrapper script ${PREFIX}/share/broctl/scripts/run-bro.
> >
> > I forgot to mention that without using the native PF_RING plugin, I am
> > able to use PF_RING with the dedicated libpcap, such as explained here:
> > https://www.bro.org/sphinx/configuration/index.html#pf-
> ring-cluster-configuration.
> >
> >
> > Therefore, I don't think that this issue is related to the pf_ring
> > network driver or something like that, but rather to bro or broctl that
> > does not set the right configuration to enable the plugin.
> >
> > R?mi
> >> How did you install bro?  Do you have more than one version of bro
> installed?
> >>
> >> What environment variables are you referring to?
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>



-- 
Mark Buchanan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170227/81edc7f2/attachment.html 

From seth at icir.org  Mon Feb 27 11:47:35 2017
From: seth at icir.org (Seth Hall)
Date: Mon, 27 Feb 2017 14:47:35 -0500
Subject: [Bro] Using native PF_RING plugin with broctl
In-Reply-To: <33024fd3-7373-568c-4cbc-a0e9524faba9@gmail.com>
References: <58B409A6.3010805@ssi.gouv.fr>
	<FFC279B5-C7AF-4519-8EBF-E60735DCCBCD@illinois.edu>
	<58B4751C.3030903@ssi.gouv.fr>
	<33024fd3-7373-568c-4cbc-a0e9524faba9@gmail.com>
Message-ID: <C76F389F-ACBA-46C8-A190-871257C8773F@icir.org>


> On Feb 27, 2017, at 2:19 PM, Gary Faulkner <gfaulkner.nsm at gmail.com> wrote:
> 
> When you built Bro did you also configure/make/make install the pf_ring 
> plugin? My recollection is that the plugins are not automatically built 
> when you build bro. They should be in the 
> <path-to-bro-source>/aux/plugins/ in the source tree. They typically 
> install into <path-to-bro>/lib/bro/plugins/.

Are there people out there that are using the pf_ring plugin to successfully load balance traffic?  I just checked the source to that plugin and I don't see where it sets up a load balanced ring. (I haven't worked on this plugin at all)

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From jlay at slave-tothe-box.net  Mon Feb 27 12:49:52 2017
From: jlay at slave-tothe-box.net (James Lay)
Date: Mon, 27 Feb 2017 13:49:52 -0700
Subject: [Bro] Using native PF_RING plugin with broctl
In-Reply-To: <C76F389F-ACBA-46C8-A190-871257C8773F@icir.org>
References: <58B409A6.3010805@ssi.gouv.fr>
	<FFC279B5-C7AF-4519-8EBF-E60735DCCBCD@illinois.edu>
	<58B4751C.3030903@ssi.gouv.fr>
	<33024fd3-7373-568c-4cbc-a0e9524faba9@gmail.com>
	<C76F389F-ACBA-46C8-A190-871257C8773F@icir.org>
Message-ID: <badb0d961164b9d004d94178ae871ecf@localhost>

On 2017-02-27 12:47, Seth Hall wrote:
>> On Feb 27, 2017, at 2:19 PM, Gary Faulkner <gfaulkner.nsm at gmail.com> 
>> wrote:
>> 
>> When you built Bro did you also configure/make/make install the 
>> pf_ring
>> plugin? My recollection is that the plugins are not automatically 
>> built
>> when you build bro. They should be in the
>> <path-to-bro-source>/aux/plugins/ in the source tree. They typically
>> install into <path-to-bro>/lib/bro/plugins/.
> 
> Are there people out there that are using the pf_ring plugin to
> successfully load balance traffic?  I just checked the source to that
> plugin and I don't see where it sets up a load balanced ring. (I
> haven't worked on this plugin at all)
> 
>   .Seth
> 
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
> 
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


 From my install notes:
###pfring
git clone https://github.com/ntop/PF_RING.git

cd PF_RING/kernel
make
sudo make install

cd PF_RING/userland/lib
./configure --prefix=/opt/pfring
make
sudo make install

cd ../libpcap
./configure --prefix=/opt/pfring
make
sudo make install

cd ../tcpdump
./configure --prefix=/opt/pfring
make
sudo make install

modprobe pf_ring enable_tx_capture=1 min_num_slots=32768
####

###bro
./configure --prefix=/opt/bro --with-pcap=/opt/pfring

pf_ring plugin
cd aux/plugins/pf_ring
./configure --bro-dist=../../.. --with-pfring=/opt/pfring 
--install-root=/opt/bro/lib/bro/plugins
sudo make install
####

1 worker from node.cfg:

[worker-1]
type=worker
host=localhost
interface=enp132s0
lb_method=pf_ring
lb_procs=6
pin_cpus=1,3,5,7,9,11


/proc/net/pf_ring/info:

PF_RING Version          : 6.5.0 
(dev:ab85fa090f7e1a9a66478815b82db6fa001a5b29)
Total rings              : 9

Standard (non ZC) Options
Ring slots               : 4096
Slot version             : 16
Capture TX               : Yes [RX+TX]
IP Defragment            : No
Socket Mode              : Standard
Total plugins            : 0
Cluster Fragment Queue   : 19736
Cluster Fragment Discard : 0


James

From jazoff at illinois.edu  Mon Feb 27 13:03:22 2017
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Mon, 27 Feb 2017 21:03:22 +0000
Subject: [Bro] Using native PF_RING plugin with broctl
In-Reply-To: <C76F389F-ACBA-46C8-A190-871257C8773F@icir.org>
References: <58B409A6.3010805@ssi.gouv.fr>
	<FFC279B5-C7AF-4519-8EBF-E60735DCCBCD@illinois.edu>
	<58B4751C.3030903@ssi.gouv.fr>
	<33024fd3-7373-568c-4cbc-a0e9524faba9@gmail.com>
	<C76F389F-ACBA-46C8-A190-871257C8773F@icir.org>
Message-ID: <904EEDCA-1FFB-4F09-9207-0870545E5604@illinois.edu>


> On Feb 27, 2017, at 2:47 PM, Seth Hall <seth at icir.org> wrote:
> 
> 
>> On Feb 27, 2017, at 2:19 PM, Gary Faulkner <gfaulkner.nsm at gmail.com> wrote:
>> 
>> When you built Bro did you also configure/make/make install the pf_ring 
>> plugin? My recollection is that the plugins are not automatically built 
>> when you build bro. They should be in the 
>> <path-to-bro-source>/aux/plugins/ in the source tree. They typically 
>> install into <path-to-bro>/lib/bro/plugins/.
> 
> Are there people out there that are using the pf_ring plugin to successfully load balance traffic?  I just checked the source to that plugin and I don't see where it sets up a load balanced ring. (I haven't worked on this plugin at all)

I can see from this thread that a number of people think they are using the plugin, but are not actually using it.

interface = eth0          # pf_ring libpcap wrapper
interface = pf_ring::eth0 # native bro pf_ring plugin


-- 
- Justin Azoff



From jlay at slave-tothe-box.net  Mon Feb 27 13:12:00 2017
From: jlay at slave-tothe-box.net (James Lay)
Date: Mon, 27 Feb 2017 14:12:00 -0700
Subject: [Bro] Using native PF_RING plugin with broctl
In-Reply-To: <904EEDCA-1FFB-4F09-9207-0870545E5604@illinois.edu>
References: <58B409A6.3010805@ssi.gouv.fr>
	<FFC279B5-C7AF-4519-8EBF-E60735DCCBCD@illinois.edu>
	<58B4751C.3030903@ssi.gouv.fr>
	<33024fd3-7373-568c-4cbc-a0e9524faba9@gmail.com>
	<C76F389F-ACBA-46C8-A190-871257C8773F@icir.org>
	<904EEDCA-1FFB-4F09-9207-0870545E5604@illinois.edu>
Message-ID: <f0ee7b38e06dc15fc4838b41850219b9@localhost>

On 2017-02-27 14:03, Azoff, Justin S wrote:
>> On Feb 27, 2017, at 2:47 PM, Seth Hall <seth at icir.org> wrote:
>> 
>> 
>>> On Feb 27, 2017, at 2:19 PM, Gary Faulkner <gfaulkner.nsm at gmail.com> 
>>> wrote:
>>> 
>>> When you built Bro did you also configure/make/make install the 
>>> pf_ring
>>> plugin? My recollection is that the plugins are not automatically 
>>> built
>>> when you build bro. They should be in the
>>> <path-to-bro-source>/aux/plugins/ in the source tree. They typically
>>> install into <path-to-bro>/lib/bro/plugins/.
>> 
>> Are there people out there that are using the pf_ring plugin to 
>> successfully load balance traffic?  I just checked the source to that 
>> plugin and I don't see where it sets up a load balanced ring. (I 
>> haven't worked on this plugin at all)
> 
> I can see from this thread that a number of people think they are
> using the plugin, but are not actually using it.
> 
> interface = eth0          # pf_ring libpcap wrapper
> interface = pf_ring::eth0 # native bro pf_ring plugin


I believe that's because I got the info from here:

https://www.bro.org/documentation/load-balancing.html

Maybe add info from 
https://www.bro.org/sphinx/components/bro-plugins/pf_ring/README.html to 
the above?  I will try out the native on next device reboot.

James

From remi.jullian at ssi.gouv.fr  Tue Feb 28 01:31:18 2017
From: remi.jullian at ssi.gouv.fr (Jullian Remi)
Date: Tue, 28 Feb 2017 10:31:18 +0100
Subject: [Bro] Using native PF_RING plugin with broctl
In-Reply-To: <904EEDCA-1FFB-4F09-9207-0870545E5604@illinois.edu>
References: <58B409A6.3010805@ssi.gouv.fr>
	<FFC279B5-C7AF-4519-8EBF-E60735DCCBCD@illinois.edu>
	<58B4751C.3030903@ssi.gouv.fr>
	<33024fd3-7373-568c-4cbc-a0e9524faba9@gmail.com>
	<C76F389F-ACBA-46C8-A190-871257C8773F@icir.org>
	<904EEDCA-1FFB-4F09-9207-0870545E5604@illinois.edu>
Message-ID: <58B54366.7030203@ssi.gouv.fr>

> 
>> On Feb 27, 2017, at 2:47 PM, Seth Hall <seth at icir.org> wrote:
>>
>>
>>> On Feb 27, 2017, at 2:19 PM, Gary Faulkner <gfaulkner.nsm at gmail.com> wrote:
>>>
>>> When you built Bro did you also configure/make/make install the pf_ring 
>>> plugin? My recollection is that the plugins are not automatically built 
>>> when you build bro. They should be in the 
>>> <path-to-bro-source>/aux/plugins/ in the source tree. They typically 
>>> install into <path-to-bro>/lib/bro/plugins/.
>>
>> Are there people out there that are using the pf_ring plugin to successfully load balance traffic?  I just checked the source to that plugin and I don't see where it sets up a load balanced ring. (I haven't worked on this plugin at all)
> 
> I can see from this thread that a number of people think they are using the plugin, but are not actually using it.
> 
> interface = eth0          # pf_ring libpcap wrapper
> interface = pf_ring::eth0 # native bro pf_ring plugin
> 
> 

Indeed, this is what I try to underline with this thread, I also believe
there is a glitch with the native PF_RING plugin.

I think that the example pointed by James Lay is using PF_RING through
the libpcap, but NOT with the native plugin. It can be proved by
breaking within Source.cc:192, the PcapSource::ExtractNextPacket() and
the underlaying pcap_next() function, should never be called, if the
plugin is properly used.

I would suggest using a libpcap compiled without PF_RING support, to
avoid confusion. This is actually how I test the plugin.

From shirkdog.bsd at gmail.com  Tue Feb 28 05:40:26 2017
From: shirkdog.bsd at gmail.com (Michael Shirk)
Date: Tue, 28 Feb 2017 08:40:26 -0500
Subject: [Bro] Using native PF_RING plugin with broctl
In-Reply-To: <58B54366.7030203@ssi.gouv.fr>
References: <58B409A6.3010805@ssi.gouv.fr>
	<FFC279B5-C7AF-4519-8EBF-E60735DCCBCD@illinois.edu>
	<58B4751C.3030903@ssi.gouv.fr>
	<33024fd3-7373-568c-4cbc-a0e9524faba9@gmail.com>
	<C76F389F-ACBA-46C8-A190-871257C8773F@icir.org>
	<904EEDCA-1FFB-4F09-9207-0870545E5604@illinois.edu>
	<58B54366.7030203@ssi.gouv.fr>
Message-ID: <CAL8PkUW2zNhMW=cmh93L-Y=F=PYVvYtnaFYdgSMC4JLN2XUxtQ@mail.gmail.com>

I vote for updated documentation for the new plugin. What James posted is
how I would set it up, which would use pf_ring enabled libpcap.

--
Michael Shirk
Daemon Security, Inc.
http://www.daemon-security.com

On Feb 28, 2017 4:33 AM, "Jullian Remi" <remi.jullian at ssi.gouv.fr> wrote:

> >
> >> On Feb 27, 2017, at 2:47 PM, Seth Hall <seth at icir.org> wrote:
> >>
> >>
> >>> On Feb 27, 2017, at 2:19 PM, Gary Faulkner <gfaulkner.nsm at gmail.com>
> wrote:
> >>>
> >>> When you built Bro did you also configure/make/make install the pf_ring
> >>> plugin? My recollection is that the plugins are not automatically built
> >>> when you build bro. They should be in the
> >>> <path-to-bro-source>/aux/plugins/ in the source tree. They typically
> >>> install into <path-to-bro>/lib/bro/plugins/.
> >>
> >> Are there people out there that are using the pf_ring plugin to
> successfully load balance traffic?  I just checked the source to that
> plugin and I don't see where it sets up a load balanced ring. (I haven't
> worked on this plugin at all)
> >
> > I can see from this thread that a number of people think they are using
> the plugin, but are not actually using it.
> >
> > interface = eth0          # pf_ring libpcap wrapper
> > interface = pf_ring::eth0 # native bro pf_ring plugin
> >
> >
>
> Indeed, this is what I try to underline with this thread, I also believe
> there is a glitch with the native PF_RING plugin.
>
> I think that the example pointed by James Lay is using PF_RING through
> the libpcap, but NOT with the native plugin. It can be proved by
> breaking within Source.cc:192, the PcapSource::ExtractNextPacket() and
> the underlaying pcap_next() function, should never be called, if the
> plugin is properly used.
>
> I would suggest using a libpcap compiled without PF_RING support, to
> avoid confusion. This is actually how I test the plugin.
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170228/4f07b717/attachment.html 

From seth at icir.org  Tue Feb 28 06:17:09 2017
From: seth at icir.org (Seth Hall)
Date: Tue, 28 Feb 2017 09:17:09 -0500
Subject: [Bro] Using native PF_RING plugin with broctl
In-Reply-To: <CACX0rUQE+RzgbHbzWUR05eLnd9xfxQKzfCgfjuPKKL7Z007riw@mail.gmail.com>
References: <CACX0rUQE+RzgbHbzWUR05eLnd9xfxQKzfCgfjuPKKL7Z007riw@mail.gmail.com>
Message-ID: <49A56BD5-77D0-4716-8D2B-2E9865F534D1@icir.org>


> On Feb 27, 2017, at 5:54 PM, fatema bannatwala <fatema.bannatwala at gmail.com> wrote:
> 
> When I configured and installed bro from source, I did:
> $./configure --prefix=/usr/local/bro/2.5 --with-pcap=/usr/local/pfring/5.6.2

Yep, you are using the libpcap wrapper here, which is currently the only way to do clustered load balancing with PF_Ring unless you do that tiny change that Mark pointed out a minute ago.  To get that more tightly integrated and configurable with broctl would take a bit more work, but as a hack that tiny change would work.

You can tell in your node.cfg if you are using the libpcap wrapper or the plugin by the interface name.  If you have use an interface name like: pf_ring::eth1, then you are using the plugin and load balancing won't work.  If you are just using an interface name like eth1 and lb_method=pf_ring, then you will be using the libpcap wrapper.  

When the pf_ring developers contributed the pf_ring plugin, it seems that they didn't do full integration with the deployment method.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From loris.leiva at gmail.com  Tue Feb 28 06:48:52 2017
From: loris.leiva at gmail.com (Loris Leiva)
Date: Tue, 28 Feb 2017 15:48:52 +0100
Subject: [Bro] SSH brute-force email notice
Message-ID: <4b0d7b09-5230-4b67-b965-d1a7cf566313@Spark>

Hello,

First of all, I am really grateful for Bro and its easy scripting. I have been using Bro in the context of my master thesis and had lots of fun using it.

I am contacting you today as I have encountered a problem that none of my google researching skills could solve. Let me try and describe it clearly.

What I am trying to acheive:
I am using the pcap file available at?https://www.bro.org/static/traces/ssh.pcap?to simulate a SSH::Password_Guessing notice using the command `broctl process`. My goal is simply to make Bro send me an email when such a notice is raised.

What is going wrong:
Even though the notice is raised, I do not receive any emails.

Hypothesis to eliminate:
- First of all, my broctl.cfg file is configured correctly and, if I raise a random notice in the `bro_init()` event, I successfully receive the email.
- I am also sure that the notice is being raise properly as a `notice.log` file gets generated with the relevant notice containing the `Notice::ACTION_EMAIL` action. I even hard-coded a print inside the module that raise the notice to make sure that this part of the code was run.

What I have tried:
- redefining?Notice::emailed_types
- redefining?Notice::alarmed_types
- adding a?Notice::policy hook containing `add n$actions[Notice::ACTION_EMAIL];`

I hope that my problem description helps. I am really struggling to understand this behaviour and cannot find similar problems online.

Please do not hesitate to contact me should you need additional information.

Thank you in advance for your support,

Best regards,
Loris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170228/d10ffa57/attachment-0001.html 

From jazoff at illinois.edu  Tue Feb 28 07:01:11 2017
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Tue, 28 Feb 2017 15:01:11 +0000
Subject: [Bro] SSH brute-force email notice
In-Reply-To: <4b0d7b09-5230-4b67-b965-d1a7cf566313@Spark>
References: <4b0d7b09-5230-4b67-b965-d1a7cf566313@Spark>
Message-ID: <3F857EE9-2C08-4ABE-8242-4615AD924BF1@illinois.edu>


> On Feb 28, 2017, at 9:48 AM, Loris Leiva <loris.leiva at gmail.com> wrote:

> What is going wrong:
> Even though the notice is raised, I do not receive any emails.
> 
> Hypothesis to eliminate:
> - First of all, my broctl.cfg file is configured correctly and, if I raise a random notice in the `bro_init()` event, I successfully receive the email.
> - I am also sure that the notice is being raise properly as a `notice.log` file gets generated with the relevant notice containing the `Notice::ACTION_EMAIL` action. I even hard-coded a print inside the module that raise the notice to make sure that this part of the code was run.

If your notice.log mentioned ACTION_EMAIL but you did not get an email then you need to look at the bro stderr log and the mail log(/var/log/mail or such) for your machine.

-- 
- Justin Azoff



From jlay at slave-tothe-box.net  Tue Feb 28 07:08:27 2017
From: jlay at slave-tothe-box.net (James Lay)
Date: Tue, 28 Feb 2017 08:08:27 -0700
Subject: [Bro] SSH brute-force email notice
In-Reply-To: <4b0d7b09-5230-4b67-b965-d1a7cf566313@Spark>
References: <4b0d7b09-5230-4b67-b965-d1a7cf566313@Spark>
Message-ID: <4757404c7c304f32ac7eaf49b59eb674@localhost>

On 2017-02-28 07:48, Loris Leiva wrote:
> Hello,
> 
> First of all, I am really grateful for Bro and its easy scripting. I
> have been using Bro in the context of my master thesis and had lots of
> fun using it.
> 
> I am contacting you today as I have encountered a problem that none of
> my google researching skills could solve. Let me try and describe it
> clearly.
> 
> WHAT I AM TRYING TO ACHEIVE:
> I am using the pcap file available at
> https://www.bro.org/static/traces/ssh.pcap [1] to simulate a
> SSH::Password_Guessing notice using the command `broctl process`. My
> goal is simply to make Bro send me an email when such a notice is
> raised.
> 
> WHAT IS GOING WRONG:
> Even though the notice is raised, I do not receive any emails.
> 
> HYPOTHESIS TO ELIMINATE:
> - First of all, my broctl.cfg file is configured correctly and, if I
> raise a random notice in the `bro_init()` event, I successfully
> receive the email.
> - I am also sure that the notice is being raise properly as a
> `notice.log` file gets generated with the relevant notice containing
> the `Notice::ACTION_EMAIL` action. I even hard-coded a print inside
> the module that raise the notice to make sure that this part of the
> code was run.
> 
> WHAT I HAVE TRIED:
> - redefining Notice::emailed_types
> - redefining Notice::alarmed_types
> - adding a Notice::policy hook containing `add
> n$actions[Notice::ACTION_EMAIL];`
> 
> I hope that my problem description helps. I am really struggling to
> understand this behaviour and cannot find similar problems online.
> 
> Please do not hesitate to contact me should you need additional
> information.
> 
> Thank you in advance for your support,
> 
> Best regards,
> Loris
> 
> Links:
> ------
> [1] https://www.bro.org/static/traces/ssh.pcap
> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro


There is sample code here:

https://www.bro.org/sphinx/scripting/index.html#raising-notices

But, I myself would love to see a full example...it's one of the things 
that often confuses me about bro; most of the time there's snippets of 
code in the documentation, but I struggle to find a full on 
setup...maybe I'm missing something.  If someone can post something step 
by step and what to add where I'll add it to my brofaq here:

https://github.com/DigiAngel/brofaq

James

From mabuchan at gmail.com  Tue Feb 28 07:10:13 2017
From: mabuchan at gmail.com (Mark Buchanan)
Date: Tue, 28 Feb 2017 09:10:13 -0600
Subject: [Bro] Using native PF_RING plugin with broctl
In-Reply-To: <49A56BD5-77D0-4716-8D2B-2E9865F534D1@icir.org>
References: <CACX0rUQE+RzgbHbzWUR05eLnd9xfxQKzfCgfjuPKKL7Z007riw@mail.gmail.com>
	<49A56BD5-77D0-4716-8D2B-2E9865F534D1@icir.org>
Message-ID: <CANQDeBPZhL08yY8O1snJ0c85-yj3LkjLEXqYbT6a6uLZRgmLGA@mail.gmail.com>

Sorry, Seth, I didn't send this to all and should have.  Here was the email
that Seth was referring to.

I'll have to check later, but I was able to make the plugin work with some
source mods, I think.  I think I tested and was unable to get it to work
natively using the broctl.conf changes that I sent to the list a few
minutes ago.

I know there was some glitch that didn't make it work out of the box, but I
was able to modify the plugin and get it to work - just don't have that
code where I am now.

Ok - found the edits I made.  Here are the diffs - however I have commented
out the second (and most necessary piece of code):

diff bro-2.5*/aux/plugins/pf_ring/src/PF_RING.cc
40a41,48
>       char app_name[4] = "bro";
>
>         if ( pfring_set_application_name(pd, app_name) != 0 )
>               {
>               Error(errno ? strerror(errno) : "unable to set app name");
>               return;
>               }
>
48a57,68
>       /* Set default cluster type */
> /*    u_int clusterId = 1;
>       cluster_type cluster_hash_type = cluster_per_flow_5_tuple;
>
>       if ( pfring_set_cluster(pd, clusterId, cluster_hash_type) != 0 )
>               {
>               Error(errno ? strerror(errno) : "unable to set cluster
mode");
>               pfring_close(pd);
>               pd = NULL;
>               return;
>               }
>  */

The issue related back to not having the cluster hash set and I believe it
wouldn't fire.  Additionally the first snippet of code sets the app name,
so when you cat /proc/net/pf_ring/<pid>.* you get "bro" out of it.

To note, I'm not a coder, so I'm happy I made it this far.  There was some
question if the cluster_type was necessary if you had the broctl.conf items
in the the file - but I can't remember the outcome of the test (but I
believe the comment out I have of the cluster type was me testing it.

Does that help?  I know I was able to get the module to work, but I think I
had to include the above items to make it work.  Additionally, there was
some glitch that omited the PFRINGClusterID from teh broctl.conf due to a
FreeBSD bug that said if PF_RING isn't needed, don't put that in there.

Mark

On Tue, Feb 28, 2017 at 8:17 AM, Seth Hall <seth at icir.org> wrote:

>
> > On Feb 27, 2017, at 5:54 PM, fatema bannatwala <
> fatema.bannatwala at gmail.com> wrote:
> >
> > When I configured and installed bro from source, I did:
> > $./configure --prefix=/usr/local/bro/2.5 --with-pcap=/usr/local/pfring/
> 5.6.2
>
> Yep, you are using the libpcap wrapper here, which is currently the only
> way to do clustered load balancing with PF_Ring unless you do that tiny
> change that Mark pointed out a minute ago.  To get that more tightly
> integrated and configurable with broctl would take a bit more work, but as
> a hack that tiny change would work.
>
> You can tell in your node.cfg if you are using the libpcap wrapper or the
> plugin by the interface name.  If you have use an interface name like:
> pf_ring::eth1, then you are using the plugin and load balancing won't
> work.  If you are just using an interface name like eth1 and
> lb_method=pf_ring, then you will be using the libpcap wrapper.
>
> When the pf_ring developers contributed the pf_ring plugin, it seems that
> they didn't do full integration with the deployment method.
>
>   .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
>
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>



-- 
Mark Buchanan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170228/c2230bd3/attachment.html 

From loris.leiva at gmail.com  Tue Feb 28 08:17:17 2017
From: loris.leiva at gmail.com (Loris Leiva)
Date: Tue, 28 Feb 2017 17:17:17 +0100
Subject: [Bro] SSH brute-force email notice
In-Reply-To: <3F857EE9-2C08-4ABE-8242-4615AD924BF1@illinois.edu>
References: <4b0d7b09-5230-4b67-b965-d1a7cf566313@Spark>
	<3F857EE9-2C08-4ABE-8242-4615AD924BF1@illinois.edu>
Message-ID: <642b841f-b9ff-4583-9274-096402cb225c@Spark>

Thank you for your answer.

I have checked the logs during my scenario and when the email doesn?t send nothing get logs at all (not even on the bro stderr log). However, when I raise a dummy notice in a bro_init() event, then I receive the email and the email gets logged properly.

Note that I am using macOS Sierra so I access my logs through the following command `log stream --predicate ?'(process == "smtpd") || (process == "smtp")' -info`.

Any idea of what could be the problem ?

Thanks again,
Loris

On 28 Feb 2017, 16:01 +0100, Azoff, Justin S <jazoff at illinois.edu>, wrote:
>
> > On Feb 28, 2017, at 9:48 AM, Loris Leiva <loris.leiva at gmail.com> wrote:
>
> > What is going wrong:
> > Even though the notice is raised, I do not receive any emails.
> >
> > Hypothesis to eliminate:
> > - First of all, my broctl.cfg file is configured correctly and, if I raise a random notice in the `bro_init()` event, I successfully receive the email.
> > - I am also sure that the notice is being raise properly as a `notice.log` file gets generated with the relevant notice containing the `Notice::ACTION_EMAIL` action. I even hard-coded a print inside the module that raise the notice to make sure that this part of the code was run.
>
> If your notice.log mentioned ACTION_EMAIL but you did not get an email then you need to look at the bro stderr log and the mail log(/var/log/mail or such) for your machine.
>
> --
> - Justin Azoff
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170228/ef97c9fe/attachment.html 

From jazoff at illinois.edu  Tue Feb 28 08:24:37 2017
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Tue, 28 Feb 2017 16:24:37 +0000
Subject: [Bro] SSH brute-force email notice
In-Reply-To: <642b841f-b9ff-4583-9274-096402cb225c@Spark>
References: <4b0d7b09-5230-4b67-b965-d1a7cf566313@Spark>
	<3F857EE9-2C08-4ABE-8242-4615AD924BF1@illinois.edu>
	<642b841f-b9ff-4583-9274-096402cb225c@Spark>
Message-ID: <404DA38B-407D-4C26-AE15-4D120D89075E@illinois.edu>


> On Feb 28, 2017, at 11:17 AM, Loris Leiva <loris.leiva at gmail.com> wrote:
> 
> Thank you for your answer.
> 
> I have checked the logs during my scenario and when the email doesn?t send nothing get logs at all (not even on the bro stderr log). However, when I raise a dummy notice in a bro_init() event, then I receive the email and the email gets logged properly.

Nothing gets logged at all? not even to notice.log?

> 
> Note that I am using macOS Sierra so I access my logs through the following command `log stream --predicate  '(process == "smtpd") || (process == "smtp")' -info`.
> 
> Any idea of what could be the problem ?
> 
> Thanks again,
> Loris

-- 
- Justin Azoff



From loris.leiva at gmail.com  Tue Feb 28 08:25:59 2017
From: loris.leiva at gmail.com (Loris Leiva)
Date: Tue, 28 Feb 2017 17:25:59 +0100
Subject: [Bro] SSH brute-force email notice
In-Reply-To: <404DA38B-407D-4C26-AE15-4D120D89075E@illinois.edu>
References: <4b0d7b09-5230-4b67-b965-d1a7cf566313@Spark>
	<3F857EE9-2C08-4ABE-8242-4615AD924BF1@illinois.edu>
	<642b841f-b9ff-4583-9274-096402cb225c@Spark>
	<404DA38B-407D-4C26-AE15-4D120D89075E@illinois.edu>
Message-ID: <6f8abe5b-9417-46bb-8b5a-39249755a6db@Spark>

Yes sorry I meant no errors get logs but weirdly I still get my notice.log entry with Notice::ACTION_EMAIL in it.

On 28 Feb 2017, 17:24 +0100, Azoff, Justin S <jazoff at illinois.edu>, wrote:
>
> > On Feb 28, 2017, at 11:17 AM, Loris Leiva <loris.leiva at gmail.com> wrote:
> >
> > Thank you for your answer.
> >
> > I have checked the logs during my scenario and when the email doesn?t send nothing get logs at all (not even on the bro stderr log). However, when I raise a dummy notice in a bro_init() event, then I receive the email and the email gets logged properly.
>
> Nothing gets logged at all? not even to notice.log?
>
> >
> > Note that I am using macOS Sierra so I access my logs through the following command `log stream --predicate '(process == "smtpd") || (process == "smtp")' -info`.
> >
> > Any idea of what could be the problem ?
> >
> > Thanks again,
> > Loris
>
> --
> - Justin Azoff
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170228/92178127/attachment.html 

From loris.leiva at gmail.com  Tue Feb 28 09:31:09 2017
From: loris.leiva at gmail.com (Loris Leiva)
Date: Tue, 28 Feb 2017 18:31:09 +0100
Subject: [Bro] SSH brute-force email notice
In-Reply-To: <6f8abe5b-9417-46bb-8b5a-39249755a6db@Spark>
References: <4b0d7b09-5230-4b67-b965-d1a7cf566313@Spark>
	<3F857EE9-2C08-4ABE-8242-4615AD924BF1@illinois.edu>
	<642b841f-b9ff-4583-9274-096402cb225c@Spark>
	<404DA38B-407D-4C26-AE15-4D120D89075E@illinois.edu>
	<6f8abe5b-9417-46bb-8b5a-39249755a6db@Spark>
Message-ID: <eeae5267-bc40-4cd8-93f0-0151c96d9329@Spark>

So I have done more tests and here are my findings.

First of all, I added the Notice::policy hook so that every notice gets sent via email (in order not to worry about accepting the right type of notice). Then I tried different ways to launch the `ssh.pcap` with bro.

- `broctl process ssh.pcap -C` => outputs the notice with ACTION_EMAIL but does not send the email (no stderr.log nor email log).
- `bro -r ssh.pcap local "Site::local_nets += { ? }"?-C` => same output, no emails.

However, if I just start bro via broctl and let it run, I start receiving random notices via emails regarding my laptop?s connections. I haven?t been able to reproduce a SSH brute-force attack but I assume it would work that way.

So I am starting to wonder if the commands `bro` and `broctl process` are actually able to send emails. Any ideas on that ?

Thanks in advance for your help,
Loris



On 28 Feb 2017, 17:27 +0100, Loris Leiva <loris.leiva at gmail.com>, wrote:
> Yes sorry I meant no errors get logs but weirdly I still get my notice.log entry with Notice::ACTION_EMAIL in it.
>
> On 28 Feb 2017, 17:24 +0100, Azoff, Justin S <jazoff at illinois.edu>, wrote:
> >
> > > On Feb 28, 2017, at 11:17 AM, Loris Leiva <loris.leiva at gmail.com> wrote:
> > >
> > > Thank you for your answer.
> > >
> > > I have checked the logs during my scenario and when the email doesn?t send nothing get logs at all (not even on the bro stderr log). However, when I raise a dummy notice in a bro_init() event, then I receive the email and the email gets logged properly.
> >
> > Nothing gets logged at all? not even to notice.log?
> >
> > >
> > > Note that I am using macOS Sierra so I access my logs through the following command `log stream --predicate '(process == "smtpd") || (process == "smtp")' -info`.
> > >
> > > Any idea of what could be the problem ?
> > >
> > > Thanks again,
> > > Loris
> >
> > --
> > - Justin Azoff
> >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170228/769f15c3/attachment.html 

From espressobeanies at gmail.com  Tue Feb 28 12:20:29 2017
From: espressobeanies at gmail.com (Espresso Beanies)
Date: Tue, 28 Feb 2017 15:20:29 -0500
Subject: [Bro] Issue with Bro reporting dropped packets
Message-ID: <CAOeD169D5c6pB4-VhnAUVpxcLXoibpq5b_+n1TO+_8z2DtE6tQ@mail.gmail.com>

Hi,

I'm trying to troubleshoot a Bro IDS that is experiencing capture loss with
dropped packets. The machine I'm using has a 16-core Intel Xeon processor,
96Gb RAM, and an Intel NIC. I have 3 Bro workers with CPU affinity enabled
and I'm using the pf_ring module on CentOS with no custom Bro scripts
running. All of my processors are running at 99% utilization.

According to my operating system, I'm dropping about 8000 packets over the
course of a day on a 300-400Mbps network. According to Bro capstats, I am
dropping about the same number of packets I'm receiving, sometimes more
than I receive. My capture_loss.log shows my workers lose about 30-50%
packets and my manager and proxy, 70-90%. I can provide any configurations
or screenshots if necessary.

I'm trying to troubleshoot where the issue lies. I initially installed Bro
with all the recommended packages (tcmalloc, etc...) and the pf_ring module
and I can see that Bro is using it. At this point, everything I see is
pointing to an application issue and I'm running Bro version 2.5. I had the
same issue with Bro v.2.4 as well.

Short of tweaking OS kernel and NIC card settings, I'm not sure where else
I could try to reduce my packet drop count in Bro. Any recommendations?

Thanks,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170228/97e01bad/attachment.html 

From al.kefallonitis at gmail.com  Tue Feb 28 13:17:02 2017
From: al.kefallonitis at gmail.com (Alex Kefallonitis)
Date: Tue, 28 Feb 2017 23:17:02 +0200
Subject: [Bro] Bro + pf_ring on a rasberry pi 3
Message-ID: <CAHv=MupwZXT2W2JvYZbBNbZzqBJ6RHB5WeYm24Etk0NoLk5+cw@mail.gmail.com>

Hi all!

After successfully compiling pf_ring and enable module on a rpi 3 arm
kernel :

pi at raspberrypi:~ $ modinfo pf_ring && cat /proc/net/pf_ring/info
filename:       /lib/modules/4.4.34-v7+/kernel/net/pf_ring/pf_ring.ko
alias:          net-pf-27
description:    Packet capture acceleration and analysis
author:         ntop.org
license:        GPL
srcversion:     159AD63EACFCF3EFC835D09
depends:
vermagic:       4.4.34-v7 SMP mod_unload modversions ARMv7
parm:           min_num_slots:Min number of ring slots (uint)
parm:           perfect_rules_hash_size:Perfect rules hash size (uint)
parm:           transparent_mode:(deprecated) (uint)
parm:           enable_debug:Set to 1 to enable PF_RING debug tracing into
the syslog (uint)
parm:           enable_tx_capture:Set to 1 to capture outgoing packets
(uint)
parm:           enable_frag_coherence:Set to 1 to handle fragments (flow
coherence) in clusters (uint)
parm:           enable_ip_defrag:Set to 1 to enable IP defragmentation(only
rx traffic is defragmentead) (uint)
parm:           quick_mode:Set to 1 to run at full speed but with upto one
socket per interface (uint)
PF_RING Version          : 6.4.1 (unknown)
Total rings              : 2

Standard (non ZC) Options
Ring slots               : 32768
Slot version             : 16
Capture TX               : Yes [RX+TX]
IP Defragment            : No
Socket Mode              : Standard
Total plugins            : 0
Cluster Fragment Queue   : 0
Cluster Fragment Discard : 0


I compiled also successfully bro with pf_ring plugin. But there is a
problem...Although rpi interface "sees" network traffic as it is plugged on
a network mirror bridge and pf_ring compiled tcpdump output does full
network packet capture :


pi at raspberrypi:~/bro-test $ ifconfig
eth0      Link encap:Ethernet  HWaddr b8:27:eb:68:1a:49
          inet addr:10.0.0.31  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: fe80::18a4:4736:aeb7:94b7/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:5912 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1317 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:358436 (350.0 KiB)  TX bytes:166018 (162.1 KiB)


pi at raspberrypi:~/bro-test $ sudo /opt/pfring/sbin/tcpdump host not 10.0.0.31
[PF_RING] mmap() failed: try with a smaller snaplen
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:00:43.045119 IP 10.0.0.2.37630 > 10.0.0.3.9200: Flags [.], seq
2264223995:2264225443, ack 4236626719, win 1444, options [nop,nop,TS val
3506664 ecr 3496553], length 1448
21:00:43.045498 IP 10.0.0.2.37630 > 10.0.0.3.9200: Flags [.], seq
1448:2896, ack 1, win 1444, options [nop,nop,TS val 3506664 ecr 3496553],
length 1448
21:00:43.045500 IP 10.0.0.2.37630 > 10.0.0.3.9200: Flags [P.], seq
2896:4096, ack 1, win 1444, options [nop,nop,TS val 3506664 ecr 3496553],
length 1200
21:00:43.045502 IP 10.0.0.2.37630 > 10.0.0.3.9200: Flags [.], seq
4096:5544, ack 1, win 1444, options [nop,nop,TS val 3506664 ecr 3496553],
length 1448
21:00:43.046343 IP 10.0.0.2.37630 > 10.0.0.3.9200: Flags [.], seq
5544:6992, ack 1, win 1444, options [nop,nop,TS val 3506664 ecr 3496553],
length 1448
21:00:43.046344 IP 10.0.0.2.37630 > 10.0.0.3.9200: Flags [P.], seq
6992:7028, ack 1, win 1444, options [nop,nop,TS val 3506664 ecr 3496553],
length 36
21:00:43.046346 IP 10.0.0.3.9200 > 10.0.0.2.37630: Flags [.], ack 7028, win
1024, options [nop,nop,TS val 3496778 ecr 3506664], length 0
^C
7 packets captured
10 packets received by filter
3 packets dropped by kernel

When i start bro with pf_ring bro exports logs only for rpi self traffic
that is to say traffic from or to 10.0.0.31 ip:

pi at raspberrypi:~/bro-test $ sudo /opt/bro/bin/bro -i pf_ring::eth0
listening on eth0


1488315827.676782 616 packets received on interface eth0, 0 dropped

pi at raspberrypi:~/bro-test $ cat conn.log
#separator \x09
#set_separator    ,
#empty_field    (empty)
#unset_field    -
#path    conn
#open    2017-02-28-21-03-39
#fields    ts    uid    id.orig_h    id.orig_p    id.resp_h    id.resp_p
proto    service    duration    orig_bytes    resp_bytes    conn_state
local_orig    local_resp    missed_bytes    history    orig_pkts
orig_ip_bytes    resp_pkts    resp_ip_bytestunnel_parents
#types    time    string    addr    port    addr    port    enum
string    interval    count    count    string    bool    bool    count
stringcount    count    count    count    set[string]
1488315814.841388    Cn0COF2Dl2RGFtlsak    10.8.0.2    60414
10.0.0.31    22    tcp    -    -    -    -    OTH    --    0    ^c    0
0    0    0    (empty)
1488315826.472327    C0a8me4cjwn36bIpZ    10.8.0.2    60414    10.0.0.31
22    tcp    -    -    -    -    OTH    --    0    ^c    0    0    0
0    (empty)
#close    2017-02-28-21-03-47


There are no errors and no capture_loss or drop packets, although base bro
plugins are enable, bro sees only limited events:

pi at raspberrypi:~/bro-test $ ls -la
total 28
drwxr-xr-x  3 pi   pi   4096 Feb 28 21:03 .
drwxr-xr-x 12 pi   pi   4096 Feb 28 20:55 ..
-rw-r--r--  1 root root  699 Feb 28 21:03 conn.log
-rw-r--r--  1 root root  253 Feb 28 21:03 packet_filter.log
-rw-r--r--  1 root root  362 Feb 28 21:03 reporter.log
drwx------  3 root root 4096 Feb 28 21:03 .state
-rw-r--r--  1 root root  428 Feb 28 21:03 weird.log


On the contrary if on the same machine bro starts with default libpcap i
get full network visibility and real traffic logs:

pi at raspberrypi:/opt/bro/logs/current $ ls
capture_loss.log  dce_rpc.log  dns.log    http.log      notice.log
stats.log   stdout.log  weird.log
conn.log          dhcp.log     files.log  kerberos.log  ssl.log
stderr.log  syslog.log  x509.log
pi at raspberrypi:/opt/bro/logs/current $


pi at raspberrypi:/opt/bro/logs/current $ tail -f conn.log
1488300721.714628    C0ZuNp3n1AAPFqOAP8    fe80::d436:4663:8865:8d25
546    ff02::1:2    547    udp    -    62.994834    784    0    S0    F
F    0    D    7    1120    0    0    (empty)
1488300873.355133    C4qmHe463s94Z4dwq1    10.0.0.3    43772    10.0.0.1
53    udp    dns    0.000407    30    105    SF    T    T    0    Dd
1    58    1    133    (empty)
1488300873.353585    CHKByP1XMjBTSqRJ7e    10.0.0.3    55014    10.0.0.1
53    udp    dns    0.000434    44    107    SF    T    T    0    Dd
1    72    1    135    (empty)
1488300882.759725    CnoqlW2FFjY3LX3q2    10.0.0.6    49704    10.0.0.5
445    tcp    -    14.935617    3786    1209    RSTO    T    T    0
ShADdaR    13    4318    9    1581    (empty)
1488300864.039916    CwI2Fng0jsi2ODWl9    10.0.0.31    123
193.93.167.241    123    udp    -    0.020522    0    48    SHR    T
F    0    Cd    0    0    1    76    (empty)
1488300874.039921    CN6xHJ2fqNoOloNr4e    10.0.0.31    123
194.116.168.41    123    udp    -    0.029709    0    48    SHR    T
F    0    Cd    0    0    1    76    (empty)
1488300933.675398    Cd7Bql2wn5TNLWBTMg    10.0.0.3    47206    10.0.0.1
53    udp    dns    0.000539    30    105    SF    T    T    0    Dd
1    58    1    133    (empty)
1488300933.674395    CHSnQs2YEeLjiqxOe3    10.0.0.3    55965    10.0.0.1
53    udp    dns    0.000245    44    107    SF    T    T    0    Dd
1    72    1    135    (empty)
1488300889.039915    CALlew1poCyja9ha2l    10.0.0.31    123
91.217.155.60    123    udp    -    0.021564    0    48    SHR    T    F
0    Cd    0    0    1    76    (empty)



So any ideas whats going on ? I couldn't find any reference of something
similar really and i am searching reading and compiling for 2 weeks :)

Bro is a great tool and combined with rpi and pf_ring very flexible and
powerful in cluster mode. So any help would be highly appreciated to help
me with this project. Thanks in advanced
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170228/86f68074/attachment.html 

From jazoff at illinois.edu  Tue Feb 28 13:23:24 2017
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Tue, 28 Feb 2017 21:23:24 +0000
Subject: [Bro] Bro + pf_ring on a rasberry pi 3
In-Reply-To: <CAHv=MupwZXT2W2JvYZbBNbZzqBJ6RHB5WeYm24Etk0NoLk5+cw@mail.gmail.com>
References: <CAHv=MupwZXT2W2JvYZbBNbZzqBJ6RHB5WeYm24Etk0NoLk5+cw@mail.gmail.com>
Message-ID: <44531C49-D4E2-4B3B-AB09-DA88B47DDD90@illinois.edu>


> On Feb 28, 2017, at 4:17 PM, Alex Kefallonitis <al.kefallonitis at gmail.com> wrote:
> 
> There are no errors and no capture_loss or drop packets, although base bro plugins are enable, bro sees only limited events:
> 
> pi at raspberrypi:~/bro-test $ ls -la
> total 28
> drwxr-xr-x  3 pi   pi   4096 Feb 28 21:03 .
> drwxr-xr-x 12 pi   pi   4096 Feb 28 20:55 ..
> -rw-r--r--  1 root root  699 Feb 28 21:03 conn.log
> -rw-r--r--  1 root root  253 Feb 28 21:03 packet_filter.log
> -rw-r--r--  1 root root  362 Feb 28 21:03 reporter.log

The reporter.log contains errors.. what does it have in it?

-- 
- Justin Azoff



From al.kefallonitis at gmail.com  Tue Feb 28 13:37:08 2017
From: al.kefallonitis at gmail.com (Alex Kefallonitis)
Date: Tue, 28 Feb 2017 23:37:08 +0200
Subject: [Bro] Bro + pf_ring on a rasberry pi 3
In-Reply-To: <44531C49-D4E2-4B3B-AB09-DA88B47DDD90@illinois.edu>
References: <CAHv=MupwZXT2W2JvYZbBNbZzqBJ6RHB5WeYm24Etk0NoLk5+cw@mail.gmail.com>
	<44531C49-D4E2-4B3B-AB09-DA88B47DDD90@illinois.edu>
Message-ID: <CAHv=Mur3fDVGeEh0jLTtFswtkZAQUEczKvrds3EYEQ5MRCtS+g@mail.gmail.com>

pi at raspberrypi:~/bro-test $ cat reporter.log
#separator \x09
#set_separator    ,
#empty_field    (empty)
#unset_field    -
#path    reporter
#open    2017-02-28-21-09-35
#fields    ts    level    message    location
#types    time    enum    string    string
1488316175.157715    Reporter::INFO    received termination signal
(empty)
1488316175.157715    Reporter::INFO    674 packets received on interface
eth0, 0 dropped    (empty)
#close    2017-02-28-21-09-35


2017-02-28 23:23 GMT+02:00 Azoff, Justin S <jazoff at illinois.edu>:

>
> > On Feb 28, 2017, at 4:17 PM, Alex Kefallonitis <
> al.kefallonitis at gmail.com> wrote:
> >
> > There are no errors and no capture_loss or drop packets, although base
> bro plugins are enable, bro sees only limited events:
> >
> > pi at raspberrypi:~/bro-test $ ls -la
> > total 28
> > drwxr-xr-x  3 pi   pi   4096 Feb 28 21:03 .
> > drwxr-xr-x 12 pi   pi   4096 Feb 28 20:55 ..
> > -rw-r--r--  1 root root  699 Feb 28 21:03 conn.log
> > -rw-r--r--  1 root root  253 Feb 28 21:03 packet_filter.log
> > -rw-r--r--  1 root root  362 Feb 28 21:03 reporter.log
>
> The reporter.log contains errors.. what does it have in it?
>
> --
> - Justin Azoff
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170228/5d59f66f/attachment.html 

From jazoff at illinois.edu  Tue Feb 28 13:48:37 2017
From: jazoff at illinois.edu (Azoff, Justin S)
Date: Tue, 28 Feb 2017 21:48:37 +0000
Subject: [Bro] Bro + pf_ring on a rasberry pi 3
In-Reply-To: <CAHv=Mur3fDVGeEh0jLTtFswtkZAQUEczKvrds3EYEQ5MRCtS+g@mail.gmail.com>
References: <CAHv=MupwZXT2W2JvYZbBNbZzqBJ6RHB5WeYm24Etk0NoLk5+cw@mail.gmail.com>
	<44531C49-D4E2-4B3B-AB09-DA88B47DDD90@illinois.edu>
	<CAHv=Mur3fDVGeEh0jLTtFswtkZAQUEczKvrds3EYEQ5MRCtS+g@mail.gmail.com>
Message-ID: <C0147792-0BB0-43A3-B18C-BD3865AF13CA@illinois.edu>


> On Feb 28, 2017, at 4:37 PM, Alex Kefallonitis <al.kefallonitis at gmail.com> wrote:
> 
> pi at raspberrypi:~/bro-test $ cat reporter.log 
> #separator \x09
> #set_separator    ,
> #empty_field    (empty)
> #unset_field    -
> #path    reporter
> #open    2017-02-28-21-09-35
> #fields    ts    level    message    location
> #types    time    enum    string    string
> 1488316175.157715    Reporter::INFO    received termination signal    (empty)
> 1488316175.157715    Reporter::INFO    674 packets received on interface eth0, 0 dropped    (empty)
> #close    2017-02-28-21-09-35
> 

ah, well that's not so bad.


The entries that you pasted from your conn.log before only had "^c" for history, which is

        ## ^       connection direction was flipped by Bro's heuristic
        ## c       packet with a bad checksum


have you tried bro using the libpcap that comes with pf_ring?

-- 
- Justin Azoff



From al.kefallonitis at gmail.com  Tue Feb 28 13:55:54 2017
From: al.kefallonitis at gmail.com (Alex Kefallonitis)
Date: Tue, 28 Feb 2017 23:55:54 +0200
Subject: [Bro] Bro + pf_ring on a rasberry pi 3
In-Reply-To: <C0147792-0BB0-43A3-B18C-BD3865AF13CA@illinois.edu>
References: <CAHv=MupwZXT2W2JvYZbBNbZzqBJ6RHB5WeYm24Etk0NoLk5+cw@mail.gmail.com>
	<44531C49-D4E2-4B3B-AB09-DA88B47DDD90@illinois.edu>
	<CAHv=Mur3fDVGeEh0jLTtFswtkZAQUEczKvrds3EYEQ5MRCtS+g@mail.gmail.com>
	<C0147792-0BB0-43A3-B18C-BD3865AF13CA@illinois.edu>
Message-ID: <CAHv=Muqs0CZ=qvkVRROAOPZ-AOa-7JLHZOScHDZn2hqbT4ZYnQ@mail.gmail.com>

No i haven't how do i enable it? Just Compile bro with
pcap=/opt/pfring/lib/libpcap.so ? It would be faster than standard libpcap
but not as fast as pf_ring?

The bad checksum staff is weird but i also tried with -C option with no
difference..anyway the whole problem seems pretty unsual

2017-02-28 23:48 GMT+02:00 Azoff, Justin S <jazoff at illinois.edu>:

>
> > On Feb 28, 2017, at 4:37 PM, Alex Kefallonitis <
> al.kefallonitis at gmail.com> wrote:
> >
> > pi at raspberrypi:~/bro-test $ cat reporter.log
> > #separator \x09
> > #set_separator    ,
> > #empty_field    (empty)
> > #unset_field    -
> > #path    reporter
> > #open    2017-02-28-21-09-35
> > #fields    ts    level    message    location
> > #types    time    enum    string    string
> > 1488316175.157715    Reporter::INFO    received termination signal
> (empty)
> > 1488316175.157715    Reporter::INFO    674 packets received on interface
> eth0, 0 dropped    (empty)
> > #close    2017-02-28-21-09-35
> >
>
> ah, well that's not so bad.
>
>
> The entries that you pasted from your conn.log before only had "^c" for
> history, which is
>
>         ## ^       connection direction was flipped by Bro's heuristic
>         ## c       packet with a bad checksum
>
>
> have you tried bro using the libpcap that comes with pf_ring?
>
> --
> - Justin Azoff
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170228/44277c40/attachment.html 

From seth at icir.org  Tue Feb 28 14:03:16 2017
From: seth at icir.org (Seth Hall)
Date: Tue, 28 Feb 2017 17:03:16 -0500
Subject: [Bro] SSH brute-force email notice
In-Reply-To: <eeae5267-bc40-4cd8-93f0-0151c96d9329@Spark>
References: <4b0d7b09-5230-4b67-b965-d1a7cf566313@Spark>
	<3F857EE9-2C08-4ABE-8242-4615AD924BF1@illinois.edu>
	<642b841f-b9ff-4583-9274-096402cb225c@Spark>
	<404DA38B-407D-4C26-AE15-4D120D89075E@illinois.edu>
	<6f8abe5b-9417-46bb-8b5a-39249755a6db@Spark>
	<eeae5267-bc40-4cd8-93f0-0151c96d9329@Spark>
Message-ID: <41B13F3A-7D6C-4FEF-A1C8-1B9829506217@icir.org>

Email sending is automatically disabled if you are reading from a PCAP.  It only works when live traffic is being read.  I believe this was originally intended to avoid people hammering themselves with email on accident when analyzing a PCAP.

  .Seth

> On Feb 28, 2017, at 12:31 PM, Loris Leiva <loris.leiva at gmail.com> wrote:
> 
> So I have done more tests and here are my findings.
> 
> First of all, I added the Notice::policy hook so that every notice gets sent via email (in order not to worry about accepting the right type of notice). Then I tried different ways to launch the `ssh.pcap` with bro.
> 
> - `broctl process ssh.pcap -C` => outputs the notice with ACTION_EMAIL but does not send the email (no stderr.log nor email log).
> - `bro -r ssh.pcap local "Site::local_nets += { ? }" -C` => same output, no emails.
> 
> However, if I just start bro via broctl and let it run, I start receiving random notices via emails regarding my laptop?s connections. I haven?t been able to reproduce a SSH brute-force attack but I assume it would work that way.
> 
> So I am starting to wonder if the commands `bro` and `broctl process` are actually able to send emails. Any ideas on that ?
> 
> Thanks in advance for your help,
> Loris
> 
> 
> 
> On 28 Feb 2017, 17:27 +0100, Loris Leiva <loris.leiva at gmail.com>, wrote:
>> Yes sorry I meant no errors get logs but weirdly I still get my notice.log entry with Notice::ACTION_EMAIL in it.
>> 
>> On 28 Feb 2017, 17:24 +0100, Azoff, Justin S <jazoff at illinois.edu>, wrote:
>>> 
>>>> On Feb 28, 2017, at 11:17 AM, Loris Leiva <loris.leiva at gmail.com> wrote:
>>>> 
>>>> Thank you for your answer.
>>>> 
>>>> I have checked the logs during my scenario and when the email doesn?t send nothing get logs at all (not even on the bro stderr log). However, when I raise a dummy notice in a bro_init() event, then I receive the email and the email gets logged properly.
>>> 
>>> Nothing gets logged at all? not even to notice.log?
>>> 
>>>> 
>>>> Note that I am using macOS Sierra so I access my logs through the following command `log stream --predicate '(process == "smtpd") || (process == "smtp")' -info`.
>>>> 
>>>> Any idea of what could be the problem ?
>>>> 
>>>> Thanks again,
>>>> Loris
>>> 
>>> --
>>> - Justin Azoff
>>> 
> _______________________________________________
> Bro mailing list
> bro at bro-ids.org
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From loris.leiva at gmail.com  Tue Feb 28 14:05:19 2017
From: loris.leiva at gmail.com (Loris Leiva)
Date: Tue, 28 Feb 2017 23:05:19 +0100
Subject: [Bro] SSH brute-force email notice
In-Reply-To: <41B13F3A-7D6C-4FEF-A1C8-1B9829506217@icir.org>
References: <4b0d7b09-5230-4b67-b965-d1a7cf566313@Spark>
	<3F857EE9-2C08-4ABE-8242-4615AD924BF1@illinois.edu>
	<642b841f-b9ff-4583-9274-096402cb225c@Spark>
	<404DA38B-407D-4C26-AE15-4D120D89075E@illinois.edu>
	<6f8abe5b-9417-46bb-8b5a-39249755a6db@Spark>
	<eeae5267-bc40-4cd8-93f0-0151c96d9329@Spark>
	<41B13F3A-7D6C-4FEF-A1C8-1B9829506217@icir.org>
Message-ID: <405520ac-0201-4203-907c-a791fb57026d@Spark>

Oh thank you so much, it all makes sense now.

Do you know if there is a way for me to enable this feature with PCAP or an alternative? I would like to simulate a scenario using a big PCAP file for a presentation and it would be great if it could generate emails.

On 28 Feb 2017, 23:03 +0100, Seth Hall <seth at icir.org>, wrote:
> Email sending is automatically disabled if you are reading from a PCAP. It only works when live traffic is being read. I believe this was originally intended to avoid people hammering themselves with email on accident when analyzing a PCAP.
>
> .Seth
>
> > On Feb 28, 2017, at 12:31 PM, Loris Leiva <loris.leiva at gmail.com> wrote:
> >
> > So I have done more tests and here are my findings.
> >
> > First of all, I added the Notice::policy hook so that every notice gets sent via email (in order not to worry about accepting the right type of notice). Then I tried different ways to launch the `ssh.pcap` with bro.
> >
> > - `broctl process ssh.pcap -C` => outputs the notice with ACTION_EMAIL but does not send the email (no stderr.log nor email log).
> > - `bro -r ssh.pcap local "Site::local_nets += { ? }" -C` => same output, no emails.
> >
> > However, if I just start bro via broctl and let it run, I start receiving random notices via emails regarding my laptop?s connections. I haven?t been able to reproduce a SSH brute-force attack but I assume it would work that way.
> >
> > So I am starting to wonder if the commands `bro` and `broctl process` are actually able to send emails. Any ideas on that ?
> >
> > Thanks in advance for your help,
> > Loris
> >
> >
> >
> > On 28 Feb 2017, 17:27 +0100, Loris Leiva <loris.leiva at gmail.com>, wrote:
> > > Yes sorry I meant no errors get logs but weirdly I still get my notice.log entry with Notice::ACTION_EMAIL in it.
> > >
> > > On 28 Feb 2017, 17:24 +0100, Azoff, Justin S <jazoff at illinois.edu>, wrote:
> > > >
> > > > > On Feb 28, 2017, at 11:17 AM, Loris Leiva <loris.leiva at gmail.com> wrote:
> > > > >
> > > > > Thank you for your answer.
> > > > >
> > > > > I have checked the logs during my scenario and when the email doesn?t send nothing get logs at all (not even on the bro stderr log). However, when I raise a dummy notice in a bro_init() event, then I receive the email and the email gets logged properly.
> > > >
> > > > Nothing gets logged at all? not even to notice.log?
> > > >
> > > > >
> > > > > Note that I am using macOS Sierra so I access my logs through the following command `log stream --predicate '(process == "smtpd") || (process == "smtp")' -info`.
> > > > >
> > > > > Any idea of what could be the problem ?
> > > > >
> > > > > Thanks again,
> > > > > Loris
> > > >
> > > > --
> > > > - Justin Azoff
> > > >
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170228/dfaf078b/attachment-0001.html 

From al.kefallonitis at gmail.com  Tue Feb 28 14:13:06 2017
From: al.kefallonitis at gmail.com (Alex Kefallonitis)
Date: Wed, 1 Mar 2017 00:13:06 +0200
Subject: [Bro] Bro + pf_ring on a rasberry pi 3
In-Reply-To: <C0147792-0BB0-43A3-B18C-BD3865AF13CA@illinois.edu>
References: <CAHv=MupwZXT2W2JvYZbBNbZzqBJ6RHB5WeYm24Etk0NoLk5+cw@mail.gmail.com>
	<44531C49-D4E2-4B3B-AB09-DA88B47DDD90@illinois.edu>
	<CAHv=Mur3fDVGeEh0jLTtFswtkZAQUEczKvrds3EYEQ5MRCtS+g@mail.gmail.com>
	<C0147792-0BB0-43A3-B18C-BD3865AF13CA@illinois.edu>
Message-ID: <CAHv=MurNJdHiPyUPQCFS8U5t9KOATSHdakAd9mau6kRvThFzZw@mail.gmail.com>

The problem is tha rpi has 100mpbs network card and i want to use cluster
and pf_ring and without knowing much seems the best option for real time
monitoring on a production network

2017-02-28 23:48 GMT+02:00 Azoff, Justin S <jazoff at illinois.edu>:

>
> > On Feb 28, 2017, at 4:37 PM, Alex Kefallonitis <
> al.kefallonitis at gmail.com> wrote:
> >
> > pi at raspberrypi:~/bro-test $ cat reporter.log
> > #separator \x09
> > #set_separator    ,
> > #empty_field    (empty)
> > #unset_field    -
> > #path    reporter
> > #open    2017-02-28-21-09-35
> > #fields    ts    level    message    location
> > #types    time    enum    string    string
> > 1488316175.157715    Reporter::INFO    received termination signal
> (empty)
> > 1488316175.157715    Reporter::INFO    674 packets received on interface
> eth0, 0 dropped    (empty)
> > #close    2017-02-28-21-09-35
> >
>
> ah, well that's not so bad.
>
>
> The entries that you pasted from your conn.log before only had "^c" for
> history, which is
>
>         ## ^       connection direction was flipped by Bro's heuristic
>         ## c       packet with a bad checksum
>
>
> have you tried bro using the libpcap that comes with pf_ring?
>
> --
> - Justin Azoff
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170301/735d8f4d/attachment.html 

From seth at icir.org  Tue Feb 28 14:20:53 2017
From: seth at icir.org (Seth Hall)
Date: Tue, 28 Feb 2017 17:20:53 -0500
Subject: [Bro] SSH brute-force email notice
In-Reply-To: <405520ac-0201-4203-907c-a791fb57026d@Spark>
References: <4b0d7b09-5230-4b67-b965-d1a7cf566313@Spark>
	<3F857EE9-2C08-4ABE-8242-4615AD924BF1@illinois.edu>
	<642b841f-b9ff-4583-9274-096402cb225c@Spark>
	<404DA38B-407D-4C26-AE15-4D120D89075E@illinois.edu>
	<6f8abe5b-9417-46bb-8b5a-39249755a6db@Spark>
	<eeae5267-bc40-4cd8-93f0-0151c96d9329@Spark>
	<41B13F3A-7D6C-4FEF-A1C8-1B9829506217@icir.org>
	<405520ac-0201-4203-907c-a791fb57026d@Spark>
Message-ID: <3BD558A1-DA18-4715-99A7-F7752D94DAD6@icir.org>


> On Feb 28, 2017, at 5:05 PM, Loris Leiva <loris.leiva at gmail.com> wrote:
> 
> Do you know if there is a way for me to enable this feature with PCAP or an alternative? I would like to simulate a scenario using a big PCAP file for a presentation and it would be great if it could generate emails.

If you don't mind modifying scripts, you can find the line here:
	https://github.com/bro/bro/blob/master/scripts/base/frameworks/notice/main.bro#L338

If you get rid of that if statement it will work.

  .Seth

--
Seth Hall
International Computer Science Institute
(Bro) because everyone has a network
http://www.bro.org/



From loris.leiva at gmail.com  Tue Feb 28 14:22:48 2017
From: loris.leiva at gmail.com (Loris Leiva)
Date: Tue, 28 Feb 2017 23:22:48 +0100
Subject: [Bro] SSH brute-force email notice
In-Reply-To: <3BD558A1-DA18-4715-99A7-F7752D94DAD6@icir.org>
References: <4b0d7b09-5230-4b67-b965-d1a7cf566313@Spark>
	<3F857EE9-2C08-4ABE-8242-4615AD924BF1@illinois.edu>
	<642b841f-b9ff-4583-9274-096402cb225c@Spark>
	<404DA38B-407D-4C26-AE15-4D120D89075E@illinois.edu>
	<6f8abe5b-9417-46bb-8b5a-39249755a6db@Spark>
	<eeae5267-bc40-4cd8-93f0-0151c96d9329@Spark>
	<41B13F3A-7D6C-4FEF-A1C8-1B9829506217@icir.org>
	<405520ac-0201-4203-907c-a791fb57026d@Spark>
	<3BD558A1-DA18-4715-99A7-F7752D94DAD6@icir.org>
Message-ID: <5cb34677-6fae-400f-9038-4f184e6204ac@Spark>

That?s perfect! Thanks again for your help.

Loris

On 28 Feb 2017, 23:20 +0100, Seth Hall <seth at icir.org>, wrote:
>
> > On Feb 28, 2017, at 5:05 PM, Loris Leiva <loris.leiva at gmail.com> wrote:
> >
> > Do you know if there is a way for me to enable this feature with PCAP or an alternative? I would like to simulate a scenario using a big PCAP file for a presentation and it would be great if it could generate emails.
>
> If you don't mind modifying scripts, you can find the line here:
> https://github.com/bro/bro/blob/master/scripts/base/frameworks/notice/main.bro#L338
>
> If you get rid of that if statement it will work.
>
> .Seth
>
> --
> Seth Hall
> International Computer Science Institute
> (Bro) because everyone has a network
> http://www.bro.org/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ICSI.Berkeley.EDU/pipermail/bro/attachments/20170228/78ff9f4c/attachment.html