From tyler.schoenke at colorado.edu  Fri Mar 23 13:51:36 2012
From: tyler.schoenke at colorado.edu (Tyler T. Schoenke)
Date: Fri, 23 Mar 2012 14:51:36 -0600
Subject: [TM] Time Machine on load balanced cluster
Message-ID: <4F6CE258.1060007@colorado.edu>

Hello, echo.... echo...

Hope someone out there is still subscribed to this list.  I am migrating
Bro to a load balanced cluster.  Since the traffic will be split between
16 or more worker nodes, I suspect I will need to run a separate
instance of Time Machine on each worker.  This means I will have to
query 16+ machines, which will get a bit cumbersome.

Is there a more elegant way to set this up?  I recall seeing TM cluster
mode in the todo meta ticket.  Are there any options available today?

Tyler

-- 
--
Tyler Schoenke
Network Security Manager
IT Security Office
University of Colorado at Boulder

From asharma at lbl.gov  Fri Mar 23 14:07:50 2012
From: asharma at lbl.gov (Aashish Sharma)
Date: Fri, 23 Mar 2012 14:07:50 -0700
Subject: [TM] Time Machine on load balanced cluster
In-Reply-To: <4F6CE258.1060007@colorado.edu>
References: <4F6CE258.1060007@colorado.edu>
Message-ID: <20120323210750.GC20744@yaksha.lbl.gov>

Hello Tyler:

The way we are currently doing this is to have an altogether separate feed going to
a standalone TM instance. Since bro can talk to TM, we just point bro
using "TimeMachineHost=<ip-addr>" in the broctl.cfg file 

Also on TM you need to enable TM to listen to the bro connections by
setting up the following in the tm.conf:

	bro_listen 1;
	bro_listen_port 47757;   # 47757 is default
        bro_listen_addr  <IP on cluster-manager> ;  # 127.0.0.1 is default

This should save you trouble to aggregating the the split traffic. 

I am sure there are better/other ways to do the same, but this is what
we have in deployment here. 

Aashish 

On Fri, Mar 23, 2012 at 02:51:36PM -0600, Tyler T. Schoenke wrote:
> Hello, echo.... echo...
> 
> Hope someone out there is still subscribed to this list.  I am migrating
> Bro to a load balanced cluster.  Since the traffic will be split between
> 16 or more worker nodes, I suspect I will need to run a separate
> instance of Time Machine on each worker.  This means I will have to
> query 16+ machines, which will get a bit cumbersome.
> 
> Is there a more elegant way to set this up?  I recall seeing TM cluster
> mode in the todo meta ticket.  Are there any options available today?
> 
> Tyler
> 
> -- 
> --
> Tyler Schoenke
> Network Security Manager
> IT Security Office
> University of Colorado at Boulder
> _______________________________________________
> Time-Machine mailing list
> Time-Machine at mailman.ICSI.Berkeley.EDU
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/time-machine

-- 
Aashish Sharma	(asharma at lbl.gov) 				 
Cyber Security, Information Technology Division  
Lawrence Berkeley National Laboratory  
http://www.lbl.gov/cyber/pgp-aashish.txt 
Office: (510)-495-2680  Cell: (510)-457-1525
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/time-machine/attachments/20120323/d08c23ce/attachment.bin 

From tyler.schoenke at colorado.edu  Mon Mar 26 07:58:51 2012
From: tyler.schoenke at colorado.edu (Tyler T. Schoenke)
Date: Mon, 26 Mar 2012 08:58:51 -0600
Subject: [TM] Time Machine on load balanced cluster
In-Reply-To: <20120323210750.GC20744@yaksha.lbl.gov>
References: <4F6CE258.1060007@colorado.edu>
	<20120323210750.GC20744@yaksha.lbl.gov>
Message-ID: <4F70842B.3000908@colorado.edu>

Hi Aashish,

By separate feed, are to talking separate mirror port?   I'm currently
splitting 10 Gbps with the load balancer.  I was wondering more how to
capture that data with TM.  Is your TM server capturing at 10 Gbps, or
whatever speed you are sending to bro?  My current worker servers only
have 1 Gb interfaces.  I haven't looked at putting a 10 Gb card into one
of them.  Can newer servers handle feeding 10 Gbps to TM?

Tyler

--
Tyler Schoenke
Network Security Manager
IT Security Office
University of Colorado at Boulder

On 3/23/12 3:07 PM, Aashish Sharma wrote:
> * PGP Signed by an unknown key
> 
> Hello Tyler:
> 
> The way we are currently doing this is to have an altogether separate feed going to
> a standalone TM instance. Since bro can talk to TM, we just point bro
> using "TimeMachineHost=<ip-addr>" in the broctl.cfg file 
> 
> Also on TM you need to enable TM to listen to the bro connections by
> setting up the following in the tm.conf:
> 
> 	bro_listen 1;
> 	bro_listen_port 47757;   # 47757 is default
>         bro_listen_addr  <IP on cluster-manager> ;  # 127.0.0.1 is default
> 
> This should save you trouble to aggregating the the split traffic. 
> 
> I am sure there are better/other ways to do the same, but this is what
> we have in deployment here. 
> 
> Aashish 
> 
> On Fri, Mar 23, 2012 at 02:51:36PM -0600, Tyler T. Schoenke wrote:
>> Hello, echo.... echo...
>>
>> Hope someone out there is still subscribed to this list.  I am migrating
>> Bro to a load balanced cluster.  Since the traffic will be split between
>> 16 or more worker nodes, I suspect I will need to run a separate
>> instance of Time Machine on each worker.  This means I will have to
>> query 16+ machines, which will get a bit cumbersome.
>>
>> Is there a more elegant way to set this up?  I recall seeing TM cluster
>> mode in the todo meta ticket.  Are there any options available today?
>>
>> Tyler
>>
>> -- 
>> --
>> Tyler Schoenke
>> Network Security Manager
>> IT Security Office
>> University of Colorado at Boulder
>> _______________________________________________
>> Time-Machine mailing list
>> Time-Machine at mailman.ICSI.Berkeley.EDU
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/time-machine
> 

From robin at icir.org  Mon Mar 26 08:07:44 2012
From: robin at icir.org (Robin Sommer)
Date: Mon, 26 Mar 2012 08:07:44 -0700
Subject: [TM] Time Machine on load balanced cluster
In-Reply-To: <4F6CE258.1060007@colorado.edu>
References: <4F6CE258.1060007@colorado.edu>
Message-ID: <20120326150744.GF29540@icir.org>


On Fri, Mar 23, 2012 at 14:51 -0600, you wrote:

> Is there a more elegant way to set this up?  I recall seeing TM cluster
> mode in the todo meta ticket.  Are there any options available today?

No, not really. A student once started to work on cluster support for
the TM, in the form of a TM proxy that would run on the manager and
relay queries/replies to/from the individual nodes. But that never got
very far unfortunately.

If you have a second mirror port, a separate TM machine with a 10G
interface (as Aashish wrote) might be the best solution right now,
assuming it can handle the load. 

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org