[Bro] Atomic operations on Broker store

Bencteux Jeffrey jeffrey.bencteux at ssi.gouv.fr
Tue Jul 18 02:56:53 PDT 2017


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi all,

I am playing around with Broker framework and I am having some trouble
trying to share elements such as set or table between multiple instances
of Bro.

What I am doing follows :

* A master store script creates a table and add it to the store
* A frontend store script retreive the table, add elements to it and
push it back to the store

I got strange results doing that such as only one element I was trying
to add was added (never the same) so I suspected the problem was
concurrency and that my operations were not atomic. Indeed, what happens
it that every call to my function in the slave script do not retreive an
up-to-date table. Retreiving and adding a new element to a table is not
an atomic operation and no function in the API is defined to do it in
one call.

If I do the same test with Broker::add_to_set() with a set instead of a
table it works. And it seems that it is because it is implemented as an
atomic operation in aux/broker/src/store/frontend.cc.

Any idea how to that with a table with existing functions? Or does it
needs further developments? If so, any hints on how to implement that?

Here is the Bro scripts I used (debug prints removed) :

master.bro:

@load base/frameworks/broker
@load broker_wrapper

const broker_port: port = 6666/tcp &redef;
redef exit_only_after_terminate = T;

global h: opaque of Broker::Handle;

event bro_init()
{
    Broker::enable();
    Broker::listen(broker_port, "127.0.0.1");

    h = Broker::create_master("test_store");

    WRAPPER::broker_table_create(h, "test");
}

event Broker::incoming_connection_established(peer_name: string)
{
    print "Conn established: ", peer_name;
}

frontend.bro:

@load base/frameworks/broker
@load broker_wrapper

const broker_port: port = 6666/tcp &redef;
redef exit_only_after_terminate = T;

global h: opaque of Broker::Handle;

event bro_init()
{
    Broker::enable();
    Broker::connect("127.0.0.1", broker_port, 1secs);
}

event Broker::outgoing_connection_established(peer_address: string,
                              peer_port: port,
                        peer_name: string)
{
    print "Conn established: ", peer_address, peer_port, peer_name;

    h = Broker::create_clone("test_store");

    WRAPPER::broker_table_insert(h, "test", Broker::data("one"),
Broker::data("this"));
    WRAPPER::broker_table_insert(h, "test", Broker::data("two"),
Broker::data("is"));
    WRAPPER::broker_table_insert(h, "test", Broker::data("three"),
Broker::data("sparta"));
}

event Broker::outgoing_connection_broken(peer_name: string, peer_port: port)
{
    print "Connection closed by remote peer";
    terminate();
}

broker_wrapper.bro:

module WRAPPER;

export {
       global broker_table_create: function(h: opaque of Broker::Handle,
name: string);
       global broker_table_insert: function(h: opaque of Broker::Handle,
name: string, key: Broker::Data, val: Broker::Data);
}

function broker_table_create(h: opaque of Broker::Handle,
                 name: string)
{
    local tab = Broker::table_create();
    Broker::insert(h, Broker::data(name), tab);
}

function broker_table_insert(h: opaque of Broker::Handle,
                 name: string,
                key: Broker::Data,
                val: Broker::Data)
{
    # look for table with name 'name'
    when (local res = Broker::lookup(h, Broker::data(name)))
    {
        # insert element "key" = "value" into table
        local status = Broker::table_insert(res$result,
                        key,
                        val);

                        print res$result;

        # insert table back into store
        Broker::insert(h, Broker::data(name), res$result);
    }
    timeout 10sec
    {
        print fmt("timeout broker_table_insert: %s key: %s val: %s",
name, key, val);
    }
}

I also did a second test:

If I try to add an element to a set, then test its existence in it with
Broker::set_contains() and it appears not to be in in the set. Any idea why?

Any help is welcome!

- -- 
Jeffrey BENCTEUX
ANSSI/COSSI/DTO/BSD
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJZbdtlAAoJEHTjT1vhqZtXV/sP/jmbNea4tCqnyX7wti2co5zc
6yL3aX7UyXHkuElzr3weKHA+oQZ3yguoQjodmigrwmBOMiFGp0bxzPvWUvYbnxWu
p7yiZZxooJNEyxkoBmauk05p79dqdhmDGLQ3kIBOGFkD0lMHx9bGPRIifEI4oDcp
hUwDi5LaEb/x//L6IPkByxxEoNrbI0HxSLS4Bl4TikUpkYbUICVsihdrlN8cPhXk
ysaO9S1jv9iJvHSOrktrhv9WuDup0HM0LDE142d/V+Dao9nu+P+HdfKd4ieRVq2O
AXFnoIp9flFksgn+PN/Y1FqF9pRWm1MUOQeUCLYTf9br7Z0z0jTR35NRogDvVFtH
at/pKuotWOozZhTPf3QnbOdpdLgXIlk6kdIqQ3ENgmkj72L8Giyn/7cEqpl0dBgB
9iR9p6VQBuI8vsRigRUvffyC7TwonCECE+CTL9AzYCv0Zs25sTZTOw7JFMS99qav
spGY5EK6H108Amc3COWQ3Gz5IUps3K4j6OwNIbOZKyaHztx8amaF6j2BPpU+V4Ox
2RuThAjW3kOECr2/fseN6HYR2AX7lNCn+HJqWHS+C+pbdOBUhGWucwYj+R+28u+h
PVlWwYxcSb0ezZxckX3w32oi0I0YkX9ejLtxdLMd2L9XY4S1nRhxeFHnESqx51gi
y6Uh6WaD+yfu9o4IXIUL
=6UD8
-----END PGP SIGNATURE-----





More information about the Bro mailing list