[Bro-Dev] Bloom filter merging

Matthias Vallentin vallentin at icir.org
Thu Jun 13 20:45:33 PDT 2013


I had a chat with Vern about this earlier today and the discussion
yielded the following idea:

Rather than reserving a fixed number of seeds at startup, we use a
keyed hashing scheme that computes a seed to construct the i'th hash
function as follows:

    seed_i = h(initial_seed || name || i)

One can fix "initial_seed" at startup via an environment variable. The
new element "name" represents a parameter that the user provide
(optionally?) as part of the BiF bloomfilter_init. The counter "i"
allows for generating an arbitrary number of hash functions. The
function "h" shall be a consistent hash function, e.g., H3 seeded with
the initial seed.

Feedback welcome,

    Matthias


More information about the bro-dev mailing list