[Bro-Dev] The BPAN project

Matthias Vallentin vallentin at icir.org
Fri Mar 16 12:46:23 PDT 2012

We've been talking a bit about our Bro community script archive BPAN [1]
quite a bit lately. Below, I summarize how I envision this project to
manifest. Feedback would be much appreciated.



    - Facilitate sharing Bro scripts in the community

    - Facilitate the integration of user-provided scripts

    - Specify a set of repositories (or paths inside) to load Bro scripts


    - Sharing scripts should be simple: one should be able to point
      someone to a remote repository and the integration happens

    - Organization by topic tags: similar to package managers, we need a
      way to categorize scripts and bundle them. For example

    - Separation between officially supported and user/experimental
      scripts. For the official ones, we offer support and make sure
      they'll always work with the latest/current version of Bro. We do
      not make such a guarantee for external scripts.

    - Integration with the Input/Output framework. That is, users should
      be able to provide custom data sources and sinks.

Implementation details:

    - github's popular fork-pull-merge model is an excellent fit for
      sharing community repositories. We may want to use
      https://github.com/broids as base for our community scripts. Users
      that want to extend the base simply perform the changes, open a
      pull request, and we can review it. The nice thing is that people
      can comment on it and follow-up changes are nicely reflected.
      Moreover, github automatically lists contributors to a certain
      file, so this encourages coders to share, participate, and get
      "reputation"---akin to stackoverflow.

    - Unlike homebrew, I believe it does not make sense for Bro to have
      single huge repository that contains all scripts and that users
      continuously branch from. Bro users probably want to organize
      their scripts into different repositories and combine multiple
      repositories ad libitum, because some repositories may contain
      sensitive elements and should not be public. Yet, private
      repositories can still be easily integrated with this new

      Having separate repositories also keeps the repository size under
      control. Although the input framework will be the main mechanism
      to source external data, I could imagine some folks would like to
      stuff some file-based intelligence in their repositories.

      However, separate repositories require integration efforts, but
      this is manageable in my eyes and outlined below.

    - Directory structure: the directory structure provides implicit
      naming and some meta data. If the directly structure is violated,
      Bro will not be able to automatically load scripts. These are the

        * Sub-directories in the repository represent "addressable
          units," e.g., names that users can refer to. I call these from
          now on a "rebro". Each rebro must have at least one file
          called '__load__.bro'. This is the entry point for Bro.

            - repo := https://www.github.com/joe-user/bromising/
            - repo/

          In this example, we have the following rebros:


          By placing __load__.bro files judiciously, users can
          explicitly decide what parts of the repo may be interpreted as
          a rebro. We may also allow putting a __load__.bro file in the
          top-level directory.

        * Each __load__.bro file contains additional meta data. Here is
          an example

            ## @version: 0.42
            ## @author: Joe Bro <bro at bro-ids.org>
            ## @license: BSD-style  (should be the default unless specified)
            ## @bro: 2.1
            ## @dependencies: b/x, https://my-repo.git/foo
            ## @tags: tls, x509, browser

          - The @-prefix inside comments may be used as a specific
            marker for parsing. But any other character would do as

          - @author: An optional field that identifies the script
            author's name and email address. If not given and the author
            hosts its repository on github, we may extract this data
            from github.

          - @license: An optional field to name the license under which
            this rebro should go

          - @bro: A mandatory field that specifies the Bro version this
            rebro is compatible with. Examples:

                * =major.minor
                    exact version match, e.g., =2.1

                * >=major.minor
                    minimum version, e.g., >=2.0

                * major.minor-major.minor
                    range of comptabile versions, e.g., 2.0-2.1

                * major.minor,major.minor
                    list of compatible versions, e.g., 1.5, 2.0beta

           - @dependencies: An optional field of internal and external
             rebro dependencies. Internal dependencies are specified
             relative to the repository top-level directory and external
             dependencies need a full-qualified rebro path.

           - @tags: An optional field that imposes a logical structure
             over rebros. Or we just keep it an
             unordered list of tags that users can combine at will.

                @tags: tls, x509, browser

             would associate three tags with this rebro. We may
             pre-define a set of categories under which users can label
             their rebros, e.g.,

                @tags: webapp/facebook, webapp/facebook

             would categorize the rebro hierarchically.

    - Let the union of all rebros be the 'universe'. We, the Bro team,
      should provide a global view of the universe by encouraging users
      to register their rebro's with us. The point of the universe is
      that it allows a global search across all existing rebros out
      there. One should be able to search the universe by tags, names,
      versions, etc.

      We may cache this meta data in a separate repository. A script
      may re-generate the meta data whenever a new rebro is added to it.

    - Configuration: Bro needs to be told which script repositories to
      integrate. I suggest all repositories sit in a one directory, that
      is controllable via BroControl.cfg, e.g.,

        Community = $PREFIX/share/bro/community

      Inside that directory, we have two subdirectories: one for all the
      repositories and one for the universe, e.g.,


    - One should be able to use BroControl to add new repositories at
      runtime and reload Bro with these scripts. Here are some examples
      that I will explain below

        > rebro add https://git.bro-ids.org/skype-analyzer [alias]
        > rebro add git-user/experimental-scripts [testing]
        > rebro rm testing
        > rebro load alias/decryptor
        > rebro load -r alias/spit-injector
        > rebro list
        > rebro list -u
        > rebro list -u -t
        > rebro pull
        > rebro deps alias
        > rebro deps alias/decryptor
        > rebro license alias


        * add: downloads a remote repository and register make all rebros
          inside it loadable

        * rm: removes a remote repository.

        * load: loads a specific rebro. The command fails unless all
          dependency requirements are met.

            --recursive|-r: downloads dependencies automatically

        * list: lists all available loadable rebros.

            --universe|-u: performs a global search in the universe
            --tags|-t:     lists rebro's by tags

        * deps: lists dependencies of a rebro.

        * license: displays licencing information about a rebro.

[1] Here are some other naming suggestions instead of BPAN:

    - Bropane (Bro Policy script Archive NEtwork; inflammable)
    - Brofuse (profuse: exuberantly plentiful)
    - Browl (A bowl full of Bro scripts)
    - Broil (Cooking with the community)
    - Brodder (Bro protestants)
    - Brovine (Grass-fed, no growth hormones)

More information about the bro-dev mailing list