[Bro-Dev] package manager progress

[Siwek, Jon]

The package manager client is at a point now where I think it would be usable.  Documentation is here:

https://bro.github.io/package-manager/

There is a branch in the ‘bro’ repo called ‘package-manager’ that simply changes CMake scripts to install ‘bro-pkg’ along with bro.  Here’s an example usage/session:

$ git clone --recursive --branch=package-manager git://bro.org/bro
...
$ cd bro && ./configure && make install
...
$ /usr/local/bro/bin/bro-pkg list all
default/jsiwek/bro-test-package
$ /usr/local/bro/bin/bro-pkg install bro-test-package
installed "bro-test-package"
loaded "bro-test-package"
$ /usr/local/bro/bin/bro packages
loaded bro-test-package plugin
loaded bro-test-package scripts
$ /usr/local/bro/bin/broctl
Test package: initialized
…

That test package shows that bro-pkg was able to install a package containing Bro scripts, a Bro plugin, and a BroControl plugin and everything should “just work” without needing any configuration.

Roadmap/TODO/Questions:

* Add a way for package’s to define “discoverability metadata”.

E.g. following the original plan for this would involve putting something like a “tags” field in each package’s pkg.meta file, but the problem with this is the client would need to either download every package to be able to search this data or have a third-party periodically aggregate it.

My current idea is that instead of putting this type of data inside the package’s metadata, the user puts it in the package source’s metadata.  They do this on first registration and may update it whenever.  That way, bro-pkg always has access to latest discoverability metadata, no need for a separate aggregation process.  It’s also something that will rarely change, so not a problem for that data to live in a repo not owned by the package author and not much increased burden for Bro Team to accept pull requests to update this data.  Thoughts?

* Automatic inter-package dependency analysis

Simply a TODO.  I put it at lower priority since I don’t think it will be common right off the bat to have complex package dependencies and users can always manually resolve dependencies at the moment.

* Is it acceptable to depend on GitPython and semantic_version python packages?

Both are replaceable implementation details, just didn’t want to write something myself if not necessary and in interest of time.

* Documentation is hosted on GitHub at the moment, move to bro.org?

Mostly just on GitHub now to be able to show something without having to touch any of the master bro/www doc generation processes, but maybe it’s a nice thing to start keeping docs more compartmentalized?  The current doc/www setup feels like it’s getting rather large/monolithic and maybe that contributes to the difficulty of approaching/understanding it when there’s breakages.  Just an idea.

* Thoughts on when to merge ‘package-manager’ branch in ‘bro’ ?

IMO, it can be done now or soon after I address responses/feedback to this email.

- Jon