[Bro-Dev] package manager progress
robin at icir.org
Wed Jul 27 16:37:58 PDT 2016
On Sun, Jul 24, 2016 at 19:45 +0000, you wrote:
> The package manager client is at a point now where I think it would be
Finally got a chance to play with it a bit. Excellent work, I really
Belows a list of just smaller things I noticed. The only larger
question I have is regading the use of submodules, also following up
on other parts of this thread. In principle, I actually quite like the
idea of using submodules; git already offers the mechanism, so why not
build on that. That said, seeing how the package manager ends up using
submodules, it's not quite the pure git model actually. If I
understood it right, it's using them really only to *find* the
external repos, but not to pinpoint a particular commit in there; the
package source never even updates the submodules. Given that approach,
I'm now wondering if a custom scheme wouldn't be the more intuitive
solution. My concern is that whoever looks at this submodule usage,
will take a while to understand what's actually happening. One could
argue that's only an implementation detail and shouldn't really matter
to anybody. On the other hand, if, for example, somebody ends up
browsing the package source repository on GitHub, I'm sure they'd be
confused by all the packages pointing to very old versions. So I'm
wondering if it would be worth switching to custom index instead of
submodules; seems that wouldn't be difficult either if indeed all we
need to do is track the external URLs somehow. Also, if you want to
track discoverability metadata there already as well, seems that the
URL could just become part of that, no?
Here's my list of other random things I noticed:
- Would suggest to rename “pkg.meta” to, say, “bro-pkg.meta”, just to
make it more explicit that it's a Bro package. That's something one
can also then search for on GitHub.
- Does "upgrade" show the packages affected and ask for confirmation?
I would suggest either doing that or require an --all option for
upgrading everything, as that's a potentially dangerous operation.
- I suppose upgrading does (better: will do) dependency checking
again, including making sure the Bro version matches the one that
update now requires?
- When installing the package manager as part of Bro, could we pull in
the Python dependencies automatically, for example by installing
them into the same prefix? Both GitHub and semantic_version are
pretty non-standard. Using them is ok I think but it would be nice
if "bro-pkg" wouldn't abort first thing because they aren't
- How about adding a note to either packages.bro or the whole
packages/ directory that's it's automatically maintained and not
supposed to be manually messed with?
- In bro-pkg.conf, has "default" in "[sources]" a special meaning, or
could it be any tag? Assuming the latter, I would just call it
"bro": "bro/jsiwek/bro-test-package" is more intuitive than
- For our default package source, do we want to support non-GitHub
repositories? If so, a naming scheme by GitHub user won't work.
- Suggest to rename "/opt/bro/var/lib/package-manager" to
"../bro-package-manager" or "../bro-pkg".
- Once we support dependencies on Bro versions, would be nice if that
worked also with the "x.y-z" scheme that git master uses (and maybe
it just does anyways).
- I like the Python API!
- Documentation (nice!):
- Python 3.x works, right? Then I'd list that explicitly.
- A quick-start guide would be helpful that just mentions the most
important steps, including basic installation along with Bro
itself (once that's merged).
- The "Installation" section becomes a bit confusing towards with
the end with all those paths. Maybe split some parts out into an
advanced section or so?
- How-tos would be helpful that show by example how to create a
(1) a pure script package, (2) and binary Bro plugin, and (3) a
Finally, my take on your questions:
> My current idea is that instead of putting this type of data inside
> the package’s metadata, the user puts it in the package source’s
Yeah, I like that.
> Automatic inter-package dependency analysis
Agree on lower priority.
> * Is it acceptable to depend on GitPython and semantic_version python packages?
It is, it just would be nice if we could help people a bit getting
these installed, see above.
> * Documentation is hosted on GitHub at the moment, move to bro.org?
Keeping these docs separate makes sense, although it would also be
nice to have the option to integrate into bro.org at least. For now,
I think it's fine to just do your own Sphinx and publish it wherever
(GitHub, RTM). We can later see what to do about bro.org. Generally,
our bro.org setup certainly needs work, it's become hard to maintain
and extend. We have been talking about some options here recently but
not settled on anything in particular yet.
> * Thoughts on when to merge ‘package-manager’ branch in ‘bro’ ?
The main question is if we see this as a 2.5 feature? If so, we should
merge soon; if not, postpone until that's out the door. Given how far
you are already, I vote for making it part of 2.5. We could declare it
experimental still for 2.5 to get some more time to iron out the
workflow before we tell people it's ok to start relying on it.
Again, all very nice work!
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin
More information about the bro-dev