[Bro-Dev] Testing and Docs for Packages

[Johanna Amann]

On Tue, Jan 17, 2017 at 04:01:19AM +0000, Siwek, Jon wrote:
> > I actually think it would be neat to do this isolated, especially given
> > that this enables testing before installing.
> 
> Not sure I follow.  Can you explain further?

Sorry - what I meant is that the tests can run before the packages are put
into the bro directory, so you can see if they will work with the
installed Bro version (or potentially system configuration) before putting
the files in. So you can use it as a prerequisite check for installation.

The other way round, you have to roll back after putting them already
there - unless I misunderstood something.

Plus - even in this case, shouldn't you be able to load the user scripts
by loading local.bro? Meaning we could even run all the tests twice - once
just with the default Bro installation, and once with the user changes,
both before installing the scripts, which could even give an indication if
Bro or other packages are at fault.

> > It also makes it easier to create something like "smokers" (Bro
> > installations that just tro tu run all testsuites of all available
> > packages with a newer version to see if something went wrong).
> 
> Can you also go into more detail on what you’re thinking there?
> 
> If there's concerns about accidentally corrupting an existing/production
> bro installation, the alternative I’d suggest would be to set up a
> separate bro-pkg config file for the smoke tests that would have bro-pkg
> install stuff in an isolated location.  This allows users to explicitly
> define the testing sandbox for themselves.

No, the idea would be more along the lines that, in this case, you might
actually never want to really install the package; you just want to see if
the tests can pass. Though, admittedly, this can once again be
accomplished by just immediately uninstalling afterwards.

Johanna