[Bro-Dev] Doc generation tests fail

[Robin Sommer]

On Thu, Apr 21, 2011 at 10:02 -0500, you wrote:

> That comment markup is using the wrong angle bracket.  Replace "##>"
> with "##<" and it should work.

Ah. :-)

>   The reason why the current markup results in a syntax error is
>   because the parser will interpret that last "##>" comment as a "##"

Ok, two thoughts:

    - is there a way to produce an error messages that makes clear
    it's a problem with the doc markup? Otherwise this can be quite
    confusing as one would normally just ignore comments when looking
    for errors the parser is reporting.

    - I'm wondering whether Bro should report this also when not
    running in doc mode. Otherwise, such things are hard to catch.

    On the other hand, I'm not sure I want to enforce rules on
    comments for cases where the doc generation isn't used.

    Perhaps a middle ground: should we have a test that ensures that
    all scripts we ship do correctly parse in doc mode? The existing
    tests reported logging.bro but only because it's a central one.
    Ones that are less used will be harder to find (and will in
    particular mess up the auto-generated pages on the web server)

> Side note: the "doc.autogen-reST-example" might still fail because (1)
> there's a missing period in one place in the baseline (2) the ordering
> of set/table values seems to be able to change between runs.

In theory, that shouldn't be the case as long as the RNG's seed is
set, which btest.cfg does. (But see below for practice).

>  I'm not sure if you want to try cherry-picking that commit or if you
>  just want to wait until merging that branch again.

Fine waiting. 

> http://www.ncsa.illinois.edu/~jsiwek/tmp/bro_logging_test_failures

Funny, some of that looks like a timezone difference! :)

And the tables do actually come out in a different order here as well.
I'm wondering whether seeding the RNG isn't sufficient. It has been in
the past afairc, but I guess same order isn't guarnateed *across*
systems but only within a single system. But then we have a problem
here ...

I saw you added a canonifier script for (2) above, but I'm guessing
that works only in this specific case? (Haven't looked closer at it). 

Robin

-- 
Robin Sommer * Phone +1 (510) 722-6541 * robin at icir.org
ICSI/LBNL    * Fax   +1 (510) 666-2956 *   www.icir.org