[Bro-Dev] case-insensitive patterns

[Vern Paxson]

Once I wound up monkeying around with the internals of the pattern-matching
code (to fix leaks, because Johanna [correctly] pushed back on adding the
&/| operators for general use if they leaked, which an old ticket indicated
they would) ... I thought what-the-heck, it's time for supporting
case-insensitive patterns.

This turned out to be tricky to implement, as I gleaned from talking with
Seth about an approach he had tried a while back but abandoned.  But I now
have it working.  Here's the blurb from the NEWS entry in the
topics/vern/case-insensitive-patterns branch:

- You can now specify that a pattern matches in a case-insensitive
  fashion by adding 'i' to the end of its specification.  So for example
  /fOO/i == "Foo" yields T, as does /fOO/i in "xFoObar".  Characters
  enclosed in quotes however keep their casing, so /"fOO"/i in "xFoObar"
  yields F, though it yields T for "xfOObar".

  You can achieve the same functionality for a subpattern enclosed in
  parentheses by adding "+i" to the open parenthesis, optionally followed
  by whitespace.  So for example "/foo|(+i bar)/" will match "BaR", but
  not "FoO".

  For both ways of specifying case-insensitivity, characters enclosed in
  double quotes maintain their case-sensitivity.  So for example /"foo"/i
  will not match "Foo", but it will match "foo".

The funky (+i ...) syntax isn't meant for general user consumption (though
it's okay if a user wants to use it directly), but rather is how I implemented
/pattern/i functionality.  Basically, /pattern/i turns into /(+i pattern)/.
That switch is necessary because the robust way to implement case-insensitive
patterns, such that they can be composed with the & and | operators and
behave as expected, is to modify the parsing of REs to turn any instance
of a letter into a character class (so that /foo/ becomes /[Ff][Oo]Oo]/,
just like people have been doing by hand for years), and also to modify
the parsing of character classes.  That requires alerting the RE scanner
that it's doing a case-insensitive (sub)pattern, which in turn requires
a prefix operator that specifies case-insensitivity.

Let me know if you have any concerns.  Otherwise, I'll tee this up
for merging early next week.

		Vern