[Bro-Dev] Proposal: Operating the Bro package manager offline
Michał Purzyński
michalpurzynski1 at gmail.com
Mon Sep 19 20:40:29 PDT 2016
One can, depending on the level of paranoia or risk management:
Use a proxy for accessing git repos with packages. For GitHub it presents a problem - because github is essential an arbitrary file upload service and without terminating SSL one cannot create good enough proxy rules around it.
What we do is we create a mirror of GitHub repositories that copy only white listed parts. This is beyond the package manager project.
I really like the idea of being able to bundle - cap - unbundle package and I think it should be supported. With possibly a configuration file to tell "bundle these packages" only.
> On 19 Sep 2016, at 18:27, Robin Sommer <robin at icir.org> wrote:
>
> At BroCon several people pointed out a need to install Bro packages on
> machines that do not have a direct external connection. One idea would
> be some kind of proxy scheme where an intermediary git repository
> mirrors packages locally; bro-pkg would then pull from there. However,
> I don't think any of us really liked that idea much. After a few
> rounds of discussions, Seth and I came up with a different idea that
> seems easier to manage: extending bro-pkg to bundle packages into
> deployment files that can be easily pushed to Bro systems simply by
> copying them over.
>
> I’ve tried to flesh this out a bit more, and would be interested to
> hear what you all think about this approach. And @Jon: Do you think
> this would be doable that way?
>
> Here’s the idea:
>
> 1. Generally, one first uses bro-pkg as usual to install packages onto
> a local Bro system that does have external Internet connectivity
> (this could be just a dummy Bro installation). One installs new
> packages there, updates existing ones, etc., until reaching a state
> that one wants to push out to the actual Bro system.
>
> 2. We add a new “bundle” command to bro-pkg that serializes the
> current state of packages into a single file on disk, a “package
> bundle”. The bundle contains the complete content of all currently
> installed packages, using some kind of suitable container format
> (could be just a ZIP file, or whatever works; the internal
> representation doesn’t really matter).
>
> 3. Users create such a bundle on the local system and then simply copy
> that bundle file over to all target Bro machines that do not have a
> external connectivity themselves, using whatever mechanism they
> have available (e.g., just scp; or maybe through some configuration
> management system like Ansible etc.).
>
> 4. On the target machine, one runs a corresponding “bro-pkg unbundle”
> command on that bundle file. That command will completely replace
> the system’s current set of packages with the bundle’s content. As
> a result, that machine will now have exactly the same packages
> installed as the original system.
>
> This would be the general scheme. A couple of people I talked to at
> BroCon confirmed that this would offer a viable solution for them, and
> that they would indeed much prefer copying files around over
> maintaining local git mirrors.
>
> Some additional thoughts on variations/extensions of this basic scheme:
>
> - I’m not quite sure if the bundle should contain just the packages
> themselves or further bro-pkg state as well, such as which packages
> are currently loaded. Right now I’m learning towards saying “just
> the packages”; that would basically treat bundles just as a
> transport mechanism to get packages over to another box. The actual
> Bro machines would still keep control over which packages to
> actually load, etc.
>
> - As it is described above, Step 1 would require having a local Bro
> installation into which packages get installed before they can be
> bundled up. It would be nice to have a mode where bro-pkg can
> operate without having a Bro around at all, just downloading
> packages locally somewhere for bundling them up. I could also see
> offering an even simpler mode where one simply lists packages to
> bundle on the command-line: “bro-pkg bundle <pkg1> <pkg2> <pkg3>”.
> That would be particularly useful with configuration management
> systems I think.
>
> - It would be neat if bro-pkg's Python library exposed operations to
> inspect & retrieve the content of a bundle, such as iterating over
> the packages inside a bundle and iterating over the files inside a
> package. That way one could easily build target-side scripts that
> process and validate bundles before going ahead and installing them
> (e.g., imposing custom restrictions on what kind of packages one
> allows to put in place; or ensuring that a bundle always contains a
> set of packages the site deems mandatory, to avoid configuration
> mistakes; or even just logging what gets pushed out).
>
> What do you guys think about this?
>
> Robin
>
> --
> Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin
> _______________________________________________
> bro-dev mailing list
> bro-dev at bro.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20160919/dd9651ec/attachment.html
More information about the bro-dev
mailing list