From slagell at illinois.edu Wed Jun 1 07:21:25 2016 From: slagell at illinois.edu (Slagell, Adam J) Date: Wed, 1 Jun 2016 14:21:25 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <20160531175413.GO2796@samurai.ICIR.org> References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <43f3ce86-2869-877f-ae66-5040f145694d@gmail.com> <20160531024430.GB1312@shogun.local> <944E9525-D2EE-4E4A-8398-63DC795505F0@illinois.edu> <20160531175413.GO2796@samurai.ICIR.org> Message-ID: > On May 31, 2016, at 12:54 PM, Matthias Vallentin wrote: > >> >> I favor ?bagger? or ?bagboy? along with ?bags?. > > I did not get the "bagger" and "bagboy" variations until I googled it, > probably because I'm not a native speaker. However, I like the grocery > bag association, that one stuck immediately. We could have a shopping > "cart" as well, to represent a collection of bags. The grocery store > theme works really well, in my opinion. Can we call an orphaned package or homeless one a ?bag lady"? ------ Adam J. Slagell Chief Information Security Officer Director, Cybersecurity Division National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From greg at broala.com Wed Jun 1 07:33:23 2016 From: greg at broala.com (Gregory Bell) Date: Wed, 1 Jun 2016 07:33:23 -0700 Subject: [Bro-Dev] CBAN naming In-Reply-To: <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> Message-ID: On Fri, May 27, 2016 at 6:17 PM, Slagell, Adam J wrote: > Think the current name is fine, but if you change it I think it helps if > it relates to things people know. So names like bpkg, bro-ports, or > bro-brew would give the immediate analogy through the name. > I like all these suggestions from Adam - and as a person who enjoys making electronic music would humbly add 'bpm' (Bro Package Manager) to the list ;-) greg > > > > On May 27, 2016, at 12:00 PM, Matthias Vallentin > wrote: > > > > To find the new name for our CBAN project, it probably make sense to > > brainstorm separately from the existing technical thread. I'd say let's > > collect some candidates and then create survey to vote on them. > > > > Here are some ideas from the existing thread: > > > > - brow > > - broil > > - broom > > - bpk > > - berk > > - bob > > - bip > > > > Looking forward to hear your ideas. > > > > Matthias > > _______________________________________________ > > bro-dev mailing list > > bro-dev at bro.org > > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > > ------ > > Adam J. Slagell > Chief Information Security Officer > Director, Cybersecurity Division > National Center for Supercomputing Applications > University of Illinois at Urbana-Champaign > www.slagell.info > > "Under the Illinois Freedom of Information Act (FOIA), any written > communication to or from University employees regarding University business > is a public record and may be subject to public disclosure." > > > > > > > > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > -- Gregory Bell, PhD CEO - Broala www.broala.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20160601/493135d4/attachment.html From vallentin at icir.org Wed Jun 1 08:19:52 2016 From: vallentin at icir.org (Matthias Vallentin) Date: Wed, 1 Jun 2016 08:19:52 -0700 Subject: [Bro-Dev] CBAN naming In-Reply-To: References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> Message-ID: > I like all these suggestions from Adam - and as a person who enjoys making > electronic music would humbly add 'bpm' (Bro Package Manager) to the list > ;-) We had a good discussion in the Bro Gitter channel about this yesterday. In fact, I suggested "bpm," but as it turns out, there exists already a package that installs a PREFIX/bin/bpm executable. To summarize the discussion: the sentiment was that calling things "packages" is the least unexpected from a user point of view, because many other package managers (e.g., Python's pip) also use the term "package" for script and compiled code. Matthias From slagell at illinois.edu Wed Jun 1 14:58:14 2016 From: slagell at illinois.edu (Slagell, Adam J) Date: Wed, 1 Jun 2016 21:58:14 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> Message-ID: <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> I have gone over a bunch of these threads and want to bring this back towards consensus, or at least consilience. Let?s pick from the following 5 options, and then we can decide if these are bundles, ports, packages, plugins, extensions, modules, or bags. Key criteria are clarity and simplicity. Does it give an intuitive sense of what it is without adding confusion? No name or analogy will be perfect, but let?s not let the perfect be the enemy of the good. And the nominees are: 1. bpkg 2. brow 3. bagger 4. bro-ports 5. bro-brew From jazoff at illinois.edu Wed Jun 1 15:12:37 2016 From: jazoff at illinois.edu (Azoff, Justin S) Date: Wed, 1 Jun 2016 22:12:37 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> Message-ID: <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> > On Jun 1, 2016, at 5:58 PM, Slagell, Adam J wrote: > > I have gone over a bunch of these threads and want to bring this back towards consensus, or at least consilience. Let?s pick from the following 5 options, and then we can decide if these are bundles, ports, packages, plugins, extensions, modules, or bags. > > Key criteria are clarity and simplicity. Does it give an intuitive sense of what it is without adding confusion? No name or analogy will be perfect, but let?s not let the perfect be the enemy of the good. > > > And the nominees are: > > 1. bpkg > 2. brow > 3. bagger > 4. bro-ports > 5. bro-brew After talking with Matthias the other day I think the best option is simply: 6. bro-package or bro-pkg We already have things like bro-cut, and bro-aux. Broctl pretty close to bro-ctl. -- - Justin Azoff From slagell at illinois.edu Wed Jun 1 15:30:05 2016 From: slagell at illinois.edu (Slagell, Adam J) Date: Wed, 1 Jun 2016 22:30:05 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu>, <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> Message-ID: <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> These are variants of #1, which I now substitute with bro-pkg > On Jun 1, 2016, at 5:12 PM, Azoff, Justin S wrote: > > 6. bro-package or bro-pkg From jsiwek at illinois.edu Thu Jun 2 08:56:18 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Thu, 2 Jun 2016 15:56:18 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> Message-ID: <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> > On Jun 1, 2016, at 5:30 PM, Slagell, Adam J wrote: > > These are variants of #1, which I now substitute with bro-pkg Related to ?pkg? or ?package? naming: if that terminology gets used, what would be done about the classic/existing usage of the term ?package? within Bro? ?Package? is currently used to refer to any collection of Bro scripts within a common directory. Just scripts, nothing else. Meaning a ?package?, as it exists now, isn?t something that the new "package manager? would know how to deal with. I don?t think it?s a good idea to let the term be overloaded in the documentation and possibly internal naming conventions in Bro code. Not that it can?t be changed, but just wondering if it?s already been decided that making those changes would not be a big deal. - Jon From slagell at illinois.edu Thu Jun 2 10:44:02 2016 From: slagell at illinois.edu (Slagell, Adam J) Date: Thu, 2 Jun 2016 17:44:02 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> Message-ID: > On Jun 2, 2016, at 10:56 AM, Siwek, Jon wrote: > > Related to ?pkg? or ?package? naming: if that terminology gets used, what would be done about the classic/existing usage of the term ?package? within Bro? > > ?Package? is currently used to refer to any collection of Bro scripts within a common directory. Just scripts, nothing else. Meaning a ?package?, as it exists now, isn?t something that the new "package manager? would know how to deal with. > > I don?t think it?s a good idea to let the term be overloaded in the documentation and possibly internal naming conventions in Bro code. Not that it can?t be changed, but just wondering if it?s already been decided that making those changes would not be a big deal. Good question. Let me look into how hard it would be to change that. ------ Adam J. Slagell Chief Information Security Officer Director, Cybersecurity Division National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From slagell at illinois.edu Thu Jun 2 13:13:28 2016 From: slagell at illinois.edu (Slagell, Adam J) Date: Thu, 2 Jun 2016 20:13:28 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> Message-ID: > On Jun 2, 2016, at 10:56 AM, Siwek, Jon wrote: > >> >> On Jun 1, 2016, at 5:30 PM, Slagell, Adam J wrote: >> >> These are variants of #1, which I now substitute with bro-pkg > > Related to ?pkg? or ?package? naming: if that terminology gets used, what would be done about the classic/existing usage of the term ?package? within Bro? > > ?Package? is currently used to refer to any collection of Bro scripts within a common directory. Just scripts, nothing else. Meaning a ?package?, as it exists now, isn?t something that the new "package manager? would know how to deal with. > > I don?t think it?s a good idea to let the term be overloaded in the documentation and possibly internal naming conventions in Bro code. Not that it can?t be changed, but just wondering if it?s already been decided that making those changes would not be a big deal. It looks like that term is just used here for ?Script packages? and on all the auto generated subpages. We can just rename that as I don?t it is not a widely used term. I do want to do that before 2.5 if we do it, though. So with that said, I propose bro-pkg, but will leave this open for another day if there are strong opinions. ------ Adam J. Slagell Chief Information Security Officer Director, Cybersecurity Division National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From vallentin at icir.org Thu Jun 2 13:37:32 2016 From: vallentin at icir.org (Matthias Vallentin) Date: Thu, 2 Jun 2016 13:37:32 -0700 Subject: [Bro-Dev] CBAN naming In-Reply-To: References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> Message-ID: <20160602203732.GA939@shogun.local> > So with that said, I propose bro-pkg, but will leave this open for > another day if there are strong opinions. I like "bro-pkg," though I wonder whether it could be even shortened to "bpkg" or "bkg." This would be the name for the command line client. How would we call the whole thing? The Bro Package Manager? Or asked differently, what would be the replacement name for CBAN? Personally, I think nothing speaks against BPM (even though there exists a bpm-tools Linux package) since we're not using it as the name for the executable. Matthias From slagell at illinois.edu Thu Jun 2 14:15:43 2016 From: slagell at illinois.edu (Slagell, Adam J) Date: Thu, 2 Jun 2016 21:15:43 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <20160602203732.GA939@shogun.local> References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> <20160602203732.GA939@shogun.local> Message-ID: > On Jun 2, 2016, at 3:37 PM, Matthias Vallentin wrote: > >> So with that said, I propose bro-pkg, but will leave this open for >> another day if there are strong opinions. > > I like "bro-pkg," though I wonder whether it could be even shortened to > "bpkg" or "bkg." It could be, and bpkg is what I original suggest, but I thought you and Justin liked bro-org better. ------ Adam J. Slagell Chief Information Security Officer Director, Cybersecurity Division National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From jan.grashoefer at gmail.com Thu Jun 2 14:40:57 2016 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Thu, 2 Jun 2016 23:40:57 +0200 Subject: [Bro-Dev] CBAN naming In-Reply-To: References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> <20160602203732.GA939@shogun.local> Message-ID: <0ed6507a-84c7-b0a5-14ee-22cd78dbd5b5@gmail.com> >> I like "bro-pkg," though I wonder whether it could be even shortened to >> "bpkg" or "bkg." > > It could be, and bpkg is what I original suggest, but I thought you and Justin liked bro-org better. The point here was that bro-pkg would align to bro-cut. Although I still like brow, I would prefer bro-pkg or bropkg to bpkg and bkg. While b(p)kg could be anything bro-pkg is clearly bro-related. Best regards, Jan From slagell at illinois.edu Fri Jun 3 07:51:41 2016 From: slagell at illinois.edu (Slagell, Adam J) Date: Fri, 3 Jun 2016 14:51:41 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <0ed6507a-84c7-b0a5-14ee-22cd78dbd5b5@gmail.com> References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> <20160602203732.GA939@shogun.local> <0ed6507a-84c7-b0a5-14ee-22cd78dbd5b5@gmail.com> Message-ID: <765FBCB8-A880-449B-8E68-DD263DA2B70B@illinois.edu> > On Jun 2, 2016, at 4:40 PM, Jan Grash?fer wrote: > > The point here was that bro-pkg would align to bro-cut. Although I still > like brow, I would prefer bro-pkg or bropkg to bpkg and bkg. While > b(p)kg could be anything bro-pkg is clearly bro-related. Agreed. And if people really need to save a few characters, that?s what aliases are for. :-) ------ Adam J. Slagell Chief Information Security Officer Director, Cybersecurity Division National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From jsiwek at illinois.edu Fri Jun 3 11:59:22 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Fri, 3 Jun 2016 18:59:22 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> Message-ID: <1E74D08D-B7EE-4833-97D8-895C45492C7D@illinois.edu> > On Jun 2, 2016, at 3:13 PM, Slagell, Adam J wrote: > > It looks like that term is just used here for ?Script packages? and on all the auto generated subpages. We can just rename that as I don?t it is not a widely used term. I do want to do that before 2.5 if we do it, though. > > So with that said, I propose bro-pkg, but will leave this open for another day if there are strong opinions. Collections of bro scripts in a directory containing an __load__.bro file were named ?packages? because that is pretty much the same way Python structures the things it calls ?packages?. So, now that I think about it, it?s actually more odd to rename them because that structural similarity is not going away. Maybe I value consistency too much, but I don?t see how any of the proposals so far are improvements to the current situation. I?m going to make a final argument for just sticking with the original names of things: 1) Keep calling ?Bro Script Packages? as ?packages?. They look just like Python packages after all. 2) Keep calling ?Bro Plugins? as ?plugins?. No one explicitly asked to rename plugins or give them an alternate name, but that is what is happening here whether people realized it or not. Why are we trying to create a synonym for something that is already called a ?plugin? ? Won?t it be more confusing if it turns out that a ?package? may also be called a ?plugin? ? And confusing that a ?package" (also called a ?plugin?) shares little similarity with a Python package, but it contains within it something that does look like a Python package, yet is named something else? 3) CBAN becomes something like ?Bro Plugin Manager? because it is dealing with plugins, not packages. In fact, ?plugin" is probably more descriptive about what is going on. In general, a plugin just means any form of extending a software?s functionality without having to directly modify that software. That could mean shared libraries, scripts, etc. This is exactly what is going on here and I don?t see how we can get any more clear or consistent than the existing names. - Jon From slagell at illinois.edu Fri Jun 3 12:39:15 2016 From: slagell at illinois.edu (Slagell, Adam J) Date: Fri, 3 Jun 2016 19:39:15 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <1E74D08D-B7EE-4833-97D8-895C45492C7D@illinois.edu> References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> <1E74D08D-B7EE-4833-97D8-895C45492C7D@illinois.edu> Message-ID: <978B160D-3DDA-4EA8-A41C-88EE40822581@illinois.edu> > On Jun 3, 2016, at 1:59 PM, Siwek, Jon wrote: > >> >> On Jun 2, 2016, at 3:13 PM, Slagell, Adam J wrote: >> >> It looks like that term is just used here for ?Script packages? and on all the auto generated subpages. We can just rename that as I don?t it is not a widely used term. I do want to do that before 2.5 if we do it, though. >> >> So with that said, I propose bro-pkg, but will leave this open for another day if there are strong opinions. > > Collections of bro scripts in a directory containing an __load__.bro file were named ?packages? because that is pretty much the same way Python structures the things it calls ?packages?. So, now that I think about it, it?s actually more odd to rename them because that structural similarity is not going away. > > Maybe I value consistency too much, but I don?t see how any of the proposals so far are improvements to the current situation. I?m going to make a final argument for just sticking with the original names of things: > > 1) Keep calling ?Bro Script Packages? as ?packages?. They look just like Python packages after all. > > 2) Keep calling ?Bro Plugins? as ?plugins?. No one explicitly asked to rename plugins or give them an alternate name, but that is what is happening here whether people realized it or not. Why are we trying to create a synonym for something that is already called a ?plugin? ? Won?t it be more confusing if it turns out that a ?package? may also be called a ?plugin? ? And confusing that a ?package" (also called a ?plugin?) shares little similarity with a Python package, but it contains within it something that does look like a Python package, yet is named something else? > > 3) CBAN becomes something like ?Bro Plugin Manager? because it is dealing with plugins, not packages. In fact, ?plugin" is probably more descriptive about what is going on. In general, a plugin just means any form of extending a software?s functionality without having to directly modify that software. That could mean shared libraries, scripts, etc. This is exactly what is going on here and I don?t see how we can get any more clear or consistent than the existing names. So what would you do, call it the tool bro-bpm, the project ?Bro Plugin Manager? (BPM), and the objects being managed plugins? Alternatively, what I earlier proposed is that we call directories of related scripts bundles, plugins are called plugins, a plugin+metadata that fits them within our package manager a package, and the whole system the Bro Package Manager (BPM) with the tool named bro-pkg. I am fine if we want to hash out these last two options a bit longer, but please no more to the mix. ------ Adam J. Slagell Chief Information Security Officer Director, Cybersecurity Division National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From dnthayer at illinois.edu Fri Jun 3 12:48:34 2016 From: dnthayer at illinois.edu (Daniel Thayer) Date: Fri, 3 Jun 2016 14:48:34 -0500 Subject: [Bro-Dev] CBAN naming In-Reply-To: <1E74D08D-B7EE-4833-97D8-895C45492C7D@illinois.edu> References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> <1E74D08D-B7EE-4833-97D8-895C45492C7D@illinois.edu> Message-ID: <5751DF12.1080703@illinois.edu> On 06/03/2016 01:59 PM, Siwek, Jon wrote: > >> On Jun 2, 2016, at 3:13 PM, Slagell, Adam J wrote: >> >> It looks like that term is just used here for ?Script packages? and on all the auto generated subpages. We can just rename that as I don?t it is not a widely used term. I do want to do that before 2.5 if we do it, though. >> >> So with that said, I propose bro-pkg, but will leave this open for another day if there are strong opinions. > > Collections of bro scripts in a directory containing an __load__.bro file were named ?packages? because that is pretty much the same way Python structures the things it calls ?packages?. So, now that I think about it, it?s actually more odd to rename them because that structural similarity is not going away. > > Maybe I value consistency too much, but I don?t see how any of the proposals so far are improvements to the current situation. I?m going to make a final argument for just sticking with the original names of things: > > 1) Keep calling ?Bro Script Packages? as ?packages?. They look just like Python packages after all. > > 2) Keep calling ?Bro Plugins? as ?plugins?. No one explicitly asked to rename plugins or give them an alternate name, but that is what is happening here whether people realized it or not. Why are we trying to create a synonym for something that is already called a ?plugin? ? Won?t it be more confusing if it turns out that a ?package? may also be called a ?plugin? ? And confusing that a ?package" (also called a ?plugin?) shares little similarity with a Python package, but it contains within it something that does look like a Python package, yet is named something else? > > 3) CBAN becomes something like ?Bro Plugin Manager? because it is dealing with plugins, not packages. In fact, ?plugin" is probably more descriptive about what is going on. In general, a plugin just means any form of extending a software?s functionality without having to directly modify that software. That could mean shared libraries, scripts, etc. This is exactly what is going on here and I don?t see how we can get any more clear or consistent than the existing names. > > - Jon > I like this idea better than anything else I've heard so far, but one small issue is we would need to be a bit careful to distinguish between "Bro plugins" (as seen by running "bro -N"), and "Bro plugin manager plugins" (as seen by running "cban list"). Not sure if this would actually cause any real confusion, just wanted to raise the issue. After all, the meaning of words like "package" already depends on the context (e.g., "I installed the Bro rpm package" vs. "I installed Justin's passive DNS script package from github"). From jan.grashoefer at gmail.com Sat Jun 4 02:27:30 2016 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Sat, 4 Jun 2016 11:27:30 +0200 Subject: [Bro-Dev] CBAN naming In-Reply-To: <978B160D-3DDA-4EA8-A41C-88EE40822581@illinois.edu> References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> <1E74D08D-B7EE-4833-97D8-895C45492C7D@illinois.edu> <978B160D-3DDA-4EA8-A41C-88EE40822581@illinois.edu> Message-ID: <47ca0690-0933-71d5-22bd-53542eb10764@gmail.com> >> 3) CBAN becomes something like ?Bro Plugin Manager? because it is dealing with plugins, not packages. In fact, ?plugin" is probably more descriptive about what is going on. In general, a plugin just means any form of extending a software?s functionality without having to directly modify that software. That could mean shared libraries, scripts, etc. This is exactly what is going on here and I don?t see how we can get any more clear or consistent than the existing names. >From my perspective scripts do not extend Bro. Scripts get executed by Bro to provide extended functionality. Calling Bro-scripts plugins for Bro is somehow like calling python-scripts plugins for the python interpreter. > Alternatively, what I earlier proposed is that we call directories of related scripts bundles, plugins are called plugins, a plugin+metadata that fits them within our package manager a package, and the whole system the Bro Package Manager (BPM) with the tool named bro-pkg. I really like that naming. What should also be defined in that context is the term module. My understanding is that a module is a functional unit that groups script-layer functionality (defined by scripts or plugins) in a dedicate namespace. Regarding the discussion about the term "package" I recommend not to think too much about details of other software and its implications for that term. A package is "a wrapper or container that covers or holds something" [1]. Some people might assume that a Bro-package looks like a python-package. But Bro is not python so their assumption might be wrong ;) Best regards, Jan [1] http://www.merriam-webster.com/dictionary/package From jsiwek at illinois.edu Sat Jun 4 08:45:18 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Sat, 4 Jun 2016 15:45:18 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <978B160D-3DDA-4EA8-A41C-88EE40822581@illinois.edu> References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> <1E74D08D-B7EE-4833-97D8-895C45492C7D@illinois.edu> <978B160D-3DDA-4EA8-A41C-88EE40822581@illinois.edu> Message-ID: <13D91574-FB59-4939-8693-A8BD2126DAD2@illinois.edu> > On Jun 3, 2016, at 2:39 PM, Slagell, Adam J wrote: > > So what would you do, call it the tool bro-bpm, the project ?Bro Plugin Manager? (BPM), and the objects being managed plugins? Yeah both the tool and the project are "Bro Plugin Manager? (or some variant of BPM abbreviation) because that is role they perform. The Bro Plugin Manager repository contains a Bro Plugin Manager client as well as a central location for plugins contributed by the community. i.e. the client helps manage a particular user?s plugins and the repository helps manage/organize the whole community?s plugins. > Alternatively, what I earlier proposed is that we call directories of related scripts bundles, plugins are called plugins, a plugin+metadata that fits them within our package manager a package My last understanding is that we?re starting out w/ metadata being entirely optional and seeing how it goes. This also makes it more convenient for people to use the tool to manage plugins they have no intention of publishing and so don?t care to bother w/ metadata. So my opinion is that there is no reason to uniquely distinguish plugin+metadata from a regular plugin. Besides that, adding metadata to a plugin doesn?t change the fact that it is still a plugin that can be used directly with bro without requiring the use the new ?plugin manager? at all. - Jon From jsiwek at illinois.edu Sat Jun 4 08:49:41 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Sat, 4 Jun 2016 15:49:41 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <5751DF12.1080703@illinois.edu> References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> <1E74D08D-B7EE-4833-97D8-895C45492C7D@illinois.edu> <5751DF12.1080703@illinois.edu> Message-ID: > On Jun 3, 2016, at 2:48 PM, Thayer, Daniel N wrote: > > I like this idea better than anything else I've heard so far, but > one small issue is we would need to be a bit careful to distinguish between "Bro plugins" (as seen by running "bro -N"), and > "Bro plugin manager plugins" (as seen by running "cban list?). Can you clarify the concern? My view is that ?bro -N? gives you a list of ?installed? plugins and ?bpm list? gives a list of ?available? plugins. In both cases, it?s still giving lists of plugins, but just filtered differently. - Jon From jsiwek at illinois.edu Sat Jun 4 08:54:21 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Sat, 4 Jun 2016 15:54:21 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <47ca0690-0933-71d5-22bd-53542eb10764@gmail.com> References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> <1E74D08D-B7EE-4833-97D8-895C45492C7D@illinois.edu> <978B160D-3DDA-4EA8-A41C-88EE40822581@illinois.edu> <47ca0690-0933-71d5-22bd-53542eb10764@gmail.com> Message-ID: > On Jun 4, 2016, at 4:27 AM, Jan Grash?fer wrote: > > From my perspective scripts do not extend Bro. Scripts get executed by > Bro to provide extended functionality. Calling Bro-scripts plugins for > Bro is somehow like calling python-scripts plugins for the python > interpreter. A difference is that Python is a general purpose language and Bro is not. So I agree, it?s weird to call python scripts plugins because they don?t really extend Python itself. But Bro scripts are intended extend Bro?s functionality (you even said so yourself :)), so it?s not weird or false that they can be categorized as a type of plugin. - Jon From slagell at illinois.edu Sat Jun 4 09:27:45 2016 From: slagell at illinois.edu (Slagell, Adam J) Date: Sat, 4 Jun 2016 16:27:45 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <13D91574-FB59-4939-8693-A8BD2126DAD2@illinois.edu> References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> <1E74D08D-B7EE-4833-97D8-895C45492C7D@illinois.edu> <978B160D-3DDA-4EA8-A41C-88EE40822581@illinois.edu> <13D91574-FB59-4939-8693-A8BD2126DAD2@illinois.edu> Message-ID: <2D266CD0-A546-471C-B9D5-ADF907177705@illinois.edu> > On Jun 4, 2016, at 10:45 AM, Siwek, Jon wrote: > > My last understanding is that we?re starting out w/ metadata being entirely optional and seeing how it goes. This also makes it more convenient for people to use the tool to manage plugins they have no intention of publishing and so don?t care to bother w/ metadata. I still strongly disagree with ALL metadata being optional, unless it is automatically cleaned up if they never ?finish? putting in required data. However, that does not have a large impact on the name. I think we are now arguing between two reasonable proposals, and I am fine giving preference to the plugin naming because it does require the least amount of changes in current naming conventions. I will leave it open this weekend for members of the project leadership to jump in if they want, but otherwise let?s go with Bro Plugin Manager (BPM) and bro-bpm. ------ Adam J. Slagell Chief Information Security Officer Director, Cybersecurity Division National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From jsiwek at illinois.edu Sat Jun 4 10:25:26 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Sat, 4 Jun 2016 17:25:26 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <2D266CD0-A546-471C-B9D5-ADF907177705@illinois.edu> References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> <1E74D08D-B7EE-4833-97D8-895C45492C7D@illinois.edu> <978B160D-3DDA-4EA8-A41C-88EE40822581@illinois.edu> <13D91574-FB59-4939-8693-A8BD2126DAD2@illinois.edu> <2D266CD0-A546-471C-B9D5-ADF907177705@illinois.edu> Message-ID: > On Jun 4, 2016, at 11:27 AM, Slagell, Adam J wrote: > > I still strongly disagree with ALL metadata being optional, unless it is automatically cleaned up if they never ?finish? putting in required data. Sorry, I was just talking about in terms of interoperability w/ the client: all metadata is optional and doesn?t magically turn a plugin into something else that can now work with it. A goal for repository submissions is to have some quality checks in place to enforce some minimum metadata to be there. > I am fine giving preference to the plugin naming because it does require the least amount of changes in current naming conventions. Right, but just to succinctly summarize all the reasons that I think point toward any form of ?package? being used for this project as a poor choice: - it's too generic and not useful in describing what it is/does (in contrast to ?plugin?) - it would create another term for what is already named a ?plugin?. Having two words for the same thing isn?t optimal. - the term is already in use within Bro for script packages - it's also already overloaded based on other contexts (e.g. binary packages) > I will leave it open this weekend for members of the project leadership to jump in if they want, but otherwise let?s go with Bro Plugin Manager (BPM) and bro-bpm. Yes, I?d go with that, too. On the exception that someone can come up w/ a longer list of convincing problems regarding the use of ?plugin? :) - Jon From robin at icir.org Sat Jun 4 11:09:11 2016 From: robin at icir.org (Robin Sommer) Date: Sat, 4 Jun 2016 11:09:11 -0700 Subject: [Bro-Dev] CBAN naming In-Reply-To: References: <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> <1E74D08D-B7EE-4833-97D8-895C45492C7D@illinois.edu> <978B160D-3DDA-4EA8-A41C-88EE40822581@illinois.edu> <13D91574-FB59-4939-8693-A8BD2126DAD2@illinois.edu> <2D266CD0-A546-471C-B9D5-ADF907177705@illinois.edu> Message-ID: <20160604180911.GO30541@icir.org> On Sat, Jun 04, 2016 at 17:25 +0000, you wrote: > On the exception that someone can come up w/ a longer list of > convincing problems regarding the use of ?plugin? :) Sorry, I have to keep the discussion going by objecting. :-) I really don't like calling these things "plugins", nor calling the whole thing the "plugin manager". I'm with Jan here: I think that would be quite misleading in terms of what I believe people associate with "plugin" normally and also with how we've used the term "plugin" so far. The primary way we've used "plugin" so far is as a compiled, binary extension. While indeed the structure also accommodates script-only plugins, that does not warrant calling a set of scripts a "plugin" in my view. Indeed I don't think most people even realize that a plugin can be just scripts. In contrast, "package" is much more generic and can therefore better accomodate a range of things without becoming confusing (it's fuzzy to begin with :) So my vote goes to "package", "bro-pkg", and "Bro Package Manager". We could then also start calling the script-only things "modules", which aligns with what the scripting language already does with its namespaces (and in fact I'm often using "module" in that way already). So a "package" would then contain script "modules" and/or binary "plugins" My second choice would be using "bundle" instead of "package", and then I guess "bro-bundle" "Bro Bundle Manager". We're introducing something new here, so it would make sense to use a new term, "bundle". Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jan.grashoefer at gmail.com Sat Jun 4 11:40:09 2016 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Sat, 4 Jun 2016 20:40:09 +0200 Subject: [Bro-Dev] CBAN naming In-Reply-To: References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> <1E74D08D-B7EE-4833-97D8-895C45492C7D@illinois.edu> <978B160D-3DDA-4EA8-A41C-88EE40822581@illinois.edu> <13D91574-FB59-4939-8693-A8BD2126DAD2@illinois.edu> <2D266CD0-A546-471C-B9D5-ADF907177705@illinois.edu> Message-ID: > Sorry, I was just talking about in terms of interoperability w/ the client: all metadata is optional and doesn?t magically turn a plugin into something else that can now work with it. > [...] > - it would create another term for what is already named a ?plugin?. Having two words for the same thing isn?t optimal. Just a side note: If we equalize the concept of a plugin/script with the concept of a bundle/package, I agree that it would be unnecessary to find a new name for the same thing. But to me a bundle/package is something different: It's a deployment unit, acting as some sort of container (usually enriched by metadata). A plugin/script itself could be used with Bro but wrapping it into the container allows to manage it (e.g. in terms of version, dependencies, etc.). From slagell at illinois.edu Sat Jun 4 14:23:40 2016 From: slagell at illinois.edu (Slagell, Adam J) Date: Sat, 4 Jun 2016 21:23:40 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <20160604180911.GO30541@icir.org> References: <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> <1E74D08D-B7EE-4833-97D8-895C45492C7D@illinois.edu> <978B160D-3DDA-4EA8-A41C-88EE40822581@illinois.edu> <13D91574-FB59-4939-8693-A8BD2126DAD2@illinois.edu> <2D266CD0-A546-471C-B9D5-ADF907177705@illinois.edu> , <20160604180911.GO30541@icir.org> Message-ID: <799DAF59-8156-4B8B-B3A7-73B0B5C2A28F@illinois.edu> > On Jun 4, 2016, at 1:09 PM, Robin Sommer wrote: > > The primary way we've used "plugin" so far is as a compiled, > binary extension. I'm not sure we all have or that the project has used it consistently in that way. It is new enough to Bro I'm not bothered if we shift it slightly. To me, the important part of a the definition of a plugin for most software is that it is an externally contributed, self/contained add-on which extends functionality. Binary or compiled don't seem very important to the definition. Module, extension, package, bundle, add-on, plugin, bag, etc.; whatever we pick long term users will acclimate. I am more concerned new users or creating confusion in our documentation. From dnthayer at illinois.edu Sat Jun 4 17:48:07 2016 From: dnthayer at illinois.edu (Thayer, Daniel N) Date: Sun, 5 Jun 2016 00:48:07 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> <1E74D08D-B7EE-4833-97D8-895C45492C7D@illinois.edu> <5751DF12.1080703@illinois.edu>, Message-ID: <8F865DA62E66F543B6104A2835719CF93A9E75F7@CITESMBX5.ad.uillinois.edu> So are you saying that the "bpm" command wouldn't have a way to show all plugins that were installed via the bpm command? I assumed that is what "bpm list" would show. ________________________________________ From: Siwek, Jon Sent: Saturday, June 04, 2016 10:49 AM To: Thayer, Daniel N Cc: bro-dev at bro.org Subject: Re: [Bro-Dev] CBAN naming > On Jun 3, 2016, at 2:48 PM, Thayer, Daniel N wrote: > > I like this idea better than anything else I've heard so far, but > one small issue is we would need to be a bit careful to distinguish between "Bro plugins" (as seen by running "bro -N"), and > "Bro plugin manager plugins" (as seen by running "cban list?). Can you clarify the concern? My view is that ?bro -N? gives you a list of ?installed? plugins and ?bpm list? gives a list of ?available? plugins. In both cases, it?s still giving lists of plugins, but just filtered differently. - Jon From jsiwek at illinois.edu Sun Jun 5 08:09:53 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Sun, 5 Jun 2016 15:09:53 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <20160604180911.GO30541@icir.org> References: <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> <1E74D08D-B7EE-4833-97D8-895C45492C7D@illinois.edu> <978B160D-3DDA-4EA8-A41C-88EE40822581@illinois.edu> <13D91574-FB59-4939-8693-A8BD2126DAD2@illinois.edu> <2D266CD0-A546-471C-B9D5-ADF907177705@illinois.edu> <20160604180911.GO30541@icir.org> Message-ID: <60990A35-0BA4-4E81-950D-DE3D703C0D50@illinois.edu> > On Jun 4, 2016, at 1:09 PM, Robin Sommer wrote: > > I really don't like calling these things "plugins", nor calling the > whole thing the "plugin manager". I'm with Jan here: I think that > would be quite misleading in terms of what I believe people associate > with "plugin? normally My argument is that it is true/factual/objective that scripts may used as a form of plugin. We don?t have any data on exactly what percent of people don?t understand that. If it is a majority, that doesn?t change the fact that the majority has a belief that is not completely true and so it makes sense to go against them in an educational effort. And that task isn?t even difficult. It takes a single sentence description: ?Bro Plugins can be either compiled code, Bro scripts or a combination of both?. > and also with how we've used the term "plugin" > so far. The primary way we've used "plugin" so far is as a compiled, > binary extension. While indeed the structure also accommodates > script-only plugins, that does not warrant calling a set of scripts a > "plugin" in my view. Indeed I don't think most people even realize > that a plugin can be just scripts. But they are going to be become aware very soon, so isn?t it more awkward to try and continue obfuscating the truth of things? > So my vote goes to "package", "bro-pkg", and "Bro > Package Manager". The only way I?d be ok with ?package? is if it completely replaces all uses of the term ?plugin?. Otherwise, I think any other idea so far is better than packages. - Jon From jsiwek at illinois.edu Sun Jun 5 08:18:27 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Sun, 5 Jun 2016 15:18:27 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> <1E74D08D-B7EE-4833-97D8-895C45492C7D@illinois.edu> <978B160D-3DDA-4EA8-A41C-88EE40822581@illinois.edu> <13D91574-FB59-4939-8693-A8BD2126DAD2@illinois.edu> <2D266CD0-A546-471C-B9D5-ADF907177705@illinois.edu> Message-ID: <86F8699F-FB8A-4D7F-AD6C-17BFCF46D05A@illinois.edu> > On Jun 4, 2016, at 1:40 PM, Jan Grash?fer wrote: > > find a new name for the same thing. But to me a bundle/package is > something different: It's a deployment unit, acting as some sort of > container (usually enriched by metadata). A plugin/script itself could > be used with Bro but wrapping it into the container allows to manage it > (e.g. in terms of version, dependencies, etc.). I see what you?re saying, but it?s not in agreement with how the ?manager? client has been designed so far: it will be able to manage arbitrary plugins without requiring them to present metadata or be wrapped in any special container. Granted, the presence of metadata allows better forms of management, but there will still be some basic plugin management going on whether or not it exists. - Jon From jsiwek at illinois.edu Sun Jun 5 08:25:09 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Sun, 5 Jun 2016 15:25:09 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <8F865DA62E66F543B6104A2835719CF93A9E75F7@CITESMBX5.ad.uillinois.edu> References: <20160527170045.GG2796@samurai.ICIR.org> <6BB7568F-3B00-40C8-A4DD-E681B6464060@illinois.edu> <32408E32-44BA-4E61-9FED-53F5090384B9@illinois.edu> <4147968B-2516-4745-B3F3-E05D7CD54CAC@illinois.edu> <104C0D40-02E0-4CF4-B7C0-C497BEEAD1C5@illinois.edu> <1A60FC1E-546A-46B8-AE68-6C8B4CC08890@illinois.edu> <1E74D08D-B7EE-4833-97D8-895C45492C7D@illinois.edu> <5751DF12.1080703@illinois.edu> <8F865DA62E66F543B6104A2835719CF93A9E75F7@CITESMBX5.ad.uillinois.edu> Message-ID: <25E7CA99-1D77-4D30-B23E-23FD57E9A5A6@illinois.edu> > On Jun 4, 2016, at 7:48 PM, Thayer, Daniel N wrote: > > So are you saying that the "bpm" command wouldn't have a way to show all plugins > that were installed via the bpm command? I assumed that is what "bpm list" > would show. No, the plugin manager would be able to show all plugins installed through it and also list ones that haven?t been installed. We?ll have to see what the exact commands come out to be, it may end up being the way you assumed. - Jon From robin at icir.org Sun Jun 5 08:55:47 2016 From: robin at icir.org (Robin Sommer) Date: Sun, 5 Jun 2016 08:55:47 -0700 Subject: [Bro-Dev] CBAN naming In-Reply-To: <8F865DA62E66F543B6104A2835719CF93A9E75F7@CITESMBX5.ad.uillinois.edu> <60990A35-0BA4-4E81-950D-DE3D703C0D50@illinois.edu> <799DAF59-8156-4B8B-B3A7-73B0B5C2A28F@illinois.edu> Message-ID: <20160605155547.GE92333@icir.org> On Sat, Jun 04, 2016 at 20:40 +0200, Jan wrote: > But to me a bundle/package is something different: It's a deployment > unit, acting as some sort of container (usually enriched by metadata). > A plugin/script itself could be used with Bro but wrapping it into the > container allows to manage it (e.g. in terms of version, dependencies, > etc.). That's the way I see it, too. I'm coming to realize something: maybe the whole confusion comes from us reusing the plugins' internal structure: we said to reuse the same directory layout for CBAN (for lack of of a better name ;-) because conveniently we already have a container format that supports both binary plugins as well as set of standalone scripts. Unfortunately, that however now leads to conflating terminology, because suddenly plugins already *are* that container. And that I argue is wrong, because people writing scripts aren't writing plugins assuming any common interpretation of that term. So what if we stepped back and ignored how we will represent/structure these things internally and just conceptually adapt the model Jan is describing: we're creating *new* containers that support both scripts and plugins, and CBAN manages these containers. And then it becomes an implementation detail how this will exactly look like. If we still want to reuse the plugin structure, fine. But it would also be ok to do something different internally if that helps avoid confusion. Whatever it is, it would only need to make sure that after "install", everything is layed out correctly for Bro to load it (which for binary plugins means following their structure at the install location). (As a side node, bringing "bro -N" into the picture makes things even more confusing because that's *really* targeting the binary extension stuff. For example, "-NN" wouldn't show the script code being added). On Sat, Jun 04, 2016 at 21:23 +0000, Adam wrote: > To me, the important part of a the definition of a plugin for most > software is that it is an externally contributed, self/contained > add-on which extends functionality. Agree in principle, but "extending functionality" with a plugin is different that just writing a new Bro script. A "plugin" typically plugs into a set of hooks that a software provides to extend things it doesn't provide out of the box; once loaded, that new functionality then becomes a core feature just as if it had been built in in the first place. I don't see, e.g., writing a script-level detector for new vulnerability XYZ like that. If a wrote new Python module implementing, say, BASE65 encoding, I wouldn't be adding a "plugin" to Python either. On Sun, Jun 05, 2016 at 15:09 +0000, Jon wrote: > My argument is that it is true/factual/objective that scripts may used as a form of plugin. Yes, but per above that's only because we decided to reuse the internal structure. To me, that's arguing from an implementation-level artefact, which isn't good starting point for defining terminology. > And that task isn?t even difficult. It takes a single sentence > description: ?Bro Plugins can be either compiled code, Bro scripts or > a combination of both?. Sure, but it's still confusing to tell people you need to write a plugin to add your new vulnerability detector; whereas so far they simply wrote a script. Look at the mailing list for how people have used the term "plugin" so far: I'm pretty sure it's been only about binary plugins. Nobody writing just a script has said "I have a question about my plugin". Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From slagell at illinois.edu Sun Jun 5 09:51:07 2016 From: slagell at illinois.edu (Slagell, Adam J) Date: Sun, 5 Jun 2016 16:51:07 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <20160605155547.GE92333@icir.org> References: <20160605155547.GE92333@icir.org> Message-ID: <0FC5D616-0715-4015-8BCE-20D74BB10619@illinois.edu> > On Jun 5, 2016, at 10:55 AM, Robin Sommer wrote: > >> To me, the important part of a the definition of a plugin for most >> software is that it is an externally contributed, self/contained >> add-on which extends functionality. > > Agree in principle, but "extending functionality" with a plugin is > different that just writing a new Bro script. A "plugin" typically > plugs into a set of hooks that a software provides to extend things it > doesn't provide out of the box; once loaded, that new functionality > then becomes a core feature just as if it had been built in in the > first place. I don't see, e.g., writing a script-level detector for > new vulnerability XYZ like that. If a wrote new Python module > implementing, say, BASE65 encoding, I wouldn't be adding a "plugin" to > Python either. I think we are working from two different mental models. You are thinking of Bro mostly from the programming language perspective and analogies with Python. I am thinking of it as a tool or piece of software. It?s both, but I am not bothered by incongruencies to Python. Regardless, it seems that you and Jon have irreconcilable differences that eliminate plugin or package. And as the PI and implementer, I give high weight to both of your opinions. Would either of you object to extensions? So while I *really* hate to do this, I will throw out bro-bee and Bro Extensions for Everyone. ------ Adam J. Slagell Chief Information Security Officer Director, Cybersecurity Division National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From jsiwek at illinois.edu Sun Jun 5 13:31:34 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Sun, 5 Jun 2016 20:31:34 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <20160605155547.GE92333@icir.org> References: <20160605155547.GE92333@icir.org> Message-ID: > On Jun 5, 2016, at 10:55 AM, Robin Sommer wrote: > > So what if we stepped back Yes, generally we may need to step back. I think this thread started based on the idea that the names of things are entirely subjective and separate from the technical design. But that?s not true ? I need to be able to write unambiguous documentation and the choice of using plugins as the top-level container type means that introducing any other name for that container is creating an ambiguity. So yes, we can revisit the choice to use plugins as the top-level container. > and ignored how we will represent/structure > these things internally and just conceptually adapt the model Jan is > describing: we're creating *new* containers that support both scripts > and plugins, and CBAN manages these containers Yes, that could be a way to do things if I understand it right as: - New Container - Script Container (e.g. what is currently called ?script package?) - Compiled Code Container (e.g. what is currently called ?plugin?) Another idea is to not have any new container and allow the Script Container and Compiled Code Container to live at the top of the hierarchy as siblings. I think they could still share a common metadata format. It would also still be possible for a person to promote their Script Container to a Compiled Code Container if they find that they need that level of functionality. > (As a side node, bringing "bro -N" into the picture makes things even > more confusing because that's *really* targeting the binary extension > stuff. For example, "-NN" wouldn't show the script code being added). Yeah, I see how that?s a bit messy now, but skeptical because it might not be much of a fuss to just change/improve the output of a command line argument. > On Sat, Jun 04, 2016 at 21:23 +0000, Adam wrote: > > If a wrote new Python module > implementing, say, BASE65 encoding, I wouldn't be adding a "plugin" to > Python either. Correct, you didn?t write a plugin to Python, but if you had an application called "Foo" that allows users to specify custom encoding schemes via Python scripts, then you wrote a plugin for ?Foo?. So while a Bro script isn?t a plugin for the Bro language, it is a plugin for the Bro application. > A "plugin" typically > plugs into a set of hooks that a software provides to extend things it > doesn't provide out of the box; once loaded, that new functionality > then becomes a core feature just as if it had been built in in the > first place. I don't see, e.g., writing a script-level detector for > new vulnerability XYZ like that. But writing such a script does extend Bro w/ new functionality that it doesn?t provide out of the box, so it seems to fit your definition of a plugin as well. The only difference I see is that a script is more limited in how far it can extend functionality, but that?s not a binary property you can use to easily categorize plugin vs. non-plugin. The way I see it: one set of ?plugin hooks" that the Bro application provides is the Bro scripting language itself and all the APIs/BIFs/events that are available within it. > On Sun, Jun 05, 2016 at 15:09 +0000, Jon wrote: > >> My argument is that it is true/factual/objective that scripts may used as a form of plugin. > > Yes, but per above that's only because we decided to reuse the > internal structure. To me, that's arguing from an implementation-level > artefact, which isn't good starting point for defining terminology. But I?m not just arguing based upon the way Bro?s plugins are implemented, I?m arguing about plugins in general. There exists applications that use scripts as a form of plugin. So I?m saying it?s actually a Good Thing that Bro plugins can be script-only or additionally use compiled code, even if it was not intentional because it fits an even wider description of the term ?plugin?. But if you still don?t like that plugins can be used for scripts-only, then we should probably revisit the design so that we aren?t using them in that way (e.g. the ideas presented near the top of this email). >> And that task isn?t even difficult. It takes a single sentence >> description: ?Bro Plugins can be either compiled code, Bro scripts or >> a combination of both?. > > Sure, but it's still confusing to tell people you need to write a > plugin to add your new vulnerability detector; They only need to write a plugin if they plan to distribute it so that it works w/ the new plugin manager tool. I don?t see how that can lead to confusion, but I do see how maybe that could be more tedious than they would like :) We can address this problem by designing the manager to seamlessly deal with both script-only containers as well as compiled-code containers like we were thinking about above. > whereas so far they > simply wrote a script. Look at the mailing list for how people have > used the term "plugin" so far: I'm pretty sure it's been only about > binary plugins. But that doesn?t negate the truth that plugins currently may be either script-only or binary. And those past discussions don?t become invalidated if we move to using script-only plugins more generally. > Nobody writing just a script has said "I have a > question about my plugin?. Because at the time they said that, they weren?t authoring a plugin, they were writing a script :) People don?t currently have a reason to put their scripts inside a plugin, so that?s why no one uses that wording and goes down that path. The incentive for them to start putting their script within a plugin would be the existence of a tool that manages plugins as a top-level container. I think it may be worth it to do another design pass to see if can find any problems w/ either the super-container or sibling-container approach as outlined at the top. I do like those ideas and only see improvements and no drawbacks (apart from implementation code complexity, which doesn?t count for much), so I don?t mind wrapping my head around it for a bit longer to see if it?s a better fit. Unless I?ve convinced you that ?plugin? as the top-level container is the way to go? - Jon From jsiwek at illinois.edu Sun Jun 5 15:26:02 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Sun, 5 Jun 2016 22:26:02 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: <0FC5D616-0715-4015-8BCE-20D74BB10619@illinois.edu> References: <20160605155547.GE92333@icir.org> <0FC5D616-0715-4015-8BCE-20D74BB10619@illinois.edu> Message-ID: > On Jun 5, 2016, at 11:51 AM, Slagell, Adam J wrote: > > Regardless, it seems that you and Jon have irreconcilable differences that eliminate plugin or package. And as the PI and implementer, I give high weight to both of your opinions. Would either of you object to extensions? > > So while I *really* hate to do this, I will throw out bro-bee and Bro Extensions for Everyone. We haven?t had a lot of time to reconcile, but my stance is that it?s not logical to choose a project name that introduces ambiguity in the naming of the top-level containers it deals with. Some paths to move forward that I see: 1) revisit the design of what the top-level container is 2) use plugins as top-level container and project name that refers to plugins 3) use plugins as top-level container and project name that is totally unrelated to the container name 4) use plugins as top-level container, a project name that refers to them by a different name, and then suggest how docs should be written or re-organized to avoid ambiguity. I?m only being slightly facetious here, I really would need extra help/effort with guidelines for what term to use in what situations. e.g. is it ok to call it a ?plugin? when referring directly to existing plugin docs, but I should use the other term otherwise? Seems better to avoid the extra effort and strain this path puts on maintaining consistent/clear docs. - Jon From vlad at grigorescu.org Mon Jun 6 08:01:57 2016 From: vlad at grigorescu.org (Vlad Grigorescu) Date: Mon, 6 Jun 2016 10:01:57 -0500 Subject: [Bro-Dev] CBAN naming In-Reply-To: References: <20160605155547.GE92333@icir.org> <0FC5D616-0715-4015-8BCE-20D74BB10619@illinois.edu> Message-ID: Having reread through the discussion, I want to try to take a step back and review some of it. I believe there are two goals in play: 1) From a user's perspective, the principle of least astonishment. Names matter, and choosing something intuitive or familiar means we're not raising the barrier to entry for a new user. 2) From a developer's perspective, submitting to "CBAN," maintaining and updating their code should be as painless as possible. Let's not forget that at the end of the day, the only thing that matters in making this project successful or not is developer contributions. If people aren't contributing, none of this has any point. The first goal is the reason we moved away from bags and carts. However, I think we're in danger of straying too far away from the second goal. I think having containers which manage both scripts and plugins would be a mistake. A scenario that I see as being quite likely is that a developer starts such a container as being script-only, but wants to expand it at some future date. Say there's some new botnet with a domain-generation algorithm that they would like Bro to predict and alert on. The script is functional, it's being deployed, but it has a noticeable performance impact. After some refactoring, the developer adds a BIF to do some string-formatting heavy lifting in C++ instead of in script-land. At this point, how does the script package get converted to a plugin? Is this a new container? Is it a refactoring of the existing container? Will bro-pkg be smart enough to remove the old scripts from the script package and make sure the scripts from the plugin are being loaded? I fear that in trying to reduce confusion over nomenclature, we're complicating the design and introducing much more confusion. I believe there are advantages to having script-only plugins. Easier iterative development is one, but also having a well defined structure with a clear place for btest (in fact, init-plugin even creates a simple btest when creating the plugin). I believe that any issues with the name of "plugin" versus "package" can and will be quickly cleared up when the project is released. I'll note that the script plugin component is actually the only one currently documented here: https://www.bro.org/sphinx-git/devel/plugins.html I also think that since many of our developer audience is on this list, and we've been spamming them about this for the past 10 days, our user outreach has already happened. :-) --Vlad On Sun, Jun 5, 2016 at 5:26 PM, Siwek, Jon wrote: > > > On Jun 5, 2016, at 11:51 AM, Slagell, Adam J > wrote: > > > > Regardless, it seems that you and Jon have irreconcilable differences > that eliminate plugin or package. And as the PI and implementer, I give > high weight to both of your opinions. Would either of you object to > extensions? > > > > So while I *really* hate to do this, I will throw out bro-bee and Bro > Extensions for Everyone. > > We haven?t had a lot of time to reconcile, but my stance is that it?s not > logical to choose a project name that introduces ambiguity in the naming of > the top-level containers it deals with. > > Some paths to move forward that I see: > > 1) revisit the design of what the top-level container is > > 2) use plugins as top-level container and project name that refers to > plugins > > 3) use plugins as top-level container and project name that is totally > unrelated to the container name > > 4) use plugins as top-level container, a project name that refers to them > by a different name, and then suggest how docs should be written or > re-organized to avoid ambiguity. I?m only being slightly facetious here, I > really would need extra help/effort with guidelines for what term to use in > what situations. e.g. is it ok to call it a ?plugin? when referring > directly to existing plugin docs, but I should use the other term > otherwise? Seems better to avoid the extra effort and strain this path > puts on maintaining consistent/clear docs. > > - Jon > > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20160606/87623cd5/attachment.html From robin at icir.org Mon Jun 6 08:53:07 2016 From: robin at icir.org (Robin Sommer) Date: Mon, 6 Jun 2016 08:53:07 -0700 Subject: [Bro-Dev] CBAN naming In-Reply-To: References: <20160605155547.GE92333@icir.org> Message-ID: <20160606155307.GN92333@icir.org> On Sun, Jun 05, 2016 at 20:31 +0000, you wrote: > - New Container > - Script Container (e.g. what is currently called ?script package?) > - Compiled Code Container (e.g. what is currently called ?plugin?) Yep, and maybe that script container could then even replace all or some of the "scripts/" part of the "compiled code container" as well. Actually, taking this one step further and also back into a circle ... If we start from this perspective: a new container format that can carry both scripts and compiled code. Then the *structure* (but not the *name*) of what's currently a plugin might end up working actually. Would it solve our issue to simply rename that format from "plugin" to "container" (or "package" or "bundle" or whatever) The source format (i.e., what a user writes) would look like this: /scripts # Contains Bro script code. /src # Contains source for binary plugin /tests # Contains tests going with container content. When installing the plugin, that turns into: /__bro_container__ # Marks a container. /scripts # Contains Bro script code. /lib/.-.so # Contains compiled binary plugin. /lib/bif/ # Contains auto-generated *.bif.bro going with the binary plugin. We could clean that up a bit for clarity, like renaming to "src/" to "plugins/" for example. (I wouldn't object to changing the code reading in the plugins to expect a different layout if that helped solving all this; that's not cast in stone.) > Another idea is to not have any new container and allow the Script > Container and Compiled Code Container to live at the top of the > hierarchy as siblings. That would work as well but I think would make it more difficult to move between the two worlds. I'd prefer a single entity being managed. > So while a Bro script isn?t a plugin for the Bro language, it is a plugin for the Bro application. Yeah, but Bro *is* the language as well. Anyways, I don't think we'll agree on this, but that's ok. :-) Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From vallentin at icir.org Mon Jun 6 10:47:18 2016 From: vallentin at icir.org (Matthias Vallentin) Date: Mon, 6 Jun 2016 10:47:18 -0700 Subject: [Bro-Dev] CBAN naming In-Reply-To: References: <20160605155547.GE92333@icir.org> <0FC5D616-0715-4015-8BCE-20D74BB10619@illinois.edu> Message-ID: <20160606174718.GA923@ninja.local> > A scenario that I see as being quite likely is that a developer starts such > a container as being script-only, but wants to expand it at some future > date. Say there's some new botnet with a domain-generation algorithm that > they would like Bro to predict and alert on. The script is functional, it's > being deployed, but it has a noticeable performance impact. After some > refactoring, the developer adds a BIF to do some string-formatting heavy > lifting in C++ instead of in script-land. At this point, how does the > script package get converted to a plugin? This scenario seems to be well supported with the "container" layout that Robin proposed. Since each package/plugin will remain in its own prefix, it can easily start with a script and later on morph into full-blown shared library plus a set of bifs. If we consider everything a self-sufficient container, then even the Bro base is just one of them. (Certainly a very special one, since it provides the most functionality.) That reminds of BROPATH. If we loosen up its semantics, it would refer to a set of prefixes where to search for containers. For example, consider the following containers: /usr/local/share/bro/base /usr/local/share/bro/policy /usr/local/share/bro/site /opt/bro/github-user-foo/bro-fancy-detector /opt/bro/github-user-foo/bro-string-algs /opt/bro/bar/bro-deep-faf This would result in the following set of "search paths:" /usr/local/share/bro /opt/bro/github-user-foo /opt/bro/bar > I fear that in trying to reduce confusion over nomenclature, we're > complicating the design and introducing much more confusion. I believe > there are advantages to having script-only plugins. Ideally, it doesn't matter from a user perspective whether a plugin is script-only or ships with shared libraries and bifs. > I believe that any issues with the name of "plugin" versus "package" can > and will be quickly cleared up when the project is released. Yeah, I think either would work. One more thing: we should avoid the term "container" because there's an entire community emerging where this term means something very different. I don't think anyone ever proposed "container" to end up in the terminology, just making sure we're not going down that road. Matthias From robin at icir.org Mon Jun 6 11:50:32 2016 From: robin at icir.org (Robin Sommer) Date: Mon, 6 Jun 2016 11:50:32 -0700 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <20160606155307.GN92333@icir.org> References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> Message-ID: <20160606185032.GU92333@icir.org> So Johanna and I just white-boarded a somewhat more generic approach that we believe would work well to cover the various use-cases without hardcoding much in terms of structure at all. The brief summary is that we would allow people to specify how to set BROPATH and BRO_PLUGIN_PATH, and how to build their binary stuff; and otherwise don't impose any furher structure on them, just some defaults. Here's the idea in more detail: - We stay with the model of a "container" that can carry both scripts and binary plugins. (We could now maybe go back to "package" again; but I'll keep using "container" in the following). - The user's git repository *is* that container, however we don't enforce much of a layout on that at all, just some defaults they can change if desired. I'll call the disk location of the local clone for a user repository . - At install time ("cban install" or whatever) gets copied into a subdirectory at a global location ("cp -rp /"). Uninstallation means removing that installation directory. - For shipping scripts: - We simply add to Bro's BROPATH. That means one can now put "@load " into local.bro and Bro will look for a __load__.bro inside the top-level container directory. That file can do whatever it wants for loading further scripts located anywhere inside the container. One can also load individual top-level scripts that way through "@load /foo.bro". - If one wants to locate the scripts somewhere else inside the container, optional meta information can set a different BROPATH relative to the top-level directory. For example, to put them into "/scripts", one would specify as meta information "bropath=scripts". - For shipping binary plugins: - Through meta information, we let the author specify a build command to build all their binary stuff, such as "./configure && make && make test". The command line client runs that command inside . - Per default, we expect that build command to create a directory /build that contains a binary plugin. We add //build to BRO_PLUGIN_PATH. - If one wants to locate the plugin elsewhere, optional meta information can set a different BRO_PLUGIN_PATH. For example, to put it into "/compiled/cool_plugin", one would specify as meta information "pluginpath=compiled/cool_plugin". - In addition, we believe we should go back to require that minimal set of meta information that we discussed earlier: a container would need at least a name, a contact, a version, and a licence. For the optional meta information described above, we preset defaults: bropath=. pluginpath=./build buildcmd= # Empty means: nothing to build. So, examples: 1. A container with just a single script pulled in through "@load foo". /META name=foo version=1.0 license=bsd contact=foo at .bar.com /__load_.bro event bro_init() { print "Foo!"; } 2. A container with multiple scripts all loaded through "@load foo". /META name=foo version=1.0 license=bsd contact=foo at bar.com /__load_.bro @load ./script1 @load ./script2 /script1.bro event bro_init() { print "Foo1"; } /script2.bro event bro_init() { print "Foo2"; } 3. Same as (2), but the scripts now in a subdirectory scripts/: /META name=foo version=1.0 license=bsd contact=foo at bar.com bropath=scripts /scripts/__load_.bro /scripts/script1.bro /scripts/script2.bro 4. A binary plugin: /META name=foo version=1.0 license=bsd contact=foo at bar.com buildcmd=./configure && make && make test /configure /src/* /tests/* (The skeleton that init-plugin creates works for this directly already, except for the meta information.) 5. A binary plugin with different build command and location: /META name=foo version=1.0 license=bsd contact=foo at bar.com buildcmd=build-all-my-stuff.sh pluginpath=compiled/cool_plugin /build-all-my-stuff.sh / -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jsiwek at illinois.edu Mon Jun 6 12:09:19 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Mon, 6 Jun 2016 19:09:19 +0000 Subject: [Bro-Dev] CBAN naming In-Reply-To: References: <20160605155547.GE92333@icir.org> <0FC5D616-0715-4015-8BCE-20D74BB10619@illinois.edu> Message-ID: > On Jun 6, 2016, at 10:01 AM, Vlad Grigorescu wrote: > > 2) From a developer's perspective, submitting to "CBAN," maintaining and updating their code should be as painless as possible. Let's not forget that at the end of the day, the only thing that matters in making this project successful or not is developer contributions. If people aren't contributing, none of this has any point. > > The first goal is the reason we moved away from bags and carts. However, I think we're in danger of straying too far away from the second goal. I think having containers which manage both scripts and plugins would be a mistake. Yeah, we do need to keep the goal of making things easier for devs/authors, but I?m wondering if having the top-level containers be plugins doesn?t actually make things a harder for people who were only writing scripts to begin with? They now have to learn what a plugin is and how to make one instead of just being able to ship their directory full of scripts and have it work. Let me know what you think. > A scenario that I see as being quite likely is that a developer starts such a container as being script-only, but wants to expand it at some future date. I?m following Matthias on this in that I see such a super-container at the top-level being able to deal w/ changing from script-only to scripts+compiled in a way that?s transparent to users. We may also want a form a pinning that allows a user to say ?don?t upgrade this if it changes from script-only to compiled code?. You had some good implementation detail questions about how to make this work well that I think all have answers, so in interest of keeping the thread shorter I won?t go in to them now. But let me know if you see one problem that will be particularly tricky that we should go into detail about. > I believe that any issues with the name of "plugin" versus "package" can and will be quickly cleared up when the project is released. I'll note that the script plugin component is actually the only one currently documented here: https://www.bro.org/sphinx-git/devel/plugins.html Yeah, I?ll concede maybe people will get used to it, but I at least need to be working from a design spec that tells me exactly what to do concerning that page. If we start using ?package? in one place I will eventually need to start talking about how a ?plugin? works and refer to that page, which is currently a user-facing doc. Do canonicalize it to use ?package?? Do I leave it alone and just have a non-sequitur transition in talking about ?packages? and then suddenly ?plugins? ? If we switch the design to instead user the super-container format, then that?s not an issue for me anymore because the relationship changes from ?is a plugin? to ?may have a plugin?. - Jon From dnthayer at illinois.edu Mon Jun 6 12:14:50 2016 From: dnthayer at illinois.edu (Daniel Thayer) Date: Mon, 6 Jun 2016 14:14:50 -0500 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <20160606185032.GU92333@icir.org> References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> Message-ID: <5755CBAA.40905@illinois.edu> On 06/06/2016 01:50 PM, Robin Sommer wrote: > - For shipping binary plugins: > > - Through meta information, we let the author specify a build > command to build all their binary stuff, such as "./configure && > make && make test". The command line client runs that command > inside . > > - Per default, we expect that build command to create a directory > /build that contains a binary plugin. We add > //build to BRO_PLUGIN_PATH. > > - If one wants to locate the plugin elsewhere, optional meta > information can set a different BRO_PLUGIN_PATH. For example, > to put it into "/compiled/cool_plugin", one would specify > as meta information "pluginpath=compiled/cool_plugin". > Could you clarify what you mean by BRO_PLUGIN_PATH here? Are you saying that after I do a "cban install cool-plugin", I'd need to manually set an environment variable in order for Bro to find the new plugin? Or are we still planning to default to /lib/bro/plugins/ ? From robin at icir.org Mon Jun 6 12:18:17 2016 From: robin at icir.org (Robin Sommer) Date: Mon, 6 Jun 2016 12:18:17 -0700 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <5755CBAA.40905@illinois.edu> References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <5755CBAA.40905@illinois.edu> Message-ID: <20160606191817.GX92333@icir.org> On Mon, Jun 06, 2016 at 14:14 -0500, you wrote: > Could you clarify what you mean by BRO_PLUGIN_PATH here? I mean we magically extend the BRO_PLUGIN_PATH so that Bro will find it there. No manual configuration, CBAN will take care of pointing Bro to the right place inside the plugin's installation directory. (And we'd need to add mechanism to Bro to actually do that.) > Or are we still planning to default to /lib/bro/plugins/ ? That default location will stay, we just add the new ones to the internal search path as well. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From johanna at icir.org Mon Jun 6 12:22:04 2016 From: johanna at icir.org (Johanna Amann) Date: Mon, 6 Jun 2016 12:22:04 -0700 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <5755CBAA.40905@illinois.edu> References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <5755CBAA.40905@illinois.edu> Message-ID: <20160606192204.GA92824@wifi241.sys.ICSI.Berkeley.EDU> On Mon, Jun 06, 2016 at 02:14:50PM -0500, Daniel Thayer wrote: > On 06/06/2016 01:50 PM, Robin Sommer wrote: > > - For shipping binary plugins: > > > > - Through meta information, we let the author specify a build > > command to build all their binary stuff, such as "./configure && > > make && make test". The command line client runs that command > > inside . > > > > - Per default, we expect that build command to create a directory > > /build that contains a binary plugin. We add > > //build to BRO_PLUGIN_PATH. > > > > - If one wants to locate the plugin elsewhere, optional meta > > information can set a different BRO_PLUGIN_PATH. For example, > > to put it into "/compiled/cool_plugin", one would specify > > as meta information "pluginpath=compiled/cool_plugin". > > > > > Could you clarify what you mean by BRO_PLUGIN_PATH here? Are you saying > that after I do a "cban install cool-plugin", I'd need to manually > set an environment variable in order for Bro to find the new plugin? > Or are we still planning to default to /lib/bro/plugins/ ? No, you do not, that happens automatically. Users also have the ability to specify a subdirectory inside of their repository that is then used as the BRO_PLUGIN_PATH. That would be done by the plugin authors if they want to change the directory structure. For the installing user, everything remains automatic. Johanna From jsiwek at illinois.edu Mon Jun 6 14:08:07 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Mon, 6 Jun 2016 21:08:07 +0000 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <20160606185032.GU92333@icir.org> References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> Message-ID: <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> > On Jun 6, 2016, at 1:50 PM, Robin Sommer wrote: > > - For shipping scripts: > > - We simply add to Bro's BROPATH. That means one > can now put "@load " into local.bro and Bro will look for > a __load__.bro inside the top-level container directory. That > file can do whatever it wants for loading further scripts > located anywhere inside the container. One can also load > individual top-level scripts that way through "@load > /foo.bro?. Similar to Daniel?s question, is there a one time setup the user does or they need to modify local.bro every time they install a new container? I was thinking there?s a directory full of scripts from containers and Bro automatically loads all of them. If a user did ?cban install ? then that container?s scripts are all auto-loaded. But if a user did ?cban unload ? then they are free to still manually load scripts from it, but otherwise it?s not auto-loaded. They can then do ?cban load ? to switch back to a default load-everything behavior. > - In addition, we believe we should go back to require that minimal > set of meta information that we discussed earlier: a container would > need at least a name, a contact, a version, and a license. Is that metadata a requirement for submission to the community repo or just for general interoperability with the container manager client? >From user perspective, the later may be fine except potentially in the use-case where a person has no plans to submit their container in the near-term, but want to use the manager client and the container structure maybe for sake of consistency or other reasons? The other potential problem is if you require metadata and then call that container a ?package?, we are back to the issue of me needing to know what to do about the current use of the term ?package? which refers to a collection of scripts that require no metadata. A ?package? can't require metadata and also not require it at the same time. If it doesn?t confuse users to rename the current ?package,? that seems like an ok path to go forward with. A rename may at least temporarily confuse/bother me because I?ve used the term ?package? consistently in the past, but I?m not a user myself and don?t know whether the usage is common among actual users, so I need someone else to decide. - Jon From robin at icir.org Mon Jun 6 14:55:09 2016 From: robin at icir.org (Robin Sommer) Date: Mon, 6 Jun 2016 14:55:09 -0700 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> Message-ID: <20160606215509.GD92333@icir.org> On Mon, Jun 06, 2016 at 21:08 +0000, you wrote: > Similar to Daniel?s question, is there a one time setup the user does > or they need to modify local.bro every time they install a new > container? In this case I was thinking modify local.bro. That said, I remember your "load"/"unload" now, that would indeed be an alternative. What let's me hesitate a bit about that is that it doesn't match how people interact with scripts currently. Right now they have to @load them explicitly (unless they are in base/, where however we don't expect user scripts). Also, how would this work when using Bro from the command line (in contrast to BroControl)? Would it still pull in all the modules that have been loaded through the client? Could that cause confusion because we wouldn't have a standardized setting anymore? Would there be a way to prevent it? > Is that metadata a requirement for submission to the community repo or > just for general interoperability with the container manager client? Mostly for submission, though local installation through the client would need some as well I think (the name[1] and the paths at least). But people could always run out out of their local clone directly by setting BROPATH and BRO_PLUGIN_PATH manually (which is probably a good way for development anyways). Johanna suggested that if we do something similar to init-plugin to create a skeleton container, it would just put a meta file in place with defaults for the small set of fields we require for submission, so it wouldn't be much of a hassle either way. [1] Actually this occurred to me while writing the earlier mail: I think we need to let people set the name rather than just using the git repository name (which I at least had in mind so far). They may want to name their repository differently then what's the container should be called. > The other potential problem is if you require metadata and then call > that container a ?package?, I'm not attached to using "package". "bundle" works for me as well, as I think "container" does (though Matthias has a point there about meanings uses of "container"). I just don't like "plugin". :-) Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jsiwek at illinois.edu Mon Jun 6 19:01:13 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Tue, 7 Jun 2016 02:01:13 +0000 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <20160606215509.GD92333@icir.org> References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> <20160606215509.GD92333@icir.org> Message-ID: <97DACE92-DA36-4B27-A4B8-1C064898E113@illinois.edu> > On Jun 6, 2016, at 4:55 PM, Robin Sommer wrote: > > On Mon, Jun 06, 2016 at 21:08 +0000, you wrote: > >> Similar to Daniel?s question, is there a one time setup the user does >> or they need to modify local.bro every time they install a new >> container? > > In this case I was thinking modify local.bro. That said, I remember > your "load"/"unload" now, that would indeed be an alternative. What > let's me hesitate a bit about that is that it doesn't match how people > interact with scripts currently. Right now they have to @load them > explicitly (unless they are in base/, where however we don't expect > user scripts). So scripts do not autoload, but plugins do? I?m thinking the procedure a user takes to get things working should be the same regardless of script-only vs. binary plugin. And if the process isn?t the same in both cases, is that also in conflict w/ the goal of a developer being able to promote a script-only thing into a binary plugin without users noticing? > Also, how would this work when using Bro from the command line (in > contrast to BroControl)? Would it still pull in all the modules that > have been loaded through the client? Could that cause confusion > because we wouldn't have a standardized setting anymore? Would there > be a way to prevent it? I was thinking that Bro command line and BroControl would have to work similarly. So it may be Bro needs a direct change to how it handles auto-loading everything within a directory, either hardcoded specifically for use with the new containers or else a path specified by a new environment variable? >> The other potential problem is if you require metadata and then call >> that container a ?package?, > > I'm not attached to using "package". "bundle" works for me as well, as > I think "container" does (though Matthias has a point there about > meanings uses of "container"). I just don't like "plugin". :-) I agree that plugin no longer matches this design when referring to top-level container. And package maybe has less severe conflicts than before, but I'd still need some clarification in how to use it within code/docs. If package is still desirable, then maybe try to silently change current usage of ?script packages? into just ?scripts? without a new container name at all for them. I also suggest ?script directory? as a potential name if needed. Then reappropriate ?package" for use in this new project. To mock up example documentation: " The Bro Package Manager is a tool that makes it easy for Bro users to install and manage third party scripts and binary plugins. It does this by providing both a communal GitHub repository where developers contribute packages they have created and a command line tool, ?bro-pkg', for users to fetch packages from that repository. The packages that users submit may contain either scripts or binary plugins for Bro and the command line client automatically handles their installation process and interdependencies with other packages. To create a package, a developer needs to 1) Create a directory and metadata file with the following required fields... (TODO: details) 2) For packages that just contain Bro scripts, see the following docs about Bro?s script loading process (TODO: there?s maybe not a good doc on the script loading process besides [1] and [2], so maybe need a better one) and then fill out these additional metadata fields... (TODO: details) 3) For packages that contain binary plugins for Bro, see the following docs about plugins: [3]. Then fill out these additional metadata fields... (TODO: details) 4...) (TODO: get into submission process, maybe talk about btest for unit tests and other ways to improve package quality ratings, etc) ? That all looks consistent except part (2) ends up pointing people toward existing docs/examples that reference ?package? but with a different meaning. I'd need a decision to be made about how/whether to change that. If people are happy w/ the new design proposal and ready to revisit naming, it may be helpful to plug in (pun intended) their ideas for naming the project/client/containers as substitutes for what I used in the above example or make a new, similar one and read it through to see if it?s still coherent. - Jon [1] https://www.bro.org/sphinx/script-reference/packages.html [2] https://www.bro.org/sphinx/scripting/index.html [3] https://www.bro.org/sphinx-git/devel/plugins.html From robin at icir.org Mon Jun 6 20:46:48 2016 From: robin at icir.org (Robin Sommer) Date: Mon, 6 Jun 2016 20:46:48 -0700 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <97DACE92-DA36-4B27-A4B8-1C064898E113@illinois.edu> References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> <20160606215509.GD92333@icir.org> <97DACE92-DA36-4B27-A4B8-1C064898E113@illinois.edu> Message-ID: <20160607034648.GH92333@icir.org> So sounds like this proposal is something you can agree with? On Tue, Jun 07, 2016 at 02:01 +0000, you wrote: > So scripts do not autoload, but plugins do? Thinking more about that I would answer: yes. Going back to the principle of least surprise, this is how it is now as well: scripts need to be loaded, plugins load automatically. I suggest we start like that, we can always add an auto-load mechanism later if it turns out that would be useful. > And if the process isn?t the same in both cases, is that also in > conflict w/ the goal of a developer being able to promote a > script-only thing into a binary plugin without users noticing? Isn't that more about moving some functionality, like bifs? I don't think a plugin would replace the scripts completely. > To mock up example documentation: I like this, with the one note that most likely they won't need to fill out any additional meta data fields in (2) and (3) because the defaults will do. > That all looks consistent except part (2) ends up pointing people > toward existing docs/examples that reference ?package? but with a > different meaning. I'd need a decision to be made about how/whether > to change that. Ok, then let's rename "package" in the current docs. I propose "module" as the replacement: it's not quite right regarding the language's module concept but close enough I would say. > naming the project/client/containers My vote: Bro Package Manager, bro-pkgs, package. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From slagell at illinois.edu Mon Jun 6 21:52:45 2016 From: slagell at illinois.edu (Slagell, Adam J) Date: Tue, 7 Jun 2016 04:52:45 +0000 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <20160607034648.GH92333@icir.org> References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> <20160606215509.GD92333@icir.org> <97DACE92-DA36-4B27-A4B8-1C064898E113@illinois.edu> <20160607034648.GH92333@icir.org> Message-ID: > On Jun 6, 2016, at 9:46 PM, Robin Sommer wrote: > >> >> That all looks consistent except part (2) ends up pointing people >> toward existing docs/examples that reference ?package? but with a >> different meaning. I'd need a decision to be made about how/whether >> to change that. > > Ok, then let's rename "package" in the current docs. I propose > "module" as the replacement: it's not quite right regarding the > language's module concept but close enough I would say. Or ?bundle", which has even less baggage, but I think module is fine. > >> naming the project/client/containers > > My vote: Bro Package Manager, bro-pkgs, package. Only difference is I don?t think the client needs the plural. I would simply go to bro-pkg. ------ Adam J. Slagell Chief Information Security Officer Director, Cybersecurity Division National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From jazoff at illinois.edu Tue Jun 7 07:32:19 2016 From: jazoff at illinois.edu (Azoff, Justin S) Date: Tue, 7 Jun 2016 14:32:19 +0000 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <20160606185032.GU92333@icir.org> References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> Message-ID: <115ECAFE-C5BA-4252-B108-96DFC5B03716@illinois.edu> > On Jun 6, 2016, at 2:50 PM, Robin Sommer wrote: > > - At install time ("cban install" or whatever) gets copied into > a subdirectory at a global location ("cp -rp > /"). Uninstallation means removing that > installation directory. One thing to think about is a distinction between installed vs. enabled for scripts and plugins. A good system that I have used is how debian sets things up for the apache configuration. When you install the apache package they setup things like mods-available and mods-enabled directories: root at b28027aa3d70:/# ls -l /etc/apache2/mods-available/|head total 524 -rw-r--r-- 1 root root 100 Oct 24 2015 access_compat.load -rw-r--r-- 1 root root 377 Oct 24 2015 actions.conf -rw-r--r-- 1 root root 66 Oct 24 2015 actions.load -rw-r--r-- 1 root root 843 Oct 24 2015 alias.conf -rw-r--r-- 1 root root 62 Oct 24 2015 alias.load root at b28027aa3d70:/# ls -l /etc/apache2/mods-enabled/|head total 0 lrwxrwxrwx 1 root root 36 Jun 7 13:46 access_compat.load -> ../mods-available/access_compat.load lrwxrwxrwx 1 root root 28 Jun 7 13:46 alias.conf -> ../mods-available/alias.conf lrwxrwxrwx 1 root root 28 Jun 7 13:46 alias.load -> ../mods-available/alias.load lrwxrwxrwx 1 root root 33 Jun 7 13:46 auth_basic.load -> ../mods-available/auth_basic.load if I install the libapache2-mod-php5 package I end up with it enabled automatically: root at b28027aa3d70:/# ls -l /etc/apache2/mods-*/php* -rw-r--r-- 1 root root 865 Apr 27 11:42 /etc/apache2/mods-available/php5.conf -rw-r--r-- 1 root root 59 Apr 27 11:42 /etc/apache2/mods-available/php5.load lrwxrwxrwx 1 root root 27 Jun 7 13:48 /etc/apache2/mods-enabled/php5.conf -> ../mods-available/php5.conf lrwxrwxrwx 1 root root 27 Jun 7 13:48 /etc/apache2/mods-enabled/php5.load -> ../mods-available/php5.load but then can easily disable it without uninstalling it: root at b28027aa3d70:/# a2dismod php5 Module php5 disabled. To activate the new configuration, you need to run: service apache2 restart root at b28027aa3d70:/# ls -l /etc/apache2/mods-*/php* -rw-r--r-- 1 root root 865 Apr 27 11:42 /etc/apache2/mods-available/php5.conf -rw-r--r-- 1 root root 59 Apr 27 11:42 /etc/apache2/mods-available/php5.load This is just hooked up in the apache config using IncludeOptional mods-enabled/*.load IncludeOptional mods-enabled/*.conf They also do this available/enabled setup for standalone conf files and sites(vhost configs) The nice thing about this system is that the base installation can include standard modules that are present but not enabled by default: root at b28027aa3d70:/# ls -l /etc/apache2/mods-*/cgi.* -rw-r--r-- 1 root root 58 Oct 24 2015 /etc/apache2/mods-available/cgi.load root at b28027aa3d70:/# a2enmod cgi root at b28027aa3d70:/# ls -l /etc/apache2/mods-*/cgi.* -rw-r--r-- 1 root root 58 Oct 24 2015 /etc/apache2/mods-available/cgi.load lrwxrwxrwx 1 root root 26 Jun 7 13:56 /etc/apache2/mods-enabled/cgi.load -> ../mods-available/cgi.load The directory/symlink thing is just one implementation, automatically editing a special .bro file and adding/removing lines would work too. So, the way this could work is that '$TOOL install foo' could both 'install' and 'enable' 'foo' and '$TOOL disable foo' could disable it without removing it. -- - Justin Azoff From jbarber at computer.org Tue Jun 7 07:45:01 2016 From: jbarber at computer.org (Jeff Barber) Date: Tue, 7 Jun 2016 10:45:01 -0400 Subject: [Bro-Dev] TYPE_INT declaration Message-ID: I just stumbled across this in src/bif_type.def. My understanding of the internal BRO type system is a bit fuzzy so I'm not 100% sure this is wrong -- and I can't point to anything that doesn't work -- but it sure doesn't *look* right. // DEFINE_BIF_TYPE(id, bif_type, bro_type, c_type, accessor, constructor) ... DEFINE_BIF_TYPE(TYPE_INT, "int", "int", "bro_int_t", "%s->AsInt()", "new Val(%s, TYPE_BOOL)") Is there some reason why "new Val(%s, TYPE_BOOL)" is correct for the TYPE_INT constructor here? (This is in master branch but has been like this since the beginning of the git repo.) Thanks, Jeff -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20160607/93ef747f/attachment.html From robin at icir.org Tue Jun 7 09:14:26 2016 From: robin at icir.org (Robin Sommer) Date: Tue, 7 Jun 2016 09:14:26 -0700 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> <20160606215509.GD92333@icir.org> <97DACE92-DA36-4B27-A4B8-1C064898E113@illinois.edu> <20160607034648.GH92333@icir.org> Message-ID: <20160607161426.GC77405@icir.org> On Tue, Jun 07, 2016 at 04:52 +0000, you wrote: > Only difference is I don?t think the client needs the plural. Yep, I meant bro-pkg, the plural was just a typo. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jsiwek at illinois.edu Tue Jun 7 10:55:14 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Tue, 7 Jun 2016 17:55:14 +0000 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <20160607034648.GH92333@icir.org> References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> <20160606215509.GD92333@icir.org> <97DACE92-DA36-4B27-A4B8-1C064898E113@illinois.edu> <20160607034648.GH92333@icir.org> Message-ID: > On Jun 6, 2016, at 10:46 PM, Robin Sommer wrote: > > So sounds like this proposal is something you can agree with? Yes. > > On Tue, Jun 07, 2016 at 02:01 +0000, you wrote: > >> So scripts do not autoload, but plugins do? > > Thinking more about that I would answer: yes. Ack. >> And if the process isn?t the same in both cases, is that also in >> conflict w/ the goal of a developer being able to promote a >> script-only thing into a binary plugin without users noticing? > > Isn't that more about moving some functionality, like bifs? I don't > think a plugin would replace the scripts completely. Yeah, I?m not seeing specific problems thinking about it more now. > Ok, then let's rename "package" in the current docs. I propose > "module" as the replacement: it's not quite right regarding the > language's module concept but close enough I would say. I?d appreciate if anyone would think a bit about whether ?package? still actually makes sense to use in the current context and doesn?t actually need a rename. My point about a package being something that can both require metadata and not require metadata might be clear enough to explain based on context? E.g. From Bro/BroControl?s point of a view packages don?t require metadata. From the package manager tool?s perspective, packages require metadata. Seems like obvious/expected behavior from user?s view and not ambiguous? - Jon From jsiwek at illinois.edu Tue Jun 7 11:29:59 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Tue, 7 Jun 2016 18:29:59 +0000 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <115ECAFE-C5BA-4252-B108-96DFC5B03716@illinois.edu> References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <115ECAFE-C5BA-4252-B108-96DFC5B03716@illinois.edu> Message-ID: <0BB29BC9-45D9-4384-B904-6581B858ED77@illinois.edu> > On Jun 7, 2016, at 9:32 AM, Azoff, Justin S wrote: > > So, the way this could work is that '$TOOL install foo' could both 'install' and 'enable' 'foo' and '$TOOL disable foo' could disable it without removing it. Yeah, that?s what I was thinking people would want/expect. > The directory/symlink thing is just one implementation, automatically editing a special .bro file and adding/removing lines would work too. Part of the process of installing the tool could be echo @load bro-pkgs.bro >> $BRO_INSTALL_DIR/site/local.bro Then the bro-pkgs.bro script just needs to be in BROPATH and the tool would automatically add/remove entries from it. That special script can be maintained in the same place where the tool is maintaining all its installed packages. - Jon From robin at icir.org Tue Jun 7 11:34:32 2016 From: robin at icir.org (Robin Sommer) Date: Tue, 7 Jun 2016 11:34:32 -0700 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <0BB29BC9-45D9-4384-B904-6581B858ED77@illinois.edu> References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <115ECAFE-C5BA-4252-B108-96DFC5B03716@illinois.edu> <0BB29BC9-45D9-4384-B904-6581B858ED77@illinois.edu> Message-ID: <20160607183432.GG77405@icir.org> On Tue, Jun 07, 2016 at 18:29 +0000, you wrote: > echo @load bro-pkgs.bro >> $BRO_INSTALL_DIR/site/local.bro Yeah, ok, I can see that. Tieing it to local.bro takes care of my primary concern, that way it takes effect only when running through BroControl (or loading local.bro expliclty). Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From robin at icir.org Tue Jun 7 15:54:52 2016 From: robin at icir.org (Robin Sommer) Date: Tue, 7 Jun 2016 15:54:52 -0700 Subject: [Bro-Dev] TYPE_INT declaration In-Reply-To: References: Message-ID: <20160607225452.GU77405@icir.org> On Tue, Jun 07, 2016 at 10:45 -0400, you wrote: > Is there some reason why "new Val(%s, TYPE_BOOL)" is correct for the > TYPE_INT constructor here? Sure looks like a typo, I just fixed it in master. Thanks! Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jan.grashoefer at gmail.com Wed Jun 8 11:30:14 2016 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Wed, 8 Jun 2016 20:30:14 +0200 Subject: [Bro-Dev] Configurable &write_expire interval In-Reply-To: References: <57050AFC.1000609@gmail.com> Message-ID: My explanations might be hard to follow without examples. So I am adding some pseudo code: > I ran into an issue while trying to make the &write_expire interval > configurable: Using a redefable constant does not work here, as the > expression only gets evaluated when the table is initialized and thus > later redefs do not influence the value. # base script: const exp_val = 5min &redef; data: table[addr] of string &write_expire=exp_val; # user script: redef exp_val = 20min; # has no effect > I thought about circumventing > this by setting the value to 0 and maintain an extra variable to check > against in my expire_func and return the right value. Unfortunately this > won't work with write/read_expire as a write or read will reset the > expiration to the initial value of 0. # base script: const exp_val = 5min &redef; function do_exp(data: table[addr] of string, idx: addr): interval { if ( is_first_call() ) return exp_val; # in case of a write, expire timer will be reset to 0 else ... } data: table[addr] of string &write_expire=0 expire_func=do_exp; > A solution could be to evaluate the interval expression every time it is > used inside the table implementation. The drawback would be that there > is no fixed value for serialization (I am not sure about the effects > here). Another solution would be to provide a bif (or implement a > language feature) to change the expire_time value from inside the > expire_func. # base script: function do_exp(data: table[addr] of string, idx: addr): interval { if ( is_first_call() ) expire exp_val; # sets expire timer instead of delay else ... } > There was a somehow similar discussion about per item expiration (see > http://mailman.icsi.berkeley.edu/pipermail/bro-dev/2016-April/011731.html) > in which Robin came up with the solution of multiple tables with > different expiration values. Again this would be a solution but doesn't > feel right (duplicate code, static and somehow counterintuitive for the > user). # base script: type exp_interval enum { ei1m, ei10m, ... }; const exp_val = ei1m &redef; data1m: table[addr] of string &write_expire=1min; data10m: table[addr] of string &write_expire=10min; ... data1d: table[addr] of string &write_expire=1day; function insert(...) { switch ( exp_val ) { case ei1m: data1m[...] = ... break; case ei10m: data10m[...] = ... break; ... } } # user script: redef exp_val = ei30m; > Maybe I am missing something regarding the loading sequence of scripts > and this problem could be solved easier. So I am open for any > suggestions or feedback! From jsiwek at illinois.edu Wed Jun 8 12:13:45 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Wed, 8 Jun 2016 19:13:45 +0000 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> <20160606215509.GD92333@icir.org> <97DACE92-DA36-4B27-A4B8-1C064898E113@illinois.edu> <20160607034648.GH92333@icir.org> Message-ID: <0FD71378-4677-4331-A80D-C3705F9C04F3@illinois.edu> > On Jun 7, 2016, at 12:55 PM, Siwek, Jon wrote: > > I?d appreciate if anyone would think a bit about whether ?package? still actually makes sense to use in the current context and doesn?t actually need a rename. I can always repose this later if I?m struggling w/ using it in the docs, but I have my head wrapped around the various contexts it can be used in now that the design is changed, so I?m not blocked on it. I?d like to move forward w/ the following unless there?s still objections, open questions, or I misunderstood general consensus: project name: Bro Package Manager client name: bro-pkg git repo name: bro-pkg design spec/proposal: use Robin's/Johanna?s top-level container name: packages changes to existing naming conventions: none - Jon From vallentin at icir.org Wed Jun 8 15:32:47 2016 From: vallentin at icir.org (Matthias Vallentin) Date: Wed, 8 Jun 2016 15:32:47 -0700 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <0FD71378-4677-4331-A80D-C3705F9C04F3@illinois.edu> References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> <20160606215509.GD92333@icir.org> <97DACE92-DA36-4B27-A4B8-1C064898E113@illinois.edu> <20160607034648.GH92333@icir.org> <0FD71378-4677-4331-A80D-C3705F9C04F3@illinois.edu> Message-ID: <20160608223247.GD11575@samurai.ICIR.org> > project name: Bro Package Manager > client name: bro-pkg > git repo name: bro-pkg > design spec/proposal: use Robin's/Johanna?s > top-level container name: packages > changes to existing naming conventions: none Looks good to me. One question though: what is the top-level container name? Do you equate one package with one container, or does the plural "packages" signify something more collection-ish? Matthias From robin at icir.org Wed Jun 8 20:01:56 2016 From: robin at icir.org (Robin Sommer) Date: Wed, 8 Jun 2016 20:01:56 -0700 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <0FD71378-4677-4331-A80D-C3705F9C04F3@illinois.edu> References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> <20160606215509.GD92333@icir.org> <97DACE92-DA36-4B27-A4B8-1C064898E113@illinois.edu> <20160607034648.GH92333@icir.org> <0FD71378-4677-4331-A80D-C3705F9C04F3@illinois.edu> Message-ID: <20160609030156.GD9283@icir.org> Sounds good. On Wed, Jun 08, 2016 at 19:13 +0000, you wrote: > git repo name: bro-pkg Do we maybe need need two repositories, one for the client and for the packages? Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jsiwek at illinois.edu Thu Jun 9 08:27:33 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Thu, 9 Jun 2016 15:27:33 +0000 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <20160608223247.GD11575@samurai.ICIR.org> References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> <20160606215509.GD92333@icir.org> <97DACE92-DA36-4B27-A4B8-1C064898E113@illinois.edu> <20160607034648.GH92333@icir.org> <0FD71378-4677-4331-A80D-C3705F9C04F3@illinois.edu> <20160608223247.GD11575@samurai.ICIR.org> Message-ID: > On Jun 8, 2016, at 5:32 PM, Matthias Vallentin wrote: > > One question though: what is the top-level container name? package > Do you equate > one package with one container, or does the plural "packages" signify > something more collection-ish? I see them as one to one. e.g. a person submits their ?package? (singular container) to the community repo which will have many such ?packages? in it that users will browse through and interact w/ on an individual basis. - Jon From jsiwek at illinois.edu Thu Jun 9 09:44:46 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Thu, 9 Jun 2016 16:44:46 +0000 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <20160609030156.GD9283@icir.org> References: <20160605155547.GE92333@icir.org> <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> <20160606215509.GD92333@icir.org> <97DACE92-DA36-4B27-A4B8-1C064898E113@illinois.edu> <20160607034648.GH92333@icir.org> <0FD71378-4677-4331-A80D-C3705F9C04F3@illinois.edu> <20160609030156.GD9283@icir.org> Message-ID: <3DCD02A9-B42C-479E-B300-DC3AA968C9BC@illinois.edu> > On Jun 8, 2016, at 10:01 PM, Robin Sommer wrote: > > Do we maybe need need two repositories, one for the client and for the > packages? I see benefits in two separate repos: 1) pull requests easier to verify ? there won?t be client bug fixes mixed in w/ package submission requests. 2) easier to avoid accidental recursive submodule updates that go out and "fetch the world" ? the client can add package repositories ?git clone? instead of as submodules located directly in client?s git repo. 3) cleaner way for client to interface w/ multiple package collections if there ever comes to exist ones not maintained by the Bro project. So repos can be: client: bro-pkg community packages: bro-pkg-community bro-pkg then would have a default URL that points to bro-pkg-community as it?s hosted on bro.org. It could also point to the bro-pkg-community github mirror, but I don?t see a reason for that. Just mentioning in case someone does. - Jon From vallentin at icir.org Thu Jun 9 13:20:52 2016 From: vallentin at icir.org (Matthias Vallentin) Date: Thu, 9 Jun 2016 13:20:52 -0700 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: References: <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> <20160606215509.GD92333@icir.org> <97DACE92-DA36-4B27-A4B8-1C064898E113@illinois.edu> <20160607034648.GH92333@icir.org> <0FD71378-4677-4331-A80D-C3705F9C04F3@illinois.edu> <20160608223247.GD11575@samurai.ICIR.org> Message-ID: <20160609202052.GQ939@shogun.local> > > Do you equate one package with one container, or does the plural > > "packages" signify something more collection-ish? > > I see them as one to one. Okay, that's what I was thinking as well. Matthias From vallentin at icir.org Thu Jun 9 13:29:23 2016 From: vallentin at icir.org (Matthias Vallentin) Date: Thu, 9 Jun 2016 13:29:23 -0700 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <3DCD02A9-B42C-479E-B300-DC3AA968C9BC@illinois.edu> References: <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> <20160606215509.GD92333@icir.org> <97DACE92-DA36-4B27-A4B8-1C064898E113@illinois.edu> <20160607034648.GH92333@icir.org> <0FD71378-4677-4331-A80D-C3705F9C04F3@illinois.edu> <20160609030156.GD9283@icir.org> <3DCD02A9-B42C-479E-B300-DC3AA968C9BC@illinois.edu> Message-ID: <20160609202923.GR939@shogun.local> > I see benefits in two separate repos: Yep. > client: bro-pkg > community packages: bro-pkg-community I'm not sure if I understand the -community suffix. The client bro-pkg makes sense to me. But the first association I have with bro-pkg-community is a community-version of bro-pkg (i.e., the client). How about just "packages?" Forking github.com/bro/packages feels like a natural and understandable descriptor. Matthias From johanna at icir.org Thu Jun 9 13:34:12 2016 From: johanna at icir.org (Johanna Amann) Date: Thu, 09 Jun 2016 13:34:12 -0700 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <20160609202923.GR939@shogun.local> References: <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> <20160606215509.GD92333@icir.org> <97DACE92-DA36-4B27-A4B8-1C064898E113@illinois.edu> <20160607034648.GH92333@icir.org> <0FD71378-4677-4331-A80D-C3705F9C04F3@illinois.edu> <20160609030156.GD9283@icir.org> <3DCD02A9-B42C-479E-B300-DC3AA968C9BC@illinois.edu> <20160609202923.GR939@shogun.local> Message-ID: On 9 Jun 2016, at 13:29, Matthias Vallentin wrote: >> I see benefits in two separate repos: > > Yep. > >> client: bro-pkg >> community packages: bro-pkg-community > > I'm not sure if I understand the -community suffix. The client bro-pkg > makes sense to me. But the first association I have with > bro-pkg-community is a community-version of bro-pkg (i.e., the > client). > How about just "packages?" Forking github.com/bro/packages feels like > a > natural and understandable descriptor. And to throw another idea in that is in line with this - what about calling the community package repository "packages" like matthias said and the client repository "package-manager"? github.com/bro/package-manager also seems quite clear :) Johanna From jsiwek at illinois.edu Thu Jun 9 14:32:19 2016 From: jsiwek at illinois.edu (Siwek, Jon) Date: Thu, 9 Jun 2016 21:32:19 +0000 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <20160609202923.GR939@shogun.local> References: <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> <20160606215509.GD92333@icir.org> <97DACE92-DA36-4B27-A4B8-1C064898E113@illinois.edu> <20160607034648.GH92333@icir.org> <0FD71378-4677-4331-A80D-C3705F9C04F3@illinois.edu> <20160609030156.GD9283@icir.org> <3DCD02A9-B42C-479E-B300-DC3AA968C9BC@illinois.edu> <20160609202923.GR939@shogun.local> Message-ID: <84110F6F-DE00-41FF-B473-CC82F1231E30@illinois.edu> > On Jun 9, 2016, at 3:29 PM, Matthias Vallentin wrote: > > I'm not sure if I understand the -community suffix. The client bro-pkg > makes sense to me. But the first association I have with > bro-pkg-community is a community-version of bro-pkg (i.e., the client). Was mostly thinking it?s nice to indicate some way in the name that the contents of the repo aren?t authored by the bro team. Doesn?t have to be a suffix, could be ?bro-community-packages? or ?community-packages?. > How about just "packages?" Forking github.com/bro/packages feels like a > natural and understandable descriptor. To me, a URL like that has a subtle implication that the packages are authored by the ?bro? account, but that could be easy to clarify in the repo description: ?These are packages contributed by the community?. I like the ?packages? + ?package-manager? combo that Johanna suggests. - Jon From vallentin at icir.org Thu Jun 9 16:26:07 2016 From: vallentin at icir.org (Matthias Vallentin) Date: Thu, 9 Jun 2016 16:26:07 -0700 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <84110F6F-DE00-41FF-B473-CC82F1231E30@illinois.edu> References: <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> <20160606215509.GD92333@icir.org> <97DACE92-DA36-4B27-A4B8-1C064898E113@illinois.edu> <20160607034648.GH92333@icir.org> <0FD71378-4677-4331-A80D-C3705F9C04F3@illinois.edu> <20160609030156.GD9283@icir.org> <3DCD02A9-B42C-479E-B300-DC3AA968C9BC@illinois.edu> <20160609202923.GR939@shogun.local> <84110F6F-DE00-41FF-B473-CC82F1231E30@illinois.edu> Message-ID: <20160609232607.GA37381@shogun.local> > I like the ?packages? + ?package-manager? combo that Johanna suggests. +1 Matthias From jan.grashoefer at gmail.com Fri Jun 10 10:08:58 2016 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Fri, 10 Jun 2016 19:08:58 +0200 Subject: [Bro-Dev] Configurable &write_expire interval In-Reply-To: References: <57050AFC.1000609@gmail.com> Message-ID: <9e4a6179-fbe3-c38f-992c-cee98c8f5316@gmail.com> As there was no feedback, I decided to use a bif (see https://github.com/bro/bro/commit/16b1032beeaaf681763785ddac1eed4128430965). It might not be the cleanest solution with respect to the bro language but it is a straight forward approach. From asharma at lbl.gov Fri Jun 10 11:05:55 2016 From: asharma at lbl.gov (Aashish Sharma) Date: Fri, 10 Jun 2016 11:05:55 -0700 Subject: [Bro-Dev] Configurable &write_expire interval In-Reply-To: References: <57050AFC.1000609@gmail.com> Message-ID: <20160610180552.GI21101@mac-4.local> HI Jan, > > A solution could be to evaluate the interval expression every time it is > > used inside the table implementation. The drawback would be that there For all of my needs above has worked fairly well. including using exp_val= 0 secs as default. Based on the value of item in the table, I could return a variable time back from the expire function thus either keeping it longer or expiring it. > > used inside the table implementation. The drawback would be that there > > is no fixed value for serialization (I am not sure about the effects Sorry, I don't quite seem to understand this drawback. Aashish On Wed, Jun 08, 2016 at 08:30:14PM +0200, Jan Grash?fer wrote: > My explanations might be hard to follow without examples. So I am adding > some pseudo code: > > > I ran into an issue while trying to make the &write_expire interval > > configurable: Using a redefable constant does not work here, as the > > expression only gets evaluated when the table is initialized and thus > > later redefs do not influence the value. > > # base script: > const exp_val = 5min &redef; > data: table[addr] of string &write_expire=exp_val; > > # user script: > redef exp_val = 20min; # has no effect > > > I thought about circumventing > > this by setting the value to 0 and maintain an extra variable to check > > against in my expire_func and return the right value. Unfortunately this > > won't work with write/read_expire as a write or read will reset the > > expiration to the initial value of 0. > > # base script: > const exp_val = 5min &redef; > > function do_exp(data: table[addr] of string, idx: addr): interval > { > if ( is_first_call() ) > return exp_val; > # in case of a write, expire timer will be reset to 0 > else > ... > } > > data: table[addr] of string &write_expire=0 expire_func=do_exp; > > > A solution could be to evaluate the interval expression every time it is > > used inside the table implementation. The drawback would be that there > > is no fixed value for serialization (I am not sure about the effects > > here). Another solution would be to provide a bif (or implement a > > language feature) to change the expire_time value from inside the > > expire_func. > > # base script: > function do_exp(data: table[addr] of string, idx: addr): interval > { > if ( is_first_call() ) > expire exp_val; # sets expire timer instead of delay > else > ... > } > > > There was a somehow similar discussion about per item expiration (see > > http://mailman.icsi.berkeley.edu/pipermail/bro-dev/2016-April/011731.html) > > in which Robin came up with the solution of multiple tables with > > different expiration values. Again this would be a solution but doesn't > > feel right (duplicate code, static and somehow counterintuitive for the > > user). > > # base script: > type exp_interval enum { ei1m, ei10m, ... }; > const exp_val = ei1m &redef; > > data1m: table[addr] of string &write_expire=1min; > data10m: table[addr] of string &write_expire=10min; > ... > data1d: table[addr] of string &write_expire=1day; > > function insert(...) > { > switch ( exp_val ) > { > case ei1m: > data1m[...] = ... > break; > case ei10m: > data10m[...] = ... > break; > ... > } > } > > # user script: > redef exp_val = ei30m; > > > Maybe I am missing something regarding the loading sequence of scripts > > and this problem could be solved easier. So I am open for any > > suggestions or feedback! > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev From jan.grashoefer at gmail.com Fri Jun 10 16:28:16 2016 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Sat, 11 Jun 2016 01:28:16 +0200 Subject: [Bro-Dev] Configurable &write_expire interval In-Reply-To: <20160610180552.GI21101@mac-4.local> References: <57050AFC.1000609@gmail.com> <20160610180552.GI21101@mac-4.local> Message-ID: Hi Aashish, > For all of my needs above has worked fairly well. including using exp_val= 0 secs as default. > > Based on the value of item in the table, I could return a variable time back from the expire function thus either keeping it longer or expiring it. Trying this, I ran into an issue with &write_expire: 1) I set &write_expire=0 min 2) In the first call of the expire_func I return a delay, e.g. 10 min 3) After e.g. 5 min there is a write, the timer is set back to 0 min (instead of 10 min) and the item immediately expires. >From my understanding, this is a result of how the delay is implemented. As far as I understand the code, a delay is realized by setting the time of the last access so that the next expiration occurs exactly after the given delay. The expiration value isn't touched. As a consequence, accessing the item within the delay time window will reset the expiration to its original value. To be clear: I think that's perfectly fine and works how I would expect it to work. It just prevents the 0-expire workaround for read/write_expire. >>> used inside the table implementation. The drawback would be that there >>> is no fixed value for serialization (I am not sure about the effects > > Sorry, I don't quite seem to understand this drawback. That's an implementation detail again. The expression given with the expire attribute is evaluated and stored when the table value is created. Thus using a const and doing a later redef doesn't lead to the intended behavior. Meanwhile I realized that the expire_func expression is also serialized. I assume this would be an option for the expire value, too. However, I could miss something. Best regards, Jan From robin at icir.org Fri Jun 10 20:52:01 2016 From: robin at icir.org (Robin Sommer) Date: Fri, 10 Jun 2016 21:52:01 -0600 Subject: [Bro-Dev] Configurable &write_expire interval In-Reply-To: References: <57050AFC.1000609@gmail.com> Message-ID: <20160611035201.GJ1139@icir.org> Sorry for being late to the party here. Your examples did actually help, I'm seeing now that the current behavior really doesn't seem right: > const exp_val = 5min &redef; > data: table[addr] of string &write_expire=exp_val; > > # user script: > redef exp_val = 20min; # has no effect I'd call this a bug actually. Redefs are supposed to take effect before anything else, so having the timeout use the original value here seems quite wrong. My immediate thought (without looking at the code) would be delaying evaluating the expression until the value is needed for the first time. At that point, the redef will have taken effect, so we should be fine. Essentially, we'd cache the evaluated value for the future once we have it. Serialization is an interesting question though. I believe there'd be nothing wrong with simply serializing the expression itself here (rather than its value). When deserializing, we'd restore the expression and make sure the cache value remains unset, so that on first use it will get evaluated. In principle, we could even go further and allow a non-constant expression for the timeout that would get evaluated every time. My main concern there would be performance, although I'm not sure if that would actually cause much overhead in the still most common case of a constant (i.e., for existing scripts). Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jan.grashoefer at gmail.com Sat Jun 11 06:36:45 2016 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Sat, 11 Jun 2016 15:36:45 +0200 Subject: [Bro-Dev] Configurable &write_expire interval In-Reply-To: <20160611035201.GJ1139@icir.org> References: <57050AFC.1000609@gmail.com> <20160611035201.GJ1139@icir.org> Message-ID: > I'd call this a bug actually. Redefs are supposed to take effect > before anything else, so having the timeout use the original value > here seems quite wrong. I agree that the behavior is at least counterintuitive :) > My immediate thought (without looking at the code) would be delaying > evaluating the expression until the value is needed for the first > time. At that point, the redef will have taken effect, so we should be > fine. Essentially, we'd cache the evaluated value for the future once > we have it. I will have a look. If I am able to fix this I will include this in the pull request for the intel updates. > Serialization is an interesting question though. I believe there'd be > nothing wrong with simply serializing the expression itself here > (rather than its value). When deserializing, we'd restore the > expression and make sure the cache value remains unset, so that on > first use it will get evaluated. Yep, meanwhile I realized serialization is already done for the expire_func statement. In case the table is serialized having a cached value set, it would be preferable to use this value, wouldn't it? > In principle, we could even go further and allow a non-constant > expression for the timeout that would get evaluated every time. My > main concern there would be performance, although I'm not sure if that > would actually cause much overhead in the still most common case of a > constant (i.e., for existing scripts). I am not sure what the actual performance impact would be. My idea would be to cache the value in case of a constant and evaluate it every time otherwise. That should combine the best of both approaches. Best regards, Jan From robin at icir.org Sat Jun 11 08:10:51 2016 From: robin at icir.org (Robin Sommer) Date: Sat, 11 Jun 2016 08:10:51 -0700 Subject: [Bro-Dev] Configurable &write_expire interval In-Reply-To: References: <57050AFC.1000609@gmail.com> <20160611035201.GJ1139@icir.org> Message-ID: <20160611151051.GA83689@icir.org> On Sat, Jun 11, 2016 at 15:36 +0200, you wrote: > I will have a look. If I am able to fix this I will include this in the > pull request for the intel updates. Please create a separate pull request for this one first (you can merge it into the intel update branch, that'll be fine). > expire_func statement. In case the table is serialized having a cached > value set, it would be preferable to use this value, wouldn't it? It's a question of semantics: what should happen if the Bro unserializing it has redef'ed the constant to a different value? I think my intuition would expect to use that modified value after unserialization. > I am not sure what the actual performance impact would be. My idea would > be to cache the value in case of a constant and evaluate it every time > otherwise. That should combine the best of both approaches. Yeah, I was thinking about that too. I'd still be curious if the overhead of re-evaluating the constant overhead becomes noticable. If you're game, you could try a little benchmark just manipulating a table plenty times and measure if that changes execution time much. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jan.grashoefer at gmail.com Mon Jun 13 12:41:55 2016 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Mon, 13 Jun 2016 21:41:55 +0200 Subject: [Bro-Dev] Configurable &write_expire interval In-Reply-To: <20160611151051.GA83689@icir.org> References: <57050AFC.1000609@gmail.com> <20160611035201.GJ1139@icir.org> <20160611151051.GA83689@icir.org> Message-ID: <35cb1370-8b6e-6bc2-1ed4-c9176cfaec42@gmail.com> > Please create a separate pull request for this one first (you can > merge it into the intel update branch, that'll be fine). I have opened a new pull request :) >> expire_func statement. In case the table is serialized having a cached >> value set, it would be preferable to use this value, wouldn't it? > > It's a question of semantics: what should happen if the Bro > unserializing it has redef'ed the constant to a different value? I > think my intuition would expect to use that modified value after > unserialization. My intuition would be that the deserialized table is identical to the serialized one. However, this is a matter of style and shouldn't be relevant now... > Yeah, I was thinking about that too. I'd still be curious if the > overhead of re-evaluating the constant overhead becomes noticable. If > you're game, you could try a little benchmark just manipulating a > table plenty times and measure if that changes execution time much. ... I did a quick measurement of the execution time for re-evaluation of a constant and couldn't detect any performance impact. So I went for this solution. A side note: I suspect that the table syncing did and still does not work as reliable as one would expect. But according to Johanna this will become deprecated soon, so I did not touch that code. Best regards, Jan From robin at icir.org Mon Jun 13 16:57:32 2016 From: robin at icir.org (Robin Sommer) Date: Mon, 13 Jun 2016 16:57:32 -0700 Subject: [Bro-Dev] Configurable &write_expire interval In-Reply-To: <35cb1370-8b6e-6bc2-1ed4-c9176cfaec42@gmail.com> References: <57050AFC.1000609@gmail.com> <20160611035201.GJ1139@icir.org> <20160611151051.GA83689@icir.org> <35cb1370-8b6e-6bc2-1ed4-c9176cfaec42@gmail.com> Message-ID: <20160613235732.GJ71212@icir.org> On Mon, Jun 13, 2016 at 21:41 +0200, you wrote: > A side note: I suspect that the table syncing did and still does not > work as reliable as one would expect. But according to Johanna this will > become deprecated soon, so I did not touch that code. If there's any obvious problem, we should still take a look, but more generally: yeah Broker will take that over as soon as things are in sufficient shape. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jan.grashoefer at gmail.com Mon Jun 13 17:35:40 2016 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Tue, 14 Jun 2016 02:35:40 +0200 Subject: [Bro-Dev] Configurable &write_expire interval In-Reply-To: <20160613235732.GJ71212@icir.org> References: <57050AFC.1000609@gmail.com> <20160611035201.GJ1139@icir.org> <20160611151051.GA83689@icir.org> <35cb1370-8b6e-6bc2-1ed4-c9176cfaec42@gmail.com> <20160613235732.GJ71212@icir.org> Message-ID: <497faa82-5ef0-fc92-be6c-f04965291e49@gmail.com> >> A side note: I suspect that the table syncing did and still does not >> work as reliable as one would expect. But according to Johanna this will >> become deprecated soon, so I did not touch that code. > > If there's any obvious problem, we should still take a look, but more > generally: yeah Broker will take that over as soon as things are in > sufficient shape. The thing is I don't get why reads only need to be propagated once per (half) &read_expire interval (see https://github.com/bro/bro/blob/master/src/Val.cc#L2326). If there was a read at the beginning of the (half) interval and another one close to the end, a peer might expire a value too early based on the first read. Seems someone put some thought into this so maybe I miss something here :) Best regards, Jan From slagell at illinois.edu Tue Jun 14 08:58:31 2016 From: slagell at illinois.edu (Slagell, Adam J) Date: Tue, 14 Jun 2016 15:58:31 +0000 Subject: [Bro-Dev] Something to think about as we build Bro Package Manager Message-ID: <35B057F5-5497-4F40-89F5-2DB80E486793@illinois.edu> http://arstechnica.com/security/2016/06/college-student-schools-govs-and-mils-on-perils-of-arbitrary-code-execution/ ------ Adam J. Slagell Chief Information Security Officer Director, Cybersecurity Division National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From seth at icir.org Tue Jun 14 22:11:07 2016 From: seth at icir.org (Seth Hall) Date: Wed, 15 Jun 2016 01:11:07 -0400 Subject: [Bro-Dev] New proposal (Re: CBAN naming) In-Reply-To: <84110F6F-DE00-41FF-B473-CC82F1231E30@illinois.edu> References: <20160606155307.GN92333@icir.org> <20160606185032.GU92333@icir.org> <07ED1AE1-0463-483C-A429-34A8E84211DA@illinois.edu> <20160606215509.GD92333@icir.org> <97DACE92-DA36-4B27-A4B8-1C064898E113@illinois.edu> <20160607034648.GH92333@icir.org> <0FD71378-4677-4331-A80D-C3705F9C04F3@illinois.edu> <20160609030156.GD9283@icir.org> <3DCD02A9-B42C-479E-B300-DC3AA968C9BC@illinois.edu> <20160609202923.GR939@shogun.local> <84110F6F-DE00-41FF-B473-CC82F1231E30@illinois.edu> Message-ID: <580AC4DD-43FB-44E6-94B5-7419AFF25CA4@icir.org> > On Jun 9, 2016, at 5:32 PM, Siwek, Jon wrote: > > I like the ?packages? + ?package-manager? combo that Johanna suggests. I like that too. It feels nice and clean. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From seth at icir.org Tue Jun 14 22:23:40 2016 From: seth at icir.org (Seth Hall) Date: Wed, 15 Jun 2016 01:23:40 -0400 Subject: [Bro-Dev] CBAN naming In-Reply-To: References: <20160605155547.GE92333@icir.org> <0FC5D616-0715-4015-8BCE-20D74BB10619@illinois.edu> Message-ID: <32D1FCCE-111E-401F-9A03-526317DC102F@icir.org> > On Jun 6, 2016, at 3:09 PM, Siwek, Jon wrote: > > If we switch the design to instead user the super-container format, then that?s not an issue for me anymore because the relationship changes from ?is a plugin? to ?may have a plugin?. I like the positioning of this because suddenly it suddenly feels very natural to explain the contents of a package (or whatever it ends up getting called). .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From robin at icir.org Fri Jun 17 08:57:02 2016 From: robin at icir.org (Robin Sommer) Date: Fri, 17 Jun 2016 08:57:02 -0700 Subject: [Bro-Dev] Configurable &write_expire interval In-Reply-To: <497faa82-5ef0-fc92-be6c-f04965291e49@gmail.com> References: <57050AFC.1000609@gmail.com> <20160611035201.GJ1139@icir.org> <20160611151051.GA83689@icir.org> <35cb1370-8b6e-6bc2-1ed4-c9176cfaec42@gmail.com> <20160613235732.GJ71212@icir.org> <497faa82-5ef0-fc92-be6c-f04965291e49@gmail.com> Message-ID: <20160617155702.GD52076@icir.org> On Tue, Jun 14, 2016 at 02:35 +0200, you wrote: > The thing is I don't get why reads only need to be propagated once per > (half) &read_expire interval > Seems someone put some thought into this so maybe I miss something > here :) That must have been me. :-) I need to look at that for a bit to see if I can remember my reasoning from many years ago (which might very well have been flawed!). Please file a ticket with your thoughts and assign it to me. Thanks, Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jan.grashoefer at gmail.com Fri Jun 17 15:53:47 2016 From: jan.grashoefer at gmail.com (=?UTF-8?Q?Jan_Grash=c3=b6fer?=) Date: Sat, 18 Jun 2016 00:53:47 +0200 Subject: [Bro-Dev] Configurable &write_expire interval In-Reply-To: <20160617155702.GD52076@icir.org> References: <57050AFC.1000609@gmail.com> <20160611035201.GJ1139@icir.org> <20160611151051.GA83689@icir.org> <35cb1370-8b6e-6bc2-1ed4-c9176cfaec42@gmail.com> <20160613235732.GJ71212@icir.org> <497faa82-5ef0-fc92-be6c-f04965291e49@gmail.com> <20160617155702.GD52076@icir.org> Message-ID: <27fed1c7-4286-1917-b9ce-62e1ea267500@gmail.com> > That must have been me. :-) I need to look at that for a bit to see if > I can remember my reasoning from many years ago (which might very well > have been flawed!). Please file a ticket with your thoughts and assign > it to me. Thanks, I have created BIT-1631 but couldn't assign it to you. Either I am just unable to find the right button or I don't have assign privileges :) Jan From seth at icir.org Thu Jun 23 09:48:03 2016 From: seth at icir.org (Seth Hall) Date: Thu, 23 Jun 2016 12:48:03 -0400 Subject: [Bro-Dev] Bro plugins + broctl plugins? Message-ID: Has any movement been made on the ability to add broctl plugins into bro plugins? I know we talked about it a few times, and it's sort of becoming necessary are more packet source plugins are showing up in the bro-plugins repository. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From slagell at illinois.edu Thu Jun 23 09:52:50 2016 From: slagell at illinois.edu (Slagell, Adam J) Date: Thu, 23 Jun 2016 16:52:50 +0000 Subject: [Bro-Dev] Bro plugins + broctl plugins? In-Reply-To: References: Message-ID: <12F17207-CE66-4AA5-8E8F-8F59B1E30C6D@illinois.edu> https://bro-tracker.atlassian.net/browse/BIT-1551 > On Jun 23, 2016, at 11:48 AM, Seth Hall wrote: > > Has any movement been made on the ability to add broctl plugins into bro plugins? I know we talked about it a few times, and it's sort of becoming necessary are more packet source plugins are showing up in the bro-plugins repository. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > > > _______________________________________________ > bro-dev mailing list > bro-dev at bro.org > http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev ------ Adam J. Slagell Chief Information Security Officer Director, Cybersecurity Division National Center for Supercomputing Applications University of Illinois at Urbana-Champaign www.slagell.info "Under the Illinois Freedom of Information Act (FOIA), any written communication to or from University employees regarding University business is a public record and may be subject to public disclosure." From seth at icir.org Thu Jun 23 10:39:50 2016 From: seth at icir.org (Seth Hall) Date: Thu, 23 Jun 2016 13:39:50 -0400 Subject: [Bro-Dev] Bro plugins + broctl plugins? In-Reply-To: <12F17207-CE66-4AA5-8E8F-8F59B1E30C6D@illinois.edu> References: <12F17207-CE66-4AA5-8E8F-8F59B1E30C6D@illinois.edu> Message-ID: <90EFE7E8-90BC-4014-9EE7-3BCDDE5DE158@icir.org> > On Jun 23, 2016, at 12:52 PM, Slagell, Adam J wrote: > > https://bro-tracker.atlassian.net/browse/BIT-1551 Great! Thanks Adam (and Daniel!). .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From seth at icir.org Thu Jun 23 11:30:41 2016 From: seth at icir.org (Seth Hall) Date: Thu, 23 Jun 2016 14:30:41 -0400 Subject: [Bro-Dev] Bro plugins + broctl plugins? In-Reply-To: <12F17207-CE66-4AA5-8E8F-8F59B1E30C6D@illinois.edu> References: <12F17207-CE66-4AA5-8E8F-8F59B1E30C6D@illinois.edu> Message-ID: <86E38AED-B6F9-4080-8337-A07D9A58E0B9@icir.org> > On Jun 23, 2016, at 12:52 PM, Slagell, Adam J wrote: > > https://bro-tracker.atlassian.net/browse/BIT-1551 I reopened this ticket since it looks like it was wrongly closed. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From dnthayer at illinois.edu Thu Jun 23 11:33:33 2016 From: dnthayer at illinois.edu (Daniel Thayer) Date: Thu, 23 Jun 2016 13:33:33 -0500 Subject: [Bro-Dev] Bro plugins + broctl plugins? In-Reply-To: <86E38AED-B6F9-4080-8337-A07D9A58E0B9@icir.org> References: <12F17207-CE66-4AA5-8E8F-8F59B1E30C6D@illinois.edu> <86E38AED-B6F9-4080-8337-A07D9A58E0B9@icir.org> Message-ID: <576C2B7D.5050705@illinois.edu> Could you specify what the problem is with the current implementation in git master? On 06/23/2016 01:30 PM, Seth Hall wrote: > >> On Jun 23, 2016, at 12:52 PM, Slagell, Adam J wrote: >> >> https://bro-tracker.atlassian.net/browse/BIT-1551 > > I reopened this ticket since it looks like it was wrongly closed. > > .Seth > > -- > Seth Hall > International Computer Science Institute > (Bro) because everyone has a network > http://www.bro.org/ > From seth at icir.org Thu Jun 23 12:27:47 2016 From: seth at icir.org (Seth Hall) Date: Thu, 23 Jun 2016 15:27:47 -0400 Subject: [Bro-Dev] Bro plugins + broctl plugins? In-Reply-To: <576C2B7D.5050705@illinois.edu> References: <12F17207-CE66-4AA5-8E8F-8F59B1E30C6D@illinois.edu> <86E38AED-B6F9-4080-8337-A07D9A58E0B9@icir.org> <576C2B7D.5050705@illinois.edu> Message-ID: <4ED3031B-E05C-4BD9-9D9C-A89F89FCE599@icir.org> > On Jun 23, 2016, at 2:33 PM, Daniel Thayer wrote: > > Could you specify what the problem is with the current implementation in git master? Oh, I looked for a commit that implements this and couldn't find it. Adam didn't seem to indicate that the work had been done when he closed it either so I thought it might have been a mistake. .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From seth at icir.org Thu Jun 23 12:29:34 2016 From: seth at icir.org (Seth Hall) Date: Thu, 23 Jun 2016 15:29:34 -0400 Subject: [Bro-Dev] Bro plugins + broctl plugins? In-Reply-To: <576C2B7D.5050705@illinois.edu> References: <12F17207-CE66-4AA5-8E8F-8F59B1E30C6D@illinois.edu> <86E38AED-B6F9-4080-8337-A07D9A58E0B9@icir.org> <576C2B7D.5050705@illinois.edu> Message-ID: <0CB00D5F-D91B-4E01-A9BC-81A5F30559F2@icir.org> > On Jun 23, 2016, at 2:33 PM, Daniel Thayer wrote: > > Could you specify what the problem is with the current implementation in git master? I see it now. I'll close again. Thanks! .Seth -- Seth Hall International Computer Science Institute (Bro) because everyone has a network http://www.bro.org/ From vallentin at icir.org Tue Jun 28 12:27:04 2016 From: vallentin at icir.org (Matthias Vallentin) Date: Tue, 28 Jun 2016 12:27:04 -0700 Subject: [Bro-Dev] [Broker] multi-topic subscriptions Message-ID: <20160628192704.GC6009@samurai.ICIR.org> If a broker endpoint subscribes to multiple topics, how many messages do you expect to receive? Consider this snippet: context ctx; auto e = ctx.spawn(); e.subscribe("/foo"); e.subscribe("/foo/bar"); e.subscribe("/foo/bar/baz"); e.publish("/foo/bar/baz", 4.2); Should the endpoint receive exactly one message or the same message three times? Personally, I think use exactly-once delivery semantics would be most intuitive. Matthias From robin at icir.org Wed Jun 29 07:53:10 2016 From: robin at icir.org (Robin Sommer) Date: Wed, 29 Jun 2016 07:53:10 -0700 Subject: [Bro-Dev] [Broker] multi-topic subscriptions In-Reply-To: <20160628192704.GC6009@samurai.ICIR.org> References: <20160628192704.GC6009@samurai.ICIR.org> Message-ID: <20160629145310.GB4516@icir.org> On Tue, Jun 28, 2016 at 12:27 -0700, you wrote: > Personally, I think use exactly-once delivery semantics > would be most intuitive. Yeah, seems most intuitive to me too. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin