From noreply at bro.org Fri May 1 00:00:25 2015 From: noreply at bro.org (Merge Tracker) Date: Fri, 1 May 2015 00:00:25 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201505010700.t4170PP7005732@bro-ids.icir.org> Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- --------------- ---------- ------------------------------------------------------------ 26007f4 [1] bro Daniel Thayer 2015-04-29 Update usage output and list of cmd-line options cb91a9c [2] bro Vlad Grigorescu 2015-04-29 A small fix to ssh/geo-data.bro. ssh can now be unset for lo [1] 26007f4 https://github.com/bro/bro/commit/26007f419ee51160717588f34ca7930f831a3761 [2] cb91a9c https://github.com/bro/bro/commit/cb91a9c10157f8fc01397f5f638595e35231e283 From noreply at bro.org Sat May 2 00:00:46 2015 From: noreply at bro.org (Merge Tracker) Date: Sat, 2 May 2015 00:00:46 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201505020700.t4270kcV001474@bro-ids.icir.org> Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- --------------- ---------- ------------------------------------------------------------ 26007f4 [1] bro Daniel Thayer 2015-04-29 Update usage output and list of cmd-line options cb91a9c [2] bro Vlad Grigorescu 2015-04-29 A small fix to ssh/geo-data.bro. ssh can now be unset for lo [1] 26007f4 https://github.com/bro/bro/commit/26007f419ee51160717588f34ca7930f831a3761 [2] cb91a9c https://github.com/bro/bro/commit/cb91a9c10157f8fc01397f5f638595e35231e283 From noreply at bro.org Sun May 3 00:00:28 2015 From: noreply at bro.org (Merge Tracker) Date: Sun, 3 May 2015 00:00:28 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201505030700.t4370S1a029061@bro-ids.icir.org> Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- --------------- ---------- ------------------------------------------------------------ 26007f4 [1] bro Daniel Thayer 2015-04-29 Update usage output and list of cmd-line options cb91a9c [2] bro Vlad Grigorescu 2015-04-29 A small fix to ssh/geo-data.bro. ssh can now be unset for lo [1] 26007f4 https://github.com/bro/bro/commit/26007f419ee51160717588f34ca7930f831a3761 [2] cb91a9c https://github.com/bro/bro/commit/cb91a9c10157f8fc01397f5f638595e35231e283 From noreply at bro.org Mon May 4 00:00:25 2015 From: noreply at bro.org (Merge Tracker) Date: Mon, 4 May 2015 00:00:25 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201505040700.t4470PUt016350@bro-ids.icir.org> Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- --------------- ---------- ------------------------------------------------------------ 26007f4 [1] bro Daniel Thayer 2015-04-29 Update usage output and list of cmd-line options cb91a9c [2] bro Vlad Grigorescu 2015-04-29 A small fix to ssh/geo-data.bro. ssh can now be unset for lo [1] 26007f4 https://github.com/bro/bro/commit/26007f419ee51160717588f34ca7930f831a3761 [2] cb91a9c https://github.com/bro/bro/commit/cb91a9c10157f8fc01397f5f638595e35231e283 From noreply at bro.org Tue May 5 00:00:25 2015 From: noreply at bro.org (Merge Tracker) Date: Tue, 5 May 2015 00:00:25 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201505050700.t4570PYC023022@bro-ids.icir.org> Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ------------------------------ f624899 [1] bro Daniel Thayer 2015-05-04 Add /sbin to PATH in btest.cfg [1] f624899 https://github.com/bro/bro/commit/f6248994e400ee95da4114dfb19551c218744aa4 From jira at bro-tracker.atlassian.net Tue May 5 06:55:00 2015 From: jira at bro-tracker.atlassian.net (Derek Ditch (JIRA)) Date: Tue, 5 May 2015 08:55:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1392) CPack brocontrol package clashes with file paths from bro-minimal package In-Reply-To: References: Message-ID: Derek Ditch created BIT-1392: -------------------------------- Summary: CPack brocontrol package clashes with file paths from bro-minimal package Key: BIT-1392 URL: https://bro-tracker.atlassian.net/browse/BIT-1392 Project: Bro Issue Tracker Issue Type: Problem Components: Bro, BroControl Affects Versions: git/master Reporter: Derek Ditch I've been building bro packages from git/master (as of today), and with EL7, yum enforces file path ownership to packages. Currently, all the packages checkout the same instance of cmake scripts, namely 'ConfigurePackaging.cmake'. Currently, this script excludes '/opt /var /var/opt'. When building brocontrol (and possibly issues with broccoli), you have to exclude paths that the bro package already owns. Namely, I had to change the exclusion line to the following to make it work: set(CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION /opt /var /var/opt /opt/bro/share/man/man8 /opt/bro/share/man /opt/bro/share /opt/bro ) I don't know how you would like to handle this. Obviously, it makes sense to have a common cmake script repo... maybe move this definition to the configure scripts for each component so that it's picked up in the initial cmake run. Also, it's worth noting that you have to use CMake >= 2.8.12 for this definition to actually work -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From noreply at bro.org Wed May 6 00:00:20 2015 From: noreply at bro.org (Merge Tracker) Date: Wed, 6 May 2015 00:00:20 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201505060700.t4670KZ1019232@bro-ids.icir.org> Open Fastpath Commits ====================== Commit Component Author Date Summary ----------- ----------- ------------- ---------- ------------------------------ f624899 [1] bro Daniel Thayer 2015-05-04 Add /sbin to PATH in btest.cfg [1] f624899 https://github.com/bro/bro/commit/f6248994e400ee95da4114dfb19551c218744aa4 From jira at bro-tracker.atlassian.net Wed May 6 15:47:00 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 6 May 2015 17:47:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1393) The --with-python configure option only partially works In-Reply-To: References: Message-ID: Daniel Thayer created BIT-1393: ---------------------------------- Summary: The --with-python configure option only partially works Key: BIT-1393 URL: https://bro-tracker.atlassian.net/browse/BIT-1393 Project: Bro Issue Tracker Issue Type: Problem Components: BroControl, trace-summary Reporter: Daniel Thayer Fix For: 2.4 When building bro with the "--with-python" configure option, some files still use the default python interpreter. This prevents, for example, BroControl from working on RHEL 5 systems. -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Wed May 6 15:47:00 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Wed, 6 May 2015 17:47:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1393) The --with-python configure option only partially works In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer reassigned BIT-1393: ---------------------------------- Assignee: Daniel Thayer > The --with-python configure option only partially works > ------------------------------------------------------- > > Key: BIT-1393 > URL: https://bro-tracker.atlassian.net/browse/BIT-1393 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl, trace-summary > Reporter: Daniel Thayer > Assignee: Daniel Thayer > Fix For: 2.4 > > > When building bro with the "--with-python" configure option, some files > still use the default python interpreter. This prevents, for example, BroControl > from working on RHEL 5 systems. -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 7 08:06:00 2015 From: jira at bro-tracker.atlassian.net (Ventz Petkov (JIRA)) Date: Thu, 7 May 2015 10:06:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1394) Github commit seems to have possible configure issues? In-Reply-To: References: Message-ID: Ventz Petkov created BIT-1394: --------------------------------- Summary: Github commit seems to have possible configure issues? Key: BIT-1394 URL: https://bro-tracker.atlassian.net/browse/BIT-1394 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: git/master Environment: * Dell C6220 (PER blade: 128GB of ram | 2 socket | 16 cores per socket | 2-10G cards) * Ubuntu 14.04.2 LTS system * Feeding in 20Gb/s links * PF_RING-6.0.3 compiled into /opt/pfring Packages installed from base (other than SSH during select-install): build-essential libnuma-dev pkg-config cmake make gcc g++ swig flex bison libpcap-dev libgeoip-dev libssl-dev python-dev zlib1g-dev libmagic-dev swig2.0 libgoogle-perftools-dev google-perftools libxml2-dev libcurl4-gnutls-dev mailutils Have also added GeoIP databases manually. pfring loaded with modprobe: modprobe pf_ring enable_tx_capture=0 min_num_slots=32768 Reporter: Ventz Petkov Priority: Low Attachments: CMakeOutput.log When checking out latest master branch (https://github.com/bro/bro/commit/1e66c6718a98675fb838205a5e55220e9794eeb7), and given the above environment, error at configure: ######################################################## bro# ./configure --with-pcap=/opt/pfring Build Directory : build Source Directory: /root/install/bro CMake Error at CMakeLists.txt:7 (include): include could not find load file: cmake/CommonCMakeConfig.cmake CMake Error at CMakeLists.txt:52 (include): include could not find load file: FindRequiredPackage -- Found sed: /bin/sed CMake Error at CMakeLists.txt:64 (FindRequiredPackage): Unknown CMake command "FindRequiredPackage". -- Configuring incomplete, errors occurred! See also "/root/install/bro/build/CMakeFiles/CMakeOutput.log". ######################################################## Attaching log file "CMakeOutput.log" -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 7 08:13:00 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Thu, 7 May 2015 10:13:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1394) Github commit seems to have possible configure issues? In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20600#comment-20600 ] Daniel Thayer commented on BIT-1394: ------------------------------------ Looks like you probably forgot to use the "--recursive" flag when you did the "git clone". > Github commit seems to have possible configure issues? > ------------------------------------------------------ > > Key: BIT-1394 > URL: https://bro-tracker.atlassian.net/browse/BIT-1394 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: * Dell C6220 (PER blade: 128GB of ram | 2 socket | 16 cores per socket | 2-10G cards) > * Ubuntu 14.04.2 LTS system > * Feeding in 20Gb/s links > * PF_RING-6.0.3 compiled into /opt/pfring > Packages installed from base (other than SSH during select-install): > build-essential libnuma-dev pkg-config cmake make gcc g++ swig flex bison libpcap-dev libgeoip-dev libssl-dev python-dev zlib1g-dev libmagic-dev swig2.0 libgoogle-perftools-dev google-perftools libxml2-dev libcurl4-gnutls-dev mailutils > Have also added GeoIP databases manually. > pfring loaded with modprobe: > modprobe pf_ring enable_tx_capture=0 min_num_slots=32768 > Reporter: Ventz Petkov > Priority: Low > Attachments: CMakeOutput.log > > > When checking out latest master branch (https://github.com/bro/bro/commit/1e66c6718a98675fb838205a5e55220e9794eeb7), and given the above environment, error at configure: > ######################################################## > bro# ./configure --with-pcap=/opt/pfring > Build Directory : build > Source Directory: /root/install/bro > CMake Error at CMakeLists.txt:7 (include): > include could not find load file: > cmake/CommonCMakeConfig.cmake > CMake Error at CMakeLists.txt:52 (include): > include could not find load file: > FindRequiredPackage > -- Found sed: /bin/sed > CMake Error at CMakeLists.txt:64 (FindRequiredPackage): > Unknown CMake command "FindRequiredPackage". > -- Configuring incomplete, errors occurred! > See also "/root/install/bro/build/CMakeFiles/CMakeOutput.log". > ######################################################## > Attaching log file "CMakeOutput.log" -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 7 08:32:00 2015 From: jira at bro-tracker.atlassian.net (Ventz Petkov (JIRA)) Date: Thu, 7 May 2015 10:32:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1394) Github commit seems to have possible configure issues? In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20601#comment-20601 ] Ventz Petkov commented on BIT-1394: ----------------------------------- Getting this with a recursive clone, but I just noticed that the sub-modules are pointed to git:// (vs http/s) Everything is behind a http/s proxy, so that might be the issue (missing sub-modules) > Github commit seems to have possible configure issues? > ------------------------------------------------------ > > Key: BIT-1394 > URL: https://bro-tracker.atlassian.net/browse/BIT-1394 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: * Dell C6220 (PER blade: 128GB of ram | 2 socket | 16 cores per socket | 2-10G cards) > * Ubuntu 14.04.2 LTS system > * Feeding in 20Gb/s links > * PF_RING-6.0.3 compiled into /opt/pfring > Packages installed from base (other than SSH during select-install): > build-essential libnuma-dev pkg-config cmake make gcc g++ swig flex bison libpcap-dev libgeoip-dev libssl-dev python-dev zlib1g-dev libmagic-dev swig2.0 libgoogle-perftools-dev google-perftools libxml2-dev libcurl4-gnutls-dev mailutils > Have also added GeoIP databases manually. > pfring loaded with modprobe: > modprobe pf_ring enable_tx_capture=0 min_num_slots=32768 > Reporter: Ventz Petkov > Priority: Low > Attachments: CMakeOutput.log > > > When checking out latest master branch (https://github.com/bro/bro/commit/1e66c6718a98675fb838205a5e55220e9794eeb7), and given the above environment, error at configure: > ######################################################## > bro# ./configure --with-pcap=/opt/pfring > Build Directory : build > Source Directory: /root/install/bro > CMake Error at CMakeLists.txt:7 (include): > include could not find load file: > cmake/CommonCMakeConfig.cmake > CMake Error at CMakeLists.txt:52 (include): > include could not find load file: > FindRequiredPackage > -- Found sed: /bin/sed > CMake Error at CMakeLists.txt:64 (FindRequiredPackage): > Unknown CMake command "FindRequiredPackage". > -- Configuring incomplete, errors occurred! > See also "/root/install/bro/build/CMakeFiles/CMakeOutput.log". > ######################################################## > Attaching log file "CMakeOutput.log" -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From lauri.vosandi at gmail.com Thu May 7 11:24:30 2015 From: lauri.vosandi at gmail.com (lauri) Date: Thu, 7 May 2015 21:24:30 +0300 Subject: [Bro-Dev] libcurl and libev integration Message-ID: Hi, This is sort of a long story so grab a cup of coffee :) I've been using ActiveHTTP::Request to issue HTTP requests from Bro scripts so far, but as it's using shell invocation of curl it is a) slow due to execution of external process b) unable to make use of HTTP keep-alive and c) hence unable to multiplex requests to same server. This causes whole lot of overhead with current ActiveHTTP::Request making it unusable for various interesting usecases, eg. querying data from ElasticSearch ;) I eventually started tweaking the code to incorporate libcurl functionality into Bro runtime and the blocking version came along just fine. Now as I am trying to make the libcurl function calls asynchronous, it's becoming really complex. There is curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd); [1] which can be used to fetch socket descriptors associated with libcurl's connections. It is designed to populate existing fd_set struct with proper file handles to be spoon-fed to select(). Note that fd_set struct internal structure is not well defined (different on Windows and POSIX); macros FD_SET, FD_CLR, FD_ISSET and FD_ZERO are designed for fd_set manipulation and there is no macro to extract socket descriptors associated with particular fd_set. Trying to work with and around Bro's FD_Set and IOSource classes makes it nearly impossible to integrate libcurl as it is without a lot of dirty code. I also noticed that you've implemented asynchronous DNS client on top of your event loop handler mechanism. I assume this was built much earlier than any event handling library was conceived so it's sort of a legacy. Now looking how stuff is done nowadays you see there are a lot of event loop libraries out there which make more efficient use of kernel such as epoll() and kqueue in contrast to select() and already have built-in methods for asynchronous file input/output. Note that libev [3] also has built-in async DNS client. My conclusion at this very early stage is that it would make sense to substitute Bro's event loop and DNS client with libev. This should make it significantly easier to integrate with other libraries such as libcurl, take a look at libuv example of libcurl [4]. I am not sure how this would affect Bro runtime logic. Comments, questions and feedback on the ideas presented above are very much welcome :) 1. http://curl.haxx.se/libcurl/c/curl_multi_fdset.html 2. http://www.mkssoftware.com/docs/man3/select.3.asp 3. https://github.com/libuv/libuv 4. http://curl.haxx.se/libcurl/c/multi-uv.html -- Lauri V?sandi tel: +372 53329412 e-mail: lauri.vosandi at gmail.com blog: http://lauri.vosandi.com/ From edthoma at sandia.gov Thu May 7 14:17:52 2015 From: edthoma at sandia.gov (Thomas, Eric D) Date: Thu, 7 May 2015 21:17:52 +0000 Subject: [Bro-Dev] [EXTERNAL] Re: [Bro] Logging VLAN IDs In-Reply-To: <20150429235907.GH10338@icir.org> References: <20150417155524.GO55440@icir.org> <20150429235907.GH10338@icir.org> Message-ID: That sounds good! Both ideas seem to add an interesting level of additional flexibility and analytic potential. -- Eric Thomas edthoma at sandia.gov On 4/29/15, 4:59 PM, "Robin Sommer" wrote: >What if we did a combination of what I suggested and your thoughts >here? We carry link-level features through to script-land inside the >connection record, and in addition allowed to transfer a custom subset >over to the connection ID for hashing? The latter could be done later >as a second step. > >Robin > >On Tue, Apr 28, 2015 at 18:32 +0000, you wrote: > >> Hi Robin, >> >> I thought more about your generalized idea and would like to follow up. >>To >> start, adding link-level features to the connection ID hash, while >>perhaps >> useful in some contexts, does not provide us the functionality we >>desire. >> I have an incoming feed of VLAN-tagged traffic (both VLAN and 802.1ah) >> with perhaps dozens of different VLANs, and I would like to handle the >> connections differently in scripts but also mainly in offline log >>analysis >> depending upon which VLANs the traffic is associated with. >> >> Initially I had proposed simply adding the VLAN Ids to the conn.log >>file, >> but that is certainly too specific of a solution. What are your thoughts >> on exposing link-level features at the script layer for connections? For >> example, if all observed VLAN tags for a connection were in a set >>variable >> of the script-level Connection record, I could then label my data by >> matching VLAN Ids, then process them differently accordingly. Thoughts? >> > > >-- >Robin Sommer * Broala, LLC * robin at broala.com * www.broala.com From jira at bro-tracker.atlassian.net Thu May 7 14:54:00 2015 From: jira at bro-tracker.atlassian.net (Vlad Grigorescu (JIRA)) Date: Thu, 7 May 2015 16:54:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1394) Github commit seems to have possible configure issues? In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20602#comment-20602 ] Vlad Grigorescu commented on BIT-1394: -------------------------------------- When working with Bro behind an HTTP proxy, I use a config based on this: https://wiki.yoctoproject.org/wiki/Working_Behind_a_Network_Proxy > vladg at dev-01 ~ % git config -l | grep proxy > http.proxy=http://proxy.example.net:3128 > core.gitproxy=/home/vladg/bin/gitproxy > vladg at dev-01 ~ % cat bin/gitproxy > #!/bin/sh > > _proxy=proxy.example.net > _proxyport=3128 > > exec socat STDIO PROXY:$_proxy:$1:$2,proxyport=$_proxyport > Github commit seems to have possible configure issues? > ------------------------------------------------------ > > Key: BIT-1394 > URL: https://bro-tracker.atlassian.net/browse/BIT-1394 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: * Dell C6220 (PER blade: 128GB of ram | 2 socket | 16 cores per socket | 2-10G cards) > * Ubuntu 14.04.2 LTS system > * Feeding in 20Gb/s links > * PF_RING-6.0.3 compiled into /opt/pfring > Packages installed from base (other than SSH during select-install): > build-essential libnuma-dev pkg-config cmake make gcc g++ swig flex bison libpcap-dev libgeoip-dev libssl-dev python-dev zlib1g-dev libmagic-dev swig2.0 libgoogle-perftools-dev google-perftools libxml2-dev libcurl4-gnutls-dev mailutils > Have also added GeoIP databases manually. > pfring loaded with modprobe: > modprobe pf_ring enable_tx_capture=0 min_num_slots=32768 > Reporter: Ventz Petkov > Priority: Low > Attachments: CMakeOutput.log > > > When checking out latest master branch (https://github.com/bro/bro/commit/1e66c6718a98675fb838205a5e55220e9794eeb7), and given the above environment, error at configure: > ######################################################## > bro# ./configure --with-pcap=/opt/pfring > Build Directory : build > Source Directory: /root/install/bro > CMake Error at CMakeLists.txt:7 (include): > include could not find load file: > cmake/CommonCMakeConfig.cmake > CMake Error at CMakeLists.txt:52 (include): > include could not find load file: > FindRequiredPackage > -- Found sed: /bin/sed > CMake Error at CMakeLists.txt:64 (FindRequiredPackage): > Unknown CMake command "FindRequiredPackage". > -- Configuring incomplete, errors occurred! > See also "/root/install/bro/build/CMakeFiles/CMakeOutput.log". > ######################################################## > Attaching log file "CMakeOutput.log" -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 7 16:30:00 2015 From: jira at bro-tracker.atlassian.net (Ventz Petkov (JIRA)) Date: Thu, 7 May 2015 18:30:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1394) Github commit seems to have possible configure issues? In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20603#comment-20603 ] Ventz Petkov commented on BIT-1394: ----------------------------------- Vlad, Thanks. The proxy is definitely an issue (but I think there is something else going on -- see bellow) -- I had first exported the http_proxy and https_proxy env variables, and then had the git global config ones set. Both still fail, but I think that makes sense since it's looking for git:// links. ############################################# With debug on: (see *bold*) git clone --recursive https://github.com/bro/bro.git Cloning into 'bro'... remote: Counting objects: 70598, done. remote: Compressing objects: 100% (24/24), done. remote: Total 70598 (delta 7), reused 0 (delta 0), pack-reused 70571 Receiving objects: 100% (70598/70598), 50.23 MiB | 12.93 MiB/s, done. Resolving deltas: 100% (46910/46910), done. Checking connectivity... done. Submodule 'aux/binpac' (git://git.bro.org/binpac) registered for path 'aux/binpac' Submodule 'aux/bro-aux' (git://git.bro.org/bro-aux) registered for path 'aux/bro-aux' Submodule 'aux/broccoli' (git://git.bro.org/broccoli) registered for path 'aux/broccoli' Submodule 'aux/broctl' (git://git.bro.org/broctl) registered for path 'aux/broctl' Submodule 'aux/broker' (git://git.bro.org/broker) registered for path 'aux/broker' Submodule 'aux/btest' (git://git.bro.org/btest) registered for path 'aux/btest' Submodule 'aux/plugins' (git://git.bro.org/bro-plugins) registered for path 'aux/plugins' Submodule 'cmake' (git://git.bro.org/cmake) registered for path 'cmake' Submodule 'src/3rdparty' (git://git.bro.org/bro-3rdparty) registered for path 'src/3rdparty' Cloning into 'aux/binpac'... *fatal: unable to connect to git.bro.org: git.bro.org[0: 192.150.187.43]: errno=No route to host* *Clone of 'git://git.bro.org/binpac' into submodule path 'aux/binpac' failed* ############################################# I was going to say I am assuming this is the problem, however, when I download the "zip" from github (vs checking anything out) and try that, same errors. wget 'https://github.com/bro/bro/archive/master.zip' and then unzip that and try to do the same: # ./configure --with-pcap=/opt/pfring Build Directory : build Source Directory: /root/ventz-install-stuff/bro-master CMake Error at CMakeLists.txt:7 (include): include could not find load file: cmake/CommonCMakeConfig.cmake CMake Error at CMakeLists.txt:52 (include): include could not find load file: FindRequiredPackage -- Found sed: /bin/sed CMake Error at CMakeLists.txt:64 (FindRequiredPackage): Unknown CMake command "FindRequiredPackage". -- Configuring incomplete, errors occurred! > Github commit seems to have possible configure issues? > ------------------------------------------------------ > > Key: BIT-1394 > URL: https://bro-tracker.atlassian.net/browse/BIT-1394 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: * Dell C6220 (PER blade: 128GB of ram | 2 socket | 16 cores per socket | 2-10G cards) > * Ubuntu 14.04.2 LTS system > * Feeding in 20Gb/s links > * PF_RING-6.0.3 compiled into /opt/pfring > Packages installed from base (other than SSH during select-install): > build-essential libnuma-dev pkg-config cmake make gcc g++ swig flex bison libpcap-dev libgeoip-dev libssl-dev python-dev zlib1g-dev libmagic-dev swig2.0 libgoogle-perftools-dev google-perftools libxml2-dev libcurl4-gnutls-dev mailutils > Have also added GeoIP databases manually. > pfring loaded with modprobe: > modprobe pf_ring enable_tx_capture=0 min_num_slots=32768 > Reporter: Ventz Petkov > Priority: Low > Attachments: CMakeOutput.log > > > When checking out latest master branch (https://github.com/bro/bro/commit/1e66c6718a98675fb838205a5e55220e9794eeb7), and given the above environment, error at configure: > ######################################################## > bro# ./configure --with-pcap=/opt/pfring > Build Directory : build > Source Directory: /root/install/bro > CMake Error at CMakeLists.txt:7 (include): > include could not find load file: > cmake/CommonCMakeConfig.cmake > CMake Error at CMakeLists.txt:52 (include): > include could not find load file: > FindRequiredPackage > -- Found sed: /bin/sed > CMake Error at CMakeLists.txt:64 (FindRequiredPackage): > Unknown CMake command "FindRequiredPackage". > -- Configuring incomplete, errors occurred! > See also "/root/install/bro/build/CMakeFiles/CMakeOutput.log". > ######################################################## > Attaching log file "CMakeOutput.log" -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 7 16:31:00 2015 From: jira at bro-tracker.atlassian.net (Ventz Petkov (JIRA)) Date: Thu, 7 May 2015 18:31:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1394) Github commit seems to have possible configure issues? In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20603#comment-20603 ] Ventz Petkov edited comment on BIT-1394 at 5/7/15 6:30 PM: ----------------------------------------------------------- Vlad, Thanks. The proxy is definitely an issue (but I think there is something else going on -- see bellow) -- I had first exported the http_proxy and https_proxy env variables, and then had the git global config ones set. Both still fail, but I think that makes sense since it's looking for git:// links. ############################################# With debug on: (see *bold*) git clone --recursive https://github.com/bro/bro.git Cloning into 'bro'... remote: Counting objects: 70598, done. remote: Compressing objects: 100% (24/24), done. remote: Total 70598 (delta 7), reused 0 (delta 0), pack-reused 70571 Receiving objects: 100% (70598/70598), 50.23 MiB | 12.93 MiB/s, done. Resolving deltas: 100% (46910/46910), done. Checking connectivity... done. Submodule 'aux/binpac' (git://git.bro.org/binpac) registered for path 'aux/binpac' Submodule 'aux/bro-aux' (git://git.bro.org/bro-aux) registered for path 'aux/bro-aux' Submodule 'aux/broccoli' (git://git.bro.org/broccoli) registered for path 'aux/broccoli' Submodule 'aux/broctl' (git://git.bro.org/broctl) registered for path 'aux/broctl' Submodule 'aux/broker' (git://git.bro.org/broker) registered for path 'aux/broker' Submodule 'aux/btest' (git://git.bro.org/btest) registered for path 'aux/btest' Submodule 'aux/plugins' (git://git.bro.org/bro-plugins) registered for path 'aux/plugins' Submodule 'cmake' (git://git.bro.org/cmake) registered for path 'cmake' Submodule 'src/3rdparty' (git://git.bro.org/bro-3rdparty) registered for path 'src/3rdparty' Cloning into 'aux/binpac'... *fatal: unable to connect to git.bro.org: git.bro.org[0: 192.150.187.43]: errno=No route to host* *Clone of 'git://git.bro.org/binpac' into submodule path 'aux/binpac' failed* ############################################# I was going to say I am assuming this is the problem, however, when I download the "zip" from github (vs checking anything out) and try that, same errors. wget 'https://github.com/bro/bro/archive/master.zip' and then unzip that and try to do the same: # ./configure --with-pcap=/opt/pfring Build Directory : build Source Directory: /root/install/bro-master CMake Error at CMakeLists.txt:7 (include): include could not find load file: cmake/CommonCMakeConfig.cmake CMake Error at CMakeLists.txt:52 (include): include could not find load file: FindRequiredPackage -- Found sed: /bin/sed CMake Error at CMakeLists.txt:64 (FindRequiredPackage): Unknown CMake command "FindRequiredPackage". -- Configuring incomplete, errors occurred! was (Author: ventz): Vlad, Thanks. The proxy is definitely an issue (but I think there is something else going on -- see bellow) -- I had first exported the http_proxy and https_proxy env variables, and then had the git global config ones set. Both still fail, but I think that makes sense since it's looking for git:// links. ############################################# With debug on: (see *bold*) git clone --recursive https://github.com/bro/bro.git Cloning into 'bro'... remote: Counting objects: 70598, done. remote: Compressing objects: 100% (24/24), done. remote: Total 70598 (delta 7), reused 0 (delta 0), pack-reused 70571 Receiving objects: 100% (70598/70598), 50.23 MiB | 12.93 MiB/s, done. Resolving deltas: 100% (46910/46910), done. Checking connectivity... done. Submodule 'aux/binpac' (git://git.bro.org/binpac) registered for path 'aux/binpac' Submodule 'aux/bro-aux' (git://git.bro.org/bro-aux) registered for path 'aux/bro-aux' Submodule 'aux/broccoli' (git://git.bro.org/broccoli) registered for path 'aux/broccoli' Submodule 'aux/broctl' (git://git.bro.org/broctl) registered for path 'aux/broctl' Submodule 'aux/broker' (git://git.bro.org/broker) registered for path 'aux/broker' Submodule 'aux/btest' (git://git.bro.org/btest) registered for path 'aux/btest' Submodule 'aux/plugins' (git://git.bro.org/bro-plugins) registered for path 'aux/plugins' Submodule 'cmake' (git://git.bro.org/cmake) registered for path 'cmake' Submodule 'src/3rdparty' (git://git.bro.org/bro-3rdparty) registered for path 'src/3rdparty' Cloning into 'aux/binpac'... *fatal: unable to connect to git.bro.org: git.bro.org[0: 192.150.187.43]: errno=No route to host* *Clone of 'git://git.bro.org/binpac' into submodule path 'aux/binpac' failed* ############################################# I was going to say I am assuming this is the problem, however, when I download the "zip" from github (vs checking anything out) and try that, same errors. wget 'https://github.com/bro/bro/archive/master.zip' and then unzip that and try to do the same: # ./configure --with-pcap=/opt/pfring Build Directory : build Source Directory: /root/ventz-install-stuff/bro-master CMake Error at CMakeLists.txt:7 (include): include could not find load file: cmake/CommonCMakeConfig.cmake CMake Error at CMakeLists.txt:52 (include): include could not find load file: FindRequiredPackage -- Found sed: /bin/sed CMake Error at CMakeLists.txt:64 (FindRequiredPackage): Unknown CMake command "FindRequiredPackage". -- Configuring incomplete, errors occurred! > Github commit seems to have possible configure issues? > ------------------------------------------------------ > > Key: BIT-1394 > URL: https://bro-tracker.atlassian.net/browse/BIT-1394 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: * Dell C6220 (PER blade: 128GB of ram | 2 socket | 16 cores per socket | 2-10G cards) > * Ubuntu 14.04.2 LTS system > * Feeding in 20Gb/s links > * PF_RING-6.0.3 compiled into /opt/pfring > Packages installed from base (other than SSH during select-install): > build-essential libnuma-dev pkg-config cmake make gcc g++ swig flex bison libpcap-dev libgeoip-dev libssl-dev python-dev zlib1g-dev libmagic-dev swig2.0 libgoogle-perftools-dev google-perftools libxml2-dev libcurl4-gnutls-dev mailutils > Have also added GeoIP databases manually. > pfring loaded with modprobe: > modprobe pf_ring enable_tx_capture=0 min_num_slots=32768 > Reporter: Ventz Petkov > Priority: Low > Attachments: CMakeOutput.log > > > When checking out latest master branch (https://github.com/bro/bro/commit/1e66c6718a98675fb838205a5e55220e9794eeb7), and given the above environment, error at configure: > ######################################################## > bro# ./configure --with-pcap=/opt/pfring > Build Directory : build > Source Directory: /root/install/bro > CMake Error at CMakeLists.txt:7 (include): > include could not find load file: > cmake/CommonCMakeConfig.cmake > CMake Error at CMakeLists.txt:52 (include): > include could not find load file: > FindRequiredPackage > -- Found sed: /bin/sed > CMake Error at CMakeLists.txt:64 (FindRequiredPackage): > Unknown CMake command "FindRequiredPackage". > -- Configuring incomplete, errors occurred! > See also "/root/install/bro/build/CMakeFiles/CMakeOutput.log". > ######################################################## > Attaching log file "CMakeOutput.log" -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 7 16:36:00 2015 From: jira at bro-tracker.atlassian.net (Ventz Petkov (JIRA)) Date: Thu, 7 May 2015 18:36:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1394) Github commit seems to have possible configure issues? In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20604#comment-20604 ] Ventz Petkov commented on BIT-1394: ----------------------------------- Just did another test -- did the recursive clone on a different system, not behind the proxy, and then moved the entire repo. That configs and compiles. The only weird thing here is that the full .zip from github didn't work, but by the name (master), I am assuming it does NOT include sub-modules? > Github commit seems to have possible configure issues? > ------------------------------------------------------ > > Key: BIT-1394 > URL: https://bro-tracker.atlassian.net/browse/BIT-1394 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: * Dell C6220 (PER blade: 128GB of ram | 2 socket | 16 cores per socket | 2-10G cards) > * Ubuntu 14.04.2 LTS system > * Feeding in 20Gb/s links > * PF_RING-6.0.3 compiled into /opt/pfring > Packages installed from base (other than SSH during select-install): > build-essential libnuma-dev pkg-config cmake make gcc g++ swig flex bison libpcap-dev libgeoip-dev libssl-dev python-dev zlib1g-dev libmagic-dev swig2.0 libgoogle-perftools-dev google-perftools libxml2-dev libcurl4-gnutls-dev mailutils > Have also added GeoIP databases manually. > pfring loaded with modprobe: > modprobe pf_ring enable_tx_capture=0 min_num_slots=32768 > Reporter: Ventz Petkov > Priority: Low > Attachments: CMakeOutput.log > > > When checking out latest master branch (https://github.com/bro/bro/commit/1e66c6718a98675fb838205a5e55220e9794eeb7), and given the above environment, error at configure: > ######################################################## > bro# ./configure --with-pcap=/opt/pfring > Build Directory : build > Source Directory: /root/install/bro > CMake Error at CMakeLists.txt:7 (include): > include could not find load file: > cmake/CommonCMakeConfig.cmake > CMake Error at CMakeLists.txt:52 (include): > include could not find load file: > FindRequiredPackage > -- Found sed: /bin/sed > CMake Error at CMakeLists.txt:64 (FindRequiredPackage): > Unknown CMake command "FindRequiredPackage". > -- Configuring incomplete, errors occurred! > See also "/root/install/bro/build/CMakeFiles/CMakeOutput.log". > ######################################################## > Attaching log file "CMakeOutput.log" -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jsiwek at illinois.edu Thu May 7 16:58:21 2015 From: jsiwek at illinois.edu (Siwek, Jon) Date: Thu, 7 May 2015 23:58:21 +0000 Subject: [Bro-Dev] libcurl and libev integration In-Reply-To: References: Message-ID: > On May 7, 2015, at 1:24 PM, lauri wrote: > > My conclusion at this very early stage is that it would make sense to > substitute Bro's event loop and DNS client with libev. I?d also vote to investigate changing over to libev (or libuv since you mention it) and I also recently suggested that as part of [1]. - Jon [1] https://bro-tracker.atlassian.net/browse/BIT-1388 From lauri.vosandi at gmail.com Thu May 7 22:16:39 2015 From: lauri.vosandi at gmail.com (lauri) Date: Fri, 8 May 2015 08:16:39 +0300 Subject: [Bro-Dev] libcurl and libev integration In-Reply-To: References: Message-ID: Hi, > I'd also vote to investigate changing over to libev (or libuv since you mention it) and I also recently suggested that as part of [1]. Thanks for the thumbs up. I think libuv makes better match due to built-in async DNS client and file input/output. Also it's used by Node.js which means that it's more tested than it's counterparts. Feel free to add reference to the initial post to the JIRA bug tracker comments section :) -- Lauri V?sandi tel: +372 53329412 e-mail: lauri.vosandi at gmail.com blog: http://lauri.vosandi.com/ From jira at bro-tracker.atlassian.net Thu May 7 23:01:00 2015 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Fri, 8 May 2015 01:01:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1392) CPack brocontrol package clashes with file paths from bro-minimal package In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20605#comment-20605 ] Johanna Amann commented on BIT-1392: ------------------------------------ We are currently moving away from the cmake generated packages to an approach of manually creating deb and rpm files (currently via the OpenSUSE Build Service). http://mailman.icsi.berkeley.edu/pipermail/bro-dev/2015-April/009837.html gives more details and download locations for nightly Bro packages. Because of that, the cpack bits in Bro are basically unmaintained. If you come up with a nice patch that makes this work again we will be happy consider it, otherwise this will probably not be fixed. > CPack brocontrol package clashes with file paths from bro-minimal package > ------------------------------------------------------------------------- > > Key: BIT-1392 > URL: https://bro-tracker.atlassian.net/browse/BIT-1392 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro, BroControl > Affects Versions: git/master > Reporter: Derek Ditch > > I've been building bro packages from git/master (as of today), and with EL7, yum enforces file path ownership to packages. Currently, all the packages checkout the same instance of cmake scripts, namely 'ConfigurePackaging.cmake'. > Currently, this script excludes '/opt /var /var/opt'. When building brocontrol (and possibly issues with broccoli), you have to exclude paths that the bro package already owns. Namely, I had to change the exclusion line to the following to make it work: > set(CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION /opt /var /var/opt /opt/bro/share/man/man8 /opt/bro/share/man /opt/bro/share /opt/bro ) > I don't know how you would like to handle this. Obviously, it makes sense to have a common cmake script repo... maybe move this definition to the configure scripts for each component so that it's picked up in the initial cmake run. > Also, it's worth noting that you have to use CMake >= 2.8.12 for this definition to actually work -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 7 23:02:00 2015 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Fri, 8 May 2015 01:02:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1392) CPack brocontrol package clashes with file paths from bro-minimal package In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20606#comment-20606 ] Johanna Amann commented on BIT-1392: ------------------------------------ Also - requiring cmake 2.8.12 as a minimum in Bro generally might be a problem since it is rather recent and probably not available in a lot of distributions that still see widespread use... > CPack brocontrol package clashes with file paths from bro-minimal package > ------------------------------------------------------------------------- > > Key: BIT-1392 > URL: https://bro-tracker.atlassian.net/browse/BIT-1392 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro, BroControl > Affects Versions: git/master > Reporter: Derek Ditch > > I've been building bro packages from git/master (as of today), and with EL7, yum enforces file path ownership to packages. Currently, all the packages checkout the same instance of cmake scripts, namely 'ConfigurePackaging.cmake'. > Currently, this script excludes '/opt /var /var/opt'. When building brocontrol (and possibly issues with broccoli), you have to exclude paths that the bro package already owns. Namely, I had to change the exclusion line to the following to make it work: > set(CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION /opt /var /var/opt /opt/bro/share/man/man8 /opt/bro/share/man /opt/bro/share /opt/bro ) > I don't know how you would like to handle this. Obviously, it makes sense to have a common cmake script repo... maybe move this definition to the configure scripts for each component so that it's picked up in the initial cmake run. > Also, it's worth noting that you have to use CMake >= 2.8.12 for this definition to actually work -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Fri May 8 05:30:01 2015 From: jira at bro-tracker.atlassian.net (Derek Ditch (JIRA)) Date: Fri, 8 May 2015 07:30:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1392) CPack brocontrol package clashes with file paths from bro-minimal package In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20607#comment-20607 ] Derek Ditch commented on BIT-1392: ---------------------------------- The OpenSuSE build service option sounds great. Any chance the respective spec files and debian control scripts can be added to the git repo so others can build it too? On Fri, May 8, 2015 at 2:02 AM, Johanna Amann (JIRA) < > CPack brocontrol package clashes with file paths from bro-minimal package > ------------------------------------------------------------------------- > > Key: BIT-1392 > URL: https://bro-tracker.atlassian.net/browse/BIT-1392 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro, BroControl > Affects Versions: git/master > Reporter: Derek Ditch > > I've been building bro packages from git/master (as of today), and with EL7, yum enforces file path ownership to packages. Currently, all the packages checkout the same instance of cmake scripts, namely 'ConfigurePackaging.cmake'. > Currently, this script excludes '/opt /var /var/opt'. When building brocontrol (and possibly issues with broccoli), you have to exclude paths that the bro package already owns. Namely, I had to change the exclusion line to the following to make it work: > set(CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION /opt /var /var/opt /opt/bro/share/man/man8 /opt/bro/share/man /opt/bro/share /opt/bro ) > I don't know how you would like to handle this. Obviously, it makes sense to have a common cmake script repo... maybe move this definition to the configure scripts for each component so that it's picked up in the initial cmake run. > Also, it's worth noting that you have to use CMake >= 2.8.12 for this definition to actually work -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Fri May 8 06:39:00 2015 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Fri, 8 May 2015 08:39:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1392) CPack brocontrol package clashes with file paths from bro-minimal package In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20608#comment-20608 ] Johanna Amann commented on BIT-1392: ------------------------------------ Currently the spec files / control scripts are available at https://build.opensuse.org/package/show/network:bro/bro-nightly - we might add them somewhere into the git-repo at the moment, but they would probably always need slight modifications to run on a specific distribution (at the moment they contain something similar to #ifdefs). > CPack brocontrol package clashes with file paths from bro-minimal package > ------------------------------------------------------------------------- > > Key: BIT-1392 > URL: https://bro-tracker.atlassian.net/browse/BIT-1392 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro, BroControl > Affects Versions: git/master > Reporter: Derek Ditch > > I've been building bro packages from git/master (as of today), and with EL7, yum enforces file path ownership to packages. Currently, all the packages checkout the same instance of cmake scripts, namely 'ConfigurePackaging.cmake'. > Currently, this script excludes '/opt /var /var/opt'. When building brocontrol (and possibly issues with broccoli), you have to exclude paths that the bro package already owns. Namely, I had to change the exclusion line to the following to make it work: > set(CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION /opt /var /var/opt /opt/bro/share/man/man8 /opt/bro/share/man /opt/bro/share /opt/bro ) > I don't know how you would like to handle this. Obviously, it makes sense to have a common cmake script repo... maybe move this definition to the configure scripts for each component so that it's picked up in the initial cmake run. > Also, it's worth noting that you have to use CMake >= 2.8.12 for this definition to actually work -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Fri May 8 10:08:01 2015 From: jira at bro-tracker.atlassian.net (Ventz Petkov (JIRA)) Date: Fri, 8 May 2015 12:08:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1394) Github commit seems to have possible configure issues? In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20609#comment-20609 ] Ventz Petkov commented on BIT-1394: ----------------------------------- Sorry for this ticket -- feel free to close it. > Github commit seems to have possible configure issues? > ------------------------------------------------------ > > Key: BIT-1394 > URL: https://bro-tracker.atlassian.net/browse/BIT-1394 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master > Environment: * Dell C6220 (PER blade: 128GB of ram | 2 socket | 16 cores per socket | 2-10G cards) > * Ubuntu 14.04.2 LTS system > * Feeding in 20Gb/s links > * PF_RING-6.0.3 compiled into /opt/pfring > Packages installed from base (other than SSH during select-install): > build-essential libnuma-dev pkg-config cmake make gcc g++ swig flex bison libpcap-dev libgeoip-dev libssl-dev python-dev zlib1g-dev libmagic-dev swig2.0 libgoogle-perftools-dev google-perftools libxml2-dev libcurl4-gnutls-dev mailutils > Have also added GeoIP databases manually. > pfring loaded with modprobe: > modprobe pf_ring enable_tx_capture=0 min_num_slots=32768 > Reporter: Ventz Petkov > Priority: Low > Attachments: CMakeOutput.log > > > When checking out latest master branch (https://github.com/bro/bro/commit/1e66c6718a98675fb838205a5e55220e9794eeb7), and given the above environment, error at configure: > ######################################################## > bro# ./configure --with-pcap=/opt/pfring > Build Directory : build > Source Directory: /root/install/bro > CMake Error at CMakeLists.txt:7 (include): > include could not find load file: > cmake/CommonCMakeConfig.cmake > CMake Error at CMakeLists.txt:52 (include): > include could not find load file: > FindRequiredPackage > -- Found sed: /bin/sed > CMake Error at CMakeLists.txt:64 (FindRequiredPackage): > Unknown CMake command "FindRequiredPackage". > -- Configuring incomplete, errors occurred! > See also "/root/install/bro/build/CMakeFiles/CMakeOutput.log". > ######################################################## > Attaching log file "CMakeOutput.log" -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From robin at icir.org Sun May 10 20:28:57 2015 From: robin at icir.org (Robin Sommer) Date: Sun, 10 May 2015 20:28:57 -0700 Subject: [Bro-Dev] libcurl and libev integration In-Reply-To: References: Message-ID: <20150511032857.GB90907@icir.org> On Thu, May 07, 2015 at 21:24 +0300, you wrote: > My conclusion at this very early stage is that it would make sense to > substitute Bro's event loop and DNS client with libev. Lots of good thoughts. That all makes sense and would be worth exploring I think. Much of the current code is indeed just legacy[1] and should really be completely redone. I wasn't aware of libuv, but that and libev both look like good candidates to get some abstraction in there. Redoing the I/O loop is a larger project though. The coding is one part but we'd also need to test it pretty thoroughly in a range of settings so that we don't break anything. If we had a volunteer to take the lead on this, that would probably help a lot. :-) Robin [1] Including such things as working around old OS versions not correctly handling select on fds coming out of pcap. -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jira at bro-tracker.atlassian.net Mon May 11 08:46:02 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Mon, 11 May 2015 10:46:02 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1388) Broker's integration in Bro's main/run loop In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20700#comment-20700 ] Jon Siwek commented on BIT-1388: -------------------------------- Just referencing a bro-dev mailing list thread w/ other discussion and ideas regarding libev or libuv integration: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/2015-May/010069.html > Broker's integration in Bro's main/run loop > ------------------------------------------- > > Key: BIT-1388 > URL: https://bro-tracker.atlassian.net/browse/BIT-1388 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro, Broker > Reporter: Jon Siwek > Fix For: 2.5 > > > * There's a cost to Broker queues being idle. Whenever Broker gets a chance to process messages, it looks for updates to all connections/message-queues/data-stores. That involves sending synchronous messages between actors, and for empty queues, it just gets back an empty deque object it needs to destroy. > * Broker queues integrate into Bro's run loop by exposing a file descriptor that's ready when the queue is non-empty. Users have the capability of adding arbitrary numbers of queues at run-time (e.g. they can freely add subscriptions to any amount of logs, events, etc.). Relying on select() may become a bottleneck if someone has hundreds of Broker queues, or could possibly break on some systems if FD_SETSIZE is limited to 1024. > Ideas on how to fix: > * Improve Bro's main run loop and dedicate an IOSource to each Broker queue (instead of sharing a single IOSource like they do now). There might be several things that could be tweaked in the main run loop, but at a minimum, epoll()/kqueue() could alternatively replace select(). Could also think about using something like libev (http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod) to abstract what particular polling backend is used. Might even be able to use libev's timers to fix how Bro's timers are currently coupled w/ there being an active IOSource consistently driving time forward. > * Move the draining of Broker queues completely off to their own threads. This maybe is adding a bit too much complexity (Broker/CAF uses threads for queues, then Bro would use more threads just to talk to those other threads...). Since CAF becomes a requirement, it may be simpler to start replacing/allowing some areas of Bro's threading to be done w/ CAF actors. And then if Broker exposed an optional API to talk directly w/ CAF actors, the integration w/ Bro may actually become more straightforward. > And those ideas don't have to be mutually exclusive. -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Mon May 11 13:58:00 2015 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Mon, 11 May 2015 15:58:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1392) CPack brocontrol package clashes with file paths from bro-minimal package In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johanna Amann updated BIT-1392: ------------------------------- Resolution: Won't Fix Status: Closed (was: Open) > CPack brocontrol package clashes with file paths from bro-minimal package > ------------------------------------------------------------------------- > > Key: BIT-1392 > URL: https://bro-tracker.atlassian.net/browse/BIT-1392 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro, BroControl > Affects Versions: git/master > Reporter: Derek Ditch > > I've been building bro packages from git/master (as of today), and with EL7, yum enforces file path ownership to packages. Currently, all the packages checkout the same instance of cmake scripts, namely 'ConfigurePackaging.cmake'. > Currently, this script excludes '/opt /var /var/opt'. When building brocontrol (and possibly issues with broccoli), you have to exclude paths that the bro package already owns. Namely, I had to change the exclusion line to the following to make it work: > set(CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION /opt /var /var/opt /opt/bro/share/man/man8 /opt/bro/share/man /opt/bro/share /opt/bro ) > I don't know how you would like to handle this. Obviously, it makes sense to have a common cmake script repo... maybe move this definition to the configure scripts for each component so that it's picked up in the initial cmake run. > Also, it's worth noting that you have to use CMake >= 2.8.12 for this definition to actually work -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Mon May 11 13:58:00 2015 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Mon, 11 May 2015 15:58:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1392) CPack brocontrol package clashes with file paths from bro-minimal package In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20701#comment-20701 ] Johanna Amann commented on BIT-1392: ------------------------------------ Closing for now since we probably will not do anything about this and it sounds like the spec/debian files also fulfill your needs. Feel free to re-open this if there is anything else... > CPack brocontrol package clashes with file paths from bro-minimal package > ------------------------------------------------------------------------- > > Key: BIT-1392 > URL: https://bro-tracker.atlassian.net/browse/BIT-1392 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro, BroControl > Affects Versions: git/master > Reporter: Derek Ditch > > I've been building bro packages from git/master (as of today), and with EL7, yum enforces file path ownership to packages. Currently, all the packages checkout the same instance of cmake scripts, namely 'ConfigurePackaging.cmake'. > Currently, this script excludes '/opt /var /var/opt'. When building brocontrol (and possibly issues with broccoli), you have to exclude paths that the bro package already owns. Namely, I had to change the exclusion line to the following to make it work: > set(CPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION /opt /var /var/opt /opt/bro/share/man/man8 /opt/bro/share/man /opt/bro/share /opt/bro ) > I don't know how you would like to handle this. Obviously, it makes sense to have a common cmake script repo... maybe move this definition to the configure scripts for each component so that it's picked up in the initial cmake run. > Also, it's worth noting that you have to use CMake >= 2.8.12 for this definition to actually work -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Tue May 12 14:35:00 2015 From: jira at bro-tracker.atlassian.net (Justin Azoff (JIRA)) Date: Tue, 12 May 2015 16:35:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1395) Elasticsearch plugin README outdated In-Reply-To: References: Message-ID: Justin Azoff created BIT-1395: --------------------------------- Summary: Elasticsearch plugin README outdated Key: BIT-1395 URL: https://bro-tracker.atlassian.net/browse/BIT-1395 Project: Bro Issue Tracker Issue Type: Problem Components: bro-aux Affects Versions: git/master Reporter: Justin Azoff Priority: Trivial The elasticsearch plugin readme still says to do: @load tuning/logs-to-elasticsearch but this file no longer exists. I'm not sure if it is loaded automatically after the plugin is installed or if a different load statement is neader. -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Tue May 12 15:17:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 12 May 2015 17:17:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1395) Elasticsearch plugin README outdated In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1395: ------------------------------ Fix Version/s: 2.4 > Elasticsearch plugin README outdated > ------------------------------------ > > Key: BIT-1395 > URL: https://bro-tracker.atlassian.net/browse/BIT-1395 > Project: Bro Issue Tracker > Issue Type: Problem > Components: bro-aux > Affects Versions: git/master > Reporter: Justin Azoff > Priority: Trivial > Labels: documentation > Fix For: 2.4 > > > The elasticsearch plugin readme still says to do: > @load tuning/logs-to-elasticsearch > but this file no longer exists. I'm not sure if it is loaded automatically after the plugin is installed or if a different load statement is neader. -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From albert.zaharovits at gmail.com Wed May 13 04:30:51 2015 From: albert.zaharovits at gmail.com (Albert Zaharovits) Date: Wed, 13 May 2015 14:30:51 +0300 Subject: [Bro-Dev] Extract complete files References: Message-ID: <664F53A1-6B17-461C-A16F-E1AEFA350932@gmail.com> Hello, I am experimenting with the Files framework in bro 2.4 beta. I would like to extract HTTP files, *without* missing_bytes. Can anyone please help me on this? Thanks, Albert From balint.martina at gmail.com Wed May 13 08:26:02 2015 From: balint.martina at gmail.com (Martina Balintova) Date: Wed, 13 May 2015 16:26:02 +0100 Subject: [Bro-Dev] dfa_state does not free its cache Message-ID: Hi, Dfa_state_cache does not follow its max size limit and it can run over this limit quite easily. I am not sure what kind of data are stored in the cache, so I am hesitant to fix the bug. Could you please take a look at it with relation to file analyzer? I use bro 2.3-397. In one of the protocol analyzers, I call file_mgr->DataIn() and later file_mgr->EndOfFile(). Valgrind shows that if you run it on traffic with a lot of files to analyze, memory builds (and is not freed) at this path: file_analysis::File::DetectMime() (File.cc:304) file_analysis::Manager::DetectMime RuleMatcher::Match RE_Match_State::Match DFa_State::Xtion DFA_State::ComputeXtion DFA_Machine::StateSetToDFA_State DFA_State::DFA_State Any hint how to fix DFA_State and RE.cc? Thank you, Martina -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20150513/81bb44dc/attachment.html From robin at icir.org Wed May 13 12:14:38 2015 From: robin at icir.org (Robin Sommer) Date: Wed, 13 May 2015 12:14:38 -0700 Subject: [Bro-Dev] dfa_state does not free its cache In-Reply-To: References: Message-ID: <20150513191438.GI96364@icir.org> On Wed, May 13, 2015 at 16:26 +0100, you wrote: > Dfa_state_cache does not follow its max size limit Yeah, that option has actually been dead for a while already, and we finally removed it recently just before the 2.4 beta (see https://github.com/bro/bro/commit/1132470b05751d1f7f25dd320758e58cdc4a6d10) We had removed the corresponding functionality with one of the earlier releases already because it was a separate code path that, due to its performance implications, had to be activated explicitly with a configure switch, meaning that probably nobody was using it anyways. It's not really a leak though. While the data structure can keep growing, the memory remains accessible and the states may be used with future traffic. That said, depending on the regexps in use, the data structure can get pretty big over time, and the memory indeed won't be reclaimed. Snother recent change in preparation for 2.4 was optimizing the file detetion regexps to cause less such memory usage. Robin -- Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin From jira at bro-tracker.atlassian.net Thu May 14 08:18:00 2015 From: jira at bro-tracker.atlassian.net (Aashish Sharma (JIRA)) Date: Thu, 14 May 2015 10:18:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: Aashish Sharma created BIT-1396: ----------------------------------- Summary: Logs disappearing on broctl restart Key: BIT-1396 URL: https://bro-tracker.atlassian.net/browse/BIT-1396 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.4 Environment: Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. Restarts happen as - broctl check; broctl restart - broctl check; broctl restart --clean - broctl restart or some variant - not precisely sure. But all log files for that duration of restarts are missing. Reporter: Aashish Sharma -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 14 08:30:02 2015 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Thu, 14 May 2015 10:30:02 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johanna Amann updated BIT-1396: ------------------------------- Environment: ? (was: Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. Restarts happen as - broctl check; broctl restart - broctl check; broctl restart --clean - broctl restart or some variant - not precisely sure. But all log files for that duration of restarts are missing. ) > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Environment: ? > Reporter: Aashish Sharma > Fix For: 2.4 > > -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 14 08:30:01 2015 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Thu, 14 May 2015 10:30:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johanna Amann updated BIT-1396: ------------------------------- Fix Version/s: 2.4 > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Environment: Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing. > Reporter: Aashish Sharma > Fix For: 2.4 > > -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 14 08:31:00 2015 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Thu, 14 May 2015 10:31:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johanna Amann updated BIT-1396: ------------------------------- Environment: (was: ?) > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 14 08:31:00 2015 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Thu, 14 May 2015 10:31:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johanna Amann updated BIT-1396: ------------------------------- Description: Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. Restarts happen as - broctl check; broctl restart - broctl check; broctl restart --clean - broctl restart or some variant - not precisely sure. But all log files for that duration of restarts are missing > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Environment: ? > Reporter: Aashish Sharma > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 14 09:28:00 2015 From: jira at bro-tracker.atlassian.net (Aashish Sharma (JIRA)) Date: Thu, 14 May 2015 11:28:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20702#comment-20702 ] Aashish Sharma commented on BIT-1396: ------------------------------------- Example: -rw-r--r-- 1 bro bro 81M May 13 13:33 conn.log.mgr.2015-05-13-13:20:11-13:33:12.gz -rw-r--r-- 1 bro bro 3.1G May 13 13:36 conn.log.mgr.2015-05-13-00:00:00-13:19:59.gz -rw-r--r-- 1 bro bro 420M May 13 14:31 conn.log.mgr.2015-05-13-13:33:24-14:29:41.gz ??? (bro was running) -rw-r--r-- 1 bro bro 1.4G May 14 00:06 conn.log.mgr.2015-05-13-16:48:02-00:00:00.gz Logs from 14:30 to 16:48 were not archived and disappeared. (Shows conn.log, but its everything) Likewise: > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 14 11:50:01 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Thu, 14 May 2015 13:50:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20703#comment-20703 ] Daniel Thayer commented on BIT-1396: ------------------------------------ Did you try looking in spool/tmp to see if there are any directories named "post-terminate-YYYY-MM-DD-HH-MM-SS-PID". Those directories would contain the missing logs if, for some reason, the log archival failed. > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 14 12:24:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 14 May 2015 14:24:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1396: ------------------------------ Priority: High (was: Normal) > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 14 12:32:00 2015 From: jira at bro-tracker.atlassian.net (Aashish Sharma (JIRA)) Date: Thu, 14 May 2015 14:32:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20704#comment-20704 ] Aashish Sharma commented on BIT-1396: ------------------------------------- Ah! Yes, I see logs in spool/tmp/post-terminate-YYYY-MM-DD-HH-MM-SS-PID. Is there a way to know if archival failed and what caused the failure. Unless one goes to explicitly account for entire days of logs, it seems like this is a silent failure. One might end up unknowingly missing chunks of logs. If I recall my workflow was, edit script, introduce a bug, broctl start fails, fix the bug, retry. Still not sure what action got archival failure. Aashish -- Aashish Sharma (asharma at lbl.gov) Cyber Security, Lawrence Berkeley National Laboratory http://go.lbl.gov/pgp-aashish Office: (510)-495-2680 Cell: (510)-612-7971 > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 14 12:51:00 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Thu, 14 May 2015 14:51:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20705#comment-20705 ] Daniel Thayer commented on BIT-1396: ------------------------------------ Did you look in stderr.log? (look in both the "spool/tmp/post-terminate-.." and "spool/manager" directories) I would expect to see some error messages there regarding log archival failure. > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 14 13:43:00 2015 From: jira at bro-tracker.atlassian.net (Aashish Sharma (JIRA)) Date: Thu, 14 May 2015 15:43:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20706#comment-20706 ] Aashish Sharma commented on BIT-1396: ------------------------------------- Yes, nothing in stderr.log - likely got over-written by one of the later bro restarts. -- Aashish Sharma (asharma at lbl.gov) Cyber Security, Lawrence Berkeley National Laboratory http://go.lbl.gov/pgp-aashish Office: (510)-495-2680 Cell: (510)-612-7971 > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 14 16:19:00 2015 From: jira at bro-tracker.atlassian.net (Aashish Sharma (JIRA)) Date: Thu, 14 May 2015 18:19:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20707#comment-20707 ] Aashish Sharma commented on BIT-1396: ------------------------------------- Um! Well the stderr.log in spool/tmp/port-terminate is there but nothing stands out in it. I can email it to you, if you'd like. Aashish > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 14 16:20:00 2015 From: jira at bro-tracker.atlassian.net (Aashish Sharma (JIRA)) Date: Thu, 14 May 2015 18:20:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20706#comment-20706 ] Aashish Sharma edited comment on BIT-1396 at 5/14/15 6:19 PM: -------------------------------------------------------------- Yes, nothing in stderr.log - likely got over-written by one of the later bro restarts. Aashish was (Author: asharma at lbl.gov): Yes, nothing in stderr.log - likely got over-written by one of the later bro restarts. -- Aashish Sharma (asharma at lbl.gov) Cyber Security, Lawrence Berkeley National Laboratory http://go.lbl.gov/pgp-aashish Office: (510)-495-2680 Cell: (510)-612-7971 > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Thu May 14 20:36:00 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Thu, 14 May 2015 22:36:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20708#comment-20708 ] Daniel Thayer commented on BIT-1396: ------------------------------------ OK, send it to me and I'll take a look. > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From asharma at lbl.gov Fri May 15 10:31:24 2015 From: asharma at lbl.gov (Aashish Sharma) Date: Fri, 15 May 2015 10:31:24 -0700 Subject: [Bro-Dev] broctl restart --clean Message-ID: <20150515173123.GF7995@yaksha.lbl.gov> Just a thought, broctl restart --clean at present does operations in the following sequence: 1) Stop running bro 2) clean up nodes 3) check configurations 4) install new config 5) start bro. If scripts are buggy, this would fail at step (3) and now I am debugging scripts while bro is not running. I think restart --clean should first check configurations (step 3) and then if success, move further or stop. buggy/typo scripts are preffered to be debugged while bro is running. Aashsih -- Aashish Sharma (asharma at lbl.gov) Cyber Security, Lawrence Berkeley National Laboratory http://go.lbl.gov/pgp-aashish Office: (510)-495-2680 Cell: (510)-612-7971 From dnthayer at illinois.edu Fri May 15 10:48:40 2015 From: dnthayer at illinois.edu (Daniel Thayer) Date: Fri, 15 May 2015 12:48:40 -0500 Subject: [Bro-Dev] broctl restart --clean In-Reply-To: <20150515173123.GF7995@yaksha.lbl.gov> References: <20150515173123.GF7995@yaksha.lbl.gov> Message-ID: <55563178.4010600@illinois.edu> To avoid user confusion, I think it would be good to simply remove the "--clean" option from the restart command. Then, one would just use the "deploy" command. It does a "check", "install", "stop", and "start" (in that order). If someone wants to do things differently, they can still use the individual ("check", "clean", "restart", etc.) commands. On 05/15/2015 12:31 PM, Aashish Sharma wrote: > Just a thought, > > broctl restart --clean at present does operations in the following sequence: > > 1) Stop running bro > 2) clean up nodes > 3) check configurations > 4) install new config > 5) start bro. > > If scripts are buggy, this would fail at step (3) and now I am debugging scripts while bro is not running. > > I think restart --clean should first check configurations (step 3) and then if success, move further or stop. > > buggy/typo scripts are preffered to be debugged while bro is running. > > Aashsih > > From jira at bro-tracker.atlassian.net Sun May 17 07:58:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Sun, 17 May 2015 09:58:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1397) broctl --help is mysterious In-Reply-To: References: Message-ID: Vern Paxson created BIT-1397: -------------------------------- Summary: broctl --help is mysterious Key: BIT-1397 URL: https://bro-tracker.atlassian.net/browse/BIT-1397 Project: Bro Issue Tracker Issue Type: Problem Components: BroControl Affects Versions: 2.4 Environment: MacOS Mavericks Reporter: Vern Paxson For a newly installed Bro 2.4 beta, issuing "{{broctl --help}}" yields the cryptic output: {{Error: unable to open database file: /usr/local/bro/spool/state.db}} -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Sun May 17 11:35:03 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Sun, 17 May 2015 13:35:03 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1397) broctl --help is mysterious In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20709#comment-20709 ] Daniel Thayer commented on BIT-1397: ------------------------------------ Do you have write access to /usr/local/bro/spool ? How did you install Bro (build from source, or install a package)? > broctl --help is mysterious > --------------------------- > > Key: BIT-1397 > URL: https://bro-tracker.atlassian.net/browse/BIT-1397 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.4 > Environment: MacOS Mavericks > Reporter: Vern Paxson > > For a newly installed Bro 2.4 beta, issuing "{{broctl --help}}" yields the cryptic output: > {{Error: unable to open database file: /usr/local/bro/spool/state.db}} -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Sun May 17 12:19:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Sun, 17 May 2015 14:19:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1397) broctl --help is mysterious In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20710#comment-20710 ] Vern Paxson commented on BIT-1397: ---------------------------------- No, I don't have write access. I had expected that ordinary users can run Bro after it's installed - is that wrong? (In any case, the error message sure is cryptic!) I installed from source. > broctl --help is mysterious > --------------------------- > > Key: BIT-1397 > URL: https://bro-tracker.atlassian.net/browse/BIT-1397 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.4 > Environment: MacOS Mavericks > Reporter: Vern Paxson > > For a newly installed Bro 2.4 beta, issuing "{{broctl --help}}" yields the cryptic output: > {{Error: unable to open database file: /usr/local/bro/spool/state.db}} -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Mon May 18 00:40:02 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Mon, 18 May 2015 02:40:02 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1397) broctl --help is mysterious In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20711#comment-20711 ] Daniel Thayer commented on BIT-1397: ------------------------------------ I think the problem is that you need to be superuser to install in /usr/local, but when you do that then all of the installed files/directories are owned by root. The user who runs broctl needs write access to the /logs and /spool directories. I always run as an ordinary user and I just install to that user's home directory. > broctl --help is mysterious > --------------------------- > > Key: BIT-1397 > URL: https://bro-tracker.atlassian.net/browse/BIT-1397 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.4 > Environment: MacOS Mavericks > Reporter: Vern Paxson > > For a newly installed Bro 2.4 beta, issuing "{{broctl --help}}" yields the cryptic output: > {{Error: unable to open database file: /usr/local/bro/spool/state.db}} -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Tue May 19 02:50:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 19 May 2015 04:50:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1397) broctl --help is mysterious In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20712#comment-20712 ] Robin Sommer commented on BIT-1397: ----------------------------------- @vern: an ordinary user can use "bro", but not necessarily broctl, as that keeps state information. That's generally ok, I think. @daniel: would be good if "broctl --help" worked for any user, independent of being root and who installed it. That shouldn't be difficult, no? Also, for other commands, could you add a check that makes sure the user running broctl has the right permissions, and give an corresponding error message otherwise? > broctl --help is mysterious > --------------------------- > > Key: BIT-1397 > URL: https://bro-tracker.atlassian.net/browse/BIT-1397 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.4 > Environment: MacOS Mavericks > Reporter: Vern Paxson > > For a newly installed Bro 2.4 beta, issuing "{{broctl --help}}" yields the cryptic output: > {{Error: unable to open database file: /usr/local/bro/spool/state.db}} -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Tue May 19 03:32:02 2015 From: jira at bro-tracker.atlassian.net (Jason (JIRA)) Date: Tue, 19 May 2015 05:32:02 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1398) PPPoE PCAP stripping laters In-Reply-To: References: Message-ID: Jason created BIT-1398: -------------------------- Summary: PPPoE PCAP stripping laters Key: BIT-1398 URL: https://bro-tracker.atlassian.net/browse/BIT-1398 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.3 Environment: Ubuntu 12.04.5 , pf_ring Reporter: Jason Priority: High Recently I discovered what I believe to be a problem with Bro's packet collection of PPPoE traffic. This occurs both on the wire and when reading in PCAP. Here is a sample SSL session over PPPoE as captured by tcpdump: 12:58:27.914864568 PPPoE [ses 0x279a] IP 192.168.110.235.25095 > 192.168.162.218.443: Flags [S], seq 2317077818, win 65535, options [mss 1380,nop,wscale 9,sackOK,TS val 139402792 ecr 0], length 0 12:58:28.091544568 PPPoE [ses 0x279a] IP 192.168.162.218.443 > 192.168.110.235.25095: Flags [S.], seq 2303200074, ack 2317077819, win 5792, options [mss 1460,sackOK,TS val 1200789536 ecr 139402792,nop,wscale 7], length 0 12:58:28.092020568 PPPoE [ses 0x279a] IP 192.168.110.235.25095 > 192.168.162.218.443: Flags [.], ack 1, win 513, options [nop,nop,TS val 139402972 ecr 1200789536], length 0 12:58:28.092579568 PPPoE [ses 0x279a] IP 192.168.110.235.25095 > 192.168.162.218.443: Flags [P.], seq 1:257, ack 1, win 513, options [nop,nop,TS val 139402972 ecr 1200789536], length 256 12:58:28.268976568 PPPoE [ses 0x279a] IP 192.168.162.218.443 > 192.168.110.235.25095: Flags [.], ack 257, win 54, options [nop,nop,TS val 1200789713 ecr 139402972], length 0 Running this capture through Bro results in a valid ssl.log: 1431435508.092579 C2fjf233dO59LO7sj9 192.168.110.235 25095 192.168.162.218 443 TLSv10 TLS_DHE_RSA_WITH_AES_256_CBC_SHA - some_website.com 7e710c9504f77e9fc8d18121ed965a25119c673b6b4e0a07b5bfcd5baadae534 - T - - - - -- But the resulting PCAP coming out of Bro for the same packets looks like this: 12:58:27.914864256 40:00:3f:06:da:8a > 45:00:00:3c:aa:49, ethertype Unknown (0x6e36), length 82: 12:58:28.091544552 40:00:30:06:93:d4 > 45:00:00:3c:00:00, ethertype Unknown (0x36ec), length 82: 12:58:28.092020256 40:00:3f:06:da:84 > 45:00:00:34:aa:57, ethertype Unknown (0x6e36), length 74: 12:58:28.092579152 40:00:3f:06:d9:82 > 45:00:01:34:aa:59, ethertype Unknown (0x6e36), length 330: 12:58:28.268976656 40:00:30:06:00:42 > 45:00:00:34:93:9a, ethertype Unknown (0x36ec), length 74: Please let me know if you need any additional information. Jason -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Tue May 19 04:56:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 19 May 2015 06:56:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20713#comment-20713 ] Robin Sommer commented on BIT-1396: ----------------------------------- Any further updates here? Missing logs (or logs that don't make it all the way) always worry me. > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Tue May 19 09:20:01 2015 From: jira at bro-tracker.atlassian.net (Doris Schioberg (JIRA)) Date: Tue, 19 May 2015 11:20:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20714#comment-20714 ] Doris Schioberg commented on BIT-1396: -------------------------------------- Ich hab das mit dem letzten Master ausprobiert und konnte den Fehler nicht reproduzieren, falls das hilft. -- Doris Schioberg Bro Outreach, Training, and Education Coordinator International Computer Science Institute (ICSI Berkeley) Phone: +1 (510) 289-8406 * doris at bro.org > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Tue May 19 10:22:01 2015 From: jira at bro-tracker.atlassian.net (Aashish Sharma (JIRA)) Date: Tue, 19 May 2015 12:22:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20715#comment-20715 ] Aashish Sharma commented on BIT-1396: ------------------------------------- I found the 'missing' logs in spool/tmp/ dir. However, as I mentioned before, this crash_dump wasn't something outstanding (likely a script bug) so, I never noticed it happen when doing broctl restart etc. So logs didn't quite disappear atleast but weren't moved to archive folder too. I think I am suppose to send Daniel stderr.out. Doing that now. Aashish -- Aashish Sharma (asharma at lbl.gov) Cyber Security, Lawrence Berkeley National Laboratory http://go.lbl.gov/pgp-aashish Office: (510)-495-2680 Cell: (510)-612-7971 > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Tue May 19 18:55:00 2015 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Tue, 19 May 2015 20:55:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1399) topic/seth/deflate-missing-headers-fix In-Reply-To: References: Message-ID: Seth Hall created BIT-1399: ------------------------------ Summary: topic/seth/deflate-missing-headers-fix Key: BIT-1399 URL: https://bro-tracker.atlassian.net/browse/BIT-1399 Project: Bro Issue Tracker Issue Type: Patch Components: Bro Affects Versions: 2.4 Reporter: Seth Hall Merge request for the topic/seth/deflate-missing-headers-fix branch. It fixes an issue that is occasionally but regularly seen where servers don't include the headers correctly for deflated content. Browsers cope with this situation just fine and this change makes Bro also deal with the situation. -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Tue May 19 18:56:00 2015 From: jira at bro-tracker.atlassian.net (Seth Hall (JIRA)) Date: Tue, 19 May 2015 20:56:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1399) topic/seth/deflate-missing-headers-fix In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seth Hall updated BIT-1399: --------------------------- Status: Merge Request (was: Open) > topic/seth/deflate-missing-headers-fix > -------------------------------------- > > Key: BIT-1399 > URL: https://bro-tracker.atlassian.net/browse/BIT-1399 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > > Merge request for the topic/seth/deflate-missing-headers-fix branch. > It fixes an issue that is occasionally but regularly seen where servers don't include the headers correctly for deflated content. Browsers cope with this situation just fine and this change makes Bro also deal with the situation. -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Tue May 19 19:29:01 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Tue, 19 May 2015 21:29:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20716#comment-20716 ] Daniel Thayer commented on BIT-1396: ------------------------------------ Did you also look in the log archive directories (i.e., directories with names like "logs/YYYY-MM-DD") for stderr.log? (the actual filename will be longer and end in ".gz") When Bro crashes or is killed by broctl, then stdout.log and stderr.log are supposed to be archived. > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From noreply at bro.org Wed May 20 00:00:23 2015 From: noreply at bro.org (Merge Tracker) Date: Wed, 20 May 2015 00:00:23 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201505200700.t4K70NWW013907@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall - 2015-05-19 - Normal topic/seth/deflate-missing-headers-fix [2] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix From noreply at bro.org Thu May 21 00:00:28 2015 From: noreply at bro.org (Merge Tracker) Date: Thu, 21 May 2015 00:00:28 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201505210700.t4L70SWh025162@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall - 2015-05-19 - Normal topic/seth/deflate-missing-headers-fix [2] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix From noreply at bro.org Fri May 22 00:00:27 2015 From: noreply at bro.org (Merge Tracker) Date: Fri, 22 May 2015 00:00:27 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201505220700.t4M70RVC024996@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall - 2015-05-19 - Normal topic/seth/deflate-missing-headers-fix [2] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix From balint.martina at gmail.com Fri May 22 04:47:11 2015 From: balint.martina at gmail.com (Martina Balintova) Date: Fri, 22 May 2015 12:47:11 +0100 Subject: [Bro-Dev] Port flipping Message-ID: Hi Robin, I have a problem where a connection missing its SYN is not flipped correctly, because the client happened to choose a port that Bro thinks is a server port (IRC, 6666). What is confusing me is the special case in NetSessions::WantConnection() that prevents the flip. Your comments are about avoiding being confused by stealth scans, but I think that the flip will happen in every case except when the client is unlucky enough to use a ?server? port number. Given that is most of the time, why have the special case at all? Martina -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.icsi.berkeley.edu/pipermail/bro-dev/attachments/20150522/4c187c96/attachment.html From jira at bro-tracker.atlassian.net Fri May 22 10:08:01 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 22 May 2015 12:08:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1400) topic/jsiwek/mime-multipart-boundary-leniency In-Reply-To: References: Message-ID: Jon Siwek created BIT-1400: ------------------------------ Summary: topic/jsiwek/mime-multipart-boundary-leniency Key: BIT-1400 URL: https://bro-tracker.atlassian.net/browse/BIT-1400 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Reporter: Jon Siwek Assignee: Seth Hall Fix For: 2.4 Seth had a private pcap showing HTTP multipart content using boundary strings containing the '<' and '>' characters which causes HTTP/MIME content parsing to fail. This branch changes it so those characters are allowed (even though not explicitly permitted by the RFC). It feels a bit hacky to me (but so do most changes I've done to HTTP/MIME analyzers), so please review and check if the analysis looks "more correct" now. I scheduled this for 2.4 because I think Seth mentioned it might be something to try to get fixed in the final release, but it might be better to put it as part of 2.5 -- it's not really a severe bug but more of an oddity from a particular HTTP implementation and Bro's behavior with respect to it hasn't changed anytime recently (i.e. it's not a regression). -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From jira at bro-tracker.atlassian.net Fri May 22 10:13:00 2015 From: jira at bro-tracker.atlassian.net (Jon Siwek (JIRA)) Date: Fri, 22 May 2015 12:13:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1401) Broker fixes for 2.4 In-Reply-To: References: Message-ID: Jon Siwek created BIT-1401: ------------------------------ Summary: Broker fixes for 2.4 Key: BIT-1401 URL: https://bro-tracker.atlassian.net/browse/BIT-1401 Project: Bro Issue Tracker Issue Type: Problem Components: Bro, Broker Reporter: Jon Siwek Fix For: 2.4 Just making a reminder ticket of what Broker changes to include in a 0.3.1 Broker release and the final 2.4 Bro release. This commit should be considered: https://github.com/bro/broker/commit/8fc6938017dc15acfb26fa29e6ad0933019781c5 -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From noreply at bro.org Sat May 23 00:00:49 2015 From: noreply at bro.org (Merge Tracker) Date: Sat, 23 May 2015 00:00:49 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201505230700.t4N70n0C004497@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall - 2015-05-19 - Normal topic/seth/deflate-missing-headers-fix [2] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix From jira at bro-tracker.atlassian.net Sat May 23 19:18:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Sat, 23 May 2015 21:18:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1397) broctl --help is mysterious In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20720#comment-20720 ] Vern Paxson commented on BIT-1397: ---------------------------------- @robin: yeah, I think that's fine. I just want the error message to be clear! > broctl --help is mysterious > --------------------------- > > Key: BIT-1397 > URL: https://bro-tracker.atlassian.net/browse/BIT-1397 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.4 > Environment: MacOS Mavericks > Reporter: Vern Paxson > > For a newly installed Bro 2.4 beta, issuing "{{broctl --help}}" yields the cryptic output: > {{Error: unable to open database file: /usr/local/bro/spool/state.db}} -- This message was sent by Atlassian JIRA (v6.5-OD-03-002#65000) From noreply at bro.org Sun May 24 00:00:24 2015 From: noreply at bro.org (Merge Tracker) Date: Sun, 24 May 2015 00:00:24 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201505240700.t4O70OF2011036@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall - 2015-05-19 - Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------ ---------- --------------------------------------------------------------- #30 [3] bro jsbarber [4] 2015-05-23 Use a common Packet format and preserve layer 2 information [5] #1 [6] bro-plugins jsbarber [7] 2015-05-23 Use a common Packet format and preserve layer 2 information [8] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #30 https://github.com/bro/bro/pull/30 [4] jsbarber https://github.com/jsbarber [5] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [6] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From noreply at bro.org Mon May 25 00:00:26 2015 From: noreply at bro.org (Merge Tracker) Date: Mon, 25 May 2015 00:00:26 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201505250700.t4P70QBk005745@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall - 2015-05-19 - Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------ ---------- --------------------------------------------------------------- #30 [3] bro jsbarber [4] 2015-05-24 Use a common Packet format and preserve layer 2 information [5] #1 [6] bro-plugins jsbarber [7] 2015-05-23 Use a common Packet format and preserve layer 2 information [8] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #30 https://github.com/bro/bro/pull/30 [4] jsbarber https://github.com/jsbarber [5] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [6] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From noreply at bro.org Tue May 26 00:00:27 2015 From: noreply at bro.org (Merge Tracker) Date: Tue, 26 May 2015 00:00:27 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201505260700.t4Q70ROK022624@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ---------- ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall - 2015-05-19 - Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------ ---------- --------------------------------------------------------------- #30 [3] bro jsbarber [4] 2015-05-24 Use a common Packet format and preserve layer 2 information [5] #1 [6] bro-plugins jsbarber [7] 2015-05-23 Use a common Packet format and preserve layer 2 information [8] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #30 https://github.com/bro/bro/pull/30 [4] jsbarber https://github.com/jsbarber [5] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [6] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From jira at bro-tracker.atlassian.net Tue May 26 08:04:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 26 May 2015 10:04:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1399) topic/seth/deflate-missing-headers-fix In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1399: --------------------------------- Assignee: Robin Sommer > topic/seth/deflate-missing-headers-fix > -------------------------------------- > > Key: BIT-1399 > URL: https://bro-tracker.atlassian.net/browse/BIT-1399 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > Assignee: Robin Sommer > > Merge request for the topic/seth/deflate-missing-headers-fix branch. > It fixes an issue that is occasionally but regularly seen where servers don't include the headers correctly for deflated content. Browsers cope with this situation just fine and this change makes Bro also deal with the situation. -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Tue May 26 08:10:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 26 May 2015 10:10:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1399) topic/seth/deflate-missing-headers-fix In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20800#comment-20800 ] Robin Sommer commented on BIT-1399: ----------------------------------- Could we postpone this to 2.5? It's a nontrivial code change, and not a regression. Messing with that code during the beta worries me a bit. > topic/seth/deflate-missing-headers-fix > -------------------------------------- > > Key: BIT-1399 > URL: https://bro-tracker.atlassian.net/browse/BIT-1399 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > Assignee: Robin Sommer > > Merge request for the topic/seth/deflate-missing-headers-fix branch. > It fixes an issue that is occasionally but regularly seen where servers don't include the headers correctly for deflated content. Browsers cope with this situation just fine and this change makes Bro also deal with the situation. -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Tue May 26 08:41:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 26 May 2015 10:41:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1401) Broker fixes for 2.4 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1401: ------------------------------ Resolution: Merged Status: Closed (was: Open) > Broker fixes for 2.4 > -------------------- > > Key: BIT-1401 > URL: https://bro-tracker.atlassian.net/browse/BIT-1401 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro, Broker > Reporter: Jon Siwek > Fix For: 2.4 > > > Just making a reminder ticket of what Broker changes to include in a 0.3.1 Broker release and the final 2.4 Bro release. > This commit should be considered: > https://github.com/bro/broker/commit/8fc6938017dc15acfb26fa29e6ad0933019781c5 -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Tue May 26 08:41:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 26 May 2015 10:41:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1401) Broker fixes for 2.4 In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20801#comment-20801 ] Robin Sommer commented on BIT-1401: ----------------------------------- Looks like this is merged already. As we aren't branching, everything in master will make it into 2.4. > Broker fixes for 2.4 > -------------------- > > Key: BIT-1401 > URL: https://bro-tracker.atlassian.net/browse/BIT-1401 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro, Broker > Reporter: Jon Siwek > Fix For: 2.4 > > > Just making a reminder ticket of what Broker changes to include in a 0.3.1 Broker release and the final 2.4 Bro release. > This commit should be considered: > https://github.com/bro/broker/commit/8fc6938017dc15acfb26fa29e6ad0933019781c5 -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Tue May 26 08:43:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 26 May 2015 10:43:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1400) topic/jsiwek/mime-multipart-boundary-leniency In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20802#comment-20802 ] Robin Sommer commented on BIT-1400: ----------------------------------- Yeah, agree that this might be better for 2.5, similar to BIT-1399 as well. > topic/jsiwek/mime-multipart-boundary-leniency > --------------------------------------------- > > Key: BIT-1400 > URL: https://bro-tracker.atlassian.net/browse/BIT-1400 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Jon Siwek > Assignee: Seth Hall > Fix For: 2.4 > > > Seth had a private pcap showing HTTP multipart content using boundary strings containing the '<' and '>' characters which causes HTTP/MIME content parsing to fail. This branch changes it so those characters are allowed (even though not explicitly permitted by the RFC). It feels a bit hacky to me (but so do most changes I've done to HTTP/MIME analyzers), so please review and check if the analysis looks "more correct" now. > I scheduled this for 2.4 because I think Seth mentioned it might be something to try to get fixed in the final release, but it might be better to put it as part of 2.5 -- it's not really a severe bug but more of an oddity from a particular HTTP implementation and Bro's behavior with respect to it hasn't changed anytime recently (i.e. it's not a regression). -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Tue May 26 08:45:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 26 May 2015 10:45:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20803#comment-20803 ] Robin Sommer commented on BIT-1396: ----------------------------------- I'm still not sure if we have a regression here: is this something 2.3 handled better? > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Tue May 26 08:51:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 26 May 2015 10:51:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1395) Elasticsearch plugin README outdated In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20804#comment-20804 ] Robin Sommer commented on BIT-1395: ----------------------------------- Needs to be {{Bro/ElasticSearch/logs-to-elasticsearch}} now. I'll fix it. > Elasticsearch plugin README outdated > ------------------------------------ > > Key: BIT-1395 > URL: https://bro-tracker.atlassian.net/browse/BIT-1395 > Project: Bro Issue Tracker > Issue Type: Problem > Components: bro-aux > Affects Versions: git/master > Reporter: Justin Azoff > Priority: Trivial > Labels: documentation > Fix For: 2.4 > > > The elasticsearch plugin readme still says to do: > @load tuning/logs-to-elasticsearch > but this file no longer exists. I'm not sure if it is loaded automatically after the plugin is installed or if a different load statement is neader. -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Tue May 26 08:53:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 26 May 2015 10:53:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1387) segfault in nb_dns.cc when nameserver is not reachable In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1387: --------------------------------- Assignee: Robin Sommer > segfault in nb_dns.cc when nameserver is not reachable > ------------------------------------------------------ > > Key: BIT-1387 > URL: https://bro-tracker.atlassian.net/browse/BIT-1387 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master, 2.3 > Environment: Ubuntu 14.10 and Debian Minimal 7.8 > Reporter: Frank Meier > Assignee: Robin Sommer > Fix For: 2.4 > > > The segfault happens, if a nameserver is set in /etc/resolv.conf, but the network > of the nameserver is not reachable: > $ cat /etc/resolv.conf > nameserver 192.168.1.1 > $ cat dns.bro > event bro_init() { > when ( local result = lookup_hostname("example.com") ) { > } > } > $ bro -v > bro version 2.3-793 > $ bro dns.bro > warning in /home/franky/bro-git/bro/scripts/base/init-bare.bro, line 1: problem initializing NB-DNS: connect(192.168.1.1): Network is unreachable > warning: can't issue DNS request > warning: can't issue DNS request > Segmentation fault (core dumped) > The segfault does not happen, if BRO_DNS_FAKE ist set to on or off: > $ BRO_DNS_FAKE=0 bro dns.bro > warning in /home/franky/bro-git/bro/scripts/base/init-bare.bro, line 1: problem initializing NB-DNS: connect(192.168.1.1): Network is unreachable > $ BRO_DNS_FAKE=1 bro dns.bro > warning in /home/franky/bro-git/bro/scripts/base/init-bare.bro, line 1: problem initializing NB-DNS: connect(192.168.1.1): Network is unreachable > Here is the backtrace: > $ gdb bro /tmp/core > GNU gdb (Ubuntu 7.8-1ubuntu4) 7.8.0.20141001-cvs > [...] > Core was generated by `bro dns.bro'. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 nb_dns_fd (nd=0x0) at /home/franky/bro-git/bro/src/nb_dns.c:176 > 176 return (nd->s); > (gdb) bt > #0 nb_dns_fd (nd=0x0) at /home/franky/bro-git/bro/src/nb_dns.c:176 > #1 0x0000000000567c1d in DNS_Mgr::AnswerAvailable (this=, timeout=0) at /home/franky/bro-git/bro/src/DNS_Mgr.cc:1425 > #2 0x000000000056c24a in DNS_Mgr::DoProcess (this=0x15c1410, flush=false) at /home/franky/bro-git/bro/src/DNS_Mgr.cc:1382 > #3 0x000000000056c420 in DNS_Mgr::Flush (this=0x15c1410) at /home/franky/bro-git/bro/src/DNS_Mgr.cc:1334 > #4 0x0000000000540126 in done_with_network () at /home/franky/bro-git/bro/src/main.cc:316 > #5 0x000000000051f679 in main (argc=, argv=) at /home/franky/bro-git/bro/src/main.cc:1216 > fix option 1: > diff --git a/src/DNS_Mgr.cc b/src/DNS_Mgr.cc > index 11fd258..08f76df 100644 > --- a/src/DNS_Mgr.cc > +++ b/src/DNS_Mgr.cc > @@ -1422,6 +1422,10 @@ void DNS_Mgr::DoProcess(bool flush) > > int DNS_Mgr::AnswerAvailable(int timeout) > { > + if (!nb_dns) { > + reporter->Warning("nb_dns_fd() failed in DNS_Mgr::WaitForReplies"); > + return -1; > + } > int fd = nb_dns_fd(nb_dns); > if ( fd < 0 ) > { > fix option 2: > diff --git a/src/nb_dns.c b/src/nb_dns.c > index 33a0083..22778e2 100644 > --- a/src/nb_dns.c > +++ b/src/nb_dns.c > @@ -172,7 +172,9 @@ nb_dns_finish(struct nb_dns_info *nd) > int > nb_dns_fd(struct nb_dns_info *nd) > { > - > + if (!nd) { > + return -1; > + } > return (nd->s); > } -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Tue May 26 09:01:00 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Tue, 26 May 2015 11:01:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20805#comment-20805 ] Daniel Thayer commented on BIT-1396: ------------------------------------ I would argue that 2.4 handles logs (slightly) better than 2.3. > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Tue May 26 09:11:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 26 May 2015 11:11:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1402) New SSL::Invalid_Server_Cert in test-suite In-Reply-To: References: Message-ID: Robin Sommer created BIT-1402: --------------------------------- Summary: New SSL::Invalid_Server_Cert in test-suite Key: BIT-1402 URL: https://bro-tracker.atlassian.net/browse/BIT-1402 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Reporter: Robin Sommer Assignee: Johanna Amann Fix For: 2.4 I'm getting two additional {{SSL::Invalid_Server_Cert}} with the private test-suite, presumably due to an OpenSSL version change regarding MD5 handling. Can we revert behavior back to the previous one with recent OpenSSL versions? {code} +XXXXXXXXXX.XXXXXX XXXXXXXXXXX X 2012 Y 443 - - - tcp SSL::Invalid_Server_Cert SSL certificate validation failed with (certificate signature failure) CN=XXX X Y 443 - bro Notice::ACTION_LOG 3600.000000 F - - - - - +XXXXXXXXXX.XXXXXX XXXXXXXXXXX X 2013 Y 443 - - - tcp SSL::Invalid_Server_Cert SSL certificate validation failed with (certificate signature failure) CN=XXX X Y - bro Notice::ACTION_LOG 3600.000000 F - - - - - {code} -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Tue May 26 09:11:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 26 May 2015 11:11:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1398) PPPoE PCAP stripping laters In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1398: ------------------------------ Priority: Normal (was: High) > PPPoE PCAP stripping laters > --------------------------- > > Key: BIT-1398 > URL: https://bro-tracker.atlassian.net/browse/BIT-1398 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.3 > Environment: Ubuntu 12.04.5 , pf_ring > Reporter: Jason > > Recently I discovered what I believe to be a problem with Bro's packet collection of PPPoE traffic. This occurs both on the wire and when reading in PCAP. > Here is a sample SSL session over PPPoE as captured by tcpdump: > 12:58:27.914864568 PPPoE [ses 0x279a] IP 192.168.110.235.25095 > 192.168.162.218.443: Flags [S], seq 2317077818, win 65535, options [mss 1380,nop,wscale 9,sackOK,TS val 139402792 ecr 0], length 0 > 12:58:28.091544568 PPPoE [ses 0x279a] IP 192.168.162.218.443 > 192.168.110.235.25095: Flags [S.], seq 2303200074, ack 2317077819, win 5792, options [mss 1460,sackOK,TS val 1200789536 ecr 139402792,nop,wscale 7], length 0 > 12:58:28.092020568 PPPoE [ses 0x279a] IP 192.168.110.235.25095 > 192.168.162.218.443: Flags [.], ack 1, win 513, options [nop,nop,TS val 139402972 ecr 1200789536], length 0 > 12:58:28.092579568 PPPoE [ses 0x279a] IP 192.168.110.235.25095 > 192.168.162.218.443: Flags [P.], seq 1:257, ack 1, win 513, options [nop,nop,TS val 139402972 ecr 1200789536], length 256 > 12:58:28.268976568 PPPoE [ses 0x279a] IP 192.168.162.218.443 > 192.168.110.235.25095: Flags [.], ack 257, win 54, options [nop,nop,TS val 1200789713 ecr 139402972], length 0 > Running this capture through Bro results in a valid ssl.log: > 1431435508.092579 C2fjf233dO59LO7sj9 192.168.110.235 25095 192.168.162.218 443 TLSv10 TLS_DHE_RSA_WITH_AES_256_CBC_SHA - some_website.com 7e710c9504f77e9fc8d18121ed965a25119c673b6b4e0a07b5bfcd5baadae534 - T - - - - -- > But the resulting PCAP coming out of Bro for the same packets looks like this: > 12:58:27.914864256 40:00:3f:06:da:8a > 45:00:00:3c:aa:49, ethertype Unknown (0x6e36), length 82: > 12:58:28.091544552 40:00:30:06:93:d4 > 45:00:00:3c:00:00, ethertype Unknown (0x36ec), length 82: > 12:58:28.092020256 40:00:3f:06:da:84 > 45:00:00:34:aa:57, ethertype Unknown (0x6e36), length 74: > 12:58:28.092579152 40:00:3f:06:d9:82 > 45:00:01:34:aa:59, ethertype Unknown (0x6e36), length 330: > 12:58:28.268976656 40:00:30:06:00:42 > 45:00:00:34:93:9a, ethertype Unknown (0x36ec), length 74: > Please let me know if you need any additional information. > Jason -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From robin at icir.org Tue May 26 09:12:43 2015 From: robin at icir.org (Robin Sommer) Date: Tue, 26 May 2015 09:12:43 -0700 Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: <20150526161243.GN33919@icir.org> On Tue, May 26, 2015 at 11:01 -0500, you wrote: > I would argue that 2.4 handles logs (slightly) better than 2.3. I believe you, generally. :) But what about this specific case? I'm sorry but I just don't get from the discussion here if this is a new problem, or something that was never handled differently/better. From jira at bro-tracker.atlassian.net Tue May 26 09:14:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Tue, 26 May 2015 11:14:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20806#comment-20806 ] Robin Sommer commented on BIT-1396: ----------------------------------- I believe you, generally. :) But what about this specific case? I'm sorry but I just don't get from the discussion here if this is a new problem, or something that was never handled differently/better. > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Tue May 26 09:48:00 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Tue, 26 May 2015 11:48:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20807#comment-20807 ] Daniel Thayer commented on BIT-1396: ------------------------------------ I don't believe there is really anything new here (the way logs get archived hasn't really changed since at least Bro 2.0), but for the next release I'd like to change the way logs are archived to make the whole procedure more robust and less confusing to the user (perhaps broctld could play a role in this). I've added a small section to the broctl user manual describing how a user could deal with this situation (hopefully that will clear up some of the confusion). > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Tue May 26 15:20:00 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Tue, 26 May 2015 17:20:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1403) topic/dnthayer/fix-2.4-beta In-Reply-To: References: Message-ID: Daniel Thayer created BIT-1403: ---------------------------------- Summary: topic/dnthayer/fix-2.4-beta Key: BIT-1403 URL: https://bro-tracker.atlassian.net/browse/BIT-1403 Project: Bro Issue Tracker Issue Type: Problem Components: BroControl Reporter: Daniel Thayer Fix For: 2.4 The branch topic/dnthayer/fix-2.4-beta in the broctl branch contains fixes for the following issues found after the 2.4 beta was released: 1) fix use of "./configure --with-python" 2) ssh_runner Python 3.4 compatibility fix 3) added a helpful message that appears first time broctl is run (tells user to run "broctl deploy") 4) fix "ps.bro" plugin output 5) Python 3 compatibility fix for warning messages when config has changed 6) Fix ssh_runner to not crash when binary data is sent 7) Improve error messages involving the SQLite state database file 8) Add sanity checks on broctl option values during broctl initialization to avoid cryptic error messages 9) Improve visibility of archive-log error messages in the stderr.log file (now a user can just grep for "archive-log" to check if any errors occurred) 10) Show "help" output when a user runs broctl non-interactively with an unknown command 11) Improve error messages related to the "env_vars" option, and don't remove quotes in the value 12) Don't use daemon threads in ssh_runner 13) Added more tests, and improved some tests 14) Improved broctl documentation -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Tue May 26 15:31:00 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Tue, 26 May 2015 17:31:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1403) topic/dnthayer/fix-2.4-beta In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1403: ------------------------------- Status: Merge Request (was: Open) > topic/dnthayer/fix-2.4-beta > --------------------------- > > Key: BIT-1403 > URL: https://bro-tracker.atlassian.net/browse/BIT-1403 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: Daniel Thayer > Fix For: 2.4 > > > The branch topic/dnthayer/fix-2.4-beta in the broctl branch contains fixes for > the following issues found after the 2.4 beta was released: > 1) fix use of "./configure --with-python" > 2) ssh_runner Python 3.4 compatibility fix for Popen > 3) added a helpful message that appears first time broctl is run (tells user to run "broctl deploy") > 4) fix "ps.bro" plugin output > 5) Python 3 compatibility fix for warning messages when config has changed > 6) Fix ssh_runner to not crash when binary data is sent > 7) Improve error messages involving the SQLite state database file > 8) Add sanity checks on broctl option values during broctl initialization to avoid cryptic error messages > 9) Improve visibility of archive-log error messages in the stderr.log file (now a user can just grep for "archive-log" to check if any errors occurred) > 10) Show "help" output when a user runs broctl non-interactively with an unknown command > 11) Improve error messages related to the "env_vars" option, and don't remove quotes in the value > 12) Don't use daemon threads in ssh_runner > 13) Added more tests, and improved some tests > 14) Improved broctl documentation -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Tue May 26 15:31:00 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Tue, 26 May 2015 17:31:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1403) topic/dnthayer/fix-2.4-beta In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1403: ------------------------------- Description: The branch topic/dnthayer/fix-2.4-beta in the broctl branch contains fixes for the following issues found after the 2.4 beta was released: 1) fix use of "./configure --with-python" 2) ssh_runner Python 3.4 compatibility fix for Popen 3) added a helpful message that appears first time broctl is run (tells user to run "broctl deploy") 4) fix "ps.bro" plugin output 5) Python 3 compatibility fix for warning messages when config has changed 6) Fix ssh_runner to not crash when binary data is sent 7) Improve error messages involving the SQLite state database file 8) Add sanity checks on broctl option values during broctl initialization to avoid cryptic error messages 9) Improve visibility of archive-log error messages in the stderr.log file (now a user can just grep for "archive-log" to check if any errors occurred) 10) Show "help" output when a user runs broctl non-interactively with an unknown command 11) Improve error messages related to the "env_vars" option, and don't remove quotes in the value 12) Don't use daemon threads in ssh_runner 13) Added more tests, and improved some tests 14) Improved broctl documentation was: The branch topic/dnthayer/fix-2.4-beta in the broctl branch contains fixes for the following issues found after the 2.4 beta was released: 1) fix use of "./configure --with-python" 2) ssh_runner Python 3.4 compatibility fix 3) added a helpful message that appears first time broctl is run (tells user to run "broctl deploy") 4) fix "ps.bro" plugin output 5) Python 3 compatibility fix for warning messages when config has changed 6) Fix ssh_runner to not crash when binary data is sent 7) Improve error messages involving the SQLite state database file 8) Add sanity checks on broctl option values during broctl initialization to avoid cryptic error messages 9) Improve visibility of archive-log error messages in the stderr.log file (now a user can just grep for "archive-log" to check if any errors occurred) 10) Show "help" output when a user runs broctl non-interactively with an unknown command 11) Improve error messages related to the "env_vars" option, and don't remove quotes in the value 12) Don't use daemon threads in ssh_runner 13) Added more tests, and improved some tests 14) Improved broctl documentation > topic/dnthayer/fix-2.4-beta > --------------------------- > > Key: BIT-1403 > URL: https://bro-tracker.atlassian.net/browse/BIT-1403 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: Daniel Thayer > Fix For: 2.4 > > > The branch topic/dnthayer/fix-2.4-beta in the broctl branch contains fixes for > the following issues found after the 2.4 beta was released: > 1) fix use of "./configure --with-python" > 2) ssh_runner Python 3.4 compatibility fix for Popen > 3) added a helpful message that appears first time broctl is run (tells user to run "broctl deploy") > 4) fix "ps.bro" plugin output > 5) Python 3 compatibility fix for warning messages when config has changed > 6) Fix ssh_runner to not crash when binary data is sent > 7) Improve error messages involving the SQLite state database file > 8) Add sanity checks on broctl option values during broctl initialization to avoid cryptic error messages > 9) Improve visibility of archive-log error messages in the stderr.log file (now a user can just grep for "archive-log" to check if any errors occurred) > 10) Show "help" output when a user runs broctl non-interactively with an unknown command > 11) Improve error messages related to the "env_vars" option, and don't remove quotes in the value > 12) Don't use daemon threads in ssh_runner > 13) Added more tests, and improved some tests > 14) Improved broctl documentation -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Tue May 26 15:37:00 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Tue, 26 May 2015 17:37:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1397) broctl --help is mysterious In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20808#comment-20808 ] Daniel Thayer commented on BIT-1397: ------------------------------------ This issue is addressed by BIT-1403. I've improved the broctl documentation, improved the SQLite database file error messages, and added "broctl help" output when a user types an unknown command (such as "broctl --help"). > broctl --help is mysterious > --------------------------- > > Key: BIT-1397 > URL: https://bro-tracker.atlassian.net/browse/BIT-1397 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.4 > Environment: MacOS Mavericks > Reporter: Vern Paxson > Fix For: 2.4 > > > For a newly installed Bro 2.4 beta, issuing "{{broctl --help}}" yields the cryptic output: > {{Error: unable to open database file: /usr/local/bro/spool/state.db}} -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Tue May 26 15:37:01 2015 From: jira at bro-tracker.atlassian.net (Daniel Thayer (JIRA)) Date: Tue, 26 May 2015 17:37:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1397) broctl --help is mysterious In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Thayer updated BIT-1397: ------------------------------- Fix Version/s: 2.4 > broctl --help is mysterious > --------------------------- > > Key: BIT-1397 > URL: https://bro-tracker.atlassian.net/browse/BIT-1397 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.4 > Environment: MacOS Mavericks > Reporter: Vern Paxson > Fix For: 2.4 > > > For a newly installed Bro 2.4 beta, issuing "{{broctl --help}}" yields the cryptic output: > {{Error: unable to open database file: /usr/local/bro/spool/state.db}} -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From noreply at bro.org Wed May 27 00:00:35 2015 From: noreply at bro.org (Merge Tracker) Date: Wed, 27 May 2015 00:00:35 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201505270700.t4R70ZmA003607@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1403 [1] BroControl Daniel Thayer - 2015-05-26 2.4 Normal topic/dnthayer/fix-2.4-beta [2] BIT-1399 [3] Bro Seth Hall Robin Sommer 2015-05-26 - Normal topic/seth/deflate-missing-headers-fix [4] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------ ---------- ---------------------------------------------------------------- #30 [5] bro jsbarber [6] 2015-05-24 Use a common Packet format and preserve layer 2 information [7] #1 [8] bro-plugins jsbarber [9] 2015-05-23 Use a common Packet format and preserve layer 2 information [10] [1] BIT-1403 https://bro-tracker.atlassian.net/browse/BIT-1403 [2] fix-2.4-beta https://github.com/bro/brocontrol/tree/topic/dnthayer/fix-2.4-beta [3] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [4] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [5] Pull Request #30 https://github.com/bro/bro/pull/30 [6] jsbarber https://github.com/jsbarber [7] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [8] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [9] jsbarber https://github.com/jsbarber [10] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From jira at bro-tracker.atlassian.net Wed May 27 12:27:00 2015 From: jira at bro-tracker.atlassian.net (Johanna Amann (JIRA)) Date: Wed, 27 May 2015 14:27:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1402) New SSL::Invalid_Server_Cert in test-suite In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johanna Amann updated BIT-1402: ------------------------------- Resolution: Fixed Status: Closed (was: Open) Fixed by setting OPENSSL_ENABLE_MD5_VERIFY in btest.cfg. Fedora introduces this non-standard environment variable in one of their distro-specific patches to OpenSSL (openssl-1.0.1e-no-md5-verify.patch); if it is not set, MD5 verification is not permitted. Committed in 5147b0bb02588f223cf04fac2ac3c3d9a7640217 > New SSL::Invalid_Server_Cert in test-suite > ------------------------------------------ > > Key: BIT-1402 > URL: https://bro-tracker.atlassian.net/browse/BIT-1402 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Robin Sommer > Assignee: Johanna Amann > Fix For: 2.4 > > > I'm getting two additional {{SSL::Invalid_Server_Cert}} with the private test-suite, presumably due to an OpenSSL version change regarding MD5 handling. Can we revert behavior back to the previous one with recent OpenSSL versions? > {code} > +XXXXXXXXXX.XXXXXX XXXXXXXXXXX X 2012 Y 443 - - - tcp SSL::Invalid_Server_Cert SSL certificate validation failed with (certificate signature failure) CN=XXX X Y 443 - bro Notice::ACTION_LOG 3600.000000 F - - - - - > +XXXXXXXXXX.XXXXXX XXXXXXXXXXX X 2013 Y 443 - - - tcp SSL::Invalid_Server_Cert SSL certificate validation failed with (certificate signature failure) CN=XXX X Y - bro Notice::ACTION_LOG 3600.000000 F - - - - - > {code} -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From noreply at bro.org Thu May 28 00:00:18 2015 From: noreply at bro.org (Merge Tracker) Date: Thu, 28 May 2015 00:00:18 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201505280700.t4S70Ira009160@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ------------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1403 [1] BroControl Daniel Thayer - 2015-05-26 2.4 Normal topic/dnthayer/fix-2.4-beta [2] BIT-1399 [3] Bro Seth Hall Robin Sommer 2015-05-26 - Normal topic/seth/deflate-missing-headers-fix [4] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------ ---------- ---------------------------------------------------------------- #30 [5] bro jsbarber [6] 2015-05-24 Use a common Packet format and preserve layer 2 information [7] #1 [8] bro-plugins jsbarber [9] 2015-05-23 Use a common Packet format and preserve layer 2 information [10] [1] BIT-1403 https://bro-tracker.atlassian.net/browse/BIT-1403 [2] fix-2.4-beta https://github.com/bro/brocontrol/tree/topic/dnthayer/fix-2.4-beta [3] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [4] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [5] Pull Request #30 https://github.com/bro/bro/pull/30 [6] jsbarber https://github.com/jsbarber [7] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [8] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [9] jsbarber https://github.com/jsbarber [10] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From jira at bro-tracker.atlassian.net Thu May 28 11:52:02 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 28 May 2015 13:52:02 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1403) topic/dnthayer/fix-2.4-beta In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1403: --------------------------------- Assignee: Robin Sommer > topic/dnthayer/fix-2.4-beta > --------------------------- > > Key: BIT-1403 > URL: https://bro-tracker.atlassian.net/browse/BIT-1403 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: Daniel Thayer > Assignee: Robin Sommer > Fix For: 2.4 > > > The branch topic/dnthayer/fix-2.4-beta in the broctl branch contains fixes for > the following issues found after the 2.4 beta was released: > 1) fix use of "./configure --with-python" > 2) ssh_runner Python 3.4 compatibility fix for Popen > 3) added a helpful message that appears first time broctl is run (tells user to run "broctl deploy") > 4) fix "ps.bro" plugin output > 5) Python 3 compatibility fix for warning messages when config has changed > 6) Fix ssh_runner to not crash when binary data is sent > 7) Improve error messages involving the SQLite state database file > 8) Add sanity checks on broctl option values during broctl initialization to avoid cryptic error messages > 9) Improve visibility of archive-log error messages in the stderr.log file (now a user can just grep for "archive-log" to check if any errors occurred) > 10) Show "help" output when a user runs broctl non-interactively with an unknown command > 11) Improve error messages related to the "env_vars" option, and don't remove quotes in the value > 12) Don't use daemon threads in ssh_runner > 13) Added more tests, and improved some tests > 14) Improved broctl documentation -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Thu May 28 12:14:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 28 May 2015 14:14:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1399) topic/seth/deflate-missing-headers-fix In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1399: ------------------------------ Fix Version/s: 2.4 > topic/seth/deflate-missing-headers-fix > -------------------------------------- > > Key: BIT-1399 > URL: https://bro-tracker.atlassian.net/browse/BIT-1399 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > Assignee: Robin Sommer > Fix For: 2.4 > > > Merge request for the topic/seth/deflate-missing-headers-fix branch. > It fixes an issue that is occasionally but regularly seen where servers don't include the headers correctly for deflated content. Browsers cope with this situation just fine and this change makes Bro also deal with the situation. -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Thu May 28 12:15:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 28 May 2015 14:15:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1399) topic/seth/deflate-missing-headers-fix In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20810#comment-20810 ] Robin Sommer commented on BIT-1399: ----------------------------------- Setting version so that we will discuss. > topic/seth/deflate-missing-headers-fix > -------------------------------------- > > Key: BIT-1399 > URL: https://bro-tracker.atlassian.net/browse/BIT-1399 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > Assignee: Robin Sommer > Fix For: 2.4 > > > Merge request for the topic/seth/deflate-missing-headers-fix branch. > It fixes an issue that is occasionally but regularly seen where servers don't include the headers correctly for deflated content. Browsers cope with this situation just fine and this change makes Bro also deal with the situation. -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Thu May 28 13:35:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 28 May 2015 15:35:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1403) topic/dnthayer/fix-2.4-beta In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1403: ------------------------------ Resolution: Merged (was: Fixed) Status: Closed (was: Merge Request) > topic/dnthayer/fix-2.4-beta > --------------------------- > > Key: BIT-1403 > URL: https://bro-tracker.atlassian.net/browse/BIT-1403 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Reporter: Daniel Thayer > Assignee: Robin Sommer > Fix For: 2.4 > > > The branch topic/dnthayer/fix-2.4-beta in the broctl branch contains fixes for > the following issues found after the 2.4 beta was released: > 1) fix use of "./configure --with-python" > 2) ssh_runner Python 3.4 compatibility fix for Popen > 3) added a helpful message that appears first time broctl is run (tells user to run "broctl deploy") > 4) fix "ps.bro" plugin output > 5) Python 3 compatibility fix for warning messages when config has changed > 6) Fix ssh_runner to not crash when binary data is sent > 7) Improve error messages involving the SQLite state database file > 8) Add sanity checks on broctl option values during broctl initialization to avoid cryptic error messages > 9) Improve visibility of archive-log error messages in the stderr.log file (now a user can just grep for "archive-log" to check if any errors occurred) > 10) Show "help" output when a user runs broctl non-interactively with an unknown command > 11) Improve error messages related to the "env_vars" option, and don't remove quotes in the value > 12) Don't use daemon threads in ssh_runner > 13) Added more tests, and improved some tests > 14) Improved broctl documentation -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Thu May 28 13:35:02 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 28 May 2015 15:35:02 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1397) broctl --help is mysterious In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1397: ------------------------------ Resolution: Merged Status: Closed (was: Open) > broctl --help is mysterious > --------------------------- > > Key: BIT-1397 > URL: https://bro-tracker.atlassian.net/browse/BIT-1397 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl > Affects Versions: 2.4 > Environment: MacOS Mavericks > Reporter: Vern Paxson > Fix For: 2.4 > > > For a newly installed Bro 2.4 beta, issuing "{{broctl --help}}" yields the cryptic output: > {{Error: unable to open database file: /usr/local/bro/spool/state.db}} -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Thu May 28 13:36:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 28 May 2015 15:36:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1395) Elasticsearch plugin README outdated In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1395: ------------------------------ Resolution: Fixed Status: Closed (was: Open) > Elasticsearch plugin README outdated > ------------------------------------ > > Key: BIT-1395 > URL: https://bro-tracker.atlassian.net/browse/BIT-1395 > Project: Bro Issue Tracker > Issue Type: Problem > Components: bro-aux > Affects Versions: git/master > Reporter: Justin Azoff > Priority: Trivial > Labels: documentation > Fix For: 2.4 > > > The elasticsearch plugin readme still says to do: > @load tuning/logs-to-elasticsearch > but this file no longer exists. I'm not sure if it is loaded automatically after the plugin is installed or if a different load statement is neader. -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Thu May 28 13:37:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 28 May 2015 15:37:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1393) The --with-python configure option only partially works In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1393: ------------------------------ Resolution: Merged Status: Closed (was: Open) > The --with-python configure option only partially works > ------------------------------------------------------- > > Key: BIT-1393 > URL: https://bro-tracker.atlassian.net/browse/BIT-1393 > Project: Bro Issue Tracker > Issue Type: Problem > Components: BroControl, trace-summary > Reporter: Daniel Thayer > Assignee: Daniel Thayer > Fix For: 2.4 > > > When building bro with the "--with-python" configure option, some files > still use the default python interpreter. This prevents, for example, BroControl > from working on RHEL 5 systems. -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Thu May 28 13:38:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 28 May 2015 15:38:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1387) segfault in nb_dns.cc when nameserver is not reachable In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20811#comment-20811 ] Robin Sommer commented on BIT-1387: ----------------------------------- Patch is merged, plus some additional checks. Yeah, DNS_Mgr could use some more work (or replacement) but closing the ticket for this issue. > segfault in nb_dns.cc when nameserver is not reachable > ------------------------------------------------------ > > Key: BIT-1387 > URL: https://bro-tracker.atlassian.net/browse/BIT-1387 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master, 2.3 > Environment: Ubuntu 14.10 and Debian Minimal 7.8 > Reporter: Frank Meier > Assignee: Robin Sommer > Fix For: 2.4 > > > The segfault happens, if a nameserver is set in /etc/resolv.conf, but the network > of the nameserver is not reachable: > $ cat /etc/resolv.conf > nameserver 192.168.1.1 > $ cat dns.bro > event bro_init() { > when ( local result = lookup_hostname("example.com") ) { > } > } > $ bro -v > bro version 2.3-793 > $ bro dns.bro > warning in /home/franky/bro-git/bro/scripts/base/init-bare.bro, line 1: problem initializing NB-DNS: connect(192.168.1.1): Network is unreachable > warning: can't issue DNS request > warning: can't issue DNS request > Segmentation fault (core dumped) > The segfault does not happen, if BRO_DNS_FAKE ist set to on or off: > $ BRO_DNS_FAKE=0 bro dns.bro > warning in /home/franky/bro-git/bro/scripts/base/init-bare.bro, line 1: problem initializing NB-DNS: connect(192.168.1.1): Network is unreachable > $ BRO_DNS_FAKE=1 bro dns.bro > warning in /home/franky/bro-git/bro/scripts/base/init-bare.bro, line 1: problem initializing NB-DNS: connect(192.168.1.1): Network is unreachable > Here is the backtrace: > $ gdb bro /tmp/core > GNU gdb (Ubuntu 7.8-1ubuntu4) 7.8.0.20141001-cvs > [...] > Core was generated by `bro dns.bro'. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 nb_dns_fd (nd=0x0) at /home/franky/bro-git/bro/src/nb_dns.c:176 > 176 return (nd->s); > (gdb) bt > #0 nb_dns_fd (nd=0x0) at /home/franky/bro-git/bro/src/nb_dns.c:176 > #1 0x0000000000567c1d in DNS_Mgr::AnswerAvailable (this=, timeout=0) at /home/franky/bro-git/bro/src/DNS_Mgr.cc:1425 > #2 0x000000000056c24a in DNS_Mgr::DoProcess (this=0x15c1410, flush=false) at /home/franky/bro-git/bro/src/DNS_Mgr.cc:1382 > #3 0x000000000056c420 in DNS_Mgr::Flush (this=0x15c1410) at /home/franky/bro-git/bro/src/DNS_Mgr.cc:1334 > #4 0x0000000000540126 in done_with_network () at /home/franky/bro-git/bro/src/main.cc:316 > #5 0x000000000051f679 in main (argc=, argv=) at /home/franky/bro-git/bro/src/main.cc:1216 > fix option 1: > diff --git a/src/DNS_Mgr.cc b/src/DNS_Mgr.cc > index 11fd258..08f76df 100644 > --- a/src/DNS_Mgr.cc > +++ b/src/DNS_Mgr.cc > @@ -1422,6 +1422,10 @@ void DNS_Mgr::DoProcess(bool flush) > > int DNS_Mgr::AnswerAvailable(int timeout) > { > + if (!nb_dns) { > + reporter->Warning("nb_dns_fd() failed in DNS_Mgr::WaitForReplies"); > + return -1; > + } > int fd = nb_dns_fd(nb_dns); > if ( fd < 0 ) > { > fix option 2: > diff --git a/src/nb_dns.c b/src/nb_dns.c > index 33a0083..22778e2 100644 > --- a/src/nb_dns.c > +++ b/src/nb_dns.c > @@ -172,7 +172,9 @@ nb_dns_finish(struct nb_dns_info *nd) > int > nb_dns_fd(struct nb_dns_info *nd) > { > - > + if (!nd) { > + return -1; > + } > return (nd->s); > } -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Thu May 28 13:39:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Thu, 28 May 2015 15:39:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1387) segfault in nb_dns.cc when nameserver is not reachable In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1387: ------------------------------ Resolution: Fixed Status: Closed (was: Open) > segfault in nb_dns.cc when nameserver is not reachable > ------------------------------------------------------ > > Key: BIT-1387 > URL: https://bro-tracker.atlassian.net/browse/BIT-1387 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: git/master, 2.3 > Environment: Ubuntu 14.10 and Debian Minimal 7.8 > Reporter: Frank Meier > Assignee: Robin Sommer > Fix For: 2.4 > > > The segfault happens, if a nameserver is set in /etc/resolv.conf, but the network > of the nameserver is not reachable: > $ cat /etc/resolv.conf > nameserver 192.168.1.1 > $ cat dns.bro > event bro_init() { > when ( local result = lookup_hostname("example.com") ) { > } > } > $ bro -v > bro version 2.3-793 > $ bro dns.bro > warning in /home/franky/bro-git/bro/scripts/base/init-bare.bro, line 1: problem initializing NB-DNS: connect(192.168.1.1): Network is unreachable > warning: can't issue DNS request > warning: can't issue DNS request > Segmentation fault (core dumped) > The segfault does not happen, if BRO_DNS_FAKE ist set to on or off: > $ BRO_DNS_FAKE=0 bro dns.bro > warning in /home/franky/bro-git/bro/scripts/base/init-bare.bro, line 1: problem initializing NB-DNS: connect(192.168.1.1): Network is unreachable > $ BRO_DNS_FAKE=1 bro dns.bro > warning in /home/franky/bro-git/bro/scripts/base/init-bare.bro, line 1: problem initializing NB-DNS: connect(192.168.1.1): Network is unreachable > Here is the backtrace: > $ gdb bro /tmp/core > GNU gdb (Ubuntu 7.8-1ubuntu4) 7.8.0.20141001-cvs > [...] > Core was generated by `bro dns.bro'. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 nb_dns_fd (nd=0x0) at /home/franky/bro-git/bro/src/nb_dns.c:176 > 176 return (nd->s); > (gdb) bt > #0 nb_dns_fd (nd=0x0) at /home/franky/bro-git/bro/src/nb_dns.c:176 > #1 0x0000000000567c1d in DNS_Mgr::AnswerAvailable (this=, timeout=0) at /home/franky/bro-git/bro/src/DNS_Mgr.cc:1425 > #2 0x000000000056c24a in DNS_Mgr::DoProcess (this=0x15c1410, flush=false) at /home/franky/bro-git/bro/src/DNS_Mgr.cc:1382 > #3 0x000000000056c420 in DNS_Mgr::Flush (this=0x15c1410) at /home/franky/bro-git/bro/src/DNS_Mgr.cc:1334 > #4 0x0000000000540126 in done_with_network () at /home/franky/bro-git/bro/src/main.cc:316 > #5 0x000000000051f679 in main (argc=, argv=) at /home/franky/bro-git/bro/src/main.cc:1216 > fix option 1: > diff --git a/src/DNS_Mgr.cc b/src/DNS_Mgr.cc > index 11fd258..08f76df 100644 > --- a/src/DNS_Mgr.cc > +++ b/src/DNS_Mgr.cc > @@ -1422,6 +1422,10 @@ void DNS_Mgr::DoProcess(bool flush) > > int DNS_Mgr::AnswerAvailable(int timeout) > { > + if (!nb_dns) { > + reporter->Warning("nb_dns_fd() failed in DNS_Mgr::WaitForReplies"); > + return -1; > + } > int fd = nb_dns_fd(nb_dns); > if ( fd < 0 ) > { > fix option 2: > diff --git a/src/nb_dns.c b/src/nb_dns.c > index 33a0083..22778e2 100644 > --- a/src/nb_dns.c > +++ b/src/nb_dns.c > @@ -172,7 +172,9 @@ nb_dns_finish(struct nb_dns_info *nd) > int > nb_dns_fd(struct nb_dns_info *nd) > { > - > + if (!nd) { > + return -1; > + } > return (nd->s); > } -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From noreply at bro.org Fri May 29 00:00:17 2015 From: noreply at bro.org (Merge Tracker) Date: Fri, 29 May 2015 00:00:17 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201505290700.t4T70HVE031595@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-28 2.4 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [3] bro yunzheng [4] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [5] #30 [6] bro jsbarber [7] 2015-05-24 Use a common Packet format and preserve layer 2 information [8] #1 [9] bro-plugins jsbarber [10] 2015-05-23 Use a common Packet format and preserve layer 2 information [11] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #31 https://github.com/bro/bro/pull/31 [4] yunzheng https://github.com/yunzheng [5] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [6] Pull Request #30 https://github.com/bro/bro/pull/30 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [9] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From jira at bro-tracker.atlassian.net Fri May 29 08:04:00 2015 From: jira at bro-tracker.atlassian.net (Stephen Hosom (JIRA)) Date: Fri, 29 May 2015 10:04:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1404) decompose_uri() builtin throws errors on URIs with select parameters In-Reply-To: References: Message-ID: Stephen Hosom created BIT-1404: ----------------------------------- Summary: decompose_uri() builtin throws errors on URIs with select parameters Key: BIT-1404 URL: https://bro-tracker.atlassian.net/browse/BIT-1404 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Affects Versions: 2.4 Reporter: Stephen Hosom URIs with odd query strings cause errors in reporter.log. For example: local something = decompose_uri("dfasjdfasdfasdf?asd"); results in: error in /usr/local/bro-master/share/bro/base/utils/urls.bro, line 79: no such index (parts[1]) http://try.bro.org/#/trybro/saved/8505 demonstrates a pretty alright example. -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Fri May 29 11:13:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 29 May 2015 13:13:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1396) Logs disappearing on broctl restart In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1396: ------------------------------ Resolution: Cannot Reproduce Status: Closed (was: Open) > Logs disappearing on broctl restart > ----------------------------------- > > Key: BIT-1396 > URL: https://bro-tracker.atlassian.net/browse/BIT-1396 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Aashish Sharma > Priority: High > Fix For: 2.4 > > > Noticed that on certain restarts of bro-2.4-beta, logs arbitrarily disappear. > Restarts happen as > - broctl check; broctl restart > - broctl check; broctl restart --clean > - broctl restart > or some variant - not precisely sure. But all log files for that duration of restarts are missing -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Fri May 29 11:16:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 29 May 2015 13:16:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1400) topic/jsiwek/mime-multipart-boundary-leniency In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1400: ------------------------------ Fix Version/s: (was: 2.4) 2.5 > topic/jsiwek/mime-multipart-boundary-leniency > --------------------------------------------- > > Key: BIT-1400 > URL: https://bro-tracker.atlassian.net/browse/BIT-1400 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Reporter: Jon Siwek > Assignee: Seth Hall > Fix For: 2.5 > > > Seth had a private pcap showing HTTP multipart content using boundary strings containing the '<' and '>' characters which causes HTTP/MIME content parsing to fail. This branch changes it so those characters are allowed (even though not explicitly permitted by the RFC). It feels a bit hacky to me (but so do most changes I've done to HTTP/MIME analyzers), so please review and check if the analysis looks "more correct" now. > I scheduled this for 2.4 because I think Seth mentioned it might be something to try to get fixed in the final release, but it might be better to put it as part of 2.5 -- it's not really a severe bug but more of an oddity from a particular HTTP implementation and Bro's behavior with respect to it hasn't changed anytime recently (i.e. it's not a regression). -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Fri May 29 11:16:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 29 May 2015 13:16:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1399) topic/seth/deflate-missing-headers-fix In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1399: ------------------------------ Fix Version/s: (was: 2.4) 2.5 > topic/seth/deflate-missing-headers-fix > -------------------------------------- > > Key: BIT-1399 > URL: https://bro-tracker.atlassian.net/browse/BIT-1399 > Project: Bro Issue Tracker > Issue Type: Patch > Components: Bro > Affects Versions: 2.4 > Reporter: Seth Hall > Assignee: Robin Sommer > Fix For: 2.5 > > > Merge request for the topic/seth/deflate-missing-headers-fix branch. > It fixes an issue that is occasionally but regularly seen where servers don't include the headers correctly for deflated content. Browsers cope with this situation just fine and this change makes Bro also deal with the situation. -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Fri May 29 11:17:00 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 29 May 2015 13:17:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1404) decompose_uri() builtin throws errors on URIs with select parameters In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer reassigned BIT-1404: --------------------------------- Assignee: Seth Hall > decompose_uri() builtin throws errors on URIs with select parameters > -------------------------------------------------------------------- > > Key: BIT-1404 > URL: https://bro-tracker.atlassian.net/browse/BIT-1404 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Stephen Hosom > Assignee: Seth Hall > Fix For: 2.4 > > > URIs with odd query strings cause errors in reporter.log. > For example: > local something = decompose_uri("dfasjdfasdfasdf?asd"); > results in: > error in /usr/local/bro-master/share/bro/base/utils/urls.bro, line 79: no such index (parts[1]) > http://try.bro.org/#/trybro/saved/8505 demonstrates a pretty alright example. -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Fri May 29 11:17:01 2015 From: jira at bro-tracker.atlassian.net (Robin Sommer (JIRA)) Date: Fri, 29 May 2015 13:17:01 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1404) decompose_uri() builtin throws errors on URIs with select parameters In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Sommer updated BIT-1404: ------------------------------ Fix Version/s: 2.4 > decompose_uri() builtin throws errors on URIs with select parameters > -------------------------------------------------------------------- > > Key: BIT-1404 > URL: https://bro-tracker.atlassian.net/browse/BIT-1404 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Bro > Affects Versions: 2.4 > Reporter: Stephen Hosom > Assignee: Seth Hall > Fix For: 2.4 > > > URIs with odd query strings cause errors in reporter.log. > For example: > local something = decompose_uri("dfasjdfasdfasdf?asd"); > results in: > error in /usr/local/bro-master/share/bro/base/utils/urls.bro, line 79: no such index (parts[1]) > http://try.bro.org/#/trybro/saved/8505 demonstrates a pretty alright example. -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From noreply at bro.org Sat May 30 00:00:33 2015 From: noreply at bro.org (Merge Tracker) Date: Sat, 30 May 2015 00:00:33 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201505300700.t4U70X91009351@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [3] bro yunzheng [4] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [5] #30 [6] bro jsbarber [7] 2015-05-29 Use a common Packet format and preserve layer 2 information [8] #1 [9] bro-plugins jsbarber [10] 2015-05-23 Use a common Packet format and preserve layer 2 information [11] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #31 https://github.com/bro/bro/pull/31 [4] yunzheng https://github.com/yunzheng [5] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [6] Pull Request #30 https://github.com/bro/bro/pull/30 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [9] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets From jira at bro-tracker.atlassian.net Sat May 30 11:29:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Sat, 30 May 2015 13:29:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1405) Notice framework documentation glitch In-Reply-To: References: Message-ID: Vern Paxson created BIT-1405: -------------------------------- Summary: Notice framework documentation glitch Key: BIT-1405 URL: https://bro-tracker.atlassian.net/browse/BIT-1405 Project: Bro Issue Tracker Issue Type: Problem Components: Documentation Reporter: Vern Paxson Priority: Low The [Notice documentation|https://www.bro.org/sphinx/frameworks/notice.html] includes the phrase "_Users should directly make modifications to the Notice::Info record given as the argument to the hook_". Presumably this is instead "should *not* directly make ..." -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Sat May 30 11:33:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Sat, 30 May 2015 13:33:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1405) Notice framework documentation confusion In-Reply-To: References: Message-ID: [ https://bro-tracker.atlassian.net/browse/BIT-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vern Paxson updated BIT-1405: ----------------------------- Description: The [Notice documentation|https://www.bro.org/sphinx/frameworks/notice.html] includes the phrase "_Users should directly make modifications to the Notice::Info record given as the argument to the hook_". Initially I read this with a presumption that it was a typo and what was meant was instead "should *not* directly make ...". Then when I got to the example I see that it actually does mean go-ahead-and-modify, though presumably it only makes sense to modify some fields (such as $actions) and not others (context provided by the Info record). So this phrasing should be clarified, maybe along the lines of "Users alter notice processing by directly modifying certain fields in the Notice::Info record given as the argument ...". (was: The [Notice documentation|https://www.bro.org/sphinx/frameworks/notice.html] includes the phrase "_Users should directly make modifications to the Notice::Info record given as the argument to the hook_". Presumably this is instead "should *not* directly make ...") Summary: Notice framework documentation confusion (was: Notice framework documentation glitch) > Notice framework documentation confusion > ---------------------------------------- > > Key: BIT-1405 > URL: https://bro-tracker.atlassian.net/browse/BIT-1405 > Project: Bro Issue Tracker > Issue Type: Problem > Components: Documentation > Reporter: Vern Paxson > Priority: Low > > The [Notice documentation|https://www.bro.org/sphinx/frameworks/notice.html] includes the phrase "_Users should directly make modifications to the Notice::Info record given as the argument to the hook_". Initially I read this with a presumption that it was a typo and what was meant was instead "should *not* directly make ...". Then when I got to the example I see that it actually does mean go-ahead-and-modify, though presumably it only makes sense to modify some fields (such as $actions) and not others (context provided by the Info record). So this phrasing should be clarified, maybe along the lines of "Users alter notice processing by directly modifying certain fields in the Notice::Info record given as the argument ...". -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Sat May 30 13:23:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Sat, 30 May 2015 15:23:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1406) Trouble locating -b documentation In-Reply-To: References: Message-ID: Vern Paxson created BIT-1406: -------------------------------- Summary: Trouble locating -b documentation Key: BIT-1406 URL: https://bro-tracker.atlassian.net/browse/BIT-1406 Project: Bro Issue Tracker Issue Type: Problem Components: Documentation Reporter: Vern Paxson I'm trying to locate the documentation for -b (run Bro in "bare" mode) but the paths I'd expect to work aren't. I don't see it or command line arguments (or flags) in the general index at [https://www.bro.org/sphinx/genindex.html], and at [https://www.bro.org/sphinx/search.html] a search on -b doesn't turn up anything. While the "using Bro from the command-line section" mentions the flag, it doesn't link to any documentation for it. -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From jira at bro-tracker.atlassian.net Sat May 30 13:31:00 2015 From: jira at bro-tracker.atlassian.net (Vern Paxson (JIRA)) Date: Sat, 30 May 2015 15:31:00 -0500 (CDT) Subject: [Bro-Dev] [JIRA] (BIT-1407) -f silently fails if base/frameworks/packet-filter isn't loaded In-Reply-To: References: Message-ID: Vern Paxson created BIT-1407: -------------------------------- Summary: -f silently fails if base/frameworks/packet-filter isn't loaded Key: BIT-1407 URL: https://bro-tracker.atlassian.net/browse/BIT-1407 Project: Bro Issue Tracker Issue Type: Problem Components: Bro Reporter: Vern Paxson I know we've been through this before (though searching the tickets in Jira, I couldn't find the thread). But to revisit: the "-f filter" option silently does nothing if base/frameworks/packet-filter isn't loaded (so the scenario here is using -b to suppress its automatic loading). This can lead to seriously confusing behavior. It would be preferable if there's either an error message indicating that the option won't be supported, or if it forced loading of packet-filter. -- This message was sent by Atlassian JIRA (v6.5-OD-04-052#65000) From noreply at bro.org Sun May 31 00:00:20 2015 From: noreply at bro.org (Merge Tracker) Date: Sun, 31 May 2015 00:00:20 -0700 Subject: [Bro-Dev] [Auto] Merge Status Message-ID: <201505310700.t4V70KvR001275@bro-ids.icir.org> Open Merge Requests =================== ID Component Reporter Assignee Updated For Version Priority Summary ------------ ----------- ---------- ------------ ---------- ------------- ---------- ------------------------------------------ BIT-1399 [1] Bro Seth Hall Robin Sommer 2015-05-29 2.5 Normal topic/seth/deflate-missing-headers-fix [2] Open GitHub Pull Requests ========================= Issue Component User Updated Title ------- ----------- ------------- ---------- ---------------------------------------------------------------- #31 [3] bro yunzheng [4] 2015-05-28 Fix BIT-1314: Detect "quantum insert" type of attacks [5] #30 [6] bro jsbarber [7] 2015-05-29 Use a common Packet format and preserve layer 2 information [8] #1 [9] bro-plugins jsbarber [10] 2015-05-23 Use a common Packet format and preserve layer 2 information [11] [1] BIT-1399 https://bro-tracker.atlassian.net/browse/BIT-1399 [2] deflate-missing-headers-fix https://github.com/bro/bro/tree/topic/seth/deflate-missing-headers-fix [3] Pull Request #31 https://github.com/bro/bro/pull/31 [4] yunzheng https://github.com/yunzheng [5] Merge Pull Request #31 with git pull --no-ff --no-commit https://github.com/yunzheng/bro.git topic/bit-1314 [6] Pull Request #30 https://github.com/bro/bro/pull/30 [7] jsbarber https://github.com/jsbarber [8] Merge Pull Request #30 with git pull --no-ff --no-commit https://github.com/jsbarber/bro.git topic/rework-packets [9] Pull Request #1 https://github.com/bro/bro-plugins/pull/1 [10] jsbarber https://github.com/jsbarber [11] Merge Pull Request #1 with git pull --no-ff --no-commit https://github.com/jsbarber/bro-plugins.git topic/rework-packets