From neto at caltech.edu  Thu Jul  3 16:15:56 2014
From: neto at caltech.edu (Naoki Eto)
Date: Thu, 3 Jul 2014 16:15:56 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: some code review (be967c5)
Message-ID: <201407032315.s63NFuEd024473@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit be967c53f4cce48b1caf9ca8b3ecfc19b7ef4a5f
Author: NaokiEto <neto at caltech.edu>
Date:   Thu Jul 3 19:03:11 2014 -0400

    some code review


>---------------------------------------------------------------

be967c53f4cce48b1caf9ca8b3ecfc19b7ef4a5f
 CMakeLists.txt                                  |   2 +
 CMakeLists.txt => CMakeLists.txt~               |   0
 README                                          | 105 ++++++++++++++++
 README~                                         | 112 +++++++++++++++++
 config.h.in => config.h                         |   6 +-
 config.h.in => config.h.in~                     |   0
 etc/timemachine.cfg.in                          |  53 ++++++--
 etc/{timemachine.cfg.in => timemachine.cfg.in~} |  29 +++--
 src/CMakeLists.txt                              |   2 +
 src/{CMakeLists.txt => CMakeLists.txt~}         |   0
 src/Connection.cc                               |  11 ++
 src/{Connection.cc => Connection.cc~}           |  11 ++
 src/DiskIndex.cc                                | 112 ++++++++++++++++-
 src/{DiskIndex.cc => DiskIndex.cc~}             | 109 +++++++++++++++-
 src/DynClass.cc                                 |  16 +++
 src/{DynClass.cc => DynClass.cc~}               |  16 +++
 src/DynClass.hh                                 |   2 +
 src/{DynClass.hh => DynClass.hh~}               |   2 +
 src/Fifo.cc                                     |  36 +++++-
 src/{Fifo.cc => Fifo.cc~}                       |  34 ++++-
 src/FifoDisk.cc                                 |  38 +++++-
 src/{FifoDisk.cc => FifoDisk.cc~}               |  36 +++++-
 src/FifoMem.cc                                  |  42 ++++++-
 src/{FifoMem.cc => FifoMem.cc~}                 |  41 +++++-
 src/FifoMem.hh                                  |   3 +-
 src/{FifoMem.hh => FifoMem.hh~}                 |   3 +-
 src/Index.cc                                    | 111 +++++++++++++++-
 src/{Index.cc => Index.cc~}                     | 111 +++++++++++++++-
 src/Index.hh                                    |   4 +-
 src/{Index.hh => Index.hh~}                     |   3 +-
 src/IndexField.cc                               |   8 +-
 src/{IndexField.cc => IndexField.cc~}           |   6 +
 src/{IndexField.hh => IndexField.hh~}           |   8 +-
 src/IndexHash.cc                                |   1 +
 src/{IndexHash.cc => IndexHash.cc~}             |   0
 src/{LogFile.cc => LogFile.cc~}                 |   0
 src/Storage.cc                                  | 160 +++++++++++++++++++++---
 src/{Storage.cc => Storage.cc~}                 | 160 +++++++++++++++++++++---
 src/cmd_parser.yy                               |   6 +-
 src/{cmd_parser.yy => cmd_parser.yy~}           |   0
 src/main.cc.in                                  |  29 ++++-
 src/{main.cc.in => main.cc.in~}                 |  29 ++++-
 src/pcapnav/pcapnav.c                           |   6 +-
 src/pcapnav/{pcapnav.c => pcapnav.c~}           |   5 +-
 src/tm.h                                        |   2 +-
 src/{tm.h => tm.h~}                             |   2 +-
 46 files changed, 1359 insertions(+), 113 deletions(-)

Diff suppressed because of size. To see it, use:

    git diff-tree --root --patch-with-stat --no-color --find-copies-harder --ignore-space-at-eol --cc be967c53f4cce48b1caf9ca8b3ecfc19b7ef4a5f

From neto at caltech.edu  Thu Jul 10 21:14:57 2014
From: neto at caltech.edu (Naoki Eto)
Date: Thu, 10 Jul 2014 21:14:57 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: started some IPv6
	implementation. Cannot display all the ipv6 packets yet,
	but does display the full ipv6 addresses in the class files.
	(256b528)
Message-ID: <201407110414.s6B4EvZr003480@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit 256b528683f3ff3320b29e25e5360c456f472ca8
Author: NaokiEto <neto at caltech.edu>
Date:   Fri Jul 11 00:01:47 2014 -0400

    started some IPv6 implementation. Cannot display all the ipv6 packets yet, but does display the full ipv6 addresses in the class files.


>---------------------------------------------------------------

256b528683f3ff3320b29e25e5360c456f472ca8
 CMakeLists.txt                              |  12 +
 CMakeLists.txt~                             |  13 +
 README                                      |   2 +-
 README~                                     |   4 +
 TypeBro.h                                   | 721 ++++++++++++++++++++++++++++
 TypeBro.h~                                  | 713 +++++++++++++++++++++++++++
 config.h                                    |   9 -
 config.h.in                                 | 106 ++++
 config.h.in~                                | 106 ++++
 src/Connection.cc                           | 436 ++++++++++++++---
 src/Connection.cc~                          | 438 ++++++++++++++---
 src/Connection.hh                           |  97 ++++
 src/{Connection.hh => Connection.hh~}       |  95 ++++
 src/FifoMem.cc                              |   6 +
 src/FifoMem.cc~                             |   6 +
 src/FifoMem.hh                              |   4 +-
 src/FifoMem.hh~                             |   2 +-
 src/H3.h                                    | 143 ++++++
 src/Hash.h                                  |  89 ++++
 src/Hash.h~                                 |  89 ++++
 src/Hash3.cc                                | 180 +++++++
 src/Hash3.cc~                               | 178 +++++++
 src/IP.cc                                   | 638 ++++++++++++++++++++++++
 src/IP.h                                    | 563 ++++++++++++++++++++++
 src/IPAddr.cc                               | 270 +++++++++++
 src/IPAddr.cc~                              | 270 +++++++++++
 src/IPAddr.h                                | 653 +++++++++++++++++++++++++
 src/IPAddr.h~                               | 653 +++++++++++++++++++++++++
 src/Index.cc                                |   5 +-
 src/Index.cc~                               |   5 +-
 src/IndexField.cc                           | 221 ++++++++-
 src/IndexField.cc~                          | 223 ++++++++-
 src/IndexField.hh                           | 126 ++++-
 src/IndexField.hh~                          | 132 ++++-
 src/Storage.cc                              |  32 +-
 src/Storage.cc~                             |  34 +-
 src/modp_numtoa.h                           | 102 ++++
 src/net_util.h                              | 235 +++++++++
 src/net_util.h~                             | 234 +++++++++
 src/packet_headers.h                        |  13 +-
 src/{packet_headers.h => packet_headers.h~} |  13 +-
 src/util.h                                  | 512 ++++++++++++++++++++
 src/util.h~                                 | 510 ++++++++++++++++++++
 43 files changed, 8657 insertions(+), 236 deletions(-)

Diff suppressed because of size. To see it, use:

    git diff-tree --root --patch-with-stat --no-color --find-copies-harder --ignore-space-at-eol --cc 256b528683f3ff3320b29e25e5360c456f472ca8

From vern at icir.org  Fri Jul 11 20:23:10 2014
From: vern at icir.org (Vern Paxson)
Date: Fri, 11 Jul 2014 20:23:10 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: started some IPv6
	implementation. Cannot display all the ipv6 packets yet,
	but does display the full ipv6 addresses in the class files.
	(256b528)
In-Reply-To: <201407110414.s6B4EvZr003480@bro-ids.icir.org> (Thu,
	10 Jul 2014 21:14:57 MST).
Message-ID: <20140712032310.E3FF62C405C@rock.ICSI.Berkeley.EDU>

>  CMakeLists.txt~                             |  13 +

How did a bunch of *~ files wind up in the repo?

		Vern

From asharma at lbl.gov  Fri Jul 11 21:11:05 2014
From: asharma at lbl.gov (Aashish Sharma)
Date: Fri, 11 Jul 2014 21:11:05 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: started some IPv6
 implementation. Cannot display all the ipv6 packets yet,
 but does display the full ipv6 addresses in the class files. (256b528)
In-Reply-To: <20140712032310.E3FF62C405C@rock.ICSI.Berkeley.EDU>
References: <201407110414.s6B4EvZr003480@bro-ids.icir.org>
	<20140712032310.E3FF62C405C@rock.ICSI.Berkeley.EDU>
Message-ID: <20140712041104.GB31228@yaksha.lbl.gov>

Its a mistake. Likely an effect of git add . 

I will work with Naoki to fix this and clean up this branch which is under development. 

Aashish 

On Fri, Jul 11, 2014 at 08:23:10PM -0700, Vern Paxson wrote:
> >  CMakeLists.txt~                             |  13 +
> 
> How did a bunch of *~ files wind up in the repo?
> 
> 		Vern
> _______________________________________________
> Time-Machine mailing list
> Time-Machine at mailman.ICSI.Berkeley.EDU
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/time-machine

-- 
Aashish Sharma	(asharma at lbl.gov) 				 
Cyber Security, 
Lawrence Berkeley National Laboratory  
http://go.lbl.gov/pgp-aashish 
Office: (510)-495-2680  Cell: (510)-612-7971
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/time-machine/attachments/20140711/2a109aed/attachment.bin 

From neto at caltech.edu  Tue Jul 15 17:49:00 2014
From: neto at caltech.edu (Naoki Eto)
Date: Tue, 15 Jul 2014 17:49:00 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: started implementation of hash
	function from bro. It crashes early still and can loop
	forever. This commit is only to save work so far. (d70ea95)
Message-ID: <201407160049.s6G0n0P0025751@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit d70ea953decb26ac4286d2ba1553dd2cf4b5985a
Author: NaokiEto <neto at caltech.edu>
Date:   Tue Jul 15 20:34:46 2014 -0400

    started implementation of hash function from bro. It crashes early still and can loop forever. This commit is only to save work so far.


>---------------------------------------------------------------

d70ea953decb26ac4286d2ba1553dd2cf4b5985a
 README~                          | 116 ------
 TypeBro.h~                       | 713 ---------------------------------
 config.h.in~                     | 115 ------
 etc/timemachine.cfg.in~          |  69 ----
 src/CMakeLists.txt               |  23 +-
 src/CMakeLists.txt~              |  97 -----
 src/Connection.cc                | 547 +++++++++++++++++++++----
 src/Connection.cc~               | 631 -----------------------------
 src/Connection.hh                | 227 +++++++++--
 src/Connection.hh~               | 400 -------------------
 src/Connections.hh               |   2 +-
 src/Desc.h                       | 196 +++++++++
 src/DiskIndex.cc                 |  12 +-
 src/DiskIndex.cc~                | 545 -------------------------
 src/DiskIndex.hh                 |  12 +-
 src/DynClass.cc                  |   4 +-
 src/DynClass.cc~                 | 260 ------------
 src/DynClass.hh~                 | 132 ------
 src/Fifo.cc~                     | 261 ------------
 src/FifoDisk.cc~                 | 345 ----------------
 src/FifoMem.cc~                  | 445 ---------------------
 src/FifoMem.hh~                  |  99 -----
 src/H3.h                         |  20 +-
 src/Hash.cc                      | 341 +++++++++-------
 src/Hash.h                       |   8 +-
 src/Hash.h~                      |  89 -----
 src/Hash3.cc                     | 180 ---------
 src/Hash3.cc~                    | 178 ---------
 src/{Hash.cc => HashAlt.cc}      |   0
 src/{Hash.hh => HashIterator.hh} |   6 +-
 src/IP.cc                        | 638 -----------------------------
 src/IP.h                         | 563 --------------------------
 src/IPAddr.cc                    | 270 -------------
 src/IPAddr.cc~                   | 270 -------------
 src/IPAddr.h                     | 653 ------------------------------
 src/IPAddr.h~                    | 653 ------------------------------
 src/Index.cc                     |  27 +-
 src/Index.cc~                    | 417 -------------------
 src/Index.hh                     |   5 +-
 src/Index.hh~                    | 362 -----------------
 src/IndexField.cc                | 316 ++++++++++++---
 src/IndexField.cc~               | 591 ---------------------------
 src/IndexField.hh                | 220 ++++++++--
 src/IndexField.hh~               | 578 ---------------------------
 src/IndexHash.cc                 |  71 +++-
 src/IndexHash.cc~                | 270 -------------
 src/IndexHash.hh                 |   2 +-
 src/LogFile.cc~                  |  65 ---
 src/Net.h                        | 116 ++++++
 src/Obj.h                        | 227 +++++++++++
 src/Queue.cc                     |   2 +-
 src/Queue.hh                     |   4 +-
 src/Reporter.h                   | 146 +++++++
 src/Storage.cc                   |   4 +-
 src/Storage.cc~                  | 699 --------------------------------
 src/bro_inet_ntop.c              | 189 +++++++++
 src/bro_inet_ntop.h              |  19 +
 src/cmd_parser.yy~               | 469 ----------------------
 src/input.h                      |  50 +++
 src/main.cc.in~                  | 842 ---------------------------------------
 src/net_util.h~                  | 234 -----------
 src/packet_headers.h~            |  63 ---
 src/tm.h                         |   1 +
 src/tm.h~                        |  75 ----
 src/util-config.h                |   1 +
 src/util-config.h.in             |   1 +
 src/util.cc                      | 103 +++++
 src/util.h                       |   4 +-
 src/util.h~                      | 510 ------------------------
 69 files changed, 2513 insertions(+), 13290 deletions(-)

Diff suppressed because of size. To see it, use:

    git diff-tree --root --patch-with-stat --no-color --find-copies-harder --ignore-space-at-eol --cc d70ea953decb26ac4286d2ba1553dd2cf4b5985a

From neto at caltech.edu  Wed Jul 16 14:52:14 2014
From: neto at caltech.edu (Naoki Eto)
Date: Wed, 16 Jul 2014 14:52:14 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: fixed the silly bug in the
	lookup function (called itself within itself) and attempted a
	fix on the deletion of the entries in old hash table. It
	seems to run to completion,
	but I have not checked rigorously yet. Index files in the indexes
	directory don't seem to be correct. The class files may not
	be entirely correct either,
	seems to only get most of the packets rather than nearly all of the
	packets. I am committing because this seems to run without
	segfault (5900dc0)
Message-ID: <201407162152.s6GLqEHg012406@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit 5900dc0822865b425793eaa88b6a870ce3de4af8
Author: NaokiEto <neto at caltech.edu>
Date:   Wed Jul 16 17:35:03 2014 -0400

    fixed the silly bug in the lookup function (called itself within itself) and attempted a fix on the deletion of the entries in old hash table. It seems to run to completion, but I have not checked rigorously yet. Index files in the indexes directory don't seem to be correct. The class files may not be entirely correct either, seems to only get most of the packets rather than nearly all of the packets. I am committing because this seems to run without segfault


>---------------------------------------------------------------

5900dc0822865b425793eaa88b6a870ce3de4af8
 CMakeLists.txt~   | 104 ------------------------------------------------------
 src/Connection.cc |  20 +++++++++--
 src/Connection.hh |  23 ++++++------
 src/DiskIndex.cc  |   3 +-
 src/Hash.cc       |   9 +++--
 src/Index.cc      |   6 +++-
 src/Index.hh      |   2 +-
 src/IndexEntry.hh |   7 ++++
 src/IndexField.hh |  61 +++++++++++++++++++++-----------
 src/IndexHash.cc  |  26 ++++++++++----
 10 files changed, 109 insertions(+), 152 deletions(-)

diff --git a/CMakeLists.txt~ b/CMakeLists.txt~
deleted file mode 100644
index 565c53a..0000000
--- a/CMakeLists.txt~
+++ /dev/null
@@ -1,104 +0,0 @@
-project(TimeMachine C CXX)
-cmake_minimum_required(VERSION 2.6.3 FATAL_ERROR)
-include(cmake/CommonCMakeConfig.cmake)
-
-file(STRINGS "${CMAKE_CURRENT_SOURCE_DIR}/VERSION" VERSION LIMIT_COUNT 1)
-
-set(PREFIX "${CMAKE_INSTALL_PREFIX}")
-set(ETC "${BRO_ETC_INSTALL_DIR}")
-
-########################################################################
-## Dependency Configuration
-
-include(FindRequiredPackage)
-
-FindRequiredPackage(FLEX)
-FindRequiredPackage(BISON)
-FindRequiredPackage(PCAP)
-FindRequiredPackage(Readline)
-FindRequiredPackage(Threads)
-
-# Optional Dependencies
-
-set(USE_BROCCOLI false)
-find_package(Broccoli)
-if (BROCCOLI_FOUND)
-    set(USE_BROCCOLI true)
-    include_directories(BEFORE ${Broccoli_INCLUDE_DIR})
-    list(APPEND OPTLIBS ${Broccoli_LIBRARY})
-endif ()
-
-if (MISSING_PREREQS)
-    foreach (prereq ${MISSING_PREREQ_DESCS})
-        message(SEND_ERROR ${prereq})
-    endforeach ()
-    message(FATAL_ERROR "Configuration aborted due to missing prerequisites")
-endif ()
-
-set(CMAKE_CXX_FLAGS "-g -Wall")
-
-#detect 32 or 64 bit compiler
-IF(CMAKE_SYSTEM_PROCESSOR MATCHES "^(i.86|x86|x86_64)$")
-    include(CheckTypeSize)
-    check_type_size("void*" SIZEOF_VOID_P BUILTIN_TYPES_ONLY)
-    if (${SIZEOF_VOID_P} EQUAL 8)
-        set(CMAKE_SYSTEM_PROCESSOR_x86 64)
-    else()
-        set(CMAKE_SYSTEM_PROCESSOR_x86 32)
-    endif()
-ENDIF()
-
-########################################################################
-## System Introspection
-
-configure_file(${CMAKE_CURRENT_SOURCE_DIR}/config.h.in
-               ${CMAKE_CURRENT_BINARY_DIR}/config.h)
-
-include_directories(BEFORE ${CMAKE_CURRENT_BINARY_DIR})
-
-########################################################################
-## Recurse on sub-directories
-
-add_subdirectory(src)
-
-configure_file(${CMAKE_CURRENT_SOURCE_DIR}/etc/timemachine.cfg.in
-               ${CMAKE_CURRENT_BINARY_DIR}/etc/timemachine.cfg)
-
-include(InstallPackageConfigFile)
-InstallPackageConfigFile(
-    ${CMAKE_CURRENT_BINARY_DIR}/etc/timemachine.cfg
-    ${ETC}
-    timemachine.cfg)
-
-########################################################################
-## Build Summary
-
-if (TimeMachine_SKIP_INSTALL)
-    set(timemachine_install_summary "Install skipped")
-else ()
-    set(timemachine_install_summary "${CMAKE_INSTALL_PREFIX}")
-endif ()
-
-if (CMAKE_BUILD_TYPE)
-    string(TOUPPER ${CMAKE_BUILD_TYPE} BuildType)
-endif ()
-
-message(
-    "\n==================|  TimeMachine Build Summary  |===================="
-    "\n"
-    "\nInstall prefix:    ${timemachine_install_summary}"
-    "\nConfig File Dir:   ${BRO_ETC_INSTALL_DIR}"
-
-    "\nDebug mode:        ${ENABLE_DEBUG}"
-    "\n"
-    "\nCC:                ${CMAKE_C_COMPILER}"
-    "\nCFLAGS:            ${CMAKE_C_FLAGS} ${CMAKE_C_FLAGS_${BuildType}}"
-    "\nCXX:               ${CMAKE_CXX_COMPILER}"
-    "\nCXXFLAGS:          ${CMAKE_CXX_FLAGS} ${CMAKE_CXX_FLAGS_${BuildType}}"
-    "\nCPP:               ${CMAKE_CXX_COMPILER}"
-    "\n"
-    "\nBroccoli:          ${USE_BROCCOLI}"
-    "\n================================================================\n"
-)
-
-include(UserChangedWarning)
diff --git a/src/Connection.cc b/src/Connection.cc
index 72650fd..b6a4f31 100644
--- a/src/Connection.cc
+++ b/src/Connection.cc
@@ -63,6 +63,9 @@ void ConnectionID4::init(proto_t proto,
     // DEBUG DEBUG DEBUG
 	tmlog(TM_LOG_NOTE, "connection 4: Connection.cc, ~line 48", "connection 4 for ipv4 initialized");
 
+    in4_addr ipv4_d_address;
+    in4_addr ipv4_s_address;
+
     v6.version = 4;
 	v6.proto=proto;
 
@@ -150,6 +153,9 @@ void ConnectionID3::init(proto_t proto,
     // DEBUG DEBUG DEBUG
 	tmlog(TM_LOG_DEBUG, "connection 3: Connection.cc, ~line 71", "connection 3 initialized");
 
+    in4_addr ipv4_d_address;
+    in4_addr ipv4_s_address;
+
     v6.version = 4;
 
 	v6.proto=proto;
@@ -446,14 +452,16 @@ bool ConnectionID3::operator==(const ConnectionID& other) const {
 	return (v6.proto == ((ConnectionID3*)&other)->v6.proto)
 		   && (v6.ip1 == ((ConnectionID3*)&other)->v6.ip1)
 		   && (v6.ip2 == ((ConnectionID3*)&other)->v6.ip2)
-		   && (v6.port2 == ((ConnectionID3*)&other)->v6.port2);
+		   && (v6.port2 == ((ConnectionID3*)&other)->v6.port2)
+           && (v6.version == ((ConnectionID3*)&other)->v6.version);
 }
 
 //TODO: MAke this inline (i.e. move to Connection.hh so that it is
 //consistent with ConnectionID4
 bool ConnectionID2::operator==(const ConnectionID& other) const {
 	return (v6.ip1 == ((ConnectionID2*)&other)->v6.ip1)
-		   && (v6.ip2 == ((ConnectionID2*)&other)->v6.ip2);
+		   && (v6.ip2 == ((ConnectionID2*)&other)->v6.ip2)
+           && (v6.version == ((ConnectionID2*)&other)->v6.version);
 }
 
 void ConnectionID4::getStr(char* s, int maxsize) const {
@@ -798,6 +806,10 @@ hash_t ConnectionID4::hash() const
 
     HashKey* newHashKey = new HashKey(&key, sizeof(key));
 
+    //memcpy(&hash_key, newHashKey->Hash(), 8);
+
+    //hash_key = newHashKey->Hash();
+
 	return newHashKey->Hash();
 	}
 
@@ -844,6 +856,8 @@ hash_t ConnectionID3::hash() const
 
     HashKey* newHashKey = new HashKey(&key, sizeof(key));
 
+    //hash_key = newHashKey->Hash();
+
 	return newHashKey->Hash();
 	}
 
@@ -890,6 +904,8 @@ hash_t ConnectionID2::hash() const
 
     HashKey* newHashKey = new HashKey(&key, sizeof(key));
 
+    //hash_key = newHashKey->Hash();
+
 	return newHashKey->Hash();
 	}
 
diff --git a/src/Connection.hh b/src/Connection.hh
index ba4e2e1..70b8151 100644
--- a/src/Connection.hh
+++ b/src/Connection.hh
@@ -188,6 +188,8 @@ public:
 
     hash_t hash() const;
 
+    hash_t hash_key;
+
 protected:
 	void init(proto_t proto, uint32_t s_ip, uint32_t d_ip,
 			  uint16_t s_port, uint16_t d_port);
@@ -195,6 +197,7 @@ protected:
                uint16_t s_port, uint16_t d_port);
 	//v_t v;
     v6_t v6;
+
 private:
 	static std::string pattern_connection4;
     static std::string pattern6_connection4;
@@ -202,10 +205,6 @@ private:
     static RE2 re6;
 
 	//in6_addr in6; // IPv6 or v4-to-v6-mapped address
-
-
-    in4_addr ipv4_d_address;
-    in4_addr ipv4_s_address;
 };
 
 class ConnectionID3: public ConnectionID {
@@ -341,6 +340,7 @@ public:
 
     hash_t hash() const;
 
+    hash_t hash_key;  
 protected:
 	void init(proto_t proto, uint32_t s_ip, uint32_t d_ip, uint16_t port);
     void init6(proto_t proto, unsigned char s_ip[], unsigned char d_ip[], 
@@ -348,13 +348,11 @@ protected:
 	//v_t v;
     v6_t v6;
 
-private:
+
+//private:
 	//in6_addr in6; // IPv6 or v4-to-v6-mapped address
 
 	//static const uint8_t v4_mapped_prefix[12]; // top 96 bits of v4-mapped-addr
-
-    in4_addr ipv4_d_address;
-    in4_addr ipv4_s_address;
 };
 
 
@@ -468,20 +466,19 @@ public:
 */
 
     hash_t hash() const;
+
+    hash_t hash_key;
+
 protected:
 	void init(uint32_t s_ip, uint32_t d_ip);
     void init6( unsigned char s_ip[], unsigned char d_ip[]);
 	//v_t v;
     v6_t v6;
 
-private:
+//private:
 	//in6_addr in6; // IPv6 or v4-to-v6-mapped address
 
 	//static const uint8_t v4_mapped_prefix[12]; // top 96 bits of v4-mapped-addr
-
-    in4_addr ipv4_d_address;
-    in4_addr ipv4_s_address;
-
 };
 
 
diff --git a/src/DiskIndex.cc b/src/DiskIndex.cc
index 96f0654..addb837 100644
--- a/src/DiskIndex.cc
+++ b/src/DiskIndex.cc
@@ -251,7 +251,7 @@ void IndexFiles<T>::writeIndex( IndexHash *ih) {
                         // a single IndexEntry entry
 	tm_time_t interval[2];  // the current interval
 	tm_time_t range[2] = {0, 0};  // First TS in index and last TS in index
-	uint64_t keysize = 0; // size of key (can be taken from IndexEntry->getKey()->getKeySize() )
+	int keysize = 0; // size of key (can be taken from IndexEntry->getKey()->getKeySize() )
 
 	lock_file_numbers();
     // gets the file name from aggregation level and file number, puts it in a char array
@@ -333,6 +333,7 @@ void IndexFiles<T>::writeIndex( IndexHash *ih) {
     
         // set the indexentry to be written to be disk to be the next visited and deleted entry from the hash table
 		ie=ih->getNextDelete();
+        tmlog(TM_LOG_NOTE, "DiskIndex:wrting", "Deleting the IndexEntry type from the hash table");
 	}
 	tmlog(TM_LOG_DEBUG, T::getIndexNameStatic().c_str(), 
 			"Heigth of tree was: %d. level=%d. we wrote %d entries.", ih->height, ih->level, count);
diff --git a/src/Hash.cc b/src/Hash.cc
index 232b603..c0d7ca6 100644
--- a/src/Hash.cc
+++ b/src/Hash.cc
@@ -20,6 +20,8 @@
 #include "Hash.h"
 
 #include "H3.h"
+
+#include "tm.h"
 const H3<hash_t, UHASH_KEY_SIZE>* h3;
 
 void init_hash_function()
@@ -142,11 +144,14 @@ HashKey::HashKey(const void* arg_key, int arg_size, hash_t arg_hash,
 HashKey::HashKey(const void* bytes, int arg_size)
 	{
 	size = arg_size;
+    tmlog(TM_LOG_NOTE, "HashKey", "Copying Key...");
     //printf("Copying key...\n");
 	key = CopyKey(bytes, size);
     //printf("Creating Hash...");
+    tmlog(TM_LOG_NOTE, "HashKey", "Creating Hash...");
 	hash = HashBytes(key, size);
 	is_our_dynamic = 1;
+    tmlog(TM_LOG_NOTE, "HashKey", "Done creating hash");
 	}
 
 void* HashKey::TakeKey()
@@ -175,8 +180,8 @@ hash_t HashKey::HashBytes(const void* bytes, int size)
 		// H3 doesn't check if size is zero
         //printf("Doing H3...\n");
 
-        //if (size != 0)
-        //    printf("The resulting hash is %lu\n", (*h3)(bytes, size));
+        if (size != 0)
+            tmlog(TM_LOG_NOTE, "HashBytes", "The resulting hash is %lu\n", (*h3)(bytes, size));
 
 		return ( size == 0 ) ? 0 : (*h3)(bytes, size);
 		}
diff --git a/src/Index.cc b/src/Index.cc
index c723609..204537f 100644
--- a/src/Index.cc
+++ b/src/Index.cc
@@ -213,6 +213,7 @@ void Index<T>::addEntry(IndexField *iqe) {
 					disk_index->writeIndex(old);
 				} else {
 					// not disk writer, clear the old hash table, though not deleted
+                    tmlog(TM_LOG_NOTE, "Index.cc", "we are clearing the old hash table");
 					old->clear();
 				}
 #ifdef TM_HEAVY_DEBUG
@@ -249,6 +250,7 @@ void Index<T>::addEntry(IndexField *iqe) {
             // 
 			if (hash_size > 2*old->getNumEntries()) { 
                 // Note that we delete cur - this means we delete the formerly old hash table, which has been written to disk
+                tmlog(TM_LOG_NOTE, "Index.cc", "we are about to delete the current (formerly old) hash table");
 				delete cur;
 				cur = new IndexHash(hash_size/2);
 			}
@@ -387,6 +389,7 @@ void Index<T>::run() {
 	int myqlen; 
     // IndexField pointer to last element in the queue
 	IndexField *iqe;
+    tmlog(TM_LOG_NOTE, "addEntry, run", "the size of the input_q is %d before the lock_queue", input_q.size());
 	lock_queue(); // Must have the lock when calling cond_wait
     // run forever
 	while(1) {
@@ -396,9 +399,10 @@ void Index<T>::run() {
 
         tmlog(TM_LOG_NOTE, "addEntry, run", "the size of the input_q is %d before the cond_wait_queue", input_q.size());
 
+        // called from the cond_broadcast_queue
 		cond_wait_queue();
 
-        tmlog(TM_LOG_NOTE, "addEntry, run", "the size of the input_q is %d", input_q.size());
+        tmlog(TM_LOG_NOTE, "addEntry, run", "the size of the input_q is after the cond_wait_queue %d", input_q.size());
 
 		// XXX: Maybe we should read from the queue in burst of, say 10, 
 		// entries, so that we don't have that many lock(), unlock() calls
diff --git a/src/Index.hh b/src/Index.hh
index 8fe5c9c..63f2bd9 100644
--- a/src/Index.hh
+++ b/src/Index.hh
@@ -193,7 +193,7 @@ protected:
 	 * YOU MUST HOLD THE queue_lock WHEN CALLING THIS */
 	void cond_broadcast_queue() {
 		pthread_cond_broadcast(&queue_cond);
-        tmlog(TM_LOG_DEBUG, "Mantainer IndexThread", "signaling to Maintainer IndexThread that we added something to the queue");
+	    tmlog(TM_LOG_DEBUG, "Mantainer IndexThread", "signaling to Maintainer IndexThread that we added something to the queue");
 	}
 	/** 
 	 * Wait for signal, that data is availabe in the queue 
diff --git a/src/IndexEntry.hh b/src/IndexEntry.hh
index 24b09af..39b96be 100644
--- a/src/IndexEntry.hh
+++ b/src/IndexEntry.hh
@@ -75,23 +75,29 @@ class IndexEntry {
 public:
 	IndexEntry(IndexField* key, tm_time_t t0, tm_time_t t1) :
 	key(key), icount(1), intlist(Interval(t0,t1)) {
+        tmlog(TM_LOG_NOTE, "IndexEntry:IndexEntry()", "Creating an IndexEntry instance with an IndexField type that has timestamp %f", key->ts);
 		curint = &intlist;
 		parent = left = right = NULL;
 		col_next = col_prev = NULL;
 		avlbal=0;
 	};
 	virtual ~IndexEntry() {
+        tmlog(TM_LOG_NOTE, "IndexEntry, deleting", "starting to delete IndexEntry type");  
 		Interval *ci;
 		Interval *ni;
+        tmlog(TM_LOG_NOTE, "IndexEntry, deleting", "about the delete the key, which is of IndexField type");
 		delete key;
+        key = NULL;
 		ci=intlist.next;
 		while(ci) {
+            tmlog(TM_LOG_NOTE, "IndexEntry, deleting", "deleting the linked list of intervals");
 			ni = ci->next;
 			delete ci;
 			ci = ni;
 		}
 	};
 	inline IndexField* getKey() {
+        tmlog(TM_LOG_NOTE, "IndexEntry:getKey()", "accessing IndexEntry's getKey() method");
 		return key;
 	}
 	virtual int update_time(tm_time_t t, tm_time_t d_t, tm_time_t iat) {
@@ -110,6 +116,7 @@ public:
 		return &intlist;
 	};
 	virtual std::string getStr() const {
+        tmlog(TM_LOG_NOTE, "IndexEntry:getStr()", "accessing IndexEntry's getStr() method");
 		std::stringstream ss;
 		ss.setf(std::ios::fixed);
 		ss << "key: " << key->getStr()
diff --git a/src/IndexField.hh b/src/IndexField.hh
index 456e948..442255e 100644
--- a/src/IndexField.hh
+++ b/src/IndexField.hh
@@ -29,7 +29,7 @@ public:
 	virtual ~IndexField() {};
 	virtual const std::string getIndexName() const = 0;
 	static const std::string getIndexNameStatic();
-	virtual const char* getConstKeyPtr() const=0;
+	virtual const unsigned char* getConstKeyPtr() const=0;
 	//  virtual char* getKeyPtr() { return NULL; }
 	virtual const int getKeySize() const=0;
 	virtual void getStr(char* s, int maxsize) const=0;
@@ -52,7 +52,7 @@ public:
 
     virtual hash_t hash() const = 0;
 
-	virtual hash_t getInt() const=0;
+	virtual const unsigned char* getInt() const= 0;
 	// The Timestamp field is only used, when the IndexField is put into
 	// the input_q of an index? Why? We need a timestamp for every entry
 	// in the input_q. If we take the TS out and use a seperate IndexQueueEntry
@@ -64,6 +64,8 @@ public:
 	
 	virtual void getBPFStr(char *, int) const = 0;
 
+    hash_t hash_key;
+
 	//  IndexField(void *);
 	
 };
@@ -125,6 +127,7 @@ public:
 		memcpy((void*)getConstKeyPtr(), p, getKeySize());
 	}
 	virtual ~IPAddress() {
+        tmlog(TM_LOG_NOTE, "IPAddress", "deleting an ipaddress type");
         //delete [] strIP;
     };
 
@@ -168,15 +171,15 @@ public:
 
 	    return newHashKey->Hash();
 	}
-	virtual hash_t getInt() const {
-		return ip_address;
+	virtual const unsigned char* getInt() const {
+		return ipv6_address.s6_addr;
 	}
-	virtual const char* getConstKeyPtr() const {
-		return (const char*)&ip_address;
+	virtual const unsigned char* getConstKeyPtr() const {
+		return ipv6_address.s6_addr;
 	}
 	//  char* getKeyPtr() { return (char*)&ip_address; }
 	virtual const int getKeySize() const {
-		return sizeof(ip_address);
+		return sizeof(ipv6_address.s6_addr);
 	}
 	virtual void getStr(char* s, int maxsize) const;
 	virtual std::string getStr() const;
@@ -199,6 +202,8 @@ public:
 
 	static IndexField* parseQuery(const char *query);
 	virtual void getBPFStr(char *, int) const;
+
+    hash_t hash_key;
 private:
 
 	/**
@@ -213,7 +218,7 @@ private:
 
 	static const uint8_t v4_mapped_prefix[12]; // top 96 bits of v4-mapped-addr
 
-	uint32_t ip_address;
+	//uint32_t ip_address;
     unsigned char ip6_address[16]; // unsigned char ip6_address[16]
  	static std::string pattern;
     static std::string pattern6;
@@ -328,11 +333,17 @@ public:
 	Port(uint16_t port): port(port) {  /* printf("Port(%u)\n", port); */
 	}
 	virtual ~Port() {}
-	virtual hash_t getInt() const {
-		return port;
+
+    // I don't think this is ever used, so returning 0 should be ok
+	virtual const unsigned char* getInt() const {
+        //unsigned char portInt[16];
+
+        //memcpy(portInt, (unsigned char*) &port, sizeof(uint16_t));
+
+		return 0;
 	}
-	virtual const char* getConstKeyPtr() const {
-		return (const char*)&port;
+	virtual const unsigned char* getConstKeyPtr() const {
+		return (const unsigned char*)&port;
 	};
 	//  char* getKeyPtr() { return (char*)&port; };
 	virtual const int getKeySize() const {
@@ -390,6 +401,8 @@ protected:
 private:
 	in6_addr ipv6_address; // IPv6 or v4-to-v6-mapped address
 
+    //const unsigned char portInt[16];
+
 	static const uint8_t v4_mapped_prefix[12]; // top 96 bits of v4-mapped-addr
 };
 
@@ -470,11 +483,11 @@ public:
 		return c_id.hash();
 	}
 
-	hash_t getInt() const {
+	const unsigned char* getInt() const {
 		return 0;
 	}
-	virtual const char* getConstKeyPtr() const {
-		return (const char*)c_id.getConstV();
+	virtual const unsigned char* getConstKeyPtr() const {
+		return (const unsigned char*)c_id.getConstV();
 	}
 	//  char* getKeyPtr() { return (char*)c_id.getV(); }
 	virtual const int getKeySize() const {
@@ -520,6 +533,8 @@ public:
 	bool operator==(const char* other_key) const {
 		return c_id==*(ConnectionID4 *)other_key;
 	}
+
+    hash_t hash_key;
 private:
 	ConnectionID4 c_id;
 	static std::string pattern_connection4;
@@ -550,11 +565,11 @@ public:
 	virtual hash_t hash() const {
 		return c_id.hash();
 	}
-	hash_t getInt() const {
+	const unsigned char* getInt() const {
 		return 0;
 	}
-	virtual const char* getConstKeyPtr() const {
-		return (const char*)c_id.getConstV();
+	virtual const unsigned char* getConstKeyPtr() const {
+		return (const unsigned char*)c_id.getConstV();
 	}
 	virtual const int getKeySize() const {
 		return sizeof(*c_id.getConstV());
@@ -602,6 +617,8 @@ public:
 		//printf("ConnectionIF::operator==(const char* other_key)\n");
 		return c_id==*(ConnectionID3 *)other_key;
 	}
+
+    hash_t hash_key;
 private:
 	ConnectionID3 c_id;
 	static std::string pattern_connection3;
@@ -630,13 +647,13 @@ public:
 	virtual hash_t hash() const {
 		return c_id.hash();
 	}
-	hash_t getInt() const {
+	const unsigned char* getInt() const {
 		return 0;
 	}
 	static IndexField* parseQuery(const char *query);
 
-	virtual const char* getConstKeyPtr() const {
-		return (const char*)c_id.getConstV();
+	virtual const unsigned char* getConstKeyPtr() const {
+		return (const unsigned char*)c_id.getConstV();
 	}
 	virtual const int getKeySize() const {
 		return sizeof(*c_id.getConstV());
@@ -680,6 +697,8 @@ public:
 	bool operator==(const char* other_key) const {
 		return c_id==*(ConnectionID2 *)other_key;
 	}
+
+    hash_t hash_key;
 private:
 	ConnectionID2 c_id;
 	static std::string pattern_connection2;
diff --git a/src/IndexHash.cc b/src/IndexHash.cc
index 32c8b63..eaefca4 100644
--- a/src/IndexHash.cc
+++ b/src/IndexHash.cc
@@ -11,24 +11,31 @@ IndexHash::IndexHash(size_t size) {
 }
 
 IndexHash::~IndexHash() {
-	clear();
+	clear(); // removing this clear feels dangerous, not sure why it is failing
 	delete []htable;
 }
 
 int IndexHash::clear() {
-	IndexEntry *col_cur, *col_next;
+	IndexEntry *col_cur, *col_nextt;
 	unsigned count = 0;
 	
 	troot = tnext = tprev = tcur = NULL;
 	for (unsigned i=0; i<numBuckets; i++) {
-		col_next = htable[i];
-		while (col_next) {
-			col_cur = col_next;
-			col_next = col_cur->col_next;
+		col_nextt = htable[i];
+		while (col_nextt) {
+            tmlog(TM_LOG_NOTE, "idxhash: clear()", "entering the while loop for deletion for bucket number %d", i);
+			col_cur = col_nextt;
+            IndexField *key_curr;
+            key_curr = col_nextt->getKey();
+            if (key_curr != NULL)
+                //tmlog(TM_LOG_NOTE, "idxhash: clear()", "the entry to delete has key with form %s", key_curr->getIndexName().c_str());
+			col_nextt = col_cur->col_next;
+            tmlog(TM_LOG_NOTE, "idxhash: clear()", "about to delete a collision list entry");
 			delete col_cur;
 			count++;
 		}
 		htable[i]=NULL;
+        //tmlog(TM_LOG_NOTE, "idxhash: clear()", "we make the bucket number %d NULL", i);
 	}
 	assert(count == numEntries);
 	numEntries = 0;
@@ -58,7 +65,6 @@ IndexEntry* IndexHash::lookup( IndexField* key) {
 	curalt = troot;
 	cmp = 0;
 #ifdef TM_HEAVY_DEBUG
-	assert(!lookup(key));
 	if (troot)
 		assert(troot->parent == NULL);
 #endif
@@ -183,6 +189,10 @@ void IndexHash::add(IndexField *key, IndexEntry *ie) {
 	ie->col_prev = NULL;
 	htable[hval] = ie;
 
+    ie->key->hash_key = hval;
+
+    tmlog(TM_LOG_NOTE, "idxhash", "this entry for which we have foudn the bucket value %d for has this timestamp %f and form %s", hval, key->ts, key->getStr().c_str());
+
     tmlog(TM_LOG_NOTE, "idx_hash", "setting an entry in the hash table at %d", hval);
 	
 	if (ie->col_next != NULL) 
@@ -329,6 +339,8 @@ void IndexHash::eraseEntry(IndexEntry *ie) {
 		 */
 		htable[ie->key->hash()%numBuckets] = ie->col_next;
 	}
+    tmlog(TM_LOG_NOTE, "IndexHash: eraseEntry", "we are trying to delete the entry at bucket number %d", ie->key->hash_key);
+    htable[ie->key->hash_key] = NULL;
 	delete ie;
 	numEntries--;
 }


From neto at caltech.edu  Fri Jul 18 23:37:32 2014
From: neto at caltech.edu (Naoki Eto)
Date: Fri, 18 Jul 2014 23:37:32 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: Some querying is working (with
	some adjust for VLAN tags) However,
	it does not take care of MPLS labels Also,
	the check for hash index conflicts has not been done The check that
	full sessions are being taken into account has not been done
	either (e047c0b)
Message-ID: <201407190637.s6J6bWx7010510@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit e047c0b1053f5e410bff1ca6d21742d69e800647
Author: NaokiEto <neto at caltech.edu>
Date:   Sat Jul 19 02:22:10 2014 -0400

    Some querying is working (with some adjust for VLAN tags)
    However, it does not take care of MPLS labels
    Also, the check for hash index conflicts has not been done
    The check that full sessions are being taken into account has not been done either


>---------------------------------------------------------------

e047c0b1053f5e410bff1ca6d21742d69e800647
 CHANGES           |   5 ++
 README            | 109 ++++++-------------------------------------
 src/Connection.cc |   6 +++
 src/Connection.hh |   3 +-
 src/DiskIndex.cc  |   2 +
 src/Fifo.cc       |  33 ++++++++++++-
 src/FifoDisk.cc   |  52 ++++++++++++++++++++-
 src/FifoMem.cc    |  40 ++++++++++++++++
 src/Index.cc      |   1 +
 src/Index.hh      |   6 ++-
 src/IndexField.cc |   8 ++--
 src/IndexHash.cc  |   4 ++
 src/Query.cc      |   7 ++-
 src/Storage.cc    | 136 ++++++++++++++++++++++++++++++++++++++++++++++++++++--
 src/Storage.hh    |  11 +++++
 15 files changed, 315 insertions(+), 108 deletions(-)

diff --git a/CHANGES b/CHANGES
index 459d484..11d9011 100644
--- a/CHANGES
+++ b/CHANGES
@@ -1,3 +1,8 @@
+0.1-4 | 2014-07-18 16:53:50 -0800
+    
+    * Some querying for IPv6 addresses is enabled. (Naoki Eto)
+
+    * VLAN tags are taken into account w/o MPLS labels (Naoki Eto)
 
 0.1-4 | 2013-02-07 14:37:50 -0800
 
diff --git a/README b/README
index 0eac795..29779af 100644
--- a/README
+++ b/README
@@ -10,107 +10,26 @@ information, please see the Time Machines's home page:
 
 	http://www.bro.org/community/time-machine.html
 
-One example way to run is:
-
-sudo timemachine -c etc/timemachine.cfg.in if you are in the time-machine directory
-
-sudo timemachine -r <fullpathnametopcapfile> -c </home/lakers/.ssh/time-machine/etc/timemachine.cfg.in>
-
-To exit, press Ctrl + D
-
-to do the read file, do the full path (pwd)
-
-tcpdump -r class_all_1403831630.092187
-
-tm-query is a python file
-
-use python tm-query to run
+To compile and link, we can do the following:
+sudo ./configure --enable-debug --prefix=<path_to_binary_file_and_config_file>
+sudo make -j 8 && sudo make install
+(the -j 8 lets it make with 8 processors)
 
+To run, do the following:
+sudo ./timemachine -r <path_to_pcap_file> -c <path_to_config_file>
 
+One example way to compile and run is:
 sudo ./configure --enable-debug --prefix=/home/lakers/tm
+sudo make -j 8 && sudo make install
+sudo ./timemachine -r <fullpathnametopcapfile> -c </home/lakers/tm/etc/timemachine.cfg.in>
 
-sudo make -j 8
-
-sudo make install
-
-
-sudo timemachine -r "/home/lakers/tenpack.pcap" -c "/home/lakers/tm/etc/timemachine.cfg"
-
-
-remote console is for components like network intrusion systems like Bro to run
-real-time queries using Time Machine
-
-I guess regular users can also use remote console, but in that case, I'm not entirely
-sure what the benefit is. Is it that multiple users can use time machine at once?
-
-
-sudo ./timemachine -r /home/lakers/randompackets.pcap -c /home/lakers/tm/etc/timemachine.cfg
-
-4 types of connections: connection4, connection3, connection2, and ip
-
-
-race condition? look at vthree log
-
-bpf_filter? O.o returns -1 when fails?
-
-
-he has some funny comments XD
-
-i learned about avl tree and pmtud path maximum transmission unit discovery method
-
-checking for cut-off is done in Fifo:addPkt, which is called in Storage.cc addPkt method
-
-setCutOff in parser
-
-
-IndexField.cc, string IPAddress is one place where IPv6 would need to be implemented
-
-also the string pattern_ipv6
-
-and string pattern ipport (near top of IndexField.cc)
-
-
-qlen is queue length?
-
-input_q is input queue?
-
-potential race conditions at addEntry?
-
-packet is determined if it matches with a class or not
-
-creates a FIFO list with classes that match a packet
-
-this fifo list contains one class. For how many classes there are in the config file, there are that many Fifo's
-
-Each Fifo contains the packets that match with it
-
-these packets are passed to input queue and are transformed into Index fields
-
-ONce the input queue has more than 10, it signals to index thread
-
-if input queue last packet is older than the oldest packet in the Fifo list and the length of the input queue
-is less than the number of entries in the hash table,
-
-then add the last packet to the hash table
-
-then eventually write the hash table term into the disk.
-
-start at definition of capture thread in storage.cc
-
-there are 5+ known data structures into play
-
-FIFO memory ring buffer
-
-input queue of IndexField's
-
-pointers for a sorted list/queue that determine the order of the timeouts of each entry in the hash table presumably
-
-the infamous hash table with IndexEntry that doubles as an AVL tree
+To exit, press Ctrl + D
 
-the writing to files on disk (not really a data structure, but meh, sort of counts)
+To do querying for ipv6 addresses, please use brackets around the ipv6 address.
+So, for example,
+query to_file "testquery.pcap" index ip "[2001:558:fe0d:1c::2]"
 
+Not using brackets around an ipv6 address will result in a querying error.
 
-seems liek the difference between held_bytes and tot_bytes (and simialr with packets) is that held_bytes only counts the bytes that are currently in the
-buffer while tot_bytes counts all the bytes, including the ones that were popped from the buffer
 
 
diff --git a/src/Connection.cc b/src/Connection.cc
index b6a4f31..b021a8d 100644
--- a/src/Connection.cc
+++ b/src/Connection.cc
@@ -806,6 +806,12 @@ hash_t ConnectionID4::hash() const
 
     HashKey* newHashKey = new HashKey(&key, sizeof(key));
 
+    tmlog(TM_LOG_NOTE, "ConnectionID4::hash()", "the hash of newhashkey is %u", newHashKey->Hash());
+
+    HashKey* newHashKeyDos = new HashKey(&key, sizeof(key));
+
+    tmlog(TM_LOG_NOTE, "ConnectionID4::hash()", "the second hash of newhashkeydos is %u", newHashKeyDos->Hash());
+
     //memcpy(&hash_key, newHashKey->Hash(), 8);
 
     //hash_key = newHashKey->Hash();
diff --git a/src/Connection.hh b/src/Connection.hh
index 70b8151..3bb3e9d 100644
--- a/src/Connection.hh
+++ b/src/Connection.hh
@@ -496,7 +496,8 @@ public:
 	}
 	Connection(Connection *c);
 	virtual ~Connection() {
-		delete c_id;
+    // we need delete [] instead of delete because we allocated an array for the ipv6 addresses?
+		delete [] c_id;
 	}
 	void addPkt(const struct pcap_pkthdr* header, const u_char* packet);
 	tm_time_t getLastTs() {
diff --git a/src/DiskIndex.cc b/src/DiskIndex.cc
index addb837..f4365e2 100644
--- a/src/DiskIndex.cc
+++ b/src/DiskIndex.cc
@@ -211,6 +211,7 @@ template <class T>
 void IndexFiles<T>::lookup(IntervalSet *iset, IndexField *key, tm_time_t t0, tm_time_t t1) {
 	int level;
 	uint64_t curfile;
+    // class IndexFileReader from DiskIndex.hh
 	IndexFileReader *ifr;
 	char *fname;
 
@@ -226,6 +227,7 @@ void IndexFiles<T>::lookup(IntervalSet *iset, IndexField *key, tm_time_t t0, tm_
 				// the intervals [t0,t1] and [first,last] intersect 
 				// ==> look for matches
 				ifr->lookupEntry(iset, key);
+
 			}
 			delete ifr;
 		}
diff --git a/src/Fifo.cc b/src/Fifo.cc
index 85f525a..7bf3468 100644
--- a/src/Fifo.cc
+++ b/src/Fifo.cc
@@ -10,6 +10,7 @@
 #include "Connection.hh"
 #include "FifoDisk.hh"
 #include "tm.h"
+#include "bro_inet_ntop.h"
 
 Fifo::Fifo() {
 	init();
@@ -96,7 +97,7 @@ void Fifo::start() {
 
 bool Fifo::matchPkt(const struct pcap_pkthdr* header,
 					const unsigned char* packet) {
-    tmlog(TM_LOG_DEBUG, "Fifo: matchPkt", "The value of this bpffilter for packet %lu is %lu", header->ts.tv_usec, bpf_filter(fp.bf_insns, (unsigned char*)packet, header->len, header->caplen));
+    tmlog(TM_LOG_DEBUG, "Fifo: matchPkt", "The value of this bpffilter for packet %lu and %lu is %lu", header->ts.tv_sec, header->ts.tv_usec, bpf_filter(fp.bf_insns, (unsigned char*)packet, header->len, header->caplen));
 
 	return bpf_filter(fp.bf_insns, (unsigned char*)packet,
 					  header->len,
@@ -113,6 +114,21 @@ uint64_t Fifo::pktEviction() {
 	for (i=0; i<pkts_to_disk && fm->getHeldPkts()>0; i++) {
         // adding packet to Fifo disk, addPkt method is from FifoDisk.cc
         // getS() method is from FifoMem.c - it gets the beginning of first valid packet in the memory ring buffer block
+
+
+        char str1[INET6_ADDRSTRLEN];
+
+        bro_inet_ntop(AF_INET6, &(IP6(fm->getS() + 4 + sizeof(struct pcap_pkthdr))->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
+
+        //char s1[INET6_ADDRSTRLEN];
+
+        //inet_pton(AF_INET6, s1, str1);
+
+        char str2[INET6_ADDRSTRLEN];
+
+        bro_inet_ntop(AF_INET6, &(IP6(fm->getS() + 4 + sizeof(struct pcap_pkthdr))->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
+
+        tmlog(TM_LOG_NOTE, "Fifo::pktEviction", "we ard adding the packet to the FifoDisk with src ip %s and dst ip %s", str1, str2);
 		fd->addPkt(fm->getS());
         // pop the packet from the memory ring buffer (I'm still not sure about the align stuff, with a_s, a_lp)
         // note that popPkt returns the size of the popped packet in bytes
@@ -143,6 +159,21 @@ bool Fifo::addPkt(const struct pcap_pkthdr* header,
 			cutoff_bytes+=header->len;
 			return false;
 		} else {
+
+            char str1[INET6_ADDRSTRLEN];
+
+            bro_inet_ntop(AF_INET6, &(IP6(packet + 4)->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
+
+            //char s1[INET6_ADDRSTRLEN];
+
+            //inet_pton(AF_INET6, s1, str1);
+
+            char str2[INET6_ADDRSTRLEN];
+
+            bro_inet_ntop(AF_INET6, &(IP6(packet + 4)->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
+
+            tmlog(TM_LOG_NOTE, "Fifo::addPkt", "the packet we are adding to FifoMem has source ip %s and destination ip %s", str1, str2);
+
 			// normal addition to Memory Fifo since this packet is not cutoff
 			fm->addPkt(header, packet);
             tmlog(TM_LOG_DEBUG, "addPkt: Fifo.cc, ~line 121", "Connection cut-off did not occur for packet %lu", header->ts.tv_usec);
diff --git a/src/FifoDisk.cc b/src/FifoDisk.cc
index 910654e..917cfcf 100644
--- a/src/FifoDisk.cc
+++ b/src/FifoDisk.cc
@@ -10,6 +10,7 @@
 #include "tm.h"
 #include "FifoDisk.hh"
 #include "Query.hh"
+#include "bro_inet_ntop.h"
 
 
 /***************************************************************************
@@ -121,11 +122,28 @@ void FifoDisk::addPkt(const pkt_ptr p) {
 						 classname.c_str(), newestTimestamp);
                 // push back the newest disk file into the list of files
 				files.push_back(new FifoDiskFile(new_file_name, pcap_handle));
+
+                tmlog(TM_LOG_NOTE, "FifoDisk: addPkt", "the new file name is: %s", new_file_name);
                 // free new_file_name since we malloced it and don't need it anymore
 				free(new_file_name);
 			}
 			unlockQueryInProgress();
 		}
+
+        char str1[INET6_ADDRSTRLEN];
+
+        bro_inet_ntop(AF_INET6, &(IP6(p + 4 + sizeof(struct pcap_pkthdr))->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
+
+        //char s1[INET6_ADDRSTRLEN];
+
+        //inet_pton(AF_INET6, s1, str1);
+
+        char str2[INET6_ADDRSTRLEN];
+
+        bro_inet_ntop(AF_INET6, &(IP6(p + 4 + sizeof(struct pcap_pkthdr))->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
+
+        tmlog(TM_LOG_NOTE, "FifoDisk::addPkt", "we are going to add in next step the packet to the FifoDisk with src ip %s and dst ip %s", str1, str2);
+
         // in the last file, the newest file, add the packet
 		files.back()->addPkt(p);
         // get the oldest time stamp (the 1 milisecond if statement appears again)
@@ -242,6 +260,8 @@ uint64_t FifoDiskFile::query( QueryRequest *qreq, QueryResult *qres, IntervalSet
 	if (is_open)
 		flush();
 
+	//char errbuf[PCAP_ERRBUF_SIZE];
+
 	pcapnav_t *ph=pcapnav_open_offline(filename.c_str());
 	if (!ph) {
 		char *pcap_errstr = pcapnav_geterr(ph);
@@ -251,11 +271,13 @@ uint64_t FifoDiskFile::query( QueryRequest *qreq, QueryResult *qres, IntervalSet
 		struct pcap_pkthdr hdr;
 		const u_char *pkt;
 
+        
 		if (pcapnav_get_timespan(ph, &tv1, &tv2) != 0) {
 			tmlog(TM_LOG_WARN, "query",  "%d pcapnav could not obtain timespan.",
 					qres->getQueryID());
-			  /* Rest of error handling */
+			  //Rest of error handling
 		}
+        
 		tmlog(TM_LOG_DEBUG, "query", "%d FifoDiskFile::query: opened file %s. timespan is [%lf,%lf]",
 				qres->getQueryID(), filename.c_str(), to_tm_time(&tv1), to_tm_time(&tv2));
 
@@ -286,6 +308,7 @@ uint64_t FifoDiskFile::query( QueryRequest *qreq, QueryResult *qres, IntervalSet
 				tmlog(TM_LOG_DEBUG, "query", "%d Interval overlapped trace start. Goto 0",
 						qres->getQueryID());
 			}
+            
 			else 
 				res = pcapnav_goto_timestamp(ph, &tv);
 			switch(res) {
@@ -304,12 +327,14 @@ uint64_t FifoDiskFile::query( QueryRequest *qreq, QueryResult *qres, IntervalSet
 				default:
 					break;
 			}
+            
 			if (res != PCAPNAV_DEFINITELY) {
 				continue;
 			}
+            
 			first_pkt_for_this_int = 1;
 			do {
-				pkt = pcapnav_next(ph, &hdr);
+				pkt = pcapnav_next(ph, &hdr);// + 4;
 				scanned_packets++;
 				if (!pkt)
 					break;
@@ -324,6 +349,29 @@ uint64_t FifoDiskFile::query( QueryRequest *qreq, QueryResult *qres, IntervalSet
 					break;
 				if (t<qreq->getT0())
 					continue;
+                //tmlog("The result of matchPkt from QueryRequest is %d 
+                char str1[INET6_ADDRSTRLEN];
+
+                bro_inet_ntop(AF_INET6, &(IP6(pkt)->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
+
+                //char s1[INET6_ADDRSTRLEN];
+
+                //inet_pton(AF_INET6, s1, str1);
+
+                char str2[INET6_ADDRSTRLEN];
+
+                bro_inet_ntop(AF_INET6, &(IP6(pkt)->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
+                //char s2[INET6_ADDRSTRLEN];
+                /*
+                if ( inet_pton(AF_INET6, s2, str2) <=0 )
+			        {
+                    tmlog(TM_LOG_ERROR, "Bad IP address: %s", s2);
+                    }
+                */
+                tmlog(TM_LOG_NOTE, "FifoDisk.cc: query", "the query packet has source ip address: %s and dst ip address %s and header time stamp %lu and %lu", \
+                str1, str2, hdr.ts.tv_sec, hdr.ts.tv_usec);
+                tmlog(TM_LOG_NOTE, "FifoDisk.cc:query", "the query parameters are that it has a time interval from %f to %f, a hash of %lu, a timestamp of %f, and a form of %s", \
+                qreq->getT0(), qreq->getT1(), qreq->getField()->hash(), qreq->getField()->ts, qreq->getField()->getStr().c_str());
 				if (qreq->matchPkt(&hdr, pkt))  {
 					matches++;
 					qres->sendPkt(&hdr, pkt);
diff --git a/src/FifoMem.cc b/src/FifoMem.cc
index 4d1debd..f82b19b 100644
--- a/src/FifoMem.cc
+++ b/src/FifoMem.cc
@@ -9,6 +9,7 @@
 #include "FifoMem.hh"
 #include "Query.hh"
 #include "Index.hh"
+#include "bro_inet_ntop.h"
 
 
 FifoMem::FifoMem(uint64_t size): size(size), oldestTimestamp(0), newestTimestamp(0) {
@@ -93,6 +94,26 @@ void FifoMem::addPkt(const struct pcap_pkthdr *header,
 		memcpy(wp, header, sizeof(struct pcap_pkthdr));
 		memcpy(wp+sizeof(struct pcap_pkthdr), packet, header->caplen);
 
+
+
+
+        char str1[INET6_ADDRSTRLEN];
+
+        bro_inet_ntop(AF_INET6, &(IP6(wp + 4 + sizeof(struct pcap_pkthdr))->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
+
+        //char s1[INET6_ADDRSTRLEN];
+
+        //inet_pton(AF_INET6, s1, str1);
+
+        char str2[INET6_ADDRSTRLEN];
+
+        bro_inet_ntop(AF_INET6, &(IP6(wp + 4 + sizeof(struct pcap_pkthdr))->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
+
+        tmlog(TM_LOG_NOTE, "FifoMem::addPkt", "we just wrote to the fifo memory ring buffer the packet with src ip %s and dst ip %s", str1, str2);
+
+
+
+
 		/* Adjust align array and a_lp 
 		 * wp now points to the packet that we just wrote */
         // if position of most recently added packet > current writing position
@@ -363,6 +384,7 @@ uint64_t FifoMem::query(QueryRequest *qreq, QueryResult *qres,
 					qres->getQueryID(), i->getStart());
 
 		if (found) {
+            //p += 4;
 			p_orig = p;
 			tmlog(TM_LOG_DEBUG, "query", "%d First packet after bin-search is: ts=%lf, addr=%p, offset=%zu len=%u",
 					qres->getQueryID(), pkt_t(p), p, p-start, ((struct pcap_pkthdr *)p)->caplen);
@@ -391,6 +413,24 @@ uint64_t FifoMem::query(QueryRequest *qreq, QueryResult *qres,
 #endif
 			while ( (p_will_wrap || (p<=lp)) 
 					&& pkt_t(p) <= i->getLast() ) {
+                char str1[INET6_ADDRSTRLEN];
+
+                bro_inet_ntop(AF_INET6, &(IP6(p)->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
+
+                char s1[INET6_ADDRSTRLEN];
+
+                inet_pton(AF_INET6, s1, str1);
+
+                char str2[INET6_ADDRSTRLEN];
+
+                bro_inet_ntop(AF_INET6, &(IP6(p)->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
+                char s2[INET6_ADDRSTRLEN];
+
+                inet_pton(AF_INET6, s2, str2);
+
+                tmlog(TM_LOG_NOTE, "FifoMem.cc: query", "the query packet has source ip address: %s and dst ip address %s", str1, str2);
+                tmlog(TM_LOG_NOTE, "FifoMem.cc:query", "the query parameters in mem are that it has a time interval from %f to %f, a hash of %lu, a timestamp of %f, and a form of %s", \
+                qreq->getT0(), qreq->getT1(), qreq->getField()->hash(), qreq->getField()->ts, qreq->getField()->getStr().c_str());
 				if (qreq->matchPkt(p) && last_match_ts < pkt_t(p))  {
 					qres->sendPkt(p);
 					if (qreq->isSubscribe()) {
diff --git a/src/Index.cc b/src/Index.cc
index 204537f..eb3e2b0 100644
--- a/src/Index.cc
+++ b/src/Index.cc
@@ -362,6 +362,7 @@ void Index<T>::lookupMem(IntervalSet *set, IndexField* key) {
 	unlock_hash();
 }
 
+// disk_index is of types IndexFiles, from DiskIndex.cc
 template <class T>
 void Index<T>::lookupDisk(IntervalSet* set, IndexField* key, tm_time_t t0, tm_time_t t1) {
 	if (disk_index)
diff --git a/src/Index.hh b/src/Index.hh
index 63f2bd9..9211820 100644
--- a/src/Index.hh
+++ b/src/Index.hh
@@ -327,7 +327,11 @@ public:
 		for (std::list<IndexType*>::iterator i=indexes.begin();
 				i!=indexes.end();
 				i++)
-			if ((*i)->getIndexName()==name) return (*i);
+			if ((*i)->getIndexName()==name) 
+            {
+                tmlog(TM_LOG_NOTE, "Indexes: getIndexByName", "we have a match with the index name %s", name.c_str());
+                return (*i);
+            }
 		return NULL;
 	};
 	int trylockDiskWrite() {
diff --git a/src/IndexField.cc b/src/IndexField.cc
index c4ae442..380577c 100644
--- a/src/IndexField.cc
+++ b/src/IndexField.cc
@@ -90,10 +90,11 @@ IndexField* IPAddress::parseQuery(const char *query) {
     //char strIP[AFtypelength];
 
 
-    // DAHHHHH and so it begins i guess
+    // WHAT and so it begins i guess
     // INET6_ADDRSTRLEN is 46 and INET_ADDRSTRLEN is 16
     //char strIP46[INET6_ADDRSTRLEN];
 
+    tmlog(TM_LOG_NOTE, "parseQuery", "the argument we pass to IPAddress is %s", ip.c_str());
 
 	return new IPAddress(ip.c_str());//, strIP46);
 }
@@ -199,6 +200,7 @@ void IPAddress::Init(const std::string& s)
 			memset(ipv6_address.s6_addr, 0, sizeof(ipv6_address.s6_addr));
 			}
 		}
+        tmlog(TM_LOG_NOTE, "IPAddress::Init", "good IP Address %s", s.c_str());
 	}
 
 std::string IPAddress::getStr() const
@@ -330,7 +332,7 @@ std::string SrcIPAddress::getStrPkt(const u_char* packet) const
 			return "<bad IPv6 address conversion";
 		else
         {
-            ss << " ip for IPv6"
+            ss << "source stuff ip for IPv6"
 
             << "[" << str << "]";
 			return ss.str();
@@ -383,7 +385,7 @@ std::string DstIPAddress::getStrPkt(const u_char* packet) const
 			return "<bad IPv6 address conversion";
 		else
         {
-            ss << " ip for IPv6"
+            ss << "dst stuff ip for IPv6"
 
             << "[" << str << "]";
 			return ss.str();
diff --git a/src/IndexHash.cc b/src/IndexHash.cc
index eaefca4..6358eb1 100644
--- a/src/IndexHash.cc
+++ b/src/IndexHash.cc
@@ -114,6 +114,10 @@ IndexEntry* IndexHash::lookup( IndexField* key) {
 }
 
 void IndexHash::add(IndexField *key, IndexEntry *ie) {
+
+    tmlog(TM_LOG_NOTE, "idx_hash", "entering the add method");
+
+
 	IndexEntry *cur, *prev;
 	int cmp;
 	cur = troot;
diff --git a/src/Query.cc b/src/Query.cc
index 44b87fb..d5dcc77 100644
--- a/src/Query.cc
+++ b/src/Query.cc
@@ -27,6 +27,7 @@ QueryRequest::~QueryRequest() {
 	pcap_close(ph);
 }
 
+// note that typedef of pkt_ptr is u char*
 bool QueryRequest::matchPkt(const pkt_ptr p) {
 	/*
 	if (!have_bpf) compileBPF();
@@ -34,12 +35,16 @@ bool QueryRequest::matchPkt(const pkt_ptr p) {
 	    ((struct pcap_pkthdr *)p)->len,
 	    ((struct pcap_pkthdr *)p)->caplen);
 	*/
+	tmlog(TM_LOG_NOTE, "QueryRequest: matchPkt(const pkt_ptr)", "determine if a packet matches");
+	// matchPkt method from Fifo
 	return matchPkt((struct pcap_pkthdr *)p, p+sizeof(struct pcap_pkthdr));
 }
 
 bool QueryRequest::matchPkt(struct pcap_pkthdr *hdr, const u_char *pkt) {
+	tmlog(TM_LOG_NOTE, "QueryRequest: matchPkt(struct pcap_pkthdr, u_char pkt)", "determine if a packet matches");
 	if (!have_bpf) compileBPF();
-	return bpf_filter(fp.bf_insns, (u_char *)pkt,
+    tmlog(TM_LOG_NOTE, "QueryRequest: matchPkt(2 args)", "The value of the bpf_filter in query request  for packet %lu and %lu is %lu", hdr->ts.tv_sec, hdr->ts.tv_usec, bpf_filter(fp.bf_insns, (unsigned char*)(pkt), hdr->len, hdr->caplen));
+	return bpf_filter(fp.bf_insns, (u_char *)(pkt),
 					  hdr->len,
 					  hdr->caplen);
 }
diff --git a/src/Storage.cc b/src/Storage.cc
index 2683579..01a13dc 100644
--- a/src/Storage.cc
+++ b/src/Storage.cc
@@ -14,6 +14,7 @@
 #include "packet_headers.h"
 #include "conf.h"
 #include "tm.h"
+#include "bro_inet_ntop.h"
 
 #define SNAPLEN 8192
 
@@ -40,6 +41,20 @@ void callback(u_char *args, const struct pcap_pkthdr *header,
 	network_time=header->ts;
 	*/ 
 
+    char str1[INET6_ADDRSTRLEN];
+
+    inet_ntop(AF_INET6, &(IP6(packet)->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
+
+    //char s1[INET6_ADDRSTRLEN];
+
+    //inet_pton(AF_INET6, s1, str1);
+
+    char str2[INET6_ADDRSTRLEN];
+
+    inet_ntop(AF_INET6, &(IP6(packet)->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
+
+    tmlog(TM_LOG_NOTE, "Storage::callback", "we are going to call addpacket of storage on packet that has source ip %s and destination ip %s", str1, str2);
+
     // DEBUG DEBUG DEBUG
 	tmlog(TM_LOG_DEBUG, "storage callback: Storage.cc, ~line 29", "Callback function for pcap_loop%lu", header->ts.tv_usec);
 
@@ -264,6 +279,60 @@ void Storage::cancelThread() {
 	fifos.push_back(f);
 }*/
 
+int get_link_header_size(int dl)
+	{
+	switch ( dl ) {
+	case DLT_NULL:
+		return 4;
+
+	case DLT_EN10MB:
+		return 14;
+
+	case DLT_FDDI:
+		return 13 + 8;	// fddi_header + LLC
+
+#ifdef DLT_LINUX_SLL
+	case DLT_LINUX_SLL:
+		return 16;
+#endif
+
+	case DLT_PPP_SERIAL:	// PPP_SERIAL
+		return 4;
+
+	case DLT_RAW:
+		return 0;
+	}
+
+	return -1;
+	}
+
+void Storage::Close()
+	{
+	if ( ph )
+		{
+		pcap_close(ph);
+		ph = 0;
+		//closed = true;
+		}
+	}
+
+
+void Storage::SetHdrSize()
+	{
+	int dl = pcap_datalink(ph);
+	hdr_size = get_link_header_size(dl);
+
+	if ( hdr_size < 0 )
+		{
+		safe_snprintf(errbuf, sizeof(errbuf),
+			 "unknown data link type 0x%x", dl);
+		Close();
+		}
+
+	datalink = dl;
+	}
+
+
 
 // note that the pcap packet header has the timestamp of the packet
 void Storage::addPkt(const struct pcap_pkthdr *header,
@@ -284,11 +353,55 @@ void Storage::addPkt(const struct pcap_pkthdr *header,
     // DEBUG DEBUG DEBUG
 	tmlog(TM_LOG_NOTE, "addPkt: Storage.cc, ~line 240", "adding packet %lu !", header->ts.tv_usec);
 
+    SetHdrSize();
+
+	// Unfortunately some packets on the link might have MPLS labels
+	// while others don't. That means we need to ask the link-layer if
+	// labels are in place. TODO: Cannot handle MPLS labels just yet
+	//bool have_mpls = false;
+
 	const unsigned char* idxpacket=packet;
 	// skip VLAN header (related to ethernet frame header) for indexing TODO: look at VLAN header more closely
     // Virtual Bridged Local Area Network for logically group network devices together, which share the same 
     // physical network. VLAN tag is 4 bytes and so a VLAN header is 4 bytes longer than a regular ethernet header
-	if (ether_type==0x8100) idxpacket+=4;
+	//if (ether_type==0x8100) idxpacket+=4;
+    
+    if (ether_type == 0x8100)
+    {
+		// Check for MPLS in VLAN.
+        
+        // TODO: Cannot handle MPLS labels just yet
+        /*
+		if ( ((idxpacket[2] << 8) + idxpacket[3]) == 0x8847 )
+			have_mpls = true;
+        */
+
+		idxpacket += 4; // Skip the vlan header
+		//pkt_hdr_size = 0;
+
+		// Check for 802.1ah (Q-in-Q) containing IP.
+		// Only do a second layer of vlan tag
+		// stripping because there is no
+		// specification that allows for deeper
+		// nesting.
+		if ( ((idxpacket[2] << 8) + idxpacket[3]) == 0x0800 )
+			idxpacket += 4;
+    }
+
+    // TODO: Cannot handle MPLS labels just yet
+    /*
+	if ( have_mpls )
+		{
+		// Skip the MPLS label stack.
+		bool end_of_stack = false;
+
+		while ( ! end_of_stack )
+			{
+			end_of_stack = *(idxpacket + 2) & 0x01;
+			idxpacket += 4;
+			}
+		}
+    */
     // DEBUG DEBUG DEBUG
     tmlog(TM_LOG_NOTE, "addPkt: Storage.cc, ~line 246", "ethernet phase/physical layer complete for packet %lu", header->ts.tv_usec);
 
@@ -381,7 +494,7 @@ void Storage::addPkt(const struct pcap_pkthdr *header,
         // go through all the possible classes
 		for (std::list<Fifo*>::iterator i=fifos.begin(); i!=fifos.end(); i++) {
     
-            tmlog(TM_LOG_DEBUG, "addPkt: Storage.cc, ~line 310", "value of matchPkt of packet %lu is %lu", header->ts.tv_usec, (*i)->matchPkt(header, packet));
+            tmlog(TM_LOG_DEBUG, "addPkt: Storage.cc, ~line 310", "value of matchPkt of packet %lu is %lu", header->ts.tv_usec, (*i)->matchPkt(header, idxpacket));
 
 			if (// packet matches this class' filter (from Fifo.cc) and
 				(*i)->matchPkt(header, packet) &&
@@ -413,6 +526,20 @@ void Storage::addPkt(const struct pcap_pkthdr *header,
             // 
 			if (TCP(packet)->th_flags & ( TH_FIN | TH_SYN | TH_RST ))
 				tcp_ctrl_flag=true;
+
+            char str1[INET6_ADDRSTRLEN];
+
+            bro_inet_ntop(AF_INET6, &(IP6(idxpacket)->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
+
+            //char s1[INET6_ADDRSTRLEN];
+
+            //inet_pton(AF_INET6, s1, str1);
+
+            char str2[INET6_ADDRSTRLEN];
+
+            bro_inet_ntop(AF_INET6, &(IP6(idxpacket)->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
+
+            tmlog(TM_LOG_NOTE, "Storage::addPkt", "we will be calling Fifo::Addpkt on packet that has source ip %s and destination ip %s", str1, str2);
         /* true if cutoff should not be done for this connection */
         // When the connection is subscribed or when a tcp control flag
         // is set, then addPkt is called with a connection NULL pointer
@@ -427,8 +554,8 @@ void Storage::addPkt(const struct pcap_pkthdr *header,
         // So, it eventually leads to the writing of the class files (woo)
         // Also, note that f->addPkt returns true if we can add the packet, false otherwise
 		if ( (( c->getSuspendCutoff() | tcp_ctrl_flag )
-				&& f->addPkt(header, packet, NULL)) ||
-				f->addPkt(header, packet, c)) {
+				&& f->addPkt(header, idxpacket, NULL)) ||
+				f->addPkt(header, idxpacket, c)) {
 			/* packet was stored in Fifo *f */
             // woo-hoo! bytes from the packet were not cut
 			uncut_bytes += header->len;
@@ -561,6 +688,7 @@ void Storage::query(QueryRequest *query_req, QueryResult *query_res) {
 	gettimeofday(&t_start, NULL);
 	//fprintf(stderr, "Query ID: %d\n",  query_res->getQueryID());
 
+    // getIndexByName is from Index.hh from class Indexes
 	IndexType* idx=indexes->getIndexByName(query_req->getField()->getIndexName());
 	if (!idx) {
 		tmlog(TM_LOG_ERROR, "query", "Tried to query index \"%s\" but it does not exist\n",
diff --git a/src/Storage.hh b/src/Storage.hh
index 1cbf41f..3e136d1 100644
--- a/src/Storage.hh
+++ b/src/Storage.hh
@@ -38,6 +38,8 @@ class StorageConfig {
 		Indexes *indexes;
 };
 
+extern int get_link_header_size(int dl);
+
 class Storage {
 public:
 	Storage(StorageConfig& conf);
@@ -100,6 +102,15 @@ public:
 	}
 	
 	friend void *capture_thread(void *arg);
+
+protected:
+    void Close();
+    void SetHdrSize();
+    int datalink;
+    char errbuf[PCAP_ERRBUF_SIZE];
+    int hdr_size;
+    //const u_char* data;
+
 private:
 	pcap_t *ph;
 	int snaplen;


From neto at caltech.edu  Mon Jul 21 12:37:36 2014
From: neto at caltech.edu (Naoki Eto)
Date: Mon, 21 Jul 2014 12:37:36 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: I seemed to have removed the
	major memory leaks. There are still some memory leaks,
	but they seem to come from the original Time Machine code,
	namely with the connections. (56b3e91)
Message-ID: <201407211937.s6LJba8v006422@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit 56b3e9177e4c881ac973e7d8e72b29a25bab45bb
Author: NaokiEto <neto at caltech.edu>
Date:   Mon Jul 21 15:14:46 2014 -0400

    I seemed to have removed the major memory leaks.
    There are still some memory leaks, but they seem to come
    from the original Time Machine code, namely with the connections.


>---------------------------------------------------------------

56b3e9177e4c881ac973e7d8e72b29a25bab45bb
 src/Connection.cc | 437 ++++++++++++++++++++++++++++++++++++------------------
 src/Connection.hh |   6 +-
 src/Hash.cc       |   4 +
 src/Hash.h        |   2 +
 src/IndexField.cc |  14 ++
 src/IndexField.hh |  53 +++++--
 src/Query.cc      |  68 +++++++++
 src/Storage.cc    |   9 +-
 src/Storage.hh    |   4 +-
 9 files changed, 438 insertions(+), 159 deletions(-)

diff --git a/src/Connection.cc b/src/Connection.cc
index b021a8d..f7c1ae8 100644
--- a/src/Connection.cc
+++ b/src/Connection.cc
@@ -72,6 +72,13 @@ void ConnectionID4::init(proto_t proto,
     ipv4_d_address.s_addr = d_ip;
     ipv4_s_address.s_addr = s_ip;
 
+	struct {
+		in6_addr ip1;
+		in6_addr ip2;
+		uint16 port1;
+		uint16 port2;
+	} key;
+
 	if (addr_port_canon_lt(s_ip,d_ip,s_port,d_port)) {
 		//    v.is_canonified=true;
 
@@ -93,7 +100,21 @@ void ConnectionID4::init(proto_t proto,
 
 		v6.port1=d_port;
 		v6.port2=s_port;
-	} else {
+
+        // this is for the hash key
+        in6_addr s6_ip;
+        in6_addr d6_ip;
+        
+        memcpy(s6_ip.s6_addr, v6.ip1, 16);
+        memcpy(d6_ip.s6_addr, v6.ip2, 16);
+
+        key.ip1 = s6_ip;
+        key.ip2 = d6_ip;
+        key.port1 = v6.port1;
+        key.port2 = v6.port2;
+
+	} 
+    else {
 		//    v.is_canonified=false;
 
         ConnectionID4(ipv4_s_address, ipv4_d_address);
@@ -114,7 +135,39 @@ void ConnectionID4::init(proto_t proto,
         
 		v6.port1=s_port;
 		v6.port2=d_port;
+
+        // this is for the hash key
+        in6_addr s6_ip;
+        in6_addr d6_ip;
+        
+        memcpy(s6_ip.s6_addr, v6.ip1, 16);
+        memcpy(d6_ip.s6_addr, v6.ip2, 16);
+
+	    key.ip1 = d6_ip;
+	    key.ip2 = s6_ip;
+	    key.port1 = v6.port2;
+	    key.port2 = v6.port1;
 	}
+
+    tmlog(TM_LOG_NOTE, "connection4: Connection.cc", "connection 4 with form %s", getStr().c_str());
+
+    init_hash_function();
+
+    HashKey* newHashKey = new HashKey(&key, sizeof(key));
+
+    tmlog(TM_LOG_NOTE, "ConnectionID4::hash()", "the hash of newhashkey is %u", newHashKey->Hash());
+
+    //HashKey* newHashKeyDos = new HashKey(&key, sizeof(key));
+
+    //tmlog(TM_LOG_NOTE, "ConnectionID4::hash()", "the second hash of newhashkeydos is %u", newHashKeyDos->Hash());
+
+    //memcpy(&hash_key, newHashKey->Hash(), 8);
+
+    hash_key = newHashKey->Hash();
+
+    delete newHashKey;
+
+    free_hash_function();
 }
 
 void ConnectionID4::init6(proto_t proto,
@@ -125,6 +178,17 @@ void ConnectionID4::init6(proto_t proto,
 
     v6.version = 6;
 	v6.proto=proto;
+
+	struct {
+		in6_addr ip1;
+		in6_addr ip2;
+		uint16 port1;
+		uint16 port2;
+	} key;
+
+	// Lookup up connection based on canonical ordering, which is
+	// the smaller of <src addr, src port> and <dst addr, dst port>
+	// followed by the other.
 	if (addr6_port_canon_lt(s_ip,d_ip,s_port,d_port)) {
 		//    v6.is_canonified=true;
         // memcpy(destination, source, size)
@@ -134,6 +198,19 @@ void ConnectionID4::init6(proto_t proto,
 		//v6.ip2=s_ip;
 		v6.port1=d_port;
 		v6.port2=s_port;
+
+        // this is for the hash key
+        in6_addr s6_ip;
+        in6_addr d6_ip;
+        
+        memcpy(s6_ip.s6_addr, v6.ip1, 16);
+        memcpy(d6_ip.s6_addr, v6.ip2, 16);
+
+        key.ip1 = s6_ip;
+        key.ip2 = d6_ip;
+        key.port1 = v6.port1;
+        key.port2 = v6.port2;
+
 	} else {
 		//    v6.is_canonified=false;
         memcpy(v6.ip1, s_ip, 16);
@@ -142,8 +219,38 @@ void ConnectionID4::init6(proto_t proto,
 		//v6.ip2=d_ip;
 		v6.port1=s_port;
 		v6.port2=d_port;
+
+        // this is for the hash key
+        in6_addr s6_ip;
+        in6_addr d6_ip;
+        
+        memcpy(s6_ip.s6_addr, v6.ip1, 16);
+        memcpy(d6_ip.s6_addr, v6.ip2, 16);
+
+	    key.ip1 = d6_ip;
+	    key.ip2 = s6_ip;
+	    key.port1 = v6.port2;
+	    key.port2 = v6.port1;
 	}
     tmlog(TM_LOG_NOTE, "connection4: Connection.cc", "connection 4 with form %s", getStr().c_str());
+
+    init_hash_function();
+
+    HashKey* newHashKey = new HashKey(&key, sizeof(key));
+
+    tmlog(TM_LOG_NOTE, "ConnectionID4::hash()", "the hash of newhashkey is %u", newHashKey->Hash());
+
+    //HashKey* newHashKeyDos = new HashKey(&key, sizeof(key));
+
+    //tmlog(TM_LOG_NOTE, "ConnectionID4::hash()", "the second hash of newhashkeydos is %u", newHashKeyDos->Hash());
+
+    //memcpy(&hash_key, newHashKey->Hash(), 8);
+
+    hash_key = newHashKey->Hash();
+
+    delete newHashKey;
+
+    free_hash_function();
 }
 
 void ConnectionID3::init(proto_t proto,
@@ -177,6 +284,55 @@ void ConnectionID3::init(proto_t proto,
     ConnectionID3(ipv4_s_address, ipv4_d_address);
 
 	v6.port2=port2;
+
+	struct {
+		in6_addr ip1;
+		in6_addr ip2;
+		uint16 port1;
+		uint16 port2;
+	} key;
+
+	// Lookup up connection based on canonical ordering, which is
+	// the smaller of <src addr, src port> and <dst addr, dst port>
+	// followed by the other.
+	if (addr6_port_canon_lt(v6.ip1, v6.ip2, 0, v6.port2))
+		{
+            in6_addr s6_ip;
+            in6_addr d6_ip;
+            
+            memcpy(s6_ip.s6_addr, v6.ip1, 16);
+            memcpy(d6_ip.s6_addr, v6.ip2, 16);
+
+		    key.ip1 = s6_ip;
+		    key.ip2 = d6_ip;
+		    key.port1 = 0;
+		    key.port2 = v6.port2;
+		}
+	else
+		{
+            in6_addr s6_ip;
+            in6_addr d6_ip;
+            
+            memcpy(s6_ip.s6_addr, v6.ip1, 16);
+            memcpy(d6_ip.s6_addr, v6.ip2, 16);
+
+		    key.ip1 = d6_ip;
+		    key.ip2 = s6_ip;
+		    key.port1 = v6.port2;
+		    key.port2 = 0;
+		}
+
+    init_hash_function();
+
+    HashKey* newHashKey = new HashKey(&key, sizeof(key));
+
+    //hash_key = newHashKey->Hash();
+
+	hash_key = newHashKey->Hash();
+
+    delete newHashKey;
+
+    free_hash_function();
 }
 
 void ConnectionID3::init6(proto_t proto,
@@ -197,6 +353,55 @@ void ConnectionID3::init6(proto_t proto,
 	//v.ip1=ip1;
 	//v.ip2=ip2;
 	v6.port2=port2;
+
+	struct {
+		in6_addr ip1;
+		in6_addr ip2;
+		uint16 port1;
+		uint16 port2;
+	} key;
+
+	// Lookup up connection based on canonical ordering, which is
+	// the smaller of <src addr, src port> and <dst addr, dst port>
+	// followed by the other.
+	if (addr6_port_canon_lt(v6.ip1, v6.ip2, 0, v6.port2))
+		{
+            in6_addr s6_ip;
+            in6_addr d6_ip;
+            
+            memcpy(s6_ip.s6_addr, v6.ip1, 16);
+            memcpy(d6_ip.s6_addr, v6.ip2, 16);
+
+		    key.ip1 = s6_ip;
+		    key.ip2 = d6_ip;
+		    key.port1 = 0;
+		    key.port2 = v6.port2;
+		}
+	else
+		{
+            in6_addr s6_ip;
+            in6_addr d6_ip;
+            
+            memcpy(s6_ip.s6_addr, v6.ip1, 16);
+            memcpy(d6_ip.s6_addr, v6.ip2, 16);
+
+		    key.ip1 = d6_ip;
+		    key.ip2 = s6_ip;
+		    key.port1 = v6.port2;
+		    key.port2 = 0;
+		}
+
+    init_hash_function();
+
+    HashKey* newHashKey = new HashKey(&key, sizeof(key));
+
+    //hash_key = newHashKey->Hash();
+
+	hash_key = newHashKey->Hash();
+
+    delete newHashKey;
+
+    free_hash_function();
 }
 
 void ConnectionID2::init( uint32_t s_ip, uint32_t d_ip) {
@@ -212,6 +417,13 @@ void ConnectionID2::init( uint32_t s_ip, uint32_t d_ip) {
     ipv4_d_address.s_addr = s_ip;
     ipv4_s_address.s_addr = d_ip;
 
+	struct {
+		in6_addr ip1;
+		in6_addr ip2;
+		uint16 port1;
+		uint16 port2;
+	} key;
+
 	if (addr_port_canon_lt(s_ip,d_ip,0,0)) {
 		//    v.is_canonified=true;
         /*
@@ -224,6 +436,18 @@ void ConnectionID2::init( uint32_t s_ip, uint32_t d_ip) {
 
         ConnectionID2(ipv4_d_address, ipv4_s_address);
 
+        // this is for the hash key
+        in6_addr s6_ip;
+        in6_addr d6_ip;
+        
+        memcpy(s6_ip.s6_addr, v6.ip1, 16);
+        memcpy(d6_ip.s6_addr, v6.ip2, 16);
+
+	    key.ip1 = s6_ip;
+	    key.ip2 = d6_ip;
+	    key.port1 = 0;
+	    key.port2 = 0;
+
 	} else {
 		//    v.is_canonified=false;
         /*
@@ -235,7 +459,31 @@ void ConnectionID2::init( uint32_t s_ip, uint32_t d_ip) {
         */
 
         ConnectionID2(ipv4_s_address, ipv4_d_address);
+
+        // this is for the hash key
+        in6_addr s6_ip;
+        in6_addr d6_ip;
+        
+        memcpy(s6_ip.s6_addr, v6.ip1, 16);
+        memcpy(d6_ip.s6_addr, v6.ip2, 16);
+
+	    key.ip1 = d6_ip;
+	    key.ip2 = s6_ip;
+	    key.port1 = 0;
+	    key.port2 = 0;
 	}
+
+    init_hash_function();
+
+    HashKey* newHashKey = new HashKey(&key, sizeof(key));
+
+    //hash_key = newHashKey->Hash();
+
+	hash_key = newHashKey->Hash();
+
+    delete newHashKey;
+
+    free_hash_function();
 }
 
 void ConnectionID2::init6( unsigned char s_ip[], unsigned char d_ip[]) {
@@ -245,6 +493,13 @@ void ConnectionID2::init6( unsigned char s_ip[], unsigned char d_ip[]) {
 
     v6.version = 6;
 
+	struct {
+		in6_addr ip1;
+		in6_addr ip2;
+		uint16 port1;
+		uint16 port2;
+	} key;
+
 	if (addr6_port_canon_lt(s_ip,d_ip,0,0)) {
 		//    v.is_canonified=true;
         // memcpy(destination, source, size)
@@ -253,13 +508,50 @@ void ConnectionID2::init6( unsigned char s_ip[], unsigned char d_ip[]) {
 
 		//v6.ip1=d_ip;
 		//v6.ip2=s_ip;
+
+        // this is for the hash key
+        in6_addr s6_ip;
+        in6_addr d6_ip;
+        
+        memcpy(s6_ip.s6_addr, v6.ip1, 16);
+        memcpy(d6_ip.s6_addr, v6.ip2, 16);
+
+	    key.ip1 = s6_ip;
+	    key.ip2 = d6_ip;
+	    key.port1 = 0;
+	    key.port2 = 0;
+
 	} else {
 		//    v.is_canonified=false;
         memcpy(v6.ip1, s_ip, 16);
         memcpy(v6.ip2, d_ip, 16);
 		//v.ip1=s_ip;
 		//v.ip2=d_ip;
+
+        // this is for the hash key
+        in6_addr s6_ip;
+        in6_addr d6_ip;
+        
+        memcpy(s6_ip.s6_addr, v6.ip1, 16);
+        memcpy(d6_ip.s6_addr, v6.ip2, 16);
+
+	    key.ip1 = d6_ip;
+	    key.ip2 = s6_ip;
+	    key.port1 = 0;
+	    key.port2 = 0;
 	}
+
+    init_hash_function();
+
+    HashKey* newHashKey = new HashKey(&key, sizeof(key));
+
+    //hash_key = newHashKey->Hash();
+
+	hash_key = newHashKey->Hash();
+
+    delete newHashKey;
+
+    free_hash_function();
 }
 
 
@@ -765,154 +1057,17 @@ std::string ConnectionID2::getStr() const {
 
 hash_t ConnectionID4::hash() const
 	{
-	struct {
-		in6_addr ip1;
-		in6_addr ip2;
-		uint16 port1;
-		uint16 port2;
-	} key;
-
-	// Lookup up connection based on canonical ordering, which is
-	// the smaller of <src addr, src port> and <dst addr, dst port>
-	// followed by the other.
-	if (addr6_port_canon_lt(v6.ip1, v6.ip2, v6.port1, v6.port2))
-		{
-            in6_addr s6_ip;
-            in6_addr d6_ip;
-            
-            memcpy(s6_ip.s6_addr, v6.ip1, 16);
-            memcpy(d6_ip.s6_addr, v6.ip2, 16);
-
-		    key.ip1 = s6_ip;
-		    key.ip2 = d6_ip;
-		    key.port1 = v6.port1;
-		    key.port2 = v6.port2;
-		}
-	else
-		{
-            in6_addr s6_ip;
-            in6_addr d6_ip;
-            
-            memcpy(s6_ip.s6_addr, v6.ip1, 16);
-            memcpy(d6_ip.s6_addr, v6.ip2, 16);
-
-		    key.ip1 = d6_ip;
-		    key.ip2 = s6_ip;
-		    key.port1 = v6.port2;
-		    key.port2 = v6.port1;
-		}
-
-    init_hash_function();
-
-    HashKey* newHashKey = new HashKey(&key, sizeof(key));
-
-    tmlog(TM_LOG_NOTE, "ConnectionID4::hash()", "the hash of newhashkey is %u", newHashKey->Hash());
-
-    HashKey* newHashKeyDos = new HashKey(&key, sizeof(key));
-
-    tmlog(TM_LOG_NOTE, "ConnectionID4::hash()", "the second hash of newhashkeydos is %u", newHashKeyDos->Hash());
-
-    //memcpy(&hash_key, newHashKey->Hash(), 8);
-
-    //hash_key = newHashKey->Hash();
-
-	return newHashKey->Hash();
+	    return hash_key;
 	}
 
 hash_t ConnectionID3::hash() const
 	{
-	struct {
-		in6_addr ip1;
-		in6_addr ip2;
-		uint16 port1;
-		uint16 port2;
-	} key;
-
-	// Lookup up connection based on canonical ordering, which is
-	// the smaller of <src addr, src port> and <dst addr, dst port>
-	// followed by the other.
-	if (addr6_port_canon_lt(v6.ip1, v6.ip2, 0, v6.port2))
-		{
-            in6_addr s6_ip;
-            in6_addr d6_ip;
-            
-            memcpy(s6_ip.s6_addr, v6.ip1, 16);
-            memcpy(d6_ip.s6_addr, v6.ip2, 16);
-
-		    key.ip1 = s6_ip;
-		    key.ip2 = d6_ip;
-		    key.port1 = 0;
-		    key.port2 = v6.port2;
-		}
-	else
-		{
-            in6_addr s6_ip;
-            in6_addr d6_ip;
-            
-            memcpy(s6_ip.s6_addr, v6.ip1, 16);
-            memcpy(d6_ip.s6_addr, v6.ip2, 16);
-
-		    key.ip1 = d6_ip;
-		    key.ip2 = s6_ip;
-		    key.port1 = v6.port2;
-		    key.port2 = 0;
-		}
-
-    init_hash_function();
-
-    HashKey* newHashKey = new HashKey(&key, sizeof(key));
-
-    //hash_key = newHashKey->Hash();
-
-	return newHashKey->Hash();
+        return hash_key;
 	}
 
 hash_t ConnectionID2::hash() const
 	{
-	struct {
-		in6_addr ip1;
-		in6_addr ip2;
-		uint16 port1;
-		uint16 port2;
-	} key;
-
-	// Lookup up connection based on canonical ordering, which is
-	// the smaller of <src addr, src port> and <dst addr, dst port>
-	// followed by the other.
-	if (addr6_port_canon_lt(v6.ip1, v6.ip2, 0, 0))
-		{
-            in6_addr s6_ip;
-            in6_addr d6_ip;
-            
-            memcpy(s6_ip.s6_addr, v6.ip1, 16);
-            memcpy(d6_ip.s6_addr, v6.ip2, 16);
-
-		    key.ip1 = s6_ip;
-		    key.ip2 = d6_ip;
-		    key.port1 = 0;
-		    key.port2 = 0;
-		}
-	else
-		{
-            in6_addr s6_ip;
-            in6_addr d6_ip;
-            
-            memcpy(s6_ip.s6_addr, v6.ip1, 16);
-            memcpy(d6_ip.s6_addr, v6.ip2, 16);
-
-		    key.ip1 = d6_ip;
-		    key.ip2 = s6_ip;
-		    key.port1 = 0;
-		    key.port2 = 0;
-		}
-
-    init_hash_function();
-
-    HashKey* newHashKey = new HashKey(&key, sizeof(key));
-
-    //hash_key = newHashKey->Hash();
-
-	return newHashKey->Hash();
+        return hash_key;
 	}
 
 // Static Member initialization
diff --git a/src/Connection.hh b/src/Connection.hh
index 3bb3e9d..20d5793 100644
--- a/src/Connection.hh
+++ b/src/Connection.hh
@@ -188,8 +188,6 @@ public:
 
     hash_t hash() const;
 
-    hash_t hash_key;
-
 protected:
 	void init(proto_t proto, uint32_t s_ip, uint32_t d_ip,
 			  uint16_t s_port, uint16_t d_port);
@@ -197,6 +195,7 @@ protected:
                uint16_t s_port, uint16_t d_port);
 	//v_t v;
     v6_t v6;
+    hash_t hash_key;
 
 private:
 	static std::string pattern_connection4;
@@ -239,7 +238,8 @@ public:
 		memcpy(&v6.ip2[12], &in4_ip2.s_addr, sizeof(in4_ip2.s_addr));
 		}
 
-	virtual ~ConnectionID3() {};
+	virtual ~ConnectionID3() {
+    };
     /*
 	uint32_t hash() const {
 		//TODO: initval
diff --git a/src/Hash.cc b/src/Hash.cc
index c0d7ca6..5e9a08a 100644
--- a/src/Hash.cc
+++ b/src/Hash.cc
@@ -31,6 +31,10 @@ void init_hash_function()
 	h3 = new H3<hash_t, UHASH_KEY_SIZE>();
 	}
 
+void free_hash_function()
+{
+    delete h3;
+}
 
 HashKey::HashKey(bro_int_t i)
 	{
diff --git a/src/Hash.h b/src/Hash.h
index c6bc51a..bc18434 100644
--- a/src/Hash.h
+++ b/src/Hash.h
@@ -88,4 +88,6 @@ protected:
 
 extern void init_hash_function();
 
+extern void free_hash_function();
+
 #endif
diff --git a/src/IndexField.cc b/src/IndexField.cc
index 380577c..14ac89f 100644
--- a/src/IndexField.cc
+++ b/src/IndexField.cc
@@ -188,6 +188,13 @@ void IPAddress::Init(const std::string& s)
 		uint32_t addr = (a[0] << 24) | (a[1] << 16) | (a[2] << 8) | a[3];
 		addr = htonl(addr);
 		memcpy(&ipv6_address.s6_addr[12], &addr, sizeof(uint32_t));
+
+        init_hash_function();
+        HashKey* newHashKey = new HashKey((void*)ipv6_address.s6_addr, sizeof(ipv6_address.s6_addr));
+
+        hash_key = newHashKey->Hash();
+
+        free_hash_function();
 		}
 
 	else
@@ -201,6 +208,13 @@ void IPAddress::Init(const std::string& s)
 			}
 		}
         tmlog(TM_LOG_NOTE, "IPAddress::Init", "good IP Address %s", s.c_str());
+
+        init_hash_function();
+        HashKey* newHashKey = new HashKey((void*)ipv6_address.s6_addr, sizeof(ipv6_address.s6_addr));
+
+        hash_key = newHashKey->Hash();
+
+        free_hash_function();
 	}
 
 std::string IPAddress::getStr() const
diff --git a/src/IndexField.hh b/src/IndexField.hh
index 442255e..dec6bf7 100644
--- a/src/IndexField.hh
+++ b/src/IndexField.hh
@@ -89,6 +89,13 @@ public:
 
 		memcpy(ipv6_address.s6_addr, v4_mapped_prefix, sizeof(v4_mapped_prefix));
 		memcpy(&ipv6_address.s6_addr[12], &in4.s_addr, sizeof(in4.s_addr));
+
+        init_hash_function();
+        HashKey* newHashKey = new HashKey((void*)ipv6_address.s6_addr, sizeof(ipv6_address.s6_addr));
+
+        hash_key = newHashKey->Hash();
+
+        free_hash_function();
     }
 /*
     IPAddress(unsigned char ip6[])
@@ -103,6 +110,13 @@ public:
     IPAddress(unsigned char ip6[])
     {
         memcpy(ipv6_address.s6_addr, ip6, 16);
+
+        init_hash_function();
+        HashKey* newHashKey = new HashKey((void*)ipv6_address.s6_addr, sizeof(ipv6_address.s6_addr));
+
+        hash_key = newHashKey->Hash();
+
+        free_hash_function();
     }
 /*
     // The inet_addr() function converts the Internet host address cp from IPv4 
@@ -125,9 +139,17 @@ public:
     }
 	IPAddress(void *p) {
 		memcpy((void*)getConstKeyPtr(), p, getKeySize());
+
+        init_hash_function();
+        HashKey* newHashKey = new HashKey((void*)ipv6_address.s6_addr, sizeof(ipv6_address.s6_addr));
+
+        hash_key = newHashKey->Hash();
+
+        free_hash_function();
 	}
 	virtual ~IPAddress() {
         tmlog(TM_LOG_NOTE, "IPAddress", "deleting an ipaddress type");
+        //delete [] ip6_address;
         //delete [] strIP;
     };
 
@@ -166,10 +188,7 @@ public:
 
 	virtual hash_t hash() const {
 		// TODO: initval
-        init_hash_function();
-        HashKey* newHashKey = new HashKey((void*)ipv6_address.s6_addr, sizeof(ipv6_address.s6_addr));
-
-	    return newHashKey->Hash();
+        return hash_key;
 	}
 	virtual const unsigned char* getInt() const {
 		return ipv6_address.s6_addr;
@@ -203,7 +222,9 @@ public:
 	static IndexField* parseQuery(const char *query);
 	virtual void getBPFStr(char *, int) const;
 
+protected:
     hash_t hash_key;
+
 private:
 
 	/**
@@ -329,8 +350,23 @@ public:
 
 class Port: public IndexField {
 public:
-	Port(): port(0) {}
+	Port(): port(0) {
+        init_hash_function();
+
+        HashKey* newHashKey = new HashKey((void*)ipv6_address.s6_addr, sizeof(ipv6_address.s6_addr));
+
+	    hash_key =  newHashKey->Hash();
+
+        free_hash_function();
+    }
 	Port(uint16_t port): port(port) {  /* printf("Port(%u)\n", port); */
+        init_hash_function();
+
+        HashKey* newHashKey = new HashKey((void*)ipv6_address.s6_addr, sizeof(ipv6_address.s6_addr));
+
+	    hash_key =  newHashKey->Hash();
+
+        free_hash_function();
 	}
 	virtual ~Port() {}
 
@@ -376,11 +412,7 @@ public:
     */
 	virtual hash_t hash() const {
 		// TODO: initval
-        init_hash_function();
-
-        HashKey* newHashKey = new HashKey((void*)ipv6_address.s6_addr, sizeof(ipv6_address.s6_addr));
-
-	    return newHashKey->Hash();
+        return hash_key;
 	}
 
 	virtual const std::string getIndexName() const {
@@ -394,6 +426,7 @@ public:
     virtual std::string getStrPkt(const u_char* packet) const;
 	virtual void getBPFStr(char *, int) const;
 protected:
+    hash_t hash_key;
 	uint16_t port;
 	static std::string pattern;
 	static RE2 re;
diff --git a/src/Query.cc b/src/Query.cc
index d5dcc77..0d87bec 100644
--- a/src/Query.cc
+++ b/src/Query.cc
@@ -43,6 +43,74 @@ bool QueryRequest::matchPkt(const pkt_ptr p) {
 bool QueryRequest::matchPkt(struct pcap_pkthdr *hdr, const u_char *pkt) {
 	tmlog(TM_LOG_NOTE, "QueryRequest: matchPkt(struct pcap_pkthdr, u_char pkt)", "determine if a packet matches");
 	if (!have_bpf) compileBPF();
+
+    uint16_t ether_type=ntohs(ETHERNET(pkt)->ether_type);
+    // ETHERTYPE_IP is EtherType 0x800, for IPv4 addresses
+    // EtherType 0x8100 is for VLAN header
+    // EtherType 0x86DD is for IPv6 addresses
+    // EtherType is a field in the ethernet header frame
+	if ( ! (ether_type==ETHERTYPE_IP || ether_type==0x8100 || ether_type==0x86DD) ) {
+
+        tmlog(TM_LOG_DEBUG, "QueryRequest::matchPkt, ~line 240", "unknown ether_type 0x%.4X", ether_type);
+
+		//    fprintf(stderr,"unknown ether_type 0x%.4X\n", ether_type);
+		return false;
+	}
+
+    // DEBUG DEBUG DEBUG
+	//tmlog(TM_LOG_NOTE, "addPkt: Storage.cc, ~line 240", "adding packet %lu !", header->ts.tv_usec);
+
+    //SetHdrSize();
+
+	// Unfortunately some packets on the link might have MPLS labels
+	// while others don't. That means we need to ask the link-layer if
+	// labels are in place. TODO: Cannot handle MPLS labels just yet
+	//bool have_mpls = false;
+
+	// skip VLAN header (related to ethernet frame header) for indexing TODO: look at VLAN header more closely
+    // Virtual Bridged Local Area Network for logically group network devices together, which share the same 
+    // physical network. VLAN tag is 4 bytes and so a VLAN header is 4 bytes longer than a regular ethernet header
+	//if (ether_type==0x8100) idxpacket+=4;
+    
+    if (ether_type == 0x8100)
+    {
+        tmlog(TM_LOG_NOTE, "QueryRequest::matchPkt", "we have a vlan tag");
+		// Check for MPLS in VLAN.
+        
+        // TODO: Cannot handle MPLS labels just yet
+        /*
+		if ( ((idxpacket[2] << 8) + idxpacket[3]) == 0x8847 )
+			have_mpls = true;
+        */
+
+		pkt += 4; // Skip the vlan header
+		//pkt_hdr_size = 0;
+
+		// Check for 802.1ah (Q-in-Q) containing IP.
+		// Only do a second layer of vlan tag
+		// stripping because there is no
+		// specification that allows for deeper
+		// nesting.
+		if ( ((pkt[2] << 8) + pkt[3]) == 0x0800 )
+			pkt += 4;
+    }
+
+    // TODO: Cannot handle MPLS labels just yet
+    /*
+	if ( have_mpls )
+		{
+		// Skip the MPLS label stack.
+		bool end_of_stack = false;
+
+		while ( ! end_of_stack )
+			{
+			end_of_stack = *(idxpacket + 2) & 0x01;
+			idxpacket += 4;
+			}
+		}
+    */
+
+
     tmlog(TM_LOG_NOTE, "QueryRequest: matchPkt(2 args)", "The value of the bpf_filter in query request  for packet %lu and %lu is %lu", hdr->ts.tv_sec, hdr->ts.tv_usec, bpf_filter(fp.bf_insns, (unsigned char*)(pkt), hdr->len, hdr->caplen));
 	return bpf_filter(fp.bf_insns, (u_char *)(pkt),
 					  hdr->len,
diff --git a/src/Storage.cc b/src/Storage.cc
index 01a13dc..af990c1 100644
--- a/src/Storage.cc
+++ b/src/Storage.cc
@@ -279,6 +279,7 @@ void Storage::cancelThread() {
 	fifos.push_back(f);
 }*/
 
+/*
 int get_link_header_size(int dl)
 	{
 	switch ( dl ) {
@@ -331,7 +332,7 @@ void Storage::SetHdrSize()
 
 	datalink = dl;
 	}
-
+*/
 
 
 // note that the pcap packet header has the timestamp of the packet
@@ -353,7 +354,7 @@ void Storage::addPkt(const struct pcap_pkthdr *header,
     // DEBUG DEBUG DEBUG
 	tmlog(TM_LOG_NOTE, "addPkt: Storage.cc, ~line 240", "adding packet %lu !", header->ts.tv_usec);
 
-    SetHdrSize();
+    //SetHdrSize();
 
 	// Unfortunately some packets on the link might have MPLS labels
 	// while others don't. That means we need to ask the link-layer if
@@ -554,8 +555,8 @@ void Storage::addPkt(const struct pcap_pkthdr *header,
         // So, it eventually leads to the writing of the class files (woo)
         // Also, note that f->addPkt returns true if we can add the packet, false otherwise
 		if ( (( c->getSuspendCutoff() | tcp_ctrl_flag )
-				&& f->addPkt(header, idxpacket, NULL)) ||
-				f->addPkt(header, idxpacket, c)) {
+				&& f->addPkt(header, packet, NULL)) ||
+				f->addPkt(header, packet, c)) {
 			/* packet was stored in Fifo *f */
             // woo-hoo! bytes from the packet were not cut
 			uncut_bytes += header->len;
diff --git a/src/Storage.hh b/src/Storage.hh
index 3e136d1..864f2d6 100644
--- a/src/Storage.hh
+++ b/src/Storage.hh
@@ -38,7 +38,7 @@ class StorageConfig {
 		Indexes *indexes;
 };
 
-extern int get_link_header_size(int dl);
+//extern int get_link_header_size(int dl);
 
 class Storage {
 public:
@@ -103,6 +103,7 @@ public:
 	
 	friend void *capture_thread(void *arg);
 
+/*
 protected:
     void Close();
     void SetHdrSize();
@@ -110,6 +111,7 @@ protected:
     char errbuf[PCAP_ERRBUF_SIZE];
     int hdr_size;
     //const u_char* data;
+*/
 
 private:
 	pcap_t *ph;


From neto at caltech.edu  Mon Jul 21 14:31:39 2014
From: neto at caltech.edu (Naoki Eto)
Date: Mon, 21 Jul 2014 14:31:39 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: It does not seem to seg fault
	anymore. I removed the bug where I overdeleted (did delete []
	instead of delete) (d3183a0)
Message-ID: <201407212131.s6LLVdo6006883@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit d3183a06a5bc6291b2da94828100534b5b8bdbed
Author: NaokiEto <neto at caltech.edu>
Date:   Mon Jul 21 17:18:34 2014 -0400

    It does not seem to seg fault anymore. I removed the bug where
    I overdeleted (did delete [] instead of delete)
    
    Also, fixed a few other of the memory leaks.
    
    It still has some memory leaks, notably in the addConn method.


>---------------------------------------------------------------

d3183a06a5bc6291b2da94828100534b5b8bdbed
 src/Connection.hh  |  2 +-
 src/Connections.cc | 43 +++++++++++++++++++++++++------------------
 src/Connections.hh |  4 ++++
 src/IndexField.hh  |  4 ++++
 src/Storage.cc     |  2 ++
 5 files changed, 36 insertions(+), 19 deletions(-)

diff --git a/src/Connection.hh b/src/Connection.hh
index 20d5793..8e91640 100644
--- a/src/Connection.hh
+++ b/src/Connection.hh
@@ -497,7 +497,7 @@ public:
 	Connection(Connection *c);
 	virtual ~Connection() {
     // we need delete [] instead of delete because we allocated an array for the ipv6 addresses?
-		delete [] c_id;
+		delete c_id;
 	}
 	void addPkt(const struct pcap_pkthdr* header, const u_char* packet);
 	tm_time_t getLastTs() {
diff --git a/src/Connections.cc b/src/Connections.cc
index 9ce3b25..efca426 100644
--- a/src/Connections.cc
+++ b/src/Connections.cc
@@ -20,31 +20,38 @@ Connections::Connections(uint32_t hash_size):
 
 Connections::~Connections() {
 	delete[] htable;
+    delete addedconn;
 	pthread_mutex_destroy(&lock_mutex);
 }
 
+void Connections::addConnHelper(ConnectionID4 *c_id)
+{
+    Connection *c;
+    unsigned hval;
 
-/* add a new connection to the table */
-Connection* Connections::addConn(ConnectionID4 *c_id) {
-	Connection *c;
-	unsigned hval;
+    c = new Connection(c_id); // c_id now owned by c
+    hval = c_id->hash()%size;
+    num_entries++;
 
-	c = new Connection(c_id); // c_id now owned by c
-	hval = c_id->hash()%size;
-	num_entries++;
+    /* LOCK-XXX: we cannot guarantee that the compiler doesn't reorder, so 
+     * we must use a lock()
+     */
+    lock();
+    c->col_next = htable[hval];
+    c->col_prev = NULL;
+    htable[hval] = c;
 
-	/* LOCK-XXX: we cannot guarantee that the compiler doesn't reorder, so 
-	 * we must use a lock()
-	 */
-	lock();
-	c->col_next = htable[hval];
-	c->col_prev = NULL;
-	htable[hval] = c;
+    if (c->col_next != NULL) 
+	    c->col_next->col_prev = c;
+    unlock();
 
-	if (c->col_next != NULL) 
-		c->col_next->col_prev = c;
-	unlock();
-	return c;
+    addedconn = c;
+}
+
+/* add a new connection to the table */
+Connection* Connections::addConn(ConnectionID4 *c_id) {
+    addConnHelper(c_id);
+    return addedconn;
 }
 
 
diff --git a/src/Connections.hh b/src/Connections.hh
index 380a366..390f8f3 100644
--- a/src/Connections.hh
+++ b/src/Connections.hh
@@ -162,6 +162,10 @@ protected:
 	// these are 
 	inline Connection* lookup(ConnectionID4 *c_id);
 	inline Connection* addPkt (const struct pcap_pkthdr*, const u_char*);
+	Connection* addedconn;
+
+	void addConnHelper(ConnectionID4 *c_id);
+
 	Connection* addConn(ConnectionID4 *c_id); 
 
 private:
diff --git a/src/IndexField.hh b/src/IndexField.hh
index dec6bf7..1328c9a 100644
--- a/src/IndexField.hh
+++ b/src/IndexField.hh
@@ -116,6 +116,8 @@ public:
 
         hash_key = newHashKey->Hash();
 
+        delete newHashKey;
+
         free_hash_function();
     }
 /*
@@ -145,6 +147,8 @@ public:
 
         hash_key = newHashKey->Hash();
 
+        delete newHashKey;
+
         free_hash_function();
 	}
 	virtual ~IPAddress() {
diff --git a/src/Storage.cc b/src/Storage.cc
index af990c1..9999cb4 100644
--- a/src/Storage.cc
+++ b/src/Storage.cc
@@ -597,6 +597,8 @@ void Storage::addPkt(const struct pcap_pkthdr *header,
 		 * end 
 		 */
 	}
+
+    //delete c;
 }
 
 


From neto at caltech.edu  Tue Jul 22 18:49:18 2014
From: neto at caltech.edu (Naoki Eto)
Date: Tue, 22 Jul 2014 18:49:18 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: made taking care of hash
	collisions more explicit (7b03c40)
Message-ID: <201407230149.s6N1nIDg018675@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit 7b03c403e07c2adaa9165c40cdffc93ed69cc786
Author: NaokiEto <neto at caltech.edu>
Date:   Tue Jul 22 21:36:46 2014 -0400

    made taking care of hash collisions more explicit


>---------------------------------------------------------------

7b03c403e07c2adaa9165c40cdffc93ed69cc786
 src/Connections.cc  |  28 +++++-
 src/Connections.hh  |  21 ++--
 src/DiskIndex.cc    |   1 +
 src/Index.cc        |   4 +-
 src/IndexHash.cc    | 268 ++++++++++++++++++++++++++++++++++++----------------
 src/IndexHash.hh    |   2 +
 src/conf_scanner.ll |   1 +
 src/main.cc.in      |  46 +++++++--
 8 files changed, 271 insertions(+), 100 deletions(-)

diff --git a/src/Connections.cc b/src/Connections.cc
index efca426..480a292 100644
--- a/src/Connections.cc
+++ b/src/Connections.cc
@@ -20,22 +20,47 @@ Connections::Connections(uint32_t hash_size):
 
 Connections::~Connections() {
 	delete[] htable;
+    addedconn = NULL;
     delete addedconn;
+    //delete addedpacket;
 	pthread_mutex_destroy(&lock_mutex);
 }
 
 void Connections::addConnHelper(ConnectionID4 *c_id)
 {
+    //Connection *c;
+    unsigned hval;
+
+    addedconn = new Connection(c_id); // c_id now owned by c
+    hval = c_id->hash()%size;
+    num_entries++;
+
+    /* LOCK-XXX: we cannot guarantee that the compiler doesn't reorder, so 
+     * we must use a lock()
+     */
+    lock();
+    addedconn->col_next = htable[hval];
+    addedconn->col_prev = NULL;
+    htable[hval] = addedconn;
+
+    if (addedconn->col_next != NULL) 
+	    addedconn->col_next->col_prev = addedconn;
+    unlock();
+
+    //addedconn = c;
+
+/*
     Connection *c;
     unsigned hval;
 
     c = new Connection(c_id); // c_id now owned by c
     hval = c_id->hash()%size;
     num_entries++;
-
+*/
     /* LOCK-XXX: we cannot guarantee that the compiler doesn't reorder, so 
      * we must use a lock()
      */
+/*
     lock();
     c->col_next = htable[hval];
     c->col_prev = NULL;
@@ -46,6 +71,7 @@ void Connections::addConnHelper(ConnectionID4 *c_id)
     unlock();
 
     addedconn = c;
+*/
 }
 
 /* add a new connection to the table */
diff --git a/src/Connections.hh b/src/Connections.hh
index 390f8f3..eddcb1c 100644
--- a/src/Connections.hh
+++ b/src/Connections.hh
@@ -162,7 +162,6 @@ protected:
 	// these are 
 	inline Connection* lookup(ConnectionID4 *c_id);
 	inline Connection* addPkt (const struct pcap_pkthdr*, const u_char*);
-	Connection* addedconn;
 
 	void addConnHelper(ConnectionID4 *c_id);
 
@@ -181,6 +180,10 @@ private:
 	// the hashtable containing the connections
 	hash_t* htable;
 
+	Connection* addedconn;
+
+	Connection* addedpacket;
+
 	/* every connection is in the hashtable and in a de-queue, where newest is
 	 * the connection that has been accessed last. I.e. oldest it the connection
 	 * that has been inactive for the longest time. 
@@ -214,24 +217,24 @@ inline Connection* Connections::lookup(ConnectionID4 *c_id) {
  */
 inline Connection* Connections::addPkt(const struct pcap_pkthdr* header, const u_char* packet) {
 	ConnectionID4* c_id=new ConnectionID4(packet);
-	Connection* c;
+	//Connection* c;
 
 #ifdef TM_HEAVY_DEBUG
 	checkme(to_tm_time(&header->ts));
 #endif
-	c = lookup(c_id);
-	if (c == NULL) {
-		c = addConn(c_id);
+	addedpacket = lookup(c_id);
+	if (addedpacket == NULL) {
+		addedpacket = addConn(c_id);
 		// c_id now belongs to c
 	}
 	else {
 		delete c_id;
-		q_remove(c);
+		q_remove(addedpacket);
 	}
 
-	c->addPkt(header, packet);
-	q_add_newest(c);
-	return c;
+	addedpacket->addPkt(header, packet);
+	q_add_newest(addedpacket);
+	return addedpacket;
 }
 
 
diff --git a/src/DiskIndex.cc b/src/DiskIndex.cc
index f4365e2..5245356 100644
--- a/src/DiskIndex.cc
+++ b/src/DiskIndex.cc
@@ -389,6 +389,7 @@ void IndexFiles<T>::aggregate(tm_time_t oldestTimestampDisk) {
 			unlink(oldest_fname);
 			file_number_oldest[num_aggregate_levels-1]++;
 		}
+        //free(oldest_fname);
 		delete ifr;
 	}
 	unlock_file_numbers();
diff --git a/src/Index.cc b/src/Index.cc
index eb3e2b0..cf9bdbb 100644
--- a/src/Index.cc
+++ b/src/Index.cc
@@ -293,6 +293,8 @@ void Index<T>::addEntry(IndexField *iqe) {
 		IndexEntry* ie_n=new IndexEntry(iqe, 
 				iqe->ts-IDX_PKT_SECURITY_MARGIN*idx_thread_iat, iqe->ts);
 		cur->add(iqe, ie_n);
+
+    	last_updated = iqe->ts;
 	} else {
         tmlog(TM_LOG_NOTE, "addEntry", "updating the entry with timestamp %f and form %s", iqe->ts, iqe->getStr().c_str());
         // the entry is already in the hash table
@@ -303,7 +305,7 @@ void Index<T>::addEntry(IndexField *iqe) {
 		delete iqe;
 	}
     // update last_updated time
-	last_updated = iqe->ts;
+	//last_updated = iqe->ts;
 
     // Note that old hash table is now the formerly current hash table. So, it is in the memory, and we can do look up on it
     // This must be the table that Aashish says that indexes do not persist, part of of index persistence (other part is
diff --git a/src/IndexHash.cc b/src/IndexHash.cc
index 6358eb1..33f79e8 100644
--- a/src/IndexHash.cc
+++ b/src/IndexHash.cc
@@ -117,95 +117,203 @@ void IndexHash::add(IndexField *key, IndexEntry *ie) {
 
     tmlog(TM_LOG_NOTE, "idx_hash", "entering the add method");
 
+    /*
+	const void* old_val = Insert(ie, key);
+
+    // A same entry was already in the table
+    if (old_val)
+    {
+        tmlog(TM_LOG_NOTE, "idx_has", "we already have the same entry!");
+        delete ie;
+        return;
+    }
+
+    else
+    {
+    */
+	    IndexEntry *cur, *prev;
+	    int cmp;
+	    cur = troot;
+	    prev = NULL;
+	    cmp = 0;
+    #ifdef TM_HEAVY_DEBUG
+	    assert(!lookup(key));
+	    if (troot)
+		    assert(troot->parent == NULL);
+	    assert(ie->parent == NULL);
+	    assert(ie->left == NULL);
+	    assert(ie->right == NULL);
+	    assert(ie->avlbal == 0);
+    #endif
+	
+        tmlog(TM_LOG_NOTE, "idx_hash", "the entry to add has key: %d", *(key->getConstKeyPtr()));
+
+        tmlog(TM_LOG_NOTE, "idx_hash", "the entry to add has timestampe: %f", key->ts);
+
+	    while (cur) {
+    #ifdef TM_HEAVY_DEBUG
+		    assert(cur->avlbal>=-1 && cur->avlbal<=1);
+    #endif
+
+		    prev = cur;
+            // determine where to put the entry?
+            // determines where current pointer to Index Entry should be (transversing through tree)
+            // this compares the 16 bytes/128-bit char arrays
+		    cmp = memcmp(key->getConstKeyPtr(), cur->getKey()->getConstKeyPtr(), key->getKeySize());
+
+	        if ( key->hash() == cur->getKey()->hash() &&
+	             cmp )
+		        {
+	                unsigned hvalconflict;
+
+	                hvalconflict = key->hash()%numBuckets;
+
+	                ie->col_next = htable[hvalconflict];
+	                ie->col_prev = NULL;
+	                htable[hvalconflict] = ie;
+                    return;
+		        }
+
+		    if (cmp > 0)
+			    cur = cur->left;
+		    else if (cmp < 0)
+			    cur = cur->right;
+		    else 
+            {
+                tmlog(TM_LOG_NOTE, "idx_hash", "the already existing entry is: %d\n", *(cur->getKey()->getConstKeyPtr()));
+			    tmlog(TM_LOG_NOTE, "idx_hash",  "tried to insert an already existing entry into the tree. numEntries=%d\n",
+					    getNumEntries());
+			    //h->add_or_update(key, ie);
+                delete ie;
+                delete key;
+			    abort();
+			    return ; // entry exists, shouldn't happen	
+		    }
+	    }
+
+	    if (!troot) {
+		    troot = ie;
+    #ifdef TM_HEAVY_DEBUG
+		    assert(troot->parent == NULL);
+		    assert(troot->left == NULL);
+		    assert(troot->right == NULL);
+		    assert(troot->avlbal == 0);
+    #endif
+	    }
+	    else {
+    #ifdef TM_HEAVY_DEBUG
+		    assert(prev);
+		    assert(!cur);
+		    assert(cmp!=0);
+    #endif
+		    // We are now at a terminal node. Insert the entry
+		    ie->parent = prev;
+		    if (cmp>0) {
+			    prev->left = ie;
+			    prev->avlbal--;
+		    }
+		    else {
+			    prev->right = ie;
+			    prev->avlbal++;
+		    }
+		    rebalance(prev);
+	    }
+
+	    unsigned hval;
+
+	    hval = key->hash()%numBuckets;
+
+	    ie->col_next = htable[hval];
+	    ie->col_prev = NULL;
+	    htable[hval] = ie;
+
+        ie->key->hash_key = hval;
+
+        tmlog(TM_LOG_NOTE, "idxhash", "this entry for which we have foudn the bucket value %d for has this timestamp %f and form %s", hval, key->ts, key->getStr().c_str());
+
+        tmlog(TM_LOG_NOTE, "idx_hash", "setting an entry in the hash table at %d", hval);
+	
+	    if (ie->col_next != NULL) 
+		    ie->col_next->col_prev = ie;
+
+	    numEntries++;
+	    return;
+    //}
+}
 
-	IndexEntry *cur, *prev;
-	int cmp;
-	cur = troot;
-	prev = NULL;
-	cmp = 0;
-#ifdef TM_HEAVY_DEBUG
-	assert(!lookup(key));
-	if (troot)
-		assert(troot->parent == NULL);
-	assert(ie->parent == NULL);
-	assert(ie->left == NULL);
-	assert(ie->right == NULL);
-	assert(ie->avlbal == 0);
-#endif
-	
-    tmlog(TM_LOG_NOTE, "idx_hash", "the entry to add has key: %d", *(key->getConstKeyPtr()));
-
-    tmlog(TM_LOG_NOTE, "idx_hash", "the entry to add has timestampe: %f", key->ts);
-
-	while (cur) {
-#ifdef TM_HEAVY_DEBUG
-		assert(cur->avlbal>=-1 && cur->avlbal<=1);
-#endif
-		prev = cur;
-		cmp = memcmp(key->getConstKeyPtr(), cur->getKey()->getConstKeyPtr(), key->getKeySize());
-		if (cmp > 0)
-			cur = cur->left;
-		else if (cmp < 0)
-			cur = cur->right;
-		else {
-            tmlog(TM_LOG_ERROR, "idx_hash", "the already existing entry is: %d\n", *(cur->getKey()->getConstKeyPtr()));
-			tmlog(TM_LOG_ERROR, "idx_hash",  "tried to insert an already existing entry into the tree. numEntries=%d\n",
-					getNumEntries());
-			//h->add_or_update(key, ie);
-			abort();
-			return ; // entry exists, shouldn't happen	
-		}
+/*
+// private
+const void* IndexHash::Insert(IndexEntry* new_entry, IndexField *key)
+{
+	//int* num_entries_ptr;
+	//int* max_num_entries_ptr;
+	unsigned h;
+    h = key->hash() % numBuckets;
+
+	IndexEntry* chain = htable[h];
+
+	int n = key->getKeySize();
+
+    // if a collisions list/element exists in that particular spot in the hash table
+	if ( chain )
+	{
+        IndexEntry *cur = chain;
+        
+        // go through all the elements in the collisions list
+	    while (cur != NULL)
+	    {
+	        IndexEntry entry = *chain;
+
+            // the hashes, length, and the keys are all the same
+            // then that means the entry with the same exact parameters
+            // already exists. So, we just return the old entry value.
+	        if ( entry.getKey()->hash() == key->hash() &&
+	             entry.getKey()->getKeySize() == n &&
+	             ! memcmp(entry.getKey()->getConstKeyPtr(), key->getConstKeyPtr(), n) )
+		        {
+                    const void* old_value;
+                    if (key->getConstKeyPtr() != 0)
+                        old_value = key->getConstKeyPtr();
+		            return old_value;
+		        }
+            // next element in collisions list
+            cur = cur->col_next;
+	    }
 	}
 
-	if (!troot) {
-		troot = ie;
-#ifdef TM_HEAVY_DEBUG
-		assert(troot->parent == NULL);
-		assert(troot->left == NULL);
-		assert(troot->right == NULL);
-		assert(troot->avlbal == 0);
-#endif
-	}
-	else {
-#ifdef TM_HEAVY_DEBUG
-		assert(prev);
-		assert(!cur);
-		assert(cmp!=0);
-#endif
-		// We are now at a terminal node. Insert the entry
-		ie->parent = prev;
-		if (cmp>0) {
-			prev->left = ie;
-			prev->avlbal--;
+	else
+		// Create new chain.
+		chain = ttbl[h] = new PList(DictEntry);
+
+	// If we got this far, then we couldn't use an existing copy
+	// of the key, so make a new one if necessary.
+	if ( copy_key )
+		{
+		void* old_key = new_entry->key;
+		new_entry->key = (void*) new char[n];
+		memcpy(new_entry->key, old_key, n);
+		delete (char*) old_key;
 		}
-		else {
-			prev->right = ie;
-			prev->avlbal++;
-		}
-		rebalance(prev);
-	}
-
-	unsigned hval;
-
-	hval = key->hash()%numBuckets;
-
-	ie->col_next = htable[hval];
-	ie->col_prev = NULL;
-	htable[hval] = ie;
 
-    ie->key->hash_key = hval;
+	// We happen to know (:-() that appending is more efficient
+	// on lists than prepending.
+	chain->append(new_entry);
 
-    tmlog(TM_LOG_NOTE, "idxhash", "this entry for which we have foudn the bucket value %d for has this timestamp %f and form %s", hval, key->ts, key->getStr().c_str());
+	if ( *max_num_entries_ptr < ++*num_entries_ptr )
+		*max_num_entries_ptr = *num_entries_ptr;
 
-    tmlog(TM_LOG_NOTE, "idx_hash", "setting an entry in the hash table at %d", hval);
-	
-	if (ie->col_next != NULL) 
-		ie->col_next->col_prev = ie;
+	// For ongoing iterations: If we already passed the bucket where this
+	// entry was put, add it to the cookie's list of inserted entries.
+	loop_over_list(cookies, i)
+		{
+		IterCookie* c = cookies[i];
+		if ( h < (unsigned int) c->bucket )
+			c->inserted.append(new_entry);
+		}
 
-	numEntries++;
-	return;
+	return 0;
 }
-
+*/
 
 // rebalance an avl tree. 
 // cur is the parent of the just inserted node
diff --git a/src/IndexHash.hh b/src/IndexHash.hh
index d123174..45717c4 100644
--- a/src/IndexHash.hh
+++ b/src/IndexHash.hh
@@ -248,6 +248,8 @@ class IndexHash {
 				w->parent = NULL;
 			}
 		}
+    //private:
+    //	const void* Insert(IndexEntry* entry, IndexField *key);
 };
 
 
diff --git a/src/conf_scanner.ll b/src/conf_scanner.ll
index 81356e0..2176765 100644
--- a/src/conf_scanner.ll
+++ b/src/conf_scanner.ll
@@ -90,6 +90,7 @@ NEWLINE \n
 			}
 \".*\"	 { conflval.s=strdup(yytext+1);
 		   conflval.s[strlen(conflval.s)-1]=0;
+           /*free(conflval.s)*/
 		   return TOK_STRING;
 		 } 
 {WHITE}
diff --git a/src/main.cc.in b/src/main.cc.in
index c154df3..a306d0f 100644
--- a/src/main.cc.in
+++ b/src/main.cc.in
@@ -67,6 +67,10 @@ struct timeval stats_last_call_ts;
 
 Storage* storage=NULL;
 
+LogFile *stats_log_file;
+LogFile *classes_log_file;
+LogFile *index_log_file;
+
 
 #define HOSTNAME_MAXLEN 32
 char hostname[HOSTNAME_MAXLEN];
@@ -354,11 +358,13 @@ void *index_aggregation_thread(void *arg) {
  *  logging interval: int*
  */
 
-void *statisticslog_thread(void* arg) {
+void *statisticslog_thread(void* arg) 
+{
+    /*
 	LogFile *stats_log_file;
 	LogFile *classes_log_file;
 	LogFile *index_log_file;
-
+    */
 #define LOG_STRING_SIZE 1000
 	struct timeval last_call_ts;
 	struct timeval now;
@@ -418,8 +424,10 @@ void *statisticslog_thread(void* arg) {
 			);
 	
 	memset(&last_call_ts, 0, sizeof(last_call_ts));
-	while (log_file) {
-		if (storage->getPcapStats(&ps) < 0 ) {
+	while (log_file) 
+    {
+		if (storage->getPcapStats(&ps) < 0 ) 
+        {
 			///TODO:
 		}
 		gettimeofday(&now, NULL);
@@ -433,7 +441,8 @@ void *statisticslog_thread(void* arg) {
 #endif
 		ru_utime = to_tm_time(&(r.ru_utime));
 		ru_stime = to_tm_time(&(r.ru_stime));
-		if (last_call_ts.tv_sec) {
+		if (last_call_ts.tv_sec) 
+        {
 			dt=to_tm_time(&now) - to_tm_time(&last_call_ts);
 
 			utime_perc = 100.*(ru_utime-last_ru_utime)/dt;
@@ -443,7 +452,8 @@ void *statisticslog_thread(void* arg) {
 			uncut_rate = (uncut_bytes-last_call_uncut_bytes)/dt/125000.;
 
 		}
-		else {
+		else 
+        {
 			link_rate = uncut_rate = 0;
 			utime_perc = stime_perc = 0.0;
 
@@ -497,7 +507,8 @@ void *statisticslog_thread(void* arg) {
 					 
 				);
 		stats_log_file->logPlain(s);
-		if (last_drop != ps.ps_drop) {
+		if (last_drop != ps.ps_drop) 
+        {
 			log_file->log("DROP" , "we dropped packets: %u", ps.ps_drop-last_drop);
 			last_drop = ps.ps_drop;
 		}
@@ -506,13 +517,15 @@ void *statisticslog_thread(void* arg) {
 		
 		// TODO: make fifos stats and index stats consistent
 		fifos = storage->getFifos();
-		for(std::list<Fifo*>::iterator i=fifos.begin(); i!=fifos.end(); i++) {
+		for(std::list<Fifo*>::iterator i=fifos.begin(); i!=fifos.end(); i++) 
+        {
 			snprintf(s, LOG_STRING_SIZE, "%.2lf %s %s", to_tm_time(&now),  (*i)->getClassname().c_str(), (*i)->getStatsStr().c_str());
 			classes_log_file->logPlain(s);
 		}
 
 		indexes = storage->getIndexes();
-		for (std::list<IndexType*>::iterator i=indexes->begin(); i!=indexes->end(); i++) {
+		for (std::list<IndexType*>::iterator i=indexes->begin(); i!=indexes->end(); i++) 
+        {
 			snprintf(s, LOG_STRING_SIZE, 
 					"%.2lf "
 					"%s "
@@ -566,8 +579,18 @@ void *statisticslog_thread(void* arg) {
 		*/
 #endif
 
+/*
+    if (!log_file)
+    {
+        delete stats_log_file;
+        delete classes_log_file;
+        delete index_log_file;  
+
+    }
+*/
 		sleep(*(int*)arg);
 	}  
+
 	// log file was closed
 	return NULL;
 }
@@ -826,6 +849,11 @@ main(int argc, char** argv) {
 			while (1) 
 				pause();
 		}
+
+        delete stats_log_file;
+        delete classes_log_file;
+        delete index_log_file; 
+
 		tmexit();
 
 }


From neto at caltech.edu  Fri Jul 25 16:16:05 2014
From: neto at caltech.edu (Naoki Eto)
Date: Fri, 25 Jul 2014 16:16:05 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: fixed some bugs mainly in
	Connection.cc and IndexField.cc runs with less dropped packets,
	still a sizeable amount of dropped packets though (10% with indexes)
	runs with less cpu usage,
	still rather large cpu usage (120% with indexes) (155a210)
Message-ID: <201407252316.s6PNG50c018012@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit 155a210838f92af4e043624a45f030a9b87b805c
Author: NaokiEto <neto at caltech.edu>
Date:   Fri Jul 25 19:02:27 2014 -0400

    fixed some bugs mainly in Connection.cc and IndexField.cc
    runs with less dropped packets, still a sizeable amount of dropped
    packets though (10% with indexes)
    runs with less cpu usage, still rather large cpu usage (120% with indexes)


>---------------------------------------------------------------

155a210838f92af4e043624a45f030a9b87b805c
 src/Connection.cc  | 152 ++++++++++++++++++++--------------------------
 src/Connection.hh  |  33 +++++++++-
 src/Connections.cc |  38 ++++++++----
 src/Connections.hh |   2 +-
 src/Fifo.cc        |  22 +++----
 src/FifoDisk.cc    |  12 ++--
 src/FifoMem.cc     |  12 ++--
 src/HashAlt.cc     | 156 -----------------------------------------------
 src/IndexField.cc  | 119 +++++++++++++++++++++++++-----------
 src/IndexField.hh  |   8 ++-
 src/Storage.cc     |  10 +--
 src/jhash3.h       | 175 -----------------------------------------------------
 src/util.h         |  24 ++++++++
 13 files changed, 269 insertions(+), 494 deletions(-)

diff --git a/src/Connection.cc b/src/Connection.cc
index f7c1ae8..95586f5 100644
--- a/src/Connection.cc
+++ b/src/Connection.cc
@@ -12,6 +12,11 @@
 #include "Query.hh"
 #include "tm.h"
 
+#ifdef __FreeBSD__
+#include <sys/socket.h>
+#endif
+
+
 static std::string pattern_ip ("(\\d+\\.\\d+\\.\\d+\\.\\d+)");
 static std::string pattern_ipport ("(\\d+\\.\\d+\\.\\d+\\.\\d+):(\\d+)");
 
@@ -21,6 +26,7 @@ static std::string pattern_ip6 ("\\[(([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}|([
 static std::string pattern_ip6port ("\\[(([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,7}:|([0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2}|([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}|([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}|([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5}|[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6})|:((:[0-9a-fA-F]{1,4}){1,7}|:)|fe80:(:[0-9a-fA-F]{0,4}){0,4}%[0-9a-zA-Z]{1,}|::(ffff(:0{1,4}){0,1}:){0,1}((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]).){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])|([0-9a-fA-F]{1,4}:){1,4}:((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]).){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]))\\]:(\\d+)");
 // stolen from stackoverflow http://stackoverflow.com/questions/53497/regular-expression-that-
 
+/*
 inline uint32_t revert_uint32(uint32_t i) {
 	uint32_t r;
 	((uint8_t*)&r)[0]=((uint8_t*)&i)[3];
@@ -38,6 +44,7 @@ inline uint16_t revert_uint16(uint16_t i) {
 
 	return r;
 }
+*/
 
 inline bool addr_port_canon_lt(uint32_t s_ip, uint32_t d_ip,
 							   uint16_t s_port, uint16_t d_port) {
@@ -49,10 +56,10 @@ inline bool addr_port_canon_lt(uint32_t s_ip, uint32_t d_ip,
 
 inline bool addr6_port_canon_lt(const unsigned char s6_ip[], const unsigned char d6_ip[],
                                 uint16_t s_port, uint16_t d_port) {
-	if (s6_ip == d6_ip)
+	if (!(memcmp(s6_ip, d6_ip, 16)))
 		return (s_port < d_port);
 	else
-		return (s6_ip < d6_ip);
+		return (memcmp(s6_ip, d6_ip, 16) < 0);
 }
 
 
@@ -72,16 +79,10 @@ void ConnectionID4::init(proto_t proto,
     ipv4_d_address.s_addr = d_ip;
     ipv4_s_address.s_addr = s_ip;
 
-	struct {
-		in6_addr ip1;
-		in6_addr ip2;
-		uint16 port1;
-		uint16 port2;
-	} key;
-
 	if (addr_port_canon_lt(s_ip,d_ip,s_port,d_port)) {
 		//    v.is_canonified=true;
 
+        // setting v6.ip1 to be  destination address and v6.ip2 to be source address
         ConnectionID4(ipv4_d_address, ipv4_s_address);
 
         /*
@@ -105,18 +106,20 @@ void ConnectionID4::init(proto_t proto,
         in6_addr s6_ip;
         in6_addr d6_ip;
  
-        memcpy(s6_ip.s6_addr, v6.ip1, 16);
-        memcpy(d6_ip.s6_addr, v6.ip2, 16);
+        // setting s6_ip to be source address and d6_ip to be dest address       
+        memcpy(s6_ip.s6_addr, v6.ip2, 16);
+        memcpy(d6_ip.s6_addr, v6.ip1, 16);
 
         key.ip1 = s6_ip;
         key.ip2 = d6_ip;
-        key.port1 = v6.port1;
-        key.port2 = v6.port2;
+        key.port1 = s_port;
+        key.port2 = d_port;
 
 	} 
     else {
 		//    v.is_canonified=false;
 
+        // setting v6.ip1 to be source address and v6.ip2 to be destination address
         ConnectionID4(ipv4_s_address, ipv4_d_address);
 
         /*
@@ -145,8 +148,8 @@ void ConnectionID4::init(proto_t proto,
 
 	    key.ip1 = d6_ip;
 	    key.ip2 = s6_ip;
-	    key.port1 = v6.port2;
-	    key.port2 = v6.port1;
+	    key.port1 = d_port;
+	    key.port2 = s_port;
 	}
 
     tmlog(TM_LOG_NOTE, "connection4: Connection.cc", "connection 4 with form %s", getStr().c_str());
@@ -179,13 +182,6 @@ void ConnectionID4::init6(proto_t proto,
     v6.version = 6;
 	v6.proto=proto;
 
-	struct {
-		in6_addr ip1;
-		in6_addr ip2;
-		uint16 port1;
-		uint16 port2;
-	} key;
-
 	// Lookup up connection based on canonical ordering, which is
 	// the smaller of <src addr, src port> and <dst addr, dst port>
 	// followed by the other.
@@ -203,13 +199,13 @@ void ConnectionID4::init6(proto_t proto,
         in6_addr s6_ip;
         in6_addr d6_ip;
         
-        memcpy(s6_ip.s6_addr, v6.ip1, 16);
-        memcpy(d6_ip.s6_addr, v6.ip2, 16);
+        memcpy(s6_ip.s6_addr, s_ip, 16);
+        memcpy(d6_ip.s6_addr, d_ip, 16);
 
         key.ip1 = s6_ip;
         key.ip2 = d6_ip;
-        key.port1 = v6.port1;
-        key.port2 = v6.port2;
+        key.port1 = s_port;
+        key.port2 = d_port;
 
 	} else {
 		//    v6.is_canonified=false;
@@ -224,13 +220,13 @@ void ConnectionID4::init6(proto_t proto,
         in6_addr s6_ip;
         in6_addr d6_ip;
         
-        memcpy(s6_ip.s6_addr, v6.ip1, 16);
-        memcpy(d6_ip.s6_addr, v6.ip2, 16);
+        memcpy(s6_ip.s6_addr, s_ip, 16);
+        memcpy(d6_ip.s6_addr, d_ip, 16);
 
 	    key.ip1 = d6_ip;
 	    key.ip2 = s6_ip;
-	    key.port1 = v6.port2;
-	    key.port2 = v6.port1;
+	    key.port1 = d_port;
+	    key.port2 = s_port;
 	}
     tmlog(TM_LOG_NOTE, "connection4: Connection.cc", "connection 4 with form %s", getStr().c_str());
 
@@ -267,8 +263,8 @@ void ConnectionID3::init(proto_t proto,
 
 	v6.proto=proto;
     
-    ipv4_d_address.s_addr = ip1;
-    ipv4_s_address.s_addr = ip2;
+    ipv4_s_address.s_addr = ip1;
+    ipv4_d_address.s_addr = ip2;
 
     /*
     IPAddr(ipv4_s_address);
@@ -285,13 +281,6 @@ void ConnectionID3::init(proto_t proto,
 
 	v6.port2=port2;
 
-	struct {
-		in6_addr ip1;
-		in6_addr ip2;
-		uint16 port1;
-		uint16 port2;
-	} key;
-
 	// Lookup up connection based on canonical ordering, which is
 	// the smaller of <src addr, src port> and <dst addr, dst port>
 	// followed by the other.
@@ -347,6 +336,7 @@ void ConnectionID3::init6(proto_t proto,
     v6.version = 6;
 
     // memcpy(destination, source, size)
+    // setting v6.ip1 to be source address and v6.ip2 to be destination address 
     memcpy(v6.ip1, ip1, 16);
     memcpy(v6.ip2, ip2, 16);
 
@@ -354,13 +344,6 @@ void ConnectionID3::init6(proto_t proto,
 	//v.ip2=ip2;
 	v6.port2=port2;
 
-	struct {
-		in6_addr ip1;
-		in6_addr ip2;
-		uint16 port1;
-		uint16 port2;
-	} key;
-
 	// Lookup up connection based on canonical ordering, which is
 	// the smaller of <src addr, src port> and <dst addr, dst port>
 	// followed by the other.
@@ -369,9 +352,11 @@ void ConnectionID3::init6(proto_t proto,
             in6_addr s6_ip;
             in6_addr d6_ip;
     
-            memcpy(s6_ip.s6_addr, v6.ip1, 16);
-            memcpy(d6_ip.s6_addr, v6.ip2, 16);
+            // setting s6_ip to have source address and d6_ip to have dest address        
+            memcpy(s6_ip.s6_addr, ip1, 16);
+            memcpy(d6_ip.s6_addr, ip2, 16);
 
+            // setting key.ip1 to be source address and key.ip2 to be dest address 
 		    key.ip1 = s6_ip;
 		    key.ip2 = d6_ip;
 		    key.port1 = 0;
@@ -382,9 +367,11 @@ void ConnectionID3::init6(proto_t proto,
             in6_addr s6_ip;
             in6_addr d6_ip;
         
-            memcpy(s6_ip.s6_addr, v6.ip1, 16);
-            memcpy(d6_ip.s6_addr, v6.ip2, 16);
+            // setting s6_ip to have source address and d6_ip to have dest address    
+            memcpy(s6_ip.s6_addr, ip1, 16);
+            memcpy(d6_ip.s6_addr, ip2, 16);
 
+            // setting key.ip1 to be dest address and key.ip2 to be source address
 		    key.ip1 = d6_ip;
 		    key.ip2 = s6_ip;
 		    key.port1 = v6.port2;
@@ -414,15 +401,8 @@ void ConnectionID2::init( uint32_t s_ip, uint32_t d_ip) {
     in4_addr ipv4_d_address;
     in4_addr ipv4_s_address;
 
-    ipv4_d_address.s_addr = s_ip;
-    ipv4_s_address.s_addr = d_ip;
-
-	struct {
-		in6_addr ip1;
-		in6_addr ip2;
-		uint16 port1;
-		uint16 port2;
-	} key;
+    ipv4_s_address.s_addr = s_ip;
+    ipv4_d_address.s_addr = d_ip;
 
 	if (addr_port_canon_lt(s_ip,d_ip,0,0)) {
 		//    v.is_canonified=true;
@@ -434,15 +414,18 @@ void ConnectionID2::init( uint32_t s_ip, uint32_t d_ip) {
         memcpy(in6.s6_addr, v6.ip2, 16);
         */
 
+        // setting v6.ip1 to dest address and v6.ip2 to source address
         ConnectionID2(ipv4_d_address, ipv4_s_address);
 
         // this is for the hash key
         in6_addr s6_ip;
         in6_addr d6_ip;
  
-        memcpy(s6_ip.s6_addr, v6.ip1, 16);
-        memcpy(d6_ip.s6_addr, v6.ip2, 16);
+        // setting s6_ip to be source address and d6_ip to be dest address       
+        memcpy(s6_ip.s6_addr, v6.ip2, 16);
+        memcpy(d6_ip.s6_addr, v6.ip1, 16);
 
+        // setting key.ip1 to be source address and key.ip2 to be dest address
 	    key.ip1 = s6_ip;
 	    key.ip2 = d6_ip;
 	    key.port1 = 0;
@@ -458,15 +441,18 @@ void ConnectionID2::init( uint32_t s_ip, uint32_t d_ip) {
         memcpy(in6.s6_addr, v6.ip2, 16);
         */
 
+        // setting v6.ip1 to be source address and v6.ip2 to be dest address
         ConnectionID2(ipv4_s_address, ipv4_d_address);
 
         // this is for the hash key
         in6_addr s6_ip;
         in6_addr d6_ip;
  
+        // ssetting s6_ip to be source address and d6_ip to be dest address    
         memcpy(s6_ip.s6_addr, v6.ip1, 16); 
         memcpy(d6_ip.s6_addr, v6.ip2, 16);
 
+        // setting key.ip1 to be dest address and key.ip2 to be source address
 	    key.ip1 = d6_ip;
 	    key.ip2 = s6_ip;
 	    key.port1 = 0;
@@ -493,13 +479,6 @@ void ConnectionID2::init6( unsigned char s_ip[], unsigned char d_ip[]) {
 
     v6.version = 6;
 
-	struct {
-		in6_addr ip1;
-		in6_addr ip2;
-		uint16 port1;
-		uint16 port2;
-	} key;
-
 	if (addr6_port_canon_lt(s_ip,d_ip,0,0)) {
 		//    v.is_canonified=true;
         // memcpy(destination, source, size)
@@ -513,11 +492,11 @@ void ConnectionID2::init6( unsigned char s_ip[], unsigned char d_ip[]) {
         in6_addr s6_ip;
         in6_addr d6_ip;
         
-        memcpy(s6_ip.s6_addr, v6.ip1, 16);
-        memcpy(d6_ip.s6_addr, v6.ip2, 16);
+        memcpy(s6_ip.s6_addr, s_ip, 16);
+        memcpy(d6_ip.s6_addr, d_ip, 16);
 
-	    key.ip1 = s6_ip;
-	    key.ip2 = d6_ip;
+	    key.ip1 = d6_ip;
+	    key.ip2 = s6_ip;
 	    key.port1 = 0;
 	    key.port2 = 0;
 
@@ -532,11 +511,11 @@ void ConnectionID2::init6( unsigned char s_ip[], unsigned char d_ip[]) {
         in6_addr s6_ip;
         in6_addr d6_ip;
         
-        memcpy(s6_ip.s6_addr, v6.ip1, 16);
-        memcpy(d6_ip.s6_addr, v6.ip2, 16);
+        memcpy(s6_ip.s6_addr, s_ip, 16);
+        memcpy(d6_ip.s6_addr, d_ip, 16);
 
-	    key.ip1 = d6_ip;
-	    key.ip2 = s6_ip;
+	    key.ip1 = s6_ip;
+	    key.ip2 = d6_ip;
 	    key.port1 = 0;
 	    key.port2 = 0;
 	}
@@ -742,8 +721,8 @@ ConnectionID2::ConnectionID2(const u_char* packet) {
 //consistent with ConnectionID4
 bool ConnectionID3::operator==(const ConnectionID& other) const {
 	return (v6.proto == ((ConnectionID3*)&other)->v6.proto)
-		   && (v6.ip1 == ((ConnectionID3*)&other)->v6.ip1)
-		   && (v6.ip2 == ((ConnectionID3*)&other)->v6.ip2)
+		   && (!memcmp(v6.ip1, ((ConnectionID3*)&other)->v6.ip1, 16))
+		   && (!memcmp(v6.ip2, ((ConnectionID3*)&other)->v6.ip2, 16))
 		   && (v6.port2 == ((ConnectionID3*)&other)->v6.port2)
            && (v6.version == ((ConnectionID3*)&other)->v6.version);
 }
@@ -751,8 +730,8 @@ bool ConnectionID3::operator==(const ConnectionID& other) const {
 //TODO: MAke this inline (i.e. move to Connection.hh so that it is
 //consistent with ConnectionID4
 bool ConnectionID2::operator==(const ConnectionID& other) const {
-	return (v6.ip1 == ((ConnectionID2*)&other)->v6.ip1)
-		   && (v6.ip2 == ((ConnectionID2*)&other)->v6.ip2)
+	return (!memcmp(v6.ip1, ((ConnectionID2*)&other)->v6.ip1, 16))
+		   && (!memcmp(v6.ip2, ((ConnectionID2*)&other)->v6.ip2, 16))
            && (v6.version == ((ConnectionID2*)&other)->v6.version);
 }
 
@@ -1100,15 +1079,18 @@ ConnectionID4* ConnectionID4::parse(const char *str) {
     }
     else
     {
-        unsigned char src_ip6[16];
-        unsigned char dst_ip6[16];
+        //unsigned char src_ip6[16];
+        //unsigned char dst_ip6[16];
 
         //const char* src_ip6;
         //char* dst_ip6;
 
-        if (inet_pton(AF_INET6, src_ip.c_str(), src_ip6) == 1 && inet_pton(AF_INET6, dst_ip.c_str(), dst_ip6) == 1)
+        in6_addr src_addr6;
+        in6_addr dst_addr6;
+
+        if (inet_pton(AF_INET6, src_ip.c_str(), &(src_addr6)) == 1 && inet_pton(AF_INET6, dst_ip.c_str(), &(dst_addr6)) == 1)
         {
-	        return new ConnectionID4(proto, src_ip6, dst_ip6, htons(src_port), htons(dst_port));
+	        return new ConnectionID4(proto, src_addr6.s6_addr, dst_addr6.s6_addr, htons(src_port), htons(dst_port));
         }
         return NULL;
     }
@@ -1117,7 +1099,7 @@ ConnectionID4* ConnectionID4::parse(const char *str) {
 void Connection::addPkt(const struct pcap_pkthdr* header, const u_char* packet) {
 	last_ts=to_tm_time(&header->ts);
 	tot_pkts++;
-	tot_pktbytes+=header->caplen;
+	tot_pktbytes+=header->caplen; //len
 }
 
 int Connection::deleteSubscription() {
diff --git a/src/Connection.hh b/src/Connection.hh
index 8e91640..64a69a9 100644
--- a/src/Connection.hh
+++ b/src/Connection.hh
@@ -87,8 +87,8 @@ public:
     }
     */
 	bool operator==(const ConnectionID& other) const { 
-		return (v6.ip1 == ((ConnectionID4*)&other)->v6.ip1)
-			   && (v6.ip2 == ((ConnectionID4*)&other)->v6.ip2)
+		return (!memcmp(v6.ip1, ((ConnectionID4*)&other)->v6.ip1, 16))
+			   && (!memcmp(v6.ip2, ((ConnectionID4*)&other)->v6.ip2, 16))
 			   && (v6.port1 == ((ConnectionID4*)&other)->v6.port1)
 			   && (v6.port2 == ((ConnectionID4*)&other)->v6.port2)
 			   && (v6.proto == ((ConnectionID4*)&other)->v6.proto);
@@ -168,6 +168,15 @@ public:
 	}
     // have the structure fields align on one-byte boundaries
 	__attribute__((packed)) v6_t;
+
+    typedef struct {
+        in6_addr ip1;
+        in6_addr ip2;
+        uint16 port1;
+        uint16 port2;
+    } 
+    __attribute__((packed)) key_t;
+
     /*
 	v_t* getV() {
 		return &v;
@@ -195,6 +204,7 @@ protected:
                uint16_t s_port, uint16_t d_port);
 	//v_t v;
     v6_t v6;
+    key_t key;
     hash_t hash_key;
 
 private:
@@ -311,6 +321,14 @@ public:
 	}
 	__attribute__((packed)) v6_t;
 
+    typedef struct {
+        in6_addr ip1;
+        in6_addr ip2;
+        uint16 port1;
+        uint16 port2;
+    }
+    __attribute__((packed)) key_t;
+
 /*
 	v_t* getV() {
 		return &v;
@@ -347,6 +365,7 @@ protected:
                uint16_t port);
 	//v_t v;
     v6_t v6;
+    key_t key;
 
 
 //private:
@@ -442,6 +461,15 @@ public:
 		//    bool is_canonified;
 	}
 	__attribute__((packed)) v6_t;
+
+    typedef struct {
+        in6_addr ip1;
+        in6_addr ip2;
+        uint16 port1;
+        uint16 port2;
+    }
+    __attribute__((packed)) key_t;
+
 /*
 	v_t* getV() {
 		return &v;
@@ -474,6 +502,7 @@ protected:
     void init6( unsigned char s_ip[], unsigned char d_ip[]);
 	//v_t v;
     v6_t v6;
+    key_t key;
 
 //private:
 	//in6_addr in6; // IPv6 or v4-to-v6-mapped address
diff --git a/src/Connections.cc b/src/Connections.cc
index 480a292..ac97e37 100644
--- a/src/Connections.cc
+++ b/src/Connections.cc
@@ -26,6 +26,7 @@ Connections::~Connections() {
 	pthread_mutex_destroy(&lock_mutex);
 }
 
+/*
 void Connections::addConnHelper(ConnectionID4 *c_id)
 {
     //Connection *c;
@@ -35,9 +36,9 @@ void Connections::addConnHelper(ConnectionID4 *c_id)
     hval = c_id->hash()%size;
     num_entries++;
 
-    /* LOCK-XXX: we cannot guarantee that the compiler doesn't reorder, so 
-     * we must use a lock()
-     */
+    //LOCK-XXX: we cannot guarantee that the compiler doesn't reorder, so 
+    //we must use a lock()
+    
     lock();
     addedconn->col_next = htable[hval];
     addedconn->col_prev = NULL;
@@ -49,18 +50,15 @@ void Connections::addConnHelper(ConnectionID4 *c_id)
 
     //addedconn = c;
 
-/*
     Connection *c;
     unsigned hval;
 
     c = new Connection(c_id); // c_id now owned by c
     hval = c_id->hash()%size;
     num_entries++;
-*/
-    /* LOCK-XXX: we cannot guarantee that the compiler doesn't reorder, so 
-     * we must use a lock()
-     */
-/*
+    //LOCK-XXX: we cannot guarantee that the compiler doesn't reorder, so 
+    //we must use a lock()
+    
     lock();
     c->col_next = htable[hval];
     c->col_prev = NULL;
@@ -71,12 +69,30 @@ void Connections::addConnHelper(ConnectionID4 *c_id)
     unlock();
 
     addedconn = c;
-*/
 }
+*/
 
 /* add a new connection to the table */
 Connection* Connections::addConn(ConnectionID4 *c_id) {
-    addConnHelper(c_id);
+ 
+    unsigned hval;
+
+    addedconn = new Connection(c_id); // c_id now owned by c
+    hval = c_id->hash()%size;
+    num_entries++;
+
+    /* LOCK-XXX: we cannot guarantee that the compiler doesn't reorder, so
+     * we must use a lock()
+     */
+    lock();
+    addedconn->col_next = htable[hval];
+    addedconn->col_prev = NULL;
+    htable[hval] = addedconn;
+
+    if (addedconn->col_next != NULL)
+        addedconn->col_next->col_prev = addedconn;
+    unlock();
+   
     return addedconn;
 }
 
diff --git a/src/Connections.hh b/src/Connections.hh
index eddcb1c..fb7d81f 100644
--- a/src/Connections.hh
+++ b/src/Connections.hh
@@ -163,7 +163,7 @@ protected:
 	inline Connection* lookup(ConnectionID4 *c_id);
 	inline Connection* addPkt (const struct pcap_pkthdr*, const u_char*);
 
-	void addConnHelper(ConnectionID4 *c_id);
+	//void addConnHelper(ConnectionID4 *c_id);
 
 	Connection* addConn(ConnectionID4 *c_id); 
 
diff --git a/src/Fifo.cc b/src/Fifo.cc
index 7bf3468..0e08e6b 100644
--- a/src/Fifo.cc
+++ b/src/Fifo.cc
@@ -116,19 +116,19 @@ uint64_t Fifo::pktEviction() {
         // getS() method is from FifoMem.c - it gets the beginning of first valid packet in the memory ring buffer block
 
 
-        char str1[INET6_ADDRSTRLEN];
+        //char str1[INET6_ADDRSTRLEN];
 
-        bro_inet_ntop(AF_INET6, &(IP6(fm->getS() + 4 + sizeof(struct pcap_pkthdr))->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
+        //bro_inet_ntop(AF_INET6, &(IP6(fm->getS() + 4 + sizeof(struct pcap_pkthdr))->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
 
         //char s1[INET6_ADDRSTRLEN];
 
         //inet_pton(AF_INET6, s1, str1);
 
-        char str2[INET6_ADDRSTRLEN];
+        //char str2[INET6_ADDRSTRLEN];
 
-        bro_inet_ntop(AF_INET6, &(IP6(fm->getS() + 4 + sizeof(struct pcap_pkthdr))->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
+        //bro_inet_ntop(AF_INET6, &(IP6(fm->getS() + 4 + sizeof(struct pcap_pkthdr))->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
 
-        tmlog(TM_LOG_NOTE, "Fifo::pktEviction", "we ard adding the packet to the FifoDisk with src ip %s and dst ip %s", str1, str2);
+        //tmlog(TM_LOG_NOTE, "Fifo::pktEviction", "we ard adding the packet to the FifoDisk with src ip %s and dst ip %s", str1, str2);
 		fd->addPkt(fm->getS());
         // pop the packet from the memory ring buffer (I'm still not sure about the align stuff, with a_s, a_lp)
         // note that popPkt returns the size of the popped packet in bytes
@@ -160,23 +160,23 @@ bool Fifo::addPkt(const struct pcap_pkthdr* header,
 			return false;
 		} else {
 
-            char str1[INET6_ADDRSTRLEN];
+            //char str1[INET6_ADDRSTRLEN];
 
-            bro_inet_ntop(AF_INET6, &(IP6(packet + 4)->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
+            //bro_inet_ntop(AF_INET6, &(IP6(packet + 4)->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
 
             //char s1[INET6_ADDRSTRLEN];
 
             //inet_pton(AF_INET6, s1, str1);
 
-            char str2[INET6_ADDRSTRLEN];
+            //char str2[INET6_ADDRSTRLEN];
 
-            bro_inet_ntop(AF_INET6, &(IP6(packet + 4)->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
+            //bro_inet_ntop(AF_INET6, &(IP6(packet + 4)->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
 
-            tmlog(TM_LOG_NOTE, "Fifo::addPkt", "the packet we are adding to FifoMem has source ip %s and destination ip %s", str1, str2);
+            //tmlog(TM_LOG_NOTE, "Fifo::addPkt", "the packet we are adding to FifoMem has source ip %s and destination ip %s", str1, str2);
 
 			// normal addition to Memory Fifo since this packet is not cutoff
 			fm->addPkt(header, packet);
-            tmlog(TM_LOG_DEBUG, "addPkt: Fifo.cc, ~line 121", "Connection cut-off did not occur for packet %lu", header->ts.tv_usec);
+            //tmlog(TM_LOG_DEBUG, "addPkt: Fifo.cc, ~line 121", "Connection cut-off did not occur for packet %lu", header->ts.tv_usec);
             // increment the stored_pkts and stored_bytes
 			stored_pkts++;
 			stored_bytes+=header->len;
diff --git a/src/FifoDisk.cc b/src/FifoDisk.cc
index 917cfcf..d7d48fe 100644
--- a/src/FifoDisk.cc
+++ b/src/FifoDisk.cc
@@ -130,19 +130,19 @@ void FifoDisk::addPkt(const pkt_ptr p) {
 			unlockQueryInProgress();
 		}
 
-        char str1[INET6_ADDRSTRLEN];
+        //char str1[INET6_ADDRSTRLEN];
 
-        bro_inet_ntop(AF_INET6, &(IP6(p + 4 + sizeof(struct pcap_pkthdr))->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
+        //bro_inet_ntop(AF_INET6, &(IP6(p + 4 + sizeof(struct pcap_pkthdr))->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
 
         //char s1[INET6_ADDRSTRLEN];
 
         //inet_pton(AF_INET6, s1, str1);
 
-        char str2[INET6_ADDRSTRLEN];
+        //char str2[INET6_ADDRSTRLEN];
 
-        bro_inet_ntop(AF_INET6, &(IP6(p + 4 + sizeof(struct pcap_pkthdr))->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
+        //bro_inet_ntop(AF_INET6, &(IP6(p + 4 + sizeof(struct pcap_pkthdr))->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
 
-        tmlog(TM_LOG_NOTE, "FifoDisk::addPkt", "we are going to add in next step the packet to the FifoDisk with src ip %s and dst ip %s", str1, str2);
+        //tmlog(TM_LOG_NOTE, "FifoDisk::addPkt", "we are going to add in next step the packet to the FifoDisk with src ip %s and dst ip %s", str1, str2);
 
         // in the last file, the newest file, add the packet
 		files.back()->addPkt(p);
@@ -154,7 +154,7 @@ void FifoDisk::addPkt(const pkt_ptr p) {
         // increment the number of held packets
 		held_pkts++;
         // increment the number of total bytes entered into this Fifo by the size of the pcap packet header struct and the length of the packet
-		tot_bytes+=sizeof(struct pcap_pkthdr)+((struct pcap_pkthdr*)p)->caplen;
+		tot_bytes+=sizeof(struct pcap_pkthdr)+((struct pcap_pkthdr*)p)->caplen; //len;
         // increment the number of total packets entered into this Fifo
 		tot_pkts++;
 	}
diff --git a/src/FifoMem.cc b/src/FifoMem.cc
index f82b19b..22c6599 100644
--- a/src/FifoMem.cc
+++ b/src/FifoMem.cc
@@ -97,19 +97,19 @@ void FifoMem::addPkt(const struct pcap_pkthdr *header,
 
 
 
-        char str1[INET6_ADDRSTRLEN];
+        //char str1[INET6_ADDRSTRLEN];
 
-        bro_inet_ntop(AF_INET6, &(IP6(wp + 4 + sizeof(struct pcap_pkthdr))->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
+        //bro_inet_ntop(AF_INET6, &(IP6(wp + 4 + sizeof(struct pcap_pkthdr))->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
 
         //char s1[INET6_ADDRSTRLEN];
 
         //inet_pton(AF_INET6, s1, str1);
 
-        char str2[INET6_ADDRSTRLEN];
+        //char str2[INET6_ADDRSTRLEN];
 
-        bro_inet_ntop(AF_INET6, &(IP6(wp + 4 + sizeof(struct pcap_pkthdr))->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
+        //bro_inet_ntop(AF_INET6, &(IP6(wp + 4 + sizeof(struct pcap_pkthdr))->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
 
-        tmlog(TM_LOG_NOTE, "FifoMem::addPkt", "we just wrote to the fifo memory ring buffer the packet with src ip %s and dst ip %s", str1, str2);
+        //tmlog(TM_LOG_NOTE, "FifoMem::addPkt", "we just wrote to the fifo memory ring buffer the packet with src ip %s and dst ip %s", str1, str2);
 
 
 
@@ -157,7 +157,7 @@ void FifoMem::addPkt(const struct pcap_pkthdr *header,
 		}
         // increment these since we are adding a packet with bytes
 		tot_pkts++;
-		tot_pktbytes+=header->caplen;
+		tot_pktbytes+=header->caplen; //len;
 		held_pkts++;
 		held_bytes+=header->caplen;
 
diff --git a/src/HashAlt.cc b/src/HashAlt.cc
deleted file mode 100644
index cb25057..0000000
--- a/src/HashAlt.cc
+++ /dev/null
@@ -1,156 +0,0 @@
-#ifndef HASH_CC
-#define HASH_CC
-
-#include <algorithm>
-#include <assert.h>
-#include <stdio.h>
-
-#include "Hash.hh"
-
-template <class K, class V>
-inline Hash<K, V>::Hash(size_t size):
-size(size),
-numEntries(0) {
-	table=new table_cell_t[size];
-}
-
-template <class K, class V>
-inline Hash<K, V>::~Hash() {
-	delete []table;
-}
-
-template <class K, class V>
-inline HashNode<K, V>::HashNode(K k, V& v):
-k(k), v(v) {}
-
-template <class K, class V>
-inline HashNode<K, V>::~HashNode() { };
-
-template <class K, class V>
-inline bool HashNode<K, V>::operator==(const K& k1) const {
-	return *k1==*k;
-}
-
-template <class K, class V>
-inline bool HashNode<K, V>::operator!=(const K& k1) const {
-	return *k1!=*k;
-}
-
-// typedef template Hash<K, V>::iterator_t hash_k_v_iterator_t;
-
-template <class K, class V>
-inline int Hash<K, V>::clear() {
-	int k=0;	
-	/* very ugly hack */
-	numEntries=0;
-	for (unsigned i=0; i<size; i++) { 
-		while(!table[i].empty()) {
-			delete table[i].back().k;
-			delete table[i].back().v;
-			table[i].pop_back();
-			k++;
-		}
-	}
-	return k;
-
-
-}
-
-/* Add (K,V) pair, updating existing K with new V
- * Returns the iterator (pointer) to its place in hash
- */
-template <class K, class V>
-inline typename Hash<K, V>::iterator_t Hash<K, V>::add_or_update(K k, V v) {
-	// look up the entry in hashtable
-	table_cell_t* entry=find_table_entry(k);
-	// search in list in hashtable entry
-	typename table_cell_t::iterator	f=find_iterator(entry, k);
-	if (f!=entry->end())
-		f->v=v;
-	else {
-		numEntries++;
-		entry->push_back(HashNode<K, V>(k, v));
-		f=find_iterator(entry, k);
-	}
-	assert(f!=entry->end());
-	return iterator_t(entry, f);
-	//return (iterator_t){entry, f};
-}
-
-template <class K, class V>
-inline void Hash<K, V>::erase(K k) {
-	// look up the entry in hashtable
-	table_cell_t* entry=find_table_entry(k);
-	// search in list in hashtable entry
-	typename table_cell_t::iterator	f=find_iterator(entry, k);
-	if (f!=entry->end()) {
-		//	  delete((*f).k);
-		numEntries--;
-		entry->erase(f);
-	}
-	//	delete((*f).k);
-}
-
-template <class K, class V>
-inline void Hash<K, V>::erase(iterator_t& it) {
-	assert(!it.c->empty());
-	assert(it.i!=it.c->end());
-	numEntries--;
-	it.c->erase(it.i);
-	//	delete((*(it.i)).k);
-}
-
-template <class K, class V>
-inline V Hash<K, V>::lookup(const K k) {
-	// look up the entry in hashtable
-	table_cell_t* entry=find_table_entry(k);
-	// search in list in hashtable entry
-	typename table_cell_t::iterator	f=find_iterator(entry, k);
-	if (f!=entry->end()) {
-//		printf("Hash::lookup(): found\n");
-		return f->v;
-	} else {
-//		printf("Hash::lookup(): not found\n");
-//		throw 1;
-		return NULL;
-	}
-}
-
-template <class K, class V>
-inline bool Hash<K, V>::isElement(const K k) {
-	// look up the entry in hashtable
-	table_cell_t* entry=find_table_entry(k);
-	// search in list in hashtable entry
-	typename table_cell_t::iterator	f=find_iterator(entry, k);
-	return f!=entry->end();
-}
-
-template <class K, class V>
-inline typename Hash<K, V>::table_cell_t* Hash<K, V>::find_table_entry(const K k) {
-	return &table[(k->hash())%size];
-}
-
-
-template <class K, class V>
-inline typename Hash<K, V>::table_cell_t::iterator Hash<K, V>::find_iterator(
-	table_cell_t* entry, const K k) {
-	typename Hash<K, V>::table_cell_t::iterator i=entry->begin();
-	for (; i!=entry->end() && !(*((*i).k)==*k) ; i++);
-	return i;
-
-	// this only worked with no-pointer k
-	//	return find(entry->begin(), entry->end(), k);
-}
-
-template <class K, class V>
-void Hash<K, V>::debugPrint() {
-	debugPrint(stderr);
-}
-
-template <class K, class V>
-void Hash<K, V>::debugPrint(FILE *fp) {
-	for (uint32_t i=0; i<size; i++)
-		fprintf(fp, "%u\t%u\n", i, table[i].size());
-}
-
-#endif
diff --git a/src/IndexField.cc b/src/IndexField.cc
index 14ac89f..e3af7a6 100644
--- a/src/IndexField.cc
+++ b/src/IndexField.cc
@@ -339,7 +339,7 @@ std::string SrcIPAddress::getStrPkt(const u_char* packet) const
 	else
 		{   
 
-        tmlog(TM_LOG_NOTE, "SrcIPAddress: getStr(u_char*)", "IPAddress, IPv6");
+        //tmlog(TM_LOG_NOTE, "SrcIPAddress: getStr(u_char*)", "IPAddress, IPv6");
 		char str[INET6_ADDRSTRLEN];
 
 		if ( ! inet_ntop(AF_INET6, IP6(packet)->ip6_src.s6_addr, str, INET6_ADDRSTRLEN) )
@@ -470,18 +470,38 @@ void Port::getBPFStr(char *str, int max_str_len) const {
 }
 
 
-SrcPort::SrcPort(const u_char* packet) {
-	switch (IP(packet)->ip_p) {
-	case IPPROTO_UDP:
-		port=(UDP(packet)->uh_sport);
-		break;
-	case IPPROTO_TCP:
-		port=(TCP(packet)->th_sport);
-		break;
-	default:
-		port=0;
-		break;
+SrcPort::SrcPort(const u_char* packet)  {
+
+    if (IP(packet)->ip_v==4)
+    { 
+	    switch (IP(packet)->ip_p) {
+	    case IPPROTO_UDP:
+		    port=(UDP(packet)->uh_sport);
+		    break;
+		case IPPROTO_TCP:
+			port=(TCP(packet)->th_sport);
+			break;
+		default:
+			port=0;
+			break;
+		}
 	}
+
+	else
+	{
+		switch (IP6(packet)->ip6_ctlun.ip6_un1.ip6_un1_nxt)
+        {
+            case IPPROTO_UDP:
+            	port=(UDP6(packet)->uh_sport); 
+                break;
+            case IPPROTO_TCP:
+				port=(TCP6(packet)->th_sport);
+				break;
+			default:
+				port=0;
+				break;
+        }
+    }		
 };
 
 std::list<SrcPort*> SrcPort::genKeys(const u_char* packet) {
@@ -506,17 +526,37 @@ void SrcPort::getBPFStr(char *str, int max_str_len) const {
 
 
 DstPort::DstPort(const u_char* packet) {
-	switch (IP(packet)->ip_p) {
-	case IPPROTO_UDP:
-		port=(UDP(packet)->uh_dport);
-		break;
-	case IPPROTO_TCP:
-		port=(TCP(packet)->th_dport);
-		break;
-	default:
-		port=0;
-		break;
+
+    if (IP(packet)->ip_v==4)
+    {
+		switch (IP(packet)->ip_p) {
+		case IPPROTO_UDP:
+			port=(UDP(packet)->uh_dport);
+			break;
+		case IPPROTO_TCP:
+			port=(TCP(packet)->th_dport);
+			break;
+		default:
+			port=0;
+			break;
+		}
 	}
+
+    else
+    {
+        switch (IP6(packet)->ip6_ctlun.ip6_un1.ip6_un1_nxt)
+        {
+            case IPPROTO_UDP:
+                port=(UDP6(packet)->uh_dport);
+                break;
+            case IPPROTO_TCP:
+                port=(TCP6(packet)->th_dport);
+                break;
+            default:
+                port=0;
+                break;
+        }
+    }
 };
 
 std::list<DstPort*> DstPort::genKeys(const u_char* packet) {
@@ -590,16 +630,19 @@ IndexField* ConnectionIF4::parseQuery(const char *query) {
     
     else
     {
-        unsigned char src_ip6[16];
-        unsigned char dst_ip6[16];
+        //unsigned char src_ip6[16];
+        //unsigned char dst_ip6[16];
 
         //const char* src_ip6;
         //char* dst_ip6;
 
-        if (inet_pton(AF_INET6, src_ip.c_str(), src_ip6) == 1 && inet_pton(AF_INET6, dst_ip.c_str(), dst_ip6) == 1)
+        in6_addr src_addr6;
+        in6_addr dst_addr6;
+
+        if (inet_pton(AF_INET6, src_ip.c_str(), &(src_addr6)) == 1 && inet_pton(AF_INET6, dst_ip.c_str(), &(dst_addr6)) == 1)
         {
-	        return new ConnectionIF4(proto, src_ip6, htons(src_port),
-	                dst_ip6, htons(dst_port));
+	        return new ConnectionIF4(proto, src_addr6.s6_addr, htons(src_port),
+	                dst_addr6.s6_addr, htons(dst_port));
         }
         return NULL;
     }
@@ -699,12 +742,15 @@ IndexField* ConnectionIF3::parseQuery(const char *query) {
     }
     else
     {
-        unsigned char src_ip6[16];
-        unsigned char dst_ip6[16];
+        //unsigned char src_ip6[16];
+        //unsigned char dst_ip6[16];
 
-        if (inet_pton(AF_INET6, src_ip.c_str(), src_ip6) == 1 && inet_pton(AF_INET6, dst_ip.c_str(), dst_ip6) == 1)
+        in6_addr src_addr6;
+        in6_addr dst_addr6;
+
+        if (inet_pton(AF_INET6, src_ip.c_str(), &(src_addr6)) == 1 && inet_pton(AF_INET6, dst_ip.c_str(), &(dst_addr6)) == 1)
         {
-            return new ConnectionIF3(proto, src_ip6, dst_ip6, htons(port));
+            return new ConnectionIF3(proto, src_addr6.s6_addr, dst_addr6.s6_addr, htons(port));
         }
         return NULL;
     }
@@ -782,12 +828,15 @@ IndexField* ConnectionIF2::parseQuery(const char *query) {
     }
     else
     {
-        unsigned char src_ip6[16];
-        unsigned char dst_ip6[16];
+        //unsigned char src_ip6[16];
+        //unsigned char dst_ip6[16];
+
+        in6_addr src_addr6;
+        in6_addr dst_addr6;
 
-        if (inet_pton(AF_INET6, src_ip.c_str(), src_ip6) == 1 && inet_pton(AF_INET6, dst_ip.c_str(), dst_ip6) == 1)
+        if (inet_pton(AF_INET6, src_ip.c_str(), &(src_addr6)) == 1 && inet_pton(AF_INET6, dst_ip.c_str(), &(dst_addr6)) == 1)
         {
-            return new ConnectionIF2(src_ip6, dst_ip6);
+            return new ConnectionIF2(src_addr6.s6_addr, dst_addr6.s6_addr);
         }
         return NULL;
     }
diff --git a/src/IndexField.hh b/src/IndexField.hh
index 1328c9a..5300ac6 100644
--- a/src/IndexField.hh
+++ b/src/IndexField.hh
@@ -95,6 +95,8 @@ public:
 
         hash_key = newHashKey->Hash();
 
+        delete newHashKey;
+
         free_hash_function();
     }
 /*
@@ -244,7 +246,7 @@ private:
 	static const uint8_t v4_mapped_prefix[12]; // top 96 bits of v4-mapped-addr
 
 	//uint32_t ip_address;
-    unsigned char ip6_address[16]; // unsigned char ip6_address[16]
+    //unsigned char ip6_address[16]; // unsigned char ip6_address[16]
  	static std::string pattern;
     static std::string pattern6;
 	static RE2 re;
@@ -361,6 +363,8 @@ public:
 
 	    hash_key =  newHashKey->Hash();
 
+        delete newHashKey;
+
         free_hash_function();
     }
 	Port(uint16_t port): port(port) {  /* printf("Port(%u)\n", port); */
@@ -370,6 +374,8 @@ public:
 
 	    hash_key =  newHashKey->Hash();
 
+        delete newHashKey;
+
         free_hash_function();
 	}
 	virtual ~Port() {}
diff --git a/src/Storage.cc b/src/Storage.cc
index 9999cb4..ae49a6e 100644
--- a/src/Storage.cc
+++ b/src/Storage.cc
@@ -41,19 +41,19 @@ void callback(u_char *args, const struct pcap_pkthdr *header,
 	network_time=header->ts;
 	*/ 
 
-    char str1[INET6_ADDRSTRLEN];
+    //char str1[INET6_ADDRSTRLEN];
 
-    inet_ntop(AF_INET6, &(IP6(packet)->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
+    //inet_ntop(AF_INET6, &(IP6(packet)->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
 
     //char s1[INET6_ADDRSTRLEN];
 
     //inet_pton(AF_INET6, s1, str1);
 
-    char str2[INET6_ADDRSTRLEN];
+    //char str2[INET6_ADDRSTRLEN];
 
-    inet_ntop(AF_INET6, &(IP6(packet)->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
+    //inet_ntop(AF_INET6, &(IP6(packet)->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
 
-    tmlog(TM_LOG_NOTE, "Storage::callback", "we are going to call addpacket of storage on packet that has source ip %s and destination ip %s", str1, str2);
+    //tmlog(TM_LOG_NOTE, "Storage::callback", "we are going to call addpacket of storage on packet that has source ip %s and destination ip %s", str1, str2);
 
     // DEBUG DEBUG DEBUG
 	tmlog(TM_LOG_DEBUG, "storage callback: Storage.cc, ~line 29", "Callback function for pcap_loop%lu", header->ts.tv_usec);
diff --git a/src/jhash3.h b/src/jhash3.h
deleted file mode 100644
index ca9fce9..0000000
--- a/src/jhash3.h
+++ /dev/null
@@ -1,175 +0,0 @@
-#ifndef JHASH3_H
-#define JHASH3_H
-/*
--------------------------------------------------------------------------------
-lookup3.c, by Bob Jenkins, May 2006, Public Domain.
-
-Modified by Gregor (hashXwords added)
-
-These are functions for producing 32-bit hashes for hash table lookup.
-hashword(), hashlittle(), hashlittle2(), hashbig(), mix(), and final() 
-are externally useful functions.  Routines to test the hash are included 
-if SELF_TEST is defined.  You can use this free for any purpose.  It's in
-the public domain.  It has no warranty.
-
-You probably want to use hashlittle().  hashlittle() and hashbig()
-hash byte arrays.  hashlittle() is is faster than hashbig() on
-little-endian machines.  Intel and AMD are little-endian machines.
-On second thought, you probably want hashlittle2(), which is identical to
-hashlittle() except it returns two 32-bit hashes for the price of one.  
-You could implement hashbig2() if you wanted but I haven't bothered here.
-
-If you want to find a hash of, say, exactly 7 integers, do
-  a = i1;  b = i2;  c = i3;
-  mix(a,b,c);
-  a += i4; b += i5; c += i6;
-  mix(a,b,c);
-  a += i7;
-  final(a,b,c);
-then use c as the hash value.  If you have a variable length array of
-4-byte integers to hash, use hashword().  If you have a byte array (like
-a character string), use hashlittle().  If you have several byte arrays, or
-a mix of things, see the comments above hashlittle().  
-
-Why is this so big?  I read 12 bytes at a time into 3 4-byte integers, 
-then mix those integers.  This is fast (you can do a lot more thorough
-mixing with 12*3 instructions on 3 integers than you can with 3 instructions
-on 1 byte), but shoehorning those bytes into integers efficiently is messy.
--------------------------------------------------------------------------------
-*/
-
-#include <stdint.h>     /* defines uint32_t etc */
-
-/*
---------------------------------------------------------------------
- This works on all machines.  To be useful, it requires
- -- that the key be an array of uint32_t's, and
- -- that the length be the number of uint32_t's in the key
-
- The function hashword() is identical to hashlittle() on little-endian
- machines, and identical to hashbig() on big-endian machines,
- except that the length has to be measured in uint32_ts rather than in
- bytes.  hashlittle() is more complicated than hashword() only because
- hashlittle() has to dance around fitting the key bytes into registers.
---------------------------------------------------------------------
-*/
-#define rot(x,k) (((x)<<(k)) ^ ((x)>>(32-(k))))
-
-/*
--------------------------------------------------------------------------------
-mix -- mix 3 32-bit values reversibly.
-
-This is reversible, so any information in (a,b,c) before mix() is
-still in (a,b,c) after mix().
-
-If four pairs of (a,b,c) inputs are run through mix(), or through
-mix() in reverse, there are at least 32 bits of the output that
-are sometimes the same for one pair and different for another pair.
-This was tested for:
-* pairs that differed by one bit, by two bits, in any combination
-  of top bits of (a,b,c), or in any combination of bottom bits of
-  (a,b,c).
-* "differ" is defined as +, -, ^, or ~^.  For + and -, I transformed
-  the output delta to a Gray code (a^(a>>1)) so a string of 1's (as
-  is commonly produced by subtraction) look like a single 1-bit
-  difference.
-* the base values were pseudorandom, all zero but one bit set, or 
-  all zero plus a counter that starts at zero.
-
-Some k values for my "a-=c; a^=rot(c,k); c+=b;" arrangement that
-satisfy this are
-    4  6  8 16 19  4
-    9 15  3 18 27 15
-   14  9  3  7 17  3
-Well, "9 15 3 18 27 15" didn't quite get 32 bits diffing
-for "differ" defined as + with a one-bit base and a two-bit delta.  I
-used http://burtleburtle.net/bob/hash/avalanche.html to choose 
-the operations, constants, and arrangements of the variables.
-
-This does not achieve avalanche.  There are input bits of (a,b,c)
-that fail to affect some output bits of (a,b,c), especially of a.  The
-most thoroughly mixed value is c, but it doesn't really even achieve
-avalanche in c.
-
-This allows some parallelism.  Read-after-writes are good at doubling
-the number of bits affected, so the goal of mixing pulls in the opposite
-direction as the goal of parallelism.  I did what I could.  Rotates
-seem to cost as much as shifts on every machine I could lay my hands
-on, and rotates are much kinder to the top and bottom bits, so I used
-rotates.
--------------------------------------------------------------------------------
-*/
-#define mix(a,b,c) \
-{ \
-  a -= c;  a ^= rot(c, 4);  c += b; \
-  b -= a;  b ^= rot(a, 6);  a += c; \
-  c -= b;  c ^= rot(b, 8);  b += a; \
-  a -= c;  a ^= rot(c,16);  c += b; \
-  b -= a;  b ^= rot(a,19);  a += c; \
-  c -= b;  c ^= rot(b, 4);  b += a; \
-}
-
-/*
--------------------------------------------------------------------------------
-final -- final mixing of 3 32-bit values (a,b,c) into c
-
-Pairs of (a,b,c) values differing in only a few bits will usually
-produce values of c that look totally different.  This was tested for
-* pairs that differed by one bit, by two bits, in any combination
-  of top bits of (a,b,c), or in any combination of bottom bits of
-  (a,b,c).
-* "differ" is defined as +, -, ^, or ~^.  For + and -, I transformed
-  the output delta to a Gray code (a^(a>>1)) so a string of 1's (as
-  is commonly produced by subtraction) look like a single 1-bit
-  difference.
-* the base values were pseudorandom, all zero but one bit set, or 
-  all zero plus a counter that starts at zero.
-
-These constants passed:
- 14 11 25 16 4 14 24
- 12 14 25 16 4 14 24
-and these came close:
-  4  8 15 26 3 22 24
- 10  8 15 26 3 22 24
- 11  8 15 26 3 22 24
--------------------------------------------------------------------------------
-*/
-#define final(a,b,c) \
-{ \
-  c ^= b; c -= rot(b,14); \
-  a ^= c; a -= rot(c,11); \
-  b ^= a; b -= rot(a,25); \
-  c ^= b; c -= rot(b,16); \
-  a ^= c; a -= rot(c,4);  \
-  b ^= a; b -= rot(a,14); \
-  c ^= b; c -= rot(b,24); \
-}
-
-uint32_t hashword(
-const uint32_t *k,                   /* the key, an array of uint32_t values */
-size_t          length,               /* the length of the key, in uint32_ts */
-uint32_t        initval);         /* the previous hash, or an arbitrary value */
-
-/* A hacked down version of hashword by Gregor */
-inline uint32_t hash3words(uint32_t a, uint32_t b, uint32_t c, uint32_t initval) 
-{
-	a +=  0xdeadbeef + (((uint32_t)3)<<2) + initval;
-	b +=  0xdeadbeef + (((uint32_t)3)<<2) + initval;
-	c +=  0xdeadbeef + (((uint32_t)3)<<2) + initval;
-
-	final(a,b,c);
-	return c;
-}
-
-/* A hacked down version of hashword by Gregor */
-inline uint32_t hash2words(uint32_t a, uint32_t b, uint32_t initval) 
-{
-	return hash3words(a,b,0,initval);
-}
-/* A hacked down version of hashword by Gregor */
-inline uint32_t hash1words(uint32_t a, uint32_t initval) 
-{
-	return hash3words(a,0,0,initval);
-}
-
-#endif
diff --git a/src/util.h b/src/util.h
index 8764a78..341f0a7 100644
--- a/src/util.h
+++ b/src/util.h
@@ -72,12 +72,35 @@ typedef uint64 bro_uint_t;
 // pointer size. They can be cast safely to a pointer, e.g. in Lists,
 // which represent their entities as void* pointers.
 //
+#ifdef linux
 #if SIZEOF_VOID_P == 8
+
 typedef uint64 ptr_compat_uint;
 typedef int64 ptr_compat_int;
 #define PRI_PTR_COMPAT_INT PRId64 // Format to use with printf.
 #define PRI_PTR_COMPAT_UINT PRIu64
+
 #elif SIZEOF_VOID_P == 4
+
+typedef uint32 ptr_compat_uint;
+typedef int32 ptr_compat_int;
+#define PRI_PTR_COMPAT_INT PRId32
+#define PRI_PTR_COMPAT_UINT PRIu32
+#else
+# error "Unusual pointer size. Please report to bro at bro.org."
+#endif
+#endif
+
+#ifdef __FreeBSD__ 
+#if SIZEOF_VOID_P  8
+
+typedef uint64 ptr_compat_uint;
+typedef int64 ptr_compat_int;
+#define PRI_PTR_COMPAT_INT PRId64 // Format to use with printf.
+#define PRI_PTR_COMPAT_UINT PRIu64
+
+#elif SIZEOF_VOID_P  4
+
 typedef uint32 ptr_compat_uint;
 typedef int32 ptr_compat_int;
 #define PRI_PTR_COMPAT_INT PRId32
@@ -85,6 +108,7 @@ typedef int32 ptr_compat_int;
 #else
 # error "Unusual pointer size. Please report to bro at bro.org."
 #endif
+#endif
 
 extern "C"
 	{


From neto at caltech.edu  Sun Jul 27 15:39:18 2014
From: neto at caltech.edu (Naoki Eto)
Date: Sun, 27 Jul 2014 15:39:18 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: making this commit to save
	latest work. This commit attempted to fix the Hash bugs (f356165)
Message-ID: <201407272239.s6RMdIEE006144@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit f3561658c55eaa146b6affcca8d7ddae7ceb5bed
Author: NaokiEto <neto at caltech.edu>
Date:   Sun Jul 27 18:26:06 2014 -0400

    making this commit to save latest work. This commit attempted to fix
    the Hash bugs


>---------------------------------------------------------------

f3561658c55eaa146b6affcca8d7ddae7ceb5bed
 src/Connection.cc | 41 +++++++++++++++++++++++----
 src/Hash.cc       |  4 +--
 src/Hash.h        |  2 +-
 src/IndexField.cc |  4 +--
 src/IndexField.hh | 10 +++----
 src/IndexHash.cc  | 85 ++++++++++++++++++++++++++++++++-----------------------
 6 files changed, 94 insertions(+), 52 deletions(-)

diff --git a/src/Connection.cc b/src/Connection.cc
index 95586f5..70e603e 100644
--- a/src/Connection.cc
+++ b/src/Connection.cc
@@ -170,7 +170,7 @@ void ConnectionID4::init(proto_t proto,
 
     delete newHashKey;
 
-    free_hash_function();
+    //free_hash_function();
 }
 
 void ConnectionID4::init6(proto_t proto,
@@ -246,7 +246,7 @@ void ConnectionID4::init6(proto_t proto,
 
     delete newHashKey;
 
-    free_hash_function();
+    //free_hash_function();
 }
 
 void ConnectionID3::init(proto_t proto,
@@ -281,6 +281,7 @@ void ConnectionID3::init(proto_t proto,
 
 	v6.port2=port2;
 
+    /*	
 	// Lookup up connection based on canonical ordering, which is
 	// the smaller of <src addr, src port> and <dst addr, dst port>
 	// followed by the other.
@@ -310,6 +311,18 @@ void ConnectionID3::init(proto_t proto,
 		    key.port1 = v6.port2;
 		    key.port2 = 0;
 		}
+	*/
+
+        in6_addr s6_ip;
+        in6_addr d6_ip;
+
+        memcpy(s6_ip.s6_addr, v6.ip1, 16);
+        memcpy(d6_ip.s6_addr, v6.ip2, 16);
+
+        key.ip1 = d6_ip;
+        key.ip2 = s6_ip;
+        key.port1 = v6.port2;
+        key.port2 = 0;
 
     init_hash_function();
 
@@ -321,7 +334,7 @@ void ConnectionID3::init(proto_t proto,
 
     delete newHashKey;
 
-    free_hash_function();
+    //free_hash_function();
 }
 
 void ConnectionID3::init6(proto_t proto,
@@ -344,6 +357,7 @@ void ConnectionID3::init6(proto_t proto,
 	//v.ip2=ip2;
 	v6.port2=port2;
 
+	/*
 	// Lookup up connection based on canonical ordering, which is
 	// the smaller of <src addr, src port> and <dst addr, dst port>
 	// followed by the other.
@@ -377,6 +391,21 @@ void ConnectionID3::init6(proto_t proto,
 		    key.port1 = v6.port2;
 		    key.port2 = 0;
 		}
+	*/
+		
+        in6_addr s6_ip;
+        in6_addr d6_ip;
+    
+        // setting s6_ip to have source address and d6_ip to have dest address
+        memcpy(s6_ip.s6_addr, ip1, 16);
+        memcpy(d6_ip.s6_addr, ip2, 16);
+
+        // setting key.ip1 to be dest address and key.ip2 to be source address
+        key.ip1 = d6_ip;
+        key.ip2 = s6_ip;
+        key.port1 = v6.port2;
+        key.port2 = 0;
+	
 
     init_hash_function();
 
@@ -388,7 +417,7 @@ void ConnectionID3::init6(proto_t proto,
 
     delete newHashKey;
 
-    free_hash_function();
+    //free_hash_function();
 }
 
 void ConnectionID2::init( uint32_t s_ip, uint32_t d_ip) {
@@ -469,7 +498,7 @@ void ConnectionID2::init( uint32_t s_ip, uint32_t d_ip) {
 
     delete newHashKey;
 
-    free_hash_function();
+    //free_hash_function();
 }
 
 void ConnectionID2::init6( unsigned char s_ip[], unsigned char d_ip[]) {
@@ -530,7 +559,7 @@ void ConnectionID2::init6( unsigned char s_ip[], unsigned char d_ip[]) {
 
     delete newHashKey;
 
-    free_hash_function();
+    //free_hash_function();
 }
 
 
diff --git a/src/Hash.cc b/src/Hash.cc
index 5e9a08a..f9be30a 100644
--- a/src/Hash.cc
+++ b/src/Hash.cc
@@ -30,12 +30,12 @@ void init_hash_function()
 	ASSERT(hmac_key_set);
 	h3 = new H3<hash_t, UHASH_KEY_SIZE>();
 	}
-
+/*
 void free_hash_function()
 {
     delete h3;
 }
-
+*/
 HashKey::HashKey(bro_int_t i)
 	{
 	key_u.i = i;
diff --git a/src/Hash.h b/src/Hash.h
index bc18434..c3762c9 100644
--- a/src/Hash.h
+++ b/src/Hash.h
@@ -88,6 +88,6 @@ protected:
 
 extern void init_hash_function();
 
-extern void free_hash_function();
+//extern void free_hash_function();
 
 #endif
diff --git a/src/IndexField.cc b/src/IndexField.cc
index e3af7a6..7190431 100644
--- a/src/IndexField.cc
+++ b/src/IndexField.cc
@@ -194,7 +194,7 @@ void IPAddress::Init(const std::string& s)
 
         hash_key = newHashKey->Hash();
 
-        free_hash_function();
+        //free_hash_function();
 		}
 
 	else
@@ -214,7 +214,7 @@ void IPAddress::Init(const std::string& s)
 
         hash_key = newHashKey->Hash();
 
-        free_hash_function();
+        //free_hash_function();
 	}
 
 std::string IPAddress::getStr() const
diff --git a/src/IndexField.hh b/src/IndexField.hh
index 5300ac6..dae0a81 100644
--- a/src/IndexField.hh
+++ b/src/IndexField.hh
@@ -97,7 +97,7 @@ public:
 
         delete newHashKey;
 
-        free_hash_function();
+        //free_hash_function();
     }
 /*
     IPAddress(unsigned char ip6[])
@@ -120,7 +120,7 @@ public:
 
         delete newHashKey;
 
-        free_hash_function();
+        //free_hash_function();
     }
 /*
     // The inet_addr() function converts the Internet host address cp from IPv4 
@@ -151,7 +151,7 @@ public:
 
         delete newHashKey;
 
-        free_hash_function();
+        //free_hash_function();
 	}
 	virtual ~IPAddress() {
         tmlog(TM_LOG_NOTE, "IPAddress", "deleting an ipaddress type");
@@ -365,7 +365,7 @@ public:
 
         delete newHashKey;
 
-        free_hash_function();
+        //free_hash_function();
     }
 	Port(uint16_t port): port(port) {  /* printf("Port(%u)\n", port); */
         init_hash_function();
@@ -376,7 +376,7 @@ public:
 
         delete newHashKey;
 
-        free_hash_function();
+        //free_hash_function();
 	}
 	virtual ~Port() {}
 
diff --git a/src/IndexHash.cc b/src/IndexHash.cc
index 33f79e8..07831f9 100644
--- a/src/IndexHash.cc
+++ b/src/IndexHash.cc
@@ -42,75 +42,78 @@ int IndexHash::clear() {
 	return count;
 }
 
-
 IndexEntry* IndexHash::lookup( IndexField* key) {
-	//IndexEntry *cur;
+/*
+    //IndexEntry *cur;
 
     tmlog(TM_LOG_NOTE, "idxhash", "checking that there is not a similar key for this timestamp: %f and info: %s", key->ts, key->getStr().c_str());
     //printf("This key has the following form: " + key->getStr() + "\n");
 
     //std::cout << "This key has the following form: " << key->getStr() << std::endl;
 
-	//cur = htable[key->hash()%numBuckets];
+    //cur = htable[key->hash()%numBuckets];
 
-	//cur = htable[key->getInt()%numBuckets];
+    //cur = htable[key->getInt()%numBuckets];
 
     tmlog(TM_LOG_NOTE, "idxhash", "the hash is: %u for this timestamp %f and form %s", key->hash(), key->ts, key->getStr().c_str());
     tmlog(TM_LOG_NOTE, "idxhash", "the index is: %u for this timestamp %f and form %s", key->hash()%numBuckets, key->ts, key->getStr().c_str());
     tmlog(TM_LOG_NOTE, "idxhash", "the number of buckets is %d for this timestampe %f and form %s", numBuckets, key->ts, key->getStr().c_str());
 
     // testing out the other method in add method to determine if this would help
-	IndexEntry *curalt;
-	int cmp;
-	curalt = troot;
-	cmp = 0;
+    IndexEntry *curalt;
+    int cmp;
+    curalt = troot;
+    cmp = 0;
 #ifdef TM_HEAVY_DEBUG
-	if (troot)
-		assert(troot->parent == NULL);
+    if (troot)
+        assert(troot->parent == NULL);
 #endif
 
     tmlog(TM_LOG_NOTE, "idx_hash: lookup", "the entry to add 'lookup' has key: %d", *(key->getConstKeyPtr()));
 
     tmlog(TM_LOG_NOTE, "idx_hash: lookup", "the entry to add 'lookup' has timestamp: %f", key->ts);
 
-	while (curalt) {
+    while (curalt) {
 #ifdef TM_HEAVY_DEBUG
-		assert(curalt->avlbal>=-1 && curalt->avlbal<=1);
+        assert(curalt->avlbal>=-1 && curalt->avlbal<=1);
 #endif
-		cmp = memcmp(key->getConstKeyPtr(), curalt->getKey()->getConstKeyPtr(), key->getKeySize());
-		if (cmp > 0)
-			curalt = curalt->left;
-		else if (cmp < 0)
-			curalt = curalt->right;
-		else {
+        cmp = memcmp(key->getConstKeyPtr(), curalt->getKey()->getConstKeyPtr(), key->getKeySize());
+        if (cmp > 0)
+            curalt = curalt->left;
+        else if (cmp < 0)
+            curalt = curalt->right;
+        else {
             tmlog(TM_LOG_NOTE, "idx_hash: lookup", "this is in lookup using the add method checker. the already existing entry is: %d\n", *(curalt->getKey()->getConstKeyPtr()));
-			tmlog(TM_LOG_NOTE, "idx_hash: lookup",  "this is in lookup using the add method checker. tried to insert an already existing entry into the tree. numEntries=%d\n",
-					getNumEntries());
+            tmlog(TM_LOG_NOTE, "idx_hash: lookup",  "this is in lookup using the add method checker. tried to insert an already existing entry into the tree. numEntries=%d\n",
+                    getNumEntries());
             break;
-			//h->add_or_update(key, ie);			
-		}
-	}
+            //h->add_or_update(key, ie);
+        }
+    }
 
     if (curalt == NULL)
         tmlog(TM_LOG_NOTE, "idxhash", "cur is NULL, which means that this entry is allegedly unique");
 
     return curalt;
+*/
 
-/*
-	while (cur != NULL) {
+    IndexEntry *cur;
+
+    cur = htable[key->hash()%numBuckets];
+
+    while (cur != NULL) {
         tmlog(TM_LOG_NOTE, "idxhash", "going through the keys: %d", *cur->getKey()->getConstKeyPtr());
-		if (*key == *cur->getKey()) {
+        if (*key == *cur->getKey()) {
         //if (*(key->getConstKeyPtr()) == *(cur->getKey()->getConstKeyPtr())) {
             tmlog(TM_LOG_NOTE, "idxhash", "the same key was found. the key is %d", *(key->getConstKeyPtr()));
-			break;
-		}
-		cur = cur->col_next;
-	}
+            break;
+        }
+        cur = cur->col_next;
+    }
     tmlog(TM_LOG_NOTE, "idx_hash", "this entry has key: %d", *(key->getConstKeyPtr()));
     if (cur == NULL)
         tmlog(TM_LOG_NOTE, "idxhash", "cur is NULL, which means that this entry is allegedly unique");
-	return cur;
-*/
+    return cur;
 }
 
 void IndexHash::add(IndexField *key, IndexEntry *ie) {
@@ -173,6 +176,15 @@ void IndexHash::add(IndexField *key, IndexEntry *ie) {
 	                htable[hvalconflict] = ie;
                     return;
 		        }
+			/*
+            if ( key->hash() == cur->getKey()->hash() &&
+                 !cmp )
+                {
+					cur->intlist = ie->intlist;
+                    delete ie;
+                    return;
+                }
+			*/
 
 		    if (cmp > 0)
 			    cur = cur->left;
@@ -180,13 +192,14 @@ void IndexHash::add(IndexField *key, IndexEntry *ie) {
 			    cur = cur->right;
 		    else 
             {
-                tmlog(TM_LOG_NOTE, "idx_hash", "the already existing entry is: %d\n", *(cur->getKey()->getConstKeyPtr()));
-			    tmlog(TM_LOG_NOTE, "idx_hash",  "tried to insert an already existing entry into the tree. numEntries=%d\n",
+                delete ie; 
+                //tmlog(TM_LOG_NOTE, "idx_hash", "the already existing entry is: %d\n", *(cur->getKey()->getConstKeyPtr()));
+			    tmlog(TM_LOG_ERROR, "idx_hash",  "tried to insert an already existing entry into the tree. numEntries=%d\n",
 					    getNumEntries());
 			    //h->add_or_update(key, ie);
-                delete ie;
+                //delete ie;
                 delete key;
-			    abort();
+			    //abort();
 			    return ; // entry exists, shouldn't happen	
 		    }
 	    }


From neto at caltech.edu  Sun Jul 27 22:15:49 2014
From: neto at caltech.edu (Naoki Eto)
Date: Sun, 27 Jul 2014 22:15:49 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: deleted two unnecessary files
	and made a change in IndexHash.cc to help avoid a segfault (cba1493)
Message-ID: <201407280515.s6S5Fnkv007735@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit cba1493bebbe530c76a0b4a4f1159edf28418ce7
Author: NaokiEto <neto at caltech.edu>
Date:   Mon Jul 28 01:03:23 2014 -0400

    deleted two unnecessary files and made a change in IndexHash.cc to
    help avoid a segfault


>---------------------------------------------------------------

cba1493bebbe530c76a0b4a4f1159edf28418ce7
 src/Desc.h       | 196 -----------------------------------------------
 src/IndexHash.cc |   2 +-
 src/Obj.h        | 227 -------------------------------------------------------
 3 files changed, 1 insertion(+), 424 deletions(-)

diff --git a/src/Desc.h b/src/Desc.h
deleted file mode 100644
index 44f22cb..0000000
--- a/src/Desc.h
+++ /dev/null
@@ -1,196 +0,0 @@
-// See the file "COPYING" in the main distribution directory for copyright.
-
-#ifndef descriptor_h
-#define descriptor_h
-
-#include <stdio.h>
-#include <set>
-#include <utility>
-
-//#include "BroString.h"
-
-typedef enum {
-	DESC_READABLE,
-	DESC_PORTABLE,
-	DESC_BINARY,
-} desc_type;
-
-typedef enum {
-	STANDARD_STYLE,
-	ALTERNATIVE_STYLE,
-	RAW_STYLE,
-} desc_style;
-
-class BroFile;
-class IPAddr;
-class IPPrefix;
-
-class ODesc {
-public:
-	ODesc(desc_type t=DESC_READABLE, BroFile* f=0);
-
-	~ODesc();
-
-	int IsReadable() const		{ return type == DESC_READABLE; }
-	int IsPortable() const		{ return type == DESC_PORTABLE; }
-	int IsBinary() const		{ return type == DESC_BINARY; }
-
-	int IsShort() const		{ return is_short; }
-	void SetShort()			{ is_short = 1; }
-	void SetShort(int s)		{ is_short = s; }
-
-	// Whether we want to have quotes around strings.
-	int WantQuotes() const	{ return want_quotes; }
-	void SetQuotes(int q)	{ want_quotes = q; }
-
-	// Whether we want to print statistics like access time and execution
-	// count where available.
-	int IncludeStats() const	{ return include_stats; }
-	void SetIncludeStats(int s)	{ include_stats = s; }
-
-	desc_style Style() const	{ return style; }
-	void SetStyle(desc_style s)	{ style = s; }
-
-	void SetFlush(int arg_do_flush)	{ do_flush = arg_do_flush; }
-
-	void EnableEscaping();
-	void AddEscapeSequence(const char* s) { escape_sequences.insert(s); }
-	void AddEscapeSequence(const char* s, size_t n)
-	    { escape_sequences.insert(string(s, n)); }
-	void AddEscapeSequence(const string & s)
-	    { escape_sequences.insert(s); }
-	void RemoveEscapeSequence(const char* s) { escape_sequences.erase(s); }
-	void RemoveEscapeSequence(const char* s, size_t n)
-	    { escape_sequences.erase(string(s, n)); }
-	void RemoveEscapeSequence(const string & s)
-	    { escape_sequences.erase(s); }
-
-	void PushIndent();
-	void PopIndent();
-	void PopIndentNoNL();
-	int GetIndentLevel() const	{ return indent_level; }
-	void ClearIndentLevel() { indent_level = 0; }
-
-	int IndentSpaces() const	{ return indent_with_spaces; }
-	void SetIndentSpaces(int i)	{ indent_with_spaces = i; }
-
-	void Add(const char* s, int do_indent=1);
-	void AddN(const char* s, int len)	{ AddBytes(s, len); }
-	void Add(const string& s)	{ AddBytes(s.data(), s.size()); }
-	void Add(int i);
-	void Add(uint32 u);
-	void Add(int64 i);
-	void Add(uint64 u);
-	void Add(double d);
-	void Add(const IPAddr& addr);
-	void Add(const IPPrefix& prefix);
-
-	// Add s as a counted string.
-	void AddCS(const char* s);
-
-	void AddBytes(const BroString* s);
-
-	void Add(const char* s1, const char* s2)
-		{ Add(s1); Add(s2); }
-
-	void AddSP(const char* s1, const char* s2)
-		{ Add(s1); AddSP(s2); }
-
-	void AddSP(const char* s )
-		{ Add(s); SP(); }
-
-	void AddCount(bro_int_t n)
-		{
-		if ( ! IsReadable() )
-			{
-			Add(n);
-			SP();
-			}
-		}
-
-	void SP()	{
-			if ( ! IsBinary() )
-				Add(" ", 0);
-			}
-	void NL()	{
-			if ( ! IsBinary() && ! is_short )
-				Add("\n", 0);
-			}
-
-	// Bypasses the escaping enabled via SetEscape().
-	void AddRaw(const char* s, int len)	{ AddBytesRaw(s, len); }
-	void AddRaw(const string &s)		{ AddBytesRaw(s.data(), s.size()); }
-
-	// Returns the description as a string.
-	const char* Description() const		{ return (const char*) base; }
-
-	const u_char* Bytes() const	{ return (const u_char *) base; }
-	byte_vec TakeBytes()
-		{
-		const void* t = base;
-		base = 0;
-		size = 0;
-
-		// Don't clear offset, as we want to still support
-		// subsequent calls to Len().
-
-		return byte_vec(t);
-		}
-
-	int Len() const		{ return offset; }
-
-	void Clear();
-
-protected:
-	void Indent();
-
-	void AddBytes(const void* bytes, unsigned int n);
-	void AddBytesRaw(const void* bytes, unsigned int n);
-
-	// Make buffer big enough for n bytes beyond bufp.
-	void Grow(unsigned int n);
-
-	/**
-	 * Returns the location of the first place in the bytes to be hex-escaped.
-	 *
-	 * @param bytes the starting memory address to start searching for
-	 *        escapable character.
-	 * @param n the maximum number of bytes to search.
-	 * @return a pair whose first element represents a starting memory address
-	 *         to be escaped up to the number of characters indicated by the
-	 *         second element.  The first element may be 0 if nothing is
-	 *         to be escaped.
-	 */
-	pair<const char*, size_t> FirstEscapeLoc(const char* bytes, size_t n);
-
-	/**
-	 * @param start start of string to check for starting with an espace
-	 *              sequence.
-	 * @param end one byte past the last character in the string.
-	 * @return The number of bytes in the escape sequence that the string
-	 *         starts with.
-	 */
-	size_t StartsWithEscapeSequence(const char* start, const char* end);
-
-	desc_type type;
-	desc_style style;
-
-	void* base;		// beginning of buffer
-	unsigned int offset;	// where we are in the buffer
-	unsigned int size;	// size of buffer in bytes
-
-	bool escape;	// escape unprintable characters in output?
-	typedef set<string> escape_set;
-	escape_set escape_sequences; // additional sequences of chars to escape
-
-	BroFile* f;	// or the file we're using.
-
-	int indent_level;
-	int is_short;
-	int want_quotes;
-	int do_flush;
-	int include_stats;
-	int indent_with_spaces;
-};
-
-#endif
diff --git a/src/IndexHash.cc b/src/IndexHash.cc
index 07831f9..d2a8326 100644
--- a/src/IndexHash.cc
+++ b/src/IndexHash.cc
@@ -198,7 +198,7 @@ void IndexHash::add(IndexField *key, IndexEntry *ie) {
 					    getNumEntries());
 			    //h->add_or_update(key, ie);
                 //delete ie;
-                delete key;
+                 // delete key;
 			    //abort();
 			    return ; // entry exists, shouldn't happen	
 		    }
diff --git a/src/Obj.h b/src/Obj.h
deleted file mode 100644
index be0d91b..0000000
--- a/src/Obj.h
+++ /dev/null
@@ -1,227 +0,0 @@
-// See the file "COPYING" in the main distribution directory for copyright.
-
-#ifndef obj_h
-#define obj_h
-
-#include <limits.h>
-
-#include "input.h"
-#include "Desc.h"
-#include "SerialObj.h"
-
-class Serializer;
-class SerialInfo;
-
-class Location : SerialObj {
-public:
-	Location(const char* fname, int line_f, int line_l, int col_f, int col_l)
-		{
-		filename = fname;
-		first_line = line_f;
-		last_line = line_l;
-		first_column = col_f;
-		last_column = col_l;
-		delete_data = false;
-
-		timestamp = 0;
-		text = 0;
-		}
-
-	Location()
-		{
-		filename = 0;
-		first_line = last_line = first_column = last_column = 0;
-		delete_data = false;
-		timestamp = 0;
-		text = 0;
-		}
-
-	virtual ~Location()
-		{
-		if ( delete_data )
-			delete [] filename;
-		}
-
-	void Describe(ODesc* d) const;
-
-	bool Serialize(SerialInfo* info) const;
-	static Location* Unserialize(UnserialInfo* info);
-
-	bool operator==(const Location& l) const;
-	bool operator!=(const Location& l) const
-		{ return ! (*this == l); }
-
-	const char* filename;
-	int first_line, last_line;
-	int first_column, last_column;
-	bool delete_data;
-
-	// Timestamp and text for compatibility with Bison's default yyltype.
-	int timestamp;
-	char* text;
-protected:
-	DECLARE_SERIAL(Location);
-};
-
-#define YYLTYPE yyltype
-typedef Location yyltype;
-YYLTYPE GetCurrentLocation();
-
-// Used to mean "no location associated with this object".
-extern Location no_location;
-
-// Current start/end location.
-extern Location start_location;
-extern Location end_location;
-
-// Used by parser to set the above.
-inline void set_location(const Location loc)
-	{
-	start_location = end_location = loc;
-	}
-
-inline void set_location(const Location start, const Location end)
-	{
-	start_location = start;
-	end_location = end;
-	}
-
-class BroObj : public SerialObj {
-public:
-	BroObj()
-		{
-		ref_cnt = 1;
-		in_ser_cache = false;
-
-		// A bit of a hack.  We'd like to associate location
-		// information with every object created when parsing,
-		// since for them, the location is generally well-defined.
-		// We could maintain a separate flag that tells us whether
-		// we're inside a parse, but the parser also sets the
-		// location to no_location when it's done, so it makes
-		// sense to just check for that.  *However*, start_location
-		// and end_location are maintained as their own objects
-		// rather than pointers or references, so we can't directly
-		// check them for equality with no_location.  So instead
-		// we check for whether start_location has a line number
-		// of 0, which should only happen if it's been assigned
-		// to no_location (or hasn't been initialized at all).
-		location = 0;
-		if ( start_location.first_line != 0 )
-			SetLocationInfo(&start_location, &end_location);
-		}
-
-	virtual ~BroObj();
-
-	// Report user warnings/errors.  If obj2 is given, then it's
-	// included in the message, though if pinpoint_only is non-zero,
-	// then obj2 is only used to pinpoint the location.
-	void Warn(const char* msg, const BroObj* obj2 = 0,
-			int pinpoint_only = 0) const;
-	void Error(const char* msg, const BroObj* obj2 = 0,
-			int pinpoint_only = 0) const;
-
-	// Report internal errors.
-	void BadTag(const char* msg, const char* t1 = 0,
-			const char* t2 = 0) const;
-#define CHECK_TAG(t1, t2, text, tag_to_text_func) \
-	{ \
-	if ( t1 != t2 ) \
-		BadTag(text, tag_to_text_func(t1), tag_to_text_func(t2)); \
-	}
-
-	void Internal(const char* msg) const;
-	void InternalWarning(const char* msg) const;
-
-	virtual void Describe(ODesc* d) const { /* FIXME: Add code */ };
-
-	void AddLocation(ODesc* d) const;
-
-	// Get location info for debugging.
-	const Location* GetLocationInfo() const
-		{ return location ? location : &no_location; }
-
-	virtual bool SetLocationInfo(const Location* loc)
-		{ return SetLocationInfo(loc, loc); }
-
-	// Location = range from start to end.
-	virtual bool SetLocationInfo(const Location* start, const Location* end);
-
-	// Set new end-of-location information.  This is used to
-	// extend compound objects such as statement lists.
-	virtual void UpdateLocationEndInfo(const Location& end);
-
-	int RefCnt() const	{ return ref_cnt; }
-
-	// Helper class to temporarily suppress errors
-	// as long as there exist any instances.
-	class SuppressErrors {
-	public:
-		SuppressErrors()	{ ++BroObj::suppress_errors; }
-		~SuppressErrors()	{ --BroObj::suppress_errors; }
-	};
-
-	bool in_ser_cache;
-
-protected:
-	friend class SerializationCache;
-
-	DECLARE_ABSTRACT_SERIAL(BroObj);
-
-	Location* location;	// all that matters in real estate
-
-private:
-	friend class SuppressErrors;
-
-	void DoMsg(ODesc* d, const char s1[], const BroObj* obj2 = 0,
-			int pinpoint_only = 0) const;
-	void PinPoint(ODesc* d, const BroObj* obj2 = 0,
-			int pinpoint_only = 0) const;
-
-	friend inline void Ref(BroObj* o);
-	friend inline void Unref(BroObj* o);
-
-	int ref_cnt;
-
-	// If non-zero, do not print runtime errors.  Useful for
-	// speculative evaluation.
-	static int suppress_errors;
-};
-
-// Prints obj to stderr, primarily for debugging.
-extern void print(const BroObj* obj);
-
-extern void bad_ref(int type);
-
-// Sometimes useful when dealing with BroObj subclasses that have their
-// own (protected) versions of Error.
-inline void Error(const BroObj* o, const char* msg)
-	{
-	o->Error(msg);
-	}
-
-inline void Ref(BroObj* o)
-	{
-	if ( ++o->ref_cnt <= 1 )
-		bad_ref(0);
-	if ( o->ref_cnt == INT_MAX )
-		bad_ref(1);
-	}
-
-inline void Unref(BroObj* o)
-	{
-	if ( o && --o->ref_cnt <= 0 )
-		{
-		if ( o->ref_cnt < 0 )
-			bad_ref(2);
-		delete o;
-
-		// We could do the following if o were passed by reference.
-		// o = (BroObj*) 0xcd;
-		}
-	}
-
-// A dict_delete_func that knows to Unref() dictionary entries.
-extern void bro_obj_delete_func(void* v);
-
-#endif


From neto at caltech.edu  Mon Jul 28 05:12:22 2014
From: neto at caltech.edu (Naoki Eto)
Date: Mon, 28 Jul 2014 05:12:22 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: added some gperftools
	capabilities (taken from bro code) (da9bca8)
Message-ID: <201407281212.s6SCCM5v010230@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit da9bca81044346cf75500dfc18cf848dffae8d16
Author: NaokiEto <neto at caltech.edu>
Date:   Mon Jul 28 07:59:49 2014 -0400

    added some gperftools capabilities (taken from bro code)


>---------------------------------------------------------------

da9bca81044346cf75500dfc18cf848dffae8d16
 CMakeLists.txt         |  36 +++
 src/pcapnav/pcapnav.c~ | 716 -------------------------------------------------
 2 files changed, 36 insertions(+), 716 deletions(-)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 7f8d305..0f688b5 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -28,6 +28,33 @@ if (BROCCOLI_FOUND)
     list(APPEND OPTLIBS ${Broccoli_LIBRARY})
 endif ()
 
+set(HAVE_PERFTOOLS false)
+set(USE_PERFTOOLS_DEBUG false)
+set(USE_PERFTOOLS_TCMALLOC false)
+
+if (NOT DISABLE_PERFTOOLS)
+   find_package(GooglePerftools)
+endif ()
+
+if (GOOGLEPERFTOOLS_FOUND)
+    set(HAVE_PERFTOOLS true)
+    # Non-Linux systems may not be well-supported by gperftools, so
+    # require explicit request from user to enable it in that case.
+    if (${CMAKE_SYSTEM_NAME} MATCHES "Linux" OR ENABLE_PERFTOOLS)
+        set(USE_PERFTOOLS_TCMALLOC true)
+
+        if (ENABLE_PERFTOOLS_DEBUG)
+            # Enable heap debugging with perftools.
+            set(USE_PERFTOOLS_DEBUG true)
+            include_directories(BEFORE ${GooglePerftools_INCLUDE_DIR})
+            list(APPEND OPTLIBS ${GooglePerftools_LIBRARIES_DEBUG})
+        else ()
+            # Link in tcmalloc for better performance.
+            list(APPEND OPTLIBS ${GooglePerftools_LIBRARIES})
+        endif ()
+    endif ()
+endif ()
+
 if (MISSING_PREREQS)
     foreach (prereq ${MISSING_PREREQ_DESCS})
         message(SEND_ERROR ${prereq})
@@ -35,6 +62,12 @@ if (MISSING_PREREQS)
     message(FATAL_ERROR "Configuration aborted due to missing prerequisites")
 endif ()
 
+if (ENABLE_PERFTOOLS_DEBUG OR ENABLE_PERFTOOLS)
+    # Just a no op to prevent CMake from complaining about manually-specified
+    # ENABLE_PERFTOOLS_DEBUG or ENABLE_PERFTOOLS not being used if google
+    # perftools weren't found
+endif ()
+
 set(CMAKE_CXX_FLAGS "-g -Wall")
 
 #detect 32 or 64 bit compiler
@@ -99,6 +132,9 @@ message(
     "\nCPP:               ${CMAKE_CXX_COMPILER}"
     "\n"
     "\nBroccoli:          ${USE_BROCCOLI}"
+    "\ngperftools found:  ${HAVE_PERFTOOLS}"
+    "\n        tcmalloc:  ${USE_PERFTOOLS_TCMALLOC}"
+    "\n       debugging:  ${USE_PERFTOOLS_DEBUG}"
     "\n================================================================\n"
 )
 
diff --git a/src/pcapnav/pcapnav.c~ b/src/pcapnav/pcapnav.c~
deleted file mode 100644
index 43781a9..0000000
--- a/src/pcapnav/pcapnav.c~
+++ /dev/null
@@ -1,716 +0,0 @@
-/*
-
-Copyright (C) 2002 - 2007 Christian Kreibich
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to
-deal in the Software without restriction, including without limitation the
-rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
-sell copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in
-all copies of the Software and its documentation and acknowledgment shall be
-given in the documentation and software packages that this Software was
-used.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
-IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
-CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
-
-*/
-#if HAVE_CONFIG_H
-# include <config.h>
-#endif
-
-#include <stdlib.h>
-#include <stdio.h>
-#include <string.h>
-#include <math.h>
-#include <errno.h>
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <unistd.h>
-
-#include "pcapnav_globals.c"
-
-#include "pcapnav_private.h"
-#include "pcapnav_header.h"
-#include "pcapnav_append.h"
-#include "pcapnav_macros.h"
-#include "pcapnav_trace.h"
-#include "pcapnav_util.h"
-#include "pcapnav_debug.h"
-
-
-/* NOTE NOTE NOTE -- offsets to the user of this library are relative
- * to the *END* of the pcap file header. Hence, offset 0 means the first
- * byte after the pcap file header. This makes offset calculation
- * easier for the user, because usually the user doesn't care about
- * the trace file header when handling offsets in a trace file.
- * We thus save the user the error-prone addition/subtraction of
- * sizeof(struct pcap_file_header) all the time.
- */
-
-static char pcap_errbuf[PCAP_ERRBUF_SIZE];
-
-// mentioned in pcapnav.h and pcapnav_debug.c
-// according to netdude online,
-// debug: enables debugging output when set to a value >= 1, and disables it when set to 0. Initialially, it is disabled.
-// calldepth_limit: you can limit the calldepth up to which debugging output is displayed, to avoid excessive logging. By default, everything is logged (loglevel 0) O.o
-void
-pcapnav_init(void)
-{
-  pcapnav_runtime_options.debug = 0;
-  pcapnav_runtime_options.calldepth_limit = -1;
-}
-
-
-pcapnav_t      *
-pcapnav_open_offline(const char *fname)
-{
-  pcapnav_t               *pn;
-  u_int32_t                magic;
-  struct pcap_file_header *filehdr = NULL;
-  FILE                    *fp;
-  struct stat              st;
-
-  D_ENTER;
-
-  if (!fname || fname[0] == '\0')
-    {
-      D(("Invalid filename: %s\n", fname));
-      errno = ENOENT;
-      D_RETURN_(NULL);
-    }
-
-  /* Allocate new pcapnav structure and initialize. */
-  
-  if (! (pn = NEW(pcapnav_t)))
-    {
-      D(("Out of memory.\n"));
-      errno = ENOMEM;
-      D_RETURN_(NULL);
-    }
-
-  if (lstat(fname, &st) < 0)
-    {
-      D(("lstat failed: %s\n", strerror(errno)));
-      goto free_return;
-    }
-
-  pn->size = st.st_size;
-
-  /* Allocate pcap handle */
-  if (! (pn->pcap = pcap_open_offline(fname, pcap_errbuf)))
-    {
-      D(("%s (from pcap, re. %s)\n", pcap_errbuf, fname));
-      /* Let's hope errno is meaningful now ... */
-      goto free_return;
-    }
-  
-  /* Hook pcap's file stream into our own structure: */
-  pn->fp = pcap_file(pn->pcap);
-
-  if ((fp = fopen(fname, "r")) == NULL)
-    {
-      D(("Could not open trace file %s for reading.\n", fname));
-      /* errno set already */
-      goto free_return;
-    }
-
-  if (fread((char *)&pn->trace.filehdr, sizeof(struct pcap_file_header), 1, fp) != 1)
-    {
-      D(("Could not read trace file header from %s\n", fname));
-      /* errno set already */
-      goto cleanup_return;
-    }
-
-  /* Look at magic to determine byte order. */
-
-  magic = pn->trace.filehdr.magic;
-  filehdr = &pn->trace.filehdr;      
-
-  if (magic != TCPDUMP_MAGIC && magic != PATCHED_TCPDUMP_MAGIC)
-    {
-      magic = SWAPLONG(magic);
-
-      if (magic != TCPDUMP_MAGIC && magic != PATCHED_TCPDUMP_MAGIC)
-	{
-	  D(("Invalid trace file %s -- didn't recognize file magic.\n", fname));
-	  goto cleanup_return;
-	}
-
-      pn->trace.swapped = TRUE;
-
-      filehdr->version_major = SWAPSHORT(filehdr->version_major);
-      filehdr->version_minor = SWAPSHORT(filehdr->version_minor);
-      filehdr->thiszone = SWAPLONG(filehdr->thiszone);
-      filehdr->sigfigs = SWAPLONG(filehdr->sigfigs);
-      filehdr->snaplen = SWAPLONG(filehdr->snaplen);
-      filehdr->linktype = SWAPLONG(filehdr->linktype);      
-    }
-  
-  /* Store the size of the pcap packet header, *as* *on* *disk*,
-   * in the handle, for convenience. It must be as on disk to
-   * work on both 64- and 32-bit architectures.
-   */
-  if (magic == PATCHED_TCPDUMP_MAGIC)
-    /*
-     * XXX - the patch that's in some versions of libpcap
-     * changes the packet header but not the magic number;
-     * we'd have to use some hacks^H^H^H^H^Hheuristics to
-     * detect that.
-     */
-    pn->trace.pkthdr_size = sizeof(struct pcapnav_patched_pkthdr);
-  else
-    pn->trace.pkthdr_size = sizeof(struct pcapnav_pkthdr);
-
-  pn->chain_buf = __pcapnav_buf_new(MAX_PACKET_SIZE(pn) *
-				    MAX_CHAIN_LENGTH);
-  if (!pn->chain_buf)
-    goto cleanup_return;
-
-  pn->search_buf = __pcapnav_buf_new(MAX_BYTES_FOR_DEFINITE_HEADER(pn));
-  if (!pn->search_buf)
-    goto cleanup_return;
-
-  /* Get length of file: */
-  if (fseek(fp, 0, SEEK_END) != 0)
-    {
-      D(("Couldn't determine file length, fseek() failed: %s\n", strerror(errno)));
-      goto cleanup_return;
-    }
-
-  if ((pn->trace.length = ftell(fp)) < 0)
-    {
-      D(("Couldn't determine file length, ftell() failed: %s\n", strerror(errno)));
-      goto cleanup_return;
-    }
-
-  fclose(fp);  
-  D_RETURN_(pn);
-  
- cleanup_return:
-  fclose(fp);
-  
- free_return:
-  FREE(pn);
-  D_RETURN_(NULL);
-}
-
-
-void	        
-pcapnav_close(pcapnav_t *pn)
-{
-  D_ENTER;
-
-  if (!pn)
-    D_RETURN;
-
-  if (pn->pcap)
-    pcap_close(pn->pcap);
-
-  /* No need to flclose(pn->fp) -- we stole it from pcap */
-
-  __pcapnav_buf_free(pn->search_buf);
-  __pcapnav_buf_free(pn->chain_buf);
-
-  FREE(pn);
-  D_RETURN;
-}
-
-
-int                
-pcapnav_get_pkthdr_size(pcapnav_t *pn)
-{
-  if (!pn)
-    return 0;
-
-  return pn->trace.pkthdr_size;
-}
-
-
-const struct pcap_file_header *
-pcapnav_get_file_header(pcapnav_t *pn)
-{
-  if (!pn)
-    return 0;
-
-  return (const struct pcap_file_header *) &pn->trace.filehdr;
-}
-
-
-off_t              
-pcapnav_get_offset(pcapnav_t *pn)
-{
-  if (!pn)
-    return 0;
-
-  return ftell(pcap_file(pn->pcap)) - sizeof(struct pcap_file_header);
-}
-
-
-int
-pcapnav_set_offset(pcapnav_t *pn, off_t offset)
-{
-  off_t result;
-
-  D_ENTER;
-  
-  if (!pn)
-    D_RETURN_(-1);
-
-  result = fseek(pcap_file(pn->pcap), offset + sizeof(struct pcap_file_header), SEEK_SET);
-  D_RETURN_(result < 0 ? -1 : 0);
-}
-
-
-void               
-pcapnav_get_timestamp(pcapnav_t *pn, struct bpf_timeval *tv)
-{
-  off_t position;
-  struct pcap_pkthdr header;
-
-  D_ENTER;
-
-  if (!pn || !tv)
-    D_RETURN;
-
-  position = pcapnav_get_offset(pn);
-  memset(tv, 0, sizeof(struct bpf_timeval));
-  
-  if (pcapnav_next(pn, &header))
-    *tv = header.ts;
-  
-  pcapnav_set_offset(pn, position);
-  D_RETURN;
-}
-
-
-pcap_t            *
-pcapnav_pcap(pcapnav_t *pn)
-{
-  if (!pn)
-    return NULL;
-
-  return pn->pcap;
-}
-
-
-
-int	        
-pcapnav_loop(pcapnav_t *pn, int num_packets,
-	     pcap_handler callback, u_char *user_data)
-{
-  int n;
-
-  D_ENTER;
-
-  if (!pn || !callback)
-    {
-      D(("Invalid input.\n"));
-      D_RETURN_(0);
-    }
-  
-  n = pcap_loop(pn->pcap, num_packets, callback, user_data);
-  D_RETURN_(n);
-}
-
-
-int	        
-pcapnav_dispatch(pcapnav_t *pn, int num_packets,
-		 pcap_handler callback, u_char *user_data)
-{
-  if (!pn || !callback)
-    {
-      D(("Invalid input.\n"));
-      return 0;
-    }
-
-  return pcap_dispatch(pn->pcap, num_packets, callback, user_data);
-}
-
-
-const u_char      *
-pcapnav_next(pcapnav_t *pn, struct pcap_pkthdr *header)
-{
-  const u_char * result;
-  struct pcap_pkthdr dummy;
-
-  D_ENTER;
-
-  if (!pn)
-    D_RETURN_(NULL);
-
-  if (!header)
-    header = &dummy;
-
-  result = pcap_next(pn->pcap, header);
-  D_RETURN_(result);
-}
-
-
-int                
-pcapnav_has_next(pcapnav_t *pn)
-{
-  int result = 1;
-  off_t position;
-
-  D_ENTER;
-
-  if (!pn)
-    D_RETURN_(0);
-
-  position = pcapnav_get_offset(pn);
-  
-  if (!pcapnav_next(pn, NULL))
-    D_RETURN_(0);
-  
-  pcapnav_set_offset(pn, position);
-
-  D_RETURN_(result);
-}
-
-
-pcapnav_result_t
-pcapnav_goto_timestamp(pcapnav_t *pn, struct bpf_timeval *timestamp)
-{
-  pcapnav_result_t result;
-
-  D_ENTER;
-  
-  if (!pn || !timestamp)
-    D_RETURN_(PCAPNAV_ERROR);
-  
-  /* Make sure timespan of trace is in pn: */
-  pcapnav_get_timespan(pn, NULL, NULL);
-  result = __pcapnav_trace_find_packet_at_timestamp(pn, timestamp);
-  
-  D_RETURN_(result);
-}
-
-
-pcapnav_result_t             
-pcapnav_goto_fraction(pcapnav_t *pn, double fraction)
-{
-  pcapnav_result_t result;
-  off_t offset = 0;
-
-  D_ENTER;
-  
-  if (!pn)
-    D_RETURN_(PCAPNAV_ERROR);
-
-  /* Make sure span of trace is in pn: */
-  pcapnav_get_timespan(pn, NULL, NULL);
-  
-  /* Assert 0 <= fraction <= 1: */
-  
-  if (fraction > 1.0)
-    fraction = 1.0;
-
-  if (fraction < 0.0)
-    fraction = 0.0;
-  
-  offset = (pn->end_offset - pn->start_offset) * fraction;
-  result = __pcapnav_trace_find_packet_at_offset(pn, offset, PCAPNAV_CMP_ANY);
-  
-  D_RETURN_(result);
-}
-
-
-pcapnav_result_t             
-pcapnav_goto_offset(pcapnav_t *pn, off_t offset, pcapnav_cmp_t boundary)
-{
-  pcapnav_result_t result;
-
-  D_ENTER;
-
-  if (!pn)
-    D_RETURN_(PCAPNAV_ERROR);
-
-  /* Make sure span of trace is in pn: */
-  pcapnav_get_timespan(pn, NULL, NULL);
-  result = __pcapnav_trace_find_packet_at_offset(pn, offset, boundary);
-  
-  D_RETURN_(result);
-}
-
-
-int             
-pcapnav_get_timespan(pcapnav_t *pn, struct bpf_timeval *start, struct bpf_timeval *end)
-{
-  D_ENTER;
-
-  if (!pn)
-    D_RETURN_(-1);
-
-  if ((pn->start_time.tv_sec == 0)  &&
-      (pn->start_time.tv_usec == 0) &&
-      (pn->end_time.tv_sec == 0)    &&
-      (pn->end_time.tv_usec == 0))
-    {
-      /* We have not yet looked up the timespan and offsets
-       * of the trace file already. */
-
-      __pcapnav_trace_find_start(pn);
-      __pcapnav_trace_find_end(pn);
-    }
-  
-  if (start)
-    *start = pn->start_time;
-  
-  if (end)
-    *end = pn->end_time;
-
-  D_RETURN_(0);
-}
-
-
-off_t
-pcapnav_get_span(pcapnav_t *pn)
-{
-  off_t result;
-
-  D_ENTER;
-
-  if (!pn)
-    D_RETURN_(0);
-
-  if ((pn->start_time.tv_sec == 0)  &&
-      (pn->start_time.tv_usec == 0) &&
-      (pn->end_time.tv_sec == 0)    &&
-      (pn->end_time.tv_usec == 0))
-    {
-      /* We have not yet looked up the timespan and offsets of the trace file. */
-
-      __pcapnav_trace_find_start(pn);
-      __pcapnav_trace_find_end(pn);
-    }
-  
-  result = pn->end_offset - pn->start_offset;
-  D_RETURN_(result);
-}
-
-
-off_t
-pcapnav_get_size(pcapnav_t *pn)
-{
-  if (!pn)
-    return 0;
-
-  return pn->size - sizeof(struct pcap_file_header);
-}
-
-
-void
-pcapnav_timeval_init(struct bpf_timeval *tv,
-		     int sec, int usec)
-{
-  if (! tv)
-    return;
-
-  memset(tv, 0, sizeof(struct bpf_timeval)); /* Let's be thorough. */
-  tv->tv_sec = sec;
-  tv->tv_usec = usec;
-}
-
-
-int
-pcapnav_timeval_cmp(const struct bpf_timeval *tv1,
-		    const struct bpf_timeval *tv2)
-{
-  if (!tv1 || !tv2)
-    return 0;
-
-  if (tv1->tv_sec < tv2->tv_sec)
-    return -1;
-
-  if (tv1->tv_sec > tv2->tv_sec)
-    return 1;
-
-  if (tv1->tv_usec < tv2->tv_usec)
-    return -1;
-
-  if (tv1->tv_usec > tv2->tv_usec)
-    return 1;
-  
-  return 0;
-}
-
-
-void               
-pcapnav_timeval_sub(const struct bpf_timeval *tv1,
-		    const struct bpf_timeval *tv2,
-		    struct bpf_timeval *tv_out)
-{
-  __pcapnav_util_timeval_sub(tv1, tv2, tv_out);
-}
-
-
-void               
-pcapnav_timeval_add(const struct bpf_timeval *tv1,
-		    const struct bpf_timeval *tv2,
-		    struct bpf_timeval *tv_out)
-{
-  __pcapnav_util_timeval_add(tv1, tv2, tv_out);
-}
-
-
-double             
-pcapnav_get_time_fraction(pcapnav_t *pn,
-			  const struct bpf_timeval *tv)
-{
-  off_t offset;
-  struct pcap_pkthdr hdr;
-  double full_span, current_span, fraction, result;
-
-  D_ENTER;
-
-  if (!pn)
-    D_RETURN_(0.0);
-
-  /* Make sure timestamps are initialized */
-
-  pcapnav_get_timespan(pn, NULL, NULL);
-  full_span = __pcapnav_util_timeval_diff(&pn->end_time, &pn->start_time);
-
-  if (!tv)
-    {
-      struct pcap_pkthdr hdr2;
-
-      /* Obtain current packet header by reading one packet
-       * and then rewinding back to original position. */
-      
-      if ((offset = ftell(pn->fp)) < 0)
-	D_RETURN_(0.0);
-      
-      if (fread((void *) &hdr, sizeof(struct pcap_pkthdr), 1, pn->fp) != 1)
-	D_RETURN_(0.0);
-      
-      if (fseek(pn->fp, offset, SEEK_SET) < 0)
-	{
-	  D(("fseek() failed: %s\n", strerror(errno)));
-	  D_RETURN_(0.0);
-	}
-
-      __pcapnav_header_extract(pn, (u_char*) &hdr, &hdr2);
-      current_span = __pcapnav_util_timeval_diff(&hdr2.ts, &pn->start_time);
-    }
-  else
-    {
-      current_span = __pcapnav_util_timeval_diff(tv, &pn->start_time);
-    }
-
-  fraction = current_span / full_span;
-
-  if (fraction < 0.0)
-    fraction = 0.0;
-
-  if (fraction > 1.0)
-    fraction = 1.0;
-
-  result = fabs(fraction);
-  D_RETURN_(result);
-}
-
-
-double             
-pcapnav_get_space_fraction(pcapnav_t *pn, off_t offset)
-{
-  double fraction, result;
-  
-  D_ENTER;
-
-  if (!pn || offset == 0)
-    D_RETURN_(0.0);
-
-  /* Make sure timestamps + offsets are initialized */
-
-  pcapnav_get_timespan(pn, NULL, NULL);
-
-  fraction =
-    ((double)(offset)) /
-    (pn->end_offset - pn->start_offset);
-
-  if (fraction < 0.0)
-    fraction = 0.0;
-
-  if (fraction > 1.0)
-    fraction = 1.0;
-
-  result = fabs(fraction);
-  D_RETURN_(fraction);
-}
-
-
-int
-pcapnav_get_current_timestamp(pcapnav_t *pn, struct bpf_timeval *tv)
-{
-  struct pcap_pkthdr hdr;
-  off_t position;
-
-  D_ENTER;
-
-  if (!pn || !tv)
-    D_RETURN_(0);
-  
-  position = pcapnav_get_offset(pn);
-  
-  if (fread((void *) &hdr, sizeof(struct pcap_pkthdr), 1, pn->fp) != 1)
-    {
-      pcapnav_set_offset(pn, position);
-      D_RETURN_(0);
-    }
-
-  *tv = hdr.ts;  
-  pcapnav_set_offset(pn, position);
-
-  D_RETURN_(1);
-}
-
-
-char *
-pcapnav_geterr(pcapnav_t *pn)
-{
-  return pcap_geterr(pn->pcap);
-}
-
-
-pcap_dumper_t *
-pcapnav_dump_open(pcap_t *pcap, const char *filename, pcapnav_dumpmode_t mode)
-{
-  if (!pcap)
-    {
-      D(("Input error.\n"));
-      return NULL;
-    }
-
-  /* If the user requests standard output, just pass
-   * through to pcap.
-   */
-  if (filename[0] == '-' && filename[1] == '\0')
-    {
-      D(("Passing through to pcap_dump_open().\n"));
-      return pcap_dump_open(pcap, filename);
-    }
-  
-  switch (mode)
-    {
-    case PCAPNAV_DUMP_APPEND_FAST:
-      return pcapnav_append_fast(pcap, filename);
-      
-    case PCAPNAV_DUMP_APPEND_SAFE:
-      return pcapnav_append_safe(pcap, filename);
-
-    case PCAPNAV_DUMP_TRUNC:
-    default:      
-      return pcap_dump_open(pcap, filename);
-    }
-  
-  return NULL;
-}


From neto at caltech.edu  Mon Jul 28 09:46:33 2014
From: neto at caltech.edu (Naoki Eto)
Date: Mon, 28 Jul 2014 09:46:33 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: added the foundation in the
	CMake for cpu profiling via gperftools (9646fd4)
Message-ID: <201407281646.s6SGkXjQ013107@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit 9646fd42c97a6f76ca62743d17e971d933140cb5
Author: NaokiEto <neto at caltech.edu>
Date:   Mon Jul 28 12:34:02 2014 -0400

    added the foundation in the CMake for cpu profiling via gperftools


>---------------------------------------------------------------

9646fd42c97a6f76ca62743d17e971d933140cb5
 CMakeLists.txt | 13 ++++++++++++-
 configure      | 16 ++++++++++++++++
 2 files changed, 28 insertions(+), 1 deletion(-)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 0f688b5..e718977 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -31,6 +31,7 @@ endif ()
 set(HAVE_PERFTOOLS false)
 set(USE_PERFTOOLS_DEBUG false)
 set(USE_PERFTOOLS_TCMALLOC false)
+set(USE_PERFTOOLS_CPU false)
 
 if (NOT DISABLE_PERFTOOLS)
    find_package(GooglePerftools)
@@ -52,6 +53,15 @@ if (GOOGLEPERFTOOLS_FOUND)
             # Link in tcmalloc for better performance.
             list(APPEND OPTLIBS ${GooglePerftools_LIBRARIES})
         endif ()
+        
+        if (ENABLE_PERFTOOLS_CPU)
+            # Enable CPU usage debugging with perftools.
+            set(USE_PERFTOOLS_CPU true)
+            include_directories(BEFORE ${GooglePerftools_INCLUDE_CPU})
+            list(APPEND OPTLIBS ${GooglePerftools_LIBRARIES_CPU})
+        else ()
+            list(APPEND OPTLIBS ${GooglePerftools_LIBRARIES})
+        endif ()
     endif ()
 endif ()
 
@@ -62,7 +72,7 @@ if (MISSING_PREREQS)
     message(FATAL_ERROR "Configuration aborted due to missing prerequisites")
 endif ()
 
-if (ENABLE_PERFTOOLS_DEBUG OR ENABLE_PERFTOOLS)
+if (ENABLE_PERFTOOLS_CPU OR ENABLE_PERFTOOLS_DEBUG OR ENABLE_PERFTOOLS)
     # Just a no op to prevent CMake from complaining about manually-specified
     # ENABLE_PERFTOOLS_DEBUG or ENABLE_PERFTOOLS not being used if google
     # perftools weren't found
@@ -135,6 +145,7 @@ message(
     "\ngperftools found:  ${HAVE_PERFTOOLS}"
     "\n        tcmalloc:  ${USE_PERFTOOLS_TCMALLOC}"
     "\n       debugging:  ${USE_PERFTOOLS_DEBUG}"
+    "\n     cpu_profile:  ${USE_PERFTOOLS_CPU}"
     "\n================================================================\n"
 )
 
diff --git a/configure b/configure
index 421a221..eaf9b7a 100755
--- a/configure
+++ b/configure
@@ -58,6 +58,9 @@ prefix=/usr/local
 CMakeCacheEntries=""
 append_cache_entry CMAKE_INSTALL_PREFIX PATH   $prefix
 append_cache_entry ENABLE_DEBUG         BOOL   false
+append_cache_entry ENABLE_PERFTOOLS     BOOL   false
+append_cache_entry ENABLE_PERFTOOLS_DEBUG BOOL false
+append_cache_entry ENABLE_PERFTOOLS_CPU BOOL   false
 
 # parse arguments
 while [ $# -ne 0 ]; do
@@ -85,6 +88,19 @@ while [ $# -ne 0 ]; do
         --enable-debug)
             append_cache_entry ENABLE_DEBUG         BOOL   true
             ;;
+        --enable-perftools)
+            append_cache_entry ENABLE_PERFTOOLS     BOOL   true
+            ;;
+        --enable-perftools-debug)
+            append_cache_entry ENABLE_PERFTOOLS     BOOL   true
+            append_cache_entry ENABLE_PERFTOOLS_DEBUG BOOL   true
+            ;;
+        --enable-perftools-cpu)
+            append_cache_entry ENABLE_PERFTOOLS_CPU BOOL true
+            ;;
+        --disable-perftools)
+            append_cache_entry DISABLE_PERFTOOLS    BOOL   true
+            ;;
         --with-broccoli=*)
             append_cache_entry Broccoli_ROOT_DIR PATH $optarg
             ;;


From neto at caltech.edu  Mon Jul 28 11:11:17 2014
From: neto at caltech.edu (Naoki Eto)
Date: Mon, 28 Jul 2014 11:11:17 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: saving my work to try to get rid
	of permissions (070782b)
Message-ID: <201407281811.s6SIBH9s013708@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit 070782ba3b1e47ce9621d198305009a2cb5dfa28
Author: NaokiEto <neto at caltech.edu>
Date:   Mon Jul 28 13:59:01 2014 -0400

    saving my work to try to get rid of permissions


>---------------------------------------------------------------

070782ba3b1e47ce9621d198305009a2cb5dfa28
 src/Index.cc | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/src/Index.cc b/src/Index.cc
index cf9bdbb..9494763 100644
--- a/src/Index.cc
+++ b/src/Index.cc
@@ -11,6 +11,7 @@
 
 #include <fstream>
 #include <queue>
+#include <gperftools/profiler.h>
 
 #include "tm.h"
 #include "types.h"
@@ -282,6 +283,7 @@ void Index<T>::addEntry(IndexField *iqe) {
     // if it is not in the Hash table (== NULL)
 	if (ie==NULL) {
 
+        ProfilerStart("IndexHash_add");
 
         tmlog(TM_LOG_DEBUG, "addEntry", "beginning to add entry with timestamp %f and form %s", iqe->ts, iqe->getStr().c_str());
 
@@ -295,6 +297,8 @@ void Index<T>::addEntry(IndexField *iqe) {
 		cur->add(iqe, ie_n);
 
     	last_updated = iqe->ts;
+
+        ProfilerStop();
 	} else {
         tmlog(TM_LOG_NOTE, "addEntry", "updating the entry with timestamp %f and form %s", iqe->ts, iqe->getStr().c_str());
         // the entry is already in the hash table


From neto at caltech.edu  Mon Jul 28 14:30:40 2014
From: neto at caltech.edu (Naoki Eto)
Date: Mon, 28 Jul 2014 14:30:40 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: updated the foundation for the
	gperftools, cpu profiler (737c7e9)
Message-ID: <201407282130.s6SLUeNH015361@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit 737c7e969142cbc77889e7655d9746a4cd06eb76
Author: NaokiEto <neto at caltech.edu>
Date:   Mon Jul 28 17:18:17 2014 -0400

    updated the foundation for the gperftools, cpu profiler


>---------------------------------------------------------------

737c7e969142cbc77889e7655d9746a4cd06eb76
 configure    | 1 +
 src/Index.cc | 2 +-
 src/Index.hh | 1 +
 3 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/configure b/configure
index eaf9b7a..2ccded7 100755
--- a/configure
+++ b/configure
@@ -96,6 +96,7 @@ while [ $# -ne 0 ]; do
             append_cache_entry ENABLE_PERFTOOLS_DEBUG BOOL   true
             ;;
         --enable-perftools-cpu)
+            append_cache_entry ENABLE_PERFTOOLS     BOOL true
             append_cache_entry ENABLE_PERFTOOLS_CPU BOOL true
             ;;
         --disable-perftools)
diff --git a/src/Index.cc b/src/Index.cc
index 9494763..fd35d8d 100644
--- a/src/Index.cc
+++ b/src/Index.cc
@@ -283,7 +283,7 @@ void Index<T>::addEntry(IndexField *iqe) {
     // if it is not in the Hash table (== NULL)
 	if (ie==NULL) {
 
-        ProfilerStart("IndexHash_add");
+        ProfilerStart("test.prof");
 
         tmlog(TM_LOG_DEBUG, "addEntry", "beginning to add entry with timestamp %f and form %s", iqe->ts, iqe->getStr().c_str());
 
diff --git a/src/Index.hh b/src/Index.hh
index 9211820..e64d50b 100644
--- a/src/Index.hh
+++ b/src/Index.hh
@@ -11,6 +11,7 @@
 #include <sstream>
 #include <pthread.h>
 #include <time.h>
+#include <gperftools/profiler.h>
 
 #include "types.h"
 #include "tm.h"


From neto at caltech.edu  Mon Jul 28 18:03:31 2014
From: neto at caltech.edu (Naoki Eto)
Date: Mon, 28 Jul 2014 18:03:31 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: gperftools with CPU profiler
	seems to be working. You can put the ProfilerStart() and
	ProfilerStop() wherever you want,
	but be wary that putting them relatively close to each other may
	result in 0 samples. (1ed43c2)
Message-ID: <201407290103.s6T13VMx016438@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit 1ed43c2e2a3fcbe99772475eda855484947fb41c
Author: NaokiEto <neto at caltech.edu>
Date:   Mon Jul 28 20:50:34 2014 -0400

    gperftools with CPU profiler seems to be working. You can put the
    ProfilerStart() and ProfilerStop() wherever you want, but be wary
    that putting them relatively close to each other may result in 0
    samples.


>---------------------------------------------------------------

1ed43c2e2a3fcbe99772475eda855484947fb41c
 README         | 2 +-
 src/Index.cc   | 4 ++--
 src/Storage.cc | 5 +++++
 src/main.cc.in | 5 +++++
 4 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/README b/README
index 29779af..52423ad 100644
--- a/README
+++ b/README
@@ -31,5 +31,5 @@ query to_file "testquery.pcap" index ip "[2001:558:fe0d:1c::2]"
 
 Not using brackets around an ipv6 address will result in a querying error.
 
-
+Note that to use gperftools' CPU profiler, you will have to first install libunwind.
 
diff --git a/src/Index.cc b/src/Index.cc
index fd35d8d..617b673 100644
--- a/src/Index.cc
+++ b/src/Index.cc
@@ -283,7 +283,7 @@ void Index<T>::addEntry(IndexField *iqe) {
     // if it is not in the Hash table (== NULL)
 	if (ie==NULL) {
 
-        ProfilerStart("test.prof");
+        //ProfilerStart("/home/lakers/timemachine_results/profile/blah.prof");
 
         tmlog(TM_LOG_DEBUG, "addEntry", "beginning to add entry with timestamp %f and form %s", iqe->ts, iqe->getStr().c_str());
 
@@ -298,7 +298,7 @@ void Index<T>::addEntry(IndexField *iqe) {
 
     	last_updated = iqe->ts;
 
-        ProfilerStop();
+        //ProfilerStop();
 	} else {
         tmlog(TM_LOG_NOTE, "addEntry", "updating the entry with timestamp %f and form %s", iqe->ts, iqe->getStr().c_str());
         // the entry is already in the hash table
diff --git a/src/Storage.cc b/src/Storage.cc
index ae49a6e..88a9c13 100644
--- a/src/Storage.cc
+++ b/src/Storage.cc
@@ -4,6 +4,7 @@
 
 #include <sstream>
 #include <iostream>
+#include <gperftools/profiler.h>
 
 #include "DynClass.hh"
 #include "types.h"
@@ -338,6 +339,8 @@ void Storage::SetHdrSize()
 // note that the pcap packet header has the timestamp of the packet
 void Storage::addPkt(const struct pcap_pkthdr *header,
 					 const unsigned char *packet) {
+
+	//ProfilerStart("/home/lakers/timemachine_results/profile/blah.prof");
 	uint16_t ether_type=ntohs(ETHERNET(packet)->ether_type);
     // ETHERTYPE_IP is EtherType 0x800, for IPv4 addresses
     // EtherType 0x8100 is for VLAN header
@@ -416,6 +419,8 @@ void Storage::addPkt(const struct pcap_pkthdr *header,
      */
 	Connection* c=conns.addPkt(header, idxpacket);
 
+    //ProfilerStop();
+
     // a flow of packets characterized by the 5-tuple of (layer 4 protocol, source ip, source port, destination ip, destination port)
     // note that for protocols other than tcp/udp, may not include source/dest ports
     tmlog(TM_LOG_NOTE, "addPkt: Storage.cc, ~line 255", "updating connection state to have the packet %lu", header->ts.tv_usec);
diff --git a/src/main.cc.in b/src/main.cc.in
index a306d0f..b6bf172 100644
--- a/src/main.cc.in
+++ b/src/main.cc.in
@@ -28,6 +28,8 @@
 #define USE_MALLINFO
 #endif
 #include <sys/ioctl.h>
+#include <gperftools/profiler.h>
+
 
 #ifdef USE_BROCCOLI
 #include "BroccoliComm.hh"
@@ -616,6 +618,8 @@ void usage() {
 
 int
 main(int argc, char** argv) {
+
+	ProfilerStart("/home/lakers/timemachine_results/profile/blah.prof");
 	const char *conffile="@BRO_ETC_INSTALL_DIR@/timemachine.cfg";
 	struct sigaction exit_action;
 	int i;
@@ -850,6 +854,7 @@ main(int argc, char** argv) {
 				pause();
 		}
 
+		ProfilerStop();
         delete stats_log_file;
         delete classes_log_file;
         delete index_log_file; 


From neto at caltech.edu  Tue Jul 29 12:32:30 2014
From: neto at caltech.edu (Naoki Eto)
Date: Tue, 29 Jul 2014 12:32:30 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: Fixed one very hidden bug (in
	ConnectionIF3, IndexField.cc) (f550318)
Message-ID: <201407291932.s6TJWUre026456@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit f55031846992cf01136bd25a61020828840f9037
Author: NaokiEto <neto at caltech.edu>
Date:   Tue Jul 29 15:18:39 2014 -0400

    Fixed one very hidden bug (in ConnectionIF3, IndexField.cc)
    
    Also made it all bro_inet_ntop instead of inet_ntop (seems like
    it is better for FreeBSD)
    
    Also, previously, I added the appropriate header files for
    FreeBSD
    
    Making this commit to save the work so far.


>---------------------------------------------------------------

f55031846992cf01136bd25a61020828840f9037
 README            |  4 ++++
 src/Connection.cc | 26 ++++++++++++++------------
 src/Fifo.cc       |  2 +-
 src/FifoDisk.cc   | 17 +++++++++++------
 src/FifoMem.cc    | 27 ++++++++++++++++-----------
 src/IndexField.cc | 20 ++++++++++----------
 src/IndexField.hh | 10 +++++-----
 src/Storage.cc    | 14 ++++++++------
 src/main.cc.in    |  3 +++
 src/rmtconsole.cc |  3 ++-
 10 files changed, 74 insertions(+), 52 deletions(-)

diff --git a/README b/README
index 52423ad..a121417 100644
--- a/README
+++ b/README
@@ -33,3 +33,7 @@ Not using brackets around an ipv6 address will result in a querying error.
 
 Note that to use gperftools' CPU profiler, you will have to first install libunwind.
 
+Also note that to use pprof to read the .prof files, you will need to install
+perl5, gv, and dot. You can use apt-get install for the gv and dot (if on Ubuntu),
+and install perl5 from source.
+
diff --git a/src/Connection.cc b/src/Connection.cc
index 70e603e..8f13496 100644
--- a/src/Connection.cc
+++ b/src/Connection.cc
@@ -12,6 +12,8 @@
 #include "Query.hh"
 #include "tm.h"
 
+#include "bro_inet_ntop.h"
+
 #ifdef __FreeBSD__
 #include <sys/socket.h>
 #endif
@@ -154,7 +156,7 @@ void ConnectionID4::init(proto_t proto,
 
     tmlog(TM_LOG_NOTE, "connection4: Connection.cc", "connection 4 with form %s", getStr().c_str());
 
-    init_hash_function();
+    //init_hash_function();
 
     HashKey* newHashKey = new HashKey(&key, sizeof(key));
 
@@ -230,7 +232,7 @@ void ConnectionID4::init6(proto_t proto,
 	}
     tmlog(TM_LOG_NOTE, "connection4: Connection.cc", "connection 4 with form %s", getStr().c_str());
 
-    init_hash_function();
+    //init_hash_function();
 
     HashKey* newHashKey = new HashKey(&key, sizeof(key));
 
@@ -324,7 +326,7 @@ void ConnectionID3::init(proto_t proto,
         key.port1 = v6.port2;
         key.port2 = 0;
 
-    init_hash_function();
+    //init_hash_function();
 
     HashKey* newHashKey = new HashKey(&key, sizeof(key));
 
@@ -407,7 +409,7 @@ void ConnectionID3::init6(proto_t proto,
         key.port2 = 0;
 	
 
-    init_hash_function();
+    //init_hash_function();
 
     HashKey* newHashKey = new HashKey(&key, sizeof(key));
 
@@ -488,7 +490,7 @@ void ConnectionID2::init( uint32_t s_ip, uint32_t d_ip) {
 	    key.port2 = 0;
 	}
 
-    init_hash_function();
+    //init_hash_function();
 
     HashKey* newHashKey = new HashKey(&key, sizeof(key));
 
@@ -549,7 +551,7 @@ void ConnectionID2::init6( unsigned char s_ip[], unsigned char d_ip[]) {
 	    key.port2 = 0;
 	}
 
-    init_hash_function();
+    //init_hash_function();
 
     HashKey* newHashKey = new HashKey(&key, sizeof(key));
 
@@ -853,11 +855,11 @@ std::string ConnectionID4::getStr() const {
 
         char str1[INET6_ADDRSTRLEN];
 
-        inet_ntop(AF_INET6, &(v6.ip1), str1, INET6_ADDRSTRLEN);
+        bro_inet_ntop(AF_INET6, &(v6.ip1), str1, INET6_ADDRSTRLEN);
 
         char str2[INET6_ADDRSTRLEN];
 
-        inet_ntop(AF_INET6, &(v6.ip2), str2, INET6_ADDRSTRLEN);
+        bro_inet_ntop(AF_INET6, &(v6.ip2), str2, INET6_ADDRSTRLEN);
 
 
         ss << " ConnectionID4 for IPv6"
@@ -928,11 +930,11 @@ std::string ConnectionID3::getStr() const {
 
         char str1[INET6_ADDRSTRLEN];
 
-        inet_ntop(AF_INET6, &(v6.ip1), str1, INET6_ADDRSTRLEN);
+        bro_inet_ntop(AF_INET6, &(v6.ip1), str1, INET6_ADDRSTRLEN);
 
         char str2[INET6_ADDRSTRLEN];
 
-        inet_ntop(AF_INET6, &(v6.ip2), str2, INET6_ADDRSTRLEN);
+        bro_inet_ntop(AF_INET6, &(v6.ip2), str2, INET6_ADDRSTRLEN);
 
         ss << " ConnectionID3 for IPv6"
 
@@ -1021,11 +1023,11 @@ std::string ConnectionID2::getStr() const {
 
         char str1[INET6_ADDRSTRLEN];
 
-        inet_ntop(AF_INET6, &(v6.ip1), str1, INET6_ADDRSTRLEN);
+        bro_inet_ntop(AF_INET6, &(v6.ip1), str1, INET6_ADDRSTRLEN);
 
         char str2[INET6_ADDRSTRLEN];
 
-        inet_ntop(AF_INET6, &(v6.ip2), str2, INET6_ADDRSTRLEN);
+        bro_inet_ntop(AF_INET6, &(v6.ip2), str2, INET6_ADDRSTRLEN);
 
         ss << " ConnectionID2 for IPv6"
 
diff --git a/src/Fifo.cc b/src/Fifo.cc
index 0e08e6b..2dbc1b9 100644
--- a/src/Fifo.cc
+++ b/src/Fifo.cc
@@ -10,7 +10,7 @@
 #include "Connection.hh"
 #include "FifoDisk.hh"
 #include "tm.h"
-#include "bro_inet_ntop.h"
+//#include "bro_inet_ntop.h"
 
 Fifo::Fifo() {
 	init();
diff --git a/src/FifoDisk.cc b/src/FifoDisk.cc
index d7d48fe..9e81224 100644
--- a/src/FifoDisk.cc
+++ b/src/FifoDisk.cc
@@ -10,7 +10,7 @@
 #include "tm.h"
 #include "FifoDisk.hh"
 #include "Query.hh"
-#include "bro_inet_ntop.h"
+//#include "bro_inet_ntop.h"
 
 
 /***************************************************************************
@@ -350,17 +350,22 @@ uint64_t FifoDiskFile::query( QueryRequest *qreq, QueryResult *qres, IntervalSet
 				if (t<qreq->getT0())
 					continue;
                 //tmlog("The result of matchPkt from QueryRequest is %d 
-                char str1[INET6_ADDRSTRLEN];
+                //char str1[INET6_ADDRSTRLEN];
 
-                bro_inet_ntop(AF_INET6, &(IP6(pkt)->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
+                //bro_inet_ntop(AF_INET6, &(IP6(pkt)->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
+
+                //inet_ntop(AF_INET6, &(IP6(pkt)->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN); 
 
                 //char s1[INET6_ADDRSTRLEN];
 
                 //inet_pton(AF_INET6, s1, str1);
 
-                char str2[INET6_ADDRSTRLEN];
+                //char str2[INET6_ADDRSTRLEN];
+
+                //bro_inet_ntop(AF_INET6, &(IP6(pkt)->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
+
+                //inet_ntop(AF_INET6, &(IP6(pkt)->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);               
 
-                bro_inet_ntop(AF_INET6, &(IP6(pkt)->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
                 //char s2[INET6_ADDRSTRLEN];
                 /*
                 if ( inet_pton(AF_INET6, s2, str2) <=0 )
@@ -368,7 +373,7 @@ uint64_t FifoDiskFile::query( QueryRequest *qreq, QueryResult *qres, IntervalSet
                     tmlog(TM_LOG_ERROR, "Bad IP address: %s", s2);
                     }
                 */
-                tmlog(TM_LOG_NOTE, "FifoDisk.cc: query", "the query packet has source ip address: %s and dst ip address %s and header time stamp %lu and %lu", \
+                //tmlog(TM_LOG_NOTE, "FifoDisk.cc: query", "the query packet has source ip address: %s and dst ip address %s and header time stamp %lu and %lu", \
                 str1, str2, hdr.ts.tv_sec, hdr.ts.tv_usec);
                 tmlog(TM_LOG_NOTE, "FifoDisk.cc:query", "the query parameters are that it has a time interval from %f to %f, a hash of %lu, a timestamp of %f, and a form of %s", \
                 qreq->getT0(), qreq->getT1(), qreq->getField()->hash(), qreq->getField()->ts, qreq->getField()->getStr().c_str());
diff --git a/src/FifoMem.cc b/src/FifoMem.cc
index 22c6599..edab252 100644
--- a/src/FifoMem.cc
+++ b/src/FifoMem.cc
@@ -9,7 +9,7 @@
 #include "FifoMem.hh"
 #include "Query.hh"
 #include "Index.hh"
-#include "bro_inet_ntop.h"
+//#include "bro_inet_ntop.h"
 
 
 FifoMem::FifoMem(uint64_t size): size(size), oldestTimestamp(0), newestTimestamp(0) {
@@ -413,23 +413,28 @@ uint64_t FifoMem::query(QueryRequest *qreq, QueryResult *qres,
 #endif
 			while ( (p_will_wrap || (p<=lp)) 
 					&& pkt_t(p) <= i->getLast() ) {
-                char str1[INET6_ADDRSTRLEN];
+                //char str1[INET6_ADDRSTRLEN];
 
-                bro_inet_ntop(AF_INET6, &(IP6(p)->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
+                //bro_inet_ntop(AF_INET6, &(IP6(p)->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
 
-                char s1[INET6_ADDRSTRLEN];
+                //inet_ntop(AF_INET6, &(IP6(p)->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
 
-                inet_pton(AF_INET6, s1, str1);
+                //char s1[INET6_ADDRSTRLEN];
 
-                char str2[INET6_ADDRSTRLEN];
+                //inet_pton(AF_INET6, s1, str1);
 
-                bro_inet_ntop(AF_INET6, &(IP6(p)->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
-                char s2[INET6_ADDRSTRLEN];
+                //char str2[INET6_ADDRSTRLEN];
 
-                inet_pton(AF_INET6, s2, str2);
+                //bro_inet_ntop(AF_INET6, &(IP6(p)->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
                 
-                tmlog(TM_LOG_NOTE, "FifoMem.cc: query", "the query packet has source ip address: %s and dst ip address %s", str1, str2);
-                tmlog(TM_LOG_NOTE, "FifoMem.cc:query", "the query parameters in mem are that it has a time interval from %f to %f, a hash of %lu, a timestamp of %f, and a form of %s", \
+                //inet_ntop(AF_INET6, &(IP6(p)->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
+
+                //char s2[INET6_ADDRSTRLEN];
+
+                //inet_pton(AF_INET6, s2, str2);
+
+                //tmlog(TM_LOG_NOTE, "FifoMem.cc: query", "the query packet has source ip address: %s and dst ip address %s", str1, str2);
+                //tmlog(TM_LOG_NOTE, "FifoMem.cc:query", "the query parameters in mem are that it has a time interval from %f to %f, a hash of %lu, a timestamp of %f, and a form of %s", \
                 qreq->getT0(), qreq->getT1(), qreq->getField()->hash(), qreq->getField()->ts, qreq->getField()->getStr().c_str());
 				if (qreq->matchPkt(p) && last_match_ts < pkt_t(p))  {
 					qres->sendPkt(p);
diff --git a/src/IndexField.cc b/src/IndexField.cc
index 7190431..27033bd 100644
--- a/src/IndexField.cc
+++ b/src/IndexField.cc
@@ -122,7 +122,7 @@ void IPAddress::getStr(char* s, int maxsize) const {
         tmlog(TM_LOG_NOTE, "IPAddress", "IPAddress, IPv4");
 		char ucp[INET_ADDRSTRLEN];
 
-		if ( ! inet_ntop(AF_INET, &ipv6_address.s6_addr[12], ucp, INET_ADDRSTRLEN) )
+		if ( ! bro_inet_ntop(AF_INET, &ipv6_address.s6_addr[12], ucp, INET_ADDRSTRLEN) )
 			tmlog(TM_LOG_ERROR, "IPAddress", "<bad IPv4 address conversion");
 		else
 			snprintf(s, maxsize, "%s", ucp);
@@ -132,7 +132,7 @@ void IPAddress::getStr(char* s, int maxsize) const {
         tmlog(TM_LOG_NOTE, "IPAddress", "IPAddress, IPv6");
 		char ucp[INET6_ADDRSTRLEN];
 
-		if ( ! inet_ntop(AF_INET6, ipv6_address.s6_addr, ucp, INET6_ADDRSTRLEN) )
+		if ( ! bro_inet_ntop(AF_INET6, ipv6_address.s6_addr, ucp, INET6_ADDRSTRLEN) )
 			tmlog(TM_LOG_ERROR, "IPAddress", "<bad IPv6 address conversion");
 		else
 			snprintf(s, maxsize, "%s", ucp);
@@ -189,7 +189,7 @@ void IPAddress::Init(const std::string& s)
 		addr = htonl(addr);
 		memcpy(&ipv6_address.s6_addr[12], &addr, sizeof(uint32_t));
 
-        init_hash_function();
+        //init_hash_function();
         HashKey* newHashKey = new HashKey((void*)ipv6_address.s6_addr, sizeof(ipv6_address.s6_addr));
 
         hash_key = newHashKey->Hash();
@@ -209,7 +209,7 @@ void IPAddress::Init(const std::string& s)
 		}
         tmlog(TM_LOG_NOTE, "IPAddress::Init", "good IP Address %s", s.c_str());
 
-        init_hash_function();
+        //init_hash_function();
         HashKey* newHashKey = new HashKey((void*)ipv6_address.s6_addr, sizeof(ipv6_address.s6_addr));
 
         hash_key = newHashKey->Hash();
@@ -225,7 +225,7 @@ std::string IPAddress::getStr() const
         tmlog(TM_LOG_NOTE, "IPAddress: getStr()", "IPAddress, IPv4");
 		char s[INET_ADDRSTRLEN];
 
-		if ( ! bro_inet_ntop(AF_INET, &ipv6_address.s6_addr[12], s, INET_ADDRSTRLEN) )
+		if ( ! bro_inet_ntop(AF_INET, &ipv6_address.s6_addr[12], s, INET_ADDRSTRLEN) ) //bro_inet_ntop(AF_INET, &ipv6_address.s6_addr[12], s, INET_ADDRSTRLEN) )
 			return "<bad IPv4 address conversion";
 		else
 			return s;
@@ -236,7 +236,7 @@ std::string IPAddress::getStr() const
         tmlog(TM_LOG_NOTE, "IPAddress: getStr()", "IPAddress, IPv6");
 		char s[INET6_ADDRSTRLEN];
 
-		if ( ! bro_inet_ntop(AF_INET6, ipv6_address.s6_addr, s, INET6_ADDRSTRLEN) )
+		if ( ! bro_inet_ntop(AF_INET6, ipv6_address.s6_addr, s, INET6_ADDRSTRLEN) ) //bro_inet_ntop(AF_INET6, ipv6_address.s6_addr, s, INET6_ADDRSTRLEN) )
 			return "<bad IPv6 address conversion";
 		else
 			return s;
@@ -269,7 +269,7 @@ std::string IPAddress::getStrPkt(const u_char* packet) const
         tmlog(TM_LOG_NOTE, "IPAddress: getStr(u_char*)", "IPAddress, IPv6");
 		char str[INET6_ADDRSTRLEN];
 
-		if ( ! inet_ntop(AF_INET6, ipv6_address.s6_addr, str, INET6_ADDRSTRLEN) )
+		if ( ! bro_inet_ntop(AF_INET6, ipv6_address.s6_addr, str, INET6_ADDRSTRLEN) )
 			return "<bad IPv6 address conversion";
 		else
         {
@@ -342,7 +342,7 @@ std::string SrcIPAddress::getStrPkt(const u_char* packet) const
         //tmlog(TM_LOG_NOTE, "SrcIPAddress: getStr(u_char*)", "IPAddress, IPv6");
 		char str[INET6_ADDRSTRLEN];
 
-		if ( ! inet_ntop(AF_INET6, IP6(packet)->ip6_src.s6_addr, str, INET6_ADDRSTRLEN) )
+		if ( ! bro_inet_ntop(AF_INET6, IP6(packet)->ip6_src.s6_addr, str, INET6_ADDRSTRLEN) )
 			return "<bad IPv6 address conversion";
 		else
         {
@@ -395,7 +395,7 @@ std::string DstIPAddress::getStrPkt(const u_char* packet) const
         tmlog(TM_LOG_NOTE, "DstIPAddress: getStr(u_char*)", "IPAddress, IPv6");
 		char str[INET6_ADDRSTRLEN];
 
-		if ( ! inet_ntop(AF_INET6, IP6(packet)->ip6_dst.s6_addr, str, INET6_ADDRSTRLEN) )
+		if ( ! bro_inet_ntop(AF_INET6, IP6(packet)->ip6_dst.s6_addr, str, INET6_ADDRSTRLEN) )
 			return "<bad IPv6 address conversion";
 		else
         {
@@ -705,7 +705,7 @@ std::string ConnectionIF3::pattern6_connection3 = "\\s*(\\w+)\\s+"
 
 RE2 ConnectionIF3::re(ConnectionIF3::pattern_connection3);
 
-RE2 ConnectionIF3::re6(ConnectionIF3::pattern_connection3);
+RE2 ConnectionIF3::re6(ConnectionIF3::pattern6_connection3);
 
 std::list<ConnectionIF3*>
 ConnectionIF3::genKeys(const u_char* packet) {
diff --git a/src/IndexField.hh b/src/IndexField.hh
index dae0a81..bbcbf1e 100644
--- a/src/IndexField.hh
+++ b/src/IndexField.hh
@@ -90,7 +90,7 @@ public:
 		memcpy(ipv6_address.s6_addr, v4_mapped_prefix, sizeof(v4_mapped_prefix));
 		memcpy(&ipv6_address.s6_addr[12], &in4.s_addr, sizeof(in4.s_addr));
 
-        init_hash_function();
+        //init_hash_function();
         HashKey* newHashKey = new HashKey((void*)ipv6_address.s6_addr, sizeof(ipv6_address.s6_addr));
 
         hash_key = newHashKey->Hash();
@@ -113,7 +113,7 @@ public:
     {
         memcpy(ipv6_address.s6_addr, ip6, 16);
 
-        init_hash_function();
+        //init_hash_function();
         HashKey* newHashKey = new HashKey((void*)ipv6_address.s6_addr, sizeof(ipv6_address.s6_addr));
 
         hash_key = newHashKey->Hash();
@@ -144,7 +144,7 @@ public:
 	IPAddress(void *p) {
 		memcpy((void*)getConstKeyPtr(), p, getKeySize());
 
-        init_hash_function();
+        //init_hash_function();
         HashKey* newHashKey = new HashKey((void*)ipv6_address.s6_addr, sizeof(ipv6_address.s6_addr));
 
         hash_key = newHashKey->Hash();
@@ -357,7 +357,7 @@ public:
 class Port: public IndexField {
 public:
 	Port(): port(0) {
-        init_hash_function();
+        //init_hash_function();
 
         HashKey* newHashKey = new HashKey((void*)ipv6_address.s6_addr, sizeof(ipv6_address.s6_addr));
 
@@ -368,7 +368,7 @@ public:
         //free_hash_function();
     }
 	Port(uint16_t port): port(port) {  /* printf("Port(%u)\n", port); */
-        init_hash_function();
+        //init_hash_function();
 
         HashKey* newHashKey = new HashKey((void*)ipv6_address.s6_addr, sizeof(ipv6_address.s6_addr));
 
diff --git a/src/Storage.cc b/src/Storage.cc
index 88a9c13..e7dfe2c 100644
--- a/src/Storage.cc
+++ b/src/Storage.cc
@@ -15,7 +15,7 @@
 #include "packet_headers.h"
 #include "conf.h"
 #include "tm.h"
-#include "bro_inet_ntop.h"
+//#include "bro_inet_ntop.h"
 
 #define SNAPLEN 8192
 
@@ -412,6 +412,8 @@ void Storage::addPkt(const struct pcap_pkthdr *header,
     // set now to be the time stamp field of the pcap packet header
 	tm_time_t now = to_tm_time(&header->ts);
 
+    //init_hash_function();
+
 	/* update connections state, classify, elephant cutoff */
     // conns is of type Connections
     /* add a packet. lookup the connection and increment byte and pkt counters if it exists,
@@ -533,19 +535,19 @@ void Storage::addPkt(const struct pcap_pkthdr *header,
 			if (TCP(packet)->th_flags & ( TH_FIN | TH_SYN | TH_RST ))
 				tcp_ctrl_flag=true;
 
-            char str1[INET6_ADDRSTRLEN];
+            //char str1[INET6_ADDRSTRLEN];
 
-            bro_inet_ntop(AF_INET6, &(IP6(idxpacket)->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
+            //bro_inet_ntop(AF_INET6, &(IP6(idxpacket)->ip6_src.s6_addr), str1, INET6_ADDRSTRLEN);
 
             //char s1[INET6_ADDRSTRLEN];
 
             //inet_pton(AF_INET6, s1, str1);
 
-            char str2[INET6_ADDRSTRLEN];
+            //char str2[INET6_ADDRSTRLEN];
 
-            bro_inet_ntop(AF_INET6, &(IP6(idxpacket)->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
+            //bro_inet_ntop(AF_INET6, &(IP6(idxpacket)->ip6_dst.s6_addr), str2, INET6_ADDRSTRLEN);
 
-            tmlog(TM_LOG_NOTE, "Storage::addPkt", "we will be calling Fifo::Addpkt on packet that has source ip %s and destination ip %s", str1, str2);
+            //tmlog(TM_LOG_NOTE, "Storage::addPkt", "we will be calling Fifo::Addpkt on packet that has source ip %s and destination ip %s", str1, str2);
         /* true if cutoff should not be done for this connection */
         // When the connection is subscribed or when a tcp control flag
         // is set, then addPkt is called with a connection NULL pointer
diff --git a/src/main.cc.in b/src/main.cc.in
index b6bf172..bce1ef6 100644
--- a/src/main.cc.in
+++ b/src/main.cc.in
@@ -45,6 +45,7 @@
 #include "LogFile.hh"
 #include "Query.hh"
 #include "Index.hh"
+#include "Hash.h"
 
 
 /***************************************************************************
@@ -731,6 +732,8 @@ main(int argc, char** argv) {
 	// Initialise cmd_parser
 	cmd_parser_init();
 
+        init_hash_function();
+
 	try {
 		storage = new Storage(storageConf);
 	} catch (const std::string& str) {
diff --git a/src/rmtconsole.cc b/src/rmtconsole.cc
index 34b8f87..0788d06 100644
--- a/src/rmtconsole.cc
+++ b/src/rmtconsole.cc
@@ -13,6 +13,7 @@
 #include "tm.h"
 #include "Storage.hh"
 #include "conf.h"
+#include "bro_inet_ntop.h"
 
 /* External functions and vars */
 extern Storage* storage;
@@ -128,7 +129,7 @@ rmtconsole_listen_thread(void *arg) {
 
 		
 		tmlog(TM_LOG_NOTE, "rmtconsole", "incoming connection from %s port %d",
-		       inet_ntop(AF_INET, &remote_addr.sin_addr,
+		       bro_inet_ntop(AF_INET, &remote_addr.sin_addr,
 		                 addrbuf, sizeof(addrbuf)),
 		       ntohs(remote_addr.sin_port));
 		// FIXME: Check return values


From neto at caltech.edu  Thu Jul 31 18:34:14 2014
From: neto at caltech.edu (Naoki Eto)
Date: Thu, 31 Jul 2014 18:34:14 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: removed tmlog by commenting them
	out (to better cpu performance) (325409e)
Message-ID: <201408010134.s711YE14021880@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit 325409e7e58e1ce7e185ecb0be7d996ebd3995fe
Author: NaokiEto <neto at caltech.edu>
Date:   Thu Jul 31 21:14:49 2014 -0400

    removed tmlog by commenting them out (to better cpu performance)
    
    removed multiple memcpy's in Connection.cc and reduced the size
    of the structs
    
    removed some creations of new and delete of HashKey and used only
    HashBytes instead to save some more cpu performance
    
    changed memcmp's arguments in == overloaded operator definitions
    to be a little more optimal
    
    removed multiple instances of init_hash for H3 to reduce cpu
    usage further. It now only runs one instance in the very beginning
    
    will investigate more of gperftool's findings


>---------------------------------------------------------------

325409e7e58e1ce7e185ecb0be7d996ebd3995fe
 src/Connection.cc  | 475 +++++++++++++++++++++++++++++------------------------
 src/Connection.hh  | 180 +++++++++++---------
 src/Connections.cc |  18 +-
 src/Connections.hh |   6 +-
 src/DiskIndex.cc   |   2 +-
 src/DynClass.cc    |   4 +-
 src/Fifo.cc        |   8 +-
 src/FifoDisk.cc    |   4 +-
 src/Hash.cc        |  10 +-
 src/Index.cc       |  62 +++----
 src/Index.hh       |  12 +-
 src/IndexEntry.hh  |  12 +-
 src/IndexField.cc  |  32 ++--
 src/IndexField.hh  |  84 ++++++++--
 src/IndexHash.cc   |  44 ++---
 src/Storage.cc     |  40 ++---
 src/cmd_parser.yy  |   6 +-
 src/main.cc.in     |   6 +-
 src/tm.h           |   4 +-
 19 files changed, 575 insertions(+), 434 deletions(-)

Diff suppressed because of size. To see it, use:

    git diff-tree --root --patch-with-stat --no-color --find-copies-harder --ignore-space-at-eol --cc 325409e7e58e1ce7e185ecb0be7d996ebd3995fe

From neto at caltech.edu  Thu Jul 31 18:40:28 2014
From: neto at caltech.edu (Naoki Eto)
Date: Thu, 31 Jul 2014 18:40:28 -0700
Subject: [TM] [git/tm] topic/naokieto/ipv6: Updated the README to include
	some instructions about using gperftools' cpu profiler (5854443)
Message-ID: <201408010140.s711eSNY021982@bro-ids.icir.org>

Repository : ssh://git at bro-ids.icir.org/time-machine

On branch  : topic/naokieto/ipv6

>---------------------------------------------------------------

commit 5854443cc8375bd42928c12eb290acd40d811134
Author: NaokiEto <neto at caltech.edu>
Date:   Thu Jul 31 21:28:12 2014 -0400

    Updated the README to include some instructions about using gperftools'
    cpu profiler


>---------------------------------------------------------------

5854443cc8375bd42928c12eb290acd40d811134
 README | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/README b/README
index a121417..d0aaaad 100644
--- a/README
+++ b/README
@@ -37,3 +37,16 @@ Also note that to use pprof to read the .prof files, you will need to install
 perl5, gv, and dot. You can use apt-get install for the gv and dot (if on Ubuntu),
 and install perl5 from source.
 
+Note that to add the appropriate directories for gperftools, the file to look at is
+cmake/FindGooglePerftools.cmake
+
+Also, make sure to change the directory in the ProfilerStart function's argument in
+main.cc.in and/or reposition the ProfilerStart and ProfilerStop
+
+To enable cpu profiler debugging, add --enable-perftools-cpu to the configuring
+So, something like:
+
+./configure --enable-perftools-cpu --enable-debug --prefix=/home/lakers/tm
+
+will work.
+