[Bro] SHA256 Hash File Analyzer

[Azoff, Justin S]

> 
> On Dec 30, 2016, at 11:00 AM, Ryan Stillions <ryanstillions at hotmail.com> wrote:
> 
> I'm curious if anyone has this turned on at scale, on production systems?  If so, can you speak to the performance impacts Seth mentioned below?  
> 
> Seth, 
> any thoughts if this would be the same with 2.5 as it was when you originally posted?   I didn't see anything specific about it in release notes, so would we be correct to assume the SHA256 analyzer would probably perform the same as what you saw back in Feb 16?


The analyzer really just delegates to openssl to do all the hashing, so you should be able to use openssl to gauge the performance impact:

$ openssl  speed md5 sha1 sha256
Doing md5 for 3s on 16 size blocks: 6879766 md5's in 3.00s
Doing md5 for 3s on 64 size blocks: 5066897 md5's in 3.00s
Doing md5 for 3s on 256 size blocks: 2814019 md5's in 3.00s
Doing md5 for 3s on 1024 size blocks: 1016906 md5's in 3.00s
Doing md5 for 3s on 8192 size blocks: 147949 md5's in 3.00s
Doing sha1 for 3s on 16 size blocks: 7763902 sha1's in 3.00s
Doing sha1 for 3s on 64 size blocks: 5420584 sha1's in 3.00s
Doing sha1 for 3s on 256 size blocks: 2965390 sha1's in 3.00s
Doing sha1 for 3s on 1024 size blocks: 1054003 sha1's in 3.00s
Doing sha1 for 3s on 8192 size blocks: 147866 sha1's in 3.00s
Doing sha256 for 3s on 16 size blocks: 4896135 sha256's in 3.00s
Doing sha256 for 3s on 64 size blocks: 2682706 sha256's in 3.00s
Doing sha256 for 3s on 256 size blocks: 1131865 sha256's in 3.00s
Doing sha256 for 3s on 1024 size blocks: 342980 sha256's in 3.00s
Doing sha256 for 3s on 8192 size blocks: 45549 sha256's in 3.00s
OpenSSL 1.0.1e-fips 11 Feb 2013
built on: Tue Sep 27 13:37:25 UTC 2016
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
md5              36692.09k   108093.80k   240129.62k   347103.91k   403999.40k
sha1             41407.48k   115639.13k   253046.61k   359766.36k   403772.76k
sha256           26112.72k    57231.06k    96585.81k   117070.51k   124379.14k

On a different machine with a different distribution and newer CPUs I get

type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
md5              50302.28k   175259.63k   373751.13k   536014.85k   632668.98k
sha1             62768.79k   170994.71k   358746.20k   509927.77k   569868.29k
sha256           50775.24k   110530.33k   188262.14k   241865.05k   270240.43k

The 1024 byte block size and below would be the most relevant for bro. Unless you're using jumbo frames bro shouldn't be doing much with blocks larger than 1500.


-- 
- Justin Azoff