[Bro] New installation crashes appear to be ssh-related
Llewellyn, Ted
Ted.Llewellyn at ftr.com
Tue Mar 31 09:36:58 PDT 2015
Robin,
I have attached the backtrace to the ticket, but here it is also:
(gdb) bt
#0 0xb76e6424 in __kernel_vsyscall ()
#1 0xb71b4661 in raise () from /lib/i386-linux-gnu/i686/cmov/libc.so.6
#2 0xb71b7a92 in abort () from /lib/i386-linux-gnu/i686/cmov/libc.so.6
#3 0xb71ad878 in __assert_fail () from /lib/i386-linux-gnu/i686/cmov/libc.so.6
#4 0x083eaabe in binpac::SSH::SSH2_KEXINIT::Parse (this=0xac7ac978,
t_begin_of_data=t_begin_of_data at entry=0xac533ff6 "",
t_end_of_data=t_end_of_data at entry=0xac534008 "\210>%\255\035",
t_context=t_context at entry=0xad9419e8, t_byteorder=t_byteorder at entry=0)
at /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:1382
#5 0x083eac60 in binpac::SSH::SSH2_Message::Parse (this=0xad22d938,
t_begin_of_data=t_begin_of_data at entry=0xac533ff6 "",
t_end_of_data=t_end_of_data at entry=0xac534008 "\210>%\255\035",
t_context=t_context at entry=0xad9419e8, t_byteorder=t_byteorder at entry=0)
at /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:1216
#6 0x083eb160 in binpac::SSH::SSH2_Key_Exchange::ParseBuffer (
this=0xab743610, t_flow_buffer=0xafd04dc0, t_context=0xad9419e8,
t_byteorder=0) at /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:1090
#7 0x083eb4d6 in binpac::SSH::SSH_Key_Exchange::ParseBuffer (this=0xaeb2e878,
t_flow_buffer=0xafd04dc0, t_context=0xad9419e8, t_byteorder=0)
at /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:520
#8 0x083eb6ff in binpac::SSH::SSH_PDU::ParseBuffer (this=0xaeb323f8,
t_flow_buffer=0xafd04dc0, t_context=0xad9419e8)
at /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:360
---Type <return> to continue, or q <return> to quit---
#9 0x083eb982 in binpac::SSH::SSH_Flow::NewData (this=0xafd635b8,
t_begin_of_data=0xac533ff0 "", t_end_of_data=0xac534008 "\210>%\255\035")
at /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:2913
#10 0x083e2855 in analyzer::SSH::SSH_Analyzer::DeliverStream (this=0xadc3e1f0,
len=24, data=0xac533ff0 "", orig=true)
at /root/bro/src/analyzer/protocol/ssh/SSH.cc:71
#11 0x08479f34 in analyzer::Analyzer::NextStream (this=0xadc3e1f0, len=24,
data=0xac533ff0 "", is_orig=true) at /root/bro/src/analyzer/Analyzer.cc:245
#12 0x0847a72c in analyzer::Analyzer::ForwardStream (this=0xae014040, len=24,
data=0xac533ff0 "", is_orig=true) at /root/bro/src/analyzer/Analyzer.cc:331
#13 0x0840ddec in analyzer::tcp::TCP_Reassembler::DeliverBlock (
this=this at entry=0xadbc1cb0, seq=16, len=len at entry=24, data=0xac533ff0 "")
at /root/bro/src/analyzer/protocol/tcp/TCP_Reassembler.cc:647
#14 0x0840e2cc in BlockInserted (start_block=<optimized out>,
this=<optimized out>)
at /root/bro/src/analyzer/protocol/tcp/TCP_Reassembler.cc:393
#15 analyzer::tcp::TCP_Reassembler::BlockInserted (this=0xadbc1cb0,
start_block=0xac648218)
at /root/bro/src/analyzer/protocol/tcp/TCP_Reassembler.cc:368
#16 0x0840db2e in analyzer::tcp::TCP_Reassembler::DataSent (this=0xadbc1cb0,
t=1427797676.2736609, seq=16, len=<optimized out>, data=0xa8d1a4a "",
replaying=true)
at /root/bro/src/analyzer/protocol/tcp/TCP_Reassembler.cc:492
---Type <return> to continue, or q <return> to quit---
#17 0x0840beeb in analyzer::tcp::TCP_Endpoint::DataSent (this=0xadc74340,
t=1427797676.2736609, seq=16, len=24, caplen=24, data=0xa8d1a4a "",
ip=0xbfbeacac, tp=0xa8d1a2a)
at /root/bro/src/analyzer/protocol/tcp/TCP_Endpoint.cc:205
#18 0x08408c76 in DeliverData (flags=..., is_orig=<optimized out>,
rel_data_seq=16, endpoint=0xadc74340, tp=0xa8d1a2a, ip=0xbfbeacac,
caplen=<optimized out>, len=<optimized out>, data=<optimized out>,
t=<optimized out>, this=0xae014040)
at /root/bro/src/analyzer/protocol/tcp/TCP.cc:947
#19 analyzer::tcp::TCP_Analyzer::DeliverPacket (this=0xae014040, len=24,
data=0xa8d1a4a "", is_orig=true, seq=18446744073709551615, ip=0xbfbeacac,
caplen=24) at /root/bro/src/analyzer/protocol/tcp/TCP.cc:1347
#20 0x0847a118 in analyzer::Analyzer::NextPacket (this=0xae014040, len=56,
data=0xa8d1a2a "\230", <incomplete sequence \335>, is_orig=true,
seq=18446744073709551615, ip=0xbfbeacac, caplen=56)
at /root/bro/src/analyzer/Analyzer.cc:222
#21 0x081951c4 in Connection::NextPacket (this=0xafd52858,
t=1427797676.2736609, is_orig=1, ip=0xbfbeacac, len=56, caplen=56,
data=@0xbfbeaa68: 0xa8d1a2a "\230", <incomplete sequence \335>,
record_packet=@0xbfbeaa70: 1, record_content=@0xbfbeaa74: 1,
hdr=0xa097074, pkt=0xa8d1a08 "", hdr_size=14) at /root/bro/src/Conn.cc:260
#22 0x08238ca0 in NetSessions::DoNextPacket (this=this at entry=0xa8d3a10,
t=1427797676.2736609,
---Type <return> to continue, or q <return> to quit---
t at entry=<error reading variable: Could not find type for DW_OP_GNU_const_type>, hdr=hdr at entry=0xa097074, ip_hdr=ip_hdr at entry=0xbfbeacac,
pkt=pkt at entry=0xa8d1a08 "", hdr_size=hdr_size at entry=14,
encapsulation=encapsulation at entry=0x0) at /root/bro/src/Sessions.cc:760
#23 0x0823a3bc in NetSessions::NextPacket (this=0xa8d3a10,
t=1427797676.2736609, hdr=0xa097074, pkt=0xa8d1a08 "", hdr_size=14)
at /root/bro/src/Sessions.cc:231
#24 0x08205de6 in net_packet_dispatch (t=1427797676.2736609, hdr=0xa097074,
pkt=0xa8d1a08 "", hdr_size=14, src_ps=0xa096f88)
at /root/bro/src/Net.cc:281
#25 0x0844d5ce in iosource::PktSrc::Process (this=0xa096f88)
at /root/bro/src/iosource/PktSrc.cc:411
#26 0x0820631a in net_run () at /root/bro/src/Net.cc:329
#27 0x0815e588 in main (argc=19, argv=0xbfbeb214) at /root/bro/src/main.cc:1212
(gdb)
Ted
-----Original Message-----
From: Robin Sommer [mailto:robin at icir.org]
Sent: Tuesday, March 31, 2015 11:42 AM
To: Llewellyn, Ted
Cc: bro at bro.org
Subject: Re: [Bro] New installation crashes appear to be ssh-related
Thanks for filing the ticket. For the core, actually what would be most helpful right now I believe is a stack backtrace. Your crash report didn't have that, it looks like there's no gdb installed. Can you install gdb and then run "gdb bro core" + "bt" as described here:
https://www.bro.org/support/reporting-problems.html#getting-more-information-after-acrash
For the core itself, I think the best thing might be to hold on to it for now, just the core won't be useful for others much anyways, as one also needs to the binary and potentially a similar system to use it.
So if you could keep binary and core somewhere until this is resolved, that would be best for now.
Robin
On Mon, Mar 30, 2015 at 23:54 +0000, you wrote:
> Robin,
>
> I submitted a ticket, 1361. It won't let me attach the core dump as it's too big. How do I upload that?
>
> Thanks,
> Ted
>
>
> -----Original Message-----
> From: Robin Sommer [mailto:robin at icir.org]
> Sent: Monday, March 30, 2015 10:39 AM
> To: Llewellyn, Ted
> Cc: bro at bro.org
> Subject: Re: [Bro] New installation crashes appear to be ssh-related
>
> Ted, mind filing a ticket so that we track this one?
>
> Robin
>
> On Sun, Mar 29, 2015 at 15:12 +0000, you wrote:
>
> > We have a new Bro installation, built from source on Debian wheezy, that keeps core dumping. It looks like it's choking on some code related to ssh. Here is the diag for the latest crash. It is identical to the other one I have:
> >
> > [BroControl] > diag
> > [bro]
> >
> > Bro 2.3-633
> > Linux 3.2.0-4-686-pae
> >
> > No gdb installed.
> >
> > ==== No reporter.log
> >
> > ==== stderr.log
> > listening on eth1, capture length 8192 bytes
> >
> > bro: /root/bro/build/src/analyzer/protocol/ssh/ssh_pac.cc:1382: int binpac::SSH::SSH2_KEXINIT::Parse(binpac::const_byteptr, binpac::const_byteptr, binpac::SSH::ContextSSH*, int): Assertion `t_dataptr_after_cookie <= t_end_of_data' failed.
> > /usr/local/bro/share/broctl/scripts/run-bro: line 100: 10307 Aborted (core dumped) nohup "$mybro" "$@"
> >
> > ==== stdout.log
> > max memory size (kbytes, -m) unlimited
> > data seg size (kbytes, -d) unlimited
> > virtual memory (kbytes, -v) unlimited
> > core file size (blocks, -c) unlimited
> >
> > ==== .cmdline
> > -i eth1 -U .status -p broctl -p broctl-live -p standalone -p local
> > -p bro local.bro broctl broctl/standalone broctl/auto
> >
> > ==== .env_vars
> > PATH=/usr/local/bro/bin:/usr/local/bro/share/broctl/scripts:/usr/loc
> > al
> > /bro/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bi
> > n
> > BROPATH=/usr/local/bro/spool/installed-scripts-do-not-touch/site::/u
> > sr
> > /local/bro/spool/installed-scripts-do-not-touch/auto:/usr/local/bro/
> > sh
> > are/bro:/usr/local/bro/share/bro/policy:/usr/local/bro/share/bro/sit
> > e
> > CLUSTER_NODE=
> >
> > ==== .status
> > RUNNING [net_run]
> >
> > ==== No prof.log
> >
> > ==== No packet_filter.log
> >
> > ==== No loaded_scripts.log
> > [BroControl] >
> >
> > This is just running the default setup, with the local subnets
> > configured, as we are just starting with Bro. This is a really low end server, but the capture interface is only running at 100 meg so there are really no resource issues. (Yes, this is a 32-bit box. It's pretty old. That's why I built from source.) The first crash occurred after a few minutes. Then it ran for nearly 24 hours before this crash. Is there something I can tweak to prevent this?
> >
> > Thanks,
> > Ted Llewellyn
> >
> >
>
> > _______________________________________________
> > Bro mailing list
> > bro at bro-ids.org
> > http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/bro
>
>
> --
> Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin
>
--
Robin Sommer * ICSI/LBNL * robin at icir.org * www.icir.org/robin
More information about the Bro
mailing list