[Xorp-users] Problems with Linux kernel and OSPF ???
Aidan Walton
awalton at wires3.net
Wed Dec 5 10:24:14 PST 2007
Hi Atanu,
We didn't have to wait too long. It just happened again, this time on a
different router altogether. I would love to give you logs, but its
broken. It seems that when logrotate does its job in the early hours,
after it has rotated the active log file, xorp stops writing anything
into the log file. This seems to happen on all the routers. I have no
idea why, maybe you do. As soon as you restart xorp the log output
beings to appear in the log file. On the routers where I have not
restarted xorp, yesterdays logs are intact in the rotated file up to
when cron runs logrotate but the new log file is empty a big fat zero
and unless I restart the xorp processes it seems to stay this way!
Inline is the output from the commands you wanted me to run and the
replay file attached. I hope this helps.
All the best
Aidan
lodgefarm:~# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
89.248.141.239 * 255.255.255.255 UH 0 0 0
ppp13
89.248.141.238 * 255.255.255.255 UH 0 0 0
ppp25
89.248.141.254 * 255.255.255.255 UH 0 0 0
ppp27
89.248.141.237 * 255.255.255.255 UH 0 0 0
ppp23
89.248.141.253 * 255.255.255.255 UH 0 0 0
ppp15
89.248.141.252 * 255.255.255.255 UH 0 0 0
ppp11
89.248.141.236 * 255.255.255.255 UH 0 0 0
ppp7
89.248.141.235 * 255.255.255.255 UH 0 0 0
ppp0
89.248.141.251 * 255.255.255.255 UH 0 0 0
ppp3
89.248.141.234 * 255.255.255.255 UH 0 0 0
ppp16
89.248.141.250 * 255.255.255.255 UH 0 0 0
ppp22
89.248.141.233 * 255.255.255.255 UH 0 0 0
ppp14
89.248.141.249 * 255.255.255.255 UH 0 0 0
ppp1
89.248.141.232 * 255.255.255.255 UH 0 0 0
ppp12
89.248.141.248 * 255.255.255.255 UH 0 0 0
ppp10
89.248.141.231 * 255.255.255.255 UH 0 0 0
ppp26
89.248.141.247 * 255.255.255.255 UH 0 0 0
ppp4
89.248.141.230 * 255.255.255.255 UH 0 0 0
ppp19
89.248.141.246 * 255.255.255.255 UH 0 0 0
ppp17
89.248.141.229 * 255.255.255.255 UH 0 0 0
ppp18
89.248.141.245 * 255.255.255.255 UH 0 0 0
ppp5
89.248.141.244 * 255.255.255.255 UH 0 0 0
ppp6
89.248.141.228 * 255.255.255.255 UH 0 0 0
ppp2
89.248.141.227 * 255.255.255.255 UH 0 0 0
ppp24
89.248.141.243 * 255.255.255.255 UH 0 0 0
ppp21
89.248.141.242 * 255.255.255.255 UH 0 0 0
ppp8
89.248.141.241 * 255.255.255.255 UH 0 0 0
ppp20
89.248.141.196 * 255.255.255.252 U 0 0 0
ath0
89.248.142.128 89.248.141.221 255.255.255.248 UG 10 0 0
lo
89.248.141.224 * 255.255.255.224 U 0 0 0
ath1
10.20.30.0 * 255.255.255.0 U 0 0 0
eth0
lodgefarm:~# xorpsh
Welcome to XORP on lodgefarm
root at lodgefarm> show ospf4 neighbor
Address Interface State ID Pri
Dead
89.248.141.197 ath0/ath0 Full 89.248.141.222 128
34
root at lodgefarm> show ospf4 database
OSPF link state database, Area 0.0.0.0
Type ID Adv Rtr Seq Age Opt Cksum
Len
Router *89.248.141.221 89.248.141.221 0x80000807 409 0x2 0x9516
48
ASExt-2 *89.248.141.224 89.248.141.221 0x8000003a 645 0x2 0x7903
36
Router 89.248.141.222 89.248.141.222 0x800008f0 405 0x2 0x77f5
60
Network 89.248.141.197 89.248.141.222 0x8000003b 409 0x2 0xaf92
32
Router 89.248.141.223 89.248.141.223 0x80000fc5 1690 0x2 0xa048
48
Network 89.248.141.194 89.248.141.222 0x8000003c 1701 0x2 0xe75a
32
ASExt-2 *89.248.142.128 89.248.141.221 0x8000003a 645 0x2 0x2792
36
ASExt-2 0.0.0.0 89.248.141.223 0x80000f82 1700 0x2 0xfdba
36
Network *89.248.141.198 89.248.141.221 0x80000001 409 0x2 0x2458
32
root at lodgefarm> show route table ipv4 unicast final
89.248.141.224/27 [connected(0)/0]
> via ath1/ath1
89.248.142.128/29 [static(1)/10]
> to 89.248.141.221 via lo/lo
89.248.141.196/30 [connected(0)/0]
> via ath0/ath0
89.248.141.221/32 [connected(0)/0]
> via lo/lo
root at lodgefarm> show interfaces
ath0/ath0: Flags:<ENABLED,BROADCAST,MULTICAST> mtu 1500
inet 89.248.141.198 subnet 89.248.141.196/30 broadcast
89.248.141.199
physical index 7
ether 0:15:6d:53:25:5c
ath1/ath1: Flags:<ENABLED,BROADCAST,MULTICAST> mtu 1500
inet 89.248.141.225 subnet 89.248.141.224/27 broadcast
89.248.141.255
physical index 8
ether 0:15:6d:54:31:b2
lo/lo: Flags:<ENABLED,LOOPBACK> mtu 16436
inet 89.248.141.221 subnet 89.248.141.221/32
physical index 1
root at lodgefarm> show route table ipv4 unicast final
89.248.141.224/27 [connected(0)/0]
> via ath1/ath1
89.248.142.128/29 [static(1)/10]
> to 89.248.141.221 via lo/lo
89.248.141.196/30 [connected(0)/0]
> via ath0/ath0
89.248.141.221/32 [connected(0)/0]
> via lo/lo
root at lodgefarm> show ospf4 neighbor detail
Address Interface State ID Pri
Dead
89.248.141.197 ath0/ath0 Full 89.248.141.222 128
36
Area 0.0.0.0, opt 0x2, DR 89.248.141.197, BDR 0.0.0.0
Up 28:58:12, adjacent 00:08:32
root at lodgefarm> show ospf4 database detail
OSPF link state database, Area 0.0.0.0
Router-LSA:
LS age 528 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x1 Link
State ID 89.248.141.221 Advertising Router 89.248.141.221 LS sequence
number 0x80000807 LS checksum 0x9516 length 48
bit Nt false
bit V false
bit E true
bit B false
Type 2 Transit network IP address of Designated router
89.248.141.198 Routers interface address 89.248.141.198 Metric 1
Type 3 Stub network Subnet number 89.248.141.221 Mask
255.255.255.255 Metric 1
As-External-LSA:
LS age 764 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x5 Link
State ID 89.248.141.224 Advertising Router 89.248.141.221 LS sequence
number 0x8000003a LS checksum 0x7903 length 36
Network Mask 0xffffffe0
bit E true
Metric 0 0
Forwarding address 89.248.141.221
External Route Tag 0
Router-LSA:
LS age 524 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x1 Link
State ID 89.248.141.222 Advertising Router 89.248.141.222 LS sequence
number 0x800008f0 LS checksum 0x77f5 length 60
bit Nt false
bit V false
bit E false
bit B false
Type 2 Transit network IP address of Designated router
89.248.141.194 Routers interface address 89.248.141.194 Metric 1
Type 3 Stub network Subnet number 89.248.141.222 Mask
255.255.255.255 Metric 1
Type 2 Transit network IP address of Designated router
89.248.141.197 Routers interface address 89.248.141.197 Metric 1
Network-LSA:
LS age 528 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x2 Link
State ID 89.248.141.197 Advertising Router 89.248.141.222 LS sequence
number 0x8000003b LS checksum 0xaf92 length 32
Network Mask 0xfffffffc
Attached Router 89.248.141.222
Attached Router 89.248.141.221
Router-LSA:
LS age 10 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x1 Link
State ID 89.248.141.223 Advertising Router 89.248.141.223 LS sequence
number 0x80000fc6 LS checksum 0x9e49 length 48
bit Nt false
bit V false
bit E true
bit B false
Type 2 Transit network IP address of Designated router
89.248.141.194 Routers interface address 89.248.141.193 Metric 1
Type 3 Stub network Subnet number 89.248.141.223 Mask
255.255.255.255 Metric 1
Network-LSA:
LS age 20 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x2 Link
State ID 89.248.141.194 Advertising Router 89.248.141.222 LS sequence
number 0x8000003d LS checksum 0xe55b length 32
Network Mask 0xfffffffc
Attached Router 89.248.141.222
Attached Router 89.248.141.223
As-External-LSA:
LS age 764 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x5 Link
State ID 89.248.142.128 Advertising Router 89.248.141.221 LS sequence
number 0x8000003a LS checksum 0x2792 length 36
Network Mask 0xfffffff8
bit E true
Metric 10 0xa
Forwarding address 89.248.141.221
External Route Tag 0
Network-LSA:
LS age 529 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x2 Link
State ID 89.248.141.198 Advertising Router 89.248.141.221 LS sequence
number 0x80000001 LS checksum 0x2458 length 32
Network Mask 0xfffffffc
Attached Router 89.248.141.221
Attached Router 89.248.141.222
As-External-LSA:
LS age 19 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x5 Link
State ID 0.0.0.0 Advertising Router 89.248.141.223 LS sequence number
0x80000f83 LS checksum 0xfbbb length 36
Network Mask 0
bit E true
Metric 10 0xa
Forwarding address 89.248.141.223
External Route Tag 0
root at lodgefarm> exit
lodgefarm:/usr/local/xorp/ospf/tools# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
89.248.141.239 * 255.255.255.255 UH 0 0 0
ppp13
89.248.141.254 * 255.255.255.255 UH 0 0 0
ppp27
89.248.141.237 * 255.255.255.255 UH 0 0 0
ppp23
89.248.141.253 * 255.255.255.255 UH 0 0 0
ppp15
89.248.141.252 * 255.255.255.255 UH 0 0 0
ppp11
89.248.141.236 * 255.255.255.255 UH 0 0 0
ppp7
89.248.141.251 * 255.255.255.255 UH 0 0 0
ppp3
89.248.141.234 * 255.255.255.255 UH 0 0 0
ppp16
89.248.141.250 * 255.255.255.255 UH 0 0 0
ppp22
89.248.141.233 * 255.255.255.255 UH 0 0 0
ppp14
89.248.141.249 * 255.255.255.255 UH 0 0 0
ppp1
89.248.141.232 * 255.255.255.255 UH 0 0 0
ppp12
89.248.141.248 * 255.255.255.255 UH 0 0 0
ppp10
89.248.141.231 * 255.255.255.255 UH 0 0 0
ppp26
89.248.141.247 * 255.255.255.255 UH 0 0 0
ppp4
89.248.141.230 * 255.255.255.255 UH 0 0 0
ppp19
89.248.141.246 * 255.255.255.255 UH 0 0 0
ppp17
89.248.141.229 * 255.255.255.255 UH 0 0 0
ppp18
89.248.141.245 * 255.255.255.255 UH 0 0 0
ppp5
89.248.141.244 * 255.255.255.255 UH 0 0 0
ppp6
89.248.141.228 * 255.255.255.255 UH 0 0 0
ppp2
89.248.141.227 * 255.255.255.255 UH 0 0 0
ppp24
89.248.141.243 * 255.255.255.255 UH 0 0 0
ppp21
89.248.141.242 * 255.255.255.255 UH 0 0 0
ppp8
89.248.141.241 * 255.255.255.255 UH 0 0 0
ppp20
89.248.141.196 * 255.255.255.252 U 0 0 0
ath0
89.248.142.128 89.248.141.221 255.255.255.248 UG 10 0 0
lo
89.248.141.224 * 255.255.255.224 U 0 0 0
ath1
10.20.30.0 * 255.255.255.0 U 0 0 0
eth0
lodgefarm:/usr/local/xorp/ospf/tools# xorpsh
Welcome to XORP on lodgefarm
root at lodgefarm> show ospf4 database
OSPF link state database, Area 0.0.0.0
Type ID Adv Rtr Seq Age Opt Cksum
Len
Router *89.248.141.221 89.248.141.221 0x80000807 723 0x2 0x9516
48
ASExt-2 *89.248.141.224 89.248.141.221 0x8000003a 959 0x2 0x7903
36
Router 89.248.141.222 89.248.141.222 0x800008f0 719 0x2 0x77f5
60
Network 89.248.141.197 89.248.141.222 0x8000003b 723 0x2 0xaf92
32
Router 89.248.141.223 89.248.141.223 0x80000fc6 204 0x2 0x9e49
48
Network 89.248.141.194 89.248.141.222 0x8000003d 215 0x2 0xe55b
32
ASExt-2 *89.248.142.128 89.248.141.221 0x8000003a 959 0x2 0x2792
36
Network *89.248.141.198 89.248.141.221 0x80000001 723 0x2 0x2458
32
ASExt-2 0.0.0.0 89.248.141.223 0x80000f83 214 0x2 0xfbbb
36
root at lodgefarm> show ospf4 database
OSPF link state database, Area 0.0.0.0
Type ID Adv Rtr Seq Age Opt Cksum
Len
Router *89.248.141.221 89.248.141.221 0x80000807 742 0x2 0x9516
48
ASExt-2 *89.248.141.224 89.248.141.221 0x8000003a 978 0x2 0x7903
36
Router 89.248.141.222 89.248.141.222 0x800008f0 738 0x2 0x77f5
60
Network 89.248.141.197 89.248.141.222 0x8000003b 742 0x2 0xaf92
32
Router 89.248.141.223 89.248.141.223 0x80000fc6 224 0x2 0x9e49
48
Network 89.248.141.194 89.248.141.222 0x8000003d 234 0x2 0xe55b
32
ASExt-2 *89.248.142.128 89.248.141.221 0x8000003a 978 0x2 0x2792
36
Network *89.248.141.198 89.248.141.221 0x80000001 743 0x2 0x2458
32
ASExt-2 0.0.0.0 89.248.141.223 0x80000f83 233 0x2 0xfbbb
36
root at lodgefarm> exit
lodgefarm:/usr/local/xorp/ospf/tools# tcpdump -i ath0 proto ospf
tcpdump: verbose output suppressed, use -v or -vv for full protocol
decode
listening on ath0, link-type EN10MB (Ethernet), capture size 96 bytes
17:03:17.694436 IP 89.248.141.198 > 224.0.0.5: OSPFv2, Hello, length: 48
17:03:21.899777 IP 89.248.141.197 > 224.0.0.5: OSPFv2, Hello, length: 48
17:03:27.786449 IP 89.248.141.198 > 224.0.0.5: OSPFv2, Hello, length: 48
17:03:31.896206 IP 89.248.141.197 > 224.0.0.5: OSPFv2, Hello, length: 48
17:03:37.694828 IP 89.248.141.198 > 224.0.0.5: OSPFv2, Hello, length: 48
17:03:41.896639 IP 89.248.141.197 > 224.0.0.5: OSPFv2, Hello, length: 48
6 packets captured
6 packets received by filter
0 packets dropped by kernel
lodgefarm:/usr/local/xorp/ospf/tools# xorpsh
Welcome to XORP on lodgefarm
root at lodgefarm> show ospf4 database
OSPF link state database, Area 0.0.0.0
Type ID Adv Rtr Seq Age Opt Cksum
Len
Router *89.248.141.221 89.248.141.221 0x80000807 813 0x2 0x9516
48
ASExt-2 *89.248.141.224 89.248.141.221 0x8000003a 1049 0x2 0x7903
36
Router 89.248.141.222 89.248.141.222 0x800008f0 809 0x2 0x77f5
60
Network 89.248.141.197 89.248.141.222 0x8000003b 813 0x2 0xaf92
32
Router 89.248.141.223 89.248.141.223 0x80000fc6 294 0x2 0x9e49
48
Network 89.248.141.194 89.248.141.222 0x8000003d 305 0x2 0xe55b
32
ASExt-2 *89.248.142.128 89.248.141.221 0x8000003a 1049 0x2 0x2792
36
Network *89.248.141.198 89.248.141.221 0x80000001 813 0x2 0x2458
32
ASExt-2 0.0.0.0 89.248.141.223 0x80000f83 304 0x2 0xfbbb
36
root at lodgefarm> show ospf4 database detail
OSPF link state database, Area 0.0.0.0
Router-LSA:
LS age 828 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x1 Link
State ID 89.248.141.221 Advertising Router 89.248.141.221 LS sequence
number 0x80000807 LS checksum 0x9516 length 48
bit Nt false
bit V false
bit E true
bit B false
Type 2 Transit network IP address of Designated router
89.248.141.198 Routers interface address 89.248.141.198 Metric 1
Type 3 Stub network Subnet number 89.248.141.221 Mask
255.255.255.255 Metric 1
As-External-LSA:
LS age 1064 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x5 Link
State ID 89.248.141.224 Advertising Router 89.248.141.221 LS sequence
number 0x8000003a LS checksum 0x7903 length 36
Network Mask 0xffffffe0
bit E true
Metric 0 0
Forwarding address 89.248.141.221
External Route Tag 0
Router-LSA:
LS age 824 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x1 Link
State ID 89.248.141.222 Advertising Router 89.248.141.222 LS sequence
number 0x800008f0 LS checksum 0x77f5 length 60
bit Nt false
bit V false
bit E false
bit B false
Type 2 Transit network IP address of Designated router
89.248.141.194 Routers interface address 89.248.141.194 Metric 1
Type 3 Stub network Subnet number 89.248.141.222 Mask
255.255.255.255 Metric 1
Type 2 Transit network IP address of Designated router
89.248.141.197 Routers interface address 89.248.141.197 Metric 1
Network-LSA:
LS age 828 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x2 Link
State ID 89.248.141.197 Advertising Router 89.248.141.222 LS sequence
number 0x8000003b LS checksum 0xaf92 length 32
Network Mask 0xfffffffc
Attached Router 89.248.141.222
Attached Router 89.248.141.221
Router-LSA:
LS age 309 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x1 Link
State ID 89.248.141.223 Advertising Router 89.248.141.223 LS sequence
number 0x80000fc6 LS checksum 0x9e49 length 48
bit Nt false
bit V false
bit E true
bit B false
Type 2 Transit network IP address of Designated router
89.248.141.194 Routers interface address 89.248.141.193 Metric 1
Type 3 Stub network Subnet number 89.248.141.223 Mask
255.255.255.255 Metric 1
Network-LSA:
LS age 320 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x2 Link
State ID 89.248.141.194 Advertising Router 89.248.141.222 LS sequence
number 0x8000003d LS checksum 0xe55b length 32
Network Mask 0xfffffffc
Attached Router 89.248.141.222
Attached Router 89.248.141.223
As-External-LSA:
LS age 1064 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x5 Link
State ID 89.248.142.128 Advertising Router 89.248.141.221 LS sequence
number 0x8000003a LS checksum 0x2792 length 36
Network Mask 0xfffffff8
root at lodgefarm> /etc/init.d/xorp restart
^
unknown command.
root at lodgefarm> exit
lodgefarm:/usr/local/xorp/ospf/tools# /etc/init.d/xorp restart
Stopping xorp: Stopping xorp
Starting xorp: Starting xorp
lodgefarm:/usr/local/xorp/ospf/tools# xorpsh
Welcome to XORP on lodgefarm
root at lodgefarm> show ospf4 database
OSPF link state database, Area 0.0.0.0
Type ID Adv Rtr Seq Age Opt Cksum
Len
Router *89.248.141.221 89.248.141.221 0x80000001 4 0x2 0xf86b
48
ASExt-2 *89.248.141.224 89.248.141.221 0x80000001 4 0x2 0xebc9
36
ASExt-2 *89.248.142.128 89.248.141.221 0x80000001 4 0x2 0x9959
36
Router 89.248.141.222 89.248.141.222 0x800008f2 4 0x2 0x73f7
60
Network 89.248.141.197 89.248.141.222 0x80000001 4 0x2 0x2458
32
root at lodgefarm> show ospf4 database
OSPF link state database, Area 0.0.0.0
Type ID Adv Rtr Seq Age Opt Cksum
Len
Router *89.248.141.221 89.248.141.221 0x80000809 1 0x2 0x8723
48
ASExt-2 *89.248.141.224 89.248.141.221 0x8000003b 8 0x2 0x7704
36
ASExt-2 *89.248.142.128 89.248.141.221 0x8000003b 8 0x2 0x2593
36
Router 89.248.141.222 89.248.141.222 0x800008f2 7 0x2 0x73f7
60
Network 89.248.141.197 89.248.141.222 0x80000001 7 0x2 0x2458
32
Router 89.248.141.223 89.248.141.223 0x80000fc6 396 0x2 0x9e49
48
Network 89.248.141.194 89.248.141.222 0x8000003d 406 0x2 0xe55b
32
ASExt-2 0.0.0.0 89.248.141.223 0x80000f83 405 0x2 0xfbbb
36
root at lodgefarm> show ospf4 database
OSPF link state database, Area 0.0.0.0
Type ID Adv Rtr Seq Age Opt Cksum
Len
Router *89.248.141.221 89.248.141.221 0x80000809 5 0x2 0x8723
48
ASExt-2 *89.248.141.224 89.248.141.221 0x8000003b 12 0x2 0x7704
36
ASExt-2 *89.248.142.128 89.248.141.221 0x8000003b 12 0x2 0x2593
36
Router 89.248.141.222 89.248.141.222 0x800008f2 11 0x2 0x73f7
60
Network 89.248.141.197 89.248.141.222 0x80000001 11 0x2 0x2458
32
Router 89.248.141.223 89.248.141.223 0x80000fc6 400 0x2 0x9e49
48
Network 89.248.141.194 89.248.141.222 0x8000003d 410 0x2 0xe55b
32
ASExt-2 0.0.0.0 89.248.141.223 0x80000f83 409 0x2 0xfbbb
36
root at lodgefarm> show ospf4 database detail
OSPF link state database, Area 0.0.0.0
Router-LSA:
LS age 18 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x1 Link
State ID 89.248.141.221 Advertising Router 89.248.141.221 LS sequence
number 0x80000809 LS checksum 0x8723 length 48
bit Nt false
bit V false
bit E true
bit B false
Type 2 Transit network IP address of Designated router
89.248.141.197 Routers interface address 89.248.141.198 Metric 1
Type 3 Stub network Subnet number 89.248.141.221 Mask
255.255.255.255 Metric 1
As-External-LSA:
LS age 25 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x5 Link
State ID 89.248.141.224 Advertising Router 89.248.141.221 LS sequence
number 0x8000003b LS checksum 0x7704 length 36
Network Mask 0xffffffe0
bit E true
Metric 0 0
Forwarding address 89.248.141.221
External Route Tag 0
As-External-LSA:
LS age 25 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x5 Link
State ID 89.248.142.128 Advertising Router 89.248.141.221 LS sequence
number 0x8000003b LS checksum 0x2593 length 36
Network Mask 0xfffffff8
bit E true
Metric 10 0xa
Forwarding address 89.248.141.221
External Route Tag 0
Router-LSA:
LS age 25 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x1 Link
State ID 89.248.141.222 Advertising Router 89.248.141.222 LS sequence
number 0x800008f2 LS checksum 0x73f7 length 60
bit Nt false
bit V false
bit E false
bit B false
Type 2 Transit network IP address of Designated router
89.248.141.194 Routers interface address 89.248.141.194 Metric 1
Type 3 Stub network Subnet number 89.248.141.222 Mask
255.255.255.255 Metric 1
Type 2 Transit network IP address of Designated router
89.248.141.197 Routers interface address 89.248.141.197 Metric 1
Network-LSA:
LS age 25 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x2 Link
State ID 89.248.141.197 Advertising Router 89.248.141.222 LS sequence
number 0x80000001 LS checksum 0x2458 length 32
Network Mask 0xfffffffc
Attached Router 89.248.141.222
Attached Router 89.248.141.221
Router-LSA:
LS age 414 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x1 Link
State ID 89.248.141.223 Advertising Router 89.248.141.223 LS sequence
number 0x80000fc6 LS checksum 0x9e49 length 48
bit Nt false
bit V false
bit E true
bit B false
Type 2 Transit network IP address of Designated router
89.248.141.194 Routers interface address 89.248.141.193 Metric 1
Type 3 Stub network Subnet number 89.248.141.223 Mask
255.255.255.255 Metric 1
Network-LSA:
LS age 424 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x2 Link
State ID 89.248.141.194 Advertising Router 89.248.141.222 LS sequence
number 0x8000003d LS checksum 0xe55b length 32
Network Mask 0xfffffffc
Attached Router 89.248.141.222
Attached Router 89.248.141.223
As-External-LSA:
LS age 423 Options 0x2 DC: 0 EA: 0 N/P: 0 MC: 0 E: 1 LS type 0x5 Link
State ID 0.0.0.0 Advertising Router 89.248.141.223 LS sequence number
0x80000f83 LS checksum 0xfbbb length 36
Network Mask 0
bit E true
Metric 10 0xa
Forwarding address 89.248.141.223
External Route Tag 0
root at lodgefarm> show ospf4 neighbor det
Possible completions:
<neighbor> Show Neighbors
detail Show Neighbors
root at lodgefarm> show ospf4 neighbor detail
Address Interface State ID Pri
Dead
89.248.141.197 ath0/ath0 Full 89.248.141.222 128
33
Area 0.0.0.0, opt 0x2, DR 89.248.141.197, BDR 89.248.141.198
Up 00:00:36, adjacent 00:00:31
root at lodgefarm> exit
lodgefarm:/usr/local/xorp/ospf/tools# xorpsh
shWelcome to XORP on lodgefarm
root at lodgefarm> show interfaces
ath0/ath0: Flags:<ENABLED,BROADCAST,MULTICAST> mtu 1500
inet 89.248.141.198 subnet 89.248.141.196/30 broadcast
89.248.141.199
physical index 7
ether 0:15:6d:53:25:5c
ath1/ath1: Flags:<ENABLED,BROADCAST,MULTICAST> mtu 1500
inet 89.248.141.225 subnet 89.248.141.224/27 broadcast
89.248.141.255
physical index 8
ether 0:15:6d:54:31:b2
lo/lo: Flags:<ENABLED,LOOPBACK> mtu 16436
inet 89.248.141.221 subnet 89.248.141.221/32
physical index 1
root at lodgefarm> exit
lodgefarm:/usr/local/xorp/ospf/tools# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
89.248.141.222 89.248.141.197 255.255.255.255 UGH 2 0 0
ath0
89.248.141.239 * 255.255.255.255 UH 0 0 0
ppp13
89.248.141.238 * 255.255.255.255 UH 0 0 0
ppp25
89.248.141.223 89.248.141.197 255.255.255.255 UGH 3 0 0
ath0
89.248.141.254 * 255.255.255.255 UH 0 0 0
ppp27
89.248.141.237 * 255.255.255.255 UH 0 0 0
ppp23
89.248.141.253 * 255.255.255.255 UH 0 0 0
ppp15
89.248.141.252 * 255.255.255.255 UH 0 0 0
ppp11
89.248.141.236 * 255.255.255.255 UH 0 0 0
ppp7
89.248.141.235 * 255.255.255.255 UH 0 0 0
ppp9
89.248.141.251 * 255.255.255.255 UH 0 0 0
ppp3
89.248.141.234 * 255.255.255.255 UH 0 0 0
ppp16
89.248.141.250 * 255.255.255.255 UH 0 0 0
ppp22
89.248.141.233 * 255.255.255.255 UH 0 0 0
ppp14
89.248.141.249 * 255.255.255.255 UH 0 0 0
ppp1
89.248.141.232 * 255.255.255.255 UH 0 0 0
ppp12
89.248.141.248 * 255.255.255.255 UH 0 0 0
ppp10
89.248.141.231 * 255.255.255.255 UH 0 0 0
ppp26
89.248.141.247 * 255.255.255.255 UH 0 0 0
ppp4
89.248.141.230 * 255.255.255.255 UH 0 0 0
ppp19
89.248.141.246 * 255.255.255.255 UH 0 0 0
ppp17
89.248.141.229 * 255.255.255.255 UH 0 0 0
ppp18
89.248.141.245 * 255.255.255.255 UH 0 0 0
ppp5
89.248.141.244 * 255.255.255.255 UH 0 0 0
ppp6
89.248.141.228 * 255.255.255.255 UH 0 0 0
ppp2
89.248.141.227 * 255.255.255.255 UH 0 0 0
ppp24
89.248.141.243 * 255.255.255.255 UH 0 0 0
ppp21
89.248.141.242 * 255.255.255.255 UH 0 0 0
ppp8
89.248.141.241 * 255.255.255.255 UH 0 0 0
ppp20
89.248.141.240 * 255.255.255.255 UH 0 0 0
ppp0
89.248.141.196 * 255.255.255.252 U 0 0 0
ath0
89.248.141.192 89.248.141.197 255.255.255.252 UG 2 0 0
ath0
89.248.142.128 89.248.141.221 255.255.255.248 UG 10 0 0
lo
89.248.141.224 * 255.255.255.224 U 0 0 0
ath1
10.20.30.0 * 255.255.255.0 U 0 0 0
eth0
default 89.248.141.197 0.0.0.0 UG 3 0 0
ath0
lodgefarm:/usr/local/xorp/ospf/tools#
On Wed, 2007-12-05 at 01:00 -0800, Atanu Ghosh wrote:
> Hi,
>
> The output that it would be good to see before and after the problem
> occurs.
> 1) $ netstat -nr
> 2) Xorp> show interfaces
> 3) Xorp> show route table ipv4 unicast final
> 4) Xorp> show ospf4 neighbor detail
> 5) Xorp> show ospf4 database detail
> 6) $ print_lsas -S save.lsas
> The print_lsas program can be found in ospf/tools directory. The program
> stores the LSA database in a form that can be replayed.
>
> You can also enable tracing in ospf:
> traceoptions {
> flag {
> all {
> disable: false
> }
> }
> }
>
> Which should show routes being added and deleted.
>
> The latest code in CVS has a "clear ospf4 database" command, it would be
> interesting to know if once the problem occurs if this solves the
> problem.
>
> It might also be interesting to keep the "ip mon" command running to
> track routes being added and deleted.
>
> Would it be possible at some off peak time to flap the ADSL link to see
> if this replicates the problem. I know that you have stated that there
> were no ADSL issues when the problem occurred, but I do wonder if we are
> seeing some issue related to dynamic interfaces.
>
> Atanu.
>
> >>>>> "Aidan" == Aidan Walton <awalton at wires3.net> writes:
>
> Aidan> Hi, The adjacency runs over a wireless link between the
> Aidan> routers. It can, very possibly, drop in and out, but as far
> Aidan> as I can see this did not happen and to be honest in the 9
> Aidan> months I have had this system up I have never seen the
> Aidan> wireless link drop, but packet corruption could be a
> Aidan> possibility and this may be less easy to diagnose. It is a
> Aidan> high power 5.8GHz connection, here in the UK this is a
> Aidan> licensed band (and yes I have a license). So I don't think
> Aidan> interference is the likely cause, though I wouldn't rule this
> Aidan> out. If I look at the logs from the same period I seen
> Aidan> nothing to indicate the interface flapped, I would see the
> Aidan> wireless dis-associate and re-associate and cypher exchange
> Aidan> and this did not happen. But as I say there could be a period
> Aidan> of high BER on the links. I thought ospf would handle this
> Aidan> reasonably gracefully? I have to say heavy BER was not
> Aidan> evident when I came to repair the network, or at least I
> Aidan> didn't detect it and in the past I have run ospf over another
> Aidan> one of my wireless links with stations 10km apart with the
> Aidan> wireless link almost non-functional, dropping packets left
> Aidan> right and centre and re-associating over and over, but xorp's
> Aidan> ospf never complained! I was beginning to suspect that this
> Aidan> was related to my adsl link on the suspect router, as this is
> Aidan> a dynamic interface and I have this defined independently of
> Aidan> xorp. If this interface flaps then the default route
> Aidan> associated with the adsl ppp session is withdrawn. The
> Aidan> default from the adsl line is not propagated into ospf
> Aidan> though, instead I use a static default with a higher metric
> Aidan> pointed at the loopback and inject this into ospf
> Aidan> instead. Then the flaps of the adsl line do not cause churn
> Aidan> in the ospf domain. I was starting to think that the addition
> Aidan> and removal of the default from the adsl line was affecting
> Aidan> the kernel table and this was upsetting xorp's ospf. However
> Aidan> this morning when this happened the adsl line was stable. As
> Aidan> far as my logs look it suddenly decided to stop functioning
> Aidan> with no correlated events from other system processes. The
> Aidan> only things in the logs at the same time is iptables dropping
> Aidan> DOS attacks, but this in normal, unfortunately far to normal.
> Aidan> show ospf4 neighbour simply stated 'full' there is only one
> Aidan> neighbour defined on this router. I didn't look this time at
> Aidan> show interfaces, but from memory of the last time this
> Aidan> happened this also was normal. The problem is that these
> Aidan> routers are mounted 10m high up telegraph poles. If I loose
> Aidan> connectivity it requires a ladder and a climbing harness to
> Aidan> get at them, this is not to mention my upset customers who,
> Aidan> as is normal with customers, do not delay in telling me they
> Aidan> have lost their Internet links. I suppose what I'm trying to
> Aidan> understand is how to be best prepared for next time, logging,
> Aidan> processes and checks during the failure period to grab as
> Aidan> much useful info before I am forced to restart xorp and get
> Aidan> my customers up and running again. This is a very short
> Aidan> period I have to say. I have a small group of business units
> Aidan> supported on this router and all hell breaks loose if this
> Aidan> happens during working hours. How can I get the maximum
> Aidan> logging info from the xorp processes? Anything I can do in
> Aidan> order that you can help me, will be dutifully carried
> Aidan> out. What next, any suggestions? Thanks Aidan I will On Tue,
> Aidan> 2007-12-04 at 12:19 -0800, Atanu Ghosh wrote:
>
> Atanu> Hi,
>
> Atanu> The scenario that you describe would be perfectly normal if
> Atanu> the connectivity between the "suspect" router and the
> Atanu> "adjacent" router is lost. Although I would expect the "show
> Atanu> ospf4 neighbor" to show the state of the adjacency to be
> Atanu> "Down" not "Full". When an OSPF router loses its adjancencies
> Atanu> the LSA database will slowly timeout, however, the routes
> Atanu> will be withdrawn as soon as the adjacencies are lost.
>
> Atanu> We will require more information to diagnose the problem next
> Atanu> time the problem occurs the output of "show interfaces" and
> Atanu> "show ospf4 neighbor" would be very useful.
>
> Atanu> XORP tracks the state of interfaces in particular the carrier
> Atanu> state. If OSPF believes that the Ethernet has been
> Atanu> disconnected it will stop attempting to send hello
> Atanu> packets. Is it possible that there is a problem with an
> Atanu> interface or cable between the two routers?
>
> Atanu> Atanu.
>
> >>>>> "Aidan" == Aidan Walton <awalton at wires3.net> writes:
>
> Aidan> Hi All, I am using xorp in a production environment,
> Aidan> admittedly a small one. I operate a local WISP and xorp is
> Aidan> running on my wireless nodes. I have a very simple
> Aidan> configuration and really I could probably get away with
> Aidan> static routing throughout the entire network, but I wanted to
> Aidan> try xorp and see just how stable it was. However as I expand
> Aidan> the network I am having second thoughts. It is not good at
> Aidan> all when a network goes up in smoke and I can't explain why
> Aidan> or predict when and what the causes are. The network has
> Aidan> been in operation 24x7 for around 9 months. I am running on a
> Aidan> Linux kernel 2.6.18-4 and for the vast majority of the time I
> Aidan> have no issues. However now for the fourth time I see the
> Aidan> same problem: Suddenly the Linux kernel and the xorp rib
> Aidan> become detached. Normally all routes in the kernel match
> Aidan> those that xorp is generating, receiving and electing as
> Aidan> active. I am running OSPF and the neighbour states remain
> Aidan> 'full' throughout but if I am not mistaken I see ospf hellos
> Aidan> only in one direction (i.e nothing being transmitted from the
> Aidan> router I suspect). The lsdb of OSPF on the suspect and
> Aidan> adjacent routers contain all the routes but they are aging
> Aidan> out slowly on the adjacent router. When I look at the kernel
> Aidan> routes those from OSPF have already vanished. I can see the
> Aidan> ospf process running on the offending router? and again I can
> Aidan> see the ospf lsdb intact and correct. When I restart xorp the
> Aidan> system recovers and the routes appear in the kernel again. I
> Aidan> suspect a problem with ospf. I tried enabling traceoptions on
> Aidan> the ospf process, but in fact I needed to restart all the
> Aidan> xorp processes before this actually became active. I now have
> Aidan> this running so if/when it happens again I might be able to
> Aidan> offer some more information. Does anyone have any experience
> Aidan> of ospf begin unstable? any suggestions how I might more
> Aidan> effectively capture some logs from this event. I do not see
> Aidan> any options for logging the fea process. Is there anything I
> Aidan> can enable to help diagnose the issue? Many thanks, and of
> Aidan> course cheers for the code in the first place. Aidan
> Aidan> _______________________________________________ Xorp-users
> Aidan> mailing list Xorp-users at xorp.org
> Aidan> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/xorp-users
-------------- next part --------------
A non-text attachment was scrubbed...
Name: save.lsas.lodgefarm-5-12-07
Type: application/octet-stream
Size: 481 bytes
Desc: not available
Url : http://mailman.ICSI.Berkeley.EDU/pipermail/xorp-users/attachments/20071205/7185b20d/attachment-0001.obj
More information about the Xorp-users
mailing list