OVS flows missing from some nodes on ODL/Openstack after restart

asked 2016-11-16 06:18:00 -0800

ashleyd gravatar image

updated 2016-11-16 08:20:59 -0800

HI,

I'm currently having an issue with ODL and OVS after a restart of the openstack environment. After restart everything comes back up but the flows and tables in OVS on some nodes are incorrect for instance

Node with flows

2016-11-16T11:31:40.101Z|00016|rconn|INFO|br-int<->tcp:192.168.0.7:6653: connecting...
2016-11-16T11:31:40.221Z|00017|rconn|INFO|br-int<->tcp:192.168.0.7:6653: connected
2016-11-16T11:31:50.106Z|00018|memory|INFO|peak resident set size grew 332% in last 66.5 seconds, from 7712 kB to 33320 kB
2016-11-16T11:31:50.106Z|00019|memory|INFO|handlers:1 ofconns:1 ports:1 revalidators:1 rules:16
2016-11-16T11:31:56.484Z|00020|connmgr|INFO|br-int<->tcp:192.168.0.7:6653: 12 flow_mods 10 s ago (12 adds)

node without

2016-11-16T11:31:40.818Z|00020|rconn|INFO|br-int<->tcp:192.168.0.7:6653: connecting...
2016-11-16T11:31:40.825Z|00021|rconn|INFO|br-int<->tcp:192.168.0.7:6653: connected
2016-11-16T11:31:49.805Z|00022|memory|INFO|peak resident set size grew 332% in last 66.1 seconds, from 7712 kB to 33320 kB
2016-11-16T11:31:49.806Z|00023|memory|INFO|handlers:1 ofconns:1 ports:1 revalidators:1 rules:4

Not sure what the corresponding odl component is and can't see anything obvious in the karaf logs, all nodes do have a connection to the ODL controller and the ODL controller shows all nodes in ui


Is there a log I should look to to see if odl is sending the flows to the nodes, I can't see it in karaf for any nodes (using openflow:ID to search)?

One error in ODL is relating to openstack instances, which is strange because this instance has been deleted before the restart

2016-11-16 11:31:39,969 | ERROR | ntDispatcherImpl | TenantNetworkManagerImpl         | 298 - org.opendaylight.netvirt.openstack.net-virt - 1.2.4.Beryllium-SR3 | getNode: br-int interface is not found for node:ovsdb://uuid/0d170698-3dad-4eb7-8676-cad4c3a7b0a0
2016-11-16 11:31:39,969 | ERROR | ntDispatcherImpl | SouthboundImpl                   | 298 - org.opendaylight.netvirt.openstack.net-virt - 1.2.4.Beryllium-SR3 | extractTerminationPointAugmentations: Node value is null
2016-11-16 11:31:39,969 | ERROR | ntDispatcherImpl | SouthboundImpl                   | 298 - org.opendaylight.netvirt.openstack.net-virt - 1.2.4.Beryllium-SR3 | readTerminationPointAugmentations: Node value is null
 2016-11-16 11:31:39,969 | INFO  | ntDispatcherImpl | TenantNetworkManagerImpl         | 298 - org.opendaylight.netvirt.openstack.net-virt - 1.2.4.Beryllium-SR3 | no node found for port:NeutronPort [portUUID=b3a04683-f4b7-4d56-a477-36deab25490b, networkUUID=bf41c322-812c-495f-93ec-f184a6189f4c, name=, adminStateUp=true, status=null, macAddress=fa:16:3e:f8:64:fd, fixedIPs=[Neutron_IPs{ipAddress='192.168.111.8', subnetUUID='6bfe7912-8369-4c20-a5bf-8b3465613aba'}], deviceID=afb040cd-2584-433c-acaf-7104f6002375, deviceOwner=compute:None, tenantID=cdca7c2ac48441aaaf926f141a6c9985, securityGroups=[NeutronSecurityGroup{securityGroupUUID='0fb02d5e-8a19-440c-a27c-3c845d617928', securityGroupName='default', securityGroupDescription='Default security group', securityGroupTenantID='cdca7c2ac48441aaaf926f141a6c9985', securityRules=[NeutronSecurityRule{securityRuleUUID='0faf6709-4ec8-40d3-bf7e-8bbe00907299', securityRuleDirection='egress', securityRuleProtocol='null', securityRulePortMin=null, securityRulePortMax=null, securityRuleEthertype='IPv4', securityRuleRemoteIpPrefix='null', securityRemoteGroupID=null, securityRuleGroupID='0fb02d5e-8a19-440c-a27c-3c845d617928', securityRuleTenantID='cdca7c2ac48441aaaf926f141a6c9985'}, NeutronSecurityRule{securityRuleUUID='6ab3d2cc-7645-4cf4-b742-af077dc9a326', securityRuleDirection='ingress', securityRuleProtocol='null', securityRulePortMin=null, securityRulePortMax=null, securityRuleEthertype='IPv4', securityRuleRemoteIpPrefix='null', securityRemoteGroupID=0fb02d5e-8a19-440c-a27c-3c845d617928, securityRuleGroupID='0fb02d5e-8a19-440c-a27c-3c845d617928', securityRuleTenantID='cdca7c2ac48441aaaf926f141a6c9985'}, NeutronSecurityRule{securityRuleUUID='fa0d1866-5901-4024-9720-d93b97fdfd9f', securityRuleDirection='egress', securityRuleProtocol='null', securityRulePortMin=null, securityRulePortMax=null, securityRuleEthertype='IPv6', securityRuleRemoteIpPrefix='null', securityRemoteGroupID ...
(more)
edit retag flag offensive close merge delete