Flow optimizations - Run flow management in a work-queue

As part of flow processing we need to maintain data structures to keep the flow action in-sync with config changes. Building these changes and also revaluating flows when config change is notified can result in significant latencies. With this change we move the flow management part to a work-queue. The flow management module is resposible to keep the flow in-sync with config changed. Few other slow operations operations such as logging/UVE also will be moved to this module in subsequent commits. Partial-Bug: #1479295 Removing the config listener After IFmap dependency manager is introduced for all the objects in Agent, the config listener does not play any role other than invoking node observers and link observers, which is taken care in dependency manager itself. As part of the same, Uuid change of node is also detected and handled. Couple of test cases are moved out of flaky tests. closes-bug: #1480124 Split flow_table.cc to create new file flow_entry.cc Move FlowEntry methods to new file flow_entry.cc and flow_entry.h No changes in functionality Partial-Bug: #1479295 Move Flow logging to Flow Management module Define a message to enqueue Flow Export requests in Flow Management module. Move FlowExport functionality from FlowTable to Flow Management module. Replace FlowExport API calls in Flow Stats collector and Flow Table with a message to Flow Management module. Partial-Bug: #1479295 Run FlowTable processing from work-queue This change is a step towards running Flow setup in multiple threads. Flow creation is a two step process, FlowHandler : FlowEntry are created and flow action are determined in this context. This stage can potentially run in multiple threads (future commits) FlowHandler runs from a workqueue in "Agent::FlowHandler" task context FlowTable : 1. Manage flow_entry_map_ which contains all flows 2. Enforce the per-VM flow limits 3. Generate events to KSync and FlowMgmt modueles FlowTable runs from a workqueue in "Agent::FlowTable" task context Partial-Bug: #1479295 Optimize packet processing ASIO context Method PktHandler::HandleRcvPkt is called from ASIO context. Following processing was done in HandleRcvPkt within this context, - Decode of the packet including decoding on tunnel headers - In case of bare-metas identification of interface based on MAC address This commit minimizes processing in ASIO context. Packet are enqueued to module work-queue baesd on the agent-header. The packet decode is subsequently done when work-queue is scheduled. Partial-Bug: #1479295 * Track static and floating ip preference based on instance ip 1> Floating ip, static route and allowed address pair in ecmp mode would have preference published based on instance-ip preference 2> If allowed-address pair address is configured in active-stdby mode, run path preference algorithm Test case for same Closes-bug:#1462093,#1461787 (cherry picked from commit 590e031) Define a new property to configure flow export rate in schema. This is a per vrouter config and part of virtual-vrouter stanza. Partial-Bug: #1486850 * Agent changes for TCP connection state awareness for faster flow aging 1> Flow eviction by vrouter If a closed or reset TCP session flow is present in a flow bucket then vrouter could evict the flow and use the slot for a new flow, agent would then delete the evicted flow internally and send a message to delete reverse flow 2> Closes or reset flow would be deleted by agent during aging cycle. 3> If a flow is stuck in SYN state for more than 180 seconds delete the flow. TBD: Update stats for deleted flow Make SYN flow timeout configurable Partial-BUG: #1362701 Run packet-parse routins in ASIO context In an earlier commit f148293, the packet-parse routines were moved out of ASIO context assuming it will improve performance. Further measurements show that, performing packet-parse in ASIO context itself does not affect performance. Hence reverting the change. Partial-Bug: #1479295 Move flow-export-rate config to global-vrouter-config stanza flow-export-rate config is being moved from virtual-router to global-vrouter-config stanza to avoid configuring this parameter on each vrouter. Partial-Bug: #1486850 Export only subset of flows instead of exporting all flows. The subset of flows to be exported are selected from a algorithm. The parameters for the algorithm are (1) configured flow export rate (2) actual flow export rate and (3) sampling threshold. As of today based on timer event, we export certain number of flows. As part of this same timer event we update the actual flow-export rate and sampling threshold parameters. Each flow which needs to be exported is subjected to the following algorithm to decide whether it needs to exported or dropped. (1) Flow samples greater than or equal to sampling threshold will always be exported, with the byte/packet counts reported as-is. (2) Flow samples smaller than the sampling threshold will be exported probabilistically, with the byte/packets counts adjusted upwards according to the probability. (3) Probability = diff_bytes/sampling_threshold (4) We generate a random number less than sampling threshold. (5) If the diff_bytes is greater than random number then the flow is dropped (6) Otherwise the flow is exported after normalizing the diff bytes and packets. The normalization is done by dividing diff_bytes and diff_pkts with probability. This normalization is used as heuristictic to account for stats of dropped flows The actual flow-export-rate will be close to the configured configured export rate and whenever there is huge deviations we adjust sampling threshold to bring the actual flow export rate close to configured flow export rate. It is not guaranteed that the actual flow export rate will always be close to configured flow export rate. Partial-Bug:1486850 Conflicts: src/ksync/ksync_sock_user.cc src/vnsw/agent/oper/global_vrouter.cc src/vnsw/agent/oper/global_vrouter.h src/vnsw/agent/oper/loadbalancer.cc src/vnsw/agent/oper/logical_interface.cc src/vnsw/agent/oper/operdb_init.cc src/vnsw/agent/oper/service_instance.cc src/vnsw/agent/oper/test/instance_manager_test.cc src/vnsw/agent/oper/vm_interface.cc src/vnsw/agent/oper/vrf.h src/vnsw/agent/pkt/SConscript src/vnsw/agent/pkt/agent_stats.cc src/vnsw/agent/pkt/flow_table.cc src/vnsw/agent/pkt/flow_table.h src/vnsw/agent/pkt/pkt.sandesh src/vnsw/agent/pkt/pkt_flow_info.cc src/vnsw/agent/pkt/test/test_pkt_parse.cc src/vnsw/agent/test/service_instance_test.cc src/vnsw/agent/test/test_util.cc src/vnsw/agent/uve/test/test_stats_mock.cc src/vnsw/agent/vrouter/flow_stats/flow_stats_collector.cc src/vnsw/agent/vrouter/flow_stats/flow_stats_collector.h src/vnsw/agent/vrouter/ksync/flowtable_ksync.cc src/vnsw/agent/vrouter/ksync/sandesh_ksync.cc Change-Id: Ieb07b7cd3890c191dc5c1cd0547a4e74caa956be
Juniper · Oct 13, 2015 · 187cf88 · 187cf88
1 parent ec50325
commit 187cf88
Show file tree

Hide file tree

Showing 78 changed files with 8,502 additions and 5,048 deletions.
diff --git a/src/db/db_entry.cc b/src/db/db_entry.cc
@@ -34,10 +34,10 @@ void DBEntryBase::SetState(DBTableBase *tbl_base, ListenerId listener,
     }
 }
 
-DBState *DBEntryBase::GetState(DBTableBase *tbl_base, ListenerId listener) {
+DBState *DBEntryBase::GetState(DBTableBase *tbl_base, ListenerId listener) const {
     DBTablePartBase *tpart = tbl_base->GetTablePartition(this);
     tbb::mutex::scoped_lock lock(tpart->dbstate_mutex());
-    StateMap::iterator loc = state_.find(listener);
+    StateMap::const_iterator loc = state_.find(listener);
     if (loc != state_.end()) {
         return loc->second;
     }

diff --git a/src/db/db_entry.h b/src/db/db_entry.h
@@ -34,7 +34,7 @@ class DBEntryBase {
 
     void SetState(DBTableBase *tbl_base, ListenerId listener, DBState *state);
     void ClearState(DBTableBase *tbl_base, ListenerId listener);
-    DBState *GetState(DBTableBase *tbl_base, ListenerId listener);
+    DBState *GetState(DBTableBase *tbl_base, ListenerId listener) const;
     const DBState *GetState(const DBTableBase *tbl_base,
                             ListenerId listener) const;
     bool is_state_empty(DBTablePartBase *tpart);

diff --git a/src/schema/vnc_cfg.xsd b/src/schema/vnc_cfg.xsd
@@ -607,6 +607,9 @@ targetNamespace="http://www.contrailsystems.com/2012/VNC-CONFIG/0">
                                  type="VxlanNetworkIdentifierModeType"/>
 <!--#IFMAP-SEMANTICS-IDL 
      Property('vxlan-network-identifier-mode', 'global-vrouter-config') -->
+<xsd:element name="flow-export-rate" type="xsd:integer"/>
+<!--#IFMAP-SEMANTICS-IDL
+     Property('flow-export-rate', 'global-vrouter-config') -->
 
 <xsd:element name="domain"        type="ifmap:IdentityType"/>
 <xsd:element name="config-root-domain"/>

diff --git a/src/vnsw/agent/cmn/agent.cc b/src/vnsw/agent/cmn/agent.cc
@@ -101,6 +101,7 @@ void Agent::SetAgentTaskPolicy() {
     initialized = true;
 
     const char *db_exclude_list[] = {
+        "Agent::FlowTable",
         "Agent::FlowHandler",
         "Agent::Services",
         "Agent::StatsCollector",
@@ -113,6 +114,12 @@ void Agent::SetAgentTaskPolicy() {
     SetTaskPolicyOne("db::DBTable", db_exclude_list, 
                      sizeof(db_exclude_list) / sizeof(char *));
 
+    const char *flow_table_exclude_list[] = {
+        AGENT_INIT_TASKNAME
+    };
+    SetTaskPolicyOne("Agent::FlowTable", flow_table_exclude_list,
+                     sizeof(flow_table_exclude_list) / sizeof(char *));
+
     const char *flow_exclude_list[] = {
         "Agent::StatsCollector",
         "io::ReaderTask",
@@ -124,6 +131,7 @@ void Agent::SetAgentTaskPolicy() {
 
     const char *sandesh_exclude_list[] = {
         "db::DBTable",
+        "Agent::FlowTable",
         "Agent::FlowHandler",
         "Agent::Services",
         "Agent::StatsCollector",
@@ -135,6 +143,7 @@ void Agent::SetAgentTaskPolicy() {
                      sizeof(sandesh_exclude_list) / sizeof(char *));
 
     const char *xmpp_config_exclude_list[] = {
+        "Agent::FlowTable",
         "Agent::FlowHandler",
         "Agent::Services",
         "Agent::StatsCollector",
@@ -167,6 +176,7 @@ void Agent::SetAgentTaskPolicy() {
                      sizeof(walk_cancel_exclude_list) / sizeof(char *));
 
     const char *ksync_exclude_list[] = {
+        "Agent::FlowTable",
         "Agent::FlowHandler",
         "Agent::StatsCollector",
         "db::DBTable",
@@ -437,7 +447,7 @@ Agent::Agent() :
     local_peer_(NULL), local_vm_peer_(NULL), linklocal_peer_(NULL),
     vgw_peer_(NULL), ifmap_parser_(NULL), router_id_configured_(false),
     mirror_src_udp_port_(0), lifetime_manager_(NULL), 
-    ksync_sync_mode_(true), mgmt_ip_(""),
+    ksync_sync_mode_(false), mgmt_ip_(""),
     vxlan_network_identifier_mode_(AUTOMATIC), headless_agent_mode_(false), 
     vhost_interface_(NULL),
     connection_state_(NULL), debug_(false), test_mode_(false),
@@ -609,7 +619,9 @@ bool Agent::isVmwareVcenterMode() const {
 
 void Agent::ConcurrencyCheck() {
     if (test_mode_) {
-       CHECK_CONCURRENCY("db::DBTable", "Agent::KSync", AGENT_INIT_TASKNAME);
+       CHECK_CONCURRENCY("db::DBTable", "Agent::KSync", AGENT_INIT_TASKNAME,
+                         "Flow::Management", "Agent::FlowHandler",
+                         "Agent::FlowTable");
     }
 }
 

diff --git a/src/vnsw/agent/cmn/agent_stats.cc b/src/vnsw/agent/cmn/agent_stats.cc
@@ -14,6 +14,7 @@
 #include <cmn/agent_stats.h>
 #include <cmn/stats_types.h>
 #include <uve/agent_uve_base.h>
+#include <vrouter/flow_stats/flow_stats_collector.h>
 
 AgentStats *AgentStats::singleton_;
 
@@ -64,6 +65,8 @@ void AgentStatsReq::HandleRequest() const {
             (stats->flow_drop_due_to_linklocal_limit());
         flow->set_flow_max_system_flows(agent->flow_table_size());
         flow->set_flow_max_vm_flows(agent->pkt()->flow_table()->max_vm_flows());
+        flow->set_flow_export_msg_drops
+            (agent->flow_stats_collector()->flow_export_msg_drops());
         flow->set_context(context());
         flow->set_more(true);
         flow->Response();

diff --git a/src/vnsw/agent/cmn/agent_stats.h b/src/vnsw/agent/cmn/agent_stats.h
@@ -7,6 +7,8 @@
 #ifndef vnsw_agent_stats_hpp
 #define vnsw_agent_stats_hpp
 
+#include <stdint.h>
+
 class AgentStats {
 public:
     static const uint64_t kInvalidFlowCount = 0xFFFFFFFFFFFFFFFF;

diff --git a/src/vnsw/agent/cmn/stats.sandesh b/src/vnsw/agent/cmn/stats.sandesh
@@ -27,6 +27,7 @@ response sandesh FlowStatsResp {
     5: u64 flow_drop_due_to_linklocal_limit;
     6: u32 flow_max_system_flows;
     7: u32 flow_max_vm_flows;
+    8: u64 flow_export_msg_drops;
 }
 
 struct XmppStatsInfo {

diff --git a/src/vnsw/agent/oper/SConscript b/src/vnsw/agent/oper/SConscript
@@ -67,6 +67,7 @@ vnswoperdb = env.Library('vnswoperdb',
                           'vn.cc',
                           'vrf.cc',
                           'vrf_assign.cc',
+                          'vrouter.cc',
                           'vxlan.cc'
                          ])
 subdirs = ['test']

diff --git a/src/vnsw/agent/oper/global_vrouter.cc b/src/vnsw/agent/oper/global_vrouter.cc
@@ -31,6 +31,7 @@
 const std::string GlobalVrouter::kMetadataService = "metadata";
 const Ip4Address GlobalVrouter::kLoopBackIp = Ip4Address(0x7f000001);
 
+
 ////////////////////////////////////////////////////////////////////////////////
 
 // Link local service
@@ -385,7 +386,7 @@ GlobalVrouter::GlobalVrouter(OperDB *oper)
       fabric_dns_resolver_(new FabricDnsResolver(this,
                            *(oper->agent()->event_manager()->io_service()))),
       agent_route_resync_walker_(new AgentRouteResync(oper->agent())),
-      forwarding_mode_(Agent::L2_L3) {
+      forwarding_mode_(Agent::L2_L3), flow_export_rate_(kDefaultFlowExportRate){
 
     DBTableBase *cfg_db = IFMapTable::FindTable(oper->agent()->db(),
               "global-vrouter-config");
@@ -437,10 +438,17 @@ void GlobalVrouter::GlobalVrouterConfig(DBTablePartBase *partition,
             resync_route = true;
             resync_vm_interface = true;
         }
+        if (cfg->IsPropertySet
+                (autogen::GlobalVrouterConfig::FLOW_EXPORT_RATE)) {
+            flow_export_rate_ = cfg->flow_export_rate();
+        } else {
+            flow_export_rate_ = kDefaultFlowExportRate;
+        }
     } else {
         DeleteLinkLocalServiceConfig();
         TunnelType::DeletePriorityList();
-        resync_route= true;
+        resync_route = true;
+        flow_export_rate_ = kDefaultFlowExportRate;
     }
 
     if (cfg_vxlan_network_identifier_mode !=                             

diff --git a/src/vnsw/agent/oper/global_vrouter.h b/src/vnsw/agent/oper/global_vrouter.h
@@ -12,6 +12,8 @@ class VnEntry;
 class VrfEntry;
 class IFMapNode;
 class AgentRouteResync;
+class AgentRouteEncap;
+class AgentUtXmlFlowThreshold;
 
 namespace autogen {
     struct LinklocalServiceEntryType;
@@ -32,6 +34,7 @@ class GlobalVrouter {
 public:
     static const std::string kMetadataService;
     static const Ip4Address kLoopBackIp;
+    static const uint32_t kDefaultFlowExportRate = 1000;
 
     struct LinkLocalServiceKey {
         Ip4Address linklocal_service_ip;
@@ -69,6 +72,7 @@ class GlobalVrouter {
     const LinkLocalServicesMap &linklocal_services_map() const {
         return linklocal_services_map_;
     }
+    uint32_t flow_export_rate() const { return flow_export_rate_; }
 
     void GlobalVrouterConfig(DBTablePartBase *partition, DBEntryBase *dbe);
     bool FindLinkLocalService(const std::string &service_name,
@@ -89,6 +93,7 @@ class GlobalVrouter {
     uint64_t PendingFabricDnsRequests() const;
     void ResyncRoutes();
 
+    friend class AgentUtXmlFlowThreshold;
 private:
     class FabricDnsResolver;
     class LinkLocalRouteManager;
@@ -110,6 +115,7 @@ class GlobalVrouter {
     boost::scoped_ptr<FabricDnsResolver> fabric_dns_resolver_;
     boost::scoped_ptr<AgentRouteResync> agent_route_resync_walker_;
     Agent::ForwardingMode forwarding_mode_;
+    uint32_t flow_export_rate_;
 };
 
 #endif // vnsw_agent_global_router_h_
diff --git a/src/vnsw/agent/oper/operdb_init.cc b/src/vnsw/agent/oper/operdb_init.cc
@@ -37,6 +37,7 @@
 #include <oper/config_manager.h>
 #include <oper/agent_profile.h>
 #include <oper/agent_sandesh.h>
+#include <oper/vrouter.h>
 #include <nexthop_server/nexthop_manager.h>
 
 using boost::assign::map_list_of;
@@ -183,6 +184,7 @@ void OperDB::CreateDBTables(DB *db) {
                                              "db.physical_device_vn.0");
     agent_->set_physical_device_vn_table(dev_vn_table);
     profile_.reset(new AgentProfile(agent_, true));
+    vrouter_ = std::auto_ptr<VRouter> (new VRouter(this));
 }
 
 void OperDB::Init() {
@@ -273,6 +275,7 @@ void OperDB::Shutdown() {
 #endif
     route_preference_module_->Shutdown();
     domain_config_->Terminate();
+    vrouter_.reset();
 }
 
 void OperDB::DeleteRoutes() {

diff --git a/src/vnsw/agent/oper/operdb_init.h b/src/vnsw/agent/oper/operdb_init.h
@@ -20,6 +20,7 @@ class InstanceManager;
 class NexthopManager;
 class AgentSandeshManager;
 class AgentProfile;
+class VRouter;
 
 class OperDB {
 public:
@@ -54,6 +55,7 @@ class OperDB {
     AgentSandeshManager *agent_sandesh_manager() {
         return agent_sandesh_manager_.get();
     }
+    VRouter *vrouter() const { return vrouter_.get(); }
 
 private:
     OperDB();
@@ -68,6 +70,7 @@ class OperDB {
     std::auto_ptr<NexthopManager> nexthop_manager_;
     std::auto_ptr<AgentSandeshManager> agent_sandesh_manager_;
     std::auto_ptr<AgentProfile> profile_;
+    std::auto_ptr<VRouter> vrouter_;
     DISALLOW_COPY_AND_ASSIGN(OperDB);
 };
 #endif
diff --git a/src/vnsw/agent/oper/test/global_vrouter.xml b/src/vnsw/agent/oper/test/global_vrouter.xml
@@ -0,0 +1,27 @@
+<?xml version="1.0"?>
+<test_suite name="vrf">
+<test name="vrouter-basic-1">
+    <!-- Virtual-router with flow-export-rate -->
+    <global-vrouter-config uuid="1" name="default-global-system-config:default-global-vrouter-config" vxlan-mode="automatic"
+     flow-export-rate="100"
+    />
+    <validate name="vrouter-basic-validate-1">
+        <global-vrouter-config uuid="1" name="default-global-system-config:default-global-vrouter-config" flow-export-rate="100" />
+    </validate>
+
+    <!-- Change flow-export-rate to 200 -->
+    <global-vrouter-config uuid="1" name="default-global-system-config:default-global-vrouter-config" vxlan-mode="automatic"
+     flow-export-rate="200"
+    />
+    <validate name="change-flow-export-rate-1">
+        <global-vrouter-config uuid="1" name="default-global-system-config:default-global-vrouter-config" flow-export-rate="200" />
+    </validate>
+
+    <!-- Remove flow-export-rate attribute. It should be set to default. -->
+    <global-vrouter-config uuid="1" name="default-global-system-config:default-global-vrouter-config" vxlan-mode="automatic" />
+    <validate name="change-flow-export-rate-2">
+        <global-vrouter-config uuid="1" name="default-global-system-config:default-global-vrouter-config" flow-export-rate="1000" />
+    </validate>
+</test>
+
+</test_suite>
diff --git a/src/vnsw/agent/oper/test/test_inet_interface.cc b/src/vnsw/agent/oper/test/test_inet_interface.cc
@@ -286,15 +286,30 @@ static void InetTestCleanup(Agent *agent, const Ip4Address &addr,
                           const Ip4Address &gw, uint8_t plen) {
     InetUnicastAgentRouteTable *table = agent->fabric_inet4_unicast_table();
 
+    table->DeleteReq(agent->local_peer(), agent->fabric_vrf_name(),
+                     Ip4Address::from_string("0.0.0.0"), 0, NULL);
+    client->WaitForIdle();
+    WAIT_FOR(1000, 1000,
+             (RouteGet(agent->fabric_vrf_name(),
+                       Ip4Address::from_string("0.0.0.0"), 0) == NULL));
+
     table->DeleteReq(agent->local_peer(), agent->fabric_vrf_name(), addr, 32,
                      NULL);
+    client->WaitForIdle();
+    WAIT_FOR(1000, 1000,
+             (RouteGet(agent->fabric_vrf_name(), addr, 32) == NULL));
+
     table->DeleteReq(agent->local_peer(), agent->fabric_vrf_name(),
                      gw, 32, NULL);
+    client->WaitForIdle();
+    WAIT_FOR(1000, 1000,
+             (RouteGet(agent->fabric_vrf_name(), gw, 32) == NULL));
+
     table->DeleteReq(agent->local_peer(), agent->fabric_vrf_name(),
                      addr, plen, NULL);
+    client->WaitForIdle();
     WAIT_FOR(1000, 1000,
              (RouteGet(agent->fabric_vrf_name(), addr, plen) == NULL));
-    client->WaitForIdle();
 }
 
 static void RestoreInetConfig(Agent *agent) {
@@ -311,24 +326,33 @@ static void DelInetConfig(Agent *agent) {
     InetUnicastAgentRouteTable *table = agent->fabric_inet4_unicast_table();
     table->DeleteReq(agent->local_peer(), agent->fabric_vrf_name(),
                      Ip4Address(0), 0, NULL);
+    client->WaitForIdle();
 }
 
-static bool RouteValidate(Agent *agent, const Ip4Address &ip, uint8_t plen,
-                          NextHop::Type nh_type) {
+static const InetUnicastRouteEntry *RouteValidate(Agent *agent,
+                                                  const Ip4Address &ip,
+                                                  uint8_t plen) {
     const InetUnicastRouteEntry *rt = NULL;
-    const NextHop *nh = NULL;
-
     WAIT_FOR(1000, 1000,
              ((rt = RouteGet(agent->fabric_vrf_name(), ip, plen)) != NULL));
-    if (rt == NULL)
+    return rt;
+}
+
+static bool RouteValidate(Agent *agent, const Ip4Address &ip, uint8_t plen,
+                          NextHop::Type nh_type) {
+    const InetUnicastRouteEntry *rt;
+    if ((rt = RouteValidate(agent, ip, plen)) == NULL)
         return false;
 
-    nh = rt->GetActiveNextHop();
+    const NextHop *nh = rt->GetActiveNextHop();
     return (nh->GetType() == nh_type);
 }
 
 TEST_F(InetInterfaceTest, physical_eth_encap_1) {
     DelInetConfig(agent_);
+    WAIT_FOR(1000, 1000,
+             (RouteGet(agent_->fabric_vrf_name(),
+                       Ip4Address::from_string("0.0.0.0"), 0) == NULL));
 
     Ip4Address ip = Ip4Address::from_string("10.10.10.10");
     Ip4Address gw = Ip4Address::from_string("10.10.10.1");
@@ -350,6 +374,7 @@ TEST_F(InetInterfaceTest, physical_eth_encap_1) {
 
     EXPECT_TRUE(RouteValidate(agent_, ip, 32, NextHop::RECEIVE));
     EXPECT_TRUE(RouteValidate(agent_, net, plen, NextHop::RESOLVE));
+    EXPECT_TRUE((RouteValidate(agent_, gw, 32) != NULL));
 
     // Cleanup config by the test
     InetTestCleanup(agent_, ip, gw, plen);

diff --git a/src/vnsw/agent/oper/test/test_oper_xml.cc b/src/vnsw/agent/oper/test/test_oper_xml.cc
@@ -70,6 +70,19 @@ TEST_F(TestVrf, vxlan_1) {
     }
 }
 
+TEST_F(TestVrf, vrouter_1) {
+    AgentUtXmlTest test("controller/src/vnsw/agent/oper/test/global_vrouter.xml");
+    AgentUtXmlOperInit(&test);
+    if (test.Load() == true) {
+        test.ReadXml();
+
+        string str;
+        test.ToString(&str);
+        cout << str << endl;
+        test.Run();
+    }
+}
+
 int main(int argc, char *argv[]) {
     GETUSERARGS();
     client = TestInit(init_file, ksync_init);