GRE over IPv4 traffic from VM to Contrail is not destributed #99

Implement per-CPU fragment queues. For now, the implementation is partially copied from Linux implementation. It will be refactored in followup PRs. Change-Id: Ib448d0515b99c9b64035222a23e0c048348526bd Partial-Jira-Bug: WIND-165

Atomic builtin functions should use 'volatile' for all pointers. Change-Id: I5949dfde57c6adfa720cb79fc6c8507884bb78ff Closes-Jira-Bug: WIND-177

Change-Id: Iabc976d1de4ce6e56957e2aec8aca36b653dbf17 Partial-Jira-Bug: WIND-165

Change-Id: I2801c30cced7913a9ce6268a4a7e108f9200ec4f Partial-Jira-Bug: WIND-165

closes-jira-bug: CEM-2807 Limitation in rte_port_ethdev_writer_tx_bulk in the pkts_mask field that we can transfer only 64 segments. The segmentation code can send a max of 128 segments, which was causing the mbuf leak. Fixed the code, by calling the tx_bulk function in a loop Change-Id: I81212f1ae32e47dd80c0938631e466db98242a3e

In scenarios where external implementations offload the full datapath to hardware, flow and vrf stats do not propogate to vrouter on the host. Add callbacks which enable such implementations to update these stats. Change-Id: I92ed304003284a0b69a0c27e37b579fe35d453b3 Partial-Bug: #1767107 Signed-off-by: Frik Botha <frederick.botha@netronome.com>

With Smet routes, we only support sender outside contrail. However, we should continue using broadcast routes for sender inside contrail. Closes Jira Bug: JCB-219096 Change-Id: I2f46809beb1b04b48ed43b29ad982d08312b12a8

This is a temporary commit to get more debug info for bug:JCB-160099 Suspecting hugepage request is not processed by vrouter code. Added debug prints in hugepage request to get more data about the crash. Partial Jira Bug: JCB-160099 Change-Id: I190265a9d0c687fe8a398b106d6a8d84266e89d3

vRouter-DPDK is unable to probe Netronome NICs, since the build system does not link to the NFP PMD. Change-Id: If5d9750a18348ae578cc413063a14cd75dd0ed3e Partial-Bug: #1774414 Signed-off-by: Frik Botha <frederick.botha@netronome.com>

Prepare to move vr_linux_fragment_assembler, VrFragmentAssembler and dpdk_fragment_assemble_queue to dp-core: 1. Introduce os-dependent vr_fragment_sync_assemble callback. 2. Rename vr_fragment_assembler to vr_fragment_assemble. 3. Fix formatting issues. Change-Id: Ia1cecaf06474df424d12fae8cca84fa9c8d8cc43 Partial-Jira-Bug: WIND-165

Move duplicated code from fragment assembler on Windows, Linux and DPDK to dp-core. Change-Id: I99c4a3b2b4df13277c21bbbec5a2060b7ca3f93b Partial-Jira-Bug: WIND-165

….1 to 5.0"

closes-jira-bug: CEM-2996 Added a null check for variable. Change-Id: I667f413a1d4604f7f2de8c7c04157c9b7317b2a0

If igmp is disabled, we should always use bcast mac for lookup. Partial Jira Bug: JCB-219141 Change-Id: Iad007832801a7c79b3d0897e877209815c53ebeb

…t_p'"

closes-jira-bug: CEM-2807 tx_bulk can support upto burst of 64 packets. Change the send size to 64 instead of 32. Change-Id: Ifbf45e9fc23dd4d8e0f69fefc745730d3fbf2726 (cherry picked from commit 26f0926)

During refactoring of KSync module on Windows, `windows/ksync` directory was created. Currently, it is only included in the debug build. This change includes it to the release build. This change also removes redundant compilation options for vRouter in debug and release modes. Closes Jira Bug: WIND-198 Change-Id: I3582f7e4da5c44f5e3fa501bce0dfe3c1cd5e0fb

Function ReallocateHeaders performs NBL copy for all packets - no matter if headers are in continuous memory segment or not. It means that if NIC driver is about to reuse original NBL in the future, it may safely assume that it is not modified. This change only affects Windows specific code. Change-Id: I5528c230db094857c7f489c7cd8a070a770fc626 Closes-JIRA-Bug: WIND-178

partial-jira-bug: CEM-3470 Disable hw checksum calculation to unblock QA tests Change-Id: I8ead412473c78d8eec55c92919bef81876c5b7f0

Tested by sending invalid ARP pkts using scapy and checked that Invalid ARP counter is getting incremented. Change-Id: I2eaff1182651e4fad2da23e9d84a360c0faf1e08 closes-jira-bug: JCB-163451

Corrected the checks and updated comments for the changes. Change-Id: I5413c59925372a6f677cb550562dcd97ae345290 partial-jira-bug: CEM-3439

closes-jira-bug: CEM-3470 The hw checksum seems to work with the old api mode. Reverting the workaround done to use sw checksum and use old api mode for now. Change-Id: I958f868c7bc3149c8e2b322ec4bd95a6dd279a6a

Signing with Visual Studio is flaky, so we'll sign the driver from SConscript. Closes Jira Bug: WIND-187 Change-Id: I2337119d06bb66c44baecdaede1d821375e53cfc

Add setting checksum to zero when LSO is enabled for packet. The checksum should be calculated by NIC in this case. According to MSDN: "Turning off Address Checksum Offloads when Large Send Offload (LSO) is enabled does not prevent the miniport driver from computing and inserting checksums in the packets generated by the LSO feature." Change-Id: Ibba695c69fa8e392e69fe472908ce66e69f90314 Closes-Jira-Bug: WIND-226

The default signing is good enough. Closes Jira Bug: WIND-236 Change-Id: Ia5484c92247caf31686e7287b92d0ba193b703e4

Due to this warning, the compilation of dkms used to fail Change-Id: Idf6b609e02762308601a62d17ac3ed506b44c509 Closes-Jira-Bug: CEM-4133

Reduce computational complexitiy of dumping packet bytes to string by removing usage of RtlStringCbCatA for every byte of the packet and by using simplified implementation of byte-to-hex conversion function. This will increase the performance of the whole packet-to-json converter. Change-Id: I24dec3b7bb7c7ab593c14afa1dc58a6668d76d20 Closes-Jira-Bug: WIND-238

Fix the MTU calculation. In platform dependent code, the default MTU is taken from the linux netdevice which is 1500 bytes. But for VIF, the MTU is 1514 bytes (max size including ether hdr). As a result, the MSS calculation would cause the size to go beyond 1514byes and the NIC was dropping large size ethernet packets. Fixed by adjusting the MTU to take care of ethernet header size. Change-Id: I377446dd6430a62b35c0d80c2bd9554b72bf3ba4 Closes-Jira-Bug: CEM-4137

Added missing include. Closes Jira Bug: WIND-244 Change-Id: I2e7a89724ff0dc1c93b89b7436399ef421e5702a

…bler"

As per linux kernel git log, the get_user_pages() was changed in 4.4.168 which also maps to Ubuntu 4.4.0-143. Fixed by checking kernel version 4.4.168. Verified by building vrouter.ko on CI VM with ubuntu 4.4.0-143 and regular centos setup. Change-Id: Ie759e567a6f5f82d1c4d1dd6ea0f21c23f7287ff closes-jira-bug: CEM-4223

….04)"

…ng runtime. command: dropstats --clear Closes-Jira-Bug: CEM-3323 Change-Id: I91283226bcfb88d13c2f6bd3fadc13e1ab75fee6

1. Handle 64-bit blocks at a time, then add remaining 32-bit, 16-bit and 8-bit blocks. 2. Add function for folding 64-bit checksums to 32 bits. Change-Id: I3526188b34708a74026cda4827280b3c961b5fd3 Closes-Jira-Bug: WIND-250

…es during runtime. command: dropstats --clear Closes-Jira-Bug: CEM-3323"

partial-jira-bug: CEM-4579 Need to set the ignore_offload_bitfield field in the ethdev conf As well pass the same to the slave device from bond config Change-Id: I2e176246c2bbfcc4a54719303f7d269b74d12577

closes-jira-bug: CEM-4659 Seems like the 18.05 dpdk has issue when sending 64 packets in burst. Reverting back to use VR_DPDK_TX_BURST_SZ Change-Id: I4d08be695a5a7968f6d07be2688bf1243b399f82

When the driver is unloaded, all NBLs should be freed before calling NdisFDeregisterFilterDriver. Part of the code responsible for freeing the pending NBLs was copied from FilterDetach to Driver unload function. Change-Id: I68feedc461669b67b6b0c06ad9035353815fb294 Closes-Jira-Bug: WIND-251

Closes-Jira-Bug: CEM-4715 Change-Id: Ic1851f3d992f1373dc9984d8c4fbe41bbdb8516a

when flows are triggered by remote ECMP source, source validation (RPF check) is done by iterating through component NH list of source composite NH. If component NH (Tunnel NH ) sip matches with packet's outer source IP, then RPF is succeded. In Multi level ECMP case, component NH is again composite NH, it should recursively call validate function and propagate the source validation status to the outermost caller. This logic assumes ECMP symmetry for outer and inner ecmp. Change-Id: Ie48ee4967287a9e230723478d4c234991c4a5254 Closes-Jira-Bug: JCB-219439

Remove invalid implementation of jhash function on Windows and FreeBSD and use vr_hash instead. Usage of proper hash value results in generating different source UDP ports for packets which will spread the packets between multiple CPUs on the receiving side. This change will not affect Linux, where jhash and vr_hash are the same implementations. Change-Id: Iab1583aa379dadb994fb9969ef03fce6dae074f4 Closes-Jira-Bug: WIND-257

…indows"

Change-Id: I821e2974c4bf649770649baef835fed2c58b34de Partial-Bug: #1800894

In case of VXLAN routing, for vxlan id's whose top 4 bits are non-zero, there was a truncation happening in mtrie. The reason was that the label was assumed to be MPLS label which is 20bits. But in case of VXLAN routing, mtrie lookup happends for VXLAN labels which are 24 bits. Fixing the mask to 24 bits. Testing The following commands are executed to program a VXLAN id (11158251) with the top 4 bits are non-zero nh --create 350 --vrf 2 --type 3 --oif 0 --smac 00:25:90:a5:39:af --dmac 00:25:90:a5:3a:fd --sip 8.0.0.2 --dip 8.0.0.3 --vxlan --sport 10000 --dport 10000 --l3_vxlan 00:25:90:a5:3a:fd rt -c -n 350 -p 10.1.1.6 -l 32 -t 11158251 -f 0 -v 2 -x 0x1 Before the fix, the label in incorrectly programmed - (vrouter-agent-dpdk)[root@a7s4-kiran /]$ rt --dump 2 --family inet |grep 10.1.1.6 10.1.1.6/32 32 L 672491 350 - After the fix, the label is properly programmed - (vrouter-agent-dpdk)[root@a7s4-kiran /]$ rt --dump 2 | grep "10\.1\.1\.6\>" 10.1.1.6/32 32 L 11158251 350 - Closes-Jira-Bug: CEM-4968 Change-Id: I033b07911c8d371b8447637c58d3cfc0eafaa1f0

…trail Network Policy Mirror Action not working-DPA2a Issue fixed by creating a new memory buffer and chained to existing memory buffer. Backported DPDK checksum from latest release to support mbuf chaining. Closes-jira-bug: JCB-219327 Merged from commit: https://review.opencontrail.org/#/c/50219/ Change-Id: I0913d32c2315342eb0b5997f722b358d3482df0f

…ng, Contrail Network Policy Mirror Action not working-DPA2a Issue fixed by creating a new memory buffer and chained to existing memory buffer. Backported DPDK checksum from latest release to support mbuf chaining. Closes-jira-bug: JCB-219327 Merged from commit: https://review.opencontrail.org/#/c/50219/"

Checksum of outer IP packet may be offloaded, so we don't need to calculate it in software. Change-Id: I3c62f8aa4013d196d8adaac7e6adc8e8edc73c32 Closes-Jira-Bug: WIND-273

Branch mispredicution could cost performance loss. Add vr_likely and vr_unlikely support to assist compiler. Change-Id: I5a0a2210ea0d1f092f2543b52d0a145f38a33335 Partial-Bug: #1769570 Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>

Change-Id: Ifbc5845a0e1e2ca1da888cfa5113273098ef70a4 Partial-Jira-Bug: WIND-277

Partial-Jira-Bug: WIND-278 Change-Id: I237ec594e8bd2987d5325333871c15c84490522e

Fix for displaying "dropstats --log <core number>" closes-jira-bug: CEM-5475 Change-Id: I0cc9e7ae06769c6692d6312ef59fbc8952b656b9

Checksum of inner TCP packet may be offloaded, so we don't need to calculate it in software. Change-Id: I7c10e3f7ff592c35331a7556cfb22601e0ee947c Closes-Jira-Bug: WIND-277

… code Change-Id: Ibf449f9aa56a7c7a2cc21129e1b346586ed5f7fb Closes-Jira-Bug: WIND-287

Change-Id: Id1285ce0e5f39d0810196864fdf9d9a890a2abf6 closes-jira-bug: CEM-5420

…ntation code"

This reverts commit 28905f3. Change-Id: If2a4257649915deaee8ec0402475b31e30050c78 Partial-Jira-Bug: WIND-291

In Juju charms with ubuntu 18.04 container and ubuntu 18.04 host, the mount point for 1G hugepages is defined in a different format in /proc/mounts. Other distributions define it as "pagesize=1G", but in this case, it is defined as "pagesize=1024M" (vrouter-agent-dpdk)[root@dvplabor10n25 /]$ cat /proc/mounts |grep -i huge cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=1024M 0 0 hugetlbfs-kvm /run/hugepages/kvm hugetlbfs rw,relatime,gid=116,mode=775,pagesize=1024M 0 0 Change-Id: I6a5729f6027d09ec7ad62a30a403ae9842ff6233 Closes-Jira-Bug: CEM-4655 (cherry picked from commit 1cf32cb)

… Windows""

closes-jira-bug: CEM-5251 Disabling promiscuous mode should be done based on the result of adding the unicast address. Drivers such as i40e doesnt support the apis to set multicast address. But the bond driver is made to enable all multicast for all the slave drives. So this shouldnt be an issue. Change-Id: Ie754fad59a215462b62da6e2ab309de13422d1fd

…e_data please also see vrouter\ksync in agent. In ksync_flow_memory we set table_path_ to some default value. Then the value is set in ksyncsandeshcontext via set_table_path fn. Please see KSyncSandeshContext::FlowTableInfoHandler(vr_flow_table_data *r) This change is needed so we get the correct value here. The dpdk/vr_dpdk_table_mem.c had shm_file filled with the value but it was a local variable so gdb showed that the value was garbage. [Based on input/debugging with Jeya about this issue and Jeya's help with DPDK] Change-Id: Ibaad63a737aa7bd3c465815e7c7bcd37c54f7e66 partial-jira-bug: CEM-5300 partial-jira-bug: CEM-6242

…ow_table_data"

Change-Id: I829da74bd7926d0ead3b1fd0b4932ae0f6306435 Closes-JIRA-Bug: WIND-256

Updated Defensive check to overcome this issue, If actual interface provided for "vifdump stop <Id>" an erro message would be displayed. closes-jira-bug: CEM-5980 Change-Id: I2e5ff28e7da15527c9ac4628ebff5aa1dee5f233

Root cause: =========== This is due a race condition between setting of the Evict Candidate flag and the Eviction of the flow. We have 2 threads here say Thread1 and Thread2. Thread1 is in the flow defer_callback function, while Thread2 is in the flow mark evict function. Now consider the following sequence (Time T0 to T4) which leads to the non eviction of the flow 100. T0 and T3 are executed in Thread1, while T1, T2 and T4 are executed as part of Thread 2. Lets assume the flow index for the flows as 100 and 200. Thread 1 Thread 2 ------------- ------------- Defer_cb_Func() Mark_Evict_Func() Time T0 - CheckEvictFlow(100) – No op, as Evict candidate flag is not set for flow 100 Time T1 – Set Evict Candidate for flow(100) Time T2 – Set Evict Candidate for flow(200) Time T3 – CheckEvictFlow(200) – Now flow 200 would get evicted since Evict Candidate flag is set. Time T4 – Schedule Defer_cb for flow(200) Now, since flow 200 already got evicted at Time T3, the callback would never be scheduled at time T4. And hence, the flow 100 would never get evicted. Was able to repro the problem by simulating the above order using instrumented code and scapy script. The problem is with Defer_cb_func() which should do the eviction only when the Evict Candidate flag is set for both flows. Fix: ==== The fix is to make sure the eviction is done only after both the flows evict candidate flags are set. Testing: ======== - Done UT for the code changes - QA has qualified the fix Change-Id: I07730ab190646260d08de6ca4fdf9bc1caf16d6e closes-jira-bug: CEM-4275 (cherry picked from commit e04c2b0)

changes to allow compilation of vrouter with pre-built dpdk libraries from dpdk releases (or upstream code) Closes-Jira-Bug: TFB-1446 Change-Id: I5ee44725b71c1472c0871c07df622b992cb318ce

Closes-Jira-Bug: CEM-6199 Change-Id: Iae8ee924fb76b0e72ce58c8ffe36a6bd0d391a3a

…icted"

… in DPDK Closes-Jira-Bug: CEM-6199"

Change-Id: I4478088464df7adc1e552c41e329ed8559b8f574 closes-jira-bug: CEM-6504

These changes were planned to be committed as part of CEM-6195, but it seems it is required for the ubuntu aws builds. So raising a separate review for this to be committed first since they are standalone and can be committed independently. Change-Id: Ifb47719eeec372ca2866afb9981c901bc34cc2a5 closes-jira-bug: CEM-7103

These are eBPF based tools that allow packet tracing and capturing stack traces in kernel vrouter module. Needs bcc-tools package to work. Change-Id: I68748bbd3cf1602adf2e1d8e45c9267165c4aa47 Closes-Jira-Bug: CEM-4318

This changeset contains changes to vtest as well the new python ut infra. A new scons target "scons vrouter-py-ut" will trigger the execution of testcases. See README.md under vtest_py directory for more details on this infra. closes-jira-bug: CEM-6195 Change-Id: I4171746062dc8e0c4f63943277dd79b676362f76

The port number needs to be converted to proper endian before checking. Because of this, there was a code loop and caused stack overflow. As a defensive measure, added a ttl to drop such packets. Closes-Bug: CEM-7113 Change-Id: I7b80c25978b896f8d1b1bfb1fd34443fa1c3031f

- Fix custom vif_name not working in vtest - Add utility functions for MAC, IP etc. - Add support for running vrouter and vtest separately Partial-Jira-Bug: CEM-7351 Change-Id: Ia0f9681e986a39ed19b49804d0141bbfa4293ea7

- Fix issue with return sandesh type for vr_flow_req - Add utility function for ipv6 address - Add more constants Partial-Jira-Bug: CEM-7351 Change-Id: I611fbd3c15c24a9da99661bdb6c21169b846b2b8

Closes-Jira-Bug: CEM-6947 Change-Id: Ie19c2462f4d38d27f727ea6f519dd177b6d20fc5

- Add mock devices for agent and vhost0 - Add UT for CEM-7713 to test port 60185 which was causing vrouter crash - Add UT for packet from VMI to fabric with MPLSoUDP encap Closes-Jira-Bug: CEM-7719 Change-Id: Idc8a1324454a39c39b3f253046fd3df098a4b38c

- Defined the basic classes to create a higher level of abstraction. - basic test case file - CI related changes to fail the job if test execution fails closes-jira-bug: CEM-7351 Change-Id: I3f58bbd991e7cd1e1770ef9a0e491dae2113fc6f

Fix the issue caused due to 626ef37 closes-jira-bug: CEM-8410 Change-Id: I87b9defde7a8a462d3ca99c9cbe69e018fe22034

QEMU was not getting response for set log base message. Another commit will address the actual implementation of VHOST_F_LOG_ALL closes-jira-bug:CEM-5848 Change-Id: I520883464e396b75ebb59f1b6e6cb19884b2d830

Closes-jira-bug:CEM-8606 In this case, route prefix configired as 2.2.2.2, vtest for "vr_route_req" ignores only ':' not '.' Change-Id: Icef0a02408f6649b9240a9c05017ec73e3943743

… nodes"

partial-jira-bug: CEM-7843 Change-Id: I7b7dc2a760311a0783db68d73757d1672eaacdb7

When vr_send_broadcast() is called during route add or delete, there will be a 2048 slab memory leak. This is because response->vr_message_buf is getting set to NULL and so vr_message_free() will not free response->vr_message_buf. response->vr_message_buf is only applicable to unicast netlink case and not for multicast netlink. Fix is to move it to unicast case only. Closes-Jira-bug: CEM-5343 Change-Id: Ia4d035e88765c55f65fedce1963000c0bced55c1

…g:CEM-8606 In this case, route prefix configired as 2.2.2.2, vtest for "vr_route_req" ignores only ':' not '.'"

Closes-jira-bug:CEM-5596 - Callback notifiacation for Master/Slave bond interface members has been implemented. - Notifying vrouter-agent about the state change - Display Bond interface member status in "vif --list" output. Updated patchset: - Included vlan info, Fwdcore to display as part of physical interface - Display NH id for all interfaces. - Sending vif interface id as part of netlink message to agent Blueprint: https://review.opencontrail.org/c/Juniper/contrail-specs/+/52710 Updated patchset: - Review comments updated. Change-Id: I5985c6c4b00d17a4320a860b66640d0990409d9c

This reverts commit 580007c. Reason for revert: Check to see if this review is causing kernel panic Closes-Bug: CE-1686 Change-Id: I98cbe19e6de5afbb154ddadb1f8c14a47126b8d3

Closes-jira-bug:CEM-8367 - Callback notifiacation for Master/Slave bond interface members has been implemented. - Notifying vrouter-agent about the state change - Display Bond interface member status in "vif --list" output. Updated patchset: - Included vlan info, Fwdcore to display as part of physical interface - Display NH id for all interfaces. - Sending vif interface id as part of netlink message to agent Blueprint: https://review.opencontrail.org/c/Juniper/contrail-specs/+/52710 Updated patchset: - Review comments updated. Change-Id: I8f0a7918faaa7bb083b0a13e5f8b06d6d22a262f

Closes-jira-bug: CEM-9120 Change-Id: I8854896451b9dbba2599df7b2cbaf16266597990

For Bond Monitoring feature, Master interface update is not sent on few platforms, so forcing here to send master interface status before sending slave interface. Closes-jira-bug: CEM-9082 Change-Id: I29a6aea27a63a95f8a58f3859aca847961257816

Change-Id: I824e2ec98f59d47bcda571432122a5fc23f85eb0 closes-jira-bug: CEM-7188

In bond monitoring, link state interrupt config was enabled for all devices. This does not hold good for af_packet devices which is created during SNAT. Fix is to disable lsc for these devices. Closes-Jira-Bug: CEM-9669 Change-Id: Ifc33e797f8f24e76bf9bf09d3abcc70322bb93d5

Spec: https://github.com/Juniper/contrail-specs/blob/master/hbf.md Summary of changes: 1) Support a new sandesh object (vrftable) to store info about HBS-left and HBS-right interfaces 2) Add new flags in VIF to indicate HBS-left and HBS-right interfaces 3) Add new flags in flow to indicate HBF-left and HBS-right flows 4) Changes in flow processing to steer the packets to HBS interfaces for HBS flows before flow action 5) Changes in flow processing to resume flow actions for packets exiting from HBS interfaces 6) Add new CLI - vrftable for displaying VRF info 7) Add UTs for important HBS cases - inter/intra compute, inter/intra VN, Default/L3 mode etc. Closes-Jira-Bug: CEM-7030 Change-Id: Ifd6e394f5a85bbd665a1e0c79acf6b14c9c353df Depends-On: I2fef0dc882219d55a9965ec9bec00df922287de3 Depends-On: If681857dc16ed25c388b57157e17a0858d63596c Depends-On: I2a8db9035ada4e6afc97c141b3ca7b8f0d9c8da7

There was a race condition between vr_get_ecmp_first_member_dev() and vrouter_put_nexthop() where the latter used to set nh->nh_component_nh[i].cnh as NULL and the former tries to access it. This would cause a crash. Fix is to add a NULL check Closes-Jira-Bug: CEM-10224 Change-Id: I5ae17a94b2f57814bc5c38582631abefdd8906b1

A spurious vr_uvh_cl_timer_setup() call was happening which was leaking fds Closes-Jira-Bug: CEM-9116 Change-Id: I1a02ba31b747469f8ebab79c603f80b8102238a4

… cores machine - Change VR_CPU_MASK to 0xFFFF to allow > 255 cores. closes-jira-bug: CEM-10622 Change-Id: I52b1b51d1615f8fa78e3883573594b4f084f2aac

Partial-Jira-Bug: CEM-7843 Change-Id: Ic73501a99b2a640701306522cbef6ec10786478b

…logical cores machine"

Crash resulting due to a race condition. In file DPDK/vr_dpdk_lcore.c: thread 1 is trying to access rx_queue->q_queue_h in dpdk_lcore_rxqs_vroute() thread 14 is at vr_dpdk_lcore_if_schedule() in which (*rx_queue_init_op)() is not done due to which rx_queue->q_queue_h is not initialised yet. Fixed by adding a null check in dpdk_lcore_rxqs_vroute() Closes-jira-bug: CEM-5837 Change-Id: I7dafc89e3eb97df1e8b4d24142c7b3ed5d054710

Create dependency for vrouter-py-ut to execute "scons vrouter" first. Skip the failing UTs. They will be fixed as part of a separate ticket. Closes-jira-bug: CEM-9436 Depends-On: Id79eb261efec2b9ace54989d336dd8cbbf1ee6b9 Change-Id: If558055538c0a16e79f8a16a8b83bdc51a3e33e1

Partial-Jira-Bug: CEM-7843 Most windows specific code was already removed from this repo. Only this minor part was remaining. Change-Id: I2062952b733bf20e9c8f24591b893a0690990320

…-py-ut to execute "scons vrouter" first. Skip the failing UTs. They will be fixed as part of a separate ticket."

Vrouter UT: test_fd_leak.py testcase fixed by give some time interval before checking the fd count. closes-jira-bug: CEM-11243 Change-Id: I3ae463c1c638afcc58997e60b67a9fe7db0a41bc

…ug: CEM-7843"

The fix is exposing some more issues relating to vrouter agent. Reverting it for now. Closes-Jira-Bug: CEM-11181 Change-Id: I6f08bab45753ca12ca31aaaff9317c4c88eb2984 Depends-on: I4d21a6c7e3fc053252bc7cfdbc2b9574e69b2b86

previous review submitted with Change-Id: I2062952b733bf20e9c8f24591b893a0690990320 Change-Id: I848a057ba12e23d2a103b2be996654a318939268

Updated SConscript to trigger a shell script which runs steps required to produce vrouter code coverage report excluding XML based testcases. Implementation: scons vrouter-py-ut:coverage Closes-Jira-Bug: CEM-9980 Change-Id: I32bb1cf3e95e3f29d7fe98ac50a336f7f481e1f6

This updates README.md for building vrouter.ko module and contrail-vrouter-dpdk binary. Closes-Jira-Bug: CEM-9798 Change-Id: I54ec5d329923a6586286256b62603d2d271e08d5

Issue: Routes were missing for couple of prefixes in the vrouter routing table. Specifically 10.60.7.0/25 and 10.60.7.128/25 prefixes. Root cause: The routing table in vrouter is implemented as a mtrie data structure. The root cause of the issue was incorrect mtrie bucket deletion logic in mtrie code. The issue was reproduced with the following sequence of route adds and delete. addr = 10.60.7.0 plen = 25 vrf = 4 operation = ADD/CHANGE nh_idx = 38 label = 410760 addr = 10.60.0.0 plen = 23 vrf = 4 operation = ADD/CHANGE nh_idx = 38 label = 410760 addr = 10.60.7.128 plen = 25 vrf = 4 operation = ADD/CHANGE nh_idx = 38 label = 410760 addr = 10.60.0.0 plen = 20 vrf = 4 operation = ADD/CHANGE nh_idx = 63 label = 609700 addr = 10.60.0.0 plen = 20 vrf = 4 operation = DELETE nh_idx = 51 After executing this sequence, it was observed that the 10.60.7.0/25 and 10.60.7.128/25 routes were getting deleted as part of 10.60.0.0/20 delete operation. This was because as part of delete operation, we were deleting the mtrie bucket containing 10.60.7.0/25 and 10.60.7.128/25 routes wrongly due to incorrect logic/assumption in code. Fix: The mtrie bucket deletion logic is changed to delete the bkt using one of the entries in the bucket instead of deleting it using the values from the route delete request. This fixes the issue of bucket with non matching prefix length being deleted. Verification: The fix was verified by rerunning the same sequence again and checking if the 10.60.7.0/25 and 10.60.7.128/25 routes are present or not. A UT script has been added for the same. The fix also passed full vrouter regression in 3.2.3.x branch. Change-Id: If558d05bb666b48bb2ae48a34ec03692c57f74fb

Removed cloned original counter from dropstats output and created new option "dropstats --debug" to display cloned original output. Closes-jira-bug: CEM-11520 Change-Id: Ibb2daf86b6e7a632f33214e24d64b2110491e99d

1. Convert run_test.sh to python script 2. Move tests directory to $SB/vrouter/tests 3. Move vtest_py vtest specific libraries to $SB/vrouter/utils/pylib 4. Implement business logic for classes mentioned in skeletal implementation CEM-11303 5. Move UTs' test_basic, test_basic1, test_basic2, test_vrf_table to new infra 6. Added pep8 check in SConscript Blueprint: https://github.com/Juniper/contrail-specs/blob/master/vtest_py.md Closes-Jira-Bug: CEM-8197 Change-Id: Ia369f2c46e0e040b6c5f7f9670a88608fd915f60

Closes-Jira-Bug: CEM-10978 Change-Id: Ieccb3de7bb3d7aea63c2711da09827ad7acaf91d

Closes-jira-bug: CEM-12596 Change-Id: Ia96779ebab14f8abf868ba93ec0cbf8b6881af0a

…uments Closes-jira-bug: CEM-12596"

Closes-Jira-Bug: TFB-1556 Change-Id: I9e706213ce9e35f848e9dbd167bae7f036442262

vt_gen_message_modules.h. then it complains about a dependency cycle when one attempts to depent the header from its generator built from the source file. use the workaround to fool the scanner and break the loop. partial-jira-bug: TFB-1557 Change-Id: I694a52161fac22ba28d93af25f94dd2b32f2cbcd

Add these DPDK performance tuning CLIs to support full CPU isolation: 1) yield 2) lcore queue length (tx) 3) lcore queue length (rx) 4) mask for service cores 5) mask for DPDK control threads Refer https://contrail-jws.atlassian.net/browse/CEM-10093 for details. Closes-Jira-Bug: CEM-12161 Change-Id: I99ec6274e23f138d6344b23ba7e6b89113ec9d6b

Closes-Jira-Bug: CEM-10978 Change-Id: I15be5ebece7c85ae1758b1e65a2df6a8b56ff1e6

Closes-Jira-Bug: CEM-11409 Change-Id: Id251228cd8012851f5860f82fc6acc3a04b5abcc

…from vt_gen_message_modules.h. then it complains about a dependency cycle when one attempts to depent the header from its generator built from the source file. use the workaround to fool the scanner and break the loop."

When running in the concurrent mode, replace with shell actions. Depend explicitly on sandesh_gen generated artifacts to establish correct requirements from vtest. closes-jira-bug: TFB-1557 Change-Id: Idfa98cc284a1241153a2b1dc1ee3e0bdd765ae87

With the agent changes to support HBS with service chaining, packets can now enter HBS-Left from fabric. In this case, stamp the dmac with the flow id before sending to hbf-l. When the packets exit from hbf-r, search both dmac and smac for flow id using the magic number. Then reconstruct the mac using the actual mac after flow lookup. Closes-Jira-Bug: CEM-12574 Change-Id: Iaab3b99c7de83005ebbb55d69cb58e24c5c758a5

run_tests.py should exit with 1 if there is any failure while running tests which would fail CI. Hence fixed the script to exit with 1 on failure of any test. Closes-Jira-Bug: CEM-13022 Change-Id: I0c5e3b2f0119695808f9b6b695794150affd44a0

When VIF is added but not connected, vru_cl->vruc_fd is not added to the fd list even though it is created. This can happen when VM is stopped. In this case, vr_uvhost_del_fds_by_arg() would not close the fd. This leads to socket leak, so added fcntl() call in vr_uvhost_del_client() to check if vruc_fd is closed or not. Closes-jira-bug: CEM-9285 Change-Id: Ia55e44b8830b96e8313691dc77873bfd133e1469

UNIX_PATH_MAX is set as 108 bytes(/usr/include/linux/un.h), but vtest absolute path overflows(crossing 108 bytes). For vtest, vr_socket_dir moved to '$sb_path/.../var/run/vrouter' to '$sb_path/.../sock' and enabled few checks. closes-jira-bug: CEM-13395 Change-Id: I5cdc8638ef46f878b6ff66e98ec184f18ba8c320

1. Add mirror support to the infra. 2. Merge HBS UTs to existing UTs. Closes-Jira-Bug: CEM-11749 Blueprint: https://github.com/Juniper/contrail-specs/blob/master/vtest_py.md Change-Id: Ib12c4b01afb3610fc75f383dd2a86ac55d618588

1. Have added vxlan.py 2. Changed usage of python list to python dictionary in object_base.py 3. Have added implementation of new packets to packet.py for new test cases 4. Have added basic test cases for the implementation of mirror and mpls classes Closes-Jira-Bug: CEM-11749 Blueprint: https://github.com/Juniper/contrail-specs/blob/master/vtest_py.md Depends-On: Ib12c4b01afb3610fc75f383dd2a86ac55d618588 Change-Id: I61ba8bdc7a27ab385f8e3f7718d57fa00aa53ca7

Removing tests, lib and run_tests.sh as we are using new python based infra. Change-Id: Ic08c5fcfcc0d5779f0d59ef71801eae8dd7e38e8 Closes-Jira-Bug: CEM-13770

Closes-Jira-Bug: CEM-11749 Blueprint: https://github.com/Juniper/contrail-specs/blob/master/vtest_py.md Change-Id: Id39926b6e84e210589969db5658319c7958c10cb

This reverts commit 47ae8ac. Reason for revert: Temporarily reverting this to unblock ubuntu kernel builds for 4.15.0-91 kernel Partial-Jira-Bug: CEM-13877 Change-Id: I9bc2d3c582503c03d63e21a7a965ce7dc31467fb

…as errors""

Add support for 2MB and 1GB huge pages for vrouter kernel module The bridge and flow table are allocated by kernel vrouter using the huge pages configured by provisioning. For more details, please refer to Blueprint - https://github.com/Juniper/contrail-specs/blob/master/ziu_vrouter_huge_pages_support.md Vrouter changes - Sandesh changes for agent to pass the hugepage memory allocated for the tables - Change to return flow table and bridge table size properly during init (even before they are initialized) - Handling huge page config received from agent and pinning the pages properly. To ensure contiguous memory, using vmap for all the pages returned by kernel in case multiple hugepages are involved (NOTE: the kernel doesn’t gaurantee contiguous memory across huge pages) Unit testing - Tested with VN, VM creation - Tested with flow traffic (including verfication of flow table and bridge table) - Tested with agent restart - Tested with default and 4M flow table size - Tested all the above with 2MB and 1GB huge page sizes - Tested flow traffic without any huge page allocation (sanity) Partial-Jira-Bug: CEM-11529 Change-Id: I6d6e16feab3e888cea14ce19a37fbec2ebe45c95

1. Create new gdb macros to dump flow details from vrouter core file. 2. Address comments and changes suggested in nexthop, vif, btable and flow files. 3. Add a readme file. Partial-Jira-Bug: CEM-8854 Change-Id: I4879cd52619d7f65517d24caefa07af6aeb4c88c

The first packet fmd was not having the Dest VRF set. This is because the packet was enqueued and sent to agent. Once agent responds, the packet is re-injected. In this path, the VRF was not set. As a result, the first packet would be dropped. Closes-Jira-Bug: CEM-11144 Change-Id: I350f46ec9f9b558c4c04d1246f130ec4a891b7c8

Fix is to reset the default to zero Closes-Jira-Bug: CEM-14618 Change-Id: Ib14f14d9743c3c53af78c314a6322ca1c538836e

scons vrouter-py-ut:test failing because of virtualenv using python3 in some sandboxes. We need to use python2 with virtualenv as the tests are not python3 compatible. Also removing test from SConscript as test folder is not present now. Setting correct socket path in run_test.py Closes-Jira-Bug: CEM-14626 Change-Id: I208c32bab142dc84d17166a6df405dfafcdb99da

Need to pin all packages in requirements.txt to guard against future failures when upstream packages get upgraded. closes-jira-bug: CEM-14661 Change-Id: I3a4e669bf765d4ae74c5cf355de9ad0ee3d121a6

Need to pin all packages in requirements.txt to guard against future failures when upstream packages get upgraded. closes-jira-bug: CEM-14661 Change-Id: Ibd935ac6583c0c9e8527d4b181ad91487483b016

… effect"

1. Add support in run_test.py to execute a single TC Eg: ./run_test.py -t test_vm_to_fabric_intra_vn.py:: TestVmToFabricIntraVn::test_vm_to_fabric_intra_vn 2. Add support in ICMP to create custom size packets 3. Add support to get a packet generated by vtest app in python 4. Add validation in UTs to validate received packet 5. Fix bugs in UTs Closes-Jira-Bug: CEM-14859 Change-Id: Ibb92a4dcc552dbe8e5e52d0653330ed81e4e249e

when kernel is using hugepages On newer ubuntu kernels, a traceback related to bad PTE entries is seen when agent is exiting. The same is also seen when flow and rt commands are used. The traceback is seen when unmap for /dev/vr_bridge and /dev/flow is being done. It is not clear what is causing this. The same is not seen on older kernels used by RHEL and Centos. To avoid this issue, agent/flow/rt need not use /dev/vr_bridge and flow devices since in kernel with hugepages mapped, agent/flow/rt can directly use the mmaped /dev/hugepages/bridge and /dev/hugepages/flow files. Vrouter changes: 1. Get the hugepage file path sent by agent and store it in the internal hugepage cfg struct along with other hugepage information. 2. Given the stored hugepage file path to flow and rt command so that they can mmap the tables properly. Testing: 1. Tested with kernel mode without hugepages enabled 2. Tested with kernel (RHEL/Centos and Ubuntu) with hugepages enabled 3. Tested with DPDK compute partial-jira-bug: CEM-14660 Change-Id: I2a916c23088221d8a2ee236bac3a4bde78384142

…n agent when kernel is using hugepages"

vr_socket_dir path length should be within UNIX_PATH_MAX(108) bytes for vRouter and vtest. With below fix, if vr_socket_dir overflows, log warning message and set vr_socket_dir to "/tmp/sock" in run_test.py script Closes-Jira-Bug: CEM-14797 Change-Id: I0c15e474d474c88ed516ec25938cff0dc7bde0e3

They have changed the fault callback signature in RHEL 8.2 kernel. Update it and use a suitable return code in case of the single failure path in the implementation. partial-jira-bug: CEM-7396 partial-jira-bug: CEM-11348 Change-Id: I63637d8127ebcc18bdfc0b2b4e512c5db0fe447c

Devloped vr_info framework to process multiple CLI requests. - Have created CLI for DPDK(dpdkinfo) and for kernel(vrinfo) - Handles CLI request in parallel. - Supported for multiple platforms(Kernel/DPDK etc.) - For DPDK, Displaying the bond master & slave information. - For Kernel, Have written a sample code to display vRouter version. - Register callback function in vr_info.h(VR_INFO_REG macro) and based on CLI request, vr_info would call those callbacks and dispaly contents in CLI. - Added (--buffsz <value>) support to set output buffer size from CLI. Closes-jira-bug: CEM-7025 Change-Id: Ic20da8f8c3e4f56acd0423c974ac060573774c20 Depends-On: Ie4733cc4cc9838bee0230b0668baa1d9ba191eb5

- Reorganize code to add basic topologies as base classes. This allows re-usability of these classes. - Move all the HBS related UTs to a separate file. Closes-Jira-Bug: CEM-14859 Change-Id: Ie0b363f0c7c823702b2422865192769d87d25dfd

1. Create new GDB macros to dump rt and bridge details from vrouter core file. 2. Minor changes to vr_vif.gdb file. 3. Source inclusions for new files in vrouter.gdb file. 4. Address comments wrt vr_bridge.gdb and vr_rtable.gdb files. Partial-Jira-Bug: CEM-8854 Change-Id: Ib317a01ba2c33973b12d1917ef5d5119ed27bcdd

Root cause: In case of local ECMP processing, as in fip is on one VM on local compute and another VM on remote compute, when the pkt is chosen to be sent to remote compute (by agent selecting the remote ecmp idx), the fabric vif rewrite (L2 rewrite) was not happening due to some special checks in the code. Fix: Changed the check condition to also check for dst mac of the pkt. If dmac of the pkt is equal to vhost0 mac, then the pkt is being routed through vrouter and we need to honor the NH found during route lookup. In such cases, we need to do the L2 rewrite using NH information. closes-jira-bug: CEM-13716 Change-Id: I90dc16b5e48aba6156eddfef7b1486b6faf10741

SIGKILL is being sent to kill vrouter instance at the end every UT. Due to SIGKILL .gcda files required for code coverage report are not being generated. Changed the kill signal SIGTERM instead. Closes-Jira-Bug: CEM-15230 Change-Id: I415f285bda991ecae49e7dfbd33d075b8018736d

The DPDK rte_flow API is not thread safe. Using default configuration, multiple dataplane threads are trying to insert or remove flows on the same port, which is not supported. Introduce one spinlock per ethdev port using DPDK implementation. The spinlock is used when managing flows for this port. Closes-Bug: TFB-1578 Signed-off-by: Gaetan Rivet <grive@u256.net> Change-Id: I24e99bef331d0a15cf749ad3c5fa4552ead1b009

The following cases would be supported with this commit - 1) Packet mode + Fragmentation + Dst ECMP 2) Packet mode + Fragmentation + Dst ECMP + Mirror dst with ECMP 3) Packet mode + Fragmentation + Dst ECMP + Mirror dst without ECMP Changes done: 1) Handle fragmentation in case of ECMP with packet mode 2) Pass packets through the fragment assembler in this case 3) Add a custom field in the hash key of the fragment assembler 4) For mirror packets, mark the custom field as 1 5) Delete frag_calc logic and in all cases, remove the fragment entry from the table when the last fragment is received 4) Handle cases where fragment addition in fragment hash table return error 5) Make the fragment hash table and assembler table scanning more aggressive 6) Move all the #defines of fragmentation in one place 7) Add more comments to important fragmentation functions 8) Add UT suite for fragmentation 9) Some fixes in UT infrastructure Closes-Jira-Bug: CEM-13355 Change-Id: I305efd007e7b3b6899f4bac356dc2d1bd44d2f1e

Fix is to calculate the number of fd's after deleting the interface. Closes-jira-bug: CEM-15560 Change-Id: Ia9a7d7dd2b4fc31d5888584a44f3eb7045d7a380

Add a flag vr_close_flow_on_tcp_rst to enable/disable disconnection of TCP flow on receiving TCP RST packet. Closes-Jira-Bug: CEM-10847 Change-Id: I828d1e1797dfdca9022fe74f9217e1e905be27e2

1. Adding code snippet in the main() to set coredump_filter for vrouter dpdk. 2. Addressing comments and making changes. Closes-Jira-Bug: CEM-14017 Change-Id: Idd6be41350728150088e8ed046a8433f7baeeb93

When vRouter was stopped, rte_exit() calls log write function. But, the log file pointer is closed by an earlier call of dpdk_exit() Hence there was a crash. Fix is to close the log file after rte_exit() Closes-Jira-Bug: CEM-15695 Change-Id: I46a42aadb44f6e08dfd0c0272fe9259e73b82f1c

… information Below commands are supported for Dpdkinfo: 1. Bond: --bond (Display the bond master and slave information) 2. Lacp: --lacp all (Display all information related to lacp) --lacp conf (Display config information) 3. Mempool: --mempool all (Show summary of used/avail mbuf for all mempools) --mempool <mempool-name> (Display information about the specified mempool) 4. Nic info: --stats eth --xstats (Display non zero values for Master and slaves) --xstats=all (Display all values for Master and slaves) --xstats=<interface-id> (Display non zero values for given interface-id, =0 (Master), =1 (Slave(0)), =2 (Slave(1))) 5. Lcore: --lcore (Display lcore information) Closes-Jira-Bug: CEM-15000 Change-Id: I2c94eb8a44df19915ca0d23d10e2824a6462dbd7

…re, app information"

Fix: When vRouter acts as gateway, Allow traffic between fabric-fabric, otherwise drop it. Closes-jira-bug: CEM-13067 Change-Id: I05e390fcbf8ce9391ea0713706a582aa93c97a4a

1. Added Wireshark plugin files that parses through the buffer 2. Added 22 sandesh object scripts that are used to parse the objects' buffers 3. Added Sandump tool (formats and converts sniffed bytes to pcapng) Closes-Jira-Bug: CEM-9260 Change-Id: Idf9a842021a0d8f5e1188d4e869f0111d64d3f41

Adding an automated UT test case for the bug. closes-jira-bug: CEM-13716 Change-Id: I25556a5a17d371a6319da28958540d3dd0308e18

warnings as errors. Closes-Jira-Bug: CEM-10978 Change-Id: Icbd33700185fd93f5e9962ef78cbf3f178883e6a

…nings as errors."

…out migration. Automated change Depends-On: I76b24276f0f9f7dbd4361d06ec9a2bad07f1349e Depends-On: Ia502f7994fe5c3935838fb31a5210e5389fa943b Depends-On: Ia92e5a026af7ebbd1b479e372e8b4b447a8e9b67 Depends-On: Ib28e845dd30bb42034ff6dbeecc2dd7cb7e85c64 Depends-On: Ib52824a4677be5745b265f3a2b9c3fac58a9b0a0 Depends-On: If2956c78c81d8a62dfdf9ad610b04e0c41a3565d Change-Id: I6fc67f02ea6e9ba1f82b23063c976fba9a779f89

GRE over IPv4 traffic from VM to Contrail is not destributed #99

Are you sure you want to change the base?

GRE over IPv4 traffic from VM to Contrail is not destributed #99

Commits on Feb 18, 2019

Commits on Feb 19, 2019

Commits on Feb 20, 2019

Commits on Feb 21, 2019

Commits on Feb 22, 2019

Commits on Feb 25, 2019

Commits on Feb 27, 2019

Commits on Feb 28, 2019

Commits on Mar 1, 2019

Commits on Mar 2, 2019

Commits on Mar 4, 2019

Commits on Mar 7, 2019

Commits on Mar 12, 2019

Commits on Mar 13, 2019

Commits on Mar 14, 2019

Commits on Mar 15, 2019

Commits on Mar 18, 2019

Commits on Mar 19, 2019

Commits on Mar 21, 2019

Commits on Mar 22, 2019

Commits on Mar 23, 2019

Commits on Mar 25, 2019

Commits on Mar 26, 2019

Commits on Mar 27, 2019

Commits on Mar 28, 2019

Commits on Mar 29, 2019

Commits on Mar 30, 2019

Commits on Apr 3, 2019

Commits on Apr 4, 2019

Commits on Apr 5, 2019

Commits on Apr 8, 2019

Commits on Apr 9, 2019

Commits on Apr 10, 2019

Commits on Apr 11, 2019

Commits on Apr 12, 2019

Commits on Apr 16, 2019

Commits on Apr 18, 2019

Commits on Apr 22, 2019

Commits on Apr 23, 2019

Commits on Apr 26, 2019

Commits on May 3, 2019

Commits on May 7, 2019

Commits on May 8, 2019

Commits on May 9, 2019

Commits on May 13, 2019

Commits on May 14, 2019

Commits on May 15, 2019

Commits on May 16, 2019

Commits on May 21, 2019

Commits on Jun 6, 2019

Commits on Jun 7, 2019

Commits on Jun 11, 2019

Commits on Jun 13, 2019

Commits on Jun 19, 2019

Commits on Jun 21, 2019

Commits on Jun 25, 2019

Commits on Jun 26, 2019

Commits on Jul 1, 2019

Commits on Jul 3, 2019

Commits on Jul 9, 2019

Commits on Jul 15, 2019

Commits on Jul 16, 2019

Commits on Jul 18, 2019

Commits on Jul 19, 2019

Commits on Jul 22, 2019

Commits on Jul 29, 2019

Commits on Jul 30, 2019

Commits on Aug 2, 2019

Commits on Aug 6, 2019

Commits on Aug 7, 2019

Commits on Aug 13, 2019

Commits on Aug 20, 2019

Commits on Aug 21, 2019

Commits on Aug 23, 2019

Commits on Aug 29, 2019

Commits on Aug 30, 2019

Commits on Sep 3, 2019