Change-Id: I12ffeccc1dec08705ac16a6a5bd1f998920b9b46

Partial-bug: #1696209
DPDK 17.02 has changed the RTE_LOG api. Need to use RTE_LOG_DP to
avoid compilation of the log funciton.

Change-Id: I674154764a4fa3529f28d383f334b83c71f7040a

To convert the hash table entry to fragment entry, hash table API's like
vr_find_free_hentry() are directly passed to Fragment Macro
VR_FRAGMENT_FROM_HENTRY(). As the function address gets passed to the
macro, function gets invoked twice leading issues like fragment never
getting added to fragment table

As a fix, the hash table API's return value is stored and that is
passed to fragment Macro

Change-Id: I9350551f6bc3a6b28eaf8acea8df2d2ec42c3eae
closes-bug: #1721251

Partial-bug: #1724326
Used tcpnodelay option for the tcp socket between
agent and dpdk vrouter to avoid inconsistent delays.
This was causing huge variation in hold entries.

Change-Id: Ide2bcfaa28c992e31129a25b444028efce2f3c15

If ACL's are configured on Vhost interface, the packets need to be
subjected to flow processing. These packets would be un-tunneled
packets. The Vhost route in VRF 0 points to L3 Receive Nh. This NH can
not be marked with policy bit as we do not want to create Flows for
outer IP fields of Tunneled packets which also get processed by L3
receive NH. Also if Policy on Vhost is disabled and if L3 receive NH is
enabled with Relaxed policy, packets have to be flow processed  for
link local processing. For this reason, policy bit on Receive NH's
outgoing interface is verified to decide whether to invoke flow processing
for packets

Change-Id: I86b8ea2ecdf1460bb0d3599675d84ad46dfcf1e8
closes-bug: #1711045

For Vxlan tunneled packets that are received on Fabric interface RPF
callback was missing, leading to no RPF validation of the vxlan packets.

Due to Tor Evpn support, Tors can be in Ecmp, which is a composite Ecmp
nexthop in Vrouter. When the first packet corressponding to unique 5 tuple is
received on Fabric (rather from VM) from one of the Ecmp sources of Ecmp
composite nexthop, the component nexthop is pinned to the source, only if
RPF callback exists. Lack of this RPF callback was making the Ecmp go
wrong

As a fix, Vxlan RPF call back is provided

Change-Id: If53a0bc76398cfc8c176a3a94a0aede8b26262b4
closes-bug: #1724681

Change-Id: I2135446723dddf526babf1f033dd1492e22e9cf3
closes-bug: #1729818

Vrouter currently allocates contiguous memory for Flow and Bridge
tables and this forces the module to be inserted into kernel at the
boot up time as huge memory is likely to be presetnt at the boot of the
sytem. This requirement is taken out with the huge pages support.

Provisioning:
If the huge page support is intended in the system, provisioning of the
compute node takes care of enabling the required number of huge pages in
the system. After enabling the huge pages successfully, the hugetable
file system is mounted at a specific location and creates the required
files (2 files as of now - vrouter_mem1 and vrouter_mem2). The location
of these files is made avaialble to Agent.

Agent changes:
Agent looks for the huge files at the specific path provided by
provisioning (through agent configuration file) and mmap()s the 1G size
of data. If succeeds, Agent shares this virtual memory address with
Vrouter kernel module through netlink socket. If the huge files do not
exist, or if the mmap fails, still it communicates the failure of this
to Vrouter kernel module. This messaging is done in blockig way, so that
no other configuration is added to Vrouter till this is communicated.
Along with 1G pages, Agent attempts to allocate same number of 2Mb
pages, which Vrouter internally uses. Even these page addresses are sent
to Vrouter kernel module using same sandesh message

Vrouter changes:
1) The module_init routine is changed in such a way that, the modules like
   fib and flow which require huge contiguous amount of memory do not
   allocate any memory in their init routines.

2) A new call back "mem" is introduced which allocates the required
   memory and is invoked after succesfully initialising the hugepages
   when Agent communicates it.

3) The modules like interface etc are initialised as usual so that
   addition of fabric and Vhost interfaces are created normally and
   pushed to cross connect mode till the other configuration is
   available

4) When Agent communicates the virtual memory address, Vrouter pins
   these pages using get_user_pages(). The "mem" callback is invoked
   which allocates the memory either from huge pages or using regular
   btable. The memory required to hold this huge page in terms of 4Kb
   pages, is another huge page of size 2Mb. This 2Mb huge memory is also
   sent by Agent. If no such huge page of size 2Mb is sent by Agent,
   Vroter allocates the memory using regular malloc.

5) As the momeory allocate from the huge pages is intended for the
   entire life time of Vrouter module free() routinues are not provided
   for these memory segments. These pages are freed from the system when
   the module is removed from the kernel.

6) If agent restarts, memeory is not reinitilised again and proper
   return codes are shared with Agent

Change-Id: I942ed42463d7a0aba7bf857df7c62d9c95b83056
closes-bug: #1728925

Fortville NICs do not allow setting the MTU while the NIC is running.

Change-Id: I4280715db2f3085041aa7bb5796665a65409effd
Closes-Bug: 1729742

Currently the healthcheck is broken if policy is enabled on Vhost
interface. Health check packets are destined to metadata IPs. The
metadata routes point to VMI's in Vhost VRF. Regular VM's routes are
present in VN's VRF. If VM is detected unreachable, HC withdraws routes
in VRF corresponding to VM, but metadata Routes are not withdrawn. HC
packets should be routed using these metadata routes, if routes are
withdrawn. If Vhost has policy enabled, flow processing happens earlier
than route look up and destination metadata IP's are Nated to VM's IP.
These Nated IP's would be looked up in VN's VRF which results in drop
nexthop as routes are withdrawn.

The solutino is not to Nat the packet till the route lookup is complete
for these metadata IP's if the policy is enabled. The required flow
processing would be completed if the nexthop is marked Policy enabled.

Change-Id: I144c36faf39b062026316a067e912eed5a2fa792
closes-bug: #1724945

Currently the Ecmp component NH is calculated for IP/IP6 packets only as
Ecmp has been supported for only for L3 packets. With the support of L2 Ecmp, the
component NH needs to be chosen even for L2 packets. As there is no Flow
available for L2 packets, the hash is calculated on Ether dst, src and
VRF

Change-Id: I61085c729da9633630088604c1a6b8db5897bba8
closes-bug: #1732285

Currently, if there is a sub interface created with a Vlan Tag on any
parent interface and if mirroring is enabled on subinterface, the Vlan
Tag of the subinterface is also carried in the mirrored packet. Incase
of Vmware compute, all the VMIs are configred in Vrouter as
subinterfaces with Pvlan. And if the mirroring is enabled on these
VMI's, as the VMI's are subinterfaces, even these vlans are carried in
the mirroring. But the expectation is not to carry these Vlan tags as
these Vlan tags are Contrail specific and not seen by the user.

As a fix, there is a new VIF flag introduced "VIF_FLAG_MIRROR_NOTAG"
which is set by Agent on Vxmware VMI's only. If these flags is seen on
VIF, Vrouter discards the  Vlan Tag before mirroring the packets

Change-Id: I9bd8c33c735159937f4b325a6ee67540f4f15f39
closes-bug: #1711459

This is sandesh change for using different multicast VRF in terface req for VN with provider network

Change-Id: I0c0310232b5e44eec780ad54d552e6750d42b9a6
partial-bug: #1728545

Currently if a VN is configured with Provider network, the VRF of the
VMI's belonging to that VN are seen as Fabric VRF in Vrouter. But the
Multicast/Broadcast tree is built in the VRF corresponding to VN. When
BUM traffic is received on such VMI, traffic needs to be replicated as
per the mutlicast tree of VN's VRF rather on Fabric VRF. To achieve
this, the VN's VRF is added to VMI as different VRF and the bridge
lookup is done for this traffic in this new VRF.

Change-Id: I38721e27cebe80c7f4d14937588ba5b6c180b112
closes-bug: #1728545

With the introduction of Xps (Xmit packet steering), the sender_cpu
needs to be cleared before dev_queue_transmit(). Not doing this results
in accessing netdev's xps map at a wrong location resulting in a
crash. Clearing this makes the hard xmit calculate the sener_cpu

Change-Id: Ifb0757ffdaa3e27b15261a9944281d9224afa8f0
closes-bug: #1733431

When L2 Ecmp is not in place, packets to/from Ecmp source are forced to
L3 packets by replying ARP requests using Vrouter's and Vhost's MAc.
This is no longer required once we have L2 Ecmp in place.

So all ARP processing which specially deals with Ecmp source side is not
reqired any more. This processing is removed from the current ARP
handling

Change-Id: I484da69b3c173891b915fb86b81c2e57d711b18a
closes-bug: #1733811

Incase of TCP a session to VRRP address, if VRRP mastership changes,
flow's RPF nexthop changes as RPF nexthop needs to start pointing to new
Master. But the TCP other end might send a FIN/FIN-ACK to old VRRP
master with the intention of tearing down the session. Due to RPF
failure these packets get dropped. Before dropping the packets itself
Flow getis marked with the required flags. Incase of last FIN-Ack packet
being dropped, the Flow gets marked as Dead but eviction does not get
kicked-off as eviction is not invoked for a packet that is dropped. Due
to this, the flow never gets evicted and Agent also might not delete
this flow as part of Aging, if these flows are for BGPaaS. To fix this,
eviction is kicked-off, if the Flow is marked Dead and even if the
packet is dropped.

Change-Id: Ib4a527477403f40bc3016c7ea58813f168a81e02
closes-bug: #1733608

rt --monitor can be used to get route creations and deletions in
live. They are broadcasted on Netlink by the vrouter kernel module.
Only routes of type AF_INET and AF_INET6 are fully printed while
routes of type AF_BRIDGE are not generated yet.

The output format is 'jsonline' to be easily parsed by external
tools.

This is an example of the output:
{"operation":"add","family":"AF_INET","vrf_id":2,"prefix":32,"address":"20.1.1.254","nh":12,"flags":{"label_valid":false, "arp_proxy":true, "arp_trap":true, "arp_flood":false}}

Change-Id: I8cb8556f1c4adda0bb0ef10f98fc38702b2942c4
Closes-bug: #1650316
(cherry picked from commit 12b9b49)

If fragmentes are being processed, fragment assembler maintains the
statistics of framents in the fragment entry to decide when to delete
the fragment. When mirroring is enabled, the mirrored packet gets
processes earlier than real packet and these fragment calculations are
invoked on the mirrored packet leading to fragment entry being deleted
if the mirrored packet is the last fragment of the packet. Once the
fragment entry is deleted and mirrored packet is out of system, the real
packet gets processes and this packet will not have a matching fragment
entry to do the flow processing resulting in packet being discarded.

As a fix, the fragment calculations are invoked only when the real
packets are processed rather for mirrored packets.

Change-Id: I30b3092622f8c661c6bebaf5ccbff6c9621cc3dc
closes-bug: #1739602
(cherry picked from commit c9bf938)

Closes-bug: #1734994
Send netlink updates to agent for the VM's port in
dpdk mode of operation when VM state changes.

Change-Id: I9bc3cf8da01ed97ea2409ea2c16239447a867924

VIF_FLAG_GRO_NEEDED and VIF_FLAG_ETREE_ROOT flags were conflicting

Change-Id: Iefa51d3561fc208690fef9b1e6942167a3d925a9
Closes-Bug: 1748261

…from the vrouter,

instead of reading each flow entries and counting hold entries based on its status.
Now with this change flow -s and flow -r will directly read the hold entries from the vrouter.

Change-Id: I2d873583ac9668d9bc38c2305d86f3256ab29d94
Closes-Bug: 1738282

…ntries from the vrouter, instead of reading each flow entries and counting hold entries based on its status. Now with this change flow -s and flow -r will directly read the hold entries from the vrouter." into R4.1

Closes-bug: #1750711
When the dpdk vrouter tries to forward the traffic
after the vif gets deleted, the vif data structure
is access to update the statistics. This is causing
the vrouter to crash. Fixed by removing the stats
update after the vif is deleted.

Change-Id: Id3dcf31a8cdf98d6a6cc92ff552963f490476c68

Make the vif argument as NULL in pkt_drop_stats() in KNI send_burst().
There may be cases when vif is deleted after the packet is sent to KNI
and when such packet hits this code, there may be a crash since the
function tries to access the deleted vif.

Change-Id: If180909a1b9828b97310329feebd32371e965c11
Closes-Bug: 1714156
(cherry picked from commit b41d869)

Closes-bug: #1734994
Fixed crash related to the port status update change.
The reset status was called with an uninitialized variable.
Moved the reset status to the correct place.

Change-Id: I8bb2eb22a860429c02857597542f713705aafa20

When calling bridge_table_dump with a request containing a wrong MAC length, the dumper structure is not deallocated on the error case.
Moving MAC size check before dumper initialization.

Change-Id: I432819d7bcf729b4b17df280440547c3bf87a529
Closes-Bug: #1736712
(cherry picked from commit c15068f)

The result of pkt_pull is checked to be below 0 in case of error
(in nh_l2_rcv and nh_handle_mcast_control_pkt).
On the other hand, pkt_pull returns a pointer, which
makes the comparison pointless.

Change-Id: I1481a58a84f8872f05cf7bc4a045d1e41f6181d6
Closes-Bug: #1739010
(cherry picked from commit fe9d4ad)

vif.c:
Initializing ret variable in vr_intf_op.

udp_util.c:
Using udp_free_client instead of udp_free, so that cl is freed as well.

nl_util.c:
Initializing attr in nl_dcb_parse_reply to DCB_ATTR_UNDEFINED.
Using nl_free_client to also free cl in nl_register_client. Otherwise, fields are freed but not the structure itself.

ini_parser.c:
Closing file before returning from parse_ini_file if ini_data allocation fails.

flow.c:
In flow_set_ip, updating match_string only if token is not null and token_length is assigned to and makes sense.
Initializing ret with 0 in flow_set_ip. It would be used uninitialized in the context of token being null.
In flow_set_match, updating match_string only if token is not null and token_length is assigned to and makes sense.

Change-Id: I04fbb636497f083099375c643d32a8997d0a4c0d
Closes-Bug: #1736160
(cherry picked from commit 5fa4855)

vr_qos.c:
In vr_fc_map_get, control may reach the generate_response label with an uninitialized resp variable.
Initializing resp to NULL.

vr_route.c:
In bridge_rtb_family_init, if algo_init fails, table is not deallocated.
Freeing table on error case.

vr_packet.h:
fmd_queue field is not initialized (it doesnt't look used anywhere either).
Removing fmd_queue from forwarding metadata structure

Change-Id: Ife54d3ff95636535faa9f05ea03bb8eae7398231
Closes-Bug: #1737141

Change-Id: I8ef5d8a355b796d52d9f09a924432135ca912029
Closes-bug: 1751362

…to R4.1

Change-Id: I0d1a20adc9ec2afb7f102e8cbbe27d42f80d81ac
partial-bug: #1742044

Currently mirror metadata is stored in index table using flow index
as key. Before the Flow eviction, the addition and deletion of this meta
data to Index table is always from Agent and hence no parallel
manipulation of Index table. After eviction, the metadata deletion from
index table can happen in parallel and hence leading to memory
corruption/leak in index table manipulation.

The right fix is making the index table SMP ready. As a stop gap fix,
the mirror metada data pointer is stored in the flow entry itself
directly.

Conflicts:
	include/vr_flow.h

closes-bug: #1572397, #1744369
Change-Id: I216d8b6c55025a6053b808fd2807338265b307e0
(cherry picked from commit a437faa)

…nto R4.1

Some NICs like XL710 need more number of tx/rx descriptors. Hence make it configurable

Change-Id: I395ef2d9c6d6ce87ad624e6097b1e77c97b27848
Closes-Bug: 1744185

Add DPDK library version in the log

Change-Id: I049ef94704b6d909bb2e1f6636416c1c123eb9c8
Closes-Bug: 1760125

Partial-bug: #1748012
Redhat doesnt support KNI and uses tap for the vhost interface
The data path code seems to check for data in all the tap
interface irrespective of whether its mgmt interface or data
vlan interface. Fixed by adding check in the data path and
fwd only for datapath traffic. Mgmt traffic will go through
a slower tapdev loop.

Change-Id: Id1ef38592f3319d1894bf981d0cc4c418cfd676f

- skb_sender_cpu_clear has been removed in kernel 4.6 [1] and is no longer
  required since 4.5 [2].

- get_user_page signature has changed in 4.6 [3] and in 4.9 [4]. Since I
  don't have a 4.6 kernel for testing, I didn't add code to support it.

- The macro page_cache_release has been finally removed [5].

[1] torvalds/linux@64d4e34
[2] torvalds/linux@52bd2d6
[3] torvalds/linux@c12d2da
[4] torvalds/linux@768ae30
[5] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=09cbfeaf1a5a67bfb3201e0c83c810cecb2efa5a

Change-Id: I569ca66145204f6e71a1083412bf2cb4caab818d
Partial-Bug: #1633387
(cherry picked from commit a0d6eb0)

- linux/vhost_dev.c:
  Rename netdev destructor to priv_destructor due to API change.
  New signature of rtnl_link_ops.validate due to API change.
- linux/vr_mem.c
  Remove vma from mem_fault function since VMA pointer now accessable
  via VMF structure.

Change-Id: I9ffc103e5e69649d2a691c7a378cfd12b2363320
Partial-Bug: #1633387
(cherry picked from commit 60c9031)

Closes-bug: #1764940
Issue because of some of the linux kernel
not having all the patches related to the retpoline
fixes. The RETPOLINE define needs to be enabled
in the Makefile based on the CONFIG_RETPOLINE.

Change-Id: I8026dcc16f7056329618e2721e932557d0349386

RHEL 7.5 has kernel version updated to 3.10.0-862.el7, which breaks
building of vrouter module.

Co-Authored-By: Raja Sivaramakrishnan <raja@juniper.net>
Co-Authored-By: Krzysztof Klimonda <krzysztof.klimonda@codilime.com>
Change-Id: Icb0467344c0023141be7e1dd9998d383bfaab14b
Closes-Bug: 1763656
(cherry picked from commit b948f9f)

…r_list

structure has changed in 4.15 releases. Also some changes introduced to
vr_host_interface.c, removing NETIF_F_UFO reference for v4.15.

Change-Id: Iaa194a5623d0f5311b708025e23076670c2e19c1
closes-bug: #1765229

Change-Id: I3bf6a426928a65b66c56ffbca1f318641f799d4b
Closes-Bug: 1736108
(cherry picked from commit c4c68b6)

Closes-bug: #1746578
Fixed the feature negotiation issue by storing
the feature in persistent store and load when
VM comes up.

Change-Id: I3f41079264f040e1b57a146eda7711b171e302ac
(cherry picked from commit 9800537)

Due to buggy virtio guest drivers, there may be invalid descriptor
addresses in the descriptor table. Add checks not to use those.

Change-Id: I48ad58a1d554e07a08ae3ed68dc0f51d8d1b028b
Closes-Bug: 1726459

…5. Timer_list structure has changed in 4.15 releases. Also some changes introduced to vr_host_interface.c, removing NETIF_F_UFO reference for v4.15." into R4.1

Change-Id: I4c0cb7f168ad285c530379f382a403b743560266
closes-bug: #1770734

This changeset provides a generic infrastructure for configuring hardware
offloads for vrouter. This defines function prototypes for callbacks which will
be registered by a vrouter offload module, which will be called at certain
points in the vrouter.ko processing (mainly around where it does configurations
of tables via sandesh messages).

Offloaded functionality includes flow, vif, MPLS, VXLAN, mirror, nexthop, layer2
and route tables. QoS API is defined but not currently supported by offload
module (offload module will return an error if configured).

Change-Id: I00119ec244bf0c3bc8fa218aae10dd1b164b47ce
Closes-Bug: #1767107
Signed-off-by: Frik Botha <frederick.botha@netronome.com>

Pass to vrouter module information about multicast group. Withouth that
user space client (vif) was unable to join vrouter netlink broadcast group
when kernel 4.10 or higher was used.

Change-Id: If1868b027ffcf015ad4c3c4c287260d432edd6b5
Closes-Bug: #1771999

Set needs_free_netdev to false because free_netdev is already
executed by vhost_dev_destructor.

Change-Id: Iad92cfffe25d510bb52f458fdf102e12aab28e3f
Closes-bug: #1772639

Advertise VHOST_USER_PROTOCOL_F_LOG_SHMFD to Qemu which is needed for
live migration and suspend operation of VM

Change-Id: Icc82218e0fb58c24076128681323171c2040e1ea
Closes-Bug: 1685409
(cherry picked from commit 95ea5f6)

The condition whether to segment a packet or not was wrong. The correct
condition is that if TSO seg size that the VM is giving in virtio
header is lesser than the packet size, it needs to be segmented.

Change-Id: I97cf8a23407dac08030c3df575b5a3522b06fd2e
Closes-Bug: 1772648

Due to a bad frontend driver, there may be memory corruption in
vrouter when it posts buffers with 0 length. Add
checks to prevent the same.

Change-Id: I9bd623b414f731704c0aa1d640aec34fec098c33
Closes-Bug: 1767225

vif get to be called only when argument is a number .

Change-Id: I67e1a16a97bc7a6884379fb77d4b2ddd59881992
Closes-Bug: #1752783

We are deleting vhost0 on agent soft reset in DPDK mode. The same is not
the case with kernel mode. For containerization, the requirement is not
to delete vhost0 on soft reset.

Change-Id: I717970b2aa43e7d76561ec21c040807c4d69bd66
Closes-Bug: 1756961
(cherry picked from commit 23cc1d3)

There was a side effect in the earlier commit that Rx direction will
have an issue due to the will_segment condition. In Rx, the packet gets
GROed and hits the same code path and has to evaluate will_segment as
false since the GROed packet needs to goto the VM. But it was evaluating
to true. Hence added an extra check.

Change-Id: I6216aef7cad3b372d1aa8286e4dc5c7f723dec2e
Closes-Bug: 1772648

vRouter-DPDK is unable to probe Netronome NICs, since the build system does not
link to the NFP PMD.

Change-Id: Ia93bd22de5dcfee6ad56bdb48ba288a7e80ef4db
Partial-Bug: #1774414
Signed-off-by: Frik Botha <frederick.botha@netronome.com>

…ed only when argument is a number ." into R4.1

separator as its a malformed request in vrouter cli.

    Closes-Bug: #1734896

Change-Id: Iad450df0d9196a90d2cf5bd51e41c79a9d00b47e

centos kernels. max_mtu is now part of net_device->extended struct. Corrected
the reference and the logic so that the initialization is not skipped.
closes-bug: #1790766

Change-Id: I9db33970943050566d3fb76986b06fa5563fddc8

vr_rflow_update_ecmp_index updates the ECMP next hop index for reverse
flows. Add offload hook such that this change reflects in hardware as
well.

Change-Id: I62a28ae276ef59557dfdf8e8cc884255150723ff
Closes-Bug: #1792378
Signed-off-by: Pieter Malan <pieter.malan@netronome.com>

Closes-bug: #1788492
The checksum code updates a wrong offset when
vlan is present for the tx interface. Modified
to calculate the offset based on the ethernet
header type.

Change-Id: I4f70359254ad2461d35c84a5612f4bf9a89325b6

Closes-bug: #1794934
During an agent restart, the vif gets deleted.
This causes the stored vhostuser feature set to
be deleted. Delete the stored feature only if
VM exits.

Change-Id: I06724440905b27b1087f294f8d9b6361144611f4

Closes-bug: #1794934
During an agent restart, the vif gets deleted. With server mode
configuration of vhostuser, we should not delete the socket
file in /var/run/vrouter.

Change-Id: I0050ea47e91151c7f3cb2f80191aa0c2bdddec70

If a packet is corrupted, the packet length in the IP header may be
incorrect and calculating checksum of such packets results in kernel
assert. Hence make sure the packet length is always less than or equal
to the skb length before calling the checksum function.

Change-Id: Ib6ed8887eafdc55726e1a31ec5e3fedf1e4fc456
Closes-Bug: 1800964