[gve] Use dummy interrupt to trigger completion writeback in DQO mode

When operating in the DQO operating mode, the device will defer
writing transmit and receive completions until an entire internal
cacheline (128 bytes) is full, or until an associated interrupt is
asserted.  Since each receive descriptor is 32 bytes, this will cause
received packets to be effectively delayed until up to three further
packets have arrived.  When network traffic volumes are very low (such
as during DHCP, DNS lookups, or TCP handshakes), this typically
induces delays of up to 30 seconds and results in a very poor user
experience.

Work around this hardware problem in the same way as for the Intel
40GbE and 100GbE NICs: by enabling dummy MSI-X interrupts to trick the
hardware into believing that it needs to write out completions to host
memory.

There is no documentation around the interrupt rearming mechanism.
The value written to the interrupt doorbell does not include a
consumer counter value, and so must be relying on some undocumented
ordering constraints.  Comments in the Linux driver source suggest
that the authors believe that the device will automatically and
atomically mask an MSI-X interrupt at the point of asserting it, that
any further interrupts arriving before the doorbell is written will be
recorded in the pending bit array, and that writing the doorbell will
therefore immediately assert a new interrupt if needed.

In the absence of any documentation, choose to rearm the interrupt
once per observed completion.  This is overkill, but is less impactful
than the alternative of rearming the interrupt unconditionally on
every poll.

Signed-off-by: Michael Brown <mcb30@ipxe.org>

src/drivers/net/gve.h

@@ -16,6 +16,7 @@ FILE_LICENCE ( GPL2_OR_LATER_OR_UBDL );
 #include <stdint.h>
 #include <ipxe/dma.h>
 #include <ipxe/pci.h>
+#include <ipxe/pcimsix.h>
 #include <ipxe/in.h>
 #include <ipxe/process.h>
 #include <ipxe/retry.h>
@@ -443,8 +444,11 @@ struct gve_irqs {
 	volatile uint32_t *db[GVE_IRQ_COUNT];
 };
 
-/** Disable interrupts */
-#define GVE_IRQ_DISABLE 0x40000000UL
+/** Disable in-order queue interrupt */
+#define GVE_GQI_IRQ_DISABLE 0x40000000UL
+
+/** Rearm out-of-order queue interrupt */
+#define GVE_DQO_IRQ_REARM 0x00000019UL
 
 /**
  * Queue resources
@@ -856,6 +860,8 @@ struct gve_nic {
 	struct net_device *netdev;
 	/** DMA device */
 	struct dma_device *dma;
+	/** Dummy MSI-X interrupt */
+	struct pci_msix msix;
 
 	/** Admin queue */
 	struct gve_admin admin;