What is NetTLP?
NetTLP is a development platform for software PCIe devices that can interact with hardware root complexes.
Developers can program their own PCIe devices with IP packet processing and debug the DMA communication with powerful network tools like tcpdump and wireshark. Software PCIe devices work in various PCIe topologies, such as CPU-to-Device, Device-to-Device, and Remote DMA. For example, DMA communication of NIC or NVMe devices can be captured by NetTLP's memory device (pseudo memory device) set in the DMA region. NetTLP provides powerful visibility into PCIe communication.
On the NetTLP platform, substances of PCIe devices are implemented as software on top of a library, called LibTLP. LibTLP, which is a software implementation of PCIe Transaction Layer, provides a well-abstracted DMA API for the software devices, and sends and receives PCIe Transaction Layer Packetes (TLPs) over UDP/IP/Ethernet. The NetTLP adapter installed on a distant host receives TLPs from the software device, and then sends the TLPs to the root complex through the physical PCIe interconnect. TLPs sent from the root complex to the NetTLP adapter are delivered to the software device through the Ethernet link by encapsulating TLPs in UDP/IP/Ethernet likewise.
So, you can implement your PCIe devices in software interacting with hardware root complexes.
And, you can observe their interactions by powerful IP networking tools like tcpdump and wireshark.
Research Paper
Yohei Kuga, Ryo Nakamura, Takeshi Matsuya, and Yuji Sekiya. "NetTLP: A Development Platform for PCIe devices in Software Interacting with Hardware", 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI '20), USENIX Association, 2020 [URL]