anchor.diff:  
   = PVE: Virtualization for Work and Play (Part 3)
   :hp-alt-title: Server Virtualization Management Part3
  -:hp-tags: Blog, Open_Source, Technology
  +:hp-tags: Blog, Open_Source, Technology, ProxMox
   :icons: image
   :linkattrs:
   :published_at: 2017-05-03
  
positive.code: 
  === Configure the VFIO Modules
  
  Identify the GPU to passthrough:
  
  . Identify all GPUs: `lspci -nn | grep VGA`
  . Capture the GPU slot IDs (first pair of numbers separated by a colon):
  .. My GPU Slot ID for passthrough is: `28:00`
  .. My GPU Slot ID for the host is: `21:00`
  . Capture the vendor IDs for the passthrough GPU Slot ID: `lspci -nns 28:00`
  .. The vendor ID for my GPU VGA device: `10de:1b80`
  .. The vendor ID for my GPU Audio device: `10de:10f0`
  
  //lspci -nns 28:00 | cut -d "]" -f 2 | cut -d "[" -f 2
  
  To enable link:https://pve.proxmox.com/wiki/Pci_passthrough[PCI passthrough^] for our VM, add the following module options (including the comma separated vendor IDs identified in the prior step). This loads the vfio-pci kernel module, which maps memory regions from the PCI bus to the VM, and activates support for IOMMU groups.
  
  . Type: `nano /etc/modprobe.d/kvm.conf` and add the following:
  ```
  options vfio_iommu_type1 allow_unsafe_interrupts=1
  options vfio-pci         ids=10de:1b80,10de:10f0
  options vfio-pci         disable_vga=1
  options kvm-amd          npt=0
  
negative.code: 
  Afterward, run:
  ```
  update-grub          # update boot loader
  update-initramfs -u  # update boot image
  reboot               # reboot machine
  ```
  
  Afterward, run `lspci -nnks 28:00` to check that the driver loaded correctly. If everything went well, for each device we should see:
  
  ```
  Kernel driver in use: vfio-pci
  ```
  
  Also, run `dmesg | grep -e AMD-Vi -e vAPIC` to check our IOMMU settings.
  
  ```
  [    0.893699] AMD-Vi: IOMMU performance counters supported
  [    0.895145] AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40
  [    0.895146] AMD-Vi: Extended features (0xf77ef22294ada):
  [    0.895146]  PPR NX GT IA GA PC GA_vAPIC
  [    0.895148] AMD-Vi: Interrupt remapping enabled
  [    0.895149] AMD-Vi: virtual APIC enabled
  [    0.895257] AMD-Vi: Lazy IO/TLB flushing enabled
  ```
  
  //iommu=pt ... AMD-Vi driver will not register itself as the dma_ops backend and allows all devices unlimited access to main memory as long as no other kernel part (currently only KVM will do so) assigns the device to another domain using the IOMMU-API.
  
  [cols="1, 8a"]
  |===
  ^.^|image:/images/icons/lightbulb.png[icon="tip",size="4x",width=56]
  |*About That*: AMD Virtual Interrupt Controller (AVIC) virtualizes local link:https://en.wikipedia.org/wiki/Advanced_Programmable_Interrupt_Controller[APIC^] registers of each vCPU via the virtual APIC (vAPIC) backing page. This allows guest access to certain APIC registers without needing to emulate the hardware behavior, and should speed up workloads that generate large amount of interrupts.
  |===
  
  == Final Thoughts
  
  Congratulations! We have our PVE server configured and ready to use. We can now begin link:https://pve.proxmox.com/wiki/VM_Templates_and_Clones[creating Virtual Machines (VMs)^] or link:https://pve.proxmox.com/wiki/Linux_Container[Containers^]. In future posts, we'll consider additional opportunties for enhancing performance and security for our server, VMs, and Containers. 
  
  Although we have configured passthrough on the server, updates to our VMs are required to leverage that feature. Because Nvidia sells a commercial line of GPUs (Quadro), they do not _support_ and actively try to inhibit passthrough on their consumer line (GeForce). We will have to consider potential workarounds to enable that functionality, which may involve future tweaks to our server settings.
  
  
  