linux/drivers
Daniel Jordan 4d83de6da2 vfio/type1: Batch page pinning
Pinning one 4K page at a time is inefficient, so do it in batches of 512
instead.  This is just an optimization with no functional change
intended, and in particular the driver still calls iommu_map() with the
largest physically contiguous range possible.

Add two fields in vfio_batch to remember where to start between calls to
vfio_pin_pages_remote(), and use vfio_batch_unpin() to handle remaining
pages in the batch in case of error.

qemu pins pages for guests around 8% faster on my test system, a
two-node Broadwell server with 128G memory per node.  The qemu process
was bound to one node with its allocations constrained there as well.

                             base               test
          guest              ----------------   ----------------
       mem (GB)   speedup    avg sec    (std)   avg sec    (std)
              1      7.4%       0.61   (0.00)      0.56   (0.00)
              2      8.3%       0.93   (0.00)      0.85   (0.00)
              4      8.4%       1.46   (0.00)      1.34   (0.00)
              8      8.6%       2.54   (0.01)      2.32   (0.00)
             16      8.3%       4.66   (0.00)      4.27   (0.01)
             32      8.3%       8.94   (0.01)      8.20   (0.01)
             64      8.2%      17.47   (0.01)     16.04   (0.03)
            120      8.5%      32.45   (0.13)     29.69   (0.01)

perf diff confirms less time spent in pup.  Here are the top ten
functions:

             Baseline  Delta Abs  Symbol

               78.63%     +6.64%  clear_page_erms
                1.50%     -1.50%  __gup_longterm_locked
                1.27%     -0.78%  __get_user_pages
                          +0.76%  kvm_zap_rmapp.constprop.0
                0.54%     -0.53%  vmacache_find
                0.55%     -0.51%  get_pfnblock_flags_mask
                0.48%     -0.48%  __get_user_pages_remote
                          +0.39%  slot_rmap_walk_next
                          +0.32%  vfio_pin_map_dma
                          +0.26%  kvm_handle_hva_range
                ...

Suggested-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Daniel Jordan <daniel.m.jordan@oracle.com>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2021-02-22 16:30:47 -07:00
..
accessibility tty: rename tty_kopen() and add new function tty_kopen_shared() 2020-12-28 16:09:11 +01:00
acpi ACPI updates for 5.12-rc1 2021-02-20 21:50:59 -08:00
amba
android Merge branch 'exec-for-v5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2020-12-15 19:29:43 -08:00
ata ata: Avoid comma separated statements 2021-01-31 08:05:46 -07:00
atm atm: idt77252: fix build broken on amd64 2021-02-15 12:36:27 -08:00
auxdisplay
base - added support for Nintendo N64 2021-02-21 13:18:26 -08:00
bcma
block xen: branch for v5.12-rc1 2021-02-21 13:06:08 -08:00
bluetooth TTY/Serial driver changes for 5.12-rc1 2021-02-20 21:28:04 -08:00
bus ARM: SoC drivers for v5.12 2021-02-20 18:42:28 -08:00
cdrom block: remove unnecessary argument from blk_execute_rq 2021-01-24 21:52:39 -07:00
char - added support for Nintendo N64 2021-02-21 13:18:26 -08:00
clk Power management updates for 5.12-rc1 2021-02-20 21:42:18 -08:00
clocksource hyperv-next for 5.12 2021-02-21 13:24:39 -08:00
connector net/connector: Add const qualifier to cb_id 2020-12-16 11:06:49 -08:00
counter counter:ti-eqep: remove floor 2021-01-14 20:56:56 +00:00
cpufreq Merge branch 'pm-opp' into pm 2021-02-15 17:01:46 +01:00
cpuidle ARM: SoC drivers for v5.11 2020-12-16 16:38:41 -08:00
crypto x86: 2021-02-21 13:31:43 -08:00
dax libnvdimm for 5.11 2020-12-24 12:18:11 -08:00
dca
devfreq Merge branches 'pm-devfreq' and 'pm-tools' 2021-02-15 17:02:04 +01:00
dio
dma dmaengine dw: Revert "dmaengine: dw: Enable runtime PM" 2021-02-08 17:36:12 +05:30
dma-buf dma-buf: cma_heap: Fix memory leak in CMA heap 2021-01-10 23:08:21 +05:30
edac Merge branch 'edac-misc' into edac-updates-for-v5.12 2021-02-15 10:06:58 +01:00
eisa
extcon extcon: max77693: Fix modalias string 2020-12-11 17:18:10 +09:00
firewire
firmware arm64 updates for 5.12 2021-02-21 13:08:42 -08:00
fpga Merge 5.10-rc7 into char-misc-next 2020-12-07 10:08:14 +01:00
fsi hwmon patches for v5.11 2020-12-15 16:06:14 -08:00
gnss
gpio gpio: ep93xx: Fix single irqchip with multi gpiochips 2021-02-10 14:47:27 +01:00
gpu x86: 2021-02-21 13:31:43 -08:00
greybus
hid Staging/IIO driver patches for 5.12-rc1 2021-02-20 21:36:51 -08:00
hsi HSI: core: fix a kernel-doc markup 2020-12-02 22:35:44 +01:00
hv Revert "Drivers: hv: vmbus: Copy packets sent by Hyper-V out of the ring buffer" 2021-02-15 10:49:11 +00:00
hwmon ACPI: Test for ACPI_SUCCESS rather than !ACPI_FAILURE 2021-01-27 18:43:07 +01:00
hwspinlock hwspinlock: sirf: Remove the redundant 'of_match_ptr' 2020-12-10 13:34:40 -06:00
hwtracing intel_th: pci: Add Alder Lake-P support 2021-01-21 18:54:43 +01:00
i2c SoC platform removal 2021-02-20 18:16:30 -08:00
i3c i3c/master/mipi-i3c-hci: Fix position of __maybe_unused in i3c_hci_of_match 2020-12-31 18:41:37 +01:00
ide block: remove unnecessary argument from blk_execute_rq 2021-01-24 21:52:39 -07:00
idle intel_idle: remove definition of DEBUG 2021-01-22 16:46:16 +01:00
iio Merge 5.11-rc5 into staging-next 2021-01-25 10:37:59 +01:00
infiniband Revert "RDMA/rxe: Remove VLAN code leftovers from RXE" 2021-01-20 13:29:28 -04:00
input TTY/Serial driver changes for 5.12-rc1 2021-02-20 21:28:04 -08:00
interconnect interconnect: imx8mq: Use icc_sync_state 2020-12-28 14:03:02 +02:00
iommu hyperv-next for 5.12 2021-02-21 13:24:39 -08:00
ipack
irqchip irqchip updates for Linux 5.12: 2021-02-15 15:41:56 +01:00
isdn misdn: dsp: select CONFIG_BITREVERSE 2021-01-05 15:50:36 -08:00
leds TTY/Serial driver changes for 5.12-rc1 2021-02-20 21:28:04 -08:00
lightnvm lightnvm: pblk: Replace guid_copy() with export_guid()/import_guid() 2021-02-14 21:27:24 -07:00
macintosh macintosh/adb-iop: Use big-endian autopoll mask 2021-01-25 13:23:38 +01:00
mailbox mailbox: arm_mhuv2: Add driver 2020-12-09 19:26:02 -06:00
mcb
md for-5.12/drivers-2021-02-17 2021-02-21 11:06:54 -08:00
media media: rockchip: rkisp1: extend uapi array sizes 2021-01-28 11:31:43 +01:00
memory Memory controller drivers for v5.12, part two 2021-02-11 13:48:49 +01:00
memstick memstick: r592: Fix error return in r592_probe() 2020-12-04 12:27:10 +01:00
message SCSI misc on 20201216 2020-12-16 13:34:31 -08:00
mfd mfd/bus: sunxi-rsb: Make .remove() callback return void 2021-01-15 13:23:36 +00:00
misc misc: rtsx: init value of aspm_enabled 2021-01-22 11:04:53 +01:00
mmc for-5.12/block-2021-02-17 2021-02-21 11:02:48 -08:00
most
mtd SoC platform removal 2021-02-20 18:16:30 -08:00
mux
net hyperv-next for 5.12 2021-02-21 13:24:39 -08:00
nfc nfc: st-nci: Remove unnecessary variable 2021-02-08 14:56:20 -08:00
ntb Big fix for IDT NTB and Intel NTB LTR management support 2020-12-27 09:22:55 -08:00
nubus
nvdimm for-5.12/block-2021-02-17 2021-02-21 11:02:48 -08:00
nvme for-5.12/drivers-2021-02-17 2021-02-21 11:06:54 -08:00
nvmem
of of/device: Update dma_range_map only when dev has valid dma-ranges 2021-01-27 14:00:14 -06:00
opp OPP: Add function to look up required OPP's for a given OPP 2021-02-04 16:30:14 +05:30
parisc
parport
pci hyperv-next for 5.12 2021-02-21 13:24:39 -08:00
pcmcia Merge branch 'pcmcia-next' of git://git.kernel.org/pub/scm/linux/kernel/git/brodo/linux 2020-12-19 12:50:10 -08:00
perf drivers/perf: Replace spin_lock_irqsave to spin_lock 2021-02-10 18:50:42 +00:00
phy phy: mediatek: allow compile-testing the dsi phy 2021-01-04 13:00:54 +05:30
pinctrl pinctrl: qcom: Don't clear pending interrupts when enabling 2021-01-18 16:07:08 +01:00
platform ACPI updates for 5.12-rc1 2021-02-20 21:50:59 -08:00
pnp PNP: pnpbios: Use DEFINE_SPINLOCK() for spinlock 2021-01-22 16:22:57 +01:00
power power: supply: Fix a typo in warning message 2020-12-13 01:00:10 +01:00
powercap powercap: intel_rapl: Use topology interface in rapl_init_domains() 2021-02-12 16:53:01 +01:00
pps
ps3 powerpc/ps3: use dma_mapping_error() 2020-12-15 22:50:12 +11:00
ptp ptp: ptp_clockmatrix: clean-up - parenthesis around a == b are unnecessary 2021-02-17 13:49:26 -08:00
pwm pwm: Changes for v5.11-rc1 2020-12-19 11:51:32 -08:00
rapidio rapidio: remove unused rio_get_asm() and rio_get_device() 2020-12-15 22:46:18 -08:00
ras
regulator regulator: Fix lockdep warning resolving supplies 2021-01-22 14:03:07 +00:00
remoteproc ARM: SoC drivers for v5.11 2020-12-16 16:38:41 -08:00
reset ARM: SoC drivers for v5.12 2021-02-20 18:42:28 -08:00
rpmsg
rtc rtc: mc146818: Dont test for bit 0-5 in Register D 2021-02-02 20:35:02 +01:00
s390 These are the v5.12 updates for the locking subsystem: 2021-02-21 12:12:01 -08:00
sbus
scsi for-5.12/block-2021-02-17 2021-02-21 11:02:48 -08:00
sfi
sh sh/intc: Restore devm_ioremap() alignment 2021-01-06 19:55:29 -05:00
siox siox: Make remove callback return void 2020-12-10 16:17:15 +01:00
slimbus slimbus: qcom: fix potential NULL dereference in qcom_slim_prg_slew() 2020-12-10 16:23:56 +01:00
soc ARM: SoC drivers for v5.12 2021-02-20 18:42:28 -08:00
soundwire soundwire: intel: fix another unused-function warning 2020-12-05 13:11:54 +05:30
spi ACPI updates for 5.12-rc1 2021-02-20 21:50:59 -08:00
spmi spmi: Add driver shutdown support 2020-12-10 10:45:36 +01:00
ssb
staging Staging/IIO driver patches for 5.12-rc1 2021-02-20 21:36:51 -08:00
target for-5.12/block-2021-02-17 2021-02-21 11:02:48 -08:00
tc
tee Simplify i2c acess in OP-TEE driver 2021-02-09 16:10:49 +01:00
thermal Scheduler updates for v5.12: 2021-02-21 12:35:04 -08:00
thunderbolt thunderbolt: Changes for v5.12 merge window 2021-02-09 13:00:07 +01:00
tty TTY/Serial driver changes for 5.12-rc1 2021-02-20 21:28:04 -08:00
uio uio: uio_hv_generic: use devm_kzalloc() for private data alloc 2020-12-09 19:59:00 +01:00
usb usb: misc: usb3503: Fix logic in usb3503_init() 2021-02-11 13:52:04 +01:00
vdpa vdpa/mlx5: Restore the hardware used index after change map 2021-02-05 10:28:04 -05:00
vfio vfio/type1: Batch page pinning 2021-02-22 16:30:47 -07:00
vhost vhost_net: avoid tx queue stuck when sendmsg fails 2021-01-19 11:13:30 -08:00
video vgacon: drop BROKEN_GRAPHICS_PROGRAMS 2021-01-07 16:17:31 +01:00
virt
virtio virtio,vdpa: features, cleanups, fixes 2020-12-24 12:06:46 -08:00
visorbus
vlynq
vme vme: switch from 'pci_' to 'dma_' API 2020-12-09 19:44:34 +01:00
w1
watchdog linux-watchdog 5.11-rc1 tag 2020-12-23 15:01:49 -08:00
xen xen: branch for v5.12-rc1 2021-02-21 13:06:08 -08:00
zorro
Kconfig
Makefile asm-generic: cross-architecture timer cleanup 2020-12-16 00:07:17 -08:00