At the moment, we look for just "python3" and "python", which is good
enough almost all of the time. But ... if you are on a platform that
uses an older Python by default and only offers a newer Python as an
option, you'll have to specify --python=/usr/bin/foo every time.

We can be kind and instead make a cursory attempt to locate a suitable
Python binary ourselves, looking for the remaining well-known binaries.

This configure loop will prefer, in order:

1. Whatever is specified in $PYTHON
2. python3
3. python
4. python3.11 down through python3.6

Notes:

- Python virtual environment provides binaries for "python3", "python",
  and whichever version you used to create the venv,
  e.g. "python3.8". If configure is invoked from inside of a venv, this
  configure loop will not "break out" of that venv unless that venv is
  created using an explicitly non-suitable version of Python that we
  cannot use.

- In the event that no suitable python is found, the first python found
  is the version used to generate the human-readable error message.

- The error message isn't printed right away to allow later
  configuration code to pick up an explicitly configured python.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

If neither --python nor --meson are specified, Meson's generated
build.ninja will invoke Python script using the interpreter *that Meson
itself is running under*; not the one identified by configure.

This is only an issue if Meson's Python interpreter is not "the first
one in the path", which is the one that is used if --python is not
specified.  A common case where this happen is when the "python3" binary
comes from a virtual environment but Meson is not installed (with pip)
in the virtual environment.  In this case (presumably) whoever set up
the venv wanted to use the venv's Python interpreter to build QEMU,
while Meson might use a different one, for example an enterprise
distro's older runtime.

So, detect whether a virtual environment is setup, and if the virtual
environment does not have Meson, use the meson submodule.  Meson will
then run under the virtual environment's Python interpreter.

Reported-by: John Snow <jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Once upon a time, "sphinx-build" on certain RPM platforms invoked
specifically a Python 2.x version, while "sphinx-build-3" was a distro
shim for the Python 3.x version.

These days, none of our supported platforms utilize a 2.x version, and
those that still have 'sphinx-build-3' make it a symbolic link to
'sphinx-build'.  Not searching for 'sphinx-build-3' will prefer
pip/venv installed versions of sphinx if they're available.

This adds an extremely convenient ability to test document building
ability in QEMU across multiple versions of Sphinx for the purposes of
compatibility testing.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20230221012456.2607692-6-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Commit eed56e9a "configure, meson: move config-poison.h to meson"
lost a "| sort -u".  Restore it.  config-poison shrinks from ~4500 to
~700 lines when all targets are enabled.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230206102037.3621709-1-armbru@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

We did not correctly handle N >= operand size.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1374


Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230114233206.3118472-1-richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Block layer patches

- Lock the graph, part 2 (BlockDriver callbacks)
- virtio-scsi: fix SCSIDevice hot unplug with IOThread
- rbd: Add support for layered encryption

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmP3tUURHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9ZQkA/9HFBrcsfSyzU5sHXcpqrcVPsvFwwzhsXN
# V6zMvBXQVEMYo6oDBSyNrniOJSYjiFLm1c+bMAaAFbo8dvVqqlkecBuZgQkFjnCy
# vXyaYeWnBSG5A91Vs30qzLObBsrX7P1Gh+bvtRvBPThC1zd8lrxMbVzlsxnTfDFo
# DsPkgiXL0SZ6YLBN5s61GBCfjvF8i0/8TPAvvwhHEo15sBgcBSTFYSftzEe9TXmH
# NHAuHnRshrd9DNnf20tVPuHCanSTsIpbx5cLYBoy81vSbjqJG4agULZLltKP3fiM
# kadpqmhJwjq+KhioLmcIjevPnUuqOMEzubaxZUm9o8jjsFPa8Isv4sIaAxyUP6e6
# aze1Xh9vUXn/JEf2/hApUY+2rz5dREL/TqpFwyzZjdqJb8PVCuy1JA1m2zLkvRPd
# Bl9pS7kabhcZOHrITnJS7Lvyy4IWeiw78trtaer0nCbKbPdQB62eswSXKYh5g+Ke
# kVJbkRSNi6lnljK5egIR3VxxM5kbGZsY4aGuyZk3Lc5yeAuPOil9swHlSO+5LFxP
# lRZOyumHbfKU6J7JbGFErrqR2fZiqKUN/6i0HZAIcjpZq1QxXlmHBbmrkXao+j5Y
# 0WcHdduH65dHT8fnBMgDZCXUfV7iBufspkCmY1v50YNJRPNmDzb4Os/Jh9qLHHMQ
# M1ae+58T0Fo=
# =gOli
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 23 Feb 2023 18:49:41 GMT
# gpg:                using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg:                issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin

: (29 commits)
  block/rbd: Add support for layered encryption
  block/rbd: Add luks-any encryption opening option
  block/rbd: Remove redundant stack variable passphrase_len
  virtio-scsi: reset SCSI devices from main loop thread
  dma-helpers: prevent dma_blk_cb() vs dma_aio_cancel() race
  scsi: protect req->aiocb with AioContext lock
  block: Mark bdrv_co_refresh_total_sectors() and callers GRAPH_RDLOCK
  block: Mark bdrv_*_dirty_bitmap() and callers GRAPH_RDLOCK
  block: Mark bdrv_co_delete_file() and callers GRAPH_RDLOCK
  block: Mark bdrv_(un)register_buf() GRAPH_RDLOCK
  block: Mark bdrv_co_eject/lock_medium() and callers GRAPH_RDLOCK
  block: Mark bdrv_co_is_inserted() and callers GRAPH_RDLOCK
  block: Mark bdrv_co_io_(un)plug() and callers GRAPH_RDLOCK
  block: Mark bdrv_co_create() and callers GRAPH_RDLOCK
  block: Mark preadv_snapshot/snapshot_block_status GRAPH_RDLOCK
  block: Mark bdrv_co_copy_range() GRAPH_RDLOCK
  block: Mark bdrv_co_do_pwrite_zeroes() GRAPH_RDLOCK
  block: Mark bdrv_co_pwrite_sync() and callers GRAPH_RDLOCK
  block: Mark public read/write functions GRAPH_RDLOCK
  block: Mark read/write in block/io.c GRAPH_RDLOCK
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Error reporting patches patches for 2023-02-23

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmP3ZogSHGFybWJydUBy
# ZWRoYXQuY29tAAoJEDhwtADrkYZT+PsP/ibioHJkJiR8yMt2/2iSwpkMrphZDmRQ
# 5sAgxCARdcp0m7maH4McCFkgtERcROip+j98FV29qI4y2P/mLkt1jyMYC+TH9r4O
# X3G997526gzZBLIJJsnYlVlJ1Gbgn+uCy4AzRLuhaKAHsYoxkP0jygoSs/eIZ9tK
# Wg2tkQ/wY4bXihrlzdOpWqU3Y0ADo2PQ29p7HWheRMDQz6JQxq82hFFs1jgGQ1aq
# 4HmcpIMX0+/LshFbDU91dL1pxW17vWT9J3xtzAsWlfBBgAh257LKvJqVD0XojL04
# FxJZ05IqTXZ04gvwgji0dcvNjdmP/dXVoGLfxAYwCFtKxiig700bdNb0+6MjCT6u
# P2tSPyQQzNQ5LYI7AgER4kMyXK22RkBXx+Q7y7QK1YXszWWSmGFZWGLA2FSg4lO6
# 5jsCgtEGixsMym/ox3XeoywSh4BgWkNXC+gKMSg/hQXgfriQmndHUOlK0ZU95I43
# 7gnPol+pU1HIEy/GDU8oMyieG513Ti1KVPZyv/FbuW75AYUDlHAXH/5OFlsuaLIR
# 1QF449xCLR5vIOOLXHbKJ9jbkcAaidhq5pOhLr7oV3yKh4H53iNB7gy8+vJ6XtBf
# tXXcYPVD8LpZxDegKNpIaeT0Nr4pyy6bYfrF+YeisVotD6PDtPALfJ9eSCWjaQsl
# DG2opOfv5xuV
# =VRxu
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 23 Feb 2023 13:13:44 GMT
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* tag 'pull-error-2023-02-23' of https://repo.or.cz/qemu/armbru

:
  rocker: Tweak stubbed out monitor commands' error messages
  migration/colo: Improve an x-colo-lost-heartbeat error message
  hw/core: Improve the query-hotpluggable-cpus error message
  replay: Simplify setting replay blockers
  qga: Drop dangling reference to QERR_QGA_LOGGING_DISABLED
  hw/acpi: Move QMP command to hw/core/
  hw/acpi: Dumb down acpi_table_add() stub
  hw/smbios: Dumb down smbios_entry_add() stub
  hw/core: Improve error message when machine doesn't provide NMIs
  dump: Assert cpu_get_note_size() can't fail
  dump: Improve error message when target doesn't support memory dump
  error: Drop superfluous #include "qapi/qmp/qerror.h"

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

QAPI patches patches for 2023-02-23

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmP3VikSHGFybWJydUBy
# ZWRoYXQuY29tAAoJEDhwtADrkYZTf98P/1tf2tPnWL0QpGXqGOq/iy2cFhcoco06
# 30t4JzTZGMZv8aUBRzlnhNgp+C7uMnXxuO7DeVN/K8VCvRfGXYz1HYFJ0NWhhMz6
# RULvVncJ7m9vFykmu3iibxvjyH6uj0R5xJ8ZNIrySLTCu+58voDF/IbZ0ep3v5nX
# 1AV1ljL9taxg2SrQ53Whbet9zfgXVFnV5wLKkLOqLGvviO2OBPG7rrtQaEX2jrsa
# SdTiOdBk1IMvG3FT6cVx3bM3kQd15UwfcJsdIYpB7QBZNoqgiyfMPsNr8HzpZlJn
# KOe3qWVFWHGMWY4MtQ1j9Ph44RPrJybvPQRMDNB3CiDYEtBWsth0fZxhw9T/tKca
# 5KgJaxecB3UsXFUBWhmvhkw+hwG+cDWHtiYZSb9AX4cqvPid1UdLnSQFWgHFGX+2
# ok0Q7gy9jYEpteVbIM8kQG0TF7xnZlv99uDK8b4MAH33roXwy70vffxpRGnngNyH
# IcLvzmDqRlrlzdvUi8Uro22VmUAUqSQKxKYt9yBJcEUV9NLi8E6g+Hcrvt7YNF9V
# jcVub4aIawEZCvnPCpOgzHD9p26ofwb2WQ245/5kzMUVi2pBYsHH6hJj7WdMPixS
# r24Ykgo4sxujW4pVy45lXzpA8uKWELCp9iKUOO6hvdoJEybVDMj9zcVn70cJgDrE
# RUle5av0n8XR
# =XV0D
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 23 Feb 2023 12:03:53 GMT
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* tag 'pull-qapi-2023-02-23' of https://repo.or.cz/qemu/armbru

:
  qapi: remove JSON value FIXME
  qapi: remove _JSONObject
  qapi/parser: add QAPIExpression type
  qapi: Add minor typing workaround for 3.6
  qapi: update pylint configuration
  qapi: Update flake8 config
  docs/devel/qapi-code-gen: Fix a missing 'may', clarify SchemaInfo
  docs/devel/qapi-code-gen: Belatedly update features documentation

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Python

Only minor testing updates.

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEE+ber27ys35W+dsvQfe+BBqr8OQ4FAmP27UcACgkQfe+BBqr8
# OQ5xSBAAlh+PSySlKnqxJm9dVi+K+8o6KOf9Oq0TAO+1lSEdAN0yMTW81TNAk50S
# HWTk2CclPVCrNxmvR1zr7ibp1BJvWLtfqcBSh+e3XQ1pPpciS2L+ny2OtuNYH5G5
# qSfLxoOXqV57gHdwkWmtC1b3AsnpcgdH84r4gUIaVPWx4fvm/JBVa0R40OjWaEZ6
# gTteIqoXN/tusBk6+ssELcNAA6jlHcVbhzS31Xi1/GDAWiu4wehqQ30zbFwvpyHn
# QN0NKeh1L6cGtjfN2PHv6tji5Z479yKYQU861BCn8SEJ052f4qLb/GBT01Fx3h+7
# 6bonnNXQrnyBNXWotYadTZMreUdDokuPF7FV4dNqd9E+552aF7WhodueO0lyyaTv
# bPHFavgyfNhfPblYqLpAWiPt+BlkZNazeWTAyRaQCqA1zHOr44K0ff1vVBGGvA2/
# xd0zGJ8xGiagz4ifIpyb3Fk9fampZkMAlJjKDfhhQzDdm/mrtdOt2uZBT2IhYX7z
# E+2+WfRE98kgAy17pzVB5GPRm+yFzWiu7H7zpGu4nQzswLWrKPrdwq8XYOZ16fL8
# NAKbn6h6CS0sOYiArr3tzQSnzBlaKCmOalsNjNCeFbuH4vTmKGamohpAW/OoBxhN
# 1X3aCdXqW0ewBrLWVHfluM0mhbq6i9ycYGi24pTikFPBqJCQP2o=
# =FJCZ
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 23 Feb 2023 04:36:23 GMT
# gpg:                using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]
# Primary key fingerprint: FAEB 9711 A12C F475 812F  18F2 88A9 064D 1835 61EB
#      Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76  CBD0 7DEF 8106 AAFC 390E

* tag 'python-pull-request' of https://gitlab.com/jsnow/qemu

:
  python: drop pipenv
  python: support pylint 2.16

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

* x86 bugfixes
* OpenBSD support for naming threads
* Refined Python support policy

# -----BEGIN PGP SIGNATURE-----
#
# iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmP0wtkUHHBib256aW5p
# QHJlZGhhdC5jb20ACgkQv/vSX3jHroNI6QgAjMvEV0N5FZYMpiuQdjebBeV+uHM6
# LThewCQa0cW5jb1X1NFBbOxYlNfE3WQeZqQF+BiVJr5wT2UsyNsPH7wTjsP387vV
# juoD7D/XZo8P4Qi+vJWo8XVBrzWEK8QS1P+NxWr+ZnsAhDx2+MR87fVmHtVBW1pI
# oDO0iyRrvVtaTAIVyNWSgZ59SLMmcH/6L4aYv5nrKYuAWx7fTneGGheKuqk55RsV
# sMv+fHolmmwKVm8tMFksw0atPwL7ZmSm1uObNHCQKdDNSoakC7YpaXa3y8LEzU7I
# B4h/PsmRpN33ggvsiuzFp9kfEHMy4QazfpoVFFqTLalhTr+XuiNTxj8xdA==
# =6eNN
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 21 Feb 2023 13:10:49 GMT
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* tag 'for-upstream' of https://gitlab.com/bonzini/qemu

:
  target/i386/gdbstub: Fix a bug about order of FPU stack in 'g' packets.
  docs: build-platforms: refine requirements on Python build dependencies
  thread-posix: add support for setting threads name on OpenBSD
  target/i386: Fix 32-bit AD[CO]X insns in 64-bit mode

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

tcg: Allow first half of insn in ram, and second half in mmio
linux-user/sparc: SIGILL for unknown trap vectors
linux-user/microblaze: SIGILL for privileged insns
linux-user: Fix deadlock while exiting due to signal
target/microblaze: Add gdbstub xml
util: Adjust cacheflush for windows-arm64
include/sysemu/os-win32: Adjust setjmp/longjmp for windows-arm64

# -----BEGIN PGP SIGNATURE-----
#
# iQFRBAABCgA7FiEEekgeeIaLTbaoWgXAZN846K9+IV8FAmP1dpkdHHJpY2hhcmQu
# aGVuZGVyc29uQGxpbmFyby5vcmcACgkQZN846K9+IV+70gf+OOM3KmsFpsJ4+68W
# v/ulVwye3RFQXv4KRtuRPeKCKMk7vXmBRj9gsyOpc23TaoYiMNbFbztpAkcc/Z/1
# +6H8QeZGLWDqiX6ashwGNm/2bqPbvY7znaCvNuLkNGCPBeJ12C19uN1BBiGdeqOe
# IXIIk1r0U6rfIDhP2PJALXOxgHd/8/onYbhU6kU5tQjM24pycW44UUGPSeV++I0e
# xWezAYOmZ4PK58bXHDPMZ0UkzuefaNmiLlfwj/4nlaWQetwQTy7BeEU6FpKolUN2
# wrvfCqth/c3SdUaZHu4DoX1yWt72L37SpO0ijvk8E+AqsvXTn9gFdWK2dsEiPEeS
# Z9abFw==
# =dxZo
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 22 Feb 2023 01:57:45 GMT
# gpg:                using RSA key 7A481E78868B4DB6A85A05C064DF38E8AF7E215F
# gpg:                issuer "richard.henderson@linaro.org"
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [full]
# Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A  05C0 64DF 38E8 AF7E 215F

* tag 'pull-tcg-20230221' of https://gitlab.com/rth7680/qemu

:
  sysemu/os-win32: fix setjmp/longjmp on windows-arm64
  util/cacheflush: fix cache on windows-arm64
  target/microblaze: Add gdbstub xml
  linux-user/microblaze: Handle privileged exception
  cpus: Make {start,end}_exclusive() recursive
  linux-user: Always exit from exclusive state in fork_end()
  linux-user/sparc: Raise SIGILL for all unhandled software traps
  accel/tcg: Allow the second page of an instruction to be MMIO

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Starting from ceph Reef, RBD has built-in support for layered encryption,
where each ancestor image (in a cloned image setting) can be possibly
encrypted using a unique passphrase.

A new function, rbd_encryption_load2, was added to librbd API.
This new function supports an array of passphrases (via "spec" structs).

This commit extends the qemu rbd driver API to use this new librbd API,
in order to support this new layered encryption feature.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
Message-Id: <20230129113120.722708-4-oro@oro.sl.cloud9.ibm.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

Ceph RBD encryption API required specifying the encryption format
for loading encryption. The supported formats were LUKS (v1) and LUKS2.

Starting from Reef release, RBD also supports loading with "luks-any" format,
which works for both versions of LUKS.

This commit extends the qemu rbd driver API to enable qemu users to use
this luks-any wildcard format.

Signed-off-by: Or Ozeri <oro@il.ibm.com>
Message-Id: <20230129113120.722708-3-oro@oro.sl.cloud9.ibm.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

Signed-off-by: Or Ozeri <oro@il.ibm.com>
Message-Id: <20230129113120.722708-2-oro@oro.sl.cloud9.ibm.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

When an IOThread is configured, the ctrl virtqueue is processed in the
IOThread. TMFs that reset SCSI devices are currently called directly
from the IOThread and trigger an assertion failure in blk_drain() from
the following call stack:

virtio_scsi_handle_ctrl_req -> virtio_scsi_do_tmf -> device_code_reset
-> scsi_disk_reset -> scsi_device_purge_requests -> blk_drain

  ../block/block-backend.c:1780: void blk_drain(BlockBackend *): Assertion `qemu_in_main_thread()' failed.

The blk_drain() function is not designed to be called from an IOThread
because it needs the Big QEMU Lock (BQL).

This patch defers TMFs that reset SCSI devices to a Bottom Half (BH)
that runs in the main loop thread under the BQL. This way it's safe to
call blk_drain() and the assertion failure is avoided.

Introduce s->tmf_bh_list for tracking TMF requests that have been
deferred to the BH. When the BH runs it will grab the entire list and
process all requests. Care must be taken to clear the list when the
virtio-scsi device is reset or unrealized. Otherwise deferred TMF
requests could execute later and lead to use-after-free or other
undefined behavior.

The s->resetting counter that's used by TMFs that reset SCSI devices is
accessed from multiple threads. This patch makes that explicit by using
atomic accessor functions. With this patch applied the counter is only
modified by the main loop thread under the BQL but can be read by any
thread.

Reported-by: Qing Wang <qinwang@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230221212218.1378734-4-stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

dma_blk_cb() only takes the AioContext lock around ->io_func(). That
means the rest of dma_blk_cb() is not protected. In particular, the
DMAAIOCB field accesses happen outside the lock.

There is a race when the main loop thread holds the AioContext lock and
invokes scsi_device_purge_requests() -> bdrv_aio_cancel() ->
dma_aio_cancel() while an IOThread executes dma_blk_cb(). The dbs->acb
field determines how cancellation proceeds. If dma_aio_cancel() sees
dbs->acb == NULL while dma_blk_cb() is still running, the request can be
completed twice (-ECANCELED and the actual return value).

The following assertion can occur with virtio-scsi when an IOThread is
used:

  ../hw/scsi/scsi-disk.c:368: scsi_dma_complete: Assertion `r->req.aiocb != NULL' failed.

Fix the race by holding the AioContext across dma_blk_cb(). Now
dma_aio_cancel() under the AioContext lock will not see
inconsistent/intermediate states.

Cc: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230221212218.1378734-3-stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

If requests are being processed in the IOThread when a SCSIDevice is
unplugged, scsi_device_purge_requests() -> scsi_req_cancel_async() races
with I/O completion callbacks. Both threads load and store req->aiocb.
This can lead to assert(r->req.aiocb == NULL) failures and undefined
behavior.

Protect r->req.aiocb with the AioContext lock to prevent the race.

Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230221212218.1378734-2-stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_co_refresh_total_sectors() need to hold a reader lock for the
graph.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-24-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_*_dirty_bitmap() need to hold a reader lock for the graph.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-23-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_co_delete_file() need to hold a reader lock for the graph.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-22-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_register_buf() and bdrv_unregister_buf() need to hold a reader lock
for the graph.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-21-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_co_eject() and bdrv_co_lock_medium() need to hold a reader lock for
the graph.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-20-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_co_is_inserted() need to hold a reader lock for the graph.

blk_is_inserted() is done as a co_wrapper_mixed_bdrv_rdlock (unlike most
other blk_* functions) because it is called a lot from other blk_co_*()
functions that already hold the lock. These calls go through
blk_is_available(), which becomes a co_wrapper_mixed_bdrv_rdlock, too,
for the same reason.

Functions that run in a coroutine and can call bdrv_co_is_available()
directly are changed to do so, which results in better TSA coverage.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-19-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_co_io_plug() and bdrv_co_io_unplug() need to hold a reader lock for
the graph.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-18-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_co_create() need to hold a reader lock for the graph.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-17-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-16-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_co_copy_range() need to hold a reader lock for the graph.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-15-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

All callers are already GRAPH_RDLOCK, so just add the annotation and
remove assume_graph_lock().

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-14-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_co_pwrite_sync() need to hold a reader lock for the graph.

For some places, we know that they will hold the lock, but we don't have
the GRAPH_RDLOCK annotations yet. In this case, add assume_graph_lock()
with a FIXME comment. These places will be removed once everything is
properly annotated.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-13-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_co_pread*/pwrite*() need to hold a reader lock for the graph.

For some places, we know that they will hold the lock, but we don't have
the GRAPH_RDLOCK annotations yet. In this case, add assume_graph_lock()
with a FIXME comment. These places will be removed once everything is
properly annotated.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-12-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_driver_*() need to hold a reader lock for the graph. It doesn't add
the annotation to public functions yet.

For some places, we know that they will hold the lock, but we don't have
the GRAPH_RDLOCK annotations yet. In this case, add assume_graph_lock()
with a FIXME comment. These places will be removed once everything is
properly annotated.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-11-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_co_pwrite_zeroes() need to hold a reader lock for the graph.

For some places, we know that they will hold the lock, but we don't have
the GRAPH_RDLOCK annotations yet. In this case, add assume_graph_lock()
with a FIXME comment. These places will be removed once everything is
properly annotated.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-10-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_co_pdiscard() need to hold a reader lock for the graph.

For some places, we know that they will hold the lock, but we don't have
the GRAPH_RDLOCK annotations yet. In this case, add assume_graph_lock()
with a FIXME comment. These places will be removed once everything is
properly annotated.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-9-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_co_flush() need to hold a reader lock for the graph.

For some places, we know that they will hold the lock, but we don't have
the GRAPH_RDLOCK annotations yet. In this case, add assume_graph_lock()
with a FIXME comment. These places will be removed once everything is
properly annotated.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-8-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This function is called in two different places:
- timer callback, which does not take the graph rdlock.
- bdrv_qed_drain_begin(), which is .bdrv_drain_begin()
  callback documented as function that does not take the lock.

Since it calls recursive functions that traverse the
graph, we need to protect them with the graph rdlock.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-7-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_co_ioctl() need to hold a reader lock for the graph.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-6-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_co_block_status() need to hold a reader lock for the graph.

For some places, we know that they will hold the lock, but we don't have
the GRAPH_RDLOCK annotations yet. In this case, add assume_graph_lock()
with a FIXME comment. These places will be removed once everything is
properly annotated.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-5-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_co_truncate() need to hold a reader lock for the graph.

For some places, we know that they will hold the lock, but we don't have
the GRAPH_RDLOCK annotations yet. In this case, add assume_graph_lock()
with a FIXME comment. These places will be removed once everything is
properly annotated.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-4-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

bdrv_mirror_top_pwritev() accesses the job object when active mirroring
is enabled. It disables this code during early initialisation while
s->job isn't set yet.

However, s->job is still set way too early when the job object isn't
fully initialised. For example, &s->ops_in_flight isn't initialised yet
and the in_flight bitmap doesn't exist yet. This causes crashes when a
write request comes in too early.

Move the assignment of s->job to when the mirror job is actually fully
initialised to make sure that the mirror_top driver doesn't access it
too early.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-3-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

It is never called outside of block.c.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-2-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>