Skip to content
Snippets Groups Projects
Select Git revision
  • feature/libtcg-rebase default protected
1 result
Search by author
  • Any Author
  • Anton Anton anjo
1 author
  1. Oct 05, 2021
  3. Oct 04, 2021
  5. Oct 03, 2021
    • Richard Henderson's avatar
      Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging · 30bd1db5
      Richard Henderson authored 
      
* -smp cleanpus (Yanan)
* Hyper-V enlightenment functionality (Vitaly)
* virtio-mem support in dump, tpm and QMP (David)
* NetBSD GCC 7.4 compiler support (Nia)

# gpg: Signature made Sun 03 Oct 2021 03:41:30 AM EDT
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]

* remotes/bonzini/tags/for-upstream:
  softmmu/memory_mapping: optimize for RamDiscardManager sections
  softmmu/memory_mapping: factor out adding physical memory ranges
  softmmu/memory_mapping: never merge ranges accross memory regions
  tpm: mark correct memory region range dirty when clearing RAM
  monitor: Rate-limit MEMORY_DEVICE_SIZE_CHANGE qapi events per device
  qapi: Include qom-path in MEMORY_DEVICE_SIZE_CHANGE qapi events
  virtio-mem-pci: Fix memory leak when creating MEMORY_DEVICE_SIZE_CHANGE event
  configure: Loosen GCC requirement from 7.5.0 to 7.4.0

Signed-off-by: default avatarRichard Henderson <richard.henderson@linaro.org>
      30bd1db5
  7. Oct 02, 2021
    • Richard Henderson's avatar
      Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2021-10-02' into staging · f50ecf54
      Richard Henderson authored 
      
QAPI patches patches for 2021-10-02

# gpg: Signature made Sat 02 Oct 2021 01:37:11 AM EDT
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]

* remotes/armbru/tags/pull-qapi-2021-10-02:
  qapi/parser: enable pylint checks
  qapi/parser: Silence too-few-public-methods warning
  qapi/parser: enable mypy checks
  qapi/parser: Add FIXME for consolidating JSON-related types
  qapi/parser: add type hint annotations (QAPIDoc)
  qapi/parser: add import cycle workaround
  qapi/parser: Introduce NullSection
  qapi/parser: clarify _end_section() logic
  qapi/parser: remove FIXME comment from _append_body_line
  qapi: Add spaces after symbol declaration for consistency
  qapi/parser: fix unused check_args_section arguments
  qapi/gen: use dict.items() to iterate over _modules
  qapi/pylintrc: ignore 'consider-using-f-string' warning

Signed-off-by: default avatarRichard Henderson <richard.henderson@linaro.org>
      f50ecf54
    • David Hildenbrand's avatar
      softmmu/memory_mapping: optimize for RamDiscardManager sections · cb83ba8c
      David Hildenbrand authored 
      
virtio-mem logically plugs/unplugs memory within a sparse memory region
and notifies via the RamDiscardManager interface when parts become
plugged (populated) or unplugged (discarded).

Currently, we end up (via the two users)
1) zeroing all logically unplugged/discarded memory during TPM resets.
2) reading all logically unplugged/discarded memory when dumping, to
   figure out the content is zero.

1) is always bad, because we assume unplugged memory stays discarded
   (and is already implicitly zero).
2) isn't that bad with anonymous memory, we end up reading the zero
   page (slow and unnecessary, though). However, once we use some
   file-backed memory (future use case), even reading will populate memory.

Let's cut out all parts marked as not-populated (discarded) via the
RamDiscardManager. As virtio-mem is the single user, this now means that
logically unplugged memory ranges will no longer be included in the
dump, which results in smaller dump files and faster dumping.

virtio-mem has a minimum granularity of 1 MiB (and the default is usually
2 MiB). Theoretically, we can see quite some fragmentation, in practice
we won't have it completely fragmented in 1 MiB pieces. Still, we might
end up with many physical ranges.

Both, the ELF format and kdump seem to be ready to support many
individual ranges (e.g., for ELF it seems to be UINT32_MAX, kdump has a
linear bitmap).

Reviewed-by: default avatarPeter Xu <peterx@redhat.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Alex Williamson <alex.williamson@redhat.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Claudio Fontana <cfontana@suse.de>
Cc: Thomas Huth <thuth@redhat.com>
Cc: "Alex Bennée" <alex.bennee@linaro.org>
Cc: Peter Xu <peterx@redhat.com>
Cc: Laurent Vivier <lvivier@redhat.com>
Cc: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: default avatarDavid Hildenbrand <david@redhat.com>
Message-Id: <20210727082545.17934-5-david@redhat.com>
Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      cb83ba8c
    • David Hildenbrand's avatar
      softmmu/memory_mapping: factor out adding physical memory ranges · 3513bb1b
      David Hildenbrand authored 
      
Let's factor out adding a MemoryRegionSection to the list, to be reused in
RamDiscardManager context next.

Reviewed-by: default avatarStefan Berger <stefanb@linux.ibm.com>
Reviewed-by: default avatarPeter Xu <peterx@redhat.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Alex Williamson <alex.williamson@redhat.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Claudio Fontana <cfontana@suse.de>
Cc: Thomas Huth <thuth@redhat.com>
Cc: "Alex Bennée" <alex.bennee@linaro.org>
Cc: Peter Xu <peterx@redhat.com>
Cc: Laurent Vivier <lvivier@redhat.com>
Cc: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: default avatarDavid Hildenbrand <david@redhat.com>
Message-Id: <20210727082545.17934-4-david@redhat.com>
Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      3513bb1b
    • David Hildenbrand's avatar
      softmmu/memory_mapping: never merge ranges accross memory regions · 602f8ea7
      David Hildenbrand authored 
      
Let's make sure to not merge when different memory regions are involved.
Unlikely, but theoretically possible.

Acked-by: default avatarStefan Berger <stefanb@linux.ibm.com>
Reviewed-by: default avatarPeter Xu <peterx@redhat.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Alex Williamson <alex.williamson@redhat.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Claudio Fontana <cfontana@suse.de>
Cc: Thomas Huth <thuth@redhat.com>
Cc: "Alex Bennée" <alex.bennee@linaro.org>
Cc: Peter Xu <peterx@redhat.com>
Cc: Laurent Vivier <lvivier@redhat.com>
Cc: Stefan Berger <stefanb@linux.ibm.com>
Signed-off-by: default avatarDavid Hildenbrand <david@redhat.com>
Message-Id: <20210727082545.17934-3-david@redhat.com>
Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      602f8ea7
    • David Hildenbrand's avatar
      tpm: mark correct memory region range dirty when clearing RAM · 45e576c7
      David Hildenbrand authored 
      
We might not start at the beginning of the memory region. Let's
calculate the offset into the memory region via the difference in the
host addresses.

Acked-by: default avatarStefan Berger <stefanb@linux.ibm.com>
Fixes: ffab1be7 ("tpm: clear RAM when "memory overwrite" requested")
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Alex Williamson <alex.williamson@redhat.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Claudio Fontana <cfontana@suse.de>
Cc: Thomas Huth <thuth@redhat.com>
Cc: "Alex Bennée" <alex.bennee@linaro.org>
Cc: Peter Xu <peterx@redhat.com>
Cc: Laurent Vivier <lvivier@redhat.com>
Cc: Stefan Berger <stefanb@linux.vnet.ibm.com>
Signed-off-by: default avatarDavid Hildenbrand <david@redhat.com>
Reviewed-by: default avatarPeter Xu <peterx@redhat.com>
Message-Id: <20210727082545.17934-2-david@redhat.com>
Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      45e576c7
    • David Hildenbrand's avatar
      monitor: Rate-limit MEMORY_DEVICE_SIZE_CHANGE qapi events per device · 77ae2302
      David Hildenbrand authored 
      
We want to rate-limit MEMORY_DEVICE_SIZE_CHANGE events per device,
otherwise we can lose some events for devices. We can now use the
qom-path to reliably map an event to a device and make rate-limiting
device-aware.

This was noticed by starting a VM with two virtio-mem devices that each
have a requested size > 0. The Linux guest will initialize both devices
in parallel, resulting in losing MEMORY_DEVICE_SIZE_CHANGE events for
one of the devices.

Fixes: 722a3c78 ("virtio-pci: Send qapi events when the virtio-mem size changes")
Suggested-by: default avatarMarkus Armbruster <armbru@redhat.com>
Reviewed-by: default avatarMarkus Armbruster <armbru@redhat.com>
Signed-off-by: default avatarDavid Hildenbrand <david@redhat.com>
Message-Id: <20210929162445.64060-4-david@redhat.com>
Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      77ae2302
    • David Hildenbrand's avatar
      qapi: Include qom-path in MEMORY_DEVICE_SIZE_CHANGE qapi events · d89dd28f
      David Hildenbrand authored 
      
As we might not always have a device id, it is impossible to always
match MEMORY_DEVICE_SIZE_CHANGE events to an actual device. Let's
include the qom-path in the event, which allows for reliable mapping of
events to devices.

Fixes: 722a3c78 ("virtio-pci: Send qapi events when the virtio-mem size changes")
Suggested-by: default avatarMarkus Armbruster <armbru@redhat.com>
Reviewed-by: default avatarMarkus Armbruster <armbru@redhat.com>
Signed-off-by: default avatarDavid Hildenbrand <david@redhat.com>
Message-Id: <20210929162445.64060-3-david@redhat.com>
Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      d89dd28f
    • David Hildenbrand's avatar
      virtio-mem-pci: Fix memory leak when creating MEMORY_DEVICE_SIZE_CHANGE event · 75b98cb9
      David Hildenbrand authored 
      
Apparently, we don't have to duplicate the string.

Fixes: 722a3c78 ("virtio-pci: Send qapi events when the virtio-mem size changes")
Cc: qemu-stable@nongnu.org
Signed-off-by: default avatarDavid Hildenbrand <david@redhat.com>
Reviewed-by: default avatarMarkus Armbruster <armbru@redhat.com>
Message-Id: <20210929162445.64060-2-david@redhat.com>
Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      75b98cb9
    • nia's avatar
      configure: Loosen GCC requirement from 7.5.0 to 7.4.0 · 3830df5f
      nia authored 
      
As discussed in issue 614, we're shipping GCC 7.4.0 as the
system compiler in NetBSD 9, the most recent stable branch,
and are still actively interested in QEMU on this platform.

The differences between GCC 7.5.0 and 7.4.0 are trivial.

Signed-off-by: default avatarNia Alarie <nia@NetBSD.org>
Reviewed-by: default avatarRichard Henderson <richard.henderson@linaro.org>
Message-Id: <YVcpe79I0rly1HJh@homeworld.netbsd.org>
Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
      3830df5f
    • John Snow's avatar
      qapi/parser: enable pylint checks · d183e048
      John Snow authored 
      
Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>

Message-Id: <20210930205716.1148693-14-jsnow@redhat.com>
Reviewed-by: default avatarMarkus Armbruster <armbru@redhat.com>
Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
      d183e048
    • John Snow's avatar
      qapi/parser: Silence too-few-public-methods warning · 18e3673e
      John Snow authored 
      
Eh. Not worth the fuss today. There are bigger fish to fry.

Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-13-jsnow@redhat.com>
Reviewed-by: default avatarMarkus Armbruster <armbru@redhat.com>
Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
      18e3673e
    • John Snow's avatar
      qapi/parser: enable mypy checks · 2e28283e
      John Snow authored 
      
Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>

Message-Id: <20210930205716.1148693-12-jsnow@redhat.com>
Reviewed-by: default avatarMarkus Armbruster <armbru@redhat.com>
Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
      2e28283e
    • John Snow's avatar
      qapi/parser: Add FIXME for consolidating JSON-related types · 15acf48c
      John Snow authored 
      
The fix for this comment is forthcoming in a future commit, but this
will keep me honest. The linting configuration in ./python/setup.cfg
prohibits 'FIXME' comments. A goal of this long-running series is to
move ./scripts/qapi to ./python/qemu/qapi so that the QAPI generator is
regularly type-checked by GitLab CI.

This comment is a time-bomb to force me to address this issue prior to
that step.

Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-11-jsnow@redhat.com>
Reviewed-by: default avatarMarkus Armbruster <armbru@redhat.com>
Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
      15acf48c
    • John Snow's avatar
      qapi/parser: add type hint annotations (QAPIDoc) · 5f0d9f3b
      John Snow authored 
      
Annotations do not change runtime behavior.
This commit consists of only annotations.

Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-10-jsnow@redhat.com>
Reviewed-by: default avatarMarkus Armbruster <armbru@redhat.com>
Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
      5f0d9f3b
    • John Snow's avatar
      qapi/parser: add import cycle workaround · e7ac60fc
      John Snow authored 
      Adding static types causes a cycle in the QAPI generator:
[schema -> expr -> parser -> schema]. It exists because the QAPIDoc
class needs the names of types defined by the schema module, but the
schema module needs to import both expr.py/parser.py to do its actual
parsing.

Ultimately, the layering violation is that parser.py should not have any
knowledge of specifics of the Schema. QAPIDoc performs double-duty here
both as a parser *and* as a finalized object that is part of the schema.

In this patch, add the offending type hints alongside the workaround to
avoid the cycle becoming a problem at runtime. See
https://mypy.readthedocs.io/en/latest/runtime_troubles.html#import-cycles


for more information on this workaround technique.

I see three ultimate resolutions here:

(1) Just keep this patch and use the TYPE_CHECKING trick to eliminate
    the cycle which is only present during static analysis.

(2) Don't bother to annotate connect_member() et al, give them 'object'
    or 'Any'. I don't particularly like this, because it diminishes the
    usefulness of type hints for documentation purposes. Still, it's an
    extremely quick fix.

(3) Reimplement doc <--> definition correlation directly in schema.py,
    integrating doc fields directly into QAPISchemaMember and relieving
    the QAPIDoc class of the responsibility. Users of the information
    would instead visit the members first and retrieve their
    documentation instead of the inverse operation -- visiting the
    documentation and retrieving their members.

My preference is (3), but in the short-term (1) is the easiest way to
have my cake (strong type hints) and eat it too (Not have import
cycles). Do (1) for now, but plan for (3).

Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-9-jsnow@redhat.com>
Reviewed-by: default avatarMarkus Armbruster <armbru@redhat.com>
Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
      e7ac60fc
    • John Snow's avatar
      qapi/parser: Introduce NullSection · f4c05aaf
      John Snow authored 
      
Here's the weird bit. QAPIDoc generally expects -- virtually everywhere
-- that it will always have a current section. The sole exception to
this is in the case that end_comment() is called, which leaves us with
*no* section. However, in this case, we also don't expect to actually
ever mutate the comment contents ever again.

NullSection is just a Null-object that allows us to maintain the
invariant that we *always* have a current section, enforced by static
typing -- allowing us to type that field as QAPIDoc.Section instead of
the more ambiguous Optional[QAPIDoc.Section].

end_section is renamed to switch_section and now accepts as an argument
the new section to activate, clarifying that no callers ever just
unilaterally end a section; they only do so when starting a new section.

Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>

Message-Id: <20210930205716.1148693-8-jsnow@redhat.com>
Reviewed-by: default avatarMarkus Armbruster <armbru@redhat.com>
Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
      f4c05aaf
    • John Snow's avatar
      qapi/parser: clarify _end_section() logic · 1e20a775
      John Snow authored 
      
The "if self._section" clause in end_section is mysterious: In which
circumstances might we end a section when we don't have one?

QAPIDoc always expects there to be a "current section", only except
after a call to end_comment(). This actually *shouldn't* ever be 'None',
so let's remove that logic so I don't wonder why it's like this again in
three months.

Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-7-jsnow@redhat.com>
Reviewed-by: default avatarMarkus Armbruster <armbru@redhat.com>
Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
      1e20a775