Skip to content
Snippets Groups Projects
Select Git revision
  • feature/libtcg-rebase default protected
1 result
Search by author
  • Any Author
  • Anton Anton anjo
1 author
  1. Sep 19, 2023
    • David Hildenbrand's avatar
      backends/hostmem-file: Add "rom" property to support VM templating with R/O files · e92666b0
      David Hildenbrand authored 
      
For now, "share=off,readonly=on" would always result in us opening the
file R/O and mmap'ing the opened file MAP_PRIVATE R/O -- effectively
turning it into ROM.

Especially for VM templating, "share=off" is a common use case. However,
that use case is impossible with files that lack write permissions,
because "share=off,readonly=on" will not give us writable RAM.

The sole user of ROM via memory-backend-file are R/O NVDIMMs, but as we
have users (Kata Containers) that rely on the existing behavior --
malicious VMs should not be able to consume COW memory for R/O NVDIMMs --
we cannot change the semantics of "share=off,readonly=on"

So let's add a new "rom" property with on/off/auto values. "auto" is
the default and what most people will use: for historical reasons, to not
change the old semantics, it defaults to the value of the "readonly"
property.

For VM templating, one can now use:
    -object memory-backend-file,share=off,readonly=on,rom=off,...

But we'll disallow:
    -object memory-backend-file,share=on,readonly=on,rom=off,...
because we would otherwise get an error when trying to mmap the R/O file
shared and writable. An explicit error message is cleaner.

We will also disallow for now:
    -object memory-backend-file,share=off,readonly=off,rom=on,...
    -object memory-backend-file,share=on,readonly=off,rom=on,...
It's not harmful, but also not really required for now.

Alternatives that were abandoned:
* Make "unarmed=on" for the NVDIMM set the memory region container
  readonly. We would still see a change of ROM->RAM and possibly run
  into memslot limits with vhost-user. Further, there might be use cases
  for "unarmed=on" that should still allow writing to that memory
  (temporary files, system RAM, ...).
* Add a new "readonly=on/off/auto" parameter for NVDIMMs. Similar issues
  as with "unarmed=on".
* Make "readonly" consume "on/off/file" instead of being a 'bool' type.
  This would slightly changes the behavior of the "readonly" parameter:
  values like true/false (as accepted by a 'bool'type) would no longer be
  accepted.

Message-ID: <20230906120503.359863-4-david@redhat.com>
Acked-by: default avatarMarkus Armbruster <armbru@redhat.com>
Signed-off-by: default avatarDavid Hildenbrand <david@redhat.com>
      e92666b0
  3. Sep 04, 2023
  5. Aug 02, 2023
  7. Jul 26, 2023
    • Markus Armbruster's avatar
      qapi: Reformat recent doc comments to conform to current conventions · 9e272073
      Markus Armbruster authored 
      
Since commit a937b6aa (qapi: Reformat doc comments to conform to
current conventions), a number of comments not conforming to the
current formatting conventions were added.  No problem, just sweep
the entire documentation once more.

To check the generated documentation does not change, I compared the
generated HTML before and after this commit with "wdiff -3".  Finds no
differences.  Comparing with diff is not useful, as the reflown
paragraphs are visible there.

Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
Message-ID: <20230720071610.1096458-7-armbru@redhat.com>
      9e272073
    • Markus Armbruster's avatar
      qapi/trace: Tidy up trace-event-get-state, -set-state documentation · e27a9d62
      Markus Armbruster authored 
      
trace-event-set-state's explanation of how events are selected is
under "Features".  Doesn't belong there.  Simply delete it, as it
feels redundant with documentation of member @name.

trace-event-get-state's explanation is under "Returns".  Tolerable,
but similarly redundant.  Delete it, too.

Cc: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
Message-ID: <20230720071610.1096458-5-armbru@redhat.com>
      e27a9d62
    • Markus Armbruster's avatar
      qapi/qdev: Tidy up device_add documentation · a9c72efd
      Markus Armbruster authored 
      
The notes section comes out like this:

    Notes

    Additional arguments depend on the type.

    1. For detailed information about this command, please refer to the
       ‘docs/qdev-device-use.txt’ file.

    2. It’s possible to list device properties by running QEMU with the
       “-device DEVICE,help” command-line argument, where DEVICE is the
       device’s name

The first item isn't numbered.  Fix that:

    1. Additional arguments depend on the type.

    2. For detailed information about this command, please refer to the
       ‘docs/qdev-device-use.txt’ file.

    3. It’s possible to list device properties by running QEMU with the
       “-device DEVICE,help” command-line argument, where DEVICE is the
       device’s name

Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
Message-ID: <20230720071610.1096458-4-armbru@redhat.com>
Reviewed-by: default avatarPhilippe Mathieu-Daudé <philmd@linaro.org>
      a9c72efd
    • Markus Armbruster's avatar
      qapi/block: Tidy up block-latency-histogram-set documentation · e893b9e3
      Markus Armbruster authored 
      
Examples come out like

    Example

       set new histograms for all io types with intervals [0, 10), [10,
       50), [50, 100), [100, +inf):

The sentence "set new histograms ..." starts with a lower case letter.
Capitalize it.  Same for the other examples.

Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
Message-ID: <20230720071610.1096458-3-armbru@redhat.com>
Reviewed-by: default avatarPhilippe Mathieu-Daudé <philmd@linaro.org>
      e893b9e3
    • Markus Armbruster's avatar
      qapi/block-core: Tidy up BlockLatencyHistogramInfo documentation · dad3c956
      Markus Armbruster authored 
      
Documentation for member @bin comes out like

    list of io request counts corresponding to histogram intervals.
    len("bins") = len("boundaries") + 1 For the example above, "bins"
    may be something like [3, 1, 5, 2], and corresponding histogram
    looks like:

Note how the equation and the sentence following it run together.
Replace the equation:

    list of io request counts corresponding to histogram intervals,
    one more element than "boundaries" has.  For the example above,
    "bins" may be something like [3, 1, 5, 2], and corresponding
    histogram looks like:

Cc: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
Message-ID: <20230720071610.1096458-2-armbru@redhat.com>
[Off by one fixed]
      dad3c956
    • Juan Quintela's avatar
      migration: skipped field is really obsolete. · 7b24d326
      Juan Quintela authored 
      
Has return zero for more than 10 years.

Specifically we introduced the field in 1.5.0

commit f1c72795
Author: Peter Lieven <pl@kamp.de>
Date:   Tue Mar 26 10:58:37 2013 +0100

    migration: do not sent zero pages in bulk stage

    during bulk stage of ram migration if a page is a
    zero page do not send it at all.
    the memory at the destination reads as zero anyway.

    even if there is an madvise with QEMU_MADV_DONTNEED
    at the target upon receipt of a zero page I have observed
    that the target starts swapping if the memory is overcommitted.
    it seems that the pages are dropped asynchronously.

    this patch also updates QMP to return the number of
    skipped pages in MigrationStats.

but removed its usage in 1.5.3

commit 9ef051e5
Author: Peter Lieven <pl@kamp.de>
Date:   Mon Jun 10 12:14:19 2013 +0200

    Revert "migration: do not sent zero pages in bulk stage"

    Not sending zero pages breaks migration if a page is zero
    at the source but not at the destination. This can e.g. happen
    if different BIOS versions are used at source and destination.
    It has also been reported that migration on pseries is completely
    broken with this patch.

    This effectively reverts commit f1c72795.

Reviewed-by: default avatarDaniel P. Berrangé <berrange@redhat.com>
Message-ID: <20230612193344.3796-2-quintela@redhat.com>
Signed-off-by: default avatarJuan Quintela <quintela@redhat.com>
      7b24d326
    • Hyman Huang(黄勇)'s avatar
      migration: Extend query-migrate to provide dirty page limit info · 15699cf5
      Hyman Huang(黄勇) authored 
      
Extend query-migrate to provide throttle time and estimated
ring full time with dirty-limit capability enabled, through which
we can observe if dirty limit take effect during live migration.

Signed-off-by: default avatarHyman Huang(黄勇) <yong.huang@smartx.com>
Reviewed-by: default avatarMarkus Armbruster <armbru@redhat.com>
Reviewed-by: default avatarJuan Quintela <quintela@redhat.com>
Message-ID: <168733225273.5845.15871826788879741674-8@git.sr.ht>
Signed-off-by: default avatarJuan Quintela <quintela@redhat.com>
      15699cf5
    • Hyman Huang(黄勇)'s avatar
      migration: Introduce dirty-limit capability · dc623955
      Hyman Huang(黄勇) authored 
      
Introduce migration dirty-limit capability, which can
be turned on before live migration and limit dirty
page rate durty live migration.

Introduce migrate_dirty_limit function to help check
if dirty-limit capability enabled during live migration.

Meanwhile, refactor vcpu_dirty_rate_stat_collect
so that period can be configured instead of hardcoded.

dirty-limit capability is kind of like auto-converge
but using dirty limit instead of traditional cpu-throttle
to throttle guest down. To enable this feature, turn on
the dirty-limit capability before live migration using
migrate-set-capabilities, and set the parameters
"x-vcpu-dirty-limit-period", "vcpu-dirty-limit" suitably
to speed up convergence.

Signed-off-by: default avatarHyman Huang(黄勇) <yong.huang@smartx.com>
Acked-by: default avatarPeter Xu <peterx@redhat.com>
Reviewed-by: default avatarJuan Quintela <quintela@redhat.com>
Message-Id: <168618975839.6361.17407633874747688653-4@git.sr.ht>
Signed-off-by: default avatarJuan Quintela <quintela@redhat.com>
      dc623955
    • Hyman Huang(黄勇)'s avatar
      qapi/migration: Introduce vcpu-dirty-limit parameters · 09f9ec99
      Hyman Huang(黄勇) authored 
      
Introduce "vcpu-dirty-limit" migration parameter used
to limit dirty page rate during live migration.

"vcpu-dirty-limit" and "x-vcpu-dirty-limit-period" are
two dirty-limit-related migration parameters, which can
be set before and during live migration by qmp
migrate-set-parameters.

This two parameters are used to help implement the dirty
page rate limit algo of migration.

Signed-off-by: default avatarHyman Huang(黄勇) <yong.huang@smartx.com>
Acked-by: default avatarPeter Xu <peterx@redhat.com>
Reviewed-by: default avatarJuan Quintela <quintela@redhat.com>
Message-Id: <168618975839.6361.17407633874747688653-3@git.sr.ht>
Signed-off-by: default avatarJuan Quintela <quintela@redhat.com>
      09f9ec99
    • Hyman Huang(黄勇)'s avatar
      qapi/migration: Introduce x-vcpu-dirty-limit-period parameter · 4d807857
      Hyman Huang(黄勇) authored 
      
Introduce "x-vcpu-dirty-limit-period" migration experimental
parameter, which is in the range of 1 to 1000ms and used to
make dirtyrate calculation period configurable.

Currently with the "x-vcpu-dirty-limit-period" varies, the
total time of live migration changes, test results show the
optimal value of "x-vcpu-dirty-limit-period" ranges from
500ms to 1000 ms. "x-vcpu-dirty-limit-period" should be made
stable once it proves best value can not be determined with
developer's experiments.

Signed-off-by: default avatarHyman Huang(黄勇) <yong.huang@smartx.com>
Reviewed-by: default avatarMarkus Armbruster <armbru@redhat.com>
Reviewed-by: default avatarJuan Quintela <quintela@redhat.com>
Message-Id: <168618975839.6361.17407633874747688653-2@git.sr.ht>
Signed-off-by: default avatarJuan Quintela <quintela@redhat.com>
      4d807857
  9. Jul 25, 2023
  11. Jul 17, 2023
  13. Jul 10, 2023
  15. Jun 30, 2023
    • Avihai Horon's avatar
      migration: Add switchover ack capability · 6574232f
      Avihai Horon authored 
      
Migration downtime estimation is calculated based on bandwidth and
remaining migration data. This assumes that loading of migration data in
the destination takes a negligible amount of time and that downtime
depends only on network speed.

While this may be true for RAM, it's not necessarily true for other
migrated devices. For example, loading the data of a VFIO device in the
destination might require from the device to allocate resources, prepare
internal data structures and so on. These operations can take a
significant amount of time which can increase migration downtime.

This patch adds a new capability "switchover ack" that prevents the
source from stopping the VM and completing the migration until an ACK
is received from the destination that it's OK to do so.

This can be used by migrated devices in various ways to reduce downtime.
For example, a device can send initial precopy metadata to pre-allocate
resources in the destination and use this capability to make sure that
the pre-allocation is completed before the source VM is stopped, so it
will have full effect.

This new capability relies on the return path capability to communicate
from the destination back to the source.

The actual implementation of the capability will be added in the
following patches.

Signed-off-by: default avatarAvihai Horon <avihaih@nvidia.com>
Reviewed-by: default avatarPeter Xu <peterx@redhat.com>
Acked-by: default avatarMarkus Armbruster <armbru@redhat.com>
Tested-by: default avatarYangHang Liu <yanghliu@redhat.com>
Acked-by: default avatarAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: default avatarCédric Le Goater <clg@redhat.com>
      6574232f
  17. Jun 27, 2023
  19. Jun 26, 2023
  21. Jun 23, 2023
  23. Jun 22, 2023
  25. Jun 20, 2023
  27. Jun 13, 2023
    • Steve Sistare's avatar
      exec/memory: Introduce RAM_NAMED_FILE flag · b0182e53
      Steve Sistare authored 
      
migrate_ignore_shared() is an optimization that avoids copying memory
that is visible and can be mapped on the target.  However, a
memory-backend-ram or a memory-backend-memfd block with the RAM_SHARED
flag set is not migrated when migrate_ignore_shared() is true.  This is
wrong, because the block has no named backing store, and its contents will
be lost.  To fix, ignore shared memory iff it is a named file.  Define a
new flag RAM_NAMED_FILE to distinguish this case.

Signed-off-by: default avatarSteve Sistare <steven.sistare@oracle.com>
Reviewed-by: default avatarPeter Xu <peterx@redhat.com>
Message-Id: <1686151116-253260-1-git-send-email-steven.sistare@oracle.com>
Signed-off-by: default avatarPhilippe Mathieu-Daudé <philmd@linaro.org>
      b0182e53
  29. Jun 09, 2023
  31. Jun 05, 2023
    • Jean-Louis Dupond's avatar
      qcow2: add discard-no-unref option · 42a2890a
      Jean-Louis Dupond authored 
      When we for example have a sparse qcow2 image and discard: unmap is enabled,
there can be a lot of fragmentation in the image after some time. Especially on VM's
that do a lot of writes/deletes.
This causes the qcow2 image to grow even over 110% of its virtual size,
because the free gaps in the image get too small to allocate new
continuous clusters. So it allocates new space at the end of the image.

Disabling discard is not an option, as discard is needed to keep the
incremental backup size as low as possible. Without discard, the
incremental backups would become large, as qemu thinks it's just dirty
blocks but it doesn't know the blocks are unneeded.
So we need to avoid fragmentation but also 'empty' the unneeded blocks in
the image to have a small incremental backup.

In addition, we also want to send the discards further down the stack, so
the underlying blocks are still discarded.

Therefor we introduce a new qcow2 option "discard-no-unref".
When setting this option to true, discards will no longer have the qcow2
driver relinquish cluster allocations. Other than that, the request is
handled as normal: All clusters in range are marked as zero, and, if
pass-discard-request is true, it is passed further down the stack.
The only difference is that the now-zero clusters are preallocated
instead of being unallocated.
This will avoid fragmentation on the qcow2 image.

Fixes: https://gitlab.com/qemu-project/qemu/-/issues/1621


Signed-off-by: default avatarJean-Louis Dupond <jean-louis@dupond.be>
Message-Id: <20230605084523.34134-2-jean-louis@dupond.be>
Reviewed-by: default avatarHanna Czenczek <hreitz@redhat.com>
Signed-off-by: default avatarHanna Czenczek <hreitz@redhat.com>
      42a2890a
  33. Jun 02, 2023
    • Eric Blake's avatar
      cutils: Adjust signature of parse_uint[_full] · bd1386cc
      Eric Blake authored 
      
It's already confusing that we have two very similar functions for
wrapping the parse of a 64-bit unsigned value, differing mainly on
whether they permit leading '-'.  Adjust the signature of parse_uint()
and parse_uint_full() to be like all of qemu_strto*(): put the result
parameter last, use the same types (uint64_t and unsigned long long
have the same width, but are not always the same type), and mark
endptr const (this latter change only affects the rare caller of
parse_uint).  Adjust all callers in the tree.

While at it, note that since cutils.c already includes:

    QEMU_BUILD_BUG_ON(sizeof(int64_t) != sizeof(long long));

we are guaranteed that the result of parse_uint* cannot exceed
UINT64_MAX (or the build would have failed), so we can drop
pre-existing dead comparisons in opts-visitor.c that were never false.

Reviewed-by: default avatarHanna Czenczek <hreitz@redhat.com>
Message-Id: <20230522190441.64278-8-eblake@redhat.com>
[eblake: Drop dead code spotted by Markus]
Signed-off-by: default avatarEric Blake <eblake@redhat.com>
      bd1386cc
  35. Jun 01, 2023
  37. May 28, 2023
  39. May 23, 2023
  41. May 22, 2023
  43. May 15, 2023
  45. May 10, 2023
    • Vladimir Sementsov-Ogievskiy's avatar
      build: move COLO under CONFIG_REPLICATION · 51e47cf8
      Vladimir Sementsov-Ogievskiy authored 
      
We don't allow to use x-colo capability when replication is not
configured. So, no reason to build COLO when replication is disabled,
it's unusable in this case.

Note also that the check in migrate_caps_check() is not the only
restriction: some functions in migration/colo.c will just abort if
called with not defined CONFIG_REPLICATION, for example:

    migration_iteration_finish()
       case MIGRATION_STATUS_COLO:
           migrate_start_colo_process()
               colo_process_checkpoint()
                   abort()

It could probably make sense to have possibility to enable COLO without
REPLICATION, but this requires deeper audit of colo & replication code,
which may be done later if needed.

Signed-off-by: default avatarVladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Acked-by: default avatarDr. David Alan Gilbert <dave@treblig.org>
Reviewed-by: default avatarJuan Quintela <quintela@redhat.com>
Message-Id: <20230428194928.1426370-4-vsementsov@yandex-team.ru>
Signed-off-by: default avatarJuan Quintela <quintela@redhat.com>
      51e47cf8
    • Markus Armbruster's avatar
      qapi: Reformat doc comments to conform to current conventions · a937b6aa
      Markus Armbruster authored 
      
Change

    # @name: Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed
    #        do eiusmod tempor incididunt ut labore et dolore magna aliqua.

to

    # @name: Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed
    #     do eiusmod tempor incididunt ut labore et dolore magna aliqua.

See recent commit "qapi: Relax doc string @name: description
indentation rules" for rationale.

Reflow paragraphs to 70 columns width, and consistently use two spaces
to separate sentences.

To check the generated documentation does not change, I compared the
generated HTML before and after this commit with "wdiff -3".  Finds no
differences.  Comparing with diff is not useful, as the reflown
paragraphs are visible there.

Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
Message-Id: <20230428105429.1687850-18-armbru@redhat.com>
Reviewed-by: default avatarJuan Quintela <quintela@redhat.com>
Acked-by: default avatarLukas Straub <lukasstraub2@web.de>
[Straightforward conflicts in qapi/audio.json qapi/misc-target.json
qapi/run-state.json resolved]
      a937b6aa
  47. May 09, 2023