Skip to content
Snippets Groups Projects
Select Git revision
  • feature/libtcg-rebase default protected
1 result
Search by author
  • Any Author
  • Anton Anton anjo
1 author
  1. Apr 30, 2021
    • Kevin Wolf's avatar
      qemu-img convert: Unshare write permission for source · 0b8fb55c
      Kevin Wolf authored 
      
For a successful conversion of an image, we must make sure that its
content doesn't change during the conversion.

A special case of this is using the same image file both as the source
and as the destination. If both input and output format are raw, the
operation would just be useless work, with other formats it is a sure
way to destroy the image. This will now fail because the image file
can't be opened a second time for the output when opening it for the
input has already acquired file locks to unshare BLK_PERM_WRITE.

Nevertheless, if there is some reason in a special case why it is
actually okay to allow writes to the image while it is being converted,
-U can still be used to force sharing all permissions.

Note that for most image formats, BLK_PERM_WRITE would already be
unshared by the format driver, so this only really makes a difference
for raw source images (but any output format).

Reported-by: default avatarXueqiang Wei <xuwei@redhat.com>
Signed-off-by: default avatarKevin Wolf <kwolf@redhat.com>
Reviewed-by: default avatarEric Blake <eblake@redhat.com>
Message-Id: <20210422164344.283389-3-kwolf@redhat.com>
Reviewed-by: default avatarVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: default avatarKevin Wolf <kwolf@redhat.com>
      0b8fb55c
  3. Mar 19, 2021
    • Kevin Wolf's avatar
      qemu-img: Use user_creatable_process_cmdline() for --object · 99b1e646
      Kevin Wolf authored 
      
This switches qemu-img from a QemuOpts-based parser for --object to
user_creatable_process_cmdline() which uses a keyval parser and enforces
the QAPI schema.

Apart from being a cleanup, this makes non-scalar properties accessible.

As a side effect, fix wrong exit codes in the object parsing error path
of 'qemu-img compare'. This was broken in commit 334c43e2 because
&error_fatal exits with an exit code of 1, while it should have been 2.

Document that exit code 0 is also returned when just requested help was
printed instead of comparing images. This is preexisting behaviour that
isn't changed by this patch, though another instance of it is added with
'--object help'.

Signed-off-by: default avatarKevin Wolf <kwolf@redhat.com>
Acked-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
Acked-by: default avatarPeter Krempa <pkrempa@redhat.com>
Reviewed-by: default avatarEric Blake <eblake@redhat.com>
      99b1e646
  5. Jan 28, 2021
  7. Dec 19, 2020
    • Markus Armbruster's avatar
      qobject: Change qobject_to_json()'s value to GString · eab3a467
      Markus Armbruster authored 
      
qobject_to_json() and qobject_to_json_pretty() build a GString, then
covert it to QString.  Just one of the callers actually needs a
QString: qemu_rbd_parse_filename().  A few others need a string they
can modify: qmp_send_response(), qga's send_response(), to_json_str(),
and qmp_fd_vsend_fds().  The remainder just need a string.

Change qobject_to_json() and qobject_to_json_pretty() to return the
GString.

qemu_rbd_parse_filename() now has to convert to QString.  All others
save a QString temporary.  to_json_str() actually becomes a bit
simpler, because GString provides more convenient modification
functions.

Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
Message-Id: <20201211171152.146877-6-armbru@redhat.com>
      eab3a467
    • Markus Armbruster's avatar
      qobject: Make qobject_to_json_pretty() take a pretty argument · 6589f459
      Markus Armbruster authored 
      
Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
Message-Id: <20201211171152.146877-4-armbru@redhat.com>
      6589f459
    • Eric Blake's avatar
      qapi: Use QAPI_LIST_PREPEND() where possible · 54aa3de7
      Eric Blake authored 
      
Anywhere we create a list of just one item or by prepending items
(typically because order doesn't matter), we can use
QAPI_LIST_PREPEND().  But places where we must keep the list in order
by appending remain open-coded until later patches.

Note that as a side effect, this also performs a cleanup of two minor
issues in qga/commands-posix.c: the old code was performing
 new = g_malloc0(sizeof(*ret));
which 1) is confusing because you have to verify whether 'new' and
'ret' are variables with the same type, and 2) would conflict with C++
compilation (not an actual problem for this file, but makes
copy-and-paste harder).

Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Message-Id: <20201113011340.463563-5-eblake@redhat.com>
Reviewed-by: default avatarMarkus Armbruster <armbru@redhat.com>
Acked-by: default avatarStefan Hajnoczi <stefanha@redhat.com>
[Straightforward conflicts due to commit a8aa94b5 "qga: update
schema for guest-get-disks 'dependents' field" and commit a10b453a
"target/mips: Move mips_cpu_add_definition() from helper.c to cpu.c"
resolved.  Commit message tweaked.]
Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
      54aa3de7
  9. Nov 11, 2020
  11. Nov 03, 2020
  13. Oct 27, 2020
  15. Sep 21, 2020
    • Eric Blake's avatar
      qemu-img: Support bitmap --merge into backing image · 14f16bf9
      Eric Blake authored 
      If you have the chain 'base.qcow2 <- top.qcow2' and want to merge a
bitmap from top into base, qemu-img was failing with:

qemu-img: Could not open 'top.qcow2': Could not open backing file: Failed to get shared "write" lock
Is another process using the image [base.qcow2]?

The easiest fix is to not open the entire backing chain of either
image (source or destination); after all, the point of 'qemu-img
bitmap' is solely to manipulate bitmaps directly within a single qcow2
image, and this is made more precise if we don't pay attention to
other images in the chain that may happen to have a bitmap by the same
name.

However, note that on a case-by-case analysis, there _are_ times where
we treat it as a feature that we can access a bitmap from a backing
layer in association with an overlay BDS.  A demonstration of this is
using NBD to expose both an overlay BDS (for constant contents) and a
bitmap (for learning which blocks are interesting) during an
incremental backup:

Base <- Active <- Temporary
          \--block job ->/

where Temporary is being fed by a backup 'sync=none' job.  When
exposing Temporary over NBD, referring to a bitmap that lives only in
Active is less effort than having to copy a bitmap into Temporary [1].
So the testsuite additions in this patch check both where bitmaps get
allocated (the qemu-img info output), and that qemu-nbd is indeed able
to access a bitmap inherited from the backing chain since it is a
different use case than 'qemu-img bitmap'.

[1] Full disclosure: prior to the recent commit 374eedd1 and
friends, we were NOT able to see bitmaps through filters, which meant
that we actually did not have nice clean semantics for uniformly being
able to pick up bitmaps from anywhere in the backing chain (seen as a
change in behavior between qemu 4.1 and 4.2 at commit 00e30f05, when
block-copy swapped from a one-off to a filter).  Which means libvirt
was already coded to copy bitmaps around for the sake of older qemu,
even though modern qemu no longer needs it.  Oh well.

Fixes: http://bugzilla.redhat.com/1877209


Reported-by: default avatarEyal Shenitzky <eshenitz@redhat.com>
Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Message-Id: <20200914191009.644842-1-eblake@redhat.com>
[eblake: more commit message tweaks, per Max Reitz review]
Reviewed-by: default avatarMax Reitz <mreitz@redhat.com>
Reviewed-by: default avatarVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
      14f16bf9
  17. Sep 17, 2020
  19. Sep 15, 2020
  21. Sep 07, 2020
    • Hanna Reitz's avatar
      qemu-img: Use child access functions · 4a2061e6
      Hanna Reitz authored 
      
This changes iotest 204's output, because blkdebug on top of a COW node
used to make qemu-img map disregard the rest of the backing chain (the
backing chain was broken by the filter).  With this patch, the
allocation in the base image is reported correctly.

Signed-off-by: default avatarMax Reitz <mreitz@redhat.com>
      4a2061e6
  23. Sep 02, 2020
  25. Jul 17, 2020
  27. Jul 14, 2020
    • Eric Blake's avatar
      qemu-img: Deprecate use of -b without -F · d9f059aa
      Eric Blake authored 
      
Creating an image that requires format probing of the backing image is
potentially unsafe (we've had several CVEs over the years based on
probes leaking information to the guest on a subsequent boot, although
these days tools like libvirt are aware of the issue enough to prevent
the worst effects).  For example, if our probing algorithm ever
changes, or if other tools like libvirt determine a different probe
result than we do, then subsequent use of that backing file under a
different format will present corrupted data to the guest.
Fortunately, the worst effects occur only when the backing image is
originally raw, and we at least prevent commit into a probed raw
backing file that would change its probed type.

Still, it is worth starting a deprecation clock so that future
qemu-img can refuse to create backing chains that would rely on
probing, to encourage clients to avoid unsafe practices.  Most
warnings are intentionally emitted from bdrv_img_create() in the block
layer, but qemu-img convert uses bdrv_create() which cannot emit its
own warning without causing spurious warnings on other code paths.  In
the end, all command-line image creation or backing file rewriting now
performs a check.

Furthermore, if we probe a backing file as non-raw, then it is safe to
explicitly record that result (rather than relying on future probes);
only where we probe a raw image do we care about further warnings to
the user when using such an image (for example, commits into a
probed-raw backing file are prevented), to help them improve their
tooling.  But whether or not we make the probe results explicit, we
still warn the user to remind them to upgrade their workflow to supply
-F always.

iotest 114 specifically wants to create an unsafe image for later
amendment rather than defaulting to our new default of recording a
probed format, so it needs an update.  While touching it, expand it to
cover all of the various warnings enabled by this patch.  iotest 301
also shows a change to qcow messages.

Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Message-Id: <20200706203954.341758-11-eblake@redhat.com>
Signed-off-by: default avatarKevin Wolf <kwolf@redhat.com>
      d9f059aa
    • Eric Blake's avatar
      block: Add support to warn on backing file change without format · e54ee1b3
      Eric Blake authored 
      
For now, this is a mechanical addition; all callers pass false. But
the next patch will use it to improve 'qemu-img rebase -u' when
selecting a backing file with no format.

Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Reviewed-by: default avatarPeter Krempa <pkrempa@redhat.com>
Reviewed-by: default avatarJán Tomko <jtomko@redhat.com>
Message-Id: <20200706203954.341758-10-eblake@redhat.com>
Signed-off-by: default avatarKevin Wolf <kwolf@redhat.com>
      e54ee1b3
    • Eric Blake's avatar
      block: Finish deprecation of 'qemu-img convert -n -o' · 25956af3
      Eric Blake authored 
      
It's been two releases since we started warning; time to make the
combination an error as promised.  There was no iotest coverage, so
add some.

While touching the documentation, tweak another section heading for
consistent style.

Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Message-Id: <20200706203954.341758-3-eblake@redhat.com>
Signed-off-by: default avatarKevin Wolf <kwolf@redhat.com>
      25956af3
    • Kevin Wolf's avatar
      qemu-img map: Don't limit block status request size · d0ceea88
      Kevin Wolf authored 
      
Limiting each loop iteration of qemu-img map to 1 GB was arbitrary from
the beginning, though it only cut the maximum in half then because the
interface was a signed 32 bit byte count. These days, bdrv_block_status
supports a 64 bit byte count, so the arbitrary limit is even worse.

On file-posix, bdrv_block_status() eventually maps to SEEK_HOLE and
SEEK_DATA, which don't support a limit, but always do all of the work
necessary to find the start of the next hole/data. Much of this work may
be repeated if we don't use this information fully, but query with an
only slightly larger offset in the next loop iteration. Therefore, if
bdrv_block_status() is called in a loop, it should always pass the
full number of bytes that the whole loop is interested in.

This removes the arbitrary limit and speeds up 'qemu-img map'
significantly on heavily fragmented images.

Signed-off-by: default avatarKevin Wolf <kwolf@redhat.com>
Message-Id: <20200707144629.51235-1-kwolf@redhat.com>
Reviewed-by: default avatarEric Blake <eblake@redhat.com>
Signed-off-by: default avatarKevin Wolf <kwolf@redhat.com>
      d0ceea88
  29. Jul 10, 2020
  31. Jul 06, 2020
  33. Jul 03, 2020
    • Kevin Wolf's avatar
      qemu-img convert: Don't pre-zero images · edafc70c
      Kevin Wolf authored 
      
Since commit 5a37b60a, qemu-img create will pre-zero the target image
if it isn't already zero-initialised (most importantly, for host block
devices, but also iscsi etc.), so that writing explicit zeros wouldn't
be necessary later.

This could speed up the operation significantly, in particular when the
source image file was only sparsely populated. However, it also means
that some block are written twice: Once when pre-zeroing them, and then
when they are overwritten with actual data. On a full image, the
pre-zeroing is wasted work because everything will be overwritten.

In practice, write_zeroes typically turns out faster than writing
explicit zero buffers, but slow enough that first zeroing everything and
then overwriting parts can be a significant net loss.

Meanwhile, qemu-img convert was rewritten in 690c7301 and zero blocks
are now written to the target using bdrv_co_pwrite_zeroes() if the
target could be pre-zeroed. This way we already make use of the faster
write_zeroes operation, but avoid writing any blocks twice.

Remove the pre-zeroing because these days this former optimisation has
actually turned into a pessimisation in the common case.

Reported-by: default avatarNir Soffer <nsoffer@redhat.com>
Signed-off-by: default avatarKevin Wolf <kwolf@redhat.com>
Message-Id: <20200622151203.35624-1-kwolf@redhat.com>
Tested-by: default avatarNir Soffer <nsoffer@redhat.com>
Reviewed-by: default avatarEric Blake <eblake@redhat.com>
Signed-off-by: default avatarKevin Wolf <kwolf@redhat.com>
      edafc70c
  35. May 28, 2020
    • Eric Blake's avatar
      qemu-img: Add convert --bitmaps option · 15e39ad9
      Eric Blake authored 
      Make it easier to copy all the persistent bitmaps of (the top layer
of) a source image along with its guest-visible contents, by adding a
boolean flag for use with qemu-img convert.  This is basically
shorthand, as the same effect could be accomplished with a series of
'qemu-img bitmap --add' and 'qemu-img bitmap --merge -b source'
commands, or by their corresponding QMP commands.

Note that this command will fail in the same scenarios where 'qemu-img
measure' omits a 'bitmaps size:' line, namely, when either the source
or the destination lacks persistent bitmap support altogether.

See also https://bugzilla.redhat.com/show_bug.cgi?id=1779893



While touching this, clean up a couple coding issues spotted in the
same function: an extra blank line, and merging back-to-back 'if
(!skip_create)' blocks.

Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Message-Id: <20200521192137.1120211-5-eblake@redhat.com>
Reviewed-by: default avatarVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
      15e39ad9
    • Eric Blake's avatar
      qemu-img: Factor out code for merging bitmaps · 6c729dd8
      Eric Blake authored 
      
The next patch will add another client that wants to merge dirty
bitmaps; it will be easier to refactor the code to construct the QAPI
struct correctly into a helper function.

Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Message-Id: <20200521192137.1120211-4-eblake@redhat.com>
Reviewed-by: default avatarVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
      6c729dd8
    • Eric Blake's avatar
      qcow2: Expose bitmaps' size during measure · 5d72c68b
      Eric Blake authored 
      It's useful to know how much space can be occupied by qcow2 persistent
bitmaps, even though such metadata is unrelated to the guest-visible
data.  Report this value as an additional QMP field, present when
measuring an existing image and output format that both support
bitmaps.  Update iotest 178 and 190 to updated output, as well as new
coverage in 190 demonstrating non-zero values made possible with the
recently-added qemu-img bitmap command (see 3b51ab4b).

The new 'bitmaps size:' field is displayed automatically as part of
'qemu-img measure' any time it is present in QMP (that is, any time
both the source image being measured and destination format support
bitmaps, even if the measurement is 0 because there are no bitmaps
present).  If the field is absent, it means that no bitmaps can be
copied (source, destination, or both lack bitmaps, including when
measuring based on size rather than on a source image).  This behavior
is compatible with an upcoming patch adding 'qemu-img convert
--bitmaps': that command will fail in the same situations where this
patch omits the field.

The addition of a new field demonstrates why we should always
zero-initialize qapi C structs; while the qcow2 driver still fully
populates all fields, the raw and crypto drivers had to be tweaked to
avoid uninitialized data.

Consideration was also given towards having a 'qemu-img measure
--bitmaps' which errors out when bitmaps are not possible, and
otherwise sums the bitmaps into the existing allocation totals rather
than displaying as a separate field, as a potential convenience
factor.  But this was ultimately decided to be more complexity than
necessary when the QMP interface was sufficient enough with bitmaps
remaining a separate field.

See also: https://bugzilla.redhat.com/1779904



Reported-by: default avatarNir Soffer <nsoffer@redhat.com>
Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Message-Id: <20200521192137.1120211-3-eblake@redhat.com>
Reviewed-by: default avatarVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
      5d72c68b
  37. May 19, 2020
    • Eric Blake's avatar
      qemu-img: Add bitmap sub-command · 3b51ab4b
      Eric Blake authored 
      
Include actions for --add, --remove, --clear, --enable, --disable, and
--merge (note that --clear is a bit of fluff, because the same can be
accomplished by removing a bitmap and then adding a new one in its
place, but it matches what QMP commands exist).  Listing is omitted,
because it does not require a bitmap name and because it was already
possible with 'qemu-img info'.  A single command line can play one or
more bitmap commands in sequence on the same bitmap name (although all
added bitmaps share the same granularity, and and all merged bitmaps
come from the same source file).  Merge defaults to other bitmaps in
the primary image, but can also be told to merge bitmaps from a
distinct image.

While this supports --image-opts for the file being modified, I did
not think it worth the extra complexity to support that for the source
file in a cross-file merges.  Likewise, I chose to have --merge only
take a single source rather than following the QMP support for
multiple merges in one go (although you can still use more than one
--merge in the command line); in part because qemu-img is offline and
therefore atomicity is not an issue.

Upcoming patches will add iotest coverage of these commands while
also testing other features.

Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Reviewed-by: default avatarMax Reitz <mreitz@redhat.com>
Message-Id: <20200513011648.166876-7-eblake@redhat.com>
Reviewed-by: default avatarVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
      3b51ab4b
    • Eric Blake's avatar
      qemu-img: Fix stale comments on doc location · 0562adf5
      Eric Blake authored 
      
Missed in commit e13c59fa.

Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Reviewed-by: default avatarMax Reitz <mreitz@redhat.com>
Message-Id: <20200513011648.166876-3-eblake@redhat.com>
Reviewed-by: default avatarVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
      0562adf5
  39. May 18, 2020
  41. Apr 30, 2020
  43. Apr 29, 2020
    • Markus Armbruster's avatar
      qemu-img: Reject broken -o "" · f62514b3
      Markus Armbruster authored 
      
qemu-img create, convert, amend, and measure use accumulate_options()
to merge multiple -o options.  This is broken for -o "":

    $ qemu-img create -f qcow2 -o backing_file=a -o "" -o backing_fmt=raw,size=1M new.qcow2
    qemu-img: warning: Could not verify backing image. This may become an error in future versions.
    Could not open 'a,backing_fmt=raw': No such file or directory
    Formatting 'new.qcow2', fmt=qcow2 size=1048576 backing_file=a,,backing_fmt=raw cluster_size=65536 lazy_refcounts=off refcount_bits=16
    $ qemu-img info new.qcow2
    image: new.qcow2
    file format: qcow2
    virtual size: 1 MiB (1048576 bytes)
    disk size: 196 KiB
    cluster_size: 65536
--> backing file: a,backing_fmt=raw
    Format specific information:
        compat: 1.1
        lazy refcounts: false
        refcount bits: 16
        corrupt: false

Merging these three -o the obvious way is wrong, because it results in
an unwanted ',' escape:

    backing_file=a,,backing_fmt=raw,size=1M
                  ~~

We could silently drop -o "", but Kevin asked me to reject it instead.

Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
Message-Id: <20200415074927.19897-10-armbru@redhat.com>
Reviewed-by: default avatarEric Blake <eblake@redhat.com>
      f62514b3