Skip to content
Snippets Groups Projects
Select Git revision
  • feature/libtcg-rebase default protected
1 result
Search by author
  • Any Author
  • Anton Anton anjo
1 author
  1. Apr 26, 2022
  3. Apr 21, 2022
  5. Apr 20, 2022
  7. Apr 06, 2022
  9. Mar 22, 2022
  11. Mar 07, 2022
  13. Jan 14, 2022
  15. Dec 28, 2021
  17. Sep 15, 2021
    • Eric Blake's avatar
      qemu-img: Add -F shorthand to convert · 1899bf47
      Eric Blake authored 
      
Although we have long supported 'qemu-img convert -o
backing_file=foo,backing_fmt=bar', the fact that we have a shortcut -B
for backing_file but none for backing_fmt has made it more likely that
users accidentally run into:

qemu-img: warning: Deprecated use of backing file without explicit backing format

when using -B instead of -o.  For similarity with other qemu-img
commands, such as create and compare, add '-F $fmt' as the shorthand
for '-o backing_fmt=$fmt'.  Update iotest 122 for coverage of both
spellings.

Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Message-Id: <20210913131735.1948339-1-eblake@redhat.com>
Reviewed-by: default avatarVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: default avatarMaxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: default avatarHanna Reitz <hreitz@redhat.com>
      1899bf47
    • Hanna Reitz's avatar
      qemu-img: Allow target be aligned to sector size · a1c62436
      Hanna Reitz authored 
      We cannot write to images opened with O_DIRECT unless we allow them to
be resized so they are aligned to the sector size: Since 9c60a5d1,
bdrv_node_refresh_perm() ensures that for nodes whose length is not
aligned to the request alignment and where someone has taken a WRITE
permission, the RESIZE permission is taken, too).

Let qemu-img convert pass the BDRV_O_RESIZE flag (which causes
blk_new_open() to take the RESIZE permission) when using cache=none for
the target, so that when writing to it, it can be aligned to the target
sector size.

Without this patch, an error is returned:

$ qemu-img convert -f raw -O raw -t none foo.img /mnt/tmp/foo.img
qemu-img: Could not open '/mnt/tmp/foo.img': Cannot get 'write'
permission without 'resize': Image size is not a multiple of request
alignment

Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1994266


Signed-off-by: default avatarHanna Reitz <hreitz@redhat.com>
Message-Id: <20210819101200.64235-1-hreitz@redhat.com>
Reviewed-by: default avatarVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
      a1c62436
  19. Aug 26, 2021
  21. Jul 21, 2021
    • Eric Blake's avatar
      qemu-img: Add --skip-broken-bitmaps for 'convert --bitmaps' · 955171e4
      Eric Blake authored 
      The point of 'qemu-img convert --bitmaps' is to be a convenience for
actions that are already possible through a string of smaller
'qemu-img bitmap' sub-commands.  One situation not accounted for
already is that if a source image contains an inconsistent bitmap (for
example, because a qemu process died abruptly before flushing bitmap
state), the user MUST delete those inconsistent bitmaps before
anything else useful can be done with the image.

We don't want to delete inconsistent bitmaps by default: although a
corrupt bitmap is only a loss of optimization rather than a corruption
of user-visible data, it is still nice to require the user to opt in
to the fact that they are aware of the loss of the bitmap.  Still,
requiring the user to check 'qemu-img info' to see whether bitmaps are
consistent, then use 'qemu-img bitmap --remove' to remove offenders,
all before using 'qemu-img convert', is a lot more work than just
adding a knob 'qemu-img convert --bitmaps --skip-broken-bitmaps' which
opts in to skipping the broken bitmaps.

After testing the new option, also demonstrate the way to manually fix
things (either deleting bad bitmaps, or re-creating them as empty) so
that it is possible to convert without the option.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1946084


Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Message-Id: <20210709153951.2801666-4-eblake@redhat.com>
[eblake: warning message tweak, test enhancements]
Reviewed-by: default avatarVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
      955171e4
    • Eric Blake's avatar
      qemu-img: Fail fast on convert --bitmaps with inconsistent bitmap · 74a4320f
      Eric Blake authored 
      
Waiting until the end of the convert operation (a potentially
time-consuming task) to finally detect that we can't copy a bitmap is
bad, comparing to failing fast up front.  Furthermore, this prevents
us from leaving a file behind with a bitmap that is not marked as
inconsistent even though it does not have sane contents.

This fixes the problems exposed in the previous patch to the iotest:
it adds a fast failure up front, and even if we don't fail early, it
ensures that any bitmap we add but do not properly populate is removed
again rather than left behind incomplete.

Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Message-Id: <20210709153951.2801666-3-eblake@redhat.com>
[eblake: add a hint to the warning message, simplify name computation]
Reviewed-by: default avatarNir Soffer <nsoffer@redhat.com>
Reviewed-by: default avatarVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
      74a4320f
  23. Jul 12, 2021
    • Eric Blake's avatar
      qemu-img: Make unallocated part of backing chain obvious in map · 8417e137
      Eric Blake authored 
      
The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0.  But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer.  This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).

The task of reconstructing a qcow2 chain was made harder in commit
0da98568 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.

An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients.  The iotests have
several examples where this distinction demonstrates the additional
accuracy.

Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: default avatarVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: default avatarEric Blake <eblake@redhat.com>
      8417e137
  25. Jul 09, 2021
  27. Jun 25, 2021
  29. Apr 30, 2021
    • Kevin Wolf's avatar
      qemu-img convert: Unshare write permission for source · 0b8fb55c
      Kevin Wolf authored 
      
For a successful conversion of an image, we must make sure that its
content doesn't change during the conversion.

A special case of this is using the same image file both as the source
and as the destination. If both input and output format are raw, the
operation would just be useless work, with other formats it is a sure
way to destroy the image. This will now fail because the image file
can't be opened a second time for the output when opening it for the
input has already acquired file locks to unshare BLK_PERM_WRITE.

Nevertheless, if there is some reason in a special case why it is
actually okay to allow writes to the image while it is being converted,
-U can still be used to force sharing all permissions.

Note that for most image formats, BLK_PERM_WRITE would already be
unshared by the format driver, so this only really makes a difference
for raw source images (but any output format).

Reported-by: default avatarXueqiang Wei <xuwei@redhat.com>
Signed-off-by: default avatarKevin Wolf <kwolf@redhat.com>
Reviewed-by: default avatarEric Blake <eblake@redhat.com>
Message-Id: <20210422164344.283389-3-kwolf@redhat.com>
Reviewed-by: default avatarVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: default avatarKevin Wolf <kwolf@redhat.com>
      0b8fb55c
  31. Mar 19, 2021
    • Kevin Wolf's avatar
      qemu-img: Use user_creatable_process_cmdline() for --object · 99b1e646
      Kevin Wolf authored 
      
This switches qemu-img from a QemuOpts-based parser for --object to
user_creatable_process_cmdline() which uses a keyval parser and enforces
the QAPI schema.

Apart from being a cleanup, this makes non-scalar properties accessible.

As a side effect, fix wrong exit codes in the object parsing error path
of 'qemu-img compare'. This was broken in commit 334c43e2 because
&error_fatal exits with an exit code of 1, while it should have been 2.

Document that exit code 0 is also returned when just requested help was
printed instead of comparing images. This is preexisting behaviour that
isn't changed by this patch, though another instance of it is added with
'--object help'.

Signed-off-by: default avatarKevin Wolf <kwolf@redhat.com>
Acked-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
Acked-by: default avatarPeter Krempa <pkrempa@redhat.com>
Reviewed-by: default avatarEric Blake <eblake@redhat.com>
      99b1e646
  33. Jan 28, 2021
  35. Dec 19, 2020
    • Markus Armbruster's avatar
      qobject: Change qobject_to_json()'s value to GString · eab3a467
      Markus Armbruster authored 
      
qobject_to_json() and qobject_to_json_pretty() build a GString, then
covert it to QString.  Just one of the callers actually needs a
QString: qemu_rbd_parse_filename().  A few others need a string they
can modify: qmp_send_response(), qga's send_response(), to_json_str(),
and qmp_fd_vsend_fds().  The remainder just need a string.

Change qobject_to_json() and qobject_to_json_pretty() to return the
GString.

qemu_rbd_parse_filename() now has to convert to QString.  All others
save a QString temporary.  to_json_str() actually becomes a bit
simpler, because GString provides more convenient modification
functions.

Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
Message-Id: <20201211171152.146877-6-armbru@redhat.com>
      eab3a467
    • Markus Armbruster's avatar
      qobject: Make qobject_to_json_pretty() take a pretty argument · 6589f459
      Markus Armbruster authored 
      
Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
Message-Id: <20201211171152.146877-4-armbru@redhat.com>
      6589f459
    • Eric Blake's avatar
      qapi: Use QAPI_LIST_PREPEND() where possible · 54aa3de7
      Eric Blake authored 
      
Anywhere we create a list of just one item or by prepending items
(typically because order doesn't matter), we can use
QAPI_LIST_PREPEND().  But places where we must keep the list in order
by appending remain open-coded until later patches.

Note that as a side effect, this also performs a cleanup of two minor
issues in qga/commands-posix.c: the old code was performing
 new = g_malloc0(sizeof(*ret));
which 1) is confusing because you have to verify whether 'new' and
'ret' are variables with the same type, and 2) would conflict with C++
compilation (not an actual problem for this file, but makes
copy-and-paste harder).

Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Message-Id: <20201113011340.463563-5-eblake@redhat.com>
Reviewed-by: default avatarMarkus Armbruster <armbru@redhat.com>
Acked-by: default avatarStefan Hajnoczi <stefanha@redhat.com>
[Straightforward conflicts due to commit a8aa94b5 "qga: update
schema for guest-get-disks 'dependents' field" and commit a10b453a
"target/mips: Move mips_cpu_add_definition() from helper.c to cpu.c"
resolved.  Commit message tweaked.]
Signed-off-by: default avatarMarkus Armbruster <armbru@redhat.com>
      54aa3de7
  37. Nov 11, 2020
  39. Nov 03, 2020
  41. Oct 27, 2020
  43. Sep 21, 2020
    • Eric Blake's avatar
      qemu-img: Support bitmap --merge into backing image · 14f16bf9
      Eric Blake authored 
      If you have the chain 'base.qcow2 <- top.qcow2' and want to merge a
bitmap from top into base, qemu-img was failing with:

qemu-img: Could not open 'top.qcow2': Could not open backing file: Failed to get shared "write" lock
Is another process using the image [base.qcow2]?

The easiest fix is to not open the entire backing chain of either
image (source or destination); after all, the point of 'qemu-img
bitmap' is solely to manipulate bitmaps directly within a single qcow2
image, and this is made more precise if we don't pay attention to
other images in the chain that may happen to have a bitmap by the same
name.

However, note that on a case-by-case analysis, there _are_ times where
we treat it as a feature that we can access a bitmap from a backing
layer in association with an overlay BDS.  A demonstration of this is
using NBD to expose both an overlay BDS (for constant contents) and a
bitmap (for learning which blocks are interesting) during an
incremental backup:

Base <- Active <- Temporary
          \--block job ->/

where Temporary is being fed by a backup 'sync=none' job.  When
exposing Temporary over NBD, referring to a bitmap that lives only in
Active is less effort than having to copy a bitmap into Temporary [1].
So the testsuite additions in this patch check both where bitmaps get
allocated (the qemu-img info output), and that qemu-nbd is indeed able
to access a bitmap inherited from the backing chain since it is a
different use case than 'qemu-img bitmap'.

[1] Full disclosure: prior to the recent commit 374eedd1 and
friends, we were NOT able to see bitmaps through filters, which meant
that we actually did not have nice clean semantics for uniformly being
able to pick up bitmaps from anywhere in the backing chain (seen as a
change in behavior between qemu 4.1 and 4.2 at commit 00e30f05, when
block-copy swapped from a one-off to a filter).  Which means libvirt
was already coded to copy bitmaps around for the sake of older qemu,
even though modern qemu no longer needs it.  Oh well.

Fixes: http://bugzilla.redhat.com/1877209


Reported-by: default avatarEyal Shenitzky <eshenitz@redhat.com>
Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Message-Id: <20200914191009.644842-1-eblake@redhat.com>
[eblake: more commit message tweaks, per Max Reitz review]
Reviewed-by: default avatarMax Reitz <mreitz@redhat.com>
Reviewed-by: default avatarVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
      14f16bf9
  45. Sep 17, 2020
  47. Sep 15, 2020
  49. Sep 07, 2020
    • Hanna Reitz's avatar
      qemu-img: Use child access functions · 4a2061e6
      Hanna Reitz authored 
      
This changes iotest 204's output, because blkdebug on top of a COW node
used to make qemu-img map disregard the rest of the backing chain (the
backing chain was broken by the filter).  With this patch, the
allocation in the base image is reported correctly.

Signed-off-by: default avatarMax Reitz <mreitz@redhat.com>
      4a2061e6
  51. Sep 02, 2020
  53. Jul 17, 2020
  55. Jul 14, 2020
    • Eric Blake's avatar
      qemu-img: Deprecate use of -b without -F · d9f059aa
      Eric Blake authored 
      
Creating an image that requires format probing of the backing image is
potentially unsafe (we've had several CVEs over the years based on
probes leaking information to the guest on a subsequent boot, although
these days tools like libvirt are aware of the issue enough to prevent
the worst effects).  For example, if our probing algorithm ever
changes, or if other tools like libvirt determine a different probe
result than we do, then subsequent use of that backing file under a
different format will present corrupted data to the guest.
Fortunately, the worst effects occur only when the backing image is
originally raw, and we at least prevent commit into a probed raw
backing file that would change its probed type.

Still, it is worth starting a deprecation clock so that future
qemu-img can refuse to create backing chains that would rely on
probing, to encourage clients to avoid unsafe practices.  Most
warnings are intentionally emitted from bdrv_img_create() in the block
layer, but qemu-img convert uses bdrv_create() which cannot emit its
own warning without causing spurious warnings on other code paths.  In
the end, all command-line image creation or backing file rewriting now
performs a check.

Furthermore, if we probe a backing file as non-raw, then it is safe to
explicitly record that result (rather than relying on future probes);
only where we probe a raw image do we care about further warnings to
the user when using such an image (for example, commits into a
probed-raw backing file are prevented), to help them improve their
tooling.  But whether or not we make the probe results explicit, we
still warn the user to remind them to upgrade their workflow to supply
-F always.

iotest 114 specifically wants to create an unsafe image for later
amendment rather than defaulting to our new default of recording a
probed format, so it needs an update.  While touching it, expand it to
cover all of the various warnings enabled by this patch.  iotest 301
also shows a change to qcow messages.

Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Message-Id: <20200706203954.341758-11-eblake@redhat.com>
Signed-off-by: default avatarKevin Wolf <kwolf@redhat.com>
      d9f059aa
    • Eric Blake's avatar
      block: Add support to warn on backing file change without format · e54ee1b3
      Eric Blake authored 
      
For now, this is a mechanical addition; all callers pass false. But
the next patch will use it to improve 'qemu-img rebase -u' when
selecting a backing file with no format.

Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Reviewed-by: default avatarPeter Krempa <pkrempa@redhat.com>
Reviewed-by: default avatarJán Tomko <jtomko@redhat.com>
Message-Id: <20200706203954.341758-10-eblake@redhat.com>
Signed-off-by: default avatarKevin Wolf <kwolf@redhat.com>
      e54ee1b3
    • Eric Blake's avatar
      block: Finish deprecation of 'qemu-img convert -n -o' · 25956af3
      Eric Blake authored 
      
It's been two releases since we started warning; time to make the
combination an error as promised.  There was no iotest coverage, so
add some.

While touching the documentation, tweak another section heading for
consistent style.

Signed-off-by: default avatarEric Blake <eblake@redhat.com>
Message-Id: <20200706203954.341758-3-eblake@redhat.com>
Signed-off-by: default avatarKevin Wolf <kwolf@redhat.com>
      25956af3
    • Kevin Wolf's avatar
      qemu-img map: Don't limit block status request size · d0ceea88
      Kevin Wolf authored 
      
Limiting each loop iteration of qemu-img map to 1 GB was arbitrary from
the beginning, though it only cut the maximum in half then because the
interface was a signed 32 bit byte count. These days, bdrv_block_status
supports a 64 bit byte count, so the arbitrary limit is even worse.

On file-posix, bdrv_block_status() eventually maps to SEEK_HOLE and
SEEK_DATA, which don't support a limit, but always do all of the work
necessary to find the start of the next hole/data. Much of this work may
be repeated if we don't use this information fully, but query with an
only slightly larger offset in the next loop iteration. Therefore, if
bdrv_block_status() is called in a loop, it should always pass the
full number of bytes that the whole loop is interested in.

This removes the arbitrary limit and speeds up 'qemu-img map'
significantly on heavily fragmented images.

Signed-off-by: default avatarKevin Wolf <kwolf@redhat.com>
Message-Id: <20200707144629.51235-1-kwolf@redhat.com>
Reviewed-by: default avatarEric Blake <eblake@redhat.com>
Signed-off-by: default avatarKevin Wolf <kwolf@redhat.com>
      d0ceea88