Skip to content
Snippets Groups Projects
  1. Nov 03, 2020
  2. Nov 02, 2020
    • Peter Maydell's avatar
      Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20201102a' into staging · 8545ae48
      Peter Maydell authored
      
      Migration and virtiofs fixes 2020-11-02
      
      Fixes for postcopy migration test hang
      A seccomp crash for virtiofsd on some !x86
      Help message and minor CID fix
      
      And another crack at Max's set.
      
      Signed-off-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
      
      # gpg: Signature made Mon 02 Nov 2020 19:54:59 GMT
      # gpg:                using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
      # gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
      # Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A  9FA9 0516 331E BC5B FDE7
      
      * remotes/dgilbert/tags/pull-migration-20201102a:
        tests/acceptance: Add virtiofs_submounts.py
        tests/acceptance/boot_linux: Accept SSH pubkey
        virtiofsd: Announce sub-mount points
        virtiofsd: Add mount ID to the lo_inode key
        meson.build: Check for statx()
        virtiofsd: Add attr_flags to fuse_entry_param
        virtiofsd: Check FUSE_SUBMOUNTS
        virtiofsd: Fix the help message of posix lock
        tools/virtiofsd: Check vu_init() return value (CID 1435958)
        virtiofsd: Seccomp: Add 'send' for syslog
        migration: Postpone the kick of the fault thread after recover
        migration: Unify reset of last_rb on destination node when recover
      
      Signed-off-by: default avatarPeter Maydell <peter.maydell@linaro.org>
      8545ae48
    • Hanna Reitz's avatar
      tests/acceptance: Add virtiofs_submounts.py · af1bb3fe
      Hanna Reitz authored
      
      This test invokes several shell scripts to create a random directory
      tree full of submounts, and then check in the VM whether every submount
      has its own ID and the structure looks as expected.
      
      (Note that the test scripts must be non-executable, so Avocado will not
      try to execute them as if they were tests on their own, too.)
      
      Because at this commit's date it is unlikely that the Linux kernel on
      the image provided by boot_linux.py supports submounts in virtio-fs, the
      test will be cancelled if no custom Linux binary is provided through the
      vmlinuz parameter.  (The on-image kernel can be used by providing an
      empty string via vmlinuz=.)
      
      So, invoking the test can be done as follows:
      $ avocado run \
          tests/acceptance/virtiofs_submounts.py \
          -p vmlinuz=/path/to/linux/build/arch/x86/boot/bzImage
      
      This test requires root privileges (through passwordless sudo -n),
      because at this point, virtiofsd requires them.  (If you have a
      timestamp_timeout period for sudoers (e.g. the default of 5 min), you
      can provide this by executing something like "sudo true" before invoking
      Avocado.)
      
      Signed-off-by: default avatarMax Reitz <mreitz@redhat.com>
      Message-Id: <20201102161859.156603-8-mreitz@redhat.com>
      Tested-by: default avatarEduardo Habkost <ehabkost@redhat.com>
      Signed-off-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
      af1bb3fe
    • Hanna Reitz's avatar
      tests/acceptance/boot_linux: Accept SSH pubkey · 0a779803
      Hanna Reitz authored
      
      Let download_cloudinit() take an optional pubkey, which subclasses of
      BootLinux can pass through setUp().
      
      Signed-off-by: default avatarMax Reitz <mreitz@redhat.com>
      Reviewed-by: default avatarPhilippe Mathieu-Daudé <philmd@redhat.com>
      Reviewed-by: default avatarWillian Rampazzo <willianr@redhat.com>
      Reviewed-by: default avatarStefan Hajnoczi <stefanha@redhat.com>
      Message-Id: <20201102161859.156603-7-mreitz@redhat.com>
      Signed-off-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
      0a779803
    • Hanna Reitz's avatar
      virtiofsd: Announce sub-mount points · 9d82f6a3
      Hanna Reitz authored
      
      Whenever we encounter a directory with an st_dev or mount ID that
      differs from that of its parent, we set the FUSE_ATTR_SUBMOUNT flag so
      the guest can create a submount for it.
      
      We only need to do so in lo_do_lookup().  The following functions return
      a fuse_attr object:
      - lo_create(), though fuse_reply_create(): Calls lo_do_lookup().
      - lo_lookup(), though fuse_reply_entry(): Calls lo_do_lookup().
      - lo_mknod_symlink(), through fuse_reply_entry(): Calls lo_do_lookup().
      - lo_link(), through fuse_reply_entry(): Creating a link cannot create a
        submount, so there is no need to check for it.
      - lo_getattr(), through fuse_reply_attr(): Announcing submounts when the
        node is first detected (at lookup) is sufficient.  We do not need to
        return the submount attribute later.
      - lo_do_readdir(), through fuse_add_direntry_plus(): Calls
        lo_do_lookup().
      
      Make announcing submounts optional, so submounts are only announced to
      the guest with the announce_submounts option.  Some users may prefer the
      current behavior, so that the guest learns nothing about the host mount
      structure.
      
      (announce_submounts is force-disabled when the guest does not present
      the FUSE_SUBMOUNTS capability, or when there is no statx().)
      
      Signed-off-by: default avatarMax Reitz <mreitz@redhat.com>
      Reviewed-by: default avatarStefan Hajnoczi <stefanha@redhat.com>
      Message-Id: <20201102161859.156603-6-mreitz@redhat.com>
      Signed-off-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
      9d82f6a3
    • Hanna Reitz's avatar
      virtiofsd: Add mount ID to the lo_inode key · d672fce6
      Hanna Reitz authored
      Using st_dev is not sufficient to uniquely identify a mount: You can
      mount the same device twice, but those are still separate trees, and
      e.g. by mounting something else inside one of them, they may differ.
      
      Using statx(), we can get a mount ID that uniquely identifies a mount.
      If that is available, add it to the lo_inode key.
      
      Most of this patch is taken from Miklos's mail here:
      https://marc.info/?l=fuse-devel&m=160062521827983
      
      
      (virtiofsd-use-mount-id.patch attachment)
      
      Suggested-by: default avatarMiklos Szeredi <mszeredi@redhat.com>
      Signed-off-by: default avatarMax Reitz <mreitz@redhat.com>
      Reviewed-by: default avatarStefan Hajnoczi <stefanha@redhat.com>
      Message-Id: <20201102161859.156603-5-mreitz@redhat.com>
      Signed-off-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
      d672fce6
    • Hanna Reitz's avatar
      meson.build: Check for statx() · 84e319a5
      Hanna Reitz authored
      
      Check whether the glibc provides statx() and if so, define CONFIG_STATX.
      
      Signed-off-by: default avatarMax Reitz <mreitz@redhat.com>
      Reviewed-by: default avatarStefan Hajnoczi <stefanha@redhat.com>
      Message-Id: <20201102161859.156603-4-mreitz@redhat.com>
      Signed-off-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
      84e319a5
    • Hanna Reitz's avatar
      virtiofsd: Add attr_flags to fuse_entry_param · 93e79851
      Hanna Reitz authored
      
      fuse_entry_param is converted to fuse_attr on the line (by
      fill_entry()), so it should have a member that mirrors fuse_attr.flags.
      
      fill_entry() should then copy this fuse_entry_param.attr_flags to
      fuse_attr.flags.
      
      Signed-off-by: default avatarMax Reitz <mreitz@redhat.com>
      Reviewed-by: default avatarStefan Hajnoczi <stefanha@redhat.com>
      Message-Id: <20201102161859.156603-3-mreitz@redhat.com>
      Signed-off-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
      93e79851
    • Hanna Reitz's avatar
      virtiofsd: Check FUSE_SUBMOUNTS · 9c6ac043
      Hanna Reitz authored
      
      FUSE_SUBMOUNTS is a pure indicator by the kernel to signal that it
      supports submounts.  It does not check its state in the init reply, so
      there is nothing for fuse_lowlevel.c to do but to check its existence
      and copy it into fuse_conn_info.capable.
      
      Signed-off-by: default avatarMax Reitz <mreitz@redhat.com>
      Reviewed-by: default avatarStefan Hajnoczi <stefanha@redhat.com>
      Message-Id: <20201102161859.156603-2-mreitz@redhat.com>
      Signed-off-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
      9c6ac043
    • Jiachen Zhang's avatar
      virtiofsd: Fix the help message of posix lock · 0429eaf5
      Jiachen Zhang authored
      
      The commit 88fc1079 disabled remote
      posix locks by default. But the --help message still says it is enabled
      by default. So fix it to output no_posix_lock.
      
      Signed-off-by: default avatarJiachen Zhang <zhangjiachen.jaycee@bytedance.com>
      Message-Id: <20201027081558.29904-1-zhangjiachen.jaycee@bytedance.com>
      Reviewed-by: default avatarPhilippe Mathieu-Daudé <philmd@redhat.com>
      Reviewed-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
      Signed-off-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
      0429eaf5
    • Philippe Mathieu-Daudé's avatar
      tools/virtiofsd: Check vu_init() return value (CID 1435958) · 26930260
      Philippe Mathieu-Daudé authored
      
      Since commit 6f5fd837, vu_init() can fail if malloc() returns NULL.
      
      This fixes the following Coverity warning:
      
        CID 1435958 (#1 of 1): Unchecked return value (CHECKED_RETURN)
      
      Fixes: 6f5fd837 ("libvhost-user: support many virtqueues")
      Reviewed-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
      Signed-off-by: default avatarPhilippe Mathieu-Daudé <philmd@redhat.com>
      Message-Id: <20201102092339.2034297-1-philmd@redhat.com>
      Signed-off-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
      26930260
    • Dr. David Alan Gilbert's avatar
      virtiofsd: Seccomp: Add 'send' for syslog · dcaac9f1
      Dr. David Alan Gilbert authored
      On ppc, and some other archs, it looks like syslog ends up using 'send'
      rather than 'sendto'.
      
      Reference: https://github.com/kata-containers/kata-containers/issues/1050
      
      
      
      Reported-by: default avatar <amulmek1@in.ibm.com>
      Signed-off-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
      Message-Id: <20201102150750.34565-1-dgilbert@redhat.com>
      Reviewed-by: default avatarDaniel P. Berrangé <berrange@redhat.com>
      Signed-off-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
      dcaac9f1
    • Peter Xu's avatar
      migration: Postpone the kick of the fault thread after recover · 5e773431
      Peter Xu authored
      
      The new migrate_send_rp_req_pages_pending() call should greatly improve
      destination responsiveness because it will resync faulted address after
      postcopy recovery.  However it is also the 1st place to initiate the page
      request from the main thread.
      
      One thing is overlooked on that migrate_send_rp_message_req_pages() is not
      designed to be thread-safe.  So if we wake the fault thread before syncing all
      the faulted pages in the main thread, it means they can race.
      
      Postpone the wake up operation after the sync of faulted addresses.
      
      Fixes: 0c26781c ("migration: Sync requested pages after postcopy recovery")
      Tested-by: default avatarChristian Schoenebeck <qemu_oss@crudebyte.com>
      Signed-off-by: default avatarPeter Xu <peterx@redhat.com>
      Message-Id: <20201102153010.11979-3-peterx@redhat.com>
      Reviewed-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
      Signed-off-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
      5e773431
    • Peter Xu's avatar
      migration: Unify reset of last_rb on destination node when recover · cc5ab872
      Peter Xu authored
      
      When postcopy recover happens, we need to reset last_rb after each return of
      postcopy_pause_fault_thread() because that means we just got the postcopy
      migration continued.
      
      Unify this reset to the place right before we want to kick the fault thread
      again, when we get the command MIG_CMD_POSTCOPY_RESUME from source.
      
      This is actually more than that - because the main thread on destination will
      now be able to call migrate_send_rp_req_pages_pending() too, so the fault
      thread is not the only user of last_rb now.  Move the reset earlier will allow
      the first call to migrate_send_rp_req_pages_pending() to use the reset value
      even if called from the main thread.
      
      (NOTE: this is not a real fix to 0c26781c mentioned below, however it is just
       a mark that when picking up 0c26781c we'd better have this one too; the real
       fix will come later)
      
      Fixes: 0c26781c ("migration: Sync requested pages after postcopy recovery")
      Tested-by: default avatarChristian Schoenebeck <qemu_oss@crudebyte.com>
      Signed-off-by: default avatarPeter Xu <peterx@redhat.com>
      Reviewed-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
      Message-Id: <20201102153010.11979-2-peterx@redhat.com>
      Signed-off-by: default avatarDr. David Alan Gilbert <dgilbert@redhat.com>
      cc5ab872
    • Tomáš Golembiovský's avatar
      qga: add implementation of guest-get-disks for Windows · c67d2efd
      Tomáš Golembiovský authored
      
      The command lists all the physical disk drives. Unlike for Linux
      partitions and virtual volumes are not listed.
      
      Example output:
      
      {
        "return": [
          {
            "name": "\\\\.\\PhysicalDrive0",
            "partition": false,
            "address": {
              "serial": "QM00001",
              "bus-type": "sata",
              ...
            },
            "dependents": []
          }
        ]
      }
      
      Signed-off-by: default avatarTomáš Golembiovský <tgolembi@redhat.com>
      Signed-off-by: default avatarMichael Roth <michael.roth@amd.com>
      c67d2efd
    • Tomáš Golembiovský's avatar
      qga: add implementation of guest-get-disks for Linux · fed39564
      Tomáš Golembiovský authored
      
      The command lists all disks (real and virtual) as well as disk
      partitions. For each disk the list of dependent disks is also listed and
      /dev path is used as a handle so it can be matched with "name" field of
      other returned disk entries. For disk partitions the "dependents" list
      is populated with the the parent device for easier tracking of
      hierarchy.
      
      Example output:
      {
        "return": [
          ...
          {
            "name": "/dev/dm-0",
            "partition": false,
            "dependents": [
              "/dev/sda2"
            ],
            "alias": "luks-7062202e-5b9b-433e-81e8-6628c40da9f7"
          },
          {
            "name": "/dev/sda2",
            "partition": true,
            "dependents": [
              "/dev/sda"
            ]
          },
          {
            "name": "/dev/sda",
            "partition": false,
            "address": {
              "serial": "SAMSUNG_MZ7LN512HCHP-000L1_S1ZKNXAG822493",
              "bus-type": "sata",
              ...
              "dev": "/dev/sda",
              "target": 0
            },
            "dependents": []
          },
          ...
        ]
      }
      
      Signed-off-by: default avatarTomáš Golembiovský <tgolembi@redhat.com>
      Reviewed-by: default avatarMarc-André Lureau <marcandre.lureau@redhat.com>
      *add missing stub for !defined(CONFIG_FSFREEZE)
      *remove unused deps_dir variable
      Signed-off-by: default avatarMichael Roth <michael.roth@amd.com>
      fed39564
    • Peter Maydell's avatar
      Merge remote-tracking branch 'remotes/nvme/tags/pull-nvme-20201102' into staging · 8680d6e3
      Peter Maydell authored
      
      nvme pull 2 Nov 2020
      
      # gpg: Signature made Mon 02 Nov 2020 15:20:30 GMT
      # gpg:                using RSA key DBC11D2D373B4A3755F502EC625156610A4F6CC0
      # gpg: Good signature from "Keith Busch <kbusch@kernel.org>" [unknown]
      # gpg:                 aka "Keith Busch <keith.busch@gmail.com>" [unknown]
      # gpg:                 aka "Keith Busch <keith.busch@intel.com>" [unknown]
      # gpg: WARNING: This key is not certified with a trusted signature!
      # gpg:          There is no indication that the signature belongs to the owner.
      # Primary key fingerprint: DBC1 1D2D 373B 4A37 55F5  02EC 6251 5661 0A4F 6CC0
      
      * remotes/nvme/tags/pull-nvme-20201102: (30 commits)
        hw/block/nvme: fix queue identifer validation
        hw/block/nvme: fix create IO SQ/CQ status codes
        hw/block/nvme: fix prp mapping status codes
        hw/block/nvme: report actual LBA data shift in LBAF
        hw/block/nvme: add trace event for requests with non-zero status code
        hw/block/nvme: add nsid to get/setfeat trace events
        hw/block/nvme: reject io commands if only admin command set selected
        hw/block/nvme: support for admin-only command set
        hw/block/nvme: validate command set selected
        hw/block/nvme: support per-namespace smart log
        hw/block/nvme: fix log page offset check
        hw/block/nvme: remove pointless rw indirection
        hw/block/nvme: update nsid when registered
        hw/block/nvme: change controller pci id
        pci: allocate pci id for nvme
        hw/block/nvme: support multiple namespaces
        hw/block/nvme: refactor identify active namespace id list
        hw/block/nvme: add support for sgl bit bucket descriptor
        hw/block/nvme: add support for scatter gather lists
        hw/block/nvme: harden cmb access
        ...
      
      Signed-off-by: default avatarPeter Maydell <peter.maydell@linaro.org>
      8680d6e3
    • Peter Maydell's avatar
      tests/qtest/npcm7xx_rng-test: Disable randomness tests · ffb4fbf9
      Peter Maydell authored
      
      The randomness tests in the NPCM7xx RNG test fail intermittently
      but fairly frequently. On my machine running the test in a loop:
       while QTEST_QEMU_BINARY=./qemu-system-aarch64 ./tests/qtest/npcm7xx_rng-test; do true; done
      
      will fail in less than a minute with an error like:
      ERROR:../../tests/qtest/npcm7xx_rng-test.c:256:test_first_byte_runs:
      assertion failed (calc_runs_p(buf.l, sizeof(buf) * BITS_PER_BYTE) > 0.01): (0.00286205989 > 0.01)
      
      (Failures have been observed on all 4 of the randomness tests,
      not just first_byte_runs.)
      
      It's not clear why these tests are failing like this, but intermittent
      failures make CI and merge testing awkward, so disable running them
      unless a developer specifically sets QEMU_TEST_FLAKY_RNG_TESTS when
      running the test suite, until we work out the cause.
      
      Signed-off-by: default avatarPeter Maydell <peter.maydell@linaro.org>
      Reviewed-by: default avatarPhilippe Mathieu-Daudé <philmd@redhat.com>
      Message-id: 20201102152454.8287-1-peter.maydell@linaro.org
      Reviewed-by: default avatarHavard Skinnemoen <hskinnemoen@google.com>
      ffb4fbf9
    • Peter Maydell's avatar
      qemu-option-trace.rst.inc: Don't use option:: markup · 5a4b0d41
      Peter Maydell authored
      
      Sphinx 3.2 is pickier than earlier versions about the option:: markup,
      and complains about our usage in qemu-option-trace.rst:
      
      ../../docs/qemu-option-trace.rst.inc:4:Malformed option description
        '[enable=]PATTERN', should look like "opt", "-opt args", "--opt args",
        "/opt args" or "+opt args"
      
      In this file, we're really trying to document the different parts of
      the top-level --trace option, which qemu-nbd.rst and qemu-img.rst
      have already introduced with an option:: markup.  So it's not right
      to use option:: here anyway.  Switch to a different markup
      (definition lists) which gives about the same formatted output.
      
      (Unlike option::, this markup doesn't produce index entries; but
      at the moment we don't do anything much with indexes anyway, and
      in any case I think it doesn't make much sense to have individual
      index entries for the sub-parts of the --trace option.)
      
      Signed-off-by: default avatarPeter Maydell <peter.maydell@linaro.org>
      Reviewed-by: default avatarDaniel P. Berrangé <berrange@redhat.com>
      Tested-by: default avatarStefan Hajnoczi <stefanha@redhat.com>
      Message-id: 20201030174700.7204-3-peter.maydell@linaro.org
      5a4b0d41
    • Peter Maydell's avatar
      scripts/kerneldoc: For Sphinx 3 use c:macro for macros with arguments · 92bb29f9
      Peter Maydell authored
      
      The kerneldoc script currently emits Sphinx markup for a macro with
      arguments that uses the c:function directive. This is correct for
      Sphinx versions earlier than Sphinx 3, where c:macro doesn't allow
      documentation of macros with arguments and c:function is not picky
      about the syntax of what it is passed. However, in Sphinx 3 the
      c:macro directive was enhanced to support macros with arguments,
      and c:function was made more picky about what syntax it accepted.
      
      When kerneldoc is told that it needs to produce output for Sphinx
      3 or later, make it emit c:function only for functions and c:macro
      for macros with arguments. We assume that anything with a return
      type is a function and anything without is a macro.
      
      This fixes the Sphinx error:
      
      /home/petmay01/linaro/qemu-from-laptop/qemu/docs/../include/qom/object.h:155:Error in declarator
      If declarator-id with parameters (e.g., 'void f(int arg)'):
        Invalid C declaration: Expected identifier in nested name. [error at 25]
          DECLARE_INSTANCE_CHECKER ( InstanceType,  OBJ_NAME,  TYPENAME)
          -------------------------^
      If parenthesis in noptr-declarator (e.g., 'void (*f(int arg))(double)'):
        Error in declarator or parameters
        Invalid C declaration: Expecting "(" in parameters. [error at 39]
          DECLARE_INSTANCE_CHECKER ( InstanceType,  OBJ_NAME,  TYPENAME)
          ---------------------------------------^
      
      Signed-off-by: default avatarPeter Maydell <peter.maydell@linaro.org>
      Reviewed-by: default avatarDaniel P. Berrangé <berrange@redhat.com>
      Tested-by: default avatarStefan Hajnoczi <stefanha@redhat.com>
      Message-id: 20201030174700.7204-2-peter.maydell@linaro.org
      92bb29f9
Loading