Testing updates:

  - .travis.yml tweaks and optimisations
  - .cirrus.yml enabled for FreeBSD CI
  - docker.py clean-ups for binfmt_misc
  - more control of vm-test builds

# gpg: Signature made Mon 11 Feb 2019 13:03:14 GMT
# gpg:                using RSA key F715F7CD46F94435F4F588658E520D61289519AE
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44
#      Subkey fingerprint: F715 F7CD 46F9 4435 F4F5  8865 8E52 0D61 2895 19AE

* remotes/stsquad/tags/pull-testing-next-110219-1:
  tests/vm: Be verbose while extracting compressed images
  docs/devel/testing: Add -a option to usermod command on docker setup
  scripts/qemu.py: allow arches use KVM for their 32bit cousins
  tests/vm: expose BUILD_TARGET, TARGET_LIST and EXTRA_CONFIGURE_OPTS
  tests/vm: add --build-target option
  tests/vm: call make check directly for netbsd/freebsd/ubuntu.i386
  tests/vm: move images to $HOME/.cache/qemu-vm/images
  tests: PEP8 cleanup of docker.py, mostly white space
  tests: docker.py be even smarter with persistent binfmt_misc
  tests: make docker.py check for persistent configs
  tests: make docker.py update use configured binfmt path
  docker: add debian-buster-arm64-cross
  archive-source.sh: Clone the submodules locally
  MAINTAINERS: Add an entry for scripts/archive-source.sh
  .travis.yml: fold --disable-tcg into alternate coroutine builds
  .travis.yml: separate tools and docs into another entry
  .travis.yml: stop requesting libffi & gettext from homebrew
  .cirrus.yml: basic compile and test for FreeBSD

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Depending of the host hardware, copying and extracting VM images can
take up to few minutes. Add verbosity to avoid the user to worry about
VMs hanging.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190129175403.18017-2-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

The option -G of usermod command will remove user from other groups
not listed, i.e.: $USER will belong only to group 'docker' after
following the documentation as is.

From usermod(8) manual page:

    If the user is currently a member of a group which is not listed,
    the user will be removed from the group. This behaviour can be
    changed via the -a option, which appends the user to the current
    supplementary group list.

This patch improves the situation by adding the -a option to the
usermod command, which will just append user to the supplementary
group list.

Cc: qemu-trivial@nongnu.org
Signed-off-by: Murilo Opsfelder Araujo <muriloo@linux.ibm.com>
Message-Id: <20190207184346.6840-1-muriloo@linux.ibm.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

A lot of architectures can run their 32 bit cousins on KVM so the
kvm_available function needs to be a little less restricting when
deciding if KVM is available.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

Now the underlying basevm support passes these along we can expose
some additional variables to our Makefile to allow more customised
tweaking of the build. For example:

  make vm-build-freebsd TARGET_LIST=aarch64-softmmu \
    EXTRA_CONFIGURE_OPTS="--disable-tools --disable-docs" \
    BUILD_TARGET=check-softfloat

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>

This allows us to invoke the build with a custom target (for the VMs
that use the {target} format string specifier). Currently OpenBSD is
still hardwired due to problems running check.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>

The "make check" target calls check-qtest which has the appropriate
system binaries as dependencies so we shouldn't need to do two steps
of make invocation. Doing it in two steps was a hangover from when our
make check couldn't run tests in parallel.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>

It's easier to move around the images then, by replacing the
subdirectory with a symlink.  Allows to share the images between
multiple qemu checkouts for example.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

My editor keeps putting squiggly lines under a bunch of the python
lines to remind me how non-PEP8 compliant it is. Clean that up so it's
easier to spot new errors.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

If we have a persistent mapping we don't need the QEMU binary copied
into the container as the kernel has already opened the file and will
pass the fd in. However the support libraries will still need to be
there.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

binfmt_misc configured with the "F" flag opens the interpreter at
config time. This means it can use an already open file-descriptor to
run QEMU so there is no point trying to copy the binary into a
container.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

When copying a QEMU binary into a linux-user docker image we should
check what the current configured binfmt_misc path is rather than
just assuming "/usr/bin/qemu-bin". Obviously if the user changes the
configuration afterwards they will break their images again.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

We can't build QEMU with this but we can use this image to build newer
arm64 testcases which need more up to date tools.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

We cloned the QEMU repository from the local storage. Since the
submodules are also available there, clone them too. This is
quicker and reduce network use.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[AJB: incorporated review suggestions from danpb]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

The scripts/archive-source.sh is used by the VM tests, it makes
sense to add it in the "Build and test automation" section.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

The alternate coroutine builds are really only of interest to people
running KVM (although I think you could use them for TCG if you really
tried). As they tend to run long lets kill two birds with one stone
and fold the --disable-tcg build into them.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

Re-building the tools and documents by default is a little wasteful as
they are not really affected by the main build options. Split tools
and documents into their own task with a minimal softmmu and
linux-user target list just to check they don't interact badly.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

The default package set installed on macOS builders from Travis already
includes libffi and gettext as shown by log messages:

  Skipping install of libffi formula. It is already up-to-date.
  Using libffi
  Skipping install of gettext formula. It is already up-to-date.
  Using gettext

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

Signed-off-by: Ed Maste <emaste@freebsd.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

More work towards libslirp

Marc-André Lureau (27):
  slirp: generalize guestfwd with a callback based approach
  net/slirp: simplify checking for cmd: prefix
  net/slirp: free forwarding rules on cleanup
  net/slirp: fix leaks on forwarding rule registration error
  slirp: add callbacks for timer
  slirp: replace trace functions with DEBUG calls
  slirp: replace QEMU_PACKED with SLIRP_PACKED
  slirp: replace most qemu socket utilities with slirp own version
  slirp: replace qemu_set_nonblock()
  slirp: add unregister_poll_fd() callback
  slirp: replace qemu_notify_event() with a callback
  slirp: move QEMU state saving to a separate unit
  slirp: do not include qemu headers in libslirp.h public API header
  slirp: improve windows headers inclusion
  slirp: add slirp own version of pstrcpy
  slirp: remove qemu timer.h dependency
  slirp: remove now useless QEMU headers inclusions
  slirp: replace net/eth.h inclusion with own defines
  slirp: replace qemu qtailq with slirp own copy
  slirp: replace remaining qemu headers dependency
  slirp: prefer c99 types over BSD kind
  slirp: improve send_packet() callback
  slirp: replace global polling with per-instance & notifier
  slirp: remove slirp_instances list
  slirp: use polling callbacks, drop glib requirement
  slirp: pass opaque to all callbacks
  slirp: API is extern C

Peter Maydell (2):
  slirp: Avoid marking naturally packed structs as QEMU_PACKED
  slirp: Don't mark struct ipq or struct ipasfrag as packed

Samuel Thibault (3):
  slirp: Avoid unaligned 16bit memory access
  slirp: replace QEMU_BUILD_BUG_ON with G_STATIC_ASSERT
  slirp: Move g_spawn_async_with_fds_qemu compatibility to slirp/

# gpg: Signature made Thu 07 Feb 2019 14:02:41 GMT
# gpg:                using RSA key E61DBB15D4172BDEC97E92D9DB550E89F0FA54F3
# gpg: Good signature from "Samuel Thibault <samuel.thibault@aquilenet.fr>" [unknown]
# gpg:                 aka "Samuel Thibault <sthibault@debian.org>" [marginal]
# gpg:                 aka "Samuel Thibault <samuel.thibault@gnu.org>" [unknown]
# gpg:                 aka "Samuel Thibault <samuel.thibault@inria.fr>" [marginal]
# gpg:                 aka "Samuel Thibault <samuel.thibault@labri.fr>" [marginal]
# gpg:                 aka "Samuel Thibault <samuel.thibault@ens-lyon.org>" [marginal]
# gpg:                 aka "Samuel Thibault <samuel.thibault@u-bordeaux.fr>" [unknown]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 900C B024 B679 31D4 0F82  304B D017 8C76 7D06 9EE6
#      Subkey fingerprint: E61D BB15 D417 2BDE C97E  92D9 DB55 0E89 F0FA 54F3

* remotes/thibault/tags/samuel-thibault: (32 commits)
  slirp: API is extern C
  slirp: pass opaque to all callbacks
  slirp: use polling callbacks, drop glib requirement
  slirp: remove slirp_instances list
  slirp: replace global polling with per-instance & notifier
  slirp: improve send_packet() callback
  slirp: prefer c99 types over BSD kind
  slirp: replace remaining qemu headers dependency
  slirp: Move g_spawn_async_with_fds_qemu compatibility to slirp/
  slirp: replace QEMU_BUILD_BUG_ON with G_STATIC_ASSERT
  slirp: replace qemu qtailq with slirp own copy
  slirp: replace net/eth.h inclusion with own defines
  slirp: remove now useless QEMU headers inclusions
  slirp: remove qemu timer.h dependency
  slirp: add slirp own version of pstrcpy
  slirp: improve windows headers inclusion
  slirp: do not include qemu headers in libslirp.h public API header
  slirp: move QEMU state saving to a separate unit
  slirp: replace qemu_notify_event() with a callback
  slirp: add unregister_poll_fd() callback
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

- add new netlink type from linux v4.18 and v4.19
- fix coverity warning (CID 1390634)
- fix ioctl(SIOCGIFCONF) crash

# gpg: Signature made Thu 07 Feb 2019 13:12:53 GMT
# gpg:                using RSA key F30C38BD3F2FBE3C
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/linux-user-for-4.0-pull-request:
  linux-user: add new netlink types
  linux-user: Check sscanf return value in open_net_route()
  Fix linux-user crashes in ioctl(SIOCGIFCONF) when ifc_buf is NULL.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

qemu-sparc queue

# gpg: Signature made Wed 06 Feb 2019 21:25:18 GMT
# gpg:                using RSA key CC621AB98E82200D915CC9C45BC2C56FAE0F321F
# gpg:                issuer "mark.cave-ayland@ilande.co.uk"
# gpg: Good signature from "Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>" [full]
# Primary key fingerprint: CC62 1AB9 8E82 200D 915C  C9C4 5BC2 C56F AE0F 321F

* remotes/mcayland/tags/qemu-sparc-20190206:
  sun4m: pass initrd size to OpenBIOS via fw_cfg interface
  sun4u: add power_mem_read routine
  hw/sparc64: Create VGA device only if it has really been requested

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Trivial patches:
* Update copyright
* Fix LGPL in target/moxie
* configure portability fix
* Drop useless inclusion of "hw/i386/pc.h"
* Mark the cpu-cluster device with user_creatable = false
* tsc210x: Fix building with no verbosity

# gpg: Signature made Wed 06 Feb 2019 15:27:35 GMT
# gpg:                using RSA key F30C38BD3F2FBE3C
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/trivial-patches-pull-request:
  hw/input/tsc210x: Fix building with no verbosity
  hw/cpu/cluster: Mark the cpu-cluster device with user_creatable = false
  hw/unicore32/puv3: Drop useless inclusion of "hw/i386/pc.h"
  hw/sparc64/sun4u: Drop useless inclusion of "hw/i386/pc.h"
  configure: Avoid non-portable 'test -o/-a'
  target/moxie: Fix LGPL information in the file headers
  qemu-common.h: Update copyright string for 2019

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Pull request

# gpg: Signature made Wed 06 Feb 2019 14:57:54 GMT
# gpg:                using RSA key DAE8E10975969CE5
# gpg: Good signature from "Marc-André Lureau <marcandre.lureau@redhat.com>" [full]
# gpg:                 aka "Marc-André Lureau <marcandre.lureau@gmail.com>" [full]
# Primary key fingerprint: 87A9 BD93 3F87 C606 D276  F62D DAE8 E109 7596 9CE5

* remotes/elmarco/tags/dump-pull-request:
  dump: Set correct vaddr for ELF dump

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Make it possible to use headers easily with C++ projects.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

This is friendlier for FFI bindings.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

It would be legitimate to use libslirp without glib. Let's
add_poll/get_revents pair of callbacks to provide the same
functionality.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

Now that polling is done per-instance, we don't need a global list of
slirp instances.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

Remove hard-coded dependency on slirp in main-loop, and use a "poll"
notifier instead. The notifier is registered per slirp instance.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

Use a more descriptive name for the callback.

Reuse the SlirpWriteCb type. Wrap it to check that all data has been written.

Return a ssize_t for potential error handling and data-loss reporting.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

Replace:
- u_char -> uint8_t
- u_short -> uint16_t
- u_long -> uint32_t
- u_int -> unsigned
- caddr_t -> char *

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

Except for the migration code which is gated by WITH_QEMU, only
include our own headers, so libslirp can be built standalone.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

Only slirp actually needs it, and will need it along in libslirp.

Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>

to remove another dependency on qemu.

Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

Some of those could have been squashed earlier, but it is easier to do
it all here.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: samuel Thibault <samuel.thibault@ens-lyon.org>

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

Remove a dependency on qemu util.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

Our API usage requires Vista, set WIN32_LEAN_AND_MEAN to fix a number
of issues (winsock2.h include order for ex, which is better to include
first for legacy reasons).

While at it, group redundants #ifndef _WIN32 blocks.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>