Skip to content
Snippets Groups Projects
Select Git revision
  • feature/libtcg-rebase default protected
1 result
Search by author
  • Any Author
  • Anton Anton anjo
1 author
  1. Apr 21, 2022
  3. Mar 22, 2022
    • John Snow's avatar
      python/utils: add VerboseProcessError · 062fd1da
      John Snow authored 
      
This adds an Exception that extends the Python stdlib
subprocess.CalledProcessError.

The difference is that the str() method of this exception also adds the
stdout/stderr logs. In effect, if this exception goes unhandled, Python
will print the output in a visually distinct wrapper to the terminal so
that it's easy to spot in a sea of traceback information.

Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
Reviewed-by: default avatarEric Blake <eblake@redhat.com>
Reviewed-by: default avatarHanna Reitz <hreitz@redhat.com>
Message-Id: <20220321201618.903471-3-jsnow@redhat.com>
Signed-off-by: default avatarHanna Reitz <hreitz@redhat.com>
      062fd1da
    • John Snow's avatar
      python/utils: add add_visual_margin() text decoration utility · be73231b
      John Snow authored 
      
>>> print(add_visual_margin(msg, width=72, name="Commit Message"))
┏━ Commit Message ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
┃ add_visual_margin() takes a chunk of text and wraps it in a visual
┃ container that force-wraps to a specified width. An optional title
┃ label may be given, and any of the individual glyphs used to draw the
┃ box may be replaced or specified as well.
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
Reviewed-by: default avatarEric Blake <eblake@redhat.com>
Acked-by: default avatarHanna Reitz <hreitz@redhat.com>
Message-Id: <20220321201618.903471-2-jsnow@redhat.com>
Signed-off-by: default avatarHanna Reitz <hreitz@redhat.com>
      be73231b
  5. Mar 07, 2022
  7. Feb 23, 2022
    • John Snow's avatar
      Revert "python: pin setuptools below v60.0.0" · 43a1119e
      John Snow authored 
      
This reverts commit 1e4d8b31.

Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
Message-id: 20220204221804.2047468-3-jsnow@redhat.com
Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      43a1119e
    • John Snow's avatar
      Python: add setuptools v60.0 workaround · 762c280d
      John Snow authored 
      Setuptools v60 and later include a bundled version of distutils, a
deprecated standard library scheduled for removal in future versions of
Python. Setuptools v60 is only possible to install for Python 3.7 and later.

Python has a distutils.sysconfig.get_python_lib() function that returns
'/usr/lib/pythonX.Y' on posix systems. RPM-based systems actually use
'/usr/lib64/pythonX.Y' instead, so Fedora patches stdlib distutils for
Python 3.7 and Python 3.8 to return the correct value.

Python 3.9 and later introduce a sys.platlibdir property, which returns
the correct value on RPM-based systems.

The change to a distutils package not provided by Fedora on Python 3.7
and 3.8 causes a regression in distutils.sysconfig.get_python_lib() that
ultimately causes false positives to be emitted by pylint, because it
can no longer find the system source libraries.

Many Python tools are fairly aggressive about updating setuptools
packages, and so even though this package is a fair bit newer than
Python 3.7/3.8, it's not entirely unreasonable for a given user to have
such a modern package with a fairly old Python interpreter.

Updates to Python 3.7 and Python 3.8 are being produced for Fedora which
will fix the problem on up-to-date systems. Until then, we can force the
loading of platform-provided distutils when running the pylint
test. This is the least-invasive yet most comprehensive fix.

References:
 https://github.com/pypa/setuptools/pull/2896
 https://github.com/PyCQA/pylint/issues/5704
 https://github.com/pypa/distutils/issues/110



Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
Message-id: 20220204221804.2047468-2-jsnow@redhat.com
Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      762c280d
    • John Snow's avatar
      Python: discourage direct setup.py install · 2ddaeb7b
      John Snow authored 
      
When invoking setup.py directly, the default behavior for 'install' is
to run the bdist_egg installation hook, which is ... actually deprecated
by setuptools. It doesn't seem to work quite right anymore.

By contrast, 'pip install' will invoke the bdist_wheel hook
instead. This leads to differences in behavior for the two approaches. I
advocate using pip in the documentation in this directory, but the
'setup.py' which has been used for quite a long time in the Python world
may deceptively appear to work at first glance.

Add an error message that will save a bit of time and frustration
that points the user towards using the supported installation
invocation.

Reported-by: default avatarDaniel P. Berrangé <berrange@redhat.com>
Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
Reviewed-by: default avatarBeraldo Leal <bleal@redhat.com>
Message-id: 20220207213039.2278569-1-jsnow@redhat.com
Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      2ddaeb7b
    • Daniel P. Berrangé's avatar
      python: support recording QMP session to a file · 5c66d7d8
      Daniel P. Berrangé authored 
      
When running QMP commands with very large response payloads, it is often
not easy to spot the info you want. If we can save the response to a
file then tools like 'grep' or 'jq' can be used to extract information.

For convenience of processing, we merge the QMP command and response
dictionaries together:

  {
      "arguments": {},
      "execute": "query-kvm",
      "return": {
          "enabled": false,
          "present": true
      }
  }

Example usage

  $ ./scripts/qmp/qmp-shell-wrap -l q.log -p -- ./build/qemu-system-x86_64 -display none
  Welcome to the QMP low-level shell!
  Connected
  (QEMU) query-kvm
  {
      "return": {
          "enabled": false,
          "present": true
      }
  }
  (QEMU) query-mice
  {
      "return": [
          {
              "absolute": false,
              "current": true,
              "index": 2,
              "name": "QEMU PS/2 Mouse"
          }
      ]
  }

 $ jq --slurp '. | to_entries[] | select(.value.execute == "query-kvm") |
               .value.return.enabled' < q.log
   false

Reviewed-by: default avatarPhilippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: default avatarDaniel P. Berrangé <berrange@redhat.com>
Message-id: 20220128161157.36261-3-berrange@redhat.com
Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      5c66d7d8
    • Daniel P. Berrangé's avatar
      python: introduce qmp-shell-wrap convenience tool · 43912529
      Daniel P. Berrangé authored 
      
With the current 'qmp-shell' tool developers must first spawn QEMU with
a suitable -qmp arg and then spawn qmp-shell in a separate terminal
pointing to the right socket.

With 'qmp-shell-wrap' developers can ignore QMP sockets entirely and
just pass the QEMU command and arguments they want. The program will
listen on a UNIX socket and tell QEMU to connect QMP to that.

For example, this:

 # qmp-shell-wrap -- qemu-system-x86_64 -display none

Is roughly equivalent of running:

 # qemu-system-x86_64 -display none -qmp qmp-shell-1234 &
 # qmp-shell qmp-shell-1234

Except that 'qmp-shell-wrap' switches the socket peers around so that
it is the UNIX socket server and QEMU is the socket client. This makes
QEMU reliably go away when qmp-shell-wrap exits, closing the server
socket.

Signed-off-by: default avatarDaniel P. Berrangé <berrange@redhat.com>
Message-id: 20220128161157.36261-2-berrange@redhat.com
[Edited for rebase. --js]
Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      43912529
  9. Feb 02, 2022
    • John Snow's avatar
      python/aqmp: add socket bind step to legacy.py · b0b662bb
      John Snow authored 
      
The synchronous QMP library would bind to the server address during
__init__(). The new library delays this to the accept() call, because
binding occurs inside of the call to start_[unix_]server(), which is an
async method -- so it cannot happen during __init__ anymore.

Python 3.7+ adds the ability to create the server (and thus the bind()
call) and begin the active listening in separate steps, but we don't
have that functionality in 3.6, our current minimum.

Therefore ... Add a temporary workaround that allows the synchronous
version of the client to bind the socket in advance, guaranteeing that
there will be a UNIX socket in the filesystem ready for the QEMU client
to connect to without a race condition.

(Yes, it's a bit ugly. Fixing it more nicely will have to wait until our
minimum Python version is 3.7+.)

Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
Reviewed-by: default avatarKevin Wolf <kwolf@redhat.com>
Message-id: 20220201041134.1237016-5-jsnow@redhat.com
Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      b0b662bb
    • John Snow's avatar
      python: upgrade mypy to 0.780 · 74a1505d
      John Snow authored 
      
We need a slightly newer version of mypy in order to use some features
of the asyncio server functions in the next commit.

(Note: pipenv is not really suited to upgrading individual packages; I
need to replace this tool with something better for the task. For now,
the miscellaneous updates not related to the mypy upgrade are simply
beyond my control. It's on my list to take care of soon.)

Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
Reviewed-by: default avatarKevin Wolf <kwolf@redhat.com>
Message-id: 20220201041134.1237016-4-jsnow@redhat.com
Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      74a1505d
    • John Snow's avatar
      python/machine: raise VMLaunchFailure exception from launch() · 50465f94
      John Snow authored 
      
This allows us to pack in some extra information about the failure,
which guarantees that if the caller did not *intentionally* cause a
failure (by capturing this Exception), some pretty good clues will be
printed at the bottom of the traceback information.

This will help make failures in the event of a non-negative return code
more obvious when they go unhandled; the current behavior in
_post_shutdown() is to print a warning message only in the event of
signal-based terminations (for negative return codes).

(Note: In Python, catching BaseException instead of Exception catches a
broader array of Exception events, including SystemExit and
KeyboardInterrupt. We do not want to "wrap" such exceptions as a
VMLaunchFailure, because that will 'downgrade' the exception from a
BaseException to a regular Exception. We do, however, want to perform
cleanup in either case, so catch on the broadest scope and
wrap-and-re-raise only in the more targeted scope.)

Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
Reviewed-by: default avatarHanna Reitz <hreitz@redhat.com>
Reviewed-by: default avatarKevin Wolf <kwolf@redhat.com>
Message-id: 20220201041134.1237016-3-jsnow@redhat.com
Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      50465f94
    • John Snow's avatar
      python/aqmp: Fix negotiation with pre-"oob" QEMU · fa73e6e4
      John Snow authored 
      
QEMU versions prior to the "oob" capability *also* can't accept the
"enable" keyword argument at all. Fix the handshake process with older
QEMU versions.

Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
Reviewed-by: default avatarHanna Reitz <hreitz@redhat.com>
Reviewed-by: default avatarKevin Wolf <kwolf@redhat.com>
Message-id: 20220201041134.1237016-2-jsnow@redhat.com
Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      fa73e6e4
  11. Jan 21, 2022