Skip to content
Snippets Groups Projects
  1. Apr 21, 2022
  2. Jan 21, 2022
  3. Jul 01, 2021
  4. Jun 01, 2021
    • John Snow's avatar
      python: add Makefile for some common tasks · 6560379f
      John Snow authored
      
      Add "make venv" to create the pipenv-managed virtual environment that
      contains our explicitly pinned dependencies.
      
      Add "make check" to run the python linters [in the host execution
      environment].
      
      Add "make venv-check" which combines the above two: create/update the
      venv, then run the linters in that explicitly managed environment.
      
      Add "make develop" which canonizes the runes needed to get both the
      linting pre-requisites (the "[devel]" part), and the editable
      live-install (the "-e" part) of these python libraries.
      
      make clean: delete miscellaneous python packaging output possibly
      created by pipenv, pip, or other python packaging utilities
      
      make distclean: delete the above, the .venv, and the editable "qemu"
      package forwarder (qemu.egg-info) if there is one.
      
      Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      Reviewed-by: default avatarCleber Rosa <crosa@redhat.com>
      Tested-by: default avatarCleber Rosa <crosa@redhat.com>
      Message-id: 20210527211715.394144-29-jsnow@redhat.com
      Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      6560379f
    • John Snow's avatar
      python: add avocado-framework and tests · 31622b2a
      John Snow authored
      
      Try using avocado to manage our various tests; even though right now
      they're only invoking shell scripts and not really running any
      python-native code.
      
      Create tests/, and add shell scripts which call out to mypy, flake8,
      pylint and isort to enforce the standards in this directory.
      
      Add avocado-framework to the setup.cfg development dependencies, and add
      avocado.cfg to store some preferences for how we'd like the test output
      to look.
      
      Finally, add avocado-framework to the Pipfile environment and lock the
      new dependencies. We are using avocado >= 87.0 here to take advantage of
      some features that Cleber has helpfully added to make the test output
      here *very* friendly and easy to read for developers that might chance
      upon the output in Gitlab CI.
      
      [Note: ALL of the dependencies get updated to the most modern versions
      that exist at the time of this writing. No way around it that I have
      seen. Not ideal, but so it goes.]
      
      Provided you have the right development dependencies (mypy, flake8,
      isort, pylint, and now avocado-framework) You should be able to run
      "avocado --config avocado.cfg run tests/" from the python folder to run
      all of these linters with the correct arguments.
      
      (A forthcoming commit adds the much easier 'make check'.)
      
      Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      Reviewed-by: default avatarCleber Rosa <crosa@redhat.com>
      Tested-by: default avatarCleber Rosa <crosa@redhat.com>
      Message-id: 20210527211715.394144-28-jsnow@redhat.com
      Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      31622b2a
    • John Snow's avatar
      python: add devel package requirements to setuptools · dbe75f55
      John Snow authored
      
      setuptools doesn't have a formal understanding of development requires,
      but it has an optional feataures section. Fine; add a "devel" feature
      and add the requirements to it.
      
      To avoid duplication, we can modify pipenv to install qemu[devel]
      instead. This enables us to run invocations like "pip install -e
      .[devel]" and test the package on bleeding-edge packages beyond those
      specified in Pipfile.lock.
      
      Importantly, this also allows us to install the qemu development
      packages in a non-networked mode: `pip3 install --no-index -e .[devel]`
      will now fail if the proper development dependencies are not already
      met. This can be useful for automated build scripts where fetching
      network packages may be undesirable.
      
      Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      Reviewed-by: default avatarCleber Rosa <crosa@redhat.com>
      Message-id: 20210527211715.394144-27-jsnow@redhat.com
      Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      dbe75f55
    • John Snow's avatar
      python: Add pipenv support · 41c1d81c
      John Snow authored
      
      pipenv is a tool used for managing virtual environments with pinned,
      explicit dependencies. It is used for precisely recreating python
      virtual environments.
      
      pipenv uses two files to do this:
      
      (1) Pipfile, which is similar in purpose and scope to what setup.cfg
      lists. It specifies the requisite minimum to get a functional
      environment for using this package.
      
      (2) Pipfile.lock, which is similar in purpose to `pip freeze >
      requirements.txt`. It specifies a canonical virtual environment used for
      deployment or testing. This ensures that all users have repeatable
      results.
      
      The primary benefit of using this tool is to ensure *rock solid*
      repeatable CI results with a known set of packages. Although I endeavor
      to support as many versions as I can, the fluid nature of the Python
      toolchain often means tailoring code for fairly specific versions.
      
      Note that pipenv is *not* required to install or use this module; this is
      purely for the sake of repeatable testing by CI or developers.
      
      Here, a "blank" pipfile is added with no dependencies, but specifies
      Python 3.6 for the virtual environment.
      
      Pipfile will specify our version minimums, while Pipfile.lock specifies
      an exact loadout of packages that were known to operate correctly. This
      latter file provides the real value for easy setup of container images
      and CI environments.
      
      Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      Reviewed-by: default avatarCleber Rosa <crosa@redhat.com>
      Message-id: 20210527211715.394144-15-jsnow@redhat.com
      Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      41c1d81c
    • John Snow's avatar
      python: add MANIFEST.in · eae4e442
      John Snow authored
      
      When creating a source or binary distribution via 'python3 setup.py
      <sdist|bdist>', the VERSION and PACKAGE.rst files aren't bundled by
      default. Create a MANIFEST.in file that instructs the build tools to
      include these so that installation from these files won't fail.
      
      This is required by 'tox', as well as by the tooling needed to upload
      packages to PyPI.
      
      Exclude the 'README.rst' file -- that's intended as a guidebook to our
      source tree, not a file that needs to be distributed.
      
      Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      Reviewed-by: default avatarVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
      Reviewed-by: default avatarCleber Rosa <crosa@redhat.com>
      Message-id: 20210527211715.394144-14-jsnow@redhat.com
      Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      eae4e442
    • John Snow's avatar
      python: add directory structure README.rst files · 93128815
      John Snow authored
      
      Add short readmes to python/, python/qemu/, python/qemu/machine,
      python/qemu/qmp, and python/qemu/utils that explain the directory
      hierarchy. These readmes are visible when browsing the source on
      e.g. gitlab/github and are designed to help new developers/users quickly
      make sense of the source tree.
      
      They are not designed for inclusion in a published manual.
      
      Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      Reviewed-by: default avatarCleber Rosa <crosa@redhat.com>
      Message-id: 20210527211715.394144-13-jsnow@redhat.com
      Signed-off-by: default avatarJohn Snow <jsnow@redhat.com>
      93128815
Loading