Skip to content
Snippets Groups Projects
  • Mark Kanda's avatar
    b9f88dc0
    qmp: Support for querying stats · b9f88dc0
    Mark Kanda authored
    
    Gathering statistics is important for development, for monitoring and
    for performance measurement.  There are tools such as kvm_stat that do
    this and they rely on the _user_ knowing the interesting data points
    rather than the tool (which can treat them as opaque).
    
    The commands introduced in this commit introduce QMP support for
    querying stats; the goal is to take the capabilities of these tools
    and making them available throughout the whole virtualization stack,
    so that one can observe, monitor and measure virtual machines without
    having shell access + root on the host that runs them.
    
    query-stats returns a list of all stats per target type (only VM
    and vCPU to start); future commits add extra options for specifying
    stat names, vCPU qom paths, and providers.  All these are used by the
    HMP command "info stats".  Because of the development usecases around
    statistics, a good HMP interface is important.
    
    query-stats-schemas returns a list of stats included in each target
    type, with an option for specifying the provider.  The concepts in the
    schema are based on the KVM binary stats' own introspection data, just
    translated to QAPI.
    
    There are two reasons to have a separate schema that is not tied to
    the QAPI schema.  The first is the contents of the schemas: the new
    introspection data provides different information than the QAPI data,
    namely unit of measurement, how the numbers are gathered and change
    (peak/instant/cumulative/histogram), and histogram bucket sizes.
    There's really no reason to have this kind of metadata in the QAPI
    introspection schema (except possibly for the unit of measure, but
    there's a very weak justification).
    
    Another reason is the dynamicity of the schema.  The QAPI introspection
    data is very much static; and while QOM is somewhat more dynamic,
    generally we consider that to be a bug rather than a feature these days.
    On the other hand, the statistics that are exposed by QEMU might be
    passed through from another source, such as KVM, and the disadvantages of
    manually updating the QAPI schema for outweight the benefits from vetting
    the statistics and filtering out anything that seems "too unstable".
    Running old QEMU with new kernel is a supported usecase; if old QEMU
    cannot expose statistics from a new kernel, or if a kernel developer
    needs to change QEMU before gathering new info from the new kernel,
    then that is a poor user interface.
    
    The framework provides a method to register callbacks for these QMP
    commands.  Most of the work in fact is done by the callbacks, and a
    large majority of this patch is new QAPI structs and commands.
    
    Examples (with KVM stats):
    
    - Query all VM stats:
    
    { "execute": "query-stats", "arguments" : { "target": "vm" } }
    
    { "return": [
         { "provider": "kvm",
           "stats": [
              { "name": "max_mmu_page_hash_collisions", "value": 0 },
              { "name": "max_mmu_rmap_size", "value": 0 },
              { "name": "nx_lpage_splits", "value": 148 },
              ... ] },
         { "provider": "xyz",
           "stats": [ ... ] }
    ] }
    
    - Query all vCPU stats:
    
    { "execute": "query-stats", "arguments" : { "target": "vcpu" } }
    
    { "return": [
         { "provider": "kvm",
           "qom_path": "/machine/unattached/device[0]"
           "stats": [
              { "name": "guest_mode", "value": 0 },
              { "name": "directed_yield_successful", "value": 0 },
              { "name": "directed_yield_attempted", "value": 106 },
              ... ] },
         { "provider": "kvm",
           "qom_path": "/machine/unattached/device[1]"
           "stats": [
              { "name": "guest_mode", "value": 0 },
              { "name": "directed_yield_successful", "value": 0 },
              { "name": "directed_yield_attempted", "value": 106 },
              ... ] },
    ] }
    
    - Retrieve the schemas:
    
    { "execute": "query-stats-schemas" }
    
    { "return": [
        { "provider": "kvm",
          "target": "vcpu",
          "stats": [
             { "name": "guest_mode",
               "unit": "none",
               "base": 10,
               "exponent": 0,
               "type": "instant" },
            { "name": "directed_yield_successful",
               "unit": "none",
               "base": 10,
               "exponent": 0,
               "type": "cumulative" },
            ... ]
        },
        { "provider": "kvm",
          "target": "vm",
          "stats": [
            { "name": "max_mmu_page_hash_collisions",
               "unit": "none",
               "base": 10,
               "exponent": 0,
               "type": "peak" },
            ... ]
        },
        { "provider": "xyz",
          "target": "vm",
          "stats": [ ... ]
        }
    ] }
    
    Signed-off-by: default avatarMark Kanda <mark.kanda@oracle.com>
    Reviewed-by: default avatarMarkus Armbruster <armbru@redhat.com>
    Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>
    b9f88dc0
    History
    qmp: Support for querying stats
    Mark Kanda authored
    
    Gathering statistics is important for development, for monitoring and
    for performance measurement.  There are tools such as kvm_stat that do
    this and they rely on the _user_ knowing the interesting data points
    rather than the tool (which can treat them as opaque).
    
    The commands introduced in this commit introduce QMP support for
    querying stats; the goal is to take the capabilities of these tools
    and making them available throughout the whole virtualization stack,
    so that one can observe, monitor and measure virtual machines without
    having shell access + root on the host that runs them.
    
    query-stats returns a list of all stats per target type (only VM
    and vCPU to start); future commits add extra options for specifying
    stat names, vCPU qom paths, and providers.  All these are used by the
    HMP command "info stats".  Because of the development usecases around
    statistics, a good HMP interface is important.
    
    query-stats-schemas returns a list of stats included in each target
    type, with an option for specifying the provider.  The concepts in the
    schema are based on the KVM binary stats' own introspection data, just
    translated to QAPI.
    
    There are two reasons to have a separate schema that is not tied to
    the QAPI schema.  The first is the contents of the schemas: the new
    introspection data provides different information than the QAPI data,
    namely unit of measurement, how the numbers are gathered and change
    (peak/instant/cumulative/histogram), and histogram bucket sizes.
    There's really no reason to have this kind of metadata in the QAPI
    introspection schema (except possibly for the unit of measure, but
    there's a very weak justification).
    
    Another reason is the dynamicity of the schema.  The QAPI introspection
    data is very much static; and while QOM is somewhat more dynamic,
    generally we consider that to be a bug rather than a feature these days.
    On the other hand, the statistics that are exposed by QEMU might be
    passed through from another source, such as KVM, and the disadvantages of
    manually updating the QAPI schema for outweight the benefits from vetting
    the statistics and filtering out anything that seems "too unstable".
    Running old QEMU with new kernel is a supported usecase; if old QEMU
    cannot expose statistics from a new kernel, or if a kernel developer
    needs to change QEMU before gathering new info from the new kernel,
    then that is a poor user interface.
    
    The framework provides a method to register callbacks for these QMP
    commands.  Most of the work in fact is done by the callbacks, and a
    large majority of this patch is new QAPI structs and commands.
    
    Examples (with KVM stats):
    
    - Query all VM stats:
    
    { "execute": "query-stats", "arguments" : { "target": "vm" } }
    
    { "return": [
         { "provider": "kvm",
           "stats": [
              { "name": "max_mmu_page_hash_collisions", "value": 0 },
              { "name": "max_mmu_rmap_size", "value": 0 },
              { "name": "nx_lpage_splits", "value": 148 },
              ... ] },
         { "provider": "xyz",
           "stats": [ ... ] }
    ] }
    
    - Query all vCPU stats:
    
    { "execute": "query-stats", "arguments" : { "target": "vcpu" } }
    
    { "return": [
         { "provider": "kvm",
           "qom_path": "/machine/unattached/device[0]"
           "stats": [
              { "name": "guest_mode", "value": 0 },
              { "name": "directed_yield_successful", "value": 0 },
              { "name": "directed_yield_attempted", "value": 106 },
              ... ] },
         { "provider": "kvm",
           "qom_path": "/machine/unattached/device[1]"
           "stats": [
              { "name": "guest_mode", "value": 0 },
              { "name": "directed_yield_successful", "value": 0 },
              { "name": "directed_yield_attempted", "value": 106 },
              ... ] },
    ] }
    
    - Retrieve the schemas:
    
    { "execute": "query-stats-schemas" }
    
    { "return": [
        { "provider": "kvm",
          "target": "vcpu",
          "stats": [
             { "name": "guest_mode",
               "unit": "none",
               "base": 10,
               "exponent": 0,
               "type": "instant" },
            { "name": "directed_yield_successful",
               "unit": "none",
               "base": 10,
               "exponent": 0,
               "type": "cumulative" },
            ... ]
        },
        { "provider": "kvm",
          "target": "vm",
          "stats": [
            { "name": "max_mmu_page_hash_collisions",
               "unit": "none",
               "base": 10,
               "exponent": 0,
               "type": "peak" },
            ... ]
        },
        { "provider": "xyz",
          "target": "vm",
          "stats": [ ... ]
        }
    ] }
    
    Signed-off-by: default avatarMark Kanda <mark.kanda@oracle.com>
    Reviewed-by: default avatarMarkus Armbruster <armbru@redhat.com>
    Signed-off-by: default avatarPaolo Bonzini <pbonzini@redhat.com>