Skip to content
Snippets Groups Projects
  • Daniel P. Berrangé's avatar
    2f7350cd
    gitlab: enable ccache for many build jobs · 2f7350cd
    Daniel P. Berrangé authored
    
    The `ccache` tool can be very effective at reducing compilation times
    when re-running pipelines with only minor changes each time. For example
    a fresh 'build-system-fedora' job will typically take 20 minutes on the
    gitlab.com shared runners. With ccache this is reduced to as little as
    6 minutes.
    
    Normally meson would auto-detect existance of ccache in $PATH and use
    it automatically, but the way we wrap meson from configure breaks this,
    as we're passing in an config file with explicitly set compiler paths.
    Thus we need to add $CCACHE_WRAPPERSPATH to the front of $PATH. For
    unknown reasons if doing this in msys though, gcc becomes unable to
    invoke 'cc1' when run from meson. For msys we thus set CC='ccache gcc'
    before invoking 'configure' instead.
    
    A second problem with msys is that cache misses are incredibly
    expensive, so enabling ccache massively slows down the build when
    the cache isn't well populated. This is suspected to be a result of
    the cost of spawning processes under the msys architecture. To deal
    with this we set CCACHE_DEPEND=1 which enables ccache's 'depend_only'
    strategy. This avoids extra spawning of the pre-processor during
    cache misses, with the downside that is it less likely ccache will
    find a cache hit after semantically benign compiler flag changes.
    This is the lesser of two evils, as otherwise we can't use ccache
    at all under msys and remain inside the job time limit.
    
    If people are finding ccache to hurt their pipelines, it can be
    disabled by setting the 'CCACHE_DISABLE=1' env variable against
    their gitlab fork CI settings.
    
    Signed-off-by: default avatarDaniel P. Berrangé <berrange@redhat.com>
    Reviewed-by: default avatarPhilippe Mathieu-Daudé <philmd@linaro.org>
    Message-Id: <20230804111054.281802-2-berrange@redhat.com>
    Signed-off-by: default avatarAlex Bennée <alex.bennee@linaro.org>
    Message-Id: <20230829161528.2707696-2-alex.bennee@linaro.org>
    2f7350cd
    History
    gitlab: enable ccache for many build jobs
    Daniel P. Berrangé authored
    
    The `ccache` tool can be very effective at reducing compilation times
    when re-running pipelines with only minor changes each time. For example
    a fresh 'build-system-fedora' job will typically take 20 minutes on the
    gitlab.com shared runners. With ccache this is reduced to as little as
    6 minutes.
    
    Normally meson would auto-detect existance of ccache in $PATH and use
    it automatically, but the way we wrap meson from configure breaks this,
    as we're passing in an config file with explicitly set compiler paths.
    Thus we need to add $CCACHE_WRAPPERSPATH to the front of $PATH. For
    unknown reasons if doing this in msys though, gcc becomes unable to
    invoke 'cc1' when run from meson. For msys we thus set CC='ccache gcc'
    before invoking 'configure' instead.
    
    A second problem with msys is that cache misses are incredibly
    expensive, so enabling ccache massively slows down the build when
    the cache isn't well populated. This is suspected to be a result of
    the cost of spawning processes under the msys architecture. To deal
    with this we set CCACHE_DEPEND=1 which enables ccache's 'depend_only'
    strategy. This avoids extra spawning of the pre-processor during
    cache misses, with the downside that is it less likely ccache will
    find a cache hit after semantically benign compiler flag changes.
    This is the lesser of two evils, as otherwise we can't use ccache
    at all under msys and remain inside the job time limit.
    
    If people are finding ccache to hurt their pipelines, it can be
    disabled by setting the 'CCACHE_DISABLE=1' env variable against
    their gitlab fork CI settings.
    
    Signed-off-by: default avatarDaniel P. Berrangé <berrange@redhat.com>
    Reviewed-by: default avatarPhilippe Mathieu-Daudé <philmd@linaro.org>
    Message-Id: <20230804111054.281802-2-berrange@redhat.com>
    Signed-off-by: default avatarAlex Bennée <alex.bennee@linaro.org>
    Message-Id: <20230829161528.2707696-2-alex.bennee@linaro.org>
crossbuild-template.yml 3.10 KiB