We are seeing instability on our CI runs which has been there since
the test was introduced. I suspect it triggers more on Travis due to
their heavy load.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>