diff --git a/Getting-Started.md b/Getting-Started.md new file mode 100644 index 0000000000000000000000000000000000000000..0b621d98b4ab578f77273599de3f10746dd76fe5 --- /dev/null +++ b/Getting-Started.md @@ -0,0 +1,181 @@ +# How to reproduce the results + +### rev.ng installation + +You will need a copy of the *rev.ng* framework. To get *rev.ng* up an running you'll need `orchestra`, a tool which automatically compiles all the software which composes *rev.ng*, and which also manages the runs of the SPECint 2006 benchmarks. + +You can retrieve the `orchestra` repository at the url https://rev.ng/gitlab/revng-bar-2019/orchestra.git + +You'll also need a copy of the [SPECint 2006 benchmarks](https://www.spec.org/cpu2006/). + +First of all, for building *rev.ng* you need some dependencies. For a debian based system, you can easily install them with: + +``` +apt-get install --no-install-recommends \ + aufs-tools \ + build-essential \ + ca-certificates \ + cmake \ + curl \ + gawk \ + git \ + libglib2.0-dev \ + m4 \ + pkg-config \ + python \ + python-pyelftools \ + python3 \ + python3-cffi \ + python3-pygraphviz \ + python3-setuptools \ + sed \ + texinfo \ + valgrind \ + zlib1g-dev +``` + +Optionally, we can also install `git-lfs`: + +``` +curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | bash +apt-get install --no-install-recommends git-lfs +``` + +Now you can proceed with the cloning of the `orchestra` repository: + +``` +git clone git@rev.ng:revng-bar-2019/orchestra.git +``` + +Then you'll need to build *rev.ng*. To do this, first move into the `orchestra` folder, and checkout the branch which contains all the tools and configurations for running the SPEC benchmarks: + +``` +git checkout feature/spec +``` + +Now you'll need to export some environment variables, which are used to prepare the build environment: + +``` +export BINARY_COMPONENTS="" +``` + +This tells `orchestra` to compile all the needed components from scratch. + +``` +export QEMU_DEFAULT_BUILD="qemu-release" +``` + +This tells `orchestra` to use the optimized version of QEMU during the translation of the binaries. + +``` +export LIBC_DEFAULT_CONFIG="gc-o2" +``` + +This tells `orchestra` to compile the `libc` library using optimizations. + +``` +export DEFAULT_TOOLCHAINS="x86-64" +``` + +This tells `orchestra` to compile only the toolchains for the `x86-64` architecture, without building all the other toolchains. + + +At this point, you can build *rev.ng* using `orchestra` with the command: + +``` +make install-revamb +``` + +### Installing SPEC + +For what concerns the SPECint benchmark installation, you can provide, with the `SPEC_ARCHIVE_URL` environment variable, an address where to retrieve an archive with the content of the SPEC DVD. The archive should be named `spec2006_int_fp.tar.gz` (or you can specify a different archive name using the `SPEC_ARCHIVE_NAME` environment variable). + +Alternatively you can directly place the archive in the `source-archives` folder inside the `orchestra` folder. + +The archive structure should be made in such way a that, at the root level, there is a folder called `spec2006_int_fp/`, which in turn contains these files and directories (the content of the SPEC DVD): +``` +benchspec +bin +config +cshrc +Docs +Docs.txt +install_archives +install.bat +install.sh +LICENSE +LICENSE.txt +MANIFEST +README +README.txt +redistributable_sources +result +Revisions +shrc +shrc.bat +tools +uninstall.sh +version.txt +``` + +### Running the benchmarks + +At this point, we are ready to run the benchmarks. `orchestra` already provides various targets which are designed for this purpose. +For example, the command: + +``` +make toolchain/x86-64/spec/native/make-gc-o2-static +``` + +is responsible of building the binaries that we will use for benchmarking the performances of the native binaries. + +To actually run the benchmarks, you can type: + +``` +make toolchain/x86-64/spec/native/use-gc-o2-static +``` + +The results of the run will be stored inside the folder `build/toolchain/x86-64/spec/native/use-gc-o2-static/result` (the path is relative to the `orchestra` folder) in an uncompressed form. A bundle of the results obtained will also be saved in `root/x86_64-gentoo-linux-musl`. + +By default, the benchmarks will execute three runs of the `ref` size of the benchmarks (which have been used to generate the data included in the paper). You can customize the SPEC invocation using the `SPEC_FLAGS` environment variable of `orchestra` (the default value is `--iterations=3 --loose --size ref int`) + +The targets seen above are the ones used for benchmarking the native binaries, and there are other 3 couples of targets for building and running the benchmarks, which are the targets for QEMU, binaries translated with *rev.ng*, and binaries translated and to which function isolation has been applied. + +In particular, the possible targets are: + +``` +native +qemu +revamb +revambisolated +``` + +For example, if we want to run the benchmarks for the binaries translated with function isolation on, we need to run: + +``` +make toolchain/x86-64/spec/revambisolated/make-gc-o2-static +make toolchain/x86-64/spec/revambisolated/use-gc-o2-static +``` + +These targets will take care of building the binaries, translating them with *rev.ng* (and applying function isolation), and running the SPEC benchmarks using the translated binaries. + +If you need to re-do a benchmark run, you can easily remove the installed benchmarks with the `clean` target, for example the commands: + +``` +make clean-toolchain/x86-64/spec/revambisolated/make-gc-o2-static +make clean-toolchain/x86-64/spec/revambisolated/use-gc-o2-static +``` + +remove the benchmarks of the binaries translated with function isolation. + +### Configure and build with orchestra + +`orchestra` provides two useful targets to inspect the available targets and configurations. The first target is `make help-components`, which prints all the targets that `orchestra` can build. + +The `make help-variables` command instead, prints all the environment variables that can be configured and which will affect the building of the components (such as the environment used above to build QEMU in release mode instead of debug mode). + +### SPEC integration in orchestra + +The integration of the SPEC benchmarks with the *rev.ng* framework is done by the `revcc` script (which you can find in the `revamb/scripts` folder), which is in charge of invoking the compilation, lifting, function isolation and recompilation of the binaries that we want to translate using *rev.ng*. + +In the `support/spec-config` folder, you can find the files used by SPEC as config files. Taking as an example the file `x86_64-gentoo-linux-musl-revambisolated.cfg`, you can see the invocation of `revcc` with the `-O2` and `-i` parameters, which are respectively in charge of instructing *rev.ng* to apply optimizations to the lifted module, and to apply function isolation before recompilation. The remaining parameters (the ones that follows `--`), represent the compiler and the parameters that will be used for compiling the original binaries (before lifting them).