TensorFlow™ ARM Setup Guide

This guide was created for versions: v0.1.0 - v0.9.1

Introduction

Codeplay and Arm have collaborated to bring TensorFlow™ support to Arm Mali™ via the SYCL™ and OpenCL™ open standards for heterogeneous computing. This guide describes how to build and run TensorFlow on an Arm Mali device.

The supported platform for this release is the HiKey 960 development board, running Debian 9. For other platforms, please adapt the instructions below.

Configuration Management

These instructions relate to the following versions:

Project Version
TensorFlow 1ca6b1d
ComputeCpp 1.0.0
CPU 32- or 64-bit ARM CPU
GPU Arm Mali G71 MP8

Notes

  • For older or newer versions of TensorFlow, please contact Codeplay for updated build documentation.
  • GPUs other than those listed above may work, but Codeplay does not support them at this time.

Prerequisites

Build TensorFlow for ARM Mali

The following steps have been verified on a clean installation of Ubuntu 16.04.3 64-bit.

  1. Set up the environment for the ARM architecture that you want to target:
    1. For 32-bit ARM CPUs:
      1. export TARGET_ARCH=armhf
    2. For 64-bit ARM CPUs:
      1. export TARGET_ARCH=arm64
  2. Install dependency packages:
    1. sudo dpkg --add-architecture $TARGET_ARCH
    2. echo "deb [arch=$TARGET_ARCH] http://ports.ubuntu.com/ xenial main restricted universe multiverse" | sudo tee -a /etc/apt/sources.list.d/arm.list
    3. echo "deb [arch=$TARGET_ARCH] http://ports.ubuntu.com/ xenial-updates main restricted universe multiverse" | sudo tee -a /etc/apt/sources.list.d/arm.list
    4. echo "deb [arch=$TARGET_ARCH] http://ports.ubuntu.com/ xenial-security main restricted universe multiverse" | sudo tee -a /etc/apt/sources.list.d/arm.list
    5. echo "deb [arch=$TARGET_ARCH] http://ports.ubuntu.com/ xenial-backports main restricted universe multiverse" | sudo tee -a /etc/apt/sources.list.d/arm.list
    6. sudo sed -i 's#deb  http://gb.archive.ubuntu.com/ubuntu#deb  [arch=amd64]  http://gb.archive.ubuntu.com/ubuntu#g ' /etc/apt/sources.list
    7. sudo sed -i 's#deb   http://security.ubuntu.com/ubuntu#deb   [arch=amd64]   http://security.ubuntu.com/ubuntu#g   ' /etc/apt/sources.list
    8. sudo apt-get update
    9. sudo apt-get install -y git libpython-all-dev:$TARGET_ARCH opencl-headers openjdk-8-jdk python python-numpy python-pip zlib1g-dev:$TARGET_ARCH
  3. Install toolchains:
    1. Register for an account on Codeplay's developer website: https://developer.codeplay.com/computecppce/latest/download
    2. From that Downloads page, download the following version: Ubuntu 16.04 > 64bit > computecpp-ce-1.0.0-ubuntu.16.04-64bit.tar.gz
    3. Install the toolchains for your target architecture:
      • For 32-bit ARM CPUs:
        1. Download the following ComputeCpp version: Ubuntu 14.04 > arm32 > computecpp-ce-1.0.0-ubuntu.14.04-arm32.tar.gz
        2. tar -xf ComputeCpp-CE-1.0.0-Ubuntu.16.04-64bit.tar.gz
        3. tar -xf ComputeCpp-CE-1.0.0-Ubuntu.14.04-ARM32.tar.gz
        4. cp ComputeCpp-CE-1.0.0-Ubuntu-16.04-x86_64/bin/compute++ ComputeCpp-CE-1.0.0-Ubuntu-14.04-ARM_32/bin
        5. wget https://releases.linaro.org/components/toolchain/binaries/6.3-2017.05/arm-linux-gnueabihf/gcc-linaro-6.3.1-2017.05-x86_64_arm-linux-gnueabihf.tar.xz
        6. tar -xf gcc-linaro-6.3.1-2017.05-x86_64_arm-linux-gnueabihf.tar.xz
        7. mkdir -p $HOME/gcc-linaro-6.3.1-2017.05-x86_64_arm-linux-gnueabihf/arm-linux-gnueabihf/libc/usr/include/arm-linux-gnueabihf
        8. ln -s /usr/include/arm-linux-gnueabihf/python2.7/ $HOME/gcc-linaro-6.3.1-2017.05-x86_64_arm-linux-gnueabihf/arm-linux-gnueabihf/libc/usr/include/arm-linux-gnueabihf
        9. export COMPUTECPP_TOOLKIT_PATH=$HOME/ComputeCpp-CE-1.0.0-Ubuntu-14.04-ARM_32
        10. export TF_SYCL_CROSS_TOOLCHAIN=$HOME/gcc-linaro-6.3.1-2017.05-x86_64_arm-linux-gnueabihf
        11. export TF_SYCL_CROSS_TOOLCHAIN_NAME=arm-linux-gnueabihf
        12. export CC_OPT_FLAGS="-march=armv7"
      • For 64-bit ARM CPUs:
        1. Check the version of GCC that is installed on your development board (not your development PC):
          1. gcc -v
          2. If the GCC version is earlier than 5.0, replace "Ubuntu 16.04" with Ubuntu 14.04" in all subsequent steps.
        2. Download the following ComputeCpp version: Ubuntu 16.04 > arm64 > computecpp-ce-1.0.0-ubuntu.16.04-arm64.tar.gz
        3. tar -xf ComputeCpp-CE-1.0.0-Ubuntu.16.04-64bit.tar.gz
        4. tar -xf ComputeCpp-CE-1.0.0-Ubuntu.16.04-ARM64.tar.gz
        5. cp ComputeCpp-CE-1.0.0-Ubuntu-16.04-x86_64/bin/compute++ ComputeCpp-CE-1.0.0-Ubuntu-16.04-ARM_64/bin
        6. wget https://releases.linaro.org/components/toolchain/binaries/6.3-2017.05/aarch64-linux-gnu/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu.tar.xz   
        7. tar -xf gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu.tar.xz  
        8. mkdir -p $HOME/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu/aarch64-linux-gnu/libc/usr/include/aarch64-linux-gnu   
        9. ln -s /usr/include/aarch64-linux-gnu/python2.7/ $HOME/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu/aarch64-linux-gnu/libc/usr/include/aarch64-linux-gnu/
        10. export COMPUTECPP_TOOLKIT_PATH=$HOME/ComputeCpp-CE-1.0.0-Ubuntu-16.04-ARM_64
        11. export TF_SYCL_CROSS_TOOLCHAIN=$HOME/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu
        12. export TF_SYCL_CROSS_TOOLCHAIN_NAME=aarch64-linux-gnu
        13. export TF_SYCL_BITCODE_TARGET=spirv64
        14. export CC_OPT_FLAGS="-march=armv8-a"
  4. Install Bazel:

    1. wget https://github.com/bazelbuild/bazel/releases/download/0.11.1/bazel_0.11.1-linux-x86_64.deb
    2. sudo apt install -y ./bazel_0.11.1-linux-x86_64.deb
    3. bazel version
    4. Check that the bazel version output from the above command is 0.11.1.

  5. Build TensorFlow:
    1. git clone http://github.com/codeplaysoftware/tensorflow
    2. cd tensorflow
    3. git checkout 1ca6b1d
    4. export TMPDIR=~/tensorflow_temp
    5. mkdir -p $TMPDIR
    6. export PYTHON_BIN_PATH=/usr/bin/python
    7. export USE_DEFAULT_PYTHON_LIB_PATH=1
    8. export TF_NEED_MKL=0
    9. export TF_NEED_JEMALLOC=1
    10. export TF_NEED_GCP=0
    11. export TF_NEED_HDFS=0
    12. export TF_ENABLE_XLA=0
    13. export TF_NEED_OPENCL_SYCL=1
    14. export TF_NEED_COMPUTECPP=1
    15. export TF_USE_DOUBLE_SYCL=0
    16. export TF_USE_HALF_SYCL=0
    17. export TF_NEED_CUDA=0
    18. export TF_NEED_VERBS=0
    19. export TF_NEED_MPI=0
    20. export TF_NEED_GDR=0
    21. export TF_NEED_S3=0
    22. export TF_NEED_KAFKA=0
    23. export TF_SET_ANDROID_WORKSPACE=0
    24. export TF_SYCL_BITCODE_TARGET=spirv64
      • You can alternatively set this parameter to 'spir', 'spir64, or 'spirv', depending on whether your OpenCL library supports SPIR 1.2 or SPIR-V and whether the GPU is 32- or 64-bit.
    25. ./configure
    26. bazel --output_user_root=$TMPDIR build --config=sycl_arm -c opt --verbose_failures --copt=-DNO_LOCAL_MEM --copt=-DEIGEN_DONT_VECTORIZE_SYCL --copt=-Wno-c++11-narrowing //tensorflow/tools/pip_package:build_pip_package
      1. Note that the NO_LOCAL_MEM and EIGEN_DONT_VECTORIZE_SYCL flags are optimizations for HiKey 960. If you are using a different platform, you will most likely want to remove those options.
      2. If you are using an Ubuntu 14.04 version of ComputeCpp, add "--copt=-D_GLIBCXX_USE_CXX11_ABI=0" before "//tensorflow/tools/pip_package:build_pip_package" in the above command.
    27. bazel-bin/tensorflow/tools/pip_package/build_pip_package $TMPDIR
  6. Rename the wheel file for the target architecture
    1. For 32-bit ARM CPUs:
      1. mv $TMPDIR/tensorflow-1.6.0rc0-cp27-cp27mu-linux_x86_64.whl $TMPDIR/tensorflow-1.6.0rc0-cp27-cp27mu-linux_arm.whl
    2. For 64-bit ARM CPUs:
      1. mv $TMPDIR/tensorflow-1.6.0rc0-cp27-cp27mu-linux_x86_64.whl $TMPDIR/tensorflow-1.6.0rc0-cp27-cp27mu-linux_aarch64.whl

Set Up the Development Board

  1. Install the operating system and Arm Mali driver according to Arm's instructions.
  2. Copy ComputeCpp-CE-1.0.0-Ubuntu.16.04-ARM64.tar.gz and $TMPDIR/tensorflow-1.6.0rc0-cp27-cp27mu-linux_aarch64.whl to your device e.g. using the 'scp' command.

All of the following commands should be run on the development board. Depending on how your development board's disk space has been partitioned, you may have to manage the available space carefully - the following requires at least 1.2GB free.

  1. Install dependency packages:
    1. apt-get -y install clinfo git python-pip
    2. pip install numpy==1.14.5 wheel==0.31.1 six==1.11.0 mock==2.0.0 enum34==1.1.6 scipy==0.18.1 sklearn
  2. Verify that the OpenCL installation is correct:
    1. clinfo
    2. if any errors are present, check the installation of the OpenCL driver.
      • It's important to have this step working correctly, or it is likely that you run into errors later when running TensorFlow.
      • For example, if the OpenCL driver cannot be found, ensure that LD_LIBRARY_PATH has been set correctly.
  3. Install Tensorflow:
    1. pip install tensorflow-1.6.0rc0-cp27-cp27mu-linux_aarch64.whl
  4. Set up ComputeCpp:
    1. tar -xf ComputeCpp-CE-1.0.0-Ubuntu.16.04-ARM64.tar.gz
    2. export LD_LIBRARY_PATH+=:$HOME/ComputeCpp-CE-1.0.0-Ubuntu-16.04-ARM_64/lib
    3. $HOME/ComputeCpp-CE-1.0.0-Ubuntu-16.04-ARM_64/bin/computecpp_info
    4. The output should show that the Mali-G71 OpenCL driver has been found, and that it doesn't support SPIR - that is ok.

Run Benchmarks

To verify the installation, you can execute some of the standard TensorFlow benchmarks. The example below shows how to run AlexNet:

  1. git clone http://github.com/tensorflow/benchmarks
  2. cd benchmarks
  3. git checkout f5d85aef2851881001130b28385795bc4c59fa38
  4. python scripts/tf_cnn_benchmarks/tf_cnn_benchmarks.py --num_batches=10 --local_parameter_device=sycl --device=sycl --batch_size=1 --forward_only=true --model=alexnet --data_format=NHWC
    1. You may see warnings about deprecated functions, but they can be safely ignored.