DarkNet Training

Hi, Today I would like to announce that my GitHub fork at https://github.com/sowson/darknet has a new update, the fork is an advanced port of DarkNet CNN from CUDA to OpenCL and tested on macOS with eGPU from Sonnet named Breakaway RX 570 Puck and on my GreenPC.

Let’s get started the training by the inspiration from the solution original author Joseph Redmon given the TED talk.

Interested how it works and how to rebuild? Why I spent some time overnight by changing it? Bare with me. The training has just began.

This is some very beginning work I made. After changing DarkNet to run on the macOS Sierra 10.13.3 when external GPU (eGPU) support was in the experimental stage. I bought Sonnet Breakaway RX 570 Puck and my journey began.

UPDATE: 2018-Sep-22, Prerequisite on latest macOS 10.13.6 on latest MacBook Pro

cd /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/
sudo ln -s MacOSX10.14.sdk MacOSX10.13.sdk

So how to make it works on macOS? Here is the command list updated Today.

/usr/bin/ruby -e “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)”
brew install opencv@2
brew link –force opencv@2
brew install libomp
brew install clblas
brew install clfft
brew install clrng
brew install cmake
brew install wget
mkdir github; cd github; git clone https://github.com/sowson/darknet.git
cd darknet; mkdir build; cmake -Bbuild -H.; cmake –build build; cp build/darknet .; rm -r build

But, I have also GreenPC, quite strong workstation, so I installed on separated SDD the CentOS GNU/Linux on it and to setup quickly all you need here is the commands list. Caution! It is the REBOOT command in the middle.

su –
yum install epel-release
yum upgrade
yum install wget
wget http://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/cuda-repo-rhel7-9.1.85-1.x86_64.rpm
rpm -i cuda-repo-rhel7-9.1.85-1.x86_64.rpm
yum clean all
yum install cuda
reboot
su –
yum install clinfo
clinfo
yum install cmake
yum instsall gcc-gfortran
yum install boost
wget https://github.com/clMathLibraries/clBLAS/releases/download/v2.12/clBLAS-2.12.0-Linux-x64.tar.gz
tar zxf clBLAS-2.12.0-Linux-x64.tar.gz
cd clBLAS-2.12.0-Linux-x64
cp bin/* /usr/bin/
cp lib64/* /usr/lib64/
cp -r lib64/cmake/* /usr/lib64/cmake/
cp -r lib64/pkgconfig/* /usr/lib64/pkgconfig/
cp -r include/* /usr/include/
yum install git
yum install opencv-devel
yum install openmpi-devel
exit
mkdir github; cd github; git clone https://github.com/sowson/darknet.git
cd darknet; make

The answer for the question how to play with it is on the video on the beggining. Remeber to understand recurrection you need to first understand recurrention. Thank you for reading and watching.

p ;).

6 Replies to “DarkNet Training”

    • Thanks, Imagine also Raspberry Pi and VC4CL and for example AI classifier for Home Sensors. Could be awesome!

  1. Excellent work!. I followed the Mac OS tutorial, but when i run the command “cd darknet; mkdir build; cmake -Bbuild -H.; cmake –build build; cp build/darknet .; rm -r build”. This is my result:

    — The C compiler identification is AppleClang 9.1.0.9020039
    — The CXX compiler identification is AppleClang 9.1.0.9020039
    — Check for working C compiler: /usr/bin/gcc
    — Check for working C compiler: /usr/bin/gcc — works
    — Detecting C compiler ABI info
    — Detecting C compiler ABI info – done
    — Detecting C compile features
    — Detecting C compile features – done
    — Check for working CXX compiler: /usr/bin/g++
    — Check for working CXX compiler: /usr/bin/g++ — works
    — Detecting CXX compiler ABI info
    — Detecting CXX compiler ABI info – done
    — Detecting CXX compile features
    — Detecting CXX compile features – done
    — Looking for CL_VERSION_2_2
    — Looking for CL_VERSION_2_2 – not found
    — Looking for CL_VERSION_2_1
    — Looking for CL_VERSION_2_1 – not found
    — Looking for CL_VERSION_2_0
    — Looking for CL_VERSION_2_0 – not found
    — Looking for CL_VERSION_1_2
    — Looking for CL_VERSION_1_2 – found
    — Found OpenCL: /System/Library/Frameworks/OpenCL.framework (found version “1.2”)
    — Configuring done
    — Generating done
    — Build files have been written to: /Users/ricardoramirez/workspace/yolo/sowson/darknet/build
    — Configuring done
    — Generating done
    — Build files have been written to: /Users/ricardoramirez/workspace/yolo/sowson/darknet/build
    cp: build/darknet: No such file or directory

    Thanks you very much for your work in advance!.

  2. Please, try (inside a cloned folder):

    mkdir b; cd b; cmake ..; make; cp darknet ..; cd ..; rm -r b

    Thanks!

    • Thanks for your response! This is the output of that command:

      MacBookProGE:darknet ricardoramirez$ mkdir b; cd b; cmake ..; make; cp darknet ..; cd ..; rm -r b
      — The C compiler identification is AppleClang 9.1.0.9020039
      — The CXX compiler identification is AppleClang 9.1.0.9020039
      — Check for working C compiler: /usr/bin/gcc
      — Check for working C compiler: /usr/bin/gcc — works
      — Detecting C compiler ABI info
      — Detecting C compiler ABI info – done
      — Detecting C compile features
      — Detecting C compile features – done
      — Check for working CXX compiler: /usr/bin/g++
      — Check for working CXX compiler: /usr/bin/g++ — works
      — Detecting CXX compiler ABI info
      — Detecting CXX compiler ABI info – done
      — Detecting CXX compile features
      — Detecting CXX compile features – done
      — Looking for CL_VERSION_2_2
      — Looking for CL_VERSION_2_2 – not found
      — Looking for CL_VERSION_2_1
      — Looking for CL_VERSION_2_1 – not found
      — Looking for CL_VERSION_2_0
      — Looking for CL_VERSION_2_0 – not found
      — Looking for CL_VERSION_1_2
      — Looking for CL_VERSION_1_2 – found
      — Found OpenCL: /System/Library/Frameworks/OpenCL.framework (found version “1.2”)
      — Configuring done
      — Generating done
      — Build files have been written to: /Users/ricardoramirez/workspace/yolo/sowson/darknet/b
      Scanning dependencies of target bindarknet
      [ 0%] Building C object CMakeFiles/bindarknet.dir/src/activation_layer.c.o
      [ 0%] Building C object CMakeFiles/bindarknet.dir/src/activations.c.o
      [ 1%] Building C object CMakeFiles/bindarknet.dir/examples/art.c.o
      [ 1%] Building C object CMakeFiles/bindarknet.dir/examples/attention.c.o
      [ 2%] Building C object CMakeFiles/bindarknet.dir/src/avgpool_layer.c.o
      [ 2%] Building C object CMakeFiles/bindarknet.dir/src/batchnorm_layer.c.o
      [ 2%] Building C object CMakeFiles/bindarknet.dir/src/blas.c.o
      [ 3%] Building C object CMakeFiles/bindarknet.dir/src/box.c.o
      [ 3%] Building C object CMakeFiles/bindarknet.dir/examples/captcha.c.o
      [ 4%] Building C object CMakeFiles/bindarknet.dir/examples/cifar.c.o
      [ 4%] Building C object CMakeFiles/bindarknet.dir/examples/classifier.c.o
      [ 5%] Building C object CMakeFiles/bindarknet.dir/examples/coco.c.o
      [ 5%] Building C object CMakeFiles/bindarknet.dir/src/col2im.c.o
      [ 5%] Building C object CMakeFiles/bindarknet.dir/src/compare.c.o
      [ 6%] Building C object CMakeFiles/bindarknet.dir/src/connected_layer.c.o
      [ 6%] Building C object CMakeFiles/bindarknet.dir/src/convolutional_layer.c.o
      [ 7%] Building C object CMakeFiles/bindarknet.dir/src/cost_layer.c.o
      [ 7%] Building C object CMakeFiles/bindarknet.dir/src/crnn_layer.c.o
      [ 7%] Building C object CMakeFiles/bindarknet.dir/src/crop_layer.c.o
      [ 8%] Building C object CMakeFiles/bindarknet.dir/src/data.c.o
      [ 8%] Building C object CMakeFiles/bindarknet.dir/src/deconvolutional_layer.c.o
      [ 9%] Building C object CMakeFiles/bindarknet.dir/src/demo.c.o
      [ 9%] Building C object CMakeFiles/bindarknet.dir/src/detection_layer.c.o
      [ 10%] Building C object CMakeFiles/bindarknet.dir/examples/detector.c.o
      [ 10%] Building C object CMakeFiles/bindarknet.dir/examples/dice.c.o
      [ 10%] Building C object CMakeFiles/bindarknet.dir/src/dropout_layer.c.o
      [ 11%] Building C object CMakeFiles/bindarknet.dir/src/gemm.c.o
      [ 11%] Building C object CMakeFiles/bindarknet.dir/examples/go.c.o
      [ 12%] Building C object CMakeFiles/bindarknet.dir/src/gru_layer.c.o
      [ 12%] Building C object CMakeFiles/bindarknet.dir/src/im2col.c.o
      [ 13%] Building C object CMakeFiles/bindarknet.dir/src/image.c.o
      [ 13%] Building C object CMakeFiles/bindarknet.dir/src/iseg_layer.c.o
      [ 13%] Building C object CMakeFiles/bindarknet.dir/examples/instance-segmenter.c.o
      [ 14%] Building C object CMakeFiles/bindarknet.dir/src/layer.c.o
      [ 14%] Building C object CMakeFiles/bindarknet.dir/src/list.c.o
      [ 15%] Building C object CMakeFiles/bindarknet.dir/src/local_layer.c.o
      [ 15%] Building C object CMakeFiles/bindarknet.dir/src/lstm_layer.c.o
      [ 15%] Building C object CMakeFiles/bindarknet.dir/examples/lsd.c.o
      [ 16%] Building C object CMakeFiles/bindarknet.dir/src/matrix.c.o
      [ 16%] Building C object CMakeFiles/bindarknet.dir/src/maxpool_layer.c.o
      [ 17%] Building C object CMakeFiles/bindarknet.dir/src/network.c.o
      [ 17%] Building C object CMakeFiles/bindarknet.dir/examples/nightmare.c.o
      [ 18%] Building C object CMakeFiles/bindarknet.dir/src/normalization_layer.c.o
      [ 18%] Building C object CMakeFiles/bindarknet.dir/src/option_list.c.o
      [ 18%] Building C object CMakeFiles/bindarknet.dir/src/parser.c.o
      [ 19%] Building C object CMakeFiles/bindarknet.dir/src/region_layer.c.o
      [ 19%] Building C object CMakeFiles/bindarknet.dir/examples/regressor.c.o
      [ 20%] Building C object CMakeFiles/bindarknet.dir/src/reorg_layer.c.o
      [ 20%] Building C object CMakeFiles/bindarknet.dir/examples/rnn.c.o
      [ 21%] Building C object CMakeFiles/bindarknet.dir/src/rnn_layer.c.o
      [ 21%] Building C object CMakeFiles/bindarknet.dir/examples/rnn_vid.c.o
      [ 21%] Building C object CMakeFiles/bindarknet.dir/src/route_layer.c.o
      [ 22%] Building C object CMakeFiles/bindarknet.dir/src/shortcut_layer.c.o
      [ 22%] Building C object CMakeFiles/bindarknet.dir/src/softmax_layer.c.o
      [ 23%] Building C object CMakeFiles/bindarknet.dir/examples/segmenter.c.o
      [ 23%] Building C object CMakeFiles/bindarknet.dir/examples/super.c.o
      [ 23%] Building C object CMakeFiles/bindarknet.dir/examples/swag.c.o
      [ 24%] Building C object CMakeFiles/bindarknet.dir/examples/tag.c.o
      [ 24%] Building C object CMakeFiles/bindarknet.dir/src/tree.c.o
      [ 25%] Building C object CMakeFiles/bindarknet.dir/src/utils.c.o
      [ 25%] Building C object CMakeFiles/bindarknet.dir/examples/voxel.c.o
      [ 26%] Building C object CMakeFiles/bindarknet.dir/examples/writing.c.o
      [ 26%] Building C object CMakeFiles/bindarknet.dir/examples/yolo.c.o
      [ 26%] Building C object CMakeFiles/bindarknet.dir/src/yolo_layer.c.o
      [ 27%] Building C object CMakeFiles/bindarknet.dir/src/upsample_layer.c.o
      [ 27%] Building C object CMakeFiles/bindarknet.dir/src/logistic_layer.c.o
      [ 28%] Building C object CMakeFiles/bindarknet.dir/src/l2norm_layer.c.o
      [ 28%] Building C object CMakeFiles/bindarknet.dir/src/activation_kernels.c.o
      [ 28%] Building C object CMakeFiles/bindarknet.dir/src/avgpool_layer_kernels.c.o
      [ 29%] Building C object CMakeFiles/bindarknet.dir/src/blas_kernels.c.o
      [ 29%] Building C object CMakeFiles/bindarknet.dir/src/col2im_kernels.c.o
      [ 30%] Building C object CMakeFiles/bindarknet.dir/src/convolutional_kernels.c.o
      [ 30%] Building C object CMakeFiles/bindarknet.dir/src/crop_layer_kernels.c.o
      [ 31%] Building C object CMakeFiles/bindarknet.dir/src/deconvolutional_kernels.c.o
      [ 31%] Building C object CMakeFiles/bindarknet.dir/src/dropout_layer_kernels.c.o
      [ 31%] Building C object CMakeFiles/bindarknet.dir/src/im2col_kernels.c.o
      [ 32%] Building C object CMakeFiles/bindarknet.dir/src/maxpool_layer_kernels.c.o
      [ 32%] Building C object CMakeFiles/bindarknet.dir/src/opencl.c.o
      [ 33%] Building C object CMakeFiles/bindarknet.dir/examples/darknet.c.o
      make[2]: *** No rule to make target `/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/System/Library/Frameworks/OpenCL.framework’, needed by `darknet’. Stop.
      make[1]: *** [CMakeFiles/bindarknet.dir/all] Error 2
      make: *** [all] Error 2
      cp: darknet: No such file or directory

      Thank for your help!

  3. Did you:

    sudo xcode-select -s /Applications/Xcode.app/Contents/Developer
    sudo xcodebuild -license accept

    ? Thanks :-).

Leave a Reply

Your email address will not be published. Required fields are marked *

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.