examples/2. Advanced usage - NLP-Cube local installation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# NLP-Cube local installation\n",
    "\n",
    "To be able to use NLP-Cube to the fullest (train models, export them, change the network structure, etc.) we need a clone of the NLP-Cube repository.\n",
    "\n",
    "NLP-Cube requires a number of dependencies installed. This tutorial will show each step in detail. We'll install on a fresh Ubuntu 18.04 with Python3. \n",
    "\n",
    "Assume we are working in folder ``/work/``. You can use any folder, including he ``~`` user home folder, but for clarity we'll use here a fixed folder (always with write access).\n",
    "\n",
    "Before cloning NLP-Cube, we need to setup the environment. \n",
    "\n",
    "### 1. Install system and python prerequisites\n",
    "\n",
    "System-dependent requirements:\n",
    "\n",
    "```\n",
    "sudo apt-get update && sudo apt-get install -y build-essential automake make cmake g++ wget git mercurial python3-pip\n",
    "```\n",
    "\n",
    "Python3 requirements:\n",
    "\n",
    "```\n",
    "pip3 install cython future scipy nltk requests xmltodict nose2\n",
    "```\n",
    "\n",
    "### 2. Install MKL\n",
    "\n",
    "[MKL](https://software.seek.intel.com/performance-libraries) is a suite of libraries which include fast, multi-core math processing. Using them will speed up training 2.5-3x. \n",
    "```\n",
    "sudo wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB \n",
    "sudo apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB\n",
    "sudo wget https://apt.repos.intel.com/setup/intelproducts.list -O /etc/apt/sources.list.d/intelproducts.list\n",
    "sudo apt-get update \n",
    "sudo apt-get install -y intel-mkl-64bit-2018.2-046\n",
    "```\n",
    "\n",
    "### 3. Install DyNet\n",
    "\n",
    "[DyNet](https://github.com/clab/dynet) is the neural processing framework that NLP-Cube is based on. Let's install it: (assuming we're in ``/work``)\n",
    "\n",
    "```\n",
    "mkdir dynet-base \n",
    "cd dynet-base \n",
    "git clone https://github.com/clab/dynet.git \n",
    "hg clone https://bitbucket.org/eigen/eigen -r b2e267d \n",
    "cd dynet \n",
    "mkdir build \n",
    "cd build \n",
    "cmake .. -DEIGEN3_INCLUDE_DIR=../../eigen -DPYTHON=/usr/bin/python3 -DMKL_ROOT=/opt/intel/mkl \n",
    "make -j 8\n",
    "cd python \n",
    "sudo python3 ../../setup.py build --build-dir=.. --skip-build install \n",
    "``` \n",
    "\n",
    "Note: use absolute paths for cmake for all its parameters. Also, ``make -j 8`` assumes an 8-core machine. Replace with your core count for a faster build (for debugging use ``make`` to build everything single-threaded)\n",
    "\n",
    "\\* If you have CUDA-enabled GPUs available, please follow the [tutorial on the official DyNet page](http://dynet.readthedocs.io/en/latest/python.html#installing-a-cutting-edge-and-or-gpu-version).\n",
    "\n",
    "Let's test DyNet was successfully installed. Open a python3 prompt and type:\n",
    "\n",
    "```\n",
    "import dynet\n",
    "```\n",
    "\n",
    "and you should get an output like:\n",
    "\n",
    "```\n",
    "[dynet] random seed: 885379706\n",
    "[dynet] allocating memory: 512MB\n",
    "[dynet] memory allocation done.\n",
    "```\n",
    "which means DyNet is up and running.\n",
    "\n",
    "### 4. Clone NLP-Cube\n",
    "\n",
    "```\n",
    "cd /work\n",
    "git clone https://github.com/adobe/NLP-Cube.git\n",
    "```\n",
    "\n",
    "You should now have a fully working NLP-Cube install. \n",
    "\n",
    "\n",
    "---\n",
    "\n",
    "The next tutorial shows how to [train your own models](./3.%20Advanced%20usage%20-%20Training%20a%20model%20on%20the%20UD%20Corpus.ipynb).\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}