Telluride_Decoding_Toolbox_TF2_Demo.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Telluride Decoding Toolbox TF2 Demo.ipynb",
      "provenance": [],
      "authorship_tag": "ABX9TyNmwgCzqMpvGFC1pyVDcnEw",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/google/telluride_decoding/blob/master/Telluride_Decoding_Toolbox_TF2_Demo.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nof9Hjq_4Xfy",
        "colab_type": "text"
      },
      "source": [
        "#Telluride Decoding Toolbox, TF2 Version\n",
        "\n",
        "This colab demonstrates the basic operation of the Telluride Decoding Toolbox. The toolbox is designed for large-scale experiments, where the raw data is stored on disk because is too large to fit into memory, or you want to run multiple decoding jobs in parallel for hyperparameter searches.\n",
        "\n",
        "\n",
        "In general, there are three stages to our decoding pipeline: ingest, decode, decide. This colab demonstrates the first two.\n",
        "\n",
        "1.   Ingest the raw data, and store it on disk in TFRecord format. This is the native format for Tensorflow, and each file consists of the raw audio and brain data, one (vector) sample per time step for one continuous trial.  Multiple forms of the data can be stored in each record.  Thus you might have 64 channels of EEG data, the audio intensity, as well as perhaps the phone class and the corresponding spectrogram.  All data is sampled as the same frame rate (i.e. 100Hz).\n",
        "2.   Build a decoding model. The decoding model can take several forms, such as linear regression, CCA or several forms of DNN. When building regression models you specify which input fields (from the TFRecord data) are used as input, along with any desired temporal context, and which field is the output and to be predicted. For a backward model you might use EEG with 30 frames of post_context to predict the current (audio) intensity.  For CCA you specify two sets of input fields, which are rotated to provide the highest correlation.\n",
        "3.    Decide to which signal the subject is attending. This code implements a State Space Model (from Maryland) and a simple Markov model suggested by KUL.\n",
        "\n",
        "It uses EEG data, first published as part of version 1 of the Telluride Decoding Toolbox. This data consists of 32 trials from four simultaneous subjects listening to 4 different audio tracks. There are 64 electrodes in each trial, split across the 4 subjects. The original data is provided in Matlab format and is downloaded by the colab.\n",
        "\n",
        "This colab shows how to download and import the toolbox, download and import the original Matlab data, and then measure linear and CCA models predicting the intensity that generated the EEG data.\n",
        "This is a work in progress.  Comments and/or questions are welcome as we finalize the documention and the code. The source code (a lousy way to document the toolkit) is available on [GitHub](https://github.com/google/telluride_decoding).\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JndnmDMp66FL",
        "colab_type": "text"
      },
      "source": [
        "##### Copyright 2018 Google LLC.\n",
        "\n",
        "Licensed under the Apache License, Version 2.0 (the \"License\");"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "cellView": "both",
        "id": "nfSvqhE9c9sM",
        "colab": {}
      },
      "source": [
        "#@title Default title text\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9rsal217gYVE",
        "colab_type": "text"
      },
      "source": [
        "#Install the right software \n",
        "(telluride-decoding plus prereqs)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bp7AAD_lyT0A",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "90f13414-1180-4770-f7a4-5df686d2c975"
      },
      "source": [
        "!pip uninstall -y telluride-decoding"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[33mWARNING: Skipping telluride-decoding as it is not installed.\u001b[0m\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9IBb9hzEO_mJ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# To get test versions and install them on this machine.\n",
        "# !pip install mock pyedflib  # Need to list these since they are not on test.pypi\n",
        "# !pip install --index-url https://test.pypi.org/project telluride-decoding"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EIni1EcWKgoK",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "bc5e621c-178e-4741-9b2c-924f7c9affc7"
      },
      "source": [
        "# To install the latest released version:\n",
        "!pip install telluride-decoding"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting telluride-decoding\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/7f/56/4716ad2387ada94d11ce0b0a7c523786ff14c164a475d1cda34975898d87/telluride_decoding-2.1.1-py3-none-any.whl (112kB)\n",
            "\u001b[K     |████████████████████████████████| 112kB 2.8MB/s \n",
            "\u001b[?25hRequirement already satisfied: pandas in /usr/local/lib/python3.6/dist-packages (from telluride-decoding) (1.0.5)\n",
            "Requirement already satisfied: attrs in /usr/local/lib/python3.6/dist-packages (from telluride-decoding) (20.1.0)\n",
            "Requirement already satisfied: matplotlib in /usr/local/lib/python3.6/dist-packages (from telluride-decoding) (3.2.2)\n",
            "Requirement already satisfied: tensorflow>=2 in /usr/local/lib/python3.6/dist-packages (from telluride-decoding) (2.3.0)\n",
            "Collecting pyedflib\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/ed/00/2ab3e04b03651a82df063beb16f5cf077751d7045432067b04b265ab8760/pyEDFlib-0.1.18.tar.gz (1.1MB)\n",
            "\u001b[K     |████████████████████████████████| 1.1MB 7.9MB/s \n",
            "\u001b[?25h  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
            "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
            "    Preparing wheel metadata ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting mock\n",
            "  Downloading https://files.pythonhosted.org/packages/cd/74/d72daf8dff5b6566db857cfd088907bb0355f5dd2914c4b3ef065c790735/mock-4.0.2-py3-none-any.whl\n",
            "Requirement already satisfied: numpy>=1.13.3 in /usr/local/lib/python3.6/dist-packages (from pandas->telluride-decoding) (1.18.5)\n",
            "Requirement already satisfied: python-dateutil>=2.6.1 in /usr/local/lib/python3.6/dist-packages (from pandas->telluride-decoding) (2.8.1)\n",
            "Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas->telluride-decoding) (2018.9)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->telluride-decoding) (1.2.0)\n",
            "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->telluride-decoding) (2.4.7)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib->telluride-decoding) (0.10.0)\n",
            "Requirement already satisfied: h5py<2.11.0,>=2.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=2->telluride-decoding) (2.10.0)\n",
            "Requirement already satisfied: scipy==1.4.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=2->telluride-decoding) (1.4.1)\n",
            "Requirement already satisfied: protobuf>=3.9.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=2->telluride-decoding) (3.12.4)\n",
            "Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=2->telluride-decoding) (1.15.0)\n",
            "Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=2->telluride-decoding) (1.12.1)\n",
            "Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=2->telluride-decoding) (1.31.0)\n",
            "Requirement already satisfied: astunparse==1.6.3 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=2->telluride-decoding) (1.6.3)\n",
            "Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=2->telluride-decoding) (0.8.1)\n",
            "Requirement already satisfied: keras-preprocessing<1.2,>=1.1.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=2->telluride-decoding) (1.1.2)\n",
            "Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=2->telluride-decoding) (3.3.0)\n",
            "Requirement already satisfied: gast==0.3.3 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=2->telluride-decoding) (0.3.3)\n",
            "Requirement already satisfied: tensorflow-estimator<2.4.0,>=2.3.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=2->telluride-decoding) (2.3.0)\n",
            "Requirement already satisfied: tensorboard<3,>=2.3.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=2->telluride-decoding) (2.3.0)\n",
            "Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=2->telluride-decoding) (1.1.0)\n",
            "Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=2->telluride-decoding) (0.35.1)\n",
            "Requirement already satisfied: google-pasta>=0.1.8 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=2->telluride-decoding) (0.2.0)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from protobuf>=3.9.2->tensorflow>=2->telluride-decoding) (49.6.0)\n",
            "Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard<3,>=2.3.0->tensorflow>=2->telluride-decoding) (2.23.0)\n",
            "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard<3,>=2.3.0->tensorflow>=2->telluride-decoding) (1.0.1)\n",
            "Requirement already satisfied: google-auth<2,>=1.6.3 in /usr/local/lib/python3.6/dist-packages (from tensorboard<3,>=2.3.0->tensorflow>=2->telluride-decoding) (1.17.2)\n",
            "Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard<3,>=2.3.0->tensorflow>=2->telluride-decoding) (1.7.0)\n",
            "Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.6/dist-packages (from tensorboard<3,>=2.3.0->tensorflow>=2->telluride-decoding) (0.4.1)\n",
            "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<3,>=2.3.0->tensorflow>=2->telluride-decoding) (3.2.2)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard<3,>=2.3.0->tensorflow>=2->telluride-decoding) (2020.6.20)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard<3,>=2.3.0->tensorflow>=2->telluride-decoding) (1.24.3)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard<3,>=2.3.0->tensorflow>=2->telluride-decoding) (3.0.4)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard<3,>=2.3.0->tensorflow>=2->telluride-decoding) (2.10)\n",
            "Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard<3,>=2.3.0->tensorflow>=2->telluride-decoding) (4.1.1)\n",
            "Requirement already satisfied: rsa<5,>=3.1.4; python_version >= \"3\" in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard<3,>=2.3.0->tensorflow>=2->telluride-decoding) (4.6)\n",
            "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard<3,>=2.3.0->tensorflow>=2->telluride-decoding) (0.2.8)\n",
            "Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard<3,>=2.3.0->tensorflow>=2->telluride-decoding) (1.3.0)\n",
            "Requirement already satisfied: importlib-metadata; python_version < \"3.8\" in /usr/local/lib/python3.6/dist-packages (from markdown>=2.6.8->tensorboard<3,>=2.3.0->tensorflow>=2->telluride-decoding) (1.7.0)\n",
            "Requirement already satisfied: pyasn1>=0.1.3 in /usr/local/lib/python3.6/dist-packages (from rsa<5,>=3.1.4; python_version >= \"3\"->google-auth<2,>=1.6.3->tensorboard<3,>=2.3.0->tensorflow>=2->telluride-decoding) (0.4.8)\n",
            "Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.6/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<3,>=2.3.0->tensorflow>=2->telluride-decoding) (3.1.0)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.6/dist-packages (from importlib-metadata; python_version < \"3.8\"->markdown>=2.6.8->tensorboard<3,>=2.3.0->tensorflow>=2->telluride-decoding) (3.1.0)\n",
            "Building wheels for collected packages: pyedflib\n",
            "  Building wheel for pyedflib (PEP 517) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for pyedflib: filename=pyEDFlib-0.1.18-cp36-cp36m-linux_x86_64.whl size=603304 sha256=94ce9bf7af800930122fb580d423f4855d9208e82937549a5b7a7b1d1f9ff7e0\n",
            "  Stored in directory: /root/.cache/pip/wheels/4c/16/32/829749d6ab56f64e81e44aed0b4b9a892c975512c15cc3edbb\n",
            "Successfully built pyedflib\n",
            "Installing collected packages: pyedflib, mock, telluride-decoding\n",
            "Successfully installed mock-4.0.2 pyedflib-0.1.18 telluride-decoding-2.1.1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gUlhfog-LKLo",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from matplotlib import pyplot as plt\n",
        "import numpy as np\n",
        "import tensorflow as tf\n",
        "\n",
        "tf.compat.v1.enable_v2_behavior()"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-Phr69woKr2c",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "e303fdce-70fe-464b-c432-4ef65f560c39"
      },
      "source": [
        "from telluride_decoding import brain_data\n",
        "from telluride_decoding import brain_model\n",
        "from telluride_decoding import decoding\n",
        "from telluride_decoding import regression\n",
        "from telluride_decoding import regression_data\n",
        "\n",
        "# Reset the plot library backend to the default. \n",
        "import matplotlib\n",
        "matplotlib.use('module://ipykernel.pylab.backend_inline')\n",
        "\n",
        "# Force flags to be parsed (none here) to set flag defaults. There are some parts\n",
        "# of the toolbox that still use FLAGS, but shouldn't :-(\n",
        "from absl import flags\n",
        "flags.FLAGS(['colab'])  # Cause flags library to set the default."
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['colab']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "v3_XRfgk_wFy",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "9bf3bb1f-c21c-4bd3-dea1-d446ffd6de38"
      },
      "source": [
        "telluride_data = regression_data.RegressionDataTelluride4()\n",
        "cache_dir = '/tmp'\n",
        "telluride4_url = 'https://drive.google.com/uc?id=0ByZjGXodIlspWmpBcUhvenVQa1k'\n",
        "\n",
        "if not telluride_data.is_data_local(cache_dir):\n",
        "  print('Downloading the Telluride4 data...')\n",
        "  telluride_data.download_data(telluride4_url, cache_dir)"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading the Telluride4 data...\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YO_1NkjdLumX",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "28925ef9-e933-4fca-992c-e68f23012c0b"
      },
      "source": [
        "tf_dir = '/tmp/telluride_tf'\n",
        "!mkdir {tf_dir}\n",
        "if not telluride_data.is_data_ingested(tf_dir):\n",
        "  print('Ingesting the Telluride4 data...')\n",
        "  telluride_data.ingest_data(cache_dir, tf_dir, 100)"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Ingesting the Telluride4 data...\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wvHe9BFWRn5_",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 170
        },
        "outputId": "3153a77a-a300-441a-8663-27d7c52a5c46"
      },
      "source": [
        "!ls {tf_dir}"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "README.txt\t    trial_09.tfrecords\ttrial_18.tfrecords  trial_27.tfrecords\n",
            "trial_01.tfrecords  trial_10.tfrecords\ttrial_19.tfrecords  trial_28.tfrecords\n",
            "trial_02.tfrecords  trial_11.tfrecords\ttrial_20.tfrecords  trial_29.tfrecords\n",
            "trial_03.tfrecords  trial_12.tfrecords\ttrial_21.tfrecords  trial_30.tfrecords\n",
            "trial_04.tfrecords  trial_13.tfrecords\ttrial_22.tfrecords  trial_31.tfrecords\n",
            "trial_05.tfrecords  trial_14.tfrecords\ttrial_23.tfrecords  trial_32.tfrecords\n",
            "trial_06.tfrecords  trial_15.tfrecords\ttrial_24.tfrecords\n",
            "trial_07.tfrecords  trial_16.tfrecords\ttrial_25.tfrecords\n",
            "trial_08.tfrecords  trial_17.tfrecords\ttrial_26.tfrecords\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mPa6kvSBRzO0",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 731
        },
        "outputId": "c1bdba4b-c8c8-40d9-e5e0-d8c32c431208"
      },
      "source": [
        "!cat {tf_dir}/README.txt"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "These files were ingested from: /tmp\n",
            "Using: ['/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py', '-f', '/root/.local/share/jupyter/runtime/kernel-4355a618-0ec2-4fc9-abcf-abeb6c49ae8d.json']\n",
            "With a output frame rate of 100Hz\n",
            "\n",
            "Feature shapes are:\n",
            "\teeg: FixedLenFeature(shape=[63], dtype=tf.float32, default_value=None)\n",
            "\tones: FixedLenFeature(shape=[1], dtype=tf.float32, default_value=None)\n",
            "\tintensity: FixedLenFeature(shape=[1], dtype=tf.float32, default_value=None)\n",
            "\n",
            "All ingested files:\n",
            "\t/tmp/telluride_tf/trial_01.tfrecords: 8297 records (82.97 seconds) \n",
            "\t/tmp/telluride_tf/trial_02.tfrecords: 8024 records (80.24 seconds) \n",
            "\t/tmp/telluride_tf/trial_03.tfrecords: 8072 records (80.72 seconds) \n",
            "\t/tmp/telluride_tf/trial_04.tfrecords: 7867 records (78.67 seconds) \n",
            "\t/tmp/telluride_tf/trial_05.tfrecords: 8297 records (82.97 seconds) \n",
            "\t/tmp/telluride_tf/trial_06.tfrecords: 8024 records (80.24 seconds) \n",
            "\t/tmp/telluride_tf/trial_07.tfrecords: 8072 records (80.72 seconds) \n",
            "\t/tmp/telluride_tf/trial_08.tfrecords: 7867 records (78.67 seconds) \n",
            "\t/tmp/telluride_tf/trial_09.tfrecords: 8297 records (82.97 seconds) \n",
            "\t/tmp/telluride_tf/trial_10.tfrecords: 8024 records (80.24 seconds) \n",
            "\t/tmp/telluride_tf/trial_11.tfrecords: 8072 records (80.72 seconds) \n",
            "\t/tmp/telluride_tf/trial_12.tfrecords: 7867 records (78.67 seconds) \n",
            "\t/tmp/telluride_tf/trial_13.tfrecords: 8297 records (82.97 seconds) \n",
            "\t/tmp/telluride_tf/trial_14.tfrecords: 8024 records (80.24 seconds) \n",
            "\t/tmp/telluride_tf/trial_15.tfrecords: 8072 records (80.72 seconds) \n",
            "\t/tmp/telluride_tf/trial_16.tfrecords: 7867 records (78.67 seconds) \n",
            "\t/tmp/telluride_tf/trial_17.tfrecords: 8297 records (82.97 seconds) \n",
            "\t/tmp/telluride_tf/trial_18.tfrecords: 8024 records (80.24 seconds) \n",
            "\t/tmp/telluride_tf/trial_19.tfrecords: 8072 records (80.72 seconds) \n",
            "\t/tmp/telluride_tf/trial_20.tfrecords: 7867 records (78.67 seconds) \n",
            "\t/tmp/telluride_tf/trial_21.tfrecords: 8297 records (82.97 seconds) \n",
            "\t/tmp/telluride_tf/trial_22.tfrecords: 8024 records (80.24 seconds) \n",
            "\t/tmp/telluride_tf/trial_23.tfrecords: 8072 records (80.72 seconds) \n",
            "\t/tmp/telluride_tf/trial_24.tfrecords: 7867 records (78.67 seconds) \n",
            "\t/tmp/telluride_tf/trial_25.tfrecords: 8297 records (82.97 seconds) \n",
            "\t/tmp/telluride_tf/trial_26.tfrecords: 8024 records (80.24 seconds) \n",
            "\t/tmp/telluride_tf/trial_27.tfrecords: 8072 records (80.72 seconds) \n",
            "\t/tmp/telluride_tf/trial_28.tfrecords: 7867 records (78.67 seconds) \n",
            "\t/tmp/telluride_tf/trial_29.tfrecords: 8297 records (82.97 seconds) \n",
            "\t/tmp/telluride_tf/trial_30.tfrecords: 8024 records (80.24 seconds) \n",
            "\t/tmp/telluride_tf/trial_31.tfrecords: 8072 records (80.72 seconds) \n",
            "\t/tmp/telluride_tf/trial_32.tfrecords: 7867 records (78.67 seconds) \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BreFhALzSXP8",
        "colab_type": "text"
      },
      "source": [
        "#Decode the Telluride4 EEG Data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gGIgK7AxdpNd",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# If you have problems or get unexplained errors, you might want to turn on\n",
        "# log messages with these calls.  Change the False to True to enable the logging.\n",
        "if False:\n",
        "  from absl import logging\n",
        "  logging.set_verbosity(logging.INFO)\n",
        "  logging.set_stderrthreshold('info')\n",
        "  logging.info('Testing')"
      ],
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "t4RW8KvxR6Oi",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "telluride4_options = decoding.DecodingOptions()\n",
        "telluride4_options.input_field = 'eeg'\n",
        "telluride4_options.output_field = 'intensity'\n",
        "telluride4_options.input2_field = 'intensity'\n",
        "\n",
        "telluride4_options.tfexample_dir = tf_dir\n",
        "\n",
        "telluride4_options.dnn_regressor = 'cca'\n",
        "telluride4_options.post_context = 21\n",
        "telluride4_options.input2_pre_context = 15\n",
        "telluride4_options.input2_post_context = 15\n",
        "telluride4_options.test_metric = 'cca_pearson_correlation_first'\n",
        "telluride4_options.shuffle_buffer_size = 0  # No need when training a CCA model\n",
        "telluride4_options.cca_dimensions = 5\n"
      ],
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XzZ4iB4HpFOr",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 629
        },
        "outputId": "963c6e42-cfd0-4515-b30e-07bd3eeb75bb"
      },
      "source": [
        "print(telluride4_options.experiment_parameters('\\n'))"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "attended_field=attend\n",
            "batch_norm=False\n",
            "batch_size=512\n",
            "cca_dimensions=5\n",
            "check_file_pattern=\n",
            "correlation_frames=100\n",
            "correlation_reducer=lda\n",
            "data=tfrecords\n",
            "debug=False\n",
            "dnn_regressor=cca\n",
            "dropout=0.0\n",
            "epoch_count=100\n",
            "frame_rate=100.0\n",
            "hidden_units=20-20\n",
            "input2_field=intensity\n",
            "input2_post_context=15\n",
            "input2_pre_context=15\n",
            "input_field=eeg\n",
            "learning_rate=0.05\n",
            "loss=mse\n",
            "min_context=0\n",
            "output_field=intensity\n",
            "post_context=21\n",
            "pre_context=0\n",
            "random_mixup_batch=False\n",
            "regularization_lambda=0.1\n",
            "saved_model_dir=None\n",
            "shuffle_buffer_size=0\n",
            "summary_dir=/tmp/tf\n",
            "tensorboard_dir=None\n",
            "test_file_pattern=\n",
            "test_metric=cca_pearson_correlation_first\n",
            "tfexample_dir=/tmp/telluride_tf\n",
            "tfexample_pattern=\n",
            "train_file_pattern=\n",
            "validate_file_pattern=\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VGse-kfSpaws",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "telluride4_data = regression.get_brain_data_object(telluride4_options)"
      ],
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RjSPOeEvUZcd",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Get the actual TF dataset, as an example of the full TDT data object.\n",
        "telluride4_dataset = telluride4_data.create_dataset('train')\n",
        "\n",
        "# Create the brain model (CCA as specified above.)\n",
        "brain_model = decoding.create_brain_model(telluride4_options, telluride4_dataset)"
      ],
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YTGYNV68VIsF",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "093dc99c-eabe-4670-cbdb-9291270fe5f9"
      },
      "source": [
        "# Now train the regressor. Since this regressor is built with CCA, only one\n",
        "# pass through the data is needed to collect the statistics and create the model.\n",
        "train_results, test_results = decoding.train_and_test(telluride4_options, telluride4_data, brain_model)"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "504/504 [==============================] - 8s 15ms/step - loss: 0.1268 - cca_pearson_correlation_first: 0.1268\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BKKYXbOtr6z3",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "d50cdcb5-0bba-4894-8bda-4bf2dee4c269"
      },
      "source": [
        "test_results"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'cca_pearson_correlation_first': 0.1267634630203247,\n",
              " 'loss': 0.1267634630203247}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GpUViWh2o652",
        "colab_type": "text"
      },
      "source": [
        "#Run complete jackknife test"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MxzZWT-34ABq",
        "colab_type": "text"
      },
      "source": [
        "Run a complete jackknife (leave-one-out) test using the Telluride4 data. This takes a while to run, but gives us error bounds (really standard deviations).  Compute the test-set correlation when using CCA."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Q6YnjLzrd6RC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "telluride4_regression = regression.Telluride4CCA(telluride4_options)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HzvguAUBpQm0",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "4c7341fc-3def-4cda-a34d-eab3187066a0"
      },
      "source": [
        "reg_values = np.power(10.0, np.arange(-3, 2, 1))\n",
        "results = telluride4_regression.jackknife_over_regularizations(telluride4_options, \n",
        "                                                               reg_values)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Before creating the first dataset train, validate, and test: allbut , /tmp/telluride_tf/trial_10.tfrecords , /tmp/telluride_tf/trial_10.tfrecords\n",
            "16/16 [==============================] - 0s 27ms/step - loss: -0.0280 - cca_pearson_correlation_first: -0.0280\n",
            "15/15 [==============================] - 0s 24ms/step - loss: -0.0192 - cca_pearson_correlation_first: -0.0192\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.0431 - cca_pearson_correlation_first: 0.0431\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.0090 - cca_pearson_correlation_first: 0.0090\n",
            "16/16 [==============================] - 0s 27ms/step - loss: 0.0156 - cca_pearson_correlation_first: 0.0156\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.0345 - cca_pearson_correlation_first: 0.0345\n",
            "15/15 [==============================] - 0s 24ms/step - loss: 0.0634 - cca_pearson_correlation_first: 0.0634\n",
            "15/15 [==============================] - 0s 28ms/step - loss: -0.0136 - cca_pearson_correlation_first: -0.0136\n",
            "16/16 [==============================] - 0s 27ms/step - loss: -0.0710 - cca_pearson_correlation_first: -0.0710\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.0384 - cca_pearson_correlation_first: 0.0384\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.0059 - cca_pearson_correlation_first: 0.0059\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.0431 - cca_pearson_correlation_first: 0.0431\n",
            "16/16 [==============================] - 0s 28ms/step - loss: 0.1104 - cca_pearson_correlation_first: 0.1104\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.0018 - cca_pearson_correlation_first: 0.0018\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.0387 - cca_pearson_correlation_first: 0.0387\n",
            "15/15 [==============================] - 0s 26ms/step - loss: -0.0264 - cca_pearson_correlation_first: -0.0264\n",
            "16/16 [==============================] - 0s 25ms/step - loss: -0.1021 - cca_pearson_correlation_first: -0.1021\n",
            "15/15 [==============================] - 0s 27ms/step - loss: -8.8024e-04 - cca_pearson_correlation_first: -8.8024e-04\n",
            "15/15 [==============================] - 0s 28ms/step - loss: -0.0516 - cca_pearson_correlation_first: -0.0516\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.0877 - cca_pearson_correlation_first: 0.0877\n",
            "16/16 [==============================] - 0s 27ms/step - loss: 0.1723 - cca_pearson_correlation_first: 0.1723\n",
            "15/15 [==============================] - 0s 27ms/step - loss: -0.0237 - cca_pearson_correlation_first: -0.0237\n",
            "15/15 [==============================] - 0s 28ms/step - loss: -0.0416 - cca_pearson_correlation_first: -0.0416\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.0764 - cca_pearson_correlation_first: 0.0764\n",
            "16/16 [==============================] - 0s 26ms/step - loss: -0.0469 - cca_pearson_correlation_first: -0.0469\n",
            "15/15 [==============================] - 0s 29ms/step - loss: -0.0059 - cca_pearson_correlation_first: -0.0059\n",
            "15/15 [==============================] - 0s 28ms/step - loss: -0.0607 - cca_pearson_correlation_first: -0.0607\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.0319 - cca_pearson_correlation_first: 0.0319\n",
            "16/16 [==============================] - 0s 26ms/step - loss: 0.1411 - cca_pearson_correlation_first: 0.1411\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.0750 - cca_pearson_correlation_first: 0.0750\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.0214 - cca_pearson_correlation_first: 0.0214\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1317 - cca_pearson_correlation_first: 0.1317\n",
            "Writing summary data for one jackknife: Jackknife test result test=telluride4, regularization lambda=0.001, trial=0, mean correlation=0.02029889329787693, std=0.06304294896872671, test count=32\n",
            "Jackknife parameters:attended_field=attend,batch_norm=True,batch_size=100,cca_dimensions=5,check_file_pattern=,correlation_frames=100,correlation_reducer=lda,data=tfrecords,debug=False,dnn_regressor=cca,dropout=0.0,epoch_count=1,frame_rate=100.0,hidden_units=20-20,input2_field=intensity,input2_post_context=15,input2_pre_context=15,input_field=eeg,learning_rate=0.05,loss=mse,min_context=0,output_field=eeg,post_context=21,pre_context=0,random_mixup_batch=False,regularization_lambda=0.001,saved_model_dir=None,shuffle_buffer_size=0,summary_dir=/tmp/tf,tensorboard_dir=None,test_file_pattern=/tmp/telluride_tf/trial_10.tfrecords,test_metric=cca_pearson_correlation_first,tfexample_dir=/tmp/telluride_tf,tfexample_pattern=,train_file_pattern=allbut,validate_file_pattern=/tmp/telluride_tf/trial_10.tfrecords\n",
            "\n",
            "16/16 [==============================] - 0s 26ms/step - loss: 0.1018 - cca_pearson_correlation_first: 0.1018\n",
            "15/15 [==============================] - 0s 28ms/step - loss: 0.1139 - cca_pearson_correlation_first: 0.1139\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.1114 - cca_pearson_correlation_first: 0.1114\n",
            "15/15 [==============================] - 0s 28ms/step - loss: 0.1562 - cca_pearson_correlation_first: 0.1562\n",
            "16/16 [==============================] - 0s 23ms/step - loss: 0.0813 - cca_pearson_correlation_first: 0.0813\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1690 - cca_pearson_correlation_first: 0.1690\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.1635 - cca_pearson_correlation_first: 0.1635\n",
            "15/15 [==============================] - 0s 28ms/step - loss: 0.1254 - cca_pearson_correlation_first: 0.1254\n",
            "16/16 [==============================] - 0s 27ms/step - loss: 0.1835 - cca_pearson_correlation_first: 0.1835\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.1306 - cca_pearson_correlation_first: 0.1306\n",
            "15/15 [==============================] - 0s 28ms/step - loss: 0.1665 - cca_pearson_correlation_first: 0.1665\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.1061 - cca_pearson_correlation_first: 0.1061\n",
            "16/16 [==============================] - 0s 27ms/step - loss: 0.1015 - cca_pearson_correlation_first: 0.1015\n",
            "15/15 [==============================] - 0s 28ms/step - loss: 0.0513 - cca_pearson_correlation_first: 0.0513\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.0906 - cca_pearson_correlation_first: 0.0906\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.1356 - cca_pearson_correlation_first: 0.1356\n",
            "16/16 [==============================] - 0s 26ms/step - loss: 0.0661 - cca_pearson_correlation_first: 0.0661\n",
            "15/15 [==============================] - 0s 28ms/step - loss: 0.0168 - cca_pearson_correlation_first: 0.0168\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.1499 - cca_pearson_correlation_first: 0.1499\n",
            "15/15 [==============================] - 0s 22ms/step - loss: 0.1753 - cca_pearson_correlation_first: 0.1753\n",
            "16/16 [==============================] - 0s 25ms/step - loss: 0.0272 - cca_pearson_correlation_first: 0.0272\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.0497 - cca_pearson_correlation_first: 0.0497\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.1415 - cca_pearson_correlation_first: 0.1415\n",
            "15/15 [==============================] - 0s 24ms/step - loss: 0.1421 - cca_pearson_correlation_first: 0.1421\n",
            "16/16 [==============================] - 0s 23ms/step - loss: 0.1158 - cca_pearson_correlation_first: 0.1158\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.0117 - cca_pearson_correlation_first: 0.0117\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.1065 - cca_pearson_correlation_first: 0.1065\n",
            "15/15 [==============================] - 0s 23ms/step - loss: 0.1017 - cca_pearson_correlation_first: 0.1017\n",
            "16/16 [==============================] - 0s 24ms/step - loss: 0.1050 - cca_pearson_correlation_first: 0.1050\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.1322 - cca_pearson_correlation_first: 0.1322\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.0948 - cca_pearson_correlation_first: 0.0948\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.1408 - cca_pearson_correlation_first: 0.1408\n",
            "Writing summary data for one jackknife: Jackknife test result test=telluride4, regularization lambda=0.01, trial=0, mean correlation=0.1114157076808624, std=0.04458894398903352, test count=32\n",
            "Jackknife parameters:attended_field=attend,batch_norm=True,batch_size=100,cca_dimensions=5,check_file_pattern=,correlation_frames=100,correlation_reducer=lda,data=tfrecords,debug=False,dnn_regressor=cca,dropout=0.0,epoch_count=1,frame_rate=100.0,hidden_units=20-20,input2_field=intensity,input2_post_context=15,input2_pre_context=15,input_field=eeg,learning_rate=0.05,loss=mse,min_context=0,output_field=eeg,post_context=21,pre_context=0,random_mixup_batch=False,regularization_lambda=0.01,saved_model_dir=None,shuffle_buffer_size=0,summary_dir=/tmp/tf,tensorboard_dir=None,test_file_pattern=/tmp/telluride_tf/trial_10.tfrecords,test_metric=cca_pearson_correlation_first,tfexample_dir=/tmp/telluride_tf,tfexample_pattern=,train_file_pattern=allbut,validate_file_pattern=/tmp/telluride_tf/trial_10.tfrecords\n",
            "\n",
            "16/16 [==============================] - 0s 26ms/step - loss: 0.1044 - cca_pearson_correlation_first: 0.1044\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1231 - cca_pearson_correlation_first: 0.1231\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1167 - cca_pearson_correlation_first: 0.1167\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.1514 - cca_pearson_correlation_first: 0.1514\n",
            "16/16 [==============================] - 0s 27ms/step - loss: 0.0809 - cca_pearson_correlation_first: 0.0809\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1749 - cca_pearson_correlation_first: 0.1749\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.1628 - cca_pearson_correlation_first: 0.1628\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1283 - cca_pearson_correlation_first: 0.1283\n",
            "16/16 [==============================] - 0s 28ms/step - loss: 0.1723 - cca_pearson_correlation_first: 0.1723\n",
            "15/15 [==============================] - 0s 24ms/step - loss: 0.1220 - cca_pearson_correlation_first: 0.1220\n",
            "15/15 [==============================] - 0s 23ms/step - loss: 0.1629 - cca_pearson_correlation_first: 0.1629\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.1260 - cca_pearson_correlation_first: 0.1260\n",
            "16/16 [==============================] - 0s 24ms/step - loss: 0.1020 - cca_pearson_correlation_first: 0.1020\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.0404 - cca_pearson_correlation_first: 0.0404\n",
            "15/15 [==============================] - 0s 24ms/step - loss: 0.0968 - cca_pearson_correlation_first: 0.0968\n",
            "15/15 [==============================] - 0s 24ms/step - loss: 0.1204 - cca_pearson_correlation_first: 0.1204\n",
            "16/16 [==============================] - 0s 24ms/step - loss: 0.0605 - cca_pearson_correlation_first: 0.0605\n",
            "15/15 [==============================] - 0s 30ms/step - loss: 0.0120 - cca_pearson_correlation_first: 0.0120\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1605 - cca_pearson_correlation_first: 0.1605\n",
            "15/15 [==============================] - 0s 28ms/step - loss: 0.1846 - cca_pearson_correlation_first: 0.1846\n",
            "16/16 [==============================] - 0s 28ms/step - loss: 0.0304 - cca_pearson_correlation_first: 0.0304\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.0526 - cca_pearson_correlation_first: 0.0526\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.1395 - cca_pearson_correlation_first: 0.1395\n",
            "15/15 [==============================] - 0s 29ms/step - loss: 0.1474 - cca_pearson_correlation_first: 0.1474\n",
            "16/16 [==============================] - 0s 26ms/step - loss: 0.1029 - cca_pearson_correlation_first: 0.1029\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.0187 - cca_pearson_correlation_first: 0.0187\n",
            "15/15 [==============================] - 0s 28ms/step - loss: 0.1090 - cca_pearson_correlation_first: 0.1090\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.0966 - cca_pearson_correlation_first: 0.0966\n",
            "16/16 [==============================] - 0s 24ms/step - loss: 0.1060 - cca_pearson_correlation_first: 0.1060\n",
            "15/15 [==============================] - 0s 29ms/step - loss: 0.1345 - cca_pearson_correlation_first: 0.1345\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.0980 - cca_pearson_correlation_first: 0.0980\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1360 - cca_pearson_correlation_first: 0.1360\n",
            "Writing summary data for one jackknife: Jackknife test result test=telluride4, regularization lambda=0.1, trial=0, mean correlation=0.11169882147805765, std=0.04485586837626001, test count=32\n",
            "Jackknife parameters:attended_field=attend,batch_norm=True,batch_size=100,cca_dimensions=5,check_file_pattern=,correlation_frames=100,correlation_reducer=lda,data=tfrecords,debug=False,dnn_regressor=cca,dropout=0.0,epoch_count=1,frame_rate=100.0,hidden_units=20-20,input2_field=intensity,input2_post_context=15,input2_pre_context=15,input_field=eeg,learning_rate=0.05,loss=mse,min_context=0,output_field=eeg,post_context=21,pre_context=0,random_mixup_batch=False,regularization_lambda=0.1,saved_model_dir=None,shuffle_buffer_size=0,summary_dir=/tmp/tf,tensorboard_dir=None,test_file_pattern=/tmp/telluride_tf/trial_10.tfrecords,test_metric=cca_pearson_correlation_first,tfexample_dir=/tmp/telluride_tf,tfexample_pattern=,train_file_pattern=allbut,validate_file_pattern=/tmp/telluride_tf/trial_10.tfrecords\n",
            "\n",
            "16/16 [==============================] - 0s 25ms/step - loss: 0.0871 - cca_pearson_correlation_first: 0.0871\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.1284 - cca_pearson_correlation_first: 0.1284\n",
            "15/15 [==============================] - 0s 28ms/step - loss: 0.1238 - cca_pearson_correlation_first: 0.1238\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.1413 - cca_pearson_correlation_first: 0.1413\n",
            "16/16 [==============================] - 0s 25ms/step - loss: 0.0814 - cca_pearson_correlation_first: 0.0814\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1756 - cca_pearson_correlation_first: 0.1756\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.1488 - cca_pearson_correlation_first: 0.1488\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.1260 - cca_pearson_correlation_first: 0.1260\n",
            "16/16 [==============================] - 0s 26ms/step - loss: 0.1554 - cca_pearson_correlation_first: 0.1554\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1110 - cca_pearson_correlation_first: 0.1110\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1585 - cca_pearson_correlation_first: 0.1585\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.1304 - cca_pearson_correlation_first: 0.1304\n",
            "16/16 [==============================] - 0s 27ms/step - loss: 0.1005 - cca_pearson_correlation_first: 0.1005\n",
            "15/15 [==============================] - 0s 30ms/step - loss: 0.0200 - cca_pearson_correlation_first: 0.0200\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.1030 - cca_pearson_correlation_first: 0.1030\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1061 - cca_pearson_correlation_first: 0.1061\n",
            "16/16 [==============================] - 0s 26ms/step - loss: 0.0422 - cca_pearson_correlation_first: 0.0422\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.0013 - cca_pearson_correlation_first: 0.0013\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1681 - cca_pearson_correlation_first: 0.1681\n",
            "15/15 [==============================] - 0s 28ms/step - loss: 0.1816 - cca_pearson_correlation_first: 0.1816\n",
            "16/16 [==============================] - 0s 27ms/step - loss: 0.0144 - cca_pearson_correlation_first: 0.0144\n",
            "15/15 [==============================] - 0s 23ms/step - loss: 0.0467 - cca_pearson_correlation_first: 0.0467\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1371 - cca_pearson_correlation_first: 0.1371\n",
            "15/15 [==============================] - 0s 29ms/step - loss: 0.1536 - cca_pearson_correlation_first: 0.1536\n",
            "16/16 [==============================] - 0s 25ms/step - loss: 0.1022 - cca_pearson_correlation_first: 0.1022\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.0275 - cca_pearson_correlation_first: 0.0275\n",
            "15/15 [==============================] - 0s 28ms/step - loss: 0.1071 - cca_pearson_correlation_first: 0.1071\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1058 - cca_pearson_correlation_first: 0.1058\n",
            "16/16 [==============================] - 0s 26ms/step - loss: 0.0875 - cca_pearson_correlation_first: 0.0875\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.1323 - cca_pearson_correlation_first: 0.1323\n",
            "15/15 [==============================] - 0s 28ms/step - loss: 0.1015 - cca_pearson_correlation_first: 0.1015\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.1218 - cca_pearson_correlation_first: 0.1218\n",
            "Writing summary data for one jackknife: Jackknife test result test=telluride4, regularization lambda=1.0, trial=0, mean correlation=0.10712250381766353, std=0.04694820653993128, test count=32\n",
            "Jackknife parameters:attended_field=attend,batch_norm=True,batch_size=100,cca_dimensions=5,check_file_pattern=,correlation_frames=100,correlation_reducer=lda,data=tfrecords,debug=False,dnn_regressor=cca,dropout=0.0,epoch_count=1,frame_rate=100.0,hidden_units=20-20,input2_field=intensity,input2_post_context=15,input2_pre_context=15,input_field=eeg,learning_rate=0.05,loss=mse,min_context=0,output_field=eeg,post_context=21,pre_context=0,random_mixup_batch=False,regularization_lambda=1.0,saved_model_dir=None,shuffle_buffer_size=0,summary_dir=/tmp/tf,tensorboard_dir=None,test_file_pattern=/tmp/telluride_tf/trial_10.tfrecords,test_metric=cca_pearson_correlation_first,tfexample_dir=/tmp/telluride_tf,tfexample_pattern=,train_file_pattern=allbut,validate_file_pattern=/tmp/telluride_tf/trial_10.tfrecords\n",
            "\n",
            "16/16 [==============================] - 0s 30ms/step - loss: 0.0379 - cca_pearson_correlation_first: 0.0379\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.0783 - cca_pearson_correlation_first: 0.0783\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.0837 - cca_pearson_correlation_first: 0.0837\n",
            "15/15 [==============================] - 0s 32ms/step - loss: 0.1189 - cca_pearson_correlation_first: 0.1189\n",
            "16/16 [==============================] - 0s 23ms/step - loss: 0.1094 - cca_pearson_correlation_first: 0.1094\n",
            "15/15 [==============================] - 0s 24ms/step - loss: 0.1005 - cca_pearson_correlation_first: 0.1005\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1055 - cca_pearson_correlation_first: 0.1055\n",
            "15/15 [==============================] - 0s 23ms/step - loss: 0.0787 - cca_pearson_correlation_first: 0.0787\n",
            "16/16 [==============================] - 0s 25ms/step - loss: 0.0966 - cca_pearson_correlation_first: 0.0966\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.1039 - cca_pearson_correlation_first: 0.1039\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1368 - cca_pearson_correlation_first: 0.1368\n",
            "15/15 [==============================] - 0s 28ms/step - loss: 0.1130 - cca_pearson_correlation_first: 0.1130\n",
            "16/16 [==============================] - 0s 29ms/step - loss: 0.1174 - cca_pearson_correlation_first: 0.1174\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.0171 - cca_pearson_correlation_first: 0.0171\n",
            "15/15 [==============================] - 0s 25ms/step - loss: 0.0382 - cca_pearson_correlation_first: 0.0382\n",
            "15/15 [==============================] - 0s 28ms/step - loss: 0.1305 - cca_pearson_correlation_first: 0.1305\n",
            "16/16 [==============================] - 0s 25ms/step - loss: 0.0034 - cca_pearson_correlation_first: 0.0034\n",
            "15/15 [==============================] - 0s 28ms/step - loss: -0.0237 - cca_pearson_correlation_first: -0.0237\n",
            "15/15 [==============================] - 0s 29ms/step - loss: 0.1380 - cca_pearson_correlation_first: 0.1380\n",
            "15/15 [==============================] - 0s 28ms/step - loss: 0.1432 - cca_pearson_correlation_first: 0.1432\n",
            "16/16 [==============================] - 0s 24ms/step - loss: 0.0091 - cca_pearson_correlation_first: 0.0091\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.0255 - cca_pearson_correlation_first: 0.0255\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1301 - cca_pearson_correlation_first: 0.1301\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1214 - cca_pearson_correlation_first: 0.1214\n",
            "16/16 [==============================] - 0s 29ms/step - loss: 0.1191 - cca_pearson_correlation_first: 0.1191\n",
            "15/15 [==============================] - 0s 28ms/step - loss: -0.0209 - cca_pearson_correlation_first: -0.0209\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.0569 - cca_pearson_correlation_first: 0.0569\n",
            "15/15 [==============================] - 0s 24ms/step - loss: 0.0982 - cca_pearson_correlation_first: 0.0982\n",
            "16/16 [==============================] - 0s 25ms/step - loss: 0.0594 - cca_pearson_correlation_first: 0.0594\n",
            "15/15 [==============================] - 0s 26ms/step - loss: 0.0882 - cca_pearson_correlation_first: 0.0882\n",
            "15/15 [==============================] - 0s 27ms/step - loss: 0.1103 - cca_pearson_correlation_first: 0.1103\n",
            "15/15 [==============================] - 0s 24ms/step - loss: 0.1271 - cca_pearson_correlation_first: 0.1271\n",
            "Writing summary data for one jackknife: Jackknife test result test=telluride4, regularization lambda=10.0, trial=0, mean correlation=0.08286182796291541, std=0.04749923574835863, test count=32\n",
            "Jackknife parameters:attended_field=attend,batch_norm=True,batch_size=100,cca_dimensions=5,check_file_pattern=,correlation_frames=100,correlation_reducer=lda,data=tfrecords,debug=False,dnn_regressor=cca,dropout=0.0,epoch_count=1,frame_rate=100.0,hidden_units=20-20,input2_field=intensity,input2_post_context=15,input2_pre_context=15,input_field=eeg,learning_rate=0.05,loss=mse,min_context=0,output_field=eeg,post_context=21,pre_context=0,random_mixup_batch=False,regularization_lambda=10.0,saved_model_dir=None,shuffle_buffer_size=0,summary_dir=/tmp/tf,tensorboard_dir=None,test_file_pattern=/tmp/telluride_tf/trial_10.tfrecords,test_metric=cca_pearson_correlation_first,tfexample_dir=/tmp/telluride_tf,tfexample_pattern=,train_file_pattern=allbut,validate_file_pattern=/tmp/telluride_tf/trial_10.tfrecords\n",
            "\n",
            "[[-0.02801742 -0.01922425  0.0430541   0.00896166  0.01561401  0.03449707\n",
            "   0.06336199 -0.01358989 -0.07103132  0.03839004  0.00587622  0.04314323\n",
            "   0.1103896   0.00183432  0.03866505 -0.02636089 -0.10211705 -0.00088024\n",
            "  -0.0515963   0.08771359  0.17228761 -0.02369294 -0.04160016  0.07640848\n",
            "  -0.04687332 -0.00592373 -0.06074947  0.03194896  0.14106689  0.07495023\n",
            "   0.02139918  0.13165931]\n",
            " [ 0.1018148   0.1139183   0.11136338  0.15617758  0.08125008  0.16903512\n",
            "   0.16349329  0.12540969  0.18353295  0.13055781  0.16653495  0.10610141\n",
            "   0.10153139  0.051329    0.09061357  0.13557617  0.0660762   0.01678178\n",
            "   0.14989331  0.17526764  0.02715171  0.04974486  0.14147772  0.14207718\n",
            "   0.11584555  0.01165354  0.10654999  0.10166831  0.10503945  0.13223344\n",
            "   0.09477329  0.14082919]\n",
            " [ 0.10439828  0.12308415  0.11673553  0.15136459  0.08092128  0.17491516\n",
            "   0.16279806  0.12826082  0.17225344  0.12196419  0.16293785  0.12595978\n",
            "   0.10196687  0.0403662   0.09677465  0.1204356   0.06047959  0.01195545\n",
            "   0.16047709  0.18461795  0.03044276  0.05264908  0.13947798  0.14743456\n",
            "   0.10287723  0.01872266  0.10901244  0.09663711  0.10597445  0.13446951\n",
            "   0.09801205  0.13598591]\n",
            " [ 0.08713228  0.12839398  0.12383863  0.14129156  0.0813634   0.17562251\n",
            "   0.1487629   0.12599854  0.15535644  0.11101569  0.15851361  0.13043082\n",
            "   0.1005211   0.01997608  0.10295601  0.10608857  0.04222003  0.00128821\n",
            "   0.16807388  0.18157294  0.01442212  0.04674664  0.13712388  0.15359455\n",
            "   0.10217506  0.02747912  0.10706925  0.10575257  0.08751087  0.13232487\n",
            "   0.10152766  0.12177635]\n",
            " [ 0.03794347  0.07828502  0.0836713   0.11888757  0.1093551   0.10054632\n",
            "   0.1055242   0.07872117  0.09658462  0.10392952  0.13675012  0.11296192\n",
            "   0.11742792  0.01707339  0.03823322  0.13052513  0.00336007 -0.0236564\n",
            "   0.13801527  0.14323953  0.00906106  0.02551937  0.13013487  0.12135704\n",
            "   0.11905231 -0.02094707  0.05693844  0.09815223  0.05936264  0.08818185\n",
            "   0.11029545  0.12709187]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-XrfAZ7srNT8",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        },
        "outputId": "7f2985aa-d6aa-4ad1-b8c0-3f0e578c83d6"
      },
      "source": [
        "results"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "OrderedDict([(0.001, (0.02029889329787693, 0.06304294896872671)),\n",
              "             (0.01, (0.1114157076808624, 0.04458894398903352)),\n",
              "             (0.1, (0.11169882147805765, 0.04485586837626001)),\n",
              "             (1.0, (0.10712250381766353, 0.04694820653993128)),\n",
              "             (10.0, (0.08286182796291541, 0.04749923574835863))])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NXmhkz2wf9T_",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 283
        },
        "outputId": "4fec931a-0c62-4ad2-e0ef-4d4e0357cb4e"
      },
      "source": [
        "reg_values = list(results.keys())\n",
        "result_means = [results[k][0] for k in reg_values] \n",
        "result_stddev = [results[k][1] for k in reg_values] \n",
        "plt.errorbar(reg_values, result_means, result_stddev)\n",
        "matplotlib.pyplot.xscale('log')\n",
        "plt.xlabel('Regularization Value')\n",
        "plt.ylabel('Jackknifed Correlation');"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAEKCAYAAADenhiQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZwU9Z3/8debGU7lBlEBBZVEwQNlJGoSTcRzN4qJGjFs1AQ1JnE3WXf3F92sGonJxk2y2SS6KvEC4xk1kSgRb5ONF8PhMSiCiBEUGAEBgRGG+fz+qBps256hh+menuP9fDz6MV3fqvrWp8txPtS3vl0fRQRmZmaF1KnUAZiZWfvj5GJmZgXn5GJmZgXn5GJmZgXn5GJmZgXn5GJmZgVXXuoAWoMBAwbEsGHDSh2GmVmbMnv27HcjYmCudU4uwLBhw6isrCx1GGZmbYqkNxta52ExMzMrOCcXMzMrOCcXMzMrOCcXMzMrOCcXMzMrOCcXMzMrOCcXMzMrOCcXs1bqjOuf4Yzrnyl1GGY7xMnFWoz/WJp1HE4uZmZWcE4uZtZu+Oq49ShpcpF0gqQFkhZJujjH+iMlzZFUK+m0rHVbJc1LX9Mz2odLei7t8y5JXVris5iZ2YdKllwklQHXACcCI4EzJY3M2uxvwDnA7Tm62BQRo9PXyRntVwG/iIh9gDXApIIHb2ZmjSrllctYYFFELI6IzcCdwPjMDSJiSUS8CNTl06EkAUcD96RNU4FTCheymZnlo5TJZTDwVsby0rQtX90kVUp6VlJ9AukPvBcRtdvrU9L56f6V1dXVTY3dzMwa0ZbruewZEcsk7QU8LuklYG2+O0fEFGAKQEVFRRQpRjOzDqmUVy7LgKEZy0PStrxExLL052LgSeBgYBXQR1J90mxSn2ZmVhilTC6zgBHp7K4uwARg+nb2AUBSX0ld0/cDgE8D8yMigCeA+pllZwP3FzxyMzNrVMmSS3pf5EJgJvAKcHdEVEmaLOlkAEmHSloKnA5cL6kq3X0/oFLSCyTJ5CcRMT9d9z3gIkmLSO7B3Nhyn8rMzKDE91wiYgYwI6vtsoz3s0iGtrL3exo4oIE+F5PMRDMzsxLxN/TNzKzgnFzMzKzgnFzMzKzgnFzMzKzgnFzMzKzgnFzMzKzgnFzMzKzgnFzMzKzgnFzMzKzgnFzMzKzgnFzMzKzgnFzMzKzgnFzMzKzgnFzMzDqoM65/hjOuf6YofTu5mJlZwZU0uUg6QdICSYskXZxj/ZGS5kiqlXRaRvtoSc9IqpL0oqQzMtbdIukNSfPS1+iW+jxmZpYoWbEwSWXANcCxwFJglqTpGRUlAf4GnAP8a9buG4GzImKhpN2B2ZJmRsR76fp/i4h7ivsJzMysIaWsRDkWWJRWjkTSncB4YFtyiYgl6bq6zB0j4rWM929LWgkMBN7DzMxKrpTDYoOBtzKWl6ZtTSJpLNAFeD2j+UfpcNkvJHVtXphmZtZUbfqGvqTdgFuBr0VE/dXNJcC+wKFAP+B7Dex7vqRKSZXV1dUtEq+ZWUdRyuSyDBiasTwkbcuLpF7Ag8D3I+LZ+vaIeCcSHwA3kwy/fUxETImIioioGDhw4A59ADMzy62UyWUWMELScEldgAnA9Hx2TLf/PTAt+8Z9ejWDJAGnAC8XNGozM9uukiWXiKgFLgRmAq8Ad0dElaTJkk4GkHSopKXA6cD1kqrS3b8MHAmck2PK8W2SXgJeAgYAV7bgxzIzM0o7W4yImAHMyGq7LOP9LJLhsuz9fgv8toE+jy5wmGZm1kRt+oa+mZm1Tk4uZmZWcE4uZmZWcE4uZmZWcE4uZmZWcE4uZmZWcE4uZmZWcCX9not1LJtr61i5voYfz3iFiCACAtKfyTKQrKP+/Yfr6relfjnHuiBZiIx+tvWb2Xfmvo0do5F+PhZfVj98rN/IiPPDD5h9DurXv7lqAwCnX/c0vbp1pme3cnp260yv7snPbcvZP7uX071zGclDKsxKw8nFWsTGzbUsWLGejZu3Mu2ZJQghgQBJbPszmNmWtT75W5nZzsf6ob5d9d3lOE6Ofuu3I7s9e116gA/3+3g/2/rvBKJT7n4yPnNDn3P52k0EUNZJLF9Xw8KVtayr2cL6mlq21m1LTzmVdVKafMo/mpjS9722k6h6diunW+eyJv5XNvuQk4sVXV1d8M93zWPj5q18YtDOPPzPR5U6pDahvrb5necf/pH2iGDTlq2sr6ll3aYtrKupZX3Nhz/X1y9vylyu5a3VG5N9arbw/ge1267EGtKlrNNHks+HSeqjiSrziiozkfXsVk7nMo+8d1TbTS6SBgLnAcMyt4+IrxcvLGtPfvrwAmZWrWCPfj3o26NLqcNp8yTRo0s5PbqUM6hXtx3qo64ueH9zbQOJKElU6zIS07pNW1hfs4UV62q2tW/cvHW7x+neuSxJTN0/mnR6bbtKyriC6tr549t2LadTJw/vtUX5XLncD/wFeBTY/m+TWYZ7Zi/l2idf58yxe/D6yvWlDsdSnTop/QPfGei+Q33Ubq3j/Q9qWbcpMxE1cAX1QbK8duNmlq7euG2bD2rrtnucnl3LP5qYujd8BfXexi307OYBmdYgn/8KPSIiZ8Ets8Y8/8ZqLrnvRY7Yuz+Tx4/iH254rtQhWQGVl3WiT48u9GnG1egHtVu3XR1lX0Gty0pUydVTLSvX1/B69YfLtVn3nwR87ebnOX7UrhwzchADdnYx2lLIJ7k8IOnv0icYm+XlzVUb+MatlQzt24NrJ47x2Lvl1LW8jK47l+1wAogIarbUpVdMW7jw9rm8t3EzC1e+zxMLXqLT71+iYs9+HDdqEMeP2pWh/XoU+BNYQ/JJLt8B/l3SZmBL2hYR0at4YVlbtq5mC5OmVlIXcOM5h9K7R+dSh2TtlCS6dymje5cydunVjd7dO9O7e2fuPP8w5r+zjplVK3i4ajlXPvgKVz74CiN368Xxo3bl+P0H8clBPT1du4i2m1wiomdLBGLtQ+3WOr592xyWvLuBaZPGMnzATqUOyTogSYzavTejdu/NRcd+gjdXbeDhqhU8VLWc/3nsNX7x6Gvs2b9HkmhGDeLgoX09caDA8rrzlVaGPDJdfDIiHijEwSWdAPwSKANuiIifZK0/Evgf4EBgQmZJY0lnA/+RLl4ZEVPT9jHALSR3KWcA34nY3qRLK5TJD8znLwvf5SdfOoAj9h5Q6nDMANiz/06cd+RenHfkXqxcX8Mj81cws2oFN//1Dab8eTEDe3bl2JHJ0Nnhe/WnS7mHcZsrn6nIPwEOBW5Lm74j6dMRcUlzDiypDLgGOBZYCsySND0i5mds9jfgHOBfs/btB1wOVJB8mXl2uu8a4FqSqdPPkSSXE4A/NSdWy8/Up5cw7Zk3Oe+zw5kwdo9Sh2OW0y49uzHxU3sy8VN7sq5mC0+8upKHq1bwh7nLuP25v9GzWzlH77sLx4/alaM+MZCdunr22Y7I56z9HTA6IuoAJE0F5gLNSi7AWGBRRCxO+70TGA9sSy4RsSRdlz1f8XjgkYhYna5/BDhB0pNAr4h4Nm2fBpyCk0vRPfVaNVf8sYpj9tuFi0/cr9ThmOWlV7fOjB89mPGjB1OzZSt/XfQuM6uW8+grK7l/3tt0Ke/EZ/cZsG3mWb+d/D2tfOWbkvsAq9P3vQt07MHAWxnLS4FPNWPfwelraY52K6KFK9Zz4W1z+MSgnvxywsGUeeza2qBuncsYt98gxu03iNqtdVS+uYaZVct5uGoFj726kk73waHD+qUTAnZlcJ8d+35QR5FPcvlPYK6kJ0imkB8JXFzUqFqApPOB8wH22MNDODtq1fsf8PWps+jauYwbzznUQwjWLpSXdeKwvfpz2F79uewLI6l6ex0zq5Yzs2o5kx+Yz+QH5rP/4F4cPzJJNCN22dkzz7LkM1vsjnS46dC06XsRsbwAx14GDM1YHpK25bvv57L2fTJtH5JPnxExBZgCUFFR4Rv+O+CD2q1c8NvZrFj3AXedf5j/JWftkiT2H9yb/Qf35l+O+yRvvLthW6L5+SOv8fNHXmP4gJ22fZdm9JA+nnlGI8lF0r4R8aqkQ9Km+uGm3SXtHhFzmnnsWcAIScNJEsAE4Ct57jsT+LGkvunyccAlEbFa0jpJh5Hc0D8L+HUz47QcIoJL7nuJWUvW8OszD+bgPfpufyezdmD4gJ244Ki9ueCovVmxroaH5yffpbnxL29w/VOLGdTrw5lnh+3Vv8N+gbixK5eLSIaNfp5jXQBHN+fAEVEr6UKSRFEG3BQRVZImA5URMV3SocDvgb7ASZKuiIhRaRL5IUmCAphcf3Mf+BYfTkX+E76ZXxTXPvU6981ZxnePGcFJB+1e6nDMSmJQr2589bA9+ephe7J24xYeX7CCmS+v4N7Zy/jts3+jV7dyxu03iONHDeLITwykR5eOM2zc4CeNiPPTtydGRE3mOkk79ijWjx9jBsl04cy2yzLez+Kjw1yZ290E3JSjvRLYvxDxWW4PvfwO//XQAk4+aHe+M25EqcMxaxV69+jMFw8ewhcPHkLNlq38ZWH9zLMV/H7uMrqWd+LITwxMZp7tt0uznsnWFuSTRp8GDsmjzTqAl5au5bt3zWP00D7812kH+iamWQ7dOpdx7MhBHDsymXn2/JLVPFy1gplVy3lk/grKOolPDU9mnh07chC7t8P7lY3dc9mVZBpvd0kHw7bCeb0AP/2tA1q+toZzp82i/05dmXLWGFcqNMtDeVknjth7AEfsPYDLTxrJS8vWphMCVnD59Coun17FgUN6b3sUzT67tI8nbjV25XI8ybfjhwD/ndG+Hvj3IsZkrdDGzbWcO20W79fUcs83j2CXngUZGTXrUCRx4JA+HDikD/92/L68Xv3+tkTz05kL+OnMBew1cKc00ezKQUN6t9nRgcbuuUwFpko6NSLubcGYrJWpqwsuuusFqt5exw1nVbDfbn4gtlkh7D1wZ771uX341uf24Z21m9Jnni1nyp8Xc+2Tr7Nrr27bpjiPHd6vTc08y+d7LvdK+ntgFNAto31yMQOz1uNnDy/goarl/Mff78e4/QaVOhyzdmm33t056/BhnHX4MN7buJnHXlnJzKrl3F35FtOeeZPe3Tszbr/kmWdHjhhI9y6te1g6nwdXXkdyj+XzwA3AacDzRY7LWol7Zy/lf598nTPHDmXSZ4aXOhyzDqFPjy6cOmYIp44ZwqbNW3nqtWoerlrOo/NXcN+cZXTr3Imj0pln4/Yd1CprJuUzW+yIiDhQ0osRcYWkn+PvjnQIs5as5uJtZYr3b7Njv2ZtWfcuZZyw/66csP+ubNlax3OLVyfPPJuf3Ksp7yQO26s/x48axLEjd2XX3q3jfmg+yWVT+nOjpN2BVcBuxQvJWoO/rdrIN26dzZC+PfjfiYe0qbFes/aqc1knPjNiAJ8ZMYArTh7FC0vf21Zt89L7q7j0/ipGD+2zbebZXgN3Llms+SSXByT1AX4KzCH5dv4NRY3KSmpdzRa+PnUWW+uCG8+uaPdf9jJrizp1Egfv0ZeD9+jLxSfuy6KV65mZfpfmqode5aqHXmXELjtz/KhdOW7UIA4Y3LIzz/K5of/D9O29kh4AukXE2uKGZaVSu7WOC2+fu61McSn/5WNm+dtnl57ss0tPvv35fXj7vU08nE5xvvap17n6iUXs3rsbx6WJZuywfpQXeTSisS9RfqmRdUTEfcUJyUrphw/M58+vVbtMsVkbtnuf7pzz6eGc8+nhrNmwmUdfSco63/H837jl6SX07dGZcfsNYs2GzfTuXpzJAI1duZzUyLoAOnxyOeP6ZwC46xuHlziSwpj2zBKmPvMm537GZYrN2ou+O3Xh9IqhnF4xlA0f1PLn16q3lQxYX1PLHv2K8+iZxr5E+bWiHNFapT+/Vs0Vf5zPuH134ZK/c5lis/Zop67lnHjAbpx4wG5srq3jpF//Hz2K9H2Z7Q66SRok6UZJf0qXR0qaVJRorCQWrljPt2+bw4hdduaXZ7pMsVlH0KW8E316dKZLeXHuveTT6y0kNVfqi3a8Bny3KNFYi1u9YTOTplZuK1O8s8sUm1kB5JNcBkTE3UAdJEW+gK1FjcpaxAe1W7ng1tksX1fDb84a4zLFZlYw+SSXDZL6k9zEJy0hXJCpyJJOkLRA0iJJF+dY31XSXen65yQNS9snSpqX8aqTNDpd92TaZ/26XQoRa3sTEfz7fS/z/JLV/Oz0g1ym2MwKKp8xkIuA6cDekv4KDCR5vlizSCoDrgGOBZYCsyRNj4j5GZtNAtZExD6SJgBXAWdExG3AbWk/BwB/iIh5GftNTCtSWgOufep17p2zlO8eM4KTXabYzAqs0eSSJoCj0tcnSQqGLYiILQU49lhgUUQsTo91JzAeyEwu44EfpO/vAa6WpIiIjG3OBO4sQDwdRn2Z4pNcptjMiqTRYbGI2AqcGRG1EVEVES8XKLFAUuXyrYzlpWlbzm3Sez1rgf5Z25wB3JHVdnM6JHapGnjegaTzJVVKqqyurt7Rz9DmvLxsLf981wuMHtqHn7pMsZkVST73XP4q6WpJn5V0SP2r6JHlQdKngI0R8XJG88SIOAD4bPr6aq59I2JKRFRERMXAgQNbINrSW762hklTZ9G3R2eXKTazosrnnsvo9GdmcbAAjm7msZcBQzOWh6RtubZZKqkc6E3yVOZ6E8i6aomIZenP9ZJuJxl+m9bMWNu8TZu3ct60SpcpNrMWkc89l+kR8YsiHHsWMELScJIkMgH4StY204GzgWdIJhE8Xn+/RVIn4MskVyf18ZYDfSLiXUmdgS8AjxYh9jalri646O55vPz2WpcpNrMWkdc9l2IcOL2HciHJFzRfAe6OiCpJkyWdnG52I9Bf0iKSWWuZ05WPBN6qnxCQ6grMlPQiMI8kaf2mGPG3JT9/ZAF/enk53/87lyk2s5aRz7DYXyVdDdwFbKhvjIg5zT14RMwAZmS1XZbxvgY4vYF9nwQOy2rbAIxpblztyX1zlnLNEy5TbGYtq5T3XKzIZi1ZzcX3vsThe7lMsZm1rHyKhX2+JQKxwqovUzy4b3eu/QeXKTazlrXd5CKpN3A5yT0OgKeAya5G2Xqtq9nCpFZYpri91L1pKT5f1pbl88/Zm4D1JDOzvgysA24uZlC242q31vGPt8/ljXc3cO3EQ1ym2MxKIp97LntHxKkZy1dImtfg1lZSVz74Ck+9Vs1/fukAjtjHZYrNrDTyuXLZJOkz9QuSPg1sKl5ItqNufWYJtzy9hHM/M5wzXabYzEoonyuXC4Bp6b0XgDXAOUWLyHbIXxZW8wOXKTazViKf2WIvAAdJ6pUuryt6VNYki1au51suU2xmrUiDw2KSLpI0qX45ItZFxDpJkyS5zHErsXrDZr5+SyVdy8u44ewKlyk2s1ahsb9EE8n6BnzqVqAS+J+iRGR5yyxTfOf5hzGkb49Sh2RWUp6+3Xo0dkO/PFftlojYTFI0zEoou0zxIS5TbGatSGPJpZOkjz3lMFebtbzrnlrMvXOW8p1xLlNsZq1PY8nlp8CDko6S1DN9fQ54APhZi0RnOT308nL+a+arnHTQ7nz3GJcpNrPWp8F7LhExTVI1yQMr9yd5WGUVcFlE/KmF4rMsSZnieRw0xGWKzaz1anRqUZpEnEhaiRXrajh3aqXLFJtZq+d5q23Eps1bOXdqJetqtnCvyxSbWStX0uewSzpB0gJJiyRdnGN9V0l3peufkzQsbR8maZOkeenruox9xkh6Kd3nV2oH40Z1dcG//C4pU/yrCQe7TLGZtXolSy6SyoBrgBOBkcCZkkZmbTYJWBMR+wC/AK7KWPd6RIxOXxdktF8LnAeMSF8nFOsztJT/fuQ1ZryUlCk+ZqQn65lZ69fgsJikixrbMSL+u5nHHgssiojF6fHuBMYD8zO2GQ/8IH1/D3B1Y1ciknYDekXEs+nyNOAU2vB9o9/PXcrVTyxiwqEuU2xmbUdjVy4901cF8E1gcPq6ADikAMceDLyVsbw0bcu5TUTUAmuB/um64ZLmSnpK0mcztl+6nT4BkHS+pEpJldXV1c37JEVSuWQ137vHZYrNrO1pbCryFQCS/gwcEhHr0+UfAA+2SHQNewfYIyJWSRoD/EHSqKZ0EBFTgCkAFRUVUYQYm+Wt1R8tU9yl3GWKzaztyOcv1iBgc8by5rStuZYBQzOWh6RtObeRVA70BlZFxAcRsQogImYDrwOfSLcfsp0+W731aZni2lZWptjMLF/5JJdpwPOSfpBetTwHTC3AsWcBIyQNl9QFmABMz9pmOnB2+v404PGICEkD0wkBSNqL5Mb94oh4B1gn6bD03sxZwP0FiLXF1G6t4x/vmMviapcpNrO2K596Lj+S9Ceg/r7G1yJibnMPHBG1ki4EZgJlwE0RUSVpMlAZEdOBG4FbJS0CVpMkIIAjgcmStgB1wAURsTpd9y3gFqA7yY38NnUz/8oHX+HJBS5TbGZtW75fouwBrIuIm9OrhuER8UZzDx4RM4AZWW2XZbyvAU7Psd+9wL0N9FlJ8riaNufWZ9/klqeXMMllis2sjdvusJiky4HvAZekTZ2B3xYzqI7oLwur+cH0Ko7edxf+3WWKzayNy+eeyxeBk4ENABHxNskUZSuQRSvf31am+FcuU2xm7UA+yWVzRATJU5GRtFNxQ+pY1mzYzKSps+ha3sllis2s3cgnudwt6Xqgj6TzgEeB3xQ3rI5hc20d3/jtbN5ZW8OUsypcptjM2o3GHv/SNf0+yc8kHQusAz5JUs/lkRaLsJ2KCL7/+5d4/o3V/HLCaJcpNrN2pbExmGeAQyTdGhFfBZxQCuj6Py/md7OTMsXjR+d8Qo2ZWZvVWHLpIukrwBGSvpS9MiLuK15Y7dvMquVc9dCrfOHA3Vym2MzapcaSywXARKAPcFLWugCcXHbAy8vW8t0753HgkD787PSD/DBKM2uXGntw5f8B/yepMiJubMGY2q3MMsW/cZliM2vHGruhf3REPA6s8bBY823avJXzpiVliu+5wGWKzaz07vrG4UXru7FhsaOAx/n4kBh4WKxJ6ssUv7RsLb/5agUjd3eZYjNr3xobFrs8/fm1lgunffrFoy5TbGYdy3a/Di6pK3AqMCxz+4iYXLyw2o8/zF3Grx9PyhSf+1mXKTazjiGfZ43cT1JeeDbwQXHDaV9mv7ma/3fPixy2Vz+XKTazDiWf5DIkIk4oeiTtzFurN3L+tKRM8XX/MMZlis2sQ8nnL97Tkg4oeiTtSH2Z4i1b61ym2Mw6pHySy2eA2ZIWSHpR0kuSXizEwSWdkPa7SNLFOdZ3lXRXuv45ScPS9mMlzU5jmS3p6Ix9nkz7nJe+dilErPn6SJnifxjjMsVm1iHlMyx2YjEOLKkMuAY4FlgKzJI0PSLmZ2w2CVgTEftImgBcBZwBvAucFBFvS9qfpFRy5gO6JqYVKVvcj2YkZYp//MUD+LTLFJtZB5XPlcsxEfFm5gv4ZgGOPRZYFBGLI2IzcCcwPmub8cDU9P09wDhJioi5adEygCqgezqrraR+++yb3PzXpEzxVz7lMsVm1nHlk1xOlTSxfkHSNcDAAhx7MPBWxvJSPnr18ZFtIqKWZNZa/+z4gDkRkTmT7eZ0SOxSNTBFS9L5kiolVVZXVzfncwDwfwvf5XKXKTYzA/IbFjsVmC6pDjgBeC8iJhU3rPxIGkUyVHZcRvPEiFgmqSdwL/BVYFr2vhExBZgCUFFREc2JY9HK9/nmbbNdptjMLNXglYukfpL6Ad2Bc4H/B6wHrkjbm2sZMDRjeUjalnMbSeVAb2BVujwE+D1wVkS8Xr9DRCxLf64HbicZfiuaLVvrXKbYzCxLY38JZ5M8Q0wZP/8+fQWwVzOPPQsYIWk4SRKZAHwla5vpwNkkhctOAx6PiJDUB3gQuDgi/lq/cZqA+kTEu5I6A18gKctcFHURLFz5Ph/U1nHHeYe5TLGZWaqxZ4sNB5DUKSLqMtdJavYjfSOiVtKFJDO9yoCbIqJK0mSgMiKmAzcCt0paBKwmSUAAFwL7AJdJuixtOw7YAMxME0sZSWL5TXNjbcjb721ifU0tv5wwmjF7ukyxmVm9fMZwbgC+Xr8gaSeSK4pxzT14RMwAZmS1XZbxvgY4Pcd+VwJXNtDtmObGla9de3WjW+cylyk2M8uSz2yxZZL+F0BSX+AR4LdFjaqNKC/rxICdSz4D2sys1dlucomIS4H3JV0HPAz8PCJuLnpkZmbWZjVWiTKz+uRzwKXA80BI+pIrUZqZWUMau+eSXYFyLtA5bXclSjMza1Bjs8VcgdLMzHbIdu+5SJqafq+kfrmvpJuKG5aZmbVl+cwWOzAi3qtfiIg1wMHFC8nMzNq6fJJLp3QKMpA8Fob8vh9jZmYdVD5J4ufAM5J+R/IImNOAHxU1KjMza9O2m1wiYpqk2cDn06YvZRX0MjMz+4i8hrfSZ35VA90AJO0REX8ramRmZtZm5TNb7GRJC4E3gKeAJcCfihyXmZm1Yfnc0P8hcBjwWvqk5HHAs0WNyszM2rR8ksuWiFhFMmusU0Q8AVQUOS4zM2vD8rnn8p6knYG/ALdJWklSN8XMzCynfK5cTgY2At8BHgIWkVR4bDZJJ0haIGmRpItzrO8q6a50/XOShmWsuyRtXyDp+Hz7NDOz4mvsqcjrSR5Q+ZHm9Odlkl4Hvh8Rj+3IgSWVAdcAxwJLgVmSpmdNc54ErImIfSRNAK4CzpA0kqQq5Shgd+BRSZ9I99len2ZmVmSNPbiyZ0Pr0sSwP3Bb+nNHjAUWRcTitM87gfFAZiIYD/wgfX8PcLUkpe13RsQHwBtpGeSx6Xbb69PMzIosn2Gxj4mIrRHxAvDrZhx7MPBWxvLStC3nNhFRC6wF+jeybz59mplZke1QcqkXEdcXKpCWJul8SZWSKqurq0sdjplZu9Ks5NJMy4ChGctD0rac20gqB3oDqxrZN58+AYiIKRFREREVAwcObMbHMDOzbKVMLrOAEZKGS+pCcoN+etY204Gz0/enAY9HRKTtE9LZZMOBESQlmPPp08zMiqxkj86PiFpJFwIzgTLgpvQZZpOByoiYDtwI3PBxkAsAAAuwSURBVJresF9Nkizqn3V2N8mN+lrg2xGxFSBXny392czMOrqS1mWJiBnAjKy2yzLe1wCnN7Dvj8jx6P9cfZqZWcsq5bCYmZm1U04uZmZWcE4uZmZWcE4uZmZWcE4uZmZWcE4uZmZWcE4uZmZWcE4uZmZWcE4uZmZWcE4uZmZWcE4uZmZWcE4uZmZWcE4uZmZWcE4uZmZWcE4uZmZWcE4uZmZWcCVJLpL6SXpE0sL0Z98Gtjs73WahpLPTth6SHpT0qqQqST/J2P4cSdWS5qWvc1vqM5mZ2YdKdeVyMfBYRIwAHkuXP0JSP+By4FPAWODyjCT0s4jYFzgY+LSkEzN2vSsiRqevG4r6KczMLKdSJZfxwNT0/VTglBzbHA88EhGrI2IN8AhwQkRsjIgnACJiMzAHGNICMZuZWZ5KlVwGRcQ76fvlwKAc2wwG3spYXpq2bSOpD3ASydVPvVMlvSjpHklDCxizmZnlqbxYHUt6FNg1x6rvZy5EREiKHei/HLgD+FVELE6b/wjcEREfSPoGyVXR0Q3sfz5wPsAee+zR1MObmVkjipZcIuKYhtZJWiFpt4h4R9JuwMocmy0DPpexPAR4MmN5CrAwIv4n45irMtbfAPxXI/FNSfugoqKiycnNzMwaVqphsenA2en7s4H7c2wzEzhOUt/0Rv5xaRuSrgR6A9/N3CFNVPVOBl4pcNxmZpaHUiWXnwDHSloIHJMuI6lC0g0AEbEa+CEwK31NjojVkoaQDK2NBOZkTTn+p3R68gvAPwHntOSHMjOzRNGGxRqTDl+Ny9FeCZybsXwTcFPWNksBNdDvJcAlBQ3WzMyazN/QNzOzgnNyMTOzgnNyMTOzgnNyMTOzgnNyMTOzgnNyMTOzgnNyMTOzgnNyMTOzgnNyMTOzgnNyMTOzgnNyMTOzgnNyMTOzgivJgyvbi7u+cXipQzAza5V85WJmZgXn5GJmZgXn5GJmZgVXkuQiqZ+kRyQtTH/2bWC7s9NtFko6O6P9SUkL0iqU8yTtkrZ3lXSXpEWSnpM0rGU+kZmZZSrVlcvFwGMRMQJ4LF3+CEn9gMuBTwFjgcuzktDEiBidvlambZOANRGxD/AL4KpifggzM8utVMllPDA1fT8VOCXHNscDj0TE6ohYAzwCnNCEfu8BxknKWRLZzMyKp1TJZVBEvJO+Xw4MyrHNYOCtjOWlaVu9m9MhsUszEsi2fSKiFlgL9M8VgKTzJVVKqqyurm7GRzEzs2xF+56LpEeBXXOs+n7mQkSEpGhi9xMjYpmknsC9wFeBaU3pICKmAFMAKioqmnp8MzNrRNGSS0Qc09A6SSsk7RYR70jaDViZY7NlwOcylocAT6Z9L0t/rpd0O8k9mWnpPkOBpZLKgd7AquZ/GjMza4pSfUN/OnA28JP05/05tpkJ/DjjJv5xwCVp0ugTEe9K6gx8AXg0q99ngNOAxyNiu1cls2fPflfSmxlNvUmG1PJZHgC8u71j7KDs4xZyv8a2aeq6jn6+Glvv89W09c09X1C8c+bz9XF7NrgmIlr8RXIf5DFgIUli6Je2VwA3ZGz3dWBR+vpa2rYTMBt4EagCfgmUpeu6Ab9Lt38e2GsH45uS7zJQWcTzNKVY+zW2TVPXdfTz1dh6n6+WPV/FPGc+X017leTKJSJWAeNytFcC52Ys3wTclLXNBmBMA/3WAKcXIMQ/NnG5WHb0OPns19g2TV3X0c9XY+t9vpq23ueraetb7flSmrlsB0mqjIiKUsfRVvh8NY3PV9P5nDVNsc6XH//SfFNKHUAb4/PVND5fTedz1jRFOV++cjEzs4LzlYuZmRWck4uZmRWck4uZmRWck0sRSdpP0nWS7pH0zVLH09pJOkXSb9KyCceVOp7WTtJekm6UdE+pY2mtJO0kaWr6ezWx1PG0doX8nXJyaYCkmyStlPRyVvsJaS2ZRZI+ViogU0S8EhEXAF8GPl3MeEutQOfrDxFxHnABcEYx4y21Ap2vxRExqbiRtj5NPHdfAu5Jf69ObvFgW4GmnK9C/k45uTTsFrIe8S+pDLgGOBEYCZwpaaSkAyQ9kPWqL2B2MvAgMKNlw29xt1CA85X6j3S/9uwWCne+OppbyPPckTyTsP7p6ltbMMbW5BbyP18FU6pni7V6EfHnHJUsxwKLImIxgKQ7gfER8Z8kzzjL1c90YLqkB4HbixdxaRXifKWlE34C/Cki5hQ34tIq1O9XR9SUc0dSqmMIMI8O+o/pJp6v+YU6boc82c2wvRozHyHpc5J+Jel62v+VSy5NOl/APwLHAKdJuqCYgbVSTf396i/pOuBgSZcUO7hWrqFzdx9wqqRrabnHxLQFOc9XIX+nfOVSRBHxJGmZANu+iPgV8KtSx9FWpM/o64hJOG/pswi/Vuo42opC/k75yqVp6uvF1BuStlluPl9N4/O143zumqbo58vJpWlmASMkDZfUBZhAUkPGcvP5ahqfrx3nc9c0RT9fTi4NkHQHSdGxT0paKmlSRNQCF5IUMnsFuDsiqkoZZ2vh89U0Pl87zueuaUp1vvzgSjMzKzhfuZiZWcE5uZiZWcE5uZiZWcE5uZiZWcE5uZiZWcE5uZiZWcE5uVi7JGmrpHmSXpb0R0l9inCMJyVVNHGfyZKO2YFjnZL51Nod7Serzx6SVknqldX+B0kNljyQ9H5zjmsdg5OLtVebImJ0ROwPrAa+XeqAJJVFxGUR8egO7H4KyaPRAWhGP9tExEaSL9F9MSPG3sBn8EMerZmcXKwjeIb06cKS9pb0kKTZkv4iad+M9mclvSTpyvp/nadPtn6gviNJV0s6J/sAkq6VVCmpStIVGe1LJF0laQ5wuqRbJJ0mqSK9spqXHjPS7c+TNEvSC5LuTa8ujiApdPXTdPu96/tJ9xknaW7az02SumYc+wpJc9J1++Y4N3eQPPqj3hdJEk4nSY9l7Ds+x2du8NxIGiPpqfQ8z5S0Wx7/nawdcXKxdk1JUaRxfPjcpCnAP0bEGOBfgf9N238J/DIiDiB5/HhTfT8iKoADgaMkHZixblVEHBIRd9Y3RERlemU1GngI+Fm66r6IODQiDiJ5LMekiHg6jf/f0n1ez/h83UiKQZ2Rxl4OZJbUfjciDgGuTT9vtpnAIZL6p8sTSBJODfDFdN/PAz+XpHxOhKTOwK+B09LzfBPwo3z2tfbDycXaq+6S5gHLgUHAI5J2Bo4Afpeuux6o/xf14cDv0vc7UtTty+nVyVxgFBlDWMBdDe2U3ts4BKgvy7t/ekX1EjAx7asxnwTeiIjX0uWpwJEZ6+9Lf84GhmXvHBGbSRLXaZIGAAeTJBwBP5b0IvAoyZXfoO3EkhnT/iTnfB5JZdEhee5r7YTruVh7tSkiRkvqQfLH8tsk/8J/L71ayFctH/1HWLfsDSQNJ7kqODQi1ki6JWu7Dbk6lrQ/8APgyIioL8F7C3BKRLyQDjF9rgmx5vJB+nMrDf//fgdwKUlCuT8itqTHHgiMSZeX8PHP3tC5EVAVEYc3M3Zrw3zlYu1aetP6n4B/ATYCb0g6HZKyypIOSjd9Fjg1fZ95D+JNYKSkrumMs3E5DtOLJIGslTSIpC55o9K+7gDOiojqjFU9gXfSoaWJGe3r03XZFgDDJO2TLn8VeGp7x8/yJDCCJAHfkbb1BlamieXzwJ459mvo3CwABko6HJJhMknbuwKzdsbJxdq9iJgLvAicSfIHe5KkF4AqkrrhAN8FLkqHgfYB1qb7vgXcDbyc/pybo/8X0vZXSYbU/ppHWONJ/mD/pv7Gftp+KfBc2serGdvfCfxbeuN+74xj15BUWvxdOpRWB1yXx/Ez468D7gH682Fiug2oSPs8KyuW+v1ynpt0qO004Kr0PM8jGY60DsSP3Dcj+c4HyVBaSJoAnBkRH5shZWb58T0Xs8QY4Op0RtR7wNdLHI9Zm+YrFzMzKzjfczEzs4JzcjEzs4JzcjEzs4JzcjEzs4JzcjEzs4JzcjEzs4L7//bWQO34W9KFAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zqj5fCM1kRh-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}