lectures/lecture6/lecture6.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Лекция 6. Введение в итерационные методы"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## План на сегодня\n",
    "\n",
    "### Часть 1.\n",
    "\n",
    "Концепция **итерационных методов** для линейных систем:\n",
    "- Метод Ричардсона и его сходимость\n",
    "- Метод Чебышёва и его сходимость "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Итерационные методы\n",
    "\n",
    "- Если мы хотим достичь сложности $\\mathcal{O}(N)$ в решении разреженных линейных систем, то прямые методы нам не подходят.\n",
    "\n",
    "- Если нам нужно решить частичную задачу на собственные значения, то полное спектральное разложение оказывается слишком дорогой операцией.\n",
    "\n",
    "- Для обеих задач мы будем использовать итерационные методы, основанные на Крыловских подпространствах, которые рассматривают матрицу как **линейный оператор** для преобразования одного вектора в другой (\"чёрный ящик\"), а не как двумерный массив с числами."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Матрица как \"чёрный ящик\"\n",
    "\n",
    "- Сейчас рассмотрим альтернативный взгляд на то, что такое матрица: далее матрица – это линейный оператор, который действует на вектор, и результат этого действия можно вычислить за $\\mathcal{O}(N)$ операций.\n",
    "\n",
    "- **Единственная информация, которая нам доступна:** мы знаем о матрице только то, как умножать её на вектор!\n",
    "\n",
    "- Можно ли решить линейную систему, используя только процедуру умножения матрицы на вектор?\n",
    "\n",
    "- Конечно, мы можем умножить матрицу на столбцы единичной матрицы и восстановить матрицу, но это не то, что нам нужно."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Метод Ричардсона (a.k.a метод простой итерации)\n",
    "\n",
    "Простейшая идея приводит нас к **методу Ричардсона**.\n",
    "\n",
    "$$Ax = f,$$\n",
    "$$\\tau  (Ax - f) = 0,$$\n",
    "$$x - \\tau (Ax - f) = x,$$\n",
    "$$x_{k+1} = x_k - \\tau (Ax_k - f),$$\n",
    "   \n",
    "где $\\tau > 0$ параметр метода, который надо выбрать так, чтобы метод сходился."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Связь с обыкновенными дифференциальными уравнениями (ОДУ)\n",
    "\n",
    "- Метод Ричардсона имеет глубокую связь с ОДУ.\n",
    "\n",
    "- Рассмотрим задачу, зависящую от времени ($A=A^*>0$)\n",
    "\n",
    "$$\\frac{dy}{dt} + A y = f, \\quad y(0) = y_0.$$\n",
    "\n",
    "Тогда $y(t) \\rightarrow A^{-1} f$ при $t \\rightarrow \\infty$, и **схема Эйлера** выглядит следующим образом\n",
    "\n",
    "$$\\frac{y_{k+1} - y_k}{\\tau} = -A y_k + f,$$\n",
    "\n",
    "что также приводит к методу Ричардсона\n",
    "\n",
    "$$ y_{k+1} = y_k - \\tau(Ay_k -f). $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Сходимость метода Ричардсона\n",
    "\n",
    "- Пусть $x_*$ решение, введём вектор ошибки $e_k = x_{k} - x_*$, тогда  \n",
    "\n",
    "$$ e_{k+1} = (I - \\tau A) e_k, $$\n",
    "\n",
    "поэтому если $\\Vert I - \\tau A \\Vert < 1$ в некоторой норме, метод сходится. \n",
    "\n",
    "- Для случая симметричной положительно определённой матрицы всегда можно выбрать $\\tau$ так, чтобы метод сошёлся.\n",
    "\n",
    "- Поведение метода для несимметричной матрицы будет рассмотрено далее"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Выбор оптимального параметра\n",
    "\n",
    "- Параметр $\\tau$, который минимизирует $\\|I - \\tau A\\|_2$ для $A = A^* > 0$, выражается следующим образом (получите это выражение!)\n",
    "\n",
    "$$ \\tau_\\mathrm{opt} = \\frac{2}{\\lambda_{\\min} + \\lambda_{\\max}}, $$\n",
    "\n",
    "где $\\lambda_{\\min}$ – минимальное собственное значение, а $\\lambda_{\\max}$ – максимальное собственное значение матрицы $A$.\n",
    "\n",
    "- Итак, чтобы найти оптимальный параметр нам нужно знать **границы спектра** матрицы $A$, и мы можем вычислить их с помощью **степенного метода**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Число обусловленности и скорость сходимости\n",
    "\n",
    "- Даже с оптимальным выбором значения параметра $\\tau$, ошибка на следующей итерации связана с ошибкой на текущей итерации следующим образом\n",
    "\n",
    "$$\\|e_{k+1}\\|_2 \\leq q \\|e_k\\|_2 , \\quad\\rightarrow \\quad \\|e_k\\|_2 \\leq q^{k} \\|e_0\\|_2,$$\n",
    "\n",
    "где \n",
    "\n",
    "$$ q = \\frac{\\lambda_{\\max} - \\lambda_{\\min}}{\\lambda_{\\max} + \\lambda_{\\min}} = \\frac{\\mathrm{cond}(A) - 1}{\\mathrm{cond}(A)+1}, $$\n",
    "\n",
    "$$\\mathrm{cond}(A) = \\frac{\\lambda_{\\max}}{\\lambda_{\\min}} \\quad \\text{для} \\quad A=A^*>0$$\n",
    "\n",
    "число обусловенности матрицы $A$.\n",
    "\n",
    "- Далее покажем на примере сходимость метода Ричардсона..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Maximum eigenvalue = 3.6825070656623624, minimum eigenvalue = 0.08101405277100471\n",
      "Condition number = 45.45516413147952\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaMAAAEcCAYAAABqCdtUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl4FFXWBvD3JIQdEsKmiApJEFllCSqyRCUBN1QggI7jMp9CcBkXHAk4OjM6jhB0cEMxgdGZ0XFkVWRAIEFlcyNEZIeQgBsIKCSI7OR8f1Q1NE130p1Up7qr39/z9BNSXak+txL69K176x5RVRAREdkpyu4AiIiImIyIiMh2TEZERGQ7JiMiIrIdkxEREdmOyYiIiGzHZERERLZjMiIiItsxGRERke1q2B1AuGjSpIm2atXK7jCIiMLK6tWrf1LVphXtx2Tkp1atWiE/P9/uMIiIwoqIfOPPfrxMR0REtqtyMhKRhlYEQkREkavSyUhEWovINgAlIvKziAxye26IiBRaEiERETleVXpGEwBkAGgEYDiAW0VkMACo6mwAiVUPj4iIIkFVklG+qi5R1VJVzVPVYQAai8jV5vMslERERH6pSjIq8dygqlMBiIh0rcJxiYgowlSpZyQiI0SkUERauTaq6hLzn6VVCYyIiCJHpZORqn4FYAaAYaq6w8tz3asWmjMU7z2ICR9uBsu7ExH5VqWp3eZ40Vc+nttelWM7xUeb9+D1pUWYsrTI7lCIiEIWb3oNsrt7t8bAS1rguUVb8PHmPXaHQ0QUkrwuByQirwNIRtVnxAmAIlUdXsXjhC0RwcQhnVG89yAe/O9XeP+BXkhsWt/usIiIQopwLKN8IjIQwMCkpKQRhYWVv4/3+/2HcOPklYirE4N3RlyOc2JrWxckEVGIEpHVqppc0X68TFcBVZ2nqiNjY2OrdJyWjeri9d92x+4DR3Dj5BVY891ZM+OJiCIWk1E1urR1PObc1ws1a0RhWPZneP+rH+wOiYgoJPgaM5oAoLVFr1GsquMsOlbYa3tOA3zwQG/c+/ZqPDx9DVbt2Icnrm+POjWj7Q6NiMg2vuoZjQcQb9Fr7LPoOI4RX68m3rr7Mvx98RZkLyvGl9v34eVbu6LduVwAnYgiEycw+Ck5OVmDUVxveeFejJ7xNUoPH8ejaRfhnj4JiI4Sy1+HiMgOnMAQJvq0aYqFD/XBlRc1xfgPN2NY9mfY/tOvdodFRFStAkpGItJQRGaIyEnzsUpEHg1WcJGicf1ayL69O14c3gWFu3/BtS8tw7TlxThZxl4rEUWGQHtGEwGsAjAMwFgA2wE8ZxbXu8Tq4CKJiODmruchd3QKeiU2wTPzNyH99U+xbc8vdodGRBR0gSaj1ar6nKrONr8OU9UoGEX2Zrmv3k2V07xhbUy7MxkvDu+C7T/9iuteWoHJHxXi+Mkyu0MjIgoan8lIRKaIyCARcZ/i5fW6karOAtADRlKiKjrVS3okBWntm+P5xVtx4+SVWPc9q3IQkTOV1zPKADAbwH6zZtEUAHEiMt7bzqpaAoBLU1uoaYNaePW2bsi+vTt+PngUN7+2EuMXbMLhYyftDo2IyFLlJaM88xJcDwA5ABJhjBllupKTiNwjIl0AwLxExxpGQTCgwznIHZ2CYcktkb2sGNe8tAyfbvvJ7rCIiCzj8z4jEWntrSaRWVI8FUaSSgUQB+PyXQGAoZ6F9pwiWPcZBerTop/w+Jx12PHzIQxPPh+PX9cOsXVj7A6LiMgrf+8zqvJNryISq6qOH8wIlWQEAEeOn8QLeVsxbfl2NKpbE0/f1AHXdjwHIrxZlohCS7Xd9BoJiSjU1I6Jxrhr22Hu/b3QvGEt3PefAox8azV+LD1id2hERJXCFRjCWMfzYjH3/l4Yd+3FWF64F2mTluI/X3yDMt4sS0RhhskozNWIjkJGSiIWPdwXnVrG4o/vrcctOZ+jaO9Bu0MjIvIbk1EFRGSgiOSUlob21cgLG9fDf+65DBPTO2PL7l9w7YvLMfmjQhw7wZtliSj0cdVuP4XSBIaK7PnlCJ6atxHz1+5C2+YNMGFIJ3S9oJHdYRFRBOKq3RGsWYPaePU33TDtjmQcOHIcg6d8iqfmbcCvR0/YHRoRkVeVTkYiEisif7AyGLJWavvmWPxIX9x++YV4c+UO9H9hGT7essfusIiIzlKVnlE8jBtfKYQ1qB2Dp2/qiFmjeqJOzWj87s1VePjdr/DzwaN2h0ZEdAov00WI5FbxmP9gbzzUrw3mr9uF1ElLMafge3DMkIhCAZNRBKlVIxqPpF2E+Q/2Qasm9TB6xte4440v8d2+Q3aHRkQRjskoAl3UvAFmjboCT93YAQXf7Ef/F1hZlojsxWQUoaKjBHde0QqLR6egZ2JjPDN/Ewa/thIbdx6wOzQiikBMRhHuvLg6+MedyXjl1q74fv9h3Dh5BZ5btBlHjrNmEhFVHyYjgohg4CUtkDc6BTd3PQ+vflyE615ajs+Lf7Y7NCKKEExGdEqjejXx/NBL8Pbdl+F4WRluyfkc4+asQ+nh43aHRkQOx2REZ+ndpgkWP5yCkX0TMH3Vt0ibtBQL1++yOywicjAmI/KqTs1oPH5dO8y9vzea1K+FUW8XIOOtfOw+wJpJRGS9qiYjlhZ1uE4tYzH3gV7IvOZifLJlL1L/zppJRGS9SicjVd0OINPCWChExURH4d4rjZpJHc8zayZNZc0kIrJOlXpGZkKiCNGqST28M+IyZA3phM27DuDal4yaScdPsmYSEVUNx4woICKC4T0uQN6jKUhr1xzPL96Kga+swJrvSuwOjYjCGJMRVUqzBrXx6m3dkHN7d+w/dAyDX1uJp+dtZM0kIqoUJiOqkv4dzkHu6BT85rIL8MbK7ej/wjJ8wppJRBQgJiOqsoa1Y/DMzZ0wc1RP1I6Jwl1vrsIj09dg36/H7A6NiMIEkxFZpkereCx4qA8e7NcG/1u7E6mTluL9r35gzSQiqpAlyUhEZojIYLfv+4lIQyuOTeGlVo1ojDZrJl3YuC4enr4Gd725ijWTiKhcVvWMslV1jusbVV0CINmiY1tKRBJEZIyIpJpf4+yOyYlcNZP+MrA9Vu3YhwEvLsMbK7azZhIReWVVMioWkVYAICKLRWQVgDSLjm21mao6UVXzAOQAmGp3QE4VHSW4q1dr5I5OwWWt4/H0/zZi8JRPsflH1kwiojNJZa/ni0ghgDwYb+4fiUgXAAkwlgjKU9VS68I867XTAZSYCcXzuW4wemXFZjzFrv3M57JUNc1t//2q2qii10xOTtb8/HyrmhBxVBUffL0TT8/biNLDxzEqJREPXJ2E2jHRdodGREEkIqtVtcIrZTWq8BqzAawCMExEsgFsB9AawNAgJ6JUGL2ZoV6eS8DZyWamiBSrqis5ed6duU9EuqlqQbBiJuNm2Zu6nIe+bZrimfmbMPnjbViwfhcmDO6MS1vH2x0eEdmsKmvTjVXV2ao6SlXbAMgAMBHA4yKyTUSmWBYlTo31ZMNIKPt87JYBINtjWzaALPPffNezWaN6NfH3YZfgrbsvxfGTZRiW/Rkef28dDhxhzSSiSGbZ1G5V3a6qU1V1mKomwUhMllHVYlXNUNWccnZLB+DZw8k3twNGEvOcsMAEZYM+bZpi0cN9MaJPa7z75bdI/ftSLFz/o91hEZFNAk5G/k7Zru5FVM1ZcWf1mlS1xHw+AcY40lnJh5fo7FG3Zg388fr2eP/+XmhcvxZGvb0ao95azZpJRBEooGQkIjMA7HfNnHPbfrWFMVVWPHA6+XiR4Jl0zAR11iQIql6dW8bhgwd6Ycw1bfHRlj1InbQU//3yW9ZMIooggfaMcgEMU9UdHttXi8gfbL7R1d/7hUa47jOCcfluhK8dRWSkiOSLSP7evXstCZK8i4mOwn1XJmHRw33R/tyGGDdnHW6d+jmKWTOJKCIEmoziYCSkM6hqqao+D2CYJVEFkaoWuO4zMr/6rH2gqjmqmqyqyU2bNq3OMCNW6yb18N8Rl2PC4E7YuOsArnlpOV79eBtrJhE5XEDJSFWfAzBTRMaLyFVBiqlKuKJC+IuKEtxy6QVYMjoFqe2a4blFWzDwlRX4mjWTiBwr0DGj12Hc1JoGYImInBSRVSIyRUTGA+gejCD9VGx+PWOCgltyKgaFlWYNa+O127oj26yZNOi1lfjr/zbi0DHWTCJymkBvei1S1VGub8xxF9dDAfSzMLaAqGqJiBTD+9TtEvOm14CJyEAAA5OSkqoaIlXSgA7noGdiY2R9uBn/WLEdizb8iL8N6oSUi3jplMgpAh0zOuM6iTnuMtZc6mEk7B8zysPZC7R2QxVmzKnqPFUdGRsbW6XAqGoa1o7B3wZ1woyMnqhZIwp3vvElRrNmEpFjBJqM8kTkHh/P7a9qMAGIh/fZc5k4e5mgDHM7OcClreOx4ME+ePDqJHzwtVEzae4a1kwiCncBL5QqIrEw3vBnqOoBc1trAEUwSknca3mUODX2Mw7Gja3pMMaA8gDkquost/26ARgOY928BAAF3hZUDRQXSg09m388gLGz12HNdyW4sm1TPHNzR7RsVNfusIjIjb8LpVZ61W4vL9gPxgrZ1bryQnVhMgpNJ8sU//5sB55btAUA8If+bXHnFa0QHSX2BkZEAPxPRlauTbfEqYmIQld0lOB3vVpj8SN9calbzaRNu1gziSicWJaMAMDqlbpDgYgMFJGc0tKgVcUgC7RsVBdv3tUDL93SBd/tO4SBr6zA84u24Mjxk3aHRkR+qNJlOnP5nwQYEwpctYQaWxRbSOFluvCx79djeGb+Rswp+AEJTeuxZhKRjao8ZmRWbnUlmUQYs9fcE4/nbDYBoKrqyNKdTEbhZ9nWvXj8vXX4fv9h/OayCzD22ovRsHaM3WERRRQrktFinL6Z1TUaXGw+SmAkpGwYJRtKYFR6neHPi4YjJqPwdOjYCbyQuxX/WLEdTRvUwtM3dcSADufYHRZRxKjyBAZV7Q/jJtapABqpapSqJqlqf1UdBiDfLKY325y84JpqTRQy3GsmNapbExlvrca9b6/GHtZMIgop5U5gUNVZ5vI/Q83Ldmc87WX/sVYGR2SVzi3jMO/3vfHYgLZYsnkP+k1aine//JY3yxKFCL9m06nqNAClIVJEr1pxNp1zxERH4f6rkrDwoT5of25DjDVrJm3/6Ve7QyOKeH5P7VbV7ar6kYh0MVdciAhcm855EprWP1UzacPOAxjw4jLWTCKyWcD3GanqGvPm1gKbK7sSVZp7zaR+Fxs1k26cvBJrv2fNJCI7VPqmV1WdCqCxiFztSkoiMtiyyIiqQbOGtTHlt93x+m+74+eDR3HzqyvxDGsmEVW7Kq3A4Lp0ByDNLLw31ZqwiKrXNR3PQe7oFNxy6QWYtmI7+r+wDMu27rU7LKKIYclyQKo6G0AOvJd1IAoLsXVi8OygTpg+8nLUrBGFO974EqNnrMF+1kwiCjorF0otALDEquMR2eWyhMZY8GAf/P7qJHywhjWTiKqDpQulwihk5yic2h2ZasdE49H+bfG/B3ujZXxdPPTuGvzun6vw/f5DdodG5EiW1TNyOi4HFLlOlin++ekO/H2xUTPpsQFtcUdP1kwi8ke11zMicqroKMHdvVtj0cN9kdwqHk/N24ghUz7Flh9/sTs0Isfw2jMyZ8Ylw8uSP4EeH0CRqg6v4nFsx54RAYCqYu6anXhq3gYcPHoC96Yk4v6rk1CrhiMXqyeqsmovO+50TEbk7ueDR/HM/E1476sfkNi0HiYM6YwerVgzicgTL9MRBVHj+rXwwvAu+Nf/XYojx8sw9PXP8MT763DgyHG7QyMKS0xGRFWQclFTLH6kL+7u3RrvfPEt0iYtxeINP9odFlHY8TVmNAGAVYuhFqvqOIuOZRtepqOKfP1dCTJnr8XmH3/BdZ3OwV9u7IBmDWrbHRaRrao0ZiQisTDKi1thn6qG7U06IjIQwMCkpKQRhYWFdodDIe74yTLkLCvGS0sKUbtGFP54fTsMSz4fIpwGTpGJExgsxp4RBaJo70GMm7MOX27fh54JjfHs4E5o3aSe3WERVTtOYCCyUWLT+nh3xOUYP7gT1u8sxTUvLsNrn7BmEpEvTEZEQRIVJbj10guQNzoFV7ZtiokLt+CmySux7vuwvWpNFDRMRkRB1rxhbWTfnozXf9sNPx08ipteXYG/zWfNJCJ3TEZE1eSajucid3QKhvc4H1OXb8eAF5dheSFrJhEBTEZE1Sq2TgzGD+6Md0dejpioKNz+jy/x6IyvWTOJIh6TEZENLk9ojAUP9cH9VyVi7pofkDppKT74eidrJlHEYjIiskntmGg8NuBizPt9b7RsVAcP/vcr3P2vfPxQctju0IiqHZMRkc3andsQc+7rhSdvaI/Pin5G/0lL8a9Pd+BkGXtJFDmYjCrASq9UHVw1kxY/0hfdW8Xjzx9swNDXP8XW3ayZRJHBshUYRKQLjBpI+aq6xpKDhhCuwEDVRVXx/pof8PS8jUbNpCuTcP9ViayZRGHJjhUYHgeQA2CWiBSKyGALj00UMUQEg7q2RN7oFNzQuQVeXlKI619egfwd++wOjShorExGmQC6q2qSqrYBsMTCYxNFHFfNpH/+rgcOHzuJ9Nc/w5Pvr8cvrJlEDmRZMlLV7ar6ldv3HGQhssCVbZth8SN98X+9WuPtL75B2qRlyN242+6wiCzFCQxEYaBerRr408D2eO++XoirG4MR/87H/e8UYM8vR+wOjcgSlZrAYE5W8FrvSFU/qmpQoYgTGChUHDtRhpxlRXh5yTbUqRmNP17fDkO7t2TNJApJQalnJCJDAMxwfetlF1VVR075YTKiULNtz0E8PmcdvtyxD1ckNsazgzqhFWsmUYgJVjLaZv4zE0Cxt33cx42chMmIQlFZmeK/q77FhAWbcexkGR5Juwj39G6NGtG8Ak+hIVjJqAxAuqrOqUpw4YjJiELZ7gNH8Ke567Fow250aNEQWUM6o+N5sXaHRRS0+4zy4GOsiIjs414zac8vR3HTqysxfsEmHD520u7QiPxSI8D9MwDMEJFYGPcRlXjuoKo7LIiLiCrhmo7nomdiE0z4cBOylxXjw/U/4tlBndC7TRO7QyMqV6CX6foBmAnAZ/+fExiIQsNnRT/j8ffWYftPvyK9e0s8cX07xNWtaXdYFGGCNWaUDyABxrI/Rd72UdWpfh8wDIjIQAADk5KSRhQWFtodDlFAjhw/iVc+KkT20mLE1Y3Bnwd2wA2dz+U0cKo2wZzAMFJVp1UluHDEnhGFs407D2DsnLVY+30prr64Gf56c0ecF1fH7rAoAgRrAkMBAK7WSBRm2rdoiPfu64Unrm/HmkkUkgJNRiMAZInIVSLSMBgBEVFwREcJ7umTgMWP9EW3CxuxZhKFlECT0RIAiTCmeO8XkZMejxPWh0hEVjo/vi7+/X+X4oXhl2D7T7/i+peXY1LuVhw9wWngZJ9Ap3ZPBcB+PVGYc9VM6tumKf76v414eUkhFqzbhawhndD9Qt5KSNUv0AkMDVX1QBDjCVmcwEBO9vGWPXjivfXYWXoYt19+IR4b0BYNasfYHRY5QLAmMKwWkUcrGRMRhairzJpJv7uiNd763KiZlMeaSVSNAk1GHwG4JRiBEJG9XDWT5tx7BWLrxOAes2bS3l+O2h0aRYCAkpGqZgAQEVkoIhcGKSYislHXCxph3u9749G0i5C7YTdSJy3FjPzvUJnaZ0T+CigZicg+GCsw9AdQzNl0RM5Us0YUft+vDRY81AdtmzfAmFlrcdu0L/DNz7/aHRo5FGfTEZFPSc3q492Rl5+qmTTgxWV4JPUi3M2aSWSxypYdvxrAUBi9pGIAM51abtyFs+ko0v1YatRMWryRNZPIf8GaTQcRmQLjptfhABqbX3NF5N2AoySisHFObG1k394dU25jzSSyXqBjRiNg1DRKU9V4VU1W1XgYCWkYp30TOZuI4NpO5yLvkRQMS26J7GXFGPDiMqzc9pPdoVGYC7RnlAFgjKoucd+oqrMAjAUwyqrAiCh0xdaNwfjBnfHfEZcjSoDbpn2Bx2Z+jZJDx+wOjcJUoMmoG4yVu70pgjGGREQRomdiYyx8uC/uvTIRc776AamTluJ/a3dyGjgFLNBklAcg3cdzw+E7URGRQ9WOiUbmNRfjgwd64dzYOnjgna8w4t/52FV62O7QKIwEmozGAhglIq+JSBcRaWh+nQJgCIBM60O0l4gMFJGc0tJSu0MhCmkdWsTivfuuwBPXt8OKbT8hbdIy/PuzHShjzSTyQ8BTu0UkHUbZcdecTlf94gynlRx3x6ndRP779udD+OP767C88Cd0v7ARJgzuhDbNG9gdFtkgKGXHPV4gFUBrGPcZ5auqo7sOTEZEgVFVzCn4AX+dvxG/Hj2B+69Kwr1XJqJWjWi7Q6NqFPRkFGmYjIgq56eDR/H0vI344OudaNOsPiawZlJECdpNr+bBG/p6VOZ4RORcTerXwsu3dsWbd/XAr0dPIP31z/Cnuevxy5HjdodGISTQm167ikghgP3lPIiIznLVxc2weHQK7uzZCm99/g36v7AMSzaxZhIZAq30mg/jXqIcGPcVncWpkxh4mY7IOgXf7sfY2WuxdfdB3ND5XPx5YAc0bVDL7rAoCIIyZiQiZQBGquq0qgQXjpiMiKx17EQZXl9ahMkfbUOdmtF44vp2SO/eEiJS8Q9T2AjWmFEegH2VC4mI6LSaNaLwYL82WPBQb7RpVh+PzVqL3/6DNZMiVaDJKBNAlogM4mQFIrJCUrMGmJHRE3+9uSO+/q4UA15chpxlRThxsszu0KgaBXqZbh+AOPgusKeqGmjBvrDAy3REwber9DCefH8D8jbtRsfzGmLCYNZMCnf+XqZjpVciChnnxtbB1Du648P1P+JPczfgpldX4p4+rfFI6kWoHcObZZ2MN736iT0joupVeug4nl2wCdPzv8OFjeti/KBOuCKpid1hUYCCetMrEVGwxdaNQVZ6Z7wz4jIIgN9M+wJjZn2N0kO8WdaJmIyIKKRdkdgECx/ui1EpiZhd8AP6TVqK+Wt3sWaSwzAZEVHIqx0TjbHXumom1cb97xRgxL9Xs2aSgzAZEVHYcNVM+uN17bBi216kTVqGtz5jzSQnYDIiorBSIzoKI/omYPHDKehyfhyenLsBw7I/w7Y9v9gdGlUBkxERhaULGtfFW3dfiueHXoJtew/iupdW4KW8Qhw7wZtlwxGTERGFLRFBeveWyBudggEdz8ELeVtxwyvLsfobFhAIN0xGRBT2mtSvhVdu7Yo37krGwSMnkP76p/jLBxtw8OgJu0MjP/lMRiJSJiInA3zwN09Etrn64uZYPDoFd1x+If712Q70n7QUH21mzaRwUN5yQM+BS/8QUZipX6sGnrqpI27sch7Gzl6L//tnPgZe0gJ/HtgeTeqzZlKoitjlgEQkV1XT/N2fywERhZ+jJ05iyidFePXjbahXqwaeuL49hnQ7jzWTqhGXA/JBRFJFZCSAVLtjIaLgqlUjGg+nXoQFD/ZBYtP6+MPMr3HHG1/iu32H7A6NPFSqZyQig2GUHz+Lqj5fqUBE0gGUqGqel+e6AUgGUGy+brG3/QJ8PVVVvz8esWdEFN7KyhT/+eIbZC3cgpNlikf7X4S7rmiFGtER95m8WgWlhISItAaQD6CRuUkBuL+hK4CAk5GIpMIoTzHUy3MJALLcL6mJyEwRKVbV4kBfi4giU1SU4PaerdCvXXP8ae56PDN/E+au2YkJQzqhQwvWTLJboB8JsgFsB9BdVaMAlALoBiAeRknyjEAOJiIJIpINo7fjq5x5hvm6nnFkBfJaREQA0CKuDqbekYzJv+mKXaWHcePklchauBlHjp+0O7SIFmil1zIAqar6kfl9PoB7VHWNiPQDMEFVe1QqEJEiABmel9/M7WnuvSARiQOw33WZzRwDSizn8LlejsvLdEQRruTQMTy7YBNm5H+PVo3r4tnBnXBFImsmWSlYlV5LALj3Z4thjOWsgVGOvFuAxyuXmXTO6jWpaomIQEQSVLVYVXOsfF0iigxxdWtiYvoluLnLeRj33jr8ZuoXGJ58Ph6/rh1i68bYHV5ECfQyXR6AW9y+nwEg05zQkAUjWVkpHjCSj4/nvU6iICIKxBVJTbDwob7ISEnArILv0W/SUixYx5pJ1SnQZJQJY8wIAKCqswDsADALRuIYYVlkhjiLjwcR6SYiY8x/Z5mTJ3ztO1JE8kUkf+/evVaHQkQhpE7NaIy7th3m3t8L58TWwn3/Yc2k6mTJTa8i0lpVt1e8Z7nHOGvMyJzSvdrb2I6IKIyxpCpN8fYXx4yIIseJk2V4Y+V2TMrdihpRUci89mLcdukFiIrizbKBqtabXquaiCpijh0REVWLGtFRGNk3EYse7otLzo/Fk++vZ82kIAsoGYlIw4oeFsfnmkEX7xFHnMfzRESWu7BxPbx992V4Lr0zCvcYNZNeXsKaScEQaM+oBMD+Ch6WMScuFOPssaN4GKs1BD0ZichAEckpLS0N9ksRUQgSEQxNPv9UzaRJuUbNpIJvWTPJSoEmo1E+HtNgrMQQ0E2vfsqDMX3cXTdze9Cp6jxVHRkbyzu0iSJZ0wZGzaR/3JmMX46cwJAprJlkpYDuMyrnfp6pIrIN5d94WpF4eJ89lwlgJgD3185AcBIfEVG5+rVrjssSGuO5hZvxr892IHfjbjxzc0dcdXEzu0MLa5aVkBCRrgBmqGqbAH4mDsA4GPcLpcO4JJcHY8WEWW77dQMwHMAqc9+C6ppF58LZdETkafU3+5A5ex227TmIm7q0wJ9uaI/GrJl0Bn9n01mZjB6DsRxQtCUHDDFMRkTkjXvNpPpmzaTBrJl0SlCmdpdXihzGCgwFlQ2YiCgcuddMat2kHh5lzaRKCXSh1Cz4LkVeBOMynaOmnYnIQAADk5KSRhQWFtodDhGFMFfNpAkfbkaZgjWTYMNlOqfjZToi8tfOksN48v31WLJ5Dzq3jMWEwZ3RvoXVt2GGB5YdJyKySYu4Oph2p1EzaWfJYQycvII1kyrgc2p3ZVdTUNUDlQ+HiMgZRAQ3dG4lxg15AAARH0lEQVSB3klN8Lf5mzDlkyJ8uG4Xxg/ujJ6Jje0OL+SU1zPyZ7WFoK7AQEQU7uLq1sRzQy/Bf+65DGUK3Dr1c4ydvRalh47bHVpIKe+m11E4e7JCHIxZc5k4s3ZREoDHAIyxNDoiIofoldQEix7uixfztmLq8mIs2bwHT9/YAdd0PIfTwBH4bLopMEo6TPPy3GMA4lV1nIXx2Y6z6YjIaut/KEXm7LXYsPMA+rdvjqdv6ohzYmvbHVZQBGsCw3B4lAB3UwBjFQVH4dp0RGS1jufFYu79vTDu2ouxrHAv0iYtxduff4Oyssid3VyZ2XQjfWwfCo9SD0RE5F2N6ChkpBg1kzqfH4sn3l+P4TmfYdueg3aHZotAk9EEAP1FZKGIDBKRLiJytYhMh5GkfC2kSkREXrjXTNq6+yCue2k5XonAmkkB3/QqImMAjIUxmUFhlI4oATBeVZ+zPMIQwZteiSjY9vxyBE/N24j5a3ehbfMGmDCkE7pe0MjusKok6CswmCtptwZQrKpfVeogYYTJiIiqS97G3Xhy7nr8eOAI7rqiFf7Qvy3q1Qqo4k/I8DcZVbp1qloALoxKRGS51PbNcVlCPJ5btAX//HQHFm/YjWcGdcRVbZ1bM8nnmJGILBaREx7bfK7abT4cV/KQZceJyA4Nasfg6Zs6YtaonqhbMxq/e3MVHnr3K/x88KjdoQWFz8t05thQsqoOc9tW3qrdAABVHWtphCGCl+mIyC6eNZOevKE9BnUNj5pJXLXbYkxGRGS3rbt/Qebstfjq2xL0adMEzw7qhPPj69odVrmCVVyvlYh08dj2BxGZLiJ3BxokERH576LmDTBr1BV4+qYOKPhmP/q/sAzTlhfjpANulg30PqNsABmub0RkMYy16hIB5IjIsxbGRkREHqKjBHf0bIXc0SnomdgYz8zfhMGvrcTGneFdMCHQZJQGIBcARKQrgFQAmWYXbByMVRiIiCjIWsTVwT/uTMYrt3bF9/sP48bJKzAxjGsmVWY5oGLzayqMyQyuVReKACRYERQREVVMRDDwkhbIG52Cm7ueh9c+KcJ1Ly3H58U/2x1awAJNRsUAhpmF94YDKHArppeA04mKiIiqSaN6NfH80Evw9t2X4USZ4paczzFuzlqUHg6fmkmBJqOx5mM/gG4w6hq5ZACYZVFcIYP3GRFRuOjdxqiZNLJvAqav+g5pk5Zi4fof7Q7LL5VZm641jESUp6qlbttHAJjhvs1JOLWbiMLJuu+Nmkkbdx3AgA5GzaTmDau/ZlKw6hkBxjhRMoDp5uU6iEgsjAQV/vMLiYgcoFPLWMx9oBcyr7kYn2zZi9RJS/HOF9+GbM2kQO8z6gdjokKa+YgHALM3lAijxAQREYWAmOgo3HulUTOpY4tYPP7eOtwy9XMU7Q29mkmB9oyyAMwyu1ye61DkwJjUQEREIaRVk3p4Z8RlmDikMzbvOoBrX1qOyR+FVs2kQJNRNwDTfTy3H0aNIyIiCjEigmE9zkfeoylIa9cczy/eihsnr8Ca70rsDg1A4MmoAMZ4kTcZAPKqFg4REQVTswa18ept3TD1jmSUHDqOQa+txNPzNuLXo/YWXQi0ntEEADPEWCpWAcSaa9VlwbgJtrvF8RERURCktW+OyxPiMXHhFryxcjsWbfgRfxvUEVfaVDMpoJ6Rqs4CMAxGL0hg9JQKAPQAMExV11geIRERBUWD2jH4680dMXNUT9SOicJdb67CI9PXYN+vx6o9lqqUHe8Kc9UFlh0nIgpvR0+cxKsfF2HKJ9vQoHYM/nRDe9zUpUWVaybZUs9IRLo4rXckIgMBDExKShpRWFhodzhEREG15UejZtKa70rQ96Km+NvNHatUM8mSm15FpKFZq+hnESn0VbPI3G8GgNWVjDdkqeo8VR0ZGxtrdyhEREHX9pwGmH3vFfjLwPbI37EP/V9Yhg++3hn0161ozKgAxs2tS2CMEeWIyKPuO4jIeBjTulNhrFtHRERhLDpKcFev1sgdnYJeSY2R0KRe0F/T52w6EXnM/Gdr13pzIrIawEQRcd3gmgUjSY1V1eeCHSwREVWf8+LqYNqdParltcqb2j0cwASPhU8zASwGUAIjCU0EMN6pi6MSEVH1KC8ZdQPgOX1slfk1B0ZviEmIiIiqrKIxozPWiXBLPtlMREREZJXKlJAgIiKyVEXLAQ0REc8ekMIoPe513riqTrMkMiIiihgVJSNfM+R8TeFWAExGREQUkPKSERc9JSKiauEzGUXCenNERBQaLF2bzslEZC+Abyr5400A/GRhOOEiEtsdiW0GIrPdkdhmIPB2X6iqTSvaicmoGohIvj8LBTpNJLY7EtsMRGa7I7HNQPDazandRERkOyYjIiKyHZNR9cixOwCbRGK7I7HNQGS2OxLbDASp3RwzIiIi27FnREREtmMyIiIi21W0HBBVgYh0A5AMoBhAAoBiVc2zNypriUg6jLYlml+zVXWWxz6OPQ8iEgcgS1UzPLY7ss1mu4YD+BlAYxi/72KP5x3TbhFJhVFOBzDaW6SqOR77hH2bzf/HJd7i9qd9lpwDVeUjCA/zF5LrsW0mgAS7Y7OwjekAurl9HwegCMDISDkPALIBzIyE3735+872bL9T2w0jCaV6OQeO+vsGkApgv2db/W2fVeeAl+mCJwPGG5W7bBil2p0iQVULXN+oagmM9rm327HnQUQSAMR7ecpxbTZ7gFPVrQdofppOddvNae3OUI9P92r0+tPc90GYtllEEkQkG0Yy2edjN3/aZ805sDsrO/UBo4eQ4LEtzjjl9sdnQfviAKwGEOexPQHG6u0JTj8PAEaaD8+ekePabL6xZHnZ7v4J2VHtNv++z/p0D7degFPabLbDW8+owvZZdQ7YMwoC81PkWZ821Og5uD5RhzWzLQnmwysnnwdzLGGGl+1ObfNIAKs8N6o5XuTQducByDXHQwCc6g3ONP/txDaf4k/7rDwHTEbBEQ+c/oV4EdZ/pC6q2kjdLtOZUmEMhBbD2echzke7nNrmOADFIjJSRNJdX92ed1y7VTUTQAmA1SIyxtVePT2BwXFt9uBP+yw7B5xNFxxxdgdgowwA481/O/I8iEi6eswYdOO4Nrt9uk1weyOGiGSJSLy5zXHtBgBV7S4iuTAuUxYA6Of2tCPb7Maf9ll2DtgzIsuIyEgA+1R1ot2xBIt5WcLXp0Cn8vWGMx1hMFBfFebfdCaMSQsJMHpJ4d7jCUlMRkFkvnFFBPM/aIaqpnl5zknnYZj6cf+Ew9pc7PEVAGBeoo1zf3N2UrvNmWZ5qlpg/s5bwzgHqz32c0ybvfGnfVacAyaj4HD9pz1j2q/bL6wYzpOFMy9hAA47D+ZAdn4FuzmqzcAZ4wHljQs4qt2uuNXthl5VLTE/bBWbE1gc1WYv/GmfZeeAY0ZBoKolIlKMsy9vxOP04L5jmJ8gMz0HMR14HpIBJIrIcLdt3QAkiEgWgFWqOsthbXY5dWe9t+cc+rsu8vFcNswJLA5r8xn8bZ9V54A9o+DJg/EH7a6bud0xzGvqWXrmkjCpbpduHHMeVDVHVTPdHwByARSY37smNTimzW6ycXpZHACneorubzhOancxjCWuvImDMZkBcFabvfGnfdacA7tvtnLqA8YfrOcSGbkIo2VC/GhjOoz7T7q5PVJx5hIxjj4PMC5Pet706rg2m20q8tKmdKe2G0YC9lwOKAFuN/86pc0wlgNK97K9wvZZdQ5YzyiI3BaVXAXjj9g1EBr2zGvC+308XayqiW77Ou48mD2/TADDYPxnzIGRhAvM553c5iIYvYaZnm1yWrvNnn8ijIVhAaMn6G2h1LBrs/l/eByMmNNh9AbzYCSWWW77Vdg+K84BkxEREdmOY0ZERGQ7JiMiIrIdkxEREdmOyYiIiGzHZERERLZjMiIiItsxGRERke2YjIiIyHZMRmQ5c206NR8jvTyf4Ou5IMeVJSIhf5e3iGSLyP6KzpFZfTS3OmMLVDjE6M7828w1z3223fFEEq7aTcGWCWOpHPKDiMyEscbfCHNTease94CxFmAoC4cYT1Fj0dc080NL2CRRJ2DPiIIpB0Z5hXS7Awkj6TDKccwyHwW+dlTVoaoq7tvM3l+1VyL19breYgx15jprgHNW3g4LTEYUTDNhfLIfZ3cg4cDtzbxSZc3NhS/HwKPUQ7DZ9bpBlAqzRpPdgUQSJiMKtiwA3dw+bRKFujSwV1TtmIwoqMzl9ktgJCWfvE0uEJGR5kBynMd++0Wkm4isNp9f7Sro5zb4XOTr8qD5s7nmcYrMEtKe+6R7Ht8zBvPf2eYxPCtdlve6rmOOdHsuC6cri7r2Kffypvs5M8eaXCU9Zpo/v99j/4DbZJ7TmW4TKnLdL8dV9Lo+fq8+z4NnPF729Szwl27Gqt7iq6RUmONF5jkYYz7K/RumqmEyouowHkCqBW8SLnEwLgGOB5ABo37KTLfHUHO/mT5+fiqMwmmZ5veeb7Dp5s/mmccq9tzH3G81jAqX2RVd0jGPudo8VhqA6QCy3WZsjTe3w4wr0b2mjB9GuP18BowaPN0taFOGx/HjzXb49bqe/DgP7ly/52yc+Xt2Hcv1/SzzNV3tqvCDQTnxnRovMj9gDFPViWa7OPYZTHZXGOTDeQ8YnywVZpVMGG8OCrMCLIw3FQUw0u1nsow/xzOOM9LcL85zP7hV4HTbNsZtW7q5rVsFr5HgHpu5bT/cqnma21a7trm9Xq6vc+DlnOx3fw0f58kVy1kVN30c84z2lPfzVrUJxrjQGa9Rwet6xljhefCIJ93LtjiP33FceTEH+Lc7BkYPNc7j7zPOytfh4+wHe0YUdGp8ws4BMNKfy1l+ynf7t+vylvt1fteU6PgKYnNVt0wGTn0yjgMwxu3Sj+J0WXV3GfCDeTksDsYnfPfXzoNxCXOot5+zipVt0tOz+wLu5VbyPLj/Tot8PLfEvIxmxbikq5c3TN0quqpqiXJCQ1AxGVF1cV1vt+RGVx9vDJV9syjG6TdX19fuMC7NuD/OeLM0E5k/XMf0tr/7awdLldpkjsvMNMdsfJWaDyQOv89DeQnAfM6VPLIArJaq32CbCvOyn7cxKgoe3vRK1UJVi0VkFoxp3qE2UykBp98gT71RBpBsKuI6TgIAz/uGEnBmLy8YKt0m8809AcabvSvO1b5/wq84LDsPZq+qu9njHgkgS0TGqDHOExC3xDNeVUtEZAyMManEQI9FgWPPiKrTeBiXaYb7ub/PgXCrmG9iqTATpHkZyuvsvypcYnS9yZ5xCcwczHcN0lthn7eNlW2TOUEgFcZNuDnmcbwlM6+v60XQzoN5GW0ijCTXo5KH8by/qBjmZV5zVh0nMAQRe0ZUbVS1QEQKYAwSeyoCjLXMYCSGVFh0Sc+TOR05E6c/8ZfASJQuI2BMU3bN5IqD8QZaDD/HidyZn7IzYMwai4cxg8z12rPMT/dVZr4OAGS4vqqq6zJWwG0ye7MlMHobgHGeMr3sV97reu5n2Xkwp4Rnmu1xXebrZn7vSqZZMHo6PleycOPt/iJX8j1jDImsx54RVbez3sxMM2B8qs2C8Qm5MSrxxu+n6TDuI8mF8am+u/vYhBpTqrvDeMPOhTEVvLic2CtkvpGl4fT05AwYPQ6rJy9MhJHIs+B2KawKbRoBo3cwE6d/N36/rieLz8MMGOM7GW7HmuiWNOJgzLhL9vN48XCbXGGes2LzA5K/vT+qJFEN+UWMiYgqRUSKAAz1s2dENmLPiIgcjYkoPDAZEZEjmZMzKn1plaoXL9MREZHt2DMiIiLbMRkREZHtmIyIiMh2TEZERGQ7JiMiIrIdkxEREdmOyYiIiGz3/0ttdj/r5vrbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "plt.rc(\"text\", usetex=True)\n",
    "import scipy as sp\n",
    "import scipy.sparse\n",
    "import scipy.sparse.linalg as spla\n",
    "import scipy\n",
    "from scipy.sparse import csc_matrix\n",
    "n = 10\n",
    "ex = np.ones(n);\n",
    "A = sp.sparse.spdiags(np.vstack((-ex,  2*ex, -ex)), [-1, 0, 1], n, n, 'csr'); \n",
    "rhs = np.ones(n)\n",
    "ev1, vec = spla.eigsh(A, k=2, which='LA')\n",
    "ev2, vec = spla.eigsh(A, k=2, which='SA')\n",
    "lam_max = ev1[0]\n",
    "lam_min = ev2[0]\n",
    "\n",
    "tau_opt = 2.0/(lam_max + lam_min)\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "plt.close(fig)\n",
    "\n",
    "niters = 100\n",
    "x = np.zeros(n)\n",
    "res_richardson = []\n",
    "for i in range(niters):\n",
    "    rr = A.dot(x) - rhs\n",
    "    x = x - tau_opt * rr\n",
    "    res_richardson.append(np.linalg.norm(rr))\n",
    "#Convergence of an ordinary Richardson (with optimal parameter)\n",
    "plt.plot(res_richardson)\n",
    "plt.yscale(\"log\")\n",
    "plt.xlabel(\"Number of iterations, $k$\", fontsize=20)\n",
    "plt.ylabel(\"Residual norm, $\\|Ax_k - b\\|_2$\", fontsize=20)\n",
    "plt.xticks(fontsize=20)\n",
    "plt.yticks(fontsize=20)\n",
    "print(\"Maximum eigenvalue = {}, minimum eigenvalue = {}\".format(lam_max, lam_min))\n",
    "print(\"Condition number = {}\".format(lam_max.real / lam_min.real))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Вывод о влиянии числа обусловленности на сходимость\n",
    "\n",
    "- Для **плохо обусловленных** матриц норма вектора ошибки в методе Ричардсона убывает **очень** медленно.\n",
    "- Это ещё одна причина, по которой число обусловленности очень важно: кроме того, что оно показывает ошибку при решении, оно также даёт оценку числа итераций для сходимости итерационных методов.\n",
    "- Основной вопрос в разработке итерационных методов – как улучшить обусловленность матрицы?\n",
    "- Ответом является использование <font color='red'> предобусловливателей (preconditioners) </font>. Эта тема будет рассмотрена далее в курсе."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Рассмотрим несимметричную матрицу $A$\n",
    "\n",
    "Возможные варианты поведения метода Ричардсона:\n",
    "- сходимость\n",
    "- расходимость\n",
    "- почти стабильная траектория, то есть отсутствие сходимости, но и отсутствие существенного увеличения нормы вектора ошибки\n",
    "\n",
    "###### Как мы можем определить поведение метода Ричардсона для несимметричной матрицы **до** запуска метода?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1  2]\n",
      " [-1  0]]\n",
      "Spectrum of the matrix = [0.5+1.32287566j 0.5-1.32287566j]\n"
     ]
    }
   ],
   "source": [
    "# B = np.random.randn(2, 2)\n",
    "B = np.array([[1, 2], [-1, 0]])\n",
    "# B = np.array([[0, 1], [-1, 0]])\n",
    "print(B)\n",
    "x_true = np.zeros(2)\n",
    "f = B.dot(x_true)\n",
    "eigvals = np.linalg.eigvals(B)\n",
    "print(\"Spectrum of the matrix = {}\".format(eigvals))\n",
    "\n",
    "# Run Richardson iteration\n",
    "x = np.array([0, -1])\n",
    "tau = 1e-2\n",
    "conv_x = [x]\n",
    "r = B.dot(x) - f\n",
    "conv_r = [np.linalg.norm(r)]\n",
    "num_iter = 1000\n",
    "for i in range(num_iter):\n",
    "    x = x - tau * r\n",
    "    conv_x.append(x)\n",
    "    r = B.dot(x) - f\n",
    "    conv_r.append(np.linalg.norm(r))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaIAAAEcCAYAAACFy7BqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8VPW9//HXNwlJIGQb9h2SICKgEIJ17watXdCqLLZ295Lo7b1trS3gvba2ta2NttXWVpvYq/1ZrbJobbV1AW211qokLIKILGFfFMhC2Eny/f1xTmAYJslkMjNnZvJ+Ph7zCDlz5pzP90yYz3yX8/0aay0iIiJeSfE6ABER6d6UiERExFNKRCIi4iklIhER8ZQSkYiIeEqJSEREPKVEJCIinlIiEhERTykRiYiIp9K8DiAR9O3b144cOdLrMEREEkp1dfU+a22/jvZTIgrByJEjqaqq8joMEZGEYozZGsp+apoTERFPKRGJiIinlIhERMRTSkQiIuIpJSIREfGUEpGIiHhKiUhERDylRBRFu+qPcPsza2lu0XLsIiJtUSJqhzFmujGmsqGhIazX/3vTfv7v1c38Ysm7EY5MRCR5KBG1w1r7tLW2NDc3N6zXXzN5KNdOGcZv/r6JF97eE+HoRESSgxJRlH3/inFMGJLLzQtXsWXfIa/DERGJO0pEUZbZI5X7rismNdVwwyPVHD3R7HVIIiJxRYkoBob5enHP7Ims29PIj//6jtfhiIjEFSWiGPnQmP7MuXQUf3h9K0vWvud1OCIicUOJKIa+/fExjBucw9zFq3jvwFGvwxERiQtKRDGUkZbKL6+dxJETzdy8cBUtur9IRESJKNaK+vfmtunjeHXjPn73ao3X4YiIeE6JyAPXThnGx8cN4GfPr2ftrgNehyMi4iklIg8YY7jj6nPJ7dWDby5YoSHdItKtKRF5xJeVzl0zzmX9ewe563lNASQi3ZcSkYc+NKY/X7pwBP/36mZe3bDP63BERDyhROSx+Z8YS2G/LL69aBX1h497HY6ISMwpEXmsZ7ozpHvfwWP875/WYK2GdItI96JEFAfGD8nlpmln8dfVu/nTip1ehyMiElNKRHHihg8WMmVkPrf9+W221x72OhwRkZhRIooTqSmGX8yaiAVuXrhKq7qKSLfRrRKRMabAGDPXGDPV/ZnndUz+hvl68YMrxvHmlloqXtnkdTgiIjHRrRIRsMhae6e1dilQCTzgdUCBri4ewqcmDOLuJetZszO8JcpFRBJJwiUiY8wMY8zUNp4rNsaUujWeUv/9jDHFQG3r79baeiDocbxkjOHHV43Hl5XONxes1KwLIpL0EioRuYklaC3GGFMAlFtrK621S621lUCZux2gAKgPeFmtm6DiSl6vdH4+cyIb3z/IT59d53U4IiJRlRCJyO3bqcBJJrVt7FYGVARsqwDK3X/7ohReVFwyui9fvXgUv39tC39f977X4YiIRE1CJCJrbY21tsyt5bRlBrA8YFuVux2cBBY4OCGuk9Pcy8dw9sBsvv7YCtbt0SzdIpKcEiIRdcQd/XZGbcntB2pttqshSOKx1gYmr7iR2SOVB788hZ7pqXz1oWXsbjjidUgiIhGXFIkIN8G0Jp4gCgITjpuclkY7sK4anNeTB788hQNHm5hd8To76nSzq4gklzSvA4iQUO8HmmOMmYvThFcMzGlrR2NMKVAKMHz48C4H2BXjh+Tyh+vP50sPvsnsitd54IslnDM4p0vHtNayo+4IG/ceZPPeQ+yoO8LBYyc4dKyZ1BRDVkYquT3TGdmnF4X9ezNhSC6ZPVIjVCIRkVNMok2yaYzZBJS59wK1bisGqq21Jsj+Fpjmv39nlZSU2KqqqnBfHjFrdjbw1d8vo+HICW791Fg+94ERpKacUeSgmlssa3cd4PWa/VRtraV6az37Dh47+Xyv9FRyMnvQKyOVlhbLoePNNBw+wfHmFgDSU1OYOCyPj40bwCcnDGJwXs+olFFEkocxptpaW9LhfsmUiID8wOa5ZEpEAHsbj/HNBSv418b9jB2Uw1cvHsnl4weSndnjtP2ON7Ww4f1GqrfW8a+N+3i9ppaGIycAGO7rRcmIfIpH5HP2wGxG9c3Cl5WOMacnteYWy676I6zb00jVllr+uWEfa3c7gyY+NKYf118yikuK+p7xOhER6H6JKA+oAwqttTUdbe+seEpE4DSrPfPWbu5esp6afYcwBgr6ZtGndwZY2H/oGNtqD3Oi2Xlvh+b35OLCvlxU1IcLC/rQPycz7HNv2XeIP63YyaNvbGXfweOMG5zD/3xyLBcX9Y1U8UQkSXSrROS3fab/oAR3QEK1tTY/zHNNB6YXFRXN2bBhQ1fCjgprLVVujeed3QdoOHICa51lyEf0yWLc4BwmDstjmK9XxM99rKmZP6/YxS9f3MDO+iN8eEw/fnDFeIb3ify5RCQxdcdEVIGTdCr9ts0AZltrZ3blnPFWI4onR0808/C/t3DvixtparHM/8TZfOGCEaSE2HclIskr1ESUiMO3fQQfJTcPCEw4Ze52iZLMHqmUXlbI8zddxpRRPm77y9t88cE32e83EEJEpD0JUSNy+3puwblpdQbOzalLgSXW2sV++xUDs4Fl7r7LuzJIoZVqRKGx1vLYm9v5/tNv0ycrnfuuK2bS8LBaRUUkCSRt05wXlIg6Z83OBm58tJo9DUf52czzuHLiEK9DEhEPJHPTXMwYY6YbYyobGrQuUGeMH5LL0/91CcXD8/nG4yupeHkT+sIjIm1RImqHtfZpa21pbm6u16EknLxe6Tx8/fl86txB3PHsOu54dp2SkYgElSxT/EgcykhL5d5rJ9EnK53KV5zbuG75xNm6AVZETqNEJFGVkmL4wRXjAKh8pQYDzFcyEhE/SkQSdcY4ychaqHilhqyMNL7+0dFehyUicUKJqB1+Myt4HUrCM8bwwyvHceh4E79Ysp6BuZnMKhnmdVgiEgc0WKEdGqwQWcYYyq85l0tH9+WWJ1fzj3e1BLqIKBFJjPVITeH+z0/m7IHZ/Oejy3l3T6PXIYmIx5SIJOZ6Z6Tx4Jen0DsjjTkPV1F/+LjXIYmIh5SIxBMDcjK5//OT2d1whP9+bAXNLbrHSKS7UiJqh2ZWiK7JI/L54ZXj+eeGfdz5/DqvwxERjygRtUODFaLvs+cP53MfGE7FyzUsWfue1+GIiAe6NHzbGDMRZ1mGM1hrX+rKsaX7uG36OazaXs93Fq/ib1+/lMF5Pb0OSURiKKwakTHmGmNMM1CNsxxD4GNJxCKUpJeRlsq9n53E8aYWvvn4SpqaW7wOSURiKNymuXJgMzALmBzk0eG03yL+Cvr15kefGc+bW2r51UsbvQ5HRGIo3Ka5AmCGtfbJSAYj3dvVxUN5deM+7n1pAxcV9uGCgj5ehyQiMRBujWgpbfQNiXTF7VeOZ4SvFzcvXEXj0RNehyMiMRBuIioDyowxNxtjJhpjRgY+IheidzR8O/ayMtL4+azz2N1whNufWet1OCISA+EmogKgELgTZ8DCpiCPhKfh296YPMJH2QcLWVi1g6Ua0i2S9MLtIyp3f95FkiQdiS/fnDqav697n/lPruaFEfn4stK9DklEoiTcRFQMlFprfxfJYERaZaSlcvfsiVzx61e59anV/OZzxVpMTyRJhds0txyojWQgIoHGDsrhpmln8bfVe/jLql1ehyMiURJuIpoDlBtjPmyMyYlkQCL+yi4rpHh4Ht99ag27G454HY6IREG4iehFnMEKS4E6Y0xzwKMpciFKd5aaYvjFrImcaLbMXfwWLZqlWyTphNtH9ACgTwSJiZF9s7j102P53z+t4eF/b+HLF4/yOiQRiaBwE9GPrbUHIhqJSDs+d/5wlq59jzueXcclo/tR1L+31yGJSISE2zRXbYy5OaKRxCHd0Bo/jDGUX3MuvdJTuWnBSk5oYlSRpBFuInoJuDaSgcQj3dAaX/rnZPKTqyawemcD92piVJGkEVYistaWAcYY85wxZkSEYxJp0ycmDOLq4iH85u8bWb6tzutwRCQCwl2PqBZnmp+PATUaNSex9P0rxjEwJ5P//uMK6g4d9zocEekijZqThJOT2YP7ritm5m//zTcWrOShL08hNUWzLogkqnCb5uZZa+cDLwC5wCT35wvW2vnucyJRc96wPH5w5TheWb+XXy5d73U4ItIF4Q5WwBhzP84NrbOBPu7PJcaYxyMUm0i7rp0yjFklQ/nVSxt5onqH1+GISJjC7SOag7Mm0TRrrc9aW2Kt9eEko1ndYWi3eM8Yw+2fGc/FRX2Y+8Rb/H3d+16HJCJh6MrCeHOttS/6b7TWLgbmAzd0NTCRUGSkpfLbz09m7KBsyv5QzXNr9kTtXE3NLRxvauHoiWasVRepSKR0ZRmIuW08twlnRJ1ITGRn9uCR6z/Alx9axn8+Ws28y89mzqUFpIQ5gOFEcwtrdx1g2ZZaqrbUsXnfIfYcOErDkVNLl6elGPr0TmdYfi/GDsrh3KG5XDq6HwNzMyNVLJFuw4Tzzc4Y8wKw0Vr7n0GeWwiMstZOiUB8njLGTAemFxUVzdmwYYPX4UgHDh1r4tuLVvHsmj1cUtSXWz89lrMHdjw5/MFjTazcVs+bW2qp2lLLim31HDnRDMAwX0/OHpjDwJxM+vROp0dqysnX7Gs8xpb9h1i3u5HGY84dC2MH5XBN8RCunDiEftkZ0SusSAIwxlRba0s63C/MRFQMVAG/BSqBGpxaUBlQitN39FKnDxynSkpKbFVVlddhSAistTz6xjbufG4djceauHR0Py4fN5Bxg3PwZaXTYi17G4+xae9B3trRwKod9byzu5HmFkuKcRLJlJE+SkbmUzLCF1INx1rLuj2NvLJ+L39bs4dV2+vpkWq4etJQbvxQISP7ZsWg5CLxJ6qJyD3BDJwk1Dr/TWs7SJm19oGwDhqnlIgST/3h4zz46maeXLGTHXXB1zHKzkxjwpBcSkbkUzLSx6TheWRn9ujyuTe+38jD/97KgmXbOdHcwszJw/jO5WPo21s1JOleop6I/E40FRiFUyuqstYm3QyhSkSJy1rL1v2Hefe9Rg4cOYExhn7ZGQz39WKEr1fY/UiheL/xKJUv1/D717bQMz2Vm6edxRcvHBnVc4rEk5glou5AiUi6YuP7B/nB02/zzw37uLCgDz+fdR6D83p6HZZI1IWaiMK+odU9SU5bj64cVySZFPXvzcNfPZ/yayawakc9l9/zCi+8Hb1h5iKJJtwbWicZYzYAde08RMRljGH2lOH87euXMrJvFqV/qOaXSzdo6XMRujbpaR/gLpz7hkQkBCP7ZrGw7EJueXI1dy9dz/r3Gvn5rPPI7JHqdWginunKDa2l1trfRTIYke4gs0cqv5h1HmMGZvPTZ9dRd/g4lV8soXdGuP8dRRJbuH1ES4HaSAYi0p0YY7jhg4X8fOZ5vLG5luseeJ36w1pbSbqncBPRPKDcGHOVBiaIhO+ayUOp+Pxk3tnTyBcffJMDR090/CKRJBNuInoRKAQWA3VaoVUkfFPPGcD91xWzdtcBvvLQMg4d038f6V60QqtIHPjo2AH86rOT+K8/LmfOw1U89JUpZKRpAIN0D7qhNQS6oVVi5cnlO/jWwlV8ZuJg7p49EWM0C4MkrlBvaNUwHZE4cnXxUHbVH+FnL6xnRJ8sbpp2ltchiUSdEpFInPnah4vYsv8wv3xxA8N9vbhm8lCvQxKJqi5N8ZPsjDHTjTGVDQ1JN4+rxDFjDD+5agIXFfbhlj+t5q0d9V6HJBJVSkTtsNY+ba0tzc3N7XhnkQhKT0vh158rpl/vDG58ZDm1h3SPkSQvJSKROOXLSue+64rZ23iMbzy+gmbNSydJSolIJI6dNyyPH145jn9u2MfdS9Z7HY5IVCgRicS5a88fzqySofzmHxv518Z9XocjEnFKRCIJ4PtXjGNU3yy+tXCl+osk6XSYiIwxLUGm8OnooTlKRCKoV3oav7p2EnWHTjB38VvoRnRJJqHcR3QXms5HxHPjh+Qy9/Ix/Oiv7/DIG9v4wgUjvA5JJCI6TETW2nmxCEREOvbVi0fxyoZ9/OiZtVwwysfoAdlehyTSZeojEkkgKSmGn808l94ZaXzj8ZUcb2rxOiSRLuvSFD/GmKuBgmDPWWt/1pVji0hw/bMz+ek15zLn4Sp+sWQ98z9xttchiXRJWInIGDMKqALy3U0W8J8m2AJKRCJRMu2cAXz2/GFUvLKJD43pxwUFfbwOSSRs4TbNVQCbgcnW2hSgASgGfDjLiJdFJjwRacutnzqHEb5e3LxwFQ1HtLKrJK5wE9FUYK61doX7ew2AtbYeKEeJSCTqsjLSuHv2RPYcOMptf17jdTgiYQs3EdUD/jOB1gCtix/l4dSORCTKJg3P5+sfGc1TK3fx55U7vQ5HJCzhJqKlwLV+vy8E5rmDF8pxEpWIxMDXPlzIpOF53PrUGnbWH/E6HJFOCzcRzcPpIwLAWrsY2AIsxuknmtPlyEQkJGmpKdwzeyLNLZabF66kRbN0S4IJKxFZazdba+cHbJsGFFprfdbaJyMSXZQYY5Z4HYNIJI3ok8X3p4/j9ZpafvdqjdfhiHRKRG9otdZu7ngv7xhjphpjSnEGW4gklZklQ/n4uAHc9fy7rN11wOtwREIWViIyxuR09Ojk8WYYY4ImB2NMsTGmtDWJtLVfKKy1S621leG+XiSeGWO44+pzyeuVzjcXrODoiWavQxIJSVdGzdV18AiJm1geaOO5AqDcWlvpl0TK3O0iEsCXlc7PZp7H+vcOcudz73odjkhIwp3i5waCz8hdgjNQobSjA7jJZB5QDdS2sVsZzs2z/ipwRubNDDVYke7kg2f140sXjuDBf23mw2f349LR/bwOSaRdJtLrmhhjvgP4rLW3dOI1m4Aya+3SINunWWtr/LblAXXWWuP+XgoUtnP4JUGOa1tfH4qSkhJbVVUV6u4injtyvJnpv36VxqMneO4bl5Gfle51SNINGWOqrbUlHe3XpUlP27AU576ikBNRMG7CKSCgtmStrTfGYIwpsNbWqM9H5Ew901O5Z/ZErrrvX/z3Yyt46CtT6JGqyfYlPkXjL3MqbczI3Uk+ODltUDDqJxJpx/ghufzkqgm8unEftzy5Wqu6StwKd/btFtpetdXg9Pt0VV4EjnEaY0wx7tBtY0w5QZrtRJLJzJJhbK87wq9e3EDvjDRum34OxoTcKi0SE+E2zbW3fPgmnKa5uGOtXQ4sB+7saF+376kUYPjw4VGOTCR6bpo6moNHm3jwX5s5cryZH181nrQINdMdOHqCTe8fZG/jMWoPHafZrXVlpafRPzuDYb5eDM3vqeQn7QorEcVy+XBjTF47zXNR4/Y9VYIzWCHW5xeJFGMM3/30WLIyUrn3pY3U7DvIL6+dxOC8np06jrWWnfVHqNpSR9XWWqq21PHue4101OKXnZHGhKG5fPCsfnzk7P5a3lzOEPFRc2EFEWTUXOvoOJxpg84YNRe4PZo0ak6SxVMrdvI/f1qNtVD2wQKu+8AI+mVnBN33RHML7+5ppGpLLVVb66jaUseeA0cB6J2RxqTheZSM8DFucA4DcjLx9U6nR4rBAgePNfH+gWNs2X+It3c1sGyzk7QAJgzJZdaUYVxTPIRe6dEYLyXxItRRcx0mos7OktDKWhvyHCMdDN+e6TaptW4rAKqttfnEiBKRJJNt+w/z0+fe4W+r95CaYpg8PJ+xg7LJ7ZVOU3ML+w4eY/17B3ln9wGONbUAMCg3k5KRPkpG5FMyMp+zB+aQmtK55rZd9Ud4bs0eFlZtZ92eRvpkpXP9paP44oUj6Z2hhJSMIpmI2huY0CZrbWqo+7aTiCpwkk6l37YZwGxrbdRvaDXGTAemFxUVzdmwYUO0TycSUxvea+SJ5Tt5c/N+1r93kIPHmkhNMfiy0hnVN4sJQ3I5d2guJSN9DOlkM157rLVUb63j3pc28vL6vfTLzuDWT43livMGqy8pyUQyEZVyZiLKw5ndYB6nrz1UBHwHZ/XWn3Ui2DpgjruchP/2PGCRO7N367YlOEkrZlMMq0Yk3UHr8hEpnazpdMXybXV8/y9v89aOBi4o8HHXjPMY5usVs/NLdEUsEbVx8Ptxaiq/C/JcSDMruEnmFpz7gWbgrPK6FGdI9WK//YqB2cAyd9/lsR5yrUQkEj3NLZYFy7Zzx7PvYC3c/plxfGbiENWOkkC0E1Et8B/B1h0yxnwU+K21dnSnDxynlIhEom9H3WFuWrCSZVvqmDF5KD++ajwZaSG38EscCjURdeVmgrYmNp2JOytCojPGTDfGVDY0NHgdikjSG5rfi8dLL+TrHylicfUOPlv5Ou83HvU6LImBcBPRT4GPGWOeM8ZcZYyZaIz5iDFmAU6CSor536y1T1trS3Nzc70ORaRbSE0xfOtjY7jvumLe2d3IZ379Lza+f9DrsCTKwl0q/E5gPnA+8ATOlD5LgWnAvM7MvC0iEuiTEwax6IYLOd7cwqyKf7Nmp1olklnYTXPW2juttT6cNYhmAZOttT5r7V0Ri05Euq3xQ3JZdMNF9OyRyrWVr1O1pa1lyyTRdXnCKWvtcmvtE9baFZEISESk1ai+WSy+8UL6Z2fwlYeWsXqHakbJqMNEZIx5wRjTFLCtxRjT3M6jqa3jJRINVhDx3qDcnjzyHx8gp2cPvvDgG7y7p9HrkCTCQrmhdS5QYq2d5betnA5mW7DWzo9IhHFAw7dFvLd1/yFm/vbfWODJGy/Sja8JIKr3EXU3SkQi8WHDe41cc/9r9M/J5IkbLyK3Zw+vQ5J2RPU+ImPMSGPMxIBt3zbGLDDGXB/OMUVEOjJ6QDa//cJktuw7xNceXc6J5havQ5IICHewQgVQ1vqLMeYFnLnnCoFKY8xPIhCbiMgZLirsy0+udpZA/96f39YS6Ekg3EQ0DVgCYIyZhLP89jy3CnYLzuwKIiJRMatkGDd+qJDH3tzGgmXbvQ5Huqgrw7dbZ7+eijNwoXU2hU04k5MmPI2aE4lf3/7YGC4d3Zfv/eVtDetOcOEmohpglrto3mycGbFbF8Ir4FSSSmia4kckfqWmGH557ST69c7ghkeqqTt03OuQJEzhJqL57qMOKMZZl6hVGbA42ItERCLJl5XOfdcVs7fxGN9csPLkmkqSWMKda24xzsCEWUC+tfYlv6fLcSZFFRGJuvOG5fG96efw8vq9PPivzV6HI2HoSh+RxZlnboHbRIcxJhenhqSvJSISM9d9YDgfHzeA8ufWaYLUBBTufUQfxRmUMM19+ACstQ04NSXViEQkZowx/PTqc/FlpfONx1dw+HhSzDLWbYRbIyoHFrvDtQPX863EGcAgIhIz+Vnp3D1rIjX7DnH7M+94HY50QriJqBhY0MZzdUBemMeNKxq+LZJYLirqS9llzv1Fz7+9x+twJEThJqLlOP1DwZThLJKX8DR8WyTxfGvaWYwfksMtT67WUuMJoitLhc83xtyBMzAh110u/HngGk4fzi0iEjPpaSncM3sih441MXfxW5oCKAF0Zfj2LJzaj8GpIS0HpgCzrLUrIxahiEgnFfXP5n8+OZZ/vLuXR97Y5nU40oGuLBW+2F0qfDKnLxX+RMSiExEJ0xcvHMFlZ/Xjx39dy6a9B70OR9oRiaXCVwQuFR64RISISKwZY7hrxrlk9kjlpgUrtWREHAspERljcty1hvYbYza0teaQu99CoDqiUYqIhGFATiZ3XDWBt3Y0cO+LG7wOR9oQao1oOc6Nqy/i9AlVGmNu9t/BHbhQhzMbd9IsEy4iie0TEwZxTfFQfv33jVRvrfM6HAmiw0RkjPmO+89R1tpZ1toiYCVwpzEm2xjzH8aY/TgDF+a7/UR3RTHmmNF9RCLJ4bYrzmFQbk++tXAlh45p1oV4E0qNaDbwU3f6nlbzcGpG9TgzKTyAk6iSIgG10n1EIskhJ7MHd8+eyLbaw9z+zFqvw5EAoSSiYqAqYNsy92clzuzb8wMSlYhIXDl/lI+yywp5fNl2lqx9z+twxE+ofUT1/r/4JZ0KJSARSRTfmnYW5wzKYf4Tb7G38ZjX4Yiry8O3RUQSRXpaCvdcO5HGY03Me0KzLsSLtBD3u8YYE1jzsTjLhQedc85a+7suRSYiEgVnDchm/uVn88Nn1vLHN7dx3QdGeB1StxdqImprEEJbw7QtoEQkInHpyxeN5KV173P7M2s5d0geE4ZqQJKXQklEk6MehYhIDKWkGH4x+zyu+s1rXP//lvHU1y5mcF5Pr8PqtozaSDtWUlJiq6oCBw6KSKJ7d08jM+5/jSH5PVlQeiG5vXp4HVJSMcZUuwuotkuDFUSk2xozMJv7Pl9Mzd5DfPaB16k9dNzrkLol1YhCoBqRSHL7x7vvU/aHaob5elH5hckU9OvdpePtP3iMldvr2bT3IFv3H2ZX/REOHW/m6Ilm0lIM2Zk98GWlU9A3i9EDsikZmU/f3hkRKk38CLVGpETUDmPMdGB6UVHRnA0bNGGiSDJ7vWY/Nz5SzfGmFm799DnMKhlGaooJ6bXbaw/z2qZ9vFFTS/W2OrbuP3zyudyePRia35PeGWlk9kilucVy4OgJ9jUeY1fDqRVkzx6YzcfHDeTKiYO7nAjjhRJRBKlGJNI97Ko/wjcXrOTNzbWMGZDN5y8YzkfGDmBwbibGOEmpqbmFTXsPsWZnA29uruW1mn1srz0CQN/eGUwekUfx8HyKR+RzVv/sdvudDh9vYt2eRl6v2c8r6/fyxuZarIULCnyUXlbAh87qT0qIyTAeKRFFkBKRSPdhreWZt3Zz/z82sXb3AQCyM9PIyezBsaYW6g4fp7nF+dzMyUzjgoI+XFTYh4uL+lLUv/fJhBWO9w4c5U8rdvL/XtvC7oajnD0wm1s/dQ6XjO4bkbLFmhJRBCkRiXRP7+w+QNWWWja8f5CDR5vI6JGCLyudov69OWdQLkX9e4fcfNcZx5taeHrVLu55cT3ba48wdWx/bps+jmG+XhE/VzQpEUWQEpGIeOHoiWZ+/9qWk4v6fffT5zB7yrAu1bpiScO3RUQSXGaPVG74YCHP33QZ5w3LY/6Tq5nzcBUNR04Lqnh0AAAP2ElEQVR4HVpEKRGJiMS5ofm9eOT6D/C9T5/Dy+v3csWvX2XdngNehxUxSkQiIgkgJcXw1UtG8XjpBRw53sxVv3mNF99JjnWVlIhERBLI5BE+nvnvSyjq35vSP1SzsGq71yF1mRKRiEiC6Z+TyWOlF3BRYR/mLn6Lipc3eR1SlygRiYgkoN4Zafzfl6bw6XMHccez6/jdP2u8Dilsoa5HJCIicSY9LYV7Zk/EWvjRX98hNcXwlYtHeR1WpykRiYgksLRUZ/nzppYWfvD0WrLS05g1ZZjXYXWKmuZERBJcj9QU7v1sMZed1Y9b/rSaf7z7vtchdYoSkYhIEkhPS+G+64oZMyCbrz26nDU7G7wOKWRKRO0wxkw3xlQ2NCTOGyoi3VfvjDQe+soU8nql85XfL2N3wxGvQwqJElE7rLVPW2tLc3NzvQ5FRCQkA3IyeegrUzh8rIkbH1nOsaZmr0PqkBKRiEiSOWtANj+fdR4rt9fz/b+87XU4HVIiEhFJQpePH8TXPlzIY29u549vbPM6nHYpEYmIJKlvTRvDZWf147a/rOGtHfVeh9MmJSIRkSSVmmL41bUT6dc7g68/toKDx5q8DikoJSIRkSSW1yude66dxLbaw3zvqTVehxOUEpGISJI7f5SPr390NE+u2MmTy3d4Hc4ZlIhERLqB//pwEeeP9PHdp9awZd8hr8M5jRKRiEg30DonXVpqCjctXElTc4vXIZ2kRCQi0k0MzuvJD68cx4pt9VS8Ej/LRigRiYh0I1ecN5hPTRjEPUvXs3bXAa/DAZSIRES6FWMMt39mPLk90/nWwpVxMQWQEpGISDfjy0qn/JoJrNvTyK9e3OB1OEpEIiLd0UfHDmBWyVDu/8cmlm+r8zQWJSIRkW7qu58+h0G5Pfn2wlUcOe5dE50SkYhIN5Wd2YO7Zp5Lzb5D3PHsO57FoUQkItKNXVTYl+svGcXD/97Ky+v3ehJDt0pExphiY0ypMWauMWaRMabA65hERLz2nY+PYXT/3nxn0SrqDh2P+fm7TSIyxuQBJdbaSmvtnUAFsMTjsEREPJfZI5W7Z0+k7vBx/vep1VhrY3r+uEhExpgZxpipbTzXWouZ2vozzNMUAPP8fq8CCtwEJSLSrY0fkstN087ib6v38NTKnTE9t+eJyE0sD7TxXAFQ7tZillprK4GycJrUrLXLgWl+m0qAemtt/K4WJSISQ2WXFTJlZD7fe+ptdtQdjtl5PUtExpgCY0wFTk2lto3dynCa0PxVAOXhnNNa6z+5UhkwJ5zjiIgko9QUwy9mTQTga48uj9msCybWbYFBgzBmE1BmrV0aZPs0/wTiNqXVWWuN+3spUNjO4ZcEOW4pUGutXRxKfCUlJbaqqiq0woiIJLjn395D2R+q+fwFw/nRZyaEfRxjTLW1tqSj/dLCPkOUuQnnjNqStbbeGIMxpsBaW+M213XmuFOBmsDkJCIijo+PG0jZBwuoeLmGySPyuWrS0Kiez/M+onb4wEk8bTzf6X4iY0wxTk1oqfv7jPDDExFJXt/52BguHzcQX1ZG1M8VtzUiIKKj2dwBDi8CecaY1s01QNDmObf5rhRg+PDhkQxFRCTupaWm8NsvTI7JueK5RhRRbjNevrXW+D3a7FtyR+qVWGtL+vXrF8tQRUS6lbhPRLrPR0QkucVzImodKefz3+iXmOJnnVsREQlb3CYid5BCDWf2FflwbkSNeiIyxkw3xlQ2NDRE+1QiIt1W3CYi11KcGRD8Fbvbo85a+7S1tjQ3NzcWpxMR6ZbiJRH5CD5Kbh4wM2BbGafPGSciIgnMs+Hbbl/PLTj3A+UB5caYaTgzISyGkzevzjPGlAPL3H3LY9EsJyIiseFZInL7gDqs2biTlS6PfkRnMsZMB6YXFRV5cXoRkW4hLuaai3fGmL3A1jBf3hfYF8FwEoHK3D2ozN1DV8o8wlrb4Y2YSkRRZoypCmXSv2SiMncPKnP3EIsyx8tgBRER6aaUiERExFNKRNHXqWUqkoTK3D2ozN1D1MusPiIREfGUakQiIuIpJSIREfFUPC+Ml9Dc1WBLcCZuLSDBlyd3V7MtAArdnxWtM2D47dNhmRPxurizgJRba8sCtidded14ZwP7gT4473NNwPNJU2ZjzFSc+SvBKe8ma21lwD4JXWb3/259sHgiVbYul99aq0eEH+4bsSRg2yKgwOvYwizPDKDY7/c8YBNQ2pkyJ+p1ASqARZ19jxOtvO77XBFY9mQtM04CmhrkGiTN3zUwFagLLGckyxaJ8qtpLjrKcD68/FUA5R7EEgkF1plqCTg5PVM5p5cxlDIn3HVxl5j3BXkqqcrr1voesH61Pveb9FS/3ZKqzECZDfjWbp1a/jT/fUjAMhtjCowxFThJoraN3SJVtq6X3+uMnYwPnNpCQcC2POdyex9fJ8uSB1QDeQHbCwDbWs5QypyI1wUodR+BNaKkKq/7oVEeZLv/N99kK3N1YKzu9iV+/074MrvxBasRRaRskSi/akQR5n6zPONbiHVqEa3fsBOGG3eB+wgqlDIn4nVx+w8WBtmejOUtxZnh/jTW7R9K0jIvBZa4/RvAyVrgIvffyVhmIHJli1T5NVgh8nxw6o0IooAEW+bcWpsfZPNU3JVyW//YOigzIewTb9clzzpLkQRuD+U9JoR94qm8eUCNMaYU50PFB9TaUwNSkq7M1tp57peNamPMPNzY7KnBCklXZj+RKlso+3RYfiWiyAu2wF8yKgPucP8dSpkT6roYY2bYgFGBfpKqvH7fWgv8PoQxxpQbY3zutqQqcytr7WRjzBKcpsnlwEf9nk7KMrsiVbaIlF9Nc9Jprd+arbV3eh1LNLjNDW19w0tGbX2YLCA+BxlEjPu3PA9ngEIBTu0oLpvTkpkSUZS4H2ZJx/1PWmatnRbkuQ7LnCDXZZYN4R6IJCpvTcBP4OSilHn+H8xJVGbcUWVLrbXL3fd7FM41qA7YL2nKHChSZetq+ZWIIq/1P/NpQ3793qh4bC/ujHJOb76A0MqcENfF7biu6mC3pCkvnNa+H0o7f1KUuTUm63ezrrW23v2CVeP2HSVVmQNEqmwRKb/6iCLM7dyu4czmDh9u574HYUWE+w1yXmDHZKhlTpDrUgIUGmNm+20rBgqMMeXAMmvt4iQqb6uTd8QHey7J3mNw3udNbTxXwamBKslU5pMiWbZIlF81ouhYivOH7q/Y3Z6Q3Lb0cnv6dC9T/ZptQilz3F8Xa22ltXae/wNYAix3f28dwJAU5fVTwampboCTtUP/D5NkKnMNznRVweThDFyA5CpzoEiVrevlj/bNVN3xgfOHHDjlxRLiYMqPMMszA+c+k2K/x1ROn/6lwzIn6nXBaY4MvKE1qcrrxropSKwzkrjMFZw5xU8Bfjf2JkOZcab4mRFke0TKFonyaz2iKPGbPHIZzh93a4doQnHbeuvaeLrGWlvot2+HZU6k6+LW9uYBs3D+s1XiJN/l7vPJWt5NOLWFReGUJ8HKXIpT1v3upnobfNLThCqz+//2FjeWGTg1wKU4CWOx334RKVtXy69EJCIinlIfkYiIeEqJSEREPKVEJCIinlIiEhERTykRiYiIp5SIRETEU0pEIiLiKSUiERHxlBKRRJU7H511H6VBni9o67kox1VujIn7u7mNMRXGmLqOrpExZq67wFvciscY3b+/Je71rfA6nu5Ks29LLM3DmSZHQmCMWYQzr98cd1N7MxlPwZn/L57FXYzWmdB1mvulJK6SZHeiGpHESiXOUgozvA4kgczAWXZjsftY3taO1tqZ1lrjv82t9cV8tdG2zhssxnjgzpMG8T9bdtJSIpJYWYTzjf4WrwNJBH4f5GEtWe5OejmXgKUdos2r83bRVNw1l7wOpLtSIpJYKgeK/b6BisSDaag25CklIokZd3r9epyE1KZgAwmMMaVuh3JewH51xphiY0y1+3x164J9fp3Qm9pqEnRfu8Q9ziZ3iejAfWYEHj8wBvffFe4xAlerbO+8rccs9XuunFOrh7bu026Tpv81c/uWWpfuWOS+vi5g/06Xyb2mi/wGTyzxb4Lr6LxtvK9tXofAeILsG6kvNFNx+4fccs51H+3+nUrkKBFJrN0B+K/s2lV5OM1+dwBlOGuhLPJ7zHT3W9TG6x/AWSBtnvt74IfrDPe1S91j1QTu4+5XjbNKZUVHTTzuMavdY00DFgAVfqO27nC348ZV6L+GTAjm+L2+DGe9nckRKFNZwPF9bjlCOm+gEK6Dv9b3uYLT3+cu8e8fcr9AzLLW3unGrv7MWPF69UA9kvuB823T4q6EifOBYnFXd8X5QLFAqd9ryp0/zdOOU+rulxe4H36rbPptm+u3bYa7rbiDcxT4x+Zuq8NvxU53W3XrNr/zLWnrGgS5JnX+52jjOrXGcsbKmm0c87TytPf6SJUJpx/otHN0cN7AGDu8DgHxzAiyLa+9GEO4bnNxap95AX+DeV09th6hP1QjkpiyzjfrSqA0lCasEFX5/bu1Scu/zb912LOvg9haV7EsgZPflvOAuebUvVCWU8ul+ysjBG4TWB7ON3v/cy/FabacGex1kRLJMtlTo/g6XbsN8zr4v6ebgjwfjtYa3CzrtzKrtbbeavBCzOg+IvFCOU4NpxToTJNTUG18YIT7IVKDm4g49QE7OcjxagNiaO8eH3+txwy2fw1hfKh3UpfK5DanzXaP05VYO30dopQYpgJ3ApPdpsg5tp1h8hIdSkQSc9baGmPMYpyh3PE2WqmAUx+OJz8kO5FoOtJ6nAIg8AOvgNNrd9EQdpncWREKcL5ItMZZ3fYrQorDq+vg3z90h7W23hgzF6ffqTDa55bTqWlOvHIHTtPM7BD3b7PTO1LcpsKpuMnR/WYcdJRfF5oVWz9gT2v2cmsarR3ykVAbbGO4ZXIHMkzFucG20j1OsEQW9LxBxOo6tCfw/qEa3OZbd/ScBivEiGpE4glr7XJjzHKczuJAm8CZmwwnKUzFacaLOHfI8TxOfdOvx0mSrebgDEVuHbGVh/PhWUOI/UL+3G/eZTijw3w4I8Vaz73Y7SPpMvc8AGWtP621rf0hnS6TW4utB8rd49VzaqRhqOcN3C+i18FNluU4NZxQmteC3T/UmlxP6zOS6FKNSLx0xgeZayFOc005zjfjPoTxoR+iBTj3kCzB+TY/2b8vwjrDpifjfFgvwRnuXdNO7B1yP+CmcWoIchlOTSPSAxXuxEni5fg1f3WhTHNwagyLOPXehHzeQFG4Dnk4IyRLOtrR5cNvsIR7XWrcL0Ch1uwkAoy1cT8BsYhISIwxm4CZGnCQWFQjEpGkoiSUeJSIRCQpuIMtwm4yFe+oaU5ERDylGpGIiHhKiUhERDylRCQiIp5SIhIREU8pEYmIiKeUiERExFNKRCIi4qn/Dw4rxJxtXMeDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(conv_r)\n",
    "plt.yscale(\"log\")\n",
    "plt.xlabel(\"Number of iteration, $k$\", fontsize=20)\n",
    "plt.ylabel(\"Residual norm\", fontsize=20)\n",
    "plt.yticks(fontsize=20)\n",
    "_ = plt.xticks(fontsize=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaUAAAEuCAYAAAAupQ21AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3V9sHed95vHnJ1tNlWy6R0ycAhGgdY4auIDhxKIZxEVvUpiMdxs0UBzKAta53JC9TLK1GTVGYi+cdSh3s7lbUN7L+EKmkiXaNZCEKjY3QRyIlJQIAWqkZGID7KJVQxFb2EKtOO9ezDvSaHj+zMyZOTPvnO8HICi+Z+acd0TyPHz/zPuac04AADTBgborAABAjFACADQGoQQAaAxCCWgJM+vWXYe8QqwzqkUoAS1gZtOSpuuuRwHTvu6AJMmYfQeEzcw6kpadc4up8mlJM5K2JXUlbTvnLtRQRZnZvKS9Xq9vZiuSlpxze+OvGZqGlhIQvtOSlpMFvlts2Tl31jl3wTl3VtJiHd1lZjYr6cUBhywrugaAUAJaYNo5t50qW5S0kipbUSq8qmRmXd8K6kra7XecrztdeJBE9x0QNN8tNuVbQsnyLUlzybDy3XzXnXM25mrG9Vns131oZguSlL4OTB5aSkDYTkl6OVngw2df6yQes2nojLeXJZ2suxKo3911VwBoEv+GPSupI+mYc27RzJ6StKeo5dG0N87pHhMEpqTbIdRDV9Hkh8Zwzu2Z2Uzd9UD9CCXA8y2M2bgLyczWzWxd0V/ws5LmzaxTdJaYma0qCrusFnuMFWWR5zWARiGUgNseT41pTEm64EPovJkdG2XactmtLB+ibZpGvT1K6KMdCCXgtpdTX09LWoq/SE0a6Eqal3TJH3e2hjfTgbPaynqDzzkGtTvCa+4quqZLBc9HCxBKgJd8M/X31mjAzaarzrmH/LEbiu7DGfd4U783/zg8p5LH+JZV8vGh4vudctTpoqQzOY5Po5U04QgloLc59fmL3a+UcKuF4gfpZ4c9YQVjSnHL4g6+Pts9XmtK0aoKmUPJHzuusB3Y8sNkIJQAz8y6iTfsWUkbicc6kmZ8y6mr/X/R75rZtHOub9dT2WNKPnym+jx8QdESQ8n6TPvypppiPAncpwToVnfdll+FIF5dIPkGuZDoyusXBHXo1+pZ0v4WzqISY2Rm1jGzrfjG1YpNaXgrsVHT1FEPWkpAZFvSWUUtJEl6RNJy4g07OStvV727xuqw0auF5ltRS2a2rGicJ14LL/3GP6Woq7L0lRR86/K0f+2Oov/POUnrzrnzqWNnJZ0ruw4ID8sMATn5ltSL8UQHX3bdOXe4prosplcIz/kcC3Uv7+PXyOsVmpgwdN8BOaVbJX6GWi1jNb4uTVw2KK8pAgkSLSWgEN9CmVW99ynFdZmV1C3S2vGB2q1rnyVfhwXVuNcTmoUxJaAA30KJW0y1vpk65y6Y2XLBm2Vn6+y68+NOHQIJMVpKQEs0YWwoLzObT096wGQjlAAAjcFEBwBAYxBKAIDGYKJDTu9///vdvffeW3c1ACAom5ub/+ycu2fYcYRSTvfee682NjaGHwgAuMXMXs9yHN13AIDGIJQAAI1BKAEAGoNQAgA0BqEEAGgMQgkA0BiEEgCgMbhPCZVbu7yjZ//m57r+1s19j5kkJ+mASb/tsQxjr8fjsrvM9I5zOtI5pCcfvU8njh+p7BoAjAcLsuY0MzPjJvXm2UHh0hSDwqvXYwQaMB5mtumcmxl6HKGUT9tDKYTgqcKgAOscOqhnPn0/wQWMgFCqSFtCaVLDZxTp4KL7EMiOUKpIiKG0dnlHL3z/Ne3s3eg7doPREVpAf4RSRUIIJVpBzcQEDUwyQqkiTQ2lUINolNl3bUFYYRIQShVpUiiFEkRVzXRLdksOmmHX67EQMEMQbUIoVaQJobR2eUdL3/mZ/vU3vx3r67btTbJXqDU5wGhRIWSEUkXqDKWn167qpVffqLzriinQt4UQXG37YwHtRChVpI5Qenrtqr796huVPDcBNJomhhYhhSYilCoyzlAqO4wIoPEbNu41DoQUmoBQqsg4QqnMMCKImqnOsCKkUAdCqSJVhtLa5R39xepP9ZsR3pkIobDVcaMzIYVxmJhQMrNpSTOStiV1JW075y5kOG/eH3/Mf15xzp0fdl5VoTRq6+hzDx/VcyceKLFGaII6WlSEFKowEaFkZnGYzCXKViUtOee2B5w3ryi8LvmvO5I2JS07584Oes0qQumJF3+sH23t5j7v4AHphZMP8qYxgcbZoqL1jTJMSigtS7qYbOGY2aykRefcyQHnPeWcO5MqW1AUcDboNcsOpblv/lC/+Kc3c51DGCFtHCFFCwqjmJRQ2pI0l2wV+VbP9X7h4h//W0mPOOf2EuVdSVuSjg1qZZUZSnlbSIQRsiKk0DStD6U4fCQdToaLf8xpQLiY2XVFoXQpUTbWUMozhnRA0jdPEUYoruqQIqAwzCSEUlfSVq8WkQ+luSwTHhLnLCgaUzo86LgyQmnt8o6+cO5KpmP/+NiUXvr8H430ekAaIYVxm4RQmpa0WWIobUo6lx5r8o8tSFqQpKNHjz70+uuvF6+4pO6XX1GWVeuYUYdxqTqkmCwBQilHKPnQOZmcxdfPqC2lrN12BBLqVNUK9LSgJlfWUDowjspUyY8tjXJ+V9FsvaGBVAYCCSE4cfyILn/1k/rWqQd1pHNIUrSn1ajiP4F39m7oC+eu6MFnf6C1yzujPzFaI+SWUjzR4Viv2Xfp8gHPsyrp8+nJEv2M0lLKMpbEGBKarNfNvGVsukj3Xvu1vvtOujUl/GSPWXSbwyYs+GNXFE1uGBpesVFC6cN/+YpuDhhMuvuA6e//658Wem6gLmV29dG9116T0n13QdESQ0nTvnygxGy7ZCtr1odaJQYFkiT91cmPVvXSQGXK7Oqjew+ht5Q6klZTywytKxoj2k4cc8cSQn6ZoSlJySbPlKJW1+Kg1xylpXTvl1/p+9i77j6g1577D4WeF2iiMltQdO+FL2tL6e5xVKYqzrk9M1uKlxtStLBqr+64qfgfcZD1ecrM3XhlW/7sR+p6aaASJ44f0YnjR0qZbr5346a+cO6KvnjuCt17LRd0S6kOVbWUfvWNTxWtEhAUWlCTaVLGlAAEJjkGdfjdB0d6rrgFxdhTexBKAGpR5gQJwqk96L7LqaruO26YBSJldO/Rrdc8E3GfUh2qCiWJcSUgqYxw4r6n5mBMKUB0OwC3ldG9x31P4aGllNMoLaX7v/o9vfn2O30fPyBpm9YSMBDde2GipdRAX//M4DGj3ypaRRxAf2XM3mNiRHMRSmOU5a+yrLvRApOOcGonQmnMPvfw0aHHfPzr62OoCdAOZYXTF89doaeiAQilMXvuxAO6e8ho7T/+y9ua++YPx1MhoCVGnRjhFPVU0GqqFxMdchp151kp275KEnsrAWUoOjGCyRDlYqJDg504fkR/fGxq6HE/2trVEy/+eAw1AtqraPce4031oKWUUxktpdjHv76uf/yXt4ce9/vv/R395Ctj2a0daD1aTvVgRYeKlBlKktQ9/UqmpfwPmPTNxx9szS/EEy/+WD/a2i10LiGNMjy9dlUvvfpG7q3cCadiCKWKlB1KWceXYm0YZxq23FJehBSKGuVGXMIpn4kJJTObVrQl+raiTf62nXNZtkMvdF7ZoSRFf7HluT/pwx94j9a/9IlS6zAuZQdSGgGFIoqGk0l6gsWUM5mIUDKzrqSV1Hboq5KWeuw+O/J5UjWhJOUPprsPmP7q5EeD+iut6kBKY+V15MV4U3UmJZSWJV10zp1PlM1KWnTOnSz7PKm6UJLyd+VJ0rvuPqDlz34kiF+GcYeSJN1t0t8/z3qCyIdwKt+khNKWpLlk68bMOpKuO+f63jZX9Dyp2lCKZZ2VlxTCWFMdoRQL4f8HzUM4laf19yn5EOlKumMKl3Nuzz/eLfO8cfrJV+Yy3ceU9KOtXd375VdYJqWPH23t6g9O1xeKCNMo9zixbFExwYaSpCnpdpj00C9ccp9nZgtmtmFmG9euXctd0SJe+vwf6VunHsz9Dfr2q2/o3i+/ouP/hRv+0n7j6m2tIVxxOH3u4aPKunIRyxYVE2z3nZ89t9mru83MnKLuuX2z6YqeFxtH913a3Dd/qF/805uFzm1St9WogZDcmbdIF2e/5wLyoEuvmNZ3302S9S99Ind3Xizu1rv/q9+r/a+1MoPgJ1+Z06++8Sn96huf0u+/93dyn89K7CiKLr1qBR9KfoxobOfVJe7OO1jwO/bm2+/oC+eu6EM1jztV0UKJA+rDH3hP5nNGaWUBUrFwoktvuJC77zqSrks61msWXbp81PNidXTfpa1d3tGTq1d087ejPc97fucuff0zD9TSnfCRr31P/+9f+28N30vWQMvaTfitU+1Ztgn1K7Js0SR16U3SlPCTzrlLibKuojGjw2WfJzUjlGJrl3f0pXNXNGI21X5X+oe+/MrQX+S8LawsoWeSfsnYEkpUZLyp7t+/cZmUUFpRFCRnE2Xzkk4NuXm20HlSs0Iplnc1iGEOv/ugvvZn4f/1lmUyBBMeUIUi4dT2VtOkhFJH0mpquaB1RSszbCeO2ZS0HIdQlvP6aWIoxcrq1ks6YNJ//HiYf8VlCWtCCVXK26XX5lbTRISSdGuK9ylJFxXdY3QpOaXbB9AvFa1rdzbref00OZRiVYRTLJSQyjplnFBC1Wg1RSYmlMYthFCKrV3e0TN//XPt3ci/LH8eTeruy3NP1++96y797Nl/X3GNgEjecDp08C49/1g9E5GqQChVJKRQSqqy9dTLOFtURW8uppWEOuTp0jv87oO6/NVPVl6ncSCUKhJqKMXG1XrKKuu09LInc/zuXaa/+/qflvZ8QB55Wk1t6cojlCoSeiglrV3e0env/kw3xtV8ahBaSWiCrK2mNkyAIJQq0qZQSpqkgCKQ0CST0moilCrS1lBKa2NI0WWHJnvw2R9k6lYPtdVEKFVkUkIpLfSQYmt0NF30O3ZVN25mW34rtFZT1lC6exyVQfhOHD9yxw9/0yZM9PLhD7xH61/6RN3VADKJf7+yduXt3bip09+9ese5bUBLKadJbSllVXeLqkn7RwFFtXHaON13FSGURjdqcBE8mAR5JkCE0D1NKFWEUAIwTllbTU0fY2LnWQBogedOPKD/nmEjwbbsbEsoAUDDxbvcdg4NDqZ4Z9uQg4lQAoBAPPPp+2UZjnvp1TeC3W6dUAKAQJw4fkRPPHx0aDA5RVPLQ0QoAUBAso4xXX/rph589gfBtZgIJQAITDzG9LmHjw48LsTJD8Gu6OB3jp2RtK1o59jtLDvHmtm8P/6Y/7zinDtfZV0BoArxvUmDtnVxicebfi+TFGgomVlX0rJzbi5Rtmpm28657QHnzSsKr/P+646kTTObSm6VDgCheO7EA/rfP/2/Q5f8CiWYQu2+W5S0kipbkbQ85Lyuc+5S/IVzbs+fk34uAAjGM5++X4cO3jX0uBBm5YUaSvOSLqXKNnx5T75VdMp/TrrgH++WWkMAGJMTx4/o+cceGDr5IYRZecGFkg+VrqTdZLlv9fQNF/9413/kfc0FM9sws41r167lrzQAVCzr5Ifrb91s9MSH4EJJ0pR0O4R66Bs6zrnDye47b1bS3qCxKOfcWefcjHNu5p577sldYQAYl+dOPDA0mJq86kOIoZTufhvVoqTnS35OAKhNyMEUYiiVxswWJO06587UXRcAKNNzJx4YulZeEyc+1BpKZtbN8dFJnTtSi8mPPS0mp5UDQJsMWyuviRMfartPKb7XKMcpFyWdUXSzrBSNLd0aV0qEVN+xoZRlSY/keH0ACMqJ40e08fruwJtr44kPTbl/qbZQ8hMLThY4b8/MtrV/bGlKQyYsxMxsRdLSgMkSANAKWVZ9aNKNtaGOKV1QtMRQ0rQvH8iPIy0nw8vMZrlPCUBbZZn40JTxpVBDaUn7W1mLvlxS1J1nZls+hOKy+ObajplN+49ZSSeztLAAIFTDJj40ZXwpyLXvfBfekpktKxpritfCSwfLVPwPP+a02ucpCSQArffMp+/XF89dkevz+PW3bmrt8o5OHD8y1nolBRlKkuRvgk3fCJt8fE/S4dTXWTZtBIBWyjLx4Sv/62qtoRRq9x0AoIBh40tvvv1OrTfVEkoAMGGGzbKrc9IDoQQAE2jYpIcXvv/a+CqTQCgBwAR65tP3D3x8Z+/GmGpyJ0IJACbQieNHht67VMfYEqEEABOqiWNLhULJzP6TmT1mZr9XdoUAAONzpHOo72N1jC0VbSnNSDov6bqZ/cLM/oeZfYaQAoCwPPnofQNv4Bz32FKhUHLO/blz7oCkj0k6K+mYpO/ozpD6aIn1BABU4MTxI3piyNjSOLvwRhpTcs5dcs694Jz7ZCKkLsefzeyimb23jIoCAKoxbGxpnGvilTrRwYfU45LO+ZBalXSJbj0AaLZBY0vX37o5tnoUnejwoO+i+wszu7fHIU6S/DbjpySdLlxDAEDlnnz0voGPj6sLr2hL6S8VLW7655K2fDfd835G3mOKuu8k3Vo4lYVQAaDBThw/okMH+0fCuLrwiobSup/s8AeSPqyom25O0v9U1Cp6XpJ8UN0r6dejVxUAUKXnH/tI38fG1YVXNJQu+MB50Dm37Zw745ybcc5NOec+5py74o97SFFgbZVT3dv8Bn0LftfYBb9ZX97n6Pit0QFg4tW5ZUWs0H5KzrlfSjptZo8MOe6ThWo1hN+6fNk5N5coWzWz7Zw7yC4rsREgAKC/cWwAOOqU8L8tqyI5LUpKt3BWFIVMJj7YCCQASBi0evg4xpVCXftuXvt3nd3w5VnNSlovrUYA0AKDVg8fx7hScKFkZh1JXUm7yXK/3XncAhr2HLOSXs7xmgtmtmFmG9euXctZYwAIR93jSsGFknyXWxxCPQwNJUmdAefv45w76ydyzNxzzz1ZTwMA5BRiKHVGOdnM5p1z58uqDAC0zaAbS6u+iTbEUCrMd/1lbiEBwCRyAx6rerJDoSnhZcky/pOwm+xyM7NcXXDe4865sznPAYCJcqRzqO+WFVVPdqgtlOJ7jXKcclHSGUnxfUhTSrR6fCtIicfTrzetaIYeAGCAJx+9T184d2X4gRWoLZT8Ta4nC5y3Z2bb2j+2NCVpb8DNszOSjpnZqUTZtKSumS1LushYEwBEM/AmLpRGdEFRyCTvVZr25T316rYzs6ckfcw5t1R6DQEAuYU60WFJ+1tZi75c0q117bbMbGHA87yvisoBAIoJsqXku/CW4m43RfcmLffouuu5jJAfz1qS9LikeFHWFb/NBgCgJkGGknRrn6a+IeJn5h3u89i2opbVYjW1A4BwDboXqerN8ULtvgMAVGTQvUiD7mEqA6EEALjDoHuRjnQOVfrahBIAILMnH72v0ucnlAAAmTV6kz8AQLtUveDqMIQSAOCWF77/Wq2vTygBAG7ptxCrNHir9LIQSgCAWwbdhzRoq/SyEEoAAEnReNKg+5DGsVU6oQQAkFT/eJJEKAEAvLrHkyRCCQAg6em1qwMfH8d4kkQoAcDEW7u8o2+/+sbAY8YxniQRSgAw8QYtwCpVv95dEqEEABNu0AKspurXu0sKdj8lM5tWtCX6tqJN/radc323Q+9x7ilJv1a0++xKjw0CAaD1ho0lPfHw0bF13UmBhpLfOXbZOTeXKFs1s+1h4WJm85LmnHOLibIVseEfgAnz9NrVgWNJhw4e0HMnHhhjjQINJUUBspIqW5G0LOlkv5PMrCPpRefc4UTZvKTZKioJAE21dnlHLw2Z3PD8Yx8ZU21uC3VMaV77t0Lf8OWDnJZ0NlngnDsvaa734QDQTs/+zc8Hrt7QOXRwrN12seBCybd2upJ2k+XOuT3/eHfA6QuSLqYLM3T5LZjZhpltXLt2LX+lAaBBnl67OnRyw7juS0oLsftuSrodQj10FU1+6KUjadvMFhSF2pSkXd9a6ss5d1a+hTUzM1P1FvUAUJlh40jS+Cc3JIUYSp0iJyVaUF0fMnH5splNJcsAoI2yBNLnHj469skNScF1342gX5idUzRBAgBaK8vEhs6hg7UGklRzS2nI+E/abrLLzsw6A7rwetlOfZYkOecumVnHzLrcqwSgrYZNbKhzHCmptlCK7zXKccpFSWd0O1SmJN0RUv6fPYPFObdnZkqekzJoLAoAgjVsYoNU7zhSUm2h5Fslfe8pGnDenplta3933JSkvSGtnVurP/R5DABaJYRxpKRQx5QuKFpiKGnalw+y4o+7xS85NCzMACA4oQWSFG4oLWl/K2vRl0uKuvPMbMtP/46d1f7lhJYlfb6SWgJATUKZ2JAW4pTwuAtvycyWFY01xWvhpVs7Uz3Om/Nr3W1JOubPy7SQKwCEYO3yjv7zyz8NYmJDWpChJEWz5rR/qaHk43uSDvco3xaLrwJoqafXruqlV98YGEhScyY2pAUbSgCAO2UZQ5KaN46URCgBQODWLu/omb/+ufZuDJ72bYpaSE0NJIlQAoCgrV3e0envXtWNm+8MPO4uM/23xz/ayC67JEIJAAIVT2h4xw0eQTIpiECSCCUACFLWCQ1Scyc19EIoAUBAso4fSWGMIaURSgAQgDxhJEmH331QX/uz+4NpIcUIJQBouDxddaFMaOiHUAKAhsrbOgppQkM/hBIANFDWqd6xePwo5ECSCCUAaJysU71joY4f9UIoAUBDTMpkhkEIJQCoWZGxo9CmemdFKAFATfKGkdTO1lFSsKHkd4ydUWKL8yz7IpnZrG7vPvs+SVvOubOVVRQAUoqEUehTvbMKMpTMLN7Uby5Rtmpm24O2NfdBJufcmUTZvJktEEwAqlYkjCTp0MG79PxjD7Q+kKRAQ0nRJn0rqbIVRVubp7dJv+M859wdG/w5586b2aqirdIBoFRrl3f0wvdf087eDZmU6QbYpLZ316WFGkrz2h9KG5LWh5w3Y2bdHq2pTmk1AwD1bhXlCaRJC6NYcKFkZh1FY0i7yXLn3J6ZqU/oxC5IWjezk347dZnZvKTVIa+5IGlBko4ePTrqJQBosaJddLFJDaNYcKEkaUqKQqjP411Fkx/2cc4t+YkOm2a2FB83bDzJP35WkmZmZvK2vgG03KhddBJhFAsxlEbqanPOPWRm64rGny5JeqSUWgGYOKN20UmEUVqIoTQS3xW3pKjFtaqo1TQ3aNYeAMRGbRXF5xzpHNKTj95HGKXUGkp+andWu8kuOzPrDOjC6/d6K4qmkm/7rz8kH0ySDud5LgCThVbReNQWSvG9RjlOuSjpjG6PF01JuiOk/D97tnjix5MtIh9qc2a2aWazWW6+BTA5yhgrkgijPGoLJR8Og+4p6nfenplta//Y0pSkvQHdcDOStvo8ttLj+QBMoH5BlCeQ6KIrLtQxpQuKQuZSomzal/czKAQ7Q84F0GJlBFGMVtFoQg2lJUVjQcmp3Iv+Q9Kt7rpNRWNIZ51z2/4+pju66Xw34vuY6ABMnjLGiWgVlSvIUPJdeEtmtqxorCleCy8dLFOp8xbNbMGf92tfvOecW6q+1gCaoKxxIolWURWCDCVJ8isyXBrw+J56zKhj4VVg8pTVPUerqHrBhhIApMXh8w97N/RvDx2UmXT9rZsEUUAIJQDBSofQm2//RjffiWKHcaIwEUoAgtKvK67oAqhpjBPVi1AC0HhlTtlOo1XULIQSgEZIdsV9sHNIf/KH9+j//N01gmjCEEoAatOvBbSzd0PffvWNW8cRRJODUAJQqV4z4vbeurlvYkIZwXPwgOnf/O7dt54/fq0PEkTBIJQAlCrrjLiyJibQAmoXQglAblnHf8oKnjSCqL0IJQBDDWr9VDH+0wtBNBkIJQB9Wz7DuuDKlgyeZB0YE5ochBIwAfKETrrlU2UIJScmEDyQCCWgFZoaOhIz4pAPoQQ0TDpgnnz0PklqbOj0wvgPijLnqhqWbKeZmRm3sbFRdzUQiEEtmF6Bkw4YKWppyHRHWd3S3W6M/2AYM9t0zs0MPS70UDKzeUUb9WXaztzMphVtpb6taHPA7aznSoRSG+VtmWQ95k/+8B59Z3NHN26+0/e1mxg4vTD2g1FNRCiZ2ayibdFPZgkWv/X5inNuLlG2Kmkp63bohFJ1soRD2WW9giNLUGQ5ZtRdTceJlg+q1upQ8uGyJGnTf17MGErLki46584nymb9+SezvDahVI21yzs6/d2rQ8Oh7LKQgmMUhA7qljWUgpzo4Fs1i5JkZks5Tp2XtJIq25C0XlLVUNAL339tXzfXzd/uj4uyy0IIpKwtN0IHbRBkKBVhZh1FY0i7yXLn3J6Zycy6/brwzGxB0oIkHT16tPK6TqJ/2LtRdxUqMawl1q9Flx6/kYaPcRE6aIOJCSVJU1IUQn0e7yqa/LCPc+6spLNS1H1XSe0m3Ac7h7RTUzClg6OsMaVDB+/SZx86sm/yw7AJE/0ChtDBJJikUOrUXQH09+Sj99UyptQrOMqafZen9ULgAJFJCiU0WPymPO7Zd4OCI0tQECZAuWqdfedn0WW126vrzcy2lGH2nb8/aVPS4fTzmJmTNJdlBh+z7wAgv8bPvvOBtJzjlIuSzozwkvF40ZSkW6HkJ0AkHwcA1KS2UPIz3TLdG1TS6+2Z2bb2jy1NKVoRglACgJodqLsCY3ZB0RJDSdO+HABQszaE0pR6zKwzs46Zbfl7jGJL2t86W/TlAICaBTn7zo8DnVZ0b1FH0rKZzUlaTy4hJH9vUsx34S3Fyw3585fpugOAZggylPzsuYGtG3/M4R7llyRdqqhqAIARtKH7DgDQEkGuEl4nM7sm6fW669HH+yX9c92VKEFbrkNqz7W05TokrqUu/845d8+wgwilFjGzjSw3pzVdW65Das+1tOU6JK6l6ei+AwA0BqEEAGgMQqldztZdgZK05Tqk9lxLW65D4loajTElAEBj0FICADQGoQQAaAxCCQDQGEEuM4Qw+Y0WZxTtXdWVtJ1lY8Wi51VphGuZ98cf859XUus1jl0Z/79+Pcpl59xiBVXMWofC1+HPPSXp15Lep+j7UtuamCP8fM2DeXLfAAAFh0lEQVQq2vlAiq5jyzkX1mQI5xwfgX4o+uFbkDQbf8543rykpyStSFqXND+GunYVLZibLFuV1K3ivIZey7yk6cTXHUlbkhZCu5Yez7MiaTXE6/Dfl5X09YR2Lf79YDZVNl/nz1eh66+7AnwU/MYF9saoaJfh+VTZ7LA3sqLnNfRanupRthD9bRjWtaSO7/qfvRC/Jx1J11Nl84paGKFdS88grfP7Uuj6664AHwW/cYG9Mfrg66bKOsNet+h5TbsW//impE6qvCvJZfmLvinX0ufnZ6HmUCr687WsqNsxXV5nS7zotWz2qrdSf7w2/YOJDuGa1/4tODZ8eU++3/+U/5x0wT/eLbWGd75uV9JustxF24v0fd2i51WpaJ38413/0Qhl/P/6MYyXK6lgRiNex4KivdXu4GoaTxrxWi5IWvfjUfHzzStqxQaDUApQgG+MU4nX76VffYqeV6XCdXLOHXbRfl5Js5L2anoTLOP/tzPg/HEZ5To6krbNbMHM5uPPpdcwu1F+vpYk7UnaNLOn4utwgU10YPZdmLL84PZ8k3PO7dv4UNW/Me7brr7i86pUdp0WJT1f8nNmNdK1mNm8q3nmoFfoOhJ/vHWTb9xmtmxmUzW9mY/0PXHOPWRm64q6JS9JeqSUWo0RLaUwtemNcWKZ2YKkXefcmbrrkpdvrdfdQhpVv9+jc4re1IPjf6aWJM0p+uN0s45u7lEQShNunG+MPcayKj2vSqPWyb9RLDrn5kqq0ih1KXItj7ua7xVLK3Ad26nPkiTfxdqp8828yPfEzFYkXXDOXfLfmw8purbNsutXJbrvapbzB3832WVnZiP15yfeGB8q+hwZxb/0U0r8dZ34xevXbVj0vCqVVadl1d+1Uuha/ED6RrVVy6XQdTjn9sxM6t/i69sNXqGi35OOdOcEDf/eMGdmm2Y227Q/IvohlGrkQyFPN8FFSWcU2Buj/+Xf1v7ukikNGMsqel6VyqiT/4t2qe4JAiNcy4ykY2Z2KlE2LalrZsuSLo5zrGnE78mtFRP6PDZWI35Ptvo8ttLj+RqLUKqR/wE7WeC8EN8YLyj6xUnOPpv25VWcV6XCdfLdpcvJ75GfVr1d0wy83NfSawKAmT0l6WN+Blgdin5PVtLH+ZZgXTMipWLXMui9pDPk3Gap+0YpPop9KPplWkiVzSvDDYyK7s1I35w3my4rub4d7V+BYj35muqxukSW82r4vy96LfP+/3468TGrepe0KXQtPZ5nOcvPXtOuIy7rcV7lS29VcC0r2r/MUFc9bg5u8geb/AXKd9WtusRAuZ8Kuuj8X3j+mE1FP5Rnfdm8ohZVckxgStJJV/FimolFLy8q+mWJB2ST1/RLRS24s1nPq0Pea/FfX+/zdNvOuWNV17mfot8X/1hX0WyvxxW9UZ5VFLLp+7EqN8LPV3wNW4oWyl0N7ecrUb6g6Bp+7Yv20t+zpiOUAtamN0YAkAglAECDcJ8SAKAxCCUAQGMQSgCAxiCUAACNQSgBABqDUAIANAahBABoDEIJANAYhBIAoDEIJQBAYxBKAIDGIJQAAI3BJn9AwPy2C7OKto045pxb9Bvu7Umac87l3kQSqBOrhAOB8luRPJ7YK2vdP3RSUVCtSjrsat52HciD7jsgXI+nNnCbUrSn1p5z7ryilhOBhKDQUgICZWadZOiYmVPUZVfrrqnAKAgloAXMbFbSunPO6q4LMAq674B2mJN0qe5KAKMilIBA+Zl3sVlJG4nHOr71BASFUAIC5ANny8y6Zjbti5OTGhYYW0KIGFMCAuRbSUuSNn3Ry5KWk18z8w4hIpQAAI1B9x0AoDEIJQBAYxBKAIDGIJQAAI1BKAEAGoNQAgA0BqEEAGgMQgkA0BiEEgCgMf4/XLiKOx7Gf9YAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter([x[0] for x in conv_x], [x[1] for x in conv_x])\n",
    "plt.xlabel(\"$x$\", fontsize=20)\n",
    "plt.ylabel(\"$y$\", fontsize=20)\n",
    "plt.xticks(fontsize=20)\n",
    "plt.yticks(fontsize=20)\n",
    "_ = plt.title(\"$x_0 = (0, -1)$\", fontsize=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Возможные улучшения метода простой итерации\n",
    "\n",
    "- Наискорейший спуск\n",
    "- Метод Чебышёва"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Улучшение 1: метод наискорейшего спуска\n",
    "\n",
    "- Пусть мы можем изменять значения $\\tau$ на каждой итерации, то есть \n",
    "\n",
    "$$ x_{k+1} = x_k - \\tau_k (A x_k - f). $$\n",
    "\n",
    "- Возможный выбор $\\tau_k$ такой что, $\\tau_k$ минимизирует нормы текущего вектора остатков, то есть\n",
    "\n",
    "$$\\tau_k = \\arg\\min_{\\tau > 0} \\|A(x_k - \\tau (A x_k - f)) - f \\|^2_2 $$\n",
    "\n",
    "- Используя технику, аналогичную той, что использовалась для получения нормального уравнения, получим выражение для $\\tau_k$\n",
    "\n",
    "$$\\tau_k = \\frac{r_k^{\\top}r_k}{r_k^{\\top}Ar_k}, \\quad r_k = Ax_k - f$$\n",
    "\n",
    "- Такой метод называется **методом наискорейшего спуска**.\n",
    "\n",
    "- Однако он сходится так же, как и метод Ричардсона."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Улучшение 2: метод Чебышёва\n",
    "\n",
    "- Другой метод нахождения $\\tau_k$ заключается в рассмотрении следующего выражения\n",
    "\n",
    "$$e_{k+1} = (I - \\tau_k A) e_k = (I - \\tau_k A) (I - \\tau_{k-1} A)  e_{k-1} = \\ldots = p(A) e_0, $$\n",
    "\n",
    "где $p(A)$ – **матричный полином** (простейшая матричная функция)  \n",
    "\n",
    "$$ p(A) = (I - \\tau_k A) \\ldots (I - \\tau_0 A), $$\n",
    "\n",
    "и $p(0) = 1$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Оптимальный выбор шагов по времени\n",
    "\n",
    "- Вектор ошибки можно записать в виде\n",
    "\n",
    "$$e_{k+1} = p(A) e_0, $$\n",
    "\n",
    "тогда \n",
    "\n",
    "$$\\|e_{k+1}\\| \\leq \\|p(A)\\| \\|e_0\\|, $$\n",
    "\n",
    "где $p(0) = 1$ и $p(A)$ – **матричный полином**. \n",
    "\n",
    "- Для получения скорейшего уменьшения нормы вектора ошибки, нам нужно минимизировать\n",
    "\n",
    "$$\\Vert p(A) \\Vert$$ \n",
    "\n",
    "по всем возможным полиномам $p(x)$ степени $k+1$, таких что $p(0)=1$. \n",
    "- Мы будем использовать спектральную норму $\\|\\cdot\\|_2$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "##  Полиномы, наименее отклоняющиеся от 0\n",
    "\n",
    "- Важный частный случай: $A = A^* > 0$.\n",
    "\n",
    "- Тогда $A = U \\Lambda U^*$, \n",
    "\n",
    "и \n",
    "\n",
    "$$\\Vert p(A) \\Vert_2 = \\Vert U p(\\Lambda) U^* \\Vert_2 = \\Vert p(\\Lambda) \\Vert_2 = \\max_i |p(\\lambda_i)| \\overset{!}{\\leq}\n",
    "\\max_{\\lambda_\\min \\leq \\lambda {\\leq} \\lambda_\\max} |p(\\lambda)|.$$\n",
    "\n",
    "- Последнее неравенство является приближением: мы делаем <font color='red'> ключевое предположение </font> о не использовании распределения спектра между $\\lambda_\\min$ и $\\lambda_\\max$ для получения более быстрой сходимости.\n",
    "\n",
    "- Таким образом, нам нужно найти такой полином заданной степени, что $p(0) = 1$ и отклонение от 0 является минимальным на отрезке $[\\lambda_\\min, \\lambda_\\max]$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "##  Масштабирование задачи с $[\\lambda_\\min, \\lambda_\\max]$ на $[-1, 1]$\n",
    "\n",
    "- Мы можем сделать аффинное преобразование отрезка $[\\lambda_\\min, \\lambda_\\max]$ в отрезок $[-1, 1]$:\n",
    "\n",
    "$$ \\xi = \\frac{{\\lambda_\\max + \\lambda_\\min - (\\lambda_\\min-\\lambda_\\max)x}}{2}, \\quad x\\in [-1, 1]. $$\n",
    "\n",
    "- Теперь задача свелась к задаче поиска полинома, наименее отклоняющегося от 0 на отрезке $[-1, 1]$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Точное решение: полиномы Чебышёва\n",
    "\n",
    "- Точным решением этой задачи являются знаменитые **полиномы Чебышёва**, которые можно записать в виде\n",
    "\n",
    "$$T_n(x) =  \\cos (n \\arccos x), \\quad x \\in [-1, 1]$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Что вам нужно знать о полиномах Чебышёва\n",
    "\n",
    "1. Это полиномы!\n",
    "\n",
    "2. Можно выразить $T_n$ из $T_{n-1}$ и $T_{n-2}$: \n",
    "\n",
    "$$T_n(x) = 2x T_{n-1}(x) - T_{n-2}(x), \\quad T_0(x)=1, \\quad T_1(x)=x$$\n",
    "\n",
    "3. $|T_n(x)| \\leq 1$ на $x \\in [-1, 1]$.\n",
    "\n",
    "4. У них есть $n+1$ **точка альтернанса**, где максимальное абсолютное значение достигается (это необходимое и достаточное условие для **оптимальности**: об этом говорит теорема Чебышёва об альтернансе).\n",
    "\n",
    "5. Корни записываются как \n",
    "\n",
    "$$n \\arccos x_k = \\frac{\\pi}{2} + \\pi k, \\quad \\rightarrow\\quad x_k = \\cos \\frac{\\pi(2k + 1)}{2n}, \\; k = 0, \\ldots,n-1$$\n",
    "\n",
    "Мы можем их нарисовать!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Interval $x\\\\in[-1.1, 1.1]$')"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEICAYAAAC55kg0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJztvXmUHNd93/u9vW8z07MCgx0DcCdFcTCiRIlaLINRZL3IfjZI2lIkO1YEKfJJ4vjF1HGW915iJwpoS8dSbFmkbCeKpfhRpB07tjYSlC1RqwWClCkBIMEZAsQ6e0/39F5V9/1Rdat7Zqq6q7qW7ur5fc7BwUz1VtN9u373t31/jHMOgiAIYnsS6vYJEARBEN2DjABBEMQ2howAQRDENoaMAEEQxDaGjABBEMQ2howAQRDENoaMAEEQxDaGjIABjLFVxli2i6+fZYzN+vyaxxljx31+zaOMsYf8fM1+hdZssOiltU9GoEOCuvhakOWcP2rnAYyxaTvvw+b7c85P2nk9whm0Zq2tWafr2sp9emntkxHonA91+wSsIhYgY+whty4EjLGjAH4DgKXdp937E55Aa7bNGvRiXff62o90+wR6Gc29fhrAKQBHATzBOf8oY+wRANOMsWcBnNSOPQT1SzYH4H7tKZ4GcBLAlPb7I2IHwBib5ZwfYow9rt2+AuB+znnO5b/hEQCP290xtYNzfpIxNgWLC9vu/YnOoDVrjpU16MW67vW1T55Ae6YBnOCcHwJwHAA45x8CMMc5P6J9maYBvE67z4cAnGh67DLn/H4Aj0H7omn3P6091/2c8yMAHoG6W2iLFk88yhg7of3/uMn9TgD4aC+5noQv0JolLEOeQHtynPM57ecVxljWYOfzIIApxthTBo99GAA4509oC1zc/zFAdxU/BHVnNYc2aDu9U5zzHGPshPaFNn2c27s0IhDQmiUsQ0agPSsW7/cxzvkT4hdt4W9+7GltR3W0aTf2Uai7rSlY2FWJL4jmXp7Sjpl9oaYMKhBy7dxsLQZr5ro+Sl/SnofW7EZozbaAjEDn5JpK8h6D6k4/Aeius9EifwzqDkp80WYAnNZ2SDNWXrTpNY8BeFa8Huf8tMHdfyB2dXZwOxZL9Ay0ZoktUE6gc05CXdAntMX8OGPsWS3xZvjl0HZdx6HGUgHgiwCOaY+5z+LrHgfwALQvrOaaG+6qOOcPa9UV0xaf2zLa694H4D7tZ3H8WaPXM7s/4Su0Zs3XbNbCfSyvayvP2SswGirT/2iLdwaqu2zoWjPGHupkB2b2eia7PKP7uva6RP/g95q1ek5W17XF5+uJtU/hoG2AtnBdW7ytYIwdpcoOwil+rlkr9PO6pnAQIci50ZRj54vSi64xEShcWbNWcNsA9NLap3AQQRDENoY8AYIgiG1Mz+cExsbG+IEDB7p9GkSf8uyzzy5xzsf9fl1a14SX2FnXPW8EDhw4gFOnTnX7NIg+hTF2sRuvS+ua8BI765rCQQRBENsYMgIEQRDbGDICBEEQ2xgyAgRBENsYV4xAK50PxtgxtmmeptExgiAIwn8cGwGt881sQMQ0oHfb5Zg6Mm7LMafnQBAEQXSGYyOgXczNtMEfBCB0vOegjrszOkYQBEF0Aa9zApuHVIyaHHPMYqGKtVLdjaeyxVqpjq/+6DoUJVjyG5Ks4AcXVnDuer7bp2KKonD8xXNXUKxKHT/H575zAf/7h1ddPCuC6D7X1yr4+JMvYnZx3fFz9WRimDF2nDF2ijF2anFx0dJjfvm//wD//i9/5PGZbeTM1Tze/vG/xYc//yy+eOqSr6/tlIf+7O9x/2e+i3d96ls4P1/o9ukY8rcvLeBXH3sev/Wlsx0/x+e/dxFfeeGai2dFEN3nSq6M//r1l/HqSsnxc3ltBHIARrSfswCWTY5tgHP+KOd8hnM+Mz7evvO5Ksk4cy2PuSXnVtEOn/vOBVQlBXfsHsLHn3rJ0Y7VT85ey+N/PXcF9x/Zg1Q0jBNfPdftUzLkz05fAQD86d+9iucvdTYdUOEcoRBz87QIoutU6zIAIBkNO34uT4zAphF2U9rPU1AnGxkdc8TsQhGywnEtV3H6VJaRZAVPnrmOt988gX/3rluwWKji5Nl5317fCY9+cw6ZWAT/9l234ENvncLJswuYc8GtdJO1ch1PnZnHz03vQYgBXz+30NHzKBwIMTICRH9RkVQjkOgFI8AYOwZgRvtf8DSgD4YQFUQ5zvlpo2NOz+ElLZyxXKyhollIr/n+KytYLdXxztt3YubACAbiEXxvzup87+6hKBzfeGkRR2/dgWwqhnffuRsA8Mz5pS6f2UZ+8MoKapKCB2b2YN9ICrMLnRkpWeEIkw0g+oxyTQHgjifgWEBOm0H6xKZjR5p+3jIWzu2h0OeuN2La8/kK9o+m3Xx6Q755fhHRMMNbbxpHOMQwc2AY339lS2Sr5zh7PY+VYg33Hh4DAOwbTWH/aArPnF/EL77xQHdPrgkR2rtp5wAOjWfwsgMjQOEgot8Qm91E1HkwpycTw3Z5sanC5apPIaFz1wo4PDGAVEy1o6+fGsXcYhGLhaovr98p39J2/G/SjAAA3Ht4DN+bW0FdVrp1WluYWyxiJB1DNhXD4YkMXlkqQurg/DjnCFM4iOgzyvUeCgf1AucX1nH77kEAwPV82ZfXPHc9j1t2Dui/v/6gmus+daG3Q0LfmV3GDRMZ7BxK6MfuPTyG9aqEH11Z6+KZbWRuqYipMdWjOzSRQU1WcGnV/mcrc045AaLvqJAR2Ei+XMdtk0MA/PEEVoo1zOeruGVyUD92y+QgGNsYmupFfnx1Da/dm91w7I496nt35lrv9AzMLRYxNa4agcMTGQDoKC8gK6BwENF3VCXVK6ZwkEalrmAkE8NQMopra957Aue0i+XNkw1PIBENY+9wCi/3WJVNMwuFCpbWaxuMFwDsziYxkIjgbI8YgXyljqX1KqbG1Yu/MAKdvLcK5wj3xSoniAblmowQA2IuLO7Afz1khaMmK0hEwpgcSuD6mveewFltt3/zzo0X0xsmMnh5vneNwNlr6nlvNgKMMdyyc1C/vdvMLRYBQA8HDSaiGEhEOvpsFcoJEH1IpS4jEQ2DubC2A28ERGwsGQthYjDhS2L2/HwBI+kYxgfiG447SWD6gdjp37rJCACqV/Pi9UJPyF9cXFaNwMGxRpXXeCbe0WcrK9yVLwpB9BJlzQi4QeCNQHOWfCARQcGHrt3Lq2XsG0ltOe4kgekHZ6/lsWsogaFUdMttt0wOYr0q4XIPnPt8Xt3xNyevxzJxLK7bNwKKwhGmnADRZ1Tqiis9AkAfGIHmLPlAPIL1ivdG4NJqCXuGk1uOi9h1r2rxvHi9gJsNvABArccH0BOCcvP5KlKxMDLxRhvL+EAcSx0YAZmTESD6j4okI+5CUhjoMyOQiUc81++RFY6ruTL2DG/1BPQqFi2m3UsoCscrS0UcGjdupBPx94vLzgWpnDKfr2DHYGJDGGcsE8NSB+Egko0g+pFKTSZPQFCpN9qn0/EIijUZsodx7YVCBXWZG3oCg4kohpJRXMl1/0K6mflCBVVJMe2mzqZiGExEcHGl+wZsIV/FxKZ8y1gmjnxFQlWyJwuiKBzkCBD9RkWinIBOual9eiChhg+KNe+8AREz32uQEwCAPcPJnoirb+bCkmqYDrSQ1Dgwlu4NT6CgegLNjGlGYXm9Zuu5KBxE9CPlmuxKjwDQD0ag1pBUFTFkL/MClzT9biNPAFBr7q/0oBEQFTcHxoyNFwDsG0l13Qhwzk09AQC28gKcc3AKBxF9CCWGm9iQE9A8gXUP8wJil787a2wE9gyncHm1DM67X2rZzCvLRcTCIUwOGZ83oHoJV3LlrmoIFaoSynV5qyeQiQGwZwREVJA8AaLfUBPDZAQAbCwRFZ5AwUNP4PJqCeMDcdN43O7hJMp1GatdGHXZiotLJewdSba8IO4fTUFWeFc9mQWtPHRi0NgTsNMrIHJDZAOIfoMSw01URWK4qaTQywqha2sV7BpKmN4uwkSXV7sfW2/mwnJxQ/OVESJpfGG5e8nh+bx6kd/sCYjGvCUbOQFF88ZIO4joNyqSQjkBge4JREK+hIMW8lVMDLY3Ar2UF+Cc49JKyTSZLdg/qt5+yYW5pZ2yUFA9gc1GQHh6nXgCJBtB9BvlmoxEhDwBAM2yEf4khhcKlS1Jy2b2ZNULaS9VCK2V6yjWZNM8hmAsE0ckxHDVB/0lMxY0T2CzJAcADKejyJU68ATICBB9BOccFUlGMtYjRoAxdowxdpQx9pDBbdOMMc4Ym9X+PaIdP6H9f9zp6zc8gTAG4qocglfSETVJwWqpjokBc09gMBlBJh7BlVzvGAFxLu2MQDjEsGMwgWtdPPeVUg2xSAhpgwU+lIwib8PAK1p+m8JBRD9RkxVw7s4sAcDheEnG2DQAcM5PMsamGGPTm2YGj3DOWdN9c9rx49pM4g85eX1ANQKxSAihEEM6rr4pXnkCQrtmc9KyGcYYJgbjelijFxAzFnablLU2szub7KonsFqsYSQVMxR9G0xEkS9bT7jLXISDXDs9naYNzCHO+Ue1Y8egrvFpzvnD7r8qQQCVmpgl0BuewINoXNjnABxtvpFzfrLp1xnO+Zz28wc554c23d4R1bqCRET9MyLhEJLRMNar3lTm6JUrLcJBALBzMKEnOHuBq9rOflcbTwAAJrMJX2YymLFSrGM4HTO8bTARRb5iwwiInIDLngBj7CiAk9qs7CnNE9Y3RABy4neCcJuK5N58YcC5EcgCaJ6nOGp0J+1L88WmQ1NmISTt/scZY6cYY6cWFxdbnkC5tjE2lo5HsF61Jy1glQUtKdkqHASoSU2hhNkLXM2VEYuEMGpycW1mciiJ62uVrklKrxSrpuc5mIwgX7bu5YleDQ+kpKfQ2PDMab+33BARhFs0N8i6gV+J4fs45+ILAs75w9qOaVQzEBvgnD/KOZ/hnM+Mj4+3fOKKtLFediAR8aw6SDcCLcJB4vaFfLVnGsau5MrYnU1auhjuyiZQl3lHip1usFpq7QmsdRIOctkT0Nbno9qv0wBOwcKGyM7mhiDMaHgCvWEEcgBGtJ+zAJZN7qe7xtoX4Zj26zLUXVTHqBoajTcjE49g3UbIwA6L+QoYQ9sd9Y6BBGqy0jMNY1dyZezKtvZeBLu0juJu5QVWijWMGMw7ANTEcLkuoyZZ62j2ukRUC/mc3pQHM8XO5oYgzBCimb0SDnoMjYv4FICTAMAY0yeZM8Y2X+RPifsBOKT93jFq08QmI+ChJzCajiPSZq6nGIbSKyGhq7myfnFvx6RmLLpRIVSXFayVW3gCSa36y6KRF46Yh9VBR0VSGNY3RAThCBEO6glPQOyAtJBOrmlH9PSmu85teswDmjcwa3UXZUZlk5peJhHxTDZiPt+6R0CwQwsX9YIRqMsKFgpVTFpICgMNT6AbJa45zXNqlRMAYLlM1EvZCMbYcVEBpK1/ww0RQbiN2+EgRyWigOriGhw70vTzHDaVgho9plPKdVkXFwO89QSW1muGTUybEYnjhR6oEFosVME5MNlC6qKZbCqKWCSk5z/8ZFVrBGuVEwBgOS/gVU5Au+ifYIx9FOru/37O+WnG2IzBhoggXEV4AimXmsUcG4FuU9k0cDkRDesxM7dZKdZwgzY9rBUTPeQJiHPY0SaZLWCMYWIgrpfD+omYFTCSah0OstoroCjedAxrRQ3DBsdd29wQhBlik5uOuXP5DrxsRLm+sTooFQuj7NFQmeViFSMWyizjkTBG0jFc7wkjYK2stZmJgXhXPYGRjPF7PCSMgMWcAElJE/1ISRiBOBkBAGqmPNHkFiWjYZTrsuvlmaWahEpdMb1AbWY809lgdLcxE2RrxcRAoitGYKXYxhNICE+g+zkBgugWRZfDQX1gBDaq6SVjYShc1ddwE3GBstJwBQCjmZjtUYhecH2tgkiIWT5vQA1ndSOUtaq9x1nTcJC687GaEyABOaIfKdUkhEMM8UhvlIh2HTUn0PgzRGhIJE/cQt+lpq3F1kfSMSwXu28E5rVRjXbKJHcMJlCoSLpCq1/kynWkY2HETBZ3MhpGJMQsh4O8ko0giG5SrMpIx8KudcIH2ggoCoek8A0XDSEhUXb5ArasGwFrO+qxHgoHtZp/YISogPK7uilfruvJXyMYYxhMWheRo6EyRD9SrEqu5QOAgBsBSdvpRcM+eALrNsNB6RgKFclyd6tXzOcrliuDBKIXwm8l1Hylrsf9zbAjJ03hIKIfKdVk1/IBQOCNgHqBbXb3Rbmo256AHg6ymBge1WbirnQ5JDSfr9pKCgNNfQ4+J4fzZUmP+5uRiUcsewIiLUSTxYh+oliT9AFabhBoI1CX1Z1epMkICAvptiewXKwhGmYYsPjmj2rGopshoUpdxlq5bt8IDIpwUO95Aul4GCWLJcB6dVCgVzlBbKRYlZByqUcACLgRkI3CQR7lBFaKVQybDDsxQnQxdzM53GpUYytGUjFEQsx/T6DSOicAiI5wa58t5zRjmOg/ilVZH6DlBoE2ApK8NRzkZXWQ1aQw0KgiWu6iJ7BU1IxAxp4RCIUYRjMx372YfFnCYKL1Dicdj6BoURZEpsQw0YeUauQJ6NR1T8D7nMBysaaHeKwg7tvNXoElbSc/ZtMIAKoR8/PcFYWjYMETSMcj9sNB5AkQfUSxJlN1kECWRR1448/wKiewWqxZ7hEAgIF4BLFwSN+Nd4Ml7SI+NmDdeAnGMv72ORRrEhSO9jmBWNiyQKDikYAcQXSTYlVCmqqDVOpadVCzJ5D0yBPIlesYNhl2YgRjakhlpYuegAhF2QljCdRmN/8MmJD/blcdlI5HUKkreiiwFQpVBxF9hqJwtUSUPAEV4e5HQt4mhmWFY61cR7ZNqGIzoz7vpjeztF7FYCKCeMT+rmE0HffVgIku4HaegCiNK1rw9GR9xrDDkyOIHkFc1zKUGFapGySG45EQGHM3HFSo1MG5uaaNGaPpeHcTw+s1jNmsDBKMZmIo1mTfpCOEKJyVnAAAS8lhhWQjiD6jqOXDeioxzBg7xhg7yhh7yOT2E9r/x60+xiqSvDUxzBhTlURdNAJi4lXWRjgIULuGl7qZGF6vYsxGHqMZ0RntlycjGsAGLFQHAbCUHPZqqAxBdIuiVh7dMyWi2qBtMWQjJ37fxHHG2Cy0EZMWH2MJyWSnJ+Sk3ULo3Ns2Ahk1ru62rLVVltarHSWFgUbHs1+ejNVwkEiIWekVEPMEqDqI6BeEB9xLnsCDUAdsA+pF/qjBfT7IOT+kXfStPsYSIjkY3TT4PeGyEciVhSdgMxyUiaNSV1ByuVLJKkvrtY7KQ4FGMtlvT8CLcBA5AkS/IK4lvSQbkQWw0vT7qMF9pjaFfto+hjF2nDF2ijF2anFx0fTFJWWrbAQgpou5d+FdE+Egu4nhdPd6BWqSgrVyHaMdhoPGfO5zEKJw7cJBYvFbKRMlKWmi32jkBHokHGQFzvnDmhcwqg3htvKYRznnM5zzmfHxcdP76UYgvCkcFHPZEyi1HnZihtiF+1lqKRDCdZ2Gg3RPwK9wULmOVCy8xavbjB1PQCYVUaLPKLo8WhJwbgRyAEa0n7MAlptv1Hb0x7RflwFMtXuMHUQ4KBIyCAe56AmslkS82t4b382uYSH50Gk4KBOPIBYJ+aaCakU8DmgkxKyUiHJKDBN9RiMn0DuewGNQL+zQ/j8JAIyxrHbslDgG4JD2u+FjOsHUE4iGXS1tXCvXMZiIINJml7qZRlzdf0+gYQQ68wQYYxjzsbrJiow0AKRjNjwBrVmMPAGiX2g0VdoLTbfCkRHgnJ8GAC3MkxO/A3i66fYHNG9glnN+usVjbCPJW5vFANVKupmMzZVqtkNBAPR4fDfKRHXJiA49AUCdnbDikwGz6gmkYmEwZjMcFOhuGIJosFauI8SAjIvVQY6fiXP+qMGxI21u33KsE8RQGSNPwN0S0brt8lBAzU2kY+GuhoNGHRiB0XTct+qgQkWyJHnNGEM6FrGUGNabxcgTIPqEfLmOgUTUVWXcQO+RJIOhMgCQiLkbDsqV6x15AoB6Ee5GOGh5vYpENORIaGo0HfOxOqhuOeeSjodRstQnQDkBor/IV6yFTe0QbCOgewKbwkEuJ4bXSjXb5aGC0Yx/F9JmRI+A1SE4RvjZ7NZuyHwz6XgE61Y6hhWhHURGgOgP1sp1DLmYDwACbwQ02YjNHcOxMEp12bWLl+oJdGgE0v4PZwHUcJCTUBCgzhTwo9mNc458RWrbIyBIx6wNlvHaE9jc7e6WHApBmJEvW8ud2SHYRkA2/pInomFwDlSl9nLD7eCcI+/A+mZTMV17yE8WC1WMd1gZJBAlrl6XiZZqMmSFW17c6XjYVnWQFzkBrbDh8abfXZNDIQgzyBPYhFAR3RwOEjMF3MgLFGsyFN6+k9WMkXRM1x7yk+Vi55IRAlFe6rUno+sGWQ0HxSKWvBPFw+og7WI/13TINTkUgjDDahWdHQJtBGSD8ZJAo5HCjTCGrmnT4RufTUVRlRTXJ521QlE4VmyOwzRCTFLz2hPQZaQtvscJix3hir/jJa1IqBCEI6z209gh0EbAVEXUxcEydnepmxnWqor89AZWSzXICnfsCfilfdR4j60tbquJf11KmhLDRB9QkxSU6zKFg5oR4aCogWwE4M5gmYJFYTMzxEhKP42AqO13bAQy/iiJ2vW2rGpD6Z6APyWibeVQrAojEoQRTjekZgTaCMgKB2Nbv+Ru5gSch4PUC6mfyeGlgmgUcxYOSsUiSEbDnovI2V3cSYsd4Qr3VUa6rRyKVWFEgjBiTbsWkSfQRF3mW7wAwOWcQADDQW55AoCa2PYvJ2DN20pGw6hJip4TMkPm3Mvy0GMAZoRAoptyKARhhNMNqRnuZhh8RlaULZIRQFM4yAVPQBdschwO8s8TEAZnuMMu52bGMjEs+RQOGrAaDmr6fFsN11AU7llSmHP+BIAnNh1zRQ6FIIxYK9vLnVkl8J6A0U5PJIbdDAdZvUBtRg8H+aTBAzQSucMdNrg1M5yOYdXjcy9UJSSjYcQi1paj8PTa5XxkxTtPgCD8RgxeonBQE5KiGA4hsXqRsEK+IiERDVm+QG0mFlH1e/z2BIaSUdvS10YMp7zvc1AlI6zvbhIWcz5qToCMANEfeBUOCrQRkBW+RTwOaIQL3MgJFFxozlC7hv3zBFaKNX2WgVOyqajnSW27DTBiyHa7z1fhnOYLE33DmsU53HYJtBGoy8ZGwM2cQL5sXdPGjOF0FCsBNQLDqRjWqxJqLkhwmGH3PU7G1GXb7vOlcBDRTyyv15COhfXrm1s4NgLtRLO02ujjjLETTcdOiNucvLYkK4Yhj3gkBMZcyglUrKtbmqGGVPwLB60Ua64khYFGXiFX9s6I2X2Pk1HhCbTWD1I8rA4iCL9ZLlYxZmHmhl0cGYF2ollaudxJrWpiqmnQ/HHG2Cw2aq/YRjIJBzHGkIq6M13MDdW+4S6Eg0ZdCwd53+dg9z22mvhXOCcZaaJvWFqvuva9bsapJ9BONGuq6dgcGs00H+ScH9KMR8dIMjcsEQWsd5W2o2BD4tiM4VTU8wobAeccq6Uahl0MBwHw9PztDsqwmvORFU6SEUTfsLzuXBTSCKdGoKVoltYhKWqnp6EOmgc0r6BNCKlte73qCRj/CYloGBWXmsWchoOyqRjyFQmS7F1cXVCoSqjL3EVPwNs+ByHVbS8xbLVElKaKEf2DGzNCjPAlMayFifQh85zzhzUvYLQpRKRjtb1eMmkWA9ybM5wvSy6Eg9THi+y+l4gdu2ueQFqEg7zxBMp1GZLCbRlaqyWinHMaMk/0BbKmDOx0RogRTr8ibUWzNI5yzj8K6Lv8Y9rxZTRCRLaRTKqDAHW36DQnUKnLqMmKC9VBQjrCeyMgJCPc8gS87ni2KyMNWJcFkbl3HcME4SerpRoUjp70BAxFsxhjWXEHxthxzvnD2s9HoYaERC7gEBohIttIitIyHOTUE3BLta+RXPU+L+C2JyA6eb06d7sy0oD1EmDKCRD9glAB6LmcQAvRrKebjp9gjM0yxlabHvOA5g3MOhHaapcYdloialfYzAw/9YOE2NuISyWijDE1se2REShU7HdBhkMM8UiobU5A4dwvGWmC8BQx3c+pMrARjpWIjESzOOdHtP9PAhi28phOkBSOlIk0QioWxpVVZ0agkwuUEX4qiepGwMXF4mWfg25obXpbVqq/FIUGyhD9gTACPecJdBs1HGT8JU+40CcgBJucqvaJChs/wkErpRpiYVWvyC1U6QiPw0E2va2khc9X5uq8CYIIOkt6OKj3EsNdpVViOBl1IxzkjieQiUcQCTF/wkHrqmSEm01S3noCneVdrHkC1DFM9AfL61VEQsx1BVEg6EZA4YYqooA7JaL6LAGHbzxjDNmU95LMAFxtFBN4ee75Dsd3Ji30gXg5VIYg/OT6WgU7BhOedMAH2wjIiumXPKXtFDlvPX2qFSJU4bREFABG0t4lV5tZdlEyQjCciiJXrjt6L83Il+uIR0KIR+yFr6yUAJOUNNEvXF4tY89w0pPnDrQRqLeoDkrEwuAcqDpQv8yX64iEmC5T4ISsTyJyq0X3PYHhVAyywvVdu5t02pFtpQRYnSzW6ZkRRO9webWEPcMpT5470EbAbJ4A0DSC0EFyuFCRMJiMuuKCDXuYXG3GTfE4gZeJbbUj276nlYqFabIYsS2oSQqu5yvkCRihykaYl4gCQMlBXiBfqbsSCgL8kZOuywryFck1GWnBiIcdz516AlZyPtQxTPQD19cqUDjICBghKRzRFiWigDNPwA0ZaYGYLuZFXF0gcg4jaXcrCLIe9jl0+h4nLeQEOCWGiT7g8moJACgcZIQkc4RNZCOSFkXGWlGwKXHciuFUFHWZo+iCsqkZeqNY2t2GkmEvw0FayM0uyWik7WcrK+QJEMHn8moZAHkChtRlBVGTxLDVObStyFfqGIi7s6v2Q5d/RdcNctcTaJy7B+Ggcr2jnEAyFmpb/SVzkGwEEXgur5YQDjFMDiU8ef5AGwFZaaUdZG0ObSvyZfc8gUZy1bu8wIpa3Z/uAAAgAElEQVSuIOquJ6Amx933BDjnevLdLqlYBLLCUWsxo0FROEyWB0EEhkurZewcTJjmP50SWCPAOYektAoHqRfvcps5tK3IV9zLCTTkpL3zBFY98gTCWqei24nhqqSgJisdvcf6TIFaCyNAOQGiDzh3vYAbdmQ8e/7AGgFJUcMAZolhMYe2U09AkhWUajIG3DICupKol+GguvZa7uuLqNVN7p57QzKisxJRACjVzY28rNCMYSLY1CQFLy8UcOvkoGevEVgjIGtGIGyaExDVQZ01ixVcEo8TZH3JCVQxmIiYSmk4QRWRc9cTaHRkd1YiCrSu/lI4zRMggs35hQLqMsetu7wzAu5c4bpAXYsFR1sMlQGAUofhoLxLMtKCbNL7mQIrpbpe0+82w6kY5vMVV59zzcG8BiuDZfq5WexKrozra2XcvHMQ6bh3X+NyTcaL8wUsr1cRDjHsHUlh73AKsUhv7R/LNRlXciVcyalrNBkNY89wEpND3ujtWKVSl3HmWl7v5L91clBfu1Y4czUPAJ56AoE1AsITaDVjGOi8RLTQobCZGZFwCIOJiKddw15IRgiyqShevF5w9TmdTG6zMmye92l10H/79iv42JfPoSYryKai+M2fvh3/6M5drj1/TVLwF89dwZdeuIbvzi2jtkl6JRpmuOfQGN55+0686zWTrm2U7DK7uI7//fxVfPP8In54KQfFoFBsLBPH3QeH8e47d+FtN03YugA74aX5Aj7zjVl8+YVrqNQb799APIJjM3vwL3/yBj060Ioz1/JIxcLYP5r27FwdX+G0CWE5ANNijGS729s9xgp1WQsHmXzJY5EQIiHWcYlopxLHrRhOe9s1vFKsYVfWmzIyT3MCHTaLAW08Ae6vdpAb67odZ6/l8Zt/fQZvuXEcv3D3PnzmG7P453/6HFZLNbz/ngOOnltWOP7n372KT//Ny7i2VsHBsTTe94b9uPvgCHYMJiDJCl5dKeHM1TyePDOP3/jzF/Cfv3QW73n9Pnzg3oOYGPRm7TXDOceTZ+bxx996Bd9/ZQUhBrxmTxa/8hOHcXgig13ZJEKMoVST8MpSEc9fyuGbLy3iyy9cRzYVxfvvOYBfeuMBzzzmYlXCb3/tRXzuuxeQiobxf961B2+9cRw7hxKYz1fwtR9dx+e+cwF//ffX8On3TuN1B0ZaPt/zl3K4ZXLQU4/WkRFgjE0D6gQxxtgUY2y6eVyk0e3iNrPHWEXUh7d6c5zISbsdDgKEiJyHnkCphts8ih0Op6Io1WRUJdm24qcZTob2JKPth837OWO43XfBLf7zl89iIBHF7z74WmRTMfzETRP4yBdO4//+yx9DUTh+6U0HO3relxcK+PUn/h7PvZrD6w4M48TPvQZvvmFsSyhl5sAIfnYa+LfvugU/vLyGP3xmDp99Zg6f++4F/OI9B/Dhtx7yxBsVF/9PnjyPM9fy2DuSxEP/8CYcO7IHEwPGxufNN4zj/feoRR7fmV3G5793EZ96+jw++805/OIbD+BDb5ly9VyfOb+I3/jzF3B5tYz337Mf/+rojVue/x237cQv33sQ//xPn8N7P/t9fOZ903j7zTsMn29pvYrnL+XwL3/yBtfO0QinnsCDAJ7Sfp4DcBTA6Ta3j7Z5jCVkzQi06ghNWhAZM8OtqWLNDKei+sBot+GcY7lY82yHI1zXXKmOHYMuGQEXPIFW4T5F8XXGcLvvgmOurZXxzPkl/F/33ah/HrFICJ9+7zR+5X+exv/7V2egcOCX77VuCOqygke+MYtPPf0y0vEwPvnzr8W779zVNo7OGMNr92bxe++ZxsXlIj558jwefWYOX/j+q/jAvQfxT9980JXKOkVRL/6felq9+B8cS+MTD9yJd9+5y3LdfCQcwltuHMdbbhzHywsF/N7XX8Yj35zFF753ER9480F84F5n57pWruM/fekMvnjqMqbG03j8w/e03OHfvnsIf/GRN+Ef/9H38aE/eRZ/8N4jOHrrVkPw9XML4Bw4eouxkXALp9mdLICVpt9HLdze7jFgjB1njJ1ijJ1aXFw0fGG9OqidEejUEyh3XrlihhchFUGpJqMmKZ7lBLyYk5yv1BGLhDqK0+oloi2rg3ydMezKum7F02cXAADvvGPnhuOxSAi//55pvOO2HfiPf30Gf/jMnKXn+9GVNfz0730bv/PkS/gHt+3AU7/2Vvz0a3fbTqTuH03jEw++Fk/+6lvw5hvG8Mmnz+PND/8NPvON2Y43YYrC8eUXruGnPvUMPvz5Z1Guy/jEA3fiqX/1Fvzs9J6OG6cOTwzgd3/+LnztV9+Ce28Yw++eVM/1D/521nYRCefqOd73iW/gz05fwT972yF8+V+8uW2IBwCGUlF8/p++HrdODuKffeFZPPnj61vu89SZeUwOJTzz7gU9mRjWBtE/CgAzMzOGugCKlmtptdNLRp15AoypiRy38KLMUtDQDfLKCGjVTS5KR6gy0p0ZWSslojLnMCke6wpW1nUrTp6dx4HRFA6Nb20cikVC+L33TONf/Olz+K0vncV8voKH/uHNhuXCy+tV/Nevv4w/+d5FjKZjeOR9R/CO23ZuuZ9dbtgxgD/4x0fwwuU1fPypF/FfvnIOf/jMK/iVnziEnzuyx9Jnna/U8ZfPXcH/+O5FnF9Yx9R4Gr/74Gvxf7xm0tWO2RubzvUTT72IE189hz/6lnqu73n9vpYhT845Tr+6iv/ylXP4wYVV3Do5iD/+pdfh9t1Dts5hKBnF//jA6/H+P/47fOQLp/H7753WP4czV/N4+uw8PvjmKc+rm5xe4XIAhNnLAli2eHurx1hC0cNB5vdx4gkUKnVk4hFXwwnDqRjWqxJqkuJ6iZ2uIOpBoxjQHA5yzxMoVOodh9uslIgq/grItfsuOKIqyfjO7DLe94b9pheFaDiET/3CXfiPf3UGn33mFfzNi4v48FsP4d7DY0jFw5hdWMdXf3wdX/jeqyjXZfz86/bioXfcjKGUu9U9d+wZwn//J3fjBxdW8NtfexH/4a/O4GNfOYe33zSBNx0exR17stg1lEBCC9deXi3hR1fy+OZLi/j27BIqdQW37RrEp37hLrzrjklPk6J37BnCf/snd+PUhRX8zpPquX7y6fM4essOvP3mCRwaz2D3cBKyzHFptYTvzS3jiWcv49z1AsYyMXzsZ+/A/Uc690yGklH8yQfuxvv/6O/wK184jV9/x034iZsn8G/+1wsYTEbxkbcddvkv3opTI/AYgBnt5ykAJwGAMZblnOfMbjc5ZgvZamK44+qgznepZuhqnOWaaTKrU5Z1yQiPPIG0+30O+Urn73E8EkKItfYEpBZDhzzAbK27wvn5ddQkBUf2D7e8XzQcwm/+zO14203j+E9fPot//fgPN9weDjG847Yd+LX7bsLhCe+kCADgdQdG8NjxN+D5Szn85fNX8eUXruGrBmEPwf7RFB6Y2Yv7j+zFHXvs7aqdMnNgBP/f8XvwndklPH7qMr724+t44tnLhve9Y/cQfutnbsfP3LUbGRciBYMJ1RD82hd/iI995Rw+9pVziEVC+J3773TdQBvh6C/gnJ9mjM0wxo4CyDVVQzwN4IjZ7SaPsYWitE8Mp2LhjsMvbg6UEYgLdK5Ud90IrOricQHKCZQ7GygDqInJdtVfqsCgP/GgFt8FV7DbNPST2k72uUs5/PjKGqqSgl3ZJO4+OIKxjLsCg61gjOGufcO4a98w/p9/dCuurlXwwuUclos1lGsy4pEQ9oykcHg8g70j3ujl2+GNh8bwxkNjqEkKzl3P48JyCddyZU3FM4nX7Bny5DwHElE8+r4jOHVxFS/NF/CmQ2M4MOZdb0Azjq9yWpxz87EjbW7fcswuVjyBRDTsoFms8wuUGeJCuuKBdMSKx55AIhpGIhpyNRyUr9QdaaQnY5GWieG6rPjpCbiyrs04cy2PdCyMfTYuQIwxTO8bxvS+1t6DXzDGsDubxO6sN7r4bhKLhPCaPVm8Zk/Wt9dkjOF1B0YsJZbdpIfSZvbQE8MtvuMpR9VBnc2+bYWXs3pXSzWEQ8z1c27G7RGZ+bLkqPoqGQu1NPKtpMaDxpmredwyOdiXHdBEdwmuEbDSJxBtP4LQDDdlpAWNkIr7FUIrxRqGUzFPKwnEiEy3yDtIDANAKhoxLetrJzUeJBSF48y1vKciYsT2JbDfEL1PoFViOBZx1CfgVTjIi16BlWLNs3yAYDjl3kyBSl3ta3BiaBOxMMp1Y5XYdlLjQeLqWhnrVQk37yQjQLhPYI2A7gm0qQ6qSYpuMCw/t8KxXpVcTwwnY2pc3Qs56dVi3fVhMptxs9nNiXicIBkNoWLi6TUEBgO7xHXEjNn9o91PnBL9R2C/IZbCQR2OmCzWJCjcXd0ggdtxdcFKyTvJCEE2FXXNgOUdyEgLUrGI6VAZITXuZ2LYK7weNE5sbwJrBMRo2dayEWLEpD0j4IVukMDtuLpA5AS8ZCQdw1q5rpfnOsEdT8C8D0SSW0uNB4nLqyUwBkwOkREg3CfARkCEg8zvY0VawIiCg4lX7XAzri6QFY5cyfucQDYVg8IbF3AnOBGPE7QSCBQ5gX7xBHYMJHpukAvRHwR2VelS0m2axQD74aBGqMIDI5COuZ4TyJfrULh3PQKCxpxk50ZADO0ZcuBttWoWk7Qa4v7ICZQoFER4RmC/IbLFxDBgf8SkkwHo7VA9AXeNwErJW/E4gZvVTW7Ma0jFzEuA9XBQn3gCZAQIrwiuEbAgG2Fl+pQRhar7A2UEwyn34uoCvVvY45yA8DTc8GR0b8tBTiARDaMqKYbvpR4OCnhOQJIVXFurYM8wVQYR3hBYI6BYFJAD7M8ZFhcot0tEAXfj6gKvZaQFboaD8pU6YuEQ4g7i3PpgGWnr5yvp1UGBXeIAgOv5CmSFkydAeEZgvyGKpeqg9oNHjPBioIxgRKvld1M/aNUnI+CmnHS+rAr0OelwbpX415vFAu4JXF+rAAAmA6C3QwSTwBoBkRNodQ3puDqoKiEZDXtSjZH1QDrCr5zAYCKCcIi5lBOQHHdktwr3iZxA0GUjltarAICxjLefLbF9Cew3RLEkG9FpdZD7MtKCYQ+Gs6ys15CKhTsa02gHxhiGU1GsuDBdLF+uOxa7a2Xk63p1ULA9gUVtJrWf8s/E9iKwRsCKlLReImq7Wcx93SCBm3F1wUrJ+0YxgVvNbm68x8kW08XkPukTWNY8Aa+9PGL7ElgjIApCWlUHJSKd5QQKFfdlpAVuVtgIVoveS0YI3CpxVT0Bl8JBRp5AnySGl9aryKaihrOCCcINHK8sxtgxxthRxthDJrcf1/6daDp2QtzW6es2JouZ3ycUYkhEW2vOG6GGg7zxBAbiEURciqsLVkp1zxvFBMOpWMfT2ppRcwIOw0Etwn1ynySGl9drFAoiPMWREWCMTQMA5/wkgJz4ven2owBOahOXprTfAeA4Y2wWwFynr21FShpo3VVqhhtJSzMYY6oQm5vhoGLVc8kIwXAq5kplkxuGtmV1kGxtffQ6S+tVSgoTnuLUE3gQQE77eQ7A0U23TzUdm9N+B4APcs4PacajI2QLKqJAZ4NlChXnSctWuC0it1qs+5cTSEeRK9V12Y5OqNRlVCUFQx7mBEQ4KOhhlOX1GkbJEyA8xOk3JAtgpen30eYbOeePNs1dnQZwSvt5ykII6RRj7NTi4qLhC3MLiWFAExmz4Qlwzh2PPWyHWmHjjhGoSjLWq5Lef+A1w6kYarLS8cQ2oNGH4dQItNKGkvukY3hxvYpxMgKEh/iyTdLCRKc556cBgHP+sOYFjDaFiHQ04zHDOZ8ZHx83fE5ZnzFswQjYuGBVJQU1WfFEN0jgVlwdgP48fuUERjSPw4kRy2lGQMxc7pREq8RwH1QHVSUZhYrkW6iP2J60vdKZJG/nRB4AwIh2LAtg2eRpjnLOP9r0fCuc8ye0+0+ZPKYlDQG51vdLRSO2jIAbEsftGE7F8PylXPs7WmBZqyP360IhLty5Uh17R9rc2YQ1lzyB1jmB4FcHic92bIA8AcI72hqBpnCOEY8BmNF+ngJwEgAYY1nOeU77+Tjn/GHt56NQQ0IiIXwIwCOdnLjeLNbGE0jEwvqF3QpioIxXzWLAxri608HwosrIr5yAXuLqIKchvJds0tk5R8MhRELMuGPYYuFAL+O3gSe2J462SSK8o13cc+J3AE83HT/BGJtljK02PeYBxtgxALNNj7GFFQE5QJ1Da8sTcGHiVTtEXL3oIK4u8Es8TtBoduvcCLjlCQDmOR9RHRTkxPBSUW0Uo8Qw4SWOt7tGngLn/Ij2/0kAw1YeYxeR+Gu3k07FIrYSw41wkHeegIirrxZryMSdvY7uCfhYIgo4a3YTlVGuGAGTEZNyH8hG5F3KnRBEKwK7TbLqCSRslojm9YlX3n3xmuPqTller4ExIOvh+TYj3hcnfQ75ch2MuRNyM/ME6n0wVGbNh/wUQQTWCFgZNA+oZYR2Oob1L56HF1URullxoVdgtVTDUDLq2xjFSDiEwUTEUZ/DmiYZ0WoqnFXMPIF+GC/p5YQ7ghAE9huiWKwOEh3DVpub/KgO0o2AFvN1wkqxpoeX/GIkHcOKA08gV667FuIwzQn4VCJq0CXfUkbFDvmKhEQ0hHjEW3VYYnsTXCNgYbwkoF4kZIWjJlyHNqyV64hHQp7KMotEn6j+cMJKseZbPkDgtON5rVx3Ldxm6gn4EA7SCh8eb/q9pYyKXdZKzkX2CKIdgTUCupS0BdkIAKjUrBmBvIsXKDMGExFEwwzLLnQNd0NbxqmSaK7kshHoUomodrFv1r9qJ6Nii3zF+7VIEIE1AronYEE2AgBKdcnS87q5SzWDMYaRdEzXindCN1Qmh1MxrDoYLOOmoTUvEVUQCTHHfRg2aSmjYpe1sndzLQhCEFwjwK3t8uwOlvHrizeSjjvWD5JkBSsl/wXGhtMxx30CbnoCFZMZw71WHmpFE6uZvMdChgQBuNAn0C1kzlvOEhAkWihNGpGv1DExkHByapYYy8Sw5DAnsFKqgXNgvAvhoFJNRlWSbSctOeeuJ4ZLJs1iUYeSEW0kU4xoKaOi9cc8CgAzMzNtKxXyZQmHxzPWT5ggOiCwRkBReNukMNDwBKz2CqyV67hhYsDRuVlhJB3DxeWSo+dY7tL82aw+J7mOHYP2jECxJkNWuPeJYUVB2KEn0EFTo6GMSqdQOIjwg8CGg2SFWwoHpbWO3PWqxZyAi0nLVoym445zAkva4/0WGGuUuNr3ZERVkVPdIEEyFkZVUvQckaAuc8/F4zTpkxnt/1YyKrZRFI4CJYYJHwiuJ8Dbl4cC0GUZStX2noCicBSq3s0XbmY0E0OxJqNSlzsuRxVGwG+BMSdGwO1mPL36S5KRijU+N1lRPO8R0JRwn9h0zLEkCgCs1yQonLqFCe8JrCegWMwJiHBQ0YInUKhK4NzbbmGBuHA7KRNdKnRHaliUpC514MmsuayHkzQJ90ly7yWG7UDdwoRfBNYIWA0HZWyEg9yaeGUFUdGz4iA5vFSsIhYJYcChCJ1dRtOdN7utldx9jxMmMwUkhQdaQTRf9l7DiiCAIBsBbi8nYMUT8EM3SCBCKksOpCOWCjWMpWN+18JjKBlFOMQchYPcurgJT2+zPpSkKIGeJUDicYRfBNYIWB3IEg2HEIuEsF6zbgT82H2JkIoT6Yil9WpXpk6FQgzDqRiWOzBgbo2WFJgNm1cTw8E1An7MtSAIIMBGQFZ4W8kIQSYesZQY9jMc5IaI3HKx6nt5qKDTPoe1ch3RMNMv3k4Rz7M5JyAHPhxEngDhD46/Je1UExljJ7T/j1t9jBVkxbouTCoWthUO8sMIZOIRxCIhZ55Aoda10YOjmc5kL9RuYfdCWPqw+S2eQLDDQWK9puOkIEp4iyMjYFE18ThjbBaa0JZbSosK521lpAWZeMRSYtjPnABjDKPpWMfVQZxz1RPo0hDy0XS8o3NX+zDcS2TrOQGD6qBogKuDxOjRtM9Jf2L74dQTsKKa+EHO+aGmVvu2j7GisaKWiFr7kqfjERQt5gTCIYZ0zJ/dV6e7aUCtHqnLvGvhIPXcOwsHuelpmeUErFaP9SqlmoRwiCEeCW5IiwgGTleYFdXEqU2hn7aP4Zw/yjmf4ZzPjI+PG76wnZxAOh7BupWcgNah6Ve1zagDEblF0S3ss26QYCwTx3pVsjW1DQBy5ZouO+EGZjmBuqIEOidQrMpIxcK+V34R2w/PvyWc84c1L2BUa6d3BTUcZDUxHEbJUjhI8rUuezTduYicLhnRLU+gw2Y31z0BkxJRWQl2dVCxKuk9LgThJW1XWRslxZaqidpjV7T2+mWoolotH2MVRWk/UEaQikUsJ4b9lO4dzcQ69gS6bQT06qb1GnZnk5Yf57Y2k1mzWF3mCHusHeQlpZqs5zsIwkvaXvHaaKEYqiYyxrKc8xyAU2hMXjoE4BHtmGOlRZlzWPWUrSaG8z6rNo6k4yjXZZRq0gbdGyuIePxol8JBouPZTrObrHDkK+56W9FwCNEw25ITkGQl4IlhiZLChC842iq1UE18uun2BzSVxVnO+Wm3lBYVG4m/dDyMYq39sHk/Rks2M+qgYWxpvYoQU6d8dYNOmt0KFXcbxQSJaNiwTyDIieFiVULa5saAIDrB8Soz8hQ450fa3O5YadGqbASgJoZlhaMqKS0VO/3Wb2+Oq+8dSdl67NJ6FSPpeNcudMITsFPdlHNZN0iQioW35AT6ITG8K0uNYoT3BPZbYlVKGoC+o2qVF+Cc+zJfuBldRK6DruGl9VrXKoMAIB0LIx4J2UoMr2izBIZdbnAzGjYvB1w2opMQIUF0QnCNgGJNShpoFpEzL2cs12VILk68soLwBDqpEFpa755kBKA2u41l4rbkpIViqttdzgmD6WL1HpwxbIf1qkzdwoQvBNYI2In5ZrQvU6vksJ+SEQJxEe9El181At3zBAD7DWOiEmrEbU8gttUTkGTF88liXlKqUU6A8IfAfktkmx3DAFp2DXdDujcZC2MgHsFC3p4R4JxjsVDFeJckIwQjaXslriJ0JOYRuEUqttUTkALsCSgKV0tEqTqI8IHAGgFuMzEMtPEEPEpatmN8MI7Fgj0jkK9IqNQV7BhMeHRW1rA7J3mlWEUyGtYbvNzCKCcgBTgnIP4Wv+RLiO1NYI2ArFj3BMTkrfWKuRHIV7ozyWliII75fMXWYxa0+3fbCIxlYlgq1tqW3gqWizXXQ0GAlhMwGCoTCWh1UENBlDwBwnuC+S0BIHNYlo0QZZ9iUIcR3cgJAMDEQAILNj2B6z1iBEYzMdQkxVIjHqDmBLwwAqbhoIB6Ag0FUfIECO8JrBFQFA6rIV9xYRdzW41Y69Jg74mBOBYKFcu7aQCY13IIOwa7mxOwO2vYKyOwORwkyQo4R2D7BIQnQCWihB8E81sCVUDOak4gHgkhFg7pF3oj1ko1MAYM+DzJaWIwjkpdQcHibhqAHj6aGOi+JwDA8pjJ5XVvhuAkNnkCFUkBANeml/mNHg4iI0D4QGCNgKxYmzEMqDXtg8lIy3DQSqmG4VTM9w5cEdKxUyG0kK9gMBFxPcFql0aJa/c9gaqkQFFUb0oYhEQ0mMu7ROEgwkeC+S2B5gnY0FofTET1ua1GrBRrGHZZ08YKosxzoWA9OTyfr2LnUHe9AKBhBKxUN5VrMsp1GSMe9DakNo2YFBISrSRCehlRykyJYcIPAmsE7AqEDSajegWQEV7tUtshQjp2PIH5QqXrSWFArQ5iDJYS2yJk5EU4aPN0saAbgZLW2U5S0oQfBNYIcBvVQYBqBFrlBFaL9e4YgUH7nsBCvtr1fAAARMIhjGXieslqK1aL6ns/4nKjGLB1poAwBkHNCYhqKxoqQ/hBYI2A2jFs/f6DiQgKLYyAVzXs7RiIR5CMhvWKn3YoCsdCoaIbj26zYzCul6y2YnFdvY8X8w8ymzrCK3U1MRxUT0A3YuQJED4QXCNgY8YwIMJBxkaAc45VLTHsN4wxTGYTuLZWtnT/pfUq6jLHLhvTvLxkx0DCkgFb0Mta3fdgMomNzYCNi2gwl3e5JiPEgFhAS1yJYBFYf1NRrM8YBkRiWALnW6uK8hUJssK74gkAwK6hJK7mrIWDLudUY7E72/1wEABMDCbw/KVc2/uJvMG4B8qnwhMQZbZ+5QSaRq8e4px/VDt2DOoI1WnO+cOdPG+lLiMRpSHzhD843mowxo4xxo4yxh4yuG2aMcYZY7Pav0e04ye0/43mF1tC4dZnDANqw1hNVvRQQTNeqVtaZXIogas5a56AuF/PeAKDcSwXa6hJW9/XZhYKFQynoohF3N/dZjbJgvhhBLTJeCe1AUlT2ndgGgDE/G3xu13KmhEgCD9w9I20sOhHOOeMc34IwP0ATmjHjzPGZtGYP2wbmXPYUQoWncBGISFhBNwedmKVyWwSi+vVthdSoGEE7Ax395KdWnhnsY2QnJfJbD0ctMkT8DgxPAXgqPbznPb7g1C9AHHsaPMDGGPHGWOnGGOnFhcXTZ+4UlcCm9QmgofTbVnLRa8ZB8EM51xc9D/IOT+06XYdK18WxYaAHNCQiDbqFVgVnkCX5vXuGkqAc1gSkruaq2AgEfG9s9kMEeNvd+4LhapnyWw9MawZgUazmHcXUs75o01jUqcBnAKQBbDSdLdRg8fMcM5nxsfHTZ+7IsmIB7TRjQgeTlday0Uv0FznLzYdmjILIQHWvix2ZgwDrUXkuh0OEqGda2vtjcCVXLlnvACgqcS1nRHIVzybfyDkFQp6Ytg/2QjN+z3NOT/t1nNWajJ5AoRv+JUYvq951y8SZoyx+xhjR808glbY9QSEiJxRr4CYfds9I6Dupq1UCF1ZLfdMPgBoeALXWxgwzjkW170LB4VCDJl4ZEs4KO4w/2CSs5rbtF6PiqQwVK94RMYSxukAAAuDSURBVPs5C2C5k9etSJQTIPyjrRFo80Wwuuj1XIH2fCuc8ye0+0/ZOmMNO4PmAbVPADBWEl0t1hCLhLrWoTk5pF7UrVQIXV0rY3p/1utTssxIKoZYJISrLYzAaqmOuswx4eEktEw8siExHI+EbFWPGdEU7jGEMXa8aUNzFMBjAGa0m6cA2N7cAGo4i3oECL9oawTafBEMFz1jLMs5z2k/b77In0IjIXwIwCN2TligykZYv39Wi/evlraKnS0WqhjPxLtWkpeORzCYiLStECpWJeRKdd1o9AKhEMOebBJXVs3PXXRDe9nglo6HN3gCXl9EtYv+CcbYR6FuhO7nnJ9mjM1ot+U6DRFV6krXvFJi++EoHNRi0T8N4EjTXec2PeY4Y2wFwGynXxS1Osj6RTubjCIcYoZD3VUtnu524O4dSeHSaqnlfcTt+0ZSfpySZXYPJ3G5xbmLRjEvpS4yiajeJ1Cuy0hEvDUCmic8bHC8pfdgBTUxTJ4A4Q+OcwJGi55zfqTp5zkAH2r3mA5e11afQCjEMJqOYamw1ROYz1dxeDzj9JQccWA0jTPX8i3vc2GpqN+3l9gznMSTV83PXeQLdnooejcQj2BdS/pX6kqgwymUGCb8JLB1aHZmDAvGMnFjTyDffU9g32gKl1dLkGTzXoELy+pue/9Yb3kCe4ZTWC7Wtox4FFxeLSHEgEkPu5ybE8NlLScQVCqSEthZCETwCORK45yriWGbib+xga1GoFyTUahImOiyNPOB0RTqMm9ZJnpxuYjRdEzveegVRMnqlZxxSOjSahmTQ0lPxz1mEhEUqw0p6SB7AuWa9+EsghAE0ghoA6RshYMAVf9+8xQskbTstj7/vhE1xHNx2Ty2/spSEQfGeisUBKjhIAC4bJIcvrxawu5hb5PZmXgEBT0cFNyLKOccFSnYRowIFgE1AqoVsFsBOJ6JY7FQ3TDUvVeGth/QQjwXloum97m4XML+0d4KBQFqOAhoZQTK2Dvs7XkPJNRwEOcc5QB7AjVZAefBlcEmgkcgjYCsuQK2w0GZOGqysmHCmJA76LYnsGMggXgkhIsmRqBSl3FtrdJzSWEAmBiIIxpmhtVNNUnB9XxF9xa8Ih2PQOFqPiDI2juVWrBnIRDBI5BGQHgCdofCC9mC5ryAMAJeNjJZIRRi2DeSwitLxuEg4SH0YjgoFGI4MJrG3OJWA3ZtrQzO4bkRaFYSLdeCq71TkYTuUTDPnwgegVxpwhOwnxPQjEDTTNzFQhWxSEiXlegmN+4YwEvzBcPbzmrlozftGPDzlCxzeCKD2YX1LcdFiGivx70NA4nGTIGqFNwSS138LqA5DSJ4BNIIiMSw3QbfsQG1C7M5OXxdKw/thQEet0wO4NWVkp7gbObstQJi4RCmxnvPEwBUI3BxpYSqtLFM9NKK6tl4LXqnD5bRPIGghlOEJxDUnAYRPIJpBJTOwkG6J9AUDrqwVMT+kd64sN68cxAADL2Bs9fyuGFHxtMySyccnshAVjgubApnvTS/jmQ07LkREDILS4UqKlKAcwL6fOTe/JyJ/iOQK03uMCcwnIohFg7hqqbWyTnH3GKxZ3bXt+xSjcCZa8ZG4JbJQb9PyTKHJ9SO65c3hYTOXsvjpp0DjsXc2iGUVV9dKUFWeGAvon7MQiCIZgL5TWmUiNq7sIRDDAfGUnrsemm9hkJVwsEeSbbuGkpgMBHR4/+ChUIFS+u1njYCh8YzYGyjEeCc49z1PG6Z9D6PMZaJIxJieEWT1gjqRbSRGA7m+RPBI5hGQFNWsGsEAC2BqVWxzC2qF6ypLusGCRhjuGVyED++srbh+AuX1d9v7WEjkIiGsW8ktcGALRSqWC3V9TCXl4RDDDsGE3jxuupFiRxB0KjUfBmNSRA6gTQCjXCQ/cceGs/g4nIRVUnGnLZrnOoRTwAA7j44gheurG0YfvPtl5cRj4Rw177emSNgxJH9w/jBhRW9GU8YBL88mJ1DCZx+dRVAIzwVNMgTIPwmkEZAJIY79QQUDlxYKuGVpSJikVBPTeq69/AYFA58d7Yxn+dbLy/i7oMjPX9heMPBUSwXa3pI6MeasuhNO/0pa50cSkDS1saNPr2m21BimPCbQK60TpvFANUTANTY9csL6zg4mu7oebzirn3DSMfCeOb8IgB1Nu9L8+t40+GxLp9Ze14/pQ6Z+/4r6tjpr59bwK2Tg771YEwOqV3fam6l+30fnVCmcBDhM64YAW3YttltxzYPlTc6ZgfZgScgKoGee3UV3355CXcfHGnzCH+JRUJ4w9Qovn5uATVJwZdeuAYAePMNvW8E9o2ksGMwjmfOL+L6WgXPXlzFO2/f6dvri4lrfnkeXkDhIMJvHBsBbarY4ya3TQP6FKYcY2za6Jjd19SrgzrYwadiEbx2bxZ//O1XUJUU/Oz0btvP4TXvu2c/rq1V8Mg3ZvHpv53F6w+O9HRSWMAYw8/ctRtPnpnHw187BwB45x1+GgHVE7jJh0S0V4jEcJDnIRDBwvFK0y7mcyY3Pwh1GD20+xw1OWYLMXfFrmyE4D+8+zZwqAnh1+7tvWTrW28cx5sOj+LjT72ExUIVv/6Om3qio9kKH3nbYWSTUfz56St40+FRHJ7wb1cupCn8KEn1CjFQJiifNxF8vK6jywJYafp91OTYBhhjxwEcB4B9+/ZtedJ0PIyfumNnx5Oq7tybxW8fuxM7BxM9+WVjjOF37r8TT52Zx+GJDGYO9FbIqhVDySh+/z3TuLBcwgMze3x97dt2DeKz75/B224a9/V13eTGHQP4qTsmu30axDaiJ4uptRnEjwLAzMwM33z7nuEUPv3eI1seZ4djR/y9QNllciiJ999zoNun0RFvPDyGNx72/3UZY7jv1h3+v7CLHDuyp+fXJtFftDUC2q58M3NaGKgdOQBiG5sFIOoejY4RBEEQPtPWCGi7clswxrKc8xyAxwDMaIenAAjDYXSMIAiC8Bk3qoOOAZjR/hc8DQCc89PafY4CyHHOTxsdc3oOBEEQRGc4zglwzp8A8MSmY0eaft7iSXTiXRAEQRDuQ8XIBEEQ2xgyAgRBENsYMgIEQRDbGDICBEEQ2xgmtN97FcbYIoCLJjePAVjy8XTM6JXzAHrnXHrlPIDW57Kfc+57i3GbdW1Gt9/Tbr4+/e32sLyue94ItIIxdopzPtP+ntvjPIDeOZdeOQ+gt87FCd3+O7r5+vS3e/faFA4iCILYxpARIAiC2MYE3Qj0StNZr5wH0Dvn0ivnAfTWuTih239HN1+f/naPCHROgCAIgnBG0D0BgiAIwgGBNQJ25xoT24d+XRv9+ncRG/H7cw6kEbA719jjc2n5oTDGTmj/G81l8PJ1fbsodOs9MHmtnlkbbtLtv8vPz7iba7vba7kbn3MgjUAHc409weKHcpwxNgvz83X9df282HXrPTCjV9aG23Tz7/LzM+7m2u6FtdyNzzmQRqANbWcYu4iVD+WDnPNDFiexufW6fl7suvUedIKfa8NPvP67/PyMu7m2e30te/I596MR8BMrH8qUB65ru9f182LXrfeA8A8/P+Nuru1tuZZ7ctC8R3ONuwLn/GEAYIzdxxg72gO7Yd/pofegp9ZGM0Ff8z30GXtKl/9OTz7nnjQCHs017og2X86WH4r22BVt+tqydj5u0G4x+HlR6NZ7YBmv1oabdHvN99A67+ba7sm17PX6DWQ4yO5cYyevxTl/1OCfePMfQ2Mh6B8KYyyrHTuFxgd1SPvdDdq9ruHtHtGt98AQP9eGn3j9d/XQOu/m2u76Wu7G+qWOYYdou4M5AFNiN8cYe1bMWRa7B+32h31+3Q23e0W33gPCP/z8jLu5trfjWiYjQBAEsY0JZDiIIAiCcAcyAgRBENsYMgIEQRDbGDICBEEQ2xgyAgRBENsYMgIEQRDbGDICBEEQ25j/H4TAu7oOiRMgAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "x1 = np.linspace(-1, 1, 128)\n",
    "x2 = np.linspace(-1.1, 1.1, 128)\n",
    "p = np.polynomial.Chebyshev((0, 0, 0, 0, 0, 0, 0, 0, 0, 1), (-1, 1)) #These are Chebyshev series, a proto of \"chebfun system\" in MATLAB\n",
    "fig, (ax1, ax2) = plt.subplots(1, 2)\n",
    "ax1.plot(x1, p(x1))\n",
    "ax1.set_title('Interval $x\\in[-1, 1]$')\n",
    "ax2.plot(x2, p(x2))\n",
    "ax2.set_title('Interval $x\\in[-1.1, 1.1]$')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Сходимость метода Чебышёва\n",
    "\n",
    "- Заметим, что $p(x) = (1-\\tau_n x)\\dots (1-\\tau_0 x)$, поэтому корни $p(x)$ задаются как $1/\\tau_i$ и нам осталось дополнительно отобразить их из $[-1,1]$ в $[\\lambda_\\min, \\lambda_\\max]$.\n",
    "\n",
    "- Это даёт следующий результат\n",
    "\n",
    "$$\\tau_i = \\frac{2}{\\lambda_\\max + \\lambda_\\min - (\\lambda_\\max - \\lambda_\\min)x_i}, \\quad x_i = \\cos \\frac{\\pi(2i + 1)}{2n}\\quad i=0,\\dots,n-1$$\n",
    "\n",
    "- Сходимость теперь будет такой (без доказательства)\n",
    "\n",
    "$$ e_{k+1} \\leq C q^k e_0, \\quad q = \\frac{\\sqrt{\\mathrm{cond}(A)}-1}{\\sqrt{\\mathrm{cond}(A)}+1}, $$\n",
    "\n",
    "что лучше, чем у метода Ричардсона."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaMAAAEcCAYAAABqCdtUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XtwW1l+2Pnvj5REvUiBlFpqTU+rW9B4PGN70j0gx694naQFju3UvuwhpWw22a2ttYhJ9r2xiZGzW678kdWAY6dSqcqOQU12a/+Isy1yZh27UusM2TPltZO1LZEznoy719NDqFv9mG49SJDUi5Ko3/5xzgUvQAAESICXAH+fKhTIey8uzgVB/HDO+Z1zRFUxxhhjotQRdQGMMcYYC0bGGGMiZ8HIGGNM5CwYGWOMiZwFI2OMMZGzYGSMMSZyFoyMMcZEzoKRMcaYyFkwMsYYE7k9URegVRw7dkxffPHFqIthjDEtZWZm5raqPrPRcW0djERkSlUH691Xzosvvsi1a9caVzhjjNkFROTtWo5ry2AkIkkgDiTr2WeMMSYaWw5GItKjqkuNKEzJeYeAvKpOl9mXAAaAHC6w5MLHBT+LSLb0sdX2GWOMicamExhE5LSIfB/Ii8gdEfnF0L7PicibWzh3ErhcYV8cyKjquKpOq+o4kPLbjTHGtKCtZNN9EUgBvcB54D8RkV8CUNWvAmfqPaGIxH2NJQ7MVzgsBZTWarJApt7nM8YYszNsJRhdU9XXVHXR11DOAUdF5BW/v+6FklQ1p6opX9upZAiYLS2L326MMaYFbSUY5Us3qOplQETk01s4b0UiEqNMrUlV836/NdUZY0wL2lLNSEQuiMibIvJisFFVX/M/Lm6lYBX0+edYFwg9C0bGGNOCNp1Np6rfEpEcrrnurTL7+rdauDJitRzks+2S/ucMMBXKoqu4r8x5RoARgFOnTm258MYY4DtXYGUZPvNfRl0Ss4NsKbVbVReBb1XYd30r594KVZ3F9SuN1bOvzLHjwDjAwMBAzX1gKysrzM/Ps7y8zOrqas3lNqZWnZ2ddHd309fXR1dXV9TFqc+3/zms3LVgZIq05KBXEYlVaaqL1MrKCjdu3KC3t5cXX3yRvXv3IiJRF8u0EVXl8ePHLC0tcePGDU6dOtVaAWnxPTjxo1GXwuwwZYORiPwWblBp3RlxpacC5lT1/BbPE8j5+z5CCRQ+sSG8PzLz8/P09vZy7NixqIti2pSIsG/fvsJ7bH5+npMnT0ZcqhqpwuK78PGfi7okZocpG4xU9fPbXZBaqGre91OV9h314WZriDwYLS8vYxOqmu3S09PDW2+91TrB6MECPHkAPc9FXRKzw7TiEhLTuFpbWMJvj9zq6ip79+6Nuhhml9i7d29r9UsuvuPuj3w02nKYHWcnB6M+ymfPpYHhkm0pv31HsD4is11a7r22+J67P2I1I1OsUp/RF4HTDXqOnKperOVA3/dzETdeKAZkRGQQl349CYWmurRPy77qj83shCY6Y8wGloJg9Hy05TA7TqVsukv4AaYNUGmOuXV8htyGNZxQerYxppUsvgOd++CgJfiYYpUSGBZpzgwKxpjdbPE9l7zQsZN7CEwU7B1hjNk+i+9a8oIpq65gJCI9InJFRFb97aqI/L1mFc60ntnZWURk3a2/v5+xsfWTXoyNjTE4WPPq76TT6Ug67cfHxxER8vkdOda6dSy9Z8HIlFVvzWgMlzRwDvgCcB34kl9c76VGF860rkwmw9zcHHNzc8zMzJBKpchms5w5c6boA/3q1atMT++IrHzTbE9XYel9G2Nkyqp3OqAZv0xEEb9E+KSIDJZOmmp2p1gsRjy+Nol6IpHg3LlznD59mgsXLjAxMQFQuDe7wPIHoKtWMzJlVawZiciXReQXRaQntLns9EA+7fozuPE+xpQVi8VIJpNWE9qtFt919xaMTBnVmulSwFeBBb9m0ZeBmIhcKnewT8uea0IZTRuZnZ2lr69RowZMS1nywcia6UwZ1ZrpplX1s379n7PAIL7m45vlpoEZ3HpG3/YL7PUDX2lqiVvYP/i9P+f195eiLkZVP/KRHn79P2jsjMr5fJ5r166RTqfJ5XJks9nCvnQ6zdjYGKprle7Z2VnS6TTXrl0DYGBggEwmQyKRWHfM9PQ0iUSCy5cvF+3P5XKF/fl8nmQySTabLWo6TKfTjI+Ps7CwQCqVYnp6mpmZGWKxGGNjY1y65L53nTt3jjNnzqy7rsnJycI1Aeueo7SMqVSKkZGRdc//2muvVb2WtlGYfcFqRma9jWpGqOqsqn5JVT+rqh24gDMOHMUlNMyKyCowAWSaXWDTGlKpVCGTrre3l8HBQWZnZ8lkMkUfyKWmp6fp73frMl6+fJlMJkMsFuPVV18tOm54eLiQFJHL5RgeLp4hKgh4ly9fZmpqivn5+cJ5S/X393Pt2jVSqVQhEKXTaQYGBrh8+TL9/f2k08VjsYPnHBoaYmZmhomJCeLxeCE5Y3Jykv7+fuLxOFNTU5w/f55UKkUqVdySnc/nN7yWtrH4LnT1wP6ejY81u4+qbukGHNnqOVrh1t/fr7V4/fXXazquXc3MzCigo6OjOjMzozMzMzoyMqKALiwsrDt+dHRU3dvQicfjmkwmK54/OH5iYmLdtnLnLy1XuceVPl8sFlu3LZPJFD3HxMRE1eeMxWI6MjJStG1qakoBnZqa2tK1lGqZ99y/+Juq//Qnoy6F2Wa41rMNP2O3POhV3WwNxhQ5c+YMiUSCRCJBJuMqzFeuXKn6mFwuV2he20gymSx6ro0EzV5Bk1pYuNlwdnaWfD6/rgYTixXP2Rs8/9mzZxkbG2N2dm12qqBpsPQcyWSSWCy2LoOw3mtpWYvvWH+RqchmYDBNF4vFGB0d3TDIBIGilgSH0uBQzuTkJMPDw/T399Pb21vxuHA/UlCG8LZKzz81NQW4vp/+/v7C4N1q54jH4+sCYi3X0hYWbcCrqcyCkdkWFy+6idurBaTgw3t+vua5dSsaHBwknU4zODjI5cuXee2112p6XD1lSCaTzMzMsLCwQCaTYXp6mrGxscI5ytXCcrnchoGuLT1+APdv29IRpiILRmZbxGIxLl68yNjYWMUpdeLxOPF4vNCsF1bPNDy5XI7p6elCskQikag5ACQSCWKxWFHTHcDcXOVRC0HNL5FIcPXqVQYG3NqPpeeYnJwsJCzsOkvvu3tbOsJUUO8MDMZs2ujoKJcuXSqagaFUNptlcHCQ4eFhzp8/z/z8fOHYoFlsI/F4nFgsVqiFxWKxsgGukosXL5JOp0mlUgwPDzM7O7tuXr3x8XEymQypVKrQ9DY7O1vIyMtms6RSKebn5zl//nyhL2xoaKioj2jXCFZ4tT4jU8GuqxmJSEJERkVkSEQyfkE/s00uXrzI5ORkUYd/WND0FdQggtTu0lrGRi5fvsz8/DzDw8Ok0+m6aiOjo6OMjo5y5coVUqkUd+7cWRfMzp07x9DQENlsluHhYbLZLKOjo4W09ZGREaampgqp2tlslkwms3unP7IVXs0GRLXsDD8bP1DkCHBBVX+jsUVqHh94ZlT1jP89DqRVdcNpjAYGBjQYhFnNG2+8wSc/+cktl9WYWrXEe+4PxuCb/xD+p5uwpyvq0phtJCIzqjqw0XFbaabrw81Ht+38DBB5VV03yZmfMWIAyOGWJM+Fjkv67QCoak5ERrA59YxprsV34NBxC0SmopbrMxKRJHAZWNfu4ms6GVUdDG2bEJGcquaAPGWWUxeRuN9vjGkGS+s2G2iZPiMRiYtIFlfbqZR3mwJKOxey+GmKghpS0E/kAxuA9RsZ00yL71p/kamqZYKRquZUNaWq41UOGwJKe8av+e3BefqBpG/OC2pDVisypllUfTCytG5TWcsEo4342s66WpO6pS2CJrxg26SqBkErFxxjjGmCBwvw+J4105mq2iYY4fuCqgSWOICILIS2pYCNJ0Izxmzeki0dYTbWcgkMVdTa75P22Xh9wFV1q9SW5TPtRgBOnTq19RIasxvZCq+mBu0UjGqyQZ9TuWPHwY0zalqhjGlnhWBkfUamsnZqpgPWMuWMMTvE4jvQ2QUHj0VdErODtVMwCjLiisYRhYKTZcwZE4UgrbujnT5uTKNt9d0hDSlFA/jEhRzr+476cLM1WDAyJgqL79oEqWZDmw5GqnqdnZeJNo2bCigs4bcbY6JgY4xMDbZUM/IBKQp9lM+eS7N+miBL3zYmKqtPYPkHlklnNtQy2XS+7+cibrxQDMiIyCAwFaRnq2peRNIikgGu+mMz1kRnTESWfwD61IKR2VDL9Ciqal5V06o6rKqiqmf89ECTJcfN+uMmVXWs3MzeZnvMzs4yPDxMb28vIkJvby+Dg4NMT6/9SdLpNCJb73ps1Hmifo62Y2OMTI1aJhiZ1jI+Pk5/fz+zs7NcvHiRiYkJLl68yPz8fN0L5ZkWZmOMTI1appnOtI7p6WlSqRTJZHLdUuGjo6Pk8zYV4K4RLDduM3abDVjNyDRcKpUiHo+vC0SBWMzGJe8ai+/CgV7YdyjqkpgdriHBSESuiMgvhX4/KyI9jTi3aS2Tk5PkcjnSaUtgNPi0busvMhtrVDNdVlVfC35R1ddE5BXgGw06f3v4v78AH/y7qEtR3bOfgl/44qYffvXqVQDOnTtX1+NmZ2dJp9NMT0+TSCS4fPkyiUSi6JjJyUkuXbrE7OwsiUSCTCZDMpkse55r167R19dHNpslmUySTqcZGxtjYWGhqGY2NjZGOp0ubJ+cnCSdTpPLuQTMZDJJNpslHo+ve47NlDWVSjE+Po5q8VSHpeVoG0vvQcwmGTYba1QzXU5EXgQQka+LyFVgsOojTFsKPsTr/UAdHh4mlUqRzWbJ5XIMDxcPF5ucnGR4eJhkMsnExATxeJzBwcHC8wUuXLhAKpUik8kAFI5JpVKAS6wIe/XVV0kmk8RiscLzDg0NMTMzU3ie0j6urZQ1KMfkZPFk8a+++iqJRKK9AhG4PiOrGZlaqOqmbsCbwJeBV/zvLwO/BHwOOLLZ8+7UW39/v9bi9ddfr+m4dpVMJtW9rWozOjqqgE5MTKzbtrCwUNgWi8V0dHS06LGJRKKwLXhM2NzcnAI6MjJSOD4ejxf2LywsKKDZbFZVVScmJtY9bzPKGovFNJlMritnUI567dj33INF1V/vUf2jfxx1SUyEgGtaw2fsVmpGX8VNs3NORN4ExoAMMKeqi1s4r2lhQXNVvRlz4ea2M2fOFO2bnZ0ln88zNjaGiBRus7OzzM6WrjK/Jh6Pk0wmuXbtGgAXL14kl8sVHhPUkkZGRorKcPbsWcbGxiqee6tlHRkZKRprFdSS6m3a3PFsUT1Th63MTfcFVf2qqn5eVX8IN+3OGPBrIvJ9Eflyw0ppWsZnPvMZAK5cuVLX46o1TwVNcTMzM8zNzRXdJiYmqp43Ho8XHj80NEQsFuPVV18FXNPY0NBQURmCDMB0Ok1/fz+Dg+tbm7da1tKmunBTYVsJxhj1WDAyG2tYareqXlfVy6p6TlU/hgtMZpcZGhoiHo8X+mwaIZw8EI/Hi24bfYDncrmix587d47x8XHy+Tyzs7OFwBBIJpPMzMywsLBAJpNhenqasbHa38q1lDX4PZvNVixHWyiMMbJgZDZWdzCqNWVbo5tE1USsUsd+oFrTWjlBx365dPFqzYH5fJ7p6emiZrV0Ok0+n2d8fJxYLLYuGy8Qi8UYHR0lkUgUMgQbWdZUKsX09HShuS5cQ2sbi++CdEL3s1GXxLSAuoKRiFwBFoLMudD2VxpYJtPigiyyyclJ+vv7GR8fZ3JykrGxMQYHBzl79mzd57x8+TLT09MMDw8zPT3N5OQkg4OD6z70h4eHyeVyTE9Pc/bsWWKxGBcvXizsj8fjJBIJLl26tK6PZnx8nDNnzjA2NlYo7+zsbNmmuq2WNQg+ly5das9ABLD4nlvHqKMz6pKYVlBLlkNwAy4Anyuz/QjwK0BPPedrpZtl09Vvbm5Oh4aGNBaLKaCxWEyHhoZ0bm6ucEy5LLhsNls2q21mZqaQrReLxXRkZKRwTDjTLR6PK6DJZLLouUrPPzMzU7R9YWFBR0dHC4+Px+NFWXGNKmsgkUisy87bjB37nvvf/rrqP/v5qEthIkaN2XTijq2NiPwqboDrUoX9v6yqX9lMUNzpBgYGNMjKquaNN97gk5/85DaUyGzW2NgY2WyWubm5qIvSEDv2PfePPwXP/wR8ri0/EkyNRGRGVUsXPV2nrhkYVPVLIvKvRWQW+LqqfnPTJTQmItlstj0TBnaSp6uw9L7N1m1qVlcwEpHfAgQ3u0JaRBSYBa4Bedyidzv6a5CILLB+ldi0qlr2XxsLxhddvXqVXC5XGFtkmmT5A3j6BGIWjExt6p2bbk5VPx/8IiJJILgpUH/P9Dbyq8UOa2jBPREZtUDU/vL5PBcuXABgYmKi/cb07DSFtG4LRqY29Qajojxa/6E+DSAinwbOsQ01IxEZAvJaZhVXEUkAA0AOt+x4LnxcSSAaAiZLz2HaTyKRYGFhIepi7B62qJ6pU73jjKZF5Jcr7NuW/3RfG7tcYV8cyKjquKpOq+o4kPLbUdV86NgY0KequXLnMsZsQf6Gu7cBr6ZGdQUjdQNZJ0Tkl8ODX0XkNK4m0t/g8hWISFxEsrjaznyFw1JA6ZrWWdyceaUuAvXNWWOMqU2wqF7X4ahLYlpE3TMwqOqiqn4lnN7tg9QgTZwCSFVzqprytZ1KhnAJFWHX/PZSyXBNqZHqSZc3Zit27Htt8R1rojN1aeTcdK9phFMA+Wa3dbWmIOAETXX+52TpcY3S2dnJ48ePm3FqY9Z5/PgxnZ07cIaD/Du2qJ6pS8OCEUDEM3X3QXG/UIl46OcEJckYjdLd3c3SUtkxwcY03NLSEt3d3VEXo5iqLapn6ralYCQiPSLysoi84hMbolyQpd5c3Q0TF0RkRESuici1W7du1XTSvr4+FhYWuH37No8ePdq5zSimZakqjx494vbt2ywsLNDX1xd1kYo9zMOju9ZMZ+pSMbVbRF7G1TbiwBnch308tK30w19wY412vFrHFfn+qXFw0wHV8piuri5OnTrF/Pw8b731Fqurq1soqTHldXZ20t3dzalTp+jq6oq6OMXyfoyRDXg1dag2zmiMtcGs4rfl/O01XEDK4vpe8sB1dkB2mojEmpWYUKuuri5OnjzJyZMnoyyGMdEojDGyZjpTu4rBSFU/6weFJnHT5RQtJS4iv6Wql0u2rRuEuo2CZrc+Qv1BPrEhvN8Y00yF2RcsgcHUrmqfkapO+ul/hn2zXdHuMsd/oZGFq4evDeVY33zYh5utwYKRMdshfwP27IdDx6IuiWkhNSUw+GUhFltgEb1p3FRAYQm/3RizHRbfdU10Ihsfa4xXczadql5X1W/47LnTzSxUDfoonz2XBkrXuk757caY7WADXs0m1DtRKqr6bQARmRWRnkoL7TWa7/u5yFomX0ZEBoEpVZ30ZcuLSFpEMsBVf2zGmuiM2Ub5d+DjPxd1KUyLqTsYBVT1soicFpEB3LKySyLyS6r6tQaWL/x8eWqo4ajqLOunBDLGbIfHD+HeTZt9wdRtS4Neg6Y7YNAvvFd2Nm1jzC6x9J67t2Y6U6eGTAekql/FDQ61FcuM2c1s6QizSY2cKHUWNxjWGLNbBQNebfYFU6eGTpSKy1wzxuxWi+8AAj3PRV0S02IaGoyiXELCGLMDLL4L3Sehc2/UJTEtptE1I2PMbpa/YU10ZlPKpnb7zLgBtj4LtwBzqnp+i+cxxrSCxXfgudJJUIzZWNlg5OejM8aY2j1dhcX34Ef+46hLYlqQNdMZYxpj+QN4+hh6X4i6JKYFWTAyxjSGLR1htqBSn9EXgUZNhppT1YsNOpcxZqcKBrzaVEBmEyrNTXcJNzN2I8w36DzGmJ0s/7a7t2w6swmVEhgWgcVy+4wxpqz8DTh0HPYeiLokpgVZn5ExpjHyN6yJzmyaBSNjTGPk37EmOrNpuy4YiUhGRFREFkRkSkTiUZfJmJb39KnLprOakdmkTS+u18LmVFWiLoQxbeXuh7D6yIKR2bSWDEYiMgTkVXW6zL4EbiqjHG7Z8Vy544wxDVRI67YBr2ZzWi4YiUgSt6LscJl9cSCjqoOhbRMiklPVnN8UC4IZMAhc8kuaG2M2q7ConvUZmc1pmWDkA00amKHy2KUUkC3ZlgUyrAWv8SD4iMg8bkHA/oYX2JjdZDGoGVkwMpvTMgkMqppT1ZSqjlc5bAiYLdl2zW8PzpMP/TwLJETElks3ZivyN+DgMdh3KOqSmBbVMsFoIz6gxCmpNYVqQXERSYjITOljrZnOmC2yMUZmixoWjETkZRH5ZRF5uVHnrFMfVA0scVxSQ6EZz/c/TTa/aMa0OQtGZosa2Wf0a7jmsJyIKJBW1a818Pwb2bCpTVXzIpITkRG/6QxwodLx/rgRgFOn7B/NmLJU3XLjP/wLUZfEtLBGBqM0LjPtWwAicqSB526YetK8ff/UOMDAwMBWV701pj3dvQlPHlpat9mShgUjVb1e8nskE62KSMz6gIzZRrZ0hGmAtklgwPUHQcnSF6FMuRzGmMYrLB1hwchs3qZqRj5Joex6R6r6jS2VaJOC/iDW9x314WZrsGBkTDPYgFfTAHUFIxH5HHAl+LXMIQp0brVQWzCNmwooPNYo4bcbY5ph8R040Addh6MuiWlh9daMMsB1XLJClDWNPspnz6WBCXzSgZfyN2NMM1hat2mAeoNRHBja5pRtoND3c9GXIQZkRGQQmFLVSSg01aVFJANc9cdmrInOmCbK34BnPhF1KUyLqzcYTVOhr6jZfIZcuobjZlk/JZAxphlU3aJ6P/TZqEtiWly9wSgFXPFjiF7DzXxdRFXfakC5jDGt4O5NePLAxhiZLdtMM90ZYKzKMVEmMBhjtlOQ1t1rwchszWYSGAC+BMw1uCzGmFazEIwxsmBktqbeYJQARlT1K80ojDGmxSy85e4tm85sUb0zMMxSeWE7Y8xuk38LDp+AfQejLolpcfXWjC7gEhgWgBlVXWpCmYwxrWLhbWuiMw1RbzB6DTfGZxpAZN0kDKqqLbOUuTFmi/Jvw/M/EXUpTBuoN3Bcxk35Y4zZ7VYfu3WMPnUu6pKYNlBvMPqH1jRnjAFcINKnltZtGqLeBIYZEfl7TSmJMaa1FMYYvRhpMUx7qDcYfQP4G80oiDGmxdgYI9NAdQUjVU0BIiK/LyL2DjRmN8u/DdIJPc9FXRLTBupdzygYY5QAcpZNZ8wutvAWHPkodNq/vNk6y6YzxmzOwtuWvGAapq5gpKppABF5BRjGTZyaAyaiWm7cGBOR/Nvww78QdSlMm6g3gQER+TJu0Ot54Ki/nxKR/7PBZWsKEUmISFJEhkQkKyLxqMtkTMt5dA/u3bLkBdMw9fYZXcCtaTSoqq+Ftg/hpgm6qqq/2eAyNtprwGm/Kmwfbpny/ojLZExryd9w95bWbRqk3ppRChgNByIAv+z3F4DPN6pg1fhaTbLCvoSIjPjaz0iZ4077VWPBJn01ZnMKs3Vbzcg0xmaWkBitsG8O14fUVD64XMb1WZXuiwMZVR0MbZsQkZyq5qCwfHkgRQ1LmRtjSizYonqmseqtGU0DQxX2ncctMdEUIhIXkSwu4FWq0aSAbMm2LGuLAobPNQpMqep0wwtrTLvLvw17D8KhZ6IuiWkT9QajLwCfF5H/VUReFpEef/9l4HM0sZahqjlVTanqeJXDhlgfEK9REkD9ucaAvIhMNbioxrS/YOmI9WMNjdmUemdgmAXO4aYEmgEWcB/+KeDzUaZ3i0iMMrWmoFnO14aCGlHgCpC0jDpj6pS3MUamseoeOu2TFSZ9381p3Dija6q62OjC1akP1vUJhQUB52jJtnzQn2SMqYGqS2B44S9HXRLTRjY9j8cO7GuJbXSAqk6LSExERvymQeBspeP9cSMAp06dakghjWl59+fh0V2rGZmG2lQwEpGeSvt2+npHvmYXqNb/hO+fGgcYGBiwaZCMAVi47u57T0dbDtNW6h30+mlcP0u1PpbOLZVoi0QkVqWpzhizVfM+GPVZMDKNs5mJUo8CX8KNK9pJgn6fPqAQjHxiQ3i/MWYrCjWjFyMthmkvmxn0OqKqX2lGYbbCT++TY33fUR+WpGBM48xfh+6TsPdA1CUxbWQzg1538hQ608BAybaE326MaYSF69ZfZBqu3mCUBjIi8ovVkhi2QR/ls+fSrJ8myKb8MaaR5q9bf5FpuHqb6V7DBYFJgO1c6dX3/VzEJU/EcEFxEDelz6R/8ryIpEUkA1z1x2asic6YBnl0H+5+YDUj03Ats9Krz5DbsIbjZ4lo2hx5xuxqeT9BqtWMTINtaqVXY8wuNW9jjExz1L3SqzFmF1uwMUamOSwYGWNqN38dunrgQG/UJTFtxoKRMaZ2C9fdYFdbOsI0mAUjY0ztLK3bNIkFI2NMbZ6uQv6GJS+YprBgZIypzeK78PSx1YxMU1gwMsbUZuEtd281I9MEFoyMMbWxtG7TRBUHvYrIU+qfbaFp0wEZYyI2fx069kLPc1GXxLShaoHjS0Q09Y8xZgdauO6WGu+IdP1M06YqBiOb+scYU2Telo4wzWN9RsaYjam6BAbrLzJNsqn+HRH5JdzyDOuo6m9sqUTGmJ3n/jysLFnNyDRNXcFIRE4D14BgYioFwvOCKGDByJh2M++XBLOakWmSepvpssB1oF9VO4BF3LLefbilvVONLV7ziMhU1GUwpmXMz7n7vjPRlsO0rXqb6ZJAUlW/5X/PQWGF1QzwReArDSxfw4lIEtfEmIy6LMa0jPkcSIebJNWYJqg3GOWBI6Hfc8AA8G3cUuCJBpWrKhEZAvKqOl1mX8KXKYcLOrnwccHPIpLdjrIa0xbuzMGR52HPvqhLYtpUvcFoGvgbwP/lf78CXBKReSCDC1ZN5Ws2l4HhMvviQEZVB0PbJkQkp6q5ZpfNmLY1Pwd9ZXOWjGmIevuM0rg+IwBUdRJ4C5jE9RtdaFjJSohI3Ndm4sB8hcNSuH6tsCwuUBpjNkMV7uTgqPUXmeapq2akqtcuS+EqAAAcC0lEQVSBL5RsGxSR035f0/iaTQpARCoNyB1ifTC6BliygjGbdf8OrCxa8oJpqoYMem12IKqFiMQoU2tS1bzfb20MxmxGkNZtNSPTRPWOM+rZ6BhVXdp8cbakzz9/pX6rOD77zxhThzuW1m2abzPZdBtNnhrVLIqxWg7y2XZJ/3MGmCqXlef3jwAjAKdOnWpQMY1pMfNzLq07Zv8DpnnqDUafp3wwGsAlL4xsuURNpqqzwCwwVsOx48A4wMDAgM1gbnanO3MuEFlat2miehMYxivsuiwi3wcir8eLSKxKU50xpl7zc9ZEZ5qukbN2T+Oy2aIS9Af1hTf6xIbwfmNMrSyt22yTRgajYJqdSPjaUI71fUd9uNkaLBgZU697t+HRsg14NU1XbzZdtaXIBZjZcom2ZhrXfzUb2pbw240x9bIJUs02qTeBodpS5HO46YG2Qx/ls+fSwAQ+6cBL0UKziRuzowRp3dZMZ5qs3gSGyJYi930/F3FNgTEgIyKDuNTsSV++vIikfcr2VX9sJsomuv/vT79O/rtT9Jx4kWefP0Pvsy8gh56BA73QEVUWvDE1mp8D6bS0btN0m1rpNQq+T2jDYBhK3d4R8t/7N/zkjXG4gQuP3lOElT3dPN4XQ/fH6DjYy95DvXQdiiEHjkBXD+w/4m5dPdDV7W77e9Z+79wb2XWZXeLOHPS+YO8103QVg1Etsy2UE+EMDDvST/6tf8Dd+2nm5t7kvbe/z+LNt3m0dBu9d5s9D+fpXrlHbPkuR+QduvkeR+QeR+Q+e3my8cn37F8LUl3dLkjtO1yyrbs4mIW37ffb9+wHkY2fz+w+8zlLXjDbolrNqJbZFsqxtqcShw8e5KVPvcRLn3qpaPvTp8rteyv8IP+QHyw+4M+WVvhg6SEfLj5kYWmZu4t3eHB3gY6VZQ7LA7q5T7c84DAPiHU84Jk9jzm2ukLfo4f0PFmh+/5dDj79kK6nD9i7eo/Ox3eR1UcbF7BjTyhohYJU4efwfai2Fvy8/wjsPWABrd2oumB06qeiLonZBaoFo3KzLcRwyzGkKV676GPArwKjDS1dm+voEI537+d4935eer7ybEYPHq1yc/khHy6trN0vPeTacvHvSw/X16YOda7ywuGnnDr8hI8eeMzJ/U840fWIY3tW6Nu7wpGOFbrlPgdW7yGPluHhEqwsw9J78PAN9/PKEjzdoKbWsScUnGJrPx+IrW070Ot/D/18oNcFNgtkO8/dm/DoriUvmG1RMRiVm21BRL4MjKjquqXFReQ2cLSxxTMAB/Z18sLRQ7xw9FDV44KgdXN5hQ+XHnJzaYWbyy5Q3Vxe4Q+WHnLzvRUWHzxe99g9HcIz3V0c79nP8e4uTjzbxYnu/Rzv6eL44S5OHFJO7H1Eb+cDOlaW3JICD4Pbkr/P+5/z8CDvAtqDvPu9Wg1NOn1w6oWDfXCgz9/3wsGj62+HjrmA1tHIYXJmHUvrNtuo3gSG81ReG2gW+C1cxpuJQK1B6+HjVW75WtXNJR+4llcKNa8bd+5z9a158vfXB63ODuGZw0HQOrIWvHr2c/w5f9/dxdHDXXR2+NqOKjx+4IPUwlqAerBQfLs/Dw/mYfFd+OA7bh2dJw/LX4R0+OB0zAWnQ8+42+Fn4NBxOHwCDh93t0PHbV61zbj9PXd/7IeiLYfZFTaTTTcCfK3M9mFKpuIxO9P+vZ0833eQ5/sOVj1u5YkLWh8urXCrTI3r3YUHzN7IM39vfa2ns0M4emhfITgdL9wf5kT3MVfjOrmfY4f3saezSg3n0X0XlB7Mu9kA7t9xt3u34f5td3/vNvzg2+5+pUL+zMGjcPhZ6H4Wuk/6+2eh5znoOenuDx6z2lbYre/BngNw5PmoS2J2gXqD0ReBL4rI7+NWVL2OC0ApXDCy5b3bSNeeTj7ae5CP9lYPWo+ePOX23bUa1s2ltf6tm8srvL/4kD97N8+de4/Qkl5IETh6qIsTPV0uWHXv50RPF8/07OdEUOPqOcax48+xt1rQCjx+APduwd1bcPdDuPuB6/tY/sD9vvwB3HzD/ayrxY/t3Ac9H3Efvkc+6u5jz0PsBTfO5shHd1eK8+3vwbGPWYA226LeQa9j4jqavwB8FpfgILhkhrSqfqnhJTQ73r49HXwkdoCPxA5UPe7xqgtaQc0qHLyC37/7/hK3765UCFr7eMYHq6LA5fu2TvTs55nDXeyLndp4kObTVRe0lt6DpffdbfFd9/vie3D9/4HlH4A+DRWi0wWk3heh77TrSzl6xt33nYY9XZt7AXeq29+Djw5EXQqzS9TdTKeqY8CYX6TuNJBT1W81vGSm7ezt7ODkkQOcPFI9aD1Zfcrtu4/W+rSW1ydjvPGDJW4tr/C0zOCDvkP7ipoGXfAKBa7uLo73dNEVNNU911++IKuPXXDK34CFtyH/Nsxfh4W34PXfdU2HAelwNahjH4dnfhie+QQc/4S731e9D29HevzAXffLfzPqkphdYtMzMOy0mQ5M+9jT2cGzR/bz7JH9VY9bfarcuRfUtIJkjLXgdWv5Id/7YJlbd1dYLRO1Ygf3rmUMBrWrcP9W936O9zzP/t4X3deuUg8W3DicO3Nw+02486brZ8l9M5Q9KK7WdOJH4dmX4ORfgpMvuSC4k935PqAuuBqzDarNwPB14BVV3RPaVm3WbgANH29MM3WGxmnBkYrHPX2q3Ln3iA+XHhayCIvGbC2vMHfzNjeXV3hSJmgdObB3LWOwKBmjixM9pzn+3Cc4/on9HNjnx3uvPnG1qJuvw4evw4ffdbc3fm/tpN0n4SMJeO7T8NEfd7WzrsMNfoW2oJBJZ8HIbI9qgWOa4oGtUH3WbmN2pA4/huqZ7up9Ok+fKvP3HxXVtIrGbS2v8CfX73FreYVHq0/XPb57/55C0HKB62Mc7/kxjn/ib3P8M108u/8xJ+6/yf7b34X3vwXvzcBf/Cv3YOmEZ38MXvjL8OLPwAs/7cZZReX2m4DYgFezbURLe4pNWQMDA3rt2rWoi2F2AFUlf/9xoTkwCFS3lsNjtty+ckHrcNce3zTYxYuHHvMyb/Lxx6/z/N3vcHThz+hYXUER5CMvw5mzcOYVeP7HtzeTb+K/cMHyv//O9j2naUsiMqOqG2bC1Lu43otATFW/Hdr2K8BngK+r6j+rs5zGtBwRoffQPnoP7eMTVbp+VJXFB4/Xjc8Kmgs/XHrIv31vld9Z/ggPHz8LvMI+HvOSzPFTHa/zV9/7d7z0/j+i8w9/g/sdh8nFfpoPTv41Hr5wlr6jRwtJGYe79iCNnk7p9psuEcOYbVJv/04Wt7T334FCv9JZ4FvAuIicUdVfa2wRjWlNIkLs4D5iB/fx8RPdFY9TVZYePilkCt5cHuCDxRV+b/kh/zx/h2fv/Ak/evf/5Sfu/DE/Nv91Vr67l28+fZnfXv1Jpp8mkL0HffLFfp7xNS7XVFicRdizv8ag9fSpS8aI/5UGvhrGVFdvMBoEhgBE5NNAEhhV1d8QkVHgAtASwUhEplR1MOpyGCMiHDmwlyMH9vJDZYPWzwKgq0+4d/2Pefydr/HKm7/Hzz+4yqPOQ3y3N8k3Dv4cVx+d5vX3l/jm0kPuP1pdd5b9ezt8wocfkxUkYYTGaR3v7uLIyvvIk4c2DZDZVpvJfAtWTU3ikhmCCVXncCur7mgiksSVMxl1WYyph3Tu4dDHfgY+9jPw9Evw9r9h37f/BYnXf4fE7X8Jx38UXvk8fGqYu0/3Fs+EsVScRfjGB0v8wfdWuLuyfjb25N4/4yud8Pf/6BHzb8wUAteJcBZh935iB/c2vnnQ7Fr1BqMccE5EcrhJU2dDi+nFWQtUdRORISCvqtNl9iWAAX/+OG6g7brjahE8TkSymy2rMZHr6ITTP+tuv5CBP/8a/Oll+N3/BqZ+ncOf+WUO/+TfIf5M9Yn07z96UpSEcXN5hRe+9ydwA253vcD3P1zmj75/m+Uyy5Ps6+wo1K7Kpb4Hv/ce3EdHhwUtU11d2XQ+YFxhbRqgpKp+w+/7PjChqnXP2u1rKxPAcGmQEZE4kA03qYnIBG76oa0EP1XVmv9DLJvO7Hiq8NYfwR9/2aWM7+uGn0jBT/1XbkmOWv3ef+dmmEhfL2yqtjxJeIaMcsuT7O10M70H8w2GmwaPh+6PHrKg1Y6akk2nqpMicgZIANOquhjancEFqnoKGcct1DcDzFc4LIVLnAjL+ucbruf5jGlrInD633O3D/8c/mAM/vA3XI3pr4zCj4/UtpTG7TfXDXbdzPIkHywWj9O6tbzC23fu86cbLk/SVahhrfVnrQWuo4dCy5OYtrGZPiPFNZldEJFzqrokIkdwAerVuk7kajYpABFJVzhsiPXB6BqhdZVEZASoNjpvarPNesa0pBM/Cuf+DzcDxNT/DF//+zDzv8PPXYKPf7b6Y2/9BXzir2/qabe6PMmHoeVJZt5eYKFC0Cq/PElxMsbRQxssT2J2lHrHGZ0Fvo5L5f40bvmIJVVd9DWmLwJ/t1GFE5EYro+oqNakqnkRQUTiqportyqtMQY48SPwt74K3/s6/Otfg98ehr90Hn7+i+Wb7u7Pu3WimjwNUD3Lk9y6u9Y0eGu5vuVJjh0O1aqK5h0Mlifp4tjhrtqWJzFNVW/NKANMqup5P09d2DiuBtOwYIRfrE9VS6clCmwpacKYXePjn4X4X4U//E3XdJf7A/gP/wl8/OeKj7vxx+5+h8xJt29PB8/FDvBcHcuTfLj0kA+XV7i1FJ6D8CHfeXeRO/dqW54kXOsKfn6m24JWM9UbjBLA/1Jh3wIQ21px1mn0+YLMvKT/OUOVJjzf/DcCcOrUBuvjGLPT7dkHf+0i/PAvwO/8Xfjtc/DT/y2c/XXo3OOSH752AfricOqnoi5tXepZniSYNLc07T1Iwnjdr6lVbnkSF7SC/qyuQvJFMLDY3XfRtaezSVfavuoNRrO4/qJyy46ncJOr7mihpS/Gajh2HD+OamBgwCbxM+3hIy/DyDfh9y/Cv/0nbtLWxH8Ov/tfu4UD/7N/Cft7oi5lU+zp7ChMZFvN6lPlzt3iRSDXFoN0wesvPlji9t1HNS1PUlTj8tue6e5i/14LWoHNLDt+RdxINwWOiMjLuOa7JFBhlbKtEZFYlaY6Y0y99nTBv/+P3NIV/+p/hLf+EJ79FPzt34FDx6IuXeQ6O8T1L/Xs58eeq7w8yepTZb7a8iRLD/n+zbvc2tTyJGvrahWWJ2ljm0ntPoerLQiuhhEsO34uPIFqgwT9QX2ElrPwiQ3h/caYzfj0f+qWrvjOFfjZX4l22YoW1Nng5UluLj/k8er6oNWzf8+6wcTlkjEO7mvd5eQ2s+z4JDDp56YLZkNoyrLjPmsux/q+oz7cbA0WjIzZqpMvuZtpmo4O4dhhl7n3I1RuAlVVFu4/LqpZBYOLg8D1p9fnK66pFV6e5EQ4eJUErkNdOy9obWXZ8W/hUrwLROTlJtSOpnH9VOElzhO0QP+UMcbUQ0ToO7SPvhqXJ/mwJPkivDzJ7I0Fbi6tsPJkfdA6tK8zNFluELTWBhYHgawpy5NUUDUYiUgPcBnXHzQPfLHcmkX+uK8AnwM227jZR/nsuTRuqqDwWKKUvxljzK4TXp7kh5/dYHmSB0+KmgTD47RuLj3kO+/m+XDpIQ8frw9aB/d1cry7i/9h8OP8Ry8/18xL2rBmNIsLEtO42si4Tyb4zeAAEbkEjAKLwBfqeXLf93MR19wXAzIiMohLt56EQlNd2qdhX/XHZqyJzhhjqhMRjhzcy5GDlZYncVSV5ZUn6/q0ghkxjh6q3ifWCBWDkYj8qv/xdDAHnYjMAGMiMo6btTuDS2D4gqp+qd4n9xlylaYBCh8XpGMbY4xpMBGhZ/9eevbv5WPHD0dShmo1o/O4ZrnwZKhp3HRAeVwQGgMulRxjjDHG1KVaMErgJiQNu+rvx3G1IQtCxhhjtmyjiZaKBpqGgk/WApExxphGsVn/jDHGRG6jbLrPiUhpDUhxS4+XXblPVb/SkJIZY4zZNTYKRpUy5CqlcCtuvJExxhhTs2rBqCmTnhpjjDGlKgajZs03Z4wxxpQSLV320JQlIreAtzf58GPA7QYWp1XsxuvejdcMu/O6d+M1Q/3X/YKqPrPRQRaMtoGIXFPVsgkf7Ww3XvduvGbYnde9G68ZmnfdltptjDEmchaMjDHGRM6C0fYY3/iQtrQbr3s3XjPszuvejdcMTbpu6zMyxhgTOasZGWOMiZwFI2OMMZHbaDogswUikgAGgBxuhdqcqk5HW6rGEpEh3LWd8ffZYJXe0DFt+zr41Yozqpoq2d6W1+yv6zxwBziK+3vnSva3zXWLSBK3nA64651T1fGSY1r+mv3/cb5cuWu5voa8Bqpqtybc/B9kqmTbBBCPumwNvMYhIBH6PQbMASO75XUAssDEbvjb+793tvT62/W6cUEoWeY1aKv3N5AEFkqvtdbra9RrYM10zZPCfVCFZXFLtbeLuLol4YHCMvIZiq+7bV8HEYkDfWV2td01+xrgZQ3VAP236WTosHa77pSWfLtXV+sfDB9Di16ziMRFJIsLJvMVDqvl+hrzGkQdldv1hqshxEu2xdxLHn35GnB9MWAGiJVsj+Nmb4+3++sAjPhbac2o7a7Zf7BkymwPf0Nuq+v27+913+4J1QLa5Zr9dZSrGW14fY16Daxm1AT+W+S6bxvqag7BN+qW5q8l7m9ltfPr4PsSrpTZ3q7XPAJcLd2ovr+oTa97Gpjy/SFAoTY44X9ux2suqOX6GvkaWDBqjj5Y+4OU0dJv0oCq9mqomc5L4jpCc7T36xCrcF3tes0xICciIyIyFNyH9rfddatqGsgDMyIyGlyvriUwtN01l6jl+hr2Glg2XXPEoi5AhFLAJf9zW74OIjKkJRmDIW13zaFvt/HQBzEikhGRPr+t7a4bQFX7RWQK10w5C5wN7W7Law6p5foa9hpYzcg0jIiMAPOqOhZ1WZrFN0tU+hbYrip94LxKC3TUb4V/T6dxSQtxXC2p1Ws8O5IFoybyH1y7gv8HTanqYJl97fQ6nNMaxk+02TXnSu4B8E20sfCHcztdt880m1bVWf83P417DWZKjmubay6nlutrxGtgwag5gn/aorTf0B8sR/vJUNyEAW32OviO7GsbHNZW1wxF/QHV+gXa6rqDcmtoQK+q5v2XrZxPYGmray6jlutr2GtgfUZNoKp5Ecmxvnmjj7XO/bbhv0GmSzsx2/B1GADOiMj50LYEEBeRDHBVVSfb7JoDhZH15fa16d96rsK+LD6Bpc2uuUit19eo18BqRs0zjXtDhyX89rbh29QzWjwlTDLUdNM2r4OqjqtqOnwDpoBZ/3uQ1NA21xySZW1aHKBQUwx/4LTTdedwU1yVE8MlM0B7XXM5tVxfY16DqAdbtesN94YtnSJjihaaJqSGaxzCjT9JhG5JiqeIaevXAdc8WTrote2u2V/TXJlrGmrX68YF4NLpgOKEBv+2yzXjpgMaKrN9w+tr1Gtg6xk1UWhSyau4N3HQEdryfJvwQoXdOVU9Ezq27V4HX/NLA+dw/4zjuCA86/e38zXP4WoNE6XX1G7X7Wv+Z3ATw4KrCZabKLXlrtn/D1/ElXkIVxucxgWWydBxG15fI14DC0bGGGMiZ31GxhhjImfByBhjTOQsGBljjImcBSNjjDGRs2BkjDEmchaMjDHGRM6CkTHGmMhZMDLGGBM5C0am4fzcdOpvI2X2xyvta3K5MiKy40d5i0hWRBY2eo386qNT21m2erVCGcP8e3PKv/bZqMuzm9is3abZ0ripckwNRGQCN8ffBb+p2qzHn8HNBbiTtUIZC9RN+jrov7S0TBBtB1YzMs00jlteYSjqgrSQIdxyHJP+NlvpQFUdVlUJb/O1v21fibTS85Yr407n51mD9pl5uyVYMDLNNIH7Zn8x6oK0gtCH+aaWNfcTX45SstRDs0X1vE2UxK/RFHVBdhMLRqbZMkAi9G3TmJ1uEKsVbTsLRqap/HT7eVxQqqhccoGIjPiO5FjJcQsikhCRGb9/JljQL9T5PFepedA/dsqfZ84vIV16zFDp+UvL4H/O+nOUrnRZ7XmDc46E9mVYW1k0OKZq82b4NfN9TcGSHhP+8Qslx9d9Tf41nQglVEyFm+M2et4Kf9eKr0NpecocW7rA35Avq5Yr3yYl8f1F/jUY9beq72GzNRaMzHa4BCQb8CERiOGaAC8BKdz6KROh27A/bqLC4y/jFk5L+99LP2CH/GOn/blypcf442ZwK1xmN2rS8eec8ecaBF4FsqGMrUt+O75cZ8JrytTgQujxKdwaPP0NuKZUyfn7/HXU9LylangdwoK/c5biv3NwruD3Sf+cwXVt+MWgSvkK/UX+C8Y5VR3z12V9n80U9QqDdmu/G+6bpeJXycR9OCh+BVjch4oCI6HHZNzbseg8I/64WOlxhFbgDG0bDW0b8tsSGzxHPFw2v22B0GqefttMsC30fFOVXoMyr8lC+DkqvE5BWdatuFnhnEXXU+3xjbomXL9Q0XNs8LylZdzwdSgpz1CZbbGSv3GsWpnrfO+O4mqosZL3Z6yRz2O39TerGZmmU/cNexwYqaU5q0bXQj8HzVvhdv4gJbpvg7IFq1sOQOGbcQwYDTX9KGvLqoelqIFvDovhvuGHn3sa14Q5XO5xjdLIa9K17L66a7mbfB3Cf9O5Cvte881ojeiXDGp55zS0oquq5tUSGprKgpHZLkF7e0MGulb4YNjsh0WOtQ/X4L4f1zQTvhV9WPpAVovgnOWODz93s2zpmny/zITvs6m01Hw95aj5dagWAPy+IHhkgBnZ+gDbJL7Zr1wflWkeG/RqtoWq5kRkEpfmvdMyleKsfUAWPijrCDYbCc4TB0rHDcUpruU1w6avyX+4x3Ef9kE5Zyo/oqZyNOx18LWqfl/jHgEyIjKqrp+nLqHAc0lV8yIyiuuTOlPvuUz9rGZkttMlXDPN+RqPr9gR3ij+QyyJD5C+Gaps9t8WmhiDD9miJjDfmR900jfCfLmNm70mnyCQxA3CHffnKRfMyj5vGU17HXwz2hguyH1mk6cpHV+Uwzfz+qw6S2BoIqsZmW2jqrMiMovrJC41B24uM1xgSNKgJr1SPh05zdo3/jwuUAYu4NKUg0yuGO4DNEeN/URh/lt2Cpc11ofLIAuee9J/u98y/zwAqeBeVYNmrLqvyddm87jaBrjXKV3muGrPW3pcw14HnxKe9tcTNPMl/O9BMM3gajoVZ7IIKTe+KAi+RX1IpvGsZmS227oPM+8K7lttBvcN+Sib+OCv0au4cSRTuG/1/eG+CXUp1f24D+wpXCp4rkrZN+Q/yAZZS09O4WocjU5eGMMF8gyhprAtXNMFXO1ggrW/Tc3PW6rBr8MVXP9OKnSusVDQiOEy7gZqPF8foeQK/5rl/BekWmt/ZpNEdcdPYmyMMZsiInPAcI01IxMhqxkZY9qaBaLWYMHIGNOWfHLGpptWzfayZjpjjDGRs5qRMcaYyFkwMsYYEzkLRsYYYyJnwcgYY0zkLBgZY4yJnAUjY4wxkbNgZIwxJnL/PxfjXEAzzaD5AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "niters = 64\n",
    "roots = [np.cos((np.pi * (2 * i + 1)) / (2 * niters)) for i in range(niters)]\n",
    "taus = [(lam_max + lam_min - (lam_min - lam_max) * r) / 2 for r in roots]\n",
    "x = np.zeros(n)\n",
    "r = A.dot(x) - rhs\n",
    "res_cheb = [np.linalg.norm(r)]\n",
    "\n",
    "# Implementation may be non-optimal if number of iterations is not power of two\n",
    "def good_shuffle(idx):\n",
    "    if len(idx) == 1:\n",
    "        return idx\n",
    "    else:\n",
    "        new_len = int(np.ceil((len(idx) / 2)))\n",
    "        new_idx = good_shuffle(idx[:new_len])\n",
    "        res_perm = []\n",
    "        perm_count = 0\n",
    "        for i in new_idx:\n",
    "            res_perm.append(i)\n",
    "            perm_count += 1\n",
    "            if perm_count == len(idx):\n",
    "                break\n",
    "            res_perm.append(len(idx) + 1 - i)\n",
    "            perm_count += 1\n",
    "            if perm_count == len(idx):\n",
    "                break\n",
    "        return res_perm\n",
    "\n",
    "good_perm = [i for i in range(1, niters+1)]\n",
    "# good_perm = good_shuffle([i for i in range(1, niters+1)])\n",
    "# good_perm = [i for i in range(niters, 0, -1)]\n",
    "# good_perm = np.random.permutation([i for i in range(1, niters+1)])\n",
    "\n",
    "for i in range(niters):\n",
    "    x = x - 1.0/taus[good_perm[i] - 1] * r\n",
    "    r = A.dot(x) - rhs\n",
    "    res_cheb.append(np.linalg.norm(r))\n",
    "    \n",
    "plt.semilogy(res_richardson, label=\"Richardson\")\n",
    "plt.semilogy(res_cheb, label=\"Chebyshev\")\n",
    "plt.legend(fontsize=20)\n",
    "plt.xlabel(\"Number of iterations, $k$\", fontsize=20)\n",
    "plt.ylabel(\"Residual norm, $\\|Ax_k - b\\|_2$\", fontsize=20)\n",
    "plt.xticks(fontsize=20)\n",
    "_ = plt.yticks(fontsize=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Что сломалось в прекрасном методе Чебышёва?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Перестановки корней полинома Чебышёва очень сильно влияют на сходимость метода (почему?)\n",
    "- Об оптимальной перестановке можно прочитать в [этой статье](http://www.mathnet.ru/links/42ab1a270ad179d43359ef8fab53422c/zvmmf6870.pdf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Проект [Chebfun](http://www.chebfun.org/) \n",
    "\n",
    "- Проект с открытым исходным кодом для численных расчётов (с интерфейсом для [Python](https://github.com/chebpy/chebpy) и [Matlab](https://github.com/chebfun/chebfun))\n",
    "- Основан на алгоритмах, работающих с кусочно-полиномиальными интерполяциями и Чебышёвскими полиномами\n",
    "- Авторами проекта являются Nick Trefethen и его студент Zachary Battles, подробности см. в [этой статье](http://www.chebfun.org/publications/chebfun_paper.pdf)\n",
    "- Chebfun нацелен на решение следующих задач\n",
    "    - Аппроксимация\n",
    "    - Квадратуры\n",
    "    - ОДУ\n",
    "    - Уравнения с частными производными\n",
    "    - Поиск корней\n",
    "    - Одномерная глобальная оптимизация"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Улучшения метода Чебышёва\n",
    "\n",
    "- Мы сделали важное предположение о спектре: он содержится на отрезке (и нам нужны границы)\n",
    "\n",
    "- Если спектр содержится в **двух отрезках** и мы знаем границы, также можно поставить задачу оптимизации для поиска оптимального полинома"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Если спектр матрицы лежит в нескольких отрезках\n",
    "\n",
    "- Для случая двух отрезков оптимальными полиномами являются **[полиномы Золотарёва](https://www.encyclopediaofmath.org/index.php/Zolotarev_polynomials)** (выражаются через эллиптические функции). \n",
    "- Оригинальная работа опубликована в 1877, подробности см. [тут](http://gpntb.dlibrary.org/ru/nodes/4245-zolotarev-e-prilozhenie-ellipticheskih-funktsiy-k-voprosam-o-funktsiyah-naimenee-i-naibolee-otklonyayuschihsya-ot-nulya-spb-1877) \n",
    "- Для случая, когда интервалов больше двух, оптимальный полином может быть записан с помощью гиперэллиптических функций"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Как ещё можно ускорить метод Чебышёва?\n",
    "\n",
    "- Реализация метода Чебышёва требует знания границ спектра.\n",
    "\n",
    "- Метод хранит вектор $x_k$ и вычисляет новую поправку\n",
    "\n",
    "$$r_k = A x_k - f.$$\n",
    "\n",
    "- Оказывается, что если мы будем использовать больше векторов (не только $x_k$ и $r_k$), то мы можем избавиться от необходимости знать спектр и получим более быструю сходимость!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Главная идея: Крыловские пространства\n",
    "\n",
    "- Метод Чебышёва даёт аппроксимацию вида\n",
    "\n",
    "$$x_{k+1} = x_0 + p(A) r_0,$$\n",
    "\n",
    "то есть $x_{k+1}$ лежит в Крыловском пространстве.\n",
    "- Крыловское пространство определяется для матрицы $A$ следующим образом\n",
    "\n",
    "$$ \\mathcal{K}_k(A, r_0) = \\mathrm{Span}(r_0, Ar_0, A^2 r_0, \\ldots, A^{k-1}r_0 ) $$\n",
    "\n",
    "- Наиболее естественный подход теперь – это искать вектор в этом пространстве, который минимизирует норму некоторого вектора"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Идея Крыловских методов\n",
    "\n",
    "Идея заключается в минимизации заданного функционала:\n",
    "- Энергетическая норма вектора ошибки для систем с эрмитовой положительно определённой матрицей (метод сопряжённых градиентов).\n",
    "- Норма вектора остатков для систем с матрицами общего вида (методы minres и GMRES).\n",
    "- Отношение Релея для частичных задач на собстенные значения (метод Ланцоша).\n",
    "\n",
    "Для того чтобы сделать метод практичным нужно следующее\n",
    "1. Ортогонализовать векторы $A^i r_0$ Крыловского подпространства для устойчивости (процесс Ланцоша).\n",
    "2. Получить рекуррентные формулы для уменьшения сложности.\n",
    "\n",
    "Эти методы рассмотрим далее."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Выводы по первой части\n",
    "\n",
    "- Основная идея итерационных методов\n",
    "- Метод Ричардсона: эрмитов и неэрмитов случай\n",
    "- Метод Чебышёва\n",
    "- Определение Крыловских подпространств"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## План на часть 2\n",
    "\n",
    "- Процесс Арнольди и Ланцоша для ортогонализации в Крыловском подпространстве, результаты об оптимальности Крыловских подпространств\n",
    "- Главные итерационные метода: метод сопряжённых градиентов и GMRES\n",
    "- Оценки сходимости"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Решение линейных систем и минимизация функционалов\n",
    "\n",
    "- Вместо решения линейной системы, мы можем минимизировать норму вектора остатков:\n",
    "\n",
    "$$R(x) = \\Vert A x - f \\Vert^2_2.$$\n",
    "\n",
    "- Условие оптимальности первого порядка $\\nabla R(x) = 0$ даёт\n",
    "\n",
    "$$A^* A x = A^* f,$$\n",
    "\n",
    "таким образом число обусловленности возвелось в квадрат, поэтому явная минимизация нормы остатков с помощью стандартных методов оптимизации используется редко.\n",
    "\n",
    "- Для случая симметричной положительно определённой матрицы есть более простой функционал."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Функционал энергии\n",
    "\n",
    "Пусть $A = A^* > 0$, тогда следующий функционал\n",
    "\n",
    "$$\\Phi(x) = (Ax, x)  - 2(f, x)$$\n",
    "\n",
    "называется **функционалом энергии**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Свойства функционала энергии\n",
    "\n",
    "- Строго выпуклый (проверьте!)\n",
    "\n",
    "$$ \\Phi(\\alpha x + (1 - \\alpha)y) < \\alpha \\Phi(x) + (1 - \\alpha) \\Phi(y)$$\n",
    "\n",
    "- Так как он строго выпуклый, то у него есть единственный локальный минимум, который также является глобальным\n",
    "\n",
    "- Глобальный минимум $x_*$ удовлетворяет равенству\n",
    "\n",
    "$$A x_* = f.$$\n",
    "\n",
    "- В самом деле, \n",
    "\n",
    "$$\\nabla \\Phi = 2(Ax - f).$$ \n",
    "\n",
    "и условие оптимальности первого порядка $\\nabla \\Phi (x_*) = 0$ даёт\n",
    "\n",
    "$$A x_* = f.$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Аппроксимация решения на подпространстве\n",
    "\n",
    "- Пусть дано линейное $M$-мерное подпространство с базисными векторами $\\{y_1, \\dots, y_M\\}$, мы хотим найти приближённое решение в этом базисе, то есть \n",
    "\n",
    "$$A x \\approx f, \\quad x = x_0 +  \\sum_{k=1}^M c_k y_k,$$\n",
    "\n",
    "где $c$ – вектор коэффициентов.\n",
    "\n",
    "- В случае симметричной положительно определённой матрицы нам нужно минимизировать\n",
    "\n",
    "$$(Ax, x) - 2(f, x)$$ \n",
    "\n",
    "при условии \n",
    "\n",
    "$$x = x_0 + Y c,$$\n",
    "\n",
    "где $Y=[y_1,\\dots,y_M]$ матрица размера $n \\times M$ и вектор $c$ имеет размерность $M$.\n",
    "\n",
    "- Используя представление для $x$, получаем следующую задачу минимизации на вектор $c$:\n",
    "\n",
    "$$\\widehat{\\Phi}(c) = (A Y c, Y c) + 2 (Y^*Ax_0, c) - 2(f, Y c) = (Y^* A Y c, c) - 2(Y^* (f - Ax_0), c).$$\n",
    "\n",
    "- Отметим, что это тот же самый функционал энергии, но для **Галёркинской проекции** матрицы $A$:\n",
    "\n",
    "$$Y^* A Y c = Y^* (f - Ax_0) = Y^* r_0,$$\n",
    "\n",
    "что является линейной системой размера $M \\times M$ с симметричной положительно определённой матрицей, если $Y$ имеет полный ранг.\n",
    "\n",
    "- Но как выбрать $Y$?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Выбор подпространства\n",
    "\n",
    "- В Крыловском подпространстве мы генерируем всё подпространство из единственного вектора $r_0 = f - Ax_0$:\n",
    "\n",
    "$$y_0\\equiv k_0 = r_0, \\quad y_1\\equiv k_1 = A r_0, \\quad y_2\\equiv k_2 = A^2 r_0, \\ldots, \\quad y_{M-1}\\equiv k_{M-1} = A^{M-1} r_0.$$\n",
    "\n",
    "- Это даёт **Крыловское подпространство** порядка $M$\n",
    "\n",
    "$$\\mathcal{K}_M(A, r_0) = \\mathrm{Span}(r_0, Ar_0, \\ldots, A^{M-1} r_0).$$\n",
    "\n",
    "- Оригинальная работа, в которой вводятся такие подпространства \"О численном решении уравнения, которым в технических вопросах определяются частоты малых колебаний материальных систем\",  А. Н. Крылов, 1931, доступна [тут](http://www.mathnet.ru/links/a2431bc65c0764da9a1ae95c73f741f6/im5215.pdf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Решение $x_*$ лежит в Крыловском подпространстве $x_* \\in \\mathcal{K}_n(A, f)$\n",
    "\n",
    "- По теореме [Гамильтона-Кэли](https://ru.wikipedia.org/wiki/%D0%A2%D0%B5%D0%BE%D1%80%D0%B5%D0%BC%D0%B0_%D0%93%D0%B0%D0%BC%D0%B8%D0%BB%D1%8C%D1%82%D0%BE%D0%BD%D0%B0_%E2%80%94_%D0%9A%D1%8D%D0%BB%D0%B8): $p(A) = 0$, где $p(\\lambda) = \\det(A - \\lambda I)$\n",
    "- $p(A)f = A^nf + a_1A^{n-1}f + \\ldots + a_{n-1}Af + a_n f = 0$\n",
    "- $A^{-1}p(A)f = A^{n-1}f + a_1A^{n-2}f + \\ldots + a_{n-1}f + a_nA^{-1}f = 0$\n",
    "- $x_* = A^{-1}f = -\\frac{1}{a_n}(A^{n-1}f + a_1A^{n-2}f + \\ldots + a_{n-1}f)$\n",
    "- Таким образом, $x_* \\in \\mathcal{K}_n(A, f)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Плохая обусловленность естественного базиса\n",
    "\n",
    "Естественный базис в Крыловском подпространстве очень **плохо обусловлен**, поскольку \n",
    "\n",
    "$$k_i = A^i r_0 \\rightarrow \\lambda_\\max^i v,$$\n",
    "\n",
    "где $v$ собственный вектор, соответствующий максимальному собственному значению $A$, то есть $k_i$ становятся всё более и более коллинеарными для больших $i$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Condition number = 660592080.6626757\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import scipy.sparse as spsp\n",
    "%matplotlib inline\n",
    "\n",
    "n = 100\n",
    "ex = np.ones(n);\n",
    "A = spsp.spdiags(np.vstack((-ex,  2*ex, -ex)), [-1, 0, 1], n, n, 'csr'); \n",
    "f = np.ones(n)\n",
    "x0 = np.random.randn(n)\n",
    "\n",
    "subspace_order = 10\n",
    "krylov_vectors = np.zeros((n, subspace_order))\n",
    "krylov_vectors[:, 0] = f - A.dot(x0)\n",
    "for i in range(1, subspace_order):\n",
    "    krylov_vectors[:, i] = A.dot(krylov_vectors[:, i-1])\n",
    "    \n",
    "s = np.linalg.svd(krylov_vectors, compute_uv=False)\n",
    "print(\"Condition number = {}\".format(s.max() / s.min()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Решение: получить ортогональный базис в Крыловском подпространстве"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Хороший базис в Крыловском подпространстве\n",
    "\n",
    "- Для повышения устойчивости мы сначала ортогонализуем векторы из Крыловского подпространства, используя процесс Грама-Шмидта (или QR разложение)\n",
    "\n",
    "$$K_j = \\begin{bmatrix} r_0 & Ar_0 & A^2 r_0 & \\ldots & A^{j-1} r_0\\end{bmatrix} = Q_j R_j. $$\n",
    "\n",
    "- После чего решение приближается как $$x \\approx x_0 + Q_j c.$$ \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Короткий путь к соотношению Арнольди\n",
    "\n",
    "**Утверждение.** Матрица Крыловских векторов $K_j$ удовлетворяет важному рекуррентному соотношению (которое называется **соотношение Арнольди**)\n",
    "\n",
    "$$A Q_j = Q_j H_j + h_{j, j-1} q_j e^{\\top}_{j-1},$$\n",
    "\n",
    "где $H_j$ – верхне-гессенбергова матрица, а  $Q_{j+1} = [q_0,\\dots,q_j]$ имеет ортонормированные столбцы, которые образуют пространство $K_{j+1}$.\n",
    "\n",
    "- Давайте докажем его (для простоты рассмотрим $j = 3$):\n",
    "\n",
    "$$A \\begin{bmatrix} k_0 & k_1 & k_2 \\end{bmatrix} = \\begin{bmatrix} k_1 & k_2 & k_3 \\end{bmatrix} = \\begin{bmatrix} k_0 & k_1 & k_2 \\end{bmatrix} \\begin{bmatrix} 0 & 0 & \\alpha_0 \\\\ 1 & 0  & \\alpha_1 \\\\ 0 & 1  & \\alpha_2 \\\\ \\end{bmatrix} + \\begin{bmatrix} 0 & 0 & k_3  - \\alpha_0 k_0 - \\alpha_1 k_1 - \\alpha_2 k_2 \\end{bmatrix}, $$\n",
    " \n",
    "где $\\alpha_s$ будут выбраны позднее. \n",
    "\n",
    "- Обозначим $\\widehat{k}_3 = k_3  - \\alpha_0 k_0 - \\alpha_1 k_1 - \\alpha_2 k_2$.\n",
    " \n",
    "- В матричной форме,\n",
    " \n",
    " $$A K_3 = K_3 Z + \\widehat k_3 e^{\\top}_2,$$\n",
    " \n",
    " где $e_2$ – последний столбец единичной матрицы.\n",
    " \n",
    "- Пусть QR разложение матрицы $K_3$ имеет вид\n",
    "\n",
    "$$K_3 = Q_3 R_3.$$\n",
    "\n",
    "Тогда\n",
    " \n",
    " $$A Q_3 R_3 = Q_3 R_3 Z + \\widehat{k}_3 e^{\\top}_2,$$\n",
    " \n",
    " $$ A Q_3 = Q_3 R_3 Z R_3^{-1} + \\widehat{k}_3 e^{\\top}_2 R_3^{-1}.$$\n",
    "\n",
    "\n",
    "- Заметим, что\n",
    "\n",
    "$$e^{\\top}_2 R_3^{-1} = \\begin{bmatrix} 0 & 0 & 1 \\end{bmatrix} \\begin{bmatrix} * & * & * \\\\ 0 & * & * \\\\ 0 & 0 & * \\end{bmatrix}  = \\gamma e^{\\top}_2,$$\n",
    " \n",
    " и\n",
    " \n",
    " $$R_3 Z R_3^{-1} = \\begin{bmatrix} * & * & * \\\\* & * & * \\\\  0 & * & * \\\\ \\end{bmatrix},$$      \n",
    "\n",
    "в общем случае она будет искомой матрицей $H$ и будет иметь верхне-гессенбергову форму, то есть матрица имеет вид\n",
    "\n",
    "$$H_{ij} = 0, \\quad \\mbox{если } i > j + 1.$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## (Почти) соотношение Арнольди\n",
    "\n",
    "- Пусть $Q_j$ ортогональный базис в Крыловском подпространстве, тогда мы получили почти соотношение Арнольди\n",
    "\n",
    "$$A Q_j = Q_j H_j +  \\gamma\\widehat{k}_j e^{\\top}_{j-1},$$\n",
    "\n",
    "где $H_j$ верхне-гессенбергова форма и \n",
    "\n",
    "$$\\widehat{k}_j = k_j - \\sum_{s=0}^{j-1} \\alpha_s k_s.$$\n",
    "\n",
    "- Мы выберем $\\alpha_s$ таким образом, чтобы\n",
    "\n",
    "$$Q^*_j \\widehat{k}_j = 0.$$\n",
    "\n",
    "- Тогда $\\widehat{k}_j = h_{j, j-1} q_j,$ где $q_j$ последний столбец в матрице $Q_{j+1}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Соотношение Арнольди: итоговое выражение\n",
    "\n",
    "Получили рекуррентное соотношение\n",
    "\n",
    "$$A Q_j = Q_j H_j + h_{j, j-1} q_j e^{\\top}_{j-1}.$$\n",
    "\n",
    "- Это ключевая формула для эффективной генерации векторов из Крыловских подпространств.\n",
    "\n",
    "- Для несимметричного случая это выражение совпадает с модифицированным методом Грама-Шмидта.\n",
    "\n",
    "- Для симметричного случая у нас будет более простая форма (процесс Ланцоша)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Процесс Ланцоша\n",
    "\n",
    "- Если $A = A^*$, тогда\n",
    "\n",
    "$$Q^*_j A Q_j = H_j, $$\n",
    "\n",
    "таким образом $H_j$ эрмитова, и следовательно трёхдиагональна, $H_j = T_j$.\n",
    "\n",
    "- Это даёт нам короткое рекуррентное выражение для генерации векторов Арнольди $q_j$ без **полной** ортогонализации."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Процесс Ланцоша (2)\n",
    "\n",
    "$$ A Q_j = Q_j T_j + t_{j, j-1} q_j e^{\\top}_{j-1}.$$\n",
    "\n",
    "\n",
    "- Для получения вектора $q_j$ нам нужно вычислить только последний столбец\n",
    "\n",
    "$$t_{j, j-1} q_j = (A Q_j - Q_j T_j) e_{j-1} = A q_{j-1} - t_{j-1, j-1} q_{j-1} - t_{j-2, j-1} q_{j-2}. $$\n",
    "\n",
    "- Коэффициенты $\\alpha_j = t_{j-1, j-1}$ и $\\beta_j = t_{j-2, j-1}$ могут быть восстановлены из условий ортогональности\n",
    "\n",
    "$$(q_j, q_{j-1}) = 0, \\quad (q_j, q_{j-2}) = 0$$\n",
    "\n",
    "- **Все остальные ограничения на ортогональность будут выполнены автоматически!!**\n",
    "\n",
    "- И нам нужно хранить только два вектора, чтобы получить новый вектор."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## От явного процесса Ланцоша к методу сопряжённых градиентов\n",
    "\n",
    "- Теперь мы можем получить из рекуррентных формул метода Ланцоша знаменитый **метод сопряжённых градиентов**.\n",
    "\n",
    "- Для матрицы $A = A^* > 0$\n",
    "\n",
    "$$A Q_j = Q_j T_j + T_{j, j-1} q_j.$$\n",
    "\n",
    "- Вспомним, что при минимизации функционала энергии в базисе $Y$ мы получили линейную систему \n",
    "\n",
    "$$Y^* A Y c = Y^* f.$$\n",
    "\n",
    "- Здесь $Y = Q_j$, поэтому приближённое решение $Ax \\approx f$ при условии $x_j = x_0 + Q_j c_j$ может быть получено из решения линейной системы меньшей размерности\n",
    "\n",
    "$$Q^*_j A Q_j c_j = T_j c_j = Q^*_j r_0 .$$\n",
    "\n",
    "- Так как $r_0$ образует Крыловское подпрострасво первого порядка, то\n",
    "\n",
    "$$Q^*_j r_0  = \\Vert r_0 \\Vert_2^2 e_0 = \\gamma e_0.$$\n",
    "\n",
    "- У нас есть трёхдиагональная система уравнений на $c$:\n",
    "\n",
    "$$T_j c_j = \\gamma e_0$$\n",
    "\n",
    "и $x_j = Q_j c_j$.\n",
    "\n",
    "- Мы можем остановится на этом месте, но вместо решения трёхдиагональной системы мы можем получить короткие рекуррентные формулы  для каждой итерации."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- Так как $A$ положительно определена, $T_j$ также положительно определена и имеет LU разложение\n",
    "\n",
    "- $T_j = L_j U_j$, где $L_j$ бидиагональная матрица с единицами на диагонали, $U_j$ верхне-бидиагональная матрица.\n",
    "\n",
    "- Нам нужно определить одну поддиагональ в $L$ (с элементами $c_1, \\ldots, c_{j-1}$), главную диагональ $U_j$ (с элементами $d_0, \\ldots, d_{j-1}$ и наддиагональ в $U_j$ (с элементами $b_1, \\ldots, b_{j-1}$).\n",
    "\n",
    "- Для них выполнены удобные рекуррентные соотношения:\n",
    "\n",
    "$$c_i = b_i/d_{i-1}, \\quad d_i = \\begin{cases} a_1, & \\mbox{если } i = 0, \\\\\n",
    "a_i - c_i b_i, & \\mbox{если } i > 0. \\end{cases}$$\n",
    "\n",
    "- Для решения выполнено\n",
    "\n",
    "$$x_j = Q_j T^{-1}_j \\gamma e_0  = \\gamma Q_j (L_j U_j)^{-1} e_0  = \\gamma Q_j U^{-1}_j L^{-1}_j e_0.$$\n",
    "\n",
    "- Введём две новых величины:  \n",
    "\n",
    "$$P_j = Q_j U^{-1}_j, \\quad z_j = \\gamma L^{-1}_j e_0.$$\n",
    "\n",
    "- Теперь у нас есть следующее выражение для $x_j$:\n",
    "\n",
    "$$ x_j = P_j z_j$$\n",
    "\n",
    "- Из-за рекуррентных соотношений у нас есть\n",
    "\n",
    "$$P_j = \\begin{bmatrix} P_{j-1} & p_j \\end{bmatrix}, $$\n",
    "\n",
    "и \n",
    "\n",
    "$$z_j = \\begin{bmatrix} z_{j-1} \\\\ \\xi_{j} \\end{bmatrix}.$$\n",
    "\n",
    "- Для $p_j$ и $\\xi_j$ у нас есть короткие рекуррентные соотношения (из-за бидиагональной структуры)\n",
    "\n",
    "$$p_j = \\frac{1}{d_j}\\left(q_j - b_j p_{j-1} \\right), \\quad \\xi_j = -c_j \\xi_{j-1}.$$\n",
    "\n",
    "- Таким образом, мы пришли к коротким рекуррентным соотношениям для $x_j$:\n",
    "\n",
    "$$x_j = P_j z_j = P_{j-1} z_{j-1} + \\xi_j p_j = x_{j-1} + \\xi_j p_j.$$\n",
    "\n",
    "и $q_j$ ищутся из соотношения Ланцоша (см слайды ранее).\n",
    "\n",
    "- Этот метод для решения линейных систем называется **прямой метод Ланцоша**. Он тесно связан с методом сопряжённых градиентов."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Прямой метод Ланцоша\n",
    "\n",
    "- У нас есть прямой метод Ланцоша, в котором мы храним\n",
    "\n",
    "$$p_{j-1}, q_j, x_{j-1}$$ \n",
    "\n",
    "для получения новой оценки $x_j$.\n",
    "\n",
    "- Основная проблема с $q_j$: у нас есть трёхчленное рекуррентное соотношение, но в неточной арифметике ортогональность может теряться, что ведёт к ошибкам при вычислении.\n",
    "\n",
    "- Далее короткая демонстрация такого поведения"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.6432256263275\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "import scipy as sp\n",
    "import scipy.sparse as spsp\n",
    "from scipy.sparse import csc_matrix\n",
    "\n",
    "n = 128\n",
    "ex = np.ones(n);\n",
    "A = spsp.spdiags(np.vstack((ex,  -2*ex, ex)), [-1, 0, 1], n, n, 'csr'); \n",
    "rhs = np.ones(n)\n",
    "\n",
    "nit = 64\n",
    "q1 = rhs/np.linalg.norm(rhs)\n",
    "q2 = A.dot(q1)\n",
    "q2 = q2 - np.dot(q2, q1)*q1\n",
    "q2 = q2/np.linalg.norm(q2)\n",
    "qall = [q1, q2]\n",
    "for i in range(nit):\n",
    "    qnew = A.dot(qall[-1])\n",
    "    qnew = qnew - np.dot(qnew, qall[-1])*qall[-1]\n",
    "    qnew = qnew/np.linalg.norm(qnew)\n",
    "    qnew = qnew - np.dot(qnew, qall[-2])*qall[-2]\n",
    "    qnew = qnew/np.linalg.norm(qnew)\n",
    "    qall.append(qnew)\n",
    "qall_mat = np.vstack(qall).T\n",
    "print(np.linalg.norm(qall_mat.T.dot(qall_mat) - np.eye(qall_mat.shape[1])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Метод сопряжённых градиентов (CG)\n",
    "\n",
    "- Вместо использования вектора $q_j$ (последнего вектора из модифицирвоанного метода Грама-Шмидта), удобнее работать с вектором остатков\n",
    "\n",
    "$$r_j = f - A x_j.$$\n",
    "\n",
    "- Итоговые рекуррентные формулы имеют вид\n",
    "\n",
    "$$x_j = x_{j-1} + \\alpha_{j-1} p_{j-1}$$\n",
    "\n",
    "$$r_j = r_{j-1} - \\alpha_{j-1}  A p_{j-1}$$\n",
    "\n",
    "$$p_j = r_j + \\beta_j p_{j-1}.$$\n",
    "\n",
    "- Отсюда следует название сопряжённых градиентов: к градиентам $r_j$ мы прибавляем **сопряжённые направления** $p_j$.\n",
    "\n",
    "- Выполнена ортогональность остатков (проверьте!):\n",
    "\n",
    "$$(r_i, r_j) = 0, \\quad i \\ne j$$\n",
    "\n",
    "и **A-ортогональность** сопряжённых направлений:\n",
    "\n",
    "$$ (A p_i, p_j) = 0,$$\n",
    "\n",
    "которая может быть проверена по определению (проверьте!).\n",
    "\n",
    "- Уравнения на $\\alpha_j$ и $\\beta_j$ могут быть теперь определены из этих двух свойств."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Итоговые формулы для метода сопряжённых градиентов\n",
    "\n",
    "- Мы знаем, что  $(r_{j}, r_{j-1}) = 0 = (r_{j-1} - \\alpha_{j-1} A r_{j-1}, r_{j-1})$, \n",
    "\n",
    "тогда\n",
    "\n",
    "$$\\alpha_{j-1} = \\frac{(r_{j-1}, r_{j-1})}{(A r_{j-1}, r_{j-1})}.$$\n",
    "\n",
    "- Таким же способом, получим\n",
    "\n",
    "$$\\beta_{j-1} = \\frac{(r_j, r_j)}{(r_{j-1}, r_{j-1})}.$$\n",
    "\n",
    "- Вспомним, что\n",
    "\n",
    "$x_j = x_{j-1} + \\alpha_{j-1} p_{j-1}$\n",
    "\n",
    "$r_j = r_{j-1} - \\alpha_{j-1}  A p_{j-1}$\n",
    "\n",
    "$p_j = r_j + \\beta_j p_{j-1}$.\n",
    "\n",
    "- Требуется только одно умножение матрицы на вектор.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Как мы получили метод сопряжённых градиентов: обзор\n",
    "\n",
    "- Хотим найти вектор $x_*$ в Крыловском подпространстве\n",
    "- Но естественный базис плохо обусловлен, поэтому нам нужна ортогонализация\n",
    "- Вывести рекуррентные соотношения для последовательной ортогонализации векторов из базиса в Крыловско подпространстве\n",
    "    - процесс Арнольди для несимметричных матриц\n",
    "    - процесс Ланцоша для симметричных матриц\n",
    "- Правильное переписывание этих формул даёт короткие рекуррентные соотношения"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Немного истории\n",
    "\n",
    "- Более подробно история изложена [тут](https://www.siam.org/meetings/la09/talks/oleary.pdf)\n",
    "\n",
    "- Когда Хестенс работал над методом в 1936, Гарвадский профессор сказал ему, что это слишком простой метод для публикации\n",
    "- Метод сопряжённых градиентов не работает на логарифмической линейке\n",
    "<img src=\"slide_rule.jpg\">\n",
    "- CG имеет мало достоинств по сравнению с методом Гаусса при вычислении на калькуляторе.\n",
    "- CG плохо приспособлен для \"human computers\" – слишком много обменов данными.\n",
    "<img src=\"human_computers.jpg\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Свойства метода сопряжённых градиентов\n",
    "\n",
    "- Нужно хранить 3 вектора.\n",
    "\n",
    "- Поскольку он генерирует $A$-ортогональную последовательность $p_1, \\ldots, p_N$, после $n$ шагов он должен остановиться\n",
    "\n",
    "- На практике это свойство не выполняется в неточной арифметике, поэтому после того, как в 1952 году Хестенс и Штифель изобрели этот метод, он был назван **неустойчивым**.\n",
    "\n",
    "- На самом деле это замечательный итерационный метод!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## $A$-оптимальность\n",
    "\n",
    "\n",
    "- Функционал энергии может быть записан в виде\n",
    "\n",
    "$$(Ax, x) - 2(f, x) = (A (x - x_*), (x - x_*)) - (Ax _*, x_*),$$\n",
    "\n",
    "где $A x_* = f$. \n",
    "\n",
    "- С точностью до постоянного множителя\n",
    "\n",
    "$$ (A(x - x_*), (x -x_*)) = \\Vert x - x_* \\Vert^2_A,$$\n",
    "\n",
    "что является **A-нормой** вектора ошибки.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Сходимость\n",
    "\n",
    "- CG вычисляет $x_k$, который минимизирует функционал энергии на Крыловском подпространстве, то есть $x_k = p(A)f$, где $p$ – полином степени $k+1$, поэтому\n",
    "\n",
    "$$\\Vert x_k - x_* \\Vert_A  =  \\inf\\limits_{p} \\Vert \\left(p(A) - A^{-1}\\right) f \\Vert_A. $$\n",
    "\n",
    "- Используя спектральное разложение матрицы $A$ получим\n",
    "\n",
    "$$A = U \\Lambda U^*, \\quad  g = U^* f,$$ \n",
    "\n",
    "и\n",
    "\n",
    "$$\\Vert x - x_* \\Vert^2_A = \\displaystyle{\\inf_p} \\Vert \\left(p(\\Lambda) - \\Lambda^{-1}\\right) g \\Vert_\\Lambda^2 = \\displaystyle{\\inf_p}\n",
    "\\displaystyle{\\sum_{i=1}^n} \\frac{(\\lambda_i p(\\lambda_i) - 1)^2 g^2_i}{\\lambda_i} = \\displaystyle{\\inf_{q, q(0) = 1}} \\displaystyle{\\sum_{i=1}^n} \\frac{q(\\lambda_i)^2 g^2_i}{\\lambda_i}\n",
    "$$\n",
    "\n",
    "- Выбор оптимального $q$ зависит от распределения собственных значений.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Абсолютная и относительная ошибка\n",
    "\n",
    "- Имеет место следующее выражение\n",
    "\n",
    "$$\\Vert x - x_* \\Vert^2_A \\leq \\sum_{i=1}^n \\frac{g^2_i}{\\lambda_i} \\inf_{q, q(0)=1} \\max_{j} q({\\lambda_j})^2$$\n",
    "\n",
    "- Первый множитель\n",
    "\n",
    "$$\\sum_{i=1}^n \\frac{g^2_i}{\\lambda_i} = (A^{-1} f, f) = \\Vert x_* \\Vert^2_A.$$\n",
    "\n",
    "- И мы получили оценку на относительную ошибку\n",
    "\n",
    "$$\\frac{\\Vert x - x_* \\Vert_A }{\\Vert x_* \\Vert_A} \\leq \\inf_{q, q(0)=1} \\max_{j} |q({\\lambda_j})|.$$\n",
    "\n",
    "- Таким образом, если матрица имеет только 2 различных собственных значения, тогда существует полином степени 2 такой что $q({\\lambda_1}) = q({\\lambda_2}) = 0$, и в этом случае CG сходится за 2 итерации.\n",
    "\n",
    "<font color='red'>\n",
    "\n",
    "- Если собственные значения кластеризованы и имеется $l$ выбросов, тогда после первых $\\mathcal{O}(l)$ итераций CG сойдётся как будто этих выбросов не было (следовательно, эффективное число обусловленности меньше).\n",
    "    \n",
    "- Интуитивное объяснение этого факта в том, что после $\\mathcal{O}(l)$ итераций полином имеет степень больше чем $l$ и таким образом зануляет $l$ выбросов.\n",
    "</font>\n",
    "\n",
    "\n",
    "- Давайте найдём другую полезную оценку сходимости.\n",
    "- Так как\n",
    "\n",
    "$$\n",
    "\\inf_{q, q(0)=1} \\max_{j} |q({\\lambda_j})| \\leq \\inf_{q, q(0)=1} \\max_{\\lambda\\in[\\lambda_\\min,\\lambda_\\max]} |q({\\lambda})|,\n",
    "$$\n",
    "\n",
    "то используя результат, используемый в методе Чебышёва, получим такую же скорость сходимости:\n",
    "\n",
    "$$\\frac{\\Vert x_k - x_* \\Vert_A }{\\Vert x_* \\Vert_A} \\leq \\gamma \\left( \\frac{\\sqrt{\\mathrm{cond}(A)}-1}{\\sqrt{\\mathrm{cond}(A)}+1}\\right)^k.$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Остановка за конечное число шагов & кластеризация спектра\n",
    "\n",
    "1. Если $A$ имеет $m$ различных собственных значений, CG сойдётся за $m$ итераций (покажите!).\n",
    "2. Если $A$ имеет $m$ \"кластеров\" собственных значений, CG будет сходится по очереди учитывая каждый кластер.\n",
    "\n",
    "#### В результате: сходимость лучше, чем у метода Чебышёва, но чуть более высокая стоимость одной итерации"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Резюме про CG\n",
    "\n",
    "CG – это метод, которым нужно решать линейные системы с \n",
    "\n",
    "симметричными положительно определёнными матрицами:\n",
    "\n",
    "1. Требует $\\mathcal{O}(n)$ памяти\n",
    "2. Корень из числа обусловленности в оценке сходимости\n",
    "3. Автоматическое использование выбросов и кластеров собственных значений\n",
    "4. Свойство $A$-оптимальности"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Нелинейный метод сопряжённых градиентов\n",
    "\n",
    "- CG минимизирует функционал энергии, который является квадратичным по $x$\n",
    "- Формулы CG были отправной точкой для разработки методов минимизации произвольной выпуклой функции\n",
    "- Наиболее популярные расширения метода сопряжённых градиентов (также известный как нелинейный метод сопряжённых градиентов): \n",
    "    - [метод Hestenes'a-Stiefel'a](https://www.fing.edu.uy/inco/cursos/numerico/aln/hes_stief1952.pdf)\n",
    "    - [метод Polak'a-Ribiere'a](http://www.numdam.org/article/M2AN_1969__3_1_35_0.pdf) - оригинальная работа на французском :)\n",
    "    - [метод Fletcher'a–Reeves'a](https://watermark.silverchair.com/070149.pdf?token=AQECAHi208BE49Ooan9kkhW_Ercy7Dm3ZL_9Cf3qfKAc485ysgAAAkMwggI_BgkqhkiG9w0BBwagggIwMIICLAIBADCCAiUGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMiWM0crDA8bZ0Fj5HAgEQgIIB9tIM8xbxhkhniJQifurRLHIaOcnos9gvgBj8Lkc34eBUtUg4oYzjusGP_FZiHfpBoRnM409FFCs-Y5xxomumnGRQ8UIZJe7CPSmaCcSklAdwHN3Ja-fHfALctP68U3mGGeR11P8vlESoS7HZhZEmVhFb5m7jPk1ENZKLlq9CnVTz2n2bI3HPYCfDIo8ywd8RWo5gw6EwYnZ46l1ExES8AHMqlklpgzdupLhL6HxTzBEHd_qY4B2FjK4m4MpvdaX5imEnOD6YCwZMNC_qLzUdPxumcXl4oTRDyJp1kg_6PcGi98hE10InOp4aHJeIjl5FPJnuEFEBcv28asOq4vngp6N5d-4y1fpeoxcAe4d857TeLtQnGD6LlCAL_3Mh0lQ8iYLLfjw4b_V2jHEuSxKH3BalLPMS5KF2nGqqrzfbDs8x9jW-OlkeL_x68Zidv0rTHqTd0ww3kvSINDjgw6owPMLK9KxEWP5T-4dnu_kn-NXR1W0s52kUHFD1U9AWs5Zzh4RTVOXsdjGQciWzOO-XieLn8Aju-0P8mVEGjUDOjLDZtuPYM-Ep_Z9lp8TNDCi2lB5IQFbKVpBFaAItDDyVUvKNPfBovmKlnPRFLZYjLbgocnU3tcy0YB8RZegC56Gxn7g_WkphptCVPm7IDeCE9l5gcFafuY0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Несимметричные системы и обобщённый метод минимальных невязок (GMRES) [(Y. Saad, M. Schultz, 1986)](https://epubs.siam.org/doi/10.1137/0907058)\n",
    "\n",
    "- До этого мы обсуждали симметричные положительно определённые системы. Что будет если $A$ несимметричная?\n",
    "\n",
    "- Мы по-прежнему можем ортогонализовать Крыловское подпространство используя процесс Арнольди и получить\n",
    "\n",
    "$$A Q_j = Q_j H_j + h_{j,j-1}q_j e^{\\top}_{j-1}.$$\n",
    "\n",
    "- Перепишем последнее неравенство в виде\n",
    "\n",
    "$$ A Q_j = Q_j H_j + h_{j,j-1}q_j e^{\\top}_{j-1} = Q_{j+1} \\widetilde H_j, \\quad \\widetilde H_j = \n",
    "\\begin{bmatrix} h_{0,0} & h_{0,1} & \\dots & h_{0,j-2} & h_{0,j-1} \\\\ h_{1,0} & h_{1,1} & \\dots & h_{1,j-2} & h_{1,j-1} \\\\ 0& h_{2,2} &  \\dots & h_{2,j-2} & h_{2,j-1} \\\\\n",
    "0& 0 & \\ddots & \\vdots & \\vdots  \\\\\n",
    "0& 0 &  & h_{j,j-1} & h_{j-1,j-1} \\\\ 0& 0 & \\dots & 0 & h_{j,j-1}\\end{bmatrix}$$\n",
    "\n",
    "- Тогда если нам нужно минимизировать норму вектора остатков на Крыловском подпространстве, то\n",
    "\n",
    "$$x_j = x_0 + Q_j c_j $$\n",
    "\n",
    "и $x_j$ должен быть выбран следующим образом \n",
    "\n",
    "$$ \\Vert A x_j - f \\Vert_2 =  \\Vert A Q_j c_j - r_0 \\Vert_2 \\rightarrow \\min_{c_j}.$$\n",
    "\n",
    "- Используя процесс Арнольди, получим, что\n",
    "\n",
    "$$ \\Vert Q_{j+1} \\widetilde H_j c_j -  r_0 \\Vert_2 \\rightarrow \\min_{c_j}.$$\n",
    "\n",
    "- Используя унитарную инвариантность\n",
    "\n",
    "$$ \\Vert \\widetilde H_j c_j - \\gamma e_0 \\Vert_2 \\rightarrow \\min_{c_j},$$\n",
    "\n",
    "где мы используем факт, что $Q^*_{j+1} r_0 = \\gamma e_0.$\n",
    "\n",
    "- Получили линейную задачу наименьших квадратов с $(j+1)$ уравнением и $j$ неизвестными.\n",
    "\n",
    "- Матрица также верхне-гессенбергова, поэтому её QR разложение можно вычислить очень быстро.\n",
    "\n",
    "- Это позволяет вычислить $c_j$. Метод называется **GMRES** (обобщённых минимальных невязок)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Главное про GMRES\n",
    "\n",
    "- Минимизирует норму остатков напрямую\n",
    "- Без использования нормальных уравнений\n",
    "- Требуемая память растёт с числом итераций как $\\mathcal{O}(j^2)$, поэтому обычно используются **рестарты** (запуск GMRES с нового начального приближения)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Figure size 720x360 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsgAAAFgCAYAAACmDI9oAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3Xd4FWX2wPHvm14ICUlogUAIRZqUSwJSBIREdBUboO5a1p+FoKuyskiIvWAJurpWDFjWvkAsKwpqAiJ1aaFLTyihQ0gIJf39/TE3EEICSci9c8v5PM99bjIzd+ZEyeHwzpn3VVprhBBCCCGEEAYPswMQQgghhBDCkUiBLIQQQgghRAVSIAshhBBCCFGBFMhCCCGEEEJUIAWyEEIIIYQQFXiZHYAjU0pZgBggBIgFErXWmeZGJYQQ7kPysBDCDFIgV0MpFQLEaK2nWr+PA9KAtqYGJoQQbkLysBDCLMoZ50FWSo0EcrXW6VXsKx9tyASigcyqjqvBNSzATK11W+v3IcAxoJHWOvdS4hdCCGcneVgI4cqcbgTZOoIwDRhVxb5oIFlrHV9h20ylVGZtb8lprTOUUvEVNsVg/GUgSVkI4dYkDwshXJ3TPKSnlIpWSqVgjEbkVHNYApBSaVsKkFyXa1ZK5gnAA3U5jxBCuALJw0IId+GsLRY7gITKt+ys2+MrJtTyW3Jaa2X9fjQX7l9Lq+K8o4EcrXVqff0MQgjhzCQPCyFcmdO1WFTHmoDPG9XQWucqpVBKRWutM8sf9qjFeeOoZf9ceHi4joqKqs1lhBBuYNWqVUe01o3NjsNWJA8LIRxdTfOwyxTIQCgYibia/dEYD4zUmPUBkRytdYb1+5E1Gb2Iiopi5cqVtbmUEMINKKV2mR2DjUkeFkI4tJrmYVcqkEPq82TWB03mAiFKqfLNmUCVidl6+280QKtWreozFCGEcBaSh4UQLsFpHtKzN+ttwEZaa1XhVW3PnNZ6qtY6Rmsd07ixy95BFUIIu5E8LIQwi8sVyNYeOCGEECaRPCyEcHau1GJR3tcWCpzpf6uQqG2+NKlSajgwvF27dra+lBBCOCLJw0KYpLi4mOzsbAoKCswOxSH4+fnRsmVLvL296/R5lymQrU9JZ3J+D1woxsTyNk/MWutZwKyYmBiZp1MI4XYkDwthnuzsbIKCgoiKiqJCz75b0lpz9OhRsrOzadOmTZ3O4WotFukYKy1VZLFuF0IIYXuSh4UwQUFBAWFhYW5fHAMopQgLC7uk0XRnLZBDqfpp6UTOX/o0wbrd5pRSw5VSU/Py8uxxOSGEMJPkYSEcjBTHZ13qfwunKZCVUiFKqWSl1EyMpJyslEpRSo0sP8Y692ai9biRSqkJQLI9butZrz9Laz06ODjYHpcTQgi7kjwshHAXTtODXJ50a3BcBpBh+4iEEMK9SB4WQrgLpxlBFkIIIYQQzik19aILYJ6RmZlJr169iI+PJz4+ntzc6hbntB0pkOtRbXvfSkqKSfn+CbIP7bRtYEII4Sbq0oOcsX0h36S9S2lZqQ0jE8K9TZ8+vcbH5ubmEhcXR1paGmlpaYSE2H9qdadpsXAGtZ1e6Jf/fcm7ebOYMvsHOhR50d67LR2bD6B/95G0adZSmu2FEKKW6jLN24dzk1jol8cbX3zKoDZxxDSLpUfjHkQFR+GhZBxJOJ/nZ23kj33H6/WcnSMa8uzwLtXuz8jIICUlhZycHIYPH86sWbMIDQ0lISGB6dOnk56ezqhRo5g2bVqNCt7U1FQyMzMJDQ0lJSWlPn+UGpEC2UTxvf9McXEhSzJ/YJPazQ8eW/nh4FY8f/mIVkWKpsVhBHl2xD/oKhqF96BJQ1+aBPnSpKEfTYJ8CW/gi5+3p9k/hhBCOLUJN35Ohy/u5ID/fubvmMOszB8BaOjTkO6Nu9OjSQ96NO5B1/CuBHgHmBytEI5rxowZZGVlkZiYSFJSEhaLBQCLxUJGRgYzZ84EjBaK6louJkyYQHR0NMnJyYwcOZJRo0aRnp5OXFyc3X4OkALZVD4+vtx0VQI3XZUAwL5DO1iweiYbDyxlh9rDmoAjFHgshpLFNNlTRrOCQDjVmr0ne7KrsAsaLwJ9PAlt4ENooC9hgT6EBvoQ4u9NkJ83QX5eFV7eld698PWS4loIIaKateH2+35lxWcTeSnrc+Z7R/JJ85tQIafZmbeVhXsXAuClvOgU1glLEws9m/bE0sRCI79GJkcvxPkuNNJrS7feeishISEkJiaSmJhIbm4uKSkpREdHn3NcdHQ0EyZMqPY8ISEhjBxpTI4TGxtLZqZdJsE5hxTI9ehSlziNaNKW24dNPPN9SXEBW3bMZk1WGquObCDD4xhHgzYDm2lRWsZlZYE0UVHg0Yu9pX04eLyMTfuPk3e6mFNFF++l8/HyoGGlojnI15sAX08CfDwJ9PHCv+K7ryf+3l7GPuvXgb6eZ4/x9sTDQ9pChBDmqWsebtYokOFj3+HAqmvoO/tBrtgzjfE7RrOwLIGwoBLaRh4lMHgPx4u38tXmr/j0j08BiA6OxtLUgqWJBUtTCxGBEdIeJ9xedHQ0M2fOJD09nZSUFJKTk8/Zf7ER5IojxitWrCApKcnmMVemtNZ2v6iri4mJ0StXrqz38+qyMvZkL2HVth9YfSiDjNMH2WUdBPYr01yu/IgN6UDvNsPo3OEWCrUv+QUlHC8oJr+gxPoyvj5RWPX2/AKjuDZeJRQUl9UqRj9vj9oX1j7VF+QB3sbXPl7SByicn1Jqlda68ipzwgYuKQ/n7kHPvAe1dyWbo+7iQ/97WL4rn905pwDw9ymjfWQuoWHZnPbczq6Tf3Cy+AQATQOaYmlqoXez3vRp3oeWDeR5EmEfmzZtolOnTqZdv7wHOSUlhcmTJ5OWlgZAcnIyFouF+Ph4gCpHlCvLzMw8MwJtsVjOK7Brqqr/JjXNw1Ig24CtCuSqHDm8idVbviNj31JWndzDZlWCVgr/Mo3FI5DeoV3o0+46OrYfjqeXT63PX1qmOV1cyqnCEk4VlXKyqITTFQpoY1spp4tKOFlYyuniUk4Wnj2m/PgzxxSVWr8voTZ/9Lw9Ff7engT6el2gsPbE38fL+u5JgI+1EK/8OW9PfL098PMy3n08PWTkW9iFFMj2c8l5uKQIfn0KlqdAZB8Y9W8OEsrKncdYuSuHlTuPsXFfHmUaoIzGoccIC9+Lh38WOaWbOVl6DICIwAh6N+9N72bGq2lg03r5+YSozOwC2RFJgexg7FkgV5aXt5uVG79mefZClp/YzXZP4/9vUJkmxjOYPo270bv9jbSLvhrlYd6orNaawpIyo4guLDmnsD5Zofg+VbmwLizhVIWCvbxQP1uEl1JUWrtRbzDaTXy9PPDz9sS30tdnt3ni5228+3pXtd8D3wt8xs/7/P3enjIy7k6kQLafesvD61Phh0fB2x9GfAhtrzqz60RhCWt257J+bx7bDuWz7eAJth86weniEjx8DuMZuIPA4Czw206pMkafm/m34oqIPgyM7Ets01hC/Ow/fZVwTVIgn08KZAdjZoFc2ZHDm1j+x39Yvm8py07vJ9vakhFapunjHUbvpr3o03EELVv0NbVgrk/FpWXnjHCfKrR+XWx8fbq4lMKSUgqKyygsKaWwuIwC63thSRmFxaUUlpRRYH2veOyZz1TYfym/Qp4e6ryCvHIhfqYgr/ReoyK+ms/IqLk5pEC2n3rNw4e3wIy7jfernoQr/wHV5MuyMs3e3NNsPZjPtkMn2Hown60Hj7MjdxslvlvxCtyBp38WyrMIgBDPKDoE96RfRB/io68gMiRUWjJEnUiBfD4pkB1EhYdDHti2bZvZ4VRp797lLN+cyvIDK1hWeJjDnkYijiiF/gEtuLJ1HH0uv5uABk1MjtQ5aK0pKi0vrKsuqgurKKoLis8W5AUXKMiN7ZXOYy3oi0sv7Xe3fNT8bFFdg0K7ihF23+pGy6v5jJeHctsCQApk27NZHi48AT/+HdbPhC43w00fgLdfjT9esXDefDCXVfvXsi1/DTmlf6D8dqE8StDaA1XYikaqG9GBMVwe3ok2jRsQFR5IVFggjQK83fZ3R1ycFMjnkwLZwTjSCPKF6LIysnbNZ/mW71hyKINlpXmc8lB4a41F+XNleA8GdLyV6DZDXWZ02ZWUlukKhfNFCu0Ko+QFVXymsIaj5eWfv5S04aGovgWlmlHzyqPlPp4eZ/rHywv0swW/8fny730qfW9mgS4Fsv3YJA9rDUvehrRnoFVfuP0rCAi9pFOWlWkyjx5jbuYKlh9Yxtb8leSWZhn7SoIoPdGBkhOXUXKyHcG+wbQOC6BVaMCZ91ahgbQOC6BZQz+5K+TmpEA+nxTIDsZZCuTKigtPkrHxSxZlzmHh8R3ssPYvR5TCgIAWDJDRZYExal5cqisU2lUX1ee1qlQzwl5VIW6rUXMApbhoEe1befs5BfnZY6/p0oyo8MBaXFsKZHuxaR7e8A18NwZCWsMdMyG0Tb2e/sjpIyzeu5jf9yxgyb6lnCzJR+FBI8/2+BZ15mReOw4eDqOkwuMWPl4etGzkT6vQACIbGcXzyF4taRRY+4ezhXOSAvl8UiA7GGctkCvbv28VCzd8zqIDK6oYXe7JkMvvIrJl/2p78YSob+Wj5kUlxgh4UUmFgvzM9xW2F5cZLTDFpdb3sjMtMdV99vxjK1+vjFJj6gI+vieGIR1rPiuBFMj2Y/M8vGsJfP1nUB5w+5fQup9NLlNSVsL6I+tZmL2QRXsXsSlnEwDhfuH0CO9Dm8BYGpR15UBuGXtyTrHb+sovKGHZE0Np2rDmbSDCuUmBfD4pkB2MqxTIFVUcXV50fMeZ2TE6FJUwVAUypGF7LmvaE9W4IzTuaIyoeHqbHLUQtlFSahTP3p61m4lECmT7sUseProDvroVju2CPgnQ/+/QoLFNL1k+urxo7yIW71tMflE+vp6+9I3oy9BWQxnUchCN/BqRd6qYhv5e0rPsRtyhQE5MTCQzM5PY2NgLrsRX7lIKZFlJT9SIt28gfSyj6WMZzT+APdnL+G3jF8w9uo4PinOYcnoDLbat4aq1pxh66jQ9i8rwDGsLjS+D8MuM98YdIbwD1GE+ZiEciZenB14yRZ8Iawv3p8PPT8D/3oeVn8AVY6DfI+BvmyWow/3DubHdjdzY7kZKykrIOJjBvD3zmLt7LvP3zMdDedCraS+GRA5hSKshRDSIsEkcQtRWamrqmeWja/OZlJQU0tLSyMjIAGDmzJlnCuWLLThyKWQEuR45wywWtnD09FF+z/6deTt/ZemB5RSVFdPIw4dBBBCff5y+h3fjra3Nch7eRrHctAs07QrNukLTy20+6iKEI5ARZNszLQ8f3grzX4GN34JvMAxOhNgH7DYgoLVmU84m5u6ey7zd89ieux2ATqGdGNJqCNdEXUNUcJRdYhHmcPQR5FGjRjFz5swaH5+ZmUl6ejozZ84kLS2NyZMnY7FYiIuLIzU1lZycHEaPHn3Bc0iLhYNxxRaLmjpVfIpFexcxb888FuxZQH5xPsE+wcQ3ieFa/xb0OnkCz4Mb4eAGyN9/9oMNmp5bMDe7HMLbg4eneT+MEPVMCmT7MS0PH9xozHKxPR3C2sM1r0L7OLuHsev4Ln7b/Rvz9sxjzaE1aDSdQjtxTZtruCbqGhlZdkHnFINzJsKB9fV7gWaXw7WvVru7fKnpnJwchg8fzqxZswgNDSUhIYHp06czdepU4uLimDZtGiEhF18gJyEhgeTkZEaNGkVaWhqJiYncdtttWCwW0tPTSUtLu+gS1NJiIRxGgHcAV0ddzdVRV1NcWsySfUuYs3MOP+2eR2rJacL9wxnWYRjXXP0U3QNaog5tgAMbjIL5wAZY+j6UFRsn82kAzbtDRM+zr9BoYxoCIYRwRE27wB2psO1X+DkJvhwBHa+HP70ODZvbLYzWDVtzT9d7uKfrPRw6dYhfd/7KnJ1zeHPVm7y56k26N+7OtW2u5erWV9M4QO7gifoxY8YMsrKySExMJCkpCYvFAoDFYiEjI+PMCHJmZiapqalVnmPChAmkpqbSq1evcwrp3Nxc2/8AFcgIsg248whydU6XnGZh9kLmZM1hQfYCisqKiAiM4Jo213Bj2xuJDrH2EZUWw5GtsH8t7FsD+zKMfwWXFBj7/YKheQ+jWG5hMd6DI6VoFk5BRpDtxyHycEkR/O89mP8qePrCsEnQ8y5T81V2fjY/7/yZn7N+ZsuxLSgUsc1iuT76eq6OuppA75pPWygci9ktFuUjyCkpKWRmZpKYmEhubi4pKSlER0cTHx9PWlpajc6VkJBATk4OAOnp6SQlJQEQHR3NyJEjSU1NJTMz86IP6kmLhYNxiMTswE4UneC3Pb8xO2s2S/ctpVSXcnn45dzU7iaGRQ0j2Df43A+UFsPhzbBvNezNMN4Pbjw70hzYGCL7QGRv4715j1qtcCWEvUiBbD8OlYePbIdZj8KuxRA9GG54B0JamR0VmbmZ/LzzZ+ZkzWHn8Z34e/kT1yqOG9vdSGyzWDyUPIjqTBypQC5XsRWiYoF8sRHkcrm5uWdaLDIyMpg+fTrJyckkJCSQkJBwZoS6OtJiIZxKA58GDG87nOFth3Pk9BF+yvyJ77d/z4v/e5Hk5ckMaTWEG9vdSN/mffH08DSmi2tm7Uu23G2cpKTQKJL3ZUD2KtizDDb/aOzz8IaIHmeL5pa97XprUwghzhHeDv76I6z6xOhPnjIAbnjLWLLaRNEh0TzU4yEe7P4g646s47/b/8vPWT8zK3MWEYER3NDuBm6IvoHIhpGmximcz+TJk88UwxX7hOPj48+MKNdkmraKLBYL06dPZ9SoUURHR1+0OL5UMoJsAw41cuEkyp/A/n7798zOmk1eYR5NAppwS/tbGNF+BM0Cm138JCcOQ/YKo1jes9wonstbM4JbGcVyqysg6kpjJg1pyxB2JiPI9uOweTgnC765H/auhJ53wrWTwcdx2hoKSgqYt3se/93xX5buW4pGE9sslls73MrQVkPxlvntHZbZI8iOSFosHIzDJmYnUVRaxO/Zv/PNtm9YsncJHsqDQS0Hcdtlt3FFxBU1v+1XUmT0L+9ZdvZVPnNGQLhRLDfpbMyWEdbOKJod6C8q4XqkQLYfh87DpcVGX/LCfxr555pXoe0Qh/tH+4GTB/hhxw98u+1b9p7YS5hfmDFo0WEELRq0MDs8UYkUyOeTAtlBuOs8yLa0J38PqVtT+W7bdxwrPEZkUCSjOozilva3nN+rfDFaw7Es2LnYWCZ2zzLj+/I5mlHGQibNuxstGs17QPNu4BtU7z+XcE9SINueU+XhzN/h+wfh+F6jQL7un8ZMPQ6mtKyUJfuWMGPLDBbsXYDWmitbXsntl91O/xb9pVfZQUiBfD4pkB2MQ49cOKmi0iLSd6Uzfct0Mg5l4O/lz83tbubOzncSGXQJ/XElhcYtzyNb4dAfxswZ+9dC/j7rAcoYWY6wGLNmtLAY8zV7+dbLzyXcixTI9uM0ebik0FiBb94k48HjQYnGSnwO2sqw/8R+Urel8u22bzly+gjRwdHc3flurm97Pb6ekhfNJAXy+aRAdjBOk5id1JacLXz2x2fMzppNmS5jaKuh3N35bno06VF/FzlxyDrN3GrYu8roZz552Njn6WMUyS0s1sK5lyxqImpECmT7cbo8fHw/zJkAm36AZt2MmS4i6jGn1bPi0mJ+3vkzn/3xGZtzNhPqF8rtHW/ntstuI9Qv1Ozw3JIUyOeTAtnBOF1idlIHTx7k681fM2PrDPKL8unWuBujLx/NwJYDUfXdy6c15GWfLZb3ZhgFdFG+sd+ngdGSUT7K3KKXzM8sziMFsv04bR7+4weYPR5OHjFGkgdPBG9/s6Oqltaa5QeW8+nGT1m4dyG+nr7c1O4m7u16r6zWZ2dSIJ9PCmQH47SJ2UmdKj7F99u/57M/PmPvib10Cu1EQrcErmp1lW1748rK4Og2o1guL5wPrIfSImN/QPjZUebylQCDmtouHuHwpEC2H6fOw6ePwa9Pw+rPjQeIb3gHWvczO6qL2pG7g8//+Jz/7vgvaLih3Q3c3/V+mSbOTqRAPp8UyA7GqROzEysuK+anzJ+Ytm4au/N3075RexK6JRDfOt5+D5GUFBnLZpePMu/NgCNbzj4I2LCFtVjuYbRpNO4IIa3BQx5ycQdSINuPS+ThHb8ZC4zk7obYB+CqJyDA8dsXDpw8wMcbPuabrd9Qqku5Lvo6Erol0Kqh+YujuDJXK5BHjRpFbm4uFovlzFzKiYmJZGZmEhsbe2Ye5aq2lZOFQoQAvD28uandTVwffT0/7/yZqeumMv738XQO68xjvR7jiuZX2D4IL5+zbRax1m2FJ4yR5X2rra+Ms4uaAHgHGLNnNOlkFMzl78GRUjgL4c7aXgUPLoV5L8KyFFg3Hfr+Da54CPwamh1dtZoFNuOJPk/wwOUP8O+N/2bGlhnMzpzNiA4jGNN9DOH+4WaHKEyQmprKyJEja3Ts1KlTSUpKwmKxMGrUKDIyMs7smzlz5pmiODc397xt0dH1MxOMjCDbgEuMXLiA0rJSZmfN5p3V77D/5H76RfTj75a/0ynMAf6FXXDcWD770KZz38vnaQbwDjRm0GjcEZp0hMadjHfpbXZaMoJsPy6Xhw9uhN9eNv5x7d8I+o+F3qOdYu72I6ePkLI2hdStqXh7evPXLn/lr53/SgOfBmaH5lIcfQR51KhRzJw5s0bHVix0p06dCnBmNDkuLo7U1FRycnKq3DZ69Ogz55ERZCGq4OnhyfC2w7k66mqmb57O1PVTufXHW7k++noe6/UYTQKamBecX0NjZb/I3uduP30MDm85t3DeMRfWfnX2GJ8GRuEcZl3gJLydMQId2ha8/ez7cwgh7KNpF7j9S+Mu1G8vQ/pzsPQ9uPIf0Ov/HPp3P9w/nCeveJK7Ot/FO6vf4YO1HzB983Qe7vkwI9qPwFNmAKp3ycuT2ZyzuV7P2TG0I4m9E6vdn5GRQUpKCjk5OQwfPpxZs2YRGhpKQkIC06dPJz09nVGjRjFt2jRCQkIueK2Ko8CrVq06c47QUKPFKCQkhBUrVgBUua0+SIEsXJ6vpy93d7mbm9vfzMcbPuazjZ8xb/c8EroncFenuxxr6VT/RsYKf60qtYOcyjl3pPnwZti5ENb9p8JBChq1Norlyq/AMLv+GEIIG4noCXfMhN3L4LdJ8PNEWPw2DBwPPe8y2rwcVKuGrXht0Gvc0+UeXl/5Oi/+70VSt6aS1CeJnk16mh2eqAczZswgKyuLxMTEMy0SABaLhYyMjDMjyJmZmaSmplZ5jop9xBkZGeTk5GCxWEhJSTnv2PIWC1uQAlm4jSCfIMZaxnJLu1uYvGIyb656k++2fcdTVzxFn+Z9zA7vwgJCjafYKz/JXngCcnbAkW3W11bjPWsBlBScPc4/1CiUG1csnNtbHxCU0RshnE6rPvDXWcbv+rxJ8NM4WPwvGDQRut0Gno7713uX8C58POxjftn5C6+vfJ2759zN8OjhPB77OI38Gpkdnku40EivLd16662EhISQmJhIYmIiubm5pKSknNcXHB0dfd4DdZWVf7a8qG7bti2ZmZlYLBZyc3MJCwsjLCzsvG31xXF/g5xQhSVOzQ5FXEBkw0jeGfoOC7MX8uryV7n/1/sZ0X4E42LG0dDHcR98qZJvA2Np7Obdz91eVgp5eyoUzdbCecscyPjs7HFe/tCsq/UcPYzZNRp3dNhVvIS4GLfLw20Gwr2/wPa5xsN8/30IFr0Bg5Og6wiHfV5BKcU1ba5hYMuBTFs/jX9v/DeL9i4iqU8S10RdU/9z2Qu7io6OZubMmaSnp5OSknJmFopyNRlBTkxMPGfUOC4ujunTpzNy5EjS0tJISEgAqHJbfZCH9GzA5R4OcWEFJQW8v/Z9Pt34KeF+4Tzd92kGRw42OyzbOpVztnA+tAn2r4H9684ueuLlZ6zk1aLX2UVPQqMd9i9aZyIP6dmPW+ZhrWHzT0aP8qGNED0Yhr8FjaJMDuzith7byrOLn2XD0Q0MbjmYp654iqaBMm98bZj9kF55D3JKSgqTJ08mLS0NgOTkZCwWC/Hx8QBVjihXNnnyZFJSUs70KiclJTFy5MhzZqqoPPVbxW3lZB5kB+OWidnJbTyykaeXPM22Y9sY1WEUj8c+jr+X465eVe/KyiAn0yiWy5fX3r8Wik8Z+/2CrctqWwvmCAs0bG5uzE5ICmT7ces8XFYGqz6BtGdBl8LQZ4wZLxy8naq0rJQvNn3Bu6vfxdfLlxf6vcCQVkPMDstpmF0gOyIpkB2MWydmJ1ZcWsw7a97hkw2fEB0czeSBk7ks9DKzwzJPaYnxMOA+60qBezOMqaZ0qbE/qLm1WO559t3/wk8muzspkO1H8jCQlw0/PgbbfoWWveHGd40ZcBzczrydTFgwgU05m7jtstsYHzMePy/HnaXDUUiBfD4pkB2MJGbntnTfUp5c9CS5hbkkxiZyW8fbzA7JcRSfNhY9KS+Y964yHhIsF9bOOtJsLZibdjH6pAUgBbI9SR620hrWz4Q5iVB0AgZNgP5/d/jnDIpLi3l79dv8e+O/aRvcljevepM2wW3MDsuhSYF8PimQHYwkZueXU5DDk4ueZNHeRYzqMIqk3kmONR2cIzl9zNqWkXG2aD5xwLpTQWgbaHa59dXNeA9q7pY9zVIg24/k4UpOHIY5j8PG74xl7q9NhqgBZkd1UUv2LmHiwomUlJXw2qDX6N+iv9khOaxNmzbRsWNHecDRSmvN5s2bpUB2JJKYXUNpWSnvrH6HjzZ8RK+mvXhj8BuE+oWaHZZzOL7P6GE+sB4OrIMDG+BY1tn9/qFni+Ymna0rBXZ0ilXBLoUUyPYjebgam3405k7O2wOdhkPc8xDW1uyoLmjfiX08Mu8RtuduZ1yvcdzd+W4pAquQlZVFUFAQYWFhbv/fR2vN0aNHyc/Pp02bc+88SIFsIknMruWnzJ94dsmzhPmFMSV+CtHB9bPOu9spOG70MB9YDwfXW9//gNJC6wHKaNFo3h2adzNGm5t3N+aAdhFSINuP5OELKD4NS9+FhW8LmucrAAAgAElEQVQav39XPAiDEsE3yOzIqnWq+BRPLX6KtF1p3NzuZp7p+wxeHjJTbUXFxcVkZ2dTUFBw8YPdgJ+fHy1btsTb+9y7v1Igm0gSs+vZcGQDf5v7N7TWTImfQpewLmaH5BpKS+DYTji8ySie968zRp6PZ589JjjSmJ85oqfxat7DaYtmKZDtR/JwDZw4ZMydnPGZ0fZ0zSvQ+SaHbX8q02W8v+Z9UtalcFXkVbw26DV8PX3NDks4GSmQTSSJ2TXtOr6L0b+OJq8oj3eHvEtMM6lzbObkUWtrxjrYt8aYfi4n8+z+RlFni+WInsZIsxPMoCEFsv1IHq6F7JXGbBcH1kH7YfCn14xl6x3UV5u+4pXlrxDbLJZ3h7xLgHeA2SEJJyIFsokkMbuugycP8kDaAxw4eYCp8VPp0aSH2SG5j9PHjNHlfavPvnJ3n90f2vbsKHPLWKNo9nasqaGkQLYfycO1VFoCy1Ng3kuAts6dnAAeHmZHVqUfM3/kqUVP0atpL94b+p5MAydqTArkeqCUsgChQAgQDyRrrTMv/ClJzK7u8KnD3PPzPRwrOMaHwz6kc1hns0NyX6dyzi2Y962G43uNfR7exkOALWOtrxhj5NnE28dSINee5GE7y90DP40z5k6OuhJueh9CWpkdVZVm7ZjFk4ueZECLAbx11Vsy05CoESmQ64FS6hjQRmudq5QaDSRorXtd7HOSmF3f/hP7+evPf6WgpIAvr/uSyKBIs0MS5fIPGLeM9660vmdA8UljX0DY2WK5aVcI7wAhrcHTPg/7SIFce5KHTaC10Zf8yxOAgmGToMeddvs9qY0ZW2bw4v9e5E9t/sSrV77q9rM3iIuraR52vD/tNaCUGgnkaq3Tq9hnAWKATCAayKzquBpqo7XOtX6dU8dzCBfUvEFzpsZP5Y7Zd/DovEf5/NrPaeAjC2I4hKBm0Ol64wXWFQE3QfYKyF5lvG/9+ezxnj5Ge0Z4e6NgDu9g/bq9Qz/VbzbJwy5MKej1V4geBN8/BLPGwuK34Mp/QLfbHGqRkVsvu5W8wjzeXv027ULa8UC3B8wOSbgIpxtBVkrFATOBUZUTrlIqGkjRWsdX2DYTSKzJLbmLXDcN49beRZO8jFy4j//t/x9j0sacucXn6eFpdkiiJgry4PBWOFL+2ma852SeXUobICiiisK5AzSMqFOrhquMIEsediNlZbDlJ/h9svEQX0grGPAY9LgDvBxjBgmtNRMXTmR21mz+NfhfDG091OyQhANzuRFka9JNBFZR/ShCApBSaVsKkAyMuoTrjgTSLmEERLioK5pfwcTeE3lp2UtMXT+VB7s/aHZIoib8giEy1nhVVFJkTDt3ZCsc2XK2cF43HQqPnz3Op4ExZ/PVk6DNlXYN3UySh92Qh4exoEjH62HrL7BgsjHjxYLX4Ya3oV2c2RGilOL5fs+zJ38PSYuSSG2USquGjtk3LZyH040gAyildmD0oVUeudgBxFccpVBKhQDHtNbK+v1o4ELLBlWZgK2fG1VxVKQ6MnLhfhIXJPLLzl/44k9f0DW8q9nhiPqmNZw4eP6I89BnjFkzashVRpBB8rDb0hp2zDMK5Oteh6aOMyf8gZMHuOWHW2gT3IZPr/lUFhIRVXLph/SqSszlCRhoVKFfrXyfBtrW5vZe+YiF1npypfNf9DySmN3P8aLjjPhhBH6efky/frrMyymq5OoFsuRhYbY5WXOYsGACf+vxN8Z0H2N2OMIB1TQPO+YEh3UTClA5KVdQ2/WBo4GwSt/nXmoPnXBNDX0a8lL/l9h5fCdvrnrT7HCEMIvkYWGqa9tcy5/a/IkP1n7AxqMbzQ5HODFXKpDrdRkt66jICqXUaOttvSSg2s5/63ErlVIrDx8+XJ+hCCfRu3lv7ux0J9O3TGfDkQ1mhyOEGSQPC9M90ecJQv1CmbR0EmW6zOxwhJNypQK53mmtU7XWU62vUVrrjAscO1VrHaO1jmncuLE9wxQO5G89/kaYfxgv/e8lScxC1APJw6K2gn2DGRczjg1HN/Ddtu/MDkc4KZcrkK09akKYooFPAx7r9Rgbjm4gbVea2eEIYQrJw8Js17W5jp5NevLemvc4XXLa7HCEE3KlArm8Jy204sYKidrmPWtKqeFKqal5eXm2vpRwYNe1uY52Ie14d/W7lJSVmB2OEPYkeVg4BKUUYy1jOXz6MP/Z/B+zwxFOyGUKZOtDIZmc3wMXip0e6tBaz9Jajw4ODrb1pYQD8/Tw5OEeD7Pz+E5+2fmL2eEIYTeSh4Uj6dW0F/1b9OeTDZ/IKLKoNZcpkK3SMZY3rchi3S6E3VzV6ipjLs6Nn+KMUykKcQkkDwuHcX/X+zlWeIxZO2aZHYpwMs5aIIdS9dPSiZy/UlOCdbvNya09Uc5DefDXzn9lU84mlh9YbnY4QtiC5GHh8Ho17UXXsK58uvFTeXBa1IrTFMhKqRClVLJSaiZGUk5WSqUopUaWH2O9vZdoPW6kUmoCkGyvOTPl1p6o6Pq21xPiG8LMrTPNDkWIeiF5WDgbpRR3db6L3fm7ZbBC1IrTrMNYnnRrcFwGUO00QELYi6+nL9dHX8/0LdPJLcglxE8e7BfOTfKwcEZDWw8laFkQ3277liuaX2F2OMJJOM0IshDO6Ob2N1NcVsyPmT+aHYoQQrglX09frmtzHXN3zSWvUFpvRM1IgVyPpPdNVNahUQc6hXZiTtYcs0MRwi1IHhZVuandTRSVFTFv9zyzQxFOQgrkeiS9b6IqV0ddzboj69h/Yr/ZoQjh8iQPi6p0DutMiwYtZAEnUWNSIAthY/Gt4wFI3y2zXAkhhBmUUsS3jmfp/qUcLzpudjjCCUiBLISNtW7Ymg6NOsjIhRBCmCi+dTwlZSX8vud3s0MRTkAK5HokvW+iOoMjB7P28Fp5QEQIG5M8LKrTNbwrYX5hLMxeaHYowglIgVyPpPdNVOfKFldSpstYun+p2aEI4dIkD4vqeCgP+rfoz5L9SygtKzU7HOHgpEAWwg66hnclyCeIRdmLzA5FCCHc1oAWA8grzGPD0Q1mhyIcnBTIQtiBl4cX/SL6sXjfYrTWZocjhBBuqV9EPzyUB4v2ymCFuDApkIWwkwEtBnDk9BG2HNtidihCCOGWgn2DuTz8chbvXWx2KMLBSYFcj+ThEHEh/SP6A8jIhRA2JHlYXEz/Fv3ZcGQDOQU5ZociHJgUyPVIHg4RF9I4oDEdQztKgSyEDUkeFhdzZYsr0WiW7FtidijCgUmBLIQd9Y/oz5pDa8gvyjc7FCGEcEudwzrTyLeRDFaIC5ICWQg7GthyIKW6lAXZC8wORQgh3JKH8mBAiwEszF5IcWmx2eEIByUFshB21KNJD5r4N+HnnT+bHYoQQritYVHDOF50XOamF9WSAlkIO/JQHgxrM4zFexdzvOi42eEIIYRb6hfRjyCfIH7OksEKUTUpkOuRPD0tauLaqGspLiuWxCyEDUgeFjXh7elNfOt45u6eK4MVokpSINcjeXpa1ETX8K50b9ydD9Z+wKniU2aHI4RLkTwsauovHf/C6ZLTTFs3zexQhAOSAlkIO1NKMT5mPIdPH+aNVW/IynpCCGGCy0Iv44a2N/Dlpi9Zf3i92eEIByMFshAm6NGkB3d2upPpW6Yz/vfx7MnfY3ZIQgjhdsZaxtIkoAn3/XofX2/+mqLSIrNDEg5CCmQhTDIhdgKP9HyE+XvmM/y74Tyz+BnyCqVvUggh7KVxQGM+v/ZzOod15uVlL3Ptt9fy/fbvzQ5LOAApkIUwiVKK0d1GM2fEHP7c8c/MypzFHbPvIDs/2+zQhBDCbTQOaMwnwz5havxUmgc25+nFT/Pyspel/c3NSYEshMmaBDQhsXciH139ETkFOUxYMIHiMpm8Xggh7EUpRd+Ivnx6zafc1fkuvt78NTO3zjQ7LGEiKZCFcBCWphae7fss64+sZ+q6qWaHI4QQbsfTw5PxMePp27wvr614jay8LLNDEiaRArkeyfyb4lINixrGDW1vYOq6qaw5tMbscIRwOpKHxaXyUB5MGjAJXy9fkhYmyR09NyUFcj2S+TdFfUjqnUTzwOb8/be/S5EsRC1JHhb1oUlAE57r+xwbj25k/PzxMme9G5ICWQgH08CnAe8PfZ8A7wDu/eVevtv2ndkhCSGE24lrHUdibCLzs+dzx+w7ZDpONyMFshAOKDokmq+v+5peTXvxzJJn+GbrN2aHJIQQbufOzncyJW4Kh04d4p4593D41GGzQxJ2IgWyEA4q2DeYKXFT6B/Rn0nLJkm7hRBCmKBfRD8+HvYx+cX5/P23v1NcKj3J7kAKZCEcmJeHF5MHTaZZQDMmLpzIiaITZockhBBu57LQy5jUfxLrjqxjytopZocj7EAKZCEcXEOfhrxy5SvsP7mfJxY9IaMXQghhgqujrubmdjfz0YaPSNuVZnY4wsakQBbCCfRo0oPE2ER+2/Mbj/72KAUlBWaHJIQQbmdi74l0b9yd8b+PZ9aOWWaHI2xICmQhnMRfOv2F5/o+x+K9i3l47sMykiyEEHYW4B3AB3EfENs0licXPcl/t//X7JCEjUiBLIQTGdFhBC/2f5FlB5bxr4x/mR2OEEK4nQDvAN6Le4/ezXvzwtIX2JKzxeyQhA1IgSyEk7mx3Y3cftntfPbHZ8zfM9/scIQQwu34evoyeeBkQnxDGP/7eE4WnzQ7JFHPpECuR7LEqbCX8bHj6RjakacWP8W+E/vMDkcIhyF5WNhLqF8orw58ld35u3lh6Qtorc0OSdQjKZDrkSxxKuzF19OX1we9TmlZKff9cp8UyUJYSR4W9hTbLJaHuj/E7KzZvLTsJcp0mdkhiXoiBbIQTqp1w9ZMu3oaeUV5JKQlcLzouNkhCSGE2xndbTT/1+X/mL5lOh+s/cDscEQ9kQJZCCfWNbwrb1/1Ntn52UxcMFFu8QkhhJ0ppXis12Pc2PZGpqydwtzdc80OSdQDKZCFcHIxzWJ4PPZxFu5dyMytM80ORwgh3I5Simf6PkPnsM48v+R5jpw+YnZI4hJJgSyEC/hzxz/Tt3lfXl/5OhuPbDQ7HCGEcDs+nj68POBlThafZOKCiTJXvZOTAlkIF6CU4uUrXybUL5SH5j7Envw9ZockhBBup21IW57p+wzLDizjmSXPSNubE5MCWQgXEe4fzvtx71NSVsJD6Q9xrOCY2SEJIYTbubHdjTzc42F+zPyRt1e/bXY4oo6kQBbChUQHR/Pu0HfZe2Ivzy99XkYvhBDCBKO7jWZE+xF8uP5DFmYvNDscUQdSIAvhYno26ckjPR9h7u65/Jj5o9nhCCGE21FK8USfJ2gX0o7nljxHbkGu2SGJWpICWQgXdHfnu7E0sfDC0hfkoT0hhDCBj6cPkwZM4ljhMf7x+z/koT0nIwVyDSmlUsyOQYia8vTw5I3BbxDmH8Yj8x7hwMkDZockxCWTPCycTZewLjzf73mWH1jOpGWTpO3NiUiBXANKqThgtNlxCFEbYf5hvDPkHU6VnOLReY9SWFpodkhC1JnkYeGshrcdzuhuo/l227d8uelLs8MRNeSUBbJSaqQ1WVa1z6KUGq2Uiit/v8RrhQA5gDQQCafTvlF7kq9MZlPOJqasmWJ2OMKFSB4Woub+1uNvDI4czJur3iQzL9PscEQNOF2BbE2006rZFw0ka62naq3TtdZTgQTr9rqK01pnXMLnhTDVoMhB3NzuZj7Z+AmrDq4yOxzhAiQPC1E7HsqDZ/s+i7+3P08sfELu6DkBpymQlVLR1v6zaIyRhKokAJV71FKA5DpeMw5Ir8tnhXAkj8c+TmRQJOPmj2P/if1mhyOclORhIeou3D+c5/s9z8ajG3lh6QvSj+zgvMwOoKa01pkYiRelVGI1h43k/MS8Ekgr/0YpNRpoe4FLpWmt08tv6Wmt5ZaecHpBPkG8PeRt7vjpDsb+NpZPr/0Ufy9/s8MSTkbysBCXZmiroTzU/SHeX/s+lzW6jLu73G12SKIa9V4gK6VKtdae9X3eGlw3hCpGNbTWuUoplFLRWutM6+2+mogDQpVSMdbvQ6xJPd36l4QQTiU6OJrkgck8PPdhJv1vEi8NeMnskIQNmZGLJQ8LcXEJ3RPYemwr/1z1T7qGd8XS1GJ2SKIKtmixUNXuUGqIUup+pVSPCtuGKqWG1MN1Q8FIxNXsr1X/m9Y61dpDN7U8mVu/lqQsnNbAlgO5//L7+WHHD8zfM9/scIRtmZGLJQ8LcREeyoNJAybRPLA5Ty9+mtMlp80OSVTBFgVylU01SqlXgTFAO2CyUuoVAK31XGBmPVw3pB7OcR6lVIhSaoL16wnVPWhifVJ7pVJq5eHDh20RihD14sHuD9K+UXueXfKszI/s2szIxZKHhaiBQO9AXuz/Irvzd/PyspelH9kB2fMhvR1a61u11hO11lcDM5RS4637jtkxjlrRWudqrSdrrZX1vcqRC+uoRozWOqZx48b2DlOIGvP29Oa1ga9RUFLAuPnjKCotMjskYV9Ol4slDwtXFNssloRuCXy//XtmbJlhdjiiErvOYqGUilJK3Q+gtV6ttX5dKfUA0Kger2GTEQwhXEnbkLa8NOAl1h9Zz0vLXpLRCzdj61wseViImnmox0Nc2eJKXl3+KhkHZSZDR1KrAvlS+ta01tMwnlpWVWy/tTZxVKN8RCG04sYKidrmPWtKqeFKqal5eXm2vpQQlyyudRwPXP4A3277lh8zfzQ7HFELDpyLJQ8LUQseyoNXB75KRIMIHv/9cfIK5c+to6hxgVwffWta67nWJHze9prGcYFz52Ik38ojF6FArj0e6tBaz9Jajw4ODrb1pYSoF3/r8TcsTSy8suwV6Ud2Eo6ciyUPC1F7DX0a8tqg18gpyJE7eg6kNtO87dBaTyz/RinVUyk1Xmv9Ouf2rSmlVBS1f1gjV2u9s5afqSwdiAEq3qewIJPMC1ElTw9PJvWfxMhZIxn721g+GfYJAd4BZoclLszRc7HkYSFqqXNYZ8Z0H8O7a96lW3g37ux8p9khub1azYNsTbZxWusPtdargdXWvrXQSodauMAUQ9XQwM4aHhtK1Uk/EWMEpeIcmwnWl80ppYYDw9u1a2ePywlRLyIbRjJ54GQe/e1RnlnyDK8NfA2lavvrK+zJQXKx5GEh6tED3R5gc85mJq+YTHRINP0i+pkdkltTtRnKV0oNBaIr35pTSg0tvzVnq8nprT1sSRjzaI7EuI2XjrHiUmqF4yzAbcAK67EZWmu7jlzExMTolStX2vOSQlyyD9d/yFsZb/Fs32cZ2WGk2eG4JKXUKq11zMWPvOh5TMnFkoeFsK1Txaf4y09/Ibcwl9QbUgn3Dzc7JJdT0zxcqwK5hhc2ZSU9RyKJWTijMl1GQloCaw6t4T/X/4e2IRdaCVjURX0VyDW8llvnYsnDwlltPbaVv/z0F2KaxvB+3Pt4KLtOOObyapqHz2uxqIeeNTN7kIUQdeShPHh5wMuMnDWS8b+P58s/fSn9yCaSXCyEe+rQqAMTYifw4v9e5OMNH3P/5febHZJbqqoHuVcdzlO5Z83WPcgOSXrfhLNrHNCYVwa8woNzH2TCggm8ddVbeHq47SCk2SQX14HkYeEKRnUYxYoDK3g7423aNGzD0NZDzQ7J7UiLhQ3IrT3h7L7e/DUvL3uZh3s8TEJ3uzxb5RakxcJ+JA8LZ1dQUsB9v9zH9tztpA5PJbJhpNkhuYSa5mG7NbZYJ7F/xfr1LRUnuBdCOJbbL7uda9tcy5S1U1hzaI3Z4Yh6JLlYCOfg5+XH64Nex9PDk4kLJ1JcWmx2SG7FFj3IF/KqdSqiBOA/SqlorfW3tbyWEMLGlFI8fcXTrD+8nnHzx/Gf6/9Dk4AmZoflViQXCyGaN2jO8/2eZ9z8cbyy/BWe6fuM2SG5DVv1IFd5jNY6Tyk1EpigtZ6nlBpRh2s5LOl9E64kyCeIt4e8zR2z7+Cx+Y/xybBP8PH0MTssdyK5uA4kDwtXE986nvu63sdHGz6iY2hHbr3sUleEFzVhtx5kawK+DWNy+1DrtiFa63n1GoADkN434UrSd6Xz2PzHuKndTbzQ7wVZROQSOEIPsrvkYsnDwpWUlpXyyLxHWLpvKR8O+5BeTevy72cBDtiDrLX+Rmt9a4WE/Ku9ri2EqLu41nGM6T6G77d/z1ebvzI7HHGJJBcL4Xw8PTxJHphMy6CWjJs/jv0n9psdksurU4GslJpe/mCHUiqqjg95JLjaiIUQrurB7g9yVeRVvLbiNZbtX2Z2OMJKcrEQ7iPIJ4i3hrxFUWkRY38by+mS02aH5NLqOoI8VWu9RinVBhiDcbuutoLreG0hhJ15KA9eufIVohpGMf738WTnZ5sdkjBILhbCjUQHR5M8MJnNOZt5dsmz1HebrDirxgWyUupXpdQUpdQtwA7r+1DgA611Ug0+31ApdX+FEY4s6zlchlJquFJqal5entmhCFHvAr0DeXvI25TqUsb+NpZTxafMDsktSS6+MMnDwtUNbDmQRy2PMidrDp9s/MTscFxWbUaQE4GJGKsyjanwGlnD23pPAI2AJ5RS24BXgdjahevYtNaztNajg4NlQEa4plYNW/H6wNfZnrudpxY/JaMX5pBcfAGSh4U7uK/rfVwTdQ3/WvUvFmYvNDscl1TVNG9V0lqvtn75jfUFgFKqJ9AWuNhqAmla67kVPjcCyKh5qEIIR9CvRT/G9RrH6ytfZ9r6aYzuNtrskNyK5GIhhFKK5/s9T1ZeFokLEvnquq+ICo4yOyyXUqMRZKXUB9XdgtNar9Zaf1PVPqXUNuutwCFU6nOzPkmdVeuIhRCmu7vz3VwffT3vrH6H33b/ZnY4bkNysRCiXIB3AG8NeQsvDy8e/e1R8ovyzQ7JpdS0xSINmHvRo873DZAO3ApMVkodVUqNl6VNhXBuSime7fssncM688SiJ9hzfI/ZIbkLycVCiDNaNGjBPwf/kz3H9/DM4mek7a0e1bRA3oHRs1YrWuuJ1tGJMVrrdhh9bnkYvW/blVJTantOIYRj8PPy443Bb6CUYvyC8RSVFpkdkjuQXCyEOEdss1jGWsaSvjudrzd/bXY4LqOmBfLtQIZS6pdLGXXQWmdqradZJ6lvB0yuy3kclTw9LdxNiwYtmNR/En8c/YM3Vr1hdjjuQHLxRUgeFu7o7i53M7DlQF5f+Tp/HP3D7HBcQo1HkK2rLo3h7KjDL7W5UFW9c67W9yZPTwt3NKTVEO7sdCdfbvqSebtlvQkbk1x8EZKHhTvyUB681P8lQv1CGf/7eE4WnzQ7JKdX0wJ5hlJqiNY6q8Kow7BaXquuvXNCCAc3rtc4Ood15pklz3Dw5EGzw3FlkouFEFUK8QsheWAye0/s5eVlL5sdjtOrUYGstc6rh6VI69Q7J4RwfN6e3rx65asUlRbx5OInKdNlZofkkiQXCyEupFfTXozuNpofdvzAnKw5Zofj1M6bB1kpFQWE1PI8uVrrnRc55nZgtFJqBcYIRrrW+mLzdQohnESb4DZM7D2RZ5c8y6cbP+X/uv6f2SE5NcnFQoi6SOiWwNJ9S3lx6Yt0a9yNFg1amB2SU6pqoZBedTiPBnZe5JgdWutQpVQbIA6jdy64DrcHhRAO6uZ2N7No7yLeznib3s160yW8i9khOTPJxUKIWvPy8OLVK19l1KxRJC1M4uNhH+PlUeN14YSVqu8585RSpVprzyq2BwO96uH2oMOLiYnRK1euNDsMIUyRV5jHiB9G4O/lz4zhM/D38jc7JIehlFqltY6x07XcOhdLHhbu7qfMn5i4cCIP93iYhO4JZofjMGqah2v6kN4lq6feOSGEgwv2DealAS+x8/hO3ln9jtnhiEokFwvhHq6Lvo5ro67lg3UfsCVni9nhOB1b9CArG/bOOTSl1HBgeLt27cwORQhT9Wneh9suu40v/viCoa2G0qtpXboF3Jvk4rqRPCzEWU/0eYLlB5bz1OKn+Oq6r/D28DY7JKdxXouFUmpEHc6jtdbfWj9fBowEVF3P4ezk1p4QcKr4FLf8cAseyoPU4akEeAeYHZLpatNiIbn40kgeFsIwb/c8xv42loe6P8SDPR40OxzT1TQPnzeCrLX+5hKv7RLJVQhxaQK8A3ix/4vc+8u9vJXxFkl9kswOyalILhZC1IchrYZwXfR1TF03lcGRg+kU1snskJyC3XqQhRDuJ7ZZLHd0uoOvNn/FigMrzA5HCCHcUlLvJEL8Qnhy8ZMUlxabHY5TsMW8H27Z9yaEqNqjPR9lYfZCnl78NN/e8K20WtiP5GIhBGA8PP1s32d5ZN4jfLDuAx7p+YjZITk8W02MZ6EOfW9cfP5OIYSTKW+1uOfne/jnyn/ydN+nzQ7JnUguFkIAMDhyMDe0vYGP1n/EVZFX0TW8q9khOTSbFMjS9yaEqMjS1MLdne/m0z8+ZVDkIAa2HGh2SG5BcrEQoqLE3oks27+MpIVJMk/9RdR7D7LWWvqahRDnecTyCO0btee5Jc9xouiE2eG4PMnFQojKGvo0PDNP/Xur3zM7HIcmCVQIYRe+nr680O8Fjpw+wpS1U8wORwgh3FKf5n0Y0X4EX2z6gm3HtpkdjsOSAlkIYTddw7syosMIvtz0JRuObDA7HCGEcEtjLWNp4NOA55c+T2lZqdnhOCQpkIUQdvVYr8cI8w/jiUVPUFBSYHY4Qgjhdhr5NSKpdxJrD6/lk42fmB2OQ5ICuR4ppYYrpabm5eWZHYoQDquhT0Ne7P8iWXlZfLzhY7PDES5G8rAQNfOnNn8ivnU8U9ZMYc/xPWaH43CkQK5HWutZWuvRwcHBZocihEPrF9GPa9tcy0frP2JPviRmUX8kDwtRM0opJvaeiJeHF8krks0Ox+FIgSyEMMU/ev0DLw8vJq+YbHYoQgjhlpoENGFM9zH8nv07vzlrYksAACAASURBVO/53exwHIoUyEIIUzQNbMqY7mOYv2c+C7IXmB2OEEK4pTs73Umb4DYkr0imsLTQ7HAchhTIQgjT3NnpTqIaRpG8PJmi0iKzwxFCCLfj7elNUu8k9uTv4d8b/m12OA5DCmQhhGm8Pb1J6pPE7vzdfLrxU7PDEUIIt9Q3oi/xreP5cP2H7Duxz+xwHIIUyEIIU/WL6Ed863imrpvK/hP7zQ5HCCHc0uMxjwPw+srXTY7EMUiBLIQw3fiY8QC8tvI1kyMRQgj31LxBcx7o9gBpu9JYsm+J2eGYTgpkIYTpIhpEcP/l95O2K42l+5aaHY4QQrile7rcQ6ugVryy7BWKS4vNDsdUUiALIRzCPV3vITIokleWS2IWQggz+Hj6kNg7kZ3Hd/LFpi/MDsdUUiALIRyCr6cvE3tPJCsviy83fWl2OEII4ZYGthzI4MjBTFk7hYMnD5odjmmkQL4ApVSyUkorpY4ppdKUUtFmxySEKxvYciCDWg5iytopHDp1yOxwhAOQPCyE/U2InUBpWSn/XPVPs0MxjRTIF7ZDa6201o201vFa60yzAxLC1SXGJlJSVsI/V7pvYhbnkDwshJ1FBkVy7+X3MidrDisOrDA7HFM4ZYGslBqplIqrZp9FKTVaKRVX/m7v+IQQdRfZMJL/6/p/zM6a7baJ2RlIHhbCtd3X9T5aNGjBy8teprjM/Z4LcboC2Zpop1WzLxpI1lpP1Vqna62nAgmXcEsupPwvAettvpC6xi2EqLn7Lr+PiMAIXl72MiVlJWaHIyqRPCyE6/Pz8uPx2MfZnrud6Zunmx2O3TlNgayUilZKpQDRQE41hyUAKZW2pQDJdbzsVK11qtY6HZgOzK3jeYQQteDv5c+E2Alsz93Ofzb/x+xwhJXkYSHcy5DIIfSP6M97a97jyOkjZodjV0prbXYMtaaU2gEkWBNm5e3n9KhZRxuOaa2V9fvRQNsLnD6t8nkrnEsDjbTWuReKLyYmRq9cubJmP4wQokpaa8akj2H94fXMvmU2IX7OP3ColFqltY4xO476IHlYCPewM28nN/9wMze2vZHn+j1ndjiXrKZ52MsewdiDNQGfN6qhtc5VSqGUitZaZ1pv99XkfBZgmta6V+Xz1VvQQohqKaUYHzOekbNGMm39NB6PfdzskMRFSB4WwvVEBUdx+2W389Xmr7ir8120DbnQv21dh9O0WNRAKFwwcda2/y2TCrcJrT13qXULTQhRF+0btefGtjfy9eavyc7PNjsccXGSh4VwQQndEgj0CuRfq/5ldih240oFcr3ef7Um+EzrE9ijgXjggeqOtx63Uim18vDhw/UZihBu7aEeD+GhPHhn9TtmhyIuTvKwEC4oxC+Eey+/l/nZ891mdiFXKpDrXfkT2NZX4oVu61mPidFaxzRu3NieYQrh0poFNuOuzncxO2s2G49uNDscYWeSh4VwDHd2upOmAU15Y+UbOOPza7XlcgWyTAEkhOu5t+u9hPiG8ObKN90iMTs7ycNCuB4/Lz8e7vkwG45u4Jddv5gdjs25UoFc/sR0aMWNFRK1zVdfUkoNV0pNzcvLs/WlhHArQT5BjOk+hmUHlrF432KzwxHVkzwshAsbHj2c9o3a89aqtygude3FQ1ymQC7vVeP8HrhQINcey5NqrWdprUcHBwfb+lJCuJ1bO9xKywYteWPVG5SWlZodjqiC5GEhXJunhyfjeo0j+0Q2M7bOMDscm3KZAtkqHag8t53Ful0I4cS8Pb0ZaxnLtmPbmJU5y+xwRPUkDwvhwvpH9KdPsz6krE0hvyjf7HBsxlkL5FCqflo6ERhVaVuCdbvNya09IWxrWNQwuoR14YO1H1Bc5tq395yA5GEh3JBSisdiHuNY4TG+3PSl2eHYjNMUyEqpEKVUslJqJkZSTlZKpSilRpYfY729l2g9bqRSagKQbI/betbry609IWxIKcWD3R9k74m9/JT5k9nhuB3Jw0IIgC5hXRjccjCf//E5J4tPmh2OTTjNSnrlSbcGx2UAGbaPSAhhhoEtB9IptBPT1k3j+ujr8fJwmjTm9CQPCyHKJXRP4M8//ZmvN3/N/Zffb3Y49c5pRpCdgdzaE8L2lFIkdEtgd/5u5mTNMTsc4WAkDwthH13Du9K/xf+3d2/BbVz3/cC/hyJlUjfCoCTLDmVJS8kVHd8CUo7jjO3YJmJbEamxRMjTRNOZNhXhhzSTPpT4sy//9qUOOG2fMtNCbNNOG3cKgb6IlGWNAStt49RNRML1LbQsEbQUyZEtE4JkWXfr9GEX1BLEjSSAg939fmYwJBaLPb/dBQ5+e/ac3W/iX97/F1y4ekF1OCXHBLmEeGqPqDIevf1RbLh5AwbeHeAVLWga1sNElfPsPc/izOUziHwYUR1KyTFBJiLLqRE18N/jx8TZCUSPRVWHQ0TkSPetvA8P3PoA/um9f8Kla5dUh1NSTJCJyJK8a7zQGjWE3gnhuryuOhwiIkfy3+PH5KVJvHDkBdWhlBQTZCKypBpRg557enA0dRQHjx9UHQ4RkSO1r2pH+y3t+Om7P8XlLy+rDqdkmCCXEAeHEFXWk2ufxJplaxB6JwQppepwqAqwHiaqPP+9fnx68VO8fORl1aGUDBPkEuLgEKLKWlCzALvu3oUPkh/gP0/8p+pwqAqwHiaqvK+v+jruW3Ef/uG9f8DVL+1xEycmyERkaZu1zWhe0oy/f/vv2YpMRKSAEALP3vssTn1xCkPjQ6rDKQkmyERkaXU1dfjju/8Y70++j19+/EvV4RAROdKDtz2Iu5ruwsC7A7h63fqtyEyQicjyulq6cOviW/F3b/8dW5GJiBRItyKfPH8S+xP7VYczb0yQS4iDQ4jUqFtQh+/f9X28c/od/M/v/kd1OKQQ62EidR5ufhit7lYMvDuAa9evqQ5nXpgglxAHhxCp8/SGp7Fy0UqE3gmpDoUUYj1MpI4QAv57/Dh27hgOfHRAdTjzwgSZiGxh4YKF+KO7/gijn4zi0KlDqsMhInKkR29/FBtu3oCBdwbw5fUvVYczZ0yQicg2tm/Yjqb6JoTeZisyEZEK6Zs4Jc4mED0eVR3OnDFBJiLbqK+txx/e9Yf41alf4f3P3lcdDhGRI3lv92Jd4zr89N2fWnbgNBNkIrKVbRu2YVHtIvxs7GeqQyEicqQFNQuws3UnxpJjeOvTt1SHMydMkEuIo6eJ1Fu6cCme3vA0Dnx0AKcvnFYdDlUY62Gi6tDZ0ollC5dZtrGCCXIJcfQ0UXX47sbv4svrX2LPh3tUh0IVxnqYqDo01Dag+45uvH78dXx8/mPV4cwaE2Qisp3bl92Oh5sfxp7De3D5y8uqwyEicqTf3/j7EBD49w/+XXUos8YEmYhs6Xut30PyUhKvTryqOhQiIkdatXgVHr/9cQweGcSFqxdUhzMrTJCJyJYeuPUBrHetx/Njz1t2FDURkdXtvHMnPr/yOfYl9qkOZVaYIBORLQkh8N3W7+KD5AcY/WRUdThERI5034r7cGfTnZZrrGCCTES2tUXbgsabGvH82POqQyEiciQhBHa27kTibAJvfvym6nCKxgSZiGyrobYB2zdsx8HfHsTJ8ydVh0NE5EhPrH0CTfVN+Nexf1UdStGYIJcQr79JVH2sPIqaZo/1MFH1WbhgIZ7Z+AzeOPkGJs5OqA6nKEyQS4jX3ySqPqsWr0LHmg68cOQFy42iptljPUxUnXx3+FBXU4d/G/s31aEUhQkyEdnezlZ9FPXw+LDqUIiIHGl5w3I8te4p7B3fi3NXzqkOpyAmyERke/euuBdfbfoqnv/geVyX11WHQ0TkSDtbd+LitYt46chLqkMpiAkyEdmeEALfa/0eJs5OYOTUiOpwiIgcqbWpFZ6VHkQ+jFT9Jd+YIBORI3jXeLG0bilePPqi6lCIiBxr+x3bcezcsaq/Pj0TZCJyhPraenxH+w6iH0Vx9jKvcEBEpIJ3jRdL6pbgxSPV3VjBBJmIHGPbhm24cv0KXkm8ojoUIiJHaqhtwOZ1m/HasdeqerAeE2QicozWpla0ulvx4pEXq77/GxGRXW27Yxsuf3kZryZeVR1KTkyQichRtm/YjsNnDuM3yd+oDoWIyJHudN+Jje6NeOHIC6pDyYkJMhE5ylPaU6hfUI8XP6zu/m9ERHYlhMC2DdswlhzD2OSY6nCyYoJcQrzFKVH1W7ZwGbxrvNg/sR8Xr11UHQ6VGOthImvYvG4zFtYsrNrBekyQS4i3OCWyhm0btuH81fOIHouqDoVKjPUwkTU03tQI71ovXkm8gkvXLqkOZwYmyETkOG23tGHNsjV44cPq7f9GRGR32zdsx+dXP6/KxgomyETkOEIIPL3+acQ/jeOjsx+pDoeIyJHab2nH6qWr8dLR6rv1NBNkInKkreu3YoFYwDvrEREpkh6sd+jUIRw7d0x1ONPUqg7AKa5evYoTJ07g0qXq62ejQn19PZqbm1FXV6c6FHKo5Q3L8UjzI9h7dC/+5Gt/groafhaJiCqtq6ULP3nrJ3jpyEv4UduPVIczhQlyhZw4cQJLly7F2rVrIYRQHY5SUkpMTk7ixIkTWLdunepwyMG237EdB397EP/12//C42seVx0OEZHjrFy0Eg81P4S943vxg6/9ALU11ZGasotFhVy6dAlNTU2OT44B/ZRKU1MTW9NJuQdvexArG1aymwURkULb1m/DZxc/wy9O/EJ1KFOYIFcQk+MbuC2oGtTW1GLr+q144+Qb+OSLT1SHQ0TkSA81P4QVDSuqqrGCCTIROdrW9VtxXV7H/on9qkMhInKk2ppabNG24I0TbyB5Kak6HABMkMkwODhY9LyJRAJtbW3wer3wer1IpVJljIyovNYsW4N7V9yLofEhSClVh0NE5EhbWrbgmryGAxMHVIcCgAlyQUKIbvNDdTzlEg6Hi543lUqho6MD0WgU0WgULperjJERlV+n1omjqaM4fOaw6lAoC6fUw0ROdsfNd2CjeyOGx4dVhwKAV7HISwjRCyAhpRwUQrgAvA6g+KbWHP5y+H385uNz847P7M7bluH/d3415+vxeByhUAjJZBKdnZ0YHh6G2+2G3+9HOBxGLBaDz+fDwMBAUQnv4OAgEokE3G43QqFQKVeFqOKeXPckfnzoxxgeH8ZG90bV4ZBJuephIqo+W7Qt+OuRv0bibAJao6Y0FksmyEYLQkpKGcvymgdAO4AEAA16xTpjviLKcAHok1LeDABSyhSAtnkFrtiePXswMTGBQCCAvr4+eDweAIDH40E8HkckEgGgd6HI1eWit7cXmqYhGAyiu7sbPp8PsVgMHR0dFVsPolJrvKkRjzQ/glcSr+BP2/60ai4zVM1YDxNRqX1H+w7+dvRvsW98H37o+aHSWCz3KyCE6AAwAMCX5TUNQFBK6TVNiwghElLKxCyLageQSP8IAPAAGJzDcmbI19JbTjt27IDL5UIgEEAgEEAqlUIoFIKmTT9K0zQNvb29OZfjcrnQ3a2f5dy0aRMSiXlvEiLlOls68frx1/Hmx2/ioeaHVIdT1exQDxNR9VnesBzfuO0b2JfYhx987QeoEep6AlsmQTYq3QCAUQC5hjj6AWSe7w8BCCJLRV6ABr0yjkkpU0KIEaPsllkup+pomoZIJIJYLIZQKIRgMDjt9UItyOYW40OHDqGvr6/sMROV28NfeRiNNzViODHMBDkH1sNEVG5dWhcCvwhg9JNRbFq1SVkclkmQjRYDPwAIIQI5ZuvGzIp5BEA0/UQI0YP8lWvUOBWYgH5aMGWUnxJCaEIIzeqtF/39/YhG9U1iTo69Xu9Ui3K+FmRN0+Dz+ZBKpeDxeKa6ahBZWd2COjy59km8fPRlnL9yHksWLlEdUtVhPUxE5fbo7Y9icd1iDI0PMUEuBaOvmoaMVg2jQkW6QpVS7i5ykdkqX8tez8zj8UwNpuvt7Z2RAKcT5mKkW6CJ7KarpQvhw2FEj0Xx9IanVYdjOayHiWi+Gmob4F3jRfRYFH/+9T9HQ22DkjjsdJk3NzA1iCObWQ2HNFonUkaFn67459KHjogs4u7ld2PtsrUYGh9SHYpVsR4monnr1DrxxdUvcPD4QWUx2ClBLsfFeH0A+ozTgX3I039OCNEjhBgRQoycPn26DKEQUbkJIdDZ0omRT0Zw8vxJ1eFYEethIpq39lXtuHXxrUqviWynBLnkjFOBASnlbuNvzlYLY552KWX7ihUrKhkmEZXQFm0LAGDf+D7FkRDAepjIiWpEDbZoW/Dm797Epxc+VRODklLLKH0qTlHZnUKI3WfPnlUVAhHN021LbsP9q+7HcGKYt56eI9bDRDRfXS1duC6v45XEK0rKt1OCnG5VcJsnmirqsvdZk1IOSyl7Ghsby10UEZVRZ0snjp07hrdPv606FKthPUxEJbG2cS3uWXEPhsaHlDRW2CZBNgaFJDCzD5wb+t2eOKiDiIriXeNFQ20DB+vNEuthIiqlrS1bcTR1FGPJsYqXbZsE2RCDfuclM48xncosEAjA5/Ohv79fdShE87K4bjEev/1xHPjoAC5/eVl1OFbDepiISuKJtU+grqZOyWA9qybIbmQfLR3AzBHOfmN62Vm571uuO+cVeo/Xq99NNh6PAwAikQgmJyd5+2myvK6WLnx+5XP8x2//Q3Uo1Yr1MBGVVeNNjfjW6m9h/8R+XL1+taJlWyZBFkK4hBBBIUQEeqUcFEKEhBDd6XmM03sBY75uIUQvgGClTutZue9bOBye1fyJRALJ5I17AcRisalkedOmTYjF2FhE1nb/qvuxctFKdrMwYT1MRJXW1dKF5KUkfnnylxUt1zJ30ktXukXMFwcQL39E8/Dq/wNOvVvaZa66G3jqxzlfjsfjCIVCSCaT6OzsxPDwMNxuN/x+P8LhMGKxGHw+HwYGBuByFR6AHgwGEQwGp+6oNzk5CbdbH5fjcrlw6NCh0qwXkSILahagU+vEP7//z/js4mdY3rBcdUjK2aoeJiJL+OZXvgl3vRtD40P41upvVaxcyyTIViCE6ATQuX79etWhZLVnzx5MTEwgEAigr68PHo8HgH4b6ng8PpXsJhKJnF0uent7MTg4iLa2tmmJdCrFu7+S/XS1dOEf3/tH7E/sxx989Q9Uh0NFqPZ6mIhmp66mDpvXbUb4cBhnL59F402VOTvEBLmEpJTDAIbb29t35Z0xT0tvOe3YsQMulwuBQACBQACpVAqhUAiaNv3ur5qmobe3N+dyotEokskkotEoRkZG0N/fj5aWFiQSCXg8HqRSKTQ1NZV7dYjKTnNpuKvpLgyNDzFBtoii62Eisoyuli78bOxnODBxAM9sfKYiZTJBdiBN0xCJRBCLxRAKhRAMBqe9XqgFORQKAdBbjX0+H3p7exGPxxEOh9Hd3Y1oNAq/31/29SCqhK71XfirX/0VDicP4/fcv6c6HCIix9no3ogNN2/A0PgQE2Qqn/7+fkSjUQCYlhx7vd6pFuV8LcjZeDwehMNh+Hw+aJo21X2DyOqeWvsU+g/1Y2h8CH/m/jPV4RAROY4QAl1aF/5m9G8wcXYC6xrXlb9M3kq19Nrb2+XIyMi0aWNjY2htbVUUUXXiNiGr+NHPf4T//fR/EfPFUFsz93YFIcSolDLzGsFUBtnqYSKyrtMXTqNjsAPfv+v7+KHnh3NeTrH1sGUu82YFvP4mkT11tnRi8tIk/vvj/1YdChXAepjInlYsWoFv3PYNDCeGcV1eL3t5TJBLiNffJLKnh7/yMFw3uZTczYlmh/UwkX1tbdmKU1+cwqFT5b+ULBNkIqIC6hbU4al1T+Hg8YM4d+Wc6nCIiBzp0dWPYkndkorcwIkJMhFREba2bMWV61fw2kevqQ6FiMiR6mvr8cTaJxA9FsWFqxfKWhYTZCKiItzZdCe0Ro23niYiUqizpRMXr13E68dfL2s5TJBLiINDiOxLCIGuli689elbOH7uuOpwKAfWw0T25lnpQfOSZuwd31vWcpggl5CTBof4fD54vV4EAoGpaYFAAD6fD/39/XmnEVnVFm0LBASGExysV62cVA8TOVG6seLXv/s1Tn1xqmzlMEEmAMh557xsdu/ejb6+PkSjUSQSCcTjccTjcQBAJBLB5OTk1PTMaURWdsviW/DArQ9geLwylxkiIqKZtrRsgYTEvsS+spXBBJkAAOFwuOh5Ozo6pu6U5/V6MTIyglgsBq/XCwDYtGkTYrFY1mlEVte1vgsnz59E/JO46lCIiBxp9dLV8Kz0YGh8COW64R1vNa1A8NdBfJD8oKTL3OjeiMD9gZyvx+NxhEIhJJNJdHZ2Ynh4GG63G36/H+FwGLFYDD6fDwMDA3C5XHnL0jRt6v/R0dGpZbjdbgCAy+XCoUP6NQqzTSOyssdWP4ZFtYswND6E9lW8KR4RkQpdLV34izf/Au999h7uXnF3yZfPBNlB9uzZg4mJCQQCAfT19U21Ans8HsTjcUQiEQBAIpHI2eWit7d36v94PI5kMgmPx4NQKDRj3lQqVYa1IFJrUd0ifHvtt/HasdfQ9/U+NNQ2qA6JiMhxvr3223ju189h7/heJsjVTgjRCaBz/fr1eefL19JbTjt27IDL5UIgEEAgEEAqlUIoFJrWIgzoLcTmRDib9HvTSXVLSwsSiQQ8Hg9SqRSamprQ1NQ0YxqRHXS1dOHloy/j58d/js3aZtXhkEmx9TARWdvShUvx2OrHcOCjAwhsCqBuQV1Jl88EuYSklMMAhtvb23epjiUfTdMQiUQQi8UQCoUQDAanvV5MC3IgEJjWatzR0YFwOIzu7m5Eo1H4/X4AyDqNyOrabmnDTx77CR687UHVoVAGq9TDRDR/z977LHru6Sl5cgwwQXak/v5+RKNRAJiWHHu93qkW5XwtyP39/YjFYmhrawMA9PX1obu7G+FwGD6fD5qmTXXfyDaNyOpqRA0eWf2I6jCIiBxNc2mFZ5ojUa7Rf07W3t4uR0ZGpk0bGxtDa2urooiqE7cJOY0QYlRKyZF9FZCtHiYiKrYe5mXeiIiIiIhMmCATEREREZkwQSYiIiIiMmGCXEJCiE4hxO6zZ89mfZ39vW/gtiCicihUDxMRFYMJcglJKYellD2NjY0zXquvr8fk5CQTQ+jJ8eTkJOrr61WHQkQ2k68eJiIqFi/zViHNzc04ceIETp8+rTqUqlBfX4/m5mbVYRARERHNwAS5Qurq6rBu3TrVYRARERFRAexiQURERERkwgSZiIiIiMiECTIRERERkQlvNV0GQojTAI7N4i3LAXxWpnCqEdfX3ri+ua2RUq4oZzCkYz1cFK6zM3CdpyuqHmaCXAWEECPF3BfcLri+9sb1JSty4n7kOjsD13lu2MWCiIiIiMiECTIRERERkQkT5OqwW3UAFcb1tTeuL1mRE/cj19kZuM5zwD7IREREREQmbEEmKhEhhCfjebcQokMI0ZtvmlVlWd+g8bfHNM0260tkZcV8FzO/00TVrtjfmLn8BjFBVsgpyYMTEichRAeAiOm5BwCklDEAKSGEJ9s0JcGWQOb6GnqEEOMAEsY8dlrfHuMRNE2z9QGQnRXaT3bbj8V8F3N8py2riH084zttdUWsc4fxsMU6F/sbY3y2vbNdPhNkReyUPBTBtolTmrEuCdOkZwCkjP8TADpyTLOkLOsLALuklC3Ga4BN1teoXGNSyt0ANOMHxtYHQHZWaD/ZdD8W/C7m+E5bUhH7eMZ3WkGYJVXkOvuM1z1O+VzPBxNkdWyRPBTJlolTAS4ASdPzphzT7ETLaL2wy/pquPEZTRjPbX0AZHOF9pMd96NdvovFKrQPs32nrS7vOkspY1JKv/FUk1LGKxlcmRT8XAshPKbcY1Zq5xoVzZuTKqz0EbpHStkPZ627Yxj7FkIIrx1aZNKMVqY0D4AwgDY47wDILgrtJ+5H68u7D3N8p62uqM+t0YDhz/aaTbnn+ka2IFPZSSn7jSO4JjslTgWkcOOL6QIwmWOaLRh9+bqNp5PQW2Rstb7GKcm4TVpeyFls9V0sFSd+p42GDL8QwqU6lhLI+7meT+sxwARZJUdUWE5InHII48ZpOw1ALMc0uxjBjfVpMZ7bbX07pJQB439HHQDZTKH9ZMf9mPW7aJMkKZti96H5O211BZNFU7/jBIAeWF+hz7VmDFzsAeCebb9rJsjq2C15yMUJiROMg4D29MFAukXCaDFPSSnj2aYpC3iecqzvDuP5uA3Xt8fUhaQDzjsAspNCP6q22495vouvp+fJ/E5bXMEDgizfaasrtM4dmJ5AW35AZqHPtZRyUEo5aEyb9cEgbxSikHFUk4DeYd62d7ox1jMJfT37TdNsv+5kfabLXyWh/8D4pJSxbJ9hfq6tIce+G5VStuV6nawl3z7O9Z1WF21pFFhnF4AdxqxtpgF7lAMTZCIiIiIiE3axICIiIiIyYYJMRERERGTCBJmIiIiIyIQJMhERERGRCRNkIiIiIiITJshERERERCZMkC1MCNEhhJDGY8ZdcYQQWq7XyhxXUAhR9dcPFEKEhBBnVGyjShFC9AohoqrjyMcKMRJRcYw7l8liH6b3sR6gqlKrOgAqmQAAXtC+SEKICAAPgF3GJMvfVSiHTdDvoFTNrBAjERUnBsCbMc0PoNv4m6uuZT1AVYUJsj3sBtAjhOg23VaR8uuGfvekim8vIUQQQEhKWfakXErpU1l+MeVmi5GIrElKmULG7bmFEB7odW4sV73DeoCqDbtY2EME+lF5n+pArEAIkb5ffUpB2S4AvdBbrytOVfmq15uIiGg2mCDbRxCAxzhSJyIiIqI5YoJsE1LK3dBbRIP55ss2gE4I0WMMmHBlzHdGCOERQowar48aAwM1IUTUmDYuhOjOUZbHmO+MMd+M/mXGgI5py8+Mwfg/ZCzDlbmMPOWml9ljei0IYNx4mp4na/zFLtMUa97tavR7PmO8v4eo6gAABRdJREFUHDFeO5PxnpJuD3Nc5Srf+DxETAMeo6ZW+oLl5th2xWzvM1nm82TM123EKbPFRkTVIbMemO9vUL66bA6x5a1HCpWV8Vt4xvifjVlVjgmyvTwHoKOECYALeveN56APrtCM5+lHus9YJMf7BwCEoA8gBPSEdFqlYrw3ZiwrkTmPMd8ogHbo/VfzdoswljlqLMsLIAwgJIQIGbM8hxsDSAIAWgr1Qy5imcXaZSrbD6AFQFtGOSXdHhUq35+xfDf07VVUuZlmsb3Tn88Qpn8+08tJPx80ykuvU8GDLCKqCnP6DSq2LitGoXqkUFlGspyuD3dB/91JAXhmtrFQhUkp+bDoA/qIXwmgw3juMp6HjOea8bzH9J6gvtunLafHmM+VOV962RnTek3Tuo1pngJlaObYjGlnAAQz5htNTzOVF53FNjljLiPHdkrH0l3CZRa7XXOWXabtMS2uSpQPvZ/xtDIKlJsZY9HbO6OM9DRXxmfTlS9ePvjgo7wP6OMPJAAtzzyZ9cB8foPy1mWzjD1vPVJEvTk+mzqbj+p5sAXZRqTempe+okWpWslGTP+nuyaYRyinRyS7C8SWMN7XDkyNanYB6BXTr4npwcyBXH4UwThSd0FvUTSXHYN+xD7rUdLlWGaOckq+PVSVL6WMG//OpbVmttvb/Fkcz/Ha60K/xipPaRJZz6x+g2ZZlxUjZz1SqCyjFVlDga6PVJ14mTf7CUJvueyBfkpoXmT2U/hzvfpDAkaCjBvJU1uW5SUzYij2cmTpZWabP4E5JGxlWma+ckq5PSpWvnGa8RljOfPZJrPa3jk+n1OvCSG80L8TQSPOmJQy8xqtRFSl5vAbVHRdVmz5eeqRQmW1m/4ni2GCbDNSyoQQYhD6Jd9iheavMA03Ep+pBKiECV96ORqAeMZrGqa3RKhcZr5yypUAl618od/9Kt1Kkt4eo7nfUVQcJdneRstzm3FGpQdAUAjRK6Xsn2N8RFTdSl6X5qpHYPqNzdFwUNQZVqpO7GJhT89BP+1T7CCAnAOmSsWoWDpgVCjGafisV92YR/eQdPI0rQuA0bqZHuxRyWVm265ZWxLKtD2yKWn5xinEDgABKeVuYznZfpSKbUEpxz6ElDJlJMVx6HfsIiIbKmddmlmPFCrLSJoTuDFQvWSxUPmxBdmGpJRxIUQc+sCITOOAft976MlqB/Qj4pIT+uW9ArjRupiCnryn7YJ+2a/0lQhcuHEr0ln3szVOhfmhX/HADf3qB+myB41WgHIts6jtaiwPAPzpv6ZT/iXdHnnWp2TlG2csUtBbVAB9H8/4MShQbuZ8JdmHQr80XMBYl3T3DA8y+jcTke3krcuMA/sggOdMYyayKqIeKVRv+qFf1SICvT5z48ZYCnb3qmJsQbavGUmKYQ/0o98g9Na4JpRh0JchDCBqPJIA2sz9yaR+ebU26BVKFPpl4bIebRdL6teDTvcNi0Bft4Ccx21Mi1zmbLZrP/QEOghTN4JybI8Klb8LeqUfwY31L7rcTCXch3ug98P3m5bTbyyfiGyqiLrMBf3qFO1ZFzBd3nqkUFnp7hm4cQYsfZm3cv3uUokIKWXhuYiIiIhsQggxDsBXqAWZnIstyEREROQ4TI4pHybIRERE5BjGALlSd10jm2EXCyIiIiIiE7YgExERERGZMEEmIiIiIjJhgkxEREREZMIEmYiIiIjIhAkyEREREZEJE2QiIiIiIhMmyEREREREJv8HLLg4stOxeZgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import scipy.sparse.linalg as la\n",
    "from scipy.sparse import csc_matrix, csr_matrix\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import time\n",
    "\n",
    "%matplotlib inline\n",
    "n = 150\n",
    "ex = np.ones(n);\n",
    "lp1 = sp.sparse.spdiags(np.vstack((ex,  -2*ex, ex)), [-1, 0, 1], n, n, 'csr'); \n",
    "e = sp.sparse.eye(n)\n",
    "A = sp.sparse.kron(lp1, e) + sp.sparse.kron(e, lp1)\n",
    "A = csr_matrix(A)\n",
    "rhs = np.ones(n * n)\n",
    "\n",
    "plt.figure(figsize=(10, 5))\n",
    "f, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 5))\n",
    "for restart in [5, 40, 200]:\n",
    "    hist = []\n",
    "    def callback(rk):\n",
    "        hist.append(np.linalg.norm(rk) / np.linalg.norm(rhs))\n",
    "    st = time.time()\n",
    "    sol = la.gmres(A, rhs, x0=np.zeros(n*n), maxiter=200, restart=restart, callback=callback, tol=1e-16)\n",
    "    current_time = time.time() - st\n",
    "    ax1.semilogy(np.array(hist), label='rst={}'.format(restart))\n",
    "    ax2.semilogy([current_time * i / len(hist) for i in range(len(hist))], np.array(hist), label='rst={}'.format(restart))\n",
    "    \n",
    "\n",
    "ax1.legend(loc='best')\n",
    "ax2.legend(loc='best')\n",
    "ax1.set_xlabel(\"Number of outer iterations\", fontsize=20)\n",
    "ax2.set_xlabel(\"Time, sec\", fontsize=20)\n",
    "ax1.set_ylabel(r\"$\\frac{||r_k||_2}{||rhs||_2}$\", fontsize=20)\n",
    "ax2.set_ylabel(r\"$\\frac{||r_k||_2}{||rhs||_2}$\", fontsize=20)\n",
    "plt.sca(ax1)\n",
    "plt.yticks(fontsize=20)\n",
    "plt.sca(ax2)\n",
    "plt.yticks(fontsize=20)\n",
    "f.tight_layout()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3V1wG1l2H/D/EUWB0lhSA6Px7ExsjQZap+y4XLuGuMlDqpIqD7CufdjEiYFRUklVqlIWkTzlxUWskockL5FB590m5aq8RiLWfnCl8gEoj3EyIrlOxV7vZpfY2a3xZnc+QEiaGX2ROnno21ATaACNRn8R/f9VoUg2Gt2XLYgH997T54qqgoiIKG1OJd0AIiIiLwxQRESUSgxQRESUSgxQRESUSgxQRESUSgxQRESUSgxQRESUSgxQRESUSgxQRESUSqeTbgBNdunSJb1y5UrSzSAiCs3u7u7HqvratP0YoFLuypUr2NnZSboZREShEZEf+tmPQ3xERJRKDFBERJRKDFBERJRKDFBERJRKDFAxE5F20m0gIjoJmMUXExEpAygCKCfdFiKikyDzPSgRqZrg4fVcSUTWRKTsfA16HlXtqOpW8Jb69z/+9D/h4WeP4jgVEVFkMt2DMgHnNoCax3NFAE1Vrbi2bYtIV1W7MTZzJk+ffIp/8a0GTn1L8faTHApHb+GVc38T+dfLuPxqHj+XP4ufL5zDX7HOYmV5KenmEhGNlckAZYJPA8AugN6Y3eoANoe2bQJowiOgpcULVfxT6+v43sF7+O7yh/jzc98H8H288uF/wFs/zGH5s5/DTz+7hu8/+TJeO//KIGD9XP4sfj5/Dm+9+gr+xtsFnDolSf8qRJRxoqpJtyFRIrIPoK6qHY/tFXdvSUQsAAeqKubnNQBXJxy+7XFcdV7vx+rqqs5TSeLjj/4C9799B+/9+E/w3uMf40em03Th6AX+6uEruHD4Nv7f59ew1/tFHL6wn/y9f1TC137ljcDnJCKaRER2VXV12n6Z7EFNYwJREUO9K1XtiwhEpKiq3bjmlOZx6bVfwtf+9r/B18zPP/nJn+L+X7Rw/yf/E+89/il2ct8GXvk23nxV8SW5gP6Hb+DsDz8ELv4ScLYAnM0DZy1gaTnR34OIsocBylsBsAPSmOeLACKbhzI9szUAuHz5cqjH/sIXvoyvf+HL+Lr5+cc/3sF7f7GN+z+9j/eefIifvP4ID7/zr4Cdz46/8Mx54FzeBCz3o/Dy+5WLwKklAAKIDH3FmO1DX4OSeYYkOZxJFMjKBeD1X47s8AxQ3qywDygiJZgUcxFpwmP4z2F6ZluAPcQXdlvc3nxzFb/x5ip+A8D//elH+M3/8mv4k6v/GH/nV74KPO4Djw/sx+e9l98/PgAefPDye30RZROJKK3e/lvAP/njyA7PABUTVd0DsAdgI+m2jLNy+hz0xWl8snwO+KLPjPoXL4Bnj+xA9eQB8OIIgAIK81Vn+BrUHK/N+Bws0VzOhv5Z/hgGqAlExJowzLdwRAR6dA6Pjx76f9GpU/bQ3srF6BpGRJmU+Rt1x3DmlwrujSZ5wv38wtGjV/D5LAGKiCgiDFAeTK+pi9G5qAKAfppv1J2XHp3D54cMUESUPAao8ToAhvP0S2b7QhJhD4qI0oMByu4Vec30NTBaMaJuti8sPZxxDoqIKCKZTJIwc0k3Yd/PZAFoikgFdup3CxjclNswKeH3zb7NRR7eA+we1OOjT3H44hCnT2Xy7UFEKZHJv0BmjmlqT8iVGp4JThYfoHj47CEKK4WpryEiigqH+OgYPXoFANB/kpnseiJKKQYoOsYJUAdPDxJuCRFlHQMUHaOH5wCwB0VEyWOAogEBe1BElB4MUHSMnSQB9J+yB0VEyWKAouP0DJYlh4Mn7EERUbIYoGjAWVLp7NIF9qCIKHEMUDTi3OkL7EERUeIyeaNuEsyChU5ZpQpSXJXi7NJF9qCIKHEMUPG5B+BtU0KpAGAbwLWE23SMmKXPzy1dwMGT7yfcGiLKuswHKBGpwl5CY6RKuen1rMJeeqMIoDtumXYf3nYtftgLeIxYnF26gB8+Zg+KiJKV6QAlImUAtzFatRwi4hSHrbi2bYtIN8jQ3NDKvKmuin526QI+ff4pnh89x/LSctLNIaKMymSShIgURWQTdq9oXG+mDmBzaNsmgOac512HXTU9detKvcziOw8AePDsQYKtIaKsy2QPyvSA6gAgIuN6MlWMBqgdAG3nBxFZA3B1wqmOBSJz3g0RWRORtrt3libnli4CAA6eHODS2UsJt4aIsiqTAWoas17USO/KJDhARIqq2lXVLZ/HKwKoquqG2XQXwKZznFAbH4IV04NiJh8RJSmTQ3w+FICReSO34ozHKwJ4dejn/rjgZHpYOyKy89FHH814qvmdW7oAALwXiogSxR6UN68l4ANT1Y6IWGZIELDvg3pnwv5bALYAYHV1VcNsyyRmCgpnTYBiD4qIksQAFRNnKXnD19BgUlZOsQdFRMnjEN8EZi4qc06fWsYry6+wB0VEiWKA8ubMDRXcG10BK3WJDaGQl99aOYtrQhFRohigPJjkiC5G56IKmJDcsEjyuTxX1SWiRDFAjdeBXebIrWS2LzQFYK2wB0VEyWKAellhfFgDoyWQUl2iaF7iGuNjD4qIkpbJLD4zl3QT9v1IFoCmiFRgV35oAYObchsi0gRw3+yb2iUywsYeFBElLZMByswxTe0JqeoegL3oW5Qyqsjn8nh8+BhPDp9g5fRK0i0iogziEB8NiDuLb8Ue9WSqORElhQGKPOVzeQAMUESUHAYoGqGw74MCWE2CiJLDAEWe8ivsQRFRshigaMA1BTXoQTFAEVFSGKBohCpwMWcvWsh7oYgoKQxQ5On0qdM4f+Y874UiosQwQNGAuPPMwWoSRJQsBqiYiEhTRFREDkSkbZaBTzVWkyCiJGWykkRC9lVVpu+WPFV7Ed98Lo+ffv7ThFtDRFmV+QAlIlXYS2iMVCkXkRLsiuZd2LX4ul77LYrh6GnlLHyn951E2kJElOkAJSJlALcxWrUcZgiuqaoV17ZtEekGLBhrOcEQQAXALVMTMLXyK3n0n/ahqiPzU0REUctkgDLBpwFgF0BvzG51AJtD2zYBNOER0HzYcgKSiPQA3ANwLcBxIqfmq5Wz8PToKR4fPsa55XOJtomIsieTAcr0gOoAICLjqppXMRqgdgC0nR9EZA3A1QmnajtDgu7ekqruiUhJRKw09aKGO0nuahIMUEQUt0wGqGnMelFFDPWuzBpREJGiqnZVdcvn8UoAbqvqsR5TmoKTl0E9vqcHePNn3ky4NUSUNUwz91YAJgaQWVPEu3D1xszcVytY06Jnkvhe9qB4LxQRJYA9KG9eS8AHZnpeXTMkCNjDgjfG7W/2WwOAy5cvh9mUmbh7UEREcWOAisks6elm6HALAFZXV3XK7qERjFaSANiDIqJkcIhvAjMXlVnnz5zHKTnFHhQRJYIByptzn1PBvdEVsILcB3ViOF22pVNLuHjmIntQRJQIBigPJjmii9G5qALsqhOLGaA87sW1ViyuCUVEiWCAGq8Du8yRW8lsz4x8Ls8ARUSJYICye0Vec00NjFaMqJvtC80pFgvYCxdyDoqIkpDJLD4zl3QT9v1MFoCmiFRgV35oAYPU8IaINAHcN/s2F3Z4D6OVJAD7Xqg/+/jP4m8MEWVeJgOUmWOa2hNS1T0Ae9G3KL2snL0mFAvGElHcOMRHE+VzeRy+OMRnzz9LuilElDEMUDSRtcJqEkSUDAYoGvAawGM1CSJKCgMUjXAl8bEHRUSJYYCiiQY9KN4LRUQxizRAicgLETmM8hwUHq8svUEP6gl7UEQUrzh6UMxNPsHOL5/HkiyxB0VEsUvlEJ+IXEi6DVmmeDkJJSL2vVDsQRFRzFIVoETkbRH5PoC+iHwiIn/P9dxvisj3EmzeXETkQER06LGedLvcxnV18yusx0dE8UtVgALwO7Dr3eUBXAfwD0Xk7wOAqn4T9kq0J44prVRTVXEeABqqupF02/xgD4qIkpC2ALWjqvdU9YGqdlT1XQCvisivmedDX11WRKoiUh7zXElE1kSk7HwNeh73iroiUgXQCnqsqOnQVWYPioiSkLZafCN/BVX1toi8IyK/GvbJTMC5jdGq5RARpzhsxbVtW0S6sxaMNbX/nGNYAAppLDo7rtSeleOaUEQUv9T1oETkhoh8T0SuOBtV9Z759kEYJxGRoohswq5Q3huzWx3A5tC2TQDNOU9/E8DdOY8RKytn4cHTB3ihL5JuChFlSOAelIhcUNWHYTZGVb8lIl3YQ33vezx3LaTzdGEHIIjIuKrmVYwGqB0AbecHEVnD5HmxtntozyiraqrXlBoeR82v5HGkR3j07BEu5i4m0iYiyp6ZA5SIvA37j3RRRA4A/Jaq/pF57jcB/I6q/kLQBqnqAwDfGvPcD4IedxZmGG6kd2XWiIKIFFW1q6pbMx63PHzMk8DK2Tfr9p/2GaCIKDZBelBOpt0OgK8AWBMRUdU/VNVvisj2uBeGeX9T2L23IQVzjnETL0UAQeaQSvCYZ0sLGZNo7gSogycHeOvCW3E2iYgyLEiA2nHNCXUAdMy80a+p6n8f9yIReQf2IoGK+atL7AP453MeYxKvJeDDMjWwmaHDNQC4fPlyhE3x5pXFB7AeHxHFK0iAmpZp55kKboLaPa/nssLvfU9m6HALAFZXV0NPrZ+VuwdFRBSXIFl8sWTapYGZi8qMcWnm7EERURJm7kHFlWmXMGcYrgBXj9EVsFJ3D1OUzp0+h+VTy1wTiohiFSjNPGim3UlJkjDZel2MzkUVAPTTeJNtmHRolFZEkM/luaouEcVqJECFGESODRidsCQJwE4AWQWw59pWMtszx1qx2IMiolgdC1AhBhEFcKwnleIkiQK8s/YaALZhkhWMunlkDntQRBS3YwEq7CBi0qVTx8wl3YR9P5MFoCkiFdiVH1rAYJivISJNAPfNvs1FH94DRtPMAbsH9d3ed+NvDBFlVtqKxcbC3IA7tdyQqu7h+BDfQhuXxQeYJTc4xEdEMUq8WKxZiPATJ2VdRK6IyG8760BROuRX8nj49CEOXxwm3RQiyojYkiQmKMAuoPq+iFwEsAs7jfueiNRV9ddDag/NwcpZUCgePnuIwkoh6eYQUQbEliQxaV9VdVLW34U9J/SOqj4UkYsi8tuq+u/naAuFIJ8zN+s+6TNAEVEs0pAkkXd9XwOw59zjpKoPRGRhKlOk3bhisYCdJAGwmgQRxScNSRItEdmBfW9TGaZIqkviteiyRj3S+JweFBMliCguiSdJmMoT78C+Abaiqn8gIm+LyC2zS378qykug3p8vBeKiGKShh6UUzrp9tDmmogUAdzyeAlFYFKaubNQIXtQRBSXuQKUiNwF8B9V9Q/Nz+8AuD9vnTzTq/riPMegcJ09fRYrSyvsQRFRbOYd4tt0ghMwSLJYnfOYAAAR+XIYx0kLESmJyLqIVEWkmealPLwqSQCsx0dE8Zo3QHVdN9j+NxG5D6Ayb6OMV0XklnvNqZPKBKNtVd0wpZQ2ATQTbtaIafcV5HN5ZvERUWxmDlBmocLfM0u8/wCAZao+bMK+4fbmjMd7R0R6pprEfzVVJK6o6j1zrPKsbZzx/FUR8TyH6fWsiUjZ+RrwNGW41pAy9fxSWadwEitncYiPiGITZA7qm7CLp74rIpuwb8h9G0DNJDvMqg77/icLwFcA/AMAGyKyCzuzrwjgDwIcdyoTcG6b8w8/5xSHrbi2bYtIN0DB2D7sihkj50hj8dlxef3WioUPPv0g1rYQUXYFWVH3G+bbbwKAiLwNu4fwL0WkBLsi+CxrNbVdy8U7x7wIu6rEVQDfGPfCoEzwacAuq9Qbs1sddq/QzRmaGwlok6hqR0QgIpapku70xFI1DyWT0vjAJTeIKF5zp5mbYb7b5uEELE8i8luws/z+95RjeqWdh8b0WuqmTeOqmlcxGqB2ALSdH0yljKsTTtVW1Y455zUznNjFy+G+1PWeJrFWLDx6/gjPXzzH8qnlpJtDRAvOd4Ay80LvT9tv0pLvsBf/U/NJvQP7j/1e2urtmaSGIoZ6V6b3MxiaU9Ut7yN4c9aaMj24rln2I3XGZfE51SQePH2AS2cvxdgiIsoiX0kSIvL7APZDyKjrwJ6L+XUA34I939SBPef0iYj8OxH50pznCEMBGKwb5aU46wFFxJ2fXYeP9ajSxqnHd/CEqeZEFD2/Pahd2GnS7895vroZvuuYB4BBskIFwFcBfENEFPZCgR0Ad1T1T+c876yimBtqiEgVdvC77/SmvJihwzUAuHz5cgRNGXPeKc8PKpoz1ZyIYuB7iM+VyBDYuOE/M08zHLB+FXbQqgL4hXnPnbRZhgPNvlsAsLq6mppiuVaOPSgiio+vIT5VvW3ufYpl+E1VO6r6u6r6VVVNLDiludpDlHRMovmgYCx7UEQUg1nmoOqwExq+JyJ3ROS3FqHKwxhOdt2xe5dcAetEZd/5NSXLnD0oIoqV7zkoVT1lUshLsIfevgFgy0z+353x3qdj/GYIxsVk63UxOhdVANBP4821cTizdAavLL/CHhQl6unTp+j1enj06BGOjo6Sbg4ZZ86cwaVLl3Dx4sXQjuk3QPXMPUx3VfWbOH5DbQV2JYlATO/shohcTVOQgj0ntgo7WcNRgmuubFGNSzMHTLkjBihKyNOnT/GjH/0I+XweV65cwfLy8tQbzCl6qorHjx/jgw8+QC6Xw8rKSijH9TsH9U1V/QPYpYjc2x+oaktVf3eONuwC+GqCwakA76y9BkYrRpzI9HC//PxHz+fyrGhOien1esjn87h06RLOnDnD4JQSIoJz587h0qVL+Oijj0I77kyVJMLI5IvzuOOYuaSbsO9nsgA0RaQCu/JDy7SpLyINEWnCrj3o1ObL5PCew1qxOAdFiXn06BGuXLmSdDNojPPnz+OTTz4J7XgjAUpELoR0bF8fbZwMQQC/P60EUljMDbhTe0KquofjQ3yZMCmvPZ/L4wcPJhULIYrO0dERlpdZZiutTp8+jcPDw/CO5/7BrIjbgP03ap6+s8Kucj6VmYNaA7BmEhP2YJdA6qRsTopgL/3OHhQlicN66RX2v82xAGWG2kIbbjMVEaaJNEOQwpVfyePzw8/x9Ogpcku5pJtDRAts3hV1w+BkCH5ikjH+map+EUAedlJCpud8EjEhjc+5F4rLbhBR1OZebmNeJm3dGV6859r+AMDYenWUDHc1iddfeT3h1hDRIhvpQYnIhZAeM92tFXcmH3nzXU2CqeZEC69er6PfT260JNYkiRAzBKGqD8M6Fvk3qGjOIT6ihdXtdrG5uYmtrS00m83E2hFbkkSIwQ8A9gEwcSIik9LMB2tCsQdFtJA2NjbQaDRQLM687F3oYpuDCjv4UTSmfXK4mLNHbtmDIkqPTqeDSqUCnVSnzKf19XWsr68PjpmkNGTxLRwRaQd57iRYPrWM82fOswdFRJFLPItvkZiFFosAyrM8lzbTPoTlc3n2oCh1/u0f/zm+/eN0TU3/tTcv4F9//Zdnfl2tVkOhUECtVkOj0cD169exvr6OjY0N3Lp1C/1+H5Zl4ebNm2i32+h07BrWzo2y29vbqFarof4uSYg1QCWVJGGWWu+blXuHnyvBrlrehR1Aul77+WxTxxxzc5bn0sTPneDWisUeFFHEOp0O7t69i2aziXfffRedTgeNRgO7u7uwLAvdbhf9fh/tdhutVgu1Wi2UIb40iS1AJZUkYXoutzFamRwi4hSArbi2bYtIN+tFYSfJ5/L48PMPk24G0TFBeipp1u12sb+/P0hWcHpNxWJx8HWaer2OXq83db9CoYDNzfR9dl7YJAkTfBqwl/MY9y9UBzD8r7IJoAmPgJYV45Z8d1g5C989+G5MrSHKplKpdCwIlctlFAoF5PN5lMtlVCoVrK+vTzxGGoPOLBZ2Dsr0gOoAICLjKpdXMRqgdmAXq4V57RqAqxNO1Q46JJhGfrq2+RXOQRFFbbiHZFkW9vf30Wq1cOfOHTQaDdy/fx/b29sJtTB6kQYoVU1tlqBZE6qIod6VWQcKIlJU1a6qbiXTwvSychaeHD3B48PHOHv6bNLNIcqUarWKarU6mHcC7ODlhUN8J1cBGKwN5aWIjBaqnZrFt/KymsTZn2GAIopDq9VCu90eZPjduXNn0MtyvrZarUGwKpfLcwUdp8RRr9cbGwCjltoeTgxCv+IiUhKRdfN90yRoTH3O4zhrIrIjIjthLp8cFtbjI4pfqVRCr9dDrVbDtWvX0O12B8N7xWIRpVJpkJY+j62tLYjIoHd29epViAj29uJfuzXLPajQuVbg3ZjlOY99twBsAcDq6mqseaN+1hvjkhtE0fKaVyoWixPnm3Z3d0M599raGtbW/CzlF70s96AADOaiaAasx0dEcchygHLmlwruja6Alcn5J2BysVjAVdH8KXtQRBSdzAYokxzRxehcVAF21YlMBijxkWh+4cwFCAQHT9iDIqLoZDZAGR3YZY7cSmY7jbF0agkXcxfZgyKiSGUlQBXgnbXXwGjFiLrZnll+ynlZOYs9KCKK1MJm8Zm5pJuw72eyADRFpAK78kMLGNyU2xCRJoD7Zt9mVof3APiukphfybMHRUSRWtgAZeaYpvaEXOnfNAMrZ+GDTz9IuhlEtMCyMsRHM5hWLBZgPT4iih4DFB3jdx0UK2evCbVo688QUXowQFEg+Vwehy8O8dnzz5JuChEtKAYoGuUni89Uk2CiBBFFhQGKAmE1CSKKGgMUHeOnWCzgqsfHe6GIKCIMUBQIe1BEi6teryOfzyOfz6NeryfWjoW9D4qC85OXxx4U0WJy1oG6d+8eut0ubty4gW63i3a7HXtbGKAiICJtVa0MbSvhZcmlClJascJPsVgAOL98HkuyxB4U0QLp9/totVqD20dKpRIsy0KlUkGn00G5PHad1UhwiC9EIlIWkTUAXv+K9wDsmDJLuwDGrzx2AojI4F4oIkpWp9OB+J1AnmJ9ff3Yz05Q4oq6ERGRKuwlNEaqlJuezSrspTeKALpe+/nhvE5ENj2eftuUXwKAXpDjx8XvzbdWzmI1CaIFYlkWms2m5/YkLHyAEpEygNsYrVoOEXGKw1Zc27ZFpBv28JsrOAEprpg+y4cwa4U9KEqR//wN4Cf/J+lWHPeFXwG+9jszv6xWq6FQKKBWq6HRaOD69etYX1/HxsYGbt26hX6/D8uycPPmTbTbbXQ69mdqpxe1vb2NarUayq/Q7XbR7/djH94DFjhAmeDTgD2cNq7HUgcw3NvZBNCER0ALqU1V2BXVT/yaU/lcHj948IOkm0G0kDqdDu7evYtms4l3330XnU4HjUYDu7u7sCxrEDja7TZarRZqtVokpccajQbK5TJKpVLox55mYQOU6QHVAUBExvVWqhgNUDsABukqZk7p6oRT+Q42pk0bIrLmlUiRFn7f49aKhYMP2YOilAjQU0mzbreL/f19FItFABj0morF4uDrNPV6Hb3e9BmFQqGAzc3RmYmNjQ3s7e1hd3d39l8gBAsboKYx60UVMdS7MmtEQUSKqtpV1a0QzlUEUFXVDbPpLoBN5xzzHj9Ms0yz5nN5PHj6AC/0BU4J822IwlQqlY4FoXK5jEKhgHw+j3K5jEqlMpLQMMwr6PjV6XRw69atQY8tCVn+q1IARuaG3KZ/PPGvCODVoZ/7aQtOs7JyFo70CI+ePUq6KUQLZ7iHZFkW9vf3sb29Dcuy0Gg0Bvcsha3b7aJWq2F3d9dXTy0qme1BwXsJ+LmYjMCy+b4JM/ynqh0RscxwIWDfB/XOhOOsAVgDgMuXL4fdzNDkV15Wk7iYu5hwa4iyoVqtolqtDuadgPFZdkGG+Pr9PiqVCu7du5docAKyHaBC51qdd8PjuZbrx4nDhmZYcQsAVldXY19wye8JrdzLahJvXXgrugYREVqtFtrt9iDD786dO4MA4nxttVqDYFUulwMN8V27dm2QAei+98nvvFeYsjzEB2AwF0XGLDf7uXtQRBStUqmEXq+HWq2Ga9euodvtYnvbvt+/WCyiVCoN0tKD6na76Ha72NjYwLVr1449vO6PilqWe1DO/E8BwOAvrCtgnej5oTg4PSgGKKJwOYHHrVgsem53hJFpVywWU7VKdmZ7UCY5oovRuagCFiCBYR5+35+DHhSrSRBRBDIboIwO7DJHbiWzPZNmSTM/d/oclk8ts5oEEUUiKwHKqSI+rIHRihGpLUOUNiKCfC7PIT4iisTCzkGZuaSbsO85sgA0RaQCO/W7BQxuym2YlPD7Zt9ULoMRJ/Wdx2eqSXBNKCKKwMIGKDPHNLUn5EoNJ2C2MT6APSgiikxWhvgoIuxBEVFUGKBoxCxZplbOYg+KiCLBAEXHzLomp5Wz8ODpAxy9OIqkPUSUXQxQNJf8Sh4KxcNnD5NuChEtGAYomsugHh/vhSKikDFA0VzyOVaTIKJoMEDRMbMUiwXsLD6APSgiCh8DFM2FPSgiigoDVAREpO2xrSkiKiIHItI2y8Cn0izVjNmDIlo8jUYD+Xwe+XwetVoN/X4yH0AXtpJEEkSkDLtcUtnj6X1VnTWLO3YzjvDh7OmzWFlaYQ+KaEFUKhX0ej3cu3cPlmWhXq/jnXfeCWU5j1llIkCJSBX2EhojVcrNMu2rsJfeKALoeu3nh/M6EZl9GcsTzFqx2IMiWhCWZWF7e3uwMm+z2RwskMgVdUNmejW3xzznFIfdUtWOWWq9HtHwmyUiVREpm+G+1K7kO+tyZflcHg+ePoikLUQ0XafTmTnBaRx3cALsVXaTsrA9KBNkGgB2AfTG7FYHMNzb2QTQxOgyHPPaMgVsISI9APcAXAv5HHML8ha3cuxBES2qW7duoVQqxd57AhY4QJklM+oAICLjqppXMRqgdgAMkhxEZA3A1QmnavsZEnSCk/l+T0RKImK5t59U1oqFv/z4L5NuBmVc870mvtP7TtLNOOYXC7+Ixl+ffXm5Wq2GQqGAWq2GRqOB69evY319HRsbG7h16xb6/T4sy8LNmzfRbrfR6dh/gpxe1Paebzr9AAAO7UlEQVT2NqrVauB29/t93LhxA51OB4VCIZH5J2CBA9Q0ZoitiKHelVkjCiJSVNWuGfab91wlALdV9ViPKa3BaZZisYA9xMceFFG4Op0O7t69i2aziXfffRedTgeNRgO7u7uwLAvdbhf9fh/tdhutVgu1Wm2mDNxpCoUCVldX0e120el05gp4QWU2QMFeZXdSkCjCTpwIQxeunpqZF2uFdOxQBRnHtlYsPHr2CM9fPMfyqeUIWkU0XZCeSpp1u13s7+8PhtacXlOxWBx8naZer6PXGzfD8VKhUMDm5svBJMuyBj87wW93dxelUingbxNMlgNU6EkKpqdUNt83YYb/TK+sa4YLAXvI8MaE46wBWAOAy5cvh93M0Dk36z54+gCXzl5KuDVEi2F43qdcLqNQKCCfz6NcLqNSqWB9fX3iMdxBJ6hqtToIWGEcbxYLn8UXJ1XdU9UNVRVVbbjnppwsQfNoTBreM/usqurqa6+9Fk/j5+DcrMt7oYjCM9xDsiwL+/v7gyy7RqOBWi3sXC5vhULBV08sbFnuQQGw56LSOheUFJ0x0dzpQXEeiih61WoV1Wp1MPQG4FhauNusQ3x7e3vodrvH5pv6/T663S7q9Xo4v8AMshygnPmlAoBBgHLdn5Rc8n+CgqaZA+DKukQRarVaaLfbgwy/O3fuDHpZztdWqzUIVuVyeeYhuWKxiFqtduw8N27cQLFYnDqcGIXMDvGZXlMXo3NRBdhVJzIZoIIYrAn1hD0ooqiUSiX0ej3UarVBZYft7W0AdmAplUqDtPSgLMsapJQ7JY6KxSLTzBPSgV3maM+1rWS2Z9asmaqDOSj2oIhC4QQet2Kx6LndEVYQcWfwJS0rPagCvLP2GhitGFE32zMpSLWU3FIO506fYw+KiEK1sD0oM5d0E/b9TBaApohUYKd+t4DBTbkNkxJ+3+zb5PDe7PIrefagiChUCxugzBzT1J6Qqu7h+BBf5gW5F531+IgobFkZ4iPfglVEtlYs3gdFRKFigKJQ5HMc4iOicDFA0Ygg9SatnMUARUShYoCiY4KueZZfyeOz55/h2dGzcBtENCTMit0UrrD/bRigKBSsJkFxWFpawvPnz5NuBo1xeHiI06fDy71jgKJQ5FdMPT7eC0UROn/+PB4+fJh0M2iMR48eYWVlJbTjMUCRh9m76T977mfxReuLONKjCNpDZCsUCjg4OMDHH3+MZ8+ecbgvJVQVn3/+OT7++GOEuQLDwt4HRcEEnILCl177Ev7o7/5RqG0hGpbL5XD58mX0ej28//77ODriB6K0yOVyeP3110PtQTFARUBE2qpaGdp2gNFySw1V3YivZUQnXy6XwxtvvIE33ngj6aZQxBigQmSWci/CrKrr2m4BqLkXMBSR9bQGJ46aEFEaZCJAiUgV9hIaI1XKzTLtq7CX3igC6Hrt54fzOhEZKQU8FJyqAFpBzhG1oGnmRERhW/gAZXo1tzFatRwi4hSHrbi2bYtIN8yCse4Ve01vqsCCtEREky1sFp+IFE1Ppghg3JrHdQDDvZ1NAM0Im3YTwN0Ijz83DvERURosbA/K9FDqACAi46qaVzEaoHYAtJ0fRGQNwNUJp2rPOCRYVtXUrjclgfP4iIjCtbABahoz1DbSuzJrREFEiqraVdWtEM9ZHj4fERF5W9ghPh8KwPH5oSHFCM5ZApD6WkAaaEUoIqJwZbYHBe8l4OdiMgLL5vsmvIf/piZHmGHFNfPjpyLy3QDNuQTg4wCvw/8C8LtBXjhd4DZFiG3yh23yL43tSlub3vKzk2ShVIiI7AOoD6V6lwDsqurIpIuIKIBK0HTzNBCRHVVdTbodbmyTP2yTP2lsE5DOdqWxTX5keYgPwGAuioiIUibLAcoZaiu4N7oCFu9TIiJKUGYDlEmO6GJ0LqoAu+rESQ9QoWUfhoht8odt8ieNbQLS2a40tmmqzM5Bme2bsOehtlzbqgCuq+pI5QkiIopPVnpQBXhn7TUwWgKpbrYTEVGCFrYHZeaSbsK+n6kKezivAzv1u+XarwTgOoD7Zt+9k5y9R0S0KBY2QBER0cmW5Rt1T6Sgy4OEuayIx7Gr5phXzddNdy91zGss2Dcjt2CXfyrAHl6dtbZhqMeO6jo5N24D2JlQvcTrdaFfpyiWn5n3uk1pUyLvr3FtSvL9NaFNqXl/hUpV+TghD9hv6PbQtm0AxShe57NNVQAl188WgH0Aaz7apK7HAYBqiNdp5mNHfJ3aQ21yP/bjuk6wK50cwC5anIr315Q2JfL+8nGdYn9/TWlTKt5fYT+ykiSxKIIuDxLlsiJFVd1zflD701vT43xeKgDyAK6qal6nfCqeUZBjR3mduqZN12D3BJxHHR5rlQ2Z+zpFvPxMoNf5bFOs7y+fbQp0bER7nRJ9f0Um6QjJh/8H7E+OxaFtlv3PGP7rfLTHArALwBra7nwqG/vJ0Owz8kkwpOsU6NhRXSdzHM9P/OO2R3mdzO/p9Sk8sfeXV5uSfn9NuE6Jvb8mtCk1768wH+xBnRCTlgcxz3tWXw/6Oj/MMYqIpvJ7rKK8TuY4IzdKisia1/Yk8P0Vray/v4JiksTJ4Wd5EK/qF0Ff54uq5j02l+GvGkfRTPoCdjt7Gt7wwqzHjvQ6DTNrg+343D3K6+Tg+2s2fH/FgAHq5Aha1DaJYrh1ALem7NMDAD1+T9q2iCCE/xxBjh33dSqp6oaP/aK8Tm58f/nH91dMOMRHoTJrWfWm/edQ1b7H8EMoCQlRHjsM5tOqr0/Laf9d4sb313SL9P5igDphgi4PEseyImYcva6qlYCH6MIeboiirb6OHdPyKzdhVzUJKrLrxPdXYHx/RYAB6uQIujxInMuKNAG842dHEVn32OxMIM81YRzw2LFcJ3O80oS5iOH9I7tOQ/j+8onvr/gwQJ0QGnB5kKCvm5W5T6Ph5z+G+STc9Mhccv7zBm5T0GPHdZ1gJvj97BjldRrG95fvdvD9FSMGqJOlA7tMilsJ07vzQV/ni5kXaLr/k4lIeVzqrNmv7vGfsgy7WK/vUi0hHzvS62R8Bf7nByK7TmPw/TUF318xS/pGLD78P2B/+houldKG6+Y/eJSC8fO6OdpUhV3Lq+R6lGHXS5vUpqpHu3fhKmszZ5smHjvu6+Q63vbwOeK+ThhTzibJ99eENiX2/prSpkTeX+PalKb3V5gPppmfIKraF5GGKQzpLA9y7JOlUQj4upmY8e7tMU9Pa1NLRKom4+hV2P8xavO2acZjx3KdhvQwuYROJNdJji8/Y8Ee2qnAtfxM3O+vaW1K4v3l8zrF+v7y0yaXRN5fUeFyG0RElEqcgyIiolRigCIiolRigCIiolRigCIiolRigCIiolRigCIiolRigCIiolRigCIiolRigKJYmNppah5rHs8Xxz0XcbuaIpL6u9VFZFNEDpK4RnERkXURaSfdjklOQhsXCUsdURIaAIYXSaMxRGQbdg26G2ZTKsrQROArsOvspdlJaOPCYA+K4rYFezG0atINOUGqsJeaaJnHXlwnNj3MWNYFUtWaqkpS5/dzXq82UnQYoChu27B7ADeTbshJ4PojGfvSB6ZI6Trs3lvskjp/0r83vcQARUloAiiJCP8AENFYDFAUO1Xdgt0jaE7azyuBQUTWTKKANbTfgYiURGTXPL/rLGonIm2zbX/c0KJ5bdscZ19ERuYZzLIEx44/3Abz/aY5xvDqqZPO6xxzzfVcE/b6PQDg7DN1aHTSMV1tnXhdzbzXgXl62zx3MPSaUK+Hu11Rnd+8H7ZdCSdt91DetPOOuXZ+rveBx36lof2qpp3q1bYsYoCipNwCMHZV1ACctYNuAajDXjtn2/Womf3GrS90G8Am7AQOwA4I7j9cVfPajjlWd3gfs98u7FVTN3XKiqTmmLvmWBUAdwBsir28OczvUjHfNwBc9Vj/Z9Zj+nXDde46gKsArg2dJ9TrEdP560PHL8C+Xr7OO2yG6+28Pzdx/P3pHMf5uWXO5/xOUz/kLLSkV0zkIxsP2JlPCqBsfrbMz5vm56L52b3aZ9N+ix47zprZzxrezzn20LZ117aq2Vaaco6iu21m2wHsxeXc++0621zn81zNdMw1OXCfY8x1ctoydhXVAMf0e13Hnjui63GsXXGcH/Y807FzTDnvcBt9X++hczjbrKH3pjWpvVl7sAdFiVD70+wWgDU/Q2E+7bi+d4bGOq5tTnr2sVVFPdrWNa9bBewhHNgBdd01/KJ4uQS5Wx0+mOEoC/Ynave5O7CHP2ter4v7mGPOE/r1SOr8+jIjcuaefIDr7X4v7o957p7Y91pxfha8D4qS1YT9yX0N9tDGXNR7CClo9lsXJkDh5R+vax7HO7a8tvpfKts5ptf+XQT4gxnRMSedJ8zrEdv5zbDcdXOcea7JTNd7zPtz8JzYy7g3zQMi0lHVyrjXZAEDFCVGVbsi0oKdct6Ztn/Minj5h2fwByjEP7jOcYoAhu9rKuJ4bzDJY046T1QBKLLzi10Fogg7CDjXY3f8K3y1I5TrbXpe18yIwhqApoisq+pGwPadeBzio6Tdgj1Mct3n/mMnrMNi/kCUYYKmGQbyzDqcY3jS+eN1bAjKfLp3JtTjPKbXde15bIvqengJ9fwmEaEM+6bnLXMcrwDneV4PUfwbQlX7Jijtwa5ckVnsQVGiVHVPRPZg3xg5bB+w65/BDhZl2J8sQ2fSixt4+em6Dzt4Om7ATjt2MrEs2H+Yuggwz2KGdOqwM74KsLO/nHO3zKfpqI7p67qa4wFA3fnqGnIK9XpM+H1CO7/psfdh90wA+9+44bHfpPMO7xfKv6FJTW+Y38UZHixhaH4rc5LO0uAjGw8MZTaNeW44i8+CPfyisP+oOnNWnll8Q8d09iu6tpWG2wBXhpU5hwJou1839Pq22ecA5g/juDbMcF3cv+P60PMzZfH5PKav6zp0ffYxmjUX6vUY8+8Y6vnNv/OBec2u6/cevkae5x3TxmnXe2rWpPk3ce578/x9s/gQc3GIiIhShXNQRESUSgxQRESUSgxQRESUSgxQRESUSgxQRESUSgxQRESUSgxQRESUSgxQRESUSgxQRESUSv8fu0bFe/rbZAEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import scipy.sparse.linalg as la\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# Example from http://www.caam.rice.edu/~embree/39961.pdf\n",
    "\n",
    "A = np.array([[1, 1, 1],\n",
    "              [0, 1, 3],\n",
    "              [0, 0, 1]]\n",
    "            )\n",
    "rhs = np.array([2, -4, 1])\n",
    "x0 = np.zeros(3)\n",
    "\n",
    "for restart in [1, 2, 3]:\n",
    "    hist = []\n",
    "    def callback(rk):\n",
    "        hist.append(np.linalg.norm(rk)/np.linalg.norm(rhs))\n",
    "    _ = la.gmres(A, rhs, x0=x0, maxiter=20, restart=restart, callback=callback)\n",
    "    plt.semilogy(np.array(hist), label='rst={}'.format(restart))\n",
    "plt.legend(fontsize=18)\n",
    "plt.xlabel(\"Number of outer iterations\", fontsize=20)\n",
    "plt.ylabel(r\"$\\frac{||r_k||_2}{||rhs||_2}$\", fontsize=28)\n",
    "plt.xticks(fontsize=20)\n",
    "plt.yticks(fontsize=20)\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Анонс следующей лекции (25 ноября)\n",
    "\n",
    "- Продолжение знакомства с итерационными методами\n",
    "    - BiCG\n",
    "    - Minres\n",
    "    - и другие\n",
    "- Предобусловливатели\n",
    "- Итерационные методы для частичной задачи на собственные значения \n",
    "    - PINVIT\n",
    "    - LOPCG\n",
    "    - LOBPCG\n",
    "    - Jacobi-Davidson"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "from IPython.core.display import HTML\n",
    "def css_styling():\n",
    "    styles = open(\"./styles/custom.css\", \"r\").read()\n",
    "    return HTML(styles)\n",
    "css_styling()"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}