1A Foundations of Computer Science.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This course has two aims. The first is to teach programming. The second is to\n",
    "present some fundamental principles of computer science, especially algorithm\n",
    "design. Most students will have some programming experience already, but there\n",
    "are few people whose programming cannot be improved through greater knowledge\n",
    "of basic principles. Please bear this point in mind if you have extensive\n",
    "experience and find parts of the course rather slow.\n",
    "\n",
    "The programming in this course is based on the language [OCaml](https://ocaml.org)\n",
    "and mostly concerns the functional programming style. Functional programs tend\n",
    "to be shorter and easier to understand than their counterparts in conventional\n",
    "languages such as C. In the space of a few weeks, we shall cover many\n",
    "fundamental data structures and learn basic methods for estimating efficiency.\n",
    "\n",
    "The first thing you will notice about this course is that there is an\n",
    "_interactive_ version hosted online at <https://hub.cl.cam.ac.uk/>, where you\n",
    "can login with your Cambridge Raven identity and edit the code fragments in\n",
    "your browser.  You are encouraged to do so -- such edits will only persist in\n",
    "your session, and will help you to explore the world of functional programming.\n",
    "If you are using the web-based version, then you need to know a few concepts:\n",
    "\n",
    "- The notebook consists of a sequence of textual and code snippets.\n",
    "- The code snippets can be executed individually, and will “remember” the\n",
    "  results of the previous snippets.\n",
    "- To begin with, click on `Cell` / `Run All` in the menu to execute the\n",
    "  entire notebook.\n",
    "- You can later double click on any cell and modify its contents, and\n",
    "  press `Shift+Enter` to reevaluate its contents.  This will only modify\n",
    "  the current cell, so you will have to `Run All` again to see the effects\n",
    "  on the whole notebook.\n",
    "- While editing longer snippets, you can also press `Shift+Tab` while\n",
    "  typing to get more documentation hints about the code you are writing. \n",
    "\n",
    "This course is lectured by Amanda Prorok and Anil Madhavapeddy.  These notes\n",
    "are translated from Lawrence C. Paulson’s earlier course on Standard ML, which\n",
    "had credits to David Allsopp, Stuart Becker, Gavin Bierman, Chloë Brown, Silas\n",
    "Brown, Qi Chen, David Cottingham, William Denman, Robert Harle, Daniel Hulme,\n",
    "Frank King, Jack Lawrence-Jones, Joseph Lord, Dimitrios Los, Farhan Mannan,\n",
    "James Margetson, David Morgan, Alan Mycroft, Sridhar Prabhu, Frank Stajano,\n",
    "Thomas Tuerk, Xincheng Wang, Philip Withnall and Assel Zhiyenbayeva for\n",
    "pointing out errors.  The current notes have been ported to OCaml in 2019 by\n",
    "David Allsopp, Jon Ludlam and Anil Madhavapeddy, with feedback from Richard\n",
    "Sharp.\n",
    "\n",
    "Some books that are complementary to this course are:\n",
    "\n",
    "- [*OCaml from the Very Beginning*](http://ocaml-book.com) by John Whitington.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lecture 1: Introduction to Programming\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Concepts in Computer Science\n",
    "\n",
    "\n",
    "\n",
    "- Computers: a child can use them; **nobody** can fully understand them!\n",
    "- We can master complexity through levels of abstraction.\n",
    "- Focus on 2 or 3 levels at most!\n",
    "\n",
    "**Recurring issues:**\n",
    "\n",
    "- _what services_ to provide at each level\n",
    "- _how to implement_ them using lower-level services\n",
    "- _the interface:_ how the two levels should communicate\n",
    "\n",
    "A basic concept in computer science is that large systems can only be\n",
    "understood in levels, with each level further subdivided into functions or\n",
    "services of some sort. The interface to the higher level should supply the\n",
    "advertised services. Just as important, it should block access to the means by\n",
    "which those services are implemented. This _abstraction barrier_ allows one\n",
    "level to be changed without affecting levels above. For example, when a\n",
    "manufacturer designs a faster version of a processor, it is essential that\n",
    "existing programs continue to run on it. Any differences between the old and\n",
    "new processors should be invisible to the program.\n",
    "\n",
    "Modern processors have elaborate specifications, which still sometimes leave\n",
    "out important details. In the old days, you then had to consult the circuit\n",
    "diagrams.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example 1: Dates\n",
    "\n",
    "\n",
    "\n",
    "- Abstract level: dates over a certain interval\n",
    "- Concrete level: typically 6 characters: `YYMMDD` (where each character is represented by 8 bits)\n",
    "- Date crises caused by __inadequate__ internal formats:\n",
    "  * Digital’s PDP-10: using 12-bit dates (good for at most 11 years)\n",
    "  * 2000 crisis: 48 bits could be good for lifetime of universe!\n",
    "\n",
    "Digital Equipment Corporation’s date crisis occurred in 1975.  The\n",
    "PDP-10 was a 36-bit mainframe computer. It represented dates using a 12-bit\n",
    "format designed for the tiny PDP-8. With 12 bits, one can distinguish\n",
    "$2^{12} = 4096$ days or 11 years.\n",
    "\n",
    "The most common industry format for dates uses six characters: two for the\n",
    "year, two for the month and two for the day. The most common “solution” to the\n",
    "year 2000 crisis is to add two further characters, thereby altering file sizes.\n",
    "Others have noticed that the existing six characters consist of 48 bits,\n",
    "already sufficient to represent all dates over the projected lifetime of the\n",
    "universe: $2^{48}$ = $2.8\\times 10^{14}$ days = $7.7\\times 10^{11}$ years!\n",
    "\n",
    "Mathematicians think in terms of unbounded ranges, but the representation we\n",
    "choose for the computer usually imposes hard limits. A good programming\n",
    "language like OCaml lets one easily change the representation used in the\n",
    "program.  But if files in the old representation exist all over the place,\n",
    "there will still be conversion problems. The need for compatibility with older\n",
    "systems causes problems across the computer industry.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example II: Floating Point Numbers\n",
    "\n",
    "\n",
    "\n",
    "Computers have integers like `1066` and floats like $1.066\\times 10^3$.\n",
    "A floating-point number is represented by two integers.\n",
    "The concept of _data type_ involves:\n",
    "\n",
    "- how a value is represented inside the computer\n",
    "- the suite of operations given to programmers\n",
    "- valid and invalid (or exceptional) results, such as “infinity”\n",
    "Computer arithmetic can yield _incorrect answers!_\n",
    "\n",
    "In science, numbers written with finite precision and a decimal exponent are\n",
    "said to be in _standard form_. The computational equivalent is the _floating\n",
    "point number_. These are familiar to anybody who has used a scientific\n",
    "calculator.  Internally, a float consists of two integers.\n",
    "\n",
    "Because of its finite precision, floating-point computations are potentially\n",
    "inaccurate. To see an example, use your nearest electronic calculator to\n",
    "compute $(2^{1/10000})^{10000}$. I get $1.99999959$! With certain computations,\n",
    "the errors spiral out of control. Many programming languages fail to check\n",
    "whether even integer computations fall within the allowed range: you can add\n",
    "two positive integers and get a negative one!\n",
    "\n",
    "Most computers give us a choice of precisions. In 32-bit precision, integers\n",
    "typically range from $2^{31}-1$ (namely 2 147 483 647) to $-2^{31}$; reals\n",
    "are accurate to about six decimal places and can get as large as $10^{35}$ or so.\n",
    "For reals, 64-bit precision is often preferred. Early languages like Fortran\n",
    "required variables to be declared as `INTEGER`, `REAL` or `COMPLEX` and barred\n",
    "programmers from mixing numbers in a computation. Nowadays, programs handle\n",
    "many different kinds of data, including text and symbols. The concept of a\n",
    "_data type_ can ensure that different types of data are not combined in a\n",
    "senseless way.\n",
    "\n",
    "Inside the computer, all data are stored as bits. In most programming\n",
    "languages, the compiler uses types to generate correct machine code, and types\n",
    "are not stored during program execution. In this course, we focus almost\n",
    "entirely on programming in a high-level language: OCaml.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Goals of Programming\n",
    "\n",
    "\n",
    "\n",
    "- to describe a computation so that it can be done **mechanically**:\n",
    "  * Expressions compute values.\n",
    "  * Commands cause effects.\n",
    "- to do so efficiently and **correctly**, giving the right answers quickly\n",
    "- to allow easy modification as needs change\n",
    "  * Through an orderly **structure** based on abstraction principles\n",
    "  * Such as modules or classes\n",
    "\n",
    "Programming _in-the-small_ concerns the writing of code to do simple, clearly\n",
    "defined tasks. Programs provide expressions for describing mathematical\n",
    "formulae and so forth. This was the original contribution of FORTRAN, the\n",
    "FORmula TRANslator. Commands describe how control should flow from one part of\n",
    "the program to the next.\n",
    "\n",
    "As we code layer upon layer, we eventually find ourselves programming\n",
    "_in the large_ : joining large modules to solve some messy task. Programming\n",
    "languages have used various mechanisms to allow one part of the program to\n",
    "provide interfaces to other parts. Modules encapsulate a body of code, allowing\n",
    "outside access only through a programmer-defined interface. _Abstract Data\n",
    "Types_ are a simpler version of this concept, which implement a single concept\n",
    "such as dates or floating-point numbers.\n",
    "\n",
    "_Object-oriented programming_ is the most complicated approach to modularity.\n",
    "_Classes_ define concepts, and they can be built upon other classes. Operations\n",
    "can be defined that work in appropriately specialised ways on a family of\n",
    "related classes. _Objects_ are instances of classes and hold the data that is\n",
    "being manipulated.\n",
    "\n",
    "This course does not cover OCaml’s sophisticated module system, which can do\n",
    "many of the same things as classes. You will learn all about objects when you\n",
    "study Java. OCaml includes a powerful object system, although this is not used\n",
    "as much as its module system.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Why Program in OCaml?\n",
    "\n",
    "\n",
    "\n",
    "Why program in OCaml at all?\n",
    "\n",
    "* It is interactive.\n",
    "* It has a flexible notion of _data type_.\n",
    "* It hides the underlying hardware: _no crashes_.\n",
    "* Programs can easily be understood mathematically.\n",
    "* It distinguishes naming something from _updating memory_.\n",
    "* It manages storage for us.\n",
    "\n",
    "Programming languages matter. They affect the reliability, security, and\n",
    "efficiency of the code you write, as well as how easy it is to read, refactor,\n",
    "and extend. The languages you know can also change how you think, influencing\n",
    "the way you design software even when you’re not using them.\n",
    "\n",
    "What makes OCaml special is that it occupies a sweet spot in the space of\n",
    "programming language designs. It provides a combination of efficiency,\n",
    "expressiveness and practicality that is difficult to find matched by any other language.\n",
    "“ML” was originally the meta language of the LCF (Logic for Computable Functions)\n",
    "proof assistant released by Robin Milner in 1972 (at Stanford, and later at Cambridge).\n",
    "ML was turned into a compiler in order to make it easier to use LCF on different machines,\n",
    "and it was gradually turned into a full-fledged system of its own by the 1980s.\n",
    "\n",
    "The modern OCaml emerged in 1996, and the past few decades have seen OCaml\n",
    "attract a significant user base, and language improvements have been steadily\n",
    "added to support the growing commercial and academic codebases.\n",
    "OCaml is therefore the outcome of years of research into programming languages,\n",
    "and a good base to begin our journey into learning the foundations of computer\n",
    "science.\n",
    "\n",
    "Because of its connection to mathematics, OCaml programs can be designed and\n",
    "understood without thinking in detail about how the computer will run them.\n",
    "Although a program can abort, it cannot crash: it remains under the control of\n",
    "the OCaml system. It still achieves respectable efficiency and provides\n",
    "lower-level primitives for those who need them. Most other languages allow\n",
    "direct access to the underlying machine and even try to execute illegal\n",
    "operations, causing crashes.\n",
    "\n",
    "The only way to learn programming is by writing and running programs. This web\n",
    "notebook provides an interactive environment where you can modify the example\n",
    "fragments and see the results for yourself.  You should also consider\n",
    "installing OCaml on your own computer so that you try more advanced programs\n",
    "locally.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A first session with OCaml\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val pi : float = 3.14159265358979\n"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let pi = 3.14159265358979"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The first line of this simple session is a _value declaration_. It makes the\n",
    "name `pi` stand for the floating point number `3.14159`. (Such names are called\n",
    "_identifiers_.)  OCaml echoes the name (`pi`) and type (`float`) of the\n",
    "declared identifier.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : float = 7.06858347057702829\n"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pi *. 1.5 *. 1.5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The second line computes the area of the circle with radius `1.5` using the\n",
    "formula $A = \\pi r^2$. We use `pi` as an abbreviation for `3.14159`.\n",
    "Multiplication is expressed using `*.`, which is called an _infix operator_\n",
    "because it is written between its two operands.\n",
    "\n",
    "OCaml replies with the computed value (about `7.07`) and its type (again `float`).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val area : float -> float = <fun>\n"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let area r = pi *. r *. r"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "To work abstractly, we should provide the service “compute the area of a\n",
    "circle,” so that we no longer need to remember the formula. This sort of\n",
    "encapsulated computation is called a _function_. The third line declares the\n",
    "function `area`. Given any floating point number `r`, it returns another\n",
    "floating point number computed using the `area` formula; note that the function\n",
    "has type `float -> float`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : float = 12.56637061435916\n"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "area 2.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The fourth line calls the function `area` supplying `2.0` as the argument. A\n",
    "circle of radius `2` has an area of about `12.6`. Note that brackets around a\n",
    "function argument are not necessary.\n",
    "\n",
    "The function uses `pi` to stand for `3.14159`. Unlike what you may have seen in\n",
    "other programming languages, `pi` cannot be “assigned to” or otherwise updated.\n",
    "Its meaning within `area` will persist even if we issue a new `let` declaration\n",
    "for `pi` afterwards.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Raising a Number to a Power\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val npower : float -> int -> float = <fun>\n"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec npower x n =\n",
    "  if n = 0 then 1.0\n",
    "  else x *. npower x (n - 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "_Mathematical Justification_ (for $x\\not=0$):\n",
    "$$\n",
    "\\begin{aligned}\n",
    "           x^0 & = 1 \\\\\n",
    "           x^{n+1} & = x\\times x^n.\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "The function `npower` raises its real argument `x` to the power `n`, a\n",
    "non-negative integer. The function is **recursive**: it calls itself.  You\n",
    "can spot a recursive function due to the `rec` keyword in the definition:\n",
    "this indicates that any invocation of the function name should call itself.\n",
    "This concept should be familiar from mathematics, since exponentiation is defined by the\n",
    "rules shown above. You may also have seen recursion in the product rule for\n",
    "differentiation: $(u\\cdot v)' = u\\cdot v' + u'\\cdot v$. In finding the derivative of $u\\cdot v$,\n",
    "we recursively find the derivatives of $u$ and $v$, combining them to obtain the desired result.\n",
    "The recursion is meaningful because it terminates: we reduce the problem to two smaller\n",
    "problems, and this cannot go on forever. The OCaml programmer uses recursion\n",
    "heavily.  For $n\\geq0$, the equation $x^{n+1} = x\\times x^n$ yields an obvious\n",
    "computation:\n",
    "\n",
    "$$ x^3 = x\\times x^2 = x\\times x\\times x^1 = x\\times x\\times x\\times x^0 = x\\times x\\times x $$\n",
    "\n",
    "The equation clearly holds even for negative $n$. However, the corresponding\n",
    "computation runs forever:\n",
    "\n",
    "$$ x^{-1} = x\\times x^{-2} = x\\times x\\times x^{-3}=\\cdots $$\n",
    "\n",
    "Note that the function `npower` contains both an integer constant (`0`) and a\n",
    "floating point constant (`1.0`). The decimal point makes all the difference.\n",
    "OCaml will notice and ascribe different meaning to each type of constant.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val square : float -> float = <fun>\n"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let square x = x *. x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Now for a tiresome but necessary aside. In most languages, the types of\n",
    "arguments and results must always be specified. OCaml is unusual that it normally\n",
    "infers the types itself. However, sometimes it is useful to supply a hint to\n",
    "help you debug and develop your program. OCaml will still infer the types even if you don’t specify them, but in some cases\n",
    "it will use a more inefficient function than a specialised one. Some languages\n",
    "have just one type of number, converting automatically between different\n",
    "formats; this is slow and could lead to unexpected rounding errors.  Type\n",
    "constraints are allowed almost anywhere. We can put one on any occurrence of x\n",
    "in the function.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val square : float -> float = <fun>\n"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let square (x : float) = x *. x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or we can constrain the type of the function’s result:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val square : float -> float = <fun>\n"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let square x : float = x *. x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "OCaml treats the equality and comparison test specially. Expressions like `if x = y then` …\n",
    "are allowed provided `x` and `y` have the same type and equality testing is\n",
    "possible for that type. (We discuss equality further in a later lecture.)\n",
    "Note that `x <> y` is OCaml for $x\\not=y$.\n",
    "\n",
    "A characteristic feature of the computer is its ability to test for conditions\n",
    "and act accordingly.  In the early days, a program might jump to a given\n",
    "address depending on the sign of some number.  Later, John McCarthy defined\n",
    "the _conditional expression_ to satisfy:\n"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "if true then x else y = x\n",
    "if false then x else y = y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "OCaml evaluates the expression `if` $B$ `then` $E_1$ `else` $E_2$ by first evaluating $B$.\n",
    "If the result is `true` then OCaml evaluates $E_1$ and otherwise $E_2$.  Only one\n",
    "of the two expressions $E_1$ and $E_2$ is evaluated!  If both were evaluated,\n",
    "then recursive functions like `npower` above would run forever.\n",
    "\n",
    "The `if`-expression is governed by an expression of type `bool`, whose two\n",
    "values are `true` and `false`.  In modern programming languages, tests are not\n",
    "built into “conditional branch” constructs but have an independent status.\n",
    "Tests, or _Boolean expressions,_ can be expressed using relational operators\n",
    "such as `<` and `=`. They can be combined using the Boolean operators for\n",
    "negation (`not`), conjunction (written as `&&`) and disjunction (written as `||`).  New\n",
    "properties can be declared as functions: here, to test whether an integer is\n",
    "even, for example:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val even : int -> bool = <fun>\n"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let even n = n mod 2 = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## _Efficiently_ Raising a Number to a Power\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val power : float -> int -> float = <fun>\n"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec power x n =\n",
    "  if n = 1 then x\n",
    "  else if even n then\n",
    "    power (x *. x) (n / 2)\n",
    "  else\n",
    "    x *. power (x *. x) (n / 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "_Mathematical Justification_\n",
    "$$\\begin{aligned}\n",
    "           x^1 & = x \\\\\n",
    "           x^{2n} & = (x^2)^n  \\\\\n",
    "           x^{2n+1} & = x\\times(x^2)^n.\n",
    "\\end{aligned}$$\n",
    "\n",
    "For large `n`, computing powers using $x^{n+1} = x\\times x^n$ is too slow to\n",
    "be practical.  The equations above are much faster. Example:\n",
    "\n",
    "$$ 2^{12} = 4^6 = 16^3 = 16\\times 256^1 = 16\\times 256 = 4096. $$\n",
    "\n",
    "Instead of `n` multiplications, we need at most $2\\lg n$ multiplications,\n",
    "where $\\lg n$ is the logarithm of $n$ to the base $2$.\n",
    "\n",
    "We use the function `even`, declared previously, to test whether the\n",
    "exponent is even.  Integer division (`div`) truncates its result to an\n",
    "integer: dividing $2n+1$ by 2 yields $n$.\n",
    "\n",
    "A recurrence is a useful computation rule only if it is bound to terminate.\n",
    "If $n>0$ then $n$ is smaller than both $2n$ and $2n+1$.  After enough\n",
    "recursive calls, the exponent will be reduced to $1$.  The equations also hold\n",
    "if $n\\leq0$, but the corresponding computation runs forever.\n",
    "\n",
    "Our reasoning assumes arithmetic to be _exact_. Fortunately, the calculation is\n",
    "well-behaved using floating-point.\n",
    "\n",
    "Computer numbers have a finite range, which if exceeded results in the\n",
    "integer wrapping around.  You will understand this behaviour more as you\n",
    "learn about computer architecture and how modern systems represent\n",
    "numbers in memory.\n",
    "\n",
    "If integers and reals must be combined in a calculation, OCaml provides functions\n",
    "to convert between them:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int = 3\n"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "int_of_float 3.14159"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : float = 3.\n"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "float_of_int 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "OCaml’s libraries are organised using “modules”, so we may use compound\n",
    "identifiers such as `Float.of_int` to refer to library functions. There\n",
    "are many thousands of library functions available in the OCaml ecosystem,\n",
    "including text-processing and operating systems functions in addition to the\n",
    "usual numerical ones.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1.1\n",
    "\n",
    "\n",
    "One solution to the year 2000 bug involves storing years as two digits, but interpreting them such\n",
    "that 50 means 1950 and 49 means 2049. Comment on the merits and demerits of this approach.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1.2\n",
    "\n",
    "\n",
    "Using the date representation of the previous exercise, code OCaml functions to (a) compare two\n",
    "years (b) add/subtract some given number of years from another year.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1.3\n",
    "\n",
    "\n",
    "\n",
    "Why would no experienced programmer write an expression of the form `if` … `then true else false`?\n",
    "What about expressions of the form `if` … `then false else true`?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1.4\n",
    "\n",
    "\n",
    "Functions `npower` and `power` both return a `float`. The definition of `npower` returns the float\n",
    "value `1.0` in its base case. The definition of `power` does not, so how does the OCaml type checker\n",
    "know that `power` returns a `float`?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1.5\n",
    "\n",
    "\n",
    "Because computer arithmetic is based on binary numbers, simple decimals such as 0.1 often cannot be\n",
    "represented exactly. Write a function that performs the computation\n",
    "m$$\\underbrace{x+x+\\cdots+x}_{n}$$\n",
    "where $x$ has type `float`. (It is essential to use repeated addition rather than multiplication!)\n",
    "\n",
    "An error of this type has been blamed for the failure of an American Patriot Missile battery to\n",
    "intercept an incoming Iraqi missile during the [first Gulf War](https://en.wikipedia.org/wiki/MIM-104_Patriot#Failure_at_Dhahran).\n",
    "The missile hit an American Army barracks, killing 28.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1.6\n",
    "\n",
    "\n",
    "Another example of the inaccuracy of floating-point arithmetic takes the golden ratio\n",
    "$\\phi\\approx1.618\\ldots$ as its starting point:\n",
    "$$\\gamma_0 = \\frac{1+\\sqrt5}{2} \\quad\\text{and}\\quad\\gamma_{n+1} = \\frac{1}{\\gamma_n-1}.$$\n",
    "In theory, it is easy to prove that $\\gamma_n=\\cdots = \\gamma_1 = \\gamma_0$ for all $n>0$. Code this\n",
    "computation in OCaml and report the value of $\\gamma_{50}$. _Hint:_ in OCaml, $\\sqrt5$ is expressed\n",
    "as `sqrt 5.0`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lecture 2: Recursion and Efficiency\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Expression Evaluation\n",
    "\n",
    "\n",
    "\n",
    "Expression evaluation concerns expressions and the values they return. This\n",
    "view of computation may seem to be too narrow. It is certainly far removed from\n",
    "computer hardware, but that can be seen as an advantage. For the traditional\n",
    "concept of computing solutions to problems, expression evaluation is entirely\n",
    "adequate.\n",
    "\n",
    "Starting with $E_0$, the expression $E_i$ is reduced to $E_{i+1}$ until this\n",
    "process concludes with a value $v$.  A _value_ is something like a number\n",
    "that cannot be further reduced.\n",
    "\n",
    "We write $E \\rightarrow E'$ to say that $E$ is _reduced_ to $E'$.\n",
    "Mathematically, they are equal: $E=E'$, but the computation goes from $E$ to\n",
    "$E'$ and never the other way around.\n",
    "\n",
    "Computers also interact with the outside world.  For a start, they need some\n",
    "means of accepting problems and delivering solutions.  Many computer systems\n",
    "monitor and control industrial processes.  This role of computers is familiar\n",
    "now, but was never envisaged in the early days. Computer pioneers focused on\n",
    "mathematical calculations.  Modelling interaction and control requires a notion\n",
    "of _states_ that can be observed and changed.  Then we can consider\n",
    "updating the state by assigning to variables or performing input/output,\n",
    "finally arriving at conventional programs as coded in C, for instance.\n",
    "\n",
    "For now, we remain at the level of expressions, which is usually termed\n",
    "_functional programming_.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summing the first _n_ integers\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val nsum : int -> int = <fun>\n"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec nsum n =\n",
    "  if n = 0 then\n",
    "    0\n",
    "  else\n",
    "    n + nsum (n - 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The function call `nsum n` computes the sum `1 +` … `+ nz` rather naively, hence the\n",
    "initial `n` in its name:\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "nsum 3 \\Rightarrow &  3 + (nsum 2) \\\\\n",
    "       \\Rightarrow  & 3 + (2 + (num 1) \\\\\n",
    "       \\Rightarrow  & 3 + (2 + (1 + nsum 0)) \\\\\n",
    "       \\Rightarrow  & 3 + (2 + (1 + 0))\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "The nesting of parentheses is not just an artifact of\n",
    "our notation; it indicates a real problem.  The function gathers up a\n",
    "collection of numbers, but none of the additions can be performed until `nsum\n",
    "0` is reached.  Meanwhile, the computer must store the numbers in an internal\n",
    "data structure, typically the _stack_.  For large `n`, say `nsum 10000`, the\n",
    "computation might fail due to stack overflow.\n",
    "\n",
    "We all know that the additions can be performed as we go along.  How do we\n",
    "make the computer do that?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Iteratively summing the first `n` integers\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val summing : int -> int -> int = <fun>\n"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec summing n total =\n",
    "  if n = 0 then\n",
    "    total\n",
    "  else\n",
    "    summing (n - 1) (n + total)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Function `summing` takes an additional argument: a running total.  If\n",
    "`n` is zero then it returns the running total; otherwise, `summing`\n",
    "adds to it and continues.  The recursive calls do not nest; the additions are\n",
    "done immediately.\n",
    "\n",
    "A recursive function whose computation does not nest is called\n",
    "_iterative_ or _tail-recursive_. Many functions can be made iterative by\n",
    "introducing an argument analogous to `total`, which is often called an\n",
    "_accumulator_.\n",
    "\n",
    "The gain in efficiency is sometimes worthwhile and sometimes not.  The function\n",
    "`power` is not iterative because nesting occurs whenever the exponent is odd.\n",
    "Adding a third argument makes it iterative, but the change complicates the\n",
    "function and the gain in efficiency is minute; for 32-bit integers, the maximum\n",
    "possible nesting is 30 for the exponent $2^{31}-1$.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Recursion _vs_ Iteration\n",
    "\n",
    "\n",
    "\n",
    "- “Iterative” normally refers to a loop, coded using `while` for example (see the last lecture)\n",
    "- Tail-recursion is only efficient if the compiler detects it\n",
    "- Mainly it saves space (memory), though iterative code can also run after\n",
    "- Do not make programs iterative unless the gain is worth it\n",
    "\n",
    "A [classic book](https://en.wikipedia.org/wiki/Structure_and_Interpretation_of_Computer_Programs)\n",
    "by Abelson and Sussman used _iterative_ to mean _tail-recursive_. It describes\n",
    "the Lisp dialect known as Scheme. Iterative functions produce computations\n",
    "resembling those that can be done using while-loops in conventional languages.\n",
    "\n",
    "Many algorithms can be expressed naturally using recursion, but only awkwardly\n",
    "using iteration. There is a story that Dijkstra sneaked recursion into Algol-60\n",
    "by inserting the words “any other occurrence of the procedure name denotes\n",
    "execution of the procedure.” By not using the word “recursion”, he managed to\n",
    "slip this amendment past sceptical colleagues.\n",
    "\n",
    "Obsession with tail recursion leads to a coding style in which functions\n",
    "have many more arguments than necessary.  Write straightforward code first,\n",
    "avoiding only gross inefficiency.  If the program turns out to be too slow,\n",
    "tools are available for pinpointing the cause.  Always remember KISS (Keep\n",
    "It Simple, Stupid).\n",
    "\n",
    "I hope you have all noticed by now that the summation can be done even more\n",
    "efficiently using the arithmetic progression formula:\n",
    "\n",
    "$$ 1+\\cdots+n = n(n+1)/2 $$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Silly Summing the First _n_ Integers\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val sillySum : int -> int = <fun>\n"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec sillySum n =\n",
    "  if n = 0 then\n",
    "    0\n",
    "  else\n",
    "    n + (sillySum (n - 1) + sillySum (n - 1)) / 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The function calls itself $2^n$ times!  Bigger inputs mean higher costs---but\n",
    "what’s the growth rate?\n",
    "\n",
    "Now let us consider how to estimate various costs associated with a program.\n",
    "_Asymptotic complexity_ refers to how costs---usually time or space---grow with\n",
    "increasing inputs. Space complexity can never exceed time complexity, for it\n",
    "takes time to do anything with the space.  Time complexity often greatly\n",
    "exceeds space complexity.\n",
    "\n",
    "The function `sillySum` calls itself twice in each recursive step.  This\n",
    "function is contrived, but many mathematical formulas refer to a particular\n",
    "quantity more than once.  In OCaml, we can create a local binding to a computed\n",
    "value using the _local declaration_ syntax. In the following expression, `y` is\n",
    "computed once and used twice:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : float = 2.\n"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let x = 2.0 in\n",
    "let y = Float.pow x 20.0 in\n",
    "y *. (x /. y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Fast hardware does not make good algorithms unnecessary.  On the contrary,\n",
    "faster hardware magnifies the superiority of better algorithms.  Typically, we\n",
    "want to handle the largest inputs possible.  If we double our processing power,\n",
    "what do we gain?  How much can we increase $n$, the input to our function?\n",
    "With `sillySum`, we can only go from $n$ to $n+1$.  We are limited to this\n",
    "modest increase because the function’s running time is proportional to $2^n$.\n",
    "With the function `npower` defined in the previous lecture, we can go from $n$\n",
    "to $2n$: we can handle problems twice as big.  With `power` we can do much\n",
    "better still, going from $n$ to $n^2$.\n",
    "\n",
    "| complexity | 1 second   | 1 minute   | 1 hour    | gain  |\n",
    "| :--------: | ---------: | ---------: | --------: | ----: |\n",
    "| $n$        |       1000 |     60 000 | 3 600 000 | $x60$ |\n",
    "| $n \\log n$ |        140 |      4 895 |   204 095 | $x41$ |\n",
    "| $n^{2}$    |         31 |        244 |     1 897 | $x8$  |\n",
    "| $n^{3}$    |         10 |         39 |       153 | $x4$  |\n",
    "| $2^{n}$    |          9 |         15 |        21 | $+6$  |\n",
    "\n",
    "This table (excerpted from [a 40-year-old book](https://archive.org/details/designanalysisof00ahoarich)!)\n",
    "illustrates the effect of various time complexities.  The left-hand column indicates how many\n",
    "milliseconds are required to process an input of size $n$.  The other entries\n",
    "show the maximum size of $n$ that can be processed in the given time (one\n",
    "second, minute or hour).\n",
    "\n",
    "The table illustrates how large an input can be processed as a function\n",
    "of time.  As we increase the computer time per input from one second to one\n",
    "minute and then to one hour, the size of the input increases accordingly.\n",
    "\n",
    "The top two rows (complexities $n$ and $n \\lg n$) increase rapidly: for $n$, by\n",
    "a factor of 60.  The bottom two start out close together, but $n^3$ (which\n",
    "grows by a factor of 3.9) pulls well away from $2^n$ (whose growth is only\n",
    "additive).  If an algorithm’s complexity is exponential then it can never\n",
    "handle large inputs, even if it is given huge resources.  On the other hand,\n",
    "suppose the complexity has the form $n^c$, where $c$ is a constant.  (We say\n",
    "the complexity is _polynomial_.)  Doubling the argument then increases the\n",
    "cost by a constant factor.  That is much better, though if $c>3$ the algorithm\n",
    "may not be considered practical.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Comparing Algorithms: O Notation\n",
    "\n",
    "\n",
    "\n",
    "- Formally, define $f(n) = O(g(n))$ provided $|f(n)| \\leq c|g(n)|$\n",
    "- $|f(n)|$ is bounded for some constant $c$ and all _sufficiently large_ $n$.\n",
    "- Intuitively, look at the _most significant_ term.\n",
    "- Ignore _constant factors_ as they seldom dominate and are often transitory\n",
    "\n",
    "For example: consider $n^2$ instead of $3n^2+34n+433$.\n",
    "\n",
    "The cost of a program is usually a complicated formula.  Often we should\n",
    "consider only the most significant term.  If the cost is $n^2 + 99n + 900$\n",
    "for an input of size $n$, then the $n^2$ term will eventually dominate,\n",
    "even though $99n$ is bigger for $n<99$.\n",
    "The constant term $900$ may look big, but it is soon dominated by $n^2$.\n",
    "\n",
    "Constant factors in costs can be ignored unless they are large.  For one thing,\n",
    "they seldom make a difference: $100n^2$ will be better than $n^3$ in the long\n",
    "run: or _asymptotically_ to use the jargon.  Moreover, constant factors are\n",
    "seldom stable.  They depend upon details such as which hardware, operating\n",
    "system or programming language is being used.  By ignoring constant factors, we\n",
    "can make comparisons between algorithms that remain valid in a broad range of\n",
    "circumstances.\n",
    "\n",
    "The “Big O” notation is commonly used to describe efficiency---to be precise,\n",
    "_asymptotic complexity_.  It concerns the limit of a function as its\n",
    "argument tends to infinity.  It is an abstraction that meets the informal\n",
    "criteria that we have just discussed.\n",
    "In the definition, _sufficiently large_ means there is some constant $n_0$\n",
    "such that $|f(n)|\\leq c|g(n)|$ for all $n$ greater than $n_0$.  The\n",
    "role of $n_0$ is to ignore finitely many exceptions to the bound, such as the\n",
    "cases when $99n$ exceeds $n^2$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simple Facts About O Notation\n",
    "\n",
    "\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    " O(2g(n)) & \\text{ is the same as } O(g(n)) \\\\\n",
    " O(\\log_{10}n) & \\text{ is the same as } O(\\ln n)  \\\\\n",
    " O(n^2+50n+36) & \\text{ is the same as } O(n^2) \\\\[1.5ex]\n",
    " O(n^2) & \\text{ is contained in }  O(n^3) \\\\\n",
    " O(2^n) & \\text{ is contained in }  O(3^n)  \\\\\n",
    " O(\\log n) & \\text{ is contained in } O(\\sqrt n)\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "$O$ notation lets us reason about the costs of algorithms easily.\n",
    "- Constant factors such as the $2$ in $O(2g(n))$ drop out: we can use $O(g(n))$ with twice the value of $c$ in the definition.\n",
    "- Because constant factors drop out, the base of logarithms is irrelevant.\n",
    "- Insignificant terms drop out.  To see that $O(n^2+50n+36)$ is the same as $O(n^2)$, consider that $n^2+50n+36/n^2$ converges to 1 for increasing $n$. In fact, $n^2+50n+36 \\le 2n^2$ for $n\\ge 51$, so can double the constant factor\n",
    "\n",
    "If $c$ and $d$ are constants (that is, they are independent of $n$) with $0 < c < d$ then\n",
    "- $O(n^c)$ is contained in $O(n^d)$\n",
    "- $O(c^n)$ is contained in $O(d^n)$\n",
    "- $O(\\log n)$ is contained $in O(n^c)$\n",
    "\n",
    "To say that $O(c^n)$ _is contained in_ $O(d^n)$ means that the former gives\n",
    "a tighter bound than the latter.  For example, if $f(n)=O(2^n)$ then\n",
    "$f(n)=O(3^n)$ trivially, but the converse does not hold.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Common Complexity Classes\n",
    "\n",
    "\n",
    "\n",
    "- $O(1)$ is _constant_\n",
    "- $O(\\log n)$ is _logarithmic_\n",
    "- $O(n)$ is _linear_\n",
    "- $O(n\\log n)$ is _quasi-linear_\n",
    "- $O(n^2)$ is _quadratic_\n",
    "- $O(n^3)$ is _cubic_\n",
    "- $O(a^n)$ is _exponential_ (for fixed $a$)\n",
    "\n",
    "Logarithms grow very slowly, so $O(\\log n)$ complexity is excellent.  Because\n",
    "$O$ notation ignores constant factors, the base of the logarithm is\n",
    "irrelevant!\n",
    "\n",
    "Under linear we might mention $O(n\\log n)$, which occasionally is called\n",
    "_quasilinear_ and which scales up well for large $n$.\n",
    "\n",
    "An example of quadratic complexity is matrix addition: forming the sum of two\n",
    "$n\\times n$ matrices obviously takes $n^2$ additions.  Matrix\n",
    "multiplication is of cubic complexity, which limits the size of matrices that\n",
    "we can multiply in reasonable time.  An $O(n^{2.81})$ algorithm exists, but it\n",
    "is too complicated to be of much use, even though it is theoretically better.\n",
    "\n",
    "An exponential growth rate such as $2^n$ restricts us to small values of $n$.\n",
    "Already with $n=20$ the cost exceeds one million.  However, the worst case\n",
    "might not arise in normal practice.  OCaml type-checking is exponential in the\n",
    "worst case, but not for ordinary programs.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sample costs in O notation\n",
    "\n",
    "\n",
    "\n",
    "Recall that `npower` computes $x^n$\n",
    "by repeated multiplication while `nsum` naively computes the sum\n",
    "$1+\\cdots+n$.  Each obviously performs $O(n)$ arithmetic operations.  Because\n",
    "they are not tail recursive, their use of space is also $O(n)$.  The function\n",
    "`summing` is a version of `nsum` with an accumulating argument;\n",
    "its iterative behaviour lets it work in constant space.  $O$ notation spares\n",
    "us from having to specify the units used to measure space.\n",
    "\n",
    "\n",
    "| Function       | Time        | Space       |\n",
    "| :------------- | :---------- | :---------- |\n",
    "| npower, nsum   | O($n$)      | O($n$)      |\n",
    "| summing        | O($n$)      | O($1$)      |\n",
    "| $n(n+1)/2$     | O($1$)      | O($1$)      |\n",
    "| power          | O($\\log~n$) | O($\\log~n$) |\n",
    "| sillySum      | O($2^n$)    | O($n$)      |\n",
    "\n",
    "Even ignoring constant factors, the units chosen can influence the result.\n",
    "Multiplication may be regarded as a single unit of cost.  However, the cost of\n",
    "multiplying two $n$-digit numbers for large $n$ is itself an important\n",
    "question, especially now that public-key cryptography uses numbers hundreds of\n",
    "digits long.\n",
    "\n",
    "Few things can _really_ be done in constant time or stored in constant\n",
    "space.  Merely to store the number $n$ requires $O(\\log n)$ bits.  If a\n",
    "program cost is $O(1)$, then we have probably assumed that certain operations\n",
    "it performs are also $O(1)$---typically because we expect never to exceed the\n",
    "capacity of the standard hardware arithmetic.\n",
    "\n",
    "With `power`, the precise number of operations depends upon $n$ in a\n",
    "complicated way, depending on how many odd numbers arise, so it is convenient\n",
    "that we can just write $O(\\log n)$.  An accumulating argument could reduce its\n",
    "space cost to $O(1)$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Some Simple Recurrence Relations\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "Consider $T(n)$ has a cost we want to bound using $O$ notation.\n",
    "A typical _base case_ is $T(1)=1$.  Some _recurrences_ are:\n",
    "\n",
    "| Equation           | Complexity   |\n",
    "| ------------------ | ------------ |\n",
    "| $T(n+1) = T(n)+1$  | $O(n)$       |\n",
    "| $T(n+1) = T(n)+n$  | $O(n^2)$     |\n",
    "| $T(n) = T(n/2)+1$  | $O(\\log n)$  |\n",
    "| $T(n) = 2T(n/2)+n$ | $O(n\\log n)$ |\n",
    "\n",
    "To analyse a function, inspect its OCaml declaration.  Recurrence equations for\n",
    "the cost function $T(n)$ can usually be read off.  Since we ignore constant\n",
    "factors, we can give the base case a cost of one unit.  Constant work done in\n",
    "the recursive step can also be given unit cost; since we only need an upper\n",
    "bound, this unit represents the larger of the two actual costs.  We could use\n",
    "other constants if it simplifies the algebra.\n",
    "\n",
    "For example, recall our function `nsum`:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val nsum : int -> int = <fun>\n"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec nsum n =\n",
    "  if n = 0 then \n",
    "    0\n",
    "  else\n",
    "    n + nsum (n - 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Given $n+1$, it performs a constant amount of work (an addition and\n",
    "subtraction) and calls itself recursively with argument $n$.  We get the\n",
    "recurrence equations $T(0)=1$ and $T(n+1) = T(n)+1$.  The closed form is\n",
    "clearly $T(n)=n+1$, as we can easily verify by substitution.  The cost is\n",
    "_linear_.\n",
    "\n",
    "This function, given $n+1$, calls `nsum`, performing $O(n)$ work.\n",
    "Again ignoring constant factors, we can say that this call takes exactly $n$\n",
    "units.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val nsumsum : int -> int = <fun>\n"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec nsumsum n =\n",
    "  if n = 0 then\n",
    "    0\n",
    "  else\n",
    "    nsum n + nsumsum (n - 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "We get the recurrence equations $T(0)=1$ and $T(n+1) = T(n)+n$.  It is easy to\n",
    "see that $T(n)=(n-1)+\\cdots+1=n(n-1)/2=O(n^2)$.  The cost is\n",
    "_quadratic_.\n",
    "\n",
    "The function `power` divides its input $n$ into two, with\n",
    "the recurrence equation $T(n) = T(n/2)+1$.  Clearly $T(2^n)=n+1$, so\n",
    "$T(n)=O(\\log n)$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 2.1\n",
    "\n",
    "\n",
    "Code an iterative version of the function `power`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 2.2\n",
    "\n",
    "\n",
    "Add a column to the table of complexities from _The Design and Analysis of Computer Algorithms_ with the heading _60 hours:_\n",
    "\n",
    "| complexity |   1 second |   1 minute |    1 hour | 60 hours |\n",
    "| :--------: | ---------: | ---------: | --------: | -------: |\n",
    "| $n$        |       1000 |     60 000 | 3 600 000 |          |\n",
    "| $n \\log n$ |        140 |      4 895 |   204 095 |          |\n",
    "| $n^{2}$    |         31 |        244 |     1 897 |          |\n",
    "| $n^{3}$    |         10 |         39 |       153 |          |\n",
    "| $2^{n}$    |          9 |         15 |        21 |          |\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 2.3\n",
    "\n",
    "\n",
    "Let $g_1$, …, $g_k$ be functions such that $g_i(n)\\ge0$ for $i=1$, …, $k$ and all sufficiently\n",
    "large $n$.\n",
    "\n",
    "Show that if $f(n) = O(a_1 g_1(n)+\\cdots+a_k g_k(n))$ then $f(n) = O(g_1(n)+\\cdots+g_k(n))$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 2.4\n",
    "\n",
    "\n",
    "Find an upper bound for the recurrence given by $T(1) = 1$ and $T(n) = 2T(n/2)+1$.  You should be\n",
    "able to find a tighter bound than $O(n\\log n)$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lecture 3: Lists\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val x : int list = [3; 5; 9]\n"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let x = [3; 5; 9]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val y : (int * string) list = [(1, \"one\"); (2, \"two\")]\n"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let y = [(1, \"one\"); (2, \"two\")]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "A _list_ is an ordered series of elements; repetitions are significant.\n",
    "So `[3; 5; 9]` differs from `[5; 3; 9]` and from `[3; 3; 5; 9]`.  Elements in the\n",
    "list are separated with `;` when constructed, as opposed to the `,` syntax\n",
    "used for fixed-length tuples.\n",
    "\n",
    "All elements of a list must have the same type.  Above we see a list of\n",
    "integers and a list of `(integer, string)` pairs.  One can also have lists of\n",
    "lists, such as `[[3]; []; [5; 6]]`, which has type `int list list`.\n",
    "\n",
    "In the general case, if $x_1; \\ldots; x_n$ all have the same type (say\n",
    "$\\tau$) then the list $[x_1;\\ldots;x_n]$ has type $(\\tau)\\texttt{list}$.\n",
    "\n",
    "Lists are the simplest data structure that can be used to process collections\n",
    "of items.  Conventional languages use _arrays_ whose elements are\n",
    "accessed using subscripting: for example, $A[i]$ yields the $i$th element of\n",
    "the array $A$.  Subscripting errors are a known cause of programmer grief,\n",
    "however, so arrays should be replaced by higher-level data structures whenever\n",
    "possible.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int list = [3; 5; 9; 2; 10]\n"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x @ [2; 10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : (int * string) list = [(2, \"two\"); (1, \"one\")]\n"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "List.rev [(1, \"one\"); (2, \"two\")]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The infix operator `@` (also called `List.append`) concatenates two lists.\n",
    "Also built-in is `List.rev`, which reverses a list.  These are demonstrated\n",
    "in the session above.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The List Primitives\n",
    "\n",
    "\n",
    "\n",
    "There are two kinds of lists:\n",
    "\n",
    "- `[]` represents the empty list\n",
    "- `x :: l` is the list with head $x$ and tail $l$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val nil : 'a list = []\n"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let nil = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int list = [1]\n"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "1 :: nil"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int list = [1; 2]\n"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "1 :: 2 :: nil"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The operator `::` (also called `List.cons` for “construct”), puts a new element on\n",
    "to the head of an existing list.  While we should not be too preoccupied with\n",
    "implementation details, it is essential to know that `::` is an $O(1)$\n",
    "operation.  It uses constant time and space, regardless of the length of the\n",
    "resulting list.  Lists are represented internally with a linked structure;\n",
    "adding a new element to a list merely hooks the new element to the front of\n",
    "the existing structure.  Moreover, that structure continues to denote the same\n",
    "list as it did before; to see the new list, one must look at the new `::` node\n",
    "(or “cons cell”) just created.\n",
    "\n",
    "\n",
    "Here we see the element `1` being consed to the front of the list `[3; 5; 9]`:\n",
    "\n",
    "$$\n",
    "\\let\\down=\\downarrow\n",
    "\\begin{array}{*{10}{c@{\\,}}c}\n",
    ":: & \\to & \\cdots & :: & \\to &  :: & \\to &  :: & \\to & nil \\\\\n",
    "\\down &  &        & \\down &  & \\down &  & \\down  \\\\\n",
    "1     &  &        & 3     &  & 5     &  & 9\n",
    "\\end{array}\n",
    "$$\n",
    "\n",
    "Given a list, taking its first element (its “head”) or its list of\n",
    "remaining elements (its “tail”) also takes constant time.  Each\n",
    "operation just follows a link.  In the diagram above, the first down arrow\n",
    "leads to the head and the leftmost right arrow leads to the tail.  Once we\n",
    "have the tail, its head is the second element of the original list, etc.\n",
    "\n",
    "The tail is _not_ the last element; it is the _list_ of all elements\n",
    "other than the head!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Getting at the Head and Tail\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val null : 'a list -> bool = <fun>\n"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let null = function\n",
    "  | [] -> true\n",
    "  | x :: l -> false"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : bool = true\n"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "null []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : bool = false\n"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "null [1; 2; 3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "File \"[29]\", line 1, characters 7-17:\n",
      "1 | let hd (x::l) = x\n",
      "           ^^^^^^^^^^\n",
      "Warning 8: this pattern-matching is not exhaustive.\n",
      "Here is an example of a case that is not matched:\n",
      "[]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "val hd : 'a list -> 'a = <fun>\n"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let hd (x::l) = x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int = 1\n"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hd [1; 2; 3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "File \"[31]\", line 1, characters 7-17:\n",
      "1 | let tl (x::l) = l\n",
      "           ^^^^^^^^^^\n",
      "Warning 8: this pattern-matching is not exhaustive.\n",
      "Here is an example of a case that is not matched:\n",
      "[]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "val tl : 'a list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let tl (x::l) = l"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int list = [6; 5]\n"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tl [7; 6; 5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The empty list has neither head nor tail.  Applying `List.hd` `List.tl` to `[]`\n",
    "is an error---strictly speaking, an “exception”.  The function `null` can\n",
    "be used to check for the empty list beforehand.  Taking a list apart using\n",
    "combinations of `hd` and `tl` is hard to get right.  Fortunately, it is seldom\n",
    "necessary because of _pattern-matching_.\n",
    "\n",
    "The declaration of `null` above has two clauses: one for the empty list\n",
    "(for which it returns `true`) and one for non-empty lists (for which it\n",
    "returns `false`).\n",
    "\n",
    "The declaration of `hd` above has only one clause, for non-empty lists.  They\n",
    "have the form `x::l` and the function returns `x`, which is the head.  OCaml\n",
    "prints a warning to tell us that calling the function could raise an exception\n",
    "due to all possible inputs not being handled, including a counter-example (in\n",
    "this case, the empty list `[]`). The declaration of `tl` is similar to `hd`.\n",
    "\n",
    "These three primitive functions are _polymorphic_ and allow flexibility in the\n",
    "types of their arguments and results. Note their types!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : 'a list -> bool = <fun>\n"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "null"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : 'a list -> 'a = <fun>\n"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : 'a list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tl"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Symbols `'a` and `'b` are called _type variables_ and stand for any types. Code\n",
    "written using these functions is checked for type correctness at compile time.\n",
    "And this guarantees strong properties at run time, for example that the\n",
    "elements of any list all have the same type.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Computing the Length of a List\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val nlength : 'a list -> int = <fun>\n"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec nlength = function\n",
    "| [] -> 0\n",
    "| x :: xs -> 1 + nlength xs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int = 0\n"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlength []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int = 3\n"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlength [5; 6; 7]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "nlength[a; b; c] \\Rightarrow & 1 + nlength[b; c] \\\\\n",
    "   \\Rightarrow & 1 + (1 + nlength[c]) \\\\\n",
    "   \\Rightarrow & 1 + (1 + (1 + nlength[])) \\\\\n",
    "   \\Rightarrow & 1 + (1 + (1 + 0)) \\\\\n",
    "   \\Rightarrow & \\ldots \\;\\; 3\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "Most list processing involves recursion.  This is a simple example; patterns\n",
    "can be more complex.  Observe the use of a vertical bar `|` to separate the function’s\n",
    "clauses.  We have _one_ function declaration that handles two cases.\n",
    "To understand its role, consider the following faulty code:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "File \"[39]\", line 1, characters 16-22:\n",
      "1 | let rec nlength [] = 0\n",
      "                    ^^^^^^\n",
      "Warning 8: this pattern-matching is not exhaustive.\n",
      "Here is an example of a case that is not matched:\n",
      "_::_\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "val nlength : 'a list -> int = <fun>\n"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec nlength [] = 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "File \"[40]\", line 1, characters 16-40:\n",
      "1 | let rec nlength (x::xs) = 1 + nlength xs\n",
      "                    ^^^^^^^^^^^^^^^^^^^^^^^^\n",
      "Warning 8: this pattern-matching is not exhaustive.\n",
      "Here is an example of a case that is not matched:\n",
      "[]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "val nlength : 'a list -> int = <fun>\n"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec nlength (x::xs) = 1 + nlength xs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "These are two declarations, not one.  First we declare `nlength` to be a\n",
    "function that handles only empty lists.  Then we redeclare it to be a function\n",
    "that handles only non-empty lists; it can never deliver a result.  We see that\n",
    "a second `let` declaration replaces any previous one rather than extending it\n",
    "to cover new cases.\n",
    "\n",
    "Now, let us return to the declaration shown on the slide.  The length function\n",
    "is _polymorphic_ and applies to _all_ lists regardless of element\n",
    "type!  Most programming languages lack such flexibility.\n",
    "\n",
    "Unfortunately, this length computation is naive and wasteful.  Like\n",
    "`nsum` earlier, it is not tail-recursive.  It\n",
    "uses $O(n)$ space, where $n$ is the length of its input.  As usual, the\n",
    "solution is to add an accumulating argument.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Efficiently Computing the Length of a List\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val addlen : int * 'a list -> int = <fun>\n"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec addlen = function\n",
    "  | (n, []) -> n\n",
    "  | (n, x::xs) -> addlen (n + 1, xs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int = 3\n"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "addlen (0, [5; 6; 7])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "addlen(0, [a; b; c]) \\Rightarrow &  addlen(1, [b; c]) \\\\\n",
    "  \\Rightarrow  & addlen(2, [c]) \\\\\n",
    "  \\Rightarrow  & addlen(3, []) \\\\\n",
    "  \\Rightarrow  & 3\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "Patterns can be as complicated as we like.  Here, the two patterns are\n",
    "`(n, [])` and `(n, x::xs)`.\n",
    "\n",
    "Function `addlen` is again polymorphic.  Its type mentions the integer\n",
    "accumulator.\n",
    "\n",
    "Now we may declare an efficient length function.  It is simply a wrapper for\n",
    "`addlen`, supplying zero as the initial value of $n$.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val length : 'a list -> int = <fun>\n"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let length xs = addlen (0, xs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int = 4\n"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "length [5; 6; 7; 8]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The recursive calls do not nest: this version is iterative.  It takes $O(1)$\n",
    "space.  Obviously its time requirement is $O(n)$ because it takes at least $n$\n",
    "steps to find the length of an $n$-element list.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Append: List Concatenation\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val append : 'a list * 'a list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec append = function\n",
    "  | ([], ys) -> ys\n",
    "  | (x::xs, ys) -> x :: append (xs, ys)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int list = [1; 2; 3; 4]\n"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "append ([1; 2; 3], [4])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "append([1; 2; 3], [4]) \\Rightarrow & 1 :: append([2; 3], [4]) \\\\\n",
    "  \\Rightarrow & 1 :: (2 :: append([3], [4])) \\\\\n",
    "  \\Rightarrow & 1 :: (2 :: (3 :: append([], [4]))) \\\\\n",
    "  \\Rightarrow & 1 :: (2 :: (3 :: [4])) \\;; [1; 2; 3; 4]\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "Here is how append might be declared, ignoring the details of how `@` is made\n",
    "an infix operator.  This function is also not iterative.  It scans its first\n",
    "argument, sets up a string of `cons` operations (`::`) and finally does them.\n",
    "\n",
    "It uses $O(n)$ space and time, where $n$ is the length of its first argument.\n",
    "_Its costs are independent of its second argument._\n",
    "\n",
    "An accumulating argument could make it iterative, but with considerable\n",
    "complication.  The iterative version would still require $O(n)$ space and time\n",
    "because concatenation requires copying all the elements of the first list.\n",
    "Therefore, we cannot hope for asymptotic gains; at best we can decrease the\n",
    "constant factor involved in $O(n)$, but complicating the code is likely to\n",
    "increase that factor.  Never add an accumulator merely out of habit.\n",
    "\n",
    "Note append’s polymorphic type. It tells us that two lists can be joined if\n",
    "their element types agree.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reversing a List in O(n^2)\n",
    "\n",
    "\n",
    "\n",
    "Let us consider one way to reverse a list.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val nrev : 'a list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec nrev = function\n",
    "  | [] -> []\n",
    "  | x::xs -> (nrev xs) @ [x]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int list = [3; 2; 1]\n"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nrev [1; 2; 3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "nrev[a; b; c] \\Rightarrow & nrev[b; c] @ [a] \\\\\n",
    "  \\Rightarrow &  (nrev[c] @ [b]) @ [a] \\\\\n",
    "  \\Rightarrow &  ((nrev[] @ [c]) @ [b]) @ [a] \\\\\n",
    "  \\Rightarrow &  (([] @ [c]) @ [b]) @ [a] \\;; \\ldots \\;; [c; b; a]\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "This reverse function is grossly inefficient due to poor usage of append, which\n",
    "copies its first argument.  If `nrev` is given a list of length $n>0$, then\n",
    "append makes $n-1$ conses to copy the reversed tail.  Constructing the list\n",
    "`[x]` calls `cons` again, for a total of $n$ calls.  Reversing the tail\n",
    "requires $n-1$ more conses, and so forth.  The total number of conses is:\n",
    "\n",
    "$$ 0 + 1 + 2 + \\cdots + n = {n(n+1)/2} $$\n",
    "\n",
    "The time complexity is therefore $O(n^2)$.  Space complexity is only $O(n)$\n",
    "because the copies don’t all exist at the same time.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reversing a List in O(n)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val revApp : 'a list * 'a list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec revApp = function\n",
    "  | ([], ys) -> ys\n",
    "  | (x::xs, ys) -> revApp (xs, x::ys)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "revApp([a; b; c], []) \\Rightarrow & revApp([b; c], [a]) \\\\\n",
    "  \\Rightarrow & revApp([c], [b; a]) \\\\\n",
    "  \\Rightarrow & revApp([], [c; b; a]) \\\\\n",
    "  \\Rightarrow & [c; b; a]\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "Calling `revApp (xs, ys)` reverses the elements of `xs` and\n",
    "prepends them to `ys`.  Now we may declare\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val rev : 'a list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rev xs = revApp (xs, [])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int list = [3; 2; 1]\n"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rev [1; 2; 3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "It is easy to see that this reverse function performs just $n$ conses, given\n",
    "an $n$-element list.  For both reverse functions, we could count the number of\n",
    "conses precisely---not just up to a constant factor.  $O$ notation is still\n",
    "useful to describe the overall running time: the time taken by a cons\n",
    "varies from one system to another.\n",
    "\n",
    "The accumulator $y$ makes the function iterative.  But the gain in complexity\n",
    "arises from the removal of `append`.  Replacing an expensive operation (append)\n",
    "by a series of cheap operations (cons) is called _reduction in strength_\n",
    "and is a common technique in computer science.  It originated when many\n",
    "computers did not have a hardware multiply instruction; the series of products\n",
    "$i\\times r$ for $i=0$, $\\ldots, n$ could more efficiently be computed by\n",
    "repeated addition.  Reduction in strength can be done in various ways; we\n",
    "shall see many instances of removing append.\n",
    "\n",
    "Consing to an accumulator produces the result in reverse.  If\n",
    "that forces the use of an extra list reversal then the iterative function\n",
    "may be much slower than the recursive one.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lists, Strings and Characters\n",
    "\n",
    "\n",
    "\n",
    "Strings are provided in most programming languages to allow text processing.\n",
    "Strings are essential for communication with users. Even a purely numerical\n",
    "program formats its results ultimately as strings.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : char = 'a'\n"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'a'   (* a character constant *)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : string = \"a\"\n"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"a\"   (* a string constant of length 1 *)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : string = \"abc\"\n"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"abc\" (* a string constant of length 3 *)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int = 3\n"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "String.length \"abc\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : string = \"abcdef\"\n"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"abc\" ^ \"def\"  (* concatenate two strings *)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "In a few programming languages, strings simply are lists of characters, but\n",
    "this is poor design. Strings are an abstract concept in themselves. Treating\n",
    "them as lists leads to clumsy and inefficient code.\n",
    "\n",
    "Similarly, characters are not strings of size one, but are a primitive concept.\n",
    "Character constants in OCaml have the form `'c'`, where $c$ is any character.\n",
    "For example, the comma character is `','`.\n",
    "\n",
    "Special characters are coded in strings using _escape sequences_ involving the\n",
    "backslash character; among many others, a double quote is written `\"\\\\\"` and\n",
    "the newline character is written `\"\\n\"`. For example, the string\n",
    "`\"I\\nLIKE\\nCHEESE\\n\"` represents three text lines.\n",
    "\n",
    "In addition to the operators described above, the relations `<` `<=` `>` and\n",
    "`>=` work for strings and yield alphabetic order (more precisely, lexicographic\n",
    "order with respect to ASCII character codes).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 3.1\n",
    "\n",
    "\n",
    "Code a recursive function to compute the sum of a list’s elements. Then code an iterative version\n",
    "and comment on the improvement in efficiency.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 3.2\n",
    "\n",
    "\n",
    "Code a function to return the last element of a non-empty list. How efficiently can this be done?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 3.3\n",
    "\n",
    "\n",
    "Code a function to return the list consisting of the even-numbered elements of the list given as its\n",
    "argument. For example, given `[a; b; c; d]` it should return `[b; d]`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 3.4\n",
    "\n",
    "\n",
    "Consider the polymorphic types in these two function declarations:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val id : 'a -> 'a = <fun>\n"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let id x = x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val loop : 'a -> 'b = <fun>\n"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec loop x = loop x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Explain why these types make logical sense, preventing run time type errors, even for expressions\n",
    "like `id [id [id 0]]` or `loop true / loop 3`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 3.5\n",
    "\n",
    "\n",
    "Code a function `tails` to return the list of the tails of its argument. For example, given\n",
    "`[1; 2; 3]` it should return `[[1; 2; 3]; [2; 3]; [3]; []]`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lecture 4: More on Lists\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## List Utilities: take and drop\n",
    "\n",
    "\n",
    "\n",
    "Removing the first $i$ elements of a list can be done as follows:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val take : 'a list * int -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec take = function\n",
    "  | ([], _) -> []\n",
    "  | (x::xs, i) ->\n",
    "      if i > 0 then\n",
    "        x :: take (xs, i - 1)\n",
    "      else\n",
    "        []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val drop : 'a list * int -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec drop = function\n",
    "  | ([], _) -> []\n",
    "  | (x::xs, i) ->\n",
    "      if i > 0 then\n",
    "        drop (xs, i-1)\n",
    "      else\n",
    "        x::xs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "This lecture examines more list utilities, illustrating more patterns of\n",
    "recursion, and concludes with a small program for making change.\n",
    "\n",
    "The functions `take` and `drop` divide a list\n",
    "into parts, returning or discarding the first $i$ elements.\n",
    "\n",
    "$$\n",
    "xs = [\\underbrace{x_0,\\ldots,x_{i-1}}_{\\textstyle take(xs,i)},\n",
    "      \\underbrace{x_i,\\ldots,x_{n-1}}_{\\textstyle drop(xs,i)} ]\n",
    "$$\n",
    "\n",
    "Applications of `take` and `drop` will appear in future lectures.  Typically,\n",
    "they divide a collection of items into equal parts for recursive processing.\n",
    "\n",
    "The special pattern variable `_` appears in both functions.  This _wildcard\n",
    "pattern_ matches anything.  We could have written `i` in both positions, but\n",
    "the wildcard reminds us that the relevant clause ignores this argument.\n",
    "\n",
    "Function `take` is not iterative, but making it so would not improve\n",
    "its efficiency.  The task requires copying up to $i$ list elements, which must\n",
    "take $O(i)$ space and time.\n",
    "\n",
    "Function `drop` simply skips over $i$ list elements.  This requires\n",
    "$O(i)$ time but only constant space.  It is iterative and much faster than\n",
    "`take`.  Both functions use $O(i)$ time, but skipping elements is faster\n",
    "than copying them:  `drop`’s constant factor is smaller.\n",
    "\n",
    "Both functions take a list and an integer, returning a list of the same type.\n",
    "So their type is `'a list * int -> 'a list`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Linear Search\n",
    "\n",
    "\n",
    "\n",
    "- find $x$ in list $[x_1,\\ldots,x_n]$ by comparing with each element\n",
    "- obviously $O(n)$ time\n",
    "- simple & general\n",
    "- ordered searching needs only $O(\\log n)$\n",
    "- indexed lookup needs only $O(1)$\n",
    "\n",
    "_Linear search_ is the obvious way to find a desired item in a\n",
    "collection: simply look through all the items, one at a time.  If $x$ is in\n",
    "the list, then it will be found in $n/2$ steps on average, and even the worst\n",
    "case is obviously $O(n)$.\n",
    "\n",
    "Large collections of data are usually ordered or indexed so that items can be\n",
    "found in $O(\\log n)$ time, which is exponentially better than $O(n)$.  Even\n",
    "$O(1)$ is achievable (using a hash table), though subject to the usual\n",
    "proviso that machine limits are not exceeded.\n",
    "\n",
    "Efficient indexing methods are of prime importance: consider Web\n",
    "search engines.  Nevertheless, linear search is often used to search small\n",
    "collections because it is so simple and general, and it is the starting point\n",
    "for better algorithms.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Equality Tests\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val member : 'a -> 'a list -> bool = <fun>\n"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec member x = function\n",
    " | [] -> false\n",
    " | y::l ->\n",
    "    if x = y then\n",
    "      true\n",
    "    else\n",
    "      member x l"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "All the list functions we have encountered up to now have been “polymorphic”,\n",
    "working for lists of any type.  Function `member` uses linear search to report\n",
    "whether or not `x` occurs in `l`.\n",
    "\n",
    "To do this generically, it uses a special feature of OCaml known as\n",
    "“polymorphic equality”, which manifests itself via the `=`, `>=`, `<=`, `>` and\n",
    "`<` operators.  These operators inspect the _structure_ of the values using a\n",
    "consistent order.  Valid types you can compare this way include integers,\n",
    "strings, booleans, and tuples or lists of primitive types.\n",
    "\n",
    "More complex types can be compared this way within careful limits: recursive\n",
    "structures or function values will not work (we will cover function values in\n",
    "the Currying lecture later).  For now, it is sufficient to use these magic\n",
    "polymorphic equality operators.  As you get more familiar with OCaml and the\n",
    "use of higher order functions (also covered in a later lecture), you will\n",
    "encounter the use of explicit `compare` functions that are used to provide more\n",
    "complex equality tests.\n",
    "\n",
    "The presence of polymorphic equality is a contentious feature in OCaml.  While\n",
    "it provides a great ease of use in smaller codebases, it starts to become more\n",
    "dangerous when building larger OCaml-based systems.  Most large-scale users of\n",
    "OCaml tend towards not using it in important code, but it is just fine for our\n",
    "purposes while learning the beginning steps of computer science.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building a List of Pairs\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val zip : 'a list -> 'b list -> ('a * 'b) list = <fun>\n"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec zip xs ys =\n",
    "  match xs, ys with\n",
    "  | (x::xs, y::ys) -> (x, y) :: zip xs ys\n",
    "  | _ -> []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "$$ \\left.[x_1,\\ldots,x_n]\\atop\n",
    "         [y_1,\\ldots,y_n]\\right\\}\\;\\longmapsto\\;[(x_1,y_1),\\ldots,(x_n,y_n)]\n",
    "$$\n",
    "\n",
    "The _wildcard_ pattern `_` matches _anything_. The patterns are also tested\n",
    "in order of their definitions.\n",
    "\n",
    "A list of pairs of the form $[(x_1,y_1),\\ldots,(x_n,y_n)]$ associates each\n",
    "$x_i$ with $y_i$.  Conceptually, a telephone directory could be regarded as\n",
    "such a list, where $x_i$ ranges over names and $y_i$ over the corresponding\n",
    "telephone number.  Linear search in such a list can find the $y_i$ associated\n",
    "with a given $x_i$, or vice versa---very slowly.\n",
    "\n",
    "In other cases, the $(x_i,y_i)$ pairs might have been generated by applying a\n",
    "function to the elements of another list $[z_1,\\ldots,z_n]$.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val unzip : ('a * 'b) list -> 'a list * 'b list = <fun>\n"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec unzip = function\n",
    " | [] -> ([], [])\n",
    " | (x, y)::pairs ->\n",
    "     let xs, ys = unzip pairs in\n",
    "     (x::xs, y::ys)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The functions `zip` and `unzip` build and take apart lists of\n",
    "pairs: `zip` pairs up corresponding list elements and `unzip`\n",
    "inverts this operation.  Their types reflect what they do:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : 'a list -> 'b list -> ('a * 'b) list = <fun>\n"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : ('a * 'b) list -> 'a list * 'b list = <fun>\n"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unzip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "If the lists are of unequal length, `zip` discards surplus items at the\n",
    "end of the longer list.  Its first pattern only matches a pair of non-empty\n",
    "lists.  The second pattern is just a wildcard and could match anything.  OCaml\n",
    "tries the clauses in the order given, so the first pattern is tried first.\n",
    "The second only gets arguments where at least one of the lists is empty.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building a Pair of Results\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val unzip : ('a * 'b) list -> 'a list * 'b list = <fun>\n"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec unzip = function\n",
    " | [] -> ([], [])\n",
    " | (x, y)::pairs ->\n",
    "     let xs, ys = unzip pairs in\n",
    "     (x::xs, y::ys)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val revUnzip : ('a * 'b) list * 'a list * 'b list -> 'a list * 'b list =\n",
       "  <fun>\n"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec revUnzip = function\n",
    "  | ([], xs, ys) -> (xs, ys)\n",
    "  | ((x, y)::pairs, xs, ys) ->\n",
    "      revUnzip (pairs, x::xs, y::ys)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Given a list of pairs, `unzip` has to build _two_ lists of\n",
    "results, which is awkward using recursion.  The version shown about uses the\n",
    "_local declaration_ `let D in E`,\n",
    "where $D$ consists of declarations and $E$ is the expression that can use\n",
    "them. The let-construct counts as an expression and can be used\n",
    "(perhaps wrapped within parentheses) wherever an expression is expected.\n",
    "\n",
    "Note especially the declaration `let xs, ys = unzip pairs`\n",
    "which binds `xs` and `ys` to the results of the recursive call.\n",
    "In general, the declaration `let P = E` matches the\n",
    "pattern $P$ against the value of expression $E$.  It binds all the variables\n",
    "in $P$ to the corresponding values.\n",
    "\n",
    "Here is a version of `unzip` that replaces the local declaration by a\n",
    "function `conspair` for taking apart the pair of lists in the\n",
    "recursive call.  It defines the same\n",
    "computation as the previous version of `unzip` and is possibly clearer,\n",
    "but not every local declaration can be eliminated as easily.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val conspair : ('a * 'b) * ('a list * 'b list) -> 'a list * 'b list = <fun>\n"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let conspair ((x, y), (xs, ys)) = (x::xs, y::ys)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val unzip : ('a * 'b) list -> 'a list * 'b list = <fun>\n"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec unzip = function\n",
    "  | [] -> ([], [])\n",
    "  | xy :: pairs -> conspair (xy, unzip pairs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Making the function iterative yields `revUnzip` above, which is\n",
    "very simple.  Iteration can construct many results at once in different\n",
    "argument positions.  Both output lists are built in reverse order, which can\n",
    "be corrected by reversing the input to `revUnzip`.  The total costs\n",
    "will probably exceed those of `unzip` despite the advantages of\n",
    "iteration.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## An Application: Making Change\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val change : int list -> int -> int list = <fun>\n"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec change till amt =\n",
    "  if amt = 0 then\n",
    "    []\n",
    "  else\n",
    "    match till with\n",
    "    | [] -> raise (Failure \"no more coins!\")\n",
    "    | c::till ->\n",
    "        if amt < c then\n",
    "          change till amt\n",
    "        else\n",
    "          c :: change (c::till) (amt - c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "- The recursion _terminates_ when `amt = 0`.\n",
    "- Tries the _largest coin first_ to use large coins.\n",
    "- The algorithm is _greedy_ and can fail!\n",
    "\n",
    "The till has unlimited supplies of coins.  The largest coins should be tried\n",
    "first, to avoid giving change all in pennies.  The list of legal coin values,\n",
    "called `till`, is given in descending order, such as 50, 20, 10, 5,\n",
    "2 and 1.  (Recall that the head of a list is the element most easily reached.)\n",
    "The code for `change` is based on simple observations:\n",
    "\n",
    "- Change for zero consists of no coins at all.  (Note the pattern of `0` in the first clause.)\n",
    "- For a nonzero amount, try the largest available coin.  If it is small enough, use it and decrease the amount accordingly.\n",
    "- Exclude from consideration any coins that are too large.\n",
    "\n",
    "Although nobody considers making change for zero, this is the simplest way to\n",
    "make the algorithm terminate.  Most iterative procedures become simplest if,\n",
    "in their base case, they do nothing.  A base case of one instead of zero is\n",
    "often a sign of a novice programmer.\n",
    "\n",
    "The function can terminate either with success or failure.  It fails by\n",
    "raising exception `Failure` namely if `till` becomes empty while `amt` is still nonzero.\n",
    "(Exceptions will be discussed later.)\n",
    "\n",
    "Unfortunately, failure can occur even when change can be made.  The greedy\n",
    "‘largest coin first’ approach is to blame.  Suppose we have coins of values 5\n",
    "and 2, and must make change for 6; the only way is $6=2+2+2$, ignoring the 5.\n",
    "_Greedy algorithms_ are often effective, but not here.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## All Ways of Making Change\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val change : int list -> int -> int list list = <fun>\n"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec change till amt =\n",
    "  if amt = 0 then\n",
    "    [ [] ]\n",
    "  else\n",
    "    match till with\n",
    "    | [] -> []\n",
    "    | c::till ->\n",
    "        if amt < c then\n",
    "          change till amt\n",
    "        else\n",
    "          let rec allc = function\n",
    "            | [] -> []\n",
    "            | cs :: css -> (c::cs) :: allc css\n",
    "          in\n",
    "            allc (change (c::till) (amt - c)) @\n",
    "                  change till amt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Now we generalise the problem to return the list of _all possible ways_ of making change.\n",
    "Look at the type: the result is now a list of lists.\n",
    "\n",
    "The code will never raise exceptions.  It expresses failure by returning an\n",
    "empty list of solutions: it returns `[]` if the till is empty and the\n",
    "amount is nonzero.\n",
    "\n",
    "If the amount is zero, then there is only one way of making change;\n",
    "the result should be `[[]]`.  This is success in the base case.\n",
    "\n",
    "In nontrivial cases, there are two sources of solutions: to use a coin (if\n",
    "possible) and decrease the amount accordingly, or to remove the current coin\n",
    "value from consideration.\n",
    "\n",
    "The function `allc` is declared locally in order to make use\n",
    "of `c`, the current coin.  It adds an extra `c` to all the\n",
    "solutions returned by the recursive call to make change for `amt - c`.\n",
    "\n",
    "Observe the naming convention: `cs` is a list of coins, while\n",
    "`css` is a list of such lists.  The trailing `s' is suggestive of a\n",
    "plural.\n",
    "\n",
    "This complicated program, and the even trickier one on the next slide, are\n",
    "included as challenges.  Are you enthusiastic enough to work them out?  We\n",
    "shall revisit the “making change” task later to illustrate exception-handling.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## All Ways of Making Change --- Faster!\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val change : int list -> int -> int list -> int list list -> int list list =\n",
       "  <fun>\n"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec change till amt chg chgs =\n",
    "  if amt = 0 then\n",
    "    chg::chgs\n",
    "  else\n",
    "    match till with\n",
    "    | []  -> chgs\n",
    "    | c::till ->\n",
    "        if amt < 0 then\n",
    "          chgs\n",
    "        else\n",
    "          change (c::till) (amt - c) (c::chg)\n",
    "                 (change till amt chg chgs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We’ve added _another_ accumulating parameter!  Repeatedly improving simple code\n",
    "is called _stepwise refinement_.\n",
    "\n",
    "Two extra arguments eliminate many `::` and append operations from the previous\n",
    "slide’s change function.  The first, `chg`, accumulates the coins chosen so\n",
    "far; one evaluation of c::chg} replaces many evaluations of `allc`.  The\n",
    "second, `chgs`, accumulates the list of solutions so far; it avoids the need\n",
    "for append.  This version runs several times faster than the previous one.\n",
    "\n",
    "Making change is still extremely slow for an obvious reason: the number of\n",
    "solutions grows rapidly in the amount being changed.  Using 50, 20, 10, 5,\n",
    "2 and 1, there are 4366 ways of expressing 99.\n",
    "\n",
    "Our three change functions illustrate a basic technique: program development\n",
    "by stepwise refinement.  Begin by writing a very simple program and add\n",
    "requirements individually.  Add efficiency refinements last of all.\n",
    "Even if the simpler program cannot be included in the next version and has\n",
    "to be discarded, one has learned about the task by writing it.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 4.1\n",
    "\n",
    "\n",
    "Code a function to implement set union, by analogy with `inter` above. It should avoid introducing\n",
    "repetitions, for example the union of the lists `[4; 7; 1]` and `[6; 4; 7]` should be `[1; 6; 4; 7]`\n",
    "(though the order does not matter).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 4.2\n",
    "\n",
    "\n",
    "Code a function that takes a list of integers and returns two lists, the first consisting of all\n",
    "non-negative numbers found in the input and the second consisting of all the negative numbers.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 4.3\n",
    "\n",
    "\n",
    "How does this version of `zip` differ from the one above?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "File \"[73]\", line 2, characters 2-82:\n",
      "2 | ..match xs, ys with\n",
      "3 |   | (x::xs, y::ys) -> (x, y) :: zip xs ys\n",
      "4 |   | ([], [])   -> []\n",
      "Warning 8: this pattern-matching is not exhaustive.\n",
      "Here is an example of a case that is not matched:\n",
      "(([], _::_)|(_::_, []))\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "val zip : 'a list -> 'b list -> ('a * 'b) list = <fun>\n"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec zip xs ys =\n",
    "  match xs, ys with\n",
    "  | (x::xs, y::ys) -> (x, y) :: zip xs ys\n",
    "  | ([], [])   -> []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 4.4\n",
    "\n",
    "\n",
    "What assumptions do the ‘making change’ functions make about the variables `till` and `amt`?\n",
    "Describe what could happen if these assumptions were violated.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 4.5\n",
    "\n",
    "\n",
    "Show that the number of ways of making change for $n$ (ignoring order) is $O(n)$ if there are two\n",
    "legal coin values. What if there are three, four, … coin values?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 4.6\n",
    "\n",
    "\n",
    "We know nothing about the functions `f` and `g` other than their polymorphic types:\n",
    "`val f : 'a * 'b -> 'b * 'a` and `val g : 'a -> 'a list`.\n",
    "Suppose that `f (1, true)` and `g 0` are evaluated and return their results. State, with reasons,\n",
    "what you think the resulting values will be.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lecture 5: Sorting\n",
    "\n",
    "\n",
    "\n",
    "A few applications for sorting and arranging items into order are:\n",
    "\n",
    "- search\n",
    "- merging\n",
    "- duplicates\n",
    "- inverting tables\n",
    "- graphics algorithms\n",
    "\n",
    "Sorting is perhaps the most deeply studied aspect of algorithm design.\n",
    "Knuth’s series _The Art of Computer Programming_ devotes an entire\n",
    "volume to sorting and searching!  [Sedgewick](https://algs4.cs.princeton.edu/home/)\n",
    "also covers sorting.  Sorting has countless applications.\n",
    "\n",
    "Sorting a collection allows items to be found quickly.  Recall that linear\n",
    "search requires $O(n)$ steps to search among $n$ items.  A sorted collection\n",
    "admits _binary search_ which requires only $O(\\log n)$ time.  The idea\n",
    "of binary search is to compare the item being sought with the middle item (in\n",
    "position $n/2$) and then to discard either the left half or the right,\n",
    "depending on the result of the comparison.  Binary search needs arrays or\n",
    "trees, not lists; we shall come to binary search trees later.\n",
    "\n",
    "Two sorted files can quickly be _merged_ to form a larger sorted file.  Other\n",
    "applications include finding _duplicates_ that, after sorting, are adjacent.\n",
    "\n",
    "A telephone directory is sorted alphabetically by name.  The same information\n",
    "can instead be sorted by telephone number (useful to the police) or by street\n",
    "address (useful to junk-mail firms).  Sorting information in different ways\n",
    "gives it different applications.\n",
    "\n",
    "Common sorting algorithms include insertion sort, quicksort,\n",
    "mergesort and heapsort.  We shall consider the first three of\n",
    "these.  Each algorithm has its advantages.\n",
    "\n",
    "As a concrete basis for comparison, runtimes are quoted for DECstation\n",
    "computers.  (These were based on the MIPS chip, an early RISC design.)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How Fast Can We Sort?\n",
    "\n",
    "\n",
    "\n",
    "- typically count _comparisons_ $C(n)$\n",
    "- there are $n!$ permutations of $n$ elements\n",
    "- each comparison eliminates _half_ of the permutations $2^{C(n)}\\geq n!$\n",
    "- therefore $C(n)\\geq \\log(n!)\\approx n\\log n-1.44n$\n",
    "\n",
    "The usual measure of efficiency for sorting algorithms is the number of\n",
    "comparison operations required.  Mergesort requires only $O(n\\log n)$\n",
    "comparisons to sort an input of $n$ items.  It is straightforward to prove\n",
    "that this complexity is the best possible.  There\n",
    "are $n!$ permutations of $n$ elements and each comparison distinguishes two\n",
    "permutations.  The lower bound on the number of comparisons, $C(n)$, is\n",
    "obtained by solving $2^{C(n)}\\geq n!$; therefore $C(n)\\geq \\log(n!)\\approx\n",
    "n\\log n-1.44n$.\n",
    "\n",
    "In order to compare the sorting algorithms, we use the [following source](http://www.firstpr.com.au/dsp/rand31/p1192-park.pdf) of\n",
    "pseudo-random numbers. Never mind how this works: generating\n",
    "statistically good random numbers is hard.  Much effort has gone into those few\n",
    "lines of code.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val nextrandom : float -> float = <fun>\n"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let nextrandom seed =\n",
    "  let a = 16807.0 in\n",
    "  let m = 2147483647.0 in\n",
    "  let t = a *. seed in\n",
    "  t -. m *. (floor (t /. m))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val randlist : float * float list -> int -> float * float list = <fun>\n"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec randlist (seed, seeds) = function\n",
    "  | 0 -> (seed, seeds)\n",
    "  | n -> randlist (nextrandom seed, seed::seeds) (n-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "We can now bind the identifier `rs` to a list of 10,000 random numbers.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val seed : float = 1043618065.\n",
       "val rs : float list =\n",
       "  [1484786315.; 925166085.; 1614852353.; 721631166.; 173942219.; 1229443779.;\n",
       "   789328014.; 570809709.; 1760109362.; 270600523.; 2108528931.; 16480421.;\n",
       "   519782231.; 162430624.; 372212905.; 1954184989.; 898872741.; 1651521688.;\n",
       "   1114791388.; 1325968501.; 1469981427.; 465437343.; 1732504088.;\n",
       "   280054095.; 1924919450.; 1244369648.; 1524535715.; 706293012.;\n",
       "   1372325856.; 1302473561.; 941382430.; 2137445578.; 1937168414.;\n",
       "   1852570660.; 495231255.; 1092873378.; 140232191.; 328129841.; 632752255.;\n",
       "   227857208.; 1616471915.; 719842438.; 1402481130.; 745001020.; 791471334.;\n",
       "   2131048000.; 312659966.; 1389551813.; 443838892.; 854190041.; 741774068.;\n",
       "   267473377.; 1372555293.; 1539748349.; 697860888.; 1261546017.; 734770781.;\n",
       "   1512111397.; 813238415.; 1034499961.; 602256496.; 462191385.; 250718457.;\n",
       "   246489360.; 295426232.; 468306241.; 877829533.; 1130589227.; 1914364883.;\n",
       "   1479854970.; 878528585.; 1268712064.; 115837978.; 1803525169.; 689954646.;\n",
       "   1174020926.; 651968560.; 391152461.; 1776325865.; 2015344107.; 246977673.;\n",
       "   1381242649.; 1115030853.; 190703911.; 316761032.; 464218769.; 1537522160.;\n",
       "   1958981931.; 390463588.; 224009597.; 235243732.; 620352731.; 1374109567.;\n",
       "   832140633.; 675075162.; 1296171190.; 2009054653.; 1534419747.; 145880482.;\n",
       "   1649432515.; 403989126.; 1112417244.; 1290575192.; 896661113.; 218545469.;\n",
       "   1002393512.; 2131316096.; 551979127.; 932010335.; 665881436.; 1975412808.;\n",
       "   639877791.; 1781707137.; 894518191.; 568004958.; 1331430214.; 629489848.;\n",
       "   183264178.; 162027282.; 464592882.; 93302056.; 1178713033.; 1401486247.;\n",
       "   1846150129.; 1646978216.; 1104441491.; 111995009.; 66193165.; 2038880392.;\n",
       "   79340676.; 871801051.; 967550305.; 2067810758.; 1600354198.; 1746626663.;\n",
       "   1516388116.; 1308870791.; 173082747.; 189881227.; 478010722.; 739707315.;\n",
       "   255334803.; 164203714.; 1893097038.; 1587694259.; 292950569.; 918323194.;\n",
       "   41453146.; 1217297445.; 256768724.; 586494122.; 586258194.; 660494391.;\n",
       "   507554325.; 699716071.; 672895139.; 76065072.; 1594869218.; 1439459639.;\n",
       "   641123634.; 1650611940.; 177447368.; 301427463.; 525804524.; 553672425.;\n",
       "   926899509.; 794676486.; 690277940.; 2115070333.; 1062048650.; 1653192448.;\n",
       "   1808855340.; 126475289.; 1028198214.; 1739565096.; 1515748830.;\n",
       "   427491435.; 319330584.; 666483848.; 854842154.; 1853528448.; 1975611245.;\n",
       "   1905343266.; 1229802342.; 1416055428.; 2091603253.; 1068308139.;\n",
       "   198239748.; 982076370.; 1094563396.; 44402415.; 889814989.; 290736902.;\n",
       "   417580014.; 1935788352.; 595665917.; 367638848.; 894945148.; 1868608068.;\n",
       "   317883051.; 941451621.; 1595942893.; 789094274.; 1150772108.; 422742112.;\n",
       "   1444245279.; 1273601104.; 256005435.; 1742330161.; 1514599036.;\n",
       "   956344512.; 2113041793.; 293237373.; 1386995194.; 1509339194.; 891946522.;\n",
       "   1020832915.; 592544922.; 1746311153.; 1471539715.; 143832370.;\n",
       "   2041568248.; 1039556199.; 1608726047.; 1205124472.; 2123533995.;\n",
       "   1560620058.; 1837598795.; 1028172251.; 98318742.; 1405510706.;\n",
       "   1047695837.; 59221314.; 1822176683.; 1096018886.; 1528104537.;\n",
       "   1270922857.; 812074106.; 291115596.; 795788616.; 638657646.; 2034314619.;\n",
       "   1527649272.; 156357479.; 1010056202.; 1139413443.; 1110927723.;\n",
       "   1216083346.; 846825145.; 2100385733.; 315213605.; 1629637749.;\n",
       "   1139833627.; 895118866.; 296359237.; 1361440746.; 1188627020.;\n",
       "   1964199872.; 166733080.; 54185744.; 575493576.; 1810324496.; 1765549585.;\n",
       "   53514233.; 747348448.; 61758907.; 1710119765.; 188311628.; 8827553.;\n",
       "   67975851.; 1808633248.; 1290488843.; 1264775607.; 1711469075.;\n",
       "   1537468597.; 706677101.; 518290019.; 190285086.; 157683412.; 985907152.;\n",
       "   1571668636.; 632570698.; 791081325.; 1773794197.; 1787141077.;\n",
       "   1727982894.; 794213057.; 633163306.; 682601940.; 1573439414.; 1041956036.;\n",
       "   1169697582.; 758914445.; 2096291761.; 1502226099.; 1665995955.;\n",
       "   948048264.; 1596326605.; 1816773893.; ...]\n"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let seed, rs = randlist (1.0, []) 10000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Insertion Sort\n",
    "\n",
    "\n",
    "\n",
    "An insert operation does does $n/2$ comparisons on average.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val ins : 'a * 'a list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec ins = function\n",
    "  | x, [] -> [x]\n",
    "  | x, y::ys ->\n",
    "      if x <= y then\n",
    "        x :: y :: ys\n",
    "      else\n",
    "        y :: ins (x, ys)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "_Insertion sort_ takes $O(n^2)$ comparisons on average:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val insort : 'a list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec insort = function\n",
    "    | [] -> []\n",
    "    | x::xs -> ins (x, insort xs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Items from the input are copied one at a time to the output.  Each new item is\n",
    "inserted into the right place so that the output is always in order.\n",
    "\n",
    "We could easily write iterative versions of these functions, but to no purpose.\n",
    "Insertion sort is slow because it does $O(n^2)$ comparisons (and a lot of list\n",
    "copying), not because it is recursive.  Its quadratic runtime makes it nearly\n",
    "useless: it takes 174 seconds for our example while the next-worst figure is\n",
    "1.4 seconds.\n",
    "\n",
    "Insertion sort is worth considering because it is easy to code and illustrates\n",
    "the concepts.  Two efficient sorting algorithms, mergesort and heapsort, can be\n",
    "regarded as refinements of insertion sort.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Quicksort: The Idea\n",
    "\n",
    "\n",
    "\n",
    "The Quicksort algorithm has the following flow:\n",
    "\n",
    "- Choose a _pivot_ element, $a$\n",
    "- Divide to partition the input into two sublists:\n",
    "  * those _at most_ $a$ in value\n",
    "  * those _exceeding_ $a$\n",
    "- Conquer using recursive calls to sort the sublists\n",
    "- Combine the sorted lists by appending one to the other\n",
    "\n",
    "Quicksort was invented by C. A. R. Hoare, who now works at Microsoft Research,\n",
    "Cambridge.  Quicksort works by _divide and conquer,_ a basic algorithm design\n",
    "principle.  Quicksort chooses from the input some value $a$, called the\n",
    "_pivot_.  It partitions the remaining items into two parts: those $\\leq a$, and\n",
    "those $>a$.  It sorts each part recursively, then puts the smaller part before\n",
    "the greater.\n",
    "\n",
    "The cleverest feature of Hoare’s algorithm was that the partition could be done\n",
    "_in place_ by exchanging array elements.  Quicksort was invented before\n",
    "recursion was well known, and people found it extremely hard to understand.  As\n",
    "usual, we shall consider a list version based on functional programming.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Quicksort: The Code\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val quick : 'a list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec quick = function\n",
    "  | [] -> []\n",
    "  | [x] -> [x]\n",
    "  | a::bs ->\n",
    "      let rec part = function\n",
    "        | (l, r, []) -> (quick l) @ (a :: quick r)\n",
    "        | (l, r, x::xs) ->\n",
    "            if (x <= a) then\n",
    "              part (x::l, r, xs)\n",
    "            else\n",
    "              part (l, x::r, xs)\n",
    "      in\n",
    "      part ([], [], bs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Our OCaml quicksort copies the items.  It is still pretty fast, and it is much\n",
    "easier to understand.  It takes roughly 0.74 seconds to sort our list of random\n",
    "numbers.\n",
    "\n",
    "The function declaration consists of three clauses.  The first handles the\n",
    "empty list; the second handles singleton lists (those of the form `[x]`; the\n",
    "third handles lists of two or more elements.  Often, lists of length up to five\n",
    "or so are treated as special cases to boost speed.\n",
    "\n",
    "The locally declared function `part` partitions the input using `a` as the\n",
    "pivot.  The arguments `l` and `r` accumulate items for the left ($\\leq a$) and\n",
    "right ($>a$) parts of the input, respectively.\n",
    "\n",
    "It is not hard to prove that quicksort does $n\\log n$ comparisons, _in the average case_\n",
    "(see [page 94 of Aho](https://archive.org/details/designanalysisof00ahoarich)).  With random data, the pivot\n",
    "usually has an average value that divides the input in two approximately equal\n",
    "parts.  We have the recurrence $T(1) = 1$ and $T(n) = 2T(n/2)+n$, which is\n",
    "$O(n\\log n)$.  In our example, it is about 235 times faster than insertion\n",
    "sort.\n",
    "\n",
    "In the worst case, quicksort’s running time is quadratic!  An example is when\n",
    "its input is almost sorted or reverse sorted.  Nearly all of the items end up\n",
    "in one partition; work is not divided evenly.  We have the recurrence\n",
    "$T(1) = 1$ and $T(n+1) = T(n)+n$, which is $O(n^2)$.  Randomising the input\n",
    "makes the worst case highly unlikely.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Append-Free Quicksort\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val quik : 'a list * 'a list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec quik = function\n",
    "  | ([], sorted) -> sorted\n",
    "  | ([x], sorted) -> x::sorted\n",
    "  | a::bs, sorted ->\n",
    "     let rec part = function\n",
    "       | l, r, [] -> quik (l, a :: quik (r, sorted))\n",
    "       | l, r, x::xs ->\n",
    "           if x <= a then\n",
    "             part (x::l, r, xs)\n",
    "           else\n",
    "             part (l, x::r, xs)\n",
    "     in\n",
    "     part ([], [], bs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The list `sorted` accumulates the result in the _combine_ stage of\n",
    "the quicksort algorithm.  We have again used the standard technique for\n",
    "eliminating append.  Calling `quik(xs, sorted)` reverses the elements of\n",
    "`xs` and prepends them to the list `sorted`.\n",
    "\n",
    "Looking closely at `part`, observe that `quik(r, sorted)` is\n",
    "performed first.  Then `a` is consed to this sorted list.  Finally,\n",
    "`quik` is called again to sort the elements of `l`.\n",
    "\n",
    "The speedup is significant.  An imperative quicksort coded in Pascal (taken\n",
    "from [Sedgewick](https://algs4.cs.princeton.edu/20sorting/)) is just slightly faster than function\n",
    "`quik`.  The near-agreement is surprising because the computational overheads\n",
    "of lists exceed those of arrays.  In realistic applications, comparisons are\n",
    "the dominant cost and the overheads matter even less.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Merging Two Lists\n",
    "\n",
    "\n",
    "\n",
    "Merge joins two sorted lists.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val merge : 'a list * 'a list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec merge = function\n",
    "  | [], ys -> ys\n",
    "  | xs, [] -> xs\n",
    "  | x::xs, y::ys ->\n",
    "      if x <= y then\n",
    "        x :: merge (xs, y::ys)\n",
    "      else\n",
    "        y :: merge (x::xs, ys)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Generalises insert to two lists, and does at most $m+n-1$ comparisons.\n",
    "\n",
    "_Merging_ means combining two sorted lists to form a larger sorted list.\n",
    "It does at most $m+n$ comparisons, where $m$ and $n$ are the lengths of the\n",
    "input lists.  If $m$ and $n$ are roughly equal then we have a fast way of\n",
    "constructing sorted lists; if $n=1$ then merging degenerates to insertion,\n",
    "doing much work for little gain.\n",
    "\n",
    "Merging is the basis of several sorting algorithms; we look at a\n",
    "divide-and-conquer one.  Mergesort is seldom found in conventional programming\n",
    "because it is hard to code for arrays; it works nicely with lists.  It divides\n",
    "the input (if non-trivial) into two roughly equal parts, sorts them\n",
    "recursively, then merges them.\n",
    "\n",
    "Function `merge` is not iterative; the recursion is deep.  An iterative\n",
    "version is of little benefit for the same reasons that apply to\n",
    "`append` in the earlier lecture on Lists.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Top-down Merge sort\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val tmergesort : 'a list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec tmergesort = function\n",
    "  | [] -> []\n",
    "  | [x] -> [x]\n",
    "  | xs ->\n",
    "      let k = List.length xs / 2 in\n",
    "      let l = tmergesort (take (xs, k)) in\n",
    "      let r = tmergesort (drop (xs, k)) in\n",
    "      merge (l, r)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "$O(n\\log n)$ comparisons in worst case\n",
    "\n",
    "Mergesort’s _divide_ stage divides the input not by choosing a pivot (as\n",
    "in quicksort) but by simply counting out half of the elements.  The\n",
    "_conquer_ stage again involves recursive calls, and the _combine_\n",
    "stage involves merging.  Function `tmergesort` takes roughly 1.4\n",
    "seconds to sort the list `rs`.\n",
    "\n",
    "In the worst case, mergesort does $O(n\\log n)$ comparisons, with the same\n",
    "recurrence equation as in quicksort’s average case.  Because `take` and\n",
    "`drop` divide the input in two equal parts (they differ at most by\n",
    "one element), we always have $T(n) = 2T(n/2)+n$.\n",
    "\n",
    "Quicksort is nearly 3 times as fast in the example.  But it risks a\n",
    "quadratic worst case!  Merge sort is safe but slow.  So which algorithm is\n",
    "best?\n",
    "\n",
    "We have seen a _top-down_ mergesort.  _Bottom-up_ algorithms also\n",
    "exist.  They start with a list of one-element lists and repeatedly merge\n",
    "adjacent lists until only one is left.  A refinement, which exploits any\n",
    "initial order among the input, is to start with a list of increasing or\n",
    "decreasing runs of input items.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary of Sorting Algorithms\n",
    "\n",
    "\n",
    "\n",
    "- Optimal is $O(n\\log n)$ comparisons\n",
    "- Insertion sort: simple to code; too slow (quadratic) [174 secs]\n",
    "- Quicksort: fast on average; quadratic in worst case [0.53 secs]\n",
    "- Mergesort: optimal in theory; often slower than quicksort [1.4 secs]\n",
    "- _Match the algorithm to the application_\n",
    "\n",
    "Quicksort’s worst case cannot be ignored.  For large $n$, a complexity of\n",
    "$O(n^2)$ is catastrophic.  Mergesort has an $O(n\\log n)$ worst case running\n",
    "time, which is optimal, but it is typically slower than quicksort for random\n",
    "data.\n",
    "\n",
    "Non-comparison sorting deserves mentioning.  We can sort a large number of\n",
    "small integers using their radix representation in $O(n)$ time.  This result\n",
    "does not contradict the comparison-counting argument because comparisons are\n",
    "not used at all.  Linear time is achievable only if the greatest integer is\n",
    "fixed in advance; as $n$ goes to infinity, increasingly many of the items\n",
    "are the same.  It is a simple special case.\n",
    "\n",
    "Many other sorting algorithms exist. A few are outlined in the exercises.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 5.1\n",
    "\n",
    "\n",
    "Another sorting algorithm (selection sort) consists of looking at the elements to be sorted,\n",
    "identifying and removing a minimal element, which is placed at the head of the result. The tail is\n",
    "obtained by recursively sorting the remaining elements. State, with justification, the time\n",
    "complexity of this approach.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 5.2\n",
    "\n",
    "\n",
    "Implement selection sort (see previous exercise) using OCaml.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 5.3\n",
    "\n",
    "\n",
    "Another sorting algorithm (bubble sort) consists of looking at adjacent pairs of elements,\n",
    "exchanging them if they are out of order and repeating this process until no more exchanges are\n",
    "possible. State, with justification, the time complexity of this approach.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 5.4\n",
    "\n",
    "\n",
    "Implement bubble sort (see previous exercise) using OCaml.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lecture 6: Datatypes and Trees\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## An Enumeration Type\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "type vehicle = Bike | Motorbike | Car | Lorry\n"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type vehicle = Bike\n",
    "             | Motorbike\n",
    "             | Car\n",
    "             | Lorry"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "- We have declared a _new type_ named `vehicle`.\n",
    "- $\\ldots$ along with four new constants.\n",
    "- They are the _constructors_ of the datatype.\n",
    "\n",
    "The `type` declaration adds a new type to our OCaml session.  Type\n",
    "`vehicle` is as good as any built-in type and even admits\n",
    "pattern-matching.  The four new identifiers of type `vehicle` are\n",
    "called _constructors_.\n",
    "\n",
    "We could represent the various vehicles by the numbers 0--3.  However, the code would be\n",
    "hard to read and even harder to maintain.  Consider adding `Tricycle`\n",
    "as a new vehicle. If we wanted to add it before `Bike`, then all the\n",
    "numbers would have to be changed.  Using `type`, such additions are\n",
    "trivial and the compiler can (at least sometimes) warn us when it encounters a\n",
    "function declaration that doesn’t yet have a case for `Tricycle`.\n",
    "\n",
    "Representing vehicles by strings like `\"Bike\"`, `\"Car\"`, etc.,\n",
    "is also bad.  Comparing string values is slow and the compiler\n",
    "can’t warn us of misspellings like `\"MOtorbike\"`: they will make our\n",
    "code fail.\n",
    "\n",
    "Most programming languages allow the declaration of types like\n",
    "`vehicle`.  Because they consist of a series of identifiers, they are\n",
    "called _enumeration types_.  Other common examples are days of the week\n",
    "or colours.  The compiler chooses the integers for us; type-checking prevents\n",
    "us from confusing `Bike` with `Red` or `Sunday`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Declaring a Function on Vehicles\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val wheels : vehicle -> int = <fun>\n"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let wheels = function\n",
    "| Bike -> 2\n",
    "| Motorbike -> 2\n",
    "| Car -> 4\n",
    "| Lorry -> 18"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "- Datatype constructors can be used in patterns.\n",
    "- Pattern-matching is fast, even complicated nested patterns.\n",
    "\n",
    "The beauty of datatype declarations is that the new types behave as if they\n",
    "were built into OCaml. Type-checking catches common errors, such as mixing up\n",
    "different datatypes in a function like `wheels`, as well as missing\n",
    "and redundant patterns.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A Datatype with Constructor Functions\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "type vehicle = Bike | Motorbike of int | Car of bool | Lorry of int\n"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type vehicle = Bike\n",
    "             | Motorbike of int\n",
    "             | Car       of bool\n",
    "             | Lorry     of int"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "- Constructor functions (like `Lorry`) make _distinct values_.\n",
    "- Different kinds of `vehicle` can belong to one list: `[Bike, Car true, Motorbike 450]`\n",
    "\n",
    "OCaml generalises the notion of enumeration type to allow data to be associated\n",
    "with each constructor.  The constructor `Bike` is a vehicle all by itself, but\n",
    "the other three constructors are functions for creating vehicles.\n",
    "\n",
    "Since we might find it hard to remember what the various `int` and\n",
    "`bool` components are for, it is wise to include _comments_ in\n",
    "complex declarations.  In OCaml, comments are enclosed in the brackets\n",
    "`(*` and `*)`.  Programmers should comment their code to explain\n",
    "design decisions and key features of the algorithms (sometimes by citing a\n",
    "reference work).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "type vehicle = Bike | Motorbike of int | Car of bool | Lorry of int\n"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type vehicle = Bike\n",
    "             | Motorbike of int  (* engine size in CCs *)\n",
    "             | Car       of bool (* true if a Reliant Robin *)\n",
    "             | Lorry     of int  (* number of wheels *)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The list shown on the slide represents a bicycle, a Reliant Robin and a large\n",
    "motorbike.  It can be almost seen as a mixed-type list containing integers and\n",
    "booleans.  It is actually a list of vehicles; datatypes lessen the impact of\n",
    "the restriction that all list elements must have the same type.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A Finer Wheel Computation\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val wheels : vehicle -> int = <fun>\n"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let wheels = function\n",
    "| Bike -> 2\n",
    "| Motorbike _ -> 2\n",
    "| Car robin -> if robin then 3 else 4\n",
    "| Lorry w -> w"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "This function consists of four clauses:\n",
    "- A Bike has two wheels.\n",
    "- A Motorbike has two wheels.\n",
    "- A Reliant Robin has three wheels; all other cars have four.\n",
    "- A Lorry has the number of wheels stored with its constructor.\n",
    "\n",
    "There is no overlap between the `Motorbike` and `Lorry` cases.  Although\n",
    "`Motorbike` and `Lorry` both hold an integer, OCaml takes the\n",
    "constructor into account. A Motorbike is distinct from any Lorry.\n",
    "\n",
    "Vehicles are one example of a concept consisting of several varieties with\n",
    "distinct features.  Most programming languages can represent such concepts\n",
    "using something analogous to datatypes.  (They are sometimes called\n",
    "_union types_ or _variant records_ whose _tag fields_ play the\n",
    "role of the constructors.)\n",
    "\n",
    "\n",
    "A pattern may be built from the constructors of several datatypes, including\n",
    "lists. A pattern may also contain integer and string constants. There is no\n",
    "limit to the size of patterns or the number of clauses in a function\n",
    "declaration. OCaml performs pattern-matching [efficiently](https://dl.acm.org/citation.cfm?id=507641)\n",
    "(you do not need to understand the details of how it optimises them at this stage).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Error Handling: Exceptions\n",
    "\n",
    "\n",
    "\n",
    "During a computation, what happens if something goes _wrong?_\n",
    "- Division by zero\n",
    "- Pattern matching failure\n",
    "\n",
    "_Exception-handling_ lets us recover gracefully.\n",
    "- Raising an exception abandons the current computation.\n",
    "- Handling the exception attempts an alternative computation.\n",
    "- The raising and handling can be far apart in the code.\n",
    "- Errors of _different sorts_ can be handled separately.\n",
    "\n",
    "Exceptions are necessary because it is not always possible to tell in advance\n",
    "whether or not a search will lead to a dead end or whether a numerical\n",
    "calculation will encounter errors such as overflow or divide by zero. Rather\n",
    "than just crashing, programs should check whether things have gone wrong, and\n",
    "perhaps attempt an alternative computation (perhaps using a different algorithm\n",
    "or higher precision). A number of modern languages provide exception handling.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exceptions in OCaml\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "exception Failure\n"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exception Failure"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "exception NoChange of int\n"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exception NoChange of int"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "ename": "error",
     "evalue": "runtime_error",
     "output_type": "error",
     "traceback": [
      "\u001b[31mException: Failure.\nCalled from file \"toplevel/toploop.ml\", line 208, characters 17-27\n\u001b[0m"
     ]
    }
   ],
   "source": [
    "raise Failure"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "File \"[91]\", line 3, characters 2-20:\n",
      "3 |   raise (NoChange 1);\n",
      "      ^^^^^^^^^^^^^^^^^^\n",
      "Warning 21: this statement never returns (or has an unsound type.)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "- : unit = ()\n"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "try\n",
    "  print_endline \"pre exception\";\n",
    "  raise (NoChange 1);\n",
    "  print_endline \"post exception\";\n",
    "with\n",
    "  | NoChange _ ->\n",
    "      print_endline \"handled a NoChange exception\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Each `exception` declaration introduces a distinct sort of exception, which can\n",
    "be handled separately from others. If $E$ raises an exception, then its\n",
    "evaluation has failed; _handling_ an exception means evaluating another\n",
    "expression and returning its value instead. One exception handler can specify\n",
    "separate expressions for different sorts of exceptions.\n",
    "\n",
    "Exception names are _constructors_ of the special datatype `exn`.  This is a\n",
    "peculiarity of OCaml that lets exception-handlers use pattern-matching. Note that\n",
    "exception `Failure` is just an error indication, while `NoChange n` carries\n",
    "further information: the integer $n$.\n",
    "\n",
    "The effect of `raise <expr>` is to jump to the most recently-encountered\n",
    "handler that matches `<expr>`.  The matching handler can only be found\n",
    "_dynamically_ (during execution); contrast with how OCaml associates occurrences\n",
    "of identifiers with their matching declarations, which does not require running\n",
    "the program.\n",
    "\n",
    "One criticism of OCaml’s exceptions is that---unlike the Java language---nothing\n",
    "in a function declaration indicates which exceptions it might raise. One\n",
    "alternative to exceptions is to instead return a value of datatype `option`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pre exception\n",
      "handled a NoChange exception\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "File \"[91]\", line 3, characters 2-20:\n",
      "3 |   raise (NoChange 1);\n",
      "      ^^^^^^^^^^^^^^^^^^\n",
      "Warning 21: this statement never returns (or has an unsound type.)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "val x : int option = Some 1\n"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let x = Some 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val y : 'a option = None\n"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let y = None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "type 'a option = None | Some of 'a\n"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type 'a option = None | Some of 'a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "`None` signifies an error, while `Some x` returns the solution $x$.  This\n",
    "approach looks clean, but the drawback is that many places in the code would\n",
    "have to check for `None`.  Despite this, there is a builtin `option` type\n",
    "in OCaml as it is so useful.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Making Change with Exceptions\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "exception Change\n"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "val change : int list -> int -> int list = <fun>\n"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exception Change\n",
    "let rec change till amt =\n",
    "  if amt = 0 then\n",
    "    []\n",
    "  else\n",
    "    match till with\n",
    "    | [] ->\n",
    "        raise Change\n",
    "    | c::till ->\n",
    "        if amt < 0 then\n",
    "          raise Change\n",
    "        else\n",
    "          try\n",
    "             c :: change (c::till) (amt - c)\n",
    "          with Change ->\n",
    "             change till amt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "In the Lists lectures, we considered the problem of making change.  The greedy\n",
    "algorithm presented there could not express “6 using 5 and 2” because it always\n",
    "took the largest coin.  Returning the list of all possible solutions avoids\n",
    "that problem rather expensively: we only need one solution.\n",
    "\n",
    "Using exceptions, we can code a _backtracking_ algorithm: one that can undo\n",
    "past decisions if it comes to a dead end.  The exception `Change` is raised if\n",
    "we run out of coins (with a non-zero amount) or if the amount goes negative.\n",
    "We always try the largest coin, but enclose the recursive call in an exception\n",
    "handler, which undoes the choice if it goes wrong.\n",
    "\n",
    "Carefully observe how exceptions interact with recursion.  The exception\n",
    "handler always undoes the _most recent_ choice, leaving others possibly to\n",
    "be undone later.  If making change really is impossible, then eventually\n",
    "`exception Change` will be raised with no handler to catch it, and it\n",
    "will be reported at top level.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Making Change: A Trace\n",
    "\n",
    "\n",
    "\n",
    "Here is the full execution. Observe how the exception handlers nest and how\n",
    "they drop away once the given expression has returned a value.\n"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "change [5; 2] 6\n",
    "  ⇒ 5::change [5; 2] 1 with C -> change [2] 6\n",
    "  ⇒ 5::(5::change [5; 2] -4) with C -> change [2] 1\n",
    "                             with C -> change [2] 6\n",
    "  ⇒ 5::(change [2] 1) with C -> change [2] 6\n",
    "  ⇒ 5::(2::change [2] -1) with Chang -> change [] 1\n",
    "                          with C -> change [2] 6\n",
    "  ⇒ 5::(change [] 1) with C -> change [2] 6\n",
    "  ⇒ change [2] 6\n",
    "  ⇒ 2::(change [2] 4) with C -> change [] 6\n",
    "  ⇒ 2::(2::change [2] 2) with C -> change [] 4\n",
    "                         with C -> change [] 6\n",
    "  ⇒ 2::(2::(2::change [2] 0)) with C -> change [] 2\n",
    "                              with C -> change [] 4\n",
    "                              with C -> change [] 6\n",
    "  ⇒ 2::(2::[2]) with C -> change [] 4\n",
    "                with C -> change [] 6\n",
    "  ⇒ 2::[2; 2] with C -> change [] 6\n",
    "  ⇒ [2; 2; 2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Binary Trees, a Recursive Datatype\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "type 'a tree = Lf | Br of 'a * 'a tree * 'a tree\n"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type 'a tree =\n",
    "  Lf\n",
    "| Br of 'a * 'a tree * 'a tree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "![](bintree.png)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int tree = Br (1, Br (2, Br (4, Lf, Lf), Br (5, Lf, Lf)), Br (3, Lf, Lf))\n"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Br(1, Br(2, Br(4, Lf, Lf),\n",
    "            Br(5, Lf, Lf)),\n",
    "      Br(3, Lf, Lf))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "A data structure with multiple branching is called a “tree”.  Trees can\n",
    "represent mathematical expressions, logical formulae, computer programs, the\n",
    "phrase structure of English sentences, etc.\n",
    "\n",
    "_Binary trees_ are nearly as fundamental as lists.  They can provide\n",
    "efficient storage and retrieval of information.  In a binary tree, each node\n",
    "is empty ($Lf$), or is a branch ($Br$) with a label and two subtrees.\n",
    "\n",
    "OCaml lists are a datatype and could be declared as follows:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "type 'a mylist = Nil | Cons of 'a * 'a mylist\n"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type 'a mylist =\n",
    "| Nil\n",
    "| Cons of 'a * 'a mylist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "We could even declare `::` as an infix constructor.  The only\n",
    "thing we could not define is the `[...]` notation, which is\n",
    "part of the OCaml grammar (although there does exist a mechanism\n",
    "to use a _similar_ syntax for custom indexed datatypes).\n",
    "\n",
    "A recursive type does not have to be polymorphic.\n",
    "For example, here is a simple datatype of tree shapes with no attached data\n",
    "that is recursive but not polymorphic.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "type shape = Null | Join of shape * shape\n"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type shape =\n",
    "| Null\n",
    "| Join of shape * shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The datatype `'a option` (mentioned above) is the opposite -- it is\n",
    "polymorphic, but not recursive.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Properties of Binary Trees\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val count : 'a tree -> int = <fun>\n"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec count = function\n",
    "| Lf -> 0  (* number of branch nodes *)\n",
    "| Br (v, t1, t2) -> 1 + count t1 + count t2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val depth : 'a tree -> int = <fun>\n"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec depth = function\n",
    "| Lf -> 0  (* length of longest path *)\n",
    "| Br (v, t1, t2) -> 1 + max (depth t1) (depth t2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The invariant $\\texttt{count}(t)\\le 2^{\\texttt{depth}(t)} - 1$ holds in the functions above.\n",
    "\n",
    "Functions on trees are expressed recursively using pattern-matching.  Both\n",
    "functions above are analogous to \\texttt{length} on lists.  Here is a third\n",
    "measure of a tree’s size:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val leaves : 'a tree -> int = <fun>\n"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec leaves = function\n",
    "| Lf -> 1\n",
    "| Br (v, t1, t2) -> leaves t1 + leaves t2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "This function is redundant because of a basic fact about trees, which can be\n",
    "proved by induction: for every tree $t$, we have $\\texttt{leaves}(t) =\n",
    "\\texttt{count}(t)+1$.  The inequality shown on the slide also has an elementary\n",
    "proof by induction.\n",
    "\n",
    "A tree of depth 20 can store $2^{20}-1$ or approximately one million elements.\n",
    "The access paths to these elements are short, particularly when compared with\n",
    "a million-element list!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 6.1\n",
    "\n",
    "\n",
    "Give the declaration of an OCaml type for the days of the week. Comment on the practicality of such\n",
    "a type in a calendar application.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 6.2\n",
    "\n",
    "\n",
    "Write an OCaml function taking a binary tree labelled with integers and returning their sum.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 6.3\n",
    "\n",
    "\n",
    "Using the definition of `'a tree` from before:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "type 'a tree = Lf | Br of 'a * 'a tree * 'a tree\n"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type 'a tree = Lf | Br of 'a * 'a tree * 'a tree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Examine the following function declaration. What does `ftree (1, n)` accomplish?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val ftree : int -> int -> int tree = <fun>\n"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec ftree k n =\n",
    "  if n = 0 then\n",
    "    Lf\n",
    "  else\n",
    "    Br (k, ftree (2 * k) (n - 1), ftree (2 * k + 1) (n - 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 6.4\n",
    "\n",
    "\n",
    "Give the declaration of an OCaml type for arithmetic expressions that have the following\n",
    "possibilities: real numbers, variables (represented by strings), or expressions of the form $-E$,\n",
    "$E+E$, $E\\times E$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 6.5\n",
    "\n",
    "\n",
    "Continuing the previous exercise, write a function that evaluates an expression. If the expression\n",
    "contains any variables, your function should raise an exception indicating the variable name.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lecture 7: Dictionaries and Functional Arrays\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dictionaries\n",
    "\n",
    "\n",
    "\n",
    "- lookup: find an item in the dictionary\n",
    "- update (insert): replace (store) an item in the dictionary\n",
    "- delete: remove an item from the dictionary\n",
    "- empty: the null dictionary\n",
    "- Missing: exception for errors in `lookup` and `delete`\n",
    "\n",
    "Ideally, an _abstract type_ should provide these operations and hide the internal data structures.\n",
    "\n",
    "A dictionary attaches values to identifiers, called “keys”.  Before choosing\n",
    "the internal representation for a data structure, you need to specify the full\n",
    "set of operations. In fact, here we only consider `update` (associating a\n",
    "value with an identifier) and `lookup` (retrieving such a value). Deletion\n",
    "is more difficult and would limit our choices. Some applications may need\n",
    "additional operations, such as `merge (combining two dictionaries). We\n",
    "shall see that update can be done efficiently in a functional style, without\n",
    "excessive copying.\n",
    "\n",
    "An _abstract type_ provides specified operations while hiding low-level\n",
    "details, such as the data structure used to represent dictionaries. Abstract\n",
    "types can be declared in any modern programming language. Java’s _objects_\n",
    "serve this role, as do OCaml’s modules. This course does not cover modules, and we\n",
    "simply declare the dictionary operations individually.\n",
    "\n",
    "An _association list_ (a list of pairs) is the simplest dictionary representation.\n",
    "Lookup is by linear search, and therefore slow: $O(n)$. Association lists are\n",
    "only usable if there are few keys in use. However, they are general in that the\n",
    "keys do not need a concept of ordering, only equality.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "exception Missing\n"
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exception Missing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val lookup : ('a * 'b) list * 'a -> 'b = <fun>\n"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec lookup = function\n",
    "| [], a -> raise Missing\n",
    "| (x, y) :: pairs, a ->\n",
    "    if a = x then\n",
    "      y\n",
    "    else\n",
    "      lookup (pairs, a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val update : ('a * 'b) list * 'a * 'b -> ('a * 'b) list = <fun>\n"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let update (l, b, y) = (b, y) :: l"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "To enter a new `(key, value)` pair, simply “cons” it to the list with `update`.\n",
    "This takes constant time, which is the best we could hope for.  But the space\n",
    "requirement is huge: linear in the number of updates, not in the number of\n",
    "distinct keys. Obsolete entries are never deleted: that would require first\n",
    "finding them, increasing the update time from $O(1)$ to $O(n)$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Binary Search Trees\n",
    "\n",
    "\n",
    "\n",
    "A _dictionary_ associates _values_ (here, numbers) with _keys_.\n",
    "\n",
    "![](binsearch.png)\n",
    "\n",
    "Binary search trees are an important application of binary trees.  They work\n",
    "for keys that have a total ordering, such as strings.  Each branch of the tree\n",
    "carries a $(key, value)$ pair; its left subtree holds smaller keys; the right\n",
    "subtree holds greater keys.  If the tree remains reasonably balanced, then\n",
    "update and lookup both take $O(\\log n)$ for a tree of size $n$.  These times\n",
    "hold in the average case; given random data, the tree is likely to remain\n",
    "balanced.\n",
    "\n",
    "At a given node, all keys in the left subtree are smaller (or equal) while all\n",
    "trees in the right subtree are greater.\n",
    "\n",
    "An unbalanced tree has a linear access time in the worst case.  Examples\n",
    "include building a tree by repeated insertions of elements in increasing or\n",
    "decreasing order; there is a close resemblance to quicksort.  Building a binary\n",
    "search tree, then converting it to inorder, yields a sorting algorithm called\n",
    "_treesort_.\n",
    "\n",
    "Self-balancing trees, such as Red-Black trees, attain $O(\\log n)$ in the worst\n",
    "case.  They are complicated to implement.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lookup: Seeks Left or Right\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "exception Missing of string\n"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exception Missing of string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val lookup : (string * 'a) tree * string -> 'a = <fun>\n"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec lookup = function\n",
    "| Br ((a, x), t1, t2), b ->\n",
    "    if b < a then\n",
    "      lookup (t1, b)\n",
    "    else if a < b then\n",
    "      lookup (t2, b)\n",
    "    else\n",
    "      x\n",
    "| Lf, b -> raise (Missing b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Guaranteed $O(\\log n)$ access time _if_ the tree is balanced!\n",
    "\n",
    "Lookup in the binary search tree goes to the left subtree if the desired\n",
    "key is smaller than the current one and to the right if it is greater.\n",
    "It raises `Missing` if it encounters an empty tree.\n",
    "\n",
    "Since an ordering is involved, we have to declare the functions for a specific\n",
    "type, here `string`.  Now exception `Missing` mentions that type: if lookup\n",
    "fails, the exception returns the missing key.  The exception could be\n",
    "eliminated using type `option` of our earlier Datatypes lecture, using the\n",
    "constructor `None` for failure.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Update\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val update : 'a -> 'b -> ('a * 'b) tree -> ('a * 'b) tree = <fun>\n"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec update k v = function\n",
    "| Lf -> Br ((k, v), Lf, Lf)\n",
    "| Br ((a, x), t1, t2) ->\n",
    "    if k < a then\n",
    "      Br ((a, x), update k v t1, t2)\n",
    "    else if a < k then\n",
    "      Br ((a, x), t1, update k v t2)\n",
    "    else (* a = k *)\n",
    "      Br ((a, v), t1, t2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "This is also $O(\\log n)$ as it copies the path only, and _not whole subtrees!_\n",
    "\n",
    "If you are familiar with the usual update operation for this sort of tree, you\n",
    "may wonder whether it can be implemented in OCaml, where there is no direct way to\n",
    "replace part of a data structure by something else.\n",
    "\n",
    "The update operation is a nice piece of functional programming.  It searches\n",
    "in the same manner as `lookup`, but the recursive calls reconstruct a\n",
    "new tree around the result of the update.  One subtree is updated and the\n",
    "other left unchanged.  The internal representation of trees ensures that\n",
    "unchanged parts of the tree are not copied, but _shared_.\n",
    "Therefore, update copies only the path from the root to the new\n",
    "node.  Its time and space requirements, for a reasonably balanced tree, are\n",
    "both $O(\\log n)$.\n",
    "\n",
    "The comparison between $b$ and $a$ allows three cases:\n",
    "- smaller: update the left subtree; share the right\n",
    "- greater: update the right subtree; share the left\n",
    "- equal: update the label and share both subtrees\n",
    "\n",
    "Note: in the function definition, `(* a = b*)` is a comment.  Comments\n",
    "in OCaml are enclosed in the brackets `(*` and `*)`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Aside: Traversing Trees (3 Methods)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val preorder : 'a tree -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec preorder = function\n",
    "| Lf -> []\n",
    "| Br (v, t1, t2) ->\n",
    "    [v] @ preorder t1 @ preorder t2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val inorder : 'a tree -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 112,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec inorder = function\n",
    "| Lf -> []\n",
    "| Br (v, t1, t2) ->\n",
    "    inorder t1 @ [v] @ inorder t2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val postorder : 'a tree -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec postorder = function\n",
    "| Lf -> []\n",
    "| Br (v, t1, t2) ->\n",
    "    postorder t1 @ postorder t2 @ [v]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "_Tree traversal_ means examining each node of a tree in some order.  [D. E.\n",
    "Knuth](https://en.wikipedia.org/wiki/The_Art_of_Computer_Programming) has\n",
    "identified three forms of tree traversal: preorder, inorder and\n",
    "postorder.  We can code these “visiting orders” as functions\n",
    "that convert trees into lists of labels.  Algorithms based on these notions\n",
    "typically perform some action at each node; the functions above simply copy\n",
    "the nodes into lists.  Consider the tree:\n",
    "\n",
    "![](bintree2.png)\n",
    "\n",
    "- `preorder` visits the label first (“Polish notation”), yielding `ABDECFG`\n",
    "- `inorder` visits the label midway, yielding `DBEAFCG`\n",
    "- `postorder` visits the label last (“Reverse Polish”), yielding `DEBFGCA`. You will be familiar with this concept if you own an RPN calculator.\n",
    "\n",
    "What is the use of `inorder`? Consider applying it to a binary search tree: the\n",
    "result is a sorted list of pairs. We could use this, for example, to merge two\n",
    "binary search trees. It is not difficult to transform a sorted list of pairs\n",
    "into a binary search tree.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Efficiently Traversing Trees\n",
    "\n",
    "\n",
    "\n",
    "Unfortunately, the functions shown on the previous slide are quadratic in the\n",
    "worst case: the appends in the recursive calls are inefficient.  To correct\n",
    "that problem, we (as usual) add an accumulating argument.  Observe how\n",
    "each function constructs its result list and compare with how appends were\n",
    "eliminated from `quicksort` in the Sorting lecture.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val preord : 'a tree * 'a list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec preord = function\n",
    "| Lf, vs -> vs\n",
    "| Br (v, t1, t2), vs ->\n",
    "    v :: preord (t1, preord (t2, vs))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val inord : 'a tree * 'a list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec inord = function\n",
    "| Lf, vs -> vs\n",
    "| Br (v, t1, t2), vs ->\n",
    "    inord (t1, v::inord (t2, vs))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val postord : 'a tree * 'a list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec postord = function\n",
    "| Lf, vs -> vs\n",
    "| Br (v, t1, t2), vs ->\n",
    "    postord (t1, postord (t2, v::vs))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "One can prove equations relating each of these functions to its counterpart on\n",
    "the previous slide.  For example:\n",
    "\n",
    "$$ \\texttt{inord}(t, vs) = \\texttt{inorder}(t) @ vs $$\n",
    "\n",
    "These three types of tree traversal are related in that all are depth-first.\n",
    "They each traverse the left subtree in full before traversing the right\n",
    "subtree.  Breadth-first search (from the Queues lecture) is another\n",
    "possibility.  That involves going through the levels of a tree one at a time.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Arrays\n",
    "\n",
    "\n",
    "\n",
    "- A conventional array is an indexed storage area.\n",
    "  * It is updated _in place_ by the command `a.(k) <- x`\n",
    "  * The concept is inherently _imperative_.\n",
    "- A _functional array_ is a finite map from integers to data.\n",
    "  * Updating implies _copying_ to return `update(A, k, x)`\n",
    "  * The new array equals `A` except that `A[k] = x`.\n",
    "- Can we do updates efficiently?\n",
    "\n",
    "The elements of a list can only be reached by counting from the front.\n",
    "Elements of a tree are reached by following a path from the root.  An\n",
    "_array_ hides such structural matters; its elements are uniformly\n",
    "designated by number.  Immediate access to arbitrary parts of a data structure\n",
    "is called _random access_.\n",
    "\n",
    "Arrays are the dominant data structure in conventional programming languages.\n",
    "The ingenious use of arrays is the key to many of the great classical\n",
    "algorithms, such as Hoare’s original quicksort (the partition step) and\n",
    "Warshall’s transitive-closure algorithm.\n",
    "\n",
    "The drawback is that subscripting is a chief cause of programmer error.  That\n",
    "is why arrays play little role in this introductory course.\n",
    "\n",
    "Functional arrays are described below in order to illustrate another way of\n",
    "using trees to organise data.  Here is a summary of basic dictionary data\n",
    "structures in order of decreasing generality and increasing efficiency:\n",
    "- Linear search: Most general, needing only equality on keys, but inefficient: linear time.\n",
    "- Binary search: Needs an ordering on keys.  Logarithmic access time in the average case, but our binary search trees are linear in the worst case.\n",
    "- Array subscripting: Least general, requiring keys to be integers, but even worst-case time is logarithmic.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Functional Arrays as Binary Trees\n",
    "\n",
    "\n",
    "\n",
    "The path to element $i$ follows the _binary code_ for $i$ (its “subscript”).\n",
    "\n",
    "![](array1.png)\n",
    "\n",
    "This simple representation (credited to W. Braun) ensures that the tree is\n",
    "balanced.  Complexity of access is always $O(\\log n)$, which is optimal.  For\n",
    "actual running time, access to conventional arrays is much faster: it requires\n",
    "only a few hardware instructions.  Array access is often taken to be $O(1)$,\n",
    "which (as always) presumes that hardware limits are never exceeded.\n",
    "\n",
    "The lower bound for array indices is one.  The upper bound starts at zero\n",
    "(which signifies the empty array) and can grow without limit.  Inspection of\n",
    "the diagram above should make it clear that these trees are always balanced:\n",
    "the left subtree can have at most one node more than the right subtree,\n",
    "recursively all the way down.  (This assumes that the array is defined for\n",
    "subscripts $1\\ldots n$ with no gaps; an array defined only for odd numbers, for\n",
    "example, would obviously be unbalanced.)\n",
    "\n",
    "The numbers in the diagram above are not the labels of branch nodes, but\n",
    "indicate the positions of array elements. For example, the label corresponding\n",
    "to $A[2]$ is at the position shown. The nodes of a functional array are\n",
    "labelled with the data we want to store, not with these integers.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Lookup Function\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "exception Subscript\n"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "val sub : 'a tree * int -> 'a = <fun>\n"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exception Subscript\n",
    "let rec sub = function\n",
    "| Lf, _ -> raise Subscript  (* Not found *)\n",
    "| Br (v, t1, t2), k ->\n",
    "    if k = 1 then v\n",
    "    else if k mod 2 = 0 then\n",
    "      sub (t1, k / 2)\n",
    "    else\n",
    "      sub (t2, k / 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val sub : 'a tree * int -> 'a = <fun>\n"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec sub = function (* Alternative implementation *)\n",
    "| Lf, _ -> raise Subscript\n",
    "| Br (v, t1, t2), 1 -> v\n",
    "| Br (v, t1, t2), k when k mod 2 = 0 -> sub (t1, k / 2)\n",
    "| Br (v, t1, t2), k -> sub (t2, k / 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The lookup function `sub`, divides the subscript by 2 until 1 is\n",
    "reached.  If the remainder is 0 then the function follows the left subtree,\n",
    "otherwise the right.  If it reaches a leaf, it signals error by raising\n",
    "exception `Subscript`.\n",
    "\n",
    "Array access can also be understood in terms of the subscript’s binary code.\n",
    "Because the subscript must be a positive integer, in binary it has a leading\n",
    "one.  Discard this one and reverse the remaining bits.  Interpreting zero\n",
    "as _left_ and one as _right_ yields the path from the root to the\n",
    "subscript.\n",
    "\n",
    "Popular literature often explains the importance of binary as being led by\n",
    "hardware: because a circuit is either on or off.  The truth is almost the\n",
    "opposite.  Designers of digital electronics go to a lot of trouble to suppress\n",
    "the continuous behaviour that would naturally arise.  The real reason why\n",
    "binary is important is its role in algorithms: an `if-then-else` decision leads\n",
    "to binary branching.\n",
    "\n",
    "Data structures, such as trees, and algorithms, such as mergesort, use binary\n",
    "branching in order to reduce a cost from $O(n)$ to $O(\\log n)$.  Two is the\n",
    "smallest integer divisor that achieves this reduction.  (Larger divisors are\n",
    "only occasionally helpful, as in the case of B-trees, where they reduce the\n",
    "constant factor.)  The simplicity of binary arithmetic compared with decimal\n",
    "arithmetic is just another instance of the simplicity of algorithms based on\n",
    "binary choices.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Update Function\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val update : 'a tree * int * 'a -> 'a tree = <fun>\n"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec update = function\n",
    "| Lf, k, w ->\n",
    "    if k = 1 then\n",
    "      Br (w, Lf, Lf)\n",
    "    else\n",
    "      raise Subscript  (* Gap in tree *)\n",
    "| Br (v, t1, t2), k, w ->\n",
    "    if k = 1 then\n",
    "      Br (w, t1, t2)\n",
    "    else if k mod 2 = 0 then\n",
    "      Br (v, update (t1, k / 2, w), t2)\n",
    "    else\n",
    "      Br (v, t1, update (t2, k / 2, w))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The `update` function also divides the subscript repeatedly by two.  When it\n",
    "reaches a value of one, it has identified the element position.  Then it\n",
    "replaces the branch node by another branch with the new label.\n",
    "\n",
    "A leaf may be replaced by a branch, extending the array, provided no\n",
    "intervening nodes have to be generated.  This suffices for arrays without gaps\n",
    "in their subscripting.  (The data structure can be modified to allow _sparse_\n",
    "arrays, where most subscript positions are undefined.) Exception `Subscript`\n",
    "indicates that the subscript position does not exist and cannot be created.\n",
    "This use of exceptions is not easily replaced by `None` and `Some`.\n",
    "\n",
    "Note that there are two tests involving $k=1$.  If we have reached a leaf,\n",
    "it returns a branch, extending the array by one.  If we are still at a branch\n",
    "node, then the effect is to update an existing array element.\n",
    "\n",
    "A similar function can _shrink_ an array by one.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 7.1\n",
    "\n",
    "\n",
    "Draw the binary search tree that arises from successively inserting the following pairs into the\n",
    "empty tree: `(\"Alice\", 6)`\", `(\"Tobias\", 2)`\", `(\"Gerald\", 8)`\", `(\"Lucy\", 9)`. Then repeat this\n",
    "task using the order `(\"Gerald\", 8)`\", `(\"Alice\", 6)`\", `(\"Lucy\", 9)`\", `(\"Tobias\", 2)`. Why are\n",
    "results different?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 7.2\n",
    "\n",
    "\n",
    "Code an insertion function for binary search trees. It should resemble the existing `update`\n",
    "function except that it should raise the exception `Collision` if the item to be inserted is already\n",
    "present.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 7.3\n",
    "\n",
    "\n",
    "Continuing the previous exercise, it would be natural for exceptional `Collision` to return the\n",
    "value previously stored in the dictionary. Why is that goal difficult to achieve?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 7.4\n",
    "\n",
    "\n",
    "Describe an algorithm for deleting an entry from a binary search tree. Comment on the suitability of\n",
    "your approach.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 7.5\n",
    "\n",
    "\n",
    "Code the delete function outlined in the previous exercise.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 7.6\n",
    "\n",
    "\n",
    "Show that the functions `preorder`, `inorder` and `postorder` all require $O(n^2)$ time in the worst\n",
    "case, where $n$ is the size of the tree.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 7.7\n",
    "\n",
    "\n",
    "Show that the functions `preord`, `inord` and `postord` all take linear time in the size of the\n",
    "tree.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 7.8\n",
    "\n",
    "\n",
    "Write a function to remove the first element from a functional array. All the other elements are to\n",
    "have their subscripts reduced by one. The cost of this operation should be linear in the size of the\n",
    "array.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lecture 8: Functions as Value\n",
    "\n",
    "\n",
    "\n",
    "In OCaml, functions can be\n",
    "- passed as arguments to other functions,\n",
    "- returned as results,\n",
    "- put into lists, trees, etc.,\n",
    "- but _not_ tested for equality.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : (int -> int) list = [<fun>; <fun>; <fun>]\n"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[(fun n -> n * 2);\n",
    " (fun n -> n * 3);\n",
    " (fun n -> n + 1)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Progress in programming languages can be measured by what abstractions they\n",
    "admit.  Conditional expressions (descended from conditional jumps based\n",
    "on the sign of some numeric variable) and parametric types such as\n",
    "$\\alpha\\,\\texttt{list}$ are examples.  The idea that functions could be used\n",
    "as values in a computation arose early, but it took some time before the idea\n",
    "was fully realised.  Many programming languages let functions be passed as\n",
    "arguments to other functions, but few take the trouble needed to allow\n",
    "functions to be returned as results.\n",
    "\n",
    "In mathematics, a _functional_ or _higher-order function_ is a\n",
    "function that operates on other functions.  Many functionals are familiar from\n",
    "mathematics: for example, the differential operator maps functions to their\n",
    "derivatives, which are also functions.  To a mathematician, a function is\n",
    "typically an infinite, uncomputable object.  We use OCaml functions to represent\n",
    "algorithms.  Sometimes they represent infinite collections of data given by\n",
    "computation rules.\n",
    "\n",
    "Functions cannot be compared for equality. We could compare the machine\n",
    "addresses of the compiled code, but that would merely be a test of identity: it\n",
    "would regard any two separate functions as unequal even if they were compiled\n",
    "from identical pieces of source code.  Such a low-level feature has no place in\n",
    "a principled language.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Functions Without Names\n",
    "\n",
    "\n",
    "\n",
    "If functions are to be regarded as computational values, then we need a\n",
    "notation for them.  The `fun` notation expresses a non-recursive function\n",
    "value without giving the function a name.\n",
    "\n",
    "$\\tt fun x \\rightarrow E$ is the function $f$ such that $f(x)=E$.\n",
    "The function `fun n -> n*2` is a _doubling function_.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int -> int = <fun>\n"
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fun n -> n * 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int = 34\n"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(fun n -> n * 2) 17"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The main purpose of `fun`-notation is to package up small expressions that are to be\n",
    "applied repeatedly using some other function.\n",
    "The expression `fun n -> n*2` has the same value as the identifier\n",
    "`double`, declared as follows:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val double : int -> int = <fun>\n"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let double n = n * 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The `fun` notation can also do pattern matching, and the `function` keyword\n",
    "adds an anonymous variable name to pattern match against.  The following functions\n",
    "are all equivalent, with the latter definitions bound to the `is_zero` value and the earlier ones anonymous:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int -> bool = <fun>\n"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fun x -> match x with 0 -> true | _ -> false"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int -> bool = <fun>\n"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "function 0 -> true | _ -> false"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val is_zero : int -> bool = <fun>\n"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let is_zero = fun x -> match x with 0 -> true | _ -> false"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val is_zero : int -> bool = <fun>\n"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let is_zero = function 0 -> true | _ -> false"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Curried Functions\n",
    "\n",
    "\n",
    "\n",
    "A _curried function_ returns another function as its result.    \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val prefix : string -> string -> string = <fun>\n"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let prefix = fun a -> fun b -> a ^ b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val promote : string -> string = <fun>\n"
      ]
     },
     "execution_count": 129,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let promote = prefix \"Senior \""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : string = \"Junior Professor\"\n"
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prefix \"Junior \" \"Professor\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : string = \"Senior Professor\"\n"
      ]
     },
     "execution_count": 131,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "promote \"Professor\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "A short form for the definition of `prefix` is simply to pass multiple\n",
    "arguments to the function definition.  The following two definitions\n",
    "are equivalent in OCaml:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val prefix : string -> string -> string = <fun>\n"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let prefix = fun a -> fun b -> a ^ b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val prefix : string -> string -> string = <fun>\n"
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let prefix a b = a ^ b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Currying is the technique of expressing a function taking multiple arguments as nested functions, each taking a single argument.\n",
    "The `fun`-notation lets us package `n*2` as the function\n",
    "`fun n -> n * 2`, but what if there are several variables, as in\n",
    "`fun n -> n * 2 + k`?  A function of two arguments could be coded using\n",
    "pattern-matching on pairs, writing `fun (n, k) -> n * 2 + k`.\n",
    "\n",
    "Currying is an alternative, where we _nest_ the `fun`-notation:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int -> int -> int = <fun>\n"
      ]
     },
     "execution_count": 134,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fun k -> fun n -> n * 2 + k"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Applying this curried function to the argument 1 yields another function, in which `k` has been replaced by 1:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val fn : int -> int -> int = <fun>\n"
      ]
     },
     "execution_count": 135,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let fn = fun k -> fun n -> n * 2 + k"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val fn' : int -> int = <fun>\n"
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let fn' = fn 1 (* n * 2 + 1 *)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int = 7\n"
      ]
     },
     "execution_count": 137,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fn' 3  (* 3 * 2 + 1 *)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "And this function, when applied to 3, yields the result 7. The two arguments are supplied one after another.\n",
    "\n",
    "The example on the slide is similar but refers to the expression `a^b`,\n",
    "where `^` is the infix operator for string concatenation. Function `promote` binds the first argument of `prefix` to\n",
    "`\"Professor\"`; the resulting function prefixes that title\n",
    "to any string to which it is applied.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Shorthand for Curried Functions\n",
    "\n",
    "\n",
    "\n",
    "A function-returning function is just a function of two arguments.\n",
    "\n",
    "This curried function syntax is nicer than nested `fun` binders:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val prefix : string -> string -> string = <fun>\n"
      ]
     },
     "execution_count": 138,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let prefix a b = a ^ b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val dub : string -> string = <fun>\n"
      ]
     },
     "execution_count": 139,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let dub = prefix \"Sir \""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Curried functions allows _partial application_ (to the first argument).\n",
    "\n",
    "In OCaml, an $n$-argument curried function `f` can be declared using the syntax:\n",
    "\n",
    "$$\\tt let \\; f \\; x_1 \\: \\ldots \\: x_{n} \\: = \\: E$$\n",
    "\n",
    "and applied using the syntax:\n",
    "\n",
    "$$\\tt \\; E_1 \\; \\ldots \\; E_n$$\n",
    "\n",
    "If `f` is not recursive, then it is equivalent to the function expressed via nesting as follows:\n",
    "\n",
    "$$\\tt fun \\; x_1 \\; \\rightarrow \\cdots \\rightarrow fun \\; x_{n} \\rightarrow E) $$\n",
    "\n",
    "We now have two ways of expressing functions of multiple arguments: either by\n",
    "passing a pair of arguments or by currying.  Currying allows _partial application_\n",
    "which is useful when fixing the first argument yields a function\n",
    "that is interesting in its own right.  An example from mathematics is the\n",
    "function $x^y$, where fixing $y=2$ yields a function in $x$ alone, namely\n",
    "squaring. Similarly, $y=3$ yields cubing, while $y=1$ yields the identity\n",
    "function.\n",
    "\n",
    "Though the function `hd` (which returns the head of a list) is not\n",
    "curried, it may be used with the curried application syntax in some\n",
    "expressions:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : string = \"Sir Hamilton\"\n"
      ]
     },
     "execution_count": 140,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "List.hd [dub; promote] \"Hamilton\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Here `List.hd` is applied to a list of functions, and the resulting function\n",
    "`dub` is then applied to the string `\"Hamilton\"`.  The idea of\n",
    "executing code stored in data structures reaches its full development in\n",
    "_object-oriented_ programming, like in Java.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Partial Application: A Curried Insertion Sort\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val insort : ('a -> 'a -> bool) -> 'a list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 141,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let insort lessequal =\n",
    "  let rec ins = function\n",
    "    | x, [] -> [x]\n",
    "    | x, y::ys ->\n",
    "        if lessequal x y then\n",
    "           x :: y :: ys\n",
    "        else\n",
    "           y :: ins (x, ys)\n",
    "  in\n",
    "  let rec sort = function\n",
    "    | [] -> []\n",
    "    | x::xs -> ins (x, sort xs)\n",
    "  in\n",
    "  sort"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The sorting functions we discussed in earlier lectures are coded to sort real\n",
    "numbers.  They can be generalised to an arbitrary ordered type by passing the\n",
    "ordering predicate `leq` as an argument.\n",
    "\n",
    "Functions `ins` and `sort` are declared locally, referring to `lessequal`.\n",
    "Though it may not be obvious, `insort` is a curried function.  Given its first\n",
    "argument, a predicate for comparing some particular type of items, it returns\n",
    "the function `sort` for sorting lists of that type of items.\n",
    "\n",
    "Some examples of its use:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int list = [3; 5; 8; 9]\n"
      ]
     },
     "execution_count": 142,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "insort (<=) [5; 3; 9; 8]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : string list = [\"a\"; \"bee\"; \"bitten\"; \"on\"]\n"
      ]
     },
     "execution_count": 143,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "insort (<=) [\"bitten\"; \"on\"; \"a\"; \"bee\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int list = [9; 8; 5; 3]\n"
      ]
     },
     "execution_count": 144,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "insort (>=) [5; 3; 9; 8]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "An obscure point: the syntax `(<=)` denotes the comparison operator as a\n",
    "function, which is then given to `insort`.  Passing the relation $\\geq$ for\n",
    "`lessequal` gives a decreasing sort.  This is no coding trick; it is justified\n",
    "in mathematics, since if $\\leq$ is a partial ordering then so is $\\geq$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## map: the “Apply to All” Function\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val map : ('a -> 'b) -> 'a list -> 'b list = <fun>\n"
      ]
     },
     "execution_count": 145,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec map f = function\n",
    "  | [] -> []\n",
    "  | x::xs -> (f x) :: map f xs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : string list = [\"Hippy\"; \"Hoppy\"]\n"
      ]
     },
     "execution_count": 146,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "map (fun s -> s ^ \"ppy\") [\"Hi\"; \"Ho\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int list list = [[2]; [4; 6]]\n"
      ]
     },
     "execution_count": 147,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "map (map double) [[1]; [2; 3]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The functional `map` applies a function to every element of a list,\n",
    "returning a list of the function’s results.  “Apply to all” is a fundamental\n",
    "operation and we shall see several applications of it below.  We\n",
    "again see the advantages of `fun`-notation, currying and\n",
    "`map`.  If we did not have them, the first example on the slide\n",
    "would require a preliminary function declaration:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val sillylist : string list -> string list = <fun>\n"
      ]
     },
     "execution_count": 148,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec sillylist = function\n",
    "  | [] -> []\n",
    "  | s::ss -> (s ^ \"ppy\") :: sillylist ss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "An expression containing several applications of functionals---such as our\n",
    "second example---can abbreviate a long series of declarations.  Sometimes this\n",
    "coding style is cryptic, but it can be clear as crystal.  Treating functions\n",
    "as values lets us capture common program structures once and for all.\n",
    "\n",
    "In the second example, `double` is the obvious integer doubling function we\n",
    "defined earlier.  Note that \\texttt{map} is a built-in OCaml function in the\n",
    "form of `List.map`.  OCaml’s standard library includes, among much else, many\n",
    "list functions.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example: Matrix Transpose\n",
    "\n",
    "\n",
    "\n",
    "$$\n",
    "\\begin{pmatrix}\n",
    "           a   & b & c \\\\\n",
    "           d   & e & f\n",
    "   \\end{pmatrix}^T =\n",
    "   \\begin{pmatrix}\n",
    "        a & d     \\\\\n",
    "        b & e     \\\\\n",
    "        c & f\n",
    "\\end{pmatrix}\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val transp : 'a list list -> 'a list list = <fun>\n"
      ]
     },
     "execution_count": 149,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec transp = function\n",
    "  | []::_ -> []\n",
    "  | rows -> (map List.hd rows) ::\n",
    "            (transp (map List.tl rows))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "A matrix can be viewed as a list of rows, each row a list of matrix elements.\n",
    "This representation is not especially efficient compared with the conventional\n",
    "one (using arrays).  Lists of lists turn up often, though, and we can see how\n",
    "to deal with them by taking familiar matrix operations as examples.\n",
    "_ML for the Working Programmer_ goes as far as Gaussian elimination,\n",
    "which presents surprisingly few difficulties.\n",
    "\n",
    "The transpose of the matrix\n",
    "$\\left(\\begin{smallmatrix} a & b & c \\\\\n",
    "                           d & e & f\\end{smallmatrix}\\right)$\n",
    "is\n",
    "$\\left(\\begin{smallmatrix}\n",
    "        a & d     \\\\\n",
    "        b & e     \\\\\n",
    "        c & f\n",
    "   \\end{smallmatrix}\\right)$,\n",
    "which in OCaml corresponds to the following transformation on lists of lists:\n"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "[[a; b; c]; [d; e; f]] => [[a; d]; [b; e]; [c; f]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The workings of function `transp` are simple.  If `rows` is the\n",
    "matrix to be transposed, then `map hd` extracts its first column and\n",
    "`map tl` extracts its second column:\n"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "map hd rows => [a; d]\n",
    "map tl rows => [[b; c]; [e; f]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "A recursive call transposes the latter matrix, which is then given the column\n",
    "`[a; d]` as its first row.\n",
    "The two functions expressed using `map` would otherwise have to be declared\n",
    "separately.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Review of Matrix Multiplication\n",
    "\n",
    "\n",
    "\n",
    "$$\n",
    "\\begin{pmatrix} A_1 & \\cdots & A_k \\end{pmatrix}  \\cdot\n",
    "   \\begin{pmatrix}\n",
    "        B_1 \\\\ \\vdots \\\\ B_k\n",
    "   \\end{pmatrix}   =\n",
    "   \\begin{pmatrix}\n",
    "        A_1 B_1 + \\cdots + A_k B_k\n",
    "   \\end{pmatrix}\n",
    "$$\n",
    "\n",
    "The right side is the _vector dot product_ $\\vec{A}\\cdot \\vec{B}$.\n",
    "Repeat for each _row_ of $A$ and _column_ of $B$.\n",
    "\n",
    "The _dot product_ of two vectors is\n",
    "$$ (a_1,\\ldots,a_k) \\cdot (b_1,\\ldots,b_k) = a_1b_1 + \\cdots + a_kb_k $$\n",
    "\n",
    "A simple case of matrix multiplication is when $A$ consists of a single row\n",
    "and $B$ consists of a single column.  Provided $A$ and $B$ contain the same\n",
    "number $k$ of elements, multiplying them yields a $1\\times1$ matrix whose\n",
    "single element is the dot product shown above.\n",
    "\n",
    "If $A$ is an $m\\times k$ matrix and $B$ is a $k\\times n$ matrix\n",
    "then $A\\times B$ is an $m\\times n$ matrix.\n",
    "For each $i$ and $j$, the $(i,j)$ element of $A\\times B$ is the dot\n",
    "product of row $i$ of $A$ with column $j$ of $B$.\n",
    "\n",
    "$$\n",
    "\\begin{pmatrix}\n",
    "        2 & 0 \\\\\n",
    "        3 &-1 \\\\\n",
    "        0 & 1 \\\\\n",
    "        1 & 1\n",
    "   \\end{pmatrix}\n",
    "   \\begin{pmatrix}\n",
    "        1 & 0 & 2 \\\\\n",
    "        4 &-1 & 0\n",
    "   \\end{pmatrix}   =\n",
    "   \\begin{pmatrix}\n",
    "        2 & 0 & 4 \\\\\n",
    "       -1 & 1 & 6 \\\\\n",
    "        4 &-1 & 0 \\\\\n",
    "        5 &-1 & 2\n",
    "\\end{pmatrix}\n",
    "$$\n",
    "\n",
    "The (1, 1) element above is computed by\n",
    "$$ (2,0)\\cdot(1,4) = 2\\times1 + 0\\times4 = 2. $$\n",
    "\n",
    "Coding matrix multiplication in a conventional programming language usually\n",
    "involves three nested loops.  It is hard to avoid mistakes in the subscripting,\n",
    "which often runs slowly due to redundant internal calculations.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Matrix Multiplication in OCaml\n",
    "\n",
    "\n",
    "\n",
    "_Dot product_ of two vectors---a _curried function_\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "File \"[150]\", line 2, characters 2-86:\n",
      "2 | ..match xs, ys with\n",
      "3 |   | [], [] -> 0.0\n",
      "4 |   | x::xs, y::ys ->  (x *. y) +. (dotprod xs ys)\n",
      "Warning 8: this pattern-matching is not exhaustive.\n",
      "Here is an example of a case that is not matched:\n",
      "((_::_, [])|([], _::_))\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "val dotprod : float list -> float list -> float = <fun>\n"
      ]
     },
     "execution_count": 150,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec dotprod xs ys =\n",
    "  match xs, ys with\n",
    "  | [], [] -> 0.0\n",
    "  | x::xs, y::ys ->  (x *. y) +. (dotprod xs ys)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "_Matrix product_\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val matprod : float list list -> float list list -> float list list = <fun>\n"
      ]
     },
     "execution_count": 151,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec matprod arows brows =\n",
    "  let cols = transp brows in\n",
    "  map (fun row -> map (dotprod row) cols) arows"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The `transp brows` converts $B$ into a list of columns.  It yields a\n",
    "list, whose elements are the columns of $B$.  Each row of $A\\times B$ is\n",
    "obtained by multiplying a row of $A$ by the columns of $B$.\n",
    "\n",
    "Because `dotprod` is curried, it can be applied to a row of $A$.  The\n",
    "resulting function is applied to all the columns of $B$.  We have another\n",
    "example of currying and partial application.\n",
    "\n",
    "The outer `map` applies `dotprod` to each row of $A$.  The inner\n",
    "`map`, using `fun`-notation, applies `dotprod row` to each\n",
    "column of $B$.  Compare with the version in _ML for the Working\n",
    "  Programmer_ (page 89) which does not use `map` and requires two\n",
    "additional function declarations.\n",
    "\n",
    "In the dot product function, the two vectors must have the same length.\n",
    "Otherwise, exception `Match_failure` is raised.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## List Functionals for Predicates\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val exists : ('a -> bool) -> 'a list -> bool = <fun>\n"
      ]
     },
     "execution_count": 152,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec exists p = function\n",
    "| [] -> false\n",
    "| x::xs -> (p x) || (exists p xs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val filter : ('a -> bool) -> 'a list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 153,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec filter p = function\n",
    "| [] -> []\n",
    "| x::xs ->\n",
    "    if p x then\n",
    "      x :: filter p xs\n",
    "    else\n",
    "      filter p xs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "A _predicate_ is a _boolean-valued_ function.\n",
    "\n",
    "The functional `exists` transforms a predicate into a predicate over\n",
    "lists.  Given a list, `exists p` tests whether or not some list element\n",
    "satisfies `p` (making it return `true`).  If it finds one, it stops\n",
    "searching immediately, thanks to the behaviour of the lazy `||` operator.\n",
    "\n",
    "Dually, we have a functional to test whether all list elements satisfy the\n",
    "predicate.  If it finds a counterexample then it, too, stops searching.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val all : ('a -> bool) -> 'a list -> bool = <fun>\n"
      ]
     },
     "execution_count": 154,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec all p = function\n",
    "| [] -> true\n",
    "| x::xs -> (p x) && all p xs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The `filter` functional is related to `map`.  It applies a predicate to all the\n",
    "list elements, but instead of returning the resulting values (which could only\n",
    "be `true` or `false`), it returns the list of elements satisfying the\n",
    "predicate.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Applications of the Predicate Functionals\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val member : 'a -> 'a list -> bool = <fun>\n"
      ]
     },
     "execution_count": 155,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let member y xs =\n",
    "  exists (fun x -> x=y) xs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val inter : 'a list -> 'a list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 156,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let inter xs ys =\n",
    "  filter (fun x -> member x ys) xs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "_Testing whether two lists have no common elements_\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val disjoint : 'a list -> 'a list -> bool = <fun>\n"
      ]
     },
     "execution_count": 157,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let disjoint xs ys =\n",
    "  all (fun x -> all (fun y -> x<>y) ys) xs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The Lists lecture presented the function `member`, which tests whether a\n",
    "specified value can be found as a list element, and `inter`, which returns the\n",
    "“intersection” of two lists: the list of elements they have in common.\n",
    "\n",
    "But remember: the purpose of list functionals is not to replace the\n",
    "declarations of popular functions, which probably are available already.  It is\n",
    "to eliminate the need for separate declarations of ad-hoc functions.  When they\n",
    "are nested, like the calls to \\texttt{all} in \\texttt{disjoint} above, the\n",
    "inner functions are almost certainly one-offs, not worth declaring separately.\n",
    "\n",
    "Our primitives themselves can be seen as a programming language.  Part of the\n",
    "task of programming is to extend our programming language with notation for\n",
    "solving the problem at hand.  The levels of notation that we define should\n",
    "correspond to natural levels of abstraction in the problem domain.\n",
    "\n",
    "Historical Note:\n",
    "Alonzo Church’s $\\lambda$-calculus gave a simple syntax, $\\lambda$-notation,\n",
    "for expressing functions.  It is the direct precursor of OCaml’s\n",
    "`fun`-notation.  It was soon shown that his system was equivalent in\n",
    "computational power to Turing machines, and _Church’s thesis_ states that\n",
    "this defines precisely the set of functions that can be computed effectively.\n",
    "\n",
    "The $\\lambda$-calculus had a tremendous influence on the design of functional\n",
    "programming languages.  McCarthy’s Lisp was something of a false start; it\n",
    "interpreted variable binding incorrectly, an error that stood for some 20\n",
    "years.  But in 1966, Peter Landin (of Queen Mary College, University of London)\n",
    "sketched out the main features of functional languages.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 8.1\n",
    "\n",
    "\n",
    "What does the following function do, and what are its uses?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val sw : ('a -> 'b -> 'c) -> 'b -> 'a -> 'c = <fun>\n"
      ]
     },
     "execution_count": 158,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let sw f x y = f y x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 8.2\n",
    "\n",
    "\n",
    "There are many ways of combining orderings. The `lexicographic ordering` uses two keys for\n",
    "comparisons. It is specified by\n",
    "$$(x',y')<(x,y)\\iff x'<x \\vee (x'=x \\wedge y'<y).$$\n",
    "Write an OCaml function to lexicographically combine two orderings, supplied as functions. Explain\n",
    "how it allows function `insort` to sort a list of pairs.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 8.3\n",
    "\n",
    "\n",
    "Without using `map` write a function `map2` such that `map2 f` is equivalent to `map (map f)`. The\n",
    "obvious solution requires declaring two recursive functions. Try to get away with only one by\n",
    "exploiting nested pattern-matching.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 8.4\n",
    "\n",
    "\n",
    "The type `'a option`, declared below, can be viewed as a type of lists having at most one element.\n",
    "(It is typically used as an alternative to exceptions.) Declare an analogue of the function `map`\n",
    "for type `'a option`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "type 'a option = None | Some of 'a\n"
      ]
     },
     "execution_count": 159,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type 'a option = None | Some of 'a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 8.5\n",
    "\n",
    "\n",
    "Recall the making change function of Lecture 4:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val change : int list -> int -> int list list = <fun>\n"
      ]
     },
     "execution_count": 160,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec change till amt =\n",
    "  if amt = 0 then\n",
    "    [ [] ]\n",
    "  else\n",
    "    match till with\n",
    "    | [] -> []\n",
    "    | c::till ->\n",
    "        if amt < c then\n",
    "          change till amt\n",
    "        else\n",
    "          let rec allc = function\n",
    "            | [] -> []\n",
    "            | cs :: css -> (c::cs) :: allc css\n",
    "          in\n",
    "            allc (change (c::till) (amt - c)) @\n",
    "                  change till amt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Function `allc` applies the function ‘cons a `c`’ to every element of a list. Eliminate it by\n",
    "declaring a curried cons function and applying `map`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lecture 9: Sequences, or Lazy Lists\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A Pipeline\n",
    "\n",
    "\n",
    "\n",
    "$$\n",
    "\\framebox{\\emph{Producer}} \\to \\framebox{\\emph{Filter}} \\to\\cdots\\to\n",
    " \\framebox{\\emph{Filter}} \\to \\framebox{\\emph{Consumer}}\n",
    "$$\n",
    "\n",
    "- Produce sequence of items\n",
    "- Filter sequence in stages\n",
    "- Consume results as needed\n",
    "- _Lazy lists_ join the stages together\n",
    "\n",
    "Two types of program can be distinguished.  A sequential program\n",
    "accepts a problem to solve, processes for a while, and finally terminates\n",
    "with its result.  A typical example is the huge numerical simulations that are\n",
    "run on supercomputers.  Most of our OCaml functions also fit this model.\n",
    "\n",
    "At the other extreme are _reactive_ programs, whose job is to interact\n",
    "with the environment.  They communicate constantly during their operation and\n",
    "run for as long as is necessary.  A typical example is the software that\n",
    "controls many modern aircraft.  Reactive programs often consist of\n",
    "_concurrent processes_ running at the same time and communicating with\n",
    "one another.\n",
    "\n",
    "Concurrency is too difficult to consider in this course, but we can model\n",
    "simple pipelines such as that shown above.  The _Producer_ represents one\n",
    "or more sources of data, which it outputs as a stream.  The _Filter_\n",
    "stages convert the input stream to an output stream, perhaps consuming several\n",
    "input items to yield a single output item.  The _Consumer_ takes as many\n",
    "elements as necessary.\n",
    "\n",
    "The Consumer drives the pipeline: nothing is computed except in response to\n",
    "its demand for an additional datum.  Execution of the Filter stages is\n",
    "interleaved as required for the computation to go through.  The programmer\n",
    "sets up the data dependencies but has no clear idea of what happens when.  We\n",
    "have the illusion of concurrent computation.\n",
    "\n",
    "The Unix operating system provides similar ideas through its _pipes_ that\n",
    "link processes together.  In OCaml, we can model pipelines using _lazy lists_.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lazy Lists (or Streams)\n",
    "\n",
    "\n",
    "\n",
    "- Lists of possibly _infinite_ length\n",
    "- Elements _computed upon demand_\n",
    "- _Avoids waste_ if there are many solutions\n",
    "- _Infinite_ values are a useful abstraction\n",
    "\n",
    "In OCaml, we can implement laziness by _delaying evaluation_ of the tail of\n",
    "the list.\n",
    "\n",
    "Lazy lists have practical uses.  Some algorithms, like making change, can\n",
    "yield many solutions when only a few are required.  Sometimes the original\n",
    "problem concerns infinite series: with lazy lists, we can pretend they really\n",
    "exist!\n",
    "\n",
    "We are now dealing with _infinite_ (or at least unbounded) computations.\n",
    "A potentially infinite source of data is processed one element at a time, upon\n",
    "demand.  Such programs are harder to understand than terminating ones and have\n",
    "more ways of going wrong.\n",
    "\n",
    "Some purely functional languages, such as Haskell, use lazy evaluation\n",
    "everywhere.  Even the if-then-else construct can be a function, and all lists\n",
    "are lazy.  In OCaml, we can declare a type of lists such that evaluation of the\n",
    "tail does not occur until demanded.  _Delayed_ evaluation is weaker than\n",
    "_lazy_ evaluation, but it is good enough for our purposes and often the\n",
    "best compromise for performance and memory usage.\n",
    "\n",
    "The traditional word “stream” is reserved in OCaml parlance for\n",
    "input/output channels.  Let us call lazy lists _sequences_ instead.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lazy Lists in OCaml\n",
    "\n",
    "\n",
    "\n",
    "- The empty tuple `()` and its _type_ `unit`\n",
    "- Delayed version of $E$ is `fun () -> E`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "type 'a seq = Nil | Cons of 'a * (unit -> 'a seq)\n"
      ]
     },
     "execution_count": 161,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type 'a seq =\n",
    "| Nil\n",
    "| Cons of 'a * (unit -> 'a seq)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "File \"[162]\", line 1, characters 9-26:\n",
      "1 | let head (Cons (x, _)) = x\n",
      "             ^^^^^^^^^^^^^^^^^\n",
      "Warning 8: this pattern-matching is not exhaustive.\n",
      "Here is an example of a case that is not matched:\n",
      "Nil\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "val head : 'a seq -> 'a = <fun>\n"
      ]
     },
     "execution_count": 162,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let head (Cons (x, _)) = x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "File \"[163]\", line 1, characters 9-31:\n",
      "1 | let tail (Cons (_, xf)) = xf ()\n",
      "             ^^^^^^^^^^^^^^^^^^^^^^\n",
      "Warning 8: this pattern-matching is not exhaustive.\n",
      "Here is an example of a case that is not matched:\n",
      "Nil\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "val tail : 'a seq -> 'a seq = <fun>\n"
      ]
     },
     "execution_count": 163,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let tail (Cons (_, xf)) = xf ()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "$\\tt Cons(x, xf)$ has _head_ $x$ and _tail function_ $xf$\n",
    "\n",
    "The primitive OCaml type `unit` has one element, which is\n",
    "written `()`.  This element may be regarded as a 0-tuple, and\n",
    "`unit` as the nullary Cartesian product.  (Think of the connection\n",
    "between multiplication and the number 1.)\n",
    "\n",
    "The empty tuple serves as a placeholder in situations where no information is\n",
    "required.  It may:\n",
    "- appear in a data structure.  For example, a `unit`-valued dictionary represents a set of keys.\n",
    "- be the argument of a function, where its effect is to _delay evaluation_.\n",
    "- be the argument or result of a procedure. (see the Procedural Programming section)\n",
    "\n",
    "The empty tuple, like all tuples, is a constructor and is allowed in patterns;\n",
    "for example: `let f () = ...`\n",
    "\n",
    "In particular $\\tt fun \\, () \\rightarrow E$ is the function that takes an argument of\n",
    "type `unit` and returns the value of $E$ as its result.  Expression $E$\n",
    "is not evaluated until the function is called, even though the only possible\n",
    "argument is `()`.  The function simply delays the evaluation of $E$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Infinite Sequence: $k$, $k+1$, $k+2$, …\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val from : int -> int seq = <fun>\n"
      ]
     },
     "execution_count": 164,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec from k = Cons (k, fun () -> from (k+1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val it : int seq = Cons (1, <fun>)\n"
      ]
     },
     "execution_count": 165,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let it = from 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val it : int seq = Cons (2, <fun>)\n"
      ]
     },
     "execution_count": 166,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let it = tail it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val it : int seq = Cons (3, <fun>)\n"
      ]
     },
     "execution_count": 167,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let it = tail it"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Function `from` constructs the infinite sequence of integers starting\n",
    "from $k$.  Execution terminates because of the `fun` enclosing the\n",
    "recursive call.  OCaml displays the tail of a sequence as `fun`, which\n",
    "stands for some function value.  Each call to `tail` generates the next\n",
    "sequence element.  We could do this forever.\n",
    "\n",
    "This example is of little practical value because the cost of computing a\n",
    "sequence element will be dominated by that of creating the dummy function.\n",
    "Lazy lists tend to have high overheads.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Consuming a Sequence\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val get : int -> 'a seq -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 168,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec get n s =\n",
    "  if n = 0 then\n",
    "    []\n",
    "  else\n",
    "    match s with\n",
    "    | Nil -> []\n",
    "    | Cons (x, xf) -> x :: get (n-1) (xf ())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The above code gets the first $n$ elements as a list.\n",
    "`xf ()` _forces_ evaluation.\n",
    "\n",
    "The function `get` converts a sequence to a list.  It takes the\n",
    "first $n$ elements; it takes all of them if $n<0$, which can terminate only if\n",
    "the sequence is finite.\n",
    "\n",
    "In the third line of `get`, the expression `xf()` calls the tail\n",
    "function, demanding evaluation of the next element.  This operation is called\n",
    "_forcing_ the list.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sample Evaluation\n",
    "\n",
    "\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "\\tt get(2,\\, from \\; 60) \\\\\n",
    "\\tt get(2,\\, Cons(6, fun \\; () \\rightarrow from \\; (6+1))) \\\\\n",
    "\\tt 6 :: get(1,\\, from \\; (6+1)) \\\\\n",
    "\\tt 6 :: get(1,\\, Cons \\; (7,\\, fun \\; () \\rightarrow from \\; (7+1))) \\\\\n",
    "\\tt 6 :: 7 :: get(0,\\, Cons \\; (8,\\, fun \\; () \\rightarrow from \\; (8+1))) \\\\\n",
    "\\tt 6 :: 7 :: [] \\\\\n",
    "\\tt [6; 7]\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "Here we ask for two elements of the infinite sequence.  In fact, three\n",
    "elements are computed: 6, 7 and 8.  Our implementation is slightly too eager.\n",
    "A more complicated `type` declaration could avoid this problem.\n",
    "Another problem is that if one repeatedly examines some particular list\n",
    "element using forcing, that element is repeatedly evaluated.  In a lazy\n",
    "programming language, the result of the first evaluation would be stored for\n",
    "later reference.  To get the same effect in OCaml requires the use of\n",
    "references.\n",
    "\n",
    "We should be grateful that the potentially infinite computation is kept\n",
    "finite.  The tail of the original sequence even contains the unevaluated\n",
    "expression 6+1.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Joining Two Sequences\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val appendq : 'a seq -> 'a seq -> 'a seq = <fun>\n"
      ]
     },
     "execution_count": 169,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec appendq xq yq =\n",
    "  match xq with\n",
    "  | Nil -> yq \n",
    "  | Cons (x, xf) ->\n",
    "      Cons(x, fun () -> appendq (xf ()) yq)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "A more fair alternative:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val interleave : 'a seq -> 'a seq -> 'a seq = <fun>\n"
      ]
     },
     "execution_count": 170,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec interleave xq yq =\n",
    "  match xq with\n",
    "  | Nil -> yq\n",
    "  | Cons (x, xf) ->\n",
    "      Cons (x, fun () -> interleave yq (xf ()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Most list functions and functionals have analogues on sequences, but strange\n",
    "things can happen.  Can an infinite list be reversed?\n",
    "\n",
    "Function `appendq` is precisely the same idea as `append`\n",
    "from the Lists lecture; it concatenates two sequences.  If the first\n",
    "argument is infinite, then `appendq` never gets to its second argument,\n",
    "which is lost.  Concatenation of infinite sequences is not terribly\n",
    "interesting.\n",
    "\n",
    "The function `interleave` avoids this problem by exchanging the two\n",
    "arguments in each recursive call.  It combines the two lazy lists, losing no\n",
    "elements.  Interleaving is the right way to combine two potentially infinite\n",
    "information sources into one.\n",
    "\n",
    "In both function declarations, observe that each `xf ()` is enclosed\n",
    "within a ${\\tt fun () \\rightarrow \\ldots}$.  Each _force_ is enclosed within a\n",
    "_delay_.  This practice makes the functions lazy.  A force not enclosed\n",
    "in a delay, as in `get` above, runs the risk of evaluating the sequence\n",
    "in full.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Functionals for Lazy Lists\n",
    "\n",
    "\n",
    "\n",
    "Filtering lazy lists:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val filterq : ('a -> bool) -> 'a seq -> 'a seq = <fun>\n"
      ]
     },
     "execution_count": 171,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec filterq p = function\n",
    "| Nil -> Nil\n",
    "| Cons (x, xf) ->\n",
    "    if p x then\n",
    "      Cons (x, fun () -> filterq p (xf ()))\n",
    "    else\n",
    "      filterq p (xf ())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The infinite sequence $x$, $f(x)$, $f(f(x))$, …\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val iterates : ('a -> 'a) -> 'a -> 'a seq = <fun>\n"
      ]
     },
     "execution_count": 172,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec iterates f x =\n",
    "  Cons (x, fun () -> iterates f (f x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The functional `filterq` demands elements of `xq` until it finds\n",
    "one satisfying `p`.  (Recall `filter`, the analogous operation for ordinary lists.)  It\n",
    "contains a _force_ not protected by a _delay_.  If `xq` is\n",
    "infinite and contains no satisfactory element, then `filtering` runs\n",
    "forever.\n",
    "\n",
    "The functional `iterates` generalises `from`.  It creates the\n",
    "next element not by adding one but by calling the function `f`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Numerical Computations on Infinite Sequences\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val next : float -> float -> float = <fun>\n"
      ]
     },
     "execution_count": 173,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let next a x = (a /. x +. x) /. 2.0 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Close enough?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "File \"[174]\", line 3, characters 4-145:\n",
      "3 | ....match xf () with\n",
      "4 |     | Cons (y, yf) ->\n",
      "5 |         if abs_float (x -. y) <= eps then\n",
      "6 |           y\n",
      "7 |         else\n",
      "8 |           within eps (Cons (y, yf))\n",
      "Warning 8: this pattern-matching is not exhaustive.\n",
      "Here is an example of a case that is not matched:\n",
      "Nil\n",
      "File \"[174]\", line 1, characters 21-193:\n",
      "1 | .....................function\n",
      "2 | | Cons (x, xf) ->\n",
      "3 |     match xf () with\n",
      "4 |     | Cons (y, yf) ->\n",
      "5 |         if abs_float (x -. y) <= eps then\n",
      "6 |           y\n",
      "7 |         else\n",
      "8 |           within eps (Cons (y, yf))\n",
      "Warning 8: this pattern-matching is not exhaustive.\n",
      "Here is an example of a case that is not matched:\n",
      "Nil\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "val within : float -> float seq -> float = <fun>\n"
      ]
     },
     "execution_count": 174,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec within eps = function\n",
    "| Cons (x, xf) ->\n",
    "    match xf () with\n",
    "    | Cons (y, yf) ->\n",
    "        if abs_float (x -. y) <= eps then\n",
    "          y\n",
    "        else\n",
    "          within eps (Cons (y, yf))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Square Roots:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val root : float -> float = <fun>\n"
      ]
     },
     "execution_count": 175,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let root a = within 1e6 (iterates (next a) 1.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The _Newton-Raphson method_ is widely used for computing square roots.\n",
    "The infinite series $x_0, (a/x_0+x_0)/2, \\ldots{}$ converges rapidly to $\\sqrt{a}$.\n",
    "The initial approximation, $x_0$, is typically retrieved from a table, and is accurate enough\n",
    "that only a few iterations of the method are necessary.\n",
    "Calling `iterates (next a) x0` generates the _infinite series_ of\n",
    "approximations to the square root of $a$ using the Newton-Raphson method.\n",
    "To compute $\\sqrt2$, the resulting series begins 1, 1.5, 1.41667, 1.4142157, 1.414213562, …,\n",
    "and this last figure is already accurate to 10 significant digits!\n",
    "\n",
    "Function `within` searches down the lazy list for two points whose\n",
    "difference is less than `eps`.  It tests their absolute difference.\n",
    "Relative difference and other “close enough” tests can be coded.  Such\n",
    "components can be used to implement other numerical functions directly as\n",
    "functions over sequences.  The point is to build programs from small,\n",
    "interchangeable parts.\n",
    "\n",
    "Function `root` uses `within, `iterates` and `next` to\n",
    "to apply Newton-Raphson with a tolerance of $10^{-6}$\n",
    "and a (poor) initial approximation of 1.0.\n",
    "\n",
    "This treatment of numerical computation has received some attention in the\n",
    "research literature; a recurring example is Richardson extrapolation.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 9.1\n",
    "\n",
    "\n",
    "Code an analogue of `map` for sequences.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 9.2\n",
    "\n",
    "\n",
    "Consider the list function `concat`, which concatenates a list of lists to form a single list. Can\n",
    "it be generalised to concatenate a sequence of sequences? What can go wrong?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val concat : 'a list list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 176,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec concat = function\n",
    "| [] ->\n",
    "    []\n",
    "| l::ls ->\n",
    "    l @ concat ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 9.3\n",
    "\n",
    "\n",
    "Code a function to make change using lazy lists, delivering the sequence of all possible ways of\n",
    "making change. Using sequences allows us to compute solutions one at a time when there exists an\n",
    "astronomical number. Represent lists of coins using ordinary lists. (_Hint_: to  benefit from\n",
    "laziness you may need to pass around the sequence of alternative solutions as a function of type\n",
    "`unit -> (int list) seq`.)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 9.4\n",
    "\n",
    "\n",
    "A _lazy binary tree_ is either empty or is a branch containing a label and two lazy binary trees,\n",
    "possibly to infinite depth. Present an OCaml datatype to represent lazy binary trees, along with a\n",
    "function that accepts a lazy binary tree and produces a lazy list that contains all of the tree’s\n",
    "labels. (Taken from the exam question 2008 Paper 1 Question 5.)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 9.5\n",
    "\n",
    "\n",
    "Code the lazy list whose elements are all ordinary lists of zeroes and ones, namely\n",
    "`[]; [0]; [1]; [0; 0]; [0; 1]; [1; 0]; [1; 1]; [0; 0; 0]; `….  (Taken from the exam question\n",
    "2003 Paper 1 Question 5.)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 9.6\n",
    "\n",
    "\n",
    "(Continuing the previous exercise.)\n",
    "A _palindrome_ is a list that equals its own reverse. Code the lazy list whose elements are all\n",
    "palindromes of 0s and 1s, namely\n",
    "`[]; [0]; [1]; [0; 0]; [0; 0; 0]; [0; 1; 0]; [1; 1]; [1; 0; 1]; [1; 1; 1]; [0; 0; 0; 0]; `, …. You\n",
    "may take the reversal function `List.rev` as given.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lecture 10: Queues and Search Strategies\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Breadth-First v Depth-First Tree Traversal\n",
    "\n",
    "\n",
    "\n",
    "- binary trees as _decision trees_\n",
    "- look for _solution nodes_\n",
    "  * Depth-first: search one subtree in full before moving on\n",
    "  * Breadth-first: search all nodes at level $k$ before moving to $k+1$\n",
    "- finds _all_ solutions --- nearest first!\n",
    "\n",
    "Preorder, inorder and postorder tree traversals all have something in common:\n",
    "they are depth-first.  At each node, the left subtree is entirely\n",
    "traversed before the right subtree.  Depth-first traversals are easy to code\n",
    "and can be efficient, but they are ill-suited for some problems.\n",
    "\n",
    "Suppose the tree represents the possible moves in a puzzle, and the purpose\n",
    "of the traversal is to search for a node containing a solution.  Then a\n",
    "depth-first traversal may find one solution node deep in the left subtree,\n",
    "when another solution is at the very top of the right subtree.  Often we\n",
    "want the shortest path to a solution.\n",
    "\n",
    "Suppose the tree is _infinite_ or simply extremely large.  Depth-first search\n",
    "is almost useless with such trees, for if the left subtree is infinite then the\n",
    "search will never reach the right subtree.  OCaml can represent infinite trees by\n",
    "the means discussed in the lecture on laziness. Another tree representation (suitable\n",
    "for solving solitaire, for example) is by a function `next : pos -> pos list`,\n",
    "which maps a board position to a list of the positions possible after\n",
    "the next move.  For simplicity, the examples below use the OCaml datatype\n",
    "`tree`, which has only finite trees.\n",
    "\n",
    "A _breadth-first_ traversal explores the nodes horizontally rather than\n",
    "vertically.  When visiting a node, it does not traverse the subtrees until\n",
    "it has visited all other nodes at the current depth.  This is easily\n",
    "implemented by keeping a list of trees to visit.  Initially, this list\n",
    "consists of one element: the entire tree.  Each iteration removes a tree\n",
    "from the head of the list and adds its subtrees after the end of the\n",
    "list.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Breadth-First Tree Traversal --- Using Append\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val nbreadth : 'a tree list -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 177,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec nbreadth = function\n",
    "| [] -> []\n",
    "| Lf :: ts -> nbreadth ts\n",
    "| Br (v, t, u) :: ts ->\n",
    "    v :: nbreadth (ts @ [t; u])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Keeps an _enormous queue_ of nodes of search, and is a wasteful use of `append`.\n",
    "\n",
    "Breadth-first search can be inefficient, this naive implementation especially\n",
    "so.  When the search is at depth $d$ of the tree, the list contains all the\n",
    "remaining trees at depth $d$, followed by the subtrees (all at depth $d+1$) of\n",
    "the trees that have already been visited.  At depth 10, the list could already\n",
    "contain 1024 elements.  It requires a lot of space, and aggravates this with a\n",
    "gross misuse of append.  Evaluating `ts@[t, u]` copies the long list\n",
    "`ts` just to insert two elements.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## An Abstract Data Type: Queues\n",
    "\n",
    "\n",
    "\n",
    "- `qempty` is the _empty queue_\n",
    "- `qnull` _tests_ whether a queue is empty\n",
    "- `qhd` _returns_ the element at the _head_ of a queue\n",
    "- `deq` _discards_ the element at the _head_ of a queue\n",
    "- `enq` _adds_ an element at the _end_ of a queue\n",
    "\n",
    "Breadth-first search becomes much faster if we replace the lists by\n",
    "_queues_.  A queue represents a sequence, allowing elements to be taken\n",
    "from the head and added to the tail.  This is a First-In-First-Out (FIFO)\n",
    "discipline: the item next to be removed is the one that has been in the queue\n",
    "for the longest time.  Lists can implement queues, but append is a poor means\n",
    "of adding elements to the tail.\n",
    "\n",
    "Our functional arrays are suitable, provided we\n",
    "augment them with a function to delete the first array element.  (See _ML\n",
    "  for the Working Programmer_ page 156.)  Each operation would take $O(\\log\n",
    "n)$ time for a queue of length $n$.\n",
    "\n",
    "We shall describe a representation of queues that is purely functional, based\n",
    "upon lists, and efficient.  Operations take $O(1)$ time when “amortized”:\n",
    "averaged over the lifetime of a queue.\n",
    "\n",
    "A conventional programming technique is to represent a queue by an array.  Two\n",
    "indices point to the front and back of the queue, which may wrap around the\n",
    "end of the array.  The coding is somewhat tricky.  Worse, the length of the\n",
    "queue must be given a fixed upper bound.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Efficient Functional Queues: Idea\n",
    "\n",
    "\n",
    "\n",
    "- Represent the queue $x_1\\; x_2\\; \\ldots\\; x_m\\; y_n\\; \\ldots\\; y_1$ by any _pair of lists_\n",
    "$$ ([x_1,x_2,\\ldots,x_m], \\; [y_1,y_2,\\ldots,y_n])$$\n",
    "- Add new items to the _rear list_\n",
    "- Remove items from _front list_ and if empty move _rear_ to _front_\n",
    "- _Amortized_ time per operation is $O(1)$\n",
    "\n",
    "Queues require efficient access at both ends: at the front, for removal, and\n",
    "at the back, for insertion.  Ideally, access should take constant time,\n",
    "$O(1)$.  It may appear that lists cannot provide such access.  If\n",
    "`enq(q, x)` performs `q@[x]`, then this operation will be $O(n)$.  We\n",
    "could represent queues by reversed lists, implementing `enq(q, x)` by\n",
    "`x::q`, but then the `deq` and `qhd` operations would be\n",
    "$O(n)$.  Linear time is intolerable: a series of $n$ queue operations\n",
    "could then require $O(n^2)$ time.\n",
    "\n",
    "The solution is to represent a queue by a pair of lists, where\n",
    "$$ ([x_1,x_2,\\ldots,x_m], \\, [y_1,y_2,\\ldots,y_n]) $$\n",
    "represents the queue $x_1 x_2 \\ldots x_m y_n \\ldots y_1$.\n",
    "\n",
    "The front part of the queue is stored in order, and the rear part is stored in\n",
    "reverse order.  The `enq` operation adds elements to the rear part\n",
    "using cons, since this list is reversed; thus, `enq` takes constant\n",
    "time.  The `deq` and `qhd` operations look at the front part,\n",
    "which normally takes constant time, since this list is stored in order.  But\n",
    "sometimes `deq` removes the last element from the front part; when this\n",
    "happens, it reverses the rear part, which becomes the new front part.\n",
    "\n",
    "_Amortized_ time refers to the cost per operation averaged over the\n",
    "lifetime of any complete execution.  Even for the worst possible execution,\n",
    "the average cost per operation turns out to be constant; see the analysis\n",
    "below.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Efficient Functional Queues: Code\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "type 'a queue = Q of 'a list * 'a list\n"
      ]
     },
     "execution_count": 178,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type 'a queue =\n",
    "| Q of 'a list * 'a list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val norm : 'a queue -> 'a queue = <fun>\n"
      ]
     },
     "execution_count": 179,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let norm = function\n",
    "| Q ([], tls) -> Q (List.rev tls, [])\n",
    "| q -> q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val qnull : 'a queue -> bool = <fun>\n"
      ]
     },
     "execution_count": 180,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let qnull = function\n",
    "| Q ([], []) -> true\n",
    "| _ -> false"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 181,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val enq : 'a queue -> 'a -> 'a queue = <fun>\n"
      ]
     },
     "execution_count": 181,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let enq (Q (hds, tls)) x = norm (Q (hds, x::tls))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "exception Empty\n"
      ]
     },
     "execution_count": 182,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exception Empty"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 183,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val deq : 'a queue -> 'a queue = <fun>\n"
      ]
     },
     "execution_count": 183,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let deq = function\n",
    "| Q (x::hds, tls) -> norm (Q (hds, tls))\n",
    "| _ -> raise Empty"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val qempty : 'a queue = Q ([], [])\n"
      ]
     },
     "execution_count": 184,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let qempty = Q ([], [])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val qhd : 'a queue -> 'a = <fun>\n"
      ]
     },
     "execution_count": 185,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let qhd = function\n",
    "| Q (x::_, _) -> x\n",
    "| _ -> raise Empty"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The datatype of queues prevents confusion with other pairs of lists.  The empty\n",
    "queue has both parts empty.\n",
    "\n",
    "The function `norm` puts a queue into normal form, ensuring that the front part\n",
    "is never empty unless the entire queue is empty.  Functions `deq` and `enq`\n",
    "call `norm` to normalise their result.\n",
    "\n",
    "Because queues are in normal form, their head is certain to be in their\n",
    "front part, so `qhd` looks there.\n",
    "\n",
    "Let us analyse the cost of an execution comprising (in any possible order) $n$\n",
    "`enq` operations and $n$ `deq` operations, starting with an\n",
    "empty queue.  Each `enq` operation will perform one cons, adding an\n",
    "element to the rear part.  Since the final queue must be empty, each element\n",
    "of the rear part gets transferred to the front part.  The corresponding\n",
    "reversals perform one cons per element.  Thus, the total cost of the series of\n",
    "queue operations is $2n$ cons operations, an average of 2 per operation.  The\n",
    "amortized time is $O(1)$.\n",
    "\n",
    "There is a catch.  The conses need not be distributed evenly; reversing a long\n",
    "list could take up to $n-1$ of them.  Unpredictable delays make the approach\n",
    "unsuitable for _real-time programming_ where deadlines must be met.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Breadth-First Tree Traversal --- Using Queues\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val breadth : 'a tree queue -> 'a list = <fun>\n"
      ]
     },
     "execution_count": 186,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let rec breadth q =\n",
    "  if qnull q then []\n",
    "  else\n",
    "    match qhd q with\n",
    "    | Lf -> breadth (deq q)\n",
    "    | Br (v, t, u) -> v :: breadth (enq (enq (deq q) t) u)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "This function implements the same algorithm as `nbreadth` but uses a different\n",
    "data structure.  It represents queues using type `queue` instead of type\n",
    "`list`.\n",
    "\n",
    "To compare their efficiency, I applied both functions to the full binary tree\n",
    "of depth 12, which contains 4095 labels.  The function `nbreadth` took 30\n",
    "seconds while `breadth` took only 0.15 seconds: faster by a factor of 200.\n",
    "\n",
    "For larger trees, the speedup would be greater.  Choosing the right data\n",
    "structure pays handsomely.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Iterative deepening: Another Exhaustive Search\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "- Breadth-first search examines $O(b^d)$ nodes:\n",
    "$$ 1 + b + \\cdots + b^d = {b^{d+1}-1 \\over b-1}\n",
    "  \\qquad \\begin{array}[c]{r@{}l}\n",
    "            b & {} = \\hbox{branching factor}\\\\\n",
    "            d & {} = \\hbox{depth}\n",
    "          \\end{array}\n",
    "$$\n",
    "- Recompute nodes at depth $d$ instead of storing them\n",
    "- Time factor is $b/(b-1)$ if $b>1$; complexity is still $O(b^d)$\n",
    "- Space required at depth $d$ drops from $b^d$ to $d$\n",
    "\n",
    "Breadth-first search is not practical for infinite trees: it uses too much\n",
    "space. Large parts of the tree have to be stored.\n",
    "Consider the slightly more general problem of searching trees whose\n",
    "branching factor is $b$ (for binary trees, $b=2$).  Then breadth-first search\n",
    "to depth $d$ examines $(b^{d+1}-1)/(b-1)$ nodes, which is $O(b^d)$, ignoring\n",
    "the constant factor of $b/(b-1)$.  Since all nodes that are examined are also\n",
    "stored, the space and time requirements are both $O(b^d)$.\n",
    "\n",
    "_Depth-first iterative deepening_ combines the space efficiency of\n",
    "depth-first with the “nearest-first” property of breadth-first search.  It\n",
    "performs repeated depth-first searches with increasing depth bounds, each time\n",
    "discarding the result of the previous search.  Thus it searches to depth 1,\n",
    "then to depth 2, and so on until it finds a solution.  We can afford to\n",
    "discard previous results because the number of nodes is growing exponentially.\n",
    "There are $b^{d+1}$ nodes at level $d+1$; if $b\\geq2$, this number actually\n",
    "exceeds the total number of nodes of all previous levels put together, namely\n",
    "$(b^{d+1}-1) / (b-1)$.\n",
    "\n",
    "[Korf shows](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.91.288) that the time needed for iterative deepening to reach\n",
    "depth $d$ is only $b/(b-1)$ times that for breadth-first search, if $b>1$.\n",
    "This is a constant factor; both algorithms have the same time complexity,\n",
    "$O(b^d)$.  In typical applications where $b\\geq2$ the extra factor of\n",
    "$b/(b-1)$ is quite tolerable.  The reduction in the space requirement is\n",
    "exponential, from $O(b^d)$ for breadth-first to $O(d)$ for iterative\n",
    "deepening. Of course, this assumes that the tree itself is not stored in memory.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Another Abstract Data Type: Stacks\n",
    "\n",
    "\n",
    "\n",
    "- `empty` is the _empty stack_\n",
    "- `null` tests whether a stack is empty\n",
    "- `top` returns the element at the _top_ of a stack\n",
    "- `pop` discards the element at the _top_ of a stack\n",
    "- `push` adds an element at the _top_ of a stack\n",
    "\n",
    "A _stack_ is a sequence such that items can be added or removed from the head\n",
    "only.  A stack obeys a Last-In-First-Out (LIFO) discipline: the item next to be\n",
    "removed is the one that has been in the queue for the _shortest_ time.  Lists\n",
    "can easily implement stacks because both `cons` and `hd` affect the head.  But\n",
    "unlike lists, stacks are often regarded as an imperative data structure: the\n",
    "effect of `push` or `pop` is to change an existing stack, not return a new one.\n",
    "\n",
    "In conventional programming languages, a stack is often implemented by storing\n",
    "the elements in an array, using a variable (the “stack pointer”) to count them.\n",
    "Most language processors keep track of recursive function calls using an\n",
    "internal stack.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A Survey of Search Methods\n",
    "\n",
    "\n",
    "\n",
    "- Depth-first: use a _stack_  (efficient but incomplete)\n",
    "- Breadth-first: use a _queue_ (uses too much space!)\n",
    "- Iterative deepening: use depth-first to get benefits of breadth-first (trades time for space)\n",
    "- Best-first: use a _priority queue_ (heuristic search)\n",
    "\n",
    "The data structure determines the search!\n",
    "\n",
    "Search procedures can be classified by the data structure used to store\n",
    "pending subtrees.  Depth-first search stores them on a stack, which is\n",
    "implicit in functions like `inorder`, but can be made explicit.\n",
    "Breadth-first search stores such nodes in a queue.\n",
    "\n",
    "An important variation is to store the nodes in a priority queue, which\n",
    "is an ordered sequence.  The priority queue applies some sort of ranking\n",
    "function to the nodes, placing higher-ranked nodes before lower-ranked ones.\n",
    "The ranking function typically estimates the distance from the node to a\n",
    "solution.  If the estimate is good, the solution is located swiftly.  This\n",
    "method is called best-first search.\n",
    "\n",
    "The priority queue can be kept as a sorted list, although this is slow.\n",
    "Binary search trees would be much better on average, and fancier data\n",
    "structures improve matters further.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 10.1\n",
    "\n",
    "\n",
    "Suppose that we have an implementation of queues, based on binary trees, such that each operation\n",
    "takes logarithmic time in the worst case. Outline the advantages and drawbacks of such an\n",
    "implementation compared with one presented above.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 10.2\n",
    "\n",
    "\n",
    "The traditional way to implement queues uses a fixed-length array. Two indices into the array\n",
    "indicate the start and end of the queue, which wraps around from the end of the array to the start.\n",
    "How appropriate is such a data structure for implementing breadth-first search?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 10.3\n",
    "\n",
    "\n",
    "Write a version of the function `breadth` using a nested `let` construction rather than\n",
    "`match`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 10.4\n",
    "\n",
    "\n",
    "Iterative deepening is inappropriate if $b\\approx1$, where $b$ is the branching factor. What search\n",
    "strategy is appropriate in this case?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 10.5\n",
    "\n",
    "\n",
    "Consider the following OCaml function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 187,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val next : int -> int list = <fun>\n"
      ]
     },
     "execution_count": 187,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let next n = [2 * n; 2 * n + 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we regard it as representing a tree, where the subtrees are computed from the current label, what\n",
    "tree does `next 1` represent?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lecture 11: Elements of Procedural Programming\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Procedural Programming\n",
    "\n",
    "\n",
    "\n",
    "- Procedural programs can change the machine state.\n",
    "- They can interact with its environment\n",
    "- They use control structures like branching, iteration and procedures.\n",
    "\n",
    "They use data abstractions of the computer’s memory:\n",
    "- _references_ to memory cells\n",
    "- _arrays_ that are blocks of memory cells\n",
    "- _linked structures_ such as _linked lists_\n",
    "\n",
    "Procedural programming is programming in the traditional sense of the word.  A\n",
    "program _state_ is repeatedly transformed by the execution of\n",
    "_commands_ or _statements_.  A state change might be local to the\n",
    "machine and consist of updating a variable or array.  A state change might\n",
    "consist of sending data to the outside world.  Even reading data counts as a\n",
    "state change, since this act normally removes the data from the environment.\n",
    "\n",
    "Procedural programming languages provide primitive commands and control\n",
    "structures for combining them.  The primitive commands include\n",
    "_assignment_ for updating variables, and various _input/output_\n",
    "commands for communication.  Control structures include `if` and\n",
    "`match` constructs for conditional execution, and repetitive constructs\n",
    "such as `while`.  Programmers can package up their own commands as\n",
    "_procedures_ taking arguments.  The need for such “subroutines” was\n",
    "evident from the earliest days; they represent one of the first\n",
    "examples of abstraction in programming languages.\n",
    "\n",
    "OCaml makes no distinction between commands and expressions. OCaml provides built-in\n",
    "‘functions’ to perform assignment and communication, and these can be used\n",
    "in the traditional (procedural) style. OCaml programmers often follow a\n",
    "functional style for most internal computations and use imperative features\n",
    "mainly for communication with the outside world.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## OCaml Primitives for References\n",
    "\n",
    "\n",
    "\n",
    "| Syntax   | Effect  |\n",
    "| -- | -- |\n",
    "| `ref E` | _create_ a reference with _initial contents_ = value of `E` |\n",
    "| `!P`    | return _current contents_ of reference `P` |\n",
    "| `P := E` | _update_ contents of `P` to value of `E` |\n",
    "\n",
    "\n",
    "The above text presents the OCaml primitives, but most languages have analogues of\n",
    "them, often heavily disguised.  We need a means of creating references (or\n",
    "allocating storage), getting at the current contents of a reference cell, and\n",
    "updating that cell.\n",
    "\n",
    "The function `ref` creates references (also called\n",
    "“locations”).  Calling `ref` allocates a new location in memory.\n",
    "Initially, this location holds the value given by\n",
    "expression $E$.\n",
    "\n",
    "The function `!`, when applied to a reference, returns its contents.\n",
    "This operation is called _dereferencing_.  Clearly `!` is not a\n",
    "mathematical function; its result depends upon the store.\n",
    "\n",
    "The assignment `P:=E` evaluates expression $P$, which must return a\n",
    "reference $p$, and $E$.  It stores at address $p$ the value of $E$.\n",
    "Syntactically, `:=` is a function and `P:=E` is an\n",
    "expression, even though it updates the store.  Like many functions that change\n",
    "the state, it returns the value `()` of type `unit`.\n",
    "\n",
    "If $\\tau$ is some OCaml type, then $\\tau$ `ref` is the type of references\n",
    "to cells that can hold values of $\\tau$.  Please do not confuse the type\n",
    "`ref` with the function `ref`.  This table of the primitive\n",
    "functions and their types might be useful:\n",
    "\n",
    "| Syntax | OCaml Type |\n",
    "| --- | -------------- |\n",
    "| `ref` | `'a -> 'a ref` |\n",
    "| `!` | `'a ref -> 'a `|\n",
    "| `:=` | `'a ref -> 'a -> unit` |\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Trying Out References\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val p : int ref = {contents = 5}\n"
      ]
     },
     "execution_count": 188,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let p = ref 5 (* create a reference *)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 189,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : unit = ()\n"
      ]
     },
     "execution_count": 189,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p := !p + 1   (* p now holds value 6 *)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val ps : int ref list = [{contents = 77}; {contents = 6}]\n"
      ]
     },
     "execution_count": 190,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let ps = [ ref 77; p ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : unit = ()\n"
      ]
     },
     "execution_count": 191,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "List.hd ps := 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int ref list = [{contents = 3}; {contents = 6}]\n"
      ]
     },
     "execution_count": 192,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The first line declares `p` to hold a reference to an integer,\n",
    "initially 5.  Its type is `int ref`, not just `int`, so it\n",
    "admits assignment.  Assignment never changes `let` bindings: they are\n",
    "_immutable_.  The identifier `p` will always denote the reference\n",
    "mentioned in its declaration unless superseded by a new usage of `p`.\n",
    "Only the _contents_ of the reference is mutable.\n",
    "\n",
    "OCaml displays a reference value as `{contents=v}`, where value $v$ is the\n",
    "contents.  This notation is readable but gives us no way of telling whether\n",
    "two references holding the same value are actually the same reference.  To\n",
    "display a reference as a machine address has obvious drawbacks!\n",
    "\n",
    "In the first assignment, the expression `!p` yields the reference’s\n",
    "current contents, namely 5.  The assignment changes the contents of `p`\n",
    "to 6.  Most languages do not have an explicit dereferencing operator\n",
    "(like `!`) because of its inconvenience.  Instead, by convention,\n",
    "occurrences of the reference on the _left-hand_ side of the `:=`\n",
    "denote locations and those on the _right-hand_ side denote the contents.\n",
    "A special ‘address of’ operator may be available to override the convention\n",
    "and make a reference on the right-hand side to denote a location.  Logically\n",
    "this is a mess, but it makes programs shorter.\n",
    "\n",
    "The list `ps` is declared to hold a new reference (initially\n",
    "containing 77) as well as `p`.  Then the new reference is\n",
    "updated to hold 3.  The assignment to `hd ps` does _not_\n",
    "update `ps`, only the contents of a reference in that list.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Commands: Expressions with Effects\n",
    "\n",
    "\n",
    "\n",
    "- Basic commands update references, write to files, etc.\n",
    "- $C_1 ; \\ldots ; C_n$ causes a series of expressions to be evaluated and returns the value of $C_n$.\n",
    "- A typical command returns the empty tuple: `()`\n",
    "- `if` $B$ `then` $C_1$ `else` $C_2$ behaves like the traditional control structure if $C_1$ and $C_2$ have effects.\n",
    "- Other OCaml constructs behave naturally with commands, including `match` expressions and recursive functions.\n",
    "\n",
    "We use the term _command_ informally to refer to an expression that has an\n",
    "effect on the state. All expressions denote some value, but they can return\n",
    "`()`, which conveys no actual information.\n",
    "\n",
    "We need a way to execute one command after another.\n",
    "The construct $C_1 ; \\ldots ; C_n$ evaluates the expressions $C_1$\n",
    "to $C_n$ in the order given and returns the value of $C_n$.  The values\n",
    "of the other expressions are discarded; their only purpose is to change the\n",
    "state.\n",
    "\n",
    "Commands may be used with `if` and `match` much as in conventional languages.\n",
    "OCaml functions play the role of procedures.\n",
    "\n",
    "Other languages that combine the functional and imperative programming\n",
    "paradigms include Lisp (and its dialect Scheme), Scala, and even a\n",
    "systems programming language, BLISS (now long extinct).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Iteration: the `while` command\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val tlopt : 'a list -> 'a list option = <fun>\n"
      ]
     },
     "execution_count": 193,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let tlopt = function\n",
    "| [] -> None\n",
    "| _::xs -> Some xs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val length : 'a list -> int = <fun>\n"
      ]
     },
     "execution_count": 194,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let length xs =\n",
    "  let lp  = ref xs in (* list of uncounted elements *)\n",
    "  let np  = ref 0  in (* accumulated count *)\n",
    "  let fin = ref false in\n",
    "  while not !fin do\n",
    "    match tlopt !lp with\n",
    "    | None -> fin := true\n",
    "    | Some xs ->\n",
    "        lp := xs;\n",
    "        np := 1 + !np\n",
    "  done;\n",
    "  !np (* the final count is returned *)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Once we can change the state, we need to do so repeatedly.  Recursion can\n",
    "serve this purpose, but having to declare a procedure for every loop is\n",
    "clumsy, and compilers for conventional languages seldom exploit\n",
    "tail-recursion.\n",
    "\n",
    "Early programming languages provided little support for repetition.  The\n",
    "programmer had to set up loops using goto commands, exiting the loop using\n",
    "another goto controlled by an `if`.  Modern languages provide a\n",
    "confusing jumble of looping constructs, the most fundamental of which is\n",
    "`while B do C`.  The boolean expression $B$ is evaluated,\n",
    "and if true, command $C$ is executed and the command repeats.  If $B$\n",
    "evaluates to false then the `while` command terminates, perhaps without\n",
    "executing $C$ even once.\n",
    "\n",
    "OCaml’s main looping construct is `while`, which returns the value `()`.  The\n",
    "function `length` declares references to hold the list under\n",
    "examination (`lp`) and number of elements counted so far (`np`) as well\n",
    "as whether the end of the list has been reached (the boolean reference `fin`).\n",
    "While the list is non-empty, we skip over one more element (by setting it to\n",
    "its tail) and count that element.\n",
    "\n",
    "The body of the `while` loop first checks to see if the end of the list has\n",
    "been reached, in which case it sets the `fin` variable to true.  If there is a\n",
    "tail value, then two assignments are executed in sequence.  The `lp` reference\n",
    "is set to the tail of the list, and the `np` reference integer is incremented\n",
    "by one.  When the while loop terminates due to the `fin` variable being set to\n",
    "true, the expression `!np` returns the computed length as the function’s\n",
    "result.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Private, Persistent References\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 195,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "exception TooMuch of int\n"
      ]
     },
     "execution_count": 195,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exception TooMuch of int"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val makeAccount : int -> int -> int = <fun>\n"
      ]
     },
     "execution_count": 196,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let makeAccount initBalance =\n",
    "  let balance = ref initBalance in\n",
    "  let withdraw amt =\n",
    "     if amt > !balance then\n",
    "       raise (TooMuch (amt - !balance))\n",
    "     else begin\n",
    "       balance := !balance - amt;\n",
    "       !balance\n",
    "     end\n",
    "  in\n",
    "  withdraw"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "As you may have noticed, OCaml’s programming style looks clumsy compared with\n",
    "that of languages like C.  OCaml omits the defaults and abbreviations they\n",
    "provide to shorten programs.  However, OCaml’s explicitness makes it ideal for\n",
    "teaching the fine points of references and arrays.  OCaml’s references are more\n",
    "flexible than those found in other languages.\n",
    "\n",
    "The function `makeAccount` models a bank.  Calling the function with a\n",
    "specified initial balance creates a new reference `balance`) to\n",
    "maintain the account balance and returns a function (`withdraw`) having\n",
    "sole access to that reference.  Calling `withdraw` reduces the balance\n",
    "by the specified amount and returns the new balance.  You can pay money in by\n",
    "withdrawing a negative amount.  The `if`-construct prevents the account\n",
    "from going overdrawn, raising an exception.\n",
    "\n",
    "Look at the $\\tt (E_1; E_2)$ construct in the _else_ part above.\n",
    "The first expression updates the account balance and returns the trivial\n",
    "value ().  The second expression, `!balance`, returns the current\n",
    "balance but does not return the reference itself: that would allow\n",
    "unauthorised updates.\n",
    "\n",
    "This example is based on one by Dr A C Norman.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Two Bank Accounts\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val student : int -> int = <fun>\n"
      ]
     },
     "execution_count": 197,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let student = makeAccount 500"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val director : int -> int = <fun>\n"
      ]
     },
     "execution_count": 198,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let director = makeAccount 4000000;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int = 495\n"
      ]
     },
     "execution_count": 199,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "student 5        (* coach fare *)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int = 3850000\n"
      ]
     },
     "execution_count": 200,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "director 150000  (* Tesla *)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 201,
   "metadata": {},
   "outputs": [
    {
     "ename": "error",
     "evalue": "runtime_error",
     "output_type": "error",
     "traceback": [
      "\u001b[31mException: TooMuch 5.\nRaised at file \"[196]\", line 5, characters 13-39\nCalled from file \"toplevel/toploop.ml\", line 208, characters 17-27\n\u001b[0m"
     ]
    }
   ],
   "source": [
    "student 500      (* oh oh *)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Each call to `makeAccount` returns a copy of `withdraw` holding\n",
    "a _fresh_ instance of the reference `balance`.  As with a real bank\n",
    "pass-book, there is no access to the account balance except via the\n",
    "corresponding `withdraw` function.  If that function is discarded, the\n",
    "reference cell becomes unreachable; the computer will eventually reclaim it,\n",
    "just as banks close down dormant accounts.\n",
    "\n",
    "Here we see two people managing their accounts.  For better or worse, neither\n",
    "can take money from the other.\n",
    "\n",
    "We could generalise `makeAccount` to return several functions that\n",
    "jointly manage information held in shared references.  The functions might be\n",
    "packaged using OCaml records, which are not discussed in this course.\n",
    "Most procedural languages do not properly support the concept of private\n",
    "references, although _object-oriented_ languages take them as a basic theme.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## OCaml Primitives for Arrays\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 202,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : string array = [|\"a\"; \"b\"; \"c\"|]\n"
      ]
     },
     "execution_count": 202,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[|\"a\"; \"b\"; \"c\"|] (* allocate a fresh string array *)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 203,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : char array = [|'a'; 'a'; 'a'|]\n"
      ]
     },
     "execution_count": 203,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Array.make 3 'a'   (* array of size 10 with cell containing 'a' *)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 204,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val aa : int array = [|0; 10; 20; 30; 40|]\n"
      ]
     },
     "execution_count": 204,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let aa = Array.init 5 (fun i -> i * 10) (* array of size 5 initialised to (fun i) *)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 205,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int = 30\n"
      ]
     },
     "execution_count": 205,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Array.get aa 3  (* retrieve the 4th cell in the array *)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 206,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : unit = ()\n"
      ]
     },
     "execution_count": 206,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Array.set aa 3 42 (* set the 4th cell's value to 42 *)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "There are many other array operations in the `Array` module in the OCaml standard\n",
    "library.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int -> 'a -> 'a array = <fun>\n"
      ]
     },
     "execution_count": 207,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Array.make"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 208,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int -> (int -> 'a) -> 'a array = <fun>\n"
      ]
     },
     "execution_count": 208,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Array.init"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 209,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : 'a array -> int -> 'a = <fun>\n"
      ]
     },
     "execution_count": 209,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Array.get"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 210,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : 'a array -> int -> 'a -> unit = <fun>\n"
      ]
     },
     "execution_count": 210,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Array.set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "OCaml arrays are like references that hold several elements instead of one.  The\n",
    "elements of an $n$-element array are designated by the integers from 0\n",
    "to $n-1$.  The $i$th array element is usually written $A.[i]$.\n",
    "If $\\tau$ is a type then $\\tau$ `array` is the\n",
    "type of arrays (of any size) with elements from $\\tau$.\n",
    "\n",
    "Calling `Array.init i n` creates an array of the size specified in $i$\n",
    "by expression $n$.  Initially, element $A.[i]$ holds the value of $f(i)$ for $i=0$, \\ldots,~$n-1$.\n",
    "Like `ref`, it allocates mutable storage to hold the specified values.\n",
    "\n",
    "Calling `Array.get A i` returns the contents of $A.[i]$.\n",
    "\n",
    "Calling `Array.set A i E` modifies the array $A$ by storing the\n",
    "value of $E$ as the new contents of $A[i]$; it returns `()` as its value.\n",
    "\n",
    "OCaml’s arrays are much safer than C’s. In C, an array is nothing more than an\n",
    "address indicating the start of a storage area. Nothing indicates the size of\n",
    "the area.  Therefore C programs are vulnerable to _buffer overrun attacks:_ an\n",
    "attacker sends more data than the receiving program expects, overrunning the\n",
    "area of storage set aside to hold it. The attack eventually overwrites the\n",
    "program itself, replacing it with code controlled by the attacker.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Array Examples\n",
    "\n",
    "\n",
    "\n",
    "In the following session, the identifier `ar` is bound to an array of 20 elements, which\n",
    "are initially set to the squares of their subscripts.  The array’s third\n",
    "element (which actually has subscript 2) is inspected and found to be four. The\n",
    "second call to `Array.get` supplies a subscript that is out of range, so OCaml\n",
    "rejects it.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 211,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "val ar : int array =\n",
       "  [|0; 1; 4; 9; 16; 25; 36; 49; 64; 81; 100; 121; 144; 169; 196; 225; 256;\n",
       "    289; 324; 361|]\n"
      ]
     },
     "execution_count": 211,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "let ar = Array.init 20 (fun i -> i * i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 212,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int = 4\n"
      ]
     },
     "execution_count": 212,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Array.get ar 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {},
   "outputs": [
    {
     "ename": "error",
     "evalue": "runtime_error",
     "output_type": "error",
     "traceback": [
      "\u001b[31mException: Invalid_argument \"index out of bounds\".\nRaised by primitive operation at unknown location\nCalled from file \"toplevel/toploop.ml\", line 208, characters 17-27\n\u001b[0m"
     ]
    }
   ],
   "source": [
    "Array.get ar 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 214,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : int array =\n",
       "[|0; 1; 33; 9; 16; 25; 36; 49; 64; 81; 100; 121; 144; 169; 196; 225; 256;\n",
       "  289; 324; 361|]\n"
      ]
     },
     "execution_count": 214,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Array.set ar 2 33; ar"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "By calling `Array.set`, we then modify the element with subscript 2. Note\n",
    "however that we cannot modify the array’s length. If we outgrow the array, we\n",
    "have to create a new one, copy the data into it, and then forget the old array.\n",
    "Typically the new array would be double the size of the old one, so that the\n",
    "cost of copying is insignificant.\n",
    "\n",
    "OCaml provides numerous operators for modifying, computing over and searching in\n",
    "arrays. Many are analogous to functions on lists. For example,\n",
    "`Array.exists` takes a boolean-valued function and returns `true` if an\n",
    "array element satisfies it.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : bool = true\n"
      ]
     },
     "execution_count": 215,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Array.exists (fun i -> i > 200) ar"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 216,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "- : bool = false\n"
      ]
     },
     "execution_count": 216,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Array.exists (fun i -> i < 0) ar"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References: OCaml _vs_ conventional languages\n",
    "\n",
    "\n",
    "\n",
    "- We must write `!p` to get the _contents_ of `p`\n",
    "- We write just `p` for the _address_ of `p`\n",
    "- We can store private reference cells (like `balance`) in functions---analogous to elements of _object-oriented programming_\n",
    "- OCaml’s assignment syntax is $\\tt V \\, := \\, E$ instead of $V$ = $E$\n",
    "- OCaml has few control structures: `while`, `match`, `if` and `for` (the latter is not covered in this course)\n",
    "- OCaml has syntax for updating an array via the `a.[i] <- v` syntax which is the same as `Array.set a i v`.\n",
    "\n",
    "Conventional syntax for variables and assignments has hardly changed since\n",
    "Fortran, the first high-level language. In conventional languages,\n",
    "virtually all variables can be updated.  We declare something like\n",
    "`p: int`, mentioning no reference type even if the language provides\n",
    "them.  If we do not specify an initial value, we may get whatever bits were\n",
    "previously at that address.  Illegal values arising from uninitialised\n",
    "variables can cause errors that are almost impossible to diagnose.\n",
    "\n",
    "Dereferencing operators (like OCaml’s `!`) are especially unpopular, because\n",
    "they clutter the program text. Virtually all programming languages make dereferencing\n",
    "implicit (that is, automatic).\n",
    "\n",
    "It is generally accepted these days that a two-dimensional array $A$ is nothing\n",
    "but an array of arrays.  An assignment to such an array is typically written\n",
    "something like $A[i,j] {:=} x$; in C, the syntax is `A[i][j] = x`. Higher\n",
    "dimensions are treated analogously.  The corresponding OCaml code can either\n",
    "declare an array of arrays, or use the `A.(i)` syntax to calculate the linear\n",
    "offset into a single array.\n",
    "\n",
    "You can use the constructs we have learnt to easily create linked (mutable) lists as\n",
    "an alternative to arrays.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 217,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "type 'a mlist = Nil | Cons of 'a * 'a mlist ref\n"
      ]
     },
     "execution_count": 217,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type 'a mlist =\n",
    "| Nil\n",
    "| Cons of 'a * 'a mlist ref"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "It is worth mentioning that OCaml’s references fully suffice for coding the sort of linked data structures\n",
    "taught in algorithms courses, and is illustrated in the figure above. The\n",
    "programming style is a little different from the usual, but the principles are\n",
    "the same.  OCaml also provides comprehensive input/output primitives for various\n",
    "types of file and operating system.\n",
    "\n",
    "OCaml’s system of modules include _structures,_ which can be seen as encapsulated\n",
    "groups of declarations, and _signatures,_ which are specifications of\n",
    "structures listing the name and type of each component.  Finally, there are\n",
    "_functors,_ which are analogous to functions that combine a number of argument\n",
    "structures, and which can be used to plug program components together. These\n",
    "primitives are useful for managing large programming projects.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 11.1\n",
    "\n",
    "\n",
    "Comment, with examples, on the differences between an `int ref list` and an `int list ref`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 11.2\n",
    "\n",
    "\n",
    "Write a version of function `power` (Lecture 1) using `while` instead of recursion.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 11.3\n",
    "\n",
    "\n",
    "What is the effect of $\\tt while \\, C_1; \\; B \\, do \\, C_2 \\, done$ ?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 11.4\n",
    "\n",
    "\n",
    "Write a function to exchange the values of two references, `xr` and `yr`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 11.5\n",
    "\n",
    "\n",
    "Arrays of multiple dimensions are represented in OCaml by arrays of arrays. Write functions to\n",
    "(a) create an $n\\times n$ identity matrix, given $n$, and\n",
    "(b) to transpose an $m\\times n$ matrix. Identity matrices have the following form:\n",
    "$$\n",
    "\\left( {\n",
    "\\begin{array}{cccc}\n",
    "   1 & 0 &  \\cdots  & 0  \\\\\n",
    "   0 & 1 &  \\cdots  & 0  \\\\\n",
    "    \\vdots  &  \\vdots  &  \\ddots  &  \\vdots   \\\\\n",
    "   0 & 0 &  \\cdots  & 1  \\\\\n",
    " \\end{array}\n",
    " } \\right)\n",
    "$$"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "OCaml 4.08",
   "language": "OCaml",
   "name": "ocaml-jupyter"
  },
  "language_info": {
   "codemirror_mode": "text/x-ocaml",
   "file_extension": ".ml",
   "mimetype": "text/x-ocaml",
   "name": "OCaml",
   "nbconverter_exporter": null,
   "pygments_lexer": "OCaml",
   "version": "4.08.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}