![\"Dask](\"https://raw.githubusercontent.com/dask/dask/main/docs/source/images/dask-overview.svg\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3a8359e2",
+ "metadata": {},
+ "source": [
+ "### Dask jurgon: Client, Scheduler and Workers \n",
+ "\n",
+ "- Client: The user-facing entry point for cluster users. In other words, the client lives where your python code lives, and it communicates to the scheduler, passing along the tasks to be executed.\n",
+ "- Scheduler: The task manager, it sends the tasks to the workers.\n",
+ "- Workers: The ones that compute the tasks.\n",
+ "\n",
+ "Note: The Scheduler and the Workers are on the same network, they could live in your laptop or on a separate cluster\n",
+ "\n",
+ "![\"Dask](\"https://raw.githubusercontent.com/coiled/pydata-global-dask/master/images/dask-cluster.svg\"\n",)
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d78d7794",
+ "metadata": {
+ "tags": []
+ },
+ "source": [
+ "## When to use Dask?\n",
+ "\n",
+ "Before trying to use Dask, there are some questions to determine if Dask might be suitable for you. \n",
+ "\n",
+ "- Does your data fit in memory? \n",
+ " - Yes: Use pandas or numpy. \n",
+ " - No : Dask might be able to help. \n",
+ "- Do your computations take for ever?\n",
+ " - Yes: Dask might be able to help. \n",
+ " - No : Awesome.\n",
+ "- Do you have embarrassingly parallelizable code?\n",
+ " - Yes: Dask might be able to help.\n",
+ " - No?: If you are not sure here are some [examples](https://examples.dask.org/applications/embarrassingly-parallel.html) \n",
+ " - No: I'm sorry, although Dask might have some hope for you.\n",
+ " \n",
+ " \n",
+ "**Bottom Left:** You don't need Dask. \n",
+ "**Elsewhere:** Dask fair game.\n",
+ "\n",
+ "\n",
+ "![\"Dask](\"https://raw.githubusercontent.com/dask/dask-ml/main/docs/source/images/dimensions_of_scale.svg\"\n",)
\n",
+ "\n",
+ "\n",
+ "**Disclaimers:**\n",
+ "\n",
+ "1. When we say \"Dask might be able to help\" it is because you should try first to accelerate your code with Numpy and or Numba, checking types used on your Dataframes, and then maybe consider Dask. Now even when using Dask, we can't guarantee that things will be faster, it depends on what is the code behind. \n",
+ "\n",
+ "2. Even when you have large datasets, at some point you want to double check if you have reduced things to a manageable level where going back to pandas or Numpy might be the best call.\n",
+ "\n",
+ "**Best practices:**\n",
+ "\n",
+ "The learning curve to use Dask can be a bit intimidating, that's why we want to point you out to some best practices links that will make the process smoother. We will go over some of these topics but we want to leave here these links for future reference\n",
+ "\n",
+ "- Are you working with arrays? Check this [array best practices](https://docs.dask.org/en/latest/array-best-practices.html)\n",
+ "- Dealing with DataFrames? Check this [DataFrames best practices](https://docs.dask.org/en/latest/dataframe-best-practices.html)\n",
+ "- Are you trying to accelerate your code using `delayed`? Check this [delayed best practices](https://docs.dask.org/en/latest/delayed-best-practices.html)\n",
+ "- For overall good practices check [Dask good practices](https://docs.dask.org/en/latest/best-practices.html)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "ea50907f",
+ "metadata": {},
+ "source": [
+ "## Why Dask? \n",
+ "\n",
+ "If you are interested in knowing why Dask might be a good option for you we recommend you to check the Dask documentation [Why Dask?](https://docs.dask.org/en/latest/why.html)\n",
+ "\n",
+ "But if you are already convinced that Dask is right for you and/or want to learn more about it. The topics that we will cover on this mini-tutorial are:\n",
+ "\n",
+ "1. Dask Delayed: How to parallelize existing Python code and your custom algorithms. \n",
+ "2. Schedulers: Single Machine vs Distributed, and the Dashboard. \n",
+ "3. From pandas to Dask: How to manipulate bigger-than-memory DataFrames using Dask. \n",
+ "4. Dask-ML: Scalable machine learning using Dask. "
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.9.6"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/notebooks/1_Delayed.ipynb b/notebooks/1_Delayed.ipynb
new file mode 100644
index 0000000..6bdcceb
--- /dev/null
+++ b/notebooks/1_Delayed.ipynb
@@ -0,0 +1,742 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "be4dd2fc",
+ "metadata": {},
+ "source": [
+ "![\"Dask](\"https://raw.githubusercontent.com/dask/dask/main/docs/source/images/dask_horizontal_no_pad.svg\")
\n",
+ "\n",
+ "This notebook was inspired in the materials from: \n",
+ "\n",
+ "- https://github.com/coiled/pydata-global-dask/\n",
+ "- https://github.com/dask/dask-tutorial/\n",
+ "\n",
+ "# Dask Delayed\n",
+ "\n",
+ "Sometimes we have problems that are parallelizable. Dask delayed is an interface that can be use to parallelize existing Python code and custom algorithms. \n",
+ "\n",
+ "A first step to determine if we can use `dask.delayed` is to identify if there is some level of parallelism that we haven't exploit and hopefully `dask.delayed` will take care of it. We will start showing a simple example inspired on the main [Dask tutorial](https://tutorial.dask.org/), and we will it parallelize using `dask.delayed`.\n",
+ "\n",
+ "The following two functions will perform simple computations, where we use the `sleep` to simulate work. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "id": "c7cbb4e5",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from time import sleep\n",
+ "\n",
+ "def inc(x):\n",
+ " \"\"\"Increments x by one\"\"\"\n",
+ " sleep(1)\n",
+ " return x + 1\n",
+ "\n",
+ "def add(x, y):\n",
+ " \"\"\"Adds x and y\"\"\"\n",
+ " sleep(1)\n",
+ " return x + y"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "700f0a97",
+ "metadata": {},
+ "source": [
+ "Let's do some operations and time these functions using the `%%time` magic at the beginning of the cell. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "id": "1c0053fb",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "CPU times: user 933 µs, sys: 1.62 ms, total: 2.55 ms\n",
+ "Wall time: 3.01 s\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%time\n",
+ "\n",
+ "x = inc(1)\n",
+ "y = inc(2)\n",
+ "z = add(x, y)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a99dd0ec",
+ "metadata": {},
+ "source": [
+ "The execution of the cell above took three seconds, this happens because we are calling each function sequentially. The computations above can be represented by the following graph:\n",
+ "\n",
+ "![\"Dask](\"https://raw.githubusercontent.com/dask/dask/main/docs/source/images/inc-add.svg\")
\n",
+ "\n",
+ "\n",
+ "Where the circles are function calls, squares represent objects that are created by one task as output and can be inputs into other tasks, and arrows represent the dependencies between the tasks. From looking at the task graph, the opportunity for parallelization is more evident since the the two calls to the `inc` function are completely independent of one-another. Let's explore how `dask.delayed` can help us with this.\n",
+ "\n",
+ "\n",
+ "### `dask.delayed` \n",
+ "\n",
+ "Using the `dask.delayed` decorator we'll transform the `inc` and `add` functions. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "id": "31abdfa5",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from dask import delayed"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "id": "03aed361",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "CPU times: user 119 µs, sys: 24 µs, total: 143 µs\n",
+ "Wall time: 132 µs\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%time\n",
+ "\n",
+ "a = delayed(inc)(1)\n",
+ "b = delayed(inc)(2)\n",
+ "c = delayed(add)(a, b)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f5546859",
+ "metadata": {},
+ "source": [
+ "When we call the `delayed` version of the functions by passing the arguments, the original function is isn't actually called yet, that's why the execution finishes very quickly. When we called the `delayed` version of the functions, a `delayed` object is made, which keeps track of the functions to call and what arguments to pass to it. \n",
+ "\n",
+ "If we inspect `c`, we will notice that it instead of having the value five, we have what is called a `delayed` object."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "id": "33c59a9b",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Delayed('add-9b51a77b-5b91-4b92-88d8-48db94783550')\n"
+ ]
+ }
+ ],
+ "source": [
+ "print(c)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "4ae1bfee",
+ "metadata": {},
+ "source": [
+ "We can visualize this objects by doing:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "id": "ba3c6980",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAALMAAAF7CAYAAACZ2LqeAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nO3dd3BU5foH8O+mkbKQopCEwM9IQggQpYgQgaElqNSRJkUDCBeVJk3BuVev3LEil6bAFVGEhHJJMPROCiIEEYEAARIIEHpPW9K2PL8/vGSIKWSTs/vuvuf5zGQclpNzvnn8stk9++5ZDRERGLN/cQ6iEzCmFC4zkwaXmUnDSXQApV27dg2HDh0SHcPmvf7666IjKE4j2xPA2NhYDB06VHQMmyfZ/3ZA5ieARMRfFXytX79e9P8ai5G2zEx9uMxMGlxmJg0uM5MGl5lJg8vMpMFlZtLgMjNpcJmZNLjMTBpcZiYNLjOTBpeZSYPLzKTBZWbS4DIzaXCZmTS4zEwaXGYmDS4zkwaXmUmDy8ykwWVm0uAyM2lwmZk0uMxMGlxmJg0uM5MGl5lJg8vMpMFlZtLgMjNpcJmZNLjMTBpcZiYNLjOTBpeZSYPLzKTBZWbS4DIzaXCZmTS4zEwaXGYmDS4zkwaX2QpMJpPoCKrgJDqApcTGxoqOAAAgIuzatQu9evUSHQUAkJKSIjqCxUhb5qFDh4qOUMbKlStFR5CedA8zXn/9dRCRzXyNHz8eAJCYmCg8y+NfMpKuzLZEr9dj7dq1AFD6X2Y5XGYL2rNnD3JzcwEA69evR3FxseBEcuMyW9DatWvh7OwMANDpdNi9e7fgRHLjMltIQUEBNm7cCL1eDwBwdHTEmjVrBKeSG5fZQrZu3YqioqLSPxsMBmzevBk6nU5gKrlxmS1kzZo1cHR0LHObXq/Hli1bBCWSH5fZAnJycrBr1y4YDIYyt2s0GsTExAhKJT8uswVs2LChwpewjUYj9u7di3v37glIJT8uswU86d73559/tlISdeEyK+zmzZv49ddfYTQaK/x7IkJ0dLSVU6kDl1lh69evh4ND5WM1mUxISUlBVlaWFVOpA5dZYTExMeWe+P0VESEuLs5KidRD2lVzIty4cQPZ2dkICAgovc1gMKCoqAharbbMtjIvxRRFQ7IuobIRsbGxGDp0qLQr1WxIHD/MYNLgMjNpcJmZNLjMTBpcZiYNLjOTBpeZSYPLzKTBZWbS4DIzaXCZmTS4zEwaXGYmDS4zkwaXmUmDy8ykwWVm0uAyM2lwmZk0uMxMGlxmJg0uM5MGl5lJg8vMpMFlZtLgMjNpcJmZNLjMTBpcZiYNLjOTBpeZSYMvNq4wIsL169dx+/ZtPHz4ECdOnAAA7Nu3D1qtFh4eHmjUqBG8vb0FJ5UPX2y8FogIaWlpSExMxKFDh5CRkYH09HQUFBQ88XsbNGiA0NBQtGzZEl26dEH37t3h6+trhdTSiuMym8lkMiEpKQkxMTHYuXMn7ty5A29vb3Tu3BktWrRASEgIQkJC4O/vDw8Pj9J745ycHOh0Ouh0OmRlZSE9PR3p6elITU3FkSNHYDQa0bJlSwwaNAhRUVEICgoS/aPamzgQq5Y7d+7Qxx9/TI0bNyYA1L59e5o7dy798ccfZDQaa7Xv/Px82rFjB02dOpX8/f1Jo9FQp06daPXq1aTX6xX6CaQXy2V+gmvXrtGUKVPI3d2dGjRoQP/4xz/o7NmzFjuewWCgnTt30rBhw8jJyYmCgoLo+++/p+LiYosdUxJc5sqUlJTQwoULSavVkq+vL3311Vf08OFDq2a4dOkSvffee+Tq6krBwcG0a9cuqx7fznCZK7J//34KDQ0ld3d3+uKLL4TfK16+fJlee+01AkDDhg2j27dvC81jo7jMjzMYDDR79mxydHSkfv360eXLl0VHKmPbtm0UGBhI/v7+lJiYKDqOreEyP3L79m3q3r07ubq60pIlS0THqVROTg4NGjSIHB0d6V//+heZTCbRkWwFl5mIKDMzk4KDgykoKIiOHTsmOk61fPvtt+Ts7EwjR46kkpIS0XFsQazqzzOnpaXhlVdegZ+fH3bs2IEGDRqIjlRtCQkJGDBgADp06ID4+HjUrVtXdCSR1P0JrWlpaejSpQtCQ0ORlJRkV0UGgIiICOzbtw8nTpzAoEGDUFJSIjqSUKq9Z7527Ro6deqEhg0bIiEhAe7u7qIj1dipU6fQpUsXvPrqq1izZg0cHFR5H6XOe+bc3Fz07NkTXl5e2Llzp10XGQCee+45/Pzzz9i4cSNmzpwpOo4wqizz3/72N+Tk5GDnzp3w8vISHUcRPXr0wIoVKzB//nzEx8eLjiOG2Ceg1rd48WJycHCgPXv2iI5iEW+//TZ5eXnRxYsXRUexNnWdzUhPT0fr1q0xa9YszJ49W3QciygqKkJ4eDi0Wi0OHDgAjUYjOpK1qGsJaGRkJO7cuYNjx47ByUne9yWcOnUKbdu2xbJlyzBmzBjRcaxFPU8A161bh6SkJCxbtkzqIgN/PiGcMGECZs6ciXv37omOYzWquGcuKSlBcHAwXnnlFSxfvlx0HKvIzc1FaGgoRowYgXnz5omOYw3quGeOjo7GrVu38NFHH4mOYjWenp6YOXMmvvvuO9y9e1d0HKuQvsxGoxFz587FqFGj8Mwzz4iOY1XvvPMOtFotFi1aJDqKVUhf5q1bt+LChQuYNWuW6ChW5+7ujqlTp2LJkiUoKioSHcfipC9zdHQ0IiIiEBwcLDqKEGPHjoVOp8OWLVtER7E4qcv84MED7NixA1FRUaKjCNOgQQNERkYiJiZGdBSLk7rMGzZsgJOTEwYMGCA6ilAjR47E7t278eDBA9FRLErqMu/duxfdu3eHVqsVHUWoXr16wWQyITk5WXQUi5K2zESE5ORkdO/eXXSUUidPnsSiRYuQnZ2tyHbV5eXlhTZt2iApKUmR/dkqact86tQp3Lt3z6bK/Ouvv2Lq1Km4deuWItuZo0ePHkhMTFRsf7ZI2jIfP34cbm5uaNWqlegoNiE8PBznzp1DYWGh6CgWI22Zz507h6ZNm6r1XRflNGvWDCaTCefPnxcdxWKk/T+dnp6OZs2aKbrP5ORkTJw4ESEhIWjcuDGGDx+O7777Dkajsdy2v//+O4YMGYImTZogMjISixcvRkXLYKq7XW0FBwfDyckJ586dU3zftkLa5WMXL15E7969FdtfUlISevbsCU9PT4wYMQJPP/009u7di/Hjx+PixYv4+uuvS7dNTk5Gnz594OrqioEDB8LBwQEff/xxuXe1VHc7Jbi4uKBx48a4ePGi4vu2GcLeF2BhgYGBNGfOHMX2N27cOKpTpw5lZ2eX3lZYWEj+/v4UGhpaZttWrVqRt7c3Xbp0qfS2jIwMcnd3JwB05swZs7ZTSps2bejDDz9UdJ82JFbahxn5+fmKXkdi+vTp+P3338vca5aUlMDLywt5eXmltx0+fBipqamYMGECAgMDS29v2rRpmVciq7udkurWrYv8/HyL7NsWSPswQ+kyh4aG4v79+5g3bx5SUlJw+fJlnD9/Hnl5eWjYsGHpdo8ek7Zu3brcPlq2bGn2dkqqV6+e1GWW9p5Zo9Eo+kRq7ty5aNSoET799FPo9XpERkZi5cqV6NSpU5ntHr1k7OjoWG4frq6uZm+nJCXnYYukLbNWq4VOp1NkX3fv3sWHH34IT09PXL16FZs3b8ZXX32FAQMGlFta+eyzzwIA9u/fX24/ly9fNns7JSn928rWSFtmJR8fZmVlwWQyYeDAgWXKcPXq1dJPk3qkXbt2cHZ2Lvdqm8FgwNq1a83eTklcZjvl5eWl2NqGZs2aQavVYv369di6dSvOnz+PlStXomPHjqhXrx50Oh3S09MBAI0bN8bEiRNx6tQpjB07FseOHcPx48cxePBg5Obmlu6zutspKScnB56enhbZt00QfT7FUgYOHEhDhgxRbH+xsbGk1WoJAAEgHx8fWrVqFW3YsIE8PDzIycmpdNuioiIaN25c6bYAKCIigmJiYsqccqvudkooLi4mJycnio2NVWyfNkbei8D8/e9/x/bt25GamqrYPu/fv4/jx4/D398fLVq0KL3Ayv3795GdnV3u3SxXr17FqVOn0Lx589LHyBWp7na1kZaWhrCwMKSmpuL555+3yDEEk/ciMKtWrcL48eORn59f4RkDtYmPj8eQIUOg0+ng5uYmOo4lyHupgbZt26KwsFDRe2Z7dvjwYTRv3lzWIgOQ+AlgWFgYfH19pV+QXl2JiYno0aOH6BgWJW2ZNRoNunTpwmXGn2cxTpw4YVNvVLAEacsMAD179kRSUpLUL+FWx/bt2+Hg4IBu3bqJjmJRUpd58ODBMBqN6r349v/ExMSgV69e8Pb2Fh3FoqQus7e3N/r06aOKa0ZU5vbt20hISFDFtUOkLjMAjBo1CklJSVK/Xagqy5cvR926ddG3b1/RUSxO+jL36dMHTZs2xVdffSU6itU9fPgQ33zzDSZNmmSxlXi2RPoyOzo6YtasWYiJiUFWVpboOFb1n//8BwUFBZg8ebLoKFYh7SuAj9Pr9WjatGnpJzKpQU5ODpo1a4aRI0di7ty5ouNYg7yvAD7O2dkZX3/9NVauXFnh+mEZffTRRzAajfjwww9FR7EaVdwzP9KnTx9cuXIFx44dg7Ozs+g4FnPs2DG0b98eP/74I0aNGiU6jrXIu9CoIufPn0erVq0wY8YMfPrpp6LjWERhYSE6dOgALy8v7N+/X1UfnaaKhxmPNG3aFAsWLMAXX3yBPXv2iI5jEZMnT8a1a9ewatUqNRX5T6JWUos0bNgw8vX1patXr4qOoqhVq1aRRqOhTZs2iY4igryL86uSl5eHl156CRqNBr/88gt8fHxER6q1ffv2oU+fPpg2bZoqz6lDbY+ZH3f9+nV06tQJfn5+SEhIgIeHh+hINXby5El06dKl9KV7lV4sUl2PmR8XEBCAvXv34tKlS+jTp4/F3kRqaUeOHEFkZCTatWuHn376Sa1FBqCCVwCr0rRpUyQmJiIzMxOdO3fG9evXRUcyy969exEZGYnw8HBs2bIFLi4uoiMJpeoyA39eCuvAgQPQ6/Xo3Lkzfv/9d9GRnoiIsGDBAvTu3RtDhgxBfHw83N3dRccST+jzTxty9+5devnll8nFxYUWLlxIJpNJdKQK3b9/n/r3709OTk705Zdf2mxOAWK5zI8xGo30+eefk5OTE7366qt04cIF0ZHKiI+Pp8aNG1OjRo3ol19+ER3H1nCZK3Lw4EEKCwsjV1dX+uSTT6igoEBonvPnz1Pv3r1Jo9FQVFQU3b17V2geG6XO88zVYTAYsGTJEvzzn/9EnTp1MGHCBPj5+WHkyJFWeXy6fft2uLi4YNu2bVi2bBmeeeYZLF68GD179rT4se1UHN8zP8GtW7fogw8+IFdXV9JoNDRz5kw6deqUxY5XUlJCW7dupc6dOxMAatKkCa1cuZL0er3FjikJea+crxRfX1+0b98eJSUl8PT0RGxsLJ577jm0bdsWX375JX777TcYDIZaHSM3NxdbtmzBpEmTEBAQgP79+6OgoAAAUFxcjIiICDg5SXtdeMXww4wn2LJlCwYOHAij0Yjw8HAcOnQIBw4cQExMDLZv346bN2+iXr166NSpE1q0aIGQkBA0a9YMvr6+0Gq1pV/5+fnIycmBTqdDVlYW0tPTkZ6ejhMnTuDYsWMgIjz//PMYNGgQoqKiUKdOHfj7+0Oj0SAwMBAHDx6Ev7+/6HHYMvW+nF0de/fuRZ8+fWA0GkFEGDJkCNavX19mm3PnziExMREpKSk4d+4c0tPTq3WdjoYNGyI0NBQtW7ZE165d0a1bNzz11FOlf28ymeDi4gKj0QhnZ2c0btwYBw8ehJ+fn+I/pyS4zJX59ddf0bNnT+j1ehiNRri4uGDChAlYsGDBE7/35s2buHv3LnQ6XelXvXr14OXlBa1Wi4CAgGpd9Pvpp5/G/fv3Afz5bpng4GAcOHCgTOlZqTh+IFaBlJQUvPzyy6VFBv683Fd17xX9/f0VeUjg6+tbWma9Xo8LFy6ga9euOHDggPQXdKkJfgL4F8ePH8crr7yCkpKSMp+8ajAYrP4rvnHjxmX+rNfrkZGRgZdffln1lxyrCJf5MampqejWrRsKCgrKfYSw0Wi0+hOwRo0alTuLodfrkZqaip49eyr2AUSy4DL/T3p6Onr06FFhkR+x9j2zn59fhRdK1+v1+OOPP9C/f/9yn3alZlxm/PlG186dOyMvL6/Kc8bWvmf29/eHyWSq8O8MBgMOHDiAfv36obi42Kq5bJXqy/xoLXN2dnaVRXZ0dLT6WQQ/Pz/o9fpK/95gMCAhIQFDhw6t9Qs3MlB9ma9fv44mTZqUns+tjI+Pj9XfxfGk3wTOzs5wdHSEj49P6VkPNVN9mbt06YKUlBQcPXoUQ4YMgYODQ4WlFvFiRWXHdHR0hJubG8aPH49Lly5hxYoV8PX1tXI628Pnmf/nhRdewJo1a9ChQwe8//77cHJygkajKf0136hRI6tnevyeWaPRQKPRwMPDAwaDAefPn0dAQIDVM9ky1d8z/9W6devQv39/ZGVlYdq0aaXv2hZRHDc3t9LjBwUFYfny5cjMzISLiwvWrVtn9Tw2T9yKPduTmJhIAOjQoUOlt+Xm5tKcOXNo8eLFQjKNHj2aNm/eTEajsfS26dOnU0BAABUXFwvJZKN4cf7jevfujYKCAiQnJ4uOUqVr164hKCgIy5Ytw+jRo0XHsRW80OiRU6dOoVWrVti2bRt69+4tOs4TjRo1CkeOHEFaWpqqr5XxGC7zI1FRUThx4gROnjxpFxccPHv2LMLCwrBp0yb069dPdBxbwGUG/vy13aRJE/zwww8YOXKk6DjV1rdvX+Tl5eGXX34RHcUWqPfyXI+bN28efH19MWzYMNFRzPLBBx/gwIEDOHTokOgoNkH1Zc7OzsYPP/yAadOm2d3lrbp27YqXXnoJ//73v0VHsQmqL/OSJUvg6OiIsWPHio5SIzNmzMCmTZtw5swZ0VGEU3WZi4uLsWTJEkycOBGenp6i49TIgAEDEBoaWq23c8lO1WX+6aefkJ2djUmTJomOUmMODg6YMmUKYmJicOPGDdFxhFJtmY1GI+bPn4/Ro0fb/Vv4R48eDR8fH3z77beiowil2jLHx8cjMzMTU6ZMER2l1urUqYNJkyZh6dKldnvRdCWotszz58/HgAED0Lx5c9FRFDFx4kRoNBosX75cdBRhVFnmpKQkHD58GDNmzBAdRTGenp4YO3YsFi5ciJKSEtFxhFDlK4C9evVCYWGhzS8oMtejBUjfffcd3nrrLdFxrE19L2fb24Iic6l4AZL6yhwVFYU//vgDaWlpdrGgyFwqXoCkrjLb64Iic6l0AZK6FhrZ64Iic6l1AZJqymzPC4rM9WgB0ty5c0VHsSrVlNneFxSZa8aMGdi8ebOqFiCposxFRUV2v6DIXGpcgKSKMsuwoMhcDg4OmDp1qqoWIElfZqPRiAULFkixoMhco0aNUtUCJOnLLNOCInOpbQGS9GWWbUGRudS0AEnqMsu4oMhcalqAJPUrgLIuKDKXShYgyftytuwLisylggVI8pZZ9gVF5lLBAiQ5y6yWBUXmknwBkpwLjdSyoMhcsi9Akq7MDx48UM2CInPJvgBJujKrbUGRuWRegCRVmYuKirB06VJVLSgyl8wLkCp8AljZBynaumXLlmH69OnIzMy0+KdDWfL0lqXnv3z5ckyZMgWZmZl2u16lgvnHlftMk/Xr1xMA/nrCl6Xw/Gs8/9hKPzotLi6usr9StZSUFMyfP9/ix+H5V6yq+Vda5sGDB1sskD2z1kMwnn/Fqpq/VE8AmbpxmZk0uMxMGlxmJg0uM5MGl5lJg8vMpMFlZtLgMjNpcJmZNLjMTBpcZiYNLjOTBpeZSYPLzKTBZWbS4DIzaXCZmTS4zEwaXGYmDS4zkwaXmUmDy8ykwWVm0uAyM2lwmZk0uMxMGlxmJg0uM5MGl5lJg8vMpMFlZtLgMjNpcJmZNLjMTBpcZiYNLjOTBpeZSYPLzKTBZWbS4DIzaXCZmTS4zEwadlFma31eNauYvcy/0g+Ct5UfwGQyYdGiRZg2bZroKAAAIrLKcXj+Fatq/pWW2dHR0SJhaur9998XHcGqeP7mK1fmjh07Ii4uTkSWCn3//ffYu3cvPvnkE4SFhYmOY3E8/1ogG1ZSUkKenp4EgMaOHSs6jurY2fxjbfoJ4O7du5GbmwsAiI2NRXFxseBE6mJv87fpMq9duxbOzs4AAJ1Oh127dglOpC72Nn+bLXNBQQE2bdoEvV4P4M8nRGvWrBGcSj3scf42W+YtW7agqKio9M8GgwFbtmyBTqcTmEo97HH+Nlvm1atXw8GhbDy9Xo/NmzcLSqQu9jh/myxzdnY29uzZA6PRWOZ2jUaDmJgYQanUw17nb5Nl3rBhQ7lBAoDRaMS+fftw584dAanUw17nb5Nlrupfv0ajQXx8vBXTqI+9zt/mynzz5k0cPHiw0rUJJpMJ0dHRVk6lHvY8f5sr83//+99yTzweZzKZcPjwYWRlZVkxlXrY8/xtrswxMTEwGAxVbkNEiI2NtVIidbHn+Ve6ak6EGzduIDs7GwEBAaW3lZSUQKfTwcfHp8y2KSkp1o4nPXufv4bISgt0ayg2NhZDhw612jpiVpYdzT/O5h5mMFZTXGYmDS4zkwaXmUmDy8ykwWVm0uAyM2lwmZk0uMxMGlxmJg0uM5MGl5lJg8vMpMFlZtLgMjNpcJmZNLjMTBpcZiYNLjOTBpeZSYPLzKTBZWbS4DIzaXCZmTS4zEwaXGYmDS4zkwaXmUmDy8ykwWVm0uAyM2nYdJnz8vKQnZ0NALh37x4KCwsFJ1IXe5u/TVxs/Pbt20hOTkZKSgrOnj2LjIwMXLlypcIPiXF1dUVISAhCQkIQFhaGrl27Ijw8HK6urgKSy0GS+ccJK/OVK1cQExOD9evX4/Tp03B0dESbNm3QsmVLNGvWDIGBgfD29oaHhwfc3NyQm5uLhw8f4v79+8jIyEBGRgaOHz+Oixcvws3NDd26dcObb76JAQMGwM3NTcSPZFcknH8cyMoSEhKoZ8+e5ODgQPXr16fJkyfT9u3bKT8/v0b7u3z5Mq1YsYL69etHzs7OVK9ePZo8eTJlZWUpnFwOEs8/1mplTkpKovDwcAJAERERtHnzZiopKVH0GHfu3KH58+fT//3f/5GLiwuNHTuWbty4oegx7JUK5m/5Mt+8eZPeeOMN0mg01Lt3bzp8+LClD0klJSX0448/UmBgIHl6etKiRYvIYDBY/Li2SEXzt2yZt2zZQk899RQFBATQqlWrLHmoChUUFNAnn3xCderUoRdffJEuXrxo9QwiqWz+limzXq+n6dOnk0ajoTFjxpBOp7PEYart9OnT1LJlS3rqqado+/btQrNYg0rnr3yZCwoKqH///uTh4UExMTFK777GHj58SKNGjSJHR0f6/vvvRcexGBXPX9ky5+bmUpcuXcjHx4dSUlKU3LVi/vWvf5FGo6HPP/9cdBTFqXz+ypW5sLCQunfvTv7+/nT69GmldmsRS5cuJY1GQwsXLhQdRTE8f4XKbDAYaODAgeTt7U0nT55UYpcWN3fuXHJwcKC1a9eKjlJrPH8iUqrMs2fPJldXVzpw4IASu7OaGTNmkKurK504cUJ0lFrh+ROREmVOTk4mR0dHWrp0qRKBrMpoNFJkZCQFBwdTbm6u6Dg1wvMvVbsy5+bmUkBAAA0ZMqS2QYS5fv06NWjQgMaNGyc6itl4/mXUrsxTp04lHx8funv3bm2DCLVu3TpycHCgQ4cOiY5iFp5/GTUvc2pqKjk5OdHy5ctrE8BmREREUJs2bchoNIqOUi08/3JqXubBgwdT27Zt7eZ//pOkpaWRg4MDxcXFiY5SLTz/cmpW5rNnz5KDgwNt2LChpge2SYMHD6bWrVuTyWQSHaVKPP8K1azM48aNo+bNm0tzr/DI8ePHCQDt3btXdJQq8fwrFGv2ewCLiooQFxeHt99+Gw4ONv0WQrO1bt0a4eHhiI6OFh2lUjz/ypk9jY0bN0Kn02HYsGE1OqCti4qKQnx8PPLz80VHqRDPv3Jmlzk+Ph6RkZHw8/Mz+2CP27x5M2JjY2u1D0sYOnQoioqKsHv3btFRKsTzr4I5D0pMJhM9/fTTNG/evJo8pimjXbt29Oyzz9Z6P5bQvn17mjBhgugY5fD8qxTrZE7xT548iXv37qFHjx7m/6v5i0mTJtnsdRh69OiBTZs2iY5RDs//Ccyp/g8//EBardamnkWbTCbFT6Vt3bqVANjceg2ef5XMO5tx7tw5hISEKPIs+r333sNbb71V+udx48Zh0qRJuHHjBkaMGIFnnnkGQUFBGDNmDB4+fFju+1NTUxEZGQkvLy+4u7ujQ4cO2LlzZ61zAUBoaCgAICMjQ5H9KYXn/wTmVL9fv340fPhw8/6ZVeKvj9natWtHgYGBFBAQQJ07d6aZM2dS165dCQANHDiwzPcmJSWRq6srBQQE0LRp02jMmDHk6elJTk5OdPDgwVpnMxgM5OLiQmvWrKn1vpTE86+SeS+avPDCCzRz5kzzklWiomECoFmzZpX+2jIajdS2bVvy9PQs3c5oNFKrVq3I09OTzp8/X3r72bNnSaPR0BtvvKFIvsDAQJozZ44i+1IKz79K5j3MyM/PR926dc276zeDm5sbZs+eDY1GAwBwcHBAp06dkJubi2vXrgEAjh8/jtTUVLz22msIDg4u/d7Q0FB88803aN++vSJZ6tatC51Op8i+lMLzr5pZZzMsPcwGDRqUuwCft7c3AJT+YBcuXAAAPPfcc+W+f9KkSYplqVu3LvLy8hTbnxJ4/lUz657ZZDJZ9CXUqi64R/+7vuPdu3cBAAEBARbLAQBOTk4wGo0WPYa5eP5VM2syWq22wme21hQYGAgA+O2338r9XXR0NFauXKnIcfLy8ix6L1gTPP+qmV1m0WsWXnzxRbi5uSExMbHM7WfOnMHo0aOxf/9+RY5j6V/pNcHzr5pZZW7QoAFu3bpl1vDQ6PEAAATnSURBVAGU5uvri6lTp+LkyZN49913cfToUURHR2P48OFwcnLCu+++W+tjEBFu376N+vXrK5BYOTz/qpn1BDAkJASpqalmHcASPv30UxAR5s6di2XLlgEA/P39sWbNGnTo0KHW+79x4wZ0Oh2aNWtW630pief/BOacyPvmm2+ofv365nyLRel0Ojp06BCdPn2aiouLFdtvYmIiAaBbt24ptk8l8PyrZN5Co1atWuHu3bu4dOkSnn32WfP+1ViAh4cHXnrpJcX3e+TIEfj5+cHX11fxfdcGz79qZj1mDg8Ph7u7O5KSksw6iL1JSEhQZGWa0nj+VTOrzC4uLujYsSMSEhLMPpC9KCoqwsGDB22yzDz/qpl9Br5fv37Ytm0bCgoKzD6YPdi+fTuKi4vRq1cv0VEqxPOvnNllHj58OAoLC21y8boSoqOjERERgYYNG4qOUiGef+XMLnP9+vXx6quv4scffzT7YLbuxo0b2LVrF6KiokRHqRTPvwo1OXWyb98+AmB312Z7kmnTppGfnx8VFhaKjlIlnn+Fan55rg4dOlD//v1r+u02586dO+Th4UHz588XHaVaeP7l1LzM27dvJ41GQ0lJSTXdhU155513yM/PT/gnM1UXz7+c2l3Stm/fvtSiRQvFP+nT2o4ePUqOjo60evVq0VHMwvMvo3ZlzszMJDc3N5o9e3ZtdiNUUVERtW7dmrp162bzF0z8K55/GbX/GIhvv/2WHB0d7fbX3YQJE8jT05MuXLggOkqN8PxLKfMBPYMGDaKGDRvStWvXlNid1axZs4Y0Go3dXJO5Mjx/IlKqzNnZ2dSiRQsKCwujBw8eKLFLi9u3bx/VqVOHZsyYITpKrfH8iUjJD7W8evUqNW7cmDp27Eh5eXlK7dYifvvtN6pXrx6NGDHCpq4OVBs8f4U/bvjMmTPk5+dHbdu2pdu3byu5a8Xs3r2btFot9e3bV9E1uLZA5fNX/oPgMzMzKTg4mIKDgyktLU3p3dfK8uXLycXFhUaOHGn3p7Mqo+L5K19mIqJbt25Rp06dyN3dnVasWGGJQ5glPz+f3nzzTdJoNPTRRx/Z3Sk4c6l0/pYpMxGRXq+nWbNmkUajoddee42ysrIsdagq7dixg4KCgqh+/fq0c+dOIRlEUOH8LVfmR/bt20chISHk4eFBn332mdWenJw9e5YGDBhAAGjIkCF0/fp1qxzX1qho/pYvM9Gfr/J89tlnVLduXfLx8aHZs2fTnTt3LHKso0eP0uuvv04ODg7UokUL2r17t0WOY09UMn/rlPmRBw8e0OzZs8nHx4ecnZ2pX79+tH79esrOzq7VfjMyMmjOnDkUFhZGAKh169YUFxcnzWk3pUg+/1gN0f8uImZFhYWF2LhxI1avXo09e/YAANq2bYsuXbqgRYsWaNasGYKCgqDVaqHVah+tu0ZOTg6ys7ORkZGB9PR0HD9+HElJSbhy5Qp8fHwwdOhQvPnmm+jYsaO1fyS7Iun844SU+XEPHjxAcnIyEhMTcejQIaSnp5d7f5uHh0e5a6w1aNAAYWFh6Nq1KyIiItC+fXs4OztbM7oUJJq/+DL/FRHh6tWruHz5MnQ6HXQ6HR4+fAitVgtvb294enqiadOm8PLyEh1VSnY8f9srM2M1FCfX59UyVeMyM2k4AcgWHYIxBTz8f0BtYAlbB9dFAAAAAElFTkSuQmCC\n",
+ "text/plain": [
+ "![\"Dask](\"https://raw.githubusercontent.com/dask/dask/main/docs/source/images/dask_horizontal_no_pad.svg\"\n",)
\n",
+ " \n",
+ "This notebook was inspired in the materials from: \n",
+ "\n",
+ "- https://github.com/coiled/pydata-global-dask/\n",
+ "\n",
+ " \n",
+ "# Schedulers\n",
+ "\n",
+ "So far we have only seen the power of `dask.delayed` and we got familiarized with the idea of task graphs and we learn that these task graphs need to be executed to get the results of our computation. But what does it mean \"to be executed\"? Who takes care of this? Well, as you might have guess from the title of this notebook, this is the job of the Dask task scheduler. \n",
+ "\n",
+ "\n",
+ "![\"Grid](\"https://raw.githubusercontent.com/coiled/pydata-global-dask/master/images/grid_search_schedule.gif\"\n",)
\n",
+ "\n",
+ "\n",
+ "There are different task schedulers in Dask, and even though they will all compute the same result, but they might have different performances. There are two different classes of schedulers: single-machine and distributed schedulers.\n",
+ "\n",
+ "\n",
+ "## Single Machine Schedulers\n",
+ "\n",
+ "Single machine schedulers require no setup, they only use the Python standard library, and they provide basic features on on a local process or threadpool. Dask provides different single machine schedulers:\n",
+ "\n",
+ "\n",
+ "- \"threads\": The threaded scheduler executes computations with a local `concurrent.futures.ThreadPoolExecutor`. The threaded scheduler is the default choice for Dask arrays, Dask DataFrames, and Dask delayed.\n",
+ "\n",
+ "- \"processes\": The multiprocessing scheduler executes computations with a local `concurrent.futures.ProcessPoolExecutor`. The multiprocessing scheduler is the default choice for Dask Bag.\n",
+ "\n",
+ "- \"single-threaded\": The single-threaded synchronous scheduler executes all computations in the local thread, with no parallelism at all. This is particularly valuable for debugging and profiling, which are more difficult when using threads or processes.\n",
+ "\n",
+ "### Single machine schedulers in action\n",
+ "\n",
+ "Using the same examples we used in the Delayed lesson, let's see how we can modify the scheduler and how this affects the performance of our computations. "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "id": "e15fbe82",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import dask\n",
+ "from dask import delayed\n",
+ "from time import sleep"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "id": "d0b88f61",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "@delayed\n",
+ "def inc(x):\n",
+ " \"\"\"Increments x by one\"\"\"\n",
+ " sleep(1)\n",
+ " return x + 1"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "id": "a73e948a",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "Delayed('sum-de7db1d6-1e32-477b-aded-a34ba2c60cd9')"
+ ]
+ },
+ "execution_count": 3,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "data = list(range(8))\n",
+ "\n",
+ "results = []\n",
+ "for i in data:\n",
+ " y = inc(i) \n",
+ " results.append(y)\n",
+ " \n",
+ "total = delayed(sum)(results)\n",
+ "total"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "4bf0cf9f",
+ "metadata": {},
+ "source": [
+ "### The multi-threading scheduler (default)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "id": "0d6e3709",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "CPU times: user 2.09 ms, sys: 1.19 ms, total: 3.29 ms\n",
+ "Wall time: 1.01 s\n"
+ ]
+ },
+ {
+ "data": {
+ "text/plain": [
+ "36"
+ ]
+ },
+ "execution_count": 4,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "%%time \n",
+ "dask.config.set(scheduler='threads')\n",
+ "total.compute()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "id": "1bc1197d",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "CPU times: user 4.79 ms, sys: 2.42 ms, total: 7.21 ms\n",
+ "Wall time: 2.01 s\n"
+ ]
+ },
+ {
+ "data": {
+ "text/plain": [
+ "36"
+ ]
+ },
+ "execution_count": 5,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "%%time \n",
+ "dask.config.set(scheduler='threads', num_workers=4) #setting num_workers\n",
+ "total.compute()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6a294197",
+ "metadata": {},
+ "source": [
+ "### The multi-process scheduler \n",
+ "\n",
+ "Notice that we can also set the scheduler as a context manager "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "id": "e158bf20",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "CPU times: user 10.6 ms, sys: 19 ms, total: 29.7 ms\n",
+ "Wall time: 6.19 s\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%time\n",
+ "with dask.config.set(scheduler='processes'): \n",
+ " total.compute() "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d0403b14",
+ "metadata": {},
+ "source": [
+ "### The single-threaded scheduler \n",
+ "\n",
+ "Tools like `pdb` do not work well with multi threads or process, but you can work around this by using the single-threaded scheduler when debugging."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "id": "4ecb2d51",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "CPU times: user 5.29 ms, sys: 1.45 ms, total: 6.74 ms\n",
+ "Wall time: 8.04 s\n"
+ ]
+ },
+ {
+ "data": {
+ "text/plain": [
+ "36"
+ ]
+ },
+ "execution_count": 7,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "%%time\n",
+ "total.compute(scheduler=\"single-threaded\") "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b88c0692",
+ "metadata": {},
+ "source": [
+ "For more information about single-machine schedulers, and which one to choose you can visit the detailed the Dask documentation on [single-machine schedulers](https://docs.dask.org/en/latest/setup/single-machine.html). "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0ffcbd5f",
+ "metadata": {},
+ "source": [
+ "## Distributed Scheduler\n",
+ "\n",
+ "The Dask distributed scheduler, despite having \"distributed\" on its nama, it also works well on a single machine. We recommend using the distributed scheduler as it offers more features and diagnostics. You can think of the distributed scheduler as an \"advanced scheduler\". \n",
+ "\n",
+ "The distributed scheduler can be used in a cluster as well as locally. Deploying a remote Dask cluster involves additional setup that you can read more about on the Dask [setup documentation](https://docs.dask.org/en/latest/setup.html). Alternatively, you can use [Coiled](https://docs.coiled.io/user_guide/index.html#what-is-coiled) which provides a cluster-as-a-service functionality to provision hosted Dask clusters on demand, and you can try it for free. \n",
+ "\n",
+ "For now, we will set up the scheduler locally. To set up the distributed scheduler locally we need to create a `Client` object, which will let you interact with the \"cluster\" (local threads or processes on your machine)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "id": "5cd38299",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from dask.distributed import Client"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "id": "0354e700",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ "
"
+ ],
+ "text/plain": [
+ "\n",
+ " \n",
+ "\n",
+ "
\n",
+ "\n",
+ " \n",
+ "
\n",
+ "Client
\n", + "Client-3142c864-167e-11ec-9fb4-1e00ea0a0276
\n", + "| Connection method: Cluster object | \n", + "Cluster type: distributed.LocalCluster | \n", + " \n", + "
| \n", + " Dashboard: http://127.0.0.1:8787/status\n", + " | \n", + "\n", + " |
\n",
+ "
\n",
+ "
\n",
+ " \n",
+ "\n",
+ " Cluster Info
\n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ "\n",
+ "
\n",
+ "LocalCluster
\n", + "a6338b33
\n", + "| \n", + " Dashboard: http://127.0.0.1:8787/status\n", + " | \n", + "\n", + " Workers: 4\n", + " | \n", + "
| \n", + " Total threads: 8\n", + " | \n", + "\n", + " Total memory: 16.00 GiB\n", + " | \n", + "
| Status: running | \n", + "Using processes: True | \n", + "
\n",
+ " \n",
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ " Scheduler Info
\n",
+ "
\n",
+ "\n",
+ " \n",
+ "
\n",
+ "\n",
+ " \n",
+ " \n",
+ "
\n",
+ "\n",
+ " \n",
+ " \n",
+ "
\n",
+ "
\n",
+ " Scheduler
\n", + "Scheduler-3d8c0db7-910f-4c72-8c3f-79d76d73f0b5
\n", + "| \n", + " Comm: tcp://127.0.0.1:57649\n", + " | \n", + "\n", + " Workers: 4\n", + " | \n", + "
| \n", + " Dashboard: http://127.0.0.1:8787/status\n", + " | \n", + "\n", + " Total threads: 8\n", + " | \n", + "
| \n", + " Started: Just now\n", + " | \n", + "\n", + " Total memory: 16.00 GiB\n", + " | \n", + "
\n",
+ " \n",
+ "
\n",
+ "\n",
+ " \n",
+ "
\n",
+ "\n",
+ " Workers
\n",
+ "
\n",
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ " \n",
+ " Worker: 0
\n",
+ "
\n",
+ " | \n", + " Comm: tcp://127.0.0.1:57665\n", + " | \n", + "\n", + " Total threads: 2\n", + " | \n", + "
| \n", + " Dashboard: http://127.0.0.1:57666/status\n", + " | \n", + "\n", + " Memory: 4.00 GiB\n", + " | \n", + "
| \n", + " Nanny: tcp://127.0.0.1:57652\n", + " | \n", + "\n", + " |
| \n", + " Local directory: /Users/ncclementi/Documents/git/dask-mini-tutorial/notebooks/dask-worker-space/worker-dop_ge5z\n", + " | \n", + "
\n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ " \n",
+ " Worker: 1
\n",
+ "
\n",
+ " | \n", + " Comm: tcp://127.0.0.1:57668\n", + " | \n", + "\n", + " Total threads: 2\n", + " | \n", + "
| \n", + " Dashboard: http://127.0.0.1:57669/status\n", + " | \n", + "\n", + " Memory: 4.00 GiB\n", + " | \n", + "
| \n", + " Nanny: tcp://127.0.0.1:57654\n", + " | \n", + "\n", + " |
| \n", + " Local directory: /Users/ncclementi/Documents/git/dask-mini-tutorial/notebooks/dask-worker-space/worker-1wivjaz7\n", + " | \n", + "
\n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ " \n",
+ " Worker: 2
\n",
+ "
\n",
+ " | \n", + " Comm: tcp://127.0.0.1:57660\n", + " | \n", + "\n", + " Total threads: 2\n", + " | \n", + "
| \n", + " Dashboard: http://127.0.0.1:57662/status\n", + " | \n", + "\n", + " Memory: 4.00 GiB\n", + " | \n", + "
| \n", + " Nanny: tcp://127.0.0.1:57651\n", + " | \n", + "\n", + " |
| \n", + " Local directory: /Users/ncclementi/Documents/git/dask-mini-tutorial/notebooks/dask-worker-space/worker-l9wsw5bz\n", + " | \n", + "
\n",
+ " \n",
+ "
\n",
+ " \n",
+ "\n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ " \n",
+ " Worker: 3
\n",
+ "
\n",
+ " | \n", + " Comm: tcp://127.0.0.1:57659\n", + " | \n", + "\n", + " Total threads: 2\n", + " | \n", + "
| \n", + " Dashboard: http://127.0.0.1:57661/status\n", + " | \n", + "\n", + " Memory: 4.00 GiB\n", + " | \n", + "
| \n", + " Nanny: tcp://127.0.0.1:57653\n", + " | \n", + "\n", + " |
| \n", + " Local directory: /Users/ncclementi/Documents/git/dask-mini-tutorial/notebooks/dask-worker-space/worker-6c3pjpi7\n", + " | \n", + "
\n",
+ " \n",
+ "This notebook was inspired in the materials from: \n",
+ "\n",
+ "- https://github.com/coiled/pydata-global-dask/\n",
+ "\n",
+ "# Dask DataFrame\n",
+ "\n",
+ "As we learned on our Dask Delayed notebook we can parallelize functions using `dask.delayed`, but luckily for us we have multiple Dask collections that already include a parallel version of multiple functions. In this notebook we will learn about the [Dask DataFrame](https://docs.dask.org/en/latest/dataframe.html), a Pandas DataFrame interface that will automatically build parallel computations for tabular data.\n",
+ "\n",
+ "## The Dask DataFrame data model\n",
+ "\n",
+ "For the most part, a Dask DataFrame feels like a Pandas DataFrame. However, internally a Dask DataFrame is composed of many Pandas DataFrames (see the image below). \n",
+ "\n",
+ "![](\"http://dask.pydata.org/en/latest/_images/dask-dataframe.svg\")
\n",
+ "\n",
+ "Dask DataFrames are partitioned along their index into different **partitions** where each parition is a normal Pandas DataFrame. These Pandas objects may live on disk or on other machines.\n",
+ "\n",
+ "Dask DataFrames implement a well-used portion of the Pandas API which are backed by blocked algorithms that allow for parallel and out-of-core computation. In particular, Dask DataFrames support:\n",
+ "\n",
+ "- Element-wise operations: `df.x` + `df.y`, `df * df`\n",
+ "\n",
+ "- Row-wise selections: `df[df.x > 0]`\n",
+ "\n",
+ "- Loc: `df.loc[4.0:10.5]`\n",
+ "\n",
+ "- Common aggregations: `df.x.max()`, `df.max()`\n",
+ "\n",
+ "- Is in: `df[df.x.isin([1, 2, 3])]`\n",
+ "\n",
+ "- Datetime/string accessors: `df.timestamp.month`\n",
+ "\n",
+ "- Froupby-aggregate (with common aggregations): `df.groupby(df.x).y.max()`, `df.groupby('x').max()`\n",
+ "\n",
+ "among other operations. For more details about what Pandas operations are implemented in Dask, check the [Dask DataFrame API documentation](http://docs.dask.org/en/latest/dataframe-api.html). \n",
+ "\n",
+ "For many purposes Dask DataFrames can serve as drop-in replacements for Pandas DataFrames. Much like the Dask Delayed interface, Dask DataFrames are lazily evaluated. You can use use the DataFrame API to automatically build up a task graph representing complex computations and then call `compute()` to to evaluate the graph in parallel. \n",
+ "\n",
+ "## When to use Dask DataFrames\n",
+ "\n",
+ "Pandas is great for tabular datasets that fit in memory. If your data fits in memory then you should use Pandas. **Dask becomes useful when the dataset you want to analyze is larger than your machine's RAM** where you would normally run into `MemoryError`s.\n",
+ "\n",
+ "```python\n",
+ " MemoryError: ...\n",
+ "```\n",
+ "\n",
+ "During this tutorial, the example NYC dataset we're working with is only about 200MB so that you can download it in a reasonable time and exercises finish quickly, but Dask Dataframes will scale to datasets much larger than the memory on your local machine. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e37867c1",
+ "metadata": {},
+ "source": [
+ "# Getting started with Dask DataFrames\n",
+ "\n",
+ "Let's use Dask DataFrame's to explore our NYC flight dataset. Dask's `read_csv` function will automatically example wildcard characters like `\"*\"` which can, for example, be used to load an entire directory of CSV files."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "id": "50398620",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "data_dir='/Users/ncclementi/Documents/git/dask-mini-tutorial/data'\n"
+ ]
+ }
+ ],
+ "source": [
+ "%run ../prep_data.py -d flights"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "id": "4169babd",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'../data/nycflights/*.csv'"
+ ]
+ },
+ "execution_count": 2,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "import os\n",
+ "\n",
+ "files = os.path.join('../data', 'nycflights', '*.csv')\n",
+ "files"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "id": "ce4c4e87",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import dask.dataframe as dd"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "id": "b879a54b",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "Dask DataFrame Structure:
\n",
+ "\n",
+ "\n",
+ "\n",
+ " \n",
+ "
\n",
+ "
\n",
+ "| \n", + " | Date | \n", + "DayOfWeek | \n", + "DepTime | \n", + "CRSDepTime | \n", + "ArrTime | \n", + "CRSArrTime | \n", + "UniqueCarrier | \n", + "FlightNum | \n", + "TailNum | \n", + "ActualElapsedTime | \n", + "CRSElapsedTime | \n", + "AirTime | \n", + "ArrDelay | \n", + "DepDelay | \n", + "Origin | \n", + "Dest | \n", + "Distance | \n", + "TaxiIn | \n", + "TaxiOut | \n", + "Cancelled | \n", + "Diverted | \n", + "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| npartitions=10 | \n", + "\n", + " | \n", + " | \n", + " | \n", + " | \n", + " | \n", + " | \n", + " | \n", + " | \n", + " | \n", + " | \n", + " | \n", + " | \n", + " | \n", + " | \n", + " | \n", + " | \n", + " | \n", + " | \n", + " | \n", + " | \n", + " |
| \n", + " | datetime64[ns] | \n", + "int64 | \n", + "float64 | \n", + "int64 | \n", + "float64 | \n", + "int64 | \n", + "object | \n", + "int64 | \n", + "object | \n", + "float64 | \n", + "float64 | \n", + "float64 | \n", + "float64 | \n", + "float64 | \n", + "object | \n", + "object | \n", + "float64 | \n", + "float64 | \n", + "float64 | \n", + "bool | \n", + "int64 | \n", + "
| \n", + " | ... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "
| ... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "
| \n", + " | ... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "
| \n", + " | ... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "... | \n", + "
Dask Name: read-csv, 10 tasks
"
+ ],
+ "text/plain": [
+ "Dask DataFrame Structure:\n",
+ " Date DayOfWeek DepTime CRSDepTime ArrTime CRSArrTime UniqueCarrier FlightNum TailNum ActualElapsedTime CRSElapsedTime AirTime ArrDelay DepDelay Origin Dest Distance TaxiIn TaxiOut Cancelled Diverted\n",
+ "npartitions=10 \n",
+ " datetime64[ns] int64 float64 int64 float64 int64 object int64 object float64 float64 float64 float64 float64 object object float64 float64 float64 bool int64\n",
+ " ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...\n",
+ "... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...\n",
+ " ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...\n",
+ " ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...\n",
+ "Dask Name: read-csv, 10 tasks"
+ ]
+ },
+ "execution_count": 4,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "ddf = dd.read_csv(files,\n",
+ " parse_dates={'Date': [0, 1, 2]},\n",
+ " dtype={\"TailNum\": str,\n",
+ " \"CRSElapsedTime\": float,\n",
+ " \"Cancelled\": bool})\n",
+ "ddf"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7f2a3583",
+ "metadata": {},
+ "source": [
+ "Notice that the representation of the dataframe object contains no data - Dask has just done enough to read the start of the first file, and infer the column names and dtypes."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "id": "aefc5d79",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "Index(['Date', 'DayOfWeek', 'DepTime', 'CRSDepTime', 'ArrTime', 'CRSArrTime',\n",
+ " 'UniqueCarrier', 'FlightNum', 'TailNum', 'ActualElapsedTime',\n",
+ " 'CRSElapsedTime', 'AirTime', 'ArrDelay', 'DepDelay', 'Origin', 'Dest',\n",
+ " 'Distance', 'TaxiIn', 'TaxiOut', 'Cancelled', 'Diverted'],\n",
+ " dtype='object')"
+ ]
+ },
+ "execution_count": 5,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "ddf.columns"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "id": "f4495ebc",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "Date datetime64[ns]\n",
+ "DayOfWeek int64\n",
+ "DepTime float64\n",
+ "CRSDepTime int64\n",
+ "ArrTime float64\n",
+ "CRSArrTime int64\n",
+ "UniqueCarrier object\n",
+ "FlightNum int64\n",
+ "TailNum object\n",
+ "ActualElapsedTime float64\n",
+ "CRSElapsedTime float64\n",
+ "AirTime float64\n",
+ "ArrDelay float64\n",
+ "DepDelay float64\n",
+ "Origin object\n",
+ "Dest object\n",
+ "Distance float64\n",
+ "TaxiIn float64\n",
+ "TaxiOut float64\n",
+ "Cancelled bool\n",
+ "Diverted int64\n",
+ "dtype: object"
+ ]
+ },
+ "execution_count": 6,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "ddf.dtypes"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "eb0b42ec",
+ "metadata": {},
+ "source": [
+ "Dask DataFrames have an `.npartitions` attribute which tells you how many Pandas DataFrames make up a Dask DataFrame."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "id": "c685c721",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "10"
+ ]
+ },
+ "execution_count": 7,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "ddf.npartitions"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "306be7d1",
+ "metadata": {},
+ "source": [
+ "Let's look at the task graph for our Dask DataFrame to get a sense for where these partitions are coming from:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "id": "524b8739",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAABaAAAADkCAYAAABuUYMVAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdeViU9d7H8c8AAwiKCqKCqKDikiKmkolmZoZLqCfNzMxcynZtO51Oyzme8/Sc0tOmLSfTFhXKcjmmmZpllgbuG24JKiBoLiyyKALDfJ8/eqCIbfC+7/nNMJ/Xdc111UD3/U1+3e/pN8OMSUQERERERERERERERET6KnNTPQERERERERERERERNUzcgCYiIiIiIiIiIiIiQ3ADmoiIiIiIiIiIiIgM4aHipNu3b0dGRoaKUzuNtm3bon///qrHcFgZGRnYvn276jEcmslkwvjx41WP4dCWL1+uegSHFx0djZCQENVjOCz2rG7sWe3Ys7qxZ3Vjz+rGntWOPasbe1Y79qxu7Fnd2LO6sWe1Y8/qpqxnosC4ceMEAG+13MaPH6/iR+M0Pv/8c+U/I2e4Uc3KysqU/3yc4bZixQrVPyqHxp7VfWPPasee2XajmrFntt3Ys9qxZ3Xf2LPasWe23ahm7JltN/asduxZ3TdFPbMoewuO8ePHQ0R4q+Y2btw4VT8Wp6P6Z+Wot88//1z1j8ZprFixQvnPyxFvZWVlqn80ToM9q/nGntlO9c/KUW/sme3Ys+pv7Jnt2LOab+yZ7VT/rBz1xp7Zjj2r/sae2Y49q/mmsmd8D2giIiIiIiIiIiIiMgQ3oImIiIiIiIiIiIjIENyAJiIiIiIiIiIiIiJDcAOaiIiIiIiIiIiIiAzBDWgiIiIiIiIiIiIiMgQ3oImIiIiIiIiIiIjIENyAJiIiIiIiIiIiIiJDcAOaiIiIiIiIiIiIiAzBDWgiIiIiIiIiIiIiMgQ3oImIiIiIiIiIiIjIENyAJiIiIiIiIiIiIiJDcAOaiIiIiIiIiIiIiAzBDWgiIiIiIiIiIiIiMgQ3oP+f1WpVPQIRubjCwkLVI5CTs1gsKCsrUz0GNQAFBQXIzs5WPQYRuSgRQW5uruoxiMiFWSwWFBUVqR6DnJzFYoGIqB7DIbj0BnRycjKeeOIJhIaGIiAgALGxsdi8ebPqschJhYeHY8aMGarHICezb98+DBs2DP7+/mjSpAlat26Nhx56CPn5+apHIyfy6aefIjo6Gk2aNIG3tze6du2Kd999l0+u0jXJzs5G165dcdNNN6kehZyE1WpFZGQkunfvXuV2++23qx6PnEhubi5mzJgBX19f+Pv7w8/PDxMmTEBWVpbq0cjBbd68udpr0O9v9913n+oxyQls2rQJvXv3ho+PD3x9fREWFsbH1VRv69evR+/evdG4cWO0atUKd999N7Zu3ap6LKU8VA+gSlFREUaPHo0zZ87gnnvuQUBAAFatWoVRo0Zh48aNGDRokOoRyYksXrwYJ06cwODBg1WPQk5kz549GDp0KDw8PHDPPffA398fX3zxBRYuXIj9+/djx44dcHNz6ecJyQZLly7F1KlT0aVLFzzxxBMoKirCqlWrMHPmTFy6dAkvvfSS6hHJydx///04e/YsmjZtqnoUchKZmZlISkpCREQE/P39K33tj39PVJOSkhKMHDkSO3fuxPTp09G/f3/s2rULCxcuRGZmJhISElSPSA7MZDLBbDZX+7WrV6/i+PHjCA8Pt/NU5Gw2b96M4cOHo1mzZpg2bRrMZjNWrlyJmTNn4uLFi/jnP/+pekRyAsuWLcOkSZMQGhqKZ599FmfOnMHy5cuxYcMG7Nq1C126dFE9ohIuuwH94osv4vjx41i/fj1GjBgBAHjiiScQGRmJqVOn4tSpU4onJEeXmZmJf/7zn9i9ezcOHjyoehxyQu+++y6Kioqwc+dO9OrVCwDwP//zPxg6dCg2b96MVatWYfz48YqnJEf3xhtvIDw8HDt37oSfnx8A4LnnnkNYWBjee+89bkBTvbz//vvYuHEjNw2pXk6cOAEAiIuLQ2RkpOJpyFktWbIEO3bswOuvv45nnnkGwK9PiJlMJnzwwQfYs2cP+vbtq3hKclRDhgzBgQMHqv3azJkzkZ+fjwULFth5KnI2L7/8MkQEu3fvRseOHQEAr776KkJCQvDGG2/g73//O9zd3RVPSY6spKQEzz77LHx9fbFv3z40a9YMADBnzhyEhITg7rvvxv79+xVPqYbLvrRu8eLF6NmzZ8XmMwC0atUKw4YNQ2pqKnbu3KlwOnIGBQUFSE5ORtOmTREVFaV6HHJCiYmJ6NWrV8Xmc7lp06YBAHbt2qViLHIieXl5OHz4MEaMGFGx+QwAwcHBGDJkCHJyclBaWqpwQnImR44cwTPPPIO5c+ciKChI9TjkRFJSUmAymdC5c2fVo5ATi4+PR8uWLTFz5sxK97/wwgtYunQpAgMDFU1Gzmzjxo34z3/+g/j4eLRu3Vr1OOTgMjIyEBISUrH5DABNmjTBDTfcgNLSUly9elXhdOQMjh49ijNnzmDkyJEVm88A0LJlS8TExODAgQPIy8tTOKE6LrkBnZWVhdzcXAwdOrTK18ofOO/Zs8feY5GT6datG3788Uf8+OOP+Oyzz1SPQ06mtLQUw4YNw+OPP17laxkZGQD4a8tUNw8PD2zduhXPPfdcpfvz8vKQlJSEmJiYGn8dlej3rl69iokTJ+Kmm27CrFmzVI9DTubEiRNo164dCgsLsW7dOnz00UdITEzkh6JSvaSkpGDEiBHw9PTEqVOnsHbtWuzduxfBwcGYPHky2rdvr3pEcjLZ2dmYPn06JkyYgCFDhqgeh5zAHXfcgczMTKxfv77ivuPHj2PLli245ZZb4Ovrq3A6cgZnz54FANxwww1VvlZ+35EjR+w6k6NwybfgOH78OABU++qe8vdiuXDhgl1nIiLXYjab8c4771S5/8KFC3jvvfdgNpsRGxurYDJyJr6+vhgwYEDF38+bNw/p6en4+uuvUVZWhueff17hdORMnn32WZw9exbffPMNTCaT6nHIyZw4cQL5+fkIDQ3FlStXKu7v06cP4uLi0K1bN4XTkTMoLCzEL7/8glatWmHUqFFYt25dxde6du2KTz75BDfeeKPCCckZPfbYY7h06RLmzJmjehRyEjNnzsTmzZsRGxuL6OhoeHt7Y8uWLQgODsa//vUv1eOREyh/9fz3339f8XZS5Y4ePQrg1w3o6Ohou8+mmku+Arr8feqqe3Vh+TPrly5dsutMRETr1q1DREQEzpw5g9dffx0RERGqRyIn8+KLL2LevHlISUlBixYt0KhRI9UjkRNYt24d3n33XSxatIhvvUHX5MSJEygoKMDs2bORnJyMhIQEPPjggzhw4ADGjBmDy5cvqx6RHFz5/5/Nnz8fqampeOedd7B37168/fbbSE9Px5gxY/gCIaqXI0eOYPny5Xj66afRrl071eOQk2jWrBnat29f8T7QO3bsgNVqhYeHBwoKClSPR04gPDwcffv2xebNm/Hhhx+ioKAAeXl5eO+997BixQoAcNnfEHPJDWgvLy8AQE5OTpWvlT9Abt68uV1nIiLXdfLkSYwePRqjRo2Cn58fNm3axF+Bp2ty+fJlJCcn4+OPP0Z2djb69euHc+fOqR6LHNgvv/yCadOm4YEHHsAdd9yhehxyUosXL8b+/fvxl7/8BeHh4YiOjsYHH3yAp59+GikpKfjvf/+rekRycOX/X1ZcXIyVK1fi8ccfR+/evTFz5kw8/fTTuHDhAj7//HPFU5Iz+fe//w1PT088/fTTqkchJ3LTTTdhzZo1+M9//oNz587h4sWLWLVqFfLz83H77bcjLS1N9Yjk4Nzc3PDxxx+jdevWmDFjBlq3bo3WrVvj6aefxowZMwAA3bt3VzylGi65AV3+4QOnTp2q8rXyBz/8kAsisof4+Hj06tULP/zwA/7973/j8OHD1b4/PVF1RARWq7XSfeHh4Zg2bRrmzJmD0tLSSu9hR/RH77//PrKyspCXl4dp06ZV3M6cOYOzZ89i2rRpePXVV1WPSQ6uT58+6NGjR5X7R44cCQA4fPiwvUciJ9OmTRsAwI033oiuXbtW+tqoUaMAAMeOHbP7XOScTp8+jc8++wx/+tOf+JkqZLNjx47h0KFDGDx4MB555BE0b94cjRo1wtixYzF16lRcuXKFT6iSTSIiInDo0CF8+OGHePjhh/Hyyy9j165dFW9J5qob0C75HtCdO3eGyWSqdgP64MGDAIB+/frZeywicjHr1q3Dfffdh/79+2PZsmX89UCqtzlz5uCFF17A119/XbHRU65FixYAfvtQS6LqBAYGolevXkhJSal0f3FxMaxWKw4cOAA3N5d8vQLZKCMjA7t27UJUVFSVjpU/1m7ZsqWK0ciJlK+d0tLSKl8rKioCADRt2tSuM5HzWrhwISwWC+6//37Vo5ATSUpKAgDcfPPNVb5222234c0330Rubq69xyInU1JSgtTUVLRo0aLKNWjOnDkICgpy2SfGXPL/KIKDgzFo0CBs3boVJ0+erLi/tLQUn332Gdq0aYM+ffoonJCIXMELL7yApk2bYuXKldx8pmtS/j7h3377bZWvLVq0CAAQGRlp15nIucycORP79++vcuvWrRs6dOiA/fv346OPPlI9JjmwnJwc3HnnndV+ONMXX3wB4NdfaSaqTaNGjTBkyBDs3bu3yhNiX375JQC45Ac20bXZtGkT/P39ceutt6oehZzIddddBwBYuXJlla8tX74cAPgZPVSnK1euoGvXrpg5c2al+zMzM7Fq1SqMHj1a0WTqueQroIFfN35uv/123HXXXXjxxRfRvHlzzJ07F6dOncK6dev4CfBEZKjc3FwcPnwY119/Pd54441qv2fw4MGIjY2182TkTEaOHImIiAi88847aNasGYYNG4YzZ85gxYoV+OqrrxAVFcU1RESG6tmzJ/r3749FixYhICAAY8eOhdVqRXx8PDZt2oRx48bhhhtuUD0mOYE5c+agX79+uOuuu/DKK6+gbdu2+P777/HBBx9g4MCBLv0/7WS73Nxc7N27F6NGjeJv8FC9dO/eHTExMdi0aROGDx+Oe++9F6GhoVi9ejWWLVuG7t27409/+pPqMcnBNWvWDEOGDMHKlSsxdOhQ3HHHHThx4gQefPBBhISE4LXXXlM9ojIuuwEdExODuLg4PPDAAxg3bhyAXxfKm2++iREjRiiejogauoSEBIgI9u3bh3379lX7PSaTiZuHVCs3Nzd8+eWXuPfee/GPf/wD//jHPyq+NnbsWLz99tvw8HDZ1BORHZhMJnz55Zd44IEH8Oqrr1Z6z/BHHnmkxidZif4oKioKX3/9NaZNm1bpbaVGjx6NTz75ROFk5Ey2bNkCq9WK/v37qx6FnIybmxuWLVuGmTNnYtmyZfjmm28qvjZo0CB8/PHH8PT0VDghOYuPP/4YEydOxP3331/xNhy9e/fGZ599hiZNmiieTh2X/r/Su+++G3feeSf27NkDq9WKfv36wd3dXfVY5IQ6deoEEVE9BjmR2NhYrhnSRYcOHfDTTz8hNTUVP//8Mxo1aoQuXbpUfKAT0bXYu3ev6hHIibRs2RJr165Feno6jh8/jmbNmqFbt24u/T9ZdG1GjBiBjIwMHD58GBcvXkRERASCgoJUj0VOZOzYsXyMTdfM398fn376KebOnYsjR46gqKgIXbt2RZcuXfhb8mSz9u3bIyEhAYcOHcKpU6fQu3dvvuUmXHwDGgA8PDxw4403qh6DiIjomrm5uaFjx47o2LGj6lGIyIW1b98e7du3Vz0GOTmz2Yzrr79e9RhE5MJCQkIQEhKiegxyYiaTCT179kTPnj1Vj+Iw+KZIRERERERERERERGQIbkATERERERERERERkSG4AU1EREREREREREREhuAGNBEREREREREREREZghvQRERERERERERERGQIbkATERERERERERERkSG4AU1EREREREREREREhuAGNBEREREREREREREZghvQRERERERERERERGQIbkATERERERERERERkSG4AU1EREREREREREREhuAGNBEREREREREREREZghvQRERERERERERERGQIbkATERERERERERERkSG4AU1EREREREREREREhuAGNIDi4mKUlJSoHoOcXEFBgeoRyMlxDZFW7Bnpgdci0opriLRiz0gPvBaRVlxDpBV79hsPVSfOzMzE8uXLVZ2+kt27d8PNzQ19+vRRPQqAX/9s2rVrp3oMp+Aoa6i0tBTx8fGYNm2a6lEAADt27FA9gtPYvn07rFar6jEAAIsWLcLUqVNhNptVjwIRUT2C02DPasae2c5R1hB75rzYs+qxZ7Zjz2rGntnOUdYQe+a82LPqsWe2Y89qprJnyjagt2/fju3bt6s6vcPjAxzbTJgwQfUIlWzcuFH1CFRPb775puoRKvnuu+9Uj0D1xJ7Vjj2zDXtGWrFnpBV7Vjv2zDbsGWnFnpFW7FntVPXMJC7+NEpBQQECAwMBAFlZWWjcuLHiicgZjRkzBmvXrsXy5csxfvx41eOQE1q2bBnuuecejBs3DitXrlQ9Djkh9oz0wJ6RVuwZacWekR7YM9KKPSOt2LNKylz+PaDXrFmD0tJSlJaWYu3atarHISeUl5eHDRs2wGQyIT4+XvU45KQ+/fRTmEwmfPXVV3yvMbom7BlpxZ6RHtgz0oo9I63YM9IDe0ZasWeVufwGdFxcHEwmE0wmE+Li4lSPQ05o1apVsFqtEBGsX78eOTk5qkciJ5Obm4tvvvkGIgKLxYLVq1erHomcEHtGWrFnpBV7Rnpgz0gr9oy0Ys9ID+xZZS69AZ2VlYXNmzejrKwMZWVl+Pbbb5Gdna16LHIyv7+QiAi+/PJLhdOQM1qxYkWlD9pgnKi+2DPSA3tGWrFnpBV7Rnpgz0gr9oy0Ys+qcukN6Oo+FXPVqlUKJiFndeHCBWzduhVlZWUAfn2AwzhRff1+zVitVmzZsgXnz59XOBE5G/aMtGLPSA/sGWnFnpFW7BnpgT0jrdizqlx6AzouLg6//wxGEcHSpUsVTkTO5vPPP4eb22//GVmtVvz44484e/aswqnImZw9exaJiYmVnmEHGCeqH/aMtGLPSCv2jPTAnpFW7BlpxZ6RHtizqlx2AzojIwM7d+6sdFGxWq1ITExEZmamwsnImSxdurTi2fVy7u7uWLFihaKJyNn88UEy8Ou1yNXjRLZjz0gP7BlpxZ6RVuwZ6YE9I63YM9KKPauey25Af/bZZ3B3d69yv4eHB7744gsFE5GzOXXqFPbt21fpWS0AKCsrw5IlSxRNRc6mugfJIoJdu3YhLS1NzVDkVNgz0oo9Iz2wZ6QVe0ZasWekB/aMtGLPqueyG9DVXVQAwGKx8JktssmyZcvg4eFR5X4Rwf79+5GSkqJgKnImJ0+exMGDB6s8SAYYJ7Ide0ZasWekFXtGemDPSCv2jLRiz0gP7Fn1XHID+ueff8bRo0ervaiICJKSknD8+HEFk5EziYuLQ2lpabVfM5vNjBPVKT4+HmazudqvlZaW8pUaVCf2jPTAnpFW7BlpxZ6RHtgz0oo9I63Ys5q55Ab0p59+WuNFBQA8PT2xbNkyO05Ezqaui0ZpaSk++eQTO05Ezig+Pr7GB8kAcOzYMRw5csSOE5GzYc9IK/aM9MCekVbsGWnFnpEe2DPSij2rmUtuQNf2zCgAlJSUYPHixfYbiJzOsmXLar2oAL++B9nBgwftNBE5m3379uHEiRO1fo+npyc+//xzO01Ezog9I63YM9KKPSM9sGekFXtGWrFnpAf2rGYutwG9a9cupKen1/l96enp2Lt3rx0mImcjIli6dGmtF5VyrvrMFtXNlgcuJSUlWLJkSbW/vkPEnpFW7BnpgT0jrdgz0oo9Iz2wZ6QVe1a7qu/Q38Clp6dj/Pjxle7bvXs3ACAqKqrS/WlpaejTp4/dZiPnkJmZiQEDBlS678yZM9izZw/GjBlT6X6LxWLP0ciJWCyWStciEcHatWsRFRWF4ODgSt979uxZtGnTxt4jkoNjz0gr9oz0wJ6RVuwZacWekR7YM9KKPaudSfjUDcaPHw83Nzd+KAFdsy+++AITJ06E1WpVPQo5KavVCnd3d6xYsQJ33nmn6nHISbFnpBV7RlqxZ6QH9oy0Ys9IK/aM9MCeVShzubfgICIiIiIiIiIiIiL74AY0ERERERERERERERmCG9BEREREREREREREZAhuQBMRERERERERERGRIbgBTURERERERERERESG4AY0ERERERERERERERmCG9BEREREREREREREZAhuQBMRERERERERERGRIbgBTURERERERERERESG4AY0ERERERERERERERmCG9BEREREREREREREZAhuQBMRERERERERERGRIbgBTURERERERERERESG4AY0ERERERERERERERmCG9BEREREREREREREZAhuQBMRERERERERERGRIbgBTURERERERERERESG4AY0ERERERERERERERmCG9BEREREREREREREZAhuQBMRERERERERERGRIbgBTURERERERERERESG4AY0ERERERERERERERmCG9BEREREREREREREZAhuQBMRERERERERERGRIbgBTURERERERERERESG4AY0ERERERERERERERmCG9BEREREREREREREZAhuQBMRERERERERERGRIbgBTURERERERERERESG4AY0ERERERERERERERmCG9BEREREREREREREZAgP1QPYk4ggOzu74nblyhVYrVZkZmbCZDJh06ZNcHNzg6+vLwICAipuRL9XWFiI7OxsZGVlITc3FwCwa9cuAMC6devg7e0Ns9lcaQ15enqqHJkcTElJSaVrUWlpKYqKigAAO3fuRLNmzQAA/v7+FWuocePGKkcmB8OekR7YM9KKPSOt2DPSA3tGWrFnpBV7VjeTiIjqIfSWn5+PgwcPIikpCUlJSTh58iTS0tKQkZGBkpKSeh3L09MT7dq1Q2hoKDp27IiePXtW3Pz8/Az6NyDVLBYLjh49iqSkJBw6dAg///wz0tLSkJaWhvz8/HofLygoCGFhYQgNDUWPHj0QERGByMhItG3b1oDpyVGcPn264jp0+PDhijX0yy+/1PtYfn5+CA0NRWhoKLp27VpxHerWrRs8PFzquUSXwp6RVuwZ6YE9I63YM9KKPSM9sGekFXt2zcoaxAZ0VlYWNm3ahG3btiEhIQFHjhyB1WpF8+bN0bNnT3Tq1KniwhAcHIyAgAC0aNECPj4+cHNzQ9OmTQEAeXl5sFqtuHz5csUzqGfPnkV6ejrS0tKQkpKCpKQkXLp0Ce7u7ujevTsGDBiAQYMG4bbbbnO5Zy8akpKSEmzbtg0//PADfvrpJ+zevRuXL1+Gp6cnrrvuOnTr1q1iDYWEhFSsIX9/fwBAkyZN4OHhgaKiIly9ehWlpaUVa+jixYsVa+jkyZM4cuQI0tLSAABt2rRBdHQ0BgwYgJiYGHTr1k3hnwJpdeTIEXz77bdISEhAYmIizp49CwAVD2w7dOiAsLAwtG/fHoGBgRXPeprNZnh7e6NRo0awWCwoKCgAAOTk5CArKwvZ2dnIzMxEamoq0tLScOzYMRw7dgwlJSXw9fXFDTfcgIEDB+KWW27BwIEDYTabVf4xkAbsGWnFnpEe2DPSij0jrdgz0gN7RlqxZ7px3g3o06dPIz4+HmvWrMGePXvg4eGBqKgo9O/fHwMGDEDv3r3Rrl07Q86dnp6Offv2VVzE9uzZA6vViqioKIwaNQrjx49HeHi4Iecm/Vy5cgX//e9/sXLlSmzevBmFhYXo0qVLxQOOG264AV27djUkFnl5eTh48CASExMr1lFOTg5CQ0MxcuRI3HnnnRg0aBDc3d11PzfpR0SQkJCATz/9FOvXr8fp06cREBCAAQMGIDo6Gv3790dkZGRFdPRUWlqKY8eOYdeuXUhISEBCQgJSUlLQpEkTDB06FH/6058wZswYQ85N+mLPSCv2jLRiz0gP7BlpxZ6RVuwZ6YE9M0QZxInk5+fLxx9/LIMHDxY3NzcJDAyUBx98UFavXi35+fnK5srLy5NVq1bJjBkzpGXLlgJArr/+ennzzTclKytL2VxUVVlZmWzevFmmTJkiTZo0EbPZLLfffru8//77kpaWpmwui8UiiYmJ8re//U169eolAKRVq1Yyc+ZMOXjwoLK5qHopKSkye/Zs6dChgwCQyMhI+fvf/y47duyQsrIyZXOdOnVK3nvvPbn99tvFy8tLvLy8ZPTo0bJ69WopLS1VNhdVxZ6RVuwZ6YE9I63YM9KKPSM9sGekFXtmOItTbEDv3LlTJk2aJD4+PuLl5SV33nmnrFmzRkpKSlSPVoXFYpHvvvtOZsyYIX5+fuLl5SUTJ06UPXv2qB7NpZ0/f15eeukladu2rQCQqKgoefvtt+XixYuqR6vW8ePH5eWXX5bOnTsLAOnXr5/ExcUxUgqVlJTIkiVLZMCAAWIymSQoKEieeeYZh30AmpubK4sXL5aYmBhxc3OT4OBg+dvf/iYXLlxQPZpLY89IK/aMtGLPSA/sGWnFnpFW7BnpgT2zG8fegP7hhx8kJiZGAEjfvn1lwYIFkpOTo3osmxUWFspHH30kffr0EQAyfPhw2bp1q+qxXEpGRobMmjVLfHx8pFWrVvL888/L0aNHVY9lM6vVKlu2bJGJEyeKh4eHdOjQQT744AO5evWq6tFcxtWrV+X999+XsLAwMZvNcvfdd8uGDRvEYrGoHs1mp06dkpdeeklatmwpvr6+8uSTT0pmZqbqsVwKe0ZasWekFXtGemDPSCv2jLRiz0gP7JndOeYG9LZt2+TWW28VADJgwABZu3at6pE027Ztm8TGxlb6d7JararHarBOnTols2bNEm9vb2ndurXMmTNHLl++rHosTdLS0mTWrFnSqFEjadWqVYP4d3JkhYWFMm/ePAkJCRFPT0+ZPHmyJCcnqx5Lk6tXr8oHH3wgbdu2rfh3On78uOqxGjT2jLRiz0gr9peXDrcAACAASURBVIz0wJ6RVuwZacWekR7YM2UcawP64MGDMnDgQDGZTDJy5Ej56aefVI+ku61bt8rw4cMFgAwaNEgOHz6seqQG5dKlSzJr1izx8PCQTp06yYcffijFxcWqx9JVZmamPPXUU+Lr6ytBQUESHx/viBcXp2W1WmXJkiXSqlUrady4sTzzzDNy9uxZ1WPpqri4WBYuXCgdOnQQDw8Peeqpp5S+r1VDxJ6RVuwZacWekR7YM9KKPSOt2DPSA3umnGNsQOfn58tTTz0lHh4ecuONN8ru3btVj2S4nTt3yg033CBms1n+/Oc/S0FBgeqRnN6nn34qQUFBEhgYKB999JFT/QrOtbhw4YI8/PDD4ubmJrfccotT/eqaozp8+LAMGjRI3N3d5bHHHnO2N/Wvt9LSUlm4cKEEBARIcHCwfP7556pHcnrsGXumB/aMPdOKPWPPtGLP2DM9sGfsmVbsGXumFXvmMD1TvwG9du1aadeunTRv3lzmzZun9BNK7a38mbwWLVpIcHCwLFmyRPVITik5OVliYmLEZDLJ5MmTHfaDK4yyd+9e6devn5jNZpk1axafKb0Gly9fltmzZ4unp6f07t1bduzYoXoku8rJyZFZs2aJm5ubDB48WI4cOaJ6JKfEnrFnWrFn7JlW7Bl7pgf2jD3Tij1jz7Riz9gzPbBnDtUzdRvQWVlZMmrUKDGZTHL//fc3+GeyanPhwgWZOnWqmEwmueOOO5zqjc9Vslqt8tprr4mnp6f06dNHdu7cqXokZSwWi7z33nvSrFkzCQsLc7lAa5GQkCDt27cXf39/WbBggUtF6Y8SExOlV69e4uXlJW+99RZ/ddBG7Nlv2LNrw579hj27duzZb9iza8Oe/YY9uzbs2W/Ys2vHnv2GPbs27NlvHKhnajagExISpF27dtK2bVtH/5RGu9qyZYu0adNGQkNDGag6ZGVlye233y5ms1nmzp3b4H+dy1bnzp2T4cOHi9lsljfeeIOBqoXVapV///vf4uHhIbGxsXLhwgXVIzkEi8Uir7zyinh4eMiYMWMkOztb9UgOjT2rHntmO/aseuyZ7diz6rFn9cOeVY89sx17Vj32zHbsWfXYs/phz6rnAD2z7wa01WqVefPmidlsltjYWJd+FqImFy9elJEjR4qHh4fMnj3bpZ/tq8nOnTslLCxM2rZt2yDfOF6r3/93NmrUKAaqGuUPkPnfWc127twpoaGh/O+sBuxZ3dizurFntWPP6sae1Y09qx17Vjf2rG7sWe3Ys7qxZ3Vjz2rHntVNcc/stwGdk5MjI0eOFLPZLK+//jqf+auF1WqVV199VTw8PGTUqFFy6dIl1SM5jPJnREePHs1w12Hbtm0SEhIi7dq1c+lff/ujxMRECQkJkfbt28v27dtVj+PQLl68KCNGjKh4xQb9ij2zHXtWM/bMduxZ9dgz27Fn1WPPbMee1Yw9sx17Vj32zHbsWfXYM9sp7Jl9NqDPnDkjPXr0kJCQEP7qUj1s27ZNgoODJTIyUs6ePat6HKXKysrkkUceEXd3d/7qUj1cvHhRhg8fLr6+vvL111+rHke5tWvXSqNGjSQ2Npbv5Wcjq9Uqc+bMETc3N5k5c6bLvxqBPbs27Nlv2LNrw55Vxp7VH3tWGXt2bdiz37Bn14Y9q4w9qz/2rDL27Noo6JnxG9AnT56Ujh07Srdu3SQ9Pd3o0zU4aWlp0qVLFwkNDZXjx4+rHkeJ4uJimTBhgnh5ecmKFStUj+N0LBaLzJgxQzw8POTDDz9UPY4yS5cuFbPZLFOmTJHS0lLV4zid1atXi7e3t4wdO1aKiopUj6MEe6YNe8aeacWe/Yo904Y9Y8+0Ys/YM63Ys1+xZ9qwZ+yZVnbumbEb0Lt375aWLVtKVFQU30Beg+zsbOnfv7+0atVK9u7dq3ocuyooKJCYmBhp3LixbNq0SfU4Tstqtcrs2bPFZDLJ3LlzVY9jd/PmzROTySTPPfccX52hwZYtW8TPz0+GDBkieXl5qsexK/ZMH+wZe6YVe8ae6YE9Y8+0Ys/YM63YM/ZMD+wZe6aVHXtm3Ab05s2bpXHjxjJ8+HApLCw06jQuo6CgQG677TZp0qSJbNmyRfU4dpGVlSV9+vSR1q1by/79+1WP0yC8+eabFaF3FX/+85/Fzc1N5s+fr3qUBmHv3r3SqlUr6du3r8u8zx97pi/2jD3TA3tGWrFn7JlW7Bl7pgf2jLRiz9gzrezUM2M2oHft2iWNGzeWCRMmSElJiRGncEnFxcVy5513ip+fX4N/pr2wsFBuvPFGad++vZw8eVL1OA3K0qVLxc3NTf71r3+pHsVw//znP8Xd3V3i4+NVj9KgpKSkSLt27SQ6OlouX76sehxDsWfGYM9ID+wZacWekVbsGemBPSOt2DPSyg49038DOiUlRVq1aiVDhw6V4uJivQ/v8kpKSmT48OESGBgoP//8s+pxDFFSUiIjRoyQFi1ayLFjx1SP0yC9//77YjKZZNGiRapHMcyCBQsEgLz99tuqR2mQkpOTpWXLlnL77bc32PdsY8+MxZ6RHtgz0oo9I63YM9IDe0ZasWeklcE903cD+syZMxIaGio33HCDFBQU6Hlo+p3Lly9L//79JSwsrMF9+rLVapWpU6eKj4+PJCYmqh6nQfv73/8u7u7usnLlStWj6O7LL78Ud3d3efnll1WP0qCVP/s8adKkBvfebeyZfbBnpAf2jLRiz0gr9oz0wJ6RVuwZaWVgz/TbgL506ZL06tVLwsPD5fz583odlmpw8eJF6dq1q0REREhOTo7qcXTz9NNPi6enp2zcuFH1KC5h1qxZ4u3tLT/++KPqUXSzZcsW8fb2lkcffVT1KC7hu+++Ey8vL/nLX/6iehTdsGf2xZ6RHtgz0oo9I63YM9IDe0ZasWeklUE902cD2mq1ysiRI6Vt27Zy+vRpPQ5JNkhNTZXg4GAZM2ZMg3h266OPPhI3Nzf54osvVI/iMiwWi4wdO1ZatGghGRkZqsfRLC0tTfz9/eXuu++WsrIy1eO4jPj4eDGZTBIXF6d6FM3YMzXYM9KKPSM9sGekFXtGWrFnpAf2jLQyoGf6bEDPmTNHzGazJCQk6HE4qoft27eL2WyWt956S/Uomhw+fFh8fHzk+eefVz2KyyksLJTrrrtObrzxRqd+E//S0lIZMGCA9OjRo8F/8IIjevrpp8XX11eOHj2qehRN2DN12DPSij0jPbBnpBV7RlqxZ6QH9oy00rln2jegd+zYIWazWd544w09BqJr8Morr4jZbHba9+RqKIF1ZuUPMF944QXVo1yzZ555pkEE1lmVlJRIdHS0Uz/AZM/UY89IK/aMtGLPSA/sGWnFnpFW7BnpQceeaduAzsnJkdDQUBk5cmSD+BUjZ2W1WmXUqFHSrl07yc7OVj1OvU2ZMkWaN28uaWlpqkdxaYsWLRI3NzenfH+3r7/+WkwmkyxZskT1KC7t9OnTEhAQIDNmzFA9Sr2xZ46BPSM9sGekFXtGWrFnpAf2jLRiz0grHXumbQN61KhR0rZtW8nKytJyGNLBhQsXpE2bNjJ27FjVo9TL0qVLxWQyybp161SPQiIyadIkCQwMdKpP787IyJCAgACZOnWq6lFIRFavXi0mk0mWLVumepR6Yc8cB3tGemDPSCv2jLRiz0gP7BlpxZ6RVjr17No3oD///HNxc3OTrVu3ahmAdPT999+LyWSSVatWqR7FJhcvXhR/f3954oknVI9C/6+goEA6dOggd999t+pRbDZu3DgJDw932l8raogeffRRCQwMdJpX/LBnjoc9I63YM9IDe0ZasWekFXtGemDPSCsdenZtG9D5+fnSpk0beeCBB671xGSQ++67T9q2bSsFBQWqR6nT9OnTJSgoSPLy8lSPQr+zYcMGASDfffed6lHq9M033wgAWb9+vepR6Hfy8vIkODhYHn74YdWj1Ik9c1zsGWnFnpFW7BnpgT0jrdgz0oo9Iz1o7Nm1bUA/9dRT4u/vLxcuXLiWf5wMdP78eWnevLk899xzqkepVUJCgphMJlm+fLnqUagaY8aMkfDwcLl69arqUWp09epV6dy5s9x1112qR6FqxMfHi5ubm2zfvl31KLVizxwXe0Z6YM9IK/aMtGLPSA/sGWnFnpFWGntW/w3oQ4cOidlslkWLFl3LCckO3nvvPfHw8JCkpCTVo1TLYrFIr1695LbbblM9CtUgPT1dfH195ZVXXlE9So1mz54tTZo0kczMTNWjUA2GDBkiffr0EYvFonqUarFnjo89I63YM9IDe0ZasWekFXtGemDPSCsNPav/BvTAgQPlxhtvlLKysvr+o2QnZWVl0rdvXxk8eLDqUao1b9488fLykuTkZNWjUC1eeeUV8fHxkfT0dNWjVHHy5Enx9vaW119/XfUoVIujR4+Kp6en/Oc//1E9SrXYM8fHnpEe2DPSij0jrdgz0gN7RlqxZ6SVhp7VbwN6/fr1YjKZZPfu3fU9EdlZYmKiAJBvv/1W9SiVXL58WVq2bOnwv4JGIsXFxdKhQwd58MEHVY9SxbRp06RLly5SWlqqehSqw1NPPSVBQUFSVFSkepRK2DPnwZ6RVuwZ6YE9I63YM9KKPSM9sGek1TX2zGISEYGNbrrpJvj5+eHrr7+29R8hhWJiYmCxWPD999+rHqXC/Pnz8cILLyA1NRUtW7ZUPQ7VYeHChZg1axZOnjyJNm3aqB4HAJCRkYFOnTph4cKFmDJliupxqA7nzp1Dhw4d8MYbb+CRRx5RPU4F9sy5sGekFXtGWrFnpAf2jLRiz0gr9oz0cA09K7P5FdA//PCDAJBt27bVd3OcFHG0n1lJSYm0a9dOnnzySdWjkI0c8Wf26KOPSrt27aS4uFj1KGQjR/uZOdq1kermaD8zR7w2Uu0c8WfmaNdGqpuj/cwc7dpIdXO0n5kjXhupdo74M3O0ayPVzdF+Zo52baS6XcPPzPZXQMfExKC0tBRbtmyp/9Y4KTNo0CA0adLEIZ5FWrRoER577DGcOHEC7dq1Uz0O2ciRXhVx/vx5hIWFOdyztVS78ldFfPDBB5g6darqcdgzJ8WekVbsGWnFnpEe2DPSij0jrdgz0kM9e2bbK6D37t0rJpPJ4d6viuq2YcMGAaD8fXQsFouEh4fLQw89pHQOqr+ioiIJCgqS559/XvUo8uc//1lat24tV65cUT0K1dO0adOkU6dOyj9xmT1zXuwZacWekR7YM9KKPSOt2DPSA3tGWtWzZ7a9Avree+9FcnIydu3apW17nJTo06cPevTogSVLliib4csvv8S4ceOQkpKCDh06KJuDrs2rr76K1157DWfPnoW3t7eSGYqKitC6dWu89NJLePbZZ5XMQNcuOTkZXbt2xdq1axEbG6tsDvbMubFnpBV7RlqxZ6QH9oy0Ys9IK/aM9FCPnpW51fUd+fn5WL16Ne6//359piO7mz59OlatWoXCwkJlMyxduhRDhgzhgxsnNX36dBQUFGDdunXKZli9ejWuXLmC++67T9kMdO06d+6MQYMGIT4+XtkM7JnzY89IK/aMtGLPSA/sGWnFnpFW7BnpoT49q3MDeuXKlbBYLLjzzjt1GY7sb+LEibBYLFi9erWS8+fk5GD9+vWYPHmykvOTdq1atcKtt96KuLg4ZTPExcVhxIgRaNWqlbIZSJvJkydjzZo1uHTpkpLzs2fOjz0jrdgz0gN7RlqxZ6QVe0Z6YM9Iq/r0rM4N6Li4OIwZMwYBAQG6DEf25+/vj+HDhyuL07Jly+Dh4YGxY8cqOT/pY/LkyVi/fj3Onz9v93OfP38e3333HR8kO7nx48fDZDJh5cqVSs7Pnjk/9oz0wJ6RVuwZacWekR7YM9KKPSOt6tOzWjeg09PTsXXrVl5UGoDJkydj8+bNyMzMtPu54+LiMHbsWDRu3Nju5yb93HHHHWjUqBGWL19u93PHxcXB19dX6XtTkXZ+fn4YPXq0kv/ZYs8aDvaMtGLPSCv2jPTAnpFW7BlpxZ6RHmztWa0b0J9++ikCAgIwfPhwXYcj+4uNjUWzZs2wbNkyu543OTkZO3fu5EWlAfDx8cG4ceOUxCk+Ph533XUXGjVqZPdzk74mT56Mbdu2ITU11a7nZc8aDvaMtGLPSA/sGWnFnpFW7BnpgT0jrWztWa0b0GvWrMEdd9wBs9ms63Bkf15eXhgzZgzWrFlj1/OuXbsWLVq0wJAhQ+x6XjLGXXfdhT179uDs2bN2O2d6ejoOHjyICRMm2O2cZJyYmBg0bdoUX331lV3Py541HOwZ6YE9I63YM9KKPSM9sGekFXtGWtnasxo3oLOysrBnzx6MGDFC9+FIjREjRmDHjh3Iycmx2zk3bNiAYcOGwd3d3W7nJOMMHjwYjRo1wjfffGO3c27YsAG+vr4YOHCg3c5JxjGbzRg6dCg2btxot3OyZw0Pe0ZasWekFXtGemDPSCv2jLRiz0gPtvSsxg3oTZs2wcPDA7feeqshw5H93XbbbTCZTNi8ebNdzldYWIiffvqJv1LRgDRq1Ag333wzNmzYYLdzbty4Ebfeeiu8vLzsdk4y1ogRI/DDDz/g6tWrdjkfe9bwsGekFXtGemDPSCv2jLRiz0gP7BlpZUvPatyA3rp1K6KiotCkSRNDhiP7a9asGXr37o2tW7fa5Xw7duxASUkJbrnlFrucj+zjlltusdsaEhFs27aNYWpghgwZgqKiIuzevdsu52PPGh72jPTAnpFW7BlpxZ6RHtgz0oo9I61s6VmNG9AJCQkYMGCAIYOROgMGDEBCQoJdzpWYmIiwsDC0adPGLucj+xg4cCDOnz+PkydPGn6un3/+GTk5OYiOjjb8XGQ/oaGhCAkJsdu1iD1rmNgz0oo9I63YM9IDe0ZasWekFXtGeqirZ9VuQOfn5+Po0aPo37+/YYORGtHR0UhKSkJhYaHh59qxYwfXUAPUu3dveHt72yVO27dvh4+PDyIjIw0/F9lXdHQ0EhMTDT8Pe9ZwsWekFXtGemDPSCv2jLRiz0gP7BlpVVfPqt2APnDgAKxWK/r27WvocGR/ffr0QVlZGQ4dOmT4ufbt28c11AB5eXmhe/fuOHDggOHn2r9/PyIjI/nJuA1Q7969sX//fsPPw541XOwZacWekR7YM9KKPSOt2DPSA3tGWtXVs2o3oJOSkuDv74+QkBBDhyP7Cw0NhZ+fn+EPcC5cuIDz58+jZ8+ehp6H1IiIiLDLg+SDBw/y2fUGKiIiApmZmYZ/6jt71nCxZ6QH9oy0Ys9IK/aM9MCekVbsGWlVV8+q3YA+dOgQevToYehgpIbJZEL37t0Nj1NSUhKAXy9i1PBERETg8OHDhp/n8OHDvBY1UOXXhiNHjhh6Hvas4WLPSA/sGWnFnpFW7BnpgT0jrdgz0qqunlW7AX3y5El07tzZ0MEcRVJSEubPn4/c3FzVo9hNp06dDP+AghMnTqBp06Zo2bKloedxBK64hsLDw3Hu3DlD36suOzsbubm5vBY1UCEhIfDx8TH8WsSeNWzsmb5ccQ2xZ/pztXXEnunP1dYQwJ7pzRXXEHumP1dbR+yZ/lxtDQG196zaDei0tDSEhoYaOZPD+Omnn/Dkk0/i3Llzqkexm7CwMKSmphp6jvT0dISFhRl6DkfhqmsI+PVaYZTyY/Na1DCZTCa0a9fO8GsRe9awsWf6ctU1BLBnenK1dcSe6c/V1hDAnunNVdcQwJ7pydXWEXumP1dbQ0DtPauyAW21WpGRkeEyC8IVhYWFIT09HSJi2DnS0tLQvn17w45PapU/wDEyTunp6XBzc0O7du0MOwep1aFDB0PXEHvW8LFnpBV7Rnpgz0gr9oy0Ys9ID+wZaVVbz6psQOfk5KCkpARBQUF2Gc5ZiYihDxCMFBQUhKKiIuTn5xt2jvPnzyM4ONiw4zcEzryGfH194efnh/Pnzxt2jnPnzsHf3x9eXl6GncPZOfMaAn69Fl24cMGw47NntnHmdcSeOQZnXkPsmWNw5jUEsGeOwpnXEXvmGJx5DbFnjsGZ1xDAnjkKZ15HtfWsygZ0dnY2AKBFixaGDzZr1izcf//9yMzMxGOPPYbAwMCKr126dAmPPvooevTogdatW2Ps2LFYv359tcf54Ycf8Nhjj6Fz585o27YtJk6ciAULFqCsrKzS9+3evRvjx49Hhw4dMHToULz77rv1/qEePHgQQ4cORbNmzeDj44N+/fphw4YNFV+/evUqZs+ejY4dO8LLywvh4eF46KGHUFBQAACYOXMmbrrpJvzyyy9Vjv3ggw8iJiYGJSUl9Zqpvsp/tuU/ayNkZWUhICDAsOOXq20NAbavI2daQ4BjrKOAgABD11B2drZd1hCgz7XI1jUEaF9Hda0hwDmuRQEBAcjKyjLs+OxZ7dgz27BnNWPPbMOe1Yw9sw17Vjv2zDbsWc3YM9uwZzVjz2zDntWuwfdM/iAxMVEASEZGxh+/pLubb75ZunTpIj179hQA0rt3bxERycjIkNDQUPH19ZVHHnlE/vrXv8r1118vbm5u8tZbb1U6xvfffy/u7u7i7+8vjz/+uPzjH/+QAQMGCAB59tlnK75vy5Yt4uPjI/7+/vLAAw/Igw8+KM2aNZPQ0FABIEePHq1z3i1btoi3t7e0adNGnnrqKZk+fbo0bdpUPDw8JCEhQUREpk2bJu7u7jJ16lSZP3++zJo1Sxo1aiT9+/cXEZF58+YJAHn33XcrHfvMmTPi5uYmEyZM0PRnaovU1FQBILt27TLsHG3atJE333zTsOOXq2kNidi+jpxtDYk4xjqKioqq9OejtyeffFKio6MNO/7vab0W2bqGRLSvI1vWkIhzXIvmzp0roaGhhh2fPasZe2Y79qx67Jnt2LPqsWe2Y89qxp7Zjj2rHntmO/aseuyZ7dizmrlAzyxVNqA3bdokACQ3N9fwwW6++WYBIMOGDZNjx45V3D9p0iQBIDt27Ki4r7i4WIYMGSKenp6SnZ1dcf+MGTPEy8ur0rxFRUUSFBQkXbt2rbgvMjJSmjdvLqmpqRX3JScni4+Pj00LoqysTCIjI6Vp06aSkpJScf+xY8fEZDLJpEmT5OrVq2I2m2X06NGV/tn58+cLADl+/LhcuHBBPDw8ZPDgwZW+56233hIAsnHjxjr+1LS7ePGiAJAtW7YYdo5mzZrJwoULDTt+uZrWkIjt68jZ1pCIOMQ6Gjx4sDz66KOGHX/GjBkSExNj2PF/T+u1yNY1JKJtHdmyhkTEaa5F7733ngQGBhp2fPaseuxZ/bBnVbFn9cOeVcWe1Q97Vj32rH7Ys6rYs/phz6piz+qHPauei/Ss6gb0unXrBIBcvnzZ8MHKF8Tvd8azs7PFZDJJVFRUle9ftmyZAJBFixZV3Hfs2DFJSkqq9H15eXnSrVs3CQ4OFhGR7du3CwB58cUXqxzzoYceqlgQZWVlUlhYWOlmsVhERGTPnj0CQKZMmVLlGO+8847Mnz9fLl++LGazWfz8/GTfvn0VXy8/bvmxRo0aJe7u7nL+/PmK7+nXr5+0adNGysrKbPmj0yQvL8/wxefj4yOLFy827PjlqltDIvVbR3quIRGpcR3puYZE1K+jYcOGyf3332/Y8adMmSKxsbGGHf/3tF6LbFlDItqvRbasIRFxmmvRokWLxM/Pz7Djs2fsmR7YM/ZMK/aMPdOKPWPP9MCesWdasWfsmVbsmUv3zOKBPyh/PxBPT88/fskQgYGBiIqKqvj748ePQ0RQWFiICRMmVPre8jexPnnyZMV9Xbt2RXZ2Nt544w1s374daWlpSElJQX5+fsWHLPz8888AgF69elU5f/fu3Sv+eufOnYiOjq709c8++wwTJ07EiRMnAAARERFVjvH4449X/PXs2bPx0ksvoXfv3ujWrRtuueUWjBw5EsOGDYO7uzsAYMqUKfjqq6+wevVqPPTQQ0hLS8POnTvx/PPPw82tytty6678QwOKi4sNO0dxcbGyNQTUbx3puYaAmtdR+c9WjzUEqF9Hnp6eDWYNAdquRbasIUD7tcjWNeTj4+M01yIj1xB7xp7pgT1jz7Riz9gzrdgz9kwP7Bl7phV7xp5pxZ65ds+MP3sd/vgJquVvVO3l5QWz2VzpFhAQgEmTJlX6Ib722msICQnByy+/jNLSUgwdOhSLFy/GgAEDKr4nJycHACrFoZy3t3fFX7do0QKTJk2qdAsNDQUAXLx4EQDQpk2bWv99XnzxRZw4cQJ/+9vf4OPjgwULFiA2Nhbdu3fHuXPnAACxsbFo3rw5Vq5cCQD44osvAABTp06t889LD/L/b4RuMpkMO4eRx/6j6j6Ftz7rSM81BNS8jvRcQ4D6dQQYv4bK16o9aLkW2bKGAO3XIlvXEOA81yJ7XiuMxp6xZ1qxZ+yZHtgz9kwr9ow904o9Y8/0wJ6xZ1qxZw7Wsz++JtreL4kPCQmpdN+RI0cEgIwfP77K91ssFsnNzZXi4mIR+fV9ltzc3KRVq1aSn59f6Xv79OlT8ZL4//73vwJAnnjiiSrHfOGFF2x6T5avvvpKAMiTTz5Z5WtLliyRTz75RIqLiyU3N7fSr+H88ssv8vjjjwsAeeGFFyruf/jhh8XDw0OysrKkV69ednszf5HfXhL/zTffGHYOe/6K1x/XkIjt68iZ15CI2nUUExMjDzzwgGHHt/eveF3rtcjWNSSifR3ZsoZExGmuRYsWLZKmTZsadnz2rHrsWf2wZ1WxZ/XDnlXFntUPe1Y99qx+2LOq2LP6Yc+qYs/qhz2rnov0zFLlFdDlzxCUvzTe3jp16oTAwEB88803KC0trfS1V199Fc2bN8euXbsAAOnp6bBarRg7diyaNGlS8X0ZGRk4cOBAxd/37dsXZrMZ33//faXjWSwWfPbZZzbNFRUVhUaNGlU5xtGjRzF16lT8+OOP+P7779G8eXMsW7asOtjiYgAAIABJREFU4uutW7fGs88+CwDIzc2tuH/KlCmwWCyYO3cuDhw4gGnTptk0hx7KXwpv5K89eHp6KltDgO3ryJnXEKB2HZWUlFT76ga9NLQ1BGhfR7asIQBOdS0ycg2xZ9Vjz+qnoV2LHHENAeyZkdgz47Fn7JlW7Bl7pgf2jD3Tij1jz7SqtWd/3JJOTEwUAJKRkWH4znhNz45++OGHAkDuuusu2bt3r6SkpMjrr78uXl5ectttt4nVahURkfz8fGncuLH4+/vL2rVrJTk5WT755BMJCQmR5s2bi5+fn/z8888iIvLkk08KAJk+fbrs3btX9u3bJ2PGjJHmzZvb9IyEiMjzzz8vAOShhx6S3bt3y5IlS6Rnz55iNptlx44dkp+fLy1btpTw8HDZsmWLXLp0Sfbs2SN33HGHAJB169ZVOl7nzp3F3d1dfHx8JC8vT4c/UdukpqZW+8EQegoJCZE333zTsOOXq2kNidi2jpx9DYmoW0d9+/aVv/zlL4Yd/6mnnpL+/fsbdvzf03Itqs8aEtG+jupaQyLiNNeiuXPnSlhYmGHHZ89qxp7Zjj2rHntmO/aseuyZ7dizmrFntmPPqsee2Y49qx57Zjv2rGYu0DNLlQ3o5ORkASD79+83fLDa4vT222+Lt7e3ABAA4uHhIQ8//LBkZ2dX+r7ly5dL48aNK77P399flixZIitXrhRfX1/x8PAQEZGrV6/KjBkzKr4PgNx6660SFxdn84KwWCzy17/+Vdzd3SuOERQUJMuXL6/4nm+//VaCg4Mrncfb21v+9a9/VTneyy+/LADk3nvvrc8fm2a7du0SAHLq1CnDzhEZGVntp4DqrbY1JGLbOnLmNSSibh2FhobK3LlzDTv+//7v/0rnzp0NO/7vab0W2bqGRLSvI1vWkIhzXIueffbZaj+BWC/sWc3YM9uxZ9Vjz2zHnlWPPbMde1Yz9sx27Fn12DPbsWfVY89sx57VzAV6ZjGJVH4X+ZycHAQEBODbb7/F0KFDoVJBQQH279+PwsJCREREoG3bttV+X3Z2Nvbv34+goCBcd911FW92nZ2djdzcXHTq1KniezMyMnDo0CF069YNYWFh1zTX5cuXkZSUBD8/P4SHh1d5afmVK1eQlJSE06dPo0WLFujRowdatmx5TecywoYNGzBy5Ejk5+dX+lUCPd16660IDw/HggULDDl+fdiyjriG6q9JkyaYP38+pk+fbsjxFyxYgBdffLHigwJU0nsNAdrXUV1rCHD8dTR9+nT88ssv2LBhgyHHZ8/q5uzXIvaMPdMDe1YZe1Z/7FlVvBbVD3vGnumBPauMPas/9qwqXovqp5aelVXZgLZarWjUqBE++ugj3HvvvfadlOxi8eLFePTRR3H58mXDPuH0nnvuweXLl7FmzRpDjk9qFRYWokmTJvjqq68QGxtryDlWr16NcePG4cqVK1U+yZoahpEjRyIwMBBLliwx5PjsWcPHnpFW7BnpgT0jrdgz0oo9Iz2wZ6RVLT0rq/IhhG5ubmjbti3S0tLsNiDZV2pqKkJDQw17cAMA7du3R3p6umHHJ7XKrw/X+qyeLdq3bw8RwenTpw07B6mVmppq6Bpizxo+9oy0Ys9ID+wZacWekVbsGemBPSOtautZlQ1o4NeLFuPUcBl9UQGA0NBQpKamGnoOUqf8Z9u+fXvDzhEaGgoAjFMDJSJIT083/FrEnjVs7BlpxZ6RVuwZ6YE9I63YM9KKPSM91NazajegO3bsiOPHjxs6FKmTkpKCjh07GnqOTp06IT8/H+fPnzf0PKRGcnIyWrdujcaNGxt2Dn9/f/j7+yM5Odmwc5A6GRkZKCoqMvxaxJ41bOwZacWekVbsGemBPSOt2DPSij0jPdTWs2o3oHv27IlDhw7hD28PTQ2AiODIkSOIiIgw9Dw9e/YEACQlJRl6HlLj0KFDFT9jI0VERODQoUOGn4fs79ChQzCZTOjRo4eh52HPGi72jPTAnpFW7BlpxZ6RHtgz0oo9I63q6lmNG9CXLl1CRkaGocOR/aWmpqKgoMDwOAUGBiIoKIgPcBqoQ4cOGf4gGQAiIyO5hhqoQ4cOoW3btmjWrJmh52HPGi72jPTAnpFW7BlpxZ6RHtgz0oo9I63q6lmNG9Du7u7Yu3evocOR/e3duxceHh7o3r274ee6/vrruYYaoOLiYhw5cgSRkZGGn6tXr15ISkpCaWmp4eci+9q3bx969epl+HnYs4aLPSOt2DPSA3tGWrFnpBV7Rnpgz0irunpW7Qa0n58funfvjsTEREOHI/v76aefEBkZaeh7Q5Xr378/11ADtHv3bhQXF2PgwIGGnys6OhpXrlzB/v37DT8X2ddPP/2EAQMGGH4e9qzhYs9IK/aM9MCekVbsGWnFnpEe2DPSqq6eVbsBDQADBgxAQkKCYYORGgkJCXa5qAC/rqH09HRkZmb+X3t3HhT1ff9x/P1dYEEOQQ4RUXBFidV4ETUeVBrxwNaj3kkUHWNq6sTQjjWmNZPaWCeTpk2lWhszjXE8qvGq8ahnREUkajQ6mGi9wGNUFBDlUO7X749ETX4lcny+7Gd3eT1m/C/yeUmUJ/NZdtcu55F9HD58WFq0aNHg744rIhIdHS0hISGMk4vJzMyUmzdv2vVrEXvmetgzUsWekSr2jMzAnpEq9oxUsWdkhpp69oMX0P3795fjx49LQUFBgwwj+8vPz5dTp05J//797XLes88+K56enpKSkmKX88g+9u/fL3FxcXY5yzAMiY2N5d8hF5OSkiLe3t7yzDPP2OU89sz1sGdkBvaMVLFnpIo9IzOwZ6SKPSNVtenZD15ADx48WKqqqmTfvn0NMo7sb8+ePWIYhsTHx9vlPG9vb/nxj38su3btsst51PAePHggqampkpCQYLczExISJCUlRUpLS+12JjWsnTt3ynPPPSdeXl52OY89cz3sGaliz8gM7BmpYs9IFXtGZmDPSFVtevaDF9CBgYHSs2dP2bFjR4OMI/vbuXOn9OnTp8Hf1fS7hg4dKnv27JHKykq7nUkNJyUlRUpKSmTIkCF2OzMhIUGKi4slNTXVbmdSwykvL5d9+/bZ9Ztk9sz1sGekij0jVewZmYE9I1XsGaliz8gMtenZD15Ai4gMHz5ctm7dyji5gJKSEtmyZYuMHDnSrucOGzZM8vLy5ODBg3Y9lxrGunXrpFevXhIWFma3MyMiIqRr167y73//225nUsPZtWuXFBQUyPDhw+16LnvmOtgzMgN7RqrYM1LFnpEZ2DNSxZ6Rqtr27IkX0BMmTJDbt2/LgQMHzNxGGmzbtk0KCgrk+eeft+u50dHR0q1bN9mwYYNdzyXzFRcXy+bNmyUxMdHuZ48bN042bdokFRUVdj+bzLVq1SqJi4uTyMhIu57LnrkO9oxUsWdkBvaMVLFnpIo9IzOwZ6Sqtj174gV0VFSUdO/eXdatW2fqOLK/1atXy8CBAyU8PNzuZ48fP142bdok5eXldj+bzLN582YpKyuTCRMm2P3s8ePHS05ODuPk5O7evSvbtm2TSZMm2f1s9sx1sGekij0jVewZmYE9I1XsGaliz8gMte3ZEy+gRUQSExNl3bp1UlxcbNo4sq/c3FzZuXOnTJ48Wcv5L774ouTl5cm2bdu0nE/mWLlypQwdOlSCg4Ptfnb79u2ld+/e8vHHH9v9bDLPhg0bxDAMGTt2rJbz2TPnx56RGdgzUsWekSr2jMzAnpEq9oxU1aVnNV5AT548WcrKyviohBNbu3atWK1Wu7++2EORkZEyePBg+eijj7ScT+pu3LghKSkpWp7e9dDLL78smzZtkpycHG0bSM2qVatk5MiR4u/vr+V89sz5sWekij0jM7BnpIo9I1XsGZmBPSNVdelZjRfQQUFBMmrUKPnHP/5hyjiyLwDy8ccfy9ixY8XX11fbjunTp8vu3bvl4sWL2jZQ/S1fvlz8/f1l2LBh2jY8//zz4uXlxUfZndS5c+ckLS1N6zfJ7JlzY8/IDOwZqWLPSBV7RmZgz0gVe0aq6twz1MKXX34JwzCwe/fu2vzn5EC2b98OEcHx48e17qioqEB0dDRefvllrTuo7oqLi9G8eXO8+eabuqdgzpw5CA0Nxf3793VPoTqaMmUKOnTogMrKSq072DPnxZ6RKvaMzMCekSr2jFSxZ2QG9oxU1bFnFQYA1OZmOyEhQR48eCAHDx5UuSAnO4uNjZVmzZo5xOt7LVu2TGbMmCEXLlyw+zusUv0tXLhQ3nrrLcnKypKQkBCtW27duiU2m03+/Oc/y6uvvqp1C9Xe1atXpV27dvLRRx9pe63D72LPnBN7RqrYM1LFnpEZ2DNSxZ6RKvaMzFDHnlXW6iegASA9PR0igtTU1PpejpOd7du3DyKCtLQ03VMAAGVlZYiMjERSUpLuKVRLJSUlCA8Px6xZs3RPeWTmzJlo3bo1SktLdU+hWvrlL38Jm82G8vJy3VMAsGfOiD0jVewZmYE9I1XsGaliz8gM7BmpqkfPav8T0CIi/fv3Fx8fH9m5c2fdr8bJ7uLj48UwDPnss890T3lk8eLFMmfOHMnMzJSwsDDdc6gGS5culV//+tdy6dIlCQ8P1z1HRESuXbsm7dq1k6VLl8rUqVN1z6EaZGdni81mk+TkZHnllVd0z3mEPXMu7BmpYs9IFXtGZmDPSBV7RqrYMzJDPXpW+5+ABoDdu3dDRHD06NG6Xo6TnaWlpUFEsG/fPt1Tvuf+/fsIDQ3F7NmzdU+hGpSUlMBms2HGjBm6p/yPadOmoX379igrK9M9hWqQlJSE8PBwlJSU6J7yPeyZ82DPSBV7RmZgz0gVe0aq2DMyA3tGqurZs4o6XUADQFxcHHr16qX9hcrph1VUVCAmJgbx8fG6p1Rr8eLFsFqt+O9//6t7Cj3BH//4R/j4+ODq1au6p/yPzMxMNGnSBH/60590T6EnOH36NDw8PPDhhx/qnlIt9szxsWdkBvaMVLFnpIo9IzOwZ6SKPSNVCj2r+wX0V199BQ8PDyxdurSuv5XsZNGiRbBarThz5ozuKdWqqKhA9+7dERcXh6qqKt1zqBpXrlyBj4+PQ38DMX/+fHh7eyMrK0v3FKpGVVUVnnvuOfTo0cNhv4Fgzxwfe0aq2DNSxZ6RGdgzUsWekSr2jMyg0LO6X0ADwOzZs9GsWTPcunWrPr+dGlB2djYCAgIwd+5c3VOe6NixY7BYLFi7dq3uKVSN4cOHo2PHjg79FKrS0lI89dRTGDNmjO4pVI0VK1bAYrE4/FOo2DPHxZ6RGdgzUsWekSr2jMzAnpEq9oxUKfasfhfQxcXFiIyMxNSpU+vz26kBTZw4ERERESgqKtI9pUa/+MUv0KJFC9y9e1f3FPqOTz/91CFfn646e/bsgYhg+/btuqfQd9y7dw9hYWF49dVXdU+pEXvmuNgzUsWekSr2jMzAnpEq9oxUsWdkBsWe1e8CGgA2bNgAwzCwf//++n4IMtnevXthGAY+/fRT3VNqJTc3F8HBwZg5c6buKfSte/fuoU2bNpg4caLuKbU2fvx4REVFobCwUPcU+tYrr7yC0NBQ5Ofn655SK+yZ42HPSBV7RmZgz0gVe0aq2DMyA3tGqkzoWf0voAFg1KhRCA8Px+3bt1U+DJkgOzsbYWFhGD9+vO4pdfKvf/3Lqb4pc3UTJkxAaGgosrOzdU+ptevXryMkJASTJk3SPYUAbNy4EYZhYP369bqn1Al75jjYMzIDe0aq2DNSxZ6RGdgzUsWekSqTeqZ2AZ2fnw+bzYahQ4c67IuYNwaVlZUYNGgQoqKinPLpUtOmTUOzZs2QmZmpe0qj9sEHH8BisWDPnj26p9TZzp07YbFYsGzZMt1TGrXLly8jMDAQM2bM0D2lztgzx8CekRnYM1LFnpEq9ozMwJ6RKvaMVJnYM7ULaOCbNyuwWq149913VT8U1dPbb78NT09PHD9+XPeUennw4AG6du2KXr16obS0VPecRikjIwNNmjTB73//e91T6u2NN96Al5cXTp06pXtKo1RWVobevXujc+fOuH//vu459cKe6ceekSr2jFSxZ2QG9oxUsWekij0jM5jYM/ULaAD4y1/+And3dxw6dMiMD0d1cODAAbi5uWHx4sW6pyg5d+4c/Pz8MHv2bN1TGp3CwkJ06NABcXFxqKio0D2n3srLyxEbG4vo6GgUFBTontPovPbaa/D19cXZs2d1T1HCnunDnpEq9ozMwJ6RKvaMVLFnZAb2jFSZ3DNzLqCrqqowfPhwtGzZEllZWWZ8SKqFixcvIjQ0FGPGjNE9xRQrVqyAYRhYtWqV7imNRnl5OUaMGIHmzZvj+vXruucou3r1KoKCgjB69Gin/mbN2SxfvhyGYWDt2rW6pyhjz/Rgz0gVe0ZmYM9IFXtGqtgzMgN7RqoaoGfmXEAD37w7a/fu3dGuXTuneoF8Z3X79m1ER0ejR48eLvVo4uuvvw4PDw/s2LFD9xSXV1VVhWnTpqFJkyZIS0vTPcc0n3/+Oby9vTF9+nTdUxqF7du3w93dHW+99ZbuKaZhz+yLPSNV7BmZgT0jVewZqWLPyAzsGalqoJ6ZdwENuG50HU1BQQFiYmIQFRXlcv/4qqqq8NJLL8Hb29ulouuI5s6dCzc3N2zevFn3FNNt3boV7u7uTv2aac7gyJEj8PHxweTJk1FVVaV7jqnYM/tgz8gM7BmpYs9IFXtGZmDPSBV7RqoasGfmXkAD3/yYdosWLTBgwACUlJSY/eEbvdLSUgwePBjNmzfH+fPndc9pEBUVFRg1ahSCgoJw5swZ3XNc0pIlS2AYhku/K/HKlSthGAaSk5N1T3FJX3/9NQIDAzF8+HCUl5frntMg2LOGxZ6RGdgzUsWekSr2jMzAnpEq9oxUNXDPzL+ABoATJ07Az88PY8eO5bvmmqikpASjR4+Gv78/Tp48qXtOgyouLkbfvn0RERGBCxcu6J7jUpYvXw6LxdIo3kl2wYIFsFgsWLlype4pLuXcuXNo1aoVYmNjnfYdlWuLPWsY7BmZgT0jVewZqWLPyAzsGaliz0iVHXrWMBfQwDfvlujn54dBgwahsLCwoY5pNO7du4f4+Hj4+/sjNTVV9xy7yMvLQ8+ePREaGooTJ07onuMS3nvvPRiGgTfffFP3FLt54403YBgG3n//fd1TXMKxY8cQEhKCZ599Fnfu3NE9xy7YM3OxZ+yZGdgzUsWesWeq2DP2zAzsGaliz9gzVXbqWcNdQANARkYGWrZsiR49euDWrVsNeZRLy87ORkxMDFq0aIEvv/xS9xy7KioqQkJCAnx9fbFr1y7dc5xWVVUV5syZA8Mw8N577+meY3d/+9vfYLFYkJSUhMrKSt1znNa+ffvQtGlTxMfHN7rX3WLPzMGesWeq2DP2zAzsGXumij1jz1SxZ+yZGdgz9kyVHXvWsBfQAJCZmYn27dujbdu2fKpOPTz8/NlsNpd9TbGalJaW4oUXXoDVasUnn3yie47TKS8vx0svvQSr1Yo1a9bonqPN6tWr4eHhgcTERJSVleme43Q2btwIT09PTJw4sdF+/tgzNewZe6aKPfsGe6aGPWPPVLFn7Jkq9uwb7Jka9ow9U2XnnjX8BTQA3Lx5E926dUNYWFijeXqSGfbv34/Q0FA888wzjf4RncrKSiQlJcFiseCdd95xuXd0bSi3bt1CfHw8fH19sWfPHt1ztNuxYwd8fHwwZMgQ5OTk6J7jFCorKzF//nxYLBbMmjWr0f/bY8/qhz17jD2rH/bs+9izumPPvo89qx/27DH2rH7Ys+9jz+qOPfs+9qx+NPTMPhfQAHD37l2MGDEC7u7ueOedd/g0iyd4+AXFzc0No0ePbnRPpXiShQsXwsPDAwkJCbh9+7buOQ5t//79CAsLg81m42u0fcexY8cQGRmJVq1aMVA1yM7OxsCBA+Hp6YnFixfrnuMw2LPaY89+GHtWe+xZ9diz2mPPqsee1R579sPYs9pjz6rHntUee1Y99qz2NPbMfhfQwDevc5ScnAyr1YoBAwbg5s2b9jzeKdy+fRsJCQnw8PDAu+++2+gfzarOF198gbZt2yI0NBR79+7VPcfhPPx35uHhgZEjRzaaNyKoi7t372LcuHFwc3PDvHnzGKhqHDhwAC1btkRkZCQ+//xz3XMcDntWM/asZuzZk7FnNWPPasaePRl7VjP2rGbs2ZOxZzVjz2rGnj0Ze1YzzT2z7wX0Q0ePHkWbNm0QHh6OlJQUHRMc0t69exEWFoaoqCgcP35c9xyHdufOHfz85z+Hu7s7FixYgPLyct2THML169cfPSK6aNEi3XMcWlVVFRYuXAir1YohQ4YwUN8qKyvDH/7wB7i5uWHMmDG4e/eu7kkOjT2rHntWe+xZ9diz2mPPqsee1Q17Vj32rPbYs+qxZ7XHnlWPPasb9qx6DtAzPRfQAJCfn4/Ro0fDMAxMmTKlUb+G1s2bNzFp0iQYhoFx48bxC0otPXyEy9PTE127dsXhw4d1T9KmvLwcycnJaNq0Kdq1a8dvkOvg6NGjaNu2Lfz9/bF48WJUVFTonqRNamoqOnfuDC8vLz6lqw7Ys8fYs/phzx5jz+qPPXuMPasf9uwx9qx+2LPH2LP6Y88eY8/qhz17zIF6pu8C+qGtW7ciMjISAQEBSE5OblRfXCorK7FixQoEBQUhPDwcK1as0D3JKV28eBFDhw6FYRhITExsdF9cvvjiC/Ts2RMeHh5ISkpCYWGh7klO5/79+5g3bx48PT3RrVs3pKen655kV3l5eY/eRGbAgAE4e/as7klOiT1jz1SxZ+yZKvaMPTMDe8aeqWLP2DNV7Bl7Zgb2zKF6pv8CGgCKioowZ84ceHh4oEePHo3i9WzS0tIQExMDq9WKuXPnori4WPckp7d+/XqEh4cjMDAQS5cudfmnfd28eRPTpk2DYRgYNGgQzp07p3uS0zt79iwGDBgAi8WC6dOnu/w3y2VlZViyZAmaNWuG1q1bY9OmTbonOT32jD0zA3vGnqliz9gzVewZe2YG9ow9U8WesWeq2DOH6ZljXEA/9PXXX+MnP/kJRASDBg3CgQMHdE8yXUpKCuLj4yEiiI+P5yNZJissLMRvfvMbeHh4wGaz4YMPPkBJSYnuWaa6evUqZs6ciSZNmqBVq1ZYt26d7kkuZ82aNWjZsiW8vb2RlJSEa9eu6Z5kqgcPHmDJkiWIjIyE1WrFnDlzUFRUpHuWS2HPSBV7RmZgz0gVe0aq2DMyA3tGqtgz7RzrAvqhQ4cOYdiwYRAR9OvXD1u3bnX6dxs+dOgQBgwY8L0/EzWcy5cvIykpCU2aNEHz5s0xb9483Lt3T/csJZmZmUhKSoKXlxciIiKQnJyM+/fv657lskpKSvDhhx+idevWsFqtSExMdPqfYigqKkJycjLCw8Mf/ZkuXryoe5ZLY89IFXtGqtgzMgN7RqrYM1LFnpEZ2DNtHPMC+qG0tDQMHToUIoKYmBj8/e9/R25uru5ZtZaTk4PFixejW7duMAwDP/vZzxrdaxfpdv36dcyaNQs+Pj4ICQnB66+/jtOnT+ueVWtlZWXYunUrxo4dCzc3N7Rv3x7Lli1DWVmZ7mmNRmlpKf75z38iKioK7u7uGDduHLZv3+5UTyHMyMjA7NmzERwcDF9fX8yePZvvKm1n7BmpYs9IFXtGZmDPSBV7RqrYMzIDe2Z3jn0B/dCJEyeQmJgIX19fWK1WjBw5Eps2bXLIp+48ePAAGzZswIgRI+Dh4QE/Pz9MmTIFJ0+e1D2tUcvJycG8efPQpk0biAi6d++Ov/71rw77Rf7YsWN47bXXEBISAovFgri4OKxdu7ZRvWi+oykvL8fq1avRv39/GIaB5s2b41e/+pXDvqP1jRs38P7776Nr164QEbRt2xZvv/22U0XVFbFnpIo9I1XsGZmBPSNV7BmpYs/IDOyZ3TjHBfRDhYWFWLlyJQYNGgSLxYLAwEBMnToV69evR35+vrZdeXl5+OSTTzBlyhQEBATAzc0NQ4YMwerVqx3lxb7pW1VVVTh48CCmTZsGf39/uLu7Y/DgwVi0aBEuXLigbVd5eTkOHDiA3/72t+jQoQNEBB06dMCCBQtw+fJlbbuoellZWZg/fz6io6MhIujYsSN+97vfITU1Vesj7+fOnUNycjIGDRoENzc3BAQEYPr06Th06JDTP63I1bBnpIo9IzOwZ6SKPSNV7BmZgT0jVexZg6swAECc0PXr12XNmjWyZcsWOXLkiBiGITExMdK3b1+JjY2VmJgYadOmjRiGYeq5ACQrK0tOnDghhw8flsOHD8vJkyfFMAzp06ePjBw5Ul544QVp2bKlqeeS+UpKSmTLli2yceNG2bt3r9y7d09sNpvExsZKv379pFevXtKpUyexWq2mn33nzh05deqUpKenS3p6uhw+fFgKCgqkffv28tOf/lRefPFF6dWrl+nnkvmOHDkia9eulf/85z9y6dIl8ff3l379+knfvn2lb9++0rVrVwkMDDT93NLSUjlz5owcOXJE0tPT5dChQ3LlyhVp1qyZDBw4UMaNGyfDhw8XLy8v088mc7FnpIo9IzOwZ6SKPSNV7BmZgT0jVexZg6h02gvo78rPz5fPPvtMUlNT5fDhw5KRkSGVlZXi5+cnnTp1kujoaLHZbGKz2SQ0NFSCg4MlJCREmjZtKiIivr6+IiJSVFQkIiIFBQWSk5Mjubm5kp2dLVlZWZKVlSXnz5+XM2fOSGFhobi7u0uXLl0kNjZW+vfvL/Hx8RIQEKDtc0BqKioqJD09XVJSUiQ9PV2OHDny6P9zdHS0/OhHPxKbzSZt2rSRiIgICQoKevTLzc1NrFar+Pj4SH5+voiIPHjwQPLy8iQvL09u3bolV65ckaysLMnMzJSvvvpKbty4ISIiERERj76hGjx4sLRr107np4EUXbhwQXb3OoJLAAACLklEQVTv3i3p6emSlpYm165dExGR8PBw6dSpk0RFRYnNZpOIiAgJDQ2VoKAgCQ4OFi8vLzEMQwICAqS4uFjKysqksrJS8vLyJDc3V/Ly8uTatWuPvhadPXtWLly4IBUVFeLn5yd9+vSRfv36yYABA6R3797i7u6u+TNB9cWekSr2jMzAnpEq9oxUsWdkBvaMVLFnpnGNC+j/r6ioSE6fPv3o16VLlyQrK0suX74sJSUldfpYXl5ej/4yRUVFydNPPy1dunSRp59++tFfJHI9lZWVcu7cOTl9+rRkZGTI+fPnH31huHPnTp0+lpubm4SHh0ubNm2kbdu20rFjR+nSpYt07tzZWR+5olq6ceOGZGRkSEZGhpw9e1YyMzMlKytLrl+/LlVVVXX6WEFBQY++yX7qqace/R2Kjo4WNze3BvoTkG7sGaliz8gM7BmpYs9IFXtGZmDPSBV7Vm+ueQH9JAUFBZKbmyu5ublSUFAgIiKFhYUiIuLn5yciIk2bNpXg4GAJDg5+9KgF0UMPHz2/c+eO5OXlSWVlpZSWlsr9+/clICBADMOQJk2aSFBQkAQGBj56FJ7ooYePnj/8VVJSIgDk7t274u3tLZ6enuLm5va9n+TgU7Xo/2PPSBV7RqrYMzIDe0aq2DNSxZ6RGdizJ2p8F9BEREREREREREREZBeVFt0LiIiIiIiIiIiIiMg18QKaiIiIiIiIiIiIiBoEL6CJiIiIiIiIiIiIqEH8H2I/nGuscsp2AAAAAElFTkSuQmCC\n",
+ "text/plain": [
+ "\n",
+ "\n",
+ "\n",
+ " \n",
+ "
\n",
+ "
"
+ ],
+ "text/plain": [
+ " Date DayOfWeek DepTime CRSDepTime ArrTime CRSArrTime \\\n",
+ "0 1990-01-01 1 1621.0 1540 1747.0 1701 \n",
+ "1 1990-01-02 2 1547.0 1540 1700.0 1701 \n",
+ "2 1990-01-03 3 1546.0 1540 1710.0 1701 \n",
+ "3 1990-01-04 4 1542.0 1540 1710.0 1701 \n",
+ "4 1990-01-05 5 1549.0 1540 1706.0 1701 \n",
+ "\n",
+ " UniqueCarrier FlightNum TailNum ActualElapsedTime ... AirTime ArrDelay \\\n",
+ "0 US 33 NaN 86.0 ... NaN 46.0 \n",
+ "1 US 33 NaN 73.0 ... NaN -1.0 \n",
+ "2 US 33 NaN 84.0 ... NaN 9.0 \n",
+ "3 US 33 NaN 88.0 ... NaN 9.0 \n",
+ "4 US 33 NaN 77.0 ... NaN 5.0 \n",
+ "\n",
+ " DepDelay Origin Dest Distance TaxiIn TaxiOut Cancelled Diverted \n",
+ "0 41.0 EWR PIT 319.0 NaN NaN False 0 \n",
+ "1 7.0 EWR PIT 319.0 NaN NaN False 0 \n",
+ "2 6.0 EWR PIT 319.0 NaN NaN False 0 \n",
+ "3 2.0 EWR PIT 319.0 NaN NaN False 0 \n",
+ "4 9.0 EWR PIT 319.0 NaN NaN False 0 \n",
+ "\n",
+ "[5 rows x 21 columns]"
+ ]
+ },
+ "execution_count": 9,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "ddf.head()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "95fb66a7",
+ "metadata": {},
+ "source": [
+ "`.head()` triggers a computation to show the first 10 rows of the DataFrame. As you might have noticed we can perform computations on Dask DataFrames using the single machine scheduler, but we recommend you use a distributed cluster so you can see the dashboard in action."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "id": "4e816c86",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "| \n", + " | Date | \n", + "DayOfWeek | \n", + "DepTime | \n", + "CRSDepTime | \n", + "ArrTime | \n", + "CRSArrTime | \n", + "UniqueCarrier | \n", + "FlightNum | \n", + "TailNum | \n", + "ActualElapsedTime | \n", + "... | \n", + "AirTime | \n", + "ArrDelay | \n", + "DepDelay | \n", + "Origin | \n", + "Dest | \n", + "Distance | \n", + "TaxiIn | \n", + "TaxiOut | \n", + "Cancelled | \n", + "Diverted | \n", + "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", + "1990-01-01 | \n", + "1 | \n", + "1621.0 | \n", + "1540 | \n", + "1747.0 | \n", + "1701 | \n", + "US | \n", + "33 | \n", + "NaN | \n", + "86.0 | \n", + "... | \n", + "NaN | \n", + "46.0 | \n", + "41.0 | \n", + "EWR | \n", + "PIT | \n", + "319.0 | \n", + "NaN | \n", + "NaN | \n", + "False | \n", + "0 | \n", + "
| 1 | \n", + "1990-01-02 | \n", + "2 | \n", + "1547.0 | \n", + "1540 | \n", + "1700.0 | \n", + "1701 | \n", + "US | \n", + "33 | \n", + "NaN | \n", + "73.0 | \n", + "... | \n", + "NaN | \n", + "-1.0 | \n", + "7.0 | \n", + "EWR | \n", + "PIT | \n", + "319.0 | \n", + "NaN | \n", + "NaN | \n", + "False | \n", + "0 | \n", + "
| 2 | \n", + "1990-01-03 | \n", + "3 | \n", + "1546.0 | \n", + "1540 | \n", + "1710.0 | \n", + "1701 | \n", + "US | \n", + "33 | \n", + "NaN | \n", + "84.0 | \n", + "... | \n", + "NaN | \n", + "9.0 | \n", + "6.0 | \n", + "EWR | \n", + "PIT | \n", + "319.0 | \n", + "NaN | \n", + "NaN | \n", + "False | \n", + "0 | \n", + "
| 3 | \n", + "1990-01-04 | \n", + "4 | \n", + "1542.0 | \n", + "1540 | \n", + "1710.0 | \n", + "1701 | \n", + "US | \n", + "33 | \n", + "NaN | \n", + "88.0 | \n", + "... | \n", + "NaN | \n", + "9.0 | \n", + "2.0 | \n", + "EWR | \n", + "PIT | \n", + "319.0 | \n", + "NaN | \n", + "NaN | \n", + "False | \n", + "0 | \n", + "
| 4 | \n", + "1990-01-05 | \n", + "5 | \n", + "1549.0 | \n", + "1540 | \n", + "1706.0 | \n", + "1701 | \n", + "US | \n", + "33 | \n", + "NaN | \n", + "77.0 | \n", + "... | \n", + "NaN | \n", + "5.0 | \n", + "9.0 | \n", + "EWR | \n", + "PIT | \n", + "319.0 | \n", + "NaN | \n", + "NaN | \n", + "False | \n", + "0 | \n", + "
5 rows × 21 columns
\n", + "\n",
+ " \n",
+ "
"
+ ],
+ "text/plain": [
+ "\n",
+ " \n",
+ "\n",
+ "
\n",
+ "\n",
+ " \n",
+ "
\n",
+ "Client
\n", + "Client-d7668b3c-1a47-11ec-81ee-1e00ea0a0276
\n", + "| Connection method: Cluster object | \n", + "Cluster type: distributed.LocalCluster | \n", + " \n", + "
| \n", + " Dashboard: http://127.0.0.1:8787/status\n", + " | \n", + "\n", + " |
\n",
+ "
\n",
+ "
\n",
+ " \n",
+ "\n",
+ " Cluster Info
\n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ "\n",
+ "
\n",
+ "LocalCluster
\n", + "87aa4218
\n", + "| \n", + " Dashboard: http://127.0.0.1:8787/status\n", + " | \n", + "\n", + " Workers: 4\n", + " | \n", + "
| \n", + " Total threads: 8\n", + " | \n", + "\n", + " Total memory: 16.00 GiB\n", + " | \n", + "
| Status: running | \n", + "Using processes: True | \n", + "
\n",
+ " \n",
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ " Scheduler Info
\n",
+ "
\n",
+ "\n",
+ " \n",
+ "
\n",
+ "\n",
+ " \n",
+ " \n",
+ "
\n",
+ "\n",
+ " \n",
+ " \n",
+ "
\n",
+ "
\n",
+ " Scheduler
\n", + "Scheduler-5d1add51-58dc-4ebd-9d19-53c55ad336e8
\n", + "| \n", + " Comm: tcp://127.0.0.1:59070\n", + " | \n", + "\n", + " Workers: 4\n", + " | \n", + "
| \n", + " Dashboard: http://127.0.0.1:8787/status\n", + " | \n", + "\n", + " Total threads: 8\n", + " | \n", + "
| \n", + " Started: Just now\n", + " | \n", + "\n", + " Total memory: 16.00 GiB\n", + " | \n", + "
\n",
+ " \n",
+ "
\n",
+ "\n",
+ " \n",
+ "
\n",
+ "\n",
+ " Workers
\n",
+ "
\n",
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ " \n",
+ " Worker: 0
\n",
+ "
\n",
+ " | \n", + " Comm: tcp://192.168.0.205:59086\n", + " | \n", + "\n", + " Total threads: 2\n", + " | \n", + "
| \n", + " Dashboard: http://192.168.0.205:59089/status\n", + " | \n", + "\n", + " Memory: 4.00 GiB\n", + " | \n", + "
| \n", + " Nanny: tcp://127.0.0.1:59074\n", + " | \n", + "\n", + " |
| \n", + " Local directory: /Users/ncclementi/Documents/git/dask-mini-tutorial/notebooks/dask-worker-space/worker-bs2rt3dr\n", + " | \n", + "
\n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ " \n",
+ " Worker: 1
\n",
+ "
\n",
+ " | \n", + " Comm: tcp://192.168.0.205:59087\n", + " | \n", + "\n", + " Total threads: 2\n", + " | \n", + "
| \n", + " Dashboard: http://192.168.0.205:59088/status\n", + " | \n", + "\n", + " Memory: 4.00 GiB\n", + " | \n", + "
| \n", + " Nanny: tcp://127.0.0.1:59075\n", + " | \n", + "\n", + " |
| \n", + " Local directory: /Users/ncclementi/Documents/git/dask-mini-tutorial/notebooks/dask-worker-space/worker-_y3pjg3g\n", + " | \n", + "
\n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ " \n",
+ " Worker: 2
\n",
+ "
\n",
+ " | \n", + " Comm: tcp://192.168.0.205:59081\n", + " | \n", + "\n", + " Total threads: 2\n", + " | \n", + "
| \n", + " Dashboard: http://192.168.0.205:59082/status\n", + " | \n", + "\n", + " Memory: 4.00 GiB\n", + " | \n", + "
| \n", + " Nanny: tcp://127.0.0.1:59073\n", + " | \n", + "\n", + " |
| \n", + " Local directory: /Users/ncclementi/Documents/git/dask-mini-tutorial/notebooks/dask-worker-space/worker-d0xcrp9k\n", + " | \n", + "
\n",
+ " \n",
+ "
\n",
+ " \n",
+ "\n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ " \n",
+ " Worker: 3
\n",
+ "
\n",
+ " | \n", + " Comm: tcp://192.168.0.205:59080\n", + " | \n", + "\n", + " Total threads: 2\n", + " | \n", + "
| \n", + " Dashboard: http://192.168.0.205:59083/status\n", + " | \n", + "\n", + " Memory: 4.00 GiB\n", + " | \n", + "
| \n", + " Nanny: tcp://127.0.0.1:59072\n", + " | \n", + "\n", + " |
| \n", + " Local directory: /Users/ncclementi/Documents/git/dask-mini-tutorial/notebooks/dask-worker-space/worker-mun11dkj\n", + " | \n", + "
\n",
+ "\n",
+ "# Parallel and Distributed Machine Learning\n",
+ "\n",
+ "The material in this notebook was based on the open-source content from [Dask's tutorial repository](https://github.com/dask/dask-tutorial) and the [Machine learning notebook](https://github.com/coiled/data-science-at-scale/blob/master/3-machine-learning.ipynb) from data science at scale from coiled\n",
+ "\n",
+ "So far we have seen how Dask makes data analysis scalable with parallelization via Dask DataFrames. Let's now see how [Dask-ML](https://ml.dask.org/) allows us to do machine learning in a parallel and distributed manner. Note, machine learning is really just a special case of data analysis (one that automates analytical model building), so the 💪 Dask gains 💪 we've seen will apply here as well!\n",
+ "\n",
+ "(If you'd like a refresher on the difference between parallel and distributed computing, [here's a good discussion on StackExchange](https://cs.stackexchange.com/questions/1580/distributed-vs-parallel-computing).)\n",
+ "\n",
+ "\n",
+ "## Types of scaling problems in machine learning\n",
+ "\n",
+ "There are two main types of scaling challenges you can run into in your machine learning workflow: scaling the **size of your data** and scaling the **size of your model**. That is:\n",
+ "\n",
+ "1. **CPU-bound problems**: Data fits in RAM, but training takes too long. Many hyperparameter combinations, a large ensemble of many models, etc.\n",
+ "2. **Memory-bound problems**: Data is larger than RAM, and sampling isn't an option.\n",
+ "\n",
+ "Here's a handy diagram for visualizing these problems:\n",
+ "\n",
+ "![\"scaling](\"https://raw.githubusercontent.com/coiled/data-science-at-scale/master/images/dimensions_of_scale.svg\"\n",)
\n",
+ "\n",
+ "\n",
+ "In the bottom-left quadrant, your datasets are not too large (they fit comfortably in RAM) and your model is not too large either. When these conditions are met, you are much better off using something like scikit-learn, XGBoost, and similar libraries. You don't need to leverage multiple machines in a distributed manner with a library like Dask-ML. However, if you are in any of the other quadrants, distributed machine learning is the way to go.\n",
+ "\n",
+ "Summarizing: \n",
+ "\n",
+ "* For in-memory problems, just use scikit-learn (or your favorite ML library).\n",
+ "* For large models, use `dask_ml.joblib` and your favorite scikit-learn estimator.\n",
+ "* For large datasets, use `dask_ml` estimators.\n",
+ "\n",
+ "## Scikit-Learn in five minutes\n",
+ "\n",
+ "![\"sklearn](\"https://raw.githubusercontent.com/coiled/data-science-at-scale/master/images/scikit_learn_logo_small.svg\"){kind=logo}
\n",
+ "\n",
+ "\n",
+ "In this section, we'll quickly run through a typical Scikit-Learn workflow:\n",
+ "\n",
+ "* Load some data (in this case, we'll generate it)\n",
+ "* Import the Scikit-Learn module for our chosen ML algorithm\n",
+ "* Create an estimator for that algorithm and fit it with our data\n",
+ "* Inspect the learned attributes\n",
+ "* Check the accuracy of our model\n",
+ "\n",
+ "Scikit-Learn has a nice, consistent API:\n",
+ "\n",
+ "* You instantiate an `Estimator` (e.g. `LinearRegression`, `RandomForestClassifier`, etc.). All of the models *hyperparameters* (user-specified parameters, not the ones learned by the estimator) are passed to the estimator when it's created.\n",
+ "* You call `estimator.fit(X, y)` to train the estimator.\n",
+ "* Use `estimator` to inspect attributes, make predictions, etc. \n",
+ "\n",
+ "Here `X` is an array of *feature variables* (what you're using to predict) and `y` is an array of *target variables* (what we're trying to predict)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b6e7379d",
+ "metadata": {},
+ "source": [
+ "### Generate some random data"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "id": "8613c202",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from sklearn.datasets import make_classification\n",
+ "\n",
+ "# Generate data\n",
+ "X, y = make_classification(n_samples=10000, n_features=4, random_state=0)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1fbcf18d",
+ "metadata": {},
+ "source": [
+ "**Refreshing some ML concepts**\n",
+ "\n",
+ "- `X` is the samples matrix (or design matrix). The size of `X` is typically (`n_samples`, `n_features`), which means that samples are represented as rows and features are represented as columns.\n",
+ "- A \"feature\" (also called an \"attribute\") is a measurable property of the phenomenon we're trying to analyze. A feature for a dataset of employees might be their hire date, for example.\n",
+ "- `y` are the target values, which are real numbers for regression tasks, or integers for classification (or any other discrete set of values). For unsupervized learning tasks, `y` does not need to be specified. `y` is usually 1d array where the `i`th entry corresponds to the target of the `i`th sample (row) of `X`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "id": "85824219",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "array([[-0.77244139, 0.3607576 , -2.38110133, 0.08757 ],\n",
+ " [ 1.14946035, 0.62254594, 0.37302939, 0.45965795],\n",
+ " [-1.90879217, -1.1602627 , -0.27364545, -0.82766028],\n",
+ " [-0.77694695, 0.31434299, -2.26231851, 0.06339125],\n",
+ " [-1.17047054, 0.02212382, -2.17376797, -0.13421976],\n",
+ " [ 0.79010037, 0.68530624, -0.44740487, 0.44692959],\n",
+ " [ 1.68616989, 1.6329131 , -1.42072654, 1.04050557],\n",
+ " [-0.93912893, -1.02270838, 1.10093827, -0.63714432]])"
+ ]
+ },
+ "execution_count": 2,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Let's take a look at X\n",
+ "X[:8]"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "id": "7dd8ef9a",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "array([0, 0, 1, 0, 0, 0, 0, 1])"
+ ]
+ },
+ "execution_count": 3,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Let's take a look at y\n",
+ "y[:8]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "80d61bf2",
+ "metadata": {},
+ "source": [
+ "### Fitting and SVC\n",
+ "\n",
+ "For this example, we will fit a [Support Vector Classifier](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "id": "5d638ce9",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "SVC(random_state=0)"
+ ]
+ },
+ "execution_count": 4,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "from sklearn.svm import SVC\n",
+ "\n",
+ "estimator = SVC(random_state=0)\n",
+ "estimator.fit(X, y)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "4068406a",
+ "metadata": {},
+ "source": [
+ "We can inspect the learned features by taking a look a the `support_vectors_`:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "id": "b891e47a",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "array([[-0.77244139, 0.3607576 , -2.38110133, 0.08757 ],\n",
+ " [ 1.14946035, 0.62254594, 0.37302939, 0.45965795],\n",
+ " [-0.77694695, 0.31434299, -2.26231851, 0.06339125],\n",
+ " [ 0.79010037, 0.68530624, -0.44740487, 0.44692959]])"
+ ]
+ },
+ "execution_count": 5,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "estimator.support_vectors_[:4]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a175852d",
+ "metadata": {},
+ "source": [
+ "And we check the accuracy:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "id": "8300ec8e",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "0.905"
+ ]
+ },
+ "execution_count": 6,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "estimator.score(X, y)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1efac1bd",
+ "metadata": {},
+ "source": [
+ "There are [3 different approaches](https://scikit-learn.org/0.15/modules/model_evaluation.html) to evaluate the quality of predictions of a model. One of them is the **estimator score method**. Estimators have a score method providing a default evaluation criterion for the problem they are designed to solve, which is discussed in each estimator's documentation.\n",
+ "\n",
+ "### Hyperparameter Optimization\n",
+ "\n",
+ "There are a few ways to learn the best *hyper*parameters while training. One is `GridSearchCV`.\n",
+ "As the name implies, this does a brute-force search over a grid of hyperparameter combinations. Scikit-learn provides tools to automatically find the best parameter combinations via cross-validation (which is the \"CV\" in `GridSearchCV`)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "id": "b4659297",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from sklearn.model_selection import GridSearchCV"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "id": "f6c889f4",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Fitting 2 folds for each of 4 candidates, totalling 8 fits\n",
+ "[CV] END ................................C=0.001, kernel=rbf; total time= 2.9s\n",
+ "[CV] END ................................C=0.001, kernel=rbf; total time= 2.9s\n",
+ "[CV] END ...............................C=0.001, kernel=poly; total time= 1.2s\n",
+ "[CV] END ...............................C=0.001, kernel=poly; total time= 1.1s\n",
+ "[CV] END .................................C=10.0, kernel=rbf; total time= 0.8s\n",
+ "[CV] END .................................C=10.0, kernel=rbf; total time= 0.7s\n",
+ "[CV] END ................................C=10.0, kernel=poly; total time= 1.1s\n",
+ "[CV] END ................................C=10.0, kernel=poly; total time= 1.0s\n",
+ "CPU times: user 13.7 s, sys: 237 ms, total: 13.9 s\n",
+ "Wall time: 14 s\n"
+ ]
+ },
+ {
+ "data": {
+ "text/plain": [
+ "GridSearchCV(cv=2,\n",
+ " estimator=SVC(gamma='auto', probability=True, random_state=0),\n",
+ " param_grid={'C': [0.001, 10.0], 'kernel': ['rbf', 'poly']},\n",
+ " verbose=2)"
+ ]
+ },
+ "execution_count": 8,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "%%time\n",
+ "estimator = SVC(gamma='auto', random_state=0, probability=True)\n",
+ "param_grid = {\n",
+ " 'C': [0.001, 10.0],\n",
+ " 'kernel': ['rbf', 'poly'],\n",
+ "}\n",
+ "\n",
+ "# Brute-force search over a grid of hyperparameter combinations\n",
+ "grid_search = GridSearchCV(estimator, param_grid, verbose=2, cv=2)\n",
+ "grid_search.fit(X, y)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "id": "ad134157",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "({'C': 10.0, 'kernel': 'rbf'}, 0.9086000000000001)"
+ ]
+ },
+ "execution_count": 9,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "grid_search.best_params_, grid_search.best_score_"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "228deb1b",
+ "metadata": {},
+ "source": [
+ "## Compute Bound: Single-machine parallelism with Joblib\n",
+ "\n",
+ "![\"Joblib](\"https://raw.githubusercontent.com/coiled/data-science-at-scale/master/images/joblib_logo.svg\"){kind=logo}
\n",
+ "\n",
+ "In this section we'll see how [Joblib](https://joblib.readthedocs.io/en/latest/) (\"*a set of tools to provide lightweight pipelining in Python*\") gives us parallelism on our laptop. Here's what our grid search graph would look like if we set up six training \"jobs\" in parallel:\n",
+ "\n",
+ "![\"grid](\"https://raw.githubusercontent.com/coiled/data-science-at-scale/master/images/unmerged_grid_search_graph.svg\")
\n",
+ "\n",
+ "With Joblib, we can say that Scikit-Learn has *single-machine* parallelism.\n",
+ "Any Scikit-Learn estimator that can operate in parallel exposes an `n_jobs` keyword, which tells you how many tasks to run in parallel. Specifying `n_jobs=-1` jobs means running the maximum possible number of tasks in parallel."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "id": "d9bead16",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Fitting 2 folds for each of 4 candidates, totalling 8 fits\n",
+ "CPU times: user 2.22 s, sys: 107 ms, total: 2.33 s\n",
+ "Wall time: 6.52 s\n"
+ ]
+ },
+ {
+ "data": {
+ "text/plain": [
+ "GridSearchCV(cv=2,\n",
+ " estimator=SVC(gamma='auto', probability=True, random_state=0),\n",
+ " n_jobs=-1,\n",
+ " param_grid={'C': [0.001, 10.0], 'kernel': ['rbf', 'poly']},\n",
+ " verbose=2)"
+ ]
+ },
+ "execution_count": 10,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "%%time\n",
+ "grid_search = GridSearchCV(estimator, param_grid, verbose=2, cv=2, n_jobs=-1)\n",
+ "grid_search.fit(X, y)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "e7c73f87",
+ "metadata": {},
+ "source": [
+ "Notice that the computation above it is faster than before. If you are running this computation on binder, you might not see a speed-up and the reason for that is that binder instances tend to have only one core with no threads so you can't see any parallelism. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1fe6255f",
+ "metadata": {},
+ "source": [
+ "## Compute Bound: Multi-machine parallelism with Dask\n",
+ "\n",
+ "\n",
+ "In this section we'll see how Dask (plus Joblib and Scikit-Learn) gives us multi-machine parallelism. Here's what our grid search graph would look like if we allowed Dask to schedule our training \"jobs\" over multiple machines in our cluster:\n",
+ "\n",
+ "![\"merged](\"https://raw.githubusercontent.com/coiled/data-science-at-scale/master/images/merged_grid_search_graph.svg\")
\n",
+ " \n",
+ "We can say that Dask can talk to Scikit-Learn (via Joblib) so that our *cluster* is used to train a model. \n",
+ "\n",
+ "If we run this on a laptop, it will take quite some time, but the CPU usage will be satisfyingly near 100% for the duration. To run faster, we would need a distributed cluster. For details on how to create a LocalCluster you can check the Dask documentation on [Single Machine: dask.distributed](https://docs.dask.org/en/latest/setup/single-distributed.html). \n",
+ "\n",
+ "Let's instantiate a Client with `n_workers=4`, which will give us a `LocalCluster`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "id": "cdf776a5",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ " \n",
+ "
"
+ ],
+ "text/plain": [
+ "\n",
+ " \n",
+ "\n",
+ "
\n",
+ "\n",
+ " \n",
+ "
\n",
+ "Client
\n", + "Client-d607f712-1beb-11ec-83b7-1e00ea0a0276
\n", + "| Connection method: Cluster object | \n", + "Cluster type: distributed.LocalCluster | \n", + " \n", + "
| \n", + " Dashboard: http://127.0.0.1:8787/status\n", + " | \n", + "\n", + " |
\n",
+ "
\n",
+ "
\n",
+ " \n",
+ "\n",
+ " Cluster Info
\n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ "\n",
+ "
\n",
+ "LocalCluster
\n", + "7ce68b21
\n", + "| \n", + " Dashboard: http://127.0.0.1:8787/status\n", + " | \n", + "\n", + " Workers: 4\n", + " | \n", + "
| \n", + " Total threads: 8\n", + " | \n", + "\n", + " Total memory: 16.00 GiB\n", + " | \n", + "
| Status: running | \n", + "Using processes: True | \n", + "
\n",
+ " \n",
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ " Scheduler Info
\n",
+ "
\n",
+ "\n",
+ " \n",
+ "
\n",
+ "\n",
+ " \n",
+ " \n",
+ "
\n",
+ "\n",
+ " \n",
+ " \n",
+ "
\n",
+ "
\n",
+ " Scheduler
\n", + "Scheduler-d35eb497-bd1b-4230-9373-a0fc4d50ce74
\n", + "| \n", + " Comm: tcp://127.0.0.1:56106\n", + " | \n", + "\n", + " Workers: 4\n", + " | \n", + "
| \n", + " Dashboard: http://127.0.0.1:8787/status\n", + " | \n", + "\n", + " Total threads: 8\n", + " | \n", + "
| \n", + " Started: Just now\n", + " | \n", + "\n", + " Total memory: 16.00 GiB\n", + " | \n", + "
\n",
+ " \n",
+ "
\n",
+ "\n",
+ " \n",
+ "
\n",
+ "\n",
+ " Workers
\n",
+ "
\n",
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ " \n",
+ " Worker: 0
\n",
+ "
\n",
+ " | \n", + " Comm: tcp://127.0.0.1:56119\n", + " | \n", + "\n", + " Total threads: 2\n", + " | \n", + "
| \n", + " Dashboard: http://127.0.0.1:56121/status\n", + " | \n", + "\n", + " Memory: 4.00 GiB\n", + " | \n", + "
| \n", + " Nanny: tcp://127.0.0.1:56108\n", + " | \n", + "\n", + " |
| \n", + " Local directory: /Users/ncclementi/Documents/git/dask-mini-tutorial/notebooks/dask-worker-space/worker-xoihndbb\n", + " | \n", + "
\n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ " \n",
+ " Worker: 1
\n",
+ "
\n",
+ " | \n", + " Comm: tcp://127.0.0.1:56116\n", + " | \n", + "\n", + " Total threads: 2\n", + " | \n", + "
| \n", + " Dashboard: http://127.0.0.1:56122/status\n", + " | \n", + "\n", + " Memory: 4.00 GiB\n", + " | \n", + "
| \n", + " Nanny: tcp://127.0.0.1:56110\n", + " | \n", + "\n", + " |
| \n", + " Local directory: /Users/ncclementi/Documents/git/dask-mini-tutorial/notebooks/dask-worker-space/worker-jk05nngf\n", + " | \n", + "
\n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ " \n",
+ " Worker: 2
\n",
+ "
\n",
+ " | \n", + " Comm: tcp://127.0.0.1:56117\n", + " | \n", + "\n", + " Total threads: 2\n", + " | \n", + "
| \n", + " Dashboard: http://127.0.0.1:56120/status\n", + " | \n", + "\n", + " Memory: 4.00 GiB\n", + " | \n", + "
| \n", + " Nanny: tcp://127.0.0.1:56109\n", + " | \n", + "\n", + " |
| \n", + " Local directory: /Users/ncclementi/Documents/git/dask-mini-tutorial/notebooks/dask-worker-space/worker-iebvxbsd\n", + " | \n", + "
\n",
+ " \n",
+ "
\n",
+ " \n",
+ "\n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ " \n",
+ " Worker: 3
\n",
+ "
\n",
+ " | \n", + " Comm: tcp://127.0.0.1:56118\n", + " | \n", + "\n", + " Total threads: 2\n", + " | \n", + "
| \n", + " Dashboard: http://127.0.0.1:56123/status\n", + " | \n", + "\n", + " Memory: 4.00 GiB\n", + " | \n", + "
| \n", + " Nanny: tcp://127.0.0.1:56111\n", + " | \n", + "\n", + " |
| \n", + " Local directory: /Users/ncclementi/Documents/git/dask-mini-tutorial/notebooks/dask-worker-space/worker-aogycp1g\n", + " | \n", + "
\n",
+ "
| \n",
+ " \n", + " \n", + " | \n", + "
\n",
+ "
| \n",
+ " \n", + " \n", + " | \n", + "
\n", + "
![\"Coiled](\"https://raw.githubusercontent.com/coiled/data-science-at-scale/master/images/Coiled-Logo_Horizontal_RGB_Black.png\"\n",){kind=logo}
\n",
+ "\n", + "\n", + "In this section we'll see how Coiled allows us to solve machine learning problems with multi-machine parallelism in the cloud.\n", + "\n", + "Coiled, [among other things](https://coiled.io/product/), provides hosted and scalable Dask clusters. The biggest barriers to entry for doing machine learning at scale are \"Do you have access to a cluster?\" and \"Do you know how to manage it?\" Coiled solves both of those problems. \n", + "\n", + "We'll spin up a Coiled cluster (with 10 workers in this case), then instantiate a Dask Client to use with that cluster.\n", + "\n", + "To learn more about how to set up an environment you can visit Coiled documentation on [Creating software environments](https://docs.coiled.io/user_guide/software_environment_creation.html)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "id": "58d6d915", + "metadata": {}, + "outputs": [], + "source": [ + "import coiled\n", + "from dask.distributed import Client" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "id": "e50ddd7c", + "metadata": {}, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Output()" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Found software environment build\n", + "Created FW rules: coiled-dask-nccleme1a-48073-firewall\n", + "Created scheduler VM: coiled-dask-nccleme1a-48073-scheduler (type: t3.medium, ip: [])\n" + ] + }, + { + "data": { + "text/html": [ + "\n" + ], + "text/plain": [] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
\n",
+ " \n",
+ "
"
+ ],
+ "text/plain": [
+ "\n",
+ " \n",
+ "\n",
+ "
\n",
+ "\n",
+ " \n",
+ "
\n",
+ "Client
\n", + "Client-ebddc9ca-1bed-11ec-83b7-1e00ea0a0276
\n", + "| Connection method: Cluster object | \n", + "Cluster type: coiled.Cluster | \n", + " \n", + "
| \n", + " Dashboard: http://3.239.164.166:8787\n", + " | \n", + "\n", + " |
\n",
+ "
\n",
+ "
\n",
+ " \n",
+ "\n",
+ " Cluster Info
\n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ "\n",
+ "
\n",
+ "Cluster
\n", + "ncclementi-e7940354-4
\n", + "| \n", + " Dashboard: http://3.239.164.166:8787\n", + " | \n", + "\n", + " Workers: 3\n", + " | \n", + "
| \n", + " Total threads: 6\n", + " | \n", + "\n", + " Total memory: 22.43 GiB\n", + " | \n", + "
\n",
+ " \n",
+ "
\n",
+ "\n",
+ "
\n",
+ " \n",
+ " Scheduler Info
\n",
+ "
\n",
+ "\n",
+ " \n",
+ "
\n",
+ "\n",
+ " \n",
+ " \n",
+ "
\n",
+ "\n",
+ " \n",
+ " \n",
+ "
\n",
+ "
\n",
+ " Scheduler
\n", + "Scheduler-8a1a9f85-1bcf-4376-8627-d8db098a7cad
\n", + "| \n", + " Comm: tls://10.4.10.249:8786\n", + " | \n", + "\n", + " Workers: 3\n", + " | \n", + "
| \n", + " Dashboard: http://10.4.10.249:8787/status\n", + " | \n", + "\n", + " Total threads: 6\n", + " | \n", + "
| \n", + " Started: Just now\n", + " | \n", + "\n", + " Total memory: 22.43 GiB\n", + " | \n", + "
\n",
+ " \n",
+ "
\n",
+ "\n",
+ " \n",
+ "
\n",
+ "\n",
+ " Workers
\n",
+ "
\n",
+ "\n",
+ " \n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ " \n",
+ " Worker: coiled-dask-nccleme1a-48073-worker-c2dd20ad0c
\n",
+ "
\n",
+ " | \n", + " Comm: tls://10.4.29.128:41265\n", + " | \n", + "\n", + " Total threads: 2\n", + " | \n", + "
| \n", + " Dashboard: http://10.4.29.128:42261/status\n", + " | \n", + "\n", + " Memory: 7.48 GiB\n", + " | \n", + "
| \n", + " Nanny: tls://10.4.29.128:35733\n", + " | \n", + "\n", + " |
| \n", + " Local directory: /dask-worker-space/worker-ze7tje4w\n", + " | \n", + "
\n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ " \n",
+ " Worker: coiled-dask-nccleme1a-48073-worker-f0ed5c9314
\n",
+ "
\n",
+ " | \n", + " Comm: tls://10.4.28.200:41553\n", + " | \n", + "\n", + " Total threads: 2\n", + " | \n", + "
| \n", + " Dashboard: http://10.4.28.200:33961/status\n", + " | \n", + "\n", + " Memory: 7.48 GiB\n", + " | \n", + "
| \n", + " Nanny: tls://10.4.28.200:37411\n", + " | \n", + "\n", + " |
| \n", + " Local directory: /dask-worker-space/worker-wwdpgqq7\n", + " | \n", + "
\n",
+ " \n",
+ "
\n",
+ " \n",
+ "\n",
+ " \n",
+ "
\n",
+ " \n",
+ " \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
\n",
+ " \n",
+ " Worker: coiled-dask-nccleme1a-48073-worker-f475e55e68
\n",
+ "
\n",
+ " | \n", + " Comm: tls://10.4.26.58:46645\n", + " | \n", + "\n", + " Total threads: 2\n", + " | \n", + "
| \n", + " Dashboard: http://10.4.26.58:42815/status\n", + " | \n", + "\n", + " Memory: 7.48 GiB\n", + " | \n", + "
| \n", + " Nanny: tls://10.4.26.58:39903\n", + " | \n", + "\n", + " |
| \n", + " Local directory: /dask-worker-space/worker-izvh1s2b\n", + " | \n", + "