{
 "cells": [
  {
   "cell_type": "raw",
   "id": "eaf29ef7",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    ".. _data-caching:\n",
    "\n",
    "************\n",
    "Data Caching\n",
    "************\n",
    "\n",
    "This chapter discusses MSG's data caching, a key component of its performance and scaleability. Each instance of a grid data type (e.g., :f:type:`~fmsg_m/specgrid_t` in Fortran, or :py:class:`pymsg.PhotGrid` in Python) has an associated cache that temporarily stores spectroscopic and/or photometric data for use in interpolations. This grid cache is initially empty when the instance is created, but grows in size as data are loaded on-demand from disk. Eventually, once a user-definable size limit is reached, old data are discarded from the cache to make room for new.\n",
    "\n",
    "Caching Demo\n",
    "============\n",
    "\n",
    "With its caching functionality, MSG can in principle work with grids that are much larger on disk than available computer memory. The following Python code fragments showcase this capability, while also illustrating the basics of inspecting and controlling a grid cache.\n",
    "\n",
    "To get started, download the `sg-CAP18-coarse.h5 <http://user.astro.wisc.edu/~townsend/resource/download/msg/grids/CAP18/coarse/sg-CAP18-coarse.h5>`__ spectroscopic grid, storing it your working directory. Then, initialize MSG and load the grid:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "2eb4244c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import standard modules\n",
    "\n",
    "import sys\n",
    "import os\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import random as rn\n",
    "import time\n",
    "\n",
    "# Import pymsg\n",
    "\n",
    "import pymsg\n",
    "\n",
    "# Set plot parameters\n",
    "\n",
    "%matplotlib inline\n",
    "plt.rcParams.update({'font.size': 16})\n",
    "\n",
    "# Load the SpecGrid\n",
    "\n",
    "specgrid = pymsg.SpecGrid('sg-CAP18-coarse.h5')"
   ]
  },
  {
   "cell_type": "raw",
   "id": "5b4d9d5f",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "Next, inspect the state of the cache attached to the :code:`specgrid` object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "88947edc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cache usage: 128 MB\n",
      "cache limit: 128 MB\n"
     ]
    }
   ],
   "source": [
    "# Inspect the cache\n",
    "\n",
    "print(f'cache usage: {specgrid.cache_usage} MB')\n",
    "print(f'cache limit: {specgrid.cache_limit} MB')"
   ]
  },
  {
   "cell_type": "raw",
   "id": "556f2bd2",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "The :py:attr:`pymsg.SpecGrid.cache_usage` property reports the current memory usage (in megabytes) of the cache. Because we've just loaded :code:`specgrid`, this usage is zero --- no data have (yet) been read into memory. The :py:attr:`pymsg.SpecGrid.cache_limit` property likewise specifies the maximum memory usage (again, in megabytes) of the cache. By default, this limit is set to 128 MB.\n",
    "\n",
    "Let's define a function that interpolates the visible-spectrum flux for a randomly-chosen set of photospheric parameters. Since we're interested primarily in the behavior of the grid cache, the function returns its execution time rather than interpolation result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "b4d6d202",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Random flux routine\n",
    "\n",
    "def random_flux():\n",
    "    \n",
    "    start_time = time.process_time()\n",
    "    \n",
    "    # Set up the wavelength abscissa (visible-spectrum)\n",
    "    \n",
    "    lam_min = 3000.\n",
    "    lam_max = 7000.\n",
    "\n",
    "    lam = np.linspace(lam_min, lam_max, 1000)\n",
    "    \n",
    "    # Loop until a valid set of photospheric parameters is found\n",
    "    \n",
    "    while True:\n",
    "        \n",
    "        # Choose random photospheric parameters\n",
    "    \n",
    "        x = {}\n",
    "    \n",
    "        for label in specgrid.axis_labels:\n",
    "            x[label] = rn.uniform(specgrid.axis_x_min[label], specgrid.axis_x_max[label])\n",
    "            \n",
    "        # Interpolate the flux, allowing for the fact that the \n",
    "        # photospheric parameters may fall in a grid void\n",
    "\n",
    "        try:\n",
    "            F = specgrid.flux(x, lam)\n",
    "            break\n",
    "        except LookupError:\n",
    "            pass\n",
    "        \n",
    "    end_time = time.process_time()\n",
    "    \n",
    "    return end_time-start_time"
   ]
  },
  {
   "cell_type": "raw",
   "id": "05ad7b43",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "Running this function a few times (for now ignoring the return value), we can watch the cache fill up to the 128-MB limit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "44dc51b8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cache usage: 86 MB\n",
      "cache usage: 128 MB\n",
      "cache usage: 128 MB\n"
     ]
    }
   ],
   "source": [
    "# Run random_flux three times\n",
    "\n",
    "for i in range(3):\n",
    "    random_flux()\n",
    "    print(f'cache usage: {specgrid.cache_usage} MB')"
   ]
  },
  {
   "cell_type": "raw",
   "id": "5da7d36e",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "If needed this limit can be increased by changing the :py:attr:`pymsg.SpecGrid.cache_limit` property, as this example demonstrates:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "c6fe23f5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cache usage: 215 MB\n",
      "cache usage: 256 MB\n",
      "cache usage: 256 MB\n"
     ]
    }
   ],
   "source": [
    "# Increase the cache limit to 256 MB\n",
    "\n",
    "specgrid.cache_limit = 256\n",
    "\n",
    "# Run random_flux three times\n",
    "\n",
    "for i in range(3):\n",
    "    random_flux()\n",
    "    print(f'cache usage: {specgrid.cache_usage} MB')"
   ]
  },
  {
   "cell_type": "raw",
   "id": "fb3e44f6",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "Finally, the contents of the cache can be flushed (and the memory freed up) with a call to the :py:func:`pymsg.SpecGrid.flush_cache()` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "309be11b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cache usage: 256 MB\n",
      "cache usage: 0 MB\n"
     ]
    }
   ],
   "source": [
    "# Flush the cache\n",
    "\n",
    "print(f'cache usage: {specgrid.cache_usage} MB')\n",
    "\n",
    "specgrid.flush_cache()\n",
    "\n",
    "print(f'cache usage: {specgrid.cache_usage} MB')"
   ]
  },
  {
   "cell_type": "raw",
   "id": "14dea26a",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    ".. _wavelength-subset:\n",
    "\n",
    "Wavelength Subsetting\n",
    "=====================\n",
    "\n",
    "When working with spectroscopic grids, it's often the case one is interested only in a subset of the wavelength range covered by the grid. As an example, the :py:func:`random_flux` function defined above focuses on just the visible part of the electromagnetic spectrum. In such cases, you can direct MSG to cache only the data within the subset by setting the :py:attr:`pymsg.SpecGrid.cache_lam_min` and :py:attr:`pymsg.SpecGrid.cache_lam_min` properties. This has the benefit of slowing the rate at which the cache fills up, as the following example demonstrates:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "604c9e66",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cache usage: 18 MB\n",
      "cache usage: 30 MB\n",
      "cache usage: 55 MB\n"
     ]
    }
   ],
   "source": [
    "# Subset to the visible part of the spectrum\n",
    "\n",
    "specgrid.cache_lam_min = 3000.\n",
    "specgrid.cache_lam_max = 7000.\n",
    "\n",
    "# Run random_flux three times\n",
    "\n",
    "for i in range(3):\n",
    "    random_flux()\n",
    "    print(f'cache usage: {specgrid.cache_usage} MB')"
   ]
  },
  {
   "cell_type": "raw",
   "id": "a661a875",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "(compare this against the cache growth in the preceding examples).\n",
    "\n",
    "Performance Impact\n",
    "==================\n",
    "\n",
    "Let's now explore how caching can have a significant impact on MSG's performance. Run the following code, which expands the cache limit further to 512 MB, and then calls :py:func:`random_spectrum` 300 times, storing the usage and execution time after each call:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "8bb1a133",
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Flush the cache and set the limit to 512 MB\n",
    "\n",
    "specgrid.flush_cache()\n",
    "specgrid.cache_limit = 512\n",
    "\n",
    "# Allocate usage & timing arrays\n",
    "\n",
    "n = 300\n",
    "\n",
    "cache_usages_512 = np.empty(n, dtype=int)\n",
    "exec_timings_512 = np.empty(n)\n",
    "\n",
    "# Call random_flux\n",
    "\n",
    "for i in range(n):\n",
    "    exec_timings_512[i] = random_flux()\n",
    "    cache_usages_512[i] = specgrid.cache_usage"
   ]
  },
  {
   "cell_type": "raw",
   "id": "95bd286c",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "Then, plot the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "818c2eb0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'exution time (s)')"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 50672 characters]\n",
      "text/plain": [
       "<Figure size 864x432 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot cache usage and execution time\n",
    "\n",
    "fig, ax = plt.subplots(ncols=2, figsize=[12,6])\n",
    "\n",
    "ax[0].plot(cache_usages_512)\n",
    "ax[0].set_xlabel('call #')\n",
    "ax[0].set_ylabel('cache usage (MB)')\n",
    "\n",
    "ax[1].plot(exec_timings_512)\n",
    "ax[1].set_xlabel('call #')\n",
    "ax[1].set_ylabel('exution time (s)')"
   ]
  },
  {
   "cell_type": "raw",
   "id": "19c11a46",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "In the left-hand panel we see the cache usage grow and eventually asymptote toward the point where the entire grid (around 400 MB) is resident in memory. In the right-hand panel the execution time rapidly drops from ~0.2s down to ~0.01s, as more and more data can be read from memory (fast) rather than disk (slow).\n",
    "\n",
    "Let's now repeat the exercise, but for a cache limit reduced back down to 128 MB:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "54d7aad5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'exution time (s)')"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 92616 characters]\n",
      "text/plain": [
       "<Figure size 864x432 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Flush the cache and set the limit to 128 MB\n",
    "\n",
    "specgrid.flush_cache()\n",
    "specgrid.cache_limit = 128\n",
    "\n",
    "# Allocate usage & timing arrays\n",
    "\n",
    "n = 300\n",
    "\n",
    "cache_usages_128 = np.empty(n, dtype=int)\n",
    "exec_timings_128 = np.empty(n)\n",
    "\n",
    "# Call random_flux\n",
    "\n",
    "for i in range(n):\n",
    "    exec_timings_128[i] = random_flux()\n",
    "    cache_usages_128[i] = specgrid.cache_usage\n",
    "    \n",
    "# Plot cache usage and execution time\n",
    "\n",
    "fig, ax = plt.subplots(ncols=2, figsize=[12,6])\n",
    "\n",
    "ax[0].plot(cache_usages_512, label='512 MB limit')\n",
    "ax[0].plot(cache_usages_128, label='128 MB limit')\n",
    "ax[0].set_xlabel('call #')\n",
    "ax[0].set_ylabel('cache usage (MB)')\n",
    "ax[0].legend(loc=4)\n",
    "\n",
    "ax[1].plot(exec_timings_512)\n",
    "ax[1].plot(exec_timings_128)\n",
    "ax[1].set_xlabel('call #')\n",
    "ax[1].set_ylabel('exution time (s)')"
   ]
  },
  {
   "cell_type": "raw",
   "id": "03da5c24",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "The left-hand panel reveals that, as expected, the growth of the cache usage stops once the limit is reached. Because not all the data can be held in memory, the performance gains from the cache are more modest than in the previous case, with the execution times in the right-hand panel averaging 0.15s.\n",
    "\n",
    "Choosing Cache Settings\n",
    "=======================\n",
    "\n",
    "As the demonstration above makes clear, the choice of the :py:attr:`pymsg.SpecGrid.cache_usage` property can have a significant impact on the performance of MSG. Ideally, this property should be set to exceed the total size of the grid [#disk-vs-mem]_; but if that's not possible due to limited computer memory, try some of the following measures:\n",
    "\n",
    "* set the :py:attr:`pymsg.SpecGrid.cache_usage` property to at least :math:`4^{N}` times the size of a single node, where :math:`N` is the number of dimensions in a grid (see the :ref:`tensor-product-interpolation` appendix).\n",
    "* if undertaking a sequence of interpolations, reorder them so that ones with similar photospheric parameters are grouped together.\n",
    "* for spectroscopic grids, use the :py:attr:`pymsg.SpecGrid.cache_lam_min` and/or :py:attr:`pymsg.SpecGrid.cache_lam_max` properties, as discussed above in the :ref:`wavelength-subset` section.\n",
    "\n",
    "Technical Details\n",
    "=================\n",
    "\n",
    "The basic data unit of MSG's caches is a grid node, representing the spectroscopic or photometric intensity for a single set of photospheric parameters. During an interpolation the nodes required for the calculation are accessed through the cache, which reads them from disk as necessary. After every node access, the current cache usage is compared against the user-specified limit. If it exceeds this limit, then one or more nodes are evicted from the cache using a :wiki:`least-recently used (LRU) algorithm <Cache_replacement_policies#Least_recently_used_(LRU)>`.\n",
    "\n",
    "This implementation --- with eviction taking place **after** cache access --- means that in principle the cache limit can be zero. For performance reasons, however, this is not recommended.\n",
    "\n",
    ".. rubric:: Footnotes\n",
    "\n",
    ".. [#disk-vs-mem] Note that the in-memory size of a grid is typically larger than its on-disk size, due to \n",
    "                  compression."
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Raw Cell Format",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
