{ "cells": [ { "cell_type": "markdown", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "introduction" }, "source": [ "# Creating custom MDP problems in MDPax\n", "\n", "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/joefarrington/mdpax/blob/main/examples/create_custom_problem.ipynb)\n", "\n", "\n", "This tutorial shows how to implement your own Markov Decision Process (MDP) problems using MDPax. We'll implement the classic FrozenLake environment, which is a well-known example from [OpenAI's Gym/Gymnasium](https://gymnasium.farama.org/environments/toy_text/frozen_lake/).\n", "\n", "If you're running the notebook in Google Colab, you should verify that you're using a GPU instance. Click Runtime > Change runtime type and ensure \"GPU\" is selected as the Hardware accelerator. You can confirm GPU availability by running `!nvidia-smi` in a code cell.\n", "\n", "## Prerequisites\n", "\n", "Before getting started, you might find these resources helpful if you're unfamiliar with either JAX or MDPs. \n", "\n", "### JAX Background\n", "\n", "- [JAX Quickstart](https://jax.readthedocs.io/en/latest/notebooks/quickstart.html)\n", "- [JAX Tutorial](https://jax.readthedocs.io/en/latest/jax-101/index.html)\n", "- ๐Ÿ“บ [Machine Learning with JAX - From Zero to Hero | Aleksa Gordiฤ‡](https://www.youtube.com/watch?v=SstuvS-tVc0)\n", "\n", "### MDP Background\n", "\n", "- [Reinforcement Learning: An Introduction - Chapter 3 | Sutton & Barto](http://incompleteideas.net/book/RLbook2020.pdf)\n", "- ๐Ÿ“บ [Markov Decision Processes 1 - Value Iteration | Stanford CS221](https://www.youtube.com/watch?v=9g32v7bK3Co)\n", "\n", "\n", "## MDPax Problem class\n", "\n", "The `Problem` class is used to represent MDPs in MDPax. As described in the [Getting Started] notebook, it uses a functional approach to describe the MDP instead of explicitly creating transition and reward matrices. \n", "\n", "When creating a subclass of `Problem` to represent your own MDP, you will need to define an `__init__` method and the following methods marked as `@abstractmethods` in the `Problem` class:\n", "\n", "* `name`: A unique string identifier for the MDP\n", "* `_construct_state_space`: A helper function that builds an array of all possible states, with dimensions [n_states, state_dim]\n", "* `state_to_index`: A function that maps from a state vector to its index in the state space array\n", "* `_construct_action_space`: A helper function that builds an array of all possible actions, with dimensions [n_actions, action_dim]\n", "* `_construct_random_event_space`: A helper function that builds an array of all possible random events, with dimensions [n_events, event_dim]\n", "* `random_event_probability`: A function that returns the probability of a random event given a state and action\n", "* `transition`: A function that returns the next state and reward given a state, action, and random event. \n", "\n", "See the [MDPax documentation](https://mdpax.readthedocs.io/en/latest/) for detailed information about the `Problem` class.\n", "\n", "In this notebook we'll walk through implementing each of those methods for the FrozenLake problem. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Installation and imports" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you're running the notebook in Google Colab you may need to restart the runtime after installing the dependencies so that the updated packages can be loaded. You will receive a \"Warning\" message in the output of the cell below if this is the case." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "id": "setup" }, "outputs": [], "source": [ "import sys\n", "\n", "try:\n", " import mdptoolbox\n", " import mdpax\n", " import matplotlib\n", "except ImportError:\n", " if 'google.colab' in sys.modules:\n", " # Automatically install mdpax if running in Colab, environment is temporary\n", " !pip install \"mdpax[examples-colab] @ git+https://github.com/joefarrington/mdpax.git\"\n", " else:\n", " print(\"Dependencies not installed. Please follow the installation instructions in the README: https://github.com/joefarrington/mdpax\")" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "id": "imports" }, "outputs": [], "source": [ "import jax\n", "import jax.numpy as jnp\n", "import matplotlib.pyplot as plt\n", "import mdptoolbox\n", "import numpy as np\n", "\n", "from mdpax.core.problem import Problem\n", "from mdpax.utils.spaces import create_range_space\n", "from mdpax.utils.types import (\n", " ActionSpace,\n", " ActionVector,\n", " Policy,\n", " RandomEventSpace,\n", " RandomEventVector,\n", " Reward,\n", " StateSpace,\n", " StateVector,\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## FrozenLake\n", "\n", "In FrozenLake, an agent must navigate from start (S) to goal (G) on a frozen lake surface (F), avoiding holes in the ice (H).\n", "\n", "The default 4x4 map looks like this:\n", "```\n", "SFFF\n", "FHFH\n", "FFFH\n", "HFFG\n", "``` \n", "\n", "The surface can be slippery, making movement stochastic: when the agent chooses a direction, they may instead slide perpendicular to their intended direction." ] }, { "cell_type": "markdown", "metadata": { "id": "problem_intro" }, "source": [ "## 1 Step-by-step Problem definition\n", "\n", "First, we'll create our `FrozenLake` class that inherits from `mdpax.core.problem.Problem`. \n", "The base class handles much of the boilerplate, we just need to implement the required methods.\n", "\n", "In order to avoid duplication in the notebook, each part of the step-by-step definition inherits from the previous one, building up the class definition. All the cells in Part 1 therefore need to be run in order. To see the full class definition rather than working through the methods one by one you can skip ahead to [Part 2](##-2-Putting-it-all-together). " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1.1 The `__init__` method and config class" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We'll start with an outline of the class, with a placeholder for each method, and fill in the `__init__` method. We use some custom types in the function signature, which are defined in [mdpax/utils/types.py](https://github.com/joefarrington/mdpax/blob/main/src/mdpax/utils/types.py). \n", "\n", "Following the `gym` implementation, we need to support three arguments:\n", "* `desc`: A custom map\n", "* `map_name`: A key for a map in the dictionary of known maps `MAPS`\n", "* `is_slippery`: Whether the lake is slippery or not\n", "\n", "We'll comment out `super().__init__()` for now because it performs some setup procedures that rely on all the methods being defined and we are implementing them one by one." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "# Dictionary of known maps\n", "# From https://github.com/openai/gym/blob/master/gym/envs/toy_text/frozen_lake.py\n", "MAPS = {\n", " \"4x4\": [\"SFFF\", \"FHFH\", \"FFFH\", \"HFFG\"],\n", " \"8x8\": [\n", " \"SFFFFFFF\",\n", " \"FFFFFFFF\",\n", " \"FFFHFFFF\",\n", " \"FFFFFHFF\",\n", " \"FFFHFFFF\",\n", " \"FHHFFFHF\",\n", " \"FHFFHFHF\",\n", " \"FFFHFFFG\",\n", " ],\n", "}" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "class FrozenLake(Problem):\n", " \"\"\"FrozenLake MDP from OpenAI Gym.\n", "\n", " Models navigation on a grid world with slippery movement.\n", " \n", " The agent must navigate from start (S) to goal (G) on a frozen lake surface (F)\n", " where the surface is slippery and there are holes (H) that end the episode. \n", " \n", " Example 4x4 map:\n", " SFFF\n", " FHFH\n", " FFFH\n", " HFFG\n", "\n", " State Space (state_dim = 2):\n", " Vector containing:\n", " - Row: 1 element in range [0, n_rows-1]\n", " - Column: 1 element in range [0, n_cols-1]\n", "\n", " Action Space (action_dim = 1):\n", " Vector containing:\n", " - Intended movement direction: 1 element in range [0, 3]\n", " where:\n", " - 0: LEFT\n", " - 1: DOWN\n", " - 2: RIGHT\n", " - 3: UP\n", "\n", " Random Events (event_dim = 1):\n", " Vector containing:\n", " - Actual movement direction: 1 element in range [0, 3]\n", "\n", " Dynamics:\n", " 1. The agent chooses an intended movement direction\n", " 2. The agent moves in the actual direction.\n", " - If the surface is not slippery, the agent moves in the intended direction.\n", " - If the surface is slippery, the agent has a 1/3 probability of moving\n", " in the intended direction, a 1/3 probability of moving in the \n", " direction to the left of the intended direction, and a 1/3 \n", " probability of moving in the direction to the right of the intended \n", " direction.\n", " 3. The episode ends when the agent reaches the goal or falls into a hole\n", " \n", " Args:\n", " desc: Custom map layout as list of strings\n", " map_name: Key for a map in the dictionary of known maps `MAPS`\n", " is_slippery: If True, the movement direction is stochastic\n", " \"\"\"\n", " def __init__(\n", " self, \n", " desc: list[str] | None = None,\n", " map_name: str = \"4x4\",\n", " is_slippery: bool = True,\n", " ):\n", " self.desc = desc\n", " self.map_name = map_name\n", " self.is_slippery = is_slippery\n", "\n", " # Use a custom map if provided\n", " if desc is not None:\n", " self.map = desc\n", " else:\n", " self.map = MAPS[map_name]\n", "\n", " # Convert map to array for efficient lookup\n", " # Needs to be numeric so use 1 for hole and 2 for goal\n", " # 0 for frozen surface\n", " self.grid = jnp.array([\n", " [1 if c == 'H' else 2 if c == 'G' else 0 \n", " for c in row]\n", " for row in self.map\n", " ])\n", " self.n_rows, self.n_cols = self.grid.shape\n", " \n", " # This relies on all the methods being defined, so\n", " # comment out until class complete\n", " #super().__init__()\n", " \n", " @property\n", " def name(self) -> str:\n", " \"\"\"Unique identifier for this problem type.\"\"\"\n", " pass\n", " \n", " def _construct_state_space(self) -> StateSpace:\n", " \"\"\"Build array of all possible states.\n", "\n", " Returns:\n", " Array of shape [n_states, state_dim] containing all possible states\n", " \"\"\"\n", " pass\n", "\n", " def state_to_index(self, state: StateVector) -> int:\n", " \"\"\"Convert state vector to index.\n", "\n", " Args:\n", " state: Vector representation of a state [state_dim]\n", "\n", " Returns:\n", " Index of the state in state_space\n", "\n", " Note:\n", " This mapping must be consistent with the ordering in state_space\n", " \"\"\"\n", " pass\n", " \n", " def _construct_action_space(self) -> ActionSpace:\n", " \"\"\"Build an array of all possible actions.\n", "\n", " Returns:\n", " Array of shape [n_actions, action_dim] containing all possible actions\n", " \"\"\"\n", " pass\n", " \n", " def _construct_random_event_space(self) -> RandomEventSpace:\n", " \"\"\"Build an array of all possible random events.\n", "\n", " Returns:\n", " Array of shape [n_events, event_dim] containing all possible random events\n", " \"\"\"\n", " pass\n", " \n", " def random_event_probability(\n", " self,\n", " state: StateVector,\n", " action: ActionVector,\n", " random_event: RandomEventVector\n", " ) -> float:\n", " \"\"\"Calculate probability of random event given state-action pair.\n", "\n", " Args:\n", " state: Current state vector [state_dim]\n", " action: Action vector [action_dim]\n", " random_event: Random event vector [event_dim]\n", "\n", " Returns:\n", " Probability of the random event occurring\n", " \"\"\"\n", " pass\n", "\n", " def transition(\n", " self,\n", " state: StateVector,\n", " action: ActionVector,\n", " random_event: RandomEventVector\n", " ) -> tuple[StateVector, Reward]:\n", " \"\"\"Compute next state and reward for a transition.\n", "\n", " Args:\n", " state: Current state vector [state_dim]\n", " action: Action vector [action_dim]\n", " random_event: Random event vector [event_dim]\n", "\n", " Returns:\n", " Tuple containing the next state vector [state_dim] and\n", " the immediate reward\n", " \"\"\"\n", " pass" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we can create an instance using the 4x4 map with slippery movement:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "problem = FrozenLake(map_name=\"4x4\", is_slippery=True)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Array([[0, 0, 0, 0],\n", " [0, 1, 0, 1],\n", " [0, 0, 0, 1],\n", " [1, 0, 0, 2]], dtype=int32)" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "problem.grid" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1.2 The `name` property" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `name` property is used to set a checkpoint directory when one is not provided. It can therefore be a name for any problem represented by the class or, as here, provide some information about the input settings." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "class FrozenLake(FrozenLake):\n", " @property\n", " def name(self) -> str:\n", " \"\"\"Unique identifier for this problem type.\"\"\"\n", " s = \"slippery\" if self.is_slippery else \"non_slippery\"\n", " if self.desc is not None:\n", " return f\"frozen_lake_custom_{s}\"\n", " else:\n", " return f\"frozen_lake_{self.map_name}_{s}\"" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'frozen_lake_4x4_slippery'" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "problem = FrozenLake(map_name=\"4x4\", is_slippery=True)\n", "problem.name" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1.3 The `_construct_state_space` and `state_to_index` methods" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In MDPax, the `state_space` for a problem is a 2D array that contains every possible state for the MDP. Each row is a vector representing one state. This helper method is called when the class is instantiated, and builds the `state_space` array. \n", "\n", "The `state_to_index` method converts a state vector to its index in the `state_space`. The array representing the value function uses the same ordering as the `state_space` and therefore this is used to, for example, find the value of the next state during value iteration. The `transition` function returns a state vector, the index of that state is found using `state_to_index`, and then the value of the next state is identified by looking up that element in the array of values. \n", "\n", "In this problem, the state vector has two components: the row and column of the gridworld. All of the states between the bounds (0 and n_rows - 1 and 0 and n_columns - 1, for the two dimensions of our state) are valid and therefore we can use the MDPax helper function `create_range_space` to build the state space and a method to extract the index for a state." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "class FrozenLake(FrozenLake):\n", " def _construct_state_space(self) -> StateSpace:\n", " \"\"\"Build array of all possible states.\n", "\n", " Returns:\n", " Array of shape [n_states, state_dim] containing all possible states\n", " \"\"\"\n", " mins = np.zeros(2, dtype=np.int32)\n", " maxs = np.array([self.n_rows-1, self.n_cols-1], dtype=np.int32)\n", " state_space, self._state_to_index_fn = create_range_space(mins, maxs)\n", " return state_space\n", " \n", " \n", " def state_to_index(self, state: StateVector) -> int:\n", " \"\"\"Convert state vector to index.\"\"\"\n", " return self._state_to_index_fn(state)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Array([[0, 0],\n", " [0, 1],\n", " [0, 2],\n", " [0, 3],\n", " [1, 0],\n", " [1, 1],\n", " [1, 2],\n", " [1, 3],\n", " [2, 0],\n", " [2, 1],\n", " [2, 2],\n", " [2, 3],\n", " [3, 0],\n", " [3, 1],\n", " [3, 2],\n", " [3, 3]], dtype=int32)" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "problem = FrozenLake(map_name=\"4x4\", is_slippery=True)\n", "problem._construct_state_space() # 16 states for the (4x4) grid" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Array(0, dtype=int32)" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "problem.state_to_index(jnp.array([0, 0]))" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Array(7, dtype=int32)" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "problem.state_to_index(jnp.array([1, 3]))" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Array(15, dtype=int32)" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "problem.state_to_index(jnp.array([3, 3]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1.4 The `_construct_action_space` method" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `action_space` is a 2D array containing every possible action. Each row is a vector representing one action. This helper method is called when the class is instantiated, and builds the `action_space` array. \n", "\n", "In this problem, the action vector has one component: an integer representing the intended direction of movement. We reshape the array to [n_actions, 1] so that we have a 2D array which is expected by MDPax for all problems." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "class FrozenLake(FrozenLake):\n", " def _construct_action_space(self) -> ActionSpace:\n", " \"\"\"Build an array of all possible actions.\n", "\n", " Returns:\n", " Array of shape [n_actions, action_dim] containing all possible actions\n", " \"\"\"\n", " # One action for each direction\n", " return jnp.arange(4).reshape(-1, 1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Problem characteristics do not change the action space" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Array([[0],\n", " [1],\n", " [2],\n", " [3]], dtype=int32)" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "problem = FrozenLake(map_name=\"4x4\", is_slippery=True)\n", "problem._construct_action_space()" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Array([[0],\n", " [1],\n", " [2],\n", " [3]], dtype=int32)" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "problem = FrozenLake(map_name=\"4x4\", is_slippery=False)\n", "problem._construct_action_space()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1.5 The `_construct_random_event_space` method" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "MDP transitions may be stochastic, and MDPax handles this using `random_event_space`, a 2D array containing every possible random event. The `transition` function is deterministic given a state, action and random event. \n", "\n", "Each row is a vector representing one random event. This helper method is called when the class is instantiated, and builds the `random_event_space` array. \n", "\n", "In this problem, the random event vector has one component: an integer representing the actual direction of movement. We reshape the array to [n_events, 1] so that we have a 2D array which is expected by MDPax for all problems." ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "class FrozenLake(FrozenLake):\n", " def _construct_random_event_space(self) -> RandomEventSpace:\n", " \"\"\"Build an array of all possible random events.\n", "\n", " Returns:\n", " Array of shape [n_events, event_dim] containing all possible random events\n", " \"\"\"\n", " return jnp.arange(4).reshape(-1, 1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Problem characteristics do not change the random event space" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Array([[0],\n", " [1],\n", " [2],\n", " [3]], dtype=int32)" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "problem = FrozenLake(map_name=\"4x4\", is_slippery=True)\n", "problem._construct_random_event_space()" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Array([[0],\n", " [1],\n", " [2],\n", " [3]], dtype=int32)" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "problem = FrozenLake(map_name=\"8x8\", is_slippery=True)\n", "problem._construct_random_event_space()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1.6 The `random_event_probability` method" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This method returns the probability of a specific random event when in a state and taking an action. \n", "\n", "When used with an MDPAX `Solver` class, both `random_event_probability` and `transition` will be JIT compiled and transformed by JAX's `vmap`/`pmap` transformations to efficiently run on GPUs. This means they must be written using JAX array operations and avoid, for example, Python control flow which does not work with JIT. See the tutorials in the [JAX documention](https://jax.readthedocs.io/en/latest/tutorials.html) for more information, specifically the sections on [JIT compilation](https://jax.readthedocs.io/en/latest/jit-compilation.html) and ['The Sharp Bits'](https://jax.readthedocs.io/en/latest/notebooks/Common_Gotchas_in_JAX.html) for common issues. \n", "\n", "In this case, the probability depends on whether the lake is slippery or not, so we will write two helper functions - one to use when the lake is slippery and one when it isn't - to simplify the logic." ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "class FrozenLake(FrozenLake):\n", "\n", " def random_event_probability(\n", " self,\n", " state: StateVector,\n", " action: ActionVector,\n", " random_event: RandomEventVector\n", " ) -> float:\n", " \"\"\"Calculate probability of random event given state-action pair.\n", "\n", " Args:\n", " state: Current state vector [state_dim]\n", " action: Action vector [action_dim]\n", " random_event: Random event vector [event_dim]\n", "\n", " Returns:\n", " Probability of the random event occurring\n", " \"\"\"\n", " # Instead of using Python control flow, we use JAX's lax.cond function to\n", " # select the correct function to call based on whether the lake is slippery or not.\n", " # This takes the form jnp.lax.cond(condition, true_fn, false_fn, *operands)\n", " \n", " return jax.lax.cond(self.is_slippery, \n", " self._random_event_probability_slippery,\n", " self._random_event_probability_not_slippery, state, action, random_event)\n", "\n", "\n", " def _random_event_probability_slippery(self, state: StateVector, \n", " action: ActionVector, \n", " random_event: RandomEventVector) -> float:\n", " \"\"\"Calculate probability of random event given state-action pair when the lake is slippery.\"\"\"\n", " intended_direction = action[0]\n", " opposite_direction = (intended_direction + 2) % 4\n", " # 1/3 prob of each direction except opposite intended\n", "\n", " # jnp.where is like numpy.where but for JAX arrays\n", " # creates an array of 4 elements, 1/3 for each direction except opposite intended\n", " probs = jnp.where(jnp.arange(4) == opposite_direction, 0, 1/3)\n", " return probs[random_event[0]]\n", "\n", " def _random_event_probability_not_slippery(self, state: StateVector, \n", " action: ActionVector, \n", " random_event: RandomEventVector) -> float:\n", " \"\"\"Calculate probability of random event given state-action pair when the lake is not slippery.\"\"\"\n", " # Probability of intended direction is 1.0\n", " return jnp.where(action[0] == random_event[0], 1.0, 0.0)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Try when the lake is slippery" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Try to go left, get left: 0.33\n", "Try to go left, get right: 0.00\n", "Try to go left, get up: 0.33\n", "Try to go left, get down: 0.33\n" ] } ], "source": [ "problem = FrozenLake(map_name=\"4x4\", is_slippery=True)\n", "print(f\"Try to go left, get left: {problem.random_event_probability(state=jnp.array([0, 0]), action=jnp.array([0]), random_event=jnp.array([0])):.2f}\")\n", "print(f\"Try to go left, get right: {problem.random_event_probability(state=jnp.array([0, 0]), action=jnp.array([0]), random_event=jnp.array([2])):.2f}\")\n", "print(f\"Try to go left, get up: {problem.random_event_probability(state=jnp.array([0, 0]), action=jnp.array([0]), random_event=jnp.array([1])):.2f}\")\n", "print(f\"Try to go left, get down: {problem.random_event_probability(state=jnp.array([0, 0]), action=jnp.array([0]), random_event=jnp.array([3])):.2f}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And when it isn't slippery:" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Try to go left, get left: 1.00\n", "Try to go left, get right: 0.00\n", "Try to go left, get up: 0.00\n", "Try to go left, get down: 0.00\n" ] } ], "source": [ "problem = FrozenLake(map_name=\"4x4\", is_slippery=False)\n", "print(f\"Try to go left, get left: {problem.random_event_probability(state=jnp.array([0, 0]), action=jnp.array([0]), random_event=jnp.array([0])):.2f}\")\n", "print(f\"Try to go left, get right: {problem.random_event_probability(state=jnp.array([0, 0]), action=jnp.array([0]), random_event=jnp.array([2])):.2f}\")\n", "print(f\"Try to go left, get up: {problem.random_event_probability(state=jnp.array([0, 0]), action=jnp.array([0]), random_event=jnp.array([1])):.2f}\")\n", "print(f\"Try to go left, get down: {problem.random_event_probability(state=jnp.array([0, 0]), action=jnp.array([0]), random_event=jnp.array([3])):.2f}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1.7 The `transition` method" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `transition` method is deterministic because any random elements in a transition are specified by the random event argument. It returns the next state and the reward when we take a specified action in a specified state and stochastic elements resolve to the specified random event. \n", "\n", "As with `random_event_probability`, this needs to be written using JAX array operations so that it can be transformed using JAX's `vmap` and `pmap` transformations and JIT compiled. We therefore use `jnp.where` instead of Python if statements for control flow.\n", "\n", "In this problem, we have terminal states (the goal and the holes in the ice). Once the agent reaches a terminal state it stays there forever." ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [], "source": [ "class FrozenLake(FrozenLake):\n", " def transition(\n", " self,\n", " state: StateVector,\n", " action: ActionVector,\n", " random_event: RandomEventVector\n", " ) -> tuple[StateVector, Reward]:\n", " \"\"\"Compute next state and reward for a transition.\n", "\n", " Args:\n", " state: Current state vector [state_dim]\n", " action: Action vector [action_dim]\n", " random_event: Random event vector [event_dim]\n", "\n", " Returns:\n", " Tuple containing the next state vector [state_dim] and\n", " the immediate reward\n", " \"\"\"\n", " \n", " row, col = state[0], state[1]\n", " is_terminal_state = (self.grid[row, col] == 1) | (self.grid[row, col] == 2) # Goal or hole\n", " \n", " # Direction we actually move defined by random event\n", " actual_direction = random_event[0]\n", "\n", " # Calculate next row - move down or up\n", " # If we are at top or bottom of grid, stay in current row\n", " next_row = jnp.where(\n", " actual_direction == 1, # DOWN\n", " jnp.minimum(row + 1, self.n_rows - 1),\n", " jnp.where(\n", " actual_direction == 3, # UP\n", " jnp.maximum(row - 1, 0),\n", " row # No change for LEFT/RIGHT\n", " )\n", " )\n", " \n", " # Calculate next column - move left or right\n", " # If we are at left or right of grid, stay in current column\n", " next_col = jnp.where(\n", " actual_direction == 0, # LEFT\n", " jnp.maximum(col - 1, 0),\n", " jnp.where(\n", " actual_direction == 2, # RIGHT\n", " jnp.minimum(col + 1, self.n_cols - 1),\n", " col # No change for UP/DOWN\n", " )\n", " )\n", " \n", " # Construct next state\n", " next_state = jnp.array([next_row, next_col])\n", " \n", " # Stay in current state if terminal\n", " next_state = jnp.where(\n", " is_terminal_state,\n", " state,\n", " next_state\n", " )\n", " \n", " # Calculate reward (1 only when transitioning TO goal, not for staying in goal)\n", " reward = jnp.where(\n", " is_terminal_state,\n", " 0.0, # No reward in terminal states\n", " jnp.where(self.grid[next_row, next_col] == 2, 1.0, 0.0) # 1.0 for reaching goal\n", " )\n", " \n", " return next_state, reward" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [], "source": [ "problem = FrozenLake(map_name=\"4x4\", is_slippery=True)" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Moving left from start, next state: [0 0], reward: 0.0\n" ] } ], "source": [ "next_state, reward = problem.transition(\n", " state=jnp.array([0, 0]), action=jnp.array([0]), random_event=jnp.array([0]))\n", "print(f\"Moving left from start, next state: {next_state}, reward: {reward}\")" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Moving right into goal, next state [3 3], reward: 1.0\n" ] } ], "source": [ "next_state, reward = problem.transition(\n", " state=jnp.array([3, 2]), action=jnp.array([2]), random_event=jnp.array([2]))\n", "print(f\"Moving right into goal, next state {next_state}, reward: {reward}\")" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Trying to move from a hole, next state: [2 3], reward: 0.0\n" ] } ], "source": [ "next_state, reward = problem.transition(\n", " state=jnp.array([2, 3]), action=jnp.array([1]), random_event=jnp.array([1]))\n", "print(f\"Trying to move from a hole, next state: {next_state}, reward: {reward}\")" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [], "source": [ "problem = FrozenLake(map_name=\"4x4\", is_slippery=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2 Putting it all together" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here is the full class definition for the `FrozenLake` problem." ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [], "source": [ "class FrozenLake(Problem):\n", " \"\"\"FrozenLake MDP from OpenAI Gym.\n", "\n", " Models navigation on a grid world with slippery movement.\n", " \n", " The agent must navigate from start (S) to goal (G) on a frozen lake surface (F)\n", " where the surface is slippery and there are holes (H) that end the episode. \n", " \n", " Example 4x4 map:\n", " SFFF\n", " FHFH\n", " FFFH\n", " HFFG\n", "\n", " State Space (state_dim = 2):\n", " Vector containing:\n", " - Row: 1 element in range [0, n_rows-1]\n", " - Column: 1 element in range [0, n_cols-1]\n", "\n", " Action Space (action_dim = 1):\n", " Vector containing:\n", " - Intended movement direction: 1 element in range [0, 3]\n", " where:\n", " - 0: LEFT\n", " - 1: DOWN\n", " - 2: RIGHT\n", " - 3: UP\n", "\n", " Random Events (event_dim = 1):\n", " Vector containing:\n", " - Actual movement direction: 1 element in range [0, 3]\n", "\n", " Dynamics:\n", " 1. The agent chooses an intended movement direction\n", " 2. The agent moves in the actual direction.\n", " - If the surface is not slippery, the agent moves in the intended direction.\n", " - If the surface is slippery, the agent has a 1/3 probability of moving\n", " in the intended direction, a 1/3 probability of moving in the \n", " direction to the left of the intended direction, and a 1/3 \n", " probability of moving in the direction to the right of the intended \n", " direction.\n", " 3. The episode ends when the agent reaches the goal or falls into a hole\n", " \n", " Args:\n", " desc: Custom map layout as list of strings\n", " map_name: Key for a map in the dictionary of known maps `MAPS`\n", " is_slippery: If True, the movement direction is stochastic\n", " \"\"\"\n", " def __init__(\n", " self, \n", " desc: list[str] | None = None,\n", " map_name: str = \"4x4\",\n", " is_slippery: bool = True,\n", " ):\n", " self.desc = desc\n", " self.map_name = map_name\n", " self.is_slippery = is_slippery\n", "\n", " # Use a custom map if provided\n", " if desc is not None:\n", " self.map = desc\n", " else:\n", " self.map = MAPS[map_name]\n", "\n", " # Convert map to array for efficient lookup\n", " # Needs to be numeric so use 1 for hole and 2 for goal\n", " # 0 for frozen surface\n", " self.grid = jnp.array([\n", " [1 if c == 'H' else 2 if c == 'G' else 0 \n", " for c in row]\n", " for row in self.map\n", " ])\n", " self.n_rows, self.n_cols = self.grid.shape\n", " \n", " super().__init__()\n", " \n", " @property\n", " def name(self) -> str:\n", " \"\"\"A unique identifier for this problem type\"\"\"\n", " s = \"slippery\" if self.is_slippery else \"non_slippery\"\n", " if self.desc is not None:\n", " return f\"frozen_lake_custom_{s}\"\n", " else:\n", " return f\"frozen_lake_{self.map_name}_{s}\"\n", " \n", " def _construct_state_space(self) -> StateSpace:\n", " \"\"\"Build array of all possible states.\n", "\n", " Returns:\n", " Array of shape [n_states, state_dim] containing all possible states\n", " \"\"\"\n", " mins = np.zeros(2, dtype=np.int32)\n", " maxs = np.array([self.n_rows-1, self.n_cols-1], dtype=np.int32)\n", " state_space, self._state_to_index_fn = create_range_space(mins, maxs)\n", " return state_space\n", " \n", " \n", " def state_to_index(self, state: StateVector) -> int:\n", " \"\"\"Convert state vector to index.\"\"\"\n", " return self._state_to_index_fn(state)\n", " \n", " def _construct_action_space(self) -> ActionSpace:\n", " \"\"\"Build an array of all possible actions.\n", "\n", " Returns:\n", " Array of shape [n_actions, action_dim] containing all possible actions\n", " \"\"\"\n", " # One action for each direction\n", " return jnp.arange(4).reshape(-1, 1)\n", " \n", " def _construct_random_event_space(self) -> RandomEventSpace:\n", " \"\"\"Build an array of all possible random events.\n", "\n", " Returns:\n", " Array of shape [n_events, event_dim] containing all possible random events\n", " \"\"\"\n", " return jnp.arange(4).reshape(-1, 1)\n", " \n", " def random_event_probability(\n", " self,\n", " state: StateVector,\n", " action: ActionVector,\n", " random_event: RandomEventVector\n", " ) -> float:\n", " \"\"\"Calculate probability of random event given state-action pair.\n", "\n", " Args:\n", " state: Current state vector [state_dim]\n", " action: Action vector [action_dim]\n", " random_event: Random event vector [event_dim]\n", "\n", " Returns:\n", " Probability of the random event occurring\n", " \"\"\"\n", " # Instead of using Python control flow, we use JAX's lax.cond function to\n", " # select the correct function to call based on whether the lake is slippery or not.\n", " # This takes the form jnp.lax.cond(condition, true_fn, false_fn, *operands)\n", " \n", " return jax.lax.cond(self.is_slippery, \n", " self._random_event_probability_slippery,\n", " self._random_event_probability_not_slippery, state, action, random_event)\n", "\n", "\n", " def _random_event_probability_slippery(self, state: StateVector, \n", " action: ActionVector, \n", " random_event: RandomEventVector) -> float:\n", " \"\"\"Calculate probability of random event given state-action pair when the lake is slippery.\"\"\"\n", " intended_direction = action[0]\n", " opposite_direction = (intended_direction + 2) % 4\n", " # 1/3 prob of each direction except opposite intended\n", "\n", " # jnp.where is like numpy.where but for JAX arrays\n", " # creates an array of 4 elements, 1/3 for each direction except opposite intended\n", " probs = jnp.where(jnp.arange(4) == opposite_direction, 0, 1/3)\n", " return probs[random_event[0]]\n", "\n", " def _random_event_probability_not_slippery(self, state: StateVector, \n", " action: ActionVector, random_event: \n", " RandomEventVector) -> float:\n", " \"\"\"Calculate probability of random event given state-action pair when the lake is not slippery.\"\"\"\n", " # Probability of intended direction is 1.0\n", " return jnp.where(action[0] == random_event[0], 1.0, 0.0)\n", "\n", " def transition(\n", " self,\n", " state: StateVector,\n", " action: ActionVector,\n", " random_event: RandomEventVector\n", " ) -> tuple[StateVector, Reward]:\n", " \"\"\"Compute next state and reward for a transition.\n", "\n", " Args:\n", " state: Current state vector [state_dim]\n", " action: Action vector [action_dim]\n", " random_event: Random event vector [event_dim]\n", "\n", " Returns:\n", " Tuple containing the next state vector [state_dim] and\n", " the immediate reward\n", " \"\"\"\n", " \n", " row, col = state[0], state[1]\n", " is_terminal_state = (self.grid[row, col] == 1) | (self.grid[row, col] == 2) # Goal or hole\n", " \n", " # Direction we actually move defined by random event\n", " actual_direction = random_event[0]\n", "\n", " # Calculate next row - move down or up\n", " # If we are at top or bottom of grid, stay in current row\n", " next_row = jnp.where(\n", " actual_direction == 1, # DOWN\n", " jnp.minimum(row + 1, self.n_rows - 1),\n", " jnp.where(\n", " actual_direction == 3, # UP\n", " jnp.maximum(row - 1, 0),\n", " row # No change for LEFT/RIGHT\n", " )\n", " )\n", " \n", " # Calculate next column - move left or right\n", " # If we are at left or right of grid, stay in current column\n", " next_col = jnp.where(\n", " actual_direction == 0, # LEFT\n", " jnp.maximum(col - 1, 0),\n", " jnp.where(\n", " actual_direction == 2, # RIGHT\n", " jnp.minimum(col + 1, self.n_cols - 1),\n", " col # No change for UP/DOWN\n", " )\n", " )\n", " \n", " # Construct next state\n", " next_state = jnp.array([next_row, next_col])\n", " \n", " # Stay in current state if terminal\n", " next_state = jnp.where(\n", " is_terminal_state,\n", " state,\n", " next_state\n", " )\n", " \n", " # Calculate reward (1 only when transitioning TO goal, not for staying in goal)\n", " reward = jnp.where(\n", " is_terminal_state,\n", " 0.0, # No reward in terminal states\n", " jnp.where(self.grid[next_row, next_col] == 2, 1.0, 0.0) # 1.0 for reaching goal\n", " )\n", " \n", " return next_state, reward" ] }, { "cell_type": "markdown", "metadata": { "id": "solving_intro" }, "source": [ "## 3 Solving the Problem\n", "\n", "Now that we have our problem implementation, let's solve it using value iteration." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3.1 Non-slippery case" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "id": "solving" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m2025-01-05 22:05:56.574\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.core.solver\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m159\u001b[0m - \u001b[1mSolver initialized with frozen_lake_4x4_non_slippery problem\u001b[0m\n", "\u001b[32m2025-01-05 22:05:56.718\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.utils.checkpointing\u001b[0m:\u001b[36m_setup_checkpointing\u001b[0m:\u001b[36m123\u001b[0m - \u001b[1mCheckpointing not enabled\u001b[0m\n", "\u001b[32m2025-01-05 22:05:57.389\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 1 span: 1.00000\u001b[0m\n", "\u001b[32m2025-01-05 22:05:57.544\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 2 span: 0.90000\u001b[0m\n", "\u001b[32m2025-01-05 22:05:57.552\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 3 span: 0.81000\u001b[0m\n", "\u001b[32m2025-01-05 22:05:57.556\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 4 span: 0.72900\u001b[0m\n", "\u001b[32m2025-01-05 22:05:57.561\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 5 span: 0.65610\u001b[0m\n", "\u001b[32m2025-01-05 22:05:57.565\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 6 span: 0.59049\u001b[0m\n", "\u001b[32m2025-01-05 22:05:57.569\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 7 span: 0.00000\u001b[0m\n", "\u001b[32m2025-01-05 22:05:57.571\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m502\u001b[0m - \u001b[1mConvergence threshold reached at iteration 7\u001b[0m\n", "\u001b[32m2025-01-05 22:05:57.591\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m521\u001b[0m - \u001b[1mExtracting policy\u001b[0m\n", "\u001b[32m2025-01-05 22:05:57.834\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m523\u001b[0m - \u001b[1mPolicy extracted\u001b[0m\n", "\u001b[32m2025-01-05 22:05:57.835\u001b[0m | \u001b[32m\u001b[1mSUCCESS \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m525\u001b[0m - \u001b[32m\u001b[1mValue iteration completed\u001b[0m\n" ] } ], "source": [ "from mdpax.solvers.value_iteration import ValueIteration\n", "\n", "# Create problem instance\n", "problem = FrozenLake(map_name=\"4x4\", is_slippery=False)\n", "\n", "# Create and run solver\n", "solver = ValueIteration(problem, gamma=0.9, epsilon=1e-3)\n", "solution = solver.solve()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And, because this is a small problem, we can construct the transition and reward matrices and compare the result to pymdptoolbox." ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MDPax and mdptoolbox policies match: True\n" ] } ], "source": [ "P, R = problem.build_transition_and_reward_matrices()\n", "P, R = np.array(P), np.array(R)\n", "vi = mdptoolbox.mdp.ValueIteration(P, R, discount=0.9, epsilon=1e-3)\n", "vi.run()\n", "print(f\"MDPax and mdptoolbox policies match: {np.all(np.array(vi.policy) == solution.policy.flatten())}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Finally, we can visualize the policy to check that it makes sense." ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [], "source": [ "def plot_policy(problem: FrozenLake, policy: Policy) -> None:\n", " \"\"\"Visualize FrozenLake policy.\n", " \n", " Args:\n", " problem: FrozenLake problem instance\n", " policy: Policy array [n_states, 1] with action indices\n", "\n", " Returns:\n", " None\n", " \"\"\"\n", " # Create figure\n", " fig, ax = plt.subplots(figsize=(8, 8))\n", " \n", " # Plot grid\n", " ax.grid(True)\n", " ax.set_xticks(range(problem.n_cols + 1))\n", " ax.set_yticks(range(problem.n_rows + 1))\n", " \n", " # Plot cell types\n", " cell_colors = {\n", " 'S': 'lightgreen',\n", " 'F': 'lightblue',\n", " 'H': 'red',\n", " 'G': 'gold'\n", " }\n", " \n", " for i in range(problem.n_rows):\n", " for j in range(problem.n_cols):\n", " cell = problem.map[i][j]\n", " ax.add_patch(plt.Rectangle(\n", " (j, problem.n_rows-1-i),\n", " 1, 1,\n", " facecolor=cell_colors[cell],\n", " alpha=0.3\n", " ))\n", " ax.text(\n", " j+0.5, problem.n_rows-1-i+0.5,\n", " cell,\n", " ha='center', va='center'\n", " )\n", " \n", " # Plot policy arrows\n", " arrows = {0: 'โ†', 1: 'โ†“', 2: 'โ†’', 3: 'โ†‘'}\n", " \n", " for state_idx, action in enumerate(policy):\n", " state = problem.state_space[state_idx]\n", " row, col = state[0], state[1]\n", " \n", " # Skip arrows in terminal states\n", " if (problem.grid[row, col] == 1) | (problem.grid[row, col] == 2):\n", " continue\n", " \n", " ax.text(\n", " col+0.5, problem.n_rows-1-row+0.2,\n", " arrows[int(action[0])],\n", " ha='center', va='center',\n", " color='black',\n", " fontsize=20\n", " )\n", " \n", " ax.set_title('FrozenLake Policy\\n(arrows show optimal actions)')\n", " plt.show()" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAApYAAAK+CAYAAAAYDV5RAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAARTNJREFUeJzt3Xl4U2X+v/F3utOVpa20IFtRUAEZURFkk1V2FBERpKAiyiLYUZYZAUEcVBSRraLDogiC4DqgYhHKJioFUQRBZdBxZSnQQoFC0+f3hz/yNbZCW54mbXq/rotrJqc5OZ+Ex/TmJE0dxhgjAAAA4CL5eXsAAAAA+AbCEgAAAFYQlgAAALCCsAQAAIAVhCUAAACsICwBAABgBWEJAAAAKwhLAAAAWEFYAgAAwArCEgAsadWqlerVq+ftMQps4cKFcjgc+v77713bWrVqpVatWnltJgClG2EJ4ILOBUh+f8aMGePt8S6Kw+HQsGHDvD1Ggf3xsffz81N8fLzat2+v1NRUb48GAArw9gAASo9JkyapZs2abttK0xk6X9GuXTv1799fxhjt379fc+bMUevWrbVq1Sp17Njxom77ww8/tDQlgLKIsARQYB07dtS1115boOuePn1aQUFB8vPjhRHbLr/8cvXr1891+ZZbblGDBg00ffr0iw7LoKCgix0PQBnGMz6Ai5aamiqHw6GlS5fq0UcfVZUqVRQaGqrMzExJ0vLly9WoUSOVK1dO0dHR6tevn37++ec8++f3p0aNGm7Hev/999W8eXOFhYUpIiJCnTt31q5du9yuM2DAAIWHh+vnn39Wjx49FB4erpiYGD388MNyOp2Fvn/vvPOOOnfurPj4eAUHByshIUGPP/54gW7rww8/VGhoqPr06aOcnBxJ0p49e3TbbbepYsWKCgkJ0bXXXqt333230HOdU79+fUVHR2v//v2ubWvXrnU9TuXLl1f37t319ddfX/C28nuP5enTp/XYY4/p8ssvV0hIiOLi4nTrrbdq3759MsaoRo0a6t69e57bOn36tKKiojR48OAi3zcApQtnLAEUWEZGhg4fPuy2LTo62vX/H3/8cQUFBenhhx9Wdna2goKCtHDhQg0cOFDXXXedpkyZogMHDuj555/X5s2b9fnnn6t8+fK64oortGjRIrfbPXbsmJKSkhQbG+vatmjRIiUmJqpDhw566qmndPLkSSUnJ6tZs2b6/PPP3SLU6XSqQ4cOaty4sZ555hmtWbNGzz77rBISEvTAAw8U6n4vXLhQ4eHhSkpKUnh4uNauXavx48crMzNTU6dO/cv9Vq5cqdtuu029e/fW/Pnz5e/vr127dunGG29UlSpVNGbMGIWFhen1119Xjx499MYbb+iWW24p1GySdPToUR09elS1a9eWJK1Zs0YdO3ZUrVq19Nhjj+nUqVOaOXOmbrzxRm3fvj1PrJ+P0+lUly5d9NFHH+mOO+7QiBEjdPz4caWkpOirr75SQkKC+vXrp6efflpHjhxRxYoVXfv+5z//UWZmptvZVQA+zgDABSxYsMBIyvePMcasW7fOSDK1atUyJ0+edO135swZExsba+rVq2dOnTrl2r5y5UojyYwfPz7f4+Xm5pouXbqY8PBws2vXLmOMMcePHzfly5c3gwYNcrvub7/9ZqKioty2JyYmGklm0qRJbtf929/+Zho1auS2TZIZOnToee//H+/TOYMHDzahoaHm9OnTrm0tW7Y0V111lTHGmDfeeMMEBgaaQYMGGafT6bpOmzZtTP369d32y83NNU2bNjWXXXbZeec4N+8999xjDh06ZA4ePGg+/fRT06ZNGyPJPPvss8YYYxo2bGhiY2NNenq6a78vvvjC+Pn5mf79+7u2nft73b9/v9t9aNmypevy/PnzjSQzbdq0PLPk5uYaY4zZu3evkWSSk5Pdvt6tWzdTo0YN1/UA+D5eCgdQYLNnz1ZKSorbnz9KTExUuXLlXJfT0tJ08OBBDRkyRCEhIa7tnTt3Vt26dbVq1ap8j/P4449r5cqVWrhwoa688kpJUkpKio4dO6Y+ffro8OHDrj/+/v5q3Lix1q1bl+d27r//frfLzZs313//+99C3+8/3qfjx4/r8OHDat68uU6ePKk9e/bkuf5rr72m3r17a/DgwZo7d67rfaZHjhzR2rVrdfvtt7tu5/Dhw0pPT1eHDh307bffur1F4K/MmzdPMTExio2NVePGjbV582YlJSVp5MiR+vXXX7Vjxw4NGDDA7exhgwYN1K5dO7333nuFuu9vvPGGoqOjNXz48Dxfczgckn5/z2fjxo21ePFi19eOHDmi999/X3379nVdD4Dv46VwAAV2/fXXn/eHd/78E+M//PCDJKlOnTp5rlu3bl1t2rQpz/YPPvhAEydO1NixY9WzZ0/X9m+//VaS1Lp163yPHRkZ6XY5JCREMTExbtsqVKigo0eP/uX8f2XXrl169NFHtXbtWtf7Rs/JyMhwu7x//37169dPvXr10syZM92+9t1338kYo3HjxmncuHH5HuvgwYOqUqXKeefp3r27hg0bJofDoYiICF111VUKCwuTdP7H/IorrtDq1auVlZXluv6F7Nu3T3Xq1FFAwPm/XfTv31/Dhg3TDz/8oOrVq2v58uU6e/as7rrrrgIdB4BvICwBWPPHM3tFsX//fvXt21ft2rXT5MmT3b6Wm5sr6ff3WVauXDnPvn8OH39//4ua5Zxjx46pZcuWioyM1KRJk5SQkKCQkBBt375do0ePds11TlxcnOLi4vTee+8pLS3NLcTPXffhhx9Whw4d8j3eufdJnk/VqlXVtm3bi7hX9t1xxx166KGHtHjxYv3jH//Qq6++qmuvvTbfwAXguwhLAMWmevXqkqS9e/fmOdO4d+9e19cl6dSpU7r11ltVvnx5vfbaa3k+pighIUGSFBsb69GoSk1NVXp6ut588021aNHCtf2PP4H9RyEhIVq5cqVat26tm2++WevXr9dVV10lSapVq5YkKTAwsNjuwx8f8z/bs2ePoqOjC3y2Uvr9cf/000919uxZBQYG/uX1KlasqM6dO2vx4sXq27evNm/erOnTpxd6fgClG++xBFBsrr32WsXGxuqFF15Qdna2a/v777+vr7/+Wp07d3Ztu//++/XNN9/orbfeUoUKFfLcVocOHRQZGal//etfOnv2bJ6vHzp0qFjuw7kzn8YY17YzZ85ozpw5f7lPVFSUVq9erdjYWLVr10779u2T9HsUt2rVSnPnztWvv/6aZz8b9yEuLk4NGzbUyy+/rGPHjrm2f/XVV/rwww/VqVOnQt1ez549dfjwYc2aNSvP1/74mEjSXXfdpd27d+uRRx6Rv7+/7rjjjiLdBwClF2csARSbwMBAPfXUUxo4cKBatmypPn36uD5uqEaNGnrooYckSatWrdIrr7yinj176ssvv9SXX37puo3w8HD16NFDkZGRSk5O1l133aVrrrlGd9xxh2JiYvS///1Pq1at0o033phv/BREWlpanpfepd8/07Fp06aqUKGCEhMT9eCDD8rhcGjRokV5ourPoqOjlZKSombNmqlt27batGmTqlSpotmzZ6tZs2aqX7++Bg0apFq1aunAgQPasmWLfvrpJ33xxRdFug9/NHXqVHXs2FFNmjTRPffc4/q4oaioKD322GOFuq3+/fvrlVdeUVJSkj777DM1b95cWVlZWrNmjYYMGeL2+ZWdO3dWpUqVtHz5cnXs2NHto6IAlBHe/aF0AKXBuY+l2bp1a75fP/dxQ8uXL8/368uWLTN/+9vfTHBwsKlYsaLp27ev+emnn/Lcfn5/qlevnudYHTp0MFFRUSYkJMQkJCSYAQMGmLS0NNd1EhMTTVhYWJ45JkyYYP78tPdXx5VkHn/8cWOMMZs3bzY33HCDKVeunImPjzejRo0yq1evNpLMunXrXLf1x48bOue7774zcXFx5oorrjCHDh0yxhizb98+079/f1O5cmUTGBhoqlSpYrp06WJWrFiR7+P353kv9PFIxhizZs0ac+ONN5py5cqZyMhI07VrV7N792636xTk44aM+f3jlv75z3+amjVrmsDAQFO5cmVz2223mX379uU57pAhQ4wks2TJkgvOCMD3OIy5wD+7AQAooIceekjz5s3Tb7/9ptDQUG+PA8DDeI8lAMCK06dP69VXX1XPnj2JSqCM4j2WAICLcvDgQa1Zs0YrVqxQenq6RowY4e2RAHgJYQkAuCi7d+9W3759FRsbqxkzZqhhw4beHgmAl/AeSwAAAFjBeywBAABgBWEJAAAAKwhLoJg9/fTTqlu3bp7fKQ13CxculMPhUFpamrdH8boBAwaoRo0aZe7Y+XE4HIX+UHfb0tPTFRYWpvfee8+rcwClAWEJFKPMzEw99dRTGj16dJ7ffY2y7ZdfftFjjz2mHTt2eHsUr3vvvfe8Ho/nU6lSJd17770aN26ct0cBSjy+0wHFaP78+crJyVGfPn28PQpKmF9++UUTJ07MNyxfeukl7d271/NDecl7772niRMn5vu1U6dO6dFHH/XwRHndf//92r59u9auXevtUYASjbAEitGCBQvUrVs3hYSEWLm93NxcnT59Ot+vZWVlWTkGvC8wMFDBwcHeHqNECAkJUUCA9z8Z74orrlC9evW0cOFCb48ClGiEJVBM9u/fry+//FJt27bN87VnnnlGTZs2VaVKlVSuXDk1atRIK1asyHM9h8OhYcOGafHixbrqqqsUHBysDz74wPV+xPXr12vIkCGKjY1V1apVXfvNmTPHdf34+HgNHTpUx44dc319xowZ8vf3d9v27LPPyuFwKCkpybXN6XQqIiJCo0ePdm1bunSpGjVqpIiICEVGRqp+/fp6/vnnL/h4FHS/7OxsJSUlKSYmRmFhYbrlllt06NChPNcrzvv4Vy50TElq1aqV6tWrp23btqlp06YqV66catasqRdeeMF1ndTUVF133XWSpIEDB8rhcMjhcLii5c/vc/z+++/lcDj0zDPPaPbs2apVq5ZCQ0PVvn17/fjjjzLG6PHHH1fVqlVVrlw5de/eXUeOHHGb65133lHnzp0VHx+v4OBgJSQk6PHHH5fT6bzg/c5PYW7v008/VadOnVShQgWFhYWpQYMGrr/7AQMGaPbs2ZLkehwcDodr3/zeY/n555+rY8eOioyMVHh4uNq0aaNPPvnE7Trn/hvZvHnzBddTWlqaOnTooOjoaNff1913353nfrRr107/+c9/xKf0AX/N+/8MBHzUxx9/LEm65ppr8nzt+eefV7du3dS3b1+dOXNGS5cuVa9evbRy5Up17tzZ7bpr167V66+/rmHDhik6Olo1atRwvXw6ZMgQxcTEaPz48a4zlo899pgmTpyotm3b6oEHHtDevXuVnJysrVu3avPmzQoMDFTz5s2Vm5urTZs2qUuXLpKkjRs3ys/PTxs3bnQd+/PPP9eJEyfUokULSVJKSor69OmjNm3a6KmnnpIkff3119q8efN5f9tKYfYbPny4KlSooAkTJuj777/X9OnTNWzYMC1btsx1neK8j3+lIMc85+jRo+rUqZNuv/129enTR6+//roeeOABBQUF6e6779YVV1yhSZMmafz48brvvvvUvHlzSVLTpk3PO8PixYt15swZDR8+XEeOHNHTTz+t22+/Xa1bt1ZqaqpGjx6t7777TjNnztTDDz+s+fPnu/ZduHChwsPDlZSUpPDwcK1du1bjx49XZmampk6det7j5qegt5eSkqIuXbooLi5OI0aMUOXKlfX1119r5cqVGjFihAYPHqxffvlFKSkpWrRo0QWPu2vXLjVv3lyRkZEaNWqUAgMDNXfuXLVq1Urr169X48aN3a5/ofV08OBBtW/fXjExMRozZozKly+v77//Xm+++WaeYzdq1EjPPfecdu3apXr16hX6MQPKBAOgWDz66KNGkjl+/Hier508edLt8pkzZ0y9evVM69at3bZLMn5+fmbXrl1u2xcsWGAkmWbNmpmcnBzX9oMHD5qgoCDTvn1743Q6XdtnzZplJJn58+cbY4xxOp0mMjLSjBo1yhhjTG5urqlUqZLp1auX8ff3d808bdo04+fnZ44ePWqMMWbEiBEmMjLS7ZgFUZD9zt2ntm3bmtzcXNf2hx56yPj7+5tjx4555D7mp6DHNMaYli1bGknm2WefdW3Lzs42DRs2NLGxsebMmTPGGGO2bt1qJJkFCxbkOV5iYqKpXr266/L+/fuNJBMTE+N6HIwxZuzYsUaSufrqq83Zs2dd2/v06WOCgoLM6dOnXdv+vOaMMWbw4MEmNDTU7Xp/PvZfKcjt5eTkmJo1a5rq1avneXz/+Hc8dOhQ81ffjiSZCRMmuC736NHDBAUFmX379rm2/fLLLyYiIsK0aNHCta2g6+mtt94ykszWrVsveJ8//vhjI8ksW7bsgtcFyipeCgeKSXp6ugICAhQeHp7na+XKlXP9/6NHjyojI0PNmzfX9u3b81y3ZcuWuvLKK/M9xqBBg+Tv7++6vGbNGp05c0YjR450+yn0QYMGKTIyUqtWrZIk+fn5qWnTptqwYYOk388epqena8yYMTLGaMuWLZJ+P8NXr149lS9fXpJUvnx5ZWVlKSUlpVCPRWH2u++++9xeCm3evLmcTqd++OEHj9zH/BT0mOcEBARo8ODBrstBQUEaPHiwDh48qG3btl3wMfgrvXr1UlRUlOvyubNz/fr1c3sfYuPGjXXmzBn9/PPPrm1/XHPHjx/X4cOH1bx5c508eVJ79uwp9CwFub3PP/9c+/fv18iRI/M8vn/8Oy4op9OpDz/8UD169FCtWrVc2+Pi4nTnnXdq06ZNyszMdNvnQuvp3FwrV67U2bNnz3v8ChUqSJIOHz5c6NmBsoKwBLxg5cqVuuGGGxQSEqKKFSsqJiZGycnJysjIyHPdmjVr/uXt/Plr575Z1qlTx217UFCQatWq5fq69Ps32G3btunUqVPauHGj4uLidM011+jqq692vVS8adMm18u00u8vvV9++eXq2LGjqlatqrvvvlsffPDBBe9vYfarVq2a2+Vz38yPHj3qkfuYn8IcU5Li4+MVFhbmtu3yyy+X9Pv7JYvqz4/Nuci89NJL891+7jGTfn8J+ZZbblFUVJQiIyMVExOjfv36SVK+6+5CCnJ7+/btkyRrLxsfOnRIJ0+ezPP3IP3+wzW5ubn68ccf3bZfaD21bNlSPXv21MSJExUdHa3u3btrwYIFys7OznMM8//fW1mUKAbKCsISKCaVKlVSTk6Ojh8/7rZ948aNrp8UnzNnjt577z2lpKTozjvvzPeHAv54ZqgwX7uQZs2a6ezZs9qyZYs2btzoiqvmzZtr48aN2rNnjw4dOuQWXbGxsdqxY4feffdddevWTevWrVPHjh2VmJh43mMVZr8/noH9o/wem+K4jyXZXz02F3rMjh07ppYtW+qLL77QpEmT9J///EcpKSmu97sW9sP7bd9ecbrQY+NwOLRixQpt2bJFw4YN088//6y7775bjRo10okTJ9z2ORej0dHRxTs0UIoRlkAxqVu3rqTffzr8j9544w2FhIRo9erVuvvuu9WxY8d8f3K8KKpXry5JeT4D8cyZM9q/f7/r65J0/fXXKygoSBs3bnSLrhYtWujTTz/VRx995Lr8R0FBQeratavmzJmjffv2afDgwXrllVf03XffnXe2ou7njft4MceUfv+Myj9//NM333wjSa6f9vbkWa/U1FSlp6dr4cKFGjFihLp06aK2bdu6zt4V1+0lJCRIkr766qvz3l5BH4uYmBiFhobm+xmfe/bskZ+fX56ztwV1ww036IknnlBaWpoWL16sXbt2aenSpW7XOfff8hVXXFGkYwBlAWEJFJMmTZpIUp5fUejv7y+Hw+H2sSzff/+93n777Ys+Ztu2bRUUFKQZM2a4neGbN2+eMjIy3H7iPCQkRNddd51ee+01/e9//3M7m3fq1CnNmDFDCQkJiouLc+2Tnp7udjw/Pz81aNBAkvJ96fBi9/PGfbzYY0pSTk6O5s6d67p85swZzZ07VzExMWrUqJEkuV4q//PHFRWHc2ft/jj7mTNnNGfOnGK9vWuuuUY1a9bU9OnT89zPP+5b0MfC399f7du31zvvvOP2loIDBw5oyZIlatasmSIjIwt1X44ePZrnbHjDhg0l5V2b27ZtU1RUlK666qpCHQMoS/i4IaCY1KpVS/Xq1dOaNWvcPhOvc+fOmjZtmm6++WbdeeedOnjwoGbPnq3atWvryy+/vKhjxsTEaOzYsZo4caJuvvlmdevWTXv37tWcOXN03XXXud4Dd07z5s315JNPKioqSvXr15f0+8vWderU0d69ezVgwAC369977706cuSIWrdurapVq+qHH37QzJkz1bBhw/OexSnqft64jzaOGR8fr6eeekrff/+9Lr/8ci1btkw7duzQiy++6PpYooSEBJUvX14vvPCCIiIiFBYWpsaNG5/3PbVF1bRpU1WoUEGJiYl68MEH5XA4tGjRoiJ/HmNBb8/Pz0/Jycnq2rWrGjZsqIEDByouLk579uzRrl27tHr1aklyxfaDDz6oDh06yN/fX3fccUe+x548ebJSUlLUrFkzDRkyRAEBAZo7d66ys7P19NNPF/q+vPzyy5ozZ45uueUWJSQk6Pjx43rppZcUGRmpTp06uV03JSVFXbt25T2WwPl4/gfRgbJj2rRpJjw8PM9Hs8ybN89cdtllJjg42NStW9csWLDATJgwIc9HrkgyQ4cOzXO75z5K5a8+ImXWrFmmbt26JjAw0FxyySXmgQceyPfjdFatWmUkmY4dO7ptv/fee40kM2/ePLftK1asMO3btzexsbEmKCjIVKtWzQwePNj8+uuv530cCrLfX92ndevWGUlm3bp1HrmP51OQY7Zs2dJcddVVJi0tzTRp0sSEhISY6tWrm1mzZuW5vXfeecdceeWVJiAgwO2jh/7q44amTp2a72OzfPlyt+35PZabN282N9xwgylXrpyJj483o0aNMqtXr87z2Bb044YKenvGGLNp0ybTrl07ExERYcLCwkyDBg3MzJkzXV/Pyckxw4cPNzExMcbhcLj9d6A/fdyQMcZs377ddOjQwYSHh5vQ0FBz0003mY8//viCj8EfH7NzM27fvt306dPHVKtWzQQHB5vY2FjTpUsXk5aW5rbf119/bSSZNWvWXPCxAcoyhzH8CgGguGRkZKhWrVp6+umndc8993h7HHhAq1atdPjw4Qu+rxCly8iRI7VhwwZt27aNM5bAefAeS6AYRUVFadSoUZo6dWqJ+klZAAWXnp6uf//735o8eTJRCVwAZywBwCLOWAIoyzhjCQAAACsuKiyffPJJORwOjRw50tI4AFC6paamcrYSQJlV5LDcunWr5s6d6/osOgAAAJRtRfocyxMnTqhv37566aWXNHny5PNeNzs72+1DZnNzc3XkyBFVqlSJN0EDAACUQMYYHT9+XPHx8fLzK/h5yCKF5dChQ9W5c2e1bdv2gmE5ZcoUTZw4sSiHAQAAgBf9+OOPqlq1aoGvX+iwXLp0qbZv366tW7cW6Ppjx45VUlKS63JGRoaqVaumdbvWKapCVGEPDxSYM8epbzd9q/o33Ch/f37JFIqP05mjnZ9sZq2h2LHW4CkZx46qydX1FBERUaj9CrUqf/zxR40YMUIpKSkKCQkp0D7BwcEKDg7Osz2qQpSiKhGWKD7Os06FhoaqfIWK8g/gCRjFx5mTw1qDR7DW4GmFfdtioVbltm3bdPDgQV1zzTWubU6nUxs2bNCsWbOUnZ0tf3//Qg0AAAAA31CosGzTpo127tzptm3gwIGqW7euRo8eTVQCAACUYYUKy4iICNWrV89tW1hYmCpVqpRnOwAAAMoWfvMOAAAArLjod/6mpqZaGAMAAAClHWcsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsS7j0Q+kaN2ycmtVupisirlDjao01oPMApX2c5u3R4ENGDh6k+Ihyef7s37fP26PBx7DW4CmsNe8I8PYAOL+hdwzV2TNnNfXfU1WtZjUdPnhYH6/7WMfSj3l7NPiYm9q113PJc922VYqO8dI08GWsNXgKa83zCMsSLPNYprZu2qolKUvUuEVjSVKV6lV09XVXe3ky+KKgoCDFXlLZ22OgDGCtwVNYa57HS+ElWGh4qMLCw5Tyboqys7O9PQ4AAMB5EZYlWEBAgJ7+99N689U39bfYv6lXq156Ztwz2rNzj7dHgw9a88H7ql052vXnvrvu9PZI8FGsNXgKa83zeCm8hLv5lpt1U8ebtHXTVn3+2edav3q9Xnz2Rf3rhX/ptv63eXs8+JCmLVrqyedmuC6HhoV6cRr4MtYaPIW15nmEZSkQHBKsZm2bqVnbZhr+j+Eae/9YPf/484QlrAoNDVXNhARvj4EygLUGT2GteR4vhZdCtevW1qmsU94eAwAAwA1nLEuwo+lHNezOYeqV2Et169dVWHiYdm7fqRenvai2Xdt6ezwAAAA3hGUJFhoeqobXNdT8GfP1v//+TzlncxRXNU697+6tIaOHeHs8AAAANw5jjPHkATMzMxUVFaXtv2xXVKUoTx4aZYzzrFO71+xWo5Zt5B/Av6FQfJw5Odq2/iPWGoodaw2ecuzIEV1ZvYoyMjIUGRlZ4P14jyUAAACsICwBAABgBWEJAAAAKwhLAAAAWEFYAgAAwArCEgAAAFYQlgAAALCCsAQAAIAVhCUAAACsICwBAABgBWEJAAAAKwhLAAAAWEFYAgAAwArCEgAAAFYQlgAAALCCsAQAAIAVhCUAAACsICwBAABgBWEJAAAAKwhLAAAAWEFYAgAAwArCEgAAAFYQlgAAALCCsAQAAIAVhCUAAACsICwBAABgBWEJAAAAKwhLAAAAWBHg7QFwYZs/2qy0j9NUuUpl9b67t7fHAYCLtmHdWn225WPFxVdR3wEDvT0OAEs4Y1kKbF67WTMmz9DrC1739igAYMXGdWs1bcoTeu3lBd4eBYBFhCUAAACsICwBAABgBWEJAAAAKwhLAAAAWEFYAjiv7OxsjRv1d6UfPuztUQDgvHJzc/X4o//Qjz/84O1RyizCEsB5Dep3p+Ylz1Hvrp2ISwAlVm5urh66/z4lP/+cenZqr0MHD3h7pDKJsARwXp2795Cfn592f7VTvbt20pH0dG+PBABuzkXl8tcWS5KaNm+pStExXp6qbCIsAZxX7353aerMOXI4HNr91U7d3qUjcQmgxPhzVPbq01fT5rwgPz8Sxxt41AFcUJ/+iXp6xuz/i0vOXAIoAf4clbf2vkPPvfAiUelF/EpHoIx4Zd5LGjPyQSu3tXvnlxp4Ry+9k7LWyu0BQFFMnTzJFZWS9OaypXpz2dIi317n7j300quv2RitzCLpARRJZkaGt0cAUMZlZBzz9gj4E85YAmVE95691KRZiyLvvzF1rcY98ncZY1Q5Lk7/Xlz0swIAYMOjk57Q3t27tWXTRkm/vxQ+4pExRb69iMgIW6OVWYRlCbR57WbNnTpXycuTFRYelufrxhiNHz5eNS+rqbtH3O2FCVEaRZUvr6jy5Yu078bUdZo87p+uqFy+arUSLrvM7oDwaRtT12nWtGc0f8kyhYWH5/m6MUZjHxqhWrVr675hdt6yAd8XGhamRW+8rbt69tCWTRv15rKlSrjscj00eqy3RyuzCMsS5reff9PgnoN16uQp3d3tbi34z4I815n40EQteWmJJKlWnVpqdXMrD0+JsmTT+lQl3t5Tp0+dIipRJL/+8rMG9L5Np06eVN9bu2vJW+/muc6jjyTplXkvSZJqX15Hrdt38PSYKKVCQ0O16I231f+2W/Txxg2aOnmSHA6HRo4q+plLFB3vsSxhKleprIcmPCRJStucpnu636OTWSddX5/88GQtSl4kSepyexc1b9fcK3Oi7Jg6eRJRiYsSF19Fox4dL0n6bMvH6tezh06ezHJ9fcKYR7Rg7guSpO639VLLNm29MidKr9DQUL2y4i3d2KKlJGnuzOd14LdfvTxV2cQZyxLonpH3yOl06ql/PKXPNn6mbR9vkyTt3LZTOz7bIUnq1LOTnl3wrPz9/b04KcqChctWaPiguzXxyalEJYps8PARcjqdmjzun/pk8yZt/WSLJOmLz7dre9pWSVLXW27VzJfm87yGIgkNDdXLy9/UsHsGaPjfH9ElleO8PVKZRFiWUPf9/T4ZY/T0P5+W0+mUJNf/dujRQc+98pwCAvjrQ/GrULGiXn3jbW+PAR8wZGSScnNz9a8J4/I8r3Xq1l2z57/M8xouSmhoqOa/9rq3xyjTeCm8BBv88GA9/PjDbtvadm2r5199nidfAKXSsKSHNXbCJLdtHTp3UfLCRTyvAT6AsCzhHhj1gJImJkmSWndqrZlLZiowMNDLUwFA0Q1/+BGNHv+YJKntzZ0095XFPK8BPoJ/HpYCQ8cM1dAxQ709BgBYM+KR0RrxyGhvjwHAMs5YAgAAwArCEgAAAFYQlgAAALCCsAQAAIAVhCUAAACsICwBAABgBWEJAAAAKwhLAAAAWEFYAgAAwArCEgAAAFYQlgAAALCCsAQAAIAVhCUAAACsICwBAABgBWEJAAAAKwhLAAAAWEFYAgAAwArCEgAAAFYUKiyTk5PVoEEDRUZGKjIyUk2aNNH7779fXLMBAACgFClUWFatWlVPPvmktm3bprS0NLVu3Vrdu3fXrl27ims+AAAAlBIBhbly165d3S4/8cQTSk5O1ieffKKrrroq332ys7OVnZ3tupyZmSlJcuY45TzrLOy8QIE5c5z//39zvDwJfN25NcZaQ3FjrcFTnM6irbFChaX7AZ1avny5srKy1KRJk7+83pQpUzRx4sQ827/d9K1CQ0OLenigwHZsXu/tEVBGsNbgKaw1FLeTJ08WaT+HMcYUZoedO3eqSZMmOn36tMLDw7VkyRJ16tTpL6+f3xnLSy+9VDv/+4PKV6hYpKGBgnDm5GjH5vVqJynQ4fD2OPBhZ41RiqSGN7aUf0CR/70OXBDPa/CU9Kwsxd15pzIyMhQZGVng/Qr9DFinTh3t2LFDGRkZWrFihRITE7V+/XpdeeWV+V4/ODhYwcHBebb7+wfwBAyPCHQ4eAJG8TNG/gE8r8EzeF5DcSvq+ir0M2BQUJBq164tSWrUqJG2bt2q559/XnPnzi3SAAAAAPANF/05lrm5uW4vdQMAAKBsKtQZy7Fjx6pjx46qVq2ajh8/riVLlig1NVWrV68urvkAAABQShQqLA8ePKj+/fvr119/VVRUlBo0aKDVq1erXbt2xTUfAAAASolCheW8efOKaw4AAACUcvyucAAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsIyxJs5OBBio8ol+fP/n37vD0afMiA6dPV44kn8mxP3blTjm7ddOzECS9MBV/F8xo8gec17wnw9gA4v5vatddzyXPdtlWKjvHSNABw8XheA3wXYVnCBQUFKfaSyt4eAwCs4XkN8F28FA4AAAArOGNZwq354H3Vrhztuty6XXu9uGiJFyeCL1q5davCb7/dbZszN9dL08DX8bwGT+B5zTsIyxKuaYuWevK5Ga7LoWGhXpwGvuqm+vWV/MADbts+/eYb9Zs2zUsTwZfxvAZP4HnNOwjLEi40NFQ1ExK8PQZ8XFhIiGrHx7tt+yk93UvTwNfxvAZP4HnNO3iPJQAAAKwgLAEAAGAFYQkAAAArHMYY48kDZmZmKioqSrt/+FnlK1b05KFRxjhzcrRt/Ufq5HAo0OHw9jjwYWeN0XvGqFHLNvIP4K3rKD48r8FT0rOyFN2njzIyMhQZGVng/ThjCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWBHh7AFzYhnVr9dmWjxUXX0V9Bwz09jgAcNF4XgN8E2csS4GN69Zq2pQn9NrLC7w9CgBYwfMa4JsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAsgTamrlPvbp2VdeJEvl83xmjMyAf14qwZHp4MAIqG5zWgbAjw9gBw9+svP2tA79t06uRJ9b21u5a89W6e6zz6SJJemfeSJKn25XXUun0HT48JAAXG8xpQdnDGsoSJi6+iUY+OlyR9tuVj9evZQydPZrm+PmHMI1ow9wVJUvfbeqllm7ZemRMACornNaDs4IxlCTR4+Ag5nU5NHvdPfbJ5k7Z+skWS9MXn27U9baskqestt2rmS/Pl7+/vzVEBoEB4XgPKBs5YllBDRibpHxMflyQ5nU63/+3Urbtmz39ZAQH8uwBA6cHzGuD7CMsSbFjSwxo7YZLbtg6duyh54SKefAGUSjyvAb6NsCzhhj/8iEaPf0yS1PbmTpr7ymIFBgZ6dygAuAg8rwG+i7AsBUY8Mlq/HD+lV5a/oaCgIG+PAwAXjec1wDcRlgAAALCCsAQAAIAVhCUAAACsICwBAABgBWEJAAAAKwhLAAAAWEFYAgAAwArCEgAAAFYQlgAAALCCsAQAAIAVhCUAAACsICwBAABgBWEJAAAAKwhLAAAAWEFYAgAAwArCEgAAAFYQlgAAALCCsAQAAIAVhQrLKVOm6LrrrlNERIRiY2PVo0cP7d27t7hmAwAAQClSqLBcv369hg4dqk8++UQpKSk6e/as2rdvr6ysrOKaDwAAAKVEQGGu/MEHH7hdXrhwoWJjY7Vt2za1aNEi332ys7OVnZ3tupyZmSlJcjpz5MzJKey8QIGdW19njfHyJPB159YYz2kobjyvwVOKusYKFZZ/lpGRIUmqWLHiX15nypQpmjhxYp7tOz/ZrNDQ0Is5PFAgKZLEkzA8YMfm9d4eAWUEz2sobieLuJ/DmKKtzNzcXHXr1k3Hjh3Tpk2b/vJ6+Z2xvPTSS7Xzvz+ofIW/DlLgYjlzcrRj83o1vLGl/AMu6t9QwHmx1uAp59ZaO0mBDoe3x4EPS8/KUtyddyojI0ORkZEF3q/Iz4BDhw7VV199dd6olKTg4GAFBwfn2e7vH8ATMDzCP4C1Bs9grcFTAh0OwhLFqqjrq0jPgMOGDdPKlSu1YcMGVa1atUgHBgAAgG8pVFgaYzR8+HC99dZbSk1NVc2aNYtrLgAAAJQyhQrLoUOHasmSJXrnnXcUERGh3377TZIUFRWlcuXKFcuAAAAAKB0K9TmWycnJysjIUKtWrRQXF+f6s2zZsuKaDwAAAKVEoV8KBwAAAPLD7woHAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAswUYOHqT4iHJ5/uzft8/bo8HHsNbgKaw1eMKA6dPV44kn8mxP3blTjm7ddOzECS9MVTYEeHsAnN9N7drrueS5btsqRcd4aRr4MtYaPIW1BvguwrKECwoKUuwllb09BsoA1ho8hbUG+C5eCgcAAIAVnLEs4dZ88L5qV452XW7drr1eXLTEixPBV7HW4CmsNXjCyq1bFX777W7bnLm5Xpqm7CAsS7imLVrqyedmuC6HhoV6cRr4MtYaPIW1Bk+4qX59JT/wgNu2T7/5Rv2mTfPSRGUDYVnChYaGqmZCgrfHQBnAWoOnsNbgCWEhIaodH++27af0dC9NU3bwHksAAABYQVgCAADACsISAAAAVvAeyxJs+tyXvD0CygjWGjyFtQZPWDhyZL7bW9WvL/Puu54dpozhjCUAAACsICwBAABgBWEJAAAAKwhLAAAAWEFYAgAAwArCEgAAAFYQlgAAALCCsAQAAIAVhCUAAACsICwBAABgBWEJAAAAKwhLAAAAWEFYAgAAwArCEgAAAFYQlgAAALCCsAQAAIAVhCUAAACsICwBAABgBWEJAAAAKwhLAAAAWEFYAgAAwArCEgAAAFYQlgAAALCCsAQAAIAVhCUAAACsICwBAABgBWEJAAAAKwhLAAAAWBHg7QEAlBwb1q3VZ1s+Vlx8FfUdMNDb48CHsdYA38QZSwAuG9et1bQpT+i1lxd4exT4ONYa4JsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAspbKzszVu1N+Vfviwt0cBAACQRFiWWoP63al5yXPUu2sn4hIAAJQIhGUp1bl7D/n5+Wn3VzvVu2snHUlP9/ZIAACgjCMsS6ne/e7S1Jlz5HA4tPurnbq9S0fiEgAAeBVhWYr16Z+op2fM/r+45MwlAADwIn6lo5e9Mu8ljRn5oJXb2r3zSw28o5feSVlr5fYAAAAKgzOWPiYzI8PbIwAAgDKKM5Ze1r1nLzVp1qLI+29MXatxj/xdxhhVjovTvxcvtTgdAABAwRGWXhZVvryiypcv0r4bU9dp8rh/uqJy+arVSrjsMrsDwqdtTF2nWdOe0fwlyxQWHp7n68YYjX1ohGrVrq37htl5ywbKJtYaUDYQlqXUpvWpSry9p06fOkVUokh+/eVnDeh9m06dPKm+t3bXkrfezXOdRx9J0ivzXpIk1b68jlq37+DpMeEDWGtA2cF7LEupqZMnEZW4KHHxVTTq0fGSpM+2fKx+PXvo5Mks19cnjHlEC+a+IEnqflsvtWzT1itzovRjrQFlB2csS6mFy1Zo+KC7NfHJqUQlimzw8BFyOp2aPO6f+mTzJm39ZIsk6YvPt2t72lZJUtdbbtXMl+bL39/fm6OilGOtAWUDYVlKVahYUa++8ba3x4APGDIySbm5ufrXhHFyOp2S5PrfTt26a/b8lxUQwFMFLh5rDfB9vBQOQMOSHtbYCZPctnXo3EXJCxfxjR5WsdYA30ZYApAkDX/4EY0e/5gkqe3NnTT3lcUKDAz07lDwSaw1wHcRlgBcRjwyWr8cP6VXlr+hoKAgb48DH8ZaA3wTYQkAAAArCEsAAABYQVgCAADACsISAAAAVhCWAAAAsIKwBAAAgBWEJQAAAKwgLAEAAGAFYQkAAAArCEsAAABYQVgCAADACsISAAAAVhCWAAAAsIKwBAAAgBWEJQAAAKwgLAEAAGAFYQkAAAArCEsAAABYUeiw3LBhg7p27ar4+Hg5HA69/fbbxTAWAAAASptCh2VWVpauvvpqzZ49uzjmAQAAQCkVUNgdOnbsqI4dOxb4+tnZ2crOznZdzszMlCQ5nTly5uQU9vBAgZ1bX6wzFDfWGjzl3Bo7a4yXJ4GvK+oaK3RYFtaUKVM0ceLEPNt3frJZoaGhxX14QDs2r/f2CCgjWGvwlBRJIi5RjE4Wcb9iD8uxY8cqKSnJdTkzM1OXXnqpbpJUyeEo7sOjDDtrjFIkNbyxpfwDin2powxz5uRox+b1rDUUu3NrrV1TKTCA76EoPulHi7ZfsT8DBgcHKzg4OM/2QIdDgYQlipsx8g8I4Js9PIK1Bk8JDHAQlihWgf5FW1983BAAAACsICwBAABgRaFfszlx4oS+++471+X9+/drx44dqlixoqpVq2Z1OAAAAJQehQ7LtLQ03XTTTa7L534wJzExUQsXLrQ2GAAAAEqXQodlq1atZPiIAwAAAPwJ77EEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAEAACAFYQlAAAArCAsAQAAYAVhCQAAACsISwAAAFhBWAIAAMAKwhIAAABWEJYAAACwgrAsoQZMn64eTzyRZ3vqzp1ydOumYydOeGEq+KqRgwcpPqJcnj/79+3z9mjwMaw1eMpvB45qxKiXVPvq+xQS01OXJNylG9uNUvK/39PJk9neHs9nBXh7AAAlw03t2uu55Llu2ypFx3hpGvgy1hqK23/3/6Yb249W+agw/WtCf9W/qrqCgwK1c/f3enHBh6oSX0ndOjX29pg+ibAEIEkKCgpS7CWVvT0GygDWGorbkKRkBQT4K239NIWFhbi216pZWd073yBjjBen8228FA4AAHxGenqmPly7Q0MHdXKLyj9yOBwenqrs4IxlCbZy61aF33672zZnbq6XpoGvW/PB+6pdOdp1uXW79npx0RIvTgRfxVpDcfruv7/KGKM6l1Vx2x5do69OZ5+VJA0d1ElPTRrghel8H2FZgt1Uv76SH3jAbdun33yjftOmeWki+LKmLVrqyedmuC6HhoV6cRr4MtYavOGzdc8qNzdXfe+dpuz/H5iwj7AswcJCQlQ7Pt5t20/p6V6aBr4uNDRUNRMSvD0GygDWGopT7Vpxcjgc2vvtz27ba9X8/X295coFeWOsMoP3WAIAAJ9RqVKk2t3UULNeXKWsrNPeHqfMISwBAIBPmTPtfuXkOHVtyyQte2Ojvt77o/Z++5NeXbpOe775Sf7+5E9x4aVwAADgUxJqxenzTdP1r2eWa+zEV/TTz+kKDg7UlXUu1cMP3qIh93by9og+y2E8/GFOmZmZioqK0uHXXlOlsDBPHhplzFlj9J4xatSyjfwD+DcUio8zJ0fb1n/EWkOxO7fWOrVwKDCAj8xB8Uk/kqXoGn2UkZGhyMjIAu/HuWAAAABYQVgCAADACsISAAAAVhCWAAAAsIKwBAAAgBWEJQAAAKwgLAEAAGAFYQkAAAArCEsAAABYQVgCAADACsISAAAAVhCWAAAAsIKwBAAAgBWEJQAAAKwgLAEAAGAFYQkAAAArCEsAAABYQVgCAADACsISAAAAVhCWAAAAsIKwBAAAgBWEJQAAAKwgLAEAAGAFYQkAAAArCEsAAABYQVgCAADACsISAAAAVhCWAAAAsIKwBAAAgBWEJQAAAKwgLAEAAGAFYQkAAAArCEsAAABYQVgCAADACsISwHllZ2dr3Ki/K/3wYW+PAh/HWgNKP8ISwHkN6nen5iXPUe+unfiGj2LFWgNKP8ISwHl17t5Dfn5+2v3VTvXu2klH0tO9PRJ8FGsNKP0ISwDn1bvfXZo6c44cDod2f7VTt3fpyDd8FAvWGlD6EZYALqhP/0Q9PWP2/33D52wSiglrDSjdArw9AADPeGXeSxoz8kErt7V755caeEcvvZOy1srtwbew1oCyizOWAIokMyPD2yOgjGCtAaUHZyyBMqJ7z15q0qxFkfffmLpW4x75u4wxqhwXp38vXmpxOvgS1hpQdhGWQBkRVb68osqXL9K+G1PXafK4f7q+0S9ftVoJl11md0D4DNYaUHbxUjiA89q0PlWJt/fU6VOn+EaPYsVaA0o/whLAeU2dPIlv9PAI1hpQ+hGWAM5r4bIVat2+A9/oUexYa0Dpx3ssAZxXhYoV9eobb3t7DJQBrDWg9OOMJQAAAKwgLAEAAGAFYQkAAAArCEsAAABYQVgCAADACsISAAAAVhCWAAAAsIKwBAAAgBWEJQAAAKwgLAEAAGAFYQkAAAArCEsAAABYQVgCAADACsISAAAAVhCWAAAAsIKwBAAAgBWEJQAAAKwgLAEAAGAFYQkAAAArCEsAAABYQVgCAADAiiKF5ezZs1WjRg2FhISocePG+uyzz2zPBQAAgFKm0GG5bNkyJSUlacKECdq+fbuuvvpqdejQQQcPHiyO+QAAAFBKBBR2h2nTpmnQoEEaOHCgJOmFF17QqlWrNH/+fI0ZMybP9bOzs5Wdne26nJGRIUk6kpVV1JmBAjlrjE5KOnb0iPz9C73UgQJzOnN08uRJ1hqK3bm1ln5UCvR3eHsc+LAjx37vNGNMofYr1DPgmTNntG3bNo0dO9a1zc/PT23bttWWLVvy3WfKlCmaOHFinu2X33tvoQYFAACAZ6WnpysqKqrA1y9UWB4+fFhOp1OXXHKJ2/ZLLrlEe/bsyXefsWPHKikpyXX52LFjql69uv73v/8ValCgsDIzM3XppZfqxx9/VGRkpLfHgQ9jrcFTWGvwlIyMDFWrVk0VK1Ys1H7F/ppNcHCwgoOD82yPioriPwp4RGRkJGsNHsFag6ew1uApfn6F+3GcQl07Ojpa/v7+OnDggNv2AwcOqHLlyoU6MAAAAHxLocIyKChIjRo10kcffeTalpubq48++khNmjSxPhwAAABKj0K/FJ6UlKTExERde+21uv766zV9+nRlZWW5fkr8QoKDgzVhwoR8Xx4HbGKtwVNYa/AU1ho8pahrzWEK+3PkkmbNmqWpU6fqt99+U8OGDTVjxgw1bty4sDcDAAAAH1KksAQAAAD+jN8VDgAAACsISwAAAFhBWAIAAMAKwhIAAABWeDQsZ8+erRo1aigkJESNGzfWZ5995snDo4zYsGGDunbtqvj4eDkcDr399tveHgk+aMqUKbruuusUERGh2NhY9ejRQ3v37vX2WPBBycnJatCggeu37TRp0kTvv/++t8dCGfDkk0/K4XBo5MiRBd7HY2G5bNkyJSUlacKECdq+fbuuvvpqdejQQQcPHvTUCCgjsrKydPXVV2v27NneHgU+bP369Ro6dKg++eQTpaSk6OzZs2rfvr2ysrK8PRp8TNWqVfXkk09q27ZtSktLU+vWrdW9e3ft2rXL26PBh23dulVz585VgwYNCrWfxz5uqHHjxrruuus0a9YsSb//xp5LL71Uw4cP15gxYzwxAsogh8Oht956Sz169PD2KPBxhw4dUmxsrNavX68WLVp4exz4uIoVK2rq1Km65557vD0KfNCJEyd0zTXXaM6cOZo8ebIaNmyo6dOnF2hfj5yxPHPmjLZt26a2bdv+34H9/NS2bVtt2bLFEyMAQLHKyMiQ9Ps3fKC4OJ1OLV26VFlZWfwqZRSboUOHqnPnzm7dVlCF/pWORXH48GE5nU5dcsklbtsvueQS7dmzxxMjAECxyc3N1ciRI3XjjTeqXr163h4HPmjnzp1q0qSJTp8+rfDwcL311lu68sorvT0WfNDSpUu1fft2bd26tUj7eyQsAcCXDR06VF999ZU2bdrk7VHgo+rUqaMdO3YoIyNDK1asUGJiotavX09cwqoff/xRI0aMUEpKikJCQop0Gx4Jy+joaPn7++vAgQNu2w8cOKDKlSt7YgQAKBbDhg3TypUrtWHDBlWtWtXb48BHBQUFqXbt2pKkRo0aaevWrXr++ec1d+5cL08GX7Jt2zYdPHhQ11xzjWub0+nUhg0bNGvWLGVnZ8vf3/+8t+GR91gGBQWpUaNG+uijj1zbcnNz9dFHH/EeEQClkjFGw4YN01tvvaW1a9eqZs2a3h4JZUhubq6ys7O9PQZ8TJs2bbRz507t2LHD9efaa69V3759tWPHjgtGpeTBl8KTkpKUmJioa6+9Vtdff72mT5+urKwsDRw40FMjoIw4ceKEvvvuO9fl/fv3a8eOHapYsaKqVavmxcngS4YOHaolS5bonXfeUUREhH777TdJUlRUlMqVK+fl6eBLxo4dq44dO6patWo6fvy4lixZotTUVK1evdrbo8HHRERE5HmfeFhYmCpVqlTg9497LCx79+6tQ4cOafz48frtt9/UsGFDffDBB3l+oAe4WGlpabrppptcl5OSkiRJiYmJWrhwoZemgq9JTk6WJLVq1cpt+4IFCzRgwADPDwSfdfDgQfXv31+//vqroqKi1KBBA61evVrt2rXz9mhAHh77HEsAAAD4Nn5XOAAAAKwgLAEAAGAFYQkAAAArCEsAAABYQVgCAADACsISAAAAVhCWAAAAsIKwBAAAgBWEJQAAAKwgLAEAAGAFYQkAAAAr/h/ugGXEInZwZQAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_policy(problem, solution.policy)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3.2 Slippery case" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can repeat the analysis for the slippery case, again comparing the results from MDPax and pymdptoolbox." ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m2025-01-05 22:05:59.966\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.core.solver\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m159\u001b[0m - \u001b[1mSolver initialized with frozen_lake_4x4_slippery problem\u001b[0m\n", "\u001b[32m2025-01-05 22:05:59.992\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.utils.checkpointing\u001b[0m:\u001b[36m_setup_checkpointing\u001b[0m:\u001b[36m123\u001b[0m - \u001b[1mCheckpointing not enabled\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.283\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 1 span: 0.33333\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.420\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 2 span: 0.10000\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.423\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 3 span: 0.06000\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.427\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 4 span: 0.05400\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.432\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 5 span: 0.03510\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.436\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 6 span: 0.02916\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.440\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 7 span: 0.02066\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.443\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 8 span: 0.01669\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.446\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 9 span: 0.01266\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.451\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 10 span: 0.00999\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.453\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 11 span: 0.00806\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.457\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 12 span: 0.00647\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.461\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 13 span: 0.00547\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.465\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 14 span: 0.00469\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.468\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 15 span: 0.00399\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.472\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 16 span: 0.00342\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.478\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 17 span: 0.00304\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.482\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 18 span: 0.00271\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.484\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 19 span: 0.00244\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.489\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 20 span: 0.00218\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.492\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 21 span: 0.00193\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.497\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 22 span: 0.00171\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.501\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 23 span: 0.00152\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.505\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 24 span: 0.00134\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.510\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 25 span: 0.00118\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.514\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 26 span: 0.00104\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.518\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 27 span: 0.00091\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.523\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 28 span: 0.00080\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.525\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 29 span: 0.00070\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.531\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 30 span: 0.00061\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.538\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 31 span: 0.00054\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.543\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 32 span: 0.00047\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.548\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 33 span: 0.00041\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.550\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 34 span: 0.00036\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.554\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 35 span: 0.00031\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.557\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 36 span: 0.00027\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.560\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 37 span: 0.00024\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.564\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 38 span: 0.00021\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.567\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 39 span: 0.00018\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.571\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 40 span: 0.00016\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.576\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 41 span: 0.00014\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.578\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 42 span: 0.00012\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.581\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 43 span: 0.00011\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.582\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m502\u001b[0m - \u001b[1mConvergence threshold reached at iteration 43\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.584\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m521\u001b[0m - \u001b[1mExtracting policy\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.832\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m523\u001b[0m - \u001b[1mPolicy extracted\u001b[0m\n", "\u001b[32m2025-01-05 22:06:00.833\u001b[0m | \u001b[32m\u001b[1mSUCCESS \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m525\u001b[0m - \u001b[32m\u001b[1mValue iteration completed\u001b[0m\n" ] } ], "source": [ "problem = FrozenLake(map_name=\"4x4\", is_slippery=True)\n", "solver = ValueIteration(problem, gamma=0.9, epsilon=1e-3)\n", "solution = solver.solve()" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MDPax and mdptoolbox policies match: True\n" ] } ], "source": [ "P, R = problem.build_transition_and_reward_matrices()\n", "P, R = np.array(P), np.array(R)\n", "vi = mdptoolbox.mdp.ValueIteration(P, R, discount=0.9, epsilon=1e-3)\n", "vi.run()\n", "print(f\"MDPax and mdptoolbox policies match: {np.all(np.array(vi.policy) == solution.policy.flatten())}\")" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAApYAAAK+CAYAAAAYDV5RAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAARppJREFUeJzt3X98zfX///H7sZ/OfvmxjY38TirkHb3l5yS/52dSifyopPyI9q3wflcivYmSCqPefkR+haQ3lSa/pfLznRSVkErGMAxj2+v7h4/zdmzY5rnz2s5u18tllzqvc17n9TjHy2s3r3N25rAsyxIAAABwg4rYPQAAAAC8A2EJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAIY0adJE1atXt3uMbJs5c6YcDof279/vWtakSRM1adLEtpkAFGyEJYDruhQgWX0NHTrU7vFuiMPh0IABA+weI9suf+6LFCmi6OhotWjRQmvWrLF7NACQr90DACg4Ro4cqYoVK7otK0hn6LxF8+bN1aNHD1mWpX379mny5Mlq2rSpli9frtatW9/QfX/xxReGpgRQGBGWALKtdevWqlOnTrZue+7cOfn7+6tIEV4YMa1q1arq3r2763KnTp1Us2ZNTZgw4YbD0t/f/0bHA1CIccQHcMPWrFkjh8Oh+fPn64UXXlCZMmXkdDp18uRJSdLChQtVu3ZtFS1aVOHh4erevbv++OOPTOtn9VWhQgW3bX322Wdq1KiRgoKCFBISotjYWO3atcvtNr169VJwcLD++OMPdezYUcHBwYqIiNCzzz6r9PT0HD++pUuXKjY2VtHR0QoICFDlypX1yiuvZOu+vvjiCzmdTnXt2lVpaWmSpN27d+v+++9XiRIlFBgYqDp16uiTTz7J8VyX1KhRQ+Hh4dq3b59r2apVq1zPU7FixdShQwf9+OOP172vrN5jee7cOb388suqWrWqAgMDFRUVpfvuu0979+6VZVmqUKGCOnTokOm+zp07p7CwMPXt2zfXjw1AwcIZSwDZlpycrKNHj7otCw8Pd/3/K6+8In9/fz377LNKTU2Vv7+/Zs6cqd69e+uuu+7S6NGjdfjwYb311lvauHGjtm/frmLFiunWW2/V7Nmz3e73xIkTiouLU2RkpGvZ7Nmz1bNnT7Vs2VKvvfaazpw5o/j4eDVs2FDbt293i9D09HS1bNlSdevW1euvv66VK1fqjTfeUOXKlfXUU0/l6HHPnDlTwcHBiouLU3BwsFatWqWXXnpJJ0+e1Lhx46663rJly3T//ffrwQcf1PTp0+Xj46Ndu3apQYMGKlOmjIYOHaqgoCB9+OGH6tixoxYvXqxOnTrlaDZJOn78uI4fP64qVapIklauXKnWrVurUqVKevnll3X27Fm98847atCggbZt25Yp1q8lPT1dbdu21ZdffqmHHnpIgwYN0qlTp5SQkKDvv/9elStXVvfu3TV27FgdO3ZMJUqUcK37n//8RydPnnQ7uwrAy1kAcB0zZsywJGX5ZVmWtXr1akuSValSJevMmTOu9c6fP29FRkZa1atXt86ePetavmzZMkuS9dJLL2W5vYyMDKtt27ZWcHCwtWvXLsuyLOvUqVNWsWLFrD59+rjd9q+//rLCwsLclvfs2dOSZI0cOdLttn/729+s2rVruy2TZPXv3/+aj//yx3RJ3759LafTaZ07d861LCYmxrr99tsty7KsxYsXW35+flafPn2s9PR0123uvfdeq0aNGm7rZWRkWPXr17duvvnma85xad7HHnvMOnLkiJWYmGh988031r333mtJst544w3LsiyrVq1aVmRkpJWUlORa77///a9VpEgRq0ePHq5ll/5c9+3b5/YYYmJiXJenT59uSbLGjx+faZaMjAzLsixrz549liQrPj7e7fr27dtbFSpUcN0OgPfjpXAA2TZp0iQlJCS4fV2uZ8+eKlq0qOvyli1blJiYqH79+ikwMNC1PDY2VtWqVdPy5cuz3M4rr7yiZcuWaebMmbrtttskSQkJCTpx4oS6du2qo0ePur58fHxUt25drV69OtP9PPnkk26XGzVqpF9//TXHj/vyx3Tq1CkdPXpUjRo10pkzZ7R79+5Mt583b54efPBB9e3bV1OnTnW9z/TYsWNatWqVHnjgAdf9HD16VElJSWrZsqV+/vlnt7cIXM20adMUERGhyMhI1a1bVxs3blRcXJwGDx6sQ4cOaceOHerVq5fb2cOaNWuqefPm+vTTT3P02BcvXqzw8HANHDgw03UOh0PSxfd81q1bV3PmzHFdd+zYMX322Wfq1q2b63YAvB8vhQPItr///e/X/OGdK39i/MCBA5KkW265JdNtq1Wrpg0bNmRa/vnnn2vEiBEaNmyYOnfu7Fr+888/S5KaNm2a5bZDQ0PdLgcGBioiIsJtWfHixXX8+PGrzn81u3bt0gsvvKBVq1a53jd6SXJystvlffv2qXv37urSpYveeecdt+t++eUXWZalF198US+++GKW20pMTFSZMmWuOU+HDh00YMAAORwOhYSE6Pbbb1dQUJCkaz/nt956q1asWKGUlBTX7a9n7969uuWWW+Tre+1vFz169NCAAQN04MABlS9fXgsXLtSFCxf0yCOPZGs7ALwDYQnAmMvP7OXGvn371K1bNzVv3lyjRo1yuy4jI0PSxfdZli5dOtO6V4aPj4/PDc1yyYkTJxQTE6PQ0FCNHDlSlStXVmBgoLZt26YhQ4a45rokKipKUVFR+vTTT7Vlyxa3EL9022effVYtW7bMcnuX3id5LWXLllWzZs1u4FGZ99BDD+mZZ57RnDlz9I9//EMffPCB6tSpk2XgAvBehCWAPFO+fHlJ0p49ezKdadyzZ4/rekk6e/as7rvvPhUrVkzz5s3L9DFFlStXliRFRkZ6NKrWrFmjpKQkffTRR2rcuLFr+eU/gX25wMBALVu2TE2bNlWrVq20du1a3X777ZKkSpUqSZL8/Pzy7DFc/pxfaffu3QoPD8/22Urp4vP+zTff6MKFC/Lz87vq7UqUKKHY2FjNmTNH3bp108aNGzVhwoQczw+gYOM9lgDyTJ06dRQZGakpU6YoNTXVtfyzzz7Tjz/+qNjYWNeyJ598Uj/99JOWLFmi4sWLZ7qvli1bKjQ0VP/617904cKFTNcfOXIkTx7DpTOflmW5lp0/f16TJ0++6jphYWFasWKFIiMj1bx5c+3du1fSxShu0qSJpk6dqkOHDmVaz8RjiIqKUq1atfT+++/rxIkTruXff/+9vvjiC7Vp0yZH99e5c2cdPXpUEydOzHTd5c+JJD3yyCP64Ycf9Nxzz8nHx0cPPfRQrh4DgIKLM5YA8oyfn59ee+019e7dWzExMeratavr44YqVKigZ555RpK0fPlyzZo1S507d9Z3332n7777znUfwcHB6tixo0JDQxUfH69HHnlEd955px566CFFRETot99+0/Lly9WgQYMs4yc7tmzZkumld+niZzrWr19fxYsXV8+ePfX000/L4XBo9uzZmaLqSuHh4UpISFDDhg3VrFkzbdiwQWXKlNGkSZPUsGFD1ahRQ3369FGlSpV0+PBhbdq0Sb///rv++9//5uoxXG7cuHFq3bq16tWrp8cee8z1cUNhYWF6+eWXc3RfPXr00KxZsxQXF6dvv/1WjRo1UkpKilauXKl+/fq5fX5lbGysSpYsqYULF6p169ZuHxUFoJCw94fSARQElz6WZvPmzVlef+njhhYuXJjl9QsWLLD+9re/WQEBAVaJEiWsbt26Wb///num+8/qq3z58pm21bJlSyssLMwKDAy0KleubPXq1cvasmWL6zY9e/a0goKCMs0xfPhw68rD3tW2K8l65ZVXLMuyrI0bN1p33323VbRoUSs6Otp6/vnnrRUrVliSrNWrV7vu6/KPG7rkl19+saKioqxbb73VOnLkiGVZlrV3716rR48eVunSpS0/Pz+rTJkyVtu2ba1FixZl+fxdOe/1Ph7Jsixr5cqVVoMGDayiRYtaoaGhVrt27awffvjB7TbZ+bghy7r4cUv//Oc/rYoVK1p+fn5W6dKlrfvvv9/au3dvpu3269fPkmTNnTv3ujMC8D4Oy7rOP7sBAMimZ555RtOmTdNff/0lp9Np9zgAPIz3WAIAjDh37pw++OADde7cmagECineYwkAuCGJiYlauXKlFi1apKSkJA0aNMjukQDYhLAEANyQH374Qd26dVNkZKTefvtt1apVy+6RANiE91gCAADACN5jCQAAACMISwAAABhBWAJ5bOzYsapWrVqm3ykNdzNnzpTD4dCWLVvsHsV2vXr1UoUKFQrdtrPicDhy/KHupiUlJSkoKEiffvqprXMABQFhCeShkydP6rXXXtOQIUMy/e5rFG5//vmnXn75Ze3YscPuUWz36aef2h6P11KyZEk9/vjjevHFF+0eBcj3+E4H5KHp06crLS1NXbt2tXsU5DN//vmnRowYkWVYvvfee9qzZ4/nh7LJp59+qhEjRmR53dmzZ/XCCy94eKLMnnzySW3btk2rVq2yexQgXyMsgTw0Y8YMtW/fXoGBgUbuLyMjQ+fOncvyupSUFCPbgP38/PwUEBBg9xj5QmBgoHx97f9kvFtvvVXVq1fXzJkz7R4FyNcISyCP7Nu3T999952aNWuW6brXX39d9evXV8mSJVW0aFHVrl1bixYtynQ7h8OhAQMGaM6cObr99tsVEBCgzz//3PV+xLVr16pfv36KjIxU2bJlXetNnjzZdfvo6Gj1799fJ06ccF3/9ttvy8fHx23ZG2+8IYfDobi4ONey9PR0hYSEaMiQIa5l8+fPV+3atRUSEqLQ0FDVqFFDb7311nWfj+yul5qaqri4OEVERCgoKEidOnXSkSNHMt0uLx/j1Vxvm5LUpEkTVa9eXVu3blX9+vVVtGhRVaxYUVOmTHHdZs2aNbrrrrskSb1795bD4ZDD4XBFy5Xvc9y/f78cDodef/11TZo0SZUqVZLT6VSLFi108OBBWZalV155RWXLllXRokXVoUMHHTt2zG2upUuXKjY2VtHR0QoICFDlypX1yiuvKD09/bqPOys5ub9vvvlGbdq0UfHixRUUFKSaNWu6/ux79eqlSZMmSZLreXA4HK51s3qP5fbt29W6dWuFhoYqODhY9957r77++mu321z6O7Jx48br7k9btmxRy5YtFR4e7vrzevTRRzM9jubNm+s///mP+JQ+4Ors/2cg4KW++uorSdKdd96Z6bq33npL7du3V7du3XT+/HnNnz9fXbp00bJlyxQbG+t221WrVunDDz/UgAEDFB4ergoVKrhePu3Xr58iIiL00ksvuc5YvvzyyxoxYoSaNWump556Snv27FF8fLw2b96sjRs3ys/PT40aNVJGRoY2bNigtm3bSpLWr1+vIkWKaP369a5tb9++XadPn1bjxo0lSQkJCeratavuvfdevfbaa5KkH3/8URs3brzmb1vJyXoDBw5U8eLFNXz4cO3fv18TJkzQgAEDtGDBAtdt8vIxXk12tnnJ8ePH1aZNGz3wwAPq2rWrPvzwQz311FPy9/fXo48+qltvvVUjR47USy+9pCeeeEKNGjWSJNWvX/+aM8yZM0fnz5/XwIEDdezYMY0dO1YPPPCAmjZtqjVr1mjIkCH65Zdf9M477+jZZ5/V9OnTXevOnDlTwcHBiouLU3BwsFatWqWXXnpJJ0+e1Lhx46653axk9/4SEhLUtm1bRUVFadCgQSpdurR+/PFHLVu2TIMGDVLfvn31559/KiEhQbNnz77udnft2qVGjRopNDRUzz//vPz8/DR16lQ1adJEa9euVd26dd1uf739KTExUS1atFBERISGDh2qYsWKaf/+/froo48ybbt27dp68803tWvXLlWvXj3HzxlQKFgA8sQLL7xgSbJOnTqV6bozZ864XT5//rxVvXp1q2nTpm7LJVlFihSxdu3a5bZ8xowZliSrYcOGVlpammt5YmKi5e/vb7Vo0cJKT093LZ84caIlyZo+fbplWZaVnp5uhYaGWs8//7xlWZaVkZFhlSxZ0urSpYvl4+Pjmnn8+PFWkSJFrOPHj1uWZVmDBg2yQkND3baZHdlZ79JjatasmZWRkeFa/swzz1g+Pj7WiRMnPPIYs5LdbVqWZcXExFiSrDfeeMO1LDU11apVq5YVGRlpnT9/3rIsy9q8ebMlyZoxY0am7fXs2dMqX7686/K+ffssSVZERITrebAsyxo2bJglybrjjjusCxcuuJZ37drV8vf3t86dO+daduU+Z1mW1bdvX8vpdLrd7sptX0127i8tLc2qWLGiVb58+UzP7+V/xv3797eu9u1IkjV8+HDX5Y4dO1r+/v7W3r17Xcv+/PNPKyQkxGrcuLFrWXb3pyVLlliSrM2bN1/3MX/11VeWJGvBggXXvS1QWPFSOJBHkpKS5Ovrq+Dg4EzXFS1a1PX/x48fV3Jysho1aqRt27Zlum1MTIxuu+22LLfRp08f+fj4uC6vXLlS58+f1+DBg91+Cr1Pnz4KDQ3V8uXLJUlFihRR/fr1tW7dOkkXzx4mJSVp6NChsixLmzZtknTxDF/16tVVrFgxSVKxYsWUkpKihISEHD0XOVnviSeecHsptFGjRkpPT9eBAwc88hizkt1tXuLr66u+ffu6Lvv7+6tv375KTEzU1q1br/scXE2XLl0UFhbmunzp7Fz37t3d3odYt25dnT9/Xn/88Ydr2eX73KlTp3T06FE1atRIZ86c0e7du3M8S3bub/v27dq3b58GDx6c6fm9/M84u9LT0/XFF1+oY8eOqlSpkmt5VFSUHn74YW3YsEEnT550W+d6+9OluZYtW6YLFy5cc/vFixeXJB09ejTHswOFBWEJ2GDZsmW6++67FRgYqBIlSigiIkLx8fFKTk7OdNuKFSte9X6uvO7SN8tbbrnFbbm/v78qVarkul66+A1269atOnv2rNavX6+oqCjdeeeduuOOO1wvFW/YsMH1Mq108aX3qlWrqnXr1ipbtqweffRRff7559d9vDlZr1y5cm6XL30zP378uEceY1Zysk1Jio6OVlBQkNuyqlWrSrr4fsncuvK5uRSZN910U5bLLz1n0sWXkDt16qSwsDCFhoYqIiJC3bt3l6Qs97vryc797d27V5KMvWx85MgRnTlzJtOfg3Txh2syMjJ08OBBt+XX259iYmLUuXNnjRgxQuHh4erQoYNmzJih1NTUTNuw/u+9lbmJYqCwICyBPFKyZEmlpaXp1KlTbsvXr1/v+knxyZMn69NPP1VCQoIefvjhLH8o4PIzQzm57noaNmyoCxcuaNOmTVq/fr0rrho1aqT169dr9+7dOnLkiFt0RUZGaseOHfrkk0/Uvn17rV69Wq1bt1bPnj2vua2crHf5GdjLZfXc5MVjzM+u9txc7zk7ceKEYmJi9N///lcjR47Uf/7zHyUkJLje75rTD+83fX956XrPjcPh0KJFi7Rp0yYNGDBAf/zxhx599FHVrl1bp0+fdlvnUoyGh4fn7dBAAUZYAnmkWrVqki7+dPjlFi9erMDAQK1YsUKPPvqoWrduneVPjudG+fLlJSnTZyCeP39e+/btc10vSX//+9/l7++v9evXu0VX48aN9c033+jLL790Xb6cv7+/2rVrp8mTJ2vv3r3q27evZs2apV9++eWas+V2PTse441sU7r4GZVXfvzTTz/9JEmun/b25FmvNWvWKCkpSTNnztSgQYPUtm1bNWvWzHX2Lq/ur3LlypKk77///pr3l93nIiIiQk6nM8vP+Ny9e7eKFCmS6extdt1999169dVXtWXLFs2ZM0e7du3S/Pnz3W5z6e/yrbfemqttAIUBYQnkkXr16klSpl9R6OPjI4fD4faxLPv379fHH398w9ts1qyZ/P399fbbb7ud4Zs2bZqSk5PdfuI8MDBQd911l+bNm6fffvvN7Wze2bNn9fbbb6ty5cqKiopyrZOUlOS2vSJFiqhmzZqSlOVLhze6nh2P8Ua3KUlpaWmaOnWq6/L58+c1depURUREqHbt2pLkeqn8yo8ryguXztpdPvv58+c1efLkPL2/O++8UxUrVtSECRMyPc7L183uc+Hj46MWLVpo6dKlbm8pOHz4sObOnauGDRsqNDQ0R4/l+PHjmc6G16pVS1LmfXPr1q0KCwvT7bffnqNtAIUJHzcE5JFKlSqpevXqWrlypdtn4sXGxmr8+PFq1aqVHn74YSUmJmrSpEmqUqWKvvvuuxvaZkREhIYNG6YRI0aoVatWat++vfbs2aPJkyfrrrvucr0H7pJGjRppzJgxCgsLU40aNSRdfNn6lltu0Z49e9SrVy+32z/++OM6duyYmjZtqrJly+rAgQN65513VKtWrWuexcntenY8RhPbjI6O1muvvab9+/eratWqWrBggXbs2KF3333X9bFElStXVrFixTRlyhSFhIQoKChIdevWveZ7anOrfv36Kl68uHr27Kmnn35aDodDs2fPzvXnMWb3/ooUKaL4+Hi1a9dOtWrVUu/evRUVFaXdu3dr165dWrFihSS5Yvvpp59Wy5Yt5ePjo4ceeijLbY8aNUoJCQlq2LCh+vXrJ19fX02dOlWpqakaO3Zsjh/L+++/r8mTJ6tTp06qXLmyTp06pffee0+hoaFq06aN220TEhLUrl073mMJXIvnfxAdKDzGjx9vBQcHZ/polmnTplk333yzFRAQYFWrVs2aMWOGNXz48EwfuSLJ6t+/f6b7vfRRKlf7iJSJEyda1apVs/z8/KxSpUpZTz31VJYfp7N8+XJLktW6dWu35Y8//rglyZo2bZrb8kWLFlktWrSwIiMjLX9/f6tcuXJW3759rUOHDl3zecjOeld7TKtXr7YkWatXr/bIY7yW7GwzJibGuv32260tW7ZY9erVswIDA63y5ctbEydOzHR/S5cutW677TbL19fX7aOHrvZxQ+PGjcvyuVm4cKHb8qyey40bN1p33323VbRoUSs6Otp6/vnnrRUrVmR6brP7cUPZvT/LsqwNGzZYzZs3t0JCQqygoCCrZs2a1jvvvOO6Pi0tzRo4cKAVERFhORwOt78HuuLjhizLsrZt22a1bNnSCg4OtpxOp3XPPfdYX3311XWfg8ufs0szbtu2zeratatVrlw5KyAgwIqMjLTatm1rbdmyxW29H3/80ZJkrVy58rrPDVCYOSyLXyEA5JXk5GRVqlRJY8eO1WOPPWb3OPCAJk2a6OjRo9d9XyEKlsGDB2vdunXaunUrZyyBa+A9lkAeCgsL0/PPP69x48blq5+UBZB9SUlJ+ve//61Ro0YRlcB1cMYSAAzijCWAwowzlgAAADDihsJyzJgxcjgcGjx4sKFxAKBgW7NmDWcrARRauQ7LzZs3a+rUqa7PogMAAEDhlqvPsTx9+rS6deum9957T6NGjbrmbVNTU90+ZDYjI0PHjh1TyZIleRM0AABAPmRZlk6dOqXo6GgVKZL985C5Csv+/fsrNjZWzZo1u25Yjh49WiNGjMjNZgAAAGCjgwcPqmzZstm+fY7Dcv78+dq2bZs2b96crdsPGzZMcXFxrsvJyckqV66cVu9arbDiYTndPJBt6Wnp+nnDz6pxdwP5+PBLppB30tPTtPPrjexryHPsa/CU5BPHVe+O6goJCcnRejnaKw8ePKhBgwYpISFBgYGB2VonICBAAQEBmZaHFQ9TWEnCEnkn/UK6nE6nihUvIR9fDsDIO+lpaexr8Aj2NXhaTt+2mKO9cuvWrUpMTNSdd97pWpaenq5169Zp4sSJSk1NlY+PT44GAAAAgHfIUVjee++92rlzp9uy3r17q1q1ahoyZAhRCQAAUIjlKCxDQkJUvXp1t2VBQUEqWbJkpuUAAAAoXPjNOwAAADDiht/5u2bNGgNjAAAAoKDjjCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhGU+l3QkSS8OeFENqzTUrSG3qm65uuoV20tbvtpi92jwIoP79lF0SNFMX/v27rV7NHgZ9jV4CvuaPXztHgDX1v+h/rpw/oLG/XucylUsp6OJR/XV6q90IumE3aPBy9zTvIXejJ/qtqxkeIRN08Cbsa/BU9jXPI+wzMdOnjipzRs2a27CXNVtXFeSVKZ8Gd1x1x02TwZv5O/vr8hSpe0eA4UA+xo8hX3N83gpPB9zBjsVFBykhE8SlJqaavc4AAAA10RY5mO+vr4a+++x+uiDj/S3yL+pS5Muev3F17V75267R4MXWvn5Z6pSOtz19cQjD9s9ErwU+xo8hX3N83gpPJ9r1amV7ml9jzZv2Kzt327X2hVr9e4b7+pfU/6l+3vcb/d48CL1G8dozJtvuy47g5w2TgNvxr4GT2Ff8zzCsgAICAxQw2YN1bBZQw38x0ANe3KY3nrlLcISRjmdTlWsXNnuMVAIsK/BU9jXPI+XwgugKtWq6GzKWbvHAAAAcMMZy3zseNJxDXh4gLr07KJqNaopKDhIO7ft1Lvj31Wzds3sHg8AAMANYZmPOYOdqnVXLU1/e7p++/U3pV1IU1TZKD346IPqN6Sf3eMBAAC4cViWZXlygydPnlRYWJi2/blNYSXDPLlpFDLpF9L1w8ofVDvmXvn48m8o5J30tDRtXfsl+xryHPsaPOXEsWO6rXwZJScnKzQ0NNvr8R5LAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsATg8vtvBzT2lRH6av06u0cBACM4rnkWYQlAkvT7wd90f2wrTRg7Rt3v66B1q1fZPRIA3BCOa55HWALQH78fVJfYVvpt/35J0rlz59Trwfu1cd1aewcDgFziuGYPwhIo5P7843d1iW2lA/v2uS0/d/asenS5T19v2GDTZACQOxzX7ENYAoXYoT//UJfYVtr/66/y8/PT3+rUkSTdVL68SkdF6eyZM3qkSyd989VGmycFgOzhuGYvwhIopI4eSVSXNq20b+9e+fn5acr7H6h+oxhJUkREpD5c9rkiS5VWyunTeuT+Ttq+ZbPNEwPAtXFcsx9hmc9kZGRozLAx+n3/73aPAi8XVqy4bq5WTb6+voqfMUut27V3u75K1apauOwzhUdEKiq6jG4qX96mSVHQZWRk6JUX/qGDBw7YPQq8HMc1+/naPQD+JyMjQ8/3eV5LPliiTxd/qo/Wf6TwUuF2jwUv5efnp6mz5mj75m9Vt0HDLG9zc7VqWrj8MxUvUULhEZEenhDeICMjQ888+YQWzpuj/yxZrOWr1ykispTdY8FLcVyzH2GZT1welZJ0d8zdKhFRwuap4O38/f2vevC95JZbb/PQNPA2l0elJNVvFKOS4RE2TwVvx3HNXrwUng9cGZWdunfSmKljVKQIfzwACqYro7JL124aP3kKxzXAy/E33GZXRmWHrh009r2xHHwBFFhXRuV9Dz6kN6e8y3ENKAR4Kdxmb4540xWVkrR03lItnbc01/fXqlMrTZo/ycRoAJAr40aNdEWlJH20YL4+WjA/1/cX26Gj3vtgnonRAOQx/vlos1MnTtk9AgAYlZx8wu4RANiEM5Y2e/5fz+unH37SN+u+kXTxpfD+Q/vn+v6CQ4NNjQYAufLCyFe154cftGnDekkXXwof9NzQXN9fSGiIqdEA5DHC0mbOIKemLZ2mxzo8pm/WfaOl85aqYtWKGviPgXaPBgC54gwK0uzFH+uRzh21acN6fbRgvirfXFXPDBlm92gA8hgvhecDRZ1FNW3pNNWNqStJmjBigiaOnmjzVACQe06nU7MXf6z6jRpLuvi+ywljx9g8FYC8RljmE0WdRTXt42m6u8ndkqRpE6Yp8VCizVMBQO45nU7NWrREDRpf/JV6U995S4f/OmTzVADyEmGZjxR1FtW/l/xbzds318xlMxUZxW8EAFCwOZ1Ovb/wI7Vq205zl3yiUqWj7B4JQB7iPZb5TFFnUU1ZOMXuMQDAGKfTqenzPrR7DAAewBlLAAAAGEFYAgAAwAjCEoDLP0eO0p+nzmrZ6nV2jwIARnBc8yzCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADAiR2EZHx+vmjVrKjQ0VKGhoapXr54+++yzvJoNAAAABUiOwrJs2bIaM2aMtm7dqi1btqhp06bq0KGDdu3alVfzAQAAoIDwzcmN27Vr53b51VdfVXx8vL7++mvdfvvtWa6Tmpqq1NRU1+WTJ09KktLT0pV+IT2n8wLZlp6W/n//TbN5Eni7S/sY+xryGvsaPCU9PXf7WI7C0n2D6Vq4cKFSUlJUr169q95u9OjRGjFiRKblP2/4WU6nM7ebB7Jtx8a1do+AQoJ9DZ7Cvoa8dubMmVyt57Asy8rJCjt37lS9evV07tw5BQcHa+7cuWrTps1Vb5/VGcubbrpJO389oGLFS+RqaCA70tPStGPjWjWX5Odw2D0OvNgFy1KCpFoNYuTjm+t/rwPXxXENnpKUkqKohx9WcnKyQkNDs71ejo+At9xyi3bs2KHk5GQtWrRIPXv21Nq1a3XbbbdlefuAgAAFBARkWu7j48sBGB7h53BwAEbesyz5+HJcg2dwXENey+3+leMjoL+/v6pUqSJJql27tjZv3qy33npLU6dOzdUAAAAA8A43/DmWGRkZbi91AwAAoHDK0RnLYcOGqXXr1ipXrpxOnTqluXPnas2aNVqxYkVezQcAAIACIkdhmZiYqB49eujQoUMKCwtTzZo1tWLFCjVv3jyv5gMAAEABkaOwnDZtWl7NAQAAgAKO3xUOAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGGZjw3u20fRIUUzfe3bu9fu0eBFek2YoI6vvppp+ZqdO+Vo314nTp+2YSp4K45r8ASOa/bxtXsAXNs9zVvozfipbstKhkfYNA0A3DiOa4D3IizzOX9/f0WWKm33GABgDMc1wHvxUjgAAACM4IxlPrfy889UpXS463LT5i307uy5Nk4Eb7Rs82YFP/CA27L0jAybpoG347gGT+C4Zg/CMp+r3zhGY95823XZGeS0cRp4q3tq1FD8U0+5Lfvmp5/Uffx4myaCN+O4Bk/guGYPwjKfczqdqli5st1jwMsFBQaqSnS027Lfk5JsmgbejuMaPIHjmj14jyUAAACMICwBAABgBGEJAAAAIxyWZVme3ODJkycVFhamHw78oWIlSnhy0yhk0tPStHXtl2rjcMjP4bB7HHixC5alTy1LtWPulY8vb11H3uG4Bk9JSklReNeuSk5OVmhoaLbX44wlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYARhmc9kZGTolRf+oYMHDtg9CgAYwXENKDwIy3wkIyNDzzz5hOLfelOd27TQkcTDdo8EADeE4xpQuBCW+cSlg+/CeXMkSfUbxahkeITNUwFA7nFcAwofwjIfuPLg26VrN42fPEVFivDHA6Bg4rgGFE78DbfZlQff+x58SG9OeZeDL4ACi+MaUHj52j1AYTdu1EjXwVeSPlowXx8tmJ/r+4vt0FHvfTDPxGgAkCsc14DCi38+2iw5+YTdIwCAURzXgMKLM5Y2e2Hkq9rzww/atGG9pIsvGQ16bmiu7y8kNMTUaACQKxzXgMKLsLSZMyhIsxd/rEc6d9SmDev10YL5qnxzVT0zZJjdowFArnBcAwovXgrPB5xOp2Yv/lj1GzWWdPH9SRPGjrF5KgDIPY5rQOFEWOYTTqdTsxYtUYPGMZKkqe+8pcN/HbJ5KgDIPY5rQOFDWOYjTqdT7y/8SK3attPcJZ+oVOkou0cCgBvCcQ0oXHiPZT7jdDo1fd6Hdo8BAMZwXAMKD85YAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMyFFYjh49WnfddZdCQkIUGRmpjh07as+ePXk1GwAAAAqQHIXl2rVr1b9/f3399ddKSEjQhQsX1KJFC6WkpOTVfAAAACggfHNy488//9zt8syZMxUZGamtW7eqcePGWa6Tmpqq1NRU1+WTJ09KktLT05SelpbTeYFsu7R/XbAsmyeBt7u0j3FMQ17juAZPye0+lqOwvFJycrIkqUSJEle9zejRozVixIhMy3d+vVFOp/NGNg9kS4IkcRCGB+zYuNbuEVBIcFxDXjuTy/UclpW7PTMjI0Pt27fXiRMntGHDhqveLqszljfddJN2/npAxYpfPUiBG5WelqYdG9eqVoMY+fje0L+hgGtiX4OnXNrXmkvyczjsHgdeLCklRVEPP6zk5GSFhoZme71cHwH79++v77///ppRKUkBAQEKCAjItNzHx5cDMDzCx5d9DZ7BvgZP8XM4CEvkqdzuX7k6Ag4YMEDLli3TunXrVLZs2VxtGAAAAN4lR2FpWZYGDhyoJUuWaM2aNapYsWJezQUAAIACJkdh2b9/f82dO1dLly5VSEiI/vrrL0lSWFiYihYtmicDAgAAoGDI0edYxsfHKzk5WU2aNFFUVJTra8GCBXk1HwAAAAqIHL8UDgAAAGSF3xUOAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGGZjw3u20fRIUUzfe3bu9fu0eBl2NfgKexr8IReEyao46uvZlq+ZudOOdq314nTp22YqnDwtXsAXNs9zVvozfipbstKhkfYNA28GfsaPIV9DfBehGU+5+/vr8hSpe0eA4UA+xo8hX0N8F68FA4AAAAjOGOZz638/DNVKR3uuty0eQu9O3uujRPBW7GvwVPY1+AJyzZvVvADD7gtS8/IsGmawoOwzOfqN47RmDffdl12BjltnAbejH0NnsK+Bk+4p0YNxT/1lNuyb376Sd3Hj7dposKBsMznnE6nKlaubPcYKATY1+Ap7GvwhKDAQFWJjnZb9ntSkk3TFB68xxIAAABGEJYAAAAwgrAEAACAEbzHMh+bMPU9u0dAIcG+Bk9hX4MnzBw8OMvlTWrUkPXJJ54dppDhjCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGOFr9wC4vt9/O6C5789Uwyb3qH6jxnaPAy+2bvUqfbvpK0VFl1G3Xr3tHgcAUMBwxjKf+/3gb7o/tpUmjB2j7vd10LrVq+weCV5s/epVGj/6Vc17f4bdowAACiDCMh/74/eD6hLbSr/t3y9JOnfunHo9eL82rltr72AAAABZICzzqT//+F1dYlvpwL59bsvPnT2rHl3u09cbNtg0GQAAQNYIy3zo0J9/qEtsK+3/9Vf5+fnpb3XqSJJuKl9epaOidPbMGT3SpZO++WqjzZMCAAD8D2GZzxw9kqgubVpp39698vPz05T3P1D9RjGSpIiISH247HNFliqtlNOn9cj9nbR9y2abJwYAALiIsMxnwooV183VqsnX11fxM2apdbv2btdXqVpVC5d9pvCISEVFl9FN5cvbNCkAZE9GRoZeeeEfOnjggN2jAMhjfNxQPuPn56eps+Zo++ZvVbdBwyxvc3O1alq4/DMVL1FC4RGRHp4QALIvIyNDzzz5hBbOm6P/LFms5avXKSKylN1jAcgjhGU+5O/vf9WovOSWW2/z0DQAkDuXR6Uk1W8Uo5LhETZPBSAv8VI4AMC4K6OyS9duGj95iooU4dsO4M34Gw4AMOrKqLzvwYf05pR3iUqgEOClcACAUeNGjXRFpSR9tGC+PlowP9f3F9uho977YJ6J0QDkMf75CAAwKjn5hN0jALAJZywBAEa9MPJV7fnhB23asF7SxZfCBz03NNf3FxIaYmo0AHmMsAQKsfVrVmvi+Nc1fe4CBQUHZ7resiwNe2aQKlWpoicGPG3DhCiInEFBmr34Yz3SuaM2bVivjxbMV+Wbq+qZIcPsHg1AHuOlcKCQOvTnH+r14P1av3qVut3XQWdSUjLd5oXn4jRr2nt6edgQrfpihQ1ToqByOp2avfhj1W/UWNLF911OGDvG5qkA5DXCEiikoqLL6PkXXpIkfbvpK3Xv3FFnzvwvLocPfU4zpk6RJHW4v4ti7m1my5wouJxOp2YtWqIGjS/+Wtqp77ylw38dsnkqAHmJl8KBQqzvwEFKT0/XqBf/qa83btDmrzdJkv67fZu2/d/voW/X6T698950+fj42DkqCiin06n3F36kAY/10sD/95xKlY6yeyQAeYiwBAq5foPjlJGRoX8Nf1Hp6emS5Ppvm/YdNGn6+/L15VCB3HM6nZo+70O7xwDgAbwUDkAD4p7VsOEj3Za1jG2r+JmziUoAQLYRlgAkSQOffU5DXnpZktSsVRtNnTVHfn5+9g4FAChQCMsC4J8jR+nPU2e1bPU6u0eBlxv03BD9eeqsZi1cLH9/f7vHAQAUMIQlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYESOw3LdunVq166doqOj5XA49PHHH+fBWAAAAChochyWKSkpuuOOOzRp0qS8mAcAAAAFlG9OV2jdurVat26d7dunpqYqNTXVdfnkyZOSpPT0NKWnpeV080C2Xdq/2M+Q19jX4CmX9rELlmXzJPB2ud3HchyWOTV69GiNGDEi0/KdX2+U0+nM680D2rFxrd0joJBgX4OnJEgScYk8dCaX6+V5WA4bNkxxcXGuyydPntRNN92keySVdDjyevMoxC5YlhIk1WoQIx/fPN/VUYilp6Vpx8a17GvIc5f2teb1JT9fvoci7yQdz916eX4EDAgIUEBAQKblfg6H/AhL5DXLko+vL9/s4RHsa/AUP18HYYk85eeTu/2LjxsCAACAEYQlAAAAjMjxazanT5/WL7/84rq8b98+7dixQyVKlFC5cuWMDgcAAICCI8dhuWXLFt1zzz2uy5d+MKdnz56aOXOmscEAAABQsOQ4LJs0aSKLjzgAAADAFXiPJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMIKwBAAAgBGEJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMIKwBAAAgBGEJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMIKwBAAAgBGEJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMIKwBAAAgBGEJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMIKwBAAAgBGEJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMIKwBAAAgBGEJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMIKwBAAAgBGEJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMIKwBAAAgBGEZT7Va8IEdXz11UzL1+zcKUf79jpx+rQNU8FbDe7bR9EhRTN97du71+7R4GXY1+Apfx0+rkHPv6cqdzyhwIjOKlX5ETVo/rzi//2pzpxJtXs8r+Vr9wAA8od7mrfQm/FT3ZaVDI+waRp4M/Y15LVf9/2lBi2GqFhYkP41vIdq3F5eAf5+2vnDfr074wuViS6p9m3q2j2mVyIsAUiS/P39FVmqtN1joBBgX0Ne6xcXL19fH21ZO15BQYGu5ZUqllaH2LtlWZaN03k3XgoHAABeIynppL5YtUP9+7Rxi8rLORwOD09VeHDGMh9btnmzgh94wG1ZekaGTdPA2638/DNVKR3uuty0eQu9O3uujRPBW7GvIS/98ushWZalW24u47Y8vEI3nUu9IEnq36eNXhvZy4bpvB9hmY/dU6OG4p96ym3ZNz/9pO7jx9s0EbxZ/cYxGvPm267LziCnjdPAm7GvwQ7frn5DGRkZ6vb4eKX+X2DCPMIyHwsKDFSV6Gi3Zb8nJdk0Dbyd0+lUxcqV7R4DhQD7GvJSlUpRcjgc2vPzH27LK1W8+L7eokX97Rir0OA9lgAAwGuULBmq5vfU0sR3lysl5Zzd4xQ6hCUAAPAqk8c/qbS0dNWJidOCxev1456D2vPz7/pg/mrt/ul3+fiQP3mFl8IBAIBXqVwpSts3TNC/Xl+oYSNm6fc/khQQ4KfbbrlJzz7dSf0eb2P3iF7LYXn4w5xOnjypsLAwHZ03TyWDgjy5aRQyFyxLn1qWasfcKx9f/g2FvJOelqata79kX0Oeu7SvtWnskJ8vH5mDvJN0LEXhFboqOTlZoaGh2V6Pc8EAAAAwgrAEAACAEYQlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYISv3QMAAAqfdatX6dtNXykquoy69ept9zgADOGMJQDA49avXqXxo1/VvPdn2D0KAIMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEcE2pqal68fn/p6SjR+0eBQCQzxGWAK6pT/eHNS1+sh5s14a4BABcE2EJ4JpiO3RUkSJF9MP3O/VguzY6lpRk90gAgHyKsARwTQ92f0Tj3pksh8OhH77fqQfatiYuAQBZIiwBXFfXHj019u1J/4tLzlwCALLAr3QEColZ097T0MFPG7mvH3Z+p94PddHShFVG7g8A4B04YwkgV04mJ9s9AgAgn+GMJVBIdOjcRfUaNs71+uvXrNKLz/0/WZal0lFR+vec+QanAwB4A8ISKCTCihVTWLFiuVp3/ZrVGvXiP11RuXD5ClW++WazA8KrrV+zWhPHv67pcxcoKDg40/WWZWnYM4NUqUoVPTHAzFs2AHgeYQngmjasXaOeD3TWubNniUrkyqE//1CvB+/X2TNn1O2+Dpq75JNMt3nhuTjNmvaeJKlK1VvUtEVLT48JwADeYwngmsaNGklU4oZERZfR8y+8JEn6dtNX6t65o86cSXFdP3zoc5oxdYokqcP9XRRzbzNb5gRw4zhjCeCaZi5YpIF9HtWIMeOISuRa34GDlJ6erlEv/lNfb9ygzV9vkiT9d/s2bduyWZLUrtN9eue96fLx8bFzVAA3gLAEcE3FS5TQB4s/tnsMeIF+g+OUkZGhfw1/Uenp6ZLk+m+b9h00afr78vXl2xJQkPFSOADAYwbEPathw0e6LWsZ21bxM2cTlYAXICwBAB418NnnNOSllyVJzVq10dRZc+Tn52fvUACMICwBAB436Lkh+vPUWc1auFj+/v52jwPAEMISAAAARhCWAAAAMIKwBAAAgBGEJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMIKwBAAAgBGEJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMCJXYTlp0iRVqFBBgYGBqlu3rr799lvTcwEAAKCAyXFYLliwQHFxcRo+fLi2bdumO+64Qy1btlRiYmJezAcAAIACwjenK4wfP159+vRR7969JUlTpkzR8uXLNX36dA0dOjTT7VNTU5Wamuq6nJycLEk6lpKS25mBbLlgWToj6cTxY/LxyfGuDmRbenqazpw5w76GPHdpX0s6Lvn5OOweB17s2ImLnWZZVo7Wy9ER8Pz589q6dauGDRvmWlakSBE1a9ZMmzZtynKd0aNHa8SIEZmWV3388RwNCgAAAM9KSkpSWFhYtm+fo7A8evSo0tPTVapUKbflpUqV0u7du7NcZ9iwYYqLi3NdPnHihMqXL6/ffvstR4MCOXXy5EnddNNNOnjwoEJDQ+0eB16MfQ2ewr4GT0lOTla5cuVUokSJHK2X56/ZBAQEKCAgINPysLAw/lLAI0JDQ9nX4BHsa/AU9jV4SpEiOftxnBzdOjw8XD4+Pjp8+LDb8sOHD6t06dI52jAAAAC8S47C0t/fX7Vr19aXX37pWpaRkaEvv/xS9erVMz4cAAAACo4cvxQeFxennj17qk6dOvr73/+uCRMmKCUlxfVT4tcTEBCg4cOHZ/nyOGAS+xo8hX0NnsK+Bk/J7b7msHL6c+SSJk6cqHHjxumvv/5SrVq19Pbbb6tu3bo5vRsAAAB4kVyFJQAAAHAlflc4AAAAjCAsAQAAYARhCQAAACMISwAAABjh0bCcNGmSKlSooMDAQNWtW1fffvutJzePQmLdunVq166doqOj5XA49PHHH9s9ErzQ6NGjdddddykkJESRkZHq2LGj9uzZY/dY8ELx8fGqWbOm67ft1KtXT5999pndY6EQGDNmjBwOhwYPHpztdTwWlgsWLFBcXJyGDx+ubdu26Y477lDLli2VmJjoqRFQSKSkpOiOO+7QpEmT7B4FXmzt2rXq37+/vv76ayUkJOjChQtq0aKFUlJS7B4NXqZs2bIaM2aMtm7dqi1btqhp06bq0KGDdu3aZfdo8GKbN2/W1KlTVbNmzRyt57GPG6pbt67uuusuTZw4UdLF39hz0003aeDAgRo6dKgnRkAh5HA4tGTJEnXs2NHuUeDljhw5osjISK1du1aNGze2exx4uRIlSmjcuHF67LHH7B4FXuj06dO68847NXnyZI0aNUq1atXShAkTsrWuR85Ynj9/Xlu3blWzZs3+t+EiRdSsWTNt2rTJEyMAQJ5KTk6WdPEbPpBX0tPTNX/+fKWkpPCrlJFn+vfvr9jYWLduy64c/0rH3Dh69KjS09NVqlQpt+WlSpXS7t27PTECAOSZjIwMDR48WA0aNFD16tXtHgdeaOfOnapXr57OnTun4OBgLVmyRLfddpvdY8ELzZ8/X9u2bdPmzZtztb5HwhIAvFn//v31/fffa8OGDXaPAi91yy23aMeOHUpOTtaiRYvUs2dPrV27lriEUQcPHtSgQYOUkJCgwMDAXN2HR8IyPDxcPj4+Onz4sNvyw4cPq3Tp0p4YAQDyxIABA7Rs2TKtW7dOZcuWtXsceCl/f39VqVJFklS7dm1t3rxZb731lqZOnWrzZPAmW7duVWJiou68807XsvT0dK1bt04TJ05UamqqfHx8rnkfHnmPpb+/v2rXrq0vv/zStSwjI0Nffvkl7xEBUCBZlqUBAwZoyZIlWrVqlSpWrGj3SChEMjIylJqaavcY8DL33nuvdu7cqR07dri+6tSpo27dumnHjh3XjUrJgy+Fx8XFqWfPnqpTp47+/ve/a8KECUpJSVHv3r09NQIKidOnT+uXX35xXd63b5927NihEiVKqFy5cjZOBm/Sv39/zZ07V0uXLlVISIj++usvSVJYWJiKFi1q83TwJsOGDVPr1q1Vrlw5nTp1SnPnztWaNWu0YsUKu0eDlwkJCcn0PvGgoCCVLFky2+8f91hYPvjggzpy5Iheeukl/fXXX6pVq5Y+//zzTD/QA9yoLVu26J577nFdjouLkyT17NlTM2fOtGkqeJv4+HhJUpMmTdyWz5gxQ7169fL8QPBaiYmJ6tGjhw4dOqSwsDDVrFlTK1asUPPmze0eDcjEY59jCQAAAO/G7woHAACAEYQlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYMT/B/FNV7C28TGaAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_policy(problem, solution.policy)" ] }, { "cell_type": "markdown", "metadata": { "id": "results_explanation" }, "source": [ "Note that even though this is the optimal policy, the agent won't always follow the arrows exactly when `is_slippery=True` because there's a chance of sliding left or right relative to the intended direction." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Checkpoints for problems defined in notebooks" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The checkpointing demonstration in the [Getting Started notebook](https://mdpax.readthedocs.io/en/latest/notebooks/getting_started.html) uses an MDPax example problem, which is defined in a Python module and has a [Hydra](https://hydra.cc/docs/1.3/intro/) configuration. This allows the use of the `restore` method which can recreate the problem and solver using the config.\n", "\n", "If a problem is defined in a notebook, as is the case here, a lightweight version of checkpointing is enabled instead. It still saves the solver state, and so optimization can be resumed, but the user must manually recreate the problem and solver before loading the checkpoint, as shown below." ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m2025-01-05 22:06:01.534\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.core.solver\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m159\u001b[0m - \u001b[1mSolver initialized with frozen_lake_4x4_slippery problem\u001b[0m\n", "\u001b[32m2025-01-05 22:06:01.565\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.utils.checkpointing\u001b[0m:\u001b[36m_setup_checkpointing\u001b[0m:\u001b[36m147\u001b[0m - \u001b[1mLightweight checkpointing enabled - problem and solver must be reconstructed manually\u001b[0m\n", "\u001b[32m2025-01-05 22:06:01.566\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.utils.checkpointing\u001b[0m:\u001b[36m_setup_checkpointing\u001b[0m:\u001b[36m152\u001b[0m - \u001b[1mSaving checkpoints every 1 iteration(s) to /home/joefarrington/other_learning/mdpax/examples/checkpoints/create_custom_problem/initial_checkpoints\u001b[0m\n", "\u001b[32m2025-01-05 22:06:01.761\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 1 span: 0.33333\u001b[0m\n", "\u001b[32m2025-01-05 22:06:01.904\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 2 span: 0.10000\u001b[0m\n", "\u001b[32m2025-01-05 22:06:01.909\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 3 span: 0.06000\u001b[0m\n", "\u001b[32m2025-01-05 22:06:01.916\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 4 span: 0.05400\u001b[0m\n", "\u001b[32m2025-01-05 22:06:01.921\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 5 span: 0.03510\u001b[0m\n", "\u001b[32m2025-01-05 22:06:01.929\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 6 span: 0.02916\u001b[0m\n", "\u001b[32m2025-01-05 22:06:01.933\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 7 span: 0.02066\u001b[0m\n", "\u001b[32m2025-01-05 22:06:01.937\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 8 span: 0.01669\u001b[0m\n", "\u001b[32m2025-01-05 22:06:01.944\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 9 span: 0.01266\u001b[0m\n", "\u001b[32m2025-01-05 22:06:01.948\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 10 span: 0.00999\u001b[0m\n", "\u001b[32m2025-01-05 22:06:01.949\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m514\u001b[0m - \u001b[1mMaximum iterations reached\u001b[0m\n", "\u001b[32m2025-01-05 22:06:01.950\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m521\u001b[0m - \u001b[1mExtracting policy\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.140\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m523\u001b[0m - \u001b[1mPolicy extracted\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.141\u001b[0m | \u001b[32m\u001b[1mSUCCESS \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m525\u001b[0m - \u001b[32m\u001b[1mValue iteration completed\u001b[0m\n" ] } ], "source": [ "# Run the solver for 10 iterations\n", "problem = FrozenLake(map_name=\"4x4\", is_slippery=True)\n", "solver = ValueIteration(problem, gamma=0.9, epsilon=1e-3, \n", " checkpoint_dir=\"checkpoints/create_custom_problem/initial_checkpoints\", \n", " checkpoint_frequency=1)\n", "solution = solver.solve(max_iterations = 10)" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m2025-01-05 22:06:02.153\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.core.solver\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m159\u001b[0m - \u001b[1mSolver initialized with frozen_lake_4x4_slippery problem\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.184\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.utils.checkpointing\u001b[0m:\u001b[36m_setup_checkpointing\u001b[0m:\u001b[36m147\u001b[0m - \u001b[1mLightweight checkpointing enabled - problem and solver must be reconstructed manually\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.185\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.utils.checkpointing\u001b[0m:\u001b[36m_setup_checkpointing\u001b[0m:\u001b[36m152\u001b[0m - \u001b[1mSaving checkpoints every 1 iteration(s) to /home/joefarrington/other_learning/mdpax/examples/checkpoints/create_custom_problem/new_checkpoints\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.437\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 11 span: 0.00806\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.460\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 12 span: 0.00647\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.467\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 13 span: 0.00547\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.473\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 14 span: 0.00469\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.480\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 15 span: 0.00399\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.485\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 16 span: 0.00342\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.490\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 17 span: 0.00304\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.497\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 18 span: 0.00271\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.503\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 19 span: 0.00244\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.511\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 20 span: 0.00218\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.518\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 21 span: 0.00193\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.525\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 22 span: 0.00171\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.529\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 23 span: 0.00152\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.536\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 24 span: 0.00134\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.542\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 25 span: 0.00118\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.549\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 26 span: 0.00104\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.555\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 27 span: 0.00091\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.561\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 28 span: 0.00080\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.567\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 29 span: 0.00070\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.574\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 30 span: 0.00061\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.581\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 31 span: 0.00054\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.590\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 32 span: 0.00047\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.597\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 33 span: 0.00041\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.603\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 34 span: 0.00036\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.610\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 35 span: 0.00031\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.616\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 36 span: 0.00027\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.623\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 37 span: 0.00024\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.631\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 38 span: 0.00021\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.635\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 39 span: 0.00018\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.641\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 40 span: 0.00016\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.650\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 41 span: 0.00014\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.657\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 42 span: 0.00012\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.664\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m497\u001b[0m - \u001b[1mIteration 43 span: 0.00011\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.666\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m502\u001b[0m - \u001b[1mConvergence threshold reached at iteration 43\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.668\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m521\u001b[0m - \u001b[1mExtracting policy\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.872\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m523\u001b[0m - \u001b[1mPolicy extracted\u001b[0m\n", "\u001b[32m2025-01-05 22:06:02.872\u001b[0m | \u001b[32m\u001b[1mSUCCESS \u001b[0m | \u001b[36mmdpax.solvers.value_iteration\u001b[0m:\u001b[36msolve\u001b[0m:\u001b[36m525\u001b[0m - \u001b[32m\u001b[1mValue iteration completed\u001b[0m\n" ] } ], "source": [ "# load in the checkpoint and resume\n", "problem_restored = FrozenLake(map_name=\"4x4\", is_slippery=True)\n", "solver_restored = ValueIteration(problem_restored, gamma=0.9, epsilon=1e-3, \n", " checkpoint_dir=\"checkpoints/create_custom_problem/new_checkpoints\", \n", " checkpoint_frequency=1)\n", "solver_restored.load_checkpoint(checkpoint_dir=\"checkpoints/create_custom_problem/initial_checkpoints\")\n", "solution_restored = solver_restored.solve(max_iterations = 100)" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAApYAAAK+CAYAAAAYDV5RAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAARppJREFUeJzt3X98zfX///H7sZ/OfvmxjY38TirkHb3l5yS/52dSifyopPyI9q3wflcivYmSCqPefkR+haQ3lSa/pfLznRSVkErGMAxj2+v7h4/zdmzY5rnz2s5u18tllzqvc17n9TjHy2s3r3N25rAsyxIAAABwg4rYPQAAAAC8A2EJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAIY0adJE1atXt3uMbJs5c6YcDof279/vWtakSRM1adLEtpkAFGyEJYDruhQgWX0NHTrU7vFuiMPh0IABA+weI9suf+6LFCmi6OhotWjRQmvWrLF7NACQr90DACg4Ro4cqYoVK7otK0hn6LxF8+bN1aNHD1mWpX379mny5Mlq2rSpli9frtatW9/QfX/xxReGpgRQGBGWALKtdevWqlOnTrZue+7cOfn7+6tIEV4YMa1q1arq3r2763KnTp1Us2ZNTZgw4YbD0t/f/0bHA1CIccQHcMPWrFkjh8Oh+fPn64UXXlCZMmXkdDp18uRJSdLChQtVu3ZtFS1aVOHh4erevbv++OOPTOtn9VWhQgW3bX322Wdq1KiRgoKCFBISotjYWO3atcvtNr169VJwcLD++OMPdezYUcHBwYqIiNCzzz6r9PT0HD++pUuXKjY2VtHR0QoICFDlypX1yiuvZOu+vvjiCzmdTnXt2lVpaWmSpN27d+v+++9XiRIlFBgYqDp16uiTTz7J8VyX1KhRQ+Hh4dq3b59r2apVq1zPU7FixdShQwf9+OOP172vrN5jee7cOb388suqWrWqAgMDFRUVpfvuu0979+6VZVmqUKGCOnTokOm+zp07p7CwMPXt2zfXjw1AwcIZSwDZlpycrKNHj7otCw8Pd/3/K6+8In9/fz377LNKTU2Vv7+/Zs6cqd69e+uuu+7S6NGjdfjwYb311lvauHGjtm/frmLFiunWW2/V7Nmz3e73xIkTiouLU2RkpGvZ7Nmz1bNnT7Vs2VKvvfaazpw5o/j4eDVs2FDbt293i9D09HS1bNlSdevW1euvv66VK1fqjTfeUOXKlfXUU0/l6HHPnDlTwcHBiouLU3BwsFatWqWXXnpJJ0+e1Lhx46663rJly3T//ffrwQcf1PTp0+Xj46Ndu3apQYMGKlOmjIYOHaqgoCB9+OGH6tixoxYvXqxOnTrlaDZJOn78uI4fP64qVapIklauXKnWrVurUqVKevnll3X27Fm98847atCggbZt25Yp1q8lPT1dbdu21ZdffqmHHnpIgwYN0qlTp5SQkKDvv/9elStXVvfu3TV27FgdO3ZMJUqUcK37n//8RydPnnQ7uwrAy1kAcB0zZsywJGX5ZVmWtXr1akuSValSJevMmTOu9c6fP29FRkZa1atXt86ePetavmzZMkuS9dJLL2W5vYyMDKtt27ZWcHCwtWvXLsuyLOvUqVNWsWLFrD59+rjd9q+//rLCwsLclvfs2dOSZI0cOdLttn/729+s2rVruy2TZPXv3/+aj//yx3RJ3759LafTaZ07d861LCYmxrr99tsty7KsxYsXW35+flafPn2s9PR0123uvfdeq0aNGm7rZWRkWPXr17duvvnma85xad7HHnvMOnLkiJWYmGh988031r333mtJst544w3LsiyrVq1aVmRkpJWUlORa77///a9VpEgRq0ePHq5ll/5c9+3b5/YYYmJiXJenT59uSbLGjx+faZaMjAzLsixrz549liQrPj7e7fr27dtbFSpUcN0OgPfjpXAA2TZp0iQlJCS4fV2uZ8+eKlq0qOvyli1blJiYqH79+ikwMNC1PDY2VtWqVdPy5cuz3M4rr7yiZcuWaebMmbrtttskSQkJCTpx4oS6du2qo0ePur58fHxUt25drV69OtP9PPnkk26XGzVqpF9//TXHj/vyx3Tq1CkdPXpUjRo10pkzZ7R79+5Mt583b54efPBB9e3bV1OnTnW9z/TYsWNatWqVHnjgAdf9HD16VElJSWrZsqV+/vlnt7cIXM20adMUERGhyMhI1a1bVxs3blRcXJwGDx6sQ4cOaceOHerVq5fb2cOaNWuqefPm+vTTT3P02BcvXqzw8HANHDgw03UOh0PSxfd81q1bV3PmzHFdd+zYMX322Wfq1q2b63YAvB8vhQPItr///e/X/OGdK39i/MCBA5KkW265JdNtq1Wrpg0bNmRa/vnnn2vEiBEaNmyYOnfu7Fr+888/S5KaNm2a5bZDQ0PdLgcGBioiIsJtWfHixXX8+PGrzn81u3bt0gsvvKBVq1a53jd6SXJystvlffv2qXv37urSpYveeecdt+t++eUXWZalF198US+++GKW20pMTFSZMmWuOU+HDh00YMAAORwOhYSE6Pbbb1dQUJCkaz/nt956q1asWKGUlBTX7a9n7969uuWWW+Tre+1vFz169NCAAQN04MABlS9fXgsXLtSFCxf0yCOPZGs7ALwDYQnAmMvP7OXGvn371K1bNzVv3lyjRo1yuy4jI0PSxfdZli5dOtO6V4aPj4/PDc1yyYkTJxQTE6PQ0FCNHDlSlStXVmBgoLZt26YhQ4a45rokKipKUVFR+vTTT7Vlyxa3EL9022effVYtW7bMcnuX3id5LWXLllWzZs1u4FGZ99BDD+mZZ57RnDlz9I9//EMffPCB6tSpk2XgAvBehCWAPFO+fHlJ0p49ezKdadyzZ4/rekk6e/as7rvvPhUrVkzz5s3L9DFFlStXliRFRkZ6NKrWrFmjpKQkffTRR2rcuLFr+eU/gX25wMBALVu2TE2bNlWrVq20du1a3X777ZKkSpUqSZL8/Pzy7DFc/pxfaffu3QoPD8/22Urp4vP+zTff6MKFC/Lz87vq7UqUKKHY2FjNmTNH3bp108aNGzVhwoQczw+gYOM9lgDyTJ06dRQZGakpU6YoNTXVtfyzzz7Tjz/+qNjYWNeyJ598Uj/99JOWLFmi4sWLZ7qvli1bKjQ0VP/617904cKFTNcfOXIkTx7DpTOflmW5lp0/f16TJ0++6jphYWFasWKFIiMj1bx5c+3du1fSxShu0qSJpk6dqkOHDmVaz8RjiIqKUq1atfT+++/rxIkTruXff/+9vvjiC7Vp0yZH99e5c2cdPXpUEydOzHTd5c+JJD3yyCP64Ycf9Nxzz8nHx0cPPfRQrh4DgIKLM5YA8oyfn59ee+019e7dWzExMeratavr44YqVKigZ555RpK0fPlyzZo1S507d9Z3332n7777znUfwcHB6tixo0JDQxUfH69HHnlEd955px566CFFRETot99+0/Lly9WgQYMs4yc7tmzZkumld+niZzrWr19fxYsXV8+ePfX000/L4XBo9uzZmaLqSuHh4UpISFDDhg3VrFkzbdiwQWXKlNGkSZPUsGFD1ahRQ3369FGlSpV0+PBhbdq0Sb///rv++9//5uoxXG7cuHFq3bq16tWrp8cee8z1cUNhYWF6+eWXc3RfPXr00KxZsxQXF6dvv/1WjRo1UkpKilauXKl+/fq5fX5lbGysSpYsqYULF6p169ZuHxUFoJCw94fSARQElz6WZvPmzVlef+njhhYuXJjl9QsWLLD+9re/WQEBAVaJEiWsbt26Wb///num+8/qq3z58pm21bJlSyssLMwKDAy0KleubPXq1cvasmWL6zY9e/a0goKCMs0xfPhw68rD3tW2K8l65ZVXLMuyrI0bN1p33323VbRoUSs6Otp6/vnnrRUrVliSrNWrV7vu6/KPG7rkl19+saKioqxbb73VOnLkiGVZlrV3716rR48eVunSpS0/Pz+rTJkyVtu2ba1FixZl+fxdOe/1Ph7Jsixr5cqVVoMGDayiRYtaoaGhVrt27awffvjB7TbZ+bghy7r4cUv//Oc/rYoVK1p+fn5W6dKlrfvvv9/au3dvpu3269fPkmTNnTv3ujMC8D4Oy7rOP7sBAMimZ555RtOmTdNff/0lp9Np9zgAPIz3WAIAjDh37pw++OADde7cmagECineYwkAuCGJiYlauXKlFi1apKSkJA0aNMjukQDYhLAEANyQH374Qd26dVNkZKTefvtt1apVy+6RANiE91gCAADACN5jCQAAACMISwAAABhBWAJ5bOzYsapWrVqm3ykNdzNnzpTD4dCWLVvsHsV2vXr1UoUKFQrdtrPicDhy/KHupiUlJSkoKEiffvqprXMABQFhCeShkydP6rXXXtOQIUMy/e5rFG5//vmnXn75Ze3YscPuUWz36aef2h6P11KyZEk9/vjjevHFF+0eBcj3+E4H5KHp06crLS1NXbt2tXsU5DN//vmnRowYkWVYvvfee9qzZ4/nh7LJp59+qhEjRmR53dmzZ/XCCy94eKLMnnzySW3btk2rVq2yexQgXyMsgTw0Y8YMtW/fXoGBgUbuLyMjQ+fOncvyupSUFCPbgP38/PwUEBBg9xj5QmBgoHx97f9kvFtvvVXVq1fXzJkz7R4FyNcISyCP7Nu3T999952aNWuW6brXX39d9evXV8mSJVW0aFHVrl1bixYtynQ7h8OhAQMGaM6cObr99tsVEBCgzz//3PV+xLVr16pfv36KjIxU2bJlXetNnjzZdfvo6Gj1799fJ06ccF3/9ttvy8fHx23ZG2+8IYfDobi4ONey9PR0hYSEaMiQIa5l8+fPV+3atRUSEqLQ0FDVqFFDb7311nWfj+yul5qaqri4OEVERCgoKEidOnXSkSNHMt0uLx/j1Vxvm5LUpEkTVa9eXVu3blX9+vVVtGhRVaxYUVOmTHHdZs2aNbrrrrskSb1795bD4ZDD4XBFy5Xvc9y/f78cDodef/11TZo0SZUqVZLT6VSLFi108OBBWZalV155RWXLllXRokXVoUMHHTt2zG2upUuXKjY2VtHR0QoICFDlypX1yiuvKD09/bqPOys5ub9vvvlGbdq0UfHixRUUFKSaNWu6/ux79eqlSZMmSZLreXA4HK51s3qP5fbt29W6dWuFhoYqODhY9957r77++mu321z6O7Jx48br7k9btmxRy5YtFR4e7vrzevTRRzM9jubNm+s///mP+JQ+4Ors/2cg4KW++uorSdKdd96Z6bq33npL7du3V7du3XT+/HnNnz9fXbp00bJlyxQbG+t221WrVunDDz/UgAEDFB4ergoVKrhePu3Xr58iIiL00ksvuc5YvvzyyxoxYoSaNWump556Snv27FF8fLw2b96sjRs3ys/PT40aNVJGRoY2bNigtm3bSpLWr1+vIkWKaP369a5tb9++XadPn1bjxo0lSQkJCeratavuvfdevfbaa5KkH3/8URs3brzmb1vJyXoDBw5U8eLFNXz4cO3fv18TJkzQgAEDtGDBAtdt8vIxXk12tnnJ8ePH1aZNGz3wwAPq2rWrPvzwQz311FPy9/fXo48+qltvvVUjR47USy+9pCeeeEKNGjWSJNWvX/+aM8yZM0fnz5/XwIEDdezYMY0dO1YPPPCAmjZtqjVr1mjIkCH65Zdf9M477+jZZ5/V9OnTXevOnDlTwcHBiouLU3BwsFatWqWXXnpJJ0+e1Lhx46653axk9/4SEhLUtm1bRUVFadCgQSpdurR+/PFHLVu2TIMGDVLfvn31559/KiEhQbNnz77udnft2qVGjRopNDRUzz//vPz8/DR16lQ1adJEa9euVd26dd1uf739KTExUS1atFBERISGDh2qYsWKaf/+/froo48ybbt27dp68803tWvXLlWvXj3HzxlQKFgA8sQLL7xgSbJOnTqV6bozZ864XT5//rxVvXp1q2nTpm7LJVlFihSxdu3a5bZ8xowZliSrYcOGVlpammt5YmKi5e/vb7Vo0cJKT093LZ84caIlyZo+fbplWZaVnp5uhYaGWs8//7xlWZaVkZFhlSxZ0urSpYvl4+Pjmnn8+PFWkSJFrOPHj1uWZVmDBg2yQkND3baZHdlZ79JjatasmZWRkeFa/swzz1g+Pj7WiRMnPPIYs5LdbVqWZcXExFiSrDfeeMO1LDU11apVq5YVGRlpnT9/3rIsy9q8ebMlyZoxY0am7fXs2dMqX7686/K+ffssSVZERITrebAsyxo2bJglybrjjjusCxcuuJZ37drV8vf3t86dO+daduU+Z1mW1bdvX8vpdLrd7sptX0127i8tLc2qWLGiVb58+UzP7+V/xv3797eu9u1IkjV8+HDX5Y4dO1r+/v7W3r17Xcv+/PNPKyQkxGrcuLFrWXb3pyVLlliSrM2bN1/3MX/11VeWJGvBggXXvS1QWPFSOJBHkpKS5Ovrq+Dg4EzXFS1a1PX/x48fV3Jysho1aqRt27Zlum1MTIxuu+22LLfRp08f+fj4uC6vXLlS58+f1+DBg91+Cr1Pnz4KDQ3V8uXLJUlFihRR/fr1tW7dOkkXzx4mJSVp6NChsixLmzZtknTxDF/16tVVrFgxSVKxYsWUkpKihISEHD0XOVnviSeecHsptFGjRkpPT9eBAwc88hizkt1tXuLr66u+ffu6Lvv7+6tv375KTEzU1q1br/scXE2XLl0UFhbmunzp7Fz37t3d3odYt25dnT9/Xn/88Ydr2eX73KlTp3T06FE1atRIZ86c0e7du3M8S3bub/v27dq3b58GDx6c6fm9/M84u9LT0/XFF1+oY8eOqlSpkmt5VFSUHn74YW3YsEEnT550W+d6+9OluZYtW6YLFy5cc/vFixeXJB09ejTHswOFBWEJ2GDZsmW6++67FRgYqBIlSigiIkLx8fFKTk7OdNuKFSte9X6uvO7SN8tbbrnFbbm/v78qVarkul66+A1269atOnv2rNavX6+oqCjdeeeduuOOO1wvFW/YsMH1Mq108aX3qlWrqnXr1ipbtqweffRRff7559d9vDlZr1y5cm6XL30zP378uEceY1Zysk1Jio6OVlBQkNuyqlWrSrr4fsncuvK5uRSZN910U5bLLz1n0sWXkDt16qSwsDCFhoYqIiJC3bt3l6Qs97vryc797d27V5KMvWx85MgRnTlzJtOfg3Txh2syMjJ08OBBt+XX259iYmLUuXNnjRgxQuHh4erQoYNmzJih1NTUTNuw/u+9lbmJYqCwICyBPFKyZEmlpaXp1KlTbsvXr1/v+knxyZMn69NPP1VCQoIefvjhLH8o4PIzQzm57noaNmyoCxcuaNOmTVq/fr0rrho1aqT169dr9+7dOnLkiFt0RUZGaseOHfrkk0/Uvn17rV69Wq1bt1bPnj2vua2crHf5GdjLZfXc5MVjzM+u9txc7zk7ceKEYmJi9N///lcjR47Uf/7zHyUkJLje75rTD+83fX956XrPjcPh0KJFi7Rp0yYNGDBAf/zxhx599FHVrl1bp0+fdlvnUoyGh4fn7dBAAUZYAnmkWrVqki7+dPjlFi9erMDAQK1YsUKPPvqoWrduneVPjudG+fLlJSnTZyCeP39e+/btc10vSX//+9/l7++v9evXu0VX48aN9c033+jLL790Xb6cv7+/2rVrp8mTJ2vv3r3q27evZs2apV9++eWas+V2PTse441sU7r4GZVXfvzTTz/9JEmun/b25FmvNWvWKCkpSTNnztSgQYPUtm1bNWvWzHX2Lq/ur3LlypKk77///pr3l93nIiIiQk6nM8vP+Ny9e7eKFCmS6extdt1999169dVXtWXLFs2ZM0e7du3S/Pnz3W5z6e/yrbfemqttAIUBYQnkkXr16klSpl9R6OPjI4fD4faxLPv379fHH398w9ts1qyZ/P399fbbb7ud4Zs2bZqSk5PdfuI8MDBQd911l+bNm6fffvvN7Wze2bNn9fbbb6ty5cqKiopyrZOUlOS2vSJFiqhmzZqSlOVLhze6nh2P8Ua3KUlpaWmaOnWq6/L58+c1depURUREqHbt2pLkeqn8yo8ryguXztpdPvv58+c1efLkPL2/O++8UxUrVtSECRMyPc7L183uc+Hj46MWLVpo6dKlbm8pOHz4sObOnauGDRsqNDQ0R4/l+PHjmc6G16pVS1LmfXPr1q0KCwvT7bffnqNtAIUJHzcE5JFKlSqpevXqWrlypdtn4sXGxmr8+PFq1aqVHn74YSUmJmrSpEmqUqWKvvvuuxvaZkREhIYNG6YRI0aoVatWat++vfbs2aPJkyfrrrvucr0H7pJGjRppzJgxCgsLU40aNSRdfNn6lltu0Z49e9SrVy+32z/++OM6duyYmjZtqrJly+rAgQN65513VKtWrWuexcntenY8RhPbjI6O1muvvab9+/eratWqWrBggXbs2KF3333X9bFElStXVrFixTRlyhSFhIQoKChIdevWveZ7anOrfv36Kl68uHr27Kmnn35aDodDs2fPzvXnMWb3/ooUKaL4+Hi1a9dOtWrVUu/evRUVFaXdu3dr165dWrFihSS5Yvvpp59Wy5Yt5ePjo4ceeijLbY8aNUoJCQlq2LCh+vXrJ19fX02dOlWpqakaO3Zsjh/L+++/r8mTJ6tTp06qXLmyTp06pffee0+hoaFq06aN220TEhLUrl073mMJXIvnfxAdKDzGjx9vBQcHZ/polmnTplk333yzFRAQYFWrVs2aMWOGNXz48EwfuSLJ6t+/f6b7vfRRKlf7iJSJEyda1apVs/z8/KxSpUpZTz31VJYfp7N8+XJLktW6dWu35Y8//rglyZo2bZrb8kWLFlktWrSwIiMjLX9/f6tcuXJW3759rUOHDl3zecjOeld7TKtXr7YkWatXr/bIY7yW7GwzJibGuv32260tW7ZY9erVswIDA63y5ctbEydOzHR/S5cutW677TbL19fX7aOHrvZxQ+PGjcvyuVm4cKHb8qyey40bN1p33323VbRoUSs6Otp6/vnnrRUrVmR6brP7cUPZvT/LsqwNGzZYzZs3t0JCQqygoCCrZs2a1jvvvOO6Pi0tzRo4cKAVERFhORwOt78HuuLjhizLsrZt22a1bNnSCg4OtpxOp3XPPfdYX3311XWfg8ufs0szbtu2zeratatVrlw5KyAgwIqMjLTatm1rbdmyxW29H3/80ZJkrVy58rrPDVCYOSyLXyEA5JXk5GRVqlRJY8eO1WOPPWb3OPCAJk2a6OjRo9d9XyEKlsGDB2vdunXaunUrZyyBa+A9lkAeCgsL0/PPP69x48blq5+UBZB9SUlJ+ve//61Ro0YRlcB1cMYSAAzijCWAwowzlgAAADDihsJyzJgxcjgcGjx4sKFxAKBgW7NmDWcrARRauQ7LzZs3a+rUqa7PogMAAEDhlqvPsTx9+rS6deum9957T6NGjbrmbVNTU90+ZDYjI0PHjh1TyZIleRM0AABAPmRZlk6dOqXo6GgVKZL985C5Csv+/fsrNjZWzZo1u25Yjh49WiNGjMjNZgAAAGCjgwcPqmzZstm+fY7Dcv78+dq2bZs2b96crdsPGzZMcXFxrsvJyckqV66cVu9arbDiYTndPJBt6Wnp+nnDz6pxdwP5+PBLppB30tPTtPPrjexryHPsa/CU5BPHVe+O6goJCcnRejnaKw8ePKhBgwYpISFBgYGB2VonICBAAQEBmZaHFQ9TWEnCEnkn/UK6nE6nihUvIR9fDsDIO+lpaexr8Aj2NXhaTt+2mKO9cuvWrUpMTNSdd97pWpaenq5169Zp4sSJSk1NlY+PT44GAAAAgHfIUVjee++92rlzp9uy3r17q1q1ahoyZAhRCQAAUIjlKCxDQkJUvXp1t2VBQUEqWbJkpuUAAAAoXPjNOwAAADDiht/5u2bNGgNjAAAAoKDjjCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhGU+l3QkSS8OeFENqzTUrSG3qm65uuoV20tbvtpi92jwIoP79lF0SNFMX/v27rV7NHgZ9jV4CvuaPXztHgDX1v+h/rpw/oLG/XucylUsp6OJR/XV6q90IumE3aPBy9zTvIXejJ/qtqxkeIRN08Cbsa/BU9jXPI+wzMdOnjipzRs2a27CXNVtXFeSVKZ8Gd1x1x02TwZv5O/vr8hSpe0eA4UA+xo8hX3N83gpPB9zBjsVFBykhE8SlJqaavc4AAAA10RY5mO+vr4a+++x+uiDj/S3yL+pS5Muev3F17V75267R4MXWvn5Z6pSOtz19cQjD9s9ErwU+xo8hX3N83gpPJ9r1amV7ml9jzZv2Kzt327X2hVr9e4b7+pfU/6l+3vcb/d48CL1G8dozJtvuy47g5w2TgNvxr4GT2Ff8zzCsgAICAxQw2YN1bBZQw38x0ANe3KY3nrlLcISRjmdTlWsXNnuMVAIsK/BU9jXPI+XwgugKtWq6GzKWbvHAAAAcMMZy3zseNJxDXh4gLr07KJqNaopKDhIO7ft1Lvj31Wzds3sHg8AAMANYZmPOYOdqnVXLU1/e7p++/U3pV1IU1TZKD346IPqN6Sf3eMBAAC4cViWZXlygydPnlRYWJi2/blNYSXDPLlpFDLpF9L1w8ofVDvmXvn48m8o5J30tDRtXfsl+xryHPsaPOXEsWO6rXwZJScnKzQ0NNvr8R5LAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsATg8vtvBzT2lRH6av06u0cBACM4rnkWYQlAkvT7wd90f2wrTRg7Rt3v66B1q1fZPRIA3BCOa55HWALQH78fVJfYVvpt/35J0rlz59Trwfu1cd1aewcDgFziuGYPwhIo5P7843d1iW2lA/v2uS0/d/asenS5T19v2GDTZACQOxzX7ENYAoXYoT//UJfYVtr/66/y8/PT3+rUkSTdVL68SkdF6eyZM3qkSyd989VGmycFgOzhuGYvwhIopI4eSVSXNq20b+9e+fn5acr7H6h+oxhJUkREpD5c9rkiS5VWyunTeuT+Ttq+ZbPNEwPAtXFcsx9hmc9kZGRozLAx+n3/73aPAi8XVqy4bq5WTb6+voqfMUut27V3u75K1apauOwzhUdEKiq6jG4qX96mSVHQZWRk6JUX/qGDBw7YPQq8HMc1+/naPQD+JyMjQ8/3eV5LPliiTxd/qo/Wf6TwUuF2jwUv5efnp6mz5mj75m9Vt0HDLG9zc7VqWrj8MxUvUULhEZEenhDeICMjQ888+YQWzpuj/yxZrOWr1ykispTdY8FLcVyzH2GZT1welZJ0d8zdKhFRwuap4O38/f2vevC95JZbb/PQNPA2l0elJNVvFKOS4RE2TwVvx3HNXrwUng9cGZWdunfSmKljVKQIfzwACqYro7JL124aP3kKxzXAy/E33GZXRmWHrh009r2xHHwBFFhXRuV9Dz6kN6e8y3ENKAR4Kdxmb4540xWVkrR03lItnbc01/fXqlMrTZo/ycRoAJAr40aNdEWlJH20YL4+WjA/1/cX26Gj3vtgnonRAOQx/vlos1MnTtk9AgAYlZx8wu4RANiEM5Y2e/5fz+unH37SN+u+kXTxpfD+Q/vn+v6CQ4NNjQYAufLCyFe154cftGnDekkXXwof9NzQXN9fSGiIqdEA5DHC0mbOIKemLZ2mxzo8pm/WfaOl85aqYtWKGviPgXaPBgC54gwK0uzFH+uRzh21acN6fbRgvirfXFXPDBlm92gA8hgvhecDRZ1FNW3pNNWNqStJmjBigiaOnmjzVACQe06nU7MXf6z6jRpLuvi+ywljx9g8FYC8RljmE0WdRTXt42m6u8ndkqRpE6Yp8VCizVMBQO45nU7NWrREDRpf/JV6U995S4f/OmTzVADyEmGZjxR1FtW/l/xbzds318xlMxUZxW8EAFCwOZ1Ovb/wI7Vq205zl3yiUqWj7B4JQB7iPZb5TFFnUU1ZOMXuMQDAGKfTqenzPrR7DAAewBlLAAAAGEFYAgAAwAjCEoDLP0eO0p+nzmrZ6nV2jwIARnBc8yzCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADAiR2EZHx+vmjVrKjQ0VKGhoapXr54+++yzvJoNAAAABUiOwrJs2bIaM2aMtm7dqi1btqhp06bq0KGDdu3alVfzAQAAoIDwzcmN27Vr53b51VdfVXx8vL7++mvdfvvtWa6Tmpqq1NRU1+WTJ09KktLT0pV+IT2n8wLZlp6W/n//TbN5Eni7S/sY+xryGvsaPCU9PXf7WI7C0n2D6Vq4cKFSUlJUr169q95u9OjRGjFiRKblP2/4WU6nM7ebB7Jtx8a1do+AQoJ9DZ7Cvoa8dubMmVyt57Asy8rJCjt37lS9evV07tw5BQcHa+7cuWrTps1Vb5/VGcubbrpJO389oGLFS+RqaCA70tPStGPjWjWX5Odw2D0OvNgFy1KCpFoNYuTjm+t/rwPXxXENnpKUkqKohx9WcnKyQkNDs71ejo+At9xyi3bs2KHk5GQtWrRIPXv21Nq1a3XbbbdlefuAgAAFBARkWu7j48sBGB7h53BwAEbesyz5+HJcg2dwXENey+3+leMjoL+/v6pUqSJJql27tjZv3qy33npLU6dOzdUAAAAA8A43/DmWGRkZbi91AwAAoHDK0RnLYcOGqXXr1ipXrpxOnTqluXPnas2aNVqxYkVezQcAAIACIkdhmZiYqB49eujQoUMKCwtTzZo1tWLFCjVv3jyv5gMAAEABkaOwnDZtWl7NAQAAgAKO3xUOAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGGZjw3u20fRIUUzfe3bu9fu0eBFek2YoI6vvppp+ZqdO+Vo314nTp+2YSp4K45r8ASOa/bxtXsAXNs9zVvozfipbstKhkfYNA0A3DiOa4D3IizzOX9/f0WWKm33GABgDMc1wHvxUjgAAACM4IxlPrfy889UpXS463LT5i307uy5Nk4Eb7Rs82YFP/CA27L0jAybpoG347gGT+C4Zg/CMp+r3zhGY95823XZGeS0cRp4q3tq1FD8U0+5Lfvmp5/Uffx4myaCN+O4Bk/guGYPwjKfczqdqli5st1jwMsFBQaqSnS027Lfk5JsmgbejuMaPIHjmj14jyUAAACMICwBAABgBGEJAAAAIxyWZVme3ODJkycVFhamHw78oWIlSnhy0yhk0tPStHXtl2rjcMjP4bB7HHixC5alTy1LtWPulY8vb11H3uG4Bk9JSklReNeuSk5OVmhoaLbX44wlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYARhmc9kZGTolRf+oYMHDtg9CgAYwXENKDwIy3wkIyNDzzz5hOLfelOd27TQkcTDdo8EADeE4xpQuBCW+cSlg+/CeXMkSfUbxahkeITNUwFA7nFcAwofwjIfuPLg26VrN42fPEVFivDHA6Bg4rgGFE78DbfZlQff+x58SG9OeZeDL4ACi+MaUHj52j1AYTdu1EjXwVeSPlowXx8tmJ/r+4vt0FHvfTDPxGgAkCsc14DCi38+2iw5+YTdIwCAURzXgMKLM5Y2e2Hkq9rzww/atGG9pIsvGQ16bmiu7y8kNMTUaACQKxzXgMKLsLSZMyhIsxd/rEc6d9SmDev10YL5qnxzVT0zZJjdowFArnBcAwovXgrPB5xOp2Yv/lj1GzWWdPH9SRPGjrF5KgDIPY5rQOFEWOYTTqdTsxYtUYPGMZKkqe+8pcN/HbJ5KgDIPY5rQOFDWOYjTqdT7y/8SK3attPcJZ+oVOkou0cCgBvCcQ0oXHiPZT7jdDo1fd6Hdo8BAMZwXAMKD85YAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMyFFYjh49WnfddZdCQkIUGRmpjh07as+ePXk1GwAAAAqQHIXl2rVr1b9/f3399ddKSEjQhQsX1KJFC6WkpOTVfAAAACggfHNy488//9zt8syZMxUZGamtW7eqcePGWa6Tmpqq1NRU1+WTJ09KktLT05SelpbTeYFsu7R/XbAsmyeBt7u0j3FMQ17juAZPye0+lqOwvFJycrIkqUSJEle9zejRozVixIhMy3d+vVFOp/NGNg9kS4IkcRCGB+zYuNbuEVBIcFxDXjuTy/UclpW7PTMjI0Pt27fXiRMntGHDhqveLqszljfddJN2/npAxYpfPUiBG5WelqYdG9eqVoMY+fje0L+hgGtiX4OnXNrXmkvyczjsHgdeLCklRVEPP6zk5GSFhoZme71cHwH79++v77///ppRKUkBAQEKCAjItNzHx5cDMDzCx5d9DZ7BvgZP8XM4CEvkqdzuX7k6Ag4YMEDLli3TunXrVLZs2VxtGAAAAN4lR2FpWZYGDhyoJUuWaM2aNapYsWJezQUAAIACJkdh2b9/f82dO1dLly5VSEiI/vrrL0lSWFiYihYtmicDAgAAoGDI0edYxsfHKzk5WU2aNFFUVJTra8GCBXk1HwAAAAqIHL8UDgAAAGSF3xUOAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGGZjw3u20fRIUUzfe3bu9fu0eBl2NfgKexr8IReEyao46uvZlq+ZudOOdq314nTp22YqnDwtXsAXNs9zVvozfipbstKhkfYNA28GfsaPIV9DfBehGU+5+/vr8hSpe0eA4UA+xo8hX0N8F68FA4AAAAjOGOZz638/DNVKR3uuty0eQu9O3uujRPBW7GvwVPY1+AJyzZvVvADD7gtS8/IsGmawoOwzOfqN47RmDffdl12BjltnAbejH0NnsK+Bk+4p0YNxT/1lNuyb376Sd3Hj7dposKBsMznnE6nKlaubPcYKATY1+Ap7GvwhKDAQFWJjnZb9ntSkk3TFB68xxIAAABGEJYAAAAwgrAEAACAEbzHMh+bMPU9u0dAIcG+Bk9hX4MnzBw8OMvlTWrUkPXJJ54dppDhjCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGEFYAgAAwAjCEgAAAEYQlgAAADCCsAQAAIARhCUAAACMICwBAABgBGEJAAAAIwhLAAAAGOFr9wC4vt9/O6C5789Uwyb3qH6jxnaPAy+2bvUqfbvpK0VFl1G3Xr3tHgcAUMBwxjKf+/3gb7o/tpUmjB2j7vd10LrVq+weCV5s/epVGj/6Vc17f4bdowAACiDCMh/74/eD6hLbSr/t3y9JOnfunHo9eL82rltr72AAAABZICzzqT//+F1dYlvpwL59bsvPnT2rHl3u09cbNtg0GQAAQNYIy3zo0J9/qEtsK+3/9Vf5+fnpb3XqSJJuKl9epaOidPbMGT3SpZO++WqjzZMCAAD8D2GZzxw9kqgubVpp39698vPz05T3P1D9RjGSpIiISH247HNFliqtlNOn9cj9nbR9y2abJwYAALiIsMxnwooV183VqsnX11fxM2apdbv2btdXqVpVC5d9pvCISEVFl9FN5cvbNCkAZE9GRoZeeeEfOnjggN2jAMhjfNxQPuPn56eps+Zo++ZvVbdBwyxvc3O1alq4/DMVL1FC4RGRHp4QALIvIyNDzzz5hBbOm6P/LFms5avXKSKylN1jAcgjhGU+5O/vf9WovOSWW2/z0DQAkDuXR6Uk1W8Uo5LhETZPBSAv8VI4AMC4K6OyS9duGj95iooU4dsO4M34Gw4AMOrKqLzvwYf05pR3iUqgEOClcACAUeNGjXRFpSR9tGC+PlowP9f3F9uho977YJ6J0QDkMf75CAAwKjn5hN0jALAJZywBAEa9MPJV7fnhB23asF7SxZfCBz03NNf3FxIaYmo0AHmMsAQKsfVrVmvi+Nc1fe4CBQUHZ7resiwNe2aQKlWpoicGPG3DhCiInEFBmr34Yz3SuaM2bVivjxbMV+Wbq+qZIcPsHg1AHuOlcKCQOvTnH+r14P1av3qVut3XQWdSUjLd5oXn4jRr2nt6edgQrfpihQ1ToqByOp2avfhj1W/UWNLF911OGDvG5qkA5DXCEiikoqLL6PkXXpIkfbvpK3Xv3FFnzvwvLocPfU4zpk6RJHW4v4ti7m1my5wouJxOp2YtWqIGjS/+Wtqp77ylw38dsnkqAHmJl8KBQqzvwEFKT0/XqBf/qa83btDmrzdJkv67fZu2/d/voW/X6T698950+fj42DkqCiin06n3F36kAY/10sD/95xKlY6yeyQAeYiwBAq5foPjlJGRoX8Nf1Hp6emS5Ppvm/YdNGn6+/L15VCB3HM6nZo+70O7xwDgAbwUDkAD4p7VsOEj3Za1jG2r+JmziUoAQLYRlgAkSQOffU5DXnpZktSsVRtNnTVHfn5+9g4FAChQCMsC4J8jR+nPU2e1bPU6u0eBlxv03BD9eeqsZi1cLH9/f7vHAQAUMIQlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYESOw3LdunVq166doqOj5XA49PHHH+fBWAAAAChochyWKSkpuuOOOzRp0qS8mAcAAAAFlG9OV2jdurVat26d7dunpqYqNTXVdfnkyZOSpPT0NKWnpeV080C2Xdq/2M+Q19jX4CmX9rELlmXzJPB2ud3HchyWOTV69GiNGDEi0/KdX2+U0+nM680D2rFxrd0joJBgX4OnJEgScYk8dCaX6+V5WA4bNkxxcXGuyydPntRNN92keySVdDjyevMoxC5YlhIk1WoQIx/fPN/VUYilp6Vpx8a17GvIc5f2teb1JT9fvoci7yQdz916eX4EDAgIUEBAQKblfg6H/AhL5DXLko+vL9/s4RHsa/AUP18HYYk85eeTu/2LjxsCAACAEYQlAAAAjMjxazanT5/WL7/84rq8b98+7dixQyVKlFC5cuWMDgcAAICCI8dhuWXLFt1zzz2uy5d+MKdnz56aOXOmscEAAABQsOQ4LJs0aSKLjzgAAADAFXiPJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMIKwBAAAgBGEJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMIKwBAAAgBGEJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMIKwBAAAgBGEJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMIKwBAAAgBGEJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMIKwBAAAgBGEJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMIKwBAAAgBGEJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMIKwBAAAgBGEJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMIKwBAAAgBGEZT7Va8IEdXz11UzL1+zcKUf79jpx+rQNU8FbDe7bR9EhRTN97du71+7R4GXY1+Apfx0+rkHPv6cqdzyhwIjOKlX5ETVo/rzi//2pzpxJtXs8r+Vr9wAA8od7mrfQm/FT3ZaVDI+waRp4M/Y15LVf9/2lBi2GqFhYkP41vIdq3F5eAf5+2vnDfr074wuViS6p9m3q2j2mVyIsAUiS/P39FVmqtN1joBBgX0Ne6xcXL19fH21ZO15BQYGu5ZUqllaH2LtlWZaN03k3XgoHAABeIynppL5YtUP9+7Rxi8rLORwOD09VeHDGMh9btnmzgh94wG1ZekaGTdPA2638/DNVKR3uuty0eQu9O3uujRPBW7GvIS/98ushWZalW24u47Y8vEI3nUu9IEnq36eNXhvZy4bpvB9hmY/dU6OG4p96ym3ZNz/9pO7jx9s0EbxZ/cYxGvPm267LziCnjdPAm7GvwQ7frn5DGRkZ6vb4eKX+X2DCPMIyHwsKDFSV6Gi3Zb8nJdk0Dbyd0+lUxcqV7R4DhQD7GvJSlUpRcjgc2vPzH27LK1W8+L7eokX97Rir0OA9lgAAwGuULBmq5vfU0sR3lysl5Zzd4xQ6hCUAAPAqk8c/qbS0dNWJidOCxev1456D2vPz7/pg/mrt/ul3+fiQP3mFl8IBAIBXqVwpSts3TNC/Xl+oYSNm6fc/khQQ4KfbbrlJzz7dSf0eb2P3iF7LYXn4w5xOnjypsLAwHZ03TyWDgjy5aRQyFyxLn1qWasfcKx9f/g2FvJOelqata79kX0Oeu7SvtWnskJ8vH5mDvJN0LEXhFboqOTlZoaGh2V6Pc8EAAAAwgrAEAACAEYQlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYISv3QMAAAqfdatX6dtNXykquoy69ept9zgADOGMJQDA49avXqXxo1/VvPdn2D0KAIMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEcE2pqal68fn/p6SjR+0eBQCQzxGWAK6pT/eHNS1+sh5s14a4BABcE2EJ4JpiO3RUkSJF9MP3O/VguzY6lpRk90gAgHyKsARwTQ92f0Tj3pksh8OhH77fqQfatiYuAQBZIiwBXFfXHj019u1J/4tLzlwCALLAr3QEColZ097T0MFPG7mvH3Z+p94PddHShFVG7g8A4B04YwkgV04mJ9s9AgAgn+GMJVBIdOjcRfUaNs71+uvXrNKLz/0/WZal0lFR+vec+QanAwB4A8ISKCTCihVTWLFiuVp3/ZrVGvXiP11RuXD5ClW++WazA8KrrV+zWhPHv67pcxcoKDg40/WWZWnYM4NUqUoVPTHAzFs2AHgeYQngmjasXaOeD3TWubNniUrkyqE//1CvB+/X2TNn1O2+Dpq75JNMt3nhuTjNmvaeJKlK1VvUtEVLT48JwADeYwngmsaNGklU4oZERZfR8y+8JEn6dtNX6t65o86cSXFdP3zoc5oxdYokqcP9XRRzbzNb5gRw4zhjCeCaZi5YpIF9HtWIMeOISuRa34GDlJ6erlEv/lNfb9ygzV9vkiT9d/s2bduyWZLUrtN9eue96fLx8bFzVAA3gLAEcE3FS5TQB4s/tnsMeIF+g+OUkZGhfw1/Uenp6ZLk+m+b9h00afr78vXl2xJQkPFSOADAYwbEPathw0e6LWsZ21bxM2cTlYAXICwBAB418NnnNOSllyVJzVq10dRZc+Tn52fvUACMICwBAB436Lkh+vPUWc1auFj+/v52jwPAEMISAAAARhCWAAAAMIKwBAAAgBGEJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMIKwBAAAgBGEJQAAAIwgLAEAAGAEYQkAAAAjCEsAAAAYQVgCAADACMISAAAARhCWAAAAMCJXYTlp0iRVqFBBgYGBqlu3rr799lvTcwEAAKCAyXFYLliwQHFxcRo+fLi2bdumO+64Qy1btlRiYmJezAcAAIACwjenK4wfP159+vRR7969JUlTpkzR8uXLNX36dA0dOjTT7VNTU5Wamuq6nJycLEk6lpKS25mBbLlgWToj6cTxY/LxyfGuDmRbenqazpw5w76GPHdpX0s6Lvn5OOweB17s2ImLnWZZVo7Wy9ER8Pz589q6dauGDRvmWlakSBE1a9ZMmzZtynKd0aNHa8SIEZmWV3388RwNCgAAAM9KSkpSWFhYtm+fo7A8evSo0tPTVapUKbflpUqV0u7du7NcZ9iwYYqLi3NdPnHihMqXL6/ffvstR4MCOXXy5EnddNNNOnjwoEJDQ+0eB16MfQ2ewr4GT0lOTla5cuVUokSJHK2X56/ZBAQEKCAgINPysLAw/lLAI0JDQ9nX4BHsa/AU9jV4SpEiOftxnBzdOjw8XD4+Pjp8+LDb8sOHD6t06dI52jAAAAC8S47C0t/fX7Vr19aXX37pWpaRkaEvv/xS9erVMz4cAAAACo4cvxQeFxennj17qk6dOvr73/+uCRMmKCUlxfVT4tcTEBCg4cOHZ/nyOGAS+xo8hX0NnsK+Bk/J7b7msHL6c+SSJk6cqHHjxumvv/5SrVq19Pbbb6tu3bo5vRsAAAB4kVyFJQAAAHAlflc4AAAAjCAsAQAAYARhCQAAACMISwAAABjh0bCcNGmSKlSooMDAQNWtW1fffvutJzePQmLdunVq166doqOj5XA49PHHH9s9ErzQ6NGjdddddykkJESRkZHq2LGj9uzZY/dY8ELx8fGqWbOm67ft1KtXT5999pndY6EQGDNmjBwOhwYPHpztdTwWlgsWLFBcXJyGDx+ubdu26Y477lDLli2VmJjoqRFQSKSkpOiOO+7QpEmT7B4FXmzt2rXq37+/vv76ayUkJOjChQtq0aKFUlJS7B4NXqZs2bIaM2aMtm7dqi1btqhp06bq0KGDdu3aZfdo8GKbN2/W1KlTVbNmzRyt57GPG6pbt67uuusuTZw4UdLF39hz0003aeDAgRo6dKgnRkAh5HA4tGTJEnXs2NHuUeDljhw5osjISK1du1aNGze2exx4uRIlSmjcuHF67LHH7B4FXuj06dO68847NXnyZI0aNUq1atXShAkTsrWuR85Ynj9/Xlu3blWzZs3+t+EiRdSsWTNt2rTJEyMAQJ5KTk6WdPEbPpBX0tPTNX/+fKWkpPCrlJFn+vfvr9jYWLduy64c/0rH3Dh69KjS09NVqlQpt+WlSpXS7t27PTECAOSZjIwMDR48WA0aNFD16tXtHgdeaOfOnapXr57OnTun4OBgLVmyRLfddpvdY8ELzZ8/X9u2bdPmzZtztb5HwhIAvFn//v31/fffa8OGDXaPAi91yy23aMeOHUpOTtaiRYvUs2dPrV27lriEUQcPHtSgQYOUkJCgwMDAXN2HR8IyPDxcPj4+Onz4sNvyw4cPq3Tp0p4YAQDyxIABA7Rs2TKtW7dOZcuWtXsceCl/f39VqVJFklS7dm1t3rxZb731lqZOnWrzZPAmW7duVWJiou68807XsvT0dK1bt04TJ05UamqqfHx8rnkfHnmPpb+/v2rXrq0vv/zStSwjI0Nffvkl7xEBUCBZlqUBAwZoyZIlWrVqlSpWrGj3SChEMjIylJqaavcY8DL33nuvdu7cqR07dri+6tSpo27dumnHjh3XjUrJgy+Fx8XFqWfPnqpTp47+/ve/a8KECUpJSVHv3r09NQIKidOnT+uXX35xXd63b5927NihEiVKqFy5cjZOBm/Sv39/zZ07V0uXLlVISIj++usvSVJYWJiKFi1q83TwJsOGDVPr1q1Vrlw5nTp1SnPnztWaNWu0YsUKu0eDlwkJCcn0PvGgoCCVLFky2+8f91hYPvjggzpy5Iheeukl/fXXX6pVq5Y+//zzTD/QA9yoLVu26J577nFdjouLkyT17NlTM2fOtGkqeJv4+HhJUpMmTdyWz5gxQ7169fL8QPBaiYmJ6tGjhw4dOqSwsDDVrFlTK1asUPPmze0eDcjEY59jCQAAAO/G7woHAACAEYQlAAAAjCAsAQAAYARhCQAAACMISwAAABhBWAIAAMAIwhIAAABGEJYAAAAwgrAEAACAEYQlAAAAjCAsAQAAYMT/B/FNV7C28TGaAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_policy(problem_restored, solution_restored.policy)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The plot shows the same policy as when we solved the same problem uninterrupted.\n", "\n", "See the source code for the [forest management problem](https://github.com/joefarrington/mdpax/blob/main/src/mdpax/problems/forest.py) for a simple example of a problem defined in a module with a Hydra configuration." ] }, { "cell_type": "markdown", "metadata": { "id": "next_steps" }, "source": [ "## Next Steps\n", "\n", "Try modifying the problem:\n", "1. Create custom maps with different layouts\n", "2. Modify the reward structure (e.g., small negative reward for each step)\n", "3. Add new features like varying slipperiness or wind effects\n", "\n", "For more examples and documentation:\n", "- [MDPax Documentation](https://mdpax.readthedocs.io/en/latest/)\n", "- [Original FrozenLake Environment](https://www.gymlibrary.dev/environments/toy_text/frozen_lake/)" ] } ], "metadata": { "colab": { "name": "create_custom_problem.ipynb", "provenance": [], "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.4" } }, "nbformat": 4, "nbformat_minor": 2 }