Add Term-I Projects

Four projects completed in the first term. Does not include labs and
exercises.

License.txt

@@ -0,0 +1,36 @@
+PROJECT LICENSE
+
+Copyright (c) 2018 Pranav Suri
+
+The projects in this repository were submitted by Pranav Suri as part of the
+Nanodegree At Udacity.
+
+As part of Udacity Honor code, your submissions must be your own work, hence
+submitting this project as yours will cause you to break the Udacity Honor Code
+and the suspension of your account.
+
+I, the author of the project, allow you to check the code as a reference, but
+if you submit it, it's your own responsibility if you get expelled.
+
+Besides the above notice, the following license applies and this license notice
+must be included in all works derived from this project.
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,25 @@
+# Udacity: Artificial Intelligence Nanodegree
+
+This repository contains the projects completed as a part of Udacity's [Artificial Intelligence Nanodegree](https://in.udacity.com/course/artificial-intelligence-nanodegree--nd889).
+
+## Contents
+
+### Term-1: Foundations of AI
+
+#### P1: Diagonal Sudoku Solver
+In this project, an extension of a Sudoku solving agent is developed. The project is capable of solving any Classic or Diagonal Sudoku puzzle using three ideas: Constraint Propagation, Search (DFS) and Naked-Twins Strategy.
+
+#### P2: Game Playing Agent (Isolation)
+This game-playing agent uses techniques such as Iterative Deepening, Minimax, and Alpha-Beta Pruning to compete in the game of Isolation (a two-player discrete competitive game with perfect information). The different heuristics used are then compared to find the best heuristic.
+
+#### P3: Implementing a Planning Search
+A planning agent was implemented to solve deterministic logistics-planning problems for an air cargo transport system. The underlying logic makes use of a planning graph and A* search with automatically generated heuristics. The results/performance are then compared against several uninformed non-heuristic search methods (BFS, DFS, etc.)
+
+#### P4: American Sign Language Recognizer
+HMMs (Hidden Markov Models) are used to recognize words communicated using the American Sign Language (ASL). The system is trained on a dataset of videos that have been pre-processed and annotated and then tested on novel sequences.
+
+### Term-2: Deep Learning & Applications
+Yet to enroll (as of 15th Feb 2018).
+
+## License
+[Modified MIT License © Pranav Suri](/License.txt)

Term-I – AI Foundations/01 - Sudoku Solver/Classic_Sudoku.py

@@ -0,0 +1,169 @@
+#Row-Column Labels
+rows = 'ABCDEFGHI'
+cols = '123456789'
+
+def cross(a, b):
+    """Make 2-char combos from two strings.
+    Arguments: Two strings
+    """
+    return [s+t for s in a for t in b]
+
+#Defining Boxes
+boxes = cross(rows, cols)
+
+#Defining Units
+row_units = [cross(r, cols) for r in rows]
+column_units = [cross(rows, c) for c in cols]
+square_units = [cross(rs, cs) for rs in ('ABC','DEF','GHI') for cs in ('123','456','789')]
+unitlist = row_units + column_units + square_units
+
+units = dict((s, [u for u in unitlist if s in u]) for s in boxes)
+peers = dict((s, set(sum(units[s],[]))-set([s])) for s in boxes)
+
+#Testing Example
+#grid = '..3.2.6..9..3.5..1..18.64....81.29..7.......8..67.82....26.95..8..2.3..9..5.1.3..'
+
+def grid_values_unsolved(grid):
+    """Convert grid string into {<box>: <value>} dict with '.' value for empties.
+
+    Arguments:
+        grid: Sudoku grid in string form, 81 characters long
+
+    Returns:
+        Sudoku grid in dictionary form:
+        - keys: Box labels, e.g. 'A1'
+        - values: Value in corresponding box, e.g. '8', or '.' if it is empty
+    """
+    assert(len(grid)==81)
+    return dict(zip(boxes, grid))
+
+def display(values):
+    """Display the values as a 2-D grid.
+    Input: The sudoku in dictionary form
+    Output: None
+    """
+    width = 1+max(len(values[s]) for s in boxes)
+    line = '+'.join(['-'*(width*3)]*3)
+    for r in rows:
+        print(''.join(values[r+c].center(width)+('|' if c in '36' else '')
+                      for c in cols))
+        if r in 'CF': print(line)
+    return
+
+def grid_values(grid):
+    """Convert grid string into {<box>: <value>} dict with '123456789' value for empties.
+
+    Args:
+        grid: Sudoku grid in string form, 81 characters long
+    Returns:
+        Sudoku grid in dictionary form:
+        - keys: Box labels, e.g. 'A1'
+        - values: Value in corresponding box, e.g. '8', or '123456789' if it is empty
+    """
+    values = []
+    all_digits = '123456789'
+    for n in grid:
+        if n == '.':
+            values.append(all_digits)
+        elif n in all_digits:
+            values.append(n)
+
+    assert len(values)==81
+    return dict(zip(boxes, values))
+
+def eliminate(values):
+    """Eliminate values from peers of each box with a single value.
+
+    Go through all the boxes, and whenever there is a box with a single value,
+    eliminate this value from the set of values of all its peers.
+
+    Args:
+        values: Sudoku in dictionary form
+    Returns:
+        Resulting Sudoku in dictionary form after eliminating values
+    """
+    #This will create a list of boxes that are solved
+    solved_values = [box for box in values.keys() if len(values[box]) == 1]
+
+    for box in solved_values:
+        digit = values[box]
+        for peer in peers[box]:
+            values[peer] = values[peer].replace(digit,'')
+    return values
+
+def only_choice(values):
+    """Finalize all values that are the only choice for a unit.
+
+    Go through all the units, and whenever there is a unit with a value
+    that only fits in one box, assign the value to this box.
+
+    Input: Sudoku in dictionary form.
+    Output: Resulting Sudoku in dictionary form after filling in only choices.
+    """
+    for unit in unitlist:
+        for digit in '123456789':
+            dplaces = [box for box in unit if digit in values[box]]
+            if len(dplaces) == 1:
+                values[dplaces[0]] = digit
+    return values
+
+def reduce_puzzle(values):
+    """Iterate eliminate() and only_choice().
+    If at some point, there is a box with no available values, return False.
+
+    If the sudoku is solved, return the sudoku.
+    If after an iteration of both functions, the sudoku remains the same, return the sudoku.
+
+    Input: A sudoku in dictionary form.
+    Output: The resulting sudoku in dictionary form.
+    """
+    stalled = False
+    while not stalled:
+        #Check how many boxes have a determined value
+        solved_values_before = len([box for box in values.keys() if len(values[box]) == 1])
+        #Use the Eliminate Strategy
+        values = eliminate(values)
+        #Use the Only Choice Strategy
+        values = only_choice(values)
+        #Check how many boxes have a determined value, to compare
+        solved_values_after = len([box for box in values.keys() if len(values[box]) == 1])
+        #If no new values were added, stop the loop.
+        stalled = solved_values_before == solved_values_after
+        #Sanity check, return False if there is a box with zero available values:
+        if len([box for box in values.keys() if len(values[box]) == 0]):
+            return False
+    return values
+
+def search_puzzle(values):
+    """Using depth-first search and propagation,
+    create a search tree and solve the sudoku."""
+
+    #Reduce the puzzle
+    values = reduce_puzzle(values)
+    if values is False:
+        return False ##Failed early
+    if all(len(values[s]) == 1 for s in boxes):
+        return values ##Solved!
+
+    #Choose one of the unfilled squares with the fewest possibilities
+    n, s = min((len(values[s]), s) for s in boxes if len(values[s]) > 1)
+    #print(n,s)
+
+    #Recursion to solve each one of the resulting sudokus,
+    #and if one returns a value (not False), return that answer!
+    for value in values[s]:
+        new_sudoku = values.copy()
+        new_sudoku[s] = value
+        attempt = search(new_sudoku)
+        if attempt:
+            return attempt
+
+if __name__ == '__main__':
+    #Testing Example
+    print("Enter the Sudoku values row-wise.\n\nAn example is given below:")
+    print('..3.2.6..9..3.5..1..18.64....81.29..7.......8..67.82....26.95..8..2.3..9..5.1.3..\n')
+    grid = input("Enter the Sudoku string:\n")
+    print('\n')
+    display(grid_values_unsolved(grid))
+    print("\n")
+    display(search_puzzle(reduce_puzzle(grid_values(grid))))

Term-I – AI Foundations/01 - Sudoku Solver/PySudoku.py

@@ -0,0 +1,66 @@
+import sys, os, random, pygame
+sys.path.append(os.path.join("objects"))
+import SudokuSquare
+from GameResources import *
+
+digits = '123456789'
+rows = 'ABCDEFGHI'
+
+
+def play(values_list):
+    pygame.init()
+
+
+    size = width, height = 700, 700
+    screen = pygame.display.set_mode(size)
+
+    background_image = pygame.image.load("./images/sudoku-board-bare.jpg").convert()
+
+    clock = pygame.time.Clock()
+
+    # The puzzleNumber sets a seed so either generate
+    # a random number to fill in here or accept user
+    # input for a duplicatable puzzle.
+
+    for values in values_list:
+        pygame.event.pump()
+        theSquares = []
+        initXLoc = 0
+        initYLoc = 0
+        startX, startY, editable, number = 0, 0, "N", 0
+        for y in range(9):
+            for x in range(9):
+                if x in (0, 1, 2):  startX = (x * 57) + 38
+                if x in (3, 4, 5):  startX = (x * 57) + 99
+                if x in (6, 7, 8):  startX = (x * 57) + 159
+
+                if y in (0, 1, 2):  startY = (y * 57) + 35
+                if y in (3, 4, 5):  startY = (y * 57) + 100
+                if y in (6, 7, 8):  startY = (y * 57) + 165
+                col = digits[x]
+                row = rows[y]
+                string_number = values[row + col]
+                if len(string_number) > 1 or string_number == '' or string_number == '.':
+                    number = None
+                else:
+                    number = int(string_number)
+                theSquares.append(SudokuSquare.SudokuSquare(number, startX, startY, editable, x, y))
+
+        screen.blit(background_image, (0, 0))
+        for num in theSquares:
+            num.draw()
+
+        pygame.display.flip()
+        pygame.display.update()
+        clock.tick(5)
+
+    # leave game showing until closed by user
+    while True:
+        for event in pygame.event.get():
+            if event.type == pygame.QUIT:
+                pygame.quit()
+                quit()
+
+if __name__ == "__main__":
+    main()
+    sys.exit()

Term-I – AI Foundations/01 - Sudoku Solver/README.md

@@ -0,0 +1,78 @@
+# Diagonal Sudoku Solver
+> In this project, an extension of a Sudoku solving agent is developed. The project is capable of solving any Classic or Diagonal Sudoku puzzle using three ideas: Constraint Propagation, Search (DFS) and Naked-Twins Strategy.
+
+## About
+[Sudoku](https://en.wikipedia.org/wiki/Sudoku) is one of the world's most popular puzzles. It consists of a 9x9 grid, and the objective is to fill the grid with digits in such a way that each row, each column, and each of the 9 principal 3x3 sub-squares contains all of the digits from 1 to 9. The detailed rules can be found [here](http://www.conceptispuzzles.com/?uri=puzzle/sudoku/rules).
+
+This project solves classic/diagonal Sudoku puzzles using **[Constraint Propagation](https://en.wikipedia.org/wiki/Constraint_satisfaction)** and **[Search (DFS)](https://en.wikipedia.org/wiki/Search_algorithm)**. In addition to the mentioned algorithmic techniques, Sudoku-specific strategy '**[Naked-Twins](http://www.sudokudragon.com/tutorialnakedtwins.htm)**' has also been used.
+
+#### Q: How do we use constraint propagation to solve the naked-twins problem?  
+- Constraint propagation works by reducing domains of variables, strengthening constraints, or creating new ones. This leads to a reduction of the search space, making it faster to use search algorithms to traverse for the solution.
+
+- The naked twins problem refers to the situation when two boxes within the same unit (row, column, square or diagonal) have the same two possible numbers that can be filled in them. When this happens, as no other number can go in those boxes, those two numbers can't go anywhere else either. This means they can be safely removed from the possibilities on any other box that belongs to the same unit.
+
+- In this process, an additional constraint can be added that allows further reduction of the possible digits that can fill the sudoku grid. This shortens the recursion towards the solution.
+
+#### Q: How do we use constraint propagation to solve the diagonal sudoku problem?  
+- To solve diagonal sudoku, an additional constraint added by the addition of the two diagonals to unit-list. By constituting the necessary constraint which combined with Depth First Search & other reductions,  feasible solution to the diagonal Sudoku is produced.
+
+## Requirements
+This project requires **Python 3**. It is recommended to use [Anaconda](https://www.continuum.io/downloads), a pre-packaged Python distribution that contains all of the necessary libraries and software for this project. Try using the environment provided in this folder.
+
+To see the visualization of Sudoku solving, installation of pygame is a necessity.The installation instructions are available [here](http://www.pygame.org/download.shtml).
+
+## Files
+* `solutions.py` – Driver program. Solves the Sudoku.
+
+* `sudoku.py` – Consists of a class 'Sudoku' which contains all method definitions used for solving.
+
+* `solution_test.py` – To test the solution.
+
+* `PySudoku.py` – Code for visualizing the solution.
+
+* `visualize.py` – Code for visualizing the solution.
+
+* `Classic_Sudoku.py` – Solver for classic sudoku.
+
+## Example:
+#### Classic_Sudoku Solver
+```
+python classic_sudoku.py
+
+Enter the Soduko values row-wise.
+
+An example is given below:
+..3.2.6..9..3.5..1..18.64....81.29..7.......8..67.82....26.95..8..2.3..9..5.1.3..
+
+Enter the Soduko string:
+..3.2.6..9..3.5..1..18.64....81.29..7.......8..67.82....26.95..8..2.3..9..5.1.3..
+
+. . 3 |. 2 . |6 . .                4 8 3 |9 2 1 |6 5 7
+9 . . |3 . 5 |. . 1                9 6 7 |3 4 5 |8 2 1
+. . 1 |8 . 6 |4 . .                2 5 1 |8 7 6 |4 9 3
+------+------+------              -------+------+------
+. . 8 |1 . 2 |9 . .                5 4 8 |1 3 2 |9 7 6
+7 . . |. . . |. . 8                7 2 9 |5 6 4 |1 3 8
+. . 6 |7 . 8 |2 . .                1 3 6 |7 9 8 |2 4 5
+------+------+------              -------+------+------
+. . 2 |6 . 9 |5 . .                3 7 2 |6 8 9 |5 1 4
+8 . . |2 . 3 |. . 9                8 1 4 |2 5 3 |7 6 9
+. . 5 |. 1 . |3 . .                6 9 5 |4 1 7 |3 8 2
+```
+
+#### Diagonal_Sudoku Solver
+To run the code, edit the Sudoku string in `solutions.py` and run the script.
+
+![Visualization of Diagonal Sudoku Solver](images/Diagonal_Visualization_Board.gif)
+
+## Future Improvements
+This was my favorite project of the nanodegree program. There is a lot that I would love to add to take this project and my learning experience further. Some of the ideas are:
+
+- Receiving input from a camera.
+- Extending the project to more Sudoku formats.
+- Incorporating more Sudoku solving techniques.
+
+If you want to contribute to the above ideas, feel free to reach out to me or use my code to build your own fork.
+
+## License
+[Modified MIT License © Pranav Suri](/License.txt)

Term-I – AI Foundations/01 - Sudoku Solver/aind-environment.yml

@@ -0,0 +1,21 @@
+name: aind
+channels: !!python/tuple
+- defaults
+dependencies:
+- mkl=2017.0.1=0
+- numpy=1.11.3=py36_0
+- openssl=1.0.2j=0
+- pip=9.0.1=py36_1
+- python=3.6.0=0
+- readline=6.2=2
+- scikit-learn=0.18.1=np111py36_1
+- scipy=0.18.1=np111py36_1
+- setuptools=27.2.0=py36_0
+- sqlite=3.13.0=0
+- tk=8.5.18=0
+- wheel=0.29.0=py36_0
+- xz=5.2.2=0
+- zlib=1.2.8=3
+- pip:
+  - hmmlearn==0.2.0
+

Term-I – AI Foundations/01 - Sudoku Solver/objects/GameResources.py

@@ -0,0 +1,14 @@
+import os, pygame
+
+def load_image(name):
+    """A better load of images."""
+    fullname = os.path.join("images", name)
+    try:
+        image = pygame.image.load(fullname)
+        if image.get_alpha() == None:
+            image = image.convert()
+        else:
+            image = image.convert_alpha()
+    except pygame.error:
+        print("Oops! Could not load image:", fullname)
+    return image, image.get_rect()

Term-I – AI Foundations/01 - Sudoku Solver/objects/SudokuSquare.py

@@ -0,0 +1,115 @@
+import pygame
+
+from pygame import *
+
+def AAfilledRoundedRect(surface,rect,color,radius=0.4):
+
+    """
+    AAfilledRoundedRect(surface,rect,color,radius=0.4)
+
+    surface : destination
+    rect    : rectangle
+    color   : rgb or rgba
+    radius  : 0 <= radius <= 1
+    """
+
+    rect         = Rect(rect)
+    color        = Color(*color)
+    alpha        = color.a
+    color.a      = 0
+    pos          = rect.topleft
+    rect.topleft = 0,0
+    rectangle    = Surface(rect.size,SRCALPHA)
+
+    circle       = Surface([min(rect.size)*3]*2,SRCALPHA)
+    draw.ellipse(circle,(0,0,0),circle.get_rect(),0)
+    circle       = transform.smoothscale(circle,[int(min(rect.size)*radius)]*2)
+
+    radius              = rectangle.blit(circle,(0,0))
+    radius.bottomright  = rect.bottomright
+    rectangle.blit(circle,radius)
+    radius.topright     = rect.topright
+    rectangle.blit(circle,radius)
+    radius.bottomleft   = rect.bottomleft
+    rectangle.blit(circle,radius)
+
+    rectangle.fill((0,0,0),rect.inflate(-radius.w,0))
+    rectangle.fill((0,0,0),rect.inflate(0,-radius.h))
+
+    rectangle.fill(color,special_flags=BLEND_RGBA_MAX)
+    rectangle.fill((255,255,255,alpha),special_flags=BLEND_RGBA_MIN)
+
+    return surface.blit(rectangle,pos)
+
+class SudokuSquare:
+    """A sudoku square class."""
+    def __init__(self, number=None, offsetX=0, offsetY=0, edit="Y", xLoc=0, yLoc=0):
+        if number != None:
+            number = str(number)
+            self.color = (2, 204, 186)
+        else:
+            number = ""
+            self.color = (255, 255, 255)
+        # print("FONTS", pygame.font.get_fonts())
+        self.font = pygame.font.SysFont('opensans', 21)
+        self.text = self.font.render(number, 1, (255, 255, 255))
+        self.textpos = self.text.get_rect()
+        self.textpos = self.textpos.move(offsetX + 17, offsetY + 4)
+
+        # self.collide = pygame.Surface((25, 22))
+        # self.collide = self.collide.convert()
+        # AAfilledRoundedRect(pygame.display.get_surface(), (xLoc, yLoc, 25, 22), (255, 255, 255))
+        # self.collide.fill((2, 204, 186))
+        # self.collideRect = self.collide.get_rect()
+        # self.collideRect = self.collideRect.move(offsetX + 1, offsetY + 1)
+        # The rect around the text is 11 x 28
+
+        self.edit = edit
+        self.xLoc = xLoc
+        self.yLoc = yLoc
+        self.offsetX = offsetX
+        self.offsetY = offsetY
+
+    def draw(self):
+        screen = pygame.display.get_surface()
+        AAfilledRoundedRect(screen, (self.offsetX, self.offsetY, 45, 40), self.color)
+
+        # screen.blit(self.collide, self.collideRect)
+        screen.blit(self.text, self.textpos)
+
+
+    def checkCollide(self, collision):
+        if len(collision) == 2:
+            return self.collideRect.collidepoint(collision)
+        elif len(collision) == 4:
+            return self.collideRect.colliderect(collision)
+        else:
+            return False
+
+
+    def highlight(self):
+        self.collide.fill((190, 190, 255))
+        self.draw()
+
+
+    def unhighlight(self):
+        self.collide.fill((255, 255, 255, 255))
+        self.draw()
+
+
+    def change(self, number):
+        if number != None:
+            number = str(number)
+        else:
+            number = ""
+        
+        if self.edit == "Y":
+            self.text = self.font.render(number, 1, (0, 0, 0))
+            self.draw()
+            return 0
+        else:
+            return 1
+
+
+    def currentLoc(self):
+        return self.xLoc, self.yLoc

Term-I – AI Foundations/01 - Sudoku Solver/solution.py

@@ -0,0 +1,43 @@
+import sudoku as sdk
+
+def naked_twins(values):
+    """
+    Eliminate values using the naked twins strategy.
+    Args:
+        values(dict): a dictionary of the form
+        {'box_name': '123456789', ...}
+    Returns:
+        the values dictionary with the naked twins eliminated from peers.
+    """
+    sudoku = sdk.Sudoku(values, partial=True)
+    sudoku.naked_twins()
+
+    return sudoku.values
+
+def solve(grid):
+    """
+    Find the solution to a Sudoku grid.
+    Args:
+        grid(string): a string representing a sudoku grid.
+            Example: '2.............62....1....7...6..8...3...9...7...6..4...4....8....52.............3'
+    Returns:
+        The dictionary representation of the final sudoku grid. False if no solution exists.
+    """
+    diagonal_sudoku = sdk.Sudoku(grid, diag=True)
+    diagonal_sudoku.search()
+    diagonal_sudoku.display()
+
+    return diagonal_sudoku.values
+
+if __name__ == '__main__':
+    diag_sudoku_grid = '9...6...7.6.971.4...........5.....3.41.....28.7.....6...........9.854.7.5...1...4'
+    solve(diag_sudoku_grid)
+
+    try:
+        from visualize import visualize_assignments
+        visualize_assignments(sdk.assignments)
+
+    except SystemExit:
+        pass
+    except:
+        print('Could not visualize your board due to a pygame issue. Not a problem! It is not a requirement.')

Term-I – AI Foundations/01 - Sudoku Solver/solution_test.py

@@ -0,0 +1,98 @@
+import solution
+import unittest
+
+
+class TestNakedTwins(unittest.TestCase):
+    before_naked_twins_1 = {'I6': '4', 'H9': '3', 'I2': '6', 'E8': '1', 'H3': '5', 'H7': '8', 'I7': '1', 'I4': '8',
+                            'H5': '6', 'F9': '7', 'G7': '6', 'G6': '3', 'G5': '2', 'E1': '8', 'G3': '1', 'G2': '8',
+                            'G1': '7', 'I1': '23', 'C8': '5', 'I3': '23', 'E5': '347', 'I5': '5', 'C9': '1', 'G9': '5',
+                            'G8': '4', 'A1': '1', 'A3': '4', 'A2': '237', 'A5': '9', 'A4': '2357', 'A7': '27',
+                            'A6': '257', 'C3': '8', 'C2': '237', 'C1': '23', 'E6': '579', 'C7': '9', 'C6': '6',
+                            'C5': '37', 'C4': '4', 'I9': '9', 'D8': '8', 'I8': '7', 'E4': '6', 'D9': '6', 'H8': '2',
+                            'F6': '125', 'A9': '8', 'G4': '9', 'A8': '6', 'E7': '345', 'E3': '379', 'F1': '6',
+                            'F2': '4', 'F3': '23', 'F4': '1235', 'F5': '8', 'E2': '37', 'F7': '35', 'F8': '9',
+                            'D2': '1', 'H1': '4', 'H6': '17', 'H2': '9', 'H4': '17', 'D3': '2379', 'B4': '27',
+                            'B5': '1', 'B6': '8', 'B7': '27', 'E9': '2', 'B1': '9', 'B2': '5', 'B3': '6', 'D6': '279',
+                            'D7': '34', 'D4': '237', 'D5': '347', 'B8': '3', 'B9': '4', 'D1': '5'}
+    possible_solutions_1 = [
+        {'G7': '6', 'G6': '3', 'G5': '2', 'G4': '9', 'G3': '1', 'G2': '8', 'G1': '7', 'G9': '5', 'G8': '4', 'C9': '1',
+         'C8': '5', 'C3': '8', 'C2': '237', 'C1': '23', 'C7': '9', 'C6': '6', 'C5': '37', 'A4': '2357', 'A9': '8',
+         'A8': '6', 'F1': '6', 'F2': '4', 'F3': '23', 'F4': '1235', 'F5': '8', 'F6': '125', 'F7': '35', 'F8': '9',
+         'F9': '7', 'B4': '27', 'B5': '1', 'B6': '8', 'B7': '27', 'E9': '2', 'B1': '9', 'B2': '5', 'B3': '6', 'C4': '4',
+         'B8': '3', 'B9': '4', 'I9': '9', 'I8': '7', 'I1': '23', 'I3': '23', 'I2': '6', 'I5': '5', 'I4': '8', 'I7': '1',
+         'I6': '4', 'A1': '1', 'A3': '4', 'A2': '237', 'A5': '9', 'E8': '1', 'A7': '27', 'A6': '257', 'E5': '347',
+         'E4': '6', 'E7': '345', 'E6': '579', 'E1': '8', 'E3': '79', 'E2': '37', 'H8': '2', 'H9': '3', 'H2': '9',
+         'H3': '5', 'H1': '4', 'H6': '17', 'H7': '8', 'H4': '17', 'H5': '6', 'D8': '8', 'D9': '6', 'D6': '279',
+         'D7': '34', 'D4': '237', 'D5': '347', 'D2': '1', 'D3': '79', 'D1': '5'},
+        {'I6': '4', 'H9': '3', 'I2': '6', 'E8': '1', 'H3': '5', 'H7': '8', 'I7': '1', 'I4': '8', 'H5': '6', 'F9': '7',
+         'G7': '6', 'G6': '3', 'G5': '2', 'E1': '8', 'G3': '1', 'G2': '8', 'G1': '7', 'I1': '23', 'C8': '5', 'I3': '23',
+         'E5': '347', 'I5': '5', 'C9': '1', 'G9': '5', 'G8': '4', 'A1': '1', 'A3': '4', 'A2': '237', 'A5': '9',
+         'A4': '2357', 'A7': '27', 'A6': '257', 'C3': '8', 'C2': '237', 'C1': '23', 'E6': '579', 'C7': '9', 'C6': '6',
+         'C5': '37', 'C4': '4', 'I9': '9', 'D8': '8', 'I8': '7', 'E4': '6', 'D9': '6', 'H8': '2', 'F6': '125',
+         'A9': '8', 'G4': '9', 'A8': '6', 'E7': '345', 'E3': '79', 'F1': '6', 'F2': '4', 'F3': '23', 'F4': '1235',
+         'F5': '8', 'E2': '3', 'F7': '35', 'F8': '9', 'D2': '1', 'H1': '4', 'H6': '17', 'H2': '9', 'H4': '17',
+         'D3': '79', 'B4': '27', 'B5': '1', 'B6': '8', 'B7': '27', 'E9': '2', 'B1': '9', 'B2': '5', 'B3': '6',
+         'D6': '279', 'D7': '34', 'D4': '237', 'D5': '347', 'B8': '3', 'B9': '4', 'D1': '5'}
+        ]
+
+    before_naked_twins_2 = {'A1': '23', 'A2': '4', 'A3': '7', 'A4': '6', 'A5': '8', 'A6': '5', 'A7': '23', 'A8': '9',
+                            'A9': '1', 'B1': '6', 'B2': '9', 'B3': '8', 'B4': '4', 'B5': '37', 'B6': '1', 'B7': '237',
+                            'B8': '5', 'B9': '237', 'C1': '23', 'C2': '5', 'C3': '1', 'C4': '23', 'C5': '379',
+                            'C6': '2379', 'C7': '8', 'C8': '6', 'C9': '4', 'D1': '8', 'D2': '17', 'D3': '9',
+                            'D4': '1235', 'D5': '6', 'D6': '237', 'D7': '4', 'D8': '27', 'D9': '2357', 'E1': '5',
+                            'E2': '6', 'E3': '2', 'E4': '8', 'E5': '347', 'E6': '347', 'E7': '37', 'E8': '1', 'E9': '9',
+                            'F1': '4', 'F2': '17', 'F3': '3', 'F4': '125', 'F5': '579', 'F6': '279', 'F7': '6',
+                            'F8': '8', 'F9': '257', 'G1': '1', 'G2': '8', 'G3': '6', 'G4': '35', 'G5': '345',
+                            'G6': '34', 'G7': '9', 'G8': '27', 'G9': '27', 'H1': '7', 'H2': '2', 'H3': '4', 'H4': '9',
+                            'H5': '1', 'H6': '8', 'H7': '5', 'H8': '3', 'H9': '6', 'I1': '9', 'I2': '3', 'I3': '5',
+                            'I4': '7', 'I5': '2', 'I6': '6', 'I7': '1', 'I8': '4', 'I9': '8'}
+    possible_solutions_2 = [
+        {'A1': '23', 'A2': '4', 'A3': '7', 'A4': '6', 'A5': '8', 'A6': '5', 'A7': '23', 'A8': '9', 'A9': '1', 'B1': '6',
+         'B2': '9', 'B3': '8', 'B4': '4', 'B5': '37', 'B6': '1', 'B7': '237', 'B8': '5', 'B9': '237', 'C1': '23',
+         'C2': '5', 'C3': '1', 'C4': '23', 'C5': '79', 'C6': '79', 'C7': '8', 'C8': '6', 'C9': '4', 'D1': '8',
+         'D2': '17', 'D3': '9', 'D4': '1235', 'D5': '6', 'D6': '237', 'D7': '4', 'D8': '27', 'D9': '2357', 'E1': '5',
+         'E2': '6', 'E3': '2', 'E4': '8', 'E5': '347', 'E6': '347', 'E7': '37', 'E8': '1', 'E9': '9', 'F1': '4',
+         'F2': '17', 'F3': '3', 'F4': '125', 'F5': '579', 'F6': '279', 'F7': '6', 'F8': '8', 'F9': '257', 'G1': '1',
+         'G2': '8', 'G3': '6', 'G4': '35', 'G5': '345', 'G6': '34', 'G7': '9', 'G8': '27', 'G9': '27', 'H1': '7',
+         'H2': '2', 'H3': '4', 'H4': '9', 'H5': '1', 'H6': '8', 'H7': '5', 'H8': '3', 'H9': '6', 'I1': '9', 'I2': '3',
+         'I3': '5', 'I4': '7', 'I5': '2', 'I6': '6', 'I7': '1', 'I8': '4', 'I9': '8'},
+        {'A1': '23', 'A2': '4', 'A3': '7', 'A4': '6', 'A5': '8', 'A6': '5', 'A7': '23', 'A8': '9', 'A9': '1', 'B1': '6',
+         'B2': '9', 'B3': '8', 'B4': '4', 'B5': '3', 'B6': '1', 'B7': '237', 'B8': '5', 'B9': '237', 'C1': '23',
+         'C2': '5', 'C3': '1', 'C4': '23', 'C5': '79', 'C6': '79', 'C7': '8', 'C8': '6', 'C9': '4', 'D1': '8',
+         'D2': '17', 'D3': '9', 'D4': '1235', 'D5': '6', 'D6': '237', 'D7': '4', 'D8': '27', 'D9': '2357', 'E1': '5',
+         'E2': '6', 'E3': '2', 'E4': '8', 'E5': '347', 'E6': '347', 'E7': '37', 'E8': '1', 'E9': '9', 'F1': '4',
+         'F2': '17', 'F3': '3', 'F4': '125', 'F5': '579', 'F6': '279', 'F7': '6', 'F8': '8', 'F9': '257', 'G1': '1',
+         'G2': '8', 'G3': '6', 'G4': '35', 'G5': '345', 'G6': '34', 'G7': '9', 'G8': '27', 'G9': '27', 'H1': '7',
+         'H2': '2', 'H3': '4', 'H4': '9', 'H5': '1', 'H6': '8', 'H7': '5', 'H8': '3', 'H9': '6', 'I1': '9', 'I2': '3',
+         'I3': '5', 'I4': '7', 'I5': '2', 'I6': '6', 'I7': '1', 'I8': '4', 'I9': '8'}
+    ]
+
+    def test_naked_twins(self):
+        self.assertTrue(solution.naked_twins(self.before_naked_twins_1) in self.possible_solutions_1,
+                        "Your naked_twins function produced an unexpected board.")
+
+    def test_naked_twins2(self):
+        self.assertTrue(solution.naked_twins(self.before_naked_twins_2) in self.possible_solutions_2,
+                        "Your naked_twins function produced an unexpected board.")
+
+
+
+class TestDiagonalSudoku(unittest.TestCase):
+    diagonal_grid = '2.............62....1....7...6..8...3...9...7...6..4...4....8....52.............3'
+    solved_diag_sudoku = {'G7': '8', 'G6': '9', 'G5': '7', 'G4': '3', 'G3': '2', 'G2': '4', 'G1': '6', 'G9': '5',
+                          'G8': '1', 'C9': '6', 'C8': '7', 'C3': '1', 'C2': '9', 'C1': '4', 'C7': '5', 'C6': '3',
+                          'C5': '2', 'C4': '8', 'E5': '9', 'E4': '1', 'F1': '1', 'F2': '2', 'F3': '9', 'F4': '6',
+                          'F5': '5', 'F6': '7', 'F7': '4', 'F8': '3', 'F9': '8', 'B4': '7', 'B5': '1', 'B6': '6',
+                          'B7': '2', 'B1': '8', 'B2': '5', 'B3': '3', 'B8': '4', 'B9': '9', 'I9': '3', 'I8': '2',
+                          'I1': '7', 'I3': '8', 'I2': '1', 'I5': '6', 'I4': '5', 'I7': '9', 'I6': '4', 'A1': '2',
+                          'A3': '7', 'A2': '6', 'E9': '7', 'A4': '9', 'A7': '3', 'A6': '5', 'A9': '1', 'A8': '8',
+                          'E7': '6', 'E6': '2', 'E1': '3', 'E3': '4', 'E2': '8', 'E8': '5', 'A5': '4', 'H8': '6',
+                          'H9': '4', 'H2': '3', 'H3': '5', 'H1': '9', 'H6': '1', 'H7': '7', 'H4': '2', 'H5': '8',
+                          'D8': '9', 'D9': '2', 'D6': '8', 'D7': '1', 'D4': '4', 'D5': '3', 'D2': '7', 'D3': '6',
+                          'D1': '5'}
+
+    def test_solve(self):
+        self.assertEqual(solution.solve(self.diagonal_grid), self.solved_diag_sudoku)
+
+if __name__ == '__main__':
+    unittest.main()

Term-I – AI Foundations/01 - Sudoku Solver/sudoku.py

@@ -0,0 +1,209 @@
+assignments = []
+
+class Sudoku():
+    """Initializing and solving a classic/diagnol Sudoku.
+
+    ATTRIBUTES:
+        grid        : row-wise string representation of Sudoku
+        row         : string 'ABCDEFGHI'
+        cols        : string '123456789'
+        boxes       : list of all squares
+        unit_list   : list of all units
+        peers       : dictionary of peers for each box
+        values      : dictionary of box:digits
+    """
+
+    def __init__(self, grid, partial=False, diag=False,rows='ABCDEFGHI', cols='123456789'):
+        """
+            Args:
+            - grid      : 81-char string to be solved if 'partial = False'
+                          else intermediate solution as a dict of box:values
+            - partial   : init with string or partial solution
+            - diag      : boolean - diagonal Sudoku = True
+
+            Returns:
+                All class attributes initialized
+        """
+        self.grid = grid
+        self.rows = rows
+        self.cols = cols
+        self.grid_init(partial, diag)
+
+    def cross(self, A, B):
+        """
+        Cross product of elements in a and b
+        """
+        return [a+b for a in A for b in B]
+
+    def assign_value(self, box, value):
+        """Update the values dictionary.
+        Assigns a value to a given box. If it updates the board, records it.
+        """
+        self.values[box] = value
+        if len(value) == 1:
+            assignments.append(self.values.copy())
+
+    def grid_init(self, partial, diag):
+        """Setup the grid and initialize difference parameters.
+        i.e. Boxes, Values, Peers.
+        """
+        self.boxes = self.cross(self.rows,self.cols)
+
+        if partial:
+            self.values = self.grid.copy()
+        else:
+            self.values = {}
+            for box,char in zip(self.boxes,self.grid):
+                if char == '.':
+                    self.values[box] = self.cols
+                else:
+                    self.values[box] = char
+
+        rowunits = [self.cross(r,self.cols) for r in self.rows]
+        colunits = [self.cross(self.rows,c) for c in self.cols]
+        squareunits = [self.cross(rs, cs) for rs in ['ABC','DEF','GHI'] for cs in ['123','456','789']]
+
+        self.unitlist = rowunits + colunits + squareunits
+        if diag:
+            self.unitlist += [[r+c for r,c in zip(self.rows,self.cols)],
+                              [r+c for r,c in zip(self.rows,self.cols[::-1])]]
+
+        units = {box:[u for u in self.unitlist if box in u] for box in self.boxes}
+        self.peers = {box:set(sum(units[box],[]))-set([box]) for box in self.boxes}
+
+    def display(self):
+        """Display the values as a 2-D grid.
+        Args:
+            values(dict): The sudoku in dictionary form
+        """
+        width = 1+max(len(self.values[s]) for s in self.boxes)
+        line = '+'.join(['-'*(width*3)]*3)
+        for r in self.rows:
+            print(''.join(self.values[r+c].center(width)+('|' if c in '36' else '')
+                      for c in self.cols))
+            if r in 'CF': print(line)
+        print('\n')
+
+    def solved_values(self):
+        return [box for box in self.boxes if len(self.values[box]) == 1]
+
+    def eliminate(self):
+        """Eliminate values from peers of each box with a single value.
+
+        Go through all the boxes, and whenever there is a box with a single
+        value, eliminate this value from the set of values of all its peers.
+
+        Args:
+            values: Sudoku in dictionary form
+        Returns:
+            Resulting Sudoku in dictionary form after eliminating values
+        """
+        solved_values_boxes = self.solved_values()
+        for box in solved_values_boxes:
+            digit = self.values[box]
+            for peer in self.peers[box]:
+                value = self.values[peer]
+                value = value.replace(digit,'')
+                self.assign_value(peer, value)
+
+    def only_choice(self):
+        """Finalize all values that are the only choice for a unit.
+
+        Go through all the units, and whenever there is a unit with a value
+        that only fits in one box, assign the value to this box.
+
+        Input: Sudoku in dictionary form.
+        Output: Resulting Sudoku in dictionary form after filling in only
+        choices.
+        """
+        for unit in self.unitlist:
+            for digit in self.cols:
+                boxes_containing_digit =[box for box in unit if digit in self.values[box]]
+                if len(boxes_containing_digit) == 1:
+                    self.assign_value(boxes_containing_digit[0], digit)
+
+    #Naked-Twins Stragtegy
+    def find_naked_twins(self):
+        twins = [[a for a in u
+                        for b in u
+                            if (a != b) and (len(self.values[a]) == 2)
+                                        and (self.values[a] == self.values[b])
+                 ]
+                 for i,u in enumerate(self.unitlist)
+                ]
+        return twins
+
+    def eliminate_naked_twins(self, twins):
+        for i, twin in enumerate(twins):
+            if twin:
+                for box in self.unitlist[i]:
+                    if box not in twin:
+                        for digit in self.values[twin[0]]:
+                            value = self.values[box]
+                            if len(value) > 1 :
+                                value = value.replace(digit,'')
+                                self.assign_value(box, value)
+
+    def naked_twins(self):
+        """Eliminate values using the naked twins strategy.
+        Args:
+            values(dict): a dictionary of the form {'box_name': '123456789', ...}
+
+        Returns:
+            the values dictionary with the naked twins eliminated from peers.
+        """
+        #Find all instances of naked twins
+        twins = self.find_naked_twins()
+        #Eliminate the naked twins as possibilities for their units
+        self.eliminate_naked_twins(twins)
+
+    def reduce_puzzle(self):
+        """Iterate eliminate() and only_choice().
+        If at some point, there is a box with no available values, return False.
+
+        If the sudoku is solved, return the sudoku.
+        If after an iteration of both functions, the sudoku remains the same,
+        return the sudoku.
+
+        Input: A sudoku in dictionary form.
+        Output: The resulting sudoku in dictionary form.
+        """
+        stalled = False
+        while not stalled:
+            solved_values_before = self.solved_values()
+            self.eliminate()
+            self.only_choice()
+            solved_values_after = self.solved_values()
+
+            stalled = solved_values_before == solved_values_after
+
+            if len([box for box in self.boxes if len( self.values[box]) == 0]):
+                # if it is still solvable run naked_twins or not
+                self.naked_twins()
+                return True
+        return False
+
+    def search(self):
+        """Using depth-first search and propagation,
+        create a search tree and solve the sudoku.
+        """
+        # Reduce the puzzle
+        failed = self.reduce_puzzle()
+        if failed :
+            return False #Tree-leaf: not a solution
+        if all(len(self.values[box]) == 1 for box in self.boxes):
+            return True #Solved
+
+        # Choose box with min possibilities
+        n, box = min((len(self.values[box]),box) for box in self.boxes if len(self.values[box]) > 1)
+
+        # Recursion to solve each one of the resulting sudokus,
+        # and if one returns a value (not False), return that answer!
+        for digit in self.values[box]:
+            tmp = self.values.copy() # copy of the values in case of failure = 'attempt ===False'
+            self.values[box] = digit
+            attempt = self.search()
+            if attempt:
+                return attempt
+            else:
+                self.values = tmp

Term-I – AI Foundations/01 - Sudoku Solver/visualize.py

@@ -0,0 +1,17 @@
+from PySudoku import play
+
+def visualize_assignments(assignments):
+    """ Visualizes the set of assignments created by the Sudoku AI"""
+    last_assignment = None
+    filtered_assignments = []
+
+    for i in range(len(assignments)):
+        if last_assignment:
+            last_assignment_items = [item for item in last_assignment.items() if len(item[1]) == 1]
+            current_assignment_items = [item for item in assignments[i].items() if len(item[1]) == 1]
+            shared_items = set(last_assignment_items) & set(current_assignment_items)
+            if len(shared_items) < len(current_assignment_items):
+                filtered_assignments.append(assignments[i])
+        last_assignment = assignments[i]
+
+    play(filtered_assignments)

Term-I – AI Foundations/02 - Game Playing Agent/README.md

@@ -0,0 +1,70 @@
+# Build a Game-Playing Agent
+> This game-playing agent uses techniques such as Iterative Deepening, Minimax, and Alpha-Beta Pruning to compete in the game of Isolation (a two-player discrete competitive game with perfect information). The different heuristics used are then compared to find the best heuristic.
+
+![Example game of isolation](images/viz.gif)
+
+## About
+This project is an adversarial search agent to play the game 'Isolation.' Isolation is a deterministic, two-player board game of perfect information in which the players alternate turns moving a single piece from one cell to another.  Whenever either player occupies a cell, that cell becomes blocked for the remainder of the game.  The first player with no remaining legal moves loses, and the opponent is declared the winner.
+
+This project uses a version of Isolation where each agent is restricted to L-shaped movements (like a knight in chess) on a rectangular grid (like a chessboard). The agents can move to an open cell on the board that is 2-rows and 1-column or 2-columns and 1-row away from their current position on the board. Movements are blocked at the edges of the board (the board does not wrap around). However, the player can "jump" blocked or occupied spaces (just like a knight in chess).
+
+Additionally, agents will have a fixed time limit each turn to search for the best move and respond.  If the time limit expires during a player's turn, that player forfeits the match, and the opponent wins.
+
+## Requirements
+This project requires **Python 3**. It is recommended to use [Anaconda](https://www.continuum.io/downloads), a pre-packaged Python distribution that contains all of the necessary libraries and software for this project. Try using the environment provided in this folder.
+
+### Using the Board Visualization
+The `isoviz` folder contains a modified version of `chessboard.js` that can animate games played on a 7x7 board.  In order to use the board, you must run a local web server by running `python -m SimpleHTTPServer 8000` from your project directory (you can replace 8000 with another port number if that one is unavailable), then open your browser to `http://localhost:8000` and navigate to the `/isoviz/display.html` page.  Enter the move history of an isolation match (i.e., the array returned by the `Board.play()` method) into the text area and run the match. Refresh the page to run a different game.
+
+## Files
+- `game_agent.py` – Contains the code for the game-playing agent (see CustomPlayer class).
+
+- `agent_test.py` - Provided by @udacity to unit test `game_agent.py` implementation.
+
+- `tournament.py` - Provided by the @udacity staff to evaluate the performance of the game-playing agent.
+
+- `heuristic_analysis.pdf` – Contains the analysis of the various heuristics implemented in the game_agent. The metrics are obtained from `tournament.py`.
+
+- `research_review.pdf` – Reviews IBM's Deep Blue's [seminal paper](https://pdfs.semanticscholar.org/ad2c/1efffcd7c3b7106e507396bdaa5fe00fa597.pdf). A detailed blog post can be read on my blog on [this link]()
+
+## Output (`tournament.py`)
+```
+*************************
+ Evaluating: ID_Improved
+*************************
+
+Playing Matches:
+----------
+  Match 1: ID_Improved vs   Random    	Result: 15 to 5
+  Match 2: ID_Improved vs   MM_Null   	Result: 14 to 6
+  Match 3: ID_Improved vs   MM_Open   	Result: 16 to 4
+  Match 4: ID_Improved vs MM_Improved 	Result: 12 to 8
+  Match 5: ID_Improved vs   AB_Null   	Result: 14 to 6
+  Match 6: ID_Improved vs   AB_Open   	Result: 12 to 8
+  Match 7: ID_Improved vs AB_Improved 	Result: 11 to 9
+
+Results:
+----------
+ID_Improved         67.14%
+
+*************************
+   Evaluating: Student   
+*************************
+
+Playing Matches:
+----------
+  Match 1:   Student   vs   Random    	Result: 17 to 3
+  Match 2:   Student   vs   MM_Null   	Result: 14 to 6
+  Match 3:   Student   vs   MM_Open   	Result: 14 to 6
+  Match 4:   Student   vs MM_Improved 	Result: 13 to 7
+  Match 5:   Student   vs   AB_Null   	Result: 18 to 2
+  Match 6:   Student   vs   AB_Open   	Result: 13 to 7
+  Match 7:   Student   vs AB_Improved 	Result: 16 to 4
+
+Results:
+----------
+Student             75.00%
+```
+
+## License
+[Modified MIT License © Pranav Suri](/License.txt)

Term-I – AI Foundations/02 - Game Playing Agent/agent_test.py

@@ -0,0 +1,549 @@
+"""
+This file contains test cases to verify the correct implementation of the
+functions required for this project including minimax, alphabeta, and iterative
+deepening.  The heuristic function is tested for conformance to the expected
+interface, but cannot be automatically assessed for correctness.
+
+STUDENTS SHOULD NOT NEED TO MODIFY THIS CODE.  IT WOULD BE BEST TO TREAT THIS
+FILE AS A BLACK BOX FOR TESTING.
+"""
+import unittest
+import timeit
+import sys
+
+import isolation
+import game_agent
+
+from collections import Counter
+from copy import copy
+from functools import wraps
+from queue import Queue
+from threading import Thread
+from multiprocessing import TimeoutError
+from queue import Empty as QueueEmptyError
+from importlib import reload
+
+TIMEOUT = 60
+
+WRONG_MOVE = """
+The {} function failed because it returned a non-optimal move at search
+depth {}.
+Valid choices: {}
+Your selection: {}
+"""
+
+WRONG_NUM_EXPLORED = """
+Your {} search visited the wrong nodes at search depth {}.  If the number
+of visits is too large, make sure that iterative deepening is only
+running when the `iterative` flag is set in the agent constructor.
+Max explored size: {}
+Number you explored: {}
+"""
+
+UNEXPECTED_VISIT = """
+Your {} search did not visit the number of expected unique nodes at search
+depth {}.
+Max explored size: {}
+Number you explored: {}
+"""
+
+ID_FAIL = """
+Your agent explored the wrong number of nodes using Iterative Deepening and
+minimax. Remember that ID + MM should check every node in each layer of the
+game tree before moving on to the next layer.
+"""
+
+INVALID_MOVE = """
+Your agent returned an invalid move. Make sure that your function returns
+a selection when the search times out during iterative deepening.
+Valid choices: {!s}
+Your choice: {}
+"""
+
+WRONG_HEURISTIC = """
+Your agent did not use the heuristic provided to the agent constructor
+(accessed with self.score()).  Make sure that you're not directly calling
+your custom_score() heuristic function in alpha-beta or minimax search.
+"""
+
+TIMER_MARGIN = 15  # time (in ms) to leave on the timer to avoid timeout
+
+
+def curr_time_millis():
+    """Simple timer to return the current clock time in milliseconds."""
+    return 1000 * timeit.default_timer()
+
+
+def handler(obj, testcase, queue):
+    """Handler to pass information between threads; used in the timeout
+    function to abort long-running (i.e., probably hung) test cases.
+    """
+    try:
+        queue.put((None, testcase(obj)))
+    except:
+        queue.put((sys.exc_info(), None))
+
+
+def timeout(time_limit):
+    """Function decorator for unittest test cases to specify test case timeout.
+
+    The timer mechanism works by spawning a new thread for the test to run in
+    and using the timeout handler for the thread-safe queue class to abort and
+    kill the child thread if it doesn't return within the timeout.
+
+    It is not safe to access system resources (e.g., files) within test cases
+    wrapped by this timer.
+    """
+
+    def wrapUnitTest(testcase):
+
+        @wraps(testcase)
+        def testWrapper(self):
+
+            queue = Queue()
+
+            try:
+                p = Thread(target=handler, args=(self, testcase, queue))
+                p.daemon = True
+                p.start()
+                err, res = queue.get(timeout=time_limit)
+                p.join()
+                if err:
+                    raise err[0](err[1]).with_traceback(err[2])
+                return res
+            except QueueEmptyError:
+                raise TimeoutError(
+                    ("Test aborted due to timeout. Test was " +
+                     "expected to finish in less than {} second(s).").format(
+                        time_limit))
+
+        return testWrapper
+
+    return wrapUnitTest
+
+
+def makeEvalTable(table):
+    """Use a closure to create a heuristic function that returns values from
+    a table that maps board locations to constant values. This supports testing
+    the minimax and alphabeta search functions.
+
+    THIS HEURISTIC IS ONLY USEFUL FOR TESTING THE SEARCH FUNCTIONALITY -
+    IT IS NOT MEANT AS AN EXAMPLE OF A USEFUL HEURISTIC FOR GAME PLAYING.
+    """
+
+    def score(game, player):
+        row, col = game.get_player_location(player)
+        return table[row][col]
+
+    return score
+
+
+def makeEvalStop(limit, timer):
+    """Use a closure to create a heuristic function that forces the search
+    timer to expire when a fixed number of node expansions have been perfomred
+    during the search. This ensures that the search algorithm should always be
+    in a predictable state regardless of node expansion order.
+
+    THIS HEURISTIC IS ONLY USEFUL FOR TESTING THE SEARCH FUNCTIONALITY -
+    IT IS NOT MEANT AS AN EXAMPLE OF A USEFUL HEURISTIC FOR GAME PLAYING.
+    """
+
+    def score(game, player):
+        timer.invoked = True
+        if timer.time_left() < 0:
+            raise TimeoutError("Timer expired during search. You must " +
+                               "return an answer before the timer reaches 0.")
+        if limit == game.counts[0]:
+            timer.time_limit = 0
+        return 0
+
+    return score
+
+
+def makeBranchEval(first_branch):
+    """Use a closure to create a heuristic function that evaluates to a nonzero
+    score when the root of the search is the first branch explored, and
+    otherwise returns 0.  This heuristic is used to force alpha-beta to prune
+    some parts of a game tree for testing.
+
+    THIS HEURISTIC IS ONLY USEFUL FOR TESTING THE SEARCH FUNCTIONALITY -
+    IT IS NOT MEANT AS AN EXAMPLE OF A USEFUL HEURISTIC FOR GAME PLAYING.
+    """
+
+    def score(game, player):
+        if not first_branch:
+            first_branch.append(game.root)
+        if game.root in first_branch:
+            return 1.
+        return 0.
+
+    return score
+
+
+class CounterBoard(isolation.Board):
+    """Subclass of the isolation board that maintains counters for the number
+    of unique nodes and total nodes visited during depth first search.
+
+    Some functions from the base class must be overridden to maintain the
+    counters during search.
+    """
+
+    def __init__(self, *args, **kwargs):
+        super(CounterBoard, self).__init__(*args, **kwargs)
+        self.counter = Counter()
+        self.visited = set()
+        self.root = None
+
+    def copy(self):
+        new_board = CounterBoard(self._player_1, self._player_2,
+                                 width=self.width, height=self.height)
+        new_board.move_count = self.move_count
+        new_board._active_player = self._active_player
+        new_board._inactive_player = self._inactive_player
+        new_board._board_state = copy(self._board_state)
+        new_board.counter = self.counter
+        new_board.visited = self.visited
+        new_board.root = self.root
+        return new_board
+
+    def forecast_move(self, move):
+        self.counter[move] += 1
+        self.visited.add(move)
+        new_board = self.copy()
+        new_board.apply_move(move)
+        if new_board.root is None:
+            new_board.root = move
+        return new_board
+
+    @property
+    def counts(self):
+        """ Return counts of (total, unique) nodes visited """
+        return sum(self.counter.values()), len(self.visited)
+
+
+class Project1Test(unittest.TestCase):
+
+    def initAUT(self, depth, eval_fn, iterative=False,
+                method="minimax", loc1=(3, 3), loc2=(0, 0), w=7, h=7):
+        """Generate and initialize player and board objects to be used for
+        testing.
+        """
+        reload(game_agent)
+        agentUT = game_agent.CustomPlayer(depth, eval_fn, iterative, method)
+        board = CounterBoard(agentUT, 'null_agent', w, h)
+        board.apply_move(loc1)
+        board.apply_move(loc2)
+        return agentUT, board
+
+    @timeout(TIMEOUT)
+    # @unittest.skip("Skip eval function test.")  # Uncomment this line to skip test
+    def test_heuristic(self):
+        """Test output interface of heuristic score function interface."""
+        player1 = "Player1"
+        player2 = "Player2"
+        p1_location = (0, 0)
+        p2_location = (1, 1)  # top left corner
+        game = isolation.Board(player1, player2)
+        game.apply_move(p1_location)
+        game.apply_move(p2_location)
+
+        self.assertIsInstance(game_agent.custom_score(game, player1), float,
+            "The heuristic function should return a floating point")
+
+    @timeout(TIMEOUT)
+    # @unittest.skip("Skip simple minimax test.")  # Uncomment this line to skip test
+    def test_minimax_interface(self):
+        """Test CustomPlayer.minimax interface with simple input """
+        h, w = 7, 7  # board size
+        test_depth = 1
+        starting_location = (5, 3)
+        adversary_location = (0, 0)  # top left corner
+        iterative_search = False
+        search_method = "minimax"
+        heuristic = lambda g, p: 0.  # return 0 everywhere
+
+        # create a player agent & a game board
+        agentUT = game_agent.CustomPlayer(
+            test_depth, heuristic, iterative_search, search_method)
+        agentUT.time_left = lambda: 99  # ignore timeout for fixed-depth search
+        board = isolation.Board(agentUT, 'null_agent', w, h)
+
+        # place two "players" on the board at arbitrary (but fixed) locations
+        board.apply_move(starting_location)
+        board.apply_move(adversary_location)
+
+        for move in board.get_legal_moves():
+            next_state = board.forecast_move(move)
+            v, _ = agentUT.minimax(next_state, test_depth)
+
+            self.assertTrue(type(v) == float,
+                            ("Minimax function should return a floating " +
+                             "point value approximating the score for the " +
+                             "branch being searched."))
+
+    @timeout(TIMEOUT)
+    # @unittest.skip("Skip alphabeta test.")  # Uncomment this line to skip test
+    def test_alphabeta_interface(self):
+        """Test CustomPlayer.alphabeta interface with simple input """
+        h, w = 9, 9  # board size
+        test_depth = 1
+        starting_location = (2, 7)
+        adversary_location = (0, 0)  # top left corner
+        iterative_search = False
+        search_method = "alphabeta"
+        heuristic = lambda g, p: 0.  # return 0 everywhere
+
+        # create a player agent & a game board
+        agentUT = game_agent.CustomPlayer(
+            test_depth, heuristic, iterative_search, search_method)
+        agentUT.time_left = lambda: 99  # ignore timeout for fixed-depth search
+        board = isolation.Board(agentUT, 'null_agent', w, h)
+
+        # place two "players" on the board at arbitrary (but fixed) locations
+        board.apply_move(starting_location)
+        board.apply_move(adversary_location)
+
+        for move in board.get_legal_moves():
+            next_state = board.forecast_move(move)
+            v, _ = agentUT.alphabeta(next_state, test_depth)
+
+            self.assertTrue(type(v) == float,
+                            ("Alpha Beta function should return a floating " +
+                             "point value approximating the score for the " +
+                             "branch being searched."))
+
+    @timeout(TIMEOUT)
+    # @unittest.skip("Skip get_move test.")  # Uncomment this line to skip test
+    def test_get_move_interface(self):
+        """Test CustomPlayer.get_move interface with simple input """
+        h, w = 9, 9  # board size
+        test_depth = 1
+        starting_location = (2, 7)
+        adversary_location = (0, 0)  # top left corner
+        iterative_search = False
+        search_method = "minimax"
+        heuristic = lambda g, p: 0.  # return 0 everywhere
+
+        # create a player agent & a game board
+        agentUT = game_agent.CustomPlayer(
+            test_depth, heuristic, iterative_search, search_method)
+
+        # Test that get_move returns a legal choice on an empty game board
+        board = isolation.Board(agentUT, 'null_agent', w, h)
+        legal_moves = board.get_legal_moves()
+        move = agentUT.get_move(board, legal_moves, lambda: 99)
+        self.assertIn(move, legal_moves,
+                      ("The get_move() function failed as player 1 on an " +
+                       "empty board. It should return coordinates on the " +
+                       "game board for the location of the agent's next " +
+                       "move. The move must be one of the legal moves on " +
+                       "the current game board."))
+
+        # Test that get_move returns a legal choice for first move as player 2
+        board = isolation.Board('null_agent', agentUT, w, h)
+        board.apply_move(starting_location)
+        legal_moves = board.get_legal_moves()
+        move = agentUT.get_move(board, legal_moves, lambda: 99)
+        self.assertIn(move, legal_moves,
+                      ("The get_move() function failed making the first " +
+                       "move as player 2 on a new board. It should return " +
+                       "coordinates on the game board for the location " +
+                       "of the agent's next move. The move must be one " +
+                       "of the legal moves on the current game board."))
+
+        # Test that get_move returns a legal choice after first move
+        board = isolation.Board(agentUT, 'null_agent', w, h)
+        board.apply_move(starting_location)
+        board.apply_move(adversary_location)
+        legal_moves = board.get_legal_moves()
+        move = agentUT.get_move(board, legal_moves, lambda: 99)
+        self.assertIn(move, legal_moves,
+                      ("The get_move() function failed as player 1 on a " +
+                       "game in progress. It should return coordinates on" +
+                       "the game board for the location of the agent's " +
+                       "next move. The move must be one of the legal moves " +
+                       "on the current game board."))
+
+    @timeout(TIMEOUT)
+    # @unittest.skip("Skip minimax test.")  # Uncomment this line to skip test
+    def test_minimax(self):
+        """Test CustomPlayer.minimax
+
+        This test uses a scoring function that returns a constant value based
+        on the location of the search agent on the board to force minimax to
+        choose a branch that visits those cells at a specific fixed-depth.
+        If minimax is working properly, it will visit a constant number of
+        nodes during the search and return one of the acceptable legal moves.
+        """
+        h, w = 7, 7  # board size
+        starting_location = (2, 3)
+        adversary_location = (0, 0)  # top left corner
+        iterative_search = False
+        method = "minimax"
+
+        # The agent under test starts at position (2, 3) on the board, which
+        # gives eight (8) possible legal moves [(0, 2), (0, 4), (1, 1), (1, 5),
+        # (3, 1), (3, 5), (4, 2), (4, 4)]. The search function will pick one of
+        # those moves based on the estimated score for each branch.  The value
+        # only changes on odd depths because even depths end on when the
+        # adversary has initiative.
+        value_table = [[0] * w for _ in range(h)]
+        value_table[1][5] = 1  # depth 1 & 2
+        value_table[4][3] = 2  # depth 3 & 4
+        value_table[6][6] = 3  # depth 5
+        heuristic = makeEvalTable(value_table)
+
+        # These moves are the branches that will lead to the cells in the value
+        # table for the search depths.
+        expected_moves = [set([(1, 5)]),
+                          set([(3, 1), (3, 5)]),
+                          set([(3, 5), (4, 2)])]
+
+        # Expected number of node expansions during search
+        counts = [(8, 8), (24, 10), (92, 27), (418, 32), (1650, 43)]
+
+        # Test fixed-depth search; note that odd depths mean that the searching
+        # player (student agent) has the last move, while even depths mean that
+        # the adversary has the last move before calling the heuristic
+        # evaluation function.
+        for idx in range(5):
+            test_depth = idx + 1
+            agentUT, board = self.initAUT(test_depth, heuristic,
+                                          iterative_search, method,
+                                          loc1=starting_location,
+                                          loc2=adversary_location)
+
+            # disable search timeout by returning a constant value
+            agentUT.time_left = lambda: 1e3
+            _, move = agentUT.minimax(board, test_depth)
+
+            num_explored_valid = board.counts[0] == counts[idx][0]
+            num_unique_valid = board.counts[1] == counts[idx][1]
+
+            self.assertTrue(num_explored_valid, WRONG_NUM_EXPLORED.format(
+                method, test_depth, counts[idx][0], board.counts[0]))
+
+            self.assertTrue(num_unique_valid, UNEXPECTED_VISIT.format(
+                method, test_depth, counts[idx][1], board.counts[1]))
+
+            self.assertIn(move, expected_moves[idx // 2], WRONG_MOVE.format(
+                method, test_depth, expected_moves[idx // 2], move))
+
+    @timeout(TIMEOUT)
+    # @unittest.skip("Skip alpha-beta test.")  # Uncomment this line to skip test
+    def test_alphabeta(self):
+        """Test CustomPlayer.alphabeta
+
+        This test uses a scoring function that returns a constant value based
+        on the branch being searched by alphabeta in the user agent, and forces
+        the search to prune on every other branch it visits. By using a huge
+        board where the players are too far apart to interact and every branch
+        has the same growth factor, the expansion and pruning must result in
+        an exact number of expanded nodes.
+        """
+        h, w = 101, 101  # board size
+        starting_location = (50, 50)
+        adversary_location = (0, 0)  # top left corner
+        iterative_search = False
+        method = "alphabeta"
+
+        # The agent under test starts in the middle of a huge board so that
+        # every branch has the same number of possible moves, so pruning any
+        # branch has the same effect during testing
+
+        # These are the expected number of node expansions for alphabeta search
+        # to explore the game tree to fixed depth.  The custom eval function
+        # used for this test ensures that some branches must be pruned, while
+        # the search should still return an optimal move.
+        counts = [(8, 8), (17, 10), (74, 42), (139, 51), (540, 119)]
+
+        for idx in range(len(counts)):
+            test_depth = idx + 1  # pruning guarantee requires min depth of 3
+            first_branch = []
+            heuristic = makeBranchEval(first_branch)
+            agentUT, board = self.initAUT(test_depth, heuristic,
+                                          iterative_search, method,
+                                          loc1=starting_location,
+                                          loc2=adversary_location,
+                                          w=w, h=h)
+
+            # disable search timeout by returning a constant value
+            agentUT.time_left = lambda: 1e3
+            _, move = agentUT.alphabeta(board, test_depth)
+
+            num_explored_valid = board.counts[0] == counts[idx][0]
+            num_unique_valid = board.counts[1] == counts[idx][1]
+
+            self.assertTrue(num_explored_valid, WRONG_NUM_EXPLORED.format(
+                method, test_depth, counts[idx][0], board.counts[0]))
+
+            self.assertTrue(num_unique_valid, UNEXPECTED_VISIT.format(
+                method, test_depth, counts[idx][1], board.counts[1]))
+
+            self.assertIn(move, first_branch, WRONG_MOVE.format(
+                method, test_depth, first_branch, move))
+
+    @timeout(TIMEOUT)
+    # @unittest.skip("Skip iterative deepening test.")  # Uncomment this line to skip test
+    def test_get_move(self):
+        """Test iterative deepening in CustomPlayer.get_move
+
+        Placing an agent on the game board and performing ID minimax search,
+        which should visit a specific number of unique nodes while expanding.
+        By forcing the search to timeout when a predetermined number of nodes
+        have been expanded, we can then verify that the expected number of
+        unique nodes have been visited.
+        """
+
+        class DynamicTimer():
+            """Dynamic Timer allows the time limit to be changed after the
+            timer is initialized so that the search timeout can be triggered
+            before the timer actually expires. This allows the timer to expire
+            when an event occurs, regardless of the clock time required until
+            the event happens.
+            """
+            def __init__(self, time_limit):
+                self.time_limit = time_limit
+                self.invoked = False
+                self.start_time = curr_time_millis()
+
+            def time_left(self):
+                return self.time_limit - (curr_time_millis() - self.start_time)
+
+        w, h = 11, 11  # board size
+        adversary_location = (0, 0)
+        method = "minimax"
+
+        # The agent under test starts at the positions indicated below, and
+        # performs an iterative deepening minimax search (minimax is easier to
+        # test because it always visits all nodes in the game tree at every
+        # level).
+        origins = [(2, 3), (6, 6), (7, 4), (4, 2), (0, 5), (10, 10)]
+        exact_counts = [(8, 8), (32, 10), (160, 39), (603, 35), (1861, 54), (3912, 62)]
+
+        for idx in range(len(origins)):
+
+            # set the initial timer high enough that the search will not
+            # timeout before triggering the dynamic timer to halt by visiting
+            # the expected number of nodes
+            time_limit = 1e4
+            timer = DynamicTimer(time_limit)
+            eval_fn = makeEvalStop(exact_counts[idx][0], timer)
+            agentUT, board = self.initAUT(-1, eval_fn, True, method,
+                                          origins[idx], adversary_location,
+                                          w, h)
+            legal_moves = board.get_legal_moves()
+            chosen_move = agentUT.get_move(board, legal_moves, timer.time_left)
+
+            diff_total = abs(board.counts[0] - exact_counts[idx][0])
+            diff_unique = abs(board.counts[1] - exact_counts[idx][1])
+
+            self.assertTrue(timer.invoked, WRONG_HEURISTIC)
+            self.assertTrue(diff_total <= 1 and diff_unique == 0, ID_FAIL)
+
+            self.assertTrue(chosen_move in legal_moves, INVALID_MOVE.format(
+                legal_moves, chosen_move))
+
+
+if __name__ == '__main__':
+    unittest.main()

Term-I – AI Foundations/02 - Game Playing Agent/game_agent.py

@@ -0,0 +1,428 @@
+"""This file contains all the classes you must complete for this project.
+
+You can use the test cases in `agent_test.py` to help during development, and
+augment the test suite with your own test cases to further test your code.
+
+You must test your agent's strength against a set of agents with known relative
+strength using `tournament.py` and include the results in your report.
+"""
+
+import random
+import math
+
+class Timeout(Exception):
+    """Subclass base exception for code clarity."""
+    pass
+
+# Heuristic Definitions
+def penalize_corners(game, player):
+    """Heuristic which penalizes the player for moving to a corner and returns
+    a float value of the difference in legal moves of the two players.
+
+    PARAMETERS:
+        game :: `isolation.Board`
+            An instance of `isolation.Board` encoding the current state of
+            the game (e.g. player locations and blocked cells).
+
+        player :: object
+            A player instance in the current game.
+
+    RETURNS:
+        float :: The heuristic value of the current game state to a specified
+        player.
+    """
+
+    # Check if game is already over
+    if game.is_loser(player):
+        return float('-inf')
+    if game.is_winner(player):
+        return float('inf')
+
+    own_moves = len(game.get_legal_moves(player))
+    opp_moves = len(game.get_legal_moves(game.get_opponent(player)))
+
+    # Penalize for moving to corner
+    corner_weight = 4
+    if is_corner(game, game.get_player_location(player)):
+        own_moves -= corner_weight
+
+    return float(own_moves - opp_moves)
+
+def is_corner(game, player_location):
+    """Returns 'True' if player's current location is a board corner.
+    """
+    corner_positions = [(0, 0), (0, game.height - 1), \
+                        (game.width - 1, 0), \
+                        (game.width - 1, game.height - 1)]
+    return player_location in corner_positions
+
+def run_away(game, player):
+    """Heuristic to credit player moves that are farther from the opponent and
+    returns a float value of the difference in legal moves of the two players.
+
+    PARAMETERS:
+        game :: `isolation.Board`
+            An instance of `isolation.Board` encoding the current state of
+            the game (e.g. player locations and blocked cells).
+
+        player :: object
+            A player instance in the current game.
+
+    RETURNS:
+        float :: The heuristic value of the current game state to a specified
+        player.
+    """
+
+    # Check if game is already over
+    if game.is_loser(player):
+        return float('-inf')
+    if game.is_winner(player):
+        return float('inf')
+
+    opp_player = game.get_opponent(player)
+    own_moves = len(game.get_legal_moves(player))
+    opp_moves = len(game.get_legal_moves(game.get_opponent(player)))
+
+    # Reward player for choosing moves farther from opponent
+    distance = euclid_distance(game, player, opp_player)
+    own_moves += distance
+
+    return((own_moves - opp_moves))
+
+def euclid_distance(game, player, opp_player):
+    """Returns distance between current positions of two players.
+    """
+    player_location = game.get_player_location(player)
+    opp_location = game.get_player_location(opp_player)
+    x_dist = math.pow(player_location[0] - opp_location[0], 2)
+    y_dist = math.pow(player_location[1] - opp_location[1], 2)
+    return math.sqrt(x_dist + y_dist)
+
+def foresee_moves(game, player):
+    """Heuristic that returns a float of the difference in the number of legal
+    moves in the current game state plus the number of possible moves.
+
+    PARAMETERS:
+        game :: `isolation.Board`
+            An instance of `isolation.Board` encoding the current state of
+            the game (e.g. player locations and blocked cells).
+
+        player :: object
+            A player instance in the current game.
+
+    RETURNS
+        float :: The heuristic value of the current game state to a specified
+        player.
+    """
+
+    # Check if game is already over
+    if game.is_loser(player):
+        return float('-inf')
+    if game.is_winner(player):
+        return float('inf')
+
+    own_legal_moves = game.get_legal_moves(player)
+    own_moves = len(own_legal_moves)
+    for m in own_legal_moves:
+        own_moves += len(game.get_moves(m))
+
+    opp_legal_moves = game.get_legal_moves(game.get_opponent(player))
+    opp_moves = len(opp_legal_moves)
+    for m in opp_legal_moves:
+        opp_moves += len(game.get_moves(m))
+
+    return float(own_moves - opp_moves)
+
+def normalized_moves(game, player):
+    """
+    Heuristic that returns the difference of player's and opponent's move(s),
+    divided by total of all remaining legal moves.
+
+    PARAMETERS:
+        game :: `isolation.Board`
+            An instance of `isolation.Board` encoding the current state of
+            the game (e.g. player locations and blocked cells).
+
+        player :: object
+            A player instance in the current game.
+
+    RETURNS:
+        float :: The heuristic value of the current game state to a specified
+        player.
+    """
+
+    if game.is_loser(player):
+        return float("-inf")
+
+    if game.is_winner(player):
+        return float("inf")
+
+    own_moves = len(game.get_legal_moves(player))
+    opp_moves = len(game.get_legal_moves(game.get_opponent(player)))
+
+    score = float((own_moves - opp_moves) / (own_moves + opp_moves))
+    return score
+
+def custom_score(game, player):
+    """Calclate the heuristic value of a game state from the point of view of
+    the given player.
+
+    Note: this function should be called from within a Player instance as
+    `self.score()` - you should not need to call this function directly.
+
+    PARAMETERS:
+        game :: `isolation.Board`
+            An instance of `isolation.Board` encoding the current state of
+            the game (e.g. player locations and blocked cells).
+
+        player :: object
+            A player instance in the current game.
+
+    RETURNS:
+        float :: The heuristic value of the current game state to a specified
+        player.
+    """
+
+    # return penalize_corners(game, player)
+    # return run_away(game, player)
+    return foresee_moves(game, player)
+    # return normalized_moves(game, player)
+
+
+class CustomPlayer:
+    """Game-playing agent that chooses a move using your evaluation function
+    and a depth-limited minimax algorithm with alpha-beta pruning. You must
+    finish and test this player to make sure it properly uses minimax and
+    alpha-beta to return a good move before the search time limit expires.
+
+    PARAMETERS:
+        search_depth :: int (optional)
+            A strictly positive integer (i.e., 1, 2, 3,...) for the number of
+            layers in the game tree to explore for fixed-depth search i.e., a
+            depth of one (1) would only explore the immediate sucessors of the
+            current state. This parameter should be ignored when
+            iterative = True.
+
+        score_fn :: callable (optional)
+            A function to use for heuristic evaluation of game states.
+
+        iterative :: boolean (optional)
+            Flag indicating whether to perform fixed-depth search (False) or
+            iterative deepening search (True). When True, search_depth should
+            be ignored and no limit to search depth.
+
+        method :: {'minimax', 'alphabeta'} (optional)
+            The name of the search method to use in get_move().
+
+        timeout :: float (optional)
+            Time remaining (in milliseconds) when search is aborted. Should be
+            a positive value large enough to allow the function to return
+            before the timer expires.
+    """
+
+    def __init__(self, search_depth=3, score_fn=custom_score,
+                 iterative=True, method='minimax', timeout=10.):
+        self.search_depth = search_depth
+        self.iterative = iterative
+        self.score = score_fn
+        self.method = method
+        self.time_left = None
+        self.TIMER_THRESHOLD = timeout
+
+    def get_move(self, game, legal_moves, time_left):
+        """Search for the best move from the available legal moves and return a
+        result before the time limit expires.
+
+        This function must perform iterative deepening if self.iterative=True,
+        and it must use the search method (minimax or alphabeta) corresponding
+        to the self.method value.
+
+        PARAMETERS:
+            game :: `isolation.Board`
+                An instance of `isolation.Board` encoding the current state of
+                the game (e.g., player locations and blocked cells).
+
+            legal_moves :: list<(int, int)>
+                A list containing legal moves. Moves are encoded as tuples of
+                pairs of ints defining the next (row, col) for the agent to
+                occupy.
+
+            time_left :: callable
+                A function that returns the number of milliseconds left in the
+                current turn. Returning with any less than 0 ms remaining
+                forfeits the game.
+
+        RETURNS:
+        (int, int)
+            Board coordinates corresponding to a legal move; may return
+            (-1, -1) if there are no available legal moves.
+        """
+
+        self.time_left = time_left
+
+        if not legal_moves:
+            return (-1, -1)
+
+        move = None
+        try:
+            # The search method call (alpha beta or minimax) should happen in
+            # here in order to avoid timeout. The try/except block will
+            # automatically catch the exception raised by the search method
+            # when the timer gets close to expiring.
+            algorithm_name = getattr(self, self.method)
+            if self.iterative:
+                depth = 1 # Depth used for iterative deepening.
+                while True:
+                    _, move = algorithm_name(game, depth)
+                    depth += 1
+            else:
+                _, move = algorithm_name(game, self.search_depth)
+        except Timeout:
+            # Return the best move so far in-case of time-out.
+            return move
+        return move
+
+    def minimax(self, game, depth, maximizing_player=True):
+        """Implement the minimax search algorithm as described in the lectures.
+
+        PARAMETERS:
+            game :: isolation.Board
+                An instance of the Isolation game `Board` class representing
+                the current game state.
+
+            depth :: int
+                Depth is an integer representing the maximum number of plies to
+                search in the game tree before aborting.
+
+            maximizing_player :: bool
+                Flag indicating whether the current search depth corresponds to
+                a maximizing layer (True) or a minimizing layer (False).
+
+        RETURNS:
+            float :: The score for the current search branch
+
+            tuple(int, int) :: The best move for the current branch;
+            (-1, -1) for no legal moves
+        """
+
+        if self.time_left() < self.TIMER_THRESHOLD:
+            raise Timeout()
+
+        # Return heuristic value of game when search has reached max depth
+        if depth == 0:
+            # Note: No need to return the move.
+            # The max/min players keep a track of them.
+            return (self.score(game, self), None)
+
+        # Verify if there are any available legal moves
+        legal_moves = game.get_legal_moves()
+        if not legal_moves:
+            return (game.utility(self), (-1, -1))
+
+        # Maximize/minimize play accordingly
+        if maximizing_player:
+            return self.minimax_maximize(game, legal_moves, depth)
+        else:
+            return self.minimax_minimize(game, legal_moves, depth)
+
+    def minimax_maximize(self, game, legal_moves, depth):
+        """Minimax maximizer player. Returns the highest (score, move)
+        tuple found in the game."""
+        highest_score, selected_move = (float('-inf'), (-1, -1))
+        for move in legal_moves:
+            score, _ = self.minimax(game.forecast_move(move), \
+            depth - 1, False)
+            highest_score, selected_move = max((highest_score, \
+            selected_move), (score, move))
+        return (highest_score, selected_move)
+
+    def minimax_minimize(self, game, legal_moves, depth):
+        """Minimax maximizer player. Returns the lowest (score, move)
+        tuple found in the game."""
+        lowest_score, selected_move = (float('inf'), (-1, -1))
+        for move in legal_moves:
+            score, _ = self.minimax(game.forecast_move(move), depth - 1)
+            lowest_score, selected_move = min((lowest_score, \
+            selected_move), (score, move))
+        return (lowest_score, selected_move)
+
+    def alphabeta(self, game, depth, alpha=float("-inf"), beta=float("inf"),
+    maximizing_player=True):
+        """Implement minimax search with alpha-beta pruning as described in the
+        lectures.
+
+        PARAMETERS:
+            game :: `isolation.Board`
+                An instance of the Isolation game 'Board' class representing
+                the current game state.
+
+            depth :: int
+                Depth is an integer representing the maximum number of plies
+                to search in the game tree before aborting.
+
+            alpha :: float
+                Alpha limits the lower bound of search on minimizing layers.
+
+            beta :: float
+                Beta limits the upper bound of search on maximizing layers.
+
+            maximizing_player :: bool
+                Flag indicating whether the current search depth corresponds
+                to a maximizing layer (True) or a minimizing layer (False).
+
+        RETURNS:
+            float
+                The score for the current search branch.
+
+            tuple(int, int)
+                The best move for the current branch; (-1, -1) for no legal
+                moves.
+        """
+
+        if self.time_left() < self.TIMER_THRESHOLD:
+            raise Timeout()
+
+        # Return heuristic value of game when search has reached max-depth
+        if depth == 0:
+            # Move need not be returned.
+            # Min/Max players keep a track of them.
+            return (self.score(game, self), None)
+
+        # Verify if there are any available legal moves
+        legal_moves = game.get_legal_moves()
+        if not legal_moves:
+            return (game.utility(self), (-1, -1))
+
+        # Maximize/minimize play accordingly
+        if maximizing_player:
+            return self.ab_maximize_play(game, legal_moves, depth, alpha, beta)
+        else:
+            return self.ab_minimize_play(game, legal_moves, depth, alpha, beta)
+
+    def ab_maximize_play(self, game, legal_moves, depth, alpha, beta):
+        """Alphabeta maximizer player. Returns the highest score/move
+        tuple found in game, pruning the search tree."""
+        highest_score, selected_move = (float('-inf'), (-1, -1))
+        for move in legal_moves:
+            score, _ = self.alphabeta(game.forecast_move(move), \
+            depth - 1, alpha, beta, False)
+            if score > alpha:
+                alpha = score
+                highest_score, selected_move = score, move
+            if alpha >= beta:
+                break
+        return (highest_score, selected_move)
+
+    def ab_minimize_play(self, game, legal_moves, depth, alpha, beta):
+        """Alphabeta minimizer player. Returns the lowest score/move
+        tuple found in game, pruning the search tree."""
+        lowest_score, selected_move = (float('inf'), (-1, -1))
+        for move in legal_moves:
+            score, _ = self.alphabeta(game.forecast_move(move), \
+            depth - 1, alpha, beta, True)
+            if score < beta:
+                beta = score
+                lowest_score, selected_move = score, move
+            if beta <= alpha:
+                break
+        return (lowest_score, selected_move)

Term-I – AI Foundations/02 - Game Playing Agent/isolation/__init__.py

@@ -0,0 +1,11 @@
+"""
+This library provides a Python implementation of the game Isolation.
+Isolation is a deterministic, two-player game of perfect information in
+which the players alternate turns moving between cells on a square grid
+(like a checkerboard).  Whenever either player occupies a cell, that
+location is blocked for the rest of the game. The first player with no
+legal moves loses, and the opponent is declared the winner.
+"""
+
+# Make the Board class available at the root of the module for imports
+from .isolation import Board

Term-I – AI Foundations/02 - Game Playing Agent/isolation/isolation.py

@@ -0,0 +1,343 @@
+"""
+This file contains the `Board` class, which implements the rules for the
+game Isolation as described in lecture, modified so that the players move
+like knights in chess rather than queens.
+
+You MAY use and modify this class, however ALL function signatures must
+remain compatible with the defaults provided, and none of your changes will
+be available to project reviewers.
+"""
+import random
+import timeit
+from copy import copy
+
+TIME_LIMIT_MILLIS = 150
+
+
+class Board(object):
+    """Implement a model for the game Isolation assuming each player moves like
+    a knight in chess.
+
+    Parameters
+    ----------
+    player_1 : object
+        An object with a get_move() function. This is the only function
+        directly called by the Board class for each player.
+
+    player_2 : object
+        An object with a get_move() function. This is the only function
+        directly called by the Board class for each player.
+
+    width : int (optional)
+        The number of columns that the board should have.
+
+    height : int (optional)
+        The number of rows that the board should have.
+    """
+    BLANK = 0
+    NOT_MOVED = None
+
+    def __init__(self, player_1, player_2, width=7, height=7):
+        self.width = width
+        self.height = height
+        self.move_count = 0
+        self._player_1 = player_1
+        self._player_2 = player_2
+        self._active_player = player_1
+        self._inactive_player = player_2
+
+        # The last 3 entries of the board state includes initiative (0 for
+        # player 1, 1 for player 2) player 2 last move, and player 1 last move
+        self._board_state = [Board.BLANK] * (width * height + 3)
+        self._board_state[-1] = Board.NOT_MOVED
+        self._board_state[-2] = Board.NOT_MOVED
+
+    def hash(self):
+        return str(self._board_state).__hash__()
+
+    @property
+    def active_player(self):
+        """The object registered as the player holding initiative in the
+        current game state.
+        """
+        return self._active_player
+
+    @property
+    def inactive_player(self):
+        """The object registered as the player in waiting for the current
+        game state.
+        """
+        return self._inactive_player
+
+    def get_opponent(self, player):
+        """Return the opponent of the supplied player.
+
+        Parameters
+        ----------
+        player : object
+            An object registered as a player in the current game. Raises an
+            error if the supplied object is not registered as a player in
+            this game.
+
+        Returns
+        -------
+        object
+            The opponent of the input player object.
+        """
+        if player == self._active_player:
+            return self._inactive_player
+        elif player == self._inactive_player:
+            return self._active_player
+        raise RuntimeError("`player` must be an object registered as a player in the current game.")
+
+    def copy(self):
+        """ Return a deep copy of the current board. """
+        new_board = Board(self._player_1, self._player_2, width=self.width, height=self.height)
+        new_board.move_count = self.move_count
+        new_board._active_player = self._active_player
+        new_board._inactive_player = self._inactive_player
+        new_board._board_state = copy(self._board_state)
+        return new_board
+
+    def forecast_move(self, move):
+        """Return a deep copy of the current game with an input move applied to
+        advance the game one ply.
+
+        Parameters
+        ----------
+        move : (int, int)
+            A coordinate pair (row, column) indicating the next position for
+            the active player on the board.
+
+        Returns
+        -------
+        isolation.Board
+            A deep copy of the board with the input move applied.
+        """
+        new_board = self.copy()
+        new_board.apply_move(move)
+        return new_board
+
+    def move_is_legal(self, move):
+        """Test whether a move is legal in the current game state.
+
+        Parameters
+        ----------
+        move : (int, int)
+            A coordinate pair (row, column) indicating the next position for
+            the active player on the board.
+
+        Returns
+        -------
+        bool
+            Returns True if the move is legal, False otherwise
+        """
+        idx = move[0] + move[1] * self.height
+        return (0 <= move[0] < self.height and 0 <= move[1] < self.width and
+                self._board_state[idx] == Board.BLANK)
+
+    def get_blank_spaces(self):
+        """Return a list of the locations that are still available on the board.
+        """
+        return [(i, j) for j in range(self.width) for i in range(self.height)
+                if self._board_state[i + j * self.height] == Board.BLANK]
+
+    def get_player_location(self, player):
+        """Find the current location of the specified player on the board.
+
+        Parameters
+        ----------
+        player : object
+            An object registered as a player in the current game.
+
+        Returns
+        -------
+        (int, int) or None
+            The coordinate pair (row, column) of the input player, or None
+            if the player has not moved.
+        """
+        if player == self._player_1:
+            if self._board_state[-1] == Board.NOT_MOVED:
+                return Board.NOT_MOVED
+            idx = self._board_state[-1]
+        elif player == self._player_2:
+            if self._board_state[-2] == Board.NOT_MOVED:
+                return Board.NOT_MOVED
+            idx = self._board_state[-2]
+        else:
+            raise RuntimeError(
+                "Invalid player in get_player_location: {}".format(player))
+        w = idx // self.height
+        h = idx % self.height
+        return (h, w)
+
+    def get_legal_moves(self, player=None):
+        """Return the list of all legal moves for the specified player.
+
+        Parameters
+        ----------
+        player : object (optional)
+            An object registered as a player in the current game. If None,
+            return the legal moves for the active player on the board.
+
+        Returns
+        -------
+        list<(int, int)>
+            The list of coordinate pairs (row, column) of all legal moves
+            for the player constrained by the current game state.
+        """
+        if player is None:
+            player = self.active_player
+        return self.get_moves(self.get_player_location(player))
+
+    def apply_move(self, move):
+        """Move the active player to a specified location.
+
+        Parameters
+        ----------
+        move : (int, int)
+            A coordinate pair (row, column) indicating the next position for
+            the active player on the board.
+        """
+        idx = move[0] + move[1] * self.height
+        last_move_idx = int(self.active_player == self._player_2) + 1
+        self._board_state[-last_move_idx] = idx
+        self._board_state[idx] = 1
+        self._board_state[-3] ^= 1
+        self._active_player, self._inactive_player = self._inactive_player, self._active_player
+        self.move_count += 1
+
+    def is_winner(self, player):
+        """ Test whether the specified player has won the game. """
+        return player == self._inactive_player and not self.get_legal_moves(self._active_player)
+
+    def is_loser(self, player):
+        """ Test whether the specified player has lost the game. """
+        return player == self._active_player and not self.get_legal_moves(self._active_player)
+
+    def utility(self, player):
+        """Returns the utility of the current game state from the perspective
+        of the specified player.
+
+                    /  +infinity,   "player" wins
+        utility =  |   -infinity,   "player" loses
+                    \          0,    otherwise
+
+        Parameters
+        ----------
+        player : object (optional)
+            An object registered as a player in the current game. If None,
+            return the utility for the active player on the board.
+
+        Returns
+        ----------
+        float
+            The utility value of the current game state for the specified
+            player. The game has a utility of +inf if the player has won,
+            a value of -inf if the player has lost, and a value of 0
+            otherwise.
+        """
+        if not self.get_legal_moves(self._active_player):
+
+            if player == self._inactive_player:
+                return float("inf")
+
+            if player == self._active_player:
+                return float("-inf")
+
+        return 0.
+
+    def get_moves(self, loc):
+        """Generate the list of possible moves for an L-shaped motion (like a
+        knight in chess).
+        """
+        if loc == Board.NOT_MOVED:
+            return self.get_blank_spaces()
+
+        r, c = loc
+        directions = [(-2, -1), (-2, 1), (-1, -2), (-1, 2),
+                      (1, -2), (1, 2), (2, -1), (2, 1)]
+        valid_moves = [(r + dr, c + dc) for dr, dc in directions
+                       if self.move_is_legal((r + dr, c + dc))]
+        random.shuffle(valid_moves)
+        return valid_moves
+
+    def print_board(self):
+        """DEPRECATED - use Board.to_string()"""
+        return self.to_string()
+
+    def to_string(self, symbols=['1', '2']):
+        """Generate a string representation of the current game state, marking
+        the location of each player and indicating which cells have been
+        blocked, and which remain open.
+        """
+        p1_loc = self._board_state[-1]
+        p2_loc = self._board_state[-2]
+
+        col_margin = len(str(self.height - 1)) + 1
+        prefix = "{:<" + "{}".format(col_margin) + "}"
+        offset = " " * (col_margin + 3)
+        out = offset + '   '.join(map(str, range(self.width))) + '\n\r'
+        for i in range(self.height):
+            out += prefix.format(i) + ' | '
+            for j in range(self.width):
+                idx = i + j * self.height
+                if not self._board_state[idx]:
+                    out += ' '
+                elif p1_loc == idx:
+                    out += symbols[0]
+                elif p2_loc == idx:
+                    out += symbols[1]
+                else:
+                    out += '-'
+                out += ' | '
+            out += '\n\r'
+
+        return out
+
+    def play(self, time_limit=TIME_LIMIT_MILLIS):
+        """Execute a match between the players by alternately soliciting them
+        to select a move and applying it in the game.
+
+        Parameters
+        ----------
+        time_limit : numeric (optional)
+            The maximum number of milliseconds to allow before timeout
+            during each turn.
+
+        Returns
+        ----------
+        (player, list<[(int, int),]>, str)
+            Return multiple including the winning player, the complete game
+            move history, and a string indicating the reason for losing
+            (e.g., timeout or invalid move).
+        """
+        move_history = []
+
+        time_millis = lambda: 1000 * timeit.default_timer()
+
+        while True:
+
+            legal_player_moves = self.get_legal_moves()
+            game_copy = self.copy()
+
+            move_start = time_millis()
+            time_left = lambda : time_limit - (time_millis() - move_start)
+            curr_move = self._active_player.get_move(
+                game_copy, legal_player_moves, time_left)
+            move_end = time_left()
+
+            if curr_move is None:
+                curr_move = Board.NOT_MOVED
+
+            if move_end < 0:
+                return self._inactive_player, move_history, "timeout"
+
+            if curr_move not in legal_player_moves:
+                if len(legal_player_moves) > 0:
+                    return self._inactive_player, move_history, "forfeit"
+                return self._inactive_player, move_history, "illegal move"
+
+            move_history.append(list(curr_move))
+
+            self.apply_move(curr_move)

Term-I – AI Foundations/02 - Game Playing Agent/isoviz/LICENSE.txt

@@ -0,0 +1,21 @@
+Copyright 2013 Chris Oakman
+http://chessboardjs.com/
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Term-I – AI Foundations/02 - Game Playing Agent/isoviz/css/chessboard.css

@@ -0,0 +1,91 @@
+/*!
+ * chessboard.js v0.3.0
+ *
+ * Copyright 2013 Chris Oakman
+ * Released under the MIT license
+ * https://github.com/oakmac/chessboardjs/blob/master/LICENSE
+ *
+ * Date: 10 Aug 2013
+ */
+
+td {
+  font-size: 1.5em;
+}
+
+/* clearfix */
+.clearfix-7da63 {
+  clear: both;
+}
+
+/* board */
+.board-b72b1 {
+  border: 2px solid #404040;
+  -moz-box-sizing: content-box;
+  box-sizing: content-box;
+}
+
+/* square */
+.square-55d63 {
+  float: left;
+  position: relative;
+
+  /* disable any native browser highlighting */
+  -webkit-touch-callout: none;
+    -webkit-user-select: none;
+     -khtml-user-select: none;
+       -moz-user-select: none;
+        -ms-user-select: none;
+            user-select: none;
+}
+
+/* white square */
+.white-1e1d7.blocked {
+  background-color: #999999;
+  color: #333333;
+}
+
+/* black square */
+.black-3c85d.blocked {
+  background-color: #333333;
+  color: #999999;
+}
+
+/* white square */
+.white-1e1d7 {
+  background-color: #f0d9b5;
+  color: #b58863;
+}
+
+/* black square */
+.black-3c85d {
+  background-color: #b58863;
+  color: #f0d9b5;
+}
+
+/* white square */
+.square-55d63.win {
+  background-color: #339900;
+  color: #999999;
+}
+
+/* black square */
+.square-55d63.lose {
+  background-color: #990000;
+  color: #333333;
+}
+
+/* notation */
+.notation-322f9 {
+  cursor: default;
+  font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
+  font-size: 14px;
+  position: absolute;
+}
+.alpha-d2270 {
+  bottom: 1px;
+  right: 3px;
+}
+.numeric-fc462 {
+  top: 2px;
+  left: 2px;
+}

Term-I – AI Foundations/02 - Game Playing Agent/isoviz/display.html

@@ -0,0 +1,139 @@
+<!doctype html>
+<html>
+<head>
+  <meta charset="utf-8" />
+  <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1" />
+  <title>Empty Board Example</title>
+
+  <link rel="stylesheet" href="css/chessboard.css" />
+  <style>
+  #display {
+  	float: left;
+  	margin: 50px;
+  	padding: 0px 50px;
+  }
+  #board {
+  	width: 600px;
+  	display: inline-block;
+  	vertical-align: top;
+  }
+  #match {
+  	margin-left: 20px;
+  	width: 275px;
+  	display: inline-block;
+  	vertical-align: top;
+  }
+  </style>
+</head>
+<body style="font-family: monospace;">
+
+<div>
+	<form id="game_form">
+	  Player1:<br>
+	  <input type="text" name="player1" value="Player1">
+	  <br>
+	  Player2:<br>
+	  <input type="text" name="player2" value="Player2">
+	  <br>
+	  Move History:<br>
+	  <textarea rows="3" cols="120" name="moves" placeholder="[[0, 0], [3, 2], ...]"></textarea>
+	  <br>
+	  <input type="submit" id="runGame" value="Run Game">
+	</form> 
+</div>
+
+<div id="display">
+	<div id="match">
+		<table id="moves" style="text-align: center;"></table>
+	</div>
+	<div id="board"></div>
+</div>
+
+<script src="js/json3.min.js"></script>
+<script src="js/jquery-1.10.1.min.js"></script>
+<script src="js/chessboard.js"></script>
+<script>
+var init = function() {
+
+	var ind2alpha = function(xy) {
+		var alpha = "abcdefg";
+		var num = "1234567";
+		return alpha[xy[1]] + num[6 - xy[0]];
+	};
+
+	var runGame = function() {
+		
+		event.preventDefault();
+		
+		form = document.getElementById("game_form");
+		if ( !form.player1.value || !form.player2.value || !form.moves.value)
+			return;
+
+		document.getElementById("runGame").disabled = true;
+		game = {player1: form.player1.value,
+				player2: form.player2.value,
+				moves: JSON.parse(form.moves.value)};
+
+		var interval = 500;  // Length of the pause between moves (in milliseconds)
+
+		// Build the moves table by adding a header
+		var table = document.getElementById("moves");
+		var header = table.createTHead();
+		var row = header.insertRow();
+		var cell = row.insertCell();
+		cell.setAttribute("colspan", 2);
+		cell.innerHTML = "<h3>" + game["player1"] + " vs " + game["player2"] + "</h3>";
+
+		// Add the pieces in their starting positions directly to the board
+		p0 = ind2alpha(game["moves"][0]);
+		p1 = ind2alpha(game["moves"][1]);
+		pos = {}
+		pos[p0] = "wN";
+		pos[p1] = "bN";
+		board.position(pos);
+		row = table.insertRow();
+		cell = row.insertCell();
+		cell.innerHTML = "(" +  game["moves"][0] + ")";
+		cell = row.insertCell();
+		cell.innerHTML = "(" + game["moves"][1] + ")";
+		idx = 0;
+
+		// Perform moves until the list is exhausted
+		timer = window.setInterval(function() { doMove() }, interval);
+
+		function doMove() {
+
+			var start = game["moves"][idx];
+			var end = game["moves"][idx + 2];
+			var move = ind2alpha(start) + "-" + ind2alpha(end);
+			board.move(move);
+
+			// Write the player location in the moves table
+			if (idx % 2 == 0) {
+				row = table.insertRow();
+				cell = row.insertCell();
+				cell.innerHTML = "(" + game["moves"][idx + 2] + ")";
+			} else {
+				cell = row.insertCell();
+				cell.innerHTML = "(" + game["moves"][idx + 2] + ")";
+			}
+
+			idx++;
+			// quit when the game is resolved
+			if (idx >= game["moves"].length - 2) {
+				var winCell = ind2alpha(game["moves"][idx + 1]);
+				var loseCell = ind2alpha(game["moves"][idx]);
+				board.finalize(winCell, loseCell);
+				window.clearInterval(timer);
+				return;
+			}
+		};
+	};
+
+	var board = ChessBoard('board');
+	document.getElementById("game_form").addEventListener('submit', runGame);
+};
+$(document).ready(init);
+</script>
+</body>
+</html>

Term-I – AI Foundations/02 - Game Playing Agent/isoviz/js/chessboard.js

@@ -0,0 +1,1183 @@
+/*!
+ * chessboard.js v0.3.0
+ *
+ * Copyright 2013 Chris Oakman
+ * Released under the MIT license
+ * http://chessboardjs.com/license
+ *
+ * Date: 10 Aug 2013
+ */
+
+// start anonymous scope
+;(function() {
+'use strict';
+
+//------------------------------------------------------------------------------
+// Chess Util Functions
+//------------------------------------------------------------------------------
+var COLUMNS = 'abcdefg'.split('');
+
+function validMove(move) {
+  // move should be a string
+  if (typeof move !== 'string') return false;
+
+  // move should be in the form of "e2-e4", "f6-d5"
+  var tmp = move.split('-');
+  if (tmp.length !== 2) return false;
+
+  return (validSquare(tmp[0]) === true && validSquare(tmp[1]) === true);
+}
+
+function validSquare(square) {
+  if (typeof square !== 'string') return false;
+  return (square.search(/^[a-h][1-7]$/) !== -1);
+}
+
+function validPieceCode(code) {
+  if (typeof code !== 'string') return false;
+  return (code.search(/^[bw][N]$/) !== -1);
+}
+
+// TODO: this whole function could probably be replaced with a single regex
+function validFen(fen) {
+  if (typeof fen !== 'string') return false;
+
+  // cut off any move, castling, etc info from the end
+  // we're only interested in position information
+  fen = fen.replace(/ .+$/, '');
+
+  // FEN should be 8 sections separated by slashes
+  var chunks = fen.split('/');
+  if (chunks.length !== 7) return false;
+
+  // check the piece sections
+  for (var i = 0; i < 7; i++) {
+    if (chunks[i] === '' ||
+        chunks[i].length > 7 ||
+        chunks[i].search(/[^nN1-7]/) !== -1) {
+      return false;
+    }
+  }
+
+  return true;
+}
+
+function validPositionObject(pos) {
+  if (typeof pos !== 'object') return false;
+
+  for (var i in pos) {
+    if (pos.hasOwnProperty(i) !== true) continue;
+
+    if (validSquare(i) !== true || validPieceCode(pos[i]) !== true) {
+      return false;
+    }
+  }
+
+  return true;
+}
+
+// convert FEN piece code to bP, wK, etc
+function fenToPieceCode(piece) {
+  // black piece
+  if (piece.toLowerCase() === piece) {
+    return 'b' + piece.toUpperCase();
+  }
+
+  // white piece
+  return 'w' + piece.toUpperCase();
+}
+
+// convert bP, wK, etc code to FEN structure
+function pieceCodeToFen(piece) {
+  var tmp = piece.split('');
+
+  // white piece
+  if (tmp[0] === 'w') {
+    return tmp[1].toUpperCase();
+  }
+
+  // black piece
+  return tmp[1].toLowerCase();
+}
+
+// convert FEN string to position object
+// returns false if the FEN string is invalid
+function fenToObj(fen) {
+  if (validFen(fen) !== true) {
+    return false;
+  }
+
+  // cut off any move, castling, etc info from the end
+  // we're only interested in position information
+  fen = fen.replace(/ .+$/, '');
+
+  var rows = fen.split('/');
+  var position = {};
+
+  var currentRow = 7;
+  for (var i = 0; i < 7; i++) {
+    var row = rows[i].split('');
+    var colIndex = 0;
+
+    // loop through each character in the FEN section
+    for (var j = 0; j < row.length; j++) {
+      // number / empty squares
+      if (row[j].search(/[1-7]/) !== -1) {
+        var emptySquares = parseInt(row[j], 10);
+        colIndex += emptySquares;
+      }
+      // piece
+      else {
+        var square = COLUMNS[colIndex] + currentRow;
+        position[square] = fenToPieceCode(row[j]);
+        colIndex++;
+      }
+    }
+
+    currentRow--;
+  }
+
+  return position;
+}
+
+// position object to FEN string
+// returns false if the obj is not a valid position object
+function objToFen(obj) {
+  if (validPositionObject(obj) !== true) {
+    return false;
+  }
+
+  var fen = '';
+
+  var currentRow = 7;
+  for (var i = 0; i < 7; i++) {
+    for (var j = 0; j < 7; j++) {
+      var square = COLUMNS[j] + currentRow;
+
+      // piece exists
+      if (obj.hasOwnProperty(square) === true) {
+        fen += pieceCodeToFen(obj[square]);
+      }
+
+      // empty space
+      else {
+        fen += '1';
+      }
+    }
+
+    if (i !== 6) {
+      fen += '/';
+    }
+
+    currentRow--;
+  }
+
+  // squeeze the numbers together
+  // haha, I love this solution...
+  fen = fen.replace(/1111111/g, '7');
+  fen = fen.replace(/111111/g, '6');
+  fen = fen.replace(/11111/g, '5');
+  fen = fen.replace(/1111/g, '4');
+  fen = fen.replace(/111/g, '3');
+  fen = fen.replace(/11/g, '2');
+
+  return fen;
+}
+
+window['ChessBoard'] = window['ChessBoard'] || function(containerElOrId, cfg) {
+'use strict';
+
+cfg = cfg || {};
+
+//------------------------------------------------------------------------------
+// Constants
+//------------------------------------------------------------------------------
+
+var MINIMUM_JQUERY_VERSION = '1.7.0',
+  START_FEN = '7/7/7/7/7/7/7/7',
+  START_POSITION = fenToObj(START_FEN);
+
+// use unique class names to prevent clashing with anything else on the page
+// and simplify selectors
+var CSS = {
+  alpha: 'alpha-d2270',
+  black: 'black-3c85d',
+  board: 'board-b72b1',
+  chessboard: 'chessboard-63f37',
+  clearfix: 'clearfix-7da63',
+  highlight1: 'highlight1-32417',
+  highlight2: 'highlight2-9c5d2',
+  notation: 'notation-322f9',
+  numeric: 'numeric-fc462',
+  piece: 'piece-417db',
+  row: 'row-5277c',
+  sparePieces: 'spare-pieces-7492f',
+  sparePiecesBottom: 'spare-pieces-bottom-ae20f',
+  sparePiecesTop: 'spare-pieces-top-4028b',
+  square: 'square-55d63',
+  white: 'white-1e1d7'
+};
+
+//------------------------------------------------------------------------------
+// Module Scope Variables
+//------------------------------------------------------------------------------
+
+// DOM elements
+var containerEl,
+  boardEl,
+  draggedPieceEl,
+  sparePiecesTopEl,
+  sparePiecesBottomEl;
+
+// constructor return object
+var widget = {};
+
+//------------------------------------------------------------------------------
+// Stateful
+//------------------------------------------------------------------------------
+
+var ANIMATION_HAPPENING = false,
+  BOARD_BORDER_SIZE = 2,
+  CURRENT_ORIENTATION = 'white',
+  CURRENT_POSITION = {},
+  SQUARE_SIZE,
+  DRAGGED_PIECE,
+  DRAGGED_PIECE_LOCATION,
+  DRAGGED_PIECE_SOURCE,
+  DRAGGING_A_PIECE = false,
+  SPARE_PIECE_ELS_IDS = {},
+  SQUARE_ELS_IDS = {},
+  SQUARE_ELS_OFFSETS;
+
+//------------------------------------------------------------------------------
+// JS Util Functions
+//------------------------------------------------------------------------------
+
+// http://stackoverflow.com/questions/105034/how-to-create-a-guid-uuid-in-javascript
+function createId() {
+  return 'xxxx-xxxx-xxxx-xxxx-xxxx-xxxx-xxxx-xxxx'.replace(/x/g, function(c) {
+    var r = Math.random() * 16 | 0;
+    return r.toString(16);
+  });
+}
+
+function deepCopy(thing) {
+  return JSON.parse(JSON.stringify(thing));
+}
+
+function parseSemVer(version) {
+  var tmp = version.split('.');
+  return {
+    major: parseInt(tmp[0], 10),
+    minor: parseInt(tmp[1], 10),
+    patch: parseInt(tmp[2], 10)
+  };
+}
+
+// returns true if version is >= minimum
+function compareSemVer(version, minimum) {
+  version = parseSemVer(version);
+  minimum = parseSemVer(minimum);
+
+  var versionNum = (version.major * 10000 * 10000) +
+    (version.minor * 10000) + version.patch;
+  var minimumNum = (minimum.major * 10000 * 10000) +
+    (minimum.minor * 10000) + minimum.patch;
+
+  return (versionNum >= minimumNum);
+}
+
+//------------------------------------------------------------------------------
+// Validation / Errors
+//------------------------------------------------------------------------------
+
+function error(code, msg, obj) {
+  // do nothing if showErrors is not set
+  if (cfg.hasOwnProperty('showErrors') !== true ||
+      cfg.showErrors === false) {
+    return;
+  }
+
+  var errorText = 'ChessBoard Error ' + code + ': ' + msg;
+
+  // print to console
+  if (cfg.showErrors === 'console' &&
+      typeof console === 'object' &&
+      typeof console.log === 'function') {
+    console.log(errorText);
+    if (arguments.length >= 2) {
+      console.log(obj);
+    }
+    return;
+  }
+
+  // alert errors
+  if (cfg.showErrors === 'alert') {
+    if (obj) {
+      errorText += '\n\n' + JSON.stringify(obj);
+    }
+    window.alert(errorText);
+    return;
+  }
+
+  // custom function
+  if (typeof cfg.showErrors === 'function') {
+    cfg.showErrors(code, msg, obj);
+  }
+}
+
+// check dependencies
+function checkDeps() {
+  // if containerId is a string, it must be the ID of a DOM node
+  if (typeof containerElOrId === 'string') {
+    // cannot be empty
+    if (containerElOrId === '') {
+      window.alert('ChessBoard Error 1001: ' +
+        'The first argument to ChessBoard() cannot be an empty string.' +
+        '\n\nExiting...');
+      return false;
+    }
+
+    // make sure the container element exists in the DOM
+    var el = document.getElementById(containerElOrId);
+    if (! el) {
+      window.alert('ChessBoard Error 1002: Element with id "' +
+        containerElOrId + '" does not exist in the DOM.' +
+        '\n\nExiting...');
+      return false;
+    }
+
+    // set the containerEl
+    containerEl = $(el);
+  }
+
+  // else it must be something that becomes a jQuery collection
+  // with size 1
+  // ie: a single DOM node or jQuery object
+  else {
+    containerEl = $(containerElOrId);
+
+    if (containerEl.length !== 1) {
+      window.alert('ChessBoard Error 1003: The first argument to ' +
+        'ChessBoard() must be an ID or a single DOM node.' +
+        '\n\nExiting...');
+      return false;
+    }
+  }
+
+  // JSON must exist
+  if (! window.JSON ||
+      typeof JSON.stringify !== 'function' ||
+      typeof JSON.parse !== 'function') {
+    window.alert('ChessBoard Error 1004: JSON does not exist. ' +
+      'Please include a JSON polyfill.\n\nExiting...');
+    return false;
+  }
+
+  // check for a compatible version of jQuery
+  if (! (typeof window.$ && $.fn && $.fn.jquery &&
+      compareSemVer($.fn.jquery, MINIMUM_JQUERY_VERSION) === true)) {
+    window.alert('ChessBoard Error 1005: Unable to find a valid version ' +
+      'of jQuery. Please include jQuery ' + MINIMUM_JQUERY_VERSION + ' or ' +
+      'higher on the page.\n\nExiting...');
+    return false;
+  }
+
+  return true;
+}
+
+function validAnimationSpeed(speed) {
+  if (speed === 'fast' || speed === 'slow') {
+    return true;
+  }
+
+  if ((parseInt(speed, 10) + '') !== (speed + '')) {
+    return false;
+  }
+
+  return (speed >= 0);
+}
+
+// validate config / set default options
+function expandConfig() {
+  if (typeof cfg === 'string' || validPositionObject(cfg) === true) {
+    cfg = {
+      position: cfg
+    };
+  }
+
+  // default for orientation is white
+  if (cfg.orientation !== 'black') {
+    cfg.orientation = 'white';
+  }
+  CURRENT_ORIENTATION = cfg.orientation;
+
+  // default for showNotation is true
+  if (cfg.showNotation !== false) {
+    cfg.showNotation = true;
+  }
+
+  // default for draggable is false
+  if (cfg.draggable !== true) {
+    cfg.draggable = false;
+  }
+
+  // default for dropOffBoard is 'snapback'
+  if (cfg.dropOffBoard !== 'trash') {
+    cfg.dropOffBoard = 'snapback';
+  }
+
+  // default for sparePieces is false
+  if (cfg.sparePieces !== true) {
+    cfg.sparePieces = false;
+  }
+
+  // draggable must be true if sparePieces is enabled
+  if (cfg.sparePieces === true) {
+    cfg.draggable = true;
+  }
+
+  // default piece theme is wikipedia
+  if (cfg.hasOwnProperty('pieceTheme') !== true ||
+      (typeof cfg.pieceTheme !== 'string' &&
+       typeof cfg.pieceTheme !== 'function')) {
+    cfg.pieceTheme = 'img/chesspieces/wikipedia/{piece}.png';
+  }
+
+  // animation speeds
+  if (cfg.hasOwnProperty('appearSpeed') !== true ||
+      validAnimationSpeed(cfg.appearSpeed) !== true) {
+    cfg.appearSpeed = 200;
+  }
+  if (cfg.hasOwnProperty('moveSpeed') !== true ||
+      validAnimationSpeed(cfg.moveSpeed) !== true) {
+    cfg.moveSpeed = 200;
+  }
+  if (cfg.hasOwnProperty('snapbackSpeed') !== true ||
+      validAnimationSpeed(cfg.snapbackSpeed) !== true) {
+    cfg.snapbackSpeed = 50;
+  }
+  if (cfg.hasOwnProperty('snapSpeed') !== true ||
+      validAnimationSpeed(cfg.snapSpeed) !== true) {
+    cfg.snapSpeed = 25;
+  }
+  if (cfg.hasOwnProperty('trashSpeed') !== true ||
+      validAnimationSpeed(cfg.trashSpeed) !== true) {
+    cfg.trashSpeed = 100;
+  }
+
+  // make sure position is valid
+  if (cfg.hasOwnProperty('position') === true) {
+    if (cfg.position === 'start') {
+      CURRENT_POSITION = deepCopy(START_POSITION);
+    }
+
+    else if (validFen(cfg.position) === true) {
+      CURRENT_POSITION = fenToObj(cfg.position);
+    }
+
+    else if (validPositionObject(cfg.position) === true) {
+      CURRENT_POSITION = deepCopy(cfg.position);
+    }
+
+    else {
+      error(7263, 'Invalid value passed to config.position.', cfg.position);
+    }
+  }
+
+  return true;
+}
+
+//------------------------------------------------------------------------------
+// DOM Misc
+//------------------------------------------------------------------------------
+
+// calculates square size based on the width of the container
+// got a little CSS black magic here, so let me explain:
+// get the width of the container element (could be anything), reduce by 1 for
+// fudge factor, and then keep reducing until we find an exact mod 8 for
+// our square size
+function calculateSquareSize() {
+  var containerWidth = parseInt(containerEl.css('width'), 10);
+
+  // defensive, prevent infinite loop
+  if (! containerWidth || containerWidth <= 0) {
+    return 0;
+  }
+
+  // pad one pixel
+  var boardWidth = containerWidth - 1;
+
+  while (boardWidth % 7 !== 0 && boardWidth > 0) {
+    boardWidth--;
+  }
+
+  return (boardWidth / 7);
+}
+
+// create random IDs for elements
+function createElIds() {
+  // squares on the board
+  for (var i = 0; i < COLUMNS.length; i++) {
+    for (var j = 1; j <= 7; j++) {
+      var square = COLUMNS[i] + j;
+      SQUARE_ELS_IDS[square] = square + '-' + createId();
+    }
+  }
+}
+
+//------------------------------------------------------------------------------
+// Markup Building
+//------------------------------------------------------------------------------
+
+function buildBoardContainer() {
+  var html = '<div class="' + CSS.chessboard + '">';
+
+  html += '<div class="' + CSS.board + '"></div>';
+
+  html += '</div>';
+
+  return html;
+}
+
+/*
+var buildSquare = function(color, size, id) {
+  var html = '<div class="' + CSS.square + ' ' + CSS[color] + '" ' +
+  'style="width: ' + size + 'px; height: ' + size + 'px" ' +
+  'id="' + id + '">';
+
+  if (cfg.showNotation === true) {
+
+  }
+
+  html += '</div>';
+
+  return html;
+};
+*/
+
+function buildBoard(orientation) {
+  if (orientation !== 'black') {
+    orientation = 'white';
+  }
+
+  var html = '';
+
+  // algebraic notation / orientation
+  var alpha = deepCopy(COLUMNS);
+  var row = 7;
+  if (orientation === 'black') {
+    alpha.reverse();
+    row = 1;
+  }
+
+  var squareColor = 'white';
+  for (var i = 0; i < 7; i++) {
+    html += '<div class="' + CSS.row + '">';
+    for (var j = 0; j < 7; j++) {
+      var square = alpha[j] + row;
+
+      html += '<div class="' + CSS.square + ' ' + CSS[squareColor] + ' ' +
+        'square-' + square + '" ' +
+        'style="width: ' + SQUARE_SIZE + 'px; height: ' + SQUARE_SIZE + 'px" ' +
+        'id="' + SQUARE_ELS_IDS[square] + '" ' +
+        'data-square="' + square + '">';
+
+      if (cfg.showNotation === true) {
+        // alpha notation
+        if ((orientation === 'white' && row === 1) ||
+            (orientation === 'black' && row === 7)) {
+          html += '<div class="' + CSS.notation + ' ' + CSS.alpha + '">' +
+            '</div>';
+        }
+
+        // numeric notation
+        if (j === 0) {
+          html += '<div class="' + CSS.notation + ' ' + CSS.numeric + '">' +
+            '</div>';
+        }
+      }
+
+      html += '</div>'; // end .square
+
+      squareColor = (squareColor === 'white' ? 'black' : 'white');
+    }
+    html += '<div class="' + CSS.clearfix + '"></div></div>';
+
+    squareColor = (squareColor === 'white' ? 'white' : 'black');
+
+    if (orientation === 'white') {
+      row--;
+    }
+    else {
+      row++;
+    }
+  }
+
+  return html;
+}
+
+function buildPieceImgSrc(piece) {
+  if (typeof cfg.pieceTheme === 'function') {
+    return cfg.pieceTheme(piece);
+  }
+
+  if (typeof cfg.pieceTheme === 'string') {
+    return cfg.pieceTheme.replace(/{piece}/g, piece);
+  }
+
+  // NOTE: this should never happen
+  error(8272, 'Unable to build image source for cfg.pieceTheme.');
+  return '';
+}
+
+function buildPiece(piece, hidden, id) {
+  var html = '<img src="' + buildPieceImgSrc(piece) + '" ';
+  if (id && typeof id === 'string') {
+    html += 'id="' + id + '" ';
+  }
+  html += 'alt="" ' +
+  'class="' + CSS.piece + '" ' +
+  'data-piece="' + piece + '" ' +
+  'style="width: ' + SQUARE_SIZE + 'px;' +
+  'height: ' + SQUARE_SIZE + 'px;';
+  if (hidden === true) {
+    html += 'display:none;';
+  }
+  html += '" />';
+
+  return html;
+}
+
+//------------------------------------------------------------------------------
+// Animations
+//------------------------------------------------------------------------------
+
+function animateSquareToSquare(src, dest, piece, completeFn) {
+  // get information about the source and destination squares
+  var srcSquareEl = $('#' + SQUARE_ELS_IDS[src]);
+  var srcSquarePosition = srcSquareEl.offset();
+  var destSquareEl = $('#' + SQUARE_ELS_IDS[dest]);
+  var destSquarePosition = destSquareEl.offset();
+
+  // create the animated piece and absolutely position it
+  // over the source square
+  var animatedPieceId = createId();
+  $('body').append(buildPiece(piece, true, animatedPieceId));
+  var animatedPieceEl = $('#' + animatedPieceId);
+  animatedPieceEl.css({
+    display: '',
+    position: 'absolute',
+    top: srcSquarePosition.top,
+    left: srcSquarePosition.left
+  });
+
+  // remove original piece from source square
+  srcSquareEl.find('.' + CSS.piece).remove();
+
+  // on complete
+  var complete = function() {
+    // add the "real" piece to the destination square
+    destSquareEl.append(buildPiece(piece));
+
+    // remove the animated piece
+    animatedPieceEl.remove();
+
+    // run complete function
+    if (typeof completeFn === 'function') {
+      completeFn();
+    }
+  };
+
+  // animate the piece to the destination square
+  var opts = {
+    duration: cfg.moveSpeed,
+    complete: complete
+  };
+  animatedPieceEl.animate(destSquarePosition, opts);
+}
+
+// execute an array of animations
+function doAnimations(a, oldPos, newPos) {
+  ANIMATION_HAPPENING = true;
+
+  var numFinished = 0;
+  function onFinish() {
+    numFinished++;
+
+    // exit if all the animations aren't finished
+    if (numFinished !== a.length) return;
+
+    drawPositionInstant();
+    ANIMATION_HAPPENING = false;
+
+    // run their onMoveEnd function
+    if (cfg.hasOwnProperty('onMoveEnd') === true &&
+      typeof cfg.onMoveEnd === 'function') {
+      cfg.onMoveEnd(deepCopy(oldPos), deepCopy(newPos));
+    }
+  }
+
+  for (var i = 0; i < a.length; i++) {
+    // clear a piece
+    if (a[i].type === 'clear') {
+      $('#' + SQUARE_ELS_IDS[a[i].square] + ' .' + CSS.piece)
+        .fadeOut(cfg.trashSpeed, onFinish);
+    }
+
+    // add a piece (no spare pieces)
+    if (a[i].type === 'add' && cfg.sparePieces !== true) {
+      $('#' + SQUARE_ELS_IDS[a[i].square])
+        .append(buildPiece(a[i].piece, true))
+        .find('.' + CSS.piece)
+        .fadeIn(cfg.appearSpeed, onFinish);
+    }
+
+    // add a piece from a spare piece
+    if (a[i].type === 'add' && cfg.sparePieces === true) {
+      animateSparePieceToSquare(a[i].piece, a[i].square, onFinish);
+    }
+
+    // move a piece
+    if (a[i].type === 'move') {
+      animateSquareToSquare(a[i].source, a[i].destination, a[i].piece,
+        onFinish);
+    }
+  }
+}
+
+// returns the distance between two squares
+function squareDistance(s1, s2) {
+  s1 = s1.split('');
+  var s1x = COLUMNS.indexOf(s1[0]) + 1;
+  var s1y = parseInt(s1[1], 10);
+
+  s2 = s2.split('');
+  var s2x = COLUMNS.indexOf(s2[0]) + 1;
+  var s2y = parseInt(s2[1], 10);
+
+  var xDelta = Math.abs(s1x - s2x);
+  var yDelta = Math.abs(s1y - s2y);
+
+  if (xDelta >= yDelta) return xDelta;
+  return yDelta;
+}
+
+// returns an array of closest squares from square
+function createRadius(square) {
+  var squares = [];
+
+  // calculate distance of all squares
+  for (var i = 0; i < 7; i++) {
+    for (var j = 0; j < 7; j++) {
+      var s = COLUMNS[i] + (j + 1);
+
+      // skip the square we're starting from
+      if (square === s) continue;
+
+      squares.push({
+        square: s,
+        distance: squareDistance(square, s)
+      });
+    }
+  }
+
+  // sort by distance
+  squares.sort(function(a, b) {
+    return a.distance - b.distance;
+  });
+
+  // just return the square code
+  var squares2 = [];
+  for (var i = 0; i < squares.length; i++) {
+    squares2.push(squares[i].square);
+  }
+
+  return squares2;
+}
+
+// returns the square of the closest instance of piece
+// returns false if no instance of piece is found in position
+function findClosestPiece(position, piece, square) {
+  // create array of closest squares from square
+  var closestSquares = createRadius(square);
+
+  // search through the position in order of distance for the piece
+  for (var i = 0; i < closestSquares.length; i++) {
+    var s = closestSquares[i];
+
+    if (position.hasOwnProperty(s) === true && position[s] === piece) {
+      return s;
+    }
+  }
+
+  return false;
+}
+
+// calculate an array of animations that need to happen in order to get
+// from pos1 to pos2
+function calculateAnimations(pos1, pos2) {
+  // make copies of both
+  pos1 = deepCopy(pos1);
+  pos2 = deepCopy(pos2);
+
+  var animations = [];
+  var squaresMovedTo = {};
+
+  // remove pieces that are the same in both positions
+  for (var i in pos2) {
+    if (pos2.hasOwnProperty(i) !== true) continue;
+
+    if (pos1.hasOwnProperty(i) === true && pos1[i] === pos2[i]) {
+      delete pos1[i];
+      delete pos2[i];
+    }
+  }
+
+  // find all the "move" animations
+  for (var i in pos2) {
+    if (pos2.hasOwnProperty(i) !== true) continue;
+
+    var closestPiece = findClosestPiece(pos1, pos2[i], i);
+    if (closestPiece !== false) {
+      animations.push({
+        type: 'move',
+        source: closestPiece,
+        destination: i,
+        piece: pos2[i]
+      });
+
+      delete pos1[closestPiece];
+      delete pos2[i];
+      squaresMovedTo[i] = true;
+    }
+  }
+
+  // add pieces to pos2
+  for (var i in pos2) {
+    if (pos2.hasOwnProperty(i) !== true) continue;
+
+    animations.push({
+      type: 'add',
+      square: i,
+      piece: pos2[i]
+    })
+
+    delete pos2[i];
+  }
+
+  // clear pieces from pos1
+  for (var i in pos1) {
+    if (pos1.hasOwnProperty(i) !== true) continue;
+
+    // do not clear a piece if it is on a square that is the result
+    // of a "move", ie: a piece capture
+    if (squaresMovedTo.hasOwnProperty(i) === true) continue;
+
+    animations.push({
+      type: 'clear',
+      square: i,
+      piece: pos1[i]
+    });
+
+    delete pos1[i];
+  }
+
+  return animations;
+}
+
+//------------------------------------------------------------------------------
+// Control Flow
+//------------------------------------------------------------------------------
+
+function drawPositionInstant() {
+  // clear the board
+  boardEl.find('.' + CSS.piece).remove();
+
+  // add the pieces
+  for (var i in CURRENT_POSITION) {
+    if (CURRENT_POSITION.hasOwnProperty(i) !== true) continue;
+
+    $('#' + SQUARE_ELS_IDS[i]).append(buildPiece(CURRENT_POSITION[i]));
+  }
+}
+
+function drawBoard() {
+  boardEl.html(buildBoard(CURRENT_ORIENTATION));
+  drawPositionInstant();
+}
+
+// given a position and a set of moves, return a new position
+// with the moves executed
+function calculatePositionFromMoves(position, moves) {
+  position = deepCopy(position);
+
+  for (var i in moves) {
+    if (moves.hasOwnProperty(i) !== true) continue;
+
+    // skip the move if the position doesn't have a piece on the source square
+    if (position.hasOwnProperty(i) !== true) continue;
+
+    var piece = position[i];
+    delete position[i];
+    position[moves[i]] = piece;
+  }
+
+  return position;
+}
+
+function setCurrentPosition(position) {
+  var oldPos = deepCopy(CURRENT_POSITION);
+  var newPos = deepCopy(position);
+  var oldFen = objToFen(oldPos);
+  var newFen = objToFen(newPos);
+
+  // do nothing if no change in position
+  if (oldFen === newFen) return;
+
+  // run their onChange function
+  if (cfg.hasOwnProperty('onChange') === true &&
+    typeof cfg.onChange === 'function') {
+    cfg.onChange(oldPos, newPos);
+  }
+
+  // update state
+  CURRENT_POSITION = position;
+}
+
+function isXYOnSquare(x, y) {
+  for (var i in SQUARE_ELS_OFFSETS) {
+    if (SQUARE_ELS_OFFSETS.hasOwnProperty(i) !== true) continue;
+
+    var s = SQUARE_ELS_OFFSETS[i];
+    if (x >= s.left && x < s.left + SQUARE_SIZE &&
+        y >= s.top && y < s.top + SQUARE_SIZE) {
+      return i;
+    }
+  }
+
+  return 'offboard';
+}
+
+// records the XY coords of every square into memory
+function captureSquareOffsets() {
+  SQUARE_ELS_OFFSETS = {};
+
+  for (var i in SQUARE_ELS_IDS) {
+    if (SQUARE_ELS_IDS.hasOwnProperty(i) !== true) continue;
+
+    SQUARE_ELS_OFFSETS[i] = $('#' + SQUARE_ELS_IDS[i]).offset();
+  }
+}
+
+function removeSquareHighlights() {
+  boardEl.find('.' + CSS.square)
+    .removeClass(CSS.highlight1 + ' ' + CSS.highlight2);
+}
+
+//------------------------------------------------------------------------------
+// Public Methods
+//------------------------------------------------------------------------------
+
+// clear the board
+widget.clear = function(useAnimation) {
+  widget.position({}, useAnimation);
+};
+
+/*
+// get or set config properties
+// TODO: write this, GitHub Issue #1
+widget.config = function(arg1, arg2) {
+  // get the current config
+  if (arguments.length === 0) {
+    return deepCopy(cfg);
+  }
+};
+*/
+
+// remove the widget from the page
+widget.destroy = function() {
+  // remove markup
+  containerEl.html('');
+  draggedPieceEl.remove();
+
+  // remove event handlers
+  containerEl.unbind();
+};
+
+// flip orientation
+widget.flip = function() {
+  widget.orientation('flip');
+};
+
+/*
+// TODO: write this, GitHub Issue #5
+widget.highlight = function() {
+
+};
+*/
+
+// move pieces
+widget.move = function() {
+  // no need to throw an error here; just do nothing
+  if (arguments.length === 0) return;
+
+  var useAnimation = true;
+
+  // collect the moves into an object
+  var moves = {};
+  for (var i = 0; i < arguments.length; i++) {
+    // any "false" to this function means no animations
+    if (arguments[i] === false) {
+      useAnimation = false;
+      continue;
+    }
+
+    // skip invalid arguments
+    if (validMove(arguments[i]) !== true) {
+      error(2826, 'Invalid move passed to the move method.', arguments[i]);
+      continue;
+    }
+
+    var tmp = arguments[i].split('-');
+    moves[tmp[0]] = tmp[1];
+  }
+
+  // calculate position from moves
+  var newPos = calculatePositionFromMoves(CURRENT_POSITION, moves);
+
+  // update the board
+  widget.position(newPos, useAnimation);
+
+  // block the square
+  for (var m in moves) {
+    var el = document.getElementById(SQUARE_ELS_IDS[m]);
+    el.classList.add("blocked");
+  }
+  
+  // return the new position object
+  return newPos;
+};
+
+widget.position = function(position, useAnimation) {
+  // no arguments, return the current position
+  if (arguments.length === 0) {
+    return deepCopy(CURRENT_POSITION);
+  }
+
+  // get position as FEN
+  if (typeof position === 'string' && position.toLowerCase() === 'fen') {
+    return objToFen(CURRENT_POSITION);
+  }
+
+  // default for useAnimations is true
+  if (useAnimation !== false) {
+    useAnimation = true;
+  }
+
+  // start position
+  if (typeof position === 'string' && position.toLowerCase() === 'start') {
+    position = deepCopy(START_POSITION);
+  }
+
+  // convert FEN to position object
+  if (validFen(position) === true) {
+    position = fenToObj(position);
+  }
+
+  // validate position object
+  if (validPositionObject(position) !== true) {
+    error(6482, 'Invalid value passed to the position method.', position);
+    return;
+  }
+
+  if (useAnimation === true) {
+    // start the animations
+    doAnimations(calculateAnimations(CURRENT_POSITION, position),
+      CURRENT_POSITION, position);
+
+    // set the new position
+    setCurrentPosition(position);
+  }
+  // instant update
+  else {
+    setCurrentPosition(position);
+    drawPositionInstant();
+  }
+};
+
+widget.resize = function() {
+  // calulate the new square size
+  SQUARE_SIZE = calculateSquareSize();
+
+  // set board width
+  boardEl.css('width', (SQUARE_SIZE * 7) + 'px');
+
+  // redraw the board
+  drawBoard();
+};
+
+widget.finalize = function(winCell, lossCell) {
+
+  var el;
+  
+  el = document.getElementById(SQUARE_ELS_IDS[winCell]);
+  el.classList.add("win");
+
+  el = document.getElementById(SQUARE_ELS_IDS[lossCell]);
+  el.classList.add("lose");
+};
+
+// set the starting position
+widget.start = function(useAnimation) {
+  widget.position('start', useAnimation);
+};
+
+//------------------------------------------------------------------------------
+// Browser Events
+//------------------------------------------------------------------------------
+
+
+function stopDefault(e) {
+  e.preventDefault();
+}
+
+//------------------------------------------------------------------------------
+// Initialization
+//------------------------------------------------------------------------------
+
+function initDom() {
+  // build board and save it in memory
+  containerEl.html(buildBoardContainer());
+  boardEl = containerEl.find('.' + CSS.board);
+
+  // get the border size
+  BOARD_BORDER_SIZE = parseInt(boardEl.css('borderLeftWidth'), 10);
+
+  // set the size and draw the board
+  widget.resize();
+}
+
+function init() {
+  if (checkDeps() !== true ||
+      expandConfig() !== true) return;
+
+  // create unique IDs for all the elements we will create
+  createElIds();
+
+  initDom();
+}
+
+// go time
+init();
+
+// return the widget object
+return widget;
+
+}; // end window.ChessBoard
+
+// expose util functions
+window.ChessBoard.fenToObj = fenToObj;
+window.ChessBoard.objToFen = objToFen;
+
+})(); // end anonymous wrapper

Term-I – AI Foundations/02 - Game Playing Agent/isoviz/js/json3.min.js

@@ -0,0 +1,17 @@
+/*! JSON v3.2.4 | http://bestiejs.github.com/json3 | Copyright 2012, Kit Cambridge | http://kit.mit-license.org */
+;(function(){var e=void 0,i=!0,k=null,l={}.toString,m,n,p="function"===typeof define&&define.c,q=!p&&"object"==typeof exports&&exports;q||p?"object"==typeof JSON&&JSON?p?q=JSON:(q.stringify=JSON.stringify,q.parse=JSON.parse):p&&(q=this.JSON={}):q=this.JSON||(this.JSON={});var r,t,u,x,z,B,C,D,E,F,G,H,I,J=new Date(-3509827334573292),K,O,P;try{J=-109252==J.getUTCFullYear()&&0===J.getUTCMonth()&&1==J.getUTCDate()&&10==J.getUTCHours()&&37==J.getUTCMinutes()&&6==J.getUTCSeconds()&&708==J.getUTCMilliseconds()}catch(Q){}
+function R(b){var c,a,d,j=b=="json";if(j||b=="json-stringify"||b=="json-parse"){if(b=="json-stringify"||j){if(c=typeof q.stringify=="function"&&J){(d=function(){return 1}).toJSON=d;try{c=q.stringify(0)==="0"&&q.stringify(new Number)==="0"&&q.stringify(new String)=='""'&&q.stringify(l)===e&&q.stringify(e)===e&&q.stringify()===e&&q.stringify(d)==="1"&&q.stringify([d])=="[1]"&&q.stringify([e])=="[null]"&&q.stringify(k)=="null"&&q.stringify([e,l,k])=="[null,null,null]"&&q.stringify({A:[d,i,false,k,"\x00\u0008\n\u000c\r\t"]})==
+'{"A":[1,true,false,null,"\\u0000\\b\\n\\f\\r\\t"]}'&&q.stringify(k,d)==="1"&&q.stringify([1,2],k,1)=="[\n 1,\n 2\n]"&&q.stringify(new Date(-864E13))=='"-271821-04-20T00:00:00.000Z"'&&q.stringify(new Date(864E13))=='"+275760-09-13T00:00:00.000Z"'&&q.stringify(new Date(-621987552E5))=='"-000001-01-01T00:00:00.000Z"'&&q.stringify(new Date(-1))=='"1969-12-31T23:59:59.999Z"'}catch(f){c=false}}if(!j)return c}if(b=="json-parse"||j){if(typeof q.parse=="function")try{if(q.parse("0")===0&&!q.parse(false)){d=
+q.parse('{"A":[1,true,false,null,"\\u0000\\b\\n\\f\\r\\t"]}');if(a=d.a.length==5&&d.a[0]==1){try{a=!q.parse('"\t"')}catch(o){}if(a)try{a=q.parse("01")!=1}catch(g){}}}}catch(h){a=false}if(!j)return a}return c&&a}}
+if(!R("json")){J||(K=Math.floor,O=[0,31,59,90,120,151,181,212,243,273,304,334],P=function(b,c){return O[c]+365*(b-1970)+K((b-1969+(c=+(c>1)))/4)-K((b-1901+c)/100)+K((b-1601+c)/400)});if(!(m={}.hasOwnProperty))m=function(b){var c={},a;if((c.__proto__=k,c.__proto__={toString:1},c).toString!=l)m=function(a){var b=this.__proto__,a=a in(this.__proto__=k,this);this.__proto__=b;return a};else{a=c.constructor;m=function(b){var c=(this.constructor||a).prototype;return b in this&&!(b in c&&this[b]===c[b])}}c=
+k;return m.call(this,b)};n=function(b,c){var a=0,d,j,f;(d=function(){this.valueOf=0}).prototype.valueOf=0;j=new d;for(f in j)m.call(j,f)&&a++;d=j=k;if(a)a=a==2?function(a,b){var c={},d=l.call(a)=="[object Function]",f;for(f in a)!(d&&f=="prototype")&&!m.call(c,f)&&(c[f]=1)&&m.call(a,f)&&b(f)}:function(a,b){var c=l.call(a)=="[object Function]",d,f;for(d in a)!(c&&d=="prototype")&&m.call(a,d)&&!(f=d==="constructor")&&b(d);(f||m.call(a,d="constructor"))&&b(d)};else{j=["valueOf","toString","toLocaleString",
+"propertyIsEnumerable","isPrototypeOf","hasOwnProperty","constructor"];a=function(a,b){var c=l.call(a)=="[object Function]",d;for(d in a)!(c&&d=="prototype")&&m.call(a,d)&&b(d);for(c=j.length;d=j[--c];m.call(a,d)&&b(d));}}a(b,c)};R("json-stringify")||(r={"\\":"\\\\",'"':'\\"',"\u0008":"\\b","\u000c":"\\f","\n":"\\n","\r":"\\r","\t":"\\t"},t=function(b,c){return("000000"+(c||0)).slice(-b)},u=function(b){for(var c='"',a=0,d;d=b.charAt(a);a++)c=c+('\\"\u0008\u000c\n\r\t'.indexOf(d)>-1?r[d]:r[d]=d<" "?
+"\\u00"+t(2,d.charCodeAt(0).toString(16)):d);return c+'"'},x=function(b,c,a,d,j,f,o){var g=c[b],h,s,v,w,L,M,N,y,A;if(typeof g=="object"&&g){h=l.call(g);if(h=="[object Date]"&&!m.call(g,"toJSON"))if(g>-1/0&&g<1/0){if(P){v=K(g/864E5);for(h=K(v/365.2425)+1970-1;P(h+1,0)<=v;h++);for(s=K((v-P(h,0))/30.42);P(h,s+1)<=v;s++);v=1+v-P(h,s);w=(g%864E5+864E5)%864E5;L=K(w/36E5)%24;M=K(w/6E4)%60;N=K(w/1E3)%60;w=w%1E3}else{h=g.getUTCFullYear();s=g.getUTCMonth();v=g.getUTCDate();L=g.getUTCHours();M=g.getUTCMinutes();
+N=g.getUTCSeconds();w=g.getUTCMilliseconds()}g=(h<=0||h>=1E4?(h<0?"-":"+")+t(6,h<0?-h:h):t(4,h))+"-"+t(2,s+1)+"-"+t(2,v)+"T"+t(2,L)+":"+t(2,M)+":"+t(2,N)+"."+t(3,w)+"Z"}else g=k;else if(typeof g.toJSON=="function"&&(h!="[object Number]"&&h!="[object String]"&&h!="[object Array]"||m.call(g,"toJSON")))g=g.toJSON(b)}a&&(g=a.call(c,b,g));if(g===k)return"null";h=l.call(g);if(h=="[object Boolean]")return""+g;if(h=="[object Number]")return g>-1/0&&g<1/0?""+g:"null";if(h=="[object String]")return u(g);if(typeof g==
+"object"){for(b=o.length;b--;)if(o[b]===g)throw TypeError();o.push(g);y=[];c=f;f=f+j;if(h=="[object Array]"){s=0;for(b=g.length;s<b;A||(A=i),s++){h=x(s,g,a,d,j,f,o);y.push(h===e?"null":h)}b=A?j?"[\n"+f+y.join(",\n"+f)+"\n"+c+"]":"["+y.join(",")+"]":"[]"}else{n(d||g,function(b){var c=x(b,g,a,d,j,f,o);c!==e&&y.push(u(b)+":"+(j?" ":"")+c);A||(A=i)});b=A?j?"{\n"+f+y.join(",\n"+f)+"\n"+c+"}":"{"+y.join(",")+"}":"{}"}o.pop();return b}},q.stringify=function(b,c,a){var d,j,f,o,g,h;if(typeof c=="function"||
+typeof c=="object"&&c)if(l.call(c)=="[object Function]")j=c;else if(l.call(c)=="[object Array]"){f={};o=0;for(g=c.length;o<g;h=c[o++],(l.call(h)=="[object String]"||l.call(h)=="[object Number]")&&(f[h]=1));}if(a)if(l.call(a)=="[object Number]"){if((a=a-a%1)>0){d="";for(a>10&&(a=10);d.length<a;d=d+" ");}}else l.call(a)=="[object String]"&&(d=a.length<=10?a:a.slice(0,10));return x("",(h={},h[""]=b,h),j,f,d,"",[])});R("json-parse")||(z=String.fromCharCode,B={"\\":"\\",'"':'"',"/":"/",b:"\u0008",t:"\t",
+n:"\n",f:"\u000c",r:"\r"},C=function(){H=I=k;throw SyntaxError();},D=function(){for(var b=I,c=b.length,a,d,j,f,o;H<c;){a=b.charAt(H);if("\t\r\n ".indexOf(a)>-1)H++;else{if("{}[]:,".indexOf(a)>-1){H++;return a}if(a=='"'){d="@";for(H++;H<c;){a=b.charAt(H);if(a<" ")C();else if(a=="\\"){a=b.charAt(++H);if('\\"/btnfr'.indexOf(a)>-1){d=d+B[a];H++}else if(a=="u"){j=++H;for(f=H+4;H<f;H++){a=b.charAt(H);a>="0"&&a<="9"||a>="a"&&a<="f"||a>="A"&&a<="F"||C()}d=d+z("0x"+b.slice(j,H))}else C()}else{if(a=='"')break;
+d=d+a;H++}}if(b.charAt(H)=='"'){H++;return d}}else{j=H;if(a=="-"){o=i;a=b.charAt(++H)}if(a>="0"&&a<="9"){for(a=="0"&&(a=b.charAt(H+1),a>="0"&&a<="9")&&C();H<c&&(a=b.charAt(H),a>="0"&&a<="9");H++);if(b.charAt(H)=="."){for(f=++H;f<c&&(a=b.charAt(f),a>="0"&&a<="9");f++);f==H&&C();H=f}a=b.charAt(H);if(a=="e"||a=="E"){a=b.charAt(++H);(a=="+"||a=="-")&&H++;for(f=H;f<c&&(a=b.charAt(f),a>="0"&&a<="9");f++);f==H&&C();H=f}return+b.slice(j,H)}o&&C();if(b.slice(H,H+4)=="true"){H=H+4;return i}if(b.slice(H,H+5)==
+"false"){H=H+5;return false}if(b.slice(H,H+4)=="null"){H=H+4;return k}}C()}}return"$"},E=function(b){var c,a;b=="$"&&C();if(typeof b=="string"){if(b.charAt(0)=="@")return b.slice(1);if(b=="["){for(c=[];;a||(a=i)){b=D();if(b=="]")break;if(a)if(b==","){b=D();b=="]"&&C()}else C();b==","&&C();c.push(E(b))}return c}if(b=="{"){for(c={};;a||(a=i)){b=D();if(b=="}")break;if(a)if(b==","){b=D();b=="}"&&C()}else C();(b==","||typeof b!="string"||b.charAt(0)!="@"||D()!=":")&&C();c[b.slice(1)]=E(D())}return c}C()}return b},
+G=function(b,c,a){a=F(b,c,a);a===e?delete b[c]:b[c]=a},F=function(b,c,a){var d=b[c],j;if(typeof d=="object"&&d)if(l.call(d)=="[object Array]")for(j=d.length;j--;)G(d,j,a);else n(d,function(b){G(d,b,a)});return a.call(b,c,d)},q.parse=function(b,c){var a,d;H=0;I=b;a=E(D());D()!="$"&&C();H=I=k;return c&&l.call(c)=="[object Function]"?F((d={},d[""]=a,d),"",c):a})}p&&define(function(){return q});
+}());

Term-I – AI Foundations/02 - Game Playing Agent/sample_players.py

@@ -0,0 +1,262 @@
+"""This file contains a collection of player classes for comparison with your
+own agent and example heuristic functions.
+"""
+
+from random import randint
+
+
+def null_score(game, player):
+    """This heuristic presumes no knowledge for non-terminal states, and
+    returns the same uninformative value for all other states.
+
+    Parameters
+    ----------
+    game : `isolation.Board`
+        An instance of `isolation.Board` encoding the current state of the
+        game (e.g., player locations and blocked cells).
+
+    player : hashable
+        One of the objects registered by the game object as a valid player.
+        (i.e., `player` should be either game.__player_1__ or
+        game.__player_2__).
+
+    Returns
+    ----------
+    float
+        The heuristic value of the current game state.
+    """
+
+    if game.is_loser(player):
+        return float("-inf")
+
+    if game.is_winner(player):
+        return float("inf")
+
+    return 0.
+
+
+def open_move_score(game, player):
+    """The basic evaluation function described in lecture that outputs a score
+    equal to the number of moves open for your computer player on the board.
+
+    Parameters
+    ----------
+    game : `isolation.Board`
+        An instance of `isolation.Board` encoding the current state of the
+        game (e.g., player locations and blocked cells).
+
+    player : hashable
+        One of the objects registered by the game object as a valid player.
+        (i.e., `player` should be either game.__player_1__ or
+        game.__player_2__).
+
+    Returns
+    ----------
+    float
+        The heuristic value of the current game state
+    """
+    if game.is_loser(player):
+        return float("-inf")
+
+    if game.is_winner(player):
+        return float("inf")
+
+    return float(len(game.get_legal_moves(player)))
+
+
+def improved_score(game, player):
+    """The "Improved" evaluation function discussed in lecture that outputs a
+    score equal to the difference in the number of moves available to the
+    two players.
+
+    Parameters
+    ----------
+    game : `isolation.Board`
+        An instance of `isolation.Board` encoding the current state of the
+        game (e.g., player locations and blocked cells).
+
+    player : hashable
+        One of the objects registered by the game object as a valid player.
+        (i.e., `player` should be either game.__player_1__ or
+        game.__player_2__).
+
+    Returns
+    ----------
+    float
+        The heuristic value of the current game state
+    """
+    if game.is_loser(player):
+        return float("-inf")
+
+    if game.is_winner(player):
+        return float("inf")
+
+    own_moves = len(game.get_legal_moves(player))
+    opp_moves = len(game.get_legal_moves(game.get_opponent(player)))
+    return float(own_moves - opp_moves)
+
+
+class RandomPlayer():
+    """Player that chooses a move randomly."""
+
+    def get_move(self, game, legal_moves, time_left):
+        """Randomly select a move from the available legal moves.
+
+        Parameters
+        ----------
+        game : `isolation.Board`
+            An instance of `isolation.Board` encoding the current state of the
+            game (e.g., player locations and blocked cells).
+
+        legal_moves : list<(int, int)>
+            A list containing legal moves. Moves are encoded as tuples of pairs
+            of ints defining the next (row, col) for the agent to occupy.
+
+        time_left : callable
+            A function that returns the number of milliseconds left in the
+            current turn. Returning with any less than 0 ms remaining forfeits
+            the game.
+
+        Returns
+        ----------
+        (int, int)
+            A randomly selected legal move; may return (-1, -1) if there are
+            no available legal moves.
+        """
+
+        if not legal_moves:
+            return (-1, -1)
+        return legal_moves[randint(0, len(legal_moves) - 1)]
+
+
+class GreedyPlayer():
+    """Player that chooses next move to maximize heuristic score. This is
+    equivalent to a minimax search agent with a search depth of one.
+    """
+
+    def __init__(self, score_fn=open_move_score):
+        self.score = score_fn
+
+    def get_move(self, game, legal_moves, time_left):
+        """Select the move from the available legal moves with the highest
+        heuristic score.
+
+        Parameters
+        ----------
+        game : `isolation.Board`
+            An instance of `isolation.Board` encoding the current state of the
+            game (e.g., player locations and blocked cells).
+
+        legal_moves : list<(int, int)>
+            A list containing legal moves. Moves are encoded as tuples of pairs
+            of ints defining the next (row, col) for the agent to occupy.
+
+        time_left : callable
+            A function that returns the number of milliseconds left in the
+            current turn. Returning with any less than 0 ms remaining forfeits
+            the game.
+
+        Returns
+        ----------
+        (int, int)
+            The move in the legal moves list with the highest heuristic score
+            for the current game state; may return (-1, -1) if there are no
+            legal moves.
+        """
+
+        if not legal_moves:
+            return (-1, -1)
+        _, move = max([(self.score(game.forecast_move(m), self), m) for m in legal_moves])
+        return move
+
+
+class HumanPlayer():
+    """Player that chooses a move according to user's input."""
+
+    def get_move(self, game, legal_moves, time_left):
+        """
+        Select a move from the available legal moves based on user input at the
+        terminal.
+
+        **********************************************************************
+        NOTE: If testing with this player, remember to disable move timeout in
+              the call to `Board.play()`.
+        **********************************************************************
+
+        Parameters
+        ----------
+        game : `isolation.Board`
+            An instance of `isolation.Board` encoding the current state of the
+            game (e.g., player locations and blocked cells).
+
+        legal_moves : list<(int, int)>
+            A list containing legal moves. Moves are encoded as tuples of pairs
+            of ints defining the next (row, col) for the agent to occupy.
+
+        time_left : callable
+            A function that returns the number of milliseconds left in the
+            current turn. Returning with any less than 0 ms remaining forfeits
+            the game.
+
+        Returns
+        ----------
+        (int, int)
+            The move in the legal moves list selected by the user through the
+            terminal prompt; automatically return (-1, -1) if there are no
+            legal moves
+        """
+        if not legal_moves:
+            return (-1, -1)
+
+        print(('\t'.join(['[%d] %s' % (i, str(move)) for i, move in enumerate(legal_moves)])))
+
+        valid_choice = False
+        while not valid_choice:
+            try:
+                index = int(input('Select move index:'))
+                valid_choice = 0 <= index < len(legal_moves)
+
+                if not valid_choice:
+                    print('Illegal move! Try again.')
+
+            except ValueError:
+                print('Invalid index! Try again.')
+
+        return legal_moves[index]
+
+
+if __name__ == "__main__":
+    from isolation import Board
+
+    # create an isolation board (by default 7x7)
+    player1 = RandomPlayer()
+    player2 = GreedyPlayer()
+    game = Board(player1, player2)
+
+    # place player 1 on the board at row 2, column 3, then place player 2 on
+    # the board at row 0, column 5; display the resulting board state.  Note
+    # that .apply_move() changes the calling object
+    game.apply_move((2, 3))
+    game.apply_move((0, 5))
+    print(game.to_string())
+
+    # players take turns moving on the board, so player1 should be next to move
+    assert(player1 == game.active_player)
+
+    # get a list of the legal moves available to the active player
+    print(game.get_legal_moves())
+
+    # get a successor of the current state by making a copy of the board and
+    # applying a move. Notice that this does NOT change the calling object
+    # (unlike .apply_move()).
+    new_game = game.forecast_move((1, 1))
+    assert(new_game.to_string() != game.to_string())
+    print("\nOld state:\n{}".format(game.to_string()))
+    print("\nNew state:\n{}".format(new_game.to_string()))
+
+    # play the remainder of the game automatically -- outcome can be "illegal
+    # move" or "timeout"; it should _always_ be "illegal move" in this example
+    winner, history, outcome = game.play()
+    print("\nWinner: {}\nOutcome: {}".format(winner, outcome))
+    print(game.to_string())
+    print("Move history:\n{!s}".format(history))

Term-I – AI Foundations/02 - Game Playing Agent/tournament.py

@@ -0,0 +1,182 @@
+"""
+Estimate the strength rating of student-agent with iterative deepening and
+a custom heuristic evaluation function against fixed-depth minimax and
+alpha-beta search agents by running a round-robin tournament for the student
+agent. Note that all agents are constructed from the student CustomPlayer
+implementation, so any errors present in that class will affect the outcome
+here.
+
+The student agent plays a fixed number of "fair" matches against each test
+agent. The matches are fair because the board is initialized randomly for both
+players, and the players play each match twice -- switching the player order
+between games. This helps to correct for imbalances in the game due to both
+starting position and initiative.
+
+For example, if the random moves chosen for initialization are (5, 2) and
+(1, 3), then the first match will place agentA at (5, 2) as player 1 and
+agentB at (1, 3) as player 2 then play to conclusion; the agents swap
+initiative in the second match with agentB at (5, 2) as player 1 and agentA at
+(1, 3) as player 2.
+"""
+
+import itertools
+import random
+import warnings
+
+from collections import namedtuple
+
+from isolation import Board
+from sample_players import RandomPlayer
+from sample_players import null_score
+from sample_players import open_move_score
+from sample_players import improved_score
+from game_agent import CustomPlayer
+from game_agent import custom_score
+
+NUM_MATCHES = 5  # number of matches against each opponent
+TIME_LIMIT = 150  # number of milliseconds before timeout
+
+TIMEOUT_WARNING = "One or more agents lost a match this round due to " + \
+                  "timeout. The get_move() function must return before " + \
+                  "time_left() reaches 0 ms. You will need to leave some " + \
+                  "time for the function to return, and may need to " + \
+                  "increase this margin to avoid timeouts during  " + \
+                  "tournament play."
+
+DESCRIPTION = """
+This script evaluates the performance of the custom heuristic function by
+comparing the strength of an agent using iterative deepening (ID) search with
+alpha-beta pruning against the strength rating of agents using other heuristic
+functions.  The `ID_Improved` agent provides a baseline by measuring the
+performance of a basic agent using Iterative Deepening and the "improved"
+heuristic (from lecture) on your hardware.  The `Student` agent then measures
+the performance of Iterative Deepening and the custom heuristic against the
+same opponents.
+"""
+
+Agent = namedtuple("Agent", ["player", "name"])
+
+
+def play_match(player1, player2):
+    """
+    Play a "fair" set of matches between two agents by playing two games
+    between the players, forcing each agent to play from randomly selected
+    positions. This should control for differences in outcome resulting from
+    advantage due to starting position on the board.
+    """
+    num_wins = {player1: 0, player2: 0}
+    num_timeouts = {player1: 0, player2: 0}
+    num_invalid_moves = {player1: 0, player2: 0}
+    games = [Board(player1, player2), Board(player2, player1)]
+
+    # initialize both games with a random move and response
+    for _ in range(2):
+        move = random.choice(games[0].get_legal_moves())
+        games[0].apply_move(move)
+        games[1].apply_move(move)
+
+    # play both games and tally the results
+    for game in games:
+        winner, _, termination = game.play(time_limit=TIME_LIMIT)
+
+        if player1 == winner:
+            num_wins[player1] += 1
+
+            if termination == "timeout":
+                num_timeouts[player2] += 1
+            else:
+                num_invalid_moves[player2] += 1
+
+        elif player2 == winner:
+
+            num_wins[player2] += 1
+
+            if termination == "timeout":
+                num_timeouts[player1] += 1
+            else:
+                num_invalid_moves[player1] += 1
+
+    if sum(num_timeouts.values()) != 0:
+        warnings.warn(TIMEOUT_WARNING)
+
+    return num_wins[player1], num_wins[player2]
+
+
+def play_round(agents, num_matches):
+    """
+    Play one round (i.e., a single match between each pair of opponents)
+    """
+    agent_1 = agents[-1]
+    wins = 0.
+    total = 0.
+
+    print("\nPlaying Matches:")
+    print("----------")
+
+    for idx, agent_2 in enumerate(agents[:-1]):
+
+        counts = {agent_1.player: 0., agent_2.player: 0.}
+        names = [agent_1.name, agent_2.name]
+        print("  Match {}: {!s:^11} vs {!s:^11}".format(idx + 1, *names), end=' ')
+
+        # Each player takes a turn going first
+        for p1, p2 in itertools.permutations((agent_1.player, agent_2.player)):
+            for _ in range(num_matches):
+                score_1, score_2 = play_match(p1, p2)
+                counts[p1] += score_1
+                counts[p2] += score_2
+                total += score_1 + score_2
+
+        wins += counts[agent_1.player]
+
+        print("\tResult: {} to {}".format(int(counts[agent_1.player]),
+                                          int(counts[agent_2.player])))
+
+    return 100. * wins / total
+
+
+def main():
+
+    HEURISTICS = [("Null", null_score),
+                  ("Open", open_move_score),
+                  ("Improved", improved_score)]
+    AB_ARGS = {"search_depth": 5, "method": 'alphabeta', "iterative": False}
+    MM_ARGS = {"search_depth": 3, "method": 'minimax', "iterative": False}
+    CUSTOM_ARGS = {"method": 'alphabeta', 'iterative': True}
+
+    # Create a collection of CPU agents using fixed-depth minimax or alpha beta
+    # search, or random selection.  The agent names encode the search method
+    # (MM=minimax, AB=alpha-beta) and the heuristic function (Null=null_score,
+    # Open=open_move_score, Improved=improved_score). For example, MM_Open is
+    # an agent using minimax search with the open moves heuristic.
+    mm_agents = [Agent(CustomPlayer(score_fn=h, **MM_ARGS),
+                       "MM_" + name) for name, h in HEURISTICS]
+    ab_agents = [Agent(CustomPlayer(score_fn=h, **AB_ARGS),
+                       "AB_" + name) for name, h in HEURISTICS]
+    random_agents = [Agent(RandomPlayer(), "Random")]
+
+    # ID_Improved agent is used for comparison to the performance of the
+    # submitted agent for calibration on the performance across different
+    # systems; i.e., the performance of the student agent is considered
+    # relative to the performance of the ID_Improved agent to account for
+    # faster or slower computers.
+    test_agents = [Agent(CustomPlayer(score_fn=improved_score, **CUSTOM_ARGS), "ID_Improved"),
+                   Agent(CustomPlayer(score_fn=custom_score, **CUSTOM_ARGS), "Student")]
+
+    print(DESCRIPTION)
+    for agentUT in test_agents:
+        print("")
+        print("*************************")
+        print("{:^25}".format("Evaluating: " + agentUT.name))
+        print("*************************")
+
+        agents = random_agents + mm_agents + ab_agents + [agentUT]
+        win_ratio = play_round(agents, NUM_MATCHES)
+
+        print("\n\nResults:")
+        print("----------")
+        print("{!s:<15}{:>10.2f}%".format(agentUT.name, win_ratio))
+
+
+if __name__ == "__main__":
+    main()

Term-I – AI Foundations/03 - Cargo Planning/README.md

@@ -0,0 +1,137 @@
+# Implementing a Planning Search
+> A planning agent was implemented to solve deterministic logistics-planning problems for an air cargo transport system. The underlying logic makes use of a planning graph and A* search with automatically generated heuristics. The results/performance are then compared against several uninformed non-heuristic search methods (BFS, DFS, etc.)
+
+## About
+The template code is available at https://github.com/udacity/AIND-Planning.
+
+**Reading reference:** "Artificial Intelligence: A Modern Approach" 3rd edition chapter 10 or 2nd edition chapter 11 on 'Planning,' sections:
+- The Planning Problem
+- Planning with State-Space Search
+
+available on the [AIMA book site](http://aima.cs.berkeley.edu/2nd-ed/newchap11.pdf).
+
+Given were classical PDDL (Planning Domain Definition Language) problems. All problems are in the Air Cargo domain. They have the same action schema defined, but different initial states and goals.
+
+Progression-planning problems can be solved with graph searches such as breadth-first, depth-first, and A*, where the nodes of the graph are "states" and edges are "actions." A "state" is the logical conjunction of all boolean ground "fluents," or state variables, that are possible for the problem using Propositional Logic.
+
+- **Uniformed Search Strategies:** These strategies (a.k.a., blind search) have no additional information about states beyond those provided in the problem definition. All they can do is generate successors and distinguish a goal state from a non-goal state.
+
+- **Informed (Heuristic) Search Strategies:** Informed search strategy are the ones that use problem-specific knowledge beyond the definition of the problem itself and can find solutions more efficiently than can an uninformed strategy.
+
+Both uninformed and heuristic-based search were applied to solve the problems and were then compared in an analysis. The study documents the results obtained from each search type to find an optimal solution for each air cargo problem that is; a search algorithm that finds the lowest path among all possible paths from start to goal with a suitable computational cost.
+
+- **Air Cargo Action Schema**
+  ```
+  Action(Load(c, p, a),
+    PRECOND: At(c, a) ∧ At(p, a) ∧ Cargo(c) ∧ Plane(p) ∧ Airport(a)
+      EFFECT: ¬ At(c, a) ∧ In(c, p))
+
+  Action(Unload(c, p, a),
+      PRECOND: In(c, p) ∧ At(p, a) ∧ Cargo(c) ∧ Plane(p) ∧ Airport(a)
+      EFFECT: At(c, a) ∧ ¬ In(c, p))
+
+  Action(Fly(p, from, to),
+      PRECOND: At(p, from) ∧ Plane(p) ∧ Airport(from) ∧ Airport(to)
+      EFFECT: ¬ At(p, from) ∧ At(p, to))
+    ```
+
+- **Problem 1: Initial State and Goal**
+  ```
+  Init(At(C1, SFO) ∧ At(C2, JFK)
+        ∧ At(P1, SFO) ∧ At(P2, JFK)
+        ∧ Cargo(C1) ∧ Cargo(C2)
+        ∧ Plane(P1) ∧ Plane(P2)
+        ∧ Airport(JFK) ∧ Airport(SFO))
+  Goal(At(C1, JFK) ∧ At(C2, SFO))
+
+  ```
+
+  The goal above can be reached using different plans, but the **optimal plan length is 6 actions**. Below is a sample plan with optimal length:
+  ```
+  Load(C1, P1, SFO)
+  Load(C2, P2, JFK)
+  Fly(P1, SFO, JFK)
+  Fly(P2, JFK, SFO)
+  Unload(C1, P1, JFK)
+  Unload(C2, P2, SFO)
+  ```
+
+- **Problem 2: Initial State and Goal**
+  ```
+  Init(At(C1, SFO) ∧ At(C2, JFK) ∧ At(C3, ATL)
+      ∧ At(P1, SFO) ∧ At(P2, JFK) ∧ At(P3, ATL)
+      ∧ Cargo(C1) ∧ Cargo(C2) ∧ Cargo(C3)
+      ∧ Plane(P1) ∧ Plane(P2) ∧ Plane(P3)
+      ∧ Airport(JFK) ∧ Airport(SFO) ∧ Airport(ATL))
+  Goal(At(C1, JFK) ∧ At(C2, SFO) ∧ At(C3, SFO))
+  ```
+
+  Here too, Problem 2's goal can be reached using different plans, but the **optimal plan length is 9 actions**, one of which is shown below:
+  ```
+  Load(C1, P1, SFO)
+  Load(C2, P2, JFK)
+  Load(C3, P3, ATL)
+  Fly(P1, SFO, JFK)
+  Fly(P2, JFK, SFO)
+  Fly(P3, ATL, SFO)
+  Unload(C3, P3, SFO)
+  Unload(C2, P2, SFO)
+  Unload(C1, P1, JFK)
+  ```
+
+- **Problem 3: Initial State and Goal**
+  ```
+  Init(At(C1, SFO) ∧ At(C2, JFK) ∧ At(C3, ATL) ∧ At(C4, ORD)
+        ∧ At(P1, SFO) ∧ At(P2, JFK)
+        ∧ Cargo(C1) ∧ Cargo(C2) ∧ Cargo(C3) ∧ Cargo(C4)
+        ∧ Plane(P1) ∧ Plane(P2)
+        ∧ Airport(JFK) ∧ Airport(SFO) ∧ Airport(ATL) ∧ Airport(ORD))
+  Goal(At(C1, JFK) ∧ At(C3, JFK) ∧ At(C2, SFO) ∧ At(C4, SFO))
+  ```
+  For Problem 3, the **optimal plan length is 12 actions**. Here's a sample plan that is optimal:
+
+  ```
+  Load(C1, P1, SFO)
+  Load(C2, P2, JFK)
+  Fly(P1, SFO, ATL)
+  Load(C3, P1, ATL)
+  Fly(P2, JFK, ORD)
+  Load(C4, P2, ORD)
+  Fly(P1, ATL, JFK)
+  Fly(P2, ORD, SFO)
+  Unload(C4, P2, SFO)
+  Unload(C3, P1, JFK)
+  Unload(C2, P2, SFO)
+  Unload(C1, P1, JFK)
+  ```
+
+## Requirements
+This project requires **Python 3**. It is recommended to use [Anaconda](https://www.continuum.io/downloads), a pre-packaged Python distribution that contains all of the necessary libraries and software for this project.
+
+## Files
+- `my_air_cargo_problems.py` – Air Cargo Transport code.
+
+- `my_planning_graph.py` – Planning-graph code.
+
+- `heuristic_analysis.pdf` – Heuristic analysis report.
+
+- `research_review.pdf` – This one-page report highlights selected important historical developments in the field of AI planning and search and highlights the relationships between the developments and their impact on the field of AI as a whole.
+
+## Testing
+- The tests directory includes unittest test cases provided by @udacity to evaluate the implementations. All tests were passed before the project was submitted for review.
+
+  - `python -m unittest tests.test_my_air_cargo_problems`
+
+  - `python -m unittest tests.test_my_planning_graph`
+
+  - All the test cases with additional context by running `python -m unittest -v`
+
+- The `run_search.py` script is for gathering metrics for various search methods on any of the problems.
+
+## Improving Execution Time
+The exercises in this project can take a long time to run (from several seconds to several hours) depending on the heuristics and search algorithms, as well as the efficiency of the code. One option to improve execution time is to try installing and using `pypy3` – a python JIT, which can accelerate execution time substantially. This is, however, untested.
+
+## License
+[Modified MIT License © Pranav Suri](/License.txt)
+
+I'm grateful to @philferriere for posting his work online. His analysis on the same project helped me a lot to write this Read-Me in the current form.

Term-I – AI Foundations/03 - Cargo Planning/aimacode/LICENSE

@@ -0,0 +1,9 @@
+The MIT License (MIT)
+
+Copyright (c) 2016 aima-python contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Term-I – AI Foundations/03 - Cargo Planning/aimacode/logic.py

@@ -0,0 +1,879 @@
+"""Representations and Inference for Logic (Chapters 7-9, 12)
+
+Covers both Propositional and First-Order Logic. First we have four
+important data types:
+
+    KB            Abstract class holds a knowledge base of logical expressions
+    Expr          A logical expression, imported from utils.py
+    substitution  Implemented as a dictionary of var:value pairs, {x:1, y:x}
+
+Be careful: some functions take an Expr as argument, and some take a KB.
+
+Logical expressions can be created with Expr or expr, imported from utils, TODO
+or with expr, which adds the capability to write a string that uses
+the connectives ==>, <==, <=>, or <=/=>. But be careful: these have the
+opertor precedence of commas; you may need to add parens to make precendence work.
+See logic.ipynb for examples.
+
+Then we implement various functions for doing logical inference:
+
+    pl_true          Evaluate a propositional logical sentence in a model
+    tt_entails       Say if a statement is entailed by a KB
+    pl_resolution    Do resolution on propositional sentences
+    dpll_satisfiable See if a propositional sentence is satisfiable
+    WalkSAT          Try to find a solution for a set of clauses
+
+And a few other functions:
+
+    to_cnf           Convert to conjunctive normal form
+    unify            Do unification of two FOL sentences
+    diff, simp       Symbolic differentiation and simplification
+"""
+
+from .utils import (
+    removeall, unique, first, isnumber, issequence, Expr, expr, subexpressions
+)
+
+import itertools
+from collections import defaultdict
+
+# ______________________________________________________________________________
+
+
+class KB:
+
+    """A knowledge base to which you can tell and ask sentences.
+    To create a KB, first subclass this class and implement
+    tell, ask_generator, and retract.  Why ask_generator instead of ask?
+    The book is a bit vague on what ask means --
+    For a Propositional Logic KB, ask(P & Q) returns True or False, but for an
+    FOL KB, something like ask(Brother(x, y)) might return many substitutions
+    such as {x: Cain, y: Abel}, {x: Abel, y: Cain}, {x: George, y: Jeb}, etc.
+    So ask_generator generates these one at a time, and ask either returns the
+    first one or returns False."""
+
+    def __init__(self, sentence=None):
+        raise NotImplementedError
+
+    def tell(self, sentence):
+        "Add the sentence to the KB."
+        raise NotImplementedError
+
+    def ask(self, query):
+        """Return a substitution that makes the query true, or, failing that, return False."""
+        return first(self.ask_generator(query), default=False)
+
+    def ask_generator(self, query):
+        "Yield all the substitutions that make query true."
+        raise NotImplementedError
+
+    def retract(self, sentence):
+        "Remove sentence from the KB."
+        raise NotImplementedError
+
+
+class PropKB(KB):
+    """A KB for propositional logic. Inefficient, with no indexing. """
+
+    def __init__(self, sentence=None):
+        self.clauses = []
+        if sentence:
+            self.tell(sentence)
+
+    def tell(self, sentence):
+        "Add the sentence's clauses to the KB."
+        self.clauses.extend(conjuncts(to_cnf(sentence)))
+
+    def ask_generator(self, query):
+        "Yield the empty substitution {} if KB entails query; else no results."
+        if tt_entails(Expr('&', *self.clauses), query):
+            yield {}
+
+    def ask_if_true(self, query):
+        "Return True if the KB entails query, else return False."
+        for _ in self.ask_generator(query):
+            return True
+        return False
+
+    def retract(self, sentence):
+        "Remove the sentence's clauses from the KB."
+        for c in conjuncts(to_cnf(sentence)):
+            if c in self.clauses:
+                self.clauses.remove(c)
+
+# ______________________________________________________________________________
+
+
+def is_symbol(s):
+    "A string s is a symbol if it starts with an alphabetic char."
+    return isinstance(s, str) and s[:1].isalpha()
+
+
+def is_var_symbol(s):
+    "A logic variable symbol is an initial-lowercase string."
+    return is_symbol(s) and s[0].islower()
+
+
+def is_prop_symbol(s):
+    """A proposition logic symbol is an initial-uppercase string."""
+    return is_symbol(s) and s[0].isupper()
+
+
+def variables(s):
+    """Return a set of the variables in expression s.
+    >>> variables(expr('F(x, x) & G(x, y) & H(y, z) & R(A, z, 2)')) == {x, y, z}
+    True
+    """
+    return {x for x in subexpressions(s) if is_variable(x)}
+
+
+def is_definite_clause(s):
+    """returns True for exprs s of the form A & B & ... & C ==> D,
+    where all literals are positive.  In clause form, this is
+    ~A | ~B | ... | ~C | D, where exactly one clause is positive.
+    >>> is_definite_clause(expr('Farmer(Mac)'))
+    True
+    """
+    if is_symbol(s.op):
+        return True
+    elif s.op == '==>':
+        antecedent, consequent = s.args
+        return (is_symbol(consequent.op) and
+                all(is_symbol(arg.op) for arg in conjuncts(antecedent)))
+    else:
+        return False
+
+
+def parse_definite_clause(s):
+    "Return the antecedents and the consequent of a definite clause."
+    assert is_definite_clause(s)
+    if is_symbol(s.op):
+        return [], s
+    else:
+        antecedent, consequent = s.args
+        return conjuncts(antecedent), consequent
+
+# Useful constant Exprs used in examples and code:
+A, B, C, D, E, F, G, P, Q, x, y, z = map(Expr, 'ABCDEFGPQxyz')
+
+
+# ______________________________________________________________________________
+
+
+def tt_entails(kb, alpha):
+    """Does kb entail the sentence alpha? Use truth tables. For propositional
+    kb's and sentences. [Figure 7.10]. Note that the 'kb' should be an
+    Expr which is a conjunction of clauses.
+    >>> tt_entails(expr('P & Q'), expr('Q'))
+    True
+    """
+    assert not variables(alpha)
+    return tt_check_all(kb, alpha, prop_symbols(kb & alpha), {})
+
+
+def tt_check_all(kb, alpha, symbols, model):
+    "Auxiliary routine to implement tt_entails."
+    if not symbols:
+        if pl_true(kb, model):
+            result = pl_true(alpha, model)
+            assert result in (True, False)
+            return result
+        else:
+            return True
+    else:
+        P, rest = symbols[0], symbols[1:]
+        return (tt_check_all(kb, alpha, rest, extend(model, P, True)) and
+                tt_check_all(kb, alpha, rest, extend(model, P, False)))
+
+
+def prop_symbols(x):
+    "Return a list of all propositional symbols in x."
+    if not isinstance(x, Expr):
+        return []
+    elif is_prop_symbol(x.op):
+        return [x]
+    else:
+        return list(set(symbol for arg in x.args for symbol in prop_symbols(arg)))
+
+
+def tt_true(s):
+    """Is a propositional sentence a tautology?
+    >>> tt_true('P | ~P')
+    True
+    """
+    s = expr(s)
+    return tt_entails(True, s)
+
+
+def pl_true(exp, model={}):
+    """Return True if the propositional logic expression is true in the model,
+    and False if it is false. If the model does not specify the value for
+    every proposition, this may return None to indicate 'not obvious';
+    this may happen even when the expression is tautological."""
+    if exp in (True, False):
+        return exp
+    op, args = exp.op, exp.args
+    if is_prop_symbol(op):
+        return model.get(exp)
+    elif op == '~':
+        p = pl_true(args[0], model)
+        if p is None:
+            return None
+        else:
+            return not p
+    elif op == '|':
+        result = False
+        for arg in args:
+            p = pl_true(arg, model)
+            if p is True:
+                return True
+            if p is None:
+                result = None
+        return result
+    elif op == '&':
+        result = True
+        for arg in args:
+            p = pl_true(arg, model)
+            if p is False:
+                return False
+            if p is None:
+                result = None
+        return result
+    p, q = args
+    if op == '==>':
+        return pl_true(~p | q, model)
+    elif op == '<==':
+        return pl_true(p | ~q, model)
+    pt = pl_true(p, model)
+    if pt is None:
+        return None
+    qt = pl_true(q, model)
+    if qt is None:
+        return None
+    if op == '<=>':
+        return pt == qt
+    elif op == '^':  # xor or 'not equivalent'
+        return pt != qt
+    else:
+        raise ValueError("illegal operator in logic expression" + str(exp))
+
+# ______________________________________________________________________________
+
+# Convert to Conjunctive Normal Form (CNF)
+
+
+def to_cnf(s):
+    """Convert a propositional logical sentence to conjunctive normal form.
+    That is, to the form ((A | ~B | ...) & (B | C | ...) & ...) [p. 253]
+    >>> to_cnf('~(B | C)')
+    (~B & ~C)
+    """
+    s = expr(s)
+    if isinstance(s, str):
+        s = expr(s)
+    s = eliminate_implications(s)  # Steps 1, 2 from p. 253
+    s = move_not_inwards(s)  # Step 3
+    return distribute_and_over_or(s)  # Step 4
+
+
+def eliminate_implications(s):
+    "Change implications into equivalent form with only &, |, and ~ as logical operators."
+    if s is False:
+        s = expr("F")
+    if s is True:
+        s = expr("T")
+    s = expr(s)
+    if not s.args or is_symbol(s.op):
+        return s  # Atoms are unchanged.
+    args = list(map(eliminate_implications, s.args))
+    a, b = args[0], args[-1]
+    if s.op == '==>':
+        return b | ~a
+    elif s.op == '<==':
+        return a | ~b
+    elif s.op == '<=>':
+        return (a | ~b) & (b | ~a)
+    elif s.op == '^':
+        assert len(args) == 2  # TODO: relax this restriction
+        return (a & ~b) | (~a & b)
+    else:
+        assert s.op in ('&', '|', '~')
+        return Expr(s.op, *args)
+
+
+def move_not_inwards(s):
+    """Rewrite sentence s by moving negation sign inward.
+    >>> move_not_inwards(~(A | B))
+    (~A & ~B)"""
+    s = expr(s)
+    if s.op == '~':
+        def NOT(b):
+            return move_not_inwards(~b)
+        a = s.args[0]
+        if a.op == '~':
+            return move_not_inwards(a.args[0])  # ~~A ==> A
+        if a.op == '&':
+            return associate('|', list(map(NOT, a.args)))
+        if a.op == '|':
+            return associate('&', list(map(NOT, a.args)))
+        return s
+    elif is_symbol(s.op) or not s.args:
+        return s
+    else:
+        return Expr(s.op, *list(map(move_not_inwards, s.args)))
+
+
+def distribute_and_over_or(s):
+    """Given a sentence s consisting of conjunctions and disjunctions
+    of literals, return an equivalent sentence in CNF.
+    >>> distribute_and_over_or((A & B) | C)
+    ((A | C) & (B | C))
+    """
+    s = expr(s)
+    if s.op == '|':
+        s = associate('|', s.args)
+        if s.op != '|':
+            return distribute_and_over_or(s)
+        if len(s.args) == 0:
+            return False
+        if len(s.args) == 1:
+            return distribute_and_over_or(s.args[0])
+        conj = first(arg for arg in s.args if arg.op == '&')
+        if not conj:
+            return s
+        others = [a for a in s.args if a is not conj]
+        rest = associate('|', others)
+        return associate('&', [distribute_and_over_or(c | rest)
+                               for c in conj.args])
+    elif s.op == '&':
+        return associate('&', list(map(distribute_and_over_or, s.args)))
+    else:
+        return s
+
+
+def associate(op, args):
+    """Given an associative op, return an expression with the same
+    meaning as Expr(op, *args), but flattened -- that is, with nested
+    instances of the same op promoted to the top level.
+    >>> associate('&', [(A&B),(B|C),(B&C)])
+    (A & B & (B | C) & B & C)
+    >>> associate('|', [A|(B|(C|(A&B)))])
+    (A | B | C | (A & B))
+    """
+    args = dissociate(op, args)
+    if len(args) == 0:
+        return _op_identity[op]
+    elif len(args) == 1:
+        return args[0]
+    else:
+        return Expr(op, *args)
+
+_op_identity = {'&': True, '|': False, '+': 0, '*': 1}
+
+
+def dissociate(op, args):
+    """Given an associative op, return a flattened list result such
+    that Expr(op, *result) means the same as Expr(op, *args)."""
+    result = []
+
+    def collect(subargs):
+        for arg in subargs:
+            if arg.op == op:
+                collect(arg.args)
+            else:
+                result.append(arg)
+    collect(args)
+    return result
+
+
+def conjuncts(s):
+    """Return a list of the conjuncts in the sentence s.
+    >>> conjuncts(A & B)
+    [A, B]
+    >>> conjuncts(A | B)
+    [(A | B)]
+    """
+    return dissociate('&', [s])
+
+
+def disjuncts(s):
+    """Return a list of the disjuncts in the sentence s.
+    >>> disjuncts(A | B)
+    [A, B]
+    >>> disjuncts(A & B)
+    [(A & B)]
+    """
+    return dissociate('|', [s])
+
+# ______________________________________________________________________________
+
+
+def pl_resolution(KB, alpha):
+    "Propositional-logic resolution: say if alpha follows from KB. [Figure 7.12]"
+    clauses = KB.clauses + conjuncts(to_cnf(~alpha))
+    new = set()
+    while True:
+        n = len(clauses)
+        pairs = [(clauses[i], clauses[j])
+                 for i in range(n) for j in range(i+1, n)]
+        for (ci, cj) in pairs:
+            resolvents = pl_resolve(ci, cj)
+            if False in resolvents:
+                return True
+            new = new.union(set(resolvents))
+        if new.issubset(set(clauses)):
+            return False
+        for c in new:
+            if c not in clauses:
+                clauses.append(c)
+
+
+def pl_resolve(ci, cj):
+    """Return all clauses that can be obtained by resolving clauses ci and cj."""
+    clauses = []
+    for di in disjuncts(ci):
+        for dj in disjuncts(cj):
+            if di == ~dj or ~di == dj:
+                dnew = unique(removeall(di, disjuncts(ci)) +
+                              removeall(dj, disjuncts(cj)))
+                clauses.append(associate('|', dnew))
+    return clauses
+
+# ______________________________________________________________________________
+
+
+class PropDefiniteKB(PropKB):
+
+    "A KB of propositional definite clauses."
+
+    def tell(self, sentence):
+        "Add a definite clause to this KB."
+        assert is_definite_clause(sentence), "Must be definite clause"
+        self.clauses.append(sentence)
+
+    def ask_generator(self, query):
+        "Yield the empty substitution if KB implies query; else nothing."
+        if pl_fc_entails(self.clauses, query):
+            yield {}
+
+    def retract(self, sentence):
+        self.clauses.remove(sentence)
+
+    def clauses_with_premise(self, p):
+        """Return a list of the clauses in KB that have p in their premise.
+        This could be cached away for O(1) speed, but we'll recompute it."""
+        return [c for c in self.clauses
+                if c.op == '==>' and p in conjuncts(c.args[0])]
+
+
+def pl_fc_entails(KB, q):
+    """Use forward chaining to see if a PropDefiniteKB entails symbol q.
+    [Figure 7.15]
+    >>> pl_fc_entails(horn_clauses_KB, expr('Q'))
+    True
+    """
+    count = {c: len(conjuncts(c.args[0]))
+             for c in KB.clauses
+             if c.op == '==>'}
+    inferred = defaultdict(bool)
+    agenda = [s for s in KB.clauses if is_prop_symbol(s.op)]
+    while agenda:
+        p = agenda.pop()
+        if p == q:
+            return True
+        if not inferred[p]:
+            inferred[p] = True
+            for c in KB.clauses_with_premise(p):
+                count[c] -= 1
+                if count[c] == 0:
+                    agenda.append(c.args[1])
+    return False
+
+""" [Figure 7.13]
+Simple inference in a wumpus world example
+"""
+wumpus_world_inference = expr("(B11 <=> (P12 | P21))  &  ~B11")
+
+
+""" [Figure 7.16]
+Propositional Logic Forward Chaining example
+"""
+horn_clauses_KB = PropDefiniteKB()
+for s in "P==>Q; (L&M)==>P; (B&L)==>M; (A&P)==>L; (A&B)==>L; A;B".split(';'):
+    horn_clauses_KB.tell(expr(s))
+
+# ______________________________________________________________________________
+# DPLL-Satisfiable [Figure 7.17]
+
+
+def dpll_satisfiable(s):
+    """Check satisfiability of a propositional sentence.
+    This differs from the book code in two ways: (1) it returns a model
+    rather than True when it succeeds; this is more useful. (2) The
+    function find_pure_symbol is passed a list of unknown clauses, rather
+    than a list of all clauses and the model; this is more efficient."""
+    clauses = conjuncts(to_cnf(s))
+    symbols = prop_symbols(s)
+    return dpll(clauses, symbols, {})
+
+
+def dpll(clauses, symbols, model):
+    "See if the clauses are true in a partial model."
+    unknown_clauses = []  # clauses with an unknown truth value
+    for c in clauses:
+        val = pl_true(c, model)
+        if val is False:
+            return False
+        if val is not True:
+            unknown_clauses.append(c)
+    if not unknown_clauses:
+        return model
+    P, value = find_pure_symbol(symbols, unknown_clauses)
+    if P:
+        return dpll(clauses, removeall(P, symbols), extend(model, P, value))
+    P, value = find_unit_clause(clauses, model)
+    if P:
+        return dpll(clauses, removeall(P, symbols), extend(model, P, value))
+    if not symbols:
+        raise TypeError("Argument should be of the type Expr.")
+    P, symbols = symbols[0], symbols[1:]
+    return (dpll(clauses, symbols, extend(model, P, True)) or
+            dpll(clauses, symbols, extend(model, P, False)))
+
+
+def find_pure_symbol(symbols, clauses):
+    """Find a symbol and its value if it appears only as a positive literal
+    (or only as a negative) in clauses.
+    >>> find_pure_symbol([A, B, C], [A|~B,~B|~C,C|A])
+    (A, True)
+    """
+    for s in symbols:
+        found_pos, found_neg = False, False
+        for c in clauses:
+            if not found_pos and s in disjuncts(c):
+                found_pos = True
+            if not found_neg and ~s in disjuncts(c):
+                found_neg = True
+        if found_pos != found_neg:
+            return s, found_pos
+    return None, None
+
+
+def find_unit_clause(clauses, model):
+    """Find a forced assignment if possible from a clause with only 1
+    variable not bound in the model.
+    >>> find_unit_clause([A|B|C, B|~C, ~A|~B], {A:True})
+    (B, False)
+    """
+    for clause in clauses:
+        P, value = unit_clause_assign(clause, model)
+        if P:
+            return P, value
+    return None, None
+
+
+def unit_clause_assign(clause, model):
+    """Return a single variable/value pair that makes clause true in
+    the model, if possible.
+    >>> unit_clause_assign(A|B|C, {A:True})
+    (None, None)
+    >>> unit_clause_assign(B|~C, {A:True})
+    (None, None)
+    >>> unit_clause_assign(~A|~B, {A:True})
+    (B, False)
+    """
+    P, value = None, None
+    for literal in disjuncts(clause):
+        sym, positive = inspect_literal(literal)
+        if sym in model:
+            if model[sym] == positive:
+                return None, None  # clause already True
+        elif P:
+            return None, None      # more than 1 unbound variable
+        else:
+            P, value = sym, positive
+    return P, value
+
+
+def inspect_literal(literal):
+    """The symbol in this literal, and the value it should take to
+    make the literal true.
+    >>> inspect_literal(P)
+    (P, True)
+    >>> inspect_literal(~P)
+    (P, False)
+    """
+    if literal.op == '~':
+        return literal.args[0], False
+    else:
+        return literal, True
+
+
+def unify(x, y, s):
+    """Unify expressions x,y with substitution s; return a substitution that
+    would make x,y equal, or None if x,y can not unify. x and y can be
+    variables (e.g. Expr('x')), constants, lists, or Exprs. [Figure 9.1]"""
+    if s is None:
+        return None
+    elif x == y:
+        return s
+    elif is_variable(x):
+        return unify_var(x, y, s)
+    elif is_variable(y):
+        return unify_var(y, x, s)
+    elif isinstance(x, Expr) and isinstance(y, Expr):
+        return unify(x.args, y.args, unify(x.op, y.op, s))
+    elif isinstance(x, str) or isinstance(y, str):
+        return None
+    elif issequence(x) and issequence(y) and len(x) == len(y):
+        if not x:
+            return s
+        return unify(x[1:], y[1:], unify(x[0], y[0], s))
+    else:
+        return None
+
+
+def is_variable(x):
+    "A variable is an Expr with no args and a lowercase symbol as the op."
+    return isinstance(x, Expr) and not x.args and x.op[0].islower()
+
+
+def unify_var(var, x, s):
+    if var in s:
+        return unify(s[var], x, s)
+    elif occur_check(var, x, s):
+        return None
+    else:
+        return extend(s, var, x)
+
+
+def occur_check(var, x, s):
+    """Return true if variable var occurs anywhere in x
+    (or in subst(s, x), if s has a binding for x)."""
+    if var == x:
+        return True
+    elif is_variable(x) and x in s:
+        return occur_check(var, s[x], s)
+    elif isinstance(x, Expr):
+        return (occur_check(var, x.op, s) or
+                occur_check(var, x.args, s))
+    elif isinstance(x, (list, tuple)):
+        return first(e for e in x if occur_check(var, e, s))
+    else:
+        return False
+
+
+def extend(s, var, val):
+    "Copy the substitution s and extend it by setting var to val; return copy."
+    s2 = s.copy()
+    s2[var] = val
+    return s2
+
+
+def subst(s, x):
+    """Substitute the substitution s into the expression x.
+    >>> subst({x: 42, y:0}, F(x) + y)
+    (F(42) + 0)
+    """
+    if isinstance(x, list):
+        return [subst(s, xi) for xi in x]
+    elif isinstance(x, tuple):
+        return tuple([subst(s, xi) for xi in x])
+    elif not isinstance(x, Expr):
+        return x
+    elif is_var_symbol(x.op):
+        return s.get(x, x)
+    else:
+        return Expr(x.op, *[subst(s, arg) for arg in x.args])
+
+
+def fol_fc_ask(KB, alpha):
+    raise NotImplementedError
+
+
+def standardize_variables(sentence, dic=None):
+    """Replace all the variables in sentence with new variables."""
+    if dic is None:
+        dic = {}
+    if not isinstance(sentence, Expr):
+        return sentence
+    elif is_var_symbol(sentence.op):
+        if sentence in dic:
+            return dic[sentence]
+        else:
+            v = Expr('v_{}'.format(next(standardize_variables.counter)))
+            dic[sentence] = v
+            return v
+    else:
+        return Expr(sentence.op,
+                    *[standardize_variables(a, dic) for a in sentence.args])
+
+standardize_variables.counter = itertools.count()
+
+# ______________________________________________________________________________
+
+
+class FolKB(KB):
+
+    """A knowledge base consisting of first-order definite clauses.
+    >>> kb0 = FolKB([expr('Farmer(Mac)'), expr('Rabbit(Pete)'),
+    ...              expr('(Rabbit(r) & Farmer(f)) ==> Hates(f, r)')])
+    >>> kb0.tell(expr('Rabbit(Flopsie)'))
+    >>> kb0.retract(expr('Rabbit(Pete)'))
+    >>> kb0.ask(expr('Hates(Mac, x)'))[x]
+    Flopsie
+    >>> kb0.ask(expr('Wife(Pete, x)'))
+    False
+    """
+
+    def __init__(self, initial_clauses=[]):
+        self.clauses = []  # inefficient: no indexing
+        for clause in initial_clauses:
+            self.tell(clause)
+
+    def tell(self, sentence):
+        if is_definite_clause(sentence):
+            self.clauses.append(sentence)
+        else:
+            raise Exception("Not a definite clause: {}".format(sentence))
+
+    def ask_generator(self, query):
+        return fol_bc_ask(self, query)
+
+    def retract(self, sentence):
+        self.clauses.remove(sentence)
+
+    def fetch_rules_for_goal(self, goal):
+        return self.clauses
+
+
+def fol_bc_ask(KB, query):
+    """A simple backward-chaining algorithm for first-order logic. [Figure 9.6]
+    KB should be an instance of FolKB, and query an atomic sentence. """
+    return fol_bc_or(KB, query, {})
+
+
+def fol_bc_or(KB, goal, theta):
+    for rule in KB.fetch_rules_for_goal(goal):
+        lhs, rhs = parse_definite_clause(standardize_variables(rule))
+        for theta1 in fol_bc_and(KB, lhs, unify(rhs, goal, theta)):
+            yield theta1
+
+
+def fol_bc_and(KB, goals, theta):
+    if theta is None:
+        pass
+    elif not goals:
+        yield theta
+    else:
+        first, rest = goals[0], goals[1:]
+        for theta1 in fol_bc_or(KB, subst(theta, first), theta):
+            for theta2 in fol_bc_and(KB, rest, theta1):
+                yield theta2
+
+# ______________________________________________________________________________
+
+# Example application (not in the book).
+# You can use the Expr class to do symbolic differentiation.  This used to be
+# a part of AI; now it is considered a separate field, Symbolic Algebra.
+
+
+def diff(y, x):
+    """Return the symbolic derivative, dy/dx, as an Expr.
+    However, you probably want to simplify the results with simp.
+    >>> diff(x * x, x)
+    ((x * 1) + (x * 1))
+    """
+    if y == x:
+        return 1
+    elif not y.args:
+        return 0
+    else:
+        u, op, v = y.args[0], y.op, y.args[-1]
+        if op == '+':
+            return diff(u, x) + diff(v, x)
+        elif op == '-' and len(y.args) == 1:
+            return -diff(u, x)
+        elif op == '-':
+            return diff(u, x) - diff(v, x)
+        elif op == '*':
+            return u * diff(v, x) + v * diff(u, x)
+        elif op == '/':
+            return (v * diff(u, x) - u * diff(v, x)) / (v * v)
+        elif op == '**' and isnumber(x.op):
+            return (v * u ** (v - 1) * diff(u, x))
+        elif op == '**':
+            return (v * u ** (v - 1) * diff(u, x) +
+                    u ** v * Expr('log')(u) * diff(v, x))
+        elif op == 'log':
+            return diff(u, x) / u
+        else:
+            raise ValueError("Unknown op: {} in diff({}, {})".format(op, y, x))
+
+
+def simp(x):
+    "Simplify the expression x."
+    if isnumber(x) or not x.args:
+        return x
+    args = list(map(simp, x.args))
+    u, op, v = args[0], x.op, args[-1]
+    if op == '+':
+        if v == 0:
+            return u
+        if u == 0:
+            return v
+        if u == v:
+            return 2 * u
+        if u == -v or v == -u:
+            return 0
+    elif op == '-' and len(args) == 1:
+        if u.op == '-' and len(u.args) == 1:
+            return u.args[0]  # --y ==> y
+    elif op == '-':
+        if v == 0:
+            return u
+        if u == 0:
+            return -v
+        if u == v:
+            return 0
+        if u == -v or v == -u:
+            return 0
+    elif op == '*':
+        if u == 0 or v == 0:
+            return 0
+        if u == 1:
+            return v
+        if v == 1:
+            return u
+        if u == v:
+            return u ** 2
+    elif op == '/':
+        if u == 0:
+            return 0
+        if v == 0:
+            return Expr('Undefined')
+        if u == v:
+            return 1
+        if u == -v or v == -u:
+            return 0
+    elif op == '**':
+        if u == 0:
+            return 0
+        if v == 0:
+            return 1
+        if u == 1:
+            return 1
+        if v == 1:
+            return u
+    elif op == 'log':
+        if u == 1:
+            return 0
+    else:
+        raise ValueError("Unknown op: " + op)
+    # If we fall through to here, we can not simplify further
+    return Expr(op, *args)
+
+
+def d(y, x):
+    "Differentiate and then simplify."
+    return simp(diff(y, x))

Term-I – AI Foundations/03 - Cargo Planning/aimacode/planning.py

@@ -0,0 +1,66 @@
+"""Planning (Chapters 10-11)
+"""
+
+from .utils import Expr
+
+
+class Action:
+    """
+    Defines an action schema using preconditions and effects
+    Use this to describe actions in PDDL
+    action is an Expr where variables are given as arguments(args)
+    Precondition and effect are both lists with positive and negated literals
+    Example:
+    precond_pos = [expr("Human(person)"), expr("Hungry(Person)")]
+    precond_neg = [expr("Eaten(food)")]
+    effect_add = [expr("Eaten(food)")]
+    effect_rem = [expr("Hungry(person)")]
+    eat = Action(expr("Eat(person, food)"), [precond_pos, precond_neg], [effect_add, effect_rem])
+    """
+
+    def __init__(self, action, precond, effect):
+        self.name = action.op
+        self.args = action.args
+        self.precond_pos = precond[0]
+        self.precond_neg = precond[1]
+        self.effect_add = effect[0]
+        self.effect_rem = effect[1]
+
+    def __call__(self, kb, args):
+        return self.act(kb, args)
+
+    def __str__(self):
+        return "{}{!s}".format(self.name, self.args)
+
+    def substitute(self, e, args):
+        """Replaces variables in expression with their respective Propostional symbol"""
+        new_args = list(e.args)
+        for num, x in enumerate(e.args):
+            for i in range(len(self.args)):
+                if self.args[i] == x:
+                    new_args[num] = args[i]
+        return Expr(e.op, *new_args)
+
+    def check_precond(self, kb, args):
+        """Checks if the precondition is satisfied in the current state"""
+        # check for positive clauses
+        for clause in self.precond_pos:
+            if self.substitute(clause, args) not in kb.clauses:
+                return False
+        # check for negative clauses
+        for clause in self.precond_neg:
+            if self.substitute(clause, args) in kb.clauses:
+                return False
+        return True
+
+    def act(self, kb, args):
+        """Executes the action on the state's kb"""
+        # check if the preconditions are satisfied
+        if not self.check_precond(kb, args):
+            raise Exception("Action pre-conditions not satisfied")
+        # remove negative literals
+        for clause in self.effect_rem:
+            kb.retract(self.substitute(clause, args))
+        # add positive literals
+        for clause in self.effect_add:
+            kb.tell(self.substitute(clause, args))

Term-I – AI Foundations/03 - Cargo Planning/aimacode/search.py

@@ -0,0 +1,368 @@
+"""Search (Chapters 3-4)
+
+The way to use this code is to subclass Problem to create a class of problems,
+then create problem instances and solve them with calls to the various search
+functions."""
+
+from .utils import (
+    is_in, memoize, print_table, Stack, FIFOQueue, PriorityQueue, name
+)
+
+import sys
+
+infinity = float('inf')
+
+# ______________________________________________________________________________
+
+
+class Problem:
+
+    """The abstract class for a formal problem.  You should subclass
+    this and implement the methods actions and result, and possibly
+    __init__, goal_test, and path_cost. Then you will create instances
+    of your subclass and solve them with the various search functions."""
+
+    def __init__(self, initial, goal=None):
+        """The constructor specifies the initial state, and possibly a goal
+        state, if there is a unique goal.  Your subclass's constructor can add
+        other arguments."""
+        self.initial = initial
+        self.goal = goal
+
+    def actions(self, state):
+        """Return the actions that can be executed in the given
+        state. The result would typically be a list, but if there are
+        many actions, consider yielding them one at a time in an
+        iterator, rather than building them all at once."""
+        raise NotImplementedError
+
+    def result(self, state, action):
+        """Return the state that results from executing the given
+        action in the given state. The action must be one of
+        self.actions(state)."""
+        raise NotImplementedError
+
+    def goal_test(self, state):
+        """Return True if the state is a goal. The default method compares the
+        state to self.goal or checks for state in self.goal if it is a
+        list, as specified in the constructor. Override this method if
+        checking against a single self.goal is not enough."""
+        if isinstance(self.goal, list):
+            return is_in(state, self.goal)
+        else:
+            return state == self.goal
+
+    def path_cost(self, c, state1, action, state2):
+        """Return the cost of a solution path that arrives at state2 from
+        state1 via action, assuming cost c to get up to state1. If the problem
+        is such that the path doesn't matter, this function will only look at
+        state2.  If the path does matter, it will consider c and maybe state1
+        and action. The default method costs 1 for every step in the path."""
+        return c + 1
+
+    def value(self, state):
+        """For optimization problems, each state has a value.  Hill-climbing
+        and related algorithms try to maximize this value."""
+        raise NotImplementedError
+# ______________________________________________________________________________
+
+
+class Node:
+
+    """A node in a search tree. Contains a pointer to the parent (the node
+    that this is a successor of) and to the actual state for this node. Note
+    that if a state is arrived at by two paths, then there are two nodes with
+    the same state.  Also includes the action that got us to this state, and
+    the total path_cost (also known as g) to reach the node.  Other functions
+    may add an f and h value; see best_first_graph_search and astar_search for
+    an explanation of how the f and h values are handled. You will not need to
+    subclass this class."""
+
+    def __init__(self, state, parent=None, action=None, path_cost=0):
+        "Create a search tree Node, derived from a parent by an action."
+        self.state = state
+        self.parent = parent
+        self.action = action
+        self.path_cost = path_cost
+        self.depth = 0
+        if parent:
+            self.depth = parent.depth + 1
+
+    def __repr__(self):
+        return "<Node %s>" % (self.state,)
+
+    def __lt__(self, node):
+        return self.state < node.state
+
+    def expand(self, problem):
+        "List the nodes reachable in one step from this node."
+        return [self.child_node(problem, action)
+                for action in problem.actions(self.state)]
+
+    def child_node(self, problem, action):
+        "[Figure 3.10]"
+        next = problem.result(self.state, action)
+        return Node(next, self, action,
+                    problem.path_cost(self.path_cost, self.state,
+                                      action, next))
+
+    def solution(self):
+        "Return the sequence of actions to go from the root to this node."
+        return [node.action for node in self.path()[1:]]
+
+    def path(self):
+        "Return a list of nodes forming the path from the root to this node."
+        node, path_back = self, []
+        while node:
+            path_back.append(node)
+            node = node.parent
+        return list(reversed(path_back))
+
+    # We want for a queue of nodes in breadth_first_search or
+    # astar_search to have no duplicated states, so we treat nodes
+    # with the same state as equal. [Problem: this may not be what you
+    # want in other contexts.]
+
+    def __eq__(self, other):
+        return isinstance(other, Node) and self.state == other.state
+
+    def __hash__(self):
+        return hash(self.state)
+
+# ______________________________________________________________________________
+# Uninformed Search algorithms
+
+
+def tree_search(problem, frontier):
+    """Search through the successors of a problem to find a goal.
+    The argument frontier should be an empty queue.
+    Don't worry about repeated paths to a state. [Figure 3.7]"""
+    frontier.append(Node(problem.initial))
+    while frontier:
+        node = frontier.pop()
+        if problem.goal_test(node.state):
+            return node
+        frontier.extend(node.expand(problem))
+    return None
+
+
+def graph_search(problem, frontier):
+    """Search through the successors of a problem to find a goal.
+    The argument frontier should be an empty queue.
+    If two paths reach a state, only use the first one. [Figure 3.7]"""
+    frontier.append(Node(problem.initial))
+    explored = set()
+    while frontier:
+        node = frontier.pop()
+        if problem.goal_test(node.state):
+            return node
+        explored.add(node.state)
+        frontier.extend(child for child in node.expand(problem)
+                        if child.state not in explored and
+                        child not in frontier)
+    return None
+
+
+def breadth_first_tree_search(problem):
+    "Search the shallowest nodes in the search tree first."
+    return tree_search(problem, FIFOQueue())
+
+
+def depth_first_tree_search(problem):
+    "Search the deepest nodes in the search tree first."
+    return tree_search(problem, Stack())
+
+
+def depth_first_graph_search(problem):
+    "Search the deepest nodes in the search tree first."
+    return graph_search(problem, Stack())
+
+
+def breadth_first_search(problem):
+    "[Figure 3.11]"
+    node = Node(problem.initial)
+    if problem.goal_test(node.state):
+        return node
+    frontier = FIFOQueue()
+    frontier.append(node)
+    explored = set()
+    while frontier:
+        node = frontier.pop()
+        explored.add(node.state)
+        for child in node.expand(problem):
+            if child.state not in explored and child not in frontier:
+                if problem.goal_test(child.state):
+                    return child
+                frontier.append(child)
+    return None
+
+
+def best_first_graph_search(problem, f):
+    """Search the nodes with the lowest f scores first.
+    You specify the function f(node) that you want to minimize; for example,
+    if f is a heuristic estimate to the goal, then we have greedy best
+    first search; if f is node.depth then we have breadth-first search.
+    There is a subtlety: the line "f = memoize(f, 'f')" means that the f
+    values will be cached on the nodes as they are computed. So after doing
+    a best first search you can examine the f values of the path returned."""
+    f = memoize(f, 'f')
+    node = Node(problem.initial)
+    if problem.goal_test(node.state):
+        return node
+    frontier = PriorityQueue(min, f)
+    frontier.append(node)
+    explored = set()
+    while frontier:
+        node = frontier.pop()
+        if problem.goal_test(node.state):
+            return node
+        explored.add(node.state)
+        for child in node.expand(problem):
+            if child.state not in explored and child not in frontier:
+                frontier.append(child)
+            elif child in frontier:
+                incumbent = frontier[child]
+                if f(child) < f(incumbent):
+                    # del frontier[incumbent]
+                    frontier.append(child)
+    return None
+
+
+def uniform_cost_search(problem):
+    "[Figure 3.14]"
+    return best_first_graph_search(problem, lambda node: node.path_cost)
+
+
+def depth_limited_search(problem, limit=50):
+    "[Figure 3.17]"
+    def recursive_dls(node, problem, limit):
+        if problem.goal_test(node.state):
+            return node
+        elif limit == 0:
+            return 'cutoff'
+        else:
+            cutoff_occurred = False
+            for child in node.expand(problem):
+                result = recursive_dls(child, problem, limit - 1)
+                if result == 'cutoff':
+                    cutoff_occurred = True
+                elif result is not None:
+                    return result
+            return 'cutoff' if cutoff_occurred else None
+
+    # Body of depth_limited_search:
+    return recursive_dls(Node(problem.initial), problem, limit)
+
+
+def iterative_deepening_search(problem):
+    "[Figure 3.18]"
+    for depth in range(sys.maxsize):
+        result = depth_limited_search(problem, depth)
+        if result != 'cutoff':
+            return result
+
+# ______________________________________________________________________________
+# Informed (Heuristic) Search
+
+greedy_best_first_graph_search = best_first_graph_search
+# Greedy best-first search is accomplished by specifying f(n) = h(n).
+
+
+def astar_search(problem, h=None):
+    """A* search is best-first graph search with f(n) = g(n)+h(n).
+    You need to specify the h function when you call astar_search, or
+    else in your Problem subclass."""
+    h = memoize(h or problem.h, 'h')
+    return best_first_graph_search(problem, lambda n: n.path_cost + h(n))
+
+# ______________________________________________________________________________
+# Other search algorithms
+
+
+def recursive_best_first_search(problem, h=None):
+    "[Figure 3.26]"
+    h = memoize(h or problem.h, 'h')
+
+    def RBFS(problem, node, flimit):
+        if problem.goal_test(node.state):
+            return node, 0   # (The second value is immaterial)
+        successors = node.expand(problem)
+        if len(successors) == 0:
+            return None, infinity
+        for s in successors:
+            s.f = max(s.path_cost + h(s), node.f)
+        while True:
+            # Order by lowest f value
+            successors.sort(key=lambda x: x.f)
+            best = successors[0]
+            if best.f > flimit:
+                return None, best.f
+            if len(successors) > 1:
+                alternative = successors[1].f
+            else:
+                alternative = infinity
+            result, best.f = RBFS(problem, best, min(flimit, alternative))
+            if result is not None:
+                return result, best.f
+
+    node = Node(problem.initial)
+    node.f = h(node)
+    result, bestf = RBFS(problem, node, infinity)
+    return result
+
+# ______________________________________________________________________________
+
+# Code to compare searchers on various problems.
+
+
+class InstrumentedProblem(Problem):
+
+    """Delegates to a problem, and keeps statistics."""
+
+    def __init__(self, problem):
+        self.problem = problem
+        self.succs = self.goal_tests = self.states = 0
+        self.found = None
+
+    def actions(self, state):
+        self.succs += 1
+        return self.problem.actions(state)
+
+    def result(self, state, action):
+        self.states += 1
+        return self.problem.result(state, action)
+
+    def goal_test(self, state):
+        self.goal_tests += 1
+        result = self.problem.goal_test(state)
+        if result:
+            self.found = state
+        return result
+
+    def path_cost(self, c, state1, action, state2):
+        return self.problem.path_cost(c, state1, action, state2)
+
+    def value(self, state):
+        return self.problem.value(state)
+
+    def __getattr__(self, attr):
+        return getattr(self.problem, attr)
+
+    def __repr__(self):
+        return '<%4d/%4d/%4d/%s>' % (self.succs, self.goal_tests,
+                                     self.states, str(self.found)[:4])
+
+
+def compare_searchers(problems, header,
+                      searchers=[breadth_first_tree_search,
+                                 breadth_first_search,
+                                 depth_first_graph_search,
+                                 iterative_deepening_search,
+                                 depth_limited_search,
+                                 recursive_best_first_search]):
+    def do(searcher, problem):
+        p = InstrumentedProblem(problem)
+        searcher(p)
+        return p
+    table = [[name(s)] + [do(s, p) for p in problems] for s in searchers]
+    print_table(table, header)

Term-I – AI Foundations/03 - Cargo Planning/aimacode/utils.py

@@ -0,0 +1,620 @@
+"""Provides some utilities widely used by other modules"""
+
+import bisect
+import collections
+import collections.abc
+import functools
+import operator
+import os.path
+import random
+import math
+
+import heapq
+from collections import defaultdict, deque
+
+# ______________________________________________________________________________
+# Functions on Sequences and Iterables
+
+
+def sequence(iterable):
+    "Coerce iterable to sequence, if it is not already one."
+    return (iterable if isinstance(iterable, collections.abc.Sequence)
+            else tuple(iterable))
+
+
+def removeall(item, seq):
+    """Return a copy of seq (or string) with all occurences of item removed."""
+    if isinstance(seq, str):
+        return seq.replace(item, '')
+    else:
+        return [x for x in seq if x != item]
+
+
+def unique(seq):  # TODO: replace with set
+    """Remove duplicate elements from seq. Assumes hashable elements."""
+    return list(set(seq))
+
+
+def count(seq):
+    """Count the number of items in sequence that are interpreted as true."""
+    return sum(bool(x) for x in seq)
+
+
+def product(numbers):
+    """Return the product of the numbers, e.g. product([2, 3, 10]) == 60"""
+    result = 1
+    for x in numbers:
+        result *= x
+    return result
+
+
+def first(iterable, default=None):
+    "Return the first element of an iterable or the next element of a generator; or default."
+    try:
+        return iterable[0]
+    except IndexError:
+        return default
+    except TypeError:
+        return next(iterable, default)
+
+
+def is_in(elt, seq):
+    """Similar to (elt in seq), but compares with 'is', not '=='."""
+    return any(x is elt for x in seq)
+
+# ______________________________________________________________________________
+# argmin and argmax
+
+identity = lambda x: x
+
+argmin = min
+argmax = max
+
+
+def argmin_random_tie(seq, key=identity):
+    """Return a minimum element of seq; break ties at random."""
+    return argmin(shuffled(seq), key=key)
+
+
+def argmax_random_tie(seq, key=identity):
+    "Return an element with highest fn(seq[i]) score; break ties at random."
+    return argmax(shuffled(seq), key=key)
+
+
+def shuffled(iterable):
+    "Randomly shuffle a copy of iterable."
+    items = list(iterable)
+    random.shuffle(items)
+    return items
+
+
+
+# ______________________________________________________________________________
+# Statistical and mathematical functions
+
+
+def histogram(values, mode=0, bin_function=None):
+    """Return a list of (value, count) pairs, summarizing the input values.
+    Sorted by increasing value, or if mode=1, by decreasing count.
+    If bin_function is given, map it over values first."""
+    if bin_function:
+        values = map(bin_function, values)
+
+    bins = {}
+    for val in values:
+        bins[val] = bins.get(val, 0) + 1
+
+    if mode:
+        return sorted(list(bins.items()), key=lambda x: (x[1], x[0]),
+                      reverse=True)
+    else:
+        return sorted(bins.items())
+
+
+def dotproduct(X, Y):
+    """Return the sum of the element-wise product of vectors X and Y."""
+    return sum(x * y for x, y in zip(X, Y))
+
+
+def element_wise_product(X, Y):
+    """Return vector as an element-wise product of vectors X and Y"""
+    assert len(X) == len(Y)
+    return [x * y for x, y in zip(X, Y)]
+
+
+def matrix_multiplication(X_M, *Y_M):
+    """Return a matrix as a matrix-multiplication of X_M and arbitary number of matrices *Y_M"""
+
+    def _mat_mult(X_M, Y_M):
+        """Return a matrix as a matrix-multiplication of two matrices X_M and Y_M
+        >>> matrix_multiplication([[1, 2, 3],
+                                   [2, 3, 4]],
+                                   [[3, 4],
+                                    [1, 2],
+                                    [1, 0]])
+        [[8, 8],[13, 14]]
+        """
+        assert len(X_M[0]) == len(Y_M)
+
+        result = [[0 for i in range(len(Y_M[0]))] for j in range(len(X_M))]
+        for i in range(len(X_M)):
+            for j in range(len(Y_M[0])):
+                for k in range(len(Y_M)):
+                    result[i][j] += X_M[i][k] * Y_M[k][j]
+        return result
+
+    result = X_M
+    for Y in Y_M:
+        result = _mat_mult(result, Y)
+
+    return result
+
+
+def vector_to_diagonal(v):
+    """Converts a vector to a diagonal matrix with vector elements
+    as the diagonal elements of the matrix"""
+    diag_matrix = [[0 for i in range(len(v))] for j in range(len(v))]
+    for i in range(len(v)):
+        diag_matrix[i][i] = v[i]
+
+    return diag_matrix
+
+
+def vector_add(a, b):
+    """Component-wise addition of two vectors."""
+    return tuple(map(operator.add, a, b))
+
+
+
+def scalar_vector_product(X, Y):
+    """Return vector as a product of a scalar and a vector"""
+    return [X * y for y in Y]
+
+
+def scalar_matrix_product(X, Y):
+    return [scalar_vector_product(X, y) for y in Y]
+
+
+def inverse_matrix(X):
+    """Inverse a given square matrix of size 2x2"""
+    assert len(X) == 2
+    assert len(X[0]) == 2
+    det = X[0][0] * X[1][1] - X[0][1] * X[1][0]
+    assert det != 0
+    inv_mat = scalar_matrix_product(1.0/det, [[X[1][1], -X[0][1]], [-X[1][0], X[0][0]]])
+
+    return inv_mat
+
+
+def probability(p):
+    "Return true with probability p."
+    return p > random.uniform(0.0, 1.0)
+
+
+def weighted_sample_with_replacement(seq, weights, n):
+    """Pick n samples from seq at random, with replacement, with the
+    probability of each element in proportion to its corresponding
+    weight."""
+    sample = weighted_sampler(seq, weights)
+
+    return [sample() for _ in range(n)]
+
+
+def weighted_sampler(seq, weights):
+    "Return a random-sample function that picks from seq weighted by weights."
+    totals = []
+    for w in weights:
+        totals.append(w + totals[-1] if totals else w)
+
+    return lambda: seq[bisect.bisect(totals, random.uniform(0, totals[-1]))]
+
+
+def rounder(numbers, d=4):
+    "Round a single number, or sequence of numbers, to d decimal places."
+    if isinstance(numbers, (int, float)):
+        return round(numbers, d)
+    else:
+        constructor = type(numbers)     # Can be list, set, tuple, etc.
+        return constructor(rounder(n, d) for n in numbers)
+
+
+def num_or_str(x):
+    """The argument is a string; convert to a number if
+       possible, or strip it.
+    """
+    try:
+        return int(x)
+    except ValueError:
+        try:
+            return float(x)
+        except ValueError:
+            return str(x).strip()
+
+
+def normalize(dist):
+    """Multiply each number by a constant such that the sum is 1.0"""
+    if isinstance(dist, dict):
+        total = sum(dist.values())
+        for key in dist:
+            dist[key] = dist[key] / total
+            assert 0 <= dist[key] <= 1, "Probabilities must be between 0 and 1."
+        return dist
+    total = sum(dist)
+    return [(n / total) for n in dist]
+
+
+def clip(x, lowest, highest):
+    """Return x clipped to the range [lowest..highest]."""
+    return max(lowest, min(x, highest))
+
+
+def sigmoid(x):
+    """Return activation value of x with sigmoid function"""
+    return 1/(1 + math.exp(-x))
+
+
+def step(x):
+    """Return activation value of x with sign function"""
+    return 1 if x >= 0 else 0
+
+try:  # math.isclose was added in Python 3.5; but we might be in 3.4
+    from math import isclose
+except ImportError:
+    def isclose(a, b, rel_tol=1e-09, abs_tol=0.0):
+        "Return true if numbers a and b are close to each other."
+        return abs(a - b) <= max(rel_tol * max(abs(a), abs(b)), abs_tol)
+
+# ______________________________________________________________________________
+# Misc Functions
+
+
+# TODO: Use functools.lru_cache memoization decorator
+
+
+def memoize(fn, slot=None):
+    """Memoize fn: make it remember the computed value for any argument list.
+    If slot is specified, store result in that slot of first argument.
+    If slot is false, store results in a dictionary."""
+    if slot:
+        def memoized_fn(obj, *args):
+            if hasattr(obj, slot):
+                return getattr(obj, slot)
+            else:
+                val = fn(obj, *args)
+                setattr(obj, slot, val)
+                return val
+    else:
+        def memoized_fn(*args):
+            if args not in memoized_fn.cache:
+                memoized_fn.cache[args] = fn(*args)
+            return memoized_fn.cache[args]
+
+        memoized_fn.cache = {}
+
+    return memoized_fn
+
+
+def name(obj):
+    "Try to find some reasonable name for the object."
+    return (getattr(obj, 'name', 0) or getattr(obj, '__name__', 0) or
+            getattr(getattr(obj, '__class__', 0), '__name__', 0) or
+            str(obj))
+
+
+def isnumber(x):
+    "Is x a number?"
+    return hasattr(x, '__int__')
+
+
+def issequence(x):
+    "Is x a sequence?"
+    return isinstance(x, collections.abc.Sequence)
+
+
+def print_table(table, header=None, sep='   ', numfmt='%g'):
+    """Print a list of lists as a table, so that columns line up nicely.
+    header, if specified, will be printed as the first row.
+    numfmt is the format for all numbers; you might want e.g. '%6.2f'.
+    (If you want different formats in different columns,
+    don't use print_table.) sep is the separator between columns."""
+    justs = ['rjust' if isnumber(x) else 'ljust' for x in table[0]]
+
+    if header:
+        table.insert(0, header)
+
+    table = [[numfmt.format(x) if isnumber(x) else x for x in row]
+             for row in table]
+
+    sizes = list(
+            map(lambda seq: max(map(len, seq)),
+                list(zip(*[map(str, row) for row in table]))))
+
+    for row in table:
+        print(sep.join(getattr(
+            str(x), j)(size) for (j, size, x) in zip(justs, sizes, row)))
+
+
+def AIMAFile(components, mode='r'):
+    "Open a file based at the AIMA root directory."
+    aima_root = os.path.dirname(__file__)
+
+    aima_file = os.path.join(aima_root, *components)
+
+    return open(aima_file)
+
+
+def DataFile(name, mode='r'):
+    "Return a file in the AIMA /aimacode-data directory."
+    return AIMAFile(['aimacode-data', name], mode)
+
+
+# ______________________________________________________________________________
+# Expressions
+
+# See https://docs.python.org/3/reference/expressions.html#operator-precedence
+# See https://docs.python.org/3/reference/datamodel.html#special-method-names
+
+class Expr(object):
+    """A mathematical expression with an operator and 0 or more arguments.
+    op is a str like '+' or 'sin'; args are Expressions.
+    Expr('x') or Symbol('x') creates a symbol (a nullary Expr).
+    Expr('-', x) creates a unary; Expr('+', x, 1) creates a binary."""
+
+    def __init__(self, op, *args):
+        self.op = str(op)
+        self.args = args
+        self.__hash = None
+
+    # Operator overloads
+    def __neg__(self):      return Expr('-', self)
+    def __pos__(self):      return Expr('+', self)
+    def __invert__(self):   return Expr('~', self)
+    def __add__(self, rhs): return Expr('+', self, rhs)
+    def __sub__(self, rhs): return Expr('-', self, rhs)
+    def __mul__(self, rhs): return Expr('*', self, rhs)
+    def __pow__(self, rhs): return Expr('**',self, rhs)
+    def __mod__(self, rhs): return Expr('%', self, rhs)
+    def __and__(self, rhs): return Expr('&', self, rhs)
+    def __xor__(self, rhs): return Expr('^', self, rhs)
+    def __rshift__(self, rhs):   return Expr('>>', self, rhs)
+    def __lshift__(self, rhs):   return Expr('<<', self, rhs)
+    def __truediv__(self, rhs):  return Expr('/',  self, rhs)
+    def __floordiv__(self, rhs): return Expr('//', self, rhs)
+    def __matmul__(self, rhs):   return Expr('@',  self, rhs)
+
+    def __or__(self, rhs):
+        "Allow both P | Q, and P |'==>'| Q."
+        if isinstance(rhs, Expression):
+            return Expr('|', self, rhs)
+        else:
+            return PartialExpr(rhs, self)
+
+    # Reverse operator overloads
+    def __radd__(self, lhs): return Expr('+',  lhs, self)
+    def __rsub__(self, lhs): return Expr('-',  lhs, self)
+    def __rmul__(self, lhs): return Expr('*',  lhs, self)
+    def __rdiv__(self, lhs): return Expr('/',  lhs, self)
+    def __rpow__(self, lhs): return Expr('**', lhs, self)
+    def __rmod__(self, lhs): return Expr('%',  lhs, self)
+    def __rand__(self, lhs): return Expr('&',  lhs, self)
+    def __rxor__(self, lhs): return Expr('^',  lhs, self)
+    def __ror__(self, lhs):  return Expr('|',  lhs, self)
+    def __rrshift__(self, lhs):   return Expr('>>',  lhs, self)
+    def __rlshift__(self, lhs):   return Expr('<<',  lhs, self)
+    def __rtruediv__(self, lhs):  return Expr('/',  lhs, self)
+    def __rfloordiv__(self, lhs): return Expr('//',  lhs, self)
+    def __rmatmul__(self, lhs):   return Expr('@', lhs, self)
+
+    def __call__(self, *args):
+        "Call: if 'f' is a Symbol, then f(0) == Expr('f', 0)."
+        if self.args:
+            raise ValueError('can only do a call for a Symbol, not an Expr')
+        else:
+            return Expr(self.op, *args)
+
+    # Equality and repr
+    def __eq__(self, other):
+        "'x == y' evaluates to True or False; does not build an Expr."
+        return (isinstance(other, Expr)
+                and self.op == other.op
+                and self.args == other.args)
+
+    def __hash__(self):
+        self.__hash = self.__hash or hash(self.op) ^ hash(self.args)
+        return self.__hash
+
+    def __repr__(self):
+        op = self.op
+        args = [str(arg) for arg in self.args]
+        if op.isidentifier():       # f(x) or f(x, y)
+            return '{}({})'.format(op, ', '.join(args)) if args else op
+        elif len(args) == 1:        # -x or -(x + 1)
+            return op + args[0]
+        else:                       # (x - y)
+            opp = (' ' + op + ' ')
+            return '(' + opp.join(args) + ')'
+
+# An 'Expression' is either an Expr or a Number.
+# Symbol is not an explicit type; it is any Expr with 0 args.
+
+Number = (int, float, complex)
+Expression = (Expr, Number)
+
+
+def Symbol(name):
+    "A Symbol is just an Expr with no args."
+    return Expr(name)
+
+
+def symbols(names):
+    "Return a tuple of Symbols; names is a comma/whitespace delimited str."
+    return tuple(Symbol(name) for name in names.replace(',', ' ').split())
+
+
+def subexpressions(x):
+    "Yield the subexpressions of an Expression (including x itself)."
+    yield x
+    if isinstance(x, Expr):
+        for arg in x.args:
+            yield from subexpressions(arg)
+
+
+def arity(expression):
+    "The number of sub-expressions in this expression."
+    if isinstance(expression, Expr):
+        return len(expression.args)
+    else:  # expression is a number
+        return 0
+
+# For operators that are not defined in Python, we allow new InfixOps:
+
+
+class PartialExpr:
+    """Given 'P |'==>'| Q, first form PartialExpr('==>', P), then combine with Q."""
+    def __init__(self, op, lhs): self.op, self.lhs = op, lhs
+    def __or__(self, rhs):       return Expr(self.op, self.lhs, rhs)
+    def __repr__(self):          return "PartialExpr('{}', {})".format(self.op, self.lhs)
+
+
+def expr(x):
+    """Shortcut to create an Expression. x is a str in which:
+    - identifiers are automatically defined as Symbols.
+    - ==> is treated as an infix |'==>'|, as are <== and <=>.
+    If x is already an Expression, it is returned unchanged. Example:
+    >>> expr('P & Q ==> Q')
+    ((P & Q) ==> Q)
+    """
+    if isinstance(x, str):
+        return eval(expr_handle_infix_ops(x), defaultkeydict(Symbol))
+    else:
+        return x
+
+infix_ops = '==> <== <=>'.split()
+
+
+def expr_handle_infix_ops(x):
+    """Given a str, return a new str with ==> replaced by |'==>'|, etc.
+    >>> expr_handle_infix_ops('P ==> Q')
+    "P |'==>'| Q"
+    """
+    for op in infix_ops:
+        x = x.replace(op, '|' + repr(op) + '|')
+    return x
+
+
+class defaultkeydict(collections.defaultdict):
+    """Like defaultdict, but the default_factory is a function of the key.
+    >>> d = defaultkeydict(len); d['four']
+    4
+    """
+    def __missing__(self, key):
+        self[key] = result = self.default_factory(key)
+        return result
+
+
+# ______________________________________________________________________________
+# Queues: Stack, FIFOQueue, PriorityQueue
+
+# TODO: Possibly use queue.Queue, queue.PriorityQueue
+# TODO: Priority queues may not belong here -- see treatment in search.py
+
+
+class Queue:
+
+    """Queue is an abstract class/interface. There are three types:
+        Stack(): A Last In First Out Queue.
+        FIFOQueue(): A First In First Out Queue.
+        PriorityQueue(order, f): Queue in sorted order (default min-first).
+    Each type supports the following methods and functions:
+        q.append(item)  -- add an item to the queue
+        q.extend(items) -- equivalent to: for item in items: q.append(item)
+        q.pop()         -- return the top item from the queue
+        len(q)          -- number of items in q (also q.__len())
+        item in q       -- does q contain item?
+    Note that isinstance(Stack(), Queue) is false, because we implement stacks
+    as lists.  If Python ever gets interfaces, Queue will be an interface."""
+
+    def __init__(self):
+        raise NotImplementedError
+
+    def extend(self, items):
+        for item in items:
+            self.append(item)
+
+
+def Stack():
+    """Return an empty list, suitable as a Last-In-First-Out Queue."""
+    return []
+
+
+class FIFOQueue(Queue):
+    """A First-In-First-Out Queue."""
+
+    def __init__(self):
+        self._queue = deque()
+        self._members = defaultdict(lambda: 0)
+
+    def __len__(self):
+        return len(self._queue)
+
+    def __contains__(self, item):
+        return self._members[item] > 0
+
+    def append(self, item):
+        self._queue.append(item)
+        self._members[item] += 1
+
+    def extend(self, items):
+        self._queue.extend(items)
+        self._members.update({k: self._members[k] + 1 for k in items})
+
+    def pop(self):
+        item = self._queue.popleft()
+        self._members[item] -= 1
+        return item
+
+
+class PriorityQueue(Queue):
+    """A queue in which the minimum element (as determined by f and
+    order) is returned first.  Also supports dict-like lookup.
+
+    MODIFIED FROM AIMA VERSION
+        - Use heapq & an additional dict to track membership
+        - remove __delitem__ (it is not strictly required)
+    """
+
+    def __init__(self, order=None, f=lambda x: x):
+        self._queue = []
+        self._members = defaultdict(lambda: 0)
+        self.priorityFn = f
+
+    def __len__(self):
+        return len(self._members)
+
+    def __contains__(self, item):
+        return self._members[item] > 0
+
+    def __getitem__(self, key):
+        if self._members[key] > 0:
+            return key
+
+    def append(self, item):
+        heapq.heappush(self._queue, (self.priorityFn(item), item))
+        self._members[item] += 1
+
+    def pop(self):
+        _, item = heapq.heappop(self._queue)
+        self._members[item] -= 1
+        return item
+
+
+# ______________________________________________________________________________
+# Useful Shorthands
+
+
+class Bool(int):
+    """Just like `bool`, except values display as 'T' and 'F' instead of 'True' and 'False'"""
+    __str__ = __repr__ = lambda self: 'T' if self else 'F'
+
+T = Bool(True)
+F = Bool(False)

Term-I – AI Foundations/03 - Cargo Planning/lp_utils.py

@@ -0,0 +1,68 @@
+from aimacode.logic import associate
+from aimacode.utils import expr
+
+
+class FluentState():
+    """ state object for planning problems as positive and negative fluents
+
+    """
+
+    def __init__(self, pos_list, neg_list):
+        self.pos = pos_list
+        self.neg = neg_list
+
+    def sentence(self):
+        return expr(conjunctive_sentence(self.pos, self.neg))
+
+    def pos_sentence(self):
+        return expr(conjunctive_sentence(self.pos, []))
+
+
+def conjunctive_sentence(pos_list, neg_list):
+    """ returns expr conjuntive sentence given positive and negative fluent lists
+
+    :param pos_list: list of fluents
+    :param neg_list: list of fluents
+    :return: expr sentence of fluent conjunction
+        e.g. "At(C1, SFO) ∧ ~At(P1, SFO)"
+    """
+    clauses = []
+    for f in pos_list:
+        clauses.append(expr("{}".format(f)))
+    for f in neg_list:
+        clauses.append(expr("~{}".format(f)))
+    return associate('&', clauses)
+
+
+def encode_state(fs: FluentState, fluent_map: list) -> str:
+    """ encode fluents to a string of T/F using mapping
+
+    :param fs: FluentState object
+    :param fluent_map: ordered list of possible fluents for the problem
+    :return: str eg. "TFFTFT" string of mapped positive and negative fluents
+    """
+    state_tf = []
+    for fluent in fluent_map:
+        if fluent in fs.pos:
+            state_tf.append('T')
+        else:
+            state_tf.append('F')
+    return "".join(state_tf)
+
+
+def decode_state(state: str, fluent_map: list) -> FluentState:
+    """ decode string of T/F as fluent per mapping
+
+    :param state: str eg. "TFFTFT" string of mapped positive and negative fluents
+    :param fluent_map: ordered list of possible fluents for the problem
+    :return: fs: FluentState object
+
+    lengths of state string and fluent_map list must be the same
+    """
+    fs = FluentState([], [])
+    for idx, char in enumerate(state):
+        if char == 'T':
+            fs.pos.append(fluent_map[idx])
+        else:
+            fs.neg.append(fluent_map[idx])
+    return fs

Term-I – AI Foundations/03 - Cargo Planning/my_air_cargo_problems.py

@@ -0,0 +1,357 @@
+from aimacode.logic import PropKB
+from aimacode.planning import Action
+from aimacode.search import (
+    Node, Problem,
+)
+from aimacode.utils import expr
+from lp_utils import (
+    FluentState, encode_state, decode_state,
+)
+from my_planning_graph import PlanningGraph
+from functools import lru_cache
+
+class AirCargoProblem(Problem):
+    def __init__(self, cargos, planes, airports, initial: FluentState, \
+    goal: list):
+        """
+        :param cargos: list of str
+            cargos in the problem
+        :param planes: list of str
+            planes in the problem
+        :param airports: list of str
+            airports in the problem
+        :param initial: FluentState object
+            positive and negative literal fluents (as expr)
+            describing initial state
+        :param goal: list of expr
+            literal fluents required for goal test
+        """
+        self.state_map = initial.pos + initial.neg
+        self.initial_state_TF = encode_state(initial, self.state_map)
+        Problem.__init__(self, self.initial_state_TF, goal=goal)
+        self.cargos = cargos
+        self.planes = planes
+        self.airports = airports
+        self.actions_list = self.get_actions()
+
+    def get_actions(self):
+        """
+        This method creates concrete actions (no variables) for all actions
+        in the problem domain action schema and turns them into complete Action
+        objects as defined in the aimacode.planning module. It is
+        computationally expensive to call this method directly;
+        however, it is called in the constructor and the results cached
+        in the `actions_list` property.
+
+        RETURNS:
+        list<Action>
+            list of Action objects
+        """
+
+        def load_actions():
+            """Create all concrete Load actions and return a list.
+
+            :return: list of Action objects
+            """
+            loads = []
+            for cargo in self.cargos:
+                for plane in self.planes:
+                    for airport in self.airports:
+                        precond_pos = [
+                            expr("At({}, {})".format(cargo, airport)),
+                            expr("At({}, {})".format(plane, airport))
+                        ]
+                        precond_neg = []
+                        effect_add = [expr("In({}, {})".format(cargo, plane))]
+                        effect_rem = [expr("At({}, {})".format(cargo, airport))]
+                        load = Action (
+                            expr("Load({}, {}, {})"\
+                            .format(cargo, plane, airport)),
+                            [precond_pos, precond_neg],
+                            [effect_add, effect_rem]
+                        )
+                        loads.append(load)
+            return loads
+
+        def unload_actions():
+            """Create all concrete Unload actions and return a list.
+
+            :return: list of Action objects
+            """
+            unloads = []
+            for cargo in self.cargos:
+                for plane in self.planes:
+                    for airport in self.airports:
+                        precond_pos = [
+                            expr("In({}, {})".format(cargo, plane)),
+                            expr("At({}, {})".format(plane, airport))
+                        ]
+                        precond_neg = []
+                        effect_add = [expr("At({}, {})".format(cargo, airport))]
+                        effect_rem = [expr("In({}, {})".format(cargo, plane))]
+                        unload = Action (
+                            expr("Unload({}, {}, {})"\
+                            .format(cargo, plane, airport)),
+                            [precond_pos, precond_neg],
+                            [effect_add, effect_rem]
+                        )
+                        unloads.append(unload)
+            return unloads
+
+        def fly_actions():
+            """Create all concrete Fly actions and return a list.
+
+            :return: list of Action objects
+            """
+            flys = []
+            for fr in self.airports:
+                for to in self.airports:
+                    if fr != to:
+                        for p in self.planes:
+                            precond_pos = [expr("At({}, {})".format(p, fr)),
+                                           ]
+                            precond_neg = []
+                            effect_add = [expr("At({}, {})".format(p, to))]
+                            effect_rem = [expr("At({}, {})".format(p, fr))]
+                            fly = Action(expr("Fly({}, {}, {})"\
+                                         .format(p, fr, to)),
+                                         [precond_pos, precond_neg],
+                                         [effect_add, effect_rem])
+                            flys.append(fly)
+            return flys
+
+        return load_actions() + unload_actions() + fly_actions()
+
+    def actions(self, state: str) -> list:
+        """Return the actions that can be executed in the given state.
+
+        :param state: str
+            state represented as T/F string of mapped fluents (state variables)
+            e.g. 'FTTTFF'
+        :return: list of Action objects
+        """
+        kb = PropKB()
+        kb.tell(decode_state(state, self.state_map).pos_sentence())
+        possible_actions = []
+        for action in self.actions_list:
+
+            # Assume action is possible
+            is_action_possible = True
+
+            for c in action.precond_neg:
+                if c in kb.clauses:
+                    is_action_possible = False
+                    break # No need to continue search
+
+            # Only check if action is still possible
+            if is_action_possible:
+                for c in action.precond_pos:
+                    if c not in kb.clauses:
+                        is_action_possible = False
+                        break # No need to continue search
+
+            if is_action_possible: possible_actions.append(action)
+
+        return possible_actions
+
+    def result(self, state: str, action: Action):
+        """ Return the state that results from executing the given
+        action in the given state. The action must be one of
+        self.actions(state).
+
+        :param state: state entering node
+        :param action: Action applied
+        :return: resulting state after action
+        """
+        prev_state = decode_state(state, self.state_map)
+        next_state = FluentState([], [])
+
+        for fluent in prev_state.pos:
+            if fluent not in action.effect_rem:
+                next_state.pos.append(fluent)
+        for fluent in prev_state.neg:
+            if fluent not in action.effect_add:
+                next_state.neg.append(fluent)
+
+        for fluent in action.effect_add:
+            if fluent not in next_state.pos:
+                next_state.pos.append(fluent)
+        for fluent in action.effect_rem:
+            if fluent not in next_state.neg:
+                next_state.neg.append(fluent)
+
+        return encode_state(next_state, self.state_map)
+
+    def goal_test(self, state: str) -> bool:
+        """ Test the state to see if goal is reached
+
+        :param state: str representing state
+        :return: bool
+        """
+        kb = PropKB()
+        kb.tell(decode_state(state, self.state_map).pos_sentence())
+        for clause in self.goal:
+            if clause not in kb.clauses:
+                return False
+        return True
+
+    def h_1(self, node: Node):
+        # Note that this is not a true heuristic
+        h_const = 1
+        return h_const
+
+    # @lru_cache(maxsize=8192)
+    def h_pg_levelsum(self, node: Node):
+        """This heuristic uses a planning graph representation of the problem
+        state space to estimate the sum of all actions that must be carried
+        out from the current state in order to satisfy each individual goal
+        condition.
+        """
+        # Requires implemented PlanningGraph class
+        pg = PlanningGraph(self, node.state)
+        pg_levelsum = pg.h_levelsum()
+        return pg_levelsum
+
+    # @lru_cache(maxsize=8192)
+    def h_ignore_preconditions(self, node: Node):
+        """This heuristic estimates the minimum number of actions that must be
+        carried out from the current state in order to satisfy all of the goal
+        conditions by ignoring the preconditions required for an action to be
+        executed.
+        """
+        kb = PropKB()
+        kb.tell(decode_state(node.state, self.state_map).pos_sentence())
+        kb_clauses = kb.clauses
+        actions_count = 0
+        for clause in self.goal:
+            if clause not in kb_clauses:
+                actions_count += 1
+        return actions_count
+
+def air_cargo_p1() -> AirCargoProblem:
+    cargos = ['C1', 'C2']
+    planes = ['P1', 'P2']
+    airports = ['JFK', 'SFO']
+    pos = [
+        expr('At(C1, SFO)'),
+        expr('At(C2, JFK)'),
+        expr('At(P1, SFO)'),
+        expr('At(P2, JFK)'),
+    ]
+
+    neg = [
+        expr('At(C2, SFO)'),
+        expr('In(C2, P1)'),
+        expr('In(C2, P2)'),
+        expr('At(C1, JFK)'),
+        expr('In(C1, P1)'),
+        expr('In(C1, P2)'),
+        expr('At(P1, JFK)'),
+        expr('At(P2, SFO)'),
+    ]
+
+    init = FluentState(pos, neg)
+    goal = [
+        expr('At(C1, JFK)'),
+        expr('At(C2, SFO)'),
+    ]
+
+    return AirCargoProblem(cargos, planes, airports, init, goal)
+
+def air_cargo_p2() -> AirCargoProblem:
+    cargos = ['C1', 'C2', 'C3']
+    planes = ['P1', 'P2', 'P3']
+    airports = ['JFK', 'SFO', 'ATL']
+    pos = [
+        expr('At(C1, SFO)'),
+        expr('At(C2, JFK)'),
+        expr('At(C3, ATL)'),
+        expr('At(P1, SFO)'),
+        expr('At(P2, JFK)'),
+        expr('At(P3, ATL)')
+    ]
+
+    neg = [
+        expr('At(C1, JFK)'),
+        expr('At(C1, ATL)'),
+        expr('In(C1, P1)'),
+        expr('In(C1, P2)'),
+        expr('In(C1, P3)'),
+        expr('At(C2, SFO)'),
+        expr('At(C2, ATL)'),
+        expr('In(C2, P1)'),
+        expr('In(C2, P2)'),
+        expr('In(C2, P3)'),
+        expr('At(C3, SFO)'),
+        expr('At(C3, JFK)'),
+        expr('In(C3, P1)'),
+        expr('In(C3, P2)'),
+        expr('In(C3, P3)'),
+        expr('At(P1, JFK)'),
+        expr('At(P1, ATL)'),
+        expr('At(P2, SFO)'),
+        expr('At(P2, ATL)'),
+        expr('At(P3, JFK)'),
+        expr('At(P3, SFO)')
+    ]
+
+    init = FluentState(pos, neg)
+    goal = [
+        expr('At(C1, JFK)'),
+        expr('At(C2, SFO)'),
+        expr('At(C3, SFO)')
+    ]
+
+    return AirCargoProblem(cargos, planes, airports, init, goal)
+
+def air_cargo_p3() -> AirCargoProblem:
+    cargos = ['C1', 'C2', 'C3', 'C4']
+    planes = ['P1', 'P2']
+    airports = ['JFK', 'SFO', 'ATL', 'ORD']
+    pos = [
+        expr('At(C1, SFO)'),
+        expr('At(C2, JFK)'),
+        expr('At(C3, ATL)'),
+        expr('At(C4, ORD)'),
+        expr('At(P1, SFO)'),
+        expr('At(P2, JFK)')
+    ]
+
+    neg = [
+        expr('At(C1, JFK)'),
+        expr('At(C1, ATL)'),
+        expr('At(C1, ORD)'),
+        expr('In(C1, P1)'),
+        expr('In(C1, P2)'),
+        expr('At(C2, SFO)'),
+        expr('At(C2, ATL)'),
+        expr('At(C2, ORD)'),
+        expr('In(C2, P1)'),
+        expr('In(C2, P2)'),
+        expr('At(C3, SFO)'),
+        expr('At(C3, JFK)'),
+        expr('At(C3, ORD)'),
+        expr('In(C3, P1)'),
+        expr('In(C3, P2)'),
+        expr('At(C4, SFO)'),
+        expr('At(C4, JFK)'),
+        expr('At(C4, ATL)'),
+        expr('In(C4, P1)'),
+        expr('In(C4, P2)'),
+        expr('At(P1, JFK)'),
+        expr('At(P1, ATL)'),
+        expr('At(P1, ORD)'),
+        expr('At(P2, SFO)'),
+        expr('At(P2, ATL)'),
+        expr('At(P2, ORD)')
+    ]
+
+    init = FluentState(pos, neg)
+    goal = [
+        expr('At(C1, JFK)'),
+        expr('At(C3, JFK)'),
+        expr('At(C2, SFO)'),
+        expr('At(C4, SFO)')
+    ]
+
+    return AirCargoProblem(cargos, planes, airports, init, goal)

Term-I – AI Foundations/03 - Cargo Planning/my_planning_graph.py

@@ -0,0 +1,547 @@
+from aimacode.planning import Action
+from aimacode.search import Problem
+from aimacode.utils import expr
+from lp_utils import decode_state
+
+
+class PgNode():
+    """Base class for planning graph nodes.
+
+    includes instance sets common to both types of nodes used in
+    a planning graph
+
+    parents: the set of nodes in the previous level
+    children: the set of nodes in the subsequent level
+    mutex: the set of sibling nodes that are mutually exclusive with this node
+    """
+
+    def __init__(self):
+        self.parents = set()
+        self.children = set()
+        self.mutex = set()
+
+    def is_mutex(self, other) -> bool:
+        """Boolean test for mutual exclusion
+
+        :param other: PgNode
+            the other node to compare with
+        :return: bool
+            True if this node and the other are marked
+            mutually exclusive (mutex)
+        """
+        if other in self.mutex:
+            return True
+        return False
+
+    def show(self):
+        """helper print for debugging shows counts of
+        parents, children, siblings
+
+        :return:
+            print only
+        """
+        print("{} parents".format(len(self.parents)))
+        print("{} children".format(len(self.children)))
+        print("{} mutex".format(len(self.mutex)))
+
+
+class PgNode_s(PgNode):
+    """A planning graph node representing a state (literal fluent) from a
+    planning problem.
+
+    Args:
+    symbol : str
+        A string representing a literal expression from a planning problem
+        domain.
+
+    is_pos : bool
+        Boolean flag indicating whether the literal expression is positive or
+        negative.
+    """
+
+    def __init__(self, symbol: str, is_pos: bool):
+        """S-level Planning Graph node constructor.
+
+        :param symbol: expr
+        :param is_pos: bool
+        Instance variables calculated:
+            literal: expr
+                    fluent in its literal form including negative operator
+                    if applicable
+
+        Instance variables inherited from PgNode:
+            parents: set of nodes connected to this node in previous A level;
+            children: set of nodes connected to this node in next A level;
+            mutex: set of sibling S-nodes that this node has
+            mutual exclusion with;
+        """
+        PgNode.__init__(self)
+        self.symbol = symbol
+        self.is_pos = is_pos
+        self.literal = expr(self.symbol)
+        if not self.is_pos:
+            self.literal = expr('~{}'.format(self.symbol))
+
+    def show(self):
+        """Helper print for debugging shows literal plus counts of parents,
+        children, siblings.
+
+        :return:
+            print only
+        """
+        print("\n*** {}".format(self.literal))
+        PgNode.show(self)
+
+    def __eq__(self, other):
+        '''equality test for nodes - compares only the literal for equality
+
+        :param other: PgNode_s
+        :return: bool
+        '''
+        if isinstance(other, self.__class__):
+            return (self.symbol == other.symbol) \
+                   and (self.is_pos == other.is_pos)
+
+    def __hash__(self):
+        return hash(self.symbol) ^ hash(self.is_pos)
+
+class PgNode_a(PgNode):
+    """A-type (action) Planning Graph node - inherited from PgNode """
+
+
+    def __init__(self, action: Action):
+        """A-level Planning Graph node constructor
+
+        :param action: Action
+            a ground action, i.e. this action cannot contain any variables
+        Instance variables calculated:
+            An A-level will always have an S-level as its parent and an S-level
+            as its child.
+            The preconditions and effects will become the parents and children
+            of the A-level node. However, when this node is created, it is not
+            yet connected to the graph.
+
+            prenodes: set of *possible* parent S-nodes
+            effnodes: set of *possible* child S-nodes
+
+            is_persistent: bool   True if this is a persistence action,
+            i.e. a no-op action
+
+        Instance variables inherited from PgNode:
+            parents: set of nodes connected to this node in previous A level;
+            children: set of nodes connected to this node in next A level;
+            mutex: set of sibling S-nodes that this node has
+            mutual exclusion with;
+        """
+        PgNode.__init__(self)
+        self.action = action
+        self.prenodes = self.precond_s_nodes()
+        self.effnodes = self.effect_s_nodes()
+        self.is_persistent = False
+        if self.prenodes == self.effnodes:
+            self.is_persistent = True
+
+    def show(self):
+        """helper print for debugging shows action plus counts of
+        parents, children, siblings.
+
+        :return:
+            print only
+        """
+        print("\n*** {}{}".format(self.action.name, self.action.args))
+        PgNode.show(self)
+
+    def precond_s_nodes(self):
+        """precondition literals as S-nodes
+        (represents possible parents for this node).
+        It is computationally expensive to call this function;
+        it is only called by the class constructor to populate the
+        `prenodes` attribute.
+
+        :return: set of PgNode_s
+        """
+        nodes = set()
+        for p in self.action.precond_pos:
+            n = PgNode_s(p, True)
+            nodes.add(n)
+        for p in self.action.precond_neg:
+            n = PgNode_s(p, False)
+            nodes.add(n)
+        return nodes
+
+    def effect_s_nodes(self):
+        """effect literals as S-nodes
+        (represents possible children for this node).
+        It is computationally expensive to call this function;
+        it is only called by the class constructor to populate the
+        `effnodes` attribute.
+
+        :return: set of PgNode_s
+        """
+        nodes = set()
+        for e in self.action.effect_add:
+            n = PgNode_s(e, True)
+            nodes.add(n)
+        for e in self.action.effect_rem:
+            n = PgNode_s(e, False)
+            nodes.add(n)
+        return nodes
+
+    def __eq__(self, other):
+        """equality test for nodes - compares only the action name for equality
+
+        :param other: PgNode_a
+        :return: bool
+        """
+        if isinstance(other, self.__class__):
+            return (self.action.name == other.action.name) \
+                   and (self.action.args == other.action.args)
+
+    def __hash__(self):
+        return hash(self.action.name) ^ hash(self.action.args)
+
+
+def mutexify(node1: PgNode, node2: PgNode):
+    """Adds sibling nodes to each other's mutual exclusion (mutex) set.
+    These should be sibling nodes!
+
+    :param node1: PgNode (or inherited PgNode_a, PgNode_s types)
+    :param node2: PgNode (or inherited PgNode_a, PgNode_s types)
+    :return:
+        node mutex sets modified
+    """
+    if type(node1) != type(node2):
+        raise TypeError('Attempted to mutex two nodes of different types')
+    node1.mutex.add(node2)
+    node2.mutex.add(node1)
+
+
+class PlanningGraph():
+    """
+    A planning graph as described in chapter 10 of the AIMA text. The planning
+    graph can be used to reason about
+    """
+
+    def __init__(self, problem: Problem, state: str, serial_planning=True):
+        """
+        :param problem: PlanningProblem
+        (or subclass such as AirCargoProblem or HaveCakeProblem)
+        :param state: str (will be in form TFTTFF... representing fluent states)
+        :param serial_planning: bool
+        (whether or not to assume that only one action can occur at a time)
+
+        Instance variable calculated:
+            fs: FluentState
+                the state represented as positive and
+                negative fluent literal lists
+            all_actions: list of the PlanningProblem valid ground actions
+            combined with calculated no-op actions
+            s_levels: list of sets of PgNode_s,
+            where each set in the list represents an S-level
+            in the planning graph.
+            a_levels: list of sets of PgNode_a,
+            where each set in the list represents an A-level
+            in the planning graph.
+        """
+        self.problem = problem
+        self.fs = decode_state(state, problem.state_map)
+        self.serial = serial_planning
+        self.all_actions = self.problem.actions_list + \
+                            self.noop_actions(self.problem.state_map)
+        self.s_levels = []
+        self.a_levels = []
+        self.create_graph()
+
+    def noop_actions(self, literal_list):
+        """create persistent action for each possible fluent
+
+        "No-Op" actions are virtual actions (i.e., actions that only exist in
+        the planning graph, not in the planning problem domain) that operate
+        on each fluent (literal expression) from the problem domain. No op
+        actions "pass through" the literal expressions from one level of the
+        planning graph to the next.
+
+        The no-op action list requires both a positive and a negative action
+        for each literal expression. Positive no-op actions require the literal
+        as a positive precondition and add the literal expression as an effect
+        in the output, and negative no-op actions require the literal as a
+        negative precondition and remove the literal expression as an effect in
+        the output.
+
+        This function should only be called by the class constructor.
+
+        :param literal_list:
+        :return: list of Action
+        """
+        action_list = []
+        for fluent in literal_list:
+            act1 = Action(expr("Noop_pos({})".format(fluent)),\
+                                ([fluent], []), ([fluent], []))
+            action_list.append(act1)
+            act2 = Action(expr("Noop_neg({})".format(fluent)),\
+                                ([], [fluent]), ([], [fluent]))
+            action_list.append(act2)
+        return action_list
+
+    def create_graph(self):
+        """Build a Planning Graph as described in Russell-Norvig 3rd Ed 10.3
+        or 2nd Ed 11.4
+
+        The S0 initial level has been implemented for you.
+        It has no parents and includes all of the literal fluents that are part
+        of the initial state passed to the constructor.  At the start
+        of a problem planning search, this will be the same as the initial
+        state of the problem.  However, the planning graph can be built from
+        any state in the Planning Problem.
+
+        This function should only be called by the class constructor.
+
+        :return:
+            builds the graph by filling s_levels[] and a_levels[] lists with
+            node sets for each level.
+        """
+        # the graph should only be built during class construction
+        if (len(self.s_levels) != 0) or (len(self.a_levels) != 0):
+            raise Exception(
+                'Planning Graph already created;\
+                construct a new planning graph for each new state in the\
+                planning sequence')
+
+        # initialize S0 to literals in initial state provided.
+        leveled = False
+        level = 0
+        self.s_levels.append(set())  # S0 set of s_nodes - empty to start
+        # for each fluent in the initial state, add the correct literal PgNode_s
+        for literal in self.fs.pos:
+            self.s_levels[level].add(PgNode_s(literal, True))
+        for literal in self.fs.neg:
+            self.s_levels[level].add(PgNode_s(literal, False))
+        # no mutexes at the first level
+
+        # continue to build the graph alternating A,
+        # S levels until last two S levels contain the same literals,
+        # i.e. until it is "leveled"
+        while not leveled:
+            self.add_action_level(level)
+            self.update_a_mutex(self.a_levels[level])
+
+            level += 1
+            self.add_literal_level(level)
+            self.update_s_mutex(self.s_levels[level])
+
+            if self.s_levels[level] == self.s_levels[level - 1]:
+                leveled = True
+
+    def add_action_level(self, level):
+        """ add an A (action) level to the Planning Graph
+
+        :param level: int
+            the level number alternates S0, A0, S1, A1, S2, ...etc.
+            the level number is also used as the
+            index for the node set lists self.a_levels[] and self.s_levels[]
+        :return:
+            adds A nodes to the current level in self.a_levels[level]
+        """
+        previous_s_level = self.s_levels[level]
+        # Previous state level (previous to this level of actions)
+        actions = self.all_actions # List of all possible actions
+        self.a_levels.append(set()) # Initialize this level of actions
+        for action in actions:
+            a_node = PgNode_a(action)
+            if a_node.prenodes.issubset(previous_s_level):
+                for s_node in previous_s_level:
+                    # Connect action level to state and vice-versa
+                    s_node.children.add(a_node)
+                    a_node.parents.add(s_node)
+                # Add newly created action to this new level of actions
+                self.a_levels[level].add(a_node)
+
+    def add_literal_level(self, level):
+        """ add an S (literal) level to the Planning Graph
+
+        :param level: int
+            the level number alternates S0, A0, S1, A1, S2, ....etc.
+            the level number is also used as the
+            index for the node set lists self.a_levels[] and self.s_levels[]
+        :return:
+            adds S nodes to the current level in self.s_levels[level]
+        """
+        previous_a_level = self.a_levels[level - 1]
+        #Previous action level (previous to this level of states)
+        self.s_levels.append(set()) # Initialize this level of states
+        for a_node in previous_a_level:
+            for effect_node in a_node.effnodes:
+                # Connect state level to action and vice-versa
+                a_node.children.add(effect_node)
+                effect_node.parents.add(a_node)
+                # Add newly state to this new level of states
+                self.s_levels[level].add(effect_node)
+
+    def update_a_mutex(self, nodeset):
+        """ Determine and update sibling mutual exclusion for A-level nodes
+
+        Mutex action tests section from 3rd Ed. 10.3 or 2nd Ed. 11.4
+        A mutex relation holds between two actions a given level
+        if the planning graph is a serial planning graph and the pair are
+        nonpersistence actions or if any of the three conditions hold between
+        the pair:
+           Inconsistent Effects
+           Interference
+           Competing needs
+
+        :param nodeset: set of PgNode_a (siblings in the same level)
+        :return:
+            mutex set in each PgNode_a in the set is appropriately updated
+        """
+        nodelist = list(nodeset)
+        for i, n1 in enumerate(nodelist[:-1]):
+            for n2 in nodelist[i + 1:]:
+                if (self.serialize_actions(n1, n2) or
+                        self.inconsistent_effects_mutex(n1, n2) or
+                        self.interference_mutex(n1, n2) or
+                        self.competing_needs_mutex(n1, n2)):
+                    mutexify(n1, n2)
+
+    def serialize_actions(self, node_a1: PgNode_a, node_a2: PgNode_a) -> bool:
+        '''
+        Test a pair of actions for mutual exclusion, returning True if the
+        planning graph is serial, and if either action is persistent; otherwise
+        return False.  Two serial actions are mutually exclusive if they are
+        both non-persistent.
+
+        :param node_a1: PgNode_a
+        :param node_a2: PgNode_a
+        :return: bool
+        '''
+        #
+        if not self.serial:
+            return False
+        if node_a1.is_persistent or node_a2.is_persistent:
+            return False
+        return True
+
+    def inconsistent_effects_mutex(self, node_a1: PgNode_a,\
+                                    node_a2: PgNode_a) -> bool:
+        '''
+        Test a pair of actions for inconsistent effects, returning True if
+        one action negates an effect of the other, and False otherwise.
+
+        :param node_a1: PgNode_a
+        :param node_a2: PgNode_a
+        :return: bool
+        '''
+        return bool(
+            # Are actions from node 1 negated by actions in node 2?
+            set(node_a1.action.effect_add) & set(node_a2.action.effect_rem) |
+
+            # Are actions from node 2 negated by actions in node 1?
+            set(node_a2.action.effect_add) & set(node_a1.action.effect_rem)
+        )
+
+    def interference_mutex(self, node_a1: PgNode_a, node_a2: PgNode_a) -> bool:
+        '''
+        Test a pair of actions for mutual exclusion, returning True if the
+        effect of one action is the negation of a precondition of the other.
+
+        :param node_a1: PgNode_a
+        :param node_a2: PgNode_a
+        :return: bool
+        '''
+        return bool(
+            # Are actions in node 1 the negation of a precondition of node 2?
+            set(node_a1.action.effect_add) & set(node_a2.action.precond_neg) |
+            set(node_a1.action.effect_rem) & set(node_a2.action.precond_pos) |
+
+            # Are actions in node 2 the negation of a precondition of node 1?
+            set(node_a2.action.effect_add) & set(node_a1.action.precond_neg) |
+            set(node_a2.action.effect_rem) & set(node_a1.action.precond_pos)
+        )
+
+    def competing_needs_mutex(self, node_a1: PgNode_a,\
+                            node_a2: PgNode_a) -> bool:
+        '''
+        Test a pair of actions for mutual exclusion, returning True if one of
+        the precondition of one action is mutex with a precondition of the
+        other action.
+
+        :param node_a1: PgNode_a
+        :param node_a2: PgNode_a
+        :return: bool
+        '''
+        for a1_parent in node_a1.parents:
+            for a2_parent in node_a2.parents:
+                if a1_parent.is_mutex(a2_parent):
+                    return True
+        return False
+
+    def update_s_mutex(self, nodeset: set):
+        ''' Determine and update sibling mutual exclusion for S-level nodes
+
+        Mutex action tests section from 3rd Ed. 10.3 or 2nd Ed. 11.4
+        A mutex relation holds between literals at a given level
+        if either of the two conditions hold between the pair:
+           Negation
+           Inconsistent support
+
+        :param nodeset: set of PgNode_a (siblings in the same level)
+        :return:
+            mutex set in each PgNode_a in the set is appropriately updated
+        '''
+        nodelist = list(nodeset)
+        for i, n1 in enumerate(nodelist[:-1]):
+            for n2 in nodelist[i + 1:]:
+                if self.negation_mutex(n1, n2) or\
+                        self.inconsistent_support_mutex(n1, n2):
+                    mutexify(n1, n2)
+
+    def negation_mutex(self, node_s1: PgNode_s, node_s2: PgNode_s) -> bool:
+        '''
+        Test a pair of state literals for mutual exclusion, returning True if
+        one node is the negation of the other, and False otherwise.
+
+        :param node_s1: PgNode_s
+        :param node_s2: PgNode_s
+        :return: bool
+        '''
+        # Verify both state represent the same symbol
+        is_same_symbol = node_s1.symbol == node_s2.symbol
+        # Verify 1 state is the negation of the other
+        is_negation = node_s1.is_pos != node_s2.is_pos
+        return is_same_symbol and is_negation
+
+    def inconsistent_support_mutex(self, node_s1: PgNode_s, node_s2: PgNode_s):
+        '''
+        Test a pair of state literals for mutual exclusion, returning True if
+        there are no actions that could achieve the two literals at the same
+        time, and False otherwise.  In other words, the two literal nodes are
+        mutex if all of the actions that could achieve the first literal node
+        are pairwise mutually exclusive with all of the actions that could
+        achieve the second literal node.
+
+        :param node_s1: PgNode_s
+        :param node_s2: PgNode_s
+        :return: bool
+        '''
+        for s1_parent in node_s1.parents:
+            for s2_parent in node_s2.parents:
+                if not s1_parent.is_mutex(s2_parent):
+                    return False
+        return True
+
+    def h_levelsum(self) -> int:
+        '''The sum of the level costs of the individual goals
+        (admissible if goals independent)
+
+        :return: int
+        '''
+        level_sum = 0
+        goals = self.problem.goal
+        s_levels = self.s_levels
+        for goal in goals:
+            node = PgNode_s(goal, True)
+            s_levels_list = enumerate(s_levels)
+            for level, s_nodes in s_levels_list:
+                if node in s_nodes:
+                    level_sum += level
+                    break
+        return level_sum

Term-I – AI Foundations/03 - Cargo Planning/run_search.py

@@ -0,0 +1,142 @@
+import argparse
+from timeit import default_timer as timer
+from aimacode.search import InstrumentedProblem
+from aimacode.search import (breadth_first_search, astar_search,
+    breadth_first_tree_search, depth_first_graph_search, uniform_cost_search,
+    greedy_best_first_graph_search, depth_limited_search,
+    recursive_best_first_search)
+from my_air_cargo_problems import air_cargo_p1, air_cargo_p2, air_cargo_p3
+
+PROBLEM_CHOICE_MSG = """
+Select from the following list of air cargo problems. You may choose more than
+one by entering multiple selections separated by spaces.
+"""
+
+SEARCH_METHOD_CHOICE_MSG = """
+Select from the following list of search functions. You may choose more than
+one by entering multiple selections separated by spaces.
+"""
+
+INVALID_ARG_MSG = """
+You must either use the -m flag to run in manual mode, or use both the -p and
+-s flags to specify a list of problems and search algorithms to run. Valid
+choices for each include:
+"""
+
+PROBLEMS = [["Air Cargo Problem 1", air_cargo_p1],
+            ["Air Cargo Problem 2", air_cargo_p2],
+            ["Air Cargo Problem 3", air_cargo_p3]]
+SEARCHES = [["breadth_first_search", breadth_first_search, ""],
+            ['breadth_first_tree_search', breadth_first_tree_search, ""],
+            ['depth_first_graph_search', depth_first_graph_search, ""],
+            ['depth_limited_search', depth_limited_search, ""],
+            ['uniform_cost_search', uniform_cost_search, ""],
+            ['recursive_best_first_search', recursive_best_first_search, 'h_1'],
+            ['greedy_best_first_graph_search', greedy_best_first_graph_search, 'h_1'],
+            ['astar_search', astar_search, 'h_1'],
+            ['astar_search', astar_search, 'h_ignore_preconditions'],
+            ['astar_search', astar_search, 'h_pg_levelsum'],
+            ]
+
+
+class PrintableProblem(InstrumentedProblem):
+    """ InstrumentedProblem keeps track of stats during search, and this
+    class modifies the print output of those statistics for air cargo
+    problems.
+    """
+
+    def __repr__(self):
+        return '{:^10d}  {:^10d}  {:^10d}'.format(self.succs, self.goal_tests, self.states)
+
+
+def run_search(problem, search_function, parameter=None):
+
+    start = timer()
+    ip = PrintableProblem(problem)
+    if parameter is not None:
+        node = search_function(ip, parameter)
+    else:
+        node = search_function(ip)
+    end = timer()
+    print("\nExpansions   Goal Tests   New Nodes")
+    print("{}\n".format(ip))
+    show_solution(node, end - start)
+    print()
+
+
+def manual():
+
+    print(PROBLEM_CHOICE_MSG)
+    for idx, (name, _) in enumerate(PROBLEMS):
+        print("    {!s}. {}".format(idx+1, name))
+    p_choices = input("> ").split()
+
+    print(SEARCH_METHOD_CHOICE_MSG)
+    for idx, (name, _, heuristic) in enumerate(SEARCHES):
+        print("    {!s}. {} {}".format(idx+1, name, heuristic))
+    s_choices = input("> ").split()
+
+    main(p_choices, s_choices)
+
+    print("\nYou can run this selection again automatically from the command " +
+          "line\nwith the following command:")
+    print("\n  python {} -p {} -s {}\n".format(__file__,
+                                               " ".join(p_choices),
+                                               " ".join(s_choices)))
+
+
+def main(p_choices, s_choices):
+
+    problems = [PROBLEMS[i-1] for i in map(int, p_choices)]
+    searches = [SEARCHES[i-1] for i in map(int, s_choices)]
+
+    for pname, p in problems:
+
+        for sname, s, h in searches:
+            hstring = h if not h else " with {}".format(h)
+            print("\nSolving {} using {}{}...".format(pname, sname, hstring))
+
+            _p = p()
+            _h = None if not h else getattr(_p, h)
+            run_search(_p, s, _h)
+
+
+def show_solution(node, elapsed_time):
+    if node is None:
+        print("The selected planner did not find a solution for this problem. " +
+              "Make sure you have completed the AirCargoProblem implementation " +
+              "and pass all unit tests first.")
+    else:
+        print("Plan length: {}  Time elapsed in seconds: {}".format(len(node.solution()), elapsed_time))
+        for action in node.solution():
+            print("{}{}".format(action.name, action.args))
+
+if __name__=="__main__":
+    parser = argparse.ArgumentParser(description="Solve air cargo planning problems " + 
+        "using a variety of state space search methods including uninformed, greedy, " +
+        "and informed heuristic search.")
+    parser.add_argument('-m', '--manual', action="store_true",
+                        help="Interactively select the problems and searches to run.")
+    parser.add_argument('-p', '--problems', nargs="+", choices=range(1, len(PROBLEMS)+1), type=int, metavar='',
+                        help="Specify the indices of the problems to solve as a list of space separated values. Choose from: {!s}".format(list(range(1, len(PROBLEMS)+1))))
+    parser.add_argument('-s', '--searches', nargs="+", choices=range(1, len(SEARCHES)+1), type=int, metavar='',
+                        help="Specify the indices of the search algorithms to use as a list of space separated values. Choose from: {!s}".format(list(range(1, len(SEARCHES)+1))))
+    args = parser.parse_args()
+
+    if args.manual:
+        manual()
+    elif args.problems and args.searches:
+        main(list(sorted(set(args.problems))), list(sorted(set((args.searches)))))
+    else:
+        print()
+        parser.print_help()
+        print(INVALID_ARG_MSG)
+        print("Problems\n-----------------")
+        for idx, (name, _) in enumerate(PROBLEMS):
+            print("    {!s}. {}".format(idx+1, name))
+        print()
+        print("Search Algorithms\n-----------------")
+        for idx, (name, _, heuristic) in enumerate(SEARCHES):
+            print("    {!s}. {} {}".format(idx+1, name, heuristic))
+        print()
+        print("Use manual mode for interactive selection:\n\n\tpython run_search.py -m\n")

Term-I – AI Foundations/03 - Cargo Planning/tests/test_my_air_cargo_problems.py

@@ -0,0 +1,84 @@
+import os
+import sys
+import unittest
+
+from aimacode.planning import Action
+from aimacode.utils import expr
+from aimacode.search import Node
+from lp_utils import decode_state
+
+from my_air_cargo_problems import (
+    air_cargo_p1, air_cargo_p2, air_cargo_p3,
+)
+
+class TestAirCargoProb1(unittest.TestCase):
+
+    def setUp(self):
+        self.p1 = air_cargo_p1()
+
+    def test_ACP1_num_fluents(self):
+        self.assertEqual(len(self.p1.initial), 12)
+
+    def test_ACP1_num_requirements(self):
+        self.assertEqual(len(self.p1.goal),2)
+
+
+class TestAirCargoProb2(unittest.TestCase):
+
+    def setUp(self):
+        self.p2 = air_cargo_p2()
+
+    def test_ACP2_num_fluents(self):
+        self.assertEqual(len(self.p2.initial), 27)
+
+    def test_ACP2_num_requirements(self):
+        self.assertEqual(len(self.p2.goal),3)
+
+
+class TestAirCargoProb3(unittest.TestCase):
+
+    def setUp(self):
+        self.p3 = air_cargo_p3()
+
+    def test_ACP3_num_fluents(self):
+        self.assertEqual(len(self.p3.initial), 32)
+
+    def test_ACP3_num_requirements(self):
+        self.assertEqual(len(self.p3.goal),4)
+
+
+class TestAirCargoMethods(unittest.TestCase):
+
+    def setUp(self):
+        self.p1 = air_cargo_p1()
+        self.act1 = Action(
+            expr('Load(C1, P1, SFO)'),
+            [[expr('At(C1, SFO)'), expr('At(P1, SFO)')], []],
+            [[expr('In(C1, P1)')], [expr('At(C1, SFO)')]]
+        )
+
+    def test_AC_get_actions(self):
+        # to see a list of the actions, uncomment below
+        # print("\nactions for problem")
+        # for action in self.p1.actions_list:
+        #     print("{}{}".format(action.name, action.args))
+        self.assertEqual(len(self.p1.actions_list), 20)
+
+    def test_AC_actions(self):
+        # to see list of possible actions, uncomment below
+        # print("\npossible actions:")
+        # for action in self.p1.actions(self.p1.initial):
+        #     print("{}{}".format(action.name, action.args))
+        self.assertEqual(len(self.p1.actions(self.p1.initial)), 4)
+
+    def test_AC_result(self):
+        fs = decode_state(self.p1.result(self.p1.initial, self.act1), self.p1.state_map)
+        self.assertTrue(expr('In(C1, P1)') in fs.pos)
+        self.assertTrue(expr('At(C1, SFO)') in fs.neg)
+
+    def test_h_ignore_preconditions(self):
+        n = Node(self.p1.initial)
+        self.assertEqual(self.p1.h_ignore_preconditions(n),2)
+
+if __name__ == '__main__':
+    unittest.main()

Term-I – AI Foundations/03 - Cargo Planning/tests/test_my_planning_graph.py

@@ -0,0 +1,127 @@
+import os
+import sys
+import unittest
+
+from aimacode.utils import expr
+from aimacode.planning import Action
+from example_have_cake import have_cake
+
+from my_planning_graph import (
+    PlanningGraph, PgNode_a, PgNode_s, mutexify
+)
+
+
+class TestPlanningGraphLevels(unittest.TestCase):
+    def setUp(self):
+        self.p = have_cake()
+        self.pg = PlanningGraph(self.p, self.p.initial)
+
+    def test_add_action_level(self):
+        # for level, nodeset in enumerate(self.pg.a_levels):
+        #     for node in nodeset:
+        #         print("Level {}: {}{})".format(level, node.action.name, node.action.args))
+        self.assertEqual(len(self.pg.a_levels[0]), 3, len(self.pg.a_levels[0]))
+        self.assertEqual(len(self.pg.a_levels[1]), 6, len(self.pg.a_levels[1]))
+
+    def test_add_literal_level(self):
+        # for level, nodeset in enumerate(self.pg.s_levels):
+        #     for node in nodeset:
+        #         print("Level {}: {})".format(level, node.literal))
+        self.assertEqual(len(self.pg.s_levels[0]), 2, len(self.pg.s_levels[0]))
+        self.assertEqual(len(self.pg.s_levels[1]), 4, len(self.pg.s_levels[1]))
+        self.assertEqual(len(self.pg.s_levels[2]), 4, len(self.pg.s_levels[2]))
+
+
+class TestPlanningGraphMutex(unittest.TestCase):
+    def setUp(self):
+        self.p = have_cake()
+        self.pg = PlanningGraph(self.p, self.p.initial)
+        # some independent nodes for testing mutex
+        self.na1 = PgNode_a(Action(expr('Go(here)'),
+                                   [[], []], [[expr('At(here)')], []]))
+        self.na2 = PgNode_a(Action(expr('Go(there)'),
+                                   [[], []], [[expr('At(there)')], []]))
+        self.na3 = PgNode_a(Action(expr('Noop(At(there))'),
+                                   [[expr('At(there)')], []], [[expr('At(there)')], []]))
+        self.na4 = PgNode_a(Action(expr('Noop(At(here))'),
+                                   [[expr('At(here)')], []], [[expr('At(here)')], []]))
+        self.na5 = PgNode_a(Action(expr('Reverse(At(here))'),
+                                   [[expr('At(here)')], []], [[], [expr('At(here)')]]))
+        self.ns1 = PgNode_s(expr('At(here)'), True)
+        self.ns2 = PgNode_s(expr('At(there)'), True)
+        self.ns3 = PgNode_s(expr('At(here)'), False)
+        self.ns4 = PgNode_s(expr('At(there)'), False)
+        self.na1.children.add(self.ns1)
+        self.ns1.parents.add(self.na1)
+        self.na2.children.add(self.ns2)
+        self.ns2.parents.add(self.na2)
+        self.na1.parents.add(self.ns3)
+        self.na2.parents.add(self.ns4)
+
+    def test_serialize_mutex(self):
+        self.assertTrue(PlanningGraph.serialize_actions(self.pg, self.na1, self.na2),
+                        "Two persistence action nodes not marked as mutex")
+        self.assertFalse(PlanningGraph.serialize_actions(self.pg, self.na3, self.na4), "Two No-Ops were marked mutex")
+        self.assertFalse(PlanningGraph.serialize_actions(self.pg, self.na1, self.na3),
+                         "No-op and persistence action incorrectly marked as mutex")
+
+    def test_inconsistent_effects_mutex(self):
+        self.assertTrue(PlanningGraph.inconsistent_effects_mutex(self.pg, self.na4, self.na5),
+                        "Canceling effects not marked as mutex")
+        self.assertFalse(PlanningGraph.inconsistent_effects_mutex(self.pg, self.na1, self.na2),
+                         "Non-Canceling effects incorrectly marked as mutex")
+
+    def test_interference_mutex(self):
+        self.assertTrue(PlanningGraph.interference_mutex(self.pg, self.na4, self.na5),
+                        "Precondition from one node opposite of effect of other node should be mutex")
+        self.assertTrue(PlanningGraph.interference_mutex(self.pg, self.na5, self.na4),
+                        "Precondition from one node opposite of effect of other node should be mutex")
+        self.assertFalse(PlanningGraph.interference_mutex(self.pg, self.na1, self.na2),
+                         "Non-interfering incorrectly marked mutex")
+
+    def test_competing_needs_mutex(self):
+        self.assertFalse(PlanningGraph.competing_needs_mutex(self.pg, self.na1, self.na2),
+                         "Non-competing action nodes incorrectly marked as mutex")
+        mutexify(self.ns3, self.ns4)
+        self.assertTrue(PlanningGraph.competing_needs_mutex(self.pg, self.na1, self.na2),
+                        "Opposite preconditions from two action nodes not marked as mutex")
+
+    def test_negation_mutex(self):
+        self.assertTrue(PlanningGraph.negation_mutex(self.pg, self.ns1, self.ns3),
+                        "Opposite literal nodes not found to be Negation mutex")
+        self.assertFalse(PlanningGraph.negation_mutex(self.pg, self.ns1, self.ns2),
+                         "Same literal nodes found to be Negation mutex")
+
+    def test_inconsistent_support_mutex(self):
+        self.assertFalse(PlanningGraph.inconsistent_support_mutex(self.pg, self.ns1, self.ns2),
+                         "Independent node paths should NOT be inconsistent-support mutex")
+        mutexify(self.na1, self.na2)
+        self.assertTrue(PlanningGraph.inconsistent_support_mutex(self.pg, self.ns1, self.ns2),
+                        "Mutex parent actions should result in inconsistent-support mutex")
+
+        self.na6 = PgNode_a(Action(expr('Go(everywhere)'),
+                                   [[], []], [[expr('At(here)'), expr('At(there)')], []]))
+        self.na6.children.add(self.ns1)
+        self.ns1.parents.add(self.na6)
+        self.na6.children.add(self.ns2)
+        self.ns2.parents.add(self.na6)
+        self.na6.parents.add(self.ns3)
+        self.na6.parents.add(self.ns4)
+        mutexify(self.na1, self.na6)
+        mutexify(self.na2, self.na6)
+        self.assertFalse(PlanningGraph.inconsistent_support_mutex(
+            self.pg, self.ns1, self.ns2),
+            "If one parent action can achieve both states, should NOT be inconsistent-support mutex, even if parent actions are themselves mutex")
+
+
+class TestPlanningGraphHeuristics(unittest.TestCase):
+    def setUp(self):
+        self.p = have_cake()
+        self.pg = PlanningGraph(self.p, self.p.initial)
+
+    def test_levelsum(self):
+        self.assertEqual(self.pg.h_levelsum(), 1)
+
+
+if __name__ == '__main__':
+    unittest.main()

Term-I – AI Foundations/04 - ASL Recognizer/README.md

@@ -0,0 +1,43 @@
+# American Sign Language Recognizer
+> HMMs (Hidden Markov Models) are used to recognize words communicated using the American Sign Language (ASL). The system is trained on a dataset of videos that have been pre-processed and annotated and then tested on novel sequences.
+
+## About
+The template code is available at https://github.com/udacity/AIND-Recognizer.
+
+The overall goal of this project is to build a word recognizer for American Sign Language video sequences, demonstrating the power of probabilistic models. In particular, this project employs Hidden Markov Models (HMM's) to analyze a series of measurements taken from videos of American Sign Language (ASL) collected for research ([RWTH-BOSTON-104 Database](http://www-i6.informatik.rwth-aachen.de/~dreuw/database-rwth-boston-104.php)). In this video, the right-hand 'x' and 'y' locations are plotted as the speaker signs the sentence. The raw data, train, and test sets are pre-defined.
+
+In the first part of the project, a variety of feature sets are derived. Further, in Part-2, three different model selection criterion is implemented. The objective of Model Selection is to tune the number of states for each word HMM before testing on unseen data. In three methods: Log-likelihood using cross-validation folds (CV), Bayesian Information Criterion (BIC), Discriminative Information Criterion (DIC) are explored. Finally, the recognizer is and compare the effects the different combinations of feature sets and model selection criteria.
+
+### Data Description
+The data in the `/data/` directory was derived from the RWTH-BOSTON-104 Database. The hand positions (`hand_condensed.csv`) are pulled directly from the database `boston104.handpositions.rybach-forster-dreuw-2009-09-25.full.xml.`
+
+The three markers are:
+- `0`: speaker's left hand
+- `1`: speaker's right hand
+- `2`: speaker's nose
+- `X` & `Y` values of the video frame increase left-to-right and top-to-bottom.
+
+For purposes of this project, the sentences have been pre-segmented into words based on slow motion examination of the files. These segments are provided in the `train_words.csv` and `test_words.csv` files in the form of start and end frames (inclusive).
+
+The videos in the corpus include recordings from three different ASL speakers. The mappings for the three speakers to video are included in the `speaker.csv` file.
+
+## Requirements
+This project requires **Python 3** with `NumPy`, `Pandas`, `matplotlib`, `SciPy`, `scikit-learn`, `jupyter` and `hmmlearn`.
+
+It is recommended to use [Anaconda](https://www.continuum.io/downloads), a pre-packaged Python distribution that contains all of the necessary libraries and software for this project.
+
+`hmmlearn`, Version 0.2.1, contains a bug-fix related to the log function, which is used in this project. This version can be directly from its repo with the following command:
+
+`pip install git+https://github.com/hmmlearn/hmmlearn.git`
+
+## Files
+- `asl_recognizer.ipynb` - Main project notebook.
+
+- `asl_recognizer.html` – HTML export of the main notebook.
+
+- `my_model_selectors.py` – Contains model-selection class.
+
+- `my_recognizer.py` – Implements the recognizer.
+
+## License
+[Modified MIT License © Pranav Suri](/License.txt)

Term-I – AI Foundations/04 - ASL Recognizer/asl_recognizer.html

@@ -0,0 +1,15434 @@
+<!DOCTYPE html>
+<html>
+<head><meta charset="utf-8" />
+<title>asl_recognizer</title>
+
+<script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.1.10/require.min.js"></script>
+<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.3/jquery.min.js"></script>
+
+<style type="text/css">
+    /*!
+*
+* Twitter Bootstrap
+*
+*/
+/*!
+ * Bootstrap v3.3.6 (http://getbootstrap.com)
+ * Copyright 2011-2015 Twitter, Inc.
+ * Licensed under MIT (https://github.com/twbs/bootstrap/blob/master/LICENSE)
+ */
+/*! normalize.css v3.0.3 | MIT License | github.com/necolas/normalize.css */
+html {
+  font-family: sans-serif;
+  -ms-text-size-adjust: 100%;
+  -webkit-text-size-adjust: 100%;
+}
+body {
+  margin: 0;
+}
+article,
+aside,
+details,
+figcaption,
+figure,
+footer,
+header,
+hgroup,
+main,
+menu,
+nav,
+section,
+summary {
+  display: block;
+}
+audio,
+canvas,
+progress,
+video {
+  display: inline-block;
+  vertical-align: baseline;
+}
+audio:not([controls]) {
+  display: none;
+  height: 0;
+}
+[hidden],
+template {
+  display: none;
+}
+a {
+  background-color: transparent;
+}
+a:active,
+a:hover {
+  outline: 0;
+}
+abbr[title] {
+  border-bottom: 1px dotted;
+}
+b,
+strong {
+  font-weight: bold;
+}
+dfn {
+  font-style: italic;
+}
+h1 {
+  font-size: 2em;
+  margin: 0.67em 0;
+}
+mark {
+  background: #ff0;
+  color: #000;
+}
+small {
+  font-size: 80%;
+}
+sub,
+sup {
+  font-size: 75%;
+  line-height: 0;
+  position: relative;
+  vertical-align: baseline;
+}
+sup {
+  top: -0.5em;
+}
+sub {
+  bottom: -0.25em;
+}
+img {
+  border: 0;
+}
+svg:not(:root) {
+  overflow: hidden;
+}
+figure {
+  margin: 1em 40px;
+}
+hr {
+  box-sizing: content-box;
+  height: 0;
+}
+pre {
+  overflow: auto;
+}
+code,
+kbd,
+pre,
+samp {
+  font-family: monospace, monospace;
+  font-size: 1em;
+}
+button,
+input,
+optgroup,
+select,
+textarea {
+  color: inherit;
+  font: inherit;
+  margin: 0;
+}
+button {
+  overflow: visible;
+}
+button,
+select {
+  text-transform: none;
+}
+button,
+html input[type="button"],
+input[type="reset"],
+input[type="submit"] {
+  -webkit-appearance: button;
+  cursor: pointer;
+}
+button[disabled],
+html input[disabled] {
+  cursor: default;
+}
+button::-moz-focus-inner,
+input::-moz-focus-inner {
+  border: 0;
+  padding: 0;
+}
+input {
+  line-height: normal;
+}
+input[type="checkbox"],
+input[type="radio"] {
+  box-sizing: border-box;
+  padding: 0;
+}
+input[type="number"]::-webkit-inner-spin-button,
+input[type="number"]::-webkit-outer-spin-button {
+  height: auto;
+}
+input[type="search"] {
+  -webkit-appearance: textfield;
+  box-sizing: content-box;
+}
+input[type="search"]::-webkit-search-cancel-button,
+input[type="search"]::-webkit-search-decoration {
+  -webkit-appearance: none;
+}
+fieldset {
+  border: 1px solid #c0c0c0;
+  margin: 0 2px;
+  padding: 0.35em 0.625em 0.75em;
+}
+legend {
+  border: 0;
+  padding: 0;
+}
+textarea {
+  overflow: auto;
+}
+optgroup {
+  font-weight: bold;
+}
+table {
+  border-collapse: collapse;
+  border-spacing: 0;
+}
+td,
+th {
+  padding: 0;
+}
+/*! Source: https://github.com/h5bp/html5-boilerplate/blob/master/src/css/main.css */
+@media print {
+  *,
+  *:before,
+  *:after {
+    background: transparent !important;
+    color: #000 !important;
+    box-shadow: none !important;
+    text-shadow: none !important;
+  }
+  a,
+  a:visited {
+    text-decoration: underline;
+  }
+  a[href]:after {
+    content: " (" attr(href) ")";
+  }
+  abbr[title]:after {
+    content: " (" attr(title) ")";
+  }
+  a[href^="#"]:after,
+  a[href^="javascript:"]:after {
+    content: "";
+  }
+  pre,
+  blockquote {
+    border: 1px solid #999;
+    page-break-inside: avoid;
+  }
+  thead {
+    display: table-header-group;
+  }
+  tr,
+  img {
+    page-break-inside: avoid;
+  }
+  img {
+    max-width: 100% !important;
+  }
+  p,
+  h2,
+  h3 {
+    orphans: 3;
+    widows: 3;
+  }
+  h2,
+  h3 {
+    page-break-after: avoid;
+  }
+  .navbar {
+    display: none;
+  }
+  .btn > .caret,
+  .dropup > .btn > .caret {
+    border-top-color: #000 !important;
+  }
+  .label {
+    border: 1px solid #000;
+  }
+  .table {
+    border-collapse: collapse !important;
+  }
+  .table td,
+  .table th {
+    background-color: #fff !important;
+  }
+  .table-bordered th,
+  .table-bordered td {
+    border: 1px solid #ddd !important;
+  }
+}
+@font-face {
+  font-family: 'Glyphicons Halflings';
+  src: url('../components/bootstrap/fonts/glyphicons-halflings-regular.eot');
+  src: url('../components/bootstrap/fonts/glyphicons-halflings-regular.eot?#iefix') format('embedded-opentype'), url('../components/bootstrap/fonts/glyphicons-halflings-regular.woff2') format('woff2'), url('../components/bootstrap/fonts/glyphicons-halflings-regular.woff') format('woff'), url('../components/bootstrap/fonts/glyphicons-halflings-regular.ttf') format('truetype'), url('../components/bootstrap/fonts/glyphicons-halflings-regular.svg#glyphicons_halflingsregular') format('svg');
+}
+.glyphicon {
+  position: relative;
+  top: 1px;
+  display: inline-block;
+  font-family: 'Glyphicons Halflings';
+  font-style: normal;
+  font-weight: normal;
+  line-height: 1;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+}
+.glyphicon-asterisk:before {
+  content: "\002a";
+}
+.glyphicon-plus:before {
+  content: "\002b";
+}
+.glyphicon-euro:before,
+.glyphicon-eur:before {
+  content: "\20ac";
+}
+.glyphicon-minus:before {
+  content: "\2212";
+}
+.glyphicon-cloud:before {
+  content: "\2601";
+}
+.glyphicon-envelope:before {
+  content: "\2709";
+}
+.glyphicon-pencil:before {
+  content: "\270f";
+}
+.glyphicon-glass:before {
+  content: "\e001";
+}
+.glyphicon-music:before {
+  content: "\e002";
+}
+.glyphicon-search:before {
+  content: "\e003";
+}
+.glyphicon-heart:before {
+  content: "\e005";
+}
+.glyphicon-star:before {
+  content: "\e006";
+}
+.glyphicon-star-empty:before {
+  content: "\e007";
+}
+.glyphicon-user:before {
+  content: "\e008";
+}
+.glyphicon-film:before {
+  content: "\e009";
+}
+.glyphicon-th-large:before {
+  content: "\e010";
+}
+.glyphicon-th:before {
+  content: "\e011";
+}
+.glyphicon-th-list:before {
+  content: "\e012";
+}
+.glyphicon-ok:before {
+  content: "\e013";
+}
+.glyphicon-remove:before {
+  content: "\e014";
+}
+.glyphicon-zoom-in:before {
+  content: "\e015";
+}
+.glyphicon-zoom-out:before {
+  content: "\e016";
+}
+.glyphicon-off:before {
+  content: "\e017";
+}
+.glyphicon-signal:before {
+  content: "\e018";
+}
+.glyphicon-cog:before {
+  content: "\e019";
+}
+.glyphicon-trash:before {
+  content: "\e020";
+}
+.glyphicon-home:before {
+  content: "\e021";
+}
+.glyphicon-file:before {
+  content: "\e022";
+}
+.glyphicon-time:before {
+  content: "\e023";
+}
+.glyphicon-road:before {
+  content: "\e024";
+}
+.glyphicon-download-alt:before {
+  content: "\e025";
+}
+.glyphicon-download:before {
+  content: "\e026";
+}
+.glyphicon-upload:before {
+  content: "\e027";
+}
+.glyphicon-inbox:before {
+  content: "\e028";
+}
+.glyphicon-play-circle:before {
+  content: "\e029";
+}
+.glyphicon-repeat:before {
+  content: "\e030";
+}
+.glyphicon-refresh:before {
+  content: "\e031";
+}
+.glyphicon-list-alt:before {
+  content: "\e032";
+}
+.glyphicon-lock:before {
+  content: "\e033";
+}
+.glyphicon-flag:before {
+  content: "\e034";
+}
+.glyphicon-headphones:before {
+  content: "\e035";
+}
+.glyphicon-volume-off:before {
+  content: "\e036";
+}
+.glyphicon-volume-down:before {
+  content: "\e037";
+}
+.glyphicon-volume-up:before {
+  content: "\e038";
+}
+.glyphicon-qrcode:before {
+  content: "\e039";
+}
+.glyphicon-barcode:before {
+  content: "\e040";
+}
+.glyphicon-tag:before {
+  content: "\e041";
+}
+.glyphicon-tags:before {
+  content: "\e042";
+}
+.glyphicon-book:before {
+  content: "\e043";
+}
+.glyphicon-bookmark:before {
+  content: "\e044";
+}
+.glyphicon-print:before {
+  content: "\e045";
+}
+.glyphicon-camera:before {
+  content: "\e046";
+}
+.glyphicon-font:before {
+  content: "\e047";
+}
+.glyphicon-bold:before {
+  content: "\e048";
+}
+.glyphicon-italic:before {
+  content: "\e049";
+}
+.glyphicon-text-height:before {
+  content: "\e050";
+}
+.glyphicon-text-width:before {
+  content: "\e051";
+}
+.glyphicon-align-left:before {
+  content: "\e052";
+}
+.glyphicon-align-center:before {
+  content: "\e053";
+}
+.glyphicon-align-right:before {
+  content: "\e054";
+}
+.glyphicon-align-justify:before {
+  content: "\e055";
+}
+.glyphicon-list:before {
+  content: "\e056";
+}
+.glyphicon-indent-left:before {
+  content: "\e057";
+}
+.glyphicon-indent-right:before {
+  content: "\e058";
+}
+.glyphicon-facetime-video:before {
+  content: "\e059";
+}
+.glyphicon-picture:before {
+  content: "\e060";
+}
+.glyphicon-map-marker:before {
+  content: "\e062";
+}
+.glyphicon-adjust:before {
+  content: "\e063";
+}
+.glyphicon-tint:before {
+  content: "\e064";
+}
+.glyphicon-edit:before {
+  content: "\e065";
+}
+.glyphicon-share:before {
+  content: "\e066";
+}
+.glyphicon-check:before {
+  content: "\e067";
+}
+.glyphicon-move:before {
+  content: "\e068";
+}
+.glyphicon-step-backward:before {
+  content: "\e069";
+}
+.glyphicon-fast-backward:before {
+  content: "\e070";
+}
+.glyphicon-backward:before {
+  content: "\e071";
+}
+.glyphicon-play:before {
+  content: "\e072";
+}
+.glyphicon-pause:before {
+  content: "\e073";
+}
+.glyphicon-stop:before {
+  content: "\e074";
+}
+.glyphicon-forward:before {
+  content: "\e075";
+}
+.glyphicon-fast-forward:before {
+  content: "\e076";
+}
+.glyphicon-step-forward:before {
+  content: "\e077";
+}
+.glyphicon-eject:before {
+  content: "\e078";
+}
+.glyphicon-chevron-left:before {
+  content: "\e079";
+}
+.glyphicon-chevron-right:before {
+  content: "\e080";
+}
+.glyphicon-plus-sign:before {
+  content: "\e081";
+}
+.glyphicon-minus-sign:before {
+  content: "\e082";
+}
+.glyphicon-remove-sign:before {
+  content: "\e083";
+}
+.glyphicon-ok-sign:before {
+  content: "\e084";
+}
+.glyphicon-question-sign:before {
+  content: "\e085";
+}
+.glyphicon-info-sign:before {
+  content: "\e086";
+}
+.glyphicon-screenshot:before {
+  content: "\e087";
+}
+.glyphicon-remove-circle:before {
+  content: "\e088";
+}
+.glyphicon-ok-circle:before {
+  content: "\e089";
+}
+.glyphicon-ban-circle:before {
+  content: "\e090";
+}
+.glyphicon-arrow-left:before {
+  content: "\e091";
+}
+.glyphicon-arrow-right:before {
+  content: "\e092";
+}
+.glyphicon-arrow-up:before {
+  content: "\e093";
+}
+.glyphicon-arrow-down:before {
+  content: "\e094";
+}
+.glyphicon-share-alt:before {
+  content: "\e095";
+}
+.glyphicon-resize-full:before {
+  content: "\e096";
+}
+.glyphicon-resize-small:before {
+  content: "\e097";
+}
+.glyphicon-exclamation-sign:before {
+  content: "\e101";
+}
+.glyphicon-gift:before {
+  content: "\e102";
+}
+.glyphicon-leaf:before {
+  content: "\e103";
+}
+.glyphicon-fire:before {
+  content: "\e104";
+}
+.glyphicon-eye-open:before {
+  content: "\e105";
+}
+.glyphicon-eye-close:before {
+  content: "\e106";
+}
+.glyphicon-warning-sign:before {
+  content: "\e107";
+}
+.glyphicon-plane:before {
+  content: "\e108";
+}
+.glyphicon-calendar:before {
+  content: "\e109";
+}
+.glyphicon-random:before {
+  content: "\e110";
+}
+.glyphicon-comment:before {
+  content: "\e111";
+}
+.glyphicon-magnet:before {
+  content: "\e112";
+}
+.glyphicon-chevron-up:before {
+  content: "\e113";
+}
+.glyphicon-chevron-down:before {
+  content: "\e114";
+}
+.glyphicon-retweet:before {
+  content: "\e115";
+}
+.glyphicon-shopping-cart:before {
+  content: "\e116";
+}
+.glyphicon-folder-close:before {
+  content: "\e117";
+}
+.glyphicon-folder-open:before {
+  content: "\e118";
+}
+.glyphicon-resize-vertical:before {
+  content: "\e119";
+}
+.glyphicon-resize-horizontal:before {
+  content: "\e120";
+}
+.glyphicon-hdd:before {
+  content: "\e121";
+}
+.glyphicon-bullhorn:before {
+  content: "\e122";
+}
+.glyphicon-bell:before {
+  content: "\e123";
+}
+.glyphicon-certificate:before {
+  content: "\e124";
+}
+.glyphicon-thumbs-up:before {
+  content: "\e125";
+}
+.glyphicon-thumbs-down:before {
+  content: "\e126";
+}
+.glyphicon-hand-right:before {
+  content: "\e127";
+}
+.glyphicon-hand-left:before {
+  content: "\e128";
+}
+.glyphicon-hand-up:before {
+  content: "\e129";
+}
+.glyphicon-hand-down:before {
+  content: "\e130";
+}
+.glyphicon-circle-arrow-right:before {
+  content: "\e131";
+}
+.glyphicon-circle-arrow-left:before {
+  content: "\e132";
+}
+.glyphicon-circle-arrow-up:before {
+  content: "\e133";
+}
+.glyphicon-circle-arrow-down:before {
+  content: "\e134";
+}
+.glyphicon-globe:before {
+  content: "\e135";
+}
+.glyphicon-wrench:before {
+  content: "\e136";
+}
+.glyphicon-tasks:before {
+  content: "\e137";
+}
+.glyphicon-filter:before {
+  content: "\e138";
+}
+.glyphicon-briefcase:before {
+  content: "\e139";
+}
+.glyphicon-fullscreen:before {
+  content: "\e140";
+}
+.glyphicon-dashboard:before {
+  content: "\e141";
+}
+.glyphicon-paperclip:before {
+  content: "\e142";
+}
+.glyphicon-heart-empty:before {
+  content: "\e143";
+}
+.glyphicon-link:before {
+  content: "\e144";
+}
+.glyphicon-phone:before {
+  content: "\e145";
+}
+.glyphicon-pushpin:before {
+  content: "\e146";
+}
+.glyphicon-usd:before {
+  content: "\e148";
+}
+.glyphicon-gbp:before {
+  content: "\e149";
+}
+.glyphicon-sort:before {
+  content: "\e150";
+}
+.glyphicon-sort-by-alphabet:before {
+  content: "\e151";
+}
+.glyphicon-sort-by-alphabet-alt:before {
+  content: "\e152";
+}
+.glyphicon-sort-by-order:before {
+  content: "\e153";
+}
+.glyphicon-sort-by-order-alt:before {
+  content: "\e154";
+}
+.glyphicon-sort-by-attributes:before {
+  content: "\e155";
+}
+.glyphicon-sort-by-attributes-alt:before {
+  content: "\e156";
+}
+.glyphicon-unchecked:before {
+  content: "\e157";
+}
+.glyphicon-expand:before {
+  content: "\e158";
+}
+.glyphicon-collapse-down:before {
+  content: "\e159";
+}
+.glyphicon-collapse-up:before {
+  content: "\e160";
+}
+.glyphicon-log-in:before {
+  content: "\e161";
+}
+.glyphicon-flash:before {
+  content: "\e162";
+}
+.glyphicon-log-out:before {
+  content: "\e163";
+}
+.glyphicon-new-window:before {
+  content: "\e164";
+}
+.glyphicon-record:before {
+  content: "\e165";
+}
+.glyphicon-save:before {
+  content: "\e166";
+}
+.glyphicon-open:before {
+  content: "\e167";
+}
+.glyphicon-saved:before {
+  content: "\e168";
+}
+.glyphicon-import:before {
+  content: "\e169";
+}
+.glyphicon-export:before {
+  content: "\e170";
+}
+.glyphicon-send:before {
+  content: "\e171";
+}
+.glyphicon-floppy-disk:before {
+  content: "\e172";
+}
+.glyphicon-floppy-saved:before {
+  content: "\e173";
+}
+.glyphicon-floppy-remove:before {
+  content: "\e174";
+}
+.glyphicon-floppy-save:before {
+  content: "\e175";
+}
+.glyphicon-floppy-open:before {
+  content: "\e176";
+}
+.glyphicon-credit-card:before {
+  content: "\e177";
+}
+.glyphicon-transfer:before {
+  content: "\e178";
+}
+.glyphicon-cutlery:before {
+  content: "\e179";
+}
+.glyphicon-header:before {
+  content: "\e180";
+}
+.glyphicon-compressed:before {
+  content: "\e181";
+}
+.glyphicon-earphone:before {
+  content: "\e182";
+}
+.glyphicon-phone-alt:before {
+  content: "\e183";
+}
+.glyphicon-tower:before {
+  content: "\e184";
+}
+.glyphicon-stats:before {
+  content: "\e185";
+}
+.glyphicon-sd-video:before {
+  content: "\e186";
+}
+.glyphicon-hd-video:before {
+  content: "\e187";
+}
+.glyphicon-subtitles:before {
+  content: "\e188";
+}
+.glyphicon-sound-stereo:before {
+  content: "\e189";
+}
+.glyphicon-sound-dolby:before {
+  content: "\e190";
+}
+.glyphicon-sound-5-1:before {
+  content: "\e191";
+}
+.glyphicon-sound-6-1:before {
+  content: "\e192";
+}
+.glyphicon-sound-7-1:before {
+  content: "\e193";
+}
+.glyphicon-copyright-mark:before {
+  content: "\e194";
+}
+.glyphicon-registration-mark:before {
+  content: "\e195";
+}
+.glyphicon-cloud-download:before {
+  content: "\e197";
+}
+.glyphicon-cloud-upload:before {
+  content: "\e198";
+}
+.glyphicon-tree-conifer:before {
+  content: "\e199";
+}
+.glyphicon-tree-deciduous:before {
+  content: "\e200";
+}
+.glyphicon-cd:before {
+  content: "\e201";
+}
+.glyphicon-save-file:before {
+  content: "\e202";
+}
+.glyphicon-open-file:before {
+  content: "\e203";
+}
+.glyphicon-level-up:before {
+  content: "\e204";
+}
+.glyphicon-copy:before {
+  content: "\e205";
+}
+.glyphicon-paste:before {
+  content: "\e206";
+}
+.glyphicon-alert:before {
+  content: "\e209";
+}
+.glyphicon-equalizer:before {
+  content: "\e210";
+}
+.glyphicon-king:before {
+  content: "\e211";
+}
+.glyphicon-queen:before {
+  content: "\e212";
+}
+.glyphicon-pawn:before {
+  content: "\e213";
+}
+.glyphicon-bishop:before {
+  content: "\e214";
+}
+.glyphicon-knight:before {
+  content: "\e215";
+}
+.glyphicon-baby-formula:before {
+  content: "\e216";
+}
+.glyphicon-tent:before {
+  content: "\26fa";
+}
+.glyphicon-blackboard:before {
+  content: "\e218";
+}
+.glyphicon-bed:before {
+  content: "\e219";
+}
+.glyphicon-apple:before {
+  content: "\f8ff";
+}
+.glyphicon-erase:before {
+  content: "\e221";
+}
+.glyphicon-hourglass:before {
+  content: "\231b";
+}
+.glyphicon-lamp:before {
+  content: "\e223";
+}
+.glyphicon-duplicate:before {
+  content: "\e224";
+}
+.glyphicon-piggy-bank:before {
+  content: "\e225";
+}
+.glyphicon-scissors:before {
+  content: "\e226";
+}
+.glyphicon-bitcoin:before {
+  content: "\e227";
+}
+.glyphicon-btc:before {
+  content: "\e227";
+}
+.glyphicon-xbt:before {
+  content: "\e227";
+}
+.glyphicon-yen:before {
+  content: "\00a5";
+}
+.glyphicon-jpy:before {
+  content: "\00a5";
+}
+.glyphicon-ruble:before {
+  content: "\20bd";
+}
+.glyphicon-rub:before {
+  content: "\20bd";
+}
+.glyphicon-scale:before {
+  content: "\e230";
+}
+.glyphicon-ice-lolly:before {
+  content: "\e231";
+}
+.glyphicon-ice-lolly-tasted:before {
+  content: "\e232";
+}
+.glyphicon-education:before {
+  content: "\e233";
+}
+.glyphicon-option-horizontal:before {
+  content: "\e234";
+}
+.glyphicon-option-vertical:before {
+  content: "\e235";
+}
+.glyphicon-menu-hamburger:before {
+  content: "\e236";
+}
+.glyphicon-modal-window:before {
+  content: "\e237";
+}
+.glyphicon-oil:before {
+  content: "\e238";
+}
+.glyphicon-grain:before {
+  content: "\e239";
+}
+.glyphicon-sunglasses:before {
+  content: "\e240";
+}
+.glyphicon-text-size:before {
+  content: "\e241";
+}
+.glyphicon-text-color:before {
+  content: "\e242";
+}
+.glyphicon-text-background:before {
+  content: "\e243";
+}
+.glyphicon-object-align-top:before {
+  content: "\e244";
+}
+.glyphicon-object-align-bottom:before {
+  content: "\e245";
+}
+.glyphicon-object-align-horizontal:before {
+  content: "\e246";
+}
+.glyphicon-object-align-left:before {
+  content: "\e247";
+}
+.glyphicon-object-align-vertical:before {
+  content: "\e248";
+}
+.glyphicon-object-align-right:before {
+  content: "\e249";
+}
+.glyphicon-triangle-right:before {
+  content: "\e250";
+}
+.glyphicon-triangle-left:before {
+  content: "\e251";
+}
+.glyphicon-triangle-bottom:before {
+  content: "\e252";
+}
+.glyphicon-triangle-top:before {
+  content: "\e253";
+}
+.glyphicon-console:before {
+  content: "\e254";
+}
+.glyphicon-superscript:before {
+  content: "\e255";
+}
+.glyphicon-subscript:before {
+  content: "\e256";
+}
+.glyphicon-menu-left:before {
+  content: "\e257";
+}
+.glyphicon-menu-right:before {
+  content: "\e258";
+}
+.glyphicon-menu-down:before {
+  content: "\e259";
+}
+.glyphicon-menu-up:before {
+  content: "\e260";
+}
+* {
+  -webkit-box-sizing: border-box;
+  -moz-box-sizing: border-box;
+  box-sizing: border-box;
+}
+*:before,
+*:after {
+  -webkit-box-sizing: border-box;
+  -moz-box-sizing: border-box;
+  box-sizing: border-box;
+}
+html {
+  font-size: 10px;
+  -webkit-tap-highlight-color: rgba(0, 0, 0, 0);
+}
+body {
+  font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
+  font-size: 13px;
+  line-height: 1.42857143;
+  color: #000;
+  background-color: #fff;
+}
+input,
+button,
+select,
+textarea {
+  font-family: inherit;
+  font-size: inherit;
+  line-height: inherit;
+}
+a {
+  color: #337ab7;
+  text-decoration: none;
+}
+a:hover,
+a:focus {
+  color: #23527c;
+  text-decoration: underline;
+}
+a:focus {
+  outline: thin dotted;
+  outline: 5px auto -webkit-focus-ring-color;
+  outline-offset: -2px;
+}
+figure {
+  margin: 0;
+}
+img {
+  vertical-align: middle;
+}
+.img-responsive,
+.thumbnail > img,
+.thumbnail a > img,
+.carousel-inner > .item > img,
+.carousel-inner > .item > a > img {
+  display: block;
+  max-width: 100%;
+  height: auto;
+}
+.img-rounded {
+  border-radius: 3px;
+}
+.img-thumbnail {
+  padding: 4px;
+  line-height: 1.42857143;
+  background-color: #fff;
+  border: 1px solid #ddd;
+  border-radius: 2px;
+  -webkit-transition: all 0.2s ease-in-out;
+  -o-transition: all 0.2s ease-in-out;
+  transition: all 0.2s ease-in-out;
+  display: inline-block;
+  max-width: 100%;
+  height: auto;
+}
+.img-circle {
+  border-radius: 50%;
+}
+hr {
+  margin-top: 18px;
+  margin-bottom: 18px;
+  border: 0;
+  border-top: 1px solid #eeeeee;
+}
+.sr-only {
+  position: absolute;
+  width: 1px;
+  height: 1px;
+  margin: -1px;
+  padding: 0;
+  overflow: hidden;
+  clip: rect(0, 0, 0, 0);
+  border: 0;
+}
+.sr-only-focusable:active,
+.sr-only-focusable:focus {
+  position: static;
+  width: auto;
+  height: auto;
+  margin: 0;
+  overflow: visible;
+  clip: auto;
+}
+[role="button"] {
+  cursor: pointer;
+}
+h1,
+h2,
+h3,
+h4,
+h5,
+h6,
+.h1,
+.h2,
+.h3,
+.h4,
+.h5,
+.h6 {
+  font-family: inherit;
+  font-weight: 500;
+  line-height: 1.1;
+  color: inherit;
+}
+h1 small,
+h2 small,
+h3 small,
+h4 small,
+h5 small,
+h6 small,
+.h1 small,
+.h2 small,
+.h3 small,
+.h4 small,
+.h5 small,
+.h6 small,
+h1 .small,
+h2 .small,
+h3 .small,
+h4 .small,
+h5 .small,
+h6 .small,
+.h1 .small,
+.h2 .small,
+.h3 .small,
+.h4 .small,
+.h5 .small,
+.h6 .small {
+  font-weight: normal;
+  line-height: 1;
+  color: #777777;
+}
+h1,
+.h1,
+h2,
+.h2,
+h3,
+.h3 {
+  margin-top: 18px;
+  margin-bottom: 9px;
+}
+h1 small,
+.h1 small,
+h2 small,
+.h2 small,
+h3 small,
+.h3 small,
+h1 .small,
+.h1 .small,
+h2 .small,
+.h2 .small,
+h3 .small,
+.h3 .small {
+  font-size: 65%;
+}
+h4,
+.h4,
+h5,
+.h5,
+h6,
+.h6 {
+  margin-top: 9px;
+  margin-bottom: 9px;
+}
+h4 small,
+.h4 small,
+h5 small,
+.h5 small,
+h6 small,
+.h6 small,
+h4 .small,
+.h4 .small,
+h5 .small,
+.h5 .small,
+h6 .small,
+.h6 .small {
+  font-size: 75%;
+}
+h1,
+.h1 {
+  font-size: 33px;
+}
+h2,
+.h2 {
+  font-size: 27px;
+}
+h3,
+.h3 {
+  font-size: 23px;
+}
+h4,
+.h4 {
+  font-size: 17px;
+}
+h5,
+.h5 {
+  font-size: 13px;
+}
+h6,
+.h6 {
+  font-size: 12px;
+}
+p {
+  margin: 0 0 9px;
+}
+.lead {
+  margin-bottom: 18px;
+  font-size: 14px;
+  font-weight: 300;
+  line-height: 1.4;
+}
+@media (min-width: 768px) {
+  .lead {
+    font-size: 19.5px;
+  }
+}
+small,
+.small {
+  font-size: 92%;
+}
+mark,
+.mark {
+  background-color: #fcf8e3;
+  padding: .2em;
+}
+.text-left {
+  text-align: left;
+}
+.text-right {
+  text-align: right;
+}
+.text-center {
+  text-align: center;
+}
+.text-justify {
+  text-align: justify;
+}
+.text-nowrap {
+  white-space: nowrap;
+}
+.text-lowercase {
+  text-transform: lowercase;
+}
+.text-uppercase {
+  text-transform: uppercase;
+}
+.text-capitalize {
+  text-transform: capitalize;
+}
+.text-muted {
+  color: #777777;
+}
+.text-primary {
+  color: #337ab7;
+}
+a.text-primary:hover,
+a.text-primary:focus {
+  color: #286090;
+}
+.text-success {
+  color: #3c763d;
+}
+a.text-success:hover,
+a.text-success:focus {
+  color: #2b542c;
+}
+.text-info {
+  color: #31708f;
+}
+a.text-info:hover,
+a.text-info:focus {
+  color: #245269;
+}
+.text-warning {
+  color: #8a6d3b;
+}
+a.text-warning:hover,
+a.text-warning:focus {
+  color: #66512c;
+}
+.text-danger {
+  color: #a94442;
+}
+a.text-danger:hover,
+a.text-danger:focus {
+  color: #843534;
+}
+.bg-primary {
+  color: #fff;
+  background-color: #337ab7;
+}
+a.bg-primary:hover,
+a.bg-primary:focus {
+  background-color: #286090;
+}
+.bg-success {
+  background-color: #dff0d8;
+}
+a.bg-success:hover,
+a.bg-success:focus {
+  background-color: #c1e2b3;
+}
+.bg-info {
+  background-color: #d9edf7;
+}
+a.bg-info:hover,
+a.bg-info:focus {
+  background-color: #afd9ee;
+}
+.bg-warning {
+  background-color: #fcf8e3;
+}
+a.bg-warning:hover,
+a.bg-warning:focus {
+  background-color: #f7ecb5;
+}
+.bg-danger {
+  background-color: #f2dede;
+}
+a.bg-danger:hover,
+a.bg-danger:focus {
+  background-color: #e4b9b9;
+}
+.page-header {
+  padding-bottom: 8px;
+  margin: 36px 0 18px;
+  border-bottom: 1px solid #eeeeee;
+}
+ul,
+ol {
+  margin-top: 0;
+  margin-bottom: 9px;
+}
+ul ul,
+ol ul,
+ul ol,
+ol ol {
+  margin-bottom: 0;
+}
+.list-unstyled {
+  padding-left: 0;
+  list-style: none;
+}
+.list-inline {
+  padding-left: 0;
+  list-style: none;
+  margin-left: -5px;
+}
+.list-inline > li {
+  display: inline-block;
+  padding-left: 5px;
+  padding-right: 5px;
+}
+dl {
+  margin-top: 0;
+  margin-bottom: 18px;
+}
+dt,
+dd {
+  line-height: 1.42857143;
+}
+dt {
+  font-weight: bold;
+}
+dd {
+  margin-left: 0;
+}
+@media (min-width: 541px) {
+  .dl-horizontal dt {
+    float: left;
+    width: 160px;
+    clear: left;
+    text-align: right;
+    overflow: hidden;
+    text-overflow: ellipsis;
+    white-space: nowrap;
+  }
+  .dl-horizontal dd {
+    margin-left: 180px;
+  }
+}
+abbr[title],
+abbr[data-original-title] {
+  cursor: help;
+  border-bottom: 1px dotted #777777;
+}
+.initialism {
+  font-size: 90%;
+  text-transform: uppercase;
+}
+blockquote {
+  padding: 9px 18px;
+  margin: 0 0 18px;
+  font-size: inherit;
+  border-left: 5px solid #eeeeee;
+}
+blockquote p:last-child,
+blockquote ul:last-child,
+blockquote ol:last-child {
+  margin-bottom: 0;
+}
+blockquote footer,
+blockquote small,
+blockquote .small {
+  display: block;
+  font-size: 80%;
+  line-height: 1.42857143;
+  color: #777777;
+}
+blockquote footer:before,
+blockquote small:before,
+blockquote .small:before {
+  content: '\2014 \00A0';
+}
+.blockquote-reverse,
+blockquote.pull-right {
+  padding-right: 15px;
+  padding-left: 0;
+  border-right: 5px solid #eeeeee;
+  border-left: 0;
+  text-align: right;
+}
+.blockquote-reverse footer:before,
+blockquote.pull-right footer:before,
+.blockquote-reverse small:before,
+blockquote.pull-right small:before,
+.blockquote-reverse .small:before,
+blockquote.pull-right .small:before {
+  content: '';
+}
+.blockquote-reverse footer:after,
+blockquote.pull-right footer:after,
+.blockquote-reverse small:after,
+blockquote.pull-right small:after,
+.blockquote-reverse .small:after,
+blockquote.pull-right .small:after {
+  content: '\00A0 \2014';
+}
+address {
+  margin-bottom: 18px;
+  font-style: normal;
+  line-height: 1.42857143;
+}
+code,
+kbd,
+pre,
+samp {
+  font-family: monospace;
+}
+code {
+  padding: 2px 4px;
+  font-size: 90%;
+  color: #c7254e;
+  background-color: #f9f2f4;
+  border-radius: 2px;
+}
+kbd {
+  padding: 2px 4px;
+  font-size: 90%;
+  color: #888;
+  background-color: transparent;
+  border-radius: 1px;
+  box-shadow: inset 0 -1px 0 rgba(0, 0, 0, 0.25);
+}
+kbd kbd {
+  padding: 0;
+  font-size: 100%;
+  font-weight: bold;
+  box-shadow: none;
+}
+pre {
+  display: block;
+  padding: 8.5px;
+  margin: 0 0 9px;
+  font-size: 12px;
+  line-height: 1.42857143;
+  word-break: break-all;
+  word-wrap: break-word;
+  color: #333333;
+  background-color: #f5f5f5;
+  border: 1px solid #ccc;
+  border-radius: 2px;
+}
+pre code {
+  padding: 0;
+  font-size: inherit;
+  color: inherit;
+  white-space: pre-wrap;
+  background-color: transparent;
+  border-radius: 0;
+}
+.pre-scrollable {
+  max-height: 340px;
+  overflow-y: scroll;
+}
+.container {
+  margin-right: auto;
+  margin-left: auto;
+  padding-left: 0px;
+  padding-right: 0px;
+}
+@media (min-width: 768px) {
+  .container {
+    width: 768px;
+  }
+}
+@media (min-width: 992px) {
+  .container {
+    width: 940px;
+  }
+}
+@media (min-width: 1200px) {
+  .container {
+    width: 1140px;
+  }
+}
+.container-fluid {
+  margin-right: auto;
+  margin-left: auto;
+  padding-left: 0px;
+  padding-right: 0px;
+}
+.row {
+  margin-left: 0px;
+  margin-right: 0px;
+}
+.col-xs-1, .col-sm-1, .col-md-1, .col-lg-1, .col-xs-2, .col-sm-2, .col-md-2, .col-lg-2, .col-xs-3, .col-sm-3, .col-md-3, .col-lg-3, .col-xs-4, .col-sm-4, .col-md-4, .col-lg-4, .col-xs-5, .col-sm-5, .col-md-5, .col-lg-5, .col-xs-6, .col-sm-6, .col-md-6, .col-lg-6, .col-xs-7, .col-sm-7, .col-md-7, .col-lg-7, .col-xs-8, .col-sm-8, .col-md-8, .col-lg-8, .col-xs-9, .col-sm-9, .col-md-9, .col-lg-9, .col-xs-10, .col-sm-10, .col-md-10, .col-lg-10, .col-xs-11, .col-sm-11, .col-md-11, .col-lg-11, .col-xs-12, .col-sm-12, .col-md-12, .col-lg-12 {
+  position: relative;
+  min-height: 1px;
+  padding-left: 0px;
+  padding-right: 0px;
+}
+.col-xs-1, .col-xs-2, .col-xs-3, .col-xs-4, .col-xs-5, .col-xs-6, .col-xs-7, .col-xs-8, .col-xs-9, .col-xs-10, .col-xs-11, .col-xs-12 {
+  float: left;
+}
+.col-xs-12 {
+  width: 100%;
+}
+.col-xs-11 {
+  width: 91.66666667%;
+}
+.col-xs-10 {
+  width: 83.33333333%;
+}
+.col-xs-9 {
+  width: 75%;
+}
+.col-xs-8 {
+  width: 66.66666667%;
+}
+.col-xs-7 {
+  width: 58.33333333%;
+}
+.col-xs-6 {
+  width: 50%;
+}
+.col-xs-5 {
+  width: 41.66666667%;
+}
+.col-xs-4 {
+  width: 33.33333333%;
+}
+.col-xs-3 {
+  width: 25%;
+}
+.col-xs-2 {
+  width: 16.66666667%;
+}
+.col-xs-1 {
+  width: 8.33333333%;
+}
+.col-xs-pull-12 {
+  right: 100%;
+}
+.col-xs-pull-11 {
+  right: 91.66666667%;
+}
+.col-xs-pull-10 {
+  right: 83.33333333%;
+}
+.col-xs-pull-9 {
+  right: 75%;
+}
+.col-xs-pull-8 {
+  right: 66.66666667%;
+}
+.col-xs-pull-7 {
+  right: 58.33333333%;
+}
+.col-xs-pull-6 {
+  right: 50%;
+}
+.col-xs-pull-5 {
+  right: 41.66666667%;
+}
+.col-xs-pull-4 {
+  right: 33.33333333%;
+}
+.col-xs-pull-3 {
+  right: 25%;
+}
+.col-xs-pull-2 {
+  right: 16.66666667%;
+}
+.col-xs-pull-1 {
+  right: 8.33333333%;
+}
+.col-xs-pull-0 {
+  right: auto;
+}
+.col-xs-push-12 {
+  left: 100%;
+}
+.col-xs-push-11 {
+  left: 91.66666667%;
+}
+.col-xs-push-10 {
+  left: 83.33333333%;
+}
+.col-xs-push-9 {
+  left: 75%;
+}
+.col-xs-push-8 {
+  left: 66.66666667%;
+}
+.col-xs-push-7 {
+  left: 58.33333333%;
+}
+.col-xs-push-6 {
+  left: 50%;
+}
+.col-xs-push-5 {
+  left: 41.66666667%;
+}
+.col-xs-push-4 {
+  left: 33.33333333%;
+}
+.col-xs-push-3 {
+  left: 25%;
+}
+.col-xs-push-2 {
+  left: 16.66666667%;
+}
+.col-xs-push-1 {
+  left: 8.33333333%;
+}
+.col-xs-push-0 {
+  left: auto;
+}
+.col-xs-offset-12 {
+  margin-left: 100%;
+}
+.col-xs-offset-11 {
+  margin-left: 91.66666667%;
+}
+.col-xs-offset-10 {
+  margin-left: 83.33333333%;
+}
+.col-xs-offset-9 {
+  margin-left: 75%;
+}
+.col-xs-offset-8 {
+  margin-left: 66.66666667%;
+}
+.col-xs-offset-7 {
+  margin-left: 58.33333333%;
+}
+.col-xs-offset-6 {
+  margin-left: 50%;
+}
+.col-xs-offset-5 {
+  margin-left: 41.66666667%;
+}
+.col-xs-offset-4 {
+  margin-left: 33.33333333%;
+}
+.col-xs-offset-3 {
+  margin-left: 25%;
+}
+.col-xs-offset-2 {
+  margin-left: 16.66666667%;
+}
+.col-xs-offset-1 {
+  margin-left: 8.33333333%;
+}
+.col-xs-offset-0 {
+  margin-left: 0%;
+}
+@media (min-width: 768px) {
+  .col-sm-1, .col-sm-2, .col-sm-3, .col-sm-4, .col-sm-5, .col-sm-6, .col-sm-7, .col-sm-8, .col-sm-9, .col-sm-10, .col-sm-11, .col-sm-12 {
+    float: left;
+  }
+  .col-sm-12 {
+    width: 100%;
+  }
+  .col-sm-11 {
+    width: 91.66666667%;
+  }
+  .col-sm-10 {
+    width: 83.33333333%;
+  }
+  .col-sm-9 {
+    width: 75%;
+  }
+  .col-sm-8 {
+    width: 66.66666667%;
+  }
+  .col-sm-7 {
+    width: 58.33333333%;
+  }
+  .col-sm-6 {
+    width: 50%;
+  }
+  .col-sm-5 {
+    width: 41.66666667%;
+  }
+  .col-sm-4 {
+    width: 33.33333333%;
+  }
+  .col-sm-3 {
+    width: 25%;
+  }
+  .col-sm-2 {
+    width: 16.66666667%;
+  }
+  .col-sm-1 {
+    width: 8.33333333%;
+  }
+  .col-sm-pull-12 {
+    right: 100%;
+  }
+  .col-sm-pull-11 {
+    right: 91.66666667%;
+  }
+  .col-sm-pull-10 {
+    right: 83.33333333%;
+  }
+  .col-sm-pull-9 {
+    right: 75%;
+  }
+  .col-sm-pull-8 {
+    right: 66.66666667%;
+  }
+  .col-sm-pull-7 {
+    right: 58.33333333%;
+  }
+  .col-sm-pull-6 {
+    right: 50%;
+  }
+  .col-sm-pull-5 {
+    right: 41.66666667%;
+  }
+  .col-sm-pull-4 {
+    right: 33.33333333%;
+  }
+  .col-sm-pull-3 {
+    right: 25%;
+  }
+  .col-sm-pull-2 {
+    right: 16.66666667%;
+  }
+  .col-sm-pull-1 {
+    right: 8.33333333%;
+  }
+  .col-sm-pull-0 {
+    right: auto;
+  }
+  .col-sm-push-12 {
+    left: 100%;
+  }
+  .col-sm-push-11 {
+    left: 91.66666667%;
+  }
+  .col-sm-push-10 {
+    left: 83.33333333%;
+  }
+  .col-sm-push-9 {
+    left: 75%;
+  }
+  .col-sm-push-8 {
+    left: 66.66666667%;
+  }
+  .col-sm-push-7 {
+    left: 58.33333333%;
+  }
+  .col-sm-push-6 {
+    left: 50%;
+  }
+  .col-sm-push-5 {
+    left: 41.66666667%;
+  }
+  .col-sm-push-4 {
+    left: 33.33333333%;
+  }
+  .col-sm-push-3 {
+    left: 25%;
+  }
+  .col-sm-push-2 {
+    left: 16.66666667%;
+  }
+  .col-sm-push-1 {
+    left: 8.33333333%;
+  }
+  .col-sm-push-0 {
+    left: auto;
+  }
+  .col-sm-offset-12 {
+    margin-left: 100%;
+  }
+  .col-sm-offset-11 {
+    margin-left: 91.66666667%;
+  }
+  .col-sm-offset-10 {
+    margin-left: 83.33333333%;
+  }
+  .col-sm-offset-9 {
+    margin-left: 75%;
+  }
+  .col-sm-offset-8 {
+    margin-left: 66.66666667%;
+  }
+  .col-sm-offset-7 {
+    margin-left: 58.33333333%;
+  }
+  .col-sm-offset-6 {
+    margin-left: 50%;
+  }
+  .col-sm-offset-5 {
+    margin-left: 41.66666667%;
+  }
+  .col-sm-offset-4 {
+    margin-left: 33.33333333%;
+  }
+  .col-sm-offset-3 {
+    margin-left: 25%;
+  }
+  .col-sm-offset-2 {
+    margin-left: 16.66666667%;
+  }
+  .col-sm-offset-1 {
+    margin-left: 8.33333333%;
+  }
+  .col-sm-offset-0 {
+    margin-left: 0%;
+  }
+}
+@media (min-width: 992px) {
+  .col-md-1, .col-md-2, .col-md-3, .col-md-4, .col-md-5, .col-md-6, .col-md-7, .col-md-8, .col-md-9, .col-md-10, .col-md-11, .col-md-12 {
+    float: left;
+  }
+  .col-md-12 {
+    width: 100%;
+  }
+  .col-md-11 {
+    width: 91.66666667%;
+  }
+  .col-md-10 {
+    width: 83.33333333%;
+  }
+  .col-md-9 {
+    width: 75%;
+  }
+  .col-md-8 {
+    width: 66.66666667%;
+  }
+  .col-md-7 {
+    width: 58.33333333%;
+  }
+  .col-md-6 {
+    width: 50%;
+  }
+  .col-md-5 {
+    width: 41.66666667%;
+  }
+  .col-md-4 {
+    width: 33.33333333%;
+  }
+  .col-md-3 {
+    width: 25%;
+  }
+  .col-md-2 {
+    width: 16.66666667%;
+  }
+  .col-md-1 {
+    width: 8.33333333%;
+  }
+  .col-md-pull-12 {
+    right: 100%;
+  }
+  .col-md-pull-11 {
+    right: 91.66666667%;
+  }
+  .col-md-pull-10 {
+    right: 83.33333333%;
+  }
+  .col-md-pull-9 {
+    right: 75%;
+  }
+  .col-md-pull-8 {
+    right: 66.66666667%;
+  }
+  .col-md-pull-7 {
+    right: 58.33333333%;
+  }
+  .col-md-pull-6 {
+    right: 50%;
+  }
+  .col-md-pull-5 {
+    right: 41.66666667%;
+  }
+  .col-md-pull-4 {
+    right: 33.33333333%;
+  }
+  .col-md-pull-3 {
+    right: 25%;
+  }
+  .col-md-pull-2 {
+    right: 16.66666667%;
+  }
+  .col-md-pull-1 {
+    right: 8.33333333%;
+  }
+  .col-md-pull-0 {
+    right: auto;
+  }
+  .col-md-push-12 {
+    left: 100%;
+  }
+  .col-md-push-11 {
+    left: 91.66666667%;
+  }
+  .col-md-push-10 {
+    left: 83.33333333%;
+  }
+  .col-md-push-9 {
+    left: 75%;
+  }
+  .col-md-push-8 {
+    left: 66.66666667%;
+  }
+  .col-md-push-7 {
+    left: 58.33333333%;
+  }
+  .col-md-push-6 {
+    left: 50%;
+  }
+  .col-md-push-5 {
+    left: 41.66666667%;
+  }
+  .col-md-push-4 {
+    left: 33.33333333%;
+  }
+  .col-md-push-3 {
+    left: 25%;
+  }
+  .col-md-push-2 {
+    left: 16.66666667%;
+  }
+  .col-md-push-1 {
+    left: 8.33333333%;
+  }
+  .col-md-push-0 {
+    left: auto;
+  }
+  .col-md-offset-12 {
+    margin-left: 100%;
+  }
+  .col-md-offset-11 {
+    margin-left: 91.66666667%;
+  }
+  .col-md-offset-10 {
+    margin-left: 83.33333333%;
+  }
+  .col-md-offset-9 {
+    margin-left: 75%;
+  }
+  .col-md-offset-8 {
+    margin-left: 66.66666667%;
+  }
+  .col-md-offset-7 {
+    margin-left: 58.33333333%;
+  }
+  .col-md-offset-6 {
+    margin-left: 50%;
+  }
+  .col-md-offset-5 {
+    margin-left: 41.66666667%;
+  }
+  .col-md-offset-4 {
+    margin-left: 33.33333333%;
+  }
+  .col-md-offset-3 {
+    margin-left: 25%;
+  }
+  .col-md-offset-2 {
+    margin-left: 16.66666667%;
+  }
+  .col-md-offset-1 {
+    margin-left: 8.33333333%;
+  }
+  .col-md-offset-0 {
+    margin-left: 0%;
+  }
+}
+@media (min-width: 1200px) {
+  .col-lg-1, .col-lg-2, .col-lg-3, .col-lg-4, .col-lg-5, .col-lg-6, .col-lg-7, .col-lg-8, .col-lg-9, .col-lg-10, .col-lg-11, .col-lg-12 {
+    float: left;
+  }
+  .col-lg-12 {
+    width: 100%;
+  }
+  .col-lg-11 {
+    width: 91.66666667%;
+  }
+  .col-lg-10 {
+    width: 83.33333333%;
+  }
+  .col-lg-9 {
+    width: 75%;
+  }
+  .col-lg-8 {
+    width: 66.66666667%;
+  }
+  .col-lg-7 {
+    width: 58.33333333%;
+  }
+  .col-lg-6 {
+    width: 50%;
+  }
+  .col-lg-5 {
+    width: 41.66666667%;
+  }
+  .col-lg-4 {
+    width: 33.33333333%;
+  }
+  .col-lg-3 {
+    width: 25%;
+  }
+  .col-lg-2 {
+    width: 16.66666667%;
+  }
+  .col-lg-1 {
+    width: 8.33333333%;
+  }
+  .col-lg-pull-12 {
+    right: 100%;
+  }
+  .col-lg-pull-11 {
+    right: 91.66666667%;
+  }
+  .col-lg-pull-10 {
+    right: 83.33333333%;
+  }
+  .col-lg-pull-9 {
+    right: 75%;
+  }
+  .col-lg-pull-8 {
+    right: 66.66666667%;
+  }
+  .col-lg-pull-7 {
+    right: 58.33333333%;
+  }
+  .col-lg-pull-6 {
+    right: 50%;
+  }
+  .col-lg-pull-5 {
+    right: 41.66666667%;
+  }
+  .col-lg-pull-4 {
+    right: 33.33333333%;
+  }
+  .col-lg-pull-3 {
+    right: 25%;
+  }
+  .col-lg-pull-2 {
+    right: 16.66666667%;
+  }
+  .col-lg-pull-1 {
+    right: 8.33333333%;
+  }
+  .col-lg-pull-0 {
+    right: auto;
+  }
+  .col-lg-push-12 {
+    left: 100%;
+  }
+  .col-lg-push-11 {
+    left: 91.66666667%;
+  }
+  .col-lg-push-10 {
+    left: 83.33333333%;
+  }
+  .col-lg-push-9 {
+    left: 75%;
+  }
+  .col-lg-push-8 {
+    left: 66.66666667%;
+  }
+  .col-lg-push-7 {
+    left: 58.33333333%;
+  }
+  .col-lg-push-6 {
+    left: 50%;
+  }
+  .col-lg-push-5 {
+    left: 41.66666667%;
+  }
+  .col-lg-push-4 {
+    left: 33.33333333%;
+  }
+  .col-lg-push-3 {
+    left: 25%;
+  }
+  .col-lg-push-2 {
+    left: 16.66666667%;
+  }
+  .col-lg-push-1 {
+    left: 8.33333333%;
+  }
+  .col-lg-push-0 {
+    left: auto;
+  }
+  .col-lg-offset-12 {
+    margin-left: 100%;
+  }
+  .col-lg-offset-11 {
+    margin-left: 91.66666667%;
+  }
+  .col-lg-offset-10 {
+    margin-left: 83.33333333%;
+  }
+  .col-lg-offset-9 {
+    margin-left: 75%;
+  }
+  .col-lg-offset-8 {
+    margin-left: 66.66666667%;
+  }
+  .col-lg-offset-7 {
+    margin-left: 58.33333333%;
+  }
+  .col-lg-offset-6 {
+    margin-left: 50%;
+  }
+  .col-lg-offset-5 {
+    margin-left: 41.66666667%;
+  }
+  .col-lg-offset-4 {
+    margin-left: 33.33333333%;
+  }
+  .col-lg-offset-3 {
+    margin-left: 25%;
+  }
+  .col-lg-offset-2 {
+    margin-left: 16.66666667%;
+  }
+  .col-lg-offset-1 {
+    margin-left: 8.33333333%;
+  }
+  .col-lg-offset-0 {
+    margin-left: 0%;
+  }
+}
+table {
+  background-color: transparent;
+}
+caption {
+  padding-top: 8px;
+  padding-bottom: 8px;
+  color: #777777;
+  text-align: left;
+}
+th {
+  text-align: left;
+}
+.table {
+  width: 100%;
+  max-width: 100%;
+  margin-bottom: 18px;
+}
+.table > thead > tr > th,
+.table > tbody > tr > th,
+.table > tfoot > tr > th,
+.table > thead > tr > td,
+.table > tbody > tr > td,
+.table > tfoot > tr > td {
+  padding: 8px;
+  line-height: 1.42857143;
+  vertical-align: top;
+  border-top: 1px solid #ddd;
+}
+.table > thead > tr > th {
+  vertical-align: bottom;
+  border-bottom: 2px solid #ddd;
+}
+.table > caption + thead > tr:first-child > th,
+.table > colgroup + thead > tr:first-child > th,
+.table > thead:first-child > tr:first-child > th,
+.table > caption + thead > tr:first-child > td,
+.table > colgroup + thead > tr:first-child > td,
+.table > thead:first-child > tr:first-child > td {
+  border-top: 0;
+}
+.table > tbody + tbody {
+  border-top: 2px solid #ddd;
+}
+.table .table {
+  background-color: #fff;
+}
+.table-condensed > thead > tr > th,
+.table-condensed > tbody > tr > th,
+.table-condensed > tfoot > tr > th,
+.table-condensed > thead > tr > td,
+.table-condensed > tbody > tr > td,
+.table-condensed > tfoot > tr > td {
+  padding: 5px;
+}
+.table-bordered {
+  border: 1px solid #ddd;
+}
+.table-bordered > thead > tr > th,
+.table-bordered > tbody > tr > th,
+.table-bordered > tfoot > tr > th,
+.table-bordered > thead > tr > td,
+.table-bordered > tbody > tr > td,
+.table-bordered > tfoot > tr > td {
+  border: 1px solid #ddd;
+}
+.table-bordered > thead > tr > th,
+.table-bordered > thead > tr > td {
+  border-bottom-width: 2px;
+}
+.table-striped > tbody > tr:nth-of-type(odd) {
+  background-color: #f9f9f9;
+}
+.table-hover > tbody > tr:hover {
+  background-color: #f5f5f5;
+}
+table col[class*="col-"] {
+  position: static;
+  float: none;
+  display: table-column;
+}
+table td[class*="col-"],
+table th[class*="col-"] {
+  position: static;
+  float: none;
+  display: table-cell;
+}
+.table > thead > tr > td.active,
+.table > tbody > tr > td.active,
+.table > tfoot > tr > td.active,
+.table > thead > tr > th.active,
+.table > tbody > tr > th.active,
+.table > tfoot > tr > th.active,
+.table > thead > tr.active > td,
+.table > tbody > tr.active > td,
+.table > tfoot > tr.active > td,
+.table > thead > tr.active > th,
+.table > tbody > tr.active > th,
+.table > tfoot > tr.active > th {
+  background-color: #f5f5f5;
+}
+.table-hover > tbody > tr > td.active:hover,
+.table-hover > tbody > tr > th.active:hover,
+.table-hover > tbody > tr.active:hover > td,
+.table-hover > tbody > tr:hover > .active,
+.table-hover > tbody > tr.active:hover > th {
+  background-color: #e8e8e8;
+}
+.table > thead > tr > td.success,
+.table > tbody > tr > td.success,
+.table > tfoot > tr > td.success,
+.table > thead > tr > th.success,
+.table > tbody > tr > th.success,
+.table > tfoot > tr > th.success,
+.table > thead > tr.success > td,
+.table > tbody > tr.success > td,
+.table > tfoot > tr.success > td,
+.table > thead > tr.success > th,
+.table > tbody > tr.success > th,
+.table > tfoot > tr.success > th {
+  background-color: #dff0d8;
+}
+.table-hover > tbody > tr > td.success:hover,
+.table-hover > tbody > tr > th.success:hover,
+.table-hover > tbody > tr.success:hover > td,
+.table-hover > tbody > tr:hover > .success,
+.table-hover > tbody > tr.success:hover > th {
+  background-color: #d0e9c6;
+}
+.table > thead > tr > td.info,
+.table > tbody > tr > td.info,
+.table > tfoot > tr > td.info,
+.table > thead > tr > th.info,
+.table > tbody > tr > th.info,
+.table > tfoot > tr > th.info,
+.table > thead > tr.info > td,
+.table > tbody > tr.info > td,
+.table > tfoot > tr.info > td,
+.table > thead > tr.info > th,
+.table > tbody > tr.info > th,
+.table > tfoot > tr.info > th {
+  background-color: #d9edf7;
+}
+.table-hover > tbody > tr > td.info:hover,
+.table-hover > tbody > tr > th.info:hover,
+.table-hover > tbody > tr.info:hover > td,
+.table-hover > tbody > tr:hover > .info,
+.table-hover > tbody > tr.info:hover > th {
+  background-color: #c4e3f3;
+}
+.table > thead > tr > td.warning,
+.table > tbody > tr > td.warning,
+.table > tfoot > tr > td.warning,
+.table > thead > tr > th.warning,
+.table > tbody > tr > th.warning,
+.table > tfoot > tr > th.warning,
+.table > thead > tr.warning > td,
+.table > tbody > tr.warning > td,
+.table > tfoot > tr.warning > td,
+.table > thead > tr.warning > th,
+.table > tbody > tr.warning > th,
+.table > tfoot > tr.warning > th {
+  background-color: #fcf8e3;
+}
+.table-hover > tbody > tr > td.warning:hover,
+.table-hover > tbody > tr > th.warning:hover,
+.table-hover > tbody > tr.warning:hover > td,
+.table-hover > tbody > tr:hover > .warning,
+.table-hover > tbody > tr.warning:hover > th {
+  background-color: #faf2cc;
+}
+.table > thead > tr > td.danger,
+.table > tbody > tr > td.danger,
+.table > tfoot > tr > td.danger,
+.table > thead > tr > th.danger,
+.table > tbody > tr > th.danger,
+.table > tfoot > tr > th.danger,
+.table > thead > tr.danger > td,
+.table > tbody > tr.danger > td,
+.table > tfoot > tr.danger > td,
+.table > thead > tr.danger > th,
+.table > tbody > tr.danger > th,
+.table > tfoot > tr.danger > th {
+  background-color: #f2dede;
+}
+.table-hover > tbody > tr > td.danger:hover,
+.table-hover > tbody > tr > th.danger:hover,
+.table-hover > tbody > tr.danger:hover > td,
+.table-hover > tbody > tr:hover > .danger,
+.table-hover > tbody > tr.danger:hover > th {
+  background-color: #ebcccc;
+}
+.table-responsive {
+  overflow-x: auto;
+  min-height: 0.01%;
+}
+@media screen and (max-width: 767px) {
+  .table-responsive {
+    width: 100%;
+    margin-bottom: 13.5px;
+    overflow-y: hidden;
+    -ms-overflow-style: -ms-autohiding-scrollbar;
+    border: 1px solid #ddd;
+  }
+  .table-responsive > .table {
+    margin-bottom: 0;
+  }
+  .table-responsive > .table > thead > tr > th,
+  .table-responsive > .table > tbody > tr > th,
+  .table-responsive > .table > tfoot > tr > th,
+  .table-responsive > .table > thead > tr > td,
+  .table-responsive > .table > tbody > tr > td,
+  .table-responsive > .table > tfoot > tr > td {
+    white-space: nowrap;
+  }
+  .table-responsive > .table-bordered {
+    border: 0;
+  }
+  .table-responsive > .table-bordered > thead > tr > th:first-child,
+  .table-responsive > .table-bordered > tbody > tr > th:first-child,
+  .table-responsive > .table-bordered > tfoot > tr > th:first-child,
+  .table-responsive > .table-bordered > thead > tr > td:first-child,
+  .table-responsive > .table-bordered > tbody > tr > td:first-child,
+  .table-responsive > .table-bordered > tfoot > tr > td:first-child {
+    border-left: 0;
+  }
+  .table-responsive > .table-bordered > thead > tr > th:last-child,
+  .table-responsive > .table-bordered > tbody > tr > th:last-child,
+  .table-responsive > .table-bordered > tfoot > tr > th:last-child,
+  .table-responsive > .table-bordered > thead > tr > td:last-child,
+  .table-responsive > .table-bordered > tbody > tr > td:last-child,
+  .table-responsive > .table-bordered > tfoot > tr > td:last-child {
+    border-right: 0;
+  }
+  .table-responsive > .table-bordered > tbody > tr:last-child > th,
+  .table-responsive > .table-bordered > tfoot > tr:last-child > th,
+  .table-responsive > .table-bordered > tbody > tr:last-child > td,
+  .table-responsive > .table-bordered > tfoot > tr:last-child > td {
+    border-bottom: 0;
+  }
+}
+fieldset {
+  padding: 0;
+  margin: 0;
+  border: 0;
+  min-width: 0;
+}
+legend {
+  display: block;
+  width: 100%;
+  padding: 0;
+  margin-bottom: 18px;
+  font-size: 19.5px;
+  line-height: inherit;
+  color: #333333;
+  border: 0;
+  border-bottom: 1px solid #e5e5e5;
+}
+label {
+  display: inline-block;
+  max-width: 100%;
+  margin-bottom: 5px;
+  font-weight: bold;
+}
+input[type="search"] {
+  -webkit-box-sizing: border-box;
+  -moz-box-sizing: border-box;
+  box-sizing: border-box;
+}
+input[type="radio"],
+input[type="checkbox"] {
+  margin: 4px 0 0;
+  margin-top: 1px \9;
+  line-height: normal;
+}
+input[type="file"] {
+  display: block;
+}
+input[type="range"] {
+  display: block;
+  width: 100%;
+}
+select[multiple],
+select[size] {
+  height: auto;
+}
+input[type="file"]:focus,
+input[type="radio"]:focus,
+input[type="checkbox"]:focus {
+  outline: thin dotted;
+  outline: 5px auto -webkit-focus-ring-color;
+  outline-offset: -2px;
+}
+output {
+  display: block;
+  padding-top: 7px;
+  font-size: 13px;
+  line-height: 1.42857143;
+  color: #555555;
+}
+.form-control {
+  display: block;
+  width: 100%;
+  height: 32px;
+  padding: 6px 12px;
+  font-size: 13px;
+  line-height: 1.42857143;
+  color: #555555;
+  background-color: #fff;
+  background-image: none;
+  border: 1px solid #ccc;
+  border-radius: 2px;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+  -webkit-transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
+  -o-transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
+  transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
+}
+.form-control:focus {
+  border-color: #66afe9;
+  outline: 0;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0,0,0,.075), 0 0 8px rgba(102, 175, 233, 0.6);
+  box-shadow: inset 0 1px 1px rgba(0,0,0,.075), 0 0 8px rgba(102, 175, 233, 0.6);
+}
+.form-control::-moz-placeholder {
+  color: #999;
+  opacity: 1;
+}
+.form-control:-ms-input-placeholder {
+  color: #999;
+}
+.form-control::-webkit-input-placeholder {
+  color: #999;
+}
+.form-control::-ms-expand {
+  border: 0;
+  background-color: transparent;
+}
+.form-control[disabled],
+.form-control[readonly],
+fieldset[disabled] .form-control {
+  background-color: #eeeeee;
+  opacity: 1;
+}
+.form-control[disabled],
+fieldset[disabled] .form-control {
+  cursor: not-allowed;
+}
+textarea.form-control {
+  height: auto;
+}
+input[type="search"] {
+  -webkit-appearance: none;
+}
+@media screen and (-webkit-min-device-pixel-ratio: 0) {
+  input[type="date"].form-control,
+  input[type="time"].form-control,
+  input[type="datetime-local"].form-control,
+  input[type="month"].form-control {
+    line-height: 32px;
+  }
+  input[type="date"].input-sm,
+  input[type="time"].input-sm,
+  input[type="datetime-local"].input-sm,
+  input[type="month"].input-sm,
+  .input-group-sm input[type="date"],
+  .input-group-sm input[type="time"],
+  .input-group-sm input[type="datetime-local"],
+  .input-group-sm input[type="month"] {
+    line-height: 30px;
+  }
+  input[type="date"].input-lg,
+  input[type="time"].input-lg,
+  input[type="datetime-local"].input-lg,
+  input[type="month"].input-lg,
+  .input-group-lg input[type="date"],
+  .input-group-lg input[type="time"],
+  .input-group-lg input[type="datetime-local"],
+  .input-group-lg input[type="month"] {
+    line-height: 45px;
+  }
+}
+.form-group {
+  margin-bottom: 15px;
+}
+.radio,
+.checkbox {
+  position: relative;
+  display: block;
+  margin-top: 10px;
+  margin-bottom: 10px;
+}
+.radio label,
+.checkbox label {
+  min-height: 18px;
+  padding-left: 20px;
+  margin-bottom: 0;
+  font-weight: normal;
+  cursor: pointer;
+}
+.radio input[type="radio"],
+.radio-inline input[type="radio"],
+.checkbox input[type="checkbox"],
+.checkbox-inline input[type="checkbox"] {
+  position: absolute;
+  margin-left: -20px;
+  margin-top: 4px \9;
+}
+.radio + .radio,
+.checkbox + .checkbox {
+  margin-top: -5px;
+}
+.radio-inline,
+.checkbox-inline {
+  position: relative;
+  display: inline-block;
+  padding-left: 20px;
+  margin-bottom: 0;
+  vertical-align: middle;
+  font-weight: normal;
+  cursor: pointer;
+}
+.radio-inline + .radio-inline,
+.checkbox-inline + .checkbox-inline {
+  margin-top: 0;
+  margin-left: 10px;
+}
+input[type="radio"][disabled],
+input[type="checkbox"][disabled],
+input[type="radio"].disabled,
+input[type="checkbox"].disabled,
+fieldset[disabled] input[type="radio"],
+fieldset[disabled] input[type="checkbox"] {
+  cursor: not-allowed;
+}
+.radio-inline.disabled,
+.checkbox-inline.disabled,
+fieldset[disabled] .radio-inline,
+fieldset[disabled] .checkbox-inline {
+  cursor: not-allowed;
+}
+.radio.disabled label,
+.checkbox.disabled label,
+fieldset[disabled] .radio label,
+fieldset[disabled] .checkbox label {
+  cursor: not-allowed;
+}
+.form-control-static {
+  padding-top: 7px;
+  padding-bottom: 7px;
+  margin-bottom: 0;
+  min-height: 31px;
+}
+.form-control-static.input-lg,
+.form-control-static.input-sm {
+  padding-left: 0;
+  padding-right: 0;
+}
+.input-sm {
+  height: 30px;
+  padding: 5px 10px;
+  font-size: 12px;
+  line-height: 1.5;
+  border-radius: 1px;
+}
+select.input-sm {
+  height: 30px;
+  line-height: 30px;
+}
+textarea.input-sm,
+select[multiple].input-sm {
+  height: auto;
+}
+.form-group-sm .form-control {
+  height: 30px;
+  padding: 5px 10px;
+  font-size: 12px;
+  line-height: 1.5;
+  border-radius: 1px;
+}
+.form-group-sm select.form-control {
+  height: 30px;
+  line-height: 30px;
+}
+.form-group-sm textarea.form-control,
+.form-group-sm select[multiple].form-control {
+  height: auto;
+}
+.form-group-sm .form-control-static {
+  height: 30px;
+  min-height: 30px;
+  padding: 6px 10px;
+  font-size: 12px;
+  line-height: 1.5;
+}
+.input-lg {
+  height: 45px;
+  padding: 10px 16px;
+  font-size: 17px;
+  line-height: 1.3333333;
+  border-radius: 3px;
+}
+select.input-lg {
+  height: 45px;
+  line-height: 45px;
+}
+textarea.input-lg,
+select[multiple].input-lg {
+  height: auto;
+}
+.form-group-lg .form-control {
+  height: 45px;
+  padding: 10px 16px;
+  font-size: 17px;
+  line-height: 1.3333333;
+  border-radius: 3px;
+}
+.form-group-lg select.form-control {
+  height: 45px;
+  line-height: 45px;
+}
+.form-group-lg textarea.form-control,
+.form-group-lg select[multiple].form-control {
+  height: auto;
+}
+.form-group-lg .form-control-static {
+  height: 45px;
+  min-height: 35px;
+  padding: 11px 16px;
+  font-size: 17px;
+  line-height: 1.3333333;
+}
+.has-feedback {
+  position: relative;
+}
+.has-feedback .form-control {
+  padding-right: 40px;
+}
+.form-control-feedback {
+  position: absolute;
+  top: 0;
+  right: 0;
+  z-index: 2;
+  display: block;
+  width: 32px;
+  height: 32px;
+  line-height: 32px;
+  text-align: center;
+  pointer-events: none;
+}
+.input-lg + .form-control-feedback,
+.input-group-lg + .form-control-feedback,
+.form-group-lg .form-control + .form-control-feedback {
+  width: 45px;
+  height: 45px;
+  line-height: 45px;
+}
+.input-sm + .form-control-feedback,
+.input-group-sm + .form-control-feedback,
+.form-group-sm .form-control + .form-control-feedback {
+  width: 30px;
+  height: 30px;
+  line-height: 30px;
+}
+.has-success .help-block,
+.has-success .control-label,
+.has-success .radio,
+.has-success .checkbox,
+.has-success .radio-inline,
+.has-success .checkbox-inline,
+.has-success.radio label,
+.has-success.checkbox label,
+.has-success.radio-inline label,
+.has-success.checkbox-inline label {
+  color: #3c763d;
+}
+.has-success .form-control {
+  border-color: #3c763d;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+}
+.has-success .form-control:focus {
+  border-color: #2b542c;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #67b168;
+  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #67b168;
+}
+.has-success .input-group-addon {
+  color: #3c763d;
+  border-color: #3c763d;
+  background-color: #dff0d8;
+}
+.has-success .form-control-feedback {
+  color: #3c763d;
+}
+.has-warning .help-block,
+.has-warning .control-label,
+.has-warning .radio,
+.has-warning .checkbox,
+.has-warning .radio-inline,
+.has-warning .checkbox-inline,
+.has-warning.radio label,
+.has-warning.checkbox label,
+.has-warning.radio-inline label,
+.has-warning.checkbox-inline label {
+  color: #8a6d3b;
+}
+.has-warning .form-control {
+  border-color: #8a6d3b;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+}
+.has-warning .form-control:focus {
+  border-color: #66512c;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #c0a16b;
+  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #c0a16b;
+}
+.has-warning .input-group-addon {
+  color: #8a6d3b;
+  border-color: #8a6d3b;
+  background-color: #fcf8e3;
+}
+.has-warning .form-control-feedback {
+  color: #8a6d3b;
+}
+.has-error .help-block,
+.has-error .control-label,
+.has-error .radio,
+.has-error .checkbox,
+.has-error .radio-inline,
+.has-error .checkbox-inline,
+.has-error.radio label,
+.has-error.checkbox label,
+.has-error.radio-inline label,
+.has-error.checkbox-inline label {
+  color: #a94442;
+}
+.has-error .form-control {
+  border-color: #a94442;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+}
+.has-error .form-control:focus {
+  border-color: #843534;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #ce8483;
+  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #ce8483;
+}
+.has-error .input-group-addon {
+  color: #a94442;
+  border-color: #a94442;
+  background-color: #f2dede;
+}
+.has-error .form-control-feedback {
+  color: #a94442;
+}
+.has-feedback label ~ .form-control-feedback {
+  top: 23px;
+}
+.has-feedback label.sr-only ~ .form-control-feedback {
+  top: 0;
+}
+.help-block {
+  display: block;
+  margin-top: 5px;
+  margin-bottom: 10px;
+  color: #404040;
+}
+@media (min-width: 768px) {
+  .form-inline .form-group {
+    display: inline-block;
+    margin-bottom: 0;
+    vertical-align: middle;
+  }
+  .form-inline .form-control {
+    display: inline-block;
+    width: auto;
+    vertical-align: middle;
+  }
+  .form-inline .form-control-static {
+    display: inline-block;
+  }
+  .form-inline .input-group {
+    display: inline-table;
+    vertical-align: middle;
+  }
+  .form-inline .input-group .input-group-addon,
+  .form-inline .input-group .input-group-btn,
+  .form-inline .input-group .form-control {
+    width: auto;
+  }
+  .form-inline .input-group > .form-control {
+    width: 100%;
+  }
+  .form-inline .control-label {
+    margin-bottom: 0;
+    vertical-align: middle;
+  }
+  .form-inline .radio,
+  .form-inline .checkbox {
+    display: inline-block;
+    margin-top: 0;
+    margin-bottom: 0;
+    vertical-align: middle;
+  }
+  .form-inline .radio label,
+  .form-inline .checkbox label {
+    padding-left: 0;
+  }
+  .form-inline .radio input[type="radio"],
+  .form-inline .checkbox input[type="checkbox"] {
+    position: relative;
+    margin-left: 0;
+  }
+  .form-inline .has-feedback .form-control-feedback {
+    top: 0;
+  }
+}
+.form-horizontal .radio,
+.form-horizontal .checkbox,
+.form-horizontal .radio-inline,
+.form-horizontal .checkbox-inline {
+  margin-top: 0;
+  margin-bottom: 0;
+  padding-top: 7px;
+}
+.form-horizontal .radio,
+.form-horizontal .checkbox {
+  min-height: 25px;
+}
+.form-horizontal .form-group {
+  margin-left: 0px;
+  margin-right: 0px;
+}
+@media (min-width: 768px) {
+  .form-horizontal .control-label {
+    text-align: right;
+    margin-bottom: 0;
+    padding-top: 7px;
+  }
+}
+.form-horizontal .has-feedback .form-control-feedback {
+  right: 0px;
+}
+@media (min-width: 768px) {
+  .form-horizontal .form-group-lg .control-label {
+    padding-top: 11px;
+    font-size: 17px;
+  }
+}
+@media (min-width: 768px) {
+  .form-horizontal .form-group-sm .control-label {
+    padding-top: 6px;
+    font-size: 12px;
+  }
+}
+.btn {
+  display: inline-block;
+  margin-bottom: 0;
+  font-weight: normal;
+  text-align: center;
+  vertical-align: middle;
+  touch-action: manipulation;
+  cursor: pointer;
+  background-image: none;
+  border: 1px solid transparent;
+  white-space: nowrap;
+  padding: 6px 12px;
+  font-size: 13px;
+  line-height: 1.42857143;
+  border-radius: 2px;
+  -webkit-user-select: none;
+  -moz-user-select: none;
+  -ms-user-select: none;
+  user-select: none;
+}
+.btn:focus,
+.btn:active:focus,
+.btn.active:focus,
+.btn.focus,
+.btn:active.focus,
+.btn.active.focus {
+  outline: thin dotted;
+  outline: 5px auto -webkit-focus-ring-color;
+  outline-offset: -2px;
+}
+.btn:hover,
+.btn:focus,
+.btn.focus {
+  color: #333;
+  text-decoration: none;
+}
+.btn:active,
+.btn.active {
+  outline: 0;
+  background-image: none;
+  -webkit-box-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);
+  box-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);
+}
+.btn.disabled,
+.btn[disabled],
+fieldset[disabled] .btn {
+  cursor: not-allowed;
+  opacity: 0.65;
+  filter: alpha(opacity=65);
+  -webkit-box-shadow: none;
+  box-shadow: none;
+}
+a.btn.disabled,
+fieldset[disabled] a.btn {
+  pointer-events: none;
+}
+.btn-default {
+  color: #333;
+  background-color: #fff;
+  border-color: #ccc;
+}
+.btn-default:focus,
+.btn-default.focus {
+  color: #333;
+  background-color: #e6e6e6;
+  border-color: #8c8c8c;
+}
+.btn-default:hover {
+  color: #333;
+  background-color: #e6e6e6;
+  border-color: #adadad;
+}
+.btn-default:active,
+.btn-default.active,
+.open > .dropdown-toggle.btn-default {
+  color: #333;
+  background-color: #e6e6e6;
+  border-color: #adadad;
+}
+.btn-default:active:hover,
+.btn-default.active:hover,
+.open > .dropdown-toggle.btn-default:hover,
+.btn-default:active:focus,
+.btn-default.active:focus,
+.open > .dropdown-toggle.btn-default:focus,
+.btn-default:active.focus,
+.btn-default.active.focus,
+.open > .dropdown-toggle.btn-default.focus {
+  color: #333;
+  background-color: #d4d4d4;
+  border-color: #8c8c8c;
+}
+.btn-default:active,
+.btn-default.active,
+.open > .dropdown-toggle.btn-default {
+  background-image: none;
+}
+.btn-default.disabled:hover,
+.btn-default[disabled]:hover,
+fieldset[disabled] .btn-default:hover,
+.btn-default.disabled:focus,
+.btn-default[disabled]:focus,
+fieldset[disabled] .btn-default:focus,
+.btn-default.disabled.focus,
+.btn-default[disabled].focus,
+fieldset[disabled] .btn-default.focus {
+  background-color: #fff;
+  border-color: #ccc;
+}
+.btn-default .badge {
+  color: #fff;
+  background-color: #333;
+}
+.btn-primary {
+  color: #fff;
+  background-color: #337ab7;
+  border-color: #2e6da4;
+}
+.btn-primary:focus,
+.btn-primary.focus {
+  color: #fff;
+  background-color: #286090;
+  border-color: #122b40;
+}
+.btn-primary:hover {
+  color: #fff;
+  background-color: #286090;
+  border-color: #204d74;
+}
+.btn-primary:active,
+.btn-primary.active,
+.open > .dropdown-toggle.btn-primary {
+  color: #fff;
+  background-color: #286090;
+  border-color: #204d74;
+}
+.btn-primary:active:hover,
+.btn-primary.active:hover,
+.open > .dropdown-toggle.btn-primary:hover,
+.btn-primary:active:focus,
+.btn-primary.active:focus,
+.open > .dropdown-toggle.btn-primary:focus,
+.btn-primary:active.focus,
+.btn-primary.active.focus,
+.open > .dropdown-toggle.btn-primary.focus {
+  color: #fff;
+  background-color: #204d74;
+  border-color: #122b40;
+}
+.btn-primary:active,
+.btn-primary.active,
+.open > .dropdown-toggle.btn-primary {
+  background-image: none;
+}
+.btn-primary.disabled:hover,
+.btn-primary[disabled]:hover,
+fieldset[disabled] .btn-primary:hover,
+.btn-primary.disabled:focus,
+.btn-primary[disabled]:focus,
+fieldset[disabled] .btn-primary:focus,
+.btn-primary.disabled.focus,
+.btn-primary[disabled].focus,
+fieldset[disabled] .btn-primary.focus {
+  background-color: #337ab7;
+  border-color: #2e6da4;
+}
+.btn-primary .badge {
+  color: #337ab7;
+  background-color: #fff;
+}
+.btn-success {
+  color: #fff;
+  background-color: #5cb85c;
+  border-color: #4cae4c;
+}
+.btn-success:focus,
+.btn-success.focus {
+  color: #fff;
+  background-color: #449d44;
+  border-color: #255625;
+}
+.btn-success:hover {
+  color: #fff;
+  background-color: #449d44;
+  border-color: #398439;
+}
+.btn-success:active,
+.btn-success.active,
+.open > .dropdown-toggle.btn-success {
+  color: #fff;
+  background-color: #449d44;
+  border-color: #398439;
+}
+.btn-success:active:hover,
+.btn-success.active:hover,
+.open > .dropdown-toggle.btn-success:hover,
+.btn-success:active:focus,
+.btn-success.active:focus,
+.open > .dropdown-toggle.btn-success:focus,
+.btn-success:active.focus,
+.btn-success.active.focus,
+.open > .dropdown-toggle.btn-success.focus {
+  color: #fff;
+  background-color: #398439;
+  border-color: #255625;
+}
+.btn-success:active,
+.btn-success.active,
+.open > .dropdown-toggle.btn-success {
+  background-image: none;
+}
+.btn-success.disabled:hover,
+.btn-success[disabled]:hover,
+fieldset[disabled] .btn-success:hover,
+.btn-success.disabled:focus,
+.btn-success[disabled]:focus,
+fieldset[disabled] .btn-success:focus,
+.btn-success.disabled.focus,
+.btn-success[disabled].focus,
+fieldset[disabled] .btn-success.focus {
+  background-color: #5cb85c;
+  border-color: #4cae4c;
+}
+.btn-success .badge {
+  color: #5cb85c;
+  background-color: #fff;
+}
+.btn-info {
+  color: #fff;
+  background-color: #5bc0de;
+  border-color: #46b8da;
+}
+.btn-info:focus,
+.btn-info.focus {
+  color: #fff;
+  background-color: #31b0d5;
+  border-color: #1b6d85;
+}
+.btn-info:hover {
+  color: #fff;
+  background-color: #31b0d5;
+  border-color: #269abc;
+}
+.btn-info:active,
+.btn-info.active,
+.open > .dropdown-toggle.btn-info {
+  color: #fff;
+  background-color: #31b0d5;
+  border-color: #269abc;
+}
+.btn-info:active:hover,
+.btn-info.active:hover,
+.open > .dropdown-toggle.btn-info:hover,
+.btn-info:active:focus,
+.btn-info.active:focus,
+.open > .dropdown-toggle.btn-info:focus,
+.btn-info:active.focus,
+.btn-info.active.focus,
+.open > .dropdown-toggle.btn-info.focus {
+  color: #fff;
+  background-color: #269abc;
+  border-color: #1b6d85;
+}
+.btn-info:active,
+.btn-info.active,
+.open > .dropdown-toggle.btn-info {
+  background-image: none;
+}
+.btn-info.disabled:hover,
+.btn-info[disabled]:hover,
+fieldset[disabled] .btn-info:hover,
+.btn-info.disabled:focus,
+.btn-info[disabled]:focus,
+fieldset[disabled] .btn-info:focus,
+.btn-info.disabled.focus,
+.btn-info[disabled].focus,
+fieldset[disabled] .btn-info.focus {
+  background-color: #5bc0de;
+  border-color: #46b8da;
+}
+.btn-info .badge {
+  color: #5bc0de;
+  background-color: #fff;
+}
+.btn-warning {
+  color: #fff;
+  background-color: #f0ad4e;
+  border-color: #eea236;
+}
+.btn-warning:focus,
+.btn-warning.focus {
+  color: #fff;
+  background-color: #ec971f;
+  border-color: #985f0d;
+}
+.btn-warning:hover {
+  color: #fff;
+  background-color: #ec971f;
+  border-color: #d58512;
+}
+.btn-warning:active,
+.btn-warning.active,
+.open > .dropdown-toggle.btn-warning {
+  color: #fff;
+  background-color: #ec971f;
+  border-color: #d58512;
+}
+.btn-warning:active:hover,
+.btn-warning.active:hover,
+.open > .dropdown-toggle.btn-warning:hover,
+.btn-warning:active:focus,
+.btn-warning.active:focus,
+.open > .dropdown-toggle.btn-warning:focus,
+.btn-warning:active.focus,
+.btn-warning.active.focus,
+.open > .dropdown-toggle.btn-warning.focus {
+  color: #fff;
+  background-color: #d58512;
+  border-color: #985f0d;
+}
+.btn-warning:active,
+.btn-warning.active,
+.open > .dropdown-toggle.btn-warning {
+  background-image: none;
+}
+.btn-warning.disabled:hover,
+.btn-warning[disabled]:hover,
+fieldset[disabled] .btn-warning:hover,
+.btn-warning.disabled:focus,
+.btn-warning[disabled]:focus,
+fieldset[disabled] .btn-warning:focus,
+.btn-warning.disabled.focus,
+.btn-warning[disabled].focus,
+fieldset[disabled] .btn-warning.focus {
+  background-color: #f0ad4e;
+  border-color: #eea236;
+}
+.btn-warning .badge {
+  color: #f0ad4e;
+  background-color: #fff;
+}
+.btn-danger {
+  color: #fff;
+  background-color: #d9534f;
+  border-color: #d43f3a;
+}
+.btn-danger:focus,
+.btn-danger.focus {
+  color: #fff;
+  background-color: #c9302c;
+  border-color: #761c19;
+}
+.btn-danger:hover {
+  color: #fff;
+  background-color: #c9302c;
+  border-color: #ac2925;
+}
+.btn-danger:active,
+.btn-danger.active,
+.open > .dropdown-toggle.btn-danger {
+  color: #fff;
+  background-color: #c9302c;
+  border-color: #ac2925;
+}
+.btn-danger:active:hover,
+.btn-danger.active:hover,
+.open > .dropdown-toggle.btn-danger:hover,
+.btn-danger:active:focus,
+.btn-danger.active:focus,
+.open > .dropdown-toggle.btn-danger:focus,
+.btn-danger:active.focus,
+.btn-danger.active.focus,
+.open > .dropdown-toggle.btn-danger.focus {
+  color: #fff;
+  background-color: #ac2925;
+  border-color: #761c19;
+}
+.btn-danger:active,
+.btn-danger.active,
+.open > .dropdown-toggle.btn-danger {
+  background-image: none;
+}
+.btn-danger.disabled:hover,
+.btn-danger[disabled]:hover,
+fieldset[disabled] .btn-danger:hover,
+.btn-danger.disabled:focus,
+.btn-danger[disabled]:focus,
+fieldset[disabled] .btn-danger:focus,
+.btn-danger.disabled.focus,
+.btn-danger[disabled].focus,
+fieldset[disabled] .btn-danger.focus {
+  background-color: #d9534f;
+  border-color: #d43f3a;
+}
+.btn-danger .badge {
+  color: #d9534f;
+  background-color: #fff;
+}
+.btn-link {
+  color: #337ab7;
+  font-weight: normal;
+  border-radius: 0;
+}
+.btn-link,
+.btn-link:active,
+.btn-link.active,
+.btn-link[disabled],
+fieldset[disabled] .btn-link {
+  background-color: transparent;
+  -webkit-box-shadow: none;
+  box-shadow: none;
+}
+.btn-link,
+.btn-link:hover,
+.btn-link:focus,
+.btn-link:active {
+  border-color: transparent;
+}
+.btn-link:hover,
+.btn-link:focus {
+  color: #23527c;
+  text-decoration: underline;
+  background-color: transparent;
+}
+.btn-link[disabled]:hover,
+fieldset[disabled] .btn-link:hover,
+.btn-link[disabled]:focus,
+fieldset[disabled] .btn-link:focus {
+  color: #777777;
+  text-decoration: none;
+}
+.btn-lg,
+.btn-group-lg > .btn {
+  padding: 10px 16px;
+  font-size: 17px;
+  line-height: 1.3333333;
+  border-radius: 3px;
+}
+.btn-sm,
+.btn-group-sm > .btn {
+  padding: 5px 10px;
+  font-size: 12px;
+  line-height: 1.5;
+  border-radius: 1px;
+}
+.btn-xs,
+.btn-group-xs > .btn {
+  padding: 1px 5px;
+  font-size: 12px;
+  line-height: 1.5;
+  border-radius: 1px;
+}
+.btn-block {
+  display: block;
+  width: 100%;
+}
+.btn-block + .btn-block {
+  margin-top: 5px;
+}
+input[type="submit"].btn-block,
+input[type="reset"].btn-block,
+input[type="button"].btn-block {
+  width: 100%;
+}
+.fade {
+  opacity: 0;
+  -webkit-transition: opacity 0.15s linear;
+  -o-transition: opacity 0.15s linear;
+  transition: opacity 0.15s linear;
+}
+.fade.in {
+  opacity: 1;
+}
+.collapse {
+  display: none;
+}
+.collapse.in {
+  display: block;
+}
+tr.collapse.in {
+  display: table-row;
+}
+tbody.collapse.in {
+  display: table-row-group;
+}
+.collapsing {
+  position: relative;
+  height: 0;
+  overflow: hidden;
+  -webkit-transition-property: height, visibility;
+  transition-property: height, visibility;
+  -webkit-transition-duration: 0.35s;
+  transition-duration: 0.35s;
+  -webkit-transition-timing-function: ease;
+  transition-timing-function: ease;
+}
+.caret {
+  display: inline-block;
+  width: 0;
+  height: 0;
+  margin-left: 2px;
+  vertical-align: middle;
+  border-top: 4px dashed;
+  border-top: 4px solid \9;
+  border-right: 4px solid transparent;
+  border-left: 4px solid transparent;
+}
+.dropup,
+.dropdown {
+  position: relative;
+}
+.dropdown-toggle:focus {
+  outline: 0;
+}
+.dropdown-menu {
+  position: absolute;
+  top: 100%;
+  left: 0;
+  z-index: 1000;
+  display: none;
+  float: left;
+  min-width: 160px;
+  padding: 5px 0;
+  margin: 2px 0 0;
+  list-style: none;
+  font-size: 13px;
+  text-align: left;
+  background-color: #fff;
+  border: 1px solid #ccc;
+  border: 1px solid rgba(0, 0, 0, 0.15);
+  border-radius: 2px;
+  -webkit-box-shadow: 0 6px 12px rgba(0, 0, 0, 0.175);
+  box-shadow: 0 6px 12px rgba(0, 0, 0, 0.175);
+  background-clip: padding-box;
+}
+.dropdown-menu.pull-right {
+  right: 0;
+  left: auto;
+}
+.dropdown-menu .divider {
+  height: 1px;
+  margin: 8px 0;
+  overflow: hidden;
+  background-color: #e5e5e5;
+}
+.dropdown-menu > li > a {
+  display: block;
+  padding: 3px 20px;
+  clear: both;
+  font-weight: normal;
+  line-height: 1.42857143;
+  color: #333333;
+  white-space: nowrap;
+}
+.dropdown-menu > li > a:hover,
+.dropdown-menu > li > a:focus {
+  text-decoration: none;
+  color: #262626;
+  background-color: #f5f5f5;
+}
+.dropdown-menu > .active > a,
+.dropdown-menu > .active > a:hover,
+.dropdown-menu > .active > a:focus {
+  color: #fff;
+  text-decoration: none;
+  outline: 0;
+  background-color: #337ab7;
+}
+.dropdown-menu > .disabled > a,
+.dropdown-menu > .disabled > a:hover,
+.dropdown-menu > .disabled > a:focus {
+  color: #777777;
+}
+.dropdown-menu > .disabled > a:hover,
+.dropdown-menu > .disabled > a:focus {
+  text-decoration: none;
+  background-color: transparent;
+  background-image: none;
+  filter: progid:DXImageTransform.Microsoft.gradient(enabled = false);
+  cursor: not-allowed;
+}
+.open > .dropdown-menu {
+  display: block;
+}
+.open > a {
+  outline: 0;
+}
+.dropdown-menu-right {
+  left: auto;
+  right: 0;
+}
+.dropdown-menu-left {
+  left: 0;
+  right: auto;
+}
+.dropdown-header {
+  display: block;
+  padding: 3px 20px;
+  font-size: 12px;
+  line-height: 1.42857143;
+  color: #777777;
+  white-space: nowrap;
+}
+.dropdown-backdrop {
+  position: fixed;
+  left: 0;
+  right: 0;
+  bottom: 0;
+  top: 0;
+  z-index: 990;
+}
+.pull-right > .dropdown-menu {
+  right: 0;
+  left: auto;
+}
+.dropup .caret,
+.navbar-fixed-bottom .dropdown .caret {
+  border-top: 0;
+  border-bottom: 4px dashed;
+  border-bottom: 4px solid \9;
+  content: "";
+}
+.dropup .dropdown-menu,
+.navbar-fixed-bottom .dropdown .dropdown-menu {
+  top: auto;
+  bottom: 100%;
+  margin-bottom: 2px;
+}
+@media (min-width: 541px) {
+  .navbar-right .dropdown-menu {
+    left: auto;
+    right: 0;
+  }
+  .navbar-right .dropdown-menu-left {
+    left: 0;
+    right: auto;
+  }
+}
+.btn-group,
+.btn-group-vertical {
+  position: relative;
+  display: inline-block;
+  vertical-align: middle;
+}
+.btn-group > .btn,
+.btn-group-vertical > .btn {
+  position: relative;
+  float: left;
+}
+.btn-group > .btn:hover,
+.btn-group-vertical > .btn:hover,
+.btn-group > .btn:focus,
+.btn-group-vertical > .btn:focus,
+.btn-group > .btn:active,
+.btn-group-vertical > .btn:active,
+.btn-group > .btn.active,
+.btn-group-vertical > .btn.active {
+  z-index: 2;
+}
+.btn-group .btn + .btn,
+.btn-group .btn + .btn-group,
+.btn-group .btn-group + .btn,
+.btn-group .btn-group + .btn-group {
+  margin-left: -1px;
+}
+.btn-toolbar {
+  margin-left: -5px;
+}
+.btn-toolbar .btn,
+.btn-toolbar .btn-group,
+.btn-toolbar .input-group {
+  float: left;
+}
+.btn-toolbar > .btn,
+.btn-toolbar > .btn-group,
+.btn-toolbar > .input-group {
+  margin-left: 5px;
+}
+.btn-group > .btn:not(:first-child):not(:last-child):not(.dropdown-toggle) {
+  border-radius: 0;
+}
+.btn-group > .btn:first-child {
+  margin-left: 0;
+}
+.btn-group > .btn:first-child:not(:last-child):not(.dropdown-toggle) {
+  border-bottom-right-radius: 0;
+  border-top-right-radius: 0;
+}
+.btn-group > .btn:last-child:not(:first-child),
+.btn-group > .dropdown-toggle:not(:first-child) {
+  border-bottom-left-radius: 0;
+  border-top-left-radius: 0;
+}
+.btn-group > .btn-group {
+  float: left;
+}
+.btn-group > .btn-group:not(:first-child):not(:last-child) > .btn {
+  border-radius: 0;
+}
+.btn-group > .btn-group:first-child:not(:last-child) > .btn:last-child,
+.btn-group > .btn-group:first-child:not(:last-child) > .dropdown-toggle {
+  border-bottom-right-radius: 0;
+  border-top-right-radius: 0;
+}
+.btn-group > .btn-group:last-child:not(:first-child) > .btn:first-child {
+  border-bottom-left-radius: 0;
+  border-top-left-radius: 0;
+}
+.btn-group .dropdown-toggle:active,
+.btn-group.open .dropdown-toggle {
+  outline: 0;
+}
+.btn-group > .btn + .dropdown-toggle {
+  padding-left: 8px;
+  padding-right: 8px;
+}
+.btn-group > .btn-lg + .dropdown-toggle {
+  padding-left: 12px;
+  padding-right: 12px;
+}
+.btn-group.open .dropdown-toggle {
+  -webkit-box-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);
+  box-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);
+}
+.btn-group.open .dropdown-toggle.btn-link {
+  -webkit-box-shadow: none;
+  box-shadow: none;
+}
+.btn .caret {
+  margin-left: 0;
+}
+.btn-lg .caret {
+  border-width: 5px 5px 0;
+  border-bottom-width: 0;
+}
+.dropup .btn-lg .caret {
+  border-width: 0 5px 5px;
+}
+.btn-group-vertical > .btn,
+.btn-group-vertical > .btn-group,
+.btn-group-vertical > .btn-group > .btn {
+  display: block;
+  float: none;
+  width: 100%;
+  max-width: 100%;
+}
+.btn-group-vertical > .btn-group > .btn {
+  float: none;
+}
+.btn-group-vertical > .btn + .btn,
+.btn-group-vertical > .btn + .btn-group,
+.btn-group-vertical > .btn-group + .btn,
+.btn-group-vertical > .btn-group + .btn-group {
+  margin-top: -1px;
+  margin-left: 0;
+}
+.btn-group-vertical > .btn:not(:first-child):not(:last-child) {
+  border-radius: 0;
+}
+.btn-group-vertical > .btn:first-child:not(:last-child) {
+  border-top-right-radius: 2px;
+  border-top-left-radius: 2px;
+  border-bottom-right-radius: 0;
+  border-bottom-left-radius: 0;
+}
+.btn-group-vertical > .btn:last-child:not(:first-child) {
+  border-top-right-radius: 0;
+  border-top-left-radius: 0;
+  border-bottom-right-radius: 2px;
+  border-bottom-left-radius: 2px;
+}
+.btn-group-vertical > .btn-group:not(:first-child):not(:last-child) > .btn {
+  border-radius: 0;
+}
+.btn-group-vertical > .btn-group:first-child:not(:last-child) > .btn:last-child,
+.btn-group-vertical > .btn-group:first-child:not(:last-child) > .dropdown-toggle {
+  border-bottom-right-radius: 0;
+  border-bottom-left-radius: 0;
+}
+.btn-group-vertical > .btn-group:last-child:not(:first-child) > .btn:first-child {
+  border-top-right-radius: 0;
+  border-top-left-radius: 0;
+}
+.btn-group-justified {
+  display: table;
+  width: 100%;
+  table-layout: fixed;
+  border-collapse: separate;
+}
+.btn-group-justified > .btn,
+.btn-group-justified > .btn-group {
+  float: none;
+  display: table-cell;
+  width: 1%;
+}
+.btn-group-justified > .btn-group .btn {
+  width: 100%;
+}
+.btn-group-justified > .btn-group .dropdown-menu {
+  left: auto;
+}
+[data-toggle="buttons"] > .btn input[type="radio"],
+[data-toggle="buttons"] > .btn-group > .btn input[type="radio"],
+[data-toggle="buttons"] > .btn input[type="checkbox"],
+[data-toggle="buttons"] > .btn-group > .btn input[type="checkbox"] {
+  position: absolute;
+  clip: rect(0, 0, 0, 0);
+  pointer-events: none;
+}
+.input-group {
+  position: relative;
+  display: table;
+  border-collapse: separate;
+}
+.input-group[class*="col-"] {
+  float: none;
+  padding-left: 0;
+  padding-right: 0;
+}
+.input-group .form-control {
+  position: relative;
+  z-index: 2;
+  float: left;
+  width: 100%;
+  margin-bottom: 0;
+}
+.input-group .form-control:focus {
+  z-index: 3;
+}
+.input-group-lg > .form-control,
+.input-group-lg > .input-group-addon,
+.input-group-lg > .input-group-btn > .btn {
+  height: 45px;
+  padding: 10px 16px;
+  font-size: 17px;
+  line-height: 1.3333333;
+  border-radius: 3px;
+}
+select.input-group-lg > .form-control,
+select.input-group-lg > .input-group-addon,
+select.input-group-lg > .input-group-btn > .btn {
+  height: 45px;
+  line-height: 45px;
+}
+textarea.input-group-lg > .form-control,
+textarea.input-group-lg > .input-group-addon,
+textarea.input-group-lg > .input-group-btn > .btn,
+select[multiple].input-group-lg > .form-control,
+select[multiple].input-group-lg > .input-group-addon,
+select[multiple].input-group-lg > .input-group-btn > .btn {
+  height: auto;
+}
+.input-group-sm > .form-control,
+.input-group-sm > .input-group-addon,
+.input-group-sm > .input-group-btn > .btn {
+  height: 30px;
+  padding: 5px 10px;
+  font-size: 12px;
+  line-height: 1.5;
+  border-radius: 1px;
+}
+select.input-group-sm > .form-control,
+select.input-group-sm > .input-group-addon,
+select.input-group-sm > .input-group-btn > .btn {
+  height: 30px;
+  line-height: 30px;
+}
+textarea.input-group-sm > .form-control,
+textarea.input-group-sm > .input-group-addon,
+textarea.input-group-sm > .input-group-btn > .btn,
+select[multiple].input-group-sm > .form-control,
+select[multiple].input-group-sm > .input-group-addon,
+select[multiple].input-group-sm > .input-group-btn > .btn {
+  height: auto;
+}
+.input-group-addon,
+.input-group-btn,
+.input-group .form-control {
+  display: table-cell;
+}
+.input-group-addon:not(:first-child):not(:last-child),
+.input-group-btn:not(:first-child):not(:last-child),
+.input-group .form-control:not(:first-child):not(:last-child) {
+  border-radius: 0;
+}
+.input-group-addon,
+.input-group-btn {
+  width: 1%;
+  white-space: nowrap;
+  vertical-align: middle;
+}
+.input-group-addon {
+  padding: 6px 12px;
+  font-size: 13px;
+  font-weight: normal;
+  line-height: 1;
+  color: #555555;
+  text-align: center;
+  background-color: #eeeeee;
+  border: 1px solid #ccc;
+  border-radius: 2px;
+}
+.input-group-addon.input-sm {
+  padding: 5px 10px;
+  font-size: 12px;
+  border-radius: 1px;
+}
+.input-group-addon.input-lg {
+  padding: 10px 16px;
+  font-size: 17px;
+  border-radius: 3px;
+}
+.input-group-addon input[type="radio"],
+.input-group-addon input[type="checkbox"] {
+  margin-top: 0;
+}
+.input-group .form-control:first-child,
+.input-group-addon:first-child,
+.input-group-btn:first-child > .btn,
+.input-group-btn:first-child > .btn-group > .btn,
+.input-group-btn:first-child > .dropdown-toggle,
+.input-group-btn:last-child > .btn:not(:last-child):not(.dropdown-toggle),
+.input-group-btn:last-child > .btn-group:not(:last-child) > .btn {
+  border-bottom-right-radius: 0;
+  border-top-right-radius: 0;
+}
+.input-group-addon:first-child {
+  border-right: 0;
+}
+.input-group .form-control:last-child,
+.input-group-addon:last-child,
+.input-group-btn:last-child > .btn,
+.input-group-btn:last-child > .btn-group > .btn,
+.input-group-btn:last-child > .dropdown-toggle,
+.input-group-btn:first-child > .btn:not(:first-child),
+.input-group-btn:first-child > .btn-group:not(:first-child) > .btn {
+  border-bottom-left-radius: 0;
+  border-top-left-radius: 0;
+}
+.input-group-addon:last-child {
+  border-left: 0;
+}
+.input-group-btn {
+  position: relative;
+  font-size: 0;
+  white-space: nowrap;
+}
+.input-group-btn > .btn {
+  position: relative;
+}
+.input-group-btn > .btn + .btn {
+  margin-left: -1px;
+}
+.input-group-btn > .btn:hover,
+.input-group-btn > .btn:focus,
+.input-group-btn > .btn:active {
+  z-index: 2;
+}
+.input-group-btn:first-child > .btn,
+.input-group-btn:first-child > .btn-group {
+  margin-right: -1px;
+}
+.input-group-btn:last-child > .btn,
+.input-group-btn:last-child > .btn-group {
+  z-index: 2;
+  margin-left: -1px;
+}
+.nav {
+  margin-bottom: 0;
+  padding-left: 0;
+  list-style: none;
+}
+.nav > li {
+  position: relative;
+  display: block;
+}
+.nav > li > a {
+  position: relative;
+  display: block;
+  padding: 10px 15px;
+}
+.nav > li > a:hover,
+.nav > li > a:focus {
+  text-decoration: none;
+  background-color: #eeeeee;
+}
+.nav > li.disabled > a {
+  color: #777777;
+}
+.nav > li.disabled > a:hover,
+.nav > li.disabled > a:focus {
+  color: #777777;
+  text-decoration: none;
+  background-color: transparent;
+  cursor: not-allowed;
+}
+.nav .open > a,
+.nav .open > a:hover,
+.nav .open > a:focus {
+  background-color: #eeeeee;
+  border-color: #337ab7;
+}
+.nav .nav-divider {
+  height: 1px;
+  margin: 8px 0;
+  overflow: hidden;
+  background-color: #e5e5e5;
+}
+.nav > li > a > img {
+  max-width: none;
+}
+.nav-tabs {
+  border-bottom: 1px solid #ddd;
+}
+.nav-tabs > li {
+  float: left;
+  margin-bottom: -1px;
+}
+.nav-tabs > li > a {
+  margin-right: 2px;
+  line-height: 1.42857143;
+  border: 1px solid transparent;
+  border-radius: 2px 2px 0 0;
+}
+.nav-tabs > li > a:hover {
+  border-color: #eeeeee #eeeeee #ddd;
+}
+.nav-tabs > li.active > a,
+.nav-tabs > li.active > a:hover,
+.nav-tabs > li.active > a:focus {
+  color: #555555;
+  background-color: #fff;
+  border: 1px solid #ddd;
+  border-bottom-color: transparent;
+  cursor: default;
+}
+.nav-tabs.nav-justified {
+  width: 100%;
+  border-bottom: 0;
+}
+.nav-tabs.nav-justified > li {
+  float: none;
+}
+.nav-tabs.nav-justified > li > a {
+  text-align: center;
+  margin-bottom: 5px;
+}
+.nav-tabs.nav-justified > .dropdown .dropdown-menu {
+  top: auto;
+  left: auto;
+}
+@media (min-width: 768px) {
+  .nav-tabs.nav-justified > li {
+    display: table-cell;
+    width: 1%;
+  }
+  .nav-tabs.nav-justified > li > a {
+    margin-bottom: 0;
+  }
+}
+.nav-tabs.nav-justified > li > a {
+  margin-right: 0;
+  border-radius: 2px;
+}
+.nav-tabs.nav-justified > .active > a,
+.nav-tabs.nav-justified > .active > a:hover,
+.nav-tabs.nav-justified > .active > a:focus {
+  border: 1px solid #ddd;
+}
+@media (min-width: 768px) {
+  .nav-tabs.nav-justified > li > a {
+    border-bottom: 1px solid #ddd;
+    border-radius: 2px 2px 0 0;
+  }
+  .nav-tabs.nav-justified > .active > a,
+  .nav-tabs.nav-justified > .active > a:hover,
+  .nav-tabs.nav-justified > .active > a:focus {
+    border-bottom-color: #fff;
+  }
+}
+.nav-pills > li {
+  float: left;
+}
+.nav-pills > li > a {
+  border-radius: 2px;
+}
+.nav-pills > li + li {
+  margin-left: 2px;
+}
+.nav-pills > li.active > a,
+.nav-pills > li.active > a:hover,
+.nav-pills > li.active > a:focus {
+  color: #fff;
+  background-color: #337ab7;
+}
+.nav-stacked > li {
+  float: none;
+}
+.nav-stacked > li + li {
+  margin-top: 2px;
+  margin-left: 0;
+}
+.nav-justified {
+  width: 100%;
+}
+.nav-justified > li {
+  float: none;
+}
+.nav-justified > li > a {
+  text-align: center;
+  margin-bottom: 5px;
+}
+.nav-justified > .dropdown .dropdown-menu {
+  top: auto;
+  left: auto;
+}
+@media (min-width: 768px) {
+  .nav-justified > li {
+    display: table-cell;
+    width: 1%;
+  }
+  .nav-justified > li > a {
+    margin-bottom: 0;
+  }
+}
+.nav-tabs-justified {
+  border-bottom: 0;
+}
+.nav-tabs-justified > li > a {
+  margin-right: 0;
+  border-radius: 2px;
+}
+.nav-tabs-justified > .active > a,
+.nav-tabs-justified > .active > a:hover,
+.nav-tabs-justified > .active > a:focus {
+  border: 1px solid #ddd;
+}
+@media (min-width: 768px) {
+  .nav-tabs-justified > li > a {
+    border-bottom: 1px solid #ddd;
+    border-radius: 2px 2px 0 0;
+  }
+  .nav-tabs-justified > .active > a,
+  .nav-tabs-justified > .active > a:hover,
+  .nav-tabs-justified > .active > a:focus {
+    border-bottom-color: #fff;
+  }
+}
+.tab-content > .tab-pane {
+  display: none;
+}
+.tab-content > .active {
+  display: block;
+}
+.nav-tabs .dropdown-menu {
+  margin-top: -1px;
+  border-top-right-radius: 0;
+  border-top-left-radius: 0;
+}
+.navbar {
+  position: relative;
+  min-height: 30px;
+  margin-bottom: 18px;
+  border: 1px solid transparent;
+}
+@media (min-width: 541px) {
+  .navbar {
+    border-radius: 2px;
+  }
+}
+@media (min-width: 541px) {
+  .navbar-header {
+    float: left;
+  }
+}
+.navbar-collapse {
+  overflow-x: visible;
+  padding-right: 0px;
+  padding-left: 0px;
+  border-top: 1px solid transparent;
+  box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.1);
+  -webkit-overflow-scrolling: touch;
+}
+.navbar-collapse.in {
+  overflow-y: auto;
+}
+@media (min-width: 541px) {
+  .navbar-collapse {
+    width: auto;
+    border-top: 0;
+    box-shadow: none;
+  }
+  .navbar-collapse.collapse {
+    display: block !important;
+    height: auto !important;
+    padding-bottom: 0;
+    overflow: visible !important;
+  }
+  .navbar-collapse.in {
+    overflow-y: visible;
+  }
+  .navbar-fixed-top .navbar-collapse,
+  .navbar-static-top .navbar-collapse,
+  .navbar-fixed-bottom .navbar-collapse {
+    padding-left: 0;
+    padding-right: 0;
+  }
+}
+.navbar-fixed-top .navbar-collapse,
+.navbar-fixed-bottom .navbar-collapse {
+  max-height: 340px;
+}
+@media (max-device-width: 540px) and (orientation: landscape) {
+  .navbar-fixed-top .navbar-collapse,
+  .navbar-fixed-bottom .navbar-collapse {
+    max-height: 200px;
+  }
+}
+.container > .navbar-header,
+.container-fluid > .navbar-header,
+.container > .navbar-collapse,
+.container-fluid > .navbar-collapse {
+  margin-right: 0px;
+  margin-left: 0px;
+}
+@media (min-width: 541px) {
+  .container > .navbar-header,
+  .container-fluid > .navbar-header,
+  .container > .navbar-collapse,
+  .container-fluid > .navbar-collapse {
+    margin-right: 0;
+    margin-left: 0;
+  }
+}
+.navbar-static-top {
+  z-index: 1000;
+  border-width: 0 0 1px;
+}
+@media (min-width: 541px) {
+  .navbar-static-top {
+    border-radius: 0;
+  }
+}
+.navbar-fixed-top,
+.navbar-fixed-bottom {
+  position: fixed;
+  right: 0;
+  left: 0;
+  z-index: 1030;
+}
+@media (min-width: 541px) {
+  .navbar-fixed-top,
+  .navbar-fixed-bottom {
+    border-radius: 0;
+  }
+}
+.navbar-fixed-top {
+  top: 0;
+  border-width: 0 0 1px;
+}
+.navbar-fixed-bottom {
+  bottom: 0;
+  margin-bottom: 0;
+  border-width: 1px 0 0;
+}
+.navbar-brand {
+  float: left;
+  padding: 6px 0px;
+  font-size: 17px;
+  line-height: 18px;
+  height: 30px;
+}
+.navbar-brand:hover,
+.navbar-brand:focus {
+  text-decoration: none;
+}
+.navbar-brand > img {
+  display: block;
+}
+@media (min-width: 541px) {
+  .navbar > .container .navbar-brand,
+  .navbar > .container-fluid .navbar-brand {
+    margin-left: 0px;
+  }
+}
+.navbar-toggle {
+  position: relative;
+  float: right;
+  margin-right: 0px;
+  padding: 9px 10px;
+  margin-top: -2px;
+  margin-bottom: -2px;
+  background-color: transparent;
+  background-image: none;
+  border: 1px solid transparent;
+  border-radius: 2px;
+}
+.navbar-toggle:focus {
+  outline: 0;
+}
+.navbar-toggle .icon-bar {
+  display: block;
+  width: 22px;
+  height: 2px;
+  border-radius: 1px;
+}
+.navbar-toggle .icon-bar + .icon-bar {
+  margin-top: 4px;
+}
+@media (min-width: 541px) {
+  .navbar-toggle {
+    display: none;
+  }
+}
+.navbar-nav {
+  margin: 3px 0px;
+}
+.navbar-nav > li > a {
+  padding-top: 10px;
+  padding-bottom: 10px;
+  line-height: 18px;
+}
+@media (max-width: 540px) {
+  .navbar-nav .open .dropdown-menu {
+    position: static;
+    float: none;
+    width: auto;
+    margin-top: 0;
+    background-color: transparent;
+    border: 0;
+    box-shadow: none;
+  }
+  .navbar-nav .open .dropdown-menu > li > a,
+  .navbar-nav .open .dropdown-menu .dropdown-header {
+    padding: 5px 15px 5px 25px;
+  }
+  .navbar-nav .open .dropdown-menu > li > a {
+    line-height: 18px;
+  }
+  .navbar-nav .open .dropdown-menu > li > a:hover,
+  .navbar-nav .open .dropdown-menu > li > a:focus {
+    background-image: none;
+  }
+}
+@media (min-width: 541px) {
+  .navbar-nav {
+    float: left;
+    margin: 0;
+  }
+  .navbar-nav > li {
+    float: left;
+  }
+  .navbar-nav > li > a {
+    padding-top: 6px;
+    padding-bottom: 6px;
+  }
+}
+.navbar-form {
+  margin-left: 0px;
+  margin-right: 0px;
+  padding: 10px 0px;
+  border-top: 1px solid transparent;
+  border-bottom: 1px solid transparent;
+  -webkit-box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.1), 0 1px 0 rgba(255, 255, 255, 0.1);
+  box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.1), 0 1px 0 rgba(255, 255, 255, 0.1);
+  margin-top: -1px;
+  margin-bottom: -1px;
+}
+@media (min-width: 768px) {
+  .navbar-form .form-group {
+    display: inline-block;
+    margin-bottom: 0;
+    vertical-align: middle;
+  }
+  .navbar-form .form-control {
+    display: inline-block;
+    width: auto;
+    vertical-align: middle;
+  }
+  .navbar-form .form-control-static {
+    display: inline-block;
+  }
+  .navbar-form .input-group {
+    display: inline-table;
+    vertical-align: middle;
+  }
+  .navbar-form .input-group .input-group-addon,
+  .navbar-form .input-group .input-group-btn,
+  .navbar-form .input-group .form-control {
+    width: auto;
+  }
+  .navbar-form .input-group > .form-control {
+    width: 100%;
+  }
+  .navbar-form .control-label {
+    margin-bottom: 0;
+    vertical-align: middle;
+  }
+  .navbar-form .radio,
+  .navbar-form .checkbox {
+    display: inline-block;
+    margin-top: 0;
+    margin-bottom: 0;
+    vertical-align: middle;
+  }
+  .navbar-form .radio label,
+  .navbar-form .checkbox label {
+    padding-left: 0;
+  }
+  .navbar-form .radio input[type="radio"],
+  .navbar-form .checkbox input[type="checkbox"] {
+    position: relative;
+    margin-left: 0;
+  }
+  .navbar-form .has-feedback .form-control-feedback {
+    top: 0;
+  }
+}
+@media (max-width: 540px) {
+  .navbar-form .form-group {
+    margin-bottom: 5px;
+  }
+  .navbar-form .form-group:last-child {
+    margin-bottom: 0;
+  }
+}
+@media (min-width: 541px) {
+  .navbar-form {
+    width: auto;
+    border: 0;
+    margin-left: 0;
+    margin-right: 0;
+    padding-top: 0;
+    padding-bottom: 0;
+    -webkit-box-shadow: none;
+    box-shadow: none;
+  }
+}
+.navbar-nav > li > .dropdown-menu {
+  margin-top: 0;
+  border-top-right-radius: 0;
+  border-top-left-radius: 0;
+}
+.navbar-fixed-bottom .navbar-nav > li > .dropdown-menu {
+  margin-bottom: 0;
+  border-top-right-radius: 2px;
+  border-top-left-radius: 2px;
+  border-bottom-right-radius: 0;
+  border-bottom-left-radius: 0;
+}
+.navbar-btn {
+  margin-top: -1px;
+  margin-bottom: -1px;
+}
+.navbar-btn.btn-sm {
+  margin-top: 0px;
+  margin-bottom: 0px;
+}
+.navbar-btn.btn-xs {
+  margin-top: 4px;
+  margin-bottom: 4px;
+}
+.navbar-text {
+  margin-top: 6px;
+  margin-bottom: 6px;
+}
+@media (min-width: 541px) {
+  .navbar-text {
+    float: left;
+    margin-left: 0px;
+    margin-right: 0px;
+  }
+}
+@media (min-width: 541px) {
+  .navbar-left {
+    float: left !important;
+    float: left;
+  }
+  .navbar-right {
+    float: right !important;
+    float: right;
+    margin-right: 0px;
+  }
+  .navbar-right ~ .navbar-right {
+    margin-right: 0;
+  }
+}
+.navbar-default {
+  background-color: #f8f8f8;
+  border-color: #e7e7e7;
+}
+.navbar-default .navbar-brand {
+  color: #777;
+}
+.navbar-default .navbar-brand:hover,
+.navbar-default .navbar-brand:focus {
+  color: #5e5e5e;
+  background-color: transparent;
+}
+.navbar-default .navbar-text {
+  color: #777;
+}
+.navbar-default .navbar-nav > li > a {
+  color: #777;
+}
+.navbar-default .navbar-nav > li > a:hover,
+.navbar-default .navbar-nav > li > a:focus {
+  color: #333;
+  background-color: transparent;
+}
+.navbar-default .navbar-nav > .active > a,
+.navbar-default .navbar-nav > .active > a:hover,
+.navbar-default .navbar-nav > .active > a:focus {
+  color: #555;
+  background-color: #e7e7e7;
+}
+.navbar-default .navbar-nav > .disabled > a,
+.navbar-default .navbar-nav > .disabled > a:hover,
+.navbar-default .navbar-nav > .disabled > a:focus {
+  color: #ccc;
+  background-color: transparent;
+}
+.navbar-default .navbar-toggle {
+  border-color: #ddd;
+}
+.navbar-default .navbar-toggle:hover,
+.navbar-default .navbar-toggle:focus {
+  background-color: #ddd;
+}
+.navbar-default .navbar-toggle .icon-bar {
+  background-color: #888;
+}
+.navbar-default .navbar-collapse,
+.navbar-default .navbar-form {
+  border-color: #e7e7e7;
+}
+.navbar-default .navbar-nav > .open > a,
+.navbar-default .navbar-nav > .open > a:hover,
+.navbar-default .navbar-nav > .open > a:focus {
+  background-color: #e7e7e7;
+  color: #555;
+}
+@media (max-width: 540px) {
+  .navbar-default .navbar-nav .open .dropdown-menu > li > a {
+    color: #777;
+  }
+  .navbar-default .navbar-nav .open .dropdown-menu > li > a:hover,
+  .navbar-default .navbar-nav .open .dropdown-menu > li > a:focus {
+    color: #333;
+    background-color: transparent;
+  }
+  .navbar-default .navbar-nav .open .dropdown-menu > .active > a,
+  .navbar-default .navbar-nav .open .dropdown-menu > .active > a:hover,
+  .navbar-default .navbar-nav .open .dropdown-menu > .active > a:focus {
+    color: #555;
+    background-color: #e7e7e7;
+  }
+  .navbar-default .navbar-nav .open .dropdown-menu > .disabled > a,
+  .navbar-default .navbar-nav .open .dropdown-menu > .disabled > a:hover,
+  .navbar-default .navbar-nav .open .dropdown-menu > .disabled > a:focus {
+    color: #ccc;
+    background-color: transparent;
+  }
+}
+.navbar-default .navbar-link {
+  color: #777;
+}
+.navbar-default .navbar-link:hover {
+  color: #333;
+}
+.navbar-default .btn-link {
+  color: #777;
+}
+.navbar-default .btn-link:hover,
+.navbar-default .btn-link:focus {
+  color: #333;
+}
+.navbar-default .btn-link[disabled]:hover,
+fieldset[disabled] .navbar-default .btn-link:hover,
+.navbar-default .btn-link[disabled]:focus,
+fieldset[disabled] .navbar-default .btn-link:focus {
+  color: #ccc;
+}
+.navbar-inverse {
+  background-color: #222;
+  border-color: #080808;
+}
+.navbar-inverse .navbar-brand {
+  color: #9d9d9d;
+}
+.navbar-inverse .navbar-brand:hover,
+.navbar-inverse .navbar-brand:focus {
+  color: #fff;
+  background-color: transparent;
+}
+.navbar-inverse .navbar-text {
+  color: #9d9d9d;
+}
+.navbar-inverse .navbar-nav > li > a {
+  color: #9d9d9d;
+}
+.navbar-inverse .navbar-nav > li > a:hover,
+.navbar-inverse .navbar-nav > li > a:focus {
+  color: #fff;
+  background-color: transparent;
+}
+.navbar-inverse .navbar-nav > .active > a,
+.navbar-inverse .navbar-nav > .active > a:hover,
+.navbar-inverse .navbar-nav > .active > a:focus {
+  color: #fff;
+  background-color: #080808;
+}
+.navbar-inverse .navbar-nav > .disabled > a,
+.navbar-inverse .navbar-nav > .disabled > a:hover,
+.navbar-inverse .navbar-nav > .disabled > a:focus {
+  color: #444;
+  background-color: transparent;
+}
+.navbar-inverse .navbar-toggle {
+  border-color: #333;
+}
+.navbar-inverse .navbar-toggle:hover,
+.navbar-inverse .navbar-toggle:focus {
+  background-color: #333;
+}
+.navbar-inverse .navbar-toggle .icon-bar {
+  background-color: #fff;
+}
+.navbar-inverse .navbar-collapse,
+.navbar-inverse .navbar-form {
+  border-color: #101010;
+}
+.navbar-inverse .navbar-nav > .open > a,
+.navbar-inverse .navbar-nav > .open > a:hover,
+.navbar-inverse .navbar-nav > .open > a:focus {
+  background-color: #080808;
+  color: #fff;
+}
+@media (max-width: 540px) {
+  .navbar-inverse .navbar-nav .open .dropdown-menu > .dropdown-header {
+    border-color: #080808;
+  }
+  .navbar-inverse .navbar-nav .open .dropdown-menu .divider {
+    background-color: #080808;
+  }
+  .navbar-inverse .navbar-nav .open .dropdown-menu > li > a {
+    color: #9d9d9d;
+  }
+  .navbar-inverse .navbar-nav .open .dropdown-menu > li > a:hover,
+  .navbar-inverse .navbar-nav .open .dropdown-menu > li > a:focus {
+    color: #fff;
+    background-color: transparent;
+  }
+  .navbar-inverse .navbar-nav .open .dropdown-menu > .active > a,
+  .navbar-inverse .navbar-nav .open .dropdown-menu > .active > a:hover,
+  .navbar-inverse .navbar-nav .open .dropdown-menu > .active > a:focus {
+    color: #fff;
+    background-color: #080808;
+  }
+  .navbar-inverse .navbar-nav .open .dropdown-menu > .disabled > a,
+  .navbar-inverse .navbar-nav .open .dropdown-menu > .disabled > a:hover,
+  .navbar-inverse .navbar-nav .open .dropdown-menu > .disabled > a:focus {
+    color: #444;
+    background-color: transparent;
+  }
+}
+.navbar-inverse .navbar-link {
+  color: #9d9d9d;
+}
+.navbar-inverse .navbar-link:hover {
+  color: #fff;
+}
+.navbar-inverse .btn-link {
+  color: #9d9d9d;
+}
+.navbar-inverse .btn-link:hover,
+.navbar-inverse .btn-link:focus {
+  color: #fff;
+}
+.navbar-inverse .btn-link[disabled]:hover,
+fieldset[disabled] .navbar-inverse .btn-link:hover,
+.navbar-inverse .btn-link[disabled]:focus,
+fieldset[disabled] .navbar-inverse .btn-link:focus {
+  color: #444;
+}
+.breadcrumb {
+  padding: 8px 15px;
+  margin-bottom: 18px;
+  list-style: none;
+  background-color: #f5f5f5;
+  border-radius: 2px;
+}
+.breadcrumb > li {
+  display: inline-block;
+}
+.breadcrumb > li + li:before {
+  content: "/\00a0";
+  padding: 0 5px;
+  color: #5e5e5e;
+}
+.breadcrumb > .active {
+  color: #777777;
+}
+.pagination {
+  display: inline-block;
+  padding-left: 0;
+  margin: 18px 0;
+  border-radius: 2px;
+}
+.pagination > li {
+  display: inline;
+}
+.pagination > li > a,
+.pagination > li > span {
+  position: relative;
+  float: left;
+  padding: 6px 12px;
+  line-height: 1.42857143;
+  text-decoration: none;
+  color: #337ab7;
+  background-color: #fff;
+  border: 1px solid #ddd;
+  margin-left: -1px;
+}
+.pagination > li:first-child > a,
+.pagination > li:first-child > span {
+  margin-left: 0;
+  border-bottom-left-radius: 2px;
+  border-top-left-radius: 2px;
+}
+.pagination > li:last-child > a,
+.pagination > li:last-child > span {
+  border-bottom-right-radius: 2px;
+  border-top-right-radius: 2px;
+}
+.pagination > li > a:hover,
+.pagination > li > span:hover,
+.pagination > li > a:focus,
+.pagination > li > span:focus {
+  z-index: 2;
+  color: #23527c;
+  background-color: #eeeeee;
+  border-color: #ddd;
+}
+.pagination > .active > a,
+.pagination > .active > span,
+.pagination > .active > a:hover,
+.pagination > .active > span:hover,
+.pagination > .active > a:focus,
+.pagination > .active > span:focus {
+  z-index: 3;
+  color: #fff;
+  background-color: #337ab7;
+  border-color: #337ab7;
+  cursor: default;
+}
+.pagination > .disabled > span,
+.pagination > .disabled > span:hover,
+.pagination > .disabled > span:focus,
+.pagination > .disabled > a,
+.pagination > .disabled > a:hover,
+.pagination > .disabled > a:focus {
+  color: #777777;
+  background-color: #fff;
+  border-color: #ddd;
+  cursor: not-allowed;
+}
+.pagination-lg > li > a,
+.pagination-lg > li > span {
+  padding: 10px 16px;
+  font-size: 17px;
+  line-height: 1.3333333;
+}
+.pagination-lg > li:first-child > a,
+.pagination-lg > li:first-child > span {
+  border-bottom-left-radius: 3px;
+  border-top-left-radius: 3px;
+}
+.pagination-lg > li:last-child > a,
+.pagination-lg > li:last-child > span {
+  border-bottom-right-radius: 3px;
+  border-top-right-radius: 3px;
+}
+.pagination-sm > li > a,
+.pagination-sm > li > span {
+  padding: 5px 10px;
+  font-size: 12px;
+  line-height: 1.5;
+}
+.pagination-sm > li:first-child > a,
+.pagination-sm > li:first-child > span {
+  border-bottom-left-radius: 1px;
+  border-top-left-radius: 1px;
+}
+.pagination-sm > li:last-child > a,
+.pagination-sm > li:last-child > span {
+  border-bottom-right-radius: 1px;
+  border-top-right-radius: 1px;
+}
+.pager {
+  padding-left: 0;
+  margin: 18px 0;
+  list-style: none;
+  text-align: center;
+}
+.pager li {
+  display: inline;
+}
+.pager li > a,
+.pager li > span {
+  display: inline-block;
+  padding: 5px 14px;
+  background-color: #fff;
+  border: 1px solid #ddd;
+  border-radius: 15px;
+}
+.pager li > a:hover,
+.pager li > a:focus {
+  text-decoration: none;
+  background-color: #eeeeee;
+}
+.pager .next > a,
+.pager .next > span {
+  float: right;
+}
+.pager .previous > a,
+.pager .previous > span {
+  float: left;
+}
+.pager .disabled > a,
+.pager .disabled > a:hover,
+.pager .disabled > a:focus,
+.pager .disabled > span {
+  color: #777777;
+  background-color: #fff;
+  cursor: not-allowed;
+}
+.label {
+  display: inline;
+  padding: .2em .6em .3em;
+  font-size: 75%;
+  font-weight: bold;
+  line-height: 1;
+  color: #fff;
+  text-align: center;
+  white-space: nowrap;
+  vertical-align: baseline;
+  border-radius: .25em;
+}
+a.label:hover,
+a.label:focus {
+  color: #fff;
+  text-decoration: none;
+  cursor: pointer;
+}
+.label:empty {
+  display: none;
+}
+.btn .label {
+  position: relative;
+  top: -1px;
+}
+.label-default {
+  background-color: #777777;
+}
+.label-default[href]:hover,
+.label-default[href]:focus {
+  background-color: #5e5e5e;
+}
+.label-primary {
+  background-color: #337ab7;
+}
+.label-primary[href]:hover,
+.label-primary[href]:focus {
+  background-color: #286090;
+}
+.label-success {
+  background-color: #5cb85c;
+}
+.label-success[href]:hover,
+.label-success[href]:focus {
+  background-color: #449d44;
+}
+.label-info {
+  background-color: #5bc0de;
+}
+.label-info[href]:hover,
+.label-info[href]:focus {
+  background-color: #31b0d5;
+}
+.label-warning {
+  background-color: #f0ad4e;
+}
+.label-warning[href]:hover,
+.label-warning[href]:focus {
+  background-color: #ec971f;
+}
+.label-danger {
+  background-color: #d9534f;
+}
+.label-danger[href]:hover,
+.label-danger[href]:focus {
+  background-color: #c9302c;
+}
+.badge {
+  display: inline-block;
+  min-width: 10px;
+  padding: 3px 7px;
+  font-size: 12px;
+  font-weight: bold;
+  color: #fff;
+  line-height: 1;
+  vertical-align: middle;
+  white-space: nowrap;
+  text-align: center;
+  background-color: #777777;
+  border-radius: 10px;
+}
+.badge:empty {
+  display: none;
+}
+.btn .badge {
+  position: relative;
+  top: -1px;
+}
+.btn-xs .badge,
+.btn-group-xs > .btn .badge {
+  top: 0;
+  padding: 1px 5px;
+}
+a.badge:hover,
+a.badge:focus {
+  color: #fff;
+  text-decoration: none;
+  cursor: pointer;
+}
+.list-group-item.active > .badge,
+.nav-pills > .active > a > .badge {
+  color: #337ab7;
+  background-color: #fff;
+}
+.list-group-item > .badge {
+  float: right;
+}
+.list-group-item > .badge + .badge {
+  margin-right: 5px;
+}
+.nav-pills > li > a > .badge {
+  margin-left: 3px;
+}
+.jumbotron {
+  padding-top: 30px;
+  padding-bottom: 30px;
+  margin-bottom: 30px;
+  color: inherit;
+  background-color: #eeeeee;
+}
+.jumbotron h1,
+.jumbotron .h1 {
+  color: inherit;
+}
+.jumbotron p {
+  margin-bottom: 15px;
+  font-size: 20px;
+  font-weight: 200;
+}
+.jumbotron > hr {
+  border-top-color: #d5d5d5;
+}
+.container .jumbotron,
+.container-fluid .jumbotron {
+  border-radius: 3px;
+  padding-left: 0px;
+  padding-right: 0px;
+}
+.jumbotron .container {
+  max-width: 100%;
+}
+@media screen and (min-width: 768px) {
+  .jumbotron {
+    padding-top: 48px;
+    padding-bottom: 48px;
+  }
+  .container .jumbotron,
+  .container-fluid .jumbotron {
+    padding-left: 60px;
+    padding-right: 60px;
+  }
+  .jumbotron h1,
+  .jumbotron .h1 {
+    font-size: 59px;
+  }
+}
+.thumbnail {
+  display: block;
+  padding: 4px;
+  margin-bottom: 18px;
+  line-height: 1.42857143;
+  background-color: #fff;
+  border: 1px solid #ddd;
+  border-radius: 2px;
+  -webkit-transition: border 0.2s ease-in-out;
+  -o-transition: border 0.2s ease-in-out;
+  transition: border 0.2s ease-in-out;
+}
+.thumbnail > img,
+.thumbnail a > img {
+  margin-left: auto;
+  margin-right: auto;
+}
+a.thumbnail:hover,
+a.thumbnail:focus,
+a.thumbnail.active {
+  border-color: #337ab7;
+}
+.thumbnail .caption {
+  padding: 9px;
+  color: #000;
+}
+.alert {
+  padding: 15px;
+  margin-bottom: 18px;
+  border: 1px solid transparent;
+  border-radius: 2px;
+}
+.alert h4 {
+  margin-top: 0;
+  color: inherit;
+}
+.alert .alert-link {
+  font-weight: bold;
+}
+.alert > p,
+.alert > ul {
+  margin-bottom: 0;
+}
+.alert > p + p {
+  margin-top: 5px;
+}
+.alert-dismissable,
+.alert-dismissible {
+  padding-right: 35px;
+}
+.alert-dismissable .close,
+.alert-dismissible .close {
+  position: relative;
+  top: -2px;
+  right: -21px;
+  color: inherit;
+}
+.alert-success {
+  background-color: #dff0d8;
+  border-color: #d6e9c6;
+  color: #3c763d;
+}
+.alert-success hr {
+  border-top-color: #c9e2b3;
+}
+.alert-success .alert-link {
+  color: #2b542c;
+}
+.alert-info {
+  background-color: #d9edf7;
+  border-color: #bce8f1;
+  color: #31708f;
+}
+.alert-info hr {
+  border-top-color: #a6e1ec;
+}
+.alert-info .alert-link {
+  color: #245269;
+}
+.alert-warning {
+  background-color: #fcf8e3;
+  border-color: #faebcc;
+  color: #8a6d3b;
+}
+.alert-warning hr {
+  border-top-color: #f7e1b5;
+}
+.alert-warning .alert-link {
+  color: #66512c;
+}
+.alert-danger {
+  background-color: #f2dede;
+  border-color: #ebccd1;
+  color: #a94442;
+}
+.alert-danger hr {
+  border-top-color: #e4b9c0;
+}
+.alert-danger .alert-link {
+  color: #843534;
+}
+@-webkit-keyframes progress-bar-stripes {
+  from {
+    background-position: 40px 0;
+  }
+  to {
+    background-position: 0 0;
+  }
+}
+@keyframes progress-bar-stripes {
+  from {
+    background-position: 40px 0;
+  }
+  to {
+    background-position: 0 0;
+  }
+}
+.progress {
+  overflow: hidden;
+  height: 18px;
+  margin-bottom: 18px;
+  background-color: #f5f5f5;
+  border-radius: 2px;
+  -webkit-box-shadow: inset 0 1px 2px rgba(0, 0, 0, 0.1);
+  box-shadow: inset 0 1px 2px rgba(0, 0, 0, 0.1);
+}
+.progress-bar {
+  float: left;
+  width: 0%;
+  height: 100%;
+  font-size: 12px;
+  line-height: 18px;
+  color: #fff;
+  text-align: center;
+  background-color: #337ab7;
+  -webkit-box-shadow: inset 0 -1px 0 rgba(0, 0, 0, 0.15);
+  box-shadow: inset 0 -1px 0 rgba(0, 0, 0, 0.15);
+  -webkit-transition: width 0.6s ease;
+  -o-transition: width 0.6s ease;
+  transition: width 0.6s ease;
+}
+.progress-striped .progress-bar,
+.progress-bar-striped {
+  background-image: -webkit-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
+  background-image: -o-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
+  background-image: linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
+  background-size: 40px 40px;
+}
+.progress.active .progress-bar,
+.progress-bar.active {
+  -webkit-animation: progress-bar-stripes 2s linear infinite;
+  -o-animation: progress-bar-stripes 2s linear infinite;
+  animation: progress-bar-stripes 2s linear infinite;
+}
+.progress-bar-success {
+  background-color: #5cb85c;
+}
+.progress-striped .progress-bar-success {
+  background-image: -webkit-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
+  background-image: -o-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
+  background-image: linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
+}
+.progress-bar-info {
+  background-color: #5bc0de;
+}
+.progress-striped .progress-bar-info {
+  background-image: -webkit-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
+  background-image: -o-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
+  background-image: linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
+}
+.progress-bar-warning {
+  background-color: #f0ad4e;
+}
+.progress-striped .progress-bar-warning {
+  background-image: -webkit-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
+  background-image: -o-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
+  background-image: linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
+}
+.progress-bar-danger {
+  background-color: #d9534f;
+}
+.progress-striped .progress-bar-danger {
+  background-image: -webkit-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
+  background-image: -o-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
+  background-image: linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
+}
+.media {
+  margin-top: 15px;
+}
+.media:first-child {
+  margin-top: 0;
+}
+.media,
+.media-body {
+  zoom: 1;
+  overflow: hidden;
+}
+.media-body {
+  width: 10000px;
+}
+.media-object {
+  display: block;
+}
+.media-object.img-thumbnail {
+  max-width: none;
+}
+.media-right,
+.media > .pull-right {
+  padding-left: 10px;
+}
+.media-left,
+.media > .pull-left {
+  padding-right: 10px;
+}
+.media-left,
+.media-right,
+.media-body {
+  display: table-cell;
+  vertical-align: top;
+}
+.media-middle {
+  vertical-align: middle;
+}
+.media-bottom {
+  vertical-align: bottom;
+}
+.media-heading {
+  margin-top: 0;
+  margin-bottom: 5px;
+}
+.media-list {
+  padding-left: 0;
+  list-style: none;
+}
+.list-group {
+  margin-bottom: 20px;
+  padding-left: 0;
+}
+.list-group-item {
+  position: relative;
+  display: block;
+  padding: 10px 15px;
+  margin-bottom: -1px;
+  background-color: #fff;
+  border: 1px solid #ddd;
+}
+.list-group-item:first-child {
+  border-top-right-radius: 2px;
+  border-top-left-radius: 2px;
+}
+.list-group-item:last-child {
+  margin-bottom: 0;
+  border-bottom-right-radius: 2px;
+  border-bottom-left-radius: 2px;
+}
+a.list-group-item,
+button.list-group-item {
+  color: #555;
+}
+a.list-group-item .list-group-item-heading,
+button.list-group-item .list-group-item-heading {
+  color: #333;
+}
+a.list-group-item:hover,
+button.list-group-item:hover,
+a.list-group-item:focus,
+button.list-group-item:focus {
+  text-decoration: none;
+  color: #555;
+  background-color: #f5f5f5;
+}
+button.list-group-item {
+  width: 100%;
+  text-align: left;
+}
+.list-group-item.disabled,
+.list-group-item.disabled:hover,
+.list-group-item.disabled:focus {
+  background-color: #eeeeee;
+  color: #777777;
+  cursor: not-allowed;
+}
+.list-group-item.disabled .list-group-item-heading,
+.list-group-item.disabled:hover .list-group-item-heading,
+.list-group-item.disabled:focus .list-group-item-heading {
+  color: inherit;
+}
+.list-group-item.disabled .list-group-item-text,
+.list-group-item.disabled:hover .list-group-item-text,
+.list-group-item.disabled:focus .list-group-item-text {
+  color: #777777;
+}
+.list-group-item.active,
+.list-group-item.active:hover,
+.list-group-item.active:focus {
+  z-index: 2;
+  color: #fff;
+  background-color: #337ab7;
+  border-color: #337ab7;
+}
+.list-group-item.active .list-group-item-heading,
+.list-group-item.active:hover .list-group-item-heading,
+.list-group-item.active:focus .list-group-item-heading,
+.list-group-item.active .list-group-item-heading > small,
+.list-group-item.active:hover .list-group-item-heading > small,
+.list-group-item.active:focus .list-group-item-heading > small,
+.list-group-item.active .list-group-item-heading > .small,
+.list-group-item.active:hover .list-group-item-heading > .small,
+.list-group-item.active:focus .list-group-item-heading > .small {
+  color: inherit;
+}
+.list-group-item.active .list-group-item-text,
+.list-group-item.active:hover .list-group-item-text,
+.list-group-item.active:focus .list-group-item-text {
+  color: #c7ddef;
+}
+.list-group-item-success {
+  color: #3c763d;
+  background-color: #dff0d8;
+}
+a.list-group-item-success,
+button.list-group-item-success {
+  color: #3c763d;
+}
+a.list-group-item-success .list-group-item-heading,
+button.list-group-item-success .list-group-item-heading {
+  color: inherit;
+}
+a.list-group-item-success:hover,
+button.list-group-item-success:hover,
+a.list-group-item-success:focus,
+button.list-group-item-success:focus {
+  color: #3c763d;
+  background-color: #d0e9c6;
+}
+a.list-group-item-success.active,
+button.list-group-item-success.active,
+a.list-group-item-success.active:hover,
+button.list-group-item-success.active:hover,
+a.list-group-item-success.active:focus,
+button.list-group-item-success.active:focus {
+  color: #fff;
+  background-color: #3c763d;
+  border-color: #3c763d;
+}
+.list-group-item-info {
+  color: #31708f;
+  background-color: #d9edf7;
+}
+a.list-group-item-info,
+button.list-group-item-info {
+  color: #31708f;
+}
+a.list-group-item-info .list-group-item-heading,
+button.list-group-item-info .list-group-item-heading {
+  color: inherit;
+}
+a.list-group-item-info:hover,
+button.list-group-item-info:hover,
+a.list-group-item-info:focus,
+button.list-group-item-info:focus {
+  color: #31708f;
+  background-color: #c4e3f3;
+}
+a.list-group-item-info.active,
+button.list-group-item-info.active,
+a.list-group-item-info.active:hover,
+button.list-group-item-info.active:hover,
+a.list-group-item-info.active:focus,
+button.list-group-item-info.active:focus {
+  color: #fff;
+  background-color: #31708f;
+  border-color: #31708f;
+}
+.list-group-item-warning {
+  color: #8a6d3b;
+  background-color: #fcf8e3;
+}
+a.list-group-item-warning,
+button.list-group-item-warning {
+  color: #8a6d3b;
+}
+a.list-group-item-warning .list-group-item-heading,
+button.list-group-item-warning .list-group-item-heading {
+  color: inherit;
+}
+a.list-group-item-warning:hover,
+button.list-group-item-warning:hover,
+a.list-group-item-warning:focus,
+button.list-group-item-warning:focus {
+  color: #8a6d3b;
+  background-color: #faf2cc;
+}
+a.list-group-item-warning.active,
+button.list-group-item-warning.active,
+a.list-group-item-warning.active:hover,
+button.list-group-item-warning.active:hover,
+a.list-group-item-warning.active:focus,
+button.list-group-item-warning.active:focus {
+  color: #fff;
+  background-color: #8a6d3b;
+  border-color: #8a6d3b;
+}
+.list-group-item-danger {
+  color: #a94442;
+  background-color: #f2dede;
+}
+a.list-group-item-danger,
+button.list-group-item-danger {
+  color: #a94442;
+}
+a.list-group-item-danger .list-group-item-heading,
+button.list-group-item-danger .list-group-item-heading {
+  color: inherit;
+}
+a.list-group-item-danger:hover,
+button.list-group-item-danger:hover,
+a.list-group-item-danger:focus,
+button.list-group-item-danger:focus {
+  color: #a94442;
+  background-color: #ebcccc;
+}
+a.list-group-item-danger.active,
+button.list-group-item-danger.active,
+a.list-group-item-danger.active:hover,
+button.list-group-item-danger.active:hover,
+a.list-group-item-danger.active:focus,
+button.list-group-item-danger.active:focus {
+  color: #fff;
+  background-color: #a94442;
+  border-color: #a94442;
+}
+.list-group-item-heading {
+  margin-top: 0;
+  margin-bottom: 5px;
+}
+.list-group-item-text {
+  margin-bottom: 0;
+  line-height: 1.3;
+}
+.panel {
+  margin-bottom: 18px;
+  background-color: #fff;
+  border: 1px solid transparent;
+  border-radius: 2px;
+  -webkit-box-shadow: 0 1px 1px rgba(0, 0, 0, 0.05);
+  box-shadow: 0 1px 1px rgba(0, 0, 0, 0.05);
+}
+.panel-body {
+  padding: 15px;
+}
+.panel-heading {
+  padding: 10px 15px;
+  border-bottom: 1px solid transparent;
+  border-top-right-radius: 1px;
+  border-top-left-radius: 1px;
+}
+.panel-heading > .dropdown .dropdown-toggle {
+  color: inherit;
+}
+.panel-title {
+  margin-top: 0;
+  margin-bottom: 0;
+  font-size: 15px;
+  color: inherit;
+}
+.panel-title > a,
+.panel-title > small,
+.panel-title > .small,
+.panel-title > small > a,
+.panel-title > .small > a {
+  color: inherit;
+}
+.panel-footer {
+  padding: 10px 15px;
+  background-color: #f5f5f5;
+  border-top: 1px solid #ddd;
+  border-bottom-right-radius: 1px;
+  border-bottom-left-radius: 1px;
+}
+.panel > .list-group,
+.panel > .panel-collapse > .list-group {
+  margin-bottom: 0;
+}
+.panel > .list-group .list-group-item,
+.panel > .panel-collapse > .list-group .list-group-item {
+  border-width: 1px 0;
+  border-radius: 0;
+}
+.panel > .list-group:first-child .list-group-item:first-child,
+.panel > .panel-collapse > .list-group:first-child .list-group-item:first-child {
+  border-top: 0;
+  border-top-right-radius: 1px;
+  border-top-left-radius: 1px;
+}
+.panel > .list-group:last-child .list-group-item:last-child,
+.panel > .panel-collapse > .list-group:last-child .list-group-item:last-child {
+  border-bottom: 0;
+  border-bottom-right-radius: 1px;
+  border-bottom-left-radius: 1px;
+}
+.panel > .panel-heading + .panel-collapse > .list-group .list-group-item:first-child {
+  border-top-right-radius: 0;
+  border-top-left-radius: 0;
+}
+.panel-heading + .list-group .list-group-item:first-child {
+  border-top-width: 0;
+}
+.list-group + .panel-footer {
+  border-top-width: 0;
+}
+.panel > .table,
+.panel > .table-responsive > .table,
+.panel > .panel-collapse > .table {
+  margin-bottom: 0;
+}
+.panel > .table caption,
+.panel > .table-responsive > .table caption,
+.panel > .panel-collapse > .table caption {
+  padding-left: 15px;
+  padding-right: 15px;
+}
+.panel > .table:first-child,
+.panel > .table-responsive:first-child > .table:first-child {
+  border-top-right-radius: 1px;
+  border-top-left-radius: 1px;
+}
+.panel > .table:first-child > thead:first-child > tr:first-child,
+.panel > .table-responsive:first-child > .table:first-child > thead:first-child > tr:first-child,
+.panel > .table:first-child > tbody:first-child > tr:first-child,
+.panel > .table-responsive:first-child > .table:first-child > tbody:first-child > tr:first-child {
+  border-top-left-radius: 1px;
+  border-top-right-radius: 1px;
+}
+.panel > .table:first-child > thead:first-child > tr:first-child td:first-child,
+.panel > .table-responsive:first-child > .table:first-child > thead:first-child > tr:first-child td:first-child,
+.panel > .table:first-child > tbody:first-child > tr:first-child td:first-child,
+.panel > .table-responsive:first-child > .table:first-child > tbody:first-child > tr:first-child td:first-child,
+.panel > .table:first-child > thead:first-child > tr:first-child th:first-child,
+.panel > .table-responsive:first-child > .table:first-child > thead:first-child > tr:first-child th:first-child,
+.panel > .table:first-child > tbody:first-child > tr:first-child th:first-child,
+.panel > .table-responsive:first-child > .table:first-child > tbody:first-child > tr:first-child th:first-child {
+  border-top-left-radius: 1px;
+}
+.panel > .table:first-child > thead:first-child > tr:first-child td:last-child,
+.panel > .table-responsive:first-child > .table:first-child > thead:first-child > tr:first-child td:last-child,
+.panel > .table:first-child > tbody:first-child > tr:first-child td:last-child,
+.panel > .table-responsive:first-child > .table:first-child > tbody:first-child > tr:first-child td:last-child,
+.panel > .table:first-child > thead:first-child > tr:first-child th:last-child,
+.panel > .table-responsive:first-child > .table:first-child > thead:first-child > tr:first-child th:last-child,
+.panel > .table:first-child > tbody:first-child > tr:first-child th:last-child,
+.panel > .table-responsive:first-child > .table:first-child > tbody:first-child > tr:first-child th:last-child {
+  border-top-right-radius: 1px;
+}
+.panel > .table:last-child,
+.panel > .table-responsive:last-child > .table:last-child {
+  border-bottom-right-radius: 1px;
+  border-bottom-left-radius: 1px;
+}
+.panel > .table:last-child > tbody:last-child > tr:last-child,
+.panel > .table-responsive:last-child > .table:last-child > tbody:last-child > tr:last-child,
+.panel > .table:last-child > tfoot:last-child > tr:last-child,
+.panel > .table-responsive:last-child > .table:last-child > tfoot:last-child > tr:last-child {
+  border-bottom-left-radius: 1px;
+  border-bottom-right-radius: 1px;
+}
+.panel > .table:last-child > tbody:last-child > tr:last-child td:first-child,
+.panel > .table-responsive:last-child > .table:last-child > tbody:last-child > tr:last-child td:first-child,
+.panel > .table:last-child > tfoot:last-child > tr:last-child td:first-child,
+.panel > .table-responsive:last-child > .table:last-child > tfoot:last-child > tr:last-child td:first-child,
+.panel > .table:last-child > tbody:last-child > tr:last-child th:first-child,
+.panel > .table-responsive:last-child > .table:last-child > tbody:last-child > tr:last-child th:first-child,
+.panel > .table:last-child > tfoot:last-child > tr:last-child th:first-child,
+.panel > .table-responsive:last-child > .table:last-child > tfoot:last-child > tr:last-child th:first-child {
+  border-bottom-left-radius: 1px;
+}
+.panel > .table:last-child > tbody:last-child > tr:last-child td:last-child,
+.panel > .table-responsive:last-child > .table:last-child > tbody:last-child > tr:last-child td:last-child,
+.panel > .table:last-child > tfoot:last-child > tr:last-child td:last-child,
+.panel > .table-responsive:last-child > .table:last-child > tfoot:last-child > tr:last-child td:last-child,
+.panel > .table:last-child > tbody:last-child > tr:last-child th:last-child,
+.panel > .table-responsive:last-child > .table:last-child > tbody:last-child > tr:last-child th:last-child,
+.panel > .table:last-child > tfoot:last-child > tr:last-child th:last-child,
+.panel > .table-responsive:last-child > .table:last-child > tfoot:last-child > tr:last-child th:last-child {
+  border-bottom-right-radius: 1px;
+}
+.panel > .panel-body + .table,
+.panel > .panel-body + .table-responsive,
+.panel > .table + .panel-body,
+.panel > .table-responsive + .panel-body {
+  border-top: 1px solid #ddd;
+}
+.panel > .table > tbody:first-child > tr:first-child th,
+.panel > .table > tbody:first-child > tr:first-child td {
+  border-top: 0;
+}
+.panel > .table-bordered,
+.panel > .table-responsive > .table-bordered {
+  border: 0;
+}
+.panel > .table-bordered > thead > tr > th:first-child,
+.panel > .table-responsive > .table-bordered > thead > tr > th:first-child,
+.panel > .table-bordered > tbody > tr > th:first-child,
+.panel > .table-responsive > .table-bordered > tbody > tr > th:first-child,
+.panel > .table-bordered > tfoot > tr > th:first-child,
+.panel > .table-responsive > .table-bordered > tfoot > tr > th:first-child,
+.panel > .table-bordered > thead > tr > td:first-child,
+.panel > .table-responsive > .table-bordered > thead > tr > td:first-child,
+.panel > .table-bordered > tbody > tr > td:first-child,
+.panel > .table-responsive > .table-bordered > tbody > tr > td:first-child,
+.panel > .table-bordered > tfoot > tr > td:first-child,
+.panel > .table-responsive > .table-bordered > tfoot > tr > td:first-child {
+  border-left: 0;
+}
+.panel > .table-bordered > thead > tr > th:last-child,
+.panel > .table-responsive > .table-bordered > thead > tr > th:last-child,
+.panel > .table-bordered > tbody > tr > th:last-child,
+.panel > .table-responsive > .table-bordered > tbody > tr > th:last-child,
+.panel > .table-bordered > tfoot > tr > th:last-child,
+.panel > .table-responsive > .table-bordered > tfoot > tr > th:last-child,
+.panel > .table-bordered > thead > tr > td:last-child,
+.panel > .table-responsive > .table-bordered > thead > tr > td:last-child,
+.panel > .table-bordered > tbody > tr > td:last-child,
+.panel > .table-responsive > .table-bordered > tbody > tr > td:last-child,
+.panel > .table-bordered > tfoot > tr > td:last-child,
+.panel > .table-responsive > .table-bordered > tfoot > tr > td:last-child {
+  border-right: 0;
+}
+.panel > .table-bordered > thead > tr:first-child > td,
+.panel > .table-responsive > .table-bordered > thead > tr:first-child > td,
+.panel > .table-bordered > tbody > tr:first-child > td,
+.panel > .table-responsive > .table-bordered > tbody > tr:first-child > td,
+.panel > .table-bordered > thead > tr:first-child > th,
+.panel > .table-responsive > .table-bordered > thead > tr:first-child > th,
+.panel > .table-bordered > tbody > tr:first-child > th,
+.panel > .table-responsive > .table-bordered > tbody > tr:first-child > th {
+  border-bottom: 0;
+}
+.panel > .table-bordered > tbody > tr:last-child > td,
+.panel > .table-responsive > .table-bordered > tbody > tr:last-child > td,
+.panel > .table-bordered > tfoot > tr:last-child > td,
+.panel > .table-responsive > .table-bordered > tfoot > tr:last-child > td,
+.panel > .table-bordered > tbody > tr:last-child > th,
+.panel > .table-responsive > .table-bordered > tbody > tr:last-child > th,
+.panel > .table-bordered > tfoot > tr:last-child > th,
+.panel > .table-responsive > .table-bordered > tfoot > tr:last-child > th {
+  border-bottom: 0;
+}
+.panel > .table-responsive {
+  border: 0;
+  margin-bottom: 0;
+}
+.panel-group {
+  margin-bottom: 18px;
+}
+.panel-group .panel {
+  margin-bottom: 0;
+  border-radius: 2px;
+}
+.panel-group .panel + .panel {
+  margin-top: 5px;
+}
+.panel-group .panel-heading {
+  border-bottom: 0;
+}
+.panel-group .panel-heading + .panel-collapse > .panel-body,
+.panel-group .panel-heading + .panel-collapse > .list-group {
+  border-top: 1px solid #ddd;
+}
+.panel-group .panel-footer {
+  border-top: 0;
+}
+.panel-group .panel-footer + .panel-collapse .panel-body {
+  border-bottom: 1px solid #ddd;
+}
+.panel-default {
+  border-color: #ddd;
+}
+.panel-default > .panel-heading {
+  color: #333333;
+  background-color: #f5f5f5;
+  border-color: #ddd;
+}
+.panel-default > .panel-heading + .panel-collapse > .panel-body {
+  border-top-color: #ddd;
+}
+.panel-default > .panel-heading .badge {
+  color: #f5f5f5;
+  background-color: #333333;
+}
+.panel-default > .panel-footer + .panel-collapse > .panel-body {
+  border-bottom-color: #ddd;
+}
+.panel-primary {
+  border-color: #337ab7;
+}
+.panel-primary > .panel-heading {
+  color: #fff;
+  background-color: #337ab7;
+  border-color: #337ab7;
+}
+.panel-primary > .panel-heading + .panel-collapse > .panel-body {
+  border-top-color: #337ab7;
+}
+.panel-primary > .panel-heading .badge {
+  color: #337ab7;
+  background-color: #fff;
+}
+.panel-primary > .panel-footer + .panel-collapse > .panel-body {
+  border-bottom-color: #337ab7;
+}
+.panel-success {
+  border-color: #d6e9c6;
+}
+.panel-success > .panel-heading {
+  color: #3c763d;
+  background-color: #dff0d8;
+  border-color: #d6e9c6;
+}
+.panel-success > .panel-heading + .panel-collapse > .panel-body {
+  border-top-color: #d6e9c6;
+}
+.panel-success > .panel-heading .badge {
+  color: #dff0d8;
+  background-color: #3c763d;
+}
+.panel-success > .panel-footer + .panel-collapse > .panel-body {
+  border-bottom-color: #d6e9c6;
+}
+.panel-info {
+  border-color: #bce8f1;
+}
+.panel-info > .panel-heading {
+  color: #31708f;
+  background-color: #d9edf7;
+  border-color: #bce8f1;
+}
+.panel-info > .panel-heading + .panel-collapse > .panel-body {
+  border-top-color: #bce8f1;
+}
+.panel-info > .panel-heading .badge {
+  color: #d9edf7;
+  background-color: #31708f;
+}
+.panel-info > .panel-footer + .panel-collapse > .panel-body {
+  border-bottom-color: #bce8f1;
+}
+.panel-warning {
+  border-color: #faebcc;
+}
+.panel-warning > .panel-heading {
+  color: #8a6d3b;
+  background-color: #fcf8e3;
+  border-color: #faebcc;
+}
+.panel-warning > .panel-heading + .panel-collapse > .panel-body {
+  border-top-color: #faebcc;
+}
+.panel-warning > .panel-heading .badge {
+  color: #fcf8e3;
+  background-color: #8a6d3b;
+}
+.panel-warning > .panel-footer + .panel-collapse > .panel-body {
+  border-bottom-color: #faebcc;
+}
+.panel-danger {
+  border-color: #ebccd1;
+}
+.panel-danger > .panel-heading {
+  color: #a94442;
+  background-color: #f2dede;
+  border-color: #ebccd1;
+}
+.panel-danger > .panel-heading + .panel-collapse > .panel-body {
+  border-top-color: #ebccd1;
+}
+.panel-danger > .panel-heading .badge {
+  color: #f2dede;
+  background-color: #a94442;
+}
+.panel-danger > .panel-footer + .panel-collapse > .panel-body {
+  border-bottom-color: #ebccd1;
+}
+.embed-responsive {
+  position: relative;
+  display: block;
+  height: 0;
+  padding: 0;
+  overflow: hidden;
+}
+.embed-responsive .embed-responsive-item,
+.embed-responsive iframe,
+.embed-responsive embed,
+.embed-responsive object,
+.embed-responsive video {
+  position: absolute;
+  top: 0;
+  left: 0;
+  bottom: 0;
+  height: 100%;
+  width: 100%;
+  border: 0;
+}
+.embed-responsive-16by9 {
+  padding-bottom: 56.25%;
+}
+.embed-responsive-4by3 {
+  padding-bottom: 75%;
+}
+.well {
+  min-height: 20px;
+  padding: 19px;
+  margin-bottom: 20px;
+  background-color: #f5f5f5;
+  border: 1px solid #e3e3e3;
+  border-radius: 2px;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.05);
+  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.05);
+}
+.well blockquote {
+  border-color: #ddd;
+  border-color: rgba(0, 0, 0, 0.15);
+}
+.well-lg {
+  padding: 24px;
+  border-radius: 3px;
+}
+.well-sm {
+  padding: 9px;
+  border-radius: 1px;
+}
+.close {
+  float: right;
+  font-size: 19.5px;
+  font-weight: bold;
+  line-height: 1;
+  color: #000;
+  text-shadow: 0 1px 0 #fff;
+  opacity: 0.2;
+  filter: alpha(opacity=20);
+}
+.close:hover,
+.close:focus {
+  color: #000;
+  text-decoration: none;
+  cursor: pointer;
+  opacity: 0.5;
+  filter: alpha(opacity=50);
+}
+button.close {
+  padding: 0;
+  cursor: pointer;
+  background: transparent;
+  border: 0;
+  -webkit-appearance: none;
+}
+.modal-open {
+  overflow: hidden;
+}
+.modal {
+  display: none;
+  overflow: hidden;
+  position: fixed;
+  top: 0;
+  right: 0;
+  bottom: 0;
+  left: 0;
+  z-index: 1050;
+  -webkit-overflow-scrolling: touch;
+  outline: 0;
+}
+.modal.fade .modal-dialog {
+  -webkit-transform: translate(0, -25%);
+  -ms-transform: translate(0, -25%);
+  -o-transform: translate(0, -25%);
+  transform: translate(0, -25%);
+  -webkit-transition: -webkit-transform 0.3s ease-out;
+  -moz-transition: -moz-transform 0.3s ease-out;
+  -o-transition: -o-transform 0.3s ease-out;
+  transition: transform 0.3s ease-out;
+}
+.modal.in .modal-dialog {
+  -webkit-transform: translate(0, 0);
+  -ms-transform: translate(0, 0);
+  -o-transform: translate(0, 0);
+  transform: translate(0, 0);
+}
+.modal-open .modal {
+  overflow-x: hidden;
+  overflow-y: auto;
+}
+.modal-dialog {
+  position: relative;
+  width: auto;
+  margin: 10px;
+}
+.modal-content {
+  position: relative;
+  background-color: #fff;
+  border: 1px solid #999;
+  border: 1px solid rgba(0, 0, 0, 0.2);
+  border-radius: 3px;
+  -webkit-box-shadow: 0 3px 9px rgba(0, 0, 0, 0.5);
+  box-shadow: 0 3px 9px rgba(0, 0, 0, 0.5);
+  background-clip: padding-box;
+  outline: 0;
+}
+.modal-backdrop {
+  position: fixed;
+  top: 0;
+  right: 0;
+  bottom: 0;
+  left: 0;
+  z-index: 1040;
+  background-color: #000;
+}
+.modal-backdrop.fade {
+  opacity: 0;
+  filter: alpha(opacity=0);
+}
+.modal-backdrop.in {
+  opacity: 0.5;
+  filter: alpha(opacity=50);
+}
+.modal-header {
+  padding: 15px;
+  border-bottom: 1px solid #e5e5e5;
+}
+.modal-header .close {
+  margin-top: -2px;
+}
+.modal-title {
+  margin: 0;
+  line-height: 1.42857143;
+}
+.modal-body {
+  position: relative;
+  padding: 15px;
+}
+.modal-footer {
+  padding: 15px;
+  text-align: right;
+  border-top: 1px solid #e5e5e5;
+}
+.modal-footer .btn + .btn {
+  margin-left: 5px;
+  margin-bottom: 0;
+}
+.modal-footer .btn-group .btn + .btn {
+  margin-left: -1px;
+}
+.modal-footer .btn-block + .btn-block {
+  margin-left: 0;
+}
+.modal-scrollbar-measure {
+  position: absolute;
+  top: -9999px;
+  width: 50px;
+  height: 50px;
+  overflow: scroll;
+}
+@media (min-width: 768px) {
+  .modal-dialog {
+    width: 600px;
+    margin: 30px auto;
+  }
+  .modal-content {
+    -webkit-box-shadow: 0 5px 15px rgba(0, 0, 0, 0.5);
+    box-shadow: 0 5px 15px rgba(0, 0, 0, 0.5);
+  }
+  .modal-sm {
+    width: 300px;
+  }
+}
+@media (min-width: 992px) {
+  .modal-lg {
+    width: 900px;
+  }
+}
+.tooltip {
+  position: absolute;
+  z-index: 1070;
+  display: block;
+  font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
+  font-style: normal;
+  font-weight: normal;
+  letter-spacing: normal;
+  line-break: auto;
+  line-height: 1.42857143;
+  text-align: left;
+  text-align: start;
+  text-decoration: none;
+  text-shadow: none;
+  text-transform: none;
+  white-space: normal;
+  word-break: normal;
+  word-spacing: normal;
+  word-wrap: normal;
+  font-size: 12px;
+  opacity: 0;
+  filter: alpha(opacity=0);
+}
+.tooltip.in {
+  opacity: 0.9;
+  filter: alpha(opacity=90);
+}
+.tooltip.top {
+  margin-top: -3px;
+  padding: 5px 0;
+}
+.tooltip.right {
+  margin-left: 3px;
+  padding: 0 5px;
+}
+.tooltip.bottom {
+  margin-top: 3px;
+  padding: 5px 0;
+}
+.tooltip.left {
+  margin-left: -3px;
+  padding: 0 5px;
+}
+.tooltip-inner {
+  max-width: 200px;
+  padding: 3px 8px;
+  color: #fff;
+  text-align: center;
+  background-color: #000;
+  border-radius: 2px;
+}
+.tooltip-arrow {
+  position: absolute;
+  width: 0;
+  height: 0;
+  border-color: transparent;
+  border-style: solid;
+}
+.tooltip.top .tooltip-arrow {
+  bottom: 0;
+  left: 50%;
+  margin-left: -5px;
+  border-width: 5px 5px 0;
+  border-top-color: #000;
+}
+.tooltip.top-left .tooltip-arrow {
+  bottom: 0;
+  right: 5px;
+  margin-bottom: -5px;
+  border-width: 5px 5px 0;
+  border-top-color: #000;
+}
+.tooltip.top-right .tooltip-arrow {
+  bottom: 0;
+  left: 5px;
+  margin-bottom: -5px;
+  border-width: 5px 5px 0;
+  border-top-color: #000;
+}
+.tooltip.right .tooltip-arrow {
+  top: 50%;
+  left: 0;
+  margin-top: -5px;
+  border-width: 5px 5px 5px 0;
+  border-right-color: #000;
+}
+.tooltip.left .tooltip-arrow {
+  top: 50%;
+  right: 0;
+  margin-top: -5px;
+  border-width: 5px 0 5px 5px;
+  border-left-color: #000;
+}
+.tooltip.bottom .tooltip-arrow {
+  top: 0;
+  left: 50%;
+  margin-left: -5px;
+  border-width: 0 5px 5px;
+  border-bottom-color: #000;
+}
+.tooltip.bottom-left .tooltip-arrow {
+  top: 0;
+  right: 5px;
+  margin-top: -5px;
+  border-width: 0 5px 5px;
+  border-bottom-color: #000;
+}
+.tooltip.bottom-right .tooltip-arrow {
+  top: 0;
+  left: 5px;
+  margin-top: -5px;
+  border-width: 0 5px 5px;
+  border-bottom-color: #000;
+}
+.popover {
+  position: absolute;
+  top: 0;
+  left: 0;
+  z-index: 1060;
+  display: none;
+  max-width: 276px;
+  padding: 1px;
+  font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
+  font-style: normal;
+  font-weight: normal;
+  letter-spacing: normal;
+  line-break: auto;
+  line-height: 1.42857143;
+  text-align: left;
+  text-align: start;
+  text-decoration: none;
+  text-shadow: none;
+  text-transform: none;
+  white-space: normal;
+  word-break: normal;
+  word-spacing: normal;
+  word-wrap: normal;
+  font-size: 13px;
+  background-color: #fff;
+  background-clip: padding-box;
+  border: 1px solid #ccc;
+  border: 1px solid rgba(0, 0, 0, 0.2);
+  border-radius: 3px;
+  -webkit-box-shadow: 0 5px 10px rgba(0, 0, 0, 0.2);
+  box-shadow: 0 5px 10px rgba(0, 0, 0, 0.2);
+}
+.popover.top {
+  margin-top: -10px;
+}
+.popover.right {
+  margin-left: 10px;
+}
+.popover.bottom {
+  margin-top: 10px;
+}
+.popover.left {
+  margin-left: -10px;
+}
+.popover-title {
+  margin: 0;
+  padding: 8px 14px;
+  font-size: 13px;
+  background-color: #f7f7f7;
+  border-bottom: 1px solid #ebebeb;
+  border-radius: 2px 2px 0 0;
+}
+.popover-content {
+  padding: 9px 14px;
+}
+.popover > .arrow,
+.popover > .arrow:after {
+  position: absolute;
+  display: block;
+  width: 0;
+  height: 0;
+  border-color: transparent;
+  border-style: solid;
+}
+.popover > .arrow {
+  border-width: 11px;
+}
+.popover > .arrow:after {
+  border-width: 10px;
+  content: "";
+}
+.popover.top > .arrow {
+  left: 50%;
+  margin-left: -11px;
+  border-bottom-width: 0;
+  border-top-color: #999999;
+  border-top-color: rgba(0, 0, 0, 0.25);
+  bottom: -11px;
+}
+.popover.top > .arrow:after {
+  content: " ";
+  bottom: 1px;
+  margin-left: -10px;
+  border-bottom-width: 0;
+  border-top-color: #fff;
+}
+.popover.right > .arrow {
+  top: 50%;
+  left: -11px;
+  margin-top: -11px;
+  border-left-width: 0;
+  border-right-color: #999999;
+  border-right-color: rgba(0, 0, 0, 0.25);
+}
+.popover.right > .arrow:after {
+  content: " ";
+  left: 1px;
+  bottom: -10px;
+  border-left-width: 0;
+  border-right-color: #fff;
+}
+.popover.bottom > .arrow {
+  left: 50%;
+  margin-left: -11px;
+  border-top-width: 0;
+  border-bottom-color: #999999;
+  border-bottom-color: rgba(0, 0, 0, 0.25);
+  top: -11px;
+}
+.popover.bottom > .arrow:after {
+  content: " ";
+  top: 1px;
+  margin-left: -10px;
+  border-top-width: 0;
+  border-bottom-color: #fff;
+}
+.popover.left > .arrow {
+  top: 50%;
+  right: -11px;
+  margin-top: -11px;
+  border-right-width: 0;
+  border-left-color: #999999;
+  border-left-color: rgba(0, 0, 0, 0.25);
+}
+.popover.left > .arrow:after {
+  content: " ";
+  right: 1px;
+  border-right-width: 0;
+  border-left-color: #fff;
+  bottom: -10px;
+}
+.carousel {
+  position: relative;
+}
+.carousel-inner {
+  position: relative;
+  overflow: hidden;
+  width: 100%;
+}
+.carousel-inner > .item {
+  display: none;
+  position: relative;
+  -webkit-transition: 0.6s ease-in-out left;
+  -o-transition: 0.6s ease-in-out left;
+  transition: 0.6s ease-in-out left;
+}
+.carousel-inner > .item > img,
+.carousel-inner > .item > a > img {
+  line-height: 1;
+}
+@media all and (transform-3d), (-webkit-transform-3d) {
+  .carousel-inner > .item {
+    -webkit-transition: -webkit-transform 0.6s ease-in-out;
+    -moz-transition: -moz-transform 0.6s ease-in-out;
+    -o-transition: -o-transform 0.6s ease-in-out;
+    transition: transform 0.6s ease-in-out;
+    -webkit-backface-visibility: hidden;
+    -moz-backface-visibility: hidden;
+    backface-visibility: hidden;
+    -webkit-perspective: 1000px;
+    -moz-perspective: 1000px;
+    perspective: 1000px;
+  }
+  .carousel-inner > .item.next,
+  .carousel-inner > .item.active.right {
+    -webkit-transform: translate3d(100%, 0, 0);
+    transform: translate3d(100%, 0, 0);
+    left: 0;
+  }
+  .carousel-inner > .item.prev,
+  .carousel-inner > .item.active.left {
+    -webkit-transform: translate3d(-100%, 0, 0);
+    transform: translate3d(-100%, 0, 0);
+    left: 0;
+  }
+  .carousel-inner > .item.next.left,
+  .carousel-inner > .item.prev.right,
+  .carousel-inner > .item.active {
+    -webkit-transform: translate3d(0, 0, 0);
+    transform: translate3d(0, 0, 0);
+    left: 0;
+  }
+}
+.carousel-inner > .active,
+.carousel-inner > .next,
+.carousel-inner > .prev {
+  display: block;
+}
+.carousel-inner > .active {
+  left: 0;
+}
+.carousel-inner > .next,
+.carousel-inner > .prev {
+  position: absolute;
+  top: 0;
+  width: 100%;
+}
+.carousel-inner > .next {
+  left: 100%;
+}
+.carousel-inner > .prev {
+  left: -100%;
+}
+.carousel-inner > .next.left,
+.carousel-inner > .prev.right {
+  left: 0;
+}
+.carousel-inner > .active.left {
+  left: -100%;
+}
+.carousel-inner > .active.right {
+  left: 100%;
+}
+.carousel-control {
+  position: absolute;
+  top: 0;
+  left: 0;
+  bottom: 0;
+  width: 15%;
+  opacity: 0.5;
+  filter: alpha(opacity=50);
+  font-size: 20px;
+  color: #fff;
+  text-align: center;
+  text-shadow: 0 1px 2px rgba(0, 0, 0, 0.6);
+  background-color: rgba(0, 0, 0, 0);
+}
+.carousel-control.left {
+  background-image: -webkit-linear-gradient(left, rgba(0, 0, 0, 0.5) 0%, rgba(0, 0, 0, 0.0001) 100%);
+  background-image: -o-linear-gradient(left, rgba(0, 0, 0, 0.5) 0%, rgba(0, 0, 0, 0.0001) 100%);
+  background-image: linear-gradient(to right, rgba(0, 0, 0, 0.5) 0%, rgba(0, 0, 0, 0.0001) 100%);
+  background-repeat: repeat-x;
+  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#80000000', endColorstr='#00000000', GradientType=1);
+}
+.carousel-control.right {
+  left: auto;
+  right: 0;
+  background-image: -webkit-linear-gradient(left, rgba(0, 0, 0, 0.0001) 0%, rgba(0, 0, 0, 0.5) 100%);
+  background-image: -o-linear-gradient(left, rgba(0, 0, 0, 0.0001) 0%, rgba(0, 0, 0, 0.5) 100%);
+  background-image: linear-gradient(to right, rgba(0, 0, 0, 0.0001) 0%, rgba(0, 0, 0, 0.5) 100%);
+  background-repeat: repeat-x;
+  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#00000000', endColorstr='#80000000', GradientType=1);
+}
+.carousel-control:hover,
+.carousel-control:focus {
+  outline: 0;
+  color: #fff;
+  text-decoration: none;
+  opacity: 0.9;
+  filter: alpha(opacity=90);
+}
+.carousel-control .icon-prev,
+.carousel-control .icon-next,
+.carousel-control .glyphicon-chevron-left,
+.carousel-control .glyphicon-chevron-right {
+  position: absolute;
+  top: 50%;
+  margin-top: -10px;
+  z-index: 5;
+  display: inline-block;
+}
+.carousel-control .icon-prev,
+.carousel-control .glyphicon-chevron-left {
+  left: 50%;
+  margin-left: -10px;
+}
+.carousel-control .icon-next,
+.carousel-control .glyphicon-chevron-right {
+  right: 50%;
+  margin-right: -10px;
+}
+.carousel-control .icon-prev,
+.carousel-control .icon-next {
+  width: 20px;
+  height: 20px;
+  line-height: 1;
+  font-family: serif;
+}
+.carousel-control .icon-prev:before {
+  content: '\2039';
+}
+.carousel-control .icon-next:before {
+  content: '\203a';
+}
+.carousel-indicators {
+  position: absolute;
+  bottom: 10px;
+  left: 50%;
+  z-index: 15;
+  width: 60%;
+  margin-left: -30%;
+  padding-left: 0;
+  list-style: none;
+  text-align: center;
+}
+.carousel-indicators li {
+  display: inline-block;
+  width: 10px;
+  height: 10px;
+  margin: 1px;
+  text-indent: -999px;
+  border: 1px solid #fff;
+  border-radius: 10px;
+  cursor: pointer;
+  background-color: #000 \9;
+  background-color: rgba(0, 0, 0, 0);
+}
+.carousel-indicators .active {
+  margin: 0;
+  width: 12px;
+  height: 12px;
+  background-color: #fff;
+}
+.carousel-caption {
+  position: absolute;
+  left: 15%;
+  right: 15%;
+  bottom: 20px;
+  z-index: 10;
+  padding-top: 20px;
+  padding-bottom: 20px;
+  color: #fff;
+  text-align: center;
+  text-shadow: 0 1px 2px rgba(0, 0, 0, 0.6);
+}
+.carousel-caption .btn {
+  text-shadow: none;
+}
+@media screen and (min-width: 768px) {
+  .carousel-control .glyphicon-chevron-left,
+  .carousel-control .glyphicon-chevron-right,
+  .carousel-control .icon-prev,
+  .carousel-control .icon-next {
+    width: 30px;
+    height: 30px;
+    margin-top: -10px;
+    font-size: 30px;
+  }
+  .carousel-control .glyphicon-chevron-left,
+  .carousel-control .icon-prev {
+    margin-left: -10px;
+  }
+  .carousel-control .glyphicon-chevron-right,
+  .carousel-control .icon-next {
+    margin-right: -10px;
+  }
+  .carousel-caption {
+    left: 20%;
+    right: 20%;
+    padding-bottom: 30px;
+  }
+  .carousel-indicators {
+    bottom: 20px;
+  }
+}
+.clearfix:before,
+.clearfix:after,
+.dl-horizontal dd:before,
+.dl-horizontal dd:after,
+.container:before,
+.container:after,
+.container-fluid:before,
+.container-fluid:after,
+.row:before,
+.row:after,
+.form-horizontal .form-group:before,
+.form-horizontal .form-group:after,
+.btn-toolbar:before,
+.btn-toolbar:after,
+.btn-group-vertical > .btn-group:before,
+.btn-group-vertical > .btn-group:after,
+.nav:before,
+.nav:after,
+.navbar:before,
+.navbar:after,
+.navbar-header:before,
+.navbar-header:after,
+.navbar-collapse:before,
+.navbar-collapse:after,
+.pager:before,
+.pager:after,
+.panel-body:before,
+.panel-body:after,
+.modal-header:before,
+.modal-header:after,
+.modal-footer:before,
+.modal-footer:after,
+.item_buttons:before,
+.item_buttons:after {
+  content: " ";
+  display: table;
+}
+.clearfix:after,
+.dl-horizontal dd:after,
+.container:after,
+.container-fluid:after,
+.row:after,
+.form-horizontal .form-group:after,
+.btn-toolbar:after,
+.btn-group-vertical > .btn-group:after,
+.nav:after,
+.navbar:after,
+.navbar-header:after,
+.navbar-collapse:after,
+.pager:after,
+.panel-body:after,
+.modal-header:after,
+.modal-footer:after,
+.item_buttons:after {
+  clear: both;
+}
+.center-block {
+  display: block;
+  margin-left: auto;
+  margin-right: auto;
+}
+.pull-right {
+  float: right !important;
+}
+.pull-left {
+  float: left !important;
+}
+.hide {
+  display: none !important;
+}
+.show {
+  display: block !important;
+}
+.invisible {
+  visibility: hidden;
+}
+.text-hide {
+  font: 0/0 a;
+  color: transparent;
+  text-shadow: none;
+  background-color: transparent;
+  border: 0;
+}
+.hidden {
+  display: none !important;
+}
+.affix {
+  position: fixed;
+}
+@-ms-viewport {
+  width: device-width;
+}
+.visible-xs,
+.visible-sm,
+.visible-md,
+.visible-lg {
+  display: none !important;
+}
+.visible-xs-block,
+.visible-xs-inline,
+.visible-xs-inline-block,
+.visible-sm-block,
+.visible-sm-inline,
+.visible-sm-inline-block,
+.visible-md-block,
+.visible-md-inline,
+.visible-md-inline-block,
+.visible-lg-block,
+.visible-lg-inline,
+.visible-lg-inline-block {
+  display: none !important;
+}
+@media (max-width: 767px) {
+  .visible-xs {
+    display: block !important;
+  }
+  table.visible-xs {
+    display: table !important;
+  }
+  tr.visible-xs {
+    display: table-row !important;
+  }
+  th.visible-xs,
+  td.visible-xs {
+    display: table-cell !important;
+  }
+}
+@media (max-width: 767px) {
+  .visible-xs-block {
+    display: block !important;
+  }
+}
+@media (max-width: 767px) {
+  .visible-xs-inline {
+    display: inline !important;
+  }
+}
+@media (max-width: 767px) {
+  .visible-xs-inline-block {
+    display: inline-block !important;
+  }
+}
+@media (min-width: 768px) and (max-width: 991px) {
+  .visible-sm {
+    display: block !important;
+  }
+  table.visible-sm {
+    display: table !important;
+  }
+  tr.visible-sm {
+    display: table-row !important;
+  }
+  th.visible-sm,
+  td.visible-sm {
+    display: table-cell !important;
+  }
+}
+@media (min-width: 768px) and (max-width: 991px) {
+  .visible-sm-block {
+    display: block !important;
+  }
+}
+@media (min-width: 768px) and (max-width: 991px) {
+  .visible-sm-inline {
+    display: inline !important;
+  }
+}
+@media (min-width: 768px) and (max-width: 991px) {
+  .visible-sm-inline-block {
+    display: inline-block !important;
+  }
+}
+@media (min-width: 992px) and (max-width: 1199px) {
+  .visible-md {
+    display: block !important;
+  }
+  table.visible-md {
+    display: table !important;
+  }
+  tr.visible-md {
+    display: table-row !important;
+  }
+  th.visible-md,
+  td.visible-md {
+    display: table-cell !important;
+  }
+}
+@media (min-width: 992px) and (max-width: 1199px) {
+  .visible-md-block {
+    display: block !important;
+  }
+}
+@media (min-width: 992px) and (max-width: 1199px) {
+  .visible-md-inline {
+    display: inline !important;
+  }
+}
+@media (min-width: 992px) and (max-width: 1199px) {
+  .visible-md-inline-block {
+    display: inline-block !important;
+  }
+}
+@media (min-width: 1200px) {
+  .visible-lg {
+    display: block !important;
+  }
+  table.visible-lg {
+    display: table !important;
+  }
+  tr.visible-lg {
+    display: table-row !important;
+  }
+  th.visible-lg,
+  td.visible-lg {
+    display: table-cell !important;
+  }
+}
+@media (min-width: 1200px) {
+  .visible-lg-block {
+    display: block !important;
+  }
+}
+@media (min-width: 1200px) {
+  .visible-lg-inline {
+    display: inline !important;
+  }
+}
+@media (min-width: 1200px) {
+  .visible-lg-inline-block {
+    display: inline-block !important;
+  }
+}
+@media (max-width: 767px) {
+  .hidden-xs {
+    display: none !important;
+  }
+}
+@media (min-width: 768px) and (max-width: 991px) {
+  .hidden-sm {
+    display: none !important;
+  }
+}
+@media (min-width: 992px) and (max-width: 1199px) {
+  .hidden-md {
+    display: none !important;
+  }
+}
+@media (min-width: 1200px) {
+  .hidden-lg {
+    display: none !important;
+  }
+}
+.visible-print {
+  display: none !important;
+}
+@media print {
+  .visible-print {
+    display: block !important;
+  }
+  table.visible-print {
+    display: table !important;
+  }
+  tr.visible-print {
+    display: table-row !important;
+  }
+  th.visible-print,
+  td.visible-print {
+    display: table-cell !important;
+  }
+}
+.visible-print-block {
+  display: none !important;
+}
+@media print {
+  .visible-print-block {
+    display: block !important;
+  }
+}
+.visible-print-inline {
+  display: none !important;
+}
+@media print {
+  .visible-print-inline {
+    display: inline !important;
+  }
+}
+.visible-print-inline-block {
+  display: none !important;
+}
+@media print {
+  .visible-print-inline-block {
+    display: inline-block !important;
+  }
+}
+@media print {
+  .hidden-print {
+    display: none !important;
+  }
+}
+/*!
+*
+* Font Awesome
+*
+*/
+/*!
+ *  Font Awesome 4.2.0 by @davegandy - http://fontawesome.io - @fontawesome
+ *  License - http://fontawesome.io/license (Font: SIL OFL 1.1, CSS: MIT License)
+ */
+/* FONT PATH
+ * -------------------------- */
+@font-face {
+  font-family: 'FontAwesome';
+  src: url('../components/font-awesome/fonts/fontawesome-webfont.eot?v=4.2.0');
+  src: url('../components/font-awesome/fonts/fontawesome-webfont.eot?#iefix&v=4.2.0') format('embedded-opentype'), url('../components/font-awesome/fonts/fontawesome-webfont.woff?v=4.2.0') format('woff'), url('../components/font-awesome/fonts/fontawesome-webfont.ttf?v=4.2.0') format('truetype'), url('../components/font-awesome/fonts/fontawesome-webfont.svg?v=4.2.0#fontawesomeregular') format('svg');
+  font-weight: normal;
+  font-style: normal;
+}
+.fa {
+  display: inline-block;
+  font: normal normal normal 14px/1 FontAwesome;
+  font-size: inherit;
+  text-rendering: auto;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+}
+/* makes the font 33% larger relative to the icon container */
+.fa-lg {
+  font-size: 1.33333333em;
+  line-height: 0.75em;
+  vertical-align: -15%;
+}
+.fa-2x {
+  font-size: 2em;
+}
+.fa-3x {
+  font-size: 3em;
+}
+.fa-4x {
+  font-size: 4em;
+}
+.fa-5x {
+  font-size: 5em;
+}
+.fa-fw {
+  width: 1.28571429em;
+  text-align: center;
+}
+.fa-ul {
+  padding-left: 0;
+  margin-left: 2.14285714em;
+  list-style-type: none;
+}
+.fa-ul > li {
+  position: relative;
+}
+.fa-li {
+  position: absolute;
+  left: -2.14285714em;
+  width: 2.14285714em;
+  top: 0.14285714em;
+  text-align: center;
+}
+.fa-li.fa-lg {
+  left: -1.85714286em;
+}
+.fa-border {
+  padding: .2em .25em .15em;
+  border: solid 0.08em #eee;
+  border-radius: .1em;
+}
+.pull-right {
+  float: right;
+}
+.pull-left {
+  float: left;
+}
+.fa.pull-left {
+  margin-right: .3em;
+}
+.fa.pull-right {
+  margin-left: .3em;
+}
+.fa-spin {
+  -webkit-animation: fa-spin 2s infinite linear;
+  animation: fa-spin 2s infinite linear;
+}
+@-webkit-keyframes fa-spin {
+  0% {
+    -webkit-transform: rotate(0deg);
+    transform: rotate(0deg);
+  }
+  100% {
+    -webkit-transform: rotate(359deg);
+    transform: rotate(359deg);
+  }
+}
+@keyframes fa-spin {
+  0% {
+    -webkit-transform: rotate(0deg);
+    transform: rotate(0deg);
+  }
+  100% {
+    -webkit-transform: rotate(359deg);
+    transform: rotate(359deg);
+  }
+}
+.fa-rotate-90 {
+  filter: progid:DXImageTransform.Microsoft.BasicImage(rotation=1);
+  -webkit-transform: rotate(90deg);
+  -ms-transform: rotate(90deg);
+  transform: rotate(90deg);
+}
+.fa-rotate-180 {
+  filter: progid:DXImageTransform.Microsoft.BasicImage(rotation=2);
+  -webkit-transform: rotate(180deg);
+  -ms-transform: rotate(180deg);
+  transform: rotate(180deg);
+}
+.fa-rotate-270 {
+  filter: progid:DXImageTransform.Microsoft.BasicImage(rotation=3);
+  -webkit-transform: rotate(270deg);
+  -ms-transform: rotate(270deg);
+  transform: rotate(270deg);
+}
+.fa-flip-horizontal {
+  filter: progid:DXImageTransform.Microsoft.BasicImage(rotation=0, mirror=1);
+  -webkit-transform: scale(-1, 1);
+  -ms-transform: scale(-1, 1);
+  transform: scale(-1, 1);
+}
+.fa-flip-vertical {
+  filter: progid:DXImageTransform.Microsoft.BasicImage(rotation=2, mirror=1);
+  -webkit-transform: scale(1, -1);
+  -ms-transform: scale(1, -1);
+  transform: scale(1, -1);
+}
+:root .fa-rotate-90,
+:root .fa-rotate-180,
+:root .fa-rotate-270,
+:root .fa-flip-horizontal,
+:root .fa-flip-vertical {
+  filter: none;
+}
+.fa-stack {
+  position: relative;
+  display: inline-block;
+  width: 2em;
+  height: 2em;
+  line-height: 2em;
+  vertical-align: middle;
+}
+.fa-stack-1x,
+.fa-stack-2x {
+  position: absolute;
+  left: 0;
+  width: 100%;
+  text-align: center;
+}
+.fa-stack-1x {
+  line-height: inherit;
+}
+.fa-stack-2x {
+  font-size: 2em;
+}
+.fa-inverse {
+  color: #fff;
+}
+/* Font Awesome uses the Unicode Private Use Area (PUA) to ensure screen
+   readers do not read off random characters that represent icons */
+.fa-glass:before {
+  content: "\f000";
+}
+.fa-music:before {
+  content: "\f001";
+}
+.fa-search:before {
+  content: "\f002";
+}
+.fa-envelope-o:before {
+  content: "\f003";
+}
+.fa-heart:before {
+  content: "\f004";
+}
+.fa-star:before {
+  content: "\f005";
+}
+.fa-star-o:before {
+  content: "\f006";
+}
+.fa-user:before {
+  content: "\f007";
+}
+.fa-film:before {
+  content: "\f008";
+}
+.fa-th-large:before {
+  content: "\f009";
+}
+.fa-th:before {
+  content: "\f00a";
+}
+.fa-th-list:before {
+  content: "\f00b";
+}
+.fa-check:before {
+  content: "\f00c";
+}
+.fa-remove:before,
+.fa-close:before,
+.fa-times:before {
+  content: "\f00d";
+}
+.fa-search-plus:before {
+  content: "\f00e";
+}
+.fa-search-minus:before {
+  content: "\f010";
+}
+.fa-power-off:before {
+  content: "\f011";
+}
+.fa-signal:before {
+  content: "\f012";
+}
+.fa-gear:before,
+.fa-cog:before {
+  content: "\f013";
+}
+.fa-trash-o:before {
+  content: "\f014";
+}
+.fa-home:before {
+  content: "\f015";
+}
+.fa-file-o:before {
+  content: "\f016";
+}
+.fa-clock-o:before {
+  content: "\f017";
+}
+.fa-road:before {
+  content: "\f018";
+}
+.fa-download:before {
+  content: "\f019";
+}
+.fa-arrow-circle-o-down:before {
+  content: "\f01a";
+}
+.fa-arrow-circle-o-up:before {
+  content: "\f01b";
+}
+.fa-inbox:before {
+  content: "\f01c";
+}
+.fa-play-circle-o:before {
+  content: "\f01d";
+}
+.fa-rotate-right:before,
+.fa-repeat:before {
+  content: "\f01e";
+}
+.fa-refresh:before {
+  content: "\f021";
+}
+.fa-list-alt:before {
+  content: "\f022";
+}
+.fa-lock:before {
+  content: "\f023";
+}
+.fa-flag:before {
+  content: "\f024";
+}
+.fa-headphones:before {
+  content: "\f025";
+}
+.fa-volume-off:before {
+  content: "\f026";
+}
+.fa-volume-down:before {
+  content: "\f027";
+}
+.fa-volume-up:before {
+  content: "\f028";
+}
+.fa-qrcode:before {
+  content: "\f029";
+}
+.fa-barcode:before {
+  content: "\f02a";
+}
+.fa-tag:before {
+  content: "\f02b";
+}
+.fa-tags:before {
+  content: "\f02c";
+}
+.fa-book:before {
+  content: "\f02d";
+}
+.fa-bookmark:before {
+  content: "\f02e";
+}
+.fa-print:before {
+  content: "\f02f";
+}
+.fa-camera:before {
+  content: "\f030";
+}
+.fa-font:before {
+  content: "\f031";
+}
+.fa-bold:before {
+  content: "\f032";
+}
+.fa-italic:before {
+  content: "\f033";
+}
+.fa-text-height:before {
+  content: "\f034";
+}
+.fa-text-width:before {
+  content: "\f035";
+}
+.fa-align-left:before {
+  content: "\f036";
+}
+.fa-align-center:before {
+  content: "\f037";
+}
+.fa-align-right:before {
+  content: "\f038";
+}
+.fa-align-justify:before {
+  content: "\f039";
+}
+.fa-list:before {
+  content: "\f03a";
+}
+.fa-dedent:before,
+.fa-outdent:before {
+  content: "\f03b";
+}
+.fa-indent:before {
+  content: "\f03c";
+}
+.fa-video-camera:before {
+  content: "\f03d";
+}
+.fa-photo:before,
+.fa-image:before,
+.fa-picture-o:before {
+  content: "\f03e";
+}
+.fa-pencil:before {
+  content: "\f040";
+}
+.fa-map-marker:before {
+  content: "\f041";
+}
+.fa-adjust:before {
+  content: "\f042";
+}
+.fa-tint:before {
+  content: "\f043";
+}
+.fa-edit:before,
+.fa-pencil-square-o:before {
+  content: "\f044";
+}
+.fa-share-square-o:before {
+  content: "\f045";
+}
+.fa-check-square-o:before {
+  content: "\f046";
+}
+.fa-arrows:before {
+  content: "\f047";
+}
+.fa-step-backward:before {
+  content: "\f048";
+}
+.fa-fast-backward:before {
+  content: "\f049";
+}
+.fa-backward:before {
+  content: "\f04a";
+}
+.fa-play:before {
+  content: "\f04b";
+}
+.fa-pause:before {
+  content: "\f04c";
+}
+.fa-stop:before {
+  content: "\f04d";
+}
+.fa-forward:before {
+  content: "\f04e";
+}
+.fa-fast-forward:before {
+  content: "\f050";
+}
+.fa-step-forward:before {
+  content: "\f051";
+}
+.fa-eject:before {
+  content: "\f052";
+}
+.fa-chevron-left:before {
+  content: "\f053";
+}
+.fa-chevron-right:before {
+  content: "\f054";
+}
+.fa-plus-circle:before {
+  content: "\f055";
+}
+.fa-minus-circle:before {
+  content: "\f056";
+}
+.fa-times-circle:before {
+  content: "\f057";
+}
+.fa-check-circle:before {
+  content: "\f058";
+}
+.fa-question-circle:before {
+  content: "\f059";
+}
+.fa-info-circle:before {
+  content: "\f05a";
+}
+.fa-crosshairs:before {
+  content: "\f05b";
+}
+.fa-times-circle-o:before {
+  content: "\f05c";
+}
+.fa-check-circle-o:before {
+  content: "\f05d";
+}
+.fa-ban:before {
+  content: "\f05e";
+}
+.fa-arrow-left:before {
+  content: "\f060";
+}
+.fa-arrow-right:before {
+  content: "\f061";
+}
+.fa-arrow-up:before {
+  content: "\f062";
+}
+.fa-arrow-down:before {
+  content: "\f063";
+}
+.fa-mail-forward:before,
+.fa-share:before {
+  content: "\f064";
+}
+.fa-expand:before {
+  content: "\f065";
+}
+.fa-compress:before {
+  content: "\f066";
+}
+.fa-plus:before {
+  content: "\f067";
+}
+.fa-minus:before {
+  content: "\f068";
+}
+.fa-asterisk:before {
+  content: "\f069";
+}
+.fa-exclamation-circle:before {
+  content: "\f06a";
+}
+.fa-gift:before {
+  content: "\f06b";
+}
+.fa-leaf:before {
+  content: "\f06c";
+}
+.fa-fire:before {
+  content: "\f06d";
+}
+.fa-eye:before {
+  content: "\f06e";
+}
+.fa-eye-slash:before {
+  content: "\f070";
+}
+.fa-warning:before,
+.fa-exclamation-triangle:before {
+  content: "\f071";
+}
+.fa-plane:before {
+  content: "\f072";
+}
+.fa-calendar:before {
+  content: "\f073";
+}
+.fa-random:before {
+  content: "\f074";
+}
+.fa-comment:before {
+  content: "\f075";
+}
+.fa-magnet:before {
+  content: "\f076";
+}
+.fa-chevron-up:before {
+  content: "\f077";
+}
+.fa-chevron-down:before {
+  content: "\f078";
+}
+.fa-retweet:before {
+  content: "\f079";
+}
+.fa-shopping-cart:before {
+  content: "\f07a";
+}
+.fa-folder:before {
+  content: "\f07b";
+}
+.fa-folder-open:before {
+  content: "\f07c";
+}
+.fa-arrows-v:before {
+  content: "\f07d";
+}
+.fa-arrows-h:before {
+  content: "\f07e";
+}
+.fa-bar-chart-o:before,
+.fa-bar-chart:before {
+  content: "\f080";
+}
+.fa-twitter-square:before {
+  content: "\f081";
+}
+.fa-facebook-square:before {
+  content: "\f082";
+}
+.fa-camera-retro:before {
+  content: "\f083";
+}
+.fa-key:before {
+  content: "\f084";
+}
+.fa-gears:before,
+.fa-cogs:before {
+  content: "\f085";
+}
+.fa-comments:before {
+  content: "\f086";
+}
+.fa-thumbs-o-up:before {
+  content: "\f087";
+}
+.fa-thumbs-o-down:before {
+  content: "\f088";
+}
+.fa-star-half:before {
+  content: "\f089";
+}
+.fa-heart-o:before {
+  content: "\f08a";
+}
+.fa-sign-out:before {
+  content: "\f08b";
+}
+.fa-linkedin-square:before {
+  content: "\f08c";
+}
+.fa-thumb-tack:before {
+  content: "\f08d";
+}
+.fa-external-link:before {
+  content: "\f08e";
+}
+.fa-sign-in:before {
+  content: "\f090";
+}
+.fa-trophy:before {
+  content: "\f091";
+}
+.fa-github-square:before {
+  content: "\f092";
+}
+.fa-upload:before {
+  content: "\f093";
+}
+.fa-lemon-o:before {
+  content: "\f094";
+}
+.fa-phone:before {
+  content: "\f095";
+}
+.fa-square-o:before {
+  content: "\f096";
+}
+.fa-bookmark-o:before {
+  content: "\f097";
+}
+.fa-phone-square:before {
+  content: "\f098";
+}
+.fa-twitter:before {
+  content: "\f099";
+}
+.fa-facebook:before {
+  content: "\f09a";
+}
+.fa-github:before {
+  content: "\f09b";
+}
+.fa-unlock:before {
+  content: "\f09c";
+}
+.fa-credit-card:before {
+  content: "\f09d";
+}
+.fa-rss:before {
+  content: "\f09e";
+}
+.fa-hdd-o:before {
+  content: "\f0a0";
+}
+.fa-bullhorn:before {
+  content: "\f0a1";
+}
+.fa-bell:before {
+  content: "\f0f3";
+}
+.fa-certificate:before {
+  content: "\f0a3";
+}
+.fa-hand-o-right:before {
+  content: "\f0a4";
+}
+.fa-hand-o-left:before {
+  content: "\f0a5";
+}
+.fa-hand-o-up:before {
+  content: "\f0a6";
+}
+.fa-hand-o-down:before {
+  content: "\f0a7";
+}
+.fa-arrow-circle-left:before {
+  content: "\f0a8";
+}
+.fa-arrow-circle-right:before {
+  content: "\f0a9";
+}
+.fa-arrow-circle-up:before {
+  content: "\f0aa";
+}
+.fa-arrow-circle-down:before {
+  content: "\f0ab";
+}
+.fa-globe:before {
+  content: "\f0ac";
+}
+.fa-wrench:before {
+  content: "\f0ad";
+}
+.fa-tasks:before {
+  content: "\f0ae";
+}
+.fa-filter:before {
+  content: "\f0b0";
+}
+.fa-briefcase:before {
+  content: "\f0b1";
+}
+.fa-arrows-alt:before {
+  content: "\f0b2";
+}
+.fa-group:before,
+.fa-users:before {
+  content: "\f0c0";
+}
+.fa-chain:before,
+.fa-link:before {
+  content: "\f0c1";
+}
+.fa-cloud:before {
+  content: "\f0c2";
+}
+.fa-flask:before {
+  content: "\f0c3";
+}
+.fa-cut:before,
+.fa-scissors:before {
+  content: "\f0c4";
+}
+.fa-copy:before,
+.fa-files-o:before {
+  content: "\f0c5";
+}
+.fa-paperclip:before {
+  content: "\f0c6";
+}
+.fa-save:before,
+.fa-floppy-o:before {
+  content: "\f0c7";
+}
+.fa-square:before {
+  content: "\f0c8";
+}
+.fa-navicon:before,
+.fa-reorder:before,
+.fa-bars:before {
+  content: "\f0c9";
+}
+.fa-list-ul:before {
+  content: "\f0ca";
+}
+.fa-list-ol:before {
+  content: "\f0cb";
+}
+.fa-strikethrough:before {
+  content: "\f0cc";
+}
+.fa-underline:before {
+  content: "\f0cd";
+}
+.fa-table:before {
+  content: "\f0ce";
+}
+.fa-magic:before {
+  content: "\f0d0";
+}
+.fa-truck:before {
+  content: "\f0d1";
+}
+.fa-pinterest:before {
+  content: "\f0d2";
+}
+.fa-pinterest-square:before {
+  content: "\f0d3";
+}
+.fa-google-plus-square:before {
+  content: "\f0d4";
+}
+.fa-google-plus:before {
+  content: "\f0d5";
+}
+.fa-money:before {
+  content: "\f0d6";
+}
+.fa-caret-down:before {
+  content: "\f0d7";
+}
+.fa-caret-up:before {
+  content: "\f0d8";
+}
+.fa-caret-left:before {
+  content: "\f0d9";
+}
+.fa-caret-right:before {
+  content: "\f0da";
+}
+.fa-columns:before {
+  content: "\f0db";
+}
+.fa-unsorted:before,
+.fa-sort:before {
+  content: "\f0dc";
+}
+.fa-sort-down:before,
+.fa-sort-desc:before {
+  content: "\f0dd";
+}
+.fa-sort-up:before,
+.fa-sort-asc:before {
+  content: "\f0de";
+}
+.fa-envelope:before {
+  content: "\f0e0";
+}
+.fa-linkedin:before {
+  content: "\f0e1";
+}
+.fa-rotate-left:before,
+.fa-undo:before {
+  content: "\f0e2";
+}
+.fa-legal:before,
+.fa-gavel:before {
+  content: "\f0e3";
+}
+.fa-dashboard:before,
+.fa-tachometer:before {
+  content: "\f0e4";
+}
+.fa-comment-o:before {
+  content: "\f0e5";
+}
+.fa-comments-o:before {
+  content: "\f0e6";
+}
+.fa-flash:before,
+.fa-bolt:before {
+  content: "\f0e7";
+}
+.fa-sitemap:before {
+  content: "\f0e8";
+}
+.fa-umbrella:before {
+  content: "\f0e9";
+}
+.fa-paste:before,
+.fa-clipboard:before {
+  content: "\f0ea";
+}
+.fa-lightbulb-o:before {
+  content: "\f0eb";
+}
+.fa-exchange:before {
+  content: "\f0ec";
+}
+.fa-cloud-download:before {
+  content: "\f0ed";
+}
+.fa-cloud-upload:before {
+  content: "\f0ee";
+}
+.fa-user-md:before {
+  content: "\f0f0";
+}
+.fa-stethoscope:before {
+  content: "\f0f1";
+}
+.fa-suitcase:before {
+  content: "\f0f2";
+}
+.fa-bell-o:before {
+  content: "\f0a2";
+}
+.fa-coffee:before {
+  content: "\f0f4";
+}
+.fa-cutlery:before {
+  content: "\f0f5";
+}
+.fa-file-text-o:before {
+  content: "\f0f6";
+}
+.fa-building-o:before {
+  content: "\f0f7";
+}
+.fa-hospital-o:before {
+  content: "\f0f8";
+}
+.fa-ambulance:before {
+  content: "\f0f9";
+}
+.fa-medkit:before {
+  content: "\f0fa";
+}
+.fa-fighter-jet:before {
+  content: "\f0fb";
+}
+.fa-beer:before {
+  content: "\f0fc";
+}
+.fa-h-square:before {
+  content: "\f0fd";
+}
+.fa-plus-square:before {
+  content: "\f0fe";
+}
+.fa-angle-double-left:before {
+  content: "\f100";
+}
+.fa-angle-double-right:before {
+  content: "\f101";
+}
+.fa-angle-double-up:before {
+  content: "\f102";
+}
+.fa-angle-double-down:before {
+  content: "\f103";
+}
+.fa-angle-left:before {
+  content: "\f104";
+}
+.fa-angle-right:before {
+  content: "\f105";
+}
+.fa-angle-up:before {
+  content: "\f106";
+}
+.fa-angle-down:before {
+  content: "\f107";
+}
+.fa-desktop:before {
+  content: "\f108";
+}
+.fa-laptop:before {
+  content: "\f109";
+}
+.fa-tablet:before {
+  content: "\f10a";
+}
+.fa-mobile-phone:before,
+.fa-mobile:before {
+  content: "\f10b";
+}
+.fa-circle-o:before {
+  content: "\f10c";
+}
+.fa-quote-left:before {
+  content: "\f10d";
+}
+.fa-quote-right:before {
+  content: "\f10e";
+}
+.fa-spinner:before {
+  content: "\f110";
+}
+.fa-circle:before {
+  content: "\f111";
+}
+.fa-mail-reply:before,
+.fa-reply:before {
+  content: "\f112";
+}
+.fa-github-alt:before {
+  content: "\f113";
+}
+.fa-folder-o:before {
+  content: "\f114";
+}
+.fa-folder-open-o:before {
+  content: "\f115";
+}
+.fa-smile-o:before {
+  content: "\f118";
+}
+.fa-frown-o:before {
+  content: "\f119";
+}
+.fa-meh-o:before {
+  content: "\f11a";
+}
+.fa-gamepad:before {
+  content: "\f11b";
+}
+.fa-keyboard-o:before {
+  content: "\f11c";
+}
+.fa-flag-o:before {
+  content: "\f11d";
+}
+.fa-flag-checkered:before {
+  content: "\f11e";
+}
+.fa-terminal:before {
+  content: "\f120";
+}
+.fa-code:before {
+  content: "\f121";
+}
+.fa-mail-reply-all:before,
+.fa-reply-all:before {
+  content: "\f122";
+}
+.fa-star-half-empty:before,
+.fa-star-half-full:before,
+.fa-star-half-o:before {
+  content: "\f123";
+}
+.fa-location-arrow:before {
+  content: "\f124";
+}
+.fa-crop:before {
+  content: "\f125";
+}
+.fa-code-fork:before {
+  content: "\f126";
+}
+.fa-unlink:before,
+.fa-chain-broken:before {
+  content: "\f127";
+}
+.fa-question:before {
+  content: "\f128";
+}
+.fa-info:before {
+  content: "\f129";
+}
+.fa-exclamation:before {
+  content: "\f12a";
+}
+.fa-superscript:before {
+  content: "\f12b";
+}
+.fa-subscript:before {
+  content: "\f12c";
+}
+.fa-eraser:before {
+  content: "\f12d";
+}
+.fa-puzzle-piece:before {
+  content: "\f12e";
+}
+.fa-microphone:before {
+  content: "\f130";
+}
+.fa-microphone-slash:before {
+  content: "\f131";
+}
+.fa-shield:before {
+  content: "\f132";
+}
+.fa-calendar-o:before {
+  content: "\f133";
+}
+.fa-fire-extinguisher:before {
+  content: "\f134";
+}
+.fa-rocket:before {
+  content: "\f135";
+}
+.fa-maxcdn:before {
+  content: "\f136";
+}
+.fa-chevron-circle-left:before {
+  content: "\f137";
+}
+.fa-chevron-circle-right:before {
+  content: "\f138";
+}
+.fa-chevron-circle-up:before {
+  content: "\f139";
+}
+.fa-chevron-circle-down:before {
+  content: "\f13a";
+}
+.fa-html5:before {
+  content: "\f13b";
+}
+.fa-css3:before {
+  content: "\f13c";
+}
+.fa-anchor:before {
+  content: "\f13d";
+}
+.fa-unlock-alt:before {
+  content: "\f13e";
+}
+.fa-bullseye:before {
+  content: "\f140";
+}
+.fa-ellipsis-h:before {
+  content: "\f141";
+}
+.fa-ellipsis-v:before {
+  content: "\f142";
+}
+.fa-rss-square:before {
+  content: "\f143";
+}
+.fa-play-circle:before {
+  content: "\f144";
+}
+.fa-ticket:before {
+  content: "\f145";
+}
+.fa-minus-square:before {
+  content: "\f146";
+}
+.fa-minus-square-o:before {
+  content: "\f147";
+}
+.fa-level-up:before {
+  content: "\f148";
+}
+.fa-level-down:before {
+  content: "\f149";
+}
+.fa-check-square:before {
+  content: "\f14a";
+}
+.fa-pencil-square:before {
+  content: "\f14b";
+}
+.fa-external-link-square:before {
+  content: "\f14c";
+}
+.fa-share-square:before {
+  content: "\f14d";
+}
+.fa-compass:before {
+  content: "\f14e";
+}
+.fa-toggle-down:before,
+.fa-caret-square-o-down:before {
+  content: "\f150";
+}
+.fa-toggle-up:before,
+.fa-caret-square-o-up:before {
+  content: "\f151";
+}
+.fa-toggle-right:before,
+.fa-caret-square-o-right:before {
+  content: "\f152";
+}
+.fa-euro:before,
+.fa-eur:before {
+  content: "\f153";
+}
+.fa-gbp:before {
+  content: "\f154";
+}
+.fa-dollar:before,
+.fa-usd:before {
+  content: "\f155";
+}
+.fa-rupee:before,
+.fa-inr:before {
+  content: "\f156";
+}
+.fa-cny:before,
+.fa-rmb:before,
+.fa-yen:before,
+.fa-jpy:before {
+  content: "\f157";
+}
+.fa-ruble:before,
+.fa-rouble:before,
+.fa-rub:before {
+  content: "\f158";
+}
+.fa-won:before,
+.fa-krw:before {
+  content: "\f159";
+}
+.fa-bitcoin:before,
+.fa-btc:before {
+  content: "\f15a";
+}
+.fa-file:before {
+  content: "\f15b";
+}
+.fa-file-text:before {
+  content: "\f15c";
+}
+.fa-sort-alpha-asc:before {
+  content: "\f15d";
+}
+.fa-sort-alpha-desc:before {
+  content: "\f15e";
+}
+.fa-sort-amount-asc:before {
+  content: "\f160";
+}
+.fa-sort-amount-desc:before {
+  content: "\f161";
+}
+.fa-sort-numeric-asc:before {
+  content: "\f162";
+}
+.fa-sort-numeric-desc:before {
+  content: "\f163";
+}
+.fa-thumbs-up:before {
+  content: "\f164";
+}
+.fa-thumbs-down:before {
+  content: "\f165";
+}
+.fa-youtube-square:before {
+  content: "\f166";
+}
+.fa-youtube:before {
+  content: "\f167";
+}
+.fa-xing:before {
+  content: "\f168";
+}
+.fa-xing-square:before {
+  content: "\f169";
+}
+.fa-youtube-play:before {
+  content: "\f16a";
+}
+.fa-dropbox:before {
+  content: "\f16b";
+}
+.fa-stack-overflow:before {
+  content: "\f16c";
+}
+.fa-instagram:before {
+  content: "\f16d";
+}
+.fa-flickr:before {
+  content: "\f16e";
+}
+.fa-adn:before {
+  content: "\f170";
+}
+.fa-bitbucket:before {
+  content: "\f171";
+}
+.fa-bitbucket-square:before {
+  content: "\f172";
+}
+.fa-tumblr:before {
+  content: "\f173";
+}
+.fa-tumblr-square:before {
+  content: "\f174";
+}
+.fa-long-arrow-down:before {
+  content: "\f175";
+}
+.fa-long-arrow-up:before {
+  content: "\f176";
+}
+.fa-long-arrow-left:before {
+  content: "\f177";
+}
+.fa-long-arrow-right:before {
+  content: "\f178";
+}
+.fa-apple:before {
+  content: "\f179";
+}
+.fa-windows:before {
+  content: "\f17a";
+}
+.fa-android:before {
+  content: "\f17b";
+}
+.fa-linux:before {
+  content: "\f17c";
+}
+.fa-dribbble:before {
+  content: "\f17d";
+}
+.fa-skype:before {
+  content: "\f17e";
+}
+.fa-foursquare:before {
+  content: "\f180";
+}
+.fa-trello:before {
+  content: "\f181";
+}
+.fa-female:before {
+  content: "\f182";
+}
+.fa-male:before {
+  content: "\f183";
+}
+.fa-gittip:before {
+  content: "\f184";
+}
+.fa-sun-o:before {
+  content: "\f185";
+}
+.fa-moon-o:before {
+  content: "\f186";
+}
+.fa-archive:before {
+  content: "\f187";
+}
+.fa-bug:before {
+  content: "\f188";
+}
+.fa-vk:before {
+  content: "\f189";
+}
+.fa-weibo:before {
+  content: "\f18a";
+}
+.fa-renren:before {
+  content: "\f18b";
+}
+.fa-pagelines:before {
+  content: "\f18c";
+}
+.fa-stack-exchange:before {
+  content: "\f18d";
+}
+.fa-arrow-circle-o-right:before {
+  content: "\f18e";
+}
+.fa-arrow-circle-o-left:before {
+  content: "\f190";
+}
+.fa-toggle-left:before,
+.fa-caret-square-o-left:before {
+  content: "\f191";
+}
+.fa-dot-circle-o:before {
+  content: "\f192";
+}
+.fa-wheelchair:before {
+  content: "\f193";
+}
+.fa-vimeo-square:before {
+  content: "\f194";
+}
+.fa-turkish-lira:before,
+.fa-try:before {
+  content: "\f195";
+}
+.fa-plus-square-o:before {
+  content: "\f196";
+}
+.fa-space-shuttle:before {
+  content: "\f197";
+}
+.fa-slack:before {
+  content: "\f198";
+}
+.fa-envelope-square:before {
+  content: "\f199";
+}
+.fa-wordpress:before {
+  content: "\f19a";
+}
+.fa-openid:before {
+  content: "\f19b";
+}
+.fa-institution:before,
+.fa-bank:before,
+.fa-university:before {
+  content: "\f19c";
+}
+.fa-mortar-board:before,
+.fa-graduation-cap:before {
+  content: "\f19d";
+}
+.fa-yahoo:before {
+  content: "\f19e";
+}
+.fa-google:before {
+  content: "\f1a0";
+}
+.fa-reddit:before {
+  content: "\f1a1";
+}
+.fa-reddit-square:before {
+  content: "\f1a2";
+}
+.fa-stumbleupon-circle:before {
+  content: "\f1a3";
+}
+.fa-stumbleupon:before {
+  content: "\f1a4";
+}
+.fa-delicious:before {
+  content: "\f1a5";
+}
+.fa-digg:before {
+  content: "\f1a6";
+}
+.fa-pied-piper:before {
+  content: "\f1a7";
+}
+.fa-pied-piper-alt:before {
+  content: "\f1a8";
+}
+.fa-drupal:before {
+  content: "\f1a9";
+}
+.fa-joomla:before {
+  content: "\f1aa";
+}
+.fa-language:before {
+  content: "\f1ab";
+}
+.fa-fax:before {
+  content: "\f1ac";
+}
+.fa-building:before {
+  content: "\f1ad";
+}
+.fa-child:before {
+  content: "\f1ae";
+}
+.fa-paw:before {
+  content: "\f1b0";
+}
+.fa-spoon:before {
+  content: "\f1b1";
+}
+.fa-cube:before {
+  content: "\f1b2";
+}
+.fa-cubes:before {
+  content: "\f1b3";
+}
+.fa-behance:before {
+  content: "\f1b4";
+}
+.fa-behance-square:before {
+  content: "\f1b5";
+}
+.fa-steam:before {
+  content: "\f1b6";
+}
+.fa-steam-square:before {
+  content: "\f1b7";
+}
+.fa-recycle:before {
+  content: "\f1b8";
+}
+.fa-automobile:before,
+.fa-car:before {
+  content: "\f1b9";
+}
+.fa-cab:before,
+.fa-taxi:before {
+  content: "\f1ba";
+}
+.fa-tree:before {
+  content: "\f1bb";
+}
+.fa-spotify:before {
+  content: "\f1bc";
+}
+.fa-deviantart:before {
+  content: "\f1bd";
+}
+.fa-soundcloud:before {
+  content: "\f1be";
+}
+.fa-database:before {
+  content: "\f1c0";
+}
+.fa-file-pdf-o:before {
+  content: "\f1c1";
+}
+.fa-file-word-o:before {
+  content: "\f1c2";
+}
+.fa-file-excel-o:before {
+  content: "\f1c3";
+}
+.fa-file-powerpoint-o:before {
+  content: "\f1c4";
+}
+.fa-file-photo-o:before,
+.fa-file-picture-o:before,
+.fa-file-image-o:before {
+  content: "\f1c5";
+}
+.fa-file-zip-o:before,
+.fa-file-archive-o:before {
+  content: "\f1c6";
+}
+.fa-file-sound-o:before,
+.fa-file-audio-o:before {
+  content: "\f1c7";
+}
+.fa-file-movie-o:before,
+.fa-file-video-o:before {
+  content: "\f1c8";
+}
+.fa-file-code-o:before {
+  content: "\f1c9";
+}
+.fa-vine:before {
+  content: "\f1ca";
+}
+.fa-codepen:before {
+  content: "\f1cb";
+}
+.fa-jsfiddle:before {
+  content: "\f1cc";
+}
+.fa-life-bouy:before,
+.fa-life-buoy:before,
+.fa-life-saver:before,
+.fa-support:before,
+.fa-life-ring:before {
+  content: "\f1cd";
+}
+.fa-circle-o-notch:before {
+  content: "\f1ce";
+}
+.fa-ra:before,
+.fa-rebel:before {
+  content: "\f1d0";
+}
+.fa-ge:before,
+.fa-empire:before {
+  content: "\f1d1";
+}
+.fa-git-square:before {
+  content: "\f1d2";
+}
+.fa-git:before {
+  content: "\f1d3";
+}
+.fa-hacker-news:before {
+  content: "\f1d4";
+}
+.fa-tencent-weibo:before {
+  content: "\f1d5";
+}
+.fa-qq:before {
+  content: "\f1d6";
+}
+.fa-wechat:before,
+.fa-weixin:before {
+  content: "\f1d7";
+}
+.fa-send:before,
+.fa-paper-plane:before {
+  content: "\f1d8";
+}
+.fa-send-o:before,
+.fa-paper-plane-o:before {
+  content: "\f1d9";
+}
+.fa-history:before {
+  content: "\f1da";
+}
+.fa-circle-thin:before {
+  content: "\f1db";
+}
+.fa-header:before {
+  content: "\f1dc";
+}
+.fa-paragraph:before {
+  content: "\f1dd";
+}
+.fa-sliders:before {
+  content: "\f1de";
+}
+.fa-share-alt:before {
+  content: "\f1e0";
+}
+.fa-share-alt-square:before {
+  content: "\f1e1";
+}
+.fa-bomb:before {
+  content: "\f1e2";
+}
+.fa-soccer-ball-o:before,
+.fa-futbol-o:before {
+  content: "\f1e3";
+}
+.fa-tty:before {
+  content: "\f1e4";
+}
+.fa-binoculars:before {
+  content: "\f1e5";
+}
+.fa-plug:before {
+  content: "\f1e6";
+}
+.fa-slideshare:before {
+  content: "\f1e7";
+}
+.fa-twitch:before {
+  content: "\f1e8";
+}
+.fa-yelp:before {
+  content: "\f1e9";
+}
+.fa-newspaper-o:before {
+  content: "\f1ea";
+}
+.fa-wifi:before {
+  content: "\f1eb";
+}
+.fa-calculator:before {
+  content: "\f1ec";
+}
+.fa-paypal:before {
+  content: "\f1ed";
+}
+.fa-google-wallet:before {
+  content: "\f1ee";
+}
+.fa-cc-visa:before {
+  content: "\f1f0";
+}
+.fa-cc-mastercard:before {
+  content: "\f1f1";
+}
+.fa-cc-discover:before {
+  content: "\f1f2";
+}
+.fa-cc-amex:before {
+  content: "\f1f3";
+}
+.fa-cc-paypal:before {
+  content: "\f1f4";
+}
+.fa-cc-stripe:before {
+  content: "\f1f5";
+}
+.fa-bell-slash:before {
+  content: "\f1f6";
+}
+.fa-bell-slash-o:before {
+  content: "\f1f7";
+}
+.fa-trash:before {
+  content: "\f1f8";
+}
+.fa-copyright:before {
+  content: "\f1f9";
+}
+.fa-at:before {
+  content: "\f1fa";
+}
+.fa-eyedropper:before {
+  content: "\f1fb";
+}
+.fa-paint-brush:before {
+  content: "\f1fc";
+}
+.fa-birthday-cake:before {
+  content: "\f1fd";
+}
+.fa-area-chart:before {
+  content: "\f1fe";
+}
+.fa-pie-chart:before {
+  content: "\f200";
+}
+.fa-line-chart:before {
+  content: "\f201";
+}
+.fa-lastfm:before {
+  content: "\f202";
+}
+.fa-lastfm-square:before {
+  content: "\f203";
+}
+.fa-toggle-off:before {
+  content: "\f204";
+}
+.fa-toggle-on:before {
+  content: "\f205";
+}
+.fa-bicycle:before {
+  content: "\f206";
+}
+.fa-bus:before {
+  content: "\f207";
+}
+.fa-ioxhost:before {
+  content: "\f208";
+}
+.fa-angellist:before {
+  content: "\f209";
+}
+.fa-cc:before {
+  content: "\f20a";
+}
+.fa-shekel:before,
+.fa-sheqel:before,
+.fa-ils:before {
+  content: "\f20b";
+}
+.fa-meanpath:before {
+  content: "\f20c";
+}
+/*!
+*
+* IPython base
+*
+*/
+.modal.fade .modal-dialog {
+  -webkit-transform: translate(0, 0);
+  -ms-transform: translate(0, 0);
+  -o-transform: translate(0, 0);
+  transform: translate(0, 0);
+}
+code {
+  color: #000;
+}
+pre {
+  font-size: inherit;
+  line-height: inherit;
+}
+label {
+  font-weight: normal;
+}
+/* Make the page background atleast 100% the height of the view port */
+/* Make the page itself atleast 70% the height of the view port */
+.border-box-sizing {
+  box-sizing: border-box;
+  -moz-box-sizing: border-box;
+  -webkit-box-sizing: border-box;
+}
+.corner-all {
+  border-radius: 2px;
+}
+.no-padding {
+  padding: 0px;
+}
+/* Flexible box model classes */
+/* Taken from Alex Russell http://infrequently.org/2009/08/css-3-progress/ */
+/* This file is a compatability layer.  It allows the usage of flexible box 
+model layouts accross multiple browsers, including older browsers.  The newest,
+universal implementation of the flexible box model is used when available (see
+`Modern browsers` comments below).  Browsers that are known to implement this 
+new spec completely include:
+
+    Firefox 28.0+
+    Chrome 29.0+
+    Internet Explorer 11+ 
+    Opera 17.0+
+
+Browsers not listed, including Safari, are supported via the styling under the
+`Old browsers` comments below.
+*/
+.hbox {
+  /* Old browsers */
+  display: -webkit-box;
+  -webkit-box-orient: horizontal;
+  -webkit-box-align: stretch;
+  display: -moz-box;
+  -moz-box-orient: horizontal;
+  -moz-box-align: stretch;
+  display: box;
+  box-orient: horizontal;
+  box-align: stretch;
+  /* Modern browsers */
+  display: flex;
+  flex-direction: row;
+  align-items: stretch;
+}
+.hbox > * {
+  /* Old browsers */
+  -webkit-box-flex: 0;
+  -moz-box-flex: 0;
+  box-flex: 0;
+  /* Modern browsers */
+  flex: none;
+}
+.vbox {
+  /* Old browsers */
+  display: -webkit-box;
+  -webkit-box-orient: vertical;
+  -webkit-box-align: stretch;
+  display: -moz-box;
+  -moz-box-orient: vertical;
+  -moz-box-align: stretch;
+  display: box;
+  box-orient: vertical;
+  box-align: stretch;
+  /* Modern browsers */
+  display: flex;
+  flex-direction: column;
+  align-items: stretch;
+}
+.vbox > * {
+  /* Old browsers */
+  -webkit-box-flex: 0;
+  -moz-box-flex: 0;
+  box-flex: 0;
+  /* Modern browsers */
+  flex: none;
+}
+.hbox.reverse,
+.vbox.reverse,
+.reverse {
+  /* Old browsers */
+  -webkit-box-direction: reverse;
+  -moz-box-direction: reverse;
+  box-direction: reverse;
+  /* Modern browsers */
+  flex-direction: row-reverse;
+}
+.hbox.box-flex0,
+.vbox.box-flex0,
+.box-flex0 {
+  /* Old browsers */
+  -webkit-box-flex: 0;
+  -moz-box-flex: 0;
+  box-flex: 0;
+  /* Modern browsers */
+  flex: none;
+  width: auto;
+}
+.hbox.box-flex1,
+.vbox.box-flex1,
+.box-flex1 {
+  /* Old browsers */
+  -webkit-box-flex: 1;
+  -moz-box-flex: 1;
+  box-flex: 1;
+  /* Modern browsers */
+  flex: 1;
+}
+.hbox.box-flex,
+.vbox.box-flex,
+.box-flex {
+  /* Old browsers */
+  /* Old browsers */
+  -webkit-box-flex: 1;
+  -moz-box-flex: 1;
+  box-flex: 1;
+  /* Modern browsers */
+  flex: 1;
+}
+.hbox.box-flex2,
+.vbox.box-flex2,
+.box-flex2 {
+  /* Old browsers */
+  -webkit-box-flex: 2;
+  -moz-box-flex: 2;
+  box-flex: 2;
+  /* Modern browsers */
+  flex: 2;
+}
+.box-group1 {
+  /*  Deprecated */
+  -webkit-box-flex-group: 1;
+  -moz-box-flex-group: 1;
+  box-flex-group: 1;
+}
+.box-group2 {
+  /* Deprecated */
+  -webkit-box-flex-group: 2;
+  -moz-box-flex-group: 2;
+  box-flex-group: 2;
+}
+.hbox.start,
+.vbox.start,
+.start {
+  /* Old browsers */
+  -webkit-box-pack: start;
+  -moz-box-pack: start;
+  box-pack: start;
+  /* Modern browsers */
+  justify-content: flex-start;
+}
+.hbox.end,
+.vbox.end,
+.end {
+  /* Old browsers */
+  -webkit-box-pack: end;
+  -moz-box-pack: end;
+  box-pack: end;
+  /* Modern browsers */
+  justify-content: flex-end;
+}
+.hbox.center,
+.vbox.center,
+.center {
+  /* Old browsers */
+  -webkit-box-pack: center;
+  -moz-box-pack: center;
+  box-pack: center;
+  /* Modern browsers */
+  justify-content: center;
+}
+.hbox.baseline,
+.vbox.baseline,
+.baseline {
+  /* Old browsers */
+  -webkit-box-pack: baseline;
+  -moz-box-pack: baseline;
+  box-pack: baseline;
+  /* Modern browsers */
+  justify-content: baseline;
+}
+.hbox.stretch,
+.vbox.stretch,
+.stretch {
+  /* Old browsers */
+  -webkit-box-pack: stretch;
+  -moz-box-pack: stretch;
+  box-pack: stretch;
+  /* Modern browsers */
+  justify-content: stretch;
+}
+.hbox.align-start,
+.vbox.align-start,
+.align-start {
+  /* Old browsers */
+  -webkit-box-align: start;
+  -moz-box-align: start;
+  box-align: start;
+  /* Modern browsers */
+  align-items: flex-start;
+}
+.hbox.align-end,
+.vbox.align-end,
+.align-end {
+  /* Old browsers */
+  -webkit-box-align: end;
+  -moz-box-align: end;
+  box-align: end;
+  /* Modern browsers */
+  align-items: flex-end;
+}
+.hbox.align-center,
+.vbox.align-center,
+.align-center {
+  /* Old browsers */
+  -webkit-box-align: center;
+  -moz-box-align: center;
+  box-align: center;
+  /* Modern browsers */
+  align-items: center;
+}
+.hbox.align-baseline,
+.vbox.align-baseline,
+.align-baseline {
+  /* Old browsers */
+  -webkit-box-align: baseline;
+  -moz-box-align: baseline;
+  box-align: baseline;
+  /* Modern browsers */
+  align-items: baseline;
+}
+.hbox.align-stretch,
+.vbox.align-stretch,
+.align-stretch {
+  /* Old browsers */
+  -webkit-box-align: stretch;
+  -moz-box-align: stretch;
+  box-align: stretch;
+  /* Modern browsers */
+  align-items: stretch;
+}
+div.error {
+  margin: 2em;
+  text-align: center;
+}
+div.error > h1 {
+  font-size: 500%;
+  line-height: normal;
+}
+div.error > p {
+  font-size: 200%;
+  line-height: normal;
+}
+div.traceback-wrapper {
+  text-align: left;
+  max-width: 800px;
+  margin: auto;
+}
+/**
+ * Primary styles
+ *
+ * Author: Jupyter Development Team
+ */
+body {
+  background-color: #fff;
+  /* This makes sure that the body covers the entire window and needs to
+       be in a different element than the display: box in wrapper below */
+  position: absolute;
+  left: 0px;
+  right: 0px;
+  top: 0px;
+  bottom: 0px;
+  overflow: visible;
+}
+body > #header {
+  /* Initially hidden to prevent FLOUC */
+  display: none;
+  background-color: #fff;
+  /* Display over codemirror */
+  position: relative;
+  z-index: 100;
+}
+body > #header #header-container {
+  padding-bottom: 5px;
+  padding-top: 5px;
+  box-sizing: border-box;
+  -moz-box-sizing: border-box;
+  -webkit-box-sizing: border-box;
+}
+body > #header .header-bar {
+  width: 100%;
+  height: 1px;
+  background: #e7e7e7;
+  margin-bottom: -1px;
+}
+@media print {
+  body > #header {
+    display: none !important;
+  }
+}
+#header-spacer {
+  width: 100%;
+  visibility: hidden;
+}
+@media print {
+  #header-spacer {
+    display: none;
+  }
+}
+#ipython_notebook {
+  padding-left: 0px;
+  padding-top: 1px;
+  padding-bottom: 1px;
+}
+@media (max-width: 991px) {
+  #ipython_notebook {
+    margin-left: 10px;
+  }
+}
+#noscript {
+  width: auto;
+  padding-top: 16px;
+  padding-bottom: 16px;
+  text-align: center;
+  font-size: 22px;
+  color: red;
+  font-weight: bold;
+}
+#ipython_notebook img {
+  height: 28px;
+}
+#site {
+  width: 100%;
+  display: none;
+  box-sizing: border-box;
+  -moz-box-sizing: border-box;
+  -webkit-box-sizing: border-box;
+  overflow: auto;
+}
+@media print {
+  #site {
+    height: auto !important;
+  }
+}
+/* Smaller buttons */
+.ui-button .ui-button-text {
+  padding: 0.2em 0.8em;
+  font-size: 77%;
+}
+input.ui-button {
+  padding: 0.3em 0.9em;
+}
+span#login_widget {
+  float: right;
+}
+span#login_widget > .button,
+#logout {
+  color: #333;
+  background-color: #fff;
+  border-color: #ccc;
+}
+span#login_widget > .button:focus,
+#logout:focus,
+span#login_widget > .button.focus,
+#logout.focus {
+  color: #333;
+  background-color: #e6e6e6;
+  border-color: #8c8c8c;
+}
+span#login_widget > .button:hover,
+#logout:hover {
+  color: #333;
+  background-color: #e6e6e6;
+  border-color: #adadad;
+}
+span#login_widget > .button:active,
+#logout:active,
+span#login_widget > .button.active,
+#logout.active,
+.open > .dropdown-togglespan#login_widget > .button,
+.open > .dropdown-toggle#logout {
+  color: #333;
+  background-color: #e6e6e6;
+  border-color: #adadad;
+}
+span#login_widget > .button:active:hover,
+#logout:active:hover,
+span#login_widget > .button.active:hover,
+#logout.active:hover,
+.open > .dropdown-togglespan#login_widget > .button:hover,
+.open > .dropdown-toggle#logout:hover,
+span#login_widget > .button:active:focus,
+#logout:active:focus,
+span#login_widget > .button.active:focus,
+#logout.active:focus,
+.open > .dropdown-togglespan#login_widget > .button:focus,
+.open > .dropdown-toggle#logout:focus,
+span#login_widget > .button:active.focus,
+#logout:active.focus,
+span#login_widget > .button.active.focus,
+#logout.active.focus,
+.open > .dropdown-togglespan#login_widget > .button.focus,
+.open > .dropdown-toggle#logout.focus {
+  color: #333;
+  background-color: #d4d4d4;
+  border-color: #8c8c8c;
+}
+span#login_widget > .button:active,
+#logout:active,
+span#login_widget > .button.active,
+#logout.active,
+.open > .dropdown-togglespan#login_widget > .button,
+.open > .dropdown-toggle#logout {
+  background-image: none;
+}
+span#login_widget > .button.disabled:hover,
+#logout.disabled:hover,
+span#login_widget > .button[disabled]:hover,
+#logout[disabled]:hover,
+fieldset[disabled] span#login_widget > .button:hover,
+fieldset[disabled] #logout:hover,
+span#login_widget > .button.disabled:focus,
+#logout.disabled:focus,
+span#login_widget > .button[disabled]:focus,
+#logout[disabled]:focus,
+fieldset[disabled] span#login_widget > .button:focus,
+fieldset[disabled] #logout:focus,
+span#login_widget > .button.disabled.focus,
+#logout.disabled.focus,
+span#login_widget > .button[disabled].focus,
+#logout[disabled].focus,
+fieldset[disabled] span#login_widget > .button.focus,
+fieldset[disabled] #logout.focus {
+  background-color: #fff;
+  border-color: #ccc;
+}
+span#login_widget > .button .badge,
+#logout .badge {
+  color: #fff;
+  background-color: #333;
+}
+.nav-header {
+  text-transform: none;
+}
+#header > span {
+  margin-top: 10px;
+}
+.modal_stretch .modal-dialog {
+  /* Old browsers */
+  display: -webkit-box;
+  -webkit-box-orient: vertical;
+  -webkit-box-align: stretch;
+  display: -moz-box;
+  -moz-box-orient: vertical;
+  -moz-box-align: stretch;
+  display: box;
+  box-orient: vertical;
+  box-align: stretch;
+  /* Modern browsers */
+  display: flex;
+  flex-direction: column;
+  align-items: stretch;
+  min-height: 80vh;
+}
+.modal_stretch .modal-dialog .modal-body {
+  max-height: calc(100vh - 200px);
+  overflow: auto;
+  flex: 1;
+}
+@media (min-width: 768px) {
+  .modal .modal-dialog {
+    width: 700px;
+  }
+}
+@media (min-width: 768px) {
+  select.form-control {
+    margin-left: 12px;
+    margin-right: 12px;
+  }
+}
+/*!
+*
+* IPython auth
+*
+*/
+.center-nav {
+  display: inline-block;
+  margin-bottom: -4px;
+}
+/*!
+*
+* IPython tree view
+*
+*/
+/* We need an invisible input field on top of the sentense*/
+/* "Drag file onto the list ..." */
+.alternate_upload {
+  background-color: none;
+  display: inline;
+}
+.alternate_upload.form {
+  padding: 0;
+  margin: 0;
+}
+.alternate_upload input.fileinput {
+  text-align: center;
+  vertical-align: middle;
+  display: inline;
+  opacity: 0;
+  z-index: 2;
+  width: 12ex;
+  margin-right: -12ex;
+}
+.alternate_upload .btn-upload {
+  height: 22px;
+}
+/**
+ * Primary styles
+ *
+ * Author: Jupyter Development Team
+ */
+ul#tabs {
+  margin-bottom: 4px;
+}
+ul#tabs a {
+  padding-top: 6px;
+  padding-bottom: 4px;
+}
+ul.breadcrumb a:focus,
+ul.breadcrumb a:hover {
+  text-decoration: none;
+}
+ul.breadcrumb i.icon-home {
+  font-size: 16px;
+  margin-right: 4px;
+}
+ul.breadcrumb span {
+  color: #5e5e5e;
+}
+.list_toolbar {
+  padding: 4px 0 4px 0;
+  vertical-align: middle;
+}
+.list_toolbar .tree-buttons {
+  padding-top: 1px;
+}
+.dynamic-buttons {
+  padding-top: 3px;
+  display: inline-block;
+}
+.list_toolbar [class*="span"] {
+  min-height: 24px;
+}
+.list_header {
+  font-weight: bold;
+  background-color: #EEE;
+}
+.list_placeholder {
+  font-weight: bold;
+  padding-top: 4px;
+  padding-bottom: 4px;
+  padding-left: 7px;
+  padding-right: 7px;
+}
+.list_container {
+  margin-top: 4px;
+  margin-bottom: 20px;
+  border: 1px solid #ddd;
+  border-radius: 2px;
+}
+.list_container > div {
+  border-bottom: 1px solid #ddd;
+}
+.list_container > div:hover .list-item {
+  background-color: red;
+}
+.list_container > div:last-child {
+  border: none;
+}
+.list_item:hover .list_item {
+  background-color: #ddd;
+}
+.list_item a {
+  text-decoration: none;
+}
+.list_item:hover {
+  background-color: #fafafa;
+}
+.list_header > div,
+.list_item > div {
+  padding-top: 4px;
+  padding-bottom: 4px;
+  padding-left: 7px;
+  padding-right: 7px;
+  line-height: 22px;
+}
+.list_header > div input,
+.list_item > div input {
+  margin-right: 7px;
+  margin-left: 14px;
+  vertical-align: baseline;
+  line-height: 22px;
+  position: relative;
+  top: -1px;
+}
+.list_header > div .item_link,
+.list_item > div .item_link {
+  margin-left: -1px;
+  vertical-align: baseline;
+  line-height: 22px;
+}
+.new-file input[type=checkbox] {
+  visibility: hidden;
+}
+.item_name {
+  line-height: 22px;
+  height: 24px;
+}
+.item_icon {
+  font-size: 14px;
+  color: #5e5e5e;
+  margin-right: 7px;
+  margin-left: 7px;
+  line-height: 22px;
+  vertical-align: baseline;
+}
+.item_buttons {
+  line-height: 1em;
+  margin-left: -5px;
+}
+.item_buttons .btn,
+.item_buttons .btn-group,
+.item_buttons .input-group {
+  float: left;
+}
+.item_buttons > .btn,
+.item_buttons > .btn-group,
+.item_buttons > .input-group {
+  margin-left: 5px;
+}
+.item_buttons .btn {
+  min-width: 13ex;
+}
+.item_buttons .running-indicator {
+  padding-top: 4px;
+  color: #5cb85c;
+}
+.item_buttons .kernel-name {
+  padding-top: 4px;
+  color: #5bc0de;
+  margin-right: 7px;
+  float: left;
+}
+.toolbar_info {
+  height: 24px;
+  line-height: 24px;
+}
+.list_item input:not([type=checkbox]) {
+  padding-top: 3px;
+  padding-bottom: 3px;
+  height: 22px;
+  line-height: 14px;
+  margin: 0px;
+}
+.highlight_text {
+  color: blue;
+}
+#project_name {
+  display: inline-block;
+  padding-left: 7px;
+  margin-left: -2px;
+}
+#project_name > .breadcrumb {
+  padding: 0px;
+  margin-bottom: 0px;
+  background-color: transparent;
+  font-weight: bold;
+}
+#tree-selector {
+  padding-right: 0px;
+}
+#button-select-all {
+  min-width: 50px;
+}
+#select-all {
+  margin-left: 7px;
+  margin-right: 2px;
+}
+.menu_icon {
+  margin-right: 2px;
+}
+.tab-content .row {
+  margin-left: 0px;
+  margin-right: 0px;
+}
+.folder_icon:before {
+  display: inline-block;
+  font: normal normal normal 14px/1 FontAwesome;
+  font-size: inherit;
+  text-rendering: auto;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+  content: "\f114";
+}
+.folder_icon:before.pull-left {
+  margin-right: .3em;
+}
+.folder_icon:before.pull-right {
+  margin-left: .3em;
+}
+.notebook_icon:before {
+  display: inline-block;
+  font: normal normal normal 14px/1 FontAwesome;
+  font-size: inherit;
+  text-rendering: auto;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+  content: "\f02d";
+  position: relative;
+  top: -1px;
+}
+.notebook_icon:before.pull-left {
+  margin-right: .3em;
+}
+.notebook_icon:before.pull-right {
+  margin-left: .3em;
+}
+.running_notebook_icon:before {
+  display: inline-block;
+  font: normal normal normal 14px/1 FontAwesome;
+  font-size: inherit;
+  text-rendering: auto;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+  content: "\f02d";
+  position: relative;
+  top: -1px;
+  color: #5cb85c;
+}
+.running_notebook_icon:before.pull-left {
+  margin-right: .3em;
+}
+.running_notebook_icon:before.pull-right {
+  margin-left: .3em;
+}
+.file_icon:before {
+  display: inline-block;
+  font: normal normal normal 14px/1 FontAwesome;
+  font-size: inherit;
+  text-rendering: auto;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+  content: "\f016";
+  position: relative;
+  top: -2px;
+}
+.file_icon:before.pull-left {
+  margin-right: .3em;
+}
+.file_icon:before.pull-right {
+  margin-left: .3em;
+}
+#notebook_toolbar .pull-right {
+  padding-top: 0px;
+  margin-right: -1px;
+}
+ul#new-menu {
+  left: auto;
+  right: 0;
+}
+.kernel-menu-icon {
+  padding-right: 12px;
+  width: 24px;
+  content: "\f096";
+}
+.kernel-menu-icon:before {
+  content: "\f096";
+}
+.kernel-menu-icon-current:before {
+  content: "\f00c";
+}
+#tab_content {
+  padding-top: 20px;
+}
+#running .panel-group .panel {
+  margin-top: 3px;
+  margin-bottom: 1em;
+}
+#running .panel-group .panel .panel-heading {
+  background-color: #EEE;
+  padding-top: 4px;
+  padding-bottom: 4px;
+  padding-left: 7px;
+  padding-right: 7px;
+  line-height: 22px;
+}
+#running .panel-group .panel .panel-heading a:focus,
+#running .panel-group .panel .panel-heading a:hover {
+  text-decoration: none;
+}
+#running .panel-group .panel .panel-body {
+  padding: 0px;
+}
+#running .panel-group .panel .panel-body .list_container {
+  margin-top: 0px;
+  margin-bottom: 0px;
+  border: 0px;
+  border-radius: 0px;
+}
+#running .panel-group .panel .panel-body .list_container .list_item {
+  border-bottom: 1px solid #ddd;
+}
+#running .panel-group .panel .panel-body .list_container .list_item:last-child {
+  border-bottom: 0px;
+}
+.delete-button {
+  display: none;
+}
+.duplicate-button {
+  display: none;
+}
+.rename-button {
+  display: none;
+}
+.shutdown-button {
+  display: none;
+}
+.dynamic-instructions {
+  display: inline-block;
+  padding-top: 4px;
+}
+/*!
+*
+* IPython text editor webapp
+*
+*/
+.selected-keymap i.fa {
+  padding: 0px 5px;
+}
+.selected-keymap i.fa:before {
+  content: "\f00c";
+}
+#mode-menu {
+  overflow: auto;
+  max-height: 20em;
+}
+.edit_app #header {
+  -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
+  box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
+}
+.edit_app #menubar .navbar {
+  /* Use a negative 1 bottom margin, so the border overlaps the border of the
+    header */
+  margin-bottom: -1px;
+}
+.dirty-indicator {
+  display: inline-block;
+  font: normal normal normal 14px/1 FontAwesome;
+  font-size: inherit;
+  text-rendering: auto;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+  width: 20px;
+}
+.dirty-indicator.pull-left {
+  margin-right: .3em;
+}
+.dirty-indicator.pull-right {
+  margin-left: .3em;
+}
+.dirty-indicator-dirty {
+  display: inline-block;
+  font: normal normal normal 14px/1 FontAwesome;
+  font-size: inherit;
+  text-rendering: auto;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+  width: 20px;
+}
+.dirty-indicator-dirty.pull-left {
+  margin-right: .3em;
+}
+.dirty-indicator-dirty.pull-right {
+  margin-left: .3em;
+}
+.dirty-indicator-clean {
+  display: inline-block;
+  font: normal normal normal 14px/1 FontAwesome;
+  font-size: inherit;
+  text-rendering: auto;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+  width: 20px;
+}
+.dirty-indicator-clean.pull-left {
+  margin-right: .3em;
+}
+.dirty-indicator-clean.pull-right {
+  margin-left: .3em;
+}
+.dirty-indicator-clean:before {
+  display: inline-block;
+  font: normal normal normal 14px/1 FontAwesome;
+  font-size: inherit;
+  text-rendering: auto;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+  content: "\f00c";
+}
+.dirty-indicator-clean:before.pull-left {
+  margin-right: .3em;
+}
+.dirty-indicator-clean:before.pull-right {
+  margin-left: .3em;
+}
+#filename {
+  font-size: 16pt;
+  display: table;
+  padding: 0px 5px;
+}
+#current-mode {
+  padding-left: 5px;
+  padding-right: 5px;
+}
+#texteditor-backdrop {
+  padding-top: 20px;
+  padding-bottom: 20px;
+}
+@media not print {
+  #texteditor-backdrop {
+    background-color: #EEE;
+  }
+}
+@media print {
+  #texteditor-backdrop #texteditor-container .CodeMirror-gutter,
+  #texteditor-backdrop #texteditor-container .CodeMirror-gutters {
+    background-color: #fff;
+  }
+}
+@media not print {
+  #texteditor-backdrop #texteditor-container .CodeMirror-gutter,
+  #texteditor-backdrop #texteditor-container .CodeMirror-gutters {
+    background-color: #fff;
+  }
+}
+@media not print {
+  #texteditor-backdrop #texteditor-container {
+    padding: 0px;
+    background-color: #fff;
+    -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
+    box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
+  }
+}
+/*!
+*
+* IPython notebook
+*
+*/
+/* CSS font colors for translated ANSI colors. */
+.ansibold {
+  font-weight: bold;
+}
+/* use dark versions for foreground, to improve visibility */
+.ansiblack {
+  color: black;
+}
+.ansired {
+  color: darkred;
+}
+.ansigreen {
+  color: darkgreen;
+}
+.ansiyellow {
+  color: #c4a000;
+}
+.ansiblue {
+  color: darkblue;
+}
+.ansipurple {
+  color: darkviolet;
+}
+.ansicyan {
+  color: steelblue;
+}
+.ansigray {
+  color: gray;
+}
+/* and light for background, for the same reason */
+.ansibgblack {
+  background-color: black;
+}
+.ansibgred {
+  background-color: red;
+}
+.ansibggreen {
+  background-color: green;
+}
+.ansibgyellow {
+  background-color: yellow;
+}
+.ansibgblue {
+  background-color: blue;
+}
+.ansibgpurple {
+  background-color: magenta;
+}
+.ansibgcyan {
+  background-color: cyan;
+}
+.ansibggray {
+  background-color: gray;
+}
+div.cell {
+  /* Old browsers */
+  display: -webkit-box;
+  -webkit-box-orient: vertical;
+  -webkit-box-align: stretch;
+  display: -moz-box;
+  -moz-box-orient: vertical;
+  -moz-box-align: stretch;
+  display: box;
+  box-orient: vertical;
+  box-align: stretch;
+  /* Modern browsers */
+  display: flex;
+  flex-direction: column;
+  align-items: stretch;
+  border-radius: 2px;
+  box-sizing: border-box;
+  -moz-box-sizing: border-box;
+  -webkit-box-sizing: border-box;
+  border-width: 1px;
+  border-style: solid;
+  border-color: transparent;
+  width: 100%;
+  padding: 5px;
+  /* This acts as a spacer between cells, that is outside the border */
+  margin: 0px;
+  outline: none;
+  border-left-width: 1px;
+  padding-left: 5px;
+  background: linear-gradient(to right, transparent -40px, transparent 1px, transparent 1px, transparent 100%);
+}
+div.cell.jupyter-soft-selected {
+  border-left-color: #90CAF9;
+  border-left-color: #E3F2FD;
+  border-left-width: 1px;
+  padding-left: 5px;
+  border-right-color: #E3F2FD;
+  border-right-width: 1px;
+  background: #E3F2FD;
+}
+@media print {
+  div.cell.jupyter-soft-selected {
+    border-color: transparent;
+  }
+}
+div.cell.selected {
+  border-color: #ababab;
+  border-left-width: 0px;
+  padding-left: 6px;
+  background: linear-gradient(to right, #42A5F5 -40px, #42A5F5 5px, transparent 5px, transparent 100%);
+}
+@media print {
+  div.cell.selected {
+    border-color: transparent;
+  }
+}
+div.cell.selected.jupyter-soft-selected {
+  border-left-width: 0;
+  padding-left: 6px;
+  background: linear-gradient(to right, #42A5F5 -40px, #42A5F5 7px, #E3F2FD 7px, #E3F2FD 100%);
+}
+.edit_mode div.cell.selected {
+  border-color: #66BB6A;
+  border-left-width: 0px;
+  padding-left: 6px;
+  background: linear-gradient(to right, #66BB6A -40px, #66BB6A 5px, transparent 5px, transparent 100%);
+}
+@media print {
+  .edit_mode div.cell.selected {
+    border-color: transparent;
+  }
+}
+.prompt {
+  /* This needs to be wide enough for 3 digit prompt numbers: In[100]: */
+  min-width: 14ex;
+  /* This padding is tuned to match the padding on the CodeMirror editor. */
+  padding: 0.4em;
+  margin: 0px;
+  font-family: monospace;
+  text-align: right;
+  /* This has to match that of the the CodeMirror class line-height below */
+  line-height: 1.21429em;
+  /* Don't highlight prompt number selection */
+  -webkit-touch-callout: none;
+  -webkit-user-select: none;
+  -khtml-user-select: none;
+  -moz-user-select: none;
+  -ms-user-select: none;
+  user-select: none;
+  /* Use default cursor */
+  cursor: default;
+}
+@media (max-width: 540px) {
+  .prompt {
+    text-align: left;
+  }
+}
+div.inner_cell {
+  /* Old browsers */
+  display: -webkit-box;
+  -webkit-box-orient: vertical;
+  -webkit-box-align: stretch;
+  display: -moz-box;
+  -moz-box-orient: vertical;
+  -moz-box-align: stretch;
+  display: box;
+  box-orient: vertical;
+  box-align: stretch;
+  /* Modern browsers */
+  display: flex;
+  flex-direction: column;
+  align-items: stretch;
+  /* Old browsers */
+  -webkit-box-flex: 1;
+  -moz-box-flex: 1;
+  box-flex: 1;
+  /* Modern browsers */
+  flex: 1;
+}
+@-moz-document url-prefix() {
+  div.inner_cell {
+    overflow-x: hidden;
+  }
+}
+/* input_area and input_prompt must match in top border and margin for alignment */
+div.input_area {
+  border: 1px solid #cfcfcf;
+  border-radius: 2px;
+  background: #f7f7f7;
+  line-height: 1.21429em;
+}
+/* This is needed so that empty prompt areas can collapse to zero height when there
+   is no content in the output_subarea and the prompt. The main purpose of this is
+   to make sure that empty JavaScript output_subareas have no height. */
+div.prompt:empty {
+  padding-top: 0;
+  padding-bottom: 0;
+}
+div.unrecognized_cell {
+  padding: 5px 5px 5px 0px;
+  /* Old browsers */
+  display: -webkit-box;
+  -webkit-box-orient: horizontal;
+  -webkit-box-align: stretch;
+  display: -moz-box;
+  -moz-box-orient: horizontal;
+  -moz-box-align: stretch;
+  display: box;
+  box-orient: horizontal;
+  box-align: stretch;
+  /* Modern browsers */
+  display: flex;
+  flex-direction: row;
+  align-items: stretch;
+}
+div.unrecognized_cell .inner_cell {
+  border-radius: 2px;
+  padding: 5px;
+  font-weight: bold;
+  color: red;
+  border: 1px solid #cfcfcf;
+  background: #eaeaea;
+}
+div.unrecognized_cell .inner_cell a {
+  color: inherit;
+  text-decoration: none;
+}
+div.unrecognized_cell .inner_cell a:hover {
+  color: inherit;
+  text-decoration: none;
+}
+@media (max-width: 540px) {
+  div.unrecognized_cell > div.prompt {
+    display: none;
+  }
+}
+div.code_cell {
+  /* avoid page breaking on code cells when printing */
+}
+@media print {
+  div.code_cell {
+    page-break-inside: avoid;
+  }
+}
+/* any special styling for code cells that are currently running goes here */
+div.input {
+  page-break-inside: avoid;
+  /* Old browsers */
+  display: -webkit-box;
+  -webkit-box-orient: horizontal;
+  -webkit-box-align: stretch;
+  display: -moz-box;
+  -moz-box-orient: horizontal;
+  -moz-box-align: stretch;
+  display: box;
+  box-orient: horizontal;
+  box-align: stretch;
+  /* Modern browsers */
+  display: flex;
+  flex-direction: row;
+  align-items: stretch;
+}
+@media (max-width: 540px) {
+  div.input {
+    /* Old browsers */
+    display: -webkit-box;
+    -webkit-box-orient: vertical;
+    -webkit-box-align: stretch;
+    display: -moz-box;
+    -moz-box-orient: vertical;
+    -moz-box-align: stretch;
+    display: box;
+    box-orient: vertical;
+    box-align: stretch;
+    /* Modern browsers */
+    display: flex;
+    flex-direction: column;
+    align-items: stretch;
+  }
+}
+/* input_area and input_prompt must match in top border and margin for alignment */
+div.input_prompt {
+  color: #303F9F;
+  border-top: 1px solid transparent;
+}
+div.input_area > div.highlight {
+  margin: 0.4em;
+  border: none;
+  padding: 0px;
+  background-color: transparent;
+}
+div.input_area > div.highlight > pre {
+  margin: 0px;
+  border: none;
+  padding: 0px;
+  background-color: transparent;
+}
+/* The following gets added to the <head> if it is detected that the user has a
+ * monospace font with inconsistent normal/bold/italic height.  See
+ * notebookmain.js.  Such fonts will have keywords vertically offset with
+ * respect to the rest of the text.  The user should select a better font.
+ * See: https://github.com/ipython/ipython/issues/1503
+ *
+ * .CodeMirror span {
+ *      vertical-align: bottom;
+ * }
+ */
+.CodeMirror {
+  line-height: 1.21429em;
+  /* Changed from 1em to our global default */
+  font-size: 14px;
+  height: auto;
+  /* Changed to auto to autogrow */
+  background: none;
+  /* Changed from white to allow our bg to show through */
+}
+.CodeMirror-scroll {
+  /*  The CodeMirror docs are a bit fuzzy on if overflow-y should be hidden or visible.*/
+  /*  We have found that if it is visible, vertical scrollbars appear with font size changes.*/
+  overflow-y: hidden;
+  overflow-x: auto;
+}
+.CodeMirror-lines {
+  /* In CM2, this used to be 0.4em, but in CM3 it went to 4px. We need the em value because */
+  /* we have set a different line-height and want this to scale with that. */
+  padding: 0.4em;
+}
+.CodeMirror-linenumber {
+  padding: 0 8px 0 4px;
+}
+.CodeMirror-gutters {
+  border-bottom-left-radius: 2px;
+  border-top-left-radius: 2px;
+}
+.CodeMirror pre {
+  /* In CM3 this went to 4px from 0 in CM2. We need the 0 value because of how we size */
+  /* .CodeMirror-lines */
+  padding: 0;
+  border: 0;
+  border-radius: 0;
+}
+/*
+
+Original style from softwaremaniacs.org (c) Ivan Sagalaev <Maniac@SoftwareManiacs.Org>
+Adapted from GitHub theme
+
+*/
+.highlight-base {
+  color: #000;
+}
+.highlight-variable {
+  color: #000;
+}
+.highlight-variable-2 {
+  color: #1a1a1a;
+}
+.highlight-variable-3 {
+  color: #333333;
+}
+.highlight-string {
+  color: #BA2121;
+}
+.highlight-comment {
+  color: #408080;
+  font-style: italic;
+}
+.highlight-number {
+  color: #080;
+}
+.highlight-atom {
+  color: #88F;
+}
+.highlight-keyword {
+  color: #008000;
+  font-weight: bold;
+}
+.highlight-builtin {
+  color: #008000;
+}
+.highlight-error {
+  color: #f00;
+}
+.highlight-operator {
+  color: #AA22FF;
+  font-weight: bold;
+}
+.highlight-meta {
+  color: #AA22FF;
+}
+/* previously not defined, copying from default codemirror */
+.highlight-def {
+  color: #00f;
+}
+.highlight-string-2 {
+  color: #f50;
+}
+.highlight-qualifier {
+  color: #555;
+}
+.highlight-bracket {
+  color: #997;
+}
+.highlight-tag {
+  color: #170;
+}
+.highlight-attribute {
+  color: #00c;
+}
+.highlight-header {
+  color: blue;
+}
+.highlight-quote {
+  color: #090;
+}
+.highlight-link {
+  color: #00c;
+}
+/* apply the same style to codemirror */
+.cm-s-ipython span.cm-keyword {
+  color: #008000;
+  font-weight: bold;
+}
+.cm-s-ipython span.cm-atom {
+  color: #88F;
+}
+.cm-s-ipython span.cm-number {
+  color: #080;
+}
+.cm-s-ipython span.cm-def {
+  color: #00f;
+}
+.cm-s-ipython span.cm-variable {
+  color: #000;
+}
+.cm-s-ipython span.cm-operator {
+  color: #AA22FF;
+  font-weight: bold;
+}
+.cm-s-ipython span.cm-variable-2 {
+  color: #1a1a1a;
+}
+.cm-s-ipython span.cm-variable-3 {
+  color: #333333;
+}
+.cm-s-ipython span.cm-comment {
+  color: #408080;
+  font-style: italic;
+}
+.cm-s-ipython span.cm-string {
+  color: #BA2121;
+}
+.cm-s-ipython span.cm-string-2 {
+  color: #f50;
+}
+.cm-s-ipython span.cm-meta {
+  color: #AA22FF;
+}
+.cm-s-ipython span.cm-qualifier {
+  color: #555;
+}
+.cm-s-ipython span.cm-builtin {
+  color: #008000;
+}
+.cm-s-ipython span.cm-bracket {
+  color: #997;
+}
+.cm-s-ipython span.cm-tag {
+  color: #170;
+}
+.cm-s-ipython span.cm-attribute {
+  color: #00c;
+}
+.cm-s-ipython span.cm-header {
+  color: blue;
+}
+.cm-s-ipython span.cm-quote {
+  color: #090;
+}
+.cm-s-ipython span.cm-link {
+  color: #00c;
+}
+.cm-s-ipython span.cm-error {
+  color: #f00;
+}
+.cm-s-ipython span.cm-tab {
+  background: url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAMCAYAAAAkuj5RAAAAAXNSR0IArs4c6QAAAGFJREFUSMft1LsRQFAQheHPowAKoACx3IgEKtaEHujDjORSgWTH/ZOdnZOcM/sgk/kFFWY0qV8foQwS4MKBCS3qR6ixBJvElOobYAtivseIE120FaowJPN75GMu8j/LfMwNjh4HUpwg4LUAAAAASUVORK5CYII=);
+  background-position: right;
+  background-repeat: no-repeat;
+}
+div.output_wrapper {
+  /* this position must be relative to enable descendents to be absolute within it */
+  position: relative;
+  /* Old browsers */
+  display: -webkit-box;
+  -webkit-box-orient: vertical;
+  -webkit-box-align: stretch;
+  display: -moz-box;
+  -moz-box-orient: vertical;
+  -moz-box-align: stretch;
+  display: box;
+  box-orient: vertical;
+  box-align: stretch;
+  /* Modern browsers */
+  display: flex;
+  flex-direction: column;
+  align-items: stretch;
+  z-index: 1;
+}
+/* class for the output area when it should be height-limited */
+div.output_scroll {
+  /* ideally, this would be max-height, but FF barfs all over that */
+  height: 24em;
+  /* FF needs this *and the wrapper* to specify full width, or it will shrinkwrap */
+  width: 100%;
+  overflow: auto;
+  border-radius: 2px;
+  -webkit-box-shadow: inset 0 2px 8px rgba(0, 0, 0, 0.8);
+  box-shadow: inset 0 2px 8px rgba(0, 0, 0, 0.8);
+  display: block;
+}
+/* output div while it is collapsed */
+div.output_collapsed {
+  margin: 0px;
+  padding: 0px;
+  /* Old browsers */
+  display: -webkit-box;
+  -webkit-box-orient: vertical;
+  -webkit-box-align: stretch;
+  display: -moz-box;
+  -moz-box-orient: vertical;
+  -moz-box-align: stretch;
+  display: box;
+  box-orient: vertical;
+  box-align: stretch;
+  /* Modern browsers */
+  display: flex;
+  flex-direction: column;
+  align-items: stretch;
+}
+div.out_prompt_overlay {
+  height: 100%;
+  padding: 0px 0.4em;
+  position: absolute;
+  border-radius: 2px;
+}
+div.out_prompt_overlay:hover {
+  /* use inner shadow to get border that is computed the same on WebKit/FF */
+  -webkit-box-shadow: inset 0 0 1px #000;
+  box-shadow: inset 0 0 1px #000;
+  background: rgba(240, 240, 240, 0.5);
+}
+div.output_prompt {
+  color: #D84315;
+}
+/* This class is the outer container of all output sections. */
+div.output_area {
+  padding: 0px;
+  page-break-inside: avoid;
+  /* Old browsers */
+  display: -webkit-box;
+  -webkit-box-orient: horizontal;
+  -webkit-box-align: stretch;
+  display: -moz-box;
+  -moz-box-orient: horizontal;
+  -moz-box-align: stretch;
+  display: box;
+  box-orient: horizontal;
+  box-align: stretch;
+  /* Modern browsers */
+  display: flex;
+  flex-direction: row;
+  align-items: stretch;
+}
+div.output_area .MathJax_Display {
+  text-align: left !important;
+}
+div.output_area .rendered_html table {
+  margin-left: 0;
+  margin-right: 0;
+}
+div.output_area .rendered_html img {
+  margin-left: 0;
+  margin-right: 0;
+}
+div.output_area img,
+div.output_area svg {
+  max-width: 100%;
+  height: auto;
+}
+div.output_area img.unconfined,
+div.output_area svg.unconfined {
+  max-width: none;
+}
+/* This is needed to protect the pre formating from global settings such
+   as that of bootstrap */
+.output {
+  /* Old browsers */
+  display: -webkit-box;
+  -webkit-box-orient: vertical;
+  -webkit-box-align: stretch;
+  display: -moz-box;
+  -moz-box-orient: vertical;
+  -moz-box-align: stretch;
+  display: box;
+  box-orient: vertical;
+  box-align: stretch;
+  /* Modern browsers */
+  display: flex;
+  flex-direction: column;
+  align-items: stretch;
+}
+@media (max-width: 540px) {
+  div.output_area {
+    /* Old browsers */
+    display: -webkit-box;
+    -webkit-box-orient: vertical;
+    -webkit-box-align: stretch;
+    display: -moz-box;
+    -moz-box-orient: vertical;
+    -moz-box-align: stretch;
+    display: box;
+    box-orient: vertical;
+    box-align: stretch;
+    /* Modern browsers */
+    display: flex;
+    flex-direction: column;
+    align-items: stretch;
+  }
+}
+div.output_area pre {
+  margin: 0;
+  padding: 0;
+  border: 0;
+  vertical-align: baseline;
+  color: black;
+  background-color: transparent;
+  border-radius: 0;
+}
+/* This class is for the output subarea inside the output_area and after
+   the prompt div. */
+div.output_subarea {
+  overflow-x: auto;
+  padding: 0.4em;
+  /* Old browsers */
+  -webkit-box-flex: 1;
+  -moz-box-flex: 1;
+  box-flex: 1;
+  /* Modern browsers */
+  flex: 1;
+  max-width: calc(100% - 14ex);
+}
+div.output_scroll div.output_subarea {
+  overflow-x: visible;
+}
+/* The rest of the output_* classes are for special styling of the different
+   output types */
+/* all text output has this class: */
+div.output_text {
+  text-align: left;
+  color: #000;
+  /* This has to match that of the the CodeMirror class line-height below */
+  line-height: 1.21429em;
+}
+/* stdout/stderr are 'text' as well as 'stream', but execute_result/error are *not* streams */
+div.output_stderr {
+  background: #fdd;
+  /* very light red background for stderr */
+}
+div.output_latex {
+  text-align: left;
+}
+/* Empty output_javascript divs should have no height */
+div.output_javascript:empty {
+  padding: 0;
+}
+.js-error {
+  color: darkred;
+}
+/* raw_input styles */
+div.raw_input_container {
+  line-height: 1.21429em;
+  padding-top: 5px;
+}
+pre.raw_input_prompt {
+  /* nothing needed here. */
+}
+input.raw_input {
+  font-family: monospace;
+  font-size: inherit;
+  color: inherit;
+  width: auto;
+  /* make sure input baseline aligns with prompt */
+  vertical-align: baseline;
+  /* padding + margin = 0.5em between prompt and cursor */
+  padding: 0em 0.25em;
+  margin: 0em 0.25em;
+}
+input.raw_input:focus {
+  box-shadow: none;
+}
+p.p-space {
+  margin-bottom: 10px;
+}
+div.output_unrecognized {
+  padding: 5px;
+  font-weight: bold;
+  color: red;
+}
+div.output_unrecognized a {
+  color: inherit;
+  text-decoration: none;
+}
+div.output_unrecognized a:hover {
+  color: inherit;
+  text-decoration: none;
+}
+.rendered_html {
+  color: #000;
+  /* any extras will just be numbers: */
+}
+.rendered_html em {
+  font-style: italic;
+}
+.rendered_html strong {
+  font-weight: bold;
+}
+.rendered_html u {
+  text-decoration: underline;
+}
+.rendered_html :link {
+  text-decoration: underline;
+}
+.rendered_html :visited {
+  text-decoration: underline;
+}
+.rendered_html h1 {
+  font-size: 185.7%;
+  margin: 1.08em 0 0 0;
+  font-weight: bold;
+  line-height: 1.0;
+}
+.rendered_html h2 {
+  font-size: 157.1%;
+  margin: 1.27em 0 0 0;
+  font-weight: bold;
+  line-height: 1.0;
+}
+.rendered_html h3 {
+  font-size: 128.6%;
+  margin: 1.55em 0 0 0;
+  font-weight: bold;
+  line-height: 1.0;
+}
+.rendered_html h4 {
+  font-size: 100%;
+  margin: 2em 0 0 0;
+  font-weight: bold;
+  line-height: 1.0;
+}
+.rendered_html h5 {
+  font-size: 100%;
+  margin: 2em 0 0 0;
+  font-weight: bold;
+  line-height: 1.0;
+  font-style: italic;
+}
+.rendered_html h6 {
+  font-size: 100%;
+  margin: 2em 0 0 0;
+  font-weight: bold;
+  line-height: 1.0;
+  font-style: italic;
+}
+.rendered_html h1:first-child {
+  margin-top: 0.538em;
+}
+.rendered_html h2:first-child {
+  margin-top: 0.636em;
+}
+.rendered_html h3:first-child {
+  margin-top: 0.777em;
+}
+.rendered_html h4:first-child {
+  margin-top: 1em;
+}
+.rendered_html h5:first-child {
+  margin-top: 1em;
+}
+.rendered_html h6:first-child {
+  margin-top: 1em;
+}
+.rendered_html ul {
+  list-style: disc;
+  margin: 0em 2em;
+  padding-left: 0px;
+}
+.rendered_html ul ul {
+  list-style: square;
+  margin: 0em 2em;
+}
+.rendered_html ul ul ul {
+  list-style: circle;
+  margin: 0em 2em;
+}
+.rendered_html ol {
+  list-style: decimal;
+  margin: 0em 2em;
+  padding-left: 0px;
+}
+.rendered_html ol ol {
+  list-style: upper-alpha;
+  margin: 0em 2em;
+}
+.rendered_html ol ol ol {
+  list-style: lower-alpha;
+  margin: 0em 2em;
+}
+.rendered_html ol ol ol ol {
+  list-style: lower-roman;
+  margin: 0em 2em;
+}
+.rendered_html ol ol ol ol ol {
+  list-style: decimal;
+  margin: 0em 2em;
+}
+.rendered_html * + ul {
+  margin-top: 1em;
+}
+.rendered_html * + ol {
+  margin-top: 1em;
+}
+.rendered_html hr {
+  color: black;
+  background-color: black;
+}
+.rendered_html pre {
+  margin: 1em 2em;
+}
+.rendered_html pre,
+.rendered_html code {
+  border: 0;
+  background-color: #fff;
+  color: #000;
+  font-size: 100%;
+  padding: 0px;
+}
+.rendered_html blockquote {
+  margin: 1em 2em;
+}
+.rendered_html table {
+  margin-left: auto;
+  margin-right: auto;
+  border: 1px solid black;
+  border-collapse: collapse;
+}
+.rendered_html tr,
+.rendered_html th,
+.rendered_html td {
+  border: 1px solid black;
+  border-collapse: collapse;
+  margin: 1em 2em;
+}
+.rendered_html td,
+.rendered_html th {
+  text-align: left;
+  vertical-align: middle;
+  padding: 4px;
+}
+.rendered_html th {
+  font-weight: bold;
+}
+.rendered_html * + table {
+  margin-top: 1em;
+}
+.rendered_html p {
+  text-align: left;
+}
+.rendered_html * + p {
+  margin-top: 1em;
+}
+.rendered_html img {
+  display: block;
+  margin-left: auto;
+  margin-right: auto;
+}
+.rendered_html * + img {
+  margin-top: 1em;
+}
+.rendered_html img,
+.rendered_html svg {
+  max-width: 100%;
+  height: auto;
+}
+.rendered_html img.unconfined,
+.rendered_html svg.unconfined {
+  max-width: none;
+}
+div.text_cell {
+  /* Old browsers */
+  display: -webkit-box;
+  -webkit-box-orient: horizontal;
+  -webkit-box-align: stretch;
+  display: -moz-box;
+  -moz-box-orient: horizontal;
+  -moz-box-align: stretch;
+  display: box;
+  box-orient: horizontal;
+  box-align: stretch;
+  /* Modern browsers */
+  display: flex;
+  flex-direction: row;
+  align-items: stretch;
+}
+@media (max-width: 540px) {
+  div.text_cell > div.prompt {
+    display: none;
+  }
+}
+div.text_cell_render {
+  /*font-family: "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;*/
+  outline: none;
+  resize: none;
+  width: inherit;
+  border-style: none;
+  padding: 0.5em 0.5em 0.5em 0.4em;
+  color: #000;
+  box-sizing: border-box;
+  -moz-box-sizing: border-box;
+  -webkit-box-sizing: border-box;
+}
+a.anchor-link:link {
+  text-decoration: none;
+  padding: 0px 20px;
+  visibility: hidden;
+}
+h1:hover .anchor-link,
+h2:hover .anchor-link,
+h3:hover .anchor-link,
+h4:hover .anchor-link,
+h5:hover .anchor-link,
+h6:hover .anchor-link {
+  visibility: visible;
+}
+.text_cell.rendered .input_area {
+  display: none;
+}
+.text_cell.rendered .rendered_html {
+  overflow-x: auto;
+  overflow-y: hidden;
+}
+.text_cell.unrendered .text_cell_render {
+  display: none;
+}
+.cm-header-1,
+.cm-header-2,
+.cm-header-3,
+.cm-header-4,
+.cm-header-5,
+.cm-header-6 {
+  font-weight: bold;
+  font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
+}
+.cm-header-1 {
+  font-size: 185.7%;
+}
+.cm-header-2 {
+  font-size: 157.1%;
+}
+.cm-header-3 {
+  font-size: 128.6%;
+}
+.cm-header-4 {
+  font-size: 110%;
+}
+.cm-header-5 {
+  font-size: 100%;
+  font-style: italic;
+}
+.cm-header-6 {
+  font-size: 100%;
+  font-style: italic;
+}
+/*!
+*
+* IPython notebook webapp
+*
+*/
+@media (max-width: 767px) {
+  .notebook_app {
+    padding-left: 0px;
+    padding-right: 0px;
+  }
+}
+#ipython-main-app {
+  box-sizing: border-box;
+  -moz-box-sizing: border-box;
+  -webkit-box-sizing: border-box;
+  height: 100%;
+}
+div#notebook_panel {
+  margin: 0px;
+  padding: 0px;
+  box-sizing: border-box;
+  -moz-box-sizing: border-box;
+  -webkit-box-sizing: border-box;
+  height: 100%;
+}
+div#notebook {
+  font-size: 14px;
+  line-height: 20px;
+  overflow-y: hidden;
+  overflow-x: auto;
+  width: 100%;
+  /* This spaces the page away from the edge of the notebook area */
+  padding-top: 20px;
+  margin: 0px;
+  outline: none;
+  box-sizing: border-box;
+  -moz-box-sizing: border-box;
+  -webkit-box-sizing: border-box;
+  min-height: 100%;
+}
+@media not print {
+  #notebook-container {
+    padding: 15px;
+    background-color: #fff;
+    min-height: 0;
+    -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
+    box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
+  }
+}
+@media print {
+  #notebook-container {
+    width: 100%;
+  }
+}
+div.ui-widget-content {
+  border: 1px solid #ababab;
+  outline: none;
+}
+pre.dialog {
+  background-color: #f7f7f7;
+  border: 1px solid #ddd;
+  border-radius: 2px;
+  padding: 0.4em;
+  padding-left: 2em;
+}
+p.dialog {
+  padding: 0.2em;
+}
+/* Word-wrap output correctly.  This is the CSS3 spelling, though Firefox seems
+   to not honor it correctly.  Webkit browsers (Chrome, rekonq, Safari) do.
+ */
+pre,
+code,
+kbd,
+samp {
+  white-space: pre-wrap;
+}
+#fonttest {
+  font-family: monospace;
+}
+p {
+  margin-bottom: 0;
+}
+.end_space {
+  min-height: 100px;
+  transition: height .2s ease;
+}
+.notebook_app > #header {
+  -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
+  box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
+}
+@media not print {
+  .notebook_app {
+    background-color: #EEE;
+  }
+}
+kbd {
+  border-style: solid;
+  border-width: 1px;
+  box-shadow: none;
+  margin: 2px;
+  padding-left: 2px;
+  padding-right: 2px;
+  padding-top: 1px;
+  padding-bottom: 1px;
+}
+/* CSS for the cell toolbar */
+.celltoolbar {
+  border: thin solid #CFCFCF;
+  border-bottom: none;
+  background: #EEE;
+  border-radius: 2px 2px 0px 0px;
+  width: 100%;
+  height: 29px;
+  padding-right: 4px;
+  /* Old browsers */
+  display: -webkit-box;
+  -webkit-box-orient: horizontal;
+  -webkit-box-align: stretch;
+  display: -moz-box;
+  -moz-box-orient: horizontal;
+  -moz-box-align: stretch;
+  display: box;
+  box-orient: horizontal;
+  box-align: stretch;
+  /* Modern browsers */
+  display: flex;
+  flex-direction: row;
+  align-items: stretch;
+  /* Old browsers */
+  -webkit-box-pack: end;
+  -moz-box-pack: end;
+  box-pack: end;
+  /* Modern browsers */
+  justify-content: flex-end;
+  display: -webkit-flex;
+}
+@media print {
+  .celltoolbar {
+    display: none;
+  }
+}
+.ctb_hideshow {
+  display: none;
+  vertical-align: bottom;
+}
+/* ctb_show is added to the ctb_hideshow div to show the cell toolbar.
+   Cell toolbars are only shown when the ctb_global_show class is also set.
+*/
+.ctb_global_show .ctb_show.ctb_hideshow {
+  display: block;
+}
+.ctb_global_show .ctb_show + .input_area,
+.ctb_global_show .ctb_show + div.text_cell_input,
+.ctb_global_show .ctb_show ~ div.text_cell_render {
+  border-top-right-radius: 0px;
+  border-top-left-radius: 0px;
+}
+.ctb_global_show .ctb_show ~ div.text_cell_render {
+  border: 1px solid #cfcfcf;
+}
+.celltoolbar {
+  font-size: 87%;
+  padding-top: 3px;
+}
+.celltoolbar select {
+  display: block;
+  width: 100%;
+  height: 32px;
+  padding: 6px 12px;
+  font-size: 13px;
+  line-height: 1.42857143;
+  color: #555555;
+  background-color: #fff;
+  background-image: none;
+  border: 1px solid #ccc;
+  border-radius: 2px;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+  -webkit-transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
+  -o-transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
+  transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
+  height: 30px;
+  padding: 5px 10px;
+  font-size: 12px;
+  line-height: 1.5;
+  border-radius: 1px;
+  width: inherit;
+  font-size: inherit;
+  height: 22px;
+  padding: 0px;
+  display: inline-block;
+}
+.celltoolbar select:focus {
+  border-color: #66afe9;
+  outline: 0;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0,0,0,.075), 0 0 8px rgba(102, 175, 233, 0.6);
+  box-shadow: inset 0 1px 1px rgba(0,0,0,.075), 0 0 8px rgba(102, 175, 233, 0.6);
+}
+.celltoolbar select::-moz-placeholder {
+  color: #999;
+  opacity: 1;
+}
+.celltoolbar select:-ms-input-placeholder {
+  color: #999;
+}
+.celltoolbar select::-webkit-input-placeholder {
+  color: #999;
+}
+.celltoolbar select::-ms-expand {
+  border: 0;
+  background-color: transparent;
+}
+.celltoolbar select[disabled],
+.celltoolbar select[readonly],
+fieldset[disabled] .celltoolbar select {
+  background-color: #eeeeee;
+  opacity: 1;
+}
+.celltoolbar select[disabled],
+fieldset[disabled] .celltoolbar select {
+  cursor: not-allowed;
+}
+textarea.celltoolbar select {
+  height: auto;
+}
+select.celltoolbar select {
+  height: 30px;
+  line-height: 30px;
+}
+textarea.celltoolbar select,
+select[multiple].celltoolbar select {
+  height: auto;
+}
+.celltoolbar label {
+  margin-left: 5px;
+  margin-right: 5px;
+}
+.completions {
+  position: absolute;
+  z-index: 110;
+  overflow: hidden;
+  border: 1px solid #ababab;
+  border-radius: 2px;
+  -webkit-box-shadow: 0px 6px 10px -1px #adadad;
+  box-shadow: 0px 6px 10px -1px #adadad;
+  line-height: 1;
+}
+.completions select {
+  background: white;
+  outline: none;
+  border: none;
+  padding: 0px;
+  margin: 0px;
+  overflow: auto;
+  font-family: monospace;
+  font-size: 110%;
+  color: #000;
+  width: auto;
+}
+.completions select option.context {
+  color: #286090;
+}
+#kernel_logo_widget {
+  float: right !important;
+  float: right;
+}
+#kernel_logo_widget .current_kernel_logo {
+  display: none;
+  margin-top: -1px;
+  margin-bottom: -1px;
+  width: 32px;
+  height: 32px;
+}
+#menubar {
+  box-sizing: border-box;
+  -moz-box-sizing: border-box;
+  -webkit-box-sizing: border-box;
+  margin-top: 1px;
+}
+#menubar .navbar {
+  border-top: 1px;
+  border-radius: 0px 0px 2px 2px;
+  margin-bottom: 0px;
+}
+#menubar .navbar-toggle {
+  float: left;
+  padding-top: 7px;
+  padding-bottom: 7px;
+  border: none;
+}
+#menubar .navbar-collapse {
+  clear: left;
+}
+.nav-wrapper {
+  border-bottom: 1px solid #e7e7e7;
+}
+i.menu-icon {
+  padding-top: 4px;
+}
+ul#help_menu li a {
+  overflow: hidden;
+  padding-right: 2.2em;
+}
+ul#help_menu li a i {
+  margin-right: -1.2em;
+}
+.dropdown-submenu {
+  position: relative;
+}
+.dropdown-submenu > .dropdown-menu {
+  top: 0;
+  left: 100%;
+  margin-top: -6px;
+  margin-left: -1px;
+}
+.dropdown-submenu:hover > .dropdown-menu {
+  display: block;
+}
+.dropdown-submenu > a:after {
+  display: inline-block;
+  font: normal normal normal 14px/1 FontAwesome;
+  font-size: inherit;
+  text-rendering: auto;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+  display: block;
+  content: "\f0da";
+  float: right;
+  color: #333333;
+  margin-top: 2px;
+  margin-right: -10px;
+}
+.dropdown-submenu > a:after.pull-left {
+  margin-right: .3em;
+}
+.dropdown-submenu > a:after.pull-right {
+  margin-left: .3em;
+}
+.dropdown-submenu:hover > a:after {
+  color: #262626;
+}
+.dropdown-submenu.pull-left {
+  float: none;
+}
+.dropdown-submenu.pull-left > .dropdown-menu {
+  left: -100%;
+  margin-left: 10px;
+}
+#notification_area {
+  float: right !important;
+  float: right;
+  z-index: 10;
+}
+.indicator_area {
+  float: right !important;
+  float: right;
+  color: #777;
+  margin-left: 5px;
+  margin-right: 5px;
+  width: 11px;
+  z-index: 10;
+  text-align: center;
+  width: auto;
+}
+#kernel_indicator {
+  float: right !important;
+  float: right;
+  color: #777;
+  margin-left: 5px;
+  margin-right: 5px;
+  width: 11px;
+  z-index: 10;
+  text-align: center;
+  width: auto;
+  border-left: 1px solid;
+}
+#kernel_indicator .kernel_indicator_name {
+  padding-left: 5px;
+  padding-right: 5px;
+}
+#modal_indicator {
+  float: right !important;
+  float: right;
+  color: #777;
+  margin-left: 5px;
+  margin-right: 5px;
+  width: 11px;
+  z-index: 10;
+  text-align: center;
+  width: auto;
+}
+#readonly-indicator {
+  float: right !important;
+  float: right;
+  color: #777;
+  margin-left: 5px;
+  margin-right: 5px;
+  width: 11px;
+  z-index: 10;
+  text-align: center;
+  width: auto;
+  margin-top: 2px;
+  margin-bottom: 0px;
+  margin-left: 0px;
+  margin-right: 0px;
+  display: none;
+}
+.modal_indicator:before {
+  width: 1.28571429em;
+  text-align: center;
+}
+.edit_mode .modal_indicator:before {
+  display: inline-block;
+  font: normal normal normal 14px/1 FontAwesome;
+  font-size: inherit;
+  text-rendering: auto;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+  content: "\f040";
+}
+.edit_mode .modal_indicator:before.pull-left {
+  margin-right: .3em;
+}
+.edit_mode .modal_indicator:before.pull-right {
+  margin-left: .3em;
+}
+.command_mode .modal_indicator:before {
+  display: inline-block;
+  font: normal normal normal 14px/1 FontAwesome;
+  font-size: inherit;
+  text-rendering: auto;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+  content: ' ';
+}
+.command_mode .modal_indicator:before.pull-left {
+  margin-right: .3em;
+}
+.command_mode .modal_indicator:before.pull-right {
+  margin-left: .3em;
+}
+.kernel_idle_icon:before {
+  display: inline-block;
+  font: normal normal normal 14px/1 FontAwesome;
+  font-size: inherit;
+  text-rendering: auto;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+  content: "\f10c";
+}
+.kernel_idle_icon:before.pull-left {
+  margin-right: .3em;
+}
+.kernel_idle_icon:before.pull-right {
+  margin-left: .3em;
+}
+.kernel_busy_icon:before {
+  display: inline-block;
+  font: normal normal normal 14px/1 FontAwesome;
+  font-size: inherit;
+  text-rendering: auto;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+  content: "\f111";
+}
+.kernel_busy_icon:before.pull-left {
+  margin-right: .3em;
+}
+.kernel_busy_icon:before.pull-right {
+  margin-left: .3em;
+}
+.kernel_dead_icon:before {
+  display: inline-block;
+  font: normal normal normal 14px/1 FontAwesome;
+  font-size: inherit;
+  text-rendering: auto;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+  content: "\f1e2";
+}
+.kernel_dead_icon:before.pull-left {
+  margin-right: .3em;
+}
+.kernel_dead_icon:before.pull-right {
+  margin-left: .3em;
+}
+.kernel_disconnected_icon:before {
+  display: inline-block;
+  font: normal normal normal 14px/1 FontAwesome;
+  font-size: inherit;
+  text-rendering: auto;
+  -webkit-font-smoothing: antialiased;
+  -moz-osx-font-smoothing: grayscale;
+  content: "\f127";
+}
+.kernel_disconnected_icon:before.pull-left {
+  margin-right: .3em;
+}
+.kernel_disconnected_icon:before.pull-right {
+  margin-left: .3em;
+}
+.notification_widget {
+  color: #777;
+  z-index: 10;
+  background: rgba(240, 240, 240, 0.5);
+  margin-right: 4px;
+  color: #333;
+  background-color: #fff;
+  border-color: #ccc;
+}
+.notification_widget:focus,
+.notification_widget.focus {
+  color: #333;
+  background-color: #e6e6e6;
+  border-color: #8c8c8c;
+}
+.notification_widget:hover {
+  color: #333;
+  background-color: #e6e6e6;
+  border-color: #adadad;
+}
+.notification_widget:active,
+.notification_widget.active,
+.open > .dropdown-toggle.notification_widget {
+  color: #333;
+  background-color: #e6e6e6;
+  border-color: #adadad;
+}
+.notification_widget:active:hover,
+.notification_widget.active:hover,
+.open > .dropdown-toggle.notification_widget:hover,
+.notification_widget:active:focus,
+.notification_widget.active:focus,
+.open > .dropdown-toggle.notification_widget:focus,
+.notification_widget:active.focus,
+.notification_widget.active.focus,
+.open > .dropdown-toggle.notification_widget.focus {
+  color: #333;
+  background-color: #d4d4d4;
+  border-color: #8c8c8c;
+}
+.notification_widget:active,
+.notification_widget.active,
+.open > .dropdown-toggle.notification_widget {
+  background-image: none;
+}
+.notification_widget.disabled:hover,
+.notification_widget[disabled]:hover,
+fieldset[disabled] .notification_widget:hover,
+.notification_widget.disabled:focus,
+.notification_widget[disabled]:focus,
+fieldset[disabled] .notification_widget:focus,
+.notification_widget.disabled.focus,
+.notification_widget[disabled].focus,
+fieldset[disabled] .notification_widget.focus {
+  background-color: #fff;
+  border-color: #ccc;
+}
+.notification_widget .badge {
+  color: #fff;
+  background-color: #333;
+}
+.notification_widget.warning {
+  color: #fff;
+  background-color: #f0ad4e;
+  border-color: #eea236;
+}
+.notification_widget.warning:focus,
+.notification_widget.warning.focus {
+  color: #fff;
+  background-color: #ec971f;
+  border-color: #985f0d;
+}
+.notification_widget.warning:hover {
+  color: #fff;
+  background-color: #ec971f;
+  border-color: #d58512;
+}
+.notification_widget.warning:active,
+.notification_widget.warning.active,
+.open > .dropdown-toggle.notification_widget.warning {
+  color: #fff;
+  background-color: #ec971f;
+  border-color: #d58512;
+}
+.notification_widget.warning:active:hover,
+.notification_widget.warning.active:hover,
+.open > .dropdown-toggle.notification_widget.warning:hover,
+.notification_widget.warning:active:focus,
+.notification_widget.warning.active:focus,
+.open > .dropdown-toggle.notification_widget.warning:focus,
+.notification_widget.warning:active.focus,
+.notification_widget.warning.active.focus,
+.open > .dropdown-toggle.notification_widget.warning.focus {
+  color: #fff;
+  background-color: #d58512;
+  border-color: #985f0d;
+}
+.notification_widget.warning:active,
+.notification_widget.warning.active,
+.open > .dropdown-toggle.notification_widget.warning {
+  background-image: none;
+}
+.notification_widget.warning.disabled:hover,
+.notification_widget.warning[disabled]:hover,
+fieldset[disabled] .notification_widget.warning:hover,
+.notification_widget.warning.disabled:focus,
+.notification_widget.warning[disabled]:focus,
+fieldset[disabled] .notification_widget.warning:focus,
+.notification_widget.warning.disabled.focus,
+.notification_widget.warning[disabled].focus,
+fieldset[disabled] .notification_widget.warning.focus {
+  background-color: #f0ad4e;
+  border-color: #eea236;
+}
+.notification_widget.warning .badge {
+  color: #f0ad4e;
+  background-color: #fff;
+}
+.notification_widget.success {
+  color: #fff;
+  background-color: #5cb85c;
+  border-color: #4cae4c;
+}
+.notification_widget.success:focus,
+.notification_widget.success.focus {
+  color: #fff;
+  background-color: #449d44;
+  border-color: #255625;
+}
+.notification_widget.success:hover {
+  color: #fff;
+  background-color: #449d44;
+  border-color: #398439;
+}
+.notification_widget.success:active,
+.notification_widget.success.active,
+.open > .dropdown-toggle.notification_widget.success {
+  color: #fff;
+  background-color: #449d44;
+  border-color: #398439;
+}
+.notification_widget.success:active:hover,
+.notification_widget.success.active:hover,
+.open > .dropdown-toggle.notification_widget.success:hover,
+.notification_widget.success:active:focus,
+.notification_widget.success.active:focus,
+.open > .dropdown-toggle.notification_widget.success:focus,
+.notification_widget.success:active.focus,
+.notification_widget.success.active.focus,
+.open > .dropdown-toggle.notification_widget.success.focus {
+  color: #fff;
+  background-color: #398439;
+  border-color: #255625;
+}
+.notification_widget.success:active,
+.notification_widget.success.active,
+.open > .dropdown-toggle.notification_widget.success {
+  background-image: none;
+}
+.notification_widget.success.disabled:hover,
+.notification_widget.success[disabled]:hover,
+fieldset[disabled] .notification_widget.success:hover,
+.notification_widget.success.disabled:focus,
+.notification_widget.success[disabled]:focus,
+fieldset[disabled] .notification_widget.success:focus,
+.notification_widget.success.disabled.focus,
+.notification_widget.success[disabled].focus,
+fieldset[disabled] .notification_widget.success.focus {
+  background-color: #5cb85c;
+  border-color: #4cae4c;
+}
+.notification_widget.success .badge {
+  color: #5cb85c;
+  background-color: #fff;
+}
+.notification_widget.info {
+  color: #fff;
+  background-color: #5bc0de;
+  border-color: #46b8da;
+}
+.notification_widget.info:focus,
+.notification_widget.info.focus {
+  color: #fff;
+  background-color: #31b0d5;
+  border-color: #1b6d85;
+}
+.notification_widget.info:hover {
+  color: #fff;
+  background-color: #31b0d5;
+  border-color: #269abc;
+}
+.notification_widget.info:active,
+.notification_widget.info.active,
+.open > .dropdown-toggle.notification_widget.info {
+  color: #fff;
+  background-color: #31b0d5;
+  border-color: #269abc;
+}
+.notification_widget.info:active:hover,
+.notification_widget.info.active:hover,
+.open > .dropdown-toggle.notification_widget.info:hover,
+.notification_widget.info:active:focus,
+.notification_widget.info.active:focus,
+.open > .dropdown-toggle.notification_widget.info:focus,
+.notification_widget.info:active.focus,
+.notification_widget.info.active.focus,
+.open > .dropdown-toggle.notification_widget.info.focus {
+  color: #fff;
+  background-color: #269abc;
+  border-color: #1b6d85;
+}
+.notification_widget.info:active,
+.notification_widget.info.active,
+.open > .dropdown-toggle.notification_widget.info {
+  background-image: none;
+}
+.notification_widget.info.disabled:hover,
+.notification_widget.info[disabled]:hover,
+fieldset[disabled] .notification_widget.info:hover,
+.notification_widget.info.disabled:focus,
+.notification_widget.info[disabled]:focus,
+fieldset[disabled] .notification_widget.info:focus,
+.notification_widget.info.disabled.focus,
+.notification_widget.info[disabled].focus,
+fieldset[disabled] .notification_widget.info.focus {
+  background-color: #5bc0de;
+  border-color: #46b8da;
+}
+.notification_widget.info .badge {
+  color: #5bc0de;
+  background-color: #fff;
+}
+.notification_widget.danger {
+  color: #fff;
+  background-color: #d9534f;
+  border-color: #d43f3a;
+}
+.notification_widget.danger:focus,
+.notification_widget.danger.focus {
+  color: #fff;
+  background-color: #c9302c;
+  border-color: #761c19;
+}
+.notification_widget.danger:hover {
+  color: #fff;
+  background-color: #c9302c;
+  border-color: #ac2925;
+}
+.notification_widget.danger:active,
+.notification_widget.danger.active,
+.open > .dropdown-toggle.notification_widget.danger {
+  color: #fff;
+  background-color: #c9302c;
+  border-color: #ac2925;
+}
+.notification_widget.danger:active:hover,
+.notification_widget.danger.active:hover,
+.open > .dropdown-toggle.notification_widget.danger:hover,
+.notification_widget.danger:active:focus,
+.notification_widget.danger.active:focus,
+.open > .dropdown-toggle.notification_widget.danger:focus,
+.notification_widget.danger:active.focus,
+.notification_widget.danger.active.focus,
+.open > .dropdown-toggle.notification_widget.danger.focus {
+  color: #fff;
+  background-color: #ac2925;
+  border-color: #761c19;
+}
+.notification_widget.danger:active,
+.notification_widget.danger.active,
+.open > .dropdown-toggle.notification_widget.danger {
+  background-image: none;
+}
+.notification_widget.danger.disabled:hover,
+.notification_widget.danger[disabled]:hover,
+fieldset[disabled] .notification_widget.danger:hover,
+.notification_widget.danger.disabled:focus,
+.notification_widget.danger[disabled]:focus,
+fieldset[disabled] .notification_widget.danger:focus,
+.notification_widget.danger.disabled.focus,
+.notification_widget.danger[disabled].focus,
+fieldset[disabled] .notification_widget.danger.focus {
+  background-color: #d9534f;
+  border-color: #d43f3a;
+}
+.notification_widget.danger .badge {
+  color: #d9534f;
+  background-color: #fff;
+}
+div#pager {
+  background-color: #fff;
+  font-size: 14px;
+  line-height: 20px;
+  overflow: hidden;
+  display: none;
+  position: fixed;
+  bottom: 0px;
+  width: 100%;
+  max-height: 50%;
+  padding-top: 8px;
+  -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
+  box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
+  /* Display over codemirror */
+  z-index: 100;
+  /* Hack which prevents jquery ui resizable from changing top. */
+  top: auto !important;
+}
+div#pager pre {
+  line-height: 1.21429em;
+  color: #000;
+  background-color: #f7f7f7;
+  padding: 0.4em;
+}
+div#pager #pager-button-area {
+  position: absolute;
+  top: 8px;
+  right: 20px;
+}
+div#pager #pager-contents {
+  position: relative;
+  overflow: auto;
+  width: 100%;
+  height: 100%;
+}
+div#pager #pager-contents #pager-container {
+  position: relative;
+  padding: 15px 0px;
+  box-sizing: border-box;
+  -moz-box-sizing: border-box;
+  -webkit-box-sizing: border-box;
+}
+div#pager .ui-resizable-handle {
+  top: 0px;
+  height: 8px;
+  background: #f7f7f7;
+  border-top: 1px solid #cfcfcf;
+  border-bottom: 1px solid #cfcfcf;
+  /* This injects handle bars (a short, wide = symbol) for 
+        the resize handle. */
+}
+div#pager .ui-resizable-handle::after {
+  content: '';
+  top: 2px;
+  left: 50%;
+  height: 3px;
+  width: 30px;
+  margin-left: -15px;
+  position: absolute;
+  border-top: 1px solid #cfcfcf;
+}
+.quickhelp {
+  /* Old browsers */
+  display: -webkit-box;
+  -webkit-box-orient: horizontal;
+  -webkit-box-align: stretch;
+  display: -moz-box;
+  -moz-box-orient: horizontal;
+  -moz-box-align: stretch;
+  display: box;
+  box-orient: horizontal;
+  box-align: stretch;
+  /* Modern browsers */
+  display: flex;
+  flex-direction: row;
+  align-items: stretch;
+  line-height: 1.8em;
+}
+.shortcut_key {
+  display: inline-block;
+  width: 20ex;
+  text-align: right;
+  font-family: monospace;
+}
+.shortcut_descr {
+  display: inline-block;
+  /* Old browsers */
+  -webkit-box-flex: 1;
+  -moz-box-flex: 1;
+  box-flex: 1;
+  /* Modern browsers */
+  flex: 1;
+}
+span.save_widget {
+  margin-top: 6px;
+}
+span.save_widget span.filename {
+  height: 1em;
+  line-height: 1em;
+  padding: 3px;
+  margin-left: 16px;
+  border: none;
+  font-size: 146.5%;
+  border-radius: 2px;
+}
+span.save_widget span.filename:hover {
+  background-color: #e6e6e6;
+}
+span.checkpoint_status,
+span.autosave_status {
+  font-size: small;
+}
+@media (max-width: 767px) {
+  span.save_widget {
+    font-size: small;
+  }
+  span.checkpoint_status,
+  span.autosave_status {
+    display: none;
+  }
+}
+@media (min-width: 768px) and (max-width: 991px) {
+  span.checkpoint_status {
+    display: none;
+  }
+  span.autosave_status {
+    font-size: x-small;
+  }
+}
+.toolbar {
+  padding: 0px;
+  margin-left: -5px;
+  margin-top: 2px;
+  margin-bottom: 5px;
+  box-sizing: border-box;
+  -moz-box-sizing: border-box;
+  -webkit-box-sizing: border-box;
+}
+.toolbar select,
+.toolbar label {
+  width: auto;
+  vertical-align: middle;
+  margin-right: 2px;
+  margin-bottom: 0px;
+  display: inline;
+  font-size: 92%;
+  margin-left: 0.3em;
+  margin-right: 0.3em;
+  padding: 0px;
+  padding-top: 3px;
+}
+.toolbar .btn {
+  padding: 2px 8px;
+}
+.toolbar .btn-group {
+  margin-top: 0px;
+  margin-left: 5px;
+}
+#maintoolbar {
+  margin-bottom: -3px;
+  margin-top: -8px;
+  border: 0px;
+  min-height: 27px;
+  margin-left: 0px;
+  padding-top: 11px;
+  padding-bottom: 3px;
+}
+#maintoolbar .navbar-text {
+  float: none;
+  vertical-align: middle;
+  text-align: right;
+  margin-left: 5px;
+  margin-right: 0px;
+  margin-top: 0px;
+}
+.select-xs {
+  height: 24px;
+}
+.pulse,
+.dropdown-menu > li > a.pulse,
+li.pulse > a.dropdown-toggle,
+li.pulse.open > a.dropdown-toggle {
+  background-color: #F37626;
+  color: white;
+}
+/**
+ * Primary styles
+ *
+ * Author: Jupyter Development Team
+ */
+/** WARNING IF YOU ARE EDITTING THIS FILE, if this is a .css file, It has a lot
+ * of chance of beeing generated from the ../less/[samename].less file, you can
+ * try to get back the less file by reverting somme commit in history
+ **/
+/*
+ * We'll try to get something pretty, so we
+ * have some strange css to have the scroll bar on
+ * the left with fix button on the top right of the tooltip
+ */
+@-moz-keyframes fadeOut {
+  from {
+    opacity: 1;
+  }
+  to {
+    opacity: 0;
+  }
+}
+@-webkit-keyframes fadeOut {
+  from {
+    opacity: 1;
+  }
+  to {
+    opacity: 0;
+  }
+}
+@-moz-keyframes fadeIn {
+  from {
+    opacity: 0;
+  }
+  to {
+    opacity: 1;
+  }
+}
+@-webkit-keyframes fadeIn {
+  from {
+    opacity: 0;
+  }
+  to {
+    opacity: 1;
+  }
+}
+/*properties of tooltip after "expand"*/
+.bigtooltip {
+  overflow: auto;
+  height: 200px;
+  -webkit-transition-property: height;
+  -webkit-transition-duration: 500ms;
+  -moz-transition-property: height;
+  -moz-transition-duration: 500ms;
+  transition-property: height;
+  transition-duration: 500ms;
+}
+/*properties of tooltip before "expand"*/
+.smalltooltip {
+  -webkit-transition-property: height;
+  -webkit-transition-duration: 500ms;
+  -moz-transition-property: height;
+  -moz-transition-duration: 500ms;
+  transition-property: height;
+  transition-duration: 500ms;
+  text-overflow: ellipsis;
+  overflow: hidden;
+  height: 80px;
+}
+.tooltipbuttons {
+  position: absolute;
+  padding-right: 15px;
+  top: 0px;
+  right: 0px;
+}
+.tooltiptext {
+  /*avoid the button to overlap on some docstring*/
+  padding-right: 30px;
+}
+.ipython_tooltip {
+  max-width: 700px;
+  /*fade-in animation when inserted*/
+  -webkit-animation: fadeOut 400ms;
+  -moz-animation: fadeOut 400ms;
+  animation: fadeOut 400ms;
+  -webkit-animation: fadeIn 400ms;
+  -moz-animation: fadeIn 400ms;
+  animation: fadeIn 400ms;
+  vertical-align: middle;
+  background-color: #f7f7f7;
+  overflow: visible;
+  border: #ababab 1px solid;
+  outline: none;
+  padding: 3px;
+  margin: 0px;
+  padding-left: 7px;
+  font-family: monospace;
+  min-height: 50px;
+  -moz-box-shadow: 0px 6px 10px -1px #adadad;
+  -webkit-box-shadow: 0px 6px 10px -1px #adadad;
+  box-shadow: 0px 6px 10px -1px #adadad;
+  border-radius: 2px;
+  position: absolute;
+  z-index: 1000;
+}
+.ipython_tooltip a {
+  float: right;
+}
+.ipython_tooltip .tooltiptext pre {
+  border: 0;
+  border-radius: 0;
+  font-size: 100%;
+  background-color: #f7f7f7;
+}
+.pretooltiparrow {
+  left: 0px;
+  margin: 0px;
+  top: -16px;
+  width: 40px;
+  height: 16px;
+  overflow: hidden;
+  position: absolute;
+}
+.pretooltiparrow:before {
+  background-color: #f7f7f7;
+  border: 1px #ababab solid;
+  z-index: 11;
+  content: "";
+  position: absolute;
+  left: 15px;
+  top: 10px;
+  width: 25px;
+  height: 25px;
+  -webkit-transform: rotate(45deg);
+  -moz-transform: rotate(45deg);
+  -ms-transform: rotate(45deg);
+  -o-transform: rotate(45deg);
+}
+ul.typeahead-list i {
+  margin-left: -10px;
+  width: 18px;
+}
+ul.typeahead-list {
+  max-height: 80vh;
+  overflow: auto;
+}
+ul.typeahead-list > li > a {
+  /** Firefox bug **/
+  /* see https://github.com/jupyter/notebook/issues/559 */
+  white-space: normal;
+}
+.cmd-palette .modal-body {
+  padding: 7px;
+}
+.cmd-palette form {
+  background: white;
+}
+.cmd-palette input {
+  outline: none;
+}
+.no-shortcut {
+  display: none;
+}
+.command-shortcut:before {
+  content: "(command)";
+  padding-right: 3px;
+  color: #777777;
+}
+.edit-shortcut:before {
+  content: "(edit)";
+  padding-right: 3px;
+  color: #777777;
+}
+#find-and-replace #replace-preview .match,
+#find-and-replace #replace-preview .insert {
+  background-color: #BBDEFB;
+  border-color: #90CAF9;
+  border-style: solid;
+  border-width: 1px;
+  border-radius: 0px;
+}
+#find-and-replace #replace-preview .replace .match {
+  background-color: #FFCDD2;
+  border-color: #EF9A9A;
+  border-radius: 0px;
+}
+#find-and-replace #replace-preview .replace .insert {
+  background-color: #C8E6C9;
+  border-color: #A5D6A7;
+  border-radius: 0px;
+}
+#find-and-replace #replace-preview {
+  max-height: 60vh;
+  overflow: auto;
+}
+#find-and-replace #replace-preview pre {
+  padding: 5px 10px;
+}
+.terminal-app {
+  background: #EEE;
+}
+.terminal-app #header {
+  background: #fff;
+  -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
+  box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
+}
+.terminal-app .terminal {
+  float: left;
+  font-family: monospace;
+  color: white;
+  background: black;
+  padding: 0.4em;
+  border-radius: 2px;
+  -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.4);
+  box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.4);
+}
+.terminal-app .terminal,
+.terminal-app .terminal dummy-screen {
+  line-height: 1em;
+  font-size: 14px;
+}
+.terminal-app .terminal-cursor {
+  color: black;
+  background: white;
+}
+.terminal-app #terminado-container {
+  margin-top: 20px;
+}
+/*# sourceMappingURL=style.min.css.map */
+    </style>
+<style type="text/css">
+    .highlight .hll { background-color: #ffffcc }
+.highlight  { background: #f8f8f8; }
+.highlight .c { color: #408080; font-style: italic } /* Comment */
+.highlight .err { border: 1px solid #FF0000 } /* Error */
+.highlight .k { color: #008000; font-weight: bold } /* Keyword */
+.highlight .o { color: #666666 } /* Operator */
+.highlight .ch { color: #408080; font-style: italic } /* Comment.Hashbang */
+.highlight .cm { color: #408080; font-style: italic } /* Comment.Multiline */
+.highlight .cp { color: #BC7A00 } /* Comment.Preproc */
+.highlight .cpf { color: #408080; font-style: italic } /* Comment.PreprocFile */
+.highlight .c1 { color: #408080; font-style: italic } /* Comment.Single */
+.highlight .cs { color: #408080; font-style: italic } /* Comment.Special */
+.highlight .gd { color: #A00000 } /* Generic.Deleted */
+.highlight .ge { font-style: italic } /* Generic.Emph */
+.highlight .gr { color: #FF0000 } /* Generic.Error */
+.highlight .gh { color: #000080; font-weight: bold } /* Generic.Heading */
+.highlight .gi { color: #00A000 } /* Generic.Inserted */
+.highlight .go { color: #888888 } /* Generic.Output */
+.highlight .gp { color: #000080; font-weight: bold } /* Generic.Prompt */
+.highlight .gs { font-weight: bold } /* Generic.Strong */
+.highlight .gu { color: #800080; font-weight: bold } /* Generic.Subheading */
+.highlight .gt { color: #0044DD } /* Generic.Traceback */
+.highlight .kc { color: #008000; font-weight: bold } /* Keyword.Constant */
+.highlight .kd { color: #008000; font-weight: bold } /* Keyword.Declaration */
+.highlight .kn { color: #008000; font-weight: bold } /* Keyword.Namespace */
+.highlight .kp { color: #008000 } /* Keyword.Pseudo */
+.highlight .kr { color: #008000; font-weight: bold } /* Keyword.Reserved */
+.highlight .kt { color: #B00040 } /* Keyword.Type */
+.highlight .m { color: #666666 } /* Literal.Number */
+.highlight .s { color: #BA2121 } /* Literal.String */
+.highlight .na { color: #7D9029 } /* Name.Attribute */
+.highlight .nb { color: #008000 } /* Name.Builtin */
+.highlight .nc { color: #0000FF; font-weight: bold } /* Name.Class */
+.highlight .no { color: #880000 } /* Name.Constant */
+.highlight .nd { color: #AA22FF } /* Name.Decorator */
+.highlight .ni { color: #999999; font-weight: bold } /* Name.Entity */
+.highlight .ne { color: #D2413A; font-weight: bold } /* Name.Exception */
+.highlight .nf { color: #0000FF } /* Name.Function */
+.highlight .nl { color: #A0A000 } /* Name.Label */
+.highlight .nn { color: #0000FF; font-weight: bold } /* Name.Namespace */
+.highlight .nt { color: #008000; font-weight: bold } /* Name.Tag */
+.highlight .nv { color: #19177C } /* Name.Variable */
+.highlight .ow { color: #AA22FF; font-weight: bold } /* Operator.Word */
+.highlight .w { color: #bbbbbb } /* Text.Whitespace */
+.highlight .mb { color: #666666 } /* Literal.Number.Bin */
+.highlight .mf { color: #666666 } /* Literal.Number.Float */
+.highlight .mh { color: #666666 } /* Literal.Number.Hex */
+.highlight .mi { color: #666666 } /* Literal.Number.Integer */
+.highlight .mo { color: #666666 } /* Literal.Number.Oct */
+.highlight .sb { color: #BA2121 } /* Literal.String.Backtick */
+.highlight .sc { color: #BA2121 } /* Literal.String.Char */
+.highlight .sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */
+.highlight .s2 { color: #BA2121 } /* Literal.String.Double */
+.highlight .se { color: #BB6622; font-weight: bold } /* Literal.String.Escape */
+.highlight .sh { color: #BA2121 } /* Literal.String.Heredoc */
+.highlight .si { color: #BB6688; font-weight: bold } /* Literal.String.Interpol */
+.highlight .sx { color: #008000 } /* Literal.String.Other */
+.highlight .sr { color: #BB6688 } /* Literal.String.Regex */
+.highlight .s1 { color: #BA2121 } /* Literal.String.Single */
+.highlight .ss { color: #19177C } /* Literal.String.Symbol */
+.highlight .bp { color: #008000 } /* Name.Builtin.Pseudo */
+.highlight .vc { color: #19177C } /* Name.Variable.Class */
+.highlight .vg { color: #19177C } /* Name.Variable.Global */
+.highlight .vi { color: #19177C } /* Name.Variable.Instance */
+.highlight .il { color: #666666 } /* Literal.Number.Integer.Long */
+    </style>
+<style type="text/css">
+    
+/* Temporary definitions which will become obsolete with Notebook release 5.0 */
+.ansi-black-fg { color: #3E424D; }
+.ansi-black-bg { background-color: #3E424D; }
+.ansi-black-intense-fg { color: #282C36; }
+.ansi-black-intense-bg { background-color: #282C36; }
+.ansi-red-fg { color: #E75C58; }
+.ansi-red-bg { background-color: #E75C58; }
+.ansi-red-intense-fg { color: #B22B31; }
+.ansi-red-intense-bg { background-color: #B22B31; }
+.ansi-green-fg { color: #00A250; }
+.ansi-green-bg { background-color: #00A250; }
+.ansi-green-intense-fg { color: #007427; }
+.ansi-green-intense-bg { background-color: #007427; }
+.ansi-yellow-fg { color: #DDB62B; }
+.ansi-yellow-bg { background-color: #DDB62B; }
+.ansi-yellow-intense-fg { color: #B27D12; }
+.ansi-yellow-intense-bg { background-color: #B27D12; }
+.ansi-blue-fg { color: #208FFB; }
+.ansi-blue-bg { background-color: #208FFB; }
+.ansi-blue-intense-fg { color: #0065CA; }
+.ansi-blue-intense-bg { background-color: #0065CA; }
+.ansi-magenta-fg { color: #D160C4; }
+.ansi-magenta-bg { background-color: #D160C4; }
+.ansi-magenta-intense-fg { color: #A03196; }
+.ansi-magenta-intense-bg { background-color: #A03196; }
+.ansi-cyan-fg { color: #60C6C8; }
+.ansi-cyan-bg { background-color: #60C6C8; }
+.ansi-cyan-intense-fg { color: #258F8F; }
+.ansi-cyan-intense-bg { background-color: #258F8F; }
+.ansi-white-fg { color: #C5C1B4; }
+.ansi-white-bg { background-color: #C5C1B4; }
+.ansi-white-intense-fg { color: #A1A6B2; }
+.ansi-white-intense-bg { background-color: #A1A6B2; }
+
+.ansi-bold { font-weight: bold; }
+
+    </style>
+
+
+<style type="text/css">
+/* Overrides of notebook CSS for static HTML export */
+body {
+  overflow: visible;
+  padding: 8px;
+}
+
+div#notebook {
+  overflow: visible;
+  border-top: none;
+}
+
+@media print {
+  div.cell {
+    display: block;
+    page-break-inside: avoid;
+  } 
+  div.output_wrapper { 
+    display: block;
+    page-break-inside: avoid; 
+  }
+  div.output { 
+    display: block;
+    page-break-inside: avoid; 
+  }
+}
+</style>
+
+<!-- Custom stylesheet, it must be in the same directory as the html file -->
+<link rel="stylesheet" href="custom.css">
+
+<!-- Loading mathjax macro -->
+<!-- Load mathjax -->
+    <script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS_HTML"></script>
+    <!-- MathJax configuration -->
+    <script type="text/x-mathjax-config">
+    MathJax.Hub.Config({
+        tex2jax: {
+            inlineMath: [ ['$','$'], ["\\(","\\)"] ],
+            displayMath: [ ['$$','$$'], ["\\[","\\]"] ],
+            processEscapes: true,
+            processEnvironments: true
+        },
+        // Center justify equations in code and markdown cells. Elsewhere
+        // we use CSS to left justify single line equations in code cells.
+        displayAlign: 'center',
+        "HTML-CSS": {
+            styles: {'.MathJax_Display': {"margin": 0}},
+            linebreaks: { automatic: true }
+        }
+    });
+    </script>
+    <!-- End of mathjax configuration --></head>
+<body>
+  <div tabindex="-1" id="notebook" class="border-box-sizing">
+    <div class="container" id="notebook-container">
+
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<h1 id="Artificial-Intelligence-Engineer-Nanodegree---Probabilistic-Models">Artificial Intelligence Engineer Nanodegree - Probabilistic Models<a class="anchor-link" href="#Artificial-Intelligence-Engineer-Nanodegree---Probabilistic-Models">&#182;</a></h1><h2 id="Project:-Sign-Language-Recognition-System">Project: Sign Language Recognition System<a class="anchor-link" href="#Project:-Sign-Language-Recognition-System">&#182;</a></h2><ul>
+<li><a href="#intro">Introduction</a></li>
+<li><a href="#part1_tutorial">Part 1 Feature Selection</a><ul>
+<li><a href="#part1_tutorial">Tutorial</a></li>
+<li><a href="#part1_submission">Features Submission</a></li>
+<li><a href="#part1_test">Features Unittest</a></li>
+</ul>
+</li>
+<li><a href="#part2_tutorial">Part 2 Train the models</a><ul>
+<li><a href="#part2_tutorial">Tutorial</a></li>
+<li><a href="#part2_submission">Model Selection Score Submission</a></li>
+<li><a href="#part2_test">Model Score Unittest</a></li>
+</ul>
+</li>
+<li><a href="#part3_tutorial">Part 3 Build a Recognizer</a><ul>
+<li><a href="#part3_tutorial">Tutorial</a></li>
+<li><a href="#part3_submission">Recognizer Submission</a></li>
+<li><a href="#part3_test">Recognizer Unittest</a></li>
+</ul>
+</li>
+<li><a href="#part4_info">Part 4 (OPTIONAL) Improve the WER with Language Models</a></li>
+</ul>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p><a id='intro'></a></p>
+<h2 id="Introduction">Introduction<a class="anchor-link" href="#Introduction">&#182;</a></h2><p>The overall goal of this project is to build a word recognizer for American Sign Language video sequences, demonstrating the power of probabalistic models.  In particular, this project employs  <a href="https://en.wikipedia.org/wiki/Hidden_Markov_model">hidden Markov models (HMM's)</a> to analyze a series of measurements taken from videos of American Sign Language (ASL) collected for research (see the <a href="http://www-i6.informatik.rwth-aachen.de/~dreuw/database-rwth-boston-104.php">RWTH-BOSTON-104 Database</a>).  In this video, the right-hand x and y locations are plotted as the speaker signs the sentence.
+<a href="https://drive.google.com/open?id=0B_5qGuFe-wbhUXRuVnNZVnMtam8"><img src="http://www-i6.informatik.rwth-aachen.de/~dreuw/images/demosample.png" alt="ASLR demo"></a></p>
+<p>The raw data, train, and test sets are pre-defined.  You will derive a variety of feature sets (explored in Part 1), as well as implement three different model selection criterion to determine the optimal number of hidden states for each word model (explored in Part 2). Finally, in Part 3 you will implement the recognizer and compare the effects the different combinations of feature sets and model selection criteria.</p>
+<p>At the end of each Part, complete the submission cells with implementations, answer all questions, and pass the unit tests.  Then submit the completed notebook for review!</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p><a id='part1_tutorial'></a></p>
+<h2 id="PART-1:-Data">PART 1: Data<a class="anchor-link" href="#PART-1:-Data">&#182;</a></h2><h3 id="Features-Tutorial">Features Tutorial<a class="anchor-link" href="#Features-Tutorial">&#182;</a></h3><h5 id="Load-the-initial-database">Load the initial database<a class="anchor-link" href="#Load-the-initial-database">&#182;</a></h5><p>A data handler designed for this database is provided in the student codebase as the <code>AslDb</code> class in the <code>asl_data</code> module.  This handler creates the initial <a href="http://pandas.pydata.org/pandas-docs/stable/">pandas</a> dataframe from the corpus of data included in the <code>data</code> directory as well as dictionaries suitable for extracting data in a format friendly to the <a href="https://hmmlearn.readthedocs.io/en/latest/">hmmlearn</a> library.  We'll use those to create models in Part 2.</p>
+<p>To start, let's set up the initial database and select an example set of features for the training set.  At the end of Part 1, you will create additional feature sets for experimentation.</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[1]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
+<span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
+<span class="kn">from</span> <span class="nn">asl_data</span> <span class="k">import</span> <span class="n">AslDb</span>
+
+
+<span class="n">asl</span> <span class="o">=</span> <span class="n">AslDb</span><span class="p">()</span> <span class="c1"># initializes the database</span>
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="o">.</span><span class="n">head</span><span class="p">()</span> <span class="c1"># displays the first five rows of the asl database, indexed by video and frame</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt output_prompt">Out[1]:</div>
+
+<div class="output_html rendered_html output_subarea output_execute_result">
+<div>
+<table border="1" class="dataframe">
+  <thead>
+    <tr style="text-align: right;">
+      <th></th>
+      <th></th>
+      <th>left-x</th>
+      <th>left-y</th>
+      <th>right-x</th>
+      <th>right-y</th>
+      <th>nose-x</th>
+      <th>nose-y</th>
+      <th>speaker</th>
+    </tr>
+    <tr>
+      <th>video</th>
+      <th>frame</th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <th rowspan="5" valign="top">98</th>
+      <th>0</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+    </tr>
+    <tr>
+      <th>1</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+    </tr>
+    <tr>
+      <th>2</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+    </tr>
+    <tr>
+      <th>3</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+    </tr>
+    <tr>
+      <th>4</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+    </tr>
+  </tbody>
+</table>
+</div>
+</div>
+
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[2]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="o">.</span><span class="n">ix</span><span class="p">[</span><span class="mi">98</span><span class="p">,</span><span class="mi">1</span><span class="p">]</span>  <span class="c1"># look at the data available for an individual frame</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt output_prompt">Out[2]:</div>
+
+
+<div class="output_text output_subarea output_execute_result">
+<pre>left-x         149
+left-y         181
+right-x        170
+right-y        175
+nose-x         161
+nose-y          62
+speaker    woman-1
+Name: (98, 1), dtype: object</pre>
+</div>
+
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p>The frame represented by video 98, frame 1 is shown here:</p>
+<p><img src="http://www-i6.informatik.rwth-aachen.de/~dreuw/database/rwth-boston-104/overview/images/orig/098-start.jpg" alt="Video 98"></p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<h5 id="Feature-selection-for-training-the-model">Feature selection for training the model<a class="anchor-link" href="#Feature-selection-for-training-the-model">&#182;</a></h5><p>The objective of feature selection when training a model is to choose the most relevant variables while keeping the model as simple as possible, thus reducing training time.  We can use the raw features already provided or derive our own and add columns to the pandas dataframe <code>asl.df</code> for selection. As an example, in the next cell a feature named <code>'grnd-ry'</code> is added. This feature is the difference between the right-hand y value and the nose y value, which serves as the "ground" right y value.</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[3]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;grnd-ry&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;right-y&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;nose-y&#39;</span><span class="p">]</span>
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="o">.</span><span class="n">head</span><span class="p">()</span>  <span class="c1"># the new feature &#39;grnd-ry&#39; is now in the frames dictionary</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt output_prompt">Out[3]:</div>
+
+<div class="output_html rendered_html output_subarea output_execute_result">
+<div>
+<table border="1" class="dataframe">
+  <thead>
+    <tr style="text-align: right;">
+      <th></th>
+      <th></th>
+      <th>left-x</th>
+      <th>left-y</th>
+      <th>right-x</th>
+      <th>right-y</th>
+      <th>nose-x</th>
+      <th>nose-y</th>
+      <th>speaker</th>
+      <th>grnd-ry</th>
+    </tr>
+    <tr>
+      <th>video</th>
+      <th>frame</th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <th rowspan="5" valign="top">98</th>
+      <th>0</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+      <td>113</td>
+    </tr>
+    <tr>
+      <th>1</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+      <td>113</td>
+    </tr>
+    <tr>
+      <th>2</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+      <td>113</td>
+    </tr>
+    <tr>
+      <th>3</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+      <td>113</td>
+    </tr>
+    <tr>
+      <th>4</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+      <td>113</td>
+    </tr>
+  </tbody>
+</table>
+</div>
+</div>
+
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<h5 id="Try-it!">Try it!<a class="anchor-link" href="#Try-it!">&#182;</a></h5>
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[4]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">asl_utils</span> <span class="k">import</span> <span class="n">test_features_tryit</span>
+<span class="c1"># TODO add df columns for &#39;grnd-rx&#39;, &#39;grnd-ly&#39;, &#39;grnd-lx&#39; representing differences between hand and nose locations</span>
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;grnd-rx&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;right-x&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;nose-x&#39;</span><span class="p">]</span>
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;grnd-ly&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;left-y&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;nose-y&#39;</span><span class="p">]</span>
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;grnd-lx&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;left-x&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;nose-x&#39;</span><span class="p">]</span>
+
+<span class="c1"># test the code</span>
+<span class="n">test_features_tryit</span><span class="p">(</span><span class="n">asl</span><span class="p">)</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+<div class="output_subarea output_stream output_stdout output_text">
+<pre>asl.df sample
+</pre>
+</div>
+</div>
+
+<div class="output_area"><div class="prompt"></div>
+
+<div class="output_html rendered_html output_subarea ">
+<div>
+<table border="1" class="dataframe">
+  <thead>
+    <tr style="text-align: right;">
+      <th></th>
+      <th></th>
+      <th>left-x</th>
+      <th>left-y</th>
+      <th>right-x</th>
+      <th>right-y</th>
+      <th>nose-x</th>
+      <th>nose-y</th>
+      <th>speaker</th>
+      <th>grnd-ry</th>
+      <th>grnd-rx</th>
+      <th>grnd-ly</th>
+      <th>grnd-lx</th>
+    </tr>
+    <tr>
+      <th>video</th>
+      <th>frame</th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <th rowspan="5" valign="top">98</th>
+      <th>0</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+      <td>113</td>
+      <td>9</td>
+      <td>119</td>
+      <td>-12</td>
+    </tr>
+    <tr>
+      <th>1</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+      <td>113</td>
+      <td>9</td>
+      <td>119</td>
+      <td>-12</td>
+    </tr>
+    <tr>
+      <th>2</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+      <td>113</td>
+      <td>9</td>
+      <td>119</td>
+      <td>-12</td>
+    </tr>
+    <tr>
+      <th>3</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+      <td>113</td>
+      <td>9</td>
+      <td>119</td>
+      <td>-12</td>
+    </tr>
+    <tr>
+      <th>4</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+      <td>113</td>
+      <td>9</td>
+      <td>119</td>
+      <td>-12</td>
+    </tr>
+  </tbody>
+</table>
+</div>
+</div>
+
+</div>
+
+<div class="output_area"><div class="prompt output_prompt">Out[4]:</div>
+
+<div class="output_html rendered_html output_subarea output_execute_result">
+<font color=green>Correct!</font><br/>
+</div>
+
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[5]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># collect the features into a list</span>
+<span class="n">features_ground</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;grnd-rx&#39;</span><span class="p">,</span><span class="s1">&#39;grnd-ry&#39;</span><span class="p">,</span><span class="s1">&#39;grnd-lx&#39;</span><span class="p">,</span><span class="s1">&#39;grnd-ly&#39;</span><span class="p">]</span>
+ <span class="c1">#show a single set of features for a given (video, frame) tuple</span>
+<span class="p">[</span><span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="o">.</span><span class="n">ix</span><span class="p">[</span><span class="mi">98</span><span class="p">,</span><span class="mi">1</span><span class="p">][</span><span class="n">v</span><span class="p">]</span> <span class="k">for</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">features_ground</span><span class="p">]</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt output_prompt">Out[5]:</div>
+
+
+<div class="output_text output_subarea output_execute_result">
+<pre>[9, 113, -12, 119]</pre>
+</div>
+
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<h5 id="Build-the-training-set">Build the training set<a class="anchor-link" href="#Build-the-training-set">&#182;</a></h5><p>Now that we have a feature list defined, we can pass that list to the <code>build_training</code> method to collect the features for all the words in the training set.  Each word in the training set has multiple examples from various videos.  Below we can see the unique words that have been loaded into the training set:</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[6]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="n">training</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">build_training</span><span class="p">(</span><span class="n">features_ground</span><span class="p">)</span>
+<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Training words: </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">training</span><span class="o">.</span><span class="n">words</span><span class="p">))</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+<div class="output_subarea output_stream output_stdout output_text">
+<pre>Training words: [&#39;JOHN&#39;, &#39;WRITE&#39;, &#39;HOMEWORK&#39;, &#39;IX-1P&#39;, &#39;SEE&#39;, &#39;YESTERDAY&#39;, &#39;IX&#39;, &#39;LOVE&#39;, &#39;MARY&#39;, &#39;CAN&#39;, &#39;GO&#39;, &#39;GO1&#39;, &#39;FUTURE&#39;, &#39;GO2&#39;, &#39;PARTY&#39;, &#39;FUTURE1&#39;, &#39;HIT&#39;, &#39;BLAME&#39;, &#39;FRED&#39;, &#39;FISH&#39;, &#39;WONT&#39;, &#39;EAT&#39;, &#39;BUT&#39;, &#39;CHICKEN&#39;, &#39;VEGETABLE&#39;, &#39;CHINA&#39;, &#39;PEOPLE&#39;, &#39;PREFER&#39;, &#39;BROCCOLI&#39;, &#39;LIKE&#39;, &#39;LEAVE&#39;, &#39;SAY&#39;, &#39;BUY&#39;, &#39;HOUSE&#39;, &#39;KNOW&#39;, &#39;CORN&#39;, &#39;CORN1&#39;, &#39;THINK&#39;, &#39;NOT&#39;, &#39;PAST&#39;, &#39;LIVE&#39;, &#39;CHICAGO&#39;, &#39;CAR&#39;, &#39;SHOULD&#39;, &#39;DECIDE&#39;, &#39;VISIT&#39;, &#39;MOVIE&#39;, &#39;WANT&#39;, &#39;SELL&#39;, &#39;TOMORROW&#39;, &#39;NEXT-WEEK&#39;, &#39;NEW-YORK&#39;, &#39;LAST-WEEK&#39;, &#39;WILL&#39;, &#39;FINISH&#39;, &#39;ANN&#39;, &#39;READ&#39;, &#39;BOOK&#39;, &#39;CHOCOLATE&#39;, &#39;FIND&#39;, &#39;SOMETHING-ONE&#39;, &#39;POSS&#39;, &#39;BROTHER&#39;, &#39;ARRIVE&#39;, &#39;HERE&#39;, &#39;GIVE&#39;, &#39;MAN&#39;, &#39;NEW&#39;, &#39;COAT&#39;, &#39;WOMAN&#39;, &#39;GIVE1&#39;, &#39;HAVE&#39;, &#39;FRANK&#39;, &#39;BREAK-DOWN&#39;, &#39;SEARCH-FOR&#39;, &#39;WHO&#39;, &#39;WHAT&#39;, &#39;LEG&#39;, &#39;FRIEND&#39;, &#39;CANDY&#39;, &#39;BLUE&#39;, &#39;SUE&#39;, &#39;BUY1&#39;, &#39;STOLEN&#39;, &#39;OLD&#39;, &#39;STUDENT&#39;, &#39;VIDEOTAPE&#39;, &#39;BORROW&#39;, &#39;MOTHER&#39;, &#39;POTATO&#39;, &#39;TELL&#39;, &#39;BILL&#39;, &#39;THROW&#39;, &#39;APPLE&#39;, &#39;NAME&#39;, &#39;SHOOT&#39;, &#39;SAY-1P&#39;, &#39;SELF&#39;, &#39;GROUP&#39;, &#39;JANA&#39;, &#39;TOY1&#39;, &#39;MANY&#39;, &#39;TOY&#39;, &#39;ALL&#39;, &#39;BOY&#39;, &#39;TEACHER&#39;, &#39;GIRL&#39;, &#39;BOX&#39;, &#39;GIVE2&#39;, &#39;GIVE3&#39;, &#39;GET&#39;, &#39;PUTASIDE&#39;]
+</pre>
+</div>
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p>The training data in <code>training</code> is an object of class <code>WordsData</code> defined in the <code>asl_data</code> module.  in addition to the <code>words</code> list, data can be accessed with the <code>get_all_sequences</code>, <code>get_all_Xlengths</code>, <code>get_word_sequences</code>, and <code>get_word_Xlengths</code> methods. We need the <code>get_word_Xlengths</code> method to train multiple sequences with the <code>hmmlearn</code> library.  In the following example, notice that there are two lists; the first is a concatenation of all the sequences(the X portion) and the second is a list of the sequence lengths (the Lengths portion).</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[7]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="n">training</span><span class="o">.</span><span class="n">get_word_Xlengths</span><span class="p">(</span><span class="s1">&#39;CHOCOLATE&#39;</span><span class="p">)</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt output_prompt">Out[7]:</div>
+
+
+<div class="output_text output_subarea output_execute_result">
+<pre>(array([[-11,  48,   7, 120],
+        [-11,  48,   8, 109],
+        [ -8,  49,  11,  98],
+        [ -7,  50,   7,  87],
+        [ -4,  54,   7,  77],
+        [ -4,  54,   6,  69],
+        [ -4,  54,   6,  69],
+        [-13,  52,   6,  69],
+        [-13,  52,   6,  69],
+        [ -8,  51,   6,  69],
+        [ -8,  51,   6,  69],
+        [ -8,  51,   6,  69],
+        [ -8,  51,   6,  69],
+        [ -8,  51,   6,  69],
+        [-10,  59,   7,  71],
+        [-15,  64,   9,  77],
+        [-17,  75,  13,  81],
+        [ -4,  48,  -4, 113],
+        [ -2,  53,  -4, 113],
+        [ -4,  55,   2,  98],
+        [ -4,  58,   2,  98],
+        [ -1,  59,   2,  89],
+        [ -1,  59,  -1,  84],
+        [ -1,  59,  -1,  84],
+        [ -7,  63,  -1,  84],
+        [ -7,  63,  -1,  84],
+        [ -7,  63,   3,  83],
+        [ -7,  63,   3,  83],
+        [ -7,  63,   3,  83],
+        [ -7,  63,   3,  83],
+        [ -7,  63,   3,  83],
+        [ -7,  63,   3,  83],
+        [ -7,  63,   3,  83],
+        [ -4,  70,   3,  83],
+        [ -4,  70,   3,  83],
+        [ -2,  73,   5,  90],
+        [ -3,  79,  -4,  96],
+        [-15,  98,  13, 135],
+        [ -6,  93,  12, 128],
+        [ -2,  89,  14, 118],
+        [  5,  90,  10, 108],
+        [  4,  86,   7, 105],
+        [  4,  86,   7, 105],
+        [  4,  86,  13, 100],
+        [ -3,  82,  14,  96],
+        [ -3,  82,  14,  96],
+        [  6,  89,  16, 100],
+        [  6,  89,  16, 100],
+        [  7,  85,  17, 111]], dtype=int64), [17, 20, 12])</pre>
+</div>
+
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<h6 id="More-feature-sets">More feature sets<a class="anchor-link" href="#More-feature-sets">&#182;</a></h6><p>So far we have a simple feature set that is enough to get started modeling.  However, we might get better results if we manipulate the raw values a bit more, so we will go ahead and set up some other options now for experimentation later.  For example, we could normalize each speaker's range of motion with grouped statistics using <a href="http://pandas.pydata.org/pandas-docs/stable/api.html#api-dataframe-stats">Pandas stats</a> functions and <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.groupby.html">pandas groupby</a>.  Below is an example for finding the means of all speaker subgroups.</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[8]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="n">df_means</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="o">.</span><span class="n">groupby</span><span class="p">(</span><span class="s1">&#39;speaker&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">mean</span><span class="p">()</span>
+<span class="n">df_means</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt output_prompt">Out[8]:</div>
+
+<div class="output_html rendered_html output_subarea output_execute_result">
+<div>
+<table border="1" class="dataframe">
+  <thead>
+    <tr style="text-align: right;">
+      <th></th>
+      <th>left-x</th>
+      <th>left-y</th>
+      <th>right-x</th>
+      <th>right-y</th>
+      <th>nose-x</th>
+      <th>nose-y</th>
+      <th>grnd-ry</th>
+      <th>grnd-rx</th>
+      <th>grnd-ly</th>
+      <th>grnd-lx</th>
+    </tr>
+    <tr>
+      <th>speaker</th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <th>man-1</th>
+      <td>206.248203</td>
+      <td>218.679449</td>
+      <td>155.464350</td>
+      <td>150.371031</td>
+      <td>175.031756</td>
+      <td>61.642600</td>
+      <td>88.728430</td>
+      <td>-19.567406</td>
+      <td>157.036848</td>
+      <td>31.216447</td>
+    </tr>
+    <tr>
+      <th>woman-1</th>
+      <td>164.661438</td>
+      <td>161.271242</td>
+      <td>151.017865</td>
+      <td>117.332462</td>
+      <td>162.655120</td>
+      <td>57.245098</td>
+      <td>60.087364</td>
+      <td>-11.637255</td>
+      <td>104.026144</td>
+      <td>2.006318</td>
+    </tr>
+    <tr>
+      <th>woman-2</th>
+      <td>183.214509</td>
+      <td>176.527232</td>
+      <td>156.866295</td>
+      <td>119.835714</td>
+      <td>170.318973</td>
+      <td>58.022098</td>
+      <td>61.813616</td>
+      <td>-13.452679</td>
+      <td>118.505134</td>
+      <td>12.895536</td>
+    </tr>
+  </tbody>
+</table>
+</div>
+</div>
+
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p>To select a mean that matches by speaker, use the pandas <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.map.html">map</a> method:</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[9]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;left-x-mean&#39;</span><span class="p">]</span><span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;speaker&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">df_means</span><span class="p">[</span><span class="s1">&#39;left-x&#39;</span><span class="p">])</span>
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="o">.</span><span class="n">head</span><span class="p">()</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt output_prompt">Out[9]:</div>
+
+<div class="output_html rendered_html output_subarea output_execute_result">
+<div>
+<table border="1" class="dataframe">
+  <thead>
+    <tr style="text-align: right;">
+      <th></th>
+      <th></th>
+      <th>left-x</th>
+      <th>left-y</th>
+      <th>right-x</th>
+      <th>right-y</th>
+      <th>nose-x</th>
+      <th>nose-y</th>
+      <th>speaker</th>
+      <th>grnd-ry</th>
+      <th>grnd-rx</th>
+      <th>grnd-ly</th>
+      <th>grnd-lx</th>
+      <th>left-x-mean</th>
+    </tr>
+    <tr>
+      <th>video</th>
+      <th>frame</th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <th rowspan="5" valign="top">98</th>
+      <th>0</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+      <td>113</td>
+      <td>9</td>
+      <td>119</td>
+      <td>-12</td>
+      <td>164.661438</td>
+    </tr>
+    <tr>
+      <th>1</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+      <td>113</td>
+      <td>9</td>
+      <td>119</td>
+      <td>-12</td>
+      <td>164.661438</td>
+    </tr>
+    <tr>
+      <th>2</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+      <td>113</td>
+      <td>9</td>
+      <td>119</td>
+      <td>-12</td>
+      <td>164.661438</td>
+    </tr>
+    <tr>
+      <th>3</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+      <td>113</td>
+      <td>9</td>
+      <td>119</td>
+      <td>-12</td>
+      <td>164.661438</td>
+    </tr>
+    <tr>
+      <th>4</th>
+      <td>149</td>
+      <td>181</td>
+      <td>170</td>
+      <td>175</td>
+      <td>161</td>
+      <td>62</td>
+      <td>woman-1</td>
+      <td>113</td>
+      <td>9</td>
+      <td>119</td>
+      <td>-12</td>
+      <td>164.661438</td>
+    </tr>
+  </tbody>
+</table>
+</div>
+</div>
+
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<h5 id="Try-it!">Try it!<a class="anchor-link" href="#Try-it!">&#182;</a></h5>
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[10]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">asl_utils</span> <span class="k">import</span> <span class="n">test_std_tryit</span>
+<span class="c1"># TODO Create a dataframe named `df_std` with standard deviations grouped by speaker</span>
+<span class="n">df_std</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="o">.</span><span class="n">groupby</span><span class="p">(</span><span class="s1">&#39;speaker&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">std</span><span class="p">()</span>
+
+<span class="c1"># test the code</span>
+<span class="n">test_std_tryit</span><span class="p">(</span><span class="n">df_std</span><span class="p">)</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+<div class="output_subarea output_stream output_stdout output_text">
+<pre>df_std
+</pre>
+</div>
+</div>
+
+<div class="output_area"><div class="prompt"></div>
+
+<div class="output_html rendered_html output_subarea ">
+<div>
+<table border="1" class="dataframe">
+  <thead>
+    <tr style="text-align: right;">
+      <th></th>
+      <th>left-x</th>
+      <th>left-y</th>
+      <th>right-x</th>
+      <th>right-y</th>
+      <th>nose-x</th>
+      <th>nose-y</th>
+      <th>grnd-ry</th>
+      <th>grnd-rx</th>
+      <th>grnd-ly</th>
+      <th>grnd-lx</th>
+      <th>left-x-mean</th>
+    </tr>
+    <tr>
+      <th>speaker</th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+      <th></th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <th>man-1</th>
+      <td>15.154425</td>
+      <td>36.328485</td>
+      <td>18.901917</td>
+      <td>54.902340</td>
+      <td>6.654573</td>
+      <td>5.520045</td>
+      <td>53.487999</td>
+      <td>20.269032</td>
+      <td>36.572749</td>
+      <td>15.080360</td>
+      <td>0.0</td>
+    </tr>
+    <tr>
+      <th>woman-1</th>
+      <td>17.573442</td>
+      <td>26.594521</td>
+      <td>16.459943</td>
+      <td>34.667787</td>
+      <td>3.549392</td>
+      <td>3.538330</td>
+      <td>33.972660</td>
+      <td>16.764706</td>
+      <td>27.117393</td>
+      <td>17.328941</td>
+      <td>0.0</td>
+    </tr>
+    <tr>
+      <th>woman-2</th>
+      <td>15.388711</td>
+      <td>28.825025</td>
+      <td>14.890288</td>
+      <td>39.649111</td>
+      <td>4.099760</td>
+      <td>3.416167</td>
+      <td>39.128572</td>
+      <td>16.191324</td>
+      <td>29.320655</td>
+      <td>15.050938</td>
+      <td>0.0</td>
+    </tr>
+  </tbody>
+</table>
+</div>
+</div>
+
+</div>
+
+<div class="output_area"><div class="prompt output_prompt">Out[10]:</div>
+
+<div class="output_html rendered_html output_subarea output_execute_result">
+<font color=green>Correct!</font><br/>
+</div>
+
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p><a id='part1_submission'></a></p>
+<h3 id="Features-Implementation-Submission">Features Implementation Submission<a class="anchor-link" href="#Features-Implementation-Submission">&#182;</a></h3><p>Implement four feature sets and answer the question that follows.</p>
+<ul>
+<li><p>normalized Cartesian coordinates</p>
+<ul>
+<li>use <em>mean</em> and <em>standard deviation</em> statistics and the <a href="https://en.wikipedia.org/wiki/Standard_score">standard score</a> equation to account for speakers with different heights and arm length</li>
+</ul>
+</li>
+<li><p>polar coordinates</p>
+<ul>
+<li>calculate polar coordinates with <a href="https://en.wikipedia.org/wiki/Polar_coordinate_system#Converting_between_polar_and_Cartesian_coordinates">Cartesian to polar equations</a></li>
+<li>use the <a href="https://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.arctan2.html">np.arctan2</a> function and <em>swap the x and y axes</em> to move the $0$ to $2\pi$ discontinuity to 12 o'clock instead of 3 o'clock;  in other words, the normal break in radians value from $0$ to $2\pi$ occurs directly to the left of the speaker's nose, which may be in the signing area and interfere with results.  By swapping the x and y axes, that discontinuity move to directly above the speaker's head, an area not generally used in signing.</li>
+</ul>
+</li>
+<li><p>delta difference</p>
+<ul>
+<li>as described in Thad's lecture, use the difference in values between one frame and the next frames as features</li>
+<li>pandas <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.diff.html">diff method</a> and <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.fillna.html">fillna method</a> will be helpful for this one</li>
+</ul>
+</li>
+<li><p>custom features</p>
+<ul>
+<li>These are your own design; combine techniques used above or come up with something else entirely. We look forward to seeing what you come up with! 
+Some ideas to get you started:<ul>
+<li>normalize using a <a href="https://en.wikipedia.org/wiki/Feature_scaling">feature scaling equation</a></li>
+<li>normalize the polar coordinates</li>
+<li>adding additional deltas</li>
+</ul>
+</li>
+</ul>
+</li>
+</ul>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[11]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># TODO add features for normalized by speaker values of left, right, x, y</span>
+<span class="c1"># Name these &#39;norm-rx&#39;, &#39;norm-ry&#39;, &#39;norm-lx&#39;, and &#39;norm-ly&#39;</span>
+<span class="c1"># using Z-score scaling (X-Xmean)/Xstd</span>
+
+<span class="n">features_norm</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;norm-rx&#39;</span><span class="p">,</span> <span class="s1">&#39;norm-ry&#39;</span><span class="p">,</span> <span class="s1">&#39;norm-lx&#39;</span><span class="p">,</span><span class="s1">&#39;norm-ly&#39;</span><span class="p">]</span>
+
+<span class="n">mean_right_x</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;speaker&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">df_means</span><span class="p">[</span><span class="s1">&#39;right-x&#39;</span><span class="p">])</span>
+<span class="n">mean_right_y</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;speaker&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">df_means</span><span class="p">[</span><span class="s1">&#39;right-y&#39;</span><span class="p">])</span>
+<span class="n">mean_left_x</span>  <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;speaker&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">df_means</span><span class="p">[</span><span class="s1">&#39;left-x&#39;</span><span class="p">])</span>
+<span class="n">mean_left_y</span>  <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;speaker&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">df_means</span><span class="p">[</span><span class="s1">&#39;left-y&#39;</span><span class="p">])</span>
+
+<span class="n">std_right_x</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;speaker&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">df_std</span><span class="p">[</span><span class="s1">&#39;right-x&#39;</span><span class="p">])</span>
+<span class="n">std_right_y</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;speaker&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">df_std</span><span class="p">[</span><span class="s1">&#39;right-y&#39;</span><span class="p">])</span>
+<span class="n">std_left_x</span>  <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;speaker&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">df_std</span><span class="p">[</span><span class="s1">&#39;left-x&#39;</span><span class="p">])</span>
+<span class="n">std_left_y</span>  <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;speaker&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">df_std</span><span class="p">[</span><span class="s1">&#39;left-y&#39;</span><span class="p">])</span>
+
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;norm-rx&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;right-x&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="n">mean_right_x</span><span class="p">)</span> <span class="o">/</span> <span class="n">std_right_x</span>
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;norm-ry&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;right-y&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="n">mean_right_y</span><span class="p">)</span> <span class="o">/</span> <span class="n">std_right_y</span>
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;norm-lx&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;left-x&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="n">mean_left_x</span><span class="p">)</span> <span class="o">/</span> <span class="n">std_left_x</span>
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;norm-ly&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;left-y&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="n">mean_left_y</span><span class="p">)</span> <span class="o">/</span> <span class="n">std_left_y</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[12]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># TODO add features for polar coordinate values where the nose is the origin</span>
+<span class="c1"># Name these &#39;polar-rr&#39;, &#39;polar-rtheta&#39;, &#39;polar-lr&#39;, and &#39;polar-ltheta&#39;</span>
+<span class="c1"># Note that &#39;polar-rr&#39; and &#39;polar-rtheta&#39; refer to the radius and angle</span>
+
+<span class="n">features_polar</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;polar-rr&#39;</span><span class="p">,</span> <span class="s1">&#39;polar-rtheta&#39;</span><span class="p">,</span> <span class="s1">&#39;polar-lr&#39;</span><span class="p">,</span> <span class="s1">&#39;polar-ltheta&#39;</span><span class="p">]</span>
+
+<span class="n">grnd_rx</span> <span class="o">=</span>  <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;grnd-rx&#39;</span><span class="p">]</span>
+<span class="n">grnd_ry</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;grnd-ry&#39;</span><span class="p">]</span>
+<span class="n">grnd_lx</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;grnd-lx&#39;</span><span class="p">]</span>
+<span class="n">grnd_ly</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;grnd-ly&#39;</span><span class="p">]</span>
+
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;polar-rr&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">sqrt</span><span class="p">(</span><span class="n">grnd_rx</span><span class="o">*</span><span class="n">grnd_rx</span> <span class="o">+</span> <span class="n">grnd_ry</span><span class="o">*</span><span class="n">grnd_ry</span><span class="p">)</span>
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;polar-rtheta&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arctan2</span><span class="p">(</span><span class="n">grnd_rx</span><span class="p">,</span> <span class="n">grnd_ry</span><span class="p">)</span>
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;polar-lr&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">sqrt</span><span class="p">(</span><span class="n">grnd_lx</span><span class="o">*</span><span class="n">grnd_lx</span> <span class="o">+</span> <span class="n">grnd_ly</span><span class="o">*</span><span class="n">grnd_ly</span><span class="p">)</span>
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;polar-ltheta&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arctan2</span><span class="p">(</span><span class="n">grnd_lx</span><span class="p">,</span> <span class="n">grnd_ly</span><span class="p">)</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[13]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># TODO add features for left, right, x, y differences by one time step, i.e. the &quot;delta&quot; values discussed in the lecture</span>
+<span class="c1"># Name these &#39;delta-rx&#39;, &#39;delta-ry&#39;, &#39;delta-lx&#39;, and &#39;delta-ly&#39;</span>
+
+<span class="n">features_delta</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;delta-rx&#39;</span><span class="p">,</span> <span class="s1">&#39;delta-ry&#39;</span><span class="p">,</span> <span class="s1">&#39;delta-lx&#39;</span><span class="p">,</span> <span class="s1">&#39;delta-ly&#39;</span><span class="p">]</span>
+
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;delta-rx&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;right-x&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">diff</span><span class="p">()</span><span class="o">.</span><span class="n">fillna</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;delta-ry&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;right-y&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">diff</span><span class="p">()</span><span class="o">.</span><span class="n">fillna</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;delta-lx&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;left-x&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">diff</span><span class="p">()</span><span class="o">.</span><span class="n">fillna</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;delta-ly&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;left-y&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">diff</span><span class="p">()</span><span class="o">.</span><span class="n">fillna</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[14]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># TODO add features of your own design, which may be a combination of the above or something else</span>
+<span class="c1"># Name these whatever you would like</span>
+
+<span class="c1"># TODO define a list named &#39;features_custom&#39; for building the training set</span>
+
+<span class="c1"># Need to calculate mean and std again to include &#39;polar-rr&#39; and &#39;polar-lr&#39;</span>
+<span class="n">df_means</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="o">.</span><span class="n">groupby</span><span class="p">(</span><span class="s1">&#39;speaker&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">mean</span><span class="p">()</span>
+<span class="n">df_std</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="o">.</span><span class="n">groupby</span><span class="p">(</span><span class="s1">&#39;speaker&#39;</span><span class="p">)</span><span class="o">.</span><span class="n">std</span><span class="p">()</span>
+
+<span class="c1"># Normalize right arm radius</span>
+<span class="n">mean_polar_rr</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;speaker&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">df_means</span><span class="p">[</span><span class="s1">&#39;polar-rr&#39;</span><span class="p">])</span>
+<span class="n">std_polar_rr</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;speaker&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">df_std</span><span class="p">[</span><span class="s1">&#39;polar-rr&#39;</span><span class="p">])</span>
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;norm-polar-rr&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;polar-rr&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="n">mean_polar_rr</span><span class="p">)</span> <span class="o">/</span> <span class="n">std_polar_rr</span>
+
+<span class="c1"># Normalize left arm radius</span>
+<span class="n">mean_polar_lr</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;speaker&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">df_means</span><span class="p">[</span><span class="s1">&#39;polar-lr&#39;</span><span class="p">])</span>
+<span class="n">std_polar_lr</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;speaker&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">df_std</span><span class="p">[</span><span class="s1">&#39;polar-lr&#39;</span><span class="p">])</span>
+<span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;norm-polar-lr&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span><span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;polar-lr&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="n">mean_polar_lr</span><span class="p">)</span> <span class="o">/</span> <span class="n">std_polar_lr</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p><strong>Question 1:</strong>  What custom features did you choose for the features_custom set and why?</p>
+<p><strong>Answer 1:</strong> Added new features - <em>norm-polar-lr</em> &amp; <em>norm-polar-rr</em> which normalize the polar radius to account for differences in arm length.</p>
+<p>Also, as noted from the lectures, delta between values in consecutive time steps gives us an indication of speed. Hence, delta of normalized data might be a good metric.</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p><a id='part1_test'></a></p>
+<h3 id="Features-Unit-Testing">Features Unit Testing<a class="anchor-link" href="#Features-Unit-Testing">&#182;</a></h3><p>Run the following unit tests as a sanity check on the defined "ground", "norm", "polar", and 'delta"
+feature sets.  The test simply looks for some valid values but is not exhaustive.  However, the project should not be submitted if these tests don't pass.</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[15]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">unittest</span>
+<span class="c1"># import numpy as np</span>
+
+<span class="k">class</span> <span class="nc">TestFeatures</span><span class="p">(</span><span class="n">unittest</span><span class="o">.</span><span class="n">TestCase</span><span class="p">):</span>
+
+    <span class="k">def</span> <span class="nf">test_features_ground</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+        <span class="n">sample</span> <span class="o">=</span> <span class="p">(</span><span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="o">.</span><span class="n">ix</span><span class="p">[</span><span class="mi">98</span><span class="p">,</span> <span class="mi">1</span><span class="p">][</span><span class="n">features_ground</span><span class="p">])</span><span class="o">.</span><span class="n">tolist</span><span class="p">()</span>
+        <span class="bp">self</span><span class="o">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="n">sample</span><span class="p">,</span> <span class="p">[</span><span class="mi">9</span><span class="p">,</span> <span class="mi">113</span><span class="p">,</span> <span class="o">-</span><span class="mi">12</span><span class="p">,</span> <span class="mi">119</span><span class="p">])</span>
+
+    <span class="k">def</span> <span class="nf">test_features_norm</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+        <span class="n">sample</span> <span class="o">=</span> <span class="p">(</span><span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="o">.</span><span class="n">ix</span><span class="p">[</span><span class="mi">98</span><span class="p">,</span> <span class="mi">1</span><span class="p">][</span><span class="n">features_norm</span><span class="p">])</span><span class="o">.</span><span class="n">tolist</span><span class="p">()</span>
+        <span class="n">np</span><span class="o">.</span><span class="n">testing</span><span class="o">.</span><span class="n">assert_almost_equal</span><span class="p">(</span><span class="n">sample</span><span class="p">,</span> <span class="p">[</span> <span class="mf">1.153</span><span class="p">,</span>  <span class="mf">1.663</span><span class="p">,</span> <span class="o">-</span><span class="mf">0.891</span><span class="p">,</span>  <span class="mf">0.742</span><span class="p">],</span> <span class="mi">3</span><span class="p">)</span>
+
+    <span class="k">def</span> <span class="nf">test_features_polar</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+        <span class="n">sample</span> <span class="o">=</span> <span class="p">(</span><span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="o">.</span><span class="n">ix</span><span class="p">[</span><span class="mi">98</span><span class="p">,</span><span class="mi">1</span><span class="p">][</span><span class="n">features_polar</span><span class="p">])</span><span class="o">.</span><span class="n">tolist</span><span class="p">()</span>
+        <span class="n">np</span><span class="o">.</span><span class="n">testing</span><span class="o">.</span><span class="n">assert_almost_equal</span><span class="p">(</span><span class="n">sample</span><span class="p">,</span> <span class="p">[</span><span class="mf">113.3578</span><span class="p">,</span> <span class="mf">0.0794</span><span class="p">,</span> <span class="mf">119.603</span><span class="p">,</span> <span class="o">-</span><span class="mf">0.1005</span><span class="p">],</span> <span class="mi">3</span><span class="p">)</span>
+
+    <span class="k">def</span> <span class="nf">test_features_delta</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
+        <span class="n">sample</span> <span class="o">=</span> <span class="p">(</span><span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="o">.</span><span class="n">ix</span><span class="p">[</span><span class="mi">98</span><span class="p">,</span> <span class="mi">0</span><span class="p">][</span><span class="n">features_delta</span><span class="p">])</span><span class="o">.</span><span class="n">tolist</span><span class="p">()</span>
+        <span class="bp">self</span><span class="o">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="n">sample</span><span class="p">,</span> <span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">])</span>
+        <span class="n">sample</span> <span class="o">=</span> <span class="p">(</span><span class="n">asl</span><span class="o">.</span><span class="n">df</span><span class="o">.</span><span class="n">ix</span><span class="p">[</span><span class="mi">98</span><span class="p">,</span> <span class="mi">18</span><span class="p">][</span><span class="n">features_delta</span><span class="p">])</span><span class="o">.</span><span class="n">tolist</span><span class="p">()</span>
+        <span class="bp">self</span><span class="o">.</span><span class="n">assertTrue</span><span class="p">(</span><span class="n">sample</span> <span class="ow">in</span> <span class="p">[[</span><span class="o">-</span><span class="mi">16</span><span class="p">,</span> <span class="o">-</span><span class="mi">5</span><span class="p">,</span> <span class="o">-</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">],</span> <span class="p">[</span><span class="o">-</span><span class="mi">14</span><span class="p">,</span> <span class="o">-</span><span class="mi">9</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">]],</span> <span class="s2">&quot;Sample value found was </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">sample</span><span class="p">))</span>
+                         
+<span class="n">suite</span> <span class="o">=</span> <span class="n">unittest</span><span class="o">.</span><span class="n">TestLoader</span><span class="p">()</span><span class="o">.</span><span class="n">loadTestsFromModule</span><span class="p">(</span><span class="n">TestFeatures</span><span class="p">())</span>
+<span class="n">unittest</span><span class="o">.</span><span class="n">TextTestRunner</span><span class="p">()</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">suite</span><span class="p">)</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+<div class="output_subarea output_stream output_stderr output_text">
+<pre>....
+----------------------------------------------------------------------
+Ran 4 tests in 0.018s
+
+OK
+</pre>
+</div>
+</div>
+
+<div class="output_area"><div class="prompt output_prompt">Out[15]:</div>
+
+
+<div class="output_text output_subarea output_execute_result">
+<pre>&lt;unittest.runner.TextTestResult run=4 errors=0 failures=0&gt;</pre>
+</div>
+
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p><a id='part2_tutorial'></a></p>
+<h2 id="PART-2:-Model-Selection">PART 2: Model Selection<a class="anchor-link" href="#PART-2:-Model-Selection">&#182;</a></h2><h3 id="Model-Selection-Tutorial">Model Selection Tutorial<a class="anchor-link" href="#Model-Selection-Tutorial">&#182;</a></h3><p>The objective of Model Selection is to tune the number of states for each word HMM prior to testing on unseen data.  In this section you will explore three methods:</p>
+<ul>
+<li>Log likelihood using cross-validation folds (CV)</li>
+<li>Bayesian Information Criterion (BIC)</li>
+<li>Discriminative Information Criterion (DIC) </li>
+</ul>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<h5 id="Train-a-single-word">Train a single word<a class="anchor-link" href="#Train-a-single-word">&#182;</a></h5><p>Now that we have built a training set with sequence data, we can "train" models for each word.  As a simple starting example, we train a single word using Gaussian hidden Markov models (HMM).   By using the <code>fit</code> method during training, the <a href="https://en.wikipedia.org/wiki/Baum%E2%80%93Welch_algorithm">Baum-Welch Expectation-Maximization</a> (EM) algorithm is invoked iteratively to find the best estimate for the model <em>for the number of hidden states specified</em> from a group of sample seequences. For this example, we <em>assume</em> the correct number of hidden states is 3, but that is just a guess.  How do we know what the "best" number of states for training is?  We will need to find some model selection technique to choose the best parameter.</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[16]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">warnings</span>
+<span class="kn">from</span> <span class="nn">hmmlearn.hmm</span> <span class="k">import</span> <span class="n">GaussianHMM</span>
+
+<span class="k">def</span> <span class="nf">train_a_word</span><span class="p">(</span><span class="n">word</span><span class="p">,</span> <span class="n">num_hidden_states</span><span class="p">,</span> <span class="n">features</span><span class="p">):</span>
+    
+    <span class="n">warnings</span><span class="o">.</span><span class="n">filterwarnings</span><span class="p">(</span><span class="s2">&quot;ignore&quot;</span><span class="p">,</span> <span class="n">category</span><span class="o">=</span><span class="ne">DeprecationWarning</span><span class="p">)</span>
+    <span class="n">training</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">build_training</span><span class="p">(</span><span class="n">features</span><span class="p">)</span>  
+    <span class="n">X</span><span class="p">,</span> <span class="n">lengths</span> <span class="o">=</span> <span class="n">training</span><span class="o">.</span><span class="n">get_word_Xlengths</span><span class="p">(</span><span class="n">word</span><span class="p">)</span>
+    <span class="n">model</span> <span class="o">=</span> <span class="n">GaussianHMM</span><span class="p">(</span><span class="n">n_components</span><span class="o">=</span><span class="n">num_hidden_states</span><span class="p">,</span> <span class="n">n_iter</span><span class="o">=</span><span class="mi">1000</span><span class="p">)</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">lengths</span><span class="p">)</span>
+    <span class="n">logL</span> <span class="o">=</span> <span class="n">model</span><span class="o">.</span><span class="n">score</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">lengths</span><span class="p">)</span>
+    <span class="k">return</span> <span class="n">model</span><span class="p">,</span> <span class="n">logL</span>
+
+<span class="n">demoword</span> <span class="o">=</span> <span class="s1">&#39;BOOK&#39;</span>
+<span class="n">model</span><span class="p">,</span> <span class="n">logL</span> <span class="o">=</span> <span class="n">train_a_word</span><span class="p">(</span><span class="n">demoword</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="n">features_ground</span><span class="p">)</span>
+<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Number of states trained in model for </span><span class="si">{}</span><span class="s2"> is </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">demoword</span><span class="p">,</span> <span class="n">model</span><span class="o">.</span><span class="n">n_components</span><span class="p">))</span>
+<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;logL = </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">logL</span><span class="p">))</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+<div class="output_subarea output_stream output_stdout output_text">
+<pre>Number of states trained in model for BOOK is 3
+logL = -2331.1138127433196
+</pre>
+</div>
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p>The HMM model has been trained and information can be pulled from the model, including means and variances for each feature and hidden state.  The <a href="http://math.stackexchange.com/questions/892832/why-we-consider-log-likelihood-instead-of-likelihood-in-gaussian-distribution">log likelihood</a> for any individual sample or group of samples can also be calculated with the <code>score</code> method.</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[17]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">show_model_stats</span><span class="p">(</span><span class="n">word</span><span class="p">,</span> <span class="n">model</span><span class="p">):</span>
+    <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Number of states trained in model for </span><span class="si">{}</span><span class="s2"> is </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">word</span><span class="p">,</span> <span class="n">model</span><span class="o">.</span><span class="n">n_components</span><span class="p">))</span>    
+    <span class="n">variance</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="n">np</span><span class="o">.</span><span class="n">diag</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">covars_</span><span class="p">[</span><span class="n">i</span><span class="p">])</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">n_components</span><span class="p">)])</span>    
+    <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">n_components</span><span class="p">):</span>  <span class="c1"># for each hidden state</span>
+        <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;hidden state #</span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">i</span><span class="p">))</span>
+        <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;mean = &quot;</span><span class="p">,</span> <span class="n">model</span><span class="o">.</span><span class="n">means_</span><span class="p">[</span><span class="n">i</span><span class="p">])</span>
+        <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;variance = &quot;</span><span class="p">,</span> <span class="n">variance</span><span class="p">[</span><span class="n">i</span><span class="p">])</span>
+        <span class="nb">print</span><span class="p">()</span>
+    
+<span class="n">show_model_stats</span><span class="p">(</span><span class="n">demoword</span><span class="p">,</span> <span class="n">model</span><span class="p">)</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+<div class="output_subarea output_stream output_stdout output_text">
+<pre>Number of states trained in model for BOOK is 3
+hidden state #0
+mean =  [ -1.12415027  69.44164191  17.02866283  77.7231196 ]
+variance =  [ 19.70434594  16.83041492  30.51552305  11.03678246]
+
+hidden state #1
+mean =  [ -11.45300909   94.109178     19.03512475  102.2030162 ]
+variance =  [  77.403668    203.35441965   26.68898447  156.12444034]
+
+hidden state #2
+mean =  [ -3.46504869  50.66686933  14.02391587  52.04731066]
+variance =  [ 49.12346305  43.04799144  39.35109609  47.24195772]
+
+</pre>
+</div>
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<h5 id="Try-it!">Try it!<a class="anchor-link" href="#Try-it!">&#182;</a></h5><p>Experiment by changing the feature set, word, and/or num_hidden_states values in the next cell to see changes in values.</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[18]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="n">my_testword</span> <span class="o">=</span> <span class="s1">&#39;CHOCOLATE&#39;</span>
+<span class="n">model</span><span class="p">,</span> <span class="n">logL</span> <span class="o">=</span> <span class="n">train_a_word</span><span class="p">(</span><span class="n">my_testword</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="n">features_ground</span><span class="p">)</span> <span class="c1"># Experiment here with different parameters</span>
+<span class="n">show_model_stats</span><span class="p">(</span><span class="n">my_testword</span><span class="p">,</span> <span class="n">model</span><span class="p">)</span>
+<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;logL = </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">logL</span><span class="p">))</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+<div class="output_subarea output_stream output_stdout output_text">
+<pre>Number of states trained in model for CHOCOLATE is 3
+hidden state #0
+mean =  [   0.58333333   87.91666667   12.75        108.5       ]
+variance =  [  39.41055556   18.74388889    9.855       144.4175    ]
+
+hidden state #1
+mean =  [ -9.30211403  55.32333876   6.92259936  71.24057775]
+variance =  [ 16.16920957  46.50917372   3.81388185  15.79446427]
+
+hidden state #2
+mean =  [ -5.40587658  60.1652424    2.32479599  91.3095432 ]
+variance =  [   7.95073876   64.13103127   13.68077479  129.5912395 ]
+
+logL = -601.3291470028619
+</pre>
+</div>
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<h5 id="Visualize-the-hidden-states">Visualize the hidden states<a class="anchor-link" href="#Visualize-the-hidden-states">&#182;</a></h5><p>We can plot the means and variances for each state and feature.  Try varying the number of states trained for the HMM model and examine the variances.  Are there some models that are "better" than others?  How can you tell?  We would like to hear what you think in the classroom online.</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[19]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="o">%</span><span class="k">matplotlib</span> inline
+</pre></div>
+
+</div>
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[20]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">math</span>
+<span class="kn">from</span> <span class="nn">matplotlib</span> <span class="k">import</span> <span class="p">(</span><span class="n">cm</span><span class="p">,</span> <span class="n">pyplot</span> <span class="k">as</span> <span class="n">plt</span><span class="p">,</span> <span class="n">mlab</span><span class="p">)</span>
+
+<span class="k">def</span> <span class="nf">visualize</span><span class="p">(</span><span class="n">word</span><span class="p">,</span> <span class="n">model</span><span class="p">):</span>
+    <span class="sd">&quot;&quot;&quot; visualize the input model for a particular word &quot;&quot;&quot;</span>
+    <span class="n">variance</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="n">np</span><span class="o">.</span><span class="n">diag</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">covars_</span><span class="p">[</span><span class="n">i</span><span class="p">])</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">n_components</span><span class="p">)])</span>
+    <span class="n">figures</span> <span class="o">=</span> <span class="p">[]</span>
+    <span class="k">for</span> <span class="n">parm_idx</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">means_</span><span class="p">[</span><span class="mi">0</span><span class="p">])):</span>
+        <span class="n">xmin</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="nb">min</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">means_</span><span class="p">[:,</span><span class="n">parm_idx</span><span class="p">])</span> <span class="o">-</span> <span class="nb">max</span><span class="p">(</span><span class="n">variance</span><span class="p">[:,</span><span class="n">parm_idx</span><span class="p">]))</span>
+        <span class="n">xmax</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="nb">max</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">means_</span><span class="p">[:,</span><span class="n">parm_idx</span><span class="p">])</span> <span class="o">+</span> <span class="nb">max</span><span class="p">(</span><span class="n">variance</span><span class="p">[:,</span><span class="n">parm_idx</span><span class="p">]))</span>
+        <span class="n">fig</span><span class="p">,</span> <span class="n">axs</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplots</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">n_components</span><span class="p">,</span> <span class="n">sharex</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">sharey</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
+        <span class="n">colours</span> <span class="o">=</span> <span class="n">cm</span><span class="o">.</span><span class="n">rainbow</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">linspace</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">model</span><span class="o">.</span><span class="n">n_components</span><span class="p">))</span>
+        <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="p">(</span><span class="n">ax</span><span class="p">,</span> <span class="n">colour</span><span class="p">)</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="nb">zip</span><span class="p">(</span><span class="n">axs</span><span class="p">,</span> <span class="n">colours</span><span class="p">)):</span>
+            <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">linspace</span><span class="p">(</span><span class="n">xmin</span><span class="p">,</span> <span class="n">xmax</span><span class="p">,</span> <span class="mi">100</span><span class="p">)</span>
+            <span class="n">mu</span> <span class="o">=</span> <span class="n">model</span><span class="o">.</span><span class="n">means_</span><span class="p">[</span><span class="n">i</span><span class="p">,</span><span class="n">parm_idx</span><span class="p">]</span>
+            <span class="n">sigma</span> <span class="o">=</span> <span class="n">math</span><span class="o">.</span><span class="n">sqrt</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">diag</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">covars_</span><span class="p">[</span><span class="n">i</span><span class="p">])[</span><span class="n">parm_idx</span><span class="p">])</span>
+            <span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">mlab</span><span class="o">.</span><span class="n">normpdf</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">mu</span><span class="p">,</span> <span class="n">sigma</span><span class="p">),</span> <span class="n">c</span><span class="o">=</span><span class="n">colour</span><span class="p">)</span>
+            <span class="n">ax</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s2">&quot;</span><span class="si">{}</span><span class="s2"> feature </span><span class="si">{}</span><span class="s2"> hidden state #</span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">word</span><span class="p">,</span> <span class="n">parm_idx</span><span class="p">,</span> <span class="n">i</span><span class="p">))</span>
+
+            <span class="n">ax</span><span class="o">.</span><span class="n">grid</span><span class="p">(</span><span class="kc">True</span><span class="p">)</span>
+        <span class="n">figures</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">plt</span><span class="p">)</span>
+    <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">figures</span><span class="p">:</span>
+        <span class="n">p</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
+        
+<span class="n">visualize</span><span class="p">(</span><span class="n">my_testword</span><span class="p">,</span> <span class="n">model</span><span class="p">)</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+
+
+<div class="output_png output_subarea ">
+<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAYQAAAEICAYAAABfz4NwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz
+AAALEgAACxIB0t1+/AAAIABJREFUeJzsnXl8lcXV+L8nC1vYF8NOUFDEBdnFpWClFtxwAVxwt1J8
+q7UudfnpW9DW1tpqlWpV7FsVd9wVqShqRBSURRQBkS1IgCA7hCWQ5Pz+mIlcLndLcm/uTXK++Tyf
+PHfmzMyZuc+dM9szI6qKYRiGYaQlWwHDMAwjNTCDYBiGYQBmEAzDMAyPGQTDMAwDMINgGIZheMwg
+GIZhGIAZBKOaISLZIjJdRHaIyAPJ1ieRiMg4EXkugv9CERkUxm+QiORHCPu0iPwpDmoaNQgzCElE
+RC4WkTkiUigi60TkvyJykvcLWRmIiIpIl4DP3UXkbRHZ5ivJj0XkhKAwdXx8S0Vkp4jkich/RCQn
+QOZMEfnS+28SkedFpH2A/xUiMiNKfp4WkWIRaRPg9rjPX6GI7BWRfQGf/ysiOT5PhUHXBWGSGQ1s
+BBqr6s0RCzgKyagUfX4/FpFdIvKdiAyuaFyqepSq5sZRvYQS/OzGIJ8rIr+KU9qTROQ0EakrIgVB
+fnX972G7iBSIyE3xSLM6YgYhSfiH7iHgz0A20BF4FDi7HHEcBnwGLAA6A22BN4D3RWRAgOirPt6L
+gSZAD2AOcKqPZzjwgtenJXAUUATMEJFmMeqSBZwPbAMuKXNX1TGq2lBVG/q8vlz2WVWHBkTRNMC9
+oaq+HCapTsAiTYE3KkUkowLBXgS+AloAdwKvikiruCpmhKI37pk/Fvg2yG8c0BX3bJ0C3CoiQ6pU
+u1RBVe2q4gtXKRcCIyLIjAOeC+GuQBd//ywwJYTMY8B0fz8Y2A10CJOOAKuAW4Pc03A/nHv85yuA
+GRH0vQxYDdwAfBtrnoAcn6eMGMrtaWAfsNeX32Cv5+3AcmATMAloHhDmFaAAZ6imA0d599FBcb0T
+XL4Baf7J3w8C8oHbfJzPevczgfnAVuBz4Ngw+h+OM7SNAtymA2MilNckYCKwA1gI9AnwzwMG+/v6
+XtctwCLg90B+gGxPYJ6P52XgpbJ8RcuDT+cW4Btfji8D9cLo3AX4xMttxDUAyvKpwE5f3hcAzYDJ
+wAav92SgvZe/FygB9nj5R7x7N+ADYDOwBBgZw3PTDFjh768F7g/yXwucFvD5HuClZNUPybySrkBt
+vIAhQDERKkFiMwgFwJUhZE7xP6b6wH3AJxHS6ebj7BzC725gpr+/gsgG4UPgflxvpxjoHUueKIdB
+8PJPB1VkNwCzgPZAXeAJ4MUA/6uARt7vIWB+uLiCyzdYBmcQioG/+vjq4yraH4H+QDpwOa4CrRtC
+93OBxUFu/wT+GeEZ2AOc7uP+CzArwD+P/QbhPuBToDnQAWfM871fHZzRvxHIBIbjjGFZviLmwd9/
+ieuBNgcWE96IvYjr+aQB9YCTIpRtC1yvsoH/jl4B3gzwzwV+FfA5C9fouBLI8HpvBLqH0eVUnIEr
+xBn+rf7/Tn8/EGcsFMgOCHc+sCBZ9UMyLxsySg4tgI2qWhxFbqSIbA28gvxbAutChFuH+0E292mF
+kgmMgzAy6wL8wyIiHXFG6AVVXY8zDpdFCxfExqC8HhljuDHAnaqar6pFuEp0eNlwjqr+R1V3BPj1
+EJEm5dQtkFJgrKoWqepuXE/jCVX9QlVLVPUZXC/g+BBhG+JazoFsx1WG4ZihqlNUtQTXI+wRRm4k
+cK+qblbV1cD4AL/jcYbgIVXdp6qvArMD/GPJw3hVXauqm4F3gOPC6LEPN/TSVlX3qGrYeSdV3aSq
+r6nqLlXdgesVDAwnj+vF5KnqU6parKpfAa8BI8LE/6GqNgXe9DLtcMatpao2VdVPcN8JHPi9RPtO
+aixmEJLDJqBlDGPQk/yD+9MV5L8RaBMiXBtcxbXFpxVKJjAOwsi0CfCPxKW4lu98//l54GIRyYwh
+bBktg/K6OMZwnYA3AgzmYlzvKFtE0kXkPhFZLiLbcZUBxGDkIrBBVfcEpX9zkNHugGtNB1MINA5y
+a4IbxglH4AToLqBemOemLa71XMaqIL816pu/IfxjyUOwHg0Jza24Ycgv/Sqoq8LIISINROQJEVnl
+v5/pQFMRSQ8TpBPQP0jPUUDrMPHne5mLgGdwvaBOwDoRedCLFfr/gd9LtO+kxmIGITnMxLXAzqlk
+PNMI3ToaiRvq2eVl+gWuGApiCW5c/IB4RCQN13X+MAY9LgMO9Ss0CoAHcZXu6THlonKsBoYGGZN6
+qroGN4k+DDfX0AQ3PAWuwgI3VBDMLtwQRhnBlU1wmNW4lnlg+g1U9cUQcS/ElVNg67OHd68s63CV
+eBkdg/zaiYiE8S9PHiKiqgWqeo2qtgV+Dfwrwsqim4EjgP6q2hj4mXcP9/2sxg1/BurZUFWvDaNL
+e9zw7DTfmJoA/MaHu8nLbMGVT2DPK17fSbXDDEISUNVtwB+AR0XkHN9SyhSRoSJyfzmiuhs4QUTu
+FZHmItJIRK7HVdC3+bSm4Sbh3hCR3iKS4eXGiMhVvtV4C3CXuGWw9USkNfBvXKvpHwHpifcPvAYA
+hwH9cMMIxwFH41YtlXfYqCI8DtwrIp28gq1EZJj3a4QzvJtwlfyfg8KuBw4NcpuP692k+5UmkYYw
+AJ4ExohIf3FkicgZQZU+AKr6vY9/rC+784BjcMMelWUScIeINPPG//oAv5m4uY/f+ufsPNz3Ve48
+RENERgQ0PrbgKvVS/zm4vBvhFjxsFZHmwNig6ILlJwOHi8ilPh+ZItI3yvBib9xkOkAv3EqjYCbi
+nv9mPq5rcHNHtQ4zCElCVR8AbgLuwq2yWA1chxvvjDWOpcBJuBZNHq6lcz7wS1X9LEB0ODAFtzpk
+G27CsQ+u94C6JZ6X4iYdN+FWqdQHTlTVTQHxnID7AQdeVwNvqeoC3zosUNUC4GHgTP9Dj4WtcuB7
+CLGuBX8YeBu31HYHboK5v/ebiBsaWePzNCso7P8B3f3wQ1m53wCchZt0HEWU70NV5+AqkEdwFeAy
+3AR8OC7Elf0W3CTxcFXdEDWX0bkbl9eVwPu4+YYyHfcC53m9NuNW+LxeiTxEoi/whYgU4r6XG1R1
+hfcbBzzjy3skbpK/Pm5YchbwXlBcD+Pmg7aIyHg/z3AargzX4oaxyib4w9EbmOd7R90I3fIfi1ul
+tgo3kX2/qgbrUiuQA4cVDcMwjNqK9RAMwzAMwAyCYRiG4TGDYBiGYQBmEAzDMAxPRTbnShotW7bU
+nJycZKsREzt37iQrKyvZaqQcVi6hsXIJjZVLaMpbLnPnzt2oqlE3UaxWBiEnJ4c5c0ItI049cnNz
+GTRoULLVSDmsXEJj5RIaK5fQlLdcRGRVdKlqZhAMoyaydydsmtmCz76E+s2hQStomgPZx8IB7xYb
+RoIxg2AYSeL7yfDlI5CXCyVFxxy0SX/bPnD8jdB9BKSXZ1cow6ggZhAMo4opLID3boCFk6BpZ+j7
+P7Cr3dcMvboHRdth5wZY8wV88TC8Pgqm3Q7nPgs50TbRMIxKYquMDKMKWfQqPHokfPcW/PxeuG4J
+/PJBaNZ7C/WaQpOO0La3MxK/WQwXTYbM+jDxVJj5INjGAkYiMYNgGFXEvP+DV0ZCiyNgzNdw8v+L
+PBQkaXD4GXDNbDjibHj/Znj1Ati3u+p0NmoXZhAMowr44p/wzq+gyy/h8o+g5RGxh63bGEa+Bqfe
+53oYr14AJfsSp6tRezGDYBgJZuaD8N5vodu5cMGbkNkgephgROCk2+D0R+H7d+Dtq0BLo4czjPJg
+k8qGkUC+e8sN9Rw1Es57HtIq+Yvrey3s2QIf3Ql1m8LQ8bY01YgfZhAMI0GsX+BWCbXrB+c8U3lj
+UMZJd8DuzTDzAWh2KAy4MT7xGoYNGRlGAti1EV46243/X/AGZNSLX9wi8Iu/QbdzYNqtsPrz+MVt
+1G5iMggiMkRElojIMhG5PYS/iMh47/+NiPQK8MsTkQUiMl9E5gS4NxeRD0Rkqf/fLD5ZMozkoqXw
+6oWwYx1c+CY0ahs9THkRgWFPuWWqr17gDJBhVJaoBkFE0oFHgaFAd+AiEekeJDYU6Oqv0cBjQf6n
+qOpxqtonwO124ENV7Yo7yP0gQ2MY1ZHPH4CVH8Lpj7jhokRRrymMeMW9yPb6KCgtSVxaRu0glh5C
+P2CZqq7wZ7O+BAwLkhkGTFTHLKCpiLSJEu8w4Bl//wxwTjn0NoyUZN1XbsK327nQ8+rEp9eml5tY
+Xv4+fP73xKdn1GxiMQjtcAfAl5Hv3WKVUWCaiMwVkdEBMtmqus7fFwDZMWttGCnIvl2upd6gJZz1
+ZNWt/ul1DRx5HuT+AX4M3hDJMMpBVawyOklV14jIIcAHIvKdqk4PFFBVFZGQL+V7IzIaIDs7m9zc
+3IQrHA8KCwurja5VSU0ul6Xju7BxcXuO+dvXfLlgS7nCVrZcml6SSdqHfXn2vCJ6/mseaRk1Y4+L
+mvy8VIZElUssBmEN0CHgc3vvFpOMqpb9/1FE3sANQU0H1otIG1Vd54eXfgyVuKpOACYA9OnTR6vL
+3ui2j3toamq55H0Cn7wB/W+AIbf0KHf4eJRL+1J4ZXgd0mcOZOD/ViqqlKGmPi+VJVHlEsuQ0Wyg
+q4h0FpE6wIXA20EybwOX+dVGxwPbfEWfJSKNAEQkCzgNftrl923gcn9/OfBWJfNiGElh3263LUWz
+Q92Gdcmi+/lw9EUw/R4omJ88PYzqS1SDoKrFwHXAVGAxMElVF4rIGBEZ48WmACuAZcCTwP9492xg
+hoh8DXwJvKuq73m/+4BfiMhSYLD/bBjVjtxxsHmZmzeok+TTHk9/BOq3gLd/BaXFydXFqH7ENIeg
+qlNwlX6g2+MB9wr8JkS4FUDI/rOqbgJOLY+yhpFqrJ0LM/8OPX8FnX+ebG3ciWtDx7t3E74YDwNu
+SrZGRnXC3lQ2jApSsg/evhqysuG0vyVbm/10HwGHnwkf/y9szUu2NkZ1wgyCYVSQLx6G9V+7HUjr
+NU22NvsRcTpJGrx7rR2qY8SOGQTDqABb8yB3LBwxDI48N9naHEyTjm6Ce9l78O2LydbGqC6YQTCM
+cqIKU34DCAz9Z7K1CU/f30DbvjD1RthdvtcijFqKGQTDKCeLX4OlU+CUP0KTDtHlk0VaOpw1AXZt
+gmm2U5gRA2YQDKMc7NkG//0ttO4J/a9PtjbRaX2ce1lu3gTbJtuIjhkEwygHH94BO9e7lne8DrxJ
+NKfcDY07wORf21nMRmTMIBhGjKyeCXMeh37XQ9s+0eVThToN3QtrP37rTlkzjHCYQTCMGCjZB5NH
+Q+N2bu6gunHE2W5H1E/udm9VG0YozCAYRgzMfMC1sIc+AnUbJVubijH0n5BeByaPsXcTjNCYQTCM
+KGxe5lrW3c6FbsFHQ1UjGrWFwX91p7l9/Ux0eaP2YQbBMCKgpW57ivS6qf3OQaz0Hg0dT4L3b4ad
+ITecN2ozZhAMIwJznoBV0+G0B9z8QXVH0uDMCbC30C2fNYxAzCAYRhi2roJpt8Khg6HnVcnWJn60
+OhJ+9gdY+DIsejXZ2hipREwGQUSGiMgSEVkmIge98+gPxhnv/b8RkV7evYOIfCwii0RkoYjcEBBm
+nIisEZH5/jo9ftkyjMqh6tbtq1bt+chVxUm3uaWz715rQ0fGfqIaBBFJBx4FhgLdgYtEpHuQ2FCg
+q79GA49592LgZlXtDhwP/CYo7D9U9Th/HXDegmEkk3lPwvKpcOpfoGlOsrWJP2kZcM4zULTddkQ1
+9hNLD6EfsExVV6jqXuAlIHitxTBgojpmAU3LzktW1XkAqroDd+JaDRiJNWoym753G8IdOhj6HXTs
+U82hVXf3TsXi121HVMMRi0FoB6wO+JzPwZV6VBkRyQF6Al8EOF/vh5j+IyLNYtTZMBJGyT54fRRk
+1HMtaKnhs2wDbob2x8O7/2OH6RgxHqFZWUSkIfAa8DtV3e6dHwP+CKj//wBw0NSdiIzGDUORnZ1N
+bm5uVahcaQoLC6uNrlVJqpfLyv/ksHZODt3Hfcvc7zfC91WTbjLLpe319Vg3ug9PnbmTHg/NJy0j
+dcaPUv15SRaJKpdYDMIaIHCT3/beLSYZEcnEGYPnVfX1MgFVXV92LyJPApNDJa6qE4AJAH369NFB
+gwbFoHLyyc3NpbroWpWkcrnk5cL05+G4K2HY2KOrNO1kl0uHdHjtwiboRwMZ9OekqXEQyS6XVCVR
+5RJLh3g20FVEOotIHeBC4O0gmbeBy/xqo+OBbaq6TkQE+D9gsao+GBhARNoEfDwX+LbCuTCMSrI9
+3x1M3+JwGPJwsrWpeo6+AHpdAzPug+UfJFsbI1lENQiqWgxcB0zFTQpPUtWFIjJGRMZ4sSnACmAZ
+8CTwP979ROBS4OchlpfeLyILROQb4BTgxrjlyjDKQcleeGUE7NsFI1+vvnsVVZYhD7l3FF4f5d7B
+MGofMc0h+CWhU4LcHg+4V+Cg9RiqOgMIuYJbVS8tl6aGkSCm3gT5s2DEK65CrK1kNnAG8d/94aWz
+4arP3NbZRu2hhq+hMIzIzJ0Asx+FAbdA9+HJ1ib5tDwCRkxyO7u+canby8moPZhBMGotS95xL2V1
+GQqD/5JsbVKHw05zezd99yZ8/Idka2NUJdXkEEDDiC/5s9wkcpvebqiouhyHWVX0v8H1Ej69Fxq0
+guNviB7GqP7Yz8CodWxYDC+c6c4HuHgy1MlKtkaphwic+Tjs3gxTf+fKqNevkq2VkWhsyMioVRR8
+DU8PdD2CS96DrEOSrVHqkpYB578IXYbAO6NhwQvJ1shINGYQjFpD/hfwzCDIqAtXTofmXZKtUeqT
+URdGvgadfuYmmec8Hj2MUX0xg2DUCpZ/AM8OhvrN4cpP3QtoRmxkNoCL33WT7+9eCx/dZbuj1lTM
+IBg1GlX47G/w/BC3jfUV02vmdtaJpk4WXPime5v503vhzcth3+5ka2XEG5tUNmosewvdecgLJ7l3
+DIY9ZS9aVYa0DDjzCWjSET7+X1g3D4a/BIdU7bZPRgKxHoJRI1kxDR471h0ROfivMHySGYN4IAI/
+uwsumQq7NsCTfWH2Y/YCW03BDIJRo9i1Cd66Ep79BaRnwuW5cOKtNe8IzGRz2Gkw5hvoNBCm/A/8
+5yRYOyfZWhmVxQyCUSPYsxVyx8H4Q+HrZ+GkO2DM19Dp5GRrVnNpmA2jprihuC0r4Ml+8NbVsGVl
+sjUzKorNIRjVmq2rYN6/YfYjzigceR4MutvGtasKSYPjrnDl/skf4YuH4eunofsIOOH30LZ3sjU0
+yoMZBKPasXsLLHsPvn4Glr/v3I44CwaOgzY9k6paraVuYzjtb3D875xRmPsELHwZsnvA0RfB0RdC
+007J1tKIRkwGQUSGAA8D6cC/VfW+IH/x/qcDu4ArVHVepLAi0hx4GcgB8oCRqrql8lkyahp7C934
+9OqZsHwq/DADtAQat4eBf4CeV7mVL0byadwOfnE/nHwnfD0Rvn0BPrzdXYccDZ1PdVf7/vaWeCoS
+1SCISDrwKPALIB+YLSJvq+qiALGhQFd/9cedl9w/StjbgQ9V9T4Rud1/vi1+WTOqEyV7Yfsa2L4a
+tq2GTd/DxsXu2rBo/yqW7GPhpNvh8DOhbV9IS0+u3kZo6jWB/te7a8sKWPgKrJzmeg5f+BPpGrVz
+PboW3dxb480Pg8YdoGFr1+OwhQBVTyw9hH7AMlVdASAiLwHDgECDMAyY6A/KmSUiTf0RmTkRwg4D
+BvnwzwC5JMggrJ0Dm5clIubw/LjoEL4tqNo0YyXsW6ZB7gfIqf/s/2vpgfdaAqUl/n+xu0r2uYq+
+ZC8U73YvMq1Z2Y31D0PRDijaDrs3wa6N7j4QSYNmh0LLbtDtXGg/ANr1gwYt4lcORtXQ7FA46TZ3
+Fe9xO82unQsFX0HBfPcWeUnRgWEy6kODllCS2Ye8Ds5A1MmCzIaQWR8y6kF6Xbe1RloGpGX6/+kg
+6f5/mrsQb1yC/8MBx3cdYIDCuSeRTgOhUZvocpUhFoPQDlgd8Dkf1wuIJtMuSthsVV3n7wuA7FCJ
+i8hoYDRAdnY2ubm5Mah8IEsf6srat9qVO1zl6M7iKk4xVZGMUtLqlpJWpxTJbExhw0LSG5SQ3qCY
+zEOLadlzH5lN9lGnRRF1Dymibqsi6rfZQ1qd/Yvb1wBrFiQvD4mmsLCwQs92taU3NO8NzXENiqKN
+ddmztj5FG+qwd3Nd9m6qw77tmezZClu3FFOyJp2S3emU7HGX7hNK96WBpkhtXQUc89dvaN5vM5C4
+5yUlJpVVVUUkZLtVVScAEwD69OmjgwYNKnf8fbrBnvuiy8WTL7/8kn79+lVtouUhzO/ooNZQcEvJ
+t64CW17iW2M/tc58Sy3dt95E0ihb4Zyb+zkV+Q7LWL9+PSNGjOCrr75i9OjRPPDAAxWOK5XIzc09
+qFzGjRvHsmXLeO6550KGOeqoo3j00UdDlmdubi6XXHIJ+fn5IcNeccUVtG/fnj/96U+VVT2hhCqX
+MlRdj7RkH5Tu8z3Tkv29Vcp6r6UH9m7LesLBPeDAeEO5J5vG7Y/96eXKSOVSKVQ14gUMAKYGfL4D
+uCNI5gngooDPS4A2kcKWyfj7NsCSaLr07t1bqwsff/xxVJnnn39ee/furVlZWdq6dWsdMmSIfvrp
+p6qqOnbsWB01atRBYQBdunTpT58XLlyoZ511ljZu3FgbNmyogwYN0s8+++yAMEVFRTp27Fjt0qWL
+NmjQQDt16qRXXnmlrly58ieZd955R/v27asNGjTQ5s2b68UXX6yrV6/+yf+pp57SE088MWJ+Lr/8
+ck1PT9e1a9f+5PbrX/9as7KyNCsrSzMzMzU9Pf2nz0OGDNGVK1cq8JNb2fXSSy+FTOOee+7Rc889
+V0tLSyPqEguXX3653nnnnZWOpzysXLlSBw0apPXr19cjjjhCP/jgA1UN/byEewZi4eOPP9Z27dqF
+9U9G3oOf3WgMHDhQb7nllrikPWLECJ06daru2bNHs7OzD/B7+eWXdcCAAVq/fn0dOHBgXNJLNLHU
+L4EAczRK/aqqiEbZtlBEMoDvgVNxPffZwMWqujBA5gzgOtwqo/7AeFXtFymsiPwN2KT7J5Wbq+qt
+UXTZAKyKqHDq0BLYGME/G2iNy892XFukMdAIN7TWFqgLBL/m0xv4Fijy/kcCG3DDburTbYcr950+
+TBcg06e1C7fiq7mX3wg0w833rAK2eP/2XpdFQAnQwse9JEx+0oAePs51wPoQMm19Hr8LcKsDHAPM
+DRNvMJ2AfcDaGOUjkQPsjVNcsdINKMT9Hpp4Hb4FmnLw8xLuGYiFRkBn4Jsw/jlUfd4Dn91YOMLL
+5sUh7aOBxbjybI/7fZTRCDdaUg/3fIZ7xlOJaPVLMJ1UtVVUqVisBq6i/x5YDtzp3cYAY/y94FYT
+LQcWAH0ihfXuLYAPgaXANJxBiEmf6nARwSLjKoJCYEQEmXHAcyHcFeji758FpoSQeQyY7u8HA7uB
+DmHSEZwhuDXIPQ33473Hf74CmBFB38tw80U3AN9GyNOmILccn6eMGMr0aZwx2OvLb7DX83b/fG0C
+JgU+S8ArOGO5DZgOHOXdRwfF9U5w+Qak+Sd/PwhnrG/zcT7r3c8E5gNbgc+BY8PofziugmsU4Dbd
+/5YOel58eU0CJgI7gIVBv608YLC/r+913YIz4r8H8gNkewLzfDwvAy+V5StaHnw6t+CMyzYfvl6Y
+PHYBPvFyG4GXA/KpuEZKIXABriEyGdeg2eLv23v5e3ENkVIv/4h37wZ8AGzGVdwjY3humgEr/P21
+wP1h5H4F5FZF/VDZK9TzEpd4k52xmnpF+sKAIUAxESpBYjMIBcCVIWRO8T+m+sB9wCcR0unm4+wc
+wu9uYKa/v4LIBuFD4H5cz6cY6B0mTxU2CF7+6aCK7AZgFq7VVxc3fPligP9VuBZgXeAhYH64uILL
+N1gGZxCKgb/6+OrjKtofcT3jdOByXAVaN4Tu5wKLg9z+6a9wBmEPrlGVDvwFmBXgn8d+g3Af8Cmu
+59cBZ8zzvV8dnNG/EddTHI4zhmX5ipgHf/8lrsfSHNfSHhPm+3kRuBNnqOsBJ0Uo2xbA+UAD/x29
+ArwZ4J8L5AV8zsI1Oq7Eteh74oxO9zC6nIozcIU4w7/V/9/p7wcGydd6g2B7GSWHFsBGVS2OIjdS
+RLYGXkH+LXHDM8Gsw/0gm/u0QskExkEYmXUB/mERkY44I/SCqq7HGYfLooULYmNQXo+MMdwYXM8z
+X1WLcJXocD9ciar+R1V3BPj1EJEm5dQtkFJgrKoWqepuXE/jCVX9QlVLVPUZXC/g+BBhG+JazoFs
+x1WG4ZihqlNUtQTXI+wRRm4kcK+qblbV1cD4AL/jcYbgIVXdp6qv4oZvy4glD+NVda2qbgbeAY4L
+o8c+3LBeW1Xdo6ozwmVMVTep6muquktVd+B6BQPDyeN6MXmq+pSqFqvqV8BrwIgw8X+oqk2BN71M
+O5xxa6mqTVX1kwhp1UrMICSOCRH8NgEtyyqtCEzyD+5PV5D/RtyEfDBtcBXXFp9WpNXLZeOQ4eKJ
+ZZzyUlzLd77//DxwsYhkhpBdGiaOlkF5jXXVbifgjQCDuRjXO8oWkXQRuU9ElovIdvaPRUc1chHY
+oKp7gtK/Ochod8C1poMpxI1RB9IEN4wT7nkJfJtlF1AvzHPTlgOXeK8K8lujvmkZwj+WPATrEW4z
+8Vtxw5BfishCEbkqjBwi0kBEnhCRVf77mQ409S+0ljE9SM/+QXqOws3FhYo/38tchHvX6UcfxzoR
+eTCcXtWESPVLhTGDkCDULZcNx0xcC+ycSiYzjdCto5G4oZ5dXqafiLQPE8cS3Lj4AfGIWyt6Pq61
+H43LgENFpEBECoAHcZXu6SFk4/2K4GpgaJAxqaeqa4CLcS9ADmb/BC7sX0wbakXFLtwQRhnBlU1w
+mNW4lnlg+g1U9cUQcS/ElVNgj6AHsDDK8xIL63CVeBkdg/za+S1mQvmXJw8RUdUCVb1GVdsCvwb+
+JSLhTq/P8jvrAAAgAElEQVS+GTdx3F9VGwM/8+6B30+gQViNG/4M1LOhql4bRpf2uOHZab4xNQH4
+jQ93U3nzlkrE4XkJiRmEJKCq24A/AI+KyDm+pZQpIkNF5P5yRHU3cIKI3CsizUWkkYhcj6ugb/Np
+TcNNwr0hIr1FJMPLjRGRq3yr8RbgLhG5WETqiUhr4N+41uw/AtIT7x94DQAOw73Rfpy/jgZeoPzD
+RhXhceBeEenkFWwlIsO8XyOc4d2Eq+T/HBR2PXBokNt8XO8m3e/DFWkIA+BJYIyI9BdHloicEVTp
+A6Cq3/v4x/qyOw+3wuq1mHMbnknAHSLSzBv/6wP8ZuLmPn7rn7PzcN9XufMQDREZEdD42IKr1Mve
+MAwu70a4BQ9b/d5mY4OiC5afDBwuIpf6fGSKSN8ow4u9cZPpAL2Ag05t8N91Pdy8RJr/bkL1bms8
+ZhCShKo+ANwE3IVbZbEat3T3zXLEsRQ4CdfKzMO1BM8HfqmqnwWIDgem4FaHbMNNOPbB9R5Q1Zdx
+wz434irPRbgJ0xNVdVNAPCfgfsCB19XAW6q6wLcOC1S1ALeh4Zn+hx4LW0WkMOCKtQX3MPA28L6I
+7MBNMJe9DT8RNzSyxudpVlDY/wO6++GHsnK/ATgLN+k4iijfh6rOAa4BHsFVgMtwE/DhuBBX9ltw
+k8TDVXVD1FxG525cXlcC7+PmG8p03Auc5/XajFvh83ol8hCJvsAXIlKI+15uUL91DW4O5xlf3iNx
+k/z1ccOSs4D3guJ6GDcftEVExvt5htNwZbgWN4xVNsEfjt7APN876obrpQVzKe5Zfgw42d8/Wa5c
+1xCivodgGIZh1A6sh2AYhmEAZhAMwzAMjxkEwzAMA6ikQRCRISKyRESWiduPKNi/m4jMFJEiEbml
+PGENwzCMqqXCk8r+5ZHvCTgNDbfj6aIAmUNwL4KcA2xR1b/HGjYULVu21JycnArpW9Xs3LmTrKys
+ZKuRcli5hMbKJTRWLqEpb7nMnTt3o8awuV1lzkOIepKaqv4I/ChuN9RyhQ1FTk4Oc+YctIw4JUnY
+fuXVHCuX0Fi5hMbKJTTlLRcRiWmX6MoMGYU7JS3RYQ0jLpRQynxW8yrz+IZ8iilJtkqGkVRS4sS0
+SEgcjtBMBrXuSMQYSYVyKRVlXdti8jvso6iekl4MizLWMXnf17RZm0nHVZmkl1bt0YypUC6piJVL
+aFLxCM01HLh3SnvvFtewGocjNJOBdXVDkwrlMpkFLOcH2tOMk+lCl4xWrGQTczJX8V2nApp0asW5
+HIeEO2c0AaRCuaQiVi6hSVS5VMYgzAa6ikhnXGV+IW4zsUSHNYwK8zX5zOMHTuAwTuWInyr9Q2nJ
+obTkU5bxMUtoQxMGHLTNkWHUbCpsEFS1WESuA6biDtX4j7qjMcd4/8f9JmlzcJuklYrI73CHWWwP
+FbaymTGMSKxnO++ygE405+ccHrIHcBKHUcA2prGYbBpzaKV2yjaM6kWl5hBUdQpu07RAt8cD7gtw
+w0ExhTWMRFHEPl5hHvXI5Hx6khZmPYUgnE0PNlLIa8xjNCfThPpVrK1hJAd7U9moFeSylC3s5Hx6
+0pB6EWXrksFI+rCPEj6uFuetG0Z8MINg1Hi2s5s5rKIH7elEi5jCtCCLvuTwDWvYwI4Ea2gYqYEZ
+BKPGM4NlKMrJdC1XuBM5jDpkkMv3CdLMMFILMwhGjWYru5jHanrSgWYHnIwZnQbU4Xg6s5gC1rI1
+QRoaRupgBsGo0XzKMgThZMId6xuZAXSmPpl8bL0EoxZgBsGosWxmJ/PJpzcdaVzBlUJ1yeREDmM5
+G1jF5jhraBiphRkEo8byOctJRziJwyoVT19yaEAdZrEiurBhVGPMIBg1kt3s5RvWcAztoi4zjUYm
+6fSiA9+zni3sipOGhpF6mEEwaiTzyaeYUvqSE5f4+tAJEOYQ0y7ChlEtMYNg1DhKUWaTR0ea05rG
+cYmzMfU5ktZ8xWr22TbZRg3FDIJR41jKj2xlN/3i1Dsooy857GEfC2Le1NcwqhdmEIwax2zyaEQ9
+jiA7rvF2pBmtacyX5KFU7OhZw0hlzCAYNYqNFLKCjfSmI+lxfrwFoR85/MgOW4Jq1EjMIBg1ijms
+Ip00etMxIfEfRVvqk2mTy0aNxAyCUWPYRwlfk8+RtCaLuglJI5N0etCe7yigkKKEpGEYycIMglFj
+WMQ6iiimV4J6B2X0oiOlKF+Tn9B0DKOqMYNg1BjmsooWZNGJ5glNpyUN6URz5vGDTS4bNQozCEaN
+YD3byWcrvegY8mjMeNOLjmxhFyvZlPC0DKOqMINg1Ajm8QPppNEj9ImtcedIWlOfTOba5LJRgzCD
+YFR79lHCN6yhO61pQJ0qSTPDTy4vYT2F7KmSNA0j0ZhBMKo9C1lbJZPJwZRNLs+3yWWjhlApgyAi
+Q0RkiYgsE5HbQ/iLiIz3/t+ISK8AvzwRWSAi80VkTmX0MGo3c/mBlmTRMcGTycG0pCE5tGAuP1Bq
+k8tGDaDCBkFE0oFHgaFAd+AiEekeJDYU6Oqv0cBjQf6nqOpxqtqnonoYtZu1bGMNW+lDpyqZTA6m
+D53Yxm6W8WOVp20Y8aYyPYR+wDJVXaGqe4GXgGFBMsOAieqYBTQVkTaVSNMwDmAOeWSSzrFVNJkc
+zBFk05C6zLbJZaMGkFGJsO2A1QGf84H+Mci0A9YBCkwTkRLgCVWdECoRERmN612QnZ1Nbm5uJVSu
+OgoLC6uNrlVJPMtlX4byzYBdZK/PYNb3n8UlzorQPKeU5Tkb+O+sj6i/p2JtLHteQmPlEppElUtl
+DEJlOUlV14jIIcAHIvKdqk4PFvKGYgJAnz59dNCgQVWsZsXIzc2luuhalcSzXGayglIWc3bbAWS3
+jc+5BxVhO3t4mI/IOL4NgziyQnHY8xIaK5fQJKpcKjNktAboEPC5vXeLSUZVy/7/CLyBG4IyjJhQ
+lDmsogPNyI7TITgVpTH16EY28+3wHKOaUxmDMBvoKiKdRaQOcCHwdpDM28BlfrXR8cA2VV0nIlki
+0ghARLKA04BvK6GLUctYxga2sCtuR2RWlj50Yjf7WMjaZKtiGBWmwkNGqlosItcBU4F04D+qulBE
+xnj/x4EpwOnAMmAXcKUPng28ISJlOrygqu9VOBdGrWM2eWRRlyNpnWxVAMihBa1oyBfk0YP2SVnx
+ZBiVpVJzCKo6BVfpB7o9HnCvwG9ChFsB9KhM2kbtZT3bWcYGBnF43A/BqSiCMIBDeZtvWM5GutAq
+2SoZRrlJjV+TYZSDz1hOHdJTZriojGNoRyPq8RnLkq2KYVQIMwhGtWILu1jIWnrRkfpkJludA0gn
+jePpzCo2s4atyVbHMMqNGQSjWjGTFaSRxgAOTbYqIelFR+qRwWcsT7YqhlFuzCAY1YZCipjPao71
+QzOpSF0y6EMnvqOAjRQmWx3DKBdmEIxqwxespJhSTuCwZKsSkX50JoM0PrdeglHNMINgVAt2sIcv
+yaM7bWhBVrLViUhD6tKbTnxNPuvZnmx1DCNmzCAY1YKPWUIJpfycI5KtSkz8jC7UJZMPWGznLhvV
+BjMIRsqzlm3MJ5/+dKZ5ivcOyqhPHX5GV1awkWVsSLY6hhETZhCMlEZR3mcRDajDyXRJtjrloi+d
+aE4DPmAxpZQmWx3DiIoZBCOlWUwBP7CZUziCein23kE00kljMEeykULmHbALvGGkJmYQjJRlD/t4
+n0Vk04ieB2yaW304gmw60ZwP+Y6t7Eq2OoYRETMIRkqiKO/wDYUUcQbHkFZNN4sThLPpgQKvM9+G
+joyUxgyCkZLM5QcWU8DPOYL2NEu2OpWiGQ04g6PJZwvTbZ8jI4Uxg2CkHAVsZyqL6EKrlN2iorwc
+QzuOpR2fspRVbE62OoYREjMIRkpRSBGvMo/6ZDKMHjXqXIGhHE0zGvAa89hk21oYKYgZBCNl2MEe
+JjKTHexhOL3Iom6yVYordcngAvqgKBOZxSZ2JlslwzgAMwhGSrCdPUxkFtvYw8X0oyPNk61SQmhF
+Iy7leEq8UdhsRsFIIcwgGEknny08w0x2UMQo+tGphhqDMg6hEZfSnxJKeYqZLOPHZKtkGIAZBCOJ
+FFPCh3zHU3xOCaVcUoN7BsFk05jLOZ4s6vACs1natYi9FCdbLaOWU6kzlQ2jIpRQyiLWMYNlbKCQ
+42jPaXSvdm8iV5ZWNOJXnMhHLGFW25U8znRO4DB60J5M0pOtnlELqVQPQUSGiMgSEVkmIreH8BcR
+Ge/9vxGRXrGGNWoWirKe7azqtJeH+Yg3mE8pyoX04Wx61DpjUEYG6ZxGd479uh4NqMMUvuUhPuQj
+vmMNWym1nVKNKqTCPQQRSQceBX4B5AOzReRtVV0UIDYU6Oqv/sBjQP8YwxrVEEXZSzE7KGIThWxk
+J+vZzko2sZMi6AyH0YqzyKELrWrUstLK0HRrOsM4kR/YzExWMoPlzGA59ckkhxZk05hWNKQFDWlE
+XeqRaWVnxJ3KDBn1A5ap6goAEXkJGAYEVurDgImqqsAsEWkqIm2AnBjCxo0FrGEVmxIRdVjWHl5E
+Id9UaZoVIVL7UwN28lf/VwqUUkopSgmlFFPKXkrYRzFFFLOTvZQEbc/QiLp0pgWH0pKCmUsZMqBf
+orJTrRGETrSgEy3YSREr2chyNrKKTSym4ADZNIQs6lKXDOqQTh0ySCfNX0IaggT8d/ET0Yikonmp
+Lr+jqqAfnTmERglNozIGoR0csIVjPq4XEE2mXYxhARCR0cBogOzsbHJzc8ut6MrOe1nfumon7LR5
+KZuL8qs0zUQg+y2Cq1C07BLSSkFKIb1USC+B+iVC473p1NmbQeY+ocEuof6uNDJKBNjGVrZRXLir
+Qt9hTaewsDBkuTQBjiWNkrQG7GpQyu4Gyt46yr46yt7MYkrS97E7HQrTFU2DUgFNU1RwF/xU02sq
+1vhRqCm/o3hQvHgDTbe6uaVwz0ulUdUKXcBw4N8Bny8FHgmSmQycFPD5Q6BPLGFDXb1799bqwscf
+f5xsFVKSypZLQUGBnnzyydqwYUO96aab4qNUChCqXMaOHaujRo0KG6Z79+5hy/Pjjz/Wdu3ahQ17
++eWX65133lleNasc+x2FprzlAszRGOr1ykwqr4ED9iRu791ikYklbI3nhRdeoE+fPjRs2JA2bdow
+dOhQZsyYAcC4ceO45JJLDgojIixbtn+DtEWLFnH22WfTpEkTGjVqxCmnnMLnn39+QJi9e/cybtw4
+unbtSlZWFjk5OVx11VXk5eX9JDN58mT69etHVlYWLVq0YNSoUeTn72+ZPf3005x00kkR83PFFVeQ
+kZHBunXrfnIbM2YMDRs2pGHDhtSpU4fBgwf/9Hno0KHk5eUhIj+5lV0vv/xyyDQmTJhAy5Yt2b59
+Ow888EBEfaJxxRVXcNddd1UqjvKSl5fHKaecQoMGDejWrRvTpk2rcFwLFy5k0KBB8VMuwQQ/u9EY
+NGgQ7777blzSHjlyJO+//z5FRUW0bt36AL9bbrmFrl270qhRI7p168bEiRPjkmZ1RJzxqEBAkQzg
+e+BUXGU+G7hYVRcGyJwBXAecjhsSGq+q/WIJGybNDcCqCilc9bQENkbwzwZa4/KzHde7bww0wg2h
+tQXqAiuDwvUGvgWKvP+RwAagwMfREjck9z389BpsFyDTp7ULSAeae/mNQDPcvM4qYIv3b+91WQSU
+AC183EvC5CcN6OHjXAesDyHT1ufxuwC3OsAxwNww8QbTCdgHrI1RPhI5wN44xRUr3YBC3HPfxOvw
+LdCUg5+XcM9ALDQCOkPYAfgcqj7vgc9uLBzhZfPikPbRwGJcebbH/T7KaAtsBvYAWbhFMEshpV8j
+j1a/BNNJVVtFlYqlGxHuwlX03wPLgTu92xhgjL8X3Gqi5cACoE+ksDXpIkIXDVcRFAIjIsiMA54L
+4a5AF3//LDAlhMxjwHR/PxjYDXQIk47gDMGtQe5puB/vPf7zFcCMCPpehpsXugH4NkKeNgW55fg8
+ZcRQpk/jjMFeX36DvZ63++doEzAJaB4Q5hWcsdwGTAeO8u6jg+J6J7h8A9L8k78fhDPWt/k4n/Xu
+ZwLzga3A58CxYfQ/HFfBNQpwm+5/Mwc9L768JgETgR3AwqDfUB4w2N/X97puwRnx3wP5AbI9gXk+
+npeBl8ryFS0PPp1bcMZlmw9fL0weuwCfeLmNwMsB+VRcJVsIXIBriEzGNWi2+Pv2Xv5eXEOk1Ms/
+4t27AR/gKvAlwMgYnptmwAp/fy1wfxT5t4GbE1k/VPYK9bzEJd5kZ6ymXpG+MGAIUEyESpDYDEIB
+cGUImVP8j6k+cB/wSYR0uvk4O4fwuxuY6e+vILJB+BC4H9fzKQZ6h8lThQ2Cl386qCK7AZiFa/XV
+BZ4AXgzwvwrXWq4LPATMDxdXcPkGy+AMQjHwVx9ffVxF+yOuB5wOXI6rQOuG0P1cYHGQ2z/9Fc4g
+7ME1ntKBvwCzAvzz2G8Q7gM+xfX8OuCMeb73q4Mz+jfieorDccawLF8R8+Dvv8S1pJvjWtpjwnw/
+LwJ34gx1PQ6cQwwu2xbA+UAD/x29ArwZ4J8L5AV8zsI1Oq7ELYjpiTM63cPocirOwBXiDP9W/3+n
+vx8YIkx9XA93SKLqhnhcoZ6XeFy2dUVyaAFsVNVoS59GisjWwCvIvyXu4Q1mHe4H2dynFUomMA7C
+yKwL8A+LiHTEGaEXVHU9zjhcFi1cEBuD8npkjOHG4HqY+apahKtEh/thSVT1P6q6I8Cvh4g0Kadu
+gZQCY1W1SFV343oaT6jqF6paoqrP4HoBx4cI2xDXcg5kO0RcSzhDVaeoagmuR9gjjNxI4F5V3ayq
+q4HxAX7H4wzBQ6q6T1VfxQ3TlhFLHsar6lpV3Qy8AxwXRo99uGG9tqq6R1VnhMuYqm5S1ddUdZeq
+7sD1CgaGk8f1YvJU9SlVLVbVr4DXgBFh4v9QVZsCb3qZdjjj1lJVm6rqJyGCPQ58DUyNoEeNxQxC
+4pgQwW8T0LKs0orAJP/g/nQF+W8E2oQI1wZXcW3xaYWSCYyDMDJtiG2c8lJcy3e+//w8cLGIhHr9
+eGmYOFoG5XVxDOmCq3zeCDCYi3G9o2wRSReR+0RkuYhsZ/9YdFQjF4ENqronKP2bg4x2B1xrOphC
+3BxKIE1wwzjhnpfAFxB2AfXCPDdtOXAp96ogvzXqm5Yh/GPJQ7AeDcPoeytuGPJLEVkoIleFkUNE
+GojIEyKyyn8/04Gm/sXVMqYH6dk/SM9RuLm4UPHne5mLgGdwvaBOwDoReTCE/N9wcw0jg8oqFYlU
+v1QYMwgJQlUjfWEzcS2wcyqZzDRCt45G4oZ6dnmZfiLSPkwcS3Dj4gfEIyJpuO78hzHocRlwqIgU
+iEgB8CCu0j09hGy8z5BcDQwNMib1VHUNcDHuhcfB7J/Ahf3vYIX60e/CDWGUEVzZBIdZjWuZB6bf
+QFVfDBH3Qlw5BfYIegALozwvsbCOA1fudQzyayciEsa/PHmIiKoWqOo1qtoW+DXwLxHpEkb8ZtzE
+cX9VbQz8zLsHfj+BBmE1bvgzUM+GqnptGF3a44Znp/nG1ATgNz7cTYGyInI3bmeF01R1e3nzXdXE
+4XkJiRmEJKCq24A/AI+KyDm+pZQpIkNF5P5yRHU3cIKI3CsizUWkkYhcj6ugb/NpTcNNwr0hIr1F
+JMPLjRGRq3xL6BbgLhG5WETqiUhr4N+41uw/AtIT7x94DQAOw725fpy/jgZeoPzDRhXhceBeEenk
+FWwlIsO8XyOc4d2Eq+T/HBR2PRx0Rud8XO8mXUSGEHkIA+BJYIyI9Pd7d2WJyBlBlT4Aqvq9j3+s
+L7vzcCusXos5t+GZBNwhIs288b8+wG8mbu7jt/45Ow/3fZU7D9EQkREBjY8tuEq97NX14PJuhFvw
+sFVEmgNjg6ILlp8MHC4il/p8ZIpI3yjDi71xk+kAvYA5IXS+A9d4GKyqVbulQYphBiFJqOoDwE3A
+XbhVFqtxS3TfLEccS4GTcK3MPFxL8Hzgl6r6WYDocGAKbnXINtyEYx9c7wFVfRk37HMjrvJchJtc
+OzHoB3IC7gcceF0NvKWqC3zrsEBVC4CHgTP9Dz0WtopIYcB1U/Qg4NN5G3hfRHbgJpjL3nqfiBsa
+WePzNCso7P8B3f3wQ1m53wCchZt0HEWU70NV5wDXAI/gKsBluAn4cFyIK/stuEni4aq6IWouo3M3
+Lq8rgfdx8w1lOu4FzvN6bcat8Hm9EnmIRF/gCxEpxH0vN6jfogY3h/OML++RuEn++rhhyVnAe0Fx
+PYybD9oiIuP9PMNpuDJcixvGKpvgD0dvYJ7vHXXD9dKC+TOux7Qs4Pn7f+XNeE2gwu8hGIZhGDUL
+6yEYhmEYgBkEwzAMw2MGwTAMwwDMIBiGYRieanWmcsuWLTUnJyfZasTEzp07ycrKSrYaKYeVS2is
+XEJj5RKa8pbL3LlzN2oMm9tVK4OQk5PDnDkHLSNOSXJzc6vV1sRVhZVLaKxcQmPlEprylouIxLRL
+tA0ZGUZ52LIFOneGrCxo0wa6d4dXX022VoYRF8wgGEZ5GDsWfvgBrr4azjoLVOHXv3aGwjCqOWYQ
+DCNWFiyAf/0Lrr0Wxo+HCRPgpZecMbj33mRrZxiVxgyCYcSCKlx/PTRtCvfcs9+9Rw+44gr45z9h
+xYqwwQ2jOmAGwTBi4ZVX4JNPXE+gedD2TH/8I2RkwB13JEc3w4gTZhAMIxpFRXDzzdCzJ/zqVwf7
+t2sHt9wCkybBzJlVr59hxAkzCIYRjalTIT/f9QTS00PL/P73cMghcN99VaubYcQRMwiGEY1XXoFm
+zeC008LLNGwIF13kjMf2lD9fxTBCYgbBMCKxZw+89Racey5khjoRNIDhw93w0rvvVo1uhhFnzCAY
+RiTefx927IARIc9xP5ATTnAvq9mLakY1xQyCYURi0iQ3XHTqqdFl09JcT+K//4WdOxOvm2HEGTMI
+hhGOPXvg7bdjGy4qY/hw2L3bGQXDqGaYQTCMcEyd6oaLRo6MPczJJ0OrVjZsZFRLzCAYRjheecW9
+hPbzn8ceJiPD9SgmT3Y9BcOoRiTMIIjIEBFZIiLLROT2EP7dRGSmiBSJyC2J0sMwKkTZcNE558Q+
+XFTG8OFuDmHq1MToZhgJIiEGQUTSgUeBoUB34CIR6R4kthn4LfD3ROhgGJUiN9cNF51/fvnDDhrk
+ehavvRZvrQwjoSSqh9APWKaqK1R1L/ASMCxQQFV/VNXZwL4E6WAYFee996BePTjllPKHzcyE0093
+cZSWxl83w0gQiToxrR2wOuBzPtC/IhGJyGhgNEB2dja5ubmVVq4qKCwsrDa6ViXVpVz6vvEGRccc
+wzdffFGh8Id07Ej3jRuZ++ST7DjiiKjy1aVcqhorl9AkqlxS/ghNVZ0ATADo06ePVpfj9Ozov9BU
+i3LJy4MffiDrxhsrrutRR8Ff/kLvDRvcATpRqBblkgSsXEKTqHJJ1JDRGqBDwOf23s0wUp+yyeAh
+QyoeR6tW0KuXGzYyjGpCogzCbKCriHQWkTrAhcDbCUrLMOLLe+9Bp04Qw1BPRIYMgVmzYOvW+Ohl
+GAkmIQZBVYuB64CpwGJgkqouFJExIjIGQERai0g+cBNwl4jki0jjROhjGDGzdy98+KGrzEUqF9eQ
+IVBS4uIzjGpAwuYQVHUKMCXI7fGA+wLcUJJhpA4zZ7rlppUZLirj+OOhSRPX46jI8lXDqGLsTWXD
+COS999zbxuV5OzkcGRluU7z33nNnMhtGimMGwTACee89OPFEaByn0cshQ9xpa4sXxyc+w0ggZhAM
+o4yCApg/H375y/jFWRaXrTYyqgFmEAyjjLJKOx7zB2V07Ajdu9t22Ea1wAyCYZQxeTK0awfHHRff
+eIcOhenT3WS1YaQwZhAMA9xy0/ffd3sQVXa5aTBnnuninzYtvvEaRpwxg2AYADNmuBb8GWfEP+4T
+T3TLTydPjn/chhFHzCAYBsC770KdOrGdnVxeMjPd5PKUKbb7qZHSmEEwDHAGYdAgaNgwMfGfeaZb
+xTRvXmLiN4w4YAbBMJYvhyVLEjNcVMbQoW5uwoaNjBTGDIJhvPuu+59Ig9CyJQwYYAbBSGnMIBjG
+u++6nU0POyyx6ZxxBsydC+vWJTYdw6ggZhCM2k1hoTs/+cwzE59WWRpTpkSWM4wkYQbBqN1Mnere
+EUjkcFEZxxwDHTrAO+8kPi3DqABmEIzazfPPQ3Y2nHxy4tMSgbPPdkZo+/bEp2cY5cQMglF72bzZ
+zR9cfLHbqroquOQS2LMHXn21atIzjHJgBsGovbz6qhsuuuSSqkuzf3/o2hUmTqy6NA0jRswgGLWX
+556DI4+Enj2rLk0RuOwy+OQTWLWq6tI1jBgwg2DUTvLy4NNPXe8g3pvZRaOsR/Lcc1WbrmFEwQyC
+UTt5/nn3f9Soqk87JwcGDnTDRna0ppFCmEEwah+qrnX+s59Bp07J0eGyy+D77+HLL5OTvmGEwAyC
+UfuYOxe++65qJ5ODGT4c6tWzyWUjpTCDYNQ+/v53t6vpiBHJ06FxYzj3XHjhBdi2LXl6GEYAZhCM
+2sWCBTBpEtxwAzRtmlxdbrkFtm6Ff/wjuXoYhscMglG7uPtuaNQIbrop2ZpAr15u6OiBB8i0XoKR
+AphBMGoP8+fDa6/BjTdC8+bJ1sZxzz2waxcdX3gh2ZoYhhkEoxYxbpwbJvrd75KtyX6OPBIuvZS2
+b74Ja9YkWxujlmMGwagdzJ4Nb70FN9+c/LmDYMaORUpL4U9/SrYmRi3HDIJR89m8GS66CFq3ht/+
+NtnaHEznzqw74wx48kmYNi3Z2hi1GDMIRs2muBguvBB++MHNHzRunGyNQrLimmvc8NGIEe58Z8NI
+AmYQjJrNbbfBBx/AY4/BCSckW5uwlGRluYNzMjPdyWqbNiVbJaMWYgbBqJmUlsJf/woPPgjXXQdX
+X/NAQ8sAAAZRSURBVJ1sjaKTkwNvvul6M+edZ0bBqHLMIBg1j7w8+PnP4fbb4fzznVGoLpxwAjz9
+NHz+ORx1FLz+erI1MmoRCTMIIjJERJaIyDIRuT2Ev4jIeO//jYj0SpQuRi1h6VK3rv/YY2HePHjq
+KXjlFTcMU5246CKYMwfatnUGbfhw+OgjNx9iGAkkIecGikg68CjwCyAfmC0ib6vqogCxoUBXf/UH
+HvP/DSM8paWwY4fb/+eHH9wE7JIlbnXOV1+5sw2GDHFzBsnayTQe9OgBX3wB998Pf/6zmxBv2RLO
+OssZvK5doUsXaNECmjSpfkbPSEkSdZBsP2CZqq4AEJGXgGFAoEEYBkxUVQVmiUhTEWmjquvirs24
+cfv3v68i+u3eDfXrV2ma1YH+u3aFLpeycwFU91+lpe5/cbE76nLvXti58+AzBOrUcdtAPPigW6XT
+vn3iM1IVZGbCnXe6F+nee88ZhTfecD2fYOrVc1dmprv+f3v399pVHcdx/Pnaj2wokWlt4UZ64Y39
+gGJI4M3AKFPRrsKisB84BAUDIVL/gSCoiAiRDISCECKUMMqsXdovS8QsEyJTNOtCms4a3+3VxTnL
+o55tbu18z3ee9wOG59e++3xffnc+58fO+9PcDE1NyVd2AKCxBgOq90BB1yF+jzJ27kxKtheoqA5h
+HvBbZv4U1x79520zD7iiQ5DUC/QCtLe309fXN+HGdFy6xOw6Hy3WajX66zVw+zRSq9VoGeVo1lft
+uNyUXNF0SwvDra24uZmhtjZqs2ZRmzmTwTlzGOjq4u+OjmQHCHDiRPI1zVy4cGHsz/acOdDbC+vW
+0Xr+PG2nTtF25gwt/f20XLxI88AATYODaGiIploNhofRSKea0liD8TToQD3xe3TZyePHuZj+f477
+eZmkhk/a9g5gB0B3d7d7enom/iKT+Z7/qa+vj0m19QYXueSLXPJFLpe1Z6aLyqWom8qnga7MfGe6
+bKLbhBBCqBO5gFNFSS3AcWApyU7+a+BJ20cz26wANgLLSS4nvWF78Tiv+wfw65Q3uBhzgT/LbkQD
+ilzyRS75Ipd8E83lLtu3j7dRIZeMbNckbQQ+AZqBd2wflbQ+Xb8d2EfSGZwABoBnr+N1x31DjULS
+N7a7y25Ho4lc8kUu+SKXfEXlUtg9BNv7SHb62WXbM9MGNhT180MIIUxMPKkcQggBiA6hSDvKbkCD
+ilzyRS75Ipd8heRSyE3lEEII00+cIYQQQgCiQwghhJCKDqEgkjZLsqS5mWVb0uquP0l6pMz21Zuk
+VyT9mFa2/VDSrZl1lc0Fxq8MXAWSuiR9IekHSUclbUqX3yZpv6Sf039nl93WMkhqlvSdpI/S+UJy
+iQ6hAJK6gIeBk5lli4A1wN3AMuCttCpsVewH7rF9H8lDi1sgcslUBn4UWAQ8kWZSNTVgs+1FwIPA
+hjSHl4ADthcCB9L5KtoEHMvMF5JLdAjFeA14EcjesV8NvG/7H9u/kDyQN+aT2TcS25/aHinof5Ck
+VAlUPBcylYFtDwIjlYErxfYZ24fS6X6Snd88kix2pZvtAh4rp4XlkdQJrADeziwuJJfoEKaYpNXA
+aduHr1o1WnXXKnoO+DidrnouVX//15A0H7gf+BJoz5TEP8uVNd6q4nWSA8zhzLJCcmn4aqeNSNJn
+QEfOqm3AVpLLRZUzVi6296TbbCO5PFDfASrCtCBpFvAB8ILtv5QpiW7bkir1d/KSVgLnbH8rqSdv
+m6nMJTqESbD9UN5ySfcCC4DD6Qe5EzgkaTEVqO46Wi4jJD0DrASW+vIDMDd8LuOo+vv/j6RWks7g
+Pdsjg0n/PjJwlqQ7gXPltbAUS4BVkpYDNwO3SHqXgnKJS0ZTyPYR23fYnm97Psnp/wO2zwJ7gTWS
+ZkhaQDJ06FclNreuJC0jOe1dZXsgs6rSuZBUAl4oaYGkm0husO8tuU11p+QIaidwzParmVV7gbXp
+9FpgT73bVibbW2x3pvuTNcDntp+ioFziDKFO0mqvu0mGEa0BG2wPldysenoTmAHsT8+eDtpeX/Vc
+RqsMXHKzyrAEeBo4Iun7dNlW4GVgt6TnSUrfP15S+xpNIblE6YoQQghAXDIKIYSQig4hhBACEB1C
+CCGEVHQIIYQQgOgQQgghpKJDCCGEAESHEEIIIfUvipf89bvnk4cAAAAASUVORK5CYII=
+"
+>
+</div>
+
+</div>
+
+<div class="output_area"><div class="prompt"></div>
+
+
+<div class="output_png output_subarea ">
+<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAYoAAAEICAYAAABBBrPDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz
+AAALEgAACxIB0t1+/AAAIABJREFUeJzsnXeYVdXV8H9rGjPM0AeHXhSUoKAyiNjh01fBaIgRSewl
+ihhNjCWWYF4wnxo1MVFejS2xoFFssb58ihIQUVFBEQRERooMMPQ21Cnr+2Pvkcv13ju3zdw7M+v3
+POe555zd1j7n3L32XruJqmIYhmEY4chItQCGYRhGemOKwjAMw4iIKQrDMAwjIqYoDMMwjIiYojAM
+wzAiYorCMAzDiIgpCqNBISJFIjJDRLaLyH2plqcuEZHxIvJsBPcFIjIkjNsQESmNEPYpEbkjCWIa
+TQBTFClERM4TkdkiUi4ia0Tk/4nI8d4tZCEhIioivQKu+4rIGyKy1Ree00Tk2KAwOT6+JSKyQ0SW
+i8gTItIjwM8ZIvKpd98oIv8SkS4B7peIyMxa8vOUiFSKSMeAe4/4/JWLyF4RqQi4/n8i0sPnqTzo
++HmYZEYDG4CWqnpDxAdcC6koLEXk/4rIfP+cxicSl6oeqqrTkyNZ3RP87Ubhf7qIXJ6ktF8UkVNF
+pJmIlAW5NfP/h20iUiYi1ycjzcaEKYoU4T/G+4G7gCKgG/AQ8JMY4jgI+BCYD/QEOgGvAlNE5JgA
+ry/7eM8DWgGHA7OBk308I4HnvDyFwKHAHmCmiLSJUpZ84GxgK3BBzX1VHaOqBapa4PP6Qs21qg4P
+iKJ1wP0CVX0hTFLdgYWaBjNFRSQrjmAlwE3A/yZZHCMyxbhvvj/wVZDbeKA37tsaCtwkIsPqVbp0
+R1XtqOcDV1iXA+dE8DMeeDbEfQV6+fNngMkh/DwMzPDnpwC7gK5h0hFgBXBT0P0M3B/qj/76EmBm
+BHkvAlYC1wJfRZsnoIfPU1YUz+0poALY65/fKV7OW4BvgY3Ai0DbgDAvAWU4BTYDONTfHx0U15vB
+zzcgzTv8+RCgFLjZx/mMv38GMBfYAnwE9I8iL88C42vxM97nZyKwHVgADAxwXw6c4s/zvKybgYXA
+74DSAL9HAp/7eF4AJtXkq7Y8+HRuBOb55/gCkBtG5l7A+97fBlzFAP/sFdjhn/fPgTbAW8B6L/db
+QBfv/06gCtjt/T/o7/cB3gU2AYuBUVE86zbAUn9+FXBvkPtq4NSA6z8Ck+qzTEj3I+UCNMUDGAZU
+EqFwJDpFUQZcGsLPUP8nywPuBt6PkE4fH2fPEG63Ax/780uIrCimAvfiWkeVQHE0eSIGReH9PxVU
+wF0LzAK6AM2AR4HnA9wvA1p4t/uBueHiCn6+wX5wiqISuMfHl4crgNcBRwOZwMW4grVZLfmIVlHs
+Bk73cf8JmBXgvpx9iuJu4AOgLdAVp+RLvVsOrjJwHZANjMQpyZp8RcyDP/8U12JtCywCxoSR+Xlg
+LE6B5wLHR3i27XCt0Ob+Hb0EvBbgPh24POA6H1cZuRTI8nJvAPqGkeVknOIrx1UItvjfHf78JJwS
+UaAoINzZwPz6LhfS+TDTU2poB2xQ1cpa/I0SkS2BR5B7IbAmRLg1uD9qW59WKD+BcRDGz5oA97CI
+SDeccnpOVdfilMZFtYULYkNQXn8UZbgxwFhVLVXVPbjCdWSNWUhVn1DV7QFuh4tIqxhlC6QaGKeq
+e1R1F65l8qiqfqKqVar6NM5sNziBNAKZqaqTVbUK14I8PIy/UcCdqrpJVVcCEwLcBuMUxP2qWqGq
+LwOfBbhHk4cJqrpaVTcBbwJHhJGjAmfC6aSqu1U1bL+Wqm5U1VdUdaeqbse1Ik4K5x/X6lmuqk+q
+aqWqfgG8ApwTJv6pqtoaeM376YxTeoWq2lpV3wcKvPetAUG34RSX4TFFkRo2AoVR2Lhf9B/090eQ
++wagY4hwHXEF2mafVig/gXEQxk/HAPdIXAgsUtW5/vpfwHkikh1F2BoKg/K6KMpw3YFXAxTpIlxr
+qkhEMkXkbhH5VkS24QoJiEL5RWC9qu4OSv+GIGXeFVf7TgaBHa87gdww300nXG27hhVBbqvUV5dD
+uEeTh2A5CgjNTThz5qd+VNZlYfwhIs1F5FERWeHfzwygtYhkhgnSHTg6SM7zgQ5h4i/1fs4Fnsa1
+mroDa0Tkr95buf9tGRC0Fc5EZ3hMUaSGj3E1tp8mGM97hK5NjcKZjHZ6P4MCRzAFsRhnd98vHhHJ
+wDXBp0Yhx0XAgX7ESBnwV1xhfHpUuUiMlcDwICWTq6qrcJ33I3B9Ga1wZi5wBRk4k0MwO3GmkBqC
+C6HgMCtxNfnA9Jur6vMJ5Cke1uAK9xq6Bbl1FhEJ4560PKhqmapeoaqdgCuBv0cY6XQDcAhwtKq2
+BE7098O9n5U4M2qgnAWqelUYWbrgzLzv+UrWY8DVPtz13s9m3PMJbKkdjusPMjymKFKAqm4F/ht4
+SER+6mtW2SIyXETujSGq24FjReROEWkrIi1E5Ne4gvtmn9Z7uM6/V0WkWESyvL8xInKZr2XeCNwm
+brhuroh0AP6Bq2X9LSA98e6BxzHAQcAgnDniCOAw3CiqWM1P8fAIcKeIdPcCtheREd6tBU4hb8QV
+/ncFhV0LHBh0by6uNZTpR75EMoUAPA6MEZGjxZEvIj8WkZCmC/+ec3H/vSz/DMPVoGPhReBWEWnj
+KwW/DnD7GNe38huf/s9w7yuuPERCRM4JqJRsxhX21f46+Hm3wA202CIibYFxQdEF+38LOFhELvT5
+yBaRo2oxUxbjOvEBBuBGPgUzEff9t/FxXYHrmzI8pihShKreB1wP3IYb9bESuAZnT402jiXA8bga
+0HJczehs4DRV/TDA60hgMm60ylZcR+dAXGsDdUNRL8R1dm7EjZrJA45T1Y0B8RyL+2MHHr8EXlfV
++b42WaaqZcADwBm+AIiGLbL/PIpox7I/ALyBGxK8HdexfbR3m4gzsazyeZoVFPafQF9vxqh57tcC
+Z+I6O8+nlvehqrNxBcuDuIKxBNfxH47Hcc/tXFyn7y7cs0+U23F5XQZMwfVn1Mi4F/iZl2sTbsTR
+vxPIQySOAj4RkXLce7lWVZd6t/HA0/55j8INLsjDmTdnAW8HxfUArr9ps4hM8P0YpwK/wI1UKmPf
+wIJwFAOf+9ZUH0K3FMbhRs2twHWg36uqwbI0aWR/s6VhGIZh7I+1KAzDMIyImKIwDMMwImKKwjAM
+w4iIKQrDMAwjIvEsapYyCgsLtUePHvWe7o4dO8jPz6/3dGvD5IqddJXN5IoNkys25syZs0FV28cd
+QX2uF5LoUVxcrKlg2rRpKUm3Nkyu2ElX2Uyu2DC5YgOYrbbWk2E0Laor4fN/wDqbP2zUAw3K9GQY
+hmPu0/DmFe68yzFQPBoOvwjEqn5GHWCflWE0MKqr4MN7oMMRcOp9sHszvH4pfPb3VEtmNFZMURhG
+A2PRK7BpCZwwFo65Hn61EDoWw9wnUy2Z0VgxRWEYDQhVmPknaHcI9DnL3ROB/hfCms9h/cLUymc0
+TkxRGEYDouRtKJsLx90MGQFrzvY7FyQTvnwmfFjDiBdTFIbRgJj5J2jZBfqfv//9/AOg12kw/1nQ
+6tBhDSNeTFEYRgNh9Wz47gM49neQmfND9/4XwbZSWD693kUzGjmmKAyjgbBkMiDQ7/zQ7of8BJq1
+hHlmfjKSjCkKw2ggLJsKHY+E5u1Cu2fnwY9GwsKXoWJn/cpmNG5MURhGA2DvDlj5MfQ8ObK/wy+E
+veXw9ev1I5fRNDBFYRgNgO9mQnVF7Yqi+4mQ1w6Wvls/chlNA1MUhtEAWPqe68Dudnxkf5IB3U+A
+Fe/Xj1xG08AUhWE0AJZNdWs65USxgnX3k2DzUjcCyjCSgSkKw0hzdm50k+xqMzvV0P0k97tiRt3J
+ZDQtTFEYRpqzfBqgcGCUiqKoPzRrBcvN/GQkCVMUhpHmLJ0KOQXQ6ajo/Gdkur4M66cwkoUpCsNI
+c5ZNdeakzOzow3Q/ETYuhvK1dSeX0XQwRWEYaczWlW5J8Wj7J2qwfgojmZiiMIw0Ztl/3G+0/RM1
+dBwA2fmmKIzkYIrCMNKY72ZCbms44LDYwmVmQ9djrZ/CSA6mKAwjjVk5E7oeF99e2N1PgnXz3fBa
+w0gEUxSGkabsWA8bvq59NnY4evh+iu9mJk8mo2liisIw0pSVH7nfbifEF77TUZCVa+YnI3ESUhQi
+MkxEFotIiYjcEsJdRGSCd58nIgMC3JaLyHwRmSsisxORwzAaI999AJnNoNPA+MJnNYPOR7t4DCMR
+4lYUIpIJPAQMB/oC54pI3yBvw4He/hgNPBzkPlRVj1DVOP8KhtF4+W4mdD7KFfjx0v1EWPM57Nme
+PLmMpkciLYpBQImqLlXVvcAkYESQnxHARHXMAlqLSMcE0jSMJkHFTlgzB7rG2T9RQ7cT3B7aNWYs
+w4iHRBRFZ2BlwHWpvxetHwXeE5E5IjI6ATkMo9Gx6lOoroy/I7uGrseAZJr5yUiMrBSmfbyqrhKR
+A4B3ReRrVf3B9CCvREYDFBUVMX369HoWE8rLy1OSbm2YXLGTrrIFy7Xime4gPVhR/SGrp1cmFHdB
+7wHMe7OajFPmJixXumBy1TOqGtcBHAO8E3B9K3BrkJ9HgXMDrhcDHUPENR64sbY0i4uLNRVMmzYt
+JenWhskVO+kqW7Bcz5yq+vd+yYn7nRtU/2+OasWuxOVKF0yu2ABma5xlvaomZHr6DOgtIj1FJAf4
+BfBGkJ83gIv86KfBwFZVXSMi+SLSAkBE8oFTga8SkMUwGg3Vla5PIVGzUw3dT4SqvbDqs+TEZzQ9
+4jY9qWqliFwDvANkAk+o6gIRGePdHwEmA6cDJcBO4FIfvAh4VURqZHhOVd+OOxeG0YhYOx/2lidP
+UdTEs2KG2ybVMGIloT4KVZ2MUwaB9x4JOFfg6hDhlgKHJ5K2YTRWahbyS5aiyGvr1or6bgYwNjlx
+Gk0Lm5ltGGnG0inQtje06pa8OLud6MxZ1Yn1ixtNFFMUhpFGVO6B5dPhoNOSG2/3E5w5qyz2gU+G
+YYrCMNKJlR+6yXa9kqwoataLWmHzKYw4MEVhGGlEyTuQkQ09hiQ33padnTlr6ZTkxms0DUxRGEYa
+8e070O04yClIftwHn+F2zNtbnvy4jcaNKQrDSBPKy2Dtl8nvn6jh4DPdfIpv362b+I3GiykKw0gT
+agrwg06tm/i7He+2Vf0meFqsYdSCKQrDSBOWToHm7aHDEXUTf2Y29BoO3/wvVFfVTRpG48QUhWGk
+AVoN306Bg/4rvv2xo+XgM2Hnerc6rWFEiykKw0gDyr8tYMe6uuufqKHXMMjIgsVmfjJiwBSFYaQB
+m2a1A+DA/6rbdPLauDkV37xZt+kYjQtTFIaRYrQayt7uQPeToEU97P94yE9g/QLYvLTu0zIaB6Yo
+DCPFLPsP7F6dR/GV9ZPewWe638XWqjCixBSFYaSYOY9BVssKfnRW/aTX9iBofyjMfxbcvmGGERlT
+FIaRQsrXwtevQofTysjKrb90B10Dq2e7BQgNozZMURhGCvnyabf0d8cz1tRruodfDPkHwEf31muy
+RgPFFIVhpAiths8fd1uVNu+2s17Tzs6DQb+Bkrdh7bx6TdpogJiiMIwUsWwabCqBAaNTk/5Rv4Ls
+fPjoz6lJ32g4mKIwjBRQXQnTboO8dtD37NTIkNcGikfD/Odhy4rUyGA0DExRGEYK+PBeKJ0Fw/+H
+eu3EDmbwdSACH96TOhmM9McUhWHUM2u+gOnj4NCfQ79zUytLq65QPAZmP+xaFoYRClMUhlGPVO6G
+Vy9wq8T++O+plsZx2n2uQ/31S2Hlx6mWxkhHTFEYRj2xZzu8djGsXwgjnoC8tqmWyJGZA6NegZZd
+4IWfwpblqZbISDcSUhQiMkxEFotIiYjcEsJdRGSCd58nIgOiDWsYjYmVH8OjR8DCl+HkP7lVXNOJ
+5oVw3ltQuQeePAHm/csN3zUMSEBRiEgm8BAwHOgLnCsifYO8DQd6+2M08HAMYQ2jQVOxC5ZMhjdH
+u8JXq+GS9+H4NK0WFfaBC/3mSa9eAI8fBRtmFrJrc6olM1JNVgJhBwElqroUQEQmASOAhQF+RgAT
+VVWBWSLSWkQ6Aj2iCJs0Vs9249XjZd3CA/iqLHnyJAuTK3bile37NZHUn6vbJU6r3T7Ulbtc/8OO
+9bB9lTtWz3H3s/PhiEvh1L9Abqtk5ib5dB4Eo2fD/OfgP2NhzR8OY8EfoN0hbue95u2heTu3pWpm
+M7drXka222xJMtwIKmRffCJhk0qIdP3G6lKu+lpdOBSJKIrOwMqA61Lg6Cj8dI4yLAAiMhrXGqGo
+qIjp06fHLOiS+3uz+vXOMYfbR18WJRC67jC5YqduZcvIqSKncC/NCvdQNKyctoM30vqIrWTkVDPr
+i/DhysvL4/q264wu0P/xDMo+y6ZiWRHbF7Xk2w/yqdyWRWV5dqqlI32/sbqTq98982g7aFMdxR6Z
+RBRFvaCqjwGPAQwcOFCHDBkScxwD+8Duu+OX4dNPP2XQoEHxR1BHNEW51m1Yyy9/ew7zF33BRaNG
+88eb76s/2XztWCSgBp3hatZZuZDVDLLyMhHJA/KA1kCXqKKePn06wd/2+PHjKSkp4dlnnw0Z5tBD
+D+Whhx76Qbia+C644AJKS0tDhr3kkkvo0qULd9xxR2S5cqYzZOyB+92rroQ921xLqqoCqitcy0o1
+qF+jDlembYrffssu/ckpqJOoa0dV4zqAY4B3Aq5vBW4N8vMocG7A9WKgYzRhQx3FxcWaCqZNm1Yn
+8f7rX//S4uJizc/P1w4dOuiwYcP0gw8+UFXVcePG6fnnn/+DMIAuWbLke7kWLFigZ555prZs2VIL
+Cgp0yJAh+uGHH+4XZs+ePTpu3Djt1auXNm/eXLt3766XXnqpLlu27Hs/b775ph511FHavHlzbdu2
+rZ533nm6cuXK792ffPJJPe644yLm5+KLL9bMzEx9+eWXv7935ZVXan5+vubn52t2drZmZWV9fz1s
+2DBdtmyZAt/fqzkmTZoUMo0//vGPetZZZ2l1dXXkhxuGwHd58cUX69ixY+OKJ15uu+02PeywwzQz
+M1PHjRsXUq4awn0D0TBt2jTt3LlzWPdo857Mbz/w242Gk046SR9//PGQbrHKdc455+g777yju3fv
+1qKiov3cXnjhBT3mmGM0Ly9PTzrppJjiTVSu+gKYrXGW9aqKaJwL0otIFvANcDKwCvgMOE9VFwT4
++TFwDXA6zrQ0QVUHRRM2TJrrgVQsNlAIbEhynEVAB1x+tuHqXy2BFjhTXCegGbAsKFwx8BWwx4fv
+AKwHynwchTjT3jfADh+mF5Dt09oJZAJtvf8NQBtcv9EKYLN37+JlWQhUAe183IvD5CcDONzHuQ0I
+tX9aqDzlAP2AOWHiDaY7UAGsjtJ/MIHvsgewN4G44qEdTv72wK6AtEN9Y+G+gWhoAfQEwi3514Po
+8p7Mbz/w242GQ4CNYdKPVa7DgEW459kF9/+ooQXOupKL+w+G+8ajoS7KimRwiKq2iDt0IloGpwC+
+Ab4Fxvp7Y4Ax/lxwo5u+BeYDAyOFTdeDBLVxiPhaAeXAORH8jAeeDXFfgV7+fCMwOYSfh4EZ/vwU
+XIHUNUw6glMQNwXdz8D9qf/ory8BZkaQ9yJcv9O1wK5o84QrsBTIiuK5PYUrZPf653eKl/MW/x1t
+BF4E2gaEeQmnRLcCM4Cv/P3RQXG9Gfx8A9K8w58PwSnxm32cz/j7ZwBzgS3AR0D/KPLyLDA+0jfm
+n9eLwERgO7Ag6D+0HDjFn+d5WTfjlPvvgNIAv0cCn/t4XgAm1eQrUh6A2T6dG3FKZ6sPnxsmX72A
+972/DcAL/v4M/2x3+Of9c1wF5S1cRWezP+/i/d+Jq6Ds9v4f9Pf7AO8ClbgCfVQUz7oNsNSfXwXc
+G8bf5cD0BP/bSS0rknUkKlfKM9AQjmS/fGCY/9DDFo5EpygqgEtD+Bnq/2R5wN3A+xHS6ePj7BnC
+7XbgY39+CZEVxVTgXlxLSYHiaPJEDIrC+38qqIC7FpiFqyU2w5k7nw9wvwxXY2wG3A/sDBdX8PMN
+9oNTFJXAPT6+PFwBvA7XYs4ELsYVrM1qyUe0imI3rlKVCfwJmBXgvpx9iuJu4ANcS7ErTsmXercc
+XGXgOlzLcqT/dmryFTYP7FMUn+JaOG1xNfMxYfL1PDAWp8BzgeMjPNt2wNlAc/+OXgJeC3CfDlwe
+cJ2Pq4xc6uU6EqeM+oaR5WSc4ivHVQi2+N8d/vykIP+mKMIcNjM7NbQDNqhqZS3+RonIlsAjyD0L
+CLXjzRrcH7WtTyvSrjiFAWFCxVMY4v5+iEg3nHJ6TlXX4kxPF9UWLogNQXn9UZThxuBapKWqugdX
+uI705k1U9QlV3R7gliciiQxSrQbGqeoeVd2Fa5k8qqqfqGqVqj6NM60MTiCNQGaq6mRVrQKewZn3
+QjEKuFNVN6nqSmBCgNtgnIK4X1UrVPVlnLm3hmjyMEFVV6vqJuBN4IgwclTgzIOdVHW3qs4MlzFV
+3aiqr6jqTlXdjmtFnBTOP67Vs1xVn/ThvwBeAc4JE/9UVW0NvOb9dMYpvUJVba2q70dIywjAFEV0
+PJbk+DYChTWFWQRe9B/090eQ+zbc4IBgOuIKtM0+rUijr2vsqeHiicbeeiGwSFXn+utXgPNEJJZx
+lIVBeY12lGF34NUARboI15oqEpFMEblbRL4VkW24QgKiUH4RWK+qu4PSvyFImXfF1b5jIdw3Fjgq
+fyeQG+a76cT+Q85XBLmtUl+1DOEeKQ81cgXLEW78zU04c+anIrJARC4L4w8RaS4ij4rICv9+ZgCt
+/YTcUHQHjvby9fW/5+P66ULFX+r9nAs8jWs1dQfWiMhfw8mVIMkuK5JFQnKZoogCdUN0k8nHuBrb
+TxOM501C16ZG4UxGO4H3gEEiEm6c5mKc3X2/eEQkA2cWmBqFHBcBB4pImYiUAT/BFcanR5WLxFgJ
+DA9SMrmqugo4DzeR8xRcv1APH6ZmGliokRw7caaQGoILoeAwK3E1+cD0m6tqTGuxJuEbW4Mr3Gvo
+FuTWWWS/6W+B7mHzEKtcqlqmqleoaifgSuDvItIrjPcbcB3WR6tqS+BEfz/c+1mJM6PWyNdaVQtU
+9aowsnTBmXnf85Wsx4CrfbjrY8lXtNRBWZEUEpXLFEUKUNWtwH8DD4nIT33NKltEhotILLsY3w4c
+KyJ3ikhbEWkhIr/GFdw3+7Tew3X+vSoixSKS5f2NEZHLfC3zRuA2ETlPRHJFpAPwD9wIkL8FpCfe
+PfA4BjgIN1P/CH8cBjxH7OaneHgEuFNEunsB24vICO/WAqeQN+IK/7uCwq4FDgy6NxfXGsoUkWFE
+NoUAPA6MEZGj/dpm+SLyYxEJOcLEv+dc3H8vyz/DcDXoWHgRuFVE2vhKwa8D3D7G9a38xqf/M9z7
+iisPkRCRcwIqJZtxhX3N7Irg590CN9Bii4i0BcYFRRfs/y3gYBG50OcjW0SOqsVMWYzrxAcYgOvb
+CJY507+TLCDDv5N0mFWYNpiiSBGqeh9wPXAbbtTHStxQ4tdiiGMJcDzObr0cV3M8GzhNVT8M8DoS
+mIwbrbIV19E5ENfaQFVfwJmPrsMVqgtxHbXHqerGgHiOxf2xA49fAq+r6nxfmyxT1TLgAeAMXwBE
+wxYRKQ84oq3xPQC8AUwRke24ju2aWf4TcSaWVT5Ps4LC/hNvwhCRmud+LXAmrrPzfGp5H6o6G7gC
+eBBXMJbgOv7D8TjuuZ2L6/TdhXv2iXI7Lq/LgCm4/owaGfcCP/NybcKNOPp3AnmIxFHAJyJSjnsv
+16pfqgfXR/S0f96jcIML8nDmzVnA20FxPYDrb9osIhN8P8apwC9ww3rL2DewIBzFwOe+NdUHN3Is
+mAtx7+Fh4AR//nhMuW7kxD2PwjAMw2gaWIvCMAzDiIgpCsMwDCMipigMwzCMiJiiMAzDMCKS9suM
+B1JYWKg9evSo93R37NhBfn5+vadbGyZX7KSrbCZXbJhcsTFnzpwNqto+7ggSWf+jvo/Gtsx4ophc
+sZOusplcsWFyxQYJrvXUoFoUhhFMFdV8zFI2sYND6MBBFJJFMuavGYZRgykKo8GyiR38m7msZgs5
+ZDKXUpqRxQC6cQp9kMDNmw3DiBtTFEaDZCFreJ0vyUQYyQAOoYhlbOALVvIxS2lJLkfTM9ViGkaj
+wBSF0eDYwk5e50sOoAUjGUAr8gDoxQEcRHteYDbvsogutKEzwQvuGoYRKzY81mhwvMNCAM7myO+V
+RA2CMILDaUEuL/M5u6hIhYiG0agwRWE0KL5hLYtZy4n0pvV+q4HvI48czuZItrObN/myniU0jMZH
+VIpCRIaJyGIRKRGRW0K4i4hM8O7zRGRAgNtyEZkvInNFZHbA/bYi8q6ILPG/bZKTJaOxUkEVb7OA
+QgoYXEv/QxfaMISD+Zq1fMemepLQMBontSoKv1b+Q8BwoC9wroj0DfI2HOjtj9G45XoDGaqqR6jq
+wIB7twBTVbU3bnOcHyggwwhkJiVsYRencxiZUdRxBtGD5uTwId/Wg3SG0XiJpkUxCChR1aXq1rWf
+hNs1LJARwEQ/t2MWbjvDSNtv1oR52p8/TeK7vRmNmN1UMItlHEpHetAuqjA5ZDGIHixhHWVsq2MJ
+DaPxUut+FCIyEhimqpf76wtxWxdeE+DnLeBu9Rupi8hU4GZVnS0iy3Cb5VThNnB/zPvZon4PaL+p
+yGb94Z7QiMhoXCuFoqKi4kmTJiWa55gpLy+noCDcFsGpoynJVdqlgqW99nLk7FxalEc/oa4iS/l0
+8E7abszkR4tym9QzSwYmV2ykq1xDhw6dE2TRiYn6GB57vKquEpEDgHdF5GtVnRHoQVVVREJqLK9Y
+HgMYOHCgDhkypM4FDmb69OmkIt3aaCpyVaM8yDS60oYzBx4bc3hhER8XLaV/0VHMm/5Zk3hmycLk
+io10lStRojE9rWL/Tdu7+HtR+VG3yT2qug54lX179a6tMU/533WxCm80Db5hLVvYFfcEusH0JIMM
+PrK+CsNsUoD4AAAgAElEQVSIi2gUxWdAbxHpKSI5uP1q3wjy8wZwkR/9NBjYqqpr/CbtLQBEJB+3
+3+1XAWEu9ucXA68nmBejkfIpy2lJLn0oiit8AbkcSRe+ZBV7s23rX8OIlVpNT6paKSLXAO8AmcAT
+qrpARMZ490eAycDpuE3ZdwKX+uBFwKuuC4Is4DlVrdlA/W7gRRH5JW5T+FFJy5XRaFjLNpazkZPp
+Q0YC034G0oPZfMe6osokSmcYTYOo+ihUdTJOGQTeeyTgXIGrQ4RbChweJs6NwMmxCGs0PT5lOVlk
+MGA/y2bsHEALOtOaso5bUdQWDDSMGLCZ2UbaspsK5rOKfnQmj5yE4xtAV3bmK6VsSYJ0htF0MEVh
+pC0LWUMl1RyZYGuihr50IqMKvuC7pMRnGE0FUxRG2vIlpRSSn7QVYJuRxQHrsljAGvZgfRWGES2m
+KIy0ZCM7WMlmDqdrUvsTOqzJooIqFrA6aXEaRmPHFIWRlnxJKQL0p3NS422xLYP2FPAFK5Mar2E0
+ZkxRGGlHNco8SjmI9rQgN6lxC8IRdGUVW1jP9qTGbRiNFVMURtqxnI1sYzeH06VO4u9HZwThyx8s
+MGAYRihMURhpx1xWkks2h8Q5E7s2CmhGL9ozn1VUYzO1DaM2TFEYacVuKviaMg6jI1lEv0psrBxO
+F7azm2VsqLM0DKOxYIrCSCtq5k4cnqS5E+E4mAPIJYt5lNZpOobRGDBFYaQVbu5EAZ1oVafpZJHJ
+oXRiEWU2p8IwasEUhZE27Js70aVe1mLqTxcqqWYRa+o8LcNoyJiiMNKGeXU0dyIcXWhNW/L50sxP
+hhERUxRGWqAo81jFgXUwdyIcgtCfzqxgE5vZWS9pGkZDxBSFkRYsZyNb2cURdTR3Ihw1czXm2kxt
+wwiLKQojLfiSUpqRVWdzJ8LRijwOoj1zKaWa6npN2zAaClEpChEZJiKLRaRERG4J4S4iMsG7zxOR
+Af5+VxGZJiILRWSBiFwbEGa8iKwSkbn+OD152TIaEnuoYBFlHEanOp07EY4BdGU7uylhfb2nbRgN
+gVoVhYhkAg8Bw4G+wLki0jfI23Cgtz9GAw/7+5XADaraFxgMXB0U9m+qeoQ/9ttBz2g6zGMVFVQl
+bd+JWDmYIvLJsYUCDSMM0bQoBgElqrpUVfcCk4ARQX5GABPVMQtoLSIdVXWNqn4OoKrbgUVQT0Na
+jAaBosxmBZ1oRack7TsRK5lkcDhd+IZ1bGd3SmQwjHRG3HbXETyIjASGqerl/vpC4GhVvSbAz1vA
+3ao6019PBW5W1dkBfnoAM4DDVHWbiIwHLgW2ArNxLY/NIdIfjWulUFRUVDxp0qS4Mxsv5eXlFBQU
+1Hu6tdEY5NrSqop5R+7m4K9z6FCWXceShZdtV141nx29ix5Ls+n2XeLbriZLrlRjcsVGuso1dOjQ
+Oao6MO4IVDXiAYwE/hFwfSHwYJCft4DjA66nAgMDrguAOcDPAu4VAZm4Vs2dwBO1yVJcXKypYNq0
+aSlJtzYag1wv6+d6j76te7Wy7gQKIJJsT+lHOkH/o9VaXS+yBNIY3mV9YnLFBjBbaylfIx3RmJ5W
+wX7G4y7+XlR+RCQbeAX4l6r+O0BBrVXVKlWtBh7HmbiMJkQ5e1jEGg6nK9kp6MQOZgDd2MxOltpC
+gYaxH9Eois+A3iLSU0RygF8AbwT5eQO4yI9+GgxsVdU1IiLAP4FFqvrXwAAi0jHg8izgq7hzYTRI
+5rKSapRiuqVaFAB+RAcKaMbHLE21KIaRVmTV5kFVK0XkGuAdnKnoCVVdICJjvPsjwGTgdKAE2Inr
+ewA4Dmeqmi8ic/2936sb4XSviBwBKLAcuDJpuTLSnmqUOXxHD9pRSHrYdLPIZBA9+A+LKWMrHep4
+YULDaCjUqigAfME+OejeIwHnClwdItxMCL26m6peGJOkRqNiIWvYyi5O5UepFmU/iunOB5TwMUs5
+iyNTLY5hpAU2M9uodxTlA5bQngL60CHV4uxHHtkMoBtfeUVmGIYpCiMFLKKM9ZRzAr3rZTnxWDma
+HgDMYllqBTGMNMEUhVGvKMoMltCOfPrSsfYAKaA1zTmMjnzBd+ymItXiGEbKMUVh1CuLWcs6tnMC
+vchIw9ZEDcdwEHup4kO+TbUohpFyTFEY9UZNa6It+RxGp1SLE5EOtKQ/nZnFMturwmjymKIw6o35
+rKKMbb41kf6f3sn0IQPhXRamWhTDSCnp/281GgU72MM7LKQLrenXQNaFbEEuJ9CLr1lrs7WNJo0p
+CqNeeIeF7KGSM+mf1n0TwQymJ63JYwoLbWMjo8liisKoc75hLV+xmhPoRXtapFqcmMgik1Ppyzq2
+85Et7WE0UUxRGHXKHiqYzFe0p4DjOCjV4sTFIRRxKB2ZxmIzQRlNElMURp1RRTUv8Tnb2cMZ9E/J
+NqfJQBDOpD+FFPAKn7PFRkEZTQxTFEadoChv8CVL2cCZ9KMrbVItUkLkkMUoBlKN8hJzqKAq1SIZ
+Rr1hisKoE97ja+azmqEcwhEp2gs72bQjn59yBGvYxgvMZo/N2jaaCKYojKRSSRUlvffwMUs5iu4c
+30D7JcJxCEWcSX+WsZGn+JhttnCg0QQwRWEkjU3s4Ak+YnXnSgbTk9M4NC0X/UuUI+nKeRzFZnbx
+Tz5kJZtSLZJh1CmmKIyE2U0FH/ItjzGTLezi0PnNOJW+DWq+RKwcRHsu5RgE4Uk+5hU+t6U+jEZL
+VIpCRIaJyGIRKRGRW0K4i4hM8O7zRGRAbWFFpK2IvCsiS/xvw+7tbGIoyjq28x6LeID/MJWv6Uob
+ruQE2m2Maj+sBk8RLfkVJ3EivVnMWv7O+7zBl5SwjiqbnGc0Imr9R4tIJvAQ8F9AKfCZiLyhqoEL
+4AwHevvjaOBh4Ohawt4CTFXVu70CuQW4OXlZM5JFNcp2drORHWyknNVsZSnr2c4eBOhLJ47lQDo2
+wa1Dc8hiCAczgG7MYAlfsZq5lJJLNj1pRwdaUkRLCimgBblkN9AhwkbTJpqq3yCgRFWXAojIJGAE
+7LdS2ghgot8SdZaItBaRjkCPCGFHAEN8+KeB6dSRopjPKlawMe7wqw/eQznzkihRcqhNLg15T793
+U39VjVJNNdUolVRTQRWVVLGHSnayl11Bo3tyyeZACjmQQnrRnpbkJS9TDZSW5HIG/RhGX75lA4tY
+w0o2s4iy/fw1I4vm5NCMLLLJJIcsMhE2HbqbjXyBABnI9307Nca7wL6e+jToNdRvP1XUpVyD6MkB
+KVrZIBpF0RlYGXBdims11Oancy1hi1R1jT8vA4pCJS4io4HRAEVFRUyfPj0KkfdnWc+9rO1QGXO4
+GrRtNZv2lMYdvq6IWy71hY3/Fa05hIxq3FElZFVB6wqhfUU2OXuF5jszyNsp5OwVhG1sYxufh1jW
+ory8PK73VB/Ul2ytgdYIlZnN2VFQza7cavY2U/bmKBXZe9ibuYddmUp1JqhAZbMqduxcg4pX8F4b
+BJ6ngkb37dcxdSlX5aL1tN6SohapqkY8gJHAPwKuLwQeDPLzFnB8wPVUYGCksMCWoDg21yZLcXGx
+poJp06alJN3aaIpylZWV6QknnKAFBQV6/fXXxxy+IT2zcePG6fnnnx82TN++fcPmZ9q0adq5c+ew
+YS+++GIdO3ZsXHKlAyZXbACztZbyNdIRTWf2KthvxlQXfy8aP5HCrvXmKfzvuihkaVQ899xzDBw4
+kIKCAjp27Mjw4cOZOXMmAOPHj+eCCy74QRgRoaSk5PvrhQsX8pOf/IRWrVrRokULhg4dykcffbRf
+mL179zJ+/Hh69+5Nfn4+PXr04LLLLmP58uXf+3nrrbcYNGgQ+fn5tGvXjvPPP5/S0n01o6eeeorj
+jz8+Yn4uueQSsrKy2Lhxn5lvzJgxFBQUUFBQQE5ODtnZ2d9fDx8+nOXLlyMi39+rOV544YWQaTz2
+2GMUFhaybds27rvvvojy1MYll1zCbbfdllAcsfKHP/yBfv36kZWVxfjx4xOKa8GCBQwZMiQpctUH
+wd9ubQwZMoR//OMfSUl71KhRTJkyhT179tChQ4f93G688UZ69+5NixYt6NOnDxMnTkxKmo0Jccom
+ggeRLOAb4GRcIf8ZcJ6qLgjw82PgGuB0nGlpgqoOihRWRP4MbNR9ndltVfWmWmRZD6yIL6sJUQhJ
+Xw2uCOiAy882nJWhJdACZ6LrBDQDlgWFKwa+Avb48B2A9TjznXpZO+Oe+w4fpheQ7dPaCWQCbb3/
+DUAbXH/SCmCzd+/iZVkIVAHtfNyLw+QnAzjcx7kNQi61GipPOUA/YE6YeIPpDlQAq6P0H0zgu+wB
+7E0grnhoh5O/PbArIO1Q31i4byAaWgA9IazBvAfR5T2Z337gtxsNhwAbw6Qfq1yHAYtwz7ML7v9R
+QydgE7AbyMcNylnCvv9PLNRFWZEMDlHV+Ds4oml24BTAN8C3wFh/bwwwxp8LbnTTt8B8YGCksP5+
+O5yJagnwHk5RxN00qsuDBJttIeJrBZQD50TwMx54NsR9BXr5843A5BB+HgZm+PNTcAVS1zDpCE5B
+3BR0PwP3p/6jv74EmBlB3otw/VHXAruizROuwFIgK4rn9hSukN3rn98pXs5b/Pe1EXgx8FsCXsIp
+0a3ADOArf390UFxvBj/fgDTv8OdDcEr8Zh/nM/7+GcBcYAvwEdA/irw8C4yP9I355/UiMBHYDiwI
++m8tB07x53le1s045f47oDTA75HA5z6eF4BJNfmKlAdgtk/nRpzS2erD54bJVy/gfe9vA/CCvz/D
+P9sd/nn/HFdBeQtX0dnsz7t4/3fiKii7vf8ak3Uf4F2gEldpGRXFs24DLPXnVwH31uL/DeCGOP/b
+SS0rknUkKlfKM9AQjmS/fGCY/9DDFo5EpygqgEtD+Bnq/2R5wN3A+xHS6ePj7BnC7XbgY39+CZEV
+xVTgXlxLSYHiaPJEDIrC+38qqIC7FpiFqyU2Ax4Fng9wvwxXu24G3A/sDBdX8PMN9oNTFJXAPT6+
+PFwBvA7Xks4ELsYVrM1qyUe0imI3rrKVCfwJmBXgvpx9iuJu4ANcS7ErTsmXerccXGXgOlzLcqT/
+dmryFTYP7FMUn+Jq3m1xNfMxYfL1PDAWp8Bz2b/vMvjZtgPOBpr7d/QS8FqA+3Tg8oDrfFxl5FIv
+15E4ZdQ3jCwn4xRfOa5CsMX/7vDnJ4UIkwesAYbF+d9ulIrCZmanhnbABlWtbSjWKBHZEngEuWfh
+Pupg1uD+qG19WqH81FAYECZUPIUh7u+HiHTDKafnVHUtzvR0UW3hgtgQlNcfRRluDK6lWqqqe3CF
+60hv9kRVn1DV7QFueSKSyISPamCcqu5R1V24lsmjqvqJqlap6tM408rgBNIIZKaqTlbVKuAZnHkv
+FKOAO1V1k6quBCYEuA3GKYj7VbVCVV/GmYFriCYPE1R1tapuAt4EjggjRwXOPNhJVXer6sxwGVPV
+jar6iqruVNXtuFbESeH841o9y1X1SR/+C+AV4Jww8U9V1dbAa95PZ5zSK1TV1qr6fohgjwBfAu9E
+kKPJYYoiOh5LcnwbgcKawiwCL/oP+vsjyH0b0DFEuI64Am2zTyuUnxpq7Knh4onG3nohsEhV5/rr
+V4DzRCQ7irA1FAbldVGU4boDrwYo0kW41lSRiGSKyN0i8q2IbMMVEhCF8ovAelXdHZT+DUHKvCuu
+9h0L4b6xwIkYO4HcMN9NJ/Yfir4iyG2V+qplCPdIeaiRK1iOgjDy3oQzZ34qIgtE5LIw/hCR5iLy
+qIis8O9nBtDaT9QNRXfcRN4tQF//ez6uny5U/KXez7m4uVrrfBxrROSvIfz/GdeXMSroWcVCssuK
+ZJGQXKYookBVk/3yP8bV2H6aYDxvEro2NQpnMtqJ6/8ZJCJdwsSxGGd33y8eEcnAmQWmRiHHRcCB
+IlImImXAT3CF8elR5SIxVgLDg5RMrqquAs7DTew8Bdcv1MOHqZmZEKow2IkzhdQQXAgFh1mJq8kH
+pt9cVZ+PJRNJ+MbWsP8Iw25Bbp1FRMK4h81DrHKpapmqXqGqnYArgb+LSK8w3m/AdVgfraotgRP9
+/XDvZyXOjFojX2tVLVDVq8LI0gVn5n3PV7IeA6724a4P9Csit+NWmDhVVbfFkuegNNNSUSQqlymK
+FKCqW4H/Bh4SkZ/6mlW2iAwXkXtjiOp24FgRudOvndVCRH6NK7hv9mm9h+v8e1VEikUky/sbIyKX
++ZrTjcBtInKeiOSKSAfgH7hRWH8LSE+8e+BxDHAQbgb/Ef44DHiO2M1P8fAIcKeIdPcCtheREd6t
+BU4hb8QV/ncFhV0LHBh0by6uNZQpIsOIbAoBeBwYIyJH+zXP8kXkxyIScoSJf8+5uP9eln+GyZhF
+9SJwq4i08ZWCXwe4fYzrW/mNT/9nuPcVVx4iISLnBFRKNoOf/O8Ift4tcAMttohIW2BcUHTB/t8C
+DhaRC30+skXkqFrMlMW4TnyAAbi+jWCZb8VVKk5R1fiXcGjEmKJIEap6H3A9cBtu1MdK3BDj12KI
+YwlwPM5uvRxXczwbOE1VPwzwOhKYjButshXX0TkQ19pAVV/AmY+uwxWqC3GdescF/XGOxf2xA49f
+Aq+r6nxfmyxT1TLgAeAMXwBEwxYRKQ84rq89CPh03gCmiMh2XMd2zez/iTgTyyqfp1lBYf+JN2GI
+SM1zvxY4E9fZeT61vA9VnQ1cATyIKxhLcB3/4Xgc99zOxXX67sI9+0S5HZfXZcAUXH9GjYx7gZ95
+uTbhRhz9O4E8ROIo4BMRKce9l2vVL+GD6yN62j/vUbjBBXk48+Ys4O2guB7A9TdtFpEJvh/jVOAX
+uGG9ZewbWBCOYuBz35rqgxs5FsxduBZWScD39/tYM96YqXUehWEYhtG0sRaFYRiGERFTFIZhGEZE
+TFEYhmEYETFFYRiGYUSkQe1ZWVhYqD169Kj3dHfs2EF+fn69p1sbJlfspKtsJldsmFyxMWfOnA2q
+2j7uCBJZ/6O+D9uPYn9MrthJV9lMrtgwuWIDW+vJMGph3jz46U+hbVv6/+53cNddMCfaVc0NwzBF
+YTReSkvhF7+Aww+HadPgxz8mZ+NGGDsWBg6ECRNqj8MwDFMURiNl61Y49VR48034/e9h2TJ45hlm
+P/EErF8PP/sZXHstJLhLnmE0BRpUZ7ZhREVVFZx3HixZAlOmwNCh+7sXFsKkSXD++XDjjVBRAbfc
+khpZDaMBYIrCaHzceitMngx///sPlUQN2dnw3HPu99ZboU8f149hGMYPMNOT0bh4/nn485/hqqvc
+EYmsLHjqKejfH665BrZvrxcRDaOhYYrCaDxs2wa//S0MHgwPPBBdmOxsePRRWL0abrutbuUzjAaK
+KQqj8fCnP8G6dW40U3YMm+sNHuxaH//zP/DZZ7X7N4wmhikKo3GwfDn87W9wwQVw1FGxh7/rLujQ
+Aa68Eipr28rcMJoWCSkKERkmIotFpEREfjBsxO+WNcG7zxORAUHumSLyhYi8lYgchsGtt0JGhivw
+46FVK2eu+uILmDgxubIZRgMnbkXht298CLfPbF/gXBHpG+RtONDbH6OBh4PcrwUWxSuDYQDw8cdu
+uOuNN0LXrrX7D8fIkVBc7JSNtSoM43sSaVEMAkpUdam6rRYn4TayD2QEMNEvNzILaC0iHQH8vro/
+xu3NbBjx8/vfO7PRTTclFo8I/Pd/w7ffutFThmEAic2j6Izb57mGUvbtVRzJT2fc3s73AzfhNlgP
+i4iMxrVGKCoqYvr06QmIHB/l5eUpSbc2TC5osXAhxdOnU3LVVZTOnl2r/1pla9GCgQcdRMbYsXza
+qRNkZiZP2ETkShEmV2ykq1wJE+9qgsBI4B8B1xcCDwb5eQs4PuB6KjAQOAP4u783BHgrmjRt9dj9
+MblU9ayzVNu0Ud22LSrvUcn28suqoPrcc4nJFgP2LmPD5IoNUrh67Cog0CDcxd+Lxs9xwE9EZDnO
+ZPV/ROTZBGQxmiJffw2vvQZXXw0tIjZMY+Oss+DQQ+GOO6C6OnnxGkYDJRFF8RnQW0R6ikgO8Avg
+jSA/bwAX+dFPg4GtqrpGVW9V1S6q2sOH+4+qXpCALEZT5M9/htxc+M1vkhtvRgb84Q+wcKFTRIbR
+xIlbUahqJXAN8A5u5NKLqrpARMaIyBjvbTKwFCgBHgd+laC8huEoLYVnnoFf/hLax79xV1hGjoSe
+PeGvf01+3IbRwEhoUUBVnYxTBoH3Hgk4V+DqWuKYDkxPRA6jCXL//c4sdMMNdRN/ZqZrqVx3nZut
+Hc8kPsNoJNjMbKPhsX07PP44jBoFdbmH+mWXub6Pv/2t7tIwjAaAKQqj4fHUU/sWAKxLWraEyy+H
+l15ypi7DaKKYojAaFtXVbtG/Y46BQYPqPr3f/Mal+eCDdZ+WYaQppiiMhsXkyVBS4rYxrQ969HDD
+ZR97DHbsqJ80DSPNMEVhNCweeAA6d3Z7XtcX110HmzfbYoFGk8UUhdFwWLAA3nvP7UYXy34TiXLs
+sTBwoDN52QQ8owliisJoODzwAOTlwRVX1G+6Is7U9fXXMGVK/aZtGGmAKQqjYbBhg5tgd8EF0K5d
+/ac/apRboTbaLVYNoxFhisJoGDz6KOzeXfdDYsORkwO/+hW8/bZrWRhGE8IUhZH+7Nnjhqeedhr0
+Dd4bqx658kpo1sz1VRhGE8IUhZH+vPAClJW50Uep5IAD4Lzz4Omn3Sgow2gimKIw0htVt4RG375w
+6qmplsZ1au/c6eZVGEYTwRSFkd68/z7Mnev6JkRSLQ0cfjicfLLr1N6zJ9XSGEa9YIrCSG/+9jco
+LHSjndKFm2+GNWvgWdtry2gaJKQoRGSYiCwWkRIRuSWEu4jIBO8+T0QG+PtdRWSaiCwUkQUiUk/r
+MRgNigUL4I033GijvLxUS7OPU06BI490GyfZBDyjCRC3ohCRTOAhYDjQFzhXRIKHpAwHevtjNPCw
+v18J3KCqfYHBwNUhwhpNnbvugvz85O9glygirlWxeDG8/nqqpTGMOieRFsUgoERVl6rqXtze1yOC
+/IwAJvr9vWcBrUWko98O9XMAVd2O2yGvcwKyGI2NkhKYNAmuuio1E+xq4+yz4cAD4Z57XIe7YTRi
+ElEUnYGVAdel/LCwr9WPiPQAjgQ+SUAWo7Fx991uPae62sEuUbKy4MYb4ZNPYMaMVEtjGHVKQluh
+JoqIFACvAL9V1W1h/IzGma0oKipi+vTp9Segp7y8PCXp1kZjlavZ2rUc/dRTrDnzTJZ8/XVSZ0In
+85llHHggg9u0Ycd11/HlffclNCqrsb7LusLkqmdUNa4DOAZ4J+D6VuDWID+PAucGXC8GOvrzbOAd
+4Ppo0ywuLtZUMG3atJSkWxuNVq5rrlHNylJdsSIp8gSS9Gf2wAOqoDplSkLRNNp3WUeYXLEBzNY4
+y3pVTcj09BnQW0R6ikgO8AvgjSA/bwAX+dFPg4GtqrpGRAT4J7BIVf+agAxGY2PFCrcf9kUXQbdu
+qZamdq68Erp3h9//3voqjEZL3IpCVSuBa3CtgkXAi6q6QETGiMgY720ysBQoAR4HfuXvHwdcCPwf
+EZnrj9PjlcVoRPz+986EM25cqiWJjmbN4PbbYfZs+Pe/Uy2NYdQJCfVRqOpknDIIvPdIwLkCV4cI
+NxNIg2m2RlrxySfw3HMwdmzDaE3UcMEFcO+9Tu4RI1xHt2E0ImxmtpEeqML110NRkZuj0JDIzIQ7
+73TzKp5+OtXSGEbSMUVhpAcvvwwffQR33AEtWqRamtgZMcJtmXrLLW6TJcNoRJiiMFLPzp2uFdGv
+H1x6aaqliQ8ReOQR2LIlfed+GEacmKIwUs/NN8OyZW5F1szMVEsTP/36uRbFxIm2t7bRqDBFYaSW
+KVPc7nW//S0MHZpqaRJn7Fg45BA3bHbHjlRLYxhJwRSFkTo2bXKmph/9yC0A2BjIzXXzQJYvh1tv
+TbU0hpEUTFEYqUEVrr4a1q1z+zqk0zLiiXLCCW7F2//5H3jmmVRLYxgJY4rCSA1/+YtbHXb8eBgw
+INXSJJ+//AWGDIErroBPP021NIaREKYojPrnmWfgppvg5z9vvOaZ7Gx46SXo2BHOOsvtiGcYDRRT
+FEb98vbbcNllbt/pp5+GjEb8CRYWuo2Ntm6F00+HtWtTLZFhxEUj/pcaacf//q/b8KdfP7cuUrNm
+qZao7unfH155xc3aPv54WLo01RIZRsyYojDqHlU3qunMM93Q0cmToWXLVEtVf5x2GkydChs3wnHH
+wdy5qZbIMGLCFIVRt2ze7Poixo6Fc8+FmTOhQ4dUS1X/HHOMy3tWlju/5x6orEy1VIYRFaYojLqh
+ogImTIBevZzp5c9/dsNgmzdPtWSpo29fNwJq2DA3g3vQIJgzJ9VSGUatmKIwksumTXR67TU47DC4
+9lo39PWLL9z+0glsFdpo6NjR9c+8/DKsXg0DB8Ipp9Duo4+gujrV0hlGSBJSFCIyTEQWi0iJiNwS
+wl1EZIJ3nyciA6INazQQVGHJEnjySTjnHOjYkYMfeMC1HN56yy3R0b9/qqVML0Rcp/7XX8Of/gSL
+F9Nv7Fjo2RN+9Ss3Umr79lRLaRjfE/cOKyKSCTwE/BdQCnwmIm+o6sIAb8OB3v44GngYODrKsEaq
+2bsXysvdmkVbtsD69e4oLYWSEnfMm+dmVwMccABcdRWzDzuMgZdfnlrZGwKtWzsT1A03sOCOOzj0
+iy/cgoIPP+yUyYEHwqGHOpNVly7QqZPr32ndGlq1csux5+XZRklGnZPIFzYIKFHVpQAiMgkYAQQW
+9iOAiX6nu1ki0lpEOgI9ogibPMaPh3/9K+7gg3btSsslJr6XK9xezYH3a85V9z+qq6Gqat9RWekU
+xN69kU0hbdpA797O3n7ccW7ZikMOgYwMyqdPT1oemwTZ2awfOtRtqbp3r9uXY8YM+Oord0yeHLnj
+OxGLjfsAAAZzSURBVDPTrTGVne2URmamOzIy3FFj8hMJfR6Bo3ftSst+pbT/T9YF//wnnHhi3cRd
+C4kois7AyoDrUlyroTY/naMMC4CIjAZGAxQVFTE9jkKow65dtOnePeZwNVRWVrI9DWtt+8kV5k+v
+oe77QkJrfjMyUF+wVGdloVlZVGdnU52TQ1VuLlV5eVTl51PRujV7W7Vib7t2VAYPb1279vsJZeXl
+5XG9p/ogXWX7gVwnnrivUKiqInvrVppt3EjOpk1k7dhBZnk5WTt3krF3LxkVFWTs3YtUVSGVlUhV
+Fagi1dVIjbKvqRh4JFzlIojKigqy0v3bTyPqUq7vvvmGHSnqx0q/Jx2Eqj4GPAYwcOBAHTJkSOyR
+xBMmgOnTpxNXunWMyRU76SqbyRUbTVGuojqJNToSURSrgK4B1138vWj8ZEcR1jAMw0gDRKNsgv4g
+oEgW8A1wMq6Q/ww4T1UXBPj5MXANcDrOtDRBVQdFEzZMmuuBFXEJnBiFQDpuhGxyxU66ymZyxYbJ
+FRuHqGrcm9HH3aJQ1UoRuQZ4B8gEnlDVBSIyxrs/Avz/9s4t1IoqjOO/P5qmFqmZXVDwFBn4lFJh
+V0qjq3h6FBK0y0tEdKPQhKC3tKjektAisgwxMxEDu1FPaWIdO6amoZWmHSO6kKAG/x7WOrU5uKdz
+zs69pvp+MOw1a/awf8zMmm/PmjXfbCAFiT3AEeCOqnX78ZtnDda3FSRtsX1Jid+uIrwGTl3dwmtg
+hNfAkLSllfVbukdhewMpGDTWLW0oG7i3v+sGQRAE9SOezA6CIAgqiUDRP14oLdCE8Bo4dXULr4ER
+XgOjJa9B38wOgiAI/h/EFUUQBEFQSQSKIAiCoJIIFH9DXbLcSpoo6QNJX0jaLun+XD9W0juSdufP
+MYX8hkj6VNL6unjl3GKrJe2UtEPS5TXxejDvw25JKyWdWsJL0ouSeiR1N9Q19ZC0MLeDXZJubLPX
+U3k/bpP0pqTRdfBqWPawJEsaVxcvSfflbbZd0pKWvGzH1GQiPePxFXA+MAzoAqYUcjkXmJbLp5Me
+WJwCLAEW5PoFwOJCfg8BrwHr83xxL+Bl4O5cHgaMLu1FynO2FxiR51cB80t4AdcA04DuhroTeuRj
+rQsYDnTkdjGkjV43AENzeXFdvHL9RNIzYV8D4+rgBVwHvAsMz/PjW/GKK4pq/syQa/sY0Jvltu3Y
+Pmh7ay7/CuwgnXQ6SSdE8udt7XaTNAG4FVjWUF3US9IZpAa0HMD2Mds/lfbKDAVG5AwFI4HvSnjZ
+/gj4sU91M49O4HXbR23vJT1Ee1m7vGxvtN2bQvdjUtqf4l6ZZ4FHgcaRQaW97gGetH00f6enFa8I
+FNU0y35bFEmTgKnAJuBs2wfzokOUyR32HKmhNKa2LO3VARwGXspdYsskjSrtZfsA8DTwDXAQ+Nn2
+xtJeDTTzqFNbuBN4O5eLeknqBA7Y7uqzqPT2mgxcLWmTpA8lXdqKVwSKfxmSTgPeAB6w/UvjMqdr
+y7aOd5Y0C+ix3fTlzyW8SP/apwHP254K/EbqSinqlfv8O0mB7DxglKS5pb1ORF08GpG0CPgdGPwL
+Zv45l5HAY8DjpV1OwFBgLDAdeARYJQ3+XcQRKKrpT4bctiHpFFKQeNX2mlz9fX4ZFPmzp9n6J4kr
+gdmS9pG65mZIWlEDr/3Aftub8vxqUuAo7XU9sNf2YdvHgTXAFTXw6qWZR/G2IGk+MAu4PQex0l4X
+kAJ+Vz7+JwBbJZ1T2AvS8b/Gic2kq/1xg/WKQFHNJ8CFkjokDQPmAOtKiOR/A8uBHbafaVi0DpiX
+y/OAt9rpZXuh7Qm2J5G2z/u259bA6xDwraSLctVM0hsUi3qRupymSxqZ9+lM0v2m0l69NPNYB8yR
+NFxSB+n1xpvbJSXpJlL35mzbR/r4FvGy/bnt8bYn5eN/P2nAyaGSXpm1pBvaSJpMGszxw6C9TsZd
++P/SRMp++yVpdMCigh5XkboBtgGf5ekW4EzgPWA3aZTD2IKO1/LXqKfiXsDFwJa8zdYCY2ri9QSw
+E+gGXiGNQGm7F7CSdJ/kOOkkd1eVB7Aot4NdwM1t9tpD6lvvPfaX1sGrz/J95FFPpb1IgWFFPsa2
+AjNa8YoUHkEQBEEl0fUUBEEQVBKBIgiCIKgkAkUQBEFQSQSKIAiCoJIIFEEQBEElESiCIAiCSiJQ
+BEEQBJX8AceBcacrcAwzAAAAAElFTkSuQmCC
+"
+>
+</div>
+
+</div>
+
+<div class="output_area"><div class="prompt"></div>
+
+
+<div class="output_png output_subarea ">
+<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz
+AAALEgAACxIB0t1+/AAAIABJREFUeJzsnXd4VVW6h98vjZCEDoZOQFDEbmgiCozIgA0LYEFEvcrg
+jOWOOo6O3rFc2zjjjHp1xDKOXbCLiJUBFQUEFEV6lwCht1ACSb77x1qBw/Gc5CQ5Jyfle/PsJ3vv
+VfZv7bP3t+peS1QVwzAMo/aQEG8BhmEYRuViht8wDKOWYYbfMAyjlmGG3zAMo5Zhht8wDKOWYYbf
+MAyjlmGG36iSiEimiHwpIjtF5JF464klInK3iLxSgvs8Eekbxq2viOSUEPYFEbkvCjKNGoQZ/kpA
+RC4VkVkikici60TkIxHp7d1CvvQioiLSMeC4i4iMF5Ht3hhOFpFeQWFSfHxLRGSXiKwUkedFJCvA
+z9ki8q133ywir4pI6wD3K0RkainpeUFECkSkRcC5MT59eSKyT0T2Bxx/JCJZPk15QdtFYS4zCtgE
+1FfVm0u8waVQ2cZPRA4TkddFZK3/vb4WkR7ljU9Vj1bVKVGUGFOCn90I/E8RkaujdO03RGSAiNQR
+kdwgtzr+fdghIrkiclM0rlkdMcMfY/zD9SjwAJAJtAWeBM4tQxyHA18Dc4H2QEvgXeBTETk5wOtb
+Pt5LgQbA8cAs4HQfzxDgNa+nKXA0kA9MFZFGEWpJBy4EtgOXFZ9X1dGqmqGqGT6t44qPVXVQQBQN
+A85nqOq4MJdqB8zXKvCFoYgklTFIBjATyAYaAy8CH4pIRrS1Gb8gG/fMHwf8FOR2N9AJ92z1A24V
+kYGVqq6qoKq2xWjDGd88YGgJfu4GXglxXoGOfv9lYGIIP08BX/r9/sAeoE2Y6wiwCrg16HwC7gW5
+1x9fAUwtQe/lwGrgRuCnSNMEZPk0JUVw314A9gP7/P3r73XeBiwDNgNvAI0DwrwJ5OIypC+Bo/35
+UUFxfRB8fwOueZ/f7wvkAH/0cb7sz58NzAG2Ad8Ax5XhWdgBZJdwv94AXgJ2AvOArgHuK4H+fr+u
+17oVmA/8AcgJ8Hsi8J2PZxwwtjhdpaXBX+cW4Ed/H8cBqWE0dwS+8P424TJ6/L1XYJe/3xcBjYAJ
+wEavewLQ2vu/HygE9nr/T/jznYHPgC3AImBYBPe4EbDc718LPBzkvhYYEHB8LzC2Mm1CVdniLqAm
+b8BAoIASjB2RGf5c4MoQfvr5l6Yu8BDwRQnX6ezjbB/C7R5gmt+/gpIN/yTgYVztpSCUMQuVJspg
++L3/F4IM1o3AdKA1UAd4Gng9wP0qoJ53exSYEy6u4Psb7Adn+AuAv/j46uIM6gagB5AIjMQZyjoR
+pOUEb9galPAM7AXO9HE/CEwPcF/JQcP/EPAVribRBpdp53i3FFzm/nsgGRiCy/SK01ViGvz+t7ga
+ZWNgATA6jObXgTtwGXIq0LuEe9sEV0tM87/Rm8B7Ae5TgKsDjtNxhYsrgSSvexPQJYyW03EZWR4u
+g9/m/+/y+31wmYICmQHhLgTmxsM2xHuzpp7Y0gTYpKoFpfgbJiLbArcg96bAuhDh1uFevMb+WqH8
+BMZBGD/rAtzDIiJtcZnNa6q6HpcJXF5auCA2BaX1qAjDjQbuUNUcVc3HGcshxc0wqvq8qu4McDte
+RBqUUVsgRcBdqpqvqntwNYenVXWGqhaq6ou4ZrKeJUUiIvVxNbZ7VHV7CV6nqupEVS30/o8P428Y
+cL+qblHV1cDjAW49cQb/UVXdr6pv4ZqciokkDY+r6lpV3QJ8gMu0QrEf12TSUlX3qmrYfiFV3ayq
+b6vqblXdiSvl9wnnH1crWamq/1bVAlX9HngbGBom/kmq2hB4z/tphcvEmqpqQ1X9Atf8Bq6GUswO
+XEZU6zDDH1s2A00jaCN+wz+gB7Yg901AixDhWuAM1FZ/rVB+AuMgjJ8WAe4lMQJYoKpz/PGrwKUi
+khxB2GKaBqV1QYTh2gHvBmSMC3C1nUwRSRSRh0RkmYjswL30EEFmVgIbVXVv0PVvDsqc2+BKxyER
+kbo44zldVR8s5XqBHZG7gdQwz01LXGm4mFVBbmvUF2dDuEeShmAd4folbsU1H37rRx1dFcYfIpIm
+Ik+LyCr/+3wJNBSRxDBB2gE9gnQOB5qHiT/H+7kE15+ywcexTkT+7r3l+f/1A4I2wDWJ1TrM8MeW
+abgS1XkVjOdzQpd2huGaaHZ7P90DR+gEsQjXbn1IPCKSgKvyTopAx+VABz8iIhf4O864nhlRKirG
+amBQUKaRqqprcJ3Zg3F9AQ1wzUrgDBO4Kn4wu3FND8UEG5XgMKtxJe3A66ep6uuhxIpIHVwJNAf4
+TWRJjIh1OGNdTNsgt1YiImHcy5SGklDVXFW9RlVb4tL3zxJG8twMHAn0UNX6wGn+fLjfZzWu2TJQ
+Z4aqXhtGS2tcs+rnvtD0DPA7H+4m72cr7v4E1qSOx/Wn1DrM8McQX7X/M/CkiJznSz7JIjJIRB4u
+Q1T3AL1E5H4RaSwi9UTkepwh/qO/1ue4zrB3RSRbRJK8v9EicpUvBd4C3ClueGmqiDQHnsOVgv4R
+cD3x7oHbycDhQHdc9f8E4BjcKKGyNveUhzHA/SLSzgtsJiKDvVs9XAa7GWfMHwgKux7oEHRuDq62
+kuhHdpTU9ADwLDBaRHqII11EzhKRXzQV+BrQW7jO9pGqWhR5MkvlDeB2EWnkM/nrA9ym4fombvDP
+2QW436vMaSgNERkaUMjYijPexekMvt/1cPdim4g0Bu4Kii7Y/wTgCBEZ4dORLCLdSmkWzMZ1agOc
+hBvZE8xLuOe/kY/rGlzfTq3DDH+MUdVHgJuAO3GjGlYD1+FKg5HGsQTojSuhrMSVXC4Efq2qXwd4
+HQJMxI3G2I7r+OuKqw2gbujkCFzn32bcqJC6wCmqujkgnl64FzVw+y/gfVWd60t7uaqaCzwGnO1f
+6EjYJoeO4490LPVjwHjcENaduI7e4rHxL+GaNNb4NE0PCvsvoItvNii+7zcC5+A6/4ZTyu+hqrNw
+huIJnKFbiusID0UvXDv1AA5N76mRJbVE7sGldQXwKa4/oFjjPuACr2sLbkTNO+VMQ2l0A2aISB7u
+d7lRVZd7t7uBF/39HobrbK+La06cDnwcFNdjuP6arSLyuO8HGABcjBuJk8vBjvZwZAPf+dpOZ0KX
+5O/CjQpbhetQflhVg7XUCuTQ5kDDMAyjpmMlfsMwjFqGGX7DMIxahhl+wzCMWoYZfsMwjFpGWSef
+qhSaNm2qWVlZMYl7165dpKenxyTuaGEao0N10AjVQ6dpjA6x1Dh79uxNqtosIs+xnA+ivFt2drbG
+ismTJ8cs7mhhGqNDddCoWj10msboEEuNwCyN0MZWyRK/YRhlR4tg3Xew+hso2AtFhSACLbKh7SmQ
+nFZ6HEbtwAy/YVRzcmbAt4/Dsk9hd5gZlxKSoXVP6Doajr4IEsLNkmPUCszwG0Y1ZeMC+M8dsPBd
+qNsYOp0Jhw+E9v2gTgNn3AvyIWcarJgMiz+Ad4bDl/dB37uhyxAQG95RKzHDbxjVjKJCmHI3TH3A
+Nd/0vQd6/h7qhJhxJykVOg50W/8HYf7bMOUueOsiOHwAnP8ypB9W6Ukw4ozl94ZRjchbD68MgK/u
+g+NGwA3Loc+fQxv9YCQBjh4K186FM/8Jq76EMce72oBRuzDDbxjVhJwZ8PSJrvP23OfhvBcgPbLB
+e4eQkAjdroWrZ0BqQ3jpdJj2j9LDGTUHM/yGUQ1Y9SW83B+S6zqDfeKVFY8z8zi4ZiZ0uRA+vck1
+H9mcjbUDa+M3jCrO8s/h9XOhYTu4fBLUC7vmV9lJyYALx0LKKPjiHsjfCQP+5oaBGjUXM/yGUYVZ
++gmMHQxNj4QRn8WmIzYhEc591mUC0/8ORfth4GNm/GsyZvgNo4qydha8cQE06wKXf+6GbMYKSYCB
+j0JCkjP+GS3g1Ntjdz0jvkTUxi8iA0VkkYgsFZHbQrh3FpFpIpIvIreUJaxhGL9k63J47SxIz4Th
+H8XW6BcjAgP+CscOh//8Cea8EPtrGvGh1BK/iCQCTwJn4BaOniki41V1foC3LcANBC0qHmFYwzAC
+2LURXhnoxusP/wgyMivv2pIAg5+HXeth/NWQ0dx9A2DULCIp8XcHlqrqcnVreo4FBgd6UNUNqjoT
+2F/WsIZhHKRwn2ve2bEaLvnAte1XNokpMOwdN+rnrYtg08LK12DElkgMfyvcAuHF5PhzkVCRsIZR
+6/j49/DzVBj8b2hzcvx01KkHF78HiXVg7HlQkGeT+9QkqkznroiMAkYBZGZmMmXKlJhcJy8vL2Zx
+RwvTGB2qg0Y4qHPdxOYs/mdnWl/0M5uaL6cqSO90RwN+vPl4frr3CBLTplTpuX2qw+9dVTRGYvjX
+AG0Cjlv7c5EQcVhVfQZ4BqBr167at2/fCC9RNqZMmUKs4o4WpjE6VAeN4HR2SuvL1Megwxkw/NW2
+JCS2jbcsR19olQwTf5cJUzLpe2+8BYWnOvzeVUVjJPn3TKCTiLQXkRTgYmB8hPFXJKxh1Ar2b0/i
+jSHuw6wLX696UyZ3vRaaD1rHl/8LSz+OtxojGpRa4lfVAhG5DvgESASeV9V5IjLau48RkebALKA+
+UCQi/w10UdUdocLGKjGGUd3QIlj44FHsWg9XfQNpTeKt6JeIQMcbl6BrWvDOZTB6DtRvHW9VRkWI
+qI1fVScCE4POjQnYz8U140QU1jAMxzd/gy0zmjDoCWiZHW814UmsU8TQN+GZbHjrYhg5GRKT463K
+KC9VuKvGMGo2P0+FSX+CZn020O238VZTOk2OgHOehdVfuwVgjOqLGX7DiAO7N8Pbl0DDLDjiD4uq
+zbw4x1wM2aPhm7/Cko/ircYoL2b4DaOSUYXxV7lFVYa+AUnphfGWVCZ+/Xc47Fh473LYuTbeaozy
+YIbfMCqZb5+ARePhjL9Ci5PirabsJNeFIeNg/254d4SbWsKoXpjhN4xKJHcOfHYLHHE29Lgh3mrK
+T7OjYND/wYr/wNQH463GKCtm+A2jktiX5+a+SWvqpmSoLu364TjhSjjmErdy189T463GKAtm+A2j
+kpj4O9iyFC54zRn/6o4InD3GrQz29qWwZ0u8FRmRYobfMCqBH15y22l/hqw+8VYTPerUd+39ebkw
+/r9szd7qghl+w4gxmxbBh7+Fdn3gtDvjrSb6tOwK/R+Che/BzCfjrcaIBDP8hhFD9u9x7fpJqXDB
+q1VvHp5o0fP30Oks+PRmWDs73mqM0jDDbxgx5OP/hvU/wHkvQv0avBKFCJz3glsM/q1hsHdbvBUZ
+JWGG3zBixI+vwHfPwCm3wRFnxVtN7Elr6tr7t/8M719l7f1VGTP8hhEDNs6HCb+BdqfBr/433moq
+jza94PQHYeG7MOPxeKsxwmGG3zCiTP5OeGMIpGT4+fWrzDp3lcPJN8MR57gP1Wx8f9UkIsMvIgNF
+ZJGILBWR20K4i4g87t1/FJGTAtxWishcEZkjIrOiKd4wqhqq8P4VsHmRM/r1WsZbUeUjAue/5Cag
+e3OozedTFSnV8ItIIvAkMAjoAlwiIl2CvA0COvltFPBUkHs/VT1BVbtWXLJhVF2+egAWvOPm4Wn/
+q3iriR+pDeGidw/Wfgry463ICCSSEn93YKmqLlfVfcBYYHCQn8HAS+qYDjQUkRZR1moYVZrFH8Lk
+/4Fjh7vhjbWdw45xU1PkTIOPqvG8RDUR0VK63kVkCDBQVa/2xyOAHqp6XYCfCcBDqjrVH08C/qiq
+s0RkBbAdKASe9ouqh7rOKFxtgczMzOyxY8dWOHGhyMvLIyMjIyZxRwvTGB0qU+OulWl8f91J1G2x
+hxP+73sSU4siDlvT7+XyZzqw+vW2dLx+Ca0uWBNlZQep6fexNPr16zc74lYVVS1xA4YAzwUcjwCe
+CPIzAegdcDwJ6Or3W/n/hwE/AKeVds3s7GyNFZMnT45Z3NHCNEaHytK4c53qP9qp/jVTdevKsoev
+6feysED19cGq9ySoLvogepqCqen3sTSAWVqKbS3eImnqWQO0CThu7c9F5EdVi/9vAN7FNR0ZRo1g
+/254/VzYvREuneAmLDMOJSHRfbXc/ES3Xu+67+OtyIjE8M8EOolIexFJAS4Gxgf5GQ9c7kf39AS2
+q+o6EUkXkXoAIpIODAB+iqJ+w4gbRYXwznBYO8uN4GlpQxfCkpIOl4yHuo3h9bNh26p4K6rdlGr4
+VbUAuA74BFgAvKGq80RktIiM9t4mAsuBpcCzQPHS0ZnAVBH5AfgW+FBVP45yGgyj0tEi+GCUm5hs
+4KNw5LnxVlT1qdcSLv0Q9u2Cl/u7GT2N+BDRpyWqOhFn3APPjQnYV+B3IcItB46voEbDqFKoujl4
+5jwPfe6q3itpVTaZx8Lwj+DlM9x2xReuFmBULvblrmGUAVWY9Cf49v/cF6p97oq3oupHm5Ph4vdh
+82J4ZSDs3R5vRbUPM/yGESGq8Plt8PVDkD3afaRV3ZdPjBcdToehb0Lu9/DS6bB7U7wV1S7M8BtG
+BBQVujb9bx6Grr+Fs540o19RjjwXLnoPNs6DF/rY1A6ViRl+wyiFgnx4+xL4/jk49U448wkQe3Oi
+whFnuTb/7T/Dv091axIbscceX8MogZ3r4MV+MP9NGPCIm2LZSvrRJasvXD7JtfU/2x2WT4q3opqP
+GX7DCMOamfBsN7eC1tA34eSb4q2o5tKqO1zzrVul7JVfw4z/s4VcYokZfsMIQhVmjXFND4nJcNU3
+0GVIvFXVfBp1cPf6iLPh4xvcx3G2hGNsMMNvGAHsXAevnQUfXuuaIK6ZCc3tS5RKo049uOgd6Hcf
+zHsDxhwPK7+It6qahxl+w8B9ifvDS/DUsbByCgx6wnU6pjWNt7LahyTAaXfAf30DiXVcH8tHN1jp
+P5qY4TdqPWtmwr96wXsjoUkn+M130P131okbb1p1d79Ft9/CzCfh/46A7//tMmmjYpjhN2ot6+e6
+YZrPdYftq+C8F+Gqr6Fp53grM4pJyXDDZ6+Z5TLl8VfB0yfC/LcsA6gItWwZaKO2o+oWAJ/2CCx6
+3xmW3rdD79ugTv14qzPC0eJEuHIq/PQ6fHGvW8u3WRfo9Qc4ehgkp8VbYfXCDL9RK9i5Dua+Ct89
+5xZCT20Efe6GHtfbJGHVBRE49lI4+iL3XcWX98H7V7oJ844bAYXHZKB9rIkuEszwGzUSLYJdK9L5
++ls3dXLOdEChzSmudN9lqJsj3qh+JCTCMRe7DODnr2D20/DdM1C4ryvLHoDO58MR50CbXpBcN95q
+qyYRGX4RGQg8BiTilmF8KMhdvPuZwG7gClX9LpKwhhENdm+G3Dlu0q/V38CqL2HP5m4AtMiGfve6
+sfjWfl9zEIF2p7lt4OMw/uGF6PzOzBoDMx6DxBRo1QPangotTnJbwyyrEUAEhl9EEoEngTOAHGCm
+iIxX1fkB3gYBnfzWA3gK6BFhWMMoES1yQ/l2bXQTee1cAztyYMsy2LLEbYETfDVsD0eeA3sOW8ig
+6zrToE34uI2aQVoTaD4ol75/6Uz+TlcTWDkFVk6Gr/8CWuj8pdSDJke4rXFHqN8G6rd2XwynNXPD
+dxOT45qUSiGSEn93YKlfVAURGQsMBgKN92DgJb8gy3QRaSgiLYCsCMJGjXlvHvyBw7Fh/mH8VMVX
+/qlMjWE/i9cwfvz++vmZ/LDaHase/K9FfrSFuhkttcj9JkWFUFTgt/1QuN/9L8iHgr1QmA/7d7k1
+bPftgvwdkL/dzd+yZ0vo3zWtmRvp0eEMaHY0ND/BbenNnPuUKbk0aGNF/NpGnXrQ6Uy3gXu+1s+F
+dd/BhrmuoJAzHeaNCz0yqE4DSG0IqQ3cfkqG6zxOSYfEVEhKhaQ6rkaRkOwyioQkt0mia4qSBLcv
+4vcTAIHcRc2Zs8rXOiSg9uH/J6VClwtjf48iMfytgNUBxzm4Un1pflpFGBYAERkFjALIzMxkypQp
+EUg7lK8uP5WivYml+OrCgjLHXNlUB41HsbCCMUhyEQnJRSSk+P+pRSSmFpJQp4ik9AKS2hbQIL2Q
+pg32k+y3lMb51GmWT51m+0isezA32I970FbPOxh/Xl5euZ6jyqY66KwRGo+EtCMhDWgNFBUI+zan
+kL+xDvs21WH/9mT2bUumYHsyBbuSKNiVxN6diRRtTKRwTyKFexPR/ULRvgS3FSRAUVnbjTqzqATX
+5Eb72NDkmzLGWXaqTOeuqj4DPAPQtWtX7du3b5njOGYOh5RUQ/Htt9/SvXv3sgusRCpdY5hn95C2
+UDn0/IwZM+jRs8dBf770UlzKQXzJx5d+EpLccUKSKyUlJIL7jCT0pyTr169n6NChfD/5e0aNGsUj
+jzxS5mRNmTKF8jxHlc0VV1xBQUEBr7zySkj3o48+mieffDJkWqZMmcJll11GTk5O2Lhbt27Nfffd
+VyGN1eFexkOjFvnaa4Gv2Ra4c0WF7ri4NlxU6P5Pmzadnj16HjgPh9aoExJTaNShEtKgqiVuwMnA
+JwHHtwO3B/l5Grgk4HgR0CKSsKG27OxsjRWTJ0+OWdzhePXVVzU7O1vT09O1efPmOnDgQP3qq69U
+VfWuu+7S4cOH/0IjoEuWLDlwbt68eXrOOedo/fr1NSMjQ/v27atff/31IeHy8/P1rrvu0o4dO2pa
+Wpq2a9dOr7zySl2xYsUBPx988IF269ZN09LStHHjxnrppZfq6tWrD7j/+9//1lNOOaXE9IwcOVIT
+EhJ07dq1B8795je/0fT0dE1PT9fk5GRNSko6cDxw4EBdsWKFAgfOFW9jx44NeY17771Xzz//fC0q
+Kir55pZA8W89cuRIveOOO8odT1lZv369XnzxxdqiRQutX7++9urVS6dPnx7W/8iRI3/xDETK5MmT
+tVWrViXGHY20l+W9CX52S6NPnz767LPPlkPVoUyePFmHDh2qn3zyie7du1czMzMPcR83bpyefPLJ
+WrduXe3Tp0+Fr1dejbECmKWl2NbiTbSUuU9FJAlYDJwOrAFmApeq6rwAP2cB1+FG9fQAHlfV7pGE
+DXPNjcCqCPKt8tAUqMyF3jKB5rj07MDl8/WBerimr5ZAHWBFkMZ2wE9Avnc/CtgI5Po4muKa0hYD
+u3y4jkCyv9Zu3Eiqxt7/JqARrt9lFbDVu7f2WuYDhUATH3e4GmkCUDxt2VpgfQg/odKUAhwLzA4T
+bzDtcC04FVmXqfi3zgL2VTCuspCCu9dbcGko/q3mAqG+N+2Iu/crQriVRj2gPfBjGPcsopP2srw3
+2Rx8diPhSGBzGeIPR1Pcu7YA9/y1xr0fxdTDtXKk4t7BklpdYkUs7U87VW0Wkc9IcgecQV8MLAPu
+8OdGA6P9vuBG7yzDPdxdSwobz40y5IpRuFYDIA8YWoKfu4FXgjXijHVHf/wyMDFE2KeAL/1+f2AP
+0CbMdQRn8G8NOp+Ae0nv9cdXAFNL0Hs5rjn9Z+CnMqQpy6cpKYL79gLOYO7z96+/13mbf442A28A
+jQPCvInLFLcDXwJH+/s4KiiuD7z/A/c34Jr3+f2+uEz5jz7Ol/35s4E5wDbgG+C4MjwLO4DsMG5r
+fXpeAnYC84LeoZVAf79f12vdisus/wDkBPg9EfjOxzMOGFucrtLS4K9zCy4T2e7Dp4Z6b3CZ1Rfe
+3yZgnD//pb+3u/z9vgiXCU7AFVy2+v3W3v/9uExvr/f/hD/fGfgMl3kuAoZFcI+/B5b7/WuBh8P4
+uxqYUll2IPjdjsd1f6Ej3gJq8o0HBgIFlGDsiMzw5wJXhgjbz780dYGHgC9KuE5nH2f7EG73ANP8
+/hWUbPgnAQ9741EQypiFSVMWERp+7/+FIIN1IzAdV4qrg2tefD3A/Spcia4O8KjXNytUXP5caYa/
+APiLj68uzqBuwNVoE4GROENZJ4K0nOANW4Mw7mu9+5k+7geB6QHuKzlo+B8CvsLV5NrgMu0c75aC
+y9x/j6v5DcFlesXpKjENfv9bXI2tMa7kXFy4Czb8rwN34DLkVKB3Cfe2CXAhrl+1Hi6Tfi/AfQpw
+dcBxOq5wcSWuhH4iLnPpEub+nY7LyApxGfw2/3+X3+8T5L/WG36bpC22NAE2qWpBKf6Gici24g1n
+KAJpCqwLEW4d7sVr7K8Vyk9gHITxsy7APSwi0haX2byGM4yTcDWAsrApMK0iclSE4Ubjaow5qpqP
+y1yG+OZEVPV5Vd0Z4HY8zriVlyLgLlXNV9U9uJrD06o6Q1ULVfVFXFNGz5IiEZH6uBrbPaq6vQSv
+U1V1oqoWev/hVgEYBtyvqltUdTXweIBbT5zBf1RV96vqW7jm1WIiScPjqrpWVbcAH/DLZ7GY/bjm
+uJaquldVp4ZLmKpuVtW3VXW3qu7ElfL7hPOPq5WsVNV/q2qBqn4PvA0MDRP/JFVtiDPyQ3HNaiuB
+pqraUFVtRv8gaqPhf6YSr7UZaFpsnErgDf+ANvQP8G+D3DfhOsuDaYEzUFv9tUL5CYyDMH5aEFm7
+4whggarOwd3HV4FLRaQsn7w0DUyrqkY6crUd8G5A5rgAV8LLFJFEEXlIRJaJyA7cSw/wVhl0BbNR
+VfcGXf/moAy6Da50HBIRqYszntNV9cESrjUbV6srZjeQGua5acmhQ6RXBbmtUV+0DOEeSRqCdWT4
+/eD35lZc8+G3IjJPRK4KnTQQkTQReVpEVvnf50ugof/AMxTtcB+ABuocjmu/DxV/jvfTCHgRV6tp
+B6wTkb+H0xUnKtP+hKXWGX51w0Yri2m4EtV5ZQkUQuPnhC7tDMM10ez2frqLSOsw0S7CtVsfEo+I
+JOCq4ZEscX050EFEcoF7gb/jagpnRhC2oqwGBgVlGqmquga4FPdhYH9cv0qWDzPO/w81gmE3rumh
+mGCjEhxmNa6kHXj9NFV9PZRYEakDvIe7578pJW2RdniDq50FfovcNsitlZ9CJZR7mdIQSPAzqaq5
+qnqNqrbEpe+fItIxTPCbcR24PVS1PnCaP1+sM9S9/iJIZ4aqXhtGW2tcs+pnvuD0DPA7H65KrZRc
+yfYnLLWHdnnYAAAgAElEQVTO8Fcmvmr/Z+BJETnPl3ySRWSQiDxchqjuAXqJyP0i0lhE6onI9ThD
+/Ed/rc9xnWHviki2iCR5f6NF5CpfCrwFuFNELhWRVBFpDjyHG+Hwj4DriXcP3E4GDsd9yX2C347B
+NfuUtbmnPIwB7heRdl5gMxEZ7N3q4TLYzThj/kBQ2PVAh6Bzc3C1lUQ/n1RJTQ8AzwKjRaSHONJF
+5CwRqRfs0deA3sJ1to9UjerM8W8At4tII5/JXx/gNg3XBHeDf84uwP1eZU5DaYjI0IBCxlac8S5O
+Z/D9roe7F9tEpDFwV1B0wf4nAEeIyAifjmQR6VZKs2A2rlMb4CRcP1mw5kQRScX1GyT457oWTNDw
+S8zwxxhVfQS4CbgTN6phNW7o63tliGMJ0BvX7rsSV7K7EPi1qn4d4HUIMBFX0t2O6/jriqsNoKrj
+cM01v8cZyfm4jstTVHVzQDy9cC9q4PZfwPuqOteX9nJVNRc3Ad/Z/oWOhG0ikhewRVoiewwYD3wq
+IjtxHb3FX4G/hGvSWOPTND0o7L+ALr7ZoPi+3wicg2sXHk4pv4eqzgKuAZ7AGbqluI7wUPTCtVMP
+4ND0nhpZUkvkHlxaVwCf4voDijXuAy7wurbgRtS8U840lEY3YIaI5OF+lxvVT82C62N50d/vYbjO
+9rq45sTpwMdBcT2G66/ZKiKP+36AAcDFuI7vXA52tIcjG/jO13Y640ZGBTMC9yw/BZzq958tU6pr
+CKWO4zcMwzBqFlbiNwzDqGWY4TcMw6hlmOE3DMOoZZjhNwzDqGVUmWmZA2natKlmZWXFJO5du3aR
+nl61F1s1jdGhOmiE6qHTNEaHWGqcPXv2Jo3mJG3hNtxHE4tww8JuC+E+HDfh01zchFDHRxJvTZuW
+uayYxuhQHTSqVg+dpjE6VJVpmctd4pfI1tNdgZsgaauIDMJ9URdyBS7DqAooSi47WEgum8jjCDI5
+iuakVM3KsWGUi4o8zaWuxauqgWuIFc+saBhVkoXk8jHz2MFeBEinDgvI5SN+4mhacgZHkUqt/NDT
+qGGU+wMuERkCDFTVq/3xCNxcHNeF8X8L0LnYfwj3wDV3s8eOHVsuXaWRl5dHRkZG6R7jiGmMDmXR
+uP6wAhYdlU9GXgIt1yTRZHMSSfthR4Mi1jcvYH1mAfV2JnDMj6kkFZZ1ndXo6YwXpjE6xFJjv379
+Zqtq14g8R9omFLzhpgd4LuB4BH4RhRB+++FmU2wSSdzWxj853hJKpSZp/E5X6T06QV/UaZqv+0P6
+ma9r9V79UJ/Xr8P6KS816V7Gk9qukTK08VdkOOcaDp0lsLU/dwgichxuIrDBeuh8MIYRd34khw+Y
+y+E04xK6hW3LP4oWXMAJ5LCV15nJfgorWalhRI+KGP6ZQCcRaS8iKbgJlcYHehC3cMc7wAhVXRwi
+DsOIG9vZw0Tm0ZbGXEQ2yaWs23I0LTmPE1jFFr7AHmej+lLuzl1VLRCR64BPcCsdPa+q80RktHcf
+g5uSuAlurm6AAo20DcowYoiiTGAuijKY40mKcLGuY2nFSjYzjRUcTUta0CDGSg0j+lRojJqqTsRN
+Axx4bkzA/tW49S0No0rxAzksYyMD6UKjQ9ZjKZ3+HMUSNvABP3I1p5BgH8Ab1Qx7Yo1ax0728inz
+aUtjuh1YrCty6pLMII4mlx1MZ0X0BRpGjDHDb9Q6PmUBBRRxDschlG9oZmeacySZTGExW9gVZYWG
+EVvM8Bu1ilx2MI+19KQDTSj/nCmCcCbHIIh19BrVDjP8Rq1iMotIJYlev1iCt+zUI5VuZDGXtWxg
+ZxTUGUblYIbfqDWsZitL2EAvDo/a1Au96EAKSVbqN6oVZviNWoGi/IeFpJNC93J06IYjjRR60p4F
+5LKO7VGL1zBiiRl+o1awgs2sYgu96Rj1mTZ70p5UkplipX6jmmCG36jxKMoUFlOfVLJpG/X4U0mm
+Fx1YwgZy2Br1+A0j2pjhN2o8P7OVHLbSi8Mj/kK3rHQni7okM5WlMYnfMKKJGX6jxvMNy0gjhRMP
+mVMwuqSQRDeyWMwGG+FjVHnM8Bs1mvXsYAkb6E5WqZOwVZTuZJFEAtNYHtPrGEZFMcNv1Gi+YRkp
+JJZraoaykkYKJ9GWuaxhO3tifj3DKC9m+I0ay97UIn5iHSfRlrqVtGRiT9qjYHP4GFUaM/xGjSWn
+9X4EZ4wri4akcQwt+Y6f2cO+SruuYZQFM/xGjWQ3+8htUcBxtKI+dSv12r3owH4KmcmqSr2uYUSK
+GX6jRjKTVRQlwslRmJOnrGRSn44041tW2hKNRpXEDL9R43Cl7ZU03pRIM+rFRUMvDmc3+/iRnLhc
+3zBKwgy/UeP4gRx2s4/WqyunQzcU7WhMSxowjRUUoXHTYRihMMNv1CiKUKaxnJY0pMH2+D3egnAy
+HdjCLhazPm46DCMUZviNGsUictnKbnrRodyra0WLo2hOI9L4mmWolfqNKkSFDL+IDBSRRSKyVERu
+C+HeWUSmiUi+iNxSkWsZRmkoytcspxFpdKZ5vOWQQAI9ac8atrHaJm8zqhDlNvwikgg8CQwCugCX
+iEiXIG9bgBuAv5VboWFEyAo2s5ZtnEwHEuJc2i/mBNqQRopN3mZUKSpS4u8OLFXV5aq6DxgLDA70
+oKobVHUmsL8C1zGMiPiKJdSjDifQOt5SDpBMIj1oz1I22kItRpVBVMvX9igiQ4CBqnq1Px4B9FDV
+60L4vRvIU9WwJX8RGQWMAsjMzMweO3ZsuXSVRl5eHhkZGTGJO1qYxrKzvX4hP5y0lw5LU2id40bz
+VBWNBYnKjJN302hrIl3mpf7CvaroLAnTGB1iqbFfv36zVbVrJH6juxRRBVDVZ4BnALp27ap9+/aN
+yXWmTJlCrOKOFqax7LzGt6RRxLCO/Ujp6B7rqqQxkUV81WwpXfpmc1jQtwVVSWc4TGN0qCoaK9LU
+swYOmeC8tT9nGJXKOrazlI30ICvqyypGix60J5lEvra2fqMKUBHDPxPoJCLtRSQFuBgYHx1ZhhE5
+U1lKHb8QSlUljRS60o6fWMsWdsVbjlHLKbfhV9UC4DrgE2AB8IaqzhOR0SIyGkBEmotIDnATcKeI
+5IhI/WgINwxwpf0F5NKdLFIraerl8nIy7Ukgga+s1G/EmQrVi1V1IjAx6NyYgP1cqEJDLIwaxyQW
+UpfkuEzGVlYySKUb7ZjOCnrSnkysDGTEB/ty16i2LGcTy9lEbzpW+dJ+Mb3pSB2SmMTCeEsxajFm
++I1qiaL8h4XU96Xo6kIaKfSmI0vZyEo2x1uOUUsxw29USxaQy1q2048jSYrxIurRpjtZ1CeVz1lg
+c/gYccEMv1HtKKCQ/7CQw6jHsbSKt5wyk0wifTmCtWxnPuviLceohZjhN6odU1nGFnbTn6OqzJw8
+ZeU4WpNJPT5jAQWJVuo3Khcz/Ea1YiM7mcpSjqElHWkWbznlJgHhTI5lB3tZ2d4WZTcqFzP8RrVB
+USYwlxSS+DXBE8FWP9rQiO5ksbZVAavZEm85Ri3CDL9Rbfie1axmKwM4inTqxFtOVPgVR1InX/iA
+uRTYwuxGJWGG36gWbGU3n7GAdjTm+Br0TWAKSXRanMIm8uyLXqPSMMNvVHn2U8ibzEaAczk+7ksq
+RpvGW5I4ntZ8xVKWsiHecoxagBl+o0qjKBP5iVx2cB4n0Ii0eEuKCWdyDJnU5x2+t0ncjJhjht+o
+0szmZ34gh9PoxBFkxltOzEgmkYvIRhDGMYt8CuItyajBmOE3qiyLWM/HzKMjzehDp3jLiTkNSeNC
+TmITebzL9xRSFG9JRg3FDL9RJVlILm8ym+bU5wJOrHHt+uHoQFMGcjSL2cAbzLaRPkZMMMNvVDnm
+s443+Y6WNOAyelSbmTejRTeyOItjWMIGXmcm+6zZx4gyZviNKkMRylcs4W2+ozUNGV4LjX4x2bTj
+PI5nJZt5mRlsZXe8JRk1iKq5QKlR69jOHt5lDj+zhaNpyTkcW2XXz60sjqM1KSTxPj/wNF8xkC4c
+T+ta0+xlxI7a/WYZcWc/hcxiFV+xlCKKGMzxHEcrM26ezjSnOfV5nx8Yz48sYj39OJLDqBdvaUY1
+xgy/ERf2sp855PA1y9hFPu1pylkcQ2PS4y2tytGQNEbQk+ks5wuWsIj1HEVzetORFjSItzyjGlIh
+wy8iA4HHgETgOVV9KMhdvPuZwG7gClX9riLXNKovu8hnOZuYxzqWsZFCisiiCUM5ibY0jre8Kk0C
+Qi8O5wTaMIMVfMtKFpBLUzI4iuZ0pjmZ1K+201QblUu5Db+IJAJPAmcAOcBMERmvqvMDvA0COvmt
+B/CU/2/UUAopYjf72MletrKbLexmIztZw7YDHZT1SKUr7TiGlrSiYZwVVy/SSKEfR3IyHZjLGhaQ
+y1SW8hVLSSGRFjSgJQ1pTDoNqUsj0kinDikkWvOZcYCKlPi7A0tVdTmAiIwFBgOBhn8w8JKqKjBd
+RBqKSAtVjcmyQx/xU6kfvaw9Ip88fozF5aNGrDWGW/ajeBlA/cWxUnTgv1JEERuP28sqprGfIvZT
+wD4K2cv+kF+c1iOVVjQkm7a0oTGtaWhGqIKkkkw3suhGFrvIZxkbWcM21rCdb1n5i/cgASGNFFJI
+IoVEkkkkiQQS/ZaAkIAE/HHgP8C6I/PZWcIzWRV+zZrwbqeQxIBKmHK8Ioa/FbA64DiHX5bmQ/lp
+Bb9cb05ERgGjADIzM5kyZUqZBc3tsZuiUgaoauMituTnlDnuyiSuGvXQl1g04L93SygSiqSQbdu2
+kVAkJBZCaqGQXgDJ+5NJ3i+k7BPq7kkgda+QWCjATvaxk2WsZlklJSUvL69cz1FlEy2ddYGOwOGk
+kl9H2Zuq7E0tYn8KFCQp+5MLKUwsYF8i7ElUNAGKBDRBUeHABj7zD9jXhkVstfemwpSmMXm/kDKr
+EibqU9VybcAQXLt+8fEI4IkgPxOA3gHHk4CupcWdnZ2tsWLy5MkxiztamEbV3NxcPfXUUzUjI0Nv
+uummcsVRHe6jqurIkSN1+PDhYd27dOkSNi2TJ0/WVq1alRj3HXfcUVGJ1eJe1naNwCyN0H5X5AOu
+NUCbgOPW/lxZ/dR4XnvtNbp27UpGRgYtWrRg0KBBTJ06FYC7776byy677BdhRISlSw/Ozz5//nzO
+PfdcGjRoQL169ejXrx/ffPPNIWH27dvH3XffTadOnUhPTycrK4urrrqKlStXHvAzYcIEunfvTnp6
+Ok2aNGH48OHk5Bwsgbzwwgv07t27xPRcccUVnH766axbd7DiNnr0aDIyMsjIyCAlJYXk5OQDx4MG
+DWLlypWIyIFzxdu4ceNCXuOZZ56hadOm7Nixg0ceeaREPaVxxRVXcOedd1YojrKwYcMGLrnkElq2
+bEmDBg045ZRTmDFjRrnjmzdvHn379o2ewBgT/OyWRt++fXnuueeicu1hw4bx6aefkp+fT/PmzQ9x
+u+WWW+jUqRP16tWjc+fOvPTSS1G5ZnVEXEZRjoAiScBi4HScMZ8JXKqq8wL8nAVchxvV0wN4XFW7
+RxD3RmBVuYSVTlNgU4ziDkUm0ByXnh24mnN9oB6u6aslUAdYEaSxHfATkO/djwI2Ark+jqa4ZrPF
+cGAe345Asr/Wbtxoq8be/yagEZDl3bd699Zey3ygEGji414UJj0JwPF+fy2wPoSfUGlKAY4FZoeJ
+N5h2wH5/jfJS/FtnAfsqGFdZSMHd6y24NBT/VnMhZCdUR9y9XxHCrTTqAe0hbMNxFtFJe1nem2wO
+PruRcCSwuQzxh6Mp7l1bgHv+WuPej2Ja4n6TvUA6btDJEqjUebBjaX/aqWpkC1FHWjUIteEM+mJg
+GXCHPzcaGO33BTfyZxnuoS+1mSfWG2WoDkXhWg2APGBoCX7uBl4J1ogz1h398cvAxBBhnwK+9Pv9
+gT1AmzDXEZzBvzXofALuJb3XH18BTC1B7+W4fpufgZ/KkKYsn6akCO7bCziDuc/fv/5e523+WdoM
+vAE0DgjzJi5T3A58CRzt7+OooLg+8P4P3N+Aa97n9/viMuU/+jhf9ufPBuYA24BvgOPK8CzsALLD
+uK316XkJ2AnMC3xXgJVAf79f12vdisus/wDkBPg9EfjOxzMOGFucrtLS4K9zCy4T2e7Dp4Z6b3CZ
+1Rfe3yZgnD//pb+3u/z9vgiXCU7AFVy2+v3W3v/9uExvr/f/hD/fGfgMZ6gXAcMiuMffA8v9/rXA
+w6X4Hw/cXFn2INR9jNcWdwGVnuDKNfwDgQJKMHZEZvhzgStDhO3nX5q6wEPAFyVcp7OPs30It3uA
+aX7/Cko2/JOAh73xKAhlzMKkKYsIDb/3/0KQwboRmI4rxdUBngZeD3C/Clf6rQM86vXNChWXP1ea
+4S8A/uLjq4szqBtwNddEYCTOUNaJIC0neMPWIIz7Wu9+po/7QWB6gPtKDhr+h4CvcDW5NrhMO8e7
+peAy99/jan5DcJlecbpKTIPf/xZXMm6MKzkXF+KCDf/rwB24DDmVQ/vygu9tE+BCIM3/Rm8C7wW4
+TwGuDjhOxxUursQNQDkRl7l0CXP/TsdlZIW4DH6b/7/L7/cJEaYubpDJwMqyB6HuY7w2m6QttjQB
+NqlqadMrDhORbcUbzlAE0pQQI6H8uQTcS9okjJ/AOAjjZ12Ae1hEpC0us3kNZxgn4WoAZWFTYFpF
+5KgIw43G1SpzVDUfl7kM8U2OqOrzqrozwO14nHErL0XAXaqar6p7cDWHp1V1hqoWquqLuKaMniVF
+IiL1cTW2e1R1ewlep6rqRFUt9P6PD+NvGHC/qm5R1dXA4wFuPXEG/1FV3a+qb+GaYIuJJA2Pq+pa
+Vd0CfMAvn8Vi9uOa41qq6l5VnRouYaq6WVXfVtXdqroTV8rvE84/rlayUlX/raoFqvo98DYwNEz8
+k1S1Ic7ID8U1q60EmqpqQ1X9IkSwMcAPwCcl6Kix1EbD/0wlXmsz0LTYOJXAG/4Bbegf4N8GuW8C
+WoQI1wJnoLb6a4XyExgHYfy0ILJ2xxHAAlWdg7uPrwKXikhZptBsGphWVV0QYbh2wLsBmeMCXAkv
+U0QSReQhEVkmIjtwLz3AW2XQFcxGVd0bdP2bgzLoNrjScUhEpC7OeE5X1QdLuNZsXK2umN1Aapjn
+piWHDpFeFeS2Rn3RMoR7JGkI1pHh94Pfm1txzYffisg8EbkqdNJARNJE5GkRWeV/ny+Bhv4j0FC0
+A3oE6RyOa78PFX+O99MIeBFXq2kHrBORv4fw/1fgGFzzUfk6OctPZdqfsNQ6w6+qlXnjp+FKVOeV
+JVAIjZ8TurQzDNdEs9v76S4ircNEuwjXbn1IPCKSgKuGT4pA2uVABxHJBe4F/o6rKZwZQdiKshoY
+FJRppKrqGuBS3MeC/XH9Klk+TPGQoVAv9244ZAHfYKMSHGY1rqQdeP00VX09lFgRqQO8h7vnvykl
+bZF2eIOrnQWOlGsb5NbKT5USyr1MaQgk+JlU1VxVvUZVW+LS908R6Rgm+M24DtweqlofOM2fL9YZ
+6l5/EaQzQ1WvDaOtNa5Z9TNfcHoG+J0Pd1OgXxG5BzejwABV3VFauqNNJdufsNQ6w1+Z+Kr9n4En
+ReQ8X/JJFpFBIvJwGaK6B+glIveLSGMRqSci1+MM8R/9tT7HdYa9KyLZIpLk/Y0Wkat8yeYW4E4R
+uVREUkWkOfAcbpTRPwKuJ949cDsZOBz3xfYJfjsG1+xT1uae8jAGuF9E2nmBzURksHerh8tgN+OM
++QNBYdcDHYLOzcHVVhL9nFMlNT0APAuMFpEe4kgXkbNE5BfTZPoa0Fu4zvaRqhrNNRTfAG4XkUY+
+k78+wG0argnuBv+cXYD7vcqchtIQkaEBhYytOONdnM7g+10Pdy+2iUhj4K6g6IL9TwCOEJERPh3J
+ItKtlGbBbFynNsBJuH6yYM234woJ/VV1c6mJrMGY4Y8xqvoIcBNwJ25Uw2rcENf3yhDHEqA3rt13
+Ja5kdyHwa1X9OsDrEGAirqS7Hdfx1xVXG0BVx+Gaa36PM5LzcZ1cpwS9CL1wL2rg9l/A+6o615f2
+clU1FzcJ39n+hY6EbSKSF7DdVHoQ8NcZD3wqIjtxHb3FX4q/hGvSWOPTND0o7L+ALr7ZoPi+3wic
+g2sXHk4pv4eqzgKuAZ7AGbqluI7wUPTCtVMP4ND0nhpZUkvkHlxaVwCf4voDijXuAy7wurbgRtS8
+U840lEY3YIaI5OF+lxvVT9+C62N50d/vYbjO9rq45sTpwMdBcT2G66/ZKiKP+36AAcDFuI7vXA52
+tIcjG/jO13Y640ZGBfMArga0NOA3+VNZE14TKPc4fsMwDKN6YiV+wzCMWoYZfsMwjFqGGX7DMIxa
+hhl+wzCMWkaVXHO3adOmmpWVFZO4d+3aRXp61V7X1TRGh+qgEaqHTtMYHWKpcfbs2Zu0MiZpi9Vm
+8/FPjreEUjGN0aM66DSN0aGqzMdfJUv8Rg1l716YPBmmTYPp02HuXNi9251XhcMPhy5d4Oij4Zxz
+oGtXOOQjVMMwooEZfiP2LFoEzzwDL7wAW7ZAQgIcdxwMGgT160NqqjP8S5bAvHnw/vvwv/8LHTvC
+pZfC6NHQoqRpiAzDKAsRGX7/SftjuNkOn1PVh4LcOwP/xn0qfYeq/i3SsEYNZuVK+MMf4K23ICkJ
+zj8frroKeveGjIzw4bZtg3fegddeg/vug7/+FW68EW69FRo1qjT5hlFTKXVUj59B70ncxEZdgEtE
+JHgZ+C3ADcDfyhHWqGns2gX/8z/QuTN8+KHbX70a3ngDBg4s2egDNGzoMojPP4fFi+GCC+Avf4EO
+HWDMGFc7MAyj3EQynLM7sFRVl6ubC2QsbibEA6jqBlWdiZuju0xhjRrGnDlw0kmupH7BBa6Z5957
+oXnIGXVL5/DD4ZVXXLzZ2XDttXDmmbC2slZRNIyaR6lz9YjIENwqNVf74xG46VWvC+H3biCvuKmn
+jGFH4RaKIDMzM3vs2LEVSVdY8vLyyCitxBlnqqVGVVq99x6HP/UU+xs0YMHtt7PtpJOie1FVWr73
+Hoc//TRFKSks/OMf2XzKKZFrrKJUB52mMTrEUmO/fv1mq2rXiDyXNuwHN+PjcwHHI/DrYobwezdw
+S3nCBm42nHNyvCWUyiEad+9WHTZMFVTPOkt1w4bYXnzRItXsbFUR1QcfVC0qKl1jFaY66DSN0aGq
+DOeMpKlnDYcu/NDan4uEioQ1qgMbN8Lpp8Obb7p2+A8+gGaRfUNSbo44Ar76Ci6+GG6/HS6/3A0J
+NQwjIiIZ1TMT6CQi7XFG+2LcYgaRUJGwRlVnyRI3JHPNGmf4L7yw8q5dty68+qob83/nnfDzzy7T
+qV+/8jQYRjWlVMOvqgUich1uUeJE4HlVnScio737GL+S0yzcSk5FIvLfQBdV3REqbKwSY1Qe6StW
+wEUXQVER/Oc/cPLJlS9CBO64w3UAjxgB/fvDRx9BkyaVr8UwqhERjeNX1Ym4lZ0Cz40J2M/FNeNE
+FNao5vzwAyf893+7YZn/+Q8ceWR89Vx8MaSnw9Ch0LcvfPqpffBlGCVgs3MaZeO77+BXv6IwNRW+
++CL+Rr+Yc85x3wysWAH9+sH69fFWZBhVFjP8RuT89JNrTqlXjzmPPuqmVKhKnH46TJzo2vvPOIOk
+HTvircgwqiRm+I3IWLoUzjjDdapOnszeqtqUctppbq6fxYs57tZbYfv2eCsyjCqHGX6jdHJyXEl/
+/3747DNo3z7eikrmjDPgrbfIWLoUzj4b9uyJtyLDqFKY4TdKZssWGDDA/f/kEzdtcnXg7LNZ8Kc/
+wddfuxk+Cwrircgwqgxm+I3w7NkD554Ly5a5MfLZ2fFWVCY2/upX8Nhj8N578Nvf2uRuhuGx+fiN
+0BQWupLyN9+4WTX79Im3ovJx/fWQmwsPPOCGeN5zT7wVGUbcMcNv/BJVuOEGV1J+/HEYMiTeiirG
+ffc543/vvdCunZvy2TBqMWb4jV/yt7/BP//pFj65/vp4q6k4Im4e/5wcGDUKWrWCX/863qoMI25Y
+G79xKG++6Qz+RRfBgw/GW030SE52aTvmGFeDmTMn3ooMI26Y4TcO8s03bs6bU05x6+Mm1LDHo359
+93Vvw4Zw1lmuBmAYtZAa9mYb5WbpUjeCp21b17afmhpvRbGhVSv3de/OnW6M/86d8VZkGJWOGX4D
+Nm92yxmCM4pNm8ZXT6w59li3APxPP8GwYTbG36h1mOGv7ezdC+ed5+a3ef/9qjf/TqwYMACeego+
+/th1YNsYf6MWYaN6ajOqbmjj1Kkwdqxr269NXHON+zjtL3+BDh3gD3+ItyLDqBTM8Ndm7rgDXn/d
+fdx00UXxVhMfHngAVq50I5natXNNP4ZRwzHDX1t5+mk3XHPUKLjttniriR8JCW4E05o1bkRTy5bQ
+u3e8VRlGTLE2/trIxIlu7pozz4Qnn3QfONVmUlPdSKasLBg8GBYujLciw4gpERl+ERkoIotEZKmI
+/KJ4KI7HvfuPInJSgNtKEZkrInNEZFY0xRvlYOZM15xx4okwbhwkWaUPcOv0Tpzo7sfAgbB2bbwV
+GUbMKNXwi0gi8CQwCOgCXCIiwXPzDgI6+W0U8FSQez9VPUFVu1ZcslFuFi92pfzDDoMJE9yaucZB
+Dj/cGf9Nm9x9skVcjBpKJCX+7sBSVV2uqvuAscDgID+DgZfUMR1oKCJVdImmWsratW4Io4hbjLx5
+83grqppkZ8M778C8eXD++ZCfH29FhhF1REsZvywiQ4CBqnq1Px4B9FDV6wL8TAAeUtWp/ngS8EdV
+nSUiK4DtQCHwtKo+E+Y6o3C1BTIzM7PHjh1b4cSFIi8vj4wqXtKNtsakvDxOuPFGUtetY84//kFe
+FBZIr+n3MfPTTznqwQfZeOqpzL/rLjQxMcrqDlLT72VlUds19uvXb3bErSqqWuIGDAGeCzgeATwR
+5GWC45AAAAwuSURBVGcC0DvgeBLQ1e+38v8PA34ATivtmtnZ2RorJk+eHLO4o0VUNe7cqdqzp2py
+suqnn0Yt2lpxHx99VBVUR45ULSyMhqSQ1Ip7WQnUdo3ALC3FthZvkfTsrQHaBBy39uci8qOqxf83
+iMi7uKajLyPKlYyKUbyC1syZbmbKM86It6LqxY03unb+u+5yE7v94x82AsqoEUTSxj8T6CQi7UUk
+BbgYGB/kZzxwuR/d0xPYrqrrRCRdROoBiEg6MAD4KYr6jXDs2wdDh8KUKW6c+vnnx1tR9eR//gd+
+/3u3hOPtt9vUDkaNoNQSv6oWiMh1wCdAIvC8qs4TkdHefQwwETgTWArsBq70wTOBd8WVkpKA11T1
+46inwjiU/Hxn9D/80C1Actll8VZUfRFxC9Ps3eumdhBxX/tayd+oxkQ0iFtVJ+KMe+C5MQH7Cvwu
+RLjlwPEV1GiUhb174cIL3bDEJ56A3/wm3oqqPwkJ7l6qwkMPOaN///1m/I1qi329U5PYs8c16Xzy
+iZuSYdSoeCuqOSQkuK+cVd1UF3v2wCOP1LzFaoxagRn+msK2ba4jd+pU+Ne/bEHxWJCQ4NYiTk2F
+Rx919/zZZ+3rZ6PaYU9sTWDtWjfNwMKFbrbN2jrTZmWQkOBG9zRu7Eb7bNvm7nlNXbHMqJGY4a/u
+LFzojP7mza5dv3//eCuq+YjAn/8MjRrB/7d3/jFSVVcc/5z96bprpAayrFQEyapAU1ZcAYmYVbrF
+CgnVWMUaI62JbaLYRmvUrUlJlEhMq21qg0WrofUHMbFUUlGjxvUH/oJFlGVXlCBYgQXBUFh+uDsz
+p3+ct+6w7I+3dHbeG+Z8kpN58967M9+5eXPevfede+6tt8LMmZbkbcSIqJU5Tih8gDKXeeklmDbN
+xpvfeMOdfrZZsMDmR6xbB1OnQktL1IocJxTu+HMRVXjwQZg921IJr1kDkycPWMwZAq66ym66hw7B
+9OnW63KcmOOOP9c4cMDi8m+/3SJ4Vq+G0aOjVpXfTJkC779vN+HZs+GeeyCZjFqV4/SJO/5cYt06
+a9kvXw733QfPPgvl5VGrcsCWbXz3XbjxRovxr6+HtraoVTlOr7jjzwWSSZs9Om2aTdBqbLT1cj2G
+PF6UlcFjj8ETT9hNYOJEuzk7TsxwzxF3mptt7PiOO2xxkPXrYcaMqFU5/TF/vvXOxo2z0NprrrHF
+XRwnJrjjjysHD1rI4OTJsGWLxYqvWGFLBDrxZ/x4eOcdG/ZZsQLOPhuWLPGxfycWuOOPG6kUlS+/
+bI7i3nttfdyWFpg3z3PD5BpFRdDQAB9+CDU1tsB9ba1FATlOhLjjjwuplC35d/75jF+8GEaNsoid
+J5/0iUG5zsSJ8Nprtrj9nj1QV2cPf997L2plTp7ijj9qOjrg6adh0iTLqtneTmtDgzmF6dOjVudk
+ChHrvX36qSV3++gjuPBCmDWL0z74wPP8O1nFHX9UfPGFLfIxejRcdx0kEta6b21lV329R+ycqJSV
+wW232XOb+++HDRv4/p13Wq/g4Yct9YbjDDHuXbLJ3r2WLrmuzib7LFoEF1wAL74IGzfaDcAzPeYH
+FRVw112wdSstDQ02H2PBAqiqgiuvhOeeg/b2qFU6JyjuZYaSVAo+/tgc+6pVFuWRSsE558DChXD9
+9TB2bNQqnSgpKWF3fT0TFi2y4Z9ly+CppywSqLTU8i/NmQOXXgrV1f6A38kI7vgzyVdfWQRHU5Pl
+xV+92hbrBgvLvPtuG8evqfE/sHMskyZZDqYHHoC33oLnnzd74QU7fvrpcPHFliKithbOO896Do4z
+SEI5fhG5DPgTtubuY6q6uMdxCY5fjq25O19V14Upm3Ps3w/btpl9/jls2mTW2grbt3efN368Pcyb
+McNabVVV0Wl2couiIrjkErOHHoLNm+H1183efttSdoA1HsaMgQkTzMaNsx7k2LEWFXbyyZH+DCe+
+DOj4RaQQ+AtQD3wJrBGRlaqanoP2R0B1YFOBJcDUkGWHDlWbMJNIQGcndHRQsmePOewjR8wOHTI7
+eNDGVPfvt1b6vn02Jr93r4XgtbXBzp12XjqnnALnnmtd8UmTrGVfU2O52h3n/0XEhniqq7uX0mxr
+g7VrbXZwS4vZK69YhFg6w4ZZL2HEiG4bNszs1FPt2q2oMCsr67bS0m4rKYHiYhuidE4YwrT4pwCb
+g4XTEZHlwFwg3XnPBf4eLLr+nogME5EqYEyIspmjstKyVyaT3daD0AGSJSU2S7bLamth5EhruZ95
+ptmYMfadPmzjZJORI23cf86c7n2plK3EtmWLNWx27LAe6I4d1nBpbrbXffusITRI6sCu88LCo62g
+oPu1oMDOGci6PquLMNshmHr4sN24jrN8NpjSU2NPhg+3PE9DTBjHPwr4T9r7L7FW/UDnjApZFgAR
+uQm4CaCyspLGxsYQ0o7mrJkz7Q9QUIAWFKCFhd9aqrgYLSricDJJSUUFqdJSUiUlJEtKSJWVkTzp
+JJJlZSTKy0mWl5MqLu7/wjlyxFa/+uSTQesciPb29uP6/dnENWaOjOvsapj0hioFR45Q1N5O4eHD
+3dbRQcE335h1dlLQ2Yl0dFCQTCKJBJ2HDlFaWIikUkgyiaRSkErZ+1TKeteq3duABPuOeh9oSNfT
+G3Ic8xo6EwmKu6LiYjovIpFIcKCfyL1EeTmfZeGajc3DXVVdCiwFqK2t1bq6usF/SIgyjY2NHNdn
+ZxHXmBlyQSPkhk7XmBnCaByVBR1hHP924Iy0998N9oU5pzhEWcdxHCeLhHH8a4BqERmLOe15wE97
+nLMSuCUYw58K/FdVd4rIVyHKHkNTU9MeEdk2iN8xGIYDcc+R6xozQy5ohNzQ6Rozw1Bq7GN871gG
+dPyqmhCRW4CXsZDMx1V1o4j8Mjj+CLAKC+XcjIVz/qy/siG+c8iykonIWlWtHarPzwSuMTPkgkbI
+DZ2uMTPERWOoMX5VXYU59/R9j6RtK3Bz2LKO4zhOdHiuHsdxnDwjHx3/0qgFhMA1ZoZc0Ai5odM1
+ZoZYaBSNabyr4ziOMzTkY4vfcRwnr3HH7ziOk2fkjeMXkZ+IyEYRSYlIbY9jd4vIZhHZJCKzotKY
+jogsFJHtIrI+sMuj1tSFiFwW1NVmEbkraj29ISJbRWRDUHdro9YDICKPi8huEWlO23eaiLwiIp8F
+r5Fm9+tDY6yuRRE5Q0ReF5GW4D/9q2B/bOqyH42xqMu8GeMXkfFACvgr8BtVXRvsnwA8gyWjOx14
+FThbVY/N8JZFRGQh0K6qv49SR0+CjKufkpZxFbg2axlXQyIiW4FaVY3NhB4RuRhoxxIafi/Y9wDw
+taouDm6i31HVO2OmcSExuhaDBJBVqrpORE4BmoAfA/OJSV32o/FqYlCXedPiV9VWVd3Uy6G5wHJV
+/UZVP8cmoU3Jrrqc4ttsraraAXRlXHUGQFXfBL7usXsusCzYXoY5h8joQ2OsUNWdXet9qOoBoBVL
+cRObuuxHYyzIG8ffD31lFo0DC0Tk46D7HZcE/3Gur3QUeFVEmoLMr3GlUlV3BtttQGWUYvohjtci
+IjIGOA94n5jWZQ+NEIO6PKEcv4i8KiLNvVgsW6QD6F0CnAXUADuBP0QqNve4SFVrsEWCbg6GMGJN
+MAM+jmOvsbwWRaQCeA74taruTz8Wl7rsRWMs6jI2aZkzgar+4DiKhck+OiSE1SsijwL/HmI5YYms
+vgaDqm4PXneLyApsiOrNaFX1yi4RqQqSGlYBu6MW1BNV3dW1HZdrUUSKMYf6lKr+M9gdq7rsTWNc
+6vKEavEfJyuBeSJSGmQRrQY+iFhT18OhLq4Amvs6N8t8m61VREqwjKsrI9Z0FCJSHjxQQ0TKgR8S
+n/rryUrghmD7BuD5CLX0StyuRRER4G9Aq6o+mHYoNnXZl8a41GU+RfVcAfwZGAHsA9ar6qzg2G+B
+nwMJrEv2YmRCA0TkH1h3UIGtwC/Sxi8jJQhB+yPdGVcXRSzpKETkLGBF8LYIeDoOGkXkGWwVw+HA
+LuB3wL+AZ4HRwDbgalWN7OFqHxrriNG1KCIXAW8BG7BIPYAGbAw9FnXZj8ZriUFd5o3jdxzHcQwf
+6nEcx8kz3PE7juPkGe74Hcdx8gx3/I7jOHmGO37HcZw8wx2/4zhOnuGO33EcJ8/4Hzxkq2E0PmjK
+AAAAAElFTkSuQmCC
+"
+>
+</div>
+
+</div>
+
+<div class="output_area"><div class="prompt"></div>
+
+
+<div class="output_png output_subarea ">
+<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz
+AAALEgAACxIB0t1+/AAAIABJREFUeJztnXl8FdX5/99PFkhIwhqKbIIUFNG6QcFdbKmCVfGriFbc
+bSmtbalLW/uzrdqvttZWv9VqVbRVcQXXosW9RERFBaXKUhQRyhbZAwEChDy/P865MFzvMknukuQ+
+79drXnfunO1z5s79zJlzzsyIqmIYhmHkDnnZFmAYhmFkFjN+wzCMHMOM3zAMI8cw4zcMw8gxzPgN
+wzByDDN+wzCMHMOM32iSiEgXEZkuIptF5NZs60knInK9iDySIHyeiAyNEzZURJYnSPugiNyYAplG
+C8KMPwOIyHkiMktEqkVklYi8KCLH+rCYf3oRURHpG/g+QESmiEiVN8NpInJ0VJpWPr9PRWSLiCwR
+kb+LSO9AnFNF5D0fvk5EHhWRHoHwi0VkRpL6PCgitSLSNbDtHl+/ahHZISI7A99fFJHevk7VUcs5
+cYoZC6wF2qrqVQl3cBKyYX7+91kjIptE5N8iMrKheanqQapakUJ5aSX62A0Rv0JEvpuisieLyEki
+0lpEKqPCWvv/wyYRqRSRK1NRZnPEjD/N+IPrz8DvgC7AvsBdwOn1yOOrwFvAx8B+QDfgWeAVETkq
+EPUpn+95QDvgUGAW8E2fzyjgMa+nHDgI2A7MEJEOIbWUAGcBVcD5ke2qOk5VS1W11Nd1UuS7qo4I
+ZNE+sL1UVSfFKaoXMF+bwB2GIlLQgGQ/BXqoalvcSeyR4InSSBsDccf8IcDcqLDrgX64Y+tE4Oci
+Mjyj6poKqmpLmhac+VYDZyeIcz3wSIztCvT16w8DU2PEuRuY7teHAduAnnHKEWAp8POo7Xm4P8hv
+/feLgRkJ9F4ILAPGA3PD1gno7etUEGK/PQjsBHb4/TfM67wG+AxYB0wGOgbSPAlU4k5I04GD/Pax
+UXk9H71/A2Xe6NeHAsuBX/g8H/bbTwXmABuBt4FDQh4Hg4EaYHCC/TUZmAhsBuYBgwLhS4Bhfr3Y
+a90AzAd+BiwPxD0c+MDnMwl4IlKvZHXw5VwNfOT34ySgKI7mvsAbPt5a3Ikev+8V2OL39zlAB+AF
+YI3X/QLupAhwE7DL759q4E6/vT/wKrAeWAiMDrGfOwCL/foPgFuiwlcCJwW+/xZ4ItO+0BSWrAto
+yQswHKglgdkRzvgrgUtixDnR/2mKgZuBNxKU09/nuV+MsBuAd/z6xSQ2/teBW3BXL7XAwDB1oh7G
+7+M/GGVY44GZQA+gNXAv8Hgg/FKgzIf9GZgTL6/o/RsdB2f8tcAffH7FOENdDQwB8oGLcEbZOkEd
+XvCGpsBLQF6CY6AGOMXn/XtgZiB8CXuM/2bgTaAj0BN30l7uw1rhTu5XAIXAKNxJL1KvhHXw6+/h
+rig7AguAcXE0Pw5cizshFwHHJti3nXBXiW38b/Qk8FwgvAL4buB7Ca5xcQlQ4HWvBQbE0fJN3Ims
+GneC3+g/t/j1E3AnBQW6BNKdBXycTY/I1mJdPemlE7BWVWuTxBstIhuDS1R4ObAqRrpVuD9eR19W
+rDjBPIgTZ1UgPC4isi/uZPOYqn6BOwlcmCxdFGuj6npgyHTjgGtVdbmqbseZ5ahIN4yq/l1VNwfC
+DhWRdvXUFqQOuE5Vt6vqNtyVw72q+q6q7lLVh3DdZEfGy0BVT8UZ3SnAK6pal6C8Gao6VVV34a7w
+Do0TbzRwk6quV9VlwB2BsCNxhv9nVd2pqk8B7wfCw9ThDlVdqarrgeeBw+Lo2InrMummqjWqGndc
+SFXXqerTqrpVVTfjWvknxIuPuypZoqoPqGqtqn4IPA2cHSf/11W1PfCcj9MddxIrV9X2qvoGUOqj
+VwWSbsL9PjmHGX96WQeUh+gjnuwP0N1LVPhaIFb/cFecQW3wZSXqQ14bSBMrn7UxtkdzAbBAVef4
+748C54lIYYi0Ecqj6rogZLpewLOBE+MC3NVOFxHJF5GbReQzEdmE+9NDiJNZAtaoak1U+VdFnZx7
+4lrHcfEG/CJwkogkGtcJDkRuBYriHDfdcK3hCEujwlaob87GCA9Th2gdpcTm57juw/f8rKNL48RD
+RNqIyL0istT/PtOB9iKSHydJL2BIlM4xwD5x8l/u43wHeAh3VdMLWCUit/lo1f6zbSBpO1yXWM5h
+xp9e3sG1qM5oZD6vEbu1MxrXRbPVxxkcnKETxUJcv/Ve+YhIHu6S9/UQOi4E+vgZEZXAbThzPSVU
+LRrHMmBE1EmjSFVX4AazR+LGAtrhupXAGRO4S/xotuK6HiJEm0p0mmW4lnaw/Daq+nhI/QXAV0PG
+TcQqnFlH2DcqrLuISJzwxtZhN6paqarfU9VuwPeBvyaYyXMVcAAwRN1g9/F+e7zfZxmu2zKos1RV
+fxBHSw9ct+prvtE0Abjcp7vSx9mA2z/BK6lDceMpOYcZfxpR1SrgN8BdInKGb/kUisgIEbmlHlnd
+ABwtIjeJSEcRKRORH+OM+Be+rNdwg2HPishAESnw8caJyKW+FXg18Ctx00uLRGQf4H5cK+j/AuWJ
+Dw8uR+GMazDu8v8w4GDcLKH6dvc0hHuAm0SklxfYOTBFsgx3gl2HM/PfRaX9AugTtW0O7mol38/s
+SNT1AHAfME5EhoijRES+LSJf6ioQkf7+Ny72v/f5OLN7ox71jcdk4Jci0sGf5H8cCHsHNzbxE1/u
+mbjfq951SIaInB1oZGzAmXekKyt6f5fhJh5sFJGOwHVR2UXHfwHYX0Qu8PUoFJGvJ+kWHIgb1AY4
+AjezJ5qJuOO/g8/re7ixnZzDjD/NqOqtwJXAr3CzGpYBP8L1R4bN41PgWFwLZQmu5XIWcLKqvhWI
+OgqYipuNUYUb+BuEuxpA3dTJC3CDf+tws0KKgWNUdV0gn6Nxf9TgchnwD1X92Lf2KlW1ErgdONX/
+ocOwUfaexx92LvXtwBTcFNbNuIHeIT5sIq5LY4Wv08yotH8DBvhug8h+Hw+chhv8G0OS30NVZ+GM
+4k6c0S3CDYTHQnDjDKtxv/l44BxV/SBO/PpwA66unwOv4MYDIhp3AGd6XetxM2qeaWAdkvF14F0R
+qcb9LuNVdbEPux54yO/v0bjB9mJcd+JM3EB3kNtx4zUbROQOPw5wEnAubiZOJXsG2uMxEPjAX+30
+J3ZL/jrcrLCluAHlW1Q1WktOIHt3BxqGYRgtHWvxG4Zh5Bhm/IZhGDmGGb9hGEaOYcZvGIaRYzTk
+4VNpp7y8XHv37p1tGbvZsmULJSUl2ZbRIEx7djDtmae56obUaJ89e/ZaVe0cJm6TNP7evXsza1as
+abjZoaKigqFDh2ZbRoMw7dnBtGee5qobUqNdRJYmj+VoksZvGC2VT16AnVvhqydDUWOeJGQYjcCM
+3zAyxL8nwnMXufW8Atj3ODj2GvjqSdnVZeQeNrhrGBlg4RT4x6Ww3zfh4jfgqKth4+fw5NmweWW2
+1Rm5hhm/YaSZJW/Ak6Oh6xFwzrPQ63gY9nu44FWo3Q4vX5FthUauYcZvGGlkRzVMPhM69IExL0Lr
+wOPQOvaF466FeZNhUU4+McbIFmb8hpFGPnwAtq2H0/8GbTp9OfyYn0OnA+CfP3SDvoaRCcz4DSNN
+1O2Cd/8MPY6CnkfFjlPQGk691/X3T78xs/qM3MWM3zDSxH+egw2L4airEsfrfQIcdA68f5e1+o3M
+YMZvGGninVtd337/EO9fG/QD2L4J5j+Vfl2GYcZvGGmgal5blr8DQ34KefHeLBug1/FusPfDv6Vf
+m2GY8RtGGlg+uSdF7eHwS8LFF4HDLoWl02Hdp+nVZhhm/IaRYqorYe2McgZ+H1qVhk932EUgeTDn
+gfRpMwww4zeMlDP/KaBOOLSer6Av6wb9ToE5D0JdbTqUGYbDjN8wUszcJ6CkTzWdB9Q/7eGXQfUq
+u6HLSC9m/IaRQqqWwbK3oPOJqxuUvt+3oeQrNshrpBczfsNIIfOfdJ9fOXFNg9LnF8LXxsCnU6Gm
+KoXCDCOAGb9hpJC5T0DXgVDcfVuD8xgwCnbtcM/uN4x0YMZvGCliw2JY+b67C7cx9DjSDfQueDo1
+ugwjGjN+w0gRcye5z4NGNy4fyYP+Z8KiF93TPQ0j1ZjxG0aKmDfJPZCtfa/G5zXgLKitgU9fbHxe
+hhGNGb9hpIB1n8AX/258N0+EfY+DNp2tu8dID6GMX0SGi8hCEVkkItfECBcRucOHfyQiR/jtPUVk
+mojMF5F5IjI+1RUwjKbAgmfc54Fnpia/vHzo/z/w6T9hZ8PHiQ0jJkmNX0TygbuAEcAA4DsiEn1r
+ygign1/GAnf77bXAVao6ADgSuDxGWsNo9ix4BroPhnY9U5fngLNcH/9nr6QuT8OAcC3+wcAiVV2s
+qjuAJ4CRUXFGAhPVMRNoLyJdVXWVqn4AoKqbgQVA9xTqN4ysU/VfN5unf4pa+xF6nwhFHay7x0g9
+BSHidAeWBb4vB4aEiNMdWBXZICK9gcOBd2MVIiJjcVcLdOnShYqKihDSMkN1dXWT0lMfTHv6Wf50
+d6Afm7q9S0WF65dJlfZ2Qw5g3jOdaXfBW+QVaqPzC0Nz2e/RNFfdkHntYYy/0YhIKfA08FNV3RQr
+jqpOACYADBo0SIcOHZoJaaGoqKigKempD6Y9/Tx4PXzlYDjlgj3toVRp77YFHn8JetaeQL9vNTq7
+UDSX/R5Nc9UNmdcepqtnBRDsuezht4WKIyKFONN/VFWfabhUw2h6bFkN/30z9d08EfoMg9Zt7c1c
+RmoJY/zvA/1EZD8RaQWcC0yJijMFuNDP7jkSqFLVVSIiwN+ABap6W0qVG0YTYOEU0LrUzeaJpqA1
+7H8aLHwOdu1MTxlG7pHU+FW1FvgR8DJucHayqs4TkXEiMs5HmwosBhYB9wE/9NuPAS4AviEic/xy
+SqorYRjZYsEz7r26XQ5JXxkDRsG29bD0jfSVYeQWofr4VXUqztyD2+4JrCtweYx0MwBppEbDaJLU
+VMHi12DIePfqxHTx1ZOhsMR19/QZlr5yjNzB7tw1jAbyn2ehbqebb59OCoth/2/78naltywjNzDj
+N4wGMvdxaL8fdI+e3JwGDhzlB5JnpL8so+Vjxm8YDWDLalj8Ohz8nfR280ToNwIKimx2j5EazPgN
+owHMexJ0F3ztO5kpr1Up9B3h7uK17h6jsZjxG0YDmPu4u2nrKwdnrsyDznEvYl8yLXNlGi0TM37D
+qCcbl7oXqh+codZ+hP4joXU7mPNgZss1Wh5m/IZRT+Y+4T4zbfwFRa7MBc/Yi9iNxmHGbxj1ZO7j
+7r24HfbLfNmHXQy122D+k5kv22g5mPEbRj1YPc+9aSvTrf0I3QdDeX/490PZKd9oGZjxG0Y9mHU3
+5LfOnvGLwKEXu/n86xdlR4PR/DHjN4yQ1FS5gdWDz4WSztnTccj5IHkwx1r9RgMx4zeMkMx5EHZu
+gcE/zq6Ott3hqye57h6b0280BDN+wwiB1sH7d0KPo6DbwGyrgcO/C5uW7XnJu2HUBzN+wwjBopdd
+n3q2W/sR+p8BnQ6AN28CzcwbGY0WhBm/YYTgvb9A6T7pfxJnWPLy4dhr3AyjT6cmj28YQcz4DSMJ
+axfCohdh4DjIb5VtNXv42hho18ta/Ub9MeM3jCS8fo17EcqgccnjZpL8Qjjm57D8HXs7l1E/zPgN
+IwGLX4P/PAfHXQulXbKt5sscfqnrgnrzpmwrMZoTZvyGEYe6Wnjpp+6dukddkW01sSkogqOucieo
+RS9lW43RXDDjN4w4zLoH1syDk251BttUGfwj6HwQTLkMtm3IthqjOWDGbxgx2LoOpv3Gvdz8gJHZ
+VpOYgiI44yH3VrCXfpJtNUZzwIzfMKLYtROePhd2bIaT/y8zr1ZsLN0GwnG/go8esZu6jOSY8RtG
+AFV44fuuz/y0+zL7hq3Gctz/g65HOP0bl2ZbjdGUMeM3jABv3gRzHoDjf+Oefd+cyC+E/3nYXbE8
+cBys+yTbioymihm/YeAedjbjZpj2azjkAhh6fbYVNYzOA+DiCqitceb/xUfZVmQ0Rcz4jZyn6r/w
+8DB4/ZcwYJTr4mkO/frx2OcwuORNd5fxgye4fn+ty7YqoykRyvhFZLiILBSRRSJyTYxwEZE7fPhH
+InJE2LSGkS22rIa3boG7D4GVs+D0v8OoyVDQOtvKGk/5AXDJDOjYF569AO77Onw+LduqjKZCQbII
+IpIP3AV8C1gOvC8iU1R1fiDaCKCfX4YAdwNDQqY1jIygda7fe+Usdzfuwn+4m7T2+wacOgE6fjXb
+ClNL+17w3Xfh48fd1czEb7gneu5/Kux/GnQbBK1Ksq3SyAZJjR8YDCxS1cUAIvIEMBIImvdIYKKq
+KjBTRNqLSFegd4i0KWPek6BpeDHF6vlfYW5l6vPNBM1d+8erAhv8g8hU3Xrws67WLztd//aOLbBz
+K2xbD9Wr3LLuUzdFE6C4Ewz+CRxxmesXb6lIHhwyBg48071IZuFz8O4d8M6tLrysO3Ta3z32oagD
+FHeA5av35Z0P3f0B+YUuD8n33V8S9bm7oMBqlrrJmvuxPrfS7fP+Z6S/vDDG3x1YFvi+HNeqTxan
+e8i0AIjIWGAsQJcuXaioqAghbW/evPA46mry650uOQNYkIZcM0PuapfCOgrLdtKq0w5addxB+Tdr
+KDtgM2UHbKbNvluRfGXeamB1qvTuobq6ukHHcFo5EHocCPuMz2fjB+3ZsqSEbcvbsH55MZULW1Fb
+XUBtdQHU9eHzbGttEM3/WC/ssIOj27+d9tLCGH9GUNUJwASAQYMG6dChQ+udx8Fz2N0qTCXvvfce
+gwcPTn3GGaC5al+99gtGXzacJcsXceHosfz2GtdE3d2ajGp55hVAXqH7LCiCwjaQl58HtPZLZqmo
+qCDsMXz99dezaNEiHnnkkZjhBx10EHfddVfM/CoqKjj//PNZvnx5zLQXX3wxPXr04MYbb9w74JTY
+WrQO/vXqdI4Zcjy1290VVN0udyUdfaW1O03wP5fFx0M312Md9miX/FZ06jc0/QWqasIFOAp4OfD9
+l8Avo+LcC3wn8H0h0DVM2ljLwIEDtSkxbdq0RqV/9NFHdeDAgVpSUqL77LOPDh8+XN98801VVb3u
+uut0zJgxX0oD6Keffrr7+7x58/S0007Ttm3bamlpqQ4dOlTfeuutvdJs375dr7vuOu3bt6+2adNG
+e/XqpcOHD9fPP/98d5znn39ev/71r2ubNm20Y8eOet555+myZct2hz/wwAN6zDHHJKzPRRddpPn5
++bpy5crd277//e9rSUmJlpSUaGFhoRYUFOz+HtEA7N4WWZ544omYZfz2t7/V4447Tuvq6hJqCcNF
+F12k1157baPzqQ+HHXaYlpeXa1lZmR5yyCH63HPPxY0b7xgIw7Rp07R79+5xwxtS98Ye79HHbjJO
+OOEEve+++xpVpqrTffbZZ+vLL7+sNTU12qVLl73CJ02apEcddZQWFxfrCSec0OjyUklj97mqKjBL
+k3hrZBFN8gYHESkAPgG+CawA3gfOU9V5gTjfBn6Ea0cMAe5Q1cFh0sYpcw3QlO49LAfWNjBtF2Af
+XH024dpEbYEyXNdXN1yTNPrqeiAwF9juww8E1gCVPo9yXFfaJ8AWn6YvUOjL2grkA/sCm73+Drhx
+l6XABh/ew2uZD+wCOvm8F8apTx5wqNewCvgiRpxYdWoFfA2YHSffaHr5uiwKGT8RvYEdwMoU5BWW
+ruz5rUqA/XG/584YceMdA2EoA/YD4s3Y7039696Y4x32PnbDcACwrpFlgtO9D7AAtz974P4fEcpw
+vRxFuP9gvGM8GzR2nwP0UtXOoWKGOTvgDP0T4DPgWr9tHDDOrwtu9s5nwMfAoERpm9tCPc6kUena
+AdXA2QniXA88EmO7An39+sPA1Bhx7gam+/VhwDagZyzt/jdaCvw8KjwP9yf9rf9+MTAjgd4LceM2
+44G5YeuEMyAFCkLstwdxBlnn998wr/MafxytAyYDHQNpnsQZbRUwHTjIbx/r89rh83o+ev8GyrzR
+rw/FnZR/4fN82G8/FZgDbATeBg4Jc8zgJkjUAIMT7K/JwETcSXpe1H9oCTDMrxd7rRtwJ+ufAcsD
+cQ8HPvD5TAKeiNQrWR18OVfjGg1VPn1RHM19gTd8vLXAJL99ut+3W/z+PgfX4HgB13DZ4Nd7+Pg3
+4RocNT7+nX57f+BVYD3OoEeHOG4+BBb79R8At8SJ912gIhXekKqFBnpMg8vLdoWbw9LQHwUYDtSS
+wOwIZ/yVwCUx4pzo/zTFwM3AG/G0+z+SAvvFiHMD8I5fv5jExv86cAvuSqYWGBimTtTD+H38B4GV
+ge/jgZm4VlxrXPfi44HwS3EtutbAn4E5UXndGJV/MuOvBf7g8yvGGepq3BVtPnARzihbxztmvMHV
++LJeAvISHAM1uEZSPvB7YGYgfAl7jP9m4E2gI9ATd9Je7sNa4U7uV+CulkbhTnqReiWsg19/D/i3
+z38BvnEXQ/PjwLW4E3IRcGyCfdsJOAto43+jJ4HnAuEVwHcD30twjYtLcC30w3EnlwFxtHwTdyLb
+hTvBb/SfW/z6CVHxc9747c7d9NIJWKuqtUnijRaRjcElKrwc160SzSrcH6+jLytWnGAexImzKhAe
+FxHZF3eyeUxVv8CdBC5Mli6KtVF1PTBkunG4K8blqrodZ5ajfHciqvp3Vd0cCDtURNrVU1uQOuA6
+Vd2uqttwVw73quq7qrpLVR/CdWUcGS8DVT0VZ3SnAK+oJrx/doaqTlXVXbgrvEPjxBsN3KSq61V1
+GXBHIOxInOH/WVV3qupTuO7VCGHqcAewU1XXA88Dh8XRsRPXHddNVWtUdUa8iqnqOlV9WlW3qupm
+XCv/hHjxcVclS1T1AVWtVdUPgaeBs+Pk/7qqtseZ/Nm4LtAlQLmqtldVezFlFGb84ZjQwHTrgPKI
+OSVgsj9Ady9R4WtxfcbRdMUZ1AZfVqw4Ee1rA2li5ROmf/ECYIGqzvHfHwXOE5HCEGkjlEfVNdEM
+vA8D672AZwMnxgW4Fl4XEckXkZtF5DMR2YT700OIk1kC1qhqTVT5V0WdnHvi+udjMQHAG/CLwEki
+cnqC8oIz0LcCRXGOm27sPUV6aVTYCvVNyBjhYepQyZ5jZitQGkfvz3Hdh++JyDwRuTROPESkjYjc
+KyJL/e8zHWjvb/CMRS/cDaBBnWNw/fex8l/u43QAHsJd1fQCVonIbfF0NTEa6jENwow/BOqmmjaE
+d3AtqsbekvEasVs7o3FdNFt9nMEi0iMYIaB9Ia7feq98RCQPdxn+eggdFwJ9RKRSRCqB23DmGmdy
+YKMJGv8yYETUSaNIVVcA5+FuDByGG1fp7dNEJn/GmsGwFdf1ECHaVKLTLMO1tIPlt1HVx2MJj3HM
+FACpuDd4Fc6sI+wbFdZdZK9bqILhoeoQ5nhX1UpV/Z6qdgO+D/xVRPrGiX4VbgB3iKq2BY732+P9
+Pstw3ZZBnaWq+oM4WnrgulVf9Y2mCcDlPt2VyerSFGiExzQIM/40oqpVwG+Au0TkDN/yKRSRESJy
+Sz2yugE4WkRuEpGOIlImIj/GGfEvfFmv4QbDnhWRgSJS4OONE5FLfSvwauBXInKeiBSJyD7A/bgZ
+Dv8XKE98eHA5Cmdcg3GX/4cBBwOPUf/unoZwD3CTiPTyAjuLSOTdWGW4E+w6nJn/LirtF0CfqG1z
+cFcr+SIynMRdDwD3AeNEZIh/NlWJiHxbRMqiI4pIf/8bF/vf+3yc2aWiy2Ey8EsR6eBP8j8OhL2D
+G5v4iS/3TNzvVe86JENEzg40MjbgzDvSlRW9v8twEw82ikhH4Lqo7KLjvwDsLyIX+HoUisjXk3QL
+DsQNagMcgRtjidacLyJFuJNwnj+u63O12mIw408zqnorcCXwK9yshmW4qa/P1SOPT4Fjcf2+S3At
+u7OAk1X1rUDUUcBU3GyMKtzA3yDc1QCqOgnXXXMFziTn4wYuj1HVdYF8jsb9UYPLZcA/VPVj39qr
+VNVK4HbgVP+HDsNGEakOLGFbZLcDU4BXRGQzbqA3chf4RFyXxgpfp5lRaf8GDPDdBpH9Ph44Ddcv
+PIYkv4eqzgK+B9yJM7pFuIHwWAhunGE17jcfD5yjqh/EiV8fbsDV9XPgFdx4QETjDuBMr2s9bkbN
+M4Hw+tQhGV8H3hWRatzvMl79o1lwdX/I7+/RuMH2Ylx34kzcQHeQ23HjNRtE5A4/DnAScC5uGmol
+ewba4zEQ+MBf7fTHzYyK5gLcsXw3cJxfv69etW4hJJ3HbxiGYbQsrMVvGIaRY5jxG4Zh5Bhm/IZh
+GDmGGb9hGEaOEeqxzH662+2427zvV9Wbo8L7Aw/gplFdq6p/Cps2FuXl5dq7d++wdUg7W7ZsoaSk
+eb6qyLRnB9OeeZqrbkiN9tmzZ6/VVD2kDWfYn+Hm2bbCPcdjQFScr+Cmd90EXF2ftLGWlvZY5mxi
+2rODac88zVW3auYfyxymq2f3qxfVzROOvD4xePJYrarv8+VHziZNaxiZYB4rea+ZvlfKMFJNql69
+2Oi0koJXL6aLJvkavZCYdscHA7exvbWy9e3MvObB9nvmaa66IfPaW9SrF9NFfV6j19Qw7VDLLmbw
+MnUog4ceTRtaNV5cEmy/Z57mqhsyrz1MV88K9n4oVA+/LQyNSWsYKeELNlPnnwO2huosqzGM7BPG
++N8H+onIfiLSCvf8jCkh829MWsNICSvZ83qDtWzOohLDaBok7epR1VoR+RHwMm6Wzt9VdZ6IjPPh
+9/inPM7CPeWxTkR+ipu9sylW2nRVxjBisZIq2tCKWnZZi98wCNnHr6pTcU99DG67J7BeievGCZXW
+MDLJKqroRju2spM11uI3DLtz12jZ7KCWNWymG+3pTKm1+A0DM36jhVPJJhToRjs6U0o129nGjmzL
+MoysYsZvtGgiA7vO+N2LpqzVb+Q6ZvxGi2YlVbSliFKK6OzfG27Gb+Q6ZvxGi2YlVXSlHQDtKKaQ
+fBvgNXJx91ekAAAfMElEQVQeM36jxVLDTtazhW7e+AWxAV7DwIzfaMGsogqAbrTfva0zZdbiN3Ie
+M36jxbLSG3+kqwcIzOyJfpCsYeQOZvxGi2UlG+lAm70eyla+e4DXWv1G7mLGb7RY1rFl90yeCF/x
+UzrXWj+/kcOY8Rstlk1sox3Fe22zmT2GYcZvtFC2U0sNtbSNMn6b2WMYZvxGC2UT2wBoS9GXwsop
+ta4eI6cx4zdaJFXUAHypxQ/QnjZsooZd1GValmE0Ccz4jRZJpMXfLkaLP9Lvv8mfHAwj1zDjN1ok
+EVMvS2D8Vf7kYBi5hhm/0SKpYhtltCY/xiEeuQrYZMZv5Chm/EaLZBPbYvbvw55+f2vxG7mKGb/R
+ItlETcwZPQCF5FNCKzN+I2cx4zdaHIpSlaDFD66fv8oGd40cJZTxi8hwEVkoIotE5JoY4SIid/jw
+j0TkiEDYEhH5WETmiMisVIo3jFhsYye11H3prt0gbSm2Fr+RsxQkiyAi+cBdwLeA5cD7IjJFVecH
+oo0A+vllCHC3/4xwoqquTZlqw0hAVYKpnBHaUcxnrEFRBMmUNMNoEoRp8Q8GFqnqYlXdATwBjIyK
+MxKYqI6ZQHsR6ZpirYYRik0Jbt6K0I5idrLLHs9s5CRJW/xAd2BZ4Pty9m7Nx4vTHVgFKPCaiOwC
+7lXVCbEKEZGxwFiALl26UFFREUZ/Rqiurm5SeupDLmpf2W0n7A9z3/6AT3fEbtusLa+Fg+Ffs96k
+tDq/kUq/TC7u92zTXHVD5rWHMf7GcqyqrhCRrwCvish/VHV6dCR/QpgAMGjQIB06dGgGpIWjoqKC
+pqSnPuSi9tf4D4tZzElHnxi3G2clVcxnBn0HHUR/9mmk0i+Ti/s92zRX3ZB57WG6elYAPQPfe/ht
+oeKoauRzNfAsruvIMNKGm8NflLDvPtL/bwO8Ri4SxvjfB/qJyH4i0go4F5gSFWcKcKGf3XMkUKWq
+q0SkRETKAESkBDgJmJtC/YbxJZJN5QRoQysKyDPjN3KSpF09qlorIj8CXgbygb+r6jwRGefD7wGm
+AqcAi4CtwCU+eRfgWRGJlPWYqr6U8loYRoBN1LAvHRLGEcTP5TfjN3KPUH38qjoVZ+7BbfcE1hW4
+PEa6xcChjdRoGKGpQ9lMTdIWP9hNXEbuYnfuGi2KLWynDo37uIYg7Si2B7UZOYkZv9Gi2HPzVvIW
+f1uKqWY7texKtyzDaFKY8RstijA3b0Voby9kMXIUM36jRRHmcQ0R2tqUTiNHMeM3WhSbqKGQfIoo
+TBrX3sRl5Cpm/EaLIszNWxGsxW/kKmb8Rouiim2hBnYBCsinlNZm/EbOYcZvtBh2UcdqNlNOaeg0
+NpffyEXM+I0WQyWbqKWOfekYOo3dvWvkImb8Rovhv6wHoGeSxzUEae+N3+byG7mEGb/RYljGejrQ
+hrIQUzkj7EtHdlHHf9mQRmWG0bQw4zdaBIryXzbUq7UP0JtO5CEsZk2alBlG08OM32gRrGcLW9lR
+r/59gFYU0JMOLMZeCW3kDmb8Rosg0lVTX+MH+CqdqWQT1WxPtSzDaJKY8RstgmWsp5hCOlFS77R9
+6AzA59bqN3IEM36jRfBfNrAvHUPdsRtNV9pSTKF19xg5gxm/0eypZjvr2VLvgd0IgtCHcj5jDYqm
+WJ1hND3M+I1mzzI/f78h/fsR+tCZarazhupUyTKMJosZv9Hs+S8bKCCPrrRrcB59KAfgM5vWaeQA
+ZvxGs6aOOj5nLd1pT34jDud2FFNOqfXzGzlBqH+KiAwXkYUiskhErokRLiJyhw//SESOCJvWMBrK
+TnYxidmsZjOH0L3R+fWhnCWs4xO+SIE6w2i6JDV+EckH7gJGAAOA74jIgKhoI4B+fhkL3F2PtIZR
+b7ayg4nMZBGrOYWDOZx9G53nUfShnBKeYBZT+Zid9vweo4VSECLOYGCRqi4GEJEngJHA/ECckcBE
+VVVgpoi0F5GuQO8QaVPGi8xlF3Upz3fl/tup5qOU55sJmrv2zV67+vk229hJNdvZwBZ2sIuzGUh/
+9klJee0o5jKO4V8sZCaf8wmr6UgJRRRSRMFeU0WTTRpt7vu9OWpvrrphj/ZWFHAS6W8bhzH+7sCy
+wPflwJAQcbqHTAuAiIzFXS3QpUsXKioqQkjbm4+HbKUuDaMW2rGO9duXpz7jDNCitCsU1gqtdghl
+O4RuK1tRuek/VPKflJbbCvha+yJW9tjJ+sIN1BYotfkkd/tE2psRzVV7c9UNe7QX7hRazVqdgQJV
+Ey7AKOD+wPcLgDuj4rwAHBv4/jowKEzaWMvAgQO1KTFt2rRsS2gwzVV7ZWWlHnLIIVpaWqpXXnll
+tuXUm/rs9+uuu07HjBkTN3zAgAFx85s2bZp27949btqLLrpIr7322tBaInk2R5qrbtXUaAdmaRJv
+jSxh2scrgJ6B7z38tjBxwqRt8Tz22GMMGjSI0tJSunbtyogRI5gxYwYA119/Peeff/6X0ogIixYt
+2v19/vz5nH766bRr146ysjJOPPFE3n777b3S7Nixg+uvv55+/fpRUlJC7969+cMf/sCSJUt2x3nh
+hRcYPHgwJSUldOrUiTFjxrB8+Z5W0oMPPsixxx6bsD4XX3wxBQUFrFq1ave2cePGUVpaSmlpKa1a
+taKwsHD39xEjRrBkyRJEZPe2yDJp0qSYZUyYMIF27dqxadMmbr311oR6knHxxRfzq1/9qlF51Jcr
+rriCzp0707ZtWw499FD+8Y9/NDivefPmMXTo0NSJSzPRx24yhg4dyv3335+SskePHs0rr7zC9u3b
+2WefvbsAr776avr160dZWRn9+/dn4sSJKSmzOSLuRJEggkgB8AnwTZxpvw+cp6rzAnG+DfwIOAXX
+lXOHqg4OkzZOmWuApQ2tVBoohwbP8+sC7IOrzyZAgbZAGa7rqxvQGvg8Kt1AYC6w3YcfCKwBKn0e
+5biutE+ALT5NX6DQl7UVyAf2BTZ7/R1w4y5LgQ0+vIfXMh/YBXTyeS+MU5884FCvYRXEnAITq06t
+gK8Bs+PkG00vX5fwDhKf3sAOYGUK8gpLV/b8ViXA/rjfc2eMuPGOgTCUAftB3M7t3tS/7o053mHv
+YzcMBwDrGlkmON37AAtw+7MH7v8RoRuwHqjB/Sb9gE/Z8//JJo3d5wC9VLVzqJhhLgtwhv4J8Blw
+rd82Dhjn1wU3e+cz4GNgUKK0zW2hHpdQUenaAdXA2QniXA88EmO7An39+sPA1Bhx7gam+/VhwDag
+Zyzt/jdaCvw8KjwP9yf9rf9+MTAjgd4LceM244G5YeuEMyAFCkLstwdxBlnn998wr/MafxytAyYD
+HQNpnsQZbRUwHTjIbx/r89rh83o+ev8GyrzRrw/FnZR/4fN82G8/FZgDbATeBg4Jc8zgJkjUAIMT
+7K/JwETcSXpe1H9oCTDMrxd7rRtwJ+ufAcsDcQ8HPvD5TAKeiNQrWR18OVfjGg1VPn1RHM19gTd8
+vLXAJL99ut+3W/z+PgfX4HgB13DZ4Nd7+Pg34RocNT7+nX57f+BVnFEvBEaHOG4+BBb79R8AtySJ
+PwW4qjHekKqFBnpMg8vLdoWbw9LQHwUYDtSSwOwIZ/yVwCUx4pzo/zTFwM3AG/G0+z+SAvvFiHMD
+8I5fv5jExv86cAvuSqYWGBimTtTD+H38B4GVge/jgZm4Vlxr4F7g8UD4pbjWb2vgz8CcqLxujMo/
+mfHXAn/w+RXjDHU17oo2H7gIZ5St4x0z3uBqfFkvAXkJjoEaXCMpH/g9MDMQvoQ9xn8z8CbQEdeN
+Ohdv/LirqqXAFbirpVG4k16kXgnr4NffA/7t81+Ab9zF0Pw4cC3uhFzE3mN80fu2E3AW0Mb/Rk8C
+zwXCK4DvBr6X4BoXl+AmoByOO7kMiKPlm7gT2S7cCX6j/9zi10+IkaYYd8U6PBUe0diFDBu/3bmb
+XjoBa1W1Nkm80SKyMbhEhZfjDtJoVuH+eB19WbHiBPMgTpxVgfC4iMi+uJPNY6r6Be4kcGGydFGs
+jarrgSHTjcNdMS5X1e04sxzluxNR1b+r6uZA2KEi0vBnOLirjetUdbuqbsNdOdyrqu+q6i5VfQjX
+lXFkvAxU9VSc0Z0CvKKqieYaz1DVqaq6C3eFd2iceKOBm1R1vaouA+4IhB2JM/w/q+pOVX0K170a
+IUwd7gB2qup64HngsDg6duK647qpao2qzohXMVVdp6pPq+pWVd2Ma+WfEC8+7qpkiao+oKq1qvoh
+8DRwdpz8X1fV9jiTPxvXBboEKFfV9qr6Roxk9+BOcC8n0NFiMeMPx4QGplsHlEfMKQGT/QG6e4kK
+X4vrM46mK86gNviyYsWJaF8bSBMrnzD9ixcAC1R1jv/+KHCeiBSGSBuhPKquCxLE/TCw3gt4NnBi
+XIBr4XURkXwRuVlEPhORTbg/PYQ4mSVgjarWRJV/VdTJuSeu3zgWEwC8Ab8InCQipycorzKwvhUo
+inPcdGPvKdJLo8JWqG9CxggPU4dK9hwzW4HSOHp/jus+fE9E5onIpXHiISJtROReEVnqf5/pQHt/
+g2csegFDonSOgdg3bIjIch+nA/AQ7qqmF7BKRG6LEf+PwMG47qOm8jjWhnpMgzDjD4GqNvRHeQfX
+ojqjkRJeI3ZrZzSui2arjzNYRHoEIwS0L8T1W++Vj4jk4S7DXw+h40Kgj4hUikglcBvOXE+pR13q
+Q9D4lwEjok4aRaq6AjgPd2PgMNy4Sm+fJjLzPtafeyuu6yFCtKlEp1mGa2kHy2+jqo/HEh7jmCkA
+vhorbj1Zxd4z5faNCusuIhInPFQdwhzvqlqpqt9T1W7A94G/ikjfONGvwg3gDlHVtsDxfnu832cZ
+rtsyqLNUVX8QR0sPXLfqq77RNAG43Ke7MhhXRG7APUngJFXdlKyemaIRHtMgzPjTiKpWAb8B7hKR
+M3zLp1BERojILfXI6gbgaBG5SUQ6ikiZiPwYZ8S/8GW9hhsMe1ZEBopIgY83TkQu9S2bq4Ffich5
+IlIkIvsA9+NmGf1foDzx4cHlKJxxDcZd/h+GazU9Rv27exrCPcBNItLLC+wsIiN9WBnuBLsOZ+a/
+i0r7BdAnatsc3NVKvogMJ3HXA8B9wDgRGeKfTVUiIt8WkbLoiCLS3//Gxf73Ph9ndrG6HOrLZOCX
+ItLBn+R/HAh7Bzc28RNf7pm436vedUiGiJwdaGRswJl3pCsren+X4SYebBSRjsB1UdlFx38B2F9E
+LvD1KBSRryfpFhyIG9QGOAI3xhKt+Ze4RsIwVV2XtJItGDP+NKOqtwJXAr/CzWpYhpv6+lw98vgU
+OBbX77sE17I7CzhZVd8KRB0FTMXNxqjCDfwNwl0NoKqTcN01V+BMcj5ukOuYqD/C0bg/anC5DPiH
+qn7sW3uVqloJ3A6c6v/QYdgoItWB5crkScCXMwV4RUQ24wZ6I3eBT8R1aazwdZoZlfZvwADfbRDZ
+7+OB03D9wmNI8nuo6izge8CdOKNbhBsIj4XgxhlW437z8cA5qvpBnPj14QZcXT8HXsGNB0Q07gDO
+9LrW42bUPNPAOiTj68C7IlKN+13Gq380C67uD/n9PRo32F6M606ciRvoDnI7brxmg4jc4ccBTgLO
+xU1DrWTPQHs8BgIf+Kud/riZUdH8DncFtChw/P2/+la8JZB0Hr9hGIbRsrAWv2EYRo5hxm8YhpFj
+mPEbhmHkGGb8hmEYOUaY5/FnnPLycu3du3e2Zexmy5YtlJSUZFtGgzDt2cG0Z57mqhtSo3327Nlr
+NeRD2pqk8ffu3ZtZs740DTdrVFRUNKvH4gYx7dnBtGee5qobUqNdREI/0di6egyjPjz4IHzlK/C7
+30FtskcwGUbTxIzfMMJQVwe//CVccgkUFcG118Lxx8Nnn2VbmWHUGzN+w0hGTQ2cfTbcfDOMHevM
+/tFHYcECOPRQaELdkoYRBjN+w0jGX/4CzzwDt94K99wDhYVw3nnw0UfQpg38+tfZVmgY9cKM3zAS
+sWMH3H47nHgiXHklBB982bOn2/bSS9bqN5oVZvyGkYjHH4cVK+BnP4sd/sMfQvv2cNNNmdVlGI3A
+jN8w4qEKf/oTHHwwDB8eO07btjB+PDz3HHz8cWb1GUYDMeM3jHi89BLMneta+3u92ySKn/wESkvh
+97/PnDbDaARm/IYRjz/+Ebp3h3PPTRyvY0e4/HKYNAk++SQz2gyjEZjxG0YsZs+GadPgpz+FVq2S
+x7/iCsjLg/vvT782w2gkZvyGEYuJE6F1a/je98LF79IFvvUtePJJNzZgGE0YM37DiKauDp5+GkaM
+gHbtwqcbPRqWLLGpnUaTx4zfMKKZOdNN4Rw1qn7pRo50N3dNnpweXYaRIsz4DSOap55y/fqnnVa/
+dB06wEknOeO37h6jCWPGbxhB6uqc8Z98spujX19Gj4b//pe2CxakXpthpAgzfsMI8v77sGxZ/bt5
+Ipx+OrRqRedp01KryzBSiBm/YQR56inXT3/66Q1L3749nHwynd94w109GEYTxIzfMCKouumY3/qW
+M/CGMno0RWvWuEFiw2iCmPEbRoTZs2Hp0oZ380Q4/XTqCgvd1YNhNEHM+A0jwjPPQH6+m5bZGNq2
+ZcPhh8OUKTa7x2iSmPEbRoQpU+CEE9yzdxrJuqOPdm/q+s9/UiDMMFKLGb9hgDPpefMaPqgbxbqj
+j3YrU6akJD/DSCVm/IYB8Pzz7rO+N23FYXvnznDEEWb8RpMklPGLyHARWSgii0TkmhjhIiJ3+PCP
+ROQIv72niEwTkfkiMk9Exqe6AoaREqZMcS9c6dMndXmefjq88w6sXp26PA0jBSQ1fhHJB+4CRgAD
+gO+IyICoaCOAfn4ZC9ztt9cCV6nqAOBI4PIYaQ0ju2zYANOnp6ybZzenn+4Gd6dOTW2+htFIwrT4
+BwOLVHWxqu4AngCipz2MBCaqYybQXkS6quoqVf0AQFU3AwuA7inUbxiN58UXYdeulHXz7Oaww6BH
+D+vuMZocBSHidAeWBb4vB4aEiNMdWBXZICK9gcOBd2MVIiJjcVcLdOnShYqKihDSMkN1dXWT0lMf
+THtyBtx/P+07dODtrVshReVVV1dT8cYb9Bs4kH1efJG3XnmFujAvdGkCNNdjprnqhixoV9WECzAK
+uD/w/QLgzqg4LwDHBr6/DgwKfC8FZgNnJitPVRk4cKA2JaZNm5ZtCQ3GtCdh+3bVtm1VL7sspdnu
+1v7SS6qg+s9/pjT/dNJcj5nmqls1NdqBWRrCX1U1VFfPCqBn4HsPvy1UHBEpBJ4GHlXVZ0KfkQwj
+E0yfDps2pb5/P8LQoe5F7M89l578DaMBhDH+94F+IrKfiLQCzgWiOy2nABf62T1HAlWqukpEBPgb
+sEBVb0upcsNIBc89B0VFMGxYevJv3RpOPdWVU1ubnjIMo54kNX5VrQV+BLyMG5ydrKrzRGSciIzz
+0aYCi4FFwH3AD/32Y3BdQ98QkTl+OSXVlTCMBrFrl3vF4imnQJs26Stn1ChYswbefDN9ZRhGPQgz
+uIuqTsWZe3DbPYF1BS6PkW4GII3UaBjp4a23oLISzj47veWMGOFOLE89BSeemN6yDCMEdueukbs8
++aTr5jn11PSW06aNM/9nnrFn9BtNAjN+Izepq3PdPCNGuMHXdDNqlLu6ePvt9JdlGEkw4zdyk7fe
+glWr0t/NE+Hb33YDvfaMfqMJYMZv5CZPPrlnxk0mKCuD4cPdVYZ19xhZxozfyD2C3TxlZZkr96yz
+YPlyeO+9zJVpGDEw4zdyj7ffhpUrM9fNE+G009yL3J98MrPlGkYUZvxG7jFpkuvmSfVD2ZLRvr3r
+Wnr4YdixI7NlG0YAM34jt9iyxRnvmWdmtpsnwtix7mauZ5/NfNmG4THjN3KLxx+Hqir44Q+Tx00H
+J50EvXvDvfdmp3zDwIzfyCVU4a9/ha99DY45Jjsa8vLge9+DadPgk0+yo8HIecz4jdzhvffgww9d
+a1+y+CSRSy6BggK4777saTByGjN+I3f461/dXbpjxmRXR9eu7jHQDz4I27dnV4uRk5jxG7nB2rVu
+Ns+FF2ZnUDea73/faXrGXlFhZB4zfiM3eOAB17r+wQ+yrcQxbBj06QO33WZ38hoZx4zfaPlUVcEf
+/wjf+AYcfHC21Tjy8uA3v4FZs2Dy5GyrMXIMM36j5fO//+u6Vf74x2wr2Zvzz4fDDoNrroGammyr
+MXIIM36jZbNwIdx+O1x2GRxxRLbV7E1+PvzpT7B0Kdx5Z7bVGDmEGb/RsrnqKiguhhtvzLaS2Hzz
+m+7VjzfeCOvWZVuNkSOY8Rstl5degn/+0/Wld+mSbTXxueUW2LwZfv3rbCsxcgQzfqNlsmKFey5O
+377wk59kW01iDjoIfvxjuPtuN7ffMNJMqJetG0azYuNG99KTjRvhjTegVatsK0rOH/8I8+a5k1Xv
+3jB0aLYVGS0Ya/EbLYuaGjjjDDeo+8wzcPjh2VYUjshz+vv2dU8Otef4GGnEjN9oOXzxhTPNN95w
+XSbDhmVbUf1o396NSRQUwIknwmuvZVuR0UIx4zeaP6rwyCMwYAC8/rp7Js9552VbVcPYbz949VVo
+2xa+9S03PrF1a7ZVGS2MUMYvIsNFZKGILBKRa2KEi4jc4cM/EpEjwqY1jAZTVQUTJ7rW8QUXwAEH
+wJw5TeexDA3l0EPhgw+c6f/lL9C/P1x3HSxalG1lRgsh6eCuiOQDdwHfApYD74vIFFWdH4g2Aujn
+lyHA3cCQkGkNIz51dc7g1693b6765BOYPx8++gj+9S/3/J1eveCOO9zjlvPzs604NRQXuxvPRo6E
+m292dx//9rcwaJC7Ee2QQ+DAA9001c6doWNH10VkGCEIc6QMBhap6mIAEXkCGAkEzXskMFFVFZgp
+Iu1FpCvQO0Ta1HHIIbBtW8qzHbxtm/sjNkOahHbV+OuR76qwa9eeZft2jt+6FXbu/HJ+hYWw//7u
+CZff+Q4MGZLd5+unk298wy3Ll7vurJdecoPAEyZ8OW5hIRQVuSU/f88isvcSIc56kzhmGkBz1Q0B
+7eXl8M47aS8vjPF3B5YFvi/HteqTxekeMi0AIjIWGAvQpUsXKioqQkjbmwO6dycvllE0ktraWjY3
+09ZUk9EeMBaNNmn/XfPy0Px8yMujrrCQGhEK2rShtqyMnW3bUtu2Ldu6d2dbt25opE41NW4wt4lR
+XV3doGM4IUce6RZVWq9dS/GyZRRu3EhhVRWFmzaRt2PH7kV27ULq6pC6ur1PsB6JdzKmCR0z9aS5
+6oY92mtLSvg01cdNDJrMXlLVCcAEgEGDBunQhsxjTtPc54qKChqkpwlg2rODac88zVU37K29ewbK
+C2P8K4Cege89/LYwcQpDpDUMwzAySBjjfx/oJyL74Uz7XCB6rtwU4Ee+D38IUKWqq0RkTYi0X2L2
+7NlrRWRpPeqRbsqBtdkW0UBMe3Yw7ZmnueqG1GjvFTZiUuNX1VoR+RHwMpAP/F1V54nIOB9+DzAV
+OAVYBGwFLkmUNkSZncNWIBOIyCxVHZRtHQ3BtGcH0555mqtuyLz2UH38qjoVZ+7BbfcE1hW4PGxa
+wzAMI3vYnbuGYRg5hhl/OGJMmm42mPbsYNozT3PVDRnWLho1h9cwDMNo2ViL3zAMI8cw4zcMw8gx
+zPgTICLXi8gKEZnjl1MCYb/0TxxdKCInZ1NnPJrTk1FFZImIfOz38yy/raOIvCoin/rPDtnWCSAi
+fxeR1SIyN7AtrtamdKzE0d4sjnMR6Ski00RkvojME5HxfnuT3vcJdGdvv6uqLXEW4Hrg6hjbBwD/
+BloD+wGfAfnZ1hulMd/r6gO08noHZFtXAr1LgPKobbcA1/j1a4A/ZFun13I8cAQwN5nWpnasxNHe
+LI5zoCtwhF8vAz7xGpv0vk+gO2v73Vr8DWMk8ISqblfVz3E3rg3OsqZodj9VVVV3AJEnozYnRgIP
++fWHgDOyqGU3qjodWB+1OZ7WJnWsxNEej6amfZWqfuDXNwMLcI+2adL7PoHueKRdtxl/cn7sXy7z
+98AlZLynkTYlmoPGIAq8JiKz/ZNaAbqo6iq/Xgl0yY60UMTT2lx+h2Z1nItIb+Bw4F2a0b6P0g1Z
+2u85b/wi8pqIzI2xjMS9UKYPcBiwCrg1q2JbNseq6mG4l/pcLiLHBwPVXQM3i7nHzUmrp1kd5yJS
+CjwN/FRVNwXDmvK+j6E7a/u9yTyWOVuoaqg3covIfcAL/muYJ5Zmm+agcTequsJ/rhaRZ3GXtl+I
+SFd1D/zrCqzOqsjExNPa5H8HVf0ist7Uj3MRKcSZ56Oq+ozf3OT3fSzd2dzvOd/iT4Q/iCL8DxCZ
+CTEFOFdEWvsnj/YD3su0viTsfqqqiLTCPRl1SpY1xURESkSkLLIOnITb11OAi3y0i4B/ZEdhKOJp
+bfLHSnM5zkVEgL8BC1T1tkBQk9738XRndb9nY3S+uSzAw8DHwEf+x+gaCLsWN9q+EBiRba1x9J+C
+m0HwGXBttvUk0NkHN4vh38C8iFagE/A68CnwGtAx21q9rsdxl+Y7cf2vlyXS2pSOlTjam8VxDhyL
+68b5CJjjl1Oa+r5PoDtr+90e2WAYhpFjWFePYRhGjmHGbxiGkWOY8RuGYeQYZvyGYRg5hhm/YRhG
+jmHGbxiGkWOY8RuGYeQY/x+hdDdSYJ2OLwAAAABJRU5ErkJggg==
+"
+>
+</div>
+
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<h5 id="ModelSelector-class">ModelSelector class<a class="anchor-link" href="#ModelSelector-class">&#182;</a></h5><p>Review the <code>ModelSelector</code> class from the codebase found in the <code>my_model_selectors.py</code> module.  It is designed to be a strategy pattern for choosing different model selectors.  For the project submission in this section, subclass <code>SelectorModel</code> to implement the following model selectors.  In other words, you will write your own classes/functions in the <code>my_model_selectors.py</code> module and run them from this notebook:</p>
+<ul>
+<li><code>SelectorCV</code>:  Log likelihood with CV</li>
+<li><code>SelectorBIC</code>: BIC </li>
+<li><code>SelectorDIC</code>: DIC</li>
+</ul>
+<p>You will train each word in the training set with a range of values for the number of hidden states, and then score these alternatives with the model selector, choosing the "best" according to each strategy. The simple case of training with a constant value for <code>n_components</code> can be called using the provided <code>SelectorConstant</code> subclass as follow:</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[21]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">my_model_selectors</span> <span class="k">import</span> <span class="n">SelectorConstant</span>
+
+<span class="n">training</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">build_training</span><span class="p">(</span><span class="n">features_ground</span><span class="p">)</span>  <span class="c1"># Experiment here with different feature sets defined in part 1</span>
+<span class="n">word</span> <span class="o">=</span> <span class="s1">&#39;VEGETABLE&#39;</span> <span class="c1"># Experiment here with different words</span>
+<span class="n">model</span> <span class="o">=</span> <span class="n">SelectorConstant</span><span class="p">(</span><span class="n">training</span><span class="o">.</span><span class="n">get_all_sequences</span><span class="p">(),</span> <span class="n">training</span><span class="o">.</span><span class="n">get_all_Xlengths</span><span class="p">(),</span> <span class="n">word</span><span class="p">,</span> <span class="n">n_constant</span><span class="o">=</span><span class="mi">3</span><span class="p">)</span><span class="o">.</span><span class="n">select</span><span class="p">()</span>
+<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Number of states trained in model for </span><span class="si">{}</span><span class="s2"> is </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">word</span><span class="p">,</span> <span class="n">model</span><span class="o">.</span><span class="n">n_components</span><span class="p">))</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+<div class="output_subarea output_stream output_stdout output_text">
+<pre>Number of states trained in model for VEGETABLE is 3
+</pre>
+</div>
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<h5 id="Cross-validation-folds">Cross-validation folds<a class="anchor-link" href="#Cross-validation-folds">&#182;</a></h5><p>If we simply score the model with the Log Likelihood calculated from the feature sequences it has been trained on, we should expect that more complex models will have higher likelihoods. However, that doesn't tell us which would have a better likelihood score on unseen data.  The model will likely be overfit as complexity is added.  To estimate which topology model is better using only the training data, we can compare scores using cross-validation.  One technique for cross-validation is to break the training set into "folds" and rotate which fold is left out of training.  The "left out" fold scored.  This gives us a proxy method of finding the best model to use on "unseen data". In the following example, a set of word sequences is broken into three folds using the <a href="http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.KFold.html">scikit-learn Kfold</a> class object. When you implement <code>SelectorCV</code>, you will use this technique.</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[22]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.model_selection</span> <span class="k">import</span> <span class="n">KFold</span>
+
+<span class="n">training</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">build_training</span><span class="p">(</span><span class="n">features_ground</span><span class="p">)</span> <span class="c1"># Experiment here with different feature sets</span>
+<span class="n">word</span> <span class="o">=</span> <span class="s1">&#39;VEGETABLE&#39;</span> <span class="c1"># Experiment here with different words</span>
+<span class="n">word_sequences</span> <span class="o">=</span> <span class="n">training</span><span class="o">.</span><span class="n">get_word_sequences</span><span class="p">(</span><span class="n">word</span><span class="p">)</span>
+<span class="n">split_method</span> <span class="o">=</span> <span class="n">KFold</span><span class="p">()</span>
+<span class="k">for</span> <span class="n">cv_train_idx</span><span class="p">,</span> <span class="n">cv_test_idx</span> <span class="ow">in</span> <span class="n">split_method</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="n">word_sequences</span><span class="p">):</span>
+    <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Train fold indices:</span><span class="si">{}</span><span class="s2"> Test fold indices:</span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">cv_train_idx</span><span class="p">,</span> <span class="n">cv_test_idx</span><span class="p">))</span>  <span class="c1"># view indices of the folds</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+<div class="output_subarea output_stream output_stdout output_text">
+<pre>Train fold indices:[2 3 4 5] Test fold indices:[0 1]
+Train fold indices:[0 1 4 5] Test fold indices:[2 3]
+Train fold indices:[0 1 2 3] Test fold indices:[4 5]
+</pre>
+</div>
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p><strong>Tip:</strong> In order to run <code>hmmlearn</code> training using the X,lengths tuples on the new folds, subsets must be combined based on the indices given for the folds.  A helper utility has been provided in the <code>asl_utils</code> module named <code>combine_sequences</code> for this purpose.</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<h5 id="Scoring-models-with-other-criterion">Scoring models with other criterion<a class="anchor-link" href="#Scoring-models-with-other-criterion">&#182;</a></h5><p>Scoring model topologies with <strong>BIC</strong> balances fit and complexity within the training set for each word.  In the BIC equation, a penalty term penalizes complexity to avoid overfitting, so that it is not necessary to also use cross-validation in the selection process.  There are a number of references on the internet for this criterion.  These <a href="http://www2.imm.dtu.dk/courses/02433/doc/ch6_slides.pdf">slides</a> include a formula you may find helpful for your implementation.</p>
+<p>The advantages of scoring model topologies with <strong>DIC</strong> over BIC are presented by Alain Biem in this <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.58.6208&amp;rep=rep1&amp;type=pdf">reference</a> (also found <a href="https://pdfs.semanticscholar.org/ed3d/7c4a5f607201f3848d4c02dd9ba17c791fc2.pdf">here</a>).  DIC scores the discriminant ability of a training set for one word against competing words.  Instead of a penalty term for complexity, it provides a penalty if model liklihoods for non-matching words are too similar to model likelihoods for the correct word in the word set.</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p><a id='part2_submission'></a></p>
+<h3 id="Model-Selection-Implementation-Submission">Model Selection Implementation Submission<a class="anchor-link" href="#Model-Selection-Implementation-Submission">&#182;</a></h3><p>Implement <code>SelectorCV</code>, <code>SelectorBIC</code>, and <code>SelectorDIC</code> classes in the <code>my_model_selectors.py</code> module.  Run the selectors on the following five words. Then answer the questions about your results.</p>
+<p><strong>Tip:</strong> The <code>hmmlearn</code> library may not be able to train or score all models.  Implement try/except contructs as necessary to eliminate non-viable models from consideration.</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[23]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="n">words_to_train</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;FISH&#39;</span><span class="p">,</span> <span class="s1">&#39;BOOK&#39;</span><span class="p">,</span> <span class="s1">&#39;VEGETABLE&#39;</span><span class="p">,</span> <span class="s1">&#39;FUTURE&#39;</span><span class="p">,</span> <span class="s1">&#39;JOHN&#39;</span><span class="p">]</span>
+<span class="kn">import</span> <span class="nn">timeit</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[24]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># TODO: Implement SelectorCV in my_model_selector.py</span>
+<span class="kn">from</span> <span class="nn">my_model_selectors</span> <span class="k">import</span> <span class="n">SelectorCV</span>
+
+<span class="n">training</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">build_training</span><span class="p">(</span><span class="n">features_ground</span><span class="p">)</span>  <span class="c1"># Experiment here with different feature sets defined in part 1</span>
+
+<span class="n">sequences</span> <span class="o">=</span> <span class="n">training</span><span class="o">.</span><span class="n">get_all_sequences</span><span class="p">()</span>
+<span class="n">Xlengths</span> <span class="o">=</span> <span class="n">training</span><span class="o">.</span><span class="n">get_all_Xlengths</span><span class="p">()</span>
+
+<span class="k">for</span> <span class="n">word</span> <span class="ow">in</span> <span class="n">words_to_train</span><span class="p">:</span>
+    <span class="n">start</span> <span class="o">=</span> <span class="n">timeit</span><span class="o">.</span><span class="n">default_timer</span><span class="p">()</span>
+    <span class="n">model</span> <span class="o">=</span> <span class="n">SelectorCV</span><span class="p">(</span><span class="n">sequences</span><span class="p">,</span> <span class="n">Xlengths</span><span class="p">,</span> <span class="n">word</span><span class="p">,</span> 
+                       <span class="n">min_n_components</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="n">max_n_components</span><span class="o">=</span><span class="mi">15</span><span class="p">,</span> <span class="n">random_state</span> <span class="o">=</span> <span class="mi">14</span><span class="p">)</span><span class="o">.</span><span class="n">select</span><span class="p">()</span>
+    <span class="n">end</span> <span class="o">=</span> <span class="n">timeit</span><span class="o">.</span><span class="n">default_timer</span><span class="p">()</span> <span class="o">-</span> <span class="n">start</span>
+    <span class="k">if</span> <span class="n">model</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
+        <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Training complete for </span><span class="si">{}</span><span class="s2"> with </span><span class="si">{}</span><span class="s2"> states with time </span><span class="si">{}</span><span class="s2"> seconds&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">word</span><span class="p">,</span> <span class="n">model</span><span class="o">.</span><span class="n">n_components</span><span class="p">,</span> <span class="n">end</span><span class="p">))</span>
+    <span class="k">else</span><span class="p">:</span>
+        <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Training failed for </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">word</span><span class="p">))</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+<div class="output_subarea output_stream output_stdout output_text">
+<pre>Training complete for FISH with 3 states with time 0.028000007005175576 seconds
+Training complete for BOOK with 15 states with time 2.330455476010684 seconds
+Training complete for VEGETABLE with 15 states with time 0.7969696299987845 seconds
+Training complete for FUTURE with 15 states with time 2.4186781469907146 seconds
+Training complete for JOHN with 15 states with time 25.637559488997795 seconds
+</pre>
+</div>
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[25]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># TODO: Implement SelectorBIC in module my_model_selectors.py</span>
+<span class="kn">from</span> <span class="nn">my_model_selectors</span> <span class="k">import</span> <span class="n">SelectorBIC</span>
+
+<span class="n">training</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">build_training</span><span class="p">(</span><span class="n">features_ground</span><span class="p">)</span>  <span class="c1"># Experiment here with different feature sets defined in part 1</span>
+<span class="n">sequences</span> <span class="o">=</span> <span class="n">training</span><span class="o">.</span><span class="n">get_all_sequences</span><span class="p">()</span>
+<span class="n">Xlengths</span> <span class="o">=</span> <span class="n">training</span><span class="o">.</span><span class="n">get_all_Xlengths</span><span class="p">()</span>
+<span class="k">for</span> <span class="n">word</span> <span class="ow">in</span> <span class="n">words_to_train</span><span class="p">:</span>
+    <span class="n">start</span> <span class="o">=</span> <span class="n">timeit</span><span class="o">.</span><span class="n">default_timer</span><span class="p">()</span>
+    <span class="n">model</span> <span class="o">=</span> <span class="n">SelectorBIC</span><span class="p">(</span><span class="n">sequences</span><span class="p">,</span> <span class="n">Xlengths</span><span class="p">,</span> <span class="n">word</span><span class="p">,</span> 
+                    <span class="n">min_n_components</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="n">max_n_components</span><span class="o">=</span><span class="mi">15</span><span class="p">,</span> <span class="n">random_state</span> <span class="o">=</span> <span class="mi">14</span><span class="p">)</span><span class="o">.</span><span class="n">select</span><span class="p">()</span>
+    <span class="n">end</span> <span class="o">=</span> <span class="n">timeit</span><span class="o">.</span><span class="n">default_timer</span><span class="p">()</span><span class="o">-</span><span class="n">start</span>
+    <span class="k">if</span> <span class="n">model</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
+        <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Training complete for </span><span class="si">{}</span><span class="s2"> with </span><span class="si">{}</span><span class="s2"> states with time </span><span class="si">{}</span><span class="s2"> seconds&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">word</span><span class="p">,</span> <span class="n">model</span><span class="o">.</span><span class="n">n_components</span><span class="p">,</span> <span class="n">end</span><span class="p">))</span>
+    <span class="k">else</span><span class="p">:</span>
+        <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Training failed for </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">word</span><span class="p">))</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+<div class="output_subarea output_stream output_stdout output_text">
+<pre>Training complete for FISH with 2 states with time 0.09146881599735934 seconds
+Training complete for BOOK with 2 states with time 2.016845521997311 seconds
+Training complete for VEGETABLE with 2 states with time 0.7241176120005548 seconds
+Training complete for FUTURE with 2 states with time 2.1909853079996537 seconds
+Training complete for JOHN with 2 states with time 22.080462529993383 seconds
+</pre>
+</div>
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[26]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># TODO: Implement SelectorDIC in module my_model_selectors.py</span>
+<span class="kn">from</span> <span class="nn">my_model_selectors</span> <span class="k">import</span> <span class="n">SelectorDIC</span>
+
+<span class="n">training</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">build_training</span><span class="p">(</span><span class="n">features_ground</span><span class="p">)</span>  <span class="c1"># Experiment here with different feature sets defined in part 1</span>
+<span class="n">sequences</span> <span class="o">=</span> <span class="n">training</span><span class="o">.</span><span class="n">get_all_sequences</span><span class="p">()</span>
+<span class="n">Xlengths</span> <span class="o">=</span> <span class="n">training</span><span class="o">.</span><span class="n">get_all_Xlengths</span><span class="p">()</span>
+<span class="k">for</span> <span class="n">word</span> <span class="ow">in</span> <span class="n">words_to_train</span><span class="p">:</span>
+    <span class="n">start</span> <span class="o">=</span> <span class="n">timeit</span><span class="o">.</span><span class="n">default_timer</span><span class="p">()</span>
+    <span class="n">model</span> <span class="o">=</span> <span class="n">SelectorDIC</span><span class="p">(</span><span class="n">sequences</span><span class="p">,</span> <span class="n">Xlengths</span><span class="p">,</span> <span class="n">word</span><span class="p">,</span> 
+                    <span class="n">min_n_components</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="n">max_n_components</span><span class="o">=</span><span class="mi">15</span><span class="p">,</span> <span class="n">random_state</span> <span class="o">=</span> <span class="mi">14</span><span class="p">)</span><span class="o">.</span><span class="n">select</span><span class="p">()</span>
+    <span class="n">end</span> <span class="o">=</span> <span class="n">timeit</span><span class="o">.</span><span class="n">default_timer</span><span class="p">()</span><span class="o">-</span><span class="n">start</span>
+    <span class="k">if</span> <span class="n">model</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
+        <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Training complete for </span><span class="si">{}</span><span class="s2"> with </span><span class="si">{}</span><span class="s2"> states with time </span><span class="si">{}</span><span class="s2"> seconds&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">word</span><span class="p">,</span> <span class="n">model</span><span class="o">.</span><span class="n">n_components</span><span class="p">,</span> <span class="n">end</span><span class="p">))</span>
+    <span class="k">else</span><span class="p">:</span>
+        <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Training failed for </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">word</span><span class="p">))</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+<div class="output_subarea output_stream output_stdout output_text">
+<pre>Training complete for FISH with 3 states with time 0.09454512700904161 seconds
+Training complete for BOOK with 15 states with time 2.4017886569927214 seconds
+Training complete for VEGETABLE with 15 states with time 0.8214017159916693 seconds
+Training complete for FUTURE with 15 states with time 2.455093226992176 seconds
+Training complete for JOHN with 15 states with time 25.174293504998786 seconds
+</pre>
+</div>
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p><strong>Question 2:</strong>  Compare and contrast the possible advantages and disadvantages of the various model selectors implemented.</p>
+<p><strong>Answer 2:</strong> Selection using cross validation is the easier to implement compared to BIC &amp; DIC. The idea in BIC to penalize model complexity is interesting as it may also have a side effect of faster running times. Yet, I also think it might be a doubled edged sword, as in some cases it will sacrifice complexity for performance. <strong>DIC seems like the best selector</strong> because it will use the model that gives the best distinction between the word of interest and all others. This should result in the best performing model but also the most sluggish as it needs to calculate the probabilities of each word.</p>
+<p>A more formal listing of advantages &amp; disadvantages is depicted below.</p>
+<h3 id="Cross-Validation:">Cross Validation:<a class="anchor-link" href="#Cross-Validation:">&#182;</a></h3><p><strong>Advantages:</strong> Does not need a lot of data for training – as the train data is folded to simulate the behavior that the model will have in test data.</p>
+<p><strong>Disadvantages:</strong> It needs to split the sequences from the beginning each time a new state will be evaluated (overhead).</p>
+<h3 id="BIC:">BIC:<a class="anchor-link" href="#BIC:">&#182;</a></h3><p><strong>Advantages:</strong> It penalizes model's complexity (of parameters).</p>
+<p><strong>Disadvantages</strong> Not as accurate as cross-validation – this could be possible because of the requirement of more training data (as there are no folds to simulate test data).</p>
+<h3 id="DIC:">DIC:<a class="anchor-link" href="#DIC:">&#182;</a></h3><p><strong>Advantages:</strong> Better performance than BIC.</p>
+<p><strong>Disadvantages:</strong> Model complexity is not penalized which might lead to a large number of parameters.</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p><a id='part2_test'></a></p>
+<h3 id="Model-Selector-Unit-Testing">Model Selector Unit Testing<a class="anchor-link" href="#Model-Selector-Unit-Testing">&#182;</a></h3><p>Run the following unit tests as a sanity check on the implemented model selectors.  The test simply looks for valid interfaces  but is not exhaustive. However, the project should not be submitted if these tests don't pass.</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[27]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">asl_test_model_selectors</span> <span class="k">import</span> <span class="n">TestSelectors</span>
+<span class="n">suite</span> <span class="o">=</span> <span class="n">unittest</span><span class="o">.</span><span class="n">TestLoader</span><span class="p">()</span><span class="o">.</span><span class="n">loadTestsFromModule</span><span class="p">(</span><span class="n">TestSelectors</span><span class="p">())</span>
+<span class="n">unittest</span><span class="o">.</span><span class="n">TextTestRunner</span><span class="p">()</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">suite</span><span class="p">)</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+<div class="output_subarea output_stream output_stderr output_text">
+<pre>....
+----------------------------------------------------------------------
+Ran 4 tests in 39.572s
+
+OK
+</pre>
+</div>
+</div>
+
+<div class="output_area"><div class="prompt output_prompt">Out[27]:</div>
+
+
+<div class="output_text output_subarea output_execute_result">
+<pre>&lt;unittest.runner.TextTestResult run=4 errors=0 failures=0&gt;</pre>
+</div>
+
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p><a id='part3_tutorial'></a></p>
+<h2 id="PART-3:-Recognizer">PART 3: Recognizer<a class="anchor-link" href="#PART-3:-Recognizer">&#182;</a></h2><p>The objective of this section is to "put it all together".  Using the four feature sets created and the three model selectors, you will experiment with the models and present your results.  Instead of training only five specific words as in the previous section, train the entire set with a feature set and model selector strategy.</p>
+<h3 id="Recognizer-Tutorial">Recognizer Tutorial<a class="anchor-link" href="#Recognizer-Tutorial">&#182;</a></h3><h5 id="Train-the-full-training-set">Train the full training set<a class="anchor-link" href="#Train-the-full-training-set">&#182;</a></h5><p>The following example trains the entire set with the example <code>features_ground</code> and <code>SelectorConstant</code> features and model selector.  Use this pattern for you experimentation and final submission cells.</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[28]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># autoreload for automatically reloading changes made in my_model_selectors and my_recognizer</span>
+<span class="o">%</span><span class="k">load_ext</span> autoreload
+<span class="o">%</span><span class="k">autoreload</span> 2
+
+<span class="kn">from</span> <span class="nn">my_model_selectors</span> <span class="k">import</span> <span class="n">SelectorConstant</span>
+
+<span class="k">def</span> <span class="nf">train_all_words</span><span class="p">(</span><span class="n">features</span><span class="p">,</span> <span class="n">model_selector</span><span class="p">):</span>
+    <span class="n">training</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">build_training</span><span class="p">(</span><span class="n">features</span><span class="p">)</span>  <span class="c1"># Experiment here with different feature sets defined in part 1</span>
+    <span class="n">sequences</span> <span class="o">=</span> <span class="n">training</span><span class="o">.</span><span class="n">get_all_sequences</span><span class="p">()</span>
+    <span class="n">Xlengths</span> <span class="o">=</span> <span class="n">training</span><span class="o">.</span><span class="n">get_all_Xlengths</span><span class="p">()</span>
+    <span class="n">model_dict</span> <span class="o">=</span> <span class="p">{}</span>
+    <span class="k">for</span> <span class="n">word</span> <span class="ow">in</span> <span class="n">training</span><span class="o">.</span><span class="n">words</span><span class="p">:</span>
+        <span class="n">model</span> <span class="o">=</span> <span class="n">model_selector</span><span class="p">(</span><span class="n">sequences</span><span class="p">,</span> <span class="n">Xlengths</span><span class="p">,</span> <span class="n">word</span><span class="p">,</span> 
+                        <span class="n">n_constant</span><span class="o">=</span><span class="mi">3</span><span class="p">)</span><span class="o">.</span><span class="n">select</span><span class="p">()</span>
+        <span class="n">model_dict</span><span class="p">[</span><span class="n">word</span><span class="p">]</span><span class="o">=</span><span class="n">model</span>
+    <span class="k">return</span> <span class="n">model_dict</span>
+
+<span class="n">models</span> <span class="o">=</span> <span class="n">train_all_words</span><span class="p">(</span><span class="n">features_ground</span><span class="p">,</span> <span class="n">SelectorConstant</span><span class="p">)</span>
+<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Number of word models returned = </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">models</span><span class="p">)))</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+<div class="output_subarea output_stream output_stdout output_text">
+<pre>Number of word models returned = 112
+</pre>
+</div>
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<h5 id="Load-the-test-set">Load the test set<a class="anchor-link" href="#Load-the-test-set">&#182;</a></h5><p>The <code>build_test</code> method in <code>ASLdb</code> is similar to the <code>build_training</code> method already presented, but there are a few differences:</p>
+<ul>
+<li>the object is type <code>SinglesData</code> </li>
+<li>the internal dictionary keys are the index of the test word rather than the word itself</li>
+<li>the getter methods are <code>get_all_sequences</code>, <code>get_all_Xlengths</code>, <code>get_item_sequences</code> and <code>get_item_Xlengths</code></li>
+</ul>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[29]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="n">test_set</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">build_test</span><span class="p">(</span><span class="n">features_ground</span><span class="p">)</span>
+<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Number of test set items: </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">test_set</span><span class="o">.</span><span class="n">num_items</span><span class="p">))</span>
+<span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Number of test set sentences: </span><span class="si">{}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">test_set</span><span class="o">.</span><span class="n">sentences_index</span><span class="p">)))</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+<div class="output_subarea output_stream output_stdout output_text">
+<pre>Number of test set items: 178
+Number of test set sentences: 40
+</pre>
+</div>
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p><a id='part3_submission'></a></p>
+<h3 id="Recognizer-Implementation-Submission">Recognizer Implementation Submission<a class="anchor-link" href="#Recognizer-Implementation-Submission">&#182;</a></h3><p>For the final project submission, students must implement a recognizer following guidance in the <code>my_recognizer.py</code> module.  Experiment with the four feature sets and the three model selection methods (that's 12 possible combinations). You can add and remove cells for experimentation or run the recognizers locally in some other way during your experiments, but retain the results for your discussion.  For submission, you will provide code cells of <strong>only three</strong> interesting combinations for your discussion (see questions below). At least one of these should produce a word error rate of less than 60%, i.e. WER &lt; 0.60 .</p>
+<p><strong>Tip:</strong> The hmmlearn library may not be able to train or score all models.  Implement try/except contructs as necessary to eliminate non-viable models from consideration.</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[30]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># TODO implement the recognize method in my_recognizer</span>
+<span class="kn">from</span> <span class="nn">my_recognizer</span> <span class="k">import</span> <span class="n">recognize</span>
+<span class="kn">from</span> <span class="nn">asl_utils</span> <span class="k">import</span> <span class="n">show_errors</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[33]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># TODO Choose a feature set and model selector</span>
+<span class="n">features</span> <span class="o">=</span> <span class="n">features_polar</span> <span class="c1"># change as needed</span>
+<span class="n">model_selector</span> <span class="o">=</span> <span class="n">SelectorCV</span> <span class="c1"># change as needed</span>
+
+<span class="c1"># TODO Recognize the test set and display the result with the show_errors method</span>
+<span class="n">models</span> <span class="o">=</span> <span class="n">train_all_words</span><span class="p">(</span><span class="n">features</span><span class="p">,</span> <span class="n">model_selector</span><span class="p">)</span>
+<span class="n">test_set</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">build_test</span><span class="p">(</span><span class="n">features</span><span class="p">)</span>
+<span class="n">probabilities</span><span class="p">,</span> <span class="n">guesses</span> <span class="o">=</span> <span class="n">recognize</span><span class="p">(</span><span class="n">models</span><span class="p">,</span> <span class="n">test_set</span><span class="p">)</span>
+<span class="n">show_errors</span><span class="p">(</span><span class="n">guesses</span><span class="p">,</span> <span class="n">test_set</span><span class="p">)</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+<div class="output_subarea output_stream output_stdout output_text">
+<pre>
+**** WER = 0.5280898876404494
+Total correct: 84 out of 178
+Video  Recognized                                                    Correct
+=====================================================================================================
+    2: JOHN WRITE HOMEWORK                                           JOHN WRITE HOMEWORK
+    7: JOHN CAN GO CAN                                               JOHN CAN GO CAN
+   12: JOHN *WHAT *JOHN CAN                                          JOHN CAN GO CAN
+   21: JOHN *HOMEWORK *JOHN *PREFER BUT *WHAT *FUTURE *WHO           JOHN FISH WONT EAT BUT CAN EAT CHICKEN
+   25: JOHN *IX IX *WHO IX                                           JOHN LIKE IX IX IX
+   28: JOHN *FUTURE IX *FUTURE *LOVE                                 JOHN LIKE IX IX IX
+   30: JOHN LIKE *MARY *MARY *MARY                                   JOHN LIKE IX IX IX
+   36: *IX *VISIT *GIVE *GIVE *MARY *MARY                            MARY VEGETABLE KNOW IX LIKE CORN1
+   40: JOHN *GO *DECIDE *JOHN *MARY                                  JOHN IX THINK MARY LOVE
+   43: JOHN *IX BUY HOUSE                                            JOHN MUST BUY HOUSE
+   50: *JOHN *SEE BUY CAR *JOHN                                      FUTURE JOHN BUY CAR SHOULD
+   54: JOHN SHOULD NOT BUY HOUSE                                     JOHN SHOULD NOT BUY HOUSE
+   57: *MARY *GO *GO MARY                                            JOHN DECIDE VISIT MARY
+   67: *SHOULD FUTURE *MARY BUY HOUSE                                JOHN FUTURE NOT BUY HOUSE
+   71: JOHN *FUTURE *GIVE1 MARY                                      JOHN WILL VISIT MARY
+   74: *IX *GO *GO *VISIT                                            JOHN NOT VISIT MARY
+   77: *JOHN *GIVE1 MARY                                             ANN BLAME MARY
+   84: *HOMEWORK *GIVE1 *HOMEWORK *COAT                              IX-1P FIND SOMETHING-ONE BOOK
+   89: *MAN *GIVE *IX *IX IX *ARRIVE *BOOK                           JOHN IX GIVE MAN IX NEW COAT
+   90: JOHN GIVE IX SOMETHING-ONE WOMAN *ARRIVE                      JOHN GIVE IX SOMETHING-ONE WOMAN BOOK
+   92: JOHN *IX IX *IX *IX BOOK                                      JOHN GIVE IX SOMETHING-ONE WOMAN BOOK
+  100: POSS NEW CAR BREAK-DOWN                                       POSS NEW CAR BREAK-DOWN
+  105: JOHN *SEE                                                     JOHN LEG
+  107: JOHN POSS *HAVE HAVE *MARY                                    JOHN POSS FRIEND HAVE CANDY
+  108: *LOVE *HOMEWORK                                               WOMAN ARRIVE
+  113: IX CAR *IX *MARY *JOHN                                        IX CAR BLUE SUE BUY
+  119: *MARY *BUY1 IX *BLAME *IX                                     SUE BUY IX CAR BLUE
+  122: JOHN *GIVE1 BOOK                                              JOHN READ BOOK
+  139: JOHN *ARRIVE WHAT *MARY *ARRIVE                               JOHN BUY WHAT YESTERDAY BOOK
+  142: JOHN BUY YESTERDAY WHAT BOOK                                  JOHN BUY YESTERDAY WHAT BOOK
+  158: LOVE JOHN WHO                                                 LOVE JOHN WHO
+  167: JOHN *MARY *VISIT LOVE MARY                                   JOHN IX SAY LOVE MARY
+  171: *IX MARY BLAME                                                JOHN MARY BLAME
+  174: *JOHN *GIVE3 GIVE1 *YESTERDAY *JOHN                           PEOPLE GROUP GIVE1 JANA TOY
+  181: *EAT ARRIVE                                                   JOHN ARRIVE
+  184: ALL BOY *GIVE1 TEACHER *YESTERDAY                             ALL BOY GIVE TEACHER APPLE
+  189: *MARY *GO *YESTERDAY BOX                                      JOHN GIVE GIRL BOX
+  193: JOHN *GO *YESTERDAY BOX                                       JOHN GIVE GIRL BOX
+  199: *HOMEWORK *STUDENT *GO                                        LIKE CHOCOLATE WHO
+  201: JOHN *MAN *LOVE *JOHN BUY HOUSE                               JOHN TELL MARY IX-1P BUY HOUSE
+</pre>
+</div>
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[34]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># TODO Choose a feature set and model selector</span>
+<span class="c1"># TODO Recognize the test set and display the result with the show_errors method</span>
+<span class="n">features</span> <span class="o">=</span> <span class="n">features_polar</span> <span class="c1"># change as needed</span>
+<span class="n">model_selector</span> <span class="o">=</span> <span class="n">SelectorDIC</span> <span class="c1"># change as needed</span>
+
+<span class="c1"># TODO Recognize the test set and display the result with the show_errors method</span>
+<span class="n">models</span> <span class="o">=</span> <span class="n">train_all_words</span><span class="p">(</span><span class="n">features</span><span class="p">,</span> <span class="n">model_selector</span><span class="p">)</span>
+<span class="n">test_set</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">build_test</span><span class="p">(</span><span class="n">features</span><span class="p">)</span>
+<span class="n">probabilities</span><span class="p">,</span> <span class="n">guesses</span> <span class="o">=</span> <span class="n">recognize</span><span class="p">(</span><span class="n">models</span><span class="p">,</span> <span class="n">test_set</span><span class="p">)</span>
+<span class="n">show_errors</span><span class="p">(</span><span class="n">guesses</span><span class="p">,</span> <span class="n">test_set</span><span class="p">)</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+<div class="output_subarea output_stream output_stdout output_text">
+<pre>
+**** WER = 0.5337078651685393
+Total correct: 83 out of 178
+Video  Recognized                                                    Correct
+=====================================================================================================
+    2: JOHN *NEW HOMEWORK                                            JOHN WRITE HOMEWORK
+    7: JOHN CAN GO CAN                                               JOHN CAN GO CAN
+   12: JOHN *WHAT *JOHN CAN                                          JOHN CAN GO CAN
+   21: JOHN *HOMEWORK *JOHN *PREFER *CAR *WHAT *FUTURE *WHO          JOHN FISH WONT EAT BUT CAN EAT CHICKEN
+   25: JOHN *IX IX *WHO IX                                           JOHN LIKE IX IX IX
+   28: JOHN *FUTURE IX *FUTURE *LOVE                                 JOHN LIKE IX IX IX
+   30: JOHN LIKE *MARY *MARY *MARY                                   JOHN LIKE IX IX IX
+   36: *IX *EAT *GIVE *GIVE *MARY *MARY                              MARY VEGETABLE KNOW IX LIKE CORN1
+   40: JOHN *GO *GIVE *JOHN *MARY                                    JOHN IX THINK MARY LOVE
+   43: JOHN *IX BUY HOUSE                                            JOHN MUST BUY HOUSE
+   50: *JOHN *SEE BUY CAR *JOHN                                      FUTURE JOHN BUY CAR SHOULD
+   54: JOHN SHOULD NOT BUY HOUSE                                     JOHN SHOULD NOT BUY HOUSE
+   57: *MARY *GO *GO MARY                                            JOHN DECIDE VISIT MARY
+   67: *SHOULD FUTURE *MARY BUY HOUSE                                JOHN FUTURE NOT BUY HOUSE
+   71: JOHN *FUTURE *GIVE1 MARY                                      JOHN WILL VISIT MARY
+   74: *IX *GO *GO *VISIT                                            JOHN NOT VISIT MARY
+   77: *JOHN *GIVE1 MARY                                             ANN BLAME MARY
+   84: *HOMEWORK *GIVE1 *HOMEWORK *COAT                              IX-1P FIND SOMETHING-ONE BOOK
+   89: *GIVE *GIVE *IX *IX IX *ARRIVE *BOOK                          JOHN IX GIVE MAN IX NEW COAT
+   90: JOHN GIVE IX SOMETHING-ONE WOMAN *ARRIVE                      JOHN GIVE IX SOMETHING-ONE WOMAN BOOK
+   92: JOHN *IX IX *IX *IX BOOK                                      JOHN GIVE IX SOMETHING-ONE WOMAN BOOK
+  100: POSS NEW CAR BREAK-DOWN                                       POSS NEW CAR BREAK-DOWN
+  105: JOHN *SEE                                                     JOHN LEG
+  107: JOHN POSS *HAVE HAVE *MARY                                    JOHN POSS FRIEND HAVE CANDY
+  108: *LOVE *HOMEWORK                                               WOMAN ARRIVE
+  113: IX CAR *IX *MARY *JOHN                                        IX CAR BLUE SUE BUY
+  119: *MARY *BUY1 IX *BLAME *IX                                     SUE BUY IX CAR BLUE
+  122: JOHN *GIVE1 BOOK                                              JOHN READ BOOK
+  139: JOHN *ARRIVE WHAT *MARY *ARRIVE                               JOHN BUY WHAT YESTERDAY BOOK
+  142: JOHN BUY YESTERDAY WHAT BOOK                                  JOHN BUY YESTERDAY WHAT BOOK
+  158: LOVE JOHN WHO                                                 LOVE JOHN WHO
+  167: JOHN *MARY *VISIT LOVE MARY                                   JOHN IX SAY LOVE MARY
+  171: *IX MARY BLAME                                                JOHN MARY BLAME
+  174: *JOHN GROUP GIVE1 *YESTERDAY *JOHN                            PEOPLE GROUP GIVE1 JANA TOY
+  181: *EAT ARRIVE                                                   JOHN ARRIVE
+  184: ALL BOY *GIVE1 TEACHER *YESTERDAY                             ALL BOY GIVE TEACHER APPLE
+  189: *MARY *GO *YESTERDAY BOX                                      JOHN GIVE GIRL BOX
+  193: JOHN *GO *YESTERDAY BOX                                       JOHN GIVE GIRL BOX
+  199: *HOMEWORK *STUDENT *GO                                        LIKE CHOCOLATE WHO
+  201: JOHN *GIVE *LOVE *JOHN BUY HOUSE                              JOHN TELL MARY IX-1P BUY HOUSE
+</pre>
+</div>
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[35]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># TODO Choose a feature set and model selector</span>
+<span class="c1"># TODO Recognize the test set and display the result with the show_errors method</span>
+<span class="n">features</span> <span class="o">=</span> <span class="n">features_norm</span> <span class="c1"># change as needed</span>
+<span class="n">model_selector</span> <span class="o">=</span> <span class="n">SelectorCV</span> <span class="c1"># change as needed</span>
+
+<span class="c1"># TODO Recognize the test set and display the result with the show_errors method</span>
+<span class="n">models</span> <span class="o">=</span> <span class="n">train_all_words</span><span class="p">(</span><span class="n">features</span><span class="p">,</span> <span class="n">model_selector</span><span class="p">)</span>
+<span class="n">test_set</span> <span class="o">=</span> <span class="n">asl</span><span class="o">.</span><span class="n">build_test</span><span class="p">(</span><span class="n">features</span><span class="p">)</span>
+<span class="n">probabilities</span><span class="p">,</span> <span class="n">guesses</span> <span class="o">=</span> <span class="n">recognize</span><span class="p">(</span><span class="n">models</span><span class="p">,</span> <span class="n">test_set</span><span class="p">)</span>
+<span class="n">show_errors</span><span class="p">(</span><span class="n">guesses</span><span class="p">,</span> <span class="n">test_set</span><span class="p">)</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+<div class="output_wrapper">
+<div class="output">
+
+
+<div class="output_area"><div class="prompt"></div>
+<div class="output_subarea output_stream output_stdout output_text">
+<pre>
+**** WER = 0.6067415730337079
+Total correct: 70 out of 178
+Video  Recognized                                                    Correct
+=====================================================================================================
+    2: JOHN WRITE HOMEWORK                                           JOHN WRITE HOMEWORK
+    7: *MARY *CAR GO *WHAT                                           JOHN CAN GO CAN
+   12: JOHN *CAR *WHAT CAN                                           JOHN CAN GO CAN
+   21: *MARY *BOX *JOHN *BLAME *CAR *CAR *CHICKEN *WRITE             JOHN FISH WONT EAT BUT CAN EAT CHICKEN
+   25: JOHN LIKE IX *MARY IX                                         JOHN LIKE IX IX IX
+   28: *ANN LIKE *ANN *ANN *ANN                                      JOHN LIKE IX IX IX
+   30: *IX-1P *CHOCOLATE *MARY *LOVE *LOVE                           JOHN LIKE IX IX IX
+   36: MARY *MARY *YESTERDAY *WOMAN LIKE *IX                         MARY VEGETABLE KNOW IX LIKE CORN1
+   40: *MARY *JOHN *FUTURE1 MARY *MARY                               JOHN IX THINK MARY LOVE
+   43: JOHN *FUTURE BUY HOUSE                                        JOHN MUST BUY HOUSE
+   50: *POSS *SEE *BOX CAR *IX                                       FUTURE JOHN BUY CAR SHOULD
+   54: JOHN *FUTURE *SHOULD *ARRIVE HOUSE                            JOHN SHOULD NOT BUY HOUSE
+   57: *LOVE *IX *JOHN *VISIT                                        JOHN DECIDE VISIT MARY
+   67: *MARY *IX *JOHN *ARRIVE HOUSE                                 JOHN FUTURE NOT BUY HOUSE
+   71: JOHN WILL VISIT MARY                                          JOHN WILL VISIT MARY
+   74: *GO *VISIT VISIT MARY                                         JOHN NOT VISIT MARY
+   77: *JOHN BLAME MARY                                              ANN BLAME MARY
+   84: *JOHN *BOX *VISIT BOOK                                        IX-1P FIND SOMETHING-ONE BOOK
+   89: *MARY *POSS *IX *IX IX *ARRIVE COAT                           JOHN IX GIVE MAN IX NEW COAT
+   90: *SELF *IX IX *IX WOMAN BOOK                                   JOHN GIVE IX SOMETHING-ONE WOMAN BOOK
+   92: JOHN *IX IX *IX *LOVE BOOK                                    JOHN GIVE IX SOMETHING-ONE WOMAN BOOK
+  100: POSS NEW CAR BREAK-DOWN                                       POSS NEW CAR BREAK-DOWN
+  105: JOHN *POSS                                                    JOHN LEG
+  107: *MARY POSS *BOX *MARY *MARY                                   JOHN POSS FRIEND HAVE CANDY
+  108: *LOVE *HOMEWORK                                               WOMAN ARRIVE
+  113: *SHOULD CAR *IX *JOHN *BOX                                    IX CAR BLUE SUE BUY
+  119: *PREFER *BUY1 IX *JOHN *GO                                    SUE BUY IX CAR BLUE
+  122: JOHN *GIVE1 BOOK                                              JOHN READ BOOK
+  139: JOHN *BUY1 *CAR *JOHN BOOK                                    JOHN BUY WHAT YESTERDAY BOOK
+  142: JOHN BUY YESTERDAY WHAT BOOK                                  JOHN BUY YESTERDAY WHAT BOOK
+  158: LOVE JOHN WHO                                                 LOVE JOHN WHO
+  167: JOHN IX *SAY-1P LOVE *IX                                      JOHN IX SAY LOVE MARY
+  171: *LIKE *JOHN BLAME                                             JOHN MARY BLAME
+  174: *CAR *GIVE1 GIVE1 *YESTERDAY *CAR                             PEOPLE GROUP GIVE1 JANA TOY
+  181: JOHN *BOX                                                     JOHN ARRIVE
+  184: *IX *IX *GIVE1 TEACHER APPLE                                  ALL BOY GIVE TEACHER APPLE
+  189: *JANA *MARY *YESTERDAY BOX                                    JOHN GIVE GIRL BOX
+  193: *IX *YESTERDAY *YESTERDAY BOX                                 JOHN GIVE GIRL BOX
+  199: *JOHN *BOX *JOHN                                              LIKE CHOCOLATE WHO
+  201: JOHN *GIVE1 *IX *WOMAN *ARRIVE HOUSE                          JOHN TELL MARY IX-1P BUY HOUSE
+</pre>
+</div>
+</div>
+
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p><strong>Question 3:</strong>  Summarize the error results from three combinations of features and model selectors.  What was the "best" combination and why?  What additional information might we use to improve our WER?  For more insight on improving WER, take a look at the introduction to Part 4.</p>
+<p><strong>Answer 3:</strong></p>
+<table>
+<thead><tr>
+<th><strong>Model</strong></th>
+<th><strong>Features</strong></th>
+<th><strong>WER</strong></th>
+<th><strong>Correct</strong></th>
+<th><strong>Incorrect</strong></th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>SelectorCV</td>
+<td>features_polar</td>
+<td>0.5280898876404494</td>
+<td>84</td>
+<td>94</td>
+</tr>
+<tr>
+<td>SelectorDIC</td>
+<td>features_polar</td>
+<td>0.5337078651685393</td>
+<td>83</td>
+<td>95</td>
+</tr>
+<tr>
+<td>SelectorCV</td>
+<td>features_norm</td>
+<td>0.6067415730337079</td>
+<td>70</td>
+<td>108</td>
+</tr>
+</tbody>
+</table>
+<p>The best results were obtained using the <em>SelectorCV</em> model selector which makes efficient use of the training data by folding it and attempting to simulate how the model would behave on test data. However, my preferred model DIC doesn't lag much. <em>features_polar</em> produced the lowest WER (with both <em>SelectorCV &amp; SelectorDIC</em> model). This could be improved by computing the probability of a word being next to another using statistical language models (or other NLP techniques).</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p><a id='part3_test'></a></p>
+<h3 id="Recognizer-Unit-Tests">Recognizer Unit Tests<a class="anchor-link" href="#Recognizer-Unit-Tests">&#182;</a></h3><p>Run the following unit tests as a sanity check on the defined recognizer.  The test simply looks for some valid values but is not exhaustive. However, the project should not be submitted if these tests don't pass.</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[&nbsp;]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">asl_test_recognizer</span> <span class="k">import</span> <span class="n">TestRecognize</span>
+<span class="n">suite</span> <span class="o">=</span> <span class="n">unittest</span><span class="o">.</span><span class="n">TestLoader</span><span class="p">()</span><span class="o">.</span><span class="n">loadTestsFromModule</span><span class="p">(</span><span class="n">TestRecognize</span><span class="p">())</span>
+<span class="n">unittest</span><span class="o">.</span><span class="n">TextTestRunner</span><span class="p">()</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">suite</span><span class="p">)</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+</div>
+<div class="cell border-box-sizing text_cell rendered">
+<div class="prompt input_prompt">
+</div>
+<div class="inner_cell">
+<div class="text_cell_render border-box-sizing rendered_html">
+<p><a id='part4_info'></a></p>
+<h2 id="PART-4:-(OPTIONAL)--Improve-the-WER-with-Language-Models">PART 4: (OPTIONAL)  Improve the WER with Language Models<a class="anchor-link" href="#PART-4:-(OPTIONAL)--Improve-the-WER-with-Language-Models">&#182;</a></h2><p>We've squeezed just about as much as we can out of the model and still only get about 50% of the words right! Surely we can do better than that.  Probability to the rescue again in the form of <a href="https://en.wikipedia.org/wiki/Language_model">statistical language models (SLM)</a>.  The basic idea is that each word has some probability of occurrence within the set, and some probability that it is adjacent to specific other words. We can use that additional information to make better choices.</p>
+<h5 id="Additional-reading-and-resources">Additional reading and resources<a class="anchor-link" href="#Additional-reading-and-resources">&#182;</a></h5><ul>
+<li><a href="https://web.stanford.edu/class/cs124/lec/languagemodeling.pdf">Introduction to N-grams (Stanford Jurafsky slides)</a></li>
+<li><a href="https://www-i6.informatik.rwth-aachen.de/publications/download/154/Dreuw--2007.pdf">Speech Recognition Techniques for a Sign Language Recognition System, Philippe Dreuw et al</a> see the improved results of applying LM on <em>this</em> data!</li>
+<li><a href="ftp://wasserstoff.informatik.rwth-aachen.de/pub/rwth-boston-104/lm/">SLM data for <em>this</em> ASL dataset</a></li>
+</ul>
+<h5 id="Optional-challenge">Optional challenge<a class="anchor-link" href="#Optional-challenge">&#182;</a></h5><p>The recognizer you implemented in Part 3 is equivalent to a "0-gram" SLM.  Improve the WER with the SLM data provided with the data set in the link above using "1-gram", "2-gram", and/or "3-gram" statistics. The <code>probabilities</code> data you've already calculated will be useful and can be turned into a pandas DataFrame if desired (see next cell).<br>
+Good luck!  Share your results with the class!</p>
+
+</div>
+</div>
+</div>
+<div class="cell border-box-sizing code_cell rendered">
+<div class="input">
+<div class="prompt input_prompt">In&nbsp;[&nbsp;]:</div>
+<div class="inner_cell">
+    <div class="input_area">
+<div class=" highlight hl-ipython3"><pre><span></span><span class="c1"># create a DataFrame of log likelihoods for the test word items</span>
+<span class="n">df_probs</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">DataFrame</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="n">probabilities</span><span class="p">)</span>
+<span class="n">df_probs</span><span class="o">.</span><span class="n">head</span><span class="p">()</span>
+</pre></div>
+
+</div>
+</div>
+</div>
+
+</div>
+    </div>
+  </div>
+</body>
+</html>

Term-I – AI Foundations/04 - ASL Recognizer/my_model_selectors.py

@@ -0,0 +1,161 @@
+import math
+import statistics
+import warnings
+
+import numpy as np
+from hmmlearn.hmm import GaussianHMM
+from sklearn.model_selection import KFold
+from asl_utils import combine_sequences
+
+class ModelSelector(object):
+    '''Base class for model selection (strategy design pattern).
+    '''
+
+    def __init__(self, all_word_sequences: dict, all_word_Xlengths: dict,
+                this_word: str, n_constant=3,
+                 min_n_components=2, max_n_components=10,
+                 random_state=14, verbose=False):
+        self.words = all_word_sequences
+        self.hwords = all_word_Xlengths
+        self.sequences = all_word_sequences[this_word]
+        self.X, self.lengths = all_word_Xlengths[this_word]
+        self.this_word = this_word
+        self.n_constant = n_constant
+        self.min_n_components = min_n_components
+        self.max_n_components = max_n_components
+        self.random_state = random_state
+        self.verbose = verbose
+        self.n_components = range(self.min_n_components, \
+                                    self.max_n_components + 1)
+
+    def select(self):
+        raise NotImplementedError
+
+    def base_model(self, num_states):
+        # with warnings.catch_warnings():
+        warnings.filterwarnings("ignore", category=DeprecationWarning)
+        # warnings.filterwarnings("ignore", category=RuntimeWarning)
+        try:
+            hmm_model = GaussianHMM(n_components=num_states,
+                                    covariance_type="diag", n_iter=1000,
+                                    random_state=self.random_state,
+                                    verbose=False).fit(self.X, self.lengths)
+            if self.verbose:
+                print("Model created for {} with {} states"\
+                        .format(self.this_word, num_states))
+            return hmm_model
+        except:
+            if self.verbose:
+                print("failure on {} with {} states".\
+                        format(self.this_word, num_states))
+            return None
+
+
+class SelectorConstant(ModelSelector):
+    """Select the model with value self.n_constant.
+    """
+
+    def select(self):
+        """Select based on n_constant value.
+
+        :return: GaussianHMM object
+        """
+        best_num_components = self.n_constant
+        return self.base_model(best_num_components)
+
+
+class SelectorBIC(ModelSelector):
+    """Select the model with the lowest Baysian Information Criterion (BIC)
+    score -- http://www2.imm.dtu.dk/courses/02433/doc/ch6_slides.pdf.
+
+    Bayesian information criteria: BIC = -2 * logL + p * logN
+    """
+
+    def select(self):
+        """Select the best model for self.this_word based on
+        BIC score for n between self.min_n_components and self.max_n_components
+
+        :return: GaussianHMM object
+        """
+        warnings.filterwarnings("ignore", category=DeprecationWarning)
+        bic_scores = []
+        try:
+            for n in self.n_components:
+                # BIC = −2 log L + p log N
+                # L = is the likelihood of the fitted model
+                # p = is the number of parameters
+                # N = is the number of data points
+                model = self.base_model(n)
+                log_l = model.score(self.X, self.lengths)
+                p = n ** 2 + 2 * n * model.n_features - 1
+                bic_score = -2 * log_l + p * math.log(n)
+                bic_scores.append(bic_score)
+        except Exception as e:
+            pass
+
+        states = self.n_components[np.argmax(bic_scores)] \
+                    if bic_scores else self.n_constant
+        return self.base_model(states)
+
+class SelectorDIC(ModelSelector):
+    """Select best model based on Discriminative Information Criterion
+
+    Biem, Alain. "A model selection criterion for classification:
+    Application to hmm topology optimization."
+
+    Document Analysis and Recognition, 2003. Proceedings.
+    Seventh International Conference on. IEEE, 2003.
+    http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.58.6208
+    DIC = log(P(X(i)) - 1/(M-1)SUM(log(P(X(all but i))
+    """
+
+    def select(self):
+        warnings.filterwarnings("ignore", category=DeprecationWarning)
+        dic_scores = []
+        logs_l = []
+        try:
+            for n_component in self.n_components:
+                model = self.base_model(n_component)
+                logs_l.append(model.score(self.X, self.lengths))
+            sum_logs_l = sum(logs_l)
+            m = len(self.n_components)
+            for log_l in logs_l:
+                # DIC = log(P(X(i)) - 1/(M-1)SUM(log(P(X(all but i))
+                other_words_likelihood = (sum_logs_l - log_l) / (m - 1)
+                dic_scores.append(log_l - other_words_likelihood)
+        except Exception as e:
+            pass
+
+        states = self.n_components[np.argmax(dic_scores)] \
+                    if dic_scores else self.n_constant
+        return self.base_model(states)
+
+class SelectorCV(ModelSelector):
+    """Slect best model based on average log Likelihood of cross-validation
+    folds.
+    """
+    def select(self):
+        warnings.filterwarnings("ignore", category=DeprecationWarning)
+        mean_scores = []
+        # Save reference to 'KFold' in variable as shown in notebook
+        split_method = KFold()
+        try:
+            for n_component in self.n_components:
+                model = self.base_model(n_component)
+                # Fold and calculate model mean scores
+                fold_scores = []
+                for _, test_idx in split_method.split(self.sequences):
+                    # Get test sequences
+                    test_X, test_length = combine_sequences(test_idx, \
+                                            self.sequences)
+                    # Record each model score
+                    fold_scores.append(model.score(test_X, test_length))
+
+                # Compute mean of all fold scores
+                mean_scores.append(np.mean(fold_scores))
+        except Exception as e:
+            pass
+
+        states = self.n_components[np.argmax(mean_scores)] \
+                    if mean_scores else self.n_constant
+        return self.base_model(states)

Term-I – AI Foundations/04 - ASL Recognizer/my_recognizer.py

@@ -0,0 +1,54 @@
+import warnings
+from asl_data import SinglesData
+
+
+def recognize(models: dict, test_set: SinglesData):
+    """Recognize test word sequences from word models set
+
+   :param models: dict of trained models
+       {'SOMEWORD': GaussianHMM model object,
+       'SOMEOTHERWORD': GaussianHMM model object, ...}
+
+   :param test_set: SinglesData object
+
+   :return: (list, list)  as probabilities, guesses
+       both lists are ordered by the test set word_id
+       probabilities is a list of dictionaries where each key a word and
+       value is Log Liklihood.
+           [{SOMEWORD': LogLvalue, 'SOMEOTHERWORD' LogLvalue, ... },
+            {SOMEWORD': LogLvalue, 'SOMEOTHERWORD' LogLvalue, ... },
+            ]
+       guesses is a list of the best guess words ordered by the test
+       set word_id.
+           ['WORDGUESS0', 'WORDGUESS1', 'WORDGUESS2',...]
+   """
+    warnings.filterwarnings("ignore", category=DeprecationWarning)
+    probabilities = []
+    guesses = []
+    #warnings.filterwarnings("ignore", category=DeprecationWarning)
+    probabilities = []
+    guesses = []
+
+    X_lengths = test_set.get_all_Xlengths()
+    
+    for X, lengths in X_lengths.values():
+        log_l = {} # Save the likelihood of a word
+        max_score = float("-inf") # Save max score as recognizer iterates
+        best_guess = None # Save best guess as recognizer iterates
+        for word, model in models.items():
+            try:
+                # Score word using model
+                word_score = model.score(X, lengths)
+                log_l[word] = word_score
+
+                if word_score > max_score:
+                    max_score = word_score
+                    best_guess = word
+            except:
+                # Unable to process word
+                log_l[word] = float("-inf")
+
+        guesses.append(best_guess)
+        probabilities.append(log_l)
+
+    return probabilities, guesses