NAV Navbar

Examples Overview

This page contains working examples of Inkling code in conjunction with python simulator files. All of these examples and the libraries that accompany them can be found on BonsaiAI’s GitHub page and are also linked within each Example.

All of the OpenAI Gym, EnergyPlus and Gazebo examples can be trained in the cloud with Bonsai managed simulators. A full list of supported Docker containers for remotely managed simulators can be found in the Project File Reference.

If you have any suggestions of examples you’d like to see us implement please contact the support team.

Basic Walkthrough Examples

  • Basic Simulator: A project called Find the Center which walks you through how to create a simple Inkling file that connects to a basic python simulator.

OpenAI Gym Examples

  • Cart Pole: A simple control problem where a cart must make a simple move to keep a pole balanced on top.
  • Mountain Car: A simple control problem where a car must build up enough momentum to climb a hill.

Real World Examples

  • HVAC with EnergyPlus: An example of climate control machine teaching using EnergyPlus and BCVTB for simulation.
  • Gazebo Simulated Turtlebot: A robotics example of a Turtlebot robot navigating an environment in Gazebo.

Basic Simulator (Find the Center)

Find the Center Diagram

Download the full source code on GitHub if you want to run this simulator locally.

In this example, we’ll walk you through the various statements that are part of the Find the Center game, including the python simulator file and the Inkling file. This is a very basic example of Inkling and how to connect to a custom python simulator.

Find the Center is a simple game where the AI seeks the average value between two numbers. In this game, the AI begins at a random value of 0, 1, or 2. The AI then can move to a lower number by outputting -1, a higher number by outputting +1, or staying on the same number by outputting 0. The goal of Find the Center is to remain in the center of 0 and 2 (the number 1).

Inkling File

Schema
schema GameState
    Int8 value
end

The GameState schema has one field, value, with type Int8.

schema PlayerMove
    Int8{-1, 0, 1} delta
end

The PlayerMove schema has one field, delta, with type Int8. The Int8 type is constrained to three possible values: -1, 0, and 1.

schema SimConfig
    Int8 dummy
end

The SimConfig schema has one field, dummy, because there is no configuration needed in this particular example.

Concept
concept find_the_center
    is classifier
    predicts (PlayerMove)
    follows input(GameState)
    feeds output
end

This concept is named find_the_center. find_the_center expects input about the state of the game (defined by the GameState schema) and replies with output defined by the PlayerMove schema. This is the AI’s next move in the simulation.

Simulator
simulator find_the_center_sim(SimConfig)
    action (PlayerMove)
    state (GameState)
end

The simulator is called find_the_center_sim (shown in #simulator-file) and takes the schema input of SimConfig (even though it isn’t configuring anything, it’s required by the simulator). The find_the_center concept will be trained using the find_the_center_sim simulator. To define the training relationship between the simulator and the concept we must begin by defining the simulator. find_the_center_sim expects an action defined in the PlayerMove schema as input and replies with a state defined in the GameState schema as output.

Curriculum
curriculum find_the_center_curriculum
    train find_the_center
    with simulator find_the_center_sim
    objective time_at_goal
        lesson seek_center
            configure
                constrain dummy with Int8{-1}
            until
                maximize time_at_goal
end

The curriculum is named find_the_center_curriculum, and it trains the find_the_center concept using the find_the_center_sim.

This curriculum contains one lesson, called seek_center. It configures the simulation, by setting a number of constraints for the state of the simulator. The lesson trains until the AI has maximized the objective time_at_goal.

Simulator File

from __future__ import print_function
import bonsai
import sys
from bonsai.simulator import SimState
from random import randint

""" If you would like to debug your code add this back in.
def debug(*args):
    print(*args, file=sys.stderr)
"""


class BasicSimulator(bonsai.Simulator):
    """ A basic simulator class that takes in a move from the inkling file,
    and returns the state as a result of that move.
    """

    min = 0
    max = 2
    goal = 1

    def __init__(self):
        super(BasicSimulator, self).__init__()
        self.goal_count = 0
        self.value = randint(self.min, self.max)
        self.old_value = self.min

    def get_terminal(self):
        """ Restarts the simulation if the AI moved out of bounds.
        """
        if (self.value < self.min or self.value > self.max):
            # (self.value == self.goal and self.old_value == self.goal)):

            # debug("terminal")
            self.reset()
            return True
        else:
            return False

    def start(self):
        """ Start the episode by initializing value to a random number
        between the min and max.
        """
        # debug("start")
        self.goal_count = 0
        self.old_value = self.min
        self.value = randint(self.min, self.max)

    def stop(self):
        """ Stop is called after a training session is complete.
        This simple game requires no cleanup after training.
        """
        # debug("stop")
        pass

    def reset(self):
        """ Reset is called to reset simulator state before the next training session.
        """
        # debug("reset")
        self.goal_count = 0
        self.old_value = self.min
        self.value = randint(self.min, self.max)

    def advance(self, actions):
        """ Function to make a move based on output from the BRAIN as
        defined in the Inkling file.
        """
        # debug("advance", actions["delta"])
        self.value += actions["delta"]
        if self.value == self.goal:
            self.goal_count += 1

    def get_state(self):
        """ Gets the state of the simulator, whether it be a valid value or
        terminal ("game over") state.
        """
        # debug("get_state")
        # debug("state", self.value)
        self.old_value = self.value
        return SimState(state={"value": self.value},
                        is_terminal=self.get_terminal())

    def time_at_goal(self):
        """ Function to return how long the simulation has maintained the goal.
        """
        return self.goal_count

if __name__ == "__main__":

    sim = BasicSimulator()
    bonsai.run_for_training_or_prediction("find_the_center_sim", sim)

This is a basic python simulator for learning the simulator library. In this case it is used to find the center between two numbers, 0 and 2. The goal, as outlined in the Inkling file, is to reach 1. The moves that the simulator is able to make are sent from the Inkling file to the simulator and the state of the simulator is sent back to Inkling.

OpenAI Gym: Cart Pole

Cart Pole Balance

Download the full source code on GitHub if you want to run this simulator locally. If you want to run Cart Pole remotely on the Bonsai Platform as a managed simulator, create a new BRAIN selecting the Cart Pole demo on beta.bons.ai.

In this example, we’ll walk you through the various statements that are part of the Cart Pole Inkling file. Each statement is followed by an explanation of the statement.

Cart Pole is a classic control problem. OpenAI Gym describes it as:

A pole is attached by an un-actuated joint to a cart, which moves along a frictionless track. The system is controlled by applying a force of +1 or -1 to the cart. The pendulum starts upright, and the goal is to prevent it from falling over. A reward of +1 is provided for every timestep that the pole remains upright. The episode ends when the pole is more than 15 degrees from vertical, or the cart moves more than 2.4 units from the center.

Inkling File

Schema
schema GameState
    Float32 position,
    Float32 velocity,
    Float32 angle,
    Float32 rotation
end

The schema GameState names four records — position, velocity, angle, and rotation — and assigns a type to them. This information is input from the simulation.

schema Action
    Int8{0, 1} command
end

The schema Action names a record — action — and assigns it a constrained type.

schema CartPoleConfig
    Int8 episode_length,
    UInt8 deque_size
end

The schema CartPoleConfig names two records — episode_length and deque_size — and assigns each of them a type. episode_length is a signed Int8 because -1 is used for “run until pole drops”.

Concept
concept balance is classifier
    predicts (Action)
    follows input(GameState)
    feeds output
end

The concept is named balance, and it takes input from the simulator. That input is the records in the schema GameState. The balance concept outputs the move the AI should make in the simulator. This output is the record in the Action schema.

Simulator
simulator cartpole_simulator(CartPoleConfig) 
    action (Action)
    state (GameState)
end

The cartpole_simulator gets information from two schemas. The first schema, CartPoleConfig, specifies the schema for configuration of the simulation. The second schema contains the state of the simulator that is sent to the lesson.

Curriculum
curriculum balance_curriculum
    train balance
    with simulator cartpole_simulator
    objective open_ai_gym_default_objective

        lesson balancing
            configure
                constrain episode_length with Int8{-1},
                constrain deque_size with UInt8{1}
            until
                maximize open_ai_gym_default_objective
end

The curriculum’s name is balance_curriculum. It trains the balance concept with the cartpole_simulator. The objective for this curriculum is up_time. The objective measures how long the pole stays upright.

This curriculum contains one lesson, called balancing. It configures the simulation, by setting two constraints for the state of the simulator. The lesson trains until the AI has maximized the objective.

Simulator File

import gym

import bonsai
from bonsai_gym_common import GymSimulator, logging_basic_config

ENVIRONMENT = 'CartPole-v0'
RECORD_PATH = None


class CartPoleSimulator(GymSimulator):

    def __init__(self, env, record_path):
        super(CartPoleSimulator, self).__init__(env, record_path=record_path)

    def get_state(self):
        parent_state = GymSimulator.get_state(self)
        state_dict = {"position": parent_state.state[0],
                      "velocity": parent_state.state[1],
                      "angle": parent_state.state[2],
                      "rotation": parent_state.state[3]}
        return bonsai.simulator.SimState(state_dict, parent_state.is_terminal)


if __name__ == "__main__":
    logging_basic_config()
    env = gym.make(ENVIRONMENT)
    simulator = CartPoleSimulator(env, RECORD_PATH)
    bonsai.run_for_training_or_prediction("cartpole_simulator", simulator)

This is an OpenAI Gym example which uses the OpenAI environment as its simulator. For more information about the simulator used see the Bonsai Gym Common GitHub repo which is a python library for integrating a Bonsai BRAIN with Open AI Gym environments.

OpenAI Gym: Mountain Car

Mountain Car Control

Download the full source code on GitHub if you want to run this simulator locally. If you want to run Mountain Car remotely on the Bonsai Platform as a managed simulator, create a new BRAIN selecting the Mountain Car demo on beta.bons.ai

We’ve used pieces of code from this example in several places, but here we’ll walk you through all the various statements that are part of the Mountain Car Inkling file. Each statement is followed by an explanation of the statement.

Mountain Car is a classic control problem. OpenAI Gym describes it as:

A car is on a one-dimensional track, positioned between two “mountains”. The goal is to drive up the mountain on the right; however, the car’s engine is not strong enough to scale the mountain in a single pass. Therefore, the only way to succeed is to drive back and forth to build up momentum.

Inkling File

Schema
schema GameState
    Float32 x_position,
    Float32 x_velocity
end

The GameState schema names two records — x_position and y_position — and assigns a type to them.

schema Action
    Int8{0, 1, 2} command
end

The Action schema names a single record — action — and assigns a constrained type to it.

schema MountainCarConfig
    Int8 episode_length,
    UInt8 deque_size
end

The MountainCarConfig schema names two records — episode_length and deque_size — and assigns types to them.

Concept
concept high_score is classifier
    predicts (Action)
    follows input(GameState)
    feeds output
end

The concept is named high_score, and it takes input from the simulator about the state of the game (GameState schema). It outputs to the Action schema. This is the AI’s next move in the game.

Simulator
simulator mountaincar_simulator(MountainCarConfig)
    action (Action)
    state (GameState)
end

The mountaincar_simulator gets information from two schemas. The first schema, MountainCarConfig, specifies the schema for configuration of the simulation. The second schema contains the state of the simulator that is sent to the lesson.

Curriculum
curriculum high_score_curriculum
    train high_score
    with simulator mountaincar_simulator
    objective open_ai_gym_default_objective

        lesson get_high_score
            configure
                constrain episode_length with Int8{-1},
                constrain deque_size with UInt8{1}
            until
                maximize open_ai_gym_default_objective
end

The curriculum is named high_score_curriculum, and it trains the high_score concept using the mountaincar_simulator. This curriculum contains one lesson, called get_high_score. It configures the simulation, by setting two constraints for the state of the simulator.

The lesson trains until the AI has maximized the objective named score.

Simulator File

import gym

import bonsai
from bonsai_gym_common import GymSimulator

ENVIRONMENT = 'MountainCar-v0'
RECORD_PATH = None
SKIPPED_FRAME = 4

class MountainCarSimulator(GymSimulator):

    def __init__(self, env, skip_frame, record_path):
        GymSimulator.__init__(
            self, env, skip_frame=skip_frame,
            record_path=record_path)

    def get_state(self):
        parent_state = GymSimulator.get_state(self)
        state_dict = {"x_position": parent_state.state[0],
                      "x_velocity": parent_state.state[1]}
        return bonsai.simulator.SimState(state_dict, parent_state.is_terminal)

if __name__ == "__main__":
    env = gym.make(ENVIRONMENT)
    simulator = MountainCarSimulator(env, SKIPPED_FRAME, RECORD_PATH)
    bonsai.run_for_training_or_prediction("mountaincar_simulator", simulator)

This is an OpenAI Gym example which uses the OpenAI environment as its simulator. For more information about the simulator used see the Bonsai Gym Common GitHub repo which is a python library for integrating a Bonsai BRAIN with Open AI Gym environments.

EnergyPlus HVAC Optimization

EnergyPlus Graph

Download the full source code on GitHub if you want to run this simulator locally. If you want to run EnergyPlus remotely on the Bonsai Platform as a managed simulator, create a new BRAIN selecting the EnergyPlus demo on beta.bons.ai.

In this example, we’ll walk you through the various statements that are part of a sample implementation of EnergyPlus on the Bonsai Platform, including the simulator and the Inkling files. This is a real-world example of how to use the Bonsai Platform for HVAC control using BCVTB and EnergyPlus.

While this BRAIN is training, the Bonsai AI Engine launches the EnergyPlus simulator in the background for every episode. The energyplus_simulator.py then drives the simulator forward a step at a time until it finishes the episode and then relaunches it for the next episode, driving the actions into it and sending state results back to the Bonsai AI Engine.

Inkling File

Schema
schema SimState
    Int32{0:10} SolarIrradiation
end

The SimState schema defines the dictionary returned from the Python simulation’s advance method to the BRAIN.

schema SimAction
    Int32 {0, 1} shade
end

The SimAction schema defines the ‘actions’, a dictionary of control signals this AI can send to the climate control. For example: shade == night, off, day.

schema SimConfig
    Int32{-1} unused
end

The SimConfig schema in this case is not used (but is still required to be defined in Inkling) but it would define the dictionary passed as a parameter to the set_properties method of the Python simulator.

Concept
concept my_concept is classifier
   predicts (SimAction)
   follows input(SimState)
   feeds output
end

This concept is named my_concept which predicts a SimAction given a SimState. In this simple demo we just ask the Bonsai Platform to generate any model that can learn to control the server using these inputs and outputs.

Simulator
simulator energyplus_simulator(SimConfig)
    action (SimAction)
    state (SimState)
end

The simulator clause declares that a simulator named energyplus_simulator will be connecting to the server for training. This code snippet binds the previous schemas to this simulator. To define the training relationship between the simulator and the concept we must begin by defining the simulator. energyplus_simulator expects an action defined in the SimAction schema as input and replies with a state defined in the SimState schema as output.

Curriculum
curriculum my_curriculum
    train my_concept
    with simulator energyplus_simulator
    objective reward_function
        lesson my_first_lesson
            configure
                constrain unused with Int32{-1}
            until
                maximize reward_function
end

The curriculum my_curriculum trains my_concept using energyplus_simulator. The BRAIN that runs this Inkling code will try to maximize the value returned from reward_function until you stop training. reward_function is a method in the Python simulator.

This curriculum contains one lesson, called my_first_lesson. It configures the simulation, by setting a number of constraints for the state of the simulator.

Simulator Excerpt

# Excerpt of simulator class from the energyplus_simulator.py file

class EnergyPlusSimulator(Simulator):
    model = ePlus85Actuator()
    server = None

    clientState = {'SolarIrradiation': 0}
    shade = 0.
    is_terminal = True

    def start(self):
        """This method is called when training is started."""
        print("EnergyPlusSimulator: start")

    def stop(self):
        print("EnergyPlusSimulator: stop")

        graph = self.model.grapher()
        py.plot(graph, filename="graph.html")

    def readFromPtolemyClient(self):
        self.server.readFromClient()
        if self.model.fromClient and len(self.model.fromClient) == 4:
            self.clientState = {
                # 'TOut': self.model.fromClient[0],
                # 'TZone': self.model.fromClient[1],
                'SolarIrradiation': int(self.model.fromClient[2])/100
                # 'FractionShadingOn': self.model.fromClient[3]
                }

            # save the client input in our graph
            for n in range(len(self.model.fromClient)):
                value = self.model.fromClient[n]
                # scale some of the values for readability
                if n == 2:
                    value /= 100.
                self.model.data[n].append(value)

        self.is_terminal = self.model.exitFlag != 0

    def restartPtolemyServer(self):
        # set some default values for get_state
        self.is_terminal = True
        # self.clientState = {'TOut': 0.,
        #                     'TZone': 0.,
        #                     'SolarIrradiation': 0.,
        #                     'FractionShadingOn': 0. }
        self.clientState = {'SolarIrradiation': 0}

        # close the old connections if they're still open
        if self.server:
            self.server.close()

        # start a new episode
        print("EnergyPlusSimulator: starting PtolemyServer")
        self.server = PtolemyServer(self.model)

        try:
            self.server.start()
            self.server.waitForClient()
            # get initial state
            self.readFromPtolemyClient()

        except OSError as msg:
            print("EnergyPlusSimulator: error on restart:", msg)
            self.server = None

    def reset(self):
        """Called by the AI Engine to reset simulator state in between training
           and test passes.
        """
        print("EnergyPlusSimulator: reset")

    def advance(self, actions):
        print("EnergyPlusSimulator: advance ", actions)
        """Advance the simulation forward one tick. actions contains a
           dictionary of key values as defined by this simulator's action
           schema in Inkling.
        """
        self.shade = actions['shade'] * 6.  # Int32[0..1]

    def set_properties(self, **kwargs):
        print("EnergyPlusSimulator: set_properties")
        """This method is called before training is started
           or on the frame after is_terminal=True to set
           configuration properties in this simulation. See
           the configure clause of the lesson statement in
           this simulator's accompanying curriculums.
        """
        self.restartPtolemyServer()

    def get_state(self):
        """Returns a named tuple of state and is_terminal. state is a
           dictionary matching the state schema as defined in Inkling.
           is_terminal is only true when the simulator is in a "game over"
           state.
        """

        print("EnergyPlusSimulator: get_state: terminal:", self.is_terminal)

        if self.is_terminal:
            self.restartPtolemyServer()
        else:
            self.server.writeToClient([self.shade])
            self.readFromPtolemyClient()

        # you like graphs? WE HAVE GRAPHS. SO MANY GRAPHS.
        if self.is_terminal:
            graph = self.model.grapher()
            write_graph(graph)

            # clear old data
            self.model.data = ([], [], [], [], [])

        return SimState(state=self.clientState, is_terminal=self.is_terminal)

    def reward_function(self):
        print("EnergyPlusSimulator: reward_function")
        # largest reward is best reward (maximize)
        reward = 0.
        if self.model.fromClient and len(self.model.fromClient) == 4:
            # SolarIrradiation === Shades down === good
            # TOut = self.model.fromClient[0]
            SolarIrradiation = self.model.fromClient[2] / 100.

            # sun is down
            if SolarIrradiation <= 1:
                if self.shade > 0:
                    reward = -1  # shades on
                else:
                    reward = 1  # shade off

            # sun is out
            else:
                if self.shade > 0: 
                    reward = 1  # shades on
                else:
                    reward = -1  # shades off

            self.model.data[4].append(reward)

        print("EnergyPlusSimulator reward:", reward)
        return reward

The full simulator file energyplus_simulator.py for this example is with the rest of the energyplus-sample code on GitHub.

This is a Python simulator for integrating the EnergyPlus simulator into the Bonsai AI Engine. This energyplus_simulator.py file repeatedly runs the EnergyPlus simulator in the background with new actions sent from the Bonsai AI Engine by passing the state from EnergyPlus to the backend, and the action from the backend back to EnergyPlus.

For more information on the functions inside of this simulator class and how to implement them see the Library Reference.

Gazebo: Turtlebot

Turtlebot Demo Image

Download the full source code on GitHub if you want to run this simulator locally. If you want to run Gazebo Turtlebot remotely on the Bonsai Platform as a managed simulator (highly recommended), create a new BRAIN selecting the Gazebo Turtlebot demo on beta.bons.ai.

In this example, we’ll walk you through the various Inkling components and simulation scripts that are part of a sample implementation of training Turtlebot to navigate the Willow Garage office on the Bonsai Platform. Turtlebot is a low-cost mobile base for basic robotics and AI-driven robotics R&D. This is a real-world example of how to use the Bonsai Platform for robotics using Gazebo and ROS (Robotic Operating System).

This example also includes scripts for integrating Gazebo and ROS into the Bonsai Platform which can be found below in the Simulator Files section. In this case, ROS is relaying information between the simulator Gazebo and the Bonsai Platform. Turtlebot is being simulated inside of Gazebo, and ROS is communicating the action and state of the simulation back to the Bonsai servers for training.

A graphical display is not part of the Docker container (when training remotely), but a Gazebo client may additionally connect to the Gazebo server (that is part of the container) to visualize the simulation environment.

Inkling File

Schema
schema State
    Float32 heading_x,
    Float32 heading_y,
    Float32 heading_yaw,
    Float32 yaw,
    Float32 x,
    Float32 y
end

The State schema provides the BRAIN with the current x, y coordinates of the turtlebot, as well as its orientation. We also supply a heading vector to the relative orientation of the goal with respect to the Turtlebot. This provides an invariant representation across goal positions.

schema Command
    Float32 lin_vel,
    Float32 ang_vel
end

The Command schema defines lin_vel as the linear velocity and ang_vel as the angular velocity of Turtlebot in the simulation. These are the two actions that can be taken to move Turtlebot. Our BRAIN will output the desired linear and angular velocities in m/s and rad/s, respectively. A PID controller will make a best effort to achieve these targets within the time limit of the iteration.

schema TurtlebotConfig
    Int8 unused
end

The TurtlebotConfig schema in this case is not used (but is still required to be defined in Inkling).

Concept
concept go_to_point is estimator
    predicts (Command)
    follows input(State)
    feeds output
end

This concept is named go_to_point, which predicts a Command given a State of the simulation. This concept of go_to_point is based on the x, y position set as the goal.

Simulator
simulator turtlebot_office_simulator(TurtlebotConfig)
    action (Command)
    state (State)
end

The simulator clause declares that a simulator named turtlebot_office_simulator will be connecting to the server for training. This code snippet binds the previous schemas to this simulator. To define the training relationship between the simulator and the concept we must begin by defining the simulator. turtlebot_office_simulator expects an action defined in the Command schema as input and replies with a state defined in the State schema as output.

Curriculum
curriculum go_to_point_curriculum
    train go_to_point
    with simulator turtlebot_office_simulator
    objective get_reward
        lesson get_to_point
            configure
                constrain unused with Int8{-1}
            until
                maximize get_reward
end

The curriculum go_to_point_curriculum trains go_to_point using turtlebot_office_simulator. The BRAIN that runs this Inkling code will try to maximize the value returned from get_reward until you stop training. get_reward is a method in the Python simulator.

This curriculum contains one lesson, called get_to_point. If used, it would configure the simulation by setting a number of constraints for the state of the simulator. In this simple example however, there are no constraints.

Simulator Files

The three full Python files for this example are with the rest of the gazebo-turtlebot-sample code on GitHub.

bonsai_ros.py

Source code too long to display, please click Github link.

This Python file mainly contains a class that acts as a basic interface between ROS and Bonsai’s protocols. It can bring up and down the ROS stack and parse a variety of basic messages.

The purpose of this file is to manage the interactions between the BRAIN server interface and many ROS topics that are available to subscribe or publish to.

See the bonsai_ros.py file on GitHub.

bonsai_gazebo.py

Source code too long to display, please click Github link.

This Python file extends the BonsaiROS interface with methods for controlling Gazebo simulations in accordance with the Bonsai simulator protocol.

The purpose of this file is to extend bonsai_ros.py to also handle interacting with the Gazebo specific services and topics made available when using a Gazebo simulation with ROS.

See the bonsai_gazebo.py file on GitHub.

turtlebot_office_simulator.py

Source code too long to display, please click Github link.

This is a Python file for integrating the Gazebo simulator through ROS into the Bonsai AI Engine. This contains the main Bonsai-specific simulator functionality, which is the main interface that communicates with the Bonsai AI engine to drive AI training and prediction.

This file repeatedly runs each “episode” and the State of the simulation in Gazebo gets passed through ROS to the Bonsai AI Engine which then proceeds with the next Command for Turtlebot and passes that back through ROS to Gazebo.

See the turtlebot_office_simulator.py file on GitHub.