Motivation

Machine learning and data science models are just one component of a larger process. For example, a price prediction model may be used to control the purchasing of raw materials, and a customer churn model may influence the way a company interacts with its customers. These models rely on pre-existing data and their predictions are used in various business processes and other models, making the overall process complex.

To simplify this complexity, our focus is on the model itself. We use pre-existing training data and optimize our model to meet intermediate goals such as forecast error or accuracy.

../../_images/train.png

In practice, the trained model is integrated into an existing process, where it receives live data and sends its predictions to the next step in the process.

../../_images/inference.png

While simplifying the complexity of data science projects can yield useful results, there are still several challenges to overcome.

  • Determining the appropriate goal for the model, e.g. whether model accuracy is the best measure for overall process outcome.

  • Incorporating live feedback into the process and adjusting the model accordingly.

  • Reusing learned models to optimize and control the input parameters of the process.

  • Effectively monitoring the model’s performance.

To address these challenges, we may add additional complexity to our process, such as separate components and code to handle feedback.

A more effective approach is to incorporate a direct feedback mechanism into our process, allowing our models and components to receive real-time information on their performance.

../../_images/swergio.png

A direct feedback mechanism allows for a wide range of applications, such as:

  • Training a model based on a goal further down the process (e.g. using differentiable programming in a Trebuchet example)

  • Collecting live reward feedback from an environment to train a reinforcement learning model (RL example)

  • Gathering live human feedback to label uncertain data in an active learning approach (Music Generator example)

  • Optimizing the input of our process using gradient-based optimization techniques (control problems)

The swergio package facilitates the implementation of forward and backward communication between independent components.

To maintain independence between components and avoid the need for all components to be aware of each other, communication is implemented through Web Sockets, and each component is only aware of specific communication rooms. Components can send and handle incoming messages.