A case for ML platform on MDB Atlas - (Part 1)

A case for ML platform on MDB Atlas - (Part 1)

PI Hackathon 2022

Overview

The end of 2022 brought a renewed interest in Machine Learning and AI capabilities. 2023 seems to be the year where every product out in the digital landscape would rush to include AI/ML capabilities.

Without AI capabilities built-in Atlas, MongoDB customers are forced to move their data from their instance to some other services like Big Query, etc, for AI/ML and data analysis. It tends to add additional operational, as well as, engineering intricacies, not to mention, the delays to push and pull the data in and out of security boundaries.

It seems obvious that a Machine Learning Platform will be in general customer's interest to be baked into MongoDB Atlas. MongoDB acquired Realm back in 2019, which brought charting capabilities to the platform. With Charts already available in the Atlas services Data Visualisation aspect of Machine Learning is already solved.

Wouldn't it be nice if we can just provide input collection with the fields that we want to predict and output collection or fields where the result of this prediction would end up, all from MongoDB Atlas? In the use case below, we tried to achieve the same:

XGBoost Prediction and Model Training

  1. Go to the MongoAI homepage.

  2. Navigate to Regression Card from the Marketplace section below.

  3. Select one of the Models to do the prediction. We selected one of the model built with xgboost, which is customized for MongoDB Time Series Data collections.

  4. Click the Card to Open the page below.

    1. Enter the required fields MongoURI, Time Series NS, Output NS, or field.

    2. Check the box if you want to use the data for retraining purposes.

    3. Click Predict.

On the backend, it would start the processing, validation, tokenization, and prediction process and the final prediction would be available in the Output NS which can be a collection or the same field in one of the Input NS.

Now with Charting capabilities inbuilt with Atlas Charts, we can visualize and analyze this data as required. Along with predicted data, the Trained Models are also pickled and saved in MongoDB.

Tech Stack

For details about the tech stack head over to the GitHub repos below and also check out individual services and submodules for details.

Predicting MongoAI's Future :)

  1. The Platform would also serve as a "Models Marketplace" where ML practitioners would test, deploy and sell their models. On the other side of the spectrum would be the customers consuming these APIs based on their requirements, all from MongoDB Atlas.

  2. We have only integrated one of the Microservices we built, with the UI. For all other available backend APIs head over to the next article, which also details
    API specs and a ChatGPT-backed extension for supporting ML developers to enrich the MongoAI marketplace ecosystem.

Code Repo

UI

github.com/SuperMohit/mongo-ai-ui

Backend

github.com/ms143desh/Hackathon2022