# Building a Tech Course Curriculums Platform with MindsDB Models and Django

We often find ourselves overwhelmed by the abundance of free and paid online tech courses and resources. That's why today we will build a platform that curates the best tech course curriculums and organizes them based on skill level, topic, and user feedback. To improve the user experience and accelerate app growth, We will integrate MindsDB models for sentiment and text classification into the platform. In this tutorial, We will walk through MindsDB models and Django are used for the backend, and Next.js for the frontend to build this platform.

## What is MindsDB?

MindsDB is an open-source, automated machine-learning platform that enables developers to build and integrate machine-learning models into their applications quickly and easily. MindsDB provides a simple interface that allows you to train, test, and deploy machine learning models without having to write any code. It also supports a wide range of use cases, such as classification, regression, and time series forecasting.

## What's the Problem?

New programmers end up in something called the [Tutorial Hell](https://medium.com/codex/what-is-tutorial-hell-and-how-to-avoid-it-to-improve-as-a-developer-8c5376c97011?source=rss------technology-5#:~:text=Tutorial%20hell%20is%20a%20concept,a%20new%20language%20or%20technology.). Whereby they keep watching and reading various online tutorials from various content creators. Tutorials are good but most don't cover all the aspects a new programmer needs to suceed in a skill.

## What's the Solution?

This project is built with the goal of helping new programmers find quality tech curriculums and resources. The platform curates the best resources and organizes them into curriculums based on user feedback. To improve the user experience and accelerate app growth, the platform integrates MindsDB models for sentiment and text classification on user reviews.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1682895270482/477b644d-37d2-4d91-846c-ac34d1fc9da5.png align="center")

## Why do we need to classify Reviews?

When users are progressing through a curriculum or have completed it. Their reviews are important so that we can use them to reorganize the curriculums based on user needs and their learning experience.

Using AI to classify user reviews sentimental can help us categorize or filter reviews by priority and what we need to work on ASAP. Doing this manually would be time-consuming and slow down problem analysis and updates.

## Why MindsDB Models?

In this project, we used MindsDB models for sentiment and text classification to classify user curriculum reviews. The user reviews are classified by labels and sentiment. MindsDB models are used because it's easier to integrate than to install Pytorch and LLM (Large Language models).

We can train our models right in the MindsDB SQL editor. Yes, we will use SQL queries to train our AI models. [MindsDB NLP engine](https://docs.mindsdb.com/nlp/nlp-mindsdb-hf) is currently powered by Hugging Face and Open AI. It supports NLP tasks like Text Classification and Zero-Shot Classification.

By using MindsDB Python SDK, we can easily connect to our MindsDB Cloud Server and execute AI queries on our pre-trained models with just a few lines of Python code. MindsDB's large community and learning resources can help us to find solutions on how to use them to solve any problems we might encounter.

## Building The Project

We will cover the project build-up from the DB design to the product mvp. The repository for the completed project [Backend API](https://github.com/devvspaces/roadflow-api) and [Frontend Web](https://github.com/devvspaces/roadflow-web) contain installation guides to set up the project and start using it.

## Project Prerequisites

To follow through with this article, we need:

* A [MindsDB](https://mindsdb.com/) cloud account. Create a free cloud account at [**cloud.mindsdb.com**](http://cloud.mindsdb.com)**.**
    
* [Python](https://python.org/) and [NodeJs](https://nodejs.org/en) installed.
    
* [Redis](https://redis.io/) for caching
    
* Basic knowledge of [Django](https://www.djangoproject.com/), [Django Rest Framework](https://www.django-rest-framework.org/), [React](https://react.dev/), and [NextJs](https://nextjs.org/) frameworks
    

> Django Memcache can be configured instead of redis if problem occurs when installing redis.

Let's start building 🤓💻

## Productivity Foundations

### Database Design

First of we need to have an understanding of how the Database will be structured. By using [dbdiagram.io](https://dbdiagram.io/), we can design relation database structures. Here is a [database diagram](https://dbdiagram.io/d/6437cb1c8615191cfa8d9bc1) that we can use.

![](https://github.com/devvspaces/roadflow-api/raw/dev/Roadflow%20DB.png align="left")

This can help us get clarity on how the project will be structured. That's like 50% of the problem-solving done here. We would find out tables related to each other and how to categorize the apps in Django. This saved will save a lot of time instead of jumping on the project without direction.

Need the DB Schema used to generate the DB diagram - visit [here](https://dbdiagram.io/d/6437cb1c8615191cfa8d9bc1).

### Flow Design

Here is the link to the [Figma jam](https://www.figma.com/file/4sWAOaXGdd16N5AlyFVSBl/RoadTrack-Project?type=whiteboard&node-id=0%3A1&t=83f2QIdz8PNYJJ4D-1).

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1682941158904/093c3c1d-5005-4a5f-b812-a4e971ce5c05.png align="center")

Having a simple design for the application flow will go a long way to help us have a working application in our minds. Also finding solutions to other basic problems like the best authentication method to use, how to organize the API, and how the user will use the application.

## Rest API Development

Django and Django Rest Framework are used for API development and documentation. Swagger allows us to test the API endpoints from the documentation itself. The [backend repository](https://github.com/devvspaces/roadflow-api) has an installation and usage guide in README.md for setting up the project and testing it with ease.

The API is classified into three major sections:

* **Account** - Authentication and Authorization
    
* **Curriculum** - Organizing curriculums, enrollments, quizzes, and reviews.
    
* **MindsDB Classify** - Classifying texts by sentiments and labels.
    

### Account

When a user registers, they will get an OTP in their email to verify their email. The user's OTP is cached for future validations. After validation, the OTP is deleted from the cache. This Django app has the following features:

1. Register
    
2. Login
    
3. Forget password
    
4. Change password
    
5. Updating user profile
    

We used [JWT](https://jwt.io/) (JSON Web Tokens) for user authentication. This allows us to have a stateless authentication that the frontend can easily manage. [Simple JWT](https://django-rest-framework-simplejwt.readthedocs.io/en/latest/) allows us to use JWT in Django Rest Framework.

### Curriculums

This is the main application in the Django project. This application supports the following features:

* user enrollments, progress tracking,
    
* quiz creation, quiz submission,
    
* curriculum, syllabus, resources creation,
    
* reviews submission and classification.
    

The backend repository contains comments that explain how the Models, Views, and Serializers work.

Basically, the user can search for curriculums and enroll in them. An enrolled user can progress through the curriculum syllabus by completing quizzes for each lesson. At the end or anytime while taking the curriculum they can submit reviews any kind of reviews. When a new review is submitted it is classified and updated in the database.

## NLP Classification using MindsDB

Our reviews are classified and stored in the `CurriculumReview` table.

```python
class CurriculumReview(models.Model):
    """Curriculum Review Model"""

    SENTIMENT = (
        ('POS', 'Positive'),
        ('NEG', 'Negative'),
        ('NEU', 'Neutral'),
    )

    LABEL = (
        ('A', 'Course Content'),
        ('B', 'Exercises'),
        ('C', 'Course Structure'),
        ('D', 'Learning Experience'),
        ('E', 'Support'),
    )

    enrollment = models.ForeignKey(
        CurriculumEnrollment, on_delete=models.CASCADE)
    rating = models.PositiveSmallIntegerField(
        default=0,
        validators=[MinValueValidator(1), MaxValueValidator(5)]
    )
    review = models.TextField(null=False, blank=False)
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)
    sentiment = models.CharField(
        max_length=3, null=True, blank=True,
        choices=SENTIMENT
    )
    label = models.CharField(
        max_length=1, choices=LABEL, null=True, blank=True)
```

For the sentiment labels, we have:

```python
SENTIMENT = (
    ('POS', 'Positive'),
    ('NEG', 'Negative'),
    ('NEU', 'Neutral'),
)
```

For the Zero-Shot classification labels, we have:

```python
LABEL = (
    ('A', 'Course Content'),
    ('B', 'Exercises'),
    ('C', 'Course Structure'),
    ('D', 'Learning Experience'),
    ('E', 'Support'),
)
```

These can then be used as Django Charfield choices, which we can configure in the Django admin for filtering. Allowing us to filter reviews by their labels.

Also, we will see how can reuse this when querying our MindsDB server.

### Creating and Training the MindsDB AI Models

First of all, the MindsDB documentation is packed with resources to understand how we can use it.

Today we will be using the following NLP models from [Hugging Face](https://huggingface.co/):

* **Sentiment Classification** - `cardiffnlp/twitter-roberta-base-sentiment`
    
* **Zero-Shot Classification** - `facebook/bart-large-mnli`
    

As we have read earlier, MindsDB works seamlessly with Hugging Face NLP models. By running SQL queries in our MindsDB Cloud SQL editor we can easily create and train these models to be used in our project.

1. To access our Cloud editor we need to sign in - [https://cloud.mindsdb.com/login](https://cloud.mindsdb.com/login). Don't have an account yet? Create one here [https://cloud.mindsdb.com](https://cloud.mindsdb.com/login).
    
2. After signing in, we can see our free demo instance is already set up and running. Click it to access the SQL Editor.
    
3. ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1682943961569/c0b23f58-d68b-4cd5-8cb5-04ecad3c4e20.png align="center")
    
    On the right panel in the SQL Editor, we can find the Learning Hub with tutorials and resources for using MindsDB SQL Editor.
    
4. ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1682944122956/8b1c0613-42b0-4a25-8ed0-3e6a32e1cbaa.png align="center")
    
    We can create files, databases, and projects in the SQL Editor. By default, we have a project, mindsdb, already created for us. A project is a collection of data that includes all the necessary information for building, training, and testing a machine-learning model.
    

**A. Training the Sentiment model**

Copy and paste the SQL statement below into the editor.

```sql
CREATE MODEL mindsdb.hf_sentiment
PREDICT PRED
USING
engine = 'huggingface',
task = 'text-classification',
model_name = 'cardiffnlp/twitter-roberta-base-sentiment',
input_column = 'text',
labels = ['Negative', 'Neutral', 'Positive'];
```

> The labels are from the labels we want to use for sentiment in our Django project.

In `mindsdb.hf_sentiment` - `mindsdb` is the project name while `hf_sentiment` is the model name. We can use any name for the model name, but we need to take note of it because it will be required in the Django project to access the model.

To run the SQL statement, we need to highlight it in the SQL editor and click on `Run` towards the top of the screen or `Shift + Enter`.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1682944858571/a89f81d6-bee5-480a-bbb0-5e8a3a7cced2.png align="center")

Running the statement creates a new model. To check the status of the hf\_sentiment model, run the following statement.

```sql
SELECT *
FROM mindsdb.models 
WHERE name = 'hf_sentiment';
```

Status = `complete` tells us that the model training is complete.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1682945159793/d78c05be-2640-4614-a803-4c274e7dffe7.png align="center")

To test the model, we run the statement below

```sql
SELECT *
FROM mindsdb.hf_sentiment
WHERE text = 'I like you. I love you.';
```

This results in a response that contains the prediction `PRED_explain`.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1682945282331/fe676180-a008-41fc-a9c5-d9a2922bb6d5.png align="center")

The prediction tells us that the text sounds more Positive than Neutral or Negative. In our Django project, we will need to write a program that retrieves the label with the maximum result.

**B. Training the Zero-Shot classification model**

Using what we have learned when creating the sentiment model. Let's create one for Zero-Shot too using a similar process.

**Create** -> **Check Status** -> **Test**

To create the model, run the following statement.

```sql
CREATE MODEL mindsdb.hf_classify
PREDICT PRED
USING
engine = 'huggingface',
task = 'zero-shot-classification',
model_name = 'facebook/bart-large-mnli',
input_column = 'text',
candidate_labels = ['Course Content', 'Exercises', 'Course Structure', 'Learning Experience', 'Support'];
```

> We also used our Django project labels as the `hf_classify` labels here too. Also don't forget to note the model name `hf_classify`.

To check for the status of the newly created model, run the following statement.

```sql
SELECT *
FROM mindsdb.models 
WHERE name = 'hf_classify';
```

Wait until it's completed training.

To test the model, run the following statement.

```sql
SELECT *
FROM mindsdb.hf_classify
WHERE text = 'This is a nice curriculum. Explains a lot about HTML tags and css design. But needs more video contents. Thanks.';
```

We should get the following response.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1682946263170/071bb18e-784b-4b71-8152-60fb5641d2e9.png align="center")

Telling us that this is a review that tells us how the user feels about the Course Content.

### Integrating MindsDB with Python

After following the installation guide in the [backend repository README.md](https://github.com/devvspaces/roadflow-api/blob/dev/README.md), we should have installed the required dependency for integrating MindsDB because it's included in the requirements.txt file in the repository.

This is the MindsDB Python SDK. To install it explicitly using [pip](https://pypi.org/project/pip/), run the following command in the terminal.

```bash
pip install mindsdb-sdk
```

This is the program from the `src/utils/base/mindsdb.py` file in the repository.

```python
import mindsdb_sdk
from django.conf import settings
from Curriculum.models import CurriculumReview

username = settings.MINDSDB_SERVER_USERNAME
password = settings.MINDSDB_SERVER_PASSWORD

server = None


def connect_server():
    global server
    if not server:
        server = mindsdb_sdk.connect(
            'https://cloud.mindsdb.com',
            login=username,
            password=password
        )
    return server


def make_query(text: str, model: str):
    return f"""
    SELECT *
    FROM mindsdb.{model}
    WHERE text = '{text}'
    """


def classify_text(text: str):
    server = connect_server()
    project = server.get_project('mindsdb')
    max_label = ""
    max_score = 0

    df = project.query(make_query(text, "hf_classify")).fetch()
    data = df['PRED_explain'][0]
    for k, v in CurriculumReview.LABEL:
        current = data.get(v)
        if current > max_score:
            max_score = current
            max_label = k

    return max_label


def sentiment_text(text: str):
    server = connect_server()
    project = server.get_project('mindsdb')
    max_label = ""
    max_score = 0

    df = project.query(make_query(text, "hf_sentiment")).fetch()
    data = df['PRED_explain'][0]
    for k, v in CurriculumReview.SENTIMENT:
        current = data.get(v)
        if current > max_score:
            max_score = current
            max_label = k
    return max_label
```

Let's go through this program 🤓

1. **Importing the required modules**
    
    ```python
    import mindsdb_sdk
    from django.conf import settings
    from Curriculum.models import CurriculumReview
    ```
    
    We imported the `CurriculumReview` model so we can have access the labels defined in it.
    
    The Django setting is imported so we can have access to the `MINDSDB_SERVER_USERNAME` and `MINDSDB_SERVER_PASSWORD` defined when we set up the backend project.
    
2. **Creating a connection to our MindsDB cloud server.**
    
    ```python
    server = None
    
    def connect_server():
        global server
        if not server:
            server = mindsdb_sdk.connect(
                'https://cloud.mindsdb.com',
                login=username,
                password=password
            )
        return server
    ```
    
    We did it this way so that if the same Django process already has a connection created for the mindsdb\_sdk server, we will use it instead of creating it again.
    
3. **Creating SQL model queries to our** `mindsdb` **project in the cloud**
    
    ```python
    def make_query(text: str, model: str):
        return f"""
        SELECT *
        FROM mindsdb.{model}
        WHERE text = '{text}'
        """
    ```
    
    This function takes the text to classify and the model name, `hf_classify` or `hf_sentiment`, and returns a query that we can execute.
    
4. **Classifying text to label**
    
    ```python
    def classify_text(text: str):
        server = connect_server()
        project = server.get_project('mindsdb')
        max_label = ""
        max_score = 0
    
        df = project.query(make_query(text, "hf_classify")).fetch()
        data = df['PRED_explain'][0]
        for k, v in CurriculumReview.LABEL:
            current = data.get(v)
            if current > max_score:
                max_score = current
                max_label = k
    
        return max_label
    ```
    
    We first
    
    * get the connection to our MindsDB cloud
        
    * then get the project, `mindsdb`,
        
    * then use the project to make a query to the `hf_classify` model,
        
    * parse the response and get the label with the maximum result.
        
5. **Classifying text to sentiment**
    
    Works exactly like `classify_text`, we just used `CurriculumReview.SENTIMENT` when getting the label with the maximum result.
    

This is a program from `/src/Curriculum/api/base/views.py` for `RateCurriculum` view.

```python
def perform_create(self, serializer):
    cur = self.get_object()
    enrollment = self.request.user\
        .get_curriculum_enrollment(cur)
    review = serializer.save(enrollment=enrollment)

    # Classify review here, in live app use cron job
    review.sentiment = sentiment_text(review.review)
    review.label = classify_text(review.review)
    review.save()
```

Ensuring the review label is updated when a user submits a new review, we classify it and update the database.

A performance problem here is that the user has to wait for our classification algorithm to work before it returns a response. In a production environment, it will be better to do the labeling later by using a [Queue](https://aws.amazon.com/message-queue/#:~:text=A%20message%20queue%20is%20a,once%2C%20by%20a%20single%20consumer.) or [Cron job](https://gutsytechster.wordpress.com/2019/06/24/how-to-setup-a-cron-job-in-django/) that classifies reviews in the background.

## Impact

By using MindsDB models, we are able to quickly and accurately classify user curriculum reviews, allowing us to identify and address areas for improvement. Using the MindsDb cloud made it easier to integrate AI into the application. Doing this the manual way will be hard and time-consuming.

This helps us to continually improve the platform and curate the best possible curriculums for users. Additionally, by using Django and Next.js, we are able to build a scalable and secure platform that can handle a large number of users and provide a seamless user experience.

## Conclusion

In this tutorial, we showed how to use MindsDB models and Django for the backend to build a tech course curriculums platform. By implementing MindsDB models for sentiment and text classification, we can quickly and accurately classify user curriculum reviews and improve the user experience of your platform. With the power of [MindsDB](https://docs.mindsdb.com/what-is-mindsdb), we can build a scalable and secure platform that provides a seamless user experience.

Explore the features of MindsDB by checking out their [**official documentation**](https://docs.mindsdb.com/what-is-mindsdb) and joining the [**Slack community**](https://mindsdbcommunity.slack.com/join/shared_invite/zt-1syebtjn9-MLT4dYX2U7NVdorPPDnlRg).

Start building today by [**creating a free MindsDB Cloud account**](https://cloud.mindsdb.com/?_gl=1*1h5q3wc*_ga*MTUxNDYyNjM3NC4xNjgwNjAyMTQ2*_ga_7LGFPGV6XV*MTY4MjMyNDY0OC4yMi4xLjE2ODIzMjQ5OTcuMzkuMC4w)!

## Want to Contribute?

This project is [Open Source](https://opensource.com/resources/what-open-source) and can help a lot of new developers organize online free content into structured curriculums saving them from the [Tutorial Hell](https://medium.com/codex/what-is-tutorial-hell-and-how-to-avoid-it-to-improve-as-a-developer-8c5376c97011?source=rss------technology-5#:~:text=Tutorial%20hell%20is%20a%20concept,a%20new%20language%20or%20technology.).

Connect with me on [**Twitter**](https://twitter.com/netrobeweb) | [**LinkedIn**](https://www.linkedin.com/in/ayomide-ayanwola/) | [**GitHub**](http://github.com/devvspaces), and let's work together, Thanks. 🤓💻