Machine Learning Implementation example: Using Dota2 Games Results Data Set

Yes~

You might be trapped by my fancy little big Title "Machine Learning Implementationd example". 

First, I would like letting you know that my implementation does not show great performance for

Dota2 Game Results Dataset. 

However, I will add my thorough explanation and some remedy you might try your own to perform better. 

 

Before the real starter, the source code credit goes to Professor Sung Kim!

His original source code URL: github.com/hunkim/PyTorchZeroToAll/blob/master/06_logistic_regression.py

hunkim/PyTorchZeroToAll

 

Let's go! 

 

Step 1: Import the necessary Modules

from torch import nn, optim, from_numpy
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
import numpy as np

I won't explain much about the lines above. 

You will eventually find it out~

Just stay tuned and follow me closely.

 

Step 2: Prepare Dota2 Game Results Dataset

xy = np.loadtxt('/Users/kimkwangjae/Downloads/dota2Dataset/dota2Train.csv', delimiter=',', dtype=np.float32)

xy[:,0][xy[:,0]>0.0] = 1.0
xy[:,0][xy[:,0]<0.0] = 0.0
#print(np.shape(xy))

x_data = from_numpy(xy[:, 1:])
y_data = from_numpy(xy[:, [0]])
#print(f'X\'s shape: {x_data.shape} | Y\'s shape: {y_data.shape}')

X_train, X_test, y_train, y_test = train_test_split(x_data, y_data, test_size=0.33, random_state=42)

I won't describe the dataset here~

It's better for you to be familiar with the UCI dataset website man: 

archive.ics.uci.edu/ml/datasets/Dota2+Games+Results

UCI Machine Learning Repository: Dota2 Games Results Data Set

As you see, "xy" is your boy who have the full attribute.

The first column of the dataset is the target (Y). 

(Spoiler: reason for my bad performance) I'm gonna use Logistic Regression model. 

So, I need those 2nd ~ 3rd lines of codes to change the targetting binary values from (-1 & 1) to (0, 1), which refers to "lost" and "won" respectively. 

 

4th-6th lines of codes are needed for splitting {attributes, label} and {train and test data}

 

Step 3: Prepare Model (Logistic Regression) 

class Model(nn.Module):
    def __init__(self):
        """
        In the constructor we instantiate two nn.Linear module
        """
        super(Model, self).__init__()
        self.l1 = nn.Linear(116, 80)
        self.l2 = nn.Linear(80, 50)
        self.l3 = nn.Linear(50, 20)
        self.l4 = nn.Linear(20, 1)

        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        """
        In the forward function we accept a Variable of input data and we must return
        a Variable of output data. We can use Modules defined in the constructor as
        well as arbitrary operators on Variables.
        """
        out1 = self.sigmoid(self.l1(x))
        out2 = self.sigmoid(self.l2(out1))
        out3 = self.sigmoid(self.l3(out2))

        y_pred = self.sigmoid(self.l4(out3))

        return y_pred
        
        
# our model
model = Model()

# Construct our loss function and an Optimizer. The call to model.parameters()
# in the SGD constructor will contain the learnable parameters of the two
# nn.Linear modules which are members of the model.
criterion = nn.BCELoss(reduction='mean')
optimizer = optim.SGD(model.parameters(), lr=0.001)

Model structure is the one of the crucial parts of the performance. 

I might have failed over here. 

You can play with different learning rate or different optimizer if needed! 

 

If you have better structure, letting me know friends :)

 

Step 4: Training Loop 

# Training loop
for epoch in range(2500):
    # Forward pass: Compute predicted y by passing x to the model
    y_pred = model(X_train)

    # Compute and print loss
    loss = criterion(y_pred, y_train)
    print(f'Epoch: {epoch + 1}/2000 | Loss: {loss.item():.6f}')

    # Zero gradients, perform a backward pass, and update the weights.
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

This is the trainig loop code.

You could play with epoch size over here! 

 

Step 5: Test the Model and Evaluation

y_pred = model(X_test)

y_pred = y_pred.detach().numpy()
y_test = y_test.detach().numpy()

y_pred[y_pred[:,0]>0.5] = 1.0
y_pred[y_pred[:,0]<0.5] = 0.0

y_pred[:,0].astype(np.int16)
y_test[:,0].astype(np.int16)

accuracy = accuracy_score(y_test, y_pred)
accuracy *= 100
print('The accuracy in percentage is ')
print(accuracy)

If you try my full code below, the performance barely pass the 50% of accuracy.

This might be my fault but I would like to say that deep learning might be the better suit for the dataset. 

While trying out the model and the dataset, please also let me know the updates or correction you have under the comments. 

Best wishes for all your Machine Learning related Project~

from torch import nn, optim, from_numpy
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
import numpy as np


xy = np.loadtxt('/Users/kimkwangjae/Downloads/dota2Dataset/dota2Train.csv', delimiter=',', dtype=np.float32)
xy[:,0][xy[:,0]>0.0] = 1.0
xy[:,0][xy[:,0]<0.0] = 0.0
#print(np.shape(xy))
x_data = from_numpy(xy[:, 1:])
y_data = from_numpy(xy[:, [0]])
print(f'X\'s shape: {x_data.shape} | Y\'s shape: {y_data.shape}')

X_train, X_test, y_train, y_test = train_test_split(x_data, y_data, test_size=0.33, random_state=42)

class Model(nn.Module):
    def __init__(self):
        """
        In the constructor we instantiate two nn.Linear module
        """
        super(Model, self).__init__()
        self.l1 = nn.Linear(116, 80)
        self.l2 = nn.Linear(80, 50)
        self.l3 = nn.Linear(50, 20)
        self.l4 = nn.Linear(20, 1)

        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        """
        In the forward function we accept a Variable of input data and we must return
        a Variable of output data. We can use Modules defined in the constructor as
        well as arbitrary operators on Variables.
        """
        out1 = self.sigmoid(self.l1(x))
        out2 = self.sigmoid(self.l2(out1))
        out3 = self.sigmoid(self.l3(out2))

        y_pred = self.sigmoid(self.l4(out3))

        return y_pred


# our model
model = Model()


# Construct our loss function and an Optimizer. The call to model.parameters()
# in the SGD constructor will contain the learnable parameters of the two
# nn.Linear modules which are members of the model.
criterion = nn.BCELoss(reduction='mean')
optimizer = optim.SGD(model.parameters(), lr=0.001)

# Training loop
for epoch in range(2500):
    # Forward pass: Compute predicted y by passing x to the model
    y_pred = model(X_train)

    # Compute and print loss
    loss = criterion(y_pred, y_train)
    print(f'Epoch: {epoch + 1}/2000 | Loss: {loss.item():.6f}')

    # Zero gradients, perform a backward pass, and update the weights.
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

y_pred = model(X_test)

y_pred = y_pred.detach().numpy()
y_test = y_test.detach().numpy()

y_pred[y_pred[:,0]>0.5] = 1.0
y_pred[y_pred[:,0]<0.5] = 0.0

y_pred[:,0].astype(np.int16)
y_test[:,0].astype(np.int16)

accuracy = accuracy_score(y_test, y_pred)
accuracy *= 100
print('The accuracy in percentage is ')
print(accuracy)