dmlc--dgl
Minjie Wang
2cdc4d3c1d
* patched 1_first * done 2_basics * done 4_batch * done 1_gcn, 9_gat, 2_capsule * 4_rgcn.py * revert * more fix
243 行
10 KiB
Python
243 行
10 KiB
Python
"""
|
|
.. currentmodule:: dgl
|
|
|
|
Graph Classification Tutorial
|
|
=============================
|
|
|
|
**Author**: `Mufei Li <https://github.com/mufeili>`_,
|
|
`Minjie Wang <https://jermainewang.github.io/>`_,
|
|
`Zheng Zhang <https://shanghai.nyu.edu/academics/faculty/directory/zheng-zhang>`_.
|
|
|
|
In this tutorial, you learn how to use DGL to batch multiple graphs of variable size and shape. The
|
|
tutorial also demonstrates training a graph neural network for a simple graph classification task.
|
|
|
|
Graph classification is an important problem
|
|
with applications across many fields, such as bioinformatics, chemoinformatics, social
|
|
network analysis, urban computing, and cybersecurity. Applying graph neural
|
|
networks to this problem has been a popular approach recently. This can be seen in the following reserach references:
|
|
`Ying et al., 2018 <https://arxiv.org/abs/1806.08804>`_,
|
|
`Cangea et al., 2018 <https://arxiv.org/abs/1811.01287>`_,
|
|
`Knyazev et al., 2018 <https://arxiv.org/abs/1811.09595>`_,
|
|
`Bianchi et al., 2019 <https://arxiv.org/abs/1901.01343>`_,
|
|
`Liao et al., 2019 <https://arxiv.org/abs/1901.01484>`_,
|
|
`Gao et al., 2019 <https://openreview.net/forum?id=HJePRoAct7>`_).
|
|
|
|
"""
|
|
|
|
###############################################################################
|
|
# Simple graph classification task
|
|
# --------------------------------
|
|
# In this tutorial, you learn how to perform batched graph classification
|
|
# with DGL. The example task objective is to classify eight types of topologies shown here.
|
|
#
|
|
# .. image:: https://data.dgl.ai/tutorial/batch/dataset_overview.png
|
|
# :align: center
|
|
#
|
|
# Implement a synthetic dataset :class:`data.MiniGCDataset` in DGL. The dataset has eight
|
|
# different types of graphs and each class has the same number of graph samples.
|
|
|
|
from dgl.data import MiniGCDataset
|
|
import matplotlib.pyplot as plt
|
|
import networkx as nx
|
|
# A dataset with 80 samples, each graph is
|
|
# of size [10, 20]
|
|
dataset = MiniGCDataset(80, 10, 20)
|
|
graph, label = dataset[0]
|
|
fig, ax = plt.subplots()
|
|
nx.draw(graph.to_networkx(), ax=ax)
|
|
ax.set_title('Class: {:d}'.format(label))
|
|
plt.show()
|
|
|
|
###############################################################################
|
|
# Form a graph mini-batch
|
|
# -----------------------
|
|
# To train neural networks efficiently, a common practice is to batch
|
|
# multiple samples together to form a mini-batch. Batching fixed-shaped tensor
|
|
# inputs is common. For example, batching two images of size 28 x 28
|
|
# gives a tensor of shape 2 x 28 x 28. By contrast, batching graph inputs
|
|
# has two challenges:
|
|
#
|
|
# * Graphs are sparse.
|
|
# * Graphs can have various length. For example, number of nodes and edges.
|
|
#
|
|
# To address this, DGL provides a :func:`dgl.batch` API. It leverages the idea that
|
|
# a batch of graphs can be viewed as a large graph that has many disjointed
|
|
# connected components. Below is a visualization that gives the general idea.
|
|
#
|
|
# .. image:: https://data.dgl.ai/tutorial/batch/batch.png
|
|
# :width: 400pt
|
|
# :align: center
|
|
#
|
|
# Define the following ``collate`` function to form a mini-batch from a given
|
|
# list of graph and label pairs.
|
|
|
|
import dgl
|
|
import torch
|
|
|
|
def collate(samples):
|
|
# The input `samples` is a list of pairs
|
|
# (graph, label).
|
|
graphs, labels = map(list, zip(*samples))
|
|
batched_graph = dgl.batch(graphs)
|
|
return batched_graph, torch.tensor(labels)
|
|
|
|
###############################################################################
|
|
# The return type of :func:`dgl.batch` is still a graph. In the same way,
|
|
# a batch of tensors is still a tensor. This means that any code that works
|
|
# for one graph immediately works for a batch of graphs. More importantly,
|
|
# because DGL processes messages on all nodes and edges in parallel, this greatly
|
|
# improves efficiency.
|
|
#
|
|
# Graph classifier
|
|
# ----------------
|
|
# Graph classification proceeds as follows.
|
|
#
|
|
# .. image:: https://data.dgl.ai/tutorial/batch/graph_classifier.png
|
|
#
|
|
# From a batch of graphs, perform message passing and graph convolution
|
|
# for nodes to communicate with others. After message passing, compute a
|
|
# tensor for graph representation from node (and edge) attributes. This step might
|
|
# be called readout or aggregation. Finally, the graph
|
|
# representations are fed into a classifier :math:`g` to predict the graph labels.
|
|
#
|
|
# Graph convolution layer can be found in the ``dgl.nn.<backend>`` submodule.
|
|
|
|
from dgl.nn.pytorch import GraphConv
|
|
|
|
###############################################################################
|
|
# Readout and classification
|
|
# --------------------------
|
|
# For this demonstration, consider initial node features to be their degrees.
|
|
# After two rounds of graph convolution, perform a graph readout by averaging
|
|
# over all node features for each graph in the batch.
|
|
#
|
|
# .. math::
|
|
#
|
|
# h_g=\frac{1}{|\mathcal{V}|}\sum_{v\in\mathcal{V}}h_{v}
|
|
#
|
|
# In DGL, :func:`dgl.mean_nodes` handles this task for a batch of
|
|
# graphs with variable size. You then feed the graph representations into a
|
|
# classifier with one linear layer to obtain pre-softmax logits.
|
|
|
|
import torch.nn as nn
|
|
import torch.nn.functional as F
|
|
|
|
class Classifier(nn.Module):
|
|
def __init__(self, in_dim, hidden_dim, n_classes):
|
|
super(Classifier, self).__init__()
|
|
self.conv1 = GraphConv(in_dim, hidden_dim)
|
|
self.conv2 = GraphConv(hidden_dim, hidden_dim)
|
|
self.classify = nn.Linear(hidden_dim, n_classes)
|
|
|
|
def forward(self, g):
|
|
# Use node degree as the initial node feature. For undirected graphs, the in-degree
|
|
# is the same as the out_degree.
|
|
h = g.in_degrees().view(-1, 1).float()
|
|
# Perform graph convolution and activation function.
|
|
h = F.relu(self.conv1(g, h))
|
|
h = F.relu(self.conv2(g, h))
|
|
g.ndata['h'] = h
|
|
# Calculate graph representation by averaging all the node representations.
|
|
hg = dgl.mean_nodes(g, 'h')
|
|
return self.classify(hg)
|
|
|
|
###############################################################################
|
|
# Setup and training
|
|
# ------------------
|
|
# Create a synthetic dataset of :math:`400` graphs with :math:`10` ~
|
|
# :math:`20` nodes. :math:`320` graphs constitute a training set and
|
|
# :math:`80` graphs constitute a test set.
|
|
|
|
import torch.optim as optim
|
|
from torch.utils.data import DataLoader
|
|
|
|
# Create training and test sets.
|
|
trainset = MiniGCDataset(320, 10, 20)
|
|
testset = MiniGCDataset(80, 10, 20)
|
|
# Use PyTorch's DataLoader and the collate function
|
|
# defined before.
|
|
data_loader = DataLoader(trainset, batch_size=32, shuffle=True,
|
|
collate_fn=collate)
|
|
|
|
# Create model
|
|
model = Classifier(1, 256, trainset.num_classes)
|
|
loss_func = nn.CrossEntropyLoss()
|
|
optimizer = optim.Adam(model.parameters(), lr=0.001)
|
|
model.train()
|
|
|
|
epoch_losses = []
|
|
for epoch in range(80):
|
|
epoch_loss = 0
|
|
for iter, (bg, label) in enumerate(data_loader):
|
|
prediction = model(bg)
|
|
loss = loss_func(prediction, label)
|
|
optimizer.zero_grad()
|
|
loss.backward()
|
|
optimizer.step()
|
|
epoch_loss += loss.detach().item()
|
|
epoch_loss /= (iter + 1)
|
|
print('Epoch {}, loss {:.4f}'.format(epoch, epoch_loss))
|
|
epoch_losses.append(epoch_loss)
|
|
|
|
###############################################################################
|
|
# The learning curve of a run is presented below.
|
|
|
|
plt.title('cross entropy averaged over minibatches')
|
|
plt.plot(epoch_losses)
|
|
plt.show()
|
|
|
|
###############################################################################
|
|
# The trained model is evaluated on the test set created. To deploy
|
|
# the tutorial, restrict the running time to get a higher
|
|
# accuracy (:math:`80` % ~ :math:`90` %) than the ones printed below.
|
|
|
|
model.eval()
|
|
# Convert a list of tuples to two lists
|
|
test_X, test_Y = map(list, zip(*testset))
|
|
test_bg = dgl.batch(test_X)
|
|
test_Y = torch.tensor(test_Y).float().view(-1, 1)
|
|
probs_Y = torch.softmax(model(test_bg), 1)
|
|
sampled_Y = torch.multinomial(probs_Y, 1)
|
|
argmax_Y = torch.max(probs_Y, 1)[1].view(-1, 1)
|
|
print('Accuracy of sampled predictions on the test set: {:.4f}%'.format(
|
|
(test_Y == sampled_Y.float()).sum().item() / len(test_Y) * 100))
|
|
print('Accuracy of argmax predictions on the test set: {:4f}%'.format(
|
|
(test_Y == argmax_Y.float()).sum().item() / len(test_Y) * 100))
|
|
|
|
###############################################################################
|
|
# The animation here plots the probability that a trained model predicts the correct graph type.
|
|
#
|
|
# .. image:: https://data.dgl.ai/tutorial/batch/test_eval4.gif
|
|
#
|
|
# To understand the node and graph representations that a trained model learned,
|
|
# we use `t-SNE, <https://lvdmaaten.github.io/tsne/>`_ for dimensionality reduction
|
|
# and visualization.
|
|
#
|
|
# .. image:: https://data.dgl.ai/tutorial/batch/tsne_node2.png
|
|
# :align: center
|
|
#
|
|
# .. image:: https://data.dgl.ai/tutorial/batch/tsne_graph2.png
|
|
# :align: center
|
|
#
|
|
# The two small figures on the top separately visualize node representations after one and two
|
|
# layers of graph convolution. The figure on the bottom visualizes
|
|
# the pre-softmax logits for graphs as graph representations.
|
|
#
|
|
# While the visualization does suggest some clustering effects of the node features,
|
|
# you would not expect a perfect result. Node degrees are deterministic for
|
|
# these node features. The graph features are improved when separated.
|
|
#
|
|
# What's next?
|
|
# ------------
|
|
# Graph classification with graph neural networks is still a new field.
|
|
# It's waiting for people to bring more exciting discoveries. The work requires
|
|
# mapping different graphs to different embeddings, while preserving
|
|
# their structural similarity in the embedding space. To learn more about it, see
|
|
# `How Powerful Are Graph Neural Networks? <https://arxiv.org/abs/1810.00826>`_ a research paper
|
|
# published for the International Conference on Learning Representations 2019.
|
|
#
|
|
# For more examples about batched graph processing, see the following:
|
|
#
|
|
# * Tutorials for `Tree LSTM <https://docs.dgl.ai/tutorials/models/2_small_graph/3_tree-lstm.html>`_ and `Deep Generative Models of Graphs <https://docs.dgl.ai/tutorials/models/3_generative_model/5_dgmg.html>`_
|
|
# * An example implementation of `Junction Tree VAE <https://github.com/dmlc/dgl/tree/master/examples/pytorch/jtnn>`_
|