N1H111SM's Miniverse

2020/05/17 Share

Materials

# GCN

Multiple GC layers stacked together with dropout layer. 采用了nn.ModuleList()技术来实现多个层之间灵活的连接. 其中的GraphConvolution层采用的就是kipf的GCN repo.

# Graph MI Maximization

## Theoretical Procedure

Suppose the input $\{(X_1, A_1), (X_2, A_2), \cdots, (X_M, A_M)\}$ is zero-padded to $N = \max N_i$ where $N_i$ is the dimension of the adjacency matrix $A_i$

Input

After GCN

Node Representation

Graph representation
We use $\sum$ as the readout function:

## Code

GraphMIMaximizer类中，初始化时创建node_level_model，该模型返回a list of multi-scale node level features. 因为是得到总的node-level embedding时采用的是concatenate策略，所以MI estimator的input维度为其向量长度之和+graph-level embedding的长度。

# Graph Generation

• node number $N_i$
• node feature $X_i$
• edge number $E_i$
• edge connection: random permutation

# Experiments

## Inductive representation learning with InfoGraph

#### Data generation

• Input: $(N, C)$
• Target: $(N)$ where each value is $0 \leq \operatorname{targets}[i] \leq C-1$.

#### InfoGraph for unsupervised representation learning

mi_epoch=10000, cla_epoch=100, unsup=10000, train=100, test=100

mi_epoch=10000, cla_epoch=500, unsup=10000, train=100, test=100

MI在训练时的表现还是需要我们再进一步观察理解，但现在先将在simulated graph data上发现的结果进行一些总结：

• MI在maximization的过程中经常会发生sudden collapse，因为是将estimation和maximization同时进行的，这样做会不会带来一些非常糟糕的结果？可能需要参考RL中的estimation.
• MI在maximization过程中经常会发生值变为nan的情况，具体原因不明，可能是因为没有采用BN或者weight decay或者采用不适当的优化器（需要进一步调查）.