Neural-Style-Transfer-Using-GNN

Replication of Learning Graph Neural Networks for Image Style Transfer (https://arxiv.org/abs/2207.11681)

Note: Paper has no implementation.

Changes

Made

• GATv2 instead of GAT, as the graph in the network is a bipartite graph in which dynamic attention is proven to be effective instead of static attention

Planning

• Implementing other types of graph construction to experiment with which suits best for the problem. For example, using Threshold instead of KNN and etc

To Do

• Implementing Deformable Patch for style graph node construction.

Results

The reason for this type of poor stylization and blurring is because of hyperparameters which were set. For example, the patch stride in paper is chosen to be 1, while in this implementation it is 7, otherwise, the number of constructed nodes will be too big, and memory issues will be faced. Content:

Style:

Result:

Theoritical work

The idea was to utilize the fact that constructed style to content graph to be a bipartite graph in a way such that GATv2 performs much better compared to GAT in bipartite graphs, proven in the section Synthetic benchmark dictionary lookup in the paper of GATv2