We have x, a, e and i as inputs, where does i come from?

[Tunnelvisie]

Hi,

It's been a while since I had time to try my hand at GNNs specifically and ML in general, so bare with me for perhaps asking a silly question.

My objective is to do a Binary Graph Classification on a set of individual (small) graphs ~25 nodes each.

I have created a custom dataset (that should be the correct dimensions as per my 6 month old previous question ). Then using BatchLoader we create a train, test and validation set.

The issue/question I have lies in the next step. There we set up the model, including I_in = Input(shape=(), name="segment_ids_in", dtype=tf.int32). Since I'm a bit rusty, all I can remember is that this is used to identify which graph is which or something along those lines. The issue arises when we pass inputs to train_on_batch() and it's expecting a value for x, a, e and i but instead it's only getting a value for x, a and e. I've looked at some examples qm9_ecc.py and custom_dataset.py but I can't figure out where the i value in inputs in both of these examples come from. They just seem to be included in inputs / the loader / the batch in these examples.

So the question is, how do I get my inputs to also contain a value of i. And probably a maybe even more important question, since I'm a bit rusty, do I even need I_in and if yes, where do I find a bit more documentation explaining what it is?

---

First, I've created a custom dataset dataset that returns a list of Graph elements as done in the example(s). Each graph has x, a, e and y values.

class CustomDataset(Dataset):
    
    def __init__(self, **kwargs):
        super().__init__(**kwargs)

    def read(self):
          file = open(join(DATA_FOLDER, FEATURE_FOLDER, DATASET_FILE), 'rb')
          data = pickle.load(file)

        # We must return a list of Graph objects
        return [
            Graph(x=x, a=a, e=e, y=np.asarray([y])) for x, a, e, y in zip(data['x'], data['a'], data['e'], data['y'])
        ]

Subsequently, using BatchLoader I load my datasets into train, validation and test sets.

################################################################################
# PARAMETERS
################################################################################
learning_rate = 1e-3  # Learning rate
epochs = 10  # Number of training epochs
batch_size = 32  # Batch size

l2_reg = 5e-4  # Regularization rate for l2
patience = 10  # Patience for early stopping

################################################################################
# LOAD DATA
################################################################################
dataset = CustomDataset()
# Parameters
F = dataset.n_node_features  # Dimension of node features
S = dataset.n_edge_features  # Dimension of edge features
n_out = dataset.n_labels  # Dimension of the target

# Train/valid/test split
split = int(0.8 * len(dataset))
split2 = int(0.1 * len(dataset)) + split
dataset_tr, dataset_te, dataset_va = dataset[:split], dataset[split: split2], dataset[split2::]

loader_tr = BatchLoader(dataset_tr, batch_size=batch_size, epochs=epochs)
loader_va = BatchLoader(dataset_va, batch_size=batch_size)
loader_te = BatchLoader(dataset_te, batch_size=batch_size)

So far so good. However, now I set up my Model with the aforementioned I_in = Input(shape=(), name="segment_ids_in", dtype=tf.int32)

X_in = Input(shape=(F,), name="X_in")
A_in = Input(shape=(None,), sparse=True, name="A_in")
E_in = Input(shape=(S,), name="E_in")
I_in = Input(shape=(), name="segment_ids_in", dtype=tf.int32)

# EdgeConditionConvolution "I have edge level features"
X_1 = ECCConv(32, activation="relu")([X_in, A_in, E_in])
X_2 = ECCConv(32, activation="relu")([X_1, A_in, E_in])
X_3 = GlobalSumPool()([X_2, I_in])
output = Dense(n_out)(X_3)

# Build model
model = Model(inputs=[X_in, A_in, E_in, I_in], outputs=output)
optimizer = Adam(learning_rate=learning_rate)
loss_fn = BinaryCrossentropy()

Then we have the train_on_batch() function, where ultimately the error is thrown (see line 3 below):

def train_on_batch(inputs, target):
    with tf.GradientTape() as tape:
        predictions = model(inputs, training=True) # <<<<------ this inputs requires 4 instead of 3 values
        loss = loss_fn(target, predictions) + sum(model.losses)
        acc = tf.reduce_mean(binary_crossentropy(target, predictions))

    gradients = tape.gradient(loss, model.trainable_variables)
    optimizer.apply_gradients(zip(gradients, model.trainable_variables))
    return loss, acc

# Training loop
results_tr = []
for batch in loader_tr:
    step += 1

    # Training step
    inputs, target = batch
    loss, acc = train_on_batch(inputs, target)
    results_tr.append((loss, acc, len(target)))

The error:

ValueError: in user code:

    C:/ussf_gnn/pass-classification.py:214 train_on_batch  *
        predictions = model(inputs, training=True)
    C:\ussf_gnn\venv\lib\site-packages\tensorflow\python\keras\engine\base_layer.py:998 __call__  **
        input_spec.assert_input_compatibility(self.input_spec, inputs, self.name)
    C:\ussf_gnn\venv\lib\site-packages\tensorflow\python\keras\engine\input_spec.py:204 assert_input_compatibility
        raise ValueError('Layer ' + layer_name + ' expects ' +

    ValueError: Layer model expects 4 input(s), but it received 3 input tensors. Inputs received: [<tf.Tensor 'inputs:0' shape=(32, 23, 16) dtype=float64>, <tf.Tensor 'inputs_1:0' shape=(32, 23, 23) dtype=float64>, <tf.Tensor 'inputs_2:0' shape=(32, 23, 23, 7) dtype=float64>]

[Tunnelvisie]

Am I simply using the wrong Loader? After reading a bit more/testing it seems like DisjointLoader does provide an i value.

[danielegrattarola]

Hi,
it seems like you have it figured it out on you own :D

Yes, the i variable is the batch index, a vector of integers that maps each node to a graph in a batch: i[j] = k means that the j-th node belongs to the k-th graph (0-based indexing).

This only makes sense when in disjoint mode, so you will only have that variable when using a DisjointLoader.
Read more here: https://graphneural.network/data-modes/

[Tunnelvisie]

Thanks, appreciate the answer :)