Why does the order in the upsampling path of the U-Net deviate from standard u-net?

[ghost]

Hi,
I have a question about the U-Net you developed (with residual units). In every layer of the upsampling path, you first apply a transposed convolution, then a ‘normal’ convolution and then you concatanate the result with the result of the same layer of the downsampling path. However, in the link you refer to in the description of the U-Net (https://link.springer.com/chapter/10.1007/978-3-030-12029-0_40 1), the order is transposed convolution --> concatenation --> normal convolution, which is to my knowledge the standard order in the upsampling path of a U-Net. Why do you deviate from this usual order, and does the order have any influence on the result?

Thanks in advance!

[Nic-Ma]

@ericspod , could you please help share some details?

Thanks.

[ericspod]

What is going on is what is described in the paper. Here each layer of the UNet structure is represented by a Sequential containing a layer of the downsample path, the skip connection, and a layer of the upsample path. As configured here the skip connection will take the input from the downsample path, pass it to the submodule which is either the next layer down or the bottom bottleneck layer of the whole structure, and then concat the output from that with the original input which is then returned. This output is fed to the upsample block which consists of the transpose convolution followed by whatever else the network is configured for. This I believe does conform with the published description and the typical UNet definition.

[ghost]

@ericspod Thank you for your reply! However, I still don't quite understand, which is mostly caused by this image (from the paper)

In this image, the residual unit in the upsampling path consist of a transposed convolution, then the concatenation and then a normal conv2d. The way you describe it, is that the transposed convolution is directly followed by a normal conv2d. (the transposed convolution takes as input the concatenation in the layer before)

I'm sorry to bother you again, but I hope you could help me out with this!

Thanks in advance!

[ericspod]

Sorry, I did state incorrectly that the paper is describing the current UNet. I think the implementation has changed from what's described in this figure, in the upsample layer the input from the previous layer and the skip connect are concatenated together before being passed to the transposed convolution. This figure shows the skip connect data being concatenated with the output from the transposed convolution which isn't how the current MONAI implementation works. Sorry for the confusion!

[ghost]

@ericspod Thanks! Then the network is completely clear to me now. Is there any particular reason why you deviate from it? Or just because it improved the performance?

[ericspod]

The current version seemed the easier one to implement as well as being compatible with Torchscript. There is very little performance difference between subtle variations on the same architecture, how one trains and for how long has more impact.