Architectures

The following architectures are availabled in ivadomed.

ResNet

class ResNet(block, layers, num_classes=2, zero_init_residual=False, groups=1, width_per_group=64, replace_stride_with_dilation=None, norm_layer=None)

Bases: Module

ResNet model based on

“Deep Residual Learning for Image Recognition”

Parameters:

• block (nn.Module) – Basic block of the network (such as conv + bn + non-nonlinearity)

• layers (int list) – Number of blocks to stack (network depth) after each downsampling step.

• num_classes (int) – Number of GT classes.

• zero_init_residual (bool) – if True, zero-initialize the last BN in each residual branch,

• zeros (so that the residual branch starts with) –

• identity. (and each residual block behaves like an) –

• groups (int) – Number of parallel branches in the network, see “paper”

• width_per_group (int) – base width of the blocks

• replace_stride_with_dilation (tuple) – each element in the tuple indicates if we replace the 2x2 stride with a dilated convolution

• norm_layer (layer) – Custom normalization layer, if not provided BatchNorm2d is used

Initialize internal Module state, shared by both nn.Module and ScriptModule.

__init__(block, layers, num_classes=2, zero_init_residual=False, groups=1, width_per_group=64, replace_stride_with_dilation=None, norm_layer=None)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

DenseNet

class DenseNet(growth_rate=32, block_config=(6, 12, 24, 16), num_init_features=64, bn_size=4, dropout_rate=0.3, num_classes=2, memory_efficient=False)

Bases: Module

Densenet-BC model class, based on “Densely Connected Convolutional Networks”

Parameters:

• growth_rate (int) - how many filters to add each layer (k in paper) –

• block_config (list of 4 ints) –

• num_init_features (int) –

• bn_size (int) – (i.e. bn_size * k features in the bottleneck layer)

• dropout_rate (float) –

• num_classes (int) –

• memory_efficient (bool) – but slower. Default: False. See “article”

Initialize internal Module state, shared by both nn.Module and ScriptModule.

__init__(growth_rate=32, block_config=(6, 12, 24, 16), num_init_features=64, bn_size=4, dropout_rate=0.3, num_classes=2, memory_efficient=False)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Unet

class Unet(in_channel=1, out_channel=1, depth=3, dropout_rate=0.3, bn_momentum=0.1, final_activation='sigmoid', is_2d=True, n_filters=64, **kwargs)

Bases: Module

A reference U-Net model.

See also

Ronneberger, O., et al (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation ArXiv link: https://arxiv.org/abs/1505.04597

Parameters:

• in_channel (int) – Number of channels in the input image.

• out_channel (int) – Number of channels in the output image.

• depth (int) – Number of down convolutions minus bottom down convolution.

• dropout_rate (float) – Probability of dropout.

• bn_momentum (float) – Batch normalization momentum.

• final_activation (str) – Choice of final activation between “sigmoid”, “relu” and “softmax”.

• is_2d (bool) – Indicates dimensionality of model: True for 2D convolutions, False for 3D convolutions.

• n_filters (int) – Number of base filters in the U-Net.

• **kwargs –

Attributes:

• encoder (Encoder) – U-Net encoder.

• decoder (Decoder) – U-net decoder.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

__init__(in_channel=1, out_channel=1, depth=3, dropout_rate=0.3, bn_momentum=0.1, final_activation='sigmoid', is_2d=True, n_filters=64, **kwargs)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

FiLMedUnet

class FiLMedUnet(in_channel=1, out_channel=1, depth=3, dropout_rate=0.3, bn_momentum=0.1, n_metadata=None, film_layers=None, is_2d=True, n_filters=64, **kwargs)

Bases: Unet

U-Net network containing FiLM layers to condition the model with another data type (i.e. not an image).

Parameters:

• n_channel (int) – Number of channels in the input image.

• out_channel (int) – Number of channels in the output image.

• depth (int) – Number of down convolutions minus bottom down convolution.

• dropout_rate (float) – Probability of dropout.

• bn_momentum (float) – Batch normalization momentum.

• n_metadata (dict) – FiLM metadata see ivadomed.loader.film for more details.

• film_layers (list) – List of 0 or 1 indicating on which layer FiLM is applied.

• is_2d (bool) – Indicates dimensionality of model.

• n_filters (int) – Number of base filters in the U-Net.

• **kwargs –

Attributes:

• encoder (Encoder) – U-Net encoder.

• decoder (Decoder) – U-net decoder.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

__init__(in_channel=1, out_channel=1, depth=3, dropout_rate=0.3, bn_momentum=0.1, n_metadata=None, film_layers=None, is_2d=True, n_filters=64, **kwargs)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x, context=None)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

HeMISUnet

class HeMISUnet(contrasts, out_channel=1, depth=3, dropout_rate=0.3, bn_momentum=0.1, **kwargs)

Bases: Module

A U-Net model inspired by HeMIS to deal with missing contrasts.

1. It has as many encoders as contrasts but only one decoder.

2. Skip connections are the concatenations of the means and var of all encoders skip connections.

Param: contrasts: list of all the possible contrasts. [‘T1’, ‘T2’, ‘T2S’, ‘F’]

See also

Havaei, M., Guizard, N., Chapados, N., Bengio, Y.: Hemis: Hetero-modal image segmentation. ArXiv link: https://arxiv.org/abs/1607.05194

Reuben Dorent and Samuel Joutard and Marc Modat and Sébastien Ourselin and Tom Vercauteren Hetero-Modal Variational Encoder-Decoder for Joint Modality Completion and Segmentation ArXiv link: https://arxiv.org/abs/1907.11150

Parameters:

• contrasts (list) – List of contrasts.

• out_channel (int) – Number of output channels.

• depth (int) – Number of down convolutions minus bottom down convolution.

• dropout_rate (float) – Probability of dropout.

• bn_momentum (float) – Batch normalization momentum.

• **kwargs –

Attributes:

• depth (int) – Number of down convolutions minus bottom down convolution.

• contrasts (list) – List of contrasts.

• Encoder_mod (ModuleDict) – Contains encoder for each modality.

• decoder (Decoder) – U-Net decoder.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

__init__(contrasts, out_channel=1, depth=3, dropout_rate=0.3, bn_momentum=0.1, **kwargs)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x_mods, indexes_mod)

X is list like X = [x_T1, x_T2, x_T2S, x_F] indexes_mod: list of arrays like [[1, 1, 1], [1, 1, 0], [1, 0, 1], [1, 1, 0]] N.B. len(list) = number of contrasts. len(list[i]) = Batch size

Modified3DUNet

class Modified3DUNet(in_channel, out_channel, n_filters=16, attention=False, dropout_rate=0.3, bn_momentum=0.1, final_activation='sigmoid', n_metadata=None, film_layers=None, **kwargs)

Bases: Module

Code from the following repository: https://github.com/pykao/Modified-3D-UNet-Pytorch The main differences with the original UNet resides in the use of LeakyReLU instead of ReLU, InstanceNormalisation instead of BatchNorm due to small batch size in 3D and the addition of segmentation layers in the decoder.

If attention=True, attention gates are added in the decoder to help focus attention on important features for a given task. Code related to the attentions gates is inspired from: https://github.com/ozan-oktay/Attention-Gated-Networks

Parameters:

• in_channel (int) – Number of channels in the input image.

• out_channel (int) – Number of channels in the output image.

• n_filters (int) – Number of base filters in the U-Net.

• attention (bool) – Boolean indicating whether the attention module is on or not.

• dropout_rate (float) – Probability of dropout.

• bn_momentum (float) – Batch normalization momentum.

• final_activation (str) – Choice of final activation between “sigmoid”, “relu” and “softmax”.

• **kwargs –

Attributes:

• in_channels (int) – Number of channels in the input image.

• n_classes (int) – Number of channels in the output image.

• base_n_filter (int) – Number of base filters in the U-Net.

• attention (bool) – Boolean indicating whether the attention module is on or not.

• momentum (float) – Batch normalization momentum.

• final_activation (str) – Choice of final activation between “sigmoid”, “relu” and “softmax”.

Note: All layers are defined as attributes and used in the forward method.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

__init__(in_channel, out_channel, n_filters=16, attention=False, dropout_rate=0.3, bn_momentum=0.1, final_activation='sigmoid', n_metadata=None, film_layers=None, **kwargs)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x, context=None, w_film=None)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Countception

class Countception(in_channel=3, out_channel=1, use_logits=False, logits_per_output=12, name='CC', **kwargs)

Bases: Module

Countception model. Fully convolutional model using inception module and used for keypoints detection. The inception model extracts several patches within each image. Every pixel is therefore processed by the network several times, allowing to average multiple predictions and minimize false negatives.

See also

Cohen JP et al. “Count-ception: Counting by fully convolutional redundant counting.” Proceedings of the IEEE International Conference on Computer Vision Workshops. 2017.

Parameters:

• in_channel (int) – number of channels on input image

• out_channel (int) – number of channels on output image

• use_logits (bool) – boolean to change output

• logits_per_output (int) – number of outputs of final convolution which will multiplied by the number of channels

• name (str) – model’s name used for call in configuration file.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

__init__(in_channel=3, out_channel=1, use_logits=False, logits_per_output=12, name='CC', **kwargs)

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.