1 Introduction
2 Materials and Methods
2.1 The Datasets
Table 1
Segmentation set | Tumour type | Raw set | Final augmented set | Origin |
BC | 192 | 3648 | NCP | |
CRC | 82 | 1558 | NCP | |
Training | total | 274 | 5206 | NCP |
BC | 54 | 54 | NCP | |
CRC | 16 | 16 | NCP | |
Validation | total | 70 | 70 | NCP |
BC | 96 | 96 | TCGA | |
Testing | total | 96 | 96 | TCGA |
Classification set | Nucleus type | Raw set | Final augmented set | Origin |
lymphocyte nuclei | 11032 | 50950 | NCP | |
other nuclei | 10922 | 55825 | NCP | |
Training | total nuclei | 21954 | 106775 | NCP |
lymphocyte nuclei | 2588 | 2588 | NCP | |
other nuclei | 2751 | 2751 | NCP | |
Validation | total nuclei | 5339 | 5339 | NCP |
BC lymphocytes | 903 | 903 | TCGA | |
CRC lymphocytes | 1143 | 1143 | CRCHP | |
total lymphocytes | 2046 | 2046 | ||
BC other | 1195 | 1195 | TCGA | |
CRC other | 1040 | 1040 | CRCHP | |
total other | 2235 | 2235 | ||
Testing I | total nuclei | 4281 | 4281 | |
BC lymphocytes | 2949 | 2949 | JAN | |
BC other | 1921 | 1921 | JAN | |
Testing II | total nuclei | 4870 | 4870 | JAN |
Table 2
Augmentation | Parameters |
Transposition, rotation axis flipping | Perpendicular rotation angles |
CLAHE (Zuiderveld, 1994) | Cliplimit = 2.0, tilegridsize = (8, 8) |
Brightness adjustment | HSV colourspace, hue layer increased by 30 |
RGB augmentation | Random pixel value adjustments up to 0.1 |
RGB2HED colour adjustments (Ruifrok and Johnston, 2001) | Colour values adjusted within range $[0.02,0.001,0.15]$ |
Fig. 1
2.2 The Proposed Method
2.2.1 Modified Micro-Net Model
Fig. 3
(1)
\[ \textit{Dice}=\frac{2\ast \mathit{TP}}{(\mathit{TP}+\mathit{FP})+(\mathit{TP}+\textit{FN})},\]2.2.2 Multilayer Perceptron
3 Results
3.1 Nuclei Segmentation
3.1.1 Hyperparameter Tuning
Table 3
Act func | Output act func | Kernel size | DO | BN | Dice coefficient | Accuracy | Precision | Recall | f-score |
U-Net | |||||||||
elu | sigmoid | 64 | 0.2 | − | $0.78\pm 0.03$ | $0.59\pm 0.08$ | $0.66\pm 0.09$ | $0.84\pm 0.04$ | $0.74\pm 0.06$ |
Micro-Net model | |||||||||
tanh | sigmoid | 64 | − | − | $0.79\pm 0.02$ | $0.66\pm 0.06$ | $0.75\pm 0.05$ | $0.85\pm 0.05$ | $0.80\pm 0.04$ |
Our model | |||||||||
elu | sigmoid | 16 | 0.2 | − | $0.81\pm 0.02$ | $0.77\pm 0.05$ | $0.86\pm 0.04$ | $0.88\pm 0.04$ | $0.87\pm 0.03$ |
elu | sigmoid | 32 | 0.2 | − | $0.80\pm 0.02$ | $0.77\pm 0.06$ | $0.85\pm 0.04$ | $0.88\pm 0.04$ | $0.87\pm 0.04$ |
elu | sigmoid | 48 | 0.2 | − | $0.80\pm 0.02$ | $0.76\pm 0.06$ | $0.85\pm 0.04$ | $0.87\pm 0.04$ | $0.87\pm 0.03$ |
elu | sigmoid | 16 | 0.3 | − | $0.81\pm 0.02$ | $0.77\pm 0.06$ | $0.86\pm 0.04$ | $0.88\pm 0.05$ | $0.87\pm 0.04$ |
elu | sigmoid | 32 | 0.3 | − | $0.80\pm 0.02$ | $0.76\pm 0.06$ | $0.85\pm 0.05$ | $0.88\pm 0.05$ | $0.87\pm 0.04$ |
elu | sigmoid | 48 | 0.3 | − | $0.80\pm 0.02$ | $0.76\pm 0.06$ | $0.86\pm 0.04$ | $0.87\pm 0.04$ | $0.87\pm 0.03$ |
elu | sigmoid | 32 | − | + | $0.80\pm 0.02$ | $0.74\pm 0.06$ | $0.84\pm 0.05$ | $0.86\pm 0.05$ | $0.85\pm 0.03$ |
relu | sigmoid | 32 | − | + | $0.80\pm 0.02$ | $0.74\pm 0.06$ | $0.84\pm 0.05$ | $0.87\pm 0.05$ | $0.85\pm 0.03$ |
elu | softmax | 32 | − | + | $0.73\pm 0.04$ | $0.58\pm 0.08$ | $0.63\pm 0.08$ | $0.87\pm 0.05$ | $0.73\pm 0.06$ |
relu | softmax | 32 | − | + | $0.77\pm 0.03$ | $0.65\pm 0.07$ | $0.72\pm 0.07$ | $0.87\pm 0.05$ | $0.78\pm 0.0$ |
3.1.2 Model Performance Speed
Table 4
Model | Parameters | Relative loading time | Relative prediction time |
Micro-Net | 73 467 842 | 1 | 1 |
Custom-16 | 131 746 | 0.212 | 0.314 |
Custom-32 | 279 506 | 0.212 | 0.288 |
Custom-48 | 507 138 | 0.268 | 0.359 |
3.1.3 Active Contour Layer
Table 5
Mask layers | Dice coefficient | Accuracy | Precision | Recall | f-score |
2-layered | $0.81\pm 0.02$ | $0.75\pm 0.06$ | $0.85\pm 0.05$ | $0.86\pm 0.04$ | $0.85\pm 0.04$ |
1-layered | $0.80\pm 0.02$ | $0.73\pm 0.06$ | $0.84\pm 0.05$ | $0.85\pm 0.04$ | $0.84\pm 0.04$ |
3.2 Nuclei Classification
3.2.1 Hyper Parameter Tuning and Model Comparison
Table 6
Models | Accuracy | Precision | Recall | f-score |
Random forest | $0.77\pm 0.002$ | $0.69\pm 0.002$ | $0.99\pm 0.002$ | $0.82\pm 0.002$ |
Multilayer perceptron | ||||
$2048/1024/512$ | $0.78\pm 0.09$ | $0.71\pm 0.1$ | $0.99\pm 0.004$ | $0.83\pm 0.06$ |
$4096/2048/1024$ | $0.78\pm 0.003$ | $0.71\pm 0.03$ | $0.99\pm 0.0003$ | $0.82\pm 0.02$ |
Convolutional neural network | ||||
Kernels per layer: 16 | $0.76\pm 0.09$ | $0.69\pm 0.1$ | $0.98\pm 0.004$ | $0.80\pm 0.06$ |
Kernels per layer: 32 | $0.76\pm 0.09$ | $0.70\pm 0.1$ | $0.98\pm 0.004$ | $0.81\pm 0.06$ |
3.3 Workflow Evaluation
Fig. 4
3.3.1 The Effect of Colour Normalization on Overall Model Performance
(4)
\[\begin{aligned}{}& {l_{\textit{mapped}}}=\frac{{l_{\textit{original}}}-{\bar{l}_{\textit{original}}}}{{\hat{l}_{\textit{original}}}}{\hat{l}_{\textit{target}}}+{\bar{l}_{\textit{target}}},\end{aligned}\]Fig. 5
Table 7
Accuracy | Precision | Recall | f-score | |
Proposed method, original staining | 0.71 | 0.76 | 0.75 | 0.70 |
Proposed method, wt stain normalization | 0.81 | 0.80 | 0.81 | 0.80 |
Janowczyk and Madabhushi (2016) | – | 0.89 | – | 0.90 |
Alom et al. (2019) | 0.90 | – | – | 0.91 |
4 Conclusions
-
• Our proposed autoencoder structure component – convolutional texture blocks – can achieve Dice nuclei segmentation score similar to that of the Micro-Net model (our model achieved 1% higher testing Dice coefficient).
-
• Additional active contour layer in nuclei annotation masks increases nuclei segmentation accuracy by 1.5%.
-
• Lymphocyte classification by multilayer perceptron network achieves $78\pm 0.3$% testing accuracy on the private dataset (NCP), and 0.71 on the public dataset (0.81 with Reinhard stain normalization).