Automated Segmentation of Overlapping Cytoplasm in Cervical Smear Images via Contour Fragments


  • Youyi Song The Hong Kong Polytechnic University
  • Jing Qin The Hong Kong Polytechnic University
  • Baiying Lei Shenzhen University
  • Kup-Sze Choi The Hong Kong Polytechnic University


cervical cancer, overlapping objects segmentation, fragments-based graphical model


We present a novel method for automated segmentation of overlapping cytoplasm in cervical smear images based on contour fragments. We formulate the segmentation problem as a graphical model, and employ the contour fragments generated from cytoplasm clump to construct the graph. Compared with traditional methods that are based on pixels, our contour fragment-based solution can take more geometric information into account and hence generate more accurate prediction of the overlapping boundaries. We further design a novel energy function for the graph, and by minimizing the energy function, fragments that come from the same cytoplasm are selected into the same set. To construct the energy function, our fragments-based data term and pairwise term are measured from the spatial relation and shape prior, which offer more geometric information for the occluded boundary inference. Afterwards, occluded boundaries are inferred using the minimal path model, in which shape of each individual cytoplasm is reconstructed on the selected fragments set. Constructed shape is used as a constraint to locate the searching area, and curvature regulation is enforced to promote the smoothness of inference result. The inference result, in turn, is used as the shape prior to construct a high-level shape regulation energy term of the built graph, and then graph energy is updated. In other words, fragments selection and occluded boundary inference are iterative processed; this interaction makes more potential shape information accessible. Using two cervical smear datasets, the performance of our method is extensively evaluated and compared with that of the state-of-the-art approaches; the results show the superiority of the proposed method.




How to Cite

Song, Y., Qin, J., Lei, B., & Choi, K.-S. (2018). Automated Segmentation of Overlapping Cytoplasm in Cervical Smear Images via Contour Fragments. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). Retrieved from