Good news! Our new work exhibits state-of-the-art performances on the DocUNet Benchmark dataset: DocScanner: Robust Document Image Rectification with Progressive Learning
Good news! A comprehensive list of Awesome Document Image Rectification methods is available.
DocTr
DocTr: Document Image Transformer for Geometric Unwarping and Illumination Correction
ACM MM 2021 Oral
Any questions or discussions are welcomed!
Training
DocTr consists of two main components: a geometric unwarping transformer (GeoTr) and an illumination correction transformer (IllTr).
- For geometric unwarping, we train the GeoTr network using the Doc3D dataset.
- For illumination correction, we train the IllTr network based on the DRIC dataset.
Inference
- Download the pretrained models from Google Drive or Baidu Cloud, and put them to
$ROOT/model_pretrained/. - Put the distorted images in
$ROOT/distorted/. - Geometric unwarping. The rectified images are saved in
$ROOT/geo_rec/by default.python inference.py - Geometric unwarping and illumination rectification. The rectified images are saved in
$ROOT/ill_rec/by default.python inference.py --ill_rec True
Evaluation
- In the DocUNet Benchmark, the '64_1.png' and '64_2.png' distorted images are rotated by 180 degrees, which do not match the GT documents. It is ingored by most of existing works. Before the evaluation, please make a check.
- We use the same evaluation code for MS-SSIM and LD as DocUNet Benchmark dataset based on Matlab 2019a. Please compare the scores according to your Matlab version. We provide our Matlab interface file at
$ROOT/ssim_ld_eval.m. - The index of 30 document (60 images) of DocUNet Benchmark used for our OCR evaluation is
$ROOT/ocr_img.txt(Setting 1). Please refer to DewarpNet for the index of 25 document (50 images) of DocUNet Benchmark used for their OCR evaluation (Setting 2). We provide the OCR evaluation code at$ROOT/OCR_eval.py. The version of pytesseract is 0.3.8, and the version of Tesseract is recent 5.0.1.20220118. - Use the rectified images available from Google Drive or Baidu Cloud for reproducing the quantitative performance on the DocUNet Benchmark reported in the paper and further comparison. We show the performance results of DocTr in the following table. For the performance of other methods, please refer to DocScanner.
| Method | MS-SSIM | LD | ED (Setting 1) | CER | ED (Setting 2) | CER |
|---|---|---|---|---|---|---|
| GeoTr | 0.5105 | 7.76 | 464.83 | 0.1746 | 724.84 | 0.1832 |
Citation
If you find this code useful for your research, please use the following BibTeX entry.
@inproceedings{feng2021doctr,
title={DocTr: Document Image Transformer for Geometric Unwarping and Illumination Correction},
author={Feng, Hao and Wang, Yuechen and Zhou, Wengang and Deng, Jiajun and Li, Houqiang},
booktitle={Proceedings of the 29th ACM International Conference on Multimedia},
pages={273--281},
year={2021}
}
@article{feng2021docscanner,
title={DocScanner: Robust Document Image Rectification with Progressive Learning},
author={Feng, Hao and Zhou, Wengang and Deng, Jiajun and Tian, Qi and Li, Houqiang},
journal={arXiv preprint arXiv:2110.14968},
year={2021}
}
Acknowledgement
The codes are largely based on DocUNet, DewarpNet, and DocProj. Thanks for their wonderful works.