TABLE STRUCTURE RECOGNITION

1. OBJECTIVE OF RESEARCH -

Data mining is widely used to identify patterns and anomalies in large datasets, with financial documents serving as a crucial source for businesses. Extracting financial information correctly and efficiently is vital to prevent data forgery and identify significant issues in business activities. Tables are a primary format for presenting key figures in financial documents, but their style and format can vary, making table recognition a critical step in financial data forgery prevention. While academic and commercial approaches exist to recognize tables in various document formats, there is limited work on image-based table recognition in real-world scenarios, with limited training data being the main technical challenge.

The proposed challenge aims to explore the latest techniques for recognizing table structures in real financial documents and focuses on identifying the row and column count of tables from an image of a financial document.

2. RESEARCH WORK IN THE TOPIC -

• *CODE means official code and CODE means not official code

Conf.	Date	Title	Highlight	code
ACM-MM	2022	TSRFormer: Table Structure Recognition with Transformers	Detection	No
CVPR	2022	TableFormer: Table Structure Understanding with Transformers.	Sequence	No
CVPR	2022	Neural Collaborative Graph Machines for Table Structure Recognition	GNN	No
CVPR	2022	PubTables-1M: Towards comprehensive table extraction from unstructured documents	Dataset	*CODE
arXiv	2021/5/23	Multi-Type-TD-TSR -- Extracting Tables from Document Images using a Multi-stage Pipeline for Table Detection and Table Structure Recognition: from OCR to Structured Table Representations	Others	*CODE
ACM-MM	2021	Show, Read and Reason: Table Structure Recognition with Flexible Context Aggregator	GNN	No
ICCV	2021	Parsing Table Structures in the Wild	Detection	No
ICCV	2021	TGRNet: A Table Graph Reconstruction Network for Table Structure Recognition	GNN	*CODE
ICDAR Competition	2021	ICDAR 2021 Competition on Scientific Literature Parsing	Dataset	*CODE
ICDAR Competition	2021	PingAn-VCGroup's Solution for ICDAR 2021 Competition on Scientific Literature Parsing Task B: Table Recognition to HTML	Sequence	*CODE
ICDAR Competition	2021	LGPMA: Complicated Table Structure Recognition with Local and Global Pyramid Mask Alignment	Others	*CODE
WACV	2021	Global table extractor (gte): A framework for joint table identification and cell structure recognition using visual context	Others	No
CVPR Workshop	2020	CascadeTabNet: An approach for end to end table detection and structure recognition from image-based documents	Others	*CODE
ECCV	2020	Image-based table recognition: data, model, and evaluation	Dataset	*CODE
ECCV	2020	Table structure recognition using top-down and bottom-up cues	Others	*CODE
LREC	2020	TableBank: A Benchmark Dataset for Table Detection and Recognition	Dataset	*CODE
arXiv	2019/8/28	Complicated table structure recognition	Others	*CODE
ICDAR	2019	Rethinking Table Recognition using Graph Neural Networks	GNN	*CODE
ICDAR	2019	Tablenet: Deep learning model for end-to-end table detection and tabular data extraction from scanned document images	Others	No
ICDAR	2019	Res2tim: Reconstruct syntactic structures from table images.	Others	*CODE
ICDAR	2017	Deepdesrt: Deep learning for detection and structure recognition of tables in document images	Others	No

3. DATASETS -

3.1 INTRODUCTION:

Dataset	Description	dataset link
TableBank	English TableBank is a new image-based table detection and recognition dataset built with novel weak supervision from Word and Latex documents on the internet, contains 417K high-quality labeled tables.It only contain cell Topology groudtruth	TableBank
SciTSR	*English SciTSR is a large-scale table structure recognition dataset, which contains 15,000 tables in PDF format and their corresponding structure labels obtained from LaTeX source files.It contain cell Topology, cell content groudtruth	SciTSR
PubTabNet	English PubTabNet is a large dataset for image-based table recognition, containing 568k+ images of tabular data annotated with the corresponding HTML representation of the tables.It contain cell Topology, cell content and non-blank cell location groudtruth	PubTabNet
FinTabNet	English This dataset contains complex tables from the annual reports of S&P 500 companies with detailed table structure annotations to help train and test structure recognition.	FinTabNet
PubTables-1M	English A large, detailed, high-quality dataset for training and evaluating a wide variety of models for the tasks of table detection, table structure recognition, and functional analysis.	PubTables-1M
WTW	English and Chinese WTW-Dataset is the first wild table dataset for table detection and table structure recongnition tasks, which is constructed from photoing, scanning and web pages, covers 7 challenging cases like: (1)Inclined tables, (2) Curved tables, (3) Occluded tables or blurredtables (4) Extreme aspect ratio tables (5) Overlaid tables, (6) Multi-color tables and (7) Irregular tables in table structure recognition.It contain cell Topology, all cell location groudtruth	WTW
TNCR	English a new table dataset with varying image quality collected from open access websites.TNCR contains 9428 labeled tables with approximately 6621 images.their classification into 5 different classes(Full Lined,Merged Cells,No lines,Partial Lined,Partial Lined Merged Cells).	TNCR
TAL_OCR_TABLE	Chinese TAL_OCR_TABLE dataset come from TAL Form Recognition Technology Challenge.The data of comes from the real homework of students in the education scene and the scene of the test paper. It contain 16k train image and 4k test imageIt contain cell Topology, cell content and all cell location groudtruth	TAL_OCR_TABLE

3.2 COMPARISION OF DATASETS:

Dataset	Cell Topology	Cell content	Cell Location	Table Location
TableBank	✓	✕	✕	✓
SciTSR	✓	✓	✕	✓
PubTabNet	✓	✓	✓†	✓
FinTabNet	✓	✓	✓†	✓
PubTables-1M	✓	✓	✓	✓
WTW	✓	✕	✓	✓
TNCR	✕	✕	✕	✓
TAL_OCR_TABLE	✓	✓	✓	✓

^† For these datasets, cell bounding boxes are given for non-blank cells only and exclude any non-text portion of a cell.

4. PRE-TRAINED MODELS & THEIR NOTEBOOKS -

Model	Notebook(Google Colab)
CascadeTabNet	https://colab.research.google.com/drive/1hvguI08b0vTJ8WAUMcG7t9pVaf4xb8Hc?usp=share_link
Table Transformer	https://colab.research.google.com/drive/1tfkbSlukfRgkuXKpV1yzG-CX0v2yadlj?authuser=2#scrollTo=_819yZ8sABN6
YOLO	https://colab.research.google.com/drive/1b2cqhp2KHpWh4HJZpwR2-iM3NlIE9waQ?usp=share_link
Multi-Type TD TSR	https://colab.research.google.com/drive/1sODx1likW2ubE6s019B-6Z7UFihD5wHv?authuser=3#scrollTo=y7Lk83F88NHW
RetinaNet	https://colab.research.google.com/drive/1mpvbfLicRNvT4trj6ALnGyewUw9Hzbk_?usp=share_link
TableNet	https://colab.research.google.com/drive/1sHl73sjGjt-eeEtNsr2dov-8Q0ea9AQc?usp=share_link
SSD	https://colab.research.google.com/drive/12slrZG5skLgN6xKooCiiYdMvD9BVOQ8H?usp=share_link

5. OUR RESEARCH WORK (TIMELINE) -

5.1 EXAMINING THE PATTERNS OF THE LATEST MODELS:

• For Evaluation We considered Three Models:-

  • CascadeTabNet
  • Multi-Type-TD-TSR
  • TGRNet

  The Detailed report of the work can be found here.

5.2 RE-EVALUATION OF THE PRETRAINED MODELS:

• The aforementioned pre-trained models were tested on TableGraph24k that was unfamiliar to each model to assess their capabilities in predicting bounding borders and extracting structures. The results for all the three types can be seen here.