Intelligent Data Extraction on Server/Desktop

When using Python on Windows or Linux you can install the package via PIP with this command:

pip install --extra-index-url=https://pypi.apryse.com apryse-data-extraction

When using Node.js on Windows or Linux you can install the package via NPM with this command:

npm install @pdftron/data-extraction

For Windows, just copy DataExtractionModuleWindows.zip in your PDFNetC folder, then extract it locally. You should have files like

• Lib\Windows\StructuredOutput.exe
• Lib\Windows\OCRModule.exe
• Lib\Windows\TabularData\TabularData.dll
• Lib\Windows\AIPageObjectExtractor\AIPageObjectExtractor.dll

For Linux, just copy DataExtractionModuleLinux.tar.gz in your PDFNetC directory, then extract it locally. You should have files like

• Lib/Linux/StructuredOutput
• Lib/Linux/OCRModule
• Lib/Linux/TabularData/TabularData
• Lib/Linux/AIPageObjectExtractor/AIPageObjectExtractor

Please refer to the below specifications to learn more about the output JSON format.

• JSON Specification for Tabular Data and Document Structure
• JSON Specification for Form Field Identification

If you are using PIP or NPM, you may skip setting AddResourceSearchPath. Otherwise, follow the directions below.

The first thing to set up before the module can be used is the location of the Lib directory under which the external add-ons are installed, so that the SDK knows where to look for them. This is achieved via the PDFNet AddResourceSearchPath function. If a relative path is used, it is based on the end-user executable.

PDFNet.AddResourceSearchPath("../../../../../Lib/");

Note: do not specify the actual Windows, Linux, MacOS directory, where the individual executables are, but its parent folder.

For error handling purposes, it is generally advisable to test whether the module is available via the IsModuleAvailable function. Since the Data Extraction suite consists of multiple modules, an extra parameter is used to clarify the component to test.

if (!DataExtractionModule.IsModuleAvailable(DataExtractionModule.DataExtractionEngine.e_tabular))
{
   // Unable to run Data Extraction: PDFTron SDK Tabular Data module not available.
}
if (!DataExtractionModule.IsModuleAvailable(DataExtractionModule.DataExtractionEngine.e_doc_structure))
{
   // Unable to run Data Extraction: PDFTron SDK Structured Output module not available.
}
if (!DataExtractionModule.IsModuleAvailable(DataExtractionModule.DataExtractionEngine.e_form))
{
   // Unable to run Data Extraction: PDFTron SDK AIFormFieldExtractor module not available.
}

If you have the module installed but the function still returns false, please double check that the correct path was used in AddResourceSearchPath earlier.

In this mode of operation, text on each page will become a part of a single table, like in a spreadsheet document. Headers, footers, paragraphs, lists, page numbers all come out of a combined tabular structure.

There is a choice between two types of output formats, either a JSON or an Excel document. JSON is more suitable for programmatic content discovery, natural language processing, statistical analysis, artificial intelligence. Excel is ideal for standard spreadsheet operations, calculations, formulas and charting.

Specify the name of the input PDF file and the name of the output JSON file, then select the Tabular engine:

DataExtractionModule.ExtractData("table.pdf", "table.json", DataExtractionModule.DataExtractionEngine.e_tabular);

If you are going to parse the JSON right away, you may as well retrieve it as an in-memory string, instead of an external file.

Specify the name of the input PDF file, then select the Tabular engine:

string json = DataExtractionModule.ExtractData("financial.pdf", DataExtractionModule.DataExtractionEngine.e_tabular);

Specify the name of the input PDF file and the name of the output XLSX file:

DataExtractionModule.ExtractToXLSX("table.pdf", "table.xlsx");

Specify the name of the input PDF file and an output filter, such as MemoryFilter:

MemoryFilter output_xlsx_stream = new MemoryFilter(0, false);
DataExtractionModule.ExtractToXLSX("financial.pdf", output_xlsx_stream);

In this mode of operation, the full logical structure is discovered, including paragraphs, lists, tables, headers, footers, images, graphics, like in a typical word processor. Section columns are differentiated from table columns, and paragraph text is separated from tables, although table cells may contain further paragraphs.

While Tabular Data Extraction is more focused on cells, financial tables, spreadsheets, formulas, currency signs, percentages and calculations on cells, Document Structure Recognition aims at the visual accuracy of documents, such as indentation, gaps, line spacing, alignment, borders and colors.

Specify the name of the input PDF file and the name of the output JSON file, then select the Doc Structure engine:

DataExtractionModule.ExtractData("paragraphs_and_tables.pdf", "paragraphs_and_tables.json", DataExtractionModule.DataExtractionEngine.e_doc_structure);

If you are going to parse the JSON right away, you may as well retrieve it as an in-memory string, instead of an external file.

Specify the name of the input PDF file, then select the Doc Structure engine:

string json = DataExtractionModule.ExtractData("tagged.pdf", DataExtractionModule.DataExtractionEngine.e_doc_structure);

In this mode of operation, the input is assumed to be a form. We currently offer 2 Form Field Identification Engines: "Form Field Detection" and "Form Field Key-Value Extraction".

Both engines require GLIBC 2.31 or newer on Linux, such as Debian 11 or Ubuntu 10.04 or newer

Our Form Field Detection engine surveys the page layout and finds the most probable arrangement of the individual form fields, along with the type of the identified fields, such as text fields or check boxes.

The output is presented in a straightforward JSON format, where each field is made up of a type, confidence value and bounding box coordinates.

Our Form Field Key-Value Extraction engine provides all the same information as the Form Field Detection engine, with some additional data provided for fields. For each form field in the document, an attempt is made to find the corresponding key (the label or question associated with a field) and value (the data within the field).

The output is presented in a JSON format, similar to the output of the Form Field Detection engine, with extra data fields for field key, label, and value text.

Specify the name of the input PDF file and the name of the output JSON file, then select the Form engine:

DataExtractionModule.ExtractData("formfields-scanned.pdf", "formfields-scanned.json", DataExtractionModule.DataExtractionEngine.e_form);

Alternatively, you can select the Form Key-Value Extraction engine:

DataExtractionModule.ExtractData("formfields-scanned.pdf", "formfields-scanned.json", DataExtractionModule.DataExtractionEngine.e_form_key_value);

If you are going to parse the JSON right away, you may as well retrieve it as an in-memory string, instead of an external file.

Specify the name of the input PDF file, then select the Form engine:

string json = DataExtractionModule.ExtractData("formfields.pdf", DataExtractionModule.DataExtractionEngine.e_form);

Alternatively, you can select the Form Key-Value Extraction engine:

string json = DataExtractionModule.ExtractData("formfields.pdf", DataExtractionModule.DataExtractionEngine.e_form_key_value);

You can automatically add detected forms to a PDF in a single step.

PDFDoc doc = new PDFDoc("formfields.pdf");
DataExtractionModule.detectAndAddFormFieldsToPDF(doc);

PDFDoc doc = new PDFDoc("formfields.pdf");
DataExtractionModule.DetectAndAddFormFieldsToPDF(doc);

By default, this function uses the Form Field Detection engine. To use the Form Field Key-Value Extraction engine, see below.

Although the default options will satisfy most common use cases, we offer a couple of options to customize the extraction behavior and unlock lesser-used functionality.

The options object is passed as the last parameter to any extraction function, as shown below.

Use the Language option to set the preferred OCR language(s). If you work with scanned documents in languages other than English, specify one or more 3-letter ISO 639-2 language codes, separated by spaces. For example, "eng deu spa fra" for English, German, Spanish, French. You may also use comma or plus as a separator.

Supported languages:

• eng: English
• deu or ger: German
• fra or fre: French
• ita: Italian
• rus: Russian
• spa: Spanish

Note: Listing too many languages at once may hurt performance and accuracy. If you know the exact language, it is always best to use that single setting.

DataExtractionOptions options = new DataExtractionOptions();
options.SetLanguage("fra spa"); // French and Spanish
DataExtractionModule.ExtractData("table.pdf", "table.json", DataExtractionModule.DataExtractionEngine.e_tabular, options);

Use the PDFPassword option to specify a PDF password if one is required.

Encrypted PDF files that are protected by a password may only be opened when the password is specified in addition to the filename. No password is necessary for files that can be viewed without any authentication.

DataExtractionOptions options = new DataExtractionOptions();
options.SetPDFPassword("password123"); // password for input PDF
DataExtractionModule.ExtractData("table.pdf", "table.json", DataExtractionModule.DataExtractionEngine.e_tabular, options);

Use the Pages option to restrict the extraction to a selected range of pages.

This can be a single page number (such as "1" for the first page), or a range separated by a dash (such as "1-5", or "7-" for 7 and beyond). An empty string means all pages are extracted.

DataExtractionOptions options = new DataExtractionOptions();
options.SetPages("1"); // extract page 1
DataExtractionModule.ExtractData("table.pdf", "table.json", DataExtractionModule.DataExtractionEngine.e_tabular, options);

Specifies if Deep Learning is used with table recognition in the DocStructure engine. Table recognition accuracy improves at the cost of increased processing time. This only affects the DocStructure engine.

DataExtractionOptions options = new DataExtractionOptions();
options.SetDeepLearningAssist(true); // Enable Deep learning assistant
DataExtractionModule.ExtractData("table.pdf", "table.json", DataExtractionModule.DataExtractionEngine.e_DocStructure, options);

When automatically detecting form fields and adding them to a document, you can force the module to preserve any existing form annotations that are already present in the document, only adding newly detected fields.

PDFDoc doc = new PDFDoc("formfields.pdf");
DataExtractionOptions options = new DataExtractionOptions();
options.SetOverlappingFormFieldBehavior("KeepOld");
DataExtractionModule.DetectAndAddFormFieldsToPDF(doc, options);

By default, DetectAndAddFormFieldsToPDF uses the Form Field Detection engine. You can force the function to use the Form Field Key-Value Extraction engine using the "Form Extraction Engine" option.

PDFDoc doc = new PDFDoc("formfields.pdf");
DataExtractionOptions options = new DataExtractionOptions();
options.SetFormExtractionEngine("FormKeyValue");
DataExtractionModule.DetectAndAddFormFieldsToPDF(doc, options);