Section:

Smart Data Extraction on Server/Desktop

Requirements
View Demo
Package: Smart Data Extraction
Module: Data Extraction

Installation Instructions

This guide walks you through installing and configuring Apryse’s Data Extraction Module so you can start extracting data from PDFs quickly and reliably.

Using PIP with Python

When using Python on Windows or Linux you can install the package via PIP with this command:

x64 is supported, but Arm and 32 bit are not.

sh

1pip install --extra-index-url=https://pypi.apryse.com apryse-data-extraction

Using NPM with Node.js

When using Node.js on Windows or Linux you can install the package via NPM with this command:

x64 is supported, but Arm and 32 bit are not.

sh

1npm install @pdftron/data-extraction

Installing directly on other platforms

For Windows, just copy DataExtractionModuleWindows.zip in your PDFNetC folder, then extract it locally. You should have files like:

x64 is supported, but Arm and 32 bit are not.

  • Lib\Windows\StructuredOutput.exe
  • Lib\Windows\OCRModule.exe
  • Lib\Windows\TabularData\TabularData.dll
  • Lib\Windows\AIPageObjectExtractor\AIPageObjectExtractor.dll

For Linux, just copy DataExtractionModuleLinux.tar.gz in your PDFNetC directory, then extract it locally. You should have files like

  • Lib/Linux/StructuredOutput
  • Lib/Linux/OCRModule
  • Lib/Linux/TabularData/TabularData
  • Lib/Linux/AIPageObjectExtractor/AIPageObjectExtractor

JSON Output Specification

Please refer to the below specifications to learn more about the output JSON format.

Usage

If you are using PIP or NPM, you may skip setting AddResourceSearchPath. Otherwise, follow the directions below.

The first thing to set up before the module can be used is the location of the Lib directory under which the external add-ons are installed, so that the SDK knows where to look for them. This is achieved via the PDFNet AddResourceSearchPath function. If a relative path is used, it is based on the end-user executable.

1PDFNet.AddResourceSearchPath("../../../../../Lib/");

Note: do not specify the actual Windows, Linux, MacOS directory, where the individual executables are, but its parent folder.

For error handling purposes, it is generally advisable to test whether the module is available via the IsModuleAvailable function. Since the Data Extraction suite consists of multiple modules, an extra parameter is used to clarify the component to test.

1if (!DataExtractionModule.IsModuleAvailable(DataExtractionModule.DataExtractionEngine.e_tabular))
2{
3   // Unable to run Data Extraction: PDFTron SDK Tabular Data module not available.
4}
5if (!DataExtractionModule.IsModuleAvailable(DataExtractionModule.DataExtractionEngine.e_doc_structure))
6{
7   // Unable to run Data Extraction: PDFTron SDK Structured Output module not available.
8}
9if (!DataExtractionModule.IsModuleAvailable(DataExtractionModule.DataExtractionEngine.e_form))
10{
11   // Unable to run Data Extraction: PDFTron SDK AIFormFieldExtractor module not available.
12}
13if (!DataExtractionModule.IsModuleAvailable(DataExtractionModule.DataExtractionEngine.e_generic_key_value))
14{
15   // Unable to run Data Extraction: PDFTron SDK AIGenericKeyValue module not available.
16}
17if (!DataExtractionModule.IsModuleAvailable(DataExtractionModule.DataExtractionEngine.e_doc_classification))
18{
19   // Unable to run Data Extraction: PDFTron SDK AIDocClassification module not available.
20}

If you have the module installed but the function still returns false, please double check that the correct path was used in AddResourceSearchPath earlier.

Data Extraction Options

Although the default options will satisfy most common use cases, we offer a couple of options to customize the extraction behavior and unlock lesser-used functionality.

The options object is passed as the last parameter to any extraction function, as shown below.

Select OCR Language

Use the Language option to set the preferred OCR language(s). If you work with scanned documents in languages other than English, specify one or more 3-letter ISO 639-2 language codes, separated by spaces. For example, "eng deu spa fra" for English, German, Spanish, French. You may also use comma or plus as a separator.

Supported languages:

  • eng: English
  • deu or ger: German
  • fra or fre: French
  • ita: Italian
  • rus: Russian
  • spa: Spanish

Note: Listing too many languages at once may hurt performance and accuracy. If you know the exact language, it is always best to use that single setting.

1DataExtractionOptions options = new DataExtractionOptions();
2options.SetLanguage("fra spa"); // French and Spanish
3DataExtractionModule.ExtractData("table.pdf", "table.json", DataExtractionModule.DataExtractionEngine.e_tabular, options);

Specify PDF Password

Use the PDFPassword option to specify a PDF password if one is required.

Encrypted PDF files that are protected by a password may only be opened when the password is specified in addition to the filename. No password is necessary for files that can be viewed without any authentication.

1DataExtractionOptions options = new DataExtractionOptions();
2options.SetPDFPassword("password123"); // password for input PDF
3DataExtractionModule.ExtractData("table.pdf", "table.json", DataExtractionModule.DataExtractionEngine.e_tabular, options);

Select a Page Range

Use the Pages option to restrict the extraction to a selected range of pages.

This can be a single page number (such as "1" for the first page), or a range separated by a dash (such as "1-5", or "7-" for 7 and beyond). An empty string means all pages are extracted.

1DataExtractionOptions options = new DataExtractionOptions();
2options.SetPages("1"); // extract page 1
3DataExtractionModule.ExtractData("table.pdf", "table.json", DataExtractionModule.DataExtractionEngine.e_tabular, options);

Specify Regions of Interest

You can specify regions to include or exclude from analysis for each page in a document using the Inclusion Zone and Exclusion Zone options for a page. These options specify rectangles in user-space coordinates that allow developers to either include or exclude a region from analysis. For example, if a document has a table that you don't want to analyze, you could specify it's bounding box as an exclusion zone, or if a document has only one paragraph that you care about, you could use an inclusion zone. If no zones are specified for a page, the entire page is included in analysis.

Inclusion and exclusion zones can be combined to create complex regions of interest. Inclusions zones are combined by union, and exclusion zones are subtracted.

This option is only supported for the Form, FormKeyValue, and GenericKeyValue engines at this time.

Inclusion and Exclusion example

1DataExtractionOptions options = new DataExtractionOptions();
2
3RectCollection p4InclusionZones = new RectCollection();
4RectCollection p4ExclusionZones = new RectCollection();
5p4InclusionZones.AddRect(30, 432, 562, 684);
6p4ExclusionZones.AddRect(30, 657, 295, 684);
7options.AddInclusionZonesForPage(p4InclusionZones, 4);
8options.AddExclusionZonesForPage(p4ExclusionZones, 4);
9
10DataExtractionModule.ExtractData("newsletter.pdf", "newsletter_key_val_with_zones.json", DataExtractionModule.DataExtractionEngine.e_generic_key_value, options);

Deep Learning Assist

Specifies if Deep Learning is used with table recognition in the DocStructure engine. Table recognition accuracy improves at the cost of increased processing time. This only affects the DocStructure engine.

1DataExtractionOptions options = new DataExtractionOptions();
2options.SetDeepLearningAssist(true); // Enable Deep learning assistant
3DataExtractionModule.ExtractData("table.pdf", "table.json", DataExtractionModule.DataExtractionEngine.e_DocStructure, options);

Preserve existing form fields when adding to PDF

When automatically detecting form fields and adding them to a document, you can force the module to preserve any existing form annotations that are already present in the document, only adding newly detected fields.

1PDFDoc doc = new PDFDoc("formfields.pdf");
2DataExtractionOptions options = new DataExtractionOptions();
3options.SetOverlappingFormFieldBehavior("KeepOld");
4DataExtractionModule.DetectAndAddFormFieldsToPDF(doc, options);

Detect Empty Fields

NEW FEATURE

New in 11.8!

Specifies if empty fields should be recognized in the GenericKeyValue engine. The default is true. Users who don't require empty fields could benefit from setting this option to false, thus reducing processing time.

This option only affects the GenericKeyValue engine.

1DataExtractionOptions options = new DataExtractionOptions();
2options.SetDetectEmptyFields(false);
3DataExtractionModule.ExtractData("newsletter.pdf", "newsletter_key_val_with_zones.json", DataExtractionModule.DataExtractionEngine.e_generic_key_value, options);

Minimum Confidence Threshold

NEW FEATURE

New in 11.8!

Specifies the minimum confidence threshold for a class to be accepted in the DocClassification engine. The default is 0.25. Classes that don't meet the minimum threshold will not be listed in the output JSON.

This option only affects the DocClassification engine.

1DataExtractionOptions options = new DataExtractionOptions();
2options.SetMinimumConfidenceThreshold(0.7);
3DataExtractionModule.ExtractData("Email.pdf", "Email_Classified.json", DataExtractionModule.DataExtractionEngine.e_doc_classification, options);

Did you find this helpful?

Trial setup questions?

Ask experts on Discord

Need other help?

Contact Support

Pricing or product questions?

Contact Sales