Extract Data From PDF File

[dasjoe]

Hi all
I have a situation where certain information from PDF documents is entered into a Data Base. I need an easy way for a user to do this. So given a situation where a data entry program is running, then certain data is taken off a PDF document, then entered on to a data entry form etc. So it would be nice to say, have a user click on certain entries on the PDF document, or the document itself and then have the required data load on the data entry form automatically, with using cut and paste on each entry. Note the required data from the PDF document is in same each time, just different values. Anyone have any ideas ??

Many thanks

[Dnereb]

I'm afraid this is a mission impossible.
A pdf can be stored in so many diffrent ways...
the most horrific version is a picture/scan containing the text information you want.
an optimzed pdf ussually contains text as three letter fragments instead of complete lines of text. (this is a way of compressing information by pointing to a letter combination like 'ing' instead of repeating the combination everywhere it occurs)

So distilling info from a pdf... don't.
use a diffrent format like XML, HTML, TXT, RTF, etc... as PDF is a closed format, and in later versions even has read/copy restrictions and security on the file.

[chris128]

I would be inclined to agree - it would kind of defeat the entire purpose of PDFs if you could just make a program manipulate them... although I guess it wouldn't strictly be impossible, it would just be very very hard.

EDIT: I just remembered actually that I think you can get OCR software (cant remember what the O stands for but i think the CR stands for Character Recognition) and that essentialy treats the PDF like it was a picture and just tries to match parts of the PDF with character shapes... I seem to remember them being very unreliable but that was about 2 years ago, maybe things have improved. Either way, it probably doesnt help you much as I doubt you are going to try and make a full OCR app.

[dasjoe]

Thanks Dnereb
That's the conclusion I have come to, though these PDF file do allow a copy function and select all function. The format of the PDF Doc is the same everytime, an option I have (assuming format never changes) is to select All----Copy----Past to say a text box (this can be hidden) then have the program select the data from the text box, as the format is always the same, then the data should be located in the same place in the text box, thats the only approach I have currently, but thought there may be another approach.

[dasjoe]

I dont need to manipulate the PDF doc, just read selected data only that happens to be in the same location every time (fortunately)

[ntg]

Can you post an example PDF?

[dasjoe]

Can't attach sample copy of PDF Doc, clients confidentiality restricts that, but basically they are stock broker contract notes when share are purchased on the stock market. eg when a trade is made, whether sell or buy, then a contract note is issued PDF format. Now obviously the broker has a standard PDF layout and then the fields are filled in appropriately for each trade order. eg Quantity, Buy price, Contract Number, etc. So what I need is a way of extracting the fields that are entered by the broker for each particular executed order from the PDF doc in a simple process for the user.
Now the location of each field, is located in the same place every time. (thats good) but that could change, so I need a way of scanning through the PDF Doc, do some checks against the data to see if it is the correct data, eg word search, is money, is numeric etc.

Any ideas ?.
Note, as I mentioned the PDF doc are not locked for select all, or Copy etc

[stanav]

You can use iTextSharp to read the values of those AcroFields in a pdf file. But without having a sample pdf, I can't help much on this.

[dasjoe]

Thanks stanav
I cannot send copy of the PDF doc is question(Too much private info), but have located another PDF Doc that would simulate the same problem. The fields are not filled in because it's locked for saving the entered fields, but imagine the fields completed by the sender of the PDFF doc. Now I want to extract those fields and place then onto another data entry program in a quick operation for the user.

Cheers

[stanav]

OK... Did you know that not every thing you see in that pdf is a field? If you open that pdf file and put a check mark on "Highlight fields", only those that are highlighted are AcroFields, and you can extract data from them fairly easy. Anything that is not highlighted can not be extracted.

And these are the steps I did to analyze the pdf, then find out how to extract the data from those fields.
1. OPen the pdf in Acrobat. Fill in some data in those fields, then save the file to a new pdf.
2. Write some code to open that new pdf, get the fields and extract data, save it in a dictionary(of string, string) where the keys are the field names and the values are the actual values for those fields.

3. Now look at the dictionary, and compare it with the pdf, you will see the location of the fields on the pdf (based on the values), so that you can use the extracted data correctly.

And here is the function to extract the fields' data

'Add a reference of iTextSharp to your project first

Imports iTextSharp.text.pdf

Public Function GetFormData(ByVal sourcePdf As String) As Dictionary(Of String, String)
        Dim frmData As New Dictionary(Of String, String)
        Try
            'Open the pdf using pdfreader
            Dim reader As New PdfReader(sourcePdf)
            'Get the form from the pdf
            Dim frm As AcroFields = reader.AcroFields
            'get the fields from the form
            Dim fields As System.Collections.Hashtable = frm.Fields
            'Extract the data from the fields
            Dim data As String = String.Empty
            For Each key As String In fields.Keys
                data = frm.GetField(key)
                frmData.Add(key, data)
            Next
            reader.Close()
        Catch ex As Exception
            Debug.Write(ex.Message)
        End Try
        Return frmData
    End Function

And to use it, you simply call the function passing in the path to your pdf file. Something like this

 Dim fieldData As Dictionary(Of String, String) = GetFormData("put the path to your pdf file here")
        For Each key As String In fieldData.Keys
            Debug.WriteLine(String.Format("FieldName: {0}, FieldData: {1}", key, fieldData(key)))
        Next

[dasjoe]

Thanks stanav
I realise that not all on a PDF are fields, but in my case all the data that needs to be collected are contain in fields.

Thanks for the code, I need to download the iTextSharp library which i will do
One question???
Is it possible and can you give eg code on how to get the field names (I presume they have names) from a PDF Doc as I will need to extract the data one field at a time.

Thanks again

[stanav]

The function I posted in the last post returns a Dictionary which gives you both the field names and their values. The field names are used as keys for that dictionary and the the keys' values are the fields' values. So if you want to get the field names, you just read them from the dictionary.Keys collection.

[dasjoe]

Thanks Stanav
Works great, added drag/drop, so one click and mouse drag/drop and data is entered.


Now to extend this further, PDF Docs that don't contain fields.
Getting the whole thing as a text file, appears to be easy enough, but retrieving it in some sort of order is now the question, not knowing a lot about the PDf structure, is there some sort of tag to each block of text that perhaps makes it identifiable.??? Also when the text is extracted on the PDF docs that I need to extract from , the text is read in a tubular form (like a news paper), not across that page line by line as this would have made it easy to extract in some sort of identifiable order.
Any Ideas

[stanav]

Text extraction from a pdf is not as easy as you may think, especially with iTextSharp. You can use iTextSharp to extract text but only to some extends... That is, the extracted text all come out as in single blob, and you loose all the formats (lines, identations).

There may be other products out there that can do a better job with text extraction, but as far as I know, none of them will can guarantee to retain the format.

[dasjoe]

I have tried iTextSharp on non field PDF's and yes thats what I can only extract (All in one text file with no other formatting information)
Though, I would have thought that there would some tag info attached.
Anyway thanks for the help

[EfficientCEO]

I know your post was awhile ago, but if you are still looking for the answer I have it. I designed a solution for this very problem: helping people move data to and from a PDF.

Using the free Adobe Reader, my software, called Quik!, will enable you to have a submit button on your form for your user to click and submit the form data directly to your database and/or data entry system.

My company name is Efficient Technology Inc and I developed Quik! in 2001 because I didn't like filling out forms by hand. Using Quik! your web or desktop application can interact with PDFs to do the following:

- Prefill forms with existing data from a database or user-entered data
- Bundle multiple forms together into a single PDF
- Save forms without Adobe Acrobat
- Submit form data to a back-office process
- Electronically sign forms
- Communicate from your server to the form

If you'd like more information about how we can enable your applications to automate forms, please go to www.QuikForms.com or contact me directly.

[ntg]

Nice sales pitch.

[MaximilianMayrhofer]

The necromancer strikes again.