Simple ways to extract Tabular Content from PDFs
Whether you are a digital marketing firm or a garment wholesaler, setting your digital base and keeping your books digitalized has become imperative. Several sectors today function on digital documents only. Amongst all the digital copies, the most common type is PDF. Due to its strict security standards, it becomes difficult for companies to procure selected data from the documents.
Companies find it difficult to search within documents, analyze trends based on the data saved within the PDF documents, and bulk paper processing. .NET experts at DEV IT have come with efficient solutions to all your above problems. Read this article to know more.
How to extract several tables of content from PDF?
To extract tabular data from PDFs, you may leverage the following libraries:
- ITextsharp: Open-source library available to extract text and font style of the content.
- HtmlAgilityPack: Open-source library available to process HTML nodes/Tags.
Steps to extract TOC:
- Extract HTML from the PDF using ITextSharp:
- Use TextWithFontExtractionStategy class to get the font style and size information of the text from the PDF. TextWithFontExtractionStategy Class is given as a reference at the end of the blog.
- Prepare custom HTML for PDF and add page number for each page.
- Method: Refer GetPdfHTMLWithPageNo(string pdfPath)
- The above method will return the HTML from the PDF with font style and size.
2. Extract all the headers highlighted in bold using HtmlAgilityPack
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
- Use the HtmlAgilityPack and load your HTML using the class mentioned below.
- Loop through each page and node to get the content highlighted in bold. Refer GetPageWiseHaderList(Stream HTMLPath)
- Or to extract the content that has a greater font size compared to regular content you may follow the above the step.
Using the above steps, you can get the header contents from the PDF. You can also auto-tag, bookmark important keywords or content using the same TOC.
Extracting data from PDFs is easy yet technical. If you are someone who requires external help to fetch the necessary data from your digital documents, get in touch with a DEV IT expert here.
Code Snippet:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 |
public StringBuilder GetPdfHTMLWithPageNo(string pdfPath) { try { string fileName = pdfPath; if (!File.Exists(fileName)) throw new FileNotFoundException("fileName not found... Filename ==> " + fileName); using (PdfReader PdfReader = new PdfReader(fileName)) { PdfReader.unethicalreading = false; string pageWiseSearchableFlag = string.Empty; bool IsOpenedWithFullPermissions = PdfReader.IsOpenedWithFullPermissions; StringBuilder PdfText = new StringBuilder(); PdfText.Append(@"<HTML>" + Environment.NewLine); for (int page = 1; page <= PdfReader.NumberOfPages; page++) { string text = string.Empty; string outputText = string.Empty; try { TextWithFontExtractionStategy S = new TextWithFontExtractionStategy(); text = iTextSharp.text.pdf.parser.PdfTextExtractor.GetTextFromPage(PdfReader, page, S); } catch (Exception ex) { } if (!string.IsNullOrWhiteSpace(text)) { PdfText.Append(@"<div class='PageStart' PageNo=" + page.ToString() + ">" + Environment.NewLine); PdfText.Append(Encoding.UTF8.GetString(ASCIIEncoding.Convert(Encoding.Default, Encoding.UTF8, Encoding.Default.GetBytes(text)))); PdfText.Append(@"</div>" + Environment.NewLine); } else { } } PdfText.Append(@"</HTML>" + Environment.NewLine); return PdfText; } } catch (Exception ex) { throw ex; } } public class TextWithFontExtractionStategy : TextSharp.text.pdf.parser.ITextExtractionStrategy { //HTML buffer private StringBuilder result = new StringBuilder(); //Store last used properties private Vector lastBaseLine; private string lastFont; private float lastFontSize; private enum TextRenderMode { FillText = 0, StrokeText = 1, FillThenStrokeText = 2, Invisible = 3, FillTextAndAddToPathForClipping = 4, StrokeTextAndAddToPathForClipping = 5, FillThenStrokeTextAndAddToPathForClipping = 6, AddTextToPaddForClipping = 7 } public void RenderText(iTextSharp.text.pdf.parser.TextRenderInfo renderInfo) { string curFont = renderInfo.GetFont().PostscriptFontName; var baseColor = renderInfo.GetFillColor(); //Check if faux bold is used if ((renderInfo.GetTextRenderMode() == (int)TextRenderMode.FillThenStrokeText)) { curFont += "-Bold"; } //This code assumes that if the baseline changes then we're on a newline Vector curBaseline = renderInfo.GetBaseline().GetStartPoint(); Vector topRight = renderInfo.GetAscentLine().GetEndPoint(); iTextSharp.text.Rectangle rect = new iTextSharp.text.Rectangle(curBaseline[Vector.I1], curBaseline[Vector.I2], topRight[Vector.I1], topRight[Vector.I2]); Single curFontSize = rect.Height; //See if something has changed, either the baseline, the font or the font size if ((this.lastBaseLine == null) || (curBaseline[Vector.I2] != lastBaseLine[Vector.I2]) || (curFontSize != lastFontSize) || (curFont != lastFont)) { //if we've put down at least one span tag close it if ((this.lastBaseLine != null)) { this.result.AppendLine("</span>"); } //If the baseline has changed then insert a line break if ((this.lastBaseLine != null) && curBaseline[Vector.I2] != lastBaseLine[Vector.I2]) { this.result.AppendLine("<br />"); } //Create an HTML tag with appropriate styles this.result.AppendFormat("<span style=\"font-family:{0};font-size:{1}\">", curFont, curFontSize); } //Append the current text this.result.Append(renderInfo.GetText()); //Set currently used properties this.lastBaseLine = curBaseline; this.lastFontSize = curFontSize; this.lastFont = curFont; } public string GetResultantText() { //If we wrote anything, then we'll always have a missing closing tag, so close it here if (result.Length > 0) { result.Append("</span>"); } return result.ToString(); } //Not needed public void BeginTextBlock() { } public void EndTextBlock() { } public void RenderImage(ImageRenderInfo renderInfo) { } } public List<Header> GetPageWiseHaderList(Stream HTMLPath) { List<Header> lstHeaders = new List<Header>(); HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument(); doc.Load(HTMLPath); var PageNodes = doc.DocumentNode.SelectNodes("//*[@*[contains(., 'PageStart')]]"); List<string> lstBoldStyles = this.GetBoldStyleDictionary(); foreach (var page in PageNodes) { int pageNumber = Convert.ToInt32(page.Attributes.FirstOrDefault(x => x.Name == "pageno").Value); foreach (var node in page.SelectNodes(".//*[@*[contains(., 'font-family')]]")) { HtmlAgilityPack.HtmlAttributeCollection htmlAttributes = node.Attributes; var classAttribute = htmlAttributes.FirstOrDefault(x => x.Name == "style"); foreach (string val in lstBoldStyles) { if (classAttribute != null && classAttribute.Value.Contains(val)) { lstHeaders.Add(new Header { PageNo = pageNumber, HeaderValue = node.InnerText }); break; } else if (classAttribute != null && Convert.ToInt32(Convert.ToDouble(classAttribute.Value.Split(':')[2])) >= 10) { lstHeaders.Add(new Header { PageNo = pageNumber, HeaderValue = node.InnerText }); break; } } } } return lstHeaders; } public class Header { public int PageNo { get; set; } public string HeaderValue { get; set; } } |
The following two tabs change content below.
Yatin Parmar
Team Lead at Dev Information Technology Ltd.
Yatin Parmar works as a .NET Team Lead at DEV IT. With several years of experience in the industry, Yatin likes project management. He likes to manage people, priorities and risks, and is truly dedicated to be at the top of his game always. In his leisure time, Yatin is a cricket enthusiast. Nonetheless, he is a people’s person too.
Latest posts by Yatin Parmar (see all)
- Simple ways to extract Tabular Content from PDFs - February 24, 2021
- Basics of Elastic Search with KIBANA - August 20, 2019
- Isolation levels in SQL Server - June 28, 2019