Blazingly Fast HTML Content Extractor

For future use in the slopless search engine project.

goals

• Extract main content from HTML pages with high accuracy
• Normalize encoding to UTF-8 and unescape HTML entities
• Split content into semantically meaningful chunks
• Detect and handle different languages

things to try

• https://docs.rs/html-escape/latest/html_escape/
• https://docs.rs/htmlentity/1.3.2/htmlentity/data/index.html
• https://html.spec.whatwg.org/entities.json