Use Apache Tika to convert documents to "sections" - apache-tika

Question

Problem
I am looking to convert documents (primarily pdf and html) to something I can reliably search over in a search database (e.g. Elasticsearch).
To best search over long passages of text, it is often recommended to break up longer text into chunks that can be searched over instead. This gives a better chance of providing relevant results.
A common structure for this is to use hierarchical sections. An example based on Stack Overflow's Wikipedia page looks like this:
{
"sections": {
"title": "Stack Overflow",
"sections": [
{
"text": "Stack Overflow is a question and answer website for professional and enthusiast programmers. It is the flagship site of the Stack Exchange Network."
},
{
"title": "History",
"sections": [
{
"text": "The website was created by Jeff Atwood and Joel Spolsky in 2008"
},
{
"title": "Security breach",
"In early May 2019, an update was deployed to Stack Overflow's development version."
}
]
}
]
}
}
What I've learnt
I stumbled upon Apache Tika after seeing it referenced in the documentation of popular cloud based Search engines (e.g. Algolia's).
It seems like Tika is capable of translating pdfs > html, so it seems like I can initially just focus on converting html > "sections".
My rudimentary understanding of Tika is that to convert in this fashion requires some understanding of the structure of the document. However, in my case I don't necessarily have a predictable document structure.
One area I have seen this done well is in "reader view" type plugins (e.g. Mozilla's). They tend to attempt to format html documents in predictable ways using heuristics.
Question
What is the best way to use Tika to convert generic html documents to hierarchical sections (as shown above) similar to how "reader view" works?