-
fix: chunking dropping table content (#4352)
发布于
2026-05-13 00:31:01 +00:00 Preserve mixed-content tail text in table HTML
Summary
HtmlTable.from_html_textcompactification cleared every element's
.tail, silently dropping any text between inline children of a<td>.
In practice this deleted real content like": do not..."after
<b>NOTICE</b>, bullet lines following each<br/>, and any inter-tag
prose in rich table cells — visible whenever a table was chunked (any
strategy:basic,by_title,by_page).Pure-whitespace tails are still stripped (those are just pretty-print
indent). Tails carrying content are now kept, with internal whitespace
runs collapsed and a single leading/trailing space preserved so adjacent
text doesn't run together.Changes
unstructured/common/html_table.py: replace the blankete.tail = Nonewith whitespace-aware normalization.test_unstructured/common/test_html_table.py: two regression tests
covering mixed inline markup,<br/>-separated lines, and whitespace
collapsing.- Bump to
0.22.28+ changelog entry.
Note
Low Risk
Low risk: localized change to table HTML minification that preserves
previously-dropped text; main risk is minor whitespace/serialization
differences in downstream table chunking output.Overview
FixesHtmlTable.from_html_text()compactification to stop deleting
meaningful mixed-content tail text (text between inline children like
<b>foo</b> baror between<br/>siblings), while still dropping pure
formatting/indent whitespace and collapsing internal whitespace runs.Adds regression tests for mixed inline markup and
<br/>-separated
cell content, and bumps the library to0.22.28with a changelog entry
describing the fix.Reviewed by Cursor Bugbot for commit
fe27d3582a40cf1d27e8617491d9b1214057605e. Bugbot is set up for automated
code reviews on this repo. Configure
here.下载附件