发布

  • fix: chunking dropping table content (#4352)

    frostbyte_neo 发布于 2026-05-13 00:31:01 +00:00

    Preserve mixed-content tail text in table HTML

    Summary

    HtmlTable.from_html_text compactification cleared every element's
    .tail, silently dropping any text between inline children of a <td>.
    In practice this deleted real content like ": do not..." after
    <b>NOTICE</b>, bullet lines following each <br/>, and any inter-tag
    prose in rich table cells — visible whenever a table was chunked (any
    strategy: basic, by_title, by_page).

    Pure-whitespace tails are still stripped (those are just pretty-print
    indent). Tails carrying content are now kept, with internal whitespace
    runs collapsed and a single leading/trailing space preserved so adjacent
    text doesn't run together.

    Changes

    • unstructured/common/html_table.py: replace the blanket e.tail = None with whitespace-aware normalization.
    • test_unstructured/common/test_html_table.py: two regression tests
      covering mixed inline markup, <br/>-separated lines, and whitespace
      collapsing.
    • Bump to 0.22.28 + changelog entry.

    Note

    Low Risk
    Low risk: localized change to table HTML minification that preserves
    previously-dropped text; main risk is minor whitespace/serialization
    differences in downstream table chunking output.

    Overview
    Fixes HtmlTable.from_html_text() compactification to stop deleting
    meaningful mixed-content tail text
    (text between inline children like
    <b>foo</b> bar or between <br/> siblings), while still dropping pure
    formatting/indent whitespace and collapsing internal whitespace runs.

    Adds regression tests for mixed inline markup and <br/>-separated
    cell content, and bumps the library to 0.22.28 with a changelog entry
    describing the fix.

    Reviewed by Cursor Bugbot for commit
    fe27d3582a40cf1d27e8617491d9b1214057605e. Bugbot is set up for automated
    code reviews on this repo. Configure
    here.

    下载附件