Implement Polygon Boolean Operations (Union, Intersection, Difference) by tigercosmos · Pull Request #693 · solvcon/modmesh

tigercosmos · 2026-03-14T05:52:20Z

Implement Polygon Boolean Operations (Union, Intersection, Difference)

Summary

Implement polygon boolean operations using trapezoidal decomposition + Y-band sweep with X-interval merge. This approach is based on the algorithm described in:

"A New Method of Applying Polygon Boolean Operations Based on Trapezoidal Decomposition", GIScience & Remote Sensing, Vol 47, No 4 (2010)

The three operations — boolean_union, boolean_intersection, and boolean_difference — are implemented as a single unified algorithm parameterized by a boolean predicate.

Visualization

The visualization code is not included in this PR and is provided only for reference.

Order (left to right, top to down): original outlined polygonss, original colored polygons, the union reseult, the intersection result, the difference result.

Algorithm web explaination

Please use browser to read the web page:
https://gist.github.com/tigercosmos/4a358accfae620ada14a5ffa0776c668

Algorithm Overview

The algorithm has two phases:

Phase	Description
1. Decomposition	Each polygon is independently decomposed into horizontal-band trapezoids by the existing `TrapezoidalDecomposer`
2. Boolean Merge	The two trapezoid sets are combined using a Y-band sweep with an X-interval sweep inside each band

Phase 2 in Detail

Step 1: Collect critical Y-values
         ├── From trapezoid boundaries (y_bottom, y_top of each trapezoid)
         └── From edge-edge crossings between polygon1 and polygon2 trapezoids

Step 2: For each Y-band [y_low, y_high]:
         ├── Gather X-interval events from spanning trapezoids
         ├── Sort events by X-position
         └── Sweep left-to-right, applying boolean predicate

Step 3: Emit result trapezoids as CCW polygons

Boolean Predicates

Operation	Predicate	Meaning
Union	`count_p1 > 0 OR count_p2 > 0`	Inside either polygon
Intersection	`count_p1 > 0 AND count_p2 > 0`	Inside both polygons
Difference	`count_p1 > 0 AND count_p2 == 0`	Inside P1 but not P2

Worked Example

Two overlapping squares: P1 = (0,0)-(2,2), P2 = (1,1)-(3,3).

   y=3 - - - - - ┬────────────────┐- - - - - - - - - -
                 │          P2    │        band [2,3]
   y=2 ┬────────────────┐- - - - -│- - - - - - - - - -
       │   P1    │//////│         │        band [1,2]
   y=1 │- - - - -└──────┴─────────┘- - - - - - - - - -
       │                │                  band [0,1]
   y=0 └────────────────┘- - - - - - - - - - - - - - -
       x=0      x=1    x=2       x=3

Y-values: {0, 1, 2, 3} → 3 bands.

Band [1, 2] event sweep:

Events sorted by X:  P1-start@0  P2-start@1  P1-end@2  P2-end@3
                         │            │           │          │
count_p1:                1            1           0          0
count_p2:                0            1           1          0

X	Event	count_p1	count_p2	Intersection	Union	Difference
0	P1 start	1	0	-	record left=0	record left=0
1	P2 start	1	1	record left=1	(still in)	emit [0,1]
2	P1 end	0	1	emit [1,2]	(still in)	-
3	P2 end	0	0	-	emit [0,3]	-

Handling Slanted Edges

Each event stores two X-values — one at band bottom, one at band top — because polygon edges can be slanted. The emitted result is a general trapezoid with four corners:

        left_x_top ─────────── right_x_top      ← y_high
       /                              \
      /    result trapezoid            \
     /                                  \
    left_x_bottom ─────────── right_x_bottom    ← y_low

The bottom and top edges may have different widths, naturally handling triangles and trapezoids.

Edge-Crossing Y-Split

When trapezoid edges from different polygons cross within a Y-band, the crossing Y-value is detected and inserted into the critical Y-values set. This subdivides the band so that no two cross-polygon edges swap order within a single sub-band, ensuring correctness.

Before split:              After split:
┌──────────────┐           ┌──────────────┐
│  ╲   ╱       │           │   ╲  ╱       │ band [y_cross, y_top]
│    ╳         │    →      ├────╳─────────┤ ← y_cross inserted
│   ╱ ╲        │           │   ╱ ╲        │ band [y_bottom, y_cross]
│  ╱   ╲       │           │  ╱   ╲       │
└──────────────┘           └──────────────┘

Key Design Decisions

Reuses existing TrapezoidalDecomposer — no new decomposition code needed
Unified algorithm — all three operations share one function (compute_boolean_with_decomposition) parameterized by a BoolOp enum
No custom structs for intermediate data — works directly with TrapezoidPad indices and a source_of() lambda
Output as trapezoids — result polygons are emitted as CCW quadrilaterals, matching the existing trapezoidal representation
Relative epsilon — all floating-point comparisons use relative thresholds scaled by coordinate magnitude
Degenerate filtering — zero-area trapezoids from touching edges are skipped
Near-duplicate Y merge — Y-values within relative epsilon are merged to avoid degenerate bands

Files Changed

File	Changes
`cpp/modmesh/universe/polygon.hpp`	Added `detail::BoolOp` enum, `detail::compute_boolean_with_decomposition()` function, `decomposed_trapezoids()` accessor, implemented `AreaBooleanUnion/Intersection/Difference::compute()`
`tests/test_polygon3d.py`	13 boolean operation tests covering: overlapping squares, non-overlapping, triangles, containment, identical polygons, shared edges, commutativity, partial vertical overlap, concave polygons, CCW winding verification

Test Plan

AI Usage

Used Claude Code with Opus 4.6 to implement the draft. Used Codex with 5.3-codex high effort for review. Fine-tuned the implementation manually with both Claude Code and Codex support. Used AI to generate all documents and the PR description.

tigercosmos · 2026-03-14T05:58:42Z

docs/polygon_boolean_algorithm.html

@@ -0,0 +1,951 @@
+<!DOCTYPE html>


Only for PR review purpose. Please use a broswer to open it. Will delete this file once the PR is approved.

Please move the extended document in a gist. Make sure this file is not in your commit history.

tigercosmos · 2026-03-14T05:59:58Z

cpp/modmesh/universe/polygon.hpp

+ * 5. Emitting result trapezoids as polygons
+ */
+template <typename T>
+std::shared_ptr<PolygonPad<T>> compute_boolean_with_decomposition(


Key function. Please check the PR description or ‎docs/polygon_boolean_algorithm.html‎ for the details.

This function can be made to a helper class in the future, but the length is OK for now.

tigercosmos · 2026-03-14T06:00:40Z

tests/test_polygon3d.py


-    # TODO: Verify and implement boolean operations
-    # once the C++ side is complete.
+    def _compute_total_area(self, result_pad):


Addressed previous TODOs.

tigercosmos · 2026-03-14T06:12:06Z

@yungyuc Could you please take a look at this PR? The C++ implementation is about 400 LOC, but it mainly contains a single critical function and quite a few comment lines. The pytest code is around 300 LOC.

My preference would be to keep everything as is. However, if we need to strictly enforce the 500 LOC limit, I can help trim some comments and test cases.

yungyuc

The tests look exciting! Points to address:

Move the extended document (‎docs/polygon_boolean_algorithm.html) in a gist.
Add all class end marker.
Do not use auto when the return type is not written in the RHS.
Use an end marker for long functions.

yungyuc · 2026-03-14T06:35:17Z

docs/polygon_boolean_algorithm.html

@@ -0,0 +1,951 @@
+<!DOCTYPE html>


Please move the extended document in a gist. Make sure this file is not in your commit history.

yungyuc · 2026-03-14T06:36:53Z

cpp/modmesh/universe/polygon.hpp

+ *  Critical Y-values: {0, 1, 2, 3}
+ *  Three y-bands: [0,1], [1,2], [2,3]
+ *
+ *   y=3 - - - - - +----------------+- - - - - - - - - -


Did you use a tool to generate the AA? If you did, please leave a comment stating what and how as a reference that may help other contributors.

I asked AI to leave explanation comments for the code. AI generated the AA directly. However, most of the time I need to manually adjust the chart for annotation or alignment.

Hand-creation of AA usually took most of the efforts. I guess the manual adjustment did not take too much time.

yungyuc · 2026-03-14T07:12:10Z

cpp/modmesh/universe/polygon.hpp

+    Union,
+    Intersection,
+    Difference
+};


Add class end marker.

yungyuc · 2026-03-14T07:13:11Z

cpp/modmesh/universe/polygon.hpp

+namespace detail
+{
+
+enum class BoolOp


I prefer BooleanOperation instead of BoolOp, but this is a must-do. Up to the author.

yungyuc · 2026-03-14T07:17:32Z

cpp/modmesh/universe/polygon.hpp

+    }
+
+    // Step 1: Decompose both polygons using the pad's TrapezoidalDecomposer
+    auto [begin1, end1] = pad->decompose_to_trapezoid(polygon_id1);


The return type is not written in the RHS. Do not use auto here.

Use std::tie instead here.

yungyuc · 2026-03-14T07:22:11Z

cpp/modmesh/universe/polygon.hpp

+    // The source polygon is determined by the index range:
+    //   [begin1, end1) -> polygon1 (source 0)
+    //   [begin2, end2) -> polygon2 (source 1)
+    auto source_of = [begin1, end1, begin2, end2](size_t idx) -> int


Many if not all of the lambdas should be made as a member function when this helper function is refactored as a helper class for better testability and performance.

yungyuc · 2026-03-14T07:23:10Z

cpp/modmesh/universe/polygon.hpp

+    };
+
+    // Collect all unique Y values from trapezoid boundaries
+    std::set<value_type> y_set;


Watch for potential performance hotspot with STL containers in the future.

yungyuc · 2026-03-14T07:26:25Z

cpp/modmesh/universe/polygon.hpp

+    }
+
+    return result;
+}


Use an end marker for long functions.

yungyuc · 2026-03-14T07:26:59Z

tests/test_polygon3d.py


-    # TODO: Verify and implement boolean operations
-    # once the C++ side is complete.
+    def _compute_total_area(self, result_pad):


yungyuc · 2026-03-14T07:28:22Z

tests/test_polygon3d.py

+        result_diff = pad.boolean_difference(polygon1, polygon2)
+        self.assertEqual(result_diff.num_polygons, 0)
+
+    def test_boolean_shared_edge(self):


It's good to test for edge touch. Point touch should be tested too, but it can be supplemented in the future.

yungyuc · 2026-03-14T07:29:18Z

@c1ydehhx please help review if you have time.

tigercosmos · 2026-03-14T10:44:37Z

cpp/modmesh/universe/polygon.hpp

+
+    // Step 1: Decompose both polygons using the pad's TrapezoidalDecomposer
+    size_t begin1, end1, begin2, end2;
+    std::tie(begin1, end1) = pad->decompose_to_trapezoid(polygon_id1);


Removed auto.

With the type the lines look clearer. It's a pity the types cannot be written in the same line of the function call.

tigercosmos · 2026-03-14T10:44:45Z

cpp/modmesh/universe/polygon.hpp

+    size_t begin1, end1, begin2, end2;
+    std::tie(begin1, end1) = pad->decompose_to_trapezoid(polygon_id1);
+    std::tie(begin2, end2) = pad->decompose_to_trapezoid(polygon_id2);
+    std::shared_ptr<TrapezoidPad<T>> trap_pad = pad->decomposed_trapezoids();


Removed auto.

tigercosmos · 2026-03-14T10:44:59Z

cpp/modmesh/universe/polygon.hpp

+    }
+
+    return result;
+} /* end function compute_boolean_with_decomposition */


tigercosmos · 2026-03-14T10:45:12Z

cpp/modmesh/universe/polygon.hpp

+        value_type x_lo, x_hi; // X at y_low and y_high
+        int source;
+        bool is_start; // true = entering the polygon interval, false = leaving
+    }; /* end struct Event */


tigercosmos · 2026-03-14T10:46:05Z

@yungyuc The issues are addressed. Please take a look. Thank you.

Move the extended document (‎docs/polygon_boolean_algorithm.html) in a gist. -> Put the gist in the PR description.
Add all class end marker. -> Done
Do not use auto when the return type is not written in the RHS. -> Done
Use an end marker for long functions. -> Done

yungyuc

Additional/remaining points to address:

Leave a code comment to reminder to profile for the closure find_crossing in the nested loop. It looks like a performance hotspot.
Should be end enum class BooleanOperation, not end enum BooleanOperation.

yungyuc · 2026-03-14T12:55:02Z

cpp/modmesh/universe/polygon.hpp

+ *  Critical Y-values: {0, 1, 2, 3}
+ *  Three y-bands: [0,1], [1,2], [2,3]
+ *
+ *   y=3 - - - - - +----------------+- - - - - - - - - -


Hand-creation of AA usually took most of the efforts. I guess the manual adjustment did not take too much time.

yungyuc · 2026-03-14T12:57:19Z

cpp/modmesh/universe/polygon.hpp

+
+    // Step 1: Decompose both polygons using the pad's TrapezoidalDecomposer
+    size_t begin1, end1, begin2, end2;
+    std::tie(begin1, end1) = pad->decompose_to_trapezoid(polygon_id1);


With the type the lines look clearer. It's a pity the types cannot be written in the same line of the function call.

yungyuc · 2026-03-14T13:00:05Z

cpp/modmesh/universe/polygon.hpp

+            // Find the Y-value where two edges cross within the overlap range.
+            // Each edge is defined by its X-values at y_lo and y_hi.
+            // clang-format off
+            auto find_crossing = [&](value_type edge_a_x_at_bottom, value_type edge_a_x_at_top,


Leave a comment to remind us to profile later and use the timing results to refactor. An issue should be created to track it after the PR is merged.

I am not totally sure about the performance of a closure in the nested loop. My gut feeling is that this is not gonna be fast. It may be hard to profile too.

For the sake of clarity, the nested loop may use a distinct helper class. But it is for future work.

I am not seeing the comment (code remark) to remind to profile in the future. Please point it out using inline review annotation.

The beginning of the lambda is the primary location to leave a TODO comment for potential performance hotspot.

yungyuc · 2026-03-14T13:03:25Z

cpp/modmesh/universe/polygon.hpp

+    Union,
+    Intersection,
+    Difference
+}; /* end enum BooleanOperation */


Should be end enum class BooleanOperation, not end enum BooleanOperation.

Use exact declaration will help grepping.

tigercosmos · 2026-03-14T14:26:10Z

@yungyuc should be good now. :D

yungyuc

A previous point was not addressed. Please clarify.

yungyuc · 2026-03-15T02:40:13Z

cpp/modmesh/universe/polygon.hpp

+            // Find the Y-value where two edges cross within the overlap range.
+            // Each edge is defined by its X-values at y_lo and y_hi.
+            // clang-format off
+            auto find_crossing = [&](value_type edge_a_x_at_bottom, value_type edge_a_x_at_top,


I am not seeing the comment (code remark) to remind to profile in the future. Please point it out using inline review annotation.

tigercosmos · 2026-03-15T06:57:47Z

cpp/modmesh/universe/polygon.hpp

+        return bottom_point.x() + t * (top_point.x() - bottom_point.x());
+    };
+
+    // TODO: potential performance issue: std container


tigercosmos · 2026-03-15T06:59:08Z

cpp/modmesh/universe/polygon.hpp

+            value_type trap_j_right_at_bottom = lerp_x(trap_j_right_bottom, trap_j_right_top, y_lo);
+            value_type trap_j_right_at_top = lerp_x(trap_j_right_bottom, trap_j_right_top, y_hi);
+
+            // TODO: potential performance issue: closure in the nested loop


This is not the best place for the TODO comment, but it makes sense to have one note here too. Don't delete it.

tigercosmos · 2026-03-15T07:00:02Z

@yungyuc I have specified where the TODOs are. Please check the comments above. Thanks!

yungyuc

@yungyuc I have specified where the TODOs are. Please check the comments above. Thanks!

Thank you, now I see them, but one is not at a proper location. If I have time I will modify it myself, but you can do it too. Let's see who's gonna be faster.

yungyuc · 2026-03-15T10:10:12Z

cpp/modmesh/universe/polygon.hpp

+            value_type trap_j_right_at_bottom = lerp_x(trap_j_right_bottom, trap_j_right_top, y_lo);
+            value_type trap_j_right_at_top = lerp_x(trap_j_right_bottom, trap_j_right_top, y_hi);
+
+            // TODO: potential performance issue: closure in the nested loop


This is not the best place for the TODO comment, but it makes sense to have one note here too. Don't delete it.

yungyuc · 2026-03-15T10:11:09Z

cpp/modmesh/universe/polygon.hpp

+            // Find the Y-value where two edges cross within the overlap range.
+            // Each edge is defined by its X-values at y_lo and y_hi.
+            // clang-format off
+            auto find_crossing = [&](value_type edge_a_x_at_bottom, value_type edge_a_x_at_top,


The beginning of the lambda is the primary location to leave a TODO comment for potential performance hotspot.

Replace the three TODO stubs (union, intersection, difference) with a working implementation that decomposes both input polygons into trapezoids, collects critical Y-values including cross-polygon edge crossings, then sweeps X-intervals per band to emit result trapezoids. Add comprehensive tests covering overlapping squares, containment, identical polygons, shared edges, commutativity, concave polygons, and CCW winding order verification.

tigercosmos · 2026-03-15T11:36:35Z

@yungyuc I am faster. 🥇

yungyuc

@yungyuc I am faster. 🥇

Glad to lose. Will merge once CI finishes.

yungyuc · 2026-03-15T12:14:01Z

@tigercosmos Issue #694 is created for you.

tigercosmos commented Mar 14, 2026

View reviewed changes

yungyuc assigned tigercosmos Mar 14, 2026

yungyuc added the geometry Geometry entities and manipulation label Mar 14, 2026

yungyuc requested changes Mar 14, 2026

View reviewed changes

tigercosmos force-pushed the polygon_3011 branch from d2ea798 to ca52d13 Compare March 14, 2026 10:42

tigercosmos commented Mar 14, 2026

View reviewed changes

cpp/modmesh/universe/polygon.hpp

}

return result;

} /* end function compute_boolean_with_decomposition */

Copy link

Collaborator Author

tigercosmos Mar 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mark.

tigercosmos commented Mar 14, 2026

View reviewed changes

yungyuc requested changes Mar 14, 2026

View reviewed changes

tigercosmos force-pushed the polygon_3011 branch from ca52d13 to b46771d Compare March 14, 2026 14:22

yungyuc requested changes Mar 15, 2026

View reviewed changes

tigercosmos commented Mar 15, 2026

View reviewed changes

yungyuc requested changes Mar 15, 2026

View reviewed changes

tigercosmos force-pushed the polygon_3011 branch from b46771d to bc7ca7d Compare March 15, 2026 11:35

yungyuc requested changes Mar 15, 2026

View reviewed changes

yungyuc approved these changes Mar 15, 2026

View reviewed changes

yungyuc merged commit 3ea2820 into solvcon:master Mar 15, 2026
14 checks passed

yungyuc mentioned this pull request Mar 15, 2026

Profile and refactor polygon Boolean #694

Open

tigercosmos deleted the polygon_3011 branch March 15, 2026 12:21

Conversation

tigercosmos commented Mar 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Implement Polygon Boolean Operations (Union, Intersection, Difference)

Summary

Visualization

Algorithm web explaination

Algorithm Overview

Phase 2 in Detail

Boolean Predicates

Worked Example

Handling Slanted Edges

Edge-Crossing Y-Split

Key Design Decisions

Files Changed

Test Plan

AI Usage

Uh oh!

tigercosmos Mar 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tigercosmos commented Mar 14, 2026

Uh oh!

yungyuc left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yungyuc commented Mar 14, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tigercosmos commented Mar 14, 2026

Uh oh!

yungyuc left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

tigercosmos commented Mar 14, 2026 •

edited

Loading

tigercosmos Mar 14, 2026 •

edited

Loading

yungyuc left a comment •

edited

Loading

yungyuc left a comment •

edited

Loading