Project Specification: Geometric Theory Inference Engine

Target Platform: Web / TensorFlow.js Goal: Semantic reconstruction of point clouds via evolutionary constraint discovery.

1. Abstract

Standard mesh reconstruction algorithms (Poisson, Ball-Pivoting) focus on surface topology—connecting dots to form a skin. This project aims for semantic reconstruction. It seeks to discover the underlying “blueprint” or “theory” that generates the point cloud. **Figure 1:** Contrast between standard surface reconstruction (fitting a skin) versus semantic reconstruction (discovering the underlying geometric blueprint).

The system inputs a raw point cloud and outputs a Minimally-Complex Geometric Model. This model consists of a sparse set of Control Points governed by explicit Geometric Constraints (angles, distances, torsion), which reproduce the input cloud through linear combinations.

## 2. The Core Architecture

The model is treated as a structured Autoencoder where the latent space is an explicit geometric graph rather than a black-box vector.

2.1. The Data Flow

**Figure 2:** The system architecture. Unlike traditional autoencoders, the latent bottleneck is an explicit, interpretable geometric graph.

Input ($\mathbf{P}_{obs}$): A point cloud tensor of shape $(N, 3)$.
Latent Variables ($\mathbf{C}$): A set of $M$ Control Points, shape $(M, 3)$, where $M \ll N$.
Projection ($\mathbf{W}$): A weight matrix defining how Control Points combine to form the cloud.
- $\mathbf{P}_{gen} = \mathbf{W} \times \mathbf{C}$
- Note: In a “Rigid Body” interpretation, $\mathbf{W}$ represents the barycentric coordinates or attachment weights of the cloud points relative to the control “bones.”
Constraints ($\mathbf{G}$): A graph where nodes are Control Points and edges are geometric rules.

2.2. The Constraint Layers (The “Theory”)

The “Theory” is defined by the energy potential of the constraints. A valid theory has near-zero energy. The system supports the following differentiable constraint layers: **Figure 3:** Visualizing the library of differentiable geometric constraints used to define the "Theory."

2-Point Distance: $|c_i - c_j| = k$ (Defines rigid struts).
3-Point Angle: $\text{acos}(\frac{\vec{v}{ij} \cdot \vec{v}{kj}}{|\vec{v}{ij}| |\vec{v}{kj}|}) = \theta$ (Defines corners/hinges).
3-Point Line-to-Point: Distance from $c_k$ to line $\overline{c_i c_j}$ is $d$ (often $d=0$ for collinearity).
4-Point Equal-Length: $|c_i - c_j| = |c_k - c_l|$ (Symmetry/Parallelograms).
4-Point Torsion: The angle between the plane $ijk$ and plane $jkl$ is $\phi$.
4-Point Plane-to-Point: Distance from $c_l$ to plane defined by $c_i, c_j, c_k$ is $d$ (Coplanarity).

3. Hybrid Optimization Strategy

The system utilizes a Bilevel Optimization approach to solve the discrete combinatorial problem (Graph Topology) and the continuous parameter problem (Geometry) simultaneously. **Figure 4:** The Bilevel Optimization Strategy. The "Architect" (GA) modifies the graph structure, while the "Engineer" (Solver) optimizes the spatial coordinates for that structure.

### 3.1. Outer Loop: Evolutionary Algorithm (The Architect)

Role: Explores the discrete space of theories.
Operations:
- Add/Remove Control Point
- Add/Remove Constraint (e.g., “Force Point A and B to be 5 units apart”)
- Mutate Constraint Target (e.g., Change required angle from 90° to 45°)
Fitness Function: $ F = \text{Error}(\mathbf{P}{obs}, \mathbf{P}{gen}) + \lambda \cdot \text{Complexity}(\mathbf{G}) $
- Complexity is the count of constraints and control points (Occam’s Razor).
- The GA selects for the simplest graph that can be solved to match the cloud.

3.2. Inner Loop: Differentiable Solver (The Engineer)

Role: For a specific graph provided by the GA, find the optimal coordinates.
Implementation: TensorFlow.js tf.train.adam optimizer.
Loss Function: $ L = \sum (\mathbf{P}_{obs} - \mathbf{W}\mathbf{C})^2 + \alpha \sum \text{ConstraintViolations}(\mathbf{C})^2 $
Dynamics:
- During optimization, constraints are soft (penalty terms).
- A “valid solution” is achieved only if the final ConstraintViolation is effectively zero (Hard Constraint satisfaction).

4. Addressing Curved Surfaces (Research Vector)

The “Rigid Body” model works perfectly for mechanical parts (planes, beams). Analytically curved surfaces (spheres, cylinders) present a challenge for a sparse point-based theory.

Proposed Solution: Higher-Order Constraints Instead of simulating a circle with 100 control points and distance constraints (a polygon), we introduce Virtual Anchor Points. **Figure 5:** Handling curvature. Instead of using many points to approximate a curve, the system evolves a "Virtual Anchor" (center point) and a radius constraint to define the surface mathematically.

Example: To define a sphere, the theory evolves a “Center Point” (which may not exist in the cloud) and a “Radius Constraint.”
Constraint: “All points in Subset $S$ must have distance $R$ from Control Point $C_{center}$.”
This keeps the model minimal while allowing perfect curvature representation.

5. Implementation Plan: TensorFlow.js

Using TF.js allows for a “Live Lab” environment where the user can watch the theory evolve in real-time in the browser.

5.1. Technology Stack

Compute: TensorFlow.js (WebGL backend).
Visualization: Three.js (rendering the Point Cloud $\mathbf{P}$ and the Control Graph $\mathbf{C}$ overlay).
UI: React or Dat.GUI for tweaking mutation rates and $\lambda$ penalties.

5.2. Data Structures (JSON Schema)

A “Theory” can be serialized as:

{
  "controlPoints": 8,
  "constraints": [
    { "type": "distance", "indices": [0, 1], "value": 10.0 },
    { "type": "angle", "indices": [0, 1, 2], "value": 1.5707 },
    { "type": "coplanar", "indices": [0, 1, 2, 3], "value": 0.0 }
  ],
  "weights": "sparse_matrix_indices..."
}

5.3. Development Phases

Phase 1: The Solver (Inner Loop)

Build the TF.js graph.
Manually define a set of control points and constraints (e.g., a Cube).
Run the optimizer to see the points snap from random positions to a perfect Cube.
Visualize with Three.js.

Phase 2: The Projector

Implement the $\mathbf{W}$ matrix optimization.
Given a fixed Cube skeleton, optimize $\mathbf{W}$ to fit a noisy point cloud of a box.

Phase 3: The Evolution (Outer Loop)

Implement the Genetic Algorithm.
Start with a simple 2D rectangle.
Allow the system to discover that 4 points + 4 distance constraints + 1 angle constraint = Rectangle.

Phase 4: Complexity & Curvature

Introduce the complexity penalty to prune useless constraints.
Experiment with “Virtual Anchors” for circular fitting.

6. Summary

This system moves beyond fitting data to understanding data. By forcing the reconstruction through a bottleneck of geometric logic, we generate models that are editable, parametric, and human-readable, turning raw scans into CAD-ready definitions.

Choose Theme