Render to BIM — Generative Rendering-Based BIM Authoring

Conventional rendering visualizes information already authored in a model. Generative rendering instead proposes furniture and spatial arrangements that may exist only in an image. To support quantity takeoff or circulation review, these objects must be identified, reconstructed as 3D geometry, and placed within the source model coordinates.

First, an unfurnished BIM view, its camera parameters, and a depth map are saved for generative design exploration. Second, objects in the selected rendering are segmented into instances and reconstructed as meshes. Third, predicted depth is calibrated to the source BIM, placement candidates are evaluated for each object, and the result is exported to IFC.

The study compares Hunyuan3D-2.1 using isolated object crops with SAM 3D Objects, which can receive a full scene and an object mask. Recovering unseen geometry from a single view makes occlusion and image truncation particularly challenging. An additional SAM 3D Objects crop-only condition separates model differences from the effect of scene context.

MoGe-2 estimates monocular depth from the accepted rendering. Corresponding pixels on architectural elements such as walls and floors calibrate that prediction to metric depth from the source BIM. Global scale-and-shift and monotonic piecewise-linear mappings are compared using held-out architectural pixels. The saved camera then lifts calibrated depth into visible surface points in BIM coordinates.

Three candidates use complementary evidence: Depth OBB fits a box to visible surface points, reconstruction pose uses the pose returned by the reconstruction model, and FoundationPose renders and compares a mesh against the observation. Candidates are selected by explained depth minus free-space violations. The mesh is placed with its proportions preserved and written as an IfcFurnishingElement, retaining the source view, estimator, and consistency scores.

Draft Table 6 reports median shape F-score@5cm values of 0.77–0.84 for SAM 3D Objects with scene and mask across renderers. In Table 7, Depth OBB achieves [email protected] of 43.4–59.6% with estimated depth and 78.8% when source-model depth is substituted. These measure aligned shape reconstruction and placement in model coordinates respectively, and are not interchangeable accuracy measures. The tables report 99 objects per renderer, while the dataset table still contains unfinished values; a final dataset size is therefore not asserted here.

The framework produces an initial BIM representation for designer review and correction rather than a fully automatic, finished BIM. Evaluation uses ground-truth instance masks and restyled furnished scenes, excluding automatic segmentation errors and freely furnishing empty views. Tilted or wall-mounted objects, occluded geometry, and depth errors in generated imagery remain open challenges.