Back to projects
3D Reconstruction / VR / Computer Graphics2026Research Project

Immersive Reconstruction with Gaussian Splatting

A modular image-to-VR pipeline for reconstructing, optimizing and deploying high-fidelity real-world environments with 3D Gaussian Splatting.

3DGSCOLMAPLichtFeld StudioUnityOpenXRVR
Preview of Immersive Reconstruction with Gaussian Splatting

Results

Visual metrics

Target frame rate

60 FPS

Minimum target for stable, comfortable immersive interaction

Tested scene budget

~250K splats

Achieved when splats remain evenly distributed in space

Meta guidance

<150K splats

Recommended baseline for stable standalone VR rendering

Render scale

0.7-0.8

Practical quality-performance range used for VR deployment

Technical snapshot

Quick facts

Goal

Production pipeline for immersive real-world capture

Role

3DGS pipeline research and XR integration

Duration

Master GIG research project, 2026

Stack

COLMAP, LichtFeld Studio, Unity, OpenXR

Dataset

Reference datasets and real captured environments

Output

Optimized, interactive VR/MR scenes

Status

Research Project

Case study

Overview

Problem

High-quality 3DGS scenes can contain too many primitives for standalone headsets, lack explicit surfaces for collisions and remain sensitive to incomplete captures, moving objects and reflective materials. A production workflow must therefore balance reconstruction quality, splat density, GPU memory and stable stereo rendering.

Goal

Create a reproducible image-to-XR workflow that preserves visual fidelity while meeting the memory and frame-rate constraints of standalone VR hardware.

Method

Capture sharp, clean and complete image coverage; recover cameras and a sparse point cloud with COLMAP Structure-from-Motion; train, crop and clean the Gaussian representation in LichtFeld Studio; then import it into Unity through UnityGaussianSplatting and gsplatUnity. OpenXR, single-pass rendering, fixed foveated rendering and a reduced render scale are used for VR deployment, while a simplified invisible mesh supplies collision geometry.

Result

The resulting modular pipeline supports real-time immersive visualization at a stable 60 FPS when splat count and spatial density are controlled. Tests reached approximately 250,000 splats, while separating the visual 3DGS representation from a simplified mesh collider enabled practical physical interaction.

Technical foundation

Technology stack

Reconstruction

COLMAPStructure-from-Motion3DGS

Optimization

LichtFeld StudioPostShotLumaAI

XR Runtime

UnityOpenXRURPgsplatUnity

Geometry

CloudCompareBlenderSplat2Mesh

Evolution

Research timeline

  1. 2026

    Tool and dataset evaluation

    Compared reference and captured datasets across COLMAP, LichtFeld Studio, PostShot and LumaAI.

  2. 2026

    Image-to-3DGS pipeline

    Established the reconstruction, training, cleanup and splat-budget optimization stages.

  3. 2026

    XR deployment

    Integrated the optimized scene in Unity and evaluated real-time rendering and interactions in VR.

Comparison

Before / After

Before
Before
After
After

Process

Project pipeline

01

Image acquisition

Capture sharp HD images of a static scene with clean, complete multi-view coverage.

Input

02

Camera reconstruction

Use COLMAP Structure-from-Motion to estimate camera poses and generate the initial point cloud.

SfM

03

3DGS training

Generate the Gaussian scene in LichtFeld Studio and tune quality, iterations and maximum splat count.

Training

04

Cleanup and optimization

Crop the scene, remove noisy splats and control density to fit the target VR performance budget.

Optimization

05

Unity XR integration

Import the scene with 3DGS plugins and configure OpenXR, single-pass rendering, FFR and post-processing.

Runtime

06

Interaction and deployment

Align an invisible simplified mesh collider with the 3DGS scene, then deploy the interactive experience to VR.

Output

Project evidence

Media gallery

image

Reconstructed Room in VR

01
Reconstructed Room in VR

A wide view of the optimized Gaussian Splatting environment running inside the VR application.

image

VR Navigation and Controls

02
VR Navigation and Controls

The runtime view with tracked controllers and interaction rays inside the reconstructed scene.

image

Runtime Interaction Test

03
Runtime Interaction Test

A practical interaction test used to validate navigation, pointing and collision behavior in Unity VR.

image

Reconstruction Detail

04
Reconstruction Detail

A closer view showing how furniture, walls and architectural details are preserved in the immersive reconstruction.

Implementation detail

Technical notes

  • Input quality is decisive: images should be sharp, free of moving subjects and cover the scene from varied viewpoints.
  • The tested Unity setup uses SH Order 0, Opacity Scale 0.7, Splat Scale 1 and sorting every fourth frame.
  • Single-pass stereo rendering is important because multi-pass rendering processes the scene separately for each eye.
  • Reflective surfaces, dynamic subjects and uneven splat density remain major sources of artifacts and performance loss.
  • Because 3DGS has no explicit surface, an aligned simplified mesh is used only for collisions while splats provide the visual representation.
  • Global Volume color adjustments provide non-destructive lighting and color control without modifying the Gaussian data.

Knowledge base

References

paper

3D Gaussian Splatting

Foundational paper behind real-time radiance field rendering.

documentation

COLMAP

Structure-from-Motion software used to estimate cameras and reconstruct the initial point cloud.

github

LichtFeld Studio

Open-source application used to train, optimize, crop and clean Gaussian scenes.

github

UnityGaussianSplatting

Unity implementation used to display and process 3D Gaussian Splatting assets.

References

Resources

Demo