SynDefect3D

Procedurally generated photorealistic 3D synthetic dataset for indoor building defect inspection, built on Infinigen Indoors. This repository extends the Infinigen pipeline with defect-focused rendering and export workflows for RGB, material segmentation, YOLO-style detection labels, and simulation use (e.g. NVIDIA Omniverse / Isaac Sim).

Paper

Add title, venue, and arXiv or DOI link when published.

Citation

If you use this work, please cite our paper (see above) and the underlying Infinigen work. BibTeX for Infinigen Indoors (CVPR 2024):

Add your project’s BibTeX entry here.

@inproceedings{infinigen2024indoors,
    author    = {Raistrick, Alexander and Mei, Lingjie and Kayan, Karhan and Yan, David and Zuo, Yiming and Han, Beining and Wen, Hongyu and Parakh, Meenal and Alexandropoulos, Stamatis and Lipson, Lahav and Ma, Zeyu and Deng, Jia},
    title     = {Infinigen Indoors: Photorealistic Indoor Scenes using Procedural Generation},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2024},
    pages     = {21783-21794}
}

License

This project is licensed under the GNU General Public License v3.0 — see LICENSE. Infinigen components under infinigen/ follow the licenses described in infinigen/LICENSE.

Dataset

Add dataset name, download link, and a short description of splits and annotations when released.

Acknowledgements

This work builds on Infinigen (code, documentation). Please see the Infinigen repository for installation, dependencies, and the full citation list.

Additional open-source assets used in this generator:

• Air Conditioner by Daniyal Malik — Sketchfab
• Air condition Daikin by maxsbond.work — Sketchfab
• Indoor air conditioner unit by Rylae Shylna — Sketchfab
• UK wall plug socket by Geng4d — Sketchfab
• UK Plug Socket by Tenakin — Sketchfab
• Twin Plug Socket by Sousinho — Sketchfab
• Wall Power Outlet - Type I by cdcruz — Sketchfab
• Grohe G-31191001 and Grohe G-32667001 by trendforward — Sketchfab
• Modern Faucet (high poly) by Elasta Kristya — Sketchfab
• Roca Element Bidet Mixer by Toss90 — Sketchfab

---

Usage

Generation is driven by Gin configs. See infinigen/infinigen_examples/configs_indoor/ for available scenes and render settings.

Run the commands below from the infinigen/ directory (or put that directory on PYTHONPATH), after completing Infinigen’s installation.

Defect-focus cameras and YOLO / bbox exports

Defect-focus cameras are controlled by Gin, not by the render loop itself:

• camera.add_defect_focus.enabled (see infinigen_examples/configs_indoor/base_indoors.gin) defaults to False. Then you only have the usual n_camera_rigs from coarse generation, all posed with the normal indoor camera logic.
• Set it to True when running coarse (same Gin mechanism as other overrides), e.g.
  -p camera.add_defect_focus.enabled=True
  to add extra camera rigs—one additional rig per defect—with head-on views. Your render loop should still use tools/count_camera_rigs.py so rig indices stay correct as the rig count grows.

YOLO labels and bbox JSON are not produced inside Blender during --task render. The render writes RGB + material segmentation (MaterialSegmentation/, Materials/).

• Enable the packaged detection + segmentation dataset: run tools/prepare_defect_annotated_dataset.py (§3). That script always emits images/, masks/, bboxes/, and bboxes_yolo/ together (there is no Gin flag to turn YOLO off inside the preparer).
• Skip detection exports: if you only need raw frames and material-index maps, do not run §3. If you ran the preparer but train segmentation only, use masks/ (and images/) and ignore or remove bboxes/ and bboxes_yolo/.

1. Coarse scene generation (example: 10 bedrooms)

for i in $(seq -w 1 10); do
  seed=$((10#$i + 50))
  out_dir="outputs/dataset/bedroom${i}"
  mkdir -p "$out_dir"
  python -m infinigen_examples.generate_indoors \
    --seed "$seed" \
    --task coarse \
    --output_folder "$out_dir" \
    -g bedroom_minimal.gin \
    --overrides camera.spawn_camera_rigs.n_camera_rigs=10
done

• --overrides camera.spawn_camera_rigs.n_camera_rigs=10 sets how many camera rigs are stored in the blend.
• bedroom_minimal.gin (and analogous kitchen_minimal, dining_minimal, bathroom_minimal, livingroom_minimal) reduce clutter while keeping defect-related constraints; edit or combine .gin files as needed.

2. Rendering (defect pipeline)

defect_render.gin enables material-index passes; execute_tasks.resample_idx randomizes the scene per resample. The loop below counts rigs per blend with tools/count_camera_rigs.py so you do not hard-code rig indices.

Set output roots with = (not :). Example:

OUT_ROOT="${OUT_ROOT:-$(pwd)/outputs/dataset}"
OUT_FRAMES_ROOT="${OUT_FRAMES_ROOT:-$(pwd)/outputs/dataset/all_frames}"
mkdir -p "${OUT_FRAMES_ROOT}"

for room in bedroom; do
  for i in $(seq -w 1 10); do
    scene_dir="${OUT_ROOT}/${room}${i}"
    [[ -d "${scene_dir}" ]] || continue
    blend="${scene_dir}/scene.blend"
    [[ -f "${blend}" ]] || continue
    n_rigs="$(blender --background --quiet "${blend}" --python tools/count_camera_rigs.py 2>/dev/null | grep -E '^[0-9]+$' | head -n1)"
    [[ "${n_rigs}" =~ ^[0-9]+$ ]] || { echo "skip ${scene_dir}: could not count camera rigs" >&2; continue; }
    for rig in $(seq 0 $((n_rigs - 1))); do
      for rs in {0..4}; do
        python -m infinigen_examples.generate_indoors \
          --seed 0 \
          --task render \
          --input_folder "${scene_dir}" \
          --output_folder "${OUT_FRAMES_ROOT}/${room}${i}/rig${rig}_rs${rs}" \
          -g infinigen_examples/configs_indoor/defect_render.gin \
          -p render.render_image_func=@defect/render_image \
             "execute_tasks.camera_id=[${rig},0]" \
             "execute_tasks.resample_idx=${rs}"
      done
    done
  done
done

Under each rig*_rs* folder you should see Image/, MaterialSegmentation/, and Materials/ (after the compositor reorganizes outputs). Use the same layout for multiple room types by extending the for room in ... list (e.g. bathroom kitchen).

3. Defect masks, COCO-style bboxes, and YOLO labels

tools/prepare_defect_annotated_dataset.py walks an all_frames-style tree, writes a flat training pack:

• images/, masks/ (semantic defect classes)
• bboxes/ (per-image JSON, pixel boxes; spalling plugs share the spalling class with wall spalling)
• bboxes_yolo/ (YOLO detection lines: cls xc yc w h normalized; cls 0–4)
• class_names.txt, splits/, annotations_coco.json

python tools/prepare_defect_annotated_dataset.py \
  -i /path/to/all_frames \
  -o /path/to/defect_annotated_dataset

After changing class rules or bbox logic, regenerate existing files with --force. The script skips samples when all of images, masks, bboxes, and bboxes_yolo already exist unless --force is set.

One frame only (same logic as the full exporter):

python tools/prepare_defect_single_sample.py \
  -i /path/to/all_frames \
  --sample-id bathroom08_rig18_rs1_rig18_camera_0 \
  -o /tmp/one_sample_export

Quick visual check of boxes on raw frames (uses the same bbox logic as the preparer):

python tools/visualize_defect_yolo_annotations.py \
  -i /path/to/all_frames \
  --sample-id bathroom08_rig18_rs1_rig18_camera_0 \
  -o preview.png \
  --print-yolo

4. Export for Isaac Sim / Omniverse

For training embodied AI in Isaac Sim, export scenes to USD:

python -m infinigen.tools.export \
  --input_folder outputs/indoors/coarse/example \
  --output_folder outputs/my_export \
  -f usdc \
  -r 2048 \
  --omniverse

• -r 2048 sets the resolution for exported textures.

---

Contact

Questions or research discussion: shiven@link.cuhk.edu.hk

If this project is useful to your work, please consider citing our paper and starring the repository.