Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1365 commits
Select commit Hold shift + click to select a range
8dfe0eb
Added thread locks and RobotMode type for b1
spomichter Sep 18, 2025
25f988c
Merge pull request #626 - Unitree B1 ROS<-->Dimensional Navigation
spomichter Sep 18, 2025
bc3e1d7
Added build directory to gitignore
yashas-salankimatt Sep 19, 2025
12dc24b
Merge pull request #628 from dimensionalOS/gitignore_build_fix
spomichter Sep 19, 2025
725be30
pr review issues
leshy Sep 19, 2025
0513439
Merge pull request #583 - 2D/3D detection module + Pointcloud localiz…
spomichter Sep 22, 2025
94b820e
Camera calibration loading (#629)
leshy Sep 22, 2025
f8cbf0a
port navigation skills to agents2
paul-nechifor Sep 18, 2025
81aaa99
add open street maps skill
paul-nechifor Sep 21, 2025
e025466
add gps location to foxglove
paul-nechifor Sep 23, 2025
9135993
add google maps skills
paul-nechifor Sep 24, 2025
c3f8d2b
travel-points
paul-nechifor Sep 25, 2025
b11c684
my files
paul-nechifor Sep 26, 2025
582f411
revert unitree go2 gps changes
paul-nechifor Sep 26, 2025
1621482
fix api key
paul-nechifor Sep 26, 2025
abe2049
Merge pull request #630 from dimensionalOS/agent2-nav-skills
paul-nechifor Sep 26, 2025
28df1d7
fix all thread leaks
paul-nechifor Sep 30, 2025
089e6f6
fail tests which leak
paul-nechifor Oct 1, 2025
cdebf18
ignore lcm, heavy, and ros
paul-nechifor Oct 1, 2025
7595050
shared mem again
paul-nechifor Oct 1, 2025
3008b94
clean up
paul-nechifor Oct 1, 2025
cd8715b
move constants
paul-nechifor Oct 1, 2025
b601b79
add utilization
paul-nechifor Aug 31, 2025
d1d12db
prettier box
paul-nechifor Oct 1, 2025
cb6d1e5
add monitoring back
paul-nechifor Oct 1, 2025
6c07c4e
Merge pull request #650 - Shared Memory IPC integrated to Unitree Go2…
spomichter Oct 2, 2025
958d772
Merge branch 'dev' into fix-leaks
paul-nechifor Oct 2, 2025
f69a630
review comment
paul-nechifor Oct 2, 2025
c175297
Merge pull request #649 from dimensionalOS/fix-leaks
paul-nechifor Oct 2, 2025
94b87c8
fix leak
paul-nechifor Oct 3, 2025
eec11e4
Merge pull request #652 from dimensionalOS/fix-leak
paul-nechifor Oct 3, 2025
7da855f
suppress echos with counter
paul-nechifor Oct 3, 2025
6bdcc54
use uuid4
paul-nechifor Oct 4, 2025
d9a0440
Merge pull request #653 from dimensionalOS/suppress-echos-with-counter
spomichter Oct 4, 2025
8f69467
Removing websocket vis causing crazy lag
spomichter Oct 4, 2025
991dc4d
Merge pull request #656 from dimensionalOS/remove-websocket-vis
spomichter Oct 4, 2025
f34eab9
Fixed navigate to object reference in navigation.py to use module not…
spomichter Oct 3, 2025
1831d52
Removed non-module navigate to object impl, created new Bbox navigati…
spomichter Oct 3, 2025
ea30202
Added dotenv to envrc
spomichter Oct 4, 2025
d60b468
Merge pull request #655 from dimensionalOS/suppress-with-uuid
paul-nechifor Oct 4, 2025
a9078cc
Fix detection2d bug in decoding of message when subscribed
spomichter Oct 4, 2025
5b31e95
Fully working minimal 2d object tracker
spomichter Oct 4, 2025
ab3b1e7
WIP: tracking stream still not working in foxglove. Changed tracked o…
spomichter Oct 7, 2025
71b648b
Fully removed depth module from unitree go2
spomichter Oct 7, 2025
4429190
Added stuck detection for 2d object tracker, fix frame copy bug over …
spomichter Oct 7, 2025
df14537
Added back multi step navigatewithtext
spomichter Oct 7, 2025
c11f312
Implementing feedback / changes to Image and CudaImage
mdaiter Oct 7, 2025
3919c88
Add helpful comment for YUV conversion
mdaiter Oct 7, 2025
83a37e2
CI code cleanup
mdaiter Oct 7, 2025
e36c01e
Merged dev
mdaiter Oct 7, 2025
79b610f
update tests to use actual image
mdaiter Oct 7, 2025
86cdde9
Fix navigation tests
spomichter Oct 7, 2025
499c151
Merge pull request #654 - Navigate to Object and object trackers re-i…
spomichter Oct 7, 2025
781fe65
Ros bridge test fix
alexlin2 Oct 8, 2025
c0d886b
video g1 spatial mem + detection - tomerge (#651)
leshy Oct 8, 2025
e08fb2b
Update README.md
spomichter Oct 8, 2025
e999a38
Merge pull request #664 from dimensionalOS/update-readme-temp
spomichter Oct 8, 2025
c1385dc
Update and speed up CUDA PnP
mdaiter Oct 8, 2025
71c19cb
CI code cleanup
mdaiter Oct 8, 2025
5458064
Adding averages to tests
mdaiter Oct 8, 2025
8fa3e80
CI code cleanup
mdaiter Oct 8, 2025
0685a85
merge dev
mdaiter Oct 8, 2025
d0dbcbb
CI code cleanup
mdaiter Oct 8, 2025
702c8e8
Push unitree_go2 to the latest
mdaiter Oct 8, 2025
0dd8344
CI code cleanup
mdaiter Oct 8, 2025
42f410f
Merge pull request #612 from dimensionalOS/matt-image-upgrades
mdaiter Oct 8, 2025
fcd8f84
Revert "Image upgrades! Impls for CUDA + numpy, along with an abstrac…
leshy Oct 8, 2025
3efa77d
Merge pull request #665 from dimensionalOS/revert-612-matt-image-upgr…
spomichter Oct 8, 2025
cdfaaed
Reapply "Image upgrades! Impls for CUDA + numpy, along with an abstra…
paul-nechifor Oct 8, 2025
2924852
fix
paul-nechifor Oct 8, 2025
2a017dc
CI code cleanup
paul-nechifor Oct 8, 2025
15d1991
add crop back
paul-nechifor Oct 8, 2025
e886217
fix encoding
paul-nechifor Oct 9, 2025
7bc6f9e
Fix detection 3d test for floating point non determinism
spomichter Oct 9, 2025
3ba5695
Fix CI lack of timer precision for test reactive
spomichter Oct 9, 2025
66440ab
Detection second pass (#662)
leshy Oct 10, 2025
e73e82e
Added cleaner Dask exit handling for cuda and shared memory object cl…
spomichter Oct 10, 2025
f91d6e2
Fix lcm_encode in new Image and added BGR default
spomichter Oct 10, 2025
820fa59
Moved cudaimage unit tests to correct location
spomichter Oct 10, 2025
348d49c
Fix very broken to_cpu method in Image
spomichter Oct 10, 2025
80d6f4f
Major rewrite of image backend tests, seperated cuda/cpu for testing …
spomichter Oct 10, 2025
68a4985
Minor changes
spomichter Oct 10, 2025
084edff
Fixes
spomichter Oct 10, 2025
90fcc58
Simplify to_cpu Image operation
spomichter Oct 10, 2025
fe021b2
Merge branch 'temp-dev' into fix-iage
spomichter Oct 10, 2025
ab83dad
CI code cleanup
spomichter Oct 10, 2025
7a136d2
Merge pull request #671 from dimensionalOS/fix-iage
spomichter Oct 10, 2025
f095163
alibaba env var fix
leshy Oct 10, 2025
ea41664
retry decorator, better vl model query system, json query, bounding b…
leshy Oct 11, 2025
2d3f944
timestamped flakey test fix
leshy Oct 11, 2025
fd7e268
circular import bugfix
leshy Oct 11, 2025
849adf0
Fix Image __eq__
spomichter Oct 11, 2025
67a53f8
squash
paul-nechifor Oct 11, 2025
ddce50c
001
paul-nechifor Oct 11, 2025
ae6510b
002
paul-nechifor Oct 11, 2025
ace0725
better universal json and detection parsing for vlms
leshy Oct 11, 2025
3e04c3a
Merge pull request #675 from dimensionalOS/just-resources
paul-nechifor Oct 11, 2025
a37f262
forgotten context managers
paul-nechifor Oct 11, 2025
c0c99d8
Removed tests added in merge by accident
spomichter Oct 11, 2025
8546465
renamed detections2d to detections
leshy Oct 11, 2025
54ad21f
Merge pull request #676 from dimensionalOS/forgotten-context-managers
paul-nechifor Oct 11, 2025
c83ccf8
Merge branch 'dev' into temp-dev
spomichter Oct 11, 2025
08d3e16
CI code cleanup
spomichter Oct 11, 2025
98e1c24
obsoleted inconvinient detection format entirely
leshy Oct 11, 2025
638d81e
tests cleanup
leshy Oct 11, 2025
b686198
detector grid testing
leshy Oct 12, 2025
75a4abf
yolo person detector cuda
leshy Oct 12, 2025
b6be880
vlm sketch
leshy Oct 12, 2025
8f2643b
add autoconnect back
paul-nechifor Oct 11, 2025
6d35a8a
001
paul-nechifor Oct 12, 2025
a8018bb
WIP
paul-nechifor Oct 12, 2025
8056b24
completely removed dep on old detection format
leshy Oct 12, 2025
e25689c
tests fix, module config fix
leshy Oct 12, 2025
7b22223
Fix race condition in CI for dask daemon threads
spomichter Oct 12, 2025
eece78e
001
paul-nechifor Oct 12, 2025
ea238d8
detection3d split into bbox and pc
leshy Oct 12, 2025
928c76c
big detection restructure
leshy Oct 12, 2025
4e82fa9
restructure, mypy
leshy Oct 12, 2025
415eb64
base.py for types, extracted table rendering to utils
leshy Oct 12, 2025
a0a17d6
conftest typing
leshy Oct 12, 2025
ba70c4d
fix module tests
paul-nechifor Oct 12, 2025
fb5f22f
all mypy resolved
leshy Oct 12, 2025
e216d24
Re fix floating point flaky test
spomichter Oct 12, 2025
eee5ecf
Fix nvimgcodec tests from reloading Image and failing pytests when ru…
spomichter Oct 12, 2025
6bf2180
Fix potential timestamp issue if set to 0.0
spomichter Oct 12, 2025
bf95182
Added back agent image message to agent encode
spomichter Oct 12, 2025
d7a7836
Added Timestamped back to new Image
spomichter Oct 12, 2025
5aaae9a
Fixed sharpness implemented incorrectly as a method rather than prope…
spomichter Oct 12, 2025
a5443d1
Add co-author attribution
spomichter Oct 12, 2025
2bac165
Fix flaky wavefront test
spomichter Oct 12, 2025
c1f01be
Test 0.1 thread delay for dask
spomichter Oct 12, 2025
84541f1
session level fixtures
leshy Oct 12, 2025
3d599d1
moondream integrated, generic huggingface model integration
leshy Oct 12, 2025
4d007ea
slightly nicer bounding boxes, slightly better vlm tests
leshy Oct 12, 2025
ce6a923
intelligent annotation font size, model warmup function
leshy Oct 12, 2025
484ae0d
messing with detections
leshy Oct 12, 2025
d0fb0c0
color brightness for from_string
leshy Oct 12, 2025
c1fbd8e
Merge pull request #680 - CUDAImage, NumpyImage, Image refactor / imp…
spomichter Oct 13, 2025
fae2062
Added dotenv back to removed .envrc. Needed to load .env
spomichter Oct 13, 2025
ccbe62b
Merge pull request #673 - Fix .envrc and flaky test
spomichter Oct 13, 2025
fbe734a
Rename FakeRTC --> ReplayRTC
spomichter Oct 13, 2025
c0ca950
fix websocket vis performance
paul-nechifor Oct 4, 2025
4dea67e
fix
paul-nechifor Oct 8, 2025
500edac
Merge pull request #681 from dimensionalOS/rename-fake-to-replay
spomichter Oct 13, 2025
33df738
mobileclip for reid, transforms for detections
leshy Oct 13, 2025
57d5296
detection reconstruction in another module
leshy Oct 13, 2025
c4c9463
002
paul-nechifor Oct 13, 2025
fff177f
reid module, mobileclip
leshy Oct 13, 2025
37ff0f1
code review change
paul-nechifor Oct 13, 2025
98cdc40
Merge pull request #682 from dimensionalOS/fix-websocketvis-performan…
paul-nechifor Oct 13, 2025
a08e2ce
add different file
paul-nechifor Oct 14, 2025
2adbf66
Merge branch 'dev' into add-autoconnect-back2
paul-nechifor Oct 14, 2025
3771e33
quick person tracker
leshy Oct 14, 2025
342d9af
person tracker cleanup
leshy Oct 14, 2025
8010105
created new navbot class
alexlin2 Sep 18, 2025
983b371
Added super duper README, navbot fully supports ros navigation
alexlin2 Sep 20, 2025
19d3a6c
added navigate to goal and cancel navigation to navbot
alexlin2 Sep 22, 2025
6004a17
added unified readme
alexlin2 Sep 22, 2025
d7f589b
added map to world static
alexlin2 Sep 22, 2025
b4b45a2
updated README
alexlin2 Sep 24, 2025
aed04ec
made navigation module and modified navbot
alexlin2 Oct 3, 2025
3a08b48
conformed to Ivan's nav spec, made navigation module a standalone mod…
alexlin2 Oct 3, 2025
1d81aa8
reset g1 to dev
alexlin2 Oct 3, 2025
699a160
fixed some comments
alexlin2 Oct 5, 2025
22855ef
fixed all of Ivan's requests
alexlin2 Oct 7, 2025
d802a39
all navbot
alexlin2 Oct 10, 2025
001f3ba
use navbot
alexlin2 Oct 10, 2025
f617dfb
made comment changes
alexlin2 Oct 10, 2025
6b04663
should pass tests
alexlin2 Oct 10, 2025
351d167
made changes given comments
alexlin2 Oct 13, 2025
7ea7046
rebased origin
alexlin2 Oct 14, 2025
c5ae2e7
passing tests
alexlin2 Oct 14, 2025
8cdc92a
clip/mobileclip standardized implementation
leshy Oct 15, 2025
9e6c6d1
reid experiment
leshy Oct 15, 2025
a2813e8
reid simplification
leshy Oct 15, 2025
7687619
disabling single test for now
leshy Oct 15, 2025
c13395f
removing garbage files
leshy Oct 15, 2025
451b309
correct test naming
leshy Oct 15, 2025
5f810fb
renamde type.py -> base.py for embedding models
leshy Oct 15, 2025
fb95c2f
Merge branch 'dev' into detection-qwen-skills
leshy Oct 15, 2025
9dea5ee
openclip optional, passing tests
leshy Oct 15, 2025
268501d
image backend test skip
leshy Oct 15, 2025
5fdac33
removing .claude
leshy Oct 15, 2025
07761e1
tests fix
leshy Oct 15, 2025
2c5565c
mobile clip optional
leshy Oct 15, 2025
2f03976
small bug fix
alexlin2 Oct 16, 2025
72b2bb1
Merge pull request #632 from dimensionalOS/alexl_ros_nav_intergration
spomichter Oct 16, 2025
6b3e708
add more robust autoconnect
paul-nechifor Oct 14, 2025
5640800
Merge branch 'dev' into add-autoconnect-back2
paul-nechifor Oct 16, 2025
db0ff3b
CI code cleanup
paul-nechifor Oct 16, 2025
3cd5641
torch reid import issues fix
leshy Oct 16, 2025
a29e156
removing package optionality for now
leshy Oct 16, 2025
28b25d4
cli tooling theme
leshy Oct 16, 2025
9b746c8
small colors fixes
leshy Oct 16, 2025
338693f
embedding models heavy tests import fix
leshy Oct 16, 2025
8d5c0ae
resolved import issues
leshy Oct 16, 2025
c4ebc93
unified import resolution strategy
leshy Oct 16, 2025
46512b0
lcmspy rpc color
leshy Oct 16, 2025
3bbd13f
removed scrollbars
leshy Oct 16, 2025
0aa462c
disabled embedding tests for now
leshy Oct 17, 2025
4ca85ec
marking tests as gpu, not heavy
leshy Oct 17, 2025
3e44e70
Merge pull request #674 - detection pipeline rewrite, embedding, vl m…
spomichter Oct 17, 2025
c738b79
Merge pull request #687 from dimensionalOS/terminal-theme
spomichter Oct 17, 2025
1f03ca2
Fix spatial memory bug in unitree g1
alexlin2 Oct 16, 2025
0528de8
Merge branch 'dev' of github.com:dimensionalOS/dimos into dev
spomichter Oct 17, 2025
5dbf7e4
CI code cleanup
spomichter Oct 17, 2025
be94cb6
Merge pull request #689 from dimensionalOS/dev-g1-changes
spomichter Oct 17, 2025
459331a
skill stuff
paul-nechifor Oct 16, 2025
0b07d4f
fix one test
paul-nechifor Oct 20, 2025
4be099f
fix test
paul-nechifor Oct 20, 2025
20a8fc0
Merge branch 'dev' into add-autoconnect-back2
paul-nechifor Oct 20, 2025
5875f29
blueprint
paul-nechifor Oct 20, 2025
bfcdca4
Merge pull request #684 from dimensionalOS/add-autoconnect-back2
paul-nechifor Oct 23, 2025
b3afd11
ruff autofixes
paul-nechifor Oct 23, 2025
3cde19e
fix some issues
paul-nechifor Oct 22, 2025
2e3bf4a
fix
paul-nechifor Oct 22, 2025
a38c6f1
from future
paul-nechifor Oct 23, 2025
014341c
CI code cleanup
paul-nechifor Oct 23, 2025
4ce6712
Remove unused.
paul-nechifor Oct 23, 2025
f0ea7d8
fix unused
paul-nechifor Oct 23, 2025
63987b0
fix bad imports
paul-nechifor Oct 23, 2025
4fcf1d2
more autofixes
paul-nechifor Oct 23, 2025
65b6d15
add types
paul-nechifor Oct 23, 2025
7c2076e
add safe annotations
paul-nechifor Oct 23, 2025
1748c46
auto typing
paul-nechifor Oct 23, 2025
aa3f01b
new ruff
paul-nechifor Oct 23, 2025
952c87d
fix bad numpy
paul-nechifor Oct 23, 2025
9b128a5
fix ros polyfill
paul-nechifor Oct 23, 2025
c1dd083
CI code cleanup
paul-nechifor Oct 24, 2025
e75810d
remove optype
paul-nechifor Oct 24, 2025
85efd20
add remapping
paul-nechifor Oct 24, 2025
72f0623
remapping
paul-nechifor Oct 24, 2025
705333a
remove with
paul-nechifor Oct 26, 2025
dac7079
Revert "remove with"
paul-nechifor Oct 26, 2025
fe1b15c
proper rename and remove DIMOS_
paul-nechifor Oct 26, 2025
c6a12b4
add demos
paul-nechifor Oct 26, 2025
0bd2602
Merge pull request #698 from dimensionalOS/rename
paul-nechifor Oct 26, 2025
e302059
add JPEG LCM
paul-nechifor Oct 21, 2025
0361179
fix join
paul-nechifor Oct 21, 2025
2f5023a
add blueprint
paul-nechifor Oct 26, 2025
bc651f5
fix script
paul-nechifor Oct 26, 2025
fb9c902
JpegSharedMemoryEncoderMixin
paul-nechifor Oct 26, 2025
e0a74ee
jpeg_shm_channels
paul-nechifor Oct 26, 2025
f4fb8c6
update dimos-lcm
paul-nechifor Oct 27, 2025
6345dd6
Merge pull request #693 - JPEG Image compression on LCM transport
spomichter Oct 27, 2025
d7d15d2
Merge dev.
paul-nechifor Oct 27, 2025
11c6549
autofix
paul-nechifor Oct 27, 2025
6ada634
remove unused
paul-nechifor Oct 27, 2025
6788a98
remove scipy-stubs
paul-nechifor Oct 27, 2025
4bb41ec
Merge pull request #694 - Added Ruff fixes and MyPy typechecking acro…
spomichter Oct 28, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
32 changes: 32 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
{
"name": "dimos-dev",
"image": "ghcr.io/dimensionalos/dev:dev",
"customizations": {
"vscode": {
"extensions": [
"charliermarsh.ruff",
"ms-python.vscode-pylance"
]
}
},
"containerEnv": {
"PYTHONPATH": "${localEnv:PYTHONPATH}:/workspaces/dimos"
},
"postCreateCommand": "git config --global --add safe.directory /workspaces/dimos && cd /workspaces/dimos && pre-commit install",
"settings": {
"notebook.formatOnSave.enabled": true,
"notebook.codeActionsOnSave": {
"notebook.source.fixAll": "explicit",
"notebook.source.organizeImports": "explicit"
},
"editor.codeActionsOnSave": {
"source.fixAll": "explicit",
"source.organizeImports": "explicit"
},
"editor.defaultFormatter": "charliermarsh.ruff",
"editor.formatOnSave": true
},
"runArgs": [
"--cap-add=NET_ADMIN"
]
}
109 changes: 109 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Version control
.git
.gitignore
.github/

# Editor and IDE files
.vscode
.idea
*.swp
*.swo
.cursor/
.cursorignore

# Shell history
.bash_history
.zsh_history
.history

# Python virtual environments
**/venv/
**/.venv/
**/env/
**/.env/
**/*-venv/
**/*_venv/
**/ENV/


# Python build artifacts
__pycache__/
*.pyc
*.pyo
*.pyd
.Python
*.egg-info/
dist/
build/
*.so
*.dylib

# Environment file
.env
.env.local
.env.*.local

# Large data files
data/*
!data/.lfs/

# Model files (can be downloaded at runtime)
*.pt
*.pth
*.onnx
*.pb
*.h5
*.ckpt
*.safetensors
checkpoints/
assets/model-cache

# Logs
*.log

# Large media files (not needed for functionality)
*.png
*.jpg
*.jpeg
*.gif
*.mp4
*.mov
*.avi
*.mkv
*.webm
*.MOV

# Large font files
*.ttf
*.otf

# Node modules (for dev tools, not needed in container)
node_modules/
package-lock.json
package.json
bin/node_modules/

# Database files
*.db
*.sqlite
*.sqlite3

# OS generated files
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Temporary files
tmp/
temp/
*.tmp
.python-version

# Exclude all assets subdirectories
assets/*/*
!assets/agent/prompt.txt
!assets/*
7 changes: 5 additions & 2 deletions .envrc
Original file line number Diff line number Diff line change
@@ -1,2 +1,5 @@
export DIRENV_WARN_TIMEOUT=-1000m
./bin/dev
if ! has nix_direnv_version || ! nix_direnv_version 3.0.6; then
source_url "https://raw.githubusercontent.com/nix-community/nix-direnv/3.0.6/direnvrc" "sha256-RYcUJaRMf8oF5LznDrlCXbkOQrywm0HDv1VjYGaJGdM="
fi
use flake .
dotenv
5 changes: 5 additions & 0 deletions .envrc.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
if ! has nix_direnv_version || ! nix_direnv_version 3.0.6; then
source_url "https://raw.githubusercontent.com/nix-community/nix-direnv/3.0.6/direnvrc" "sha256-RYcUJaRMf8oF5LznDrlCXbkOQrywm0HDv1VjYGaJGdM="
fi
use flake .
dotenv_if_exists
2 changes: 2 additions & 0 deletions .envrc.venv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
source env/bin/activate
dotenv_if_exists
10 changes: 6 additions & 4 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1,14 +1,16 @@
# Handle line endings automatically for files Git considers text,
# converting them to LF on checkout.
* text=auto eol=lf

# Ensure Python files always use LF for line endings.
*.py text eol=lf

# Treat designated file types as binary and do not alter their contents or line endings.
*.png binary
*.jpg binary
*.gif binary
*.ico binary
*.pdf binary
*.mp4 binary
# Explicit LFS tracking for test files
/data/.lfs/*.tar.gz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text binary
*.mp4 filter=lfs diff=lfs merge=lfs -text binary
*.mov filter=lfs diff=lfs merge=lfs -text binary
*.gif filter=lfs diff=lfs merge=lfs -text binary
59 changes: 59 additions & 0 deletions .github/actions/docker-build/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: docker-build
description: "Composite action to build and push a Docker target to GHCR"
inputs:
target:
description: "Dockerfile target stage to build"
required: true
tag:
description: "Image tag to push"
required: true
freespace:
description: "Remove large pre‑installed SDKs before building to free space"
required: false
default: "false"
context:
description: "Docker build context"
required: false
default: "."

runs:
using: "composite"
steps:
- name: Free up disk space
if: ${{ inputs.freespace == 'true' }}
shell: bash
run: |
echo -e "pre cleanup space:\n $(df -h)"
sudo rm -rf /opt/ghc
sudo rm -rf /usr/share/dotnet
sudo rm -rf /usr/local/share/boost
sudo rm -rf /usr/local/lib/android
echo -e "post cleanup space:\n $(df -h)"

- uses: actions/checkout@v4

- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ github.token }}

- uses: crazy-max/ghaction-github-runtime@v3

- uses: docker/setup-buildx-action@v3
with:
driver: docker-container
install: true
use: true

- name: Build & Push ${{ inputs.target }}
uses: docker/build-push-action@v6
with:
push: true
context: ${{ inputs.context }}
file: docker/${{ inputs.target }}/Dockerfile
tags: ghcr.io/dimensionalos/${{ inputs.target }}:${{ inputs.tag }}
cache-from: type=gha,scope=${{ inputs.target }}
cache-to: type=gha,mode=max,scope=${{ inputs.target }}
build-args: |
FROM_TAG=${{ inputs.tag }}
149 changes: 149 additions & 0 deletions .github/workflows/_docker-build-template.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
name: docker-build-template
on:
workflow_call:
inputs:
from-image: { type: string, required: true }
to-image: { type: string, required: true }
dockerfile: { type: string, required: true }
freespace: { type: boolean, default: true }
should-run: { type: boolean, default: false }
context: { type: string, default: '.' }

# you can run this locally as well via
# ./bin/dockerbuild [image-name]
jobs:
build:
runs-on: [self-hosted, Linux]
permissions:
contents: read
packages: write

steps:
- name: Fix permissions
if: ${{ inputs.should-run }}
run: |
sudo chown -R $USER:$USER ${{ github.workspace }} || true

- uses: actions/checkout@v4
if: ${{ inputs.should-run }}
with:
fetch-depth: 0

- name: free up disk space
# explicitly enable this for large builds
if: ${{ inputs.should-run && inputs.freespace }}
run: |
echo -e "pre cleanup space:\n $(df -h)"
sudo rm -rf /opt/ghc
sudo rm -rf /usr/share/dotnet
sudo rm -rf /usr/local/share/boost
sudo rm -rf /usr/local/lib/android

echo "=== Cleaning images from deleted branches ==="

# Get list of all remote branches
git ls-remote --heads origin | awk '{print $2}' | sed 's|refs/heads/||' > /tmp/active_branches.txt

# Check each docker image tag against branch list
docker images --format "{{.Repository}}:{{.Tag}}|{{.ID}}" | \
grep "ghcr.io/dimensionalos" | \
grep -v ":<none>" | \
while IFS='|' read image_ref id; do
tag=$(echo "$image_ref" | cut -d: -f2)

# Skip if tag matches an active branch
if grep -qx "$tag" /tmp/active_branches.txt; then
echo "Branch exists: $tag - keeping $image_ref"
else
echo "Branch deleted: $tag - removing $image_ref"
docker rmi "$id" 2>/dev/null || true
fi
done

rm -f /tmp/active_branches.txt

USAGE=$(df / | awk 'NR==2 {print $5}' | sed 's/%//')
echo "Pre-docker-cleanup disk usage: ${USAGE}%"

if [ $USAGE -gt 60 ]; then
echo "=== Running quick cleanup (usage > 60%) ==="

# Keep newest image per tag
docker images --format "{{.Repository}}|{{.Tag}}|{{.ID}}" | \
grep "ghcr.io/dimensionalos" | \
grep -v "<none>" | \
while IFS='|' read repo tag id; do
created_ts=$(docker inspect -f '{{.Created}}' "$id" 2>/dev/null)
created_unix=$(date -d "$created_ts" +%s 2>/dev/null || echo "0")
echo "${repo}|${tag}|${id}|${created_unix}"
done | sort -t'|' -k1,1 -k2,2 -k4,4nr | \
awk -F'|' '
{
repo=$1; tag=$2; id=$3
repo_tag = repo ":" tag

# Skip protected tags
if (tag ~ /^(main|dev|latest)$/) next

# Keep newest per tag, remove older duplicates
if (!(repo_tag in seen_combos)) {
seen_combos[repo_tag] = 1
} else {
system("docker rmi " id " 2>/dev/null || true")
}
}'

docker image prune -f
docker volume prune -f
fi

# Aggressive cleanup if still above 85%
USAGE=$(df / | awk 'NR==2 {print $5}' | sed 's/%//')
if [ $USAGE -gt 85 ]; then
echo "=== AGGRESSIVE cleanup (usage > 85%) - removing all except main/dev ==="

# Remove ALL images except main and dev tags
docker images --format "{{.Repository}}:{{.Tag}} {{.ID}}" | \
grep -E "ghcr.io/dimensionalos" | \
grep -vE ":(main|dev)$" | \
awk '{print $2}' | xargs -r docker rmi -f || true

docker container prune -f
docker volume prune -a -f
docker network prune -f
docker image prune -f
fi

echo -e "post cleanup space:\n $(df -h)"

- uses: docker/login-action@v3
if: ${{ inputs.should-run }}
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

# required for github cache of docker layers
- uses: crazy-max/ghaction-github-runtime@v3
if: ${{ inputs.should-run }}

# required for github cache of docker layers
- uses: docker/setup-buildx-action@v3
if: ${{ inputs.should-run }}
with:
driver: docker-container
install: true
use: true

- uses: docker/build-push-action@v6
if: ${{ inputs.should-run }}
with:
push: true
context: ${{ inputs.context }}
file: docker/${{ inputs.dockerfile }}/Dockerfile
tags: ${{ inputs.to-image }}
cache-from: type=gha,scope=${{ inputs.dockerfile }}
cache-to: type=gha,mode=max,scope=${{ inputs.dockerfile }}
#cache-from: type=gha,scope=${{ inputs.dockerfile }}-${{ inputs.from-image }}
#cache-to: type=gha,mode=max,scope=${{ inputs.dockerfile }}-${{ inputs.from-image }}
build-args: FROM_IMAGE=${{ inputs.from-image }}
Loading
Loading