Skip to content
This repository was archived by the owner on Aug 29, 2023. It is now read-only.
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
149 commits
Select commit Hold shift + click to select a range
6e1bed1
Guard against nil in GVKForObject
luxas Dec 28, 2020
35ed90d
Handle the case where the kind directory does not exist when listing
luxas Dec 28, 2020
69e47d6
Handle the case in Storage.Patch where the underlying RawStorage retu…
luxas Dec 28, 2020
2693d1d
Add a labels filter, split name & namespace to separate filters, impl…
luxas Jan 11, 2021
9c060fb
Make pretty encoding catch also cases when the target object implemen…
luxas Jan 11, 2021
b644f73
Make FrameReader thread-safe, and implement a Single-Frame FrameReader
luxas Jan 11, 2021
6d64cff
Handle updated kyaml in tests
luxas Jan 11, 2021
29e4ca6
Always add the mutex when creating the framewriter in test
luxas Jan 11, 2021
21b8571
Move common functions to an utils file. Add a mutex to guard the sche…
luxas Jan 11, 2021
a9985cc
Never convert unknown, unstructured or partial objects
luxas Jan 11, 2021
7106536
Guard the AddKnownTypes call to the scheme by a mutex, to avoid race …
luxas Jan 11, 2021
5864288
Make it possible for the codec used internally to recognize partial o…
luxas Jan 11, 2021
951a162
Use the upstream YAML MetaFactory.
luxas Jan 11, 2021
1dd8632
Fix the code getting the extension from the content-type, earlier it …
luxas Jan 11, 2021
73f90ba
Use the "functional" options pattern from controller-runtime in the s…
luxas Jan 11, 2021
143839c
Create pluggable namespace-related bridges between our Storage system…
luxas Jan 11, 2021
df6dc0d
Add IntPtr to utils.
luxas Jan 11, 2021
590b98d
Add a new Patcher interface to the serializer, that can patch both ty…
luxas Jan 14, 2021
da0e977
Fix default value in godoc
luxas Jan 14, 2021
02e0e44
update import
luxas Jan 14, 2021
23b74bf
Add context & a Namespacer to RawStorage, and support a bit more vari…
luxas Jan 14, 2021
4ec5457
Rewrite patchutil to only operate on bytes, and add support for diffe…
luxas Jan 14, 2021
173c31a
Rewrite the key system; import most things from pkg/client
luxas Jan 14, 2021
97db70a
First snapshot of the storage system rewrite.
luxas Jan 14, 2021
d1f6dc0
Check in all the really core interfaces of the storage system
luxas Jan 18, 2021
f50276b
Check in a filesystem abstraction layer that will be used across the …
luxas Jan 18, 2021
51f6a88
format.go contains common interfaces and implementations of content-t…
luxas Jan 18, 2021
8324247
Implementations for (Unversioned)ObjectID
luxas Jan 18, 2021
a039e08
A set of common error creation and check functions
luxas Jan 18, 2021
eb6af4c
Implement a default ObjectRecognizer
luxas Jan 18, 2021
6de1f2c
Implement a sample (and simple) Namespacer that statically determines…
luxas Jan 18, 2021
53be24c
Add a definition and sample implementation of PathExcluder.
luxas Jan 18, 2021
15fadfc
Add sample Namespacer and NamespaceEnforcer implementations for Kuber…
luxas Jan 18, 2021
fb40319
Register removed files that were rewritten and moved to pkg/storage/core
luxas Jan 18, 2021
f161c96
Check in the main raw.Storage-related interfaces.
luxas Jan 18, 2021
2987841
Move and rewrite the GenericRawStorage implementation completely, now…
luxas Jan 18, 2021
d4149fe
Rewrite GenericMappedRawStorage into GenericMappedFileFinder.
luxas Jan 18, 2021
7acf4dc
A small glue file implementing for implementing the ObjectInfo interf…
luxas Jan 18, 2021
fc339e1
Add event types for both files and objects, and interfaces for unstru…
luxas Jan 19, 2021
7ff01ea
Move the directory traversal code from watcherutil to core; make it u…
luxas Jan 19, 2021
5b2fffd
File and Object update event structs have now been moved to pkg/stora…
luxas Jan 19, 2021
e0481ff
Move and refactor the filewatcher implementation. Make it more thread…
luxas Jan 19, 2021
a88a1b1
Refactor the WatchStorage implementation into an UnstructuredEventSto…
luxas Jan 19, 2021
a3fa3dc
Create a "manifest" helper constructor.
luxas Jan 19, 2021
4a2e70a
Snapshot latest modifications to pkg/storage
luxas Jan 19, 2021
c57092d
Split out a "backend" part of the storage.
luxas Jan 22, 2021
193209c
Make a dedicated Lister interface, and also ContentTypeResolver, that…
luxas Jan 23, 2021
8fdce64
Move backend and client into their own subdirectories of pkg/storage
luxas Jan 23, 2021
dc9d245
Move directories around; make pkg/storage top-level for abstract stor…
luxas Jan 23, 2021
7f9b432
Rename AferoContext to Filesystem, and add a checksum method that'll …
luxas Jan 23, 2021
51e94ad
Fixup all imports and similar changes from the earlier two restructur…
luxas Jan 23, 2021
18e9745
Change the top-level Storage; remove the Stat() method in favor for t…
luxas Jan 23, 2021
b28900c
Always expose a ContentTyper for the FileFinder: create a generic "st…
luxas Jan 23, 2021
142e5f2
Change filesystem.Generic to use the Checksum/ContentType methods.
luxas Jan 23, 2021
4204cbb
Move pkg/storage/filesystem/{watch,fileevents}. Make the Emitter inte…
luxas Jan 23, 2021
054c09a
Make the MappedFileFinder expose ContentTyper.
luxas Jan 23, 2021
72aefa8
Make the inotify emitter simpler by removing the duty of it to valida…
luxas Jan 23, 2021
5a6d859
Polish the unstructured.Storage interface a bit, and create a first i…
luxas Jan 23, 2021
6076ce0
Move the "WatchStorage" from ./watch/ to ./unstructured/event. Make i…
luxas Jan 23, 2021
567b5bf
Misc changes: added various comments for future reference, removed un…
luxas Jan 23, 2021
701c40c
Move the NamespaceEnforcer interface and the sample implementation to…
luxas Jan 25, 2021
82cb551
Minor bugfixes and improvements
luxas Jan 25, 2021
9e5262c
Make pathexcluder explicitely work without a filesystem abstraction; …
luxas Jan 25, 2021
38c5d70
Add the PathExcluder back to FileWatcher, to avoid putting a lot of b…
luxas Jan 25, 2021
b04de27
Check in the first revision of the transactional client.
luxas Jan 26, 2021
11b0a6e
Check in the VersionRef file.
luxas Jan 27, 2021
d38a7c3
Simplify the Client.List operation significantly using utilerrs.
luxas Jan 27, 2021
2a8792c
Use the slightly updated versionref interface
luxas Jan 27, 2021
5184472
Rename the handlers to hooks, and add a chain interface too, which al…
luxas Jan 27, 2021
9ea7647
Make the PathExcluder simpler; create a multi-one and a file extensio…
luxas Jan 27, 2021
a326065
Update the monitor to take in an error.
luxas Jan 27, 2021
c45e37b
Use the new pathexcluder and monitor in the filewatcher, and always e…
luxas Jan 27, 2021
1d29942
Add validation to the objectrecognizer
luxas Jan 27, 2021
a26e18b
Fix race condition in Client.List
luxas Jan 27, 2021
47dec8e
Check in the distributed interfaces and options.
luxas Jan 27, 2021
c2583b6
Check in the distributed client.
luxas Jan 27, 2021
414e62f
Move place for the GitDirectory; now it is a branchmanager and remote.
luxas Jan 27, 2021
d3894ba
Move a part of the PullRequest structs to the Github PR package; and …
luxas Jan 27, 2021
b514e58
Remove the old transaction code
luxas Jan 27, 2021
d64a0e6
Remove the old cache implementation; not needed anymore.
luxas Jan 27, 2021
0afeff2
Remove the old client code, not needed anymore.
luxas Jan 27, 2021
6a9d9bb
Remove the old SyncStorage; not needed anymore.
luxas Jan 27, 2021
3e05835
Move pkg/logs to cmd/ as it is only sample code, not something that s…
luxas Jan 27, 2021
a6f6920
Remove the last pieces of the old runtime code.
luxas Jan 27, 2021
f250867
Update the sample applications.
luxas Jan 27, 2021
ab466b4
Update go.mod
luxas Jan 27, 2021
00abe17
Remove old pieces of the generated client code.
luxas Jan 27, 2021
affc5a8
make tidy
luxas Jan 27, 2021
46acb8f
Fixup the makefile a bit
luxas Jan 27, 2021
850d735
Get rid of the separate util package
luxas Jan 27, 2021
f2aefd6
go mod update
luxas Jan 27, 2021
e8c4038
Make the serializer system more pluggable and "open".
luxas Jan 29, 2021
970642e
Create and use a sample StorageVersioner.
luxas Jan 29, 2021
33809d8
Fix some comments.
luxas Jan 29, 2021
a60636f
Add more comments
luxas Feb 1, 2021
90ac7b5
Remove unnecessary "Commons"
luxas Feb 1, 2021
d509be5
Move Namespacer to pkg/storage.
luxas Feb 1, 2021
ef8d504
Move ObjectRecognizer to the unstructured package instead.
luxas Feb 1, 2021
0d5a3b1
Rename MappedFileFinder to unstructured.FileFinder
luxas Feb 1, 2021
2b936f9
Move the (Tx|Commit)HookChains to the transactional.Client instead of…
luxas Feb 1, 2021
c0d0ff2
Fix TODO in the simple filefinder.
luxas Feb 1, 2021
097e9d1
Minor comment
luxas Feb 1, 2021
6ec2841
Minor comment
luxas Feb 1, 2021
4d3aeb1
Add sets of ObjectIDs for ease of use elsewhere in this system
luxas Feb 1, 2021
8f4b91a
Implement fmt.Stringer for ObjectIDs
luxas Feb 1, 2021
3928b90
Make the ObjectRecognizer interface better; make it use FrameReaders.
luxas Feb 1, 2021
ee9fea6
Small optimization; set a given size on the set from the initial ids;…
luxas Feb 1, 2021
f057504
Use the sets across the codebase.
luxas Feb 1, 2021
8b27604
Make FileFinder.ObjectsAt support multiple files, potentially.
luxas Feb 1, 2021
24d1b72
Minor code move: Move all Object-specific code from core to the clien…
luxas Feb 1, 2021
43efc2e
an idea for the future.
luxas Feb 1, 2021
413a6f4
Add an "OldPath" property of FileEvent; and populate that in FileWatc…
luxas Feb 4, 2021
d2c3687
Rename SchemeLock -> GetLockedScheme to avoid func naming mismatches
luxas Feb 4, 2021
33565c8
Remove commit from versionref; can't really support commits now: it's…
luxas Feb 4, 2021
9c50313
Change to GetLockedScheme
luxas Feb 4, 2021
c5fb961
Adapt to VersionRef changes
luxas Feb 4, 2021
d08bb28
Remove the ObjectID set as it's redundant; focus more on the unversio…
luxas Feb 4, 2021
1dd97f5
Adapt to set changes.
luxas Feb 4, 2021
25053f3
Support multi-frame files, duplicate IDs across files, and various Ve…
luxas Feb 5, 2021
4076935
Allow the user to plug in their wanted FrameReaderFactory for reading…
luxas Feb 5, 2021
cf792b0
Reject expired contexts in the backend
luxas Feb 5, 2021
60c90f2
Unify some locking code in the txclient
luxas Feb 5, 2021
2e257ae
Move the singleframereader, update it slightly, and add a singleframe…
luxas Feb 5, 2021
5847540
Use the singleframewriter in the backend to support any content type.
luxas Feb 5, 2021
e8949ff
Implement ListGroupKinds in storage.Lister across the system.
luxas Feb 5, 2021
653075a
Fix small bug
luxas Feb 17, 2021
8230bcc
Fix minor bugs in the txclient, and move one func to utils.go
luxas Feb 17, 2021
9320092
Create a btree high-level implementation that supports indexing and m…
luxas Feb 17, 2021
ca752ab
Add a helper function to list unique submatches
luxas Feb 18, 2021
fdca597
Refactor the unstructured FileFinder to work with the BTree instead
luxas Feb 18, 2021
c33cbd5
Add new versionref-related functions to the unstructured FileFinder, …
luxas Feb 18, 2021
0d4cfd6
Remove the old mapped cache
luxas Feb 18, 2021
0babdfd
Create a hook for automatically doing a copy-on-write for a new branc…
luxas Feb 18, 2021
7df21b4
go mod tidy
luxas Feb 18, 2021
5785701
Document the btree index from the ground up, and implement Find/List …
luxas Feb 24, 2021
2844d8c
Add unit test for the btree index impls.
luxas Feb 24, 2021
eb2a8a2
Improve ListUnique greatly, remove the hacky AdvanceLastChar, and add…
luxas Feb 24, 2021
c879b47
Update the unstructured filefinder to use the latest BTree impl
luxas Feb 24, 2021
b492c2d
Initial stab at a git interface internally
luxas Apr 14, 2021
72aa730
WIP
luxas Jun 23, 2021
0bd8db6
Merge v2_framing code
luxas Jul 27, 2021
6032359
Update the serializer package to use the new framing library
luxas Jul 27, 2021
e71ba14
Update the storage system to use the framing library
luxas Jul 27, 2021
72be7a9
WIP
luxas Jul 29, 2021
54aead6
Check in progress on framing and sanitation documentation.
luxas Aug 2, 2021
a6be8d1
WIP new options pattern
luxas Aug 2, 2021
15b389f
WIP sanitation improvements
luxas Aug 3, 2021
20e744f
WIP
luxas Aug 4, 2021
70deb2a
WIP just for demo
luxas Aug 9, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,8 @@
.vscode/

# Binary artifacts
bin/
bin/
main_test.go
vendor
*.txt
old
4 changes: 1 addition & 3 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
UID_GID ?= $(shell id -u):$(shell id -g)
GO_VERSION ?= 1.14.4
GO_VERSION ?= 1.15.6
GIT_VERSION := $(shell hack/ldflags.sh --version-only)
PROJECT := github.com/weaveworks/libgitops
BOUNDING_API_DIRS := ${PROJECT}/cmd/apis/sample
API_DIRS := ${PROJECT}/cmd/sample-app/apis/sample,${PROJECT}/cmd/sample-app/apis/sample/v1alpha1
SRC_PKGS := cmd pkg
DOCKER_ARGS := --rm
CACHE_DIR := $(shell pwd)/bin/cache
API_DOCS := api/sample-app.md api/runtime.md
BINARIES := bin/sample-app bin/sample-gitops bin/sample-watch

# If we're not running in CI, run Docker interactively
Expand Down Expand Up @@ -39,7 +38,6 @@ test-internal:
tidy: docker-tidy-internal
tidy-internal: /go/bin/goimports
go mod tidy
hack/generate-client.sh
gofmt -s -w ${SRC_PKGS}
goimports -w ${SRC_PKGS}

Expand Down
20 changes: 8 additions & 12 deletions cmd/common/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ import (
"github.com/spf13/pflag"
"github.com/weaveworks/libgitops/cmd/sample-app/apis/sample/v1alpha1"
"github.com/weaveworks/libgitops/cmd/sample-app/version"
"github.com/weaveworks/libgitops/pkg/runtime"
"github.com/weaveworks/libgitops/pkg/storage"
"github.com/weaveworks/libgitops/pkg/storage/client"
"github.com/weaveworks/libgitops/pkg/storage/core"
)

var (
Expand All @@ -25,10 +25,6 @@ func init() {
rand.Seed(time.Now().UnixNano())
}

func CarKeyForName(name string) storage.ObjectKey {
return storage.NewObjectKey(storage.NewKindKey(CarGVK), runtime.NewIdentifier("default/"+name))
}

func NewCar(name string) *v1alpha1.Car {
obj := &v1alpha1.Car{}
obj.Name = name
Expand All @@ -38,17 +34,17 @@ func NewCar(name string) *v1alpha1.Car {
return obj
}

func SetNewCarStatus(s storage.Storage, key storage.ObjectKey) error {
obj, err := s.Get(key)
func SetNewCarStatus(ctx context.Context, c client.Client, name string) error {
car := &v1alpha1.Car{}
err := c.Get(ctx, core.ObjectKey{Name: name}, car)
if err != nil {
return err
}

car := obj.(*v1alpha1.Car)
car.Status.Distance = rand.Uint64()
car.Status.Speed = rand.Float64() * 100

return s.Update(car)
return c.Update(ctx, car)
}

func ParseVersionFlag() {
Expand All @@ -75,8 +71,8 @@ func NewEcho() *echo.Echo {
func StartEcho(e *echo.Echo) error {
// Start the server
go func() {
if err := e.Start(":8888"); err != nil {
e.Logger.Info("shutting down the server")
if err := e.Start(":8881"); err != nil {
e.Logger.Info("shutting down the server", err)
}
}()

Expand Down
3 changes: 3 additions & 0 deletions pkg/logs/flag/flag.go → cmd/common/logs/flag/flag.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ import (
"github.com/spf13/pflag"
)

// TODO: Use these flags in the sample binaries?
// TODO: Move to the way controller-runtime does logs instead?

type LogLevelFlag struct {
value *logrus.Level
}
Expand Down
2 changes: 2 additions & 0 deletions pkg/logs/logs.go → cmd/common/logs/logs.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ import (
log "github.com/sirupsen/logrus"
)

// TODO: Move to the way controller-runtime does logs instead?

// Quiet specifies whether to only print machine-readable IDs
var Quiet bool

Expand Down
61 changes: 0 additions & 61 deletions cmd/sample-app/client/client.go

This file was deleted.

152 changes: 0 additions & 152 deletions cmd/sample-app/client/zz_generated.client_car.go

This file was deleted.

Loading