diff --git a/README.md b/README.md index 9bca754..1b808b2 100644 --- a/README.md +++ b/README.md @@ -14,6 +14,7 @@ __ _ __ CLI tool for performing GitOps operations ## Usage + ``` NAME: gitpos - GitOps CLI @@ -36,16 +37,21 @@ GLOBAL OPTIONS: ``` ### Planning secret application to a cluster + **NOTE:** It is expected, that the cluster's KUBECONFIG is already set up. Alternatively, the `--kubeconfig` flag can be used. ```bash gitops secrets plan kubernetes ``` + Or in short + ```bash gitops s p k8s ``` + Example output: + ``` __ _ __ \ \ ____ _(_) /_____ ____ _____ @@ -74,17 +80,22 @@ use gitops secrets apply kubernetes to apply these changes to your cluster ``` ### Applying secrets to a cluster + **NOTE:** It is expected, that the cluster's KUBECONFIG is already set up. Alternatively, the `--kubeconfig` flag can be used. ```bash gitops secrets apply kubernetes ``` + Or in short + ```bash gitops s a k8s ``` + The user will be prompted to confirm the changes before they are applied to the cluster. The prompt can be bypassed by using the `--auto-approve` flag. Example output: + ``` __ _ __ \ \ ____ _(_) /_____ ____ _____ @@ -168,7 +179,8 @@ The secrets files must follow the following format: ```yaml # target of the secret -target: < k8s | vault > +targetType: < k8s | vault > + # name of the secret name: # optional namespace of the secret (default: default) @@ -185,8 +197,8 @@ data: ##### Case 1: Secret for K8s ```yaml -# target of the secret -target: k8s +# targetType of the secret +targetType: k8s # name of the secret name: my-secret-name # optional namespace of the secret (default: default) @@ -208,13 +220,13 @@ name: my-secret-name This implies, that the filename must be a valid K8s secret name. - ##### Case 2: Secret for Vault + **NOTE:** Vault secrets are still WIP ```yaml # target of the secret -target: vault +targetType: vault # name of the secret - will be used as path in vault name: /my/secret/name # data of the secret as kv pairs @@ -222,13 +234,141 @@ data: key: value ``` - #### Secrets Templating It is possible to use Go templates in the secret files. The values will originate from sops-encrypted `values.gitops.secret.enc.y[a]ml` files. Values files can be located anywhere in the repository. The GitOps CLI will pick up all files that are located on the direct path towards the respective secret file. Values files closer to the secret file will have higher precedence. Any object structure is allowed to be used in a values file. +Example: + +```yaml +# /foo/bar/dev/values.gitops.secret.enc.yaml +environment: dev +database: + host: localhost + port: 5432 + user: postgres + password: postgres +``` + +```yaml +# /foo/bar/dev/values.gitops.secret.enc.yaml +targetType: k8s +# name of the secret - will be used as path in vault +name: my-service +namespace: "{{ .Values.environment }}" +data: + application.properties: |- + service.environment={{ .Values.environment }} + spring.datasource.url=jdbc:postgresql://{{ .Values.database.host }}:{{ .Values.database.port }}/mydb + spring.datasource.username={{ .Values.database.user }} + spring.datasource.password={{ .Values.database.password }} +``` +**NOTE** that the template string (`{{ .Values.someValue }}`) must be enclosed in quotes for sops to work properly. In the above example, the entire `application.properties` data value is considered as a string and thus does not need further quoting. + +#### Multi-cluster support +It is possible to address multiple clusters with a single GitOps repository. +To add a new cluster to the GitOps state use +``` +gitops clusters add +``` +The kubeconfig file can either be a plain text file or a sops-encrypted file. If the file is encrypted, it must adhere to the following naming convention to be decrypted properly: +``` +*.kubeconfig.secret.enc.ya?ml +``` + +To inspect the currently configured clusters use +``` +gitops clusters list +``` +``` +__ _ __ +\ \ ____ _(_) /_____ ____ _____ + \ \ / __ `/ / __/ __ \/ __ \/ ___/ + / / / /_/ / / /_/ /_/ / /_/ (__ ) +/_/ \__, /_/\__/\____/ .___/____/ + /____/ /_/ + +dev => kubeconfigs/dev.kubeconfig.secret.enc.yml +int => kubeconfigs/int.kubeconfig.secret.enc.yml +prod => kubeconfigs/prod.kubeconfig.secret.enc.yml +``` + +To remove a cluster from the GitOps state use +``` +gitops clusters remove +``` + +To check connectivity with the configured clusters use +``` +gitops clusters test +``` +``` +__ _ __ +\ \ ____ _(_) /_____ ____ _____ + \ \ / __ `/ / __/ __ \/ __ \/ ___/ + / / / /_/ / / /_/ /_/ / /_/ (__ ) +/_/ \__, /_/\__/\____/ .___/____/ + /____/ /_/ + +Cluster: __default (v1.25.3) Connected: true +Cluster: dev (v1.24.8) Connected: true +Cluster: int (v1.24.8) Connected: true +Cluster: prod (v1.24.8) Connected: true +``` + +A secret can be configured to be applied to a specific cluster using the `target` attribute in the secret file. The default value is the `__default` cluster which is inferred from the `KUBECONFIG` environment variable or the default kubeconfig file. The `target` attribute can also be set using a templating variable so that all secrets under a certain directory will be applied to a specific cluster. +```yaml +# /foo/bar/dev/values.gitops.secret.enc.yaml +target: dev +``` +```yaml +# /foo/bar/dev/myservice/service.gitops.secret.enc.yaml +targetType: k8s +target: "{{ .Values.target }}" # will be replaced with "dev" +name: my-service +data: + key: value +``` + +By default, secrets will be applied to all configured clusters. This can be limited by giving the cluster as an argument: +``` +gitops secrets plan kubernetes dev +__ _ __ +\ \ ____ _(_) /_____ ____ _____ + \ \ / __ `/ / __/ __ \/ __ \/ ___/ + / / / /_/ / / /_/ /_/ / /_/ (__ ) +/_/ \__, /_/\__/\____/ .___/____/ + /____/ /_/ + +Limiting to cluster dev + +[Loading local secrets] 100% |██████████████████████████████████████████████████| (63/63) + + +No changes to apply. +``` + +#### Directory limiter +It is possible to restrict the secrets input to a specific directory to speed up loading and decryption of secrets. This can be done by providing the `--dir` flag: +``` +gitops secrets --dir application/dev plan kubernetes + +__ _ __ +\ \ ____ _(_) /_____ ____ _____ + \ \ / __ `/ / __/ __ \/ __ \/ ___/ + / / / /_/ / / /_/ /_/ / /_/ (__ ) +/_/ \__, /_/\__/\____/ .___/____/ + /____/ /_/ + +Limiting to directory applications/dev + +[Loading local secrets] 100% |██████████████████████████████████████████████████| (1/1) + +No changes to apply. +``` +**NOTE** that the directory path must be relative to the repository root and that only forward slashes (`/`) are supported. ## Repository ### After the first clone diff --git a/cmd/gitops/kubernetes.go b/cmd/gitops/kubernetes.go index 40de8d4..1864165 100644 --- a/cmd/gitops/kubernetes.go +++ b/cmd/gitops/kubernetes.go @@ -1,6 +1,9 @@ package main import ( + "fmt" + "strings" + "github.com/TwiN/go-color" "github.com/google/uuid" "github.com/mxcd/gitops-cli/internal/k8s" @@ -8,6 +11,7 @@ import ( "github.com/mxcd/gitops-cli/internal/secret" "github.com/mxcd/gitops-cli/internal/state" "github.com/mxcd/gitops-cli/internal/util" + "github.com/schollz/progressbar/v3" log "github.com/sirupsen/logrus" "github.com/urfave/cli/v2" k8sErrors "k8s.io/apimachinery/pkg/api/errors" @@ -19,13 +23,14 @@ func applyKubernetes(c *cli.Context) error { return err } - prettyPrintPlan(p) - + // exit if there is nothing to do if p.NothingToDo() { println(color.InGreen("No changes to apply.")) exitApplication(c, true) return nil - } + } + + prettyPrintPlan(p) if !c.Bool("auto-approve") { println("GitOps CLI will apply these changes to your Kubernetes cluster.") @@ -55,45 +60,79 @@ func applyKubernetes(c *cli.Context) error { } func planKubernetes(c *cli.Context) error { + clusterLimitString := getClusterLimit(c) + p, err := createKubernetesPlan(c) if err != nil { return err } - prettyPrintPlan(p) if p.NothingToDo() { println(color.InGreen("No changes to apply.")) } else { - println(color.InBold("use"), color.InGreen(color.InBold("gitops secrets apply kubernetes")), color.InBold("to apply these changes to your cluster")) + prettyPrintPlan(p) + dirLimitString := "" + if c.String("dir") != "" { + dirLimitString = " --dir " + c.String("dir") + } + applyString := fmt.Sprintf("gitops secrets%s apply kubernetes %s", dirLimitString, clusterLimitString) + println(color.InBold("use"), color.InGreen(color.InBold(applyString)), color.InBold("to apply these changes to your cluster")) } exitApplication(c, false) return nil } func createKubernetesPlan(c *cli.Context) (*plan.Plan, error) { - err := k8s.InitKubernetesClient(c) + log.Trace("Creating Kubernetes plan") + clusterLimit := getClusterLimit(c) + + limitPrintln := false + + if clusterLimit != "" { + println("Limiting to cluster " + color.InBlue(clusterLimit)) + limitPrintln = true + } + + dirLimit := getDirLimit(c) + if dirLimit != "" { + println("Limiting to directory " + color.InPurple(dirLimit)) + limitPrintln = true + } + + if limitPrintln { + println("") + } + + err := k8s.InitClusterClients(c) if err != nil { log.Error("Failed to init Kubernetes cluster connection") return nil, err } - connectionEstablished, err := k8s.TestClusterConnection() - if !connectionEstablished || err != nil { - log.Error("Failed to connect to Kubernetes cluster") - return nil, err - } - localSecrets, err := secret.LoadLocalSecrets(secret.SecretTargetKubernetes) + + localSecrets, err := secret.LoadLocalSecretsLimited(secret.SecretTargetTypeKubernetes, dirLimit, clusterLimit) if err != nil { - log.Error("Failed to load local secrets with target ", secret.SecretTargetKubernetes) + log.Error("Failed to load local secrets with target ", secret.SecretTargetTypeKubernetes) return nil, err } - log.Trace("Loaded ", len(localSecrets), " local secrets with target ", secret.SecretTargetKubernetes) + log.Trace("Loaded ", len(localSecrets), " local secrets with target ", secret.SecretTargetTypeKubernetes) p := &plan.Plan{ - Target: secret.SecretTargetKubernetes, + TargetType: secret.SecretTargetTypeKubernetes, Items: []plan.PlanItem{}, } + bar := progressbar.NewOptions(len(localSecrets), + progressbar.OptionEnableColorCodes(true), + progressbar.OptionShowBytes(false), + progressbar.OptionSetWidth(50), + progressbar.OptionShowCount(), + progressbar.OptionSetElapsedTime(false), + progressbar.OptionSetPredictTime(false), + progressbar.OptionSetDescription("[green][Syncing local state with cluster][reset]"), + ) + for _, localSecret := range localSecrets { + bar.Add(1) // check for local secret in state // update ID in localSecret if secret exists in state // update hash in state if secret exists in state @@ -110,7 +149,8 @@ func createKubernetesPlan(c *cli.Context) (*plan.Plan, error) { planItem := plan.PlanItem{ LocalSecret: localSecret, } - remoteSecret, err := k8s.GetSecret(localSecret) + + remoteSecret, err := k8s.GetSecret(localSecret, localSecret.Target) if err != nil { if k8sErrors.IsNotFound(err) { log.Trace("Secret ", localSecret.Name, " does not exist in Kubernetes cluster") @@ -124,6 +164,9 @@ func createKubernetesPlan(c *cli.Context) (*plan.Plan, error) { planItem.ComputeDiff() p.AddItem(planItem) } + bar.Finish() + println("") + println("") updatedStateSecrets := state.GetState().Secrets[:0] for _, stateSecret := range state.GetState().Secrets { @@ -135,8 +178,8 @@ func createKubernetesPlan(c *cli.Context) (*plan.Plan, error) { } } - // only add secret to updated state, if it still exists locally - if stateSecretFound { + // only add secret to updated state, if it still exists locally or if it is not in the scope of the current dir or cluster limits + if stateSecretFound || !strings.HasPrefix(stateSecret.Path, dirLimit) || stateSecret.Target != clusterLimit { updatedStateSecrets = append(updatedStateSecrets, stateSecret) continue } @@ -148,7 +191,7 @@ func createKubernetesPlan(c *cli.Context) (*plan.Plan, error) { Name: stateSecret.Name, Namespace: stateSecret.Namespace, Type: stateSecret.Type, - }) + }, stateSecret.Target) if err != nil { // only throw error if err is not "not found" if !k8sErrors.IsNotFound(err) { @@ -180,7 +223,28 @@ func createKubernetesPlan(c *cli.Context) (*plan.Plan, error) { return p, nil } +func getClusterLimit(c *cli.Context) string { + clusterLimit := c.Args().Get(0) + if clusterLimit != "" { + log.Trace("Limiting to cluster ", clusterLimit) + } else { + log.Trace("No cluster limit set. Applying to all clusters.") + } + return clusterLimit +} + +func getDirLimit(c *cli.Context) string { + dirLimit := c.String("dir") + if dirLimit != "" { + log.Trace("Limiting to directory ", dirLimit) + } else { + log.Trace("No dir limit set. Collecting secrets in all directories.") + } + return dirLimit +} + func prettyPrintPlan(p *plan.Plan) { + println("") println("GitOps CLI computed the following changes for your cluster:") println("-------------------------------------------------------") println("") @@ -188,4 +252,4 @@ func prettyPrintPlan(p *plan.Plan) { println("") println("-------------------------------------------------------") println("") -} \ No newline at end of file +} diff --git a/cmd/gitops/main.go b/cmd/gitops/main.go index 407dd00..a3cdda3 100644 --- a/cmd/gitops/main.go +++ b/cmd/gitops/main.go @@ -2,9 +2,11 @@ package main import ( "os" - "time" - "github.com/schollz/progressbar/v3" + "github.com/TwiN/go-color" + "github.com/mxcd/gitops-cli/internal/k8s" + "github.com/mxcd/gitops-cli/internal/state" + "github.com/mxcd/gitops-cli/internal/util" log "github.com/sirupsen/logrus" "github.com/urfave/cli/v2" ) @@ -55,6 +57,15 @@ func main() { Name: "secrets", Aliases: []string{"s"}, Usage: "GitOps managed secrets", + Flags: []cli.Flag{ + &cli.StringFlag{ + Name: "dir", + Aliases: []string{"d"}, + Value: "", + Usage: "directory to limit secret discovery to", + EnvVars: []string{"GITOPS_SECRETS_DIR"}, + }, + }, Subcommands: []*cli.Command{ { Name: "apply", @@ -113,6 +124,80 @@ func main() { }, }, }, + { + Name: "clusters", + Usage: "Managing target clusters", + Subcommands: []*cli.Command{ + { + Name: "list", + Usage: "List all target clusters", + Action: func(c *cli.Context) error { + initApplication(c) + clusters := state.GetState().GetClusters() + if len(clusters) == 0 { + println("No clusters configured") + return nil + } + for _, cluster := range clusters { + println(color.InBlue(cluster.Name), " => ", cluster.ConfigFile) + } + return exitApplication(c, true) + }, + }, + { + Name: "add", + Usage: "Add a target cluster. ", + Action: func(c *cli.Context) error { + initApplication(c) + kubeconfig := "" + if c.Args().Len() == 1 { + println("Please enter location of kubeconfig file for new ", color.InBlue(c.Args().Get(0)), " cluster: ") + kubeconfig = util.StringPrompt("kubeconfig file: ") + } else if c.Args().Len() == 2 { + kubeconfig = c.Args().Get(1) + } else { + log.Fatal("Usage: gitops clusters add ") + } + err := state.GetState().AddCluster(&state.ClusterState{ + Name: c.Args().Get(0), + ConfigFile: kubeconfig, + }) + if err != nil { + return err + } + return exitApplication(c, true) + }, + }, + { + Name: "remove", + Usage: "Remove a target cluster", + Action: func(c *cli.Context) error { + initApplication(c) + if c.Args().Len() != 1 { + log.Fatal("Usage: gitops clusters remove ") + } + err := state.GetState().RemoveCluster(c.Args().Get(0)) + if err != nil { + return err + } + return exitApplication(c, true) + }, + }, + { + Name: "test", + Usage: "Test a target cluster connection", + Action: func(c *cli.Context) error { + initApplication(c) + k8s.InitClusterClients(c) + clusterClients := k8s.GetClients() + for _, clusterClient := range clusterClients { + clusterClient.PrettyPrint() + } + return exitApplication(c, false) + }, + }, + }, + }, }, } @@ -121,23 +206,3 @@ func main() { log.Fatal(err) } } - -func barTest() { - bar := progressbar.NewOptions(100, - progressbar.OptionEnableColorCodes(true), - progressbar.OptionShowBytes(false), - progressbar.OptionSetWidth(30), - progressbar.OptionSetDescription("[cyan][1/3][reset] Writing moshable file..."), - /*progressbar.OptionSetTheme(progressbar.Theme{ - Saucer: "[green]=[reset]", - SaucerHead: "[green]>[reset]", - SaucerPadding: " ", - BarStart: "[", - BarEnd: "]", - })*/) - for i := 0; i < 100; i++ { - bar.Add(1) - time.Sleep(40 * time.Millisecond) - } -} - diff --git a/internal/k8s/clients.go b/internal/k8s/clients.go new file mode 100644 index 0000000..805cc18 --- /dev/null +++ b/internal/k8s/clients.go @@ -0,0 +1,173 @@ +package k8s + +import ( + "fmt" + "os" + "regexp" + + "github.com/TwiN/go-color" + "github.com/mxcd/gitops-cli/internal/state" + "github.com/mxcd/gitops-cli/internal/util" + log "github.com/sirupsen/logrus" + "github.com/urfave/cli/v2" + "k8s.io/client-go/kubernetes" + "k8s.io/client-go/tools/clientcmd" +) + +type ClusterClient struct { + Name string + Clientset *kubernetes.Clientset + Connected bool + ClusterVersion string +} + +var clusterClients map[string] *ClusterClient = make(map[string] *ClusterClient) + +func InitClusterClients(c *cli.Context) error { + log.Trace("Initializing cluster clients") + + stateClusters := state.GetState().GetClusters() + err := InitDefaultClusterClient(c) + if err != nil && len(stateClusters) == 0 { + log.Error("No default cluster config available") + return err + } + log.Trace("Found clusters in state. Initializing cluster clients") + for _, cluster := range stateClusters { + log.Trace("Initializing cluster client for cluster: ", cluster.Name) + AddClient(cluster.Name, cluster.ConfigFile) + } + return nil +} + +func InitDefaultClusterClient(c *cli.Context) error { + kubeconfig := c.String("kubeconfig") + if kubeconfig == "" { + log.Trace("No KUBECONFIG given. Testing default locations") + // check if file exists + _, err := os.Stat(clientcmd.RecommendedHomeFile) + if os.IsNotExist(err) { + log.Trace("No KUBECONFIG found in default locations. Attempting in-cluster config") + } else if err == nil { + log.Trace("Found KUBECONFIG in default location: ", clientcmd.RecommendedHomeFile) + kubeconfig = clientcmd.RecommendedHomeFile + } + } + + log.Trace("Building Kubernetes client config from KUBECONFIG: ", kubeconfig) + config, err := clientcmd.BuildConfigFromFlags("", kubeconfig) + if err != nil { + log.Warn("No default cluster config available") + return nil + } + + log.Trace("Creating Kubernetes clientset") + clientset, err := kubernetes.NewForConfig(config) + if err != nil { + return err + } + + clusterClient := &ClusterClient{ + Name: string(util.DefaultClusterClient), + Clientset: clientset, + Connected: false, + } + clusterClient.TestConnection() + clusterClients[string(util.DefaultClusterClient)] = clusterClient + + return nil +} + +func GetClient(clusterName string) (*ClusterClient, error) { + if clusterClients[clusterName] == nil { + return nil, fmt.Errorf("client '%s' not found", clusterName) + } + return clusterClients[clusterName], nil +} + +func GetClients() map[string] *ClusterClient { + return clusterClients +} + +func AddClient(clusterName string, kubeconfigFile string) error { + log.Trace("Building Kubernetes client config from KUBECONFIG: ", kubeconfigFile) + // check if the file exists + _, err := os.Stat(kubeconfigFile) + if os.IsNotExist(err) { + log.Error("KUBECONFIG file not found: ", kubeconfigFile) + return err + } + + secretKubeconfigRegex, err := regexp.Compile(`.*\.kubeconfig\.secret\.enc\.ya?ml$`) + if err != nil { + log.Fatal(err) + } + + var kubeconfigFileData []byte = []byte{} + + if secretKubeconfigRegex.MatchString(kubeconfigFile) { + log.Trace("KUBECONFIG file is a secret. Decrypting") + kubeconfigFileData, err = util.DecryptFile(kubeconfigFile) + if err != nil { + log.Error("Failed to decrypt KUBECONFIG file: ", err) + return err + } + } else { + log.Trace("KUBECONFIG file is not a secret. Reading") + kubeconfigFileData, err = os.ReadFile(kubeconfigFile) + if err != nil { + log.Error("Failed to read KUBECONFIG file: ", err) + return err + } + } + + clientConfig, err := clientcmd.NewClientConfigFromBytes(kubeconfigFileData) + if err != nil { + log.Error("Failed to build Kubernetes client config: ", err) + return err + } + restConfig, err := clientConfig.ClientConfig() + if err != nil { + log.Error("Failed to build Kubernetes client config: ", err) + return err + } + + log.Trace("Creating Kubernetes clientset") + clientset, err := kubernetes.NewForConfig(restConfig) + if err != nil { + return err + } + + clusterClient := &ClusterClient{ + Name: clusterName, + Clientset: clientset, + Connected: false, + } + clusterClient.TestConnection() + clusterClients[clusterName] = clusterClient + return nil +} + +func (c *ClusterClient) TestConnection() (bool, error) { + log.Trace("Testing Kubernetes cluster connection") + serverVersion, err := c.Clientset.Discovery().ServerVersion() + if err != nil { + log.Warn("Failed to connect to Kubernetes cluster ", color.InBlue(c.Name), ": ", err) + c.Connected = false + return false, err + } + log.Debug("Connected to Kubernetes cluster: ", serverVersion) + c.Connected = true + c.ClusterVersion = serverVersion.String() + return true, nil +} + +func (c *ClusterClient) PrettyPrint() { + clusterInfoLine := fmt.Sprintf("Cluster: %s (%s)\t", color.InBlue(c.Name), color.InGray(c.ClusterVersion)) + if c.Connected { + clusterInfoLine = fmt.Sprintf("%sConnected: %s", clusterInfoLine, color.InGreen("true")) + } else { + clusterInfoLine = fmt.Sprintf("%sConnected: %s", clusterInfoLine, color.InRed("false")) + } + println(clusterInfoLine) +} \ No newline at end of file diff --git a/internal/k8s/kubernetes.go b/internal/k8s/kubernetes.go index fa65362..a885f86 100644 --- a/internal/k8s/kubernetes.go +++ b/internal/k8s/kubernetes.go @@ -2,71 +2,30 @@ package k8s import ( "context" - "os" "github.com/TwiN/go-color" "github.com/mxcd/gitops-cli/internal/secret" log "github.com/sirupsen/logrus" - "github.com/urfave/cli/v2" v1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "k8s.io/client-go/kubernetes" - "k8s.io/client-go/tools/clientcmd" ) -var clientset *kubernetes.Clientset - -func InitKubernetesClient(c *cli.Context) error { - kubeconfig := c.String("kubeconfig") - if kubeconfig == "" { - log.Trace("No KUBECONFIG given. Testing default locations") - // check if file exists - _, err := os.Stat(clientcmd.RecommendedHomeFile) - if os.IsNotExist(err) { - log.Trace("No KUBECONFIG found in default locations. Attempting in-cluster config") - } else if err == nil { - log.Trace("Found KUBECONFIG in default location: ", clientcmd.RecommendedHomeFile) - kubeconfig = clientcmd.RecommendedHomeFile - } - } - - log.Trace("Building Kubernetes client config from KUBECONFIG: ", kubeconfig) - config, err := clientcmd.BuildConfigFromFlags("", kubeconfig) - if err != nil { - return err - } - - log.Trace("Creating Kubernetes clientset") - clientset, err = kubernetes.NewForConfig(config) - if err != nil { - return err +func CreateSecret(s *secret.Secret, clusterName string) error { + if s.Type == "ConfigMap" { + return CreateK8sConfigMap(s, clusterName) + } else { + return CreateK8sSecret(s, clusterName) } - - return nil } -func TestClusterConnection() (bool, error) { - log.Trace("Testing Kubernetes cluster connection") - serverVersion, err := clientset.Discovery().ServerVersion() +func CreateK8sConfigMap(s *secret.Secret, clusterName string) error { + clusterClient, err := GetClient(clusterName) if err != nil { - log.Error("Failed to connect to Kubernetes cluster: ", err) - return false, err - } - log.Debug("Connected to Kubernetes cluster: ", serverVersion) - - return true, nil -} - -func CreateSecret(s *secret.Secret) error { - if s.Type == "ConfigMap" { - return CreateK8sConfigMap(s) - } else { - return CreateK8sSecret(s) + return err } -} + clientset := clusterClient.Clientset -func CreateK8sConfigMap(s *secret.Secret) error { log.Trace("Creating ConfigMap ", s.Name, " in namespace ", s.Namespace) k8sConfigMap, err := clientset.CoreV1().ConfigMaps(s.Namespace).Create(context.Background(), &v1.ConfigMap{ ObjectMeta: metav1.ObjectMeta{ @@ -86,7 +45,13 @@ func CreateK8sConfigMap(s *secret.Secret) error { return err } -func CreateK8sSecret(s *secret.Secret) error { +func CreateK8sSecret(s *secret.Secret, clusterName string) error { + clusterClient, err := GetClient(clusterName) + if err != nil { + return err + } + clientset := clusterClient.Clientset + log.Trace("Creating Secret ", s.Name, " in namespace ", s.Namespace) k8sSecret, err := clientset.CoreV1().Secrets(s.Namespace).Create(context.Background(), &v1.Secret{ ObjectMeta: metav1.ObjectMeta{ @@ -107,17 +72,22 @@ func CreateK8sSecret(s *secret.Secret) error { return err } -func UpdateSecret(s *secret.Secret) error { +func UpdateSecret(s *secret.Secret, clusterName string) error { if s.Type == "ConfigMap" { - return UpdateK8sConfigMap(s) + return UpdateK8sConfigMap(s, clusterName) } else { - return UpdateK8sSecret(s) + return UpdateK8sSecret(s, clusterName) } } -func UpdateK8sSecret(s *secret.Secret) error { - log.Trace("Updating Secret ", s.Name, " in namespace ", s.Namespace) +func UpdateK8sSecret(s *secret.Secret, clusterName string) error { + clusterClient, err := GetClient(clusterName) + if err != nil { + return err + } + clientset := clusterClient.Clientset + log.Trace("Updating Secret ", s.Name, " in namespace ", s.Namespace) k8sSecret, err := clientset.CoreV1().Secrets(s.Namespace).Update(context.Background(), &v1.Secret{ ObjectMeta: metav1.ObjectMeta{ Name: s.Name, @@ -137,7 +107,12 @@ func UpdateK8sSecret(s *secret.Secret) error { return err } -func UpdateK8sConfigMap(s *secret.Secret) error { +func UpdateK8sConfigMap(s *secret.Secret, clusterName string) error { + clusterClient, err := GetClient(clusterName) + if err != nil { + return err + } + clientset := clusterClient.Clientset log.Trace("Updating ConfigMap ", s.Name, " in namespace ", s.Namespace) k8sConfigMap, err := clientset.CoreV1().ConfigMaps(s.Namespace).Update(context.Background(), &v1.ConfigMap{ @@ -158,17 +133,22 @@ func UpdateK8sConfigMap(s *secret.Secret) error { return err } -func DeleteSecret(s *secret.Secret) error { +func DeleteSecret(s *secret.Secret, clusterName string) error { if s.Type == "ConfigMap" { - return DeleteK8sConfigMap(s) + return DeleteK8sConfigMap(s, clusterName) } else { - return DeleteK8sSecret(s) + return DeleteK8sSecret(s, clusterName) } } -func DeleteK8sConfigMap(s *secret.Secret) error { +func DeleteK8sConfigMap(s *secret.Secret, clusterName string) error { + clusterClient, err := GetClient(clusterName) + if err != nil { + return err + } + clientset := clusterClient.Clientset log.Trace("Deleting ConfigMap ", s.Name, " in namespace ", s.Namespace) - err := clientset.CoreV1().ConfigMaps(s.Namespace).Delete(context.Background(), s.Name, metav1.DeleteOptions{}) + err = clientset.CoreV1().ConfigMaps(s.Namespace).Delete(context.Background(), s.Name, metav1.DeleteOptions{}) if err != nil { return err } @@ -176,9 +156,14 @@ func DeleteK8sConfigMap(s *secret.Secret) error { return err } -func DeleteK8sSecret(s *secret.Secret) error { +func DeleteK8sSecret(s *secret.Secret, clusterName string) error { + clusterClient, err := GetClient(clusterName) + if err != nil { + return err + } + clientset := clusterClient.Clientset log.Trace("Deleting Secret ", s.Name, " in namespace ", s.Namespace) - err := clientset.CoreV1().Secrets(s.Namespace).Delete(context.Background(), s.Name, metav1.DeleteOptions{}) + err = clientset.CoreV1().Secrets(s.Namespace).Delete(context.Background(), s.Name, metav1.DeleteOptions{}) if err != nil { return err } @@ -186,15 +171,20 @@ func DeleteK8sSecret(s *secret.Secret) error { return err } -func GetSecret(s *secret.Secret) (*secret.Secret, error) { +func GetSecret(s *secret.Secret, clusterName string) (*secret.Secret, error) { if s.Type == "ConfigMap" { - return GetK8sConfigMap(s) + return GetK8sConfigMap(s, clusterName) } else { - return GetK8sSecret(s) + return GetK8sSecret(s, clusterName) } } -func GetK8sConfigMap(s *secret.Secret) (*secret.Secret, error) { +func GetK8sConfigMap(s *secret.Secret, clusterName string) (*secret.Secret, error) { + clusterClient, err := GetClient(clusterName) + if err != nil { + return nil, err + } + clientset := clusterClient.Clientset k8sConfigMap, err := clientset.CoreV1().ConfigMaps(s.Namespace).Get(context.Background(), s.Name, metav1.GetOptions{}) if err != nil { @@ -203,14 +193,20 @@ func GetK8sConfigMap(s *secret.Secret) (*secret.Secret, error) { return &secret.Secret{ Name: k8sConfigMap.Name, - Target: secret.SecretTargetKubernetes, + TargetType: secret.SecretTargetTypeKubernetes, + Target: clusterName, Namespace: k8sConfigMap.Namespace, Data: k8sConfigMap.Data, Type: "ConfigMap", }, nil } -func GetK8sSecret(s *secret.Secret) (*secret.Secret, error) { +func GetK8sSecret(s *secret.Secret, clusterName string) (*secret.Secret, error) { + clusterClient, err := GetClient(clusterName) + if err != nil { + return nil, err + } + clientset := clusterClient.Clientset k8sSecret, err := clientset.CoreV1().Secrets(s.Namespace).Get(context.Background(), s.Name, metav1.GetOptions{}) if err != nil { @@ -219,7 +215,8 @@ func GetK8sSecret(s *secret.Secret) (*secret.Secret, error) { return &secret.Secret{ Name: k8sSecret.Name, - Target: secret.SecretTargetKubernetes, + TargetType: secret.SecretTargetTypeKubernetes, + Target: clusterName, Namespace: k8sSecret.Namespace, Data: getStringData(k8sSecret), Type: string(k8sSecret.Type), diff --git a/internal/plan/plan.go b/internal/plan/plan.go index 3dd5cc9..66d7b3f 100644 --- a/internal/plan/plan.go +++ b/internal/plan/plan.go @@ -11,7 +11,7 @@ type Plan struct { // List of items in the plan Items []PlanItem // Target type of the plan - Target secret.SecretTarget + TargetType secret.SecretTargetType } type PlanItem struct { @@ -50,10 +50,10 @@ func (p *Plan) Print() { } func (p *Plan) Execute() error { - if p.Target == secret.SecretTargetKubernetes { + if p.TargetType == secret.SecretTargetTypeKubernetes { return executeKubernetesPlan(p) - } else if p.Target == secret.SecretTargetVault { + } else if p.TargetType == secret.SecretTargetTypeVault { log.Fatal("Not implemented") return nil } @@ -68,21 +68,21 @@ func executeKubernetesPlan(p *Plan) error { } if item.Diff.Type == secret.SecretDiffTypeAdded { log.Trace("Secret ", item.LocalSecret.Namespace, "/", item.LocalSecret.Name, " is new, creating...") - err := k8s.CreateSecret(item.LocalSecret) + err := k8s.CreateSecret(item.LocalSecret, item.LocalSecret.Target) if err != nil { log.Error("Failed to create secret ", item.LocalSecret.Namespace, "/", item.LocalSecret.Name, " in cluster") return err } } else if item.Diff.Type == secret.SecretDiffTypeChanged { log.Trace("Secret ", item.LocalSecret.Namespace, "/", item.LocalSecret.Name, " is modified, updating...") - err := k8s.UpdateSecret(item.LocalSecret) + err := k8s.UpdateSecret(item.LocalSecret, item.LocalSecret.Target) if err != nil { log.Error("Failed to update secret ", item.LocalSecret.Namespace, "/", item.LocalSecret.Name, " in cluster") return err } } else if item.Diff.Type == secret.SecretDiffTypeRemoved { log.Trace("Secret ", item.RemoteSecret.Namespace, "/", item.RemoteSecret.Name, " is deleted, deleting...") - err := k8s.DeleteSecret(item.RemoteSecret) + err := k8s.DeleteSecret(item.RemoteSecret, item.RemoteSecret.Target) if err != nil { log.Error("Failed to delete secret ", item.RemoteSecret.Namespace, "/", item.RemoteSecret.Name, " in cluster") return err diff --git a/internal/secret/diff.go b/internal/secret/diff.go index 1b6e869..ae5d9d8 100644 --- a/internal/secret/diff.go +++ b/internal/secret/diff.go @@ -90,6 +90,15 @@ func CompareSecrets(oldSecret *Secret, newSecret *Secret) *SecretDiff { } } + if oldSecret.TargetType != newSecret.TargetType { + diffEntries = append(diffEntries, SecretDiffEntry{ + Type: SecretDiffTypeChanged, + Key: "targetType", + OldValue: string(oldSecret.TargetType), + NewValue: string(newSecret.TargetType), + Sensitive: false, + }) + } if oldSecret.Target != newSecret.Target { diffEntries = append(diffEntries, SecretDiffEntry{ diff --git a/internal/secret/loader.go b/internal/secret/loader.go index 868dec6..f53f7fb 100644 --- a/internal/secret/loader.go +++ b/internal/secret/loader.go @@ -4,6 +4,7 @@ import ( "strings" "github.com/mxcd/gitops-cli/internal/util" + "github.com/schollz/progressbar/v3" log "github.com/sirupsen/logrus" ) @@ -12,24 +13,61 @@ import ( Applies the specified target filter Use SecretTargetAll to load all secrets */ -func LoadLocalSecrets(targetFilter SecretTarget) ([]*Secret, error) { +func LoadLocalSecrets(targetTypeFilter SecretTargetType) ([]*Secret, error) { + return LoadLocalSecretsLimited(targetTypeFilter, "", "") +} + +func LoadLocalSecretsLimited(targetTypeFilter SecretTargetType, directoryLimit string, clusterLimit string) ([]*Secret, error) { + // retrieve all secret files secretFileNames, err := util.GetSecretFiles() if err != nil { return nil, err } - secrets := []*Secret{} + + // Filter by directory limit + filteredFileNames := []string{} for _, secretFileName := range secretFileNames { + if !strings.HasPrefix(secretFileName, directoryLimit) { + log.Trace("Skipping file due to directory filter: ", secretFileName) + continue + } if strings.HasSuffix(secretFileName, "values.gitops.secret.enc.yml") || strings.HasSuffix(secretFileName, "values.gitops.secret.enc.yaml") { log.Trace("Skipping values file: ", secretFileName) continue } + filteredFileNames = append(filteredFileNames, secretFileName) + } + secretFileNames = filteredFileNames + + + secrets := []*Secret{} + bar := progressbar.NewOptions(len(secretFileNames), + progressbar.OptionEnableColorCodes(true), + progressbar.OptionShowBytes(false), + progressbar.OptionSetWidth(50), + progressbar.OptionShowCount(), + progressbar.OptionSetElapsedTime(false), + progressbar.OptionSetPredictTime(false), + progressbar.OptionSetDescription("[green][Loading local secrets][reset]"), + ) + for _, secretFileName := range secretFileNames { + bar.Add(1) secret, err := FromPath(secretFileName) if err != nil { return nil, err } - if secret.Target == targetFilter || targetFilter == SecretTargetAll { - secrets = append(secrets, secret) + if secret.TargetType != targetTypeFilter && targetTypeFilter != SecretTargetTypeAll { + log.Trace("Skipping file due to targetType filter: ", secretFileName) + continue + } + if clusterLimit != "" && secret.Target != clusterLimit { + log.Trace("Skipping file due to target filter: ", secretFileName) + continue } + secrets = append(secrets, secret) } + bar.Finish() + println("") + println("") return secrets, nil } \ No newline at end of file diff --git a/internal/secret/secret.go b/internal/secret/secret.go index 11a2311..db896a7 100644 --- a/internal/secret/secret.go +++ b/internal/secret/secret.go @@ -21,8 +21,11 @@ type Secret struct { // Path is the path to the secret file Path string - // Target is the target of the secret - Target SecretTarget + // TargetType is the target of the secret + TargetType SecretTargetType + + // Target is the target system (cluster of vault instance) + Target string // Type is the type of the secret Type string @@ -43,13 +46,14 @@ type Secret struct { Data map[string]string } -type SecretTarget string -var SecretTargetVault SecretTarget = "vault" -var SecretTargetKubernetes SecretTarget = "k8s" -var SecretTargetAll SecretTarget = "all" +type SecretTargetType string +var SecretTargetTypeVault SecretTargetType = "vault" +var SecretTargetTypeKubernetes SecretTargetType = "k8s" +var SecretTargetTypeAll SecretTargetType = "all" type SecretFile struct { - Target SecretTarget `yaml:"target"` + TargetType SecretTargetType `yaml:"targetType"` + Target string `yaml:"target"` Name string `yaml:"name,omitempty"` Namespace string `yaml:"namespace" default:"default"` Type string `yaml:"type" default:"Opaque"` @@ -98,8 +102,14 @@ func (s *Secret) Load() error { var secretFile SecretFile yaml.UnmarshalStrict(s.BinaryData, &secretFile) - s.Target = secretFile.Target + s.TargetType = secretFile.TargetType + if secretFile.Target != "" { + s.Target = secretFile.Target + } else { + s.Target = string(util.DefaultClusterClient) + } + if secretFile.Name != "" { s.Name = secretFile.Name } else { @@ -136,6 +146,7 @@ func (s *Secret) PrettyPrint() { cleartext := util.GetCliContext().Bool("cleartext") println("---") println(s.CombinedName()) + println(" targetType: " + string(s.TargetType)) println(" target: " + string(s.Target)) println(" type: " + s.Type) println(" data:") diff --git a/internal/secret/secret_test.go b/internal/secret/secret_test.go index 51d9a06..9aa0e62 100644 --- a/internal/secret/secret_test.go +++ b/internal/secret/secret_test.go @@ -18,10 +18,10 @@ func TestLoadSecret1(t *testing.T) { t.Error(err) } - assert.Equal(t, secret.Target, SecretTargetVault, "Target should be vault") - assert.Equal(t, secret.Name, "my-explicitly-named-secret", "Name should be my-explicitly-named-secret") - assert.Equal(t, secret.Namespace, "default", "Namespace should be default") - assert.Equal(t, secret.Type, "Opaque", "Type should be Opaque") + assert.Equal(t, SecretTargetTypeVault, secret.TargetType, "Target should be vault") + assert.Equal(t, "my-explicitly-named-secret", secret.Name, "Name should be my-explicitly-named-secret") + assert.Equal(t, "default", secret.Namespace, "Namespace should be default") + assert.Equal(t, "Opaque", secret.Type, "Type should be Opaque") t.Log(secret) } @@ -37,7 +37,7 @@ func TestLoadSecret2(t *testing.T) { t.Error(err) } - assert.Equal(t, SecretTargetKubernetes, secret.Target, "Target should be k8s") + assert.Equal(t, SecretTargetTypeKubernetes, secret.TargetType, "Target should be k8s") assert.Equal(t, "implicit-name", secret.Name, "Name should be implicit-name") assert.Equal(t, "default", secret.Namespace, "Namespace should be default") assert.Equal(t, "kubernetes.io/dockerconfigjson", secret.Type, "Type should be dockerconfigjson") @@ -48,7 +48,7 @@ func TestSecretComparisonTarget(t *testing.T) { Name: "myName", Namespace: "myNamespace", Type: "Opaque", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string {}, } @@ -56,7 +56,7 @@ func TestSecretComparisonTarget(t *testing.T) { Name: "myName", Namespace: "myNamespace", Type: "Opaque", - Target: SecretTargetVault, + TargetType: SecretTargetTypeVault, Data: map[string]string {}, } @@ -65,8 +65,8 @@ func TestSecretComparisonTarget(t *testing.T) { assert.Equal(t, false, diff.Equal, "Secrets should not be equal") assert.Equal(t, SecretDiffTypeChanged, diff.Type, "Diff type should be changed") assert.Equal(t, 1, len(diff.Entries), "Diff should have 1 entry") - entry := diff.GetEntry("target") - assert.NotNil(t, entry, "Diff should have an entry for target") + entry := diff.GetEntry("targetType") + assert.NotNil(t, entry, "Diff should have an entry for targetType") assert.Equal(t, SecretDiffTypeChanged, entry.Type, "DiffEntry type should be changed") } @@ -75,7 +75,7 @@ func TestSecretComparisonType(t *testing.T) { Name: "myName", Namespace: "myNamespace", Type: "Opaque", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string {}, } @@ -83,7 +83,7 @@ func TestSecretComparisonType(t *testing.T) { Name: "myName", Namespace: "myNamespace", Type: ".dockerconfigjson", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string {}, } @@ -102,7 +102,7 @@ func TestSecretComparisonChangedName(t *testing.T) { Name: "myName", Namespace: "myNamespace", Type: "Opaque", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string {}, } @@ -110,7 +110,7 @@ func TestSecretComparisonChangedName(t *testing.T) { Name: "myNameExtended", Namespace: "myNamespace", Type: "Opaque", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string {}, } @@ -130,7 +130,7 @@ func TestSecretComparisonChangedNamespace(t *testing.T) { Name: "myName", Namespace: "myNamespace", Type: "Opaque", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string {}, } @@ -138,7 +138,7 @@ func TestSecretComparisonChangedNamespace(t *testing.T) { Name: "myName", Namespace: "myNamespaceExtended", Type: "Opaque", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string {}, } @@ -158,7 +158,7 @@ func TestSecretComparisonAddData(t *testing.T) { Name: "myName", Namespace: "myNamespace", Type: "Opaque", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string {}, } @@ -166,7 +166,7 @@ func TestSecretComparisonAddData(t *testing.T) { Name: "myName", Namespace: "myNamespace", Type: "Opaque", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string { "key1": "value1", "key2": "value2", @@ -193,7 +193,7 @@ func TestSecretComparisonRemoveData(t *testing.T) { Name: "myName", Namespace: "myNamespace", Type: "Opaque", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string { "key1": "value1", "key2": "value2", @@ -204,7 +204,7 @@ func TestSecretComparisonRemoveData(t *testing.T) { Name: "myName", Namespace: "myNamespace", Type: "Opaque", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string {}, } @@ -228,7 +228,7 @@ func TestSecretComparisonChangeData1(t *testing.T) { Name: "myName", Namespace: "myNamespace", Type: "Opaque", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string { "key1": "value1", "key2": "value2", @@ -239,7 +239,7 @@ func TestSecretComparisonChangeData1(t *testing.T) { Name: "myName", Namespace: "myNamespace", Type: "Opaque", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string { "key1": "newValue1", "key2": "value2", @@ -262,7 +262,7 @@ func TestSecretComparisonChangeData2(t *testing.T) { Name: "myName", Namespace: "myNamespace", Type: "Opaque", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string { "key2": "value2", "key1": "value1", @@ -273,7 +273,7 @@ func TestSecretComparisonChangeData2(t *testing.T) { Name: "myName", Namespace: "myNamespace", Type: "Opaque", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string { "key1": "newValue1", "key2": "newValue2", @@ -300,7 +300,7 @@ func TestSecretComparisonChangeData3(t *testing.T) { Name: "myName", Namespace: "myNamespace", Type: "Opaque", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string { "key1": "value1", "key2": "value2", @@ -311,7 +311,7 @@ func TestSecretComparisonChangeData3(t *testing.T) { Name: "myName", Namespace: "myNamespace", Type: "Opaque", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string { "key1": "newValue1", "key3": "value3", @@ -342,7 +342,7 @@ func TestSecretComparisonRemoveSecret(t *testing.T) { Name: "myName", Namespace: "myNamespace", Type: "Opaque", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string { "key1": "value1", "key2": "value2", @@ -369,7 +369,7 @@ func TestSecretComparisonAddSecret(t *testing.T) { Name: "myName", Namespace: "myNamespace", Type: "Opaque", - Target: SecretTargetKubernetes, + TargetType: SecretTargetTypeKubernetes, Data: map[string]string { "key1": "value1", "key2": "value2", diff --git a/internal/state/state.go b/internal/state/state.go index b68a655..8a2007d 100644 --- a/internal/state/state.go +++ b/internal/state/state.go @@ -6,23 +6,29 @@ import ( "os" "path" + "github.com/TwiN/go-color" + "github.com/andybalholm/crlf" "github.com/mxcd/gitops-cli/internal/secret" "github.com/mxcd/gitops-cli/internal/util" + log "github.com/sirupsen/logrus" "github.com/urfave/cli/v2" "gopkg.in/yaml.v2" - "github.com/andybalholm/crlf" ) type State struct { // List of secrets in the state Secrets []*SecretState + // Map of clusters known to the state + Clusters map[string]*ClusterState } type SecretState struct { // unique uuid of the secret ID string // Target is the target of the secret - Target secret.SecretTarget + TargetType secret.SecretTargetType + // Target is the target system (cluster of vault instance) + Target string // Name of the secret Name string // Namespace of the secret @@ -35,6 +41,13 @@ type SecretState struct { BinaryDataHash string } +type ClusterState struct { + // Name of the cluster + Name string + // Kubeconfig file of the cluster + ConfigFile string +} + var state *State func LoadState(c *cli.Context) error { @@ -46,6 +59,7 @@ func LoadState(c *cli.Context) error { // Create new state state = &State{ Secrets: []*SecretState{}, + Clusters: map[string]*ClusterState{}, } return nil } else { @@ -109,6 +123,7 @@ func (s *State) GetByPath(path string) *SecretState { func (s *State) Add(secret *secret.Secret) *SecretState { stateSecret := &SecretState{ ID: secret.ID, + TargetType: secret.TargetType, Target: secret.Target, Path: secret.Path, BinaryDataHash: secret.BinaryDataHash, @@ -123,6 +138,7 @@ func (s *State) Add(secret *secret.Secret) *SecretState { // TODO prohibit update of the secret type func (s *SecretState) Update(secret *secret.Secret) { secret.ID = s.ID + s.TargetType = secret.TargetType s.Target = secret.Target s.Name = secret.Name s.Namespace = secret.Namespace @@ -140,4 +156,61 @@ func (s *State) SetSecrets(secrets []*SecretState) { func GetState() *State { return state -} \ No newline at end of file +} + +type ClusterExistsError struct{} +func (m *ClusterExistsError) Error() string { + return "Cluster already exists" +} + +type ClusterNotFoundError struct{} +func (m *ClusterNotFoundError) Error() string { + return "Cluster could not be found" +} + +type ClusterNameReservedError struct{} +func (m *ClusterNameReservedError) Error() string { + return "The given cluster name is reserved" +} + +func (s *State) GetCluster(name string) (*ClusterState, error) { + if s.Clusters[name] == nil { + log.Error("Cluster " + color.InBlue(name) + " not defined in state") + return nil, &ClusterNotFoundError{} + } + return s.Clusters[name], nil +} + +func (s *State) AddCluster(cluster *ClusterState) error { + if cluster.Name == string(util.DefaultClusterClient) { + log.Error("Cluster name " + color.InBlue(cluster.Name) + " is reserved") + return &ClusterNameReservedError{} + } + if s.Clusters == nil { + s.Clusters = map[string]*ClusterState{} + } + if s.Clusters[cluster.Name] != nil { + log.Error("Cluster " + color.InBlue(cluster.Name) + " already defined in state") + return &ClusterExistsError{} + } + s.Clusters[cluster.Name] = cluster + println(color.InGreen("Added cluster "), color.InBlue(cluster.Name)) + return nil +} + +func (s *State) GetClusters() map[string]*ClusterState { + if s.Clusters == nil { + s.Clusters = map[string]*ClusterState{} + } + return s.Clusters +} + +func (s *State) RemoveCluster(name string) error { + if s.Clusters[name] == nil { + log.Error("Cluster " + color.InBlue(name) + " not defined in state") + return &ClusterNotFoundError{} + } + delete(s.Clusters, name) + println(color.InRed("Removed cluster "), color.InBlue(name)) + return nil +} diff --git a/internal/util/util.go b/internal/util/util.go index 17cf10e..df2fa7c 100644 --- a/internal/util/util.go +++ b/internal/util/util.go @@ -16,6 +16,11 @@ import ( "go.mozilla.org/sops/v3/decrypt" ) +type ClusterClientName string +const ( + DefaultClusterClient ClusterClientName = "__default" +) + // go over all files in the current directory (recursively) // and find all files that end with .secret.enc.yaml or .secret.enc.yml // return a list of these files diff --git a/test_assets/config-map.gitops.secret.enc.yml b/test_assets/config-map.gitops.secret.enc.yml index 871867b..ee1a0e9 100644 --- a/test_assets/config-map.gitops.secret.enc.yml +++ b/test_assets/config-map.gitops.secret.enc.yml @@ -1,4 +1,4 @@ -target: ENC[AES256_GCM,data:TnzF,iv:8RvsNPJdWMWDcmYiUGqeIjv1i4xzx81kMrxoUZdPQN0=,tag:iwsUoJDBPdvYFR2jTrgM1g==,type:str] +targetType: ENC[AES256_GCM,data:EBcw,iv:15E2bA+UjadooqJ5IZKR8NfU4RSgH4Z82CAmyd5BPOA=,tag:iKpdEQyq1myM5H5/8qcdqA==,type:str] type: ENC[AES256_GCM,data:Ixo2bmAJ9dxQ,iv:tSpGTtUfl30Vc1UducwZ9RCluMqgftBE11HKz73qWvE=,tag:m9XEsvq1fxB3efJWY5Rq7A==,type:str] name: ENC[AES256_GCM,data:lCy3q0v+N8tV6Z6Hgg==,iv:1wu8YbEWo/aWWYNDe71IR253+FaeUoW/hkAhXXh0vRQ=,tag:VrVVSpt6laysulM62MdMJQ==,type:str] namespace: ENC[AES256_GCM,data:zu7mVhFv0Q==,iv:jALmIjBkgsD44TRKfkXd2H9w4D8xh+McU9hTIaCA7lQ=,tag:x4OX4rvgen4o0+dt824ZJA==,type:str] @@ -20,8 +20,8 @@ sops: cEs5TzdxVEM3TGoySE4yVnUvR1FqaGsKK4PFub3b84AKbcxKB5Wf/SPQPFiSJec3 CbpGb0A9lmok0a4owoCIAhRA+UaUplpTp4IkL40xVgPYwvvnXfl4Jw== -----END AGE ENCRYPTED FILE----- - lastmodified: "2023-04-05T11:28:22Z" - mac: ENC[AES256_GCM,data:I3w0upkHYHYOSZP/cV1uKnNAjb3uHrE+EzCQ/YHw95Rk59T+chg1UN2n/nCpRwUZ85a0bAYDpIpI9IA3r0g7uu//+8Li0jv38oofCN4g0bkMJtHEbSGcLvbjzzK+hNdz8Qr8BjiWN4uvi9cXv3bFczh0gkRh4f3AGotB392C3G4=,iv:Se7j3Pj11DT9/qJ2USdkCrAUPHi9YEN9/z3VSNV/sbQ=,tag:uEP4vxDcGWi6Qne+Bw6mqA==,type:str] + lastmodified: "2023-04-27T13:13:18Z" + mac: ENC[AES256_GCM,data:1Pda0G7boRt4zRgsw33Vzc+ef42G+DJfmde+wTT3ITiAI+OzJj+Xo+8JZqu6kJ7WcvLC1z+xwpDq90uedua8x8bE/+jyXEw2rFJgCL9C1j+gPCwxfYz4GQrF58t2jEm3dFPUbdIUWzk2ArV13I55xjiDxzHBeGqvzQ//wD43Pi0=,iv:dV9xCsPsHVaoCRjmoYEEkJUTB97bKEPa0rces4xcoug=,tag:5SjEJ5HbMuhWJGPLi3kXFQ==,type:str] pgp: [] unencrypted_suffix: _unencrypted version: 3.7.3 diff --git a/test_assets/implicit-name.gitops.secret.enc.yml b/test_assets/implicit-name.gitops.secret.enc.yml index e8d2eca..c02b641 100644 --- a/test_assets/implicit-name.gitops.secret.enc.yml +++ b/test_assets/implicit-name.gitops.secret.enc.yml @@ -1,4 +1,4 @@ -target: ENC[AES256_GCM,data:TnzF,iv:8RvsNPJdWMWDcmYiUGqeIjv1i4xzx81kMrxoUZdPQN0=,tag:iwsUoJDBPdvYFR2jTrgM1g==,type:str] +targetType: ENC[AES256_GCM,data:Bz2y,iv:X5bnyD9OoRqkOnHuGe7UYJNZXmNGi92RvfBlKv8GfZc=,tag:ulwLBvEu5Hi02TdFqzSYNg==,type:str] type: ENC[AES256_GCM,data:9kKk5ejO+M1rh9hNsLHyPF23ya5M/ORysamWRRJ9,iv:ihbeO5kl+2yZDef6uaXtqPdA3068U1okRb+Rz80BspA=,tag:Ely3VgdM+eTxrrLF3kJoHA==,type:str] data: .dockerconfigjson: ENC[AES256_GCM,data:9ZVhy/0VRasWTyBj6xlYoAaD3Nok6OWzdObahT/ZdPK5JBBFV29Cdo5tzHBi8+hZsGtItukiNv4K0HsCoFYg9c+O+Xnt4YabzNA6P9Ve0DM6MsVwHzZ+Uq38ovX+,iv:wiSbrtYytn+iqw59QHB5ou5W9119lOG5Cgioo52Ufvs=,tag:YnA41Im33cvvMbKMns7HXw==,type:str] @@ -17,8 +17,8 @@ sops: cEs5TzdxVEM3TGoySE4yVnUvR1FqaGsKK4PFub3b84AKbcxKB5Wf/SPQPFiSJec3 CbpGb0A9lmok0a4owoCIAhRA+UaUplpTp4IkL40xVgPYwvvnXfl4Jw== -----END AGE ENCRYPTED FILE----- - lastmodified: "2023-03-30T20:42:57Z" - mac: ENC[AES256_GCM,data:iqDnp1W0VoMGu+wWdT9BJowxmmLgz3I9Lihct+XnWJmfnhsMZHcLWMiEo4R39vGq7uVJ2mJN5t63HZ4pVc7jwMXudICDhMCPrjgR0e7gbup30muzsx9fYehNqKhAzuiZV7Y4Kpuxy+HPESjOZSLnkthAUsbYt9uJLqJR4ESNFfk=,iv:F/cmT2x3PCCqsyZ/GOSyIA1YpnPGzOImHLeRdvwTr/0=,tag:UWqISTmkj2rH4guwDmoQPQ==,type:str] + lastmodified: "2023-04-27T13:13:29Z" + mac: ENC[AES256_GCM,data:+6jI/4o6fpvYWoHBIaKjeJ3GdSXbIrJPqbIflgffv5H6mFznhSGxOrpPO/Gg5A1K1+e9dvtEY83dzsppBP3a9GbLpvj3cajjkKvQF4sWnhvkmSFRd87bs/TU3AbKMwG+NkM9KmtUXAB3LlfG+jf2aUI17mTiCrKWKWVACv7fBuM=,iv:egGgfsz9OW9lAAsgPKusba4R/fyoGGQZE6CKp1bWtck=,tag:mS6vQTuChttWyG/RbUDrHw==,type:str] pgp: [] unencrypted_suffix: _unencrypted version: 3.7.3 diff --git a/test_assets/my-secret-name.gitops.secret.enc.yml b/test_assets/my-secret-name.gitops.secret.enc.yml index e2c2068..6af77de 100644 --- a/test_assets/my-secret-name.gitops.secret.enc.yml +++ b/test_assets/my-secret-name.gitops.secret.enc.yml @@ -1,5 +1,5 @@ #ENC[AES256_GCM,data:hZQlceO258gvqevH+HTKjcF+tXPzacrwHQlRQ/MUgEAn68DPK6lGNgw=,iv:92biM3PcYcBLhdBjqk5P5uI6nmCFOGCwOCRMcG8myj4=,tag:XY/dMPDpeS63G3POWp1Slw==,type:comment] -target: ENC[AES256_GCM,data:u2Bx,iv:Wkar4VvmR/uqRcAQ/TwOIejajOJovoQGgcKjW/frhRM=,tag:ctBM3Ybq1HB0+7dJmv6ZNw==,type:str] +targetType: ENC[AES256_GCM,data:28Gp,iv:2J1QU9VKl7GjXQ4TPVDlhdw8Ihk59y3++7FK5TR0aIc=,tag:/dfAdiBLQeJZYKa5vvabSQ==,type:str] data: fizz: ENC[AES256_GCM,data:t9D+5w==,iv:t2K8aE2JAZA+x04F9dKfiQvqYAt2uFkMe0ouBQ1St5Y=,tag:xXgUrREwfVg0VL4uwagFTQ==,type:str] sops: @@ -17,8 +17,8 @@ sops: cEs5TzdxVEM3TGoySE4yVnUvR1FqaGsKK4PFub3b84AKbcxKB5Wf/SPQPFiSJec3 CbpGb0A9lmok0a4owoCIAhRA+UaUplpTp4IkL40xVgPYwvvnXfl4Jw== -----END AGE ENCRYPTED FILE----- - lastmodified: "2023-04-03T18:28:42Z" - mac: ENC[AES256_GCM,data:XQX/t1+dEOTuQ214eWIjN9LEXYB06KO3f+trEdRuauOsCyMuKpkmPBM/5WYIzJHxnsHtQhnhM9vsSAFl+Gj846R+18fu2+aNxTXh+OC3feGiaXI1tA1CzlnS/hr6fHjHj0foNBHbLq9sBGgrMqDKPo9ygKmHhGlZLpxTUhmUI/o=,iv:zDiNOVim02RlHrmedooI13uALWMSFdG1by+mToPtluE=,tag:+DvKeuuWSDVL+lMdcgjMxg==,type:str] + lastmodified: "2023-04-27T13:13:39Z" + mac: ENC[AES256_GCM,data:0s+LUo4YdTwqOCakBcOoVOwiI6+3b3EnAZuvXAcUcp6XzwHcoQeaT7s2Xjwsx91l2dwJRpe7t3J/jsI01Kd7u0eFeNfvEVZg11TG8QuQb/Brs5CrfGBJIFQf5wiz6W1zrd0LmO1gfZoyNZqO/GOqYc24izFV7V/ZsyQby95IAMQ=,iv:GCVA2eKKw51ehIzR2qSo0bd7ZV1edczFII0DNkEZcVQ=,tag:3PGqPgVFR+Tgxh41WPYZAQ==,type:str] pgp: [] unencrypted_suffix: _unencrypted version: 3.7.3 diff --git a/test_assets/subdirectory/subdir-secret.gitops.secret.enc.yml b/test_assets/subdirectory/subdir-secret.gitops.secret.enc.yml index 696672e..8bbb1a0 100644 --- a/test_assets/subdirectory/subdir-secret.gitops.secret.enc.yml +++ b/test_assets/subdirectory/subdir-secret.gitops.secret.enc.yml @@ -1,4 +1,4 @@ -target: ENC[AES256_GCM,data:u2Bx,iv:Wkar4VvmR/uqRcAQ/TwOIejajOJovoQGgcKjW/frhRM=,tag:ctBM3Ybq1HB0+7dJmv6ZNw==,type:str] +targetType: ENC[AES256_GCM,data:7bS+,iv:sfcPxuhXqC1uZpHthWv/+b+LDdTg/2ka7ZHvptGA13k=,tag:CD4+RFdsLRCjyJ4ZF6qSZQ==,type:str] name: ENC[AES256_GCM,data:XEosXffVDAYjp7CXKdZiX/7kyc0=,iv:4/ZEVJ+fVHsjwV5zzmLVTLsGpTB8G7cBZ+GmjpQ8Vew=,tag:01mQ+bajyx14Pk2olBFgZA==,type:str] data: stage: ENC[AES256_GCM,data:mL5EMvAl+M1BXCwD6cdbXjz49w==,iv:slf+JqQIOXwxurFGn6WDRJEvxhOKxZmnwX/Wux8zaE8=,tag:xOwvuly4LhrHMLu/kZu/GQ==,type:str] @@ -6,7 +6,7 @@ data: password: ENC[AES256_GCM,data:3oZSTltX2/W9ioA8SksuXZ8e/kI1n+6cmfpDntjW,iv:FhwEyHwcnEXvR7rJjilCtuANViGeKee9bS7TEz/agjU=,tag:R7Qac/kS4A3OPxOLgIxL5g==,type:str] namespace: ENC[AES256_GCM,data:81/6YL6DAzeu+bxia6W9gtmn8BDvU+A=,iv:Bu48oJFs1BbolXXm8OfzYlTQVIuZz1xD3r7Y35fqL48=,tag:R3J85gErq6/M+aUoRlxUug==,type:str] foo: ENC[AES256_GCM,data:DIWYKz7hyPQbk+Qqhj3I/gc=,iv:NCJT5taLIUhJ+7trz6QQn3cPm+WrBphXCcKceE7OF/M=,tag:lnKOKd+aALlPzx3YrFTiyg==,type:str] - bar: ENC[AES256_GCM,data:U3c=,iv:AgE/Dqie5h9T9L3FWE9Ky6ruTVbBRLh3ZYPp+NFz6ns=,tag:zpB+jGDz5VH1JwJDCr40yw==,type:int] + bar: ENC[AES256_GCM,data:tGU=,iv:UQ2TLc4d/vVB1VxzkY+WN4t8ykaal8FYJdsw9SWFZTk=,tag:k6kgdKdpR/tDMb+Y9zbb3Q==,type:int] sops: kms: [] gcp_kms: [] @@ -22,8 +22,8 @@ sops: cEs5TzdxVEM3TGoySE4yVnUvR1FqaGsKK4PFub3b84AKbcxKB5Wf/SPQPFiSJec3 CbpGb0A9lmok0a4owoCIAhRA+UaUplpTp4IkL40xVgPYwvvnXfl4Jw== -----END AGE ENCRYPTED FILE----- - lastmodified: "2023-04-04T23:02:29Z" - mac: ENC[AES256_GCM,data:bCSFbPCQ8kay5kZyGVDkdhklZldpsMsFid8qAHT5uAHM+R3rZMobC2bo+PboiMREnM+VfOfIMDuHT+ulXPmug6TLs3c1R0TLO/5M+lc15CHb1pbXbvGYcx/q5fLkXb/tlCFE9E4ZNfCs0/XzLXQLK2Ftd8CZ5ei2Nsw2riRiHkc=,iv:+dAesjtRV3nVpqyXAe5c52U/b8FL2DOZaUyYzWafNqw=,tag:si62a2hshGd0LsJAp747Fg==,type:str] + lastmodified: "2023-04-27T13:23:18Z" + mac: ENC[AES256_GCM,data:9M8PnXeLmPkG6e2nW7prSMPDTdNGw1wOhy5gdb+fO2bO3u28rs1S0vqNopOwCcWgBXuiWHnFAHjOVXV+ZzzWgiyCPyEjnhehoTMCQRUhIxW0N5fdlw4fI0m0KHy70RFsLcebBBoliaM5bQB/x0pFQ43dJuLUXKRCXvIgAmQUGQ0=,iv:svHrart7VTfP62jDig0AF1WGxFc73qmDSsEAfvMqVm8=,tag:0aVuBQil49Z+v2Z7L9rpxA==,type:str] pgp: [] unencrypted_suffix: _unencrypted version: 3.7.3 diff --git a/test_assets/test.gitops.secret.enc.yml b/test_assets/test.gitops.secret.enc.yml index 317bef0..e4bf1db 100644 --- a/test_assets/test.gitops.secret.enc.yml +++ b/test_assets/test.gitops.secret.enc.yml @@ -1,4 +1,4 @@ -target: ENC[AES256_GCM,data:CfkeOvo=,iv:hJ17tYgAr5rlnbvzYuRVzO6AGSKMg/RUD/HpfzCERq8=,tag:g1cB0E545a67/ayXLkINUQ==,type:str] +targetType: ENC[AES256_GCM,data:Vn+aih8=,iv:GPeNQKXCCRCjIuGh6rTr5Uu5zf0pwjdmBFSfH4ZMxJs=,tag:mPT+vviltDPrhXhG10VUDA==,type:str] name: ENC[AES256_GCM,data:63DjF629p7PE2B1BvZGCrT3WyL/i2ZgH020=,iv:Cxs7yEcJgjbF35BN2USiAL8cT2Dgzvys0fnU6qyxhHM=,tag:/QNEmwztc+2kv+4o5lAGYw==,type:str] data: foo: ENC[AES256_GCM,data:6nMi,iv:GENeVxfUbdiDOR0H0h3RI8AVnaPh21KqSnI0NTyDF3c=,tag:r+xE7X6Q2dDtzssz5tfe7g==,type:str] @@ -17,8 +17,8 @@ sops: cEs5TzdxVEM3TGoySE4yVnUvR1FqaGsKK4PFub3b84AKbcxKB5Wf/SPQPFiSJec3 CbpGb0A9lmok0a4owoCIAhRA+UaUplpTp4IkL40xVgPYwvvnXfl4Jw== -----END AGE ENCRYPTED FILE----- - lastmodified: "2023-03-27T23:29:05Z" - mac: ENC[AES256_GCM,data:Jm7O2LeJFcWuhKk8RlDy1L9nni1A1VbBfrx+DApMH1Bu2SjfzcIY3JHXGt3vsgSVFKeOYy7zAD7BrYCpo5J1Qn4Ukolbv/lYjWswmig67qWS3x7z3t1mtXGsjTMkkgVR0BaKtNUw2RBWmGBIPRvHJ+r1pbO9Ww5Jnnppr8BfT78=,iv:bDnq7NVJHaF2gTWwYwjeOTpEjJwtaKY61ojiBgardTU=,tag:OTIjim0wFTH0UtHohaWkxQ==,type:str] + lastmodified: "2023-04-27T13:13:49Z" + mac: ENC[AES256_GCM,data:zL8Vz05QomC+xy1wiAMnE0bRu6x2NQvE8O8C7ye3EkzCffZBGcTO0/Pg89oO9D0PfBjN4WQo6Q609+vDeWslnXeb0hzJoqE8+Z4c6TqYBeKxWYM8X1zvYIaa6ApLBXaQx2jMKXtzIcSbIf9Gl6+XEPObORTxjN5lNjNVDZ7vPOU=,iv:C90bi8/9beayPzwhJ2jhUVmz2QYg1PG+uwVPoPg74pA=,tag:zi5fn6e12r8Ai3S2+X7mxA==,type:str] pgp: [] unencrypted_suffix: _unencrypted version: 3.7.3