Parse .env vars to directory vars

[angelhodar]

Hey there, so I have been reading the documentation and I havent found a way to dump a list of variables already set in my .env file as childs of a secrets directory in my account. There is an option called -i but this dumps a full .env file in one child, but I am talking about adding each variable of the file as child in the tree, either considering some tree structure from the var names (it would be the reverse implementation of the proposal in #286) or just add each var as direct child. It could be added as a --dump option in the write command:

secrethub write -d/--dump path/to/env/file path/to/secrethub/dir

Example

// .env file
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
DB_USERNAME
DB_PASSWORD

would map to

org/repo/my-app/
    aws/
        access_key_id
        secret_access_key
    db/
        username
        password

What do you think?

[SimonBarendse]

Hi @angelhodar , thanks for opening this issue!

Is your use case to migrate a .env environment to SecretHub?

I imagine adding an import command for this. We can start by adding .env file import as a first use case and I expect more sources to be added.

The command could look something like this:

secrethub import dot-env <destination directory>

Import secrets form a .env file containing a key=value pair on each line.

Flags:
--env-file    Path to the .env file to import the secrets from. Defaults to `.env`

Example:

For your example, you'd use:

secrethub import dot-env org/repo/my-app

What do you think?

---

I believe mapping the environment variables to a directory structure will be challenging. In your example the secret name access_key_id still contains underscores. How do we decide which underscores become directories and which should remain in the secret name?

[mackenbach]

Cool suggestion! +1 on the idea of making a secrethub import command.

For the underscore case you mention, we can do something interactive by adding an -i flag. Similar to how git rebase -i works, opening up the editor and allowing you to edit things, e.g.

AWS_ACCESS_KEY_ID     => company/repo/aws_access_key_id
AWS_SECRET_ACCESS_KEY => company/repo/aws_secret_access_key
DB_USERNAME           => company/repo/db_username
DB_PASSWORD           => company/repo/db_password

Upon save (and exiting the editor) it would then proceed to write all those secrets to the paths specified by the user.

Then by default the import command could auto replace underscores with slashes, and if the user wants more control, they can edit the mapping themselves.

[angelhodar]

Hey @SimonBarendse, glad you like my idea, I find it very helpful for already existing .env files in a project, so people can easily migrate them to secrethub with just a few commands and be accesible for other team members. The problem of the underscores can be solved by specifying a -d/--deep option to let the user specify how many underscores are used as dirs. The default value could be 0, so each var in the .env file would be added as direct child of the specified dir, then a value of 1 would produce the output I showed in my first message, and a value of 2 would output this:

org/repo/my-app/
    aws/
        access/
            key_id
        secret/
            access_key
    db/
        username
        password

But i prefer the idea of @mackenbach, as its much more robust and makes the env vars naming of the user completely independent from the import command. In fact it could be done the reverse way, for example parse a secrethub dir to a -env file for a working project that uses an already configured package like dotenv, so secrethub can be used just once to configure the env files to start using the project. Just imagine you start a project and you add .env files and configure it in your project, then you migrate them to a dir in secrethub, invite a team member and just using the import command that team member would receive the .env files, everything secure. I have had this situation with a team mate, we used Telegram to pass .env files and this would be much more elegant and secure 👍

It would be somehow an alternative to secrethub run, somehow...

[SimonBarendse]

Yes, this is exactly the kind of use case we're aiming to solve with SecretHub. It should be both easier and more secure to share your secrets.

Do you know about .env file support for the run command? You can create a secrethub.env file which is similar to a normal .env file, except it replaces the actual secret values with a reference to their paths on SecretHub. Because it now doesn't contain any secrets, you can just check it into source control and easily share it with your colleagues and collaborate on it.

You can read more about it here: https://secrethub.io/docs/reference/cli/run/#environment-files

Bonus: you can define template variables in the secrethub.env file, making it possible to use the same template for different environments. You'll pass the environment to use when starting the app, e.g. secrethub run --var env=prd -- ./run-your-app.
This is an equivalent that uses an environment variable to pass the template variable: SECRETHUB_VAR_env=prd secrethub run -- ./run-your-app.

We could add to the import command that it creates the secrethub.env file corresponding to the .env file you have just imported. Then the only thing you'd have to do is check that file in source control after the import and your colleagues can start using secrethub run to get the environment from your .env file.

Alternatively, your colleagues could use the --secrets-dir flag from #286, in which case you wouldn't need a secrethub.env nor a .env file.

---

Are you proposing to get the .env file including the secrets in your file system instead as an alternative to secrethub run? You can achieve this using secrethub inject. However, you'd lose masking and would have to make sure the .env file is cleaned up after. Do you have a specific reason you want the .env file on the file system? Maybe we can add something to secrethub run to support your use case.

[angelhodar]

My proposal was more in the direction of using secrethub as a way to pass env vars between team mates faster and without the need of using secrethub specific files or commands (secrethub.env and secrethub run/inject). I say this because if I already have a project setup to work with some .env.XXXX files (being XXXX the environment), i find a bit redundant to handle another env var to specify what is already specified in my NODE_ENV var used in package.json (to show an example). And with that import command my team mates can get the .env files in their own machines very quickly and securely. Please dont take this words as argument against secrethub.env or run/inject commands, they are really great ideas, I just wanted to focus on the env vars sharing feature of secrethub.

[SimonBarendse]

Would adding this to your package.json be helpful?

{ "scripts" :
  { "start" : "secrethub run --var env=${NODE_ENV} -- node server.js"
  }
}

Then your colleagues will just run npm run start, so they don't even have to be aware of SecretHub being used under the hood.

This becomes even easier when we have the CLI as npm package as proposed in this ticket: #27. Then you can add SecretHub as a dev dependency to your project so that your colleagues will have it automatically.