As noted e.g. at Recommended # of iterations when using PKBDF2-SHA256?, it is important to take steps to protect the password for the secret key from brute force attacks. It appears that the signify code uses bcrypt_pbkdf to do this, but the parameters like iteration count used aren't noted in the documentation. It should be possible to increase the count over time.
Can the number of iterations/rounds be modified via the command-line tool? If not can that feature be added?