Devise S3 stack schema

[riastradh-probcomp]

AWS CloudFormation templates -- of which we have a draft in the aws/ directory on the 20170505-riastradh-aws-ami branch -- cannot refer to other templates except by reference to S3 URLs. This means we can't separate our schema -- e.g., common infrastructure (VPC, front end subnets, security groups, &c.), per-instance resources (EC2 instance, back end subnets), and a single file combining it all -- into multiple local files in the git repository: we have to upload it to S3 before we can even test it. Ugh!

[axch]

Some things I can imagine doing, if this is helpful:

• Replicate the functionality of the outer caller in an ad-hoc script, or even just documentation (if there are only two, and one is to be invoked rarely, that may work)
• Can one template describe multiple stacks and let the user select which one(s) to start?
• Preemptively request increases in the limits that force VPCs etc. to be common (I think the console has at least some lists of limits applicable to our account, with current values) (may need to do this anyway as, e.g., public IPs per VPC may be limited also).
• One of the above and file a bug against cloudformation.

[axch]

Maybe relevant: The following invocation worked, even though the documentation implied that it wouldn't:

aws cloudformation validate-template --template-body file:///home/axch/work/pcp/probcomp-stack/aws/stack.yaml

Perhaps a similar thing in the appropriate place may circumvent the need for S3?

[riastradh-probcomp]

aws cloudformation validate-template ... does not recursively descend into the referenced templates.

[riastradh-probcomp]

Replicate the functionality of the outer caller in an ad-hoc script, or even just documentation (if there are only two, and one is to be invoked rarely, that may work)

What does outer caller mean here?

Can one template describe multiple stacks and let the user select which one(s) to start?

Not sure what you mean here. A stack, in CloudFormation terminology, is just another kind of AWS resource that is a composition of many other AWS resources. You can nest them and parametrize them. Thus, you can write a parametrized template for common infrastructure, a parametrized notebook for a notebook, and a single top-level nonparametrized template composing them into a fleet. In notation I'm inventing on the spot:

common_infrastructure(name, vpc_cidr) :=
    vpc = vpc(name='${name}/vpc', cidr=vpc_cidr)
    pubnet1 = subnet(...)
    pubnet2 = subnet(...)
    gateway = ...
    ...

notebook(name, subnet_cidr) :=
    ... private subnet, EC2 instance, load balancer ...

fleet :=
    infra = common_infrastructure(name='XYZ demo', vpc_cidr=10.0.0.0/16)
    user_baker = notebook(name='baker', vpc=infra.vpc, subnet_cidr=10.0.1.0/24)
    user_fletcher = notebook(name='fletcher', vpc=infra.vpc, subnet_cidr=10.0.2.0/24)
    user_miller = notebook(name='miller', vpc=infra.vpc, subnet_cidr=10.0.3.0/24)
    user_cooper = notebook(name='cooper', vpc=infra.vpc, subnet_cidr=10.0.4.0/24)
    user_smith = notebook(name='smith', vpc=infra.vpc, subnet_cidr=10.0.5.0/24)

and then you can maintain the fleet as a single unit, incrementally making updates to the set of users, to the choice of parametrized subtemplates, &c.

Preemptively request increases in the limits that force VPCs etc. to be common (I think the console has at least some lists of limits applicable to our account, with current values) (may need to do this anyway as, e.g., public IPs per VPC may be limited also).

I don't think there is a relevant limit on public IP addresses. There is a limit on elastic IP addresses per region, but we don't need those as long as we let Amazon's load balancer handle HTTPS.