Migrate away from the dependency for campus depositors to know Ingest Server IP addresses

[terrywbrady]

Problem

• AutoScaling groups will make this difficult
• Does not support disaster recovery
• Campus networking policies may prevent this in the future

Solutions

• Explore NAT gateway to simplify the network
  • Per Martin, it may not be feasible to allocate a blocked range in our new account
• Explore better authentication as we explore content (presigned urls)

[terrywbrady]

IAS meeting 11/28

• IAS will ask AWS if the NAT gateway is a good solution
• Merritt team should consider why this is still needed
• think about the impact on the linux migration

[elopatin-uc3]

@elopatin-uc3 to document each campus library and/or organization that is whitelisting Ingest host IP addresses - then set up dialog with campus contacts to determine if another option is available and considered a better practice in terms of security.

[elopatin-uc3]

Using IP addresses:

• Block of IP addresses, but you need to pay for these when you're not using them
  • Determine maximum number of instances
• NAT gateway; spins up an EC2 instance for you to run this
  • Give the gateway one IP address to use and make this public-facing
  • Private subnet involved

Ingest and Storage:

• Algorithm to determine ratio of ingest to storage servers, or mirror, or each service having its own method to determine number of workers needed.

Autoscaling Groups (ASG)

Instances vs. containers:

• for containers; however we are likely to need specialized EC2 instances (ingest, storage, replic, access) vs. containers for inventory and audit
• Fargate for containers