Flexible Topologies in Traffic Ops and Traffic Portal by zrhoffman · Pull Request #4633 · apache/trafficcontrol

zrhoffman · 2020-04-13T21:23:21Z

What does this PR (Pull Request) do?

This PR

Adds some functionality to Traffic Ops and Traffic Portal described in the Flexible Topologies blueprint

Closes:

Traffic Ops	Traffic Portal
Implement Topologies CRUD endpoints #4565	TP: Provide the ability to create/update/delete a topology #4563
Add Flexible Topologies tables and modify deliveryservice table #4564	TP: Provide the ability to assign a delivery service topology #4562
	TP: Provide the ability to view all delivery services assigned to a topology #4561
	TP: Provide the ability to view all cache groups assigned to a topology #4560
	TP: Provide the ability to view all servers utilized by a topology #4559
	TP: Provide the ability to view all topologies #4558

Which Traffic Control components are affected by this PR?

Documentation
Traffic Control Clients: Go, Python
Traffic Ops
Traffic Portal

What is the best way to verify this PR?

Run Traffic Ops API tests
Run Traffic Portal tests
Build docs

You can also navigate in TP to tp.domain.com/#!/topologies and CRUD topologies.

The following criteria are ALL met by this PR

This PR includes tests
This PR includes documentation
This PR includes an update to CHANGELOG.md
This PR includes any and all required license headers
This PR ensures that database migration sequence is correct
This PR DOES NOT FIX A SERIOUS SECURITY VULNERABILITY (see the Apache Software Foundation's security guidelines for details)
^{^{closes TP: Provide the ability to view all topologies #4558
closes TP: Provide the ability to view all servers utilized by a topology #4559
closes TP: Provide the ability to view all cache groups assigned to a topology #4560
closes TP: Provide the ability to view all delivery services assigned to a topology #4561
closes TP: Provide the ability to assign a delivery service topology #4562
closes TP: Provide the ability to create/update/delete a topology #4563
closes Add Flexible Topologies tables and modify deliveryservice table #4564
closes Implement Topologies CRUD endpoints #4565}}

zrhoffman · 2020-04-15T18:44:05Z

This PR is ready for review.

zrhoffman · 2020-04-22T17:00:27Z

Rebased onto master

- Create topology table - Create topology_cachegroup table - Create topology_cachegroup_parents table - Goose migration to add topology column to deliveryservice table

…nt view

rawlinp

I got most of the way through reviewing this, except for the real meat of the API implementation and validation. I'll hopefully finish up the rest early next week.

…would not filter. (#16)

- Fix UpdateTopologies function definition

different" warning in 36e0854

- EdgeCacheGroupType -> CacheGroupEdgeTypeName - MidCacheGroupType -> CacheGroupMidTypeName

rawlinp

Ok, I got through the rest of the TO changes. We need to be adamant about defer rows.Close() whenever we run a DB query -- otherwise, we may end up depleting all of the available DB connections over time if we're not careful.

rawlinp · 2020-05-13T16:42:12Z

+	return interfaces, nil, nil, http.StatusOK
+}
+
+func (topology *TOTopology) removeParents() error {


rawlinp · 2020-05-13T16:43:25Z

+	return nil, nil, http.StatusOK
+}
+
+func (topology *TOTopology) addParents() (error, error, int) {


heartwarming :D

rawlinp

Unit tests, TO API tests, manual CRUD testing via TP, and DB up/down migration were all successful.

mitchell852 added WIP "Work-in-Progress" - do not merge! (use 'draft' pull requests from now on) Traffic Portal v1 related to Traffic Portal version 1 Traffic Ops related to Traffic Ops labels Apr 13, 2020

ocket8888 added this to the Flexible Topologies milestone Apr 14, 2020

rawlinp reviewed Apr 14, 2020

View reviewed changes

Comment thread traffic_ops/traffic_ops_golang/routing/routes.go Outdated

zrhoffman changed the title ~~WIP: Flexible Topologies in Traffic Ops and Traffic Portal~~ Flexible Topologies in Traffic Ops and Traffic Portal Apr 15, 2020

mitchell852 removed the WIP "Work-in-Progress" - do not merge! (use 'draft' pull requests from now on) label Apr 15, 2020

mitchell852 added the new feature A new feature, capability or behavior label Apr 15, 2020

mitchell852 reviewed Apr 20, 2020

View reviewed changes

Comment thread traffic_ops/traffic_ops_golang/routing/routes.go Outdated

mitchell852 reviewed Apr 22, 2020

View reviewed changes

Comment thread traffic_ops/app/db/migrations/20200402133144_create_topology_tables.sql Outdated

zrhoffman force-pushed the topologies branch from d7e2b85 to 782f5f8 Compare April 22, 2020 17:00

ocket8888 suggested changes Apr 22, 2020

View reviewed changes

zrhoffman and others added 16 commits April 22, 2020 13:32

Topology Goose migrations

83f1492

- Create topology table - Create topology_cachegroup table - Create topology_cachegroup_parents table - Goose migration to add topology column to deliveryservice table

POST /topologies

8b60fa6

Create all nodes in a single query and all parents in a single query

70b90dd

Validation so far

165e98e

Topologies endpoints for TO Python client

e73d292

Topologies endpoint functions for Go client library

5377035

GET /topologies

3c65025

adds toplogy views

3c13fcc

adds ability to add cache groups to a top

e0f0a83

adds fake origin layer

9b4efec

only allow org_loc to be at top of topology and moved some files around

9cf09ac

adds the ability to see all servers assigned to a topology cg

dbb52a2

adds the ability to view servers for a cachgroup from the cg assignme…

215eb7b

…nt view

adds the ability to select a 2nd parent cg

32d01d3

adds the ability to assign a second parent to nodes

640bd0a

some code cleanup

986fe09

upgrades TP from api v2 to v3 (#14)

0568cee

rawlinp suggested changes May 1, 2020

View reviewed changes

reverts TP to v2 and introduces v3 for topologies (#15)

c34afcd

mattjackson220 mentioned this pull request May 5, 2020

Add service category feature to TO and TP #4518

Merged

7 tasks

mitchell852 and others added 15 commits May 6, 2020 09:07

fixes bug where if you select all and then filter, the all selection …

db6e364

…would not filter. (#16)

Do not use pointers for topology nodes

855560d

Fix warnings

36e0854

Group import with other TO imports

d04cfd8

UpdateTopologyByID -> UpdateTopology

0dcac11

No need to load Parameters objects for Topologies tests

067fac7

Swap comments

1d93481

Make a struct for invalid topologies instead of twi separate slices

bee8af9

Make failed assertions fatal

1a10131

- Test UpdateTopologies

95c9731

- Fix UpdateTopologies function definition

Fix mixed up variable names from resolving "Receiver names are

e86c7b2

different" warning in 36e0854

Retrieve all cache groups for topology validation in a single query

4146f5b

Rename cachegroup type constants

64adb7a

- EdgeCacheGroupType -> CacheGroupEdgeTypeName - MidCacheGroupType -> CacheGroupMidTypeName

Move parentCachegroup2 fixture to v3 API tests

3a9d316

Correct term: Topology -> Cache Group

4fb9ef4

rawlinp suggested changes May 13, 2020

View reviewed changes

zrhoffman added 5 commits May 13, 2020 12:01

gofmt

9101c1f

Defer closing DB connections

73fb323

No need to delete nodes and parents when deleting an entire topology

908e9fb

Remove unnecessary blank argument

c3cbf21

Use log.Close() to close responses

cf84207

zrhoffman requested a review from rawlinp May 13, 2020 19:25

adds changelog entry and shows topologies menu item in TP (#17)

7430420

rawlinp approved these changes May 13, 2020

View reviewed changes

rawlinp merged commit 88f4f2a into apache:master May 13, 2020

zrhoffman removed this from the Flexible Topologies milestone Oct 16, 2020

Conversation

zrhoffman commented Apr 13, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR (Pull Request) do?

Which Traffic Control components are affected by this PR?

What is the best way to verify this PR?

The following criteria are ALL met by this PR

Uh oh!

Uh oh!

zrhoffman commented Apr 15, 2020

Uh oh!

Uh oh!

Uh oh!

zrhoffman commented Apr 22, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rawlinp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rawlinp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rawlinp May 13, 2020

Choose a reason for hiding this comment

Uh oh!

rawlinp May 13, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rawlinp left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

zrhoffman commented Apr 13, 2020 •

edited

Loading