CLOSED - Oauth integration

.dependency_license

@@ -105,6 +105,10 @@ traffic_portal/app/src/assets/js/chartjs/angular-chart\..*, BSD
 traffic_portal/app/src/assets/css/jsonformatter\..*, Apache
 traffic_portal/app/src/assets/js/jsonformatter\..*, Apache
 traffic_portal/app/src/assets/js/fast-json-patch\..*, MIT
+traffic_portal/app/src/assets/css/colReorder.dataTables\..*, MIT
+traffic_portal/app/src/assets/js/colReorder.dataTables\..*, MIT
+traffic_ops/traffic_ops_golang/vendor/github\.com/dgrijalva/.*, MIT
+traffic_ops/traffic_ops_golang/vendor/github\.com/lestrrat-go/.*, MIT
 
 # Ignored - Do not report.
 \.DS_Store, Ignore # Created automatically OSX.

.rat-excludes

@@ -41,6 +41,7 @@ j[mM]enu(?:\.jquery)?\.(?:css|js)(?: MIT. Properly documented in LICENSE ){0}
 downloadjs\-min.*\.js(?:             MIT. Properly documented in LICENSE ){0}
 jquery\-ui\..*(?:                    MIT. Properly documented in LICENSE ){0}
 jquery\.dataTables\..*(?:            MIT. Properly documented in LICENSE ){0}
+colReorder\.dataTables\..*(?:        MIT. Properly documented in LICENSE ){0}
 .*underscore.*(?:                    MIT. Properly documented in LICENSE ){0}
 .*moment.*(?:                        MIT. Properly documented in LICENSE ){0}
 .*jsdiff.*(?:                        BSD 3-clause. Properly documented in LICENSE ){0}

CHANGELOG.md

@@ -19,6 +19,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
   - /api/1.4/cdns/dnsseckeys/refresh `GET`
   - /api/1.1/cdns/name/:name/dnsseckeys `GET`
   - /api/1.4/cdns/name/:name/dnsseckeys `GET`
+  - /api/1.4/user/login/oauth `POST`
 - To support reusing a single riak cluster connection, an optional parameter is added to riak.conf: "HealthCheckInterval". This options takes a 'Duration' value (ie: 10s, 5m) which affects how often the riak cluster is health checked.  Default is currently set to: "HealthCheckInterval": "5s".
 - Added a new Go db/admin binary to replace the Perl db/admin.pl script which is now deprecated and will be removed in a future release. The new db/admin binary is essentially a drop-in replacement for db/admin.pl since it supports all of the same commands and options; therefore, it should be used in place of db/admin.pl for all the same tasks.
 - Added an API 1.4 endpoint, /api/1.4/cdns/dnsseckeys/refresh, to perform necessary behavior previously served outside the API under `/internal`.
@@ -31,6 +32,8 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 - In Traffic Portal, removes the need to specify line breaks using `__RETURN__` in delivery service edge/mid header rewrite rules, regex remap expressions, raw remap text and traffic router additional request/response headers.
 - In Traffic Portal, provides the ability to clone delivery service assignments from one cache to another cache of the same type. Issue #2963.
 - Traffic Ops now allows each delivery service to have a set of query parameter keys to be retained for consistent hash generation by Traffic Router.
+- Added an API 1.4 endpoint, /api/1.4/user/login/oauth to handle SSO login using OAuth.
+- Added /#!/sso page to Traffic Portal to catch redirects back from OAuth provider and POST token into the API.
 
 ### Changed
 - Traffic Router, added TLS certificate validation on certificates imported from Traffic Ops
@@ -47,13 +50,17 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 - Modified Traffic Router logging format to include an additional field for DNS log entries, namely `rhi`. This defaults to '-' and is only used when EDNS0 client subnet extensions are enabled and a client subnet is present in the request. When enabled and a subnet is present, the subnet appears in the `chi` field and the resolver address is in the `rhi` field.
 - Changed traffic_ops_ort.pl so that hdr_rw-<ds>.config files are compared with strict ordering and line duplication when detecting configuration changes.
 - Traffic Ops (golang), Traffic Monitor, Traffic Stats are now compiled using Go version 1.11. Grove was already being compiled with this version which improves performance for TLS when RSA certificates are used.
+- Fixed issue #3497: TO API clients that don't specify the latest minor version will overwrite/default any fields introduced in later versions
 - Issue 3476: Traffic Router returns partial result for CLIENT_STEERING Delivery Services when Regional Geoblocking or Anonymous Blocking is enabled.
 - Upgraded Traffic Portal to AngularJS 1.7.8
 - Issue 3275: Improved the snapshot diff performance and experience.
 - Issue #3605: Fixed Traffic Monitor custom ports in health polling URL.
 - Issue 3587: Fixed Traffic Ops Golang reverse proxy and Riak logs to be consistent with the format of other error logs.
 - Database migrations have been collapsed. Rollbacks to migrations that previously existed are no longer possible.
+- Issue #3750: Fixed Grove access log fractional seconds.
 - Issue #3646: Fixed Traffic Monitor Thresholds.
+- Added fields to traffic_portal_properties.json to configure SSO through OAuth.
+- Added field to cdn.conf to configure whitelisted URLs for Json Key Set URL returned from OAuth provider.
 
 ## [3.0.0] - 2018-10-30
 ### Added

ISSUE_TEMPLATE.md

@@ -0,0 +1,68 @@
+<!--
+************ STOP!! ************
+If this issue identifies a security vulnerability, DO NOT submit it! Instead, contact
+the Apache Software Foundation Security Team at security@trafficcontrol.apache.org and follow the
+guidelines at https://www.apache.org/security/ regarding vulnerability disclosure.
+-->
+
+<!--
+- For *SUPPORT QUESTIONS*, use the
+[Traffic Control slack channels](https://traffic-control-cdn.slack.com) or [Traffic Control mailing lists](http://trafficcontrol.apache.org/mailing_lists/).
+- Before submitting, please **SEARCH GITHUB** for a similar issue or PR. -->
+
+## I'm submitting a ...
+<!-- (check all that apply with "[x]") -->
+<!--- security vulnerability (STOP!! - see above)-->
+- [ ] bug report
+- [ ] new feature / enhancement request
+- [ ] improvement request (usability, performance, tech debt, etc.)
+- [ ] other <!--(Please do not submit support requests here - see above)-->
+
+## Traffic Control components affected ...
+<!-- (check all that apply with "[x]") -->
+- [ ] CDN in a Box
+- [ ] Documentation
+- [ ] Grove
+- [ ] Traffic Control Client
+- [ ] Traffic Monitor
+- [ ] Traffic Ops
+- [ ] Traffic Ops ORT
+- [ ] Traffic Portal
+- [ ] Traffic Router
+- [ ] Traffic Stats
+- [ ] Traffic Vault
+- [ ] unknown
+
+## Current behavior:
+<!-- Describe how the bug manifests / how the current features are insufficient. -->
+
+## Expected / new behavior:
+<!-- Describe what the behavior would be without the bug / how the feature would improve Traffic Control -->
+
+## Minimal reproduction of the problem with instructions:
+<!--
+If the current behavior is a bug or you can illustrate your feature request better with an example,
+please provide the *STEPS TO REPRODUCE* and include the applicable TC version.
+-->
+
+## Anything else:
+<!-- e.g. stacktraces, related issues, suggestions how to fix -->
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->

LICENSE

@@ -243,10 +243,15 @@ For the JSON formatter component:
 
 For the DataTables component:
 @traffic_portal/app/src/assets/css/jquery.dataTables.min_1.10.9.css
-@traffic_portal/app/src/assets/js/jquery.dataTables.min_1.10.16.js
+@traffic_portal/app/src/assets/js/jquery.dataTables.min_1.10.19_patched.js
 @traffic_portal/app/src/assets/images/sort_*
 ./licenses/MIT-datatables
 
+For the DataTables ColReorder component:
+@traffic_portal/app/src/assets/css/colReorder.dataTables.min_1.5.1.css
+@traffic_portal/app/src/assets/js/colReorder.dataTables.min_1.5.1.js
+./licenses/MIT-ColReorder
+
 For the moment.js component:
 @traffic_portal/app/src/assets/js/moment-min_2.22.1.js
 ./licenses/MIT-momentjs
@@ -427,3 +432,11 @@ The modern-go/concurrent component is used under the Apache 2.0 license:
 The modern-go/reflect2 component is used under the Apache 2.0 license:
 @vendor/github.com/modern-go/reflect2/*
 ./vendor/github.com/modern-go/reflect2/LICENSE
+
+For the lestrrat-go/jwx component:
+@traffic_ops/traffic_ops_golang/vendor/github.com/lestrrat-go/jwx/*
+./traffic_ops/traffic_ops_golang/vendor/github.com/lestrrat-go/jwx/LICENSE
+
+For the dgrijalva/jwt-go component:
+@traffic_ops/traffic_ops_golang/vendor/github.com/dgrijalva/jwt-go/*
+./traffic_ops/traffic_ops_golang/vendor/github.com/dgrijalva/jwt-go/LICENSE

PULL_REQUEST_TEMPLATE.md

@@ -46,7 +46,7 @@ it includes tests (and most should), outline here the steps needed to run the
 tests. If not, lay out the manual testing procedure and please explain why
 tests are unnecessary for this Pull Request. -->
 
-## If this is a bug fix, what versions of Traffic Ops are affected?
+## If this is a bug fix, what versions of Traffic Control are affected?
 <!-- If this PR fixes a bug, please list here all of the affected versions - to
 the best of your knowledge. It's also pretty helpful to include a commit hash
 of where 'master' is at the time this PR is opened (if it affects master),

docs/source/admin/quick_howto/anonymous_blocking.rst

@@ -50,29 +50,29 @@ Configure Anonymous Blocking
 		An optional element. It includes a list of :abbr:`CIDR (Classless Inter-Domain Routing)` blocks indicating the IPv4 and IPv6 subnets that are allowed by the rule. If this list exists and the value is not ``null``, client IPs will be matched against the :abbr:`CIDR (Classless Inter-Domain Routing)` list, and if there is any match, the request will be allowed. If there is no match in the white list, further anonymous blocking logic will continue.
 
 
-#. Add the following three Anonymous Blocking parameters in Traffic Portal into CRConfig.json:
+#. Add the following three Anonymous Blocking :ref:`Parameters` in Traffic Portal with the "CRConfig.json" :ref:`parameter-config-file`, and ensure they are assigned to all of the Traffic Routers that should perform Anonymous Blocking:
 
 	``anonymousip.policy.configuration``
-		The HTTP URL of the Anonymous Blocking configuration file. Traffic Router will fetch the file from this URL.
+		The URL of the Anonymous Blocking configuration file. Traffic Router will fetch the file from this URL.
 	``anonymousip.polling.url``
-		The HTTP URL of the Anonymous IP Database. Traffic Router will fetch the file from this URL.
+		The URL of the Anonymous IP Database. Traffic Router will fetch the file from this URL.
 	``anonymousip.polling.interval``
 		The interval that Traffic Router polls the Anonymous Blocking configuration file and Anonymous IP Database.
 
 	.. figure:: anonymous_blocking/01.png
-		:scale: 100%
+		:width: 40%
 		:align: center
 
-#. Enable Anonmyous Blocking for a :term:`Delivery Service`
+#. Enable Anonmyous Blocking for a :term:`Delivery Service` using the :ref:`Delivery Services view in Traffic Portal <tp-services-delivery-service>` (don't forget to save changes!)
 
 	.. figure:: anonymous_blocking/02.png
-		:scale: 100%
+		:width: 40%
 		:align: center
 
-#. Go to :menuselection:`Tools --> Snapshot CRConfig`, perform :guilabel:`Diff CRConfig` and click :guilabel:`Write CRConfig`.
+#. Go to :ref:`the Traffic Portal CDNs view <tp-cdns>`, click on :guilabel:`Diff CDN Config Snapshot`, and click :guilabel:`Perform Snapshot`.
 
-	.. figure:: regionalgeo/03.png
-		:scale: 70%
+	.. figure:: anonymous_blocking/03.png
+		:width: 40%
 		:align: center
 
 

docs/source/admin/quick_howto/ciab.rst

@@ -201,9 +201,9 @@ The enroller runs within CDN in a Box using :option:`--dir` which provides the a
 
 Auto Snapshot/Queue-Updates
 ---------------------------
-An automatic snapshot of the current Traffic Ops CDN configuration/toplogy will be performed once the "enroller" has finished loading all of the data and a minimum number of servers have been enrolled.  To enable this feature, set the boolean ``AUTO_SNAPQUEUE_ENABLED`` to ``true`` [8]_.  The snapshot and queue-updates actions will not be performed until all servers in ``AUTO_SNAPQUEUE_SERVERS`` (comma-delimited string) have been enrolled.  The current enrolled servers will be polled every ``AUTO_SNAPQUEUE_POLL_INTERVAL`` seconds, and each action (snapshot and queue-updates) will be delayed ``AUTO_SNAPQUEUE_ACTION_WAIT`` seconds [9]_.
+An automatic :term:`Snapshot` of the current Traffic Ops CDN configuration/topology will be performed once the "enroller" has finished loading all of the data and a minimum number of servers have been enrolled. To enable this feature, set the boolean ``AUTO_SNAPQUEUE_ENABLED`` to ``true`` [8]_. The :term:`Snapshot` and :term:`Queue Updates` actions will not be performed until all servers in ``AUTO_SNAPQUEUE_SERVERS`` (comma-delimited string) have been enrolled. The current enrolled servers will be polled every ``AUTO_SNAPQUEUE_POLL_INTERVAL`` seconds, and each action (:term:`Snapshot` and :term:`Queue Updates`) will be delayed ``AUTO_SNAPQUEUE_ACTION_WAIT`` seconds [9]_.
 
-.. [8] Automatic Snapshot/Queue-Updates is enabled by default in :file:`infrastructure/cdn-in-a-box/variables.env`.
+.. [8] Automatic :term:`Snapshot`/:term:`Queue Updates` is enabled by default in `variables.env`_.
 .. [9] Server poll interval and delay action wait are defaulted to a value of 2 seconds.
 
 Mock Origin Service

docs/source/admin/quick_howto/ds_requests.rst

@@ -18,7 +18,7 @@
 *************************
 Delivery Service Requests
 *************************
-When enabled in :file:`traffic_portal_properties.json`, Delivery Service Requests are created when *all* users attempt to create, update or delete a :term:`Delivery Service`. This allows users with higher level permissions ("operations" or "admin") to review the changes for completeness and accuracy before deploying the changes. In addition, most :term:`Delivery Service` changes require cache configuration updates (aka queue updates) and/or a CDN :term:`Snapshot`. Both of these actions are reserved for users with elevated permissions.
+When enabled in :file:`traffic_portal_properties.json`, Delivery Service Requests are created when *all* users attempt to create, update or delete a :term:`Delivery Service`. This allows users with higher level permissions ("operations" or "admin") to review the changes for completeness and accuracy before deploying the changes. In addition, most :term:`Delivery Service` changes require configuration updates (i.e. :term:`Queue Updates`) and/or a CDN :term:`Snapshot`. Both of these actions are reserved for users with elevated permissions.
 
 A list of the Delivery Service requests associated with your :term:`Tenant` can be found under :menuselection:`Services --> Delivery Service Requests`
 
@@ -53,10 +53,10 @@ Reject the Delivery Service Request
 	Rejecting a Delivery Service Request will set status to 'rejected' and the request can no longer be modified. This will auto-assign the request to the user doing the rejection.
 
 Fulfill the Delivery Service Request
-	Fulfilling a Delivery Service Request will show the requested changes and, once committed, will apply the desired changes and set status to 'pending'. The request is pending because many types of changes will require :term:`cache server` configuration updates (aka queue updates) and/or a CDN snapshot. Once queue updates and/or CDN snapshot is complete, the request should be marked 'complete'.
+	Fulfilling a Delivery Service Request will show the requested changes and, once committed, will apply the desired changes and set status to 'pending'. The request is pending because many types of changes will require :term:`cache server` configuration updates (i.e. :term:`Queue Updates`) and/or a CDN :term:`Snapshot`. Once :term:`Queue Updates` and/or CDN :term:`Snapshot` is complete, the request should be marked 'complete'.
 
 Complete the Delivery Service Request
-	Only after the Delivery Service Request has been fulfilled and the changes have been applied can a Delivery Service Request be marked as 'complete'. Marking a Delivery Service Request as 'complete' is currently a manual step because some changes require :term:`cache server` configuration updates (aka queue updates) and/or a CDN :term:`Snapshot`. Once that is done and the changes have been deployed, the request status should be changed from 'pending' to 'complete'.
+	Only after the Delivery Service Request has been fulfilled and the changes have been applied can a Delivery Service Request be marked as 'complete'. Marking a Delivery Service Request as 'complete' is currently a manual step because some changes require :term:`cache server` configuration updates (i.e. :term:`Queue Updates`) and/or a CDN :term:`Snapshot`. Once that is done and the changes have been deployed, the request status should be changed from 'pending' to 'complete'.
 
 Delete the Delivery Service request
 	Delivery Service Requests with a status of 'draft' or 'submitted' can always be deleted entirely if appropriate.

docs/source/admin/quick_howto/index.rst

@@ -27,6 +27,7 @@ Traffic Control is a complicated system, and documenting it is not trivial. Some
 	ds_requests
 	federations
 	multi_site
+	oauth_login
 	regionalgeo
 	static_dns
 	steering