Skip to content
This repository was archived by the owner on Nov 24, 2025. It is now read-only.

updated /deliveryservices/{{ID}}/servers and /deliveryservices/{{ID}}… #4676

Merged
ocket8888 merged 9 commits into from
Jun 15, 2020
Merged

updated /deliveryservices/{{ID}}/servers and /deliveryservices/{{ID}}… #4676

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
136c787
updated /deliveryservices/{{ID}}/servers and /deliveryservices/{{ID}}…
mattjackson220 Apr 30, 2020
40b4c7f
updates per comments
mattjackson220 May 6, 2020
456f4cb
updates per comments
mattjackson220 May 6, 2020
968f4f4
reverted mgmt ip info change and updated for parity with /servers work
mattjackson220 May 8, 2020
8bde2b0
update to fix docs
mattjackson220 May 8, 2020
0d6ed83
added unit testing
mattjackson220 May 13, 2020
0e7b22f
updates to integrate better with other PRs
mattjackson220 May 26, 2020
093b98f
updated to change IpAddresses to IPAddresses
mattjackson220 Jun 11, 2020
87d61c8
minor update to fix miss in rebase
mattjackson220 Jun 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
- Updated /servers/details to use multiple interfaces in API v3
- Added [Edge Traffic Routing](https://traffic-control-cdn.readthedocs.io/en/latest/admin/traffic_router.html#edge-traffic-routing) feature which allows Traffic Router to localize more DNS record types than just the routing name for DNS delivery services
- Astats csv support - astats will now respond to `Accept: text/csv` and return a csv formatted stats list
- Updated /deliveryservices/{{ID}}/servers to use multiple interfaces in API v3
- Updated /deliveryservices/{{ID}}/servers/eligible to use multiple interfaces in API v3

### Fixed
- Fixed the `GET /api/x/jobs` and `GET /api/x/jobs/:id` Traffic Ops API routes to allow falling back to Perl via the routing blacklist
Expand Down
49 changes: 33 additions & 16 deletions docs/source/api/v3/deliveryservices_id_servers.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,16 +54,22 @@ Response Structure
:iloIpNetmask: The IPv4 subnet mask of the lights-out-management port\ [#ilowikipedia]_
:iloPassword: The password of the of the lights-out-management user - displays as ``******`` unless the requesting user has the 'admin' role)\ [#ilowikipedia]_
:iloUsername: The user name for lights-out-management\ [#ilowikipedia]_
:interfaceMtu: The :abbr:`MTU (Maximum Transmission Unit)` to configure for ``interfaceName``
:interfaces: An array of interface and IP address information

.. seealso:: `The Wikipedia article on Maximum Transmission Unit <https://en.wikipedia.org/wiki/Maximum_transmission_unit>`_
:max_bandwidth: The maximum allowed bandwidth for this interface to be considered "healthy" by Traffic Monitor. This has no effect if `monitor` is not true. Values are in kb/s. The value `null` means "no limit".
:monitor: A boolean indicating if Traffic Monitor should monitor this interface
:mtu: The :abbr:`MTU (Maximum Transmission Unit)` to configure for ``interfaceName``

.. seealso:: `The Wikipedia article on Maximum Transmission Unit <https://en.wikipedia.org/wiki/Maximum_transmission_unit>`_

:name: The network interface name used by the server.

:ipAddresses: An array of the IP address information for the interface

:address: The IPv4 or IPv6 address and subnet mask of the server - applicable for the interface ``name``
:gateway: The IPv4 or IPv6 gateway address of the server - applicable for the interface ``name``
:service_address: A boolean determining if content will be routed to the IP address

:interfaceName: The network interface name used by the server
:ip6Address: The IPv6 address and subnet mask of the server - applicable for the interface ``interfaceName``
:ip6Gateway: The IPv6 gateway address of the server - applicable for the interface ``interfaceName``
:ipAddress: The IPv4 address of the server- applicable for the interface ``interfaceName``
:ipGateway: The IPv4 gateway of the server- applicable for the interface ``interfaceName``
:ipNetmask: The IPv4 subnet mask of the server- applicable for the interface ``interfaceName``
:lastUpdated: The time and date at which this server was last updated, in an ISO-like format
:mgmtIpAddress: The IPv4 address of the server's management port
:mgmtIpGateway: The IPv4 gateway of the server's management port
Expand Down Expand Up @@ -121,13 +127,6 @@ Response Structure
"iloIpNetmask": "",
"iloPassword": "",
"iloUsername": "",
"interfaceMtu": 1500,
"interfaceName": "eth0",
"ip6Address": "fc01:9400:1000:8::100",
"ip6Gateway": "fc01:9400:1000:8::1",
"ipAddress": "172.16.239.100",
"ipGateway": "172.16.239.1",
"ipNetmask": "255.255.255.0",
"lastUpdated": "2018-11-14 21:08:44+00",
"mgmtIpAddress": "",
"mgmtIpGateway": "",
Expand All @@ -146,7 +145,25 @@ Response Structure
"tcpPort": 80,
"type": "EDGE",
"typeId": 11,
"updPending": false
"updPending": false,
"interfaces": [{
"ipAddresses": [
{
"address": "172.16.239.100",
"gateway": "172.16.239.1",
"service_address": true
},
{
"address": "fc01:9400:1000:8::100",
"gateway": "fc01:9400:1000:8::1",
"service_address": true
}
],
"max_bandwidth": 0,
"monitor": true,
"mtu": 1500,
"name": "eth0"
}]
}
]}

Expand Down
49 changes: 33 additions & 16 deletions docs/source/api/v3/deliveryservices_id_servers_eligible.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,16 +59,22 @@ Response Structure
:iloIpNetmask: The IPv4 subnet mask of the lights-out-management port\ [#ilowikipedia]_
:iloPassword: The password of the of the lights-out-management user - displays as ``******`` unless the requesting user has the 'admin' role)\ [#ilowikipedia]_
:iloUsername: The user name for lights-out-management\ [#ilowikipedia]_
:interfaceMtu: The :abbr:`MTU (Maximum Transmission Unit)` to configure for ``interfaceName``
:interfaces: An array of interface and IP address information

.. seealso:: `The Wikipedia article on Maximum Transmission Unit <https://en.wikipedia.org/wiki/Maximum_transmission_unit>`_
:max_bandwidth: The maximum allowed bandwidth for this interface to be considered "healthy" by Traffic Monitor. This has no effect if `monitor` is not true. Values are in kb/s. The value `null` means "no limit".
:monitor: A boolean indicating if Traffic Monitor should monitor this interface
:mtu: The :abbr:`MTU (Maximum Transmission Unit)` to configure for ``interfaceName``

.. seealso:: `The Wikipedia article on Maximum Transmission Unit <https://en.wikipedia.org/wiki/Maximum_transmission_unit>`_

:name: The network interface name used by the server.

:ipAddresses: An array of the IP address information for the interface

:address: The IPv4 or IPv6 address and subnet mask of the server - applicable for the interface ``name``
:gateway: The IPv4 or IPv6 gateway address of the server - applicable for the interface ``name``
:service_address: A boolean determining if content will be routed to the IP address

:interfaceName: The network interface name used by the server
:ip6Address: The IPv6 address and subnet mask of the server - applicable for the interface ``interfaceName``
:ip6Gateway: The IPv6 gateway address of the server - applicable for the interface ``interfaceName``
:ipAddress: The IPv4 address of the server- applicable for the interface ``interfaceName``
:ipGateway: The IPv4 gateway of the server- applicable for the interface ``interfaceName``
:ipNetmask: The IPv4 subnet mask of the server- applicable for the interface ``interfaceName``
:lastUpdated: The time and date at which this server was last updated, in an ISO-like format
:mgmtIpAddress: The IPv4 address of the server's management port
:mgmtIpGateway: The IPv4 gateway of the server's management port
Expand Down Expand Up @@ -114,13 +120,6 @@ Response Structure
"iloIpNetmask": "",
"iloPassword": "",
"iloUsername": "",
"interfaceMtu": 1500,
"interfaceName": "eth0",
"ip6Address": "fc01:9400:1000:8::100",
"ip6Gateway": "fc01:9400:1000:8::1",
"ipAddress": "172.16.239.100",
"ipGateway": "172.16.239.1",
"ipNetmask": "255.255.255.0",
"lastUpdated": "2018-10-30 16:01:12+00",
"mgmtIpAddress": "",
"mgmtIpGateway": "",
Expand All @@ -139,7 +138,25 @@ Response Structure
"tcpPort": 80,
"type": "EDGE",
"typeId": 11,
"updPending": false
"updPending": false,
"interfaces": [{
"ipAddresses": [
{
"address": "172.16.239.100",
"gateway": "172.16.239.1",
"service_address": true
},
{
"address": "fc01:9400:1000:8::100",
"gateway": "fc01:9400:1000:8::1",
"service_address": true
}
],
"max_bandwidth": 0,
"monitor": true,
"mtu": 1500,
"name": "eth0"
}]
}
]}

Expand Down
26 changes: 14 additions & 12 deletions lib/go-tc/deliveryservice_servers.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,8 @@ const (
Eligible
)

type DSServersAttrResponse struct {
Response []DSServer `json:"response"`
}

type DSServer struct {
// DSServerBase contains the base information for a Delivery Service Server.
type DSServerBase struct {
Cachegroup *string `json:"cachegroup" db:"cachegroup"`
CachegroupID *int `json:"cachegroupId" db:"cachegroup_id"`
CDNID *int `json:"cdnId" db:"cdn_id"`
Expand All @@ -85,13 +82,6 @@ type DSServer struct {
ILOIPNetmask *string `json:"iloIpNetmask" db:"ilo_ip_netmask"`
ILOPassword *string `json:"iloPassword" db:"ilo_password"`
ILOUsername *string `json:"iloUsername" db:"ilo_username"`
InterfaceMtu *int `json:"interfaceMtu" db:"interface_mtu"`
InterfaceName *string `json:"interfaceName" db:"interface_name"`
IP6Address *string `json:"ip6Address" db:"ip6_address"`
IP6Gateway *string `json:"ip6Gateway" db:"ip6_gateway"`
IPAddress *string `json:"ipAddress" db:"ip_address"`
IPGateway *string `json:"ipGateway" db:"ip_gateway"`
IPNetmask *string `json:"ipNetmask" db:"ip_netmask"`
LastUpdated *TimeNoMod `json:"lastUpdated" db:"last_updated"`
MgmtIPAddress *string `json:"mgmtIpAddress" db:"mgmt_ip_address"`
MgmtIPGateway *string `json:"mgmtIpGateway" db:"mgmt_ip_gateway"`
Expand All @@ -114,3 +104,15 @@ type DSServer struct {
ServerCapabilities []string `json:"-" db:"server_capabilities"`
DeliveryServiceCapabilities []string `json:"-" db:"deliveryservice_capabilities"`
}

// DSServerV11 contains the legacy format for a Delivery Service Server.
type DSServerV11 struct {
DSServerBase
LegacyInterfaceDetails
}

// DSServer contains information for a Delivery Service Server.
type DSServer struct {
DSServerBase
ServerInterfaces *[]ServerInterfaceInfo `json:"interfaces" db:"interfaces"`
}
2 changes: 1 addition & 1 deletion lib/go-tc/servers.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ func (sii *ServerInterfaceInfo) Value() (driver.Value, error) {
}

// Scan implements the sql.Scanner interface
// expects json.RawMessage and unmarshals to a deliveryservice struct
// expects json.RawMessage and unmarshals to a ServerInterfaceInfo struct
func (sii *ServerInterfaceInfo) Scan(src interface{}) error {
b, ok := src.([]byte)
if !ok {
Expand Down
4 changes: 4 additions & 0 deletions lib/go-util/ptr.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,10 @@ func UintPtr(u uint) *uint {
return &u
}

func Uint64Ptr(u uint64) *uint64 {
return &u
}

func Int64Ptr(i int64) *int64 {
return &i
}
Expand Down
59 changes: 44 additions & 15 deletions traffic_ops/traffic_ops_golang/deliveryservice/eligible.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ import (
"database/sql"
"errors"
"net/http"
"strconv"
"strings"

"github.com/apache/trafficcontrol/lib/go-tc"
Expand Down Expand Up @@ -63,6 +62,26 @@ func GetServersEligible(w http.ResponseWriter, r *http.Request) {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("getting eligible servers: "+err.Error()))
return
}

if inf.Version.Major < 3 {
v11ServerList := []tc.DSServerV11{}
for _, srv := range servers {
v11server := tc.DSServerV11{}
v11server.DSServerBase = srv.DSServerBase

interfaces := *srv.ServerInterfaces
legacyInterface, err := tc.InterfaceInfoToLegacyInterfaces(interfaces)
if err != nil {
api.HandleErr(w, r, inf.Tx.Tx, http.StatusInternalServerError, nil, errors.New("converting to server detail v11: "+err.Error()))
return
}
v11server.LegacyInterfaceDetails = legacyInterface

v11ServerList = append(v11ServerList, v11server)
}
api.WriteResp(w, r, v11ServerList)
return
}
api.WriteResp(w, r, servers)
}

Expand All @@ -86,13 +105,26 @@ s.ilo_ip_gateway,
s.ilo_ip_netmask,
s.ilo_password,
s.ilo_username,
COALESCE(s.interface_mtu, ` + strconv.Itoa(JumboFrameBPS) + `) as interface_mtu,
s.interface_name,
s.ip6_address,
s.ip6_gateway,
s.ip_address,
s.ip_gateway,
s.ip_netmask,
ARRAY (
SELECT ( json_build_object (
Copy link
Copy Markdown
Contributor

@rawlinp rawlinp Jun 15, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this does worry me a bit still. When reviewing @ocket8888's PR I found that this query using json_build_object is like 12x slower than querying the interfaces and IPs separately (without using json_build_object) and manually populating the servers structs with that data. Also, we should really be making the same query here as we would in the /servers API. It seems like we could make this better by sharing that code/query here.

Locally (which I typically assume performs much better than in Prod), for a DS w/ ~1000 servers assigned, this API takes 850ms for me, whereas before this PR, it takes ~75ms.

Copy link
Copy Markdown
Contributor

@ocket8888 ocket8888 Jun 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True enough. This is also still how /servers/details is being done.

'ipAddresses', ARRAY (
SELECT ( json_build_object (
'address', ip_address.address,
'gateway', ip_address.gateway,
'serviceAddress', ip_address.service_address
))
FROM ip_address
WHERE ip_address.interface = interface.name
AND ip_address.server = s.id
),
'max_bandwidth', interface.max_bandwidth,
'monitor', interface.monitor,
'mtu', COALESCE (interface.mtu, 9000),
'name', interface.name
))
FROM interface
WHERE interface.server = s.id
) AS interfaces,
s.last_updated,
s.mgmt_ip_address,
s.mgmt_ip_gateway,
Expand Down Expand Up @@ -132,6 +164,7 @@ AND (t.name LIKE 'EDGE%' OR t.name LIKE 'ORG%')

servers := []tc.DSServer{}
for rows.Next() {
serverInterfaceInfo := []tc.ServerInterfaceInfo{}
s := tc.DSServer{}
err := rows.Scan(
&s.Cachegroup,
Expand All @@ -148,13 +181,7 @@ AND (t.name LIKE 'EDGE%' OR t.name LIKE 'ORG%')
&s.ILOIPNetmask,
&s.ILOPassword,
&s.ILOUsername,
&s.InterfaceMtu,
&s.InterfaceName,
&s.IP6Address,
&s.IP6Gateway,
&s.IPAddress,
&s.IPGateway,
&s.IPNetmask,
pq.Array(&serverInterfaceInfo),
&s.LastUpdated,
&s.MgmtIPAddress,
&s.MgmtIPGateway,
Expand All @@ -180,6 +207,8 @@ AND (t.name LIKE 'EDGE%' OR t.name LIKE 'ORG%')
if err != nil {
return nil, errors.New("scanning delivery service eligible servers: " + err.Error())
}
s.ServerInterfaces = &serverInterfaceInfo

eligible := true

if !strings.HasPrefix(s.Type, "ORG") {
Expand Down
Loading