www.egi.eu EGI-Engage is co-funded by the Horizon 2020 Framework Programme

of the European Union under grant number 654142

STFC Daresbury Labs, Warrington, UK

GOCDB A Site/Service Registry and

CMDBdavid.meredith@stfc.ac.k

204/22/23

https://wiki.egi.eu/wiki/GOCDB

Insert footer here

304/22/23

A Configuration Management Database (CMBD) for e-

Infrastructures• Portal+ REST API to register + manage domain objects in an e-Infrstr:

– Projects, NGIs, Sites, Services/Endpoints/Types, ServiceGroups, Downtimes, Users, Roles, Contacts

• Static attributes, manual input + validation, mandatory/optional • Multi-tenant (1 or more projects hosted in same instance) • Comprehensive Role based permissions model• Enforces a number of Business Rules and policies • Extensible; add custom (Key=Value) pairs to domain objects • Fine grained resource filtering/grouping using tagging

• Defines what resources should be present, rather than live/current status of services/infrstr

• Bootstraps other systems: Top BDII, Monitoring, Ops portal, Accounting, ACLs.• GUI is legacy, could be modernised, but the backend Domain-Model is pretty

solid/extensible.

Insert footer here

404/22/23

Domain Model Comparison

Insert footer here

GLUE2 (subset) GOCDB (subset)

GO

CD

B

(NGI)

504/22/23

Projects/Sites/Services/ServiceGroups

Insert footer here5

EGI

EUDAT

EUDAT

EGI

604/22/23

Group Management, Roles, Rules

Insert footer here

R

• Projects, NGIs, Sites + ServiceGroups self-manage their own users: – Users request Roles over objects– Users with existing roles Grant,

Deny, Revoke requests

• Roles enable fine-grained Actions over objects • Enforces a variety of business rules:

• ‘NGI’ or ‘Project’ level role needed to update the CertificationStatus of a child Site (e.g. suspend site)

• Prevents sites self-certifying• Many others…

704/22/23 Insert footer here

Resource Grouping With Scope Tags

Insert footer here

1. Resource owners tag their NGIs, Sites, Services, ServiceGroups with one or more scope tags

2. Tags used to define resource categories/groups without duplicating • Single resource can be tagged multiple times• Maintains integrity of information across different groups, projects, etc…

3. E.g. EGI filters resources to include only ‘EGI’ tagged resources, new tags can be added as required

Service A Service B

Scope Tags Filter using ‘scope’ and ‘scope_match’ (Portal+API)

EGI EGITEST TEST

CLIP

804/22/23

Extension Properties: Add Custom (Key=Value) Pairs toNGIs, Sites, Services, Endpoints,

ServiceGroups

Insert footer here

Sample Glue attributes as extension properties on a ServiceEndpoint

Sample Glue attributes as extension propertieson a Service

904/22/23

REST style API to Query in XML

Insert footer here.... 9

• Queries are filtered using URL parameters

• Proprietary XML

• Similar to GLUE2 XML: flat rather than deeply nested XML docs

• Could render same data in GLUE2 XML/JSON

Extensions followGLUE2 XML

• API is read only• Also published on failover server

(goc.dl.ac.uk, sync’d hourly)

1004/22/23

Current Roadmap

Insert footer here

• Federated Identity Access (SAML/Shib/IdP)– Alternative to x509 to authenticate users– Done; testing underway on gocdb-test

• Improve Role Model for multi-tenant– Projects hosted in same instance can define

different Roles/rules per-project – Done; testing to start soon

• Enhance the Change Logging (EUDAT) – Record every role request, denial, acceptance,

revocation, deletion (Done, released v5.4)– Record every change to a domain object (who did what,

when, pre-post diff). TODO

Coming soon:v5.5

1104/22/23

Future Roadmap (under review)

To Consider: Move GOCDB into the InfoSys space?

Insert footer here

1204/22/23

1. Extend GOC’s data model for InfoSys

1. Add new attributes to existing objects (~trivial)

2. Add new object types to domain model e.g. GLUE2 Share (~doable)

3. Render GOC’s data in GLUE2 XML/JSON (~doable)

2. Browse/upload (key=value) .properties file for adding/updating a bulk of attributes defined on a Site, Service, Endpoint (approved)1. EUDAT: publish K=V template files for their community (or upload xml/json?)2. Approved, see RT: https://rt.egi.eu/rt/Ticket/Display.html?id=9427

3. A REST service to POST .props files / CRUD operations (~doable)– Would enable client-scripting for adding/updating dynamic attributes– Impt: Could use existing Role/Authentication model

1. Existing user registers a new GOCDB account using a host cert2. Use the host cert to request Roles over target sites/services3. Existing user grants role requests 4. Use host cert to authenticate the script on HTTP POST/PUT5. This account can be self-managed as normal; revoke roles, delete…

Insert footer here

onetime

Candidate Items/ Future Roadmap

1304/22/23

SummaryNow:•GOCDB currently supports static attributes + manual input/editing•Role based permissions model enforces a range of business rules/policy•Records what resources should be available, e.g. for bootstrapping BDIIs•Data model is extensible via custom (Key=Val) pairs

Future: Consider moving GOCDB more into the InfoSys space ?•Addition of a REST services for CRUD + dynamic attributes has been discussed in the past, but was not explored further… •Time to re-consider? –Happy to record new RT if requested by TF –Would need some further-investigation, load-testing etc..

Misc/FYI•EUDAT funded new dev on 6mth project + EGI-Engage funding confirmed•I’ll be away for next 2 weeks, but will re-engage after hols

Insert footer here

1404/22/23

Extra slides

Insert footer here

1504/22/23

Resource Filtering using Scope-Tags + Custom Extension Properties

Insert footer here

Filtering by scope Tags in API•get_site&scope=EGI,CLIP&scope_match=any|all

Filtering by custom Extension Properties (Key=value) pairs in API•get_service&extensions=(VO=)AND(VO2=bar) NOT(V04=)

1. Resources can be tagged using one or more Scope Tags2. Allows filtering in Portal and API3. Used to declare project affiliations + resource grouping/categories 4. No duplication of information

Filter using a combination of scope tags and custom properties