Meet criteria from questionnaire to evaluate FAIRness of training resources

[amyheather]

This questionnaire is currently being developed by dRTP Skills CHARTED project. It will be used to evaluate how FAIR - Findable, Accessible, Interoperable and Reusable - training materials are. At present, they are organising these into essential, recommended and nice-to-have elements. The current questionnaire has been sourced from: https://docs.google.com/spreadsheets/d/1uS71sZPBtd69t12m_rXDlcbM2OLzbBI-0mrvD7ufwr0/edit?usp=sharing

About you and your training resource

N/A - just for purposes of questionnaire

• What is your name?
• What is your email address?
• What is the name of your training resource?
• Is your training resource delivered as part of a wider programme organised by a project or organisation?
• What is the name of your project or organisation?
• What is your organisation's aim in providing the training resource?

What your training resource does

• Do you provide a clear, high-level overview of your training resource to prospective learners?
• Do you provide a clear, high-level overview of your training resource to other educators/trainers?
• Does your description clearly describe the type of learner who should use your training resource?
• Do you publish case studies to show how your training resource has been used by yourself and others? --> See Impact page
• Does your description include a clear set of learning outcomes for the resource?
• Does your description include a clear set of prerequisite knowledge required to begin the resource?
• Does your description include a clear set of related keywords identifying the tools and techniques (or similar) to which the resource relates?
• Does your description include a clear statement of the estimated time commitment required to complete the training?

Your training resource's identity

• Is the name of your training resource unique?
• Is your training resource name free from trademark violations?

How you deliver your training resource

• Does your training resource have a persistent identifier, e.g. DOI?
• Is your training resource intended for self-study?
• Is your training resource available for free?
• Is your training resource delivered online?
• Is your training resource delivered in-person? N/A
• Do you provide any tools/oportunities for your learners to communicate with each other before/during/after training? --> Giscus + GitHub discussions + GitHub issues
• Do you provide learners with computing resources or software to take part in your training resource? e.g. access to virtual machines, HPC services --> No, but mention this and explain why on guide introduction
• Is your training hosted in an established, third-party collaboration system or repository? e.g. Google Suite, GitHub, GitLab, BitBucket, Zenodo
• Does your training resource describe how a learner can get help with using your training resource?
• Do you describe what support, if any, you provide to learners?
• Does your training resource have an e-mail address or forum that is solely for supporting learners?
• Are e-mails to your support e-mail address received by more than one person? --> Multiple emails provided, but encourage github issues/discussion, which are seen by all
• Do you have a process for learners to report problems with your training resource and to manage resolving these issues?
• Is your training resource cross-platform compatible? e.g. can learners carry it out using two or more of Windows, Unix/Linux and Mac OS X, or can be used from within two or more of Internet Explorer, Chrome, Firefox and Safari?
• Do you indicate which sections/modules of your training resource are specific to particular tools/hardware/domains, and which are general?

How you support trainers

N/A

• Do you describe what support, if any, you provide to trainers?
• Do you provide trainers with instructions for delivering your resource? e.g. instructor notes, recommended learner-to-trainer ratios, lesson plans
• Do you provide training for your trainers?
• Do you provide training for external trainers?
• Do you provide info on recommended set-up for trainers?

How you prepare learners for your training

• Are preparatory instructions for learners provided before the training resource is delivered?
• Do your preparatory instructions include a "quick start" guide, that provides a short overview of how to verify that the learner is ready to begin the training resource?
• Do you provide a comprehensive index to all the tools and commands used in your training resource?
• Do you provide troubleshooting information that describes the symptoms and step-by-step solutions for problems and error messages that learners may encounter while following the preparatory instructions? --> Have provided links for NHS readers on problem solving installation of open source. Otherwise, feel to be sufficient, as these are common open tools, with plenty of external resources available for setting up.

Relationships with other training

• Do you list other training resources that would give learners the required prerequisite knowledge to use your training? --> Not addressed. Have pre-requisite pages on DES, RAP and FOSS. Have pretty basic info in book on functions and classes. Suggest they need basic Python and R, but don't have specific resources would recommend, leaving it open.
• Do you list other training resources that your training resource will give learners the prerequesite knowledge to take? --> N/A
• Does your audience description use terminology shared with other training and/or common terminology in your community? e.g. DIRECT Framework roles, CaRCC Job Family --> Yes, just using standard terms.
• Do your learning outcomes use terminology shared with other training and/or common terminology in your community? e.g. DIRECT Framework competencies / skill levels, HPC Certification Forum competencies/skills, prerequisites of other training resources --> Yes, just using standard terms.
• Do your prerequisites use terminology shared with other training and/or common terminology in your community? e.g. DIRECT Framework competencies / skill levels, HPC Certification Forum competencies/skills, learning outcomes of other training resources --> Yes, as just standard programming prerequisites.
• Do your keywords use terminology shared with other training and/or common terminology in your community? --> Yes, they reflect common DES/health/RSE vocabulary.
• Do you re-use any content from other training resources? --> Not directly, but do take inspiration from and acknowledge other resources like the HSMA Little Book of DES

Advertising your training

• Do you have a defined process for advertising your training resource? --> see dissemination.md
• Do you evaluate where learners found your training and how easy it was for them to find? --> On feedback page, we ask them to let us know how they found it
• Does your training resource appear in training registries or collections of materials? e.g. TeSS, Glittr, GOBLET
• Do you have a mailing list of similar mechanism where prospective learners can subscribe to be alerted when training is available? --> N/A
• Do you create machine-readable metadata describing the training resource? --> HTML YAML front-matter and JSON-LD on index.qmd
• Does your metadata use a recognised metadata format? e.g. Bioschemas or RDA --> Schema.org plus the Bioschemas TrainingMaterial 1.0 profile
• Do you include your metadata within the material? e.g. within the slides, as HTML headers or in a README file --> embedded on the homepage as HTML YAML front‑matter and as a JSON‑LD block in the page HTML
• Does your metadata include the persistent identifier of the training resource? --> JSON-LD has DOI and homepage has DOI in human-readable form
• Does your metadata include the availability of the training resource? --> JSON-LD lists licenses
• Does your metadata include the intended audience of the training resource? --> JSON-LD has an audience list
• Does your metadata include the learning outcomes for the training resource? --> JSON-LD has main learning outcomes under teaches
• Does your metadata include the prerequisites for the training resource? --> JSON-LD describes using competencyRequired
• Does your metadata include the keywords for the training resource? --> JSON-LD has keywords
• If your training resource is not openly available, do you still make the metadata, URL and unique identifier of your training resource openly available? --> N/A
• Do you test how easily learners can find your training resources using search engines or chatbots? e.g. Google, ChatGPT --> See findability_checks.md

Your training resource and accessibility

• Have you defined accessibility conventions or standards? e.g. legal requirements, organisational policy, community standards --> W3C Easy Checks, as stated in README
• Does your training resource adhere to defined accessibility conventions or standards? --> See Work through 'Easy Checks – A First Review of Web Accessibility' #188
• Do your preparatory instructions adhere to defined accessibility conventions or standards? --> N/A - part of site, covered in prior bullet
• Do you consider different levels of accessibility? (e.g. disabilities, neurodivergence, other personal constraints) --> Yes, within the scope of the W3C lightweight framework
• Do you use any tools to evaluate or verify that your training resource adheres to accessibility conventions or standards? --> See CONTRIBUTING.md for details
• Do you provide your training resource in a natural language that matches the target audience?
• Do you provide your training resource using terminology or examples that matches the target audience?
• Do you provide variants of your training resource to better match different audiences? e.g. in multiple languages, using different examples --> Available with two languages: Python or R
• If your training resource is not openly available, do you describe how learners may gain access to it in clear language? N/A
• If your training resource is not openly available, are the instructions for gaining access and metadata available in a natural language that matches the target audience? N/A

Open standards and your training resource

• Is your training resource source material written in an open format? e.g. OpenDocument format (Google Suite, OpenOffice), text files including .md files, data files, image formats
• Does your training resource make use of data in open data formats? e.g. GIF, SVG, HTML, XML, tar, zip, CSV, JSON, NetCDF, or domain specific ones
• Does your training resource use communication using open communications protocols? e.g. HTTP, FTP, XMPP, SOAP over HTTP, or domain-specific ones --> Yes. The training resource is delivered as a static website over standard HTTP/HTTPS, which are open, widely adopted Internet protocols

Open standards and your training resource

• Is your training resource architecture and design modular?
• Does your training resource use an accepted training design or convention? --> The design is based on common practice and FAIR training guidance.
• Do you review your training resource on a regular basis?
• Do multiple people contribute to the development of your training resource?

How you manage versions of your training resource

• Is your training resource source material stored in a repository under revision control?
• Are releases tagged in the repository?
• Is there a branch of the repository that is always stable? (i.e. ready for a course to be carried out)
• Do you back-up your repository?
• Do you store your learner or trainer instructions under revision control with your training resource source material?
• Do you publish your release history e.g. release data, version numbers, key features of each release etc. on your web site or in your training resource?

Managing source material for your training resource

• Do you create your training resource using source material in a different format than that in which it is delivered? e.g. PDF documents from Powerpoint, HTML from Markdown
• Do you have instructions for creating the training resource from the source material?
• Do you create your training resource from the source material using an automated tool? e.g. Make, ANT, Maven, CMake, Python setuptools, or R package tools
• Does your documentation list all third-party dependencies required to create the training resource from the source material?
• Does your documentation list the version number for all third-party dependencies required to create the training resource from the source material?
• Does your software list the web address, and licences for all third-party dependencies required to create the training resource from the source material and say whether the dependencies are mandatory or optional?
• Can you download dependencies required to create the training resource from the source material using a dependency management tool or package manager? e.g. Ivy, Maven, Python pip or setuptools, PHP Composer, Ruby gems, or R PackRat
• Do you have tests or verification steps that can be run after your training resource has been created from the source material to show whether the creation has been successful? --> Created by GitHub actions, will fail and send error if this is unsuccesful

How you test your training resource

• Do you collect feedback from your learners?
• Do you have a process for interpreting that that feedback in order to improve your resources? --> No, not felt to be necessary to have a formal process for this resource.
• Do you use a common template for collecting feedback? --> No, prefer to leave open-ended for this resource.
• Do you do 'peer-review' of your training resource & its delivery?
• Do you have an automated test suite to verify your training resource? e.g. to verify commands work successfully, to check spelling/structure --> Quarto has a validator, it won't run if YAML etc. are valid, it will also fail if any of the active code fails. Where possible, all code runs when book renders, so problems in code will be identified as it will fail to build.
• Do you have a framework to periodically (e.g. nightly) run your tests on the latest version of the source material? --> N/A
• Do you use continuous integration, automatically running tests whenever changes are made to your source material? --> N/A
• Are your test results publicly visible? --> N/A
• Are all manually-run tests documented? --> N/A
• Do you check that your metadata is correct, e.g. using a validator? --> Quarto has built-in validator for YAML

How you engage with your community

• Do you have an established community of learners? --> Not currently
• Does your project or organisation have resources (e.g. blog, Twitter, RSS feed, Facebook page, wiki, mailing list) that are regularly updated with information about your training resources? e.g. new training resources, availability of training --> No, but won't set up, as there won't be changes like this.
• Do you provide success stories or endorsements from learners who have used your training resource? --> None presently, but could incorporate on homepage or Impact page if received
• Do you list important partners and collaborators that are involved in your training resource?
• Do you list third-party publications that refer to your training resource or link to a resource where these are available? --> On "Impact" page
• Can learners subscribe to notifications of changes to your training resource? e.g. new/updated/deprecated material --> No, won't be adding this.
• If your training resources are open source, do you have a governance model? --> No, we don’t currently have a documented governance model (for reference here, a governance model is referring to us having defined roles like maintainer or contributor, a decision making process for how changes are approved and release process, as well as succession and maintenance plans).

How you manage contributions

• Do you accept contributions (e.g. bug fixes, enhancements, documentation updates, tutorials) from people who are not part of your project or organisation?
• Do you have a contributions policy?
• Is your contributions' policy publicly available?
• Do contributors keep the copyright/IP of their contributions? MIT licence.
• Do you require contributors have persistent identifiers, e.g. ORCID? --> Not required but list if provided
• Does your metadata include the persistent identifiers of the contributors?
• Do you publicly list contributions to your training resource?
• Do you define contributions using recognised roles, e.g. CRediT? --> I use All Contributors Contribution Types

Your training resource's copyright and licensing

• Does your documentation clearly state the copyright owners of your training resource and instructions? --> Documentation states who to cite. LICENSE file says copyright STARS.
• Does each of your source files include a copyright statement? --> No, I rely on repository-level LICENSE file.
• Does your documentation clearly state the licence of your training resource?
• Is your training resource released under an open source licence?
• Is your training resource released under an OSI-approved open-source licence?
• Does each of your source files include a licence header? --> No, I rely on repository level LICENSE file
• Do you have a recommended citation for your training resource?
• Have you ensured that your licence is suitable with your specific institute/organisational rules or guidance?
• Do you appropriately cite materials used in your training resource, or that inspired the resource?

Your plans for the future

• Does your project or organisation provide a training roadmap (a list of planned training for the next 3, 6 and 12 months)? N/A
• Do you provide a description of how your training resource is funded, and the period over which funding is guaranteed?
• Do you evaluate how long you will be able to support your training resource for? --> No plans.
• Do you evaluate how long your training resource will remain relevant for? --> No plans.
• Do you make timely announcements of the deprecation of training resources? --> No plans.