CI4TOSCA Prototype

The CI4TOSCA Prototype is a Go-based tool for generating CI/CD pipelines from TOSCA build descriptions. It serves as the reference implementation of the CI4TOSCA framework, which introduces a declarative build stage to unify artifact creation within a TOSCA model.

This prototype supports Jenkins, GitHub Actions, and GitLab CI pipelines, integrates with Artifactory and MinIO for artifact storage, and features a plugin-based architecture for extensibility.

---

Overview

Traditional DevOps workflows maintain separate scripts for build and deployment, leading to redundancy, complexity, and vendor lock-in. CI4TOSCA addresses this by embedding build instructions directly in the TOSCA model and transforming them into executable pipelines, reducing configuration effort and increasing maintainability.

This prototype was evaluated through 576 pipeline executions and demonstrated:

• An 83% reduction in CI/CD configuration files
• A 40% reduction in lines of configuration code
• Full tool-switching support with no model changes

For evaluation details, see:
=> CI4TOSCA Evaluation

---

Features

• Declarative Build Descriptions: Define build logic directly within TOSCA service templates.
• Pipeline Generation: Generate pipelines for Jenkins, GitHub Actions, and GitLab CI.
• Plugin Architecture: Easily add new artifact types, pipeline tools, and storage backends.
• Artifact Storage Support: Uploads to Artifactory, MinIO, S3, and other backends.
• TOSCA Version Support: Supports TOSCA 1.3, 2.0, and future versions via YAML templates.
• Preview Mode: View generated pipelines without writing them to disk.
• Git Integration: Automatically clones repositories referenced in TOSCA inputs.
• Component-Level Configuration: Assign individual pipeline tools and outputs per service.

---

Installation

Prerequisites

• Go 1.20 or later
• Valid TOSCA build description file (v1.3 or v2.0)

Steps

git clone https://github.com/CI4TOSCA/Prototype
cd Prototype
go mod tidy
go build -o prototype

---

Usage

Generate a Single Pipeline

go run orchestrator.go \
  --tosca path/to/tosca.yaml \
  --tool Jenkins \
  --output Jenkinsfile \
  --storage https://artifact-store.example.com \
  --storageUser user \
  --storagePassword pass

Generate Multiple Pipelines

go run orchestrator.go \
  --tosca path/to/tosca.yaml \
  --tool comp1=GitHub,comp2=Jenkins \
  --output comp1=github.yaml,comp2=jenkinsfile \
  --storage https://artifact-store.example.com \
  --storageUser user \
  --storagePassword pass

Preview a Pipeline

go run orchestrator.go \
  --tosca path/to/tosca.yaml \
  --tool GitHub \
  --storage https://artifact-store.example.com \
  --storageUser user \
  --storagePassword pass \
  --preview

Testing

Run all tests:

go test ./... -v

---

Command-Line Options

Flag	Description
--tosca	Path to the TOSCA input file.
--tool	Pipeline tool (global or per-component).
--output	Output file(s) (global or per-component).
--storage	Artifact storage URL.
--storageUser	User name for the storage backend.
--storagePassword	Password for the storage backend.
--preview	Preview the pipeline without saving it.
--executeURL	URL of pipeline tool.
--executeUser	User for pipeline tool.
--executeToken	Token for pipeline tool.
--notificationMethod	Notification type (e.g. slack).
--notificationWebHook	Webhook URL for notifications.

---

Extending the CI4TOSCA Prototype

The prototype is structured for extensibility. Each plugin type implements a common interface and is registered dynamically.

Add a New Pipeline Tool

1. Implement the Pipeline interface in pipelines/:

type MyCustomPipeline struct {
    BasePipeline
}

func (p *MyCustomPipeline) transform(stages []models.Stage) (string, error) { ... }
func (p *MyCustomPipeline) execute() (string, error) { ... }
func (p *MyCustomPipeline) validate() error { ... }

2. Register it:

func init() {
    pipelines.RegisterPipeline("MyCustomPipeline", func(base pipelines.BasePipeline) pipelines.Pipeline {
        return &MyCustomPipeline{BasePipeline: base}
    })
}

---

Add a New Artifact Type

1. Implement the Artifact interface in artifacts/:

type MyCustomArtifact struct {
    BaseArtifact
}

func (m *MyCustomArtifact) GetType() string {
    return m.Type
}
func (m *MyCustomArtifact) GenerateStages(buildDescription models.BuildDescription, artifactStorageURL string) ([]models.Stage, error) { ... }
func (m *MyCustomArtifact) GetBuildDescription(artifactType string, componentName string, artifactInputs map[string]interface{}) models.BuildDescription { ... }

2. Register it:

func init() {
    artifacts.RegisterArtifact("MyCustomArtifact", func(base artifacts.BaseArtifact) artifacts.Artifact {
        return &MyCustomArtifact{BaseArtifact: base}
    })
}

---

Add a New Storage Backend

1. Implement the ArtifactStorage interface in storage/:

type MyCustomStorage struct { ... }

func NewCustomStorage(artifactStorageURL string) (*MyCustomStorage, error) { ... }
func (s *MyCustomStorage) GenerateUploadCommand(filePath string, repoPath string) (string, error) { ... }
func (s *MyCustomStorage) GetArtifactStorageURLs(filePath string, repoPath string) string { ... }

2. Update the switch logic in storage_handler/NewStorageHandler(...) to support your backend via URL scheme.

---

Add a New Notifier

1. Implement the Notifier interface in notifications/:

type MyNotifier struct {
    WebhookURL string
}

func (m *MyNotifier) Send(message string) error { ... }

2. Register it:

func init() {
    RegisterNotifier("MyNotifier", NewMyNotifier)
}

func NewMyNotifier(url string) Notifier {
    return &MyNotifier{WebhookURL: url}
}

---

Example TOSCA Files

TOSCA 1.3 Example

tosca_definitions_version: tosca_simple_yaml_1_3

node_types:
  my_application:
    derived_from: tosca.nodes.SoftwareComponent
    metadata:
      name: my_application
      version: 1.0.0

    artifacts:
      my_image_source:
        file: "./source.tar.gz"
        type: CI4TOSCA.Sources

      my_application_binary: # For Binary images
        file: my_image
        type: CI4TOSCA.Binary.Image
        interfaces:
          Standard:
            build:
              inputs:
                architecture:
                  type: "x86_64"
                directory: "src/"
    
                build_steps:
                  - step: compile
                    command: "gcc -o my_app main.c"
                  - step: package
                    command: "tar -czf my_app.tar.gz my_app"

      my_application_docker: # For Docker images
        file: "https://github.com/CI4TOSCA/Evaluation/3_pipeline_files/dockerfiles/VueRailsDockerfile"
        type: CI4TOSCA.Docker.Image
        properties:
          image_name: "vue-rails"

      my_application_npm: # For NPM images
        file: "https://github.com/khaledosman/angular-realworld-example-app"
        type: CI4TOSCA.NPM.Image
        properties:
          build_command: "run" # optional

      my_application_maven: # For Maven images
        file: "https://github.com/spring-projects/spring-petclinic"
        type: CI4TOSCA.Jar.Image

      my_application_extern: # For images build by external pipelines
        file: "https://github.com/my-org/my-repo/deploy.yml"
        type: CI4TOSCA.ExternalPipeline.Image
        properties:
          api_token: "XYZ"
          body: "my_body"  # optional

topology_template:
  node_templates:
    my_application:
      type: my_application
      requirements:
        - host:
          node: my_host

    my_host:
      type: tosca.nodes.Compute

TOSCA 2.0 Example

tosca_definitions_version: tosca_2_0

service_template:
  node_types:
    my_application:
      derived_from: tosca.nodes.SoftwareComponent
      metadata:
        name: my_application
        version: 1.0.0

      artifacts:
        my_image_source:
          file: "./source.tar.gz"
          type: CI4TOSCA.Sources

        my_application_binary: # For Binary images
          file: my_image
          type: CI4TOSCA.Binary.Image
          interfaces:
            Standard:
              build:
                inputs:
                  architecture:
                    type: "x86_64"
                  directory: "src/"

                  build_steps:
                    - step: compile
                      command: "gcc -o my_app main.c"
                    - step: package
                      command: "tar -czf my_app.tar.gz my_app"

        my_application_docker: # For Docker images
          file: "https://github.com/CI4TOSCA/Evaluation/3_pipeline_files/dockerfiles/VueRailsDockerfile"
          type: CI4TOSCA.Docker.Image
          properties:
            image_name: "vue-rails"

        my_application_npm: # For NPM images
          file: "https://github.com/khaledosman/angular-realworld-example-app"
          type: CI4TOSCA.NPM.Image
          properties:
            build_command: "run" # optional

        my_application_maven: # For Maven images
          file: "https://github.com/spring-projects/spring-petclinic"
          type: CI4TOSCA.Jar.Image

        my_application_extern: # For images build by external pipelines
          file: "https://github.com/my-org/my-repo/deploy.yml"
          type: CI4TOSCA.ExternalPipeline.Image
          properties:
            api_token: "XYZ"
            body: "my_body"  # optional

  node_templates:
    my_application:
      type: my_application
      requirements:
        - host:
            node: my_host

    my_host:
      type: tosca.nodes.Compute

---

License

This project is licensed under the MIT License. See the LICENSE file for details.