Gitlab Runner Service
As a part of the Common Build Infrastructure (CBI), the GitLab Runner Service aims to assist Eclipse projects in enabling GitLab CI within their projects.
It allows the execution of GitLab CI pipelines through runners deployed in the secure and scalable Eclipse Foundation’s Kubernetes infrastructure by allocating the necessary resources.
Projects hosted on gitlab.eclipse.org
can now take advantage of this new service.
The use of this service is currently not extendable to other GitLab instances like gitlab.com , which, beyond the installation and the use of runners, would require additional work from the Eclipse Foundation for managing Eclipse projects on gitlab.com, similar to what exists with github.
|
This service is powered by GRAC! (GitLab Runner As Code), a tool developed by the foundation that enables rapid deployment of this service, simplifying configuration and maintenance.
What is a GitLab runner?
A GitLab Runner is an agent that works with GitLab CI pipeline. It executes tasks (jobs) defined in your project .gitlab-ci.yml
file, by automating various stages such as building, testing, and even deploying code.
For more information, start with: https://docs.gitlab.com/ee/topics/build_your_application.html
Leverage gitlab runner configuration and maintenance with Grac!
The goal of GRAC! is to make the administration of gitlab runner instances on a Kubernetes based infrastructure bearable. It uses automation, templates and a configuration-as-code approach. GRAC! consist of a set of custom shell scripts and a jsonnet generation.
Following the successfull implementation of jenkins instance as code at Eclipse Foundation with JIRO, and over +250 instances in production, there is a demand from projects to extend this support to GitLab CI.
So as a natural extension, GRAC! is born. Under the woods: same approach and technologies.
Grac! helps reduce administration overhead for recurring task like setting up a new gitlab runner for groups/projects, configuration, allowing ressources, handling specific requests, maintaining runners version, etc.
Quick start!
Request a runner
Runner can be requested by filling a ticket on the [helpdesk](https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/new). Please ensure your project lead approved with a +1 the request.
First integration
Create your first .gitlab-ci.yml
file in your project.
default:
tags:
- origin:eclipse = allow to target eclipse runner
stages:
- build
my_build_job:
stage: build
script:
- echo "example job 1"
You can find many template [here](https://docs.gitlab.com/ee/ci/examples/=cicd-templates)
Provided Services
Pipeline template: basic features
The Eclipse Foundation provides a set of functionnality to address needs related to control, construction, and publication. All of these functionalities are available within the [gitlab-ci-templates](https://gitlab.eclipse.org/eclipsefdn/it/releng/gitlab-runner-service/gitlab-ci-templates/) project, and can be imported into project pipelines.
Here is an overview of the features :
* Compliance
: use of librairies [REUSE](https://reuse.software/), [DCO check tool](https://github.com/christophebedard/dco-check), and an ECA script validation [Eclipse Contributor Agreement](https://www.eclipse.org/legal/ECA.php)
* Docker image management
: docker image build based on the buildkit Eclipse Foundation infrastructure and publish to a registry like dockerhub
* Quality
: Control docker image quality with [hadolint](https://github.com/hadolint/hadolint) tool
NOTE about [Gitlab Auto DevOps](https://docs.gitlab.com/ee/topics/autodevops/)
Pipeline template: Auto Devops
GitLab provide what’s called Auto DevOps
which is a comprehensive set of predefined CI/CD pipelines and setting, that aim to streamline and automate the software development lifecycle, from code creation to deployment and monitoring. Auto DevOps is designed to simplify the process by providing a preconfigured CI/CD setup.
This feature clearly provides a time-saving advantage in terms of configuration and CI pipeline deployment. However, this mode is not entirely suited to the operational constraints of the Eclipse Foundation’s infrastructure. That’s why the foundation offers some of these functionalities with an equivalent ready-to-use pipeline for our infrastructure.
Avalaible features: * [Container Scanning](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Container-Scanning.gitlab-ci.yml) * [Sast Analysis](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST-IaC.gitlab-ci.yml) * [Sast IaC Analysis](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST-IaC.gitlab-ci.yml) * [Secret Detection](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Secret-Detection.gitlab-ci.yml) * [Dependency Scanning](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Dependency-Scanning.gitlab-ci.yml)
All autodevops available in a specific pipeline [pipeline-autodevops.gitlab-ci.yml](https://gitlab.eclipse.org/eclipsefdn/it/releng/gitlab-runner-service/gitlab-ci-templates/-/blob/main/pipeline-autodevops.gitlab-ci.yml)
For more information, please read the documentation [here] (https://gitlab.eclipse.org/eclipsefdn/it/releng/gitlab-runner-service/gitlab-ci-templates/-/blob/main/README.md)
Known limitations When using OpenShift to run a GitLab Runner Fleet, we do not support some GitLab features given OpenShift’s security model. Features requiring Docker-in-Docker might not work. |
For Auto DevOps, the following features are not supported yet: * Auto Code Quality * Auto License Compliance (License scanning of CycloneDX files is supported on OpenShift) * Auto Browser Performance Testing * Auto Build * Operational Container Scanning (Note: Pipeline Container Scanning is supported)
Pipeline template: Full feature
We recommand using the full pipeline [pipeline.gitlab-ci.yml](https://gitlab.eclipse.org/eclipsefdn/it/releng/gitlab-runner-service/gitlab-ci-templates/-/blob/main/pipeline.gitlab-ci.yml). Including all feature from: * Compliance * Autodevops * Docker management
Dockerhub publication
The Eclipse Foundation owns the eclipse organization and a couple of other project specific organizations at https://hub.docker.com. You can ask to get a repository being created on one of these organizations. We will set permissions so that committers have write access to this repo (you will need to share your Docker Hub ID with us).
You can also ask us to create a project specific organization. The organization name needs to follow the pattern eclipse-<projectname>.
Note that we don’t grant admin permissions on any Eclipse Foundation owned organization. We recognize that this means that you will have to go through us for all new repo creation, but we can’t grant organizations-wide admin permission assigned to committers for security reasons.
In all cases, requests should be directed to the GitLab HelpDesk.
repo.eclipse.org
see: [Services/Nexus](https://wiki.eclipse.org/Services/Nexus)
Signing tool
see: [CBI Signing tool](https://wiki.eclipse.org/CBI=Signing_tool)
Supply Chain Security Best Practices
The Eclipse Foundation has authored an Open Source Software Supply Chain Best Practices document. We highly recommend Eclipse OSS projects read, understand and adopt these best practices as part of their role in the Supply Chain.
Service Level Agreement (SLA)
Most CBI services are Tier 2 - Best Effort, which means they are expected to be available at all times, and rapid restoration can be expected in the event of an outage. Eclipse Strategic Members can contact Webmaster in certain cases of off-hours support.
Please see [IT_SLA](https://wiki.eclipse.org/IT_SLA) for more information on the Eclipse Foundation IT Services SLA.
Support
-
[HelpDesk](https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/new)
-
[releng room](https://chat.eclipse.org/=/room/=eclipsefdn.releng:matrix.eclipse.org)
-
[gitlab ci templates](https://gitlab.eclipse.org/eclipsefdn/it/releng/gitlab-runner-service/gitlab-ci-templates)
-
[Grac!](https://gitlab.eclipse.org/eclipsefdn/it/releng/gitlab-runner-service/gitlab-runner-as-code)
FAQ
How to target a specific runner
Q: How can I target a specific runner for my GitLab CI/CD job?
A: To target a specific runner for your job, you can use runner tags. Runner tags are labels assigned to runners that help you select the right runner for specific jobs.
Each Eclipse foundation gitlab runner comes with a set of default labels.
I.e:
* prefix:ef-grac
* cluster:okd-c1
* type:groups
* instance:technologie.cbi.grac
* ctx:releng-group
* executor:kubernetes
* kubenamespace:ef-grac-technologie-cbi-grac
* concurrent:8
* outputLimit:100000
* image:ubuntu:20.04
* cpuLimit:8000
* cpuRequest:1000
* memoryLimit:16384
* memoryRequest:2048
* serviceCpuLimit:8000
* origin:eclipse
Q: How do I assign extra tags to runners?
A: Tags can be assigned to runners during runner registration, i.e my-tag, java, docker, etc., based on the capabilities or characteristics of the runner. This can be requested at any time, at runner creation or later by simply filling out a request in the helpdesk support.
Q: How do I use runner tags in my CI/CD job configuration?
A: In your .gitlab-ci.yml file, you can use the tags
keyword at the top level description for pipeline definition or in the job configuration to specify the runner tags that the pipeline or the job should target.
At pipeline level:
default:
tags:
- origin:eclipse
At job level:
my_job:
tags:
- origin:eclipse
script:
- echo "Running on a Eclipse foundation runner"
Q: What if no runner matches the specified tags?
A: If no runner matches the specified tags, the job won’t run. Make sure that your runner tags correspond to the capabilities of your runners and the requirements of your jobs.
Q: Is there a default runner if no tags are specified?
A: GitLab CI will automatically select a runner that doesn’t have any tags assigned if you don’t specify tags for a job. However, for more control and specificity, it’s recommended to use runner tags, espacially the origin:eclipse
tag.