GitLab 12.3 with web application firewall and performance analysis
This month’s GitLab 12.3 release is especially interesting after a substantial week in which we held the first GitLab user conference in Brooklyn, New York, and announced the completion of the E-Series funding phase: raised $ 268 million . With this money, we can significantly improve all our offers for DevOps, including monitoring, security and planning.
Web application firewall
Modern web applications are exposed to new risks from everywhere, including every connected client that sends traffic. The Web Application Firewall (WAF) provides monitoring and rules for protecting applications in a production environment. In GitLab 12.3, we present the first version of the web application firewall built into the GitLab SDLC platform. He will be monitoring and reporting security issues for Kubernetes clusters. In future releases, we will expand WAF's capabilities so that it can block malicious traffic, create and manage firewall rules, and receive information in the early stages of development to take action and reduce risks.
Productivity Analytics First Release
The teams that are responsible for delivering the software always need the right information and analytics to increase productivity and efficiency. Too often, inconspicuous bottlenecks and clutters keep them waiting and wasting time instead of taking on new features. Starting with release 12.3, we offer new analytics features to help teams and leaders better understand the productivity and effectiveness of groups and projects. Productivity Analytics helps teams and their leaders find the best ways to increase productivity. Initially focusing on the time it takes to merge merge requests, GitLab will allow you to study the data in detail and find out what and how can be improved. In many organizations, managers are involved in several projects, and the analytic workspace at the group level provides information on productivity and productivity in several projects. These two features are the first in a whole series of updates aimed at providing information and analytics to increase efficiency.
Enhanced Compliance
Compliance with policies and procedures gives developers a lot of problems. It’s easier for many GitLab users to enforce compliance when developers work together in the same application. GitLab 12.3 introduces several features that simplify actions to reduce compliance risks. Merge request approval rules prevent code merging that introduces unsupported licenses. Requiring approval from the code owner for each branch helps protect the branches , because the code owner must approve all changes.
And that's not all!
There are so many cool features in GitLab 12.3 that it’s just impossible to talk about everyone (although I really want to). More convenient viewing of resource information with a global view for cluster-level environments and deployments , more efficient Git retrieval with compressed Git HTTP links in Git , more efficient keystroke checks for the next and previous unresolved discussions .
This Month's Most Valuable Employee ( MVP ) - Cédric Tabin
Through Cedric's efforts, GitLab 12.3 introduced a new keyword for the CI job, which allows interrupted builds . He worked on this feature for over 9 months and collaborated with our review teams to include it in the release.
Thank you more, Cedric, for your invaluable work!
Main features of GitLab 12.3
Web Application Firewall for Kubernetes Ingress
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
GitLab now adds the web application firewall (WAF) modsecurity plugin to the cluster when you install the Ingress application on a Kubernetes cluster.
WAF determines whether incoming HTTP or HTTPS traffic contains malicious code, such as SQL injection, crossite scripting, or trojans. WAF already has effective rules, OWASP ModSecurity Core Rules (CRS), which define different types of attacks without additional configuration.
The documentation describes how to view WAF logs and find out what kind of malicious traffic your application is exposed to in a production environment.
Productivity Analytics
PREMIUM, ULTIMATE, SILVER, GOLD
Now there are relatively few data sources and analytics, and managers need this information to understand the productivity of the team, project and group. As Peter Drucker once said: “What can be measured can be improved.” Guided by this principle, we are releasing the first version of Productivity Analytics to help executives understand the typical patterns and find the best methods to increase overall productivity. This release focuses on how much time it takes to merge a merge request based on size. Users can use existing filters and study detailed entries down to a specific author or label in a group within a specified date range. In future versions of Productivity Analytics, we will add additional data so that we can find dependencies that increase the time of active development or expectations.
In this first release of Productivity Analytics, we did not collect historical data for new metrics so that this background process would not interfere with the transition from 12.2 to 12.3. You can follow the task where we are working on it .
Global view for cluster-level environments and deployments
PREMIUM, ULTIMATE, SILVER, GOLD
It’s convenient for operators to set up a cluster at the group level to provide developers with an application development platform. Scaling cluster resources is not easy. This requires a global view of resource use. The new Environments section of the cluster page provides an overview of all projects that use the Kubernetes cluster, including prepared environments and deployments and the number of Pods in each environment.
Approval of merge requests to prevent merging of prohibited licenses
ULTIMATE, GOLD
If you have strict license restrictions, you can configure License Compliance to prohibit merging when a forbidden license is present in the merge request. This will prevent licenses that are explicitly prohibited. Now you can configure approvers for the License-Check group in the project settings and require verification in accordance with the instructions described in the documentation .
Other improvements in GitLab 12.3
Analytics Workspace
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
Engineers and product specialists can be members of different GitLab groups and projects, but analytics is usually developed at the project level. Therefore, we have created a workspace where users can collect information from different groups, subgroups and projects. The analytics workspace simplifies the analysis and management of team metrics for participants and leaders. Workspace will be available at the Core level. But in some cases, certain features will be available for the Enterprise Edition. As the analytics workspace evolves, we guarantee that existing analytics functionality at the project level will be available to Community Edition users when moving to a new workspace. In GitLab 12.3, we release the first version of Productivity Analytics at the group and project level and Cycle Analytics at the group level. In the next releases, it will be possible to choose different groups and subgroups and transfer all the analytics functions to the instance. We'd love to hear your feedback on a strategy for analytics and value stream management .
Notifications for Design Management
PREMIUM, ULTIMATE, SILVER, GOLD
In GitLab 12.2, we released the first version of Design Management. Continuous development requires users to be notified of these activities. Conversations in designs will now create tasks for the mentioned users and send notifications according to their settings. This ensures that they don’t miss important reviews and can take action. In the next release, we will add these conversations to the main discussion tab for convenience.
API for merge request approval rules
STARTER, PREMIUM, ULTIMATE, BRONZE, SILVER, GOLD
Approval rules for merge requests allow you to specify who should participate in the code review - you assign approvers and the minimum number of approvals. Approval rules are displayed in the merge request widget, so the next reviewer is easy to see.
In GitLab 12.3, support for approval rules has been added to the API for projects and merge requests.
Keyboard shortcuts for next and previous unresolved discussion
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
Testing, discussing, and resolving feedback is at the heart of GitLab code review. With the Jump to next unresolved discussion button, you can easily move from discussion to discussion.
In GitLab 12.3, the new “n” and “p” keyboard shortcuts allow you to go to n ext (next) and p revious (previous) unresolved discussions in the merge requests to make it easier to see the changes.
An API requiring a merge request approval from a code owner on a branch
PREMIUM, ULTIMATE, SILVER, GOLD
The approval of merge requests restricts the sending of code to secure branches, and this allows you to improve the quality of the code and implement compliance control measures. But not all merge requests are designed for stable branches, and not all stable branches require the same control.
In GitLab 12.3, you can require approval from the code owner for some branches (via the API) to prevent changes being sent to files directly or merging changes without approving the code owner.
Note. This feature is only available through the API in GitLab 12.3. In GitLab 12.4, it will be available in the secure branch settings. Follow the news in task 13251 .
Flexible “rules” keyword for controlling pipline behavior
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
only/except
rules in pipelines can imply different implicit actions, and the more you add them, the more difficult it is to understand whether a particular task will be performed in different situations. We introduce the new rules:
syntax, which will greatly simplify the implementation and understanding of complex rules. This syntax is optional and can exist in one pipeline, but not in the same tasks, as the current only/except
approach.
only / except: external_pull_requests for external repositories
CORE, STARTER, PREMIUM, ULTIMATE, SILVER, GOLD
In GitLab CI, you can work with external repositories to use them for version control, and GitLab for CI / CD. Until now, CI_PIPELINE_SOURCE
always showed push, because it was based on a pull
mirror, not an external repository or webhook. Therefore, GitLab incorrectly supported only/except: merge_requests
. In release 12.3, we removed this limitation.
Removing container images from CI / CD
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
The GitLab Container Registry allows users to collect and send images and tags to a project using the GitLab CI / CD. Changes to the Container Registry are made by the CI Registry User service account, which is called from the .gitlab-ci.yml
predefined environment variable CI_REGISTRY_USER
. Previously, a service account could send new tags to the registry, but it lacked permissions to delete these tags. This prevented the removal of images related to branches, which led to additional storage costs and complicated navigation on the registry interface, because there were a lot of extra tags.
In version 12.3, we expanded the permissions of CI_REGISTRY_USER
and allowed it to remove image tags so that it could remove branch-related tags within the normal CI / CD workflow and use GitLab CI / CD to automate cleaning scripts. This task is part of a big epic to reduce the cost of Container Registry thanks to improved storage management.
Verifying domains when running full active DAST scans
ULTIMATE, GOLD
Now you can guarantee that DAST only runs active domain scans that are specifically configured for DAST scanning.
So you’ll be sure that active DAST scans are not accidentally executed in domains that provide content or are used as work.
In passive DAST scans, nothing has changed. They didn’t bother anyone.
SAST Spotbugs Analyzer Updated for Java 11
ULTIMATE, GOLD
The SAST SpotBugs analyzer has been updated and can now scan Java 11 code by setting the SAST_JAVA_VERSION
environment SAST_JAVA_VERSION
in the project.
Pipeline Run Pipeline Button for Marge Requests
PREMIUM, ULTIMATE, SILVER, GOLD
Pipelines for merge requests have recently received a new way to launch a pipeline in the context of a merge request, but you could use only push for this. In this release, we added a button that launches a new pipeline, and restarting failed pipelines is now much easier.
User Defined CI Variables for docker build with Auto DevOps
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
CI variables allow you to customize the execution of processes for building the application in the CI pipeline. Starting with GitLab 12.3, user-defined variables can be accessed at the docker build
stage in Auto DevOps. Data is provided as a new build secret
value.
Derive one or more variables using the AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES
variable, and it will be available for use in docker build
.
Knative for cluster and instance clusters
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
Group and instance clusters now support the installation of Knative, a Kubernetes -based platform for deploying and managing serverless loads. Thanks to this, several projects will be able to use the GitLab Serverless features on the same cluster.
Line Charts for Metric Panels
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
Often users want to choose the type of chart depending on the metric (for example, a line chart for the CPU, a chart with areas for disk space). To do this, we added line charts to improve the dashboard.
Quick steps to add and remove Zoom appointments in tasks
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
In emergency situations, synchronized collaboration is very important. We optimize the process of starting the conference and attracting all the necessary specialists by integrating this function directly into the task using Zoom.
When a user starts a Zoom meeting, he can attach it to the task with a quick action by entering the URL of the meeting (for example, /zoom https://gitlab.zoom.us/s/123456
). A button with direct access to the conference call will appear at the top of the task. When the incident is resolved, the Zoom meeting can be deleted with the /remove_zoom
.
This is a public feature on GitLab.com, and in self-managed instances you need to use the switch. If you want to use this feature in a self-managed GitLab instance, operators can turn on the switch for the issue_zoom_integration
function. In the release of GitLab 12.4 next month, we plan to remove the function switch and make the integration of tasks with Zoom publicly available to all users of self-managed instances.
Geo shows latency for secondary nodes during push operation via Git HTTP
PREMIUM, ULTIMATE
Getting large amounts of data can be time consuming if the user is far away. Replication of repositories with Geo speeds up the process of cloning and obtaining large repositories, since it creates read-only secondary nodes next to the remote user. Secondaries are behind the primary, so GitLab now shows an approximate replication delay when using git push
over HTTP. Users get more information when using the Geo node, they may notice an increase in delay and report it to system administrators.
Due to protocol limitations, this message is not available when using git pull
.
Disabling two-factor authentication for some OAuth providers
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
If an organization uses mandatory two-factor authentication and an identity provider that also uses 2FA , users may be unhappy with double authentication. Thanks to community input, you can now disable 2FA for some OAuth providers in GitLab. So organizations that use providers with 2FA will be much more convenient to log on to GitLab.
Thanks for the contribution, dodocat !
IP restriction supports multiple subnets
ULTIMATE, GOLD
As part of the development of the function of restricting group actions by IP address, GitLab 12.3 provides the ability to specify several IP subnets. This is very convenient for geographically distributed organizations: instead of specifying a single range that is granted too many permissions, large organizations can now restrict incoming traffic depending on specific needs.
GitLab Runner 12.3
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
Today we released GitLab Runner 12.3! GitLab Runner is an open source project,
which is used to run CI / CD jobs and send the results back to GitLab.
Changes:
- Added documentation with the best methods to get started .
- Updated PowerShell ErrorActionPreference documentation .
- Invalid service errors now indicate a specific service .
- Commands compatible with PowerShell Core can now be used to create a directory .
- Fixed exit with zero exit code when cmdlets fail .
- Long path support included .
- A variable with a short token value is provided .
- The default PATH for the secondary image is set .
A complete list of changes can be found in the GitLab Runner change log: CHANGELOG .
Performance enhancements
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
We continue to improve GitLab performance with each release for GitLab instances of any size.
Some improvements in GitLab 12.3:
- The output of tasks and merge requests through the API is accelerated due to the removal of the N + 1 request .
- Quickly delete related events when a parent is deleted .
- GetCommitSignatures RPC ported from Ruby to Go .
- GitLab Shell authorization checks migrated from Ruby to Go .
Status and number of discussions in Design Management
PREMIUM, ULTIMATE, SILVER, GOLD
In GitLab 12.2, we introduced the first version of Design Management, which allows you to load designs directly into tasks. They were loaded on a separate tab in the tasks, and the actions in each version of the design were incomprehensible to users. Now, when loading designs, status icons are added to each version that distinguish new designs from changed old ones. We also added a number of discussions to the designs to give more information to users. We are pleased that these additions to Design Management will improve the collaboration and discussion on GitLab for designers and engineers.
Git HTTP Link Compression
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
When changes are received in the Git repository, the Git server lists all branches and tags in the repository. This is the so-called link declaration, which can weigh many megabytes if the project is large.
In GitLab 12.3, when receiving via HTTP, link declarations will be compressed for supported clients in order to reduce the amount of transmitted data and speed up receiving operations.
On a typical weekday, GitLab.com processes around 850 GB of HTTP link ads. After enabling link compression, this volume was reduced by about 70%.
Audit Logs for Git Push Events (Beta)
STARTER, PREMIUM, ULTIMATE, BRONZE, SILVER, GOLD
Git history can be rewritten to change commits, authors, and timestamps and leave a clear and understandable story for future developers. But for auditing this is a problem.
In GitLab 12.3, Git push events that send commits, rewrite history, or otherwise change the repository can be added to the audit log. Audit logs for push events are disabled by default, so as not to harm the performance of GitLab instances due to high Git write traffic.
In the next release, audit logs for Git push events will be enabled by default. Stay tuned for news in task 7865 .
Smarter default commits in the Web IDE
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
Previously, a commit to the current branch was selected by default in the Web IDE. But in this case, users with permissions could accidentally send changes to the master or other protected branches. Now, when making changes to the Web IDE, the default commit options do not allow sending changes to the wrong branch. Smarter commit options prevent accidental sending to the wizard and secure branches for users with write permissions. If the user does not have write permission, information is provided on why the options are not available. In addition, new commit options support committing to non-default branches with or without an existing merge request.
Different waiting times for jobs in CI / CD pipelines
CORE, STARTER, PREMIUM, ULTIMATE
Different tasks have different execution characteristics, so the waiting time may also vary. You can configure the timeout:
specifying the timeout:
keyword timeout:
in the job in .gitlab-ci.yml
and a number that will indicate how many minutes you need to wait before the job crashes.
Thanks for the contribution, Michal Siwek !
The interruptible keyword indicates whether a job can be safely canceled.
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
Using the new interruptible
keyword, you can specify whether the task should be canceled if it is no longer needed after a new start of the same task. By default, the keyword is false
and can be used for tasks that can be safely stopped. This value can be specified only if automatic unnecessary pipelines cancellation is enabled.
This allows you to avoid duplication of unnecessary tasks in the pipelines, reduce costs and increase the efficiency of pipelines.
Due to a bug in Runner, some executing programs do not stop running tasks after canceling. We plan to fix this in 12.4.
Check Status for Pipeline Triggers
PREMIUM, ULTIMATE, SILVER, GOLD
Recently, we improved the way pipelines in each project launch each other, but one was missing - the triggering pipeline must wait or confirm the successful completion of the next pipeline. It was possible to do this through an API poll, but in this release we introduced depend
and wait
strategies that solve this problem automatically. If you select depend
, the previous pipeline will wait for this pipeline to complete and verify its success before completing the launch task. If you select wait
, the pipeline will wait for completion, but will continue to do its own thing even in the event of a failure.
API Endpoint for Docker Group Tag / Image Output
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
Using the GitLab Container Registry, you can collect and send Docker tags / images to projects from the command line, CI / CD or API. But until the release of GitLab 12.3, we did not provide tag and image information at the group level, although users often ask for it.
We have added two API endpoints that will show which images and tags exist at the group level. This is the first step in improving the visibility and search of the Container Registry. Then we use the API to create a group-level browser in the Container Registry user interface.
SAST scan without Docker-in-Docker
ULTIMATE, GOLD
SAST scans can optionally be performed without Docker-in-Docker.
That is, you can configure SAST scanning so that it does not require elevated privileges.
Editing ignore vulnerabilities
ULTIMATE, GOLD
The reasons for ignoring the vulnerability can now be edited and deleted.
This way you can add and change context for the vulnerability if you have more information.
More convenient initial setup of Pages
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
To work with Pages more conveniently, we have added a banner that notifies users of the approximate initial setup time. We understand how annoying it is when a congratulatory message appears and the page is not available. A banner helps you understand what to expect.
Mapping Kubernetes Cluster Used for Deployment
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
The job details page now displays the name of the Kubernetes cluster that was used for the particular deployment. Project owners and maintainers see a link with the name of the cluster that leads to the cluster details page.
JupyterHub for group-level Kubernetes clusters
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
Group-level clusters now support the installation of JupyterHub , a multi-user service for easily launching notebooks and creating instructions for operators. This extends the availability of JupyterHub for project and group clusters.
Slash command closes task in Slack
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
The solution to modern IT incidents is not complete without chats. This tool should be tightly integrated with the systems you manage and the tools where you fix the situation. It is advisable to minimize the context and switching between tools when you are working on the restoration of services and notifying third-party stakeholders.
In release 12.3, we added an additional slash command to the command set in our Slack-based ChatOps. Now you can close tasks in Slack without opening other tools - just find the task and close it manually. You can close the task right where you work.
Geo natively supports Docker Registry replication
PREMIUM, ULTIMATE
Geo natively supports Docker Registry replication between the primary and secondary Geo nodes. Geo users can now use the Docker Registry at the nearest secondary site. This approach does not consider storage and can be used for storage of objects, for example S3, or local storage.
When using distributed object storage (for example, S3) for the Docker Registry, the primary and secondary Geo nodes can use the same type of storage. This approach does not use native Geo replication.
API actions are included in the group IP address restriction
ULTIMATE, GOLD
GitLab 12.0 introduces a restriction on group actions by IP address . We developed this feature and included actions through the API in it. Incoming requests will now be rejected if they do not meet the group limit. This solves an important problem for enterprises with strict requirements and an advanced approach to access control, since actions in the user interface and through the API are taken into account here.
System hooks for updating project members and groups
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
System hooks provide ample opportunities for automation by launching requests as a result of various events on GitLab. Thanks to community input, changes to project members and groups are now supported in system hooks. This is a great addition for those who need a new level of supervision and automation to change participants.
Thanks for your input, Brandon Williams!
Signing Email with S / MIME
CORE, STARTER, PREMIUM, ULTIMATE, FREE, BRONZE, SILVER, GOLD
GitLab email notifications can now be signed with S / MIME for added instance-level security.
Thanks for the contribution, Siemens, @bufferoverflow and @dlouzan !
Omnibus Enhancements
CORE, STARTER, PREMIUM, ULTIMATE
- SSL authentication for external Postgres servers is now supported by specifying a certificate in gitlab.rb .
- GitLab 12.3 includes Mattermost 5.14 , an open source alternative to Slack , which’s new release includes improved keyboard accessibility, improved integration with Jira, and more. This version includes security updates , and we recommend that you upgrade.
Deprecated Features
.gitlab-ci.yml
, , GitLab 12.3
.gitlab-ci.yml, DEP_SCAN_DISABLE_REMOTE_CHECKS DS_DISABLE_REMOTE_CHECKS
, — .
, .
GitLab 12.0 .
: GitLab 12.3
gitlab-monitor gitlab-exporter.
GitLab Monitor, gitlab-monitor gitlab-exporter. gitlab-exporter — - Prometheus, GitLab, GitLab Monitor , GitLab. Omnibus, gitlab.rb .
: 22 2019 .
.
GitLab 10.0 , . , , .
, .
: 22 2019 .
All Articles