The merger of OpenTracing and OpenCensus: the path to convergence

Authors: Ted Young, Pritam Shah and the Committee for Technical Specifications (Carlos Alberto, Bogdan Drutu, Sergey Kanzhelev and Yuri Shkuro).

The joint project acquired the name: http://opentelemetry.io

Very, very short:

• We are creating a new unified set of libraries and specifications for the possibility of monitoring telemetry. It will combine the OpenTracing and OpenCensus projects, and also provide a supported way for migration.
• The reference implementation in Java will be available on April 24, work on implementations in other languages ​​will begin in full from May 8, 2019. You can view the schedule here .
• By September 2019, parity is planned with existing projects for C #, Golang, Java, NodeJS, and Python. A lot of work awaits us, but we can handle it if we work in parallel. If you are interested in participating in this project, please register and let us know how you would like to contribute.
• When the implementation in each language becomes mature, the corresponding OpenTracing and OpenCensus projects will be closed. This means freezing old projects, and the new project will continue to support existing tools for two years, using backward compatibility.

Project Overview

We are merging! The highest goal is to collect the OpenTracing and OpenCensus projects into one common project.

The core of the new project will be a set of clean and well-thought-out interfaces, including the traditional assembly of libraries that implement these interfaces in the form of so-called SDK The cherry on the cake will be the recommended standards for data and wire protocols, including common infrastructure parts.

The result will be a complete telemetry system suitable for monitoring microservices and other types of modern distributed systems, compatible with most major OSS and commercial core programs.

Main events

April 24 - A reference candidate is submitted for review.

8.05 - A team is formed, starting to work in all languages.

05.20 - The official launch of the project at Kubecon Barcelona.

6.09 - Implementations in C #, Golang, Java, NodeJS, and Python achieve parity with peers.

6.11 - The official completion of the OpenTracing and OpenCensus projects.

11.20 - Farewell party in honor of the completion of projects at the Observability Summit, Kubecon San Diego.

Chronology of convergence

Migration for each language includes an SDK assembly ready for industrial deployment, tools for popular libraries, documentation, CI, backward compatibility tools, and the closure of related OpenCensus and OpenTracing projects (sunset). We set an ambitious goal for September 2019 - to achieve parity for the languages ​​C #, Golang, Java, NodeJS and Python. We will shift the sunset date until all languages ​​are ready. But it is preferable to avoid this.

When viewing goals, please think about your personal participation, let us know by filling out the registration form , or by greeting in the Gitter chats of the OpenTracing and OpenCensus projects . View the chart as an infographic here .

Target: First draft of an interlanguage specification (completion by May 8)

It is important to work together, even when working in parallel in different languages. An interlanguage specification provides guidance for the project. It sounds prosaic, but it is a guarantee of supporting a holistic system that seems familiar regardless of the programming language.

Mandatory requirements for the first draft specification for language X:

• Definitions of general terminology.
• A model for describing distributed transactions, statistics, and metrics.
• Clarifications on important issues that arose during implementation.

This goal blocks the rest of the work, the first draft should be completed by May 8.

Target: first draft for data specification (completion by July 6)

The data specification defines a common data format for traces and metrics, so data exported by all processes can be processed by the same telemetry infrastructure, regardless of the data generation process. This includes the data schema for the trace model described in the cross-language specification. Also included are metadata definitions for common operations that the trace uses to capture: for example, HTTP requests, errors, and database requests. These semantic conventions are an example.

The first draft is based on the current OpenCensus data format and will contain the following:

• A data schema that implements an interlanguage specification.
• Metadata definitions for general operations.
• JSON and Protobuf definitions.
• Implementation of reference customers.

Please note that there is also a wired protocol that distributes in-band traces, and which we would also like to standardize. The Trace-Context distribution format is being developed through the W3C.

Target: Parity for all major supported languages ​​(completed by September 6)

We must achieve parity for the current language ecosystem in the order of replacing old projects with new ones.

• Interface definitions for tracing, metrics, and context distribution based on an interlanguage specification.
• A ready-to-use SDK that implements these interfaces, exporting Trace-Data. If possible, an SDK will be created by migrating an existing implementation from OpenCensus.
• Toolkit for popular libraries currently covered by OpenTracing and OpenCensus.

We also appreciate backward compatibility and want to ensure a smooth transition from existing projects.

• The new SDK will have backward compatibility with current OpenTracing interfaces. They will allow legacy OpenTracing tools to work together with new tools in the same process, which will allow users to gradually transfer their experience.
• When the new SDK is ready, an update plan will be created for current OpenCensus users. As with OpenTracing, legacy tools can continue to work alongside new ones.
• By November, both OpenTracing and OpenCensus will be closed for accepting changes. Backward compatibility with legacy tools will be maintained for two years.

Creating the best-in-class SDK for each language requires a lot of work, and this is what we need most.

Target: core documentation (completion by September 6)

A critical success factor for any open source project is documentation. We want first-class documentation and training tools, our technical writers are the most active developers on the project. Training developers to properly monitor software is one of the most important impacts we want to have on the world.

The following parts of the documentation are minimum necessary to start:

• Project Orientation.
• Observability 101.
• Beginning of work.
• Language Guides (separately for each).

We invite writers of any level! Our new site is based on Hugo using common markup, so it's very easy to contribute.

Target: Registry v1.0 (completion by July 6)

Registry is another critical component, an improved version of OpenTracing Registry .

• Easily find libraries, plugins, installers, and other components.
• Easy management of Registry components.
• You can find out which SDK features are available in each language.

If you are interested in design, interface, and UX, we have a great project for personal participation.

Target: infrastructure for testing and software release (completion by September 6)

In order for us to continue delivering safe code that you can rely on, we have a project commitment to create quality pipelines for testing and releasing software. Please let us know if you can take care of the conveyors for testing, measuring performance and releasing software. We clearly indicate the level of production readiness, and the maturity of the testing infrastructure will be the main deciding factor for us.

Purpose: closing the OpenTracing and OpenCensus projects (completion by November 6)

We plan to start closing old projects from September 6, if the new project reaches parity with them. 2 months later, at a parity of all languages, we plan to close the OpenTracing and OpenCensus projects. This should be understood as follows:

• repositories will be frozen, no more edits will be made.
• A two-year support period is planned for the current toolkit.
• users will be able to upgrade to the new SDK using the same tools.
• a gradual update will be possible.

Join now

We will be happy for any help, as this is a huge project. If you are interested in learning about observability - now is the time!

• Fill out the registration form for direct participation and joining the team.
• Contact us on OpenTracing and OpenCensus Gitter.
• Check out the reference Java implementation that is under development.