tool with Apache Arrow, Parquet and DataFusion 朱霜 2023.06.18 1. Introduction 2. Duo - Observability duet: Logging and Tracing • What is Duo? • How does it work? 3. Apache Arrow, Parquet and DataFusion Introduction • ID: Folyd • GitHub: @folyd • 博客: https://folyd.com • ⼯作：字节跳动 (⽕⼭引擎) Duo - Observability duet: Logging and Tracing https://github.com/duo-rs/duo Logging and Tracing Stack powerful reduce complexity? Duo - Observability duet: Logging and Tracing Duo - Observability duet: Logging and Tracing Duo - Observability duet: Logging and Tracing Duo - Observability duet: Logging and Tracing

themself • User Scaling-up/Scaling-down replicate manually or via Kubernetes Autoscaler Observability - Logging • Based on Uber zap • Auto logging with TraceID、ServiceName、Framework Version… Structured logging for friendly searching • Extract business logger from framework logger Observability - Metrics • Based on Kubernetes/Prometheus/Grafana • Builtin framework and Go runtime collectors，Export Prometheus annotations for service discovery and metrics scraping Observability - Metrics Kubernetes Prometheus Annotations Observability - Metrics Naming • Metric name should be snake_case • User metric

instrumentation Przemysław Delewski February 18, 2025 Warsaw, PolandAgenda 1. Short bio 2. Observability domain 3. OpenTelemetry 4. History of OpenTelemetry go compile time instrumentation 5. Demo engineer at Quesma, where we work on a data gateway. Before that, I spent over a decade working on observability at Sumo Logic and Dynatrace in different roles, with a short break in navigation software at happening is crucial for solving problems.Observability - tool for complexityOpenTelemetryWhat is OpenTelemetry 1. Open source implementation of observability concepts 2. Framework and toolkit (set of

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452 13.5. Observability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . the following table will be used: Web Filter Order ServerHttpObservationFilter (Micrometer Observability) Ordered.HIGHEST_PRECEDENCE + 1 WebFilterChainProxy (Spring Security) -100 HttpExchangesWebFilter exclude=* Yaml management: endpoints: jmx: exposure: exclude: "*" 13.5. Observability Observability is the ability to observe the internal state of a running system from the outside. It

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452 13.5. Observability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . the following table will be used: Web Filter Order ServerHttpObservationFilter (Micrometer Observability) Ordered.HIGHEST_PRECEDENCE + 1 WebFilterChainProxy (Spring Security) -100 HttpExchangesWebFilter exclude=* Yaml management: endpoints: jmx: exposure: exclude: "*" 13.5. Observability Observability is the ability to observe the internal state of a running system from the outside. It

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509 13.5. Observability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . exclude=* Yaml management: endpoints: jmx: exposure: exclude: "*" 13.5. Observability Observability is the ability to observe the internal state of a running system from the outside. It register any number of ObservationRegistryCustomizer beans to further configure the registry. Observability support relies on the Context Propagation library for forwarding the current observation across

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509 13.5. Observability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . exclude=* Yaml management: endpoints: jmx: exposure: exclude: "*" 13.5. Observability Observability is the ability to observe the internal state of a running system from the outside. It register any number of ObservationRegistryCustomizer beans to further configure the registry. Observability support relies on the Context Propagation library for forwarding the current observation across

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 507 13.5. Observability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . exclude=* Yaml management: endpoints: jmx: exposure: exclude: "*" 13.5. Observability Observability is the ability to observe the internal state of a running system from the outside. It register any number of ObservationRegistryCustomizer beans to further configure the registry. Observability support relies on the Context Propagation library for forwarding the current observation across

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509 13.5. Observability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . exclude=* Yaml management: endpoints: jmx: exposure: exclude: "*" 13.5. Observability Observability is the ability to observe the internal state of a running system from the outside. It register any number of ObservationRegistryCustomizer beans to further configure the registry. Observability support relies on the Context Propagation library for forwarding the current observation across

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 507 13.5. Observability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . exclude=* Yaml management: endpoints: jmx: exposure: exclude: "*" 13.5. Observability Observability is the ability to observe the internal state of a running system from the outside. It register any number of ObservationRegistryCustomizer beans to further configure the registry. Observability support relies on the Context Propagation library for forwarding the current observation across