OpenTelemetry

otel

installation

doc https://opentelemetry.io/docs/collector/installation/

❯ helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
"open-telemetry" has been added to your repositories
❯ helm pull open-telemetry/opentelemetry-collector
❯ tar -zxvf opentelemetry-collector-0.102.1.tgz  -C ./monitor

❯ helm template my-release ./opentelemetry-collector/  --values ./opentelemetry-collector/value.yaml
Error: execution error at (opentelemetry-collector/templates/NOTES.txt:6:3): [ERROR] 'mode' must be set. See https://github.com/open-telemetry/opentelemetry-helm-charts/blob/main/charts/opentelemetry-collector/UPGRADING.md for instructions.

Use --debug flag to render out invalid YAML

mode**=**<daemonset**|**deployment**|**statefulset>

 exporters:
   debug: {}
   otlp:
     endpoint: local.org:11800
     tls:
       insecure: true
receivers: 
   prometheus:
     config:
       scrape_configs:
        - job_name: 'nginx-monitoring'
          scrape_interval: 10s
          metrics_path: "/metrics"
          static_configs:
             - targets: ['local.org:9145']
               labels:
                  service: nginx-service
                  service_instance_id: e2e-test-instance
        - job_name: 'apisix-monitoring'
          scrape_interval: 10s
          metrics_path: "/apisix/prometheus/metrics"
          static_configs:
             - targets: ['apisix-prometheus-metrics.apisix:9091']
               labels:
                 skywalking_service: showcase-apisix-service
        - job_name: "vm-monitoring"
          scrape_interval: 12s
          static_configs:
             - targets: ["k8s01:9100","k8s02:9100","k8s03:9100","k8s04:9100","k8s05:9100","k8s06:9100","k8s07:9100","k8s08:9100"]

exporter.yaml

<a href="/tool/apm/exporter/#xx-exporter" target="_blank">exporter.yaml</a>

service

nginx

source https://github.com/nginxinc/nginx-otel

deb https://nginx.org/packages/debian/pool/nginx/n/

仓库版本nginx-module-otel_1.26.x+0.1.0-x ,openrestry 1.25.x (截止2024.9.8 )

nginx: [emerg] module “/opt/proxy/openresty/nginx/modules/ngx_otel_module.so” version 1026000 instead of 1025003 in /opt/proxy/openresty/nginx/conf/nginx.conf:18

单独编译

❯ git clone --depth 1 https://github.com/nginxinc/nginx-otel.git
❯ mkdir build && cd build &&   cmake -DNGX_OTEL_NGINX_BUILD_DIR=/home/cs/data/modules/nginx/nginx-1.26.2/objs  ..
....
-- Configuring done
-- Generating done
-- Build files have been written to: /opt/proxy/modules/nginx-otel/build

❯ make

跟openresty一起编译

src=/home/cs/data/modules/nginx/src  &&  \
./configure --prefix=/opt/proxy/openresty --without-http_memcached_module  --with-http_stub_status_module  --with-http_ssl_module --with-http_gzip_static_module --with-http_v2_module --with-luajit --with-pcre-jit --with-stream=dynamic  --with-http_geoip_module=dynamic   --with-stream_geoip_module=dynamic   --with-pcre=$src/pcre2-10.39   --with-zlib=$src/zlib-1.3.1  --with-openssl=$src/openssl-3.3.1   --with-http_random_index_module    --add-dynamic-module=/opt/proxy/modules/nginx-accesskey-2.0.3  --add-dynamic-module=/opt/proxy/modules/nginx-otel 

nginx.conf

nginx.conf demo
load_module modules/ngx_otel_module.so;


http {


##################################################### 
  #https://nginx.org/en/docs/ngx_otel_module.html  
  otel_exporter {
        endpoint localhost:4317;
            interval    5s;
            batch_size  512;
            batch_count 4;
    }


    otel_service_name  "openresty-1.25.3";
    # otel_trace on; #http处全局开启


    # trace 10% of requests
    split_clients $otel_trace_id $ratio_sampler {
        90%     on;
        *       off;
    }


    # or we can trace 10% of user sessions
    split_clients $cookie_sessionid $session_sampler {
        90%     on;
        *       off;
    }
##################################################### 


   log_format  local  '$remote_addr - $upstream_addr - [$time_local] $status "$request $http_referer" $upstream_bytes_sent $http_host';


   server {
       listen       443 ssl;
       server_name  local.org  ~^([\w-]+)\.local\.org$;


        ssl_certificate      conf.d/ssl/local.org/local.org.server.crt;
        ssl_certificate_key  conf.d/ssl/local.org/local.org.server.key;


        ssl_session_cache    shared:SSL:1m;
        ssl_session_timeout  5m;


        ssl_ciphers  HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers  on;


        access_log  logs/local.log  local;


         location / {


         #####################################################
            otel_trace $ratio_sampler;
            otel_trace_context propagate;
         #####################################################


            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;


            proxy_pass http://xxxxx;


        }


    }


}  

以上编译nginx-otel操作会自动安装依赖，需要翻

❯ tree ./ -L 2
 .
├──  _deps
│ ├──  grpc-build
│ ├──  grpc-src
│ ├──  grpc-subbuild
│ ├──  otelcpp-build ## https://github.com/open-telemetry/opentelemetry-cpp/blob/main/INSTALL.md
│ ├──  otelcpp-src
│ └──  otelcpp-subbuild
├──  cmake_install.cmake
├──  CMakeCache.txt
├──  CMakeFiles
│ ├──  3.25.1
│ ├──  cmake.check_cache
│ ├──  CMakeDirectoryInformation.cmake
│ ├──  CMakeOutput.log
│ ├──  CMakeRuleHashes.txt
│ ├──  CMakeScratch
│ ├──  Makefile.cmake
│ ├──  Makefile2
│ ├──  ngx_otel_module.dir
│ ├──  pkgRedirects
│ ├──  progress.marks
│ └──  TargetDirectories.txt
├──  gens
├──  http_archives
│ └──  opencensus-proto-0.3.0
├──  Makefile
├──  ngx_otel_module.so
├──  opentelemetry
│ └──  proto
└──  protos
├──  src
└──  test