Nginx Ingress 源请求 IP 相关 Header 重写问题

问题说明

  • Nginx Ingress 代理下的服务，通过 X-Forwarded-For 和 X-Real-IP 获取请求 IP 时，发现获取到的 IP 都是 Master 节点的 iP
  • 开启一个 Nginx 的 Deployment，配置 log_format 为如下配置
1

    log_format    accesslog '{"log_time":"$time_iso8601","remote_addr":"$remote_addr","status":"$status","request":"$request","req_body":"$request_body","ip_list":"$proxy_add_x_forwarded_for","upstream":"$upstream_addr","upstream_response_time":"$upstream_response_time","req_time":"$request_time","req_host":"$scheme://$host:$server_port","req_status":"$request_completion","user_real_ip_by_CDN":"$http_Ali_Cdn_Real_Ip","x-Forwarded-For":"$http_x_forwarded_for","x-Original-Forwarded-For":"$http_x_original_forwarded_for","X-Real-IP":"$http_x_real_ip"}';
  • 通过 nginx-ingress 访问这个 pod，日志如下
    • x-Original-Forwarded-For 为真实 IP
    • x-Forwarded-For 获取的为 Master 节点的 IP
1

{"log_time":"2023-08-17T02:55:16+00:00","remote_addr":"10.233.82.209","status":"304","request":"GET /XXXXXX/XXXXXX.png HTTP/1.1","req_body":"-","ip_list":"192.168.0.101, 10.233.82.209","upstream":"-","upstream_response_time":"-","req_time":"0.000","req_host":"http://XXXXXX:80","req_status":"OK","user_real_ip_by_CDN":"-","x-Forwarded-For":"192.168.0.101","x-Original-Forwarded-For":"XXX.XXX.XXX.XXX, XXX.XXX.XXX.XXX","X-Real-IP":"192.168.60.101"}
  • ingress 默认会重写 proxy_set_header X-Forwarded-For $remote_addr

问题处理

  • 更改 ingress 所在所在的 namespace 下的 configmap
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

$ k  -n XXXXXX-ingress-nginx get cm
NAME                       DATA   AGE
ingress-nginx-controller   1      149d
kube-root-ca.crt           1      149d
$ k -n XXXXXX-ingress-nginx edit cm ingress-nginx-controller
data:
  allow-snippet-annotations: "true"
  # 新增
  use-forwarded-headers: "true"
  forwarded-for-header: "X-Forwarded-For"
  compute-full-forwarded-for: "true"
  # end
  • 再次访问，查看 nginx pod 日志
    • 发现 x-Forwarded-For 和 X-Real-IP 没有被重写，可正常获取客户端 IP
1

{"log_time":"2023-08-17T03:12:29+00:00","remote_addr":"10.233.69.230","status":"304","request":"GET /XXXXXX/XXXXXX.png HTTP/1.1","req_body":"-","ip_list":"XXX.XXX.XXX.XXX, XXX.XXX.XXX.XXX, 192.168.6.101, 10.233.69.230","upstream":"-","upstream_response_time":"-","req_time":"0.000","req_host":"http://XXXXXX:80","req_status":"OK","user_real_ip_by_CDN":"-","x-Forwarded-For":"XXX.XXX.XXX.XXX, XXX.XXX.XXX.XXX, 192.168.6.101","x-Original-Forwarded-For":"XXX.XXX.XXX.XXX, XXX.XXX.XXX.XXX","X-Real-IP":"XXX.XXX.XXX.XXX"}

参数说明

  • 官方文档：https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#use-forwarded-headers
  • use-forwarded-headers：Ingress 前有负载等设备，设置为 true，如果 Ingress 直接暴露在互联网上或在 L3 负载均衡设备后，则应为 false
    • true：Nginx 将传入的 X-Forwarded-* 头传给 upstream
    • false：Nginx 忽略传入的 X-Forwarded-* 标头，进行重写
  • forwarded-for-header：指定获取请求 IP 的 Header，默认为 X-Forwarded-For
  • compute-full-forwarded-for：将远程地址附加到X-Forwarded-For报头，而不是替换它