wireshark 抓 HTTPs/TLS 流量包

发布时间： 更新时间： 总字数：1643 阅读时间：4m 作者： IP上海 分享 网址

在 Linux 上抓 HTTPs/TLS 流量包

介绍

  • 参考 SSL/TLS 概览
  • NSS 可以写入Key logs(密钥日志)，以便外部程序解密 TLS 连接。Wireshark 1.6.0 及以上版本可以使用这些日志文件来解密数据包。通过 Wireshark -> Edit -> Preferences -> Protocols -> TLS -> (Pre)-Master-Secret log filename 告诉 Wireshark 密钥文件的位置。
  • 通过将环境变量 SSLKEYLOGFILE 设置为指向一个文件，即可启用密钥日志记录。
  • 注意：从 mozilla_projects_nss_nss_3_24_release_notes 开始（仅用于 Firefox 48 和 49 仅在 Firefox 48 和 49 中使用），对于使用 Makefile 的优化编译会默认禁用 SSLKEYLOGFILE 方法（通过 build.sh 使用 gyp 的编译不受影响）。发行商可以在编译时重新启用它（使用 NSS_ALLOW_SSLKEYLOGFILE=1 make 变量）。Firefox 官方二进制文件也是这样做的。(参见 bug 1188657）。值得注意的是，Debian 没有启用 选项，请参见 Debian bug 842292。
  • key log 文件由一系列的行组成，注释行以尖字符（’#’）开头，会被忽略。密文格式为 <Label> <space> <ClientRandom> <space> <Secret>：
  • <Label> 描述了以下 secret.
  • <ClientRandom> is 32 bytes Random value from the Client Hello message, encoded as 64 hexadecimal characters.
  • <Secret> depends on the Label (see below).

The following labels are defined, followed by a description of the secret:

  • RSA: 48 bytes for the premaster secret, encoded as 96 hexadecimal characters (removed in NSS 3.34)
  • CLIENT_RANDOM: 48 bytes for the master secret, encoded as 96 hexadecimal characters (for SSL 3.0, TLS 1.0, 1.1 and 1.2)
  • CLIENT_EARLY_TRAFFIC_SECRET: the hex-encoded early traffic secret for the client side (for TLS 1.3)
  • CLIENT_HANDSHAKE_TRAFFIC_SECRET: the hex-encoded handshake traffic secret for the client side (for TLS 1.3)
  • SERVER_HANDSHAKE_TRAFFIC_SECRET: the hex-encoded handshake traffic secret for the server side (for TLS 1.3)
  • CLIENT_TRAFFIC_SECRET_0: the first hex-encoded application traffic secret for the client side (for TLS 1.3)
  • SERVER_TRAFFIC_SECRET_0: the first hex-encoded application traffic secret for the server side (for TLS 1.3)
  • EARLY_EXPORTER_SECRET: the hex-encoded early exporter secret (for TLS 1.3).
  • EXPORTER_SECRET: the hex-encoded exporter secret (for TLS 1.3)

RSA 形式允许记录使用 RSA 密钥协议的密码套件，是 Wireshark 1.6.0 支持的第一种形式。它已被 CLIENT_RANDOM取代，后者也适用于其他密钥协议算法（如基于 Diffie-Hellman 的算法），并从 Wireshark 1.8.0 开始支持。

The TLS 1.3 lines are supported since NSS 3.34 (bug 1287711) and Wireshark 2.4 (EARLY_EXPORTER_SECRET exists since NSS 3.35, bug 1417331). The size of the hex-encoded secret depends on the selected cipher suite. It is 64, 96 or 128 characters for SHA256, SHA384 or SHA512 respectively.

  • 有两种方法抓包：
    1. 客户端使用浏览器获取对称加密的密钥
    2. 使用 SSL 私钥证书，需满足
    • 加密算法不能使用 DHE、ECDHE
    • 报文须完整，包括 TCP 握手信息

浏览器获取对称加密的密钥方法

安装 wireshark

sudo add-apt-repository universe
# sudo add-apt-repository ppa:wireshark-dev/stable
sudo apt install wireshark

# 配置
sudo dpkg-reconfigure wireshark-common

配置环境变量

# /etc/profile
export SSLKEYLOGFILE=/tmp/sslkeylog.txt

需要重启 Linux 机器

tcpdump 抓包

  • tcpdump 抓包
    • 抓包 tcpdump -i eth0 tcp and port 443 -s0 -nn -w first.cap
  • 使用 浏览器 打开网页，浏览 https 的页面，即可抓取解密的包
  • 打开 wireshark，配置：编辑 > 首选项/Preferences > Protocols > SSL/TLS > (Pre)-Master-Secret log filename 为 SSLKEYLOGFILE 的路径

wireshark 直接抓包

  • 为 wireshark 配置 SSL/TLS 证书
    • 编辑 > 首选项/Preferences > Protocols > SSL/TLS > 导入RSA Key和密码(若存在) > 抓包
  • 为 wireshark 配置 SSLKEYLOGFILE
    • 编辑 > 首选项/Preferences > Protocols > SSL/TLS > (Pre)-Master-Secret log filename 为 SSLKEYLOGFILE 的路径

Python 获取对称加密的密钥方法

Python urllib3 通过配置 SSLKEYLOGFILE 实现抓取 SSL 对称加密密钥，参考

$ export SSLKEYLOGFILE=/tmp/keylogfile.txt
$ pip install requests
$ cat test.py
import requests

resp = requests.get("https://baidu.com")
print(resp.status_code)
print(resp.headers)
print(resp.text)

# 执行 Python 脚本
$ python3 test.py
200
...

# 查看 Key logs
$ cat $SSLKEYLOGFILE
# TLS secrets log file, generated by OpenSSL / Python
CLIENT_RANDOM f96ff44084a6be28bdcf26a96ad673cf36c6f1df3ac43e0cb0c9017dd7e93229 2843cad38b5a3f96c789adc45f07ada5195f29d059779301d5c1db8441e0dfcc536664ba99f97bdf1810466a9e9d6336
  • 采用 tcpdump 抓包、wireshark 解包即可

Golang 获取对称加密的密钥方法

通过 tls 包输出 Key log 到 os.Stdout

	// Typically the log would go to an open file:
	// w, err := os.OpenFile("tls-secrets.txt", os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600)
	w := os.Stdout

	client := &http.Client{
		Transport: &http.Transport{
			TLSClientConfig: &tls.Config{
				KeyLogWriter:       w,
				InsecureSkipVerify: true, // test server certificate is not trusted.
			},
		},
	}
  • server.go
// https://pkg.go.dev/crypto/tls#example-X509KeyPair-HttpServer
package main

import (
	"crypto/tls"
	"log"
	"net/http"
	"os"
	"time"
)

func main() {
	certPem := []byte(`-----BEGIN CERTIFICATE-----
MIIBhTCCASugAwIBAgIQIRi6zePL6mKjOipn+dNuaTAKBggqhkjOPQQDAjASMRAw
DgYDVQQKEwdBY21lIENvMB4XDTE3MTAyMDE5NDMwNloXDTE4MTAyMDE5NDMwNlow
EjEQMA4GA1UEChMHQWNtZSBDbzBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABD0d
7VNhbWvZLWPuj/RtHFjvtJBEwOkhbN/BnnE8rnZR8+sbwnc/KhCk3FhnpHZnQz7B
5aETbbIgmuvewdjvSBSjYzBhMA4GA1UdDwEB/wQEAwICpDATBgNVHSUEDDAKBggr
BgEFBQcDATAPBgNVHRMBAf8EBTADAQH/MCkGA1UdEQQiMCCCDmxvY2FsaG9zdDo1
NDUzgg4xMjcuMC4wLjE6NTQ1MzAKBggqhkjOPQQDAgNIADBFAiEA2zpJEPQyz6/l
Wf86aX6PepsntZv2GYlA5UpabfT2EZICICpJ5h/iI+i341gBmLiAFQOyTDT+/wQc
6MF9+Yw1Yy0t
-----END CERTIFICATE-----`)
	keyPem := []byte(`-----BEGIN EC PRIVATE KEY-----
MHcCAQEEIIrYSSNQFaA2Hwf1duRSxKtLYX5CB04fSeQ6tF1aY/PuoAoGCCqGSM49
AwEHoUQDQgAEPR3tU2Fta9ktY+6P9G0cWO+0kETA6SFs38GecTyudlHz6xvCdz8q
EKTcWGekdmdDPsHloRNtsiCa697B2O9IFA==
-----END EC PRIVATE KEY-----`)
	cert, err := tls.X509KeyPair(certPem, keyPem)
	if err != nil {
		log.Fatal(err)
	}
	cfg := &tls.Config{
		Certificates: []tls.Certificate{cert},
		KeyLogWriter: os.Stdout,
	}
	srv := &http.Server{
		Addr:         ":8000",
		TLSConfig:    cfg,
		ReadTimeout:  time.Minute,
		WriteTimeout: time.Minute,
	}
	log.Fatal(srv.ListenAndServeTLS("", ""))
}

source repo / download raw
	# 获取 ClientHello demo
	cfg := &tls.Config{
		Certificates: []tls.Certificate{*cert},
		GetConfigForClient: func(clientHello *tls.ClientHelloInfo) (*tls.Config, error) {
			collectInfos.collectClientHello(clientHello)
			currentClientHello = clientHello
			return nil, nil
		},
	}
  • client.go
// https://pkg.go.dev/crypto/tls#example-Config-KeyLogWriter
package main

import (
	"crypto/tls"
	"fmt"
	"log"
	"net/http"
	"os"
)

func main() {
	client := &http.Client{
		Transport: &http.Transport{
			TLSClientConfig: &tls.Config{
				KeyLogWriter:       os.Stdout,
				InsecureSkipVerify: true, // test server certificate is not trusted.
			},
		},
	}
	resp, err := client.Get("https://127.0.0.1:8000")
	if err != nil {
		log.Fatalf("Failed to get URL: %v", err)
	}
	resp.Body.Close()

	fmt.Println(resp.StatusCode, resp.Header, resp.Body)
}

source repo / download raw
  • 启动服务
# server
$ go run ./server.go
CLIENT_HANDSHAKE_TRAFFIC_SECRET 53d756ab252a819bd0713fe6b8437e770eaf888a5a79ba6064262fdec2a3ebaa cbcdb4060e505fea35131fef28cab62ef92dd6e628c4beb6c3dd5fac3e4e3dbf
SERVER_HANDSHAKE_TRAFFIC_SECRET 53d756ab252a819bd0713fe6b8437e770eaf888a5a79ba6064262fdec2a3ebaa 825c15a820e077438866a8885105d49b3dedbfbbb473d52a04b11df6671b1f76
CLIENT_TRAFFIC_SECRET_0 53d756ab252a819bd0713fe6b8437e770eaf888a5a79ba6064262fdec2a3ebaa 18e8d1eea03f650649e15fd67a024ef82cf593dd404532d95bbee07901130bd4
SERVER_TRAFFIC_SECRET_0 53d756ab252a819bd0713fe6b8437e770eaf888a5a79ba6064262fdec2a3ebaa 05d694d6f3c9e87c6fa70a0c912616a5180014295b1ae17234452ec67f727edc

# client
$ go run ./client.go
CLIENT_HANDSHAKE_TRAFFIC_SECRET 53d756ab252a819bd0713fe6b8437e770eaf888a5a79ba6064262fdec2a3ebaa cbcdb4060e505fea35131fef28cab62ef92dd6e628c4beb6c3dd5fac3e4e3dbf
SERVER_HANDSHAKE_TRAFFIC_SECRET 53d756ab252a819bd0713fe6b8437e770eaf888a5a79ba6064262fdec2a3ebaa 825c15a820e077438866a8885105d49b3dedbfbbb473d52a04b11df6671b1f76
CLIENT_TRAFFIC_SECRET_0 53d756ab252a819bd0713fe6b8437e770eaf888a5a79ba6064262fdec2a3ebaa 18e8d1eea03f650649e15fd67a024ef82cf593dd404532d95bbee07901130bd4
SERVER_TRAFFIC_SECRET_0 53d756ab252a819bd0713fe6b8437e770eaf888a5a79ba6064262fdec2a3ebaa 05d694d6f3c9e87c6fa70a0c912616a5180014295b1ae17234452ec67f727edc
404 map[Content-Length:[19] Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]] &{0xc000184040 {0 0} true <nil> 0x120dfe0 0x120e0c0}

Nginx 获取 TLS key logs

  • NGINX 版本信息
$ nginx -V
nginx version: nginx/1.18.0 (Ubuntu)
built with OpenSSL 3.0.2 15 Mar 2022
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -ffile-prefix-map=/build/nginx-zctdR4/nginx-1.18.0=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fPIC -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -Wl,-z,now -fPIC' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --modules-path=/usr/lib/nginx/modules --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-compat --with-debug --with-pcre-jit --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_v2_module --with-http_dav_module --with-http_slice_module --with-threads --add-dynamic-module=/build/nginx-zctdR4/nginx-1.18.0/debian/modules/http-geoip2 --with-http_addition_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_sub_module

# 安装依赖
sudo apt install -y libssl-dev

# 编译
cc sslkeylog.c -shared -o libsslkeylog.so -fPIC -ldl

# 复制动态连接库到 /usr/local/lib/
cp libsslkeylog.so /usr/local/lib/
  • 生成证书
cd /etc/nginx

# 生成私钥
openssl genrsa -out server.key 2048

# 生成公钥（证书）
openssl req -new -x509 -key server.key -out server.pem -days 3650
  • 配置 /etc/nginx/nginx.conf
/etc/nginx/nginx.conf ... 
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
	worker_connections 768;
	# multi_accept on;
}

# 新增
env SSLKEYLOGFILE;
env LD_PRELOAD;
#

http {

	##
	# Basic Settings
	##

	sendfile on;
	tcp_nopush on;
	types_hash_max_size 2048;
	# server_tokens off;

	# server_names_hash_bucket_size 64;
	# server_name_in_redirect off;

	include /etc/nginx/mime.types;
	default_type application/octet-stream;

	##
	# SSL Settings
	##

	ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
	ssl_prefer_server_ciphers on;

  # 新增
  ssl_certificate /etc/nginx/server.pem;
  ssl_certificate_key /etc/nginx/server.key;
  ssl_session_cache off;
  ssl_session_tickets off;
  #

	access_log /var/log/nginx/access.log;
	error_log /var/log/nginx/error.log;

	gzip on;

  server {
    listen 443 ssl;

    root /var/www/html;
  }

	include /etc/nginx/conf.d/*.conf;
	include /etc/nginx/sites-enabled/*;
}
  • 启动 nginx
SSLKEYLOGFILE=/tmp/keylogfile.txt LD_PRELOAD=/usr/local/lib/libsslkeylog.so nginx
  • 访问 nginx
$ curl \
  -o /dev/null \
  -v \
  --tlsv1.2 \
  --resolve "t.xiexianbin.cn:443:127.0.0.1" \
  --cacert server.pem \
  https://t.xiexianbin.cn/
  • 查看对称加密密钥
$ tail -f /tmp/keylogfile.txt
tail -f /tmp/keylogfile.txt
SERVER_HANDSHAKE_TRAFFIC_SECRET c3a77115f84538b339cdfd7f83adc2887dedcb948bd119e425858aefbc3aa2f5 2e529237805d1c1f68b8b2ccb55542e037f810a55d80e17f4791587b6019d142b74a6156b1aa7a4659d005e907ff53b5
EXPORTER_SECRET c3a77115f84538b339cdfd7f83adc2887dedcb948bd119e425858aefbc3aa2f5 6aed6f55bc908679c2f6c7c58a46d1d908d79dad24a788153823a884a5dd6c38333c6bbf2d2eb8afbf709ded0c2a1af2
SERVER_TRAFFIC_SECRET_0 c3a77115f84538b339cdfd7f83adc2887dedcb948bd119e425858aefbc3aa2f5 14546a11ec065a28c6e279e27cce15a69e23e8eb85724a767541f2f2d01bd94fc93d5bf06a4fedf517e990a1da693a75
CLIENT_HANDSHAKE_TRAFFIC_SECRET c3a77115f84538b339cdfd7f83adc2887dedcb948bd119e425858aefbc3aa2f5 fc988b90492ce0de264086c9eddd320b7b9ec411a3ed6adf887ae070ad1a90ac58dfe48a973a6c3ca92f4e28bb5b07b6
CLIENT_TRAFFIC_SECRET_0 c3a77115f84538b339cdfd7f83adc2887dedcb948bd119e425858aefbc3aa2f5 eb5f57f4301df44f97e5c08fbb499dd98c48ee0346527cfeb9e33422f2a5bc1b6d7794150c72fcd2514af241f840a983

参考

  1. https://wiki.wireshark.org/TLS
  2. https://hg.mozilla.org/projects/nss/file/cc1d6f0fd6b0ec8fabf6bd249fe1c96c9c3f261d/doc/rst/legacy/key_log_format/index.rst
最近更新
相关文章
最新评论
Home Archives Categories Tags Statistics
本文总阅读量 次 本站总访问量 次 本站总访客数 人