How to more accurately detect OpenSSL 3.0 and later versions from the shared object libssl.so

[cfc4n]

background

@xxxxxliil reported a defect in #609 and determined that this was a problem caused by the failure to search for version number information in the dynamic link library corresponding to OpenSSL 3.0 and newer.

这个问题本质上是 openssl 3 以上的 libssl.so .rodata 段中不再存在版本信息，且开 issue 时对于检查不到版本且根据文件后缀回退到默认的 3.0.0 时的日志等级为 info，导致使用时很难注意到是版本没有正确匹配导致的输出信息有误，同时发生的现象还有 keylog 和 pcap 都不能进行抓取解密的 tls 流量。

@xxxxxliil gave a fix in PR #661. After discussions between @cfc4n and @blaisewang, I have a clearer understanding of these differences, and for this reason, I have given a more precise fix.

This article explains some of the considerations for the new approach.

OpenSSL changes

In 2019, OPENSSL_VERSION_TEXT information is stored in include/openssl/opensslv.h (Generated by build.info )

refer:

openssl/openssl@f386632
openssl/openssl#10203

Later, the OpenSSL 1.1.x version will build libssl.so\libcrypto.so, and the .rodata section in the two link library files contains OPENSSL_VERSION_TEXT information.

• 1.1.x : opensslv.h--> crypto.h --> ssl_init.h
• 3.x : opensslv.h--> crypto.h --> ???

After OpenSSL 3.0, the introduction of the header file was changed. In the relevant code of libssl.so, OPENSSL_VERSION related strings are no longer referenced, so the OPENSSL_VERSION_TEXT information is no longer included in the 3.0 version of libssl.so.

via #661 (comment)

Solutions

• When the version symbol OPENSSL_VERSION_TEXT cannot be detected from libssl.so.*, determine whether the current dynamic link library is libssl.so.3
• If so, try to read the libcrypto.so.3 file in the same directory and detect the version number information.
• If not, the original process will be followed and the default version number will be used.
• If version number information is detected, the bpf bytecode of the corresponding version is used.
• If no version number is detected, the default version number is used.

potential problems

1. Trying to read the libcrypto.so.3 file may not be in the same directory as libssl.so.3.
2. There may be a version mismatch between the two files.

For these problems, it should be a small probability event, unless you manually install many versions and mess up the system environment. However, some warning information will be printed to prompt the user to perform manual detection.

I will send a new PR to fix it.

refer

openssl/openssl#10203
openssl/openssl#19801

[cfc4n]

@xxxxxliil @blaisewang Have a better idea?

[blaisewang]

Currently, I don’t have a better idea. I believe the workaround suggested by @xxxxxliil is feasible. While dynamic linking doesn’t necessarily guarantee accuracy, it is still a good solution.

In the long term, I think a better approach might be to engage with the OpenSSL community/developers, review the OpenSSL code, and identify an appropriate location to embed version-specific information directly into the libssl.so binary. We need to explain the significance of the eCapture project and clarify potential misunderstandings, hoping that the OpenSSL community can understand our work. If this can be accomplished, we won’t have to worry about this issue in the future.

[xxxxxliil]

除非有一种自动从内存中搜索各种偏移量的方案，否则不太可能存在比这个更好的方案了。
跑个题，如果特定软件自己提供了一份更改过偏移量的 *ssl/tls，有什么方法可以适配吗？

[xxxxxliil]

I will send a new PR to fix it.

我希望可以在 #661 做到你希望的更改，请告诉我应该怎么做

[cfc4n]

@xxxxxliil 查看这个diff文件，理解改动点的作用，也完全替换到 #661内的修改，只需要在commit message的最后面，增加 Co-authored-by: CFC4N <cfc4n.cs@gmail.com> 即可。

Subject: [PATCH] release: v0.8.10
---
Index: user/module/probe_openssl.go
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/user/module/probe_openssl.go b/user/module/probe_openssl.go
--- a/user/module/probe_openssl.go	(revision fcddaebc87ff4f8c4991f6f3b5870b300fcded2b)
+++ b/user/module/probe_openssl.go	(date 1732636820000)
@@ -198,8 +198,71 @@
 		}
 	}
 
-	// 未找到对应的bpf文件，尝试从so文件中获取
-	err := m.detectOpenssl(soPath)
+	err, verString := m.detectOpenssl(soPath)
+	if errors.Is(err, ErrProbeOpensslVerNotFound) {
+		// 未找到版本号， try libcrypto.so.x
+		if strings.Contains(soPath, "libssl.so.3") {
+			m.logger.Warn().Err(err).Str("soPath", soPath).Msg("OpenSSL/BoringSSL version not found.")
+			m.logger.Warn().Msg("Try to detect libcrypto.so.3. If you have doubts")
+			m.logger.Warn().Msg("See https://github.com/gojue/ecapture/discussions/675 for more information.")
+
+			// 从libssl.so.3中获取libcrypto.so.3的路径
+			var libcryptoName = "libcrypto.so.3"
+			var imd []string
+			imd, err = getImpNeeded(soPath)
+			if err == nil {
+				for _, im := range imd {
+					// 匹配 包含crypto.so字符的动态链接库库名
+					if strings.Contains(im, "crypto.so") {
+						libcryptoName = im
+						break
+					}
+				}
+			}
+			soPath = strings.Replace(soPath, "libssl.so.3", libcryptoName, 1)
+			m.logger.Info().Str("soPath", soPath).Str("imported", libcryptoName).Msg("Try to detect imported crypto.so ")
+			err, verString = m.detectOpenssl(soPath)
+			if err != nil {
+				if !errors.Is(err, ErrProbeOpensslVerNotFound) {
+					return err
+				} else {
+					m.logger.Warn().Err(err).Str("soPath", soPath).Msg("OpenSSL(libcropty.so.3) version not found.")
+				}
+			} else {
+				m.logger.Info().Str("soPath", soPath).Str("imported", libcryptoName).Str("version", verString).Msg("OpenSSL/BoringSSL version found from imported crypto.so")
+			}
+		}
+	}
+
+	if err != nil {
+		m.logger.Error().Str("soPath", soPath).Err(err).Msg("OpenSSL/BoringSSL version check failed")
+		return err
+	}
+
+	m.conf.(*config.OpensslConfig).SslVersion = verString
+	m.logger.Info().Str("origin versionKey", verString).Str("versionKeyLower", verString).Send()
+	// find the sslVersion bpfFile from sslVersionBpfMap
+
+	isAndroid := m.conf.(*config.OpensslConfig).IsAndroid
+	androidVer := m.conf.(*config.OpensslConfig).AndroidVer
+	bpfFileKey := verString
+	if isAndroid {
+		// sometimes,boringssl version always was "boringssl 1.1.1" on android. but offsets are different.
+		// see kern/boringssl_a_13_kern.c and kern/boringssl_a_14_kern.c
+		// Perhaps we can utilize the Android Version to choose a specific version of boringssl.
+		// use the corresponding bpfFile
+		bpfFileKey = fmt.Sprintf("boringssl_a_%s", androidVer)
+	}
+	bpfFile, found := m.sslVersionBpfMap[bpfFileKey]
+	if found {
+		m.sslBpfFile = bpfFile
+		m.logger.Info().Bool("Android", isAndroid).Str("library version", bpfFileKey).Msg("OpenSSL/BoringSSL version found")
+		return nil
+	}
+
+	bpfFile = m.getSoDefaultBytecode(soPath, isAndroid)
+	m.sslBpfFile = bpfFile
+	m.logger.Error().Str("sslVersion", sslVersion).Str("bpfFile", bpfFile).Msg("OpenSSL/BoringSSL version not found, used default version, if you want to use the specific version, please set the sslVersion parameter with `--ssl_version=\"openssl x.x.x\"`")
 	return err
 }
 
Index: user/module/probe_openssl_lib.go
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/user/module/probe_openssl_lib.go b/user/module/probe_openssl_lib.go
--- a/user/module/probe_openssl_lib.go	(revision fcddaebc87ff4f8c4991f6f3b5870b300fcded2b)
+++ b/user/module/probe_openssl_lib.go	(date 1732636693194)
@@ -16,8 +16,8 @@
 
 import (
 	"debug/elf"
+	"errors"
 	"fmt"
-	"github.com/gojue/ecapture/user/config"
 	"os"
 	"regexp"
 	"strings"
@@ -50,6 +50,11 @@
 	SupportedOpenSSL34Version0    = 0 // openssl 3.4.0
 )
 
+var (
+	ErrProbeOpensslVerNotFound         = errors.New("OpenSSL/BoringSSL version not found")
+	ErrProbeOpensslVerBytecodeNotFound = errors.New("OpenSSL/BoringSSL version bytecode not found")
+)
+
 // initOpensslOffset initial BpfMap
 func (m *MOpenSSLProbe) initOpensslOffset() {
 	m.sslVersionBpfMap = map[string]string{
@@ -147,27 +152,27 @@
 
 }
 
-func (m *MOpenSSLProbe) detectOpenssl(soPath string) error {
+func (m *MOpenSSLProbe) detectOpenssl(soPath string) (error, string) {
 	f, err := os.OpenFile(soPath, os.O_RDONLY, os.ModePerm)
 	if err != nil {
-		return fmt.Errorf("can not open %s, with error:%v", soPath, err)
+		return fmt.Errorf("can not open %s, with error:%v", soPath, err), ""
 	}
 	r, e := elf.NewFile(f)
 	if e != nil {
-		return fmt.Errorf("parse the ELF file  %s failed, with error:%v", soPath, err)
+		return fmt.Errorf("parse the ELF file  %s failed, with error:%v", soPath, err), ""
 	}
 
 	switch r.FileHeader.Machine {
 	case elf.EM_X86_64:
 	case elf.EM_AARCH64:
 	default:
-		return fmt.Errorf("unsupported arch library ,ELF Header Machine is :%s, must be one of EM_X86_64 and EM_AARCH64", r.FileHeader.Machine.String())
+		return fmt.Errorf("unsupported arch library ,ELF Header Machine is :%s, must be one of EM_X86_64 and EM_AARCH64", r.FileHeader.Machine.String()), ""
 	}
 
 	s := r.Section(".rodata")
 	if s == nil {
 		// not found
-		return fmt.Errorf("detect openssl version failed, cant read .rodata section from %s", soPath)
+		return fmt.Errorf("detect openssl version failed, cant read .rodata section from %s", soPath), ""
 	}
 
 	sectionOffset := int64(s.Offset)
@@ -177,12 +182,12 @@
 
 	_, err = f.Seek(0, 0)
 	if err != nil {
-		return err
+		return err, ""
 	}
 
 	ret, err := f.Seek(sectionOffset, 0)
 	if ret != sectionOffset || err != nil {
-		return err
+		return err, ""
 	}
 
 	versionKey := ""
@@ -191,7 +196,7 @@
 	// OpenSSL 3.2.0 23 Nov 2023
 	rex, err := regexp.Compile(`(OpenSSL\s\d\.\d\.[0-9a-z]+)`)
 	if err != nil {
-		return nil
+		return err, ""
 	}
 
 	buf := make([]byte, 1024*1024) // 1Mb
@@ -233,46 +238,55 @@
 	_ = f.Close()
 	//buf = buf[:0]
 
-	var bpfFile string
-	var found bool
-	if versionKey != "" {
-		versionKeyLower := strings.ToLower(versionKey)
-		m.conf.(*config.OpensslConfig).SslVersion = versionKeyLower
-		m.logger.Info().Str("origin versionKey", versionKey).Str("versionKeyLower", versionKeyLower).Msg("OpenSSL/BoringSSL version found")
-		// find the sslVersion bpfFile from sslVersionBpfMap
-		bpfFile, found = m.sslVersionBpfMap[versionKeyLower]
-		if found {
-			m.sslBpfFile = bpfFile
-			return nil
-		}
-	}
+	if versionKey == "" {
+		return ErrProbeOpensslVerNotFound, ""
+	}
+
+	versionKeyLower := strings.ToLower(versionKey)
+
+	return nil, versionKeyLower
+}
+
+func (m *MOpenSSLProbe) getSoDefaultBytecode(soPath string, isAndroid bool) string {
+	var bpfFile string
 
-	isAndroid := m.conf.(*config.OpensslConfig).IsAndroid
-	androidVer := m.conf.(*config.OpensslConfig).AndroidVer
 	// if not found, use default
 	if isAndroid {
-		// sometimes,boringssl version always was "boringssl 1.1.1" on android. but offsets are different.
-		// see kern/boringssl_a_13_kern.c and kern/boringssl_a_14_kern.c
-		// Perhaps we can utilize the Android Version to choose a specific version of boringssl.
-		// use the corresponding bpfFile
-		bpfFildAndroid := fmt.Sprintf("boringssl_a_%s", androidVer)
-		bpfFile, found = m.sslVersionBpfMap[bpfFildAndroid]
-		if found {
-			m.sslBpfFile = bpfFile
-			m.logger.Info().Str("BoringSSL Version", androidVer).Msg("OpenSSL/BoringSSL version found")
-		} else {
-			bpfFile, _ = m.sslVersionBpfMap[AndroidDefauleFilename]
-			m.logger.Warn().Str("BoringSSL Version", AndroidDefauleFilename).Msg("OpenSSL/BoringSSL version not found, used default version")
-		}
-	} else {
-		if strings.Contains(soPath, "libssl.so.3") {
-			bpfFile, _ = m.sslVersionBpfMap[Linuxdefaulefilename30]
-			m.logger.Warn().Str("OpenSSL Version", Linuxdefaulefilename30).Msg("OpenSSL/BoringSSL version not found from shared library file, used default version")
-		} else {
-			bpfFile, _ = m.sslVersionBpfMap[Linuxdefaulefilename111]
-			m.logger.Warn().Str("OpenSSL Version", Linuxdefaulefilename111).Msg("OpenSSL/BoringSSL version not found from shared library file, used default version")
-		}
+		bpfFile, _ = m.sslVersionBpfMap[AndroidDefauleFilename]
+		m.logger.Warn().Str("BoringSSL Version", AndroidDefauleFilename).Msg("OpenSSL/BoringSSL version not found, used default version")
+		return bpfFile
+	}
+
+	if strings.Contains(soPath, "libssl.so.3") {
+		bpfFile, _ = m.sslVersionBpfMap[Linuxdefaulefilename30]
+		m.logger.Warn().Str("OpenSSL Version", Linuxdefaulefilename30).Msg("OpenSSL/BoringSSL version not found from shared library file, used default version")
+	} else {
+		bpfFile, _ = m.sslVersionBpfMap[Linuxdefaulefilename111]
+		m.logger.Warn().Str("OpenSSL Version", Linuxdefaulefilename111).Msg("OpenSSL/BoringSSL version not found from shared library file, used default version")
+	}
+	return bpfFile
+}
+
+func getImpNeeded(soPath string) ([]string, error) {
+	var importedNeeded []string
+	f, err := os.OpenFile(soPath, os.O_RDONLY, os.ModePerm)
+	if err != nil {
+		return importedNeeded, fmt.Errorf("can not open %s, with error:%v", soPath, err)
 	}
-	m.sslBpfFile = bpfFile
-	return nil
+
+	elfFile, err := elf.NewFile(f)
+	if err != nil {
+		return importedNeeded, fmt.Errorf("parse the ELF file  %s failed, with error:%v", soPath, err)
+	}
+
+	// 打印外部依赖的动态链接库
+	is, err := elfFile.DynString(elf.DT_NEEDED)
+	//is, err := elfFile.ImportedSymbols()
+	if err != nil {
+		return importedNeeded, err
+	}
+	for _, s := range is {
+		importedNeeded = append(importedNeeded, s)
+	}
+	return importedNeeded, nil
 }