Exception: [Agent Request] Error: [Errno 24] Too many open files

[nowanti]

使用场景

• 在多线程(2~N)情况下，执行 agent，一段时间（几百个请求）后报错。
• Ubuntu24.04.1 / MacOS15.3 (24D60) / Windows 10 均有出现。
• Agently 版本从 3.4.2.3 退回到 3.4.2.0 后，同样的代码，未见错误，能完整跑完几千个请求。

agent 示例代码

from typing import Dict, Any
from Agently import AgentFactory


agent_factory = (
    AgentFactory()
    .set_settings("current_model", "OAIClient")
    .set_settings("model.OAIClient.auth", {"api_key": llm.api_key})
    .set_settings("model.OAIClient.url", llm.base_url)
    .set_settings("is_debug", is_debug)
)


@agent_retry
def extract(name: str, data: str) -> Dict[str, Any]:
    agent = agent_factory.create_agent().set_settings(
        "model.OAIClient.options",
        {
            "model": "qwen2.5:14b",
            "temperature": 0.7,
            "max_tokens": 128,
        },
    )

    result = (
        agent.input({"name": name, "data": data})
        .instruct(
            {
                "任务": "请从给定文本{input.data}中提取第一个完整的XX，信息不足时参考{input.name}如果不确定的请返回『不确定』。",
            }
        )
        .output(
            {
                "entity": (str, "完整的XX"),
                "other": (str, " 其他信息，可空"),
            }
        )
        .start()
    )
    return result

依赖情况

➜ poetry show --tree
agently 3.4.2.3 Agently, a framework to build applications based on language model powered intelligent agents.
├── aiohttp *
│   ├── aiohappyeyeballs >=2.3.0 
│   ├── aiosignal >=1.1.2 
│   │   └── frozenlist >=1.1.0 
│   ├── attrs >=17.3.0 
│   ├── frozenlist >=1.1.1 (circular dependency aborted here)
│   ├── multidict >=4.5,<7.0 
│   ├── propcache >=0.2.0 
│   └── yarl >=1.17.0,<2.0 
│       ├── idna >=2.0 
│       ├── multidict >=4.0 (circular dependency aborted here)
│       └── propcache >=0.2.0 (circular dependency aborted here)
├── beautifulsoup4 *
│   ├── soupsieve >1.2 
│   └── typing-extensions >=4.0.0 
├── duckduckgo-search *
│   ├── click >=8.1.8 
│   │   └── colorama * 
│   ├── lxml >=5.3.0 
│   └── primp >=0.11.0 
├── erniebot *
│   ├── aiohttp * 
│   │   ├── aiohappyeyeballs >=2.3.0 
│   │   ├── aiosignal >=1.1.2 
│   │   │   └── frozenlist >=1.1.0 
│   │   ├── attrs >=17.3.0 
│   │   ├── frozenlist >=1.1.1 (circular dependency aborted here)
│   │   ├── multidict >=4.5,<7.0 
│   │   ├── propcache >=0.2.0 
│   │   └── yarl >=1.17.0,<2.0 
│   │       ├── idna >=2.0 
│   │       ├── multidict >=4.0 (circular dependency aborted here)
│   │       └── propcache >=0.2.0 (circular dependency aborted here)
│   ├── bce-python-sdk * 
│   │   ├── future >=0.6.0 
│   │   ├── pycryptodome >=3.8.0 
│   │   └── six >=1.4.0 
│   ├── colorlog * 
│   │   └── colorama * 
│   ├── jsonschema >=4.19 
│   │   ├── attrs >=22.2.0 (circular dependency aborted here)
│   │   ├── jsonschema-specifications >=2023.03.6 
│   │   │   └── referencing >=0.31.0 
│   │   │       ├── attrs >=22.2.0 (circular dependency aborted here)
│   │   │       ├── rpds-py >=0.7.0 
│   │   │       └── typing-extensions >=4.4.0 
│   │   ├── referencing >=0.28.4 (circular dependency aborted here)
│   │   └── rpds-py >=0.7.1 (circular dependency aborted here)
│   ├── requests >=2.20 
│   │   ├── certifi >=2017.4.17 
│   │   ├── charset-normalizer >=2,<4 
│   │   ├── idna >=2.5,<4 (circular dependency aborted here)
│   │   └── urllib3 >=1.21.1,<3 
│   ├── tenacity * 
│   └── typing-extensions >=4.8.0 (circular dependency aborted here)
├── httpx *
│   ├── anyio * 
│   │   ├── idna >=2.8 
│   │   ├── sniffio >=1.1 
│   │   └── typing-extensions >=4.5 
│   ├── certifi * 
│   ├── httpcore ==1.* 
│   │   ├── certifi * (circular dependency aborted here)
│   │   └── h11 >=0.13,<0.15 
│   ├── idna * (circular dependency aborted here)
│   └── socksio ==1.* 
├── json5 *
├── litellm *
│   ├── aiohttp * 
│   │   ├── aiohappyeyeballs >=2.3.0 
│   │   ├── aiosignal >=1.1.2 
│   │   │   └── frozenlist >=1.1.0 
│   │   ├── attrs >=17.3.0 
│   │   ├── frozenlist >=1.1.1 (circular dependency aborted here)
│   │   ├── multidict >=4.5,<7.0 
│   │   ├── propcache >=0.2.0 
│   │   └── yarl >=1.17.0,<2.0 
│   │       ├── idna >=2.0 
│   │       ├── multidict >=4.0 (circular dependency aborted here)
│   │       └── propcache >=0.2.0 (circular dependency aborted here)
│   ├── click * 
│   │   └── colorama * 
│   ├── httpx >=0.23.0 
│   │   ├── anyio * 
│   │   │   ├── idna >=2.8 (circular dependency aborted here)
│   │   │   ├── sniffio >=1.1 
│   │   │   └── typing-extensions >=4.5 
│   │   ├── certifi * 
│   │   ├── httpcore ==1.* 
│   │   │   ├── certifi * (circular dependency aborted here)
│   │   │   └── h11 >=0.13,<0.15 
│   │   ├── idna * (circular dependency aborted here)
│   │   └── socksio ==1.* 
│   ├── importlib-metadata >=6.8.0 
│   │   └── zipp >=3.20 
│   ├── jinja2 >=3.1.2,<4.0.0 
│   │   └── markupsafe >=2.0 
│   ├── jsonschema >=4.22.0,<5.0.0 
│   │   ├── attrs >=22.2.0 (circular dependency aborted here)
│   │   ├── jsonschema-specifications >=2023.03.6 
│   │   │   └── referencing >=0.31.0 
│   │   │       ├── attrs >=22.2.0 (circular dependency aborted here)
│   │   │       ├── rpds-py >=0.7.0 
│   │   │       └── typing-extensions >=4.4.0 (circular dependency aborted here)
│   │   ├── referencing >=0.28.4 (circular dependency aborted here)
│   │   └── rpds-py >=0.7.1 (circular dependency aborted here)
│   ├── openai >=1.61.0 
│   │   ├── anyio >=3.5.0,<5 (circular dependency aborted here)
│   │   ├── distro >=1.7.0,<2 
│   │   ├── httpx >=0.23.0,<1 (circular dependency aborted here)
│   │   ├── jiter >=0.4.0,<1 
│   │   ├── pydantic >=1.9.0,<3 
│   │   │   ├── annotated-types >=0.6.0 
│   │   │   ├── pydantic-core 2.27.2 
│   │   │   │   └── typing-extensions >=4.6.0,<4.7.0 || >4.7.0 (circular dependency aborted here)
│   │   │   └── typing-extensions >=4.12.2 (circular dependency aborted here)
│   │   ├── sniffio * (circular dependency aborted here)
│   │   ├── tqdm >4 
│   │   │   └── colorama * (circular dependency aborted here)
│   │   └── typing-extensions >=4.11,<5 (circular dependency aborted here)
│   ├── pydantic >=2.0.0,<3.0.0 (circular dependency aborted here)
│   ├── python-dotenv >=0.2.0 
│   ├── tiktoken >=0.7.0 
│   │   ├── regex >=2022.1.18 
│   │   └── requests >=2.26.0 
│   │       ├── certifi >=2017.4.17 (circular dependency aborted here)
│   │       ├── charset-normalizer >=2,<4 
│   │       ├── idna >=2.5,<4 (circular dependency aborted here)
│   │       └── urllib3 >=1.21.1,<3 
│   └── tokenizers * 
│       └── huggingface-hub >=0.16.4,<1.0 
│           ├── filelock * 
│           ├── fsspec >=2023.5.0 
│           ├── packaging >=20.9 
│           ├── pyyaml >=5.1 
│           ├── requests * (circular dependency aborted here)
│           ├── tqdm >=4.42.1 (circular dependency aborted here)
│           └── typing-extensions >=3.7.4.3 (circular dependency aborted here)
├── openai *
│   ├── anyio >=3.5.0,<5 
│   │   ├── idna >=2.8 
│   │   ├── sniffio >=1.1 
│   │   └── typing-extensions >=4.5 
│   ├── distro >=1.7.0,<2 
│   ├── httpx >=0.23.0,<1 
│   │   ├── anyio * (circular dependency aborted here)
│   │   ├── certifi * 
│   │   ├── httpcore ==1.* 
│   │   │   ├── certifi * (circular dependency aborted here)
│   │   │   └── h11 >=0.13,<0.15 
│   │   ├── idna * (circular dependency aborted here)
│   │   └── socksio ==1.* 
│   ├── jiter >=0.4.0,<1 
│   ├── pydantic >=1.9.0,<3 
│   │   ├── annotated-types >=0.6.0 
│   │   ├── pydantic-core 2.27.2 
│   │   │   └── typing-extensions >=4.6.0,<4.7.0 || >4.7.0 (circular dependency aborted here)
│   │   └── typing-extensions >=4.12.2 (circular dependency aborted here)
│   ├── sniffio * (circular dependency aborted here)
│   ├── tqdm >4 
│   │   └── colorama * 
│   └── typing-extensions >=4.11,<5 (circular dependency aborted here)
├── pyreadline3 *
├── pyyaml *
├── requests *
│   ├── certifi >=2017.4.17 
│   ├── charset-normalizer >=2,<4 
│   ├── idna >=2.5,<4 
│   └── urllib3 >=1.21.1,<3 
├── tencentcloud-sdk-python *
│   └── requests >=2.16.0 
│       ├── certifi >=2017.4.17 
│       ├── charset-normalizer >=2,<4 
│       ├── idna >=2.5,<4 
│       └── urllib3 >=1.21.1,<3 
├── tornado *
└── websockets *
beaker 1.13.0 A Session and Caching library with WSGI Middleware
dynaconf 3.2.7 The dynamic configurator for your Python Project
httpx 0.28.1 The next generation HTTP client.
├── anyio *
│   ├── idna >=2.8 
│   ├── sniffio >=1.1 
│   └── typing-extensions >=4.5 
├── certifi *
├── httpcore ==1.*
│   ├── certifi * 
│   └── h11 >=0.13,<0.15 
├── idna *
└── socksio ==1.*
loguru 0.7.3 Python logging made (stupidly) simple
├── colorama >=0.3.4
└── win32-setctime >=1.0.0
openai 1.61.1 The official Python library for the openai API
├── anyio >=3.5.0,<5
│   ├── idna >=2.8 
│   ├── sniffio >=1.1 
│   └── typing-extensions >=4.5 
├── distro >=1.7.0,<2
├── httpx >=0.23.0,<1
│   ├── anyio * 
│   │   ├── idna >=2.8 
│   │   ├── sniffio >=1.1 
│   │   └── typing-extensions >=4.5 
│   ├── certifi * 
│   ├── httpcore ==1.* 
│   │   ├── certifi * (circular dependency aborted here)
│   │   └── h11 >=0.13,<0.15 
│   ├── idna * (circular dependency aborted here)
│   └── socksio ==1.* 
├── jiter >=0.4.0,<1
├── pydantic >=1.9.0,<3
│   ├── annotated-types >=0.6.0 
│   ├── pydantic-core 2.27.2 
│   │   └── typing-extensions >=4.6.0,<4.7.0 || >4.7.0 
│   └── typing-extensions >=4.12.2 (circular dependency aborted here)
├── sniffio *
├── tqdm >4
│   └── colorama * 
└── typing-extensions >=4.11,<5
pandas 2.2.3 Powerful data structures for data analysis, time series, and statistics
├── numpy >=1.26.0
├── python-dateutil >=2.8.2
│   └── six >=1.5 
├── pytz >=2020.1
└── tzdata >=2022.7

报错日志

Exception in thread Thread-646 (start_in_theard):
Traceback (most recent call last):
  File "/home/nowanti/.cache/pypoetry/virtualenvs/adhoc-projects-2JC5bUP8-py3.12/lib/python3.12/site-packages/Agently/Agent/Agent.py", line 301, in start_in_theard
    reply = loop.run_until_complete(self.start_async(request_type))
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/nowanti/.cache/pypoetry/virtualenvs/adhoc-projects-2JC5bUP8-py3.12/lib/python3.12/site-packages/Agently/Agent/Agent.py", line 280, in start_async
    raise(e)
  File "/home/nowanti/.cache/pypoetry/virtualenvs/adhoc-projects-2JC5bUP8-py3.12/lib/python3.12/site-packages/Agently/Agent/Agent.py", line 213, in start_async
    event_generator = await self.request.get_event_generator(request_type)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nowanti/.cache/pypoetry/virtualenvs/adhoc-projects-2JC5bUP8-py3.12/lib/python3.12/site-packages/Agently/Request/Request.py", line 117, in get_event_generator
    response_generator = await request_plugin_export["request_model"](request_data)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nowanti/.cache/pypoetry/virtualenvs/adhoc-projects-2JC5bUP8-py3.12/lib/python3.12/site-packages/Agently/plugins/request/OAIClient.py", line 206, in request_model
    "http_client": httpx.AsyncClient(**httpx_params),
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nowanti/.cache/pypoetry/virtualenvs/adhoc-projects-2JC5bUP8-py3.12/lib/python3.12/site-packages/httpx/_client.py", line 1402, in __init__
    self._transport = self._init_transport(
                      ^^^^^^^^^^^^^^^^^^^^^
  File "/home/nowanti/.cache/pypoetry/virtualenvs/adhoc-projects-2JC5bUP8-py3.12/lib/python3.12/site-packages/httpx/_client.py", line 1445, in _init_transport
    return AsyncHTTPTransport(
           ^^^^^^^^^^^^^^^^^^^
  File "/home/nowanti/.cache/pypoetry/virtualenvs/adhoc-projects-2JC5bUP8-py3.12/lib/python3.12/site-packages/httpx/_transports/default.py", line 297, in __init__
    ssl_context = create_ssl_context(verify=verify, cert=cert, trust_env=trust_env)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nowanti/.cache/pypoetry/virtualenvs/adhoc-projects-2JC5bUP8-py3.12/lib/python3.12/site-packages/httpx/_config.py", line 40, in create_ssl_context
    ctx = ssl.create_default_context(cafile=certifi.where())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/ssl.py", line 708, in create_default_context
    context.load_verify_locations(cafile, capath, cadata)
OSError: [Errno 24] Too many open files

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/home/nowanti/.cache/pypoetry/virtualenvs/adhoc-projects-2JC5bUP8-py3.12/lib/python3.12/site-packages/Agently/Agent/Agent.py", line 305, in start_in_theard
    raise Exception(f"[Agent Request] Error: { str(e) }")                    
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Exception: [Agent Request] Error: [Errno 24] Too many open files

[Maplemx]

收到，我查一下哪些更改可能引起这个问题