Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Question] Task exception was never retrieved #77

Open
chanpu9 opened this issue Sep 18, 2021 · 0 comments
Open

[Question] Task exception was never retrieved #77

chanpu9 opened this issue Sep 18, 2021 · 0 comments

Comments

@chanpu9
Copy link

chanpu9 commented Sep 18, 2021

Hello,

I tried to catch all possible errors but I could not success. How should I write the code to catch the errors such as timeout, program crash and server down.

from aiohttp import ClientSession, TCPConnector
import asyncio
from pypeln.task import TaskPool
import sys
import tqdm
import signal

async def get_pokemon(session, url, proxy):
    try:
        async with session.get(url, proxy=proxy, ssl=False, timeout=15) as resp:
            pokemon = await resp.json()
            if resp.status == 302:
                print(pokemon['name'])
                return pokemon['name']
            else:
                raise Exception('Failed ' + url) from None
    except:
        raise Exception('Failed ' + url) from None

async def _main():
    proxy = 'http://127.0.0.1:8080'
    limit = 10
    connector = TCPConnector(limit=None, force_close=True)
    try:
        async with ClientSession(connector=connector) as session, TaskPool(limit) as tasks:
            for number in range(1, 1000000000):
                url = f'https://pokeapi.co/api/v2/pokemon/{number}'
                await tasks.put(get_pokemon(session, url, proxy))
    except Exception as error:
        raise Exception(error) from None
loop = asyncio.get_event_loop()          

try:
    loop.run_until_complete(_main())
except Exception as error:
    print("Error :" + error)
    for task in asyncio.Task.all_tasks(loop):
        loop.call_soon_threadsafe(task.cancel())
        
    loop.stop()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant