Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to install via meltano add #16

Open
anden-akkio opened this issue Sep 25, 2023 · 4 comments
Open

Unable to install via meltano add #16

anden-akkio opened this issue Sep 25, 2023 · 4 comments

Comments

@anden-akkio
Copy link

anden-akkio commented Sep 25, 2023

Overview

Attempting to install this via meltano add but am running into an error that is extremely vague to the point that I'm not really sure where to start debugging.

Other extractors install fine. For example, just ran meltano --log-level=debug add extractor tap-mysql --variant singer-io and it went through fine.

CLI Output

(venv) ➜  meltano git:(tech/testing-improvements) ✗ meltano --log-level=debug add extractor tap-mysql --variant meltanolabs
2023-09-25T16:20:26.201060Z [debug    ] /etc/localtime found
2023-09-25T16:20:26.202827Z [debug    ] 1 found:
 {'/etc/localtime is a symlink to': 'America/Chicago'}
2023-09-25T16:20:26.204682Z [debug    ] Creating DB engine for project at '/Users/andenacitelli/meltano-service/src/meltano' with DB URI 'sqlite://Users/andenacitelli/meltano-service/src/meltano/.meltano/meltano.db'
2023-09-25T16:20:27.204517Z [debug    ] Found plugin parent            parent=tap-mysql plugin=tap-mysql source=<DefinitionSource.HUB: 1>
2023-09-25T16:20:27.220241Z [debug    ] Locked plugin definition       path=PosixPath('/Users/andenacitelli/meltano-service/src/meltano/plugins/extractors/tap-mysql--meltanolabs.lock')
Added extractor 'tap-mysql' to your Meltano project
Variant:        meltanolabs
Repository:     https://github.com/MeltanoLabs/tap-mysql
Documentation:  https://hub.meltano.com/extractors/tap-mysql--meltanolabs

Installing extractor 'tap-mysql'...
2023-09-25T16:20:27.287477Z [debug    ] Packages for 'extractors/tap-mysql' have changed so performing a clean install.
2023-09-25T16:20:27.287819Z [debug    ] Removed old virtual environment for 'extractors/tap-mysql'
2023-09-25T16:20:27.287859Z [debug    ] Creating virtual environment for 'extractors/tap-mysql'
2023-09-25T16:20:27.706016Z [debug    ] Upgrading pip for 'extractors/tap-mysql'
2023-09-25T16:20:27.706127Z [debug    ] Upgrading with args '--upgrade pip' in existing virtual environment for 'extractors/tap-mysql'
2023-09-25T16:20:28.759323Z [debug    ] Installing with args 'git+https://github.com/MeltanoLabs/tap-mysql.git' into virtual environment for 'extractors/tap-mysql'
Extractor 'tap-mysql' could not be installed: failed to install plugin 'tap-mysql'.

2023-09-25T16:20:32.134568Z [debug    ] Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to
join our friendly Slack community.

Failed to install plugin(s) 
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/andenacitelli/meltano-service/venv/lib/python3.10/site-packages/meltano/cli/__init__.py:1 │
│ 02 in _run_cli                                                                                   │
│                                                                                                  │
│    99 │   """
│   100 │   try:                                                                                   │
│   101 │   │   try:  # noqa: WPS225, WPS505                                                       │
│ ❱ 102 │   │   │   cli(obj={"project": None})                                                     │
│   103 │   │   except ProjectReadonly as err:                                                     │
│   104 │   │   │   raise CliError(                                                                │
│   105 │   │   │   │   f"The requested action could not be completed: {err}",                     │
│                                                                                                  │
│ /Users/andenacitelli/meltano-service/venv/lib/python3.10/site-packages/click/core.py:1157 in     │
│ __call__                                                                                         │
│                                                                                                  │
│   1154 │                                                                                         │
│   1155 │   def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any:                           │
│   1156 │   │   """Alias for :meth:`main`."""
│ ❱ 1157 │   │   return self.main(*args, **kwargs)                                                 │
│   1158                                                                                           │
│   1159                                                                                           │
│   1160 class Command(BaseCommand):                                                               │
│                                                                                                  │
│ /Users/andenacitelli/meltano-service/venv/lib/python3.10/site-packages/meltano/cli/cli.py:42 in  │
│ main                                                                                             │
│                                                                                                  │
│    39 │   │   │   args: Positional arguments for the Click group.                                │
│    40 │   │   │   kwargs: Keyword arguments for the Click group.                                 │
│    41 │   │   """                                                                                │
│ ❱  42 │   │   return super().main(*args, windows_expand_args=False, **kwargs)                    │
│    43                                                                                            │
│    44                                                                                            │
│    45 @click.group(                                                                              │
│                                                                                                  │
│ /Users/andenacitelli/meltano-service/venv/lib/python3.10/site-packages/click/core.py:1078 in     │
│ main                                                                                             │
│                                                                                                  │
│   1075 │   │   try:                                                                              │
│   1076 │   │   │   try:                                                                          │
│   1077 │   │   │   │   with self.make_context(prog_name, args, **extra) as ctx:                  │
│ ❱ 1078 │   │   │   │   │   rv = self.invoke(ctx)                                                 │
│   1079 │   │   │   │   │   if not standalone_mode:                                               │
│   1080 │   │   │   │   │   │   return rv                                                         │
│   1081 │   │   │   │   │   # it's not safe to `ctx.exit(rv)` here!                               │
│                                                                                                  │
│ /Users/andenacitelli/meltano-service/venv/lib/python3.10/site-packages/meltano/cli/utils.py:612  │
│ in invoke                                                                                        │
│                                                                                                  │
│   609 │   │   enact_environment_behavior(self.environment_behavior, ctx)                         │
│   610 │   │   if ctx.obj.get("tracker"):                                                         │
│   611 │   │   │   ctx.obj["tracker"].add_contexts(CliContext.from_click_context(ctx))            │
│ ❱ 612 │   │   super().invoke(ctx)                                                                │
│   613                                                                                            │
│   614                                                                                            │
│   615 class InstrumentedDefaultGroup(InstrumentedGroupMixin, DefaultGroup, DYMGroup):            │
│                                                                                                  │
│ /Users/andenacitelli/meltano-service/venv/lib/python3.10/site-packages/click/core.py:1688 in     │
│ invoke                                                                                           │
│                                                                                                  │
│   1685 │   │   │   │   super().invoke(ctx)                                                       │
│   1686 │   │   │   │   sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)                    │
│   1687 │   │   │   │   with sub_ctx:                                                             │
│ ❱ 1688 │   │   │   │   │   return _process_result(sub_ctx.command.invoke(sub_ctx))               │
│   1689 │   │                                                                                     │
│   1690 │   │   # In chain mode we create the contexts step by step, but after the                │
│   1691 │   │   # base command has been invoked.  Because at that point we do not                 │
│                                                                                                  │
│ /Users/andenacitelli/meltano-service/venv/lib/python3.10/site-packages/meltano/cli/utils.py:661  │
│ in invoke                                                                                        │
│                                                                                                  │
│   658 │   │   if ctx.obj.get("tracker"):                                                         │
│   659 │   │   │   ctx.obj["tracker"].add_contexts(CliContext.from_click_context(ctx))            │
│   660 │   │   │   ctx.obj["tracker"].track_command_event(CliEvent.started)                       │
│ ❱ 661 │   │   super().invoke(ctx)                                                                │
│   662                                                                                            │
│                                                                                                  │
│ /Users/andenacitelli/meltano-service/venv/lib/python3.10/site-packages/click/core.py:1434 in     │
│ invoke                                                                                           │
│                                                                                                  │
│   1431 │   │   │   echo(style(message, fg="red"), err=True)                                      │
│   1432 │   │                                                                                     │
│   1433 │   │   if self.callback is not None:                                                     │
│ ❱ 1434 │   │   │   return ctx.invoke(self.callback, **ctx.params)                                │
│   1435 │                                                                                         │
│   1436 │   def shell_complete(self, ctx: Context, incomplete: str) -> t.List["CompletionItem"]:  │
│   1437 │   │   """Return a list of completions for the incomplete value. Looks                   │
│                                                                                                  │
│ /Users/andenacitelli/meltano-service/venv/lib/python3.10/site-packages/click/core.py:783 in      │
│ invoke                                                                                           │
│                                                                                                  │
│    780 │   │                                                                                     │
│    781 │   │   with augment_usage_errors(__self):                                                │
│    782 │   │   │   with ctx:                                                                     │
│ ❱  783 │   │   │   │   return __callback(*args, **kwargs)                                        │
│    784 │                                                                                         │
│    785 │   def forward(                                                                          │
│    786 │   │   __self, __cmd: "Command", *args: t.Any, **kwargs: t.Any  # noqa: B902             │
│                                                                                                  │
│ /Users/andenacitelli/meltano-service/venv/lib/python3.10/site-packages/meltano/cli/params.py:27  │
│ in decorate                                                                                      │
│                                                                                                  │
│   24 │   │   if database_uri:                                                                    │
│   25 │   │   │   ProjectSettingsService.config_override["database_uri"] = database_uri           │
│   26 │   │                                                                                       │
│ ❱ 27 │   │   return func(*args, **kwargs)                                                        │
│   28 │                                                                                           │
│   29 │   return functools.update_wrapper(decorate, func)                                         │
│   30                                                                                             │
│                                                                                                  │
│ /Users/andenacitelli/meltano-service/venv/lib/python3.10/site-packages/meltano/cli/params.py:75  │
│ in decorate                                                                                      │
│                                                                                                  │
│   72 │   │   │   │   except MigrationError as err:                                               │
│   73 │   │   │   │   │   raise CliError(str(err)) from err                                       │
│   74 │   │   │                                                                                   │
│ ❱ 75 │   │   │   func(project, *args, **kwargs)                                                  │
│   76 │   │                                                                                       │
│   77 │   │   return functools.update_wrapper(decorate, func)                                     │
│   78                                                                                             │
│                                                                                                  │
│ /Users/andenacitelli/meltano-service/venv/lib/python3.10/site-packages/click/decorators.py:33 in │
│ new_func                                                                                         │
│                                                                                                  │
│    30 │   """                                                                                    │
│    31 │                                                                                          │
│    32 │   def new_func(*args: "P.args", **kwargs: "P.kwargs") -> "R":                            │
│ ❱  33 │   │   return f(get_current_context(), *args, **kwargs)                                   │
│    34 │                                                                                          │
│    35 │   return update_wrapper(new_func, f)                                                     │
│    36                                                                                            │
│                                                                                                  │
│ /Users/andenacitelli/meltano-service/venv/lib/python3.10/site-packages/meltano/cli/add.py:163 in │
│ add                                                                                              │
│                                                                                                  │
│   160 │   │                                                                                      │
│   161 │   │   if not success:                                                                    │
│   162 │   │   │   tracker.track_command_event(CliEvent.failed)                                   │
│ ❱ 163 │   │   │   raise CliError("Failed to install plugin(s)")                                  │
│   164 │                                                                                          │
│   165 │   _print_plugins(plugins)                                                                │
│   166 │   tracker.track_command_event(CliEvent.completed)                                        │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
CliError: Failed to install plugin(s)

The above exception was the direct cause of the following exception:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/andenacitelli/meltano-service/venv/lib/python3.10/site-packages/meltano/cli/__init__.py:1 │
│ 12 in _run_cli                                                                                   │
│                                                                                                  │
│   109 │   │   except MeltanoError as err:                                                        │
│   110 │   │   │   handle_meltano_error(err)                                                      │
│   111 │   │   except Exception as err:                                                           │
│ ❱ 112 │   │   │   raise CliError(f"{troubleshooting_message}\n{err}") from err                   │
│   113 │   except CliError as cli_error:                                                          │
│   114 │   │   cli_error.print()                                                                  │
│   115 │   │   sys.exit(1)                                                                        │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
CliError: Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to
join our friendly Slack community.

Failed to install plugin(s)

Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to
join our friendly Slack community.

Failed to install plugin(s)

meltano.yml

default_environment: staging
environments:
- name: staging
- name: production
plugins:
  extractors:
  - name: tap-postgres
    variant: transferwise
    pip_url: pipelinewise-tap-postgres
  - name: tap-s3-csv
    variant: transferwise
    pip_url: pipelinewise-tap-s3-csv
  - name: tap-mysql
    variant: meltanolabs
    pip_url: git+https://github.com/MeltanoLabs/tap-mysql.git
  loaders:
  - name: target-s3
    variant: crowemi
    pip_url: git+https://github.com/crowemi/target-s3.git
  - name: target-gcs
    variant: datateer
    pip_url: git+https://github.com/Datateer/target-gcs.git
  - name: target-postgres
    variant: meltanolabs
    pip_url: git+https://github.com/MeltanoLabs/target-postgres.git
  mappers:
  - name: meltano-map-transformer
    variant: meltano
    pip_url: git+https://github.com/MeltanoLabs/meltano-map-transform.git
    executable: meltano-map-transform
project_id: 05ff139b-599a-4d08-9a70-ee167a1dcdaf
version: 1

What I've Tried

  • Wipe .meltano/ and try again
  • meltano install --clean
  • Wipe lock files and try again
@anden-akkio anden-akkio changed the title Installation issues Unable to install via meltano add Sep 25, 2023
@visch
Copy link
Member

visch commented Sep 25, 2023

I know there's a bug that was introduced somewhere in meltano that doesn't show the pip installation logs for some reason. I'm not sure if it was 3.x that caused this but meltano/meltano#8145 should have addressed it.

You could try running the add command with a 2.x version of meltano as well or run with the version that's on main.

I'm pretty sure this is failing during the installation of tap-mysql so seeing those pip logs is important here!

@anden-akkio
Copy link
Author

anden-akkio commented Sep 25, 2023

Attempted install on 2.x.x and it's indeed a pip installation failure:

Extractor 'tap-mysql' could not be installed: failed to install plugin 'tap-mysql'.
  Running command git clone --filter=blob:none --quiet https://github.com/MeltanoLabs/tap-mysql.git /private/var/folders/kt/k_mss0s12y5dkzvyfbbkcrsm0000gn/T/pip-req-build-uks9lc5u
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [22 lines of output]
      Trying pkg-config --exists mysqlclient
      Command 'pkg-config --exists mysqlclient' returned non-zero exit status 1.
      Trying pkg-config --exists mariadb
      Command 'pkg-config --exists mariadb' returned non-zero exit status 1.
      Traceback (most recent call last):
        File "/Users/andenacitelli/meltano-service/src/meltano/.meltano/extractors/tap-mysql/venv/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/andenacitelli/meltano-service/src/meltano/.meltano/extractors/tap-mysql/venv/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/Users/andenacitelli/meltano-service/src/meltano/.meltano/extractors/tap-mysql/venv/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/private/var/folders/kt/k_mss0s12y5dkzvyfbbkcrsm0000gn/T/pip-build-env-7uf0tg37/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 355, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/private/var/folders/kt/k_mss0s12y5dkzvyfbbkcrsm0000gn/T/pip-build-env-7uf0tg37/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 325, in _get_build_requires
          self.run_setup()
        File "/private/var/folders/kt/k_mss0s12y5dkzvyfbbkcrsm0000gn/T/pip-build-env-7uf0tg37/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 341, in run_setup
          exec(code, locals())
        File "<string>", line 154, in <module>
        File "<string>", line 48, in get_config_posix
        File "<string>", line 27, in find_package_name
      Exception: Can not find valid pkg-config name.
      Specify MYSQLCLIENT_CFLAGS and MYSQLCLIENT_LDFLAGS env vars manually
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Looks like it wants mysql or mariadb to be present. I see there are some instructions in the README that I missed as to this end. Our deployment is on Linux so we could theoretically use APT, but we do our work on MacOS so I'm not really sure whether an equivalent exists. Advice?

Worth noting that this isn't a blocker because the singer-io implementation does the job sufficiently for now with no extra config. Happy to see this through to improve Meltano though, as we'd like to switch to a more modern connector w/ more consistent updates!

@visch Edited some more information in you may want to take a look at, appreciate your help

@visch
Copy link
Member

visch commented Nov 21, 2023

@anden-akkio While we could use the pure python implementation here it's slower and not the recommended method according to SQLAlchemy. The steps here https://www.bytebase.com/blog/how-to-install-mysql-client-on-mac-ubuntu-centos-windows/ look find from a quick google search , we could add a reference for MacOS installation instructions to the Readme. If you could get the steps and add that it'd be great!

@gruckion
Copy link

gruckion commented Jul 6, 2024

I am also running into an issue on Mac whilst trying to get this to work.

❯ meltano add extractor tap-mysql --variant meltanolabs

Extractor 'tap-mysql' already exists in your Meltano project

To switch from the current 'transferwise' variant to 'meltanolabs':
1. Update `variant` and `pip_url` in your `meltano.yml` project file:
        name: tap-mysql
        variant: meltanolabs
        pip_url: git+https://github.com/MeltanoLabs/tap-mysql.git
2. Reinstall the plugin:
        meltano install extractor tap-mysql
3. Check if the configuration is still valid (and make changes until it is):
        meltano config tap-mysql list

To learn more, visit https://docs.meltano.com/guide/plugin-management#switching-from-one-variant-to-another

Alternatively, to keep the existing 'tap-mysql' with variant 'meltanolabs', add variant 'meltanolabs' as a separate plugin with its own unique name:
        meltano add extractor tap-mysql--meltanolabs --inherit-from tap-mysql --variant meltanolabs

To learn more, visit https://docs.meltano.com/guide/plugin-management#multiple-variants

2024-07-06T18:29:47.382967Z [info     ] Installing extractor 'tap-mysql'
2024-07-06T18:29:51.396660Z [info     ] Logged pip install output to /Users/sigex/workdir/meltano-first-tutorial/meltano-project-github/.meltano/logs/pip/extractors/tap-mysql/pip.log
2024-07-06T18:29:51.397011Z [error    ] Extractor 'tap-mysql' could not be installed: Failed to install plugin 'tap-mysql'.
2024-07-06T18:29:51.397064Z [info     ] Collecting pipelinewise-tap-mysql
  Using cached pipelinewise_tap_mysql-1.6.0-py3-none-any.whl.metadata (23 kB)
Collecting pendulum==2.1.2 (from pipelinewise-tap-mysql)
  Using cached pendulum-2.1.2.tar.gz (81 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting pipelinewise-singer-python==1.* (from pipelinewise-tap-mysql)
  Using cached pipelinewise_singer_python-1.3.0-py3-none-any.whl.metadata (2.6 kB)
Collecting mysql-replication==0.43 (from pipelinewise-tap-mysql)
  Using cached mysql_replication-0.43.0-py3-none-any.whl
Collecting PyMySQL==1.1.* (from pipelinewise-tap-mysql)
  Using cached PyMySQL-1.1.1-py3-none-any.whl.metadata (4.4 kB)
Collecting plpygis==0.2.1 (from pipelinewise-tap-mysql)
  Using cached plpygis-0.2.1-py3-none-any.whl.metadata (2.3 kB)
Collecting tzlocal==2.1 (from pipelinewise-tap-mysql)
  Using cached tzlocal-2.1-py2.py3-none-any.whl.metadata (8.2 kB)
Collecting python-dateutil<3.0,>=2.6 (from pendulum==2.1.2->pipelinewise-tap-mysql)
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
Collecting pytzdata>=2020.1 (from pendulum==2.1.2->pipelinewise-tap-mysql)
  Using cached pytzdata-2020.1-py2.py3-none-any.whl.metadata (2.3 kB)
Collecting pytz<2021.0 (from pipelinewise-singer-python==1.*->pipelinewise-tap-mysql)
  Using cached pytz-2020.5-py2.py3-none-any.whl.metadata (21 kB)
Collecting jsonschema==3.2.0 (from pipelinewise-singer-python==1.*->pipelinewise-tap-mysql)
  Using cached jsonschema-3.2.0-py2.py3-none-any.whl.metadata (7.8 kB)
Collecting simplejson==3.17.2 (from pipelinewise-singer-python==1.*->pipelinewise-tap-mysql)
  Using cached simplejson-3.17.2-cp312-cp312-macosx_13_0_arm64.whl
Collecting backoff==1.10.0 (from pipelinewise-singer-python==1.*->pipelinewise-tap-mysql)
  Using cached backoff-1.10.0-py2.py3-none-any.whl.metadata (12 kB)
Collecting ciso8601 (from pipelinewise-singer-python==1.*->pipelinewise-tap-mysql)
  Using cached ciso8601-2.3.1-cp312-cp312-macosx_11_0_arm64.whl.metadata (25 kB)
Collecting attrs>=17.4.0 (from jsonschema==3.2.0->pipelinewise-singer-python==1.*->pipelinewise-tap-mysql)
  Using cached attrs-23.2.0-py3-none-any.whl.metadata (9.5 kB)
Collecting pyrsistent>=0.14.0 (from jsonschema==3.2.0->pipelinewise-singer-python==1.*->pipelinewise-tap-mysql)
  Using cached pyrsistent-0.20.0-cp312-cp312-macosx_10_9_universal2.whl.metadata (27 kB)
Collecting setuptools (from jsonschema==3.2.0->pipelinewise-singer-python==1.*->pipelinewise-tap-mysql)
  Using cached setuptools-70.2.0-py3-none-any.whl.metadata (5.8 kB)
Collecting six>=1.11.0 (from jsonschema==3.2.0->pipelinewise-singer-python==1.*->pipelinewise-tap-mysql)
  Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Using cached pipelinewise_tap_mysql-1.6.0-py3-none-any.whl (42 kB)
Using cached pipelinewise_singer_python-1.3.0-py3-none-any.whl (24 kB)
Using cached plpygis-0.2.1-py3-none-any.whl (21 kB)
Using cached PyMySQL-1.1.1-py3-none-any.whl (44 kB)
Using cached tzlocal-2.1-py2.py3-none-any.whl (16 kB)
Using cached backoff-1.10.0-py2.py3-none-any.whl (31 kB)
Using cached jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Using cached pytz-2020.5-py2.py3-none-any.whl (510 kB)
Using cached pytzdata-2020.1-py2.py3-none-any.whl (489 kB)
Using cached ciso8601-2.3.1-cp312-cp312-macosx_11_0_arm64.whl (15 kB)
Using cached attrs-23.2.0-py3-none-any.whl (60 kB)
Using cached pyrsistent-0.20.0-cp312-cp312-macosx_10_9_universal2.whl (83 kB)
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Using cached setuptools-70.2.0-py3-none-any.whl (930 kB)
Building wheels for collected packages: pendulum
  Building wheel for pendulum (pyproject.toml): started
  error: subprocess-exited-with-error
  
  × Building wheel for pendulum (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  Building wheel for pendulum (pyproject.toml): finished with status 'error'
  ERROR: Failed building wheel for pendulum
Failed to build pendulum
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (pendulum)

Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to
join our friendly Slack community.

Failed to install plugin(s)

I get the same issue with meltano add extractor tap-mysql and meltano add extractor tap-mysql --variant meltanolabs

I can confirm that adding the singer io one does go through.

❯ meltano add extractor tap-mysql --variant singer-io

Added extractor 'tap-mysql' to your project
Variant:        singer-io
Repository:     https://github.com/singer-io/tap-mysql
Documentation:  https://hub.meltano.com/extractors/tap-mysql--singer-io

2024-07-06T18:31:28.562833Z [info     ] Installing extractor 'tap-mysql'
2024-07-06T18:31:34.006730Z [info     ] Installed extractor 'tap-mysql'

To learn more about extractor 'tap-mysql', visit https://hub.meltano.com/extractors/tap-mysql--singer-io

But I then run into another issue

❯ meltano config tap-mysql set host localhost
meltano config tap-mysql set port 3306
meltano config tap-mysql set user root
meltano config tap-mysql set password root
meltano config tap-mysql set database mydatabase
2024-07-06T18:32:22.712434Z [info     ] The default environment 'dev' will be ignored for `meltano config`. To configure a specific environment, please use the option `--environment=<environment name>`.
/Users/sigex/.local/pipx/venvs/meltano/lib/python3.12/site-packages/meltano/cli/interactive/config.py:404: RuntimeWarning: Unknown setting 'host'
  value, metadata = settings.set_with_metadata(
Extractor 'tap-mysql' setting 'host' was set in `meltano.yml`: 'localhost'
2024-07-06T18:32:23.212641Z [info     ] The default environment 'dev' will be ignored for `meltano config`. To configure a specific environment, please use the option `--environment=<environment name>`.
/Users/sigex/.local/pipx/venvs/meltano/lib/python3.12/site-packages/meltano/cli/interactive/config.py:404: RuntimeWarning: Unknown setting 'port'
  value, metadata = settings.set_with_metadata(
Extractor 'tap-mysql' setting 'port' was set in `meltano.yml`: '3306'
2024-07-06T18:32:23.703357Z [info     ] The default environment 'dev' will be ignored for `meltano config`. To configure a specific environment, please use the option `--environment=<environment name>`.
/Users/sigex/.local/pipx/venvs/meltano/lib/python3.12/site-packages/meltano/cli/interactive/config.py:404: RuntimeWarning: Unknown setting 'user'
  value, metadata = settings.set_with_metadata(
Extractor 'tap-mysql' setting 'user' was set in `meltano.yml`: 'root'
2024-07-06T18:32:24.183287Z [info     ] The default environment 'dev' will be ignored for `meltano config`. To configure a specific environment, please use the option `--environment=<environment name>`.
/Users/sigex/.local/pipx/venvs/meltano/lib/python3.12/site-packages/meltano/cli/interactive/config.py:404: RuntimeWarning: Unknown setting 'password'
  value, metadata = settings.set_with_metadata(
Extractor 'tap-mysql' setting 'password' was set in `meltano.yml`: 'root'
2024-07-06T18:32:24.698580Z [info     ] The default environment 'dev' will be ignored for `meltano config`. To configure a specific environment, please use the option `--environment=<environment name>`.
/Users/sigex/.local/pipx/venvs/meltano/lib/python3.12/site-packages/meltano/cli/interactive/config.py:404: RuntimeWarning: Unknown setting 'database'
  value, metadata = settings.set_with_metadata(
Extractor 'tap-mysql' setting 'database' was set in `meltano.yml`: 'mydatabase'
❯ meltano config tap-mysql test

2024-07-06T18:32:33.918991Z [info     ] The default environment 'dev' will be ignored for `meltano config`. To configure a specific environment, please use the option `--environment=<environment name>`.
Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to
join our friendly Slack community.

Plugin configuration is invalid
Catalog discovery failed: command ['/Users/sigex/workdir/meltano-first-tutorial/meltano-project-github/.meltano/extractors/tap-mysql/venv/bin/tap-mysql', '--config', '/Users/sigex/workdir/meltano-first-tutorial/meltano-project-github/.meltano/run/tap-mysql/tap.b470d9dc-bed2-4d11-bee6-e7329e2fd72c.config.json', '--discover'] returned 1 with stderr:
 Traceback (most recent call last):
  File "/Users/sigex/workdir/meltano-first-tutorial/meltano-project-github/.meltano/extractors/tap-mysql/venv/bin/tap-mysql", line 5, in <module>
    from tap_mysql import main
  File "/Users/sigex/workdir/meltano-first-tutorial/meltano-project-github/.meltano/extractors/tap-mysql/venv/lib/python3.12/site-packages/tap_mysql/__init__.py", line 24, in <module>
    import tap_mysql.sync_strategies.binlog as binlog
  File "/Users/sigex/workdir/meltano-first-tutorial/meltano-project-github/.meltano/extractors/tap-mysql/venv/lib/python3.12/site-packages/tap_mysql/sync_strategies/binlog.py", line 21, in <module>
    from tap_mysql.connection import connect_with_backoff, MySQLConnection, make_connection_wrapper
  File "/Users/sigex/workdir/meltano-first-tutorial/meltano-project-github/.meltano/extractors/tap-mysql/venv/lib/python3.12/site-packages/tap_mysql/connection.py", line 17, in <module>
    match_hostname = ssl.match_hostname
                     ^^^^^^^^^^^^^^^^^^
AttributeError: module 'ssl' has no attribute 'match_hostname'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants