Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GCC now required at runtime #185

Closed
firefart opened this issue Apr 12, 2021 · 12 comments
Closed

GCC now required at runtime #185

firefart opened this issue Apr 12, 2021 · 12 comments

Comments

@firefart
Copy link

firefart commented Apr 12, 2021 •
edited
Loading

I noticed an odd behaviour when upgrading ethon 0.12.0 to 0.13.0. Version 0.13.0 seems to require ffi >= 1.15.0 which seems to invoke GCC at runtime. As we are running inside a minimal alpine docker image there is no gcc available and I would rather not install it as a runtime dependency.

The following happens when invoking a rake task:

#18 3.540 rake aborted!
#18 3.540 Bundler::GemRequireError: There was an error while trying to load the gem 'typhoeus'.
#18 3.540 Gem Load Error is: Compilation error generating constants :
#18 3.540 	sh: gcc: not found
#18 3.540 Backtrace for gem load error is:
#18 3.540 /usr/local/bundle/gems/ffi-1.15.0/lib/ffi/tools/const_generator.rb:131:in `block in calculate'
#18 3.540 /usr/local/lib/ruby/2.7.0/tempfile.rb:291:in `open'
#18 3.540 /usr/local/bundle/gems/ffi-1.15.0/lib/ffi/tools/const_generator.rb:110:in `calculate'
#18 3.540 /usr/local/bundle/gems/ffi-1.15.0/lib/ffi/tools/const_generator.rb:49:in `initialize'
#18 3.540 /usr/local/bundle/gems/ethon-0.13.0/lib/ethon/libc.rb:17:in `new'
#18 3.540 /usr/local/bundle/gems/ethon-0.13.0/lib/ethon/libc.rb:17:in `<module:Libc>'
#18 3.540 /usr/local/bundle/gems/ethon-0.13.0/lib/ethon/libc.rb:7:in `<module:Ethon>'
#18 3.540 /usr/local/bundle/gems/ethon-0.13.0/lib/ethon/libc.rb:2:in `<main>'
#18 3.540 /usr/local/bundle/gems/bootsnap-1.7.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
#18 3.540 /usr/local/bundle/gems/bootsnap-1.7.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
#18 3.540 /usr/local/bundle/gems/bootsnap-1.7.3/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
#18 3.540 /usr/local/bundle/gems/bootsnap-1.7.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
#18 3.540 /usr/local/bundle/gems/bootsnap-1.7.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
#18 3.540 /usr/local/bundle/gems/activesupport-6.1.3.1/lib/active_support/dependencies.rb:332:in `block in require'
#18 3.540 /usr/local/bundle/gems/activesupport-6.1.3.1/lib/active_support/dependencies.rb:299:in `load_dependency'
#18 3.540 /usr/local/bundle/gems/activesupport-6.1.3.1/lib/active_support/dependencies.rb:332:in `require'
#18 3.540 /usr/local/bundle/gems/ethon-0.13.0/lib/ethon.rb:16:in `<main>'
#18 3.540 /usr/local/bundle/gems/bootsnap-1.7.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
#18 3.540 /usr/local/bundle/gems/bootsnap-1.7.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
#18 3.540 /usr/local/bundle/gems/bootsnap-1.7.3/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
#18 3.540 /usr/local/bundle/gems/bootsnap-1.7.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
#18 3.540 /usr/local/bundle/gems/bootsnap-1.7.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
#18 3.540 /usr/local/bundle/gems/activesupport-6.1.3.1/lib/active_support/dependencies.rb:332:in `block in require'
#18 3.540 /usr/local/bundle/gems/activesupport-6.1.3.1/lib/active_support/dependencies.rb:299:in `load_dependency'
#18 3.540 /usr/local/bundle/gems/activesupport-6.1.3.1/lib/active_support/dependencies.rb:332:in `require'
#18 3.540 /usr/local/bundle/gems/typhoeus-1.4.0/lib/typhoeus.rb:2:in `<main>'
#18 3.540 /usr/local/bundle/gems/bootsnap-1.7.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
#18 3.540 /usr/local/bundle/gems/bootsnap-1.7.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
#18 3.540 /usr/local/bundle/gems/bootsnap-1.7.3/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
#18 3.540 /usr/local/bundle/gems/bootsnap-1.7.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
#18 3.540 /usr/local/bundle/gems/bootsnap-1.7.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/runtime.rb:66:in `block (2 levels) in require'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/runtime.rb:61:in `each'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/runtime.rb:61:in `block in require'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/runtime.rb:50:in `each'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/runtime.rb:50:in `require'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler.rb:173:in `require'
#18 3.540 /app/config/application.rb:9:in `<top (required)>'
#18 3.540 /app/Rakefile:6:in `require'
#18 3.540 /app/Rakefile:6:in `<top (required)>'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/lib/rake/rake_module.rb:29:in `load'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/lib/rake/rake_module.rb:29:in `load_rakefile'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/lib/rake/application.rb:703:in `raw_load_rakefile'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/lib/rake/application.rb:104:in `block in load_rakefile'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/lib/rake/application.rb:186:in `standard_exception_handling'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/lib/rake/application.rb:103:in `load_rakefile'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/lib/rake/application.rb:82:in `block in run'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/lib/rake/application.rb:186:in `standard_exception_handling'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/lib/rake/application.rb:80:in `run'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/exe/rake:27:in `<top (required)>'
#18 3.540 /usr/local/bundle/bin/rake:23:in `load'
#18 3.540 /usr/local/bundle/bin/rake:23:in `<top (required)>'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/cli/exec.rb:63:in `load'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/cli/exec.rb:63:in `kernel_load'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/cli/exec.rb:28:in `run'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/cli.rb:494:in `exec'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/cli.rb:30:in `dispatch'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/cli.rb:24:in `start'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/exe/bundle:49:in `block in <top (required)>'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/friendly_errors.rb:130:in `with_friendly_errors'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/exe/bundle:37:in `<top (required)>'
#18 3.540 /usr/local/bundle/bin/bundle:23:in `load'
#18 3.540 /usr/local/bundle/bin/bundle:23:in `<main>'
#18 3.540 Bundler Error Backtrace:
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/runtime.rb:69:in `rescue in block (2 levels) in require'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/runtime.rb:65:in `block (2 levels) in require'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/runtime.rb:61:in `each'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/runtime.rb:61:in `block in require'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/runtime.rb:50:in `each'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/runtime.rb:50:in `require'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler.rb:173:in `require'
#18 3.540 /app/config/application.rb:9:in `<top (required)>'
#18 3.540 /app/Rakefile:6:in `require'
#18 3.540 /app/Rakefile:6:in `<top (required)>'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/lib/rake/rake_module.rb:29:in `load'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/lib/rake/rake_module.rb:29:in `load_rakefile'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/lib/rake/application.rb:703:in `raw_load_rakefile'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/lib/rake/application.rb:104:in `block in load_rakefile'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/lib/rake/application.rb:186:in `standard_exception_handling'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/lib/rake/application.rb:103:in `load_rakefile'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/lib/rake/application.rb:82:in `block in run'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/lib/rake/application.rb:186:in `standard_exception_handling'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/lib/rake/application.rb:80:in `run'
#18 3.540 /usr/local/bundle/gems/rake-13.0.3/exe/rake:27:in `<top (required)>'
#18 3.540 /usr/local/bundle/bin/rake:23:in `load'
#18 3.540 /usr/local/bundle/bin/rake:23:in `<top (required)>'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/cli/exec.rb:63:in `load'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/cli/exec.rb:63:in `kernel_load'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/cli/exec.rb:28:in `run'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/cli.rb:494:in `exec'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/cli.rb:30:in `dispatch'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/cli.rb:24:in `start'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/exe/bundle:49:in `block in <top (required)>'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/lib/bundler/friendly_errors.rb:130:in `with_friendly_errors'
#18 3.540 /usr/local/bundle/gems/bundler-2.2.16/exe/bundle:37:in `<top (required)>'
#18 3.540 /usr/local/bundle/bin/bundle:23:in `load'
#18 3.540 /usr/local/bundle/bin/bundle:23:in `<main>'
@firefart
Copy link
Author

Another project failing now at runtime because of a missing gcc

#16 2.194 /usr/local/bundle/gems/ffi-1.15.0/lib/ffi/tools/const_generator.rb:131:in `block in calculate': Compilation error generating constants : (RuntimeError)
#16 2.197 sh: gcc: not found
#16 2.197 from /usr/local/lib/ruby/2.7.0/tempfile.rb:291:in `open'
#16 2.197 from /usr/local/bundle/gems/ffi-1.15.0/lib/ffi/tools/const_generator.rb:110:in `calculate'
#16 2.197 from /usr/local/bundle/gems/ffi-1.15.0/lib/ffi/tools/const_generator.rb:49:in `initialize'
#16 2.197 from /usr/local/bundle/gems/ethon-0.13.0/lib/ethon/libc.rb:17:in `new'
#16 2.197 from /usr/local/bundle/gems/ethon-0.13.0/lib/ethon/libc.rb:17:in `<module:Libc>'
#16 2.197 from /usr/local/bundle/gems/ethon-0.13.0/lib/ethon/libc.rb:7:in `<module:Ethon>'
#16 2.197 from /usr/local/bundle/gems/ethon-0.13.0/lib/ethon/libc.rb:2:in `<top (required)>'
#16 2.197 from /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
#16 2.197 from /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
#16 2.197 from /usr/local/bundle/gems/ethon-0.13.0/lib/ethon.rb:16:in `<top (required)>'
#16 2.197 from /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
#16 2.197 from /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
#16 2.197 from /usr/local/bundle/gems/typhoeus-1.4.0/lib/typhoeus.rb:2:in `<top (required)>'
#16 2.197 from /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
#16 2.197 from /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
#16 2.197 from /usr/local/bundle/gems/cms_scanner-0.13.3/lib/cms_scanner.rb:4:in `<top (required)>'
#16 2.197 from /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
#16 2.197 from /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
#16 2.197 from /usr/local/bundle/gems/wpscan-3.8.17/lib/wpscan.rb:8:in `<top (required)>'
#16 2.197 from /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
#16 2.197 from /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'

@stuartgibson
Copy link

Same issue here with Alpine image

@firefart firefart mentioned this issue Apr 13, 2021
Closed
@Kjarrigan
Copy link
Member

Yes. GCC is now required by ethon 0.13.0 - see #166

@firefart
Copy link
Author

Hi @Kjarrigan ,
sorry but I think GCC should never be a runtime dependency. Is there any technical requirement for this or is there a chance to disable this behaivour? Installing gcc for a library to make http reuqests seems like a bit of an overkill

@Kjarrigan
Copy link
Member

This is an indirect "dependency" as ffi uses gcc for the FFI::ConstGenerator and I'm not aware that there is a way to disable it but feel free to open a MR with a proposal on how to adjust this to fit your needs.
I'm using an alpine docker container in a lot of cases myself, but already usually have build-base installed anyways for other dependent gems.

@sullerandras
Copy link

Downgrading ethon from 0.13.0 to 0.12.0 solved the issue for me. I also don't want to add gcc to the production docker image (we only install it for the duration of running bundle install).

@leshik
Copy link

leshik commented Apr 19, 2021 •
edited
Loading

@sullerandras @Kjarrigan @firefart you can use FFI master, and then install tcc instead of gcc, which is only 256 kb in size. Then set the CC env to tcc and that's it.

@deryni
Copy link

deryni commented Apr 22, 2021

Forgive the potentially silly question but is this new FFI dependency not equivalent to Etc.sysconf(Etc::SC_OPEN_MAX)? Or is that not usable here for some reason?

@mindw
Copy link

mindw commented Apr 24, 2021

@sullerandras @Kjarrigan @firefart you can use FFI master, and then install tcc instead of gcc, which is only 256 kb in size. Then set the CC env to tcc and that's it.

Just trying to clarify things, the issue isn't GCC, but any compiler. This is commonly suggested by Docker image security guidelines https://sysdig.com/blog/dockerfile-best-practices/.

Thanks!

@Kjarrigan Kjarrigan mentioned this issue Apr 26, 2021
Merged
@Kjarrigan
Copy link
Member

@i0rek could you please deploy a new ethon version - I just merged/reverted the problematic MR since this obviously is a problem for a lot of ethon/typhoeus users. I also created a new issue #192 as a reminder that his is something we eventually need to fix.

@Kjarrigan Kjarrigan reopened this Apr 26, 2021
@hanshasselberg
Copy link
Member

Released ethon 0.14.0.

This was referenced Apr 26, 2021
Closed
Closed
@Kjarrigan
Copy link
Member

Everyone seems to be happy now. So closing. Will add a CHANGELOG entry soon-ish.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@hanshasselberg @firefart @deryni @Kjarrigan @sullerandras @leshik @mindw @stuartgibson