debugging builds

[jvolkening]

I'm trying to put together my first bioconda recipes and am having trouble debugging. I follow the the instructions here using simulate-travis.py but the builds are failing and all I can see toward the end of the output is

Source cache directory is: /opt/conda/conda-bld/src_cache
Downloading source to cache: tpp_5.0.0.tar.gz
Downloading https://downloads.sourceforge.net/project/sashimi/Trans-Proteomic%20Pipeline%20%28TPP%29/TPP%20v5.0%20%28Typhoon%29%20rev%200/TPP_5.0.0-src.tgz
Success
Extracting download
Applying patch: '/opt/recipe/tpp5.0.0.patch'
BUILD START (revised): tpp-5.0.0-0
Package: tpp-5.0.0-0
source tree in: /opt/conda/conda-bld/tpp_1506024870342/work/TPP_5.0.0-src
Traceback (most recent call last):
  File "/opt/conda/bin/conda-build", line 6, in <module>
    sys.exit(conda_build.cli.main_build.main())
  File "/opt/conda/lib/python3.5/site-packages/conda_build/cli/main_build.py", line 342, in main
    execute(sys.argv[1:])
  File "/opt/conda/lib/python3.5/site-packages/conda_build/cli/main_build.py", line 333, in execute
    noverify=args.no_verify)
  File "/opt/conda/lib/python3.5/site-packages/conda_build/api.py", line 97, in build
    need_source_download=need_source_download, config=config)
  File "/opt/conda/lib/python3.5/site-packages/conda_build/build.py", line 1522, in build_tree
    config=config)
  File "/opt/conda/lib/python3.5/site-packages/conda_build/build.py", line 1145, in build
    utils.check_call_env(cmd, env=env, cwd=src_dir)
  File "/opt/conda/lib/python3.5/site-packages/conda_build/utils.py", line 628, in check_call_env
    return _func_defaulting_env_to_os_environ(subprocess.check_call, *popenargs, **kwargs)
  File "/opt/conda/lib/python3.5/site-packages/conda_build/utils.py", line 624, in _func_defaulting_env_to_os_environ
    return func(_args, **kwargs)
  File "/opt/conda/lib/python3.5/subprocess.py", line 581, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/bin/bash', '-x', '-e', '/opt/conda/conda-bld/tpp_1506024870342/work/TPP_5.0.0-src/conda_build.sh']' returned non-zero exit status 2


15:44:02 BIOCONDA ERROR BUILD FAILED recipes/tpp, CONDA_BOOST=1.64;CONDA_GMP=5.1;CONDA_GSL=1.16;CONDA_HDF5=1.8.17;CONDA_HTSLIB=1.5;CONDA_NCURSES=5.9;CONDA_NPY=112;CONDA_PERL=5.22.0;CONDA_PY=27;CONDA_R=3.3.2;MACOSX_DEPLOYMENT_TARGET=10.9
15:44:04 BIOCONDA ERROR BUILD SUMMARY: of 2 recipes, 2 failed and 0 were skipped. Details of recipes and environments follow.
15:44:04 BIOCONDA ERROR BUILD SUMMARY: FAILED recipe canu-1.6-pl5.22.0_1.tar.bz2, environment CONDA_GMP=5.1;CONDA_NCURSES=5.9;CONDA_HTSLIB=1.5;CONDA_PY=27;CONDA_NPY=112;CONDA_HDF5=1.8.17;CONDA_GSL=1.16;CONDA_PERL=5.22.0;CONDA_BOOST=1.64;MACOSX_DEPLOYMENT_TARGET=10.9;CONDA_R=3.3.2
15:44:04 BIOCONDA ERROR BUILD SUMMARY: FAILED recipe tpp-5.0.0-0.tar.bz2, environment CONDA_GMP=5.1;CONDA_NCURSES=5.9;CONDA_HTSLIB=1.5;CONDA_PY=27;CONDA_NPY=112;CONDA_HDF5=1.8.17;CONDA_GSL=1.16;CONDA_PERL=5.22.0;CONDA_BOOST=1.64;MACOSX_DEPLOYMENT_TARGET=10.9;CONDA_R=3.3.2

which doesn't seem to give any specifics. I can't find any relevant logs in /tmp/miniconda and I'm assuming that since it is a Docker build the relevant output gets lost with the instance.

I can run the build.sh just fine outside of Docker, so I'm assuming there is something wrong in the Docker environment. How can I go about debugging this? Any advice is appreciated.

FYI, the recipe in question is here and is related to galaxyproteomics/tools-galaxyp#108

[bgruening]

Do you have more output above your pasted output? You can also increase the debug output.
You could also manually change into the container and run the commands to test it.

Please also have a look at my attempt for TPP one year ago: https://github.com/bioconda/bioconda-recipes/compare/tpp?expand=1

TPP is not an easy one to start with :(

Fee also free to start a WIP PR or a branch so that we can see the travis log and help you.

[jvolkening]

Okay, I found this further up:

common.mk:160: *** Value of build TARGET doesn't appear to be a machine-vendor-system triplet. Stop.

which explains what is wrong but not why it's failing in Docker and not on my system.

You could also manually change into the container and run the commands to test it.

Can you (easily) guide me on how to do this or direct me to relevant documentation?

Fee also free to start a WIP PR or a branch so that we can see the travis log and help you.

Is it considered okay/polite to submit a PR that I know is going to fail testing?

[jvolkening]

Please also have a look at my attempt for TPP one year ago: https://github.com/bioconda/bioconda-recipes/compare/tpp?expand=1

Yes, that was my jumping-off point but I decided to go with TPP 5.0 because I can't get 4.8 to compile anymore even on my regular system.

[bgruening]

Is it considered okay/polite to submit a PR that I know is going to fail testing?

In worse cases I think this is not a problem. And TPP is a worse case and deserves help from as much people as we can get :)

It fails probably in Docker for you, because in Docker there are very less libraries installed - you really need to specify all dependencies, even zlib and such.

Try to do docker run -i -t condaforge/linux-anvil bash and try to compile TPP step by step.

[jvolkening]

Try to do docker run -i -t condaforge/linux-anvil bash and try to compile TPP step by step.

Thanks, this is exactly what I was looking for. I will see where I can take this and then return if I have further questions.