Unable to install bioperl on Mac OS X - macos

I'm trying to get biotools working on my Mac so that I can run some Perl5 code that uses Bio::DB::Sam, but am stymied.
Mac OS X 10.10.4
perl 5.18.2
upgraded CPAN as per INSTALL instructions
'brew install expat' tells me that expat-2.1.0_1 is already installed
'sudo perl -MCPAN -e shell'
'install CJFIELDS/BioPerl-1.6.924.tar.gz'
'Do you want to run the Bio::DB::GFF or Bio::DB::SeqFeature::Store live database tests?' => 'n'
Install all
'Do you want to run tests that require connection to servers across the internet' => 'n'
Eventually get (with some lines deleted):
Running Build test
t/Align/AlignStats.t ................... ok
t/Align/AlignUtil.t .................... ok
t/Align/Graphics.t ..................... skipped: The optional module GD (or dependencies thereof) was not installed
...
t/AlignIO/msf.t ........................ ok
t/AlignIO/nexml.t ...................... skipped: The optional module Bio::Phylo (or dependencies thereof) was not installed
t/AlignIO/nexus.t ...................... ok
...
t/Assembly/ContigSpectrum.t ............ ok
t/Assembly/IO/bowtie.t ................. skipped: The optional module Bio::DB::Sam (or dependencies thereof) was not installed
t/Assembly/IO/sam.t .................... skipped: The optional module Bio::DB::Sam (or dependencies thereof) was not installed
t/Assembly/core.t ...................... ok
t/Cluster/UniGene.t .................... ok
Afterwards, I test with:
perl -e "use Bio::DB::Sam;"
and get:
Can't locate Bio/DB/Sam.pm in #INC (you may need to install the Bio::DB::Sam module) (#INC contains: /Library/Perl/5.18/darwin-thread-multi-2level /Library/Perl/5.18 /Network/Library/Perl/5.18/darwin-thread-multi-2level /Network/Library/Perl/5.18 /Library/Perl/Updates/5.18.2/darwin-thread-multi-2level /Library/Perl/Updates/5.18.2 /System/Library/Perl/5.18/darwin-thread-multi-2level /System/Library/Perl/5.18 /System/Library/Perl/Extras/5.18/darwin-thread-multi-2level /System/Library/Perl/Extras/5.18 .) at -e line 1.
BEGIN failed--compilation aborted at -e line 1.
I get the same results when cloning bioperl-live from GitHub (sync'd to revision 73c446c69a77) and trying to install that way.
Note that I have installed samtools 0.1.18 (to match the version on our cluster) by:
downloading the .tar.gz
running 'make'
copying 'samtools', 'bcftools/bcftools', and 'misc/*.pl' to ~/debarcer-packages/bin, which is on my path
Afterward, I get this:
$ which samtools
/Users/gvwilson/debarcer-packages/bin/samtools
This build did not produce a '.so' file, even though there is a rule in the samtools-0.1.18 Makefile that looks like it (maybe?) ought to produce one.

The module Bio::DB::Sam provides bindings to an older version of samtools that did not rely htslib. This is an important point because you may run into issues using SAM/BAM files generated with samtools or other aligners because most tools use htslib these days.
For building the module, you are on the right track with the version you are using but it is difficult to build if you do not know the correct flags. I previously provided a solution to do this and I'll show a better way here (just use a package manager for the Perl module).
wget http://sourceforge.net/projects/samtools/files/samtools/0.1.18/samtools-0.1.18.tar.bz2
tar xjf samtools-0.1.18.tar.bz2 && cd samtools-0.1.18
make CFLAGS=-fPIC
export SAMTOOLS=`pwd`
The last command will allow you to install the Perl module without looking for the PATH to samtools and being prompted for it. Note, the extra CFLAGS argument may not be needed on your Mac, so try without it first. It is required on Linux, and since the module uses so much memory you will likely only be using this on a Linux machine. Now, install the Perl module.
cpanm Bio::DB::Sam
or cpan if you prefer. That should get you a working Bio::DB::Sam. I don't know what you are trying to do but I will mention that the fine folks over at EBI have developed bindings to htslib called Bio::DB::HTS based on Lincoln Stein's XS code in the Bio::DB::Sam module. This is really what you should be using because the version of SAMtools mentioned above is really old and not being developed. That is my opinion and a word of caution though, nothing wrong with Bio::DB::Sam.
edit:
You find it easier to manage Perl without using the "system" Perl, and here is one solution. Other people may have their preferred method, but perlbrew (combined with cpanminus) will make this type of work fun and much less of a pain (and they are popular choices). That would be my first step: set up perlbrew, install Perl 5.22, then install cpanminus. That might sound challenging but it is just a few commands. Something along the lines of:
curl -L http://install.perlbrew.pl | bash
source ~/perl5/perlbrew/etc/bashrc
perlbrew install perl-5.22.1
perlbrew switch perl-5.22.1
perlbrew install-cpanm
should do the trick. That will give you a kick-ass Perl with some nice features not available with your "system" Perl. This is a good idea because using /usr/bin/perl requires sudo, it involves messing with the system libraries which might cause an issue, and the recent Apple changes mean that working with root directories/libraries is completely unstable.

You need to install an additional (optional) module to use samtools. That's what the The optional module Bio::DB::Sam message is about. You don't need it for the rest of BioPerl, so it's not a hard dependency.
For Bio::DB::Sam, you need samtools-0.1.17 (the latest version the module works with according to its docs). I downloaded the source and ran make. There were some warnings, but it appears to work. From your question, I don't think you had a problem here.
I then installed Bio::DB::Sam:
$ cpan Bio::DB::Sam
There were some compiler warnings, but the module passed its tests and installed. The cpan command took care of dependencies too, so it also installed BioPerl for me.
If you need some environment variables, you can set them for a one-off run of the command:
$ CFLAGS=... SAMTOOLS=... cpan Bio::DB::Sam
Note that installing Bio::DB::Sam prompted me for the location of samtools. I pointed it at the build directory:
$ cpan5.22.0 Bio::DB::Sam
Running install for module 'Bio::DB::Sam'
Configuring L/LD/LDS/Bio-SamTools-1.43.tar.gz with Build.PL
This module requires samtools 0.1.10 or higher (samtools.sourceforge.net).
Please enter the location of the bam.h and compiled libbam.a files: /Users/brian/Downloads/samtools-0.1.17
I'm betting there's not something complicated like the answer SES gave. You just need an optional module. The README for Bio::DB::Sam notes some problems that people might have and offers so workarounds, but I didn't run into these problems and my setup is close to yours.
Note that Alien::SamTools is a Perl package that installs the non-Perl samtools, but it says it installs 0.1.19. Maybe that works too, but that's not what Bio::DB::Sam says on the tin.

Related

How to specify external library file locations when installing CPAN modules

I'm trying to get Perl's Finance::Quote module working after MacOS upgrade to 11.4. One of the dependencies is B::Keywords. B::Keywords installation fails a test with this error:
> sudo cpan B::Keywords
[...]
Can't open /System/Library/Perl/5.30/darwin-thread-multi-2level/CORE/keywords.h: No such file or directory at t/11keywords.t line 25
Digging around, I see that keywords.h exists on my system in this location:
/System/Volumes/Data/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Perl/5.30/darwin-thread-multi-2level/CORE/keywords.h
Is there a way to tell cpan (or cpanm, or some other installation tool) where to look for these headers? Or, any other approach to get this working would be welcome.
It seems like they removed the keywords.h from the CORE directory (relative to the path given from $Config{archlibexp}) for the system perl on macOS 10.14, see this bug report.
The reason you are not able to install B::Keywords is due to a failed test 11keywords.t see line 24. Some possible solutions:
Install the module without running the tests (sudo cpan -T B::Keywords)
Submit an issue at the GitHub issue tracker so the author of the module can fix the problem.
Install the module with perlbrew instead of using the system perl (I tested this and it worked fine here).

pp (perl compiler) issue - still has a dependency

I'm trying to use pp (the perl compiler) to create an application that can run independent of the perl installed library and interpreter.
It successfully creates a compiled executable although I had to use the -x -c options to get it to find dependencies successfully. It will run on my machine but when I try it on another machine I get this error so clearly there is still some dependency:
501 Protocol scheme 'https' is not supported (LWP::Protocol::https not installed)
I am running it on MacOS 10.14.1 if that makes any difference. Thanks!
LWP::Protocol::https is loaded dynamically when needed, so pp has no way of knowing it's needed by default.
Solution 1
Pass -x to pp, and make sure the module is actually loaded in the run pp uses to determine the modules to include. This would probably be achieved by using LWP to make an HTTPS request during that run. --xargs=... might come in useful for this.
Solution 2
Pass -M LWP::Protocol::https to pp. You could also pass -M 'LWP::Protocol::**' to get all protocols handlers you have installed.
Solution 3
Add use LWP::Protocol::https (); to your script or an included module. Including a comment indicating why you are doing this would be appropriate.
You were building Net::SSLeay on MacOS 10.14 linking it to libssl.44.dylib which is not present on MacOS 10.12 where you try to run it.
I've found it annoying having to switch between build and test systems to find out which of the libraries are missing or incompatible and need to be packed.
I am now using the following strategy:
I use perlbrew instead of system perl.
For alien dependencies I use homebrew instead of the system libraries.
I build the packed executable using pp and run the resulting program with export DYLD_PRINT_LIBRARIES=YES being set (on the development machine)
I examine the list of loaded libraries and add all those referenced in the homebrew directory tree (/usr/local/opt/ and /usr/local/cellar/in my case) using pp -l /full/path/name -l ...
I rebuild the executable.
I still check on a target machine before deploying, but chances are very high now that it just works.

bootstrap cpan without make

I want to use the cpan tool get a library from CPAN (HTTP::Tiny::Multipart). (Or should I be using something else??) It is written in pure Perl so I hope I don't need make for this. Can this be done?
At first I couldn't get things to download at all, but eventually I found out the issue was proxies, and now with the right environment variables set I can successfully reach CPAN. But now I get error messages about a missing make.
There are three common installers for Perl module distributions. To use these installers, one follows one of the following two sequences of steps:
perl Makefile.PL # ExtUtils::MakeMaker or compatible
make
make test
make install
or
perl Build.PL # Module::Build or compatible
./Build
./Build test
./Build install
cpan handles downloading the distributions and installing prerequisites, but it doesn't replace the distribution's installer. When it comes time to install a download distribution, it uses one of the above sequence (depending on whether the distribution includes a file named Build.PL or not).
HTTP-Tiny-Multipart (the distribution that provides HTTP::Tiny::Multipart) uses ExtUtils::MakeMaker as its installer, so installing it requires make.
The underlying issue is that you are using a gutted environment. It's not the purpose of git to provide perl to you.
If you want the full unix emulation environment for which git and its perl were built[1], install Cygwin. Alternatively, you could also just as easily install a native Perl such as ActivePerl or Strawberry Perl.
Technically, git and its perl are built for MSYS, which is a subset of Cygwin.

How to force the INC and LIB path defaults for CPAN builds (cpanm+perlbrew)

Background
Working with OS X and using macports for additional utilities and libraries. The macports are installing everything into /opt/local. But the default perl in macports is 5.12, specifying an higher version like to break many ports, so I'm using my own 5.16 perl installed with perlbrew.
So, I have libraries in the default /usr/lib (provided by Apple) but want linking XS based perl modules with the macports provided libs and includes at /opt/local.
Now the question, when installing modules with cpanm (into my perlbrew's perl tree), is here some way how to specify globally the default search path for libraries and includes for cpanm (to the modules build process) to search /opt/local/{lib|include} at first?
AFAIK the cpanm havent any switches for this, so maybe, some ENV variables? Or any simpler solution as edit every Makefile.PL?
Install half manually. Run cpanm --look Some::Dist and supply the appropriate INC and LIBS or --include_dirs and --extra_compiler_flags to the build tool on the command line to configure, then finish off with the build, test, install steps and close the subshell.
Alternatively, if you need to fully automate this, then use a full-featured CPAN client instead of cpanm. For example, run the cpan shell and configure makepl_arg and mbuildpl_arg. This is then supplied to subsequent installations (instead of just one, as above).

Errors when installing a Perl module using make - Mac OSX 10.7

The problem:
I can't seem to install perl modules correctly, JSON-2.53 in particular.
I have done the following:
Searched for a similar problem and tried its solution - did not work.
perl ".../config.h, needed by `Makefile'" not working after OSX Lion upgrade
Installed XCode command line developer utilities (c compiler, make, etc)
Read version compatibility documentation on this particular perl module: http://metacpan.org/pod/JSON
Ran the following commands to make and install the desired perl module:
$perl Makefile.PL
Welcome to JSON (v.2.53)
If you install JSON::XS v.2.27, it makes JSON faster.
************************** CAUTION **************************
This is 'JSON version 2' and there are many differences *
to version 1.xx *
Please check your applications useing old version. *
See to 'INCOMPATIBLE CHANGES TO OLD VERSION' and 'TIPS' *
Writing Makefile for JSON
(verified that the Makefile has been written)
$make
make: *** No rule to make target `/System/Library/Perl/5.12/darwin-thread-multi-2level/CORE/config.h', needed by `Makefile'. Stop.
What does that error even mean? What can I do to successfully make install this module?
Here are some additional items that may help you assist me in debugging this issue:
$which make
/Applications/Xcode.app/Contents/Developer/usr/bin/make
$which perl
/usr/bin/perl
$perl -v
This is perl 5, version 12, subversion 3 (v5.12.3) built for darwin-thread-multi-2level
I think you need to download and reinstall XCode. If I recall correctly for 10.7, after downloading Xcode from the app store it drops an installer into your Applications folder. You need to run it and try installing the command line tools again (from Xcode's prefernces pane). I know you mentioned you did this already, but a bit more background might explain why it's worth another try.
Here are the relevant lines in the Makefile from my Mac:
PERL_INC = /System/Library/Perl/5.12/darwin-thread-multi-2level/CORE
# Where is the Config information that we are using/depend on
CONFIGDEP = $(PERL_ARCHLIB)$(DFSEP)Config.pm $(PERL_INC)$(DFSEP)config.h
Later on in the Makefile CONFIGDEP is used as a dependency in a target. I believe in your case make is looking for /System/Library/Perl/5.12/darwin-thread-multi-2level/CORE/config.h and can't find it. The error you're seeing is make's obtuse way of saying file not found.
config.h contains specific information about the OS but is not needed for running scripts. It's only referenced when you want to compile a module. With stock OSX you get enough perl to execute scripts. Install XCode and you get the bits (like config.h) to do perl "development". I use quotes because you can write and run perl scripts without Xcode. But as you discovered, compiling a module requires the additional files Xcode provides. (Incidentally, RedHat does the same thing. You have to install the perl-devel package to get config.h. The perl runtime is in a separate package.)
Here are some things you can try:
Verify /System/Library/Perl/5.12/darwin-thread-multi-2level/CORE/config.h exists. If not, Xcode command line utilities were not installed properly. Try it again.
If config.h exists, check its content and make sure it looks sane. It's a C header file and consists of comments and #define statements.
If you don't have access to view config.h, you have a permission issue. Try using sudo make as a bypass. Disk Utility (found in Applications -> Utilities) might be able to permanently fix this.
You could risk changing the Makefile by removing "$(PERL_INC)$(DFSEP)config.h" from CONFIGDEP. I did this on my 10.8 Mac and it worked without issue (it passed all tests as well). However, if you don't find the root cause of your config.h issue, the next time you want to install a perl module you may find yourself right back where you started.
I had this exact same error, whilst this may not be a solution for you.... after reinstalling an updated xcode compatible with the OSX version (+rebooting after the install) I still had the error - to cut a long story short I noticed there was no config.h in /CORE/ after the error.....the solution that worked was to touch config.h and create the file first and then re-run the make. Hope this helps someone.

Resources