Bug #116683 for Bio-SamTools: Bio-SamTools-1.43 is confused about what various SAMtools build dirs supply where - and so am I

Got a bit further If I link three flles from within the htslib-1.3/htslib/ directory up into the top build directpory for samtools-1.3 $ ln -s htslib-1.3/htslib/khash.h . $ ln -s htslib-1.3/htslib/faidx.h . $ ln -s htslib-1.3/htslib/hts_defs.h . then I get to defeat the fact that the way the #includes are defined differently within lib/Bio/DB/Sam.xs #include "bam.h" #include "khash.h" #include "faidx.h" and in the samtools-1.3/sam.h #include "htslib/sam.h" and i ovecome the fact that the Bio-SamTools Build.PL doesn't generate a correct set of -I/path/to/include/dirs. However, I now see that there's probably some even greater mis-match between what Bio-SamTools-1.43 and samtools-1.3 think about how the internal structures are defined: lib/Bio/DB/Sam.xs: In function 'XS_Bio__DB__Bam__Alignment_l_aux': lib/Bio/DB/Sam.xs:575: error: 'bam1_t' has no member named 'l_aux' lib/Bio/DB/Sam.xs:576: error: 'bam1_t' has no member named 'l_aux' Has anyone out there compiled Bio-SamTools-1.43 against a SMAtools version and, if so, which version ?

I do hope that someone is using this software to study worms, as this issue seems to have been one massive can of them, however I think I have accounted for all of them now. Based on what I'd seen so far, I decided to go looking in the past versions of SAMtools, in case it was possible to determine which one of them might match the perculiar layout of header files that Bio-SamTools clearly expected to be there in the SAMtools it was pointed to. Here's what I found, amongst the ones I had lying around from previous encounters into the SAMtools universe SAMtools 1.2 * doesn't have a plain htslib subdir but htslib-1.2.1 * bam.h has the note about htslib version doesn't have l_aux SAMtools 1.1 (Github doesn't bundle htslib) * doesn't have an htslib subdir AT ALL * bam.h has the note about htslib version doesn't have l_aux * Doesn't build: Makefile:86: ../htslib/htslib.mk: No such file or directory SAMtools 1.1 (Sourceforge) * doesn't have a plain htslib subdir but htslib-1.1 * bam.h has the note about htslib version doesn't have l_aux SAMtools 1.0 (Sourceforge) * doesn't have a plain htslib subdir but htslib-1.0 * bam.h has the note about htslib version doesn't have l_aux SAMtools 0.1.19 (Sourceforge) * doesn't have an htslib subdir AT ALL * bam.h DOES HAVE a bam1_t type which has an l_aux * Might have all the required header files So now I go and build a SAMtools 0.1.19 and point Bio-SamTools at that and get a bit further, but it fails trying to do this gcc -shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions \ -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic \ -o blib/arch/auto/Bio/DB/Sam/Sam.so lib/Bio/DB/Sam.o c_bin/bam2bedgraph.o \ -L/path/to/samtools-0.1.19 -lbam \ -lpthread -lz /usr/bin/ld: /path/to/samtools-0.1.19/libbam.a(bgzf.o): \ relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making \ a shared object; recompile with -fPIC /path/to/samtools-0.1.19/libbam.a: could not read symbols: Bad value However, that -lbam in the compilation looked to me as though it was trying to link in a dynamic library and not the static one it was pointed to as part of the configuration - surely not ? So I thought to go and create a libbam.so dynamic library, using all of the object files that SAMtools 0.1.19 produces in creating the static library. After the main build doing its: ar -csru libbam.a bgzf.o kstring.o bam_aux.o bam.o bam_import.o sam.o \ bam_index.o bam_pileup.o bam_lpileup.o bam_md.o razf.o faidx.o bedidx.o \ knetfile.o bam_sort.o sam_header.o bam_reheader.o kprobaln.o bam_cat.o I go and do a gcc -shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions \ -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic \ -o libbam.so \ bgzf.o kstring.o bam_aux.o bam.o bam_import.o sam.o \ bam_index.o bam_pileup.o bam_lpileup.o bam_md.o razf.o faidx.o bedidx.o \ knetfile.o bam_sort.o sam_header.o bam_reheader.o kprobaln.o bam_cat.o and I find I can get the Bio-SamTools module to build, however it fails to run any of its tests, notionally because the libbam.so isn't available to it at run time. I say "notionally" because it was that point, having seemingly built the Bio-SamTools module but failed the tests, that I decided to take a look at the Bio-SamTools test source lying around within the Perl build directorires, and what do you think I saw ... There's a README File and it says, right at the top, ---8<------8<------8<------8<------8<------8<------8<------8<------8<--- This is a Perl interface to the SAMtools sequence alignment interface. It ONLY works on versions of Samtools up to 0.1.17. It does not work on version 1.0 or higher due to major changes in the library structure. ---8<------8<------8<------8<------8<------8<------8<------8<------8<--- so there's the answer to my question from the previous posts: I should be building against 0.1.17. Seems a shame that no-one could be bothered to tell me that but hey-ho, no matter. Armed with that knowledge and the knowledge that Bio-SamTools is going to try and link against a non-existant dynamic libray that I'll need to create for myself, I added, to my list of potential SAMtools versions SAMtools 0.1.17 (Sourceforge) * doesn't have an htslib subdir AT ALL * bam.h DOES HAVE a bam1_t type which has an l_aux * Might have all the required header files and proceed as follows, noting that I'll still need to create a link to get around the way that the htslib headers are referred to in the various parts of Bio-SamTools tar xf amtools-0.1.17.tar.bz2 cd samtools-0.1.17 cp Makefile Makefile.orig vim Makefile make ln -s . htslib where the Makefile gets altered as follows 2c2 < CFLAGS= -g -Wall -O2 #-m64 #-arch ppc --- Show quoted text
I then build the shared library as shown above, adding in the -fPIC, and make it available, to the yet to be built Bio-SamTools module, by poking it into LD_LIBRARY_PATH export LD_LIBRARY_PATH=/path/to/samtools-0.1.17 The build of Bio-SamTools module no runs through and we see the following files being installed created into the Perl heirachy (paths truncated to avoid wrapping) Installing .../lib/perl5/x86_64-linux-thread-multi/auto/Bio/DB/Sam/Sam.so Installing .../lib/perl5/x86_64-linux-thread-multi/auto/Bio/DB/Sam/Sam.bs Installing .../bin/bam2bedgraph Installing .../lib/perl5/x86_64-linux-thread-multi/Bio/DB/Sam.pm Installing .../lib/perl5/x86_64-linux-thread-multi/Bio/DB/Sam/Segment.pm Installing .../lib/perl5/x86_64-linux-thread-multi/Bio/DB/Sam/Constants.pm This (OK, make that "All this") seems to have done the trick. I hope documenting this might be useful to someone, although, given the lack of feedback, it's quite possible that no-one, bar the user I was looking to install it for on my resources, is even using it anymore? Footnote: I suppose it's possible that, even though the README says that it only works with 0.1.17, there's clearly enough similarity between 0.1.17 and 0.1.19 failing to build that Bio-SamTools-1.43 might even compile aganst that later version, as long as the hoop-jumping above id performed, however I'm quitting whilst I am ahead, and will leave that as an exercise for the reader, should this thread ever have one, I guess.