Using cctbx::uctbx::unit_cell in c++ code / boosted to Python

newer
Strange behaviour of resolution...

Graeme.Winter＠diamond.ac.uk

25 Jan 2013 25 Jan '13

1:46 p.m.

Hi Folks, Been working on some c++ coding using boost etc (which I have to say is great) however I found one bit of weirdness. If I want to generate a cctbx::uctbx::unit_cell in c++ code then access this from Python I get Traceback (most recent call last): File "tst_x2tbx.py", line 67, in <module> tst_x2tbx(sys.argv[1]) File "tst_x2tbx.py", line 6, in tst_x2tbx import x2tbx File "/Users/graeme/svn/cctbx/sources/x2tbx/__init__.py", line 7, in <module> ext = boost.python.import_ext("x2tbx_ext", optional = False) File "/Users/graeme/svn/cctbx/sources/cctbx_project/boost_adaptbx/boost/python.py", line 36, in import_ext + [" "+p for p in sys.path])) ImportError: __import__("x2tbx_ext"): dlopen(/Users/graeme/svn/cctbx/prebuild/lib/x2tbx_ext.so, 2): Symbol not found: cctbx::uctbx::unit_cell::unit_cell(scitbx::af::tiny const&) Referenced from: /Users/graeme/svn/cctbx/prebuild/lib/x2tbx_ext.so Expected in: flat namespace in /Users/graeme/svn/cctbx/prebuild/lib/x2tbx_ext.so sys.path: …. unless I have import cctbx.uctbx # explicit import at the top of the Python code - have I done something wrong with the SConscript? I found something similar with import scitbx.array_family.flex # explicit import which libtbx.clean_clutter complains about as I don't reference it (hence the additional comment) Equally I could just leave the additional clutter there but it feels like I should not need it... Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

Show replies by date

Graeme.Winter＠diamond.ac.uk

25 Jan 25 Jan

2:21 p.m.

Interesting, adding env.Append(LIBS=['cctbx']) To my SConscript made this go away. Nothing like looking at other examples… Cheerio, Graeme On 25 Jan 2013, at 13:46, wrote:

...

Hi Folks,

Been working on some c++ coding using boost etc (which I have to say is great) however I found one bit of weirdness.

If I want to generate a cctbx::uctbx::unit_cell in c++ code then access this from Python I get

Traceback (most recent call last): File "tst_x2tbx.py", line 67, in <module> tst_x2tbx(sys.argv[1]) File "tst_x2tbx.py", line 6, in tst_x2tbx import x2tbx File "/Users/graeme/svn/cctbx/sources/x2tbx/__init__.py", line 7, in <module> ext = boost.python.import_ext("x2tbx_ext", optional = False) File "/Users/graeme/svn/cctbx/sources/cctbx_project/boost_adaptbx/boost/python.py", line 36, in import_ext + [" "+p for p in sys.path])) ImportError: __import__("x2tbx_ext"): dlopen(/Users/graeme/svn/cctbx/prebuild/lib/x2tbx_ext.so, 2): Symbol not found: cctbx::uctbx::unit_cell::unit_cell(scitbx::af::tiny const&) Referenced from: /Users/graeme/svn/cctbx/prebuild/lib/x2tbx_ext.so Expected in: flat namespace in /Users/graeme/svn/cctbx/prebuild/lib/x2tbx_ext.so sys.path: ….

unless I have

import cctbx.uctbx # explicit import

at the top of the Python code - have I done something wrong with the SConscript? I found something similar with

import scitbx.array_family.flex # explicit import

which libtbx.clean_clutter complains about as I don't reference it (hence the additional comment)

Equally I could just leave the additional clutter there but it feels like I should not need it...

Thanks,

Graeme

-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

Luc Bourhis

2:21 p.m.

New subject: Using cctbx::uctbx::unit_cell in c++ code / boosted to Python

On 25 Jan 2013, at 14:46, [email protected] wrote:

...

Equally I could just leave the additional clutter there but it feels like I should not need it...

You have a function "generate_unit_cell" written in C++ whose body creates a new unit_cell object and that is exposed to Python with Boost Python inside a module x2tbx. When Python loads that module, the linker of your OS is asked to resolve all symbols in your x2tbx.so. It finds a constructor cctbx::uctbx::unit_cell::unit_cell(scitbx::af::tiny const&) but it does not know what code it shall run for it. That's because that code is in cctbx/uctbx/uctbx.cpp that has been compiled and linked into a library libcctbx. Thus you have 2 solutions here: (1) either you link that library with your x2tbx.so (modify the SConscript for that), or (2) just import a Boost Python wrapper that has been linked with that library. The line "import cctbx.uctbx" belongs to solution (2) because the uctbx Python module loads the Boost Python uctbx wrappers that is linked with libcctbx. As for solution (1), beware Dragons here! It will reliably work only if all libraries are dynamic. Currently, libcctbx is a static library on MacOS and you will run into troubles on that platform with that solution (1). As it has been discussed in the past on this very forum, there is no good reason that we keep using static libs on MacOS but until somebody spends time to fix that, you are stuck with solution (2). Best wishes, Luc

Graeme.Winter＠diamond.ac.uk

2:25 p.m.

New subject: Using cctbx::uctbx::unit_cell in c++ code / boosted to Python

Hi Luc, Funny. On my OS X system solution #1 appeared to work well! I looked at some other SConscripts on how to do this & copied. Thanks for the description though - this hybrid programming stuff takes a little getting used to... Cheerio, Graeme From: [email protected] [mailto:[email protected]] On Behalf Of Luc Bourhis Sent: 25 January 2013 14:21 To: cctbx mailing list Subject: Re: [cctbxbb] Using cctbx::uctbx::unit_cell in c++ code / boosted to Python On 25 Jan 2013, at 14:46, [email protected]mailto:[email protected] wrote: Equally I could just leave the additional clutter there but it feels like I should not need it... You have a function "generate_unit_cell" written in C++ whose body creates a new unit_cell object and that is exposed to Python with Boost Python inside a module x2tbx. When Python loads that module, the linker of your OS is asked to resolve all symbols in your x2tbx.so. It finds a constructor cctbx::uctbx::unit_cell::unit_cell(scitbx::af::tiny const&) but it does not know what code it shall run for it. That's because that code is in cctbx/uctbx/uctbx.cpp that has been compiled and linked into a library libcctbx. Thus you have 2 solutions here: (1) either you link that library with your x2tbx.so (modify the SConscript for that), or (2) just import a Boost Python wrapper that has been linked with that library. The line "import cctbx.uctbx" belongs to solution (2) because the uctbx Python module loads the Boost Python uctbx wrappers that is linked with libcctbx. As for solution (1), beware Dragons here! It will reliably work only if all libraries are dynamic. Currently, libcctbx is a static library on MacOS and you will run into troubles on that platform with that solution (1). As it has been discussed in the past on this very forum, there is no good reason that we keep using static libs on MacOS but until somebody spends time to fix that, you are stuck with solution (2). Best wishes, Luc -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

Luc Bourhis

2:30 p.m.

New subject: Using cctbx::uctbx::unit_cell in c++ code / boosted to Python

On 25 Jan 2013, at 15:25, [email protected] wrote:

...

Funny. On my OS X system solution #1 appeared to work well!

Does the following work too? import x2tbx import cctbx.uctbx Best wishes, Luc

Graeme.Winter＠diamond.ac.uk

2:32 p.m.

New subject: Using cctbx::uctbx::unit_cell in c++ code / boosted to Python

Yep. Works just fine. Where by work I assume you mean, does typing the following into a new cctbx.python session result in no apparent errors. Very pleased actually with how well everything is working. Cheerio, Graeme From: [email protected] [mailto:[email protected]] On Behalf Of Luc Bourhis Sent: 25 January 2013 14:30 To: cctbx mailing list Subject: Re: [cctbxbb] Using cctbx::uctbx::unit_cell in c++ code / boosted to Python On 25 Jan 2013, at 15:25, [email protected]mailto:[email protected] wrote: Funny. On my OS X system solution #1 appeared to work well! Does the following work too? import x2tbx import cctbx.uctbx Best wishes, Luc -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

Luc Bourhis

3:47 p.m.

New subject: Using cctbx::uctbx::unit_cell in c++ code / boosted to Python

On 25 Jan 2013, at 15:32, [email protected] wrote:

...

Yep. Works just fine.

I'm surprised! Mmhmmm… I'll investigate that. Basically, those 2 imports should end up bringing into your Python process two copies of each symbol in libcctbx and I was under the impression that it should spell trouble.

...

Very pleased actually with how well everything is working.

I have fond memories of the first time I got a Boost Python module working too! Best wishes, Luc

Phzwart

4:18 p.m.

New subject: Using cctbx::uctbx::unit_cell in c++ code / boosted to Python

...

...
Very pleased actually with how well everything is working.

I have fond memories of the first time I got a Boost Python module working too!

That never stops!

Nathaniel Echols

3:47 p.m.

New subject: Using cctbx::uctbx::unit_cell in c++ code / boosted to Python

On Fri, Jan 25, 2013 at 5:46 AM, wrote:

...

unless I have

import cctbx.uctbx # explicit import

at the top of the Python code - have I done something wrong with the SConscript? I found something similar with

import scitbx.array_family.flex # explicit import

which libtbx.clean_clutter complains about as I don't reference it (hence the additional comment)

Equally I could just leave the additional clutter there but it feels like I should not need it...

FYI, this will not be complained about: import cctbx.uctbx # import dependency since in practice there are multiple other situations (some involving just pure Python, no Boost) where we've found it necessarily to import something but not directly reference it again. -Nat

4190

Age (days ago)

4190

Last active (days ago)

List overview

Download

8 comments

4 participants

participants (4)

Graeme.Winter＠diamond.ac.uk
Luc Bourhis
Nathaniel Echols
Phzwart