What Richard says. DIALS and xia2 is Python >= 2.7 though. This comes up every ~9 months (last time as far as I can tell http://phenix-online.org/pipermail/cctbxbb/2016-September/001453.html) and should probably be mentioned in a few README.MDs or so. Maybe even put in a test to guarantee compatibility? (Should not be too difficult to build something on top of https://github.com/ghewgill/pyqver) -Markus ________________________________________ From: [email protected] [[email protected]] on behalf of [email protected] [[email protected]] Sent: Tuesday, June 06, 2017 15:14 To: [email protected] Subject: Re: [cctbxbb] Backwards compatibility; Python 2.3? Hi Nick, I think officially we now support Python >= 2.5 as far as running general cctbx code, but try to support installing cctbx (i.e. via the libtbx bootstrap mechanism) with Python >= 2.4. Cheers, Richard Dr Richard Gildea Data Analysis Scientist Tel: +441235 77 8078 Diamond Light Source Ltd. Diamond House Harwell Science & Innovation Campus Didcot Oxfordshire OX11 0DE ________________________________________ From: [email protected] [[email protected]] on behalf of Nicholas Devenish [[email protected]] Sent: 06 June 2017 15:08 To: [email protected] Subject: [cctbxbb] Backwards compatibility; Python 2.3? Hi All, I'm starting to go through and learn some of the structure of these packages, starting with libtbx. What's the supported targets supposed to be? There seems to be code specifically marked as backwards compatibility for Python 2.3 (2003), and packages explicitly included that were merged into the standard library around the same time (e.g. Optik). Is this really as far back as we need to support? Thanks, Nick Devenish _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

On Wed, Jun 7, 2017 at 12:26 PM, Marcin Wojdyr wrote:

It makes sense to repeat this kind of question from time to time. At some point Python 2.5 won't be needed anymore.

So, the whole codebase supports 2.5, or just the bootstrapping part? As far as I know, bootstrapping (and thus via 2.7) is the only documented way of installing? Are there any opportunities to switch over to bootstrapped python earlier in the process? If it is the whole package, are there any automated tests with 2.5? (If not, do we know that it's all still 2.5 compatible?) Nick

Another way of asking the question: what specific Python 2.6 and/or 2.7 features could/would we use that we currently can't due to the restriction of supporting Python 2.5? Cheers, Richard Dr Richard Gildea Data Analysis Scientist Tel: +441235 77 8078 Diamond Light Source Ltd. Diamond House Harwell Science & Innovation Campus Didcot Oxfordshire OX11 0DE ________________________________________ From: [email protected] [[email protected]] on behalf of [email protected] [[email protected]] Sent: 09 June 2017 13:43 To: [email protected] Subject: Re: [cctbxbb] Backwards compatibility; Python 2.3? Folks, Does *anyone* actually use Python < 2.7 with CCTBX for anything other than kicking off bootstrap? If not, then we could relax this back to using Python 2.7+ (which is a spritely 7 years old already) Thanks Graeme -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Nicholas Devenish Sent: 08 June 2017 09:00 To: cctbx mailing list Subject: Re: [cctbxbb] Backwards compatibility; Python 2.3? On Wed, Jun 7, 2017 at 12:26 PM, Marcin Wojdyr wrote:

It makes sense to repeat this kind of question from time to time. At some point Python 2.5 won't be needed anymore.

So, the whole codebase supports 2.5, or just the bootstrapping part? As far as I know, bootstrapping (and thus via 2.7) is the only documented way of installing? Are there any opportunities to switch over to bootstrapped python earlier in the process? If it is the whole package, are there any automated tests with 2.5? (If not, do we know that it's all still 2.5 compatible?) Nick _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Others can correct me, but I believe that we use Python < 2.7 for building Phenix on several platforms because this is what the system Python is (needed to start the bootstrap). We have to support older platforms for compatibility reasons. Unfortunately, end users aren’t as likely to keep their os as up to date as developers would like.

On Jun 9, 2017, at 5:43 AM, [email protected] wrote:

Folks,

Does *anyone* actually use Python < 2.7 with CCTBX for anything other than kicking off bootstrap?

If not, then we could relax this back to using Python 2.7+ (which is a spritely 7 years old already)

Thanks Graeme

-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Nicholas Devenish Sent: 08 June 2017 09:00 To: cctbx mailing list Subject: Re: [cctbxbb] Backwards compatibility; Python 2.3?

On Wed, Jun 7, 2017 at 12:26 PM, Marcin Wojdyr wrote:

...

It makes sense to repeat this kind of question from time to time. At some point Python 2.5 won't be needed anymore.

So, the whole codebase supports 2.5, or just the bootstrapping part? As far as I know, bootstrapping (and thus via 2.7) is the only documented way of installing? Are there any opportunities to switch over to bootstrapped python earlier in the process?

If it is the whole package, are there any automated tests with 2.5? (If not, do we know that it's all still 2.5 compatible?)

Nick _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

-- Paul Adams Division Director, Molecular Biophysics & Integrated Bioimaging, Lawrence Berkeley Lab Division Deputy for Biosciences, Advanced Light Source, Lawrence Berkeley Lab Adjunct Professor, Department of Bioengineering, U.C. Berkeley Vice President for Technology, the Joint BioEnergy Institute Laboratory Research Manager, ENIGMA Science Focus Area Building 33, Room 347 Building 80, Room 247 Building 978, Room 4126 Tel: 1-510-486-4225, Fax: 1-510-486-5909 http://cci.lbl.gov/paul Lawrence Berkeley Laboratory 1 Cyclotron Road BLDG 33R0345 Berkeley, CA 94720, USA. Executive Assistant: Louise Benvenue [ [email protected] ][ 1-510-495-2506 ] --

Hi Graeme, good points. I hope others in the group can weigh in, but I’m okay with limiting Python 2.5 to bootstrap and have everything else Python 2.7 compliant. I’m not sure what the implications, if any, for rest of the code base are, but given that we distribute with Python 2.7 then I guess things should be okay. We do make Phenix builds and run tests on several platforms with a system python 2.5. Thoughts anyone else? Cheers, Paul

On Jun 9, 2017, at 1:54 PM, [email protected] wrote:

Dear Paul,

For bootstrap I completely appreciate the need for e.g. Python 2.5 or 2.6 compatibility - however does this then necessarily apply to the whole of cctbx and therefore by extension to dials etc. when the very first thing bootstrap does is install Python 2.7?

The root of my question is trying to explain things to our new starter Nick (who initiated this thread) then coming to a halt as I realised I did not understand why some of these peculiarities were present - beyond "historical reasons"

If we were to limit bootstrap.py to Python 2.5 compatibility and subsequently assume everything else is "allowed" 2.7 language level would that cause any problems? It may allow cleanup / removal of things which were from the future when added but are now part of the standard library. Do we even test on a python 2.5 machine?

I'd also like to acknowledge that I appreciate how important these choices are to Phenix, hence trying to understand fully

Hope this makes sense, best wishes Graeme

________________________________________ From: [email protected] [[email protected]] on behalf of Paul Adams [[email protected]] Sent: 09 June 2017 16:58 To: cctbx mailing list Subject: Re: [cctbxbb] Backwards compatibility; Python 2.3?

Others can correct me, but I believe that we use Python < 2.7 for building Phenix on several platforms because this is what the system Python is (needed to start the bootstrap). We have to support older platforms for compatibility reasons. Unfortunately, end users aren’t as likely to keep their os as up to date as developers would like.

...

On Jun 9, 2017, at 5:43 AM, [email protected] wrote:

Folks,

Does *anyone* actually use Python < 2.7 with CCTBX for anything other than kicking off bootstrap?

If not, then we could relax this back to using Python 2.7+ (which is a spritely 7 years old already)

Thanks Graeme

-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Nicholas Devenish Sent: 08 June 2017 09:00 To: cctbx mailing list Subject: Re: [cctbxbb] Backwards compatibility; Python 2.3?

On Wed, Jun 7, 2017 at 12:26 PM, Marcin Wojdyr wrote:

...

It makes sense to repeat this kind of question from time to time. At some point Python 2.5 won't be needed anymore.

So, the whole codebase supports 2.5, or just the bootstrapping part? As far as I know, bootstrapping (and thus via 2.7) is the only documented way of installing? Are there any opportunities to switch over to bootstrapped python earlier in the process?

If it is the whole package, are there any automated tests with 2.5? (If not, do we know that it's all still 2.5 compatible?)

Nick _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

-- Paul Adams Division Director, Molecular Biophysics & Integrated Bioimaging, Lawrence Berkeley Lab Division Deputy for Biosciences, Advanced Light Source, Lawrence Berkeley Lab Adjunct Professor, Department of Bioengineering, U.C. Berkeley Vice President for Technology, the Joint BioEnergy Institute Laboratory Research Manager, ENIGMA Science Focus Area

Building 33, Room 347 Building 80, Room 247 Building 978, Room 4126 Tel: 1-510-486-4225, Fax: 1-510-486-5909 http://cci.lbl.gov/paul

Lawrence Berkeley Laboratory 1 Cyclotron Road BLDG 33R0345 Berkeley, CA 94720, USA.

Executive Assistant: Louise Benvenue [ [email protected] ][ 1-510-495-2506 ] --

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

-- Paul Adams Division Director, Molecular Biophysics & Integrated Bioimaging, Lawrence Berkeley Lab Division Deputy for Biosciences, Advanced Light Source, Lawrence Berkeley Lab Adjunct Professor, Department of Bioengineering, U.C. Berkeley Vice President for Technology, the Joint BioEnergy Institute Laboratory Research Manager, ENIGMA Science Focus Area Building 33, Room 347 Building 80, Room 247 Building 978, Room 4126 Tel: 1-510-486-4225, Fax: 1-510-486-5909 http://cci.lbl.gov/paul Lawrence Berkeley Laboratory 1 Cyclotron Road BLDG 33R0345 Berkeley, CA 94720, USA. Executive Assistant: Louise Benvenue [ [email protected] ][ 1-510-495-2506 ] --

Colleagues In related news, it looks like you are gearing up for a Phenix release 1.12 For this, if you are including DIALS I would recommend pulling from the 1.6 branch rather than the trunk - we are likely to put out a 1.6.1 on there sometime soon, and 1.6 is there at the moment, and this is stable and tested (i.e. better than a random pull from master) How does this sound? Thanks Graeme From: [email protected] [mailto:[email protected]] On Behalf Of Billy Poon Sent: 10 June 2017 19:34 To: cctbx mailing list Subject: Re: [cctbxbb] Backwards compatibility; Python 2.3? Hi all, I agree that we should just target Python 2.7 for running cctbx programs and support Python >= 2.5 for the installation process since we provide our own copy of Python in our installers. We can work on removing checks and cleaning up code paths for specific versions of Python 2 when we migrate to Python 3. On Fri, Jun 9, 2017 at 4:08 PM Paul Adams mailto:[email protected]> wrote: Hi Graeme, good points. I hope others in the group can weigh in, but I’m okay with limiting Python 2.5 to bootstrap and have everything else Python 2.7 compliant. I’m not sure what the implications, if any, for rest of the code base are, but given that we distribute with Python 2.7 then I guess things should be okay. We do make Phenix builds and run tests on several platforms with a system python 2.5. Thoughts anyone else? Cheers, Paul

On Jun 9, 2017, at 1:54 PM, [email protected]mailto:[email protected] wrote:

Dear Paul,

For bootstrap I completely appreciate the need for e.g. Python 2.5 or 2.6 compatibility - however does this then necessarily apply to the whole of cctbx and therefore by extension to dials etc. when the very first thing bootstrap does is install Python 2.7?

The root of my question is trying to explain things to our new starter Nick (who initiated this thread) then coming to a halt as I realised I did not understand why some of these peculiarities were present - beyond "historical reasons"

If we were to limit bootstrap.py to Python 2.5 compatibility and subsequently assume everything else is "allowed" 2.7 language level would that cause any problems? It may allow cleanup / removal of things which were from the future when added but are now part of the standard library. Do we even test on a python 2.5 machine?

I'd also like to acknowledge that I appreciate how important these choices are to Phenix, hence trying to understand fully

Hope this makes sense, best wishes Graeme

________________________________________ From: [email protected]mailto:[email protected] [[email protected]mailto:[email protected]] on behalf of Paul Adams [[email protected]mailto:[email protected]] Sent: 09 June 2017 16:58 To: cctbx mailing list Subject: Re: [cctbxbb] Backwards compatibility; Python 2.3?

Others can correct me, but I believe that we use Python < 2.7 for building Phenix on several platforms because this is what the system Python is (needed to start the bootstrap). We have to support older platforms for compatibility reasons. Unfortunately, end users aren’t as likely to keep their os as up to date as developers would like.

...

On Jun 9, 2017, at 5:43 AM, [email protected]mailto:[email protected] wrote:

Folks,

Does *anyone* actually use Python < 2.7 with CCTBX for anything other than kicking off bootstrap?

If not, then we could relax this back to using Python 2.7+ (which is a spritely 7 years old already)

Thanks Graeme

-----Original Message----- From: [email protected]mailto:[email protected] [mailto:[email protected]mailto:[email protected]] On Behalf Of Nicholas Devenish Sent: 08 June 2017 09:00 To: cctbx mailing list Subject: Re: [cctbxbb] Backwards compatibility; Python 2.3?

On Wed, Jun 7, 2017 at 12:26 PM, Marcin Wojdyr mailto:[email protected]> wrote:

...

It makes sense to repeat this kind of question from time to time. At some point Python 2.5 won't be needed anymore.

So, the whole codebase supports 2.5, or just the bootstrapping part? As far as I know, bootstrapping (and thus via 2.7) is the only documented way of installing? Are there any opportunities to switch over to bootstrapped python earlier in the process?

If it is the whole package, are there any automated tests with 2.5? (If not, do we know that it's all still 2.5 compatible?)

Nick _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

-- Paul Adams Division Director, Molecular Biophysics & Integrated Bioimaging, Lawrence Berkeley Lab Division Deputy for Biosciences, Advanced Light Source, Lawrence Berkeley Lab Adjunct Professor, Department of Bioengineering, U.C. Berkeley Vice President for Technology, the Joint BioEnergy Institute Laboratory Research Manager, ENIGMA Science Focus Area

Building 33, Room 347 Building 80, Room 247 Building 978, Room 4126 Tel: 1-510-486-4225, Fax: 1-510-486-5909 http://cci.lbl.gov/paul

Lawrence Berkeley Laboratory 1 Cyclotron Road BLDG 33R0345 Berkeley, CA 94720, USA.

Executive Assistant: Louise Benvenue [ [email protected]mailto:[email protected] ][ 1-510-495-2506 ] --

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

-- Paul Adams Division Director, Molecular Biophysics & Integrated Bioimaging, Lawrence Berkeley Lab Division Deputy for Biosciences, Advanced Light Source, Lawrence Berkeley Lab Adjunct Professor, Department of Bioengineering, U.C. Berkeley Vice President for Technology, the Joint BioEnergy Institute Laboratory Research Manager, ENIGMA Science Focus Area Building 33, Room 347 Building 80, Room 247 Building 978, Room 4126 Tel: 1-510-486-4225, Fax: 1-510-486-5909 http://cci.lbl.gov/paul Lawrence Berkeley Laboratory 1 Cyclotron Road BLDG 33R0345 Berkeley, CA 94720, USA. Executive Assistant: Louise Benvenue [ [email protected]mailto:[email protected] ][ 1-510-495-2506 ] -- _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb -- -- Billy K. Poon Research Scientist, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory 1 Cyclotron Road, M/S 33R0345 Berkeley, CA 94720 Tel: (510) 486-5709 Fax: (510) 486-5909 Web: https://phenix-online.org