simtbx

Graeme.Winter＠diamond.ac.uk

3 May 2017 3 May '17

8:41 a.m.

Hi Folks We have noticed that simtbx, part of the core cctbx now, depends on dials This means if you do not have a dials build your cctbx will be broken This is probably not the way we want things? I thought simtbx was a separate repo? Cheerio Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

Attachments:

attachment.html (text/html — 3.2 KB)

Show replies by date

James Holton

3 May 3 May

2:39 p.m.

Yesterday, Aaron and I were trying to set things up so that simtbx would be available to people who aren't James Holton. It's been checked in and working for over a month now, and only a few days ago I discovered that a fresh "bootstrap" has a broken simtbx, despite my "git status" telling me all my files are up to date. I asked Aaron and Nick for help with this. I thought what we did was make cctbx.xfel depends on simtbx, since fitting simulations to stills is the whole reason for porting my stuff into python. I set up simtbx with a constructor that takes a dxtbx detector as input, but does that make it depend on dials? I did not expect that. Sorry, didn't mean to break anything! Would much appreciate any help on this. -James On 5/3/2017 1:41 AM, [email protected] wrote:

...

Hi Folks

We have noticed that simtbx, part of the core cctbx now, depends on dials

This means if you do not have a dials build your cctbx will be broken

This is probably not the way we want things?

I thought simtbx was a separate repo?

Cheerio Graeme

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

markus.gerstel＠diamond.ac.uk

2:49 p.m.

Hi James, It looks that you now have two separate places where simtbx code is kept. One is in the simtbx repository, where you fixed the sign earlier, and the other one is in the cctbx repository, where you just now fixed it again. Simtbx repo: https://github.com/dials/simtbx Simtbx folder in cctbx repo: https://github.com/cctbx/cctbx_project/tree/master/simtbx I guess suppose there are three options here: a. move simtbx into the module directory, add simtbx to the bootstrap script so it is initialised correctly. b. leave simtbx inside the cctbx_project directory and repository, but as its own git repository (using git submodule) c. leave simtbx inside the cctbx_project directory, delete the separate git repository NB: This issue is independent of resolving the dials-dependency issue. -Markus From: [email protected] [mailto:[email protected]] On Behalf Of James Holton Sent: 03 May 2017 15:40 To: Winter, Graeme (DLSLtd,RAL,LSCI) ; [email protected] Subject: Re: [cctbxbb] simtbx Yesterday, Aaron and I were trying to set things up so that simtbx would be available to people who aren't James Holton. It's been checked in and working for over a month now, and only a few days ago I discovered that a fresh "bootstrap" has a broken simtbx, despite my "git status" telling me all my files are up to date. I asked Aaron and Nick for help with this. I thought what we did was make cctbx.xfel depends on simtbx, since fitting simulations to stills is the whole reason for porting my stuff into python. I set up simtbx with a constructor that takes a dxtbx detector as input, but does that make it depend on dials? I did not expect that. Sorry, didn't mean to break anything! Would much appreciate any help on this. -James On 5/3/2017 1:41 AM, [email protected]mailto:[email protected] wrote: Hi Folks We have noticed that simtbx, part of the core cctbx now, depends on dials This means if you do not have a dials build your cctbx will be broken This is probably not the way we want things? I thought simtbx was a separate repo? Cheerio Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

Nicholas Sauter

2:52 p.m.

Markus, We want option c. simtbx should be in the cctbx_project directory. No separate git repository. We need to double check we've gotten all the files in cctbx_project, then remove them from the separate repo. James, could you look into this? Nick Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713 On Wed, May 3, 2017 at 7:49 AM, wrote:

...

Hi James,

It looks that you now have two separate places where simtbx code is kept. One is in the simtbx repository, where you fixed the sign earlier, and the other one is in the cctbx repository, where you just now fixed it again.

Simtbx repo: https://github.com/dials/simtbx

Simtbx folder in cctbx repo: https://github.com/cctbx/ cctbx_project/tree/master/simtbx

I guess suppose there are three options here:

a. move simtbx into the module directory, add simtbx to the bootstrap script so it is initialised correctly.

b. leave simtbx inside the cctbx_project directory and repository, but as its own git repository (using git submodule)

c. leave simtbx inside the cctbx_project directory, delete the separate git repository

NB: This issue is independent of resolving the dials-dependency issue.

-Markus

*From:* [email protected] [mailto:cctbxbb-bounces@ phenix-online.org] *On Behalf Of *James Holton *Sent:* 03 May 2017 15:40 *To:* Winter, Graeme (DLSLtd,RAL,LSCI) ; [email protected] *Subject:* Re: [cctbxbb] simtbx

Yesterday, Aaron and I were trying to set things up so that simtbx would be available to people who aren't James Holton. It's been checked in and working for over a month now, and only a few days ago I discovered that a fresh "bootstrap" has a broken simtbx, despite my "git status" telling me all my files are up to date. I asked Aaron and Nick for help with this.

I thought what we did was make cctbx.xfel depends on simtbx, since fitting simulations to stills is the whole reason for porting my stuff into python.

I set up simtbx with a constructor that takes a dxtbx detector as input, but does that make it depend on dials? I did not expect that.

Sorry, didn't mean to break anything! Would much appreciate any help on this.

-James

On 5/3/2017 1:41 AM, [email protected] wrote:

Hi Folks

We have noticed that simtbx, part of the core cctbx now, depends on dials

This means if you do not have a dials build your cctbx will be broken

This is probably not the way we want things?

I thought simtbx was a separate repo?

Cheerio Graeme

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

markus.gerstel＠diamond.ac.uk

2:53 p.m.

Hi Nick, Sorry, outsider perspective here. I don’t know what simtbx does or is supposed to do. What is the benefit of having code in a repository that, according to your earlier mail, is explicitly not used in the project? Or more bluntly: We (and ccp4) distribute dials and, as part of it, the cctbx_project repository. Why should we ship dead code? -Markus From: [email protected] [mailto:[email protected]] On Behalf Of Nicholas Sauter Sent: 03 May 2017 15:52 To: cctbx mailing list Subject: Re: [cctbxbb] simtbx Markus, We want option c. simtbx should be in the cctbx_project directory. No separate git repository. We need to double check we've gotten all the files in cctbx_project, then remove them from the separate repo. James, could you look into this? Nick Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713 On Wed, May 3, 2017 at 7:49 AM, mailto:[email protected]> wrote: Hi James, It looks that you now have two separate places where simtbx code is kept. One is in the simtbx repository, where you fixed the sign earlier, and the other one is in the cctbx repository, where you just now fixed it again. Simtbx repo: https://github.com/dials/simtbx Simtbx folder in cctbx repo: https://github.com/cctbx/cctbx_project/tree/master/simtbx I guess suppose there are three options here: a. move simtbx into the module directory, add simtbx to the bootstrap script so it is initialised correctly. b. leave simtbx inside the cctbx_project directory and repository, but as its own git repository (using git submodule) c. leave simtbx inside the cctbx_project directory, delete the separate git repository NB: This issue is independent of resolving the dials-dependency issue. -Markus From: [email protected]mailto:[email protected] [mailto:[email protected]mailto:[email protected]] On Behalf Of James Holton Sent: 03 May 2017 15:40 To: Winter, Graeme (DLSLtd,RAL,LSCI) mailto:[email protected]>; [email protected]mailto:[email protected] Subject: Re: [cctbxbb] simtbx Yesterday, Aaron and I were trying to set things up so that simtbx would be available to people who aren't James Holton. It's been checked in and working for over a month now, and only a few days ago I discovered that a fresh "bootstrap" has a broken simtbx, despite my "git status" telling me all my files are up to date. I asked Aaron and Nick for help with this. I thought what we did was make cctbx.xfel depends on simtbx, since fitting simulations to stills is the whole reason for porting my stuff into python. I set up simtbx with a constructor that takes a dxtbx detector as input, but does that make it depend on dials? I did not expect that. Sorry, didn't mean to break anything! Would much appreciate any help on this. -James On 5/3/2017 1:41 AM, [email protected]mailto:[email protected] wrote: Hi Folks We have noticed that simtbx, part of the core cctbx now, depends on dials This means if you do not have a dials build your cctbx will be broken This is probably not the way we want things? I thought simtbx was a separate repo? Cheerio Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Pavel Afonine

2:57 p.m.

Same questions here as Markus asked.. What is simtbx? First time hear about it. How cctbx can benefit from it? Pavel On 5/3/17 07:53, [email protected] wrote:

...

Hi Nick,

Sorry, outsider perspective here. I don’t know what simtbx does or is supposed to do.

What is the benefit of having code in a repository that, according to your earlier mail, is explicitly not used in the project?

Or more bluntly: We (and ccp4) distribute dials and, as part of it, the cctbx_project repository.

Why should we ship dead code?

-Markus

*From:*[email protected] [mailto:[email protected]] *On Behalf Of *Nicholas Sauter *Sent:* 03 May 2017 15:52 *To:* cctbx mailing list *Subject:* Re: [cctbxbb] simtbx

Markus,

We want option c. simtbx should be in the cctbx_project directory. No separate git repository. We need to double check we've gotten all the files in cctbx_project, then remove them from the separate repo. James, could you look into this?

Nick

Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division

Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713

On Wed, May 3, 2017 at 7:49 AM, mailto:[email protected]> wrote:

Hi James,

It looks that you now have two separate places where simtbx code is kept. One is in the simtbx repository, where you fixed the sign earlier, and the other one is in the cctbx repository, where you just now fixed it again.

Simtbx repo: https://github.com/dials/simtbx

Simtbx folder in cctbx repo: https://github.com/cctbx/cctbx_project/tree/master/simtbx

I guess suppose there are three options here:

a.move simtbx into the module directory, add simtbx to the bootstrap script so it is initialised correctly.

b.leave simtbx inside the cctbx_project directory and repository, but as its own git repository (using git submodule)

c.leave simtbx inside the cctbx_project directory, delete the separate git repository

NB: This issue is independent of resolving the dials-dependency issue.

-Markus

*From:*[email protected] mailto:[email protected] [mailto:[email protected] mailto:[email protected]] *On Behalf Of *James Holton *Sent:* 03 May 2017 15:40 *To:* Winter, Graeme (DLSLtd,RAL,LSCI) mailto:[email protected]>; [email protected] mailto:[email protected] *Subject:* Re: [cctbxbb] simtbx

Yesterday, Aaron and I were trying to set things up so that simtbx would be available to people who aren't James Holton. It's been checked in and working for over a month now, and only a few days ago I discovered that a fresh "bootstrap" has a broken simtbx, despite my "git status" telling me all my files are up to date. I asked Aaron and Nick for help with this.

I thought what we did was make cctbx.xfel depends on simtbx, since fitting simulations to stills is the whole reason for porting my stuff into python.

I set up simtbx with a constructor that takes a dxtbx detector as input, but does that make it depend on dials? I did not expect that.

Sorry, didn't mean to break anything! Would much appreciate any help on this.

-James

On 5/3/2017 1:41 AM, [email protected] mailto:[email protected] wrote:

Hi Folks

We have noticed that simtbx, part of the core cctbx now, depends on dials

This means if you do not have a dials build your cctbx will be broken

This is probably not the way we want things?

I thought simtbx was a separate repo?

Cheerio Graeme

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Nicholas Sauter

2:59 p.m.

I'm stepping in the shower now. Pavel, I can answer about simtbx later. Never knew that we had to explain ourselves to everyone? What is mmtbx? What is scitbx? Nick Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713 On Wed, May 3, 2017 at 7:57 AM, Pavel Afonine wrote:

...

Same questions here as Markus asked.. What is simtbx? First time hear about it. How cctbx can benefit from it? Pavel

On 5/3/17 07:53, [email protected] wrote:

Hi Nick,

Sorry, outsider perspective here. I don’t know what simtbx does or is supposed to do.

What is the benefit of having code in a repository that, according to your earlier mail, is explicitly not used in the project?

Or more bluntly: We (and ccp4) distribute dials and, as part of it, the cctbx_project repository.

Why should we ship dead code?

-Markus

*From:* [email protected] [mailto:cctbxbb-bounces@ phenix-online.org ] *On Behalf Of *Nicholas Sauter *Sent:* 03 May 2017 15:52 *To:* cctbx mailing list *Subject:* Re: [cctbxbb] simtbx

Markus,

We want option c. simtbx should be in the cctbx_project directory. No separate git repository. We need to double check we've gotten all the files in cctbx_project, then remove them from the separate repo. James, could you look into this?

Nick

Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division

Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713

On Wed, May 3, 2017 at 7:49 AM, wrote:

Hi James,

It looks that you now have two separate places where simtbx code is kept. One is in the simtbx repository, where you fixed the sign earlier, and the other one is in the cctbx repository, where you just now fixed it again.

Simtbx repo: https://github.com/dials/simtbx

Simtbx folder in cctbx repo: https://github.com/cctbx/ cctbx_project/tree/master/simtbx

I guess suppose there are three options here:

a. move simtbx into the module directory, add simtbx to the bootstrap script so it is initialised correctly.

b. leave simtbx inside the cctbx_project directory and repository, but as its own git repository (using git submodule)

c. leave simtbx inside the cctbx_project directory, delete the separate git repository

NB: This issue is independent of resolving the dials-dependency issue.

-Markus

*From:* [email protected] [mailto:cctbxbb-bounces@ phenix-online.org] *On Behalf Of *James Holton *Sent:* 03 May 2017 15:40 *To:* Winter, Graeme (DLSLtd,RAL,LSCI) ; [email protected] *Subject:* Re: [cctbxbb] simtbx

Yesterday, Aaron and I were trying to set things up so that simtbx would be available to people who aren't James Holton. It's been checked in and working for over a month now, and only a few days ago I discovered that a fresh "bootstrap" has a broken simtbx, despite my "git status" telling me all my files are up to date. I asked Aaron and Nick for help with this.

I thought what we did was make cctbx.xfel depends on simtbx, since fitting simulations to stills is the whole reason for porting my stuff into python.

I set up simtbx with a constructor that takes a dxtbx detector as input, but does that make it depend on dials? I did not expect that.

Sorry, didn't mean to break anything! Would much appreciate any help on this.

-James

On 5/3/2017 1:41 AM, [email protected] wrote:

Hi Folks

We have noticed that simtbx, part of the core cctbx now, depends on dials

This means if you do not have a dials build your cctbx will be broken

This is probably not the way we want things?

I thought simtbx was a separate repo?

Cheerio Graeme

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing [email protected]http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Graeme.Winter＠diamond.ac.uk

3:03 p.m.

Nick Shower notwithstanding, I think what happened over here is this took out our build, the board went red and surprise was expressed and questions were raised. If I broke Phenix by bundling in xia2 into something, or some other project, I would expect the same level of questioning from LBL coming my way… But sounds like you can resolve this with people in the same building then let us know over here what the conclusion was. Best wishes Graeme From: [email protected] [mailto:[email protected]] On Behalf Of Nicholas Sauter Sent: 03 May 2017 15:59 To: cctbx mailing list Subject: Re: [cctbxbb] simtbx I'm stepping in the shower now. Pavel, I can answer about simtbx later. Never knew that we had to explain ourselves to everyone? What is mmtbx? What is scitbx? Nick Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713 On Wed, May 3, 2017 at 7:57 AM, Pavel Afonine mailto:[email protected]> wrote: Same questions here as Markus asked.. What is simtbx? First time hear about it. How cctbx can benefit from it? Pavel On 5/3/17 07:53, [email protected]mailto:[email protected] wrote: Hi Nick, Sorry, outsider perspective here. I don’t know what simtbx does or is supposed to do. What is the benefit of having code in a repository that, according to your earlier mail, is explicitly not used in the project? Or more bluntly: We (and ccp4) distribute dials and, as part of it, the cctbx_project repository. Why should we ship dead code? -Markus From: [email protected]mailto:[email protected] [mailto:[email protected]] On Behalf Of Nicholas Sauter Sent: 03 May 2017 15:52 To: cctbx mailing list mailto:[email protected] Subject: Re: [cctbxbb] simtbx Markus, We want option c. simtbx should be in the cctbx_project directory. No separate git repository. We need to double check we've gotten all the files in cctbx_project, then remove them from the separate repo. James, could you look into this? Nick Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713tel:(510)%20486-5713 On Wed, May 3, 2017 at 7:49 AM, mailto:[email protected]> wrote: Hi James, It looks that you now have two separate places where simtbx code is kept. One is in the simtbx repository, where you fixed the sign earlier, and the other one is in the cctbx repository, where you just now fixed it again. Simtbx repo: https://github.com/dials/simtbx Simtbx folder in cctbx repo: https://github.com/cctbx/cctbx_project/tree/master/simtbx I guess suppose there are three options here: a. move simtbx into the module directory, add simtbx to the bootstrap script so it is initialised correctly. b. leave simtbx inside the cctbx_project directory and repository, but as its own git repository (using git submodule) c. leave simtbx inside the cctbx_project directory, delete the separate git repository NB: This issue is independent of resolving the dials-dependency issue. -Markus From: [email protected]mailto:[email protected] [mailto:[email protected]mailto:[email protected]] On Behalf Of James Holton Sent: 03 May 2017 15:40 To: Winter, Graeme (DLSLtd,RAL,LSCI) mailto:[email protected]>; [email protected]mailto:[email protected] Subject: Re: [cctbxbb] simtbx Yesterday, Aaron and I were trying to set things up so that simtbx would be available to people who aren't James Holton. It's been checked in and working for over a month now, and only a few days ago I discovered that a fresh "bootstrap" has a broken simtbx, despite my "git status" telling me all my files are up to date. I asked Aaron and Nick for help with this. I thought what we did was make cctbx.xfel depends on simtbx, since fitting simulations to stills is the whole reason for porting my stuff into python. I set up simtbx with a constructor that takes a dxtbx detector as input, but does that make it depend on dials? I did not expect that. Sorry, didn't mean to break anything! Would much appreciate any help on this. -James On 5/3/2017 1:41 AM, [email protected]mailto:[email protected] wrote: Hi Folks We have noticed that simtbx, part of the core cctbx now, depends on dials This means if you do not have a dials build your cctbx will be broken This is probably not the way we want things? I thought simtbx was a separate repo? Cheerio Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Pavel Afonine

3:10 p.m.

Hi Nick,

...

Pavel, I can answer about simtbx later.

Thanks! I'm away till May 11th. So email is best way.

...

Never knew that we had to explain ourselves to everyone?

Perhaps it's a fine idea to let others know if something is being added. Of course, as you point out, we don't have to do it.. but then people may wonder and ask questions. Sounds reasonable to me.

...

The fact is cctbx is a multipurpose repo contributed by several groups for different purposes.

I thought cctbx is not a collection of random stuff but rather something more integral..

...

What is mmtbx? What is scitbx?

I'm sure you (and others) know. Yet, what is simtbx? -;) All the best, Pavel

James Holton

3:28 p.m.

"simtbx" is the name we are giving to my diffraction image simulation codes. You may or may not of heard of their standalone versions as MLFSOM, fastBragg (now nanoBragg), nonBragg and nearBragg. Nick and I thought the capability of creating absolutely realistic diffraction data from a completely independent code base would be an excellent way to generate test data for cctbx, dials, phenix, and even third-party packages. There is also the idea of fitting a simulation directly to diffraction images, which I have had some success doing independently, but we hope this will work better under cctbx. Until recently, all this was standalone c code, with home pages here: http://bl831.als.lbl.gov/~jamesh/mlfsom/ http://bl831.als.lbl.gov/~jamesh/nanoBragg/ http://bl831.als.lbl.gov/~jamesh/nearBragg/ http://bl831.als.lbl.gov/~jamesh/nonBragg/ I have spent the bulk of my time for three months trying to wire these codes into the boost/Python framework of cctbx. I put docstrings into every single one of my objects, but since I am new to Python I don't know where people look for documentation. Again, sorry this broke everything. It's been in there since February and I was surprised to find that a new developer here couldn't access it. This is why I've been asking so many questions about tests I should perform before checking things in, but apparently it was all for naught. -James On 5/3/2017 7:57 AM, Pavel Afonine wrote:

...

Same questions here as Markus asked.. What is simtbx? First time hear about it. How cctbx can benefit from it? Pavel

On 5/3/17 07:53, [email protected] wrote:

...
Hi Nick,

Sorry, outsider perspective here. I don’t know what simtbx does or is supposed to do.

What is the benefit of having code in a repository that, according to your earlier mail, is explicitly not used in the project?

Or more bluntly: We (and ccp4) distribute dials and, as part of it, the cctbx_project repository.

Why should we ship dead code?

-Markus

*From:*[email protected] [mailto:[email protected]] *On Behalf Of *Nicholas Sauter *Sent:* 03 May 2017 15:52 *To:* cctbx mailing list *Subject:* Re: [cctbxbb] simtbx

Markus,

We want option c. simtbx should be in the cctbx_project directory. No separate git repository. We need to double check we've gotten all the files in cctbx_project, then remove them from the separate repo. James, could you look into this?

Nick

Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division

Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713

On Wed, May 3, 2017 at 7:49 AM, mailto:[email protected]> wrote:

Hi James,

It looks that you now have two separate places where simtbx code is kept. One is in the simtbx repository, where you fixed the sign earlier, and the other one is in the cctbx repository, where you just now fixed it again.

Simtbx repo: https://github.com/dials/simtbx

Simtbx folder in cctbx repo: https://github.com/cctbx/cctbx_project/tree/master/simtbx

I guess suppose there are three options here:

a.move simtbx into the module directory, add simtbx to the bootstrap script so it is initialised correctly.

b.leave simtbx inside the cctbx_project directory and repository, but as its own git repository (using git submodule)

c.leave simtbx inside the cctbx_project directory, delete the separate git repository

NB: This issue is independent of resolving the dials-dependency issue.

-Markus

*From:*[email protected] mailto:[email protected] [mailto:[email protected] mailto:[email protected]] *On Behalf Of *James Holton *Sent:* 03 May 2017 15:40 *To:* Winter, Graeme (DLSLtd,RAL,LSCI) mailto:[email protected]>; [email protected] mailto:[email protected] *Subject:* Re: [cctbxbb] simtbx

Yesterday, Aaron and I were trying to set things up so that simtbx would be available to people who aren't James Holton. It's been checked in and working for over a month now, and only a few days ago I discovered that a fresh "bootstrap" has a broken simtbx, despite my "git status" telling me all my files are up to date. I asked Aaron and Nick for help with this.

I thought what we did was make cctbx.xfel depends on simtbx, since fitting simulations to stills is the whole reason for porting my stuff into python.

I set up simtbx with a constructor that takes a dxtbx detector as input, but does that make it depend on dials? I did not expect that.

Sorry, didn't mean to break anything! Would much appreciate any help on this.

-James

On 5/3/2017 1:41 AM, [email protected] mailto:[email protected] wrote:

Hi Folks

We have noticed that simtbx, part of the core cctbx now, depends on dials

This means if you do not have a dials build your cctbx will be broken

This is probably not the way we want things?

I thought simtbx was a separate repo?

Cheerio Graeme

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Aaron Brewster

3:58 p.m.

Hi, sorry I thought things were good based on my local tests. However, it looks like we need to fix some compile errors on different platforms. I've reverted the change for now, but I'll be putting it back soon (probably later today) after we've done some more testing. Regarding Graeme's original point: "simtbx, part of the core cctbx now, depends on dials. This means if you do not have a dials build your cctbx will be broken". This is untrue. A core cctbx build means you run bootstrap.py --builder=cctbx. You'll get 'cctbx', 'cbflib', 'scitbx', 'libtbx', 'iotbx', 'mmtbx', 'smtbx', 'dxtbx', 'gltbx', 'wxtbx'. This does means several packages in cctbx_project would not be configured. If instead you ran python /path/to/libtbx/configure.py cctbx in an empty folder, you would get a similar list of packages. About dependencies, there is a circular dependency between xfel and dials. xfel depends on dials and dials depends on xfel. It's been there for a while. The root cause is dials.stills_process depends on xfel code, specifically xfel.mono_simulation. It would be nice to break this cycle, but it would involve shuffling a lot of code around. Instead, xfel is an optional dependency of dials and dials is a required dependency of xfel. simtbx will be an optional dependency of xfel. If you'd like to see the dependency chains, use libtbx.show_module_dependencies xfel libtbx.show_module_dependencies dials libtbx.show_module_dependencies cctbx -Aaron On Wed, May 3, 2017 at 8:28 AM, James Holton wrote:

...

"simtbx" is the name we are giving to my diffraction image simulation codes. You may or may not of heard of their standalone versions as MLFSOM, fastBragg (now nanoBragg), nonBragg and nearBragg. Nick and I thought the capability of creating absolutely realistic diffraction data from a completely independent code base would be an excellent way to generate test data for cctbx, dials, phenix, and even third-party packages. There is also the idea of fitting a simulation directly to diffraction images, which I have had some success doing independently, but we hope this will work better under cctbx. Until recently, all this was standalone c code, with home pages here:

http://bl831.als.lbl.gov/~jamesh/mlfsom/ http://bl831.als.lbl.gov/~jamesh/nanoBragg/ http://bl831.als.lbl.gov/~jamesh/nearBragg/ http://bl831.als.lbl.gov/~jamesh/nonBragg/

I have spent the bulk of my time for three months trying to wire these codes into the boost/Python framework of cctbx. I put docstrings into every single one of my objects, but since I am new to Python I don't know where people look for documentation.

Again, sorry this broke everything. It's been in there since February and I was surprised to find that a new developer here couldn't access it. This is why I've been asking so many questions about tests I should perform before checking things in, but apparently it was all for naught.

-James

On 5/3/2017 7:57 AM, Pavel Afonine wrote:

Same questions here as Markus asked.. What is simtbx? First time hear about it. How cctbx can benefit from it? Pavel

On 5/3/17 07:53, [email protected] wrote:

Hi Nick,

Sorry, outsider perspective here. I don’t know what simtbx does or is supposed to do.

What is the benefit of having code in a repository that, according to your earlier mail, is explicitly not used in the project?

Or more bluntly: We (and ccp4) distribute dials and, as part of it, the cctbx_project repository.

Why should we ship dead code?

-Markus

*From:* [email protected] [mailto:cctbxbb-bounces@ phenix-online.org ] *On Behalf Of *Nicholas Sauter *Sent:* 03 May 2017 15:52 *To:* cctbx mailing list *Subject:* Re: [cctbxbb] simtbx

Markus,

We want option c. simtbx should be in the cctbx_project directory. No separate git repository. We need to double check we've gotten all the files in cctbx_project, then remove them from the separate repo. James, could you look into this?

Nick

Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division

Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713

On Wed, May 3, 2017 at 7:49 AM, wrote:

Hi James,

It looks that you now have two separate places where simtbx code is kept. One is in the simtbx repository, where you fixed the sign earlier, and the other one is in the cctbx repository, where you just now fixed it again.

Simtbx repo: https://github.com/dials/simtbx

Simtbx folder in cctbx repo: https://github.com/cctbx/ cctbx_project/tree/master/simtbx

I guess suppose there are three options here:

a. move simtbx into the module directory, add simtbx to the bootstrap script so it is initialised correctly.

b. leave simtbx inside the cctbx_project directory and repository, but as its own git repository (using git submodule)

c. leave simtbx inside the cctbx_project directory, delete the separate git repository

NB: This issue is independent of resolving the dials-dependency issue.

-Markus

*From:* [email protected] [mailto:cctbxbb-bounces@ phenix-online.org] *On Behalf Of *James Holton *Sent:* 03 May 2017 15:40 *To:* Winter, Graeme (DLSLtd,RAL,LSCI) ; [email protected] *Subject:* Re: [cctbxbb] simtbx

Yesterday, Aaron and I were trying to set things up so that simtbx would be available to people who aren't James Holton. It's been checked in and working for over a month now, and only a few days ago I discovered that a fresh "bootstrap" has a broken simtbx, despite my "git status" telling me all my files are up to date. I asked Aaron and Nick for help with this.

I thought what we did was make cctbx.xfel depends on simtbx, since fitting simulations to stills is the whole reason for porting my stuff into python.

I set up simtbx with a constructor that takes a dxtbx detector as input, but does that make it depend on dials? I did not expect that.

Sorry, didn't mean to break anything! Would much appreciate any help on this.

-James

On 5/3/2017 1:41 AM, [email protected] wrote:

Hi Folks

We have noticed that simtbx, part of the core cctbx now, depends on dials

This means if you do not have a dials build your cctbx will be broken

This is probably not the way we want things?

I thought simtbx was a separate repo?

Cheerio Graeme

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing [email protected]http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing [email protected]http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Graeme.Winter＠diamond.ac.uk

4:03 p.m.

Aaron Last command exceedingly useful thank you Will explore Think the dependency was indirect will verify Cheers Graeme On 3 May 2017, at 16:59, Aaron Brewster mailto:[email protected]> wrote: Hi, sorry I thought things were good based on my local tests. However, it looks like we need to fix some compile errors on different platforms. I've reverted the change for now, but I'll be putting it back soon (probably later today) after we've done some more testing. Regarding Graeme's original point: "simtbx, part of the core cctbx now, depends on dials. This means if you do not have a dials build your cctbx will be broken". This is untrue. A core cctbx build means you run bootstrap.py --builder=cctbx. You'll get 'cctbx', 'cbflib', 'scitbx', 'libtbx', 'iotbx', 'mmtbx', 'smtbx', 'dxtbx', 'gltbx', 'wxtbx'. This does means several packages in cctbx_project would not be configured. If instead you ran python /path/to/libtbx/configure.py cctbx in an empty folder, you would get a similar list of packages. About dependencies, there is a circular dependency between xfel and dials. xfel depends on dials and dials depends on xfel. It's been there for a while. The root cause is dials.stills_process depends on xfel code, specifically xfel.mono_simulation. It would be nice to break this cycle, but it would involve shuffling a lot of code around. Instead, xfel is an optional dependency of dials and dials is a required dependency of xfel. simtbx will be an optional dependency of xfel. If you'd like to see the dependency chains, use libtbx.show_module_dependencies xfel libtbx.show_module_dependencies dials libtbx.show_module_dependencies cctbx -Aaron On Wed, May 3, 2017 at 8:28 AM, James Holton mailto:[email protected]> wrote: "simtbx" is the name we are giving to my diffraction image simulation codes. You may or may not of heard of their standalone versions as MLFSOM, fastBragg (now nanoBragg), nonBragg and nearBragg. Nick and I thought the capability of creating absolutely realistic diffraction data from a completely independent code base would be an excellent way to generate test data for cctbx, dials, phenix, and even third-party packages. There is also the idea of fitting a simulation directly to diffraction images, which I have had some success doing independently, but we hope this will work better under cctbx. Until recently, all this was standalone c code, with home pages here: http://bl831.als.lbl.gov/~jamesh/mlfsom/ http://bl831.als.lbl.gov/~jamesh/nanoBragg/ http://bl831.als.lbl.gov/~jamesh/nearBragg/ http://bl831.als.lbl.gov/~jamesh/nonBragg/ I have spent the bulk of my time for three months trying to wire these codes into the boost/Python framework of cctbx. I put docstrings into every single one of my objects, but since I am new to Python I don't know where people look for documentation. Again, sorry this broke everything. It's been in there since February and I was surprised to find that a new developer here couldn't access it. This is why I've been asking so many questions about tests I should perform before checking things in, but apparently it was all for naught. -James On 5/3/2017 7:57 AM, Pavel Afonine wrote: Same questions here as Markus asked.. What is simtbx? First time hear about it. How cctbx can benefit from it? Pavel On 5/3/17 07:53, [email protected]mailto:[email protected] wrote: Hi Nick, Sorry, outsider perspective here. I don’t know what simtbx does or is supposed to do. What is the benefit of having code in a repository that, according to your earlier mail, is explicitly not used in the project? Or more bluntly: We (and ccp4) distribute dials and, as part of it, the cctbx_project repository. Why should we ship dead code? -Markus From: [email protected]mailto:[email protected] [mailto:[email protected]] On Behalf Of Nicholas Sauter Sent: 03 May 2017 15:52 To: cctbx mailing list mailto:[email protected] Subject: Re: [cctbxbb] simtbx Markus, We want option c. simtbx should be in the cctbx_project directory. No separate git repository. We need to double check we've gotten all the files in cctbx_project, then remove them from the separate repo. James, could you look into this? Nick Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713tel:(510)%20486-5713 On Wed, May 3, 2017 at 7:49 AM, mailto:[email protected]> wrote: Hi James, It looks that you now have two separate places where simtbx code is kept. One is in the simtbx repository, where you fixed the sign earlier, and the other one is in the cctbx repository, where you just now fixed it again. Simtbx repo: https://github.com/dials/simtbx Simtbx folder in cctbx repo: https://github.com/cctbx/cctbx_project/tree/master/simtbx I guess suppose there are three options here: a. move simtbx into the module directory, add simtbx to the bootstrap script so it is initialised correctly. b. leave simtbx inside the cctbx_project directory and repository, but as its own git repository (using git submodule) c. leave simtbx inside the cctbx_project directory, delete the separate git repository NB: This issue is independent of resolving the dials-dependency issue. -Markus From: [email protected]mailto:[email protected] [mailto:[email protected]mailto:[email protected]] On Behalf Of James Holton Sent: 03 May 2017 15:40 To: Winter, Graeme (DLSLtd,RAL,LSCI) mailto:[email protected]>; [email protected]mailto:[email protected] Subject: Re: [cctbxbb] simtbx Yesterday, Aaron and I were trying to set things up so that simtbx would be available to people who aren't James Holton. It's been checked in and working for over a month now, and only a few days ago I discovered that a fresh "bootstrap" has a broken simtbx, despite my "git status" telling me all my files are up to date. I asked Aaron and Nick for help with this. I thought what we did was make cctbx.xfel depends on simtbx, since fitting simulations to stills is the whole reason for porting my stuff into python. I set up simtbx with a constructor that takes a dxtbx detector as input, but does that make it depend on dials? I did not expect that. Sorry, didn't mean to break anything! Would much appreciate any help on this. -James On 5/3/2017 1:41 AM, [email protected]mailto:[email protected] wrote: Hi Folks We have noticed that simtbx, part of the core cctbx now, depends on dials This means if you do not have a dials build your cctbx will be broken This is probably not the way we want things? I thought simtbx was a separate repo? Cheerio Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

R.D. Oeffner

5 Jun 5 Jun

11:14 a.m.

Dear James, I'm curious about simtbx. Does it allow me to supply a set of atomic coordinates in an ASU and then spit out corresponding diffraction images? I suppose it can't generate twinned data sets. Many thanks, Rob On 03/05/2017 16:28, James Holton wrote:

...

"simtbx" is the name we are giving to my diffraction image simulation codes. You may or may not of heard of their standalone versions as MLFSOM, fastBragg (now nanoBragg), nonBragg and nearBragg. Nick and I thought the capability of creating absolutely realistic diffraction data from a completely independent code base would be an excellent way to generate test data for cctbx, dials, phenix, and even third-party packages. There is also the idea of fitting a simulation directly to diffraction images, which I have had some success doing independently, but we hope this will work better under cctbx. Until recently, all this was standalone c code, with home pages here:

http://bl831.als.lbl.gov/~jamesh/mlfsom/ [1] http://bl831.als.lbl.gov/~jamesh/nanoBragg/ [2] http://bl831.als.lbl.gov/~jamesh/nearBragg/ [3] http://bl831.als.lbl.gov/~jamesh/nonBragg/ [4]

I have spent the bulk of my time for three months trying to wire these codes into the boost/Python framework of cctbx. I put docstrings into every single one of my objects, but since I am new to Python I don't know where people look for documentation.

Again, sorry this broke everything. It's been in there since February and I was surprised to find that a new developer here couldn't access it. This is why I've been asking so many questions about tests I should perform before checking things in, but apparently it was all for naught.

-James

On 5/3/2017 7:57 AM, Pavel Afonine wrote: Same questions here as Markus asked.. What is simtbx? First time hear about it. How cctbx can benefit from it? Pavel

On 5/3/17 07:53, [email protected] wrote:

Hi Nick,

Sorry, outsider perspective here. I don't know what simtbx does or is supposed to do.

What is the benefit of having code in a repository that, according to your earlier mail, is explicitly not used in the project?

Or more bluntly: We (and ccp4) distribute dials and, as part of it, the cctbx_project repository.

Why should we ship dead code?

-Markus

FROM: [email protected] [mailto:[email protected]] ON BEHALF OF Nicholas Sauter SENT: 03 May 2017 15:52 TO: cctbx mailing list SUBJECT: Re: [cctbxbb] simtbx

Markus,

We want option c. simtbx should be in the cctbx_project directory. No separate git repository. We need to double check we've gotten all the files in cctbx_project, then remove them from the separate repo. James, could you look into this?

Nick

Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division

Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713

On Wed, May 3, 2017 at 7:49 AM, wrote:

Hi James,

It looks that you now have two separate places where simtbx code is kept. One is in the simtbx repository, where you fixed the sign earlier, and the other one is in the cctbx repository, where you just now fixed it again.

Simtbx repo: https://github.com/dials/simtbx [5]

Simtbx folder in cctbx repo: https://github.com/cctbx/cctbx_project/tree/master/simtbx [6]

I guess suppose there are three options here:

a. move simtbx into the module directory, add simtbx to the bootstrap script so it is initialised correctly.

b. leave simtbx inside the cctbx_project directory and repository, but as its own git repository (using git submodule)

c. leave simtbx inside the cctbx_project directory, delete the separate git repository

NB: This issue is independent of resolving the dials-dependency issue.

-Markus

FROM: [email protected] [mailto:[email protected]] ON BEHALF OF James Holton SENT: 03 May 2017 15:40 TO: Winter, Graeme (DLSLtd,RAL,LSCI) ; [email protected] SUBJECT: Re: [cctbxbb] simtbx

Yesterday, Aaron and I were trying to set things up so that simtbx would be available to people who aren't James Holton. It's been checked in and working for over a month now, and only a few days ago I discovered that a fresh "bootstrap" has a broken simtbx, despite my "git status" telling me all my files are up to date. I asked Aaron and Nick for help with this.

I thought what we did was make cctbx.xfel depends on simtbx, since fitting simulations to stills is the whole reason for porting my stuff into python.

I set up simtbx with a constructor that takes a dxtbx detector as input, but does that make it depend on dials? I did not expect that.

Sorry, didn't mean to break anything! Would much appreciate any help on this.

-James

On 5/3/2017 1:41 AM, [email protected] wrote:

Hi Folks

We have noticed that simtbx, part of the core cctbx now, depends on dials

This means if you do not have a dials build your cctbx will be broken

This is probably not the way we want things?

I thought simtbx was a separate repo?

Cheerio Graeme

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb [7]

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb [7] _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb [7] _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb [7] Links: ------ [1] http://bl831.als.lbl.gov/~jamesh/mlfsom/ [2] http://bl831.als.lbl.gov/~jamesh/nanoBragg/ [3] http://bl831.als.lbl.gov/~jamesh/nearBragg/ [4] http://bl831.als.lbl.gov/~jamesh/nonBragg/ [5] https://github.com/dials/simtbx [6] https://github.com/cctbx/cctbx_project/tree/master/simtbx [7] http://phenix-online.org/mailman/listinfo/cctbxbb

James Holton

10:38 p.m.

Yes, that's what it does. If you look at tst_nanoBragg_minimal.py you will see some code for running it. Currently, I have the call to "sfall" commented out for testing purposes, but you can uncomment it to make structure factors from a PDB as shown. Twinning is fairly straightforward. You can cheat by "pre-twinning" the data (square the Fs, re-index, weighted sum, then square root). However, what I generally do for twinning is generate two diffraction images with different crystal orientations and then add them together. My plan is to set this up so you do the image manipulations as flex arrays. This would also allow background to be done separately. Then you would use "noisify" to add the final noise and generate the image files using the new dxtbx output functions. Working on getting those "pythonized" today. In the meantime, it should be a simple matter to generate noiseless test images. Or, if you're in a hurry, I can probably tell you how to do something using the standalone code. Like this twinning example: http://bl831.als.lbl.gov/~jamesh/challenge/twin/ Does that help? -James On 6/5/2017 4:14 AM, R.D. Oeffner wrote:

...

Dear James,

I'm curious about simtbx. Does it allow me to supply a set of atomic coordinates in an ASU and then spit out corresponding diffraction images? I suppose it can't generate twinned data sets.

Many thanks,

Rob

On 03/05/2017 16:28, James Holton wrote:

...
"simtbx" is the name we are giving to my diffraction image simulation codes. You may or may not of heard of their standalone versions as MLFSOM, fastBragg (now nanoBragg), nonBragg and nearBragg. Nick and I thought the capability of creating absolutely realistic diffraction data from a completely independent code base would be an excellent way to generate test data for cctbx, dials, phenix, and even third-party packages. There is also the idea of fitting a simulation directly to diffraction images, which I have had some success doing independently, but we hope this will work better under cctbx. Until recently, all this was standalone c code, with home pages here:

http://bl831.als.lbl.gov/~jamesh/mlfsom/ http://bl831.als.lbl.gov/~jamesh/nanoBragg/ http://bl831.als.lbl.gov/~jamesh/nearBragg/ http://bl831.als.lbl.gov/~jamesh/nonBragg/

I have spent the bulk of my time for three months trying to wire these codes into the boost/Python framework of cctbx. I put docstrings into every single one of my objects, but since I am new to Python I don't know where people look for documentation.

Again, sorry this broke everything. It's been in there since February and I was surprised to find that a new developer here couldn't access it. This is why I've been asking so many questions about tests I should perform before checking things in, but apparently it was all for naught.

-James

On 5/3/2017 7:57 AM, Pavel Afonine wrote:

...
Same questions here as Markus asked.. What is simtbx? First time hear about it. How cctbx can benefit from it? Pavel

On 5/3/17 07:53, [email protected] wrote:

...
Hi Nick,

Sorry, outsider perspective here. I don't know what simtbx does or is supposed to do.

What is the benefit of having code in a repository that, according to your earlier mail, is explicitly not used in the project?

Or more bluntly: We (and ccp4) distribute dials and, as part of it, the cctbx_project repository.

Why should we ship dead code?

-Markus

*From:*[email protected] [mailto:[email protected]] *On Behalf Of *Nicholas Sauter *Sent:* 03 May 2017 15:52 *To:* cctbx mailing list *Subject:* Re: [cctbxbb] simtbx

Markus,

We want option c. simtbx should be in the cctbx_project directory. No separate git repository. We need to double check we've gotten all the files in cctbx_project, then remove them from the separate repo. James, could you look into this?

Nick

Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division

Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713

On Wed, May 3, 2017 at 7:49 AM, mailto:[email protected]> wrote:

Hi James,

It looks that you now have two separate places where simtbx code is kept. One is in the simtbx repository, where you fixed the sign earlier, and the other one is in the cctbx repository, where you just now fixed it again.

Simtbx repo: https://github.com/dials/simtbx

Simtbx folder in cctbx repo: https://github.com/cctbx/cctbx_project/tree/master/simtbx https://github.com/cctbx/cctbx_project/tree/master/simtbx

I guess suppose there are three options here:

a.move simtbx into the module directory, add simtbx to the bootstrap script so it is initialised correctly.

b.leave simtbx inside the cctbx_project directory and repository, but as its own git repository (using git submodule)

c.leave simtbx inside the cctbx_project directory, delete the separate git repository

NB: This issue is independent of resolving the dials-dependency issue.

-Markus

*From:*[email protected] mailto:[email protected] [mailto:[email protected] mailto:[email protected]] *On Behalf Of *James Holton *Sent:* 03 May 2017 15:40 *To:* Winter, Graeme (DLSLtd,RAL,LSCI) mailto:[email protected]>; [email protected] mailto:[email protected] *Subject:* Re: [cctbxbb] simtbx

Yesterday, Aaron and I were trying to set things up so that simtbx would be available to people who aren't James Holton. It's been checked in and working for over a month now, and only a few days ago I discovered that a fresh "bootstrap" has a broken simtbx, despite my "git status" telling me all my files are up to date. I asked Aaron and Nick for help with this.

I thought what we did was make cctbx.xfel depends on simtbx, since fitting simulations to stills is the whole reason for porting my stuff into python.

I set up simtbx with a constructor that takes a dxtbx detector as input, but does that make it depend on dials? I did not expect that.

Sorry, didn't mean to break anything! Would much appreciate any help on this.

-James

On 5/3/2017 1:41 AM, [email protected] mailto:[email protected] wrote:

Hi Folks

We have noticed that simtbx, part of the core cctbx now, depends on dials

This means if you do not have a dials build your cctbx will be broken

This is probably not the way we want things?

I thought simtbx was a separate repo?

Cheerio Graeme

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Nicholas Sauter

3 May 3 May

2:57 p.m.

Markus, This is an old argument, not a new one. But the same logic, you would have us remove all our xfel code, since it has dials dependency. The fact is cctbx is a multipurpose repo contributed by several groups for different purposes. Not dead code. Nick Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713 On Wed, May 3, 2017 at 7:53 AM, wrote:

...

Hi Nick,

Sorry, outsider perspective here. I don’t know what simtbx does or is supposed to do.

What is the benefit of having code in a repository that, according to your earlier mail, is explicitly not used in the project?

Or more bluntly: We (and ccp4) distribute dials and, as part of it, the cctbx_project repository.

Why should we ship dead code?

-Markus

*From:* [email protected] [mailto:cctbxbb-bounces@ phenix-online.org] *On Behalf Of *Nicholas Sauter *Sent:* 03 May 2017 15:52 *To:* cctbx mailing list *Subject:* Re: [cctbxbb] simtbx

Markus,

We want option c. simtbx should be in the cctbx_project directory. No separate git repository. We need to double check we've gotten all the files in cctbx_project, then remove them from the separate repo. James, could you look into this?

Nick

Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division

Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713

On Wed, May 3, 2017 at 7:49 AM, wrote:

Hi James,

It looks that you now have two separate places where simtbx code is kept. One is in the simtbx repository, where you fixed the sign earlier, and the other one is in the cctbx repository, where you just now fixed it again.

Simtbx repo: https://github.com/dials/simtbx

Simtbx folder in cctbx repo: https://github.com/cctbx/ cctbx_project/tree/master/simtbx

I guess suppose there are three options here:

a. move simtbx into the module directory, add simtbx to the bootstrap script so it is initialised correctly.

b. leave simtbx inside the cctbx_project directory and repository, but as its own git repository (using git submodule)

c. leave simtbx inside the cctbx_project directory, delete the separate git repository

NB: This issue is independent of resolving the dials-dependency issue.

-Markus

*From:* [email protected] [mailto:cctbxbb-bounces@ phenix-online.org] *On Behalf Of *James Holton *Sent:* 03 May 2017 15:40 *To:* Winter, Graeme (DLSLtd,RAL,LSCI) ; [email protected] *Subject:* Re: [cctbxbb] simtbx

Yesterday, Aaron and I were trying to set things up so that simtbx would be available to people who aren't James Holton. It's been checked in and working for over a month now, and only a few days ago I discovered that a fresh "bootstrap" has a broken simtbx, despite my "git status" telling me all my files are up to date. I asked Aaron and Nick for help with this.

I thought what we did was make cctbx.xfel depends on simtbx, since fitting simulations to stills is the whole reason for porting my stuff into python.

I set up simtbx with a constructor that takes a dxtbx detector as input, but does that make it depend on dials? I did not expect that.

Sorry, didn't mean to break anything! Would much appreciate any help on this.

-James

On 5/3/2017 1:41 AM, [email protected] wrote:

Hi Folks

We have noticed that simtbx, part of the core cctbx now, depends on dials

This means if you do not have a dials build your cctbx will be broken

This is probably not the way we want things?

I thought simtbx was a separate repo?

Cheerio Graeme

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Graeme.Winter＠diamond.ac.uk

2:59 p.m.

HI Markus I think over this side of the project we are more keen on splitting things up i.e. xia2, dials, i19 modules which could all be considered as “one thing” Looks like the DIALS-West side approach is to have all things under one roof i.e. [gw56@ws133 cctbx_project]$ ls boost_adaptbx cma_es dox.sphinx iota prime site.py tbxx cbflib_adaptbx cootbx dxtbx iotbx README.md site.pyc ucif cctbx crys3d fable libtbx rstbx smtbx wxtbx chiltbx cudatbx fftw3tbx mmtbx scitbx sphinx xfel clipper_adaptbx dox gltbx omptbx simtbx spotfinder [gw56@ws133 cctbx_project]$ cd ../ [gw56@ws133 modules]$ ls allsvn ccp4io dials_scratch phenix_regression annlib ccp4io_adaptbx dlstbx scons annlib_adaptbx cctbx_project eigen tntbx bestmx clipper gui_resources xia2 boost cxi_user i19 xia2_regression bpcx_regression dials labelit cbflib dials_regression labelit_regression so more projects in cctbx_project than in my modules directory… To CCP4 / DIALS this could be “dead code” but to cctbx.xfel crew this is bread and butter stuff That said, I can see some argument to bundling fewer things in cctbx_project… Best wishes Graeme From: [email protected] [mailto:[email protected]] On Behalf Of [email protected] Sent: 03 May 2017 15:54 To: [email protected] Subject: Re: [cctbxbb] simtbx Hi Nick, Sorry, outsider perspective here. I don’t know what simtbx does or is supposed to do. What is the benefit of having code in a repository that, according to your earlier mail, is explicitly not used in the project? Or more bluntly: We (and ccp4) distribute dials and, as part of it, the cctbx_project repository. Why should we ship dead code? -Markus From: [email protected]mailto:[email protected] [mailto:[email protected]] On Behalf Of Nicholas Sauter Sent: 03 May 2017 15:52 To: cctbx mailing list mailto:[email protected]> Subject: Re: [cctbxbb] simtbx Markus, We want option c. simtbx should be in the cctbx_project directory. No separate git repository. We need to double check we've gotten all the files in cctbx_project, then remove them from the separate repo. James, could you look into this? Nick Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713 On Wed, May 3, 2017 at 7:49 AM, mailto:[email protected]> wrote: Hi James, It looks that you now have two separate places where simtbx code is kept. One is in the simtbx repository, where you fixed the sign earlier, and the other one is in the cctbx repository, where you just now fixed it again. Simtbx repo: https://github.com/dials/simtbx Simtbx folder in cctbx repo: https://github.com/cctbx/cctbx_project/tree/master/simtbx I guess suppose there are three options here: a. move simtbx into the module directory, add simtbx to the bootstrap script so it is initialised correctly. b. leave simtbx inside the cctbx_project directory and repository, but as its own git repository (using git submodule) c. leave simtbx inside the cctbx_project directory, delete the separate git repository NB: This issue is independent of resolving the dials-dependency issue. -Markus From: [email protected]mailto:[email protected] [mailto:[email protected]mailto:[email protected]] On Behalf Of James Holton Sent: 03 May 2017 15:40 To: Winter, Graeme (DLSLtd,RAL,LSCI) mailto:[email protected]>; [email protected]mailto:[email protected] Subject: Re: [cctbxbb] simtbx Yesterday, Aaron and I were trying to set things up so that simtbx would be available to people who aren't James Holton. It's been checked in and working for over a month now, and only a few days ago I discovered that a fresh "bootstrap" has a broken simtbx, despite my "git status" telling me all my files are up to date. I asked Aaron and Nick for help with this. I thought what we did was make cctbx.xfel depends on simtbx, since fitting simulations to stills is the whole reason for porting my stuff into python. I set up simtbx with a constructor that takes a dxtbx detector as input, but does that make it depend on dials? I did not expect that. Sorry, didn't mean to break anything! Would much appreciate any help on this. -James On 5/3/2017 1:41 AM, [email protected]mailto:[email protected] wrote: Hi Folks We have noticed that simtbx, part of the core cctbx now, depends on dials This means if you do not have a dials build your cctbx will be broken This is probably not the way we want things? I thought simtbx was a separate repo? Cheerio Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Graeme.Winter＠diamond.ac.uk

2:55 p.m.

Nick Fine, so long as not configured by default otherwise can take out cctbx builds as a side effect (as happened earlier) Best wishes Graeme From: Nicholas Sauter [mailto:[email protected]] Sent: 03 May 2017 15:52 To: cctbx mailing list Cc: Winter, Graeme (DLSLtd,RAL,LSCI) Subject: Re: [cctbxbb] simtbx Markus, We want option c. simtbx should be in the cctbx_project directory. No separate git repository. We need to double check we've gotten all the files in cctbx_project, then remove them from the separate repo. James, could you look into this? Nick Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713 On Wed, May 3, 2017 at 7:49 AM, mailto:[email protected]> wrote: Hi James, It looks that you now have two separate places where simtbx code is kept. One is in the simtbx repository, where you fixed the sign earlier, and the other one is in the cctbx repository, where you just now fixed it again. Simtbx repo: https://github.com/dials/simtbx Simtbx folder in cctbx repo: https://github.com/cctbx/cctbx_project/tree/master/simtbx I guess suppose there are three options here: a. move simtbx into the module directory, add simtbx to the bootstrap script so it is initialised correctly. b. leave simtbx inside the cctbx_project directory and repository, but as its own git repository (using git submodule) c. leave simtbx inside the cctbx_project directory, delete the separate git repository NB: This issue is independent of resolving the dials-dependency issue. -Markus From: [email protected]mailto:[email protected] [mailto:[email protected]mailto:[email protected]] On Behalf Of James Holton Sent: 03 May 2017 15:40 To: Winter, Graeme (DLSLtd,RAL,LSCI) mailto:[email protected]>; [email protected]mailto:[email protected] Subject: Re: [cctbxbb] simtbx Yesterday, Aaron and I were trying to set things up so that simtbx would be available to people who aren't James Holton. It's been checked in and working for over a month now, and only a few days ago I discovered that a fresh "bootstrap" has a broken simtbx, despite my "git status" telling me all my files are up to date. I asked Aaron and Nick for help with this. I thought what we did was make cctbx.xfel depends on simtbx, since fitting simulations to stills is the whole reason for porting my stuff into python. I set up simtbx with a constructor that takes a dxtbx detector as input, but does that make it depend on dials? I did not expect that. Sorry, didn't mean to break anything! Would much appreciate any help on this. -James On 5/3/2017 1:41 AM, [email protected]mailto:[email protected] wrote: Hi Folks We have noticed that simtbx, part of the core cctbx now, depends on dials This means if you do not have a dials build your cctbx will be broken This is probably not the way we want things? I thought simtbx was a separate repo? Cheerio Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom _______________________________________________ cctbxbb mailing list [email protected]mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

Nicholas Sauter

2:49 p.m.

Graeme, Yes, we want simtbx to be in cctbx_project. No, simtbx should not be configured by default when you install cctbx_project, only when you specifically ask for it, such as when xfel is configured, which already does depend on dials. And no, I was not aware of the dials dependency. Could you point it out specifically? I was only aware of it's dependence on dxtbx. Nick Nicholas K. Sauter, Ph. D. Senior Scientist, Molecular Biophysics & Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720 (510) 486-5713 On Wed, May 3, 2017 at 1:41 AM, wrote:

...

Hi Folks

We have noticed that simtbx, part of the core cctbx now, depends on dials

This means if you do not have a dials build your cctbx will be broken

This is probably not the way we want things?

I thought simtbx was a separate repo?

Cheerio Graeme

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

2598

Age (days ago)

2631

Last active (days ago)

List overview

Download

17 comments

7 participants

participants (7)

Aaron Brewster
Graeme.Winter＠diamond.ac.uk
James Holton
markus.gerstel＠diamond.ac.uk
Nicholas Sauter
Pavel Afonine
R.D. Oeffner

simtbx

markus.gerstel＠diamond.ac.uk

markus.gerstel＠diamond.ac.uk

R.D. Oeffner

tags

participants (7)