I sent you the files. To answer your question, I had weight optimization. Surprisingly, it worked well with phenix 1.8.2. But I'm ready to believe that there is a bug in my data (maybe not tolerated by this version of phenix) Thanks 2013/10/8, Nathaniel Echols :

Whenever you see the bond and angle RMSDs blow up like this, it means that the weight between the X-ray and geometry restraint terms was not calculated correctly. Probably the quickest solution is to set wxc_scale much lower - the default is 0.5 but at this resolution I think 0.1 would be more appropriate, and you could try setting it even lower. (Note that if you have weight optimization turned on this will have no effect.) However, the fact that you didn't see this before suggests that there may be a bug somewhere that is specific to your data. Could you please send us input files and log file at [email protected] so one of us can take a look? (These will be kept private, obviously.)

thanks, Nat

On Mon, Oct 7, 2013 at 12:46 PM, Jan van Agthoven wrote:

...

Hi everyone, I've been working on a low resolution structure (3.6A) which was previously refined with phenix 1.8_1069 (with a reference model, strategy=individual_sites,individual_adp ). I used the same input files and parameters and ended up with significantly different R factors.

Start: r_work = 0.2850 r_free = 0.3189 bonds = 0.005 angles = 1.059

Final: r_work = 0.2609 r_free = 0.3160 bonds = 0.003 angles = 0.997

For the old version

and

Start: r_work = 0.2779 r_free = 0.3183 bonds = 0.006 angles = 1.141 Final: r_work = 0.2889 r_free = 0.3303 bonds = 0.021 angles = 2.737

with the new version 1.8.4

Remarkably it also came with an increased number of C-beta deviation (0 against 16!).

Adding tls refinement or ramachandran restrains did not help.

Now because I was using the .eff file of an old version of phenix, it came with a long list of unused parameters. These parameters however were all used by default.

What should I do? Should I go back to this old phenix, or is there a way to improve the result?

Thanks, _______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

Okay, everyone experience problems like this in 1.8.4, could you please answer the following survey: 1) Are you using secondary structure restraints? 2) If you turn off secondary structure restraints, does that fix the problem? 3) If you leave secondary structure restraints on, but set the parameter refinement.secondary_structure.h_bond_restraints.remove_outliers to True, does that also yield a reasonably correct result? (GUI users, you should be able to just do a parameter search for "remove_outliers".) Jan: using the files you sent me, the fix in (3) does appear to solve the problem. Ryan: if you have 1.3Å data and you answered yes to (1), you really shouldn't use secondary structure restraints at high resolution. (I'm not sure there's any fixed cutoff at which they become irrelevant or dangerous, but my guess would be anything below 2.0Å doesn't need them.) Anyone else: I'm hoping you all downloaded 1.8.4 before this past weekend (or if you're using 64-bit Linux, before this morning). The installers currently online have been patched to fix the problem (I think) - note however that using parameters from restored runs will *not* work properly. thanks, Nat On Tue, Oct 8, 2013 at 9:31 AM, Ryan Spencer wrote:

Hi Nat,****

** **

I’m having a similar issue only in reverse. I’m working with a 1.3A data set and halfway during refinement the RMS angles and bonds shoot up (0.02/2.3) and the Rfree/Rwork drops by two points (17/15). When I don’t use the X-ray/stereochemistry and X-ray/ADP weight optimization the Rfree/Rwork values shoot up 10 points on the second run and the RMS angles/bonds drop significantly. Any help would be appreciated. I’m running the refinement on 1.8.4-1496.****

** **

Ryan Spencer****

University of California, Irvine****

Nowick Research Group ****

** **

*From:* [email protected] [mailto: [email protected]] *On Behalf Of *Nathaniel Echols *Sent:* Tuesday, October 08, 2013 4:24 AM *To:* PHENIX user mailing list *Subject:* Re: [phenixbb] Worse R factor with newer version of phenix****

** **

Whenever you see the bond and angle RMSDs blow up like this, it means that the weight between the X-ray and geometry restraint terms was not calculated correctly. Probably the quickest solution is to set wxc_scale much lower - the default is 0.5 but at this resolution I think 0.1 would be more appropriate, and you could try setting it even lower. (Note that if you have weight optimization turned on this will have no effect.) However, the fact that you didn't see this before suggests that there may be a bug somewhere that is specific to your data. Could you please send us input files and log file at [email protected] so one of us can take a look? (These will be kept private, obviously.)****

** **

thanks,****

Nat****

** **

On Mon, Oct 7, 2013 at 12:46 PM, Jan van Agthoven wrote:****

Hi everyone, I've been working on a low resolution structure (3.6A) which was previously refined with phenix 1.8_1069 (with a reference model, strategy=individual_sites,individual_adp ). I used the same input files and parameters and ended up with significantly different R factors.

Start: r_work = 0.2850 r_free = 0.3189 bonds = 0.005 angles = 1.059

Final: r_work = 0.2609 r_free = 0.3160 bonds = 0.003 angles = 0.997

For the old version

and

Start: r_work = 0.2779 r_free = 0.3183 bonds = 0.006 angles = 1.141 Final: r_work = 0.2889 r_free = 0.3303 bonds = 0.021 angles = 2.737

with the new version 1.8.4

Remarkably it also came with an increased number of C-beta deviation (0 against 16!).

Adding tls refinement or ramachandran restrains did not help.

Now because I was using the .eff file of an old version of phenix, it came with a long list of unused parameters. These parameters however were all used by default.

What should I do? Should I go back to this old phenix, or is there a way to improve the result?

Thanks, _______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb****

** **

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

Dr Felix Frolow Professor of Structural Biology and Biotechnology, Department of Molecular Microbiology and Biotechnology Tel Aviv University 69978, Israel Acta Crystallographica F, co-editor e-mail: [email protected] Tel: ++972-3640-8723 Fax: ++972-3640-9407 Cellular: 0547 459 608 On Oct 9, 2013, at 03:30 , Nathaniel Echols wrote:

Okay, everyone experience problems like this in 1.8.4, could you please answer the following survey:

My problems started earlier as the only version I can use without significant diversion R factors (several %) is 1.8.2.1309 (of those still visible on nightly builds).

1) Are you using secondary structure restraints?

NO

2) If you turn off secondary structure restraints, does that fix the problem?

Irrelevant

3) If you leave secondary structure restraints on, but set the parameter refinement.secondary_structure.h_bond_restraints.remove_outliers to True, does that also yield a reasonably correct result? (GUI users, you should be able to just do a parameter search for "remove_outliers".)

Irrelevant

Jan: using the files you sent me, the fix in (3) does appear to solve the problem.

Ryan: if you have 1.3Å data and you answered yes to (1), you really shouldn't use secondary structure restraints at high resolution. (I'm not sure there's any fixed cutoff at which they become irrelevant or dangerous, but my guess would be anything below 2.0Å doesn't need them.)

Anyone else: I'm hoping you all downloaded 1.8.4 before this past weekend (or if you're using 64-bit Linux, before this morning). The installers currently online have been patched to fix the problem (I think) - note however that using parameters from restored runs will *not* work properly.

thanks, Nat

On Tue, Oct 8, 2013 at 9:31 AM, Ryan Spencer wrote: Hi Nat,

I’m having a similar issue only in reverse. I’m working with a 1.3A data set and halfway during refinement the RMS angles and bonds shoot up (0.02/2.3) and the Rfree/Rwork drops by two points (17/15). When I don’t use the X-ray/stereochemistry and X-ray/ADP weight optimization the Rfree/Rwork values shoot up 10 points on the second run and the RMS angles/bonds drop significantly. Any help would be appreciated. I’m running the refinement on 1.8.4-1496.

Ryan Spencer

University of California, Irvine

Nowick Research Group

From: [email protected] [mailto:[email protected]] On Behalf Of Nathaniel Echols Sent: Tuesday, October 08, 2013 4:24 AM To: PHENIX user mailing list Subject: Re: [phenixbb] Worse R factor with newer version of phenix

Whenever you see the bond and angle RMSDs blow up like this, it means that the weight between the X-ray and geometry restraint terms was not calculated correctly. Probably the quickest solution is to set wxc_scale much lower - the default is 0.5 but at this resolution I think 0.1 would be more appropriate, and you could try setting it even lower. (Note that if you have weight optimization turned on this will have no effect.) However, the fact that you didn't see this before suggests that there may be a bug somewhere that is specific to your data. Could you please send us input files and log file at [email protected] so one of us can take a look? (These will be kept private, obviously.)

thanks,

Nat

On Mon, Oct 7, 2013 at 12:46 PM, Jan van Agthoven wrote:

Hi everyone, I've been working on a low resolution structure (3.6A) which was previously refined with phenix 1.8_1069 (with a reference model, strategy=individual_sites,individual_adp ). I used the same input files and parameters and ended up with significantly different R factors.

Start: r_work = 0.2850 r_free = 0.3189 bonds = 0.005 angles = 1.059

Final: r_work = 0.2609 r_free = 0.3160 bonds = 0.003 angles = 0.997

For the old version

and

Start: r_work = 0.2779 r_free = 0.3183 bonds = 0.006 angles = 1.141 Final: r_work = 0.2889 r_free = 0.3303 bonds = 0.021 angles = 2.737

with the new version 1.8.4

Remarkably it also came with an increased number of C-beta deviation (0 against 16!).

Adding tls refinement or ramachandran restrains did not help.

Now because I was using the .eff file of an old version of phenix, it came with a long list of unused parameters. These parameters however were all used by default.

What should I do? Should I go back to this old phenix, or is there a way to improve the result?

Thanks, _______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

Thanks, It works much better! Sent from my iPod On Oct 8, 2013, at 8:30 PM, Nathaniel Echols wrote:

Okay, everyone experience problems like this in 1.8.4, could you please answer the following survey:

1) Are you using secondary structure restraints? 2) If you turn off secondary structure restraints, does that fix the problem? 3) If you leave secondary structure restraints on, but set the parameter refinement.secondary_structure.h_bond_restraints.remove_outliers to True, does that also yield a reasonably correct result? (GUI users, you should be able to just do a parameter search for "remove_outliers".)

Jan: using the files you sent me, the fix in (3) does appear to solve the problem.

Ryan: if you have 1.3Å data and you answered yes to (1), you really shouldn't use secondary structure restraints at high resolution. (I'm not sure there's any fixed cutoff at which they become irrelevant or dangerous, but my guess would be anything below 2.0Å doesn't need them.)

Anyone else: I'm hoping you all downloaded 1.8.4 before this past weekend (or if you're using 64-bit Linux, before this morning). The installers currently online have been patched to fix the problem (I think) - note however that using parameters from restored runs will *not* work properly.

thanks, Nat

On Tue, Oct 8, 2013 at 9:31 AM, Ryan Spencer wrote:

...

Hi Nat,

I’m having a similar issue only in reverse. I’m working with a 1.3A data set and halfway during refinement the RMS angles and bonds shoot up (0.02/2.3) and the Rfree/Rwork drops by two points (17/15). When I don’t use the X-ray/stereochemistry and X-ray/ADP weight optimization the Rfree/Rwork values shoot up 10 points on the second run and the RMS angles/bonds drop significantly. Any help would be appreciated. I’m running the refinement on 1.8.4-1496.

Ryan Spencer

University of California, Irvine

Nowick Research Group

From: [email protected] [mailto:[email protected]] On Behalf Of Nathaniel Echols Sent: Tuesday, October 08, 2013 4:24 AM To: PHENIX user mailing list Subject: Re: [phenixbb] Worse R factor with newer version of phenix

Whenever you see the bond and angle RMSDs blow up like this, it means that the weight between the X-ray and geometry restraint terms was not calculated correctly. Probably the quickest solution is to set wxc_scale much lower - the default is 0.5 but at this resolution I think 0.1 would be more appropriate, and you could try setting it even lower. (Note that if you have weight optimization turned on this will have no effect.) However, the fact that you didn't see this before suggests that there may be a bug somewhere that is specific to your data. Could you please send us input files and log file at [email protected] so one of us can take a look? (These will be kept private, obviously.)

thanks,

Nat

On Mon, Oct 7, 2013 at 12:46 PM, Jan van Agthoven wrote:

Hi everyone, I've been working on a low resolution structure (3.6A) which was previously refined with phenix 1.8_1069 (with a reference model, strategy=individual_sites,individual_adp ). I used the same input files and parameters and ended up with significantly different R factors.

Start: r_work = 0.2850 r_free = 0.3189 bonds = 0.005 angles = 1.059

Final: r_work = 0.2609 r_free = 0.3160 bonds = 0.003 angles = 0.997

For the old version

and

Start: r_work = 0.2779 r_free = 0.3183 bonds = 0.006 angles = 1.141 Final: r_work = 0.2889 r_free = 0.3303 bonds = 0.021 angles = 2.737

with the new version 1.8.4

Remarkably it also came with an increased number of C-beta deviation (0 against 16!).

Adding tls refinement or ramachandran restrains did not help.

Now because I was using the .eff file of an old version of phenix, it came with a long list of unused parameters. These parameters however were all used by default.

What should I do? Should I go back to this old phenix, or is there a way to improve the result?

Thanks, _______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb

_______________________________________________ phenixbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/phenixbb