Hi Maike,

could you please send me input files and I will provide the solution for your issue?

1) reduce deviation from planarity restraints to very small value or zero  -> ligand still gets distorted in refinement

Setting esd to very small number means huge contribution of that restrain to refinement target because corresponding weight is 1/esd**2. So I'm not surprised it did not work.

2) Set all bond angle restraints to ideal values and checked sums of angles in planar rings -> ligand still gets distorted in refinement

You need a plane and phenix.refine allows you to put any desired atoms into a plane, so no need to work-around this using angles.

3) Used phenix.elbow with - -final-geometry on a pdb of a planar ligand —> ligand still gets distorted in refinement

I hoped Nigel addresses this one.

4) Used the grade server (grade.globalphasing.org) to generate cif file from SMILES string—> phenix.refine stops with message “Unknown file format: grade-ligand.cif”.

External to Phenix tools do not guarantee to provide inputs good for Phenix.

Does anyone have another suggestion,

See above: send me files and I will make it work.

except for installing an extra QM package?

QM in this particular case would be a "cannon to kill a fly".

Pavel