[cctbxbb] Overloading Python builtin names with local variables

[email protected] Graeme.Winter at diamond.ac.uk
Fri May 3 00:45:05 PDT 2019


Morning all,

As a side-effect of looking at some of the Python3 refactoring I stumbled across a few places where e.g. range is used as a variable name. While this is fine and legal, when moving the code to Python3

xrange => range

and

from builtins import range

is a back-port of Python3 range to Python2 which allows the existing behaviour of xrange generator to be maintained

This is fine except for where range is a variable name and we then get a “variable referenced before assignment” type error

As a general statement I would say overloading Python reserved names with local variables is untidy at best, and can easily cause confusion (as well as real problems as identified above) - I have therefore taken the liberty of adding a tool to libtbx which can be used to find such variables to give people a chance to avoid them:

Grey-Area dxtbx :) [master] $ libtbx.find_reserved_names format/
Checking format/
format/FormatSMVRigakuSaturn.py:239 "format"
        format = self._scan_factory.format("SMV")

format/FormatTIFFRayonixESRF.py:36 "format"
        format = {LITTLE_ENDIAN: "<", BIG_ENDIAN: ">"}[order]

format/FormatTIFFRayonixESRF.py:29 "bytes"
        width, height, depth, order, bytes = FormatTIFFRayonix.get_tiff_header(

format/FormatCBFMini.py:231 "format"
        format = self._scan_factory.format("CBF")

format/FormatTIFFRayonix.py:176 "format"
        format = self._scan_factory.format("TIFF")

format/FormatTIFFRayonix.py:31 "bytes"
        width, height, depth, order, bytes = FormatTIFF.get_tiff_header(image_file)

format/FormatTIFFRayonix.py:74 "bytes"
        width, height, depth, order, bytes = FormatTIFF.get_tiff_header(image_file)

format/FormatSMVRigakuEiger.py:243 "format"
        format = self._scan_factory.format("SMV")

format/FormatSMVADSC.py:212 "format"
        format = self._scan_factory.format("SMV")

format/Registry.py:88 "format"
        for format in sorted(self._formats, key=lambda x: x.__name__):

format/FormatSMVJHSim.py:141 "format"
        format = self._scan_factory.format("SMV")

format/FormatSMVRigakuPilatus.py:188 "format"
        format = self._scan_factory.format("SMV")

format/FormatSMVADSCSN928.py:47 "format"
        format = self._scan_factory.format("SMV")

format/FormatRAXISIVSpring8.py:134 "format"
        format = self._scan_factory.format("RAXIS")

format/FormatTIFFRayonixSPring8.py:31 "bytes"
        width, height, depth, order, bytes = FormatTIFFRayonix.get_tiff_header(

format/FormatSMVADSCmlfsom.py:37 "format"
        format = self._scan_factory.format("SMV")

format/FormatTIFFBruker.py:174 "format"
        format = self._scan_factory.format("TIFF")

format/FormatTIFFBruker.py:31 "bytes"
        width, height, depth, order, bytes = FormatTIFF.get_tiff_header(image_file)

format/FormatTIFFBruker.py:71 "bytes"
        width, height, depth, order, bytes = FormatTIFF.get_tiff_header(image_file)

format/FormatCBFMiniADSCHF4M.py:26 "format"
    for format in ["%a_%b_%d_%H:%M:%S_%Y"]:

format/FormatCBFMiniADSCHF4M.py:169 "format"
        format = self._scan_factory.format("CBF")

format/FormatSMVADSCNoDateStamp.py:54 "format"
        format = self._scan_factory.format("SMV")

format/FormatSMVRigakuSaturnNoTS.py:228 "format"
        format = self._scan_factory.format("SMV")

format/FormatPYunspecified.py:152 "format"
            format = ""

format/FormatSMVRigakuA200.py:234 "format"
        format = self._scan_factory.format("SMV")

format/FormatCBFMiniEigerPhotonFactory.py:131 "format"
        format = self._scan_factory.format("CBF")

format/FormatSMVTimePix_SU.py:97 "format"
        format = self._scan_factory.format("SMV")

format/FormatCBFMiniPilatus.py:83 "format"
        format = self._scan_factory.format("CBF")

format/FormatCBFMiniPilatusHelpers.py:22 "format"
    for format in ["%Y-%b-%dT%H:%M:%S", "%Y-%m-%dT%H:%M:%S", "%Y/%b/%d %H:%M:%S"]:

format/FormatSMVADSCSNAPSID19.py:91 "format"
        format = self._scan_factory.format("SMV")

format/FormatCBFMiniEiger.py:216 "format"
        format = self._scan_factory.format("CBF")

format/FormatSMVNOIR.py:218 "format"
        format = self._scan_factory.format("SMV")

format/FormatSMVCMOS1.py:197 "format"
        format = self._scan_factory.format("SMV")

format/FormatRAXIS.py:238 "format"
        format = self._scan_factory.format("RAXIS")


is an example where “format” is scattered all over the place but is also a useful function - replacing with fmt will mean the same and avoid this overloading, as an example.

This uses AST parsing to work through the code and find variable assignments - doing this for dials found lots of examples of “file” and “type” being used.

How much people care about this is a local issue, but I thought the tool would be useful

All the best Graeme


-- 
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom



More information about the cctbxbb mailing list