Author(s)

• segment_and_split_map: Tom Terwilliger

Purpose

The routine segment_and_split_map will identify the asymmetric unit of a map (typically cryo-EM) and contiguous regions of density within the asymmetric unit of the map.

Usage

How segment_and_split_map works:

If you have a CCP4-style (mrc, etc) map and a sequence file, you can use segment_and_split_map to split the map into smaller pieces suitable for model-building or viewing. The tool segment_and_split_map will will find where your molecule is in the map and cut out and work with just that part of the density. If your map has been averaged based on NCS symmetry and you supply a file with that NCS information (.ncs_spec, biomtr.dat, etc), segment_and_split_map will find the asymmetric unit of NCS and work with that.

Finally, the segment_and_split_map tool will cut the density in the asymmetic unit of your map into small pieces of connected density and write out a map for each one.

All the maps that are written by segment_and_split_map are superimposable on each other. They typically are all shifted from the original map to place the origin of the maps on the grid point (0,0,0).

Output files from segment_and_split_map

shifted_map.ccp4: Original map, shifted to place the origin on grid point (0,0,)

shifted_ncs.ncs_spec: NCS operators (if any), shifted to match shifted_map.ccp4.

shifted_pdb.pdb: Input PDB file (if any), shifted to match shifted_map.ccp4.

box_map_au.ccp4: Same as shifted_map.ccp4, except that everything except the asymmetric unit of NCS is zeroed out (map shows the asymmetric unit only).

box_mask_au.ccp4: Mask showing location of NCS asymmetric unit. Superimposes on box_map_au.ccp4 and shifted_map.ccp4.

segment_and_split_map_info.pkl: Pickled file with information about the segmentation. Used in phenix.map_to_model and to restore a shifted PDB file to original location.

Shifting the map to the origin

Most crystallographic maps have the origin at the corner of the map ( grid point [0,0,0]), while most cryo-EM maps have the orgin in the middle of the map. To make a consistent map, any maps with an origin not at the corner are shifted to put the origin at grid point [0,0,0]. This map is the shifted map that is used for further steps in model-building. At the conclusion of model-building, the model is shifted back to superimpose on the original map.

Finding the region containing the molecule

By default (density_select=True), the region of the map containing density is cut out of the entire map. This is particularly useful if the original map is very large and the molecule only takes up a small part of the map. This portion of the map is then shifted to place the origin at grid point [0,0,0]. (At the conclusion of model-building, the final model is shifted back to superimpose on the original map.) The region containing density is chosen as a box containing all the points above a threshold, typically 5% of the maximum in the map.

Finding the NCS asymmetric unit of the map

If you supply NCS matrices describing the NCS used to average the map (if any), then segment_and_split_map will try to define a region of the map that represents the NCS asymmetric unit. Application of the NCS operators to the NCS asymmetric unit will generate the entire map, and application to a model built into the asymmetric unit will generate the entire model. Normally identification of the NCS asymmetric unit and segmentation of the map (below) are done as a single step, yielding an asymmetric unit and a set of contiguous regions of density within that asymmetric unit. The asymmetric unit of NCS will be written out as a map to the segmentation_dir directory, superimposed on the shifted map (so that they can be viewed together in Coot).

Segmentation of the map

By default (segment=True) the map or NCS asymmetric unit of the map will be segmented (cut into small pieces) into regions of connected density. This is done by choosing a threshold of density and identifying contiguous regions where all grid points are above this threshold. The threshold is chosen to yield regions that have a size corresponding to about 50 residues. The regions of density are written out to the segmentation_dir directory and are superimposed on the shifted map (if you load the shifted map in Coot and a region map in Coot, they should superimpose.)

Examples

Standard run of segment_and_split_map:

Running segment_and_split_map is easy. From the command-line you can type:

phenix.segment\_and\_split\_map my_map.map seq.fa ncs_file=find_ncs.ncs_spec

where my_map.map is a CCP4, mrc or other related map format, seq.fa is a sequence file, and find_ncs.ncs_spec is an optional file specifying any NCS operators used in averaging the map. This can be in the form of BIOMTR records from a PDB file as well.

Possible Problems

Specific limitations and problems:

Literature

Additional information

List of all available keywords

• input_files
  • seq_file = None Sequence file (unique chains only, 1-letter code, chains separated by blank line or greater-than sign.) Can have chains that are DNA/RNA/protein and all can be present in one file.
  • map_file = None File with CCP4-style map
  • half_map_file = None Half map (two should be supplied) for FSC calculation. Must have grid identical to map_file
  • ncs_file = None File with NCS information (typically point-group NCS with the center specified). Typically in PDB format. Can also be a .ncs_spec file from phenix. Created automatically if ncs_type is specified.
  • pdb_file = None Optional PDB file matching map_file to be offset
  • pdb_to_restore = None Optional PDB file to restore to position matching original map_file. Used in combination with info_file=xxx.pkl and restored_pdb=yyyy.pdb
  • info_file = None Optional pickle file with information from a previous run. Can be used with pdb_to_restore to restore a PDB file to to position matching original map_file.
  • target_ncs_au_file = None Optional PDB file to partially define the ncs asymmetric unit of the map. The coordinates in this file will be used to mark part of the ncs au and all points nearby that are not part of another ncs au will be added.
  • input_weight_map_pickle_file = None Weight map pickle file
• output_files
  • magnification_map_file = magnification_map.ccp4 Input map file with magnification applied. Only written if magnification is applied.
  • magnification_ncs_file = magnification_ncs.ncs_spec Input NCS with magnification applied. Only written if magnification is applied.
  • shifted_map_file = shifted_map.ccp4 Input map file shifted to new origin.
  • shifted_sharpened_map_file = shifted_sharpened_map.ccp4 Input map file shifted to new origin and sharpened.
  • shifted_pdb_file = shifted_pdb.pdb Input pdb file shifted to new origin.
  • shifted_ncs_file = shifted_ncs.ncs_spec NCS information shifted to new origin.
  • shifted_used_ncs_file = shifted_used_ncs.ncs_spec NCS information (just the part that is used) shifted to new origin.
  • output_directory = segmented_maps Directory where output files are to be written applied.
  • box_map_file = box_map_au.ccp4 Output map file with one NCS asymmetric unit, cut out box
  • box_mask_file = box_mask_au.ccp4 Output mask file with one NCS asymmetric unit, cut out box
  • box_buffer = 5 Buffer (grid units) around NCS asymmetric unit in box_mask and map
  • au_output_file_stem = shifted_au File stem for output map files with one NCS asymmetric unit
  • write_intermediate_maps = False Write out intermediate maps and masks for visualization
  • write_output_maps = True Write out maps
  • remainder_map_file = remainder_map.ccp4 output map file with remainder after initial regions identified
  • output_info_file = segment_and_split_map_info.pkl Output pickle file with information about map and masks
  • restored_pdb = None Output name of PDB restored to position matching original map_file. Used in combination with info_file=xxx.pkl and pdb_to_restore=xxxx.pdb
  • output_weight_map_pickle_file = weight_map_pickle_file.pkl Output weight map pickle file
• crystal_info
  • chain_type = *None PROTEIN RNA DNA Chain type. Determined automatically from sequence file if not given. Mixed chain types are fine (leave blank if so).
  • is_crystal = False Defines whether this is a crystal (or cryo-EM). Normally set is_crystal along with use_sg_symmetry.
  • use_sg_symmetry = False If you set use_sg_symmetry=True then the symmetry of the space group will be used. For example in P1 a point at one end of the unit cell is next to a point on the other end. Normally for cryo-EM data this should be set to False and for crystal data it should be set to True.
  • resolution = None Nominal resolution of the map. This is used later to decide on resolution cutoffs for Fourier inversion of the map. Note: the resolution is not cut at this value, it is cut at resolution*d_min_ratio if at all.
  • space_group = None Space group (used for boxed maps)
  • unit_cell = None Unit Cell (used for boxed maps)
  • solvent_content = None Solvent fraction of the cell. Used for ID of solvent content in boxed maps.
  • solvent_content_iterations = 3 Iterations of solvent fraction estimation. Used for ID of solvent content in boxed maps.
  • wang_radius = None Wang radius for solvent identification. Default is 1.5* resolution
  • buffer_radius = None Buffer radius for mask smoothing. Default is resolution
  • pseudo_likelihood = None Use pseudo-likelihood method for half-map sharpening. (In development)
• reconstruction_symmetry
  • ncs_type = None Symmetry used in reconstruction. For example D7, C3, C2 I (icosahedral),T (tetrahedral), or ANY (try everything and use the highest symmetry found). Not needed if ncs_file is supplied. Note: ANY does not search for helical symmetry
  • ncs_center = None Center (in A) for NCS operators (if ncs is found automatically). If set to None, first guess is the center of the cell and then if that fails, found automatically as the center of the density in the map.
  • optimize_center = None Optimize position of NCS center. Default is False if ncs_center is supplied or center of map is used and True if it is found automatically).
  • helical_rot_deg = None helical rotation about z in degrees
  • helical_trans_z_angstrom = None helical translation along z in Angstrom units
  • two_fold_along_x = None Specifies if D or I two-fold is along x (True) or y (False). If None, both are tried.
  • random_points = 100 Number of random points in map to examine in finding NCS
  • identify_ncs_id = True If NCS is not point-group symmetry, try each possible operator when evaluating NCS and choose the one that results in the most uniform density at NCS-related points.
  • min_ncs_cc = 0.75 Minimum NCS CC to keep operators when identifying automatically
  • n_rescore = 5 Number of NCS operators to rescore
  • op_max = 14 If ncs_type is ANY, try up to op_max-fold symmetries
  • tol_r = 0.02 tolerance in rotations for point group or helical symmetry
  • abs_tol_t = 2 tolerance in translations (A) for point group or helical symmetry
  • max_helical_operators = None Maximum helical operators (if extending existing helical operators)
  • rel_tol_t = .05 tolerance in translations (fractional) for point group or helical symmetry
  • require_helical_or_point_group_symmetry = False normally helical or point-group symmetry (or none) is expected. However in some cases (helical + rotational symmetry for example) this is not needed and is not the case.
• map_modification
  • magnification = None Magnification to apply to input map. Input map grid will be scaled by magnification factor before anything else is done.
  • b_iso = None Target B-value for map (sharpening will be applied to yield this value of b_iso). If sharpening method is not supplied, default is to use b_iso_to_d_cut sharpening.
  • b_sharpen = None Sharpen with this b-value. Contrast with b_iso that yield a targeted value of b_iso. B_sharpen greater than zero is sharpening. Less than zero is blurring.
  • b_blur_hires = 200 Sharpen high_resolution data (higher than d_cut) with b_sharpen plus b_blur_hires. Reduces sharpening (or causes blurring) at high resolution. If None and b_sharpen is positive (sharpening) then high-resolution data is left as is (not sharpened). If None and b_sharpen is negative (blurring) high-resolution data is also blurred.
  • resolution_dependent_b = None If set, apply resolution_dependent_b (b0 b1 b2). Log10(amplitudes) will start at 1, change to b0 at half of resolution specified, changing linearly, change to b1/2 at resolution specified, and change to b1/2+b2 at d_min_ratio*resolution
  • normalize_amplitudes_in_resdep = False Normalize amplitudes in resolution-dependent sharpening
  • d_min_ratio = 0.833 Sharpening will be applied using d_min equal to d_min_ratio times resolution. Default is 0.833
  • scale_max = 100000 Scale amplitudes from inverse FFT to yield maximum of this value
  • input_d_cut = None High-resolution limit for sharpening
  • rmsd = None RMSD of model to true model (if supplied). Used to estimate expected fall-of with resolution of correct part of model-based map. If None, assumed to be resolution times rmsd_resolution_factor.
  • rmsd_resolution_factor = 0.25 default RMSD is resolution times resolution factor
  • fraction_complete = None Completness of model (if supplied). Used to estimate correct part of model-based map. If None, estimated from max(FSC).
  • auto_sharpen = True Automatically determine sharpening using kurtosis maximization or adjusted surface area
  • auto_sharpen_methods = no_sharpening b_iso *b_iso_to_d_cut resolution_dependent model_sharpening half_map_sharpening target_b_iso_to_d_cut None Methods to use in sharpening. b_iso searches for b_iso to maximize sharpening target (kurtosis or adjusted_sa). b_iso_to_d_cut applies b_iso only up to resolution specified, with fall-over of k_sharpen. Resolution dependent adjusts 3 parameters to sharpen variably over resolution range. Default is b_iso_to_d_cut .
  • box_in_auto_sharpen = False Use a representative box of density for initial auto-sharpening instead of the entire map.
  • density_select_in_auto_sharpen = False Choose representative box of density for initial auto-sharpening with density_select method (choose region where there is high density). Normally use instead density_select=True which carries out density_select at start of segmentation.
  • allow_box_if_b_iso_set = False Allow box_in_auto_sharpen (if set to True) even if b_iso is set. Default is to set box_n_auto_sharpen=False if b_iso is set.
  • soft_mask = True Use soft mask (smooth change from inside to outside with radius based on resolution of map). Required if you use half-map sharpening without a model, otherwise optional.
  • use_weak_density = False When choosing box of representative density, use poor density (to get optimized map for weaker density)
  • discard_if_worse = None Discard sharpening if worse
  • local_sharpening = None Sharpen locally using overlapping regions. NOTE: Best to turn off local_aniso_in_local_sharpening if NCS is present. If local_aniso_in_local_sharpening is True and NCS is present this can distort the map for some NCS copies because an anisotropy correction is applied based on local density in one copy and is transferred without rotation to other copies.
  • local_aniso_in_local_sharpening = None Use local anisotropy in local sharpening. Default is True unless NCS is present.
  • overall_before_local = True Apply overall scaling before local scaling
  • select_sharpened_map = None Select a single sharpened map to use
  • read_sharpened_maps = None Read in previously-calculated sharpened maps
  • write_sharpened_maps = None Write out local sharpened maps
  • smoothing_radius = None Sharpen locally using smoothing_radius. Default is 2/3 of mean distance between centers for sharpening
  • box_center = None You can specify the center of the box (A units)
  • box_size = 40 40 40 You can specify the size of the boxes to use (grid units)
  • target_n_overlap = 10 You can specify the targeted overlap of boxes in local sharpening
  • restrict_map_size = True Restrict box map to be inside full map (required for cryo-EM data)
  • restrict_z_turns_for_helical_symmetry = 1 Restrict Z turns for helical symmetry. Number of turns of helix going each direction in Z is specified.
  • restrict_z_distance_for_helical_symmetry = None Restrict Z distance (+/- this distance from center) for helical symmetry.
  • remove_aniso = True You can remove anisotropy (overall and locally) during sharpening
  • cc_cut = 0.2 Estimate of minimum highly reliable CC in half-map FSC. Used to decide at what CC value to smooth the remaining CC values.
  • max_cc_for_rescale = 0.2 Used along with cc_cut and scale_using_last to correct for small errors in FSC estimation at high resolution. If the value of FSC near the high-resolution limit is above max_cc_for_rescale, assume these values are correct and do not correct them.
  • scale_using_last = 3 If set, assume that the last scale_using_last bins in the FSC for half-map or model sharpening are about zero (corrects for errors int the half-map process).
  • max_box_fraction = 0.5 If box is greater than this fraction of entire map, use entire map.
  • density_select_max_box_fraction = 0.95 If box is greater than this fraction of entire map, use entire map for density_select. Default is 0.95
  • mask_atoms = True Mask atoms when using model sharpening
  • mask_atoms_atom_radius = 3 Mask for mask_atoms will have mask_atoms_atom_radius
  • value_outside_atoms = None Value of map outside atoms (set to mean to have mean value inside and outside mask be equal)
  • k_sharpen = 10 Steepness of transition between sharpening (up to resolution ) and not sharpening (d < resolution). Note: for blurring, all data are blurred (regardless of resolution), while for sharpening, only data with d about resolution or lower are sharpened. This prevents making very high-resolution data too strong. Note 2: if k_sharpen is zero, then no transition is applied and all data is sharpened or blurred. Note 3: only used if b_iso is set.
  • optimize_k_sharpen = None Optimize value of k_sharpen. Only applies for auto_sharpen_methods b_iso_to_d_cut and b_iso
  • optimize_d_cut = None Optimize value of d_cut. Only applies for auto_sharpen_methods b_iso_to_d_cut and b_iso
  • adjust_region_weight = True Adjust region_weight to make overall change in surface area equal to overall change in normalized regions over the range of search_b_min to search_b_max using b_iso_to_d_cut.
  • region_weight_method = initial_ratio *delta_ratio b_iso Method for choosing region_weights. Initial_ratio uses ratio of surface area to regions at low B value. Delta ratio uses change in this ratio from low to high B. B_iso uses resolution-dependent b_iso (not weights) with the formula b_iso=5.9*d_min**2
  • region_weight_factor = 1.0 Multiplies region_weight after calculation with region_weight_method above
  • region_weight_buffer = 0.1 Region_weight adjusted to be region_weight_buffer away from minimum or maximum values
  • region_weight_default = 30. Region_weight adjusted to be region_weight_default if no information available
  • target_b_iso_ratio = 5.9 Target b_iso ratio : b_iso is estimated as target_b_iso_ratio * resolution**2
  • signal_min = 3.0 Minimum signal in estimation of optimal b_iso. If not achieved, use any other method chosen.
  • target_b_iso_model_scale = 0. For model sharpening, the target_biso is scaled (normally zero).
  • search_b_min = -100 Low bound for b_iso search.
  • search_b_max = 300 High bound for b_iso search.
  • search_b_n = 21 Number of b_iso values to search.
  • residual_target = 'adjusted_sa' Target for maximization steps in sharpening. Can be kurtosis or adjusted_sa (adjusted surface area)
  • sharpening_target = 'adjusted_sa' Overall target for sharpening. Can be kurtosis or adjusted_sa (adjusted surface area). Used to decide which sharpening approach is used. Note that during optimization, residual_target is used (they can be the same.)
  • region_weight = 40 Region weighting in adjusted surface area calculation. Score is surface area minus region_weight times number of regions. Default is 40. A smaller value will give more sharpening.
  • sa_percent = 30. Percent of target regions used in calulation of adjusted surface area. Default is 30.
  • fraction_occupied = 0.20 Fraction of molecular volume targeted to be inside contours. Used to set contour level. Default is 0.20
  • n_bins = 20 Number of resolution bins for sharpening. Default is 20.
  • max_regions_to_test = 30 Number of regions to test for surface area in adjusted_sa scoring of sharpening
  • eps = None
  • k_sol = 0.35 k_sol value for model map calculation. IGNORED (Not applied)
  • b_sol = 50 b_sol value for model map calculation. IGNORED (Not applied)
• segmentation
  • density_select = True Run map_box with density_select=True to cut out the region in the input map that contains density. Useful if the input map is much larger than the structure. Done before segmentation is carried out.
  • density_select_threshold = 0.05 Choose region where density is this fraction of maximum or greater
  • get_half_height_width = None Use 4 times half-width at half-height as estimate of max size
  • mask_threshold = None threshold in identification of overall mask. If None, guess volume of molecule from sequence and NCS copies.
  • grid_spacing_for_au = 3 Grid spacing for asymmetric unit when constructing asymmetric unit.
  • radius = None Radius for constructing asymmetric unit.
  • value_outside_mask = 0.0 Value to assign to density outside masks
  • density_threshold = None Threshold density for identifying regions of density. Applied after normalizing the density in the region of the molecule to an rms of 1 and mean of zero.
  • starting_density_threshold = None Optional guess of threshold density
  • iteration_fraction = 0.2 On iteration of finding regions, assume target volume is this fraction of the value on previous iteration
  • max_overlap_fraction = 0.05 Maximum fractional overlap allowed to density in another asymmetric unit. Definition of a bad region.
  • remove_bad_regions_percent = 1 Remove the worst regions that are part of more than one NCS asymmetric unit, up to remove_bad_regions_percent of the total
  • require_complete = True Require all NCS copies to be represented for a region
  • split_if_possible = True Split regions that are split in some NCS copies. If None, split if most copies are split.
  • write_all_regions = False Write all regions to ccp4 map files.
  • max_per_au = None Maximum number of regions to be kept in the NCS asymmetric unit
  • max_per_au_ratio = 5. Maximum ratio of number of regions to be kept in the NCS asymmetric unit to those expected
  • min_ratio_of_ncs_copy_to_first = 0.5 Minimum ratio of the last ncs_copy region size to maximum
  • min_ratio = 0.1 Minimum ratio of region size to maximum to keep it
  • max_ratio_to_target = 3 Maximum ratio of grid points in top region to target
  • min_ratio_to_target = 0.3 Minimum ratio of grid points in top region to target
  • min_volume = 10 Minimum region size to consider (in grid points)
  • residues_per_region = 50 Target number of residues per region
  • seeds_to_try = 10 Number of regions to try as centers
  • iterate_with_remainder = True Iterate looking for regions based on remainder from first analysis
  • weight_rad_gyr = 0.1 Weight on radius of gyration of group of regions in NCS AU relative to weight on closeness to neighbors. Normalized to largest cell dimension with weight=weight_rad_gyr*300/cell_max
  • expand_size = None Grid points to expand size of regions when excluding for next round. If None, set to approx number of grid points to get expand_target below
  • expand_target = 1.5 Target expansion of regions (A)
  • mask_additional_expand_size = 1 Mask expansion in addition to expand_size for final map
  • exclude_points_in_ncs_copies = True Exclude points that are in NCS copies when creating NCS au. Does not apply if add_neighbors=True
  • add_neighbors = True Add neighboring regions around the NCS au. Turns off exclude_points_in_ncs_copies also.
  • add_neighbors_dist = 1. Max increase in radius of gyration by adding region to keep it.
• control
  • verbose = False Verbose output
  • shift_only = None Shift map and half_maps and stop
  • sharpen_only = None Sharpen map and stop
  • check_ncs = None Check the NCS symmetry by estimating NCS correlation and stop
  • resolve_size = None Size of resolve to use.