CCP4i: Graphical User Interface
	Reflection Data Utilities Module

Tasks in this module:

Analyse Data Quality (TRUNCATE)

Calculate Fs & Phases (SFALL)

SFALL - Task Window Layout

Convert to/modify/extend MTZ (Import)

Import - Project/Crystal/Dataset Names

Import - Data Harvesting in TRUNCATE

Import - Task Window Layout

Import - The Import Task

Import - The Import File Format

Import - NA4toMTZ

Import - CIF2MTZ

Import - The MTZ Column Types

Convert from MTZ (Export)

Export - Task Window Layout

Edit MTZ File (SFTOOLS)

SFTOOLS - Task Window Layout

Merge MTZ Files (CAD)

CAD - Task Window Layout

Edit MTZ Datasets

Reindex Reflections - Reindex

SF File Analysis (SFTOOLS)

Phase Analysis (Phistats)

Sigma-A

Convert FoM to/from HL

Phase Comparison

Clipper Reflection Utilities

Calculate E's

Combine Phases

Map to Structure Factors

Model to Structure Factors

Calculate Sigma-A Weights

Specialist Help is available on:

FreerUnique - how to create a complete unique list of reflections with your data

The layout of each task window, i.e. the number of folders present, and whether these folders are open or closed by default, depends on the choices made in the Protocol folder of the task (see Introduction). Although certain folders are closed by default, there are specific reasons why you should or may want to look at them. These reasons are described in the Task Window Layout sections below.

Analyse Data Quality - Truncate

The program Truncate is used to obtain structure factor amplitudes from intensities, however it can performs useful analyses on the reflection data, including graphs of anisotropy, cumulative intensity statistics and indications of twinning.

See program documentation: CTruncate, Truncate

Calculate Fs & Phases - SFALL

This is an interface to the SFALL program which generates Fcalc and PHIcalc from either input coordinates or a map. You must give the name of an input coordinate or map file and also an MTZ file into which the calculated Fs and PHIs are added. You will need to identify the FP and SIGFP in the MTZ file. If a FreeR column is also identified, the FreeR set of reflections will be excluded from the calculation. By default the calculated Fs and PHIs will cover the same resolution range as the one already in the MTZ file.

SFALL - Task Window Layout

The Task Window for the SFALL Task is relatively simple. The Protocol folder has the choice between generating structure factors from coordinates or a map (after which the Files folder is adapted accordingly), and the option to scale the input Fp to the Fcalc.

After the Protocol and Files folders, the 'Crystal Parameters' folder has the option of adapting resolution range, space group and/or cell parameters. The 'Program Parameters' folder allows the adaptation of Maximum atom radius used to build atom/residue mask and the amount to add to atomic Bfactors.

See program documentation: SFALL.

Convert to/modify/extend MTZ - Import

Standard CCP4 programs deal with reflection files in so-called MTZ format. Files in different formats, namely ascii MTZ ('na4'), X-PLOR/CNS, SHELX, MULTAN, TNT, mmCIF and user defined ascii, may be converted to MTZ format. Additionally, MTZ files may be extended to create a full unique set of reflections with various options to deal with freeRflags. The default setting is to create this set. At the start of a structure determination, it is advisable to always create this full unique set of reflections, which contains (h, k, l, F, SIGF, (I, SIGI), ..., FreeR_flag) for all observed reflections to the resolution limit available, plus entries for any unobserved reflection, all with freeRflags assigned. This never does any harm, and it is much harder to do satisfactorily at a later stage. For more information about a full unique set with freeRs, see also FreerUnique.

Import - Project/Crystal/Dataset Names

When importing data with F2MTZ, and creating a new MTZ file, it is important to set sensible names for the Crystal, Project and Dataset. These will be used to set a data structure for reflection information, and also to control Data Harvesting (see the Data Harvesting page for background details). The crystal name should identify the physical crystal used, and the dataset should identify a dataset taken from a physical crystal (e.g. a MAD experiment may take 3 datasets from a crystal at different wavelengths). The project may correspond to the CCP4i project, but does not have to. Input these in the 'MTZ Project, Crystal & Dataset Names' folder directly beneath the Files folder.

Import - Data Harvesting in TRUNCATE

TRUNCATE, part of the Import Task when SHELX or MULTAN data is imported, is needed to convert intensities to structure factors. This is a genuine call for data harvesting, which is why the first folder beneath the Protocol and Files folders is concerned with 'Data Harvesting'. The user has the choice whether to allow data harvesting or not, but it is, of course, recommended. The user should also set Crystal, Project and Dataset Names, as above, which will also be used to control harvesting from Truncate.

See Data Harvesting in CCP4i for implications for the Interface.

Import - Task Window Layout

Features to look out for in the Convert to MTZ & Standardise Task are:

Import - The Import Task

The full "import" (i.e. Convert to MTZ & Standardise) task contains the following steps:

1. F2MTZ - to convert any ASCII style reflection file to MTZ format. If no full unique set of reflections with freeRflags is required, this will be the only step (but this is not the default). For "ascii MTZ" and mmCIF, the specialised programs NA4TOMTZ and CIF2MTZ are used instead.

2. unique - to create a full unique list of reflections (indices) with dummy F and SIGF, to the required resolution. This file is then used as a template list in the following steps.

   Then, depending on whether there were freeRflags already in your data or not, i.e. whether you chose to

   1. keep existing FreeR data
   2. import FreeR data from another MTZ file
   3. generate FreeR data

   one of the following happens:

3. If there were freeRflags already in your data set:

   • CAD - to combine your data (from step 1, i.e. option a, or from an existing MTZ file, i.e. option b) with the unique template list (from step 2), sort it to standard order, and place it in the CCP4 asymmetric unit. Reflections not featuring in your data set will acquire a "missing number flag" ("NaN" by default), which can be replaced with real data as and when it becomes available.

     N.B. CAD can not deal with more than 29 columns.

   • freerflag - in COMPLETE mode, to add a random freeRflag to all reflections which do not already have one.

   • mtzutils - to remove superfluous columns (created by step 2) from the file.

4. If freeRflags are to be assigned from scratch for all reflections (i.e. option c):

   • freerflag - to add a random freeRflag to all reflections in the file created in step 2, according to the chosen fraction.

   • CAD - as in step 3, using all columns from your data, but using only the freeRflag column from the unique file from step 2, so that the superfluous columns are automatically removed at this stage.

   • freerflag - as in step 3, to ensure freeRflags are assigned to every reflection.

Import - The Import File Format

Pay special attention to the folder Detailed specification of import file format:

• The program F2MTZ requires all values to be described as real numbers, i.e. Fortran format type F. The Interface has the default Fortran format for all types of reflection files, but this may have to be changed if a non-standard reflection file is used or if more or fewer columns are to be included. This is an example of a Fortran format as it should be entered:
  

  (5X,3F5.0,6X,2F10.0,7X,F10.0,/,10X,2F10.0,8X,F10.0,6X,F10.0)

  This complicated statement describes the following reflection file format:
  

  INDE     6    0    0 FOBS=  1259.884     0.000 SIGMA=    38.561
      FPART=     0.000     0.000 WEIGHT=     1.000 TEST=         0

  • For more information on Fortran formats, see a Fortran Manual.
  • TNT input is split into two files (working and test sets separately). The Interface will automatically merge those files and insert freeRflags but can only do so if you add F8.0 to the format statement.
• If there are header lines in the reflection file not containing actual data, these lines should be skipped. The program will not run if the SKIP value is incorrect.
  • If the input is from TNT, the merging of the files can only happen successfully if the headers of the two input files contain the same number of lines, i.e. the SKIP value is the same for both files.
• The output data type and label must be given correctly for every column of data in the reflection file.
  • For X-PLOR input, the Interface is assuming freeRflags are already assigned, and will initiate the necessary swap of the values (see FreerUnique) by denoting the column type X for the freeRflag column.
  • If the input is from SHELX, the Intensities have to be converted to Fs. The column labels for the program Truncate, which is to perform the conversion, need to match those of the input file exactly.

Import - NA4toMTZ

The CCP4 Suite of programs is released with a set of example files. MTZ files are distributed in 'na4' format, which is a type of ASCII. In order to convert these to the MTZ format needed to run the programs, the program NA4toMTZ is provided. In the Interface, this is included within the 'F2MTZ' part of the 'Convert to MTZ & Standardise' task.

When browsing for an 'na4' file, do not forget to change the file extension to '.na4'.

Import - CIF2MTZ

Structure factor files obtained from the PDB will be in mmCIF format. This option can be used to convert these files to MTZ format. With a well-formed mmCIF file, you should not need to change the defaults or add any information.

Import - The MTZ Column Types

H

index h,k,l

F

structure amplitude, F

G

structure amplitude associated with F(+) or F(-)

L

standard deviation of column of type G

D

anomalous difference

Q

standard deviation of anything: J,F,D or other

P

phase angle in degrees

W

weight (of some sort)

A

phase probability coefficients (Hendrickson/Lattman)

B

BATCH number

These data types are not usually present, but are legal:

S

1/d**2 = 4 (sin theta/lambda)**2

J

intensity

K

intensity associated with I(+) or I(-)

M

standard deviation of column of type K

E

normalised structure factor

Y

M/ISYM, packed partial/reject flag and symmetry number

I

any other integer

R

any other real

U

partial FC (see CAD)

V

partial PHIC (see CAD)

In addition, the program F2MTZ accepts the column type X for freeRflags originating from X-PLOR/CNS, and will reset the column type to I in the resulting MTZ file (see FreerUnique).

N.B. The column types of structure amplitudes and phases may not be any other than F and P, respectively. It is also essential to have correct column types for phases and anomalous differences, to distinguish:

• phases which will require changing if the reflection is moved to a symmetry equivalent;
• anomalous differences which require changing sign if the reflection is changed to a Friedel partner.

See program documentation: F2MTZ, NA4toMTZ, CIF2MTZ, UNIQUE, FREERFLAG, CAD, MTZUTILS, MTZLIB

Convert from MTZ - Export

MTZ files may be converted to several other formats, i.e. X-PLOR/CNS, MULTAN/RANTAN, SHELX, TNT, MAIN, SCALEPACK, USER and mmCIF. To output an mmCIF file for deposition, use the task interface "Structure Factors for Deposition" in the Validation & Deposition module instead.

Export - Task Window Layout

Features to look out for in the Convert from MTZ Task are:

See program documentation: MTZ2VARIOUS, MTZDUMP,MTZDMP, MTZTONA4,NA4TOMTZ, MTZUTILS

For more information on mmCIF, see The mmCIF Home Page at the IUCr.

Edit MTZ File (SFTOOLS)

Analyse, manipulate and convert structure factor files.

SFTOOLS - Task Window Layout

Features to look out for in the Edit MTZ File Task are:

See program documentation: SFTOOLS, MTZUTILS, REINDEX, CAD

Merge MTZ Files (CAD)

This task interfaces to the CAD program which can be used to:

1. Delete columns from an MTZ file
2. Merge data from two or more MTZ files
3. Reset the resolution range in the MTZ file(s)
4. Change the sort order or HKL limits or space group

To input more than one MTZ file, click on the Add input MTZ file button. By default all the data in the input MTZ file is put into the output file but you can change the Input option from 'all columns' to 'selected columns' and then select the columns using the Add column button. If you want to have the majority of the columns in the file, then click on the List All Columns button and then delete the columns you do not require using the Delete selected item option under the Edit list menu button. You will then need to select the column by clicking on one of the fields for that column with the right mouse button. See also Extending Frames and Toggle Frames.

CAD can not deal with more than 29 columns.

Do not include columns H, K and L in input. These are transferred to output automatically, and only upset the program.

Two special data types are used to signal that you are preparing data for translation functions of various types. They are:

U

partial FC

V

partial PHIC

There must be only one FCpart PHICpart per input file, and they must be the last items specified for LABIN. CAD generates equivalent reflections using only the ROTATIONAL part of the primitive symmetry operator (i.e. if the spacegroup is P212121, these reflections are analysed as though the spacegroup was P222). This is allowed for in the TFFC and RSEARCH programs.

CAD - Task Window Layout

Features to look out for in the CAD Task are:

See program documentation: CAD, SFTOOLS

Edit MTZ Datasets

The columns of an MTZ file are divided between a number of datasets, which are each identified by a Crystal name (XNAME) and a Dataset name (DNAME). Crystals in turn belong to a Project (PNAME). See the description of Data Harvesting for background information. The program CAD can be used to change the attributes of a dataset, and to change the assignment of columns to datasets, and this functionality is accessible through the Edit MTZ Datasets interface.

The top half of the interface lists the current datasets and their attributes. Any attribute can be edited, and datasets may be added or deleted. The lower half provides a pull-down menu for each column allowing the user to select a dataset from the current list. Note that the input to CAD is taken from the list of columns, so that surplus datasets are ignored, and datasets deleted from the dataset list but retained in the column list are kept.

In principle, dataset information should be generated and propagated automatically, so this interface should only be needed to fix problems.

See Data Harvesting in CCP4i for implications for the Interface.

See program documentation: CAD

Reindex Reflections - Reindex

This is the option to use for reindexing 'standard' MTZ files, i.e. non-multi-record MTZ.

See program documentation: REINDEX

SF File Analysis (SFTOOLS)

This is an interface to the data analysis tools of SFTOOLS which will perform several forms of analysis of Structure Factor data. The user should select an MTZ file and the data columns to analyse. The results are displayed automatically by The Loggraph Utility. The results are also saved in a .log file which can subsequently be viewed with the View Log Graphs option in the View Files from Job menu on the right hand side of the Main Window of CCP4i.

See program documentation: SFTOOLS

Phase Analysis (Phistats)

This task will take an MTZ file with two phase sets and analyse the differences between them.

See program documentation: PHISTATS

Sigma-A

See program documentation: SIGMAA

Convert FoM to/from HL

Run the Clipper utility HLTOFOM to convert between Hendrickson-Lattman (HL) coefficients and phase/figure-of-merit (Phi/FoM) representations. This can be useful if data is in one representation but a particular task requires that data in the other.

Phase Comparison

Perform analysis and comparison of two phase sets. This task allows the comparison of phase sets in different representations: map coefficients (amplitude and phase angles), Hendrickson-Lattman coefficients, or phase and weight (phase angle plus figure-of-merit).

Calculate E's

Run the Clipper CECALC utility to generate normalised structure factor amplitudes from reflection data in an MTZ file.

Combine Phases

Combine two phase sets both expressed as Hendrickson-Lattman coefficients.

Map to Structure Factors

Use the Clipper CINVFFT program to generate structure factors from an existing map.

Model to Structure Factors

Use the Clipper CSFCALC program to generate structure factors from an existing model (set of atomic coordinates).

Calculate Sigma-A Weights

Use the Clipper CSIGMAA program to generate sigma-A weights structure factors calculated structure factors.

Valid XHTML 1.0! Valid CSS!