Processing of Electron Diffraction Patterns acquired on a Slow-scan CCD Camera

Jaap Brink and Man Wei Tam*

W.M. Keck Center for Computational Biology and Verna and Marrs McLean Department of Biochemistry Baylor College of Medicine One Baylor Plaza Houston, TX 77030, U.S.A.

*Current address: Spear, Leeds & Kellog 99 Gresham St., London EC2 V7PH


We have developed two software packages for the processing of electron diffraction patterns acquired on a slow-scan CCD camera. Automated indexing and processing of these electron diffraction patterns is possible using a program called AUTO. To obtain a set of reliable reflections, AUTO applies a number of corrections, that are necessary due to the acquisition of the patterns with a slow-scan CCD camera. To index and process unknown lattices in electron diffraction patterns obtained from, for instance, a new protein or a new crystal form, an interactive program called EDP is available. It supports indexing in an interactive manner through a graphical user interface. Processing of the electron diffraction pattern is done using a script created by EDP. Both programs run on UNIX platforms, with EDP requiring the X Window System.


A high resolution structure determination by electron crystallography requires the knowledge of both amplitudes and phases. In the case of two-dimensional (2D-) crystals the amplitudes can be readily obtained from electron diffraction patterns. A slow-scan CCD camera has been shown to be an excellent medium to acquire these patterns at a very low dose, as compared to recording them on photographic film (Brink and Chiu, 1994). Processing of electron diffraction patterns typically entails: (i) indexing of the reciprocal lattice, (ii) generating the lattice parameters, (iii) refinement of these lattice parameters, and (iv) extraction of the intensities of all reflections using the refined parameters (Baldwin and Henderson, 1984). CCD data requires additional steps due to artifacts present in the patterns. These are a pronounced spike, which runs through the central beam in the pattern, as well as the presence of high intensity-pixels caused by x-ray photons (Brink and Chiu, 1994). The spike is the result of the intense, undiffracted beam hitting a few pixels on the chip. A lot of charge is generated at these pixels, which on CCD chips that are not equipped with special anti-blooming detectors will overflow into adjacent pixels - a process also referred to as 'blooming'. The direction of the overflow reflects the principal manner in which the chip is manufactured. Though the length of the spike can be reduced by summing several patterns each acquired at a relatively short exposure (Brink et al., 1995), a correction for it is still necessary. We developed a program called AUTO, which processes an electron diffraction pattern automatically, which includes the application of both corrections. Automated processing of electron diffraction data is desirable when large amounts of data require processing, which would become a reality with the wider-spread use of slow-scan CCD cameras. Automated processing is in fact very similar to the type of processing done in X-ray crystallography. However, automated processing requires a certain familiarity with the diffraction data. In the case of a new crystal form, or when diffraction data of a new protein is available, the initial processing is preferably done interactively. For this purpose, a program with a graphical user interface, called EDP, was developed.


Electron diffraction patterns of glucose-embedded crotoxin complex crystals are acquired using a Gatan (model 679) 1024 x 1024 slow-scan CCD camera, which is attached to a JEOL4000EX intermediate voltage electron cryomicroscope and interfaced to a Macintosh IIfx (Brink and Chiu, 1994). The patterns are initially stored in the Gatan DigitalMicrograph format (Mooney et al., 1990). Conversion to the MRC file format[a] is done using CONVGATAN (Brink and Chiu, 1994). Elsewhere, we have demonstrated that more than 100 electron diffraction patterns could be acquired from a single glucose-embedded crotoxin complex crystal kept at -167oC under constraints of radiation damage and statistical definition (Brink and Chiu, 1994). In the case of 2D-crystals, these patterns could be acquired as part of a tilt series (Brink and Chiu, 1994). Particularly when the crystals display a variable thickness between them, as was found for crotoxin complex and gp32*I (Grant et al., 1991; Jeng and Chiu, 1983), one needs a large number of electron diffraction patterns to sort out the crystals in terms of their thickness using just the electron diffraction intensities. Automated processing of the diffraction patterns is then highly desirable. Additionally, it allows the user to plan experiments that otherwise might not be done. For instance, one may want to determine the maximum number of electron diffraction patterns that can be obtained from a single crystal without significant damage to the structure (Brink and Chiu, 1994). Alternatively, the electron dose could be determined, which yields the optimal statistical definition of the patterns (Brink and Chiu, 1994). Quick radiation damage experiment are also possible, which reveal the optimal electron dose for high resolution imaging (Hayward and Stroud, 1981). In all these examples AUTO can be useful, since it allows the user to process and interpret the required electron diffraction results quickly.


General description of AUTO

Fig. 1. Flow diagram showing the steps involved in processing of an electron diffraction pattern acquired on a slow-scan CCD camera by the program AUTO. All forms of data, either binary (images) or ASCII (listings), are indicated by rectangles, whereas programs are indicated by diamonds. Coordinates of the central beam (cx, cy) and the lattice vectors (ax, ay, bx, by) are obtained after processing the original pattern by AUTOCENT, RBS, AUTOZERO, FIX_BAD_PIXELS and AUTOINDEX. GINTEN uses these coordinates to extract the intensities of all reflections in the original pattern. CHECK_SPOTS is the final filtering step in this procedure, that produces a list of integrated intensities.

AUTO will process an electron diffraction pattern acquired on a slow-scan CCD camera using 1024 x 1024 pixels in a fully automated manner in 2 to 3 min. on a Silicon Graphics Indigo workstation with an R4000 processor. In the case of crotoxin complex intensities of approximately 300 reflections are extracted up to 3 Å resolution. A total of 8 Mb of memory is needed for processing an electron diffraction pattern. To improve the accuracy of the radial background subtraction, the patterns - stored as 2-byte integers - are processed as 4-byte floating-point data. AUTO calls seven different programs written in FORTRAN 77, which themselves are embedded in small C-shell scripts (Fig. 1). These programs are AUTOCENT, RBS, AUTOZERO, FIX_BAD_PIXELS, AUTOINDEX, GINTEN, and CHECK_SPOTS. Both RBS and GINTEN are derived from MRC-programs, which were ported to the UNIX environment by others (Baldwin and Henderson, 1984; Prasad et al., 1990). AUTOCENT and AUTOINDEX were modified from original MRC programs to work within AUTO and to correctly process the CCD data. Since no formal mechanism exists to pass results from one FORTRAN program to the next, the relevant information is written into the extra fields of the header of the MRC file. For instance, AUTOCENT determines the coordinates of the central undiffracted beam and writes them into the header. RBS will extract these coordinates from the header, and use them in the radial background subtraction. To optimize disk space, all input and output images are compressed using a modified Lempel-Ziv algorithm (Welch, 1984). Electron diffraction patterns of crotoxin complex are thus reduced by approximately 95%. Execution of AUTO ultimately yields a list of integrated intensities of the reflections present in the processed electron diffraction pattern.

Usage of AUTO

AUTO is called from the command line requiring seven arguments for processing, five of which are the filename with the diffraction pattern, the threshold used to search for reflections in AUTOINDEX, the type of Ewald sphere curvature correction and two raster values used by GINTEN for the background assessment and intensity integration. The remaining arguments are a text identifier and the maximum radius out to which AUTOINDEX and GINTEN should process the pattern. Optional arguments are the approximate coordinates for the position of the central beam, and whether or not to save the intermediate background-subtracted, spike-removed and x-ray pixel--filtered diffraction pattern. AUTOCENT will determine the position of the central beam in the electron diffraction pattern initially based upon a center-of-gravity calculation performed on the whole pattern, followed by a refinement using local intensity gradients. The optional command-line supplied coordinates allow AUTOCENT to skip the computationally intensive center-of-gravity calculation and to start directly with the refinement. RBS performs a radial background subtraction using the coordinates of the central beam as stored in the header by AUTOCENT (Fig. 2A).

Fig. 2. Electron diffraction patterns acquired on the Gatan (model 679) 1024 x 1024 slow-scan CCD camera at intermediate stages of processing by AUTO. The patterns were obtained from a glucose-embedded crotoxin complex crystal tilted to 45deg. and acquired at -167deg.C using 400 keV electrons in a JEOL4000EX intermediate voltage electron cryomicroscope. The electron dose for this pattern was 0.08 e/Å2. (A) Pattern shown after determination of the position of the central beam by AUTOCENT and subsequent background subtraction by RBS. (B) Same pattern, now after removal of the spike by AUTOZERO. Lattice vectors are indicated in (A). The untilted lattice vectors are a = b = 38.8 Å with = 90o.

AUTOZERO removes the spike in an iterative way starting from the position of the central beam, again as read from the header. Pixels in the spike are set to zero using a threshold criterion applied to pairs of pixels separated by 1 pixel initially. On each next iteration this separation is increased until finally no new pixels are found by the program (Fig. 2B). Erased pixels are written to disk and used later by CHECK_SPOTS to filter out reflections that are too close to the spike. Before the pattern can be indexed by AUTOINDEX, all pixels associated with x-ray photons are removed using FIX_BAD_PIXELS. Hard x-ray pixels with an intensity around 4000 counts are removed using a fixed threshold, whereas soft x-ray pixels with an intensity around 100 counts are removed after a statistical evaluation of the immediate area around each suspected soft x-ray pixel. For comparison, the strongest reflections in patterns acquired from glucose-embedded crotoxin complex crystals would contain 100 counts per pixel. Removal of x-ray pixels is accomplished by replacing each of these pixels with the mean intensity of the 8 neighboring pixels (Brink and Chiu, 1994). Additionally, a single chip defect specific to our camera is corrected at this point by reading its location of a list and treating the defective pixel as an x-ray pixel. Multiple defects of this sort can easily be handled in this way. The radial background subtracted, spike-removed and x-ray pixel--filtered electron diffraction pattern, as shown in figure 2B, is next processed by AUTOINDEX. This program essentially computes a real-space auto correlation function of the pattern to obtain the lattice vectors. The determined lattice vectors are added to the coordinates of the central beam stored in the MRC header of the original file. GINTEN refines the lattice using the center of gravity positions of all reflections in the original file while accounting for the Ewald sphere curvature and projector lens distortions. The list with integrated intensities for all the reflections created by GINTEN, is processed by CHECK_SPOTS. This program acts as a filter by removing all suspect reflections using an intensity and a sigma threshold, as well as a distance constraint of the reflections with respect to pixels in the spike. Omitting this final step results in R-factors related to Friedel symmetry of around 90%, which is typically caused by a handful of reflections lying very close to the spike and therefore having very large integrated intensities. Filtering by CHECK_SPOTS removes these reflections and consequently yields reliable R-factors, which typically range from 10% to 20%. These R-factors have been observed for well over 1000 electron diffraction patterns acquired from glucose-embedded crotoxin complex crystals (Brink and Chiu, 1994).

General description of EDP

Electron diffraction patterns of a new protein crystal or crystal form are best indexed and processed using EDP until enough familiarity is obtained with the patterns to process them using AUTO. EDP has been designed starting from the indexing program used for Fourier transforms, SPECTRA (Schmid et al., 1993). EDP is written in ANSI-C using the MOTIF widget set and runs under the X Window System[b]. The program has been developed on Silicon Graphics Indigo workstations under IRIX version 5.3. Therefore, it should be relatively straightforward to port the software to other UNIX operating systems also running under X. EDP consists of a graphical user interface to several windows, that allows the user to quickly and interactively index the lattice of an electron diffraction pattern, and to write out a script for the non-interactive processing of that pattern. EDP's main window consists of a viewport, five control buttons as well as two panels, that are used to display results of the lattice indexing and subsequent refinement (Fig. 3). Rescaling the EDP main window will also rescale the viewport. On a workstation's console, for instance, the user can therefore view most of a diffraction pattern of 1024 x 1024 pixels without averaging.

Electron diffraction patterns are opened by EDP using a standard X Window System-file browser pulled down from the menu item 'File'. EDP displays the MRC file using low and high cut-off values for the display derived from the minimum, the maximum and the mean intensity read from the header portion of the MRC file (Fig. 3).

Fig. 3. EDP's main window with an electron diffraction pattern shown inside the viewport after indexing and lattice refinement. The pattern is identical to that shown in Fig. 2. Red boxes indicate the reflections used to triangulate the lattice; blue boxes are drawn around each reflection indicating the correctness of indexing the pattern which extends to 3 Å resolution. Yellow boxes indicate the main lattice directions. Additionally, four other windows are shown. Top right is the window labeled 'Point Definition Dialog', which is activated after selecting a reflection using the mouse. Below it is the 'Image Information' window activated by the EDP VARS button. Bottom right shows the window activated by the buttons EDIT SCRIPT and WRITE SCRIPT. Bottom left shows the 'Image Control Panel' window activated by the EDP INFO button.

The five control buttons of the EDP main window activate input and output windows (Fig. 3). For instance, the button EDP INFO will bring up the 'Image Information' window, which shows actual intensities of pixels in the displayed electron diffraction pattern and their coordinates as the cursor is moved across the pattern. The button EDP VARS will bring up the 'Image Control Panel' window, which offers control over several features inside EDP. For instance, this window has the controls for the low- and high-cut values used for display. Also, the magnification of the pattern displayed inside the viewport can be set, which allows the user to zoom into an area if so desired. When the image dimensions on the screen exceed the viewport dimensions, the pattern can be scrolled inside the viewport. The search radius parameter shown in this window is used by EDP to define a circular area centered on a selected reflection over which the program will search for the highest intensity-pixel. A bilinear interpolation over this pixel and its neighboring pixels results in sub-pixel accuracy for the reflection's position.

Usage of EDP

Indexing of the electron diffraction pattern is done by selecting three reflections (Fig. 3). The best lattice is obtained, when these reflections are separated from each other by the largest number of pixels. This triangulation approach was found to be adequate for obtaining an initial set of lattice parameters. The subsequent non-interactive processing is done by GINTEN, which comprises a further refinement of the lattice. Assigning the Miller indices h and k to the selected reflection is done inside the 'Point Definition Dialog' window (Fig. 3). The determined coordinate with the highest intensity is shown. The selected reflections are also displayed in the top results portion of the EDP main window together with their positions. The button REFINE LATTICE on the EDP main window will activate the lattice refinement algorithm, which uses the refined positions of the three reflections. The user can visually judge the indexing from the proper placement of the boxes around each reflection (Fig. 3). The size of the boxes can be reduced by specifying a smaller 'Search Radius', allowing the user to judge more rigorously the fit of the computed lattice. The numerical result of the lattice refinement, i.e. the lattice parameters, is displayed in the EDP main window.

The newly indexed electron diffraction pattern can be processed in a non-interactive way using a script created by EDP. The buttons EDIT SCRIPT and WRITE SCRIPT on the EDP main window will activate a window labeled 'Script Parameters' (Fig. 3). This window offers control over the parameters used by the programs, which perform the non-interactive processing, namely RBS and GINTEN. Two environmental variables read by EDP at start-up ensure that the proper names for these executables are used in the script. Most of the values in the script are loaded at start-up from a file, called '.edprc', residing in the user's home directory. The lattice parameters, however, determined by EDP are directly written into the processing script. The untilted lattice parameters on the other hand, are loaded from file; they are used by GINTEN to compute the tilt parameters for each electron diffraction pattern according to the formulation by Shaw and Hills (Shaw and Hills, 1981).


Currently, EDP does not correct the CCD data as AUTO does. This is considered for a future version of EDP. Also, some of the programs called by AUTO could generate a background-subtracted, spike-erased and x-ray pixel--filtered electron diffraction pattern, which is subsequently indexed using EDP instead of AUTOINDEX. For new crystal forms or new crystals with rather weak diffraction this would facilitate proper indexing.


We have discussed two software tools for the processing of electron diffraction patterns acquired using a slow-scan CCD camera. AUTO allows for the automated processing, whereas EDP supports indexing using a graphical user interface combined with non-interactive processing. Electron crystallography has resulted in reconstructions of biological macromolecules to atomic resolution (Henderson et al., 1990; Jap et al., 1991; Kühlbrandt et al., 1994). For 2D-crystals processing of electron diffraction patterns would benefit from use of these tools. However, they might also be applied to helical or icosahedral specimens. For instance, high resolution electron diffraction amplitudes can be obtained from helical systems, as was shown for instance on rafts obtained from TMV and flagella (Jeng et al., 1989; Ruiz et al., 1994). Also, incoherent sums of images of single helical objects or virus particles might yield amplitudes at the desired resolution (Jeng, et al., 1989; Rey et al., 1995; Schmid et al., 1993). AUTO and EDP are available as source code via the World Wide Web using the URL


This work is supported by grants from the National Institutes of Health (RR02250), the National Science Foundation (NSFBIR-9413229) and the W.M. Keck Foundation. We acknowledge valuable discussions with Drs. W. Chiu, M.F. Schmid and M.B. Sherman, as well as Ms. P. Thuman-Commike. We express our sincere gratitude to Dr. R. Henderson for making available some of the programs discussed here. We are indebted to Dr. M.F. Schmid and Mr. R. Dargahi for using the program framework of SPECTRA for the development of EDP.


Baldwin, J. and Henderson, R. (1984) Measurement and evaluation of electron diffraction patterns from two-dimensional crystals., Ultramicroscopy 14, 319-336.

Brink, J. and Chiu, W. (1994) Applications of a slow-scan CCD camera in protein electron crystallography, J. Struct. Biol. 113, 23-34.

Brink, J., Sherman, M. B. and Chiu, W. (1995) Application of a slow-scan CCD camera in protein electron crystallography at 400 kv. in Proc. 53nd Ann. Mtg. Microsc. Soc. Am., Kansas City, MI, in press, San Francisco Press, San Francisco.

Grant, R. A., Schmid, M. F. and Chiu, W. (1991) Analysis of symmetry and three-dimensional reconstruction of thin gp32*I crystals, J. Mol. Biol. 217, 551-562.

Hayward, S. B. and Stroud, R. M. (1981) Projected Structure of Purple Membrane Determined to 3.7 Å Resolution by Low Temperature Electron Microscopy, J. Mol. Biol. 151, 491-517.

Henderson, R., Baldwin, J. M., Ceska, T. A., Zemlin, F., Beckmann, E. and Downing, K. H. (1990) Model for the structure of bacteriorhodopsin based on high-resolution electron cryo-microscopy, J. Mol. Biol. 213, 899-929.

Jap, B. K., Walian, P. J. and Gehring, K. (1991) Structural architecture of an outer membrane channel as determined by electron crystallography, Nature 350, 167-170.

Jeng, T. W. and Chiu, W. (1983) Low dose electron microscopy of the crotoxin complex thin crystal., J. Mol. Biol. 164, 329-346.

Jeng, T. W., Crowther, R. A., Stubbs, G. and Chiu, W. (1989) Visualization of alpha-helices in tobacco mosaic virus by cryo-electron microscopy, J. Mol. Biol. 205, 251-257.

Kühlbrandt, W., Wang, D. N. and Fujiyoshi, Y. (1994) Atomic model of plant light-harvesting complex, Nature 367, 614-621.

Mooney, P. E., Fan, G. Y., Truong, K. V., Bui, D. B. and Krivanek, O. L. (1990) Slow-scan CCD camera for transmission electron microscopy. in Proc. XIIth Int. Cong. Electron Microsc., Seattle, WA, pp. 164-165, San Francisco Press, San Francisco.

Prasad, B. V. V., Degn, L. L., Jeng, T.-W. and Chiu, W. (1990) Estimation of allowable errors for tilt parameter determination in protein electron crystallography, Ultramicroscopy 33, 281-285.

Rey, F. A., Heinz, F. X., Mandl, C., Kunz, C. and Harrison, S. C. (1995) The envelope glycoprotein from tick-borne encephalitis virus at 2 Å resolution, Nature 375, 291-298.

Ruiz, T., Ranck, J.-L., Diaz-Avalos, R., Caspar, D. L. D. and DeRosier, D. J. (1994) Electron diffraction of helical particles, Ultramicroscopy 55, 383-395.

Schmid, M. F., Dargahi, R. and Tam, M. W. (1993) SPECTRA: a system for processing electron images of crystals, Ultramicroscopy 48, 251-264.

Shaw, P. J. and Hills, G. J. (1981) Tilted specimen in the electron microscope: A simple specimen holder and the calculation of tilt angles for crystalline specimens, Micron 12, 279-282.

Welch, T. A. (1984) A Technique for High Performance Data Compression, IEEE Computer 17, 8-19.