Computational CryoEM Methods

This repository is a curated list for computational cryo-EM methods mainly target for single particle analysis!

You will find the paper and associated software for the popular algorithms used in the cryo-EM field.

The format for the item is [software link] - [Paper link] (Available in which package) and the headings may contain a ink to a review paper.

• Computational CryoEM Methods
• Resources
• Workflow
  • Motion correction
  • CTF estimation
  • Particle picking
  • 2D classification
  • Ab-initial Model
  • 3D refinement
  • 3D variability analysis
  • Postprocessing
• Conventions
• Tips
  • Format Conversion
  • Performing Focus Classification
  • Display images
  • Using Chimera
• Contributing

Resources

Preface

• Protein Folding Problem - Great video that describes the importance of understanding protein structure.
• What is cryoEM - Great video that describes how cryoEM can help us understanding the protein structure.

Introduction

• A quick introduction video for cryo-EM
• Cryo-EM workflow by Nvidia - Describe the blueprint of cryo-EM using GPU processing.
• A Primer to Single-Particle Cryo-Electron Microscopy - A great review to start with.
• Principles of cryo-EM single-particle image processing - Highlight the principles when dealing with a cryo-EM dataset.
• Single-particle cryo-electron microscopy: Mathematical theory, computational challenges, and opportunities - Review that covers recent updates.
• Computational Methods for Single-Particle Electron Cryomicroscopy - A comprehensive review for the mathematical background behind single-particle cryo-EM methods.
• Cryo-EM: Breakthroughs in Chemistry, Structural Biology, and Statistical Underpinnings - Review that covers statsitical problems behind cryo-EM.

Courses

• Cryo-EM Principles
• Getting Started in Cryo-EM
• Electron Microscopy Training by MRC LAB
• nysbc training course

Resources

• 3DEM Methods - A great wiki that collects papers or books for computational methods.
• A collective resource - A great repository that covers single particle analysis, model building and tomography.
• Math behind CryoEM - A great repository that collects the materials which elaborate the math behind single-particle analysis.
• Chimera Tutorial

Events and News

• CryoEM and AI reveal a structure of SARS-CoV-2 Nsp2, a multifunctional protein involved in key host processes
• 3DEM Events
• NRAMM Events

Software

• Scipion - An integrated platform that allows users to use a variety of methods in the same framework. [Documentation]
• Relion - A comprehensive package that utilizes Bayesian approach for 2D classification and 3D refinement. [Documentation]
• CryoSparc 2 - A general package that employs stochastic gradient descent, branch and bound as well as GPU acceleration for rapid reconstructions.
• Sphire - A general package that contains neural network methods for particle picking, denoising and classification.
• CisTEM - An easy-to-use framework that implements a complete pipeline for single-particle analysis.
• EMAN2 - A comprehensive package that contains a python interface and handy scripts for common tasks.
• SPIDER - A well-known package that implements image processing methods for electron microscopy.
• ASPIRE - A software that used algorithms based on rigorous mathematical theory.

Workflow

Import Data

1. From EMPIAR or EMDB.

3. From tutorial data set.

3. Generate simulation data
   • Discrete
   • Continuous
   • From CryoDrgn - Use write_starfile.py to convert to star format

Image Formation model

1. TEM Simulator - Simulation of transmission electron microscope images of biological specimens

2. InSilicoTEM - Image formation modeling in cryo-electron microscopy

3. CisTEM_Simulate - Cryo-TEM simulations of amorphous radiation-sensitive samples using multislice wave propagation

Motion correction

• Whole frame alignment

1. Unblur - Measuring the optimal exposure for single particle cryo-EM using a 2.6 Å reconstruction of rotavirus VP6 ( Use in CisTEM)
2. Full-frame motion correction - (Use in CryoSparc)
3. Optical Flow - Alignment of direct detection device micrographs using a robust Optical Flow approach (Use in Xmipp)

• Patch-based/Per-particle based alignment

1. MotionCorr2 - MotionCor2: anisotropic correction of beam-induced motion for improved cryo-electron microscopy (Use in Relion and CryoSparc)
2. Alignparts - Alignment of cryo-EM Movies of Individual Particles by Optimization of Image Translations (Use in CryoSparc(local, patch))
3. Warp - Contains Patch-based motion correction.
4. FlexAlign - FlexAlign: An Accurate and Fast Algorithm for Movie Alignment in Cryo-Electron Microscopy - (Use in Scipion)

• Damage compensation

  • This functionality is available in most of the motion correction tool
  • The dark/gain corrected is also conducted in this stage

CTF estimation

• Whole frame

1. CTFFIND4 - CTFFIND4: Fast and accurate defocus estimation from electron micrographs ( Use in CisTEM)
2. gCTF - Gctf: Real-time CTF determination and correction

• Patch-based/tilt data

1. Patch-Based CTF Estimation - Real-time cryo-electron microscopy data preprocessing with Warp ( Use in CryoSparc, Warp)
2. goCTF - goCTF: Geometrically optimized CTF determination for single-particle cryo-EM
3. Warp - Contains Patch-based CTF estimation.
4. novaCTF - Efficient 3D-CTF correction for cryo-electron tomography using NovaCTF improves subtomogram averaging resolution to 3.4 Å

• Denoising micrograph

1. Restore - Enhancing SNR and generating contrast for cryo-EM images with convolutional neural networks
2. Topaz - Topaz-Denoise: general deep denoising models for cryoEM
3. JANNI - Just Another Noise 2 Noise Implementation
4. Warp - Contains methods base on noise2noise and deconvolution filter

Particle picking

• Semi-supervised picking

1. Topaz - Positive-unlabeled convolutional neural networks for particle picking in cryo-electron micrographs. [Video]
2. Cryolo - SPHIRE-crYOLO is a fast and accurate fully automated particle picker for cryo-EM. [Video]
3. Xmipp - A pattern matching approach to the automatic selection of particles from low-contrast electron micrographs
4. EPicker - EPicker is an exemplar-based continual learning approach for knowledge accumulation in cryoEM particle picking

• Template-based picking

1. Relion, CryoSparc - Use 2D class averages or 3D projection for more accurate particle picking

• Automatic picking

1. DoG - DoG Picker and TiltPicker: software tools to facilitate particle selection in single particle electron microscopy
2. LoG - The Laplacian of Gaussian and Arbitrary Z-Crossings Approach Applied to Automated Single Particle Reconstruction (Use in Relion auto-picking)
3. APPLE - APPLE picker: Automatic particle picking, a low-effort cryo-EM framework (Use in ASPIRE auto-picking)
4. KLT picker - KLT picker: Particle picking using data-driven optimal templates

• Denoising particle and dimenison reduction

1. 2SDR - Two-stage dimension reduction for noisy high-dimensional images and application to Cryogenic Electron Microscopy
2. CWF - Denoising and covariance estimation of single particle cryo-EM images (Use in ASPIRE)
3. GAN - Generative adversarial networks as a tool to recover structural information from cryo-electron microscopy data

2D classification

• Multirefence alignment based classification

1. ISAC - Iterative Stable Alignment and Clustering of 2D Transmission Electron Microscope Images. [GPU version]
2. CL2D - A clustering approach to multireference alignment of single-particle projections in electron microscopy

• Maximum-likelihood based classification

1. Relion - Bayesian (Empirical Bayes) approach
2. CryoSparc - Bayesian with Branch and bound method
3. CisTEM - Maximum likelihood method
4. SubspaceEM - SubspaceEM: A fast maximum-a-posteriori algorithm for cryo-EM single particle reconstruction

• Mixed approach classification

6. ROME - Massively parallel unsupervised single-particle cryo-EM data clustering via statistical manifold learning

• Automatic selection of 2D classes

1. Cryoassess -High-Throughput Cryo-EM Enabled by User-Free Preprocessing Routines
2. Cinderella - Cinderella: Deep learning based binary classification tool

• New clustering methods

1. γ-SUP - γ-SUP: A clustering algorithm for cryo-electron microscopy images of asymmetric particles
2. Wasserstein-k-means - Wasserstein K-Means for Clustering Tomographic Projections

3D Tomographic Reconstruction

1. A Survey of the Use of Iterative Reconstruction Algorithms in Electron Microscopy
2. ASTRA

Ab-initial model

• Class-averages based method

1. Simple-Single-particle cryo-EM-Improved Ab Initio 3D Reconstruction With SIMPLE/PRIME

• Particles based method

1. CryoSparc - cryoSPARC: algorithms for rapid unsupervised cryo-EM structure determination. [Slides]

3D refinement

• 3D Homogeneous Refinemnet

1. Relion - RELION: Implementation of a Bayesian approach to cryo-EM structure determination. [Video]
2. CryoSparc - Use Expectation-Maximization with branch and bound method for higher resolution
3. OPUS-SSRI - Sparseness and Smoothness Regularized Imaging for improving the resolution of Cryo-EM single-particle reconstruction

• New 3D reconstruction paradigm

1. Pose Estimation with VAE-GAN-Estimation of Orientation and Camera Parameters from Cryo-Electron Microscopy Images with Variational Autoencoders and Generative Adversarial. [Related work]
2. CryoGAN - CryoGAN: A New Reconstruction Paradigm for Single-Particle Cryo-EM Via Deep Adversarial Learning
3. Orientation recovery with Siamese neural network - Learning to recover orientations from projections in single-particle cryo-EM
4. CryoAI - Amortized Inference of Poses for Ab Initio Reconstruction of 3D Molecular Volumes from Real Cryo-EM Images

• 3D classification

1. Relion, CryoSparc - Perturb the initial model and use projection matching with weighted assignment
2. LCTC - An Efficient Method to Quantify Structural Distributions in Heterogeneous cryo-EM Datasets

• 3D non-uniform Refinemnet

1. CryoSparc - Non-uniform refinement: Adaptive regularization improves single particle cryo-EM reconstruction
2. SideSplitter - Mitigating Local Over-fitting During Single Particle Reconstruction with SIDESPLITTER. [Video]

3D variability analysis

1. 3DVA - 3D Variability Analysis: Directly resolving continuous flexibility and discrete heterogeneity from single particle cryo-EM images ( Use in CryoSparc)
2. CryoDRGN - CryoDRGN: Reconstruction of heterogeneous structures from cryo-electron micrographs using neural networks (For processing with large dataset see here)
3. ManifoldEM - Retrieving functional pathways of biomolecules from single-particle snapshots
4. DMSA - Recovery of conformational continuum from single-particle cryo-EM data: Optimization of ManifoldEM informed by ground-truth studies
5. AlphaCryo4D - Deep manifold learning reveals hidden dynamics of proteasome autoregulation
6. BioEM - A Bayesian approach for extracting free energy profiles from cryo-electron microscopy experiments using a path collective variable
7. 3DFlex - 3D Flexible Refinement: Structure and Motion of Flexible Proteins from Cryo-EM ( Use in CryoSparc)
8. e2gmm - Deep learning based mixed-dimensional GMM for characterizing variability in CryoEM ( Use in EMAN2)
9. VAE - Inferring a Continuous Distribution of Atom Coordinates from Cryo-EM Images using VAEs
10. cryoFIRE - Amortized Inference for Heterogeneous Reconstruction in Cryo-EM
11. Atomic VAE - Heterogeneous reconstruction of deformable atomic models in Cryo-EM
12. Diffusion Prior - Latent Space Diffusion Models of Cryo-EM Structures

• Latent space analysis

1. Polaris - POLARIS: Path of Least Action Analysis on Energy Landscapes

• Focus classifcation

1. localrec - Localized reconstruction of subunits from electron cryomicroscopy images of macromolecular complexes (Use in Scipion)
2. Multi-body refinement - Characterisation of molecular motions in cryo-EM single-particle data by multi-body refinement in RELION (Use in Relion)

Postprocessing

• Per-particle based motion and ctf refinement

1. CTF refinement - Relion3/CisTEM/CryoSparc, 3D Reference required
2. Ewald sphere correction - New tools for automated high-resolution cryo-EM structure determination in RELION-3 (Use relion_reconstruct --reverse_curvature)
3. High-order aberrations - Estimation of high-order aberrations and anisotropic magnification from cryo-EM data sets in RELION-3.1
4. Bayesian Polishing - A Bayesian approach to beam-induced motion correction in cryo-EM single-particle analysis (Use in Relion, 3D Reference required)
5. M - Multi-particle cryo-EM refinement with M visualizes ribosome-antibiotic complex at 3.5 Å inside cells. [Video]

• Local resolution and directional resolution

1. Blocres - One number does not fit all: Mapping local variations in resolution in cryo-EM reconstructions
2. ResMap - Quantifying the Local Resolution of Cryo-EM Density Maps
3. MonoRes - MonoRes: Automatic and Accurate Estimation of Local Resolution for Electron Microscopy Maps (Use in Scipion)
4. 3DFSC - Addressing preferred specimen orientation in single-particle cryo-EM through tilting
5. MonoDir - Measuring local-directional resolution and local anisotropy in cryo-EM maps (Use in Scipion)

• Sharpening and local filtering to enhance interpretability

1. Confidence Maps - Thresholding of cryo-EM density maps by false discovery rate control (Use in ccpem)
2. DeepEnhancer-DeepEMhancer: a deep learning solution for cryo-EM volume post-processing
3. LocScaleModel-based local density sharpening of cryo-EM maps (Use in ccpem)
4. LocalDeblur - Automatic local resolution-based sharpening of cryo-EM maps
5. EM-GAN - Improved Protein Structure Modeling Using Enhanced Cryo-EM Maps With 3D Deep Generative Networks

• Denoising 3D volume

1. Warp - Based on Noise2Noise
2. Topaz - Contains 3D denoise functionality
3. Relion - Exploiting prior knowledge about biological macromolecules in cryo-EM structure determination

• Map to model

1. ModelAngelo - ModelAngelo: Automated Model Building in Cryo-EM Maps

Visualization

1. Chimera - UCSF Chimera--a Visualization System for Exploratory Research and Analysis
2. ChimeraX - UCSF ChimeraX: Meeting modern challenges in visualization and analysis

Conventions

3DEM Convention

• 3DEM Conventions
• Resolution Measure
• Symmetry

Image contrast

White on Black - Relion, Xmipp, EMAN2, CryoSparc (Cryo-EM data is typically recorded as Black on White but will invert during processing)

Black on White - Frealign (CisTEM)

Pixel size

For the format defined in RELION, the actual pixel size is calculated as rlnDetectorPixelSize * 10000 / rlnMagnification. In Relion 3.1 it has been replaced with rlnImagePixelSize.

FSC calculation

Using e2proc3d

Mask generation in CryoSparc

No Mask:

This is the raw FSC calculated between two independent half-maps reconstructed from the data. There is no masking applied, so both the structure and solvent are included in this FSC.

Spherical:

This is the FSC calculated after applying a soft spherical mask to both half maps. The outer radius of the soft sphere is equal to half the volume box-size (i.e. the sphere extends to the faces of the box in all directions). The inner radius is 85 percent of the outer radius. Between inner and outer radii, a soft cosine edge transitions from a mask value of one to a value of zero.

Loose:

This is the FSC calculated after applying a soft solvent mask to both half maps. The loose mask is calculated as follows. First, the density map is thresholded at 50% of the maximum density value. The resulting volume is dilated to create a soft mask. Voxels in the mask that are within 25 angstroms of the thresholded region receive a mask value of 1.0. Voxels between 25 and 40 angstroms fall off with a soft cosine edge, and voxels outside 40 angstroms receive a value of 0.0.

Tight:

This is the same as the loose mask, except the dilation distances are 6 angstroms for the value 1.0 distance and 12 angstroms for the value 0.0 distance.

Corrected:

This is the FSC curve calculated using the tight mask with correction by noise substitution [1]. The two half maps have their phases randomized beyond a certain resolution, then the tight mask is applied to both, and an FSC is calculated. This FSC is used along with the original FSC before phase randomization to compute the corrected FSC as in [1]. This accounts for correlation effects induced by masking. The resolution at which phase randomization begins is the resolution at which the no-mask FSC drops below the FSC = 0.143 criteria.

Chen, S. et al. High-resolution noise substitution to measure overfitting and validate resolution in 3D structure determination by single particle electron cryomicroscopy. Ultramicroscopy 135, 24–35 (2013).

Sharpen and filtering in CryoSparc

The map_filtered output in non-uniform refinement is generated as follows, after refinement has converged:

1. both raw, unfiltered halfmaps are averaged together
2. the raw map is filtered using the Gold-Standard FSC curve
3. the filtered map is sharpened using the Guinier-plot estimated b-factor
4. the sharpened map is outputted as _map_sharp_local.mrc (confusing filename… sorry)
5. the sharpened map is locally filtered using a local resolution estimate computed from the half-maps and the locally filtered map is outputted as _map_filtered.mrc

Tips

Format Conversion

1. Using PyRelion Operate Star files

2. Using PyEM Convert metadata

3. Using EMAN2 Convert binary data

4. Using SPHIRE Convert binary data

Parse Star file

1. Using StarTool
2. Using Custom function like here

Parse CryoSparc file

1. Using PyEM
2. Using Custom function like here. See full tutorial here

Performing Focus Classification

With Chimera

CisTEM

A focus mask is defined as a sphere specified by radius and x,y,z coordinates of the sphere center, all given in Å. In order to find x,y,z, in CisTEM

1. Open the 3D map in Chimera. The map should be assigned model #0.
2. Display the command line by selecting Tools > General Controls > Command Line.
3. Execute vop threshold #0 maximum -1000 setMaximum 1.0 (this assumes that the minimum voxel value in the 3D map is larger than 1000). This will create a new model #1.
4. Execute shape sphere radius 30 color red mesh true on the Command Line. This will generate a sphere with a 30 Å radius, numbered model #2 (change the radius as needed).
5. Open Tools > General Controls > Model Panel and deactivate models #0 and #1. Then move the sphere (model #2) into model #1 until it is fully contained in that volume.
6. Execute mask #1 #2 on the Command Line. This will generate a new model #3 of a solid sphere with a 30 Å radius.
7. Close models #1 and #2 in Model Panel and make sure model #0 is visible but deactive.
8. Select model #3 (the solid sphere) in Volume Viewer and change its color to red (or something easily distiguishable from the original 3D map) and make it transparent.
9. Move model #3 to the desired location of the focus mask in the 3D map.
10. Execute vop resample #3 onGrid #0 on the Command Line. This will generate a copy of the solid sphere as a new model #1, now resampled in the same coordinate system as the original 3D map.
11. Execute measure center #1 on the Command Line. This will display the x,y,z coordinates of the mask in pixel coordinates below the command line. These coordinates have to be converted to Å by multiplying them with the pixel size of the 3D map before they can be used in cisTEM's Manual Refine panel.

Local classification in CryoSparc

Display images

Micrographs

In Matlab use grayImage = uint8(255 * mat2gray(originalImage)); imshow(grayImage);

Using Chimera

Create synthetic map from PDB

1. Open chimera and import that protein: File -> Fetch By ID… -> Select PDB and type the PDB ID of the protein -> Fetch. Now you will see the 3D structure of the protein.
2. Create synthetic map from PDB model: Modify the density of the protein: Tools -> General Controls -> Command Line -> Type: molmap #0 5 (molmap = is command that generates a density map from the specified atom; # is atom specification, i.e. number assigned to the model by default; 5 = resolution).
3. Store reference map as MRC: Save to a file: Tools -> Volume Data -> Volume Viewer -> File -> Save map as… -> Give it the protein_PDB_ID.mrc.

Dispaly local resolution

Flip handedness of model and map

Slice view

volume #0 planes z,220 step 1 level -1 style surface

Visualizing dynamic movies

Using Script

Contributing

Your contributions are highly appreciated! Please take a look at the contribution guidelines first.