Invited Article: A uniﬁed evaluation of iterative...

Invited Article: A unified evaluation of iterative projection algorithms forphase retrieval

S. Marchesinia�

Lawrence Livermore National Laboratory, 7000 East Avenue, Livermore, California 94550-9234and Center for Biophotonics Science and Technology, University of California, Davis,2700 Stockton Boulevard, Suite 1400, Sacramento, California 95817

�Received 28 March 2006; accepted 5 November 2006; published online 25 January 2007;publisher error corrected 19 April 2007�

Iterative projection algorithms are successfully being used as a substitute of lenses to recombine,numerically rather than optically, light scattered by illuminated objects. Images obtainedcomputationally allow aberration-free diffraction-limited imaging and the possibility of usingradiation for which no lenses exist. The challenge of this imaging technique is transferred from thelenses to the algorithms. We evaluate these new computational “instruments” developed for thephase-retrieval problem, and discuss acceleration strategies. © 2007 American Institute of Physics.�DOI: 10.1063/1.2403783�

I. INTRODUCTION

Crystallographers routinely image molecular structuresof several thousand atoms by phasing the diffraction patternof a structure replicated in a periodic system. Likewise,computationally retrieving the phase of a diffraction patternis becoming increasingly successful at imaging—withseveral millions of resolution elements—objects as complexas biological cells, nanotubes, and nanoscale aerogel struc-tures. Diffraction microscopy �the imaging of isolatedobjects by diffraction and computational phase retrieval�promises a three-dimensional �3D� resolution limited onlyby radiation damage, wavelength, the collected solid angle,and the number of x rays or electrons collected. This capa-bility provides an extremely valuable tool for understandingnanoscience and cellular biology. Recent estimates1 of thedose and flux requirements of x-ray diffraction on single ob-jects indicate that attractive resolution values �about 10 nmfor life science and 2–4 nm for material science� should bepossible at a modern synchrotron. Atomic resolution couldbe accomplished using pulses of x rays that are shorter thanthe damage process itself2,3 using femtosecond pulses froman x-ray free-electron laser.3�a� Alternatively the radiationdamage limit could be eliminated by continuously replacingthe exposed samples, such as laser-aligned molecules4 withidentical ones.

In the fields of electron microscopy5 and astronomicalimaging,6 iterative projection algorithms have been used torecover the phase information in a variety of problems. Theevaluation of the aberrations in the Hubble space telescopedescribed by Fienup in Ref. 7 remains perhaps the mostprominent example of successful phase reconstructions in theastronomical community. Nugent and collaborators appliedsimilar techniques to characterize x-ray lenses 7�a�. In elec-

tron diffraction microscopy,5,8 Zuo and co-workers imaged asingle isolated nanotube at atomic resolution,9 and Wu et al.imaged defects at atomic resolution.10

An important review, which attempted to integrate theapproaches of the optical and crystallographic communities,appeared in 1990.11 The connection was made between the“solvent-flattening” or “density-modification” techniques ofcrystallography12 and the compact support requirements ofthe iterative projection algorithms. The importance of finesampling of the intensity of the measured diffraction patternwas recognized at an early stage.13

The observation by Sayre in 1952 �Ref. 14� that Braggdiffraction undersamples the diffracted intensity pattern wasimportant and led to more specific proposals by the sameauthor for x-ray diffractive microscopy of nonperiodicobjects.15,16 These ideas, combined with the rapid develop-ment of computational phase retrieval in the wider opticscommunity, especially the “support constraint,”5,6,17,18 en-abled the first successful use of coherent x-ray diffractionmicroscopy �CXDM�.

Since the first proof of principle demonstration ofCXDM by a team at Stony Brook,19 a number of groups havebeen working to bring these possibilities into reality.

Robinson and co-workers at the University of Illinoishave applied the principles of CXDM to hard x-ray experi-ments on microcrystalline particles. Such data have been re-constructed tomographically to produce a 3D image at 40 nmresolution.20,20�a� Miao �now at UCLA� and co-workers madeconsiderable progress in pushing the CXDM method atSpring-8 Japan to higher resolution in two-dimensional �2D��7 nm�, higher x-ray energies, and to a limited form of 3D.21

They have also made the first application of CXDM to abiological sample.22

A diffraction chamber dedicated to diffractionmicroscopy23 has been used to image biological cells24,25 atthe Advanced Light Source in Berkeley.26 Using the samechamber, a collaboration between Berkeley and Livermore

a�Present address: Lawrence Berkeley National Laboratory, 1 CyclotronRoad, Berkeley CA 94720; electronic mail: [email protected]

REVIEW OF SCIENTIFIC INSTRUMENTS 78, 011301 �2007�

0034-6748/2007/78�1�/011301/10/$23.00 © 2007 American Institute of Physics78, 011301-1

Downloaded 23 Jul 2007 to 129.49.23.145. Redistribution subject to AIP license or copyright, see http://rsi.aip.org/rsi/copyright.jsp

http://dx.doi.org/10.1063/1.2403783

http://dx.doi.org/10.1063/1.2403783

http://dx.doi.org/10.1063/1.2403783

laboratories and Arizona State University produced 3D im-aging at 10�10�40 nm resolution of test samples27 as wellas aerogel foams.28

In this article the computational instruments that enabledthese and other results are reviewed. Section II introduces thephase problem and the experimental requirements for dif-fraction microscopy, Sec. III describes the concepts of sets ofimages and their projectors. In Sec. IV the iterative projec-tion algorithms published in the literature are summarizedand tested on simple geometric sets. In Sec. V the connectionbetween projection- and gradient-based methods and relatedacceleration strategies are discussed.

II. THE PHASE PROBLEM

When we record the diffraction pattern intensity scat-tered by an object, the phase information is missing. Apartfrom normalization factors, an object of density ��r�, r beingthe coordinates in the object �or real� space, generates a dif-fraction pattern equal to the modulus square of the Fouriertransform �FT� �̃�k�:

I�k� = ��̃�k��2,

I�k� = �̃†�k��̃�k� , �1�

where k represent the coordinate in the Fourier �or Recipro-cal� space. The inverse Fourier transform �IFT� of the mea-sured intensity I provides the autocorrelation ��−r��r� ofthe object:

IFT�I�k�� = ��− r� � ��r� . �2�

The phase-retrieval problem consists of solving �̃ in Eq. �1�or � in Eq. �2�, using some extra prior knowledge. In diffrac-tion microscopy, solving such a problem is performed withgigaelement large-scale optimization algorithms, describedin the following section.

Since the intensity represents the FT of the autocorrela-tion function, and the autocorrelation is twice as large as theobject, the diffraction pattern intensity should be sampled atleast twice as finely as the amplitude to capture all possibleinformation on the object. Finer sampling adds a 0-paddingregion around the recovered autocorrelation function, whichadds no further information �Shannon theorem�. Less thancritical sampling in the Fourier domain causes aliasing in theobject space. A periodic repetition of the same structure pro-vides a stronger signal, enabling the measurement of the dif-fraction pattern before the structure is damaged. However,while an isolated object generates a continuous diffractionpattern that can be sampled as finely as desired, a periodicrepetition of the same object generates only a subset of thepossible diffraction intensities. Crystallography therefore hasto deal with an aliased autocorrelation function, also knownas the Patterson function. This reduced information can becompensated by other prior knowledge, such as the atomicnature of the object being imaged, knowledge of a portion ofthe object, presence of heavy atoms, and information ob-tained with anomalous diffraction. Other information in-cludes the presence of a solvent in the crystal. By varying thesampling rate of a diffraction pattern it was shown16,29,30 thatless than critical sampling was sufficient to solve the phase

problem. This was possible because the number of equations�measured intensities in Eq. �1�� in the 2D and 3D phase-retrieval problems is larger than the number of unknowns�resolution elements in the object�. The number of unknownsdefines the number of independent equations, or the mini-mum required sampling rate. Although no general proof hasbeen provided that limited sampling removes only redundantequations, such a minimum required sampling rate suggeststhat when the solvent exceeds 50% of the crystal volume, thealgorithms developed in the optical community, using tech-niques to dynamically refine the solvent regions31 may beable obtain ab initio structural information from crystals.

Coherence is required to properly sample the FT of theautocorrelation of the object.32 According to the Schelltheorem,33 the autocorrelation of the illuminated object ob-tained from the recorded intensity is multiplied by the com-plex degree of coherence. The beam needs to fully illuminatethe isolated object, and the degree of coherence must belarger than its autocorrelation.

Diffraction microscopy solves the phase problem by us-ing the knowledge that the object being imaged is isolated; itis assumed to be 0 outside a region called support S:

��r� = 0, if r � S . �3�

This support is equivalent to the solvent in crystallography.Equations �1� and �3� can be combined to obtain a multi-dimensional system of quadratic equations in the ��r�variables:

��r�S

��r�exp�ik · r��2= I�k� , �4�

which is a quadratic equation in the ��r� variables with co-efficients cr,r��k�=exp �ik�r-r��:

�r,r��S

cr,r��k��r��r�� = I�k� . �5�

Each value of I�k� in reciprocal space defines an ellipsoid�Eq. �5�� in the multidimensional space of the unknowns��r�, �r�S. If the number of independent equations equalsthe number of unknowns, the system has a single solution��r�. The intersection of these ellipsoids forms our solution.Unfortunately this system of equations is difficult to solve,and has an enormous number of local minima. Constantphase factors, inversion with respect to the origin �enantio-morphs�, and origin shifts ��±r+r0�ei�0 are undeterminedand considered equivalent solutions. The presence of mul-tiple nonequivalent solutions in 2D and higher-dimensionalphase retrieval problems is rare34; it occurs when the densitydistribution of the object can be described as the convolutionof two or more noncentrosymmetric distributions. Simplehomometric structures for which the phase problem is notunique35 exist in nature, but such nonuniqueness is less likelyfor more complex structures.

The presence of noise and limited prior knowledge�loose constraints� increases the number of solutions withinthe noise level and constraints. Confidence that the recoveredimage is the correct and unique one can be obtained by re-peating the phase-retrieval process using several randomstarts. Repeatability of the recovered images as a function of

011301-2 S. Marchesini Rev. Sci. Instrum. 78, 011301 �2007�


resolution measures the effective phase-retrieval transferfunction,24,27 which can be decomposed in unconstrainedamplitudes modes24 and phase aberrations.36

In the early 1980s, the development of iterative algo-rithms with feedback by Fienup produced a remarkably suc-cessful optimization method capable of extracting phaseinformation.6,18,37 The important theoretical insight that theseiterations may be viewed as projections in Hilbert space38,39

has allowed theoreticians to analyze and improve on the ba-sic Fienup algorithm.40–43

These algorithms try to find the intersection between twosets, typically the set of all the possible objects with a givendiffraction pattern �modulus set�, and the set of all the ob-jects that are constrained within a given area or support vol-ume �or outside a solvent region in crystallography�. Thesearch for the intersection is based on the information ob-tained by projecting the current estimate on the two sets. Anerror metric is obtained by evaluating the distance betweenthe current estimate and a given set. The error metric and itsgradient are used in conjugate-gradient �CG�-based methodssuch as SPEDEN.44

III. SETS, PROJECTORS AND METRICS

An image of a density distribution can be described as asequence of n pixel values. For an image of n pixels, thereare n coordinates. The magnitude of the density at a pixeldefines the value of that coordinate. Thus a single vector inthis n-dimensional space defines an image. For complex im-ages the number of coordinates increases by a factor of 2.Axes of the multidimensional space are formed by any se-quence of n pixels with all but one pixel equal to 0. Anexample is x= �x ,0 ,0� in a 3-pixel solution space. The originof this space is the image with all the pixels equal to 0. Thecomponents on these axes form the real or object space. Thesame object can be described in terms of any anothern-dimensional orthogonal �or linearly independent� bases.Axes can be rotated, shifted, inverted and so on, and theproper linear transform must be applied to obtain the com-ponents in the new basis. The basis used to describe theimage must have at least n components, but more can beused if it helps to describe the properties of the algorithm.For example the values could be left to have a real and animaginary component, doubling the number of dimensionsused to describe the object.

One important basis is the momentum or Fourier space.While the vector in the n-dimensional space representing animage is unaltered on transforming from real to reciprocalspace, its components in the new axes are altered �Fourier-transformed�. The distance between two points in then-dimensional space is independent of this transformation�Parseval theorem�. The lengths and the angles between vec-tors will be our guide to describe the behavior, convergenceand error metrics of these algorithms.

We consider two sets, S �support� and M �modulus�.When the image belongs to both sets simultaneously, wehave reached a solution. If the properties of the object beingimaged are known a-priori to be limited in a support region,we know that in the n-dimensional space of the pixel values,

some values must be zero. Images that satisfy this rule�Eq. �3�� form the support constraint set. A projection ontothis set �Ps� involves setting to 0 the components outside thesupport, while leaving the rest of the values unchanged�Fig. 1�a��:

Ps��r� = ��r� if r � S

0 otherwise,� �6�

and its complementary projector Ps=I−Ps.The values in every pixel in Fourier space can be de-

scribed using two components, the real and imaginary parts,or amplitude and phase, both defining a point in a complexplane. In an intensity measurement we obtain the amplitudeor modulus in every pixel that defines a circle in a complexplane. These circles define the modulus constraint�Fig. 1�b��. When every complex-valued pixel lies on thecircle defined by the corresponding modulus, the image sat-

FIG. 1. Examples of sets and projectors: �a� Support: The axes represent thevalues on 3 pixels of an image � known to be 0 outside the support S. Thevertical axis ��r3� represents a pixel outside �r3�S�, while the horizontalplane represents pixels inside S. The projection on this set is performedsimply by setting to 0 all the pixels outside the support. �b� Modulus: A pixel�in Fourier space� with a given complex value is projected on the closestpoint on the circle defined by the radius m. If there is some uncertainty inthe value of the radius m±�, the circle becomes a band. The circle is anon-convex set, since the linear combination between two points on thesame set �1 and �2 does not lie on the set. Also represented in the figure isthe projection on the real axis �reality projection�.

011301-3 Phase retrieval: Projection algorithms Rev. Sci. Instrum. 78, 011301 �2007�


isfies this constraint and it belongs to the modulus set. Seg-ments joining two points on a circle do not belong to thecircle; therefore the linear combination of two images is out-side the set: the set is nonconvex. These sets are problematicbecause of the presence of local minima and undefinedprojections.

The projection of a point in each complex plane onto thecorresponding circle is accomplished by taking the point onthe circle closest to the current one, setting the modulus tothe measured one �I�k�, and leaving the phase unchanged�Fig. 1�b��:

P̃m�̃�k� = P̃m��̃�k��ei��k� = �I�k�ei��k�, �7�

where we have defined the reciprocal space representation ofthe projector:

Pm = F−1P̃mF , �8�

and F and F−1 represent the forward and inverse Fouriertransforms respectively.

This operator is demonstrated to be a projector on thenonconvex �Fig. 1�b�� set of the magnitude constraint.45 Thesame paper discusses the problems of multi valued projec-tions for nonconvex sets, which do not satisfy the require-ments for gradient-based minimization algorithms, and therelated non-smoothness of the squared set distance metric,which may lead to numerical instabilities. See also Ref. 46for a follow-up discussion on the non-smooth analysis.

A projector P is an operator that takes to the closestpoint of a set from the current point �. A repetition of thesame projection is equal to one projection alone �P2=P�; itseigenvalues must therefore be �=0,1. Another operator usedhere is the reflector R= I+2�P−I�=2P−I, which applies thesame step as the projector but moves twice as far �Fig. 2�. Inthe case of the support constraint, the whole image space canbe described in terms of the eigenvectors of the correspond-ing linear projector. These eigenvectors with eigenvalues of1 �0� are the images with all the pixels equal to 0, except forone pixel inside �outside� the support. The modulus projectoris a nonlinear operator:

Pm�a + b� � Pm�a� + Pm�b� ,

Pm��a� � �Pm�a� , �9�

and it cannot be described in terms of eigenvalues and eigen-vectors.

The Euclidean length � of a vector � is defined as:

� 2 = �† · � = �r

��r��2 = �k

��̃�k��2. �10�

The sum is extended to the measured portion of the diffrac-tion pattern. If part of the reciprocal space is not measured, itshould not be included in the sum. In fact the sum should beweighted with the experimental noise ��k�:

� 2 =�k

1�2�k� ��̃�k��2

�k1

�2�k�

, �11�

with ��k�= for values of k not measured. The distancefrom the current point to the set P�−� is the basis for ourerror metric. Typically the errors in real �s� and reciprocalspace �m� are defined in terms of their distance to the cor-responding sets:

s�� = Ps� − � ,

m�� = Pm� − � , �12�

or their normalized version x��=x�� / Px� . Another errormetric used in the literature is given by the distance betweenthe two sets: s,m��= Pm�−Ps� . The projector Pm moves� to the closest minimum of m

2 �Pm��=0, providing a sim-ple relation with the gradient ��m

2 �� Refs. 18 and 45�:

Pm� = � + �Pm − I�� = � −1

2��m

2 �� , �13�

where ��m2 �� is proportional to ��m��:

��m2 �� = 2m��m�� . �14�

For �̃�k�=0, m is nondifferentiable, and the projector Pm ismultivalued.45 The presence of complex zeros ��k�=0� isconsidered of fundamental importance in the phase-retrievalproblem,47 and the phase vortices associated with these zeroscause stagnation in iterative algorithms.48 Several methodshave been proposed to solve this problem.21,36,48–50 Similarlythe projector Ps minimizes the error s

2:

�I − Ps�� = �s =1

2��s

2�� . �15�

IV. ITERATIVE PROJECTION ALGORITHMS

Several algorithms based on these concepts have nowbeen proposed and a visual representation of their behavior isuseful to characterize the algorithm in various situations, inorder to help choose the most appropriate one for a particularproblem. In this section the projection algorithms publishedin the literature are summarized �see also Table I� and testedon simple geometrical sets.

The following algorithms require a starting point �0,which is generated by assigning a random phase to the mea-sured object amplitude �modulus� in the Fourier domain��̃�k��=m�k�=�I�k�. The first algorithm called error reduc-

FIG. 2. The reflector applies the same step as the projector �P−I� twice:R�=I�+2�P−I��.



tion �ER� �Gerchberg and Saxton5,38,51� is simply �Fig. 3�a��

��n+1� = PsPm��n�, �16�

and by projecting back and forth between two sets, it con-verges to the local minimum. The name of the algorithm isdue to the steps moving along the gradient of the error metric�see Eq. �13��:

PsPm� = Ps� −1

2�sm

2 �� , �17�

where �s=Ps� is the component of the gradient in the sup-port. Figure 3�a� shows that the step size is far from opti-mum, but that it guarantees linear convergence. A line searchalong this gradient direction would considerably speed up

the convergence to a local minimum and will be discussed inSec. V.

The solvent flipping �SF� algorithm52 is obtained byreplacing the support projector Ps with its reflectorRs=2Ps−I �Fig. 3�b��:

��n+1� = RsPm��n�, �18�

which multiplies the charge density � outside the supportby −1. The hybrid input-output �HIO�6,18 �Fig. 3�c�� is basedon nonlinear feedback control theory and can be expressed as

��n+1��x� = Pm��n��x� if x � S ,

�I − �Pm��n��x� otherwise.� �19�

Equations �13� and �15� can be used to describe the steps��=��n+1�−��n�� in terms of the gradients of the error met-rics. In Sec. V it will be shown that this algorithm seeks thesaddle point:

min�s

max�s

L��, L�� = m2 �� − s

2�� 20�

by moving in the descent–ascent direction ��−Ps+�Ps��L��see Sec. V for details�, rather than in the simple error-minimization direction.

It is often used in conjunction with the ER algorithm,alternating several HIO and one or more ER iterations�HIO�20�+ER�1� in our case�. In particular one or more ERsteps are used at the end of the iteration. Elser40 pointed outthat the iterate �n can converge to a fixed point ��n+1=�n�,

TABLE I. Summary of various algorithms.

Algorithm Iteration ��n+1�=ER PsPm��n�

SF RsPm��n�

HIO �Pm��n��r� r�S

�I−�Pm��n��r� r�S DM �I+�Ps��1+ s�Pm− sI�−�Pm��1+ m�Ps− mI��n�

ASR 12 �RsRm+I��n�

HPR 12 �Rs�Rm+ ��−1�Pm�+ I+ �1−��Pm��n�

RAAR � 12��RsRm+I�+ �1−��Pm��n�

FIG. 3. Geometric representation ofvarious algorithms using a simplifiedversion of the constraint: two lines in-tersecting. �a� Error reduction algo-rithm: we start from a point on themodulus constraint by assigning a ran-dom phase to the diffraction pattern.The projection onto the modulus con-straint finds the point on the set whichis nearest to the current one. The ar-rows indicate the gradients of the errormetric. �b� The speed of convergenceis increased by replacing the projectoron the support with the reflector. Thealgorithm jumps between the modulusconstraint �solid diagonal line� and itsmirror image with respect to the sup-port constraint �dotted line�. �c� Hy-brid input–output, see text �Eq. �19��.The space perpendicular to the supportset is represented by the vertical dottedline S. �d� Difference map, see text�Eq. �22��.



which may differ from the solution � �Ps�=Pm�=��. How-ever the solution � can be easily obtained from the fixedpoint:

�mn = Pm�n, �21�

�sn = �1 +

1

��PsPm�n −

1

�Ps�

n,

where �m and �s should coincide, or else their difference canbe used as an error metric. See Ref. 40 for further details.

The difference map �DM� is a general set ofalgorithms,40 which requires four projections �two time-consuming modulus constraint projections� �Fig. 3�d��:

��n+1� = �I + �Ps��1 + s�Pm − sI�

− �Pm��1 + m�Ps − mI��n�; �22�

the solution corresponding to the fixed point is describedin the same article.40 We will use in the upcoming testswhat Elser suggested as the optimum, with s=−�−1 and m=�−1.

The averaged successive reflections �ASR�41 algorithmis

��n+1� =1

2�RsRm + I��n�. �23�

The Hybrid Projection Reflection �HPR�42 algorithm is de-rived from a relaxation of the ASR:

��n+1� =1

2�Rs�Rm + �� − 1�Pm� + I + �1 − ��Pm��n�. �24�

It is equivalent to HIO if positivity �Sec. IV.A� is not en-forced, but it is written in a recursive form, instead of acase-by-case form such as Eq. �19�. It is also equivalent tothe DM algorithm for s=−1, m=�−1. Finally the relaxedaveraged alternating reflectors �RAAR� algorithm43:

��n+1� = �1

2��RsRm + I� + �1 − ��Pm��n�. �25�

For �=1, HIO, HPR, ASR, and RAAR coincide.The first test is performed on the simplest possible case:

find the intersection between two lines. Figure 4 shows thebehavior of the various algorithms. The two sets are repre-sented by a horizontal blue line �support� and a tilted blackline �modulus�. ER simply projects back and forth betweenthese two lines, and moves along the support line in thedirection of the intersection. SF projects onto the modulus,“reflects” on the support, and moves along the reflection ofthe modulus constraint onto the support. The solvent flippingalgorithm is slightly faster than ER thanks to the increased inthe angle between projections and reflections. HIO and vari-ants �ASR, DM, HPR� move in a spiral around the intersec-tion, eventually reaching the intersection. For similar �RAAR behaves somewhere in between ER and HIO with asharper spiral, reaching the solution much earlier. Alternating20 iterations of HIO and 1 of ER �HIO�20�+ER�1�� consid-erably speeds up the convergence.

When a gap is introduced between the two lines�Fig. 4�b��, so that they do not intersect, HIO and variantsmove away from this local minimum in search of another

“attractor” or local minimum. This shows how these algo-rithms escape from local minima and explore the multidi-mensional space for other minima. ER, SF, and RAAR con-verge to or near the local minimum. By varying � RAARbecomes a local minimizer for small �, and becomes likeHIO for ��1. ER, SF, and HIO+ER converge to the localminimum in these tests.

A more realistic example is shown in Fig. 5. Here thecircumference of two circles represents a nonconvex set�modulus constraint�, while the support constraint is repre-sented by a line. The convex set represents a simplifiedmodulus constraint in a phase-retrieval problem. The advan-tage of this example is the simplicity in the “modulus” pro-jector operator �it projects onto the closest circle�.

We start from a position near the local minimum. ER,SF, and HIO+ER fall into this trap �Fig. 5�. HIO and variantsmove away from the local minimum, “find” the other circle,and converge to the center of the circle. In the center theprojection on the modulus constraint becomes “multivalued,”and its distance metric is “nonsmooth.” Such a point is un-stable, and the algorithms start spiraling toward the solution.For �=0.75, RAAR does not reach the solution, but con-verge close to the local minimum.

FIG. 4. The basic features of the iterative projection algorithms can beunderstood by this simple model of two lines intersecting �a�. The aim is tofind the intersection. The ER algorithm and the solvent flipping algorithmsconverge in some gradient-type fashion �the distance to the two sets neverincreases�, the solvent flip method being slightly faster when the angle be-tween the two lines is small. HIO and variants move following a spiral path.The lagrangian �L=m

2 −s2� is represented in grayscale, and the descent-

ascent directions ��−�s ,�s��L� are indicated by arrows. When the two lines

do not intersect �b�, HIO and variants keep moving in the direction of thegap between the two lines, away from the local minimum. ER, SF, andRAAR converge at �or close to� the local minimum.



A. Positivity

The situation changes slightly when we consider thepositivity constraint. The previous definitions of the algo-rithms still apply, just replacing Ps with Ps+:

Ps+ = ��r� if r � S and ��r� � 0,

0 otherwise.� �26�

The only difference is for HIO, which becomes

��n+1� = Pm��n��r� if r � S and Pm��n��r� � 0,

�1 − �Pm��n� otherwise.��27�

HIO and variants follow the saddle-point direction, movingaway from local minima �Fig. 6�, but as they approach thesolution they react differently to the positivity constraint,with HIO “bounching” at the x=0 axis, and ASR/HPR/DMproceeding more smoothly toward the solution.

V. STEEPEST DESCENT, CONJUGATE GRADIENT,AND MIN-MAX ALGORITHMS

As discussed in Sec. IV, the error reduction �ER� algo-rithm moves in the direction of the steepest descent18; how-ever the step length is not optimized to reach the local mini-mum in that direction, since it is only one component of thefull gradient �Fig. 3�a��. Such strategy is generally referred toas reduced gradient method. Figure 7 shows the error metricm as a function of two unknown pixel values in a simpletwo-dimensional phase-retrieval problem, and the behaviorof the ER algorithm toward the local minima.

The simplest acceleration strategy, the steepest descentmethod, uses the steepest direction �gradient� and performs aline search of the local minimum in the descent direction:

min�m2 �� + �� , �28�

�� = −1

2�sm

2 �� = − Ps�I − Pm�� ,

where �s=Ps�� is the gradient with respect to �s. At a mini-mum any further movement in the direction of the current

step increases the error metric; the gradient direction must beperpendicular to the current step. In other words the currentstep and the next step become orthogonal:

�

��m

2 �� + �� = ��Ps�I − Pm�� + ��s��r,

0 = ��s��I − Pm�� + ��s��r, �29�

where �x �y�r=R�x† ·y�. The line search algorithm can usem

2 , and/or its derivative in Eq. �29�. This optimization shouldbe performed in reciprocal space, where the modulus projec-tor is a diagonal operator and is fast to compute �Eq. �7��,while the support projection requires two Fourier transforms:

P̃s = FPsF−1. �30�

The steepest descent method is known to be inefficient in thepresence of long narrow valleys, where imposing that suc-cessive steps be perpendicular causes the algorithm to zigzagdown the valley. This problem is solved by the nonlinearconjugate gradient method.53–59 Instead of moving in the di-rection of steepest descent ��s, we move in the conjugatedirection ��s:

��s�n� = �s

�n� if n = 1,

��s�n� + s��s

�n−1� otherwise,� �31�

with s given by the Polak–Ribière method57:

s =��s

�n��s�n� − ��s

�n−1��r

��s�n−1� 2 , �32�

and forced to be positive: =max� s ,0� to improve its reli-ability. The presence of local minima shown in the previouschapters, however, will cause stagnation of steepest andconjugate gradient methods, preventing global convergence�Fig. 7�c��.

A. Feedback and the saddle-point problem

The ability to escape local minima demonstrated byinput-output feedback-based algorithms �Fig. 7�d�� makesthem superior to the methods based on simple gradient mini-mization of the error. However, as in the ER algorithm, the

FIG. 5. The horizontal line represents a support constraint, while the twocircles represent a nonconvex constraint, i.e., the modulus constraint. Thegradient-type �ER and SF� algorithms converge to the local minimum, whileHIO and variants follow the descent-ascent direction ��−�s ,�s�L� indicatedby the arrows.

FIG. 6. Positivity constraint: the support constraint is represented by a hori-zontal line originating from 0 �x�0�. A barrier due to the positivity con-straint changes the behavior of the algorithms, which no longer follow thedescent–ascent direction. HIO bounces on the x=0 axis, while the otheralgorithms are smoother.



step length is not optimized, the algorithm keeps moving inthe same direction for several steps, and sometimes over-shoots. Combining the ideas of the conjugate gradient or thesteepest descent methods and IO feedback could consider-ably speed-up convergence. Given the lagrangian L definedas the difference between the two errors:

L�� = m2 �� − s

2�� , �33�

using Eqs. �13� and �15� we obtain the gradient

�L�� = 2�Ps − Pm�� . �34�

The step �� used in HIO �Eq. �19�� can be expressed interms of this gradient �L:

�� = ��n+1� − ��n�

= �Ps�Pm − I� − �PsPm�

= �Ps�Pm − Ps� − �Ps�Pm − Ps��

FIG. 7. A simple 2D phase-retrieval problem: only two variables �pixel values� are unknown. The solution–the global minimum–is the top minimum in thefigures. The colormap and contour lines represents the error metric m��s�, and the descent direction is indicated by the arrows. The error reduction algorithm�a� proceeds toward the local minimum without optimizing the step length and stagnates at the local minima. The steepest descent method �b� moves towardthe local minimum with a zigzag trajectory, while the conjugate gradient method reaches the solution faster �c�. The HIO method generally converges to theglobal minimum, however some rare starting points converge to a local minimum �d�. The saddle-point optimization with optimized step length �Eq. �37��stagnates in the same local minimum as HIO �e�. The conjugate gradient version avoids stagnation �f�. The saddle-point optimization using a two dimensionalsearch of the saddle point reaches the global minimum from a larger range of starting points than HIO �g�. The conjugate gradient version �h�, �i� reaches thesolution faster if the conjugate directions ��s,s are obtained independently from ��s,s �i�, rather than their sum ��=��s+��s.



= �− Ps + �Ps1

2� L�� . �35�

HIO/HPR/ASR algorithms move toward the minimum of Lin the subspace �s, and the maximum in the subspace �s,using a reduced gradient optimization strategy, where thestep is proportional to the gradient but with one sign reversal�Eq. �35��. In other words, they seek the saddle point:

min�s

max�s

L��s + �s� . �36�

Min-max or saddle-point problems arise in fields as variousas game theory, economics, physics, engineering, andprimal–dual optimization methods. Function minimization iseasier than saddle-point optimization because a simple func-tion evaluation can tell us if a new point is better than theprevious one. The saddle can be higher or lower than thecurrent value, although the direction toward the saddle isindicated by the two gradient components. One option is toalternate minimization in the direction �s and maximizationin the direction �s of L. Such a strategy is similar to alter-nating HIO and ER algorithms and can be performed usingoff-the-shelf optimization routines, but it can be slow. Opti-mization of the step length, a multiplicative factor �, is ob-tained by increasing � until the current and next search di-rections become perpendicular to one another �Fig. 7�e��:

��Ps − �Ps� � L�� + ��r = 0,

��Ps�Pm − I� − �PsPm�� + ��r = 0. �37�

In analogy to the conjugate gradient method, one could sub-stitute the search direction �� with ��, as in Eq. �31��Fig. 7�f��. A more robust strategy involves replacing theone-dimensional �1D� search with a 2D optimization of thesaddle point �Fig. 7�g��:

min�

max�

��,�� ,

��,�� = L�� + ��s + ��s� . �38�

Once the 2D min–max problem is solved, the new directionscan be obtained by following the conjugate gradient scheme�Fig. 7�i��.

VI. CONCLUSIONS

Lensless imaging owes its success as an effective tool toobserve nanoscale systems to the advances made in phase-retrieval algorithms. The new instruments replacing lensesare the iterative projection algorithms for phase retrieval.These algorithms can be grouped in two categories: �1� localminimizers such as ER, SF, steepest descent, and conjugate-gradient methods, with Solvent Flip having some moderateability to escape local minima52; �2� more global minimizerssuch as HIO, DM, ASR, HPR, which use a feedback to reachthe solution. RAAR and ER+HIO fall somewhere in be-tween the two categories, depending on an adjustable param-eter. A simple benchmark is shown for comparison in Fig. 8and summarized in Table II. The test consisted in solving aphase-retrieval problem without assuming positivity �nor re-ality� of the object, and the support region was slightly largerthan the object, and was repeated 100 times for each algo-rithm. Many algorithms surprisingly failed, and only theones shown in Fig. 9 succeeded. HIO appears to be the mosteffective algorithm, and it is significantly improved in termsof speed and reliability when the 2D step size optimization�SO2D�, as described in Eq. �38� is applied. Further im-provements in reliability are achieved by performing asaddle-point optimization in a four-dimensional space�SO4D�. Minimization algorithms, although not very power-ful at solving the phase problem, can be used to polish up asolution, improving the values of the error metric consider-ably. The algorithms described here use as prior knowledgethe support region. Algorithms that use a simple threshold toreplace the support29 or more sophisticated support

TABLE II. Benchmark of various algorithms.

Algorithm No. of iterationsfor 50% success

Success after10 000 iterations

HIO/HPR 2790 82%HIO/HPR+ER 2379 82.6%ASR 1697a 42%SO2D 656 100%SO4D 605 100%Others �10 000 0%

a42% success, the algorithm either reconstruct in a limited number of itera-tions or never.

FIG. 8. Test figure used for benchmarking. The object of 1282 elements issurrounded by empty space. The whole image has 2562 elements.

FIG. 9. Percentage of successful reconstructions over many tests startingfrom random phases as a function of number of iterations. The support is theonly constraint. Positivity and reality are not enforced, and the support isloose: it is larger than the object by one additional row and column



refinement31 have not been discussed. Various projection al-gorithms combined with some form of threshold have pro-duced remarkable reconstructions of single isolated objects�HIO and RAAR �Ref. 27�, DM �Ref. 24��, as well as singleand powder crystals �SF and HIO with support refinementRefs. 53, 54, and 60–62��, but a full comparison of the algo-rithms behavior applied to this type of constraint have notbeen discussed.

ACKNOWLEDGMENTS

This work was performed under the auspices of the U.S.Department of Energy by the Lawrence Livermore NationalLaboratory under Contract No. W-7405-ENG-48 and theDirector, Office of Energy Research. This work was partiallyfunded by the National Science Foundation through theCenter for Biophotonics. The Center for Biophotonics, aNational Science Foundation Science and TechnologyCenter, is managed by the University of California, Davis,under Cooperative Agreement No. PHY0120999. The authoracknowledge useful discussions with H. N. Chapman, M. R.Howells, J. C. H. Spence and D. R. Luke.

1 M. R. Howells et al., J. Electron Spectrosc. Relat. Phenom. � cond-mat/0502059�.

2 J. C. Solem and G. C. Baldwin, Science 218, 229 �1982�.3 R. Neutze, R. Wouts, D. van der Spoel, E. Weckert, and J. Hajdu, Nature

406, 752 �2000�; H. N. Chapman et al., Nat. Phys. 2, 789 �2006�.4 J. C. H. Spence and R. B. Doak, Phys. Rev. Lett. 92, 198102 �2004�.5 R. Gerchberg and W. Saxton, Optik �Stuttgart� 35, 237 �1972�.6 J. R. Fienup, Opt. Lett. 3, 27 �1978�.7 J. R. Fienup, J. C. Marron, T. J. Schulz, and J. H. Seldin, Appl. Opt. 32,1747 �1993�; H. M. Quiney, A. G. Peele, Z. Cai, D. Paterson, and K. A.Nugent, Nat. Phys. 2, 101 �2006�.

8 J. C. H. Spence, U. Weierstall, and M. Howells, Philos. Trans. R. Soc.London, Ser. A 360, 875 �2002�.

9 J. M. Zuo, I. Vartanyants, M. Gao, R. Zhang, and L. A. Nagahara, Science300, 1419 �2003�.

10 J. Wu, U. Weierstall, and J. C. H. Spence, Nat. Mater. 4, 912 �2005�.11 R. P. Millane, J. Opt. Soc. Am. A 7, 394 �1990�.12 International Tables for Crystallography, edited M. G. Rossmann and E.

Arnold �Kluwer Academic, Dordrecht, 2001�, Vol. F.13 R. H. T. Bates, Optik �Stuttgart� 61, 247 �1982�.14 D. Sayre, Acta Crystallogr. 5, 843 �1952�.15 D. Sayre, Lect. Notes Phys. 112, 229 �1980�.16 J. Miao, D. Sayre, and H. N. Chapman, J. Opt. Soc. Am. A 15, 1662

�1998�.17 J. R. Fienup, Opt. Eng. �Bellingham� 19, 297 �1980�.18 J. R. Fienup, Appl. Opt. 21, 2758 �1982�.19 J. Miao, P. Charalambous, J. Kirz, and D. Sayre, Nature 400, 342 �1999�.20 G. J. Williams, M. A. Pfeifer, I. A. Vartanyants, and I. K. Robinson,

Phys. Rev. Lett. 90, 175501 �2003�;M. A. Pfeifer, G. J. Williams, I. A.Vartanyants, R. Harder, and I. K. Robinson, Nature 442, 63 �2006�.

21 J. Miao et al., Phys. Rev. Lett. 89, 088303 �2002�.22 J. W. Miao, K. O. Hodgson, T. Ishikawa, C. A. Larabell, M. A. LeGros,

and Y. Nishino, Proc. Natl. Acad. Sci. U.S.A. 100, 110 �2003�.23 T. Beetz et al., Nucl. Instrum. Methods Phys. Res. A 545, 459 �2005�.

24 D. Shapiro, P. Thibault, T. Beetz, V. Elser, M. Howells, C. Jacobsen, J.Kirz, E. Lima, H. Miao, A. Neiman, and D. Sayre, Proc. Natl. Acad. Sci.U.S.A. 102, 15343 �2005�.

25 P. Thibault, V. Elser, C. Jacobsen, D. Shapiro, and D. Sayre, ActaCrystallogr. A62, 248–261 �2006�.

26 M. R. Howells, P. Charalambous, H. He, S. Marcesini, and J. C. H.Spence, Proc. SPIE 4783, 65 �2002�.

27 H. N. Chapman, A. Barty, S. Marchesini, A. Noy, C. Cui, M. R. Howells,R. Rosen, H. He, J. C. H. Spence, U. Weierstall, T. Beetz, C. Jacobsen,and D. Shapiro, J. Opt. Soc. Am. A 23, 1179 �2006�.

28 A. Barty et al. �unpublished�.29 R. P. Millane, J. Opt. Soc. Am. A 13, 725 �1996�.30 W. McBride, N. L. O’Leary, and L. J. Allen, Phys. Rev. Lett. 93, 233902

�2004�.31 S. Marchesini, H. He, H. N. Chapman, S. P. Hau-Riege, . A. Noy, M. R.

Howells, U. Weierstall, and J. C. H. Spence, Phys. Rev. B 68, 140101�R��2003�.

32 J. C. H. Spence, U. Weierstall, and M. R. Howells, Ultramicroscopy 101,149 �2004�.

33 J. W. Goodman, Statistical Optics �Wiley, New York, 1985�.34 R. Barakat and G. Newsam, J. Math. Phys. 25, 3190 �1984�.35 M. J. Buerger, Vector Space and its Application in Crystal Structure In-

vestigation �Wiley, New York, 1959�.36 S. Marchesini, H. N. Chapman, A. Barty, M. R. Howells, J. C. H. Spence,

C. Cui, U. Weierstall, and A. M. Minor, IPAP Conf. Ser. 7, 380 �2006�.37 J. N. Cederquist, J. R. Fienup, J. C. Marron, and R. G. Paxman, Opt. Lett.

13, 619 �1988�.38 A. Levi and H. Stark, J. Opt. Soc. Am. A 1, 932 �1984�.39 H. Stark, Image Recovery: Theory and Applications �Academic,

New York, 1987�.40 V. Elser, J. Opt. Soc. Am. A 20, 40 �2003�.41 H. H. Bauschke, P. L. Combettes, and D. R. Luke, J. Opt. Soc. Am. A 19,

1334 �2002�.42 H. H. Bauschke, P. L. Combettes, and D. R. Luke, J. Opt. Soc. Am. A 20,

1025 �2003�.43 D. R. Luke, Inverse Probl. 21, 37 �2005�.44 S. P. Hau-Riege et al., Acta Crystallogr. A60, 294 �2004�.45 D. R. Luke, J. V. Burke, and R. G. Lyon, SIAM Rev. 44, 169 �2002�.46 J. V. Burke and D. R. Luke, SIAM J. Control Optim. 42, 576 �2003�.47 P.-T. Chen, M. A. Fiddy, C.-W. Liao, and D. A. Pommet, J. Opt. Soc. Am.

A 13, 1524 �1996�.48 J. R. Fienup and C. C. Wackerman, J. Opt. Soc. Am. A 3, 1897 �1986�.49 T. Isernia, G. Leone, R. Pierri, and F. Soldovieri, J. Opt. Soc. Am. A 16,

1845 �1999�.50 G. Oszlányi and A. Süto, Acta Crystallogr. �2005�.51 L. M. Brègman, Sov. Math. Dokl. 6, 688 �1965�.52 J. P. Abrahams and A. W. G. Leslie, Acta Crystallogr. D52, 30 �1996�.53 G. Oszlányi and A. Süto, Acta Crystallogr. A60, 134 �2004�.54 W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Nu-

merical Recipes in C �Cambridge University Press, Cambridge, 1992�.55 M. R. Hestenes, Conjugate Direction Methods in Optimization �Springer,

New York, 1980�.56 R. Fletcher and C. M. Reeves, Comput. J. 7, 149 �1964�.57 M. J. D. Powell, Lect. Notes Math. 1066, 122 �1984�.58 E. Polak and G. Ribiére, Rev. Fr. Inform. Rech. Oper. 16, 35 �1969�.59 R. Fletcher and C. M. Reeves, Comput. J. 7, 149 �1964�.60 E. Polak, Computational Methods in Optimization �Academic, New York,

1971�.61 J. S. Wu, J. C. H. Spence, M. O’Keeffe, and T. L. Groy, Acta Crystallogr.

A60, 326 �2004�.62 J. Wu, K. Leinenweber, and J. C. H. Spence, Nat. Mater. 5, 647 �2006�.



Invited Article: A uniﬁed evaluation of iterative...

Documents

Transcript of Invited Article: A uniﬁed evaluation of iterative...