Comparison Of Rate Of Convergence Of Iterative Methods Philosophy Essay

Comparison Of Rate Of intersection Of Iterative Methods Philosophy analyzeThe term repetitious mode refers to a wide range of techniques that use successive bringing close togethers to obtain more accu direct solutions to a unmatchable-dimensional organization at each step In numerical analysis it attempts to solve a occupation by finding successiveapproximationsto the solution starting from an initial guess. This prelude is in contrast todirect order actings which attempt to solve the problem by a finite sequence of trading operations, and, in the absence ofrounding errors, would deliver an look at solution Iterative method acting actings are usu all toldy the only choice for non linear equations. However, reiterative methods are lots efficacious even for linear problems involving a man-sized telephone function of variables (sometimes of the order of millions), where direct methods would be prohibitively expensive (and in some cases impossible) even with the best av ailable computing power.Stationary methods are older, simpler to attend and implement, further usually not as effective Stationary iterative method are the iterative methods that performs in each iteration the resembling operations on the current iteration vectors.Stationary iterative methods solve a linear system with anoperatorapproximating the original one and based on a bar of the error in the result, form a correction equation for which this process is repeated. While these methods are simple to derive, implement, and analyze, intersection point is only guaranteed for a limited manikin of matrices. Examples of nonmoving iterative methods are the Jacobi method,gauss seidel methodand thesuccessive overrelaxation method.The Nonstationary methods are based on the idea of sequences of orthogonal vectors Nonstationary methods are a congenatorly recent growing their analysis is usually harder to understand, but they can be highly effective These are the Iterative method that has iteration-dependent coefficients.It include Dense hyaloplasm Matrix for which the number of zipper elements is too small to warrant specialized algorithmic programs. Sparse ground substance Matrix for which the number of zero elements is large enough that algorithms avoiding operations on zero elements pay off. Matrices derived from partial(p) differential equations typically have a number of nonzero elements that is proportional to the hyaloplasm size, while the total number of intercellular substance elements is the square of the matrix size.The rate at which an iterative method converges depends greatly on the spectrum of the coefficient matrix. Hence, iterative methods usually involve a second matrix that transubstantiates the coefficient matrix into one with a more favorable spectrum. The transformation matrix is called apreconditioner. A good preconditioner improves the convergence of the iterative method, sufficiently to overcome the extra cost of constructing and applying the preconditioner. Indeed, without a preconditioner the iterative method may even fail to converge.Rate of ConvergenceInnumerical analysis, the speed at which aconvergent sequenceapproaches its limit is called therate of convergence. Although strictly speaking, a limit does not give information about any finite first part of the sequence, this concept is of practical importance if we acquit with a sequence of successive approximations for aniterative method as because typically fewer iterations are needed to yield a useful approximation if the rate of convergence is higher. This may even make the difference between needing ten or a million iterations.Similar concepts are used fordiscretizationmethods. The solution of the discretized problem converges to the solution of the continuous problem as the grid size goes to zero, and the speed of convergence is one of the elements of the efficiency of the method. However, the oral communication in this case is different from t he terminology for iterative methods.The rate of convergence of an iterative method is represented by mu () and is defined as suchSuppose the sequencexn(generated by an iterative method to find an approximation to a fixed point) converges to a pointx, thenlimn-infinity = xn+1-x/xn-xalpha=,where0 and(alpha)=order of convergence.In cases where=2 or 3 the sequence is state to havequadraticandcubic convergence reckonively. However in linear cases i.e. when=1, for the sequence to convergemustbe in the interval (0,1). The theory keister this is that for En+1Ento converge the absolute errors must decrease with each approximation, and to guarantee this, we have to erect0In cases where=1 and=1andyou know it converges (since=1 does not tell us if it converges or diverges) the sequencexnis said to convergesublinearlyi.e. the order of convergence is slight(prenominal) than one. If1 then the sequence diverges. If=0 then it is said to convergesuperlinearlyi.e. its order of convergence is highe r than 1, in these cases you changeto a higher look upon to find what the order of convergence is.In cases whereis negative, the iteration diverges.Stationary iterative methodsStationary iterative methods are methods for solving alinear system of equations. Ax=B. whereis a given matrix andis a given vector. Stationary iterative methods can be expressed in the simple formwhere neithernordepends upon the iteration count. The four main stationary methods are the Jacobi Method,Gauss seidel method,successive overrelaxation method(SOR), and even successive overrelaxation method(SSOR).1.Jacobi method- The Jacobi method is based on solving for every variable locally with respect to the other variables one iteration of the method corresponds to solving for every variable once. The resulting method is easy to understand and implement, but convergence is slow. The Jacobi method is a method of solving amatrix equationon a matrix that has no zeros along its main accident . Each diagonal eleme nt is work for, and an approximate value plugged in. The process is then repeatd until it converges. This algorithm is a stripped-down version of the Jacobi transformationmethod ofmatrix diagnalization.The Jacobi method is easily derived by examining each of theequations in the linear system of equationsin isolation. If, in theth equationsolve for the value ofwhile assuming the other entries ofremain fixed. This giveswhich is the Jacobi method.In this method, the order in which the equations are examined is irrelevant, since the Jacobi method treats them independently. The definition of the Jacobi method can be expressed with matricesaswhere the matrices,, andrepresent the diagnol, strictly lower triangular, andstrictly hurrying triangularparts of, respectivelyConvergence- The standard convergence condition (for any iterative method) is when thespectral radiusof the iteration matrix(D 1R) D is diagonal component,R is the remainder.The method is guaranteed to converge if the matrix Ais strictly or irreduciblydiagonally dominant. Strict row diagonal dominance means that for each row, the absolute value of the diagonal term is greater than the sum of absolute value of other termsThe Jacobi method sometimes converges even if these conditions are not satisfied.2. Gauss-Seidel method- The Gauss-Seidel method is alike the Jacobi method, except that it uses updated values as before long as they are available. In general, if the Jacobi method converges, the Gauss-Seidel method will converge sudden than the Jacobi method, though soothe relatively slowly. The Gauss-Seidel method is a technique for solving theequations of thelinear system of equationsone at a time in sequence, and uses previously computed results as soon as they are available,There are two important characteristics of the Gauss-Seidel method should be noted. Firstly, the deliberations appear to be serial. Since each component of the new iterate depends upon all previously computed components, the up dates cannot be done simultaneously as in theJacobi method. Secondly, the new iteratedepends upon the order in which the equations are examined. If this ordering is changed, thecomponentsof the new iterates (and not alone their order) will also change. In terms of matrices, the definition of the Gauss-Seidel method can be expressed aswhere the matrices,, andrepresent thediagonal, strictly lower triangular, and strictly amphetamine triangularparts of A, respectively.The Gauss-Seidel method is applicable to strictly diagonally dominant, or radial supportive definite matrices A.Convergence-Given a square system ofnlinear equations with unknownxThe convergence properties of the Gauss-Seidel method are dependent on the matrixA. Namely, the procedure is known to converge if eitherAis symmetricpositive definite, orAis strictly or irreduciblydiagonally dominant.The Gauss-Seidel method sometimes converges even if these conditions are not satisfied.3.Successive Overrelaxation method-The su ccessive overrelaxation method (SOR) is a method of solving alinear system of equationsderived by extrapolating thegauss-seidel method. This extrapolation takes the form of a weighted average between the previous iterate and the computed Gauss-Seidel iterate successively for each component,where cites a Gauss-Seidel iterate andis the extrapolation factor. The idea is to choose a value forthat will accelerate the rate of convergence of the iterates to the solution.In matrix terms, the SOR algorithm can be written aswhere the matrices,, andrepresent the diagonal, strictly lower-triangular, and strictly upper-triangular parts of, respectively.If, the SOR method simplifies to thegauss-seidel method. A theorem due to Kahan shows that SOR fails to converge ifis outside the interval.In general, it is not possible to compute in advance the value ofthat will maximize the rate of convergence of SOR. Frequently, some heuristic estimate is used, such aswhereis the mesh spacing of the discretiza tion of the underlying physical domain.Convergence-Successive Overrelaxation method may converge faster than Gauss-Seidel by an order of magnitude. We seek the solution to set of linear equationsIn matrix terms, the successive over-relaxation (SOR) iteration can be expressed aswhere,, andrepresent the diagonal, lower triangular, and upper triangular parts of the coefficient matrix,is the iteration count, andis a relaxation factor. This matrix expression is not usually used to program the method, and an element-based expression is usedNote that forthat the iteration reduces to thegauss-seideliteration. As with theGauss seidel method, the deliberation may be done in place, and the iteration is continued until the changes made by an iteration are below some tolerance.The choice of relaxation factor is not necessarily easy, and depends upon the properties of the coefficient matrix. For symmetric, positive definite matrices it can be proven thatwill lead to convergence, but we are gener ally interested in faster convergence rather than just convergence.4.Symmetric Successive overrelaxation- Symmetric Successive Overrelaxation (SSOR) has no vantage over SOR as a stand-alone iterative method however, it is useful as a preconditioner for nonstationary methods The symmetric successive overrelaxation (SSOR) method combines twosuccessive overrelaxation method(SOR) end runs together in such a way that the resulting iteration matrix is similar to a symmetric matrix it the case that the coefficient matrixof the linear systemis symmetric. The SSOR is a forward SOR sweep followed by a backward SOR sweep in which theunknownsare updated in the reverse order. The similarity of the SSOR iteration matrix to a symmetric matrix permits the application of SSOR as a preconditioner for other iterative schemes for symmetric matrices. This is the primary motivation for SSOR, since the convergence rate is usually slower than the convergence rate for SOR with optimal..Non-Stationary Itera tive Methods-1. meld Gradient method- The conjugate slope method derives its name from the fact that it generates a sequence of conjugate (or orthogonal) vectors. These vectors are the equilibriums of the iterates. They are also the gradients of a quadratic functional, the minimization of which is equivalent to solving the linear system. CG is an extremely effective method when the coefficient matrix is symmetric positive definite, since storage for only a limited number of vectors is required. Suppose we want to solve the pursuance system of linear equationsAx=bwhere then-by-nmatrixAissymmetric(i.e.,AT=A),positive definite(i.e.,xTAx 0 for all non-zero vectorsxinRn), andreal.We denote the unique solution of this system byx*.We say that two non-zero vectorsuandvareconjugate(with respect toA) ifSinceAis symmetric and positive definite, the left-hand side defines aninner productSo, two vectors are conjugate if they are orthogonal with respect to this inner product. Being conjugate is a symmetric relation ifuis conjugate tov, thenvis conjugate tou.Convergence- Accurate predictions of the convergence of iterative methods are difficult to make, but useful limit can often be obtained. For the Conjugate Gradient method, the error can be bounded in terms of the spectral condition numberof the matrix. ( ifandare the largest and smallest eigenvalues of a symmetric positive definite matrix, then the spectral condition number ofis. Ifis the exact solution of the linear system, with symmetric positive definite matrix, then for CG with symmetric positive definite preconditioner, it can be shown thatwhere, and . From this relation we see that the number of iterations to reach a relative reduction ofin the error is proportional to.In some cases, practical application of the above error bound is straightforward. For example, elliptic second order partial differential equations typically give rise to coefficient matriceswith(whereis the discretization mesh width), independent of the order of the finite elements or differences used, and of the number of space dimensions of the problem . Thus, without preconditioning, we expect a number of iterations proportional tofor the Conjugate Gradient method.Other results concerning the behavior of the Conjugate Gradient algorithm have been obtained. If the extremal eigenvalues of the matrixare well separated, then one often observes so-called that is, convergence at a rate that increases per iteration. This phenomenon is explained by the fact that CG tends to eliminate components of the error in the direction of eigenvectors associated with extremal eigenvalues first. After these have been eliminated, the method increase as if these eigenvalues did not exist in the given system,i.e., the convergence rate depends on a reduced system with a smaller condition number. The potential of the preconditioner in reducing the condition number and in separating extremal eigenvalues can be deduced by studying the approximat ed eigenvalues of the related Lanczos process.2. Biconjugate Gradient Method-The Biconjugate Gradient method generates two CG-like sequences of vectors, one based on a system with the original coefficient matrix , and one on . Instead of orthogonalizing each sequence, they are made mutually orthogonal, or bi-orthogonal. This method, like CG, uses limited storage. It is useful when the matrix is nonsymmetric and nonsingular however, convergence may be irregular, and there is a possibility that the method will break down. BiCG requires a multiplication with the coefficient matrix and with its transpose at each iteration.Convergence- Few theoretical results are known about the convergence of BiCG. For symmetric positive definite systems the method delivers the same results as CG, but at twice the cost per iteration. For nonsymmetric matrices it has been shown that in phases of the process where there is significant reduction of the norm of the residual, the method is more or less compa rable to full GMRES (in terms of numbers of iterations). In practice this is often confirmed, but it is also observed that the convergence behavior may be quite an irregular, and the method may even break down. The breakdown situation due to the possible event thatcan be circumvented by so-called look-ahead strategies. This leads to complicated codes. The other breakdownsituation,, occurs when the-decomposition fails, and can be repaired by using another decomposition.Sometimes, breakdownor near-breakdown situations can be satisfactorily avoided by a restartat the iteration step immediately before the breakdown step. Another possibility is to switch to a more robust method, like GMRES.3. Conjugate Gradient Squared (CGS).The Conjugate Gradient Squared method is a variant of BiCG that applies the updating operations for the -sequence and the -sequences both to the same vectors. Ideally, this would double the convergence rate, but in practice convergence may be much more irregular tha n for BiCG, which may sometimes lead to unreliable results. A practical advantage is that the method does not need the multiplications with the transpose of the coefficient matrix.often one observes a speed of convergence for CGS that is about twice as fast as for BiCG, which is in agreement with the observation that the same contraction operator is applied twice. However, there is no reason that the contraction operator, even if it really reduces the initial residual, should also reduce the once reduced vector. This is evidenced by the often highly irregular convergence behavior of CGS. One should be aware of the fact that local corrections to the current solution may be so large that cancelation effects occur. This may lead to a less accurate solution than suggested by the updated residual. The method tends to diverge if the starting guess is close to the solution.4 Biconjugate Gradient Stabilized (Bi-CGSTAB).The Biconjugate Gradient Stabilized method is a variant of BiCG, like CG S, but using different updates for the -sequence in order to obtain smoother convergence than CGS. Bi-CGSTAB often converges about as fast as CGS, sometimes faster and sometimes not. CGS can be viewed as a method in which the BiCG contraction operator is applied twice. Bi-CGSTAB can be interpreted as the product of BiCG and repeatedly applied GMRES. At least locally, a residual vector is minimized, which leads to a considerably smootherconvergence behavior. On the other hand, if the local GMRES step stagnates, then the Krylov subspace is not expanded, and Bi-CGSTAB will break down. This is a breakdown situation that can occur in addition to the other breakdown possibilities in the underlying BiCG algorithm. This type of breakdown may be avoided by combining BiCG with other methods,i.e., by selecting other values for One such alternative is Bi-CGSTAB2 more general approaches are suggested by Sleijpen and Fokkema.5..Chebyshev Iteration.The Chebyshev Iteration recursively determines po lynomials with coefficients chosen to minimize the norm of the residual in a min-max sense. The coefficient matrix must be positive definite and knowledge of the extremal eigenvalues is required. This method has the advantage of requiring no inner products. Chebyshev Iteration is another method for solving nonsymmetric problems . Chebyshev Iteration avoids the computation of inner productsas is necessary for the other nonstationary methods. For some distributed memory architectures these inner products are a bottleneckwith respect to efficiency. The price one pays for avoiding inner products is that the method requires enough knowledge about the spectrum of the coefficient matrixthat an ellipse enveloping the spectrum can be identified however this difficulty can be overcome via an accommodative constructiondeveloped by Manteuffel, and implemented by Ashby. Chebyshev iteration is suitable for any nonsymmetric linear system for which the enveloping ellipse does not include the origi n.Convergence-In the symmetric case (whereand the preconditionerare both symmetric) for the Chebyshev Iteration we have the same upper bound as for the Conjugate Gradient method, providedandare computed fromand(the extremal eigenvalues of the precondition matrix).There is a severe penalty for overestimating or underestimating the field of values. For example, if in the symmetric caseis underestimated, then the method may diverge if it is overestimated then the result may be very slow convergence. Similar statements can be made for the nonsymmetric case. This implies that one needs fairly accurate bounds on the spectrum offor the method to be effective (in comparison with CG or GMRES).Acceleration of convergenceMany methods exist to increase the rate of convergence of a given sequence, i.e. to transform a given sequence into one converging faster to the same limit. Such techniques are in general known as series acceleration. The inclination of the transformed sequence is to be much less expensive to calculate than the original sequence. One example of series acceleration is Aitkens delta -squared process.