# Online Econometrics Textbook - Regression Extensions - Restricted Least Squares and Restricted Maximum Likelihood

 Sometimes it is necessary to impose linear constraints on OLS estimation. This case of OLS estimation is called restricted least squares (RLS). Assume that the constraints are contained in (III.V-1) where b is the RLS parameter vector (K*1), r is a (H*1) vector, R is a (H*K) matrix, K is the number of parameters, and H is the number of restrictions. A solution to RLS is only possible if H is smaller than K and if the rank of R is equal to H. Otherwise, two or more linear restrictions are linearly dependent or inconsistent. Suppose K = 3, H = 2 and we wish to restrict the least squares estimates according to (III.V-2) then eq. (III.V-1) applied to (III.V-2) would look like (III.V-3) from which it can be clearly seen that R has rank H = 2. The Lagrangian function can be deduced as follows (III.V-4) which can be derived w.r.t. the parameter vector and the Lagrange multiplier vector (III.V-5) (III.V-6) On combining (III.V-5) and (III.V-6) we obtain (III.V-7) (III.V-8) which is the RLS estimator. The SSR are computed by the following formula (III.V-9) which can be written more explicitly (III.V-10) or on using X'(y - Xb) = 0 (III.V-11) or simply (III.V-12) This result is very important since it can be used in tests for linear restrictions. First we deduce from the RLS estimator (III.V-8) that (III.V-13) If we define (III.V-14) (for which proof is quite easy), we may write (III.V-15) Now since (III.V-16) (III.V-17) Since (III.V-18) a general test for linear restrictions is thus given by (III.V-19) (it can be shown that the nominator of (III.V-19) is independent of the denominator). Obviously, restrictions on parameters cannot only be imposed on least squares estimators. There is an equivalent approach with respect to MLE, the so-called restricted maximum likelihood estimation (RMLE). The greatest advantage of RMLE over RLS is that it is possible to impose nonlinear restrictions on the parameters. The most important disadvantage however might be the computational (numerical) complexity of RMLE.

No news at the moment...

© 2000-2012 All rights reserved. All Photographs (jpg files) are the property of Corel Corporation, Microsoft and their licensors. We acquired a non-transferable license to use these pictures in this website.
The free use of the scientific content in this website is granted for non commercial use only. In any case, the source (url) should always be clearly displayed. Under no circumstances are you allowed to reproduce, copy or redistribute the design, layout, or any content of this website (for commercial use) including any materials contained herein without the express written permission.

Information provided on this web site is provided "AS IS" without warranty of any kind, either express or implied, including, without limitation, warranties of merchantability, fitness for a particular purpose, and noninfringement. We use reasonable efforts to include accurate and timely information and periodically updates the information without notice. However, we make no warranties or representations as to the accuracy or completeness of such information, and it assumes no liability or responsibility for errors or omissions in the content of this web site. Your use of this web site is AT YOUR OWN RISK. Under no circumstances and under no legal theory shall we be liable to you or any other person for any direct, indirect, special, incidental, exemplary, or consequential damages arising from your access to, or use of, this web site.

Contributions and Scientific Research: Prof. Dr. E. Borghers, Prof. Dr. P. Wessa
Please, cite this website when used in publications: Xycoon (or Authors), Statistics - Econometrics - Forecasting (Title), Office for Research Development and Education (Publisher), http://www.xycoon.com/ (URL), (access or printout date).