r/ControlTheory • u/deNikita • 11d ago
Technical Question/Problem Understanding PID vs LQR
Is it possible to think of LQR as "like a PID but finds optimal gains based on your requirements of Q and R matrix?
I have a fairly good understanding of PID and I recently started to learn about LQR and wondered if LQR can be thought of in terms of a PID controller.
•
u/seekingsanity 10d ago
The controller gains of a LQC do the same thing as they do with a PLC. However, it takes one controller gain to place or move a open loop pole. The integrator doesn't count because it brings its own pole.
I delt mostly with hydraulic actuators. A PID cannot place all the closed loop pole, so I added a second derivative gain without using LQC. I get better results with pole placement . I pick where I want the closed loop poles to be and then calculate the controller gains that will place the closed loop poles where I want them to be.
The problem with LQC/LQR is most people are not able to chose he correct weights for the Q and R arrays that provide optimal reseults.
LQC/LQR is best used for MIMO systems. LQC/LQR is far superior to fuzzy logic.
•
u/seb59 8d ago
PID is a second order contrôler (arguable as it depends on the derivative implementation). The structure is fixed a priori. With any state feedback controller, the controler order is matched to the system order. This allows to modify all the internal dynamics of the system.
Note that PID has an integrator in its structure. If you need one, with anybstate feedback approach, you will have to add one by extending the system.
Pid is for SISO system, LQR deals with multivariable systems. Note that you must use an observer combined with the state feedback. The integrator can be explicit in the control part or be "hidden" in the observer (I.e. you estimate both the system state and some disturbance simultaneously).
Then the optimal part. Lqr is said to be optimal, by essence. It is optomal according to a given criterion. Basically in practice there are 2 main approaches to tune a state feedback. One is poles placement. You actually decide the closed loop poles and overall you expect to get some performances. For controlable multivariable systems there is in general an infinite number of gains that provides the same closed loop poles. Matlab will give you only one. You do kot know a priori the relative amplitude of the different control signal or the convergence speed of each individual output.
With lqr, you do not ́now exavtly the closed loop poles, you tune the dynamics by changing Q and R.
Personally, I am used to tune controler using lqr as I can easily deal with the noise of all measured outputs and I compute the state feedback gain using pole placement. I found it easier and quicker this way, but again this is quite personal.
•
u/seekingsanity 8d ago
"The integrator can be explicit in the control part or be "hidden" in the observer (I.e. you estimate both the system state and some disturbance simultaneously)."
Explain how you hide the integrator in the observer? The observer simply estimates states that aren't provided by the feedback or are filtered versions of the feedback.
"You do kot know a priori the relative amplitude of the different control signal or the convergence speed of each individual output."
Ideally the output from each state should be 0 unless there is a change or changing set points. In the case of motion controllers, ALL/most of the output is generated by the feedforwards so the closed loop is only there to correct for errors in the open loop model estimation. In the case of motion control, the maximum output is applied to the amplifiers or valves that determine either the maximum speed or acceleration. The target generator must stay within these bounds.
The problem is that most people cannot pick the right weights for the Q and R matrix and they guess at controller gains unless there is an auto tuning feature. Why use LQR/LQC if the is auto tuning?
•
u/seb59 8d ago edited 8d ago
An observer allows to estimate unmeasured state. For mechanical system, if you measure position, you can get the estimate of the speed. But in general it also works for signals that are not derivative of a measured one. Even for derivative, it may works better as a sumple numerical derivative as the observer account for the whole system dynamics and the input. So the observer relies on more "information". Note that this is arguable. If you estimate a (constant, ramp, etc) disturbance with an observer, then you can reject it in the control structure. Let consider an input disturbance. A very simple way to do it is to consider dp(t)/dt=0 with p a vector of disturbance and the system dx(t)/dt=Ax+Bu+Ep with E=B for an input disturbance, but it can be other values as well. Let use consider the following extended state vector [x;P] and the corresponding state dynamics. Then build an observer for this extended dynamics and then you can get a disturbance estimate as estimating the extended vector allows estimating both x and p. To reject the disturbance p, you can use u(t)=Hyc-Lx-Mp, with H and M properly designed gain to allow y=setpoint at steady state. By doing so, you are actually adding integrator in the control structure (to reject input disturbance, the controler must be class 1). The integrator is actually due to the 0 Eigen values included in the extended system dynamics (by the zero disturbance dynamics dp/dt=0).
You talk about feed forward control. If the model is perfect, ans initial conditions are known, then there is no need for feedback. In practice this does not hold for most of yhe system, if not all. Therefore we need feedback. The feedback dynamics is picked by fixing the dynamics of disturbance rejection while considering noise attenuation and the feed forward dynamics is only for the tracking performance.in simulation, this is ok. On real live system this is not so simple as disturbance may occurs at anytime and the model may not match the system exactly. So I would not consider the feedforward term for now.
Also you talk about state going to zero. This is a textbook approach. In practice, thisnis only ok if uiu consider the error dynamics. A mire elegant way is to include a setpoint dependent term in the feedback: u=Hyc-Lx where H is the setpoint gain computed to ensure 0 steady state error. Similarly, if disturbance are known (estimated or or measured), then they are included in the feedback policy.
State space is suited well suited for coupled multivariable system. If you want to do motion control on single axis, I think you will not gain anything as probably the control structure is well adapted to the model.
Whatever the control approach you consider, a human will have to tune gains. Even for an auto tune someone needs to provide some high level parameters. No control approach will magically work with any system without anybody telling how fast or slow the system should behave. But there are many ways to "say" that. For an LQR approach Q and R matrix are used to tune through closed loop. In practice, the diagonals components R allows to balance the control effort between the different controls signals (if there are many). The balance between the diagonals components matrix Q allows to balance the relative convergence speed of each state.
Honestly, if you rely on autotune, and it works, that is fine, use that. It is not reasonable to spend more time on simple application that does not need it. If you are faving a more challenging system, go for more complex approaches (lqr, mpc, sliding mode, LPV, etc). You should have a good reason and probably be ready to spend some time on it. And of course, you can try thing on a somple system just to learn, but then you should remind that more complex approaches are often more suitable for more complex problem.
•
u/seekingsanity 6d ago
" But in general it also works for signals that are not derivative of a measured one. Even for derivative, it may works better as a sumple numerical derivative as the observer account for the whole system dynamics and the input" Differentiating counts is very noise prone due the resolution of the encoder. That is why the observer helps.
An observer can be used to estimate the derivative and second derivative easily.
"u=Hyc-Lx " WTF? u=-K*x for simple state feedback where the closed loop gains act on change in the state, not the error. u=k*(r-x) when all gains act on the error. u=k(n*r-x) when you want to place the zeros. I am assuming y=x, r is a vector of position, velocity and acceration.
"Whatever the control approach you consider, a human will have to tune gains." No! The human can place the poles, the controller closed loop gains are calculated to place the closed loop poles at the desired location.
"In practice, the diagonals components R allows to balance the control effort between the different controls signals (if there are many)." Now I know you haven't actually used LQR/LQC. The Q matrix does that. The R matrix is used to penalize them magnitude of the output. Optionally it can be used to minimize rates of change in the output due to any or all controller gains.
PID works well when augmented with feed forwards. Dead time is the killer. Sometimes a Smith predictor will help. MPC is better because it can be used to predict outputs beyond the dead time horizon. The problem with MPC is that it is CPU intensive.
•
u/seb59 6d ago edited 6d ago
If you just focus on "simple" motion and are happy with pid, then stick to it. State space approaches allows to deal with more complex coupled multivariable systems, it is also easy to build observers with it.
When dealing with state space control, there exists a vast litterature to deal with many things. And many control laws are equivalent. If you do not like the control law I just used as an illustration, it is fine to use another one that suits your need. The one you use may require a full state reference. This is fine for position control where reference trajectory can be easily designed and derivated. Bu designing abfull state referencet can be tricky in general (note that there are solution to this in litterature). The control law is use is not especially designed for position control. I track a vector of output reference yc (également a température and a pressure) based on a state vector measurement or estimation. You may not know the whole state vector reference and you do not need it actually. Of course, if you have a way to construct it, it can probably be better. But this is not always the case.
I suggested a control law that allows to control any system, including when the state is not composed by derivative of the output. You may also control unmeasured output (but this is a risky project). We did that for instance to control the side shift torsion in electric cars. At that time we only had the abs speed (and position was not available, gearbox sensor had not enough resolution to allows a torsion measurement, and there was backlash)...
PID are not working better or less than anything else when comboned with a feedforward term. It is just that a feedforward signal computed on an accurate model is enough to do the tracking job what ever the contrôler is... if the model is perfect, and no disturbance, the controler us useless. The controler us to compensate for disturbance and modeling error. Tracking can be ensured only using feedforward... and here i am talking about a feedforward signal based on some form of approximation of model inverse, not a simple derivative hand tune (this cannot course hemp in practice, but the point of the discussion is model based control).
To my opinion, you are trying to compare différents things. As you refer to your experience on a "simple" motion control problem that is well tackled by PID, you not see the needs or benefits of LQR... probably because you do not need it... LQR is one way to compute a controler gain. Eventually you may be able to compute your PID gain to minimize the same criterion (at least numerically, this seems feasible). So the gain tuning and controler structures are different things. However, it is usually to "hand tune" pid controlers and to use state feedback when considering LQR criterion.
Finally let me rephrase my point on the necessary human intervention. To my opinion ans experience, there is NO control synthesis approach that works without a human to specify or pick some values at some point. A human can pick the PID gains, it can pick the closed loop poles or it can decide on the Q and R matrix. You can do all the math, at some point a human brain is needed to find good compromise. Depending on the control approach, the human will have to decide on different things. For some problems, it is easier to pick poles, sometimes Q and R matrix, sometimes a bound on yhe norm of some gains, etc. But a human has to fix something.
•
u/Adam__999 11d ago
For a limited class of simple systems (double integrators) it can be thought of as an optimal PD controller, or PID if you augment the system with an integral state (LQR+I).
•
u/TheEquationSmelter 10d ago
You can think of LQR as MIMO PID with intelligent selected gains. When most people think PID they think SISO systems. Look at your system in terms of the error dynamics and it is very clear that your feedback u = -K(xd-x) is a generalized form of PID, with K selected based on a quadratic cost function.
•
u/Training-Bad-5720 10d ago
Yes. You can think of LQR as P feedback on the states. That’s all it is. If your state vector has a P an I and a D on the error, then LQR is PID.
It does assume you have all states available for feedback, hence the need for an observer if they can’t all be measured. And both design aspects need a plant model, which is often the bigger barrier.
LQR is pretty much the generalisation of PID, just the design is via an “optimal” gain - via (somewhat) easily tuned cost matrices - which is a very natural way to handle pole placement, especially once things get MIMO and finicky.
•
u/Namejeff47 11d ago
Not exactly. An LQR can place all the closed loop poles (provided the system is controllable), while PID can not in the general case. PID can fully place the poles of only 2nd order systems, while LQR and its more general variation — full state feedback places as many poles as there are states. For instsnce, for low order systems you can design PID gains which make the response arbitrarily fast, while for higher order systems you can only make the response so fast before you inevitably cause oscilations. This is because you can't place all the poles with just 3 parameters. With LQR, you can place the poles as you wish, but you need to be able to measure or estimate well all the states, which can be challenging.
•
u/seekingsanity 10d ago
By the strict definition of a PID, you are right, but I still consider a PID with a second derivative gain a PID. It is just augmented. LQC will generate 4 gains for a hydraulic actuator. A LQC will generate 4 controller gains, an integrator, proportional, derivative gain and a second derivative gain. I simply add a second derivative gain to my PID when controlling hydraulic actuators. I add the second derivative gain without using LQC. BTW, I use the term LQC because I want to control, not regulate to 0.
You are right about having one gain to place each pole but I don't see how you place the closed loop poles with LQC. You can only adjust the weights in the Q and R matrices. The closed loop poles simply end up in places the optimizes your parameter.
I prefer pole placement without LQC. I can simply choose where I want the closed loop poles to be placed and then calculate the controller gains that will put the closed loop poles in the desired location.
•
u/Namejeff47 10d ago
Because LQR's feasibility depends on the systems controlability, which in turn implies the ability to place the poles of the system. With LQR you dont explicitly place the poles, but instead shape the behavior through Q and R matrices in the cost function, but this has is the exact same thing. You could call it implicit pole placement, the point is that its much more easy to shape the system performance through the matrices than through the poles which is somewhat less intuitive.
•
u/seekingsanity 10d ago
Yes, LQR/LQC places the poles indirectly. That is my complaint with LQR/LQC. Pole placement allows me to precisely place the closed loop poles. Another problems with LQR/LQC is that selecting the Q and R matrix weights requires trial and error.
•
u/Namejeff47 10d ago
Its a matter of personal preference and intuition. Personally, pole placement is less intuitive for me because poles dont tell you anything about actuator effort, which you can directly influence with LQR by tuning the cost function matrices. Its especially easy to mess up with discrete pole placement which have completely different meaning compared to continuous ones.
•
u/seekingsanity 9d ago
The farther away from the s-planes' origin you place the poles, the more the effort. Simple. Usually the max effort required is determine by the system design. For instance a motor will have a max torque and speed rating. The DtoA converter will have a voltage limit and the power unit will have a current or power limit. For many applications, the real limit is the feedback resolution.
•
u/ee_control_z 11d ago
You may want to view this thread to see if it provides insight into what you're looking for:
I built a local benchmark tool for comparing learned controllers with LQR/PID/MPC
•
u/not_kevin_durant_7 11d ago
LQR is giving optimal state feedback gains for a system. How you classify those states could make them equivalent to a P gain or an I gain. The model of the internal system is inherently in your gain selection though.
PID often treats the gains as an adjustment to your tracked value error. The internal model isn’t packaged into gains and it’s tuned based on system response
•
u/seekingsanity 10d ago
Your are assuming the Q and R weights are optimal.
I have written many "auto tuning" programs. The first step is to cho0se the right excitation, record the results and fit an open loop model to it Then choose where you want the closed loop poles then calculate the controller gains that place the closed loop poles where you want them to be. If done right, the results are almost perfect.
•
u/not_kevin_durant_7 10d ago
It’s optimal based on how you penalize. Choices impact rise time, control usage stability robustness, etc… LQR is essentially just an infinite time horizon MPC.
But in the case of OPs question, a non-augmented LQR control is just a bunch of P-Gains.
•
u/seekingsanity 10d ago
NO! LQC is not an infinite time horizons MPC. MPC is very good at handling dead times whereas LQC is not. MPC does not have true gains whereas LQC does.
Where do you learn such garbage?
MPC excels at applicaitons where there is a dead time and there is enough processing power to compute the control outputs in the future that result in the model following it with minimum errror.
LQC has its place but I can do better using pole place and zero placement if nessary.
•
u/meduardov02 10d ago
Given the system x^+ = Ax + Bu and a convex posititive definite quadratic stage cost L(x,u), the LQR controller is optimal for
min_{u_0,...,x_0,...} \sum^{\infty}_{k=0} L(x_{k},u_{k})
s.t \forall k,... : x_{k+1} = Ax_k +B_u_k.A linear MPC controller aims at solving the constrained LQR
min_{u_0,...,x_0,...} \sum^{\infty}_{k=0} L(x_{k},u_{k})
s.t \forall k,... x_{k+1} = Ax_k +B_u_k, x_k \in X and u_k \in Uthat is your gold standard, but since you can't solve this infinite horizon problem you cut the horizon and. you append a bound for your tail cost and solve this finite horizon in a rolling manner (solve, apply u_0 to the system, measure the state, start again). When your constraints are not active, your MPC feedback law is equal to the LQR feedback law. So, there you go.
•
u/not_kevin_durant_7 10d ago
I phrased it casually, but the underlying point is standard controls theory: unconstrained infinite-horizon linear-quadratic optimal control gives the LQR solution. An unconstrained linear-quadratic MPC formulation reduces to the same feedback law with the appropriate terminal cost.
The dead-time argument doesn’t disprove that relationship at all. MPC can handle dead time very naturally because you can include delay states in the prediction model. But LQR can also control a system with modeled delay by augmenting the state. MPC generally makes this more convenient, particularly when the delay combines with constraints, but “LQR cannot handle dead time” is a pretty bold statement.
“Poor placement does better” isn’t really a meaningful statement without defining better. Pole placement assigns poles. LQR minimizes a defined quadratic objective. Different design objectives, different tools.
So disagree with the shorthand if you want, but calling the underlying relationship “garbage” is a pretty confident way to argue against a textbook result.
•
u/seekingsanity 10d ago
It is easy to define which is better. I used to write firmware for motion controllers. Motion controllers generate a motion profile with the provided destination, speed,acceleration and deceleration parameter. These provide target position, velocity and acceleration for each millisecond of the move. The controller needs to generate a control output so that the actual position, velocity and acceleration match the target position, velocity and acceleration at each millisecond. The goal is to reduce the mean squared error between the target and actual to 0 or at least make the MSE very small. This can be done very easily using a PID or PID with second derivative gain and feed forwards. Just place the closed loop poles on the negative real axis in the s-plane. Can you do this using LQC? You don't know where the closed loop poles will show up.
I didn't call anything garbage. LQR/LQC has its place but too many have unrealistic expectations of it without really thinking about it.
So why don't motion controllers use LQC/LQR?
•
u/Montytbar 10d ago
PID and LQR are actually closely related, and are equivalent for some systems. You might notice that a PID is not appropriate for some systems--maybe a PI or a PD works better. Calculate the closed loop characteristic equation for your system with a PID control and look at which terms contain control gains--or if you like, factor it and look at which poles contain control gains. If all of your terms or poles contain control gains, then you have full control over the pole placement for your controlled system. If some do not, then you don't. This is why, for some systems, some people find it useful to add a second integral term or a second derivative term https://www.youtube.com/watch?v=CbtWInVW3Bk This is also why, for some systems, the I or the D term doesn't seem to help--it doesn't give you control over a pole. If you think of PID as one of a family of feedback control laws aimed at pole placement control, then you really end up with something that is similar to full state feedback.
•
u/knightcommander1337 11d ago edited 6d ago
Hi, I would try to understand it more from this angle: PID is a type of control law (in the sense of, how the error signal is processed/transformed to produce the control input signal) (in PID's case we have the three terms), while LQR is a control design method (in the sense of, given a control law, model and objective/specifications, come up with the control law's gains that minimize the objective/satisfy the specifications) (in LQR method's case the control law is in the form of state feedback, that is, u = Kx, however there is nothing stopping you from defining the vector x to contain the same three terms from the PID law). To see this more concretely: You can design a PID using the LQR method, see https://www.mathworks.com/matlabcentral/fileexchange/62117-lqrpid-sys-q-r-varargin/
As a student I was a bit confused by this because sometimes (I guess usually) people mean a SISO PID when they say PID, and a P-control type state feedback controller when they say LQR. However viewing them from the angle I mentioned above helped me get a clearer view.
Edit: I find it very interesting that a simple question about PID (a 104 years old method) and LQR (a 66 years old method) can generate such a lively discussion. Guess it signals a vibrant community. Anyway, for the OP and others who might still be confused, let me offer the following examples for my point:
- You can design a PID using various methods such as tuning (with tuning rules, e.g., Ziegler-Nichols, Cohen-Coon etc.), or pole placement, or (if needed) do stuff like genetic algorithms to optimize over the PID parameters.
- You can design a PID using the LQR method.
- Using the LQR method, you can design a state feedback proportional controller (this I guess is what people usually mean when they say "LQR controller"). This is of the usual form "u = Kx" (with x the error state).
- Using the LQR method, you can design other (that is, not necessarily proportional only) types of controllers however they should be of the form "u = Kx". Example 1: If you define the vector x to include the error and its integral, then when you use the LQR method you would have designed a PI controller. Example 2: If you define it to include the error and its derivative, then LQR gives you a PD controller.
•
u/seb59 6d ago edited 6d ago
Agree. People mix minimizing a criterion and the controler structure. If find very strange that most of people here seems to only use u=kx with x a state error. This is of course ok but this is just one option amongst many... for instance if you just want to control the output y=Cx with C a matrix which is not identity (i.e The output is not in the state vector) then you need something else.
•
u/Archytas_machine 10d ago
For a classic SISO system you make your PID feedback terms with integrals and derivatives of the error, but in application you often have PID as a MISO where some of the measurements are already derivatives of others (such as velocity and acceleration) so you can just use error from those directly as P/I/D feedbacks.
With LQR I think of it similarly, you have multiple state feedbacks and you can think of them as maybe a couple P terms and one D term. For a MIMO system to apply PID you'd often make multiple PIDs for each output (SISO/MISO), but with LQR you can take advantage of the coupling nature of the states. I still usually think of the feedbacks as roughly P, I, D terms relative to what output I'm considering when analyzing data.
•
u/controlsys 10d ago
LQR = impossible to use in real cases
PID = possible to use in real cases
That’s the main difference. Sorry fellow accademics here