If one of the lines given by the equation \(2x^2 + axy + 3y^2 = 0\) coincides with one of those given by \(2x^2 + bxy - 3y^2 = 0\) and the other lines represented by them are perpendicular then what…

The sum of the squares is $26$. Write the common line as $x=ry$ and the other two as $x=uy$ and $x=vy$, so the root products give $ru=\frac32$ and $rv=-\frac32$, hence $v=-u$. Perpendicularity gives $uv=-1$ and $u=\pm1$, so $r=\frac32u$ and the root sums give $a=-5u,\ b=-u$. The value $36$ includes the cross term $2ab$ from squaring $a+b$.