HOMOGENEOUS EQUATION: Each equation of the system of following linear equations
a_{11}x_1+a_{12}x_2+a_{13}x_3=0\;\;\;\;\;\;\;\;\;\;\;\;\;(1).
a_{21}x_1+a_{22}x_2+a_{23}x_3=0\;\;\;\;\;\;\;\;\;\;\;\;\;(2).
a_{31}x_1+a_{32}x_2+a_{33}x_3=0\;\;\;\;\;\;\;\;\;\;\;\;\;(3).
is always satisfied by x_1= 0,\;\; x_2= 0\;\; and \;\;x_3 = 0, so such a system is always consistent. The solution (0, 0, 0) of the above homogeneous equations (i),\;\; (ii),\;\; and\;\; (iii) is called the trivial solution. Any other solution of equations (i),\;\; (ii)\;\; and\;\; (iii) other than the trivial solution is called a non-trivial solution.
The above system can be written as
AX = O\;\; where \;\begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix}=\begin{bmatrix}0\\0\\0\end{bmatrix}
If \left|A\right|\neq0, then A is non-singular and A^{-1} exists, that is
A^{-1}(AX)=(A^{-1}A)X=OX=O
\;\;(A^{-1}A)X=O
⇒X=O
i.e,\;\begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix}=\begin{bmatrix}0\\0\\0\end{bmatrix}.
In this case the system of homogeneous equations possesses only the trivial solution.
Now we consider the case when the system has a non-trivial solution. Multiplying the equations (i), (ii) and (iii) by A_{11},\;\; A_{21}\;\; and A_{31} respectively and adding the resulting equations (where A_{11}, A_{21} and A_{31} are cofactors of the corresponding elements of A),
we have
(a_{11}+A_{11} + a_{21} A_{21}+a_{31} A_{31})x_1+(a_{12} A_{11} + a_{22} A_{21}+a_{32} A_{31})x_2 +(a_{13} A_{11} + a_{23} A_{21}+a_{33} A_{31})x_3= 0.
that is,
\left|A\right|x_1=0 .
Similarly, we can get
\left|A\right|x_2=0\;\;\; and\;\;\; \left|A\right|x_3=0
For a non-trivial solution, at least one of x_1 ,\; x_2\; and x_3 is different from zero
Let x_1 ≠ 0 , then from \left|A\right|x_1=0., we have \left|A\right|=0.
For example, the system
\left.\begin{array}{l}x_1+x_2+x_3=0\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;(i)\\x_1-x_2+3x_3=0\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;(ii)\\x_1+3x_2-x_3=0\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;(iii)\\\end{array}\right\}.
has a non-trivial solution because
\left|A\right|=\begin{vmatrix}1&1&1\\1&-1&3\\1&3&-1\end{vmatrix}
⇒\left|A\right|=\begin{vmatrix}1&0&0\\1&-2&2\\1&2&-2\end{vmatrix}.
by \;\;C_2 -C_1, C_3-C_1.
⇒\left|A\right|=\begin{vmatrix}-2&2\\2&-2\end{vmatrix}
⇒\left|A\right|=0
Adding (i) \;and\; (ii) , we get
\;\;\;2x_1+4x_3=0
⇒x_1=-2x_3
And subtracting (ii) \;and \;(i) , we get
\;\;\;2x_2 - 2x_3 = 0
⇒ x_2 = x_3
Putting x_1 = -2x_3 and x_2 = x_3 in (III), we see that (-2x_3) + 3(x_3) - x_3= 0, which shows that the equation (i), (ii) \;and \;(iii) are satisfied by
x_1 = -2t, x_2= t \;\;and \;\;x_3= t for any real value of t.
Thus the system consisting of (i), (ii) \;and \;(iii) has infinitely many solutions.
But the system
\left.\begin{array}{l}x_1+x_2+x_3=0\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;(i)\\x_1-x_2+3x_3=0\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;(ii)\\x_1+3x_2-2x_3=0\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;(iii)\\\end{array}\right\}.
has only the trivial solution.
because in this case
\left|A\right|=\begin{vmatrix}1&1&1\\1&-1&3\\1&3&-2\end{vmatrix}
⇒\left|A\right|=\begin{vmatrix}1&0&0\\1&-2&2\\1&2&-3\end{vmatrix}.
by \;\;C_2 -C_1, C_3-C_1.
⇒\left|A\right|=\begin{vmatrix}-2&2\\2&-3\end{vmatrix}.
⇒\left|A\right|=6\neq0.
Solving the first two equations of the above system, we get
x_1= -2x-3 and x_2= x_3.
Putting x_1 = -2x_3 and x2= x_3 in the expression x_1+ 3x_2 -2x_3
we have,
- 2x_3 +3(x_3) - 2x_3= - x_3,
that is, the third equation is not satisfied by putting
x_1= -2x_3 and x_2= x_3
But it is satisfied only if x_3=0.
Thus the above system has only the trivial solution.
you can also check homogenous function.