The question is for the finite-dimensional spaces. One wants, for a unitary operator, to see the product of how many 'symmetries' result in the unitary operator. The motivation is exactly like any other, such as the prime decomposition of rings. We emphasize the von Neumann algebra. Consider (bounded, in this case) unitary involutions (self-adjoint and it will imply a symmetry) $Q^* Q = QQ^* = \mathbb{1}$ and $Q^2=1$.

Theorem [Halmos-Kakatani, 1958 and Fillmore, 1966] - In an infinite dimensional von Neumann algebra $\mathcal{M}$, every unitary symmetry can be decomposed into four symmetries.

In this, one basically says that a set of unitaries $S^4(\mathcal{M})$ will decompose to product four symmetries in $\mathcal{M}$. The proof can be found in Halmos-Kakatani. But this will assume that we have an infinite dimensional algebra (like type $II_\infty$, type $I_\infty$, type$ III$). For finite-dimensional cases, like type $II_1$ one runs into a determinant of unitaries which are $\pm 1$. For this, Radjavi has a theorem for type $I_n$ with matrices in $M_n(\mathbb{C})$ with determinant $\pm 1$ saying that every unitary can be decomposed into four symmetries. But the determinant here has a catch of a slightly different definition of a central-valued determinant. You may refer to this paper by Bhat and Radjavi. But one safely says that in type $I_n$, we can decompose all the unitaries into finitely many symmetries (also see Broise, 1967).

For type $II_1$, [where we have a maximal entropy state], the story is a little different than type $I_n$ for where $S^4(\mathcal{M})$ is norm-closed and not norm-dense in the set of unitaries.

Theorem [Bhat, 2022] - In type $II_1$ algebra $\mathcal{M}$, every unitary can be decomposed into six symmetries in $\mathcal{M}$.

Theorem [Bhat] - For the type $II_1$, if any unitary operator $Q$ has a finite spectrum, it can be decomposed into four symmetries $\mathcal{M}$.

Now, why not three symmetries? The answer is that the product of the symmetries is not norm-dense in the set of unitaries. This can be verified for any von Neumann algebra.

So, the answer to 'Are four symmetries enough', and I don't know one, is that you don't know and it is somewhere between four and six. It is interesting to see such questions (and many are open questions) in these works.