If all your devices are functioning just the way you want in an all HDMI environment, then by all means carry on. Our goal isn't to convince you to switch from HDMI cables to TOSLINK.
For the majority of consumer setups, there will be absolutely no discernible difference between audio quality when using an HDMI cable or a TOSLINK cable. The TOSLINK system is still capable of carrying up to 7.1 channels of very high-resolution audio.
