The (subtly) crunchy, heavy, compressed (both seemingly dynamically and data-wise) quality to the sound of the SC5000 M and non-M appear to be the result of problems both with their main audio processing and further with the SPDIF outs. Right now there’s not much we can do about the main audio processing but wait for a firmware improvement.
I think there’s an issue with the sample rate and bit depth conversions from the aforementioned main processing on them to the SPDIFs that are fixed-point 24bit/96khz. If the main processing on the players is not that format, which it probably isn’t and could be 32bit float and/or heavily oversampled (very common methods), then it appears to be doing poor-quality real-time conversion from that to the digital outputs that can’t be any higher than 24/96 and have to be fixed-point math. There may also be a compressor/limiter function on the main processing and/or the SPDIF, too. For now you can mitigate some of this by using the analog outputs instead even to a digital mixer. Since you’re already using a mixer without digital inputs, for the time being again you’re stuck with this until it’s improved. You are using your players in the most optimal manner possible, at least. I don’t know if that’s going to feel like good or bad news for you.
FWIW, the X1800’s fidelity is not so great compared to the old x1700, the NXS2, and a lot of quality analog mixers, even. So you might want to keep that in mind if you’re thinking about getting one, as it’s basically more of the same you don’t like on the players’ sound heaped on top… other than roll off – the mixer is flat and doesn’t make that worse. You’ve got a nice mixer without digital inputs, apparently, so it might be worth not upgrading until the X1800’s firmware is improved… hopefully that is enough to boost its sound quality and it’s not a poor quality hardware clock or power supply to blame on the mixer design. The integration between the players and the mixer is really amazing, so it’s worth looking into at some point in the future. Admittedly, I’ve used mixers that sound worse than the X1800, so YMMV.
The particular dark, overly-warm sound quality on the players is the result of a surprisingly-low-corner-frequency low-pass filter with a steep order. It’s like the players are only 32khz instead of actually 96khz. This affects all the outputs, so using the analog ones doesn’t avoid this, either. Denon DJ has promised to try and improve the high frequency reproduction on a future firmware. Hopefully this is not the only thing they improve in the sound quality department on the Prime system.
The platter tension knob just doesn’t work as advertised. The jogs also have some variation in their feel & resistance. All I use the tension knob for is to bring jogs that are thankfully low in resistance and high in smoothness up to the “lowest common denominator” of the higher resistance, less smooth jogs that I can’t get smoother or more free-spinning. That way at least they feel pretty consistent from deck to deck. There are some other threads on this topic including a guy that cleaned his and lubed the friction thing in there with silicone grease.
I don’t use quantize or sync, so I can’t comment on those.
As for customer support, I don’t know how long you’re been trying to contact them, but they’re usually communicative. Today is a holiday in the USA and they’re also not open on weekends. Might have something to do with it.