Troubleshooting call quality degradation for VoIP phones when using handset versus speaker

Phones are a combination of Polycom VVX150 and VVX 201. When using the speaker, calls are perfect, when using the handset, the voice quality degrades due to something network related. Location is using VLANS, has a Velocloud router, and uses Zscaler for a proxy.

Does anyone know what packet header information is related to how the calls are treated internal to the phone? Apparently, at least with the Polycom phones, packet headers do contain information about whether they are to/from the speaker or the handset within a VoIP phone and something on the network is treating the handset packets different than the speaker packets.

Thanks in advance for any suggestions.