Agree with Chip Zoller.
Assuming you can afford downtime (network disruption on that VM), maybe this one is easier to follow.
Create a new vSwitch and portgroup on the source host that matches the vSwitch and portgroup on the destination host.
You don't need to attach the vSwitch to any vmnic/uplink.
Reconnect that VM to the new portgroup on the new vSwitch on source host and migrate to destination host.
You should be able to migrate now as the portgroup on source host matches the portgroup on destination host.