When unregistering the DPLL subsystem the priv pointer is different then
the one used for registration which cause failure in unregistering.
Fixes: 09eeb3aecc ("ptp_ocp: implement DPLL ops")
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Vadim Fedorenko <vadim.fedorenko@linux.dev>
Signed-off-by: David S. Miller <davem@davemloft.net>