After playing the stern yet dogmatic Lady Mary for six seasons and three films, the actor is leaving Downton Abbey for the last time ...