A Nottingham university will pay off staff to resign as it looks to reduce the number of students on certain courses.