Oughton will play a key role in promoting efficiency across the public sector, taking over responsibility for IT procurement and project management.
He will take over from Gershon, a former managing director of GEC, from the beginning of April.
Oughton, 51, is current deputy chief executive of the Office of Government Commerce.
Mike Thompson, director of research at analyst firm Butler Group, said, "Gershon shook things up. Career civil servants do not typically do that."