According to union leaders, the rate cuts mean some staff in the strategic outsourcing section of IBM Global Services could be at least £5,000 a year worse off.
Under the new pay regime, which came into force last month, stand-by rates for compensating staff for being on call during unsociable hours will be £2 per hour - half the national minimum wage.
Union leaders warned that other companies may follow IBM's example and cut pay rates for IT staff across the UK in a bid to gain a competitive edge in an uncertain market.
"What IBM seeks to do today other companies will seek to do tomorrow," said Peter Skyte, national secretary of union Amicus MSF. "Companies will be seeking a competitive advantage by undercutting IBM and create a downward spiral."
Staff working in the Warwick, Manchester and Portsmouth offices of IBM's strategic outsourcing service delivery business are understood to be affected by the changes.
Last year union leaders accused IBM of offering its IT staff a miserly redundancy package, as it consulted its workforce on hundreds of possible job losses as part of a global restructuring. Since then, IBM has declined to reveal how many UK redundancies it has made, adding that it was constantly "re-balancing" skills and resources.
IBM declined to comment on the overtime rate cuts.