Lloyds Banking Group was fined £4.3m by the City watchdog after up to 140,000 customers had payment protection insurance compensation payments delayed.
The customers were not paid redress within 28 days of receiving a decision letter and almost 9,000 had to wait more than six months for compensation, the Financial Services Authority (FSA) said.
The failings relate to Lloyds TSB Bank, Lloyds TSB Scotland and Bank of Scotland, leading to a total fine of £4.3m.
The regulator imposed a large fine because Lloyds' redress payments systems fell "way below the standard the FSA expects".
Nearly one quarter of 582,206 decision letters agreeing to pay PPI compensation between May 2011 and March 2012 did not result in payouts being made within the target of 28 days.
The regulator said that around 87,000 customers had to wait more than 45 days, 56,000 over 60 days, 29,000 past 90 days and 8,800 in excess of six months.
The FSA said that Lloyds had not planned sufficiently "at the outset" and its systems were not able to process the large volumes of PPI payouts in a timely way.
Lloyds apologised to customers and acknowledged it underestimated the scale of the PPI scandal.
The banking giant said "almost all" customers who were due redress during the review period have been paid in full and customers have not been left out of pocket because of the delays.
Lloyds said in a statement: "When we took the lead in 2011 to compensate customers on PPI, we had not fully anticipated the volume of complaints to be processed at the outset and experienced some administrative errors as we scaled up our systems and processes.
"We acknowledge that this led to some customers not being compensated on time and we apologise to those customers whose payments were delayed.
"It is important to note that almost all customers who were due redress during the review period have now been paid in full."
Many banks have had to compensate customers over the mis-selling of payment protection insurance (PPI). Columnist James Moore has said the reasons banks were keen to sell PPI on top of personal loans is that the loans themselves weren't very profitable.