In the past some employers would play the PAYE system, holding on to the PAYE deductions until the last payment date of the year, and then paying the balance due before interest was charged. That is no longer possible under real time information (RTI), as interest is now charged on late paid PAYE and CIS deductions on a monthly basis throughout the tax year.
The interest is applied to payments which fall due on and after 19 May 2014, which are paid late. If you check your business tax dashboard on the HMRC website, you may see interest accrued from 19th of the month. Where you pay the PAYE due electronically the payment is due by 22nd of the month. In the case of an electronic payment the interest charged from 19th to 22nd should be cancelled.
However, there is a known problem which HMRC are working to fix. If your business made no full payment submissions (FPS) for a tax month (perhaps because no wages were paid), HMRC may have estimated the PAYE due and added that estimated amount to your PAYE account. The estimated PAYE debt is called a "specified charge". Interest accrues on the specified charge as if it was real PAYE due.
The way to get rid of a specified charge is to submit a nil Employer Payment Summary (EPS) for each of the tax months for which the specified charge has been raised for. This should also remove the interest accrued on the specified charge, but currently it doesn't. HMRC say this interest will not be pursued, and it will be removed from the PAYE account when the software is fixed.