It was an old-fashioned football atmosphere, with packed stands and the tie produced the perfect recipe for an old-fashioned cup upset as the second division side took an early lead.
It was a catalogue of disasters for Kilmarnock, and Simon Ford in particular. His poor pass was collected by Dean Keenan who lofted the ball upfield.
Ford seemed to expect goalkeeper Alan Combe to clear, but he stayed back allowing Bryan Prunty to nip in and bundle the ball home, with a suspicion of handball, just to add to the drama.
Kilmarnock took control and dominated possession, but were wasteful in the final third, their crossing particularly woeful, leaving the travelling United fans to have a ball behind their goal.
Killie burst their bubble four minutes after the restart, and Ford made amends for his earlier mistake. Willie Gibson sent over an excellent free-kick and Ford rose highest to head the ball down past the despairing dive of Stephen Grindlay in the United goal.
Ford had another attempt in the 66th minute from another Gibson cross, but this time the keeper made a brilliant block.
Kilmarnock striker David Fernandez was sensationally shown a red card for an elbow thrown at Martyn Campbell, and the match seemed to take a new twist, but Killie refused to hand the initiative over, and seven minutes later, the ten men took the lead.
Mehdi Taouil released Craig Bryson with a great back-heel, and the youngster set off on a mazy dribble into the box. Grindlay saved his shot well, but the rebound fell to Taouil who drilled the ball in off the post.
Killie were rampant, and Ford scored his second with ten minutes left when he met Taouil's corner and glanced a header past the stranded keeper to send the home fans into ecstasy.
Killie's reward is a trip to Inverness in the fifth round.