The club ended sixth in the league again in the 2004-05 season, despite the surprise departure of Cooper just before the end of the season. In the play-off semi-final, they defeated Tranmere Rovers 6-5 on penalties after the sides had each won their home leg 2-0. The club couldn't win promotion though, as in the final however they lost 4-2 to Sheffield Wednesday after extra time. Hartlepool had been leading 2-1 with 8 minutes of regular time to go, but a controversial penalty decision in the 82nd minute which also saw Chris Westwood sent off allowed Sheffield Wednesday to level the scores making it 2-2 at 90 minutes. Hartlepool missing a key defender struggled in extra time and conceded two goals. Following this achievement Cooper's assistant Martin Scott was appointed as Manager.

The 2005-06 season saw the side slip down the division to the relegation places helped in part by poor management, an indecisive board room and key player injuries. Manager Martin Scott was suspended after an alleged fight with a player in the changing rooms, which resulted in his dismissal. Youth team coach Paul Stephenson was put in charge until the end of the season and despite remaining undefeated in his first five matches in charge, he could not stop the club being relegated into the fourth tier in May 2006. Some felt that Hartlepool's relegation was unfair given that Rotherham United had escaped administration, and therefore a 10 point deduction and demotion, by delaying a CVA meeting until after the season had ended.