Kyle Lohse (4-1) pitched seven scoreless innings for the win. He scattered four hits and one walk while striking out two to give St. Louis its fifth win in the last seven games.
"Our defense made some great plays," Lohse said. "A whole lot of [credit] to them."
J.A. Happ (1-4) took the loss, allowing two runs on five hits and two walks in five innings. He also struck out five for the Astros, who suffered their fourth loss in the past six games.
"Tonight was no exception. I thought the effort was outstanding all the way around," Astros manager Brad Mills said.
Holliday's two-out solo shot in the fourth gave the Cardinals the lead, and it was a 2-0 game after Tyler Greene added a line-drive homer in the fifth.
The Cardinals added four more runs in the seventh, all of them with two outs.
Jon Jay singled with one out and was moved to second on a sacrifice bunt by Lohse. The next five batters reached base, and runs scored on singles by Nick Punto, Albert Pujols, Holliday, and Lance Berkman to give the Cardinals a six- run advantage.
They needed all six of those runs when Houston fought back in the last two innings against the Cardinals' bullpen.
Fernando Salas came in to pitch and gave up a fielder's choice grounder to Angel Sanchez that scored Inglett, and a Hunter Pence single to center that brought home Bourgeois. Salas struck out Carlos Lee for the second out, but allowed an RBI single to Brett Wallace that plated Sanchez before getting pinch-hitter Brian Bogusevic to pop out to end the inning.
Eduardo Sanchez entered in the ninth and struck out the first batter he faced. But Inglett singled and scored when Bourgeois followed with a triple to the left-center alley. Bourn's sac fly then made it a one-run game, and Angel Sanchez doubled to put the tying run in scoring position.
However, Eduardo Sanchez got Pence to swing and miss on a 2-2 slider to end the game and record his first major league save.
Houston turned five double plays...Wallace extended his hitting streak to seven games and has now hit safely in 15 of his last 16 games...Only 58 of Happ's 101 pitches went for strikes.