More than half of South Africa’s runs came in a 117-run fourth wicket partnership between Dean Elgar (88) and Rassie van der Dussen (68).
But Elgar and Van der Dussen were unable to dominate as England dried up the runs, with Elgar eventually falling to a lofted drive when he tried to lift the scoring rate.
Anderson took three for 34, while Broad and Sam Curran took two wickets apiece.
Anderson took two wickets with the second new ball, bowling from the northern, Kelvin end.
All the other wickets by the fast bowlers were taken from the southern, Wynberg end where they got bounce and some sideways movement.
A crack just outside the line of off stump created uncertainty for right-handed batsmen.
Twelve of the 18 wickets in the match have fallen from that end.
Ben Stokes held four catches at second slip, three of them athletic diving efforts.
Off-spinner Dom Bess played an important role, bowling accurately from the northern end while England captain Joe Root rotated his faster bowlers from the more helpful end.
Bess had spells of nine and 14 overs and finished the day with one for 62 off 27 overs.
Bess made a vital breakthrough when Elgar lost patience and sliced the ball to Root, running back from mid-off.
Elgar and Van der Dussen rescued South Africa from a precarious 40 for three.
Broad took the first two wickets in a hostile six-over opening spell before Anderson was given a turn from the Wynberg end and dismissed South African captain Faf du Plessis.
Elgar played solidly in making his runs off 180 balls with ten fours but Van der Dussen had a precarious existence, especially early in his innings.
His first scoring shot was a four edged at catchable height through a gap in the slips cordon.
He was given out leg before wicket to Anderson but was reprieved on review because of an inside edge and gloved a catch behind off Broad only to be called back because of a no-ball.
He also survived a difficult chance to Stokes at second slip off Anderson.
Curran took two wickets shortly before the second new ball was due, dismissing the dangerous Quinton de Kock before Van der Dussen fell to one of Stokes’ catches when the second new ball was only four overs away.
England earlier added only seven runs to their overnight total of 262 for nine before Anderson was caught at slip off Kagiso Rabada, leaving Ollie Pope unbeaten on 61.
South Africa won the first Test at Centurion by 107 runs. (AFP)