The telecoms company is to be banned from setting up 5G, but will remain involved in 3G and 4G.
Mr Duncan Smith said allowing Huawei to work on these also posed a continued “risk” to national security.
But the government said it would “ensure” the UK’s communications system was as “secure as it possibly can be”.
Huawei, which has repeatedly said it would not cause harm to any country, predicted the UK would now be pushed “into the digital slow lane”, with higher bills for consumers.
In January, ministers announced the company would be kept out of the sensitive core of the 5G network – including national intelligence – but be allowed involvement in up to 35% of other parts.
This prompted criticism from backbench Conservative MPs, marshalled by Mr Duncan Smith, who called Huawei an arm of the Chinese Communist Party and a risk to the UK.
The US, with which the UK shares much of its intelligence, also applied diplomatic pressure for a rethink.
Under its revised plans, the government says Huawei will not be allowed to install any equipment for the 5G network from next year – and its existing equipment will be removed by 2027.
But Mr Duncan Smith told the House of Commons that the head of BT thought the removal could happen two years earlier.
He said: “I do think he [Mr Dowden] can do it quicker than this… There’s no reason why it can’t [happen].”
The government thought it had made its decision on Huawei earlier this year. It wanted to get on with delivering faster internet and thought Huawei was best placed to ensure speedy upgrades.
But since then the US has continued to apply pressure – with its decision to impose new sanctions on China a crucial factor.
Meanwhile, dozens of Tory backbenchers continued their opposition – and refused to fall in line. They have cautiously welcomed the announcement that the UK is moving away from Huawei – but they want things to move faster. Some are also uneasy that the company’s technology will stay in the 3G and 4G network.
This decision wasn’t without cost though. Ministers admit it will delay the rollout of 5G across the UK and will cost significant amounts of money – into the billions. They’ve also had to consider warnings from telecoms providers about service provision.
Ultimately though, the combination of political pressure – international and domestic – has won the debate in government.
Mr Duncan Smith added that there were “contradictions” in banning Huawei from 5G but not 3G and 4G, which would undergo “software upgrades”by Huawei “for the next decade”.
“So, if they’re a risk to us in 5G, why are they not a risk to us generally?” he asked.
Culture Secretary Oliver Dowden replied: “The reality of the 5G network is that it is fundamentally different and it’s a recognition of that fundamental difference that we are imposing these rules for 5G.
“Of course, over time… 5G will be replaced by 6G, and in all of that Huawei will be absent.”
He also said: “There is of course no such thing as a perfectly secure network, but the responsibility of the government is to ensure that it is as secure as it possibly can be.”
Labour’s shadow foreign secretary Lisa Nandy described the announcement as a “welcome, long-overdue step” but accused the government of having “no consistent approach” to China.
Speaking to the BBC, she said said that, while one department is “seeking to ban them from the 5G network”, another is “considering handing over technology involved in our nuclear industry to the Chinese government.”
SNP culture spokesman John Nicolson said it had been wrong in the first place to allow Huawei near the “nervous system” of the UK’s telecoms network.
And Labour MP Chris Bryant told the Commons there was “unity” among MPs in opposition to the company’s further involvement in 5G, saying: “I wish the government would listen to its own backbenchers.”
The US has claimed China could use Huawei to “spy, steal or attack” the UK – but the company denies this and its founder has said he would rather shut the company down than do anything to damage its clients.
Sanctions imposed in May by Washington have limited China’s access to US chip technology, which prompted the UK’s National Cyber Security Centre to launch a review of the use of Huawei.