'Wonderful' care villages 'at risk' - families
But campaigners said communities like Stanley Grange should not be penalised for the failures of large care homes.
David Wilks, the chairman of Our Lives Our Choices, which is now a national campaign group, said: "We entirely agree with moving people out of large mental hospitals… we entirely agree with the moves after the Winterbourne View report.
"But we believe that because of this position, potentially up to 40 village communities have closed over the last 15 years."
Mr Wilks, whose brother Bernard was at Stanley Grange for 35 years, added: "To lump in these types of villages with those [large care home] settings is completely nonsensical. They are completely different".
The group held a national conference earlier to try to persuade the government to change its policies towards village-style complexes for some learning disabled adults.
Its members argue people with learning disabilities were far happier living in places like Stanley Grange than they would be in flats in towns.
Many of the campaigners are family members who argue, for people with this level of complex disabilities, life in the wider community is less safe and more isolated than in a specialist environment.
The CQC said it would base each assessment of the most appropriate care on its merits.
It added: "It is for the provider to demonstrate how their service meets the expectations to deliver safe, effective, compassionate, high-quality care, and ensure autistic people and people with a learning disability are given the same choices, dignity, independence and good access to local communities as anyone else."
