I grew up going to church every Sunday with my mom and dad and we usually sat with my grandparents. The church never mentioned anything about Mary and Joseph having other kids(maybe I just didn't pay attention) but I don't think The Catholic church is cool with people thinking that Mary wasn't a virgin after or before having Jesus.