Yes - but unfortunately that means that neither of you can sponsor the other, and likewise cannot sponsor the child. Until, that is, one of you is back in compliance.
With the child - that adds further complication that getting health insurance may be more difficult (need to check provincial rules). There may be other matters that get complicated as well. You'd best at least look into this and esp private health insurance for the child.
You raised whether crossing together would improve chances - it would be rather easier to respond if one of you was in compliance with the RO. As it stands, I honestly have no idea what tactic would be best.
A small additional complication (that perhaps others could comment on): while, as noted, CBSA cannot refuse entry to a PR with valid status - therefore both you and your spouse cannot be turned away (on the assumption as above that a 44(1) wasn't issued and your PR revoked), even if they 'report' you (will be subject to appeal).
BUT: this does not apply to your child (who is not a PR, if I read correctly). Now obviously with a young child, denying the child entry constructively makes it impossible for at least one parent to enter). Your child remains a foreign national and entry subject to CBSA discretion (even if it's not that often that it happens).
Would they do this? I've no idea. I've no doubt that it's possible and chances even good that it has happened once in the past - or in other words, it may be rare or even very rare (I've no idea) but can't be excluded.
Important: caveat that I don't claim to be an expert and this to best of my knowledge; if this turns out to be wrong, I'll mea culpa, but I think worth knowing about the possibilty.
Hello
Two months back, I was trying to cross the border from Detroit to Windsor (was not meeting residency obligation but PR card had another 1 year of expiration - crossing the border after a long gap). The officer asked me a lot of questions and eventually took signature from me on a report but I don't remember getting any copy of that report. I don't even remember what I saw on the report. He did say that I can surrender my PR card while returning from Canada at the border and he also said it's fine if I don't do that, and we can figure out that later. He also said he is putting some comments or notes on my file, not sure what it was.