Living in Sri Lanka shouldn't be a problem for you if you chose to marry your Sri Lankan husband. If your marriage is truly genuine, you chose to marry each other because you want to be with each other physically, not for one spouse to gain status to live in another country. You should be able to say, "I can live in either Canada or Sri Lanka, because I love my husband, and I only want to be with him", rather than "I only want us to live in Canada, I don't want to live in Sri Lanka, because it's so poor, no jobs, etc". Have you ever asked yourself, if you could look in a crystal ball and see he can never come to Canada, would you have chosen to marry him?
You also need to remember, one of the most common questions during an interview is, "If they refuse your application, what will you do?". If you say, I will never live in Sri Lanka, they will flatly deny you because they think you focus all your plans around living in Canada, and made no plans for alternative living plans should the application fail. If living in Canada is really so important to you, then you shouldn't have married someone from another country if there is even a possibility of him being rejected.
I wish you best of luck.