You'd have to fill in IMM5476 and cancel you representative in it.
Fill Section C
Then mail it or upload through Web Form. There's nothing negative to happen. Your representative is legally bound to not disclose any information since it would relate to the time when he was representing you in good faith and if there was something adverse (he knew about) he shouldn't have accepted your representation. If there is money involved (like you got to pay later) it can be a bit tricky though. Again if you have a complete legit application, there is no worrying.
Cheers.