That letter is usually referred to as the passport request letter, that's what confused me.
Read it again, carefully - at least in some, there are instructions to send either to the regional visa office, OR to Ottawa. In one I'm looking at, Step 4 says submit your documents; sub-step 1 to regional office; 2 - In Canada to the Case Processing Centre CPC-O (more details), submitting your documents in Canada.