These sort of meaningless protocolary f-ups happen all the time. Gone are the days when the word of kings held sway and the most trivial slight was a casus belli. But today this isn't the case, so even if it were an international funeral, it's no big embarrassment. The Zouaves or the Swiss Guards won't be storming Parliament Hill any time soon.
According to the PMO, he did eat the host, which is permissible according to most Protestant denominations, as they by and large practice "open communion" (i.e. not restricted to members of the particular organization). From a Catholic perspective it was an affront as Catholics are not "in communion" with the PM's church and thus the eucharist is to be refused to non-Catholics.
The officiating priest, if he did know the PM wasn't Catholic, probably did the right thing to present Mr Harper with the host (which of course was the "wrong" thing to do according to Catholic doctrine), as refusing the host to the PM would have disrupted the service further.
IMO the proper etiquette for non-Catholics attending a Catholic service is the following: stand when everyone stands; sit when others sit; remain sitting when the congregation kneels; do not cross oneself; participate in the hand-shaking sign of peace; and do not line up to receive communion.