is it ethical to punish a person who doesn't remember the crime they committed ?

it entirely depends on the purpose and reason for the act i.e. the punishment
the PURPOSE :
the purpose of the act or the punishment can be (this is what i can think of at the moment) as follows:

1. to ensure he never repeats the act perhaps because of the immorality of the act or the unethical nae of the act
2. to seek personal pleasure out of the punishment; to see the other person suffer because he/ she has caused certain level of discomfort to you.

if you want if because of the second reason whether or not he person remembers his/her mistake is irrelevant.

if your reasoning for the punishment is the first one, there is no obvious point of punishing him/her for a deed he doesn't remember.
in fact, if the punishment is apparent and he/she develops a negative attitude towards you, and will have worse repercussions than before which a wise man would avoid. unless there is strong emotional bias of the guilty with the punisher, in which case any kind of punishment will be incapable of creeping into the emotional bond.