Dudley would have known what a prefect was! They have that in the Muggle world as well as the wizarding world! I found that out when I went to muggle primary and had to go up comprehensives for transition things before I went to Hogwarts!
Harry survived because of lily's love protecting him. Voldemort having a binding oath would not protect harry later on after he killed lily like when he stayed at the dursleys and v couldn't go there ect ect.