The guy got frightened as relationship try swinging too quickly