I’m not going to give the whole rundown as Grammar Girl does a pretty good job of it. Suffice to say (or Suffice it to say, or Suffices to say, egad), that age is a major determining factor.
It seems younger people like “on accident.” No one seemed to have a good answer to this other than young people like the dichotomy of “on purpose” vs. “on accident.”
While not scientific, British and Australian commenters seemed to prefer “by accident.” I wonder what Canadians use.
I like “by accident,” but then again, I’ve been told my usage of English is precise to the point pedantry and bordering on baroque. Maybe this is just the march of progress passing me by, or maybe this whole business is just something that happened on accident.