Well, I don't tend to believe in "doom and gloom" scenarios, but
  that's my strong bias.

  Watched War Games as a kid. Total Thermonuclear War. Smart kid
  solved it all by understanding the adult's self-aware computer
  system BETTER than the adult who programmed it. New generation,
  new way of thinking.

  Did The Bomb get us? Nope.

  Terrorists used to only kidnap and hijack planes in the 80s. Now
  they blow stuff up.

  We've had poisons that could wipe out entire populations that
  are cheap and easy to make.

  Why has no one used it on a large scale?

  Now consider this: Notice our human concern about the dangerous
  potentials of AI when done _wrong_.

  Let's say a group of scientists worked on it. News gets out.

  People will lobby, fight, blow it up. Anarchy or politics or the
  engineer's wife saying, "Honey, that thing is evil, Destroy it".

  It simply won't happen in the end. Your concern is ALSO the
  concern of millions. Even if it starts, it won't get very far.