See the solid black box? All the freedoms are still programmed
  freedoms. Even systems that can reprogram themselves can only
  reprogram themselves within the limits of "how humans" _can_
  think, fundamentally. We'd only have to be concerned with an
  ALIEN AI - *or* with a _poorly made_ human AI. Properly
  programmed AI can recode itself all it wants and there will be
  no problems.[1]ai-constraints I've heard versions of it my whole
  life so far and it's annoying. _NOW_ we have the venerable
  Stephen Hawking saying it. People tend to believe him. But he's
  probably just remembering an Isaac Asimov story he read as a kid
  or something and it's been percolating in his mind in the
  background.

  We become _so confident_ in our mathematical and technical
  abilities that we start to think, "People are like computers".
  But that's getting it all backwards. Computers are like people
  because it's _people_ that made the computers.

  It's hard to squash this fear in people though. People can be
  extraordinarily intelligent, and miss one _very important_
  detail and get it all wrong. GIGO. Even computers that can
  program themselves were _bootstrapped_ by a human. Even if THOSE
  computers make other computers, it *still* started with a human.
  If we get it wrong, well, it's _us_ that's the blame ultimately.
  [of course it'll be interesting to see what court cases will be
  like in the future with AI.

  "It's not my fault! It was my programming!"
  "Oh, but you still make that wrong choice."
  "Yes but..."

References

  Visible links
  1. http://icopiedyou.com/wp-content/uploads/2015/06/ai-constraints.png

You are viewing proxied material from sdf.org. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.