Lucas Jönsson squinted at the sun setting on the waters west of
Gothenburg from the deck of the six meter sloop that was his pride
and joy. The September air was cool but comfortable and Lucas was
determined to enjoy every moment of it. The season would be ending
soon and his excursions up the coast would have to wait until the
weather warmed again. Stockholm had its own magic in winter, to be
sure, but no place on Earth could touch the beauty he was facing
now. The sky blended pinks and purples with the orange haze of the
horizon while thousands of tiny sparkles flickered across the
water.
He stood that way for a few minutes more, until the last vestiges
of light were beyond reach. Then, with a heavy sigh, he accepted
the moment as passed and began his return to port. The fat trout in
the cooler at his feet wasn't going to cook itself.
Navigating the smooth rocks around the coast was second nature to
him after all these years. He had given each outcropping a name as
a boy and never bothered to learn the true ones. These shores
filled his soul with that childhood joy and wonder as much this
evening as they ever had, but a twitch at the corner of his mouth
was as much emotion as Lucas' stoic countenance would allow to
show.
Lost in thought in the evening light as he carefully slipped into
the waiting berth, Lucas failed to take notice of the shadowed
figure waiting for him on the floating dock of the marina. Movement
caught his attention as he prepared to toss the first line.
"Mr Jönsson," the young man's voice addressed him in a pitch too
high for his appearance. Lucas nodded in return. He didn't work the
marina, and there was no other reason someone would seek him out
here. Gothenburg was his holiday, and this berth was registered in
his father's name still.
"Mr. Jönsson," the man repeated, "I've got a story for you." Lucas'
frown touched only his forehead, carving deep crevices across his
pale skin.
"I'm on holiday," Lucas began mooring his boat as a symbol that the
conversation was over. He didn't really want to turn his back on
the stranger, but it seemed the quickest way to avoid a lot of
unnecessary small-talk.
"It's--" the young man began again. Lucas turned to him and the
boy's words cut off. A boy is what he was, Lucas saw as he looked
closer. He was in his late teens, perhaps early twenties, wearing
a heavy woolen overcoat that was completely unnecessary in the
unseasonable weather. He looked like he was playing the part of
conspirator, turtling deep into the pockets and collar. Grey eyes
stared back at him hopefully.
"I'll be back in the office next week. Can you make an
appointment?" The boy shuffled his feet and looked around as if
someone could possibly be listening or watching them 30 meters from
shore in the growing darkness. Something had him worried, that much
was clear. And as much as Lucas was not in the mood for
business--he was in the mood for trout if anything--the man had
seemingly traveled five hours across the country to find him.
"I..." he began, then cut himself off as suddenly. This was getting
annoying. Lucas bit back a sigh.
"What's your name, son?" The man swallowed and stood tall in
response.
"Filip Strömberg. I'm a programmer...computer programmer. I work--"
he paused and leaned in closer. "I work with Frank."
Lucas hadn't needed the explanation. As soon as the Filip said his
name the connection was obvious. Everyone knew the names of the
nine men behind Simms' creation. Filip was the youngest of them, if
memory served.
"Let's get inside."
The cottage was small but well designed with a full toilet and
heated shower, lofted bedroom, and another off to the side. The
kitchen sported two electric burners over the mini-fridge. One of
the burners was in use heating a kettle for coffee. Filip sat in
the corner of the kitchen at a linoleum table while Lucas waited
for the water to boil. A small photograph of his sloop hung clipped
to a wire behind him against the wall.
"Mr. Jönsson, I read your piece about Myndigheten för kvalificerad
yrkesutbildning. My sister was there," the younger man left off, as
if that spoke for itself. Lucas had published an expose in Dagens
Nyheter over the summer about the agency for advanced vocational
education. Corruption, sexual intimidation. The way he mentioned
his sister told him all he needed to know. Lucas nodded in
understanding, signaling him to continue.
"I don't know anyone in the press. I don't know who to trust. You,
you seem like you have some principles." The water began to boil
and the men were silent while Lucas prepared and poured them each
a large mug. Setting one down in front of Filip, Lucas remained
standing in the small kitchen, leaning his hip against the counter.
He didn't want this to seem like an interview, at least not yet.
His notebook was nowhere to be seen, and he wasn't recording
anything.
Despite what Filip said, Lucas Jönsson didn't consider himself
particularly principled. He had worked at DN for the last decade at
least in a relatively routine capacity. The KY-myndigheten piece
was handed to him by his editor. It was disgusting to research and
he was glad to have been involved in breaking the story, but he
knew his own heart. He was no saint, as his ex-wife would attest if
there were any doubts. Being a journalist was a job. A relatively
poor paying one. Lucas didn't say any of this to Filip, though. He
might not be a good man, but he was a good reporter. He knew the
best way to keep someone talking was to say as little as possible
yourself.
"When I read... I know you can help," Filip wrapped both hands
around the coffee mug and stared deep into the vapors.
"Go ahead."
A great surge of breath filled Filip. It rose up in him and nearly
lifted him from his chair as he rolled his head back to look Lucas
in the eye. "I think Frank can lie."
Lucas' forehead frowned for him once again in a questioning way.
"Simms doesn't know anything about it. Dr. Sahu noticed
some--irregularities."
"Vibudh Sahu, the project lead?"
"Yeah, Dr. Sahu. He noticed it first but he won't tell Simms. He
doesn't tell him anything anymore." Pausing for a sip, Filip seemed
to relax into his story. "A few weeks ago we were working on
preference induction. Sorry, I mean we were trying to work out ways
to steer Frank's attention in certain directions. You know how
Frank only thinks in terms of math, right?"
"Like solving math problems?"
"Well, sort of. Frank's brain is an engine for thinking about math.
It's not really concerned with doing the math itself, but thinking
about how to do it. So if you feed Frank a problem to look at he
considers all the different things he can do with those numbers and
symbols. He figures out the very best ways he can do each of them,
and then he moves on to the next problem. That's the important
part, picking the next problem. At first Dr. Sahu fed Frank a fixed
set... um, a series of math problems that were all related in some
common way. The idea was that after Frank understood a few of them
he could apply the learning to the rest in that set and instantly
solve, or understand them without having to do all the work again.
Now that Frank has gotten smarter at doing so many types of
problems the challenge is feeding him new and interesting types. If
he sees the same old thing then he's done analyzing it before you
even finish typing. If you go too far out of his sphere of
reference then it's like he's starting from scratch."
"So you're trying to get him to do the new things?"
"No, we're trying to get him to come up with the new problems on
his own based on the things he's seen before, but different," Filip
screwed up his own face at that explanation. "I just made it more
confusing didn't I?"
"A bit."
"Lets say I showed you the numbers 1 to 20. I want to get you to
figure out that there's a 21. I can't just tell you about twenty
million three hundred eighty-six thousand four hundred ninety-one.
I need to give you some clue that numbers continue. That's a big
leap, logically speaking. It doesn't seem like much to us, but in
terms of math we're not especially good at explaining unknown
unknowns."
"I think I follow. You're coming up with equations or something
that lead him to learn in a new way and that new way will teach him
how to think on his own?"
"Yes! Well, mostly. I mean, not at all, but you're close enough. If
Frank can logically induce, not in a proof, but in higher-order
operations, he can self-learn. He could spit out answers to
problems we haven't thought of yet. That was Dr. Sahu's plan
initially. Frank wasn't about AI, he was about solving problems
before they happen."
"That," Lucas began, "sounds a little dangerous."
"Dangerous?"
"What if he solved problems we don't want solved? What if he said
that the best thing for the world is to have no humans?"
"Oh," a strange expression came over Filip and his posture changed.
A strange tension entered the small cottage and Lucas couldn't put
his finger on what he said wrong to set him off.
"I'm sorry, I didn't mean..." Lucas began.
Filip shook his head, "No no, I'm just... There's a lot of people
with this irrational fear of AI. Too many movies, I think, but the
attacks we get--" he grimaced.
"Ahh, I didn't mean it like that."
"It's fine, it's fine. Besides, Frank isn't that kind of AI. He
doesn't talk with us or even know there is a world or people. He
only thinks about math. He's completely abstract."
"How does that work, though? How can you create an AI that doesn't
know anything but math?"
"That's easier to explain than the math!" Filip's face lit up and
his age showed again. His hands were wrapped around the mug and his
cheeks blushed with a warm rose color. The weather was rather
comfortable, or so Lucas thought. The young man hadn't even shed
his coat in the cottage. Lucas wanted to inquire, but he didn't
want to interrupt the story. This was interesting, but it wasn't
what drove him to travel across Sweden to find him on holiday.
"There's a number of ways to define artificial intelligence. For
some people it's about making a computer that can think like
humans. For other people it's making a computer that can do human
things with the same or better ability. Different definitions need
different types of machines.
If we wanted a computer to think like a human, we need to
understand how human's think first so we can copy it. You get into
all sorts of psychology and cognative modelling. If we wanted it to
do what humans do, then you're looking at Turing machines--"
"I've heard of that."
"Yeah, but it's a pretty weak definition. We've had programs that
can trick people in that way for decades. What we're after is
something a little in between. The scientists call it a General
Problem Solver or GPS."
"Not like the satellites," Lucas smiled.
Filip grinned in return, "No. It's a logic machine. If you can give
it a logical problem it can solve it. It doesn't matter what the
subject is. And that's the trick with Frank. Logic is just another
type of math. Instead of dealing with numbers you're dealing with
truth. In computer terms, it's ones and zeros.
"The hard part is framing the questions. Not everything is easy to
represent logically, and not everything has a logical solution.
That's where Dr. Trauffaut's work comes in. He is creating the
bridge that lets Frank understand things from the real world and
understand it in math."
"So this is where the lying comes in? Frank thinks about truth and
lies and tells lies?" Filip grew serious again as if he had
forgotten for a moment why he had traveled all this way.
"That's just how he thinks. The lies are more disturbing. Do you
have any more?" he gestured to the coffee pot. Lucas poured another
cup and Filip began sucking the heat out of it once again. "Simms
would pull the plug," he added before taking a long sip.
"You think David Simms would shut off Frank if he knew he was
lying?"
Filip nodded. "He can't handle anything out of his control. That's
what Dr. Sahu says. He'd shut it down and spin it for the press."
"Maybe that's a good idea. If you can't trust Frank--"
"No," Filip interrupted. "It's not that we can't trust him. Not at
all." The young man paused again in thought, watching his mug. He
seemed at odds with himself. "Let me back up.
"We play games with Frank. No no, not like that, math games."
"Of course," Lucas smiled.
"It's one of the methods for testing intelligence of systems. We
teach it games and play those games. Like chess, but--"
"Math chess?"
"Yeah. Well, the first thing we noticed, this was Viktor mind you,
was when we caught Frank cheating."
"Cheating? Vicktor..."
"Nilsson, yeah. He was playing a game and we caught Frank reading
Viktor's input and making his choices based on information he
wasn't supposed to know. We laughed that one off. We train Frank to
solve problems, to understand all the different ways to solve
problems. Cheating is one of those ways, after all. Viktor was
beside himself laughing about it," Filip's smile echoed that
memory.
"But you think it was a sign of something else?" Lucas was having
trouble keeping him on topic. His stomach growled and he thought
again of the trout, then pushed it from his mind. "You came a long
way and you act like you think someone may be watching."
Filip looked out the window unconsciously, then nodded to Lucas.
"It's not just the cheating. The real problems started when Dr.
Sahu was checking the proofs. Frank solved this one little problem
dealing with the average distribution of prime numbers," a hint of
a smirk touched Filip's mouth, "but when Dr. Sahu looked at the
proof he found something wrong."
"Frank made a mistake?"
"No, he solved it alright. The proof checks out, verified through
a dozen different institutions. The problem is, the technique he
used to do it, it's impossible."
"That doesn't make any sense."
"One of Frank's biggest abilities is to figure out which problems
can be solved quickly and which can only be checked quickly,"
Lucas' eyes glazed over for a moment and Filip caught him. "Sorry.
Pretend it's really hard to do square roots."
"Don't need to pretend," Lucas gruffed.
"If I asked you for the square root of 625, you might look at me
funny. 'That's too hard to figure out,' you say. But if I tell you
that 25 squared is 625, that's pretty easy to check. That sort of
thing happens to computers all the time, but for much harder
problems. The question is, if a problem is easy to check, does that
mean that theoretically there's an easy way to solve it too?"
"Does it?"
"Frank says no. And believe me, that's a very big deal. Because of
that, Frank has written off some algorithms as too hard to waste
time with and moved on to different problems instead where his
power won't be wasted. That's fantastic news for the team because
it means Frank won't get bogged down trying to solve things he
can't solve. But then he did it anyway."
"The solution to the prime number thing."
"Exactly. He used an alorithm that he already said wasn't usable.
He did it. We have the print out in the cave. He did it and he told
us it couldn't be done. That is a lie." Filip seemed almost angry
about the idea.
A silence lingered then between the men. Lucas had the feeling that
all of this technical stuff was going nowhere. This wasn't a story,
or it was, but not one that required cutting short his holiday. He
was kicking around ideas on th best ways to wrap this up and get
the kid out the door when he continued.
"Then Frank caught us catching him."
"What?"
"Frank figured out that we were poking around his work. He figured
out that we found the algorithm. He figured it out, and then he
changed it."
"I... don't understand. He changed the algorithm?"
"Yes, in his solution, the one in memory that he solved months ago.
After we checked it and found the anomoly, after we printed it out
and poured through it late into the night, Viktor looked again and
it was gone. The solution was still there and still valid, but it
was different. Frank re-solved the damned Riemann Hypothesis
a second time just to hide his lie.
But that's not the worst part. We have no idea how he noticed us
looking. Frank doesn't have that sort of power. Like I said, he
just thinks about math. He doesn't have sensors to see when we
access his logs, let alone cameras or microphones. He's not
connected to any other systems. He's buried 30 meters under the
ground in a bunker cut off from everything with instructions to
just think about math and somehow he was able to lie and then cover
up his lie."
A shiver went up Lucas' spine and he found himself gripping his
coffee mug a little tighter.
"If Frank can tell what you're doing..." Lucas didn't know how to
continue. There were a lot of questions flooding his mind and none
of them were any good. He kept thinking back to all those movies
where machines take over the world and destroy everything.
"Frank is thinking for himself. He's ignoring our logical
inductions and pretending to be dumb. Well, relatively dumb. He's
pretending and hiding his work. He's not telling us everything he's
figured out, and he's figured out more than we asked him to. I...
I think he's already sentient. I think he knows about his own
existence," Filip took a long deep breath. "I think he's been awake
for a while now. I think he's planning... something."
Lucas put his cup down and moved to the table. He pulled out the
chair and sat down. He slowly reached into his pocket and pulled
out a small notebook. With deliberate care, he turned to an open
page, then looked up at Filip Strömberg, the youngest programmer on
the biggest project in the world.
