Ambiguity, Approximation and Probability

Logo (the programming language)

Probabilistic programming does in 50 lines of code what used to take thousands

Apr 2015, phys.org

I wanted to put some stuff up here about the state of computer programming, because the way we smell is akin to a special kind of computer program, and one which does not act like the kind we know.

I should start like this – I grew up on Logo, and then NES video games, therefore my experience with, and thinking upon, computer programming is ‘coded’ according to this top-down style. Someone writes the code, and the computer executes the code. There are no surprises (unless you have bugs to fix). You tell the turtle (that’s what they call the cursor in Logo) what to do and it does exactly that. Look at the picture above. That little triangle (the turtle) was told to go 100 spaces, rotate 90 degrees, then go 100 more spaces, etc, until a square is born.

King Koopa was told to jump every time you throw fireballs at him, or whatever he does. There is no fuzzy logic here. Everything is clear, concise, exact, predictable. (Again, that’s when the program runs as intended; surely this kind of programming is unpredictable when it goes wrong.)

Enter a new kind of programming. With the dual advent of big data and big processors to crunch it, we are seeing a different approach. The computing power is now so capable that it is asked to figure out its own program from the data given. This helps with a lot of the problems faced in computing today. With such variety in the data (this is ultimately what big data is about – not lots of quantity, but lots of different qualities) we can no longer write programs equipped to work with such variety. The program required for that ends up being as big as the dataset.

This is where we see the parallels to smells and olfaction. The amount of smells we could potentially be exposed to is infinite and multifarious. Vision has only a few categories. Things can look light or dark, a binary classification, or they can be categorized by their color on the spectrum, which is a discrete classification. They have a shape, a size, maybe a texture category. Odors, however, cannot be organized this way. There are too many and they are too different from eachother. Therefore, olfactory perception is distinct from our other senses. In order for us to create an artificial intelligence that can smell, we would have to come up with a different kind of programming.

Facial recognition provides a good visual analogy to the olfaction problem. What a face looks like isn’t really dependent on its color or its shape, but the combination of these features, the whole. And that makes a lot of initial parameters, in fact, infinite parameters. Face-rec uses these new types of programs, and they are almost the opposite, in every way, of what programming has been. I’ll let this guy describe them:

“When you think about probabilistic programs, you think very intuitively when you're modeling. You don't think mathematically. It's a very different style of modeling.” … “The code can be generic if the learning machinery is powerful enough to learn different strategies for different tasks.”

- Tejas Kulkarni, an MIT graduate student in brain and cognitive sciences, phys.org

In the same way that we are not born already knowing every smell we will ever encounter, these programs must ‘learn on the fly.’ This is an advance in computing, but also it foreshadows a very different world, where information is not distinct, discrete, exact, etc. It is instead more like that thing you smell but you don’t know what it is, but you swear you know yet you don’t know…you know what I’m talking about? Doesn’t sound like the kind of output your computer would produce.

POST SCRIPT

[lots of good explaining in this article, so I just copied most of it]

A Grand Unified Theory of Artificial Intelligence

Mar 2010, phys.org

Embracing uncertainty

In probabilistic AI, by contrast, a computer is fed lots of examples of something — like pictures of birds — and is left to infer, on its own, what those examples have in common. This approach works fairly well with concrete concepts like “bird,” but it has trouble with more abstract concepts — for example, flight, a capacity shared by birds, helicopters, kites and superheroes. You could show a probabilistic system lots of pictures of things in flight, but even if it figured out what they all had in common, it would be very likely to misidentify clouds, or the sun, or the antennas on top of buildings as instances of flight. And even flight is a concrete concept compared to, say, “grammar,” or “motherhood.”

As a research tool, Goodman has developed a computer programming language called Church — after the great American logician Alonzo Church — that, like the early AI languages, includes rules of inference. But those rules are probabilistic. Told that the cassowary is a bird, a program written in Church might conclude that cassowaries can probably fly. But if the program was then told that cassowaries can weigh almost 200 pounds, it might revise its initial probability estimate, concluding that, actually, cassowaries probably can’t fly.

“With probabilistic reasoning, you get all that structure for free,” Goodman says. A Church program that has never encountered a flightless bird might, initially, set the probability that any bird can fly at 99.99 percent. But as it learns more about cassowaries — and penguins, and caged and broken-winged robins — it revises its probabilities accordingly. Ultimately, the probabilities represent all the conceptual distinctions that early AI researchers would have had to code by hand. But the system learns those distinctions itself, over time — much the way humans learn new concepts and revise old ones.