We have been pleased with the results so far and have
confidence that HTMs can be applied to many other
problems we have not tried. We also know of problems
that our current implementation of HTM will not solve
today, and we have yet to build very large networks.
Time will tell what obstacles we will encounter in the
future, but for now we are encouraged by how well the
technology is working.
The idea of
marrying computer science with brain research
came to me soon after I graduated in electrical
engineering from Cornell University, in 1979. For many
years I pursued the interest part time while working as
an engineer for a number of companies (including two I
cofounded-Palm Computing and Handspring), and for one
year I worked at it full time, as a graduate student in
biophysics at the University of California at Berkeley.
As Carver Mead once said, "If we really
understand a system, we will be able to build it"
In 2002, with the encouragement of some neuroscientist
friends, I created the Redwood Neuroscience Institute.
For three years, I worked with about 10 other scientists
there on all aspects of neocortical anatomy, physiology,
and theory. During those years, more than 100 other
scientists visited the RNI for discussion and debate,
and I made progress on HTM theory.
By 2004 I had developed the essence of HTM, but the
theory was still rooted in biology (I published part of
that biological theory in 2004 in my
book On Intelligence). I did not know
how to turn the biological theory into a
practical technology. A colleague of mine, Dileep
George, was aware of my work and created the missing
link. He showed how HTM could be modeled as a type of
Bayesian network, a well-known technique for resolving
ambiguity by assigning relative probabilities in
problems with many conflicting variables. George also
demonstrated that we could build machines based on HTM.
His prototype application was a vision system that
recognized line drawings of 50 different objects,
independent of size, position, distortion, and noise.
Although it wasn't designed to solve a practical
problem, it was impressive, for it did what no other
vision system we were aware of could do.
In 2005, with a theory of the neocortex, a
mathematical expression of that theory, and a working
prototype, George and I decided to start Numenta, in
Menlo Park, Calif. Our experience in industry and
academia taught us that people move more quickly in
industry, especially if there is an opportunity to build
exciting products and new businesses. Today, the RNI
continues as the Redwood Center for Theoretical
Neuroscience at UC Berkeley. George and 15 other
employees work at Numenta, and I split my time between
Numenta and Palm.
To help
Numenta jump-start an industry built on the
ideas of HTM, we embarked on creating a set of tools
that allow anyone to experiment with HTMs and to use
HTMs to solve real-world problems. By making the tools
broadly available, providing source code to many parts
of the tools, and encouraging others to extend and
commercialize their applications and enhancements, we
hope to attract engineers, scientists, and entrepreneurs
to learn about HTM. The tools together constitute an
experimental platform for HTM that is meant to attract
many developers by giving them the chance to create
successful businesses.
We realize it will take a lot of time and effort to
learn an entirely new way of computing. One of the
things I have learned is to never become an obstacle to
a developer working hard on a new platform. To this end
we have created lots of documentation and several
examples. We provide complete source code for two of the
platform's three components.
The first part is the run-time engine. This component
is a set of C++ routines that manage the creation,
training, and running of HTM networks on standard
computer hardware. It's highly scalable, meaning you can
run the entire platform on anything from a laptop with a
single CPU to PCs with multiple cores, to a large
computer cluster. Significant development can be done on
a single CPU; however, as applications grow in size,
multiple CPUs may be needed for performance.
Today the run-time engine runs on Linux. Our employees
and customers use both PCs and Macs. When designing
HTM-based systems, the developer needs to experiment
with different configurations. Running concurrent tests
saves a lot of time. Our tools make parallel testing
easier.