The Digital Library by Brewster Kahle

Brewster Kahle

"If we go and bring universal access to all human knowledge it will be remembered as one of the great things humankind has ever done."

- Brewster Kahle

Brewster Kahle has stated his goal as creating a "Universal access to all knowledge," and his catalog of inventions and institutions created for this purpose are more like the web's Greatest Hits list.

When a friend of his posed a simple, yet life-changing question: "What can you do with your life that is worthwhile?" Kahle came up with two answers. The first, developing a microchip to ensure the privacy of telephone conversations, in which did not succeed. And second, which he answered 32 years later, when he created the digital-age version of the Great Library of Alexandria.

With his work, Kahle is known as one of the people who has been successful in spite of the fact that he has never been after that kind of success. Kahle has been pushing protocols for the benefit of technology and humanity.


Early Life

Brewster Kahle was born on October 22nd, 1960, in New York. He grew up in Scarsdale, New York, and attended Scarsdale High School. In 1982, he graduated from the Massachusetts Institute of Technology in with a Bachelor of Science in computer science and engineering, where he was a member of the Chi Phi Fraternity. As a student, he studied artificial intelligence with W. Daniel Hillis and Marvin Minsky.


Career

WAIS

After graduating college, Brewster Kahle joined Thinking Machines, a parallel supercomputer maker, where he was the lead engineer on the company's main product, the Connection Machine, for six years (1983–1989).

Brewster Kahle and Harry Morris who were developers left Thinking Machine to found WAIS, Inc. in Menlo Park, California with Bruce Gilliat. The searching system was developed in the late 1980s as a project of Thinking Machines, Apple Computer, Dow Jones, and KPMG Peat Marwick. The WAIS protocol and servers were primarily promoted by Thinking Machines Corporation (TMC) of Cambridge, Massachusetts.

In 1992, the presidential campaign of Ross Perot used WAIS as a campaign wide information system, connecting the field offices to the national office. Later, Perot Systems adopted WAIS to better access the information in its corporate databases. Other early clients were the Environmental Protection Agency, Library of Congress, and the Department of Energy and later the Wall Street Journal and Encyclopedia Britannica.

The WAIS system was the internet's first publishing and distributed search system and a precursor to the World Wide Web. Public WAIS is often used as a full text search engine for individual internet Gopher servers, supplementing the popular Veronica system which only searches the menu titles of Gopher sites. WAIS and Gopher share the World Wide Web's client–server architecture and a certain amount of its functionality.

WAIS, Inc. was sold to AOL in 1995 for $15 million.

Alexa

In 1996, Kahle and Gilliat founded Alexa. The name was chosen in homage to the Library of Alexandria, drawing a parallel between the largest repository of knowledge in the ancient world and the potential of the internet to become a similar store of knowledge.

Alexa created an operation which includes archiving of web pages as they are crawled. The stored database served as the basis of creation of the Internet Archive, a non-profit digital library Kahle founded at the same time with Alexa, accessible through the Wayback Machine. In 1998, Alexa donated two terabytes size of data to the Library of Congress.

The company offered a toolbar that gave internet users suggestions on where to go next, based on the traffic patterns of its user community. Alexa also offered context for each site visited: to whom it was registered, how many pages it had, how many other sites pointed to it, and how frequently it was updated.

Alexa was sold to Amazon.com in 1999 for $250 million in Amazon stock. Since then, Alexa has taken several changes to provide traffic data, global rankings and other information on more than 30 million websites. The Alexa website that sat on #1,331 (December 2012) on its own ranking system, claims to have 6 million unique visitors each month.

Internet Archive

At the same time as he started Alexa, Kahle founded the Internet Archive, which he continues to direct. The Internet Archive is a non-profit digital library with the stated mission of "universal access to all knowledge."

Although the Internet Archive has gathered information of the World Wide Web since 1996 from web crawlers that also used for Alexa, it did not make this collection available until 2001 when it developed the Wayback Machine. Kahle was inspired to create the Wayback Machine after visiting the offices of Alta Vista, where he was struck by the immensity of the task being undertaken and achieved: to store and index everything that was on the web. "I was standing there, looking at this machine that was the size of five or six Coke machines, and there was an 'aha moment' that said, 'You can do everything'," stated Kahle.

The Wayback Machine is a digital time capsule maintained with content from Alexa Internet. The service enables users to see archived versions of web pages across time, which the Archive calls a "three dimensional index." The name Wayback Machine is a reference to a segment from The Rocky and Bullwinkle Show in which Mr. Peabody and Sherman use a time machine called the "WABAC machine" (pronounced "Wayback") to witness, participate in, and, more often than not, alter famous events in history.

In January 2013, the company announced a ground-breaking milestone of 240 billion URLs.


Digital Access For The Digital Library

Brewster Kahle's work has been around archiving the World Wide Web, and with it, to create a massive amount of database in a form of "library". "Knowledge lives in lots of different forms over time," Kahle has said. "First it was in people's memories, then it was in manuscripts, then printed books, then microfilm, CD-ROMS, now on the digital internet. Each one of these generations is very important." Kahle envisions collecting one copy of every book ever published. "We're not going to get there, but that's our goal," he said. "We want to see books live forever." Pointing out that even digital books have a physical home on a hard drive somewhere, he sees saving the physical artifacts of information storage as a way to hedge against the uncertainty of the future.

Kahle began by having conventional shipping containers modified as climate-controlled storage units, in which each container can hold about 40,000 volumes. Kahle has gathered about 500,000 books. He thinks the warehouse itself is large enough to hold about a million titles, with each one given a barcode that identifies the cardboard box, pallet and shipping container in which it resides.

For the digital library he maintained, Kahle explains the importance of packaging enough meta-data (information about the information) into the archive, "since we don't know what future researchers will be interested in, and that it might be more problematic in finding data rather than preserving data." His view of universal information accessible to everyone has made him critical of Google's book digitization, especially of Google's exclusivity in restricting other search engines digital access to the books they archive.


Awards

Brewster Kahle

With his work and dedications, Brewster Kahle has been awarded:

  • Bachelor of Science - Massachusetts Institute of Technology, 1982.
  • National Academy of Engineering, 2010.
  • American Academy of Arts and Sciences, member 2005.
  • Library of Congress NDIIP advisory board.
  • NSF Cyber Infrastructure advisory board.
  • 2004, Paul Evan Peters Award from the Coalition of Networked Information (CNI).
  • 2007, Knowledge Trust Honors award recipient.
  • 2008, Robert B. Downs Intellectual Freedom Award from the University of Illinois.
  • Public Knowledge, IP3 award recipient.
  • 2009, "50 Visionaries Changing Your World", Utne Reader.
  • 2010, Honorary Doctor of Laws, University of Alberta.
  • 2010, Zoia Horn Intellectual Freedom Award.
  • 2012, Software and Information Industry of America Peter Jackson Award SIIA Peter Jackson Award.
  • 2012, Inducted into the Internet Hall of Fame.
  • 2013, LITA/Library Hi Tech Award for Outstanding Communication in Library and Information Technology.


Personal Life

Brewster Kahle is a member of the Internet Hall of Fame, a Fellow of the American Academy of Arts and Sciences, a member of the National Academy of Engineering, and serves on the boards of the Electronic Frontier Foundation, Public Knowledge, the European Archive (Internet memory) and the Television Archive. He is a member of the advisory board of the National Digital Information Infrastructure and Preservation Program of the Library of Congress, and is a member of the National Science Foundation Advisory Committee for Cyberinfrastructure.

In 2010 he was given an honorary doctorate in computer science from Simmons College, where he studied library science in the 1980s.

Kahle and his wife, Mary Austin, run the Kahle/Austin Foundation. The Foundation supports the Free Software Foundation for its GNU project, among other projects.

In 2012, Kahle and banking veteran Jordan Modell established Internet Archive Federal Credit Union to serve people in New Brunswick and Highland Park, New Jersey, as well as participants in programs that alleviate poverty in those areas.

As a digital librarian, Brewster Kahle has been working to provide universal access to all knowledge for more than twenty-five years.