Internet Archive

The Internet Archive is a non-profit digital
library with the stated mission of “universal access to all knowledge”. It provides permanent storage of and free
public access to collections of digitized materials, including websites, music, moving
images, and nearly three million public-domain books. As of October 2012, its collection topped
10 petabytes. In addition to its archiving function, the
Archive is an activist organization, advocating for a free and open Internet. The Internet Archive allows the public to
upload and download digital material to its data cluster, but the bulk of its data is
collected automatically by its web crawlers, which work to preserve as much of the public
web as possible. Its web archive, The Wayback Machine, contains
over 150 billion web captures. The Archive also oversees one of the world’s
largest book digitization projects. Founded by Brewster Kahle in 1996, the Archive
is a 501(c)(3) non-profit operating in the United States. It has an annual budget of $10 million, derived
from a variety of sources: revenue from its Web crawling services, various partnerships,
grants, donations, and the Kahle-Austin Foundation. Its headquarters are in San Francisco, California,
where about 30 of its 200 employees work. Most of its staff work in its book-scanning
centers. The Archive has data centers in three Californian
cities, San Francisco, Redwood City, and Richmond. Its collection is mirrored for stability and
endurance at the Bibliotheca Alexandrina in Egypt. The Archive is a member of the International
Internet Preservation Consortium. This non-profit digital library was officially
designated as a library by the State of California in 2007. History Kahle founded the Archive in 1996 at the same
time that he began the for-profit web crawling company Alexa Internet. Kahle was motivated to found the Archive after
a macramé hobby website of his was lost when its host unexpectedly filed for bankruptcy. In 1996, The Internet Archive had begun to
archive and preserve the World Wide Web. The archived content wasn’t available until
2001, when it developed the Wayback Machine. In late 1999, the Archive expanded its collections
beyond the Web archive, beginning with the Prelinger Archives. Now the Internet Archive includes texts, audio,
moving images, and software. It hosts a number of other projects: the NASA
Images Archive, the contract crawling service Archive-It, and the wiki-editable library
catalog and book information site Open Library. Recently, the Archive has begun working to
provide specialized services relating to the information access needs of the print-disabled;
publicly accessible books were made available in a protected Digital Accessible Information
System format. According to its website: Most societies place importance on preserving
artifacts of their culture and heritage. Without such artifacts, civilization has no
memory and no mechanism to learn from its successes and failures. Our culture now produces more and more artifacts
in digital form. The Archive’s mission is to help preserve
those artifacts and create an Internet library for researchers, historians, and scholars. In August 2012, the Archive announced that
it has added BitTorrent to its file download options for over 1.3 million existing files,
and all newly uploaded files. This method is the fastest means of downloading
media from the Archive, as files are served from two Archive data centers, in addition
to other torrent clients which have downloaded and continue to serve the files. On November 6, 2013, the Internet Archive’s
headquarters in San Francisco’s Richmond District caught fire, destroying equipment and damaging
some nearby apartments. According to the Archive, it lost:
a side-building housing one of 30 of its scanning centers
cameras, lights, and scanning equipment worth hundreds of thousands of dollars
“maybe 20 boxes of books and film, some irreplaceable, most already digitized, and some replaceable”
The non-profit Archive sought donations to cover the estimated $600,000 in damages. World Wide Web archiving
Wayback Machine The Internet Archive has capitalized on the
popular use of the term “WABAC Machine” from a segment of the old Rocky and Bullwinkle
cartoon, and uses the name “Wayback Machine” for its service that allows archives of the
World Wide Web to be searched and accessed. This service allows users to view archived
web pages. The Wayback Machine was created as a joint
effort between Alexa Internet and the Internet Archive when a three-dimensional index was
built to allow for the browsing of archived web content. Millions of websites and their associated
data are saved in a gigantic database. The service can be used to see what previous
versions of websites used to look like, to grab original source code from websites that
may no longer be directly available, or to visit websites that no longer even exist. The Internet Archive Terms of Use specify
that users of the Wayback Machine are not to download data from the collection. Not all websites are available because many
website owners choose to exclude their sites. As with all sites based on data from web crawlers,
the Internet Archive misses large areas of the web for a variety of other reasons. International biases have also been found
in its coverage, although this does not seem to be the result of a deliberate policy. The use of the term “Wayback Machine” in the
context of the Internet Archive has become so common that “Wayback Machine” and “Internet
Archive” are almost synonymous. This usage occurs in popular culture, e.g.,
in the television show Law and Order: Criminal Intent, an extra playing a computer tech uses
the “Wayback Machine” to find an archive of a student’s Facebook style website. Snapshots usually take at least 6–18 months
to be added. The Save Page Now archiving feature was made
available in October 2013, accessible on the lower right of the Wayback Machine’s main
page. Once a target URL is entered and saved, if
the target website permits access via robots.txt, the web page will become part of the Wayback
Machine. Archive-It Created in early 2006, Archive-It is a web
archiving subscription service that allows institutions and individuals to build and
preserve collections of digital content and create digital archives. Archive-It allows the user the option to customize
their capture or exclusion of web content they want to preserve for cultural heritage
reasons. Through a web application, Archive-It partners
can harvest, catalog, manage, browse, search and view their archived collections. In terms of accessibility, the archived websites
are full text searchable within seven days of capture. Content collected through Archive-It is captured
and stored as a WARC file. A primary and back-up copy is stored at the
Internet Archive data centers. A copy of the WARC file can be given to subscribing
partner institutions for geo-redundant preservation and storage purposes to their best practice
standards. The data captured through Archive-It is periodically
indexed into the Internet Archive’s general archive. As of March 2014, Archive-It had over 275
partner institutions in 46 U.S. states and 16 countries that have captured over 7.4 billion
URLs for over 2,444 public collections. Archive-It partners are universities and college
libraries, state archives, federal institutions, museums, law libraries and cultural organizations,
including the Electronic Literature Organization, North Carolina State Archives and Library,
Stanford University, Columbia University, American University in Cairo, Georgetown Law
Library and many others. Books collections
Text collection The Internet Archive Text Archive collection
includes digitized books and special collections from various libraries and cultural heritage
institutions from around the world. The Internet Archive operates 33 scanning
centers in five countries, digitizing about 1,000 books a day for a total of over 2 million
books, financially supported by libraries and foundations. As of July 2013, the collection included 4.4
million books with over 15 million downloads per month. As of November 2008, when there were about
1 million texts, the entire collection was over 0.5 petabytes, which includes raw camera
images, cropped and skewed images, PDFs, and raw OCR data. Between about 2006 and 2008 Microsoft Corporation
had a special relationship with Internet Archive texts through its Live Search Books project,
scanning over 300,000 books which were contributed to the collection, as well as financial support
and scanning equipment. On May 23, 2008, Microsoft announced it would
be ending the Live Book Search project and no longer scanning books. Microsoft made its scanned books available
without contractual restriction and donated its scanning equipment to its former partners. Around October 2007, Archive users began uploading
public domain books from Google Book Search. As of November 2013 there were over 900,000
Google-digitized books in the Archive’s collection: the books are identical to the copies found
on Google, except without the Google watermarks, and are available for unrestricted use and
download. Brewster Kahle revealed in 2013 that, after
seeing all the books coming in, the Archive found out that this archival effort was set
up by Aaron Swartz, who with a “bunch of friends” downloaded books slow enough and from enough
computers to stay within Google’s restrictions. They did this to ensure public access to the
public domain: restricting access to works unencumbered by copyright is compared to locking
up a national park with high walls and armed guards to keep everyone out. The Archive ensured the items attributed and
linked back to Google, which never complained, while libraries “grumbled”. According to Kahle, this is an example of
Swartz’s “genius” to work on what could give the most to the public good for millions people;
another example was PACER. In fact, besides books, the Archive offers
free and anonymous public access to more than four millions court opinions, legal briefs,
or exhibits uploaded from the United States Federal Courts’ PACER electronic document
system via the RECAP web browser plugin. All of these documents are in the public domain,
but had been kept from the public behind a federal court paywall. On the Archive, they’ve been accessed by over
6 millions people. Number of texts for each language
Number of texts for each decade Open Library The Open Library is another project of the
Internet Archive. The site seeks to include a web database for
every book ever published: it holds 23 million catalog records of books. It also seeks to be a web-accessible public
library: it contains the full texts of about 1,600,000 public domain books, which are fully
readable, downloadable and full-text searchable; it offers access to an e-book lending program
for over 250,000 recent books not in the public domain, in partnership with over 1,000 library
partners from 6 countries. Open Library is a free/open source software
project, with its source code freely available on the Open Library site. Internet Archive Lending Library
The Internet Archive Lending Library is a digital library of ebooks at . This
is a new system to loan digital books over the Internet. The current technology behind this loaning
system is Adobe’s Content Server which uses digital rights management to ensure only one
person can see a particular book at one time. This collection contains over 12,000 items. Media collections In addition to web archives, the Internet
Archive maintains extensive collections of digital media that are attested by the uploader
to be in the public domain in the United States or licensed under a license that allows redistribution,
such as Creative Commons licenses. Media are organized into collections by media
type, and into sub-collections by various criteria. Each of the main collections includes an “Community”
sub-collection where general contributions by the public are stored. Moving image collection
The Internet Archive holds a collection of approximately 3,863 feature films. Additionally, the Internet Archive’s Moving
Image collection includes: newsreels, classic cartoons, pro- and anti-war propaganda, The
Video Cellar Collection, Skip Elsheimer’s “A.V. Geeks” collection, and ephemeral material
from Prelinger Archives, such as advertising, educational, and industrial films and amateur
and home movie collections. Subcategories of this collection include:
IA’s Brick Films collection, which contains stop-motion animation filmed with Lego bricks,
some of which are “remakes” of feature films. IA’s Election 2004 collection, a non-partisan
public resource for sharing video materials related to the 2004 United States Presidential
Election. IA’s FedFlix collection, Joint Venture NTIS-1832
between the National Technical Information Service and Public.Resource.Org that features
“the best movies of the United States Government, from training films to history, from our national
parks to the U.S. Fire Academy and the Postal Inspectors”
IA’s Independent News collection, which includes sub-collections such as the Internet Archive’s
World At War competition from 2001, in which contestants created short films demonstrating
“why access to history matters”. Among their most-downloaded video files are
eyewitness recordings of the devastating 2004 Indian Ocean earthquake. IA’s September 11th Television Archive, which
contains archival footage from the world’s major television networks of the terrorist
attacks of September 11, 2001, as they unfolded on live television. Some of the films available on the Internet
Archive are: Machinima archive
One of the sub-collections of the Internet Archive’s Video Archive is the Machinima Archive. This small section hosts many Machinima videos. Machinima is a digital artform in which computer
games, game engines or software engine are used in a sandbox mode like mode to create
motion pictures, recreate plays or even publish presentations/keynotes. The archive collects a range of Machinima
films from internet publishers such as Rooster Teeth and as well as independent
producers. The sub collection is a collaborative effort
between the Internet Archive, the How They Got Game research project at Stanford University,
the Academy of Machinima Arts and Sciences and TV News Search & Borrow In September 2012, the Internet Archive launched
the TV News Search & Borrow service for searching U.S. national news programs. The service is built on closed captioning
transcripts and allows user to search and stream 30-second video clips. Upon launch, the service contained “350,000
news programs collected over 3 years from national U.S. networks and stations in San
Francisco and Washington D.C.” According to Kahle, the service was inspired
by the Vanderbilt Television News Archive, a similar library of televised network news
programs. In contrast to Vanderbilt, which limits access
to streaming video to individuals associated with subscribing colleges and universities,
the TV News Search & Borrow allows open access to its streaming video clips. In 2013, the Archive received an additional
donation of “approximately 40,000 well-organized tapes,” from the estate of a Philadelphia
woman, Marion Stokes. Stokes “had recorded more than 35 years of
TV news in Philadelphia and Boston with her VHS and Betamax machines.” Audio collection The Audio Archive includes music, audio books,
news broadcasts, old time radio shows and a wide variety of other audio files. There are over 200,000 free digital recordings
in the collection. The subcollections include audio books and
poetry, podcasts, non-English audio and many others. The Live Music Archive sub-collection includes
over 100,000 concert recordings from independent artists, as well as more established artists
and musical ensembles with permissive rules about recording their concerts such as the
Grateful Dead, and more recently, The Smashing Pumpkins. Also, Jordan Zevon has allowed Internet Archive
to host a definitive collection of his father Warren Zevon concert recordings. The catalog ranges from 1976–2001 and contains
1,137 free songs. Netlabels The Archive has a collection of freely distributable
music that is streamed and available for download via its Netlabels service. The music in this collection generally have
Creative Commons-license catalogs of virtual record labels. NASA Images
The NASA Images archive was created through a Space Act Agreement between the Internet
Archive and NASA to bring public access to NASA’s image, video, and audio collections
in a single, searchable resource. The IA NASA Images team worked closely with
all of the NASA centers to keep adding to the ever-growing collection. The site launched in July 2008
and had more than 100,000 items online at the end of its hosting in 2012. Open Educational Resources
Open Educational Resources is a digital collection at . This collection contains
hundreds of free courses, video lectures, and supplemental materials from universities
in the United States and China. The contributors of this collection are ArsDigita
University, Hewlett Foundation, MIT, Monterey Institute and Naropa University. Other services and endeavors
Physical media Voicing a strong reaction to the idea of books
simply being thrown away, and inspired by the Svalbard Global Seed Vault, Kahle now
envisions collecting one copy of every book ever published. “We’re not going to get there, but that’s
our goal”, he said. Alongside the books, Kahle plans to store
the Internet Archive’s old servers, which were replaced in 2010. Software
The Internet Archive is “the largest collection of historical software online in the world”,
spanning 50 years of computer history in terabytes of computer magazines and journals, books,
shareware discs, FTP websites, video games, etc. In 2013 the Internet Archive began to provide
abandonware video games browser-playable via MESS, for instance the Atari 2600 game E.T.
the Extra-Terrestrial. Controversies and legal disputes Omni magazine
In a story at his Web site headed “What the heck is going on at Internet Archive?”, author
Steven Saylor noted, “Sometime in 2012, the entire run of Omni magazine was uploaded
at Internet Archive…Since those old issues must contain hundreds of works still under
copyright by numerous contributors, how is this legal?” At least one contributor to the magazine,
author Steve Perry, has publicly complained that he never gave permission for his work
to be uploaded, and it has been noted that all issues containing the work of Harlan Ellison
have apparently been taken down. Glenn Fleishman, investigating the question
“Who Owns Omni?”, writes that “Almost all of the authors, photographers, and artists
whose work appeared in the magazine had signed contracts that granted only short-term rights….[No
one] could simply reprint or post the content from older issues.” Grateful Dead
In November 2005, free downloads of Grateful Dead concerts were removed from the site. John Perry Barlow identified Bob Weir, Mickey
Hart, and Bill Kreutzmann as the instigators of the change, according to a New York Times
article. Phil Lesh commented on the change in a November
30, 2005, posting to his personal website: It was brought to my attention that all of
the Grateful Dead shows were taken down from right before Thanksgiving. I was not part of this decision making process
and was not notified that the shows were to be pulled. I do feel that the music is the Grateful Dead’s
legacy and I hope that one way or another all of it is available for those who want
it. A November 30 forum post from Brewster Kahle
summarized what appeared to be the compromise reached among the band members. Audience recordings could be downloaded or
streamed, but soundboard recordings were to be available for streaming only. Concerts have since been re-added. National security letter On May 8, 2008, it was revealed that the Internet
Archive successfully challenged an FBI National Security Letter asking for logs on an undisclosed
user. Uncensored hosting
On August 17, 2011, Middle East Media Research Institute published “Al-Qaeda, Jihadis Infest
the San Francisco, California-Based ‘Internet Archive’ Library” which detailed how members
can post anonymously and enjoy free uncensored hosting. Opposition to Google Books settlement
The Internet Archive is a member of the Open Book Alliance, which has been among the most
outspoken critics of the Google Book Settlement. The Archive advocates an alternative digital
library project. Opposition to SOPA and PIPA bills
The Internet Archive blacked out its website for twelve hours on January 18, 2012, in protest
of the Stop Online Piracy Act and the PROTECT IP Act bills, two pieces of pending legislation
in the United States Congress that they claim will “negatively affect the ecosystem of web
publishing that led to the emergence of the Internet Archive”. This occurred in conjunction with the English
Wikipedia blackout, as well as numerous other protests across the Internet. Ceramic Archivists collection The Great Room of the Internet Archive features
a collection of nearly 100 ceramic figures by Nuala Creed representing employees of the
Internet Archive. This collection, commissioned by Brewster
Kahle and sculpted by Nuala Creed, is ongoing. List of digitizing sponsors for ebooks This is a list of some digitizing sponsors
for ebooks in the Internet Archive. See also
Lists of Internet Archive’s collections References Further reading
Kahle, Brewster, “Archiving the Internet” at the Wayback Machine
Ringmar, Erik, “Liberate and Disseminate,” Times Higher Education Supplement, April 10,
2008. External links
Official website Official blog Internet Archive Mirror at the Bibliotheca
Alexandrina, Egypt The Audio Archive collection at the Internet
Archive [more] The Internet Archive Lending Library collection
at the Internet Archive [more] The Internet Archive Press collection at the
Internet Archive [more] The Internet Archive Presents collection at
the Internet Archive [more] The Moving Image Archive collection at the
Internet Archive [more] The Netlabels collection at the Internet Archive
[more] The Text collection at the Internet Archive
[more] The Other Minds Archive – New Music Resource
from Web Archiving at, details of Internet
Archive operations Pictures and descriptions of the Wayback Machine
hardware in 2003, with cost information Current Petabox storage hardware
Earliest known website at the Wayback Machine Internet Archive
Early websites from 1996 Brewster Kahle. “Scanning Center Fire — Please Help Rebuild”. Internet Archive Blogs. 

Leave a Reply

Your email address will not be published. Required fields are marked *