Birth pains of a digital library

Ever since the first bytes, the Internet has been seen as an opportunity to spread an unlimited amount of knowledge. For thousands of years, libraries have tried to complete such a task, and the development of information technology quite logically brought about the idea of digital knowledge storage. Whether fortunately or unfortunately, the occurring data layer touches the current world order in a way in which no-one has an integral picture of its ultimate impacts.
There are fewer and fewer physical obstacles in the way of information dissemination; technology develops and is becoming cheaper, and the biggest problem in spreading knowledge lies in its substantial value. As in certain cases, knowledge cannot be measured in terms of money; it is at times protected as eagerly as the Chinese governing class protected the silk-making secrets at the beginning of the first millennium. Where big distances and the protectionist way of life were of great help for the Chinese, now it is mostly copyright and patent legislation that must manage the protection of the spread of useful knowledge or application.

Copyright and patent acts are by nature contradictory. On the one hand, the efforts of each person and company must be worthily rewarded; on the other hand, there is a question of where we would have reached with our culture in the context of royalties? Even civilisation would have been different if a Sumer corporation had managed to patent the wheel. Society has been developing together with the spread of knowledge and the example of Mesopotamia is demagogic, but it characterises the incompatible nature of the problem. A pharmaceutical company invests billions of dollars in the preparation of medicines, which they would like to earn back. What is more important when millions of people die in Africa each year with no money for medicine but who could get help from generic drugs: the company’s right to receive a worthy reward for its work or the lives of humans? The world faces such choices every day and there are no easy solutions in the current organisation of the economy.

Ever since data digitisation, people have asked whether it is possible to create an integral digital library in the first place due to the references made to patent and copyright acts? While the technological barrier of converting data into bytes and the saving thereof remains lower and lower each year and entire libraries could be copied in practically a single click, legislation, however, gets more complicated. A recent example is ACTA (Anti Counterfeit Trade Agreement) with which developed countries try to hinder the production and spread of counterfeit products by the harmonisation of legislation on intellectual property rights. The documents leaking from the initial discussions reveal, however, that under the slogan of protecting the rights of authors, they were seriously discussing about whether countries or Internet service providers should follow the web activities of people and decrease the role of courts in the punishment for infringements.

The problem with legislation like ACTA is not that much related to the protection of intellectual property rights but it restricts the development of the economic system. The Internet has practically turned the world into a village where information moves from one side to another in a heartbeat, but legislation is in the era where knowledge moved from one part of the world to another on camels or by sailing ships.

Let us take the film industry, which constitutes a remarkable share of the US economy. There is a movie rental company called Netflix that operates in the United States of America and Canada, the service of which has developed to the point where roughly 17,000 different pieces of cinema art available on DVD could be watched any time by video streaming with an Internet connection of sufficient speed, and 10,000 different DVDs could be rented by post. Even the cost of the service is sensible and one would think that many people would be prepared to use the service for about a hundred kroons in Estonia, as well. This would also be in the interests of Netflix, but nevertheless the service has not reached Europe yet, and that is not because of technological obstacles, but rather legal obstacles. Film producers (as with representatives of many other business fields) still find that people are not equal, and at times a European citizen must pay much more money for the same product.

Information wants to be free?

When you develop the topic, then it is odd to discover that the majority of Hollywood production is freely accessible for all movie lovers over the Internet. The solutions are called file-sharing applications, one of the most popular and advanced right now being BitTorrent. Series parts reach the network within a few hours of the TV premiere, the most popular books are published in a digital form on average a day or two after the presentation, and films of hand-camera quality a day or two after reaching the big cinema. Sometimes even weeks before the premiere. Let alone computer programmes.

Next to official distribution channels, a huge parallel universe has occurred with no central junction living in all the computers of people who use file-sharing applications that no companies or authorities are able to control. It is a dragon where three heads occur after chopping off one that is demonstrated by the saga of The Pirate Bay. Now, they are trying to ban the problem, but the genie is out of the bottle and sooner or later the media industry must learn to cope with the situation. The above-mentioned BitTorrent network without central junction where the data exchange rate is in correlation with the number of users is not an enemy, but a huge opportunity. Why they have not been able to apply it to the benefit of all parties is a matter of business models.

Therefore, the current situation could be described by a slogan saying that as long as people have a little bit of curiosity left, information wants to be free, no matter how horrible that would be for the wallets of companies or private persons. So far, no reasonable solutions have been found for the problem and the situation continues where practically every Internet user may become a criminal in an unintentional and unknown way.

Too many choices?

The situation is much better with pieces that are not covered by cultural heritage and copyright. Thoughout the world, governments have begun to digitise their history at a faster or slower pace, depending on their resources. The Japanese Parliament allocated 12.6 billion yen (ca 2 billion kroons) to digitise the entire repository of the Japanese National Library in two years, almost all European countries contribute large sums of money to digitisation of their heritage, and the Estonian National Library has also begun with it. With the help of the European Union, the development of a digital library called a common Europeana has been started, which includes more than 14.6 million photos, texts, sounds, and videos as of writing this. With the support of world famous universities and research establishments, a digital Encyclopaedia of Life will be completed by 2017 to teach people all about the 1.8 million known plants, animals, and species of other life with the help of abundant multimedia.

The digitisation of “official knowledge”, whether scientific works, museum collections, or classical literature, is usually performed “all, at once, and now”. The situation is much more complicated with the evaluation and saving of digital network information production consisting of billions of devices and lots of people. When the national daily papers are stored in full, then what will be the attitude towards web diaries? Is the web page of each company equally important? For example, if the views of PR expert Daniel Vaarik or Director of the Institute of Technology of the University of Tartu, Erik Puura, should be stored due to the social position of the persons, what should be the attitude towards information like “what I did today” published by random people? The latter might be of even more interest to researchers in the sense of context; also, the digital print of several small enterprises could provide a better overview of the economic situation of a particular moment than a large enterprise of that era.

Probably, we should accept the inevitable that it is impossible to store everything and that future researchers will depend on data storage choices. The Head of the Digital Projects of the British Library, Adam Farquhar, finds that in a sense we have a clearer overview of the beginning of the twentieth century than of the twenty first one. The situation gets more confusing due to the never-ending and mostly unstructured information in zettabytes.

In spite of it all, every piece of knowledge archived for the sake of the future is important, which in turn leads to the question of the format for storing data, both in the sense of data carriers and software. The most famous case is of course NASA, which at one point during updating its hardware and software discovered that there was no device in the organisation that would be able to read the data carriers of the Apollo mission to the moon. That particular problem was solved, however, but would it have been possible to solve in fifty years’ time? Contemporary data storage devices are largely in a state where it is more secure storing information in a ceramic and electronic form in the perspective of centuries. The situation is more complicated with software formats: there are hundreds, if not thousands, of ways of saving documents, sound, photos, and videos. Even the database formats are different and although the situation improves year-by-year, lifting information from one database to another is quite troublesome.

In spite of technological restrictions, the future is still digital. Data storage gets cheaper and cheaper and the price of a gigabyte has decreased below one kroon. Recently, the scientists of a Hong Kong University demonstrated how to store data in the form of a modified DNA into E.coli bacteria. One gram of bacteria (about ten million cells) contains nine hundred bytes of data, which they say would be preserved forever as the data is transferred together with the multiplication of bacteria.1 The example was brought towards what direction storage technology takes, but until then security for the future is only provided by the annual rewriting of data and the formatting thereof, if necessary.

Data without search like a car without fuel

One of the biggest problems of the big global digital library is finding and searching for data. On the one hand, Google, Bing, and other search engines do the big work in indexing the data. On the other hand, a big share of the Internet is hidden from search engines. The third question relates to all kinds of personal information concerning people so that they have begun to talk about a “right to be (digitally) forgotten”. Each step made in the Internet leaves a trace, let alone every photo and twitter left in a social network.

When storing old time literature, a photo could be taken of book pages and inserted into the archive. A page of a book is definitely better if that book were not available, but the situation would be a lot better if the substance of the book were structured. Which means an amount of additional work and would not be feasible with current opptions, unfortunately. Scanners and other devices mean that books are digitised with a record of them as a photo file. Optical character recognition software (OCR) helps transform the photo files made of texts into the language understood (and searched) by the computer, the ability of which is sufficient in big languages but not satisfactory in smaller ones.

For a while now, the majority of information is born electronically and at some point also the entire heritage would be translated into the language of the machines. Sooner or later, the development of technology will bring about a virtual zero where all information is digital and this way accessible at any moment for every person connected to the network. The Internet is already uncovered by its measures; people do not need so much knowledge but rather the orientation skills to handle it. Next to creating new knowledge, the skills to administer and analyse the data is at least as important.

The global network is huge, at times with quite dark corners. It is an unstructured mass of data, which is completely unmapped with its opportunities. Search engines do a good job by indexing, but also for them a large amount of the network is invisible. We could also add the problem of millions of web pages being developed and closed every day, let alone the forgotten data piles of digital refusal disposal sites. Semantics or connecting information with meaning is currently a big challenge of the global network, and only after we have solved that we could talk about a huge and all-embracing digital library.

Order a digital book and help digitise cultural heritage

There are millions of printed materials in European libraries originating from the 17th-20nd centuries that every library is gradually digitising. As the funds for cultural establishments are fairly restricted everywhere, companies and private persons can also assist in digitations of the written word. The name of the service is EOD: e-book on demand offered by the Estonian National Library.

By ordering the digitisation of a piece in the collection of the national library, it would be added to the public digital catalogue. The conversion of a work of more than 200 pages to the full text search PDF-format costs the client 35 euro.

E-book on demand is a part of a bigger European Union library digitisation project that began in 2006, Books2ebooks, with twenty libraries from ten countries participating in it. Sixteen of these have opened their digital archives, and in September of this year they introduced a demo version of a common search engine. Together, they are looking for digital recording technology solutions and improvements in the EOD network.

More information: books2ebooks.eu and the digital library of the Estonian National Library located at digar.nlib.ee.


Digitisation is of huge assistance for historians

Historians directly benefit from the digitisation of intellectual heritage. While before you practically had to travel to a land far-far away to work with historical materials, now items such as the Sumerian cuneiform scripts are available in a digital form to everyone. For historians dealing with patristics, the works of Greek and Latin priests are available in several translations and publications.

All Estonian archives deal with digitisation, which in the form of church registers opens new opportunities for studying genealogy, for example. Digital map collections help visualise ancient settlements, etc. Comparing various sources has always been important in history, and it is remarkably easier in a digital form.

http://www.digitaljournal.com/article/300831

http://www.bibliotecapleyades.net/default.htm
-mõned digiraamatukogud on täiesti olemas ja kasutatavad