Legal Deposit in Australia: from print to digital

Meredith BattenNational Library of Australia

Good afternoon. It is a great honour to present a lecture at today’s international symposium commemorating the 70th anniversary of legal deposit in Japan. I would like to begin by congratulating the Diet Library on the Library’s 70th anniversary and wish you every success in this and coming years.

1

The National Library of Australia has its origins in the Commonwealth Parliamentary Library, which was established in 1901 and served both members of Parliament and the nation. The National Library Act of 1960 formally separated the National Library from the Parliamentary Library and a new building for the National Library’s collections and services was opened in 1968.

2

The National Library is also celebrating an important anniversary this year as 2018 marks the 50th anniversary of the opening of our building.

3

The Library’s collection size is currently estimated at 10 million items with 5.69 petabytes of digital storage. It has approximately 380 staff.

4

One of the Library’s strengths is its Asian Collections, which contain about 650,000 items, including a significant collection of Japanese material. The 1864 map of Tokyo, shown here, is just one of those items.

5

In 2016/17, the National Library welcomed half a million onsite visitors

6

The Library also had 1.2 million offsite visitors, plus 22.3 million visits to Trove, the online library aggregator and search engine hosted by the National Library.

7

In accordance with the National Library Act, the Library’s primary role is to ensure that Australia’s documentary heritage is collected, preserved and made accessible for the benefit of all Australians, both now and in the future.

8

As the 2011 IFLA Statement on Legal Deposit recognises, legal deposit is fundamental to the National Library of Australia’s ability to carry out its statutory role – just as it is for other national libraries around the world.

9

Australia has a long history of legal deposit. The earliest legal deposit provisions were formulated at the state level before Australia became an independent nation. The earliest state legislation was passed by the state of Victoria in 1869, followed by the states of South Australia in 1878 and New South Wales in 1879. Today nearly all Australian states and territories have their own legal deposit scheme. The state schemes vary slightly as to type of material, number of copies and libraries to be supplied, but most now make some provision for digital material. These state legal deposit laws work in conjunction with the federal legal deposit scheme established at the national level, which is my focus today.

10

The first national legal deposit scheme was introduced in Australia in the Copyright Act of 1912. Under this law, Australian publishers were required to deliver the ‘best’ copy of each book published to the Librarian of the Commonwealth Parliament within a month of publication. The books supplied under this legislation later formed the foundation of the National Library’s national collection when the Library was formally established in 1960.

11

The 1912 Act was eventually superseded by the Copyright Act of 1968 and publishers were obliged to supply copies to the National Library in place of the Parliamentary Library. Otherwise, the legal deposit provisions remained virtually unchanged for over a century.For most of this period, they served the country well - ensuring that Australia’s publications were collected, preserved and made accessible for all Australians. However, the emergence of electronic format publishing in the 1980s and online publishing in the 1990s, meant that an increasing proportion of Australian publications could not be acquired under legal deposit. This is because the legislation only prescribed deposit of print material. While the Library set up voluntary deposit schemes to collect electronic material, they were labour-intensive to manage and had limited success, often resulting in patchy holdings. While the Library had considerably more success in web archiving, this activity was limited to freely available online material.Over the next two decades, the Library lobbied for the legal deposit provisions to be extended to electronic publications, making submissions to a number of Commonwealth government enquiries on the subject. During this period, the Library also developed its understanding of the concerns of publishers and the requirements for an effective and workable scheme, as well as learning from the experience of other countries. This resulted in Australian legislation that was innovative, flexible and fit for purpose in the 21st century.

12

In February 2016, new legislation governing the national scheme came into effect, which extended legal deposit to electronic publications for the first time.The new legal deposit provisions are format and technology neutral. They cover all Australian publications whether published offline (such as print or CD-ROM) or online (such as ebooks, ejournals and websites). The provisions also require that deposited material is free of digital rights management technology, enabling the Library to ensure its long-term preservation.A particularly innovative feature of the legislation is that it allows the Library to request online material without any restriction based on place of publication or distribution. This enables the Library to request self-published works by Australians, even when the authors are using online services based overseas.

13

The legislation covers all publications that are literary, dramatic, musical or artistic works distributed in any format. These include books, serials, maps, music scores and websites.Material that is exempt from the scheme includes audio-visual and gaming material; software applications and material of limited or restricted distribution.

14

In terms of publisher obligations, the legislation makes a distinction between offline and online publications.Publishers of offline publications are required to deliver their work to the National Library within one month of publication. Publishers still need to supply the ‘best copy’ (e.g. hardback over paperback).Publishers of online publications are only required to deliver their work within one month of receiving a request from the Library. This request can be automated, for example via a web harvester. If online publishers do not receive a request, they are not obliged to deposit. Publishers who produce works that are both offline and online such as a hardcopy book and an ebook version are required to deposit the offline copy within one month of publication unless the Library requests the online version. Increasingly, the Library is encouraging government, serial and bulk commercial publishers to deposit the online version in place of print. This is in line with the Library’s strategic priority to scale up the Library’s digital collecting.

15

There are three mechanisms through which the Library currently acquires online electronic publications. Firstly, the edeposit service, which I will talk more about soon; secondly, bulk deposit mechanisms for publishers wishing to deposit large numbers of files; and, thirdly, web archiving of online material freely available through the web, both through selective harvesting of particular websites and an annual harvest of the whole .au domain. Australia was the first national library to undertake web harvesting, establishing a web archive in 1996, and we now have a national web archive collection amounting to more than half a petabyte of data or 9 billion files. The new legal deposit provisions assist with large scale bulk collecting as the legislation specifically provides for collecting using harvest robot agents. They also enable us to archive websites without seeking permission.

16

The edeposit service was released on the same day the new legislation came into effect in 2016. The service is a web-based portal or web form through which publishers can deposit individual ebooks, emaps, emusic scores or ejournals and is the public interface of an end-to-end digital collection management system. The service was developed as part of a six year project to re-develop the Library’s digital infrastructure. The primary aim of the service is to provide Australian publishers with a simple, secure and effective way of depositing their online publications in compliance with the legal deposit provisions - and to maintain publishers’ trust in the National Library as the custodian of the national collection.

17

The edeposit service is fully integrated with other National Library systems, such as the Library’s management system (Voyager) and RefTracker (used for serials communications), Trove and off the shelf products, such as Preservica, a proprietary digital preservation management system.

18

This slide shows the current workflow for individual books, music scores and maps.Much of the process is completely automated and requires no intervention by staff. The publisher submits their digital publication and metadata via the edeposit service. The file and metadata are automatically uploaded to the Library’s Digital management system. Staff are notified when a publication is deposited by an email alert. They then upgrade the catalogue record. Importantly, as soon as metadata is transferred to the Library’s catalogue and national database the material is discoverable and available for use by readers.Long-term preservation of content is critical to successful implementation of the Library’s mandate. Data and files, whether deposited through the edepositservice or a bulk process, are transferred to Preservica via an automated process to ensure ongoing preservation. Digital Preservation staff perform regular post-ingest audits to ensure the quality and completeness of ingested content and metadata.Published content from offline electronic publications, such as CD-ROMs, is also ingested to Preservica. Unlike the edeposit ingests, this is a manual process as it was not economical to automate it given the low and decreasing volume of offline electronic material.

19

The new legislation has enabled the National Library to collect 11,200 ebooks, 300 electronic music scores, 1500 ejournal titles and 26,300 electronic map files in the first two years of its operation. In addition, the Library has conducted three whole of domain harvests and selectively archived over 5000 online publications during this period.From July 2017 to April 2018, online publications have made up approximately 40 percent of books and serials acquired on legal deposit. For government, serials and maps, acquisition of online publications is beginning to overtake acquisition of publications in print.

20

Edeposit has brought a number of efficiencies for the Library, publishers and users. These include a streamlined workflow for publishers to deposit material with the Library, the reuse of publisher metadata to create library catalogue records and the availability of serial issues immediately upon deposit by publishers. This contrasts with print serial issues, for which there can be a delay of up to two or three weeks between publication of a serial issue and its availability to Library readers.

21

Through the edeposit service, publishers select the access conditions to be applied to their publication. This makes it possible for publishers to agree to a broader level of access than permitted under the Australian Copyright Act. The proportion of material available off-site is significant – about 40 percent of material deposited so far allows offsite access, meaning that these publications are available to Australians wherever they live or might be.A large proportion of this content comes from community and local organisations, covering a wide range of subject matter. Many of these titles are 'born digital' and may not have been accessible via a Library anywhere before.

22

A number of factors led to the successful implementation of electronic deposit in Australia at the Library, including an innovative system and stakeholder engagement with both staff and publishers. Staff quickly embraced the shift to digital and the investment in training, communication and involvement of staff in workshops and workflow discussions undoubtedly contributed to this result.

23

Even more critical was the emphasis given to engagement with publishers. This included webinars for publishers explaining the new legislation, which were presented by Library staff under the auspices of the Australian Publishers’ Association. We also produced a legal deposit fact sheet emailed out to publishers and provided new web content written in plain language and avoiding library jargon, as well as offering a single point of contact for publishers’ queries. In developing bulk deposit mechanisms, the Library worked closely with a few large publishers, such as Penguin Random House and Wiley, and we are indebted to their assistance through this process.

24

The success of the Library’s efforts, particularly around publisher engagement, was borne out by the results of a publisher survey conducted earlier this year.The online survey was undertaken to assess the level of trust publishers have in the National Library as a leader and partner. The survey was distributed to about 1600 publishers. Over 500 responses were received – a 30 percent response rate.

25

The results of the survey revealed a high level of awareness and compliance with legal deposit amongst those publishers who responded. 90 percent of respondents deposited all their publications with the Library, while another nine percent deposited some of their publications. Only one percent, representing nine individual publishers, responded that they did not deposit with the Library.

26

Reasons given by publishers for not depositing either any or some of their publications included:lack of awareness of legal deposit obligations (e.g. some publishers thought they had to only deposit with their State Library);depositing publications in only one format (e.g. print and not digital or vice versa); and Other reasons relating to the nature of the publication or its recent publication.

27

More than half of respondents were depositing some or all of the publications via the Library’s edeposit service, while 40 percent were still mailing or delivering hard copies to the Library. Another 4 percent had their publications archived by the Library’s web harvester.Publishers depositing through the edeposit service clearly recognised its benefits. For example, thirty-eight percent of respondents acknowledged that edeposit saved them time and postage costs. A number of respondents also commented on the ease of using the new service to deposit digital publications. For example, one respondent noted ‘Quick and easy to work with when depositing.’

28

Particularly pleasing was the high level of trust publishers had in the Library, especially the Library’s role in preserving digital publications as evidenced by some of the comments received by respondents, including the quote shown here.

29

Despite the successful implementation of electronic deposit, a number of issues or challenges remain for the Library. These include the changing nature or evolution of publishing, technical limitations of infrastructure, access and a changing workforce.

30

Publishing is undergoing a transformation in Australia as it is elsewhere. Online publishing has made it easier for everyone to be a publisher, resulting in the growth of small, independent and niche publishers, as well as self-publishing. In addition, Australian publications and Australian publishing are not confined to the Australian continent. Many self-publishers, for example, have their works published on overseas hosting platforms, such as Amazon. The challenge for the Library is to find new ways of raising awareness about legal deposit and selling its benefits to all Australian publishers, no matter their size or where their platform may be located. The recent publisher survey revealed that publishers in particular valued the Library’s digital preservation activities, so that is one aspect we can focus on in future promotional activities.

31

Technical limitations of infrastructure can also be a challenge. For example, the Library’s current edeposit service places restrictions on the type of publication formats it will accept, as well as file size. Some types of publications, particularly multimedia publications, already fall outside the current capability of the edeposit service. Investment in updating technical infrastructure will be critical to ensuring the Library’s ongoing capacity to collect new innovative electronic publications.Collecting and preserving are only two-thirds of the equation. The last, and no less important part, is the ability to provide access to these publications. Under the current Copyright Act, access to electronic publications is restricted to the Library’s premises unless the publisher allows broader access. About 60 percent of the Library’s current intake of electronic publications received through the edeposit service can only be viewed in the National Library’s reading rooms. This includes all commercial in-copyright content. Web content harvested under the annual whole of domain harvests is also not yet accessible. However, the Library is currently working to deliver all our web archive collections, including the domain harvests, through a single search and delivery service via Trove by the end of 2018. Innovative technical solutions will be employed to index the web data and create improved relevant ranked search results, opening up a wealth of text-searchable Australian historic web content for the first time.

32

The final challenge is the Library’s changing workforce and skills base. Traditionally, the Library’s collection processing areas have employed about 25 percent or more of the National Library’s total staff. This work can be labour intensive and some of the standards in use date back to printed catalogue cards or even earlier. Inside the shift from the physical to the digital collection is an even bigger revolution: away from using data created by qualified professional staff towards making much greater use of the data created by publishers. For example, books deposited by large commercial publishers such as Penguin Random House come with a large amount of publisher metadata, including publisher subject headings and summaries. By harnessing such metadata, the end goal is to provide greater access for users, and to shift staff stewardship over the catalogue to oversight, rather than creating individual records. This also enables the Library to manage our resourcing into the future, without facing ever-increasing costs as our collection increases.However, legal deposit publications in print remain a strong component of collecting and will do for some years to come. With a shrinking resource base, another of the Library’s challenges is to find similar efficiencies in managing the print intake as we have for digital. To this end, the Library has embarked on a review of how collection description work is undertaken at the Library. The key objective of this review is to re-assess the Library’s collection description policies, documentation procedures, and the workflows and processes broadly associated with collection management matters. 33

Despite these challenges, the future holds enormous promise. As a result of the success of the Library’s edeposit service, the National Library is now collaborating with Australian State and Territory Libraries to develop the National edeposit network, or NED as it is known. This new service will address, at least in part, the challenges that I have just described.

34

35

NED is an online service for the deposit, management, archiving, preservation, discovery and delivery of published electronic material across Australia and will be employed by the National Library together with the nine State and Territory Libraries. The new service is expected to be released in early 2019.NED will offer benefits to publishers, users and libraries.

36

Benefits to publishers include: an easy, one-stop online deposit and options for bulk deposit and the ability to meet both state and national legislative responsibilities through a single deposit; as well as, digital preservation and long-term access to their content; and options for access conditions.

37

The benefits to users are significant as NED will result in a vastly improved level of access to digital publications. These benefits include access to publications collected by all the libraries within library premises and more publications openly accessible. In addition, there will be less delays in availability of publications as access will be provided via the publisher metadata almost as soon as the material is deposited.

38

The benefits to libraries are also obvious. In particular, NED provides economies of scale, and an ability to offer a service that some individual State Libraries might struggle to do aloneThe existing infrastructure for digital preservation and long-term, secure storage is a major benefit.Another advantage with this approach is that together the Libraries have a collective voice when negotiating with publishers with consistent policies and strength in numbers.

This slide gives a brief preview of the beta version of the new National edepositnetwork and shows the entry or home page of the web portal for publishers.The development of NED leverages off the work undertaken to develop the National Library’s own edeposit service, but with an improved user interface designed in consultation with publishers. It also has an enhanced admin or back-end system, which will allow the State Libraries to individually manage their workflows, including the legal deposit intake of publishers from their own state.

39

NED will offer a streamlined design and deposit workflow for publishers. It will cater for the deposit of books, maps, music and serials – just as the current edeposit service does. NED will also capture publisher metadata, which will be transformed into basic catalogue records that the Libraries can then update and enhance.NED will also offer a broader set of access options for publishers, ranging from completely open access to access only through the National and relevant State or Territory Libraries. The Libraries will be negotiating with commercial publishers collectively to obtain permission for access to be provided to publications at all the National, State and Territory Libraries. If successful, this will provide a much improved access to these publications than is currently possible.

40

The advent of the National edeposit network in 2019 marks another significant chapter in Australia’s legal deposit story. In collaboration with the State and Territory Libraries, NED will help the National Library meet some of those challenges I mentioned earlier. It will enable us to share resources and thus improve our collective capability to collect, describe and preserve Australia’s documentary heritage. Most importantly, it will also enable the National and State Libraries to provide an improved level of access to our joint collections for all Australians wherever they may be. Thank you.

I would like to acknowledge the assistance of my National Library colleagues in the preparation of this paper, in particular Amelia McKenzie, Libby Cass, Paul Koerbin and Aileen Weir.

41