Cultural institutions, government agencies and private companies are already digitising their archives to make certain their longevity. Members of the public now have access to the historical records of organisations including the Public Record Office Victoria, the State Library of Western Australia and even Fairfax Media, publishers of The Age and the Sydney Morning Herald.

But who has time to browse through more than 380 shelf kilometres of material housed by the National Archives of Australia? Or more than 700,000 photographs stored at the National Library of Australia?

More importantly, how do you even find something amongst the fragile debris of decades gone by?

This is where digitisation comes in. It’s the ultimate research source for historians – and it’s available now.

1. Is digitisation just (photo)copying?

In its simplest form, digitisation is copying. Physical media – such as pieces of paper, photographs, maps and paintings – are converted into a digital format that can be shared, discovered and stored forever.

But beyond the copying process, digitisation offers many benefits for both cultural institutions and the general public. The most important is the preservation of Australia’s cultural and historical records.

Archives are full of fragile materials: glass plate negatives, photographs, old documents. Overusing or mishandling these items will inflict further damage and, in some cases, disintegration.

The National Archives added 1.75 shelf kilometres of records to its collection in 2012–2013; the National Library acquired 132,634 items.

For this reason, conservators say many archival items are so fragile that they cannot be lent to the public. Now they can – and digitisation is the key.

“Libraries historically lend items to researchers, the Australian public, whoever,” says Nicki Mackay-Sim, curator of pictures and manuscripts at the National Library of Australia. “But we really don’t want to hand over a fragile item like a glass plate negative every time someone wants to see it.”

Digitisation, she adds, allows archivists to make an “electronic copy” of an item that can be lent to the public without risk to the original.

The only problem is time. Digitisation is a slow process and archival collections grow by the day.

2. But who cares about dusty relics?

Archives can be boring. They can include rooms full of old newspapers, photographs and dreary content.

But Zoë D’Arcy, director of the digital and online access division at the National Archives of Australia, says cultural institutions have a duty to make their archival collections meaningful to everyone.

“People tend to think of archives as buildings full of things that are locked and hidden away. But in reality, we have collections full of items that the public will find interesting, such as records about their family or home town.

“Digitising this content means we can get it up and into forms where people can find it. So, in a large way, digitisation is about keeping our collection alive and relevant to the Australian public.”

This approach has already been well received. Digitised copies of Australian newspapers are consistently rated as the most popular content available on the National Library’s Trove database. This year, around 1.1 million pages of newspapers were digitised, bringing the total number of pages in the digital newspaper collection to more than 10 million.

Still, digitisation brings more than history into the modern world. Improved search features and simultaneous 24-hour access for multiple users are just some of the benefits of combining digitisation with the power of the Internet.

And users are taking advantage of this convenience, according to Ms D’Arcy. In 2012–2013, 95 per cent of users accessed the National Archives collection via the Internet. The rest visited one of its reading rooms.

A similar trend is unfolding at the National Library.

“More and more visual historians are coming to expect a digital copy of the pictures they are researching,” says Ms Mackay-Sim. “This is the way they want to receive them – online and with as much information about the item as possible.”

This is good news for Australia’s cultural institutions, Ms D’Arcy suggests, because it means the contents of the nation’s archives are resonating with the public.

“Everything we digitise and make available online ultimately becomes well known to the Australian public. This makes it a very visible part of our collection.”

3. It’s magic! From physical to digital …

Digitising archives is not as simple as scanning a document or photographing a book. Specialist equipment that can handle anything from small photographic negatives to large maps is required long before digitisation can occur.

Then there are staffing requirements. Conservators, photographers and information retrieval experts all come at a significant cost.

Many physical formats – particularly early photographic negatives – are highly unstable and are slowly deteriorating, which adds urgency to making digital copies, says Zoë D’Arcy of the National Archives.

The National Library appointed seven of its staff in December 2012 to sort, clean and digitise more than 18,000 glass plate negatives donated by Fairfax Media earlier that year. But progress is slow. Working together, the team can only process around 160 plates every day.

Similar practices are undertaken at the National Archives, Ms D’Arcy says.

“We’ve got a team of digitisers who work with a whole series of flatbed scanners. They take our documents apart and scan each individual page. Then they put the documents back together and load the scanned copy onto our online catalogue.”

Visual collections, such as photographs and glass plate negatives, take longer to digitise. Images must be reproduced at the best quality, with all possible colour information retained. They are then imported into software such as Photoshop for further editing.

Archivists go to these lengths because they want to preserve our national history. But time is against them. Many physical formats – particularly early photographic negatives – are highly unstable and are slowly deteriorating, which adds urgency to making digital copies, says Ms D’Arcy. “In essence, this digital copy becomes the master version, the best version of that item you’ll ever get.”

The library’s Enemark Collection – 500 nitrate negatives depicting rural townships from NSW and the ACT – is one such example. Digitised by the Library in 2013, the negatives are so fragile that they have rarely been removed from storage since being acquired in 1965. Now, digitisation has extended the life of these precious images.

Fairfax Media is also looking for archival immortality. Chris Berry, the company’s director of information services, says Fairfax is shipping more than 25 million prints and negatives to the United States to be catalogued, digitised and stored in climate-controlled facilities. The company is taking this step because the images are irreplaceable visual records of the past 100 years and are of “enormous public interest”.

But how do people find this material on the Internet?

“That’s the real challenge that everyone’s facing,” Ms D’Arcy says. “At the moment, our users search the [National Archives] collection by item title, government department and date. We call this metadata.”

Metadata is simply information describing something else, but it is vital when it comes to making digitised archives discoverable by search engines.

And, according to the National Library’s Nicki Mackay-Sim, there are three “descriptive pieces of information” that should always be recorded: “who the author or artist was; when the item was produced; and, what it is documenting”.

Where an image was taken will also become increasingly important in the future, she predicts, because location data will provide valuable information for historical records.

“Fortunately, many digital cameras and smartphones now save location information with each image.”

While digital technology provides time, date and location information, it cannot provide context. Nothing captures the area beyond the camera lens, or even the background to a government decision. But digitisation can help.

Digitised newspapers contain names, specific dates and historical information that can help to identity a particular event. Likewise, digitised books on renowned criminals and their crimes may give context to old court and police photographs.

This is information that metadata cannot provide, and cultural institutions are recognising this.

“At the moment, to make our records discoverable, we provide a lot of information about those records,” says Ms D’Arcy. “What we want to do [now] is make the information in those records discoverable.”

To achieve this, the National Archives is looking at using optical character recognition software to “read” the words printed on their paper documents. But this will be time-consuming. All paper documents – including those already catalogued – must be scanned via the OCR software and placed online, before the information they contain may be found by search engines.

4. Digitise and destroy?

Archives are designed to store physical material for as long as possible. In fact, the National Library aims to preserve its collection for at least 300 years. But with the progressive digitisation of Australia’s cultural and historical archives, can institutions justify keeping the original material?

“The National Archives house the records of the Commonwealth Government,” says Ms D’Arcy. “The whole point of storing these records is that the government departments who created them have said: ‘These files are so valuable that they should be kept forever’.”

The library’s Ms Mackay-Sim expresses a similar view. “The whole point of the National Library is to collect documentary heritage for the future. So it is our policy that we never destroy anything.”

But “forever” is a long time. The National Library digitised only 3.3 per cent of its entire collection in 2012–2013, while the National Archives managed 4 per cent. With forward estimates indicating similar progress over the next few years, full digitisation of both collections will take at least 25 years.

“As we’ve explored the collection, we’ve realised that some of the images have a higher value to other people than we originally thought.” — Chris Berry, on Fairfax Media’s picture archives

Technological change and the ongoing deterioration of original materials may also force cultural institutions to alter their preservation policies sooner rather than later.

“It’s a question we constantly ask ourselves,” says Ms D’Arcy. “At the moment, when we digitise a file, we keep the physical item. But that could change in the future.”

For the National Library, there is no debating this issue. “We would only dispose of an item if it was in such poor condition that you could no longer work out what it represented,” Ms Mackay-Sim says. “But hopefully we would have digitised that item prior to it decomposing.”

And what of the economics of preserving archives? The National Archives added 1.75 shelf kilometres of records to its collection in 2012–2013, while the National Library acquired 132,634 items.

“There are budget limitations for storing our large collections,” Ms Mackay-Sim acknowledges. “And we are collecting and collecting and collecting.

“Down the track, if we don’t have enough funding for further storage facilities, do we continue by digitising and then disposing of the original material? Personally, I would hate to think we got to that point.”

For Fairfax Media, the high cost of storing its collection is a deciding factor. Mr Berry says the State Library of NSW offered to preserve part of the image collection in their new cold storage facilities.

“You have to put photographs in cold storage if you want to preserve them, but there are massive costs involved in keeping the material properly.

“Our collection is so big that it would have completely filled the [NSW] State Library’s new facilities.”

Fairfax was advised to cull part of its collection before starting the digitisation process, but according to Mr Berry, the company wanted to keep its collection intact.

“Our photos show things that have always been of interest to people. We don’t yet know to whom some of the photos will be of interest … but as we’ve explored the collection, we’ve realised that some of the images have a higher value to other people than we originally thought.

“So we feel it is a good thing that we haven’t gone down that route of culling images, because the whole collection is going to be digitised and made available online.”

“That’s my biggest worry. An amazing new format will pop up in the future and we’ll have already digitised so much of our collection [in an older format]. And so we’ll have to shift everything to this new format.” — Nicki Mackay-Sim, of the National Library 

Fairfax will keep the photographic negatives from its collection, but plans to sell the original prints after the digitisation project is complete. Other institutions are not following Fairfax’s lead, but will continue collecting physical material.

Fortunately, the number of physical items being acquired by cultural institutions is gradually decreasing. According to the National Archives, 91 per cent of Australian federal government agencies are either working in a digital-only environment or are planning to do so within one year. But of these agencies, 28 per cent still manage and store more than 20 per cent of their digital files in physical form.

To counteract this, the National Archives is promoting the benefits of a paperless workplace. It also announced earlier this year that it will not be accepting physical copies of born-digital material after 2015.

5. The digital headache.

The last 10 years have seen digital cameras and smart phones rapidly replace film-based cameras. Images are now captured in a digital format, ready for further use, with no darkroom development necessary. 

But the digital format is not without problems. Image capture technologies have improved to such an extent that file formats once thought to provide “best quality” reproductions have been quickly superseded.

Such is the way of our digital world – and cultural institutions are not immune.

“If you think of our image collection, it started with paintings and sketches,” says the National Library’s Ms Mackay-Sim. “Then the format shifted to nitrate negatives, glass plate negatives and acetate negatives. Now, of course, we’ve got digital formats, which are constantly changing.

“That’s my biggest worry. An amazing new format will pop up in the future and we’ll have already digitised so much of our collection [in an older format]. And so we’ll have to shift everything to this new format.”

Doing so will be neither quick nor easy. The National Archives’ digital collection already contains around 11 million items, while the National Library’s collection is more than 2.62 million gigabytes in size.

There are also archival problems that need to be resolved when storing digital files.

“Not only do we digitise analogue material, we also take in born-digital material,” Ms Mackay-Sim continues. “Obviously, more and more photographs that we acquire are born-digital. We need somewhere to put them, and we need to have a preservation system in place.

“So digital is a bit of a headache, too, because it means people are taking a lot more photographs, but are not necessarily culling them. And that’s a major problem.”