Website Archival and Preservation

Photo by NOAA on Unsplash

Introduction

Michael Harley has been at it again in a follow-up post titled: What happens to our blogs when we die?. And, as usual his thoughts and ideas are really quite excellent. I highly recommend reading the original article.

But, as seems to be becoming a habit with me, I have some additional thoughts that might be of interest. In fact, in this case, I might be able to provide an interesting idea or two that Michael hadn’t considered.

Markdown & HTML

First, Michael made the point that he had been using some 11ty macros for processing his website. And now, because he is thinking of longer term storage of his posts, he is undoing the macros and converting them back into pure Markdown.

Something to think about: Markdown allows inline HTML, at least in part. So things like image handling could be converted into HTML. Why is this useful? And doesn’t it reduce the readability of the original files?

Yes, it does a little… But there is a long term gain to be had here. When using this approach you can use CSS classes with your image. This will allow you to have a consistent set of properties for images on your website. But more importantly: when producing an ePub version of the posts, you could substitute a new stylesheet that adjusts those properties for e-book and print mediums.

So, something as simple as <img class="img-class" src="foobar.png"> is all that is needed, and won’t need to have attributes adjusted manually.

Generating E-Books

The idea of making an e-book archive of my websites was something I hadn’t considered before. But it’s really a good idea, and something I wish I had thought of before.

One of the things I had considered recently was at least having another backup of my articles stored locally. I decided to add the RSS feeds for my sites to Thunderbird. That way I would have local copies of everything from my site, and a quick way to verify that posts published when they were scheduled.

But, generating an ePub is quite literally just as easy. All I did was use Calibre. I created a custom News source recipe for all my sites, told it to download 31 days of articles on the 28th of each month. (Have to deal with that silly February issue.) The news system uses the RSS feeds from sites and translates the XML output into XHTML for ePub books.

Now, I didn’t dig into this part, but I think the point I was making above could be handled with Calibre… That is, you could implement the custom stylesheet as part of the XML conversion process. That would mean less time spent on creating a home-grown system for converting blogposts into ePub format.

But, wait, there are more possibilities here… If your site provides an RSS feed for comments, you can make an e-book of them. You would have to merge them with the original post later, but you can at least archive them this way. Also, if you are on Mastodon, or a similar site, you can get RSS feeds for your posts on there too. I don’t think you get the responses, but at least you can archive your posts.

There is one consideration to all of this: you might want to remove Calibre’s footer that has the original link for the article / post, and the retrieval date. Although personally, I think having both the published date and the retrieval date in the e-book is a good idea. Also, I don’t know how this would look for Mastodon posts.

Another Archive

On the subject of digital archives. Something else to consider: The Internet Archive’s Wayback Machine. While this is something that might suffer a fate at some point down the road, I have some faith that they will likely survive most of the technology and platforms we are using today.

One of the things that I like about micro.blog is that they have built-in actions for the Wayback Machine. IE, when a post is published on my site, the Wayback Machine gets a copy automatically. No waiting for a bot to scan my site.

Another cool thing about the Wayback Machine is that it will capture the changes to our sites and content over time. Something that we can’t easily do with building archival e-books or printed books. In fact, a yearly book can suffer from the issue of changes / edits made to articles after their initial publication.

Given the Internet Archive has been around since 1996, and they have managed to continually expand their storage facilities I think it’s safe to say they will be around for another 30, 60 or maybe even 100 years. Long term archival of our digital lives can be accomplished with the service of Internet Archive.

A Real Book

This part really caught my attention:

Then the part I’m actually looking forward to: a printed book, once a year. I did the math and it’s a real book, not a pamphlet.

Given that I am a person who has started doing a lot of my writing using fountain pens (see: Why Fountain Pens?) I sat up and took notice when I read this part. Michael didn’t go into the cost(s) of printing a book. I wonder if Michael has thought this through completely? I’m not trying to sound condescending here. My interest in the publishing industry goes back for decades. One of the companies I worked for in the past was owned by one of the largest publishers in the world.

The reason I mention this there are some special considerations for printing a book that is designed to last for a long time. When considering those options, the costs start adding up quickly. I’ll cover a few of them:

Paper Quality: you want to choose a paper that is reasonably heavy, at least 60gsm, but 80gsm is preferable. It should also be an acid-free paper. You want to make certain that the paper will not be affected by environmental issues, such as UV light.

Ink Quality: You want archival quality, pigmented ink. Not the typical dye ink. Dyed inks can tend to fade over time. Things like UV light, and other environmental issues can affect the ink on a book.

Binding: Assuming you want the most durable, longest lasting book, you would want a hardcover book. For a hardcover book you want a saddle-stitched binding. This is really difficult to find from an on-demand print shop. Lulu, for example, offers case wrapping for hard cover or perfect binding for paperback books.

Print Type: Another concern about printing. Since you have images in your book, you probably want to go with color printing. While you can go with a standard color print, it’s not going to be the best print quality (and, I’m not even certain you can do this with archival inks). In which case you probably want to go with full quality color printing.

Layout: doing the typesetting and layout for a book isn’t the same as we tend to do on a daily basis. There is a very specific formatting for books, templates that need to be followed. And there is the process of proofing your book. If you add in hard copy proofs, that’s an additional expense. And don’t forget the other matter for a book: the table of contents, index, and any appendices. Oh, and handling footnotes or end-notes. Also, images: doing layouts with pictures can be more complicated and cause your page count to soar. Not to mention that you will want to have photo-ready quality images available for your book.

And there’s more to consider. For example, you probably want to get multiple copies, and potentially have someplace to store an extra copy or two of each book (a bank safe deposit box, and a fireproof safe are a good start). Basically, think of it as disaster recovery for your printed works. :)

This starts getting very costly. The lowest price I could get for a book (from Lulu) was around $20. That doesn’t have the archival quality ink, paper, binding, or print quality. But, personally, I think most people would be happy with it if I was selling a book as a mass-market type of thing. But this isn’t mass-market, this is personal.

I checked a few other sites (I’m not naming them because this was an ad-hoc search, and I have no experience with any of these printers), and found that a fully spec’ed out book would be around $900, per book. Five copies would be around $4,500 (I’m assuming the three for personal use / storage, and two to be stored with other family members or at other locations for disaster recovery).

And none of this takes into account the layout, proofing, etc. process. This is all on you. If you want to get a professional involved in the layout process it’s going to cost quite a bit more. I’m sure there is some middle-ground to be found here, but it’s going to take a lot of investigation, and some give-and-take to come up with a good option.

Conclusion

Hopefully this is useful to Michael, and anyone reading Michael’s articles. Personally, archiving our posts for posterity is something I hadn’t considered before. But Michael’s articles have put me in the mindset of how do I do this with the least amount of friction possible. And, I see the possibility of accomplishing this task quite easily.

One of the things I like about the idea of using Calibre’s News Fetcher to generate e-books is that it will work with any platform: WordPress, Hugo, 11ty, Bear Blog, Micro.pub, write.as, and so on. As long as your site provides an RSS feed, you can make an e-book. And the whole process can be easily automated so you don’t have to think about it all the time.

As I mentioned in my previous article (Embeds Did Serious Harm to one of My Websites): I am now allergic to using embeds - especially 3rd party embeds as they can lead to serious issues for a website. Given I have that policy now, the only embeds I would need to consider would be the ones on CerebralMix Archive, but I have a lot more work to do on that site.

Personally, I don’t think there is much of an issue with links being in the printed version of the e-book. It’s a fact that this is a physical archive of a digital medium. There are expectations of limitations of the medium.

The trickiest part of this whole process would be the printing process. The options that you have to consider are the trickiest, and can have serious impact on the books quality. And there are options to think about in terms of storage of the books (aka, the disaster recovery options).

And, I think I can leave this topic here. Michael’s done an excellent job in inspiring me to look at some possibilities for doing something similar to what he is doing, and possibly going further. If I do come up with something else, I will write about it here.