Other Scanning Old Magazines - a Worklog

IT'S SO GOOD.



Lift the squisher, lift the blade, pop it in, lower the squisher, lower the blade, lift the blade, lift the squisher, you're done. Maybe a minute to line everything up?

It does still cut at an angle, like when I'm doing it by hand:



...but we're talking a few mill. I can live with that.

Still have to cut the fold-outs by hand, mind, so that's an annoyance. Definitely going to speed things up, though!

 

Yeah, this was all a very sound investment. It now takes me about 30 minutes to go from a bound mag to:



Look at how straight that is! Hell, the two-page spread is almost perfectly lined up and I've done nothing!

Yeah, the page isn't white enough and the black isn't black enough and the colours are faded, but... that's all fixable in post. Programmatically, even.

Sure, the scanner and guillotine weren't cheap, but for the time they'll save me? Bargain.

EDIT:
1:1 crop of the unprocessed file straight off the scanner:



Page is dark, bleedthrough text is distracting, colours are muted. Quick ImageMagic one-liner...


Bleedthrough is reduced, though still present, colours are brighter, blacks are blacker. Could I do better? Yeah, probably. Would anybody care? Probably not, this is perfectly readable.

 

Opinions, please. Here's a (scaled) half-page from the latest scan:



And here it is with what I've been doing, which is 20% saturation boost and one contrast-enhancement flag¹:



But here it is with *two* contrast-enhancement flags:



One flag or two, what do we think?

EDIT:
Or the secret fourth option, two contrast bumps and a 3% brightness increase?



1: ImageMagick works with command-line flags/switches/options/whatever, and you can tell it to use its contrast-enhancement filter by specifying "-contrast" where "-" means an option and "contrast" is the option. Makes sense, right? Well, you can also tell it to reduce the contrast... by specifying "+contrast". Yeah, minus means increase and plus means decrease. What a topsy-turvy world!

 

Okay, okay, hear me out... what if... two contrasts, 105% brightness, *and* 140% saturation?

Yes, you lose most of the page texture - but that'll really help come compression time. The colours are bright, though the blacks are a little crushed for my liking.

Some (scaled!) full-page shots:





(Peep Nimoy's Vantablack jumper to see what I mean about the crushed blacks)



(The "Real World Computing" section has, for some reason, yellow pages, which is retained even after processing - note the actual white background on the screenshot. Also, no, I didn't notice the folded corner on these pages before I dumped them. Whoops!)



(This one's an interesting test, 'cos it's the back cover - nice thick, shiny card, so there's no bleedthrough (which is caused 'cos it's a simultaneous-duplex scanner, so there are lights shining on both sides of the paper at the same time) and the background was *already* white. I was worried the covers (and card inserts) would need special handling, but it looks to have survived intact!)

EDIT:

Two contrasts, 104% brightness, 140% saturation, *and* gamma 1.1?



Brings out the shadows a bit more, I think?

 

Did a whole-mag process, as a test.

With "convert -define trim:edges=east,west -fuzz 15% -trim -deskew 40% -contrast -contrast -gamma 1.1 -modulate 104,140,100 +repage" it took 7m6s to process 594 pages. However, it also reminded me why I stopped using deskew: it only took to page 17 to decide a mostly-straight page should in fact be rotated to about 45%.

*Without* deskew, 3m40s. Now, as much as I would like to use this as an excuse to get a processor with double the cores (16C32T would literally double the performance, as I'm spawning individual workers - and that's before you even get into single-threaded enhancements since they made my trusty old 2700X), even if I halved that I'd only be saving a little over a minute each time...

And the result? From the scanner:



Processed:



1:1 crop:



That'll do, I reckon!

EDIT:
What I'll probably do is use this workflow for making the PDFs, *but* keep the original unprocessed scans to-hand - compressed as 90% "visually lossless" JPEGXL. That way if I change my mind, I can reprocess them whenever I want.

EDIT EDIT:
6m19s to compress all Issue 65's pages to JPEGXL, 90% quality effort 7. Original size 9.1GB, compressed size 1.1GB(!).

And how "visually lossless" is "visually lossless"?



One vertical half of that image (saved as a PNG, obviously) is the original PNG from the scanner; the other half is JPEGXL "visually lossless" 90% quality. Go ahead, pixel-peek, you tell me which is which!

 

Issue 60 scanned. Tried to fan it out a bit to avoid it cutting too much off the back... and ended up cutting too much off the front instead, and then cut even more off the back 'cos it hadn't cut enough and the pages were still stitched and glued.



PC Pro: OMPUTING IN THE REAL WORLD. Whoops.

 

I know I should be concentrating on scanning, but I had another fiddle with the post-processing stuff - 'cos this had a particularly challenging page: a Kingpin review, with VERY VERY DARK SCREENSHOTS.

How dark? They were pretty much impossible see using what I was doing before. So, a new approach:
convert -define trim:edges=east,west -fuzz 15% -trim -contrast -gamma 1.15 -modulate 106,160,100 -auto-level +repage

"-auto-level" is doing the hard work there. It's a risk, 'cos I have less control over what it's doing... but so long as I keep an eye out for outliers (and don't delete the scans!) I should be good.

Quick tests (scaled):







It's not perfect, but I think I'm letting perfect be the enemy of good again... It's so tempting to start fiddling with the scanner brightness/contrast, but I should just stick with what I'm doing now and crack on with actually scanning paper. Too tempting to sit there bikeshedding for weeks!

EDIT:
Actually, I'm a bit aggressive on the saturation there. Poor Jon looks orange.

convert -define trim:edges=east,west -fuzz 15% -trim -contrast -gamma 1.15 -modulate 108,115,100 -auto-level +repage?

 

Scanned Issue 30. Got complacent: it was all going so well, and I'd trimmed maybe a little more off the spine than I needed to, so I started grabbing stacks of pages between inserts rather than separating them by hand first.

It was great! I was scanning stuff so quickly!

Yeah, until I hit a couple of pages that were stuck together and the scanner went YOMF. Nothing destroyed, but there's a few pages in this scan that are a little crumplier than I would have liked...

Lesson learned. Doesn't matter if it takes ages, separate every page by hand before scanning!

EDIT:
Here's one victim:



Less obvious once it's been tweaked, tho':

 

The little manual (double-sided A4 sheet, but at least it's in actual English) says yes: there's a tool for putting under the blade so you don't lose a finger, then you undo a surprisingly high number of screws and it drops out. It suggests finding someone who sharpens knives or lawnmower blades, but if you do screw it up they sell replacements.

 

Was reading through the ImageMagick docs (as you do), and discovered that I'm... not really supposed to use -auto-level. "The operator is not typically used for real-life images, image scans, or JPEG format images, as a single 'out-rider' pixel can set a bad min/max values for the -level operation. On the other hand it is the right operator to use for color stretching gradient images being used to generate Color lookup tables, distortion maps, or other 'mathematically' defined images."

So, fine, let's do it properly. Here's a scan:



Here's the old method:



And here's the new one:



Looks pretty similar, right? Let's pixel-peek:



1:1 scale, old approach on top and new approach underneath. The old method has blacker blacks and higher contrast, but the new method is closer to reality. Maybe.

The new method? convert -define trim:edges=east,west -fuzz 15% -trim -level 10%,85% +repage. In other words, I'm just blacking out the bottom 10% and whiting out the upper 15% (high, I know, but it's to get the page white) and stretching what remains, no gamma correction. There's no automatic anything in there (other than trimming), so it's less likely to go wrong... and it's faster. 3s to process five pages the old way, 2.3s the new way. So, from ~7 minutes to process a 700-page scan to 5m30s-ish. Nothing to be sniffed at!

Few other page examples - opinions welcome!

 

Actually, I'm throwing too much away there. Most I can get away with and still retain detail is convert -define trim:edges=east,west -fuzz 15% -trim -level 8%,88% +repage:



The pale stripe on the right is the original scan, and you can see the gridlines in the spreadsheet. At 85% white point they're gone; at 88%, there's just the barest hint.

I can go higher on the white point, of course, but then I'm getting discoloured page backgrounds...



That's 8%,95%. Hmm. So the question is: really nice-looking text pages, but a loss of detail in screenshots with light colours in 'em, or retain the detail but the pages look mucky.

Or just use 88%, which loses some detail and makes the pages look a bit mucky, as a compromise.