Case Mod 'Catalogue'

There used to be a sort of case mod index here somewhere, where you could find mods of a particular case by going to a particular manufacturers' page, then they'd all be listed, grouped by case model. Does this still exist?

Edit: durr, i wonder if google would work. herp derp.



Any chance this is going to get updated? Maybe Alex's first proper job! ha ha

 

Hmm. TBH I didn't even know this existed. Paging @Dogbert666 and @Maki role . I wonder how up to date it is and if it's something we could advertise more.

 

I don't think it's been updated since 2012 and whilst it should be an awesome library of mods, it relies on people maintaining they're chose picture accounts. There's so many logs with missing pictures. I tried to have a look through some of wayne's logs a few weeks ago and was gutted to find them decimated with 'picture not founds'

 

So instead of using email maybe they submit them on a form or something? Would be an interesting thing to talk about.

 

If the pics got stored 'on site' for prosperity, that would solve that problem, but data storage costs and people have some gigantic pics now.

I was trying to look through a few logs yesterday and you can only get one photo on your screen at a time and when they have no description or explanation, it's quite hard to follow them along.

Also rather than a form, if someone marks a log as 'case mod - complete', it could get flagged to be put into those archives, much like motm. You'd just need to check which case it was and shuffle it into the right folder, which wouldn't take that much, would it?

 

Will speak to @Dogbert666 about it when I'm back in Monday and see if we can find time and a good way of doing this without killing the storage of the servers

 

File size limit on the uploader, resize to a maximum on-server and transcode to JPEG at 90 quality (Guetzli, if you've got the CPU grunt and memory - I can cut the size of a JPEG by over a third by running it through Guetzli followed by jpgcrush...) Problem sorted.

(Speaking of, why did the main site switch to PNG? The files are huuuuuuuge, like ten times bigger than they need to be, and I'm uploading JPEGs anyway so we're gaining nothing quality-wise...)

 

Wait does the cropper convert everything to PNG? I'm actually looking into some automated image compression stuff now ready to try out when I'm back in the office Monday. I got our Google insight score up by compressing some static images we use and after seeing the impact that made I've been looking into the best way of automating the process and a way of compressing all old images

 

I dunno if it's the cropper that's doing it, but *everything* on the main site is PNG. Like, everything. (Apart from that one JPEG at the top, there.)


I mean, just look at the main story image today: it's a 615x250 PNG at 259.32KB. The source JPEG was 223KB, and that's 1067x600!

I raised it with Matt the day of the switchover, but it must have got lost in the noise. As it stands, though, you guys are absolutely burning bandwidth you really don't need to - that Star Wars PNG saved as a JPG at 95 (not even 90) quality and with no clever Guetzli/jpgcrush shenanigans is 77.1KB...

(The same file passed through Guetzli and jpgcrush, incidentally, is 40KB.)

 

i was thinking to save with 75 quality as that seems to be the standard. I'll 100% look into this on Monday. Don't suppose you know any tools i can run to compress them by giving it a directory and it goes through all the sub-directories?

 

I wouldn't go below 90, honestly. This ain't the early 90s, and 75'll look horrid if there's any amount of red in the image.

find. Here's a quick demo: I've created a folder with two sub-folders, and there are JPEG files in all the folders:

Code:

blacklaw@xerxes:/tmp/jpgtest$ du -ch
1.1M   ./folder1
392K   ./folder2
1.8M   .
1.8M   total
blacklaw@xerxes:/tmp/jpgtest$ find . -iname '*jpg' -type f -exec jpegoptim --strip-all --all-progressive --max=90 '{}' \;
./activision-call-of-duty-infinite-warfare.jpg 800x534 24bit P JFIF  [OK] 142505 --> 92337 bytes (35.20%), optimized.
./activision-cod-elite.jpg 614x384 24bit P JFIF  [OK] 29595 --> 20235 bytes (31.63%), optimized.
./folder1/bbctaster.jpg 1361x1075 24bit P JFIF  [OK] 372011 --> 214251 bytes (42.41%), optimized.
./folder1/bbc-micro-logo.jpg 700x832 24bit P JFIF  [OK] 46582 --> 34647 bytes (25.62%), optimized.
./folder1/bbc-bitesize-algoburger.jpg 800x450 24bit P JFIF  [OK] 99970 --> 61675 bytes (38.31%), optimized.
./folder1/bbc-micro-bit-2.jpg 614x345 24bit P JFIF  [OK] 69182 --> 43112 bytes (37.68%), optimized.
./folder1/bbc-micro-bit.jpg 624x500 24bit P JFIF  [OK] 54603 --> 37500 bytes (31.32%), optimized.
./folder1/bbc-model-b.jpg 1200x800 24bit P JFIF  [OK] 169718 --> 122520 bytes (27.81%), optimized.
./folder1/bbc-the-gamechangers.jpg 614x409 24bit P JFIF  [OK] 136358 --> 82509 bytes (39.49%), optimized.
./folder1/bbc-microbit-ghalfacree.jpg 800x800 24bit P JFIF  [OK] 165110 --> 95708 bytes (42.03%), optimized.
./folder2/cherry-mx-dev-kit.jpg 1500x561 24bit N JFIF  [OK] 72302 --> 48941 bytes (32.31%), optimized.
./folder2/cherry-mx-board-6-0.jpg 800x800 24bit P JFIF  [OK] 94403 --> 58055 bytes (38.50%), optimized.
./folder2/cherry-mx-board-6-0-ergonomic.jpg 1800x646 24bit P JFIF  [OK] 73478 --> 50758 bytes (30.92%), optimized.
./folder2/cherry-mx-speed.jpg 614x614 24bit P JFIF  [OK] 33593 --> 20743 bytes (38.25%), optimized.
./folder2/cherry-mx-rgb-nature-white.jpg 900x438 24bit P JFIF  [OK] 45503 --> 31002 bytes (31.87%), optimized.
./folder2/cherry-mx-rgb-red.jpg 800x800 24bit P JFIF  [OK] 63179 --> 37808 bytes (40.16%), optimized.
./activision-cod-blops-iii.jpg 800x450 24bit P JFIF  [OK] 138429 --> 84475 bytes (38.98%), optimized.
blacklaw@xerxes:/tmp/jpgtest$ du -ch
700K   ./folder1
260K   ./folder2
1.2M   .
1.2M   total

That's nice and quick, but if you wanna get *really* clever you'd have a core or six running Guetzli at a low priority and thumbing through all the folders. You'll need anything up to six gig of RAM per Guetzli process, though, and it's slow as molasses. Bloody good space savings, though:

Code:

blacklaw@xerxes:/tmp/jpgtest$ find . -iname '*jpg' -type f -exec guetzli --quality 90 '{}' '{}' \;
blacklaw@xerxes:/tmp/jpgtest$ find . -iname '*jpg' -type f -exec jpegoptim --strip-all --all-progressive '{}' \;
./activision-call-of-duty-infinite-warfare.jpg 800x534 24bit N JFIF  [OK] 67527 --> 64458 bytes (4.54%), optimized.
./activision-cod-elite.jpg 614x384 24bit N JFIF  [OK] 13559 --> 13213 bytes (2.55%), optimized.
./folder1/bbctaster.jpg 1361x1075 24bit N JFIF  [OK] 161067 --> 152190 bytes (5.51%), optimized.
./folder1/bbc-micro-logo.jpg 700x832 24bit N JFIF  [OK] 27342 --> 25781 bytes (5.71%), optimized.
./folder1/bbc-bitesize-algoburger.jpg 800x450 24bit N JFIF  [OK] 41634 --> 40606 bytes (2.47%), optimized.
./folder1/bbc-micro-bit-2.jpg 614x345 24bit N JFIF  [OK] 30146 --> 28776 bytes (4.54%), optimized.
./folder1/bbc-micro-bit.jpg 624x500 24bit N JFIF  [OK] 28805 --> 27311 bytes (5.19%), optimized.
./folder1/bbc-model-b.jpg 1200x800 24bit N JFIF  [OK] 85989 --> 78751 bytes (8.42%), optimized.
./folder1/bbc-the-gamechangers.jpg 614x409 24bit N JFIF  [OK] 59974 --> 57955 bytes (3.37%), optimized.
./folder1/bbc-microbit-ghalfacree.jpg 800x800 24bit N JFIF  [OK] 70399 --> 66801 bytes (5.11%), optimized.
./folder2/cherry-mx-dev-kit.jpg 1500x561 24bit N JFIF  [OK] 38474 --> 35948 bytes (6.57%), optimized.
./folder2/cherry-mx-board-6-0.jpg 800x800 24bit N JFIF  [OK] 44790 --> 42527 bytes (5.05%), optimized.
./folder2/cherry-mx-board-6-0-ergonomic.jpg 1800x646 24bit N JFIF  [OK] 41048 --> 37857 bytes (7.77%), optimized.
./folder2/cherry-mx-speed.jpg 614x614 24bit N JFIF  [OK] 16434 --> 15414 bytes (6.21%), optimized.
./folder2/cherry-mx-rgb-nature-white.jpg 900x438 24bit N JFIF  [OK] 21081 --> 20293 bytes (3.74%), optimized.
./folder2/cherry-mx-rgb-red.jpg 800x800 24bit N JFIF  [OK] 26901 --> 25606 bytes (4.81%), optimized.
./activision-cod-blops-iii.jpg 800x450 24bit N JFIF  [OK] 66301 --> 62731 bytes (5.38%), optimized.
blacklaw@xerxes:/tmp/jpgtest$ du -ch
492K   ./folder1
188K   ./folder2
828K   .
828K   total

So, in my little demo we went from 1.8MB of JPEGs to 1.2MB (a 33% saving) then to 0.8MB (a 56% saving from the originals) - and that's going from already-pretty-optimised JPEG to JPEG with no perceptible loss of image quality, bar perhaps on images with enough red to make the blocking obvious.

Seriously, lookit:



Half that image is from the original source JPEG, 142KB; the other half is from the Guetzli'd version, 64KB. You go ahead and tell me which side is which...

The bigger issue, though, is the use of PNGs; you'll save far, far more bandwidth moving the PNGs to JPEGs than recompressing the few existing JPEGs the site has. I'd suggest the following approach:

1. Fix the image uploader so it's saving JPEGs instead of PNGs.
2. Convert all existing PNGs to JPEGs using find -exec.
3. Tell the website to look for JPEGs where it was looking for PNGs.
4. Delete the old PNGs using find -delete (optional, if you've plenty of drive space don't bother.)
5. Optimise all JPEGs with find -exec jpegoptim. (This will include all the JPEGs from Step 2.)
6. (Optional) Have a few worker processes paging through the images and recompressing them through Guetzli (and another jpegoptim pass). For bonus points, set up an inotify monitor so every time a new JPEG image is saved it'll automatically get Guetzli'd in-place.
7. Congratulations, you've just slashed the site's bandwidth requirements by 75+ percent.

If you want to get really clever about this, drop find and use GNU Parallel instead - that'll launch as many workers as you have processor threads (or any number of your choosing), meaning on a decent 48-core webserver you'll be finished in one-48th of the time. Just make sure you nice -19 it, though, or it ain't going to be doing much web serving!

Oh, one last thing: if you do use Guetzli, compile from source. It's slow as you like, and a quick -Ofast -ffast-math -march=native -flto makes one heck of a difference.

 

That find exec is a really neat trick. I’m thinking imagemagick May be the way to go as it can convert them and optimise all in one I think. For the cropper I think it should be as easy as changing the file extension on the output part. The switch for old files may be a bit tricky as we may have to in and edit all old articles. One thing I’m not sure on is making duplicates as I have no idea if the server will handle that.I’ll pass on the ideas along with some specific solutions for us to Ares. Thanks for the help Gareth.

 

ImageMagick will convert, but not optimise. If you've the CPU grunt you're better off using Guetzli, which accepts a PNG as input and will spit out significantly smaller JPEGs than IM.

sed. The work of seconds, that.

Not sure what you mean, there.

 

Space isn’t something we have a ton of if I remember correctly so I’m not sure if we can make another set of all images. How would that be done in seconds? It would need a database operation for each article (I think) which would slow it down. Also sorry for the poorly formatted replies I’m on my phone until I get home later today. So the process would be most on an off peak time when no articles are going up compress all images and make the swap then have it so that future files are all created as jpegs by default in the cropper then only need optimisation and not format converting.

 

Good lord what have I started. I was just trying to find a P180 mod where they converted it to R-ATX.

 

This is a good thing. It’s somethig that’s needed doing and I just happened to look into it Wednesday. now Gareth has gave some input I think I know how to do it properly. It should help speed up the website by a bunch and reduce the amount of bandwidth you guys use loading it.

 

If you've got more than a couple of gigs of PNGs, I'd be amazed.

No, you'd just use sed (Stream EDitor) to replace image links that end in .PNG to end in .JPG instead. Alternatively, you could use a rewrite rule in Apache or Nginx or whatever webserver you're using, leave the articles themselves alone.

So am I!

No, you need to change the cropper/uploader first, or you're going to have PNGs that get missed.

Also, there's no need to do it off-peak; just nice -19 the processes, like I said, and they'll back off when the web server needs resources.

Seriously, if bit wants to hire me at consultancy rates I reckon I could have it done in a morning (the interactive part, anyway, then leave Guetzli or whatever running on the images.) This stuff used to be my bread and butter.