Thell.dev

The first stage of the process will be to convert it into a grid. On each of these grids, an MD5 (or perhaps a SHA256) is generated. It'll then search for images where there's matching MD5s. In theory, there should never be any unless the image is a straight duplicate. This is the low hanging fruit part - it can stop the process if it finds a match here.

But what if we modify a square?

We drew a line in. That square now gets a different MD5. But, since collisions CAN happen - for example, if someone does a plain white background, or transparency, and whatever grid size we use ends up capturing fully transparent/white squares - that's enough to throw it off. So instead, it'll go off how many squares are the same. This is a good time to introduce confidence scoring.

Being Confident

If all but one square returns an MD5 match to an existing image, we can say pretty confidently it's a repost. Let's say 98% confident. The system will reclassify that post as stolen, and change it to a reblog.

But what about more complicated scenarios? The grid system isn't the only method we'll be using, and they're less clear cut. We need to assign a cutoff or two - how confident should the system be to act autonomously? If it's less confident, it should ask a mod to review it manually. At the same time, at some point, the system should be confident enough that it's not a repost that it doesn't bother us. We'll settle on 90% confidence for autonomous action and 20% confidence for not bothering us until we've refined the system. Now, let's go over the other methods we'll be using.

Size Matters

Resizing something is a common way to get around filters of this kind, so we need to keep records of different sizes of the art too. We'll also need to check for images being flipped - the same way that people uploading TV episodes on YouTube do to get around the copyright filters. Ading borders is something we need to check as well.

Why so Blue?

Another common way of getting around filters is changing the colours of something - either the colour itself, or the saturation of it. Checking this will be a pain in the ass, but is essential to a comprehensive theft prevention system.

I See You

Finally, the part that'll take the longest - visual comparisons. If all the above fail, there's a chance the thief is skilled. So, we pass it to an AI to look at. This, once we've gotten it working right, is all the above on steroids. It'll be able to look at it and say whether it's seen it before, as well as a confidence score. If it gets to this stage, unless it's 100% certain, a mod will likely be required for intervention - after all, we've seen what visual recognition is like with Tumblr's porn filter.

This step has some nice bonuses, however. It'll be able to see if it's been blurred, is a cropped version blown up to full size, or whether it's seen something similar before but watermarks have been removed or text altered. It might even be able to tell whether it's a trace of someone else's work or what's been used as a reference - however, we're consciously choosing not to intervene on that stuff, and it's a waste of resources to try.

Here's what it makes of our test piece (rendered in paint - right now, we get a text readout of pixel areas that I've had to translate into something easy to read). As you can see, it's far from perfect, which is why manual mod intervention will be mandatory for this stage.

Closing Thoughts

The above is about half the system we're implementing, excluding the experiments that are more curiosity rather than something serious to include. We're not listing them all here because the post will drag on a bit, and because we want there to be some element of secrecy so you can't find holes to defeat it.

It's a pretty complex system and our aim is that any given art piece should take no longer than 20 minutes to verify. In an ideal world, it'd be 5 minutes - but we're unlikely to have the budget for that any time soon, and as more art is uploaded, the longer it'll inherently take.

Suffice it to say - art theft, while a unique system, is flawed. We want to fix it, and we want to share with you what we're doing to improve on it.

Thanks for reading!