Broken Bones Automation [part 2]

Last time I've talked about the benefits of employing advanced automation in the filter creation "pipeline.

Today I want to go a bit more into the details about that.

One of the large challenges in filter updating is understanding how to tier items. Tiering items does more than just inform the user and give them a certain benefit in terms of speed and comfort. They also affect the player's perception and thus enjoyment of the game. Now I'm not saying that I can make people drool by playing the "exalted orb sound", but on-point highlighting and tiering adds these precious moments of enjoyment and excitement, between a certain sound or minimap icon to the moment you actually pick the item up. Maybe it'll be a mirror this time?

The problem I want to address in this post is: how do we let an algorithm tier items?

First, we're using data provided by poe.ninja by tapping it's API. I always post-evaluate the data (this is *very* important for shaper/elder items), but I've found it to be very reliable. Alternatively I have my own Trade-API Query tool, but I don't want to reinvent the bicycle for no reason.

By the way all actions - parsing the filter, grabbing economy data, adjusting tier lists, everything - is done via a special tool I've written called FilterPolishZ. I've been working on that tool over the last months and it's a big step up from the last version (FilterPolish) that I've been using for over 2 years.

I use that tool to grab data for uniques, unique maps, divination cards, fossils, resonators, shaper/elder/normal basetypes and fragments from the poe.ninja API. We store these files (in JSON format) for now. We also store them for each league and day in a separated location. More on that later.

I then parse the filter seed filter. This is done using a special tool I've (recently written) for that purpose - FilterPolishZ. I've completely rewritten it compared to the last iteration over the last months and will publish the - fairly fancy and good looking - tool in a couple of weeks (with some extra access for high tier patreon supporters).

Parsing involves breaking the filter into an internal representation of FilterEntries, FilterLines, FilterTokens and several utility constructs. I won't bore you with technical details here, unless you guys are curious.

Using several "markers", we create an internal representation of the current tierlists.

Now for the interesting part.

FilterPolishZ provides a special screen - a way to create "ItemMetaInformationSheets", special tables that includes several "tags" about every unique. Things such as "Uncommon Unique", "High Stat-Variety", "Non-Drop", "Meta-Bias", "Melee-Aspect", "Prophecy Item", "Prophecy Outcome Item". There's a lot of different tags each with a special purpose. These tags are attached by me for every single base. Quite a lot of work, but they're basically the representation of my understanding of the game, combined with meta-information.

This is the key to automated tiering.

Without it, unique Ruby Flasks will end up in a mid-tier category, because there's no way knowing, that only Dying Sun is a droppable flask (and that the other ruby flask never drops). Without it, there's no way establishing that gold rings have a huge price span depending on stats and there's no way to give certain divination card a higher tier due to the fact that they have just 1 card.

In short there's a crazy amount of exceptions out there and the "ItemMetaInformationSheet" is describing this.

The next step is basically running the data above through an automated pipeline that evaluates certain rule combinations, with economy data and decides if an item needs to change it's tier.

This is a fairly complicated process, so I'll describe it in the next post an go into some interesting details.