A problem I noticed while running Tetris, is that when the agent is exploring, it is basically putting pieces in random positions (well, that’s exactly what it’s doing). This is regardless of the height culling that I tried to implement. So, it was putting pieces on top of spires because next to those spires was a low point that made the spire a viable state to put a piece in. So a new strategy was required.
I have theorised that a reworking of the splitting system would work. Here’s a diagram:
IMAGE LOST
So, the new split system will split states not only into states of size 4 (or less) but also split them along large drops/climbs of size 3 or more. This is because putting a piece halfway on the drop would be stupid playing. By providing the agent with smarter (and fewer) choices, it will make better decisions.
Instead of having variable sized states, all states could remain of size 4, but null integers (some arbitrary number) could represent nogo zones. So a state {-2} will become {-2, null, null} where null could be an out-of-bounds number or just null if using Integer objects.
Another point of interest is that due to states being cut off by height, 1-wide chasms will disappear (eg: {0, -5, 5, 0, 1…}. So the -5, 5 will be cut and the agent will forget it exists. A special state/s (designed for vertical I-pieces, but not limited to) will need to be created for the three situations:
– {5, 1, 0…} A large jump at the start: {-3, null, null}.
– {…-5, 5, 0…} A gap somewhere in the middle: {null, -3, null}.
– {…0, 1, -5} A gap at the end: {null, null, -3}.
The nulls signify where other legal units can go, but shouldn’t unless forced to. The 3’s are technically out-of-bounds (a substate can only go down to -2), so this makes these states special from other states.
An example state being split up in this method:
So the contour field array for this would be {5, -1, 0, 1, -2, -5, 4, 2, -1, 3, -6}.
It would split into this: {-3, null, null}, {-1, 0, 1}, {0,1, -2}, {null, -3, null}, {2, -1, null}, {-1, 3, null}, {null, null, -3}.
Note that the 5th and 6th states arestate is of size 3. This is due to the states being separated by a gap of 4. And the 6th state is now reduced to a size 1 spire, which is ignored.
Note that this new method produces less states, even before culling. With culling, the number of states for an agent to choose would decrease once more. The total number of unique states that exist would be 158 (5^3 + 5^2 + 5 + 3). I think.