I started up some Ms. PacMan experiments (finally) to test the learner on the environment. There was the full Ms. PacMan experience, the no dots version, and the no power dots version (with slowed ghosts). Judging by the preliminary results, the environment/learner is still inadequate.
The no dots Ms. PacMan environment is fastest (simply due to the low number of objects), and has completed a single run so far. However, the results of the run show no signs of convergence, nor of using any useful rules. Its first rule (while not first by a lot, the most useful slot anyway) is toGhost. But the problem is that there are no rules in that slot that state the ghost needs to be edible when going to it. This is most likely due to the problem of pre-goal unification. It is likely that not every ghost was edible when the goal was met, hence causing Ms. PacMan to fail.
In fact, perhaps the problem lies in early convergence. Perhaps the agent did have a nice edible rule but then the rule was lost and the slot never really recovered. But I would think that it should eventually recover. I think a problem with the current system is how the slots are updated. Only if they are simply used will they be included in updates, so the ordering of the rules doesn’t matter. Perhaps slot counts need to have positioning included as well.
The no power dots version seems to have performed better, with the performance file improving in bursts. But a worrying development recently occurred where the performance dropped back down again. Maybe a rule was removed? Maybe it was the edible ghosts rule again? I need to sort that out… Because the second most used rule according to the readable generator file is the toGhost rule.
The regular rules Ms. PacMan isn’t really doing great. Sure, it’s not doing terrible, but it’s getting fairly average scores.