As the deadline is drawing closer, I need to start forming my thesis proposal. This is the thing that essentially says what I’ll be doing for the next 3-4 years. Which is a big deal. And I have to make sure that what I do will translate into that amount of work. I think it will, but I’ll have to talk it over.
Anyway, here’re my main points:
- Relational environment. There is no leniency on this. I will be working in a relational environment. Any other environment essentially makes the rest of the goals useless.
- Random rules. The rules for an environment will be (on the first time anyway) randomly generated. However, in accordance with the Cyc storage goal, pre-known rules could be loaded in.
- Goal is defined. In the initial state specification, the agent knows the goal. This allows the agent to concentrate the randomly generated rules solely on constants present in the goal, as opposed to all constants. Furthermore, this can allow the agent to systematically approach the goal through heuristical action analysis.
- Evolving rules (CE, Genetic, mix of both?). This part is somewhat negotiable, but it does make a good method of evolving the random rules. The mixture of the two is in the regeneration of new rules within the cross entropy distribution. Because the rules are of a relation nature, the typical condition and action generators become less effective. Hence, I can instead use Genetic operations on the elite rules. Perhaps even cross-breeding. Or even cross-breeding between distributions!
- Cyc storage for knowledge transfer. Theoretically, Cyc could be used to infer the rules and such, which is also a plus for it. Unfortunately, I cannot install it on this machine, but perhaps back home it may work. The big question is speed. The second benefit of Cyc is that I can store the state specs and resulting policies, as well as other information. This can be used in other domains for forms of transfer learning, and also as a method of agent memory. However, I’m still not 100% sold on this yet.
I suppose a proposal is just that. It isn’t set in stone, and as many people have said, they have deviated from their original proposal.