Tag: phd project

Another idea struck me while reading.

When determining subgoals in a problem (assuming the agent uses subgoals), they could be states that allow access to the agents ‘other’ functions. For example, say the main goal was to move to the table and pick up a cup. A sub goal would be to first move to the table. Once the agent is within reach of the cup, it has access to it’s ‘pickup’ action.

These extra features could also be agents within themselves. An ‘agent’ would consist of a manager, which oversees the large tasks, and many ‘workers,’ each specialised in their domains. Each worker would gather experience as it performs its task. So a robot may become quite good at walking, but not so good at talking if it was trained on problems that required silent travel.

The manager would appropriate reward among the workers and would also oversee when to activate a worker. In the above cup problem, it would first activate the ‘walker worker’ then when it senses that ‘pickup thing worker’ can be activated, it activates that, recording how well it performs so it can appropriate reward later.

Just typing my ideas down while I work my way through the book.

Within the structure of the agent, I should try and keep it as modular as possible, such that code could easily be reworked without having to rework the entire agent. Of course, this is how I should program anyway, but I should remember this anyway.

This will come in handy when testing multiple policy types and knowledge storage. By having a generic policy interface, the agent could easily swap policy types and make my experiments more streamlined. Also, mostly with regards to policy and the long-term goal, newer policy classes should have methods for reading in the data collected from older policies. So if say, I designed a policy that was good, but not great and rigged it up to a robot which learned a whole lot of information about the world, I would want to transfer the knowledge the bot learned into the newer policy so it doesn’t have to relearn a whole lot of stuff.

On a side note: I freakin’ hate Macs. But I have to use one in the ML lab.

One of my ideas for my PhD was to utilise Cyc, the common sense ontology. It already has a bunch of background information and a fully functional first-order logic structure, complete with a powerful inference engine.

So, a thought occurred to me while at Dal’s parents place, which I immediately wrote down so I could post it here for future reference.

A common issue in general purpose AI is having too many facts for the agent to consider. I’m sure the book has several good ways of addressing this issue, but I haven’t got to it yet. For now, I have an idea of utilising Cyc’s microtheory structure to contain relevant information in specific areas. For instance, if an agent is solving a problem in a building, it will have these MTs: ProblemMt, for the facts relating to the problem itself; RoomMt, for the facts relating the room it is currently inside; and BuildingMt, for the facts relating to the building.

This allows the agent to focus on problems at the appropriate scale, and still know where it stands in the grand scheme of things. Similar MTs could be grouped together for generalisation purposes as well, such as grouping all rooms together.