Just typing my ideas down while I work my way through the book.
Within the structure of the agent, I should try and keep it as modular as possible, such that code could easily be reworked without having to rework the entire agent. Of course, this is how I should program anyway, but I should remember this anyway.
This will come in handy when testing multiple policy types and knowledge storage. By having a generic policy interface, the agent could easily swap policy types and make my experiments more streamlined. Also, mostly with regards to policy and the long-term goal, newer policy classes should have methods for reading in the data collected from older policies. So if say, I designed a policy that was good, but not great and rigged it up to a robot which learned a whole lot of information about the world, I would want to transfer the knowledge the bot learned into the newer policy so it doesn’t have to relearn a whole lot of stuff.
On a side note: I freakin’ hate Macs. But I have to use one in the ML lab.