OK, I have completed my initial look at the changes in AI since last I played with it and I figured I would send out my plan of attack on creating an improved AI.  I am hoping for constructive criticism/comments to the plan in case there is something I have overlooked that should be included in the planning stage.

First off, I think the thing to do is to have my new AI initially inherit most everything from the formula AI.  This will allow me to take advantage of the existing code while not creating an inconsistent syntax for doing math in WML.  (Well, my old dfool AI already has an inconsistent syntax as it predated formula AI, but it was never complete).  The new AI (I will call it fogged_ai for now) will replace a few of the functions currently defined in formula AI to make it impossible for the AI to have access to information that a player wouldn't have.  The most obvious of these functions to change would be the enemy unit list, but there are others involving pathfinding as well.  Note that if Fog of War and Shroud is turned off for the AI side, these functions should behave identically to the current functions.

The next step will be to implement the orders functionality that I had abandonded a year ago.  The order functionality would be written as some combination of formula_ai functions and C++ routines.  The idea will be to develop a library of useful formula_ai functions.

The testing of the AI will be on three fronts.  First, there will be a special test scenario for checking the correctness of the AI's non-cheating.  Second, I will be developing the AI for the players side in the HttT campaign (activated by the "droid 1" command).  Finally, I will be testing as an AI in standard single map games.  The first case is necessary to make sure that having the AI unaware of invisible units doesn't break things.  I had started doing this before, but I am not sure what state it is in now.  The HttT development should both prove that the scripting of the AI is sufficient for most campaign purposes as well as forcing me to develop a good list of formula AI library routines for campaign designers use.  It will also provide the examples of how to use it.  The final case is in some ways the most important.  The key will be that the fogged AI when working on an unscripted map must have a reasonable set of defaults.  I plan on having a special AI WML file that defines the default behaviour in these cases.  This should provide for an easy mechanism for changing/improving the AIs default behaviour on unscripted maps.  I will probably also release some simple shell scripts demonstrating how to run AI v. AI in batch mode to evaluate AI success.  There may be some additions to wesnoth command line options to make this easier.

Once items within the fogged AI have reached a certain level of maturity, if desired they could be moved down to become part of or replace elements of the original formula AI.  In addition, once the fogged AI is able to regularly beat the default AI on unscripted maps, it can be slid in to replace it.  Neither of these actions need to or should take place before thorough testing of the 3 test cases by people other than myself.  While I have confidence that my tackling the above three test cases should result in a fairly robust AI, I also recognize that it is a common human failing to be unable to recognize errors in ones own creations.  The difficult side to this, as always, will be getting volunteers to test things that have not been pushed into the default AI.  Never the less, I do think that it is important (as discussed in a different email thread) to have more than one person looking at changes that will effect how the default AI works before it gets pushed into the default AI.  So, expect a call for volunteers.

Key differences that I see in the fogged_AI vs. formula AI:
1)  The AI will explicitly not have access to information that would be invisible to a human player playing the same side.
2)  The default fogged AI will operate based upon orders.  These will likely be largely built up out of the formula AI, but should provide an easier mechanism for non-experts to manipulate the AI's behaviour in intuitive AI than the low level formula code.  This should expand the library of formula AI code.  Most of the library should work with both fogged and formula AI.  Any that only works in the fogged AI will be obviously marked as such.
3)  Unit based AI will not by default run before other orders.  There will be a command to explicitly call a units default AI, and I can imagine building a mechanism for unit based AI functions.  So, for example, one might call my_unit.scout() as part of a general scouting function to activate special scouting code for that unit (if any exists).  If none exists a default scout() function would be called.   
4)  It will be possible to run a default version of the AI on maps that have no specific AI written for it.  There will be a WML configuration file that defines this behaviour.

Key differences that I see in the fogged_AI vs. default AI.
1) Same as above.
2) Since it is based on formula AI, it will obviously be much more flexible.  It should be pretty straight forward to modify the AIs behaviour with parameters that have obvious effects. 
3) Elimination of things like the North-West bias.  When choosing between equally good moves(by whatever measure), the AI will choose randomly from amongst them.
4) Ultimately, it should be able to regularly beat the default AI on a large number of maps without customized AI scripts for those maps. 
5) The fogged AI will be able to complete at least one campaign(HttT) on easy.

I am not crazy enough to issue a timeline for this project, however I expect to have some functional form ready for the next stable release, although it will likely not have achieved all of the above goals by then.  As stated before comments and criticisms are welcome.

Darth Fool

-------------------------------------------------------------------
       "In theory, theory and practice are the same,
                but in practice they're different."
-------------------------------------------------------------------
John W. C. McNabb
-------------------------------------------------------------------