Sunday, February 25, 2007

Luc Steels and Talking Heads

The ultimate goal of my project, tacit here so far, is the unsupervised development of basic communication between simulated agents. I'll be looking for the basics of language, such as:

  • Lexical convergence: words are created, propagated, and established in usage.
  • Simple grammar: tough without direct constraints on communicative methods.
  • Labeling: faced with limited time, clever agents may label one another with names.
  • Questioning: directed requests for specific information. Altogether unlikely.
The overall ambition of the project reminds me of my many failed endeavors, past projects too overwhelming and aggressive. So with my wisdom of two-and-twenty, I'll limit my expectations to the first goal, lexical convergence, and see where it takes me. In some heartening news, it's already been empirically demonstrated.

Luc Steels (Luc Steels!) in 1998 carried out an experiment wonderfully nicknamed Talking Heads (ps). It involves two cameras looking at a scene of colored shapes, and discussing these shapes with one another without any prior consensus on their language. He went to great pains to ensure the entire process was machine-driven, involving these four steps:
  1. Sensory Processing: determines bounding boxes for each image-segment.
  2. Categorization: finds a feature-set binary tree with universally distinct branches for each image-segment, based on color, position, shape, orientation, etc.
  3. Discrimination: traversing the feature-set tree for the chosen image-segment.
  4. Lexicon-lookup: agents have a private lexicon of words seen and created, along with corresponding success rates and remaining ambiguities.
The experiment's success speaks to the viability of lexicon-convergence techniques. He even incorporated some limited ability for syntax! When you have a name like Luc Steels, horizons must shiver and acquiesce at your mighty footfall.

A scintilla of unease comes when reading that each camera can gauge where the other one is pointing. I understand the essentiality of communication grounding, but why does this strike me somehow as cheating? It worries me that my simulation, in part, lacks this clue, as a firm foundational path. My advisor, Prof. Taylor, I think picked up on this immediately, and so the matter warrants further consideration.