Finite State Analysis of Behavior
in the Lobster
Joseph Ayers
Marine Science Center
Northeastern University
East Point, Nahant, MA 01908
lobster@neu.edu
Finite state Analysis of animal behavior. A. Digital movie of lobster behaving in an aquarium.


Analyzing Lobster Behavior Under well illuminated conditions, lobsters are exceptionally visual creatures and when under observation, much of their behvaior will consist of them reacting to the observer. In order to maximize the degree of sterotypy during behavior it is necessary to blind specimens. This insulates them from perturbations due to movements of the observer, conspecifics and constrains their behavior to proprioceptive and tactilly exteroceptive inputs. Our analytical procedure proceeds in the following steps:
  1. Preparing the specimens
  2. Adjusting Illumination
  3. Videotaping Behavioral Sequences
  4. Selection of Behavioral Sequences
  5. Cataloging Videotapes
  6. Creating Quicktime Movies
  7. Finite State Analysis
  8. Graphical Analysis
  9. Segmentation of Sequences



Preparing specimens for analysis: Specimens are identified by painting a number on the dorsal carapace and/or the claws so that they can be recoginized as individuals from videotape. Specimens are reversibly blinded passing a 2cm wide strip of aluminum foil under their eyecups and then twisting it over the rostrum to stabilize the eyecops. The aluminum foil is then further stabilized by adhering a three-quartered folded piece of duct tape over the foil. The position of the joints of the theirthird legs bear specific joint markers which aid in behavioral analysis.



Adjusting Illumination

Illumination is optimized by filming during prime daylight hours. Viewing underwater animals presents problems of surface reflectance and distortion. We elimimate these variables using a light box which is suspended from a downward looking camera. The light box prevents refractive distortion by flattening any ripples, The black shroud eliminates surface reflectance.

Videotaping behavioral sequences

Two night vision cameras are used for filming the specimens. One camera is placed above a designated filming area ensuring a top view of the specimens, while the second is place on the side of the designated filming area optimizing all viewing angles.

Digitizing Behavioral Sequences

Following the filming process, videotapes are reviewed and edited to digital videotapes. Digitizing behavioral sequences then allows us the capability of finite behavioral analysis.

Reverse Engineering Locomotory Behavior

We have developed computer controlled video technology for reverse animation and kinematic analysis of animal behavior (Ayers, 1989, Ayers, 1992). This multi-media system allows correlated acquisition of kinematic and electrophysiological data by simultaneously recording behavior in the video signal and electrophysiology on the audio channels of a high resolution digital VCR. We developed extensions to a public domain image analysis program (NIH Image) which include the capability for color based acquisition and image segmentation as well as time-based quantification of kinematic parameters and correlated analog acquisition (ColorImage, Ayers and Fletcher, 1990; Ayers, 1992). This system allows us to measure animal orientation, joint angles from video on a frame by frame basis to establish the detailed movement strategies kinematics of compensatory, orientational and taxic reflexes as well as the underlieing neuromuscular control signals. As a result it has been possible to establish the coordination patterns and control signals underlieing omnidirectional walking (Ayers and Davis, 1977) as well as undulatory swimming (Ayers, 1989).


Radio-button panel used in constructing the state diagram

To directly transit from behavior to robotic controls, we perform finite state analysis task groups that mediate locomotion and searching individually to determine which synergistic sets are operant during different behavioral acts (Schlichting and Ayers, 1996; Ayers, Mehta and Dragich, 1997). Our analysis of the sequencing of these task groups borrows from a technique utilized by astronomers to detect motion of galactic objects. As the analysis proceeds through each frame of the digital movie, the program flashes between temporally adjacent frames of the movie with a brief pause after each cycle. Appendages that are moving the most flash in these projections. A panel of buttons that represent different states of the task groups (eg. elevation vs depression of the chelipeds, etc. are available to the investigator to specify which groups are active



By clicking on the appropriate buttons for each frame, it is possible to efficiently quantify the activity of all task groups at high temporal resolution from video tapes of specimens behaving in a variety of situations. These state diagrams are used to establish control sequences for the robots based on the behavior of the model organisms.


Finite-state diagram indicating state changes of the different task groups during behavior
What are the units of behavior

We define the fundamental atom of lobster behavior as an action component. An action component is a particular state which may be triggered by a releaser during which the state of all the task groups is held stabilly.

We define a sequence as an orderied series of action components which are linked as a unit. Theses sequences correspond to modal action patterns.