This illusion was taken from Akiyoshi Kitaoka’s website.

The fish shapes appear to drift in the directions of their purple heads. The illusion also works without color. The proposed explanation depends on contrast latencies, where higher-contrast elements (the black tail and yellow background) are processed first. The lower-contrast heads arrive slightly delayed and produce the impression of motion. Based on this hypothesis, we predicted that coloring the fish with two middle-luminance grays should allow us to reverse the direction of the perceived motion by switching the background from light to dark. This effectively changes which luminance is the ‘low-contrast’ one, and alters the predicted direction of the illusory motion. This appears to have worked, as is visible in the gif below:

When the luminance of the background is changed slowly over time, a strong illusory motion is perceived within the fish as the luminance of the background passes through the range contained within them. The interactive demo below allows an exploration of this effect:
-Mouse Y controls background luminance
-Clicking inverts the luminance at any point

Some further experimental gifs follow:

Blurred version to remove high-frequency spatial information:

Version with a gradient for the background:

Version with the background colors as the middle luminances. Any perceived motion here might theoretically be due to delays of perception of parts of the gradients between black and white within the fish: