Shuo & Fred
Our aim was to test whether fruit flies expand or reduce their behavioral repertoire upon receiving randomly timed rewards. This was motivated by BF Skinner's seminal experiment on 'superstition in the pigeon', in which he could show that pigeons would develop idiosyncratic, 'superstitious', behaviors in response to rewards presented at regular intervals .
Figure 1. Experimental setup
We acquired video data from single fruit flies freely walking in a circular arena. The temperature of the arena was kept at a temperature of 17°C, which is below the preferred temperature of Drosophila melanogaster [2,3]. We used an infrared laser to heat up the arena to around 19°C (control condition), or to deliver short (3s) pulses of heat 'rewards' (random reward condition). The overall laser power was kept constant during both conditions. During the random reward condition, the intervals between successive reward deliveries followed a Poisson distribution (mean interval: 10s).
We extracted the centroid position and orientation of the fly for each video frame, and used this information to create a new video in which the fly was centered and rotated (facing right) for each frame. The goal was to have a representation of the fly in which only the legs move.
On this new video we used principal component analysis (PCA) followed by a wavelet transform, to create a spatio-temporal representation of the behavioral dynamics of the fly (see  for details).
This high-dimensional representation was then embedded into two dimensions, using t-SNE , forming a 'behavioral map', containing all behavioral patterns that were present in the video data. We used kernel density estimation to calculate probability densities from these behavioral maps, for both the control and the random reward condition. Subtraction of the probability densities resulted in a difference map, which was used to identify behavioral patterns that were omitted from or added to the behavioral repertoire during the random reward condition.
We used Bonsai for data acquisition and preprocessing of video data. Python and Matlab were used for video analysis. For automated detection and classification of movement patterns we used Laurens van der Maaten's tsne toolbox (http://lvdmaaten.github.io/tsne/), following the analysis pipeline described in .
 Skinner BF (1948) ’Superstition’ in the pigeon. Journal of Experimental Psychology 38: 168–172. doi:10.1037/h0055873.
 Dillon ME, Wang G, Garrity PA, Huey RB (2009) Review: Thermal preference in Drosophila. Journal of thermal biology 34: 109–119. doi:
 Shih HW, Wu CL, Chang SW, Liu TH, Sih-Yu Lai J, Fu TF, et al. (2015) Parallel circuits control temperature preference in Drosophila during ageing. Nature Communications 6. doi:10.1038/ncomms8775.
 Berman GJ, Choi DM, Bialek W, Shaevitz JW (2014) Mapping the stereotyped behaviour of freely moving fruit flies. Journal of The Royal
Society Interface 11: 20140672. doi:10.1098/rsif.2014.0672.
 van der Maaten L, Hinton G (2008) Visualizing data using t-sne. Journal of Machine Learning Research 9: 2579–2605.