{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ec7211ca-a104-4c3d-b528-102841bfd937",
   "metadata": {
    "id": "ec7211ca-a104-4c3d-b528-102841bfd937"
   },
   "source": [
    "# Part 1\n",
    "\n",
    "**Motivation**: Perceptual decision-making is a core aspect in everyday cognition. In difficult perceptual situations, for instance when driving a car through heavy rain and trying to read a traffic sign, perceptual decision-making can be error prone. Standard models assume that these errors are either due to:\n",
    "*   a high *noise level in the sensory input* (raindrops on the windshield), or\n",
    "*   *internal brain processes* (neuronal noise) or\n",
    "*   a *mixture of both*.\n",
    "\n",
    "\n",
    "In the laboratory, perceptual decision-making is usually investigated by linking the perception of incoming sensory information to making a choice among several alternatives. For example, in the widely-used random dot motion task, participants are required to report the net direction of a cloud of moving dots within a certain time frame. (Park et al., 2016)\n",
    "\n",
    "The perceptual decisions we are going to look at in the following exercises depend on these parameters:\n",
    "\n",
    "**input, properties of the stimulus**\n",
    "- coherence, i.e. the percentage of coherently moving dots in the cloud of dots;\n",
    "![image.png](data:image/png;base64,[encoded data, 51728 characters])\n",
    "\n",
    "- fast noise, i.e. sensory input from all incoherently moving dots, which are masking the signal and change very rapidly from frame to frame;\n",
    "\n",
    "**internal brain processes**\n",
    "\n",
    "- slow noise, which varies from trial to trial or even slower i.e. it is fixed within a trial;\n",
    "- fast inner noise, which changes a lot within a trial;\n",
    "- lambda, which represents either leaky evidence accumulation (λ < 0, imagine filling a bucket with a hole in it), perfect accumulation (λ = 0, our bucket holds all the water we pour into it) or instable accumulation (λ > 0, this bucket keeps multiplying the water we've poured into it).\n",
    "Lambda characterizes memory: do we forget? do we remember perfectly? does the evidence overwhelm us?\n",
    "\n",
    "Overall, the models we are going to look at are important because they give us an idea about what kind of transformation happens in the brain such that it turns the known input (stimulus) into known output (choices of participants).\n",
    "\n",
    "And if we know what this latent process looks like than we know what the neural computations are likely aiming to implement. We can correlate that latent process with neural activity of a single neuron and at neural populational level, and we'll look at that in Part 2 of this exercise.\n",
    "\n",
    "However, let us start with the latent process itself, and Part 1 of this exercise is dedicated to getting acquainted with it.\n",
    "\n",
    "If you run the plot below, you'll see 5 traces of that latent process, depending on the parameter values you pick. Try different values, see how they influence the traces. And once again let's remember why this is exciting: we are now looking at the hidden unobservable process, that is transforming the incoming signal (represented as green 'evidence pulse' in the plot) into a decision to press rightward (if the trace ends above 0; it will be coloured blue so you can check yourself) or leftward (if the trace ends below 0; it will be coloured orange so you can check yourself). Those are not just some abstract lines -- these are links between what people see and what people perceive and consequently act upon.\n",
    "\n",
    "This is a 'door' into inner workings of the brain without opening the scull, inserting electrodes or putting people into various brain scanners. Just from observing the behavior and relating it to the stimulus.\n",
    "\n",
    "### Recap of the parameters\n",
    "(to put them conveniently near the plot)\n",
    " - coherence (input, in the stimulus);\n",
    " - fast noise (input, in the stimulus);\n",
    " - slow noise (in the brain);\n",
    " - fast inner noise (in the brain);\n",
    " - lambda (in the brain): leakiness (λ < 0), instability (λ > 0) or perfect accumulation (λ = 0).\n",
    "\n",
    "In the plot below we show 5 traces of the accumulator variable (aka latent process). For movement in the rightward direction we define positive input. Thus, if at the end of the trace the accumulator variable is positive this corresponds to the rightward choice, and negative –- to the leftward choice.\n",
    "Try different values of the parameters and familiarize yourself with how the latent process responds to them. Can you describe those regularities?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8c86d00f-9df3-478b-a5b4-d4457f727aec",
   "metadata": {
    "id": "8c86d00f-9df3-478b-a5b4-d4457f727aec"
   },
   "outputs": [],
   "source": [
    "from utils import plot_sims\n",
    "\n",
    "plot_sims()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e6cce32-6db5-46fa-bd46-93b04eb4d38e",
   "metadata": {
    "id": "1e6cce32-6db5-46fa-bd46-93b04eb4d38e"
   },
   "source": [
    "### Model-free Analyses\n",
    "\n",
    "In a real experiment we do not just collect 5 trials, we collect thousands of them. When exploring such data we usually start with model-free analyses.\n",
    "\n",
    "A model is a set of assumptions that a system behaves a certain way. For example, in the simulations above we assumed that the perception of dots movement, which participant communicates to us via the final choice (left or right keypress), depends on the noise level in relation to signal strength. Those are the parameters we explored above.\n",
    "\n",
    "It is important to understand though that every parametrization is a hypothesis: we assume that this is what matters, this is what governs the latent process and consequently the observed behavior.\n",
    "\n",
    "Hypotheses, however, are not facts, but educated (and sometimes wild) guesses. They might be wrong. As statistics teaches, before applying any method -- test if the assumptions of that method are met.\n",
    "\n",
    "One way to do that is by performing model-free analyses: finding ways to represent the data without assuming what's going on. We illustrate this approach by two plots here:\n",
    "- performance, i.e. % of correct choices (y-axis) after seeing moving dots for t seconds (x-axis);\n",
    "- psychophysical kernels, i.e. how much the same ‘bit’ of evidence at any point in time influenced the final choice.\n",
    "\n",
    "We know the ground truth, because here we are working not with the data from participants, but with 12 thousand simulations of choices (2000 trials per condition, 6 conditions: 3 coherences (0.2;0.4;0.8) x 2 directions of motion (rightward and leftward)).\n",
    "\n",
    "This gives us an opportunity to see how each assumption (parameter value) would manifest in those model-free plots. You can now explore it by changing values on the sliders for individual parameters and their various combinations.\n",
    "\n",
    "Can we distinctively recognize some parameter presence and maybe even say something about their values (e.g. whether lambda is positive, negative or zero) from the shape of these model-free plots? Do any parameters produce similar effect? The latter could indicate that the model including all the proposed parameters might be ill-defined for the dataset of this kind.\n",
    "\n",
    "Once again, play with parameters and describe the influence of each: can we see their signatures in model-free plots? Are they unique? If we had data with unknown ground-truth (parameter values) could we tell just by looking at model-free plots what are those values most likely?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9cd91779-f97c-4881-a7b0-65f782f21180",
   "metadata": {
    "id": "9cd91779-f97c-4881-a7b0-65f782f21180"
   },
   "outputs": [],
   "source": [
    "from utils import plot_model_free_analysis_conditions\n",
    "import numpy as np\n",
    "\n",
    "plot_model_free_analysis_conditions(\n",
    "    C=np.concatenate(([0], np.ones(10))),\n",
    "    ks=[0.2, 0.4, 0.8]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8937caf-c1f9-44c1-bcb5-7da7aae8d2a5",
   "metadata": {
    "id": "e8937caf-c1f9-44c1-bcb5-7da7aae8d2a5"
   },
   "source": [
    "Finally, to conclude part 1, let us look at two datasets (dashed lines on plots below) via model-free plots.\n",
    "\n",
    "As in the real experiment, the ground truth, i.e. the true generating model and it's parameters, are unknown to you.\n",
    "\n",
    "Let us assume in this exercise that we decided to proceed with the model we've been exploring so far. As experimenters we know the input coherencies of the stimuli we've shown to participants (see k values in plot legends). However, we do not know the amount of noise that our random dots masking the signal produced. And, of course, we have no idea what the inner parameters are.\n",
    "\n",
    "The task here is to find such parameter values that seem like it could be a match to the data.\n",
    "\n",
    "Do you need one parameter for that or a combination of parameters? Is the solution you found unique or could you find multiple solutions?\n",
    "\n",
    "Write down the solution(s) you found. We'll discuss them together.\n",
    "\n",
    "What else can possibly influence our ability to infer those values from data? Tip: look at the errorbars, they show 95% confidence intervals and quantify our uncertainty about any point estimates. The wider they are the less we can trust the position and exact shape of those curves.\n",
    "\n",
    "The size of the dataset influences the precision of our estimates. In this exercise you have two example datasets with a pair of model-free plots for each. Can you identify from the plots which dataset is 8 times larger than the other one?\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4293d849-168d-430f-b135-751df049779f",
   "metadata": {
    "id": "4293d849-168d-430f-b135-751df049779f"
   },
   "outputs": [],
   "source": [
    "from pymatreader import read_mat\n",
    "from utils import plot_model_free_analysis_conditions_vs_baseline\n",
    "\n",
    "dataset_1 = read_mat('dataset1.mat')\n",
    "\n",
    "plot_model_free_analysis_conditions_vs_baseline(\n",
    "    baseline_data=dataset_1\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9a4b8c18-1d04-4805-a34e-544d235519ab",
   "metadata": {
    "id": "9a4b8c18-1d04-4805-a34e-544d235519ab"
   },
   "outputs": [],
   "source": [
    "from pymatreader import read_mat\n",
    "from utils import plot_model_free_analysis_conditions_vs_baseline\n",
    "\n",
    "dataset_3 = read_mat('dataset3.mat')\n",
    "\n",
    "plot_model_free_analysis_conditions_vs_baseline(\n",
    "    baseline_data=dataset_3\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c092e28-1a2d-4860-9663-736668641117",
   "metadata": {
    "id": "0c092e28-1a2d-4860-9663-736668641117"
   },
   "source": [
    "# Part 2\n",
    "\n",
    "Now let's look at the neural data recorded in monkeys performing random dots task.\n",
    "\n",
    "In part 1 we mentioned that behavioral models describe latent processes which we can later relate to the neural activity.\n",
    "\n",
    "It is important that you do not mix those: behavioral models are only based on data (stimuli and choices) and are our best guess at what happens inbetween, they give us hypotheses about 'what does brain do'; neural data in this part is more about 'how does brain do that'.\n",
    "\n",
    "That accumulator variable a(t) above is never actually observed by experimenters, it is just their best explanation of how the brain could solve the problem. With actual neural recordings we might finally get access to it.\n",
    "\n",
    "And, of course, our assumptions from behavioral models (part 1) have to make sense given what we observe in neural data. If we say in part 1 that information is accumulated somehow (be it like water in the proverbial bucket or something else) than we expect that neurons are implementing this accumulation somehow.\n",
    "\n",
    "Questions then are: do they? Can we see any support for this idea? How would we see it by the way?\n",
    "\n",
    "Remember the qualitative similarity which we aimed for when inferring parameters that could've generated the datasets in part 1?\n",
    "\n",
    "Let us now look for that qualitative similarity between the accumulator variable a(t), i.e. latent process in the very first plot of part 1, and the neural data: can you find neurons with firing rates (y-axis) resembling accumulation?\n",
    "\n",
    "All the plots start with the onset of the stimulus (0 at x-axis) and show averages across all trials for individual neurons."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fe4fc38a-e2f5-4c7b-b8db-0fd5f769413d",
   "metadata": {
    "id": "fe4fc38a-e2f5-4c7b-b8db-0fd5f769413d"
   },
   "outputs": [],
   "source": [
    "from pymatreader import read_mat\n",
    "\n",
    "neural_data = read_mat('neural_data.mat')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8b76d7b5-0cc6-44bf-a874-bf0f9e8a83ae",
   "metadata": {
    "id": "8b76d7b5-0cc6-44bf-a874-bf0f9e8a83ae"
   },
   "outputs": [],
   "source": [
    "from utils import plot_single_neuron\n",
    "\n",
    "plot_single_neuron(neural_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6viqTYAWLKAk",
   "metadata": {
    "id": "6viqTYAWLKAk"
   },
   "source": [
    "We can also explore if the neuron responds differently to one experimental condition compared to the other. Let us once again look at trial averages for individual neurons.\n",
    "\n",
    "Can you find neurons that prefer (have a higher response rate to) right choice? What about left choice? Do all neurons have such a preference? It is called selectivity.\n",
    "\n",
    "Is there any difference in neural selectivity for correct and incorrect trials?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ca88ad6f-443f-4693-8ec1-193fb1a71af8",
   "metadata": {
    "id": "ca88ad6f-443f-4693-8ec1-193fb1a71af8"
   },
   "outputs": [],
   "source": [
    "from utils import plot_neuron_by_choice\n",
    "\n",
    "plot_neuron_by_choice(neural_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9tgTnwsbMAdX",
   "metadata": {
    "id": "9tgTnwsbMAdX"
   },
   "source": [
    "As you remember from part 1, right or left choice are only two of experimental conditions in random dots. Coherency is another condition, determining the strength of the signal controlled by the experimenter.\n",
    "\n",
    "In the plots below you can investigate how neurons respond to three different coherency levels, where 0% coherency means that there is no signal at all in the stimulus, just noise.\n",
    "\n",
    "Would you say that any neurons are selective for coherency? Is there a difference between correct and incorrect trials?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6198f115-9a11-4459-9849-cf07e04fe756",
   "metadata": {
    "id": "6198f115-9a11-4459-9849-cf07e04fe756"
   },
   "outputs": [],
   "source": [
    "from utils import plot_neuron_by_coherence\n",
    "\n",
    "plot_neuron_by_coherence(neural_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "faBWPsbdM9rn",
   "metadata": {
    "id": "faBWPsbdM9rn"
   },
   "source": [
    "We looked at trial averages for individual neurons so far. We can also look at the neural population averages.\n",
    "\n",
    "To do that let's first introduce a measure Δ that quantifies the strength of the preference each neuron has for rightward choice compared to leftward choice.\n",
    "\n",
    "Negative values of Δ show that leftward choice is preferred over rightward."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f9498e86-6f2d-4010-82ac-9d924a5af29c",
   "metadata": {
    "id": "f9498e86-6f2d-4010-82ac-9d924a5af29c"
   },
   "outputs": [],
   "source": [
    "from utils import calculate_deltas, plot_deltas\n",
    "\n",
    "deltas = calculate_deltas(neural_data)\n",
    "plot_deltas(deltas)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "vFWrlasNOKzd",
   "metadata": {
    "id": "vFWrlasNOKzd"
   },
   "source": [
    "Now you can use the Δ plots above to select the threshold delta level. Only neurons, which have the absolute Δ value above the threshold would form the population and their activity will be averaged in the population average plot below.\n",
    "\n",
    "What is the influence of increasing the Δ threshold?\n",
    "\n",
    "Do you think that neural population codes (i.e. is selective) for the choice?\n",
    "\n",
    "Does this population average signal seem to you more or less noisy than individual trial averages?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "54d93ba0-96b1-43fc-b8d4-772a78071147",
   "metadata": {
    "id": "54d93ba0-96b1-43fc-b8d4-772a78071147"
   },
   "outputs": [],
   "source": [
    "from utils import plot_aggregated_neurons\n",
    "\n",
    "plot_aggregated_neurons(neural_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0NeSC_7TO_LV",
   "metadata": {
    "id": "0NeSC_7TO_LV"
   },
   "source": [
    "Finally, to make a connection between intuitions gained in both parts of this exercise, let us circle back to part 1 and generate simulated behavioral data of the same dimensions as neural data.\n",
    "\n",
    "Just for comparison: what would we get if we averaged over accumulator variable across all trials? (You can try to answer this question before running the plot. Start with: do you expect it to come out noisier than individual a(t) latent process or quite the opposite?)\n",
    "\n",
    "Does it resemble neural population average in any way?\n",
    "\n",
    "What is different?\n",
    "\n",
    "What do you think it means? Does our model from part 1 seem to capture everything with its assumptions or do we see evidence that this is not the whole story?\n",
    "\n",
    "Can you find parameters that increase qualitative similarity between the two?\n",
    "\n",
    "Is the solution unique?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "69d2b0e1-2364-4bdf-a25f-b5f5e0102232",
   "metadata": {
    "id": "69d2b0e1-2364-4bdf-a25f-b5f5e0102232"
   },
   "outputs": [],
   "source": [
    "from utils import plot_sims_conditions\n",
    "\n",
    "plot_sims_conditions(neural_data)"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
