Skip to content
Snippets Groups Projects
Commit cf4f80cd authored by Jäkel, Frank's avatar Jäkel, Frank
Browse files

Yes-no task ready.

parent 2563558d
No related branches found
No related tags found
No related merge requests found
import numpy as np
import pandas as pd
pd.set_option('precision',2)
from psychopy import core, visual, event, monitors
from imageio import imread
......@@ -63,6 +64,9 @@ def load_data(subject):
'trial':[],
'stimulus':[],
'response':[],
'hit':[],
'false_alarm':[],
'correct':[],
'RT':[]}
data = pd.DataFrame(data)
return data
......@@ -191,9 +195,12 @@ def run_block(subject,intensity,p=0.5,task='yes-no'):
'block': block,
'intensity': intensity,
'p': p,
'trial': i,
'stimulus': s,
'response': r,
'trial': int(i),
'stimulus': int(s),
'response': int(r),
'hit': int(s and r),
'false_alarm': int(not(s) and r),
'correct': int(s==r),
'RT': rt}
data = data.append(ndata, ignore_index=True)
# after a number of trials have been done save and quit
......@@ -203,3 +210,26 @@ def run_block(subject,intensity,p=0.5,task='yes-no'):
win.close()
core.quit()
return data
def summarize(data, group = ['intensity','p'], mode='all'):
grouped = data.groupby(group)
# hit rate
N1 = grouped['stimulus'].sum()
H = grouped['hit'].sum() / N1
# false alarm rate
N0 = grouped.size()-grouped['stimulus'].sum()
FA = grouped['false_alarm'].sum() / N0
# percent correct
N = N0 + N1
PC = grouped['correct'].sum() / N
# aggregate
if mode == 'pc':
summary = pd.DataFrame([PC, N]).T
summary.columns=['PC','N']
elif mode == 'roc':
summary = pd.DataFrame([FA, H, N0, N1]).T
summary.columns=['FA','H','N0','N1']
elif mode == 'all':
summary = pd.DataFrame([FA, H, PC, N0, N1, N]).T
summary.columns=['FA','H','PC','N0','N1','N']
return summary
%% Cell type:markdown id: tags:
# Signal Detection Tutorial
Signal detection theory is among the most successful theories in all of cognitive science and psychology. You will run a little signal detection experiment on yourself and analyze the data.
When you're at the ear doctor, she will increase the intensity of a tone until you can hear it. Until you can detect it. She does this to measure how sensitive your hearing is. Imagine you are a radiologist and have to decide whether on an x-ray there is a tumor present or not. This is also a detection task. You have to detect the tumor. We will be simulating this last task with our stimuli but what we will learn is applicable more generally to all detection tasks.
%% Cell type:markdown id: tags:
## Stimuli
Instead of detecting a tumor, in our task you have to detect the letter *A* in a noisy image. Running the code in the next cell will generate a plot. In it you see a noisy image *without* the letter *A* on the left. We call this the noise-only condition. On the right you see a noisy image *with* the letter *A*. This is the signal-plus-noise condition.
%% Cell type:code id: tags:
``` python
# we need psychopy and a few other libraries (pandas, numpy, matplotib, etc.)
# they are all imported in `signal_detection.py` in the same folder
from signal_detection import *
# in this cell press shift+enter to run only this cell and generate the plot!
imshow_stimuli(125)
```
%% Cell type:markdown id: tags:
The one parameter of the function `imshow_stimuli` allows you to change the intensity of the stimulus. Play with it! When it is at zero there is no signal (just like on the left). The *A* is clearly visible with the intensity being at `125`. You can probably still kinda see the *A* when you set the parameter to `50`. You should know you're looking at grayscale images where `0` is black and `255` is white. The noise-only pixels have a mean of `127`, i.e. gray, and the standard deviation of the noise was set to `25`. The pixels of the *A* have a mean of `127` plus the intensity that you've set and the same standard deviation.
%% Cell type:markdown id: tags:
## Some Cautionary Notes
The question we want to answer is: How well can you detect the *A*. Or more generally, how can we measure your sensitivity in any detection task? In a second step, which we will not address here, we could then compare your sensitivity to everyone else's in the class to find out who's the best *A*-detector (or the best radiologist or who has a hearing problem). The reason why we won't attempt a comparison is that you're all doing this task on different computers and with different monitors. Some monitors might be better than other monitors and allow for a higher contrast. Also your detection performance will depend on the luminance of the monitor and the lighting in your room. Hence, if we measure your performance on the *A*-detection task we're really measuring your performance *on your setup*. Therefore, any comparison is not meaningful psychologically (but note that some radiologists might also have better setups than other radiologists and we might therefore also care about the setup in some applications). Given that we expect the lighting in your room to also play a role you should try to keep all conditions as constant as possible, e.g. by sitting at the same desk and by closing the curtains and switching off all lights. Also make sure that you're well rested and able to concentrate while you're doing the experiments. We want to measure your best possible performance and not you're performance when you're tired or worn out. Psychological data are already noisy enough as they are, we don't want to introduce additional, unnecessary variability. With these comments out of the way. How do we measure sensitivity?
%% Cell type:markdown id: tags:
## The Yes-No Task
The first idea one could have is the following: We choose an intensity, show the *A* with this intensity to a subject 100 times and record how often the subject reports to see the *A*. The trouble with this experiment ist that the subject could just lie and always say "I see it". A simple fix is to introduce catch trials. On each trial we flip a coin an there either is (signal trial) or there is not (noise-only or catch trial) an *A*. In this way we can offer the performance objectively.
A block in the experiment will always consist of 50 trials. In each trial you will first see a fixation cross that you should look at. Then, very briefly, the stimulus will show up. With probability `p=0.5` there will be an *A*. Your task is to then press either `y` (for yes) or `n` (for no) to answer the question whether you detected the *A*. Let's try this:
%% Cell type:code id: tags:
``` python
data = run_block(subject='test',intensity=100) # run this cell by shift+enter
```
%% Cell type:markdown id: tags:
This was an easy block with a high intensity (`intensity=100`). Let's look at how well you did.
%% Cell type:code id: tags:
``` python
subject = 'test' # put in your intials here
run_block(subject,100) # and run this cell by shift+enter
data = load_data('test')
summarize(data,'block',mode='pc')
```
%% Cell type:markdown id: tags:
The first column (block) shows you the blocks you've done and the other colums how many trials you did and how well you did in each block. The first thing you want to look at is the column N. It says you did 50 trials. The column PC shows the proportion correct. Since this was an easy block hopefully the number will be close to 1. If not you need to practice this task a little more until you make only 1 or 2 mistakes in a block of 50 trials. These mistakes are not because you didn't see the *A* but because you accidentally pressed the wrong button. As the task is pretty fast paced this can happen and it's really hard to bring down the number of mistakes down to zero even if you see all *A*s. As we want to measure how well you can see the *A* it's important that you really learn the response mapping and make as few "finger errors" as possible. So really do practice this task by re-running the `run_block`-cell above (with `subject='test'` and a clearly visible `intensity=100`) before you move on.
%% Cell type:markdown id: tags:
## Psychometric Functions
%% Cell type:code id: tags:
``` python
int(np.random.rand()<0.5)
```
%% Cell type:markdown id: tags:
## Threshold and Sensitvity
%% Cell type:code id: tags:
``` python
```
%% Cell type:code id: tags:
``` python
```
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment