Final Project Description 

In the last half of the class quarter, class participants will design, execute, and present a final project. This year, the final project will be to contribute a model, metric, or dataset to the Brain-and-Behavior Score (BBScore) repository. BBScore is a code package that we use in the NeuroAILab to organize and standardize model-brain comparisons.

Essentially, each BBScore result consists of a pretrained model, run (in inference mode) on a set of stimulus inputs, and then compared to brain data using a well-motivated metric of comparison. For example, in this study, the authors compare a convolutional neural network, on a set of naturalistic image inputs, to electrophysiology data from two areas of macaque ventral visual pathway. Or for example, in this paper, the authors compare a language model, on a set of sentences from short stories, to fMRI data from the human language brain areas. BBScore helps standarize these types of comparisons so that results can be shared and compared across models, datasets, and metrics.

In the final project for this class, you will contribute to BBScore. This will take the form of:

• Adding a new model and then running it on an existing benchmark dataset and metric, and drawing some conclusion about the way model structure does or does not affect neural alignment. Existing benchmarks include data for the visual system, the auditory system, and the language system.
• ... adding a new metric, running it on a set of existing models and dataset(s) and then showing how it helps support a novel conclusion about model comparisons. We have a large library of existing models to help you get started on this type of project.
• ... adding a new dataset and then running it on a set of existing models and metrics. To do this type of project, you will find a public dataset that we have not yet integrated in to BBScore -- or, if you happen to have data of your own you wish contribute, that could work well too.
• ... or, some combination of the above.

We will provide you a tutorial on how to use BBScore in the first half of the class quarter; we will then work with you to help you select the model, metric, or dataset you want to contribute. In the last third of the quarter, you will need to validate your project -- that is, make sure that it can basically run. You will give a short presentation about your contribution during the end-of-qurater exams period, at the conclusion of which you will turn in a short Jupyter notebook illustrating your contribution as running BBScore code, and contribute it as a Pull Request to the BBScore public repo.