Main Page
How to Do Science Stuff
This wiki contains tutorials and other information about various tasks a member of a cognitive science lab might want to do.
Contents
Overview
This is a compendium of how-tos for various science-related tasks, with special emphasis on cognitive science.
Web-based experiments
Data collection venues
There are many venues for collecting data via the Web, including your own website and a variety of professional services. For a discussion of which venue to choose, visit Which venue should I choose?. Below are links to pages for specific venues.
Research-focused venues
Online labor markets
- Amazon Mechanical Turk - The largest online labor market, frequently used by researchers.
- ShortTask.com - An Amazon Mechanical Turk competitor.
- oDesk.com - Focuses on long-term jobs. Workers download software that tracks time they spend working on projects, logs their keystrokes, and takes periodic screenshots. It is not clear whether researchers have used this to run experiments.
- Guru.com - Bills itself as a freelancer clearinghouse. Workers' claims of specific skills are verified, and work history and portfolios are available.
It is not clear whether researchers have used this to run experiments.
Ensuring Data Quality
Main article: Ensuring Data Quality
- Data quality is an issue in all experiments, not just experiments on the Internet. Many of the same procedures used in the lab can be used online, including the inclusion of catch trials. There is some evidence that paying participants may decrease data quality.
Additional Resources
- There are a number of good blogs that discuss running experiments on the Web, including CrowdFlower's blog, GamesWithWords, A Computer Scientist in a Business School, and Experimental Turk.
Obtaining Technical Skills
React
- CodeAcademy Learn ReactJS: Part 1 and Learn ReactJS: Part II are pretty decent and fairly in depth. However, discussion of routing is a little out of date.
- This simple tic-tac-toe tutorial is a good recap for brushing up on React components, or after finishing the CodeAcademy lessons [1](here).
- Routing with React changed significantly with v4. This tutorial from auth0 is a good starting place [2](here).
- However, note that their implementation of registerServiceWorker is out-of-date. You should use the default that comes with create-react-app instead.
Recommendations for Pushkin developers
You need to be comfortable with:
- Javascript
- jsPsych
- ReactJS
- Redux
- NodeJS
Tutorials
Other
Platforms for online studies: http://l3atbc.org/comparisonTable.html
List of known MOEs: https://docs.google.com/document/d/1T0_vMgrsJgdksTyQ_G-e9R0Y2wwf6e4FlsSpIUpj3iU/edit?ts=5af1f8de
Neuroscience
EEG
Overview: A step-by-step tutorial on processing ERP results using EEGLab.
Original Author: Joshua Hartshorne
Eye-Tracking
Overview: A tutorial on creating, running, and analyzing studies using E-Prime and Tobii.
Original Author: Tracy Brookhyser and Manizeh Khan
Analysis and Graphing
- This wiki contains a growing number of resources on how to analyze data using R. Those unfamiliar with R should read An Introduction to R and/or consult the R wiki. Harald Baayen's Analyzing Linguistic Data is also useful, whether one is analyzing linguistic data or not.
Using R
Statistical Analysis
Visual Display of Data
An example of a complex barplot in R.
About How To Do Science Stuff
How To Do Science Stuff is maintained by Joshua Hartshorne. Want to contribute? Email gameswithwords@gmail.com.
