Crafting a Research Agenda

Fall 2023

Course Info

CSCI 654, 30393D
M/W 10-11:50pm
Instructor: Barath Raghavan
Office Hours: Th 10-11am

Overview

In this course, we will aim to understand how to formulate a research agenda in Computer Science, examine trends that exist within sub-areas of CS, and identify fruitful and problematic research directions. We will examine barriers to scientific research advancement and how to avoid them when possible. We will practice thinking meta-systematically ("thinking outside of the box") when approaching the selection of research problems and developing research solutions. We will also study prior paradigm-shifting papers in CS research and guidance from eminent Computer Scientists who have articulated what makes for good CS research and strong CS research communities.

The course will explore the hybrid nature of Computer Science -- its existence at the intersection of mathematics, engineering, psychology, philosophy, statistics, and other disciplines -- and how to evaluate progress in CS research as we identify fruitful avenues of future study. Our focus will not be on any one area of CS; we will examine research agendas in a variety of areas.

The course is intended for CS PhD students in their second year or later who are learning to articulate their own research objectives and plans.

Course Structure

The course will involve discussion, presentation, and group work during class. In addition, students will work individually on an independent project, described below, to describe the past and potential-future research in an area of CS, possibly one that is of interest to the student(s) for future study such as in a dissertation, postdoc, or faculty position.


Project

Computer Science research is diverse in both topic and method, building upon a wide array of adjacent disciplines and also having a rich body of practice of its own. Your task is to develop a research agenda in an area of CS and present this agenda in a variety of forms: 1) as a research whitepaper surveying prior work and describing a future agenda (8+ pages ACM format or equivalent), 2) as a grant one-pager that describes why such work should be funded, 3) as a 30-minute research presentation, 4) as a 10-minute general audience presentation, and 5) as a popular-science article. Even if you choose to work on your own, you'll be paired up and will be providing peer feedback on a regular basis. (To this end, you will need to make a complete draft of each milestone available to your partner five days in advance of each deadline, to allow them time to give you feedback and for you to make edits.)

Readings and Responses

Per-class assignments will focus on assigned readings and responses to those readings. For each class's assignment, add your responses to an appropriately titled section of your Google Doc, with your most recent additions at the top of the doc. Each day's assignment is due at the start of class that day.

Grading

Details of grading will be provided in class. Grades will be determined as follows:

25% Reading responses
25% Peer feedback
50% Project

Schedule

Date Class Notes Reading Due Assignment Due
M 8/21
Welcome		 Notes - -
W 8/23
Foundations		 -
  • Why did you sign up for this course? (1 paragraph)
  • Describe your current/recent research using the Heilmeier catechism (2-3 paragraphs), and ask the Five Whys on why you are working on it (1 paragraph).
  • Describe two instances of ontological remodeling in any areas of CS, one successful and one failed; also describe how they align with Chapman's analysis. (2-3 paragraphs each)
M 8/28
Foundations		 -
  • Describe a recent-ish idea (within the last decade) in your area that has changed the way people think, leading to fundamentally-new research. (2 paragraphs)
  • Apply Chapman's AI progress criteria to the topics described by Liskov and Shenker. (2-3 paragraphs each)
  • Where are abstractions (per Liskov) used in your area of CS to extract simplicity (per Shenker)? (2 paragraphs)
W 8/30
Foundations		 -
  • Apply Chapman's AI progress criteria to the topics described by Rogaway. (2-3 paragraphs)
  • How are definitions (per Rogaway) used in your area of CS, and what issues have arisen or might arise from those definitions? (3-4 paragraphs)
  • What are mistakes in writing you have made per McEnerney and Chiang? (1-2 paragraphs)
M 9/4
No Class
  • Propose, at a high level, three new research topics/themes/directions that excite you; use the Heilmeier catechism. (2-3 paragraphs each)
  • Revise (and include the before and after for) your three proposals using Chapman's guidance on problem formulations and vocabulary.
W 9/6
Ideas		 -
  • For two of the three topics you proposed previously, produce a conceptual diagram of the research space following Meadows. Describe the leverage point(s) at which you are intervening. (2-3 paragraphs + diagram for each topic)
M 9/11
Stepping Stones		 -
  • Describe some research activity you or others do in your area(s) of interest that involves reasonableness; in your description, use at least 8 of the terms (in bold) from Chapman. (2-3 paragraphs)
  • Describe two research approaches that have been "made to work", referencing two or more of the five techniques from Chapman. (2 paragraphs each)
W 9/13
Nebulosity
  • Apply at least five of Rittel and Webber's ten "distinguishing properties" to a problem domain in your area of CS that has encountered difficulty in recent years. (3-4 paragraphs)
  • Relate Rittel and Webber's "distinguishing properties" to Chapman's guidance on problem formulation. (2-3 paragraphs)
  • Identify three "messy" "kinds" in any area of CS; what makes them messy? (1 paragraph each)
M 9/18
Meta-rationality
  • Intense research period: spend an intense period (~30 hours across two days) exploring the first steps of your research agenda; use the results to revise your agenda; describe your insights. (2-3 paragraphs)
  • Propose your semester-long research topic; identify any meta-rational steps involved in your thinking. (4-5 paragraphs)
W 9/20
Writing
  • Who are your readers for each of the documents you are writing, and what is the distinctive value that they may be looking for and getting from in your work: (1) research whitepaper, (2) grant one-pager, (3) research presentation, (4) general-audience presentation, and (5) popular-science article? (1 paragraph each)
  • Re-write the abstract and introduction from your research agenda document in two ways: 1) to be as bad as possible (similar to the bad writing Orwell describes) and 2) to be as clear as possible (following Orwell's guidance) and with a focus on the importance to your research community (per McEnerney).
M 9/25 Project Updates Milestone 1: Progress update. Write the abstract and introduction of your research agenda document, and outline the full document. (Standard two-column template of your choice, ideally using Overleaf.) Place a link to the document in your shared directory.
W 9/27
Shadowing		 -
  • Ask the best researcher you know (other than your advisor) to let you shadow them as they do research (ideally an early-stage project), ideally for at least a few hours (on video chat or in person).
  • Apply Chapman's methods and periodically ask questions to understand what they're doing (and/or ask them to narrate). Document what you saw. (3-4 paragraphs)
M 10/2
Eras		 -
  • Describe two instances (past or present) where it is apparent a research project/agenda in CS was strongly influenced by the era in which it was pursued. (2 paragraphs each)
W 10/4
 - Project updates Milestone 2: Progress update. Roughly 6 pages of prose and at least 3 figures (diagrams, graphs, or tables).
M 10/9
Community		 Guest: Aurojit Panda -
W 10/11
Community
  • Describe three ideas for improving your research scene that require no outside resources. (2-3 sentences each)
M 10/16
Society		 -
  • Describe how you might follow two or more of Silberman's recommendations in your proposed research agenda. (2 paragraphs each)
  • Describe three recent research projects in any area(s) of CS; what are the embedded, unarticulated values/politics (per Winner) of these projects? (2 paragraphs each)
W 10/18
Society
  • In the use or generation of data and data types in your work, how do you or can you address and/or deal with their social construction? (4 paragraphs)
M 10/23
Society		 -
  • What notions of fairness are common between the Briscoe and Narayanan? Where would they agree? Where would they disagree? (2-3 paragraphs)
W 10/25
 -
  • Milestone 3: Progress update. Complete draft of research agenda.
  • Allow yourself to wonder about examples of nebulosity in the context of any research topics; describe two such examples. (2-3 sentences each)
  • Explore those two examples of nebulosity; describe your understanding of them. (2-3 sentences each)
  • Play with the examples, modifying them and thus exploring the space; describe your explorations. (2-3 sentences each)
M 10/30
Theories		 -
  • Examine two research areas of CS and using the terminology of Lakatos and Havstad, describe whether they are each progressive, degenerative, or static research programs. (2 paragraphs each)
  • Identify four cognitive biases, three fallacies, and two paradoxes from the lists that have arisen in any area of CS at any time in the past, especially those that have been a barrier to the advancement of research. (2-3 sentences each)
W 11/1
Metacognition		 - -
M 11/6
Collaboration		 -
  • What are some boundary objects in your field at the intersection of it and another area? (1-2 paragraphs)
W 11/8
Advice		 -
  • What three pieces of advice were most surprising to you? (2-3 sentences each)
M 11/13 - Project Updates Milestone 4: Progress update. Draft of popular science article, grant one-pager, and slide decks.
W 11/15 -
  • What are two 'packages', in Fujimura's terminology, in some area of CS and what made them successful? (2-3 paragraphs)
M 11/20 - Project Updates -
W 11/22
No Class		 - - -
M 11/27
Mock Program Committee		 - - -
W 11/29
Mock Program Committee		 - - -

In addition to the assigned readings in the schedule above, this is an incomplete list of papers to consider for presentation in class; this list is incomplete and will be expanded: