Sundry notes and code bits for Stanford's Programming in Journalism class (COMM 177P/277P).
- Syllabus
- Assignments and Grading
- Bookshelf - recommended books, tutorials, cheatsheets, etc.
- Code solutions - A private repo containing solutions to class exercises.
- DataKit - overview and details on install and usage
- Getting Help - Resources and strategies for finding help.
- Glossary - technical terms used in class
- Python - overview, tutorials, etc.
- Technical setup and FAQ - recommended and required software (all free)
- Workflow advice - working on the command line, etc.
- Course overview - syllabus, grading, etc.
- Discuss the history of code and data analysis in journalism
- Preview of automation with Bash
- How to Succeed in this Course
- The Owl, Problem Solving, and the Unix Workbench
- Bash intro
- Bash drill
- Assignment 0 - Unix practice and failed banks script
- Misc
- Software installs -- everyone must have Python installed and GitHub configured by end of class.
- Unix Power Tools for Data Wrangling
- Bash drill - Yep. Again. This time using the "Blind-folded" and "Plain English" variations.
- Assignment 1 - Python reading/practice and code challenge
- Review Assignment 0 (failed banks) if everyone submitted...
- Python Intro and resources links to misc docs/tutorials on this GitHub repo
- Python overview and coding contexts
- Questions about Assignment 1? It covered a lot of important fundamentals:
- Python interactive shell
- Expressions
- Basic data types
- Variables
- Flow control (conditions and blocks, if/else/elif etc.)
- Python Syntax Crash Course
- Assignment 2 - Python lists/dicts and DataKit overview
No class.
- Questions about Assignment 2 on Python lists/dicts?
- Complete configuration steps
- Pop Quiz: Lists/Dicts (on Tmux in the cloud for a little taste of Pair Programming)
- Dictionaries crash course
- Reading and writing text files
- Choose-your-own adventure pair coding challenge (election data or covid in prisons)
- Assignment 3 - Practical Python skills and Failed Banks in Python
- Automating workflows
- Overview of DataKit
- Complete Tech Setup configuration and install DataKit
- Continue work in-class on Assignment 3 - FDIC python script (due by next class)
- DataKit workflow review
- Wherefore, virtual environments and pipenv?
- Libraries - Overview and practice practical skills
- Learn how to use code libraries included with Python or offered by third parties
- Learn how to download remote files
- Learn how to work with CSVs
- Continue pair programming on Assignment 3 - FDIC python script (due by next class)
- Assignment 4 - Level up on Python functions, sorting and web basics (due by next class)
- APIs and the News overview and presentation
- Working with APIs - we didn't have time to work through this one, but it's got some good background and exercises. Check it out!
- Quakebot exercise - hands-on practice with a JSON feed
- Assignment 5 - Senate compromisers Python script
- Quakebot exercise solutions?
- The Art of Writing Functions
- Assignment 5 - Senate compromisers Python script
- Assignment 6 - Final project story idea pitches
- Web scraping for the news - a high-level overview of web scraping in a news context.
- Web scraping 101 - how to dissect a website, various obstacles to scraping and strategies for overcoming them
- Web scraping exercises - some websites to challenge your newfound scraping skills
- Web scraping resources - tutorials, etc.
- Data Pipelines with Modules
- API Services
- Refinitiv Entity Extraction
- Project (extra credit): FDA Recall Entities - Scrape FDA recalls and extract entities using the OpenCalais/Refinitiv API
- Assignment 9 - First Python Notebook
- Discuss data journalism readings
- Whirlwind Tour of the Data Journalism Process
- Final Project - Present your story idea and begin data sleuthing & coding as teams.
- Data Analysis with Jupyter and pandas Overview
- Coding workshop: Work on Final Project or FDA Recalls project
Classes canceled for President's Day.
- Final project check-ins
- Coding workshop: Work on Final Project or FDA Recalls project
- Review solutions to FDA Recalls project
- Coding workshop: Work on Final Project
- Final project check-ins
- Coding workshop: Work on Final Project
- Coding workshop: Work on Final Project
- Final Project presentations
- Course retrospective
- What's next?