Getting Started¶
We're glad to have you joining us, taking a few minutes to read the following pages will help you be a better member of our community:
- Our Code of Conduct is important to us, and helps us maintain a healthy community.
- We also have a guide to help you learn where to get help that you should look over.
If you are new to open source, or unfamiliar to contributing to open source projects, it might be beneficial to read this guide. We welcome and value all contributions to Open States, including (but not limited to!) code contributions.
This guide assumes a basic familiarity with using the command line, git, and Python. If you are unfamiliar with something that is mentioned in this guide, we encourage you to read the linked resources.
No matter how experienced you are, it is a good idea to read through this section before diving into Open States' code.
No worries if you aren't an expert though, we'll walk you through the steps. And as for Python, if you've written other languages like Javascript or Ruby you'll probably be just fine. Here's a great guide on getting started installing and managing Python versions.
Don't be afraid to ask for help either!
Project Overview¶
Open States is a fairly large & somewhat complex project comprised of many moving parts with a long history.
As you look to contribute, it may be beneficial to understand a little bit about the various components.
These repositories make up the core of the project, if you're looking to contribute there's a 95% chance one of these is what you want.
- openstates-scrapers - Open States' scrapers. All code related to getting information from a website and storing it in the Open States database lives here. What is a scraper?
- people - Open States people & committee data, maintained as editable YAML files.
- openstates-core - Open States data model & scraper backend.
- openstates.org - Powers OpenStates.org website & GraphQL API.
- api-v3 - Powers API v3.
- documentation - you're reading it now.
Installing Prerequisites¶
poetry¶
If you're working on the people
repo, api-v3
, or want to work on scrapers without Docker, you'll need poetry
to build your Python virtual environment.
Note
If you haven't used poetry
before, it is similar to pipenv
, pip
, and conda
in that it manages a Python virtualenv on your behalf.
Installing Poetry
The official poetry docs recommend installing with:
curl -sSL https://install.python-poetry.org | python3 -
Then within each repo you check out, be sure to run:
poetry install
Which will fetch the correct version of dependencies.
docker & docker-compose¶
When working on scrapers or openstates.org, you have the option to use Docker.
The first thing you will need to do is get a working docker environment on your local machine. We'll do this using Docker. No worries if you aren't familiar with Docker, you'll barely have to touch it beyond what this guide explains.
Install Docker and docker-compose (if not already installed on your local system):
(a) Installing Docker:
- On OSX: Docker for Mac
- On Windows: Docker for Windows
- On Linux: Use your package manager of choice or follow Docker's instructions.
(Docker Compose is probably already installed by step 1(a) if not, proceed to step 1(b))
(b) Installing docker-compose:
- For easy installation on macOS, Windows, and 64-bit Linux.
Ensure that Docker and docker-compose are installed locally:
$ docker --version
Docker version 19.03.4, build 9013bf5
$ docker-compose --version
docker-compose version 1.24.1, build 4667896b
Of course, your versions will differ, but ensure they are relatively recent to avoid strange issues.
pre-commit¶
To help keep the code as managable as possible we strongly recommend you use pre-commit to make sure all commits adhere to our preferred style.
-
Within each repo you check out, run
pre-commit install
after checking out. It should look something like:$ pre-commit install pre-commit installed at .git/hooks/pre-commit
Note
If you're running flake8
and black
yourself via your editor or
similar this isn't strictly necessary, but we find it helps ensure
commits don't fail linting. We require all PRs to pass linting!
Recent Major Work¶
To give a sense of recent priorities, here are major milestones from the past few years:
- Federal Data & Committee Data - 2021
- API v3 - Q3 2020
- Legislation Tracking - Q1 2020
- Restoration of Historical Legislator Data - Q4 2019
- Full Text Search - Q4 2019
- 2019 Legislative Session Updates - Q1 2019
- OpenStates.org 2019 rewrite - Q1 2019
- OpenStates GraphQL API - Q4 2018
- Scraper Overhaul - Throughout much of 2017 we reworked our scrapers to be more resilient and to use an updated tech stack, replacing the one that powered the site from 2011-2016.