Contributing

Contributing#

Contributing to Smart Health#

This document describes the processes and standards for contributing code to the Smart Health data pipeline.

Contribution Workflow#

The general steps for contributing code are:

  1. Clone the code repository into your development environment (see the Development Setup section below for prerequisites and installation steps).

  2. Create an issue that describes the user requirements, context, and possible solutions.

  3. Create a branch for that issue.

  4. Check out that branch.

  5. Write your code and unit tests.

  6. Run linters, formatters, and all unit tests locally.

  7. Commit your changes to your local branch.

  8. Push the changes to GitHub.

  9. Create a pull request (PR).

  10. Submit the PR for Code review using the GitHub PR review system.

  11. Merge PR after approval.

Development Setup#

The Smart Health pipeline is written primarily in Python and orchestrated using Snakemake.

Development Environment#

To contribute to the codebase, you require an environment that meets the prerequisites listed below. However, when working with real participant data, you must use a secure virtual machine (such as RONIN) within the Secure Data Environment (SDE) to ensure data privacy.

Prerequisites#

  • Git: For version control.

  • uv: The high-performance Python package installer.

  • GitHub Authentication: A Fine-Grained Personal Access Token (recommended) or an SSH key.

Installation Steps#

  1. Clone the Repository:

    git clone git@github.com:Data-Connect-UoS/bhf-smarthealth.git
cd BHF-SmartHealth

  2. Install Dependencies (Developer Mode): As a contributor, you need the full suite of testing, linting, and documentation tools. Use uv sync --all-groups to install both the core runtime dependencies and the development dependencies defined in the [dependency-groups].dev section of pyproject.toml.

  3. Install pre-commit hooks: There are some pre-commit hooks defined in .pre-commit-config.yaml that prevent common mistakes from being committed. To setup the pre-commits, run uv run pre-commit install. Every time that new pre-commit hooks get added, you need to run uv run pre-commit autoupdate so pre-commit can update the local configurations.

  4. Configure Git Filter (Notebooks): Run the following command to strip Notebook metadata and outputs during commits, which helps prevent merge conflicts.

    git config filter.strip-notebook-output.clean 'jupyter nbconvert --ClearOutputPreprocessor.enabled=True --ClearMetadataPreprocessor.enabled=True --to=notebook --stdin --stdout --log-level=ERROR'

Line Endings#

To maintain consistency across different operating systems, we encourage LF (Line Feed) line endings.

  • VS Code/IDE Config: If using an IDE like VS Code or PyCharm, ensure line ending conversion is set to LF (Unix/Linux style) when editing files.

  • Git Config: You can configure Git to handle line endings automatically. Consult the official Git documentation on Configuring Git to handle line endings.

Ignored Files (.gitignore)#

The .gitignore file defines which files and directories should be excluded from version control. This prevents accidental commits of build artifacts, sensitive data, and local environment files.

You must ensure that the following key files and directories are ignored:

  • Sensitive Credentials: secrets.yaml.

  • Local Outputs/Data: The data/ directory, which stores intermediate and log files as configured in config.yaml.

  • Environments: Virtual environment directories such as .venv, env/, and venv/.

  • Build Artifacts: The documentation build directory (docs/_build/), code distribution directory (dist/), and other build files.

  • Caches: Python caches (__pycache__/), Pytest cache (.pytest_cache/), and Ruff cache (.ruff_cache/).

  • Internal Pipeline Files: The .snakemake/ directory.

Coding Standards & Tools#

Code Organization#

The Python code is organized into a Python package (e.g., src/bhf-smarthealth) and uses Snakemake workflow files (.smk) for pipeline orchestration.

Continuous Integration (CI/CD) Actions#

The following actions are automatically performed by the CI/CD pipeline. Knowing these helps you focus on writing code, as the checks are enforced automatically.

CI/CD Action

Description

Trigger (When it runs)

Linting & Formatting

Runs Ruff to check code style and automatically format Python scripts in src/.

On Pull Request to main.

Unit Testing

Runs Pytest on the tests/ directory to ensure existing functionality is preserved.

On Pull Request to main.

Notebook Cleansing

Clears Jupyter Notebook outputs and metadata using jupyter nbconvert, and automatically commits the clean files.

On every push to any branch.

Documentation Build Check

Builds the Sphinx documentation (uv run make html) to check for rendering or syntax errors.

On Pull Request to main.

Documentation Deployment

Builds and deploys the generated documentation to GitHub Pages.

On push to main.

Linting and Formatting#

We use Ruff for fast linting and code formatting. Before committing code, please run the linter and formatter:

uv run ruff check src/ tests/ workflow/scripts/ --fix
uv run ruff format src/ tests/ workflow/scripts/

Buidling Documentation#

We use Sphinx with sphinx-autoapi to generate API reference documentation from docstrings in the Python scripts.

  • Please use Numpy Style Docstrings for all functions, classes, and modules.

Local Documentation Build Steps#

To build and view the documentation locally for testing:

  1. Install make: Ensure the build utility is installed.

    sudo apt-get install build-essential

  2. Build Static Pages: Move into the docs directory and execute make html. This generates static web pages from the Sphinx sources and outputs them to the docs/build/html directory.

    cd docs
uv run make html

  3. View Locally: You can view the site by either:

    • Opening the docs/build/html/index.html file directly in your browser.

    • Using Python’s built-in server module (run from the docs directory):

      # Serves the site at http://localhost:8000
uv run python -m http.server -d build/html

Logging#

For logging and printing status messages to assist with debugging pipeline runs, the Smart Health pipeline uses Python’s built-in logging module. Ensure messages are logged with appropriate levels of severity (e.g., INFO, WARNING, ERROR. See the logging documentation for more information).

For example:

 logger.info(f"Loaded Daily Summary data from {json_path}")
 logger.debug(f"Loaded Daily Summary data contains columns: {data_dict.keys()}")

Testing#

We run unit tests using the pytest framework to ensure new code does not break existing functionality.

Run Tests#

Before submitting a Pull Request, you must run all tests locally using pytest:

# Run all tests 
uv run pytest

For more complex pipeline testing, you should perform a dry-run of the relevant Snakemake workflow.

uv run snakemake -s workflow/<PipelineFile> --dryrun
On this page