bhf_smarthealth.parse.summary_parser

This module contains functions to process raw Fitbit daily summary data ingressed to the Smart Health pipeline

Attributes

logger

Functions

load_json(→ Dict[str, Any])	Load the raw JSON file, reshape it into a flat dictionary, and validate
validate_data(...)	Uses Pydantic model to convert and validate the dictionary containig the raw data.
create_dataframe(→ pandas.DataFrame)	Creates a dataframe from the Pydantic model with nested FitbitSummaryDatapoints under the daily activity fields. From these nested fields, the "value" is extracted.

Module Contents

bhf_smarthealth.parse.summary_parser.logger

bhf_smarthealth.parse.summary_parser.load_json(json_path: str | pathlib.Path) → Dict[str, Any]

Load the raw JSON file, reshape it into a flat dictionary, and validate data consistency against filename metadata.

The function extracts the Sheffield ID and date from the filename and compares the filename date against all date portions of timestamps found within the data for consistency. Files with inconsistent dates (either externally or internally inconsistent) will raise a ValueError.

Parameters:

json_path (str) – Path to the data file to be processed.

Returns:

A flat dictionary containing the raw, unvalidated Fitbit daily summary data, with sheffield_id and date appended

Return type:

Dict[str, Any]

Raises:

ValueError –

• If the filename date is known but data dates do not match it (External Consistency Failure).

• If the filename date is unknown (Mandatory Error).

bhf_smarthealth.parse.summary_parser.validate_data(data_dict: Dict[str, Any]) → bhf_smarthealth.validate.fitbit_schema.FitbitSummaryData

Uses Pydantic model to convert and validate the dictionary containig the raw data. This should raise an error if there is an issue.

Parameters:

data_dict (dict[str, str]) – Dictionary of the raw data

Returns:

Pydantic model class containing the converted data

Return type:

FitbitSummaryData

Raises:

DataValidationError – Raised if the pydantic schema validation fails

bhf_smarthealth.parse.summary_parser.create_dataframe(summary_data: bhf_smarthealth.validate.fitbit_schema.FitbitSummaryData) → pandas.DataFrame

Creates a dataframe from the Pydantic model with nested FitbitSummaryDatapoints under the daily activity fields. From these nested fields, the “value” is extracted.

Parameters:

summary_data (FitbitSummaryData) – Pydantic model class containing the converted data

Returns:

DataFrame containing data

Return type:

pd.DataFrame