Skip to content

demographic_tools

Helpers for grouping ages and standardizing gender labels.

By default, ages are grouped into the population-QA buckets from "0-15" through "75+". Pass age_groups to use different age buckets.

demographic_tools._parse_age_group_label(label, sort_key)

Parse an age-band label into normalized bounds and display metadata.

Parameters:

Name Type Description Default
label str

Age group label, e.g. "0-15", "16-24", or "75+".

required
sort_key int

1-based position of the label in the configured age group list.

required

Returns:

Type Description
NormalizedAgeGroup

Tuple containing the inclusive lower bound, exclusive upper bound (or None for an open-ended group), original label, and sort key.

Raises:

Type Description
ValueError

If label is not a valid finite range or open-ended age-group label, or if its lower bound is greater than its upper bound.

demographic_tools._normalize_age_groups(age_groups)

Return parsed configured age groups or default population-QA age bands.

Parameters:

Name Type Description Default
age_groups Optional[Sequence[str]]

Ordered age group labels to normalize. If omitted, uses the default population-QA bands from "0-15" through "75+".

required

Returns:

Type Description
tuple[NormalizedAgeGroup, ...]

Parsed age groups containing their bounds, original labels, and one-based sort keys in the supplied order.

Raises:

Type Description
ValueError

If age_groups is empty or contains an invalid age-group label.

demographic_tools._age_condition(age, lower, upper)

Build a Spark predicate that selects one normalized age band.

Parameters:

Name Type Description Default
age Column

Spark column containing numeric ages.

required
lower int

Inclusive lower bound of the age band.

required
upper Optional[int]

Exclusive upper bound of the age band. Use None for an open-ended age band.

required

Returns:

Type Description
Column

Boolean Spark column that is true when an age falls within the band.

Examples:

A finite age band produces an inclusive lower and exclusive upper check:

>>> _age_condition(F.col("age"), lower=16, upper=25)
age >= 16 AND age < 25

An open-ended age band only checks its lower bound:

>>> _age_condition(F.col("age"), lower=75, upper=None)
age >= 75

demographic_tools._age_value_expr(age_col)

Build a numeric age expression from an age or age-band column.

Parameters:

Name Type Description Default
age_col str

Name of a column whose values are numeric ages or age-band labels beginning with an age, such as "65-74" or "75+".

required

Returns:

Type Description
Column

Integer Spark column containing the age or the leading age in each age-band label.

Examples:

Input: age = 24

Output: 24

Input: age = "16-24"

Output: 16

Input: age = "75+"

Output: 75

demographic_tools.age_group_expr(age_col='age', *, age_groups=None, unknown_label=None)

Build a Spark column expression for normalized age groups.

Parameters:

Name Type Description Default
age_col str

Name of the input age column.

'age'
age_groups Optional[Sequence[str]]

Optional custom age buckets. If omitted, population QA buckets are used. Labels must use formats like "0-15", "16-24", or "75+".

None
unknown_label Optional[str]

Label to use when age is null or does not match a bucket. If omitted, unmatched ages remain null.

None

Returns:

Type Description
Column

Spark column expression producing an age_group value.

demographic_tools.age_group_sort_key_expr(age_col='age', *, age_groups=None)

Build a Spark column expression for age-group sort ordering.

Parameters:

Name Type Description Default
age_col str

Name of the input age column.

'age'
age_groups Optional[Sequence[str]]

Optional custom age buckets. The 1-based position in the list is used as the sort key.

None

Returns:

Type Description
Column

Spark column expression producing an integer sort_key.

demographic_tools.gender_label_expr(gender_col='gender', *, scheme='numeric', unknown_label=None)

Build a Spark column expression for normalized gender labels.

Parameters:

Name Type Description Default
gender_col str

Name of the input gender column.

'gender'
scheme str

"numeric" maps 1 to "M" and 2 to "F". "numeric_or_text" also accepts "Male" and "Female".

'numeric'
unknown_label Optional[str]

Label to use for non-matching values. If omitted, non-matches become null.

None

Returns:

Type Description
Column

Spark column expression producing a gender_label value.

Raises:

Type Description
ValueError

If scheme is not supported.

demographic_tools.with_age_gender_columns(df, *, age_col='age', gender_col='gender', age_groups=None, gender_scheme='numeric', include_sort_key=False, unknown_label=None)

Add normalized demographic columns to a Spark DataFrame.

Parameters:

Name Type Description Default
df DataFrame

Input Spark DataFrame.

required
age_col str

Name of the input age column.

'age'
gender_col str

Name of the input gender column.

'gender'
age_groups Optional[Sequence[str]]

Optional custom age buckets. If omitted, population QA buckets are used. Labels must use formats like "0-15", "16-24", or "75+".

None
gender_scheme str

"numeric" for 1/2 values, or "numeric_or_text" to also accept "Male"/"Female".

'numeric'
include_sort_key bool

Add a sort_key column using the age group list order or explicit sort keys.

False
unknown_label Optional[str]

Label to use for null/unknown demographic values. If omitted, unknowns are null.

None

Returns:

Type Description
DataFrame

DataFrame with age_group and gender_label columns, and optionally sort_key.

Examples:

Population QA buckets:

with_age_gender_columns(df)

Visual-report buckets with chart ordering:

with_age_gender_columns(df, age_groups=["0-17", "18-24", "25-34", "75+"], gender_scheme="numeric_or_text", include_sort_key=True, unknown_label="Unknown")