API¶
Extraction¶
afx_audio.extract_all.extract_all_features(signal, sr, config, return_metadata=False)
¶
Extract all features specified in the config from the audio signal.
Args:
signal: Audio signal (1D np.ndarray)
sr: Sample rate
config: Config dict with a 'features' block mapping feature names to
extractor keyword arguments (or a list of names for default
parameters). Optional keys: 'aggregation'
(default 'mean'), 'flatten' (default True), 'normalize'
('zscore', 'minmax' or None), 'aggregation_params' (extra
keyword arguments for aggregate_features, e.g. window_size
and hop_length for 'sliding_window'), 'preserve_shape'
(True skips aggregation and returns per-frame arrays) and
'coverage' (True adds '
afx_audio.extract_all.extract_segments(signal, sr, config, segment_length, hop=None)
¶
One aggregated feature vector per fixed-length segment of a long signal.
The signal is cut into segments of segment_length seconds every
hop seconds (default: no overlap; a trailing partial segment is
dropped, and a signal shorter than one segment is a single segment),
and extract_all_features runs on each with config.
Returns:
| Type | Description |
|---|---|
(names, X, starts)
|
column names as in flatten_features, an array |
ndarray
|
of shape (n_segments, n_features), and each segment's start time |
ndarray
|
in seconds. |
afx_audio.registry.register_feature(name, func, *, units='unspecified', rows='1', overwrite=False)
¶
Make func available under name in configs, FEATURE_MAP and feature_info.
func(signal, sr, **params) must return {name: array} with the
per-frame values on the last axis (or a single value). It may accept
return_metadata and add {'metadata': {'times': ...}}; if it
does not, the argument is dropped for it. Its keyword parameters
become the feature's configurable parameters (any parameter is
accepted if it takes **kwargs). The batch extractor and
AFXFeatureExtractor pass registrations on to their worker processes,
which needs func to be picklable: defined at module level, not a
lambda or a nested function.
afx_audio.registry.unregister_feature(name)
¶
Remove a feature added with register_feature, restoring the built-in one it replaced.
afx_audio.batch_extractor.extract_features_from_folder(folder_path, config_path, output_path, file_exts=None, save_format=None, features=None, aggregation=None, flatten=None, normalize=None, n_jobs=None, sample_rate=None, verbose=False, coverage=None, hash_files=True, dataset_stats=False)
¶
Extract features from all audio files in a folder (recursively).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
folder_path
|
str
|
Root folder to search for audio files |
required |
config_path
|
str | dict[str, Any]
|
Path to a config JSON, or an already loaded config
dict (for example from |
required |
output_path
|
str
|
Path to save output (npy, json, csv, or parquet). Its folder must exist. |
required |
file_exts
|
Sequence[str] | None
|
File extensions to include, with or without the dot,
matched case-insensitively. None takes |
None
|
save_format
|
str | None
|
Output format ('npy', 'json', 'csv', 'parquet'); None infers it from the output_path extension. |
None
|
features
|
list[str] | None
|
Optional list of feature names to extract. Names missing from the config are added with default parameters; unknown names raise ValueError. |
None
|
aggregation
|
str | None
|
Aggregation method to use (validated) |
None
|
flatten
|
bool | None
|
Whether to flatten features after aggregation. None keeps the config file's setting (default True if unset there). |
None
|
normalize
|
str | None
|
Normalization method ('zscore', 'minmax', or None) |
None
|
n_jobs
|
int | None
|
Number of worker processes for parallel extraction.
None or 1 = serial; 0 or negative = all available cores.
Results are identical to serial extraction (same ordering).
Workers never fork: they start with forkserver where fork is
the default (Linux before Python 3.14), otherwise with the
method set by |
None
|
sample_rate
|
int | None
|
Override the config's sample_rate (audio is resampled to this rate on load). |
None
|
verbose
|
bool
|
Print per-file progress to stdout. Off by default; the
|
False
|
coverage
|
bool | None
|
Add a |
None
|
hash_files
|
bool
|
Record the sha256 of every audio file in the provenance record. Costs one extra read per file. |
True
|
dataset_stats
|
bool
|
Also write per-column count, mean, variance
(ddof=0), min, max and NaN count over the flattened feature
vectors to ' |
False
|
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
The results dict, keyed by path relative to folder_path. Files that |
dict[str, Any]
|
failed are collected under the 'errors' entry of the returned dict |
dict[str, Any]
|
and written to ' |
dict[str, Any]
|
main output). A sidecar this run does not write ('.errors.json' |
dict[str, Any]
|
with no failures, '.stats.json' without dataset_stats, and |
dict[str, Any]
|
'.provenance.json' for parquet) is removed if an earlier run to the |
dict[str, Any]
|
same path left one. |
Notes
- 'npy' saves a pickled dict; load it back with
np.load(path, allow_pickle=True).item(). Prefer 'json' for interchange with other tools; NaN is written as null there. - 'csv' and 'parquet' write one row per file with scalar columns;
vector features are expanded to
name_0, name_1, ...columns. 'parquet' requires pandas and pyarrow (pip install "afx-audio[parquet]") and is the recommended format for dataset-scale extraction. - Every output records how it was made (afx-audio, numpy and scipy
versions, the validated config, its hash and preset id, sample rate and
resampling, file hashes): in the parquet schema metadata under
the key 'afx', otherwise in '
.provenance.json'. Read it with afx_audio.read_provenance.
afx_audio.batch_extractor.default_extensions()
¶
File extensions read by default: those of every format the installed libsndfile reads.
For example .wav, .flac, .ogg, .oga, .opus, .mp3, .aif, .aiff, .au, .caf, .w64 and .rf64; .mp3 needs libsndfile 1.1 or later and .opus a build with Opus.
afx_audio.io.io.load_audio(path, sr=22050, offset=0.0, duration=None)
¶
Load an audio file as a mono float32 signal.
Channels are averaged. With sr set the audio is resampled with a
polyphase filter (scipy.signal.resample_poly); sr=None keeps the
file's rate. offset and duration (seconds, in the file's
timeline) read only part of the file. path may also be a file
object, as accepted by soundfile.
Returns:
| Type | Description |
|---|---|
tuple[ndarray, int]
|
(signal, sample_rate) |
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
the file does not exist. |
ValueError
|
no samples in the requested range, or a negative offset or non-positive duration. |
RuntimeError
|
soundfile cannot read the file. |
Configuration¶
afx_audio.utils.config_loader.load_config(config_path)
¶
Load and validate a JSON config file for feature extraction. Args: config_path: Path to config.json Returns: Config dictionary (see validate_config for the checks applied)
afx_audio.utils.config_loader.validate_config(config, require_sample_rate=True)
¶
Check a config dict and return it.
features may be a list of names as a shorthand for default
parameters, and sample_rate a float with an integral value; the
returned config is then a copy with the list expanded to {name: {}}
and the rate an int. Otherwise the config itself is returned.
Raises ValueError for unknown top-level keys, a missing (unless
require_sample_rate is False) or non-positive sample_rate, a
missing features block, feature names with no extractor, parameters
the extractor does not accept, non-positive frame sizes and counts, an
unknown aggregation or normalize value, sliding_window aggregation
without window_size and hop_length in aggregation_params, and a
non-boolean coverage or coverage with 'stack' or preserve_shape, and
a preset block other than {"name":
afx_audio.presets.load_preset(preset)
¶
Load a domain feature pack as a validated AFX config dict.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
preset
|
str
|
|
required |
Returns:
A config dict ready for extract_all_features /
extract_features_from_folder. The metadata keys (preset,
title, description) are kept and ignored by the extractors;
preset is what provenance records report as preset_id.
Raises:
ValueError: If the preset name or version is unknown.
afx_audio.presets.resolve_preset(preset)
¶
The exact name@version id for 'name' (latest version) or 'name@version'.
Raises ValueError for an unknown name or version, listing what exists.
afx_audio.presets.list_presets(*, versions=False)
¶
Sorted preset names, or every name@version id when versions is True.
afx_audio.presets.preset_info(preset)
¶
Id, name, version, title, description, sample rate and feature names of a preset.
afx_audio.registry.feature_info(name)
¶
Group, units, output rows, parameters with defaults and a one-line summary for a feature.
rows is the size of the first axis of the per-frame output ('1'
means the output is 1-D, one value per frame). undefined says when
a frame is NaN, None if never (or unknown, for custom features).
torch is None without a torch implementation, else the
afx_audio.torch.functional function and the rtol and atol to which
its output is tested equal to the NumPy extractor in float64.
afx_audio.registry.list_features(group=None)
¶
Feature names, optionally only those of one group (see GROUPS).
afx_audio.utils.batch_utils.flatten_features(features)
¶
(column names, 1-D float vector) for a dict of aggregated features.
The single array from aggregation='stack' gives names stack_0, stack_1, ...
afx_audio.utils.batch_utils.feature_names(features)
¶
Column names for features_to_vector(features): name for a
single value, name_0, name_1, ... (C order) for anything larger.
egemaps and egemaps_lld rows carry openSMILE's names
(EGEMAPS_NAMES, EGEMAPS_LLD_NAMES) in place of the index:
<row> for one value per row ('mean', 'std', 'skew', 'kurtosis'),
<row>_<stat> for the keys of 'summary' and 'advanced_summary',
and <row>_<j> for several values per row (windows of
'sliding_window', frames of preserve_shape).
Aggregation¶
afx_audio.utils.aggregator.aggregate_features(features, method='mean', flatten=True, normalize=None, window_size=None, hop_length=None, norm_stats=None, sliding_agg='mean', coverage=False)
¶
Aggregate features using the specified method, then optionally flatten and normalize each feature.
Args:
features: Dict of feature arrays
method: Aggregation method ('mean', 'std', 'summary', 'advanced_summary',
'skew', 'kurtosis', 'sliding_window', 'stack')
flatten: Whether to flatten each feature to 1D (ignored if method='stack')
normalize: Normalization method ('zscore', 'minmax', or None)
window_size: Window size for sliding window aggregation (required if method='sliding_window')
hop_length: Hop length for sliding window aggregation (required if method='sliding_window')
norm_stats: Optional precomputed dataset-level statistics per feature
key. For 'zscore': {'mean': {feat: m, ...}, 'std': {feat: s, ...}};
for 'minmax': {'min': {...}, 'max': {...}}.
sliding_agg: Inner aggregation for method='sliding_window'
('mean', 'std', 'max', 'min').
coverage: Add <name>_valid_frac after each feature's statistics,
the fraction of frames they are based on (see valid_frame_mask).
One value per clip, or one per window with 'sliding_window'.
Never normalized. Not available with 'stack'.
Returns:
Dict of aggregated (and optionally flattened/normalized) features, or a stacked numpy array if method='stack'.
Note on normalization
Without norm_stats, normalization is computed within this call
(i.e., per file, across the values of each feature). That rescales
every file by its own statistics and is NOT comparable across files.
For ML pipelines, compute mean/std (or min/max) per feature across
the whole dataset and pass them via norm_stats.
afx_audio.utils.aggregator.valid_fraction(values)
¶
Fraction of frames on which every row of values is finite, see valid_frame_mask.
eGeMAPS¶
afx_audio.extractors.egemaps.egemaps(signal, sr)
¶
The 88 eGeMAPS v02 functionals as {name: value}, in the order of EGEMAPS_NAMES.
Provenance and dataset statistics¶
afx_audio.provenance.config_hash(config)
¶
First 12 hex characters of the sha256 of computed_config(config) as canonical JSON.
Key order, labels, flatten and parameters written at their default
value do not change the hash, so a preset, the same preset copied into
a file, and the config the scikit-learn transformer builds from it all
hash alike.
afx_audio.provenance.read_provenance(path)
¶
Provenance record of a batch output.
path is the output itself or its .provenance.json sidecar. A
parquet file, recognised by its content and not by its extension, is
read from its schema metadata; any other output from the sidecar next
to it. Reading parquet needs pyarrow (the parquet extra).
afx_audio.dataset_stats.load_stats(path)
¶
Dataset statistics written by extract_features_from_folder(..., dataset_stats=True).
path is the batch output or its .stats.json file. Returns the
record with count and nan_count as int64 arrays and mean,
var, std, min, max as float64 arrays (NaN where a column
had no non-NaN value), aligned with columns.
afx_audio.sklearn.scaler_from_stats(stats, columns=None)
¶
A fitted StandardScaler from afx_audio.load_stats output.
Equivalent to StandardScaler().fit(X) on the matrix the statistics
came from (NaN ignored in fit, as sklearn does), without loading that
matrix. columns selects and orders a subset of stats['columns'];
the scaler then expects data with those columns in that order. A
column with no non-NaN value has no mean or scale, so it raises
ValueError unless left out through columns.
Extractors¶
afx_audio.extractors.time_domain
¶
Time-domain feature extractors.
Every extractor takes frame_size, hop_length and center. With
both sizes set the signal is cut into frames of frame_size samples
every hop_length samples (trailing samples that do not fill a frame are
dropped) and the result has shape (n_frames,). center=True (the
default) first zero-pads frame_size // 2 samples on both sides, so frame i
is centred on sample i * hop_length as in librosa and in the centered STFT.
With either
size left as None, or an uncentered clip shorter than one frame, the whole
signal is analyzed as a single unpadded frame and the result has shape (1,).
Features are computed for all frames at once on a (n_frames, frame_size) strided view, in float64.
extract_zero_crossing_rate(signal, sr=None, *, frame_size=2048, hop_length=512, center=True, threshold=1e-10, return_metadata=False)
¶
Zero-crossing rate: sign changes between consecutive samples divided by the frame length.
Samples with |x| <= threshold count as zero and zero counts as positive, so low-level noise around silence does not register as crossings. Equal to librosa.feature.zero_crossing_rate with the same center (centered frames are padded by repeating the edge samples, as librosa does). Defaults to 2048/512 framing; pass None for one value over the whole clip.
extract_variance(signal, sr=None, *, frame_size=None, hop_length=None, center=True, return_metadata=False)
¶
Variance of the samples (population, ddof=0).
extract_entropy(signal, sr=None, *, frame_size=None, hop_length=None, center=True, num_bins=100, return_metadata=False)
¶
Shannon entropy (bits) of the amplitude histogram with num_bins equal-width bins.
extract_crest_factor(signal, sr=None, *, frame_size=None, hop_length=None, center=True, return_metadata=False)
¶
Peak amplitude over RMS. NaN for an all-zero frame.
extract_kurtosis(signal, sr=None, *, frame_size=None, hop_length=None, center=True, fisher=True, return_metadata=False)
¶
Sample kurtosis m4 / m2^2 (biased moments), minus 3 when fisher.
Matches scipy.stats.kurtosis, including NaN for frames whose variance is zero to machine precision.
extract_short_time_energy(signal, sr=None, *, frame_size=None, hop_length=None, center=True, return_metadata=False)
¶
Sum of squared samples.
extract_rms_energy(signal, sr=None, *, frame_size=None, hop_length=None, center=True, return_metadata=False)
¶
Root mean square of the samples.
extract_energy_ratio(signal, sr=None, *, frame_size=None, hop_length=None, center=True, split=0.5, return_metadata=False)
¶
Energy before split (fraction of the frame length) over energy after it plus 1e-10. NaN for silence.
extract_mobility(signal, sr=None, *, frame_size=None, hop_length=None, center=True, return_metadata=False)
¶
Hjorth mobility, RMS(x') / RMS(x) with x' the first difference.
Hjorth (1970), "EEG analysis based on time domain properties". RMS is used instead of standard deviation, so a DC offset lowers the value. 1e-10 is added to the denominator; NaN for an all-zero frame.
extract_complexity(signal, sr=None, *, frame_size=None, hop_length=None, center=True, return_metadata=False)
¶
RMS(x'') / (RMS(x') + 1e-10), the mobility of x'. NaN for a constant frame, silence included.
extract_sample_entropy(signal, sr=None, *, m=2, r=0.2, max_samples=20000)
¶
Sample entropy -ln(A/B) (Richman and Moorman, 2000, Am. J. Physiol. 278(6)).
B counts ordered pairs i != j of the first N - m templates of length m
within Chebyshev distance r * std(signal), A the same for the N - m
templates of length m + 1, so both counts use the same template set.
Candidate pairs come from sorting the templates by their first sample;
when there are too many (long or near-constant frames) a KD-tree counts
instead. The worst case is still quadratic, so use it frame-wise and
keep frames below max_samples. NaN when A or B is 0 or the frame
has fewer than m + 2 samples.
extract_autocorrelation_variance(signal, sr=None, *, frame_size=None, hop_length=None, center=True, max_lag=100, return_metadata=False)
¶
Variance of the raw autocorrelation over lags 0..max_lag-1.
extract_autocorr_snr(signal, sr=None, *, frame_size=None, hop_length=None, center=True, max_lag=100, return_metadata=False)
¶
Peak over mean (plus 1e-10) of the autocorrelation over lags 0..max_lag-1. NaN for an all-zero frame.
extract_autocorr_zero_crossings(signal, sr=None, *, frame_size=None, hop_length=None, center=True, max_lag=100, return_metadata=False)
¶
Number of sign changes of the autocorrelation over lags 0..max_lag-1.
extract_autocorr_peak_width(signal, sr=None, *, frame_size=None, hop_length=None, center=True, max_lag=100, return_metadata=False)
¶
Lags from the first to the last autocorrelation value at or above half the peak, inclusive.
NaN for an all-zero frame, whose autocorrelation has no peak.
extract_autocorr_decay_rate(signal, sr=None, *, frame_size=None, hop_length=None, center=True, max_lag=100, return_metadata=False)
¶
Negative least-squares slope of log(|acf| + 1e-10) over lags 1..max_lag-1.
NaN with fewer than 3 lags and for an all-zero frame, where the slope would only be that of the 1e-10 floor.
extract_autocorr_coeffs_deviation(signal, sr=None, *, frame_size=None, hop_length=None, center=True, max_lag=100, return_metadata=False)
¶
Standard deviation of the autocorrelation over lags 0..max_lag-1.
extract_max_peak_autocorr_envelope(signal, sr=None, *, frame_size=None, hop_length=None, center=True, max_lag=100, return_metadata=False)
¶
Largest autocorrelation value over lags 1..max_lag-1. NaN when only lag 0 exists.
afx_audio.extractors.frequency_domain
¶
Spectral shape descriptors computed from the STFT magnitude.
With frame_size and hop_length set (the default) every extractor
returns one value per STFT frame, shape (n_frames,), from the shared and
memoized STFT in afx_audio.methods.stft (periodic Hann window, n_fft =
frame_size, zero padding of frame_size // 2 on both sides when
center). Centroid, bandwidth, rolloff, skewness, kurtosis, slope, crest,
decrease, entropy and flatness also accept frame_size=None or hop_length=None to use one
Hann-windowed FFT of the whole clip, shape (1,).
extract_spectral_centroid(signal, sr, *, frame_size=2048, hop_length=512, center=True, return_metadata=False)
¶
Magnitude-weighted mean frequency in Hz, as librosa.feature.spectral_centroid. 0 for a silent frame.
extract_spectral_bandwidth(signal, sr, *, frame_size=2048, hop_length=512, center=True, return_metadata=False)
¶
Magnitude-weighted standard deviation of frequency around the centroid, in Hz (librosa, p=2).
extract_spectral_rolloff(signal, sr, *, frame_size=2048, hop_length=512, roll_percent=0.85, center=True, power=1.0, return_metadata=False)
¶
Spectral rolloff: the lowest bin frequency at which the cumulative sum
of |X|^power reaches roll_percent of the frame total.
power=1 sums magnitudes, as librosa.feature.spectral_rolloff does
(identical output for the same framing); power=2 sums energy.
0 Hz for an all-zero frame.
extract_spectral_skewness(signal, sr, *, frame_size=2048, hop_length=512, center=True, return_metadata=False)
¶
Third standardized moment of frequency under the magnitude spectrum. NaN for a silent or single-bin frame.
extract_spectral_slope(signal, sr, *, frame_size=2048, hop_length=512, center=True, return_metadata=False)
¶
Least-squares slope of 20 log10(|X| + 1e-10) against frequency, in dB per Hz.
NaN for a silent frame, where the slope would only be that of the 1e-10 floor, and for a single-bin spectrum.
extract_spectral_kurtosis(signal, sr, *, frame_size=2048, hop_length=512, center=True, fisher=True, return_metadata=False)
¶
Fourth standardized moment of frequency under the magnitude spectrum, minus 3 when fisher.
The kurtosis of the spectrum seen as a distribution over frequency
(Peeters, 2004, "A large set of audio features for sound
description"), not the time-domain spectral kurtosis used for fault
detection. 0 (or 3 without fisher) is the value of a Gaussian
shape; NaN for a silent or single-bin frame.
extract_spectral_crest(signal, sr, *, frame_size=2048, hop_length=512, center=True, return_metadata=False)
¶
Largest over mean magnitude of the spectrum, per frame.
1 for a flat spectrum; a sinusoid gives about frame_size / 4 with the Hann window (its energy spreads over three bins). NaN for a silent frame.
extract_spectral_decrease(signal, sr, *, frame_size=2048, hop_length=512, center=True, return_metadata=False)
¶
Spectral decrease (Peeters, 2004): sum_k (a_k - a_1) / (k - 1) / sum_k a_k over bins k = 2..K.
a_k is the magnitude of bin k, with a_1 the DC bin, as in the Timbre Toolbox and Lerch's pyACA. Negative when the spectrum rises above its first bin on average. NaN when bins 2..K carry no energy (silence) or there is a single bin.
extract_spectral_entropy(signal, sr, *, frame_size=2048, hop_length=512, center=True, return_metadata=False)
¶
Shannon entropy in bits of the magnitude spectrum normalized to sum 1.
At most log2(frame_size // 2 + 1) for a flat spectrum, near 0 for a single tone. NaN for a silent frame (magnitude sum at most 1e-10), which has no distribution to normalize.
extract_spectral_flatness(signal, sr, *, frame_size=2048, hop_length=512, center=True, return_metadata=False)
¶
Geometric over arithmetic mean of the power spectrum (Wiener entropy).
Near 1 for white noise, near 0 for tones. Power is floored at 1e-10, as librosa.feature.spectral_flatness does with amin=1e-10, so silent frames give 1.
extract_spectral_contrast(signal, sr, *, frame_size=2048, hop_length=512, n_bands=6, fmin=200.0, quantile=0.02, center=True, return_metadata=False)
¶
Spectral contrast (Jiang et al., 2002, Proc. IEEE ICME), shape (n_bands + 1, n_frames).
The magnitude spectrum is split into octave bands starting at fmin
(the first band is 0..fmin); in each band and frame, peak and valley
are the means of the top and bottom quantile of magnitudes, and
contrast is power_to_db(peak) - power_to_db(valley) with amin 1e-10 and
top_db 80. This follows librosa.feature.spectral_contrast exactly,
including its use of 10 log10 on magnitudes, so values are half the
20 log10 magnitude ratio.
extract_spectral_flux(signal, sr, *, frame_size=2048, hop_length=512, center=True, return_metadata=False)
¶
Euclidean distance between consecutive magnitude spectra, shape (n_frames - 1,).
Centering pads by reflection here (constant padding elsewhere). A clip with a single frame gives [0.0]. flux[i] compares frames i and i + 1 and carries the time of frame i + 1.
afx_audio.extractors.cepstral_features
¶
Spectrogram and cepstral extractors: mel spectrogram, MFCC and deltas, GFCC, CQT and chroma.
All of them start from the shared STFT in afx_audio.methods.stft (periodic Hann
window, frame_size = n_fft) and return (n_dims, n_frames) arrays. With
center=True the signal is zero-padded by frame_size // 2 on both sides
first, as in librosa.
clear_feature_cache()
¶
Drop every cached filterbank.
get_cache_stats()
¶
Number of cached items, in total and per kind.
extract_mfcc(signal, sr, *, n_mfcc=13, frame_size=2048, hop_length=512, n_mels=128, fmin=0.0, fmax=None, center=True, top_db=80.0, return_metadata=False)
¶
Mel-frequency cepstral coefficients, shape (n_mfcc, n_frames).
Power spectrogram -> Slaney mel filterbank (n_mels bands, fmin..fmax,
fmax defaults to sr / 2) -> 10 log10 clamped at 1e-10 and floored
top_db below the clip's maximum -> orthonormal DCT-II (Davis and
Mermelstein, 1980, IEEE Trans. ASSP 28(4)). This is
librosa.feature.mfcc with the same n_fft, hop_length and center.
extract_mfcc_delta(signal, sr, *, n_mfcc=13, frame_size=2048, hop_length=512, n_mels=128, fmin=0.0, fmax=None, center=True, width=9, order=1, return_metadata=False)
¶
MFCC deltas, shape (n_mfcc, n_frames), key 'mfcc_delta'.
Savitzky-Golay derivative over width frames (clipped to the largest
odd value not above n_frames) with polynomial fits at the edges, as
librosa.feature.delta computes it; in the interior, order 1 is the
regression d[t] = sum_n n x[t+n] / sum_n n^2. order is the
derivative order (2 = acceleration). All zeros when fewer than 3 frames
are available.
extract_mfcc_delta_delta(signal, sr, *, n_mfcc=13, frame_size=2048, hop_length=512, n_mels=128, fmin=0.0, fmax=None, center=True, width=9, return_metadata=False)
¶
Second-order MFCC deltas (acceleration), see extract_mfcc_delta.
extract_gfcc(signal, sr, *, n_gfcc=13, frame_size=2048, hop_length=512, n_gammatone=64, fmin=50.0, fmax=None, center=True, return_metadata=False)
¶
Gammatone frequency cepstral coefficients, shape (n_gfcc, n_frames).
The MFCC pipeline with the mel filterbank replaced by n_gammatone
4th-order gammatone magnitude responses on the ERB-rate scale
(Glasberg and Moore, 1990), each normalized to unit area. Shao and
Wang (2008, Proc. ICASSP) filter in the time domain and compress with
a cube root; this variant works on the STFT and uses dB, so the values
are not interchangeable with theirs.
extract_chroma_cqt(signal, sr, *, frame_size=2048, hop_length=512, n_bins=84, bins_per_octave=12, fmin=C1_HZ, center=True, return_metadata=False)
¶
Chromagram from the STFT-based CQT approximation, shape (12, n_frames).
Each CQT bin (see extract_cqt) is assigned to pitch class round(69 + 12 log2(f / 440)) mod 12 (0 = C) and summed over octaves; each frame is L1-normalized.
extract_chroma_stft(signal, sr, *, frame_size=2048, hop_length=512, center=True, tuning=0.0, normalize=True, return_metadata=False)
¶
Chromagram from the STFT power spectrum, shape (12, n_frames), row 0 = C.
Power spectrogram projected on librosa's chroma filterbank (Gaussian
pitch-class weights per bin, Gaussian octave weighting centred on 880 Hz),
each frame scaled to a maximum of 1 when normalize. Identical to
librosa.feature.chroma_stft(..., tuning=tuning); librosa estimates the
tuning when it is not given, AFX uses tuning (fractions of a
semitone from A440) as is.
extract_cqt(signal, sr, *, hop_length=512, n_bins=84, bins_per_octave=12, fmin=None, frame_size=2048, center=True, n_fft=None, return_metadata=False)
¶
Constant-Q magnitude approximated from the STFT, shape (n_bins, n_frames).
Bin k is centered at fmin * 2^(k / bins_per_octave) (fmin defaults to
C1, 32.7 Hz) and is a triangular, L1-normalized weighting of the STFT
magnitudes between its neighbours. This is not a true constant-Q
transform: the time resolution is that of the frame_size STFT at
every frequency, and low bins narrower than the STFT bin spacing fall
back to the nearest STFT bin. Only the bin geometry follows Brown (1991,
J. Acoust. Soc. Am. 89(1)); the per-bin window length does not.
n_fft is an alias for frame_size.
extract_melspectrogram(signal, sr, *, n_mels=128, frame_size=2048, hop_length=512, fmin=0.0, fmax=None, center=True, return_metadata=False)
¶
Mel power spectrogram, shape (n_mels, n_frames).
|STFT|^2 projected on a Slaney-scale, Slaney-normalized triangular filterbank (librosa.feature.melspectrogram with power=2 and htk=False). Values are linear power, not dB.
extract_tonnetz(signal, sr, *, frame_size=2048, hop_length=512, center=True, tuning=0.0, return_metadata=False)
¶
Tonal centroid features (Harte, Sandler and Gasser, 2006), shape (6, n_frames).
The L1-normalized chroma_stft is projected on the circles of fifths,
minor thirds and major thirds (x and y of each), as
librosa.feature.tonnetz does; librosa defaults to chroma_cqt as input,
so pass it chroma=librosa.feature.chroma_stft(..., tuning=0) to
compare.
extract_onset_strength(signal, sr, *, frame_size=2048, hop_length=512, n_mels=128, fmin=0.0, fmax=None, lag=1, max_size=1, return_metadata=False)
¶
Spectral-flux onset strength, shape (n_frames,), as librosa.onset.onset_strength.
The centered mel power spectrogram in dB (ref 1, top_db 80) is
differenced over lag frames against a max_size-bin maximum
filter of itself along frequency, negative changes are dropped and the
bands averaged. With max_size > 1 this is the SuperFlux vibrato
suppression of Böck and Widmer (2013, Proc. DAFx-13). The envelope is shifted by lag + frame_size // (2 *
hop_length) frames so peaks line up with onsets in time, and trimmed
to the number of mel frames.
afx_audio.extractors.harmonic_features
¶
Harmonic feature extractors: pitch (YIN), total harmonic distortion and harmonics-to-noise ratio.
extract_pitch(signal, sr, *, frame_size=2048, hop_length=512, center=True, fmin=50.0, fmax=2000.0, threshold=0.1, return_metadata=False)
¶
Fundamental frequency in Hz per frame by YIN, NaN for unvoiced frames.
A frame is unvoiced when the cumulative mean normalized difference has
no dip below threshold and its minimum is not below 0.5, or when
the estimate falls outside [fmin, fmax] (see afx_audio.utils.pitch.yin,
which returns 0 for these frames). With return_metadata=True,
metadata['voiced'] is the boolean voicing mask. The longest
detectable period is frame_size // 2 samples, so fmin is effectively
max(fmin, 2 sr / frame_size). center zero-pads frame_size // 2
samples on both sides first, as librosa.yin does. Shape (n_frames,).
extract_thd(signal, sr, *, frame_size=None, hop_length=None, center=True, n_harmonics=4, return_metadata=False)
¶
Total harmonic distortion sqrt(sum_h A_h^2) / A_1, as a ratio.
A_1 is the largest non-DC bin of the unwindowed FFT, A_h the largest bin within +/-2 bins of h times its index, for h = 2..n_harmonics + 1 below Nyquist. Assumes the strongest component is the fundamental, so it is meant for tonal signals (test tones, machinery hum). NaN for frames with fewer than 2 bins or no energy. Framing as in the time-domain extractors; shape (n_frames,).
extract_hnr(signal, sr, *, frame_size=2048, hop_length=512, center=True, fmin=50.0, fmax=2000.0, return_metadata=False)
¶
Harmonics-to-noise ratio in dB per frame, 10 log10(r / (1 - r)).
The dB mapping is that of Boersma (1993, Proc. Institute of Phonetic Sciences 17); the correlation differs from his window-corrected autocorrelation. r is the maximum over lags sr / fmax .. sr / fmin of the normalized cross-correlation of the two segments overlapping at each lag (the NCCF of Talkin, 1995, "A robust algorithm for pitch tracking"):
r(tau) = sum_j x[j] x[j+tau] / sqrt(sum_j x[j]^2 * sum_j x[j+tau]^2)
for j = 0..N-tau-1. It is 1 for a perfectly periodic frame. The biased estimate R(tau) / R(0) would cap r at (N - tau) / N, about 2 dB HNR for a pure tone with a 400-sample frame and a 160-sample period.
The longest lag searched is frame_size // 2, as in YIN, so every lag keeps at least half the frame in the overlap and fmin is effectively max(fmin, 2 sr / frame_size). Longer lags compare so few samples that white noise reads as harmonic (r = 1 at a single-sample overlap).
r does not depend on signal level; it is clipped to (1e-10, 1 - 1e-10),
so HNR lies within +/-100 dB. NaN for silent frames (zero energy). center
zero-pads frame_size // 2 samples on both sides first; a clip still
shorter than one frame is zero-padded to one frame. Shape (n_frames,).
afx_audio.extractors.linear_prediction
¶
Linear-prediction features: LPC coefficients and formant frequencies.
extract_lpc(signal, sr=None, *, order=12, frame_size=None, hop_length=None, center=True, window='hamming', pre_emphasis=0.0, return_metadata=False)
¶
Linear prediction coefficients a_1..a_p per frame, shape (order, n_frames).
Autocorrelation method with Levinson-Durbin recursion on each
window-weighted frame (symmetric window, None for rectangular),
after optional pre-emphasis x[n] - pre_emphasis * x[n-1]. A(z) =
1 + a_1 z^-1 + ... + a_p z^-p, the sign convention of librosa.lpc
(which uses Burg's method, so values differ slightly). Framing as in
the time-domain extractors; silent frames give zeros, frames not
longer than order give NaN.
extract_formants(signal, sr, *, n_formants=3, frame_size=512, hop_length=128, center=True, order=None, pre_emphasis=0.97, fmin=90.0, max_bandwidth=400.0, return_metadata=False)
¶
Formant frequencies F1..Fn in Hz per frame from LPC roots, shape (n_formants, n_frames).
Each pre-emphasized, Hamming-windowed frame is fitted with an LPC
polynomial of order (default 2 + sr // 1000). Roots in the upper
half plane give resonances at f = angle * sr / (2 pi) with bandwidth
-sr / pi * ln|z|; those above fmin and narrower than
max_bandwidth are sorted by frequency and the lowest n_formants
kept. NaN where a frame has fewer candidates (silence, unvoiced
sounds). The classic method of Markel and Gray (1976); for speech use
frames of 20 to 30 ms and resample to about 10 kHz or 2 kHz per
expected formant, since high orders also fit spurious peaks.
Signal processing building blocks¶
afx_audio.methods.stft.stft(signal, frame_size=2048, hop_length=512, window='hann', center=True, pad_mode='constant')
¶
Complex STFT, shape (frame_size // 2 + 1, n_frames), as librosa.stft with n_fft = win_length = frame_size.
window is any scipy.signal.get_window name, used in its periodic
form, or None for rectangular. center pads frame_size // 2 samples
on both sides with pad_mode ('constant' as in librosa >= 0.10;
'reflect' falls back to zeros for signals shorter than the pad). A
signal shorter than one frame is zero-padded to one frame. float32
input gives complex64 output (computed in double precision). The
result is read-only because it is shared through the cache.
afx_audio.methods.mel.mel_filterbank(n_mels, n_fft, sr=22050, fmin=0.0, fmax=None, norm='slaney', htk=False)
¶
Triangular mel filterbank, shape (n_mels, n_fft // 2 + 1), equal to librosa.filters.mel.
Filter m rises from edge m to center m + 1 and falls to edge m + 2 of
mel_frequencies. norm='slaney' divides each filter by half its
width in Hz so every band has the same area; norm=None leaves a
peak of 1. Warns, like librosa, when a filter falls between FFT bins
and is all zero.
afx_audio.methods.chroma.chroma_filterbank(sr, n_fft, n_chroma=12, tuning=0.0, ctroct=5.0, octwidth=2.0)
¶
STFT-to-chroma weights, shape (n_chroma, n_fft // 2 + 1), equal to librosa.filters.chroma.
Every FFT bin spreads over neighbouring pitch classes with a Gaussian
whose width is the bin's spacing in chroma units (so low bins are
broad), each bin's weights are L2-normalized, then scaled by a Gaussian
over octaves centred ctroct octaves above 27.5 Hz (5 -> 880 Hz) with
octwidth octaves of standard deviation. Row 0 is C. tuning is
the deviation from A440 in fractions of a chroma bin.
afx_audio.methods.delta.compute_delta(features, width=9, order=1)
¶
Delta features along the last axis, same as librosa.feature.delta.
A Savitzky-Golay filter of length width and polynomial order
order returns the order-th derivative; in the interior this is
the regression formula of delta_kernel for order 1. Edges are fitted
with the same polynomial (mode='interp') rather than padded.
width is clipped to the largest odd value not above n_frames.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
features
|
ndarray
|
(n_features, n_frames) |
required |
width
|
int
|
odd window length, at least 3 |
9
|
order
|
int
|
1 for delta, 2 for delta-delta |
1
|
afx_audio.methods.lpc.lpc(frames, order)
¶
LPC polynomials of each row of frames by Levinson-Durbin recursion.
Returns (a, err): a has shape (n_frames, order + 1) with a[:, 0] = 1,
the coefficients of A(z) = 1 + a_1 z^-1 + ... + a_p z^-p that minimize
the prediction error of x[n] + sum_k a_k x[n-k] (the convention of
librosa.lpc, which uses Burg's method instead); err is the final
prediction error power. Frames are used as given, so window them first.
A silent frame gives A(z) = 1.
afx_audio.utils.pitch.yin(y, frame_length, hop_length, fmin=50.0, fmax=2000.0, sr=22050, threshold=0.1)
¶
Frame-wise f0 in Hz, 0 for frames judged unvoiced.
Frames are taken without padding. The lag search runs over
[sr / fmax, min(sr / fmin, frame_length / 2)]. In each frame the first
dip of the cumulative mean normalized difference below threshold is
followed to its local minimum (steps 3-4 of the paper); without such a
dip the global minimum is used if it is below 0.5. The lag is refined by
parabolic interpolation, and estimates outside [fmin, fmax] are set to 0.
Returns an array of shape (n_frames,); a signal shorter than one frame gives [0.0].
scikit-learn¶
afx_audio.sklearn.AFXFeatureExtractor
¶
Bases: BaseEstimator, TransformerMixin
Extract AFX audio features as fixed-length vectors, sklearn-style.
Each input sample is either a path to an audio file (str / os.PathLike,
loaded and resampled to sample_rate) or a raw 1-D signal
(assumed to already be at sample_rate). transform returns a
2-D array of shape (n_samples, n_features_out).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
config
|
dict[str, Any] | str | None
|
Config dict ( |
None
|
features
|
list[str] | dict[str, dict[str, Any]] | None
|
Feature names to extract, in output order (names missing from the config get default parameters), or a dict of names to parameters that replaces the config's features block. |
None
|
sample_rate
|
int | None
|
Target sample rate for loading/assuming signals. None takes the config's sample_rate (22050 if it has none). |
None
|
aggregation
|
str | None
|
Temporal aggregation per file ('mean', 'std', 'summary', 'advanced_summary', 'skew', 'kurtosis', 'stack'). None takes the config's aggregation ('mean' if it has none). |
None
|
normalize
|
str | None
|
Per-sample normalization applied by AFX ('zscore' or
'minmax'). None takes the config's normalize (none if it has
none). This normalizes each sample's feature vector
independently; for train/test consistent scaling, leave it
unset in both and add a |
None
|
n_jobs
|
int | None
|
Parallel extraction over samples via joblib (None/1 = serial, -1 = all cores). |
None
|
coverage
|
bool | None
|
Add a |
None
|
Attributes (after fit):
config_: The resolved config actually used for extraction.
feature_names_out_: Expanded output column names (vector features
become name_0, name_1, ...).
n_features_out_: Output dimensionality.
provenance_: afx-audio, numpy, scipy and Python versions, the
resolved config and its afx_audio.config_hash, the preset
id, the sample rate, the registered custom features the config
uses and the fit time, as written next to batch outputs.
PyTorch¶
afx_audio.torch.module.AFXFeatureModule
¶
Bases: Module
Extract a configurable set of AFX features as torch tensors.
Mirrors the extract_all_features config convention: features
maps feature names to parameter dicts. Constant tensors (Hann windows,
mel, gammatone and chroma filterbanks, DCT matrices) are precomputed in
__init__ and registered as buffers, so the module moves with
.to(device) and builds no filterbank per forward. The buffers are
stored in float64 and cast to the input dtype in forward: the
output has the dtype of the input, float64 input gets float64
constants, and float32 input keeps float32 math. .float() or
.half() on the module lowers the stored precision.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
features
|
dict[str, dict[str, Any]] | list[str]
|
Dict like |
required |
sample_rate
|
int
|
Audio sample rate the module expects. |
22050
|
aggregation
|
str
|
'none' (keep per-frame values), 'mean', 'std', or 'mean_std' (both, concatenated; 'summary' is accepted as on the NumPy path) over the time axis. |
'none'
|
output
|
str
|
'dict' returns |
'dict'
|
coverage
|
bool
|
Add |
False
|
Example::
module = AFXFeatureModule(
features={'mfcc': {'n_mfcc': 13}, 'spectral_centroid': {}},
sample_rate=16000, aggregation='mean', output='vector',
).to('cuda')
feats = module(waveforms) # (batch, dim), differentiable
from_config(config, *, output='dict', skip_unsupported=False)
classmethod
¶
Build the module from an AFX config, as the NumPy path reads it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
config
|
dict[str, Any] | str | PathLike
|
Config dict, path to a config JSON, or a preset ('speech' or 'speech@1', ...). |
required |
output
|
str
|
'dict' or 'vector', as in the constructor. |
'dict'
|
skip_unsupported
|
bool
|
Drop features without a torch implementation with an AFXWarning naming them, instead of raising. |
False
|
The torch-supported features keep their parameters. sample_rate
(22050 if absent), aggregation ('mean' if absent) and
coverage are taken from the config; preserve_shape maps to
aggregation 'none'. Aggregations other than 'mean', 'std' and
'summary', and normalize, have no torch equivalent and raise
ValueError, as do unsupported features unless skip_unsupported.
feature_names()
¶
Configured feature names, in output order.
get_feature_names_out()
¶
Column names of the output='vector' tensor, named as on the NumPy path.
With 'mean_std' these are the 'summary' names (mfcc_mean_0, ..., mfcc_std_12); with 'mean' or 'std' the plain names (mfcc_0, ...). Not defined for aggregation='none', whose length depends on the input duration.
forward(waveform)
¶
Features of a (time,) or (batch, time) waveform at sample_rate.
Tensors have the waveform's dtype. A dict of tensors for output='dict', otherwise one (batch, dim) or (dim,) tensor.
Warnings¶
Every warning AFX emits is an AFXWarning, so
warnings.filterwarnings('ignore', category=afx.AFXWarning) silences AFX
without hiding NumPy or SciPy warnings.
afx_audio._warnings.AFXWarning
¶
Bases: UserWarning
Base category of every warning AFX emits.
afx_audio._warnings.AFXDeprecationWarning
¶
Bases: AFXWarning, DeprecationWarning
An AFX name or parameter that will be removed; shown by default only for calls from __main__.