Snorkel’s API Documentation¶
If you’re looking for technical details on Snorkel’s API, you’re in the right place.
For more narrative walkthroughs of Snorkel fundamentals or example use cases, check out our homepage and our tutorials repo.
Snorkel Analysis Package¶
Generic model analysis utilities shared across Snorkel.
Calculate one or more scores from user-specified and/or user-defined metrics. |
|
Return data point indices bucketed by label combinations. |
|
Evaluate a standard metric on a set of predictions/probabilities. |
Snorkel Augmentation Package¶
Programmatic data set augmentation: TF creation and data generation utilities.
Apply all TFs in order to each data point. |
|
Apply each TF individually to each data point. |
|
Apply a single TF to each data point. |
|
Sample sequences of TFs according to a distribution. |
|
TF applier for a Pandas DataFrame. |
|
Naive random augmentation policy. |
|
TF applier for a list of data points. |
|
Base class for TFs. |
|
Decorate functions to create TFs. |
Snorkel Classification Package¶
PyTorch-based multi-task learning framework for discriminative modeling.
Manager for checkpointing model. |
|
Manager for checkpointing model. |
|
A DataLoader that uses the appropriate collate_fn for a |
|
A dataset where both the data fields and labels are stored in as dictionaries. |
|
A class to manage logging during training progress. |
|
Manager for checkpointing model. |
|
A class for writing logs. |
|
Manager for checkpointing model. |
|
A classifier built from one or more tasks to support advanced workflows. |
|
A single operation (forward pass of a module) to execute in a Task. |
|
A single task (a collection of modules and specified path through them). |
|
A class for logging to Tensorboard during training process. |
|
A class for training a MultitaskClassifier. |
|
Calculate cross-entropy loss when targets are probabilities (floats), not ints. |
Snorkel Labeling Package¶
Programmatic data set labeling: LF creation, models, and analysis utilities.
LF applier for a Dask DataFrame. |
|
Run analyses on LFs using label matrix. |
|
LF applier for a list of data points (e.g. |
|
A model for learning the LF accuracies and combining their output labels. |
|
Base class for labeling functions. |
|
Majority class label model. |
|
Majority vote label model. |
|
Special labeling function type for spaCy-based LFs. |
|
LF applier for a Pandas DataFrame. |
|
Parallel LF applier for a Pandas DataFrame. |
|
Random vote label model. |
|
LF applier for a Spark RDD. |
|
Special labeling function type for SpaCy-based LFs running on Spark. |
|
Filter out examples not covered by any labeling function. |
|
Decorator to define a LabelingFunction object from a function. |
|
Decorator to define an NLPLabelingFunction object from a function. |
|
Decorator to define a SparkNLPLabelingFunction object from a function. |
Snorkel Map Package¶
Generic utilities for data point to data point operations.
Base class for |
|
Define a mapper from a function. |
|
Base class for any data point to data point mapping in the pipeline. |
|
Decorate a function to define a LambdaMapper object. |
|
Convert |
Snorkel Preprocess Package¶
Preprocessors for LFs, TFs, and SFs.
alias of |
|
Convenience class for definining preprocessors from functions. |
|
Base class for preprocessors. |
|
Preprocessor that parses input text via a SpaCy model. |
|
Convert |
|
Decorate functions to create preprocessors. |
Snorkel Slicing Package¶
Programmatic data set slicing: SF creation, monitoring utilities, and representation learning for slices.
SF applier for a Dask DataFrame. |
|
Special labeling function type for spaCy-based LFs. |
|
Parallel SF applier for a Pandas DataFrame. |
|
SF applier for a Pandas DataFrame. |
|
SF applier for a list of data points. |
|
A slice-aware classifier that supports training + scoring on slice labels. |
|
A module for combining the weighted representations learned by slices. |
|
Base class for slicing functions. |
|
Modify a dataloader in-place, adding labels for slice tasks. |
|
Add slice labels to dataloader and creates new slice tasks (including base slice). |
|
Decorator to define a NLPSlicingFunction child object from a function. |
|
Return a dataframe with examples corresponding to specified |
|
Decorator to define a SlicingFunction object from a function. |
Snorkel Utils Package¶
General machine learning utilities shared across Snorkel.
Filter out examples from arrays based on specified labels to filter. |
|
Convert an array of predictions into an array of probabilistic labels. |
|
Convert an array of probabilistic labels into an array of predictions. |
|
Convert an array to a (possibly flattened) array of ints. |