snorkel.preprocess.preprocessor

class snorkel.preprocess.preprocessor(name=None, pre=None, memoize=False, memoize_key=None)

Bases: snorkel.map.core.lambda_mapper

Decorate functions to create preprocessors.

See snorkel.map.core.lambda_mapper for details.

Example

>>> @preprocessor()
... def combine_text_preprocessor(x):
...     x.article = f"{x.title} {x.body}"
...     return x
>>> from snorkel.preprocess.nlp import SpacyPreprocessor
>>> spacy_preprocessor = SpacyPreprocessor("article", "article_parsed")

We can now add our preprocessors to an LF.

>>> preprocessors = [combine_text_preprocessor, spacy_preprocessor]
>>> from snorkel.labeling.lf import labeling_function
>>> @labeling_function(pre=preprocessors)
... def article_mentions_person(x):
...     for ent in x.article_parsed.ents:
...         if ent.label_ == "PERSON":
...             return ABSTAIN
...     return NEGATIVE

__init__(name=None, pre=None, memoize=False, memoize_key=None)

Initialize self. See help(type(self)) for accurate signature.

Return type

None

Methods

__init__([name, pre, memoize, memoize_key])

Initialize self.

__call__(f)

Wrap a function to create a LambdaMapper.

Parameters

f (Callable[[Any], Optional[Any]]) – Function executing the mapping operation

Returns

New LambdaMapper executing operation in wrapped function

Return type

LambdaMapper