huggingface-transformers
DistilBert for self-supervision - switch heads for pre-training: MaskedLM and SequenceClassification
huggingface-transformers