Alterlab Text As Data
Published by alterlab-ieu in alterlab-academic-skills
What this skill does
Analyzes text as social-science data — topic modeling (BERTopic with embeddings + class-based TF-IDF, LDA/NMF via scikit-learn or gensim), document embeddings (sentence-transformers), dictionary/lexicon methods, and supervised text classification — choosing the method that matches the inferential goal (discovery vs measurement vs prediction) and validating topic reliability rather than trusting one stochastic run. It uses the verified stack (BERTopic, scikit-learn, gensim CoherenceModel, spaCy,
Add Alterlab Text As Data to your agent
Review the source and files first. When you are ready, copy the prompt instruction or use the CLI command supported by your environment.
Install with a prompt
Paste this into a compatible coding agent:
add this skill "alterlab-text-as-data" from https://github.com/alterlab-ieu/alterlab-academic-skillsInstall with the CLI
Run this command in a controlled environment after reviewing the repository:
npx skills add https://github.com/alterlab-ieu/alterlab-academic-skills --skill alterlab-text-as-dataSkill instructions
Text-as-Data — Match the Method to the Inferential Goal
Skill type: ANALYSIS MODULE. Turns a corpus into measurements. The discipline is choosing by goal — discovery (what themes exist?), measurement (how much of concept X?), or prediction (label new documents) — and validating that a topic solution is reliable, not a single lucky stochastic run.
Core Mission
PICK BY GOAL: DISCOVERY vs MEASUREMENT vs PREDICTION. THEN VALIDATE THE TOPICS — ONE RUN IS NOT A RESULT.
When to Use This Skill
- "Run topic modeling on my corpus (BERTopic / LDA)."
- "Measure how much each document expresses concept X (dictionary/lexicon)."
- "Embed my documents and cluster / compare them."
- "Classify these texts into categories."
Does NOT Trigger
| The request is really about… | Route to | Why not this skill |
|---|---|---|
| Training / fine-tuning a transformer model | alterlab-transformers | Model training, not corpus measurement. |
| Humanities close-reading / annotation of texts | alterlab-digital-humanities | Interpretive, not quantitative text-as-data. |
| Whether a text method fits the question at all | alterlab-ssci-design-gate | Design routing, upstream. |
| Plain tabular statistics | alterlab-statistical-analysis | No text. |
Method by goal (verified stack, pinned)
| Goal | Method | Verified call |
|---|---|---|
| Discovery (emergent themes, contextual) | BERTopic (v0.17) | from bertopic import BERTopic; topics, probs = BERTopic().fit_transform(docs); get_topic_info(), get_topic(0). Plug embeddings via embedding_model=SentenceTransformer("all-MiniLM-L6-v2"), a vectorizer_model=CountVectorizer(min_df=10). |
| Discovery (bag-of-words, classic) | LDA / NMF | sklearn: LatentDirichletAllocation(n_components=k).fit(CountVectorizer().fit_transform(docs)); or gensim LdaModel(corpus, num_topics=k, id2word=dictionary). |
| Topic quality | coherence | gensim CoherenceModel(model=lda, texts=tok, dictionary=d, coherence="c_v").get_coherence(). |
| Measurement (how much of concept X) | dictionary / lexicon | count validated lexicon terms; report reliability and validate against hand-coding. |
| Embeddings / similarity | sentence-transformers (v6) | SentenceTransformer("all-MiniLM-L6-v2").encode(texts) → cluster / cosine-compare. |
| Prediction (label documents) | supervised | TfidfVectorizer() → a sklearn classifier; report held-out F1, not in-sample fit. |
| Linguistic features (POS, entities) | spaCy (v3) | nlp = spacy.load("en_core_web_sm"); doc = nlp(text). |
Method-selection helper (stdlib): scripts/text_method_router.py. Full patterns, preprocessing,
and the topic-reliability procedure: references/text_stack.md.
The reliability discipline (topic models are stochastic)
LDA (and, to a lesser degree, BERTopic via UMAP) gives different topics on different runs. A single run is not a result. Required:
- Replicate across seeds/runs; align topics across runs (e.g. Hungarian matching) and score stability (top-word overlap, RBO); report a prototype (most-representative) solution.
- Coherence, not just perplexity — choose K by
c_vcoherence and human readability, not log-likelihood alone. - Determinism where possible — fix
random_state(sklearn LDA) and UMAP'srandom_state(BERTopic) for reproducibility; still report stability across different seeds. - Validate measurement — a dictionary/topic used as a measure of a concept needs validation against human coding (precision/recall or correlation), exactly like any other instrument.
Output Template
GOAL: discovery | measurement | prediction
METHOD: <BERTopic / LDA / dictionary / embeddings / supervised> + why it matches the goal
MODEL: <call + K selection by coherence; embedding/vectorizer choices>
RELIABILITY: <seeds run; topic stability; coherence c_v; determinism settings>
VALIDATION: <if used as a measure: agreement with human coding>
CLAIM SCOPE: descriptive/measurement of the corpus; generalization scoped to the corpus/sampling
References
references/text_stack.md— BERTopic/LDA/gensim/spaCy/embeddings patterns, preprocessing, topic-reliability procedure.scripts/text_method_router.py— stdlib router from goal + corpus features to the right method.
Part of the AlterLab Academic Skills suite.
Files included
- evals/evals.json
- references/text_stack.md
- scripts/text_method_router.py
- SKILL.md
More skills
Alterlab Social Science Methods
alterlab-ieu/alterlab-academic-skills
No summary provided.
Audit unavailable78 installsAlterlab Mermaid
alterlab-ieu/alterlab-academic-skills
No summary provided.
Audit unavailable67 installsAlterlab Qualitative Analysis
alterlab-ieu/alterlab-academic-skills
No summary provided.
Audit unavailable35 installs

