Turn A Pile Of Notes Into A Searchable Library.

---
name: knowledge-library-extraction
description: Extract, organize, and distill a personal knowledge library from raw notes into structured intelligence with through-lines. Use this skill whenever you say "extract my library," "process my notes," "run the extraction," "build a codex," "distill my notes," "organize my second brain," "run CODE on my notes," or any variation of wanting to turn a large collection of personal notes into organized, categorized, searchable wisdom. Also trigger when you have a folder of notes from conferences, coaching programs, books, courses, or mentors and want to find the patterns, through-lines, and best insights across all of them. This is the bridge between a messy second brain and a structured intelligence asset.
---

# Knowledge Library Extraction

Turn a raw collection of personal notes into a structured intelligence library with through-lines, attributed lessons, and searchable wisdom.

## What This Produces

From raw notes (Evernote, Notion, markdown, text files), this process produces:

1. **Clean text files**, one per note, stripped of formatting, with title and date preserved
2. **Structured JSON cards**, one per note, with lessons extracted, speakers attributed, personal reflections separated, and a top quote identified
3. **Through-lines**, the 7-12 recurring philosophies that run across the entire collection, with anchor quotes and top lessons per theme
4. **A through-lines report**, a human-readable document showing the patterns, with an observations section that surfaces what a human reading linearly would never see

## When to Use This

- Processing notes from coaching programs, conferences, masterminds, or courses
- Organizing a personal knowledge management library (second brain)
- Preparing content for a Codex-style artifact or interactive experience
- Building a Claude Project knowledge base from personal notes
- Finding patterns across years of learning from mentors, speakers, or thought leaders
- Creating a searchable, categorized wisdom library from unstructured notes

## Prerequisites

- A collection of notes in one of these formats: Evernote export (HTML), Notion export (markdown), plain text files, or markdown files
- Notes should represent a coherent body of knowledge (one mentor's world, one topic area, one program series, etc.)
- Minimum ~20 notes for meaningful through-line discovery. Sweet spot is 50-200.

## The Process: CODE

**C**apture → **O**rganize → **D**istill → **E**xecute

---

### Phase 1: CAPTURE (Export and Convert)

**Goal:** Get all notes into clean, uniform text files.

**Input:** Raw notes in their native format
**Output:** A folder of clean .txt files, one per note

#### Step 1.1: Export from source

**From Evernote:**
- Select all notes → File → Export
- Choose "Multiple Web Pages (.html)"
- Export Note Attributes: keep only Note Title, Created Date, Tags
- Deselect: Author, Note Reminders, Source URL, Tasks, Updated Date, Thumbnail
- Why: Each attribute adds HTML clutter. Title, date, and tags are the only metadata we need.

**From Notion:**
- Select pages → Export → Markdown & CSV
- Choose "Everything" for content
- Include subpages if applicable

**From other sources:**
- Get files into .txt or .md format, one file per note
- Ensure each file has a clear title (filename or first line)

#### Step 1.2: Batch convert to clean text

Run this in Claude Code. The prompt below is calibrated for Evernote HTML exports. Adjust the input parsing for other formats.

**Claude Code Prompt:**
```
Convert all HTML files in [INPUT FOLDER] from Evernote HTML export format to clean, structured text files. Each HTML file is one note exported from Evernote with "Multiple Web Pages (.html)" format.

Create output folder: [OUTPUT FOLDER]
One .txt file per input .html file, with the same filename (but .txt extension).

Evernote HTML structure:
- Note title: <meta itemprop="title" content="THE TITLE">
- Created date: <meta itemprop="created" content="20200923T235313Z">
- Tags (if any): <meta itemprop="tag" content="TAG NAME"> (multiple possible)
- Content: inside <en-note> tags
- Ignore: <style> blocks, SVG <symbol> blocks, all CSS

Output format per .txt file:
TITLE: [note title from meta tag]
DATE: [created date, reformatted to readable format like "September 23, 2020"]
TAGS: [comma-separated tags if any, or "None"]

---

[Clean text content with these rules:]
- Strip ALL HTML tags
- Preserve line breaks where block elements (p, div, li, h1-h4, br) indicate them
- Collapse 3+ consecutive newlines to 2 (one blank line)
- Preserve bullet characters (•, -, *)
- Preserve bold: text in <b> or <strong> wrapped in ** on each side
- Strip all CSS, SVG, script content
- Trim whitespace per line, remove whitespace-only lines

Use Python with html.parser (no pip installs). Print progress per file. Print summary at end: total converted, total characters, any failures.

Do not ask me any questions, just build it and verify it works.
```

**Verification:** Check 3 random output files. Title should be clean, date readable, content should be the actual note text with no HTML artifacts.

---

### Phase 2: ORGANIZE (Extract Structured Cards)

**Goal:** Turn each clean text file into a structured JSON card with lessons extracted, speakers attributed, and reflections separated.

**Input:** Folder of clean .txt files
**Output:** A single all-cards.json file with one card per note

#### Step 2.1: Define your extraction schema

Each note becomes a card with this structure:

```json
{
  "id": "note-001",
  "title": "Original note title",
  "date": "September 23, 2020",
  "source": "Program or context name",
  "source_confidence": "high | medium | low",
  "speakers": ["List of people whose wisdom appears"],
  "lessons": [
    {
      "text": "Exact lesson text. PRESERVE ORIGINAL LANGUAGE",
      "speaker": "Who said it",
      "type": "quote | principle | tactic | mindset | story | reflection"
    }
  ],
  "personal_reflections": [
    "Lines that are the note-taker's personal processing"
  ],
  "top_quote": "Single most powerful line in this note"
}
```

#### Step 2.2: Define source detection rules

Customize these for your specific library. The idea is to map a signal in each note to the program or context it came from. Examples:

- Title contains a program keyword → "[Program name]"
- Content is about a topic area you studied → "[Topic bucket name]"
- Title contains a shorthand or acronym you use → "[Program name]"
- Default → "[General bucket name]"

#### Step 2.3: Define speaker detection rules

- Lines preceded by a name + dash/colon = that speaker's content
- Default speaker = the primary teacher unless attributed otherwise
- First-person lines about the note-taker's life = "[Name] (reflection)"
- Lines starting with ** = personal commentary
- Unsure = "Unknown"

#### Step 2.4: Run extraction in batches

Process 20 notes per Claude Code prompt. Do NOT use parallel sub-agents (they crash). Sequential batches are reliable.

**First batch prompt:**
```
Process the first 20 .txt files from [CLEAN NOTES FOLDER] (alphabetically). For each note, read the file and produce a structured JSON card with: id, title, date, source, source_confidence, speakers, lessons (with text, speaker, type), personal_reflections, and top_quote.

[INSERT YOUR SOURCE DETECTION RULES]
[INSERT YOUR SPEAKER DETECTION RULES]

Lesson types: quote (standalone memorable line), principle (operating philosophy), tactic (specific actionable strategy), mindset (belief or identity statement), story (narrative, capture key point), reflection (personal processing)

CRITICAL: Preserve original language. Do not summarize or rephrase. The exact words are the product.

Save to [OUTPUT]/batch-01.json. Print files processed, lesson count per file, and top_quote from each note.

Do not ask me any questions, just build it and verify it works.
```

**Subsequent batch prompt (repeat until all notes processed):**
```
Continue processing notes from [CLEAN NOTES FOLDER]. Use the exact same extraction rules and JSON card structure from the previous batch. Process the next 20 files alphabetically (skip any already processed). Save to [OUTPUT]/batch-XX.json (increment number). Print files processed, lesson count per file, and top_quote from each.

If this is the LAST batch (fewer than 20 remaining), after saving, merge ALL batch files into a single [OUTPUT]/all-cards.json. Generate [OUTPUT]/extraction-summary.md with: total notes, breakdown by source, breakdown by speaker, total lessons, total reflections, top 20 quotes. Print summary.

Do not ask me any questions, just build it and verify it works.
```

**Verification:** Spot-check 3-4 cards against notes you know well. Are lessons preserved verbatim? Are speakers correctly attributed? Are your personal reflections separated?

---

### Phase 3: DISTILL (Through-Line Discovery)

**Goal:** Find the 7-12 recurring philosophies that run across the entire library.

**Input:** all-cards.json
**Output:** through-lines.json + through-lines-report.md

#### Step 3.1: Run the clustering

**Claude Code Prompt:**
```
Read [PATH]/all-cards.json. Analyze ALL cards and discover the natural through-lines, the 7-12 recurring themes that run across the entire corpus regardless of source, speaker, or year.

A through-line is NOT a generic category like "Leadership" or "Mindset." It is a specific recurring philosophy named in the language of the world these notes come from.

For each through-line, identify:
- Name (in the source's language, using the words the teacher would use)
- One-sentence description
- How many notes touch this theme
- Which sources/programs it appears across
- Which speakers contribute to it
- 3-5 strongest lessons (EXACT original language)
- Single most powerful anchor quote

Save:
1. [PATH]/through-lines.json (structured data with all lessons mapped)
2. [PATH]/through-lines-report.md (human-readable report with: summary, each through-line with anchor quote and top lessons, uncategorized notes list, and 2-3 paragraph Observations section surfacing what a human reading linearly would never see)

Target: 7-12 through-lines. Fewer than 7 = too broad. More than 12 = consolidate.
PRESERVE ORIGINAL LANGUAGE in all quotes.

Print all through-line names, note counts, and anchor quotes.

Do not ask me any questions, just build it and verify it works.
```

#### Step 3.2: Review and iterate

This is the human step. Read the through-lines report and ask:

1. **Does each through-line feel like the world it came from?** If a name sounds like a textbook category, rename it in the source's language.
2. **Are any through-lines actually the same idea expressed differently?** Merge them.
3. **Is anything major missing?** Run the runner-up analysis:

```
Look at all-cards.json. If you were forced to pick 5 MORE through-lines beyond the current ones, what would they be? For each: proposed name, note count, anchor quote, why it did not make the cut. Also check: is there a distinct theme around [TOPIC YOU SUSPECT IS MISSING]? Report findings, do not save files.
```

4. **Swap, add, or merge** based on your review, then re-run the clustering with updates.

#### Step 3.3: Finalize

Once through-lines are locked, run one final update prompt to regenerate both files with the final structure. This is your permanent content architecture.

---

### Phase 4: EXECUTE (Build From the Asset)

The extraction is complete. You now have a structured intelligence library that can express itself in multiple forms:

**As a Claude Project:** Load all-cards.json and through-lines.json as project knowledge. Query the library conversationally: "What have I learned about X?" "Find every lesson about Y." "Write something using the principles from Z."

**As an interactive artifact:** Build a web experience (Claude Code) that lets someone explore the through-lines visually with layered depth.

**As a randomizer:** Surface one lesson at a time from the full library. Daily wisdom from the collection.

**As a content engine:** Each through-line is a content pillar. Each anchor quote is a post. Each top lesson is a thread. The library becomes source material for months of content.

**As a searchable database:** Import the structured cards into Notion or any database tool. Filter by speaker, source, type, or through-line.

The data is the asset. The expression is the choice.

---

## Key Principles

- **PRESERVE ORIGINAL LANGUAGE.** This is the #1 rule. If someone said "Don't quit 5 minutes before the miracle," that is the lesson, word for word. Never rewrite it into "the speaker advises persistence near the finish line."
- **Sequential batches beat parallel agents.** Parallel sub-agents in Claude Code crash under heavy load. 20 notes per batch, one batch at a time, is reliable.
- **The through-lines are the irreversible decision.** Visual design can change in a day. The content architecture cannot. Get the through-lines right before building anything on top of them.
- **A master knows what to leave out.** Every note does not need to be categorized. Stubs, receipts, and action-item lists can stay uncategorized without harming the library.
- **Max subscription note:** If using Claude Code through a Max subscription (rather than a separate API key), do NOT write prompts that make API calls from scripts. All processing should happen within Claude Code's native conversation.

## Typical Scale

| Library Size | Notes | Conversion Time | Extraction Time | Clustering Time |
|-------------|-------|----------------|-----------------|-----------------|
| Small | 20-50 | 5 min | 2-3 batches, ~30 min | 1 prompt, ~10 min |
| Medium | 50-150 | 10 min | 5-8 batches, ~1 hour | 1-2 prompts, ~20 min |
| Large | 150-300 | 15 min | 10-15 batches, ~2 hours | 2-3 prompts, ~30 min |
| Massive | 300+ | 20 min | 15+ batches, ~3 hours | Multiple prompts, ~1 hour |

## Troubleshooting

- **Claude Code "Not responding"**: Stop it, start a fresh session, re-paste the same batch prompt. Each batch reads from disk, no conversation history needed.
- **Through-lines feel generic**: The names need to be in the language of the world the notes came from rather than academic categories. Ask: "Would the teacher recognize this as their language?"
- **Too many uncategorized notes**: Run a targeted analysis on the uncategorized set looking for hidden clusters of 5+ notes sharing a theme.
- **Lessons come back paraphrased when they should be verbatim**: This is the most common extraction failure. Spot-check early and add emphasis to the "PRESERVE ORIGINAL LANGUAGE" instruction if needed.