How we count word frequency
The word order of the decks and the chart on the back of a card are Cardavela's own count. We read film subtitles, speech transcripts, books and articles and counted how often every word of the decks occurs, together with its forms.
Text read, millions of words
| Language | Speech | Films | Books | Internet | Words and phrases counted |
|---|---|---|---|---|---|
| English | 103 | 150 | 40 | 81 | 21 547 |
| Spanish | 3 | 100 | 15 | 51 | 21 751 |
| Hebrew | — | 80 | — | 50 | 20 544 |
| Arabic | — | 80 | — | 50 | 19 401 |
No open speech transcripts or books of that size were found for Hebrew and Arabic, so their chart shows films and the internet only.
Sources
Films and series
OpenSubtitles 2018, monolingual text of the OPUS corpusFilm and series subtitles from opensubtitles.org. P. Lison, J. Tiedemann. OpenSubtitles2016: Extracting Large Parallel Corpora from Movie and TV Subtitles. LREC 2016.
OPUS and opensubtitles.org terms
Speech
YouTube-Commons (PleIAs)Transcripts of openly licensed videos, original language only, no machine translation.
CC BY 4.0
Speech
Switchboard, ISIP word alignmentsEnglish telephone conversations, 1990s.
Open release by ISIP, Mississippi State University
Books
Project Gutenberg330 English and 166 Spanish books in the public domain.
Public domain
Internet
Wikipedia, 2023-11-01 dumpArticles in English, Spanish, Hebrew and Arabic.
CC BY-SA 4.0
How a word's place is counted
- The word and its forms are counted: “am”, “is”, “was” for “be”; “gave up”, “giving up” for “give up”. Hebrew and Arabic prefixes are included (ו, ה, ב / و, ال, ب).
- Frequency in each kind of text is per million words, so a large set of one kind does not outweigh a small one.
- The place is a weighted sum of log frequencies. English: speech and films 0.35 each, books and internet 0.15; Spanish: films 0.45, speech 0.25, internet 0.2, books 0.1; Hebrew and Arabic: films 0.7, internet 0.3.
- Phrases are ranked among phrases and words among words: the card says “Top 500 phrases” or “Top 1,000 words”.
- Shares are shown only for words seen at least 30 times; a small set of text where the word is expected fewer than 5 times is left out of the shares.
- Words spelled alike share one count. A meaning is not placed far above its translation in the other languages, so the Hebrew “shovel” does not stand next to the particle “את”.
- Hebrew, Spanish and Arabic verbs are counted by their dictionary form, so their place is not below the English verb of the same meaning.
Levels A0–C2
Level words A1–B2 follow the CEFR-J Wordlist 1.5 (Yukio Tono, Tokyo University of Foreign Studies), C1–C2 the Octanove Vocabulary Profile C1/C2 1.0 (Octanove Labs, CC BY-SA 4.0). Card translations and examples were proofread with the help of AI.