For a language-learning platorm I'm building I need to analyze the difficulty of a foreign-language text for L2 learners in terms of CEFR.
However, as CEFR is made up of only Can do? questions, and not a fixed vocabulary list, I'm finding it difficult to properly assess. Currently, I'm running each text through the Flesch-Kincaid readability ease algorithm, and have the following rough coresspondings to CEFR levels:
# LINGUAPRESS estimates # 0-50 C2 Mastery / Proficiency # 50-60 C1 Advanced # 60-70 B2 Upper intermediate # 70-80 B1 Intermediate # 80-90 A2 Elementary # 90-100 A1 Beginner
Running 10 texts at A1 (black, average 97.5) and A2 (red, average 86.2), in English, roughly correspond to these levels.
However, I feel using FK is very inaccurate, especially for the later levels. This is because the algorithm doesn't take into account the complexity of words/frequency, just their syllable count and uniqueness.
I thought of combining FK with a measure of the frequency of words in the text compared to a wordlist (ex. using a weighted average). This could be used in conjunction with the estimates of CEFR vocab sizes StackExchange post But, I don't think this is the most efficient way, and don't know how much more accurate it would be.
Are there any algorithms, reading scores, or suggestions for calculating CEFR difficulty of Romance language texts?