PyRouge
A Python implementation of the rouge summarization metric. Support both Rouge-l and Rouge-2 metrics.
How to use it
- Check out following test code.
from PyRouge.pyrouge import Rouge
r = Rouge()
system_generated_summary = "The Kyrgyz President pushed through the law requiring the use of ink during the upcoming Parliamentary and Presidential elections In an effort to live up to its reputation in the 1990s as an island of democracy. The use of ink is one part of a general effort to show commitment towards more open elections. improper use of this type of ink can cause additional problems as the elections in Afghanistan showed. The use of ink and readers by itself is not a panacea for election ills."
manual_summmary = "The use of invisible ink and ultraviolet readers in the elections of the Kyrgyz Republic which is a small, mountainous state of the former Soviet republic, causing both worries and guarded optimism among different sectors of the population. Though the actual technology behind the ink is not complicated, the presence of ultraviolet light (of the kind used to verify money) causes the ink to glow with a neon yellow light. But, this use of the new technology has caused a lot of problems. "
[precision, recall, f_score] = r.rouge_l([system_generated_summary], [manual_summmary])
print("Precision is :"+str(precision)+"\nRecall is :"+str(recall)+"\nF Score is :"+str(f_score))- Which gives the following output
Precision is :0.446058091286
Recall is :0.439672801636
F Score is :0.442843380487
-
Also you can check that if both the summaries are same, you get a score of 1.
-
Note that you can also use it with numbers as follows.
from PyRouge.pyrouge import Rouge
r = Rouge()
print(r.rouge_l([[1, 7, 6, 7, 5], [0, 2, 8, 3, 5]], [[1, 2, 3, 4, 5], [3, 9, 5]]))