GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → castorini → Anserini

castorini / Anserini

A Lucene toolkit for replicable information retrieval research

Programming Languages

java
68154 projects - #9 most used programming language

Labels

information-retrievallucene

Projects that are alternatives of or similar to Anserini

Solrplugins
Dice Solr Plugins from Simon Hughes Dice.com
Stars: ✭ 86 (-84.99%)
Mutual labels:  information-retrieval, lucene
Lucene Solr
Apache Lucene and Solr open-source search software
Stars: ✭ 4,217 (+635.95%)
Mutual labels:  information-retrieval, lucene
Vectorsinsearch
Dice.com repo to accompany the dice.com 'Vectors in Search' talk by Simon Hughes, from the Activate 2018 search conference, and the 'Searching with Vectors' talk from Haystack 2019 (US). Builds upon my conceptual search and semantic search work from 2015
Stars: ✭ 71 (-87.61%)
Mutual labels:  information-retrieval, lucene
solr
Apache Solr open-source search software
Stars: ✭ 651 (+13.61%)
Mutual labels:  information-retrieval, lucene
LuceneTutorial
A simple tutorial of Lucene for LIS 501 Introduction to Text Mining students at the University of Wisconsin-Madison (Fall 2021).
Stars: ✭ 62 (-89.18%)
Mutual labels:  information-retrieval, lucene
RelevancyTuning
Dice.com tutorial on using black box optimization algorithms to do relevancy tuning on your Solr Search Engine Configuration from Simon Hughes Dice.com
Stars: ✭ 28 (-95.11%)
Mutual labels:  information-retrieval, lucene
lucene
Apache Lucene open-source search software
Stars: ✭ 1,009 (+76.09%)
Mutual labels:  information-retrieval, lucene
SolrConfigExamples
Examples of Solr configuration entries for Solr plugins and Conceptual Search\Semantic Search from Simon Hughes Dice.com
Stars: ✭ 26 (-95.46%)
Mutual labels:  information-retrieval, lucene
Nlp Projects
word2vec, sentence2vec, machine reading comprehension, dialog system, text classification, pretrained language model (i.e., XLNet, BERT, ELMo, GPT), sequence labeling, information retrieval, information extraction (i.e., entity, relation and event extraction), knowledge graph, text generation, network embedding
Stars: ✭ 360 (-37.17%)
Mutual labels:  information-retrieval
Sequence Semantic Embedding
Tools and recipes to train deep learning models and build services for NLP tasks such as text classification, semantic search ranking and recall fetching, cross-lingual information retrieval, and question answering etc.
Stars: ✭ 435 (-24.08%)
Mutual labels:  information-retrieval
Ssm
💕 build SSM from 0 👉🏽👉🏽 distributed micro service.
Stars: ✭ 3,383 (+490.4%)
Mutual labels:  lucene
Sparkler
Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
Stars: ✭ 362 (-36.82%)
Mutual labels:  information-retrieval
Screenfetch
Fetches system/theme information in terminal for Linux desktop screenshots.
Stars: ✭ 3,339 (+482.72%)
Mutual labels:  information-retrieval
Stroom
Stroom is a highly scalable data storage, processing and analysis platform.
Stars: ✭ 344 (-39.97%)
Mutual labels:  lucene
Cdqa
⛔ [NOT MAINTAINED] An End-To-End Closed Domain Question Answering System.
Stars: ✭ 500 (-12.74%)
Mutual labels:  information-retrieval
Getaltname
Extract subdomains from SSL certificates in HTTPS sites.
Stars: ✭ 320 (-44.15%)
Mutual labels:  information-retrieval
Resin
Hardware-accelerated vector-based search engine. Available as a HTTP service or as an embedded library.
Stars: ✭ 529 (-7.68%)
Mutual labels:  information-retrieval
Pisa
PISA: Performant Indexes and Search for Academia
Stars: ✭ 489 (-14.66%)
Mutual labels:  information-retrieval
Awesome Elasticsearch
A curated list of the most important and useful resources about elasticsearch: articles, videos, blogs, tips and tricks, use cases. All about Elasticsearch!
Stars: ✭ 4,168 (+627.4%)
Mutual labels:  lucene
Ip Tracer
Track any ip address with IP-Tracer. IP-Tracer is developed for Linux and Termux. you can retrieve any ip address information using IP-Tracer.
Stars: ✭ 399 (-30.37%)
Mutual labels:  information-retrieval
View All Similar Projects ➔

Anserini

Build Status codecov Generic badge Maven Central PyPI LICENSE doi

Anserini is an open-source information retrieval toolkit in Java built on Lucene that aims to bridge the gap between academic information retrieval research and the practice of building real-world search applications. Among other goals, our effort aims to be the opposite of this. Anserini grew out of a reproducibility study of various open-source retrieval engines in 2016 (Lin et al., ECIR 2016). See Yang et al. (SIGIR 2017) and Yang et al. (JDIQ 2018) for overviews.

Getting Started

Many Anserini features are exposed in the Pyserini Python interface. If you're looking for basic indexing and search capabilities, you might want to start there. A low-effort way to try out Anserini is to look at our online notebooks, which will allow you to get started with just a few clicks. For convenience, we've pre-built a few common indexes, available to download here.

You'll need Java 11 and Maven 3.3+ to build Anserini. Clone our repo with the --recurse-submodules option to make sure the eval/ submodule also gets cloned. Then, build using using Maven:

mvn clean package appassembler:assemble

The tools/ directory, which contains evaluation tools and other scripts, is actually this repo, integrated as a Git submodule (so that it can be shared across related projects). Build as follows (you might get warnings, but okay to ignore):

cd tools/eval && tar xvfz trec_eval.9.0.4.tar.gz && cd trec_eval.9.0.4 && make && cd ../../..
cd tools/eval/ndeval && make && cd ../../..

With that, you should be ready to go!

Regression Experiments

Anserini is designed to support experiments on various standard IR test collections out of the box. The following experiments are backed by rigorous end-to-end regression tests with run_regression.py and the Anserini replicability promise. For the most part, these runs are based on default parameter settings.

Replication Guides

The experiments described below are not associated with rigorous end-to-end regression testing and thus provide a lower standard of replicability. For the most part, manual copying and pasting of commands into a shell is required to replicate our results.

TREC-COVID and CORD-19

MS MARCO

Other Experiments

Additional Documentation

How Can I Contribute?

If you've found Anserini to be helpful, we have a simple request for you to contribute back. In the course of replicating baseline results on standard test collections, please let us know if you're successful by sending us a pull request with a simple note, like what appears at the bottom of the Robust04 page. Replicability is important to us, and we'd like to know about successes as well as failures. Since the regression documentation is auto-generated, pull requests should be sent against the raw templates. In turn, you'll be recognized as a contributor.

Beyond that, there are always open issues we would appreciate help on!

Release History

Historical Notes

  • Anserini was upgraded to Java 11 at commit 17b702d (7/11/2019) from Java 8. Maven 3.3+ is also required.
  • Anserini was upgraded to Lucene 8.0 as of commit 75e36f9 (6/12/2019); prior to that, the toolkit uses Lucene 7.6. Based on preliminary experiments, query evaluation latency has been much improved in Lucene 8. As a result of this upgrade, results of all regressions have changed slightly. To replicate old results from Lucene 7.6, use v0.5.1.

References

Acknowledgments

This research is supported in part by the Natural Sciences and Engineering Research Council (NSERC) of Canada. Previous support came from the U.S. National Science Foundation under IIS-1423002 and CNS-1405688. Any opinions, findings, and conclusions or recommendations expressed do not necessarily reflect the views of the sponsors.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].