Yahoo ticker downloader

Produces .csv, .xlsx, .json and .yaml files (All files contain same data but in a different format) for stocks, ETF, futures, indexes, mutual funds, currency, warrants and bonds. The ticker symbol, company name and exchange are saved for all symbols.

It gets its data from https://finance.yahoo.com/lookup/_. Please note: it is not possible to get all the symbols due to limitations set by Yahoo.

Requirements

Python 2.7 or Python 3.5+

Install

From python package manager (preferred):

.. code:: bash

pip install Yahoo-ticker-downloader

From source:

.. code:: bash

python setup.py install

Example Usage

.. code::

usage: YahooTickerDownloader.py [-h] [-i] [-e] [-E EXCHANGE] [-s SLEEP] [-p]
                                [type]

positional arguments:
  type                  The type to download, this can be: generic

optional arguments:
  -h, --help            show this help message and exit
  -i, --insecure        use HTTP instead of HTTPS
  -e, --export          export immediately without downloading (Only useful if
                        you already downloaded something to the .pickle file)
  -E EXCHANGE, --Exchange EXCHANGE
                        Only export ticker symbols from this exchange (the
                        filtering is done during the export phase)
  -s SLEEP, --sleep SLEEP
                        The time to sleep in seconds between requests
  -p, --pandantic       Stop and warn the user if some rare assertion fails

For example to download all stock symbols you run it like:

.. code:: bash

YahooTickerDownloader.py

The program takes a few weeks before it is finished. The program supports suspending and resuming a download. Press CTRL+C to suspend download. Restart the program in the same working directory to resume downloading. It is possible to export partially downloaded results using the -e flag.

Example of CSV output:

.. code::

Ticker	Name	Exchange	exchangeDisplay	Type	TypeDisplay
JNUG	Direxion Daily Jr Gld Mnrs Bull 3X ETF	ASE	NYSE MKT	E	ETF
DWDP	DowDuPont Inc.	NYQ	NYSE	S	Equity
E	Eni S.p.A.	NYQ	NYSE	S	Equity
EQH	AXA Equitable Holdings, Inc.	NYS	NYSE	S	Equity
XOM	Exxon Mobil Corporation	NYQ	NYSE	S	Equity
ETP	Energy Transfer Partners, L.P.	NYQ	NYSE	S	Equity
ES=F	E-mini S&P 500 Index Futures,Jufuture	CME	Chicago Mercantile Exchange	F	Futures
NQ=F	E-mini Nasdaq 100 Index Futuresfuture	CME	Chicago Mercantile Exchange	F	Futures
GE=F	Eurodollar Futures,Sep-2018future	CME	Chicago Mercantile Exchange	F	Futures
...ect

Depending on the type you are downloading, you will get between 3.000 and 100.000+ entries.

Further resources

Download history for symbols: ystockquote_

Changelog

• Version 3.0.1 (2018-12-01)

  • Removed reppy dependency

• Version 3.0.0 (2018-05-27)

  • Switched over to different JSON api (searchassist)

• Version 2.2.0 (2018-01-31)

  • Continue exporting to different formats if one export fails ( #41 )
  • Check robots.txt ( anti-feature )

• Version 2.1.1 (2017-08-02)

  • A too old requests dependency was listed ( #35 )

• Version 2.1.0 (2017-05-10)

  • Added market parameter ( pull request #33 )

• Version 2.0.1 (2017-05-07)

  • Fixed issue where all downloads except stock and currency stopped working.

• Version 2.0.0 (2017-05-05)

  • Switched over to JSON api

• Version 1.0.0 (2017-04-04)

  • Reverted some changes from 0.10.0. Bond is back. Reverted back to English site instead of German.
  • Resolved CSV issue again. Closes #23 and #16.
  • Merged #26 Workaround Y! b>2000 limit
  • Scraper now scrapes a lot more at the expense of runtime.
  • Support for python2 is back. Latest python 2 & 3 are supported.
  • Removed xls support
  • Added xlsx support (#29)

• Version 0.10.1 (2017-02-04)

  • More descriptive help message

• Version 0.10.0 (2017-02-02)

  • Removed bond downloading option.
  • Uses different yahoo source. Fixes #18
  • Removed python2 from classifiers. Related to #16

• Version 0.9.0 (unreleased)

  • Added a flag to restrict output to specific stock exchanges.

• Version 0.8.1 (2016-08-17)

  • Workaround for #7 : downloading interruption
  • Solution for #9 : UnicodeEncodeError

• Version 0.7.0 (2016-03-20)

  • Added --export option. It will transcode the .pickle file immediately to the desired output formats.

• Version 0.6.0 (unreleased)

  • Add 3 retries with an exponential back-off if HTTPError or ChunkedEncodingError is raised when processing _fetchHtml.

• Version 0.5.0 (2015-08-16)

  • Allows downloading using a insecure connection.
  • The temporarily download file-names now include the ticker type.

• Version 0.4.0 (2014-10-28)

  • Warrant symbols can now be downloaded.
  • Bond symbols can now be downloaded.

• Version 0.3.0 (2014-08-14)

  • Use HTTPS instead of HTTP
  • Retry to fetch a page if it contains no symbols (A "fix" for issue #4)
  • Renamed all 'Curreny' to 'Currency'
  • Relative imports are used
  • Fix: .csv file it outputs is encoded in UTF-8 when using python2
  • Performance: Considerable reduced memory consumption
  • It now outputs .json, .yaml and .xls files in addition to .csv

.. _https://finance.yahoo.com/lookup/: https://finance.yahoo.com/lookup/ .. _ystockquote: https://pypi.python.org/pypi/ystockquote/