Google Search Node.js
Use the serpapi npm package instead. See the migration document for more details. |
|---|
google-search-results-nodejs is a MIT-licensed Node.js package that meant to scrape search results from Google, Bing, Baidu, Yahoo and 10+ more search engines with a SerpApi backend. SerpApi provides a Playground to get you started quickly by testing API interactively.
Find SerpApi documentation at: https://serpapi.com/search-api
Find SerpApi package at: https://www.npmjs.com/package/google-search-results-nodejs
Table of Contents
Requirements
- ES6 basic understanding
- Node 7+ and NPM installed
Installation
$ npm install google-search-results-nodejsQuick start
The following example runs a search for "coffee" using your secret API key which you can find at SerpApi Dashboard page.
Open in the online IDE (Replit).
const SerpApi = require('google-search-results-nodejs')
const search = new SerpApi.GoogleSearch("<your-serpapi-api-key>")
search.json({
q: "Coffee",
location: "Austin, TX"
}, (result) => {
console.log(result)
})How SerpApi backend works
How to set SerpApi key
api_key can be set globally using a singleton pattern:
// https://serpapi.com/manage-api-key
const SerpApi = require("google-search-results-nodejs")
const search = new SerpApi.GoogleSearch("<your-serpapi-api-key>")api_key can be read from the environment variable:
const search = new SerpApi.GoogleSearch(process.env.API_KEY);api_key can be provided for each request:
const SerpApi = require("google-search-results-nodejs")
const search = new SerpApi.GoogleSearch()
let result = search.json({
api_key: "<your-serpapi-api-key>", // https://serpapi.com/manage-api-key
q: "Coffee", // search query
location: "Austin, TX", // location of the search
}, (data) => {
console.log(data)
})Google Search API Capability
const SerpApi = require("google-search-results-nodejs")
const search = new SerpApi.GoogleSearch()
query_params = {
api_key: "asdewqe1231241asm", // Your SerpApi API key, https://serpapi.com/manage-api-key
q: "coffee", // Search query.
google_domain: "google.com", // Google domain to use.
location: "Austin, Texas, United States", // Location requested for the search.
uule: "w+CAIQICINVW5pdGVkIFN0YXRlcw", // Google encoded location you want to use for the search.
ludocid: "CID ID", // ID (CID) of the Google My Business listing you want to scrape.
lsig: "AB86z5W5r155sIcs3jqfYkm9Y8Fp", // Force the knowledge graph map view to show up.
device: "desktop|mobile|tablet", // Device used when making a search.
hl: "en", // Language of the search.
gl: "gl", // Country of the search.
lr: "lang_en|lang_fr", // One or multiple languages to limit the search to.
safe: "active|off", // Level of filtering for adult content.
nfpr: "1|0", // Exclusion of results from an auto-corrected query that is spelled wrong.
num: "100", // Number of results per page.
start: "20", // Pagination offset.
ijn:"1", // Page number for Google Images.
tbm: "nws|isch|shop|lcl|vid", // Type of search: news, images, shopping. local, video results.
tbs: "custom to be search criteria", // Advanced search for patents, dates, news, videos, images, apps, or text contents
async: True|False, // Allow async request.
no_cache: True|False // Force SerpApi to fetch the Google results even if a cached version is already present
}
const callback = (data) => {
console.log(data) // create a callback
}
search.json(query_params, callback) // Show result as JSON
search.html(query_params, callback) // Show result as HTML fileSupported Engines
Google Search and 20+ other engines 👇
| Engine | Class name |
|---|---|
| Google Search Engine | GoogleSearch() |
| Google Maps | GoogleSearch() |
| Google Jobs | GoogleSearch() |
| Google Trends | GoogleSearch() |
| Google Autocomplete | GoogleScholarSearch() |
| Google About This Result | GoogleSearch() |
| Google Lens | GoogleSearch() |
| Google Finance | GoogleSearch() |
| Google Related Questions | GoogleScholarSearch() |
| Google Scholar | GoogleScholarSearch() |
| Google Play Store | GoogleSearch() |
| Google Product | GoogleSearch() |
| Google Immersive Product | GoogleSearch() |
| Google Reverse Image | GoogleSearch() |
| Google Events | GoogleSearch() |
| Google Local Services | GoogleSearch() |
| Bing | BingSearch() |
| Baidu | BaiduSearch() |
| DuckDuckGo | DuckDuckGoSearch() |
| Yahoo | YahooSearch() |
| Yandex | YandexSearch() |
| eBay | EbaySearch() |
| Youtube | YoutubeSearch() |
| Walmart | WalmartSearch() |
| HomeDepot | HomeDepotSearch() |
| Apple App Store | AppleAppStoreSearch() |
| Naver | NaverSearch() |
| Yelp | YelpSearch() |
Example by specification
We love open source, continuous integration and Test Driven Development (TDD). We are using Mocha to test our infrastructure around the clock to achieve the best Quality of Service (QoS).
The directory test/ includes specification/examples.
Set your API key:
export API_KEY="<your-serpapi-api-key>"Run all tests:
npm testExtra APIs
Location API
const search = new SerpApi.GoogleSearch("<your-serpapi-api-key>")
search.location("Austin", 3, (data) => {
console.log(data)
})Prints the first three (3) locations matching Austin (Texas, Texas, Rochester)
[
{
"id":"585069bdee19ad271e9bc072",
"google_id":200635,
"google_parent_id":21176,
"name":"Austin, TX",
"canonical_name":"Austin,TX,Texas,United States",
"country_code":"US",
"target_type":"DMA Region",
"reach":5560000,
"gps":[
-97.7430608,
30.267153
],
"keys":[
"austin",
"tx",
"texas",
"united",
"states"
]
}, ... other results
]Search Archive API
The first search result returns a search_id which can be provided to get the search result from the archive. The following code will print the search from the archive.
var search = new SerpApi.GoogleSearch("<your-serpapi-api-key>")
search.json({q: "Coffee", location: "Portland" }, (search_result) => {
// search in archive for the search just returned
search.search_archive(search_result.search_metadata.id, (archived_search) => {
console.log(archived_search)
})
})Account API
The following code snippet will print your account information.
const search = new SerpApi.GoogleSearch("<your-serpapi-api-key>")
search.account((data) => {
console.log(data)
})Promise and callback
This API was developped using basic callback to handle response. Exception are just throw away with interceptor.
If you want to take advantage of the promise to block the request, here is how we'll do:
const util = require('util')
function getJson(parameter, resolve, reject) {
const search = new SerpApi.GoogleSearch("<your-serpapi-api-key>")
try {
search.json(parameter, resolve)
} catch (e) {
reject(e)
}
}
const blockFn = util.promisify(getJson)
blockFn[util.promisify.custom](parameter).then((data) => {
expect(data.local_results[0].title.length).toBeGreaterThan(5)
done()
}).catch((error) => {
console.error(error)
done()
})Practical code example using Google Images API
Open in the online IDE (Replit)
const SerpApi = require("google-search-results-nodejs");
const search = new SerpApi.GoogleSearch(process.env.API_KEY); // your serpapi API key, https://serpapi.com/manage-api-key
const searchQuery = "coffee";
const params = {
q: searchQuery, // what we want to search
engine: "google", // parsing engine
hl: "en", // parameter defines the language to use for the Google search
gl: "us", // parameter defines the country to use for the Google search
tbm: "isch", // parameter defines the type of search you want to do (isch - Google Images)
};
const getJson = () => {
return new Promise((resolve) => {
search.json(params, resolve);
});
};
const getResults = async () => {
const imagesResults = [];
while (true) {
const json = await getJson();
if (json.images_results) {
imagesResults.push(...json.images_results);
params.ijn ? (params.ijn += 1) : (params.ijn = 1);
} else break;
}
return imagesResults;
};
getResults().then((result) => console.dir(result, { depth: null }));Reference:
- test:
test/ExampleSpec.js - documentation: https://nodejs.org/docs/latest-v8.x/api/util.html#util_util_promisify_original
Coding style
This API is using callback to run in non-blocking code. Here we are trying to follow the spirit of NodeJS.
Reference:
- https://nodejs.org/en/docs/guides/blocking-vs-non-blocking/
- https://nodejs.org/en/docs/guides/dont-block-the-event-loop/
For pratical example, you can see the test located under test/ folder.
Run regression
To run the regression suite.
export API_KEY="your api key"
make testError Management
SerpApi keeps error management simple:
- backend service error or search fail.
- client error.
If it's a backend error, a simple error message is returned as string in the server response. If it's a client error, then a SerpApiClientException is raised.
Change log
- 2.1
- add support for Naver, HomeDepot, AppleStoreApp, DuckDuckGo
- defeature location if it is not supported by the search engine
- 2.0.1
- fix classes loading.
- 2.0
- Refractor class name: SearchResult -> Search
- 1.2
- stable version to support all the basic search API.