All Projects → spatie → Laravel Sitemap

spatie / Laravel Sitemap

Licence: mit
Create and generate sitemaps with ease

Projects that are alternatives of or similar to Laravel Sitemap

Sitemap Generator Cli
Creates an XML-Sitemap by crawling a given site.
Stars: ✭ 214 (-83.85%)
Mutual labels:  google, sitemap, seo
Laravel Seo Tools
Laravel Seo package for Content writer/admin/web master who do not know programming but want to edit/update SEO tags from dashboard
Stars: ✭ 99 (-92.53%)
Mutual labels:  sitemap, laravel, seo
Laravel Robots Middleware
Enable or disable the indexing of your app
Stars: ✭ 259 (-80.45%)
Mutual labels:  google, laravel, seo
Sitemap Generator Crawler
Script that generates a sitemap by crawling a given URL
Stars: ✭ 169 (-87.25%)
Mutual labels:  sitemap, xml, seo
Sitemap Generator
Easily create XML sitemaps for your website.
Stars: ✭ 273 (-79.4%)
Mutual labels:  google, sitemap, seo
Stream Parser
⚡ PHP7 / Laravel Multi-format Streaming Parser
Stars: ✭ 391 (-70.49%)
Mutual labels:  xml, laravel
Nextjs Sitemap Generator
Generate sitemap.xml from nextjs pages
Stars: ✭ 395 (-70.19%)
Mutual labels:  sitemap, xml
Sitemap Module
Sitemap Module for Nuxt
Stars: ✭ 539 (-59.32%)
Mutual labels:  sitemap, seo
Awesome Seo
Google SEO研究及流量变现
Stars: ✭ 942 (-28.91%)
Mutual labels:  google, seo
Laravel Google Drive Demo
Laravel & Google Drive Storage - Demo project with Laravel 5.4
Stars: ✭ 299 (-77.43%)
Mutual labels:  google, laravel
Laravel Google Calendar
Manage events on a Google Calendar
Stars: ✭ 787 (-40.6%)
Mutual labels:  google, laravel
Serp
Google Search SERP Scraper
Stars: ✭ 40 (-96.98%)
Mutual labels:  google, seo
Laravel Missing Page Redirector
Redirect missing pages in your Laravel application
Stars: ✭ 378 (-71.47%)
Mutual labels:  laravel, seo
Laravel Referer
Remember a visitor's original referer
Stars: ✭ 339 (-74.42%)
Mutual labels:  laravel, seo
Next Sitemap
Sitemap generator for next.js. Generate sitemap(s) and robots.txt for all static/pre-rendered/dynamic/server-side pages.
Stars: ✭ 426 (-67.85%)
Mutual labels:  sitemap, seo
Laravel Paginateroute
Laravel router extension to easily use Laravel's paginator without the query string
Stars: ✭ 306 (-76.91%)
Mutual labels:  laravel, seo
Schema Org
A fluent builder Schema.org types and ld+json generator
Stars: ✭ 894 (-32.53%)
Mutual labels:  google, seo
Laravel Google Ads
Google Ads API for Laravel
Stars: ✭ 56 (-95.77%)
Mutual labels:  google, laravel
Response Xml
The missing XML support for Laravel's Response class.
Stars: ✭ 56 (-95.77%)
Mutual labels:  xml, laravel
Lashop
Simple shop based on Laravel 7.3
Stars: ✭ 60 (-95.47%)
Mutual labels:  laravel, seo

Generate sitemaps with ease

Latest Version on Packagist Software License Test Status Code Style Status Total Downloads

This package can generate a sitemap without you having to add urls to it manually. This works by crawling your entire site.

use Spatie\Sitemap\SitemapGenerator;

SitemapGenerator::create('https://example.com')->writeToFile($path);

You can also create your sitemap manually:

use Carbon\Carbon;
use Spatie\Sitemap\Sitemap;
use Spatie\Sitemap\Tags\Url;

Sitemap::create()

    ->add(Url::create('/home')
        ->setLastModificationDate(Carbon::yesterday())
        ->setChangeFrequency(Url::CHANGE_FREQUENCY_YEARLY)
        ->setPriority(0.1))

   ->add(...)

   ->writeToFile($path);

Or you can have the best of both worlds by generating a sitemap and then adding more links to it:

SitemapGenerator::create('https://example.com')
   ->getSitemap()
   ->add(Url::create('/extra-page')
        ->setLastModificationDate(Carbon::yesterday())
        ->setChangeFrequency(Url::CHANGE_FREQUENCY_YEARLY)
        ->setPriority(0.1))

    ->add(...)

    ->writeToFile($path);

You can also control the maximum depth of the sitemap:

SitemapGenerator::create('https://example.com')
    ->configureCrawler(function (Crawler $crawler) {
        $crawler->setMaximumDepth(3);
    })
    ->writeToFile($path);

The generator has the ability to execute JavaScript on each page so links injected into the dom by JavaScript will be crawled as well.

You can also use one of your available filesystem disks to write the sitemap to.

SitemapGenerator::create('https://example.com')->getSitemap()->writeToDisk('public', 'sitemap.xml');

You can also add your models directly by implementing the \Spatie\Sitemap\Contracts\Sitemapable interface.

use Spatie\Sitemap\Contracts\Sitemapable;
use Spatie\Sitemap\Tags\Url;

class Post extends Model implements Sitemapable
{
    public function toSitemapTag(): Url | string | array
    {
        return route('blog.post.show', $this);
    }
}

Now you can add a single post model to the sitemap or even a whole collection.

use Spatie\Sitemap\Sitemap;

Sitemap::create()
    ->add($post)
    ->add(Post::all());

This way you can add all your pages super fast without the need to crawl them all.

Support us

We invest a lot of resources into creating best in class open source packages. You can support us by buying one of our paid products.

We highly appreciate you sending us a postcard from your hometown, mentioning which of our package(s) you are using. You'll find our address on our contact page. We publish all received postcards on our virtual postcard wall.

Installation

First, install the package via composer:

composer require spatie/laravel-sitemap

The package will automatically register itself.

If you want to update your sitemap automatically and frequently you need to perform some extra steps.

Configuration

You can override the default options for the crawler. First publish the configuration:

php artisan vendor:publish --provider="Spatie\Sitemap\SitemapServiceProvider" --tag=config

This will copy the default config to config/sitemap.php where you can edit it.

use GuzzleHttp\RequestOptions;
use Spatie\Sitemap\Crawler\Profile;

return [

    /*
     * These options will be passed to GuzzleHttp\Client when it is created.
     * For in-depth information on all options see the Guzzle docs:
     *
     * http://docs.guzzlephp.org/en/stable/request-options.html
     */
    'guzzle_options' => [

        /*
         * Whether or not cookies are used in a request.
         */
        RequestOptions::COOKIES => true,

        /*
         * The number of seconds to wait while trying to connect to a server.
         * Use 0 to wait indefinitely.
         */
        RequestOptions::CONNECT_TIMEOUT => 10,

        /*
         * The timeout of the request in seconds. Use 0 to wait indefinitely.
         */
        RequestOptions::TIMEOUT => 10,

        /*
         * Describes the redirect behavior of a request.
         */
        RequestOptions::ALLOW_REDIRECTS => false,
    ],
    
    /*
     * The sitemap generator can execute JavaScript on each page so it will
     * discover links that are generated by your JS scripts. This feature
     * is powered by headless Chrome.
     */
    'execute_javascript' => false,
    
    /*
     * The package will make an educated guess as to where Google Chrome is installed. 
     * You can also manually pass it's location here.
     */
    'chrome_binary_path' => '',

    /*
     * The sitemap generator uses a CrawlProfile implementation to determine
     * which urls should be crawled for the sitemap.
     */
    'crawl_profile' => Profile::class,
    
];

Usage

Generating a sitemap

The easiest way is to crawl the given domain and generate a sitemap with all found links. The destination of the sitemap should be specified by $path.

SitemapGenerator::create('https://example.com')->writeToFile($path);

The generated sitemap will look similar to this:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <url>
        <loc>https://example.com</loc>
        <lastmod>2016-01-01T00:00:00+00:00</lastmod>
        <changefreq>daily</changefreq>
        <priority>0.8</priority>
    </url>
    <url>
        <loc>https://example.com/page</loc>
        <lastmod>2016-01-01T00:00:00+00:00</lastmod>
        <changefreq>daily</changefreq>
        <priority>0.8</priority>
    </url>

    ...
</urlset>

Customizing the sitemap generator

Define a custom Crawl Profile

You can create a custom crawl profile by implementing the Spatie\Crawler\CrawlProfile interface and by customizing the shouldCrawl() method for full control over what url/domain/sub-domain should be crawled:

use Spatie\Crawler\CrawlProfile;
use Psr\Http\Message\UriInterface;

class CustomCrawlProfile extends CrawlProfile
{
    public function shouldCrawl(UriInterface $url): bool
    {
        if ($url->getHost() !== 'localhost') {
            return false;
        }
        
        return $url->getPath() === '/';
    }
}

and register your CustomCrawlProfile::class in config/sitemap.php.

return [
    ...
    /*
     * The sitemap generator uses a CrawlProfile implementation to determine
     * which urls should be crawled for the sitemap.
     */
    'crawl_profile' => CustomCrawlProfile::class,
    
];

Changing properties

To change the lastmod, changefreq and priority of the contact page:

use Carbon\Carbon;
use Spatie\Sitemap\SitemapGenerator;
use Spatie\Sitemap\Tags\Url;

SitemapGenerator::create('https://example.com')
   ->hasCrawled(function (Url $url) {
       if ($url->segment(1) === 'contact') {
           $url->setPriority(0.9)
               ->setLastModificationDate(Carbon::create('2016', '1', '1'));
       }

       return $url;
   })
   ->writeToFile($sitemapPath);

Leaving out some links

If you don't want a crawled link to appear in the sitemap, just don't return it in the callable you pass to hasCrawled.

use Spatie\Sitemap\SitemapGenerator;
use Spatie\Sitemap\Tags\Url;

SitemapGenerator::create('https://example.com')
   ->hasCrawled(function (Url $url) {
       if ($url->segment(1) === 'contact') {
           return;
       }

       return $url;
   })
   ->writeToFile($sitemapPath);

Preventing the crawler from crawling some pages

You can also instruct the underlying crawler to not crawl some pages by passing a callable to shouldCrawl.

Note: shouldCrawl will only work with the default crawl Profile or custom crawl profiles that implement a shouldCrawlCallback method.

use Spatie\Sitemap\SitemapGenerator;
use Psr\Http\Message\UriInterface;

SitemapGenerator::create('https://example.com')
   ->shouldCrawl(function (UriInterface $url) {
       // All pages will be crawled, except the contact page.
       // Links present on the contact page won't be added to the
       // sitemap unless they are present on a crawlable page.
       
       return strpos($url->getPath(), '/contact') === false;
   })
   ->writeToFile($sitemapPath);

Configuring the crawler

The crawler itself can be configured to do a few different things.

You can configure the crawler used by the sitemap generator, for example: to ignore robot checks; like so.

SitemapGenerator::create('http://localhost:4020')
    ->configureCrawler(function (Crawler $crawler) {
        $crawler->ignoreRobots();
    })
    ->writeToFile($file);

Limiting the amount of pages crawled

You can limit the amount of pages crawled by calling setMaximumCrawlCount

use Spatie\Sitemap\SitemapGenerator;

SitemapGenerator::create('https://example.com')
    ->setMaximumCrawlCount(500) // only the 500 first pages will be crawled
    ...

Executing Javascript

The sitemap generator can execute JavaScript on each page so it will discover links that are generated by your JS scripts. You can enable this feature by setting execute_javascript in the config file to true.

Under the hood, headless Chrome is used to execute JavaScript. Here are some pointers on how to install it on your system.

The package will make an educated guess as to where Chrome is installed on your system. You can also manually pass the location of the Chrome binary to executeJavaScript().

Manually adding links

You can manually add links to a sitemap:

use Spatie\Sitemap\SitemapGenerator;
use Spatie\Sitemap\Tags\Url;

SitemapGenerator::create('https://example.com')
    ->getSitemap()
    // here we add one extra link, but you can add as many as you'd like
    ->add(Url::create('/extra-page')->setPriority(0.5))
    ->writeToFile($sitemapPath);

Adding alternates to links

Multilingual sites may have several alternate versions of the same page (one per language). Based on the previous example adding an alternate can be done as follows:

use Spatie\Sitemap\SitemapGenerator;
use Spatie\Sitemap\Tags\Url;

SitemapGenerator::create('https://example.com')
    ->getSitemap()
    // here we add one extra link, but you can add as many as you'd like
    ->add(Url::create('/extra-page')->setPriority(0.5)->addAlternate('/extra-pagina', 'nl'))
    ->writeToFile($sitemapPath);

Note the addAlternate function which takes an alternate URL and the locale it belongs to.

Manually creating a sitemap

You can also create a sitemap fully manual:

use Carbon\Carbon;

Sitemap::create()
   ->add('/page1')
   ->add('/page2')
   ->add(Url::create('/page3')->setLastModificationDate(Carbon::create('2016', '1', '1')))
   ->writeToFile($sitemapPath);

Creating a sitemap index

You can create a sitemap index:

use Spatie\Sitemap\SitemapIndex;

SitemapIndex::create()
    ->add('/pages_sitemap.xml')
    ->add('/posts_sitemap.xml')
    ->writeToFile($sitemapIndexPath);

You can pass a Spatie\Sitemap\Tags\Sitemap object to manually set the lastModificationDate property.

use Spatie\Sitemap\SitemapIndex;
use Spatie\Sitemap\Tags\Sitemap;

SitemapIndex::create()
    ->add('/pages_sitemap.xml')
    ->add(Sitemap::create('/posts_sitemap.xml')
        ->setLastModificationDate(Carbon::yesterday()))
    ->writeToFile($sitemapIndexPath);

the generated sitemap index will look similar to this:

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <sitemap>
      <loc>http://www.example.com/pages_sitemap.xml</loc>
      <lastmod>2016-01-01T00:00:00+00:00</lastmod>
   </sitemap>
   <sitemap>
      <loc>http://www.example.com/posts_sitemap.xml</loc>
      <lastmod>2015-12-31T00:00:00+00:00</lastmod>
   </sitemap>
</sitemapindex>

Create a sitemap index with sub-sequent sitemaps

You can call the maxTagsPerSitemap method to generate a sitemap that only contains the given amount of tags

use Spatie\Sitemap\SitemapGenerator;

SitemapGenerator::create('https://example.com')
    ->maxTagsPerSitemap(20000)
    ->writeToFile(public_path('sitemap.xml'));

Generating the sitemap frequently

Your site will probably be updated from time to time. In order to let your sitemap reflect these changes, you can run the generator periodically. The easiest way of doing this is to make use of Laravel's default scheduling capabilities.

You could set up an artisan command much like this one:

namespace App\Console\Commands;

use Illuminate\Console\Command;
use Spatie\Sitemap\SitemapGenerator;

class GenerateSitemap extends Command
{
    /**
     * The console command name.
     *
     * @var string
     */
    protected $signature = 'sitemap:generate';

    /**
     * The console command description.
     *
     * @var string
     */
    protected $description = 'Generate the sitemap.';

    /**
     * Execute the console command.
     *
     * @return mixed
     */
    public function handle()
    {
        // modify this to your own needs
        SitemapGenerator::create(config('app.url'))
            ->writeToFile(public_path('sitemap.xml'));
    }
}

That command should then be scheduled in the console kernel.

// app/Console/Kernel.php
protected function schedule(Schedule $schedule)
{
    ...
    $schedule->command('sitemap:generate')->daily();
    ...
}

Changelog

Please see CHANGELOG for more information what has changed recently.

Testing

First start the test server in a separate terminal session:

cd tests/server
./start_server.sh

With the server running you can execute the tests:

$ composer test

Contributing

Please see CONTRIBUTING for details.

Security

If you discover any security related issues, please email [email protected] instead of using the issue tracker.

Credits

Support us

Spatie is a webdesign agency based in Antwerp, Belgium. You'll find an overview of all our open source projects on our website.

Does your business depend on our contributions? Reach out and support us on Patreon. All pledges will be dedicated to allocating workforce on maintenance and new awesome stuff.

License

The MIT License (MIT). Please see License File for more information.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].