All Projects → streamnative → terraform-provider-pulsar

streamnative / terraform-provider-pulsar

Licence: Apache-2.0 License
Terraform provider for managing Apache Pulsar entities

Programming Languages

go
31211 projects - #10 most used programming language
Makefile
30231 projects
shell
77523 projects
Dockerfile
14818 projects

Projects that are alternatives of or similar to terraform-provider-pulsar

pulsar-helm-chart
Helm Chart for an Apache Pulsar Cluster
Stars: ✭ 31 (+47.62%)
Mutual labels:  apache-pulsar
SharpPulsar
One million topics client for Apache Pulsar - that is the goal!
Stars: ✭ 23 (+9.52%)
Mutual labels:  apache-pulsar
pulsarctl
a CLI for Apache Pulsar written in Go
Stars: ✭ 106 (+404.76%)
Mutual labels:  apache-pulsar
supernova
🌌 Apache Pulsar client for Haskell
Stars: ✭ 35 (+66.67%)
Mutual labels:  apache-pulsar
tgip-cn
TGIP-CN (Thank God Its Pulsar) is a weekly live video streaming about Apache Pulsar in Chinese.
Stars: ✭ 96 (+357.14%)
Mutual labels:  apache-pulsar
examples
Apache Pulsar examples and demos
Stars: ✭ 41 (+95.24%)
Mutual labels:  apache-pulsar
pulsar-io-kafka
Pulsar IO Kafka Connector
Stars: ✭ 24 (+14.29%)
Mutual labels:  apache-pulsar
tgip
TGIP (TGI Pulsar) is a weekly live video streaming about Apache Pulsar and its ecosystem.
Stars: ✭ 17 (-19.05%)
Mutual labels:  apache-pulsar
aop
AMQP on Pulsar protocol handler
Stars: ✭ 93 (+342.86%)
Mutual labels:  apache-pulsar
reactive-pulsar
Reactive Streams adapter for Apache Pulsar Java Client
Stars: ✭ 45 (+114.29%)
Mutual labels:  apache-pulsar
pulsar-beat-output
Elastic Beats Output to Apache Pulsar
Stars: ✭ 51 (+142.86%)
Mutual labels:  apache-pulsar
pulsar-flume-ng-sink
An Apache Flume Sink implementation to publish data to Apache pulsar
Stars: ✭ 19 (-9.52%)
Mutual labels:  apache-pulsar
zio-pulsar
Apache Pulsar client for Scala with ZIO and ZIO Streams integration.
Stars: ✭ 16 (-23.81%)
Mutual labels:  apache-pulsar
pulsar-tracing
Tracing instrumentation for Apache Pulsar clients.
Stars: ✭ 13 (-38.1%)
Mutual labels:  apache-pulsar

terraform-provider-pulsar

A Terraform provider for managing Apache Pulsar Entities.

Contents

Requirements

  • Terraform 0.10+
  • Go 1.16 or higher (to build the provider plugin)

Installation

  • Clone this repository and cd into the directory
  • Run make build, it will out a file named terraform-provider-pulsar
  • Copy this terraform-provider-pulsar bin file to your terraform plugin directory
  • Typically this plugin directory is ~/.terraform.d/plugins/
  • On Linux based 64-bit devices, this directory can be ~/.terraform.d/plugins/linux_amd64

Testing the Apache Pulsar Terraform Provider

  • Change directory to the project </path/to/provider/terraform-provider-pulsar>
  • In order to test the provider, you can run make test
  • In order to run the full suite of Acceptance tests, run make testacc

Note: Acceptance tests create real resources, and often cost money to run.

Provider Configuration

Example

Example provider with apache pulsar cluster, running locally with authentication disabled.

terraform {
  required_providers {
    pulsar = {
      versions = ["1.0.0"]
      source = "registry.terraform.io/apache/pulsar"
    }
  }
}

provider "pulsar" {
  web_service_url = "http://localhost:8080"
  token = "my_auth_token"
}
Property Description Required
web_service_url URL of your Apache Pulsar Cluster Yes
token Authentication Token for your Apache Pulsar Cluster, which is required only if your cluster has authentication enabled No
tls_trust_certs_file_path Path to a custom trusted TLS certificate file No
tls_allow_insecure_connection Boolean flag to accept untrusted TLS certificates No
api_version Used to request Apache Pulsar API service, default by 0, which represents use default version No
Resources

pulsar_tenant

A resource for managing Pulsar Tenants, can update admin roles and allowed clusters for a tenant.

Example

provider "pulsar" {
  web_service_url = "http://localhost:8080"
}

resource "pulsar_tenant" "my_tenant" {
  tenant           = "thanos"
  allowed_clusters = ["pulsar-cluster-1"]
  admin_roles      = ["godmode"]
}

Properties

Property Description Required
tenant Name of the Tenant that you want to create Yes
allowed_clusters An Array of clusters, accessible by this tenant No
admin_roles Admin Roles to be assumed by this Tenant No

pulsar_cluster

A resource for managing Apache Pulsar Clusters, can update various properties for a given cluster.

Example

provider "pulsar" {
  web_service_url = "http://localhost:8080"
}

resource "pulsar_cluster" "my_cluster" {
  cluster = "eternals"

  cluster_data {
    web_service_url    = "http://localhost:8080"
    broker_service_url = "http://localhost:6050"
    peer_clusters      = ["skrulls", "krees"]
  }
}

Properties

Property Description Required
cluster Name of the Cluster that you want to create Yes
cluster_data A Map of required fields for the cluster Yes
web_service_url Required in cluster data, pointing to your broker web service Yes
web_service_url_tls Pointing to your broker web service via tls No
broker_service_url Required in cluster data for broker discovery Yes
broker_service_url_tls Required in cluster data for broker discovery via tls No
peer_clusters Required in cluster data for adding peer clusters Yes

pulsar_namespace

A resource for creating and managing Apache Pulsar Namespaces, can update various properties for a given namespace.

Example

provider "pulsar" {
  web_service_url = "http://localhost:8080"
}

resource "pulsar_cluster" "test_cluster" {
  cluster = "skrulls"

  cluster_data {
    web_service_url    = "http://localhost:8080"
    broker_service_url = "http://localhost:6050"
    peer_clusters      = ["standalone"]
  }
}

resource "pulsar_tenant" "test_tenant" {
  tenant           = "thanos"
  allowed_clusters = [pulsar_cluster.test_cluster.cluster, "standalone"]
}

resource "pulsar_namespace" "test" {
  tenant    = pulsar_tenant.test_tenant.tenant
  namespace = "eternals"

  enable_deduplication = true

  // If defined partially, plan would show difference
  // however, none of the mising optionals would be changed
  namespace_config {
    anti_affinity                  = "anti-aff"
    max_consumers_per_subscription = "50"
    max_consumers_per_topic        = "50"
    max_producers_per_topic        = "50"
    replication_clusters           = ["standalone"]
  }

  dispatch_rate {
    dispatch_msg_throttling_rate  = 50
    rate_period_seconds           = 50
    dispatch_byte_throttling_rate = 2048
  }

  retention_policies {
    retention_minutes    = "1600"
    retention_size_in_mb = "10000"
  }

  backlog_quota {
    limit_bytes  = "10000000000"
    limit_seconds = "-1"
    policy = "consumer_backlog_eviction"
    type = "destination_storage"
  }

  persistence_policies {
    bookkeeper_ensemble                   = 1   // Number of bookies to use for a topic, default: 0
    bookkeeper_write_quorum               = 1   // How many writes to make of each entry, default: 0
    bookkeeper_ack_quorum                 = 1   // Number of acks (guaranteed copies) to wait for each entry, default: 0
    managed_ledger_max_mark_delete_rate   = 0.0 // Throttling rate of mark-delete operation (0 means no throttle), default: 0.0
  }

  permission_grant {
    role    = "some-role"
    actions = ["produce", "consume", "functions"]
  }
}

Properties

Property Description Required
tenant Name of the Tenant managing this namespace Yes
namespace name of the namespace Yes
enable_deduplication Message deduplication state on a namespace No
namespace_config Configuration for your namespaces like max allowed producers to produce messages No
dispatch_rate Apache Pulsar throttling config No
retention_policies Data retention policies No
schema_validation_enforce Enable or disable schema validation No
schema_compatibility_strategy Set schema compatibility strategy No
backlog_quota Backlog Quota for all topics No
persistence_policies Persistence policies for all topics under a given namespace No
permission_grant Permission grants on a namespace. This block can be repeated for each grant you'd like to add No
schema_compatibility_strategy
  • AutoUpdateDisabled
  • Backward
  • Forward
  • Full
  • AlwaysCompatible
  • BackwardTransitive
  • ForwardTransitive
  • FullTransitive

pulsar_topic

A resource for creating and managing Apache Pulsar Topics, can update partitions for a given partition topic.

Example

provider "pulsar" {
  web_service_url = "http://localhost:8080"
}

resource "pulsar_topic" "sample-topic-1" {
  tenant     = "public"
  namespace  = "default"
  topic_type = "persistent"
  topic_name = "partition-topic"
  partitions = 4                     # partitions > 0 means this is a partition topic

  permission_grant {
    role    = "some-role"
    actions = ["produce", "consume", "functions"]
  }

  retention_policies {
    retention_time_minutes = 1600
    retention_size_mb = 20000
  }
}

resource "pulsar_topic" "sample-topic-2" {
  tenant     = "public"
  namespace  = "default"
  topic_type = "persistent"
  topic_name = "non-partition-topic"
  partitions = 0                     # partitions = 0 means this is a non-partition topic

  permission_grant {
    role    = "some-role"
    actions = ["produce", "consume", "functions"]
  }

  retention_policies {
    retention_time_minutes = 1600
    retention_size_mb = 20000
  }
}

Properties

Property Description Required
tenant Name of the Tenant managing this topic Yes
namespace Name of the Namespace for this topic Yes
topic_type Topic persistence (persistent, non-persistent) Yes
topic_name Name of the topic Yes
partitions Number of partitions (0 for non-partitioned topic, > 1 for partitioned topic) Yes
permission_grant Permission grants on a topic. This block can be repeated for each grant you'd like to add. Permission grants are also inherited from the topic's namespace. No
retention_policies Data retention policies No

Importing existing resources

All resources could be imported using the standard terraform way.

Example

terraform import pulsar_cluster.standalone standalone

Contributing

Terraform is the work of thousands of contributors. We appreciate your help!

To contribute, please read the contribution guidelines: Contributing to Terraform - Apache Pulsar Provider

Issues on GitHub are intended to be related to bugs or feature requests with provider codebase. See https://www.terraform.io/docs/extend/community/index.html for a list of community resources to ask questions about Terraform.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].