Custom Providers

Most Terraform users never need to write a custom provider - the existing ecosystem covers thousands of services. But when your platform, internal tool, or open-source project has no provider, building one is the only way to bring it under Terraform management. This page walks through the full lifecycle: why, how, and where to publish.

---

When to Write a Custom Provider

Scenario	Example
Platform control	You’re building a product or internal platform and want users to manage it via Terraform
Unsupported OSS	An open-source tool you depend on has no Terraform provider
Custom data processing	You need provider-defined functions for domain-specific transformations

Technical prerequisites:

• Go - providers must be written in Go; no other language is supported
• Terraform Plugin Framework - the current-generation SDK, supporting Protocol v6. It abstracts away gRPC internals so you don’t need to understand the wire protocol between Terraform and the provider binary

---

Developer Environment Setup

Prerequisites

Install the Go toolchain and (optionally) a Go-aware IDE extension (e.g. the Go extension for VS Code). Verify versions match what the scaffolding template requires.

Bootstrapping from the Scaffolding Template

HashiCorp maintains a Terraform Provider Scaffolding repository. Clone it and customise:

# Rename the Go module to your provider
go mod edit -module github.com/YOUR_ORG/terraform-provider-YOUR_SERVICE
go mod tidy

Cleanup checklist:

File	Action
.github/dependabot.yml	Update to your organisation’s settings
.github/CODEOWNERS, CODE_OF_CONDUCT.md	Update or remove
.copywrite.hcl	Delete (HashiCorp internal)
main.go	Update the module import path so go generate works
README.md	Replace the project description

Configuring Developer Overrides

To test locally without publishing, tell Terraform to load your compiled binary instead of downloading from a registry:

1. Build and install: go install - binary lands in $(go env GOBIN) (default $HOME/go/bin)
2. Edit ~/.terraformrc:

provider_installation {
  dev_overrides {
    "your-org/your-provider" = "/home/you/go/bin"
  }
  direct {}   # ← keep this or Terraform can't download ANY other provider
}

Caution

Without the direct {} block, Terraform loses access to all standard providers and will fail on every init that needs anything besides your local build.

---

Provider Design

The Plugin Framework provides three foundational capabilities that every provider component (provider, data source, resource, function) uses.

Schemas

Schemas define the shape of configuration blocks - the attributes users write in HCL and the values Terraform tracks in state. Each attribute carries metadata:

Metadata	Purpose
Type	string, bool, int64, list, map, object, etc.
Required / Optional / Computed	Controls whether the user must supply the value, may supply it, or Terraform calculates it
Default value	Fallback if the user omits the attribute
Sensitive	Framework automatically redacts the value from CLI output and logs
Deprecation message	Warns users that an attribute will be removed in a future version

Error Handling and Logging

The Diagnostics object

Most framework functions receive a resp with a Diagnostics field. Append errors and warnings to it instead of returning immediately on the first failure - this lets the user see all problems at once rather than fixing them one at a time.

The tflog package

Use tflog (not Go’s standard log) for all provider logging. Its levels (tflog.Debug, tflog.Info, tflog.Error, etc.) map directly to the TF_LOG environment variable.

Caution

Never call log.Fatal or any Fatal method. An early exit prevents Terraform from receiving error details and can leave the user with a corrupted state file. Instead, append the error to Diagnostics, log with tflog, and return.

Testing Framework

Provider tests use the same Go testing package but with Terraform-specific helpers:

Test type	Prefix	Runs against real APIs?	Gated by
Unit tests	Test	❌ No - isolated, fast, free	Always run
Acceptance tests	TestAcc	✅ Yes - may incur costs	TF_ACC=1

---

Provider Interface

The provider interface is the entry point. Implementing provider.Provider tells Terraform what the provider is called, how it authenticates, and which components it ships.

Required Functions

Function	Responsibility
Metadata	Sets the provider name and version (the name becomes the prefix for all resources)
Schema	Returns the schema defining user-facing configuration parameters
Configure	Validates inputs, initialises the API client, and attaches it to resp.DataSourceData and resp.ResourceData
Resources	Returns constructors for every resource the provider offers
DataSources	Returns constructors for every data source
Functions	Returns constructors for every provider-defined function
New	Factory that creates a provider instance

Provider Schema and Model

The schema typically exposes authentication parameters - host URL, client ID, client secret, access tokens. Best practice is to mark them all optional so users can fall back to environment variables without changing code:

schema.Schema{
    Attributes: map[string]schema.Attribute{
        "host": schema.StringAttribute{
            Optional:            true,
            MarkdownDescription: "API host URL. Falls back to `MYSERVICE_HOST` env var.",
        },
        "token": schema.StringAttribute{
            Optional:  true,
            Sensitive: true,
        },
    },
}

Schema attributes map to a Go struct (the provider model) used to pass configuration values internally.

The Configure Function - Validation Pattern

For every parameter, follow this strict order:

1. Check for Unknown - if the value depends on an uncreated resource, return a helpful diagnostic (the user needs to apply the dependency first)
2. Pull from environment variable - populate the value from os.Getenv as the default
3. Override with config - if the user supplied a value in the provider block, it takes precedence
4. Collect errors gracefully - append all validation failures to Diagnostics before returning; never halt on the first error

Once validated, the authenticated client is attached to resp.DataSourceData and resp.ResourceData.

Note

The client is deliberately not attached to Functions. Functions are pure logical transformations - they must never call external APIs or interact with state.

Provider Test Setup

Configure a factory function that spins up a provider server instance for acceptance tests, plus a pre-check function that verifies required environment variables (API tokens, etc.) are present before any test runs.

---

Data Source Implementation

Data sources read existing data and populate state. They never create, modify, or destroy infrastructure.

The Data Source Interface

Function	Responsibility
Metadata	Returns the data source name
Schema	Defines parameters (user inputs) and attributes (computed results)
Configure	Retrieves the API client from the provider
Read	Core logic: looks up data and writes it to state

Schema Design

Differentiate between what the user provides and what the provider returns:

• required - user must supply this (e.g., a search query or ID)
• optional - user may supply this
• computed - provider generates this (e.g., data fetched from an API); value may be unknown at plan time

The Read Function

The heart of every data source, following a consistent four-step pattern:

1. Parse user configuration into the Go data model
2. Call the API using the configured client
3. Map API results back into the data model (may require type coercion)
4. Save the populated model into Terraform state

func (d *AccountDataSource) Read(ctx context.Context, req datasource.ReadRequest, resp *datasource.ReadResponse) {
    var model AccountModel
    resp.Diagnostics.Append(req.Config.Get(ctx, &model)...)

    account, err := d.client.GetAccount(ctx, model.Username.ValueString())
    if err != nil {
        resp.Diagnostics.AddError("API Error", err.Error())
        return
    }

    model.ID          = types.StringValue(account.ID)
    model.DisplayName = types.StringValue(account.DisplayName)

    resp.Diagnostics.Append(resp.State.Set(ctx, &model)...)
}

Registration

Add the data source constructor to the provider’s DataSources function. After compilation, users access it via the standard data block.

Testing

Use resource.Test - the helper starts a provider server, runs the Terraform binary, and cleans up automatically.

• Define tests as resource.TestStep structs with Config (raw HCL) and Check (assertions)
• Data sources typically need only a single step (read and verify)
• Use resource.TestCheckResourceAttr for individual assertions or resource.ComposeAggregateTestCheckFunc to bundle them
• Include a PreCheck that verifies required environment variables

---

Resource Implementation (CRUD)

Resources are the most complex provider component - they create, read, update, and destroy real infrastructure.

Schema and Configuration

Resource schemas are more complex than data source schemas because they must specify:

• Which attributes can be updated in place vs. which changes force a destroy-and-replace
• PlanModifiers like UseStateForUnknown - prevents Terraform from displaying “known after apply” for fields that won’t actually change

The Configure function is identical to a data source’s: retrieve the API client from the provider.

CRUD Operations

Create

Runs once during initial resource creation:

1. Read desired parameters from the Terraform plan
2. Call the API to create the resource
3. Write generated attributes (ID, timestamps, etc.) back into the model and save to state

Caution

Normalization is critical. If the API alters submitted data (e.g., wrapping text in HTML tags, trimming whitespace, case-folding), your provider must normalize the response to match the user’s original input. Failing to do this causes eternal state drift - Terraform will detect a “change” on every single plan, forever.

Read

Refreshes state against reality:

1. Pull the resource ID from current state
2. Call the API to fetch current attributes
3. Normalize and update the model
4. Save to state

Normalization is equally important here.

Update

Triggered when the user’s desired state diverges from actual state but the change doesn’t require replacement:

1. Use the existing ID to update the resource via API
2. Write the API response back to state

Delete

Usually the simplest operation - call the API with the resource ID and delete it. If no error is returned, Terraform automatically removes the resource from local state.

Registration and Testing

• Register by adding the constructor to the provider’s Resources function
• Tests use multi-step resource.TestCase - typically: create → verify → import → update → verify
• Write helper functions to generate the HCL configuration strings for different test scenarios dynamically

---

Provider-Defined Functions

Introduced in Terraform v1.8, provider-defined functions are pure logical transformations - they take parameters, process them, and return a result. No API calls, no state interaction, no client configuration.

Note

If a provider containing functions is used with a Terraform version older than v1.8, the functions are silently ignored. Users will receive an error only if they attempt to call one.

The Function Interface

Function	Responsibility
Metadata	Returns the function name
Definition	Defines documentation, input parameters (e.g., function.StringParameter), and return type
Run	Loads parameters → performs transformation → returns result or errors

Registration and Usage

Register by adding the constructor to the provider’s Functions method. Users call functions with the namespaced syntax:

output "result" {
  value = provider::myservice::normalize_id(var.raw_id)
}

Testing Functions

Writing functions is simple; testing them is disproportionately complex because you must cover known values, null inputs, and unknown inputs.

Aspect	How it differs from resource testing
Test helper	resource.UnitTest (not resource.Test) - no API, no PreCheck needed
Version gating	Skip on Terraform < v1.8.0
Validation approach	Execute the function inside an output block; assert with resource.TestCheckOutput
Simulating unknowns	Use terraform_data as a dependency that isn’t resolved until apply

---

Testing the Provider

Testing spans all component types. Here’s a consolidated view of the framework:

Component	Test type	Helper	Steps	Key considerations
Provider	Acceptance	Factory function + pre-check	Config + auth validation	Verify env vars before running
Data source	Acceptance	resource.Test	Single step (read + check)	PreCheck for API credentials
Resource	Acceptance	resource.Test	Multi-step (create → import → update)	Helper functions for HCL generation
Function	Unit	resource.UnitTest	Multiple (known, null, unknown)	Version gate; use output blocks for assertion

Running tests:

# Unit tests only (fast, free)
go test ./...

# Acceptance tests (hits real APIs)
TF_ACC=1 go test ./... -v -timeout 30m

---

Publishing to the Registry

Once the provider is developed and tested, publish it so users can install it via terraform init.

1 · Generate Documentation

The tfplugindocs tool reads your schemas (including markdownDescription fields) and auto-generates consistent docs:

go generate ./...

Place practical usage examples in the examples/ folder - tfplugindocs automatically pulls them into the generated documentation.

2 · Create a GPG Key

Registries require cryptographically signed releases:

gpg --full-generate-key
# Select: RSA and RSA, keysize 4096 (only supported option)

Export both keys. Add the private key and passphrase as GitHub repository secrets:

Secret name	Value
GPG_PRIVATE_KEY	Exported private key
PASSPHRASE	Key passphrase

3 · Register the Provider

Registry	Process
Terraform Registry	Upload your public key in User Settings → link your GitHub repository via the “Publish” menu. Future releases sync automatically.
OpenTofu Registry	Submit your signing key and provider via GitHub Issues on the OpenTofu registry repository.

4 · Create a Release

Tag a semantic version on GitHub. The scaffolding template includes a release.yml GitHub Actions workflow that:

1. Uses .goreleaser.yml to build the provider for all target architectures
2. Signs the binaries with your GPG key
3. Uploads everything to the GitHub release

Once the workflow completes, users can install the provider immediately:

terraform init   # downloads and installs the new provider

Tip

For internal enterprise providers that shouldn’t be public, distribute via a private registry (Artifactory, GitLab Package Registry) or a network mirror configured in .terraformrc.