Profiles & Configuration¶

Prefect's settings are documented and type-validated. By modifying the default settings, you can customize various aspects of the system.

Settings can be viewed from the CLI or the UI.

You can override the setting for a profile with environment variables.

Prefect profiles are groups of settings that you can persist on your machine. When you change profiles, all of the settings configured in the profile are applied. You can apply profiles to individual commands or set a profile for your environment.

Commonly configured settings¶

This section describes some commonly configured settings for Prefect installations. See Configuring settings for details on setting and unsetting configuration values.

PREFECT_API_URL¶

The PREFECT_API_URL value specifies the API endpoint of your Prefect Cloud workspace or Prefect server instance.

For example, using a local Prefect server instance.

PREFECT_API_URL="http://127.0.0.1:4200/api"

Using Prefect Cloud:

PREFECT_API_URL="https://api.prefect.cloud/api/accounts/[ACCOUNT-ID]/workspaces/[WORKSPACE-ID]"

PREFECT_API_URL setting for agents

When using workers, agents, and work pools that can create flow runs for deployments in remote environments, PREFECT_API_URL must be set for the environment in which your worker or agent is running.

If you want the worker or agent to communicate with Prefect Cloud or a Prefect server instance from a remote execution environment such as a VM or Docker container, you must configure PREFECT_API_URL in that environment.

Running the Prefect UI behind a reverse proxy

When using a reverse proxy (such as Nginx or Traefik) to proxy traffic to a locally-hosted Prefect UI instance, the Prefect server also needs to be configured to know how to connect to the API. The PREFECT_UI_API_URL should be set to the external proxy URL (e.g. if your external URL is https://prefect-server.example.com/ then set PREFECT_UI_API_URL=https://prefect-server.example.com/api for the Prefect server process). You can also accomplish this by setting PREFECT_API_URL to the API URL, as this setting is used as a fallback if PREFECT_UI_API_URL is not set.

PREFECT_API_KEY¶

The PREFECT_API_KEY value specifies the API key used to authenticate with your Prefect Cloud workspace.

PREFECT_API_KEY="[API-KEY]"

PREFECT_HOME¶

The PREFECT_HOME value specifies the local Prefect directory for configuration files, profiles, and the location of the default Prefect SQLite database.

PREFECT_HOME='~/.prefect'

PREFECT_LOCAL_STORAGE_PATH¶

The PREFECT_LOCAL_STORAGE_PATH value specifies the default location of local storage for flow runs.

PREFECT_LOCAL_STORAGE_PATH='${PREFECT_HOME}/storage'

Database settings¶

Prefect provides several self-hosting database configuration settings you can read about here.

Logging settings¶

Prefect provides several logging configuration settings that you can read about in the logging docs.

Configuring settings¶

The prefect config CLI commands enable you to view, set, and unset settings.

Command	Description
set	Change the value for a setting.
unset	Restore the default value for a setting.
view	Display the current settings.

Viewing settings from the CLI¶

The prefect config view command will display settings that override default values.

$ prefect config view
PREFECT_PROFILE="default"
PREFECT_LOGGING_LEVEL='DEBUG'

You can show the sources of values with --show-sources:

$ prefect config view --show-sources
PREFECT_PROFILE="default"
PREFECT_LOGGING_LEVEL='DEBUG' (from env)

You can also include default values with --show-defaults:

$ prefect config view --show-defaults
PREFECT_PROFILE='default'
PREFECT_AGENT_PREFETCH_SECONDS='10' (from defaults)
PREFECT_AGENT_QUERY_INTERVAL='5.0' (from defaults)
PREFECT_API_KEY='None' (from defaults)
PREFECT_API_REQUEST_TIMEOUT='30.0' (from defaults)
PREFECT_API_URL='None' (from defaults)
...

Setting and clearing values¶

The prefect config set command lets you change the value of a default setting.

A commonly used example is setting the PREFECT_API_URL, which you may need to change when interacting with different Prefect server instances or Prefect Cloud.

# use a local Prefect server
prefect config set PREFECT_API_URL="http://127.0.0.1:4200/api"

# use Prefect Cloud
prefect config set PREFECT_API_URL="https://api.prefect.cloud/api/accounts/[ACCOUNT-ID]/workspaces/[WORKSPACE-ID]"

If you want to configure a setting to use its default value, use the prefect config unset command.

prefect config unset PREFECT_API_URL

Overriding defaults with environment variables¶

All settings have keys that match the environment variable that can be used to override them.

For example, configuring the home directory:

# environment variable
export PREFECT_HOME="/path/to/home"

# python
import prefect.settings
prefect.settings.PREFECT_HOME.value()  # PosixPath('/path/to/home')

Configuring the server's port:

# environment variable
export PREFECT_SERVER_API_PORT=4242

# python
prefect.settings.PREFECT_SERVER_API_PORT.value()  # 4242

Configuration profiles¶

Prefect allows you to persist settings instead of setting an environment variable each time you open a new shell. Settings are persisted to profiles, which allow you to move between groups of settings quickly.

The prefect profile CLI commands enable you to create, review, and manage profiles.

Command	Description
create	Create a new profile.
delete	Delete the given profile.
inspect	Display settings from a given profile; defaults to active.
ls	List profile names.
rename	Change the name of a profile.
use	Switch the active profile.

The default profile starts out empty:

$ prefect profile get
[default]

If you configured settings for a profile, prefect profile inspect displays those settings:

$ prefect profile inspect
PREFECT_PROFILE = "default"
PREFECT_API_KEY = "pnu_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
PREFECT_API_URL = "http://127.0.0.1:4200/api"

You can pass the name of a profile to view its settings:

$ prefect profile create test
$ prefect profile inspect test
PREFECT_PROFILE="test"

Creating and removing profiles¶

Create a new profile with no settings:

$ prefect profile create test
Created profile 'test' at /Users/terry/.prefect/profiles.toml.

Create a new profile foo with settings cloned from an existing default profile:

$ prefect profile create foo --from default
Created profile 'cloud' matching 'default' at /Users/terry/.prefect/profiles.toml.

Rename a profile:

$ prefect profile rename temp test
Renamed profile 'temp' to 'test'.

Remove a profile:

$ prefect profile delete test
Removed profile 'test'.

Removing the default profile resets it:

$ prefect profile delete default
Reset profile 'default'.

Change values in profiles¶

Set a value in the current profile:

$ prefect config set VAR=X
Set variable 'VAR' to 'X'
Updated profile 'default'

Set multiple values in the current profile:

$ prefect config set VAR2=Y VAR3=Z
Set variable 'VAR2' to 'Y'
Set variable 'VAR3' to 'Z'
Updated profile 'default'

You can set a value in another profile by passing the --profile NAME option to a CLI command:

$ prefect --profile "foo" config set VAR=Y
Set variable 'VAR' to 'Y'
Updated profile 'foo'

Unset values in the current profile to restore the defaults:

$ prefect config unset VAR2 VAR3
Unset variable 'VAR2'
Unset variable 'VAR3'
Updated profile 'default'

Inspecting profiles¶

See a list of available profiles:

$ prefect profile ls
* default
cloud
test
local

View the current profile:

$ prefect profile get
[default]
VAR=X

View another profile:

$ prefect profile get foo
[foo]
VAR=Y

View multiple profiles:

$ prefect profile get default foo
[default]
VAR=X

[foo]
VAR=Y

View all settings for a profile:

$ prefect profile inspect cloud
PREFECT_API_URL='https://api.prefect.cloud/api/accounts/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx
x/workspaces/xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
PREFECT_API_KEY='xxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'

Using profiles¶

The profile default is used by default. There are several methods to switch to another profile.

The recommended method is to use the prefect profile use command with the name of the profile:

$ prefect profile use foo
Profile 'test' now active.

Alternatively, you may set the environment variable PREFECT_PROFILE to the name of the profile:

$ export PREFECT_PROFILE=foo

Or, specify the profile in the CLI command for one-time usage:

$ prefect --profile "foo" ...

Note that this option must come before the subcommand. For example, to list flow runs using the profile foo:

$ prefect --profile "foo" flow-run ls

You may use the -p flag as well:

$ prefect -p "foo" flow-run ls

You may also create an 'alias' to automatically use your profile:

$ alias prefect-foo="prefect --profile 'foo' "
# uses our profile!
$ prefect-foo config view

Conflicts with environment variables¶

If setting the profile from the CLI with --profile, environment variables that conflict with settings in the profile will be ignored.

In all other cases, environment variables will take precedence over the value in the profile.

For example, a value set in a profile will be used by default:

$ prefect config set PREFECT_LOGGING_LEVEL="ERROR"
$ prefect config view --show-sources
PREFECT_PROFILE="default"
PREFECT_LOGGING_LEVEL='ERROR' (from profile)

But, setting an environment variable will override the profile setting:

$ export PREFECT_LOGGING_LEVEL="DEBUG"
$ prefect config view --show-sources
PREFECT_PROFILE="default"
PREFECT_LOGGING_LEVEL='DEBUG' (from env)

Unless the profile is explicitly requested when using the CLI:

$ prefect --profile default config view --show-sources
PREFECT_PROFILE="default"
PREFECT_LOGGING_LEVEL='ERROR' (from profile)

Profile files¶

Profiles are persisted to the PREFECT_PROFILES_PATH, which can be changed with an environment variable.

By default, it is stored in your PREFECT_HOME directory:

$ prefect config view --show-defaults | grep PROFILES_PATH
PREFECT_PROFILES_PATH='~/.prefect/profiles.toml'

The TOML format is used to store profile data.