Skip to content

prefect.settings

Prefect settings management.

Each setting is defined as a Setting type. The name of each setting is stylized in all caps, matching the environment variable that can be used to change the setting.

All settings defined in this file are used to generate a dynamic Pydantic settings class called Settings. When instantiated, this class will load settings from environment variables and pull default values from the setting definitions.

The current instance of Settings being used by the application is stored in a SettingsContext model which allows each instance of the Settings class to be accessed in an async-safe manner.

Aside from environment variables, we allow settings to be changed during the runtime of the process using profiles. Profiles contain setting overrides that the user may persist without setting environment variables. Profiles are also used internally for managing settings during task run execution where differing settings may be used concurrently in the same process and during testing where we need to override settings to ensure their value is respected as intended.

The SettingsContext is set when the prefect module is imported. This context is referred to as the "root" settings context for clarity. Generally, this is the only settings context that will be used. When this context is entered, we will instantiate a Settings object, loading settings from environment variables and defaults, then we will load the active profile and use it to override settings. See enter_root_settings_context for details on determining the active profile.

Another SettingsContext may be entered at any time to change the settings being used by the code within the context. Generally, users should not use this. Settings management should be left to Prefect application internals.

Generally, settings should be accessed with SETTING_VARIABLE.value() which will pull the current Settings instance from the current SettingsContext and retrieve the value of the relevant setting.

Accessing a setting's value will also call the Setting.value_callback which allows settings to be dynamically modified on retrieval. This allows us to make settings dependent on the value of other settings or perform other dynamic effects.

PREFECT_HOME = Setting(Path, default=Path('~') / '.prefect', value_callback=expanduser_in_path) module-attribute

Prefect's home directory. Defaults to ~/.prefect. This directory may be created automatically when required.

PREFECT_EXTRA_ENTRYPOINTS = Setting(str, default='') module-attribute

Modules for Prefect to import when Prefect is imported.

Values should be separated by commas, e.g. my_module,my_other_module. Objects within modules may be specified by a ':' partition, e.g. my_module:my_object. If a callable object is provided, it will be called with no arguments on import.

PREFECT_DEBUG_MODE = Setting(bool, default=False) module-attribute

If True, places the API in debug mode. This may modify behavior to facilitate debugging, including extra logs and other verbose assistance. Defaults to False.

PREFECT_CLI_COLORS = Setting(bool, default=True) module-attribute

If True, use colors in CLI output. If False, output will not include colors codes. Defaults to True.

PREFECT_CLI_PROMPT = Setting(Optional[bool], default=None) module-attribute

If True, use interactive prompts in CLI commands. If False, no interactive prompts will be used. If None, the value will be dynamically determined based on the presence of an interactive-enabled terminal.

PREFECT_CLI_WRAP_LINES = Setting(bool, default=True) module-attribute

If True, wrap text by inserting new lines in long lines in CLI output. If False, output will not be wrapped. Defaults to True.

PREFECT_TEST_MODE = Setting(bool, default=False) module-attribute

If True, places the API in test mode. This may modify behavior to facilitate testing. Defaults to False.

PREFECT_UNIT_TEST_MODE = Setting(bool, default=False) module-attribute

This variable only exists to facilitate unit testing. If True, code is executing in a unit test context. Defaults to False.

PREFECT_TEST_SETTING = Setting(Any, default=None, value_callback=only_return_value_in_test_mode) module-attribute

This variable only exists to facilitate testing of settings. If accessed when PREFECT_TEST_MODE is not set, None is returned.

PREFECT_API_TLS_INSECURE_SKIP_VERIFY = Setting(bool, default=False) module-attribute

If True, disables SSL checking to allow insecure requests. This is recommended only during development, e.g. when using self-signed certificates.

PREFECT_API_URL = Setting(str, default=None) module-attribute

If provided, the URL of a hosted Prefect API. Defaults to None.

When using Prefect Cloud, this will include an account and workspace.

PREFECT_API_KEY = Setting(str, default=None, is_secret=True) module-attribute

API key used to authenticate with a the Prefect API. Defaults to None.

PREFECT_API_ENABLE_HTTP2 = Setting(bool, default=True) module-attribute

If true, enable support for HTTP/2 for communicating with an API.

If the API does not support HTTP/2, this will have no effect and connections will be made via HTTP/1.1.

PREFECT_CLIENT_MAX_RETRIES = Setting(int, default=5) module-attribute

The maximum number of retries to perform on failed HTTP requests.

Defaults to 5. Set to 0 to disable retries.

See PREFECT_CLIENT_RETRY_EXTRA_CODES for details on which HTTP status codes are retried.

PREFECT_CLIENT_RETRY_JITTER_FACTOR = Setting(float, default=0.2) module-attribute

A value greater than or equal to zero to control the amount of jitter added to retried client requests. Higher values introduce larger amounts of jitter.

Set to 0 to disable jitter. See clamped_poisson_interval for details on the how jitter can affect retry lengths.

PREFECT_CLIENT_RETRY_EXTRA_CODES = Setting(str, default='', value_callback=status_codes_as_integers_in_range) module-attribute

A comma-separated list of extra HTTP status codes to retry on. Defaults to an empty string. 429, 502 and 503 are always retried. Please note that not all routes are idempotent and retrying may result in unexpected behavior.

PREFECT_CLOUD_API_URL = Setting(str, default='https://api.prefect.cloud/api', value_callback=check_for_deprecated_cloud_url) module-attribute

API URL for Prefect Cloud. Used for authentication.

PREFECT_CLOUD_URL = Setting(str, default=None, deprecated=True, deprecated_start_date='Dec 2022', deprecated_help='Use `PREFECT_CLOUD_API_URL` instead.') module-attribute

PREFECT_UI_URL = Setting(Optional[str], default=None, value_callback=default_ui_url) module-attribute

The URL for the UI. By default, this is inferred from the PREFECT_API_URL.

When using Prefect Cloud, this will include the account and workspace. When using an ephemeral server, this will be None.

PREFECT_CLOUD_UI_URL = Setting(str, default=None, value_callback=default_cloud_ui_url) module-attribute

The URL for the Cloud UI. By default, this is inferred from the PREFECT_CLOUD_API_URL.

PREFECT_UI_URL will be workspace specific and will be usable in the open source too.

In contrast, this value is only valid for Cloud and will not include the workspace.

PREFECT_API_REQUEST_TIMEOUT = Setting(float, default=60.0) module-attribute

The default timeout for requests to the API

PREFECT_EXPERIMENTAL_WARN = Setting(bool, default=True) module-attribute

If enabled, warn on usage of experimental features.

PREFECT_PROFILES_PATH = Setting(Path, default=Path('${PREFECT_HOME}') / 'profiles.toml', value_callback=template_with_settings(PREFECT_HOME)) module-attribute

The path to a profiles configuration files.

PREFECT_RESULTS_DEFAULT_SERIALIZER = Setting(str, default='pickle') module-attribute

The default serializer to use when not otherwise specified.

PREFECT_RESULTS_PERSIST_BY_DEFAULT = Setting(bool, default=False) module-attribute

The default setting for persisting results when not otherwise specified. If enabled, flow and task results will be persisted unless they opt out.

PREFECT_TASKS_REFRESH_CACHE = Setting(bool, default=False) module-attribute

If True, enables a refresh of cached results: re-executing the task will refresh the cached results. Defaults to False.

PREFECT_TASK_DEFAULT_RETRIES = Setting(int, default=0) module-attribute

This value sets the default number of retries for all tasks. This value does not overwrite individually set retries values on tasks

PREFECT_FLOW_DEFAULT_RETRIES = Setting(int, default=0) module-attribute

This value sets the default number of retries for all flows. This value does not overwrite individually set retries values on a flow

PREFECT_FLOW_DEFAULT_RETRY_DELAY_SECONDS = Setting(Union[int, float], default=0) module-attribute

This value sets the retry delay seconds for all flows. This value does not overwrite individually set retry delay seconds

PREFECT_TASK_DEFAULT_RETRY_DELAY_SECONDS = Setting(Union[float, int, List[float]], default=0) module-attribute

This value sets the default retry delay seconds for all tasks. This value does not overwrite individually set retry delay seconds

PREFECT_LOCAL_STORAGE_PATH = Setting(Path, default=Path('${PREFECT_HOME}') / 'storage', value_callback=template_with_settings(PREFECT_HOME)) module-attribute

The path to a directory to store things in.

PREFECT_MEMO_STORE_PATH = Setting(Path, default=Path('${PREFECT_HOME}') / 'memo_store.toml', value_callback=template_with_settings(PREFECT_HOME)) module-attribute

The path to the memo store file.

PREFECT_MEMOIZE_BLOCK_AUTO_REGISTRATION = Setting(bool, default=True) module-attribute

Controls whether or not block auto-registration on start up should be memoized. Setting to False may result in slower server start up times.

PREFECT_LOGGING_LEVEL = Setting(str, default='INFO', value_callback=debug_mode_log_level) module-attribute

The default logging level for Prefect loggers. Defaults to "INFO" during normal operation. Is forced to "DEBUG" during debug mode.

PREFECT_LOGGING_INTERNAL_LEVEL = Setting(str, default='ERROR', value_callback=debug_mode_log_level) module-attribute

The default logging level for Prefect's internal machinery loggers. Defaults to "ERROR" during normal operation. Is forced to "DEBUG" during debug mode.

PREFECT_LOGGING_SERVER_LEVEL = Setting(str, default='WARNING') module-attribute

The default logging level for the Prefect API server.

PREFECT_LOGGING_SETTINGS_PATH = Setting(Path, default=Path('${PREFECT_HOME}') / 'logging.yml', value_callback=template_with_settings(PREFECT_HOME)) module-attribute

The path to a custom YAML logging configuration file. If no file is found, the default logging.yml is used. Defaults to a logging.yml in the Prefect home directory.

PREFECT_LOGGING_EXTRA_LOGGERS = Setting(str, default='', value_callback=get_extra_loggers) module-attribute

Additional loggers to attach to Prefect logging at runtime. Values should be comma separated. The handlers attached to the 'prefect' logger will be added to these loggers. Additionally, if the level is not set, it will be set to the same level as the 'prefect' logger.

PREFECT_LOGGING_LOG_PRINTS = Setting(bool, default=False) module-attribute

If set, print statements in flows and tasks will be redirected to the Prefect logger for the given run. This setting can be overridden by individual tasks and flows.

PREFECT_LOGGING_TO_API_ENABLED = Setting(bool, default=True) module-attribute

Toggles sending logs to the API. If False, logs sent to the API log handler will not be sent to the API.

PREFECT_LOGGING_TO_API_BATCH_INTERVAL = Setting(float, default=2.0) module-attribute

The number of seconds between batched writes of logs to the API.

PREFECT_LOGGING_TO_API_BATCH_SIZE = Setting(int, default=4000000) module-attribute

The maximum size in bytes for a batch of logs.

PREFECT_LOGGING_TO_API_MAX_LOG_SIZE = Setting(int, default=1000000) module-attribute

The maximum size in bytes for a single log.

PREFECT_LOGGING_TO_API_WHEN_MISSING_FLOW = Setting(Literal['warn', 'error', 'ignore'], default='warn') module-attribute

Controls the behavior when loggers attempt to send logs to the API handler from outside of a flow.

All logs sent to the API must be associated with a flow run. The API log handler can only be used outside of a flow by manually providing a flow run identifier. Logs that are not associated with a flow run will not be sent to the API. This setting can be used to determine if a warning or error is displayed when the identifier is missing.

The following options are available:

  • "warn": Log a warning message.
  • "error": Raise an error.
  • "ignore": Do not log a warning message or raise an error.

PREFECT_SQLALCHEMY_POOL_SIZE = Setting(int, default=None) module-attribute

Controls connection pool size when using a PostgreSQL database with the Prefect API. If not set, the default SQLAlchemy pool size will be used.

PREFECT_SQLALCHEMY_MAX_OVERFLOW = Setting(int, default=None) module-attribute

Controls maximum overflow of the connection pool when using a PostgreSQL database with the Prefect API. If not set, the default SQLAlchemy maximum overflow value will be used.

PREFECT_LOGGING_COLORS = Setting(bool, default=True) module-attribute

Whether to style console logs with color.

PREFECT_LOGGING_MARKUP = Setting(bool, default=False) module-attribute

Whether to interpret strings wrapped in square brackets as a style. This allows styles to be conveniently added to log messages, e.g. [red]This is a red message.[/red]. However, the downside is, if enabled, strings that contain square brackets may be inaccurately interpreted and lead to incomplete output, e.g. DROP TABLE [dbo].[SomeTable];" outputs DROP TABLE .[SomeTable];.

PREFECT_AGENT_QUERY_INTERVAL = Setting(float, default=15) module-attribute

The agent loop interval, in seconds. Agents will check for new runs this often. Defaults to 15.

PREFECT_AGENT_PREFETCH_SECONDS = Setting(int, default=15) module-attribute

Agents will look for scheduled runs this many seconds in the future and attempt to run them. This accounts for any additional infrastructure spin-up time or latency in preparing a flow run. Note flow runs will not start before their scheduled time, even if they are prefetched. Defaults to 15.

PREFECT_ASYNC_FETCH_STATE_RESULT = Setting(bool, default=False) module-attribute

Determines whether State.result() fetches results automatically or not. In Prefect 2.6.0, the State.result() method was updated to be async to facilitate automatic retrieval of results from storage which means when writing async code you must await the call. For backwards compatibility, the result is not retrieved by default for async users. You may opt into this per call by passing fetch=True or toggle this setting to change the behavior globally. This setting does not affect users writing synchronous tasks and flows. This setting does not affect retrieval of results when using Future.result().

PREFECT_API_BLOCKS_REGISTER_ON_START = Setting(bool, default=True) module-attribute

If set, any block types that have been imported will be registered with the backend on application startup. If not set, block types must be manually registered.

PREFECT_API_DATABASE_PASSWORD = Setting(str, default=None, is_secret=True) module-attribute

Password to template into the PREFECT_API_DATABASE_CONNECTION_URL. This is useful if the password must be provided separately from the connection URL. To use this setting, you must include it in your connection URL.

PREFECT_API_DATABASE_CONNECTION_URL = Setting(str, default=None, value_callback=default_database_connection_url, is_secret=True) module-attribute

A database connection URL in a SQLAlchemy-compatible format. Prefect currently supports SQLite and Postgres. Note that all Prefect database engines must use an async driver - for SQLite, use sqlite+aiosqlite and for Postgres use postgresql+asyncpg.

SQLite in-memory databases can be used by providing the url sqlite+aiosqlite:///file::memory:?cache=shared&uri=true&check_same_thread=false, which will allow the database to be accessed by multiple threads. Note that in-memory databases can not be accessed from multiple processes and should only be used for simple tests.

Defaults to a sqlite database stored in the Prefect home directory.

If you need to provide password via a different environment variable, you use the PREFECT_API_DATABASE_PASSWORD setting. For example:

PREFECT_API_DATABASE_PASSWORD='mypassword'
PREFECT_API_DATABASE_CONNECTION_URL='postgresql+asyncpg://postgres:${PREFECT_API_DATABASE_PASSWORD}@localhost/prefect'

PREFECT_API_DATABASE_ECHO = Setting(bool, default=False) module-attribute

If True, SQLAlchemy will log all SQL issued to the database. Defaults to False.

PREFECT_API_DATABASE_MIGRATE_ON_START = Setting(bool, default=True) module-attribute

If True, the database will be upgraded on application creation. If False, the database will need to be upgraded manually.

PREFECT_API_DATABASE_TIMEOUT = Setting(Optional[float], default=10.0) module-attribute

A statement timeout, in seconds, applied to all database interactions made by the API. Defaults to 10 seconds.

PREFECT_API_DATABASE_CONNECTION_TIMEOUT = Setting(Optional[float], default=5) module-attribute

A connection timeout, in seconds, applied to database connections. Defaults to 5.

PREFECT_API_SERVICES_SCHEDULER_LOOP_SECONDS = Setting(float, default=60) module-attribute

The scheduler loop interval, in seconds. This determines how often the scheduler will attempt to schedule new flow runs, but has no impact on how quickly either flow runs or task runs are actually executed. Defaults to 60.

PREFECT_API_SERVICES_SCHEDULER_DEPLOYMENT_BATCH_SIZE = Setting(int, default=100) module-attribute

The number of deployments the scheduler will attempt to schedule in a single batch. If there are more deployments than the batch size, the scheduler immediately attempts to schedule the next batch; it does not sleep for scheduler_loop_seconds until it has visited every deployment once. Defaults to 100.

PREFECT_API_SERVICES_SCHEDULER_MAX_RUNS = Setting(int, default=100) module-attribute

The scheduler will attempt to schedule up to this many auto-scheduled runs in the future. Note that runs may have fewer than this many scheduled runs, depending on the value of scheduler_max_scheduled_time. Defaults to 100.

PREFECT_API_SERVICES_SCHEDULER_MIN_RUNS = Setting(int, default=3) module-attribute

The scheduler will attempt to schedule at least this many auto-scheduled runs in the future. Note that runs may have more than this many scheduled runs, depending on the value of scheduler_min_scheduled_time. Defaults to 3.

PREFECT_API_SERVICES_SCHEDULER_MAX_SCHEDULED_TIME = Setting(timedelta, default=timedelta(days=100)) module-attribute

The scheduler will create new runs up to this far in the future. Note that this setting will take precedence over scheduler_max_runs: if a flow runs once a month and scheduler_max_scheduled_time is three months, then only three runs will be scheduled. Defaults to 100 days (8640000 seconds).

PREFECT_API_SERVICES_SCHEDULER_MIN_SCHEDULED_TIME = Setting(timedelta, default=timedelta(hours=1)) module-attribute

The scheduler will create new runs at least this far in the future. Note that this setting will take precedence over scheduler_min_runs: if a flow runs every hour and scheduler_min_scheduled_time is three hours, then three runs will be scheduled even if scheduler_min_runs is 1. Defaults to 1 hour (3600 seconds).

PREFECT_API_SERVICES_SCHEDULER_INSERT_BATCH_SIZE = Setting(int, default=500) module-attribute

The number of flow runs the scheduler will attempt to insert in one batch across all deployments. If the number of flow runs to schedule exceeds this amount, the runs will be inserted in batches of this size. Defaults to 500.

PREFECT_API_SERVICES_LATE_RUNS_LOOP_SECONDS = Setting(float, default=5) module-attribute

The late runs service will look for runs to mark as late this often. Defaults to 5.

PREFECT_API_SERVICES_LATE_RUNS_AFTER_SECONDS = Setting(timedelta, default=timedelta(seconds=5)) module-attribute

The late runs service will mark runs as late after they have exceeded their scheduled start time by this many seconds. Defaults to 5 seconds.

PREFECT_API_SERVICES_PAUSE_EXPIRATIONS_LOOP_SECONDS = Setting(float, default=5) module-attribute

The pause expiration service will look for runs to mark as failed this often. Defaults to 5.

PREFECT_API_SERVICES_CANCELLATION_CLEANUP_LOOP_SECONDS = Setting(float, default=20) module-attribute

The cancellation cleanup service will look non-terminal tasks and subflows this often. Defaults to 20.

PREFECT_API_DEFAULT_LIMIT = Setting(int, default=200) module-attribute

The default limit applied to queries that can return multiple objects, such as POST /flow_runs/filter.

PREFECT_SERVER_API_HOST = Setting(str, default='127.0.0.1') module-attribute

The API's host address (defaults to 127.0.0.1).

PREFECT_SERVER_API_PORT = Setting(int, default=4200) module-attribute

The API's port address (defaults to 4200).

PREFECT_SERVER_API_KEEPALIVE_TIMEOUT = Setting(int, default=5) module-attribute

The API's keep alive timeout (defaults to 5). Refer to https://www.uvicorn.org/settings/#timeouts for details.

When the API is hosted behind a load balancer, you may want to set this to a value greater than the load balancer's idle timeout.

Note this setting only applies when calling prefect server start; if hosting the API with another tool you will need to configure this there instead.

PREFECT_UI_ENABLED = Setting(bool, default=True) module-attribute

Whether or not to serve the Prefect UI.

PREFECT_UI_API_URL = Setting(str, default=None, value_callback=default_ui_api_url) module-attribute

The connection url for communication from the UI to the API. Defaults to PREFECT_API_URL if set. Otherwise, the default URL is generated from PREFECT_SERVER_API_HOST and PREFECT_SERVER_API_PORT. If providing a custom value, the aforementioned settings may be templated into the given string.

PREFECT_SERVER_ANALYTICS_ENABLED = Setting(bool, default=True) module-attribute

When enabled, Prefect sends anonymous data (e.g. count of flow runs, package version) on server startup to help us improve our product.

PREFECT_API_SERVICES_SCHEDULER_ENABLED = Setting(bool, default=True) module-attribute

Whether or not to start the scheduling service in the server application. If disabled, you will need to run this service separately to schedule runs for deployments.

PREFECT_API_SERVICES_LATE_RUNS_ENABLED = Setting(bool, default=True) module-attribute

Whether or not to start the late runs service in the server application. If disabled, you will need to run this service separately to have runs past their scheduled start time marked as late.

PREFECT_API_SERVICES_FLOW_RUN_NOTIFICATIONS_ENABLED = Setting(bool, default=True) module-attribute

Whether or not to start the flow run notifications service in the server application. If disabled, you will need to run this service separately to send flow run notifications.

PREFECT_API_SERVICES_PAUSE_EXPIRATIONS_ENABLED = Setting(bool, default=True) module-attribute

Whether or not to start the paused flow run expiration service in the server application. If disabled, paused flows that have timed out will remain in a Paused state until a resume attempt.

PREFECT_API_TASK_CACHE_KEY_MAX_LENGTH = Setting(int, default=2000) module-attribute

The maximum number of characters allowed for a task run cache key. This setting cannot be changed client-side, it must be set on the server.

PREFECT_API_SERVICES_CANCELLATION_CLEANUP_ENABLED = Setting(bool, default=True) module-attribute

Whether or not to start the cancellation cleanup service in the server application. If disabled, task runs and subflow runs belonging to cancelled flows may remain in non-terminal states.

PREFECT_EXPERIMENTAL_ENABLE_EVENTS_CLIENT = Setting(bool, default=True) module-attribute

Whether or not to enable experimental Prefect work pools.

PREFECT_EXPERIMENTAL_WARN_EVENTS_CLIENT = Setting(bool, default=False) module-attribute

Whether or not to warn when experimental Prefect work pools are used.

PREFECT_EXPERIMENTAL_ENABLE_WORK_POOLS = Setting(bool, default=True) module-attribute

Whether or not to enable experimental Prefect work pools.

PREFECT_EXPERIMENTAL_WARN_WORK_POOLS = Setting(bool, default=False) module-attribute

Whether or not to warn when experimental Prefect work pools are used.

PREFECT_EXPERIMENTAL_ENABLE_WORKERS = Setting(bool, default=True) module-attribute

Whether or not to enable experimental Prefect workers.

PREFECT_EXPERIMENTAL_WARN_WORKERS = Setting(bool, default=False) module-attribute

Whether or not to warn when experimental Prefect workers are used.

PREFECT_EXPERIMENTAL_WARN_VISUALIZE = Setting(bool, default=False) module-attribute

Whether or not to warn when experimental Prefect visualize is used.

PREFECT_RUNNER_PROCESS_LIMIT = Setting(int, default=5) module-attribute

Maximum number of processes a runner will execute in parallel.

PREFECT_WORKER_HEARTBEAT_SECONDS = Setting(float, default=30) module-attribute

Number of seconds a worker should wait between sending a heartbeat.

PREFECT_WORKER_QUERY_SECONDS = Setting(float, default=10) module-attribute

Number of seconds a worker should wait between queries for scheduled flow runs.

PREFECT_WORKER_PREFETCH_SECONDS = Setting(float, default=10) module-attribute

The number of seconds into the future a worker should query for scheduled flow runs. Can be used to compensate for infrastructure start up time for a worker.

PREFECT_EXPERIMENTAL_ENABLE_ARTIFACTS = Setting(bool, default=True) module-attribute

Whether or not to enable experimental Prefect artifacts.

PREFECT_EXPERIMENTAL_WARN_ARTIFACTS = Setting(bool, default=False) module-attribute

Whether or not to warn when experimental Prefect artifacts are used.

PREFECT_EXPERIMENTAL_ENABLE_WORKSPACE_DASHBOARD = Setting(bool, default=True) module-attribute

Whether or not to enable the experimental workspace dashboard.

PREFECT_EXPERIMENTAL_WARN_WORKSPACE_DASHBOARD = Setting(bool, default=False) module-attribute

Whether or not to warn when the experimental workspace dashboard is enabled.

Setting

Bases: Generic[T]

Setting definition type.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
class Setting(Generic[T]):
    """
    Setting definition type.
    """

    def __init__(
        self,
        type: Type[T],
        *,
        deprecated: bool = False,
        deprecated_start_date: Optional[str] = None,
        deprecated_end_date: Optional[str] = None,
        deprecated_help: str = "",
        deprecated_when_message: str = "",
        deprecated_when: Optional[Callable[[Any], bool]] = None,
        deprecated_renamed_to: Optional["Setting"] = None,
        value_callback: Callable[["Settings", T], T] = None,
        is_secret: bool = False,
        **kwargs,
    ) -> None:
        self.field: pydantic.fields.FieldInfo = Field(**kwargs)
        self.type = type
        self.value_callback = value_callback
        self._name = None
        self.is_secret = is_secret
        self.deprecated = deprecated
        self.deprecated_start_date = deprecated_start_date
        self.deprecated_end_date = deprecated_end_date
        self.deprecated_help = deprecated_help
        self.deprecated_when = deprecated_when or (lambda _: True)
        self.deprecated_when_message = deprecated_when_message
        self.deprecated_renamed_to = deprecated_renamed_to
        self.deprecated_renamed_from = None
        self.__doc__ = self.field.description

        # Validate the deprecation settings, will throw an error at setting definition
        # time if the developer has not configured it correctly
        if deprecated:
            generate_deprecation_message(
                name="...",  # setting names not populated until after init
                start_date=self.deprecated_start_date,
                end_date=self.deprecated_end_date,
                help=self.deprecated_help,
                when=self.deprecated_when_message,
            )

        if deprecated_renamed_to is not None:
            # Track the deprecation both ways
            deprecated_renamed_to.deprecated_renamed_from = self

    def value(self, bypass_callback: bool = False) -> T:
        """
        Get the current value of a setting.

        Example:
        ```python
        from prefect.settings import PREFECT_API_URL
        PREFECT_API_URL.value()
        ```
        """
        return self.value_from(get_current_settings(), bypass_callback=bypass_callback)

    def value_from(self, settings: "Settings", bypass_callback: bool = False) -> T:
        """
        Get the value of a setting from a settings object

        Example:
        ```python
        from prefect.settings import get_default_settings
        PREFECT_API_URL.value_from(get_default_settings())
        ```
        """
        value = settings.value_of(self, bypass_callback=bypass_callback)

        if not bypass_callback and self.deprecated and self.deprecated_when(value):
            # Check if this setting is deprecated and someone is accessing the value
            # via the old name
            warnings.warn(self.deprecated_message, DeprecationWarning, stacklevel=3)

            # If the the value is empty, return the new setting's value for compat
            if value is None and self.deprecated_renamed_to is not None:
                return self.deprecated_renamed_to.value_from(settings)

        if not bypass_callback and self.deprecated_renamed_from is not None:
            # Check if this setting is a rename of a deprecated setting and the
            # deprecated setting is set and should be used for compatibility
            deprecated_value = self.deprecated_renamed_from.value_from(
                settings, bypass_callback=True
            )
            if deprecated_value is not None:
                warnings.warn(
                    (
                        f"{self.deprecated_renamed_from.deprecated_message} Because"
                        f" {self.deprecated_renamed_from.name!r} is set it will be used"
                        f" instead of {self.name!r} for backwards compatibility."
                    ),
                    DeprecationWarning,
                    stacklevel=3,
                )
            return deprecated_value or value

        return value

    @property
    def name(self):
        if self._name:
            return self._name

        # Lookup the name on first access
        for name, val in tuple(globals().items()):
            if val == self:
                self._name = name
                return name

        raise ValueError("Setting not found in `prefect.settings` module.")

    @name.setter
    def name(self, value: str):
        self._name = value

    @property
    def deprecated_message(self):
        return generate_deprecation_message(
            name=f"Setting {self.name!r}",
            start_date=self.deprecated_start_date,
            end_date=self.deprecated_end_date,
            help=self.deprecated_help,
            when=self.deprecated_when_message,
        )

    def __repr__(self) -> str:
        return f"<{self.name}: {self.type.__name__}>"

    def __bool__(self) -> bool:
        """
        Returns a truthy check of the current value.
        """
        return bool(self.value())

    def __eq__(self, __o: object) -> bool:
        return __o.__eq__(self.value())

    def __hash__(self) -> int:
        return hash((type(self), self.name))

value

Get the current value of a setting.

Example: 

from prefect.settings import PREFECT_API_URL
PREFECT_API_URL.value()

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
132
133
134
135
136
137
138
139
140
141
142
def value(self, bypass_callback: bool = False) -> T:
    """
    Get the current value of a setting.

    Example:
    ```python
    from prefect.settings import PREFECT_API_URL
    PREFECT_API_URL.value()
    ```
    """
    return self.value_from(get_current_settings(), bypass_callback=bypass_callback)

value_from

Get the value of a setting from a settings object

Example: 

from prefect.settings import get_default_settings
PREFECT_API_URL.value_from(get_default_settings())

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
def value_from(self, settings: "Settings", bypass_callback: bool = False) -> T:
    """
    Get the value of a setting from a settings object

    Example:
    ```python
    from prefect.settings import get_default_settings
    PREFECT_API_URL.value_from(get_default_settings())
    ```
    """
    value = settings.value_of(self, bypass_callback=bypass_callback)

    if not bypass_callback and self.deprecated and self.deprecated_when(value):
        # Check if this setting is deprecated and someone is accessing the value
        # via the old name
        warnings.warn(self.deprecated_message, DeprecationWarning, stacklevel=3)

        # If the the value is empty, return the new setting's value for compat
        if value is None and self.deprecated_renamed_to is not None:
            return self.deprecated_renamed_to.value_from(settings)

    if not bypass_callback and self.deprecated_renamed_from is not None:
        # Check if this setting is a rename of a deprecated setting and the
        # deprecated setting is set and should be used for compatibility
        deprecated_value = self.deprecated_renamed_from.value_from(
            settings, bypass_callback=True
        )
        if deprecated_value is not None:
            warnings.warn(
                (
                    f"{self.deprecated_renamed_from.deprecated_message} Because"
                    f" {self.deprecated_renamed_from.name!r} is set it will be used"
                    f" instead of {self.name!r} for backwards compatibility."
                ),
                DeprecationWarning,
                stacklevel=3,
            )
        return deprecated_value or value

    return value

Settings

Bases: SettingsFieldsMixin

Contains validated Prefect settings.

Settings should be accessed using the relevant Setting object. For example: 

from prefect.settings import PREFECT_HOME
PREFECT_HOME.value()

Accessing a setting attribute directly will ignore any value_callback mutations. This is not recommended: 

from prefect.settings import Settings
Settings().PREFECT_PROFILES_PATH  # PosixPath('${PREFECT_HOME}/profiles.toml')

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
@add_cloudpickle_reduction
class Settings(SettingsFieldsMixin):
    """
    Contains validated Prefect settings.

    Settings should be accessed using the relevant `Setting` object. For example:
    ```python
    from prefect.settings import PREFECT_HOME
    PREFECT_HOME.value()
    ```

    Accessing a setting attribute directly will ignore any `value_callback` mutations.
    This is not recommended:
    ```python
    from prefect.settings import Settings
    Settings().PREFECT_PROFILES_PATH  # PosixPath('${PREFECT_HOME}/profiles.toml')
    ```
    """

    def value_of(self, setting: Setting[T], bypass_callback: bool = False) -> T:
        """
        Retrieve a setting's value.
        """
        value = getattr(self, setting.name)
        if setting.value_callback and not bypass_callback:
            value = setting.value_callback(self, value)
        return value

    @validator(PREFECT_LOGGING_LEVEL.name, PREFECT_LOGGING_SERVER_LEVEL.name)
    def check_valid_log_level(cls, value):
        if isinstance(value, str):
            value = value.upper()
        logging._checkLevel(value)
        return value

    @root_validator
    def post_root_validators(cls, values):
        """
        Add root validation functions for settings here.
        """
        # TODO: We could probably register these dynamically but this is the simpler
        #       approach for now. We can explore more interesting validation features
        #       in the future.
        values = max_log_size_smaller_than_batch_size(values)
        values = warn_on_database_password_value_without_usage(values)
        values = warn_on_misconfigured_api_url(values)
        return values

    def copy_with_update(
        self,
        updates: Mapping[Setting, Any] = None,
        set_defaults: Mapping[Setting, Any] = None,
        restore_defaults: Iterable[Setting] = None,
    ) -> "Settings":
        """
        Create a new `Settings` object with validation.

        Arguments:
            updates: A mapping of settings to new values. Existing values for the
                given settings will be overridden.
            set_defaults: A mapping of settings to new default values. Existing values for
                the given settings will only be overridden if they were not set.
            restore_defaults: An iterable of settings to restore to their default values.

        Returns:
            A new `Settings` object.
        """
        updates = updates or {}
        set_defaults = set_defaults or {}
        restore_defaults = restore_defaults or set()
        restore_defaults_names = {setting.name for setting in restore_defaults}

        return self.__class__(
            **{
                **{setting.name: value for setting, value in set_defaults.items()},
                **self.dict(exclude_unset=True, exclude=restore_defaults_names),
                **{setting.name: value for setting, value in updates.items()},
            }
        )

    def with_obfuscated_secrets(self):
        """
        Returns a copy of this settings object with secret setting values obfuscated.
        """
        settings = self.copy(
            update={
                setting.name: obfuscate(self.value_of(setting))
                for setting in SETTING_VARIABLES.values()
                if setting.is_secret
                # Exclude deprecated settings with null values to avoid warnings
                and not (setting.deprecated and self.value_of(setting) is None)
            }
        )
        # Ensure that settings that have not been marked as "set" before are still so
        # after we have updated their value above
        settings.__fields_set__.intersection_update(self.__fields_set__)
        return settings

    def to_environment_variables(
        self, include: Iterable[Setting] = None, exclude_unset: bool = False
    ) -> Dict[str, str]:
        """
        Convert the settings object to environment variables.

        Note that setting values will not be run through their `value_callback` allowing
        dynamic resolution to occur when loaded from the returned environment.

        Args:
            include_keys: An iterable of settings to include in the return value.
                If not set, all settings are used.
            exclude_unset: Only include settings that have been set (i.e. the value is
                not from the default). If set, unset keys will be dropped even if they
                are set in `include_keys`.

        Returns:
            A dictionary of settings with values cast to strings
        """
        include = set(include or SETTING_VARIABLES.values())

        if exclude_unset:
            set_keys = {
                # Collect all of the "set" keys and cast to `Setting` objects
                SETTING_VARIABLES[key]
                for key in self.dict(exclude_unset=True)
            }
            include.intersection_update(set_keys)

        # Validate the types of items in `include` to prevent exclusion bugs
        for key in include:
            if not isinstance(key, Setting):
                raise TypeError(
                    "Invalid type {type(key).__name__!r} for key in `include`."
                )

        env = {
            # Use `getattr` instead of `value_of` to avoid value callback resolution
            key: getattr(self, key)
            for key, setting in SETTING_VARIABLES.items()
            if setting in include
        }

        # Cast to strings and drop null values
        return {key: str(value) for key, value in env.items() if value is not None}

    class Config:
        frozen = True

value_of

Retrieve a setting's value.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1366
1367
1368
1369
1370
1371
1372
1373
def value_of(self, setting: Setting[T], bypass_callback: bool = False) -> T:
    """
    Retrieve a setting's value.
    """
    value = getattr(self, setting.name)
    if setting.value_callback and not bypass_callback:
        value = setting.value_callback(self, value)
    return value

post_root_validators

Add root validation functions for settings here.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
@root_validator
def post_root_validators(cls, values):
    """
    Add root validation functions for settings here.
    """
    # TODO: We could probably register these dynamically but this is the simpler
    #       approach for now. We can explore more interesting validation features
    #       in the future.
    values = max_log_size_smaller_than_batch_size(values)
    values = warn_on_database_password_value_without_usage(values)
    values = warn_on_misconfigured_api_url(values)
    return values

with_obfuscated_secrets

Returns a copy of this settings object with secret setting values obfuscated.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
def with_obfuscated_secrets(self):
    """
    Returns a copy of this settings object with secret setting values obfuscated.
    """
    settings = self.copy(
        update={
            setting.name: obfuscate(self.value_of(setting))
            for setting in SETTING_VARIABLES.values()
            if setting.is_secret
            # Exclude deprecated settings with null values to avoid warnings
            and not (setting.deprecated and self.value_of(setting) is None)
        }
    )
    # Ensure that settings that have not been marked as "set" before are still so
    # after we have updated their value above
    settings.__fields_set__.intersection_update(self.__fields_set__)
    return settings

to_environment_variables

Convert the settings object to environment variables.

Note that setting values will not be run through their value_callback allowing dynamic resolution to occur when loaded from the returned environment.

Parameters:

Name Type Description Default
include_keys

An iterable of settings to include in the return value. If not set, all settings are used.

 required
exclude_unset bool

Only include settings that have been set (i.e. the value is not from the default). If set, unset keys will be dropped even if they are set in include_keys.

 False

Returns:

Type Description
Dict[str, str]

A dictionary of settings with values cast to strings
Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
def to_environment_variables(
    self, include: Iterable[Setting] = None, exclude_unset: bool = False
) -> Dict[str, str]:
    """
    Convert the settings object to environment variables.

    Note that setting values will not be run through their `value_callback` allowing
    dynamic resolution to occur when loaded from the returned environment.

    Args:
        include_keys: An iterable of settings to include in the return value.
            If not set, all settings are used.
        exclude_unset: Only include settings that have been set (i.e. the value is
            not from the default). If set, unset keys will be dropped even if they
            are set in `include_keys`.

    Returns:
        A dictionary of settings with values cast to strings
    """
    include = set(include or SETTING_VARIABLES.values())

    if exclude_unset:
        set_keys = {
            # Collect all of the "set" keys and cast to `Setting` objects
            SETTING_VARIABLES[key]
            for key in self.dict(exclude_unset=True)
        }
        include.intersection_update(set_keys)

    # Validate the types of items in `include` to prevent exclusion bugs
    for key in include:
        if not isinstance(key, Setting):
            raise TypeError(
                "Invalid type {type(key).__name__!r} for key in `include`."
            )

    env = {
        # Use `getattr` instead of `value_of` to avoid value callback resolution
        key: getattr(self, key)
        for key, setting in SETTING_VARIABLES.items()
        if setting in include
    }

    # Cast to strings and drop null values
    return {key: str(value) for key, value in env.items() if value is not None}

Profile

Bases: BaseModel

A user profile containing settings.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
class Profile(pydantic.BaseModel):
    """
    A user profile containing settings.
    """

    name: str
    settings: Dict[Setting, Any] = Field(default_factory=dict)
    source: Optional[Path]

    @pydantic.validator("settings", pre=True)
    def map_names_to_settings(cls, value):
        if value is None:
            return value

        # Cast string setting names to variables
        validated = {}
        for setting, val in value.items():
            if isinstance(setting, str) and setting in SETTING_VARIABLES:
                validated[SETTING_VARIABLES[setting]] = val
            elif isinstance(setting, Setting):
                validated[setting] = val
            else:
                raise ValueError(f"Unknown setting {setting!r}.")

        return validated

    def validate_settings(self) -> None:
        """
        Validate the settings contained in this profile.

        Raises:
            pydantic.ValidationError: When settings do not have valid values.
        """
        # Create a new `Settings` instance with the settings from this profile relying
        # on Pydantic validation to raise an error.
        # We do not return the `Settings` object because this is not the recommended
        # path for constructing settings with a profile. See `use_profile` instead.
        Settings(**{setting.name: value for setting, value in self.settings.items()})

    def convert_deprecated_renamed_settings(self) -> List[Tuple[Setting, Setting]]:
        """
        Update settings in place to replace deprecated settings with new settings when
        renamed.

        Returns a list of tuples with the old and new setting.
        """
        changed = []
        for setting in tuple(self.settings):
            if (
                setting.deprecated
                and setting.deprecated_renamed_to
                and setting.deprecated_renamed_to not in self.settings
            ):
                self.settings[setting.deprecated_renamed_to] = self.settings.pop(
                    setting
                )
                changed.append((setting, setting.deprecated_renamed_to))
        return changed

    class Config:
        arbitrary_types_allowed = True

validate_settings

Validate the settings contained in this profile.

Raises:

Type Description
ValidationError

When settings do not have valid values.
Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
def validate_settings(self) -> None:
    """
    Validate the settings contained in this profile.

    Raises:
        pydantic.ValidationError: When settings do not have valid values.
    """
    # Create a new `Settings` instance with the settings from this profile relying
    # on Pydantic validation to raise an error.
    # We do not return the `Settings` object because this is not the recommended
    # path for constructing settings with a profile. See `use_profile` instead.
    Settings(**{setting.name: value for setting, value in self.settings.items()})

convert_deprecated_renamed_settings

Update settings in place to replace deprecated settings with new settings when renamed.

Returns a list of tuples with the old and new setting.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
def convert_deprecated_renamed_settings(self) -> List[Tuple[Setting, Setting]]:
    """
    Update settings in place to replace deprecated settings with new settings when
    renamed.

    Returns a list of tuples with the old and new setting.
    """
    changed = []
    for setting in tuple(self.settings):
        if (
            setting.deprecated
            and setting.deprecated_renamed_to
            and setting.deprecated_renamed_to not in self.settings
        ):
            self.settings[setting.deprecated_renamed_to] = self.settings.pop(
                setting
            )
            changed.append((setting, setting.deprecated_renamed_to))
    return changed

ProfilesCollection

" A utility class for working with a collection of profiles.

Profiles in the collection must have unique names.

The collection may store the name of the active profile.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
class ProfilesCollection:
    """ "
    A utility class for working with a collection of profiles.

    Profiles in the collection must have unique names.

    The collection may store the name of the active profile.
    """

    def __init__(
        self, profiles: Iterable[Profile], active: Optional[str] = None
    ) -> None:
        self.profiles_by_name = {profile.name: profile for profile in profiles}
        self.active_name = active

    @property
    def names(self) -> Set[str]:
        """
        Return a set of profile names in this collection.
        """
        return set(self.profiles_by_name.keys())

    @property
    def active_profile(self) -> Optional[Profile]:
        """
        Retrieve the active profile in this collection.
        """
        if self.active_name is None:
            return None
        return self[self.active_name]

    def set_active(self, name: Optional[str], check: bool = True):
        """
        Set the active profile name in the collection.

        A null value may be passed to indicate that this collection does not determine
        the active profile.
        """
        if check and name is not None and name not in self.names:
            raise ValueError(f"Unknown profile name {name!r}.")
        self.active_name = name

    def update_profile(
        self, name: str, settings: Mapping[Union[Dict, str], Any], source: Path = None
    ) -> Profile:
        """
        Add a profile to the collection or update the existing on if the name is already
        present in this collection.

        If updating an existing profile, the settings will be merged. Settings can
        be dropped from the existing profile by setting them to `None` in the new
        profile.

        Returns the new profile object.
        """
        existing = self.profiles_by_name.get(name)

        # Convert the input to a `Profile` to cast settings to the correct type
        profile = Profile(name=name, settings=settings, source=source)

        if existing:
            new_settings = {**existing.settings, **profile.settings}

            # Drop null keys to restore to default
            for key, value in tuple(new_settings.items()):
                if value is None:
                    new_settings.pop(key)

            new_profile = Profile(
                name=profile.name,
                settings=new_settings,
                source=source or profile.source,
            )
        else:
            new_profile = profile

        self.profiles_by_name[new_profile.name] = new_profile

        return new_profile

    def add_profile(self, profile: Profile) -> None:
        """
        Add a profile to the collection.

        If the profile name already exists, an exception will be raised.
        """
        if profile.name in self.profiles_by_name:
            raise ValueError(
                f"Profile name {profile.name!r} already exists in collection."
            )

        self.profiles_by_name[profile.name] = profile

    def remove_profile(self, name: str) -> None:
        """
        Remove a profile from the collection.
        """
        self.profiles_by_name.pop(name)

    def without_profile_source(self, path: Optional[Path]) -> "ProfilesCollection":
        """
        Remove profiles that were loaded from a given path.

        Returns a new collection.
        """
        return ProfilesCollection(
            [
                profile
                for profile in self.profiles_by_name.values()
                if profile.source != path
            ],
            active=self.active_name,
        )

    def to_dict(self):
        """
        Convert to a dictionary suitable for writing to disk.
        """
        return {
            "active": self.active_name,
            "profiles": {
                profile.name: {
                    setting.name: value for setting, value in profile.settings.items()
                }
                for profile in self.profiles_by_name.values()
            },
        }

    def __getitem__(self, name: str) -> Profile:
        return self.profiles_by_name[name]

    def __iter__(self):
        return self.profiles_by_name.__iter__()

    def items(self):
        return self.profiles_by_name.items()

    def __eq__(self, __o: object) -> bool:
        if not isinstance(__o, ProfilesCollection):
            return False

        return (
            self.profiles_by_name == __o.profiles_by_name
            and self.active_name == __o.active_name
        )

    def __repr__(self) -> str:
        return (
            f"ProfilesCollection(profiles={list(self.profiles_by_name.values())!r},"
            f" active={self.active_name!r})>"
        )

names: Set[str] property

Return a set of profile names in this collection.

active_profile: Optional[Profile] property

Retrieve the active profile in this collection.

set_active

Set the active profile name in the collection.

A null value may be passed to indicate that this collection does not determine the active profile.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
def set_active(self, name: Optional[str], check: bool = True):
    """
    Set the active profile name in the collection.

    A null value may be passed to indicate that this collection does not determine
    the active profile.
    """
    if check and name is not None and name not in self.names:
        raise ValueError(f"Unknown profile name {name!r}.")
    self.active_name = name

update_profile

Add a profile to the collection or update the existing on if the name is already present in this collection.

If updating an existing profile, the settings will be merged. Settings can be dropped from the existing profile by setting them to None in the new profile.

Returns the new profile object.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
def update_profile(
    self, name: str, settings: Mapping[Union[Dict, str], Any], source: Path = None
) -> Profile:
    """
    Add a profile to the collection or update the existing on if the name is already
    present in this collection.

    If updating an existing profile, the settings will be merged. Settings can
    be dropped from the existing profile by setting them to `None` in the new
    profile.

    Returns the new profile object.
    """
    existing = self.profiles_by_name.get(name)

    # Convert the input to a `Profile` to cast settings to the correct type
    profile = Profile(name=name, settings=settings, source=source)

    if existing:
        new_settings = {**existing.settings, **profile.settings}

        # Drop null keys to restore to default
        for key, value in tuple(new_settings.items()):
            if value is None:
                new_settings.pop(key)

        new_profile = Profile(
            name=profile.name,
            settings=new_settings,
            source=source or profile.source,
        )
    else:
        new_profile = profile

    self.profiles_by_name[new_profile.name] = new_profile

    return new_profile

add_profile

Add a profile to the collection.

If the profile name already exists, an exception will be raised.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
def add_profile(self, profile: Profile) -> None:
    """
    Add a profile to the collection.

    If the profile name already exists, an exception will be raised.
    """
    if profile.name in self.profiles_by_name:
        raise ValueError(
            f"Profile name {profile.name!r} already exists in collection."
        )

    self.profiles_by_name[profile.name] = profile

remove_profile

Remove a profile from the collection.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1753
1754
1755
1756
1757
def remove_profile(self, name: str) -> None:
    """
    Remove a profile from the collection.
    """
    self.profiles_by_name.pop(name)

without_profile_source

Remove profiles that were loaded from a given path.

Returns a new collection.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
def without_profile_source(self, path: Optional[Path]) -> "ProfilesCollection":
    """
    Remove profiles that were loaded from a given path.

    Returns a new collection.
    """
    return ProfilesCollection(
        [
            profile
            for profile in self.profiles_by_name.values()
            if profile.source != path
        ],
        active=self.active_name,
    )

get_extra_loggers

value_callback for PREFECT_LOGGING_EXTRA_LOGGERSthat parses the CSV string into a list and trims whitespace from logger names.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
231
232
233
234
235
236
def get_extra_loggers(_: "Settings", value: str) -> List[str]:
    """
    `value_callback` for `PREFECT_LOGGING_EXTRA_LOGGERS`that parses the CSV string into a
    list and trims whitespace from logger names.
    """
    return [name.strip() for name in value.split(",")] if value else []

debug_mode_log_level

value_callback for PREFECT_LOGGING_LEVEL that overrides the log level to DEBUG when debug mode is enabled.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
243
244
245
246
247
248
249
250
251
def debug_mode_log_level(settings, value):
    """
    `value_callback` for `PREFECT_LOGGING_LEVEL` that overrides the log level to DEBUG
    when debug mode is enabled.
    """
    if PREFECT_DEBUG_MODE.value_from(settings):
        return "DEBUG"
    else:
        return value

only_return_value_in_test_mode

value_callback for PREFECT_TEST_SETTING that only allows access during test mode

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
254
255
256
257
258
259
260
261
def only_return_value_in_test_mode(settings, value):
    """
    `value_callback` for `PREFECT_TEST_SETTING` that only allows access during test mode
    """
    if PREFECT_TEST_MODE.value_from(settings):
        return value
    else:
        return None

default_ui_api_url

value_callback for PREFECT_UI_API_URL that sets the default value to PREFECT_API_URL if set otherwise it constructs an API URL from the API settings.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
def default_ui_api_url(settings, value):
    """
    `value_callback` for `PREFECT_UI_API_URL` that sets the default value to
    `PREFECT_API_URL` if set otherwise it constructs an API URL from the API settings.
    """
    if value is None:
        # Set a default value
        if PREFECT_API_URL.value_from(settings):
            value = "${PREFECT_API_URL}"
        else:
            value = "http://${PREFECT_SERVER_API_HOST}:${PREFECT_SERVER_API_PORT}/api"

    return template_with_settings(
        PREFECT_SERVER_API_HOST, PREFECT_SERVER_API_PORT, PREFECT_API_URL
    )(settings, value)

status_codes_as_integers_in_range

value_callback for PREFECT_CLIENT_RETRY_EXTRA_CODES that ensures status codes are integers in the range 100-599.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
def status_codes_as_integers_in_range(_, value):
    """
    `value_callback` for `PREFECT_CLIENT_RETRY_EXTRA_CODES` that ensures status codes
    are integers in the range 100-599.
    """
    if value == "":
        return set()

    values = {v.strip() for v in value.split(",")}

    if any(not v.isdigit() or int(v) < 100 or int(v) > 599 for v in values):
        raise ValueError(
            "PREFECT_CLIENT_RETRY_EXTRA_CODES must be a comma separated list of "
            "integers between 100 and 599."
        )

    values = {int(v) for v in values}
    return values

template_with_settings

Returns a value_callback that will template the given settings into the runtime value for the setting.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
def template_with_settings(*upstream_settings: Setting) -> Callable[["Settings", T], T]:
    """
    Returns a `value_callback` that will template the given settings into the runtime
    value for the setting.
    """

    def templater(settings, value):
        if value is None:
            return value  # Do not attempt to template a null string

        original_type = type(value)
        template_values = {
            setting.name: setting.value_from(settings) for setting in upstream_settings
        }
        template = string.Template(str(value))
        return original_type(template.substitute(template_values))

    return templater

max_log_size_smaller_than_batch_size

Validator for settings asserting the batch size and match log size are compatible

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
321
322
323
324
325
326
327
328
329
330
331
332
333
def max_log_size_smaller_than_batch_size(values):
    """
    Validator for settings asserting the batch size and match log size are compatible
    """
    if (
        values["PREFECT_LOGGING_TO_API_BATCH_SIZE"]
        < values["PREFECT_LOGGING_TO_API_MAX_LOG_SIZE"]
    ):
        raise ValueError(
            "`PREFECT_LOGGING_TO_API_MAX_LOG_SIZE` cannot be larger than"
            " `PREFECT_LOGGING_TO_API_BATCH_SIZE`"
        )
    return values

warn_on_database_password_value_without_usage

Validator for settings warning if the database password is set but not used.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
def warn_on_database_password_value_without_usage(values):
    """
    Validator for settings warning if the database password is set but not used.
    """
    value = values["PREFECT_API_DATABASE_PASSWORD"]
    if (
        value
        and not value.startswith(OBFUSCATED_PREFIX)
        and (
            "PREFECT_API_DATABASE_PASSWORD"
            not in values["PREFECT_API_DATABASE_CONNECTION_URL"]
        )
    ):
        warnings.warn(
            "PREFECT_API_DATABASE_PASSWORD is set but not included in the "
            "PREFECT_API_DATABASE_CONNECTION_URL. "
            "The provided password will be ignored."
        )
    return values

warn_on_misconfigured_api_url

Validator for settings warning if the API URL is misconfigured.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
def warn_on_misconfigured_api_url(values):
    """
    Validator for settings warning if the API URL is misconfigured.
    """
    api_url = values["PREFECT_API_URL"]
    if api_url is not None:
        misconfigured_mappings = {
            "app.prefect.cloud": (
                "`PREFECT_API_URL` points to `app.prefect.cloud`. Did you"
                " mean `api.prefect.cloud`?"
            ),
            "account/": (
                "`PREFECT_API_URL` uses `/account/` but should use `/accounts/`."
            ),
            "workspace/": (
                "`PREFECT_API_URL` uses `/workspace/` but should use `/workspaces/`."
            ),
        }
        warnings_list = []

        for misconfig, warning in misconfigured_mappings.items():
            if misconfig in api_url:
                warnings_list.append(warning)

        parsed_url = urlparse(api_url)
        if parsed_url.path and not parsed_url.path.startswith("/api"):
            warnings_list.append(
                "`PREFECT_API_URL` should have `/api` after the base URL."
            )

        if warnings_list:
            example = (
                'e.g. PREFECT_API_URL="https://api.prefect.cloud/api/accounts/[ACCOUNT-ID]/workspaces/[WORKSPACE-ID]"'
            )
            warnings_list.append(example)

            warnings.warn("\n".join(warnings_list), stacklevel=2)

    return values

get_current_settings

Returns a settings object populated with values from the current settings context or, if no settings context is active, the environment.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
def get_current_settings() -> Settings:
    """
    Returns a settings object populated with values from the current settings context
    or, if no settings context is active, the environment.
    """
    from prefect.context import SettingsContext

    settings_context = SettingsContext.get()
    if settings_context is not None:
        return settings_context.settings

    return get_settings_from_env()

get_settings_from_env

Returns a settings object populated with default values and overrides from environment variables, ignoring any values in profiles.

Calls with the same environment return a cached object instead of reconstructing to avoid validation overhead.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
def get_settings_from_env() -> Settings:
    """
    Returns a settings object populated with default values and overrides from
    environment variables, ignoring any values in profiles.

    Calls with the same environment return a cached object instead of reconstructing
    to avoid validation overhead.
    """
    # Since os.environ is a Dict[str, str] we can safely hash it by contents, but we
    # must be careful to avoid hashing a generator instead of a tuple
    cache_key = hash(tuple((key, value) for key, value in os.environ.items()))

    if cache_key not in _FROM_ENV_CACHE:
        _FROM_ENV_CACHE[cache_key] = Settings()

    return _FROM_ENV_CACHE[cache_key]

get_default_settings

Returns a settings object populated with default values, ignoring any overrides from environment variables or profiles.

This is cached since the defaults should not change during the lifetime of the module.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
def get_default_settings() -> Settings:
    """
    Returns a settings object populated with default values, ignoring any overrides
    from environment variables or profiles.

    This is cached since the defaults should not change during the lifetime of the
    module.
    """
    global _DEFAULTS_CACHE

    if not _DEFAULTS_CACHE:
        old = os.environ
        try:
            os.environ = {}
            settings = get_settings_from_env()
        finally:
            os.environ = old

        _DEFAULTS_CACHE = settings

    return _DEFAULTS_CACHE

temporary_settings

Temporarily override the current settings by entering a new profile.

See Settings.copy_with_update for details on different argument behavior.

Examples:

>>> from prefect.settings import PREFECT_API_URL
>>>
>>> with temporary_settings(updates={PREFECT_API_URL: "foo"}):
>>>    assert PREFECT_API_URL.value() == "foo"
>>>
>>>    with temporary_settings(set_defaults={PREFECT_API_URL: "bar"}):
>>>         assert PREFECT_API_URL.value() == "foo"
>>>
>>>    with temporary_settings(restore_defaults={PREFECT_API_URL}):
>>>         assert PREFECT_API_URL.value() is None
>>>
>>>         with temporary_settings(set_defaults={PREFECT_API_URL: "bar"})
>>>             assert PREFECT_API_URL.value() == "bar"
>>> assert PREFECT_API_URL.value() is None
Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
@contextmanager
def temporary_settings(
    updates: Mapping[Setting, Any] = None,
    set_defaults: Mapping[Setting, Any] = None,
    restore_defaults: Iterable[Setting] = None,
) -> Settings:
    """
    Temporarily override the current settings by entering a new profile.

    See `Settings.copy_with_update` for details on different argument behavior.

    Examples:
        >>> from prefect.settings import PREFECT_API_URL
        >>>
        >>> with temporary_settings(updates={PREFECT_API_URL: "foo"}):
        >>>    assert PREFECT_API_URL.value() == "foo"
        >>>
        >>>    with temporary_settings(set_defaults={PREFECT_API_URL: "bar"}):
        >>>         assert PREFECT_API_URL.value() == "foo"
        >>>
        >>>    with temporary_settings(restore_defaults={PREFECT_API_URL}):
        >>>         assert PREFECT_API_URL.value() is None
        >>>
        >>>         with temporary_settings(set_defaults={PREFECT_API_URL: "bar"})
        >>>             assert PREFECT_API_URL.value() == "bar"
        >>> assert PREFECT_API_URL.value() is None
    """
    import prefect.context

    context = prefect.context.get_settings_context()

    new_settings = context.settings.copy_with_update(
        updates=updates, set_defaults=set_defaults, restore_defaults=restore_defaults
    )

    with prefect.context.SettingsContext(
        profile=context.profile, settings=new_settings
    ):
        yield new_settings

load_profiles

Load all profiles from the default and current profile paths.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
def load_profiles() -> ProfilesCollection:
    """
    Load all profiles from the default and current profile paths.
    """
    profiles = _read_profiles_from(DEFAULT_PROFILES_PATH)

    user_profiles_path = PREFECT_PROFILES_PATH.value()
    if user_profiles_path.exists():
        user_profiles = _read_profiles_from(user_profiles_path)

        # Merge all of the user profiles with the defaults
        for name in user_profiles:
            profiles.update_profile(
                name,
                settings=user_profiles[name].settings,
                source=user_profiles[name].source,
            )

        if user_profiles.active_name:
            profiles.set_active(user_profiles.active_name, check=False)

    return profiles

load_current_profile

Load the current profile from the default and current profile paths.

This will not include settings from the current settings context. Only settings that have been persisted to the profiles file will be saved.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
def load_current_profile():
    """
    Load the current profile from the default and current profile paths.

    This will _not_ include settings from the current settings context. Only settings
    that have been persisted to the profiles file will be saved.
    """
    from prefect.context import SettingsContext

    profiles = load_profiles()
    context = SettingsContext.get()

    if context:
        profiles.set_active(context.profile.name)

    return profiles.active_profile

save_profiles

Writes all non-default profiles to the current profiles path.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1890
1891
1892
1893
1894
1895
1896
def save_profiles(profiles: ProfilesCollection) -> None:
    """
    Writes all non-default profiles to the current profiles path.
    """
    profiles_path = PREFECT_PROFILES_PATH.value()
    profiles = profiles.without_profile_source(DEFAULT_PROFILES_PATH)
    return _write_profiles_to(profiles_path, profiles)

load_profile

Load a single profile by name.

Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1899
1900
1901
1902
1903
1904
1905
1906
1907
def load_profile(name: str) -> Profile:
    """
    Load a single profile by name.
    """
    profiles = load_profiles()
    try:
        return profiles[name]
    except KeyError:
        raise ValueError(f"Profile {name!r} not found.")

update_current_profile

Update the persisted data for the profile currently in-use.

If the profile does not exist in the profiles file, it will be created.

Given settings will be merged with the existing settings as described in ProfilesCollection.update_profile.

Returns:

Type Description
Profile

The new profile.
Source code in /home/runner/work/docs/docs/prefect_source/src/prefect/settings.py 
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
def update_current_profile(settings: Dict[Union[str, Setting], Any]) -> Profile:
    """
    Update the persisted data for the profile currently in-use.

    If the profile does not exist in the profiles file, it will be created.

    Given settings will be merged with the existing settings as described in
    `ProfilesCollection.update_profile`.

    Returns:
        The new profile.
    """
    import prefect.context

    current_profile = prefect.context.get_settings_context().profile

    if not current_profile:
        raise MissingProfileError("No profile is currently in use.")

    profiles = load_profiles()

    # Ensure the current profile's settings are present
    profiles.update_profile(current_profile.name, current_profile.settings)
    # Then merge the new settings in
    new_profile = profiles.update_profile(current_profile.name, settings)

    # Validate before saving
    new_profile.validate_settings()

    save_profiles(profiles)

    return profiles[current_profile.name]