prefect_databricks.rest
¶
This is a module containing generic REST tasks.
HTTPMethod
¶
Bases: Enum
Available HTTP request methods.
Source code in prefect_databricks/rest.py
33 34 35 36 37 38 39 40 41 42 |
|
execute_endpoint
async
¶
Generic function for executing REST endpoints.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
endpoint |
str
|
The endpoint route. |
required |
databricks_credentials |
DatabricksCredentials
|
Credentials to use for authentication with Databricks. |
required |
http_method |
HTTPMethod
|
Either GET, POST, PUT, DELETE, or PATCH. |
GET
|
params |
Dict[str, Any]
|
URL query parameters in the request. |
None
|
json |
Dict[str, Any]
|
JSON serializable object to include in the body of the request. |
None
|
**kwargs |
Dict[str, Any]
|
Additional keyword arguments to pass. |
{}
|
Returns:
Type | Description |
---|---|
Response
|
The httpx.Response from interacting with the endpoint. |
Examples:
Lists jobs on the Databricks instance.
from prefect import flow
from prefect_databricks import DatabricksCredentials
from prefect_databricks.rest import execute_endpoint
@flow
def example_execute_endpoint_flow():
endpoint = "/2.1/jobs/list"
databricks_credentials = DatabricksCredentials.load("my-block")
params = {
"limit": 5,
"offset": None,
"expand_tasks": True,
}
response = execute_endpoint(
endpoint,
databricks_credentials,
params=params
)
return response.json()
Source code in prefect_databricks/rest.py
89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 |
|
serialize_model
¶
Recursively serializes pydantic.BaseModel
into JSON;
returns original obj if not a BaseModel
.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
obj |
Any
|
Input object to serialize. |
required |
Returns:
Type | Description |
---|---|
Any
|
Serialized version of object. |
Source code in prefect_databricks/rest.py
45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
|
strip_kwargs
¶
Recursively drops keyword arguments if value is None,
and serializes any pydantic.BaseModel
types.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
**kwargs |
Dict
|
Input keyword arguments. |
{}
|
Returns:
Type | Description |
---|---|
Dict
|
Stripped version of kwargs. |
Source code in prefect_databricks/rest.py
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 |
|