prefect_gcp.deployments.steps
¶
Prefect deployment steps for code storage in and retrieval from Google Cloud Storage.
PullFromGcsOutput
¶
Bases: TypedDict
The output of the pull_from_gcs
step.
Source code in prefect_gcp/deployments/steps.py
30 31 32 33 34 35 36 37 |
|
PullProjectFromGcsOutput
¶
Bases: PullFromGcsOutput
Deprecated. Use PullFromGcsOutput
instead.
Source code in prefect_gcp/deployments/steps.py
40 41 42 |
|
PushProjectToGcsOutput
¶
Bases: PushToGcsOutput
Deprecated. Use PushToGcsOutput
instead.
Source code in prefect_gcp/deployments/steps.py
25 26 27 |
|
PushToGcsOutput
¶
Bases: TypedDict
The output of the push_to_gcs
step.
Source code in prefect_gcp/deployments/steps.py
16 17 18 19 20 21 22 |
|
pull_from_gcs
¶
Pulls the contents of a project from an GCS bucket to the current working directory.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
bucket |
str
|
The name of the GCS bucket where files are stored. |
required |
folder |
str
|
The folder in the GCS bucket where files are stored. |
required |
project |
Optional[str]
|
The GCP project the bucket belongs to. If not provided, the project will be inferred from the credentials or the local environment. |
None
|
credentials |
Optional[Dict]
|
A dictionary containing the service account information and project used for authentication. If not provided, the application default credentials will be used. |
None
|
Returns:
Type | Description |
---|---|
PullProjectFromGcsOutput
|
A dictionary containing the bucket, folder, and local directory where files were downloaded. |
Examples:
Pull from GCS using the default environment credentials:
build:
- prefect_gcp.deployments.steps.pull_from_gcs:
requires: prefect-gcp
bucket: my-bucket
folder: my-folder
Pull from GCS using credentials stored in a block:
build:
- prefect_gcp.deployments.steps.pull_from_gcs:
requires: prefect-gcp
bucket: my-bucket
folder: my-folder
credentials: "{{ prefect.blocks.gcp-credentials.dev-credentials }}"
Pull from to an GCS bucket using credentials stored in a service account file:
build:
- prefect_gcp.deployments.steps.pull_from_gcs:
requires: prefect-gcp
bucket: my-bucket
folder: my-folder
credentials:
project: my-project
service_account_file: /path/to/service_account.json
Source code in prefect_gcp/deployments/steps.py
158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 |
|
pull_project_from_gcs
¶
Deprecated. Use pull_from_gcs
instead.
Source code in prefect_gcp/deployments/steps.py
253 254 255 256 257 258 |
|
push_project_to_gcs
¶
Deprecated. Use push_to_gcs
instead.
Source code in prefect_gcp/deployments/steps.py
150 151 152 153 154 155 |
|
push_to_gcs
¶
Pushes the contents of the current working directory to a GCS bucket, excluding files and folders specified in the ignore_file.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
bucket |
str
|
The name of the GCS bucket where files will be uploaded. |
required |
folder |
str
|
The folder in the GCS bucket where files will be uploaded. |
required |
project |
Optional[str]
|
The GCP project the bucket belongs to. If not provided, the project will be inferred from the credentials or the local environment. |
None
|
credentials |
Optional[Dict]
|
A dictionary containing the service account information and project used for authentication. If not provided, the application default credentials will be used. |
None
|
ignore_file |
The name of the file containing ignore patterns. |
'.prefectignore'
|
Returns:
Type | Description |
---|---|
PushToGcsOutput
|
A dictionary containing the bucket and folder where files were uploaded. |
Examples:
Push to a GCS bucket:
build:
- prefect_gcp.deployments.steps.push_to_gcs:
requires: prefect-gcp
bucket: my-bucket
folder: my-project
Push to a GCS bucket using credentials stored in a block:
build:
- prefect_gcp.deployments.steps.push_to_gcs:
requires: prefect-gcp
bucket: my-bucket
folder: my-folder
credentials: "{{ prefect.blocks.gcp-credentials.dev-credentials }}"
Push to a GCS bucket using credentials stored in a service account file:
build:
- prefect_gcp.deployments.steps.push_to_gcs:
requires: prefect-gcp
bucket: my-bucket
folder: my-folder
credentials:
project: my-project
service_account_file: /path/to/service_account.json
Source code in prefect_gcp/deployments/steps.py
45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 |
|