-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Quickstart refinements #13244
base: main
Are you sure you want to change the base?
Quickstart refinements #13244
Conversation
I think we need to smooth out our quickstart. This is just a start.
✅ Deploy Preview for prefect-docs-preview ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you Bill for the quick updates. I've added a few suggestions. And we should fix the typo before releasing.
With Prefect, you can go from a Python script to a production-ready workflow that runs remotely in a few minutes. | ||
|
||
Let's get started! | ||
Prefect is a workflow orchestration platform for building data pipelines, background jobs, LLM Chains, and more. With Prefect, you can elevate a Python script into a scheduled, observable, resilient, remotely execuateble workflow in a few minutes. Let's get started! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Small typo in executable- sorry on mobile so I can't make a suggested change.
``` | ||
|
||
Let's make this script schedulable, observable, resilient, and capable of running anywhere. | ||
Copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works. Let's make this script schedulable, observable, resilient, and capable of running anywhere. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the "let's make this" section still meant to be in here? It feels a little disjointed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's meant to be a transition from "you just have a script" to "look what Prefect can do for you". Is that transition improved by the suggested change added below?
|
||
## Step 2: Connect to Prefect's API | ||
|
||
Much of Prefect's functionality is backed by an API. | ||
The easiest way to get started is to use the API hosted by Prefect: | ||
Much of Prefect's functionality is backed by an API. You can [host Prefect server](/guides/host/) yourself if you'd like, but the easiest way to get started is to use the API hosted by Prefect Cloud: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does the mention of server here increase complexity?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this distracts from the flow. Perhaps we move the info about self-hosted instances to the Next steps section at the end of the quickstart?
Line 257 could easily be modified to include a mention of self-hosted instances.
@@ -174,10 +170,9 @@ prefect work-pool create my-managed-pool --type prefect:managed | |||
You should see a message in the CLI that your work pool was created. | |||
Feel free to check out your new work pool on the **Work Pools** page in the UI. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this line helpful? I'd vote to remove it as it pulls the user away from the main focus.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question. I think we want to get the user checking out the UI at some point, but maybe not until they run the deployment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Something I've been wondering. Can this entire workflow be completed in the UI or CLI? I think we should minimize the amount we ask people to bounce between the UI and CLI. One handoff is probably fine, multiple handoffs less so.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You could do it all in Python code + CLI. The UI is a core part of the product that I think we want folks interacting with. That said, I hear you on bouncing back and forth being distracting.
@@ -192,13 +187,11 @@ if __name__ == "__main__": | |||
entrypoint="my_gh_workflow.py:repo_info", | |||
).deploy( | |||
name="my-first-deployment", | |||
work_pool_name="my-managed-pool", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually a content from line 186 - I suggest we point out more clearly that here they're using code we've stored in GitHub.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When I went through this quickstart for the first time, I was definitely confused about how GitHub fits into it. More clarification would be awesome.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you. Minor edits and notes.
We can have Prefect Cloud run our flow code for us with a Prefect Managed work pool. | ||
|
||
Let's create a [Prefect Managed work pool](/guides/managed-execution/) so that Prefect can run our flows for us. | ||
Prefect refers to infrastructure that can run worflows as [work pools](/concepts/work-pools/). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Prefect refers to infrastructure that can run worflows as [work pools](/concepts/work-pools/). | |
Prefect refers to infrastructure that can run workflows as a [work pool](/concepts/work-pools/). |
|
||
Let's create a [Prefect Managed work pool](/guides/managed-execution/) so that Prefect can run our flows for us. | ||
Prefect refers to infrastructure that can run worflows as [work pools](/concepts/work-pools/). | ||
Prefect can orchestrate workflows that run remotely on many [types of infrastructure](/work-pools/#work-pool-types), but here we'll use a [Prefect Managed work pool](/guides/managed-execution/) so that Prefect can run this flow for us. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Prefect can orchestrate workflows that run remotely on many [types of infrastructure](/work-pools/#work-pool-types), but here we'll use a [Prefect Managed work pool](/guides/managed-execution/) so that Prefect can run this flow for us. | |
Prefect can orchestrate workflows that run remotely on many [types of infrastructure](/work-pools/#work-pool-types). We'll use a [Prefect Managed work pool](/guides/managed-execution/) so that Prefect can run this flow for us. |
@@ -174,10 +170,9 @@ prefect work-pool create my-managed-pool --type prefect:managed | |||
You should see a message in the CLI that your work pool was created. | |||
Feel free to check out your new work pool on the **Work Pools** page in the UI. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question. I think we want to get the user checking out the UI at some point, but maybe not until they run the deployment.
|
||
## Step 2: Connect to Prefect's API | ||
|
||
Much of Prefect's functionality is backed by an API. | ||
The easiest way to get started is to use the API hosted by Prefect: | ||
Much of Prefect's functionality is backed by an API. You can [host Prefect server](/guides/host/) yourself if you'd like, but the easiest way to get started is to use the API hosted by Prefect Cloud: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Much of Prefect's functionality is backed by an API. You can [host Prefect server](/guides/host/) yourself if you'd like, but the easiest way to get started is to use the API hosted by Prefect Cloud: | |
Much of Prefect's functionality is backed by an API. You can [host a Prefect server instance](/guides/host/) yourself if you'd like, but the easiest way to get started is to use the API hosted by Prefect Cloud: |
With Prefect, you can go from a Python script to a production-ready workflow that runs remotely in a few minutes. | ||
|
||
Let's get started! | ||
Prefect is a workflow orchestration platform for building data pipelines, background jobs, LLM Chains, and more. With Prefect, you can elevate a Python script into a scheduled, observable, resilient, remotely execuateble workflow in a few minutes. Let's get started! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Prefect is a workflow orchestration platform for building data pipelines, background jobs, LLM Chains, and more. With Prefect, you can elevate a Python script into a scheduled, observable, resilient, remotely execuateble workflow in a few minutes. Let's get started! | |
Prefect is a workflow orchestration platform for building data pipelines, background jobs, LLM Chains, and more. With Prefect, you can elevate a Python script into a scheduled, observable, resilient, remotely executable workflow in a few minutes. Let's get started! |
``` | ||
|
||
Let's make this script schedulable, observable, resilient, and capable of running anywhere. | ||
Copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works. Let's make this script schedulable, observable, resilient, and capable of running anywhere. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works. Let's make this script schedulable, observable, resilient, and capable of running anywhere. | |
Copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works. | |
Now let's use Prefect to make this script schedulable, observable, resilient, and capable of running anywhere. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's an extra leading space on line 53, but I think this helps to differentiate the setup from the next steps.
I might go a step further and encourage people to spin up an IDE to save them a step later when we ask them to run pip/python commands.
Copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works. Let's make this script schedulable, observable, resilient, and capable of running anywhere. | |
Create a new project in your favorite IDE, and then copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works. | |
Now let's use Prefect to make this script schedulable, observable, resilient, and capable of running anywhere. |
``` | ||
|
||
Let's make this script schedulable, observable, resilient, and capable of running anywhere. | ||
Copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works. Let's make this script schedulable, observable, resilient, and capable of running anywhere. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's meant to be a transition from "you just have a script" to "look what Prefect can do for you". Is that transition improved by the suggested change added below?
@@ -192,13 +187,11 @@ if __name__ == "__main__": | |||
entrypoint="my_gh_workflow.py:repo_info", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Making the source a Prefect repo would be good.
``` | ||
|
||
Let's make this script schedulable, observable, resilient, and capable of running anywhere. | ||
Copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works. Let's make this script schedulable, observable, resilient, and capable of running anywhere. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's an extra leading space on line 53, but I think this helps to differentiate the setup from the next steps.
I might go a step further and encourage people to spin up an IDE to save them a step later when we ask them to run pip/python commands.
Copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works. Let's make this script schedulable, observable, resilient, and capable of running anywhere. | |
Create a new project in your favorite IDE, and then copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works. | |
Now let's use Prefect to make this script schedulable, observable, resilient, and capable of running anywhere. |
With Prefect, you can go from a Python script to a production-ready workflow that runs remotely in a few minutes. | ||
|
||
Let's get started! | ||
Prefect is a workflow orchestration platform for building data pipelines, background jobs, LLM Chains, and more. With Prefect, you can elevate a Python script into a scheduled, observable, resilient, remotely execuateble workflow in a few minutes. Let's get started! | ||
|
||
## Setup |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's best to use task-based titles. For example, see https://developers.google.com/style/headings
## Setup | |
## Set up your environment |
|
||
## Step 1: Install Prefect | ||
|
||
Make sure you have Python installed in a virtual environment. Then install Prefect: | ||
|
||
```bash | ||
pip install -U prefect |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I could go either way on this. People who care about virtual environments will tend to already know how to do this, but we can save some typing by putting those commands inline.
pip install -U prefect | |
# Set up a virtual environment | |
python -m venv env | |
source env/bin/activate | |
# Install Prefect | |
pip install -U prefect |
```bash | ||
pip install -U prefect | ||
``` | ||
|
||
See the [install guide](/getting-started/installation/) for more detailed installation instructions, if needed. | ||
See the [install guide](/getting-started/installation/) for more detailed installation instructions. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or maybe the virtual environment setup commands could go here.
|
||
## Step 2: Connect to Prefect's API | ||
|
||
Much of Prefect's functionality is backed by an API. | ||
The easiest way to get started is to use the API hosted by Prefect: | ||
Much of Prefect's functionality is backed by an API. You can [host Prefect server](/guides/host/) yourself if you'd like, but the easiest way to get started is to use the API hosted by Prefect Cloud: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this distracts from the flow. Perhaps we move the info about self-hosted instances to the Next steps section at the end of the quickstart?
Line 257 could easily be modified to include a mention of self-hosted instances.
@@ -100,27 +108,20 @@ def get_contributors(repo_info: dict): | |||
contributors = response.json() | |||
return contributors | |||
|
|||
|
|||
@flow(log_prints=True) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@flow(log_prints=True) | |
# This method is the flow entrypoint | |
@flow(log_prints=True) |
@@ -130,7 +131,7 @@ python my_gh_workflow.py | |||
|
|||
</div> | |||
|
|||
Now when we run this script, Prefect will automatically track the state of the flow run and log the output where we can see it in the UI and CLI. | |||
Prefect will automatically track the state of the flow run and log the output where we can see it in the UI and CLI. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think it's necessary to specify where the output appears, since it appears in all interfaces.
Prefect will automatically track the state of the flow run and log the output where we can see it in the UI and CLI. | |
Prefect will automatically track the state of the flow run and log the output. |
@@ -174,10 +170,9 @@ prefect work-pool create my-managed-pool --type prefect:managed | |||
You should see a message in the CLI that your work pool was created. | |||
Feel free to check out your new work pool on the **Work Pools** page in the UI. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Something I've been wondering. Can this entire workflow be completed in the UI or CLI? I think we should minimize the amount we ask people to bounce between the UI and CLI. One handoff is probably fine, multiple handoffs less so.
|
||
We have a flow function and we have a work pool where we can run our flow remotely. | ||
Let's package both of these things, along with the location for where to find our flow code, into a [deployment](/concepts/deployments/) so that we can schedule our workflow to run remotely. | ||
We now have a flow and a work pool where we can run it. Let's package both of these things, into a [deployment](/concepts/deployments/) so that we can run our workflow remotely. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We now have a flow and a work pool where we can run it. Let's package both of these things, into a [deployment](/concepts/deployments/) so that we can run our workflow remotely. | |
We now have a flow and a work pool where we can run it. Let's package them into a [deployment](/concepts/deployments/) so that we can run our workflow remotely. |
@@ -192,13 +187,11 @@ if __name__ == "__main__": | |||
entrypoint="my_gh_workflow.py:repo_info", | |||
).deploy( | |||
name="my-first-deployment", | |||
work_pool_name="my-managed-pool", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When I went through this quickstart for the first time, I was definitely confused about how GitHub fits into it. More clarification would be awesome.
This pull request is stale because it has been open 14 days with no activity. To keep this pull request open remove stale label or comment. |
I think we need to smooth out our quickstart. This is just a start.