Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quickstart refinements #13244

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Quickstart refinements #13244

wants to merge 2 commits into from

Conversation

billpalombi
Copy link
Contributor

I think we need to smooth out our quickstart. This is just a start.

I think we need to smooth out our quickstart. This is just a start.
Copy link

netlify bot commented May 4, 2024

Deploy Preview for prefect-docs-preview ready!

Name Link
🔨 Latest commit 3220936
🔍 Latest deploy log https://app.netlify.com/sites/prefect-docs-preview/deploys/6636e14b3c0aef00086ea9ad
😎 Deploy Preview https://deploy-preview-13244--prefect-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link
Member

@zhen0 zhen0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you Bill for the quick updates. I've added a few suggestions. And we should fix the typo before releasing.

With Prefect, you can go from a Python script to a production-ready workflow that runs remotely in a few minutes.

Let's get started!
Prefect is a workflow orchestration platform for building data pipelines, background jobs, LLM Chains, and more. With Prefect, you can elevate a Python script into a scheduled, observable, resilient, remotely execuateble workflow in a few minutes. Let's get started!
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small typo in executable- sorry on mobile so I can't make a suggested change.

```

Let's make this script schedulable, observable, resilient, and capable of running anywhere.
Copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works. Let's make this script schedulable, observable, resilient, and capable of running anywhere.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the "let's make this" section still meant to be in here? It feels a little disjointed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's meant to be a transition from "you just have a script" to "look what Prefect can do for you". Is that transition improved by the suggested change added below?


## Step 2: Connect to Prefect's API

Much of Prefect's functionality is backed by an API.
The easiest way to get started is to use the API hosted by Prefect:
Much of Prefect's functionality is backed by an API. You can [host Prefect server](/guides/host/) yourself if you'd like, but the easiest way to get started is to use the API hosted by Prefect Cloud:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the mention of server here increase complexity?

Copy link

@djsauble djsauble May 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this distracts from the flow. Perhaps we move the info about self-hosted instances to the Next steps section at the end of the quickstart?

Line 257 could easily be modified to include a mention of self-hosted instances.

@@ -174,10 +170,9 @@ prefect work-pool create my-managed-pool --type prefect:managed
You should see a message in the CLI that your work pool was created.
Feel free to check out your new work pool on the **Work Pools** page in the UI.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this line helpful? I'd vote to remove it as it pulls the user away from the main focus.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. I think we want to get the user checking out the UI at some point, but maybe not until they run the deployment.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something I've been wondering. Can this entire workflow be completed in the UI or CLI? I think we should minimize the amount we ask people to bounce between the UI and CLI. One handoff is probably fine, multiple handoffs less so.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could do it all in Python code + CLI. The UI is a core part of the product that I think we want folks interacting with. That said, I hear you on bouncing back and forth being distracting.

@@ -192,13 +187,11 @@ if __name__ == "__main__":
entrypoint="my_gh_workflow.py:repo_info",
).deploy(
name="my-first-deployment",
work_pool_name="my-managed-pool",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually a content from line 186 - I suggest we point out more clearly that here they're using code we've stored in GitHub.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I went through this quickstart for the first time, I was definitely confused about how GitHub fits into it. More clarification would be awesome.

Copy link
Contributor

@discdiver discdiver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. Minor edits and notes.

We can have Prefect Cloud run our flow code for us with a Prefect Managed work pool.

Let's create a [Prefect Managed work pool](/guides/managed-execution/) so that Prefect can run our flows for us.
Prefect refers to infrastructure that can run worflows as [work pools](/concepts/work-pools/).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Prefect refers to infrastructure that can run worflows as [work pools](/concepts/work-pools/).
Prefect refers to infrastructure that can run workflows as a [work pool](/concepts/work-pools/).


Let's create a [Prefect Managed work pool](/guides/managed-execution/) so that Prefect can run our flows for us.
Prefect refers to infrastructure that can run worflows as [work pools](/concepts/work-pools/).
Prefect can orchestrate workflows that run remotely on many [types of infrastructure](/work-pools/#work-pool-types), but here we'll use a [Prefect Managed work pool](/guides/managed-execution/) so that Prefect can run this flow for us.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Prefect can orchestrate workflows that run remotely on many [types of infrastructure](/work-pools/#work-pool-types), but here we'll use a [Prefect Managed work pool](/guides/managed-execution/) so that Prefect can run this flow for us.
Prefect can orchestrate workflows that run remotely on many [types of infrastructure](/work-pools/#work-pool-types). We'll use a [Prefect Managed work pool](/guides/managed-execution/) so that Prefect can run this flow for us.

@@ -174,10 +170,9 @@ prefect work-pool create my-managed-pool --type prefect:managed
You should see a message in the CLI that your work pool was created.
Feel free to check out your new work pool on the **Work Pools** page in the UI.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. I think we want to get the user checking out the UI at some point, but maybe not until they run the deployment.


## Step 2: Connect to Prefect's API

Much of Prefect's functionality is backed by an API.
The easiest way to get started is to use the API hosted by Prefect:
Much of Prefect's functionality is backed by an API. You can [host Prefect server](/guides/host/) yourself if you'd like, but the easiest way to get started is to use the API hosted by Prefect Cloud:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Much of Prefect's functionality is backed by an API. You can [host Prefect server](/guides/host/) yourself if you'd like, but the easiest way to get started is to use the API hosted by Prefect Cloud:
Much of Prefect's functionality is backed by an API. You can [host a Prefect server instance](/guides/host/) yourself if you'd like, but the easiest way to get started is to use the API hosted by Prefect Cloud:

With Prefect, you can go from a Python script to a production-ready workflow that runs remotely in a few minutes.

Let's get started!
Prefect is a workflow orchestration platform for building data pipelines, background jobs, LLM Chains, and more. With Prefect, you can elevate a Python script into a scheduled, observable, resilient, remotely execuateble workflow in a few minutes. Let's get started!
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Prefect is a workflow orchestration platform for building data pipelines, background jobs, LLM Chains, and more. With Prefect, you can elevate a Python script into a scheduled, observable, resilient, remotely execuateble workflow in a few minutes. Let's get started!
Prefect is a workflow orchestration platform for building data pipelines, background jobs, LLM Chains, and more. With Prefect, you can elevate a Python script into a scheduled, observable, resilient, remotely executable workflow in a few minutes. Let's get started!

```

Let's make this script schedulable, observable, resilient, and capable of running anywhere.
Copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works. Let's make this script schedulable, observable, resilient, and capable of running anywhere.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works. Let's make this script schedulable, observable, resilient, and capable of running anywhere.
Copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works.
Now let's use Prefect to make this script schedulable, observable, resilient, and capable of running anywhere.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an extra leading space on line 53, but I think this helps to differentiate the setup from the next steps.

I might go a step further and encourage people to spin up an IDE to save them a step later when we ask them to run pip/python commands.

Suggested change
Copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works. Let's make this script schedulable, observable, resilient, and capable of running anywhere.
Create a new project in your favorite IDE, and then copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works.
Now let's use Prefect to make this script schedulable, observable, resilient, and capable of running anywhere.

```

Let's make this script schedulable, observable, resilient, and capable of running anywhere.
Copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works. Let's make this script schedulable, observable, resilient, and capable of running anywhere.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's meant to be a transition from "you just have a script" to "look what Prefect can do for you". Is that transition improved by the suggested change added below?

@@ -192,13 +187,11 @@ if __name__ == "__main__":
entrypoint="my_gh_workflow.py:repo_info",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making the source a Prefect repo would be good.

@discdiver discdiver self-requested a review May 5, 2024 01:29
```

Let's make this script schedulable, observable, resilient, and capable of running anywhere.
Copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works. Let's make this script schedulable, observable, resilient, and capable of running anywhere.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an extra leading space on line 53, but I think this helps to differentiate the setup from the next steps.

I might go a step further and encourage people to spin up an IDE to save them a step later when we ask them to run pip/python commands.

Suggested change
Copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works. Let's make this script schedulable, observable, resilient, and capable of running anywhere.
Create a new project in your favorite IDE, and then copy this script into a file named `my_gh_workflow.py`. Run it locally to ensure your Python environment works.
Now let's use Prefect to make this script schedulable, observable, resilient, and capable of running anywhere.

With Prefect, you can go from a Python script to a production-ready workflow that runs remotely in a few minutes.

Let's get started!
Prefect is a workflow orchestration platform for building data pipelines, background jobs, LLM Chains, and more. With Prefect, you can elevate a Python script into a scheduled, observable, resilient, remotely execuateble workflow in a few minutes. Let's get started!

## Setup
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's best to use task-based titles. For example, see https://developers.google.com/style/headings

Suggested change
## Setup
## Set up your environment


## Step 1: Install Prefect

Make sure you have Python installed in a virtual environment. Then install Prefect:

```bash
pip install -U prefect
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could go either way on this. People who care about virtual environments will tend to already know how to do this, but we can save some typing by putting those commands inline.

Suggested change
pip install -U prefect
# Set up a virtual environment
python -m venv env
source env/bin/activate
# Install Prefect
pip install -U prefect

```bash
pip install -U prefect
```

See the [install guide](/getting-started/installation/) for more detailed installation instructions, if needed.
See the [install guide](/getting-started/installation/) for more detailed installation instructions.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or maybe the virtual environment setup commands could go here.


## Step 2: Connect to Prefect's API

Much of Prefect's functionality is backed by an API.
The easiest way to get started is to use the API hosted by Prefect:
Much of Prefect's functionality is backed by an API. You can [host Prefect server](/guides/host/) yourself if you'd like, but the easiest way to get started is to use the API hosted by Prefect Cloud:
Copy link

@djsauble djsauble May 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this distracts from the flow. Perhaps we move the info about self-hosted instances to the Next steps section at the end of the quickstart?

Line 257 could easily be modified to include a mention of self-hosted instances.

@@ -100,27 +108,20 @@ def get_contributors(repo_info: dict):
contributors = response.json()
return contributors


@flow(log_prints=True)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
@flow(log_prints=True)
# This method is the flow entrypoint
@flow(log_prints=True)

@@ -130,7 +131,7 @@ python my_gh_workflow.py

</div>

Now when we run this script, Prefect will automatically track the state of the flow run and log the output where we can see it in the UI and CLI.
Prefect will automatically track the state of the flow run and log the output where we can see it in the UI and CLI.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's necessary to specify where the output appears, since it appears in all interfaces.

Suggested change
Prefect will automatically track the state of the flow run and log the output where we can see it in the UI and CLI.
Prefect will automatically track the state of the flow run and log the output.

@@ -174,10 +170,9 @@ prefect work-pool create my-managed-pool --type prefect:managed
You should see a message in the CLI that your work pool was created.
Feel free to check out your new work pool on the **Work Pools** page in the UI.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something I've been wondering. Can this entire workflow be completed in the UI or CLI? I think we should minimize the amount we ask people to bounce between the UI and CLI. One handoff is probably fine, multiple handoffs less so.


We have a flow function and we have a work pool where we can run our flow remotely.
Let's package both of these things, along with the location for where to find our flow code, into a [deployment](/concepts/deployments/) so that we can schedule our workflow to run remotely.
We now have a flow and a work pool where we can run it. Let's package both of these things, into a [deployment](/concepts/deployments/) so that we can run our workflow remotely.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
We now have a flow and a work pool where we can run it. Let's package both of these things, into a [deployment](/concepts/deployments/) so that we can run our workflow remotely.
We now have a flow and a work pool where we can run it. Let's package them into a [deployment](/concepts/deployments/) so that we can run our workflow remotely.

@@ -192,13 +187,11 @@ if __name__ == "__main__":
entrypoint="my_gh_workflow.py:repo_info",
).deploy(
name="my-first-deployment",
work_pool_name="my-managed-pool",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I went through this quickstart for the first time, I was definitely confused about how GitHub fits into it. More clarification would be awesome.

Copy link
Contributor

This pull request is stale because it has been open 14 days with no activity. To keep this pull request open remove stale label or comment.

@github-actions github-actions bot added the status:stale This may not be relevant anymore label May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status:stale This may not be relevant anymore
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants