Prefect Profile Banner
Prefect Profile
Prefect

@PrefectIO

6,165
Followers
152
Following
303
Media
1,526
Statuses

Modern workflow orchestration for data and ML engineers. Get the code: Chat in Slack:

Joined November 2017
Don't wanna be here? Send us removal request.
Pinned Tweet
@PrefectIO
Prefect
9 months
๐Ÿšจ New website, new brand, new event-driven workflows. Take it for a spin: At Prefect, we make complex workflows simpler, not harder. How, you might ask?๐Ÿงต
1
10
26
@PrefectIO
Prefect
4 years
It's our most-requested feature ever... and now the Prefect UI is open-source!ย ๐ŸŽ‰ ๏ฟผ Runย `prefect server start`ย to get going - it's really that easy.
213
2K
27K
@PrefectIO
Prefect
4 years
The open-source Prefect platform just got an upgrade so big, we had to create two new repos just to hold it! ๐ŸŽ‰๐ŸŽ‰ Run `prefect server start` to get going - it's really that easy.
51
360
7K
@PrefectIO
Prefect
4 years
Thousands of users have already tried the new, open-source Prefect Platform! ๐Ÿš€ Run `prefect server start` to get going - it's really that easy.
11
192
4K
@PrefectIO
Prefect
4 years
The Prefect UI is now open-source! ๐ŸŽ‰ Run `prefect server start` to get going โ€” itโ€™s really that easy.
23
241
3K
@PrefectIO
Prefect
4 years
Thanks for being part of our community! We couldn't have reached this milestone without you, and we can't wait to show you what's next. Happy engineering!
1
44
2K
@PrefectIO
Prefect
4 years
To celebrate its first anniversary, we just open-sourced Prefectโ€™s UI and API! Run `prefect server start` to get going - itโ€™s really that easy.
6
66
456
@PrefectIO
Prefect
3 years
We have Liftoff! Prefect is launching four key initiatives this quarter: ๐Ÿ“ˆ Expansion ๐Ÿš€ Product ๐Ÿค Partnerships ๐Ÿ’ฌ Community Join us (and Marvin) to see how we're transforming data engineering.
8
43
395
@PrefectIO
Prefect
3 years
Introducing simple, success-based pricing for Prefect Cloud! โœ… Only successful tasks count; never pay for retries or failures ๐ŸŽ Every plan includes 10,000 free tasks / month ๐Ÿ“‰ Automatic discounts as usage scales Even Marvin's excited about it.
2
26
267
@PrefectIO
Prefect
4 years
ONE THOUSAND! ๐Ÿ˜… It's been less than a year since we launched our Slack community with 20 members and 1 simple rule: Be nice. Thank you for making Prefect the best data community around - we can't wait to show you what's coming next. Join us here:
0
29
255
@PrefectIO
Prefect
4 years
Curious about @vuejs ? In this Friday's livestream, @____nicholas will give @lauralorenzdev (and you!) an introduction to Vue development by building a brand new tile for our open-source UI's dashboard. All experience levels are welcome!
0
16
214
@PrefectIO
Prefect
3 years
Love @expectgreatdata ? Wish you could get validation reports in the Prefect UI? Now anyone can create a native UI integration for any library with only one line of code - just like we did for Great Expectations. Check out our blog to see how easy it is.
1
8
203
@PrefectIO
Prefect
3 years
Say ๐Ÿ‘‹ to Orion, our second-generation workflow engine! It's DAG-free, has an observable rules engine, and delivers an incredible developer experience from test to production. You might just love your workflows again.
6
23
135
@PrefectIO
Prefect
4 years
8
9
116
@PrefectIO
Prefect
9 months
Want to make your complex workflows simpler? Schedule pure Python using a simple @flow decorator. Develop locally, deploy to prod to build, observe, and react to data pipelines. Get started with workflow orchestration with this simple tutorial:
6
8
104
@PrefectIO
Prefect
4 years
It's finally here! Check out the public release of Prefect Cloud, including the free Scheduler tier. What will you build?
4
22
110
@PrefectIO
Prefect
3 years
We're excited to share that we raised $32M from Tiger Global and @BessemerVP ! We're so grateful to have such an incredible community of users, advisors, and investors, all working together to build a new kind of workflow system. Happy engineering!
2
10
95
@PrefectIO
Prefect
5 years
To celebrate its second anniversary, Prefect is now open source! What will you build?
3
18
95
@PrefectIO
Prefect
2 years
Modern Data Stack Mount Rushmore, we'll start: Observability @montecarlodata Transformation: @getdbt ETL: @AirbyteHQ Reverse ETL: @getcensus BI: @_hex_tech Warehouse/Lake: @SnowflakeDB ML: @huggingface Orchestration and Coordination: @PrefectIO How do you stack?
7
11
72
@PrefectIO
Prefect
4 years
@jks Prefect is a Python automation framework for data science and engineering.
5
9
62
@PrefectIO
Prefect
5 years
What a day! Since launch, Prefect became the #12 trending repo on GitHub, and reached #2 among all Python projects! Thank you for your support as we build a new kind of workflow tool.
0
14
68
@PrefectIO
Prefect
5 years
Who wants coffee?? To celebrate 1,000 followers, we're giving away 3 tins from our friends @CompassCoffeeDC , plus some exclusive Prefect gear! On 7/22 we'll randomly select one of our followers to win. To enter, just like this tweet. ๐Ÿš€โ˜•๏ธ๐Ÿงข
Tweet media one
1
1
61
@PrefectIO
Prefect
2 years
We have an official Prefect 2.0 release, and it's graduating to beta! ๐Ÿ‘ฉโ€๐Ÿ’ป Finally, Code as Workflows ๐Ÿ‘‰ Apache 2.0, top to bottom ๐Ÿค“ Ephemeral API โ›ฐ๏ธ Streaming use cases ๐Ÿ’ป Brand new UI, included More on how Prefect 2.0 helps you orchestrate your stack:
2
17
61
@PrefectIO
Prefect
3 years
Tweet media one
2
7
53
@PrefectIO
Prefect
3 years
We're excited to announce that we acquired Slate Data! Slate's Chicago-based team of amazing client-facing engineers has joined Prefect's customer success and integrations teams, so our users will benefit from their presence immediately. Welcome aboard!
2
18
51
@PrefectIO
Prefect
5 years
We โค๏ธ @dask_dev , and you should too! In our latest post, @markov_gainz shows how easy it is to deploy your Prefect flows on a distributed Dask cluster. (spoiler alert: set two environment variables and you're done)
1
16
48
@PrefectIO
Prefect
4 years
It's the biggest release we've ever done: Prefect Server and UI are graduating to standalone open-source projects! ๐Ÿพ Upgrade to 0.13.0 and get four months of performance improvements, features, and UX upgrades with one command: `prefect server start`
0
10
43
@PrefectIO
Prefect
2 years
Verified by Twitter? So 2008. Verified by Prefect is the new blue check. Today launches our new Premier Technology Partner Program within our partner ecosystem, read on ๐Ÿ‘‰
1
7
43
@PrefectIO
Prefect
4 years
We launched the new, open-source Prefect platform exactly one month ago. Since then, weโ€™ve shot through: โญ๏ธ 2150 GitHub stars ๐Ÿฆ 2180 Twitter followers ๐Ÿค 795 Slack members Itโ€™s incredible to see the community grow at this rate. Thanks for your support!
1
7
39
@PrefectIO
Prefect
4 years
You might have noticed that the internet broke last Friday. ...unless you use Prefect Cloud. Our CTO @markov_gainz explains how the Prefect Hybrid Model allows our SaaS users to mitigate failure, even during global internet outages:
2
9
38
@PrefectIO
Prefect
4 years
Prefect 0.12.0 is out! ๐Ÿ—บ Depth-first execution ๐Ÿ“ฆ Docker environments can load flows from anywhere ๐ŸŽˆ Lightweight @dask_dev serialization ๐Ÿ”Ž @expectgreatdata task ๐ŸŒก API health checks ๐Ÿ’Ž Persistent Server volumes ๐Ÿ“‘ Improved tab completion And more...
1
6
38
@PrefectIO
Prefect
2 years
๐Ÿ™Œ Prefect Cloud just got twice as good! ๐Ÿ“ˆ In the last 6 months, the average team doubled its usage, so we're doubling our free tier to keep up. ๐Ÿ†“ Every plan now includes 20,000 free runs every month! ๐Ÿค— More people than ever can use Prefect Cloud - and we're ready for them!
0
4
35
@PrefectIO
Prefect
2 years
Prefect Core is now Prefect 1.0! This release is the product of 4 years of invention and refinement. After rigorous testing, we published a 1.0 release candidate and invited feedback from our community a month ago. 1.0 is a rock solid, stable release:
0
7
34
@PrefectIO
Prefect
4 years
We're hiring! Check out our open positions, and if you don't see one that's a prefect fit, shoot us an email with your ideal role and we'll see what we can do! - Head of Growth - Front end engineer - Back end engineer - DevOps engineer #MozillaLifeboat
2
10
32
@PrefectIO
Prefect
3 years
Today weโ€™re introducing KV Store: managed metadata for Prefect Cloud flows! Use it to track useful information across runs: ๐Ÿ”ข the last processed value ๐Ÿค easily share state across flows โš™๏ธ custom configuration Learn more from the lead developer here:
1
5
34
@PrefectIO
Prefect
5 years
Prefectโ€™s never gonna let you down. The newest example from @dask_dev : - runs a Prefect ETL pipeline - in a Dask cluster - to make a GIF (because the internet) Try it yourself with @mybinderteam :
0
4
35
@PrefectIO
Prefect
5 years
Today, we're announcing a scheduler for the internet! Prefect Scheduler is a free workflow system that supports any execution environment, powered by Prefect Cloud. It's the product we've always wanted to build. Join the waitlist now!
1
9
33
@PrefectIO
Prefect
5 years
You may have heard: we love @dask_dev ! @markov_gainz details how Dask enhances our engine:
0
6
32
@PrefectIO
Prefect
3 years
Do you work for a ๐Ÿš€ startup, ๐ŸŽ“ academic institution, ๐ŸŒŽ or non-profit? Weโ€™re running Lighthouse Partner programs for these 3 groups that include free access to Prefect Cloud. Please email hello @prefect .io to apply (startups must be VC-backed and Series A or earlier). Thanks!
1
9
29
@PrefectIO
Prefect
2 years
Itโ€™s ๐Ÿ•day! Reply with an open-source project youโ€™ve contributed to or RT to thank an open-source contributor, and weโ€™ll send you a pizza. REALLY (til 4pm ET ๐Ÿ˜‰) Thereโ€™s no catch. Just pies for pi. Use the form in our bio to send us your address to get your โ€˜ZA!
50
14
31
@PrefectIO
Prefect
4 years
We've posted 3 new case studies that you'll have to read to believe: - @SymphonyRM saw efficiency gains of 99% - Slate Data saw a 1700x increase in productivity - @clearcover saved days over building and testing Airflow DAGs Check them out at
1
7
28
@PrefectIO
Prefect
4 years
Today weโ€™re excited to welcome @jcristharif to the Prefect team and look forward to powering the next generation of @dask_dev applications!
1
4
28
@PrefectIO
Prefect
2 years
And just like that, Prefect 2.0 is out of beta! Thanks to all of our contributors, community members and the Prefect team. Prefect Cloud 2.0 is also generally available today - sign up at to start coordinating your dataflow!
0
7
29
@PrefectIO
Prefect
3 years
๐Ÿ‘‹ from @KubeCon_ !
Tweet media one
1
5
29
@PrefectIO
Prefect
3 years
Two years ago, we launched the Prefect Slack Community with 20 people. Today, we welcomed our ๐ŸŒŸ10,000th๐ŸŒŸ member! Thank you for joining this amazing community of data engineers. Prefect wouldn't be where it is without such positive and enthusiastic contributors! ๐Ÿฅ‚
@PrefectIO
Prefect
5 years
We heard you loud and clear: come join our shiny new Slack community! Letโ€™s discover the best ways to work together.
1
3
16
0
3
28
@PrefectIO
Prefect
3 years
Every metric tells a story.
Tweet media one
0
2
28
@PrefectIO
Prefect
2 years
Itโ€™s not everyday you wake up and see your name in Forbes! Amazing companies were listed on Americaโ€™s Best Startup Employers 2022, proud to be named among them.
1
5
28
@PrefectIO
Prefect
4 years
Weโ€™ve just released a HUGE new version of Prefect Cloud: ๐ŸŽจ enjoy a fully-redesigned UI ๐Ÿ“Š visualize flows and run progress with new schematics ๐Ÿค– monitor all connected Prefect Agents ๐Ÿ•ต๏ธโ€โ™€๏ธ jump straight to error logs from any dashboard โฐ keep an eye on late runs
Tweet media one
2
5
28
@PrefectIO
Prefect
3 years
Wondering what we've been up to with @Microsoft ? Check out the first steps of how we're working together to support our users, and join us at #MSBuild to learn more!
0
6
27
@PrefectIO
Prefect
3 years
Say ๐Ÿ‘‹ to Prefect 0.14.9! This is a very special release, driven entirely by the Prefect community. It also includes @fivetran 's very first workflow integration! Please give them a warm welcome and let us know how we should continue collaborating.
2
2
27
@PrefectIO
Prefect
4 years
Say your @dask_dev cluster was a chicken. With Prefect 0.12.5's new resource manager API, you can control its entire lifecycle, from ๐Ÿฅš to ๐Ÿ—, from inside your flow. Pro tip: it also works with non-chickens like Spark clusters, containers, or databases.
Tweet media one
Tweet media two
1
5
25
@PrefectIO
Prefect
5 years
Prefect and Airflow share inspiration, but take very different approaches to certain aspects of workflow management. Weโ€™ve put together a document that highlights key differences and why those decisions were made.
0
8
25
@PrefectIO
Prefect
5 years
Here's an excellent interview of our friend @mrocklin by @unbalancedparen about what @dask_dev is, why itโ€™s awesome, and what other tools leverage it. Prefect even got a shoutout!
1
5
26
@PrefectIO
Prefect
4 years
๐Ÿ‘€ Have you ever wanted to fix your task code mid flow run? Stay tuned! Hot-reloading Flows coming soonโ€ฆ
0
3
26
@PrefectIO
Prefect
2 years
Toss a rock, and there's a 30% chance you'll hit a Prefect engineer (so... please don't throw too hard). We're ecstatic to see more and more industry surveys that reflect Prefect's rapid adoption among mature data organizations.
Tweet media one
2
3
26
@PrefectIO
Prefect
3 years
The new flow run page is here! In the advanced options, you'll find the new "run config" feature (Prefect 0.14+). Now you can change your execution environment, configuration, labels, or even infrastructure on a per-run basis.
Tweet media one
Tweet media two
2
4
26
@PrefectIO
Prefect
3 years
Weโ€™re excited to be a partner sponsor at @PyCon , along with many of our favorite partners! Looking for a Prefect opportunity? Come chat with @jlowin and @markov_gainz at the job fair this Sunday from 1-3pm ET!
Tweet media one
0
8
25
@PrefectIO
Prefect
2 years
Excited to share the biggest release of Prefect 2.0 to date. Highlights include: ๐Ÿ” Flow Run retries ๐Ÿ”” Configurable notifications ๐Ÿ“ฆ Flow packaging and deployment UX improvements and... drum roll please... ๐Ÿ”ฒ BLOCKS...! Read the full update here:
1
1
25
@PrefectIO
Prefect
2 years
๐Ÿ‘€๐Ÿคฏ It's official!!! See you all on Wednesday the 27th ๐Ÿ‘Œ
Tweet media one
0
2
25
@PrefectIO
Prefect
2 years
Thrilled to be a part of the new @dbt_labs Technology Partner Program alongside all our besties!
Tweet media one
0
1
24
@PrefectIO
Prefect
3 years
"Prefect is a lot of things. We think it is the best possible tool for dataflow automation. We use Prefect for data modeling, ETL, ML, basically anywhere that data automation is needed." Learn how @gabcoyne and Slate use Prefect and @getdbt together!
0
5
24
@PrefectIO
Prefect
4 years
"If you think Airflow is a great tool, you might find that Prefect is an even greater tool." Awesome to hear our friends @mrocklin and @hugobowne on with @TobiasMacey - thanks for making us your "Pick"!
@CoiledHQ
Coiled
4 years
Coiled's @mrocklin and @hugobowne recently spoke with @TobiasMacey on @Podcast__init__ about #Dask , building nimble tools for scaling #datascience , #OSS interplay with commercial needs, & how #python is eating the world. Check it out here!
0
4
15
0
4
23
@PrefectIO
Prefect
5 years
Thanks to @Zach_Angell for the first community addition to our task library: support for AWS Lambda! Do you want to contribute to an open source project? Would you like a new car? At least one of those is possible at
2
5
24
@PrefectIO
Prefect
1 year
Another Thursday, another release! Prefect 2.8.7 is live with โš™๏ธ Tons of engine performance and reliability improvements ๐Ÿ“บ Results, a visual representation of output from your tasks & flows, are live in the UI ๐Ÿ“ˆ Flow run graph now supports subflows + thousands of nodes
Tweet media one
2
4
25
@PrefectIO
Prefect
4 years
We're delighted to be included on @Accel 's #Open100 list, along with many companies we admire. Happy engineering!
@Accel
Accel
4 years
#OpenSource has exploded, so weโ€™ve launched the #Open100 , a collection of the fastest-growing OSS start-ups globally. Check out the list, and read top company building tips from the experts at @Snyk , @Rasa , @Confluent , @Segment , @Sentry and @Vercel . #OSS
Tweet media one
2
27
88
0
7
23
@PrefectIO
Prefect
4 years
๐Ÿ‘€
0
1
23
@PrefectIO
Prefect
4 years
We were so excited about the Server + UI launch, we forgot to say ๐Ÿ‘‹ to Prefect 0.13.0! ๐Ÿ’ซ New Server + UI compatibility (of course) ๐Ÿ•ธ Webhook flow storage ๐Ÿšซ Cancel in-flight work (on @dask_dev ) ๐Ÿ” Improved version-locks for global run-once semantics
1
3
23
@PrefectIO
Prefect
2 years
Prefect Cloud surged in November โ€” usage jumped 50% and we crossed โœจhalf a billionโœจ tasks processed for our users. Happy engineering, Prefectionists!
0
3
24
@PrefectIO
Prefect
2 years
Twinkle twinkle 10,000 @github stars Next stop: 100,000 ๐Ÿš€
Tweet media one
3
5
24
@PrefectIO
Prefect
4 years
This week we're excited to welcome Allyson Lubimir and Mariia Kerimova to the Prefect engineering team! Watch this space, things are about to move very quickly...
0
3
22
@PrefectIO
Prefect
3 years
Weโ€™re proud to support @boss_librarian โ€™s Digital Citizenship program at @JATrojans by donating 15 laptops to the school library in recognition of October being anti-bullying month. We love the program K.C. runs and the impact it has and look forward to working together more!
0
2
24
@PrefectIO
Prefect
2 years
Prepare 2 b excited with our latest release: ๐Ÿ–ฅ๏ธ Windows support for 2.0 โŒจ๏ธ Support for type-checking โœจ Oh, and a new UI ๐Ÿคฉ More on all of that here:
2
3
22
@PrefectIO
Prefect
3 years
Interested in deploying @PrefectIO Server on @Microsoft Azure AKS? Join us this Friday 3/19 at 4ET/1PT for a livestream walkthrough on deploying and upgrading your Prefect Server!
Tweet media one
2
4
21
@PrefectIO
Prefect
2 years
Tour De Prefect has begun! Team members from across the country have congregated in Chicago for the week for cross-org learning and IRL bonding.
Tweet media one
0
1
22
@PrefectIO
Prefect
3 years
Say ๐Ÿ‘‹ to Prefect Orion 2.0a4! This release is all about executors: โคต๏ธ Each subflow can have its own executor โœจ Expanded support for Dask clusters ๐Ÿ›  Lots of CLI improvements Thanks to our community members that helped prioritize this roadmap!
1
3
23
@PrefectIO
Prefect
3 years
Say ๐Ÿ‘‹ to Prefect 0.15! It's a whole new flow run experience: โœจ Brand new flow run CLI ๐Ÿค Subflow result passing ๐Ÿ•ต๏ธ Agentless execution ๐Ÿ” New Cloud auth patterns Thanks to all who contributed over many months of work!
1
4
22
@PrefectIO
Prefect
4 years
Did you know Prefect has two different Python APIs? The *Functional API* will be familiar to anyone coming from a data science background. Data engineers may be more comfortable with the *Imperative API*. Hate choices? Use both!
Tweet media one
Tweet media two
0
0
22
@PrefectIO
Prefect
4 years
Our new Interactive API is out of beta. Now you can leverage the full power of Prefect Cloud's GraphQL API -- without leaving your browser! ๐Ÿ‘จโ€๐Ÿ’ป Interact with all of your Prefect data โœ… Take advantage of GraphQL autocomplete ๐Ÿ“– Easily browse the schema and documentation
0
2
21
@PrefectIO
Prefect
2 years
We turn 4 this week! We're so excited to be leaving toddlerhood behind as we embark on our next chapter. Take a look what we accomplished over this past year: All that ^ and we haven't even had our first day of kindergarten yet!
Tweet media one
0
2
21
@PrefectIO
Prefect
3 years
Life, the universe, and everyone.
Tweet media one
0
1
21
@PrefectIO
Prefect
5 years
Remember when daylight saving time would break your DAGs? Prefect remembers. Thatโ€™s why everything in Prefect is timezone-aware by default.
0
1
22
@PrefectIO
Prefect
2 years
It's #InternationalWomensDay , and no one embodies #GirlPower like our COO @Smoses_Supposes . She bulldozed a new way of work creating an inspiring culture and will go to extreme heights - not just for a climb - but to make someones wild dream become a Prefect reality. TY Sarah.
Tweet media one
2
3
21
@PrefectIO
Prefect
1 year
Interested in learning the basics of MLOps? Prefect is a part of the @DataTalksClub ML Zoomcamp! ๐Ÿ‘‡ to join the upcoming live cohort or follow along with the self-paced curriculum.
1
3
22
@PrefectIO
Prefect
3 years
Thank you to part of our sales organization who teamed up for yesterday's beach cleanup! While we are laser-focused on delivering value to our community and users, Prefect will always be committed to making the world a better place. ๐ŸŒŽ
Tweet media one
2
2
20
@PrefectIO
Prefect
2 years
To fail is human; to log a failure, divine
Tweet media one
1
2
20
@PrefectIO
Prefect
2 years
We are now officially a ๐Ÿฅง enthusiast community
Tweet media one
0
0
20
@PrefectIO
Prefect
4 years
Looking to get started with @dask_dev ? @lauralorenzdev and @jsignell from @saturn_cloud just posted an excellent beginner's guide to some of the most common questions, like: ๐Ÿ—บ Where to host your cluster? ๐Ÿ›  How many workers? ๐Ÿงต How many threads?
0
8
21
@PrefectIO
Prefect
2 years
Want a chance to win one a Prefect keyboard? We'll be raffling them at our meetups tomorrow night in San Jose and Washington DC! Sign up โฌ‡๏ธ DC: San Jose:
Tweet media one
2
4
21
@PrefectIO
Prefect
4 years
"You can obtain a highly available, scalable, distributed system that will make the orchestration of your data pipelines for ETL & ML much more enjoyable and will free up your time to work with data and generate value out of it." Awesome tutorial, Anna!
0
2
21
@PrefectIO
Prefect
2 years
Collaborators are live on Prefect Cloud 2! Life is better with friends, and beta users can add up to 3 collaborators to their workspace for free. Give it a go at . More at Marvins collaborating pictured courtesy of DALL-E Mini ๐Ÿฆ†
Tweet media one
1
3
21
@PrefectIO
Prefect
2 years
"Prefect makes work that's generally kind of a pain not a pain" - George Coyne, Sr. Solutions Architect
0
2
20
@PrefectIO
Prefect
3 years
Are you a Prefectionist? We're hiring! Check out our open roles here:
0
4
20
@PrefectIO
Prefect
4 years
Sure, we all love automation. But sometimes you just need a human in the loop. Here's how easy it is to make your Prefect flows pause for manual approval:
0
3
20