ray @raydistributed Twitter profile

Last Seen Profiles

@BodyByElite

@harleyf

@markclaflin

@changmin88_IG

@LaurenBushnell3

@muttskull

@Deji

@brooke_bball25

@epilepsyireland

@YUVI_GB_11

@odotte_official

@DSI_UChicago

@alephic2

@pe_muas_nafsu

@patunikos

@turian

@oncecaldas

@SagrikaKissu

@otafiire

@_julianmichael_

@leytonkins

@ladyballsmcgee

@pitcoin_

@vlett23qx

@lPrjme

@MGMRewards

@milky_succubus

@k_myon_RPS13

@InVisionApp

@h_i_r_o_t_a

@RafaBelattini

@webkinzgurl2007

@mordekoiso

@liulianjingrua

@KasperLoock

@flamekr

ray

@raydistributed

1 year

Distributed fine-tuning LLM is more cost effective than fine-tuning on a single instance! Check out the blog post on how to fine-tune and serve LLM simply, cost effectively using Ray + DeepSpeed and 🤗

Blog | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

3

71

300

ray

@raydistributed

1 year

Ray is a powerful ML framework, but with great power comes massive documentation. How can we make it more accessible? Now, using @LangChainAI and Ray, we can build and deploy a doc search engine in about 100 lines of code -- with a self-hosted LLM! 1/n

Open Source LLM Search Engine with LangChain on Ray

Waleed, Head of Engineering at Anyscale, explains how to use LangChain and Ray Serve to build a search engine using LLM embeddings and a vector database.Blog...

www.youtube.com

9

55

291

ray

@raydistributed

3 years

Announcing a new Ray + 🤗 @huggingface integration! RAG is a new NLP model that uses external documents to augment its knowledge. We’ve integrated Ray with RAG: - 🚄Speeding up retrieval calls by 2x - 💫Improving the scalability of fine tuning Blog:

Retrieval Augmented Generation with Huggingface Transformers and Ray

Huggingface Transformers recently added the Retrieval Augmented Generation (RAG) model, a new NLP architecture that leverages external…

medium.com

2

47

182

ray

@raydistributed

4 years

We're releasing RaySGD, a pytorch library that makes distributed training cheap and simple! Features: - fp16 training support - elastic training (automatic fault tolerance) - Integrated distributed HPO (w/ RayTune) - intuitive and pytorch-friendly APIs

Faster and Cheaper Pytorch with RaySGD

Distributed training is annoying to set up and expensive to run. Here’s a library to make distributed Pytorch training simple and cheap.

medium.com

1

57

178

ray

@raydistributed

1 year

Announcing Ray 2.4.0: Infrastructure for LLM training, tuning, inference, and serving. 🧠 LLM features 💽 Ray data for ease of use & stability 📊 Serve observability 🤖 RLlib’s module for custom reinforcement learning 🏢Ray scalability for large clusters

Announcing Ray 2.4.0: Infrastructure for LLM training, tuning, inference, and serving

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

0

40

166

ray

@raydistributed

4 years

ML serving infra has evolved, and there are 3 key requirements - Framework agnostic ( @TensorFlow , @PyTorch , pure Python, ...) - Pure Python (intuitive for developers) - Out of the box scalability Why? How does this relate to Ray and @huggingface ? 🤗 👇

The Simplest Way to Serve your NLP Model in Production with Pure Python

From scikit-learn to Hugging Face Pipelines, learn the simplest way to deploy ML models using Ray Serve.

medium.com

2

47

153

ray

@raydistributed

4 years

You can now tune your @huggingface transformer Trainer with RayTune () in 1 line of code! ⚡️Access Bayesian Optimization, Population-based Training to superpower your model 🧙‍♂️Use Multi-GPU and Multi-node support Blog post:

0

31

119

ray

@raydistributed

9 months

@BytedanceTalk , the company behind TikTok, uses Ray for fast & cheap offline inference with multi-modal #LLMs . They generate embeddings for a staggering 200 TB of image and text data using a model with >10B parameters. 🧵 Thread below 👇

How ByteDance Scales Offline Inference with Multi-Modal LLMs

ByteDance, the company behind Tiktok, leverages multi-modal models to enable many applications, such as text-based image retrieval or object detection.

www.anyscale.com

2

31

119

ray

@raydistributed

4 years

Ray 1.0 is up on Github and PyPI (w/ new beautiful docs - )! 🎉This is a huge and important release, with many new APIs and tons of new committers! 🔖 Read about Ray 1.0 on our blog post ()

Announcing Ray 1.0 | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

0

56

114

ray

@raydistributed

3 years

🎉 Say hello to Ray Lightning — a faster and simpler path to multi-node distributed training for @PyTorchLightnin ⚡️. Change 1 line to scale your PyTorch Lightning training to a multi-node GPU cluster. Give it a try and let us know what you think!

Introducing Ray Lightning: Multi-node PyTorch Lightning training made easy | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

0

26

101

ray

@raydistributed

1 year

Part 2 of our Ray + LangChain Series is ready, in this part we’ll show you how to turbocharge generation of embeddings. See the video(9 minutes) at and blog post at

This link will take you to a page that’s not on LinkedIn

lnkd.in

1

34

102

ray

@raydistributed

8 months

The team @MetaAI has done a tremendous amount to move the field forward with the Llama models. We're thrilled to collaborate to help grow the Llama ecosystem.

Anyscale and Meta Collaborate to Advance the Llama-2 Ecosystem

We are excited to announce collaboration between Meta and Anyscale to bolster the Llama ecosystem.

www.anyscale.com

2

20

89

ray

@raydistributed

4 years

hyperparameter tuning for #NLProc is often overlooked, but by using @huggingface transformers + tuning techniques such as PBT, you can increase model accuracy by up to 5% on certain fine-tuning tasks *without increasing your compute budget*! 🔖 read it:

0

24

89

ray

@raydistributed

3 years

JAX is a system for high-performance machine learning research and numerical computing. At #RaySummit , @GoogleAI 's @SingularMattrix will show how JAX is used in #neuralnet training, probabilistic programming & more. Register to join live or on-demand

0

21

82

ray

@raydistributed

2 years

Data scientist != infra engineer. Thanks @marksaroufim for joining our Ray Meetup last week and sharing how to make it easier to train large-scale #ML jobs in #opensource . If you missed it, you can watch the recording here:

Ray Train, PyTorch, TorchX, and distributed deep learning | Anyscale

Welcome to our second Ray meetup, where we focus on Ray’s native libraries for scaling machine learning workloads. We'll discuss Ray Train, a production-ready distributed training library for deep...

www.anyscale.com

1

13

83

ray

@raydistributed

4 years

excited to see Ray Tune integrated into the awesome 🤗 @huggingface Transformers!

Sylvain Gugger

@GuggerSylvain

4 years

Hyperparameter search with optuna or Ray Tune is now fully integrated in Trainer (support for TF coming soon!) Tutorials coming soon but in the meantime the docs are a good way to get started with it

2

31

178

0

12

71

ray

@raydistributed

1 year

ICYM our blogs on Ray and Generative AI. We have a three-part series on how to use Ray to productionize common generative AI model workloads. Here are parts 1 and 2: 👉 👉 #Ray for #GenerativeAI #workloads

High-Performance LLM Training at 1000 GPU Scale With Alpa & Ray

In part 2 of our generative AI blog series, we cover how Ray empowers large language models (LLM) frameworks such as Alpa.

www.anyscale.com

0

12

56

ray

@raydistributed

8 months

🎉 Announcing Ray Serve and Anyscale Services general availability! Teams at @LinkedIn , @Samsara , @AntGroup + many more have been using Ray to serve LLMs & multi-modal applications in a flexible, performant and scalable way. Read more about the GA release and how companies have…

1

17

54

ray

@raydistributed

8 months

Cloud TPUs from @googlecloud are one of the most cost-effective ways to train and serve LLMs. In 2.7, Ray finally will support TPUs natively -- Ray enables a more intuitive TPU developer experience, allowing you to train and serve on massive TPU pods with ease. Learn more at…

0

9

45

ray

@raydistributed

3 years

Deep RL has become fairly capable of optimizing reward; however, how do you choose the reward function to be optimized? @pabbeel will discuss some recent progress in this area in his #RaySummit talk "Human-in-the-Loop Reinforcement Learning" Register:

0

8

41

ray

@raydistributed

1 year

Offline Batch Inference: Comparing Ray, Apache Spark & SageMaker. Image classification benchmarks show that #Ray outperforms while linearly scaling to TB-level data sizes 💽 📈 SageMaker Batch Transform by 17x 📊 Apache Spark by 2x and 3x

Offline Batch Inference | Ray, Apache Spark & SageMaker

We conduct a comparison of three different solutions for offline batch inference: AWS SageMaker Batch Transform, Apache Spark, and Ray Data.

www.anyscale.com

0

8

38

ray

@raydistributed

4 years

What enables Ray to be so much faster than Python multiprocessing? A combination of efficient handling of numerical data through @ApacheArrow and a set of abstractions more appropriate for building stateful services/actors.

1

14

35

ray

@raydistributed

3 years

🎉🍾🥳 Ray 1.3 is out! Featuring: * Published scalability limits () * Ray Client enabled by default * Object spilling is now turned on by default. * Faster autoscaling for Ray Tune * R2D2 @PyTorch and TF implementation for RLlib

Releases · ray-project/ray

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. - ray-project/ray

github.com

0

9

30

ray

@raydistributed

3 years

Growing demand for applications & HW specialization create huge burdens for learning systems at the center of intelligent applications today. At #RaySummit , see how @tqchenml addresses these challenges using the @XGBoostProject @ApacheTVM systems he built

0

10

29

ray

@raydistributed

3 years

🎉 Microsoft Researchers have developed FLAML (Fast Lightweight AutoML) which can now utilize Ray Tune for distributed hyperparameter tuning to scale up FLAML's resource-efficient & easily parallelizable algorithms across a cluster! 🎉 Learn more:

Fast AutoML with FLAML + Ray Tune | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

1

5

30

ray

@raydistributed

3 years

🔥 Modin () is a popular library that can scale your pandas workflows by changing one line of code -- using Ray! Learn how below:

How to Speed Up Pandas with Modin

The pandas library provides easy-to-use data structures like pandas DataFrames as well as tools for data analysis. One issue with pandas…

towardsdatascience.com

0

8

30

ray

@raydistributed

1 year

Blog: @LangChainAI provides an amazing suite of tools for everything around LLMs. There are tools (chains) for prompting, indexing, generating and summarizing text. While an amazing tool, using Ray with it can make LangChain even more powerful. 2/n

Building an LLM Open-Source Search Engine in 100 Lines

In part 1 of a new blog series, we show how to build a search engine in 100 lines using LLM embeddings and a vector database.

www.anyscale.com

1

3

29

ray

@raydistributed

1 year

Use gang-scheduling on Ray Clusters on #Kubernetes w/ #KubeRay & Multi-Cluster-App-Dispatcher (MCAD) to scale training #GLUE workloads 👉 Easy MCAD + KubeRay integration to scale Ray Clusters on #k8s 👉 Accelerate fine-tune #NLU tasks w/ multiple GPUs

Gang Scheduling Ray Clusters on Kubernetes with MCAD

Unblock your ML workload using KubeRay with the Multi-Cluster-App-Dispatcher (MCAD) Kubernetes controller.

www.anyscale.com

1

14

28

ray

@raydistributed

21 days

With Ray 2.11.0, we switched to weekly releases (previously every 6 weeks)! This is a huge change and will get features and fixes to users faster. This has been a big investment in our overall velocity.

Release Ray-2.11.0 · ray-project/ray

Release Highlights [data] Support reading Avro files with ray.data.read_avro [train] Added experimental support for AWS Trainium (Neuron) and Intel HPU. Ray Libraries Ray Data 🎉 New Features: S...

github.com

0

3

31

ray

@raydistributed

3 years

As technology has advanced, ML architectures have evolved. One way to see it is in terms of generations: - 1st gen involved "fixed function" pipelines - 2nd gen involved programmability within the pipeline What will be the next gen of ML architectures?

The 3rd Generation of Production ML Architectures | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

0

7

28

ray

@raydistributed

3 years

✨Ray is becoming a critical component for the next generation of ML platforms! Check out this recent blog post about how @Uber is leveraging Ray for elastic deep learning with Horovod to enable their rapidly growing usage of deep learning:

0

2

27

ray

@raydistributed

2 years

Exciting talk from @dariogila with @IBM on the future of quantum computing, and how @raydistributed could be the key for its success.

0

8

28

ray

@raydistributed

4 years

Imagine if your random forest classifier training/tuning was 30x faster while getting 5% more accurate. Wouldn't that be awesome? Today, by leveraging the RAPIDS library with Ray Tune, you can do that. See how in exciting new post: #GTC2020 #RayTune

30x Faster Hyperparameter Search with RayTune and RAPIDS

With RayTune and RAPIDS you can now tune Random Forest Classifiers 30x faster — while getting a 5% accuracy boost.

medium.com

RAPIDS AI

@RAPIDSai

4 years

With @rapidsai and @raydistributed #RayTune , you can now tune Random Forest Classifiers 30x faster -- while getting a 5% accuracy boost. See how.

0

21

58

0

12

28

ray

@raydistributed

4 years

0.8.6 is out! - Support for Windows (alpha)! - Releasing Ray Serve, a scalable model-serving library! Check out a tutorial for serving @PyTorch models: - Ray Dashboard now supports GPU monitoring! And more! Release notes:

Release ray-0.8.6 · ray-project/ray

Highlight Experimental support for Windows is now available for single node Ray usage. Check out the Windows section below for known issues and other details. Have you had troubles monitoring GPU ...

github.com

0

6

28

ray

@raydistributed

3 years

Check out the new @MLflow and @raydistributed integrations for tuning models, tracking experiments, and deploying models.

Ray & MLflow: Taking Distributed Machine Learning Applications to Production

By Amog Kamsetty and Archit Kulkarni

medium.com

1

2

27

ray

@raydistributed

4 years

"A Step-by-Step Guide to Scaling Your First Python Application in the Cloud" by Bill Chambers . You'll learn how to install Ray, create an app, test on your local machine, spin up a Ray cluster in the cloud, deploy your app, ... and more!

A Step-by-Step Guide to Scaling Your First Python Application in the Cloud

Every idea needs a Medium

link.medium.com

0

6

27

ray

@raydistributed

3 years

🎉🍾🥳 Ray 1.5 is out! Featuring: - Ray Datasets now in alpha - LightGBM on Ray in beta - The Ray cluster launcher now has support for launching clusters on Aliyun - RLlib added an improved "input API" for customizing offline datasets Learn more ⬇️

1

2

28

ray

@raydistributed

11 months

Announcing Ray 2.5 release features: 👉 Support #LLMs training with Ray Train 👉 Serve #LLMs with Ray Serve 👉 Multi-GPU learner stack in #RLlib for cost efficiency & scalable RL-agent training 👉 Performant & improved approach to batch inference at scale

Ray 2.5 | Training & Serving for LLMs, Multi-GPU Training & More

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

1

10

27

ray

@raydistributed

3 years

First sessions for #RaySummit program are up! Join the annual gathering of the global @raydistributed community for the latest in distributed computing. Speakers include @TravisAddair @eric_brewer @tqchenml @slbird @dawnsongtweets & more ➡️

0

13

25

ray

@raydistributed

1 year

Streaming distributed execution across CPUs and GPUs: Learn how Ray Data streaming works and how to use it for your own ML pipelines.

How to Stream Distributed Execution Across CPUs & GPUs

This blog post delves into how Ray Data streaming works and how to use it for your own ML pipelines distributed across both CPU and GPU devices.

www.anyscale.com

1

5

26

ray

@raydistributed

4 years

ML serving is broken - Ray Serve can fix it! Thread (1/n) 🙁Wrapping models in Flask doesn’t scale 🙁TorchServe, TFServing requires setting up a traditional web server 😊 Ray Serve lets you deploy your ML models with a simple Python interface!

Machine Learning Serving is Broken

And How Ray Serve Can Fix it

medium.com

1

5

25

ray

@raydistributed

2 years

Distributed libraries allow improved performance by exploiting the full bandwidth of distributed memory, and giving greater programmability. But how does that actually work? What does the code look like? Learn more ⬇️

Data Ingest in a Third-Generation ML Architecture | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

0

6

26

ray

@raydistributed

1 year

Ray 2.3.0 Released with: ⭐️ Observability enhancements ⭐️ Ray Dataset Streaming ⭐️Boost in Ray core performance ⭐️Gym/Gymnasium library in #RLlib ⭐️ Support ARM & Python 3.11 ⭐️ Support multiple applications in Ray Serve (developer preview)

Announcing Ray 2.3: performance improvements, new features and new platforms

The Ray 2.3 release features exciting improvements across the Ray ecosystem. In this blog post, we will highlight new features, performance enhancements, and support for new platforms.

www.anyscale.com

0

3

23

ray

@raydistributed

3 years

🎉🍾🥳 Ray 1.4 is out! Highlights include: - Ray Serve has a new deployment centric API! - Ray now has support for namespaces. (Docs: ) - RLlib now has multi-GPU support for PyTorch models! Learn more ⬇️

Releases · ray-project/ray

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. - ray-project/ray

github.com

0

6

24

ray

@raydistributed

3 years

🎉 New Introductory Tutorial on Reinforcement Learning (RL) with OpenAI Gym, RLlib, and Google Colab! 🎉 The tutorial explores: - What is RL - The OpenAI Gym CartPole Environment - The Role of Agents in RL & how to train them using RLlib

0

7

24

ray

@raydistributed

2 years

💥🎉 Ray version 1.9 is here! Featuring: ✅ Ray Train is now in beta! ✅ Ray Datasets now supports groupby and aggregations! ✅ Ray Docker images for multiple CUDA versions are now provided!

Ray version 1.9 has been released | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

1

2

24

ray

@raydistributed

3 years

💥🎉 Ray version 1.8 is here! Featuring: ✅ Ray SGD has been renamed to Ray Train ✅ Ray Datasets, now beta, has a new integration with Ray Train for scalable ML ingest ✅ Experimental support for Ray on Apple Silicon (M1 Macs)

0

5

23

ray

@raydistributed

1 year

Ray continues to enable #ML teams innovate at scale & unleash new use cases. @Spotify shares how #Ray helps #ML practitioners innovate & how they built ML platform atop Ray.

Spotify Engineering

@SpotifyEng

1 year

"Our goal for Spotify’s ML Platform has always been to create a seamless user experience for ML practitioners who want to take an ML application from development to production..." And so, we introduced @raydistributed to our @Spotify ecosystem.

0

14

58

0

5

22

ray

@raydistributed

3 years

🎉 New Tutorial on Serverless Kafka Stream Processing with Ray! Featuring: - Ray Clusters that autoscale to meet the demands of a stream processing job - How Ray can be paired with @apachekafka Learn more ⬇️

0

6

23

ray

@raydistributed

3 years

Ray 1.2 is up on GitHub and PyPI ()! 🎉 This is an important release with many new APIs and tons of new committers. Some highlights 👇

Release Release ray-1.2.0 · ray-project/ray

Release v1.2.0 Notes Highlights Ray client is now in beta! Check out more details here: https://docs.ray.io/en/master/ray-client.html XGBoost-Ray is now in beta! Check out more details about this ...

github.com

2

4

22

ray

@raydistributed

2 years

Co-creator of @PyTorch at Meta AI @soumithchintala shares how various project co-exist with @raydistributed at #raysummit .

0

4

23

ray

@raydistributed

4 years

🙌🙌 With the v3.0 release, you can use Ray to train @spacy_io on one or more remote machines, potentially speeding up your training process.

Introducing spaCy v3.0 · Explosion

spaCy v3.0 is a huge release! It features new transformer-based pipelines that get spaCy's accuracy right up to the current state-of-the-art, and a new workflow system to help you take projects from...

explosion.ai

spaCy

@spacy_io

4 years

IT'S HERE! Today we're releasing spaCy nightly, the first candidate for the upcoming v3.0. 🛸 Transformer-based pipelines for SOTA models ⚙️ New training & config system 🧬 Models using any framework 🪐 Manage end-to-end workflows 🔥 New & improved APIs

12

165

535

0

9

21

ray

@raydistributed

4 years

Surprisingly, most popular key-value stores don't support shared-memory! The Plasma Store, part of @ApacheArrow , does. In conjunction with Arrow’s data layout, this enables super fast sharing of data between multiple processes on the same machine.

1

13

23

ray

@raydistributed

2 years

The brains behind the operation 🧠

0

2

22

ray

@raydistributed

10 months

The Ray 2.6.1 released with : 🎏 Streaming responses in Serve for real-time capabilities 🎏 📀🏃‍♀️Ray Data streaming integration w/Train 🏃‍♀️☁️Distributed Training & Tuning sync with cloud storage persistence 🤖 Alpha release of the Multi-GPU Learner API 📙 Ray Gallery examples 👇

1

3

22

ray

@raydistributed

3 years

Announcing a collaboration between PyCaret + Ray! 🔥PyCaret () is a popular low-code ML library in Python. A new contributed blog shows how #PyCaret integrated Ray's tune-sklearn () to simplify model tuning!

Bayesian Hyperparameter Optimization with tune-sklearn in PyCaret

Here’s a situation every PyCaret user is familiar with: after selecting a promising model or two from compare_models(), it’s time to tune…

medium.com

0

4

21

ray

@raydistributed

4 years

New blog post, "Scaling Python Asyncio with Ray" by Simon Mo

Scaling Python Asyncio with Ray

Every idea needs a Medium

link.medium.com

0

12

22

ray

@raydistributed

3 years

In Ray 1.0.1, we're releasing Population-based Bandits (PB2), a new method for tuning neural networks published in #NeurIPS2020 by @jparkerholder and @nguyentienvu ! 🚀 PB2 can perform up to 6x more efficiently than methods like Hyperband, PBT. 🔖 Read:

Population Based Bandits: Provably Efficient Online Hyperparameter Optimization | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

0

7

21

ray

@raydistributed

3 years

At #RaySummit , @vanpelt will discuss the @weights_biases tool Tables + new Artifacts features that let you visualize & query datasets & model evaluations at the example level as well as integrate with Ray. Register:

0

6

21

ray

@raydistributed

1 month

Very impressive to see how @canva is using LLMs and image generation to transform the design world.

Anyscale

@anyscalecompute

1 month

Canva is a leader in generative AI and modernized their AI platform with @raydistributed . Some key challenges - Scaling training on more GPUs and far more data. - Unifying generative AI and non-generative models. - Flexibility to support different clouds and accelerators. This…

1

8

19

1

7

20

ray

@raydistributed

3 years

🎉 Ant Group has developed Ant Ray Serving which is an online service framework based on Ray, which provides users with a Serverless platform to publish Java/Python code as online services & allows them to focus on their own business logic 🎉 Learn more:

Building Highly Available and Scalable Online Applications on Ray at Ant Group | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

0

5

20

ray

@raydistributed

3 years

4 common patterns of serving ML models in production are: pipeline, ensemble, business logic, & online learning. Implementing these patterns typically involves a tradeoff between easy development and production readiness. Learn how Ray Serve changes this

0

6

20

ray

@raydistributed

1 year

As part of our efforts on #observability , a novel feature: "Automatic and optimistic memory scheduling for ML workloads in Ray" 👉 minimal configuration 👉 policy-based mitigation of #OOM errors w/retriable tasks 👉 debug OOM problems w/ the monitor

Automatic Memory Scheduling for ML Workloads in Ray

Learn about Ray's new out of memory (OOM) monitor and detection feature — all part of our efforts to make Ray easy to observe & debug for ML engineers.

www.anyscale.com

1

6

21

ray

@raydistributed

2 years

Distributed C++ systems are more difficult to put into production than single machine systems due to communication, deployment, and fault tolerance issues. The new Ray C++ API was designed to help to address these problems. Learn more ⬇️

Modern Distributed C++ with Ray | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

0

2

21

ray

@raydistributed

3 years

⚡️In Ray 1.2, we’re improving Ray support for distributed data processing! Featuring: - 💿External storage support - ✨Support for Python data processing libraries Use @ApacheSpark , @dask_dev DataFrames alongside ML libraries on Ray like Horovod! Blog:

Data processing support in Ray

Authors: Sang Cho, Alex Wu, Clark Zinzow, Eric Liang, Stephanie Wang

medium.com

0

5

20

ray

@raydistributed

3 years

💥🎉 Ray version 1.7 is here! Featuring: ✅ Ray SGD v2, now alpha, introduces APIs that focus on ease of use and composability ✅ Ray Workflows is in alpha. Try it out for your large data, ML, and business workflows ✅ Major enhancements to the C++ API

1

5

21

ray

@raydistributed

3 years

🎉 Introducing Distributed XGBoost Training with Ray! Featuring: - Distributed training by only changing three lines of code - Distributed hyperparameter tuning with Ray Tune - Support for Pandas, Modin, & even Dask Dataframes! Learn more ⬇️

Introducing Distributed XGBoost Training with Ray | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

1

6

19

ray

@raydistributed

3 years

You can configure and Scale ML with @Hydra_Framework and Ray on AWS or local Ray clusters. Blog Post:

Configuring and Scaling ML with Hydra + Ray

Launch your Hydra applications on the cloud with the new Hydra-Ray integration!

medium.com

0

5

21

ray

@raydistributed

2 years

There’s an even divide between developers choosing a generic #Python web server such as @FastAPI and a specialized ML serving solution framework. Check out our latest blog post for more on each option and explore why you might choose one over the other:

Ray Serve + FastAPI: The best of both worlds | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

0

5

21

ray

@raydistributed

2 years

#RaySummit is almost here! Don’t miss out on: 🌁 In-person networking in SF 🎒 3 in-depth Ray training sessions ⚙️ 40+ technical sessions and lightning talks 🎤 Speakers from @MetaAI , @Spotify , @IBM & more ...and much more!

1

8

19

ray

@raydistributed

3 years

🎉 Really exciting blog from @UberEng on moving distributed @XGBoostProject onto Ray along with parallel efforts to move Elastic #Horovod onto Ray! This is a critical step towards a unified distributed compute backend for end-to-end machine learning workflows at Uber!

Uber Engineering

@UberEng

3 years

New on our blog today! Members of our engineering team describe how they co-developed Distributed XGBoost on Ray with the Ray team @raydistributed to tackle various production challenges of doing distributed machine learning at scale. read more:

1

12

32

0

3

19

ray

@raydistributed

3 years

Ray has many ML integrations such as Horovod and 🤗 to data processing frameworks such as Spark, Modin, and Dask. But what does it mean to be "integrated with Ray"? And what benefits does it provide to library developers and users? Learn more ⬇️

Ray Distributed Library Patterns | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

0

7

19

ray

@raydistributed

4 years

Since it was first released Ray Tune is a leading way of scaling ML tuning. But there's a gap - experiment management & ML tracking. To close this, we're happy to announce an integration with @weights_biases ! Read about it here:

1

7

20

ray

@raydistributed

2 years

As modern hardware systems get more complex, it’s becoming more difficult to design integrated circuit implementations. Check out the blog post from the @IBMResearch team to learn how they use AI/ML-driven chip design and Ray to solve this challenge:

Infusing AI and ML into integrated circuit design for faster chip delivery, better chip performance...

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

0

14

19

ray

@raydistributed

3 years

After training a #MachineLearning model, the model needs to be deployed for online serving and offline processing. At #RaySummit , @simon_mo_ will walk through the journey of deploying ML models in production and how Ray Serve was built. Register:

0

8

19

ray

@raydistributed

4 years

The performance numbers resulting from the ongoing re-architecture are impressive! Here's why:

How Ray Uses gRPC (and Arrow) to Outperform gRPC

This blog post explains how the Ray 0.8 release uses gRPC and Arrow to provide a distributed Python API that is both fast and simple.

medium.com

0

2

20

ray

@raydistributed

2 years

🎉 Ray 1.12 is here! This release includes the alpha of Ray AI Runtime (AIR), a new, unified experience for seamless integration across the Ray ecosystem. 📢 Shoutout to all of the community members who supported this release. Learn all about it here:

Ray 1.12: Ray AI Runtime (alpha), usage data collection, and more | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

1

11

19

ray

@raydistributed

3 years

A distributed shuffle is a data intensive-operation that usually calls for a system built specifically for that purpose. Even though its core API contains no shuffle operations, Ray can do it in just a few lines of Python. Learn how 👇

Executing a distributed shuffle without a MapReduce system

Author: Stephanie Wang

medium.com

0

2

18

ray

@raydistributed

1 year

Last week, we released Ray 2.3. ICYMI: ⭐️ Observability enhancements ⭐️ Ray Dataset Streaming ⭐️ Boost in Ray core performance ⭐️Gym/Gymnasium library in #RLlib ⭐️ Support ARM & Python 3.11 ⭐️ Support multiple applications in Ray Serve

Announcing Ray 2.3: performance improvements, new features and new platforms

The Ray 2.3 release features exciting improvements across the Ray ecosystem. In this blog post, we will highlight new features, performance enhancements, and support for new platforms.

www.anyscale.com

0

1

19

ray

@raydistributed

2 years

🎉 How to Speed Up XGBoost Model Training Tutorial! 🎉 The tutorial explores approaches to speeding up XGBoost training like: - Changing tree construction algorithm - Cloud computing - Distributed XGBoost on Ray

How to Speed Up XGBoost Model Training | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

0

3

18

ray

@raydistributed

4 years

Using @raydistributed with @scikit_learn . @AmeerHajAli shows you how. . The technique leverages Ray's implementation of joblib. He also shows performance measurements of Ray vs. other tools, Loky, Multiprocessing, and Dask.

Easy Distributed Scikit-Learn Training with Ray

TL;DR: Scale your scikit-learn applications to a cluster with Ray’s implementation of joblib’s backend.

medium.com

0

10

18

ray

@raydistributed

3 years

🎉New blog post on the most popular RL talks from Ray Summit 2021! Including: - 24x Speedup for RL (Raoul Khouri) - Orchestrating Robotics Operations with SageMaker + RLlib ( @SahikaGenc ) - Offline RL with RLlib ( @edilmop ) - Neural MMO ( @jsuarez5341 )

Blog | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

0

7

16

ray

@raydistributed

11 months

Introducing RLlib Multi-GPU Stack for Cost-Efficient, Scalable, Multi-GPU RL Agents Training ⭐️ Achieve up to 1.7x infrastructure cost savings ⭐️ Use RLlib workers to scale out the batch collection ⭐️ Use Data distributed parallel to scale out GPUs #RL

RLlib Multi-GPU Stack | Affordable, Scalable RL Agents Training

Learn how you can achieve up to 1.7x infrastructure cost savings by using the newly introduced multi-GPU training stack in RLlib.

www.anyscale.com

0

3

18

ray

@raydistributed

3 years

#RaySummit highlight: lead Horovod maintainer @TravisAddair will show how Ludwig combines Dask on Ray for distributed out-of-memory data preprocessing, Horovod on Ray for distributed training, and Ray Tune for hyperparameter optimization. Register free at

0

8

17

ray

@raydistributed

8 months

Want to learn how to build and evaluate production RAG app with @llama_index and @raydistributed ? Join #RaySummit Training Day! 1️⃣ Implement reliable eval methods for LLMs 2️⃣ Run experiments to optimize app components 3️⃣ Take best configs to production

0

7

18

ray

@raydistributed

3 years

💥 🎉 Ray version 1.6 is here! Featuring: ✅ Ray Datasets adds native support for large-scale data loading ✅ Ray Autoscaler adds TPU support ✅ Ray Lightning brings fast & easy parallel training to PyTorch Lightning ✅ Runtime Environments are now GA

Ray version 1.6 is released | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

0

2

17

ray

@raydistributed

3 years

New blog comparing the memory management and performance of "Dask-on-Ray'' versus #Dask with its built-in scheduler. Check it out ⬇️

Analyzing Memory Management & Performance in Dask-on-Ray

In this blog, we compare the memory management and performance of Dask versus Ray (or "Dask-on-Ray'') with its built-in scheduler.

www.anyscale.com

0

2

16

ray

@raydistributed

3 years

A 3rd generation ML platform offers full programmability for ML workflows & includes a programmable compute layer. Check out this blog to learn: - How Ray improves performance by 3-9x in production workloads. - Emerging patterns of distributed compute

Blog | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

0

2

17

ray

@raydistributed

2 years

Are your ML pipelines getting longer, wider, and more dynamic? Learn how #RayServe makes it easier than ever to compose complex deployment graphs, and see a real-world example of how to build and deploy a deployment graph with the API ⬇️

Multi-model composition with Ray Serve deployment graphs | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

0

4

17

ray

@raydistributed

2 years

🎉 Ray 1.11 is here! Highlights include: ✅ Ray no longer starts Redis by default ✅ A new, more intuitive experience for the Ray docs ✅ Python 3.9 support is now stable Check out the release blog for the details:

Ray 1.11: Redisless Ray, a docs redesign, and Python 3.9 support | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

0

2

16

ray

@raydistributed

2 years

🎉 We're excited to announce Ray Datasets, a data loading and preprocessing library built on Ray. Check out our blog, where we review the current state of distributed training and model scoring pipelines and how Datasets can help 💪

Ray Datasets for large-scale machine learning ingest and scoring | Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly.

www.anyscale.com

0

4

17

ray

@raydistributed

4 years

A Software Engineering Daily podcast with @anyscale cofounder, Ion Stoica, where he discusses @raydistributed , the state of ML/AI, and related topics:

Anyscale with Ion Stoica - Software Engineering Daily

Machine learning applications are widely deployed across the software industry. Most of these applications used supervised learning, a process in which labeled data sets are used to find correlati...

softwareengineeringdaily.com

0

5

17

ray

@raydistributed

2 years

Watch Co-Founder @robertnishihara discuss the future of Ray live now at #raysummit !

0

1

17

ray

@raydistributed

4 years

Python multiprocessing is a staple of parallel Python, but scalable Python apps have new requirements: 1) multiple machines 2) stateful services/actors that communicate 3) failure handling 4) efficient large numerical data. Why? 👇

1

5

17

ray

@raydistributed

3 years

Excellent tutorial series by @psychothan on Scaling Data Science with Python and Ray! It includes: - An Introduction to Distributed Computing - Using remote functions (tasks) in Ray with Python Check it out:

Introduction to Distributed Computing with the Ray Framework

In this video, I give a brief introduction to distributed computing concepts and show how the Ray framework provides elegant abstractions for scaling data sc...

www.youtube.com

0

8

15

ray

@raydistributed

3 years

Thanks for a wonderful #RaySummit ! If you missed #RaySummit or want to watch anything again, all the keynotes & sessions will be available on-demand until July 24, 2021!

0

3

15

ray

@raydistributed

1 year

Thanking all of you in the #Ray community for all your contributions, small or big, in 2022 and wishing you all a HAPPY NEW YEAR 2023 🎇 🥂🍾 from the #RayTeam at @anyscalecompute Onwards & upwards for #Ray in 2023!

0

2

16

ray

@raydistributed

4 years

Serialization and deserialization are fundamental components of any distributed system (typically bottlenecks). @ApacheArrow solves some of the key serialization issues related to performance, shared memory, and framework independence.

1

4

16

ray

@raydistributed

9 months

New LLM service that app developers can try for free. #ArtificialIntelligence #ML #AI #LLMs

Goku Mohandas

@GokuMohandas

9 months

We ( @pcmoritz & I) have been productionizing LLM apps (more later) but at the heart are OSS LLMs served via @anyscalecompute Endpoints. - ✅ Drop-in sub for OpenAI - ☁️ Deploy on own cloud if needed - 💸 < $1 / M tokens for Llama-2-70b Try it for free 👉

3

7

58

0

1

14

ray

@raydistributed

1 year

📣 If you are a Ray user, we want to hear your story at #RaySummit 2023! Participation from the Ray community is what makes Ray Summit successful! We're accepting proposals for lightning talks, technical talks, and case studies.

1

2

16