Skip to content

[pull] master from ray-project:master#1234

Merged
pull[bot] merged 6 commits intofishbone:masterfrom
ray-project:master
Sep 5, 2023
Merged

[pull] master from ray-project:master#1234
pull[bot] merged 6 commits intofishbone:masterfrom
ray-project:master

Conversation

@pull
Copy link

@pull pull bot commented Sep 5, 2023 •
edited
Loading

See Commits and Changes for more details.

Created by pull[bot]

Can you help keep this open source service alive? 💖 Please sponsor : )

vitsai and others added 6 commits September 6, 2023 00:25
@vitsai
[core] Expose Redis getter to Python and use to retrieve session name (…
25c0e57 
…#39194)

If only GCS communicates with Redis, there is no way to initialize the log directories using persisted values. However, these directories are required to exist before starting GCS. Expose a function to the Python layer specifically to retrieve these keys from Redis and set them. Follow-ups will ensure that these keys are only set and retrieved through this interface.
@angelinalg
[Doc] [Clusters] Update text to remove references to Ray Client (#39217)
5b75428 
Because we removed the Ray Client option in the diagram (#38337), the text needs to be updated, too.

Signed-off-by: angelinalg <122562471+angelinalg@users.noreply.github.com>
@GeneDer
[CI] Skip getting logs for long running jobs (#39146)
3fe8b6c 
For long running jobs (over ~4 hours) we are getting Exited with status -1 (agent lost) during Fetching Results step due to we are collecting and downloading too many logs. Buildkite agent seems to be exited automatically during this process. This PR skips downloading ray logs for those long running jobs so the test can finish successfully.
@raulchen
[data] Removing unnecessary data copy in convert_udf_returns_to_numpy (…
a38eb53 
…#39188)

---------

Signed-off-by: Hao Chen <chenh1024@gmail.com>
@fishbone
[core] Ship the libjemalloc.so with wheel (#39241)
10c461c 
## Why are these changes needed?
In [PR](#39192), jemalloc is added, but it's not delivered with the wheel. This PR fixed it.
@peytondmurray @maxpumperla
[Doc] Use autodoc_mock_imports to mock deps for sphinx builds (#38817)
74e97de 
This PR is the first in a series needed to update Sphinx to the latest version. In this PR, we

- Remove a lot of external dependencies that aren't necessary for building the docs. The one exception to this is tune-sklearn, a project owned by ray-project which actually does have documentation hosted on the Ray docs site.
- External dependencies are now mocked out using Sphinx's autodoc_mock_imports mechanism, which is used by both autodoc and autosummary. The old module mocking mechanism has been removed in favor of this. The one exception to this is packaging.version.Version: Ray currently uses Version to modify some behaviors depending on the version of certain dependencies that the user has installed. 

---------

Signed-off-by: pdmurray <peynmurray@gmail.com>
Co-authored-by: Max Pumperla <max.pumperla@googlemail.com>
@pull pull bot requested a review from ericl as a code owner September 5, 2023 20:31
@pull pull bot added the ⤵️ pull label Sep 5, 2023
@pull pull bot merged commit 74e97de into fishbone:master Sep 5, 2023
pull bot pushed a commit that referenced this pull request Apr 29, 2024
@shrekris-anyscale
[Serve] Allow methods to pass type @serve.batch type hint (ray-proj…
e3fb472 
…ect#45004)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

Without this change, `mypy` raises a type warning when decorating
methods with `@serve.batch`:

<img width="504" alt="Screenshot 2024-04-26 at 5 14 00 PM"
src="https://github.com/ray-project/ray/assets/92341594/fd535d23-2a41-4f62-bb38-5f41a8bf53d5">

With this change, `mypy` no longer raises a type warning for methods:

<img width="505" alt="Screenshot 2024-04-26 at 5 14 35 PM"
src="https://github.com/ray-project/ray/assets/92341594/b14c4167-30e9-417e-a920-d694629b8038">

We should be aware that functions with an extra parameter before the
input list will now pass the type check. This allows users to define a
method outside a class and then assign it to a class:

<img width="329" alt="Screenshot 2024-04-26 at 5 17 30 PM"
src="https://github.com/ray-project/ray/assets/92341594/54043cf3-677a-48b8-a83b-10bec2ee6053">

## Related issue number

<!-- For example: "Closes #1234" -->

N/A

## Checks

- [X] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [X] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - This change relies on existing tests.

Signed-off-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
pull bot pushed a commit that referenced this pull request May 1, 2024
@JoshKarpel
Allow configuring Serve control loop interval, add related docs (ray-…
23d05bb 
…project#45063)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

In our experiments, adjusting this value upward helps the Serve
Controller keep up with a large number of autoscaling metrics pushes
from a large number of `DeploymentHandle`s (because the loop body is
blocking, so increasing the interval lets more other code when the
control loop isn't running), at the cost of control loop responsiveness
(since it doesn't run as often).

## Related issue number

<!-- For example: "Closes #1234" -->

Closes ray-project#44784 ... for now!

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [x] I've run `scripts/format.sh` to lint the changes in this PR.
- [x] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [x] This PR is not tested :(

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>
pull bot pushed a commit that referenced this pull request May 3, 2024
@nikitavemuri
[Jobs] Move JobSupervisor and JobLogStorageClient to separate files (r…
4ac54f1 
…ay-project#45116)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?
- Refactor to move `JobSupervisor` and `JobLogStorageClient` to separate
file for ray-project#45086
- This PR contains no logical changes, so no unit tests added

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [x] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [x] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
- [x] Unit tests -- no logical changes so existing unit tests are
sufficient
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Nikita Vemuri <nikitavemuri@anyscale.com>
Co-authored-by: Nikita Vemuri <nikitavemuri@anyscale.com>
pull bot pushed a commit that referenced this pull request May 4, 2024
@GeneDer
[Serve] update serve build multi app doc (ray-project#45134)
8ed218f 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

A user reported the `--multi-app` flag doesn't work. This option is
already deprecated from the previous migration work and by default the
command will generate multi-app config. This PR updated the docs on
example command and the resulting config. This seems to be the only
place that's still mentioning `--multi-app`.

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: Gene Su <e870252314@gmail.com>
pull bot pushed a commit that referenced this pull request May 11, 2024
@antoni-jamiolkowski
[Doc] Update key-concepts.md (ray-project#45245)
3295641 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

The text in the [DeploymentHandle (composing
deployments)](https://docs.ray.io/en/latest/serve/key-concepts.html#deploymenthandle-composing-deployments)
paragraph of the Ray Serve documentation is not consistent with the code
snippet linked below it.

<!-- Please give a short summary of the change and the problem this
solves. -->

## Related issue number

There isn't related issue. Should I create one?

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: Antoni Jamiołkowski <50143141+antoni-jamiolkowski@users.noreply.github.com>
pull bot pushed a commit that referenced this pull request May 22, 2024
@maxpumperla
[docs] fix ray assistant, closes ray-project#45416 (ray-project#45491)
edfb1d5 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: Max Pumperla <max.pumperla@googlemail.com>
pull bot pushed a commit that referenced this pull request Jun 3, 2024
@pcmoritz
Make Ray compile on Python 3.12 (ray-project#45653)
679a2ac 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

Fixes ray-project#45477

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(
pull bot pushed a commit that referenced this pull request Jun 9, 2024
@bthananjeyan
[Cluster launcher] [Azure] add option for eviction policy in azure VM…
e38e576 
… template (ray-project#45397)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?
This is necessary to give users the option to pick an eviction policy
(Delete vs Deallocate) when nodes in the cluster are pre-empted.

<!-- Please give a short summary of the change and the problem this
solves. -->
This exposes an additional option that can be set in the `node_config`
attribute in the cluster YAML file.

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [x] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [x] This PR is not tested :(


---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
  - https://app.asana.com/0/0/1207339070862431

Signed-off-by: bthananjeyan <brijen@ambirobotics.com>
pull bot pushed a commit that referenced this pull request Jun 9, 2024
@shrekris-anyscale @GeneDer
[Serve] Keep object_store_memory an int (ray-project#45781)
9b56f4d 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

Ray Serve converts `object_store_memory` into a float, but Ray Core
requires it to be an int. This change makes Serve keep it an int.

**Note:** I added the unit test to `test_regression.py`, but it makes
more sense to keep it in `test_api.py`. I'm going to hold off on adding
it there until ray-project#45780 is
resolved. If I add it to `test_api.py` now, the test passes and then
hangs while cleaning up.

## Related issue number

<!-- For example: "Closes #1234" -->

Closes ray-project#45321

## Checks

- [X] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [X] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [X] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Signed-off-by: shrekris-anyscale <92341594+shrekris-anyscale@users.noreply.github.com>
Co-authored-by: Gene Der Su <gdsu@ucdavis.edu>
pull bot pushed a commit that referenced this pull request Jun 11, 2024
@shrekris-anyscale
[Serve] Remove object_store_memory from Ray Serve (ray-project#45814)
84379cf 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

This change removes the `object_store_memory` option from
`ray_actor_options`. Ray Core doesn't support setting
`object_store_memory` at the task or actor level, so it's not meaningful
to set it in a Serve deployment.

Note that ray-project#45321 has affected Serve since at least as far back as Ray
2.7, so `object_store_memory` hasn't been available in Serve recently
anyways.

## Related issue number

<!-- For example: "Closes #1234" -->

Closes ray-project#45321

## Checks

- [X] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [X] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
pull bot pushed a commit that referenced this pull request Jun 11, 2024
@chris-ray-zhang
Update Nav Link to be "Try Ray on Anyscale" (ray-project#45851)
25c7d7a 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: Chris Zhang <chris@anyscale.com>
pull bot pushed a commit that referenced this pull request Jun 18, 2024
@vincent-pli
[doc]Make vllm example works with latest vllm version (ray-project#45742
367dcb6 
)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

## Related issue number
ray-project#45739 

<!-- For example: "Closes #1234" -->

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [x] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: vincent-pli <justdoit.pli@gmail.com>
pull bot pushed a commit that referenced this pull request Jun 27, 2024
@MortalHappiness
[Doc] Replace "docs/" with "doc/" (ray-project#46209)
e9109e6 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

See the description in the corresponding issue for details.

## Related issue number

<!-- For example: "Closes #1234" -->

Resolves: ray-project#46207

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [x] I've run `scripts/format.sh` to lint the changes in this PR.
- [x] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [x] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [x] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
pull bot pushed a commit that referenced this pull request Jun 27, 2024
@JoshKarpel
[Serve] Allow configuring handle cache size and controller max concur…
324c094 
…rency (ray-project#46278)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

The Serve Controller's scalability depends a lot on how many tasks are
active on it. These two config attributes control how many long poll
clients might be active in the system, and how many of their requests
the controller can handle concurrently, and we've found that tuning them
is important when trying to tune the scalability of a Serve cluster.

Thoughts on whether I should discuss these settings on the Performance
page in the Serve docs, like I did for
https://github.com/ray-project/ray/pull/45063/files ?

<!-- Please give a short summary of the change and the problem this
solves. -->

## Related issue number


<!-- For example: "Closes #1234" -->

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [x] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [x] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [x] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>
pull bot pushed a commit that referenced this pull request Jun 29, 2024
@JoshKarpel
[Serve] Optimize DeploymentDetails.deployment_route_prefix_not_set() (
85f9a08 
ray-project#46305)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

We've identified this method as a hotspot when the Serve Controller is
tracking a lot of apps, taking ~5% of total time at 3k apps. Serializing
the whole object to check if a field is set is unnecessary in both
Pydantic v1 and v2... if you're willing to use a dunder field in v1 :)

After this change, this check takes no significant time.

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [x] I've run `scripts/format.sh` to lint the changes in this PR.
- [x] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [x] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [x] This PR is not tested :(

This behavior didn't seem to be tested anywhere already, and it looks
like it's just protecting a deprecation, so I didn't add a unit test,
but I checked that it does have the desired behavior manually:
```python
DeploymentDetails(
        name="foo",
        status=DeploymentStatus.HEALTHY,
        status_trigger=DeploymentStatusTrigger.UNSPECIFIED,
        message="m",
        deployment_config=DeploymentSchema(
            name="foo",
            route_prefix="/foo",
        ),
        target_num_replicas=1,
        replicas=[],
    )
```
```
pydantic.error_wrappers.ValidationError: 1 validation error for DeploymentDetails
deployment_config
  Unexpectedly found a deployment-level route_prefix in the deployment_config for deployment "foo". The route_prefix in deployment_config within DeploymentDetails should not be set; please set it at the application level. (type=value_error)
```

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>
pull bot pushed a commit that referenced this pull request Jul 12, 2024
@davidxia @anyscalesam
docs: warn about running multiple local Ray instances (ray-project#45836
8f70e02 
)

See
https://discuss.ray.io/t/error-in-ray-job-submit-on-local-machine-if-multiple-clusters-are-running-at-the-same-time/14723/7?u=davidxia

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

Will prevent user confusion

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [x] I've run `scripts/format.sh` to lint the changes in this PR.
- [x] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [x] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: David Xia <david@davidxia.com>
Co-authored-by: Samuel Chan <116198444+anyscalesam@users.noreply.github.com>
pull bot pushed a commit that referenced this pull request Jul 12, 2024
@GeneDer @shrekris-anyscale @angelinalg
[Serve] Add status code to retry when request timed out (ray-project#…
a92e27a 
…46527)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

It's not immediately apparent what's the status code returned by Serve
when the request timed out. Added a note section on the timeout setup so
user knows what status code to expect and potentially retry.

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Gene Su <e870252314@gmail.com>
Signed-off-by: Gene Der Su <gdsu@ucdavis.edu>
Co-authored-by: shrekris-anyscale <92341594+shrekris-anyscale@users.noreply.github.com>
Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com>
pull bot pushed a commit that referenced this pull request Jul 12, 2024
@JoshKarpel
[Serve] Fix return type of `_DeploymentHandleBase._get_or_create_rout…
306cd9e 
…er` (ray-project#46480)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

I'm definitely *not* using this private method in a hack in our code,
and `mypy` is complaining about it.

<!-- Please give a short summary of the change and the problem this
solves. -->

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [x] I've run `scripts/format.sh` to lint the changes in this PR.
- [x] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [x] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [x] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>
pull bot pushed a commit that referenced this pull request Jul 26, 2024
@akshay-anyscale
Add max queued requests docs (ray-project#46778)
2f4302a 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: akshay-anyscale <122416226+akshay-anyscale@users.noreply.github.com>
pull bot pushed a commit that referenced this pull request Jul 26, 2024
@akshay-anyscale
Update VLLM serving example for compatibility with new vLLM release (r…
519aeab 
…ay-project#46697)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: akshay-anyscale <122416226+akshay-anyscale@users.noreply.github.com>
pull bot pushed a commit that referenced this pull request Jul 31, 2024
@zcin
[serve] add cmdline options to control which microbenchmarks to run (r…
e079a90 
…ay-project#46798)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

Add command line flags that can be used to control which microbenchmarks
to run. The release test pipeline will use `--run-all`, but when running
the script for debugging, these flags make it a lot easier to run only
the tests you want.

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com>
pull bot pushed a commit that referenced this pull request Oct 5, 2024
@pcmoritz @bveeramani
[Data][Docs] Add reference to parquet predicate pushdown (ray-project…
5a84f14 
…#47881)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

Small fix to make predicate pushdown more discoverable

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>
Co-authored-by: Balaji Veeramani <balaji@anyscale.com>
pull bot pushed a commit that referenced this pull request Oct 5, 2024
@pcmoritz @bveeramani
[Data][Doc] Add tip about how to understand map_batches format (ray-p…
48a0444 
…roject#47394)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

Add a note to help users understand the format inside map_batches.

<!-- Please give a short summary of the change and the problem this
solves. -->

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>
Co-authored-by: Balaji Veeramani <bveeramani@berkeley.edu>
pull bot pushed a commit that referenced this pull request Oct 17, 2024
@MengjinYan @angelinalg
Fix Style Changes Suggested in ray-project#47871 (ray-project#48030)
35d95a7 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

This PR fixed the style suggestions in
ray-project#47871.

The original PR got accidentally merged before the suggestions are
fixed.

<!-- Please give a short summary of the change and the problem this
solves. -->

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>
Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com>
pull bot pushed a commit that referenced this pull request Oct 17, 2024
@GeneDer
[Serve] refactor add_grpc_address helper (ray-project#48033)
4dfa033 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

Follow up on ray-project#47720 to move the
helper into default_impl.py

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Gene Su <e870252314@gmail.com>
pull bot pushed a commit that referenced this pull request Oct 21, 2024
@win5923
[CI] Fix: flake8 rule B108: Found useless Tuple expression. Consider …
39cbdcc 
…either assigning it to a variable or removing it. (ray-project#48118)

## Why are these changes needed?

While running the pre-commit hook of flake8, the following error occurs
if Python version is 3.12. It's because the version of flake8 is too
old.

![image](https://github.com/user-attachments/assets/7c103728-2e48-42f3-8b2f-b47ab93e560b)

version:
- python: 3.12.7
- flake8: 7.1.1 
- flake8-bugbear: 24.8.19


<!-- Please give a short summary of the change and the problem this
solves. -->

## Related issue number

Closes ray-project#48065

<!-- For example: "Closes #1234" -->

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [x] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: win5923 <ken89@kimo.com>
pull bot pushed a commit that referenced this pull request Oct 23, 2024
@LeoLiao123
[CI] Fix flake8 rule E225: Missing whitespace around operator (ray-pr…
9c0064b 
…oject#48188)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->
See ray-project#47991 
When running the following `flake8` command to check for errors:
```
flake8 --select E225 --extend-exclude python/ray/core/generated,python/ray/serve/generated/,python/ray/cloudpickle/,python/ray/_private/runtime_env/_clonevirtualenv.py,doc/external/,python/ray/dashboard/client/node_modules
```
the following error occurs : 

![image](https://github.com/user-attachments/assets/e595a58e-677d-480f-9490-f52e62e4f0cf)



## Related issue number
Closes ray-project#48059 
<!-- For example: "Closes #1234" -->

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [x] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: LeoLiao123 <leoyeepaa@gmail.com>
pull bot pushed a commit that referenced this pull request Oct 26, 2024
@slfan1989 @thomasdesr
[Java] Upgrade Commons-io to 2.14 (ray-project#44437)
23cc23b 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

[Java] Upgrade Commons-io to 2.14

commons-io can be upgraded to 2.14.0. commons-io 2.7 is an older
version. commons-io 2.14.0 has been verified for a long time and has no
direct or indirect CVE issues.

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: Shilun Fan <slfan1989@apache.org>
Co-authored-by: Thomas Desrosiers <681004+thomasdesr@users.noreply.github.com>
pull bot pushed a commit that referenced this pull request Oct 26, 2024
@vigneshka
Add IsHeadNode tag to node metrics (ray-project#47872)
dbcf4cc 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

Adding IsHeadNode tag to node metrics

<img width="1823" alt="Screenshot 2024-10-24 at 6 36 57 PM"
src="https://github.com/user-attachments/assets/855919db-b08e-4966-ae50-79c6de78bd90">
<img width="1818" alt="Screenshot 2024-10-24 at 6 36 47 PM"
src="https://github.com/user-attachments/assets/cb323682-d1c5-451a-98b2-eb99aff938a1">
<img width="1818" alt="Screenshot 2024-10-24 at 6 37 28 PM"
src="https://github.com/user-attachments/assets/f783cd67-e7da-4230-9f02-fa2d625a17e3">
<img width="1824" alt="Screenshot 2024-10-24 at 6 38 08 PM"
src="https://github.com/user-attachments/assets/08998ab1-7702-4fb3-8dea-76e5c8ab5232">


## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [x] I've run `scripts/format.sh` to lint the changes in this PR.
- [x] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [x] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [x] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [x] Unit tests
   - [x] Release tests
   - [x] This PR is not tested :(

---------

Signed-off-by: Vignesh Hirudayakanth <vignesh@anyscale.com>
pull bot pushed a commit that referenced this pull request Nov 16, 2024
@akyang-anyscale
[Serve] Modify max_concurrency in actor options to respect `max_ong…
d560ecd 
…oing_requests` (ray-project#47681) (ray-project#48274)

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->
This PR modifies the actor_options used when deploying replicas.
Deployment will use the configured `max_ongoing_requests` attribute of
the deployment config as the replica's `max_concurrency` if
the concurrency is not explicitly set. This is to prevent replica's
`max_concurrency` from capping
`max_ongoing_requests`.

## Related issue number

<!-- For example: "Closes #1234" -->
Closes ray-project#47681



Signed-off-by: akyang-anyscale <alexyang@anyscale.com>
pull bot pushed a commit that referenced this pull request Nov 16, 2024
@akyang-anyscale
[Serve] Expose ProxyStatus class to API docs (ray-project#43394) (ray…
bb4efd5 
…-project#48299)

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->
This PR moves `ProxyStatus` out of the `_private` directory, allowing it
to be included in the API docs. This is the final attribute of
`ServeStatus` that needs to be included in the documentation.

## Related issue number

<!-- For example: "Closes #1234" -->
Closes ray-project#43394

---------

Signed-off-by: akyang-anyscale <alexyang@anyscale.com>
pull bot pushed a commit that referenced this pull request Nov 16, 2024
@pcmoritz @brycehuang30 @angelinalg
[Debugger] Clarify that only one paused task can be debugged at a time (
1e5c582 
ray-project#48415)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

I was initially confused that I couldn't join another paused task while
a debugger was in "continue" mode.

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>
Co-authored-by: bhuang <bhuang@anyscale.com>
Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com>
pull bot pushed a commit that referenced this pull request Nov 16, 2024
@alexeykudinkin
[Data] Fix OutputBlockBuffer to avoid repeatedly copying remainder …
9b92557 
…block (ray-project#48266)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

Currently, inside `OutputBlockBuffer` we're

1. Repeatedly copying remainder of the original block, bringing total #
of bytes copied to O(N^2) (where N is the size of the original block)
2. Creating potentially very large blocks (like in
ray-project#48236) that could overflow
underlying Arrow data types.

This change addresses both of these issues, by establishing following
protocol where

1. Finalized target blocks *are* copied, while
2. Remainder block is NOT (therefore continuing referencing original
block)

Addresses ray-project#48236

<!-- Please give a short summary of the change and the problem this
solves. -->

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>
pull bot pushed a commit that referenced this pull request Nov 16, 2024
@pcmoritz
Make it possible to use Ray distributed debugger without setting RAY_…
c7263e4 
…DEBUG (ray-project#48301)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

Currently in order to use the distributed debugger, the user has to set
`RAY_DEBUG=1`. This has two disadvantages:

1. It is disruptive to the workflow and much more overhead than just
adding the `breakpoint()` instruction and re-running the program (since
the runtime environment has to be updated and the user needs to make
sure that the driver uses the flag too e.g. by restarting the python
kernel or in the worst case the container).
2. It is very easy to forget this step and then get the impression that
the debugger is not working.

There is no reason to require `RAY_DEBUG=1` to be set (the CLI debugger
works without the flag too and in particular the flag has no impact on
performance unless the debugger is actually entered). The reason this
flag was originally introduced was as a feature flag to switch between
the CLI debugger and the UI debugger. Now that the UI debugger is
getting more mature, it is better to make it the default and let people
who want to use the CLI debugger use a `RAY_DEBUG=legacy` flag.

This PR also renames the `RAY_PDB` flag to `RAY_DEBUG_POST_MORTEM` and
unifies the usage of the flag between the old and new debugger (in
particular, with the new debugger, post mortem debugging is now off
unless the user activates it).

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>
pull bot pushed a commit that referenced this pull request Nov 16, 2024
@andrewsykim
[Docs][KubeRay] Update KubeRay + Kueue guides to use newer versions o…
11aca59 
…f Kueue (ray-project#48564)

## Why are these changes needed?

Update KubeRay + Kueue guides to use newer versions of Kueue

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [X] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: Andrew Sy Kim <andrewsy@google.com>
pull bot pushed a commit that referenced this pull request Nov 16, 2024
@richardliaw
[data] Project Operator for select_columns (ray-project#48635)
31b8b41 
## Why are these changes needed?

Add Project operator to select_columns.

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
pull bot pushed a commit that referenced this pull request Nov 16, 2024
@bveeramani
[Data] Relax timeout on TFRecords benchmark (ray-project#48688)
4277ccc 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

The TFRecords release tests typically takes around 1680-1750s to
complete. Because the timeout is set to 1800s, if there's minor
variation in the job runtime, the job can timeout.

To avoid flakiness, this PR relaxes the timeout.

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
pull bot pushed a commit that referenced this pull request Nov 16, 2024
@bveeramani
[Data] Remove prefetch_batches and prefetch_blocks parameters of …
e0d4fd5 
…`iter_rows` (ray-project#48704)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

The `prefetch_blocks` and `prefetch_batches` parameters of `iter_rows`
have been deprecated for more than 6 months. In accordance with our API
policy, this PR removes them.

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
pull bot pushed a commit that referenced this pull request Nov 16, 2024
@bveeramani
[Data] Deprecate iter_tf_batches (ray-project#48693)
ba22e74 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

We recommend `to_tf` over `iter_tf_batches`. To avoid confusion, we
shouldn’t have two similar APIs, especially if we always prefer one.

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
pull bot pushed a commit that referenced this pull request Nov 16, 2024
@richardliaw
[data] Sort with None (ray-project#48750)
134e5ec 
## Why are these changes needed?

Adds a Sentinel value for making it possible to sort.

Fixes ray-project#42142 

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
pull bot pushed a commit that referenced this pull request Nov 16, 2024
@mohitjain2504 @richardliaw @GeneDer
Update kubernetes.md (ray-project#48082)
3435c25 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?
Fixed typo

<!-- Please give a short summary of the change and the problem this
solves. -->

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: mohitjain2504 <87856435+mohitjain2504@users.noreply.github.com>
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
Co-authored-by: Gene Der Su <gdsu@ucdavis.edu>
pull bot pushed a commit that referenced this pull request Nov 19, 2024
@ruisearch42
[core][compiled graphs] Fix flaky test CI command (ray-project#48761)
d4476ac 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->

Seeing the following errors for ":ray: core: flaky gpu tests" target:

```

[2024-11-15T17:50:08Z] ________ test_torch_tensor_nccl_overlap_timed[ray_start_regular1-True] _________
--
  | [2024-11-15T17:50:08Z]
  | [2024-11-15T17:50:08Z] ray_start_regular = RayContext(dashboard_url='127.0.0.1:8265', python_version='3.9.20', ray_version='3.0.0.dev0', ray_commit='{{RAY_COMMIT_SHA}}')
  | [2024-11-15T17:50:08Z] overlap_gpu_communication = True
  | [2024-11-15T17:50:08Z]
  | [2024-11-15T17:50:08Z]     @pytest.mark.parametrize(
  | [2024-11-15T17:50:08Z]         "ray_start_regular, overlap_gpu_communication",
  | [2024-11-15T17:50:08Z]         [({"num_cpus": 4}, False), ({"num_cpus": 4}, True)],
  | [2024-11-15T17:50:08Z]         indirect=["ray_start_regular"],
  | [2024-11-15T17:50:08Z]     )
  | [2024-11-15T17:50:08Z]     def test_torch_tensor_nccl_overlap_timed(ray_start_regular, overlap_gpu_communication):
  | [2024-11-15T17:50:08Z]         if not USE_GPU:
  | [2024-11-15T17:50:08Z]             pytest.skip("NCCL tests require GPUs")
  | [2024-11-15T17:50:08Z]
  | [2024-11-15T17:50:08Z] >       assert (
  | [2024-11-15T17:50:08Z]             sum(node["Resources"].get("GPU", 0) for node in ray.nodes()) >= 4
  | [2024-11-15T17:50:08Z]         ), "This test requires at least 4 GPUs"
  | [2024-11-15T17:50:08Z] E       AssertionError: This test requires at least 4 GPUs
  | [2024-11-15T17:50:08Z] E       assert 2.0 >= 4
  | [2024-11-15T17:50:08Z] E        +  where 2.0 = sum(<generator object test_torch_tensor_nccl_overlap_timed.<locals>.<genexpr> at 0x7f6c8799e200>)
```

This PR makes the config consistent with ":ray: core: multi gpu tests".

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
pull bot pushed a commit that referenced this pull request Nov 20, 2024
@JoshKarpel
[Serve] Handle multiple changed objects per `LongPollHost.listen_for_…
c6d26fd 
…change` RPC (ray-project#48803)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

Currently, in the `LongPollHost`/`LongPollClient`, if multiple objects
are updated that a `listen_for_change` request is waiting for *before
the async task in the host can run again*, only one of those updated
objects will be returned. This is inefficient because the
`LongPollClient` will immediately do a `listen_for_change` RPC again,
and that will see outdated snapshot IDs for the updates that weren't
returned and get all of the missed updates.

This is because of an asymmetry between
https://github.com/ray-project/ray/blob/b75cb793e437aa617d61dcb13e5f5d2fcc83ee68/python/ray/serve/_private/long_poll.py#L252-L272
, which looks for *all* outdated keys, and
https://github.com/ray-project/ray/blob/b75cb793e437aa617d61dcb13e5f5d2fcc83ee68/python/ray/serve/_private/long_poll.py#L309
, which only looks at a single complete `Event`, even if multiple events
completed during the
[`wait`](https://github.com/ray-project/ray/blob/b75cb793e437aa617d61dcb13e5f5d2fcc83ee68/python/ray/serve/_private/long_poll.py#L289-L293).

To prove that the `wait` can indeed see multiple completed `Event`s, see
this example:
```python
from asyncio import wait, Event, run, create_task, FIRST_COMPLETED


async def main():
    a = Event()
    b = Event()

    wait_for_a = create_task(a.wait())
    wait_for_b = create_task(b.wait())

    a.set()
    b.set()

    done, pending = await wait([wait_for_a, wait_for_b], return_when=FIRST_COMPLETED)

    print(f"{len(done)=}")
    print(f"{len(pending)=}")

run(main())

# len(done)=2
# len(pending)=0
```

Generally this won't be a big issue because most `listen_for_change`
requests in the current Serve setup are asking for a very small number
of keys and are likely to only get one key update anyway. But, as I've
been discussing with @edoakes and @zcin on Slack, I'd like to group up
the `DeploymentHandle` `listen_for_change` RPCs under a single
`LongPollClient`, which will be requesting many keys and is therefore
more likely to hit this situation.

To complement this change, I also changed `LongPollHost.notify_changed`
so that it takes multiple updates at the same time.

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [x] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [x] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [x] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Josh Karpel <josh.karpel@gmail.com>
pull bot pushed a commit that referenced this pull request Nov 20, 2024
@mimiliaogo
Pass logging_config to controller global logger (ray-project#48790)
9a19f79 
## Why are these changes needed?

<!-- Please give a short summary of the change and the problem this
solves. -->
Currently in serve.run the logging_config is not passed to controller.
This PR add this arguments into the function call so the logging_config
can be correctly specified for system-level logging.

## Related issue number
Closes ray-project#48652 
<!-- For example: "Closes #1234" -->


### Example
```
logging_config = {"log_level": "DEBUG", "logs_dir": "./mimi_debug"}
handle: DeploymentHandle = serve.run(app, logging_config=logging_config)
```

### Before
controller logs aren't saved in the specified logs_dir

<img width="326" alt="image"
src="https://github.com/user-attachments/assets/0d316428-e7a7-48e0-8d9d-1692a3045a4a">

### After
controller logs are correctly configured

<img width="325" alt="image"
src="https://github.com/user-attachments/assets/e05aba0b-75cd-4cd4-9a92-4ef8cdd84cce">

Signed-off-by: Mimi Liao <mimiliao2000@gmail.com>
pull bot pushed a commit that referenced this pull request Nov 21, 2024
@kevin85421
[core][autoscaler] Make the head group name consistent with the head …
a2a68b8 
…Pod's `ray.io/group` label. (ray-project#48840)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

The value of the `ray.io/group` label in the head Pod is `headgroup`,
whereas `KUBERAY_TYPE_HEAD` is `head-group`.

<img width="502" alt="image"
src="https://github.com/user-attachments/assets/9a06e643-d235-4237-a16a-ce131f3d9666">


## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: kaihsun <kaihsun@anyscale.com>
pull bot pushed a commit that referenced this pull request Nov 22, 2024
@akyang-anyscale
[Serve] Propagate replica constructor error to deployment status mess…
d4c2c8d 
…age and print num retries left (ray-project#48531)

## Why are these changes needed?

This change will surface the replica constructor error as soon as the
replica constructor fails for whatever reason. The exception will be
populated in the deployment status so that it's viewable from the ray
dashboard. Additionally, the number of replica constructor retries left
will also be updated in the error message. This will help users more
quickly debug a deployment that is failing to start.

## Related issue number

<!-- For example: "Closes #1234" -->
Closes ray-project#35604

Signed-off-by: akyang-anyscale <alexyang@anyscale.com>
pull bot pushed a commit that referenced this pull request Nov 23, 2024
@alexeykudinkin
Fixing pyarrow dep constraints in setup.py (ray-project#48886)
56413d2 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

This is a follow-up to a recent change upgrading minimal supported
PyArrow version from 6.0.1 to 9.0.0

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>
pull bot pushed a commit that referenced this pull request Nov 24, 2024
@ryanaoleary @kevin85421 @rickyyx
[Core][Autoscaler] Configure idleTimeoutSeconds per node type (ray-pr…
4a29571 
…oject#48813)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

Adds `idle_timeout_s` as a field to `node_type_configs`, enabling the v2
autoscaler to configure idle termination per worker type.

This PR depends on a change in KubeRay to the RayCluster CRD, since we
want to support passing `idleTimeoutSeconds` to individual worker groups
such that they can specify a custom idle duration:
ray-project/kuberay#2558

## Related issue number

Closes ray-project#36888

<!-- For example: "Closes #1234" -->

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [x] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [x] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: ryanaoleary <ryanaoleary@google.com>
Signed-off-by: ryanaoleary <113500783+ryanaoleary@users.noreply.github.com>
Co-authored-by: Kai-Hsun Chen <kaihsun@apache.org>
Co-authored-by: Ricky Xu <xuchen727@hotmail.com>
pull bot pushed a commit that referenced this pull request Nov 24, 2024
@kevin85421
[core][autoscaler] Health check logs are not visible in the autoscale…
559dd67 
…r container's stdout (ray-project#48905)

<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

* The Autoscaler container doesn't display information like `print("The
Ray head is ready. Starting the autoscaler.")` in STDOUT/STDERR for some
reason. To display logs to STDOUT/STDERR, we need to explicitly specify
`flush` in `print()` or use the logging module. I don't know why the
flush isn't triggered. The default end of `print` is `\n`, which should
trigger a line-buffered flush.

* Change `logging.warn` to `logging.warning` because `logging.warn` is
deprecated. See [this
doc](https://docs.python.org/3/library/logging.html#logging.Logger.warning)
for more details.

<img width="794" alt="image"
src="https://github.com/user-attachments/assets/12796aaa-ae7e-4986-96c8-94a0a42591b6">

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: kaihsun <kaihsun@anyscale.com>
pull bot pushed a commit that referenced this pull request Dec 2, 2024
@kevin85421 @jjyao
[core] Ray Core / Ray Data logging configuration leads to unexpected …
bf98e0d 
…behavior (ray-project#48958)

### Issue

In the Ray codebase,
[logging.config.dictConfig](https://docs.python.org/3/library/logging.config.html#logging.config.dictConfig)
may be called multiple times. However, we found that if a logger’s child
loggers are set before the logger is set via `dictConfig`, it may cause
issues.

*
[Example1](https://gist.github.com/kevin85421/24849e06c61f221fd95063a4ce81ca8f)
(incremental: False): The logger `Ray.data` loses its original handler
and uses the `Ray` logger’s handler after the Ray logger is set via
`dictConfig`.
  ```
2024-11-27 04:32:06,213 - Ray.data - INFO - This is an INFO log from
Ray.data.
2024-11-27 04:32:06,213 - Ray.data - WARNING - This is a WARNING log
from Ray.data.
  2024-11-27 04:32:06,213 - Ray.data - INFO - Ray data propagate False
  abc Ray - DEBUG - This is a DEBUG log from Ray.
  abc Ray - ERROR - This is an ERROR log from Ray.
  abc Ray.data - INFO - Another INFO log from Ray.data.
  abc Ray.data - INFO - Ray data propagate True
  ``` 
*
[Example2](https://gist.github.com/kevin85421/9cf6ee70ceec42be3de888174d0c8e6a)
(incremental: True): It looks like `Ray.data`’s handlers are removed
after the `Ray` logger is set via `dictConfig`.
  ```
2024-11-27 04:35:25,379 - Ray.data - INFO - This is an INFO log from
Ray.data.
2024-11-27 04:35:25,379 - Ray.data - WARNING - This is a WARNING log
from Ray.data.
  2024-11-27 04:35:25,379 - Ray.data - INFO - Ray data propagate False
  This is an ERROR log from Ray.
2024-11-27 04:35:25,379 - Ray.data - INFO - Another INFO log from
Ray.data.
  2024-11-27 04:35:25,379 - Ray.data - INFO - Ray data propagate False
  ```

* CPython implementation
  * Case 1: `incremental` is `False`
* If an existing logger is also a child logger of a new logger, the
child logger’s handlers will be reset, and its `propagate` attribute
will be set to true.
* In
[Example1](https://gist.github.com/kevin85421/24849e06c61f221fd95063a4ce81ca8f),
`Ray.data` is not only an existing logger but also a child logger of
Ray. Therefore, its handlers will be reset, and propagate will be set to
true.
* See the function for more details:
https://github.com/python/cpython/blob/71ede1142ddad2d31cc966b8fe4a5aff664f4d53/Lib/logging/config.py#L193-L196
 * Case 2: `incremental` is `True`
    * No handlers & filters will be added to the new logger.
* See the function for more details:
https://github.com/python/cpython/blob/71ede1142ddad2d31cc966b8fe4a5aff664f4d53/Lib/logging/config.py#L906-L915


### Solution

Instead of using `dictConfig` to set the root logger and the Ray logger,
call other functions to set the loggers explicitly.


## Related issue number

Closes ray-project#48732

<!-- For example: "Closes #1234" -->

## Checks

* Test 1
  ```python
  import ray
  import logging
  import ray.data
  
ray.init(logging_config=ray.LoggingConfig(encoding="TEXT",
log_level="INFO"))
  
  
  root_logger = logging.getLogger()
  root_logger.info("root logger")
  
  ray_logger = logging.getLogger("ray")
  ray_logger.info("ray logger")
  
  ray_data_logger = logging.getLogger("ray.data")
  ray_data_logger.info("ray data logger")
  
  @ray.remote
  def f():
      root_logger = logging.getLogger()
      root_logger.info("root logger")
      ray_data_logger = logging.getLogger("ray.data")
      ray_data_logger.info("ray data logger")
  
  
  ray.get(f.remote())
  ```
<img width="1440" alt="image"
src="https://github.com/user-attachments/assets/e522a257-28c5-4b3c-ad62-c41e4cd61664">


* Test 2
  ```python
  import ray
  import logging
  
  def report_logger(logger):
      # Collect this logger and its parents
      loggers = []
      current_logger = logger
      while current_logger:
          loggers.append(current_logger)
if not current_logger.parent or current_logger.parent == current_logger:
              break
          current_logger = current_logger.parent
  
      # Report the configuration of each logger in the hierarchy
print(f"Logging configuration for '{logger.name}' and its hierarchy:")
for log in reversed(loggers): # Start from the root and go down to the
given logger
print(f"\nLogger: {log.name or 'root'} (Level:
{logging.getLevelName(log.level)})")
          if log.handlers:
              print("  Handlers:")
              for handler in log.handlers:
print(f" - {handler.__class__.__name__} (Level:
{logging.getLevelName(handler.level)})")
          else:
              print("  No handlers configured")
  
  print("BEFORE")
  report_logger(logging.getLogger("ray.data"))
  print()
  
  import ray.data
ray.init(logging_config=ray.LoggingConfig(encoding="TEXT",
log_level="INFO"))
  
  print("AFTER:")
  report_logger(logging.getLogger("ray.data"))
  ```
<img width="1189" alt="image"
src="https://github.com/user-attachments/assets/9129b22a-f436-40ca-9f42-f1ecacf6c515">

Signed-off-by: kaihsun <kaihsun@anyscale.com>
Signed-off-by: Kai-Hsun Chen <kaihsun@apache.org>
Co-authored-by: Jiajun Yao <jeromeyjj@gmail.com>
pull bot pushed a commit that referenced this pull request Dec 2, 2024
@richardliaw
[data] Expression support for filters (ray-project#49016)
59ca821 
## Why are these changes needed?

Optimizes filtering in Ray Data, and introduces a new expression-based
syntax for filtering.

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ericl ericl Awaiting requested review from ericl

Assignees

No one assigned

Labels

⤵️ pull

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@vitsai @angelinalg @GeneDer @raulchen @fishbone @peytondmurray