You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/source/reference/api-server/api-server-tunning.rst
+9-7Lines changed: 9 additions & 7 deletions
Original file line number
Diff line number
Diff line change
@@ -15,7 +15,9 @@ The number of requests that the API server can handle concurrently is proportion
15
15
16
16
.. note::
17
17
18
-
Though a task (or job) can run for any length of time, concurrent tasks does not occupy the concurrency. Because once a task is submitted to the cluster, it will be detached and no longer takes any resources off the API server.
18
+
Requests are queued and processed by the API server. Therefore, they only take resources off the API server when they are in queue or being processed. Once requests are processed and remote clusters start doing real work, they no longer require API server's resources or count against its concurrency limit.
19
+
20
+
For example, long-running requests for ``launch`` and ``exec`` no longer take resources off the API server once a cluster has been provisioned, or once a job has been submitted to a cluster, respectively.
19
21
20
22
It is recommended to tune the resources allocated to the API server based on the expected concurrency to avoid queuing:
21
23
@@ -35,7 +37,7 @@ It is recommended to tune the resources allocated to the API server based on the
35
37
36
38
.. note::
37
39
38
-
If you specify a resources that is lower than the minimum recommended resources (4 CPUs with 8GB of memory) for team usage, an error will be raised on ``helm upgrade``. You can specify ``--set apiService.skipResourcesCheck=true`` to skip the check if performance and stability is not an issue for you scenario.
40
+
If you specify a resources that is lower than the recommended minimum resources for team usage (4 CPUs with 8GB of memory, which is also the default value when ``apiService.resources`` are not specified), an error will be raised on ``helm upgrade``. You can specify ``--set apiService.skipResourcesCheck=true`` to skip the check if performance and stability is not an issue for you scenario.
39
41
40
42
.. dropdown:: Why set the same value for the limits and requests?
41
43
@@ -91,8 +93,8 @@ The following table shows the maximum concurrency for different resource configu
91
93
- 256 Long requests, 589 Short requests
92
94
- 10~100 users
93
95
94
-
Queuing requests and polling status asynchronously
There is no limit on the number of queued requests. To avoid request blocking, you can either (1) allocate more resources to increase the maximum concurrency (described above), or (2) :ref:`submit requests asynchronously <async>` (``--async``) and poll the status asynchronously.
98
100
@@ -118,7 +120,7 @@ The requests will be queued on the API server and be processed in submission ord
118
120
5667cff2-e953-4b80-9e5f-546cea83dc59 alice sky.jobs.launch a few secs ago RUNNING
119
121
120
122
Checking the logs of a request
121
-
^^^^^^^^^^^^^^^^^^^^^^^^
123
+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
122
124
123
125
There should be some ``RUNNING`` requests that occupy the concurrency limit. Usually the ``RUNNING`` requests make progress and finally your requests will be processed, but if the ``RUNNING`` requests are stuck, you can inspect the request log with:
124
126
@@ -128,7 +130,7 @@ There should be some ``RUNNING`` requests that occupy the concurrency limit. Usu
128
130
$ sky api logs <request_id>
129
131
130
132
Canceling a request
131
-
^^^^^^^^^^^^^^^^
133
+
^^^^^^^^^^^^^^^^^^^
132
134
133
135
If the request is stuck according to the log, e.g. retrying to launch VMs that is out of stock, you can cancel the request with:
134
136
@@ -147,7 +149,7 @@ To avoid this, it is recommended to run ``sky logs`` and ``sky jobs logs`` with
147
149
148
150
sky logs --no-follow my_cluster
149
151
150
-
Commands that execute jobs like ``sky jobs launch`` and ``sky exec`` will also tail the logs of the job after the job is started by default. You can add ``--async`` flag to submit the job without tailing the logs:
152
+
Commands that execute tasks like ``sky jobs launch`` and ``sky exec`` will also tail the logs of the task after the task is started by default. You can add ``--async`` flag to submit the job without tailing the logs:
0 commit comments