You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/source/reference/api-server/api-server-tunning.rst
+8-6Lines changed: 8 additions & 6 deletions
Original file line number
Diff line number
Diff line change
@@ -10,8 +10,8 @@ Tuning API server resources
10
10
11
11
The number of requests that the API server can handle concurrently is proportional to the resources (CPU cores and memory) allocated to it. Internally, requests are categorized into two different types and handled in a first-in-first-out manner:
12
12
13
-
* ``Long-running request``: request that takes long time and more resources to run, including ``launch``, ``exec``, ``jobs.launch``, etc.
14
-
* ``Short-running request``: request that takes short time or less resources to run, including ``status``, ``logs``, etc.
13
+
* ``Long-running requests``: requests that take longer time and more resources to run, including ``launch``, ``exec``, ``jobs.launch``, etc.
14
+
* ``Short-running requests``: requests that take shorter time or less resources to run, including ``status``, ``logs``, etc.
15
15
16
16
.. note::
17
17
@@ -94,7 +94,9 @@ The following table shows the maximum concurrency for different resource configu
94
94
Queuing requests and polling status asynchronously
There is no limit on the number of queued requests, i.e. despite increasing the allocated resources to improve the maximum concurrency, you can also submit requests with :ref:`async<async>` (``--async``) and poll the status asynchronously to avoid blocking. For example:
97
+
There is no limit on the number of queued requests. To avoid request blocking, you can either (1) allocate more resources to increase the maximum concurrency (described above), or (2) :ref:`submit requests asynchronously <async>` (``--async``) and poll the status asynchronously.
98
+
99
+
For example:
98
100
99
101
.. code-block:: bash
100
102
@@ -115,7 +117,7 @@ The requests will be queued on the API server and be processed in submission ord
115
117
767182fd-0202-4ae5-b2d7-ddfabea5c821 alice sky.jobs.launch a few secs ago PENDING
116
118
5667cff2-e953-4b80-9e5f-546cea83dc59 alice sky.jobs.launch a few secs ago RUNNING
117
119
118
-
Check logs for a request
120
+
Checking the logs of a request
119
121
^^^^^^^^^^^^^^^^^^^^^^^^
120
122
121
123
There should be some ``RUNNING`` requests that occupy the concurrency limit. Usually the ``RUNNING`` requests make progress and finally your requests will be processed, but if the ``RUNNING`` requests are stuck, you can inspect the request log with:
@@ -125,7 +127,7 @@ There should be some ``RUNNING`` requests that occupy the concurrency limit. Usu
125
127
# Replace <request_id> with the actual request id from the ID column
126
128
$ sky api logs <request_id>
127
129
128
-
Cancel a request
130
+
Canceling a request
129
131
^^^^^^^^^^^^^^^^
130
132
131
133
If the request is stuck according to the log, e.g. retrying to launch VMs that is out of stock, you can cancel the request with:
@@ -137,7 +139,7 @@ If the request is stuck according to the log, e.g. retrying to launch VMs that i
137
139
Avoid concurrent logs requests
138
140
------------------------------
139
141
140
-
If you run ``sky logs`` to tail the logs of a task, the log tailing will keep taking off the resources of the API server as long as the task being tailed is running. So concurrent log requests will occupy the concurrency and make other requests to be delayed.
142
+
If you run ``sky logs`` to tail the logs of a task, the log tailing will keep taking resources off the API server as long as the task being tailed is still running. Thus, concurrent log requests will occupy the concurrency limit and potentially delay other requests.
141
143
142
144
To avoid this, it is recommended to run ``sky logs`` and ``sky jobs logs`` with ``--no-follow`` flag if there is a large number of concurrent log requests:
0 commit comments