<p>This tip covers the long jobs that can block the next request to your server.</p><p>By the end, you can point at one slow job and name what it is bound by. You learn that name first, so you can see what is blocking. The fix comes in the next tip.</p><p>Keep one picture in mind. A customer starts a long job. A second customer sends a request before that job is done. If the server cannot take that request while the first job is still running, the second customer waits. A job is bound by the one thing that sets how long it takes, and that thing is different each time. So we will walk through four jobs.</p><p>The first job is bound by a reply from outside the server. That is called I/O bound. I/O means input and output. A customer asks your app a question. Your server sends it to a language model through the OpenAI API, then waits for the answer. The processor is free the whole time. The graphics card writing the answer sits in OpenAI's data center, not in your server. If your server cannot take another request while it waits, the next customer is waiting on a company that is not even yours.</p><p>The second job is bound by the processor. That is called CPU bound. A customer asks for a year of invoices as one document, and your server lays out every page itself. Nothing outside is slow. The processor is busy from start to end. If the server has no processor time left for the next customer, that customer waits for a turn.</p><p>The third job is bound by the graphics card. That is called GPU bound. A customer asks your own server to draw a picture from a sentence, and the card inside that server does the drawing. If the server stays inside that drawing and cannot take the next request, that request waits. It can be another picture, or a normal call that never needs the card. The processor can still have spare time. If your server only asks another company to draw it, you are back to the OpenAI call. That is I/O bound, because the card is not yours.</p><p>The fourth job is bound by the memory. That is called memory bound. A customer uploads a giant spreadsheet, and the server holds the whole file in memory before it reads a single row. Other requests can slow down, or fail, because there is no room left for them. The processor still has spare time.</p><p>The wrong turn is to call all four jobs slow and only watch the processor. During the OpenAI call, and during that upload, the processor has room to spare. So the processor graph looks fine while customers wait. The fix you pick depends on what the job is bound by.</p><p>Leave with this check. When a long job blocks the next request, name what it is bound by: a reply from outside, the processor, the graphics card, or the memory.</p><p>Now take the quiz on this tip. Each question gives you one slow job, and you name which of the four it is. Then pick one slow request from a server you know, and give it that same label.</p><p>In the next tip we put that label to work. We will talk through asynchronous work, threads, multiprocessing, and background jobs, and how each one lets the second request run without waiting for the first. I will meet you there.</p>

The four long jobs that can block the next request: I/O, CPU, GPU, and memory bound. Name what a slow job is bound by before you pick a fix.

I/O, CPU, GPU, or memory bound? | Tech Tips by Moh

The graphics card, because a picture is being drawn

A reply from outside, because the card is not yours

The processor, because the server is waiting

Nothing. The processor graph shows the server is fine

I/O, CPU, GPU, or memory bound?

<div class="course-slide cs-center">
  <p class="cs-kicker">Long jobs</p>
  <h2 class="cs-title cs-at-2-2">They can <span class="cs-hl">block</span> the next request</h2>
  <p class="cs-pill cs-at-8">Name what it is bound by</p>
  <p class="cs-pill cs-at-12-1">The fix comes in the next tip</p>
</div>

<div class="course-slide">
  <p class="cs-kicker">One picture</p>
  <h2 class="cs-title">Two customers, one server</h2>
  <div class="cs-grid cs-cols-2">
    <div class="cs-card cs-at-1-6"><p class="cs-card-label">First customer</p><p class="cs-card-title">Starts a long job</p></div>
    <div class="cs-card cs-at-4"><p class="cs-card-label">Second customer</p><p class="cs-card-title">Sends a request before it is done</p></div>
  </div>
  <p class="cs-text cs-at-7-9">The server <span class="cs-bad">cannot take it</span> while the first job is still running</p>
  <p class="cs-pill cs-at-11-1">The second customer waits</p>
</div>

<div class="course-slide cs-center">
  <p class="cs-kicker">The name</p>
  <h2 class="cs-title cs-at-1-3"><span class="cs-hl">Bound by</span> the one thing that sets how long it takes</h2>
  <p class="cs-pill cs-at-4-9">Different for each job</p>
  <div class="cs-row cs-at-7-3">
    <span class="cs-chip">Reply from outside</span>
    <span class="cs-chip">Processor</span>
    <span class="cs-chip">Graphics card</span>
    <span class="cs-chip">Memory</span>
  </div>
</div>

<div class="course-slide cs-center">
  <p class="cs-kicker">Job 1</p>
  <h2 class="cs-title cs-at-0-8">Bound by a <span class="cs-hl">reply from outside</span></h2>
  <p class="cs-pill cs-at-4">I/O bound</p>
  <p class="cs-pill cs-at-6">I/O means input and output</p>
</div>

<div class="course-slide">
  <p class="cs-kicker">I/O bound · example</p>
  <div class="cs-row">
    <span class="cs-chip">Your server</span>
    <span class="cs-arrow cs-at-2-1">→</span>
    <span class="cs-chip cs-at-2-1">OpenAI API</span>
    <span class="cs-arrow cs-at-4">→</span>
    <span class="cs-chip cs-at-4">Waits for the answer</span>
  </div>
  <div class="cs-grid cs-cols-3">
    <div class="cs-card cs-at-5-6"><p class="cs-card-label">Processor</p><p class="cs-card-title">Free the whole time</p></div>
    <div class="cs-card cs-at-8"><p class="cs-card-label">Graphics card</p><p class="cs-card-title">In OpenAI's data center, not your server</p></div>
    <div class="cs-card cs-card-accent cs-at-12-9"><p class="cs-card-label">Next customer</p><p class="cs-card-title">Waits on a company that is not yours</p></div>
  </div>
</div>

<div class="course-slide cs-center">
  <p class="cs-kicker">Job 2</p>
  <h2 class="cs-title cs-at-1-2">Bound by the <span class="cs-hl">processor</span></h2>
  <p class="cs-pill cs-at-3-4">CPU bound</p>
  <p class="cs-pill cs-at-6-4">A year of invoices, laid out by your server</p>
</div>

<div class="course-slide cs-center">
  <p class="cs-kicker">CPU bound</p>
  <h2 class="cs-title cs-at-0-7">Busy from <span class="cs-hl">start to end</span></h2>
  <div class="cs-grid cs-cols-2">
    <div class="cs-card cs-at-3-1"><p class="cs-card-label">Next customer</p><p class="cs-card-title">No processor time left</p></div>
    <div class="cs-card cs-card-danger cs-at-6-4"><p class="cs-card-label">So</p><p class="cs-card-title">Waits for a turn</p></div>
  </div>
</div>

<div class="course-slide cs-center">
  <p class="cs-kicker">Job 3</p>
  <h2 class="cs-title cs-at-0-8">Bound by the <span class="cs-hl">graphics card</span></h2>
  <p class="cs-pill cs-at-3">GPU bound</p>
  <p class="cs-pill cs-at-5-7">The card inside your own server draws the picture</p>
</div>

<div class="course-slide">
  <p class="cs-kicker">GPU bound</p>
  <h2 class="cs-title cs-at-0-5">The server stays inside the drawing</h2>
  <p class="cs-text cs-at-4-1">So the next request <span class="cs-bad">waits</span>. It can be:</p>
  <div class="cs-row">
    <span class="cs-chip cs-at-6">Another picture</span>
    <span class="cs-chip cs-at-7-2">A normal call that never needs the card</span>
  </div>
  <p class="cs-pill cs-at-9-1">The processor can still have spare time</p>
</div>

<div class="course-slide cs-center">
  <p class="cs-kicker">Not your card</p>
  <h2 class="cs-title cs-at-1-5">Another company draws it</h2>
  <div class="cs-grid cs-cols-2">
    <div class="cs-card cs-at-3-6"><p class="cs-card-label">Same wait as</p><p class="cs-card-title">The OpenAI call</p></div>
    <div class="cs-card cs-card-accent cs-at-5-9"><p class="cs-card-label">I/O bound</p><p class="cs-card-title">The card is not yours</p></div>
  </div>
</div>

<div class="course-slide cs-center">
  <p class="cs-kicker">Job 4</p>
  <h2 class="cs-title cs-at-0-8">Bound by the <span class="cs-hl">memory</span></h2>
  <p class="cs-pill cs-at-2-9 cs-until-7-6">Memory bound</p>
  <p class="cs-pill cs-at-7-6 cs-until-10-8">The whole file, before one row</p>
  <p class="cs-pill cs-at-10-8">No room left. Other requests slow down or fail.</p>
  <p class="cs-pill cs-at-15-3">The processor still has spare time</p>
</div>

<div class="course-slide">
  <p class="cs-kicker">Wrong turn</p>
  <h2 class="cs-title cs-at-1-5">Call all four <span class="cs-bad">slow</span></h2>
  <p class="cs-text cs-at-3-5">and only watch the processor</p>
  <div class="cs-grid cs-cols-2">
    <div class="cs-card cs-at-5-3"><p class="cs-card-label">OpenAI call and the upload</p><p class="cs-card-title">The processor has room to spare</p></div>
    <div class="cs-card cs-card-danger cs-at-10-6"><p class="cs-card-label">The processor graph</p><p class="cs-card-title">Looks fine while customers wait</p></div>
  </div>
  <p class="cs-pill cs-at-13-9">The fix depends on what the job is bound by</p>
</div>

<div class="course-slide">
  <p class="cs-kicker">Leave with this</p>
  <p class="cs-text cs-at-1-4">When a long job blocks the next request</p>
  <h2 class="cs-title cs-at-4-1">Name what it is <span class="cs-hl">bound by</span></h2>
  <div class="cs-grid cs-cols-4">
    <div class="cs-card cs-at-5-7"><p class="cs-card-label">I/O</p><p class="cs-card-title">A reply from outside</p></div>
    <div class="cs-card cs-at-7-8"><p class="cs-card-label">CPU</p><p class="cs-card-title">The processor</p></div>
    <div class="cs-card cs-at-8-5"><p class="cs-card-label">GPU</p><p class="cs-card-title">The graphics card</p></div>
    <div class="cs-card cs-at-9-6"><p class="cs-card-label">Memory</p><p class="cs-card-title">The memory</p></div>
  </div>
</div>

<div class="course-slide cs-center">
  <p class="cs-kicker">After this</p>
  <h2 class="cs-title cs-at-0-6">Take the quiz</h2>
  <p class="cs-pill cs-at-4-8">Name which of the four it is</p>
  <p class="cs-pill cs-at-6-9">Then label one slow request you know</p>
</div>

<div class="course-slide">
  <p class="cs-kicker">Next tip</p>
  <h2 class="cs-title cs-at-1-2">Put the label to work</h2>
  <div class="cs-row">
    <span class="cs-chip cs-at-3-6">Asynchronous work</span>
    <span class="cs-chip cs-at-5-3">Threads</span>
    <span class="cs-chip cs-at-5-9">Multiprocessing</span>
    <span class="cs-chip cs-at-7-2">Background jobs</span>
  </div>
  <p class="cs-pill cs-at-9">The second request runs without waiting for the first</p>
</div>