Add Capacity to Juice Pools

Scaling GPU Capacity with Juice Pools and GPU Trader

Overview

Announcing the Juice Managed Template on GPU Trader. Now users of Juice can augment their pooled GPU resources with on-demand capacity, scaling up when needed. Juice Labs provides software “Juice” that enables GPU-over-IP, allowing your GPUs to become remote virtual resources: pooled, shared, and accessed by AI and graphics workloads. This allows you to share and better utilize your existing GPUs in your organization. But what if you need additional capacity beyond your fleet? That is where GPU Trader can help. The Juice Managed Template allows users to quickly rent and integrate on-demand capacity from GPU Trader to new or existing GPU Pools on Juice.

In this blog, you will learn how to rent an on-demand instance from GPU Trader, install the Juice Agent using our managed template, add the capacity to your Juice GPU pool, and access the capacity from your workstation. If you are new to Juice, their documentation provides detailed instructions on configuration.

These are the major steps we will cover:

1. Things you’ll need… to be successful.
2. Uh oh! I’m out of GPUs.
3. Yes! Extra capacity.
4. Adding a GPU to your Juice Pool.
5. Let’s Go!

Things you’ll need… to be successful

Here are some prerequsites for completing this tutorial:

1. Juice system requirements: You will need to use Windows 10/11 or Debian-based distributions of Linux on version 11 and greater, such as Ubuntu 20.04 and greater.
2. Install the Juice CLI App. CLI App Installation documentation.
3. Log in to the Juice CLI App. CLI App documentation.
4. A credit card for renting a GPU.

Let’s get started!

Uh oh! I’m out of GPUs

Log in to the Juice console at https://app.juicelabs.co/ or use your terminal to run the juice gpu list command to see how many GPUs are in your pool. Currently, we have two.

For this example, we’ll run a simple PyTorch Python script showing the number of available GPUs.

# pip install torch

import torch
print("PyTorch version:", torch.__version__)

# List all available GPUs
if torch.cuda.is_available():
    num_gpus = torch.cuda.device_count()
    print(f"Number of GPUs available: {num_gpus}")
    for i in range(num_gpus):
        print(f"GPU {i}: {torch.cuda.get_device_name(i)}")
else:
    print("No GPUs available")


# Test if PyTorch can access each GPU
if torch.cuda.is_available():
    num_gpus = torch.cuda.device_count()
    for i in range(num_gpus):
        try:
            # Perform a simple computation on the GPU
            device = torch.device(f'cuda:{i}')
            a = torch.tensor([1.0, 2.0], device=device)
            b = torch.tensor([1.0, 1.0], device=device)
            c = a + b
            print(f"PyTorch successfully accessed GPU {i}")
        except RuntimeError as e:
            print(f"Error during GPU {i} access:", e)
else:
    print("No GPUs available to test PyTorch access")

To run this script and access the GPUs in our Juice Pool, we wrap the script with the Juice CLI, passing in the option to request the number of GPUs we wish to use. Let’s use four GPUs. Hint: It won’t work.

(.venv) ubuntu@ip-172-31-42-91:~/pytorch$ juice run --num-gpus 4 python3 pytorch_gpu_test.py
Juice GPU, v10.1.0-44

Because there are currently only 2 GPUs in our pool, we get an error saying we do not have enough capacity.

2025-03-24T21:05:33.895Z session/request.go:385 I] Waiting for session request 478be361-af81-4053-afc4-faa34262ecb8
Error: No available GPUs were found.

So, what do we do if we want to utilize more GPUs than exist in our pool because our workload requires it, or we want to accelerate it? GPU Trader is an AI platform that allows users to find additional capacity and use it on demand. To find a GPU, log in to the GPU Trader Console at https://console.gputrader.io.

Yes! We found extra capacity

Log in and rent an on-demand GPU instance from GPU trader to find additional capacity and add it to our Juice Pool.

To find an instance:

1. Navigate to https://console.gputrader.io
2. Click “Sign in” at the bottom left of the console to create a GPU Trader account or log in if you already have one.
3. Browse the inventory, using the “Filters” to find the GPU that fits your requirements. Hint: Available inventory is highlighted in “Green,” and inventory that has been rented is highlighted in “Gray.”

4. Click the “Price/hr” button to rent the instance. Users can choose the number of cards with which they want to configure the instance. After reviewing the configuration, click “Continue” to enter payment information. If you have rented before, your payment information is saved, and you will proceed directly to the instance configuration step.

Adding a GPU to your Juice Pool

If you made it this far, you have successfully rented a GPU! Now we can configure the instance with the Juice Agent and add it to your Juice Pool.

1. Click “Add Stack” on your instance configuration page.
2. Choose the “Juice Labs Agent” managed template.
3. Now you must configure the Juice Agent to communicate with your Juice Pool. We are using Windows and will use the juice command in the terminal. If you are using Linux, use ./juice.

In the template’s YAML, you will notice three environment variables; only two need to be added: JUICE_TOKEN and JUICE_POOL. The GPU Trader platform will automatically add the JUICE_HOST_IP using ${EXTERNAL_IP}.

services:
  juice-agent:
    image: juicelabs/agent:latest
    # container_name: juice-agent
    environment:
      - JUICE_TOKEN=${JUICE_TOKEN}
      - JUICE_POOL=${JUICE_POOL}
      - JUICE_HOST_IP=${EXTERNAL_IP}
    ports:
      - 7865:7865/udp
    restart: unless-stopped

For the next two steps, add the Key and Value, and then click “Add” to add the variable to your template.

3.1 JUICE_TOKEN: Generate the Machine-to-Machine token from your terminal using the following command:

juice m2m create -d '<token name>'

Refer to the Juice M2M Tokens documentation for help. Example output: image Copy the token output into the environment variable “Value.” 3.2 JUICE_POOL: In your terminal, run the command juice pool ls to list your pool IDs. Copy the ID of the pool you wish to use and paste it into the environment variable “Value.” Example output: image 3.3 If you did the last two steps correctly, your template should look like this example: image

4. If your template looks like the example above with the outputs you generated for the JUICE_TOKEN and JUICE_POOL, click “Use Template” to instantiate the stack on your instance. Once it is complete, it should look like this:

You did it! You’ve added eight more GPUs to your pool.

Let’s Go!

To verify the GPUs are in your pool, you can log in to the Juice console or use the terminal to run the juice gpu list command. You can see that we now have eight GPUs available in our pool.

Now, when we request 4 GPUs in our client program, we can successfully access them from the pool, and the program enumerates the GPUs:

(.venv) ubuntu@ip-172-31-42-91:~/pytorch$ juice run --num-gpus 4 python3 pytorch_gpu_test.py
Juice GPU, v10.1.0-44

2025-03-24T21:05:42.856Z session/request.go:385 I] Waiting for session request 5fa67c6c-aa68-4292-8c3d-a95eea7c4ad8
2025-03-24T21:05:43.981Z session/request.go:448 I] Session 8fc58b6d-2a6c-44e4-aaae-37d8a2a4a61a ready
2025/03/24 21:05:44 Waiting for /home/ubuntu/.config/Juice GPU/logs/client-8fc58b6d-2a6c-44e4-aaae-37d8a2a4a61a.log to appear...
PyTorch version: 2.6.0+cu124
Number of GPUs available: 4
GPU 0: Juice GPU [NVIDIA A100-SXM4-80GB]
GPU 1: Juice GPU [NVIDIA A100-SXM4-80GB]
GPU 2: Juice GPU [NVIDIA A100-SXM4-80GB]
GPU 3: Juice GPU [NVIDIA A100-SXM4-80GB]
PyTorch successfully accessed GPU 0
PyTorch successfully accessed GPU 1
PyTorch successfully accessed GPU 2
PyTorch successfully accessed GPU 3

When you are satisfied with the output, you can keep the GPU in our pool for as long as needed or release the rented instance until it is needed again.

To release the instance, return to the instance configuration page, click the “Kebab” menu, and select “End rental.”

That’s it!