Running a Torch Model on Infernet

In this tutorial, we will run a Pytorch model on Infernet, using our infernet-container-starter (opens in a new tab) examples repository. If you already followed our ONNX example, you will notice it uses the same ML model as well as contracts.

Model: This example uses a pre-trained model to classify iris flowers. The code for the model is located at the simple-ml-models (opens in a new tab) repository.

Hardware Requirements

Any laptop or desktop computer should be able to run this tutorial.

Tutorial Video

Install Pre-requisites

For this tutorial you'll need to have the following installed.

Web2 Off-Chain Compute

Ensure `docker` & `foundry` exist

To check for docker, run the following command in your terminal:

docker --version
# Docker version 25.0.2, build 29cf629 (example output)

You'll also need to ensure that docker-compose exists in your terminal:

which docker-compose
# /usr/local/bin/docker-compose (example output)

To check for foundry, run the following command in your terminal:

forge --version
# forge 0.2.0 (551bcb5 2024-02-28T07:40:42.782478000Z) (example output)

Clone the starter repository

Much like our Running an ONNX Model example, we're going to clone this repository. All of the code and instructions for this tutorial can be found in the projects/torch-iris (opens in a new tab) directory of the repository.

# Clone locally
git clone --recurse-submodules https://github.com/ritual-net/infernet-container-starter
# Navigate to the repository
cd infernet-container-starter

Build the `torch-iris` container

Simply run the following command to build the torch-iris container:

make build-container project=torch-iris

Deploy the `torch-iris` container with Infernet

You can run a simple command to deploy the torch-iris container along with bootstrapping the rest of the Infernet node stack in one go:

make deploy-container project=torch-iris

Check the running containers

At this point it makes sense to check the running containers to ensure everything is running as expected.

docker container ps

You should expect to see something like this:

CONTAINER ID   IMAGE                                             COMMAND                  CREATED         STATUS          PORTS                   NAMES
0dbc30f67e1e   ritualnetwork/example-torch-iris-infernet:latest  "hypercorn app:creat…"   8 seconds ago   Up 7 seconds   0.0.0.0:3000->3000/tcp   torch-iris
47758185b1cc   ritualnetwork/infernet-node:1.3.1                 "/app/entrypoint.sh"     8 seconds ago   Up 7 seconds   0.0.0.0:4000->4000/tcp   infernet-node
49cc4b28f8d1   redis:7.4.0                                       "docker-entrypoint.s…"   8 seconds ago   Up 7 seconds   0.0.0.0:6379->6379/tcp   infernet-redis
16e96b377a15   fluent/fluent-bit:3.1.4                           "/fluent-bit/bin/flu…"   8 seconds ago   Up 7 seconds   2020/tcp, 24224/tcp      infernet-fluentbit
0b2c077302f5   ritualnetwork/infernet-anvil:1.0.0                "anvil --host 0.0.0.…"   8 seconds ago   Up 7 seconds   0.0.0.0:8545->3000/tcp   infernet-anvil

Notice that five different containers are running, including the infernet-node and the torch-iris containers.

Create an off-chain compute job

With the container running, we can make a POST request to the Infernet node to create a job.

curl -X POST "http://127.0.0.1:4000/api/jobs" \
-H "Content-Type: application/json" \
-d '{"containers": ["torch-iris"], "data": {"input": [[1.0380048, 0.5586108, 1.1037828, 1.712096]]}}'

You should get an output similar to the following:

{
    "id": "074b9e98-f1f6-463c-b185-651878f3b4f6"
}

ℹ️

The inputs provided above correspond to an iris flower with the following characteristics:

Sepal Length: 5.5cm
Sepal Width: 2.4cm
Petal Length: 3.8cm
Petal Width: 1.1cm

By vectorizing & standardizing these inputs, we get the following vector:

[1.0380048, 0.5586108, 1.1037828, 1.712096]

Refer to this function in the model's repository (opens in a new tab) for more information on how the input is scaled.

For more context on the Iris dataset, refer to the UCI Machine Learning Repository (opens in a new tab).

Collect job status

To check for the current status of the job (including any results or errors), we can poll the /api/jobs (opens in a new tab) endpoint with the ID of our job from above:

curl -X GET "http://127.0.0.1:4000/api/jobs?id=074b9e98-f1f6-463c-b185-651878f3b4f6"

You will get a response similar to this:

[
    {
        "id": "6d5e47f0-5907-4ab2-9523-862dccb80d67",
        "result": {
            "container": "torch-iris",
            "output": {
                "output": [
                    0.0016699483385309577,
                    0.021144982427358627,
                    0.977185070514679
                ]
            }
        },
        "status": "success"
    }
]

Congratulations! You have successfully setup the Infernet Node and the torch-iris container. Now let's move on to calling our service from a smart contract (a la web3 request).

Additional Resources

To look at the node configuration, code, and other resources for the torch-iris container, reference the Infernet Container Starter (opens in a new tab) repository.
To build an Infernet-compatible container image, reference the Container documentation (opens in a new tab).

Web3 On-Chain Subscriptions

To test the Web3 workflow of the Infernet Node, we will need to scaffold some additional components. Below, we:

Deploy an Anvil (opens in a new tab) local testnet node with the Infernet SDK (opens in a new tab) contracts already setup (via our infernet-anvil (opens in a new tab) image).
Deploy a simple Infernet Consumer contract (opens in a new tab) for our Web3 demo application.
Deploy an Infernet Node ready to service our requests.

Then, once this preliminary infrastructure is setup, we:

Make a subscription request to our newly-deployed consumer contract, creating an on-chain subscription.
Monitor the full lifecycle of the subscription, from the:

node listening to the subscription event,
to the node processing the subscription,
to the node sending the result back to the our consumer contract on-chain.

Setup

Follow steps 1 through 5 above to build and deploy the torch-iris container.

Inspect the Anvil node

Notice from Check the running containers that you have an Anvil node running on port 8545 and an Infernet Node on port 4000.

By default, the infernet-anvil (opens in a new tab) image used deploys the Infernet SDK (opens in a new tab) and other relevant contracts for you:

Coordinator: 0x5FbDB2315678afecb367f032d93F642f64180aa3
Primary node: 0x70997970C51812dc3A010C7d01b50e0d17dc79C8

Deploy a consumer

Next, we must deploy a consumer contract that implements the simple Infernet SDK interface (opens in a new tab).

Our IrisClassifier (opens in a new tab) contract is a simple, example consumer. All this contract does is request a compute output from our infernet node, to classify an iris flower based on its sepal and petal dimensions, and upon receiving the result, use the forge console to print the result. We can deploy this contract via the associated forge project (opens in a new tab).

We are going to make the same request as above in our web-2 example, but this time using a smart contract (opens in a new tab). Since floats are not supported in Solidity, we convert all floats to uint256 by multiplying the input vector entries by 1e6:

uint256[] memory iris_data = new uint256[](4);
iris_data[0] = 1_038_004;
iris_data[1] = 558_610;
iris_data[2] = 1_103_782;
iris_data[3] = 1_712_096;

Anvil logs

During this process, it is useful to look at the logs of the Anvil node to see what's going on. To follow the logs, in a new terminal, run:

docker logs -f infernet-anvil

Deploying the contract

Once ready, to deploy the IrisClassifier consumer contract, in another terminal, run:

make deploy-contracts project=torch-iris

You should expect to see similar Anvil logs:

eth_sendRawTransaction
eth_getTransactionReceipt

Transaction: 0x23ca6b1d1823ad5af175c207c2505112f60038fc000e1e22509816fa29a3afd6
Contract created: 0x663f3ad617193148711d28f5334ee4ed07016602
Gas used: 476669

Block Number: 1
Block Hash: 0x6b026b70fbe97b4a733d4812ccd6e8e25899a1f6c622430c3fb07a2e5c5c96b7
Block Time: "Wed, 17 Jan 2024 22:17:31 +0000"

eth_getTransactionByHash
eth_getTransactionReceipt
eth_blockNumber

From our logs, we can see that the IrisClassifier contract has been deployed to address 0x663f3ad617193148711d28f5334ee4ed07016602.

Call the contract

Now, let's call the contract initiating a request to the Infernet Node. In the same terminal, run:

make call-contract project=torch-iris

You should first expect to see an initiation transaction sent to the IrisClassifier contract:

eth_getTransactionReceipt

Transaction: 0xe56b5b6ac713a978a1631a44d6a0c9eb6941dce929e1b66b4a2f7a61b0349d65
Gas used: 123323

Block Number: 2
Block Hash: 0x3d6678424adcdecfa0a8edd51e014290e5f54ee4707d4779e710a2a4d9867c08
Block Time: "Wed, 17 Jan 2024 22:18:39 +0000"
eth_getTransactionByHash

Shortly after that you should see another transaction submitted from the Infernet Node which is the result of your on-chain subscription and its associated job request:

eth_sendRawTransaction


_____  _____ _______ _    _         _
|  __ \|_   _|__   __| |  | |  /\   | |
| |__) | | |    | |  | |  | | /  \  | |
|  _  /  | |    | |  | |  | |/ /\ \ | |
| | \ \ _| |_   | |  | |__| / ____ \| |____
|_|  \_\_____|  |_|   \____/_/    \_\______|


predictions: (adjusted by 6 decimals, 1_000_000 = 100%, 1_000 = 0.1%)
Setosa:  1015
Versicolor:  14391
Virginica:  984593

Transaction: 0x77c7ff26ed20ffb1a32baf467a3cead6ed81fe5ae7d2e419491ca92b4ac826f0
Gas used: 111091

Block Number: 3
Block Hash: 0x78f98f4d54ebdca2a8aa46c3b9b7e7ae36348373dbeb83c91a4600dd6aba2c55
Block Time: "Mon, 19 Feb 2024 20:33:00 +0000"

eth_blockNumber
eth_newFilter
eth_getFilterLogs

🎉 Congratulations! You have successfully created an on-chain subscription request!

Next steps

This container is for demonstration purposes only, and is purposefully simplified for readability and ease of comprehension. For a production-ready version of this code, check out:

The Torch Inference Workflow (opens in a new tab): A Python class that can run any Torch model from a variety of storage sources.
The Torch Inference Service (opens in a new tab): A production-ready, Infernet (opens in a new tab)-compatible container that works out-of-the-box with minimal configuration, and serves Torch inference using the Torch Inference Workflow.

Running an ONNX Model TGI Inference with Mistral-7B