Guide to use dockers from Replicate built with Cog

In this documentation, we will look at how to use an AI application as an API using dockers built with Cog.

Table of Contents

• Introduction to Cog
• How to use Cog in jarvislabs ?
• Where to find Docker image name ?
• Access the API
• Test your Cog fastAPI server

Introduction to Cog

Packaging and deploying machine learning model in a container can be hard if you are not comfortable with dockers. Cog, a open-source tool from replicate helps in building dockers, it makes deploying AI Models easier. You can represent all the required system libraries, python libraries and your prediction file in a single Yaml file.

Over the last few years the AI community has deployed a lot of AI models as APIs on Replicate. You can pick up any such model and run them on Jarvislabs.ai.

How to use Cog in Jarvislabs ?

Follow below steps to get to bring your favourite AI application as an API on Jarvislabs.ai.

Choose the Cog framework

The Frameworks page has a bunch of different options. Pick the one called "Cog".

Set the GPU type and other configuration

Once you choose the framework, you will be redirected to the launch instance page where you can choose any GPU and set the storage size required for your docker container to run.

Note Since this is a read only dockers, we will not be able to pause/resume the instances.

Enter the Docker image name

Pick any publicly available model from Replicate and enter the docker image name in the "Docker Image" field. Hit launch instance, wait for few minutes while we build the docker and bring it up for you.

Note Depending on the docker image, the size of the docker and model weights would vary. So the time taken to bring up the docker would vary.

Make sure to enter the correct docker image name.

Where to find Docker image name ?

From the replicate explore page, choose your required model

Click the Cog and you will something like below. Just copy the docker image reference after word "predict".

Note Some images do not have cog tab, you will not be able to run them on Jarvislabs.ai.

Breaking down the docker image name

r8.im/tencentarc/photomaker
@sha256:ddfc2b08d209f9fa8c1eca692712918bd449f695dabb4a958da31802a9570fe4

r8.im indicates that you are using a replicate model.

tencentarc is the username and phototmaker is the name of the model. After this "@" is encrypted id.

Access the API

Once the docker is up and running, you will be redirected to the instance page. Click the API button to access the Cog FastAPI server. You can explore, test and integrate the API with your application.

Test your Cog fastAPI server

Testing the model is very simple. You can test the model by sending a request to the API.

Example

Here I picked stability-ai/sdxl as a replicate model but you can choose anything. Once you click the API button you will see the swagger UI, which is powered by FastAPI.

Click the Predictions function to send a request and click the try it out button to edit the Request Body. Make sure to enter all the details correctly to get the output. Here I'm going to fill the prompt and image size (width & height). That's all I'm going to do.

Prompt : A red car driving in a highway
Width : 1024
height : 1024

Hopefully you will get a reponse with the output(A image data or URL).