# dat1.co

> Dat1 is a serverless GPU platform for running custom generative AI models at scale, offering fast, cost-efficient inference, privacy compliance, and zero hardware management.

Dat1 provides a machine learning model hosting solution that eliminates the complexity and expense of managing hardware for large AI models. The platform automatically handles scaling, reduces operational and hardware costs by efficiently sharing GPUs, and charges only for the time your model is actively running inference—no costs for idle time or timeouts.

**Key Features:**

* **Serverless GPU Inference:** Deploy custom AI models without managing hardware or scaling logistics.
* **Pay-per-second Pricing:** Only pay for the seconds your model is processing requests, with no charges for idle time or timeouts.
* **Low Cold Start Times:** Achieves 15–20 second cold starts for 70GB models, with optimization support for further improvement.
* **Privacy and Compliance:** Fully GDPR and CCPA-compliant, following top cybersecurity best practices.
* **Part of Nvidia Inception:** Recognized as part of Nvidia’s Inception program for AI startups.

**How It Works:**

* Upload your model weights and Python code as a deployment package using the `dat1-cli`.
* Dat1 manages all hardware logistics and distributes your model across its infrastructure.
* Scaling is handled automatically based on demand.
* You are only billed for active inference time.


## CLI and Deployment

The `dat1-cli` is a command-line interface for interacting with the Dat1 platform.

### Installation

To install the CLI, run the following command:

```bash
pip install dat1-cli
```


### CLI Usage

* **Login:** Initialize with your API key.

```bash
dat1 login
```

* **Initialize Project:** Creates a `dat1.yaml` configuration file in your project's root directory.

```bash
dat1 init
```

* **Deploy Model:** Uploads your model to the platform.

```bash
dat1 deploy
```

* **Serve Locally:** Launch your model in a local environment for testing. Requires Docker, a CUDA-compatible GPU, and the NVIDIA Container Toolkit.

```bash
dat1 serve
```


### Configuration (`dat1.yaml`)

The `dat1.yaml` file configures your model for deployment. It specifies the model name and files to exclude from the upload using glob patterns.

```yaml
model_name: <your model name>
exclude:
  - '**/.git/**'
  - '**/.idea/**'
  - '*.md'
  - '*.jpg'
  - .dat1.yaml
  - .DS_Store
```


### Code Examples

The platform expects a `handler.py` file in your project's root that contains a FastAPI app.

#### **Basic Handler**

This example shows a handler with a health check (`/`) and an inference endpoint (`/infer`).

```python
from fastapi import Request, FastAPI
from vllm import LLM, SamplingParams
import os

# Model initialization Code
# This code should be placed before the FastAPI app is initialized
llm = LLM(model=os.path.expanduser('./'), load_format="safetensors", enforce_eager=True)

app = FastAPI()

@app.get("/")
async def root():
    return "OK"

@app.post("/infer")
async def infer(request: Request):
    # Inference Code
    request = await request.json()
    prompts = request["prompt"]
    sampling_params = SamplingParams(temperature=0.8, top_p=0.95)
    outputs = llm.generate(prompts, sampling_params)
    return {
        "response" : outputs[0].outputs[0].text
    }
```


#### **Streaming Responses with Server-Sent Events (SSE)**

To stream responses, first update your `dat1.yaml` to specify the response type:

```yaml
model_name: chat_completion
response_type: sse
exclude:
  - '**/.git/**'
  - '**/.idea/**'
```

Then, modify your `handler.py` to return a generator with `EventSourceResponse`:

```python
from fastapi import Request, FastAPI
from sse_starlette.sse import EventSourceResponse
import json

app = FastAPI()

@app.get("/")
async def root():
    return "OK"

async def response_generator():
    for i in range(10):
        yield json.dumps({"response": f"Response {i}"})

@app.post("/infer")
async def infer(request: Request):
    return EventSourceResponse(response_generator(), sep="\n")
```


## Docs

* [Homepage](https://dat1.co): Main website with product overview, sign-up, and contact information.
* [GitHub Repository](https://github.com/dat1-co/dat1-cli): Official command-line interface for deploying and managing models.
* [Privacy Policy](https://dat1.co/privacy-policy): Details on data handling, GDPR/CCPA compliance, and user rights.
* [Pricing](https://dat1.co/pricing): Explanation of pay-per-second billing and example costs.


## Optional

* [Crunchbase](https://www.crunchbase.com/organization/dat1): Crunchbase
* [LinkedIn](https://www.linkedin.com/company/dat1): LinkedIn