# WebSocket Tokens

The Kubernetes API proxy module supports custom WebSocket protocols
to better support the GitLab web frontend.

Examples of custom WebSocket protocols are:

- [Watch Aggregator API](/doc/watch-aggregator-api.md)
- [Graph API](/doc/graph_api.md)

## Problem

The web frontend exclusively uses browser cookies to authenticate
with the Kubernetes API proxy.
This process is defined in detail in
[Kubernetes User Access docs](/doc/kubernetes_user_access.md).

To establish WebSocket connections the browser provides the
[WebSocket API](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API).

We are running into a limitation with this WebSocket API because it does
not allow to:

- send cookies to subdomains (cross-origin), even when the cookies `domain` is set correctly
  (otherwise works with the [Fetch API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API))
- send authorization headers nor arbitrary headers

The Kubernetes Agent Server (KAS) is often hosted on a subdomain relative to
the web frontend. For example on GitLab.com, KAS is at `kas.gitlab.com`.

The only possibility we have to send custom authentication data
is the WebSocket URI and subprotocol. The WebSocket URI is considered insecure
because it is often logged transparently by proxies.

## Solution

The solution here is to provide a dedicated non-WebSocket endpoint in KAS which when
provided with a valid session cookie, CSRF token and agent ID will return a JWT
that encodes an authorized request.

The JWT can be used in a subsequent WebSocket API request as a specially
encoded WebSocket subprotocol in the `Sec-Websocket-Protocol` header.
When the JWT is valid, the request is granted.

We have to use the WebSocket subprotocol for the token because:

- we can't send any headers with the WebSocket API.
- we don't want to sent the token in the URI because it's usually logged by proxies.
- it's what is common to do, for example, Kubernetes does it.

The subprotocol must be in the following format:

```plaintext
websocket-token.kas.gitlab.com.<token>
```

The following diagram depicts such a WebSocket token request flow:

```mermaid
sequenceDiagram
  actor f as Frontend
  participant r as GitLab Rails
  participant k as KAS

  f->>+r: Environment Page
  r->>-f: Cookie, CSRF Token

  f->>+k: GET /k8s-proxy/watch?kas-websocket-token-request
  Note over f,k: providing the cookie and CSRF token
  k->>-f: short-lived JWT using KAS<->KAS key
  f->>+k: WebSocket Upgrade Request, GET /k8s-proxy/watch, with header Sec-Websocket-Protocol: websocket-token.kas.gitlab.com.<short-lived JWT>
  k->>-f: Upgrade
```

In JavaScript this may look like this:

```javascript
// retrieve token via cookie based authentication
const response = await fetch(
  "https://gdk.test:8080/-/k8s-proxy/watch?kas-websocket-token-request",
  headers: {
    "Gitlab-Agent-Id": 1,
    "X-Csrf-Token": document.head.querySelector('meta[name="csrf-token"]').content,
  }
)
const token = (await response.json()).token

// establish websocket connection with the retrieved token
ws = new WebSocket(
  "wss://gdk.test:8080/-/k8s-proxy/watch",
  ["gitlab-agent-watch-api", `websocket-token.kas.gitlab.com.${token}`]
)
```

### Implementation Details

#### Token Request

- Must send the `kas-websocket-token-request` URI query parameter. It doesn't require a value.
- Must use the `GET` method.
- Must send [session cookie authentication](/doc/kubernetes_user_access.md) parameters.

#### Token Request Response

When requesting the token with the `kas-websocket-token-request` query parameter a
success response will be a JSON object with a single `token` field containing the JWT.

```json
{
  "token": "..."
}
```

#### JWT

The JWT must use the `HS3-512` (**H**MAC **S**HA**3** **512**) signing method.

The JWT has the following registered claims:

- Issuer: `gitlab-kas`
- Audience: `gitlab-kas`
- Subject: the user ID that requested the token
- ExpiresAt: `<now>+5s`
- NotBefore: `<now>-5s`
- IssuedAt:  `<now>`

The JWT must contain claims for:

- Endpoint (`endpoint`): the URL (`<host>/<path>`) of the request that was used to request the token. For example `gitlab.example.com/-/k8s-proxy/watch`
- Agent ID (`agent_id`)
- Impersonation Config (`impersonation_config`)

An example JWT content may look like this:

```json
// header
{
  "alg": "HS3-512",
  "typ": "JWT"
}

// payload
{
  "iss": "gitlab-kas",
  "sub": "user-id:42",
  "aud": [
    "gitlab-kas"
  ],
  "exp": 1731062878,
  "nbf": 1731062868,
  "iat": 1731062873,
  "endpoint": "gdk.test:8080/-/k8s-proxy/watch",
  "agent_id": 1,
  "impersonation_config": {
    "username": "gitlab:user:root",
    "groups": [
      "gitlab:user",
      "gitlab:project_role:20:guest",
      "gitlab:project_role:20:reporter",
      "gitlab:project_role:20:developer",
      "gitlab:project_role:20:maintainer",
      "gitlab:project_role:20:owner"
    ],
    "extra": [
      {
        "key": "agent.gitlab.com/id",
        "val": [
          "1"
        ]
      },
      {
        "key": "agent.gitlab.com/username",
        "val": [
          "root"
        ]
      },
      {
        "key": "agent.gitlab.com/access_type",
        "val": [
          "session_cookie"
        ]
      },
      {
        "key": "agent.gitlab.com/config_project_id",
        "val": [
          "20"
        ]
      }
    ]
  }
}
```
