# JSON over HTTP

> How to read and write an HTML Clay document as JSON over HTTP, with rules in the URL or published by the file itself.

Source: https://htmlclay.com/docs/json-http/

Any `.html` or `.htmlclay` file HTML Clay serves can be read and written as JSON over HTTP, on the same local address your tab uses. The rules used to extract that JSON, and the page-side API that shares them, are on the ClayJS [data plugin](https://clayjs.com/docs/plugins/data/) page; this page covers the HTTP side.

## Save the example file

Save this as `todo.htmlclay` in your home folder, then open it with HTML Clay:

```html
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <title>Tasks</title>
</head>
<body>
  <h1>Today</h1>
  <ol>
    <li class="todo" data-done="no">Buy milk</li>
    <li class="todo" data-done="no">Book tickets</li>
  </ol>
  <script src="https://clayjs.com/v1/clay.js?plugins=sync"></script>
  <script src="https://clayjs.com/v1/clay-data.js"></script>
</body>
</html>
```

The ClayJS script enables live updates. Plain `.html` files work the same way.

## Read with rules in the request

Replace `htmlclay` with the hostname in your open tab's address. Your tab's address looks like `http://htmlclay.localhost:1212/todo.htmlclay`. The port is the same for every folder.

Run this in your terminal:

```bash
base='http://htmlclay.localhost:1212'
curl --get "$base/todo.htmlclay" \
  --data-urlencode 'data={"title":"h1","items":".todo[]"}'
```

Keep that terminal open for the following examples.

The `data` parameter carries the rules, which curl encodes for the URL. `title` reads the heading. The `[]` in `.todo[]` collects every matching task's text. The response is that data as JSON.

Ask only for the keys you need. With `data={"title":"h1"}`, the response contains only `title`:

```json
{"title":"Today"}
```

## Write with rules in the request

The next request changes the heading to "Tomorrow" and the first task to "Buy bread".

Send the rules in the URL and the new values as the body:

```bash
curl "$base/todo.htmlclay" \
  --url-query 'data={"title":"h1","items":".todo[]"}' \
  -H 'Content-Type: application/json' \
  --data '{"title":"Tomorrow","items":["Buy bread","Book tickets"]}'
```

The `--data` option makes this a POST, and `--url-query` encodes the rules into the URL. HTML Clay returns the updated values:

```json
{
  "title": "Tomorrow",
  "items": ["Buy bread", "Book tickets"]
}
```

Open `todo.htmlclay` in your text editor. The heading now contains `Tomorrow`, and the first task contains `Buy bread`. The open tab updates too. The rules you sent are not saved into the file.

A key you leave out is left alone. An array you send becomes the whole list: a task missing from it is removed, and a new task is a copy of the first task's element, with your text inside it. Keep at least one task in the list because once the list is empty, a write cannot add tasks back. New text replaces what was inside the selected element. Each write that changes the file is saved as a version.

Writes reach only the elements the rules select. These rules change text; attribute rules can also change allowed attributes. Writes cannot insert scripts, styles or event handlers. HTML you send is saved as plain text. A key the rules do not define causes a `400` response. Keep the JSON body under 1 MB.

## Write only if the file is unchanged

Change the second task only if the file still matches your read. First, read with response headers:

```bash
curl -i "$api"
```

The `Etag` header identifies the file you read. Both URLs return it, for reads and writes. Its value looks like this:

```
Etag: 8f02c16e7b4d9a30
```

Replace the sample value below with your exact `Etag`, then change the second task:

```bash
curl "$api" \
  -H 'Content-Type: application/json' \
  -H 'If-Match: 8f02c16e7b4d9a30' \
  --data '{"title":"Tomorrow","items":["Buy bread","Book train tickets"]}'
```

If the file still matches, the response shows the changed task:

```json
{
  "title": "Tomorrow",
  "items": ["Buy bread", "Book train tickets"]
}
```

If the file has changed, HTML Clay returns `412` and writes nothing. Read the file again before deciding what to send. Without `If-Match`, the write skips this check.

## Reading a file as JSON

Any `.html` or `.htmlclay` file HTML Clay will serve can also be read as JSON, using the same extraction rules as the Hyperclay platform. Two URL forms, with caller supplied rules taking precedence on both.

You supply the rules, and the document's own rules tag is ignored:

```bash
curl --get 'http://htmlclay.localhost:1212/notes.htmlclay' \
  --data-urlencode 'data={title:"h1",items:".todo[]"}'
```

Without `?data=`, the file supplies its own:

```bash
curl 'http://htmlclay.localhost:1212/_/api/notes.htmlclay'
```

Replace `htmlclay` with the hostname in your tab's address bar; the port is the same for every
folder. Browsers and `curl` resolve `*.localhost` to loopback by themselves. Other programs ask
the system resolver, which macOS answers, Linux answers with systemd-resolved, and Windows usually
does not (PowerShell's `Invoke-RestMethod` fails with "remote name could not be resolved"). A
program can skip the hostname entirely: `127.0.0.1:1212/_/api/<path>` reaches whichever open folder
serves that file, for any caller that is not a browser (no `Origin`, no `Sec-Fetch-Site`):

```powershell
Invoke-RestMethod http://127.0.0.1:1212/_/api/Documents/notes/notes.htmlclay
```

The path is relative to your home folder, the same path the address bar shows. If no open folder
contains the file, the answer is a 404 `No open folder`; open the file in HTML Clay first.

The second reads a tag the page publishes:

```html
<h1>Hello</h1>
<ul>
  <li class="todo" data-done="yes">one</li>
  <li class="todo" data-done="no">two</li>
</ul>

<script data-rules-name="api" data-rules-version="1">
  {title: "h1", items: [".todo", {text: ".", done: "@data-done"}]}
</script>
```

```json
{"title":"Hello","items":[{"text":"one","done":"yes"},{"text":"two","done":"no"}]}
```

`data-rules-version` must be exactly `"1"`; anything else is a `400`. The tag is found by
`script[data-rules-name~="api"]`, so the attribute is a space-separated token list and a script
may publish several names. The first match wins. The token must match `^[a-zA-Z0-9_-]+$`.

### Rule syntax

Rules are relaxed JSON: unquoted keys, single-quoted strings, trailing commas, and bare selectors
are all fine. Comments are not. Strict JSON is tried first and used as-is when it parses.

| Rule | Result |
|---|---|
| `"sel"` | trimmed text of the first match, or `null` when nothing matches |
| `"sel[]"` | list of every match's trimmed text. Empty stays `[]`, never `null` |
| `"sel@name"` | property or attribute `name` of the first match |
| `"@name"` | property or attribute `name` of the context node itself |
| `"."` | trimmed text of the context node |
| `["sel", {…}]` | one object per match, the shape evaluated with that match as context |

Nesting is capped at 20 levels; the 21st fails.

### Which selectors are accepted

The selector gate is an allowlist, not a denylist, and it refuses rather than guesses whenever the
two engines would read a construct differently. A refusal is a `400` that names the construct and
the reason.

Accepted pseudo-classes: `:not`, `:has`, `:nth-child`, `:nth-last-child`, `:nth-of-type`,
`:nth-last-of-type`, `:first-child`, `:last-child`, `:first-of-type`, `:last-of-type`,
`:only-child`, `:only-of-type`, `:root`, `:first`, `:last`, `:eq`, `:nth`, `:contains`,
`:checked`, and `:scope` at the start of a nested rule's selector followed by a space or `>`
(`:scope > section.checklist` matches the context's direct children). A `:scope` anywhere else, or
in a top-level rule, is refused.

Refused pseudo-classes, each with its own reason: `:matches`, `:matchesown`, `:containsown`,
`:haschild`, `:lang`, `:enabled`, `:disabled`, `:empty`, `:focus`, `:target`, `:gt`, `:lt`,
`:even`, `:odd`, `:is`, `:where`, `:parent`, `:header`, `:selected`, `:button`,
`:submit`, `:text`, `:password`, `:radio`, `:file`, `:image`, `:reset`, `:input`, `:link`,
`:visited`, `:hover`, `:active`. Pseudo-elements (`::before` and friends) are refused too, as are
CSS comments and `:has()` whose argument starts with a combinator.

`:first`, `:last`, `:eq`, `:nth`, `:contains`, and `:checked` are post-filters, so they are legal
only at the very end of a selector, and at most one positional per comma group. `li:first span`
is refused for that reason.

Attribute clauses accept the operators `=`, `~=`, `|=`, `^=`, `$=`, `*=`, and `!=`. The
regular-expression operator `[attr#=…]` is refused, so is the `s` flag in `[a=b s]`, so is any
other flag, and so is an empty value with `^= $= *= != ~=`, because the two engines disagree
about what it means.

A successful data response carries `Content-Type: application/json; charset=utf-8`,
`Cache-Control: no-store`, and `X-Content-Type-Options: nosniff`.

`@name` reads a DOM property when the name is in the property allowlist, and otherwise the
literal attribute. The properties, which shadow any same-named attribute: `textContent`,
`innerText`, `innerHTML`, `outerHTML`, `value`, `checked`, `selected`, `disabled`, `readOnly`,
`type`, `tagName`, `nodeName`, `nodeType`, `nodeValue`, `childElementCount`, `id`, `className`,
`classList`, `baseURI`, `offsetWidth`, `offsetHeight`, `clientWidth`, `clientHeight`,
`scrollWidth`, `scrollHeight`, `dataset`, `currentSrc`, `duration`, `paused`, `title`,
`documentURI`, `contentType`. `href`, `src`, and `action` are deliberately absent and read as
plain attributes, so the round trip stays byte-stable.

A rule ending in `@--name` reads one CSS custom property from the element's `style` attribute, or
`null` when it is absent.

One asymmetry that is real and easy to hit: the property branch keeps an empty string, the
attribute branch turns it into `null`. So `@className` on `class=""` is `""`, while `@data-x` on
`data-x=""` is `null`.

`sel@name[]` is not supported by either engine. HTML Clay refuses it with a `400` naming the form
that works: `["sel", {"name": "@name"}]`.

### What a data request is, and is not

A data request is a different response shape on the normal read path, not a second way into the
filesystem. It runs the same home-directory, hidden-file, trusted-root, permission and real-path
checks as an ordinary GET, in the same order, and returns the same refusal. Specifically it:

- cannot read a file an ordinary request could not read
- creates no editing session: no save token, no edit-mode cookie, no identity, no version
- strips any save token an external writer left on disk before extraction, so a capability
  cannot be read back out
- returns `Cache-Control: no-store`
- sends no CORS headers
- accepts only `.html` and `.htmlclay` documents
- carries an `ETag` header, the stamp of the stored bytes, for use as `If-Match` on a write

**The absence of CORS headers is deliberate.** Only a program you run yourself, such as `curl`,
or a page on this same origin can read it. A page served by another local server on your machine
cannot.

Bare `/_/api` is a `400`, not an index. A non-`.html`/`.htmlclay` path is a `404` on the
`/_/api` route, and on the query route the `?data=` parameter is ignored and the file is served
normally. Error bodies are
`{"error": "...", "message": "...", "details": "...", "example": "?data={title:\"h1\",items:\".item\"}"}`.

### Where this differs from the Hyperclay platform

Each row was measured against the platform's own engine rather than assumed.

| Behavior | HTML Clay | Platform |
|---|---|---|
| A selector the two engines read differently (`:is`, `:empty`, `:gt`, `:matches`, CSS comments, the `[a=b s]` flag) | refused with a reason | answered, sometimes differently |
| A positional that is not last, such as `li:first span` | refused | answered |
| Any broken selector | `400` | `400` or `500`, depending on the parser's wording |
| `@type` on an element | the `type` attribute | `"tag"`, the internal node type |
| `@readOnly` | the real value | always `false` |
| `sel@href[]` | refused, naming the array form that works | silently `[]` |
| A repeated `?data=` parameter | `400` | `500` |
| Caching | none | five minutes |
| CORS | none | enabled |

The full ledger, with the measurement behind every row, is
`internal/dataapi/testdata/selector-parity.json` in the repository.

## Writing a file as JSON

Caller supplied rules work on both URL forms, for GET and POST. No embedded tag is required:

```bash
curl --get 'http://htmlclay.localhost:1212/soup.htmlclay' \
  --data-urlencode 'data={title:h1}'

curl --request POST \
  'http://htmlclay.localhost:1212/soup.htmlclay?data=%7Btitle%3Ah1%7D' \
  --header 'Content-Type: application/json' \
  --data '{"title":"Lunch"}'
```

Replace `htmlclay` with the hostname in your tab's address bar; the port is the same for every
folder. The same requests work with `/_/api/soup.htmlclay?data=%7Btitle%3Ah1%7D`, and with `.html`
files.

Supplied rules replace the embedded mapping for that request, even if the tag is malformed.
They are never merged with it or saved into the file. Without `data`, `/_/api/` uses the
embedded `api` tag as before. An empty, malformed, or repeated `data` parameter returns
`400`; it never falls back to the tag. POST bodies remain strict JSON, with no rules envelope.
The response uses the same rules as the request. GET and POST return the source file's
`ETag`; send it as `If-Match` to refuse a stale write with `412`.

`POST /_/api/{path}` with a JSON body writes it into the document through the document's own
`api` rules tag. The body has the shape a GET returns, and the answer is the fresh extraction
with an `ETag` header.

```bash
curl -X POST 'http://htmlclay.localhost:1212/_/api/notes.htmlclay' \
  -H 'Content-Type: application/json' \
  -d '{"title":"Groceries","items":[{"text":"eggs","done":"no"}]}'
```

What a write does:

- Applies each key through its rule: text rules write text (never parsed as HTML), `@attr` rules
  write the attribute, `@--name` rules set one CSS custom property in the element's `style`
  attribute (an empty string or `null` removes it) while leaving the other declarations untouched,
  `@checked`/`@selected`/`@disabled` toggle presence, lists match rows by
  content, keep their order, grow by cloning the first row (or a `cms-template` seed when empty)
  and shrink by removing rows. A key you leave out is left alone; `null` clears a text value.
- Rewrites only the changed elements. Every other byte, line endings and quoting included, is
  kept. The splice is verified by reparsing; when it cannot be verified the whole document is
  rendered instead. A body that changes nothing writes nothing.
- Commits like a save: a version is recorded, open tabs update over live sync, and the watcher
  does not report it as an outside edit.
- Honors `If-Match`: a stale stamp answers `412` with the current `ETag`, and nothing is written.

What a write refuses, always `400` with nothing written:

- `Write refused`, `details` a list of `{target, reason}`: an `on*` handler, a raw `@style` write
  (reason `attribute "style" is not content`), `srcdoc`, a save or region marker (`no-save`,
  `no-data`, `contenteditable`, `editor-ui` and the rest), a `javascript:`, `vbscript:` or `data:`
  URL in a URL attribute, any element inside `<script>`, `<style>`, `<template>`, `<iframe>`,
  `<object>`, `<noscript>` or the head's metadata tags, a text write that would delete the rules
  tag, and `@innerHTML` or `@outerHTML`.
- An `@--name` write is refused with `Write refused` when the new value breaks a rule: more than
  200 characters; `;`, `{`, `}`, `<`, `>` or backslash; a comment or control character; unbalanced
  quotes or brackets; or `url()`, `image()`, `image-set()`, `cross-fade()`, `element()`, `paint()`,
  `src()` or `expression()`. Each refusal names the reason, such as
  `"url()" is not allowed in a custom property value`. Writing back the value already on the
  element is always accepted.
- `Write rejected`, `details` `{unknownKeys, unmatched}`: a key the selected rules do not define, or a
  rule whose selector matches nothing in the document.
- `Shape mismatch`, `Cannot grow list` (an empty list with no seed to clone), `No api rules tag`,
  and the rules-tag errors a GET returns.

Who can write:

- A request with no `Origin` and no `Sec-Fetch-Site`, such as `curl`, a script or an agent, is
  admitted with no token.
- A browser page on the same origin must send the target file's save token in a `Save-Token`
  header. A token for a different file is refused. `same-site` and cross-site requests are `403`.
- The file must be one HTML Clay has open, or an HTML Clay file inside a trusted folder, which
  is registered on the spot. Anything else is `403 Not writable`, with the same answer whether or
  not the file exists. A trusted file that belongs to another folder is `409` naming that folder's
  address.
- Bodies must be `application/json` (`415`) and at most 1 MB (`413`).

## Who can read and write

A program on your computer can read and write through this API. If you can open the page in your browser, you can read it through the API.

A trusted folder is one you have allowed HTML Clay to save files in. Writes require an opened file, or a `.htmlclay` file in a trusted folder.

A save token is the identifier HTML Clay gives a page to let it save its file. A browser page sends the target file's token in a `Save-Token` header. A program on your computer needs no token.

Other websites cannot read the API through your browser. HTML Clay sends no CORS permission headers and refuses requests from other websites that try to write. A page HTML Clay serves at the same address can read it.

[Hyperclay Local](https://hyperclaylocal.com) serves the same URLs for the files in its folder, for reads and writes.

## Troubleshooting

-   `400` for an unknown key: the key is not in the rules you used. Send a key the rules define.
-   `400` for the `data` parameter: it is empty, malformed or sent twice. Send it once, with valid rules.
-   `405` on a POST to the file's own address: add the `data` parameter, or post to `/_/api/`.
-   `403` on a write: open the file in HTML Clay, or use a `.htmlclay` file in a trusted folder. An unopened `.html` file is not enough. A browser request also needs the file's save token.
-   `412` on a write: read the latest file, then revise your change and retry with its new `Etag`.
-   The file changed but the tab did not: check that the ClayJS script loads the `sync` plugin.

## Where next

For how open tabs receive file changes, see [Live updates](https://htmlclay.com/docs/live-sync/). To send a request from the page to a program, see [Programs](https://htmlclay.com/docs/programs/). [Agent editing](https://htmlclay.com/docs/guide/let-a-program-edit/) walks through attaching a program from your terminal. The [wire protocol reference](https://github.com/panphora/htmlclay/blob/main/docs/wire.md) covers that connection in detail.