JSON over HTTP

Any .html or .htmlclay file HTML Clay serves can be read and written as JSON over HTTP, on the same local address your tab uses. The rules used to extract that JSON, and the page-side API that shares them, are on the ClayJS data plugin page; this page covers the HTTP side.

Save the example file#

Save this as todo.htmlclay in your home folder, then open it with HTML Clay:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <title>Tasks</title>
</head>
<body>
  <h1>Today</h1>
  <ol>
    <li class="todo" data-done="no">Buy milk</li>
    <li class="todo" data-done="no">Book tickets</li>
  </ol>
  <script src="https://clayjs.com/v1/clay.js?plugins=sync"></script>
  <script src="https://clayjs.com/v1/clay-data.js"></script>
</body>
</html>

The ClayJS script enables live updates. Plain .html files work the same way.

Read with rules in the request#

Replace htmlclay with the hostname in your open tab’s address. Your tab’s address looks like http://htmlclay.localhost:1212/todo.htmlclay. The port is the same for every folder.

Run this in your terminal:

base='http://htmlclay.localhost:1212'
curl --get "$base/todo.htmlclay" \
  --data-urlencode 'data={"title":"h1","items":".todo[]"}'

Keep that terminal open for the following examples.

The data parameter carries the rules, which curl encodes for the URL. title reads the heading. The [] in .todo[] collects every matching task’s text. The response is that data as JSON.

Ask only for the keys you need. With data={"title":"h1"}, the response contains only title:

{"title":"Today"}

Write with rules in the request#

The next request changes the heading to “Tomorrow” and the first task to “Buy bread”.

Send the rules in the URL and the new values as the body:

curl "$base/todo.htmlclay" \
  --url-query 'data={"title":"h1","items":".todo[]"}' \
  -H 'Content-Type: application/json' \
  --data '{"title":"Tomorrow","items":["Buy bread","Book tickets"]}'

The --data option makes this a POST, and --url-query encodes the rules into the URL. HTML Clay returns the updated values:

{
  "title": "Tomorrow",
  "items": ["Buy bread", "Book tickets"]
}

Open todo.htmlclay in your text editor. The heading now contains Tomorrow, and the first task contains Buy bread. The open tab updates too. The rules you sent are not saved into the file.

A key you leave out is left alone. An array you send becomes the whole list: a task missing from it is removed, and a new task is a copy of the first task’s element, with your text inside it. Keep at least one task in the list because once the list is empty, a write cannot add tasks back. New text replaces what was inside the selected element. Each write that changes the file is saved as a version.

Writes reach only the elements the rules select. These rules change text; attribute rules can also change allowed attributes. Writes cannot insert scripts, styles or event handlers. HTML you send is saved as plain text. A key the rules do not define causes a 400 response. Keep the JSON body under 1 MB.

Write only if the file is unchanged#

Change the second task only if the file still matches your read. First, read with response headers:

curl -i "$api"

The Etag header identifies the file you read. Both URLs return it, for reads and writes. Its value looks like this:

Etag: 8f02c16e7b4d9a30

Replace the sample value below with your exact Etag, then change the second task:

curl "$api" \
  -H 'Content-Type: application/json' \
  -H 'If-Match: 8f02c16e7b4d9a30' \
  --data '{"title":"Tomorrow","items":["Buy bread","Book train tickets"]}'

If the file still matches, the response shows the changed task:

{
  "title": "Tomorrow",
  "items": ["Buy bread", "Book train tickets"]
}

If the file has changed, HTML Clay returns 412 and writes nothing. Read the file again before deciding what to send. Without If-Match, the write skips this check.

Reading a file as JSON#

Any .html or .htmlclay file HTML Clay will serve can also be read as JSON, using the same extraction rules as the Hyperclay platform. Two URL forms, with caller supplied rules taking precedence on both.

You supply the rules, and the document’s own rules tag is ignored:

curl --get 'http://htmlclay.localhost:1212/notes.htmlclay' \
  --data-urlencode 'data={title:"h1",items:".todo[]"}'

Without ?data=, the file supplies its own:

curl 'http://htmlclay.localhost:1212/_/api/notes.htmlclay'

Replace htmlclay with the hostname in your tab’s address bar; the port is the same for every folder. Browsers and curl resolve *.localhost to loopback by themselves. Other programs ask the system resolver, which macOS answers, Linux answers with systemd-resolved, and Windows usually does not (PowerShell’s Invoke-RestMethod fails with “remote name could not be resolved”). A program can skip the hostname entirely: 127.0.0.1:1212/_/api/<path> reaches whichever open folder serves that file, for any caller that is not a browser (no Origin, no Sec-Fetch-Site):

Invoke-RestMethod http://127.0.0.1:1212/_/api/Documents/notes/notes.htmlclay

The path is relative to your home folder, the same path the address bar shows. If no open folder contains the file, the answer is a 404 No open folder; open the file in HTML Clay first.

The second reads a tag the page publishes:

<h1>Hello</h1>
<ul>
  <li class="todo" data-done="yes">one</li>
  <li class="todo" data-done="no">two</li>
</ul>

<script data-rules-name="api" data-rules-version="1">
  {title: "h1", items: [".todo", {text: ".", done: "@data-done"}]}
</script>

{"title":"Hello","items":[{"text":"one","done":"yes"},{"text":"two","done":"no"}]}

data-rules-version must be exactly "1"; anything else is a 400. The tag is found by script[data-rules-name~="api"], so the attribute is a space-separated token list and a script may publish several names. The first match wins. The token must match ^[a-zA-Z0-9_-]+$.

Rule syntax#

Rules are relaxed JSON: unquoted keys, single-quoted strings, trailing commas, and bare selectors are all fine. Comments are not. Strict JSON is tried first and used as-is when it parses.

Nesting is capped at 20 levels; the 21st fails.

Which selectors are accepted#

The selector gate is an allowlist, not a denylist, and it refuses rather than guesses whenever the two engines would read a construct differently. A refusal is a 400 that names the construct and the reason.

Accepted pseudo-classes: :not, :has, :nth-child, :nth-last-child, :nth-of-type, :nth-last-of-type, :first-child, :last-child, :first-of-type, :last-of-type, :only-child, :only-of-type, :root, :first, :last, :eq, :nth, :contains, :checked, and :scope at the start of a nested rule’s selector followed by a space or > (:scope > section.checklist matches the context’s direct children). A :scope anywhere else, or in a top-level rule, is refused.

Refused pseudo-classes, each with its own reason: :matches, :matchesown, :containsown, :haschild, :lang, :enabled, :disabled, :empty, :focus, :target, :gt, :lt, :even, :odd, :is, :where, :parent, :header, :selected, :button, :submit, :text, :password, :radio, :file, :image, :reset, :input, :link, :visited, :hover, :active. Pseudo-elements (::before and friends) are refused too, as are CSS comments and :has() whose argument starts with a combinator.

:first, :last, :eq, :nth, :contains, and :checked are post-filters, so they are legal only at the very end of a selector, and at most one positional per comma group. li:first span is refused for that reason.

Attribute clauses accept the operators =, ~=, |=, ^=, $=, *=, and !=. The regular-expression operator [attr#=…] is refused, so is the s flag in [a=b s], so is any other flag, and so is an empty value with ^= $= *= != ~=, because the two engines disagree about what it means.

A successful data response carries Content-Type: application/json; charset=utf-8, Cache-Control: no-store, and X-Content-Type-Options: nosniff.

@name reads a DOM property when the name is in the property allowlist, and otherwise the literal attribute. The properties, which shadow any same-named attribute: textContent, innerText, innerHTML, outerHTML, value, checked, selected, disabled, readOnly, type, tagName, nodeName, nodeType, nodeValue, childElementCount, id, className, classList, baseURI, offsetWidth, offsetHeight, clientWidth, clientHeight, scrollWidth, scrollHeight, dataset, currentSrc, duration, paused, title, documentURI, contentType. href, src, and action are deliberately absent and read as plain attributes, so the round trip stays byte-stable.

A rule ending in @--name reads one CSS custom property from the element’s style attribute, or null when it is absent.

One asymmetry that is real and easy to hit: the property branch keeps an empty string, the attribute branch turns it into null. So @className on class="" is "", while @data-x on data-x="" is null.

sel@name[] is not supported by either engine. HTML Clay refuses it with a 400 naming the form that works: ["sel", {"name": "@name"}].

What a data request is, and is not#

A data request is a different response shape on the normal read path, not a second way into the filesystem. It runs the same home-directory, hidden-file, trusted-root, permission and real-path checks as an ordinary GET, in the same order, and returns the same refusal. Specifically it:

• cannot read a file an ordinary request could not read
• creates no editing session: no save token, no edit-mode cookie, no identity, no version
• strips any save token an external writer left on disk before extraction, so a capability cannot be read back out
• returns Cache-Control: no-store
• sends no CORS headers
• accepts only .html and .htmlclay documents
• carries an ETag header, the stamp of the stored bytes, for use as If-Match on a write

The absence of CORS headers is deliberate. Only a program you run yourself, such as curl, or a page on this same origin can read it. A page served by another local server on your machine cannot.

Bare /_/api is a 400, not an index. A non-.html/.htmlclay path is a 404 on the /_/api route, and on the query route the ?data= parameter is ignored and the file is served normally. Error bodies are {"error": "...", "message": "...", "details": "...", "example": "?data={title:\"h1\",items:\".item\"}"}.

Where this differs from the Hyperclay platform#

Each row was measured against the platform’s own engine rather than assumed.

The full ledger, with the measurement behind every row, is internal/dataapi/testdata/selector-parity.json in the repository.

Writing a file as JSON#

Caller supplied rules work on both URL forms, for GET and POST. No embedded tag is required:

curl --get 'http://htmlclay.localhost:1212/soup.htmlclay' \
  --data-urlencode 'data={title:h1}'

curl --request POST \
  'http://htmlclay.localhost:1212/soup.htmlclay?data=%7Btitle%3Ah1%7D' \
  --header 'Content-Type: application/json' \
  --data '{"title":"Lunch"}'

Replace htmlclay with the hostname in your tab’s address bar; the port is the same for every folder. The same requests work with /_/api/soup.htmlclay?data=%7Btitle%3Ah1%7D, and with .html files.

Supplied rules replace the embedded mapping for that request, even if the tag is malformed. They are never merged with it or saved into the file. Without data, /_/api/ uses the embedded api tag as before. An empty, malformed, or repeated data parameter returns 400; it never falls back to the tag. POST bodies remain strict JSON, with no rules envelope. The response uses the same rules as the request. GET and POST return the source file’s ETag; send it as If-Match to refuse a stale write with 412.

POST /_/api/{path} with a JSON body writes it into the document through the document’s own api rules tag. The body has the shape a GET returns, and the answer is the fresh extraction with an ETag header.

curl -X POST 'http://htmlclay.localhost:1212/_/api/notes.htmlclay' \
  -H 'Content-Type: application/json' \
  -d '{"title":"Groceries","items":[{"text":"eggs","done":"no"}]}'

What a write does:

• Applies each key through its rule: text rules write text (never parsed as HTML), @attr rules write the attribute, @--name rules set one CSS custom property in the element’s style attribute (an empty string or null removes it) while leaving the other declarations untouched, @checked/@selected/@disabled toggle presence, lists match rows by content, keep their order, grow by cloning the first row (or a cms-template seed when empty) and shrink by removing rows. A key you leave out is left alone; null clears a text value.
• Rewrites only the changed elements. Every other byte, line endings and quoting included, is kept. The splice is verified by reparsing; when it cannot be verified the whole document is rendered instead. A body that changes nothing writes nothing.
• Commits like a save: a version is recorded, open tabs update over live sync, and the watcher does not report it as an outside edit.
• Honors If-Match: a stale stamp answers 412 with the current ETag, and nothing is written.

What a write refuses, always 400 with nothing written:

• Write refused, details a list of {target, reason}: an on* handler, a raw @style write (reason attribute "style" is not content), srcdoc, a save or region marker (no-save, no-data, contenteditable, editor-ui and the rest), a javascript:, vbscript: or data: URL in a URL attribute, any element inside <script>, <style>, <template>, <iframe>, <object>, <noscript> or the head’s metadata tags, a text write that would delete the rules tag, and @innerHTML or @outerHTML.
• An @--name write is refused with Write refused when the new value breaks a rule: more than 200 characters; ;, {, }, <, > or backslash; a comment or control character; unbalanced quotes or brackets; or url(), image(), image-set(), cross-fade(), element(), paint(), src() or expression(). Each refusal names the reason, such as "url()" is not allowed in a custom property value. Writing back the value already on the element is always accepted.
• Write rejected, details {unknownKeys, unmatched}: a key the selected rules do not define, or a rule whose selector matches nothing in the document.
• Shape mismatch, Cannot grow list (an empty list with no seed to clone), No api rules tag, and the rules-tag errors a GET returns.

Who can write:

• A request with no Origin and no Sec-Fetch-Site, such as curl, a script or an agent, is admitted with no token.
• A browser page on the same origin must send the target file’s save token in a Save-Token header. A token for a different file is refused. same-site and cross-site requests are 403.
• The file must be one HTML Clay has open, or an HTML Clay file inside a trusted folder, which is registered on the spot. Anything else is 403 Not writable, with the same answer whether or not the file exists. A trusted file that belongs to another folder is 409 naming that folder’s address.
• Bodies must be application/json (415) and at most 1 MB (413).

Who can read and write#

A program on your computer can read and write through this API. If you can open the page in your browser, you can read it through the API.

A trusted folder is one you have allowed HTML Clay to save files in. Writes require an opened file, or a .htmlclay file in a trusted folder.

A save token is the identifier HTML Clay gives a page to let it save its file. A browser page sends the target file’s token in a Save-Token header. A program on your computer needs no token.

Other websites cannot read the API through your browser. HTML Clay sends no CORS permission headers and refuses requests from other websites that try to write. A page HTML Clay serves at the same address can read it.

Hyperclay Local serves the same URLs for the files in its folder, for reads and writes.

Troubleshooting#

• 400 for an unknown key: the key is not in the rules you used. Send a key the rules define.
• 400 for the data parameter: it is empty, malformed or sent twice. Send it once, with valid rules.
• 405 on a POST to the file’s own address: add the data parameter, or post to /_/api/.
• 403 on a write: open the file in HTML Clay, or use a .htmlclay file in a trusted folder. An unopened .html file is not enough. A browser request also needs the file’s save token.
• 412 on a write: read the latest file, then revise your change and retry with its new Etag.
• The file changed but the tab did not: check that the ClayJS script loads the sync plugin.

Where next#

For how open tabs receive file changes, see Live updates. To send a request from the page to a program, see Programs. Agent editing walks through attaching a program from your terminal. The wire protocol reference covers that connection in detail.