TSH Reference: Complete Grammar

This is the complete reference for TestScript Shorthand (TSH) as implemented by FHIR Frog, extracted from TSH.g4 and verified against the TSHProcessor in fhir-frog-shorthand. Use this page for exact syntax, edge cases, and unsupported constructs.

Document outline

A TSH file follows this structure:

TestScript: <id>
[Title: "..."]
[Description: "..."]
Status: #<code>
[Url: "..."]

[Fixture: <name>
  * resource = "..."
  * autocreate = false
  * autodelete = false
]...

[Setup:
  [* <action>]...
]

Test: "<name>"
  [* <action>]...

[Test: "<name>"
  [* <action>]...
]...

[Teardown:
  [* <action>]...
]

• TestScript: required; must come first
• Metadata (Title, Description, Status, Url): optional except Status, which is required
• Fixtures: optional; must come before Setup/Test/Teardown
• Setup, Test, Teardown: phases; at least one Test is required
• Actions: within each phase; detailed below

Metadata fields

TestScript (required)

TestScript: my-test

Sets the TestScript id. The value is a token (letters, digits, hyphen, underscore).

Title (optional)

Title: "Human-Readable Title"

Sets TestScript.title.

Description (optional)

Description: "Narrative description of what this script tests."

Sets TestScript.description.

Status (required)

Status: #draft

Sets TestScript.status. Common values: #draft, #active, #retired. See the FHIR PublicationStatus CodeSystem.

Url (optional)

Url: "http://example.org/TestScript/my-test"

Sets TestScript.url (canonical URL). If omitted, frog_tsh_compile derives one from the id.

Fixtures

Fixture: <name>
  * resource = "<path>"
  * autocreate = false
  * autodelete = false

All three properties are required (autocreate and autodelete are not used by the runner but must be present).

Fixture name

The name (after Fixture:) is a token (alphanumeric, underscore). Use it in from "<name>" clauses.

resource property

Path to a file, relative to the TestScript's directory (or absolute within the workspace). Examples:

* resource = "patient.json"
* resource = "../fixtures/patient-1.json"

The file must exist when the script runs. If not, frog_run_local fails with "Resource can not be null".

autocreate / autodelete

Must be set to false. They are parsed but not used by the FHIR Frog runner (design decision).

Actions

Within a phase (Setup, Test, Teardown), a line starting with * is an action.

* operation <type> <resource> [/<id>] [from "<fixture>"] [as <responseId>]
* assert <form>
* variable <name> = <fhirpath> from <responseId>

Operations

* operation <type> <resource>[/<id>] [from "<fixture>"] [as <responseId>]

Operation types

• create: HTTP POST with a request body (from a fixture)
• search: HTTP GET to a resource type's search endpoint
• read: HTTP GET to a specific resource (see Known limits for why it does not run)
• update: HTTP PUT (with a fixture body)
• delete: HTTP DELETE (see Known limits)

Resource

Must be a valid FHIR resource type (e.g. Patient, Observation, Bundle).

Resource id (optional)

Follows the resource name with a /:

* operation read Patient/123

Known limit: id-based read/delete do not run. See Known limits.

from clause (create, update)

* operation create Patient from "p1" as p_created

References a fixture name (from a Fixture: block). The fixture is required for create and update; it is an error to omit it.

The fixture name is a string literal (in quotes), not a reference to a variable.

as clause (optional)

* operation search Patient as search_resp

Gives a name to the response, so later actions can reference it in assertions or variable extractions.

Assertions

Assertions follow operations and validate responses.

* assert <form>

Forms

Form	Compiles to	Example
responseCode = <code>	assert.responseCode	* assert responseCode = 200
responseCode in (<code1>, <code2>)	assert.responseCode (first code only)	* assert responseCode in (200, 201)
resource = <Type>	assert.resource	* assert resource = Patient
contentType = "<type>"	assert.contentType	* assert contentType = "application/fhir+json"
expression "<fhirpath>"	assert.expression	* assert expression "Bundle.entry.count() > 0"
minimumId = "<id>"	assert.minimumId	* assert minimumId = "p1"
validateProfile = "<url>"	assert.validateProfile	* assert validateProfile = "http://example.org/Profile/MyPatient"

Assertions are placed immediately after an operation and evaluate that operation's response.

Variables

* variable <name> = <fhirpath> from <responseId>

Extracts data from a prior operation's response using a FHIRPath expression.

• name: identifier (alphanumeric, underscore)
• fhirpath: dot-separated path (e.g. resource.id, Bundle.entry.first().resource.name.first().text)
• responseId: name from a prior as <responseId> clause

Example:

TestScript: VariableExample
Title: "Variable Example"
Status: #draft
Url: "http://example.org/TestScript/variables"

Fixture: p1
  * resource = "fixtures/patient.json"
  * autocreate = false
  * autodelete = false

Test: "Extract and use a variable"
  * operation create Patient from "p1" as p1_created
  * assert responseCode = 201
  * variable patient_id = resource.id from p1_created
  * operation search Patient as search_resp
  * assert expression "Bundle.entry.where(resource.id=$patient_id).count() = 1"

Tokenization traps

String literals

Resource paths and contentType values are string literals (in double quotes):

* resource = "patient.json"          # Correct
* contentType = "application/fhir+json"  # Correct
* resource = patient.json            # ERROR: not a string

Fixture names in from clauses

Fixture names are also quoted:

Fixture: p1
  * resource = "patient.json"
  * autocreate = false
  * autodelete = false

Test: "Example"
  * operation create Patient from "p1" as p_created
  * operation search Patient as search_resp

FHIRPath in expressions

FHIRPath is a quoted string:

* assert expression "Bundle.entry.where(resource.name.where(use='official')).count() > 0"

Nested quotes are allowed (single quotes for FHIRPath string literals).

Keywords are case-sensitive

TestScript: MyTest    # Correct
testscript: MyTest    # ERROR
Fixture: f            # Correct
fixture: f            # ERROR

Compilation process

The frog_tsh_compile tool:

1. Parses the TSH file using TSH.g4
2. Applies the TSHProcessor to convert grammar nodes to TestScript JSON structure
3. Adds required fields (e.g. encodeRequestUrl: true, default url if missing)
4. Validates fixture references
5. Emits a JSON TestScript

Compiler warnings

The compiler warns about constructs that are unsupported or partially supported:

• ${var} in operation URLs: parsed but never substituted at runtime
• in operator with multiple codes: compiled but keeps only the first
• Assertions in teardown: parsed but dropped with a warning (teardown may contain operations only)
• Search parameters in operation URL: parsed as operation.resource but fails at runtime

Silent drops

The compiler silently:

• Drops assertions in Teardown phases
• Omits variables from Teardown actions
• Does not validate that fixture files exist (validation happens at run time)

Known limits (do not add to TSH)

These are not syntax errors; they parse successfully but do not run:

Construct	Why	Workaround
read Patient/123 or read Patient/<id>	Client-side HTTP 400 before reaching the server. The id is not substituted by any variable.	Use search Patient with assertions; or use JSON TestScript with "params": "/123".
delete Patient/123	Client-side HTTP 400.	Use JSON TestScript or fixture cleanup via Teardown.
search Patient?family=Smith (search with params in URL)	Parameters cannot be encoded in the operation; the whole string becomes resource field and the engine fails.	Use JSON TestScript with "params": "?family=Smith".
${variable} in operation URLs	Variables are not substituted in paths.	Use variables in assertions only. For request parameterization, use JSON TestScript.
Transaction or Batch operations	Not yet supported in TSH.	Use JSON TestScript with type: "transaction" or "batch".
System-level operations (e.g. $expand, $validate)	Not supported in TSH.	Use JSON TestScript.

Complete example with all constructs

TestScript: CompleteExample
Title: "Complete TSH Example"
Description: "Demonstrates all supported TSH constructs."
Status: #draft
Url: "http://example.org/TestScript/complete"

Fixture: patient_1
  * resource = "fixtures/patient-1.json"
  * autocreate = false
  * autodelete = false

Fixture: patient_2
  * resource = "fixtures/patient-2.json"
  * autocreate = false
  * autodelete = false

Setup:
  * operation create Patient from "patient_1" as p1_created
  * assert responseCode = 201
  * assert resource = Patient
  * variable p1_id = resource.id from p1_created

Test: "Search and validate"
  * operation search Patient as search_resp
  * assert responseCode = 200
  * assert resource = Bundle
  * assert contentType = "application/fhir+json"
  * assert expression "Bundle.entry.count() >= 1"

Test: "Create another patient and search"
  * operation create Patient from "patient_2" as p2_created
  * assert responseCode = 201
  * operation search Patient as all_patients
  * assert responseCode = 200
  * assert expression "Bundle.entry.count() >= 2"

Teardown:
  * operation search Patient as final_search

References

• FHIR TestScript specification
• TestScript Shorthand grammar (TSH.g4)
• Choosing a TestScript format
• TestScript Shorthand conceptual guide