-
Notifications
You must be signed in to change notification settings - Fork 34
/
Copy pathtemplate.haml
211 lines (165 loc) · 14.5 KB
/
template.haml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
-# This template is used for generating the HTML representation of the test suite manifests.
- require 'cgi'
- require 'digest'
!!! 5
%html
%head
%meta{"http-equiv" => "Content-Type", :content => "text/html;charset=utf-8"}
%title
= manifest['name']
%link{rel: "alternate", href: "#{json_file}"}
%link{rel: "stylesheet", href: "https://www.w3.org/StyleSheets/TR/base"}
%body
%p
%a{href: "http://www.w3.org/"}
%img{src: "http://www.w3.org/Icons/w3c_home", alt: "W3C", height: 48, width: 72}
%h1<=manifest['name']
:markdown
#{manifest['description']}
:markdown
This is an HTML version of a test manifest. The JSON-LD version of this manifest may be found at
[#{json_file}](#{json_file}). The manifest vocabulary is described in the [JSON-LD Test Vocabulary](vocab.html) ([JSON-LD](vocab.jsonld), [Turtle](vocab.ttl)) and is based on the [RDF Test Vocabulary](http://www.w3.org/TR/2014/NOTE-rdf11-testcases-20140225/).
The JSON-LD Test Suite is a set of tests that can
be used to verify JSON-LD Processor conformance to the set of specifications
that constitute JSON-LD. The goal of the suite is to provide an easy and
comprehensive JSON-LD testing solution for developers creating JSON-LD Processors.
The [JSON-LD Framing Specification](https://w3.org/TR/json-ld11-framing) maintains its own
[test suite](https://w3c.github.io/json-ld-framing/tests/).
## General instructions for running the JSON-LD Test suites
- if manifest['sequence'].first.is_a?(String) || manifest['sequence'].any? {|te| te['@type'].include?('jld:CompactTest')}
:markdown
### [compact](compact-manifest.html) tests have _input_, _expected_ and _context_ documents.
The _expected_ results can be compared using [JSON-LD object comparison](#json-ld-object-comparison) with the processor output. Additionally, if the `ordered` option is not set, result should be expanded and compared with the expanded _expected_ document also using [JSON-LD object comparison](#json-ld-object-comparison).
For **NegativeEvaluationTests**, the result is a string associated with the expected error code.
- if manifest['sequence'].first.is_a?(String) || manifest['sequence'].any? {|te| te['@type'].include?('jld:ExpandTest')}
:markdown
### [expand](expand-manifest.html) tests have _input_ and _expected_ documents.
The _expected_ results can be compared using [JSON-LD object comparison](#json-ld-object-comparison) with the processor output.
Expansion tests may have a `expandContext` option, which is treated
as an IRI relative to the manifest.
For **NegativeEvaluationTests**, the result is a string associated with the expected error code.
- if manifest['sequence'].first.is_a?(String) || manifest['sequence'].any? {|te| te['@type'].include?('jld:HtmlTest')}
:markdown
### [html](html-manifest.html) tests have _input_ and _expected_ documents and an optional _context_ document.
The _expected_ results can be compared using [JSON-LD object comparison](#json-ld-object-comparison) with the processor output after potentially remapping blank node identifiers (see below). Additionally, if the result is compacted and the `ordered` option is not set, result should be expanded and compared with the expanded _expected_ document also using [JSON-LD object comparison](#json-ld-object-comparison).
For **NegativeEvaluationTests**, the result is a string associated with the expected error code.
- if manifest['sequence'].first.is_a?(String) || manifest['sequence'].any? {|te| te['@type'].include?('jld:FlattenTest')}
:markdown
### [flatten](flatten-manifest.html) tests have _input_ and _expected_ documents and an optional _context_ document.
The _expected_ results can be compared using [JSON-LD object comparison](#json-ld-object-comparison) with the processor output after potentially remapping blank node identifiers (see below). Additionally, if the result is compacted and the `ordered` option is not set, result should be expanded and compared with the expanded _expected_ document also using [JSON-LD object comparison](#json-ld-object-comparison).
For **NegativeEvaluationTests**, the result is a string associated with the expected error code.
- if manifest['sequence'].first.is_a?(String) || manifest['name'].include?('Remote')
:markdown
### [remote-doc](remote-doc-manifest.html) tests have _input_ and _expected_ documents.
The _expected_ results can be compared using [JSON-LD object comparison](#json-ld-object-comparison) with the processor output.
For **NegativeEvaluationTests**, the result is a string associated with the expected error code.
Options may be present to describe the intended HTTP behavior:
* _contentType_: Content-Type of the returned HTTP payload, defaults to the appropriate type for the _input_ suffix.
* _httpStatus_: The HTTP status code to return, defaults to `200`.
* _redirectTo_: The HTTP _Content-Location_ header value.
* _httpLink_: The HTTP _Link_ header value.
- if manifest['sequence'].first.is_a?(String) || manifest['sequence'].any? {|te| te['@type'].include?('jld:FromRdfTest')}
:markdown
### [fromRdf](fromRdf-manifest.html) tests have _input_ and _expected_ documents.
The _expected_ results can be compared using [JSON-LD object comparison](#json-ld-object-comparison) with the processor output.
- if manifest['sequence'].first.is_a?(String) || manifest['sequence'].any? {|te| te['@type'].include?('jld:ToRdfTest')}
:markdown
### [toRdf](toRdf-manifest.html) tests have _input_ and _expected_ documents.
Some tests require the use of [JSON Canonicalization Scheme](https://tools.ietf.org/html/draft-rundgren-json-canonicalization-scheme-05) to properly generate RDF Literals from JSON literal values. This algorithm is non-normative, but is assumed to be used to properly compare results. These tests are marked using the `useJCS` option.
The _expected_ results can be compared using [RDF Dataset Isomorphism](https://www.w3.org/TR/rdf11-concepts/#dfn-dataset-isomorphism).
A **PositiveSyntaxTest** looks specifically for syntax-related issues. A **PositiveSyntaxTest** succeeds when no error is found when processing.
ToRdf tests may have a `expandContext` option, which is treated
as an IRI relative to the manifest.
:markdown
Unless `processingMode` is set explicitly in a test entry, `processingMode` is compatible with both `json-ld-1.0` and `json-ld-1.1`.
Test results that include a context input presume that the context is provided locally, and not from the referenced location, thus the results will include the content of the context file, rather than a reference.
Developers are encouraged to make a local copy of the test suite (available on [GitHub](https://github.com/w3c/json-ld-api/tree/main/tests)) and simulate the behavior of fetching test files remotely and setting HTTP headers as described in a particular test entry.
<h2 id="json-ld-object-comparison">JSON-LD Object comparison</h2>
If algorithms are invoked with the `ordered` flag set to `true`, simple JSON Object comparison may be used, as the order of all arrays will be preserved (except for _fromRdf_, unless the input quads are also ordered). If `ordered` is `false`, then the following algorithm will ensure arrays other than values of `@list` are compared without regard to order.
JSON-LD Object comparison compares JSON objects, arrays, and values recursively for equality.
* JSON objects are compared entry by entry without regard to the ordering of entries within the object. Each entry must have a corresponding entry in the object being compared to. Values are compared recursively.
* JSON arrays are generally compared without regard to order (the lone exception being if the referencing key is `@list`). Each item within the array must be equivalent to an item in the array being compared to by using the comparison algorithm recursively. For values of `@list`, the order of these items is significant.
* JSON values are compared using strict equality.
* Values of `@language`, and other places where language tags may be used are specified in lowercase in the test results. Implementations should either normalize language tags for testing purposes, or compare language tags in a case-independent way.
Note that some tests require re-expansion and comparison, as list values may exist as values of properties that have `@container: @list` and the comparison algorithm will not consider ordering significant.
# Running tests
The top-level [manifest](manifest.jsonld) references the specific test manifests, which in turn reference each test associated with a particular type of behavior.
Implementations create their own infrastructure for running the test suite. In particular, the following should be considered:
* _remote-doc_ tests will likely not return expected HTTP headers, so the _options_ should be used to determine what headers are associated with the input document.
* Test case properties identifying a file (_input_, _output_, _context_, _expectContext_, and _frame_) are presumed to have a media type appropriate for the file extension.
* `application/ld+json` for `.jsonld`
* `text/html` for `.html`
* `application/n-quads` for `.nq`
* The media type for the file associated with the _input_ property can be overridden using the `contentType` option.
* Some algorithms, particularly _fromRdf_, may not preserve the order of statements listed in the input document, and provision should be taken for performing unordered array comparison, for arrays other than values of `@list`. (This may be difficult for compacted results, where array value ordering is dependent on the associated term definition).
* Some _toRdf_ tests require the use of [JSON Canonicalization Scheme](https://tools.ietf.org/html/draft-rundgren-json-canonicalization-scheme-05) to properly generate RDF Literals from JSON literal values. This algorithm is non-normative, but is assumed to be used to properly compare results using [RDF Dataset Isomorphism](https://www.w3.org/TR/rdf11-concepts/#dfn-dataset-isomorphism). These tests are marked using the `useJCS` option.
* When comparing documents after flattening, framing or generating RDF, blank node identifiers may not be predictable. Implementations using the JSON-LD 1.0 algorithm, where output is always sorted and blank node identifiers are generated sequentially from `_:b0` may continue to use a simple object comparison. Otherwise, implementations should take this into consideration. (One way to do this may be to reduce both results and _expected_ to datsets to extract a bijective mapping of blank node labels between the two datasets as described in [RDF Dataset Isomorphism](https://www.w3.org/TR/rdf11-concepts/#dfn-dataset-isomorphism)).
* Some tests may have a `requires` property, indicating some optional behavior described by a test vocabulary term.
# Contributing Tests
If you would like to contribute a new test or a fix to an existing test,
please follow these steps:
1. Notify the JSON-LD mailing list, public-json-ld-wg@w3.org,
that you will be creating a new test or fix and the purpose of the
change.
2. Clone the git repository: git://github.com/w3c/json-ld-api.git
3. Make your changes and submit them via github, or via a 'git format-patch'
to the [JSON-LD Working Group mailing list](mailto:json-ld-wg@w3.org).
## Distribution
Distributed under the [W3C Test Suite License](http://www.w3.org/Consortium/Legal/2008/04-testsuite-license). To contribute to a W3C Test Suite, see the [policies and contribution forms](http://www.w3.org/2004/10/27-testcases).
## Disclaimer
UNDER THE EXCLUSIVE LICENSE, THIS DOCUMENT AND ALL DOCUMENTS, TESTS AND SOFTWARE THAT LINK THIS STATEMENT ARE PROVIDED "AS IS," AND COPYRIGHT HOLDERS MAKE NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, OR TITLE; THAT THE CONTENTS OF THE DOCUMENT ARE SUITABLE FOR ANY PURPOSE; NOR THAT THE IMPLEMENTATION OF SUCH CONTENTS WILL NOT INFRINGE ANY THIRD PARTY PATENTS, COPYRIGHTS, TRADEMARKS OR OTHER RIGHTS.
COPYRIGHT HOLDERS WILL NOT BE LIABLE FOR ANY DIRECT, INDIRECT, SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF ANY USE OF THE DOCUMENT OR THE PERFORMANCE OR IMPLEMENTATION OF THE CONTENTS THEREOF.
- if manifest['baseIri']
%dl
%dt="baseIri"
%dd=manifest['baseIri']
- if manifest['sequence']
%section
%h2
Test sequence:
- if manifest['sequence'].first.is_a?(String)
%ul
- manifest['sequence'].each do |man|
- man_name = man.sub('.jsonld', '')
%li
%a{href: "#{man_name}.html"}<=man_name
- else
%dl.entries
- manifest['sequence'].each do |entry|
%dt{id: entry['@id'][1..-1]}
="Test #{entry['@id'][1..-1]} #{entry['name']}"
%dd
%dl.entry
%dt="id"
%dd=entry['@id']
%dt="Type"
%dd="#{Array(entry['@type']).join(', ')}"
%dt="Purpose"
%dd=entry['purpose']
%dt="input"
%dd
%a{href: entry['input']}=entry['input']
- if entry['context']
%dt="context"
%dd
%a{href: entry['context']}=entry['context']
- if entry['frame']
%dt="frame"
%dd
%a{href: entry['frame']}=entry['frame']
%dt="expect"
%dd
- if entry['@type'].to_s.include?('Negative')
=entry['expectErrorCode']
- else
%a{href: entry['expect']}=entry['expect']
- if entry['option']
%dt="Options"
%dd
%dl.options
- entry['option'].each do |k, v|
%dt=k
%dd=v
- if entry['requires']
%dt="Requires"
%dd= entry['requires']