Post-deployment verification

Once a deployment has been made to Staging or Production a process of verifying that deployment should be performed.

Obviously for the Staging deployment much more extensive testing of the deployed content will be performed to ensure the release is as desired for publication. However a shorter “smoke” test of the deployment success should be run against the Staging server before testing begins and on the Production server before the deployment is considered successful.

Objectives

The objectives of this process are to verify that:

  • the FHIR endpoint is
    • up and available,
    • has loaded the required content,
    • has security correctly configured; AND
  • the Syndication endpoint is
    • up and available,
    • contains the right entries in the feed,
    • the entries in the feed are available and match the SHA256 listed,
    • has security correctly configured.

Tests

The following tests can be implemented either through automation or manually using the FHIR and Syndication APIs and Snapper and OntoCommand user interfaces.

FHIR endpoint

Availability

The FHIR endpoint’s metadata endpoint can be hit to determine whether the server is up and ready to service requests at all. This can be found at /fhir/metadata on the endpoint in question.

This should respond with a CapabilityStatement resource. This can be done in a web browser and will result in a file download containing the CapabilityStatement, or a tool like Postman.

This can also be checked manually by navigating to the base FHIR endpoint and opening OntoCommand. This will hit the FHIR endpoint and require it to be available.

Content loaded

Using FHIR resource search operations it is possible to return the number of each resource type loaded in the server. To get an accurate picture this will need to be performed with credentials that have complete read access to all communities, otherwise some resources may not be returned in the searches.

This can be checked against the expected count of resources of each type from the syndication feed used as the preload feed for the release.

A more thorough test can use the same method to confirm the url and version elements of each of the FHIR resources against the resources in the syndication feed to ensure all are present and report if one is not. This would require automation.

Security

Using an array of test credentials with different levels of authority and community permissions against known resources can be used as a method to determine whether the security is correctly configured and being applied. This can be done manually through OntoCommand or Snapper of course, however it is possible to also automate this testing using the API.

Syndication tests

Availability

Performing a GET request on the Ontoserver’s endpoint /synd/syndication.xml should return an XML Atom document and an HTTP 200 response code. This is a quick and simple test to ensure the endpoint us available.

Correct content

The XML Atom document returned from the GET request to /synd/syndication.xml can be compared to the syndication feed configured as the preload feed for the Ontoserver container, corresponding to the release content. All the content in the feed should be present in the Ontoserver instance’s syndication feed response. These can be checked against each other manually, although if a large quantity of items is present this is best automated.

The syndication feed also needs to be requested with credentials that have read access to all resources, including all required community permissions. If the credentials used to get the syndication feed from Ontoserver don’t have read access to some of the resources they will simply not appear in the syndication feed Atom XML, which will make it falsely appear that they are not loaded in the server.

Feed entries are available

A further test of the syndication feed is to download entries to ensure the server can produce them. The required URLs to download them are present in the feed XML document itself.

For a very large syndication feed with very large artefacts (such as SNOMED CT indexes) a randomised or selected partial set may be chosen. Additionally once entries are downloaded their SHA256 can be calculated to ensure it matches the SHA256 fingerprint advertised in the syndication feed.

Security

Security for the syndication endpoint can be checked by using test credentials with a series of different read access levels. Resources which are not readable should not appear in the requested syndication.xml Atom document. Requests to download from URLs in Atom feeds retrieved by users higher privileges should be rejected for users who do not have entries with that URL in their syndication feed.