Info |
---|
This exercise is intended to teach you how to use Snowstorm to:
|
Last Updated:
On this page
Table of Contents | ||
---|---|---|
|
Tip |
---|
Please visit the Pre-requisites page to see software that may be useful during the day. In summary, we recommend using:
|
1. Accessing the server
You will be given an IP ip-address and a you can download the training.pem private key of an to access the AWS instance to access. Go to the directory that where you have put saved the training.pem file and run:
Code Block | ||||
---|---|---|---|---|
| ||||
chmod 700 training.pem ssh -i training.pem ubuntu@<ip-address> |
Warning |
---|
This should work without any problems on Linux and OSX. For Windows, because of the difference in using SSH, please follow the instructions on the Windows set up page. |
On the server you will find:
- /opt/releases - containing the following:
SnomedCT_InternationalRF2_PRODUCTION_20190731T120000Z20210731T120000Z.zip (July 2019 2021 International Edition)
- SnomedCT_ManagedServiceSEManagedServiceDK_PRODUCTION_SE1000052DK1000005_20191130T120000Z20210930T120000Z.zip (November 2019 Swedish September 2021 Danish Extension)
SnomedCT_InternationalRF2_PRODUCTION_20200131T120000Z20220131T120000Z.zip (January 2020 International Edition2022 International Edition)
- SnomedCT_InternationalRF2ManagedServiceDK_PRODUCTION_DK1000005_2020309T120000Z20220331T120000Z.zip (March 2020 International Edition)SnomedCT_ManagedServiceSE_PRODUCTION_SE1000052_20200531T120000Z.zip (May 2020 Swedish 2022 Danish Extension)
/opt/snowstorm - currently empty
/opt/sct-browser - the SNOMED CT browsercontains the most recent released Snowstorm jar file
- /opt/Frontend-Interaction-Demonstration - containing the following:
- demo front end HTML for some exercises
Elasticsearch v6v7.89.8 2 has already been installed on the server. You can check that Elasticsearch is running with the following command:
Code Block | ||||
---|---|---|---|---|
| ||||
curl "http://localhost:9200" |
When normally installing Snowstorm, you You will need to download the most recent release of Snowstorm (currently v4.10.2) into the /opt/snowstorm folder:
Code Block | ||||
---|---|---|---|---|
| ||||
cd /opt/snowstorm
sudo wget https://github.com/IHTSDO/snowstorm/releases/download/4.10.2/snowstorm-4.10.2.jar |
from the GitHub repository releases page. However, we have put the Snowstorm executable jar file on the server already.
2. Starting Snowstorm
To start snowstorm you will need to run the following command (it is also document in the repository readme files - https://github.com/IHTSDO/snowstorm/blob/master/docs/getting-started.md).commands.
(Documentation available starting-snowstorm)
The simplest thing is to leave an SSH window open with this command running so you can easily access the log output. Alternatively, you can run the following commands. We are using tmux to enable multiple terminals in one window (you can skip using tmux but you will need to open up another ssh session for the remainder of the day's exercises) :
Code Block | ||||
---|---|---|---|---|
| ||||
tmux cd /opt/snowstorm tmux sudo java -Xms4g -Xmx4g -jar snowstorm*.jar --elasticvc.save.batch-size=10000 | sudo tee -a snowstorm.log |
The pipe at the end of the command will log all the output to the We are using tmux so that if you have an internet connection issue and get disconnected Snowstorm will keep running. If this happens you can reconnect to the machine and type "tmux a" to continue where you left off.
We are using tee to see the output in the window and copy it to a snowstorm.log file in the same directory.
Note |
---|
Either: Switch out of the tmux session by typing ctrl+B and then D on your keyboard. This will take you back to the original session and command line prompt Or: Make sure start a another, new terminal window ( a new SSH session) and ssh back in to the server so that you can run other commands without killing the running Snowstorm server |
3. Loading the initial data
Now, you will need to Next load the SNAPSHOT of the July 2019 July 2021 International Edition. Instructions on how to do that are in the Snowstorm readme - https://github.com/IHTSDO/snowstorm/blob/master/docs/loading-snomed.md - which uses the command line.
(Documentation available loading-snomed)
We will However, we are going to use the REST API to run the first importimports. Access the server Snowstorm swagger interface using the AWS instance IP ip-address and port 8080, for example - http://<ip-address>:8080/
First we need to start the import process by creating a new import job. Look for the Import endpoint endpoints and then create a new import using the start-local-file-import endpoint with the following importRequest parameter
Code Block | ||||
---|---|---|---|---|
| ||||
{ "branchPath": "MAIN", "createCodeSystemVersion": true, "filePath": "/opt/releases/SnomedCT_InternationalRF2_PRODUCTION_20210731T120000Z.zip", "type": "SNAPSHOT" } |
... and then click on 'Try it now' and then note the id of the import as you will need it for the next step (it will look something like - d0b30d96-3714-443e-99a5-2f282b1f1b0)
We now need to give the server the file. Normally, you would do this through the swagger interface, but to reduce network issues, find the correct file and we will use curl to do this from the server directly
language | bash |
---|---|
theme | Confluence |
.
The Response Headers will contain the location of the newly created job. Copy this URL into a new tab to monitor the status of the job.
You can watch the Snowstorm log to see how this is progressing through tmux (if you are running it as above)
Code Block | ||||
---|---|---|---|---|
| ||||
tmux attach |
or simply to the import endpoint or fetch the job status from the import job URL - http://<ip-address>:8080/imports/<import_id> .
This should take no more than around 30 minutes.
Warning |
---|
Make sure that you wait for the import to complete before going any further forward. You can see it is completed by looking at the status of the import - |
If you want to check that you have imported successfully, you can do this by seeing how many SNOMED CT concepts there are on the MAIN branch with the following URL - http://<ip-address>:8080/MAIN/concepts?activeFilter=true&offset=0&limit=1
- which show that there are 350,830 active concepts.a total of 474,074 active and inactive concepts.
Code System Registry
The Snowstorm Code System Registry should now have a single code system listed here: http://<ip-address>:8080/codesystems
And a single SNOMEDCT version listed here: http://<ip-address>:8080/codesystems/SNOMEDCT/versions
Check the registry throughout this exercise to see your changes reflected.
4. Loading the local edition or extension
As this is the first time we will be importing the local edition or extension (SE Danish extension in this instance), we will need to create a CodeSystem on the server.
On the swagger interface , and in the ‘Code Systems’ section of the swagger interface, look for the ‘Create a code system’ endpoint. Use
Use the following in the request to create the CodeSystem. Note the dependantVersionEffectiveTime allows you to create a CodeSystem dependent on a specific International Edition version.
Code Block | ||||
---|---|---|---|---|
| ||||
{ "branchPathname": "MAIN/SNOMEDCT-SEDanish Extension", "shortName": "SNOMEDCT-SEDK", "countryCode": "sedk", "defaultLanguageCodebranchPath": "svMAIN/SNOMEDCT-DK", "dependantVersionEffectiveTime": 2019073120210731, "namedefaultLanguageCode": "Swedish Extensiondk" } |
click
on'Try it now'.
You now need to import the local extension or edition. First we Like last time, we need to start the import process by creating a new import job. Look for the Import endpoint endpoints and then create a new import using the start-local-file-import endpoint with the following importRequest parameter
Code Block | ||||
---|---|---|---|---|
| ||||
{ "branchPath": "MAIN/SNOMEDCT-SEDK", "createCodeSystemVersion": true, "filePath": "/opt/releases/SnomedCT_ManagedServiceDK_PRODUCTION_DK1000005_20210930T120000Z.zip", "type": "SNAPSHOT" } |
click
on'Try it now'
and then note the id of the import as you will need it for the next step (it will look something like - d0b30d96-3714-443e-99a5-2f282b1f1b0).
Copy the "location" from the Response Headers and open this in a new browser tab.
Refresh the import job in your browser to see the status change from WAITING_FOR_FILE to RUNNING and then COMPLETED. You can also see how this is progressing in the Snowstorm log.
We now need to give the server the file. Normally, you would do this through the swagger interface, but to reduce network issues, find the correct file and we will use curl to do this from the server directly
Code Block | ||||
---|---|---|---|---|
| ||||
curl -X POST --header 'Content-Type: multipart/form-data' --header 'Accept: application/json' -F file=@/opt/releases/SnomedCT_ManagedServiceSE_PRODUCTION_SE1000052_20191130T120000Z.zip 'http://localhost:8080/imports/<import id>/archive' |
You can watch log to see how this is progressing, or simply to the import endpoint - http://<ip address>:8080/imports/<import id>
This should take no more than a few minutes.
Warning |
---|
Make sure that you wait for the import status to change to complete COMPLETED before going any further forward. You can see it is completed by looking at the status of the import - |
5. Upgrading MAIN to a new International Edition
Every 6 months month now, there is a new International Edition and it is important to keep your terminology server up to date. An upgrade is an import again, but for the International Edition, it is a DELTA import onto MAIN. For the purpose of this exercise, we will only import the relevant International Edition release that the Danish extension depends on.
Option A
The International Edition can be upgraded by importing a SNAPSHOT of the new changes onto the MAIN branch.
We now First, we need to create an import job as aboveusing this configuration:
Code Block | ||||
---|---|---|---|---|
| ||||
{ "branchPath": "MAIN", "createCodeSystemVersion": true, "typefilePath": "DELTA" } |
and then click on 'Try it now' and then note the id of the import as before.
We now need to give the server the fileas before
Code Block | ||||
---|---|---|---|---|
| ||||
curl -X POST --header 'Content-Type: multipart/form-data' --header 'Accept: application/json' -F file=@/opt/releases/SnomedCT_InternationalRF2_PRODUCTION_20200131T120000Z20220131T120000Z.zip", 'http://localhost:8080/imports/<import id>/archive' |
Warning |
---|
Make sure that you wait for the import to complete before going any further forward. You can see it is completed by looking at the status of the import - |
Unusually, there was an interim release of the International Edition in March 2020 before the next Swedish release to respond to this year's COVID-19 pandemic and so we need to import that before the Swedish release.
"type": "SNAPSHOT"
} |
Option B
The International Edition can be upgraded by importing a generated DELTA of the new changes onto the MAIN branch.
We now need to create an import job using this configuration:
Do the same as above with the following import job infoCode Block | ||||
---|---|---|---|---|
| ||||
{
"branchPath": "MAIN",
"createCodeSystemVersion": true,
" typefilePath":"DELTA "
}And the following curl command Code Block | | |||
|
click 'Try it now'. As before use the URL in the location Response Header to open a new browser tab. Refresh the page to monitor the import status.
Warning |
---|
Make sure that you wait for import status COMPLETED before going any further forward. |
6.1 Upgrading the extension/edition branch to the new International Edition (merging the branch)
We now need to upgrade the local extension/edition branch . to use the newly imported International Edition release content.
In our example, we will now merge rebase the MAIN branch into the /SNOMEDCT-SE branch DK branch using the CodeSystem upgrade endpoint using giving the shortname, short name SNOMEDCT-SEDK:
Code Block | ||||
---|---|---|---|---|
| ||||
{ "contentAutomations": false, "newDependantVersion": 2020030920220131 } |
You can check this has been successful by checking the status of the branch and seeing if it is forwardFORWARD.
6.2 Upgrading to a new extension or edition
Using what you have already done, see if you can do this making note the type and the branch that you will import into.
Do this by creating an import job as before, using this configuration:
Code Block | ||||
---|---|---|---|---|
| ||||
{
"branchPath": "MAIN/SNOMEDCT-DK",
"createCodeSystemVersion": true,
"filePath": "/opt/releases/SnomedCT_ManagedServiceDK_PRODUCTION_DK1000005_20220331T120000Z.zip",
"type": "SNAPSHOT"
} |
click 'Try it now' and use the location Response Header to open a new browser tab. Refresh the page to monitor the import status.
Info |
---|
Code System Registry AdviceWhen updating Snowstorm, entries only appear in the registry when each process is complete. When accessing content a version branch from the registry should be used e.g. "MAIN/SNOMECT-DK/2021-09-30", not "MAIN/SNOMEDCT-DK". This will avoid your implementation being able to see inconsistent content during an upgrade. |